Just like packer said, a lot depends on where you live. In some places, a 32 would be near the top of all applicants. In others, there could be 20+ people applying with higher math scores than that. If you lived in a place like Mississippi or Idaho or Rhode Island, where either a) the total pool of applicants is not very competitive or b) there are not many service academy applicants to begin with, then a 32 probably wouldn't be anything to worry about. However, if you live in a place like NoVa or Texas or near a major metropolitan area where there is a lot of people or a large military presence then I would say try to bring the 32 up.
All of this aside, even if you live in the middle of nowhere it wouldn't hurt to try to take the ACT again if you are financially capable. If you somehow dont win your MOC's vacancy, you would need really good scores to be competitive for a spot off the NWL