There are many different threads and posts around with similar questions. The basic premise of the question is: "what are my chances". The problem with such a question is: We have absolutely no idea who your competition is.
In other words, you could be great in something, and there's someone who is better. The average person, assuming they received a nomination from their representative and their senators, is competing against all 3 of those lists of applicants. If you don't receive an appointment there, you'll then compete against all others in the same boat, at a national level. So it's impossible to say what your score will be.
It's not like: "Get (X) score, and you receive an appointment". A score that got an appointment this year, could be too low next year. It depends on everyone else's scores.
I, and others here, know personally of people with a 4.0gpa and damn near perfect ACT/SAT scores who DID NOT receive an appointment. That's all they had. It's good you have leadership. I'm not quite sure what the admission's counselor meant by you barely met the athletic involvement minimum. But the truth is; they really don't want to hear excuses of why you weren't involved in athletics. There is nothing about your application that isn't similar to someone else who was able to manage also doing sports. They were IB/AP students, scouts, JrROTC, class officer, volunteer, etc... and yet they still were able to do athletics. So again; there's no way to say how much improved grades or CFA will affect your chances to offset sports. Next year's appointments will be LESS than this years. So, your chances are already going to be more difficult.
Remember; sports is not just about athletics and being in good shape. It's also about team work. It's about being competitive and having drive. It's about having a specific job that is part of a larger mission. It's about being challenged and not quitting.
Sorry I couldn't give you the answer you're looking for. There may be some who are more encouraging that me, but the truth is, we don't know your chances or if they improve. Like i said, I know personally straight "A" students who basically ACED the ACT/SAT and they didn't receive an appointment. My only advice is to find a way to get involved with athletics. Best of luck to you.