I had the exact same issue as you. I was weak on physical fitness, specifically pullups. I started January of my junior year going to the gym every other day and focusing on exercises that would help with pullups. Don't neglect the other parts of the CFA, but if you are weak with pullups, make half of your workout focused on muscles needed for the pullup. Once I got to where I could do one pullup (which took about two months) I built a pullup bar and attached it to my ceiling and did one every time I passed by it. Once you can do two pullups, do two each time you pass it. I did my CFA in August and did five or six pullups and received an appointment for the Class of 2019.
Being able to do pullups is not out of reach, but start training TODAY. I thought they would keep me out of the Academy, but they didn't. It's Saturday, go to the gym, talk to someone who works there, tell them your goals, and get on the machines until you can do pullups (or if you can already do them great! Get a bar and do them until you can't do any more).
And YES this is something you should worry about. If you fail one section of your CFA you will not get in. I'm know I barely passed the minimums for my pullups, but I also know I made up for it in other sections. You have months left to train, you can do it.
Sent from my iPhone using Tapatalk