Can you join the swim team and earn a varisty letter now?
It comes down to how much you effort you want to put in for your West Point application? What do you have to give up in order to join your school swim team and earn a varsity letter this year (I am assuming if you think you could get a letter of recommendation for your school's swim coach, you should be good enough to make your school team).
From my perspective, a way to increase your chance of getting accepted by West Point is making West Point #1 priority. Some candidates tell me something along the line "I am too busy right now so I cannot work on my West Point application."
What it comes down to is that the admissions office has a finite time to review your application. If you are a typical competitive candidate, good SAT/ACT, good grades, good leadership, varisty letters and etc, the admissions office don't have to spend extra time looking into your application. So if you are not a typical candidate, you are almost asking the admissions office to ignore your application (Disclaimer - this is my opinion only).
Good luck.