To be quite honest, all the things you say you enjoy are physical challenges, and Beast is not all that physically challenging.
Beast is more a challenge to your ability to adapt to changes and a surrounding that is probably different from any other you've been in before. Your character and your ability to think, adapt, and overcome difficult tasks (like how to get food passed down the table without being able to say anything, or how to organize a group of the untrained to do something both they and you have never done before without hurting themselves or failing) will be tested more than your body will be.
Good character and a brain that you use are more important for West Point than physical ability. Everyone here has physical ability. It's part of how we get in. Once you get here no one really cares about how physically fit you are unless you're deficient (IE can't pass the APFT). Outside of that people want to know that you're competent, not an idiot, can use your brain and have good character (and not necessarily in that order).
West Point graduates aren't famous for their physical ability, they're famous because they can be thrown into a random unfamiliar strange and difficult situation and adapt, improvise, and improve the situation all the while maintaining impeccable character.