I guess I'll give you my perspective. I should have been in the Class of 2014 and accepted my appointment but decided against it shortly before I was to report on R-Day for a variety of reasons. I went through my freshman year of college, joined a fraternity, and produced mediocre academic results by my standards (no, it had nothing to do with the fraternity). I haven't been a complete waste of space, but I haven't felt like I've been doing too much with my life either.
I've been contemplating ROTC at my college for a long time now and am about to do it. It seems to be more intense than your average ROTC program, and it's pretty well-respected nationally. One of my good friends from high school did it for his freshman year and got in to West Point on his second try. He's up there right now, about to finish Beast, and I'm really proud of him. I think about my decision not to go to West Point almost every day and it still bothers me. I've even thought about re-applying, but I doubt USMA lets me back in since I backed out so close to R-Day. Given what I've learned over the past year, I would jump at the opportunity to go, so I may go through the process again.
My point to you is that West Point isn't for everybody and a lot of people should do ROTC instead. However, if you really are certain it's the right college for you, go for it, and don't look back. I think my personal experience is as good as anyone's in regards to this issue. When you're up there, always remember that most of the things you think you are missing out on don't really matter in the grand scheme of things. If West Point is really something you want, then a lot of that other stuff should rightfully be considered horse**** to you.