Do you want to run in the mountains, or do you want to run in the sand? If you've never done either, they both suck. And that pretty much sums up the classic rivalry between VMI and The Citadel.
In all seriousness, it's hard to answer that question because it's so broad. More importantly, it's very difficult to compare the two unless you have personal experiences with both. TurkishRunner hit on some of the basic (third hand) observations of one who has visited both. I've never been to The Citadel, but I can answer specific questions you might have about my experiences at VMI.
Other than that, my best advice is to check out the academic programs (to include cocurriculars) of both schools and see which one most closely matches what you want a degree in and the quality/type of education you're looking for. Here are a couple of examples: VMI has a nationally recognized Arabic program, but The Citadel does not offer Arabic. On the flip side, The Citadel offers a major and a minor in Criminal Justice, but you can't get either of those at VMI.
First figure out what you're looking for, and then compare it to all the colleges you're even remotely considering. Academics should be your #1 priority but don't forget the implications of "cocurricular" education, which includes the Honor System, the Ratline, athletics, leadership training and everything else you'll be expected to accomplish while succeeding academically.
Visit all the colleges on your list, if you can. There's only so much information you can glean from recruiting brochures and the Internet.