You may want to search this forum as this question has been raised many times and there are lots of great answers posted, including the pros and cons, distances, costs, etc.
The key is that there are relatively few hotels in Annapolis -- especially close to USNA -- so it is important to book early to get your first choice.