If you are flying into town, rent the smallest car possible that will still fit you and your party. My W and I always rented a subcompact or smaller.
When trying to park in Annapolis, you will thank me for this suggestion. You might find street parking, if you're lucky, but I guarantee you...