Last year, there was an application in the Permit to Report package for incoming Midshipmen to join the Navy Federal Credit Union which included the option to have their pay directly deposited to the account. There is a branch on the Yard which is extremely convenient.
The one expense I do remember having to pay for is for our Mid's printer. They are told the make/model, and there are stacks of them piled up in the Mid Store during PPW, which is when we bought it. IIRC, it includes at least one printer cartridge, but those are also available at the Mid Store (no tax on purchases there). I think the printer was about $100.
I'm a little fuzzy on the Yard Card, but I think you can buy one online starting in early August. You will need your Plebe's Alpha code (their USNA identifier that is unique to them for their duration at USNA), but they will have had to memorize it starting on I-Day, so all you have to do is ask them! Once you load the card with $$, it can be used at the Mid Store, Drydock (restaurant/snack bar on the Yard), and the vending machines throughout the Yard. It is very similar to the type of card that students at civilian colleges use to pay for food, etc.