I would also add to this that it probably will not be a Coast Guard funded master's, as many Cadets later go on to take advantage of. I would say engineering/applied physics is possible but probably a very long stretch, so just keep in mind it will be on your own time and dime, even if that dime is a GI Bill.
I just want to throw this out as food for thought, not to discourage you. This is the second post I've seen in as many days about studying physics by you. Make sure you are really okay with studying EE - 4 years here is hard when you absolutely love what you're doing, and I couldn't even imagine surviving here in an engineering major if I was settling for something. If physics is really where your heart is and EE is just the closest thing we offer but you aren't all that interested, then perhaps another academy or ROTC may be a better fit, or maybe you don't like the idea of those. I'm just throwing this out there for consideration.
@ekb1398 makes a very good point here, you need to be certain EE is what you want to study. In part, because EE is one of the more technically difficult majors CGA offers. Not very many people graduate with an EE degree. I asked an EE classmate out of curiosity - when we were at CGA (not too long ago), the average number of people who declared EE their freshman year was around 30. The number of people who graduated as an EE was in the low teens, in my class's case, 13. Some of the remaining 17 did not graduate and some changed majors - voluntarily and because they couldn't meet the requirements. OP, if you had to change majors, is there something else you could switch to and still be happy/satisfied?
From an undergraduate perspective, you could study physics as a Marine and Environmental Science (MES) major. To graduate, you have to pick two tracks to study from three options - Physics, Chemistry, and Biology. A lot of people go Chemistry/Biology or Physics/Biology, not many do Physics/Chemistry but you have the option.
To answer your question about graduate school (or the Advanced Education Program), the CG does not currently sponsor a Physics graduate degree. So, the CG will not pay for you to get a Physics degree. However, you can use Tuition Assistance to fund your own degree and pay out of pocket as well. Nothing stops you from working during the day and going to school at night, that's how a lot of people get their degree. You will also have the GI Bill available to you 8 years after you graduate (it takes three years to accrue the full benefits, but you must serve your 5 year commitment first if you graduated after 2014).
The graduate programs the CG will sponsor are:
Academy Company Officer
Leadership Studies
Academy Instructor
Acquisitions Systems Management
Aero Eng Avionics Project Management
Aero Eng STEM MBA
Aero Eng Structures
Asia Pacific Center for Security Studies
Aviation Safety Systems
C4IT Strategic Management
Central Security Service(NSA/CSS) Fellowship
Chemical Engineering
Civil Engineering
Civil Engineering Management
Civil Engineering MBA
Communications, Computer, and Electrical Engineering
Damage Ctrl Industrial Tech
Electrical Power Systems & Controls
Emergency Management Environmental Management
Financial Accounting
Financial Management
Fire Protection Engineering
Healthcare Admin
Homeland Security
Human Resources Management
Industrial Engineering
Industrial Hygiene
Information Assurance
Information Technology Management
International Affairs(Olmstead Program)
International Affairs Policy
Law (JD)
Law (LLM)
Marine Affairs/LMR
Marine Engineering
Marine Science(Oceanography)
Maritime Operations
Mechanical Engineering
Naval Engineering
Naval Institute Fellowship
Ocean Engineering
Operations Research
Organizational Leadership
Performance Technology
Physician Assistant
Public Administration
Public Affairs
Reserve Resources Management
Strategic Intelligence
Transportation Management
Western Hemisphere Institute for Security
The one way you could possibly get a Physics degree paid for is if you applied to the Academy Instructor program and there was a opening for a Physics Instructor. That is definitely not a guarantee though, as it all depends on people rotating and you getting the one slot that dozens of people would apply for.
You mention graduate school being far off - it is. In reality, you have four years of CGA and then at least your first two years as a JO before you can even hope to go to graduate school. In reality, you may not go until your third tour - which is nine years away from now. Focus on enjoying your senior year, focus on staying physically fit, and focus on the short term goals. It's great to have dreams, I would never discourage you from asking long-term questions. But, slow down just a little.