Then catch the Short Line into Highland Falls. This one is less frequent and would require some planning, but I think it puts you at the WP Visitor's Center.
snip
I don't know what you need to do after that to actually get into West Point.
If you can get to visitor center or anywhere in downtown highland falls you are within walking distance of the South entrance gate.
There is a campus shuttle that will take you pretty much anywhere you need to go! I think it runs every half hour, but admissions could confirm.
We mainly used it from Hotel Thayer just inside the South gate, but my impression was that it had stopped in Highland Falls earlier. (Was full of civvies headed to work on campus)
All that said- for visits to USMA I'd still suggest renting a car. By the time you do planes, trains, taxies, etc a car rental is normally cheaper and will save much time. It's a no-brainer if it's more than one traveling.
USMA is not hard to get to or find, especially if starting from the NJ side of things. Head up Pallisades Parkway, or drop on to 9W headed north and ride along the hudson. Both will get you to Highland Falls. Just make sure you are on the west side of the Hudson!!!! (there is also a 9E)
In most cases you park your car at Buffalo Soldier field just inside the South gate and use the campus shuttle. Parking is available but usually full anywhere near admissions, so it's just easier.
For a quick pickup/drop-off, it's OK (per admission) to drive to the admissions building, and wait in front with flashers on and vehicle occupied. Pretty common for parents picking up candidates after overnight visit.
and for hotels, having tried 2-3 different options, our experience was that Hotel Thayer is nicer and more convenient than the alternatives. You'll pay almost as much to be 20-30 minutes away in a budget hotel. IE: Hotel Thayer $113 vs econobudget for $99. No contest. Holiday Inn Express was the same as Hotel Thayer and much further away.
Though it looks close on the map, Newburgh is over the mountain and a good 30 minutes away. And is not a particularly nice place to stay. Waterfront is OK to visit, but downtown is quite rough.