NYC & South, train is probably the best option. Take the ferry across to Garrison if running, or a large cab bill if not.
To Boston the train is a wash cost wise when compared with renting a car. And much slower than a car. Enterprise will pick you up at the PX, you get your car at Newburgh. Get some buddies to share with you and the rental car can be even better.
You'll likely pay underage premium, but sometimes the rental companies have coupons waving it for AAA members, etc.