My SEAL friend said it's a great deal of mental stress. The better you are physically, the less that adds to the stress. Swimming and running are big parts. Half Nelsons and take downs, not so much (wrestling keeps you fit, but swimming and running will do, and being a wiz at swimming is one less than that will pile on you).
I'm no SEAL, but that's the general feel from a friend or two.