Looking at the class of 2017 profile, 90% of the class plays varsity sports. That doesn't mean they were recruited for sports, but being a SA, I would imagine a good percentage have been. My daughter has been recruited for rowing.
(Just my thinking) all the SA are very physical fitness oriented. It makes sense therefore that the varsity teams are intense. Also, in camparison to other D1 schools, the # of students is low - heck we have high schools bigger than most SA here in MO. My point is to "cover" all the sports teams, most students are playing a sport.
Frankly, I think you should take it as a compliment being awarded an appointment to a SA without sports. My daughter's test scores were fine, but not great - rowing got her over this bump.