Does it count?
Activities of any kind - athletic or otherwise - conducted or sponsored outside the school system do indeed "count" in the sense these are recognized. If there are officers or any positions of leadership within these organizations, and if you hold one or more, this should be noted in your profile. If you are particularly good in rugby and a standout on the team, or if the team is recognized in whatever "league" or manner in which you compete, your coach may augment your file with a letter noting such. Again, remember, Admissions is looking for fit, scholastically able and athletically proficient candidates ... but the heaviest weight will always fall to transcript and test scores. If your transcript and test scores (plus class position) are good, AND you are active in extracurricular clubs, etc., AND play a sport, it demonstrates that you should be able to manage the demands at West Point, which, essentially, will pile all of the above on you for four academic years.