There's nothing wrong with it per se. To each their own. I don't see playing soccer or rock climbing as "distraction", on the contrary. I mean distraction as in distracting your mind from the present moment.
For example, I used to go running and weightlifting while listening to music and podcasts for example, because I was in the mindset of physical exercise == unpleasant & time consuming therefore I must distract myself "to get it over with". VR workouts sound a lot like an idea coming from that same mindset.
For example, I used to go running and weightlifting while listening to music and podcasts for example, because I was in the mindset of physical exercise == unpleasant & time consuming therefore I must distract myself "to get it over with". VR workouts sound a lot like an idea coming from that same mindset.