Meta researchers proved that Quest 2 body tracking was possible without the use of additional trackers.
The current VR systems come with a headset and a handheld controller. They only measure your head and track your hands. A class of algorithms known as inverse kinematics can estimate the position of your elbows and torso. However, this is not always accurate for your legs and only occasionally for elbows. There are many possible solutions to each set of head or hand positions.
Due to limitations in IK, not all VR apps show your hands and some only show your upper body. SteamVR tracking support PC headsets wore extra trackers such HTC Vive Tracker. However, the cost of these three trackers for body tracking is over $350 so this feature is not supported in most games.
Meta researchers have shown a system that can calculate a plausible body pose using only the Quest 2 tracking data and its controllers in . There are no additional trackers or sensors.
The avatar motion is very similar to the real user. Researchers claim that the avatar motion matches the user’s real movement more closely than worn IMU trackers, which are devices with an accelerometer and gyroscope like Pico 4’s Pico Fit Band. However, Pico claims that it is developing its own machine-learning algorithm.
There is a catch in Quest 2 Body Tracking
This system, as you can see in the video, is intended to create a realistic full-body pose. It does not match your exact hands position. QuestSim is not designed to see your avatar body, but only other people’s. The latency of the system is 160ms, which is more than 11 frames at 72Hz.
However, it would still be much better to see the full body motions of avatars of others than the often criticized legless upper bodies Meta’s avatars have. Is this or something similar coming to Quest 2?”
Andrew Bosworth, Meta CTO, certainly hinted at this last week. When Bosworth was asked about leg tracking in an Instagram, “ask me any question” Bosworth replied .
“Yeah, we’ve been made fun a lot because of the legless avatars. I think that’s fair and it’s quite funny.
It can be very distressing to have legs on your avatar that are not the same as your real legs. However, you don’t have to worry about it if you put legs on someone else.
We are currently working to make legs look natural to someone who is just watching – because they don’t know your legs – but you will see your legs when you look at them. This is our current strategy.
However, the short-term solution may not be as good as this algorithm. Machine learning research papers are typically written on high-powered PC GPUs with a low framerate. The runtime performance of this system is not mentioned in the paper.
Meta Connect’s annual AR/VR event takes place within just over 2 weeks . Therefore, any body tracking announcements would likely take place during this time.
Andrew Bosworth, Meta CTO, certainly hinted at this last week. When Bosworth was asked about leg tracking in an Instagram, “ask me any question” Bosworth replied .
“Yeah, we’ve been made fun a lot because of the legless avatars. I think that’s fair and it’s quite funny.
It can be very distressing to have legs on your avatar that are not the same as your real legs. However, you don’t have to worry about it if you put legs on someone else.
We are currently working to make legs look natural to someone who is just watching – because they don’t know your legs, but you will see your legs when you look at them. This is our current strategy.
However, the short-term solution may not be as good as this algorithm. Machine learning research papers are typically written on high-powered PC GPUs with a low framerate. The runtime performance of this system is not mentioned in the paper.
The Meta Connect annual AR/VR Event takes place within just over 2 weeks . Therefore, any body tracking announcements would likely take place during this event.
Magic leap 2 also dong some great experiments in this field.