JPL Uses Kinect 2 and Oculus Rift to Control Robot

Posted by at 12:51 pm on December 31, 2013

nasa-vr-hand
We haven’t put a person on Mars yet, but we are getting closer to doing it virtually. NASA’s Jet Propulsion Laboratory (JPL) has built a rig that allows an operator to use the Oculus Rift virtual reality headset to see through a robot’s image sensors and the Kinect 2 from Xbox One to control the robot’s arm.

The Kinect tracks the user’s movement, specifically how the arm moves, rotates and clenches. The robot arm follows, though with a slight lag due to the interface and the latency involved with operating a robot so far away. (The video shows a lag of a few seconds, but between Earth and Mars the radio delay is several minutes.)

The interface shows how the person’s hand is moving and how the robot arm is responding, keeping the user updated on the lag’s effect in real time. The Oculus Rift headset allows the user to virtually see the environment around the robot, in 3D, for better understanding of the situation.

NASA’s Jet Propulsion Laboratory website shows how the agency could use such tech? Imagine a new Mars Rover in which the user controls the robot, its cameras and wheels and arms, through motion control. And with the Rift, the user can actually perceive the environment in 3D. It would be the closest thing to putting someone on Mars without actually making the trip. JPL has also shown that this telepresence technology could be used to better control the Robonaut 2 robot on the International Space Station.

This isn’t JPL’s first experiment with these technologies. The lab has used Oculus Rift to allow a person to explore the Mars terrain around the Mars Rover, compiled via photographs taken by the Rover. JPL has also used the Rift with the Virtuix Omni treadmill to allow a person to walk around a virtual Mars. All of these technologies together could someday lead to a complete telepresence on other planets.

Leave a Reply

Sign Up For Our Newsletter

Sign up to receive breaking news
as well as receive other site updates

Enter your Email


Preview | Powered by FeedBlitz

Log in

Copyright © 2008 - 2020 · StreetCorner Media , LLC· All Rights Reserved ·