Robotic Control using Proprioception

In 2015 I had the opportunity to work at the Los Alamos Dynamics Summer School as a summer student on a human robotic interface project. Along with two other students we explored ways of building a more intuitive control interface for a human to control a robot.

We were building our device around the idea of using sensory substitution to extend the human proprioceptive sense. Proprioception is the sense that allows us to be aware of the location of our limbs and body in space. For example, when you catch a ball you don’t consciously think about where your arm is located, your proprioceptive sense takes care of that. Sensory substitution is phenomenon where the brain can be tricked to accept extra or replacement senses. A cool example of this uses tongue electrodes to replace a lack of vision by mapping an electrode intensity to a pixel shade.

We built a simulation using python and the Robot Operating System (ROS) and connected it to a wearable device and control system. The wearable device encoded the state of the robot as tactile vibrations on the operators forearm. We took our complete system to a local museum and performed a human subjects experiment on museum visitors.

Our complete experiment was published in the SPIE conference and is available as a PDF here.