Enabling End-Users to Program New Robot Skills

Mobile manipulators like the PR2 have the physical capability to do a range of useful tasks for humans, however their actual capabilities are limited by the software applications written by highly specialized programmers.  Instead, Maya Cakmak from Georgia Tech, envisions robots that can be programmed by their end-users for their own specific needs.  This past summer, Maya worked on developing a spoken dialog interface that allows a user to program new skills by physically moving PR2’s two arms and using simple speech commands.  

Imagine purchasing a brand new “programmable” robot.  How would you know what to do, to make the robot do something?  This is not a problem for many of our daily appliances as they considerably limit possible user actions.  For functionality like robot programming with a verbal dialog interface, however, it is important to guide the user with appropriate feedback from the robot and provide supplementary materials such as user manuals, tutorials, or instructional videos.  Maya conducted a user study that replicates the described scenario.  Participants in this study (15 men and 15 women, ages 19-70) with no prior knowledge of how to program the robot were left alone with the robot and a combination of supplementary materials.  They had to figure out on their own how to program different skills such as picking up medicine from a cabinet or folding a towel.

The study revealed that information presented in the user manual easily gets overlooked and instructional videos are most useful in jump starting the interaction. In addition, trial-and-error plays a crucial role especially for achieving a certain proficiency level.

User studies like Maya’s, provide important insights into how the interface and the supplementary material should be designed to improve the learnability of end-user programmable robots. Check out the video for sample interactions and look for Maya and Leila Takayama’s upcoming publication for more details.