Commanding Robots Using Natural Language

Would you like to see a robot give a high five by just asking it to? Brian Thomas, from the Brown University Robotics Group, came up with a way to command robots to give a high five and perform many other actions using spoken or written language. During his internship at Willow Garage, Brian developed a system, called RoboFrameNet, so that robots respond to pre-programmed spoken or written human language actions.

In RoboFrameNet, speech or text input is transferred into robot actions through an intermediary called semantic frames. Semantic frames, developed for the FrameNet Project, describe a scene (or series of actions), such as navigating in a hallway, picking up an object, or following a person. Because RoboFrameNet is expandable, programmers can easily add new actions and processing capabilities to the robots.

RoboFrameNet is integrated into the Android application manager from Willow Garage. RFNServer lets developers integrate their work with RoboFrameNet.

For more details, see the video, and check out the software in the RoboFrameNet package on