Intuitive Robot Interfaces with Web Robotics

During his internship, Russell Toris from Worcester Polytechnic Institute worked on improving the ROS JavaScript libraries to make creating intuitive user interfaces easier for researchers. These improvements have helped to lower the bar for both robotics researchers who want to design web interfaces, and web designers who want to connect their work to robotics.

Using HTML5, JavaScript, and other web technologies has proven to be a great way to expose robotics to a diverse user base across the Internet.  We can use these technologies to allow non-expert users to program robots by creating intuitive, browser-based user interfaces.  As part of the Robot Web Tools effort, Russell has worked on creating high-level libraries designed for easily creating intuitive, browser-based user interfaces.  As a demonstration of the power of these tools, he used the latest software from ROS and the Robot Web Tools projects to develop a user interface that allows users to easily direct the robot to manipulate objects. Users can now drag-and-drop objects in their web browser in order to direct the robot to perform simple pick-and-place tasks. 

By using such an interface, non-expert users can now easily instruct a robot to manipulate the world by simply specifying how they want to world to look. The robot begins by perceiving any known objects on the table, and displays them to the user as 3D meshes. The user is then able to drag-and-drop objects around in order to specify how he or she wants the objects arranged. Users can also specify configurations of objects and save them as templates, as a form of programming by demonstration. Having such an interface available in web browsers means that non-expert users with a wide array of operating systems and browsers can perform manipulation tasks remotely, using their own computers or mobile devices.

This work was done in collaboration with Kaijen Hsiao from Willow Garage, Sarah Osentoski from Bosch, and Chad Jenkins from Brown University. For more information, see ros3djsSharedAutonomyToolkit, and