Willow Garage Blog

PR2GRASP: From Perception and Reasoning to Grasping

Household robotics is an area that can potentially change our lives for the better. Most importantly, disabled and elderly people can become less dependent on others if household robots are available to clean, cook, bring clothes and medication, and perform other home chores.  Unfortunately, robust household robotics is also one of the more difficult areas of the field.  At Penn, the PR2 recipients will tackle a number of the challenges facing the personal robotics field.

One such challenge is navigation in dynamic environments. For personal robots to be effective, they must be able to navigate while taking into account the movements of dynamic obstacles such as people.  To this end, the Penn group will use the PR2 to continue their work on such issues as motion planning for navigation and people tracking in cluttered spaces for navigation.    

Another project on Penn's list is spring-loaded door opening with the PR2.  While the PR2 can already open doors at Willow Garage, the robot is not yet capable of opening spring-loaded doors. In order to function robustly in real environments, a personal robot must be equipped to manipulate a variety of doors.  Because the PR2 has two arms, the robot might soon be able to open a spring-loaded door without the door closing on the robot before it releases the door handle and travels through the doorway.  This ability to open more challenging doors can make the PR2, and other robots, more useful in our everyday lives.

In addition to navigation and door-opening, the team at Penn aims to modify the PR2 hardware and create support for modular end-effectors.  Penn is one of the only Beta Sites to propose hardware changes, and the team hopes to improve the PR2's capabilities by making available end-effectors that are capable in ways that the current grippers are not.

Penn's proposal also includes a variety of other diverse projects, including: visual localization and pose estimation of objects for grasping; transferring natural handheld objects between the PR2, humans, and other robots; learning salient features for better perceptual processing by the PR2; planning and controls for two-arm manipulation; and establishing telepresence through the PR2 with the Pennochio project.

To see what's currently happening on the PR2GRASP project, check out www.pr2grasp.com.

The Team

This project includes the efforts of eight researchers coming from three different areas of Engineering and Science and specializing in such diverse areas of robotics as vision, planning, controls, machine learning, haptics and hardware design. These joint efforts are being led by Maxim Likhachev.

• Prof. Maxim Likhachev: Search-Based Planning in Robotics
• Prof. Vijay Kumar: Controls of Dynamic and Multi-Agent Systems
• Prof. Mark Yim: Modular Robotics
• Prof. Jianbo Shi: Computer Vision
• Prof. Camillo J. Taylor: Computer Vision and Robotics
• Prof. Kostas Daniilidis: Space and Motion Perception in Robotics
• Prof. Katherine J. Kuchenbecker: Haptics and Teleoperation
• Prof. Daniel D. Lee: Machine Learning in Robotics

The team also includes an excellent group of graduate students and postdocs, including the following: Ben Cohen, Steve Gray, Soonkyum Kim, Mike Phillips, Cody Phillips, Matt Piccoli, Joseph Romano.

Presentation

Below is a video of the Penn team presenting their proposal to the rest of the PR2 Beta Program participants. You can download the slides as PDF.

This article written with assistance from Maxim Likhachev.

Developing the Personal Robotics Market: Enabling New Applications Through Novel Sensors and Shared Autonomy

Limited access to quality robotics research platforms is one significant obstacle to the development of new robotics applications.  Bosch’s PR2 Remote Lab will extend access to the PR2 platform to additional research labs which did not receive their own PR2.  Of course, Bosch also has its own plans for its PR2 beyond sharing it with the rest of the world.  The robotics team at the Bosch Research and Technology Center North America (RTC) will focus on developing solutions for the personal robotics market by making robots safer, affordable and capable.  The Bosch robotics team in Palo Alto closely collaborates with experts from Bosch Corporate Research based in Stuttgart, Germany, who bring in additional expertise in sensing, signal processing and manufacturing.

Bosch plans to integrate its latest sensor technology in the PR2 to explore new applications and lower costs.  These sensors include accelerometers, gyros, force sensors, air pressure sensors and a new near range sensing system.  These sensors will also be shared with other PR2 Beta Sites.

In the long run, Bosch is focused on developing the robotics market.  That means building affordable robots that work reliably in complex environments – not an easy task.  One approach to this problem that Bosch will explore is shared autonomy: including a human in the loop to help out on those tasks still too difficult for a robot to quickly or reliably complete on its own.  The goal is to combine the strengths of autonomous systems and human operators to improve performance and reduce the cost of commercial systems.

Bosch has been a contributor to ROS since March 2009.  They will extend their 3D reconstruction work, previously mentioned here, to ROS and the PR2.  This will allow the PR2 to build accurate, textured 3D models of its environment.

See the Bosch press release here.

The Team

The Bosch team combines the expertise of researchers with a background in computer science, mechanical, electrical and aerospace engineering with the corporate expertise in sensing, perception, signal proccessing and manufacturing.  Several team members are also active in the Stanford Racing Team working on autonomous vehicles.

• Jan Becker: Principal Investigator
• Christian Bersch: Planning, Perception
• Charles DuHadway: Shared Autonomy, Remote Lab
• Benjamin Pitzer: 3D Reconstruction, Remote Lab
• Soeren Kammel: 3D Perception, Image Processing
• Lukas Marti: Sensors, Signal Processing

The team also includes some great interns: Hao Dang (Columbia), Adam Stambler (Rutgers), Michael Styer (Stanford), Joerg Wagner and Sebastian Haug (both Stuttgart). Researchers at other Bosch locations will also contribute to the project.

Presentation

Below is a video of Jan Becker presenting Bosch's proposal to the rest of the PR2 Beta Program participants. You can download the slides as PDF.

Article written with assistance from Charles DuHadway.

STAIR on PR2

The PR2's arrival at Stanford University is a homecoming of sorts.  The PR (Personal Robot) project emerged from Ken Salisbury's lab at Stanford in the form of PR1.  Moreover, what was once the STAIR lab's Switchyard system has morphed into today's ROS.  We're excited to see ROS and the PR2 at Stanford, ready for more tinkering and growth.

The STAIR (STanford AI Robot) project seeks to develop the software needed to put a general-purpose robot in every home.  The team is making the PR2 the next robotic platform for STAIR, and will develop open-source software towards this goal.  The lab will transition the STAIR perception, grasping, and manipulation software to enable the PR2 to carry out three applications: fetching items, inventory taking, and clearing a dining room table.

The first component that the group plans to transition to the PR2 is their software for robotic perception, much of which is already released as open-source, but not yet adapted for the PR2.  They have developed algorithms for embodied perception, which take advantage of the physical presence of the robot to recognize objects more accurately.  The team has also developed algorithms that choose where to foveate (i.e., point a robot head or camera) to efficiently find/recognize objects, and use multiple views of an object to facilitate better recognition.  The STAIR lab proposes to integrate these perception tools with the PR2, and thereby develop an object detector with an unprecedented level of performance.

The second component to be transitioned will enable the PR2 to grasp novel objects.  The group's grasping algorithm explicitly reasons about the placement of multiple fingers on an object, and they will transition this algorithm to the PR2, modifying it to work with the robot’s grippers and sensors.  Additionally, the STAIR team will address the problem of smoothly placing objects on a surface. Finally, the group will use their algorithms to address the problem of grasping novel objects from previously known object classes. Specifically, they will develop class-specific algorithms for grasping tableware and cutlery.

The third component that the group will transition to the PR2 is the ability to use novel doors and elevators.  The STAIR lab will integrate their door opening methods on the PR2, allowing for generalizability to a broader range of doors and door handles than has been possible with their prior hardware platforms.  They've already made quick progress on this.  Within twelve hours of setting up their PR2, the group ported their existing door-opening code. This is impressively fast given how different the STAIR 1 Robot is from the PR2.

The Team

The project is led by Prof. Andrew Ng and will include contributions from Prof. Ken Salisbury and Prof. Oussama Khatib.

The team is also comprised of a group of graduate students, including:

• Morgan Quigley
• Ellen Klingbeil
• Adam Leeper 
• Adam Coates
• Quoc Le
• Adam Vogel
• Sai Prashanth
• Deepak Rao

Presentation

Below is a video of Morgan Quigley and Adam Leeper presenting Stanford's proposal to the rest of the PR2 Beta Program participants. You can download the slides as PDF.

Article written with assistance from Morgan Quigley.

We're packing up a PR2 and heading to CVPR in San Francisco.  We'll be at the CVPR 2010: Computer Vision for Human-Robot Interaction Workshop on Monday, from 2-6 pm.  The PR2 will be showing a variety of demos at the poster session on Tuesday night, from 5:20-9:00 pm.  Hope to see you there!