Robots for Humanity: The PR2 as Assistive Robot

“Ask a man in solitary confinement if he wants just an hour of freedom a day. I tell you, he will live for it. Robots are my freedom.” 
– Henry Evans (Forbes, Dec 13, 2012)
Robots for Humanity is a collaboration between Willow Garage, the Healthcare Robotics Lab at Georgia Tech led by Charlie Kemp, the Man and Machines Group at Oregon State led by Bill Smart, and Henry and Jane Evans.

Henry Evans is mute and quadriplegic, having suffered a brainstem stroke when he was just 40 years old. Following extensive therapy, Henry regained the ability to move his head and use a finger, which allows him to operate computers. Last year, Henry caught a TV interview of Georgia Tech Professor Charlie Kemp showing research with the Willow Garage PR2 robot. Willow Garage and Professor Kemp were contacted by Henry shortly afterwards, and we have been collaborating since then. We are currently exploring ways for Henry to use a PR2 robot as his surrogate. Every day, people take for granted the simple act of scratching an itch. In Henry's case, 2-3 times every hour of every day he gets an itch he can't scratch. With the aid of a PR2, Henry was able to scratch an itch for himself for the first time in 10 years. While this is only a first step, it demonstrates how people with severe physical disabilities could use personal robots to gain independence. In another example, Henry recently used the PR2 to shave his cheek. We are actively researching ways for Henry and others to perform tasks like these on a daily basis.

Currently, Henry uses a head tracker to operate a variety of experimental user interfaces. These interfaces allow him to directly move the robot's body, including its arms and head. They also let him invoke autonomous actions, such as navigating in a room and reaching out to a location. Robots that complement human abilities are extremely valuable, especially when they help us do things that we can't do by ourselves.  Our goal is to get robots in homes to help people like Henry and Jane Evans.

Robots for Humanity was recently featured on CBS Evening News.

Some of the highlights of the Robots for Humanity initiative are as follows:

  • In October 2011, Henry gave Halloween candy to children through the PR2 robot at a local mall.  As a child would approach the robot and hold out his or her candy bag, Henry would command the robot to pick up a candy from the side table and then place it inside the child’s bag.
  • In Henry’s home, he has demonstrated navigating the robot through his home to find and deliver to himself objects such as a towel from a drawer in the kitchen, and small food items from inside the fridge.
  • One of Henry's very first accomplishments was to manipulate the PR2 to scratch his own nose--the first time he had been able to do so in over a decade. Since then, Henry has not only used the PR2 to shave himself, but also tele-operated a PR2 located at Georgia Tech and shaved Prof. Kemp remotely.

The Robots for Humanity team at Willow Garage is led by Kaijen Hsiao and Matei Ciocarlie.  Their approach is to research and develop a diverse suite of open source software tools that blend the capabilities of the user and the robot. This has resulted in what is, to the best of our knowledge, the first example of a mobile manipulation platform operated by a motor impaired person using only a head-tracker single-button mouse as an input device, and used for varied and unscripted manipulation tasks in a real home as well as limited forms of social interaction.

Henry controlling the PR2 & grasping a towel from his kitchen drawer

The goal of putting robots into real homes to help people with disabilities is a long-term vision for our project. By actively involving the users, Henry and Jane Evans, in our participatory design process, we have made tangible progress towards assistive capabilities that are both useful and usable.  We also anticipate that by putting robots into the real homes of people with disabilities early and often, we can better direct our research to overcome the real-world obstacles to the use of mobile manipulators as an effective assistive technology.

Our future challenges include enabling Henry and Jane to use a PR2 in their home for longer durations, and evaluating our methods with other people with motor impairments.

Robots for Humanity is made up of the following individuals and institutions:

Henry and Jane Evans

Willow Garage, Menlo Park, CA:

Kaijen Hsiao, Matei Ciocarlie, Steve Cousins, Andreas Paepcke, Caroline Pantofaru, Leila Takayama

Healthcare Robotics Lab, Georgia Institute of Technology, Atlanta, GA:

Charlie Kemp, Tiffany Chen, Phillip Grice, Kelsey Hawkins, Advait Jain, Chih-Hung King, Hai Nguyen, Sarvagya "Survy" Vaish

Oregon State University, Corvallis, OR:

Bill Smart and Daniel Lazewatsky

 

Code written for this project is almost entirely public; you can try it out yourself if you have a PR2, or try much of it in simulation if you do not.
Here are links to the code packages for some of the interfaces that Henry uses:

The interface for teleoperating the PR2 to do scratching, shaving, and other tasks near one's own body can be found in the ROS package assistive_teleop.

The interface for graphically scripting state machines to perform gestures or automate manipulation tasks can be found in the ROS package rcommander.

The interface for teleoperating the PR2 to perform remote manipulation tasks can be found in the ROS package pr2_interactive_manipulation.

Here is a video, created in March of 2012, explaining the interface for remote manipulation, and showing Henry using it to pass out candy at Halloween and also fetch a towel from his kitchen to bring back to his wheelchair in the living room.


Project group portrait as of November 2011:

Robots for Humanity