Willow Garage Blog

March 1, 2011

Willow Garage and the National Institute of Standards (NIST) are launching a new type of robot perception challenge aimed at both:

  • establishing which perception problems have solutions and
  • advancing perception solutions to new domains.

Figure: A 3D textured point cloud of some of the objects. In this image 4 of the 5 objects are recognized with coordinate systems overlaid representing the object pose relative to the robot.

Most existing challenges in computer vision and perception aim at making progress in complex object class identification tasks such as pursued by the PASCAL Visual Object Classes challenge. These contests are useful, for example, in creating progress in web image retrieval tasks. In contrast, the Solutions for Perception Challenge aims at advancing sensing based robotics by identifying domains where roboticists can rely on having a working perception solution. Another key difference from other perception challenges is that to be of use for robotics, a perception solution must not only identify objects but calculate the 3D pose of the object relative the the robot.

First Challenge

The challenge will debut at the IEEE International Conference on Robotics and Automation (ICRA) 2011 in Shanghai, China, on May 9-13, 2011. There will be a somewhat unusual cash prize structure for contestants: up to $10,000 dollars will be awarded exponentially starting at $3.50 for a first prize winner who achieves 80% recognition and increasing exponentially from there. This prize structure reflects the increasing difficulty of achieving scores closer to 100%. The first contest focuses on recognizing rigid textured objects.

Contestants can make use of ROS infrastructure and code stubs along with OpenCV and PCL libraries for 2D and 3D image and point cloud processing.

Joining the contest

To join the contest, please send an email with subject "Join" to: solutionsinperception@gmail.com.  Please see the challenge wiki for instructions, data and starter code links


This challenge is organized by Gary Bradski (Willow Garage) and Tsai Hong (NIST). Further encouragement and support has come from Thomas Kalil, Associate Director for Policy at the White House Office of Science and Technology and from Elena Messina, Chief of Intelligent Systems Division at NIST.

February 28, 2011

IROS 2011 will celebrate the last 50 years of robotics with a series of special symposia. One of these symposia is on Mobile Manipulation. This symposium will consist of a combination of invited talks by leading researchers and practitioners in mobile manipulation and invited papers addressing the state of the art in this field and future directions. Mobile manipulation covers a wide variety of areas and we expect to showcase the breadth of research happening in this exciting area at this symposium.  

The symposium will cover both the theoretical and practical aspects of mobile manipulation with a special focus on implementation on real-world systems. The topics of interest for this symposium cover a wide-variety of sub-fields in robotics but we will focus on work in these areas as applied to mobile manipulation.

  • indoor mobile manipulation including personal and service robots
  • field (outdoor) mobile manipulation systems
  • space based manipulators
  • aerial manipulation systems
  • underwater manipulators
  • humanoid robots
  • perception
  • motion planning
  • control
  • task planning
  • human-robot interaction

Plenary Speakers

  • Oliver Brock, TU Berlin


If you would like your IROS submission to be considered for being part of the symposium, please send the organizers a title, list of authors and a tentative abstract in advance (on or before March 14th, see contact details below). We will work with the conference program committee to make the assignment of papers to the symposia and conference sessions on the subject of mobile manipulation. Please also look at the general list of instructions for authors wanting to submit papers for the symposium.

The deadline for submission to IROS is March 14, 2011. Submission details can be found at: http://www.iros2011.org/cfp.  We look forward to your contributions for IROS 2011, Mobile Manipulation Special Symposium.


  • Sachin Chitta, Willow Garage
  • Max Bajracharya, JPL


  • iros-mobile-manipulation-symposium AT willowgarage DOT com


February 23, 2011

PR2When The George Washington University began the push to transform its School of Engineering and Applied Sciences into a leading institution two years ago, it ranked the emerging field of robotics as a key technology investment. The university recruited Evan Drumwright last Spring to help build a robotics program for the engineering school.

Drumwright comes to GWU from USC, where he focused his research on humanoid robots. At the time of his hiring, Drumwright had plans in place to build a mobile manipulator robot. These plans included purchasing an anthropomorphic arm, usable base, an array of sensors, and a host of other discrete parts. Drumwright was in the processing of acquiring all of the hardware he needed and finalizing plans to spend the next year or so building his own robot when he heard about the PR2 from Willow Garage. A few weeks ago, Drumwright took delivery of his PR2 and is needless to say excited about having a ready-made platform for R&D.

According to Drumwright, "I wasn't looking forward to spending upwards of one year simply building a robot so that we could commence our research. There was no guarantee that all of the components would work together as planned. Instead, we now have a state-of-the-art PR2, that's already assembled and tested. To say that we accelerated our research is an understatement; we had the PR2 out of crate and functional within 30 minutes."

Drumwright calls the PR2 "the most advanced mobile manipulator in the world" and sees it as putting GWU in a position to conduct unique research. "There is a large amount of code already available for the PR2," says Drumwright, "and a considerable community of PR2 users online. These advantages allow my student researchers to get robot code up and working in a short period of time."

We'll reach out to Drumwright again in a few months, after he's had a chance to really get his research on the PR2 underway. Some of the research areas Drumwright is exploring include robot manipulation of soft-body objects, bi-manual manipulation, and the interplay of robots within the human environment. In addition, Drumwright is looking forward to the possibilities for collaborating with DC-based researchers outside of GWU.

February 11, 2011

When: Monday, May 9, 2011

Where: ICRA 2011 Shanghai, China.

Website: http://www.willowgarage.com/workshops/2011/icra_spme

Submission deadline: April 2, 2011

Previous workshops have brought together researchers working on use of semantic information in a variety of domains including mapping for mobile robots, mobile manipulation, and semantic robot vision. The main focus of this full-day workshop is on the autonomous acquisition of semantic information in intelligent robots and systems including perception, mapping and exploration, like the use of semantic knowledge to guide the acquisition of information from the environment. The workshop will provide a common forum where people can exchange ideas and experiences on all aspects of this problem, and identify the common questions and concerns.

The workshop will include invited talks on the state of the art in semantic perception, mapping and exploration as well as their use in different areas of mobile manipulation research. In addition, we welcome submissions in all related areas, including but not limited to:

  • Semantic robot vision and scene interpretation for mobile manipulation
  • Segmentation and annotation of natural scenes (e.g. from images or point clouds)
  • Exploration strategies for semantic mapping and knowledge acquisition
  • Semantic approaches for long-term operation in dynamic environments
  • Ontologies and efficient representations for managing semantic information in robotics
  • Use of semantic information in mapping (e.g. registration of sensory information) or knowledge acquisition


We solicit paper submissions, optionally accompanied by a video, both of which will be reviewed (not double-blind) by the program committee. An optional interactive poster session will be organized if there is a larger number of submissions. Accepted papers and videos will be archived at the Internet Archive. If there is sufficient interest, we will pursue publication of a special journal issue to include the best papers.

Papers should be in PDF, conform to the ICRA regular paper style guidelines, and be a maximum of 8 pages in length (shorter papers are welcome). Videos should be in the MPEG format, 3-5 minutes in length, and easily viewed with free video players (please try playing your video on a couple of different machines before submitting).

For submissions, please visit https://www.easychair.org/conferences/?conf=spme2011

Important Dates:

  • Submissions Due: April 2, 2011
  • Notification of Acceptance: April 18, 2011
  • Final Papers Due: May 1, 2011
  • Workshop at ICRA: May 9, 2011

Program Committee:

  • Francesco Amigoni, Politecnico di Milano, Italy
  • Michael Beetz, Technische Universitaet Muenchen, Germany
  • Sven Behnke, University of Bonn, Germany
  • Tom Duckett, University of Lincoln, UK
  • Joachim Hertzberg, University of Osnabrueck, Germany
  • Patric Jensfelt, KTH Royal Institute of Technology, Sweden
  • Jim Little, University of British Columbia, Canada
  • Eitan Marder-Eppstein, Willow Garage, USA
  • Bhaskara Marthi, Willow Garage, USA
  • Markus Vincze, TU Vienna, Austria
  • David Silver, Carnegie Mellon University, USA
  • Bertrand Douillard, University of Sydney, Australia


February 10, 2011

When: Friday, May 13, 2011

Where: ICRA 2011 Shanghai, China.

Website: http://mobilemanipulation.org/icra2011

Submission deadline: Update - The deadline has been extended to March 21, 2011

The purpose of this full-day workshop is to provide a forum for researchers across various fields to discuss methods for integrating perception and manipulation in the context of a unified mobile manipulation system. Recent years have seen tremendous progress toward the goal of autonomous mobile robots that can interact with unstructured, dynamic, and human-centered environments. Research in mobile manipulation has gained momentum with the development of integrated systems, (like Willow Garage's PR2 robot, Intel's HERB, DLR's Justin, the iCub, HRP2, etc.) with the necessary onboard sensing, computation, manipulation and mobility capabilities to start tackling such environments. Mobile manipulation is a highly integrative, interdisciplinary field. The workshop will bring together researchers from academia and industry from diverse disciplines to discuss innovations, applications, success stories, and future prospects for mobile manipulation. While contributions in all the relevant fields will be welcome, this workshop will focus on methods of integrating perception with manipulation and how methods in one field may inform another. The workshop aims to spark vibrant discussion with speakers from a wide range of disciplines, as well as lively panel discussions that address the current state of the art and future directions.

The workshop will include invited talks on the state of the art in different areas of mobile manipulation research. The list of invited speakers includes:

  • Matt Mason (CMU)
  • Tamim Asfour (KIT)
  • Stefano Soatto (UCLA)
  • Katherine Kuchenbecker (Penn)
  • Kurt Konolige (Willow Garage)
  • Satoshi Kagami (AIST)
  • Xiaofeng Ren (Intel)

The program committee includes:

  • Aaron Edsinger
  • Aaron Dollar
  • Ashutosh Saxena
  • Michael Beetz
  • Danica Kragic
  • Kei Okada
  • Ravi Balasubramanian

This workshop is the most recent in a series of workshops at RSS, ICRA and IROS centered on mobile manipulation.


We solicit paper submissions, optionally accompanied by a video, both of which will be reviewed (not double-blind) by the program committee. An optional interactive poster session will be organized if there is a larger number of submissions. Accepted papers and videos will be archived at the Internet Archive.  If there is sufficient interest, we will pursue publication of a special journal issue to include the best papers.

Topics of interest include (but are not limited to):

  • Mobile manipulation systems
  • Perception of unstructured environments
  • Modeling for mobile manipulation
  • Task and motion planning, planning under uncertainty, planning for sensing
  • Robust motion control
  • Learning by demonstration, learning by instruction

Papers should be in PDF, conform to the ICRA regular paper style guidelines, and be a maximum of 8 pages in length (shorter papers are welcome).  Videos should be in the MPEG format, 3-5 minutes in length, and easily viewed with free video players (please try playing your video on a couple of different machines before submitting).

Email submissions to: icra-mobile-manipulation-2011 at willowgarage.com. Please do not attach video files to email; include a URL instead.

Important Dates

  • Submission deadline: March 7, 2011
  • Notification of acceptance: March 28, 2011


  • Sachin Chitta, Willow Garage
  • Siddhartha Srinivasa, Intel Labs
  • Oliver Brock, TU Berlin
  • Dmitry Berenson, CMU
  • Dov Katz, TU Berlin
February 3, 2011

Thumbnail image for 3Dturtle.jpgWe were absolutely thrilled with eighteen entries to the ROS 3D Contest. The community really impressed us with creativity and technical prowess in these entries, and choosing the prizes was a difficult process. In fact, so difficult that we bent the rules and created two new prizes: 4th Overall and 2nd place Most Useful.

For the Overall prizes, we selected the entries that both amazed us and embraced the spirit of the contest: inspiring and providing building blocks for future Kinect hackers. We've been able to try many of these entries on our own thanks to the great code and documentation, and we hope that others will as well. It was hard to pick a favorite, but we kept coming back to Garratt Gallagher's Customizable Buttons. We tried it on our own desks, and it just puts a smile on your face: you draw a button whereever you please and press it. It made us feel like we were in a cartoon world where we could bend the rules of the universe. Garratt was also the most prolific -- with his six different entries, you can deconstruct the various components that he was able to assemble to produce very different results.

For second place, we selected Quadrotor Altitude and Obstacle Avoidance by the STARMAC project at Berkeley. We loved seeing what a Quadrotor with an Atom processor could do, and they went the extra mile to make sure others with quadrotors had a good starting point. For third place, Taylor Veltrop's Humanoid Teleoperation entry just kept getting better and better. He recently used his library to win best performance in the Robot Athlete Cup 2011 competition by doing robo-ikebana. We added a fourth place for Person Tracking and Reconstruction from a Mobile Base with a 7 DOF Manipulator by Chris Burbridge & Lorenzo Riano. Using a robot to turn a Kinect into a 3D scanner holds many possibilities.

For Most Useful, we were wowed. First prize went to RGBD-6D-SLAM from the University of Freiburg. SLAM with a Kinect holds the potential to unlock many applications, from creating 3D maps and 3D models to cheap autonomous navigation and much, much more. Normally "bleeding edge" means no one else can run it, but, in their case, they produced a 6D-SLAM solution that we were able to download and use in our own offices. There is much to improve, but the potential is huge. For second place, we chose ETH Zurich's Automatic Calibration of Extrinsic Parameters. Anyone mounting a Kinect on their robot should take a look at using their library.

Finally, our PrimeSense Dev Kit 5.0 Awards (thanks PrimeSense) go to Michael Ferguson and the Chemnitz University of Technology. We are confident, from their entries, that they will be able to put them to good use.

Thanks everyone!


1st Place ($3000): Customizable Buttons, Garratt Gallagher
2nd Place ($2000): Quadrotor Altitude and Obstacle Avoidance, Patrick Bouffard
3rd Place ($1000): Humanoid Teleoperation, Taylor Veltrop
4th Place ($500): Person Tracking and Reconstruction from a Mobile Base with a 7 DOF Manipulator, Chris Burbridge & Lorenzo Riano

Most Useful:

1st Place ($2000): RGBD-6D-SLAM, Felix Endres, Juergen Hess, Nikolas Engelhard, Juergen Sturm, Daniel Kuhner, Philipp Ruchti, and Wolfram Burgard
2nd Place ($1000): Automatic Calibration of Extrinsic Parameters, François Pomerleau, Francis Colas and Stéphane Magnenat

PrimeSense Dev Kit 5.0 Awards:

January 14, 2011

ICRA 2011 Workshop on Manipulation Under Uncertainty, Shanghai, China

The organizers invite you to submit a paper for review to the ICRA 2011 Workshop on Manipulation Under Uncertainty. This full-day workshop will consist of two contributed paper sessions and six invited speakers. In order to encourage discussion and exchange of ideas between particpants, all contributed papers will be presented in a poster session format, prefaced by short overviews from the podium. All accepted work will be published in a citable digital archive of the proceedings.

Important Dates

  • submission deadline: March 1st, 2011
  • notification of acceptance: on or before March 20th, 2011
  • workshop: May 13th, 2011


This workshop will focus on robot manipulation strategies that are robust to the uncertainty inherent in unstructured environments. Recently, the problem of uncertainty has been addressed in different ways, including: identifying manipulation strategies that are robust in task-specific situations, designing manipulator hardware that makes classes of manipulation activities more robust, and proposing planning algorithms that explicitly model uncertain variables. Our goal is to better understand all of these approaches by fostering discussion amongst the involved researchers.

If you have original research relevant to the area of manipulation under uncertainty, we would be very interested in receiving your submission. Our interests include, but are not limited to, the following topics in the context of manipulation:

  • explicitly modeling and/or planning under uncertainty (including general planning algorithms that could be used in the context of manipulation, even if they are not explicitly designed for such)
  • strategies for implicitly reducing uncertainty in specific, manipulation scenarios
  • manipulator designs that effectively handle uncertainty
  • new sensor technologies and their applications
  • other methods and techniques for manipulation in real-world scenarios affected by variability and uncertainty


All submissions should be in PDF format using the ICRA template. Papers should be approximately 4-6 pages in length. Submissions should be emailed to muu11@willowgarage.com by March 1st, 2011. Notification of acceptance will be on or before March 20th. All submission will be reviewed for relevance to the topic of the workshop as well as scientific merit. Reviews will be carried out by the organizing committee, with additional input from external reviewers as needed.

Invited speakers (confirmed)

  • Prof. Aaron Dollar, Yale University
  • Prof. Ken Goldberg, University of California, Berkeley
  • Prof. Kamal Gupta, Simon Fraser University
  • Prof. Matt Mason, Carnegie Mellon University
  • Prof. Joris De Schutter, Katholieke Universiteit (K.U.) Leuven
  • Prof. Jeff Trinkle, Rensselaer Polytechnic Institute


Contact the workshop organizers via email at muu11@willowgarage.com

Tentative schedule

  • 9:00 - 11:00: 3 invited talks, 40 mins each (including questions)
  • 11:00 - 11:15: podium poster introductions, 3 slides / poster
  • 11:15 - 12:00: poster session / free discussion / coffee and snacks
  • 12:00 - 1:30: lunch break
  • 1:30 - 3:30: 3 invited talks, 40 mins each (including questions)
  • 3:30 - 3:45: podium poster introductions, 3 slides / poster
  • 3:45 - 4:30: poster session / free discussion / coffee and snacks
  • 4:30 - 5:00: plenary discussion

Note: this tentative schedule is preliminary and subject to change.


  • Matei Ciocarlie (Willow Garage)
  • Kris Hauser (Indiana University)
  • Kaijen Hsiao (Willow Garage)
  • Robert Platt (M.I.T.)
  • Bill Smart (Washington University in St. Louis and Willow Garage)
January 11, 2011

Matthew Robards from the Australian National University recently finished up his work at Willow Garage working on reinforcement learning methods for robotics. In particular, created a value learner that can a robot distinguish between good and bad outcomes. Please watch the video above for an overview or view the slides below (download PDF) for more technical information. You can download the software from the value_learner package on ROS.org.

January 10, 2011

We think that ROS and the PR2 are great tools for educators. Both platforms allow students to focus on building the relevant parts of a system while incorporating less topical components from the open source community. Students get started faster and complete more impressive projects. Even more importantly, students can take components built in ROS to their next course, research project or job without worrying about licensing.

We've started a wiki page to list courses using ROS or the PR2, and to discuss teaching-related issues. Here are some course examples that you can use for inspiration:

Short Courses

University (Undergraduate & Graduate) Courses

If you're teaching a course using ROS or the PR2, please post a link at ros.org/wiki/Courses. If you have advice on setting up labs, course computers, or any other teaching-related topic, post those too. By sharing material, we'll all create effective courses more quickly.

January 6, 2011

TechFestThe PR2 robot is going on a trip to India making stops in Mumbai and Kanpur.  The PR2's first stop will be at the IIT Bombay Techfest, Asia's largest student organized science and technology festival. Techfest is expected to attract almost 20000 visitors this year. It is organized annually by IIT Bombay, one of the leading science and technology institutions in India. The PR2 is part of the "Rebooting Tomorrow" exhibition at Techfest, which showcases cutting edge technology from around the world. Look for us and the PR2 at Techfest from January 7th-9th, 2011.  The PR2 will be in the Main Building Foyer (MB Foyer) from 9:30 AM to 4:00 PM on all three days. See the Techfest schedule for a map of the venue and more details.

The PR2 will also be making a stop at Techkriti, IIT Kanpur's technology and science festival between February 17-20, 2011. More details on the PR2's trip to Kanpur will be posted soon on our blog.

The PR2's trip to the two IITs has been  made possible by a generous sponsorship from the TATA group. We thank them for this opportunity to demonstrate the PR2 in India.