TUM Rosie and PR2 James make pancakes together

It probably happens hundreds if not thousands of times every day; two college roommates get together to make some pancakes for dinner.  The only difference this time is that the parties involved were robots.  In this case, two robots from the Munich-based cluster of excellence CoTeSys (Cognition for Technical Systems) starred as chefs, demonstrating their capabilities in the world of robotics -- and in the kitchen.

In the video below, you can see James, a PR2 Beta Program robot, opening and closing cupboards and drawers, removing the pancake mix from the refrigerator, and handing it over to Rosie.  Rosie is another robot at TUM that is deployed in the Assistive Kitchen environment of the Intelligent Autonomous Systems Group.  Rosie cooks and flips the pancakes, and then delivers them back to James.

Behind this domestic tableau is a demonstration of the capabilities of service bots.  This includes characteristics such as learning, probabilistic inference, and action planning.

James uses the Web for problem solving, just like we would.  To retrieve the correct bottle of pancake mix from the fridge, it looks up a picture on the Web and then goes online to find the cooking instructions.

Rosie makes use of gravity compensation when pouring the batter, with the angle and the time for pouring the pancake mix adjusted depending on the weight of the mix.  The manipulation of the spatula comes in to play when Rosie's initially inaccurate depth estimation is resolved by sensors detecting contact with the pancake maker.

In the course of the demonstration, both robots cope with mechanical inaccuracies, shifting furniture, obstacles, and execution errors.  Failures in the task execution are dealt with using learned knowledge about the processes involved, the tools and their usage. If the spatula has not been grasped correctly, for example, the robot will notice this and adjust its grasp.

Rosie and James are not ready yet for haute cuisine, but the video provides an excellent sense of the future capabilities of service robots.

Much of the software used in this demonstration within the CoTeSys cluster will soon be made available to robot researchers in the TUM ROS repository.

As a reminder, it's just a few more days until the first CoTeSys-ROS Fall School on Cognition-Enabled Mobile Manipulation. It's being held from November 1-6, 2010 at Technische Universität München (TUM) in Munich, Germany.