Robot Perception

Our approach in the field of Robot Perception encompasses numerous research goals and pursuits. For instance, we investigate lifelong mapping and environment reconstruction. Current robot mapping systems take one sweep through an environment, construct a map off-line, and then use it until the environment changes enough to require another sweep. Our goal is to continuously update a detailed geometrical map as the robot goes about its tasks, using both vision and laser sensors to recognize places, and label spaces and objects therein.

Another perception focus is reliable object detection, registration, and tracking. Manipulating tabletop objects is one of the main goals of the PR2 robot. Our aim is to have near-perfect, real-time recognition of common small objects. One critical part of this project is the creation of CAD models, either through an off-line modeling system, or with "in-hand" creation for novel objects.

Additionally, we work on detecting people and understanding their actions. Our robots are meant to co-exist with people in everyday environments. Initially, we detect people to navigate in a more friendly fashion; we are working towards tracking face pose and body motion as part of interacting with people in cooperative tasks.

Relevant Blog Posts

Publications

2012
A General Framework for Tracking Multiple People from a Moving Camera Choi, Wongun., Pantofaru, Caroline., and Savarese, Silvio IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 11/2012, (2012) Abstract  Download: pami_preprint.pdf (1.78 MB)
2011
CAD-Model Recognition and 6 DOF Pose Aldoma, Aitor., Blodow, Nico., Gossow, David., Gedikli, Suat., Rusu, Radu., Vincze, Markus., and Bradski, Gary ICCV 2011, 3D Representation and Recognition (3dRR11), 11/2011, Barcelona, Spain, (2011)
Detecting and Tracking People using an RGB-D Camera via Multiple Detector Fusion Choi, Wongun., Pantofaru, Caroline., and Savarese, Silvio Workshop on Challenges and Opportunities in Robot Perception, at the International Conference on Computer Vision (ICCV), 11/2011, (2011)  Download: detecting_people_2011.pdf (9.96 MB)
ORB: An Efficient Alternative to SIFT or SURF Rublee, Ethan., Rabaud, Vincent., Konolige, Kurt., and Bradski, Gary International Conference on Computer Vision, 11/2011, Barcelona, (2011)  Download: orb_final.pdf (932.61 KB)
A Side of Data with My Robot: Three Datasets for Mobile Manipulation in Human Environments Ciocarlie, Matei., Pantofaru, Caroline., Hsiao, Kaijen., Bradski, Gary., Brook, Peter., and Dreyfuss, Ethan IEEE Robotics & Automation Magazine, Special Issue: Towards a WWW for Robots, 06/2011, Volume 18, Issue 2, (2011)  Download: ram_2011_datasets.pdf (2.63 MB)
User Observation & Dataset Collection for Robot Training Pantofaru, Caroline Proc. of Human-Robot Interaction (HRI), 03/2011, Lausanne, CH, (2011) Abstract  Download: lbr251-pantofaru.pdf (494.1 KB)
Using Depth Information to Improve Face Detection Burgin, Walker., Pantofaru, Caroline., and Smart, William D. Proc. of Human-Robot Interaction (HRI), 03/2011, Lausanne, CH, (2011)  Download: walker-pantofaru-smart.pdf (832.95 KB)
2010
Fast 3D Recognition and Pose Using the Viewpoint Feature Histogram Rusu, Radu Bogdan., Bradski, Gary., Thibaux, Romain., and Hsu, John Proceedings of the 23rd IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 10/2010, Taipei, Taiwan, (2010)  Download: Rusu10IROS.pdf (1.98 MB)
Sparse Sparse Bundle Adjustment Konolige, Kurt British Machine Vision Conference, 08/2010, Aberystwyth, Wales, (2010)  Download: ssba.pdf (1.16 MB)
3D Pose Estimation, Tracking and Model Learning of Articulated Objects from Dense Depth Video using Projected Texture Stereo Sturm, Jürgen., Konolige, Kurt., Stachniss, Cyrill., and Burgard, Wolfram Proc. of the Workshop RGB-D: Advanced Reasoning with Depth Cameras at Robotics: Science and Systems (RSS), 06/2010, Zaragoza, Spain, (2010) Abstract  Download: sturm10rssws[1].pdf (953.56 KB)
View-based maps Konolige, Kurt., Bowman, James., Chen, J. D.., Mihelich, Patrick., Calonder, Michael., Lepetit, Vincent., and Fua, Pascal International Journal of Robotics Research (IJRR), Volume 29, Issue 10, Number 10, (2010)  Download: viewmap.pdf (2.24 MB)
2009
View-based maps Konolige, Kurt., Bowman, James., Chen, J. D.., Mihelich, Patrick., Calonder, Michael., Lepetit, Vincent., and Fua, Pascal Proceedings of Robotics: Science and Systems, June, Seattle, USA, (2009)  Download: viewmap.pdf (2.29 MB)
Partial View Modeling and Validation in 3D Laser Scans for Grasping Blodow, Nico., Rusu, Radu Bogdan., Marton, Zoltan Csaba., and Beetz, Michael The 9th IEEE-RAS International Conference on Humanoid Robots (Humanoids), 12/2009, Paris, France, (2009)
Real-time CAD Model Matching for Mobile Manipulation and Grasping Klank, Ulrich., Pangercic, Dejan., Rusu, Radu Bogdan., and Beetz, Michael The 9th IEEE-RAS International Conference on Humanoid Robots (Humanoids), 12/2009, Paris, France, (2009) Abstract