3D Object Modeling

Research Engineer Romain Thibaux recently worked on teaching the PR2 how to determine the shape of new objects. Using its stereo cameras, the robot can obtain a 3D view of an object.  However, in a single view, many parts of the object are not visible. Romain's work incorporates many such views, taken from different angles, into a complete 3D model. The approach uses probabilities to calculate how much evidence is present to suggest that the space does in fact contain part of an object, or does not. These probabilities are then extended to unobserved regions using the heat equation, which diffuses the probabilities through space just as heat diffuses through a frying pan. In the end, the hot region is the region of high probability, and the boundary between the hot and cold regions is the surface of the object.

Romain's active perception approach allows the robot to determine the 3D shapes of novel objects in a way that is not possible with static cameras. This dynamic approach augments the PR2's perception capabilities and will allow for better manipulation, object identification, and rendering of acquired objects in simulation.

To learn more, check out the video below, or take a look at the model assembler and heat equation solver at ROS.org.