Web-based Interactive Markers and Realtime 3D Point Cloud Streaming
At Willow Garage, we believe in the power of the web enabling new applications in robotics. The web browser, taking advantage of emerging HTML5 web standards such as WebGL, websockets, and unified video streaming, can be a powerful and versatile frontend for accessing, operating and gathering information from robots. Today we are announcing a set of new Open Source libraries for 3D visualization and interaction, promoting the development of new web-based frontends for ROS systems.
On the robot side, dedicated ROS nodes throttle the transmission of TF information and provide precomputed transforms for the Interactive Markers client. The 3D meshes and textures needed for the robot model are provided by an HTTP file server. In addition, the depth and color information from the Kinect is jointly encoded into a compressed video stream that is provided via HTTP. In order to increase the dynamic range of the streamed depth image, it is split into two individual frames that encode the captured depth information from 0 to 3 meters and from 3 to 6 meters, respectively. Furthermore, compression artifacts are reduced by filling areas of unknown depth information with interpolated sample data. A binary mask is used to detect and omit these samples during decoding. Once this video stream is received by the web browser, it is assigned to a WebGL texture object which allows for fast rendering of the point cloud on the GPU. Here, a vertex shader is used to reassemble the depth and color data followed by generating a colored point cloud. In addition, a filter based on local depth variance is used to further reduce the impact of video compression distortion.