Giving ears to PR2 with 8Sounds and ManyEars
During his visit, François Ferland from ntRoLab at Université de Sherbrooke in Québec, Canada was able to integrate a microphone line array into the PR2. A microphone array can enhance speech and sound recognition in real-world settings by allowing the localization, tracking, and separation of multiple simultaneous sources. The 8Sounds and ManyEars project is a complete open hardware and software solution that enables robots to hear sounds from their environment and know where they came from. It consists of an 8 microphone array, a compact low-power USB sound card (8 Sounds), and a software package (ManyEars) to locate the orientation of up to 4 simultaneous sources and separate the signal of up to two of these sources.
Both halves of the project have been developed at IntRoLab from Université de Sherbrooke in Québec, Canada.
8Sounds is a low cost, USB-powered sound card with 8 mono inputs and one stereo output. The card was designed with undergrad students to solve problems we encountered when using traditional, musician-oriented sound cards. These cards are usually bulky, require external power supplies, and have features such as MIDI ports that are not typically needed in robotic applications. Our solution is compact, draws very little power, and is easy to install on a robot. The 8Sounds system also includes powered, omnidirectional electret microphones with differential signaling for low noise acquisition. The sound card uses the USB Audio Class 2.0 protocol, and does not require special drivers in either Linux or Mac OS X.
The ManyEars algorithm, running on any conventional PC and released under the GNU Public License (GPL), performs real-time beamforming for localization, particle filtering for tracking, and Geometric Source Separation (GSS). It’s an easy to integrate C library and a ROS package is also available. A Qt-based GUI is also available for tuning parameters.
This demonstration shows the integration of the whole system on a PR2. The microphones have been installed on the upper part of its torso and the 8Sounds card is tucked under the shell, below the PR2’s network router. The microphones do not have to be in a particular geometrical configuration or orientation, only their position relative to a (user-chosen) reference point is needed.
The 8Sounds and ManyEars system is simple to integrate into many types of situations where sound localization and tracking is needed, such as human-robot interaction. It can be installed on robots as small as the TurtleBot and as complex as the PR2. By being both open hardware and open software, robot developers can easily customize and enhance any part of the system to their applications.
For more details on the system please refer to F. Grondin, D. Létourneau, F. Ferland, V. Rousseau, F. Michaud (2013), “The ManyEars Open Framework - Microphone array open software and open hardware system for robotic applications”, Autonomous Robots, 34:217-232.
The related ROS packages are available here. The PR2-specific demonstration code is currently under the “jn0_patches” branch, but will eventually be merged into the master branch.
For more information visit the project website for links to schematics, BOMs, and relevant software repositories.