These AI-powered headphones track and translate every voice around you
What's the story
Researchers at the University of Washington have developed a groundbreaking prototype of headphones that can monitor their surroundings, identify different speakers, and translate each linguistic thread with minimal delay. The device, dubbed Spatial Speech Translation, was created using commercially available components. Building on previous research that used headphones to isolate individual voices from group discussions, this innovative device promises to revolutionize communication across language barriers.
Device design
How do these headphones work?
The Spatial Speech Translation system combines Sony SH-100XM4 noise-canceling headphones with SonicPresence's SP15C - Binaural headphones. The latter captures sound from two different sources, just like humans hear. As the microphones detect sounds, the audio is sent to a mobile device running neural network models in real-time. The translated output is relayed back through the headphones in 1-2 seconds, but users preferred 3-4-second delay during testing for enhanced accuracy.
AI integration
The headphones adapt to the user's movement
The Spatial Speech Translation system not only recognizes different voices in a group conversation but also maintains the natural rhythms of speech, making the translated output sound authentic. It even adapts as users move around a room or turn their heads, leveraging artificial intelligence (AI) to focus on different conversational threads. "Our algorithms work a little like radar," said Tuochao Chen, lead study author and doctoral student at UW's Allen School.
Future plans
Currently, the headphones support 3 languages
Currently, the Spatial Speech Translation system supports conversational Spanish, French, and German. However, researchers believe it could eventually handle around 100 languages. They are now working on improving both the speed and accuracy of this innovative device. The code that powers these headphones has been made open-source by the researchers for others to experiment with.