Google's new AI model converts sign language to text
What's the story
Google has unveiled a new artificial intelligence (AI) model, called Sign-Language-to-Text (SL2T), at its recent Made by Google event. The innovative technology is designed to convert sign language directly into written text. The SL2T model will be integrated into the Gboard and Live Transcribe apps on the Pixel 11 series, making communication easier for hearing-impaired users.
Functionality
SL2T can be used in web search, text messaging
The SL2T model uses a phone's camera to capture sign language, enabling users to communicate in various digital platforms like web search, text messages, emails, documents, or Google Gemini queries.
In face-to-face conversations, users can use Live Transcribe to sign their responses instead of typing them out.
The feature currently supports American Sign Language (ASL) translated into English.
Development
Trained on over 100,000 hours of data
Google trained the SL2T model on over 100,000 hours of data from more than 50 sign languages, with ASL accounting for about a quarter of the training data.
The company says this multi-language training helped the model learn common structures across different sign languages.
To ensure user privacy, an on-device MediaPipe Holistic model tracks points on the signer and converts them into geometric landmark coordinates instead of sending raw camera footage to a server.
Efficiency
SL2T is the most capable sign language translation model yet
The SL2T model translates directly from the landmark sequence into text, skipping the intermediate system of annotations known as 'glosses.'
Google says this approach prevents missing elements such as facial expressions, spatial constructions, and other non-manual markers that carry meaning in sign languages.
On the FLEURS-ASL benchmark, SL2T is the most capable sign language translation model to date, with a 70 BLEURT zero-shot score, substantially higher than previous results.
Twitter Post
A live demo shared by Sundar Pichai
One of the most important updates coming out of Made by Google today is our new sign-to-text feature in Gboard & Live Transcribe.
— Sundar Pichai (@sundarpichai) August 12, 2026
Built in partnership with the Deaf community, this can help people who sign ASL communicate more seamlessly with their phone or with those who don't… pic.twitter.com/Je0T7wn3bM