source: Google DeepMind: Putting sign language AI into users’ hands
level: technical
Google DeepMind introduced SL2T, a multilingual sign-language-to-text model that translates sign language into written text. The model powers new features in Gboard and Live Transcribe on Pixel 11, starting with American Sign Language (ASL) to English. Users can sign to search, draft messages, or ask Gemini to perform tasks. In Live Transcribe, they can sign responses in conversations instead of typing. Google says signing is faster and more natural than typing for many Deaf users.
SL2T was trained on over 100,000 hours of data across more than 50 sign languages, with about a quarter in ASL. It achieves a zero-shot score of 70 BLEURT on the FLEURS-ASL benchmark, higher than any previously reported score. The model processes pose landmark coordinates from MediaPipe Holistic rather than raw video, protecting privacy. It handles left-handed signing and one-handed signing. Occasional errors remain in rare signs, rapid fingerspelling, and tense without context.
Google worked with Deaf communities through the AI Sign Language Advisory Committee and co-authored an impact report. The model builds on decades of research but is the first to reach consumer products at this scale. Google plans to expand to more sign languages and devices. The technology could improve access for the estimated 70 million Deaf and hard of hearing people worldwide, though current limitations mean it is not yet a full replacement for human interpreters.
why it matters: This is the first large-scale deployment of sign language AI in consumer products, potentially improving digital access for millions of Deaf users.
source: Google DeepMind: Putting sign language AI into users’ hands