National Association of Broadcasters

winner of the NAB Technology Innovation Award 2020

Automated Sign Language to produce signs helping the Deaf, not the hearing…

Automated Sign Language Production is the automatic translation from a spoken language to a sign language. The ultimate goal would be to automatically produce a video of a signer that is indistinguishable from a real person. If done well, this could be an important technology for the Deaf community, increasing the quantity of sign language content available. But of course, generating translations of the same quality as a human interpreter is extremely challenging.

Previous work into computational sign language has mainly focused on the recognition of sign language from a video, or a translation to spoken language, the opposite task from production. However, recognition is more applicable to the hearing, as it allows them to understand the Deaf. A native sign language user does not need a system that recognises sign. However, systems that can accurately translate and produce sign would be of use to the Deaf. Not to replace human interpreters, but to provide translation into native sign language when an interpreter is not available.

In the Content4All project, we having been working on an initial technology for automatic sign language production, starting with a focus on comprehension of the productions rather than the visual quality. We have utilised the state of the art research in deep learning and machine translation to develop a prototype that translates from German to DGS within a limited topic. The current production is in the form of a skeleton signer, with the body, hand and face joints matching the movements of a signer.

This is very much a work in progress and we know that a lot more work is required before the technology can be used by the end user. However, we believe it is important to get early feedback from the Deaf community on our sign language production technology, in order to appreciate how understandable our current productions are and how they can be improved. We believe the Deaf communities should be empowered and be involved in all steps of the development of any technology that is targeting their native languages. Hence, we created this survey to get your invaluable feedback and to guide the next steps of the research. Initially we are focusing on understandable sign language production and we would appreciate your opinions on the comprehension of sign rather than the look of the avatar. We greatly value your input into our technology and by completing this survey we will strive to take all opinions on board. Check the survey in the survey tab on our website
Update note: The survey is now closed, thank you for your participation!