Dublin Nov 30th 2021: The SignON project has received Horizon 2020 funding of €5.6M. SignON is a 3-year project led by ADAPT at DCU to address the communication gap between users of spoken languages and deaf sign language users.
The SignON EU consortium of 17 European partners led by Prof. Andy Way, Professor of Computing at Dublin City University, Ireland (coordinator), and Dr. Dimitar Shterionov Assistant Professor in Cognitive Science and Artificial Intelligence at Tilburg University, The Netherlands (scientific lead), have received this award to conduct state of the art research and develop a mobile solution for automatic translation between sign and oral (written and spoken) languages.
SignON is a user-centric and community-driven project that aims to facilitate the exchange of information among deaf and hard of hearing, and hearing individuals across Europe targeting the Irish, British, Dutch, Flemish and Spanish Sign, and English, Irish, Dutch, Spanish oral languages.
Through collaboration with these European deaf and hard of hearing communities, researchers will define use-cases, co-design, and co-develop the SignON service and application. The objective of the research project is the fair, unbiased, and inclusive spread of information and digital content in European society.
The SignON communication service will be more than an advanced machine translation system. Behind the scenes, SignON will incorporate sophisticated machine learning capabilities that will allow (i) learning new Sign and oral languages; (ii) style-, domain- and user-adaptation and (iii) automatic error correction, based on user feedback. To the user, SignON will deliver signed conversations via a life-like avatar built with the latest graphics technologies.
To ensure wide uptake, improved sign language detection and synthesis, as well as multilingual speech processing for everyone, the project will deploy the SignON service as a smartphone application running on standard modern devices. While the application is designed as a light-weight interface, the SignON framework will be distributed on the cloud where the computationally intensive tasks will be executed.
Speaking about the project Prof Andy Way says:
“When I first worked on sign language MT 15 years ago, the field was very small. In 2022, we will see a special issue of the Machine Translation journal appearing dedicated to this topic. Now that ISL is a fully-fledged official language in Ireland, it is great to see this work continuing to thrive. I am pleased to coordinate the SignON project, which will develop a free, open application and framework for conversion between video (capturing and understanding Sign language), audio and text and translation between Sign and spoken languages.”
The SignON project will commence on 01/01/2021 and the consortium is currently recruiting a wide range of experts in the fields of Natural Language Processing (NLP) Machine Learning (ML), Deep Learning (DL), Machine Translation (MT) Linguistics, Deaf studies, education, 3D graphics, and others to join the SignON team.
For more details, please contact D.Shterionov@tilburguniversity.edu
Partners: Dublin City University (DCU) – coordinator, Ireland; Fincons Group (FINC), Switzerland; Instituut voor de Nederlandse Taal (INT), The Netherlands; University of the Basque Country (UPV/EHU), Spain; The National Microelectronics Applications Centre Ltd (MAC), Ireland; Pompeu Fabra University (UPF), Spain; Technological University Dublin (TU Dublin), Ireland; Trinity College Dublin (TCD), Ireland; VRT, Belgium; Ghent University (UGent), Belgium; Vlaams GebarentaalCentrum (Flemish Sign Language Centre – VGTC), Belgium, University College Dublin (UCD), Ireland; Stichting Katholieke Universiteit (RU), The Netherlands; Nederlandse TaalUnie (NTU), The Netherlands; Katholieke Universiteit Leuven (KU Leuven), Belgium; European Union of the Deaf (EUD), Belgium; Tilburg University (TiU) – scientific lead, The Netherlands.
Share this article:
Interesting read in @RTEBrainstorm by @AdaptCentre Dr. Mani Dhingra, Digital Twin Ecosystem Manager @MaynoothUni @smartdublin @AphraK @scienceirel