Periodic Reporting for period 1 - SignON (SignON - Sign Language Translation Mobile Application and Open Communications Framework)
Berichtszeitraum: 2021-01-01 bis 2022-06-30
SignON is a Horizon 2020 project which aims to develop a communication service that translates between signed and spoken (in both text and audio modalities) languages and caters for the communication needs between deaf, hard of hearing and hearing individuals.
Reducing this communication gap is the overall objective of this project. To achieve this objective SignON aims to provide a communication service delivered through a mobile application, which has been co-designed and co-developed with the relevant user communities. This co-creation approach, rigidly followed by the SignON partners, where potential deaf, hard of hearing and hearing users are involved in the research, design and development activities makes SignON a unique endeavour with strong social and societal impact.
To ensure that all activities within the consortium as well as towards the community follow correct ethical principles and respect personal privacy and legislations, the SignON Research Ethics Committee (REC) provides guidance for all of these activities and is on hand to offer support when needed.
To date, the SignON project has successfully developed working versions of the SignON App and Framework according to user requirements and expectations, established a repository for the code-base which will be openly accessible after the end of the project and, which, acts as a platform where development can take place in parallel. The SignON Framework has been developed through a distributed and highly scalable approach which allows for components to operate in parallel (to accommodate multiple users without overhead) as well as to be redesigned, improved and replaced seamlessly. Software components for ASR, SLR, MT, NLP and synthesis have been developed and integrated in the framework. State of the art models to facilitate the translation pipeline have been built and released.
Through continuous communication and dissemination activities, we keep the main stakeholders and the general public informed about our progress.
The project will continue over the next 18 months. During this time, we will endeavour to complete all tasks that were specified in the Grant Agreement, resulting in the SignOn communication service that will be delivered through a mobile application, with the inclusion of the 5 sign languages and 4 spoken languages that were to be addressed in our project, supported by the final version of the 3D virtual character and synthesis component.
The expectation is that the project will continue to be publicised and spoken about via continuous communication and dissemination activities which will lead to a continuously growing community, even beyond the lifespan of SignON. Appropriate use-cases and usage domains will be identified and investigated and will feed into the SignON Sustainable Exploitation, Innovation and IPR plan. All of these expected results will be subject to continuous ethical and privacy monitoring.
In addition to this, and of more fundamental importance, we hope that what we will have achieved will be a step towards bridging the communication gap between deaf, hard of hearing and hearing individuals. We would hope that the co-creation process in our project would instill confidence in the project and its results, to allow the respective communities to communicate more easily and more freely. We would also like to see progress towards the establishment of a repository of sign language data that would allow further research to be performed in this domain.
The potential impacts for this project are huge. Besides the impact on scientific research in terms of technical advancements in multilingual speech processing and sign language recognition on mobile devices, the societal impact of having access to an easily accessible communication service in lieu of interpreters would create a huge benefit to society. The SignON application is not presented as a tool to replace interpreters and current means of communication but as an option available to the general public if it is required and appropriate. There has also been a noticeable impact on the various partners in the consortium, not least, an improved awareness of the interests, needs and requirements of DHH people as well as deaf culture and communities.