Translation app provides a helping hand for sign language users
Across the EU, around one person in every 1 000 is a deaf sign language user. While this allows them to communicate with other sign language users, deaf people often find themselves marginalised in public settings unless an interpreter is present. The Accessible Translation (AT) (website in Turkish) project aims to provide a real-time translation service based on digital avatars. “Many competing applications developed in this field depend on equipment such as gloves, sensors, etc.,” says AT project coordinator Özer Çelik. “Accessible Translation works on all platforms without the need for any special hardware.” The application can run on any internet-connected device that has a camera, such as a smartphone or tablet.
Machine learning
Within the app, developed by Turkish firm Nevisoft IT Technology Industries, the movements of the sign language speaker are captured by the camera by artificial intelligence and converted into individual words. A natural language processing module is then used to translate these words into coherent phrases and read aloud or displayed as text. Sign language grammar structure is quite different from speech language grammar structure, according to Çelik. “There are no concepts such as prefixes or suffixes in sign language, and time concepts are expressed as words,” he explains. For this reason, the app also translates speech into sign language for deaf users, rather than simply transcribing it as text. “Since the vocabulary of sign language is very minimal compared to the language of speech, individuals with hearing disabilities have difficulty understanding text because they do not know as many written words,” says Çelik. “The best, permanent solution is sign language translation.” The signs in the translation are performed by a filmed actor, rather than an animation. “We realised that a real human model should be used to increase the comprehensibility of sign language,” notes Çelik, explaining that animations cannot adequately capture nuances in hand movements and facial expressions.
Going global
There are over 200 different signed languages in use around the world. After programming the most widely spoken of these, such as International Sign Language, more will be added to the AT service at the customers’ request. The system is aimed at businesses with deaf or hearing-impaired individuals in their service portfolio, such as banks, universities and hospitals. Çelik says EU-funding played an important role in preparing the company to enter the European market: “The feasibility research, determination of potential partnerships, improvement of the business model and research of sales channels were realised thanks to the funding received.” Çelik says the company is also developing new products, such as an application that can allow sign language users to converse over the telephone with people who do not sign. “Our main goal is to be a leading company that solves communication problems experienced by hearing-impaired individuals. In this sense, we are working to become global.”
Keywords
AT, deaf, hearing, sign, language, app, machine learning, artificial intelligence, animation, model, grammar, text
