Helping the visually impaired access socially relevant information
Over 30 million partially sighted and blind people live in Europe. Traditional remedies such as glasses do not always solve problems with sight, impacting quality of life and hampering social and occupational needs. If visual impairment is present in childhood, this can present additional problems, for example if children cannot access socially relevant information. “A child’s inability to decipher socially relevant visual cues may affect their overall development and quality of life,” explains Lucia Schiatti, Marie Skłodowska-Curie experienced researcher at the Italian Institute of Technology. Monica Gori, coordinator of the Unit for Visually Impaired People group at the IIT adds:“It is important, in the context of visual and cognitive rehabilitation, to identify such a gap through quantitative measures and enable a timely intervention, before atypical behavioural patterns may emerge,” she adds. In the EU-funded TIRESIA project, Schiatti and her colleagues from the Unit for Visually Impaired People, directed by Monica Gori – in collaboration with researchers Andrei Barbu and Boris Katz from Massachusetts Institute of Technology – worked towards developing an innovative augmented reality (AR) system to extend residual sight among children. The team designed unique digital filters to automatically boost vision, and deliver social and visual information to users. “The goal was to assess whether sensory issues, such as low visual acuity and oculomotor disorders, as well as cognitive impairments, cause differences in visual attention patterns that we can detect via eye tracking,” says Schiatti.
Developing AR visual filters
The team based their designs of the AR filters on identifying differences in gaze patterns between sighted and visually impaired groups, including adults and children – with different types of visual impairments. Following visual tasks, the researchers used artificial neural networks to automatically enhance areas of images where sighted individuals tend to focus their attention. They also trained several deep learning models to perform visual tasks, such as emotion recognition and classification of social interactions. “The goal was to design a training protocol to guide visual social attention of children who show difficulties in catching socially relevant cues,” explains Schiatti.
A deeper look at the impacts of visual impairment in children
Through their research, the team found that visual impairments often affect the social orienting mechanism: the bias that humans have towards orienting their gaze (and attention) towards socially relevant information, such as facial expressions. “This suggests that visual impairments may lead to autistic-like social attention and behaviours,” says Schiatti. Preliminary data also indicated a lower capability of perceiving social interactions among children aged 7-10 than for older children and adults, suggesting that this might be a critical age period for the development of social skills. “Both results confirm the significance of TIRESIA’s research and the importance of developing quantitative tools for the assessment and training of social skills in the context of visual rehabilitation,” notes Schiatti.
Refining the approach through eye-tracking technologies
Following the successful completion of the project, the team hopes to refine their approach in collaboration with clinicians and users, to develop a protocol for training children with visual issues. “This will include the design of a specific application and user interface, potentially linked to a set of eye-tracking technologies existing on the market,” adds Schiatti. “Such an approach could be extended to other use cases where it is important to introduce quantitative metrics to evaluate and train social skills, for instance neurodevelopmental disorders such as autism.”
Keywords
TIRESIA, visual impairment, children, augmented reality, filters, autism, bias, social, information