Increasing independence and a voice for the profoundly disabled
For people with profound intellectual and multiple disabilities (PIMD), interaction with the environment is not so easy. In most cases, they are probably dependent on family and caregivers to carry out even basic functions due to severe physical and sensory impairments. And if their communication is not through conventional symbols and at the pre-verbal level, this makes life even more difficult.
Platform INSENSION: a prosthesis for communication
The EU-funded INSENSION project, a consortium of computer scientists, experts in special education and professionals in care provision for those with intellectual disabilities, has designed and developed an ICT platform to support people with PIMD. “We developed and evaluated mechanisms for detecting various behavioural signals of people with PIMD. These are then interpreted as decisions concerning situations in their surroundings,” outlines Michal Kosiedowski, project coordinator. The study is the first ever performed in actual PIMD care facilities, with participants who have this type of disability that threatens their opportunity for self-determination. “As a result, the lives of individuals with PIMD will be enriched with a wider range of opportunities, and their life quality increase accordingly.” Especially for carers working with a new child with complex problems, it’s a time-consuming process to learn the signals necessary to communicate. For example, in their home environment, this could take 3 months. The INSENSION platform can reduce this crucial period radically.
Assessment tools
To develop the system, the researchers devised two questionnaires. They enable assessment of how an individual with PIMD interacts and communicates with others. The long questionnaire gives a comprehensive view of the person and the expression of their inner feelings. Indications included an array of indicators of emotion – pre-verbal communication, challenging behaviours, ways of expressing pleasure and displeasure, pain and mood. “Due to the complexity of the information we collected, we had to narrow the scope of the indicators,” explains Kosiedowski. A new streamlined questionnaire gives data to be interpreted as the desire to change aspects of the person’s environment due to the lack of comfort. To give a wider view of the PIMD population, INSENSION researchers collected data in the Global PIMD Atlas. The idea was to potentially create sub-groups where all individuals share certain behavioural signals. “This is a possibility for a group born with the same genetic syndrome,” Kosiedowski points out.
‘Smart room’ and interaction personalisation
The INSENSION system uses AI and the IoT to provide a way of communication for the individual. For personalisation of the system, participants with disabilities are filmed, and the video footage and audio data are stored in a database for the AI to work on. “The camera looks at the person with a disability, then the AI components can identify this person and interpret their behaviours,” states Kosiedowski. “The hands, arms and the whole body are monitored for gestures. And points on the face, which are tracked for movement are an indication of specific facial expression.” The system then logs what it understands to be significant gestures and what they mean. “Project results are pretty promising,” concludes Kosiedowski. “Our work has been the first step on the way to tackling the enormous challenge of using advanced ICT to support people with PIMD in their interaction with their surroundings.” Work will continue to refine the technology to make it more reliable.
Keywords
INSENSION, PIMD, disability, communication, ICT, AI, IoT, Global PIMD Atlas, profound intellectual and multiple disabilities
