Skip to main content
European Commission logo
English English
CORDIS - EU research results
CORDIS
CORDIS Web 30th anniversary CORDIS Web 30th anniversary

The first mental joystick based on Artificial Intelligence for Multi-Modal Human Machine Interaction

Article Category

Article available in the following languages:

Helping locked-in patients communicate with the outside world again

There is currently no cure for having a locked-in state, but new mind-controlled tools let patients communicate with computers.

Over 5 million people in the world are suffering from paralysis and communication limits, and thousands of them are either in a locked-in state or complete locked-in state: they are conscious and retain cognitive abilities, but are unable to speak, move or make facial expressions. There is currently no cure. “Physiotherapy, continuous treatment and exercises can slow down the worsening of the conditions, but from a mere medical point of view there is no chance to recover from it,” explains Alessandro Tozzo, business and strategic consultant at LiquidWeb and BrainControl project coordinator. In the EU-funded BrainControl project, researchers developed a suite of brain-controlled technologies to allow patients to use computers and gain back some control. “BrainControl’s primary goal is to give these people a real chance to communicate with the outside world again,” Tozzo says.

Linking the mind with computers

The BrainControl project developed a range of medical devices specifically conceived to help people hit by serious disabilities to regain self-confidence and communicate with the outside world again. The BrainControl BCI system consists of an EEG headset to measure brain activity, twinned with software that allows a patient to communicate with an electronic device such as a tablet or computer. BrainControl Sensory is designed to help patients with residual voluntary movements of the body. Boasting a variety of sensors such as an eye tracker, motion sensors and mouse emulators, the device uses these movements to stimulate interaction with the outside world. A third solution, SMART, allows doctors and caregivers to create a customised clinical picture for each individual patient. This software is a user-friendly platform with patient assessment tests and rehabilitation exercises, aimed at enhancing and preserving the patient’s abilities. “BrainControl SMART can be used by any type of patients through the most common interaction methods such as mouse, keyboard and mouse emulator but also with more advanced methods such as eye pointer and brain-computer interface,” Tozzo adds.

Internal and initial tests

During the BrainControl project, the team tested the systems internally to improve their technology. While there have been no clinical trials so far, several studies were carried out with Goldsmiths, University of London. The team is currently performing an accuracy study on healthy volunteers to prove the accuracy of their system. “So far, we have tested 25 people, and the accuracy after the first training session is 72 %, while after the fifth and last one it is at 90 %, which is an excellent result,” says Tozzo.

Opening up the world for those with a locked-in state

“The grant we received from the EU allowed us to finalise our solutions, test them and release them onto the market,” says Tozzo. The team is now working to develop more sophisticated solutions in the diagnostic environment, increasing the library of tests and introducing more biofeedback mechanisms that could give more objective data to the clinician to define the most accurate diagnosis and rehabilitation path for their patient. The technology could help bring communication back to millions of people who suffer from these debilitating diseases. “Being able to answer a few easy questions represents an insurmountable obstacle for them,” Tozzo remarks. “Getting back in touch with family, friends or doctors is a revolutionary achievement.”

Keywords

BrainControl, locked-in state, diagnosis, communication, headset, mind, control, computer

Discover other articles in the same domain of application