Humanoid robots with social skills could redefine elderly care
Over the last decade social robots have been deployed in various public spaces like museums, airports, banks and hospitals. Beyond basic navigation and object manipulation, these robots need to communicate naturally with people. While there are already forms of artificial intelligence tools interacting with humans, they often lack the sophistication for complex social tasks that do not require human oversight. “For instance, joining a discussion group or escorting someone through a populated building requires a robot to accurately perceive and interpret signals from multiple humans (amongst speakers) to respond appropriately. This demands advanced perception and learning capabilities for autonomous decision-making,” notes Xavier Alameda-Pineda, coordinator of the EU-funded SPRING project. SPRING successfully unveiled social robots capable of effectively interacting within complex and unpredictable public spaces. The robots were tested in the Broca gerontology day-care hospital. “Patients, often accompanied by friends or family, spend long hours at the hospital, waiting for results or scheduling appointments. Our goal was to offer a system that not only provides practical information but also eases the stress of their day,” outlines Alameda-Pineda. With testing on over 90 patients and companions, along with 30 medical staff, SPRING saw continuous improvements in usability and acceptability scores through its experimental waves. “Usability and evaluation were conducted in a day-care hospital using a full-sized humanoid robot, which differs from the usual settings of nursing facilities and the use of smaller, pet-like robots,” adds Alameda-Pineda. Researchers developed innovative software architecture for the newly developed humanoid robot, which featured five modules for perception (self-localisation, human localisation, speech processing, human behaviour analysis, person manager) and three for action (experimenter interface, multi-party conversation, non-verbal behaviour generation) processes.
More natural interactions
Social robotic platforms struggle with natural and smooth interactions because it is challenging to create a system that reliably understands social situations. This system needs to be adaptable across various scenarios and evolve with changes in social interactions. To tackle these issues, researchers used the ROS for human-robot interaction (ROS4HRI) standard that allowed robots to manage information about people, like their speech, facial expressions and body movements and position. This made it easier for robots to adapt and respond in different social settings.
Engaging in multi-modal dialogue
Humans communicate using a mix of different signals, such as gestures, speech, nodding, and even silence, to express themselves and understand others. While hitherto research has explored these cues individually, combining them effectively on a robot is still challenging. “SPRING made progress on this by helping robots use sound and sight to identify people and understand feelings in spoken words. Work also focused on enabling robots to use gestures that match what they are saying, making interactions more natural,” states Alameda-Pineda.
Boosting content control
Large language models and multi-modal generative models could enable open-world conversations and interactions. However, they often operate like a ‘black box’, making them difficult to control. This is critical especially in sensitive areas like healthcare or when interacting with children, as well as in handling issues like racist or sexist remarks. “SPRING addressed these challenges by using prompts to guide the models, but this method does not always guarantee satisfactory results. Understanding the context is crucial, as it helps define the boundaries of acceptable responses,” notes Alameda-Pineda.
Improving personalisation and adaptation
Another challenge for social robots is developing the ability to adapt and personalise their interactions with their surroundings autonomously. This implies they would not need specific data for each customer, environment or task when used in new settings. SPRING explored ways to achieve this, working on the technology needed to make these advancements possible.
Keywords
SPRING, humanoid robot, social robot, perception, day-care hospital, elderly care
