Periodic Reporting for period 3 - RAMCIP (Robotic Assistant for MCI patients at home)
Okres sprawozdawczy: 2017-01-01 do 2018-06-30
The RAMCIP project develops a novel robot that can provide proactive and discreet assistance to elderly people with Mild Cognitive Impairment (MCI) and dementia in early stages in their own home, to support their independent living and quality of life. The RAMCIP robot is aimed to provide assistance to the user in daily activities related to nutrition, medication, usage of electric appliances, socialization, as well as safe domestic locomotion, through user activities monitoring and robot interventions to facilitate the user in corresponding tasks and help counteract for observed abnormalities.
To this end, the RAMCIP robot comprises three major components: (a) cognitive functions based on advanced user and home environment modelling and monitoring, allowing the robot to decide when and how to assist the user, (b) adaptive multimodal human robot communication interfaces, and (c) advanced, dextrous and safe robotic manipulation capabilities, enabling grasping and manipulation of a variety of home objects, as well as safe physical HRI.
During the last project period, the final integrated version of the RAMCIP robot (second version – V2) has been developed. The RAMCIP robot has been successfully applied in the pilot trials of the project, which have been held in two pilot sites, in Lublin, Poland and in Barcelona, Spain. In Lublin, the robot operated within a simulated apartment, where 20 end users experienced HRI in the target use cases. In Barcelona, the robot operated in 12 different real apartments, i.e. the real homes of 12 end users, staying at each home for more than a week. The RAMCIP robot successfully established all of the target use cases in both pilot sites.
For more information on the RAMCIP project, please visit: www.ramcip-project.eu
To this end, the RAMCIP robot comprises three major components: (a) cognitive functions based on advanced user and home environment modelling and monitoring, allowing the robot to decide when and how to assist the user, (b) adaptive multimodal human robot communication interfaces, and (c) advanced, dextrous and safe robotic manipulation capabilities, enabling grasping and manipulation of a variety of home objects, as well as safe physical HRI.
During the last project period, the final integrated version of the RAMCIP robot (second version – V2) has been developed. The RAMCIP robot has been successfully applied in the pilot trials of the project, which have been held in two pilot sites, in Lublin, Poland and in Barcelona, Spain. In Lublin, the robot operated within a simulated apartment, where 20 end users experienced HRI in the target use cases. In Barcelona, the robot operated in 12 different real apartments, i.e. the real homes of 12 end users, staying at each home for more than a week. The RAMCIP robot successfully established all of the target use cases in both pilot sites.
For more information on the RAMCIP project, please visit: www.ramcip-project.eu
The main results achieved by the project can be summarized as follows:
- A hierarchical framework for semantic modelling of the end user’s home, along with metric mapping, has been developed. Methods for reconstruction, recognition and state tracking of domestic objects, both large environment objects and small household ones have been developed.
- Methods for human detection, person identification and pose tracking have been developed, as well as for action recognition and monitoring of complex activities, aiming to assess user behaviour and identify abnormalities relevant to the target use cases.
- The high-level robot behaviour policies in the scope of the project’s use cases have been defined and a POMDP-based approach for the RAMCIP robot cognitive functions was developed.
- Touch-screen, speech and gestures modalities, the robot’s AR display and the methods for robot affective input and output have been implemented. The Communication Decision Maker (CDM) module, orchestrating the UIs was developed.
- Methods enabling safe, human-aware navigation of the RAMCIP platform, as well as the safe reaching and grasping of target objects have been researched and developed.
- Control schemes for physical HRI, with respect to ensuring the user safety, were developed. Unintentional contact controllers were also established, targeted at reducing the interaction force via increasing the robotic system compliance.
- New H/W components have been developed for the RAMCIP robot, while developing the second robot version (V2), including a new mobile platform, smaller and more appealing than the initial one, more complex user interface with tablet PC, loudspeakers, microphone and light sensor integrated into one part as well as further elaborated robot arm and hand modules.
- The final (V2) integrated RAMCIP robot has been developed.
- The RAMCIP robot has been evaluated at the two project pilot sites; in Barcelona, the robot was evaluated with 12 different end users, at their real homes.
- Several horizontal activities for diffusing project objectives, concepts and achievements to key stakeholders, and the general public have taken place, including RAMCIP robot demonstration during 2018 in major exhibitions, including Hannover Messe and Automatica.
- The RAMCIP research efforts have led to scientific publications in highly-ranked scientific conferences and journals, as well as to a series of open datasets.
- A hierarchical framework for semantic modelling of the end user’s home, along with metric mapping, has been developed. Methods for reconstruction, recognition and state tracking of domestic objects, both large environment objects and small household ones have been developed.
- Methods for human detection, person identification and pose tracking have been developed, as well as for action recognition and monitoring of complex activities, aiming to assess user behaviour and identify abnormalities relevant to the target use cases.
- The high-level robot behaviour policies in the scope of the project’s use cases have been defined and a POMDP-based approach for the RAMCIP robot cognitive functions was developed.
- Touch-screen, speech and gestures modalities, the robot’s AR display and the methods for robot affective input and output have been implemented. The Communication Decision Maker (CDM) module, orchestrating the UIs was developed.
- Methods enabling safe, human-aware navigation of the RAMCIP platform, as well as the safe reaching and grasping of target objects have been researched and developed.
- Control schemes for physical HRI, with respect to ensuring the user safety, were developed. Unintentional contact controllers were also established, targeted at reducing the interaction force via increasing the robotic system compliance.
- New H/W components have been developed for the RAMCIP robot, while developing the second robot version (V2), including a new mobile platform, smaller and more appealing than the initial one, more complex user interface with tablet PC, loudspeakers, microphone and light sensor integrated into one part as well as further elaborated robot arm and hand modules.
- The final (V2) integrated RAMCIP robot has been developed.
- The RAMCIP robot has been evaluated at the two project pilot sites; in Barcelona, the robot was evaluated with 12 different end users, at their real homes.
- Several horizontal activities for diffusing project objectives, concepts and achievements to key stakeholders, and the general public have taken place, including RAMCIP robot demonstration during 2018 in major exhibitions, including Hannover Messe and Automatica.
- The RAMCIP research efforts have led to scientific publications in highly-ranked scientific conferences and journals, as well as to a series of open datasets.
The key innovation points of the project can be summarized as follows:
- Novel methods for modelling and monitoring of the user’s domestic environment and activities, along with cognitive functions that enable on that basis, the provision of proactive and discreet assistance interventions in the scope of the RAMCIP target use cases.
- An adaptive multi-modal human robot communication approach, fusing speech, touch-screen, gestures, an AR display and empathic communication channels.
- Advanced methods for safe, human-aware robot navigation and manipulator reaching motion with human-like characteristics, novel grasping methods exploiting environmental contacts; a novel approach for slippage detection during object transfer; advanced control schemes are investigated for object- hand-over and to ensure safety under both intentional and unintentional contacts.
- Multi-fingered adaptive gripper with intelligence and compliance. “Smart Grasping” (trademark secured) allows one tool to handle a wide range of interactions without needing complex programming. Potential applications are in Mobile service robots, flexible automation, logistics.
- The hardware developed within RAMCIP, with iterations through external funding now progressing as both a stand-alone component for use by robot systems integrators and as part of larger systems developed by Shadow for specific customers.
- The integrated RAMCIP robot prototype (2nd version) has been built; it includes a mobile platform with elevation mechanism and head including facial expressions display and an AR display, manipulator arm and novel hand.
- The RAMCIP robot has been evaluated in the real homes of 12 different end users in Barcelona (Spain) and in a simulated apartment in Lublin (Poland) with 20 end users.
- Evaluation focused on robot acceptability, usability and societal impact. Furthermore, in ACE additional studies have been conducted regarding quality of life and facial emotion recognition possible modification through HRI, as well as the possible influence of anosognosia in the acceptance of a service robot.
The RAMCIP robot has proved during the pilot trials its capability of assisting MCI users in a series of daily activities, being assessed by end users as an overall usable and acceptable robotic system. By assisting into the establishment of nutritional and medication activities, as well as by facilitating the user in avoiding risky situations such as falls, or also, through early detection and intervention in case of emergencies related to the user’s health state or the state of the environment, two significant impact dimensions on the primary RAMCIP user are denoted. The first relates to the suppression of the possibility for risky situations to emerge, which can lead into hospitalization. The second concerns the increased capability of the user to perform typical daily activities, without the need for constant supervision of a caregiver. The RAMCIP impacts diffuse also to the primary user’s human caregiver; by being capable to undertake caregiving tasks for some periods of time, the RAMCIP robot can relieve accordingly the caregiver’s burden, who becomes capable to spend more time for her/himself.
- Novel methods for modelling and monitoring of the user’s domestic environment and activities, along with cognitive functions that enable on that basis, the provision of proactive and discreet assistance interventions in the scope of the RAMCIP target use cases.
- An adaptive multi-modal human robot communication approach, fusing speech, touch-screen, gestures, an AR display and empathic communication channels.
- Advanced methods for safe, human-aware robot navigation and manipulator reaching motion with human-like characteristics, novel grasping methods exploiting environmental contacts; a novel approach for slippage detection during object transfer; advanced control schemes are investigated for object- hand-over and to ensure safety under both intentional and unintentional contacts.
- Multi-fingered adaptive gripper with intelligence and compliance. “Smart Grasping” (trademark secured) allows one tool to handle a wide range of interactions without needing complex programming. Potential applications are in Mobile service robots, flexible automation, logistics.
- The hardware developed within RAMCIP, with iterations through external funding now progressing as both a stand-alone component for use by robot systems integrators and as part of larger systems developed by Shadow for specific customers.
- The integrated RAMCIP robot prototype (2nd version) has been built; it includes a mobile platform with elevation mechanism and head including facial expressions display and an AR display, manipulator arm and novel hand.
- The RAMCIP robot has been evaluated in the real homes of 12 different end users in Barcelona (Spain) and in a simulated apartment in Lublin (Poland) with 20 end users.
- Evaluation focused on robot acceptability, usability and societal impact. Furthermore, in ACE additional studies have been conducted regarding quality of life and facial emotion recognition possible modification through HRI, as well as the possible influence of anosognosia in the acceptance of a service robot.
The RAMCIP robot has proved during the pilot trials its capability of assisting MCI users in a series of daily activities, being assessed by end users as an overall usable and acceptable robotic system. By assisting into the establishment of nutritional and medication activities, as well as by facilitating the user in avoiding risky situations such as falls, or also, through early detection and intervention in case of emergencies related to the user’s health state or the state of the environment, two significant impact dimensions on the primary RAMCIP user are denoted. The first relates to the suppression of the possibility for risky situations to emerge, which can lead into hospitalization. The second concerns the increased capability of the user to perform typical daily activities, without the need for constant supervision of a caregiver. The RAMCIP impacts diffuse also to the primary user’s human caregiver; by being capable to undertake caregiving tasks for some periods of time, the RAMCIP robot can relieve accordingly the caregiver’s burden, who becomes capable to spend more time for her/himself.