Periodic Reporting for period 1 - PERSEO (European Training Network on PErsonalized Robotics as SErvice Oriented applications)
Période du rapport: 2021-01-01 au 2022-12-31
The ETN on PErsonalized Robotics as SErvice Oriented applications - PERSEO - aims to train a new generation of research and professional figures to face the research challenges of the market of personal robots. The personal robotics domain presents several research challenges mainly related to the need for a high degree of personalization of the robot behaviour with respect to the specific user’s needs and preferences. Companies need to build solutions rooted in a deeper analysis of human specificities before developing products for people. At the same time, academia needs to nurture the development of an extended research community with a set of interdisciplinary skills to investigate different robot’s capabilities for understanding and modelling the interaction with human beings, for adapting the robot’s behaviour to the context, and software integration mechanisms that allow an easy personalized configuration approach to limit the static and costly customization processes of a novel robotic system.
The development of personalizable robots is accelerating. However, only a small amount of such robots is deployed in society, tackling very specific tasks in well-controlled environments. In PERSEO three scenarios will serve as integration milestones to assess the ESRs research progression and to evaluate the benefit of the service-oriented approach for the integration of the research projects: Personal Robots as Companions for Elderly People, Personal Robots for Physical Rehabilitation and Assistance, and Personal Robots for Edutainment.
The PERSEO’s research program is organized into three Research Themes, corresponding to three different WPs, aimed at investigating the personalization of robot capabilities at different levels of possible human-robot interaction, namely “Physical”, “Cognitive”, and “Social”. This requires a set of research skills ranging from computer science and AI to automation, ethics, and psychology. Fellows will deepen their knowledge in projects covering a specific research theme but, through training events, secondments, and peer collaborations will gain extensive knowledge in other areas so leading to a multidisciplinary research programme and the integration with other projects to achieve the proposed Integration Milestones. The project also aims to train Early Stage Researchers to address social, legal, and ethical issues arising from personal robots' uptake, as such skills are fundamental to achieving technological innovation that aligns with European social, ethical, and legal values.
The development of personalizable robots is accelerating. However, only a small amount of such robots is deployed in society, tackling very specific tasks in well-controlled environments. In PERSEO three scenarios will serve as integration milestones to assess the ESRs research progression and to evaluate the benefit of the service-oriented approach for the integration of the research projects: Personal Robots as Companions for Elderly People, Personal Robots for Physical Rehabilitation and Assistance, and Personal Robots for Edutainment.
The PERSEO’s research program is organized into three Research Themes, corresponding to three different WPs, aimed at investigating the personalization of robot capabilities at different levels of possible human-robot interaction, namely “Physical”, “Cognitive”, and “Social”. This requires a set of research skills ranging from computer science and AI to automation, ethics, and psychology. Fellows will deepen their knowledge in projects covering a specific research theme but, through training events, secondments, and peer collaborations will gain extensive knowledge in other areas so leading to a multidisciplinary research programme and the integration with other projects to achieve the proposed Integration Milestones. The project also aims to train Early Stage Researchers to address social, legal, and ethical issues arising from personal robots' uptake, as such skills are fundamental to achieving technological innovation that aligns with European social, ethical, and legal values.
15 Early-Stage Researchers (ESR) were recruited. The ESRs participated in several cross-sectorial environments within and outside the consortium in Industrial, Academic and public engagement events. During the first PERSEO residential workshops, they lectured on technical aspects related to personalization as well as on law, psychology, and management. A winter school on Interdisciplinary Research Methods was organised to provide an extensive understanding of the ethical aspects of personalized human-robot interaction and
a Summer School on Service-Based and Cloud Robotics was focused on service-based and cloud-robotics technologies for supporting the personalisation in human-robot interaction.
ESRs were also involved in the organisation of an international conference. The event was planned to enrich the training activities of the ESR with transferable skills related to teamwork and job management, and the opportunity to be involved in the organization of a large scientific event. ESRs also planned and updated every 6 months, their Personal Career Development Plan, and they planned the secondments within the consortium's partners, which will start on the second reporting period, i.e. from January 2023.
a Summer School on Service-Based and Cloud Robotics was focused on service-based and cloud-robotics technologies for supporting the personalisation in human-robot interaction.
ESRs were also involved in the organisation of an international conference. The event was planned to enrich the training activities of the ESR with transferable skills related to teamwork and job management, and the opportunity to be involved in the organization of a large scientific event. ESRs also planned and updated every 6 months, their Personal Career Development Plan, and they planned the secondments within the consortium's partners, which will start on the second reporting period, i.e. from January 2023.
To create personalisation and adaptation in robots, the technical work in the project has been divided into 3 main Work Packages (WPs), aimed at defining the role of personalization at different levels of possible HRI, namely “Physical”, “Cognitive”, and “Social:
WP3 - Personalization in Physical Interaction is focused on providing personalization in tasks that involve physical contact between the robot and people, and physical contact of the robot with the surrounding environment. Based on that, we conducted a detailed analysis of the SLAM techniques for sensing a robot's surroundings, we identified the semantic information that a map offers to allow robots to move in dynamic cluttered environments, and an initial paradigm has been investigated for improving the legibility, predictability and transparency of robots' movements in human-centred environments. An initial framework, including a graphical interface for non-expert users, has also been developed to provide relevant information for a kinesthetic teaching process. The work performed includes also techniques for 3D body pose estimations.
WP4 - Personalization in Cognitive Interaction aims at endowing robots with high-level meta-cognition capabilities. An initial component has been developed that extracts low- and high-level human information over the observed human actions in an RGB video, and generates multiple stable predictions of the next actions that the observed human might perform. Then, a Dynamic Bayesian Network that uses Bayesian inverse planning to represent how people infer other’s goals or preferences and to allow robots to behave in an environment based on their beliefs and preferences about the world via the principle of rational belief, and a spiking neural network model on open-source datasets have been developed. The work also includes a first model to generate synthetic instructions of the path that the robot has traversed using a GAN, and a component for learning behaviours from a forward reinforcement model with human feedback.
WP5 - Personalization in Social Interaction is in charge of enabling robots to recognise and interpret the social cues displayed by humans to know when, and how to adapt their behaviours to the individuals' preferences and states. Based on this, a detailed investigation brought to the development of a task-incremental continual learning approach of facial expressions, and an initial model has been developed for assessing engagement in real-time face-to-face human-robot social interactions. This WP has also identified anthropomorphism, manipulation and persuasion, and social imaginary as the main mediating mechanisms between personalisation and resulting attitudes and trust towards robots.
In order to ensure the wide reach of PERSEO, the Consortium's strategy for dissemination and exploitation of the project's results has been actively carried out to reach different target groups. The consortium engaged scientific communities by participating in 31 international conferences and 27 workshops and organising 8 international conferences and 3 workshops. The ESRs also count 12 scientific peer-reviewed publications by December 2022. The general audience has been successfully reached by engaging them in active communication via different channels. For supporting short but clear visual and text communication, the social media of the project (Twitter, Instagram, Facebook, YouTube and website) has been used. The consortium also engaged the general public during national and international fairs, events, and press releases via radio/tv and video (i.e. a total of 93 events).
WP3 - Personalization in Physical Interaction is focused on providing personalization in tasks that involve physical contact between the robot and people, and physical contact of the robot with the surrounding environment. Based on that, we conducted a detailed analysis of the SLAM techniques for sensing a robot's surroundings, we identified the semantic information that a map offers to allow robots to move in dynamic cluttered environments, and an initial paradigm has been investigated for improving the legibility, predictability and transparency of robots' movements in human-centred environments. An initial framework, including a graphical interface for non-expert users, has also been developed to provide relevant information for a kinesthetic teaching process. The work performed includes also techniques for 3D body pose estimations.
WP4 - Personalization in Cognitive Interaction aims at endowing robots with high-level meta-cognition capabilities. An initial component has been developed that extracts low- and high-level human information over the observed human actions in an RGB video, and generates multiple stable predictions of the next actions that the observed human might perform. Then, a Dynamic Bayesian Network that uses Bayesian inverse planning to represent how people infer other’s goals or preferences and to allow robots to behave in an environment based on their beliefs and preferences about the world via the principle of rational belief, and a spiking neural network model on open-source datasets have been developed. The work also includes a first model to generate synthetic instructions of the path that the robot has traversed using a GAN, and a component for learning behaviours from a forward reinforcement model with human feedback.
WP5 - Personalization in Social Interaction is in charge of enabling robots to recognise and interpret the social cues displayed by humans to know when, and how to adapt their behaviours to the individuals' preferences and states. Based on this, a detailed investigation brought to the development of a task-incremental continual learning approach of facial expressions, and an initial model has been developed for assessing engagement in real-time face-to-face human-robot social interactions. This WP has also identified anthropomorphism, manipulation and persuasion, and social imaginary as the main mediating mechanisms between personalisation and resulting attitudes and trust towards robots.
In order to ensure the wide reach of PERSEO, the Consortium's strategy for dissemination and exploitation of the project's results has been actively carried out to reach different target groups. The consortium engaged scientific communities by participating in 31 international conferences and 27 workshops and organising 8 international conferences and 3 workshops. The ESRs also count 12 scientific peer-reviewed publications by December 2022. The general audience has been successfully reached by engaging them in active communication via different channels. For supporting short but clear visual and text communication, the social media of the project (Twitter, Instagram, Facebook, YouTube and website) has been used. The consortium also engaged the general public during national and international fairs, events, and press releases via radio/tv and video (i.e. a total of 93 events).