Periodic Reporting for period 3 - CoLLaboratE (Co-production CeLL performing Human-Robot Collaborative AssEmbly)
Reporting period: 2021-10-01 to 2022-03-31
The CoLLaboratE is a 42-month project that aims to enable genuine human–robot collaboration for performing assembly tasks in a production cell that is designed to provide a safe collaborative environment. The project builds upon state-of-the-art methods for teaching the robot assembly tasks using different means of human demonstration, equips the robots and AGV mobile platforms with basic collaboration skills and provides efficient safety strategies for a fenceless approach within the production cell. As a result, closer collaboration will be achievable and efficient production plans making optimal use of the available resources can be designed and executed.
For CoLLaboratE to successfully realize its vision, a few scientific and technological objectives have been set throughout the project duration. These are listed in the following points:
1) To equip the robotic agents with basic collaboration skills easily adaptable to specific tasks
2) To develop a framework that enables non-experts teaching human-robot collaborative tasks from demonstration
3) The development of technologies that will enable autonomous assembly policy learning and policy improvement
4) To develop advanced safety strategies allowing effective human robot cooperation with no barriers and ergonomic performance monitoring
5) To develop techniques for controlling the production line while making optimal use of the resources by generating efficient production plans, employing reconfigurable hardware design, and utilising AGV’s with increased autonomy
6) To investigate the impact of Human-Robot Collaboration to the workers’ job satisfaction, as well as test easily applicable interventions in order to increase trust, satisfaction and performance
7) To validate CoLLaboratE system’s ability to facilitate genuine collaboration between robots and humans
For CoLLaboratE to successfully realize its vision, a few scientific and technological objectives have been set throughout the project duration. These are listed in the following points:
1) To equip the robotic agents with basic collaboration skills easily adaptable to specific tasks
2) To develop a framework that enables non-experts teaching human-robot collaborative tasks from demonstration
3) The development of technologies that will enable autonomous assembly policy learning and policy improvement
4) To develop advanced safety strategies allowing effective human robot cooperation with no barriers and ergonomic performance monitoring
5) To develop techniques for controlling the production line while making optimal use of the resources by generating efficient production plans, employing reconfigurable hardware design, and utilising AGV’s with increased autonomy
6) To investigate the impact of Human-Robot Collaboration to the workers’ job satisfaction, as well as test easily applicable interventions in order to increase trust, satisfaction and performance
7) To validate CoLLaboratE system’s ability to facilitate genuine collaboration between robots and humans
During the first 18 months of the project the consortium has made significant progress towards the main objectives:
- The use-cases were defined from a functional, layout and architectural point of view, to extract properly the main requirements.
- A questionnaire for the evaluation of the social aspects that affect the perception of the job quality and job satisfaction was prepared.
- Different modalities to teach a robot assembly tasks by demonstration were developed, including visual cues, kinesthetic teaching and mobile app interfaces.
- Novel collaborative skills were developed for efficient and safe interaction between the human and the robot during collaborative task execution, including novel methods for adaptive control of robotic manipulators during object handling, load sharing, compliant control schemes with dynamic obstacle avoidance and safety through novel designs of tactile sensors. Collaboration with AGVs is also achieved by recognizing gestures of the worker with novel skeleton tracking algorithms and translating them into actions to command the mobile robot.
- Methods were developed for efficient collaboration between human workers, production supervisors, robot manipulators and AGVs, including user-friendly graphical interfaces, novel human skeleton tracking algorithms for obstacle avoidance with robot manipulators and AGVs, ergonomic monitoring of the human and high level task planners for optimal resource allocation.
In the next 18 months of CoLLaboratE, the consortium continued their efforts in accordance to the work-plan to finalize all the modules needed for collaborative assembly and to start the integration efforts for the deployment of the four use-cases. The achievements include:
- Amendments and contingency measures to handle the delays of the COVID pandemic, while turning all meeting to virtual ones (WP1)
- Finalization of the user-requirements and system architecture, while defining the use-cases to facilitate integration (WP2)
- Finalization of the basic collaboration skills for efficient, adaptive and safe interaction (WP3),
- Finalization of the different modalities to teach assembly tasks by demonstration to a robot (WP4)
- Finalization of the high-level solutions for monitoring ergonomics, interfacing with the robot manipulators and the AGVs and overall task planning (WP5)
In the last 6 months, the consortium continued their efforts in accordance with the work-plan to integrate all the modules of the completed WPs (WP2-WP5) and deploy the four use-cases.
- All 4 use-cases were completed and validated
- The job quality questionnaire was evaluated in all use-cases
- The final deliverables were completed, including the consolidated evaluation and lessons learned
- The exploitation plan and market analysis were completed
- The use-cases were defined from a functional, layout and architectural point of view, to extract properly the main requirements.
- A questionnaire for the evaluation of the social aspects that affect the perception of the job quality and job satisfaction was prepared.
- Different modalities to teach a robot assembly tasks by demonstration were developed, including visual cues, kinesthetic teaching and mobile app interfaces.
- Novel collaborative skills were developed for efficient and safe interaction between the human and the robot during collaborative task execution, including novel methods for adaptive control of robotic manipulators during object handling, load sharing, compliant control schemes with dynamic obstacle avoidance and safety through novel designs of tactile sensors. Collaboration with AGVs is also achieved by recognizing gestures of the worker with novel skeleton tracking algorithms and translating them into actions to command the mobile robot.
- Methods were developed for efficient collaboration between human workers, production supervisors, robot manipulators and AGVs, including user-friendly graphical interfaces, novel human skeleton tracking algorithms for obstacle avoidance with robot manipulators and AGVs, ergonomic monitoring of the human and high level task planners for optimal resource allocation.
In the next 18 months of CoLLaboratE, the consortium continued their efforts in accordance to the work-plan to finalize all the modules needed for collaborative assembly and to start the integration efforts for the deployment of the four use-cases. The achievements include:
- Amendments and contingency measures to handle the delays of the COVID pandemic, while turning all meeting to virtual ones (WP1)
- Finalization of the user-requirements and system architecture, while defining the use-cases to facilitate integration (WP2)
- Finalization of the basic collaboration skills for efficient, adaptive and safe interaction (WP3),
- Finalization of the different modalities to teach assembly tasks by demonstration to a robot (WP4)
- Finalization of the high-level solutions for monitoring ergonomics, interfacing with the robot manipulators and the AGVs and overall task planning (WP5)
In the last 6 months, the consortium continued their efforts in accordance with the work-plan to integrate all the modules of the completed WPs (WP2-WP5) and deploy the four use-cases.
- All 4 use-cases were completed and validated
- The job quality questionnaire was evaluated in all use-cases
- The final deliverables were completed, including the consolidated evaluation and lessons learned
- The exploitation plan and market analysis were completed
The CoLLaboratE partners have produced more than 50 open-access scientific publications in international conferences and journals. The progress made by the project beyond the state of the art is highlighted in the following achievements:
- A correct DMP formulation for encoding orientation trajectories for controlling a robot in the Cartesian space
- A framework that allows incremental refining of existing robot policy through kinesthetic guidance
- An ergodic control algorithm for insertion-like tasks which requires stochastic exploration for task achievement.
- A human tracking algorithms that can be used to construct constraints to increase the safety for human-robot collaboration.
- A method for variable admittance controller to allow easy and precise manipulation of large objects with high inertia.
- A method for high tracking accuracy in following a desired trajectory under model and task uncertainties
- A novel Dynamic Movement Primitive formulation that supports backward reproduction of a learned trajectory
- Several behavior primitives such as search patterns, manipulability behaviors, impedance behaviors and reactive behaviors
- A framework where the initial demonstration evolves into a natural physical human-robot collaboration
- A key-frame extraction framework that performs semantic analysis on videos of human demonstrations
- An intuitive and easy-to-use mobile interface which relies on augmented reality to teach robot paths
- A robot control scheme that enforces the dynamic active constraints
- A framework for human intention recognition
- A production planner that enables the allocation of tasks to the available resources
- Ergonomic monitoring module that models human dynamics
- A gesture recognition module that permits to the collaborative robot to adapt its actions
- An optimization procedure for optimal layout and configuration of the reconfigurable fixtures in the work-cell
- A study to identify factors that influence job satisfaction and quality of the workers
The expected potential impact from the project, as it has been identified by the potential end-users, is highlighted as follows:
- Introduction of human-robot collaboration on continuous production lines and tasks that were previously deemed not appropriate for robot automation
- Ergonomic improvement, particularly in intensive tasks performed manually such as riveting.
- Easy integration in production, drastically reducing the cost of programming.
- Versatility and adaptability, as the robot can be easily taught various collaborative activities.
- A correct DMP formulation for encoding orientation trajectories for controlling a robot in the Cartesian space
- A framework that allows incremental refining of existing robot policy through kinesthetic guidance
- An ergodic control algorithm for insertion-like tasks which requires stochastic exploration for task achievement.
- A human tracking algorithms that can be used to construct constraints to increase the safety for human-robot collaboration.
- A method for variable admittance controller to allow easy and precise manipulation of large objects with high inertia.
- A method for high tracking accuracy in following a desired trajectory under model and task uncertainties
- A novel Dynamic Movement Primitive formulation that supports backward reproduction of a learned trajectory
- Several behavior primitives such as search patterns, manipulability behaviors, impedance behaviors and reactive behaviors
- A framework where the initial demonstration evolves into a natural physical human-robot collaboration
- A key-frame extraction framework that performs semantic analysis on videos of human demonstrations
- An intuitive and easy-to-use mobile interface which relies on augmented reality to teach robot paths
- A robot control scheme that enforces the dynamic active constraints
- A framework for human intention recognition
- A production planner that enables the allocation of tasks to the available resources
- Ergonomic monitoring module that models human dynamics
- A gesture recognition module that permits to the collaborative robot to adapt its actions
- An optimization procedure for optimal layout and configuration of the reconfigurable fixtures in the work-cell
- A study to identify factors that influence job satisfaction and quality of the workers
The expected potential impact from the project, as it has been identified by the potential end-users, is highlighted as follows:
- Introduction of human-robot collaboration on continuous production lines and tasks that were previously deemed not appropriate for robot automation
- Ergonomic improvement, particularly in intensive tasks performed manually such as riveting.
- Easy integration in production, drastically reducing the cost of programming.
- Versatility and adaptability, as the robot can be easily taught various collaborative activities.