Impact Aware Manipulation by Dexterous Robot Control and Learning in Dynamic Semi-Structured Logistic Environments

Project Information

I.AM.

Grant agreement ID: 871899

DOI 10.3030/871899

Project closed

EC signature date 5 December 2019

Start date 1 January 2020

End date 31 March 2024

Funded under

• INDUSTRIAL LEADERSHIP - Leadership in enabling and industrial technologies - Information and Communication Technologies (ICT)

Total cost € 4 585 485,00

EU contribution € 4 357 985,00

Coordinated by

TECHNISCHE UNIVERSITEIT EINDHOVEN
Netherlands

Deliverables

Open Research Data Pilot (3)

Data management plan (update D6.3)

The plan for the management of data with specific focus on the IAM impact motion dataset of WP1 Data management plan M6 and update M40

Data management plan (2nd update D6.3)

The plan for the management of data, with specific focus on the I.AM. impact motion dataset of WP1. Data management plan (M6) and update (M40)

Data Management Plan

The plan for the management of data, with specific focus on the I.AM. impact motion dataset of WP1. Data management plan (M6) and update (M40)

Documents, reports (15)

Human safety in impact aware manipulation

This deliverable is a technical report describing how human safety can be ensured within I.AM. software framework to provide both human-safe and impact-aware manipulation, including implementation details of safety modules on a Franka Emika Panda robot manipulator and testing and benchmarking results of dynamic tossing and dynamic boxing.

Dissemination Plan

Publication of the I.AM. dissemination plan.

Physics Engine API for Learning, Planning, Sensing, and Control

Documentation of the API enabling the access to the physics engine for, e.g., requesting post-impact velocities given desired postures and pre-impact velocities, loading and modifying robot, enviroment, and objects kinematic and dynamic descriptions. This deliverable includes the result of numerical testing, performed in conjuction of the valation scenarios, to document the expected functioning and access to the physics engine via the API.

Minutes of the milestone review consortium meeting (focus: final validation TOSS, BOX, and GRAB scenarios)

This deliverable is a document summarizing the reflections and decisions of the whole consortium taken in order to ensure the successful final validation of the three validation scenarios (TOSS, BOX, GRAB) in the final project review.

I.AM. software integration policy

This deliverable and its update regard the I.AM. software integration policy (M6) and the testing of the software integration for the three validation scenarios (TOSS, BOX, GRAB), release public API software.

I.Control report

Collect papers published with an executive summary of main results and proposed algorithms. Separate technical annex gathering the final performances obtained for each modeling benchmark. This final report summarizes the main achievements, limitations and future directions related to I.Control (T4.1, T4.2, T4.3, T2.3). This includes also the publication of the impact aware QP robot control software (I.Control) as open source sofware (on GitHub or similar open platform).

I.Model Report

Collect papers published with an executive summary of main results and proposed algorithms. Separate technicalannex gathering the final performances obtained for each modeling benchmark. This final report summarizes themain achievements, limitations and future directions related to I.Model (T1.1, T1.2, T1.3, T1.4, and T2.1).

Minutes of the milestone review consortium meeting (focus: TOSS scenario)

This deliverable is a document summarizing the reflections and decisions of the whole consortium taken in order to ensure up the reaching of all milestones up to M30 (in particular, the experimental execution of the TOSS scenario).

Impact Posture Planning for Dynamic Manipulation

This deliverable will be composed of online published papers or technical report about the theoretical foundation for the impact posture planning This is include also the publication of the impact posture planning as open source sofware on GitHub or similar open platform

I.Sense report

Collect papers published with an executive summary of main results and proposed algorithms. Separate technical annex gathering the final performances obtained for each modeling benchmark. This final report summarizes the main achievements, limitations and future directions related to I.Sense (T3.1, T3.2). This includes also the publication of the aim-aware impact monitoring pipeline and reflex decision tree software (I.Sense) as opensource sofware (on GitHub or similar open platform).

Minutes of the milestone review consortium meeting (focus: integration)

This deliverable is a document summarizing the reflections and decisions of the whole consortium taken in order to ensure up the reaching of all milestones up to M18 (in particular, software integration and the numerical simulation of the TOSS scenario).

I.AM. software integration policy (update D5.1)

This deliverable and its update regard the IAM software integration policy M6 and the testing of the software integration for the three validation scenarios TOSS BOX GRAB release public API software

Minutes of the milestone review consortium meeting (focus: BOX and GRAB scenarios)

This deliverable is a document summarizing the reflections and decisions of the whole consortium taken in order to ensure up the reaching of all milestones up to M42 in particular the experimental execution of the GRAB and BOX scenarios

Gender Action Plan

Gender action plan to actively address gender unbalance in the consortium.

I.Learn report

Collect papers published with an executive summary of main results and proposed algorithms. Separate technical annex gathering the final performances obtained for each modeling benchmark. This final report summarizes the main achievements, limitations and future directions related to I.Learn (T2.2, T3.3). This includes also the publication of the learning software (I.Learn), not already published in D2.1, as open source sofware (on GitHub or similar open platform).

Demonstrators, pilots, prototypes (3)

Scenario 3 (GRAB) report

This deliverable is a technical report describing the implementation details of I.AM. technology on dual arm KUKA system, including testing and benchmarking results of dynamic (=non-zero contact speed, object- enviroment impact aware) grabbing in comparison with standard (=almost zero-speed, impact unaware) picking and manipulation.

Scenario 2 (BOX) report

This deliverable is a technical report describing the implementation details of I.AM. technology on a Franka Emika Panda robot manipulator, including testing and benchmarking results of dynamic (= non-zero speed, object- enviroment aware) boxing in comparison with standard (= almost zero-speed, object-enviroment unaware) boxing.

Scenario 1 (TOSS) report

This deliverable is a technical report describing the implementation details of I.AM. technology on UR and Panda robots, including testing and benchmarking results of dynamic (=non-zero speed, object-environment aware) tossing in comparison with standard (=almost zero-speed, object-environment unaware) placing on a moving conveyor belt.

Websites, patent fillings, videos etc. (1)

Website and software repository

Publication of the I.AM. website with feeds from relevant social networks

Data sets, microdata, etc (2)

Publication of I.AM. dataset (update D1.1)

Release of I.AM. dataset. The dataset will contain object-enviroment, robot-object, and robot-object-environment collisions data. Precise details of the dataset will be defined in T1.2. The final version will also include an associated scientific paper.

Publication of I.AM. dataset

First release of I.AM. dataset. The dataset will contain object-enviroment, robot-object, and robot-object-environment collisions data. Precise details of the dataset will be defined in T1.2. The final version (later deliverable) will also include an associated scientific paper.

Publications

Conference proceedings (22)

Towards a Reference Framework for Tactile Robot Performance and Safety Benchmarking

Author(s): Kirschner RJ, Kurdas A, Karacan K, Junge P, Baradaran Birjandi SA, Mansfeld N, Abdolshah S, Haddadin S
Published in: International Conference on Intelligent Robots and Systems (IROS) 2021, 2021
Publisher: IEEE

Experimental Analysis of Impact Forces in Constrained Collisions According to ISO/TS 15066

Author(s): Robin Jeanne Kirschner, Nico Mansfeld, Saeed Abdolshah and Sami Haddadin
Published in: 2021 IEEE International Conference on Intelligence and Safety for Robotics (ISR), 2021
Publisher: IEEE

Mean Reflected Mass: A Physically Interpretable Metric for Safety Assessment and Posture Optimisation in Human-Robot Interaction

Author(s): T. Steinecker, A. Kurdas, N. Mansfeld, M. Hamad, R. J. Kirschner, S. Abdolshah, S. Haddadin
Published in: 2022
Publisher: IEEE
DOI: 10.1109/icra46639.2022.9811582

An MPC Framework For Planning Safe & Trustworthy Robot Motions

Author(s): M. Eckhoff, R. J. Kirschner, E. Kern, S. Abdolshah, S. Haddadin
Published in: 2022
Publisher: IEEEE
DOI: 10.1109/icra46639.2022.9812160

CSM: Contact Sensitivity Maps for Benchmarking Robot Collision Handling Systems

Author(s): Kirschner RJ, Jantalia J, Mansfeld N, Abdolshah S, Haddadin S
Published in: IEEE International Conference on Robotics and Automation (ICRA), 2021
Publisher: IEEE

Rm-Code: Proprioceptive Real-Time Recursive Multi-Contact Detection, Isolation and Identification

Author(s): Vorndamme J and Haddadin S
Published in: International Conference on Intelligent Robots and Systems (IROS) 2021, 2021
Publisher: IEEE

Online Payload Identification for Tactile Robots Using the Momentum Observer

Author(s): A. Kurdas, M. Hamad, J. Vorndamme, N. Mansfeld, S. Abdolshah and S. Haddadin
Published in: ICRA 2022, 2022
Publisher: IEEE
DOI: 10.1109/icra46639.2022.9811691

Learning Suction Cup Dynamics from Motion Capture: Accurate Prediction of an Object's Vertical Motion during Release

Author(s): Menno Lubbers; Job van Voorst; Maarten Jongeneel; Alessandro Saccon
Published in: 2022
Publisher: IEEE
DOI: 10.1109/iros47612.2022.9982211

A Robotics Perspective on Experimental Injury Biomechanics of Human Body Upper Extremities

Author(s): Mazin Hamad, Alexander Kurdas, Saeed Abdolshah and Sami Haddadin
Published in: 2021 IEEE International Conference on Intelligence and Safety for Robotics (ISR), 2021
Publisher: IEEE

Model-Adaptive High-Speed Collision Detection for Serial-Chain Robot Manipulators

Author(s): Seyed Ali Baradaran Birjandi, Sami Haddadin
Published in: 2020 IEEE Robotics and Automation Letters, 2020
Publisher: IEEE

Geometric Savitzky-Golay Filtering of Noisy Rotations on SO(3) with Simultaneous Angular Velocity and Acceleration Estimation

Author(s): M. Jongeneel, A. Saccon
Published in: 2022
Publisher: IEEE
DOI: 10.1109/iros47612.2022.9981409

Functional Mode Switching for Safe and Efficient Human-Robot Interaction

Author(s): P. Svarny, M. Hamad, A. Kurdas, M. Hoffmann, A. Saeed and S. Haddadin
Published in: 2022
Publisher: IEEE
DOI: 10.1109/humanoids53995.2022.10000070

Robot-Safe Impacts with Soft Contacts Based on Learned Deformations

Author(s): Dehio, Niels; Kheddar, Abderrahmane
Published in: ICRA, Issue 1, 2021
Publisher: IEEE

Notion on the Correct Use of the Robot Effective Mass in the Safety Context and Comments on ISO/TS 15066

Author(s): Kirschner RJ, Mansfeld N, Peña GG, Abdolshah S, Haddadin S.
Published in: 2021 IEEE International Conference on Intelligence and Safety for Robotics (ISR), 2021
Publisher: IEEE

Learning to Hit: A statistical Dynamical System based approach

Author(s): Harshit Khurana, Michael Bombile, Aude Billard
Published in: IROS 2021, 2021
Publisher: IEEE/RSJ International Conference on Intelligent Robots and Systems

Experimental Injury Biomechanics of Human Body Upper Extremities: Anatomy, Injury Severity Classification, and Impact Testing Setups

Author(s): Mazin Hamad, Alexander Kurdas, Saeed Abdolshah and Sami Haddadin
Published in: 2021 IEEE International Conference on Intelligence and Safety for Robotics (ISR), 2021
Publisher: IEEE

Predicting the Post-Impact Velocity of a Robotic Arm via Rigid Multibody Models: an Experimental Study

Author(s): Ilias Aouaj; Vincent Padois; Alessandro Saccon
Published in: IEEE International Conference on Robotics and Automation (ICRA), 2021
Publisher: IEEE

Robot Contact Reflexes: Adaptive Maneuvers in the Contact Reflex Space

Author(s): J. Vorndamme, L. F. C. Figueredo, S. Haddadin
Published in: 2022
Publisher: IEEE
DOI: 10.1109/iros47612.2022.9981904

Robust Cartesian Kinematics Estimation for Task-Space Control Systems

Author(s): Seyed Ali Baradaran Birjandi; Niels Dehio; Abderrahmane Kheddar; Sami Haddadin
Published in: 2022
Publisher: IEEE
DOI: 10.1109/iros47612.2022.9981233

Robot control for simultaneous impact tasks via QP based reference spreading

Author(s): J van Steen, N van de Wouw, A Saccon
Published in: 2022
Publisher: IEEE
DOI: 10.23919/acc53348.2022.9867812

Real-Time IMU-Based Learning: A Classification of Contact Materials

Author(s): C. M. Valle, A. A. Kurdas, E. P. Fortunić, S. Abdolshah, S. Haddadin
Published in: 2022
Publisher: IEEE
DOI: 10.1109/iros47612.2022.9981139

Model-Based 6D Visual Object Tracking with Impact Collision Models

Author(s): Maarten Jongeneel; Alexandre Bernardino; Nathan van de Wouw; Alessandro Saccon
Published in: 2022 American Control Conference (ACC), 2022
Publisher: IEEE
DOI: 10.23919/acc53348.2022.9867622

Peer reviewed articles (6)

Dual-Arm Control for Coordinated Fast Grabbing and Tossing of an Object: Proposing a New Approach

Author(s): Michael Bombile; Aude Billard
Published in: IEEE Robotics & Automation Magazine, Issue 1, 2022, ISSN 1070-9932
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/mra.2022.3177355

Dual-arm box grabbing with impact-aware model predictive control utilizing soft deformable end-effector pads

Author(s): N. Dehio, Y. Wang, A. Kheddar,
Published in: IEEE Robotics and Automation Letters, 2022, ISSN 2377-3766
Publisher: IEEE
DOI: 10.1109/lra.2022.3158433

The Franka Emika Robot: A Reference Platform for Robotics Research and Education

Author(s): S. Haddadin, S. Parusel, L. Johannsmeier, S. Golz, S. Gabl, F. Walch, M. Sabaghian, C. Jähne, L. Hausperger, S. Haddadin
Published in: IEEE Robotics & Automation Magazine, 2022, ISSN 1070-9932
Publisher: Institute of Electrical and Electronics Engineers
DOI: 10.1109/mra.2021.3138382

On Inverse Inertia Matrix and Contact-Force Model for Robotic Manipulators at Normal Impacts

Author(s): Yuquan Wang; Niels Dehio; Abderrahmane Kheddar
Published in: IEEE Robotics and Automation Letters, Issue 4, 2022, ISSN 2377-3766
Publisher: IEEE
DOI: 10.48550/arxiv.2109.04756

Predicting impact-induced joint velocity jumps on kinematics-controlled manipulators

Author(s): Y. Wang, N. Dehio, A. Kheddar,
Published in: IEEE Robotics and Automation Letters, 2022, ISSN 2377-3766
Publisher: IEEE
DOI: 10.1109/lra.2022.3167614

Expectable Motion Unit: Avoiding Hazards From Human Involuntary Motions in Human-Robot Interaction

Author(s): Robin Jeanne Kirschner; Henning Mayer; Lisa Burr; Nico Mansfeld; Saeed Abdolshah; Sami Haddadin
Published in: IEEE Robotics and Automation Letters, Issue 1, 2022, ISSN 2377-3766
Publisher: IEEE
DOI: 10.48550/arxiv.2109.07201