Periodic Reporting for period 3 - SoMa (Soft-bodied intelligence for Manipulation)
Reporting period: 2018-05-01 to 2019-04-30
Robots accomplish tasks by physically interacting with their environment. In principle, robots can be useful in a broad range of applications, ranging from industrial assembly lines and product distribution centers, to the care of the elderly, and for entertainment and education. One of the major limitations for autonomous robots in all of these applications is the ability to grasp and manipulate objects, something that is an absolute requirement for most robotic applications. In spite of many decades of active research, the dexterity and versatility of human manipulation remained out of reach for robots, fundamentally limiting the applicability of robotic technology.
The project “Soft Manipulation” (SOMA) achieved a breakthrough in robotic manipulation by copying the principles of human manipulation and by transferring these principles to robotic applications. To achieve this, SOMA explored a new avenue of robotic manipulation: manipulation with the environment, as opposed to manipulation of or in the environment. In our approach, the physical constraints imposed by objects in the environment and the manipulandum itself are not regarded as obstacles, but rather as opportunities to guide functional hand pre-shaping, adaptive grasping, and affordance-guided manipulation of objects. The exploitation of these opportunities, which we refer to as environmental constraints (EC), enabled robust grasping and manipulation
in dynamic, open, and highly variable environments.
A key ingredient for the exploitation of EC is softness of hands, i.e. their embodied ability to comply and adapt to features of the environment. The traditional paradigm for robotic manipulation is not able to address this shift of focus: state-of-the-art grasp planners are targeted towards rigid hands and objects, and are an attempt to find algorithmic solutions to inherently complex, often ill-posed problems. Further complicating matters, the requirements of planning for soft, uncertain interactions between hand and environment is entirely beyond the state of the art. However, this is how humans most often use their hands.
SOMA developed a comprehensive new view of robotic manipulation, including novel hands, sensing, control, planning and perception. Taken together, these components deliver versatile, robust, cost-effective, and safe robotic grasping and manipulation capabilities. We successfully demonstrated Soft Manipulation technology in an open manipulation problem in the food logistics industry: the handling of irregularly shaped, flexible, and easily damageable goods, such as fruit and vegetables. We also successfully demonstrated the SOMA technology in an entertainment application, where safety, comfort, and human/robot interaction were the most important metrics for success.
The results of the project are published here: http://soma-project.eu/
The project “Soft Manipulation” (SOMA) achieved a breakthrough in robotic manipulation by copying the principles of human manipulation and by transferring these principles to robotic applications. To achieve this, SOMA explored a new avenue of robotic manipulation: manipulation with the environment, as opposed to manipulation of or in the environment. In our approach, the physical constraints imposed by objects in the environment and the manipulandum itself are not regarded as obstacles, but rather as opportunities to guide functional hand pre-shaping, adaptive grasping, and affordance-guided manipulation of objects. The exploitation of these opportunities, which we refer to as environmental constraints (EC), enabled robust grasping and manipulation
in dynamic, open, and highly variable environments.
A key ingredient for the exploitation of EC is softness of hands, i.e. their embodied ability to comply and adapt to features of the environment. The traditional paradigm for robotic manipulation is not able to address this shift of focus: state-of-the-art grasp planners are targeted towards rigid hands and objects, and are an attempt to find algorithmic solutions to inherently complex, often ill-posed problems. Further complicating matters, the requirements of planning for soft, uncertain interactions between hand and environment is entirely beyond the state of the art. However, this is how humans most often use their hands.
SOMA developed a comprehensive new view of robotic manipulation, including novel hands, sensing, control, planning and perception. Taken together, these components deliver versatile, robust, cost-effective, and safe robotic grasping and manipulation capabilities. We successfully demonstrated Soft Manipulation technology in an open manipulation problem in the food logistics industry: the handling of irregularly shaped, flexible, and easily damageable goods, such as fruit and vegetables. We also successfully demonstrated the SOMA technology in an entertainment application, where safety, comfort, and human/robot interaction were the most important metrics for success.
The results of the project are published here: http://soma-project.eu/
We completed the analysis of human grasping that was envisioned for the project. Additionally, we investigated the role of wrist movement in coordination with finger movement for successful grasping. We now have an understanding of human grasping strategies and their use of stiffness. These findings have guided hand design and use case strategies.
We improved sensor technologies for soft hands and adapted the wrist designs for better usability. We developed a gripper that includes built-in embodied constraints for scooping, and a gripper with opposing PneuFlex actuators. We improved the perceived qualities of soft hands based on use case studies by DISNEY. Finally, we evaluated the resilience of the SOMA hands in impact scenarios.
We studied novel ways to define the closure signature so to achieve pinch grasps using soft hands. The closure signature was used to regrasp objects from edges. We investigated possible controllers and strategies to identify object state and exploit environmental constraints. We developed a novel control framework, which allows compliant positioning of an object after grasping. Finally, we investigated how to replicate human stiffness in a robot performing a collaborative task.
We integrated our multi-object heuristic for grasping form piles in the IFCO. We increased the SOMA Planner robustness by implementing an interface for reactive behavior generation and added new reactive controllers. We added two new grasp strategies one exploiting the corner of the IFCO, the other uses pivoting for edge grasping.
We evaluated updated hand designs for the RBO and CLASH hands. We devised a damage metric based on experiments using a sensor glove. We integrated the latest hardware and software components to the system running at OCADO. We evaluated the final system on complex use case scenarios.
We have evaluated the perceived qualities of robot grasping and SOMA hands using crowdsourcing, and used our findings to improve SOMA hand designs. Strategies for hand and arm control in handshaking have been investigated. The most recent SOMA technologies have been integrated in the system running at DISNEY, and adapted to fit the use case needs. The SOMA hands have also been integrated in a fast and robust system for human-robot handovers.
We improved sensor technologies for soft hands and adapted the wrist designs for better usability. We developed a gripper that includes built-in embodied constraints for scooping, and a gripper with opposing PneuFlex actuators. We improved the perceived qualities of soft hands based on use case studies by DISNEY. Finally, we evaluated the resilience of the SOMA hands in impact scenarios.
We studied novel ways to define the closure signature so to achieve pinch grasps using soft hands. The closure signature was used to regrasp objects from edges. We investigated possible controllers and strategies to identify object state and exploit environmental constraints. We developed a novel control framework, which allows compliant positioning of an object after grasping. Finally, we investigated how to replicate human stiffness in a robot performing a collaborative task.
We integrated our multi-object heuristic for grasping form piles in the IFCO. We increased the SOMA Planner robustness by implementing an interface for reactive behavior generation and added new reactive controllers. We added two new grasp strategies one exploiting the corner of the IFCO, the other uses pivoting for edge grasping.
We evaluated updated hand designs for the RBO and CLASH hands. We devised a damage metric based on experiments using a sensor glove. We integrated the latest hardware and software components to the system running at OCADO. We evaluated the final system on complex use case scenarios.
We have evaluated the perceived qualities of robot grasping and SOMA hands using crowdsourcing, and used our findings to improve SOMA hand designs. Strategies for hand and arm control in handshaking have been investigated. The most recent SOMA technologies have been integrated in the system running at DISNEY, and adapted to fit the use case needs. The SOMA hands have also been integrated in a fast and robust system for human-robot handovers.
The SOMA Project established Soft Manipulation as a highly promising and capable approach to robot manipulation. We demonstrated substantial progress in two industrial applications: in the automated handling of fruit and vegetables in distribution logistics and in safe and comfortable human/robot interaction in the context of the entertainment industry. Our methodological approach of iteratively evaluating our scientific progress in realistic industrial applications delivered targeted advances with an effective use of resources.
The progress beyond the state of the art achieved in the two industrial applications crucially depended on progress in the component technologies and their close integration. The project obtained substantial, novel insights into the principles that produce dexterous abilities in humans. These principles provided a validation for the overarching concepts behind SOMA and steered the advancement and development of novel component technologies. The project substantially advanced robotic hand design within three different and complementary design paradigms. These novel hand designs were augmented with innovative sensor technologies and sensing solutions that enabled closed-loop feedback in grasping and manipulation. Following the goal of manipulation with (rather than in) the environment, the project developed novel control methods as well as novel planning approaches, leveraging the capabilities of soft hands towards the generation of robust and versatile robot manipulation skills.
In summary, SOMA developed a novel approach to robot manipulation. By pursuing an integrated approach to research, advances were motivated by insights obtained from industrial applications. This enabled us to perform groundbreaking research in the component technologies required for robot manipulation while at the same time delivering integrated industrial manipulation systems, validating the effectiveness of Soft Manipulation Technology both in terms of science and in terms of industrial application. Our industrial partners will continue to exploit the benefits of Soft Manipulation Technology beyond the end of this project. This is probably the most convincing way of validating the proposed concepts and demonstrating their future potential.
The progress beyond the state of the art achieved in the two industrial applications crucially depended on progress in the component technologies and their close integration. The project obtained substantial, novel insights into the principles that produce dexterous abilities in humans. These principles provided a validation for the overarching concepts behind SOMA and steered the advancement and development of novel component technologies. The project substantially advanced robotic hand design within three different and complementary design paradigms. These novel hand designs were augmented with innovative sensor technologies and sensing solutions that enabled closed-loop feedback in grasping and manipulation. Following the goal of manipulation with (rather than in) the environment, the project developed novel control methods as well as novel planning approaches, leveraging the capabilities of soft hands towards the generation of robust and versatile robot manipulation skills.
In summary, SOMA developed a novel approach to robot manipulation. By pursuing an integrated approach to research, advances were motivated by insights obtained from industrial applications. This enabled us to perform groundbreaking research in the component technologies required for robot manipulation while at the same time delivering integrated industrial manipulation systems, validating the effectiveness of Soft Manipulation Technology both in terms of science and in terms of industrial application. Our industrial partners will continue to exploit the benefits of Soft Manipulation Technology beyond the end of this project. This is probably the most convincing way of validating the proposed concepts and demonstrating their future potential.