Periodic Reporting for period 3 - REFILLS (Robotics Enabling Fully-Integrated Logistics Lines for Supermarkets)
Reporting period: 2019-01-01 to 2020-12-31
REFILLS proposes to develop solutions allowing robots to improve logistic processes in a supermarket, revolutionising their current structure. The conventional in-store logistics processes are split in three scenarios. In the first scenario, mobile robots would inspect shelves and generate semantic environment maps for layout identification and store monitoring. The second scenario employs robots for three tasks: autonomous sorting of cases from mixed case pallets in the backroom; autonomous transportation of trolleys from the backroom to the shop floor, and assistance to clerks. In the third and final scenario, the robots handle a wide variety of products and refill shelves autonomously.
Supermarket clerks perform a number of time-consuming, repetitive, unergonomic, monotonous and wearing tasks. REFILLS aims at providing physical and cognitive help by performing autonomously some operations (shelf monitoring, depalletizing, pre-sorting, transporting) and assisting the clerks in other operations (spot finding, products replenishment).
Even while automation is improving the experience of customers at retail shops both in terms of ordering and customer comfort, there is still a lack of automation in the logistics management of the retail stores. Most of the logistics costs arise from items handling, items transportation, shelves replenishment and backroom management. The automation introduced by REFILLS may not only reduce the cost of logistics, but it also enables retail stores to function as a hub that can be used by online and delivery services thanks to robotic system solutions.
REFILLS aspires to create autonomous, cooperating and collaborative logistics robots for supermarkets by facing the following scientific and technological objectives:
- Cognition-enabled control for shelf management — Autonomous mobile robots with visual sensors generate large amounts of valuable data. These data are used to build semantic maps of the store and manipulation knowledge bases to perform combined task and motion planning for in-store manipulation activities.
- Perception and control for interacting with objects — Multimodal sensing capability (e.g. visual, force, tactile and proximity) is exploited to allow the robot to recognize objects and find their pose, as well as to safely grasp and manipulate a large variety of objects even in the presence of uncertainties.
- Design of a unique set of robotic modules and integration for in-store logistics application — A new set of robotic modules, built on existing, commercially available platforms and controllers and conceived with the following criteria: low-cost, modularity, control parametrization, optimal kinematics, dependability in operation.
Supermarket clerks perform a number of time-consuming, repetitive, unergonomic, monotonous and wearing tasks. REFILLS aims at providing physical and cognitive help by performing autonomously some operations (shelf monitoring, depalletizing, pre-sorting, transporting) and assisting the clerks in other operations (spot finding, products replenishment).
Even while automation is improving the experience of customers at retail shops both in terms of ordering and customer comfort, there is still a lack of automation in the logistics management of the retail stores. Most of the logistics costs arise from items handling, items transportation, shelves replenishment and backroom management. The automation introduced by REFILLS may not only reduce the cost of logistics, but it also enables retail stores to function as a hub that can be used by online and delivery services thanks to robotic system solutions.
REFILLS aspires to create autonomous, cooperating and collaborative logistics robots for supermarkets by facing the following scientific and technological objectives:
- Cognition-enabled control for shelf management — Autonomous mobile robots with visual sensors generate large amounts of valuable data. These data are used to build semantic maps of the store and manipulation knowledge bases to perform combined task and motion planning for in-store manipulation activities.
- Perception and control for interacting with objects — Multimodal sensing capability (e.g. visual, force, tactile and proximity) is exploited to allow the robot to recognize objects and find their pose, as well as to safely grasp and manipulate a large variety of objects even in the presence of uncertainties.
- Design of a unique set of robotic modules and integration for in-store logistics application — A new set of robotic modules, built on existing, commercially available platforms and controllers and conceived with the following criteria: low-cost, modularity, control parametrization, optimal kinematics, dependability in operation.
In the first scenario, layout identification and shelf monitoring were intended to be the data basis for all REFILLS robotic modules. They turned out to be a very valuable source of data for other processes at retail stores as well.
In the second scenario, a depalletizing cell was developed where a robot equipped with a suitably reconfigurable gripper and with visual and proximity sensors is capable to pick boxes of different sizes and dimensions from a mixed case pallet. This system may have great potential also in warehouses.
In the third scenario, the capabilities of the robots to handle and manipulate a wide variety of products autonomously in various environments including retail stores, using vision and tactile sensing, have been enhanced.
Specific hardware was developed by using a modular approach: a scanning and pointing unit allows to identify products within the shelves and to assist clerks in finding the right spot for replenishment; a robotic handling unit is used to push boxes from a trolley and lift them to a height which is ergonomically suted to be lifted by clerks, and is also capable of refilling the shelves autonomously; a carrier module allows to transport the trolleys from the backroom to the shelves, as well as to move the other robotic systems within the store. A logistic and store management system software allows communication between the robots and with the clerks through smartphones or tablets and can also provide customers with useful information on the products within the store.
REFILLS partners produced 13 papers published on proceedings of international conferences, 10 papers published on international journals, 2 PhD thesis and 3 patents. In addition, the main results of the project will be collected in an organic form on a volume in the Springer Tracts in Advanced Robotics (STAR) series with the contributions of all the REFILLS partners.
The limitations imposed by Covid-19 pandemic made not possible the full integration of all the subsystems developed by the partners in a unique demonstrator. The complete public demonstration of the three scenarios of REFILLS was planned at the Automatica 2020 industrial fair, which was originally planned in June 2020. Due to Covid-19 pandemic, the fair was first postponed to December 2020 and then cancelled. The demonstrations of robot-assisted scenarios at each partner site have been collected in a professional final demonstration video to be used for dissemination in future events.
The research developed within REFILLS has stimulated the creation of two new startup companies:
Ubica Robotics GmbH is a spin-off of the University of Bremen with the aim of further developing autonomous shelf scanning robots and bringing it to the market. The spin-off project is funded by the German Federal Ministry of Economic Affairs and Energy with the “EXIST Transfer of Research” Grant for outstanding research-oriented projects that involve expensive and high-risk resource development.
Neabotics srl is a spin-off of the University of Naples Federico II and CREATE with the aim of transferring outstanding research and development results in the field of service robotis and logistics to the market. In particular, Neabotics will exploit the developments related to the depalletization task.
In the second scenario, a depalletizing cell was developed where a robot equipped with a suitably reconfigurable gripper and with visual and proximity sensors is capable to pick boxes of different sizes and dimensions from a mixed case pallet. This system may have great potential also in warehouses.
In the third scenario, the capabilities of the robots to handle and manipulate a wide variety of products autonomously in various environments including retail stores, using vision and tactile sensing, have been enhanced.
Specific hardware was developed by using a modular approach: a scanning and pointing unit allows to identify products within the shelves and to assist clerks in finding the right spot for replenishment; a robotic handling unit is used to push boxes from a trolley and lift them to a height which is ergonomically suted to be lifted by clerks, and is also capable of refilling the shelves autonomously; a carrier module allows to transport the trolleys from the backroom to the shelves, as well as to move the other robotic systems within the store. A logistic and store management system software allows communication between the robots and with the clerks through smartphones or tablets and can also provide customers with useful information on the products within the store.
REFILLS partners produced 13 papers published on proceedings of international conferences, 10 papers published on international journals, 2 PhD thesis and 3 patents. In addition, the main results of the project will be collected in an organic form on a volume in the Springer Tracts in Advanced Robotics (STAR) series with the contributions of all the REFILLS partners.
The limitations imposed by Covid-19 pandemic made not possible the full integration of all the subsystems developed by the partners in a unique demonstrator. The complete public demonstration of the three scenarios of REFILLS was planned at the Automatica 2020 industrial fair, which was originally planned in June 2020. Due to Covid-19 pandemic, the fair was first postponed to December 2020 and then cancelled. The demonstrations of robot-assisted scenarios at each partner site have been collected in a professional final demonstration video to be used for dissemination in future events.
The research developed within REFILLS has stimulated the creation of two new startup companies:
Ubica Robotics GmbH is a spin-off of the University of Bremen with the aim of further developing autonomous shelf scanning robots and bringing it to the market. The spin-off project is funded by the German Federal Ministry of Economic Affairs and Energy with the “EXIST Transfer of Research” Grant for outstanding research-oriented projects that involve expensive and high-risk resource development.
Neabotics srl is a spin-off of the University of Naples Federico II and CREATE with the aim of transferring outstanding research and development results in the field of service robotis and logistics to the market. In particular, Neabotics will exploit the developments related to the depalletization task.
The application scenarios of REFILLS concern with the four main processes of grocery in-store logistics. In-store logistics is a domain where technologies for automatic shelf replenishment processes are strongly needed for cost and clerk health/ergonomic reasons and for being able to offer better services to customers. By providing innovative robotic solutions for the new domain of in-store logistics, Europe has the challenge to increase, maintain and strengthen its market share of 50% in professional service robotics by 2020. The industrial participants of the REFILLS Consortium: KUKA, Swisslog, INTEL and DM (end user) are committed to this European challenge and want to lead the process and contribute to fill the gap with USA and China.
REFILLS was a great opportunity to validate robotic technologies in logistic applications where robots work close to humans. This experience and the knowledge we gathered can be used in fields with similar problems such as hospitals or intelligent factories.
REFILLS was a great opportunity to validate robotic technologies in logistic applications where robots work close to humans. This experience and the knowledge we gathered can be used in fields with similar problems such as hospitals or intelligent factories.