Periodic Reporting for period 3 - SWEEPER (Sweet Pepper Harvesting Robot)
Berichtszeitraum: 2018-02-01 bis 2018-10-31
In modern high-tech greenhouses there is a high demand to automate labour. Availability of a skilled workforce performing repetitive tasks in a hot and humid environment is decreasing rapidly. Reduced capacity and resulting increase in labour costs is putting a major pressure on the competitiveness of the European greenhouse sector. Labour costs has an expected increase of 40% in the coming ten years. Without further mechanization, greenhouse food production will most likely migrate out of Europe.
Greenhouse robot harvesters have entered a medium level of readiness (TRL5), and the transition from science to effective use, the so called ‘Technological Innovation Gap’, is a remaining hurdle to overcome. In CROPS (FP7), extensive research on agricultural robotics showed that robotic sweet pepper harvesting is technically possible. Now SWEEPER’s target is to demonstrate feasibility to bring the robot towards market readiness by focussing on market’s demands for robotic abilities in modern greenhouse horticulture (TRL9). The approach is to translate fundamental research results from CROPS into a robust, operationally deployable harvesting robot and to evaluate it under full operational greenhouse conditions.
Putting the first greenhouse harvest robots onto the market will solve current labour capacity problems in greenhouses and will open up a new high tech commercial area. Increasing efficiency and reducing labour dependence also ensures Europe’s leading greenhouse food production yields and competiveness, as well as Europe’s leading role in agricultural robotics. Although SWEEPER aims at harvesting sweet pepper, knowledge and technology can be migrated easily to other vegetable crops like cucumber or tomato.
SWEEPER is an assembly of several subsystems: a mobile autonomous platform, a robotic arm holding an end-effector, and post-harvest logistics. The end-effector contains a camera for detection of sweet pepper and obstacles in order to use precise eye-hand control. The robot will evolve from a Basic, towards an Advanced and Final System. Submodules and prototypes are tested in laboratory and commercial greenhouses. The existing sweet pepper production system will be optimized in order to let the robot perform best. The Final System will be evaluated for performance and demonstrated in a commercial greenhouse. An economic model for evaluating market feasibility using evaluation results will serve as input for a business plan to support market introduction of the robot as follow-up to the project.
Greenhouse robot harvesters have entered a medium level of readiness (TRL5), and the transition from science to effective use, the so called ‘Technological Innovation Gap’, is a remaining hurdle to overcome. In CROPS (FP7), extensive research on agricultural robotics showed that robotic sweet pepper harvesting is technically possible. Now SWEEPER’s target is to demonstrate feasibility to bring the robot towards market readiness by focussing on market’s demands for robotic abilities in modern greenhouse horticulture (TRL9). The approach is to translate fundamental research results from CROPS into a robust, operationally deployable harvesting robot and to evaluate it under full operational greenhouse conditions.
Putting the first greenhouse harvest robots onto the market will solve current labour capacity problems in greenhouses and will open up a new high tech commercial area. Increasing efficiency and reducing labour dependence also ensures Europe’s leading greenhouse food production yields and competiveness, as well as Europe’s leading role in agricultural robotics. Although SWEEPER aims at harvesting sweet pepper, knowledge and technology can be migrated easily to other vegetable crops like cucumber or tomato.
SWEEPER is an assembly of several subsystems: a mobile autonomous platform, a robotic arm holding an end-effector, and post-harvest logistics. The end-effector contains a camera for detection of sweet pepper and obstacles in order to use precise eye-hand control. The robot will evolve from a Basic, towards an Advanced and Final System. Submodules and prototypes are tested in laboratory and commercial greenhouses. The existing sweet pepper production system will be optimized in order to let the robot perform best. The Final System will be evaluated for performance and demonstrated in a commercial greenhouse. An economic model for evaluating market feasibility using evaluation results will serve as input for a business plan to support market introduction of the robot as follow-up to the project.
The Basic System was assembled using a commercial manipulator mounted on a manually operated commercial harvesting platform. It had a new end-effector including an RGB camera, laser distance sensor, and illumination. It did harvest peppers, but it was not robust enough to meet targeted specifications, so the end-effector concept was abandoned.
The Advanced System contained a newly designed, compacter end-effector with a colour and depth camera, high intensity flash LED-lights and a vibrating knife cutting tool, mounted on the manipulator and an autonomous guided platform. A deep-learning image recognition algorithm was added in order to let the robot approach the fruit from an optimal direction avoiding any obstacles. Tests in a commercial greenhouse showed that it could harvest fruit fully autonomously.
The robot was further improved towards a Final System. The fruit cutting tool was enhanced, a fruit catching device was added and the robot was adapted to put harvested peppers in a container. Performance was evaluated at a commercial grower for harvesting yield, cycle time and fruit damages. Based on good results from optimal cropping experiments, best crop types were selected, and two crop varieties, of which one with longer peduncles, were used. Evaluation was done in an existing double stem-row cropping system, in which the optimal cropping system was simulated by manually modifying the crop and taking only single-stem rows into account. In total, 262 fruits were attempted to be harvested. The robot was demonstrated at a Dutch commercial greenhouse and videos of the working robot were made public.
The Advanced System contained a newly designed, compacter end-effector with a colour and depth camera, high intensity flash LED-lights and a vibrating knife cutting tool, mounted on the manipulator and an autonomous guided platform. A deep-learning image recognition algorithm was added in order to let the robot approach the fruit from an optimal direction avoiding any obstacles. Tests in a commercial greenhouse showed that it could harvest fruit fully autonomously.
The robot was further improved towards a Final System. The fruit cutting tool was enhanced, a fruit catching device was added and the robot was adapted to put harvested peppers in a container. Performance was evaluated at a commercial grower for harvesting yield, cycle time and fruit damages. Based on good results from optimal cropping experiments, best crop types were selected, and two crop varieties, of which one with longer peduncles, were used. Evaluation was done in an existing double stem-row cropping system, in which the optimal cropping system was simulated by manually modifying the crop and taking only single-stem rows into account. In total, 262 fruits were attempted to be harvested. The robot was demonstrated at a Dutch commercial greenhouse and videos of the working robot were made public.
The project yielded numerous exploitable results of which many were made public. Main exploitable result is the sweet pepper harvester. Further exploitable results are: an autonomous mobile platform; a (patented) harvesting device (end-effector); post-harvest logistics for grasping & storing; fruit localization and maturity detection, deep learning tool for obstacle detection and avoidance; a ROS-framework for top-level robot control, motion planning and control for manipulator and platform, visual servoing and a user interface. Other supporting results are: 4 sweet pepper image databases; crop management strategies to obtain an open crop structure for robotic harvesting; test scenario’s; and an economic simulation tool.
In the existing double-stem row growing system 18% of ripe fruit were harvested, and the average cycle time to pick one fruit was 24 s. Here, SWEEPER performs about 4 times better than CROPS (6% and 106 s). A good progress; remaining bottlenecks are mainly speed, fruit clustering, occlusion of peppers by leafs and the fact that in a double-stem row cropping system only half of the fruit can be reached and harvested. Taking into account the optimized cropping system, it performed better. In a simulated optimal crop, for a single stem-row assumption, with most occluding leaves and fruit clusters pruned away beforehand, 61% of ripe fruit were harvested. This shows the potential that breeding or other ways to enhance fruit visibility and decrease fruit clustering will have for future robotic harvesting. This would require a whole systems approach in which crop breeding, a new single-stem row crop production system and enhancement of the robot performance form key elements. A fully autonomous robotic sweet pepper harvester, should be viable within 5-10 years.
In the meantime, the robot may be employed in an existing double-stem row cropping system to work as a co-assistant to manual pepper pickers. It requires a crop with lesser clustering and leaf occlusion, and further incremental enhancements for cycle-time (<10s), harvesting yield (~50%) and production losses (<1%). With small adaptations, results can also be used for other crops. In cucumber cropping systems, leaves are pruned continuously, which makes robotic harvesting even more easier than for sweet pepper.
SWEEPER focussed on a challenging field for robotic applications in high-tech greenhouses with very unstructured and variable environments. Robots will replace manual labour, at least partly, in those cases where labour and cost form a bottleneck. Introduction of robots will initiate new jobs for robotic operators and maintenance, as well as change pickers work by having a robot as co-worker. Growers will profit by being more independent from availability of skilled personnel. Machine building companies will profit by new businesses like selling or leasing robots. Results are applicable to other agricultural domains like f.i. open field fruit harvesting. Dissemination activities triggered world-wide a vast amount of requests for collaboration in research and business propositions for harvesting other crops, f.i. cucumber, tomato, grapes, apple, cardamom, banana, and coco palms. The consortium will exploit the results in future R&D and private-public projects.
In the existing double-stem row growing system 18% of ripe fruit were harvested, and the average cycle time to pick one fruit was 24 s. Here, SWEEPER performs about 4 times better than CROPS (6% and 106 s). A good progress; remaining bottlenecks are mainly speed, fruit clustering, occlusion of peppers by leafs and the fact that in a double-stem row cropping system only half of the fruit can be reached and harvested. Taking into account the optimized cropping system, it performed better. In a simulated optimal crop, for a single stem-row assumption, with most occluding leaves and fruit clusters pruned away beforehand, 61% of ripe fruit were harvested. This shows the potential that breeding or other ways to enhance fruit visibility and decrease fruit clustering will have for future robotic harvesting. This would require a whole systems approach in which crop breeding, a new single-stem row crop production system and enhancement of the robot performance form key elements. A fully autonomous robotic sweet pepper harvester, should be viable within 5-10 years.
In the meantime, the robot may be employed in an existing double-stem row cropping system to work as a co-assistant to manual pepper pickers. It requires a crop with lesser clustering and leaf occlusion, and further incremental enhancements for cycle-time (<10s), harvesting yield (~50%) and production losses (<1%). With small adaptations, results can also be used for other crops. In cucumber cropping systems, leaves are pruned continuously, which makes robotic harvesting even more easier than for sweet pepper.
SWEEPER focussed on a challenging field for robotic applications in high-tech greenhouses with very unstructured and variable environments. Robots will replace manual labour, at least partly, in those cases where labour and cost form a bottleneck. Introduction of robots will initiate new jobs for robotic operators and maintenance, as well as change pickers work by having a robot as co-worker. Growers will profit by being more independent from availability of skilled personnel. Machine building companies will profit by new businesses like selling or leasing robots. Results are applicable to other agricultural domains like f.i. open field fruit harvesting. Dissemination activities triggered world-wide a vast amount of requests for collaboration in research and business propositions for harvesting other crops, f.i. cucumber, tomato, grapes, apple, cardamom, banana, and coco palms. The consortium will exploit the results in future R&D and private-public projects.