Periodic Reporting for period 3 - SeaClear (SEarch, identificAtion and Collection of marine Litter with Autonomous Robots)
Reporting period: 2023-01-01 to 2023-12-31
Today's oceans contain 26-66 million tons of waste, with approximately 94% located on the seafloor. So far, collection efforts have focused mostly on surface waste, with only a few local efforts to gather underwater waste, always using human divers. No solution exists that exploits autonomous robots for underwater litter collection; the SeaClear project will develop the first. We will create a mixed team of Unmanned Underwater, Surface, and Aerial Vehicles – UUVs, USVs, UAVs – to find and collect litter from the seabed and from the water column, focusing on coastal areas since that is where waste inflow concentrates. The UAVs and an inspection UUV map the litter, aiming to establish correlations between surface and underwater litter. A collection UUV then gathers litter, using a combined suction-gripper manipulator. The end goal is to operate the robots autonomously, without remote human intervention. When fully operational, the SeaClear system aims to detect and classify underwater litter with 80% success rate, and collect it with a 90% success rate; all this at 70% reduced cost compared to human divers.
The SeaClear system will be displayed at three demo sites where an autonomous robot system will be verified: one demo site location for the purpose of cleaning ports in the Hamburg port area with the end-user Hamburg Port Authority (Germany), another two in the area of Dubrovnik, namely near Lokrum Island and one in the area of the Mali Ston Bay, with Regional agency DUNEA (Croatia) as end-user for these areas. With these three demo sites, SeaClear has an overview in completely different sectors, given the fact that we have a pilot site of port location emphasizing maritime industries and two other locations representing protected nature areas, one from the tourism sector and the other from the mariculture sector, namely the shellfish industry. By including all of the mentioned areas, the SeaClear system will face different waste fractions, both from inland and sea origin, and obstacles that need to be solved in order for such a system to be fully functional.
The SeaClear system will be displayed at three demo sites where an autonomous robot system will be verified: one demo site location for the purpose of cleaning ports in the Hamburg port area with the end-user Hamburg Port Authority (Germany), another two in the area of Dubrovnik, namely near Lokrum Island and one in the area of the Mali Ston Bay, with Regional agency DUNEA (Croatia) as end-user for these areas. With these three demo sites, SeaClear has an overview in completely different sectors, given the fact that we have a pilot site of port location emphasizing maritime industries and two other locations representing protected nature areas, one from the tourism sector and the other from the mariculture sector, namely the shellfish industry. By including all of the mentioned areas, the SeaClear system will face different waste fractions, both from inland and sea origin, and obstacles that need to be solved in order for such a system to be fully functional.
The SeaClear consortium has successfully achieved its objectives as stated in the DoA until now with significant progress in system, software, and hardware development as well as in development of use cases, outreach, and community building, as well as successful demonstration of the SeaClear system in three demo sites in Hamburg and Dubrovnik (Bistrina and Lokrum).
Important highlights include: A cost effectiveness study based on the current development status has been set up, benchmarking the robotic system with divers, proving SeaClears positive impact in terms of cost and performance. The robot hardware in the system (USV, inspection UUV, collection UUV, gripper, collection basket, and interfaces) has been developed and integrated, including fully novel gripper and an advanced design of the collection basket. The SeaCat USV has been equipped with 3 launching and recovery systems for the 2 ROVs and the basket. Video and acoustic detection and classification sensors have been selected and added to the USV and the ROVs. In addition, complete hardware integration and trials have been conducted in Marseilles.
Automated lawnmower mapping has been demonstrated in the field tests, and proof-of-concept lab experiments have been performed in path-planning for fast, active mapping. Moreover, we have successfully carried out the automatic litter collection sequence in field demonstrations, and demonstrated the data-driven online learning control framework in simulations. In addition, field demonstrations showed autonomous coordinated control of the tethered UAV landing procedure and of the cable slack.
A dataset with underwater images has been generated and enhanced; this is currently the largest publicly available data set of labelled underwater litter. In addition, it is the first underwater litter data set for shallow-water environments. We have designed an improved image-enhancement and sensor fusion pipeline for underwater object classification via convolutional neuronal networks. Moreover, a state-of-the-art deep learning architecture has been developed for the underwater litter detection and classification, including successful validation in real-time application in Bistrina.
The SeaClear‘s Shore Operation Center, consisting of an end-user-oriented web application and a standalone operator application, has been deployed and validated. In addition, the final network topology layout has been deployed during sea trials and final demonstrations. We successfully demonstrated the SeaClear system in operational environments in Hamburg and Dubrovnik. The results of these demonstrations have been thoroughly analysed and used to benchmark our project against the various KPIs defined in the project proposal.
We had impactful press releases for tests and demos, with TV, online, and print appearances including e.g. a dedicated ZDF documentary and prime-time-news appearances, with a total estimated reach in the millions of people. In addition, we have set up active social media channels, complemented by intense outreach at e.g. trade fairs, museum exhibitions, and science popularization events. We also developed educational material including cartoons, school lesson templates, and infographics. Academic dissemination and impact includes 4 workshops, many scientific papers, spin-off lectures, and 8 PhD thesis projects.
Important highlights include: A cost effectiveness study based on the current development status has been set up, benchmarking the robotic system with divers, proving SeaClears positive impact in terms of cost and performance. The robot hardware in the system (USV, inspection UUV, collection UUV, gripper, collection basket, and interfaces) has been developed and integrated, including fully novel gripper and an advanced design of the collection basket. The SeaCat USV has been equipped with 3 launching and recovery systems for the 2 ROVs and the basket. Video and acoustic detection and classification sensors have been selected and added to the USV and the ROVs. In addition, complete hardware integration and trials have been conducted in Marseilles.
Automated lawnmower mapping has been demonstrated in the field tests, and proof-of-concept lab experiments have been performed in path-planning for fast, active mapping. Moreover, we have successfully carried out the automatic litter collection sequence in field demonstrations, and demonstrated the data-driven online learning control framework in simulations. In addition, field demonstrations showed autonomous coordinated control of the tethered UAV landing procedure and of the cable slack.
A dataset with underwater images has been generated and enhanced; this is currently the largest publicly available data set of labelled underwater litter. In addition, it is the first underwater litter data set for shallow-water environments. We have designed an improved image-enhancement and sensor fusion pipeline for underwater object classification via convolutional neuronal networks. Moreover, a state-of-the-art deep learning architecture has been developed for the underwater litter detection and classification, including successful validation in real-time application in Bistrina.
The SeaClear‘s Shore Operation Center, consisting of an end-user-oriented web application and a standalone operator application, has been deployed and validated. In addition, the final network topology layout has been deployed during sea trials and final demonstrations. We successfully demonstrated the SeaClear system in operational environments in Hamburg and Dubrovnik. The results of these demonstrations have been thoroughly analysed and used to benchmark our project against the various KPIs defined in the project proposal.
We had impactful press releases for tests and demos, with TV, online, and print appearances including e.g. a dedicated ZDF documentary and prime-time-news appearances, with a total estimated reach in the millions of people. In addition, we have set up active social media channels, complemented by intense outreach at e.g. trade fairs, museum exhibitions, and science popularization events. We also developed educational material including cartoons, school lesson templates, and infographics. Academic dissemination and impact includes 4 workshops, many scientific papers, spin-off lectures, and 8 PhD thesis projects.
The key innovation in SeaClear is that our project is the first to develop an autonomous, robotic solution for cleaning litter off the sea floor. This involves significant advances beyond the state of the art in litter classification, machine learning for mapping, and cooperative and shared control. Significant results include the UUV pose estimation using the UAV with experimental verification, the SeaClear underwater litter data set which is the first in its kind for shallow-water environments, the experimental validation of the aerial mapping and litter detection despite glare on the ocean surface, a qualitative experimental validation of camera- and sonar-based mapping, improvements on the mixed model-based and learning control strategy for grasping objects with the collection ROV and the transfer of the visual servo control to the automated basket approach.
System integration has been succesfully accomplished, intregating the unmanned surface vessel, the observation ROV, the collection ROV, the aerial drone, and the collection basket. In addition, appropriate server backbone has been designed and implemented. A Shore Operation Center (SOC) has been tested for commanding the robots to specific waypoints, while the SeaClear Service Layers enable monitoring the robots over the WebUI. The individual and integrated hardware and software components of the SeaClear robotic system have been tested and validated during test campaigns in Dubrovnik, Hamburg, and Marseille.
A business plan has been defined for system sales as well as usage of the system as a service. Exploitable results have been listed, and potential end users have been identified, together with a preliminary list of targeted stakeholders for immediate marketing actions.
In addition, the fact that the end-users are involved in all steps of the project significantly increases the chances of successful deployment of the system after the end of the project.
System integration has been succesfully accomplished, intregating the unmanned surface vessel, the observation ROV, the collection ROV, the aerial drone, and the collection basket. In addition, appropriate server backbone has been designed and implemented. A Shore Operation Center (SOC) has been tested for commanding the robots to specific waypoints, while the SeaClear Service Layers enable monitoring the robots over the WebUI. The individual and integrated hardware and software components of the SeaClear robotic system have been tested and validated during test campaigns in Dubrovnik, Hamburg, and Marseille.
A business plan has been defined for system sales as well as usage of the system as a service. Exploitable results have been listed, and potential end users have been identified, together with a preliminary list of targeted stakeholders for immediate marketing actions.
In addition, the fact that the end-users are involved in all steps of the project significantly increases the chances of successful deployment of the system after the end of the project.