Periodic Reporting for period 2 - FLEXIGROBOTS (Flexible robots for intelligent automation of precision agriculture operations)
Periodo di rendicontazione: 2022-07-01 al 2023-12-31
Digital technologies are increasingly utilized in agriculture to enhance efficiency, safety, and productivity. While the adoption of sensors, robots, and other devices by farmers is growing, there remains room for improvement to fully leverage the potential of digital solutions. Various areas, including disease detection, soil property analysis, coordinated robot usage, and support for manual tasks like harvesting, could benefit from advancements in digital technologies, including AI models. Currently, many of these tasks are performed manually, which can be time-consuming and costly.
FlexiGroBots aims at addressing these challenges. The population is growing, increasing the productivity of the fields leads to a food security scenario. Also, the early detection of diseases and application of phytosanitary products would reduce the application of pesticides, which can be toxic if not managed carefully, increasing safety for the population. Finally, because of the gain in efficiency when using multiple robots, the society will benefit from improvements in the economical and decarbonization aspects.
Having these aspects in mind, FlexiGroBots proposes a platform to support the next generation of digital solutions for agriculture, facilitating tools for the definition and testing of AI models, some horizontal services based on AI models, tools for the management of fleets with multiple robots and an infrastructure for managing a rich Data Space for Agriculture.
The solutions provided have been validated through three pilots focused on three different crops (grapevine, rapeseed and raspberry) in different locations (Spain, Finland, Serbia and Lithuania), which integrate UAVs and UGVs, as well as other data sources (as satellite images). The results obtained have shown not only that the objectives of the project have been fulfilled, but also that the KPIs established for each pilot have been reached. The creation of superscenarios has also proven that FlexiGroBots solutions enable the integration of various robotic systems and that it simplifies their management when working collaboratively.
FlexiGroBots aims at addressing these challenges. The population is growing, increasing the productivity of the fields leads to a food security scenario. Also, the early detection of diseases and application of phytosanitary products would reduce the application of pesticides, which can be toxic if not managed carefully, increasing safety for the population. Finally, because of the gain in efficiency when using multiple robots, the society will benefit from improvements in the economical and decarbonization aspects.
Having these aspects in mind, FlexiGroBots proposes a platform to support the next generation of digital solutions for agriculture, facilitating tools for the definition and testing of AI models, some horizontal services based on AI models, tools for the management of fleets with multiple robots and an infrastructure for managing a rich Data Space for Agriculture.
The solutions provided have been validated through three pilots focused on three different crops (grapevine, rapeseed and raspberry) in different locations (Spain, Finland, Serbia and Lithuania), which integrate UAVs and UGVs, as well as other data sources (as satellite images). The results obtained have shown not only that the objectives of the project have been fulfilled, but also that the KPIs established for each pilot have been reached. The creation of superscenarios has also proven that FlexiGroBots solutions enable the integration of various robotic systems and that it simplifies their management when working collaboratively.
One of FlexiGroBots main outcomes is the creation of an AI and data architecture that enables the design, implementation, training, testing and serving of AI models for Agriculture. To achieve that, FlexiGroBots has set up a Kubernetes cluster, where several technologies are deployed: Kubeflow to develop and deploy AI pipelines, MLflow to register AI models, MinIO to store the data and Kserve to deploy AI models in production. Additionally, the infrastructure for a Data Space for Agriculture has been implemented based on IDSA components, providing connectors to interact with the Data Space. Finally, a graphical web interface has been implemented to improve user experience.
In FlexiGroBots, 10 AI horizontal applications have been developed, targeting features as people detection, tracking, and action recognition, which are vital for ensuring safety in human-robot interactions; object detection, disease management, anonymization for privacy compliance, automatic dataset generation to improve AI model training, 3D virtualization for enhanced agricultural analysis, and specific tools for fruit disease detection, pest detection, and weed management. These advancements collectively contribute to the safety, efficiency, and sustainability of agricultural practices.
Finally, the MCC has been implemented, providing support for both UGVs and UAVs, since it allows control and management of the autonomous vehicles. A graphical interface has been designed to ease the use of this tool.
Regarding the pilots, apart from completing the implementation of the use cases, in the last half of the project the efforts have been directed at building a superscenario for each, so that FlexiGroBots solutions could be tested in multi-robot environments.
In pilot 1 (grapevines-Spain), UGVs were used to support harvesting and field monitoring, while UAVs were used to collect high-resolution images. Their superscenario consisted of a fleet of robots equipped with sensors and cameras and connected to a wireless network. Their work is orchestrated by a Fleet Manager within the MCC, and the information is presented to the farmer through a web platform.
Pilot 2 (rapeseed-Finland) supported activities like silage harvesting, Rumex weeding, and pest management. UAVs have also been used, for example, following and tracking autonomous tractors in the field. In their superscenario, 9 UAVs and UGVs have worked collaboratively.
In pilot 3 (raspberry-Serbia and Lithuania), a set of soil sampling robots has been deployed to make a chemical analysis of the soil. The system has also a sprayer used to add chemicals to the plants. In their superscenario, the integration of up to 3 robots working collaboratively in a real environment was successfully tested.
It should be noted that all the KPIs have been reached, so FlexiGroBots can be considered a success. All the results have been actively communicated (scientific publications, social networks, press notes, website). Additionally, a set of virtual demonstrations were held to present the solutions to DIHs and the related community to increase FlexiGroBots visibility and impact and pave the way for the exploitation of all the solutions.
In FlexiGroBots, 10 AI horizontal applications have been developed, targeting features as people detection, tracking, and action recognition, which are vital for ensuring safety in human-robot interactions; object detection, disease management, anonymization for privacy compliance, automatic dataset generation to improve AI model training, 3D virtualization for enhanced agricultural analysis, and specific tools for fruit disease detection, pest detection, and weed management. These advancements collectively contribute to the safety, efficiency, and sustainability of agricultural practices.
Finally, the MCC has been implemented, providing support for both UGVs and UAVs, since it allows control and management of the autonomous vehicles. A graphical interface has been designed to ease the use of this tool.
Regarding the pilots, apart from completing the implementation of the use cases, in the last half of the project the efforts have been directed at building a superscenario for each, so that FlexiGroBots solutions could be tested in multi-robot environments.
In pilot 1 (grapevines-Spain), UGVs were used to support harvesting and field monitoring, while UAVs were used to collect high-resolution images. Their superscenario consisted of a fleet of robots equipped with sensors and cameras and connected to a wireless network. Their work is orchestrated by a Fleet Manager within the MCC, and the information is presented to the farmer through a web platform.
Pilot 2 (rapeseed-Finland) supported activities like silage harvesting, Rumex weeding, and pest management. UAVs have also been used, for example, following and tracking autonomous tractors in the field. In their superscenario, 9 UAVs and UGVs have worked collaboratively.
In pilot 3 (raspberry-Serbia and Lithuania), a set of soil sampling robots has been deployed to make a chemical analysis of the soil. The system has also a sprayer used to add chemicals to the plants. In their superscenario, the integration of up to 3 robots working collaboratively in a real environment was successfully tested.
It should be noted that all the KPIs have been reached, so FlexiGroBots can be considered a success. All the results have been actively communicated (scientific publications, social networks, press notes, website). Additionally, a set of virtual demonstrations were held to present the solutions to DIHs and the related community to increase FlexiGroBots visibility and impact and pave the way for the exploitation of all the solutions.
FlexiGroBots has achieved significant advancements beyond the state of the art in various fields. Regarding AI and data space infrastructure, the implementation of the IDS reference testbed and the introduction of connectors has revolutionized the sharing and accessing of information, setting the basis for an agricultural Data Space. This includes integration with MinIO as a data lake solution and Kubeflow at the core of the AI platform.
The capability to automatically generate metadata for georeferenced datasets simplifies their indexing within the data cube. This achievement has been carried out focusing on redeploying the geospatial components and services (i.e. Open Data Cube, Mapserver, and PostGIS database) into the common infrastructure.
The AI platform has been enhanced with tools such as Katib, the AI models have been deployed and the Data Space (DS) connectors have been configured to enable connections to it. These advancements can potentially have a favorable socio-economic impact by facilitating data sharing among stakeholders, thus enabling new data-centric business models and fostering the growth of the AI ecosystem in Europe.
Horizontal AI models offer significant benefits in agriculture. Object identification and tracking enable tractor recognition through drone vision, providing farmers with valuable insights. Anonymization models integrate GDPR and ELSE considerations into AI, fostering innovation. These models can enhance the agricultural sector by facilitating robot deployment in the field.
The MCC has developed a solution for managing multiple robots in the field, aiming to automate tasks, optimize routes, and enable self-healing mechanisms for task reassignment. This initiative aims to reduce risks in agricultural robotics, boost adoption, and support extended operations.
The project's pilots have developed new ways to spot diseases from drone pictures and have improved robots for outdoor work. These changes highlight how robots can do more in farming, like checking crops, treating diseases, and removing weeds. This could greatly change farming, showing that digital tools can be used in many ways.
The capability to automatically generate metadata for georeferenced datasets simplifies their indexing within the data cube. This achievement has been carried out focusing on redeploying the geospatial components and services (i.e. Open Data Cube, Mapserver, and PostGIS database) into the common infrastructure.
The AI platform has been enhanced with tools such as Katib, the AI models have been deployed and the Data Space (DS) connectors have been configured to enable connections to it. These advancements can potentially have a favorable socio-economic impact by facilitating data sharing among stakeholders, thus enabling new data-centric business models and fostering the growth of the AI ecosystem in Europe.
Horizontal AI models offer significant benefits in agriculture. Object identification and tracking enable tractor recognition through drone vision, providing farmers with valuable insights. Anonymization models integrate GDPR and ELSE considerations into AI, fostering innovation. These models can enhance the agricultural sector by facilitating robot deployment in the field.
The MCC has developed a solution for managing multiple robots in the field, aiming to automate tasks, optimize routes, and enable self-healing mechanisms for task reassignment. This initiative aims to reduce risks in agricultural robotics, boost adoption, and support extended operations.
The project's pilots have developed new ways to spot diseases from drone pictures and have improved robots for outdoor work. These changes highlight how robots can do more in farming, like checking crops, treating diseases, and removing weeds. This could greatly change farming, showing that digital tools can be used in many ways.