ROBOT FLEETS FOR HIGHLY EFFECTIVE AGRICULTURE AND FORESTRY MANAGEMENT

Información del proyecto

RHEA

Identificador del acuerdo de subvención: 245986

Proyecto cerrado

Fecha de inicio 1 Agosto 2010

Fecha de finalización 31 Julio 2014

Financiado con arreglo a

• Specific Programme "Cooperation": Nanosciences, Nanotechnologies, Materials and new Production Technologies

Coste total € 8 892 394,80

Aportación de la UE € 6 599 286,00

Coordinado por

AGENCIA ESTATAL CONSEJO SUPERIOR DE INVESTIGACIONES CIENTIFICAS
Spain

Final Report Summary - RHEA (ROBOT FLEETS FOR HIGHLY EFFECTIVE AGRICULTURE AND FORESTRY MANAGEMENT)

Executive Summary:
4.1.1 Executive summary
The European Union requires an important amount of pesticides a year to maintain its current food production. This represents a major economic burden for farmers and an unsustainable chemical load for the surroundings. Reducing this input has become vital to maintain the European productivity and best of all to preserve EU’s environment.
Precise management of agricultural land is being made possible due to the availability of new technologies, including global positioning systems (GPS), geographic information systems (GIS), advanced sensors, automation of agricultural machinery, and high-resolution image sensing. Consequently, the concept of Precision Agriculture has emerged as the management strategy that uses information technologies to collect and process data from multiple sources in order to facilitate decisions associated with crop production.
Within this context, RHEA was envisaged to design, develop, test and assess a new generation of automatic and robotic systems for both chemical and physical effective weed management aiming to diminish the use of agricultural chemical inputs, improving crop quality, health and safety for humans, and reducing production costs by means of sustainable crop management. To achieve this overall objective, a fleet of heterogeneous ground and aerial robots has been developed and equipped with innovative sensors, enhanced end-effectors and improved decision control algorithms to cover a large variety of European agricultural and forestry products, including agricultural wide-row crops (maize), narrow-row crops (winter wheat) and woody perennials (olive groves).

The project RHEA focused on a number of scientific and technical objectives aiming to detect up to 90% of the weed patches by developing new perception systems, eradicate up to 90% of the weed detected by developing innovative agricultural implements based on physical and chemical features and reduce the use of agricultural inputs by about 75%. These objectives have been achieved with the development of a heterogeneous fleet made up of two aerial robots and three ground mobile robots capable of carrying perception and actuation systems.
Every aerial robot, based on a drone tailor made for a 2-kg payload, carries a remote perception system that consists of a set of two vision cameras operating in the visible and Near-infra Red spectra, respectively. Thus, the action of the two drones equipped with the remote sensing system provides information of the overall crop field. The data obtained are analyzed off-line by the weed mapping system to identify and locate the weed patches in wheat and maize crops.
The ground robots have been based on a commercial tractor chassis and equipped with a ground sensing system consisting of a vision camera attached at the top/front of the robots. This system takes pictures of the crops and analyses them in real time to determine the presence of weed patches and identify the crop rows. The ground robots can also be equipped with innovative agricultural implements for precise, real-time spraying that have succeeded at saving the stated target for chemicals in the application of herbicides in wheat crops and pesticides in olive groves. A third implement has been developed to destroy weeds based on physical (mechanical and thermal) devices that have achieved to destroy the weed removal target.
These subsystems (perception, actuation and the mobile robots) are coordinated accurately by the Mission Manager, a software package developed to define the missions and control the robots and related equipment, that interacts with the operator in charge of the mission through the Graphic User Interface as well as with the robots via a wireless communication system that interexchange commands and data including the positions of the robots provided by the Localization System based on GPS techniques and placed onboard the robots. Finally, safety to humans and animals, as well as to the robots themselves, is provided by the safety system based on vision and laser components and techniques.
RHEA met its main and specific objectives and the consortium concluded the project with two demonstrations to the EC representatives and the agriculture community (videos available as attached files). The project results have been disseminated through publications of articles in journals (20) and conferences (76), and participations in international fairs (2). The exploitation of the complete system has been assessed as challenging, but several subsystems will be commercially available with a little extra effort.


Project Context and Objectives:
4.1.2 Summary description of project context and objectives
European Union countries use more than 280,000 tons of pesticides in agriculture and forestry tasks; practically one half consists of herbicides. Clearly, these products make a significant contribution to maintain food production: each euro invested in pesticides returns 4 euros in crops saved. Considering that total sales of pesticides in Europe are currently higher than 7,000 million euro a year, we can estimate that, in Europe, pesticides may provide about 28,000 million euro a year in saved crops. However, such assessments do not consider the indirect, but substantial, environmental and economic costs associated with pesticide use. It has been estimated that only 0.1% of applied pesticides reach the target pests, while the bulk of each pesticide application (99.9%) is left to the surrounding environment, which represents a substantial chemical impact for the environment and increases the risk of undesirable side effects on human food, water and natural ecosystems. The economic value of these environmental impacts has been estimated to total about 8,000 million dollar a year in the USA. This is an important cost for local and national administrations; but pesticide use is also difficult to bear economically by farmers. The use of precise management technologies in agricultural, coined as Precision Agriculture, which has been under development for the last two decades, can provide substantial savings of herbicides and pesticides by making use of available new technologies such as global positioning systems (GPS), geographic information systems (GIS), automated agricultural machinery, and high-resolution image systems and sensors.
Although the scientific and technological bases of Precision Agriculture are mostly known and robust, the commercial application of these new technologies is still very limited in most European countries. To overcome this situation new agricultural processes and methods should be devised. A modern approach is to use existing Information and Communication Technologies (ICT) and design and build improved pest and crop sensors along with enhanced actuators to perform the proper pest control actions. Mobile platforms are to be provided in order to move those sensors and required actuators all over the working field.
In this context, the main idea of the RHEA consortium, formed by 19 multidisciplinary, experienced groups from 15 institution/companies (three research centres, 4 universities, 7 SMEs and one large industry) of 8 European countries, was to build, conduct experiments and evaluate a new generation of automatic and robotic systems for chemical and physical (mechanical and thermal) weed management. The robotic systems consists of a fleet of heterogeneous ground and aerial robots developed and endowed with advanced sensors, innovative end-effectors and improved decision control algorithms and devoted to reduce the use of chemical inputs in agriculture and forestry as well as to reduce production costs while improving crop quality, health and safety to humans and animals (See Fig. 1.1 in attached file). The RHEA system covers a large variety of European products, including agricultural wide-row crops (maize), narrow-row crops (winter wheat) and woody perennials (olive groves). This last application allows RHEA to be easily extended to forestry –woody perennial.
Thus, the RHEA project aimed to apply robotic systems to agriculture crops with an important impact in improving the economy and protecting the environment as well as in sustaining countryside regions by launching new high technological jobs and allowing women to access jobs traditionally reserved for men. RHEA can be considered as a cooperative robotic system, falling within an emerging area of research and technology with a large number of applications, as reported in the Strategic Research Agenda for Robotics in Europe (euRobotics aisbl).
To achieve the RHEA main goal, some specific technical and scientific objectives were defined at the project proposal time and refined during the first semester of the project development. Thus, the RHEA consortium has focused its activity in the development of:

1.- Advanced systems and algorithms for weed mapping based on computer vision in two modes: remote sensing and ground sensing.
Thus, this objective defined two different systems:
- The Remote Perception System was envisaged to collect information relevant to the generation of weed infestation maps (Weep Mapping System) by using special devices that provide images in the multispectral range, including a near infra-red band for reliable detection of vegetation. The Remote Perception System was devised to be put onboard the aerial robots and able to collect images at a height of 4 to 150 meters with a great stability, giving large possibilities in terms of spatial resolution. The target of this objective was to detect up to 90% of the weed patches
- The Ground Perception System was foreseen to be installed on the ground vehicles, with the following purposes:
- The detection and location of natural structures to be treated in order to guide the action of the actuators: in–row weeds and intra-row weeds. The target of this objective was to detect up to 90% of the weed patches.
- The detection of elements useful for achieving an optimal and safe autonomous navigation in real time, i.e. estimating the row position for guiding purposes, detecting obstacles to avoid collisions, etc.

2.- Innovative algorithms for decision-making modules, including behaviors such as coordination, cooperation and collaboration.
Decision-making will be performed in two different subsystems: Mission Manager and High-level Decision Making System.
- The Mission Manager was designed to be in charge of collecting data provided by the operator, Remote Perception System and Mobile Units as well as making decisions based on the information obtained. The operator provides information related with the type of mission (treatment) and field specifications (crop type, dimensions of mission fields, geographical position, etc.). The Remote Sensing System sends data related with the position and size of the weed patches. The mobile units provide their status mainly (position, orientation, status of different modules, errors, etc.). With that information, the Mission Manger makes decisions on the number of aerial units for inspecting the mission field; selects the number of ground mobile units to accomplish the task; and provides an action plan for each mobile unit (Mission Planner). In addition, the Mission Manager was also devised to be in charge of supervising the mission and taking real-time decisions at unexpected events (Mission Supervisor).
- The High-Level Decision Making System, on board the ground mobile units, was designed to consider the actions sent by the Mission Manager and the information from the on-board sensing systems in order to generate the outputs for the Actuation System: high-level commands to control the ground mobile units and treatment devices (steering, traction, brakes, etc., opening/closing nozzles, folding/unfolding spraying booms, etc.).

3.- Enhanced actuators for precise, real-time spraying, to reduce chemicals by about 75%.
After selecting the final crops for testing the overall RHEA system (Milestone 1), two implements were devised:
- A patch sprayer consisting of twelve high-speed, independent solenoid valves mounted on a horizontal sprayer boom with an equidistant spacing of 0.5 m. Thus the High-level Decision Making System can control (open/close) every nozzle independently providing a spatial transversal resolution of about 0.5 m.
- A canopy sprayer to apply pesticides optimizing the doses by adapting the spraying to the canopy sizes and improving the liquid management. The system was based on two vertical stacks with four equal-separated, articulated nozzles each. The detection of canopies was envisaged to be done by using a system based on ultrasonic sensors. This sensor system acts directly on the actuation system; thus, this implement is not controlled by the High-level Decision Making System and works autonomously.

4.- Improved end effectors to destroy weeds based on physical systems aimed to destruct up to 90% of the detected weeds.
This physical implement was based on mechanical and thermal tools. The mechanical weed removal tool consisted in shallow soil tillage interventions carried out with different “end-effector” tools characterized by static or elastic blades attached to the main frame of the implement. The thermal weed removal tool was based on rod burners fed with Liquefied Petroleum Gas for open flame or infrared applications (dry heat), steam and activated steam dispensers (humid heat), etc.

5.- A fleet of mobile robots –ground and aerial vehicles– capable of acquiring images of the task field and either applying mechanical or chemical processes for crop and weed management.
- The aerial robots, in charge of carrying the Remote Sensing System, were based on a drone configuration. Two drones were envisaged to be built featuring about 2-kg payload with about 30-minutes flying autonomy. The drones had to exhibit communication and location capabilities as well as manual and automatic remote control (Aerial unit high-level controller).
- The ground robots relied on wheeled platforms and were designed to carry the relevant agricultural implement and equipment for autonomous navigation. The ground vehicles were expected to feature a speed up to 3 km/h, motion on slopes up to 15 degrees with adaptation to uneven terrain with irregularities in the range +-20 cm as well as real-time communications and GPS location capabilities (Communication and Location System). Three units were built to carry each one of the three implements to be developed and coordinated to behave as a fleet of robots. The consortium agreed to adapt commercial medium-sized vehicles to configure the ground mobile units.

6.- Robot guidance devices and algorithms based on computer vision (a forward-looking view of the crop rows and obstacle avoidance). The ground robots will follow the rows at a speed of about 6 km/h with a positioning accuracy of about +-2 cm.
Algorithms to steer the ground robots (crop row tracking) were developed based on the information provided by the Ground Perception System and the Location System. The row tracking system main objective is to steer a ground robot in wide-row crop applications (maize) where the robot is prevented from stepping on the crop rows. In narrow-row crop applications the steering using the row tracking system has no sense, and in olive groves the trajectories can be defined easily in advance. Thus, in these last two applications the steering was performed by using the Location System (GPS trajectory definition and feedback) exclusively.

7.- Human-machine interfaces for monitoring/controlling autonomous outdoor vehicles.
The objective was to develop a friendly Graphic User Interface (GUI) for system operation, monitoring, information record-keeping and operation optimization. The GUI system was connected with the fleet of robots in real time to display the current state of operations in 3D; i.e. position, orientation, speed and status of every robot. It allows the user to send instructions to the fleet of robots and includes a simulator to permit the operator to test new operations rapidly, before the instructions are actually sent to the real system. This helps in managing emergency situations where the user has to define new operations quickly (for example, the failure of one robot).

8.- New strategies for planning missions with teams of robots and re-planning them after the failure of a number of robots. An overall scanning of the terrain of up to 95% is expected.
The objective was to determine the best routes (multi-path plan) for a fleet of n robots to treat a complete working area while minimising some cost criteria, normally time and money. The cost function can be expressed in terms of the herbicide/pesticide tank capacity, the number of turns required in the headlines or the time spent in the whole treatment. Re-planning the mission was considered as a new plan of the non-treated area with n-i robots after the failure of i robots. The failures are detected by the Mission Supervisor, which is in charge of monitoring the overall system status: acquiring data from the mobile units and inferring their status while identifying and solving any malfunction in the fleet or informing the operator.
These initial project objectives were complemented with other aspects mentioned in the project proposal and highlighted as significant objectives by the consortium at the beginning of the project development:

9.- Safety system
Safety of people around vehicles is a must in autonomous outdoors robots and was considered essential in the RHEA project. EU regulation was adopted as a strong requirement, and a safety system was defined to be implemented in the robots. This safety system works independently of the mobile unit controller to stay active even in the case of failure or malfunction in the controller. Three sub-systems were envisaged to be installed in every ground mobile unit:
- Manual safety system: It consisted of a number of emergency buttons distributed around the unit to be set on by humans around. This sub-system halts the robot and stops the engine.
- Proximity safety system: It was based on a range finder (Laser) to detect objects in the robot’s path. This subsystem halts the robot and pauses the engine, until the obstacle is out of sight.
- 3D obstacle detection system: It was based on the information provided by the Ground Detection System. Information with the number of obstacle in the front of the robot and their positions are sent to the vehicle controller in charge of making decisions (continue, halt, etc.).
1
0.- New energy power system
RHEA was devised to use conventional fuel engines for the ground mobile units to ensure the energetic autonomy for an economically profitable mission time. However, RHEA gave us a chance to check new energy sources. Thus, a hybrid energy pack consisting of a fuel cell and a solar panel was installed in every Ground Mobile Unit. Each energy pack included a hydrogen fuel cell system, hydrogen storage on metal hydride tanks, a solar photovoltaic system, batteries as well as power electronics and the main control system.
The whole RHEA system was broken down into six main subsystems organized in a hierarchical manner (See Fig. 1.2 in attached file). The Base Station, where the operator defines the mission through the GUI and takes control of the robot fleet, is at the top of this hierarchy; the Mission Manager, which defines the missions (Mission Planner) and takes automatic control of the Ground Mobile Units (Mission Supervisor), Aerial Mobile Units (Aerial Unit High-level Controller), and related equipment, is placed on a layer below. Those Mobile Units carry the Perception System that consists of the Remote Perception System on board the Aerial Mobile Units and the Ground Perception System on board the Ground Mobile Units. In addition, the Ground Mobile Units also carry the actuation equipment, which consists of physical (mechanical and thermal) and chemical (sprayers) tools to destroy weeds or apply pesticide. All mobile units are located through the Localization System based on GPS technology and the information exchanged by both mobile and fixed equipment is interconnected by the Communication System. The main goal of the project is to develop and coordinate these subsystems to apply efficiently precise agricultural techniques in order to minimize the use of chemical products in agricultural and forestry tasks. See Fig. 1.3 (in attached file) for a general view of the fleet.
To achieve these objectives, the work was divided into 11 work packages (WP), covering the aspects of the definition of technical requirements and specifications (WP1), subsystem design and development (WP2 to WP7), integration and tests (WP8), dissemination and exploitation (WP9), management (WP10) and final demonstration (WP11). To perform a periodic assessment of the fulfillment of the objectives in order to execute corrective actions, the consortium defined six evaluation points. Overcoming these milestones means an agreement by the project participants as regards the technical description of the whole system (MS1, month 6), the design of subsystems (MS2, month 12), the evaluation of individual subsystem prototypes (MS3, month 36), the delivery of the fleet (MS4, month 42), the final system integration (MS5, month 46) and in-field demonstration (MS6, month 46).


Project Results:

4.1.3 A description of the main S&T results/foregrounds
The RHEA project has been managed through three reporting periods. The First period covering from month 1st to month 12th (August 1, 2010 to July 31, 2011), the Second Reporting Period covering month 13th to month 30th (August 1, 2011 to January 31, 2013) and the Third Reporting Period beginning in month 31st and ending in month 48th (February 1, 2013 to July 31, 2014). A description of the main scientific and technical achievements in the project RHEA is commented below grouped into work packages.
WP1 - TECHNICAL REQUIREMENTS, SPECIFICATIONS AND SYSTEM BREAKDOWN
Leader: CSIC
Participants: All
Duration (planned/actual): Month 1 to month 6/ Month 2 to month 6

Main S&T results:
This activity started at the time of the kick-off meeting (month 2) and was essential for the participants in order to understand the concept philosophy behind the project idea. Along the development of this activity, the consortium held three technical meetings attended by most of the participants that resulted extremely important for the convergence of participant’s individual interests into the projects objectives.
The work performed and the progress made are described in the deliverable D1.1 - “Technical requirements, specifications and project breakdown”, whose unanimously approval by the member of the General Assembly meant to overpass the first project milestone (MS1). The following S&T results can be pointed out:
Selection of relevant test scenarios
The Annex I – Description of Work (DoW) stated the final system would be focused on three broad scenarios trying to extend the final system to a large number of different crops. These scenarios were narrow-row crops (wheat, barley, rye, etc.), wide-row crops (maize, sunflower, sugar beet, etc.) and woody perennial (olive trees, forest land, etc.). However, the consortium realised that the design of agricultural implements strongly depends on the specific crop rather than the crop type. Thus, after discussing the specific potential crops for final testing by all the participants, the Scientific and Technical Board decided to focus the development of the project on wheat as a representative crop of narrow-row crop, maize as the most important example of wide-row crops, and olive tree as a species of woody perennial.
Definition of the final tests and measurable criteria for final project assessment
Detailed definition of tests, experiments and measurements to perform the final assessment of the project results were stated. They were defined in terms of the precise seeds to be used and the required planting time for every crop. In addition, the way in which the weed patches would be determined were defined as well as the techniques for assessment of the herbicide/pesticide application processes (strips of water-sensitive paper or similar technique) in wheat and olive tree, respectively. In maize scenarios, a manual counting of the weeds into quadrats was selected as the most adequate assessment method.
Definition of the overall system features

After the selection of the specific target crops to be treated, the consortium focused mainly on:
- Defining the operational dimensions of the treatment devices (5.5 m-wide patch sprayer, 3 m-wide physical tool, 1.5 to 4 m-high canopy sprayer), mass distribution, tank capacities, power supply, etc.
- Selecting the basic dimensions, payloads, speed and operation time of the aerial and ground mobile units including the basic safety systems
- Determining the extrinsic and intrinsic features of the perception system, specifically number of cameras and configuration, sensor type, resolution, pixel size, focal length, etc.
- Selecting the positioning technique and accuracy of the location system along with the generation and transportation of the correction signal
- Describing the basic communication network structure including system architecture, communication paradigms and services as well as the related protocols and software; and
- Deciding the required operation voltages, power and dimensions of the fuel cell and solar panels
System breakdown, interface characterization and identification of potential bottlenecks
Starting with the initial decomposition of the overall scheme into systems sketched in the DoW, the consortium went deeper in defining every module and subsystem, appointing responsible partners in charge of every system, defining the computing structure and physical interfaces.
One important achievement was the identification of every piece of equipment to be provided by each participant and the relevant agreement on that. Finally, the consortium, in a General Assembly, approved a commitment to distribute the costs of any essential equipment missed in this identification (included in deliverable D1.1-“Technical requirements, specifications and project breakdown”) among the consortium members.

WP2 – MISSION MANAGER
Leader: CSIC
Participants: FTW, CY, UP, UCM, SAP, UPM, UF, CM
Duration (planned/actual): Month 7 to month 36/ Month 7 to month 36
Main S&T results:
The Mission Manager is divided into two modules: (a) The Mission Planner in charge of elaborating the mission strategy, and (b) the Mission Supervisor responsible for monitoring the mission progress. In addition, each one of these modules is broken down into modules related to aerial units and modules related to ground units (see Fig. 2.1).
Design of the Mission Planner

The Mission Planner was divided into two modules:
- The Aerial Mission Planner (AMP) related to the aerial/inspection mission, which determines the best path to cover a crop area by using a set of aerial vehicles
- The Ground Mission Planner (GMP) related to the treatment mission that, given a crop, determines the configuration of the set of ground vehicles (type and number of vehicles) as well as the path for each of the selected vehicles in order to accomplish the treatment task efficiently

Taking into account the specifications included in D1.1-“Technical requirements, specifications and project breakdown”, the following inputs for the Mission Planner were identified:
- The type of mission to be carried out
- The features of every available robot, and
- The main features of the mission field including previous information obtained about the field acquired by the aerial remote system, farmer's knowledge, etc.
With these data, the Mission Planner was configured to organize the distribution of units as well as the appropriated sequence of mobile unit actions to accomplish the mission. A detailed description of this module can be found in D2.1 – “Description of the Mission Planner“.
Design of the Mission Supervisor
The Mission Supervisor was split into two modules that are related to the supervision/monitoring of
- The aerial vehicles inspection mission and
- The ground vehicles inspection/treatment mission
both represented in Figure 2.1 by the modules AMS (Aerial Mission Supervisor) and GMS (Ground Mission Supervisor) respectively. Thus, the AMS supervises the aerial vehicles carrying out the inspection mission according to the inspection plan generated by the Aerial Mission Planner (AMP), whilst the GMS supervises the ground fleet during the treatment mission according to the plan generated by the Ground Mission Planner (GMP). A proper alarm management became an essential requirement in the supervisors' implementation. Thus the designed and developed supervision systems are able to detect different failures, dangerous situations such as vehicles out of theirs paths, inappropriate speeds, wrong states of implements, collisions among vehicles, etc. The supervision systems also detect important inflection points such as plan completed or mission over. More information about the supervisors’ performances as well as more implementation details are provided in deliverable D2.4 - “Mission Supervisor”.
Figure 2.1 also displays the connections among all the modules in the Base Station. In fact, we can see that the communication management of both supervisors and both planners with the rest of modules of the Base Station, mainly the Graphical User Interface (GUI), is conducted by an important module called Mission Manager Dispatcher (MMD). The MMD was devised to be in charge of controlling the workflow in the Base Station as well as the management of the communications among the modules in the Base Station. More information can be found in deliverable D2.5 (Mission Manager User Guide) along with other important issues such as (a) the messages that are exchanged among the modules in the Base Station and between the Base Station and other elements of the RHEA architecture (aerial and ground mobile units, and the user portable device; and (b) the global structure of the database used in the BS. In particular, the description of all the tables and their fields are provided in the document.
Other important report that explains the workflow of the Base Station is deliverable D7.4 – “Fully Functional Base Station” and it contains the format and syntax of the files used for data exchanging among Base Station modules.
Development of the Mission Planner
Ground Mission Planner
Several strategies (Simulated Annealing, Genetic Algorithms, NSGA-II) to achieve the most appropriate distribution of the ground units in the field (multi-path planning) were developed and tested. General situations were taken into account, for example a non-rectangular distribution of the crop. Furthermore, the constraints in the turning capability of the vehicles were considered, resulting in the analysis of different kind of manoeuvres and how these influence the efficiency of the robots plans.
Aerial Mission Planner:
Activities related with the development of the Mission Planner can be summarized as:
- Determination of altitude required considering camera parameters and spatial resolution requirements
- Automatic cell decomposition of the request field (delimited by poly-lines)
- Area assignment for two drones
- Optimal path planning for two drones without considering distances between them
Figure 2.2 shows the kinds of paths generated for the two Planners developed. Finally and related to the Mission Planner, the basic specifications for the design of a robotic weeding decision support system based on spatio-temporal information on weed infestations was provided. Detailed information is available in deliverable D2.3 – “Mission Planner”.
Development of the Mission Supervisor
Ground Mission Supervisor:
One of the most important results in the development of the Mission Manager for the Ground Mobile Units were the statement of the procedure for characterizing and treating the alarms in the Supervisor. An alarm is a message that a module sends to the operator or other related modules about faults or malfunctions detected in the system. These alarms are detailed in D2.2 – “Description of the Mission Supervisor”.
Aerial Mission Supervisor:
The main results achieved for supervising the aerial fleet can be summarized as:
- A fully operational interpreter to convert a general mission definition into AirRobot drone language was developed considering the AirRobot protocols. This includes initial setup, waypoint definition, safety border definition and mission commanding as well as full telemetry caption
- A software architecture was designed and developed in order to control two drones with the same hardware (PC) and performed flight supervision of the two drones
- Supervision rules were implemented
- Specification of the requirements of the aerial unit control for GUI was settled
A full mission definition and execution has been performed with a real drone in order to validate the Aerial Supervisor prototype.
Individual tests and assessment
The objectives of the Manager Manger were stated in qualitative terms. However, many simulation examples were performed to assess numerically the generation of missions achieving 100% of examples solved with 100% of field surface covered. These results along with the functionalities exhibited during tests and demos assessed positively the Mission Manager.

WP3 – PERCEPTION SYSTEM
Leader: IRS
Participants: CSIC, CV, UP, UCM, UPM, UF
Duration (planned/actual): Month 6 to month 36/ Month 7 to month 36
Main S&T results:
The Perception System was divided into Remote Perception System, Ground Perception System and obstacle/pedestrian Detection System. The main achievements can be divided into statement and development of every subsystem described below:
Statement of the Remote Perception System
The statement of the guidelines to accomplish a remote sensing system for weed detection was split into:
- The image processing requirements and
- Selection of the embedded imaging devices.
This part was devoted to the definition of the hardware and mechanical interfaces as well as the software related with mosaicking and image analysis.
Relevant details were reported in deliverable D3.1 – “Description of the Remote Sensing Equipment”.
Statement of the Ground Perception Equipment: camera based devices
Statements to accomplish a sensing system to be on-board ground robots for weed detection and navigation. These statements were split into:
- Specification of a Ground Sensing Equipment (camera-based devices) to perceive the environment for both optimal and safe navigation and to detect, with the requested accuracy, the natural elements to be treated. The specifications were focused on defining the equipment for vehicle’s guidance and safe navigation (obstacle detection), as well as on describing the techniques and algorithms to be implemented.
- Selection of the pertinent equipment and description of methods and algorithms. The selection was focused on the specific detection of weed in maize crops.

Significant details were reported in deliverable D3.2 – “Ground Sensing Equipment”.
Statement of the Laser based equipment for obstacle detection with ground robots
The technical requirements and selection of commercial Laser-based equipment to be used on Ground Mobile Units were stated. A state of the art on laser applications in agricultural tasks and the description of commercially available devices, prior to the final selection, were the main parts of this study, which was thus divided into:
- State of the art on laser applications in agriculture and description of commercially available devices
- Statement of technical requirements and election of equipment for obstacle detection in ground robots and reported in deliverable D3.3 – “Description of the laser-based ground sensing equipment”.
Development of a Remote Perception System
Three different activities were considered in the development of the Remote Perception System: (1) acquiring the images, (2) mosaicking the images and (3) extracting weed patches.
Image acquisition device:

Two approaches were studied:
- An approach based on a single SLR camera to get simultaneously the red and near-infrared bands
- An approach using two coupled twin cameras (one devoted to near-infrared)
This last solution was studied because:
- Standard colour images are required for simulators and Graphical User Interfaces (GUI)
- Practical trials with the single SLR approach have shown processing issues (due to Bayer matrix) reducing the effective spatial resolution
- A new high-end camera combining low foot-print (compact) and very high resolution with no Bayer matrix has appeared on the market (Sigma DP2 Merril)

Hardware integration of this twin-camera solution on an aerial unit was tested to evaluate the registration of visible and NIR images. After first trials using a feature point approach, it appeared to work only at very high spatial resolution. An alternative approach based on Fourier-Mellin transform was successfully developed and tested. This new approach consisted in partitioning the original images in a set of small image portions, and in identifying rotation, translation and scale change between all of them using Fourier spectrum analysis. A global homographic transformation model was then computed, including lens radial distortion. It was applied successfully on the images provided by the Sigma DP2 Merril coupled cameras.
Aerial image mosaicking:
An algorithmic approach for the registration of visible and near infrared coupled images was designed, implemented and tested. The bulk of the mosaicking process was based on an open-source package (MicMac, IGN, France) which was usually used through successive command lines under Linux OS. However, a new Windows version was implemented and its operation fully automatized. Additional software modules were developed and fully tested: picture synchronization with the flight log in order to manage the image processing inputs, automatic detection of ground targets for geo-referencing, communication with the base station, etc.
Weed patch detection:
Two Object-Based Image Analysis (OBIA) algorithms for weed patch detection in maize and wheat crops were successfully developed and tested on various real crop situations (multitemporal studies and different locations). The potential of generating bare soil and soil vegetation fraction (weeds and crop rows) maps as a first step in the accurate subsequent discrimination of weeds and crop rows was explored. Such an approach demonstrated the ability to discriminate accurately weed patches grown between crop rows to design a field program of site-specific weed management at early growth stage. Summarizing, the following performances were evaluated: i) different vegetation indices, ii) Otsu’s thresholding method for vegetation fraction mapping, and iii) the OBIA procedures for detection of wheat and maize crop-rows and weed patches using UAV imagery.
Development of the Ground Perception System: biological-object identification
A final Ground Perception System based on a SVS-VISTEK camera connected to a computer based on a National Instruments CompactRIO was fully designed and integrated in the ground mobile units (See Figs. 3.1b and 5.2). The system (hardware/software) was fully integrated with a High Level Decision Making System (HLDMS) and a mechanical and thermal implement for weed control in maize crops.
The equipment was tested at different stages and environmental conditions in the following scenarios:
- A road with four green painted lines, accurately spaced, simulating maize crops, for precision works
- A fake maize field made by planting wheat and simulating a maize field with crop rows spaced 75cm
- A real maize fields in May-July 2013 and October 2013
- A fake maize field, manually prepared from a wheat field (January-February 2014).
The ground sensing equipment was successfully validated for row-lines detection and inter-row weed densities computation in accordance with the requirements made for the mechanical and thermal implement. The row-detection was also used for precise guidance as a complement to the GPS-based guidance system. Additionally, the integration includes:
- Verification of correct overlapping between consecutive images and their corresponding areas in the real fields to cover the full space for a unique treatment, when required
- Synchronization between weeds detection and the lightning of burners
- Synchronization between row detection and the steering wheel system
Development of the Ground Perception System: 3D structure
Obstacle detection and tree structure characterization using laser and Kinect
A laser device for obstacle detection and emergency managing has been integrated in all ground vehicles (see Safety Systems, later). Its usage for tree volume measurement has been evaluated with bushes at different defoliation stages, and also with real olive trees. Kinect sensor results were promising for vegetation sensing, but finally it was discarded due to integration handicaps.
Obstacle and pedestrian detection using video cameras
Different approaches were developed and tested for obstacle and pedestrian detection during the course of the project. After developing and testing several algorithms, the following were assessed as the most appropriated for vehicles:
- Pedestrian detection
A detection system using Multiple Contours of Pedestrians was implemented. After several rounds of optimization and testing, this algorithm can now be used for detection of pedestrians. Furthermore based on the known intrinsic camera parameters the detection of pedestrians was mapped on the ground in 3D space. This algorithm was developed in order to fulfil the defined safety and warning zones in front of vehicles. The system in its final version is a very fast and efficient way for detecting the dedicated shape of humans.
- Obstacle detection using cluster centres
A novel colour clustering algorithms in combination with a normalized Euclidian distance calculation was used for detection of general objects. Three cluster centres are automatically calculated by a dedicated calibration software using images from cameras on vehicles. Object size and sensibility of the detection algorithm can be freely adjusted in the configuration. Again, the detected objects are mapped on the ground in 3D space.
Individual tests and assessment
Regarding the Remote Perception System and Weed Mapping System, the conducted tests revealed that about 95% of weed patches were discriminated with an average coefficient of Determination of R2=0.89 and a root mean square error of 0.02. However, note that the relationship of estimated versus observed weed densities strongly depends on every studied field.
Concerning the Weed Detection System, success ranging from 88% to 90% for a maize growth stage ranging from 30 cm-high to 5 cm-high, respectively, were achieved in the detection of weed patches. Thus, by performing the treatment in crops about 5 cm high (usual maize height during treatment), the system achieves the expected objective.

WP4 – ACTUATION SYSTEM
Leader: UP
Participants: CSIC, UCM, SAP, UPM, UF, BL, CM
Duration (planned/actual): Month 7 to month 36/ Month 7 to month 36
Main S&T results:
The Actuation System is in charge of acting on the crops for weed and pest control. For achieving this primary, this systems was divided into: the High-level Decision Making System, the Low-level Actuation System and the Device System (three different agricultural implements) for the crops selected in D1.1 - “Technical requirements, specifications and project breakdown”. Thus, the design, development, individual assessment, integration in the rest of the systems and overall assessment of these modules were the main activities performed in the Actuation System work package.
High-Level Decision Making System
The High Level Decision Making System (HLDMS) was devised as the main controller on-board the ground robots (Ground Mobile Units) in charge of elaborating commands for the Low-level Actuation System and the Device System with the information coming from the sensor systems, Mission Manager and the operator interfaces. Starting with the technical requirements and specifications stated in deliverable D1.1 – “Technical requirements, specifications and project breakdown”, a hardware and a software architectures for the High Level Decision Making System were stated, implemented and assessed based on a centralised system relaying on a National Instruments cRIO computer. In addition, the communication (in terms of commands, responses and related parameters) between the HLDMS and the rest of subsystems connected to it was defined and implemented. In particular, the communication between the HLDMS with: the Mission Manager, the Weed Detection System (Ground Perception System), the Guidance and Obstacle detection System, the Low-level Actuation System and the User Portable Device have been implemented and evaluated. The related definitions of commands, responses and their associated parameters were detailed in deliverable D4.1 - “Description of the HLDMS”.
Several versions of a HLDMS were tested ranging from that in which the HLDMS is a slave of the Mission Manager (the one used in the final demonstration) to the version in which the HLDMS of a given robot takes the control of a whole fleet. The implementation of the HLDMS was assessed as efficient and reliable.
Device System and Low Level Actuation System
The Device System (DS) was formed by the application tools (agricultural implements) and their relevant positioning systems. The Low-level Actuation System (LLAS) consists of the controllers of the Device System and it was divided into two subsystems: the Ground Mobile Unit Controller (GMUC) and the Actuation Controller (AC).
The GMUC, a PC-based computer located on-board the GMU, was devised to be in charge of controlling the vehicle and thus dedicated to interact with the vehicle devices (steering, speed controller, brakes, hydraulic pumps, etc.) through ISOBUS, communicate with the Location System, and the Actuator Controller. It was also devised to be in charge of making the vehicle to follow the trajectories specified by the HLDMS as a function of the information obtained from the Mission Manager, the operator interfaces and the sensorial information.
The AC, located on the implements, was in charge of the control of the different mechanisms that define the DS. It was based on a Programmable Logic Controller (PLC) to generate the required electrical signals to actuate all the devices involved in the application tools: nozzles, motors, valves, regulators, etc. The detailed definition of these elements was reported in deliverable D4.2 - “Description of the Low Level Actuation System” and details about their implementation were reported in D4.3 – “High-Level Decision Making System” and D4.4 – “Low-Level Actuation System”. The GMUC and LLAS were assessed positively with the three implements developed in the project.

New end-effectors and actuators for treatment application
Two implements were developed based on spraying techniques: (a) a spray boom for herbicides application on winter wheat and (b) an air-blast sprayer for olive tree canopy treatment.
The patch sprayer consisted of a typical boom 5.5 m wide with independently actuated nozzles spaced about 0.5 m (See Fig. 4.1a). The HLDMS can control those nozzles configuring an effective herbicide sprayer with a spatial transversal resolution of about 0.5 m. This development was particularized for wheat crops and its functionalities along with the integration features with the HLDMS were tested and positively assessed.
Both theoretical and experimental (laboratory and field) studies of agrochemical particle paths from injection into water flow to nozzle were conducted to determine the mechanical behaviour of the herbicide on the precise boom sprayer. The solenoid valve/nozzles response time was the time elapsed from the control pulse activation to the nozzle LED indicator’s photon emission (spray appears at the nozzle orifice). This time was calculated theoretically and experimentally by the use of the flow speed (m/s) in each of the sections within the prototype spray system. Traditionally, spraying was done at 200 L/ha, but when broad-spectrum contact herbicides or high levels of penetration are used it is recommended the use of higher volume (between 400 L/ha and 1000 L/ha). Three different scenarios were studied: low volume application (100 L/ha), standard volume application (200 L/ha) and high volume application (400 L/ha).
The canopy sprayer was based on two vertical stacks with four equal-separated, articulated nozzles each to apply pesticides in a vertical area ranging from 1.5 m to 4 m high (See Fig. 4.1b). The device can optimize the doses by adapting the nozzle orientation to the canopy sizes improving thus the pesticide management. The canopies were detected autonomously by using a sensing system based on ultrasonic sensors. This system was developed taking into account the particularities of the pesticide application in olive groves. This autonomous implement was checked in an olive grove fulfilling its initial requirements.

New tools for thermal and mechanical weed control
A weed removal tool was developed for weed control in maize. It consisted of a mechanical and a thermal tool 3 m wide with 4 couples of burners and mechanical end-effectors to tackle 4 crop rows simultaneously (See Fig. 4.1c). The mechanical tool consisted in shallow soil tillage actions performed by different end-effectors. These tools were characterized by static blades for intra-row weed removal. The thermal device was based on current robotic technology and consisted of a deployed mechanical structure to carry thermal tools based on rod burners fed by LPG for open flame or infrared applications (dry heat), steam and activated steam dispensers (humid heat).
Individual tests and assessment of each development
Realization of individual tests of the HLDMS and GMUC as well as many synchronization and integration tests with all the other related sub-systems (LLAS, DS) were assessed positively. Concerning the implements, the patch sprayer exhibited savings of 96.65% in the applied liquid for hectare for a field infested with weeds in about 3.24 % of the field. For an infested surface of about 10 % of the total surface, the saving falls to 90%. However, 75% (the objective target) is achieved for a weed infestation area smaller than 25% of the total area.
The canopy sprayer achieved an average reduction of the sprayed dose of about 50% in the test and demo field with olive trees smaller than conventional trees (the demo orchard was planted at the beginning of the project and the trees could not grow up to the expected stage). However, it is estimated to save over 50% in conventional plants (canopies about 3.5-m high).
Finally, experiments and a subsequent study on the effect of the Liquid Petroleum Gas (LPG) doses in maize crops and weeds for different growth stages concluded that the achievable weed killing rate can reach a weed reduction of more than 90% in terms of both density and biomass at harvest, obtaining maize yields similar or higher to yields treated with traditional methods.

WP5 – MOBILE UNITS
Leader: CNHi
Participants: CSIC, CV, UP, TRO, UPM, AR, UF, BL
Duration (planned/actual): Month 7 to month 36/ Month 7 to month 36
Main S&T results:
Configuration and development of the Ground Mobile Unit
The DoW specified the GMU to be based on small platforms tailored made for featuring a length of about 1.5 m, maximum speed of about 6 km/h, weight about 400 kg and payloads up to 350 kg. Since the first technical meeting, the participant in charge of providing these units pointed out the difficulty of devising new structures in a short time to let the rest of the participants perform tests and modifications with enough time to assure the success of the project. Furthermore, the first evaluations of the required GMU payloads to configure a profitable final system increased drastically the initial estimation. Based on these two drawbacks, the participant in charge proposed to adapt commercial vehicles as GMUs: a medium-sized tractor weighing about 1900 Kg with a maximum payload of 1600 kg. The Project Officer accepted the modification provided the initial scientific and technical project objectives were kept. Finally, the consortium adopted the medium-side vehicle as GMU for the three application scenarios (See Fig 5.1).
After a detailed analysis of the possible configurations and sizes of the GMUs, a decision was made for using a commercial agriculture vehicle chassis instead of a tailored-made robot structure featuring an engine of 51 HP, a weigh of about 1900 kg, dimensions of about 4 m long and 1.8 m wide, capable of carrying payloads up to 1600 kg, and equipped with 4×4 wheel drive, a 3-point hitch and a Power Take Off (PTO) connection.

The incidence of this consortium decision on the final results was identified as essential to achieve the project objectives and provided:
- Greater GMU reliability: they were vehicles tested and improved for a long time, while robotic prototypes are always prone to malfunctions and thus modifications during the development of the whole project
- Improved robustness: prototypes are also weak systems when facing different working conditions not taken into account during the design phase
- Easier standardization: the proposed vehicle already fulfilled a large number of standards and they are already homologated for many tasks
- Earlier availability: the delivery of the GMUs was advanced nine months with respect to the initial working plan allowing the participants in charge of the GMU controller to advance their developments
- Easiness for expanding/modifying the system: small mobile robots are designed with a very limited payload jeopardizing the inclusion of new subsystems; however, the proposed vehicle had a large payload/weight ratio allowing the designers to modify the system without the serious limitation of subsystem weight.
- Easiness for facing unforeseen problems: for example, those derived from the adaptation of the GMUs to specific requirements appearing from different subsystem, namely: the type and dimension of the GMU wheels and tyres, the length of the wheel shafts (to cover two or three crop rows underneath the GMU), extra payloads, etc.

Nevertheless, this solution presented the following drawbacks:
- Limitation in the GMU ground clearance: this feature was estimated in about 1 m in the DoW, while the proposed vehicles had a ground clearance of about 0.4 m. In any case, the 0.4-m ground clearance of the agricultural vehicle is enough for the application of weed control techniques, which are performed when the crops are short. The 1-m GMU clearance can be considered, indeed, an unrealistic feature estimation at the time of writing the proposal, and an over dimensioned requirement at the present time.
- Worsen safety: heavier vehicles carrying heavier loads make a mobile system more unsafe. This was the main drawback of the new proposal because safety was one of the main issues in the RHEA project. The only way to maintain safety in a reasonable level increasing the total mass of the system is to improve significantly the automatic safety systems. This was an important activity in the project (See the paragraph “Safety system of the Mobile Units” below).
A detailed description of the ground mobile units, definition of their dimensions, statement of their functionalities and features of their associated controllers, paying special attention to the communication between modules and the distribution of electronic controllers and drivers as well as the distribution of the supplied power were reported in deliverable D5.1 - “Ground Mobile Unit features and assessment measurements” and deliverable D5.5 – “Ground Mobile Units”. Figure 5.2 illustrates the distribution of subsystems on board a GMU.
Configuration and development of the Aerial Mobile Unit
The configuration of the aerial mobile units was devoted to define the features of a drone to meet the requirements stated in D1.1- “Technical requirements, specifications and project breakdown“. The final drone was envisaged as an extension of the commercial drone AirRobot AR100-B into a new prototype, AR200, featuring the project requested characteristics (See Fig. 5.3). This scaling-up work was done through an intermediate assessment step that used the demonstrator AR100-X6, which features a modular payload concept allowing the integration of heavier different payloads like commercial off-the-shelf video and photo cameras. That intermediate step defined two phases. In the first phase, several field tests with the AR100-X6 were performed that showed great drone stability while performing different flight manoeuvres; specially, hovering and slow flight in low altitude yielded an outstanding quality of the taken pictures and the real time video stream. These tests also verified that the AR100-X6 exhibits higher wind stability than the AR100-B. On basis of the experience with the AR100-X6, the efficiency and the characteristics of a coaxial rotor system were tested on a special engine test rig. The propulsion concepts were promising for the AR200, which was developed in the second phase.
The technical specifications of the Aerial Mobile Unit were gathered in deliverable D5.2 – “Aerial Mobile Unit features and assessment measure”, which includes the drone features that can be summarised as: diameter about 2.20 m; weight (main structure) of 4.5 kg; weight with a normal battery about 6.3 kg; weight with a battery of 16ah about 7.7 kg; maximum payload (10Ah battery) about 3 kg; maximum payload (16Ah battery) about 1.6 kg; flight duration around 40 minutes; and working distance greater than 5000 m.
The High Level Controller for the Aerial Units was developed in C++ on a windows platform so as to be executed on the base station computer integrated with the GUI. This controller aimed at driving the drones during their flight by using two USBs connections to their respective BKS consoles. The High level controller translates the mission definition created by the aerial planner (series of waypoints for each drone) to the specific protocol used by the drones, and therefore, it is in charge of create the right commands for the drones to execute the complete mission. This modular solution provided high flexibility in case of changes in drone protocols or even drones manufacturer showing high reliability during the tests.
The Aerial Unit High level controller was also in charge of performing real time alarm managing as well as obtaining the telemetry data from the aerial units and translating them to meaningful physical variables.

Safety system of the Mobile Units
The safety aspects regarding the mobile units, both ground and aerial, were evaluated for the subsequent development of their safety system. The activity was divided into three phases. In the first phase, a general study was carried out on current safety ISO and EN standards to find out possible connections between them. The second phase relied on the development of a specific design structure and a suitable strategy for RHEA. Finally, the work focused on every individual kind of vehicle, adjusting the safety requirements and specifications to their specific characteristics. In particular, the intrinsic risk associated to the GMU operations was analysed and a guide for carrying out the evaluation on potential risk and hazardous situations, including solutions and safety measures, was proposed.
The study of the current safety standards, general principle of design (ISO 12199), guidelines for risk assessment on ground and aerial units as well as recommendations and conclusions of the work carried out were summarized in D5.3 – “Safety system of the mobile units: definition and assessment measures”.
Regarding the specific modules developed and integrated in the ground units, three safety levels can be distinguished:
- Manual safety system: It was based on a PLC device in charge of brake activation and engine shut down control of the GMU when the emergency buttons placed on the GMUs or the emergency button on the GMU remote controller are pressed. This system does not depend of the main controller (HLDMS) actions and can act by its own. Note that after an engine shut down, a manual start up is required.
- Proximity safety system: It was based on a range finder (Laser) installed on the middle of the vehicle's front in a push–broom configuration (inclination) for the detection of obstacles along the vehicle trajectory (See Fig. 5.2). This subsystem was connected to the PLC of the Safety System to activate the brake and pause the engine or even shut it down if an obstacle is detected in the safety zone. Intensive tests were conducted to define the safety zones, of variable size depending on vehicle’s actual speed (dynamic size) and crop characteristics (fixed parameters). Also experiments were conducted to measure lasers’ precision and adjust their configuration to real agricultural working conditions.
- 3D obstacle detection system: Based on the data provided by the Ground Detection System. After analyzing those data, the information with the number of obstacle in the front of the robot and their positions are sent to the HLDMS in charge of making decisions (continue, halt, etc.). Those decisions can halt the vehicle and set on/off the brakes, but cannot shut down the GMU engine. A software tool for auto parameterization of obstacles was also made available (See Fig. 5.4).
Definition and Development of the alternative power systems for mobile units
Conventional fuel engines were envisaged for mobile units to ensure the energetic autonomy for reasonable periods. However, mobile units have been equipped with complementary energy sources: a fuel cell and a solar panel on-board every GMU. Thus, the GMUs were provided with a hybrid energy system including a fuel cell system, a solar panel and the tractors alternator, all of them charging 12 V batteries for providing power to the electronic equipment.
The fuel cell was based on a Proton exchange Membrane (PEM) fuel cell type, also known as Polymer exchange membrane fuel cell. This type of cell typically operates on pure (99.99%) hydrogen fuel. The PEM fuel cell combines the hydrogen fuel with the oxygen from the atmosphere to produce water, heat (up to 90°C) and electricity. PEM fuel cells are considered to have the highest energy density of all the fuel cells, and due to the nature of the reaction they have the quickest start up time (less than 1 sec) so they are favored for applications such as vehicles and portable power.
Hydrogen PEM Fuel Cell Systems are fueled with hydrogen, which can be stored in a variety of ways. Hydrogen storage in solids (or in Metal Hydrides) makes possible to store larger quantities of hydrogen in smaller volumes at low pressure and at temperatures close to room temperature. It is important to highlight the fact that Metal hydrides have the potential for reversible on-board hydrogen storage and release at low temperatures and pressures which is vital in our application. The final fuel cell developed features: PEM fuel cell power - 1.5 kW/12 V; charging current: >25 A; weight: 25 Kg. Hydrogen tank - 4 pieces; pressure: 10 bar; total capacity: 3 Nm3 (Normal Cubic Meter); weight: 32 kg. Battery pack - type: Pb Deep Cycle; Number of pieces: 2; total capacity: 360 Ah; weight: 80 kg. The placement of the fuel cell in the GMU is indicated in Fig. 5.2. Figure 5.5 shows the fuel cell components.
Solar panels generate electricity by converting the energy in photons into electric current. On the top of the tractors roof a solar panel was mounted providing continuous charging on the batteries during the daylight. There are different types and the selected for this application features: Type: Mono – crystalline; total power: 180 W/12V; charging current: 5.3 A; weight: 15 kg. The placement of the solar panels is illustrated in Fig 5.2.

Assessment of subsystem
The mobile units were assessed positively: the AMU based on a hex-rotor achieved to carry up to 3 kg (initial objective was 1.5 kg) with mission duration of about 40 minutes (initial objective was about 30 minutes) and the GMU based on a commercial chassis featured to carry the implements, work on slope greater that 15o (final feature 18o) and negotiate irregularities of about ±20 cm (many hours working on natural terrain assessed the GMU as ideal for the different applications).
The GMU Safety System consisted of the Safety System based on a laser and the Obstacle Detection System based on machine vision. The first subsystem was assessed as capable of detecting objects in the region of interest with 100% of succeed (along a very large number of experiments) while the second subsystem performed more than 98% of succeed in detecting persons and more than 93% in detecting general obstacles.
The energy hybrid system tested under laboratory conditions achieved high levels of efficiency (over 90% in total) reducing thus the environmental CO2 emissions.
WP6 – COMMUNICATION AND LOCALIZATION SYSTEMS
Leader: FTW
Participants: CSIC, SAP, UPM, AR, UF, CNHi, BL
Duration (planned/actual): Month 7 to month 36/ Month 7 to month 36

Main S&T results:
The RHEA communication system was structured in (a) the On-board Network, in charge of communicating the different computers and subsystems on-board the GMUs, and (b) the Fleet Network, based on a wireless network to communicate the robots in the fleet with the base station (Fleet Network). The localization system took care of the relative GPS position of the robots in the fleet with respect to the Base Station, where the GPS base antenna was located.
Communication Architecture and Infrastructure
A high-level communication system architecture was specified starting with the definition of the two different communication paradigms (publish/subscribe and request/reply) and services. Regarding the Fleet Network, a preliminary interface based on TCP/UDP messages or a Fleet Network API was described. Furthermore, the IP addressing scheme of the overall system was defined. In this way, it is possible to unambiguously address every component located on-board a Ground Mobile Unit (GMU) or on a remote GMU/Base Station. Regarding traffic prioritization, Differentiated Services for the On-Board Network and an 802.11e approach for the Fleet Network were chosen. Finally the architecture of the communication system software (Fleet Middleware) executed on a Linux-based Wireless Router was defined. The results of this task were documented in deliverable D6.1 – “Architecture and Infrastructures of the Communication and Location Systems”.
Regarding the communication and location infrastructure, the On-Board Network was built on a managed Ethernet switch (Moxa EDS-510A-3GT) and the Fleet Network on a Linux-based Wireless Router providing 802.11b/g 802.11a ZigBee PRO and GPRS interfaces. Preliminary tests with the equipment were performed focusing on the transmission delay and round trip time.

User Portable Device
A User Portable Device (UPD) was used by an operator in the field to interact with the fleet of robots. This device allowed the operator to be informed of the status of individual ground units and treatment implements as well as to take control of them if needed. A rugged tablet PC (model Algiz 7) equipped with GPS module was used as UPD. UPD location information was exchanged with the base station and the GMU in order to avoid crashing. Localization tests on the User Portable Device were performed by utilizing correction signal, RTCM 2.0 from two different sources: i) remote mount point where correction data are fetched over the internet and ii) the base station receiver which acts as a reference point. In both cases, NTRIP protocol was used to transfer correction signal. Position accuracy was measured in both, static and mobile scenario.
On-board Network Functions
The On-Board Network System, the physical connectivity of application computing platforms to the communication infrastructure and the required power supplies were specified in deliverable D6.4 - “On-Board Network System and Relevant Pre-Integration Tests”. In addition, all RHEA defined application commands were described and mapped on the On-Board Network communication prioritization scheme including the specification of the On-Board Network communication middleware which is to be linked and executed on the client (application) side. Services for transmission, asynchronous and synchronous reception of application commands and according On-Board Network API functions were defined and reported in the referred deliverable. Finally, the configuration and verification of the On-Board Network System, a test bed for system verification, pre-integration and integration tests were defined as well as a verification functionality to verify the RHEA application system after the integration phase were defined.

Localization System
For the localization system a GPS Trimble BX982 was evaluated and selected to be used in the Base Station (reference antenna) and on-board the Ground Mobile Units (rover antenna). The BX982 GNSS receiver was configured as an autonomous, reference station. This receiver offered centimetre-level accuracy based on RTK solutions and sub-meter accuracy code-phase solutions. Streamed outputs from the receiver provided detailed information, including the time, position, heading, quality assurance numbers, and the number of tracked satellites. The receiver also outputs a one pulse per second (1 PPS) strobe signal which lets remote devices precisely synchronize time. The main task of this GNSS receiver was to provide and transmit the GPS correction signal to each mobile unit, in order to determinate the location of the ground mobile unit accurately, getting centimetre accuracy on RTK-DGPS receivers and decimetre precision on DGPS receivers. A single connection to the BX982 via RS232, USB, Ethernet or CAN delivers both centimetre accurate positions and less than a tenth of a degree (2 meter baseline) heading accuracy.
At the base station cabin, which is located next to the mission field and equipped with a computer and software specially designed to plan and control the missions of the ground and flying units, the reference GPS-RTK receiver (BX982 GNSS) was strategically located next to the wireless router for future connections (Ethernet connection). The mounting location of the RTK-GPS antenna (Zephyr) on the base station allowed good satellite geometry to be obtained. The rover GPS-RTK receiver, with the heading information available, was located and fixed in the frame into the ground mobile unit (GMU).
For measuring the heading of the GMU, a double GPS antenna was used. A standard way to assess the accuracy, repeatability and reliability of the positioning system and a dedicated software tool was proposed to be used in order to ensure communication between the GPS devices with other systems. The results of this task were documented in the deliverable D6.3-“Description of the Localization functions and definitions of all pre-integration”.
Fleet network solution
Based on the requirements and system architecture defined in the deliverable D6.1 the fleet-network communication solution was designed and verified (See Fig. 6.1). The results in this activity include:
Design of the protocol between the High-Level-Decision-Making-System (HLDMS), Base Station (BS), User-Portable-Device (UPD) and Wireless Router (WR) components: The protocol defined a custom command frame structure used on top of TCP/IP (UDP/IP). As a result, the communication was highly portable to various platforms used by different partners of the RHEA consortium like LabView (HLDMS), Java (UPD), C++ (BS, WR).
Development of communication between GMUC and GMU: A gateway between the CAN bus of the GMU and the ISOBUS CAN was provided in the GMUs. The role of this gateway was to implement a handshaking protocol before allowing control of the GMU by the GMUC (An example of such protocol is described in the Annex C of the Part 7 of the ISOBUS standard). In order to initiate acceptance of GMUC commands by the GMU, the GMUC had to match its command value to the current status of the GMU before being allowed to send a new command or set point value. A LabView program simulating the GMU CAN communication and a gateway were provided to facilitate the development of the communication between the GMUC and the GMU.

Assessment
The communication system was not assessed quantitatively, but the large number of total hours giving service to the rest of subsystems assessed it as very positive. Regarding the Location System, the GNSS receiver showed a horizontal accuracy of 2.5 cm and a vertical accuracy of 3.7 cm on a continuous real-time basis, which assessed the system as proper for locating vehicles in agricultural tasks.

WP 7: BASE STATION AND GRAPHICAL USER INTERFACE
Leader: CY
Participants: CSIC, UPM, AR, UF, CNHi, CM
Duration (planned/actual): Month 7 to month 36/ Month 7 to month 36
Main S&T results:
Base Station configuration
Specification of the Base Station, including the model of the environment, the models of the robots, the models of the sensors and actuators and the models of the communication systems were defined. These specifications also described the functionality of the system from a user interface perspective and from an overall system perspective. All the relationships between the Base Station and the real world system were exhaustively reported in deliverable D7.1 - “Specifications of the Base Station”.
Graphical User Interface design and development
This activity was devoted to develop the Graphical User Interface where the model of the real system is represented in 3D and where the user can interact in different ways to obtain more information about what is currently going on with the real system (state variables, sensor measurements, camera images, etc.) or send commands to the system. The core of the GUI consisted of the commercial package Webots 7 robot simulator that computes physics and dynamics simulation and represents it in a 3D space. Another independent software module was developed in order to provide a single interface for the different partner’s modules, for example, the Aerial Units Mission Planner and the Ground Units Mission Planner. This additional program constituted the entry point for the human operators and allowed them to manage and set the parameters and the 3D environment to simulate or supervise. A more detailed description can be found in the document D7.2 – “Graphical User Interface prototype”.

Simulation of the operational setup
Some models of the real operational setup were developed to be able to reproduce the setup accurately in simulation. Those models included: AMUs, GMUs, actuation systems (a model of each implement) as well as a model of the field highlighting the treated area from non-treated area. To guarantee the best performance, the model of the field simply consisted on a 2D texture that can be applied to uneven virtual terrains. A more detailed description can be found in the report D7.2 – “Graphical User Interface prototype”.
For the aerial mission, the system can load and execute the plan from the file generated by the Aerial Mission Planner and represent accurately the drone trajectories and the area on the 3D space recorded by the AMU pictures.
Integration of the Base Station and GUI
The activity was focused on delivering a fully functional Base Station and GUI connected to the real system. All the network interfaces were implemented, connected to the real devices and tested through the Graphical User Interface. These include the implementation of the protocols defined in D7.1 - “Specifications of the Base Station” and the connection of the data received from the real system to the Graphical User Interface and simulation models documented in D7.2 – “Graphical User Interface Prototype” and D7.3 – “Simulation Models”, respectively. Different levels of tests were performed to assess the functionality, usability, accuracy, reliability and efficiency of the whole system. The relevant documentation was reported in D7.4 - “Graphical User Interface prototype”.
A particular attention was paid to monitor and control the GMU interface that is displayed during remote control and mission execution. It was really important to have a robust program able to handle variable communication delays and interruptions for testing purposes. The final version was a complete solution able to display all the status information and, at the same time, it allowed the operator to control the unit in a safe way. Figure 7.1 illustrates how the GUI looks like.

Assessment
The GUI was assessed qualitatively because of its nature. Along the intensive tests and demos conducted mainly in April and May 2014, the GUI achieved all expected functionalities and was positively assessed by the consortium.
WP 8: SYSTEM INTEGRATION AND PROJECT ASSESSMENT
Leader: CSIC
Participants: All
Duration (planned/actual): Month 25 to month 48/ Month 22 to month 48
Main S&T results:
Intermediate system integration and assessment
The activity regarding system integration was divided into several stages. The first one took place at the end of the second year and consisted in checking the system electro-mechanical components: mechanical interfaces, electric interfaces, equipment housed on-board the mobile units, location and attachment of cameras and sensor systems, power supplies, etc. Part of this integration was made at the GMU manufacturer (CNHi) facilities that provided the first GMU and part was made at AirRobot facilities to perform the related integration between the AMU and the Remote Perception System (IRS). Most of the partners shipped all subsystem to CNHi (Zedelgem, Belgium), including the Base Station and the three implements. An actuation plan (from May 2, 2012 to May 23, 2012) was stated in order to optimize the work. In this integration meeting, a number of problems were detected and solved in the following months.

The second stage of the integration process started in November 2012 (month 28), when the first two GMUs were delivered at the coordinator facilities (CSIC-CAR) and the partners involved in the GMUs joined together in Madrid to reintegrate GMU-1 and integrate GMU-2. Both GMUs remained fully integrated at the coordinator facilities to proceed with software integration and the preliminary tests. In addition, AirRobot sent the first AMU to the coordinator facilities to allow the partner in charge (UPM-EII) to fix the AMU controller. Some tests on real flights were conducted at that time. Since that date, the integration activities of the different partners at the coordinator facilities (CSIC-CAR) and the final demo facilities (CSIC-ICA) were intensive with an important dedication of the participants. Furthermore, the following integration tasks were performed on October 10-12, 2012 at CSIC-CAR facilities:
- Software update and testing of the GMUC (BL)
- Software update and testing of the GMU (CNHi)
- Software/Hardware update and testing of the communication system (FTW)
- Installation and checking of the energy packs (TRO)
- Installation and checking of the power distribution system (UPM-EIA and CSIC-CAR)
- Installation and checking of the safety system-laser (UPM-EIA)
- Installation of cameras and software and testing of the Ground Perception System (UCM)

These activities were complemented with the following integration activities
- Low-level actuation system in maize implement (UP and BL), Pisa, Italy, on October 10-12, 2012
- Low-level actuation system in wheat implement (SAP and BL), Seville, Spain, on December 3-5, 2012
Tests of the selected missions in real fields and assessment
In the period May 6-16, 2013, after the delivery of the 3rd GMU and the three developed implements, the consortium performed integration work at the coordinator facilities. For this period, CSIC-ICA cultivated a maize field of about 100 m × 40 m where the physical weed control implement were tested. This field was also used as the scenario to tune and assess the ground perception system and the row-tracking algorithms (UCM and CSIC-CAR). The thermal part of the physical implement was also checked (UP and CSIC-CAR). In addition, the following tasks were carried out:
- Installation and checking of subsystems on-board GMU-3 (CSIC-CAR, SAP, UCM, UPM-EIA, CV, BL, CNHi, TRO, FTW)
- Checking of implements and mechanical/electrical attachment to the GMU (UP, SAP, UF, CNHi, BL, UPM-EIA, CSIC-CAR)
- Integration of the aerial mobile units and remote perception system (AR, IRS, UPM-EII, CSIC-CAR, CSIC-IAS)
- Integration of subsystem in the Base Station (CSIC-CAR, CSIC-IAS, CY, SAP, FTW, IRS, UPM-EII)
This integration finished on May 15, 2013 with a general checking of the fleet performed by all the beneficiary representatives.

In October 2013, CSIC-ICA cultivated a new maize field at their facilities fulfilling the features demanded to a field to apply precision agriculture techniques. Many tests were conducted in this new field by UP and especially by UCM and CSIC-CAR. In addition, some integration activities were carried out not related with the maize field:
- Integration of the Aerial Mission Planer and Supervisor (CSIC-CAR and UPM-EII)
- Software integration in the Mission Manager (CSIC-IAS)
- Checking/improving the GMU (CNHi)
- Software installation and checking of the Obstacle Detection System(CV and CSIC-CAR)
- Checking/improving the Safety System (UPM-EIA and CNHi)

In November 2013, FTW performed intensive communication tests between the User Portable Device (UPD) and Base Station (BS), as well as between the UPD and a High Level Decision Making System in the real final scenario. Thus, FTW successfully tested the complete set of monitoring and controlling commands for Ground Mobile Unit (GMU) and three implements. At the same time, FTW and CSIC-CAR checked the first mission of a GMU in the olive grove. The communications were assessed robust enough for communicating the Base Station with a GMU placed 600-m away from the base station with no obstacles in between. Also, communication with a GMU inside the olive grove where tested successfully (Olive grove located about 100 m from the base station).
At the beginning of January 2014, CSIC-ICA cultivated wheat in a field to support the demo to be held on January 30, 2014 (See Fig. 1.3a). Since the beginning of January until the demo day, the following activities related with equipment integration were accomplished:
- Checking of the safety system (UPM-EIA and CNHi)
- Sprayer checking and integration in the GMU (SAP and CSIC-CAR)
- Integrating the GUI in the BS (CY and CSIC-CAR)
- Checking the implements and GMUC (BL and CSIC-CAR)
- Checking the maize implement (UP)
- Checking the canopy sprayer (UF)
- Checking the 3 GMUs (CNHi)
- Checking the obstacle detection system (CV and CSIC-CAR)

In April 2014, AR, UPM-EII, IRS, CY, and CSIC-CAR performed some tests with the AMUs at CSIC-CAR. The tests were successful; but some fine-tuning were performed and assessed. Finally, during May 2014, prior to the final demo, all the participants were active in defining, programming and performing the final demonstration held on May 21, 2014.

Redesign and reintegration of required subsystems
The preliminary integration activities allowed the consortium to perform improvements on the system. Since the first integration, the most important modifications were related with: the physical weed implement; communications; ultrasound sensor system of the canopy spraying implement; reinstallation of both doors in GMU-1 and software modifications; and redesign of some AMU subsystems (propulsion system, batteries, etc.).

Final project assessment
The final project assessment was made after the final demo held on May 21, 2014. The final assessment is reported in deliverable D8.5 – “Final Integrated System Assessment”. The consortium agreed that the main initial objectives of the project were achieved successfully.

WP 9: DISSEMINATION, EXPLOITATION AND TRAINING
Leader: SAP
Participants: CSIC, CV, CY, UP, UCM, TRO, UPM, R, UF, CNHi, BL, CM
Duration (planned/actual): Month 3 to month 48/ Month 3 to month 48
Main S&T results:

Project website management
The RHEA website was (http://www.rhea-project.eu) designed and launched in the first semester of the project. Ever since, the activity was devoted to the management and updating of the different sections as well as the uploading all the relevant information at the due time. This information includes the deliverables, the slides and videos of all the presentation given at the technical meetings and general assemblies. Thus, the RHEA website was a valuable repository of the material generated in the development of the project. In addition, all the material related with the project was included: demo videos, news in the media, proceedings of RHEA-2011 Workshop, proceedings of RHEA-2012 (http://www.rhea-conference.eu/2012/) and RHEA-2014 (http://www.rhea-conference.eu/2014/) conferences, disseminating material such as leaflets, posters, etc.
Intellectual Property Rights (IPR) handling
The Intellectual Property Rights among the partners of the RHEA consortium are defined in the Consortium Agreement signed by all partner representatives on June 29, 2010 (before the official outset of the project). The purpose of this Consortium Agreement was to specify with respect to the project the relationship among the partners, in particular concerning the organisation of the work between the partners, the management of the project and the rights and obligations of the partners concerning inter alia liability, access rights and dispute resolution. This document thus states the procedures to manage joint ownership, transfer of foreground, publications, access rights for implementations and access rights for use. Just a few IPR matters appeared in the RHEA project that were solved without affecting the development of the project and keeping the good mood among partners along the duration of the project.

Dissemination plan
The dissemination activities were reported in five deliverables (D9.1 to D9.5) with the general title of “Dissemination plan and Report (I to V)”. During the first year of the project two documents were issued. The first one (D9.1 - “Dissemination plan and Report (I)”) reported the results of the activities carried out in the first project semester to establish an identity for the RHEA project. Thus, the consortium designed a logotype and a graphical profile consisting of templates for presentations (MS–Powerpoint) and reports (MS–Word). In addition, a project leaflet and a poster were designed for quick dissemination of the launch of the project. By the end of the first six-month period (January 2011), the consortium published the first issue of the RHEA Newsletter (biannual publication) with the aim of keeping informed the participants and other interested people about the progress of the project. Deliverable D9.1 also included a list of the most relevant scientific journals and conferences to encourage the RHEA participant to disseminate the project results in those scenarios.
The activities in dissemination, along the second semester of the project, were reported in deliverable D9.2 - “Dissemination plan and Report (II)”. This document described some modifications on the RHEA dissemination tools, reported the interchange of student and scientists among participants, and listed the scientific and technical articles presented and published in international conferences (12), journals (1) and EC dissemination events (1). In addition, the second RHEA Newsletter was issued (July 2011) describing, in an easy way, the activities of the consortium and the project progress. RHEA Newsletters 2, originally written in English, was translated into Spanish and Italian to ease the dissemination, specifically among farmers. Versions of the RHEA Newsletters in these three languages were issued until the end of the project, distributed to a list of potential stakeholders created with inputs from all the project participants and made available at the RHEA website.
Regarding the second year of the project (August 2011 to July 2012), D9.3 - “Dissemination plan and Report (III)” informed about the dissemination of the project results in a similar format than previous reports. In this period, the consortium issued two Newsletters (Issue N. 3, January 2012; Issue N. 4, July 2012) and published a number of articles in conferences (21), scientific journals (3) and technical magazine (1) as well as 8 pieces of news in the Internet. In this period, the consortium also celebrated the RHEA2011 - First International Workshop on Robotics and Associated High Technologies and Equipment for Agriculture (ISBN: 978-84-615-6184-1) in Montpellier, France, hosted by IRSTEA on September 9, 2011 and attended by 36 participants. In addition, four short videos presenting (a) the major characteristics of the autonomous ground vehicle, (b) the fuel cell systems, (c) various simulations of the individual units and (d) the robot fleet spraying herbicides in a field were produced and made available at YouTube (URLs for these videos are available at deliverable D9.3).
The activities conducted from August 2012 to July 2013, third year of the project, are reported in deliverable D9.4 - “Dissemination plan and Report (IV)” and the achieved results can be pointed out as two new videos presenting several simulations of the robot fleet, equipped with different implements and conducting different missions that were produced and uploaded at YouTube as well as two pieces of news appeared in the Internet regarding the RHEA project (All those URLs are available in deliverable D9.4). In this period, Newsletter 5 (January 2013) and Newsletter 6 (July 2013) were produced and disseminated as usual through the RHEA website and the RHEA stakeholder distribution list.
An important dissemination event was RHEA2012 - The First International Conference on Robotics and Associated High Technologies and Equipment for Agriculture (http://www.rhea-conference.eu/2012/) held in Pisa, Italy, on September 19 to 21, 2012. The conference sponsored by the RHEA consortium was hosted by the University of Pisa and attended by more than 70 participants coming from 13 countries. The program included a total 53 oral communications about different aspects related with robotics applied to precision agriculture. The conference proceedings were published with ISBN 978-88-6741-021-7. During the third year of the project, a total of 18 communications were presented in different national and international conferences, 8 articles in scientific journal were published, 1 book chapter and 4 technical collaborations appeared in technical magazines. All these publications disseminated some of the project results at different levels and for different kinds of audiences.
Deliverable D9.5 - “Dissemination plan and Report (V)” put together the activity of the RHEA consortium along the fourth year of the project. During this year, the consortium produced three new videos about simulated RHEA missions that were uploaded to You Tube and several videos of real missions that took place in January and May (2014) available at the RHEA website. Moreover, Newsletter 7 (January 2014) was produced in the three usual languages and distributed. Apart from this activities related with dissemination in the Internet, the consortium was present in the social media network through Facebook (https://www.facebook.com/RheaProject) and Twitter (https://twitter.com/RHEAProject2014).
In November 2013, RHEA presented a stand in Hanover Fair (Agritechnica 2013), the largest fair of agricultural machinery in the world. The stand included a scale model of the ground unit and a real aerial unit, as well as videos, posters, leaflets and a press dossier. A lot of people and press representatives visited the stand and were interested in its content. As a result, a total of 23 articles have been published in the best specialized technical media of 13 different countries in Europe and North America. In February 2014, RHEA was again active in presenting a stand in the International Fair of agricultural machinery (FIMA), celebrated in Saragossa (Spain). The stand included a RHEA ground vehicle equipped with the specially designed patch sprayer. Numerous videos of the RHEA fleet in action and its individual components were presented.
During this last year, there were more than 30 pieces of news about different aspects related with the RHEA project disseminated to the media (TV, radio, newspapers and the Internet). References are available in D9.5. Regarding scientific and technical publications, the consortium published 38 communications to conferences (25 of them in RHEA2014), 14 articles in journals and 2 articles in technical magazines.

Besides this typical activity, the consortium was also involved in non-typical dissemination activities such as:
- CNHi was involved in the working group ISO/IEC TC 23/WG 15 for the creation on a new standard: ISO 18947- Safety of highly automated agricultural machines.
- FTW presented RHEA at the event “100 Years of the Experimental Farm of the Agricultural University in Vienna”. May 8th 2013 (Poster).
- SAP presented RHEA at the event “New challenges to improve eco-efficient agriculture”. Workshop at University of Seville (Spain), November 6, 2013.
- Cyberbotics was present with a booth at: a) IROS Conference + iREX exhibition, Tokyo (Japan, 3-8 November 2013 and b) Innorobo trade fair, Lyon (France), 19-21 March 2014.
- UPM-EIA presented a talk at the FIMA 2014 about the future of agriculture and the RHEA project., February 14, 2014.
- UPM-EIA is involved in the working groups ISO/IEC TC 23/WG 1 & 5 for the standardization of the electronic communication ISOBUS and wireless networks in agriculture.
The final demo of the RHEA project was held in Arganda del Rey, Madrid, Spain at CSIC-ICA facilities on May 21, 2014. This demo was intended to present the overall system as a project result to the EC representatives and it was planned as a part of the RHEA-2014 (Second International Conference on Robotics and associated High technologies and Equipment for Agriculture and forestry) with the aim of spreading the dissemination of the project results as much as possible and attracting as much attendees as possible. A total of 120 persons attended the event. Four different robot fleet missions were planned and executed: 1) weed patch spraying in a wheat field; 2) site-specific mechanical/thermal weed control in a maize field; 3) site-specific canopy spraying in an olive grove; and 4) Assessment of the safety system using laser systems and video processing systems.
As a final dissemination event to the scientific community, the RHEA consortium organized RHEA2014 - The Second International Conference on Robotics and Associated High Technologies and Equipment for Agriculture (http://www.rhea-conference.eu/2014/) that took place in Madrid, Spain, on May 21-23, 2014. The meeting was hosted by the Institute of Agricultural Sciences of CSIC (CSIC-ICA) and attended by 93 participants. The program included a total 62 oral communications on various aspects related with robotics applied to precision agriculture. A total of 25 presentations about RHEA results were presented at the conference. Other presentations (37) came from different projects and scientists all over the world. The conference proceedings were published with ISBN 978-84-697-0248-2.
The last scientific and demonstrative event in the project was about “The RHEA Project: results and acquisitions” and took place in San Piero a Grado (Pisa, Italy) on July 18th, 2014. The event was organized by the “Sezione Centro-Ovest of the Accademia dei Georgofili” in collaboration with the CiRAA “Enrico Avanzi” of the University of Pisa and attended by about 60 people including researchers, technicians, contractors and farmers.
Three lectures regarding (a) general information about the innovation achieved in the RHEA project, (b) the implement for Physical Weed Control (PWC) designed and realized by the University of Pisa and (c) the airblast sprayer developed by the University of Florence were given prior to a field demonstration of the functioning of the aforementioned operative machines. The PWC implement was exhibited in a maize field with artificial weed infestation showing the possibility to automatically change the LPG biological dose according to the weed cover. The airblast sprayer worked in a short rotation forest plant, allowing automatically changing the flow of active mixture of air, the speed of air and the orientation of the dispensers.
A final discussion about the future scenario of the management of the agricultural practices pointed out that it is moving towards a more and more intensive application of the principles of precision agriculture and use of sensors, automation and innovative systems of communication and actuation.

Exploitation plan
The main goal in this activity was focused on outlining the path towards the exploitation of the project results, which is reported in deliverable D9.12 - “Exploitation Plan (IV)”. This final document was an update of previous yearly exploitation reports (D9.9 to D9.11) issued at the end of every project year.
The first report (D9.9) issued at the end of the first year defined the structure of the exploitation documents and focused on analysing the market and identifying the technology trends. At the end of the second project year, the Exploitation Plan (D9.10) was delivered identifying some subsystems that could be used individually in agriculture that could also be adapted to other industries. These systems are at the forefront of technology and are of great interest for industries. The following potential subsystem were envisaged as exploitable at that time (middle of the project development):
- The Perception System
- The 3D vision algorithms for obstacle detection
- The Communication System
- The Graphic User Interface
- The Actuation Systems – three implements
- The alternative energy supply system
- The Mission Manager and
- The Safety System for autonomous agricultural vehicles
Regarding the exploitation of the whole system, RHEA was considered a complex system composed of different mobile units and several high-tech perception-action devices and, as a result, expensive. The RHEA system requires an intensive use to make the overall system economically feasible, which could be accomplished by using the "contractor approach". That means, a specialized company provides the service to individual growers. A prospective plan for placing the system in the market would be:
STEP 1: Solving the safety aspects for persons, crops and vehicles is of paramount importance because it can settle the intermediate steps between current tractors and fully autonomous tractors; however, we will still need to grow confidence on the technical level and on the perception of the end users, as well. Drastic changes are usually not well perceived by end users and a technology adapting period of about 5 to 7 years is to be expected to go from an ordinary tractor operation to a fully autonomous operation. In the beginning, tractors that have both a manual mode (still driveable by human) and autonomous mode are easier to transport to the different fields and can be used in manual mode for tasks that are still too complex to automate.
STEP 2: The tractors operate in fleets. One of the tractors in the fleet is still manned and the rest are not, but they can be supervised by the operator from the manned vehicle.
STEP 3: None of the vehicles in the fleet is manned but they are supervised by a person near the field in e.g. a truck/trailer that contains an onsite office. The same person loads/unloads the vehicles on site and takes them from one field to another one since the unmanned vehicles cannot go on the public road
STEP 4: The person in the truck is only there to get the robots transported over the public road from field to field and the supervision is done centrally by the office of the specialised company similar as a contractor.
In any case, a few RHEA partners are evaluating the commercial possibilities of the following subsystems:
Perception system (University Complutense of Madrid, UCM)

The first thing expected to be commercialized is the weed identifier. The development made on perception of crop with outside lightening condition can be used and adapted to a lot of other applications in forestry, for example, to identify trunks and leaves, estimate wood volumes, etc. Currently UCM is involved in the project entitled “Past, present and future of mountain forests: monitoring and modelling the effects of climate change and management on forest dynamics” for the period 2013-2017. The Spanish National Institute for Agriculture and Food Research and Technology is leading this project through the Forest Research Centre (CIFOR). The main activities are focused on image acquisition and processing based on a stereoscopic system patented by the CIFOR and equipped with fish-eye lenses. The aim is to determine 3D structures with the goal of determining trunk volumes and trees canopies for monitoring woodland changes and variations along the time. Texture classification-based approaches are also applied with identical purpose. The stereoscopic system works in unstructured environments, where the knowledge and background acquired during RHEA is being decisive.
3D vision algorithms for obstacle detection (CogVis)
The development made can be used in a large number of applications ranging from unmanned robots in general agricultural and forestry tasks to autonomous rovers in planetary exploration or vehicles in structured environments such as cars in roads. Manned vehicles also can benefit from obstacle avoidance system to assist the driver and reduce the risk of accident.
Communication and computer graphic interfaces (Cyberbotics)

The GUI to interface the user and the whole autonomous system with its simulation possibility, the feedback from the mobile units and the possibility to send new instructions can be adapted to other applications and other industries. In fact, the use of this kind of fleets of robots is growing in parallel with the need to interact with them. Fleets of aerial low-cost robots can provide a broad field of applications regarding inspection of large areas.
It is difficult and not very interesting to apply for software patents. But almost all the features and tools developed for interfacing and simulating the mobile units have already been exploited and included inside the latest distributed version of Cyberbotics' main product. This provided an added value for our customers and it improved the set of tools available to the company that were and will be used in other projects.
Actuation systems (University of Pisa, University of Florence and SAP)
New thermal and mechanical tools for weed removal in inter- and intra-rows were developed and tested in the project. New herbicide spraying techniques based on information from the perception system allowed an application of herbicide on only the necessary spots. That will drastically reduce the financial and environmental cost of these products in agriculture. Moreover, new olive trees canopy spraying technique will allow a more accurate application of product on canopy of trees and therefore reduce the waste of it. The institutions/company in charge of developing the Actuation System have expressed their interest in improving and exploiting these implements no matter if they finally protect them or not.
Safety of autonomous agricultural vehicles (Polytechnical University of Madrid – School of Agronomic Engineers)
Safety is a key point that has to be solved before the extension of autonomous agricultural prototypes to production models. New legislation has to be written on the liability in case of problem between the manufacturers and the user. The experience acquire with the RHEA project can help the legislative committees and consortium in this direction. New safety systems developed during the project could be used on commercial machines. There is a lack at the laser industry of a certified laser system for outdoor use (safety lidar) in agricultural environments, so here is a challenging possibility for R&D and possible patenting among RHEA partners (UPM-EIA, BlueBotics, etc.).
UPM-EIA is currently trying to apply the acquired knowledge on lidar to new developments of guidance and precise positioning of agricultural machinery inside olive groves. There is also planned a new collaboration (research stay at Germany) to develop algorithms for precise detection of vegetables combining laser and other technologies. The results of these planned projects will be analysed and combined with previous RHEA results, in order to develop exploitable applications.

Automatic mosaicking tool for VIS-NIR aerial images (IRSTEA)
In the frame of the RHEA project, a complete processing chain has been developed including synchronisation of camera pictures with flight log data, visible-near infrared registration, automatic detection of ground targets for georeferencing, mosaicking process, etc. This processing chain and its various components can be of great interest for any UAV service company dealing with aerial image acquisition for agricultural or environmental purpose. As a first step, it has been presented in a French seminar about UAV in June 2014 (http://drone.teledetection.fr/) where many private companies were attending and have confirmed their interest for such a tool. IRSTEA is presently looking for funds to recruit temporarily a computer science engineer, in order to achieve the conditioning of the software and propose it as an open source package. The building of a direct collaboration project with a specific UAV service company is also presently in progress (September 2014).

WP 10: MANAGEMENT
Leader/Participants: CSIC-CAR
Duration (planned/actual): Month 1 to month 48/ Month 1 to month 48
Main S&T results:
Project website management
The management of the RHEA project were planned to achieve the following objectives:
1.- To centralize and handle all project related matters with the Commission, being responsible for the timely availability of all reports, cost statements and deliverables,
2.- To coordinate and distribute responsibilities amongst persons and groups, decision process and information flow, as well as reporting,
3.- To release rules and methods allowing managing risks with respect to the general performance,
4.- To launch and maintain the project’s website.
To accomplish these objectives, the management activities were divided into three coordination activities with different goals and actions: financial and administrative, technical and scientific and strategic. That is,

FINANCIAL AND ADMINISTRATIVE:
- Communication between the consortium and the EC
The actions was based on the communication with the EC Project Officer and the EC Financial Officer
- Management and control of the project resources
The action relied on the Monitoring of the resources allocated for every planned activity
- Management and control of the financial and administrative resources
The actions consisted in the distribution of the advanced payment and assistance on administrative matters to the participants

TECHNICAL AND SCIENTIFIC:
- Assessment of the technical progress of the project development
The actions consisted in the permanent checking of the on-going tasks, interaction among tasks and identification of technical conflicts
- Support for the solution of conflicts, delays, and difficulties inside the WPs
The actions were based on a continuous contact and dialogue with the group leaders and S&T Board members
- Support for the communication among participants
The actions relied on continuous communications with all participants through e-mail lists (participant list, group leader list and S&T board list).
- Evaluation and approval of deliverables
The activity was focused on checking the contents of deliverables and quality of presentation
- Supervision of gender equity
The main action was to check continuously the gender balance in the project activities

STRATEGIC:
- Organization and control of technical/scientific meetings and General Assemblies
The activities were divided into (a) preparation of the agendas of the meetings and general assemblies; (b) issuing of the final minutes of the meetings and (c) organization and revision of the material presented in the meetings and assemblies to be made available at the project website.

The achievements of the project management along the development of the project can be summarised as:

FIRST REPORTING PERIOD:
- Convergence of all different ideas of the project participants about the RHEA project into a well-defined final system described in D1.1 – “Technical requirements, specifications and project breakdown”- The approval of this deliverable meant to achieve Milestone MS1.
- Agreement of the consortium on the equipment to be provided for the final system
- Commitment of the consortium about providing any equipment missed in the agreement on the final equipment
- Agreement on the system design which opens the development phase of the project (Milestone MS2)
- Management of initial delays to match the initial Work Programme at the end of this period
- Organization of four scientific and technical meetings and two General Assemblies

SECOND REPORTING PERIOD:
- Management and distribution of the first project payment
- Organization of the amendment to the budget to adjust the resources planned to the actual expended resources (Person months, travels, consumable material and durable equipment). The Project Officer accepted the proposed modifications without applying for a real amendment to the EC
- Organization of the Second Reporting Period and related justification
- Evaluation of nine deliverables
- Organization of six scientific and technical meetings and two General Assemblies
- Reorganization of meeting venues to optimize travel costs

THIRD REPORTING PERIOD:
- Management and distribution of the second project payment
- Organization of the amendment to the Grant Agreement to adjust the resources planned to the actual expended resources (Person months, travels, consumable material and durable equipment). The Project Officer suggested the consortium to apply officially for this amendment on August 6, 2013 in order to make official every small modification made to the budget
- Organization of an advanced Third Reporting Period and related justification two months before the end of the reporting period
- Evaluation of 38 deliverables
- Reorganization of meeting venues to optimize travel costs. After the consortium accepted to move all remaining meetings to Madrid, one more modification was agreed consisting in reducing the number of S&T meetings to allow the partners to come to Madrid for integration activities when really needed, maintaining the travel expenses
- Applying for permission to the Spanish Aerospace Safety Agency in order the consortium to be allowed to perform flights with the RHEA AMU during integration tasks and demonstrations. NOTE: we were warned that in Spain there was not any regulation for flying aircrafts under 25 Kg; thus, it was mandatory to apply for specific permission to the aforementioned agency.
- Handling the distribution of costs among partners willing to support the consortium attendance to AGRITECHNICA and FIMA fairs
- Organization of six scientific and technical meetings and four General Assemblies
- Management of the final project documents (3rd periodic report, final report, administration tables, FormCs, etc.)
- Organization of the Final Demonstration and Final Review Meeting

WP 11: FINAL DEMONSTRATION
Leader: CSIC
Participants: CV, CY, UP, UCM, TRO, SAP, UPM, AR, UF, IRS, CNHi, BL
Duration (planned/actual): Month 45 to month 48/ Month 45 to month 48
Main S&T results:
Final demo to the scientific and technical community and EC project officer
The final demo along with the final project review took place at CSIC-CAR and CSIC-ICA facilities on May 21, 2014. The three planned demos in wheat, maize and olive trees were carried out. Unfortunately, the aerial mission and the remote sensing tests were not performed because a sudden change in the Spanish regulation for drones (flights for drones were forbidden until the new regulation comes into force). However, the consortium checked the aerial missions and remote perception system on January, 30, 2014. The Project Technical Adviser attended that demo and the results on this subsystem are considered as project final tests.
The main activities can be summarised as
- Wheat field, maize field and olive grove preparation for tests and final demonstrations (Ready in May 2014);
- Demonstration of an aerial mission on a wheat field (January 2014);
- Demonstration of a patch spraying mission with one GMU (all subsystems involved) (January and May 2014);
- Demonstration of a 3-GMU mission faking a generic mission (all subsystems involved) (January and May 2014);
- Demonstration of the safety system (January and May 2014);
- Successful functioning of the rest of subsystems (Mission Manager, Remote Perception System, Ground Perception System, Actuation System, Communication and Location Systems, Graphical User Interface, etc.).


Potential Impact:
4.1.4 The potential impact
4.1.4.1 POTENTIAL IMPACT: SOCIO-ECONOMIC
Evaluating how the impact of the RHEA results can change the lives of the current and future EC community, several socio-economic important aspects such as demographics, employment and income levels, and quality of life of the community appear to be influenced. However, RHEA seems not to have effect in other socio-economic aspects as retail/service and housing market analyses, demand for public services as well as aesthetic quality of the community. Thus, the arguments of RHEA for socio-economic impact are:
Demographics:
RHEA provides high technology equipment that requires important teams of specialists for mission control and maintenance purposes. Engineers in electronics, communications and computer sciences as well as roboticists will be required around the application of RHEA fleets. These facts will force the creation of high technology jobs within the agricultural sector. These teams of specialists would cover the jobs shifted with the modernization of the field tasks and would contribute to sustain the number of inhabitants in rural areas. Moreover, robotization will completely release the farmers from physical exertion and this will let any human to access jobs traditionally reserved for healthy men regardless gender or physical disabilities.
Employment and income:
After automation and robotization of factories, the next step is undoubtedly the automation of farms. The advantages of automated farms over traditional farms are known and to reach a profitable automation of the first industry is a matter of time. Using perception and decision-making systems, automatic tools and vehicles will allow the agricultural processes to be more efficient in terms of time and money and will protect the environment; however, taking advantage of the stage of development in this new area in Europe will bring the opportunity of strengthening the leading position of our industry in this sector by advancing the commercialization of autonomous systems for agriculture worldwide. Definitely, this strategy will increase the occupation of European high-quality professionals as well as the personal incomes -especially in the countryside- of EU citizens. Moreover, the large number of different systems depending of very different technologies will allow the European SMEs involved to reinforce their market positions through high-tech specialization keeping the final assembly, commercialization and maintenance in large industries.
Quality of life:
RHEA reduces the amount of herbicide in about 75% to obtain the same production than traditional agriculture; then, RHEA drastically decreases the agrochemical residues on both environment and crops. Moreover, RHEA integrates new fuel cell and solar technology power systems into agriculture machinery attempting to reduce emissions. This reduction of inputs on the environment weakens the risk of adverse side effects on humans, animals, water and natural ecosystems. At the same time, the reduction of agrochemicals on crops diminishes the pre-market treatments of crops, thus also improving quality and marketability of harvested products with a strong influence on health.
The application of small robots as those in RHEA brings benefits on safety with respect to the current vehicles. A small robot diminishes the risk of leaving a big and powerful machine to wander free around the field. In addition, the RHEA system keeps the operator as a fleet supervisor in a comfortable cabin away from the field and the robots. This reduces the number of accidents in workers and thus improves the operator’s conditions at work. These aspects make the operator’s safety conditions to progress radically and also help to improve the quality of life of agriculture and forest workers.
4.1.4.2 POTENTIAL IMPACT: SOCIETAL IMPLICATIONS
The RHEA project has developed technologies that can help to increase productivity in the agricultural and forestry sectors protecting the environment, increasing employments and incomes to EU citizens, improving quality of lives for workers and people in general, maintaining inhabitants in rural areas, and diminishing the barriers to access jobs no matter the gender or disabilities of potential workers. Furthermore, the success of the RHEA project in terms of awareness and societal implications are indicated by the following results:
Prototypes:
The RHEA project has successfully developed, tested and assessed a fleet of heterogeneous robots (aerial and terrestrial) equipped with application tools for treatments in weed (mechanical and thermal tool, patch sprayer) and pest control (canopy sprayer) with a broad diffusion to farmers.
Real scenarios:
The RHEA system left the laboratory to perform missions in real scenarios under real working conditions. This has qualified the system as a truly working development by the society in general and the scientific community in particular.
True demonstrations:
The RHEA consortium finished the project with two demonstrations of the complete systems in real scenarios. The first one was held in January 2014 attended by the RHEA Project Technical Advisor. The final demo held in May 2014 was attended by the EC representatives (Project Technical Advisor and Project Officer), academia and industry members counting up to 120 attendees.
Integration of different technologies:
All project partners, academic, SMEs and large industries alike, coming from unrelated technological areas (robotics, agriculture, electronics, perception systems, location, decision making, new power systems, simulation, etc.) have actively contributed to the success of RHEA. This shows to society that integration of very dissimilar technologies can be done in a holistic manner at a reasonable cost.
Dissemination in the academia:
A large number of articles in scientific journals (20), papers in conferences and technical magazines (76) and chapters in books (2) were published so that our research has been made available to the society, and specially the scientific and technical community.
Dissemination in the society:
The RHEA project echoed in the media mass (TV, radio, press, Internet, etc.) disseminating research activities and results carried out with public funds from the EU.

4.1.4.3 MAIN DISSEMINATION ACTIVITIES
The RHEA consortium gave a significant importance to dissemination of results by designing a complete Work Package for Dissemination, exploitation and Training (WP-9). Thus, these activities were already detailed in Section 4.1.3 regarding to Description of the main S&T results/foregrounds of WP-9 and can be summarised as following:
Project website:
The RHEA website (http://www.rhea-project.eu) has been an important and the first mean of communication for the society. It has been used for disseminating news, results and activities; but it has also been a crucial tool for exchanging information among members in the consortium and an essential repository of the material produced in the development of the project. Moreover, material related with the project such as demo videos, news appeared in the media, edited proceedings of RHEA-2011 Workshop, RHEA-2012 and RHEA-2014 conferences and disseminating material (leaflets, posters, etc.) has been made available at the RHEA website.
General information:
A biannual newsletter (RHEA Newsletter) were edited in three languages (English, Italian and Spanish), disseminated through e-mail lists and made available at the RHEA website. A large number of leaflets with information on the description of the RHEA project and objectives, the international conferences organized in the development of the project as well as publicity of the final demos were distributed in conferences, workshops and fairs.

Scientific and technical results:
The RHEA partners have disseminated the project results in both contributed and invited presentations at important workshops and conferences:
- The RHEA Project; 5th Conference on the FP7 of the European Union in Spain (in Spanish); KURSAAL, San Sebastian, July 20-21, 2011, (CSIC-CAR).
- Robot fleets for highly effective agriculture and forestry management; 5th Workshop of the MANUFUTURE AET-Community – Agricultural Engineering Strategies for HORIZON 2020; Convention Center (CC), Deutsche Messe AG, Hannover, Germany, November 11, 2011, (CSIC-CAR).
- Robot fleet for agriculture; The Spanish Summit of Unmanned Vehicles Exhibition (UNVEX-12); Auditorium Hotel, Madrid, April 23-25, 2012, (CSIC-CAR).
- Robots versus pests; Global Forum for Innovations in Agriculture, Abu Dhabi, February 3-5, 2014, (CSIC-CAR).
- The future of agriculture and the RHEA project, FIMA 2014, February 14, 2014, (UPM-EIA).

In addition, the RHEA consortium disseminated videos through You Tube video-sharing website and registered profiles in Twitter (https://twitter.com/RHEAProject2014) and Facebook (https://www.facebook.com/RheaProject) online social networking services.
Organization of international conferences:
The RHEA consortium has organized two important disseminations events under the common name of RHEA: “International Conference on Robotics and Associated High-technologies and Equipment for Agriculture and Forestry” that served as a meeting point for discussions with colleagues from other consortia and locations and helped to disseminate the project results. The First RHEA international conference (RHEA-2012) took place in Pisa, Italy, and was devoted to “Applications of automated systems and robotics for crop protection in sustainable precision agriculture”, while the Second RHEA international conference (RHEA-2014) was held in Madrid, Spain, focused on “New trends in mobile robotics, perception and actuation for agriculture and forestry”. Many hard copies of the proceedings have been distributed and the electronic version made available at the RHEA website.
Participation in international Fairs:
The RHEA consortium has disseminated results in two important international fairs. In Agritechnica 2013 (Hanover, Germany, November 10-14, 2013) a scaled model of the RHEA ground mobile unit and a real aerial mobile unit were exhibited along with videos and posters. Additionally leaflets and press dossiers were distributed to visitors. More than 140 people from universities (14%), machinery manufactures (15%), journalists (13%) and general public (58%) assisted at the RHEA booth and up to 23 articles have been identified in specialized technical publications from 13 different countries. In February 10-15, 2014, the RHEA consortium was present in the International Fair of Agricultural Machinery (FIMA) held in Saragossa, Spain. The stand included a RHEA ground mobile unit equipped with the patch sprayer developed in the project. The project results were illustrated with videos of the RHEA fleet in action including individual subsystems and components. The participation in FIMA has provided direct promotion in the fair official catalogue and on the fair website passing informative promotion to over 400 worldwide media (See fair booths in Fig. 9.1).
Demonstrations and exhibitions:
One important dissemination event in the RHEA project was the final demonstration that took place at CSIC-ICA facilities (Arganda del Rey, Madrid, Spain) on May 21, 2014 prior to the Final Project Review. This demo presented the overall system as a project result to the EC representatives, academia, industry and local farmers. In the afternoon, the consortium gave a training seminar to non-RHEA partner attendees to illustrate details of the fleet components and perform additional demos. See Fig. 1 for a general view of the two demonstrations.
A post-final demonstration of the LPG implement and the canopy sprayer was organized at San Piero a Grado (Pisa, Italy) for Italian researchers, technicians, contractors and farmers.

4.1.4.4 MAIN ACTIVITIES IN THE EXPLOITATION OF RESULTS
The project exploitation plan was carried out as a main activity in Work Package 9 where a prospective exploitation plan for the complete fleet was outlined (See Section 4.1.3). This plan was assessed as very complex to achieve; however, the exploitation of some intermediate results, subsystems and components could be made available at the market for specific applications. To stimulate the exploitation of those project results, the consortium organised two important events: an Exploitation Strategy Seminars (ESS) and a lecture on IPR and patent applications.
Exploitation Strategy Seminars:
The ESS, held in Seville on November 11, 2012 was suggested by the RHEA project officer to the consortium and taught by an EC ESS expert. During this event, the RHEA consortium learnt and discussed about (a) understanding Intellectual Capital, (b) valorisation and Value of Intellectual Capital, and (c) defining key exploitable results and IP related issues. Some potential individual results (reported in Section 4.1.3) were identified as exploitable.
Lecture on patenting activities and CNHi proposal:
In August 27, 2014, during the S&T meeting in Arnsberg, Germany, CNH Industrial (CNHi) organized a lecture on patent application given by a CNHi patent attorney to stimulate patent application in the RHEA project. Patents ensure some protection to the innovations developed by the partners during the project and give competitive advantages in case of commercialization by excluding the use of the inventions in commercial products by the competitors. Another important aspect regarding commercialization is to ensure that other patents are not blocking the commercialization of the system currently developed. This is linked to another advantage of applying for a patent: at the beginning of the patenting process, there is a patents search investigating if similar inventions have not already been patented and, thus, tries to ensure that the system using the invention is free for commercialization.
Nevertheless, the RHEA members have not applied for any patent so far. The reasons seem to be the heavy cost of a patent application, the lack of infrastructure and resources from companies and institutions to support patent application and the difficulties to define every partner’s rights in developments normally carried out by many partners. To relieve these drawbacks, the CNHi patent department offered their infrastructure to patent the inventions developed by the consortium that are relevant for CNHi. This would require further IPR negotiation, but one possibility is that the partner developing the invention stays the only owner of the IP but CNHi, in exchange of carrying the work and cost related to the patenting process, is granted the freedom and exclusivity to use this IP in the agricultural field while the partner can value his invention in other application fields. In any case, a few partners are still evaluating the application of a patent for the following subsystems: (See Section 4.1.3 for additional details).
Prospective individual exploitable system:

The potential subsystems under patent/exploitation consideration are (as advanced in Section 4.1.3-WP9):
- Perception system (University Complutense of Madrid)
- 3D vision algorithms for obstacle detection (CogVis)
- Communication and computer graphic interfaces(Cyberbotics)
- Actuation systems (University of Pisa, University of Florence and SAP)
- Safety of autonomous agricultural vehicles and precise local positioning (Polytechnical University of Madrid – School of Agronomic Engineers)
- Automatic mosaicking tool for VIS-NIR aerial images (IRSTEA)


List of Websites:

http://www.rhea-project.eu/

final1-5-figures-and-tablesl.pdf

Documentos relacionados

• final1-5-figures-and-tablesl.pdf