Final Report Summary - CONMAR (Cognitive robotics: cooperative control and navigation of multiple marine robots for assisted human diving operations)
Navigation plays a key role in marine robotics and is currently the subject of intensive research worldwide, both from a theoretical and practical standpoint. This is especially true in underwater scenarios, where autonomous marine vehicles must travel over large distances to acquire scientific ocean data at unprecedented scales, while being able to geo-reference the data acquired by resorting to advanced navigation systems. Navigation, in this context, refers to the task of estimating the position of a vehicle underwater. In a broader context, it refers to the demanding task of estimating its position and pose, as well as the corresponding velocities. The complexity of the problem at hand stems essentially from the fact that submerged vehicles cannot have access to absolute navigation data such as those provided by the global positioning system (GPS). In fact, in many operational scenarios the only navigation data accessible are those acquired by on-board resident sensor units that may include an attitude and heading reference system (AHRS) and a Doppler velocity logs (DVL). Affordable, classical navigation systems that rely solely on these data will necessarily exhibit errors that grow with time, and for this reason they must be complemented with external aids. The latter may include GPS position fixes at the sea surface (if the vehicle is allowed to re-surface periodically) or even underwater by resorting to acoustic systems capable of estimating the position of the vehicle with respect to a set of transponders or with respect to a companion support ship (which must then transmit the estimated position of the underwater vehicle to the vehicle itself using an acoustic modem).
Clearly, acoustics play a key role in the above process. It is at this point that the specific conditions of the water medium come to the fore, for they impose extremely stringent limitations on the bandwidth of the acoustic transmission channel and are at the root of temporary transmission losses. Clearly, much work remains to be done before a new breed of advanced, cost effective navigation systems become available. This justifies the surge of activity on the development of different types of navigation methods that include single beacon navigation, terrain-based navigation, geomagnetic-based navigation, and navigation by resorting to selected reference objects, to name but a few. Also important and extremely actual is the concept of cooperative navigation, where multiple vehicles share their navigational resources to collectively increase the performance of their individual navigation systems. It can be stated that the research field of marine robot navigation offers a wide variety of methods to explore, but is yet far from offering a unique, generic, cost-effective solution with high accuracy. The challenges are formidable, and there is ample room for advanced research and development, namely in the field of cognitive robotics where humans and robots must co-exist and cooperate in the execution of demanding tasks. Navigation problems related to the latter (as well as the problems that arise in the process of coordinating the motion of humans and robots) are still at their infancy. This seemingly simple observation was at the root of the research project entitled 'Cognitive robotics: cooperative control and navigation of multiple marine robots for assisted human diving operations' (CONMAR) submitted for funding in the scope of the 'Marie Curie actions - Intra-European Fellowships' (FP7-PEOPLE-2009-IEF) of the Seventh Framework Programme (FP7), to afford the fellow researcher Dr Thomas Glotzbach, (thomas.glotzbach@tu-ilmenau.de via e-mail) the possibility to carry out research in the abovementioned area at the host institution Instituto Superior Técnico, Lisbon, Portugal, http://www.isr.ist.utl.pt under the supervison of Prof. António Pascoal (antonio@isr.ist.utl.pt via e-mail).
The project aimed to contribute to the area of cooperative navigation in marine environments, with special emphasis on mission scenarios with human divers in the loop. Namely, the objective was to develop a tracking system to estimate the position of a human diver (carrying an acoustic device for range measurements and communications) using a small group of autonomous surface vehicles capable of measuring their distances to diver and estimating his / her current position, as well as the corresponding velocity and course angle. The tasks addressed in the project were motivated by a larger mission concept whereby the vehicles are asked to supervise the diver and guide him / her along a pre-planned path by issuing heading commands that the diver must follow. The commands are sent via the acoustic channel and the diver receives them by examining the light patterns on light emitting diode (LED)s installed on his / her diving mask. This overall concept required the systems studied in the scope of CONMAR to be robust with respect to temporary communication losses between the diver and the surface craft and also with respect to acoustic outliers.
In the course of CONMAR, a tracker system was developed as a key element for the execution of a diver-surface vehicle mission. The tracker resorts to an extended Kalman filter (EKF) and sought inspiration from the concept of GPS intelligent buoys (GIB), a commercially available system in which several passive surface buoys receive acoustic pings transmitted by an underwater target at regular time intervals and collectively compute the position of the target. In striking contrast to what happens with the GIB system, however, in the mission scenario at the core of CONMAR it is not possible to assume that the clocks of the surface vehicles and that of the computer hardware carried by the diver (responsible for initiating the transmission of acoustic pings) are synchronised. To overcome this problem, the communication cycle must be initialised by one of the surface vehicles, by sending a reference ping that is answered by the acoustic modem carried by the diver to all surface vehicles. Therefore, the underlying measurement model had to be adapted to the specific situation at hand. Two different measurement models were developed and validated in simulation:
(i) a so-called simple model and
(ii) an advanced one.
For the simple model, the drawback is that if the ping emitted by the target is not received by the surface vehicle in charge of sending the reference ping, the measurements from the other surface vehicles cannot be used in the corresponding interrogation cycle. This is clearly a waste of the communication resources and a violation of the robustness requirements underlying the development of such a system. Using the advanced measurement model, all available measurements can be used for estimation purposes. In the presence of temporary communication losses, this approach yields far better performance. This was verified extensively in simulations using a specially developed software tool that allows the user to emulate mission scenarios involving human divers and robots in the loop. The tool will become an integral part of NetMarSyS, a software suite for the simulation of cooperative multiple vehicle navigation and control systems developed at the host institution IST.
Central to the implementation of a system capable of estimating the position of a target underwater (of which a diver is an example) is the correct placement of the companion surface vehicles. To study this issue, and in parallel with the task described above, research was done on the problem of optimal sensor placement for target positioning in three dimensions, with the restriction that the sensors be placed on a plane (the water surface). The work generalised to a three-dimensional (3D) setting results available for the localisation of wheeled robots in two dimensions. Inspired by previous work done on this subject at IST, the problem was cast in the form of an estimation theoretical problem that involved the computation of so-called Cramér-Rao-Bounds to compute the best positioning performance that can be achieved with any unbiased target position estimator. The results obtained lend themselves to intuitive physical explanation. For example, in the case where the variance of the range measurements increases with distance, there is an optimal value for the radius of the circumference where the sensor must be located, with the centre directly above the underwater target. Extensive simulations proved useful in assessing the efficacy of the algorithm developed for optimal sensor placement.
The project culminated with the execution of sea trials in Lisbon, Portugal with three autonomous surface robots and a human diver. The experimental set-up and the human resources needed were made available through the Dynamical Systems and Ocean Robotics Laboratory of ISR/IST. The tests showed clearly that the systems developed for underwater target positioning and cooperative motion control (of the diver and the companion robots) work extremely well under real conditions and hold considerable promise for the development of future robotics systems to assist human divers during demanding commercial and scientific missions.
In conclusion, CONMAR contributed to the challenging task of developing navigation methods for autonomous marine robots. Within its defined area of application, it effectively contributed to the development of a prototype of a tracking system to assist human divers that has good potential to further be enhanced and witness the transition from the laboratory to the real world. Systems enabling extensive interaction between humans and marine robots are still at their infancy, but are expected to play an ever increasing role in the future. In this respect, the contribution of CONMAR (complemented by contributions done by the hosting group at ISR / IST) can be described as a pioneering effort. Additionally, some of the results can certainly be transferred to groups interested in marine robotic applications and are therefore of interest to all groups operating robots underwater, e.g. offshore industry, or marine scientists. The methods advanced (and proven in practice) offer a reliable and robust navigation solution for a number of scenarios involving marine robots and human divers. The theoretical research done will boost further investigation in this challenging and promising area with the objective of developing increasingly robust and reliable navigation solutions for applications in a vast number of mission scenarios. Future research activities will aim at extending the results obtained to deal with multiple vehicles and humans underwater, a situation that calls for the development of systems that will effectively afford divers access to robot 'buddies' to cooperatively perform missions at sea.
One of the central objectives of the Marie Curie actions is also to train the fellow researcher engaged and to boost his / her abilities to carry out independent scientific work. In what concerns the present researcher, he could definitely broaden his research horizons in topics such as dynamical systems theory and cooperative navigation and control of maritime robots. At the same time, he became familiar with the steps required to transition from the laboratory to the real world, effectively witnessing the different phases of system design, simulation, implementation, and testing. After having completed the CONMAR project, he was employed by Ilmenau University of Technology and given a position of senior researcher. In the meantime, he is preparing his application for a postdoctoral lecture qualification (the German Habilitation). The research group that he is supervising at Illmenau deals with mission planning and navigation of marine robots. In this respect, the experience that he acquired during his stay in Lisbon, Portugal has proven extremely valuable, for it has allowed him to capitalise on his experience during his fellowship both in terms of scientific knowledge and soft skills (such as project management) as well as planning and execution of sea trials. Additionally, he is involved in teaching at the university. It is scheduled for him to become responsible for giving lectures in modelling / process analyses from 2013 or 2014 on and to organise a new course on Estimation and Decision that will rely heavily on the knowledge that he acquired during his fellowship. Additionally, the cooperation between the host institution, the Instituto Superior Técnico, and Ilmenau University of Technology, has been strengthened; this is clearly demonstrated by the fact that both institutions are partners in the new European research project MORPH (FP7-288704).
Clearly, acoustics play a key role in the above process. It is at this point that the specific conditions of the water medium come to the fore, for they impose extremely stringent limitations on the bandwidth of the acoustic transmission channel and are at the root of temporary transmission losses. Clearly, much work remains to be done before a new breed of advanced, cost effective navigation systems become available. This justifies the surge of activity on the development of different types of navigation methods that include single beacon navigation, terrain-based navigation, geomagnetic-based navigation, and navigation by resorting to selected reference objects, to name but a few. Also important and extremely actual is the concept of cooperative navigation, where multiple vehicles share their navigational resources to collectively increase the performance of their individual navigation systems. It can be stated that the research field of marine robot navigation offers a wide variety of methods to explore, but is yet far from offering a unique, generic, cost-effective solution with high accuracy. The challenges are formidable, and there is ample room for advanced research and development, namely in the field of cognitive robotics where humans and robots must co-exist and cooperate in the execution of demanding tasks. Navigation problems related to the latter (as well as the problems that arise in the process of coordinating the motion of humans and robots) are still at their infancy. This seemingly simple observation was at the root of the research project entitled 'Cognitive robotics: cooperative control and navigation of multiple marine robots for assisted human diving operations' (CONMAR) submitted for funding in the scope of the 'Marie Curie actions - Intra-European Fellowships' (FP7-PEOPLE-2009-IEF) of the Seventh Framework Programme (FP7), to afford the fellow researcher Dr Thomas Glotzbach, (thomas.glotzbach@tu-ilmenau.de via e-mail) the possibility to carry out research in the abovementioned area at the host institution Instituto Superior Técnico, Lisbon, Portugal, http://www.isr.ist.utl.pt under the supervison of Prof. António Pascoal (antonio@isr.ist.utl.pt via e-mail).
The project aimed to contribute to the area of cooperative navigation in marine environments, with special emphasis on mission scenarios with human divers in the loop. Namely, the objective was to develop a tracking system to estimate the position of a human diver (carrying an acoustic device for range measurements and communications) using a small group of autonomous surface vehicles capable of measuring their distances to diver and estimating his / her current position, as well as the corresponding velocity and course angle. The tasks addressed in the project were motivated by a larger mission concept whereby the vehicles are asked to supervise the diver and guide him / her along a pre-planned path by issuing heading commands that the diver must follow. The commands are sent via the acoustic channel and the diver receives them by examining the light patterns on light emitting diode (LED)s installed on his / her diving mask. This overall concept required the systems studied in the scope of CONMAR to be robust with respect to temporary communication losses between the diver and the surface craft and also with respect to acoustic outliers.
In the course of CONMAR, a tracker system was developed as a key element for the execution of a diver-surface vehicle mission. The tracker resorts to an extended Kalman filter (EKF) and sought inspiration from the concept of GPS intelligent buoys (GIB), a commercially available system in which several passive surface buoys receive acoustic pings transmitted by an underwater target at regular time intervals and collectively compute the position of the target. In striking contrast to what happens with the GIB system, however, in the mission scenario at the core of CONMAR it is not possible to assume that the clocks of the surface vehicles and that of the computer hardware carried by the diver (responsible for initiating the transmission of acoustic pings) are synchronised. To overcome this problem, the communication cycle must be initialised by one of the surface vehicles, by sending a reference ping that is answered by the acoustic modem carried by the diver to all surface vehicles. Therefore, the underlying measurement model had to be adapted to the specific situation at hand. Two different measurement models were developed and validated in simulation:
(i) a so-called simple model and
(ii) an advanced one.
For the simple model, the drawback is that if the ping emitted by the target is not received by the surface vehicle in charge of sending the reference ping, the measurements from the other surface vehicles cannot be used in the corresponding interrogation cycle. This is clearly a waste of the communication resources and a violation of the robustness requirements underlying the development of such a system. Using the advanced measurement model, all available measurements can be used for estimation purposes. In the presence of temporary communication losses, this approach yields far better performance. This was verified extensively in simulations using a specially developed software tool that allows the user to emulate mission scenarios involving human divers and robots in the loop. The tool will become an integral part of NetMarSyS, a software suite for the simulation of cooperative multiple vehicle navigation and control systems developed at the host institution IST.
Central to the implementation of a system capable of estimating the position of a target underwater (of which a diver is an example) is the correct placement of the companion surface vehicles. To study this issue, and in parallel with the task described above, research was done on the problem of optimal sensor placement for target positioning in three dimensions, with the restriction that the sensors be placed on a plane (the water surface). The work generalised to a three-dimensional (3D) setting results available for the localisation of wheeled robots in two dimensions. Inspired by previous work done on this subject at IST, the problem was cast in the form of an estimation theoretical problem that involved the computation of so-called Cramér-Rao-Bounds to compute the best positioning performance that can be achieved with any unbiased target position estimator. The results obtained lend themselves to intuitive physical explanation. For example, in the case where the variance of the range measurements increases with distance, there is an optimal value for the radius of the circumference where the sensor must be located, with the centre directly above the underwater target. Extensive simulations proved useful in assessing the efficacy of the algorithm developed for optimal sensor placement.
The project culminated with the execution of sea trials in Lisbon, Portugal with three autonomous surface robots and a human diver. The experimental set-up and the human resources needed were made available through the Dynamical Systems and Ocean Robotics Laboratory of ISR/IST. The tests showed clearly that the systems developed for underwater target positioning and cooperative motion control (of the diver and the companion robots) work extremely well under real conditions and hold considerable promise for the development of future robotics systems to assist human divers during demanding commercial and scientific missions.
In conclusion, CONMAR contributed to the challenging task of developing navigation methods for autonomous marine robots. Within its defined area of application, it effectively contributed to the development of a prototype of a tracking system to assist human divers that has good potential to further be enhanced and witness the transition from the laboratory to the real world. Systems enabling extensive interaction between humans and marine robots are still at their infancy, but are expected to play an ever increasing role in the future. In this respect, the contribution of CONMAR (complemented by contributions done by the hosting group at ISR / IST) can be described as a pioneering effort. Additionally, some of the results can certainly be transferred to groups interested in marine robotic applications and are therefore of interest to all groups operating robots underwater, e.g. offshore industry, or marine scientists. The methods advanced (and proven in practice) offer a reliable and robust navigation solution for a number of scenarios involving marine robots and human divers. The theoretical research done will boost further investigation in this challenging and promising area with the objective of developing increasingly robust and reliable navigation solutions for applications in a vast number of mission scenarios. Future research activities will aim at extending the results obtained to deal with multiple vehicles and humans underwater, a situation that calls for the development of systems that will effectively afford divers access to robot 'buddies' to cooperatively perform missions at sea.
One of the central objectives of the Marie Curie actions is also to train the fellow researcher engaged and to boost his / her abilities to carry out independent scientific work. In what concerns the present researcher, he could definitely broaden his research horizons in topics such as dynamical systems theory and cooperative navigation and control of maritime robots. At the same time, he became familiar with the steps required to transition from the laboratory to the real world, effectively witnessing the different phases of system design, simulation, implementation, and testing. After having completed the CONMAR project, he was employed by Ilmenau University of Technology and given a position of senior researcher. In the meantime, he is preparing his application for a postdoctoral lecture qualification (the German Habilitation). The research group that he is supervising at Illmenau deals with mission planning and navigation of marine robots. In this respect, the experience that he acquired during his stay in Lisbon, Portugal has proven extremely valuable, for it has allowed him to capitalise on his experience during his fellowship both in terms of scientific knowledge and soft skills (such as project management) as well as planning and execution of sea trials. Additionally, he is involved in teaching at the university. It is scheduled for him to become responsible for giving lectures in modelling / process analyses from 2013 or 2014 on and to organise a new course on Estimation and Decision that will rely heavily on the knowledge that he acquired during his fellowship. Additionally, the cooperation between the host institution, the Instituto Superior Técnico, and Ilmenau University of Technology, has been strengthened; this is clearly demonstrated by the fact that both institutions are partners in the new European research project MORPH (FP7-288704).