Periodic Reporting for period 3 - SmokeBot (Mobile Robots with Novel Environmental Sensors for Inspection of Disaster Sites with Low Visibility)
Période du rapport: 2017-01-01 au 2018-06-30
Context of the project
SmokeBot is driven by application needs for robots that operate in domains with restricted visibility. The exemplary target application are civil robots supporting fire brigades in search and rescue missions, e.g. in post-disaster management operations in response to fires. Previous to this project, sensor technology and the related cognitive approaches could not cope with such demanding conditions. SmokeBot addresses this shortcoming and can thus bring about a step change for robotics. It will deliver software and hardware components which facilitate robot systems to perform under harsh conditions of smoke, dust or fog. This will be demonstrated through integration of the project results in an industrial prototype of a Low Visibility Explorer Robot for providing situational awareness based on a commercial platform (from partner TAUR). In close collaboration with TAUR (robotics industry) and end users (FDDO, advisory group), SmokeBot will crucially improve the abilities of the selected platform, thus increasing safety of rescue staff and European citizens as well as improving the product of a European robotics company in an important market. An even wider impact is expected through the development of novel sensors and the corresponding cognitive approaches. In addition to traditional sensors such as LiDAR and cameras, which are strongly affected by smoke or dust, this sensing unit will include also a novel 3D radar camera, a thermal camera, and high-bandwidth gas sensors (radar, gas, thermal and vision sensors: the RGT-V sensor unit). Fusion of sensor modalities will allow the inclusion of measurements from LiDAR or a camera into the world model when they occasionally penetrate through e.g. smoke. In addition, SmokeBot will develop the means to integrate prior knowledge in the form of crude human sketch maps to allow for robust mapping and navigation even under low visibility in a harsh environment. Sensor technology from SmokeBot will result in new products to be brought to market after the project. Software developed will be made available as open source.
Objectives
SmokeBot has three high-level objectives: hardware and software development of a novel sensor suite for low visibility conditions ([O1]), a set of perception modules to enhance the cognitive abilities of mobile robots, especially for use in disaster response ([O2]), and integration of the project results in a prototype for a commercial Low Visibility Explorer Robot ([O3]).
To achieve these objectives, SmokeBot brings together leading researchers in millimetre- and sub-millimetre-wave radar systems, gas sensor systems, artificial olfaction, mobile robotics, navigation, and 3D perception. The consortium also includes a system integrator (TAUR) producing tele-operated and highly rugged robots for civil first responders and a large fire department (FDDO) that represents the interests of end users.
SmokeBot is driven by application needs for robots that operate in domains with restricted visibility. The exemplary target application are civil robots supporting fire brigades in search and rescue missions, e.g. in post-disaster management operations in response to fires. Previous to this project, sensor technology and the related cognitive approaches could not cope with such demanding conditions. SmokeBot addresses this shortcoming and can thus bring about a step change for robotics. It will deliver software and hardware components which facilitate robot systems to perform under harsh conditions of smoke, dust or fog. This will be demonstrated through integration of the project results in an industrial prototype of a Low Visibility Explorer Robot for providing situational awareness based on a commercial platform (from partner TAUR). In close collaboration with TAUR (robotics industry) and end users (FDDO, advisory group), SmokeBot will crucially improve the abilities of the selected platform, thus increasing safety of rescue staff and European citizens as well as improving the product of a European robotics company in an important market. An even wider impact is expected through the development of novel sensors and the corresponding cognitive approaches. In addition to traditional sensors such as LiDAR and cameras, which are strongly affected by smoke or dust, this sensing unit will include also a novel 3D radar camera, a thermal camera, and high-bandwidth gas sensors (radar, gas, thermal and vision sensors: the RGT-V sensor unit). Fusion of sensor modalities will allow the inclusion of measurements from LiDAR or a camera into the world model when they occasionally penetrate through e.g. smoke. In addition, SmokeBot will develop the means to integrate prior knowledge in the form of crude human sketch maps to allow for robust mapping and navigation even under low visibility in a harsh environment. Sensor technology from SmokeBot will result in new products to be brought to market after the project. Software developed will be made available as open source.
Objectives
SmokeBot has three high-level objectives: hardware and software development of a novel sensor suite for low visibility conditions ([O1]), a set of perception modules to enhance the cognitive abilities of mobile robots, especially for use in disaster response ([O2]), and integration of the project results in a prototype for a commercial Low Visibility Explorer Robot ([O3]).
To achieve these objectives, SmokeBot brings together leading researchers in millimetre- and sub-millimetre-wave radar systems, gas sensor systems, artificial olfaction, mobile robotics, navigation, and 3D perception. The consortium also includes a system integrator (TAUR) producing tele-operated and highly rugged robots for civil first responders and a large fire department (FDDO) that represents the interests of end users.
In all work packages, work in the first period was carried out according to Annex I. All deliverables that were due during the first period have been submitted to the commission in time.
There was one milestone so far and this milestone has been achieved.
The development of the mechanically pivoting radar (MPR) is completed and two MPR prototypes have been delivered to partners LUH and ORU, ready for integration on locally available robot platforms. Initial developments towards the SmokeBot Low Visibility Explorer prototype are also completed. This concerns the user interface software (taurob Commander) on the customized remote control, the development of shock resistant and waterproof wireless repeaters and the deployment mechanism for dropping wireless repeaters by the robot. Corresponding functionality and the developed novel sensors will be demonstrated during the first review meeting.
Further, the technical specifications of the module structure and system blocks of the Gas Sensing Unit (GSU) for SmokeBot have been designed. In the following period, the GSU will be fully characterized and prototypes for testing and integration on the robot will be produced. Algorithms will be developed to compensate for cross-sensitivities and the effect of ambient conditions.
All objectives for the period have been reached and the project is on track. Resources have also been used according to plan.
There was one milestone so far and this milestone has been achieved.
The development of the mechanically pivoting radar (MPR) is completed and two MPR prototypes have been delivered to partners LUH and ORU, ready for integration on locally available robot platforms. Initial developments towards the SmokeBot Low Visibility Explorer prototype are also completed. This concerns the user interface software (taurob Commander) on the customized remote control, the development of shock resistant and waterproof wireless repeaters and the deployment mechanism for dropping wireless repeaters by the robot. Corresponding functionality and the developed novel sensors will be demonstrated during the first review meeting.
Further, the technical specifications of the module structure and system blocks of the Gas Sensing Unit (GSU) for SmokeBot have been designed. In the following period, the GSU will be fully characterized and prototypes for testing and integration on the robot will be produced. Algorithms will be developed to compensate for cross-sensitivities and the effect of ambient conditions.
All objectives for the period have been reached and the project is on track. Resources have also been used according to plan.
Progress beyond the state of the art
Major scientific work and results obtained beyond the state of the art during the first period include:
• Research and experiments regarding the influence of smoke on LiDAR and radar that showed the feasibility of registration-based SLAM with radar sensors for the first time.
• First results on sensor fusion between radar and LiDAR, which are very encouraging and show how the advantages of both sensor modalities can be combined to a fused scan.
• Tests with new metal oxide (MOX) materials on gas sensors that show excellent sensitivity to CO (doped with SnO2) and NO2 (doped with WO3).
• The development of a gas identification approach that computes class posteriors by coupling pairwise probabilities between the compounds to a confidence model based on an estimation of gas concentration. The proposed approach has shown a high success rate and a better per-formance compared to standard classification approaches.
• A novel algorithm for topological map matching of sketch maps with high uncertainty was de-veloped and tested with a sketch map GUI that was also implemented in SmokeBot.
So far 3 publications with SmokeBot results were accepted.
Expected potential impact
The expected direct impact of SmokeBot is two-fold. On one hand, SmokeBot is expected to crucially improve the abilities of the platform of partner TAUR, thus increasing safety of rescue staff and European citizens as well as improving the product of a European robotics company in an important market. The development of a prototype, which demonstrates the improved abilities of robot platforms that support first responders especially in scenarios with low visibility (the SmokeBot Low Visibility Explorer), has started in the first period as planned.
An even wider impact is expected through the development of a novel sensor unit and the corresponding cognitive approaches. First prototypes of the mechanically pivoting radar (MPR) are available after period one and the development of a 3D radar camera is on track. First prototypes of the Gas Sensing Unit (GSU), which was designed in period one, are expected in M18 also according to plan. Initial results on sensor fusion between radar and LiDAR show the complementary advantages of these sensor modalities.
The methods developed for topological map matching of crude sketch maps and the approaches devel-oped to correct misinterpretations in thermal images caused by unknown emissivity values and reflections will also be useful outside of the context of low-visibility environments.
Major scientific work and results obtained beyond the state of the art during the first period include:
• Research and experiments regarding the influence of smoke on LiDAR and radar that showed the feasibility of registration-based SLAM with radar sensors for the first time.
• First results on sensor fusion between radar and LiDAR, which are very encouraging and show how the advantages of both sensor modalities can be combined to a fused scan.
• Tests with new metal oxide (MOX) materials on gas sensors that show excellent sensitivity to CO (doped with SnO2) and NO2 (doped with WO3).
• The development of a gas identification approach that computes class posteriors by coupling pairwise probabilities between the compounds to a confidence model based on an estimation of gas concentration. The proposed approach has shown a high success rate and a better per-formance compared to standard classification approaches.
• A novel algorithm for topological map matching of sketch maps with high uncertainty was de-veloped and tested with a sketch map GUI that was also implemented in SmokeBot.
So far 3 publications with SmokeBot results were accepted.
Expected potential impact
The expected direct impact of SmokeBot is two-fold. On one hand, SmokeBot is expected to crucially improve the abilities of the platform of partner TAUR, thus increasing safety of rescue staff and European citizens as well as improving the product of a European robotics company in an important market. The development of a prototype, which demonstrates the improved abilities of robot platforms that support first responders especially in scenarios with low visibility (the SmokeBot Low Visibility Explorer), has started in the first period as planned.
An even wider impact is expected through the development of a novel sensor unit and the corresponding cognitive approaches. First prototypes of the mechanically pivoting radar (MPR) are available after period one and the development of a 3D radar camera is on track. First prototypes of the Gas Sensing Unit (GSU), which was designed in period one, are expected in M18 also according to plan. Initial results on sensor fusion between radar and LiDAR show the complementary advantages of these sensor modalities.
The methods developed for topological map matching of crude sketch maps and the approaches devel-oped to correct misinterpretations in thermal images caused by unknown emissivity values and reflections will also be useful outside of the context of low-visibility environments.