Periodic Reporting for period 2 - INTREPID (Intelligent Toolkit for Reconnaissance and assessmEnt in Perilous Incidents)
Periodo di rendicontazione: 2022-06-01 al 2023-11-30
INTREPID aims to create a unique platform, seamlessly integrating Intelligence Amplification and eXtended Reality concepts with unprecedented Smart Cybernetic Assistants and innovative deep indoor Networking and Positioning capabilities, to improve and accelerate the exploration and assessment of disaster zones. The project will validate its effectiveness in challenging, complex and complementary pilots to support rescue operations in areas that are complex or dangerous to explore.
The overall objectives of the project are manifold :
- from a methodological point of view, it aims at :
* closely studying the need of End users in these first hours of a disaster to extract meaningful requirements and enable the user-centric development of a solution
* offering a 360° approach including an adapted field-validation methodogy, to go beyond the mere development of tools, deep into the "added value" evaluation
- from a technical point of view, it aims at :
* developing advanced and innovative cybernetics assistants (indoor drone, outdoor drone, robot) that can provide autonomous exploration capabilities, replacing FR in hazardous situations
* studying and developping symbiotic operation capabilities between cybernetics assistants to enable advanced exploration
* provide accurate and real-time situation awareness with indoor/outdoor agent positioning, real time update of the 3D environment, advanced automatic detection of point of interest, and innovative visualisation solutions
* propose doctrine-based decision support, including mission recommandation for all agent and path planning from position to target
The overall objectives of the project are manifold :
- from a methodological point of view, it aims at :
* closely studying the need of End users in these first hours of a disaster to extract meaningful requirements and enable the user-centric development of a solution
* offering a 360° approach including an adapted field-validation methodogy, to go beyond the mere development of tools, deep into the "added value" evaluation
- from a technical point of view, it aims at :
* developing advanced and innovative cybernetics assistants (indoor drone, outdoor drone, robot) that can provide autonomous exploration capabilities, replacing FR in hazardous situations
* studying and developping symbiotic operation capabilities between cybernetics assistants to enable advanced exploration
* provide accurate and real-time situation awareness with indoor/outdoor agent positioning, real time update of the 3D environment, advanced automatic detection of point of interest, and innovative visualisation solutions
* propose doctrine-based decision support, including mission recommandation for all agent and path planning from position to target
1. Technical work
Regarding the implementation, the global architecture of the platform has been defined and developped, based on a message exchange mechanisms (message broker) and a common data model consisting of JSON messages. Servers and front ends, in addition of all required repositories with adequate INTREPID data storage capabilities have been implemented and deployed in time for pilot 1 that occured in M13 of the project.
Regarding the specific modules of the platform.
- the "wheeled-legged" UGVhas been designed, as well as its autonomous navigation feature
- the indoor UAV has been constructed and tested during pilot 1
- a first version of the positioning toolkit for agents has been developped and tested during pilot 1
- the automatic building model reconstruction has been developped and integrated
- a first version of the Objects of Interest recognition algorithm has been developped and tested
- the authoring tool for doctrine definition has been implemented
- a first version of the Intelligence Amplification recommendation engine has been developed and tested during pilot 1, with automatic provision of path to target
Regarding integration, a target hardware infrastructure has been defined and tools have been setup to replicate this infrastructure. The iterative and continuous integration strategy leads to a live integration session at Valencia. Additional remote sessions allow the partners to adjust their data to be exchanged, given by the operational point of view.
This, in addition to the constant monitoring of the functional chains and workflows, ensures a consistent behaviour of the system.
2. Pilots and demonstrations
A common-ground field-validation methodology has been set up to be applied to trainings and pilot executions, generally applicable to tools dedicated to First Responders.
Work done in M13-M16 involved the preparation and execution of the project’s first pilot in Stockholm, Sweden that occured from 2 to 5 November 2021. The pilot included workshop opportunities for end-users and technical partners.
From M17 until the execution of the second pilot, the focus remains on the detailed preparation of the 2nd demonstration that will occurs in Marseilles, France, from 17th to 21st October 2022.
3. SEL impact
The first stage of the project has focused in studying the applicable legislation and ethical principles. This benchmark study is the common framework to all the activities in the project. It will be the reference for compliance in the future development of the research.
The subsequent phase corresponds to the description of the main legal and ethical issues to be followed during the rest of the time in the project.
Regarding the implementation, the global architecture of the platform has been defined and developped, based on a message exchange mechanisms (message broker) and a common data model consisting of JSON messages. Servers and front ends, in addition of all required repositories with adequate INTREPID data storage capabilities have been implemented and deployed in time for pilot 1 that occured in M13 of the project.
Regarding the specific modules of the platform.
- the "wheeled-legged" UGVhas been designed, as well as its autonomous navigation feature
- the indoor UAV has been constructed and tested during pilot 1
- a first version of the positioning toolkit for agents has been developped and tested during pilot 1
- the automatic building model reconstruction has been developped and integrated
- a first version of the Objects of Interest recognition algorithm has been developped and tested
- the authoring tool for doctrine definition has been implemented
- a first version of the Intelligence Amplification recommendation engine has been developed and tested during pilot 1, with automatic provision of path to target
Regarding integration, a target hardware infrastructure has been defined and tools have been setup to replicate this infrastructure. The iterative and continuous integration strategy leads to a live integration session at Valencia. Additional remote sessions allow the partners to adjust their data to be exchanged, given by the operational point of view.
This, in addition to the constant monitoring of the functional chains and workflows, ensures a consistent behaviour of the system.
2. Pilots and demonstrations
A common-ground field-validation methodology has been set up to be applied to trainings and pilot executions, generally applicable to tools dedicated to First Responders.
Work done in M13-M16 involved the preparation and execution of the project’s first pilot in Stockholm, Sweden that occured from 2 to 5 November 2021. The pilot included workshop opportunities for end-users and technical partners.
From M17 until the execution of the second pilot, the focus remains on the detailed preparation of the 2nd demonstration that will occurs in Marseilles, France, from 17th to 21st October 2022.
3. SEL impact
The first stage of the project has focused in studying the applicable legislation and ethical principles. This benchmark study is the common framework to all the activities in the project. It will be the reference for compliance in the future development of the research.
The subsequent phase corresponds to the description of the main legal and ethical issues to be followed during the rest of the time in the project.
An extensive study has been conducted on the role and potential collaboration of cybernetics assistants in disaster areas, in order to enable symbiotic use cases. At the end of the project, it is expected for the user to be able to launch a collaborative mission and for the platform to automatically assign orders to the associated assistants. It will widen the actions that are accessible to UxV to another level, for instance allowing drones to fly much further with UGV transport and/or charging capabilities.
The toolkit for positioning of agents has been developped, using sensor fusion and mapping algorithms, but without requiring pre-installed infrastructure. Study and analysis of the stability and accuracy of such positioning has been conducted, and it is expected at the end of the project to provide a positioning system and process that will accurately position the agent within a building without drift. The impact of this capability goes beyond SAR operations since it can be applied to many contexts.
A global building-centered workflow has been designed and implemented, based on a first model of the area that is updated in real-time to give an exact representation of the area and also used to compute paths and provide realistic visualisation of the scene through advanced front ends such as an Augmented Reality COP. In addition, new storage and provision mechanisms has been developped in order to efficiently handle the heavy datasets that contain the information. At the end of the project, it is expected to integrate and display all relevant elements of the operation within the building and to exploit them in operation, for instance when providing paths to First Responders. As BIM modeling of the buildings is being more and more prevalent, this will lead to path to BIM-centered operation workflows requirements and best practices.
Finally, huge progress has been made in the general field of doctrine-based decision support. First by providing tools to model the doctrine without requiring specific competences. Then by developping an Intelligence Amplification that integrated both this static model with learnt premises (dynamic) in order to provide context-based mission recommendation. It is expected at the end of the project to obtain a dynamic and justified flow of recommendation that directly and automatically exploit the current status of the operation and the pre-defined doctrine elements. This is expected to have a major impact on the SAR operations as it will alleviate the cognitive load of users.
In order to structure the evaluation and verification process, the INTREPID project has developed a field validation methodology based on the notion of "expected value" of tool dedicated to First Responders. This methodology is expected to be the subject of a webinar in November 2022, and it is designed to be general enough to be applicable beyond the scope of INTREPID.
The toolkit for positioning of agents has been developped, using sensor fusion and mapping algorithms, but without requiring pre-installed infrastructure. Study and analysis of the stability and accuracy of such positioning has been conducted, and it is expected at the end of the project to provide a positioning system and process that will accurately position the agent within a building without drift. The impact of this capability goes beyond SAR operations since it can be applied to many contexts.
A global building-centered workflow has been designed and implemented, based on a first model of the area that is updated in real-time to give an exact representation of the area and also used to compute paths and provide realistic visualisation of the scene through advanced front ends such as an Augmented Reality COP. In addition, new storage and provision mechanisms has been developped in order to efficiently handle the heavy datasets that contain the information. At the end of the project, it is expected to integrate and display all relevant elements of the operation within the building and to exploit them in operation, for instance when providing paths to First Responders. As BIM modeling of the buildings is being more and more prevalent, this will lead to path to BIM-centered operation workflows requirements and best practices.
Finally, huge progress has been made in the general field of doctrine-based decision support. First by providing tools to model the doctrine without requiring specific competences. Then by developping an Intelligence Amplification that integrated both this static model with learnt premises (dynamic) in order to provide context-based mission recommendation. It is expected at the end of the project to obtain a dynamic and justified flow of recommendation that directly and automatically exploit the current status of the operation and the pre-defined doctrine elements. This is expected to have a major impact on the SAR operations as it will alleviate the cognitive load of users.
In order to structure the evaluation and verification process, the INTREPID project has developed a field validation methodology based on the notion of "expected value" of tool dedicated to First Responders. This methodology is expected to be the subject of a webinar in November 2022, and it is designed to be general enough to be applicable beyond the scope of INTREPID.