Final Report Summary - INACHUS (Technological and Methodological Solutions for Integrated Wide Area Situation Awareness and Survivor Localisation to Support Search and Rescue Teams)
Executive Summary:
In a duration of 48 months (January 2015 – December 2018) and implemented by a consortium of 20 partners INACHUS achieved a significant time reduction related to Urban Search and Rescue (USaR) phase by providing wide-area situation awareness solutions for improved detection and localisation of the trapped victims assisted by simulation tools for predicting structural failures and a holistic decision support mechanism incorporating operational procedures and resources of relevant actors. The INACHUS methodology is user-centric, involving end-users as much and as often as possible. End-users (First Responders, FRs and Urban Search and Rescue, USaR teams) are regularly being consulted to collect their insights, needs and remarks. Based on these requirements, the INACHUS project carefully designs, implements and evaluates, with end-users, a seamlessly integrated platform which provides the appropriate tools to enable FRs and USaR crews to respond to varied abnormal events, not limited to a specific emergency case or crisis event.
The INACHUS System is structured around the individual functionality of the partially integrated subsystems from individual components and the dataflow between them. This dataflow is defined between the mapping and building assessment software subsystem, the real 3D mapping tools and the victim localization solutions to the SaR-ESS and the INACHUS emergency support system via the communication infrastructure. The dataflow between the victim localization solutions and the SaR-ESS is being processed throughout the Real-Time Communication Gateway.
Reflecting the nature of the work, the INACHUS is organized in 12 Work Packages (WPs) each with a set of deliverables and milestones. The WPs follow the logical phases of the implementation of the project and include consortium management and assessment of progress and results. INACHUS spanned across 4 periods of 12-month duration each. In the first period the overall system architecture, a mixture of technical and user requirements and subsequent specifications, emerged. Moreover, in the same period early developments of the INACHUS components were realised that have been finally prototyped during the second period. .In the 2nd integration activities began towards the interfacing and interworking of the INACHUS integrated system. During the same period the individual INACHUS components have been preliminary evaluated under blast conditions over the execution of the component testing and the first pilot event. During the 3rd period full scale developments and deployments occurred in order to launch the first INACHUS integrated prototype. In the final period of the project, INACHUS solution was exhaustively tested in two pilots validating the components and the overall suggested solution as well. INACHUS widely disseminated its interim and final results, featuring scientific publications, a vivid online presence though its website and social media, an end user group of more than 100 organisations, regular newsletter releases and a well-established clustering network with other projects and initiatives in the field of USaR and disaster events. Noteworthy, INACHUS concluded in a CWA in the field of USaR robots paving the way to future standardisation in this field. At present a discussion is ongoing on the next concrete steps that need to be followed for exploiting further INACHUS outcomes and turning the consortium from a R&D focused entity into an effort to unlock the commercial potential of the project.
Project Context and Objectives:
Crisis incidents may result in difficult working conditions for Urban Search-and-Rescue (USaR) crews. INACHUS concept was driven towards achieving a significant time reduction related to Urban Search and Rescue (USaR) phase by providing:
1) Wide-area situational awareness solutions for improved detection and localisation of trapped victim.
2) Simulation tools for predicting structural failures.
3) Holistic decision support mechanism aiding in prioritization and mission coordination.
The INACHUS system that has been delivered encompasses novel technologies originating from different domains, seamlessly interworking with the aim to shorten the required time of USaR activities. In more detail, the INACHUS work delivered simulation and realistic visualisation methods on the scale of city quarters and for individual buildings. This was realised by the development and provision of a building library by using AEM, FEM and DEM approaches to identify and enhance the achievable prediction quality. Furthermore, a mapping and cavity identification tool was developed aiming at automatically suggesting rescue paths to trapped victims and USaR teams. Equally important was the development of 3D Digital surface models on affected areas, resulting in structural damage accurate assessments, especially if combined with the mapping and cavity identification tool. For wide area surveillance cases, wide area surveillance was performed thanks to 3D laser scanners embedded on fixed wing UAVs, covering large areas during hours by automatic or piloted navigation collecting digital elevation and surface models (DEM/DSM) of high resolution and high quality (5cm).INACHUS INACHUS developed and took advantage of deploying several novel victim localisation technologies: The Ground-Based Seismic Sensor system (GBSS) is a tool for detecting and locating knocking signals from victims trapped in debris heaps; The SurfaceRadar, also called HumanFinder, is a UWB beam steering radar dedicated to searching for survivals from disasters; the Mobile Phone Detector (MPD) detects mobile phones under rubble by making use of a directional antenna, the source of the strongest signal on this known communication channel can be approached; the Robotradar / Stickradar a miniaturized radar system, one of several sensors integrated in the INACHUS Robot; detecting movements in five directions around the robot body. Moreover, INACHUS developed a snake-like robot prototype designed and manufactured to help USaR teams finding and communicating with victims under a collapsed building. The robot apart from the robot radar has two video cameras for inspection and guidance and works as a sensor platform as well including an electronic nose to detect human presence and dangerous gases, and an infrared camera to detect human presence in case of poor visibility. Output of all carried sensors is visible in the integrated graphical user interface running on a tablet PC that acts as the remote controller. The robot is capable in entering in small holes (20cm x 20cm of cross-section) and contains a power supply, an industrial grade PC and all necessary communication devices. The e-nose sensor can also detect many air related hazards in confined spaces. These include the most typical asphyxiation hazards, toxic gases and combustible gases. Last but not least INACHUS robot makes use of a Real-Time Locating System (RTLS) which provides the feature of determining the absolute and relative position of the robot head module in the rubble. The infrared camera used in the INACHUS robot is a miniaturized LWIR camera (resulting in operations, which do not depend on illumination conditions). An area detection methodology is applied in order to identify regions with the temperature of a human body (34ºC-41ºC). This temperature range is adjustable according to the user needs and the specific targets to be detected. A crucial parameter for the successful functioning of INACHUS solution in real conditions are the stable and reliable communications. The communication platform that INACHUS developed manages the seamless interoperation, the interconnection between other networks (cellular, etc.) and provides redundancy and recovery functionality. For extending the capabilities and range of the crisis network in case of failures and/or congestion and to provide a distributed architecture, portable gateways and a long-range communication solution where designed and implemented. The gateways support the functionality of any other mobile USaR center as an alternative and cost-effective solution in an integrated ad-hoc manner but with redundancy, extendibility and security capabilities to assure robustness, flexibility and reliability.
Further to above, in order to support decision making in disaster events and to plan efficiently INACHUS project developed three novel tools aiming at merging information from the various tools used in the field and from simulation and modelling tools. Multi-source information engine (MIFE) acts as the data fusion model engine for the INACHUS solution resulting in the location of possible survivors and/or the presence of dangerous gases. This information is sent to both a portal & mobile application for search & rescue operation (SaR-ESS) and to the Common Operational Picture (COP). The former acts as the digital transformer of the collected information sharing them to responsible teams and persons for proper decision-making. The latter (COP) links end users with the entire INACHUS system by providing a comprehensive map-centric view on the incident site. The rich visualization capabilities of the COP in 2D and 3D allows the end user an intuitive access to the incident situation, which increases the overall situational awareness and aids the decision process of USaR personnel on strategic as well as tactic and operational levels.
For designing, developing, prototyping, testing, evaluating and ultimately delivering the abovementioned system, the 48-month work implementation has been divided in 12 work packages as follows: WP1 on Scenarios definition, user/system requirements and specifications, WP2 on Framework design and interoperability issues, WP3 on Simulation tool for structural damage analysis and casualty estimation, WP4 on Wide-area surveillance tools for monitoring of collapsed buildings, WP5 on Victim localisation solutions, WP6 on INACHUS Emergency Support System (SaR-ESS), WP7 on Secure communication and positioning issues, WP8 on System Integration, WP9 on Pilot implementation and validation of the INACHUS platform, WP10 on Dissemination, Exploitation and Training activities, WP11 on Evaluation and consideration of societal Impacts, legal/ethical issues and standardisation and WP12 on Project management, quality assurance and reporting. In the following of present section, a description of the entirety of project objectives at WP-level can be found including their accomplishment status.
WP1 Objectives and Accomplishment status:
- To define the user requirements (services) for INACHUS (Accomplished and delivered in D1.2 & D1.1)
- To define the system requirements for INACHUS (Accomplished and delivered in D1.2)
- To define evaluation criteria and objectives for demonstration for INACHUS (Accomplished and delivered in D1.1 D1.2 D1.3)
WP2 Objectives and Accomplishment status:
- To design and deploy a framework according to the WP1 requirements (Accomplished and delivered in D2.1)
- To deploy all the required connectors to integrate all the pilots and scenarios in the INACHUS system framework (Accomplished and delivered in D2.2)
- To define the criteria on which the solution was validated, and the plan for the final integration before the pilot tests (Accomplished and delivered in D2.2)
WP3 Objectives and Accomplishment status:
- To provide 3D simulation methods for prediction of survival spaces after disastrous events (Accomplished and delivered in D3.1 D3.2)
- To provide pre-calculated results and fast computations methods for on-site assessment of first responders (Accomplished and delivered in D3.3)
- To provide realistic 3D representation of partially failed buildings and debris heaps (Accomplished and delivered in D3.3 D3.4 D3.5)
WP4 Objectives and Accomplishment status:
- Wide-area disaster scene assessment based on nested integration of space borne information and results of numerical analysis (from WP3), and population distribution mapping, for 3D modelling and USaR Prioritization (Accomplished and delivered in D4.1)
- To provide high-resolution and high-quality 3D Digital surface models coming from laser cameras and photogrammetry data on the most affected areas (Accomplished and delivered in D4.2)
- To perform an exploitation of this 3D data in relation with the library of collapsed building models coming from WP3 using a semantic analysis or matching method (Accomplished and delivered in D4.3)
- To assess the structural damage to deliver maps of survival space and rescue path as inputs to the WP6 (Accomplished and delivered in D4.1 D4.3)
WP5 Objectives and Accomplishment status:
- To provide sensor deployment solutions and increase speed of detection and localisation of trapped victims under rubble (Accomplished and delivered in D5.1 and other WP5 deliverables)
- Design and develop the snake robot used as sensor platform (Accomplished and delivered in D5.6)
- Other auxiliary sensors serve as monitoring and localisation of mechanical vibration (Accomplished and delivered in D5.2)
- Evaluate methods for prioritisation of search areas by means of triangulating the location of radiation originating from personal handsets (mobile phones) buried under rubble (Accomplished and delivered in D5.1)
- Provide preliminary indication of presence of trapped victims using CW radar, e-nose and IR camera mounted on the snake robot (Accomplished and delivered in D5.3 D5.4 D5.5)
- Investigate optimal distributed seismic sensor network configurations and operational modes with the aim of reducing mechanical interference to the radar, which in turn increases probability of detection (Accomplished and delivered in D5.2)
- To test and evaluate the proposed sensors under real life-like conditions. (Accomplished and delivered in D5.6 and other WP5 deliverables)
WP6 Objectives and Accomplishment status:
- Development of the INACHUS back-end to support the in- situ operations at both operational and strategic levels (Accomplished and delivered in D6.1 D6.2)
- Development of INACHUS front-end tools to enable a common operation picture among main actors while an operation evolves for enhanced situation awareness and better decision making (Accomplished and delivered in D6.1 D6.2)
WP7 Objectives and Accomplishment status:
- Development of the INACHUS communications and positioning platform and its subsystems (Accomplished and delivered in D7.1
WP8 Objectives and Accomplishment status:
- To integrate all the hardware and software components defined for the INACHUS system framework, taking into account the different pilots/scenarios established (Accomplished and delivered in D8.1 D8.2 D8.3)
WP9 Objectives and Accomplishment status:
- To demonstrate the use of INACHUS for each case (type of crisis) (Accomplished and delivered in D9.1 D9.2)
- To validate the impact of INACHUS for each case with external experts (Accomplished and delivered in D9.1 D9.2)
- To initially acquaint the FRs to work with INACHUS- parallel activities with WP10-Training Activities (Accomplished and delivered in D9.1 D9.2)
- To test the development of INACHUS on a local scale (site in the Netherlands) and at a transnational scale (site in France) (Accomplished and delivered in D9.1 D9.2)
- To validate the imaging (mainly airborne based) techniques (WP4 outcomes) plus the simulator of WP3 in the site in southern France (Accomplished and delivered in D9.1 D9.2)
- To test and evaluate the proposed sensors under real life conditions (Accomplished and delivered in D9.1 D9.2)
WP10 Objectives and Accomplishment status:
- To establish systematic channels and means for disseminating the project objectives, activity, progress and technological outcome to potential INACHUS stakeholders (Accomplished and delivered in D10.1)
- To ensure the alignment of the project activity with calendar events of relevant EU programmes & initiatives (Accomplished and delivered in D10.1 D10.3)
- To share the technical results with the scientific community interested in the topics addressed by the project, in order to promote the research and receive useful inputs from other scientists and International Communities (Accomplished and delivered in D10.3)
- To ensure adequate training of the end users participating in the field tests and availability of a comprehensive training package at the end of the project to facilitate the uptake of the INACHUS system by interested entities (Accomplished and delivered in D10.2 D10.5)
- To define the vision of using the project outcome and create relative awareness within Civil Protection administrations and Security organizations throughout the Member States of the European Union (Accomplished and delivered in D10.4)
- To develop the strategic approach, define the appropriate business plan and elaborate a suitable market model which can support the perspective of commercializing the project results (Accomplished and delivered in D10.4)
- To prepare support products, including documentation, in a form that can be understandable and accepted by potential users, help in technology transfer and provide necessary advices and support (Accomplished and delivered in D10.1 D10.5)
WP11 Objectives and Accomplishment status:
- To investigate current state of the ethical and societal issues in the USaR, including a limited literature review on victims’ perspective (Accomplished and delivered in D11.1)
- To define ethical and societal requirements for the solution, and to formulate ethical guidelines for the end-users (Accomplished and delivered in D11.1 in various technical notes, as well as in the INACHUS Code Of Conduct and in the organizational guidelines for data protection. The latter contents are embedded in the INACHUS Training Kit)
- To verify the application of the ethical requirements and user guidelines in pilots (Accomplished and delivered as part of INACHUS validation process. In addition a separate Privacy and Data Protection Impact Assessment (DPIA) is provided.in D11.1)
- To highlight the potential societal impacts of the proposed system and its business model, and provide recommendations on these in order ensure a successful adoption of the proposed technologies (Accomplished and delivered in D11.1 and finalized as part of the INACHUS exploitation plan D10.4)
- To exploit the current available standards, best practices and legal issues considerations when developing new technologies in the USaR operations (Accomplished and delivered in D11.3)
- To conduct a comprehensive investigation of all regulatory framework and legal aspects, regarding Information and Communication Security, to ensure compliance with current regulatory requirements and, where appropriate, develop recommendations for further regulatory considerations by the appropriate bodies. (Accomplished and delivered in D11.3)
- To foster Interoperability and interchange-ability of equipment and systems, to further define the new/ revised/ integrated/ proposed pre-normative standards based on the outcomes of INACHUS (Accomplished and delivered in D11.2
- To launch and promote the debates on USaR related pre-normative standards in the context of the European Standardisation Organizations through the new Standardisation process vehicles, like CEN/CENELEC Workshop Agreement or ETSI Industrial Specification Group (Accomplished and delivered in D11.2)
WP12 Objectives and Accomplishment status:
- To perform the project objectives in a qualitative way and to meet the project schedule and deadlines (technical coordination) (Accomplished and delivered in D12.1)
- To perform the financial and administrative tasks of the project (Accomplished and delivered through EC periodic and current EC final report)
Project Results:
The present section describes all project S & T results/foregrounds for the entire project duration, the latter being of 48 months starting on January 2015 until end of December 2018. The project lifecycle comprised of 12 distinct work packages has been divided in four periods each of them comprising of 12 months. All S & T results/foregrounds have been delivered in the context of the first nine technical work packages of the project, as the latter three are dealing with horizontal activities, namely dissemination, exploitation and project management.
First period (M1-M12) took place from January 2015 until December 2015 during which the work of WP1 was under full implementation while technical WPs i.e. WP2, WP3, WP4, WP5 and WP8 have commenced. The latter ones capitalise mainly on the end user requirements extracted - through questionnaires and a workshop - from a plethora of end users originating from different categories. Of equal importance input are considered the technical requirements, the INACHUS subsystems’/components’ specifications as well as the overall INACHUS architecture that set the basis for further developments and research activities. At the same time all WPs previously mentioned fuel with their outputs WP8 (started on M12) towards the very first INACHUS component functional and validation testing before integration.
Second period (M13-M24) has been realised from January 2016 until December 2016, during which the entirety of technical WPs, namely WP3, WP4, WP5, WP6, WP7, WP8 and WP9 have been running in parallel showcasing significant progress towards functional prototypes while WP1 and WP2 concluded. Noteworthy, the 2nd period includes the beginning of the integration activities (under WP8) towards the interfacing and interworking of the INACHUS integrated system. During the same period the individual INACHUS components have been preliminary evaluated under blast conditions over the execution of the component testing. Noteworthy, WP9 on Pilot Implementation started in the 2nd period providing an internal plan including Key Performance Indicators for the whole system, use cases as well as the description of the mock up exercises to validate the INACHUS system’s usability. The first pilot was realised during this period, providing valuable information about the INACHUS components by testing them in real conditions.
The third period of INACHUS (M25-M36) has been implemented during January 2017 until December 2017 providing almost finalised prototype of INACHUS solution. The main achievements for the aforementioned period was the establishment of the building library; the finalization and integration of the mapping and the cavity identification tool in the INACHUS system; a prototype validation of the tools to retrieve 3D data from the collapsed buildings using aerial and ground measurements, photogrammetric images from RGB cameras and 3D point clouds from Laser cameras;•an e-nose prototype including CO2, O2, NH3, CO, H2S, combustibles, and temperature and humidity sensors. Also the radar prototype and the thermal camera were completed and integrated in the INACHUS robot. For the robot, various parts were 3D printed and assembled into the final system. Furthermore, the SaR-ESS, was delivered while the COP was almost finalized. The integration interfaces between all the components and subsystems have been defined and the implementation and the compliance validation of the individual components with these interfaces have been tested and validated. Last, the second pilot of INACHUS project and the simulated exercises were successfully performed, whereas already the third pilot has been planned.
In the fourth and final period of INACHUS (M37-M48 i.e. January 2018 until December 2018) focus was given in the integration and testing of INACHUS system during the pilots. Some additional implementations in the mapping tool increasing the value of the tool for the end users and thereby simultaneously the probability of real usage of the tool in future were realised. In addition, 3D data exploitation using aerial laser cameras and image analysis by drones was finalised and SAR-ESS, a complex system composed by numerous modules and components developed to control, visualize and facilitate the search-and-rescue operations and protocols was finally delivered. Moreover, the final version of the INACHUS integrated system was finally deployed and tested during two pilots, on April 2018 in Weeze, Germany and on November 2018 in Roquebilliere, France. In both pilots the comments and feedback received by external participants were very positive for the features and user-friendliness of the INACHUS system.
The most important S & T results/foregrounds of the above mentioned WPs’ progress will be briefly but concisely described below for the entire INACHUS lifecycle:
INACHUS technical work commenced with WP1 which has initially focused on the State-of-the-Art analysis of current USaR ICT systems and on the extraction of the recommendations coming from the end-users. The information collected was analysed according to the technological aspects of navigation, communication platform and motion control of unmanned systems focusing on USaR operations. In parallel, special emphasis has been given on the formation of the INACHUS end-user community, as expected from T1.2 and to the validation of end-user requirements. This was realised by the preparation of a questionnaire and a series of workshops with end-users. The first end user workshop occurred on April 8 and 9, 2015 in Leiden, The Netherlands. The interactive, international workshop was attended by a diverse group of USaR workers and others related to the field, as well as INACHUS technical and end user partners. During the workshop, technical partners provided demonstrations of prototypes or mock ups of system components, and there were many opportunities for discussions and feedback. The goal of this first workshop was to collect user requirements. In addition, scenario elements, training needs and mission requirements were discussed. During the second workshop, hosted by EPLFM in Gardanne, France on June 10, 2015 the user requirements were validated. WP1 activities concluded in M15 with the delivery of D1.1 & D1.2 both related to end-user requirements as discussed and elaborated during the dedicated workshops (D1.1.) and as incorporated with the outcomes of the interviews and questionnaires (D1.2). Furthermore, list of EURs included specific operational end users requirements as collected by end-users in the context of T1.3 including operational reports from major disaster events such as the Haiti earthquake, Nepal earthquake and Japan tsunami. This task concluded by D1.3 a report including a detailed description of the USaR international procedures and guidelines, mainly elaborated by the INSARAG guidelines. Last but not least, T1.4 translated the operational needs into technological needs. Apart from EURs WP1 contributed in INACHUS project by generating the first use cases (UCs).
The work delivered in WP2, commenced in first period, includes the design and development of all the system components into a common Framework, and interoperability and integration matters were taken into consideration. The main outcome of T2.1 for the first period was a document entitled “Analysis of Regulatory Framework, Standardization and Interoperability Issues” incorporating a thorough review of the regulatory framework, the standardization and the interoperability issues through an investigation of existing standards and regulations regarding Information and Communication Security to ensure interoperability and interchange-ability of equipment and systems. These were mapped with respect to the applicable system components that comprise the INACHUS framework. The output of this work had the objective of feeding the INACHUS technical work packages (namely, WP3, WP4, WP5, WP6 and WP7) with inputs on standards and practices issues, on which the INAHCUS systems providers can base their work in order to ensure interoperability and security of the components as a working whole. WP2 produced 2 key deliverables for the success of the final INACHUS solution. D2.1 including the INACHUS System Architecture was produced under T2.2 and T2.3. In T2.2 conceptual and functional diagrammatic depiction of the INACHUS System (Conceptual Diagram and Functional Diagram), was created. Furthermore, T2.2 proceeded with an interactive process whereby input and feedback was provided by all the technical partners who were responsible for providing components or parts of components to the overall INACHUS System. The next step of this effort was the high-level conceptual diagram and the functional model. All delivered technical requirements, had been produced and updated, so that the technical requirements that were mapped from the end-user requirements could also be matched with the system’s specifications. Furthermore, the Technical Requirements were defined and mapped from corresponding End User Requirements. The process involved the categorisation of Technical Requirements according to System Component, and their type (“Functional”/“Non-Functional”). Additional information included the responsible partner, their subtype, the Use Case they cover, whether they are technically feasible, their feasibility constraints (if applicable), their interdependencies and their compliance with standards/regulations (if applicable). A careful cross-check of the System Specifications and Technical Requirements was conducted in order to ensure that all requirements deemed “mandatory” and “important” would be satisfied by the corresponding system components. In T2.3 the INACHUS architecture was designed. All partners active in the corresponding technical WPs contributed to the definition of the architecture of the corresponding parts of the INACHUS framework. The INACHUS architecture was complemented by the system specifications (D2.1 Annex 2) where information about the various component interfaces is presented. Note that the Annex 2 was a ‘live document’ until the end of T2.3 because the continuous development of the various INACHUS components was on going. Thus, the refinement and fine-tuning of the Technical Requirements, the updates on the system specifications and the cross checking of technical requirements to system specifications continued through the end of T2.3. Therefore, the technical team, having this information available, was able to continue the development of the INACHUS components and further deepen the analysis of the INACHUS framework contributing to the integrations activities, such as deployment diagram, interface control documents, interoperations among components, validation criteria, etc. Equally important, T2.4 was responsible for the development of a detailed plan for the integration process from the early stages of the project. This task was also responsible for defining the criteria for which the system components’ interconnection(s) would be validated. The D2.2 document presents a holistic view of the INACHUS platform integration that combines the general development procedures established in System Engineering with the development procedures established in the DOW and with the needs that the specific requirements and the specific design solutions generate for these pre-defined methodologies. This document first analyses these specific constrictions, and then it provides the specific solution procedures, the organization charts of responsibilities to execute them and the timing in order to optimize their execution including alternatives for contingency, and, finally, it defines criteria in order to validate the whole process.
WP3 provided simulation and realistic visualisation methods both on the scale of city quarters and for individual buildings. The methods provides USaR the capability to continuously improve the situational awareness of rescue teams starting from hour one after the incident occurs, up to a point in time in which the situation is fully assessed and responded to. More specifically, T3.1 aimed at the deployment of a software tool for vulnerability assessment of city quarters against blast scenarios enlarged by a module for evaluation of the expected damage under earthquake loading. The software tool included features not only based on a relative coordinate system set by the user but also by using real world coordinates that can be extracted directly form the mapping tool results. After the map is imported, the scale factor is automatically determined and each city object is referenced with real-world data. In addition, there is a feature of linking real-time earthquake data from shakemaps.org. Hence, it is possible to import ground accelerations directly, instead of estimating the intensity by the distance of the building to the epicentre. This means on the other hand that the effect of soil conditions on the earthquake’s intensity is now explicitly incorporated. T3.2 run in parallel with T3.1 aiming to conduct a literature review of available high-quality data sets for existing partly and totally collapsed buildings. Besides understanding the occurrence, reasons and the nature of building collapse, a further key objective of the literature survey was to identify suitable collapse cases, which can be used to judge the accuracy of the different simulation methods in predicting collapse for the single building assessment. The study of the collapse cases has started from the investigation of a database including 130 real collapse cases that occurred all around the world due to several causes as documented in D3.1. All of these events have been investigated evaluating the type of incident and the number of losses, the structural composition (materials and static scheme) and the dynamic trigger, the type of collapse (partial – total) and the collapse shape, the available official information and the potential use of simulation tools for supporting USaR teams. The review showed that many published cases are missing crucial information necessary for proper modelling. Relevant real cases from the formulated including relevant data were studied extensively. Moreover, three academic research examples were selected to be studied: Shirai et al. (2005) triaxial seismic testing, Peng (2015) progressive collapse experiment due to columns removal and Yu et al. (2014) blast testing of structural component. Additionally, several collapse cases taken from the ASI client database of controlled demolitions were examined. The data collected for each one of these case studies has guided WP3 to assess the selection criteria for the simulation cases according to the INACHUS objectives and to define the final validation strategy for the three available simulation methods (FEM, AEM and DEM). The validation was implemented in three different levels: a component level (Yu et al., 2014), a mid-size level (Peng, 2015) and a full-size level (simulation of the Pyne Gould Building, Christchurch). Two additional cases, namely an academic case (Shirai et. al.,2005) and the A.P Murrah building, was modelled to enrich the validation. The latter has been already modelled by ASI with a previous model of the software ELS but it was re-modelled and implemented specifically for the INACHUS project.
Chosen cases provided a variety in terms of the size of collapsed structures, the level of the damage to the structure, and the cause of collapse by earthquake or blast. The involved partners spent a great amount of efforts in D3.1. D3.1 gives an overview of relevant hazards and trends in literature, evaluates the different hazards with regards to their consideration within the INACHUS project and identifies different collapse shapes. One chapter is dedicated to the above described real collapse cases, discusses several cases and lays out the validation strategy as mentioned. The second part of the deliverable documents the predefined buildings for the mapping tool and for the detailed collapse simulations. The final work in this task consisted of the modelling of the chosen validation examples. At component level and mid-size level validation, applied element method and finite element method were used for the modelling while at full-size level validation applied element method was used. Details on the outcomes of modelling validation are included in D3.2. T3.3 typical building was defined including structural details and typical characteristics prone to collapse risks. A building library was prepared including building models that were later used for T3.4. Four buildings were chosen from the range of typical buildings defined in Task 3.3 as reported in D3.1. Chosen buildings are a multi-Family house, a high-rise building an office tower and a historic building. Artificial earthquake histories were generated, scaled and applied to the models in different directions. Explosion scenarios were modelled by sudden element removal, as further explained in D3.3. Next steps included simulations by using AEM and DEM in the four different types of buildings and are fully documented in D3.3 and in related annexes. Furthermore, an evaluation of speed enhancement of AEM and DEM models was implemented under T3.5 by a detailed study in order to assess a trade-off of accuracy, complexity and speed and thereby optimize the models towards fast on-site operations. Concerning AEM method, for the simplified and the refined model, three level of analysis were performed for an earthquake-loading scenario: “fast”, “medium” and “slow” depending on different parameter settings. Hence, a total amount of six simulations was analysed. Despite the geometrical and structural simplifications, the “slow” and the “medium” analysis show a similar final collapse shape. The medium analysis in particular matched the INACHUS request of building quickly a model (about three hours), which gives a realistic and usable debris pile within a reasonable calculation time (about 13 hours). The fast analysis provided different results for both the refined and the simplified model with a total collapse of the central core and can therefore be seen as not valid. With regards to the DEM method, despite the already mentioned pre-processing routine, which was tested and further improved within T3.5 the utilization of a customized blender release, the “Fracture modifier” (FM) was undertaken. More efficient object management in the FM compared to the Blender standard release led to a considerable speed improvement (by a factor of ca 35 to 40). The utilization of the FM can be seen as a major step towards “real-time” simulation. The modified, more effective object handling in the FM has been noticed favourably by Blender´s core developers. Details concerning the pre-processing routing and the usage of the FM are reported in D3.5. Next step in the development efforts under WP3 included the development of cavity identification and data processing (T3.6) and the creation of output and library for the Decision Support System (T3.7). A new software tool was developed able to identify cavities from simulation results delivered in specified data formats with features such as the automatic suggestions of rescue paths. This function sorts the cavities by size and connects the centre of gravity of the largest ten cavities by searching for the shortest distance between theses points. Finally, the shortest distance to the exterior is searched and indicated. T3.7 based on the analysis performed under T3.4 contributed by creating a database containing the initial buildings, the different collapse cases and the pre-simulated cavity files, each as .obj file and in the native file format. In addition to this, a short report was provided for each building.
WP4 commenced in the first period of the project focusing on 3D data mapping. The role of 3D mapping of collapsed areas in INACHUS was to provide the USaR personnel with accurate and timely 3D models to support situation assessment, localization of survival spaces and rescue paths and planning of actions. To be able to do this as efficiently as possible, a strategy is needed for how and when the respective 3D mapping systems should be used and how their respective strengths should be exploited, so as to offer maximal support in the search and rescue phase. T4.1 contained 2 distinct research focus area: work with the International Charter “Space and major disaster”, and the dasymetric population mapping. The research approach was initially given with a focus on Europe, yet aim at ultimate global applicability, not least because of relevant technical and organizational developments elsewhere that can be of benefit to Europe, but also because European strengths and assets can be fruitfully deployed in extra-European settings, enlarging the influence of Europe. Recent disaster events that took place outside of Europe (e.g. the 2015 Nepal earthquake) also constituted interesting cases to test some of the approaches being developed. The population mapping component of the INACHUS project has several different components to be considered. This is because remote sensing has been continuously used for population mapping in a dasymetric mapping approach, contributing through several proxies of population (e.g. built-up areas, nightlights in satellite imagery, land use land cover maps, among others). This approach having the target scale indicated at 50m spatial resolution, requires datasets with suitable resolution, in the case remote sensing data is to be considered. The tests carried out by ITC focused on the use of two main sets of data. A high-resolution urban land use land cover map from the Copernicus Programme, the Urban Atlas 2012 (UA12), available for 695 European cities; a building and household surface covering 3 Portuguese cities with the resolution almost matching the UA12, however with a general coarser resolution, especially in peri-urban areas. The experiments conducted considered three main different situations, where a dasymetric mapping was approached and used. This intended to simulate a situation where: 1) only urban LULC maps is available, 2) when there is information regarding the buildings which can be coupled with the LULC maps to determine the building function and 3) considering the households as a unit, which cannot be directly observed through remote sensing sensors. In this last situation the UA12 was also used to enable the comparison between all the approaches. As observed, to consider the household as proxy for residential population presence overall gives the best results. This shows the sensitivity of dasymetric mapping approaches to the used datasets as proxies of population, especially functional surface used. Even when considering an urban LULC map coupled with a building surface with the buildings divided by the number of floors still results in errors of up to 38%. To consider only the LULC maps as a proxy for population made the generated population map unusable. It seems that, at least for the considered cities, the household approach is the only one with results that may be considered. The function is of extreme importance in urban and high-resolution dasymetric mapping approaches, where an urban LULC map may not be sufficient to provide population maps at a resolution of 50x50 m².
The population maps obtained before must be coupled with early damage information. A thorough examination of the potential contributors of these graded damage maps to the INACHUS project was performed. The International Charter due to its copyright restrictions does not have the general authority to release of their reference and grading damage maps in GIS-ready format, as well as the used satellite imagery, to the public. Namely the vector information, since the imagery may be bounded to commercial agreements between these institutions and private satellite imagery providers such as IKONOS. The institutions and organizations contributing for the International Charter may decide, however, to publicly release their vector data, independently from the charter. UNOSAT, member of the International Charter, makes the vector information public through their website. However, UNOSAT is not involved in all the activations of the International Charter. The recently established Copernicus Emergency Management Service (EMS), also aims at disaster response, producing reference and damage maps for any area of the globe. Likewise the International Charter it may acquire private satellite imagery as base for their reference and grading damage maps, apart from the use of Sentinel series satellite imagery. In 2015, the Joint Research Centre of the European Commission, has contracted the Compagnia Generale Ripreseaeree (CGR) to acquire nadir and oblique images in a rapid mapping and response context, which are then assessed by the EMS. Contrasting with the International Charter, the EMS provides the vector information regarding all of their activations publicly, through their website, easing the incorporation of such information in GIS environments and consequently used in INACHUS.
Within T4.2 acting as the core 3D mapping task of the project, two different techniques were studied for collecting 3D data for disaster areas: 3D laser from airborne and ground measurement and 3D from airborne imaging. Digital elevation/surface models (DEM/DSM) of high resolution and high quality were fused from these two approaches (laser imagery and normal imagery). The images from the visual sensors can be draped onto the 3D data, thereby enriching the 3D models and making them easier to interpret for an operator. One main motivation to use two complementary data acquisition techniques (laser, images) is that both platforms can be deployed individually. The subsequent data interpretation is possible on a 24/7 basis if only one of these techniques is available. If, however, both are available and fused (see task 4.3) even better and more reliable analysis is possible. In order to provide enhanced visions to the USaR team in all weather conditions (rain, snow, fog, haze, dust wind etc.) two kinds of sensors to fulfil the target were distinguished: (i) large Field of View sensors (often passive visible/IR Camera or Radar systems) and, (ii) high spatial resolution and small FOV optical laser sensors prototypes. FOI was involved to obtain 3D data point clouds from the ground and ONERA collected 3D laser data from drones. Airborne sensors provide wide-area coverage while ground-based sensing produces dense high-accuracy data to complement the laser data from the UAV. First experiments were done in Toulouse, France in March 2016. The main objective was to share a common scenario and to identify the risks in using three different techniques on the same field test. During the third period of the project, the INACHUS consortium organized experiments to illustrate the concepts and to display first 3D data to the end-users in real-time. These experiments were shared with all the WP4 technical partners on a common scenario in Lyon, France (pilot 2 of the project). Real-time registrations were done on the 3D point clouds collected by the different systems. In Task 4.2.2 one of the indicated objectives was a view planning framework that allows a complete 3D reconstruction, which also covers the façades of the buildings. In spite of the fact that damage detection is not a part of this task, a view planning tailored for the Urban Search and Rescue (USaR) teams must consider early damage evidences (related with Task 4.5). The objective of USaR teams is the detection of buildings that may entrap victims. Therefore, the view planning must be able to discard non-damaged areas, while focusing on the partial and total collapsed ones. To achieve this, an incremental approach started being implemented within INACHUS, where earlier damage evidences are used to prioritize the following ones. The task was divided in two main sections:
• 1st: a nadir flight to have an overview over the scene to:
• 2nd: oblique views considering hot spots determined earlier
In March 2016, Toulouse, WP4 tested all of their sensors. Both nadir and oblique views were acquired. In the following link an online visualization of the point cloud is presented: December, 2016.
ICCS established workflows for fast and reliable image-based 3D modelling of disaster scenes using commercial (Agisoft PhotoScan) and open-source (MicMac and MeshLab) software. The objectives of the conducted research were (a) the a-priori establishment of a proper workflow for fast and reliable image-based 3D modelling and the a-priori establishment of the parameters that have to be set in each step of the photogrammetric process, so that a predefined plan can be applied in emergency situations, without wasting valuable time in the field for the determination of the pipeline and the parameter tuning; (b) the evaluation of commercial and open-source software implementing image-based 3D modelling. Several experiments for the establishment of 3D modelling workflows for USaR purposes via earthquake imagery were conducted and a thorough analysis on the parameters of the various modelling steps was made. Two UAV image datasets of a small region in Mirabello, Italy, around the Church of Saint Paul, acquired after the earthquake of 2012 were used in the experiments. PhotoScan and the combination of the open-source tools MicMac and MeshLab were used. The performance of the tests was evaluated in terms of time required for the creation of 3D textured models and number of generated dense cloud points and comparisons were made among the generated 3D models. Also, a study on multi-scale 3D modelling of damaged areas was performed. The generation of models at different levels of detail in an emergency scenario can be adopted in the case that a first draft 3D model of a scene is required to visualize the areas with damages (low resolution model) and detailed 3D models of areas with damages are needed to be used by USaR teams to locate possible victims. Based on previous tasks developments and achievements T4.3 focused in 3D data fusion. 3D data point clouds were collected during pilot 2 and were fused in a unique coordinate system. The outcomes of this task are detailed in D4.2 including the testing and refinements of 3D registration algorithm using multiple datasets;, final adjustments to the 3D data registration algorithm based on experience from tests during pilots, registration of the aerial laser scanner data, successful automatic fusion of ground-based laser scanning data with 3D photogrammetry models, and set up a tool for evaluating 3D registration accuracy. In addition, a tool for automatic construction of 3D occupancy grid from registered ground-based laser, as a means to high-light unexplored areas and segment possible voids was created. The dataset was exchanged with T4.5 for post-processing and data exploitation. T4.4 contributed in WP4 by developing a scenario filtering facilitating work under T4.5 where 3D data were exploited though semantic analysis and model matching. The design and implementation of the filtering approach was finished by the end of the second period. The first step was to analyze the models stored in the existing database (WP3). Then based on the format of these models a filter was implemented considering several parameters (soil, etc.).
In order to split up the tasks technical challenges (TC) have been defined and for all of those results were obtained:
TC 1: Initial scene annotation/simple features
TC 2: Model matching (link to WP3) and cavity identification based on WP3-model
TC 3: Semantic data analysis, without damage model from WP3
TC 4: Data exchange to COP
Part of the results achieved are described hereafter for each TCs.
Flight planning app – automatizing the light-weight (lw) UAV image capture and the damage detection routine
The app allows to automatically perform the lw-UAV flights and also the damage detection from the collected images in a near-real-time approach. Efforts on this period were on the testing and integration of the several components of the system such as the damage detection procedure and the communication between the UAV and a ground station for a faster processing. The developed procedure allows a flightplan to be executed, with real-time streaming of the image data and their processing, which means that an orthorectified image mosaic with initial damage indications is provided as soon as the lw UAV lands.
Multi-resolution feature fusion for the image classification of building damages
The INACHUS system takes advantage of remote sensing imagery at different geographical scales to derive the damaged state of a given region. This is performed with satellite imagery that allows for first responders to have an overview of the extent of the damage within a given region, delineating rescue efforts accordingly. However, these image data do not allow identifying damages on the façades or smaller signs of damage due to the low resolution. Aerial imagery captured from manned platforms enables the surveying of the façades and in higher resolution when compared with the satellite. Nonetheless there may be areas which are occluded from this aerial flight due to urban morphology for example. The unmanned aerial vehicle can then survey these occluded areas and at a higher resolution than the aerial manned ones. Currently, the image classification of building damages is performed considering separately each of the resolution levels using convolutional neural networks. In this approach we assess the impact of considering this multi-resolution image information in the image classification of building damages.
To accomplish this, a multi-resolution feature fusion using convolutional neural networks was used. Three different feature fusion approaches were tested. These were designed to capture different feature information from each of the resolutions network. Overall the multi-resolution feature fusion approach improved the accuracy of the image classification of building damages. This was also the case when considering a new geographical region as validation of the approach. The improvements in accuracy reached 2% considering the satellite and unmanned aerial vehicle resolution level; while the aerial manned imagery decreased the image classification accuracy when comparing with the standard approach.
Data Fusion Mediation server
The data exchange with the data fusion mediation server (linked with WP8 activities) is done through JSON files. The server needs the WP3 cavities and rescue paths. WP4 cavities and rescue paths are also needed when 1) there is no model matching between WP3 collapse-simulations and, 2) in the case the collapse simulation does not match the remote sensing derived models. The generation of the JSON files in the appropriate format were reported in the previous period. In the current period the focus was on how to feed these files to the Data Fusion Mediation Server. This file transfer, by indication of WP8, was performed using POST requests, a request method supported by HTTP.
Wide area surveillance is performed thanks to 3D laser scanners embedded on fixed wing UAVs that cover large areas. 3D point clouds are transmitted to the USaR ground-station and can be displayed on the Common Operational Picture (COP) to give a global view of the area. For smaller areas, such as a city block, 3D models of buildings, rubble and cavities can be retrieved from high resolution 3D data acquired with smaller laser imaging systems embedded on rotary wing UAVs by automatic or piloted navigation. 3D point cloud, fused visible and infrared videos are real-time transferred and displayed on the COP and USaR devices.
Experiments were organized in Roquebillière, France to validate the INACHUS tools. Partners, end users and USaR teams had were involved in a common scenario. A significant time reduction related to the USaR phase can be achieved by wide area situation awareness solutions. 3D laser imaging systems embedded on UAVs can operate in all weather conditions at day or night. The 3D laser scanner performs the strategic surveillance of the environment for various worldwide operations.
WP5 commenced in the first period of the project and concluded in the fourth period. Within INACHUS, a range of victim localisation solutions was developed. These solutions comprise surface sensors (sensor devices to be used and placed on and around the rubble) and the INACHUS snake robot, the latter being a versatile sensor deployment platform, being able to enter the pile of rubble enhancing the technical search capabilities of the end users. The surface sensors are:
• The mobile phone detector, which is a device that seeks to find the approximate location of victims through the signals emitted by their mobile phones. Four methods for victim localisation have been studied, with three effectively applicable on-site:
o The "network" method targets to locate a single phone by reading its list of the six neighbouring mobile network cells and using this information to triangulate its position if the base station location of the listed cells can be obtained. However, this information is not open data. Consequently, this method could not be investigated and tested further.
o The "triangulation" method is dedicated to locate multiple phones at once with a post-processing step: several measurements (complete scans of the environment) are sampled at different locations. The approximate victim location is subsequently inferred from these spatially distributed measurements of signal strength by means of a triangulation algorithm.
o The "proximity" method is dedicated to locate only one device at a time, directly on the field: a "silent" call is made by the prototype, thus entering in continuous communication with the device. By regularly sampling power measurements, the position of the signal source, i.e. the mobile device, can be discovered.
o The "ringing" strategy in which a mobile device can be forced to ring can be considered as an additional aid in finding victims simply by acoustics. This approach could be especially helpful when used in combination with the acoustic sensors (Task 5.4).
The prototype developed was tested successfully during the pilots organised within the duration of the project. Furthermore, a graphical user interface for location presentation in the COP as pinpoints as well as an integration of a density-map in the COP to visualise the concentration of mobile devices were developed. A detailed analysis on the personal handset positioning task is provided in D5.1.
• Ground-based seismic sensors forming a network of distributed vibrational sensors (ground sensors) that pick up vibrations at different points. With this system, vibrations generated by victims through knocking, hitting, etc. can be automatically detected and positioned. The interference of background noise is reduced through an adaptive background suppression technique that improves the usability of the system in real, noisy environments. The system was demonstrated at a field trial in June 15 2016 and was well received by the end users. The system presents detections as an image with colours indicating the location of the detected signals (a.k.a. “heat map”) gives the users a better overview and understanding of what is going on at the site. The system can continuously record data and then analyse them at a higher speed. In this way, a user can “fast forward” through previously data or reanalyse them more carefully or with other parameter settings. Knowing the position of the seismic sensors is needed for the heat map display but not for the background suppression and detection functionality of the system. Positioning of the individual sensors can be done by marking the position in a 3D model or a map (if available), using a positioning solution (e.g. Real-Time Location System, RTLS) or measured manually. With highly sensitive seismic sensors, very weak signals can be detected. However, it can be difficult to distinguish these signals from noise. Finding suitable threshold and parameter values requires extensive testing in a large variety of realistic scenarios, which is not possible to do within the scope of INACHUS. In practice a threshold has to be adjusted by the operator that governs the trade-offs between detections and false alarms in the scenario at hand. The signal detection module is still on a basic level and can be further developed to detect periodic patterns (e.g. repeated knocking) or signals reoccurring at the same location. As a spin-off of the fruitful discussions with end-users it was decided to design a plug-in hardware between the INACHUS seismic sensor system and the Delsar seismic/acoustic listening device in order to add automatic detection and noise cancellation functions to the Delsar system. This feature was successful demonstrated during the pilot 4 event. Moreover, an acoustic system has been built intended for mounting on the INACHUS mobile platform to enable communication with victims trapped in rubble. The system is made from standard audio components to facilitate integration on the platform and was successfully demonstrated to end users in a field trial.
• The Surface Radar, whose purpose is to detect trapped victims by their movements, e.g. the chest moving when breathing. The penetration capabilities of the radar enable it to 'see-thru' objects and therefore making it possible to detect movements of trapped victims behind obstacles. This helps Urban Search and Rescue (USaR) teams to increase the possibilities to find survivors as well as to speed up the whole procedure. The Surface Radar is an advanced UWB radar with nine transmitters and nine receivers with corresponding antennas, all simultaneously active and controlled in a timed manner. It is based on an earlier design that is further developed and improved by INACHUS. It provides information on whether there is a human or not in its scanning scope by measuring the slight movements from the trapped victim. It enhances the high range resolution of an impulse UWB radar with beamforming to get better location information of the target. The Surface Radar is a stand-alone system that is also integrated in the INACHUS environment to supply information about searched areas and detections. The Surface Radar is portable and of the size of a common middle-sized suitcase. It is wirelessly controlled from a ruggedized tablet computer by a single operator. The sensor is placed on top of the rubble pile. For best results a flat area with a minimum of air gaps between the sensor and the material on the pile should be found. The operator starts the measurement from some distance, 2-3 meters away, to avoid undesired influence on the sensitive measurement. The vibration environment on the site needs to be low in order to utilize the highest sensitivity. The Surface Radar is simple to operate and has only a switch for power and a connector that is used to charge the internal battery and if a wired data connection is desired. The device is ruggedized to endure rough environments and handling. In INACHUS deliverable D5.7 a more complete description of this topic is given.
The INACHUS Robot Snake carries the following sensors:
• A thermal long-wave infra-red (LWIR) camera that was used to detect the temperature variations due to heat produced by the victims. In order to detect and locate survived victims, several different sensors were used. These included mobile phone detectors, gas detectors (chemical sensors for human biomarkers), LWIR sensors, optical sensors, seismic and radar sensors. These sensors exploit different physical principles in order to maximize the detection probability and to narrow the identified volume of interest. In addition, a prototype of a snake robot mechanism is developed in the INACHUS project that is equipped with a miniaturized LWIR-camera. This thermal sensor in combination with robust and reliable detection algorithms developed is able to detect humans by thermal radiation. For efficient victim detection in uncontrolled emergency environments, the system exploits the thermal information captured by the LWIR sensor. In order to define with high accuracy which parts of the thermal image should be considered as foreground/background an area detection methodology is applied in order to identify regions with the temperature of a human body (34ºC-41ºC). This temperature range is adjustable according to the user needs and the specific targets to be detected. Objects with the desirable temperature appear in the thermal image with different pixel values. A detailed presentation of the infrared camera used in INACHUS is included in D5.3.
• The Electronic Nose Sensor (e-nose) is located in a compartment of the INACHUS robot snake. The e-nose can provide information about the concentration of gases originating from metabolic activity, as well as indicators of some specific hazardous conditions. E-nose was prototyped under T5.2. Three prototypes have been developed. The main objective for the e-nose, which is human presence detection, was successfully validated during the first pilot test with the second prototype device. The design progressed further with the third prototype including more, even non-mandatory, features while preserving the validated qualities of the second prototype. Tests performed with the third prototype show it has the same human presence detection capabilities which were validated during the third and fourth pilots. It also adds air related hazard detection such as toxic (CO, CO2, H2S, O2) and combustible air environment detection. The third prototype system is compatible for integration in the INACHUS robot and the INACHUS framework. The third prototype of the electronic nose, which is the final prototype developed for the INACHUS project and was the one planned to be integrated in the INACHUS robot (snake-like robot) in WP8, for Task 8.3. The device is designed to include all the technically feasible end user requirements in the constrained space of the INACHUS robot. For these reasons, a completely custom designed device was built using only a few off the shelf components. The electronics and mechanical design were made from scratch. The design of the electronics started before the mechanical design but the completion of both of them required constant exchange of information. The mechanical dimensions of the boards are a result of the allocated space and mounting positions assigned from the mechanical design. The allocated size for the boards in the mechanical design depends on the circuit complexity and the electronic component selection. Software design is relatively independent of the hardware design with the exception of the low-level firmware: its design requires a completed schematic diagram and its implementation is most favourable when at least the assembled PCBs are available. The control interface software of the electronic nose that was hosted on the control interface tablet of the INACHUS robot not planned to be available at the expected delivery date of deliverable D5.4. A standalone interface is available and used for testing purposes. Nevertheless, the design of the integrated interface is mature. Completion of its design and its implementation in the specific platform was successfully completed as part of WP8, Task 8.3. The 3rd prototype’s design is based on the sensor selection and system architecture of the successful 2nd prototype and is extended to implement the non-mandatory end user requirements that deal with safety aspects. The safety aspects that are implemented are the detection of some specific toxic and combustibles atmospheres and require the addition of three extra sensors, for CO, H2S and Combustibles. The CO and H2S sensors are to be in different packages so that there is the possibility, in the future, of building different configurations that can detect other gases. The implementation of the non-mandatory safety aspects is considered to be of added value for the electronic nose especially by the end users. This benefit comes with a significant cost that can be understood by the end users in many aspects of the tool: Size, weight, volume, training, operating, servicing and certification are affected. For the 3rdprototype, replacing the sensors in the field is not an option. The storage of the e-nose does not require the presence of a power source to keep the sensors biased. A combination of sensor selection and circuit design keeps the electrochemical sensors biased even without power. Little or no recovery time is needed for the sensors to be operational after storage. Some design features were included mainly in the electronics that would allow for the configuration of a standalone e-nose with the utilization of some additional modules. These features allow the connection of external devices to serve as the user interface. A tablet could be connected to one of the serial communication ports or a low-cost dedicated interface with vandal proof buttons, LED indicators and a buzzer could be connected to the exposed General Purpose Input/output pins available at an expansion header and the extra actuator output (speaker). The possibility of implementing a standalone e-nose sensor was investigated and a high-level solution was proposed but its implementation was not realized as there is no such obligation and the required resources are not available.
• The Robot Radar, a subsystem in the INACHUS robot, that detects movements of buried victims. The system is a CW-radar operating at 1.2 GHz and utilizes the Doppler effect. It detects movements but has no distance resolution due to the single-frequency that is used. The system has antennas in five different directions which the radar transmitter and receiver rapidly is switched among. This gives a direction estimation of the detection. One big challenge with this radar system is the requirement of the small physical size to fit into the INACHUS robot in combination with high penetration capability that generally means low frequencies which in turn requires large antennas. FOI had the task to design the antennas, each with a maximum size of 30x60 mm2. Each antenna consists of aperture coupled microstrip patches. The low frequency and the small size required a relatively high permittivity of the dielectric material used for the antenna design. TMM6 with epsilon ≈ 6 was therefore chosen as dielectric for the antenna. Cinside designed the RF- and controller electronics that were also restricted regarding the physical size but that did not pose any real problems. The power consumption of the radar system is quite low, but the voltages need to be high-quality and low-noise so they are generated internally from the 24V robot system voltage. Several versions of the radar hardware were developed and tested during the project. A spin-off product was the StickRadar that uses the same radar system but instead of being mounted in the robot it is mounted on a telescopic stick. With its own specially developed operator interface in a rugged wireless Android device, the StickRadar is a stand-alone system that also can communicate with the INACHUS environment or other systems.
• Apart from the aforementioned sensors, the INACHUS robot snake can also carry two video cameras for navigation (one at each end of the INACHUS robot snake) and microphone and speaker for communication with trapped victims.
In the last task of WP5 T5.6 the INACHUS robot design, locomotion mechanism and operator interface has been implemented. More specifically and as detailed, in D5.6 INACHUS robot acts as a mobile platform with the objective to carry several sensors inside the rubble and closer to the victims. Some of the sensors are standard (Video cameras, Audio Communication System), while others are more advanced (Robot Radar, IR-Camera, E-Nose). Audio Communication System was not planned in the description of work; however, it was clear from the first end-user workshops that this functionality is very important. Therefore, it was decided within WP5 that INACHUS robot had to include this device. All the robot parts were manufactured in the 3D CAD design software SOLIDWORKS. Most of the parts were manufactured by 3D-printing them in ABSplus material using the uPrint SE Plus 3D printer from Stratasys. This approach allowed to iteratively design parts, 3D print them, and then test them. When plastic material was not strong enough, the design was updated including metals parts to strengthen the mechanical properties. The design of the track module consists of the following four groups of components.
1. Rubber track and drive wheels.
2. Motor module.
3. Driveline.
4. Chassis.
The first link (module) of the INACHUS robot is the robot head. It is composed by four main parts: the yellow lid/box containing and protecting most of the sensors and electronics; the black track chassis connecting the two tracks to the lid/box; the tracks modules and the joint module. The central module has the largest available payload capacity among the three link modules. It is called standard module because it can be replicated to obtain a robot with more than three links. The electronic nose is the main payload of this module. The tail module and its joint mechanism carry the rear end video camera, the Moog multiplexer, the RTLS beacon, two LEDs, the power distribution board under the yellow cover. In addition, two electric motors are inside the joint and are driven by the embedded custom-made motor controller cards. The motor driver (VNH5019) is a board from Pololu (https://www.pololu.com/product/1451) and was bought from www.farnell.com with the product number 2475453. The module contains a fully integrated H-bridge that can be used for bidirectional speed control of a single brushed DC motor. It has a small size and is relatively low cost. It runs on a supply voltage in the range 5.5-24V. A custom-made circuit board utilizing a microcontroller (LPC1549) from NXP was made to run the low-level control algorithms based on commands from the CAN-bus. A power distribution board was designed and built by SINTEF. There are two identical power distribution boards in the INACHUS robot. The first board is in the robot head module and the second is in the robot tail module. Both communicate over the CAN-bus and supply 12V to cameras and LEDs and provide 5V power to raspberry pi to RTLS beacon and to Moog Multiplexer. The Operator Control Unit (OCU) is one of the main components of the INACHUS robot prototype. Inside the case we find a power supply, an industrial PC, specialized communication hardware and plug connectors. These hardware components are necessary to power the robot, run the control algorithms, connect the robot with its sensors, and communicate wirelessly to the TAUROB Tablet PC and the INACHUS infrastructure. The hardware is only one part of the system; proper software and a robot operator are the other parts necessary to operate the robot. Ever since the first interaction with the USaR teams, at the end-user workshops held in the INACHUS project, it was clear that an important part of the robot was going to be the human-machine interface. The end-users highlighted the importance to have:
• a solid and rugged user interface (Technical Requirement TR-N-RS/25-1), while simple to use (TR-N-RS/05-1, TR-N-RS/08-1);
• camera views, feedback and alarms from sensors visible in the GUI (TR-N-RS/18-1, TR-N-RS/16-1, TR-N-RS/17-1, respectively), both in bright and low light conditions (TR-N-RS/21-1);
• camera views showing a horizon to help with orientation (TR-N-RS/28-1);
• the possibility to operate the robot from several meters away from the entrance point (TR-N-RS/29-1).
To be able to fulfil the described requirements, different solutions were investigated, such as ordering off-the-shelf components, designing parts, and assembling at SINTEF. Finally, TAUROB Tablet PC was selected given that it is a provider of teleoperation solutions for Chemical, Biological, Radiological and Nuclear (CBRN) first responders, Explosive Ordnance Disposal (EOD) teams, fire-fighters and search and rescue teams. Further details on the INACHUS robot are provided within D5.6.
WP6 was the dedicated Work Package of INACHUS for decision-making and planning matters. WP6 commenced on M6 of the project included the following task activities resulting in two deliverables D6.1 on SaR-ESS interfaces with other systems and on Emergency Support System including Common Operational Picture and accompanying documentation. More specifically, The Search and Rescue Emergency Support System (SaR-ESS) represents the cerebral centre of the INACHUS project, and controls and operates the modules and the subcomponents that build up the INACHUS system. Furthermore, measurements and data from various sensors and legacy systems is collected by the SaR-ESS through secure data flow pipelines. The design of the SaR-ESS reflects the effort to pre-empt malfunctions and miscommunication of INACHUS as a whole system, but also to introduce innovative technological solutions, integration techniques and interfaces that come from the rich scientific and technical background of the project consortium in order to facilitate and contribute to future projects as well. The figure below illustrates the Search and Rescue Emergency Support System (SaR-ESS) architecture that has been drafted by the respective partners and which has been constantly updated as the system evolved to its final form. T6.1 and T6.2 are related to the development and deployment of the Multi Source Information Fusion Engine (MIFE) and its integration with the SaR-ESS. The Multi-source Information Fusion Engine (MIFE), is a data fusion module in which data from the different sensing elements coming from the field are combined, examined and processed. The MIFE includes the Data Fusion Mediation Server, which incorporates expert reasoning applied on the collected data from various sensors, using the INACHUS Ontological Model as a basis for the structure of the INACHUS data and their interrelations, and conforming to intelligent algorithms and pre-defined rules. The end result is the location of possible survivors. This result is published in the Fusion Backend, which in turn sends this information to the COP as an alert message. The data is also stored in database for possible future reference. The development of the MIFE API commenced based on the Java Spring Boot framework. The implementation of the MIFE connects to the RTC-Gateway and receives data from sensors. Listener classes have been implemented to receive the sensor data and loader classes to extract data from JSON files. In addition, loader classes have been implemented to extract cavities data from JSON files, which are sent to the Gateway to test initial rules on the Complex Event Processor (CEP). The Data Models for the Cavities and Alert messages was also defined, and the MIFE was updated to output alert information to RTC-Gateway to be displayed on the COP, which includes amongst other information, the possible location of survivors. SaR-ESS Portal consists of a group of components that together they form the administration dashboard of the INACHUS Platform. Alongside with the COP Dashboard, they shape the means of controlling and organizing a search and rescue operation. With the SaR-ESS Portal the Rescue operator can perform tasks, among others, such as sectorization, worksites’ management, teams’ management, monitor of the operation, manage the INSARAG Forms submitted at the field of operation etc. The SaR-ESS Portal is divided in two major parts the frontend and the backend. These two components communicate with the COP server, the authorization server and the mobile application. SaR-ESS Mobile application refers to an android (version 5.5.1 onwards) application oriented for tablet devices (with screen size 7” and up) which is focusing in supporting the work of the rescuer at the field of operation. The Environment Services (ES) encompasses the components that are necessary to serve map data and operational information to the instances of the Common Operational Picture (COP). This module consists of two components: The ES server, which provides GIS data and other geo-localized datasets, and the COP server, which provides the operational data and necessary metadata for the COPs. The Common Operational Picture (COP) is a map-centric supervision tool with advanced visualization features and annotation capabilities for collaborative management of crisis relief activities and real-time supervision of the crisis-affected region. The COP gives an overview about the scene in 2D and 3D, and is enriched with operational data in form of annotations such as text, icons, geo-localized photos, as well as the sectorization plan, worksite positions and team locations. Moreover, the datasets that are produced by WP3 and WP4 can be displayed and navigated directly on the map in the 3D view. The visualization of all this information in the most legible and user-friendly way possible is the challenge in the development of this component. The COP is available as a desktop application and a mobile application for Android.
Crisis management poses specific technical challenges that INACHUS addressed and they are related to:
• the large number of network subscribers that need to be served (crowd and first responders)
• the lack of prioritization schemes to support the first responders in most types of wireless communication (currently only TETRA claims to adopt prioritization schemes)
• the scarce nature of communication resources (i.e. the limited total wireless bandwidth available for a specific cell/access point) and the bottlenecks that appear in cases of congestion (e.g. in 3G mobile, signalling resources can be congested before the actual data bandwidth is consumed)
• the total time required for establishing a two-party or a multi-party call (normally for 3G mobile in the order of 5-8 sec, exceeds 20sec in cases of congestion with similar number in 4G band) and respectively the time required for one to register a wireless client to an access point (cases of WiFi/WiMax)
All these aspects were thoroughly analysed and tackled under WP7. All tasks in this WP aimed at the deployment of a reliable and stable communication platform. The communication platform that INACHUS developed plays a central role in the topology envisioned by the INACHUS project, and manages the seamless interoperation, the interconnection between other networks (cellular, etc.), as well as provide redundancy and recovery functionality. A multi-layer architecture is followed for the communication and the routing as well as the mobile gateway functionality. The mobile gateway can be accessed by the operator, from the control centre as well as locally by the first responders, when/if needed; as well as remotely controlled and maintained by specialized maintenance staff. For extending the capabilities and range of the crisis network in case of failures and/or congestion and to provide a distributed architecture, portable gateways and a long-range communication solution are envisaged. These gateways can be positioned according to relevant and appropriate network dimensioning, even deployed by personnel where needed (i.e. mobile gateways) and they can be automatically or manually switched on. The gateways support the functionality of any other mobile USaR center as an alternative and cost-effective solution in an integrated ad-hoc manner but with redundancy, extendibility and security capabilities to assure robustness, flexibility and reliability. The communication platform is complemented by a novel Real-Time Localization System (RTLS) - based on wireless communication and in particular focused on robustness and efficiency - which is crucial for USaR operations. Of special concern is the localization of targets under the rubble, which could be of irregular density and composed of heterogeneous materials (maybe metallic) that greatly increase the signal distortion and multi-path effect. Special focus was placed in technologies, protocols and algorithms resilient to multipath errors. Combined approaches were envisaged in order to compensate the weak points of the selected technologies. According to the momentary needs, the system should switch automatically between different technologies and adjust parameters (such as frequency bands) to give the best-effort localization.
The INACHUS localization system (RTLS) consists of clusters of devices with locating capabilities called beacons. Their purpose is to work in a cooperative way to relatively locate other beacons or tags. Absolute positions are given relatively to a known reference point. Beacons can be deployed as a separate device or be integrated within INACHUS mobile gateways.
In order to increase the accuracy and reliability of the system two localization domains were defined: surface location and underground location. Both domains work cooperatively to give relative and absolute positioning, although each of them targets specific challenges.
• Surface RTLS: probed state of the art RTLSs targeted to indoor environments were researched as the scenario expected on the surface are similar to an indoor environment. Integration with GPS was also explored to enable global positioning when environment changes from indoor-like to open-space/outdoor. The surface beacons were designed to allow a progressive, fast, in real time deployment and with little or no configuration at all. The system creates dynamic clusters and locate each element (beacon/tag) iteratively. Apart from the surface location, the Surface RTLS is the backbone and reference system for the positioning of the robot under the rubble.
• Underground RTLS: positioning an element in an environment full of irregular and heterogeneous obstacles is prone to high range distortions and multi-path effects. Technologies more suited for multipath-prone environments (such as UWB) in addition to error-aware algorithms were targeted in order to estimate and reduce the undesirable accuracy degradation. Additionally, a specific underground localization infrastructure was designed to bind the error. As the most effective approach, the underground nodes were attached to the power wire of the robot. This deployment is made at small intervals (≤1m) to minimize the ranging errors and increase the localization granularity.
WP8 deals with the INACHUS system integration activities. The integration process was defined for the INACHUS platform as a progressive and iterative process where technology components were first individually developed and verified (functional and inter-operational) and subsequently integrated to conform and validate the ready-to-deploy INACHUS framework platform. The general objective of WP8 was therefore to integrate all the hardware and software components defined for this INACHUS system framework. Partial objectives were the supervision of the execution of the Integration Plan, the definition and execution of the Integration protocol and the traceability and quality assurance of the Integration. The methodology of this integration process was structured in two steps: The first, and preliminary step, was focused into control that the individual components were ready for the integration. The second was the integration itself. For the first step, it was necessary to have functional validation traceability of the components and the compliance validation with interfaces (defined in the corresponding Interface Control Documents). The second step was the progressive integration of the components in a platform system as it was defined in the deliverables D8.2 and D8.3 following the Integration Protocol that was also defined as deliverable D8.1. The progressive integration was structured in partial integration levels that in the case of the Robot Snake and standalone sensors were supported in a specific component for integration: The Embedded Communication System. These partial integration Levels and finally the Integrated Platform was technically validated against technical requirements and use cases scenarios.
This methodology was structured around five tasks: T8.1 Functional and Validation tests before integration, T8.2 Embedded Communication system for the Mobile Platform., T8.3 Installation of sensors and communication module on the Mobile Platform, T8.4 Integration of the INACHUS System Framework and finally T8.5 Quality evaluation and continuous common connector update.
Task 8.1 concluded with the first definition of the interface and communication protocols compiled in the corresponding Interface Control Documents (ICDs) and with the individual components compliance validation according that defined interoperations.
T8.2 concluded with the development and deployment of a specific software component for integration: the Embedded Communication System (ECS) ,defined first for the mobile platform and finally also extended for the integration of the stand-alone sensors (Mobile Phone Detector, Seismic sensors and Surface Radar):. The ECS acts as an internal interface between the robot sensors and the robot OCU and as an external interface between the mobile platform (and the stand-alone sensors) with the ESS/COP via the Communication Infrastructure and the Real-time Communications Gateway (RTG). Additionally, it acts as a data broker (in this case without Data Base) to access the Robot Snake and sensor data. The ECS application provides a common API for all sensors and for the Robot Control System implemented as a library/interface that hides all the complexity of the real time communication processing and interoperation. The ECS contains two interfaces: Local and Remote. The Local interface is used by the Mobile Platform processors (Robot OCU and sensors) to interact with the ECS and to send data to any internal interested clients. On the other hand, the Remote interface is used by remote clients (ESS) to access data produced by the processors by means of a protocol structured in four layers where the two lower layers are standard and the two-upper layer are specific protocols for INACHUS. WebSockets protocol and STOMP protocol are the lower layers, the INACHUS ECS Policy layer and the JSON Models & Metamodels are the top levels. The higher layer of the remote API interface defined is then the INACHUS Data Model architecture that is the final information interchanged with the MIFE, through the RTC- Gateway also via the STOMP protocol.
Task 8.3 was focused in the specific physical integration (mechanical, electrical & communications) of the Mobile Platform. As indicated, in WP5, the INACHUS robot was designed and developed. In task 8.3 the e-Nose, LWIR (infrared camera) and Doppler radar units, all also developed in WP5, were integrated on the INACHUS robot with the on-board electronics, communications and also with the RTLS developed a WP7.
The INACHUS robot is structured into three modules (Head, Central and Tail) with joints between them, where each module has two tracks (self-contained, integrating the motor and electronics) for improved propulsion/performance. The modules are connected by active joints, meaning that the joints are actuated and therefore it is possible to change the relative position of two adjacent links. This modular snake-like body allows for carrying several sensors on the modules. Some of the sensors are Commercial Off-The-Shelf (Video cameras, Audio Communication System), while others are the sensors developed in WP5 (Robot Radar, IR-Camera, E-Nose) and WP7 (RTLS). Additionally, the robot provided components to support the integration: communications, power supply and processing units. All these components, physically integrated with functionally interoperated by means of the ECS, constitute the integrated Mobile Platform.
The task 8.3 concluded with this Mobile Platform fully integrated and ready to interoperate as a subsystem in the INACHUS platform. The mobile platform, defined to be deployed in the Worksites, provides real-time information related to the victim’s location to the INACHUS backend applications (ESS in Sector Command/Base of Operations) via the ECS and the INACHUS communications infrastructure (WP7).
As indicated, Tasks 8.4 and 8.5 involved the core activities of the INACHUS platform integration. Task 8.4 encompassed the work performed in the technical WPs to bring the INACHUS System Framework composed of heterogeneous technologies and layers with variable degree of complexity. So, in this task all the modules previously developed and independently validated were integrated together progressively to build the INACHUS platform meanwhile Task 8.5 was focused on quality assurance of that integration.
INACHUS platform is structured around a high number of complex components with also a high number of interoperations, all defined in the corresponding ICDs. Some of the interoperations are simple file exchange mechanisms, like file sharing, but others present high complexity interfaces, like the real-time interoperations conformed by means of the ECS. In order to manage these different levels of complexity, a progressive structure of five differentiate levels of integration were defined, from components to subsystems, with a Level 5 where the platform was validated as a complete system in a series of testing scenarios supported in the Use Cases defined for INACHUS.
This integral functional/interoperability validation was performed following a strict planning in order to leverage the possible interoperability errors of each element/layer interconnection as soon as possible. The traceability of technical quality of this process was covered by confirming that the individual developments were functional and compliant with the technical requirements, by confirming that the components were compliant with the interfaces and protocols defined in the Interface Control Documents and by confirming that the subsystems and the Platform as a whole were functional, with composability guarantee, and also compliant with the technical requirements.
The WP8 concluded with the Integration Protocol fully executed, in physical integration workshops complemented with virtual Integration Workshops, the physical Integration of the Mobile Platform finished and the complete INACHUS Integrated System fully defined and tested ready for demonstrations. Thist last technical WP9 of INACHUS started during the second period of the project and ended in the fourth period. It included integration activities related to pilots which occurred with the following details:
• Pilot 1, Sweden: Testing and validation of system components in realistic environments
o Victim localisation tools, coordination tool prototype
• Pilot 2, France: Small scale catastrophe for performance assessment
o Wide-area surveillance tools (drones, laser scanners) to monitor collapsed buildings; Simulation tools for structural damage analysis and casualty estimation
• Pilot 3, The Netherlands: Explosion in an industrial installation and buildings collapse due to sabotage, terrorist attack
o Initial demo of the integrated INACHUS system
• Pilot 4, French / Italian border: Final testing and validation of the international adaptability of the system
o Final demo of the entire integrated INACHUS system
Potential Impact:
Potential impact (including the socio-economic impact and the wider societal implications of the project so far) :
INACHUS main outcome, namely the delivery of a robust, resilient and secure platform for the provision of specialised ad-hoc services, facilities and support for first responders that operate at crises scenes.(the INACHUS tools, sensors, communication systems and building assessment software) reaches a wide applicability at an EU level and provides important enhancements at the service of Civil Protection and Crisis Management and more specifically in Urban Search and Rescue activities. The following high-level potential impacts evidence usability and crisis functions of INACHUS:
The increased effectiveness of FR operations through the employment of INACHUS leads to:
- Increased situation awareness
- Fast response of the communication infrastructure (e.g. priority services and QoS)
- Increased communications coverage with ad-hoc networking capabilities,
- reduction of injury and loss of life of the FRs and ultimately civilians by gaining faster, more accurate and safer localisation information and supporting guidance information.
- Preparedness may be enhanced, as the INACHUS system may act a simulation software
- A robust Situational Awareness mechanism that allows decision makers at both strategic and tactical levels to make well informed decisions has been developed.
- The CWA initiated by INACHUS consortium orientates the development of guidelines and consensus reaching process on the most efficient and harmonized ways to deploy secure
- INACHUS focused on the cooperative idea of sharing know-how in crises’ early stages, helping bring all EU countries into a coordinated Search and Rescue Operations’ handling framework. Emergency organizations in Europe were therefore able to harmonize their procedures through a common crisis database.
Since project conceptualisation, 11 high level objectives/Expected final results have been foreseen that drove the entire work implementation from idea to delivery throughout the 48 moths of project duration. As explained previously the entirety of project objectives have been accomplished and delivered, hence in the following we report on high level accomplishments and their potential impact (including the socio-economic impact and the wider societal implications of the project so far). By successfully delivering the high-level objectives, INACHUS may claim on achieving a wide site of associated potential impacts as per below.
1. INACHUS successfully developed simulation tools for estimating the number and locations of survival spaces created after a structural collapse
2. INACHUS deployed successfully Decision and Planning Components for Advanced Casualty and Damage Estimation.
Related potential impacts:
By successfully delivering the above high-level objectives, INACHUS may claim on achieving a wide site of associated potential impacts as per below.
- Stimulation of innovation in USaR field is achieved in the EU and beyond.
- Emergency response crews can be provided with critical and timely information on damage in monitored facilities so that danger can be pin-pointed and emergency response directed with precision
- Disaster results in less human victims
3. Successful integration of existing and novel sensors
Related potential impacts:
- INACHUS introduces a cost-effective and of high-performance mixture of sensors resulting in accurate, flexible and faster USaR activities in the field
- European leading position in this market which gains momentum over the last years in various application domains
4. INACHUS advanced the state of the art by developing a robust snake robot mechanism together with a snake
Related potential impacts:
- To result in accurate victim positioning without risking USaR team’s members lives
- To enhance methodologies of searching for entrapped victims
- To pave the way for alternative means of USaR activities
5. INACHUS successfully addressed interconnection issues between the developed devices
Related potential impacts:
- to decrease time of reaction
- to drastically increase the efficiency of the relevant actors
6. Successful enhancement in data fusion and analysis techniques
Related potential impacts
- To provide the rescue teams with a clearer picture of the situation at hand and eliminating redundant time to search
- To determine escape route, so that rescue teams can reach these potential survivors and extract them from the debris in the minimum possible time
7. Successful INACHUS system integration
Related potential impacts:
- To provide a reliable and stable system supporting USaR activities in the field
8. Successful contribution to standards/best practices and guidelines for the USaR operations through strong user presence
Related potential impacts:
- To provide a common approach from all stakeholders in USaR robotic platforms interoperability issues
- To enhance the technical and procedural interoperability in USaR offered technologies
9. Consideration of Societal Impact, Legal and Ethical issues of the proposed solution.
Related potential impacts (Points of Obj. 1 also apply):
- To provide a secure solution covering all Ethical and Legal issues respecting USaR crews and entrapped victims’ fundamental rights privacy
10. Arrangement of several field tests
Related potential impacts:
- To provide a stable and reliable solution tested in real conditions to USaR and CP entities and organization
- To provide a solution tested in different scenarios by different USaR teams.
11. Development of training package and extensive training courses
Related potential impacts:
- To increase the trust of USaR teams to the INACHUS solution
- To familiarise and create a user-friendly support tool for USaR activities
Main dissemination activities :
Public awareness activities aimed to spread the information about the project to the broad community to set the ground for exploitation of the results or possible technology transfer. Major awareness/communication and dissemination activities included: Project public website, Six-monthly newsletters, Social networks to communicate with larger audiences Videos showing the INACHUS system at work at YouTube, Press releases, INACHUS leaflet and poster, Presentation of the results to the public at large, Project workshops, Presentations/ scientific publications.
INACHUS Website:
A project website (www.inachus.eu) has been developed in order to disseminate the project outcomes; it includes project objectives and background, significant achievements, technology news, consortium contacts, scientific publications and presentations, etc.
Project Newsletters:
Four newsletters have been produced that depict the project's main outcomes together with dissemination activities and upcoming events related to the project itself.
Social Networks:
Facebook https://www.facebook.com/pages/Inachus-USaR-Research-Project/1649612961918245
Average of 3 posts per months (280 followers and 520 posts)
Twitter https://twitter.com/InachusUsar
4 tweets or retweets a week (a total of 1107 tweets, 185 followers)
LinkedIn https://www.linkedin.com/grp/home?gid=8385769
A total of 122 connections, 77 discussions launched, 84 members)
YouTube Channel https://www.youtube.com/channel/UCBz08Jf7tVT08x5LevztXcQ
20 videos from technical partners.
Press Releases and Radio/TV Interviews:
8 Press releases during the project: for Inachus Events, Achievements and Fields tests. During the second field test held in France a journalist from a National newspaper (Le Progrés) made interviews and a paper, and during the third field test of INACHUS project, held in Weeze, Germany, a journalist from a Dutch Fire and Rescue magazine Een-Een-Twee made a visit on the pilot site to interview partners and end users. Those interviews lead to a newspaper article, published during the summer.
Two more journalists made a visit to the pilot site:
- A journalist from a German National Radio (WDR5) made interviews of ends users and partners during the pilot. Those lead to a report of 2min30s on Friday 20 April morning on WDR5 in the "Morgenecho". Follow the link bellow to listen to the interview: https://www1.wdr.de/radio/wdr5/sendungen/morgenecho/index.html
- A journalist from RP online made a visit on the pilot site and carried out interviews and pictures report on the victim localization tools. This leads to a paper in German: http://www.rp-online.de/nrw/staedte/kevelaer/roboter-schlange-sucht-verschuettete-aid-1.7528750
- Two press releases about INACHUS final conference: one in French to inform local press and one in English for International Press. Link on our website:
- TV interview on National TV Channel (France 3) about the large USAR exercise where INACHUS final pilot was mentioned. http://www.entente-valabre.com/blog/262/48/sauvetage-deblaiement-interview-du-lt-col-p-tosello-a-france-3
- Newspaper article about the National USAR exercise in a Regional Newspaper (Nice-Matin), collocated with INACHUS final pilot (see above)
https://www.nicematin.com/vie-locale/photos-des-pompiers-de-la-france-entiere-ont-choisi-larriere-pays-pour-des-exercices-grandeur-nature-277464
- An article in “Air and Cosmos” (National reference on aeronautic, space and defence) where the pilot 4 is mentioned (above).
- Others papers on general support were also published during the project lifetime: concrete.org Nice Matin
INACHUS Leaflet and Poster:
INACHUS leaflet was first designed and printed on M3. Brochure was designed and printed on M13. Technical poster has been designed as well. In addition to the technical poster in the format of a standing roll-up banner (80cm × 200cm), a version in the format DIN A0 has been prepared with the intention to present it at conferences, where A0 is the common poster format for poster sessions. The logo poster has been reprinted in 2 copies in M34.
Clustering with other projects and initiatives
Clustering with ICARUS and DARIUS projects has been performed this first year. Furthermore, INACHUS has also identified RECONASS and EUCONCIP as relevant projects. In the second year, INACHUS considered the option to organize a common technical workshop with SPARTACUS with a focus on the communication and localization technologies, but this opportunity was cancelled by SPARTACUS partners. First exchanges have been started with DESTERIO, MOBNET, TRADR and eVACUATE. Liaison relationships have been established with the MOBNET and eVACUATE projects. MOBNET presented its developments in a special session during the 6th INACHUS plenary at Freiburg. The INACHUS Exploitation Cluster workshop occurred in Weeze, Germany, April 2018 was formed by the following projects:
DRIVER+, IN-PREP, TOXI-Triage, CAMELOT, beAWARE, ANYWHERE, STORM, GHOST, CIP-SEC, SMR, SAYSO.
INACHUS Workshops:
The Final Conference was collocated with the INSARAG Regional (Asia, Europe, Middle East and Africa) meeting in Valabre, Gardanne France and was an opportunity for EPLFM to present INACHUS project to deciders of INSARAG. INACHUS was present during the INSARAG meeting during the two days with a stand in the tent where the standing lunch was served. People were available and EPLFM had very good contacts with many INSARAG representants from different countries: Germany, France, Estonia, Czech Republic, Israel, Hungary, Morocco, Tunisia, Italy, Poland, England, Belgium, Jordania and OCHA, as well as a representant from the EC. All of them were willing to learn more about the project, took the envelope from the Final event as well as leaflet and newsletters. Most of them communicate their mail to stay in touch with the project. Two French industrials specialized in the USAR technologies were willing to take contact with the partners of some tools.
Presentations in Fora/Events/Conferences related to Urban Search and Rescue:
During the fourth reporting period several of the project partners contributed to disseminate information about the project and its results at international events and meetings. These are: the International Lidar Mapping Forum (ILMF) and ASPRS Annual Conference 2018 in Denver, Colorado, USA , the Security Counterterror Expo in London, UK , the European Robotic Forum 2018 in Finland, the Swedish Microwave Days in Sweden, the ISPRS Technical Commission II Symposium 2018 “Towards Photogrammetry 2018” in Riva del Garda, Italy, the 5th International Workshop on Emergency Networks for Public Protection and Disaster Relief 2018 (EN4PPDR ‘18) in Limassol, Cyprus and the Forum Innovation Defense 2018, in Paris, France, ASCE Structures Congress 2018, in Forth Worth, Texas.
Moreover, 4 papers were published in the IEEE Transactions on Network and Service Management journal, in the Een-een-Twee: Tijdshrift over Brandweer en Hulverlening, the Engineering Structures journal and the Remote Sensing journal.
Two workshops were organized by the INACHUS project consortium. A CEN/CENELEC Workshop in Brussels, Belgium and an INACHUS Exploitation Clustering Workshop in Weeze, Germany. Also, an interactive demonstration was held at a field trial in Revinge, Sweden.
Furthermore, during 2018 INACHUS presented its outcomes in two field tests, one in Weeze, Germany in April 2018 and another one, in Roquebillière, France in November 2018. A final event was organised in Valabre, France in October 2018.
More specifically the following dissemination activities took place:
• A paper entitled ‘’3D laser imaging techniques on UAVs: Detection and localization of collapsed buildings and trapped victims’’ was presented during the International Lidar Mapping Forum (ILMF) and ASPRS Annual Conference 2018 in Denver, Colorado, USA, 5-7 February 2018
• A keynote lecture and a project presentation entitled ‘’ A 3D Urban Planning Tool quantifying Urban Risk and Resilience’’ were performed at the Security Counterterror Expo in London, UK, 6-7 March 2018.
• A presentation entitled “Snake Robots for emergency response’’ was presented during the European Robotic Forum, in Finland, 15 March 2018.
• The 3rd field test was held in Weeze, Germany on 20 April 2018
• A CEN/CENELEC Workshop “Interoperability framework for functional components of a modular search and rescue robotic platform” was organised by the INACHUS project consortium in Brussels, Belgium, 9 May 2018.
• A poster presentation entitled “Low Cost Aperture Coupled Patch Antenna Design” was presented at the Swedish Microwave Days, in Sweden, 24-25 May 2018.
• A paper entitled ‘’Satellite image classification of building damages using satellite and airborne image samples in a deep learning approach’’ was presented at the ISPRS Technical Commission II Symposium 2018 “Towards Photogrammetry 2018” , in Riva del Garda, Italy, 3-7 June 2018.
• An interactive demonstration and a discussion organized by FOI, among the stakeholders was held in Revinge, Sweden, 26 June 2018.
• A paper entitled “INACHUS: Europese innovative voor Urban Search and Rescue” was published in the Een-een-Twee: Tijdshrift over Brandweer en Hulverlening (Dutch journal, professional USAR), June 2018
• The paper ‘’Reliability of Collapse Simulation-comparing Finite and Applied Element Method at different levels’’ was published at Engineering Structures journal, September 2018
• A paper under the title “A resilient, multi-access communication solution for USaR operations: the INACHUS approach” was presented at the 5th International Workshop on Emergency Networks for Public Protection and Disaster Relief 2018 (EN4PPDR ‘18) Limassol, Cyprus, 15-17 October 2018
• A paper under the title “Enhancing Information Resilience in Disruptive Information-Centric Networks” was published in the IEEE Transactions on Network and Service Management.
• A paper entitled “Multi-Resolution Feature Fusion for Image Classification of Building Damages with Convolutional Neural Networks” was published in Remote Sensing journal, October 2018
• The final event took place in Valabre, France on 16 October 2018
• INACHUS project was presented to INSARAG regional meeting participants in Valabre, (Oct 17 & 18) France, where INACHUS was invited.
• A paper entitled “PROGNOSEFÄHIGE NUMERISCHE SIMULATIONEN ALS NACHWEISMETHODE: VOM BAUTEIL ZUM BAUWERK“ was presented in the 8th workshop Bau-Protect 2018, Munich, Germany, 13-14 November 2018
• The final field test was held in Roquebillière, France on 14-15 November 2018
• The INACHUS project activities related to 3D laser imaging techniques on UAVs, detection and, localization of collapsed buildings and trapped victims were presented in the Forum Innovation Defence 2018, Paris, France, 22-24 November 2018.
• The Journal Paper “Reliability of Collapse Simulation- comparing Finite and Applied Element Method at different levels” has been published in Engineering Structures 176:265-278, December 2018.
INACHUS Stakeholder’s community
The number of members reached the 105 persons in the end of the project. The stakeholder community matrix, sorted out accordingly to country, type of stakeholder has been updated and the new file has been uploaded in the Redmine: https://redmine.iccs.gr/projects/inachus/dmsf?folder_id=2486
4.1.5 Exploitation of results
INACHUS showcases a wide set of exploitation activities, witnessing the great potential of the system’s commercialisation. INACHUS outcomes have been grouped into 3 sub-categories as following:
1. INACHUS Single Devices
Within this group, there are all the systems and devices developed in INACHUS including:
• Victim localization devices such as the Snake Robot, the e-Nose sensor, the ground-based seismic Sensor system, the surface radar, the miniaturized IR-Camera system for helmet, the Stick Radar,
• Emergency Support Systems such as the INACHUS Common Operational Picture (COP) and the INACHUS Mobile application,
• Secured Communication positioning systems such as the INACHUS Mobile Gateway
2. INACHUS Technologies, S/W and Simulations
Within this group, there are the different technologies, simulations and software developed in INACHUS including:
• Simulations such as Tool for Structural Damage Analysis and Casualty Estimation, Collapse simulation and cavity identification and 3D mapping simulations
• Technologies such as 3D mapping technology and Victim detection and recognition technology
• Software such as administration Software to support ESS-SAR and MIFE.
3. INACHUS platform
This group consists of the overall platform of INACHUS developed within the frames of this project including:
• USaR supportive systems and INACHUS Robot Localization (RTLS). IPRs of the Consortium Members:
The overall INACHUS solution considered in this plan is a solution that includes at least the top level organizational solutions that provide situation awareness, support and communication services and finally some of the bottom level solutions such as sensors and simulation tools. The core/backbone of the INACHUS solution is the COP, the SaR-ESS, ECS, MIFE and mobile gateway. The INACHUS solution is designed to be modular and expandable. It includes a variety of solutions for same or similar applications. Depending on the needs and size of the client organization or the needs of a specific deployment a configuration with different types and numbers of sensor systems, drones, simulation tools, communication gateways can be interconnected to the backbone as well as configure the backbone itself. Applications/organizations that do not require the core solutions could use a combination of the individual exploitable results. The USaR market for solutions of the INACHUS size, determined mainly by cost, has many potential user organizations but not that many buyers. The nature of the INACHUS solution which is in essence part of the humanitarian sector is also a valuable realization when seeking for buyers. The approach for performing the market analysis of the overall solution considers the potential user organizations. Ideas on how to address the buyer’s situation are available but reside on the establishment of some sort of end user recognition/ satisfaction and also should take into account the various specificities and differentiations existing in USaR teams coming from different countries in E.U.
During the project lifetime, several partners tried to exploit their outcomes through different channels, including the end-users already participating in INACHUS project. Below we are listing these tangible exploitation activities, as performed from each corresponding partner.
INACHUS short stories of exploitation activities as performed during project lifetime
DIGINEXT: DIGINEXT had been in direct exchange with the INSARAG working group on information management to investigate options how the INACHUS COP, possibly with the connection to the ESS, can be acquired by INSARAG USaR teams. Although the working group members are very enthusiastic about the adoption of the INACHUS COP and its benefits for real USaR operations, no funding could be secured so far by the information management working group to finance the licence fees and maintenance of this software.
Recently, DIGINEXT has made a partnership with the school of application of civil security of EPLFM, and is in advanced contact with the firefighter brigades of several French departments, for example SDIS57. EPFLM is now adopting a DIGINEXT product as operational and training solution, which is compatible with the INACHUS COP. It can be expected that the firefighters that are also USaR professionals will therefore opt to adopt the INACHUS COP to continue with this software also for their USaR operations
EXODUS: EXO has initiated discussions with USaR NL in order to investigate the possibilities of commercializing the ESS portal and mobile application through USaR NL an INSARAG Group member. A physical meeting was held in Athens (EXO premises) where EXO presented the mobile application and portal to USaR NL representative. Within this meeting and several remote discussions that followed-up this meeting, EXO had the opportunity to prepare and propose a high-level functionality matrix of the mobile and portal application for USaR Teams as well as a time estimation towards the finalization and productization of the SaR-ESS and communicate it the end-users. As INSARAG is now in a process of refining their operational procedures and specifically investing on new technologies and tools that will further strengthen and facilitate their day to day operations, EXO is expecting to follow-up these initial exploitation activities and by exploiting this transitional period that INSARAG group is currently running to invest on the ESS portal and mobile application and introduce it as an integral tool for USaR Teams towards the optimization of their tactical operations and field reporting procedures.
CINSIDE: Cinside has presented and demonstrated three different radar systems to localize victims for end-users and stakeholders during the INACHUS project, in pilots and exercises. During this period, we have collected feedback and general information that have been very important for the development and will be for the future marketing activities. Our goal is to bring at least one of the radars to the rescue market. The network of rescue professionals we have gained during the project is invaluable for tests and verifications, and hopefully also as the first reference customers, those are SBFF, EPLFM and USAR.NL within the project and numerous external e.g. UK-ISAR and MSB (Sweden). We have also found a potential reseller of our coming rescue products, LEADER France.
EMI: EMI stays in contact with VirtualCitySystems GmbH, a leading corporation for the generation and management of 3D city models. VirtualCitySystems GmbH is very interested to integrate the algorithms of the mapping software, a result of INACHUS WP3, into their 3D city engines and signed a letter of interest during the project period of INACHUS. First steps for the development of an interface to the mapping software and first workshops with German seismological services and institutions for crisis management are performed. Future steps intends an automated integration of a web based service to apply the mapping software via a 3D city model, which are currently available for many public authorities.
FOI: In 2018 FOI and the Swedish Civil Contingencies Agency (MSB) initiated a dialogue about possible routes towards a product based on the seismic system development in INACHUS. These discussions are likely to continue and deepen during 2019. It is expected that decisions will be taken regarding whether to aim for such a product and then – if yes – how to organize and support development towards it.
Research Highlights: Apart from the aforementioned activities, a core group of partners participating in INACHUS (ICCS, EXO, EPLFM, SINTEF, DXT, FOI and SBFF) joint their forces and by exploiting the outcomes/knowledge gained through their involvement in INACHUS project, they submitted two new proposals (CURSORS and INGENIOUS) under SEC Call 2018 in DRS-02 Topic that were finally funded by the European Commission, giving them as such the opportunity to further extend, exploit and improve their work as developed in INACHUS the previous years after its end.
Standardisation and Patenting:
INACHUS initiated a CEN Workshop Agreement (CWA) in order to provide guidance on the technical and procedural interoperability of urban search and rescue (USaR) robotic platforms. More specifically, the standardisation initiative taken by INACHUS focused on interoperability (hardware, software) between the USaR robotic platforms and the equipment, sensors and tools that are attached to them. It also provides guidance on the principles for enabling USaR robotic platforms (various types of them such as drones, snake-like, robots with wheels, legs, etc.) to operate in all ground search environments. In this way a generic platform can be adapted, designed and built for any possible search and rescue (SaR) scenario on the ground. The final outcome of this initiative was a submitted CWA to the CEN/CENELEC entitled “Urban search and rescue (UsaR) robotic platform technical and procedural interoperability - Guide and the components of them” with identification number CWA 17357. This CWA can act as the starting point in developing interoperable USaR robotic platform solutions and is for use by organizations responsible for designing, manufacturing, configuring, customizing and maintaining USaR robotic platforms, tools, equipment and sensors (e.g. drones, snake-like robots, mobile robots with wheels, cameras, lasers, environmental analysis sensors, etc.). The CWA is also for use by integrators and providers of SaR support systems in general, first responder organizations, operators of the USaR systems, public authorities and end-users in the field dealing with USaR mission organization and execution. Last but not least the CWA is also of interest in the procurement of USaR platforms.
Given that a CWA document is created and supported by an open list of organizations and individuals this specific CWA document will be circulated among all related actors in order to follow its recommendations (date of official publication to CEN/CENELEC website was 27 February 2019, link: https://standards.cen.eu/dyn/www/f?p=204:110:0::::FSP_PROJECT,FSP_ORG_ID:67489,2449074&cs=1BC7B9E467856890E384D9915FDABAB4E ). Furthermore, as a CWA can be an initial step in the development of a European Standard all partners will support it after the completion of the project by expanding the involved entities and organizations list and by promoting its recommendations within EU related projects and initiatives either ongoing or forthcoming.
List of Websites:
Address of project public website and relevant contact details :
As projected from the various INACHUS dissemination activities, a public website and other social groups have been created since the on-line presence of a project or initiative are nowadays considered to be a powerful tool towards diffusion of knowledge to all possible directions and audiences, permits interaction with individuals, groups, initiatives and projects that share similar interests and increases significantly the overall impact.
-Project Website: http://www.inachus.eu/
-Facebook https://www.facebook.com/pages/Inachus-USaR-Research .
-Twitter https://twitter.com/InachusUsar
-LinkedIn https://www.linkedin.com/grp/home?gid=8385769
-YouTube Channel https://www.youtube.com/channel/UCBz08Jf7tVT08x5LevztXcQ
The INACHUS Consortium and Contact details :
1. Coordinator: INSTITUTE OF COMMUNICATION AND COMPUTER SYSTEMS (ICCS), Angelos Amditis, a.amditis@iccs.gr, Greece
Partners:
2. EXODUS ANONYMOS ETAIREIA PLIROFORIKIS. (EXODUS S.A) Dimitris Petrantonakis, dpetr@exodussa.com, Greece
3. Totalforsvarets Forskningsinstitut (FOI), Gustav Tolt, gustav.tolt@foi.se, Andreas Gustafsson, andreas.gustafsson@foi.se, Sweden
4. Crisisplan B.V. (CBV), Maureen Weller, weller@crisisplan.nl, Werner Overdijk, overdijk@crisisplan.nl, The Netherlands
5. Office National D'etudes Et De Recherches Aerospatiales (ONERA), Nicolas RIVIERE, nicolas.riviere@onera.fr, France
6. TEKNIKER (TEK), Roberto Calvo, rcalvo@tekniker.es, Spain
7. Fraunhofer Institute for High-Speed Dynamics, Ernst-Mach-Institut, EMI (EMI),
Dr. Kai Fischer, kai.fischer@emi.fraunhofer.de, Germany
8. CINSIDE AB (CINSIDE), Dan Axelsson, danaxe@cinside.eu, Sweden
9. ASI Applied Science International Europe, SRL (ASI), Steven Scoba, sscoba@appliedscienceint.com, Italy
10. DIGINEXT (DXT), Stéphane Collins, stephane.collins@diginext.fr, France
11. Laurea University of Applied Sciences (LUAS), Seija Tiainen, seija.tiainen@laurea.fi, Finland
12. Entente Pour la Foret Mediterranee (EPLFM), Dr Frédérique Giroud, f.giroud@valabre.com, France
13. Stiftelsen SINTEF (SINTEF), Giancarlo Marafioti, giancarlo.marafioti@sintef.no, Norway
14. Universiteit TWENTE / Department of Earth Systems Analysis, Faculty of Geo-Information Science and Earth Observation (ITC), Norman Kerle, kerle@itc.nl, The Netherlands
15. Schussler-Plan Ingenieurgesellschaft Mbh (ScPl), Dr. Andreas Bach, abach@schuessler-plan.de , Germany
16. Södertörns brandförsvarsförbund (SBFF), Henrik Nyman, henrik.nyman@sbff.se, Sweden
17. TELINT RTD Consultancy Services LTD (TELINT), Effie Makri, emakri@telint.eu, United Kingdom
18. BYTE COMPUTER ANONYMI VIOMICHANIKIEMPORIKI ETAIREIA (BYTE), Nikolaos Bezerianos, bezerianos@byte.gr, Greece
19. Mikrosystimata Mikrorois Gia Genetikous Elegkous Kai Moriaki Diagnostiki Epe (M2G), Spyridon Blionas, sbli@micro2gen.com, Greece
20. INSTITUUT FYSIEKE VEILIGHEID (IFV), Marco Boulogne, Marco.boulogne@vrh.nl, The Netherlands
In a duration of 48 months (January 2015 – December 2018) and implemented by a consortium of 20 partners INACHUS achieved a significant time reduction related to Urban Search and Rescue (USaR) phase by providing wide-area situation awareness solutions for improved detection and localisation of the trapped victims assisted by simulation tools for predicting structural failures and a holistic decision support mechanism incorporating operational procedures and resources of relevant actors. The INACHUS methodology is user-centric, involving end-users as much and as often as possible. End-users (First Responders, FRs and Urban Search and Rescue, USaR teams) are regularly being consulted to collect their insights, needs and remarks. Based on these requirements, the INACHUS project carefully designs, implements and evaluates, with end-users, a seamlessly integrated platform which provides the appropriate tools to enable FRs and USaR crews to respond to varied abnormal events, not limited to a specific emergency case or crisis event.
The INACHUS System is structured around the individual functionality of the partially integrated subsystems from individual components and the dataflow between them. This dataflow is defined between the mapping and building assessment software subsystem, the real 3D mapping tools and the victim localization solutions to the SaR-ESS and the INACHUS emergency support system via the communication infrastructure. The dataflow between the victim localization solutions and the SaR-ESS is being processed throughout the Real-Time Communication Gateway.
Reflecting the nature of the work, the INACHUS is organized in 12 Work Packages (WPs) each with a set of deliverables and milestones. The WPs follow the logical phases of the implementation of the project and include consortium management and assessment of progress and results. INACHUS spanned across 4 periods of 12-month duration each. In the first period the overall system architecture, a mixture of technical and user requirements and subsequent specifications, emerged. Moreover, in the same period early developments of the INACHUS components were realised that have been finally prototyped during the second period. .In the 2nd integration activities began towards the interfacing and interworking of the INACHUS integrated system. During the same period the individual INACHUS components have been preliminary evaluated under blast conditions over the execution of the component testing and the first pilot event. During the 3rd period full scale developments and deployments occurred in order to launch the first INACHUS integrated prototype. In the final period of the project, INACHUS solution was exhaustively tested in two pilots validating the components and the overall suggested solution as well. INACHUS widely disseminated its interim and final results, featuring scientific publications, a vivid online presence though its website and social media, an end user group of more than 100 organisations, regular newsletter releases and a well-established clustering network with other projects and initiatives in the field of USaR and disaster events. Noteworthy, INACHUS concluded in a CWA in the field of USaR robots paving the way to future standardisation in this field. At present a discussion is ongoing on the next concrete steps that need to be followed for exploiting further INACHUS outcomes and turning the consortium from a R&D focused entity into an effort to unlock the commercial potential of the project.
Project Context and Objectives:
Crisis incidents may result in difficult working conditions for Urban Search-and-Rescue (USaR) crews. INACHUS concept was driven towards achieving a significant time reduction related to Urban Search and Rescue (USaR) phase by providing:
1) Wide-area situational awareness solutions for improved detection and localisation of trapped victim.
2) Simulation tools for predicting structural failures.
3) Holistic decision support mechanism aiding in prioritization and mission coordination.
The INACHUS system that has been delivered encompasses novel technologies originating from different domains, seamlessly interworking with the aim to shorten the required time of USaR activities. In more detail, the INACHUS work delivered simulation and realistic visualisation methods on the scale of city quarters and for individual buildings. This was realised by the development and provision of a building library by using AEM, FEM and DEM approaches to identify and enhance the achievable prediction quality. Furthermore, a mapping and cavity identification tool was developed aiming at automatically suggesting rescue paths to trapped victims and USaR teams. Equally important was the development of 3D Digital surface models on affected areas, resulting in structural damage accurate assessments, especially if combined with the mapping and cavity identification tool. For wide area surveillance cases, wide area surveillance was performed thanks to 3D laser scanners embedded on fixed wing UAVs, covering large areas during hours by automatic or piloted navigation collecting digital elevation and surface models (DEM/DSM) of high resolution and high quality (5cm).INACHUS INACHUS developed and took advantage of deploying several novel victim localisation technologies: The Ground-Based Seismic Sensor system (GBSS) is a tool for detecting and locating knocking signals from victims trapped in debris heaps; The SurfaceRadar, also called HumanFinder, is a UWB beam steering radar dedicated to searching for survivals from disasters; the Mobile Phone Detector (MPD) detects mobile phones under rubble by making use of a directional antenna, the source of the strongest signal on this known communication channel can be approached; the Robotradar / Stickradar a miniaturized radar system, one of several sensors integrated in the INACHUS Robot; detecting movements in five directions around the robot body. Moreover, INACHUS developed a snake-like robot prototype designed and manufactured to help USaR teams finding and communicating with victims under a collapsed building. The robot apart from the robot radar has two video cameras for inspection and guidance and works as a sensor platform as well including an electronic nose to detect human presence and dangerous gases, and an infrared camera to detect human presence in case of poor visibility. Output of all carried sensors is visible in the integrated graphical user interface running on a tablet PC that acts as the remote controller. The robot is capable in entering in small holes (20cm x 20cm of cross-section) and contains a power supply, an industrial grade PC and all necessary communication devices. The e-nose sensor can also detect many air related hazards in confined spaces. These include the most typical asphyxiation hazards, toxic gases and combustible gases. Last but not least INACHUS robot makes use of a Real-Time Locating System (RTLS) which provides the feature of determining the absolute and relative position of the robot head module in the rubble. The infrared camera used in the INACHUS robot is a miniaturized LWIR camera (resulting in operations, which do not depend on illumination conditions). An area detection methodology is applied in order to identify regions with the temperature of a human body (34ºC-41ºC). This temperature range is adjustable according to the user needs and the specific targets to be detected. A crucial parameter for the successful functioning of INACHUS solution in real conditions are the stable and reliable communications. The communication platform that INACHUS developed manages the seamless interoperation, the interconnection between other networks (cellular, etc.) and provides redundancy and recovery functionality. For extending the capabilities and range of the crisis network in case of failures and/or congestion and to provide a distributed architecture, portable gateways and a long-range communication solution where designed and implemented. The gateways support the functionality of any other mobile USaR center as an alternative and cost-effective solution in an integrated ad-hoc manner but with redundancy, extendibility and security capabilities to assure robustness, flexibility and reliability.
Further to above, in order to support decision making in disaster events and to plan efficiently INACHUS project developed three novel tools aiming at merging information from the various tools used in the field and from simulation and modelling tools. Multi-source information engine (MIFE) acts as the data fusion model engine for the INACHUS solution resulting in the location of possible survivors and/or the presence of dangerous gases. This information is sent to both a portal & mobile application for search & rescue operation (SaR-ESS) and to the Common Operational Picture (COP). The former acts as the digital transformer of the collected information sharing them to responsible teams and persons for proper decision-making. The latter (COP) links end users with the entire INACHUS system by providing a comprehensive map-centric view on the incident site. The rich visualization capabilities of the COP in 2D and 3D allows the end user an intuitive access to the incident situation, which increases the overall situational awareness and aids the decision process of USaR personnel on strategic as well as tactic and operational levels.
For designing, developing, prototyping, testing, evaluating and ultimately delivering the abovementioned system, the 48-month work implementation has been divided in 12 work packages as follows: WP1 on Scenarios definition, user/system requirements and specifications, WP2 on Framework design and interoperability issues, WP3 on Simulation tool for structural damage analysis and casualty estimation, WP4 on Wide-area surveillance tools for monitoring of collapsed buildings, WP5 on Victim localisation solutions, WP6 on INACHUS Emergency Support System (SaR-ESS), WP7 on Secure communication and positioning issues, WP8 on System Integration, WP9 on Pilot implementation and validation of the INACHUS platform, WP10 on Dissemination, Exploitation and Training activities, WP11 on Evaluation and consideration of societal Impacts, legal/ethical issues and standardisation and WP12 on Project management, quality assurance and reporting. In the following of present section, a description of the entirety of project objectives at WP-level can be found including their accomplishment status.
WP1 Objectives and Accomplishment status:
- To define the user requirements (services) for INACHUS (Accomplished and delivered in D1.2 & D1.1)
- To define the system requirements for INACHUS (Accomplished and delivered in D1.2)
- To define evaluation criteria and objectives for demonstration for INACHUS (Accomplished and delivered in D1.1 D1.2 D1.3)
WP2 Objectives and Accomplishment status:
- To design and deploy a framework according to the WP1 requirements (Accomplished and delivered in D2.1)
- To deploy all the required connectors to integrate all the pilots and scenarios in the INACHUS system framework (Accomplished and delivered in D2.2)
- To define the criteria on which the solution was validated, and the plan for the final integration before the pilot tests (Accomplished and delivered in D2.2)
WP3 Objectives and Accomplishment status:
- To provide 3D simulation methods for prediction of survival spaces after disastrous events (Accomplished and delivered in D3.1 D3.2)
- To provide pre-calculated results and fast computations methods for on-site assessment of first responders (Accomplished and delivered in D3.3)
- To provide realistic 3D representation of partially failed buildings and debris heaps (Accomplished and delivered in D3.3 D3.4 D3.5)
WP4 Objectives and Accomplishment status:
- Wide-area disaster scene assessment based on nested integration of space borne information and results of numerical analysis (from WP3), and population distribution mapping, for 3D modelling and USaR Prioritization (Accomplished and delivered in D4.1)
- To provide high-resolution and high-quality 3D Digital surface models coming from laser cameras and photogrammetry data on the most affected areas (Accomplished and delivered in D4.2)
- To perform an exploitation of this 3D data in relation with the library of collapsed building models coming from WP3 using a semantic analysis or matching method (Accomplished and delivered in D4.3)
- To assess the structural damage to deliver maps of survival space and rescue path as inputs to the WP6 (Accomplished and delivered in D4.1 D4.3)
WP5 Objectives and Accomplishment status:
- To provide sensor deployment solutions and increase speed of detection and localisation of trapped victims under rubble (Accomplished and delivered in D5.1 and other WP5 deliverables)
- Design and develop the snake robot used as sensor platform (Accomplished and delivered in D5.6)
- Other auxiliary sensors serve as monitoring and localisation of mechanical vibration (Accomplished and delivered in D5.2)
- Evaluate methods for prioritisation of search areas by means of triangulating the location of radiation originating from personal handsets (mobile phones) buried under rubble (Accomplished and delivered in D5.1)
- Provide preliminary indication of presence of trapped victims using CW radar, e-nose and IR camera mounted on the snake robot (Accomplished and delivered in D5.3 D5.4 D5.5)
- Investigate optimal distributed seismic sensor network configurations and operational modes with the aim of reducing mechanical interference to the radar, which in turn increases probability of detection (Accomplished and delivered in D5.2)
- To test and evaluate the proposed sensors under real life-like conditions. (Accomplished and delivered in D5.6 and other WP5 deliverables)
WP6 Objectives and Accomplishment status:
- Development of the INACHUS back-end to support the in- situ operations at both operational and strategic levels (Accomplished and delivered in D6.1 D6.2)
- Development of INACHUS front-end tools to enable a common operation picture among main actors while an operation evolves for enhanced situation awareness and better decision making (Accomplished and delivered in D6.1 D6.2)
WP7 Objectives and Accomplishment status:
- Development of the INACHUS communications and positioning platform and its subsystems (Accomplished and delivered in D7.1
WP8 Objectives and Accomplishment status:
- To integrate all the hardware and software components defined for the INACHUS system framework, taking into account the different pilots/scenarios established (Accomplished and delivered in D8.1 D8.2 D8.3)
WP9 Objectives and Accomplishment status:
- To demonstrate the use of INACHUS for each case (type of crisis) (Accomplished and delivered in D9.1 D9.2)
- To validate the impact of INACHUS for each case with external experts (Accomplished and delivered in D9.1 D9.2)
- To initially acquaint the FRs to work with INACHUS- parallel activities with WP10-Training Activities (Accomplished and delivered in D9.1 D9.2)
- To test the development of INACHUS on a local scale (site in the Netherlands) and at a transnational scale (site in France) (Accomplished and delivered in D9.1 D9.2)
- To validate the imaging (mainly airborne based) techniques (WP4 outcomes) plus the simulator of WP3 in the site in southern France (Accomplished and delivered in D9.1 D9.2)
- To test and evaluate the proposed sensors under real life conditions (Accomplished and delivered in D9.1 D9.2)
WP10 Objectives and Accomplishment status:
- To establish systematic channels and means for disseminating the project objectives, activity, progress and technological outcome to potential INACHUS stakeholders (Accomplished and delivered in D10.1)
- To ensure the alignment of the project activity with calendar events of relevant EU programmes & initiatives (Accomplished and delivered in D10.1 D10.3)
- To share the technical results with the scientific community interested in the topics addressed by the project, in order to promote the research and receive useful inputs from other scientists and International Communities (Accomplished and delivered in D10.3)
- To ensure adequate training of the end users participating in the field tests and availability of a comprehensive training package at the end of the project to facilitate the uptake of the INACHUS system by interested entities (Accomplished and delivered in D10.2 D10.5)
- To define the vision of using the project outcome and create relative awareness within Civil Protection administrations and Security organizations throughout the Member States of the European Union (Accomplished and delivered in D10.4)
- To develop the strategic approach, define the appropriate business plan and elaborate a suitable market model which can support the perspective of commercializing the project results (Accomplished and delivered in D10.4)
- To prepare support products, including documentation, in a form that can be understandable and accepted by potential users, help in technology transfer and provide necessary advices and support (Accomplished and delivered in D10.1 D10.5)
WP11 Objectives and Accomplishment status:
- To investigate current state of the ethical and societal issues in the USaR, including a limited literature review on victims’ perspective (Accomplished and delivered in D11.1)
- To define ethical and societal requirements for the solution, and to formulate ethical guidelines for the end-users (Accomplished and delivered in D11.1 in various technical notes, as well as in the INACHUS Code Of Conduct and in the organizational guidelines for data protection. The latter contents are embedded in the INACHUS Training Kit)
- To verify the application of the ethical requirements and user guidelines in pilots (Accomplished and delivered as part of INACHUS validation process. In addition a separate Privacy and Data Protection Impact Assessment (DPIA) is provided.in D11.1)
- To highlight the potential societal impacts of the proposed system and its business model, and provide recommendations on these in order ensure a successful adoption of the proposed technologies (Accomplished and delivered in D11.1 and finalized as part of the INACHUS exploitation plan D10.4)
- To exploit the current available standards, best practices and legal issues considerations when developing new technologies in the USaR operations (Accomplished and delivered in D11.3)
- To conduct a comprehensive investigation of all regulatory framework and legal aspects, regarding Information and Communication Security, to ensure compliance with current regulatory requirements and, where appropriate, develop recommendations for further regulatory considerations by the appropriate bodies. (Accomplished and delivered in D11.3)
- To foster Interoperability and interchange-ability of equipment and systems, to further define the new/ revised/ integrated/ proposed pre-normative standards based on the outcomes of INACHUS (Accomplished and delivered in D11.2
- To launch and promote the debates on USaR related pre-normative standards in the context of the European Standardisation Organizations through the new Standardisation process vehicles, like CEN/CENELEC Workshop Agreement or ETSI Industrial Specification Group (Accomplished and delivered in D11.2)
WP12 Objectives and Accomplishment status:
- To perform the project objectives in a qualitative way and to meet the project schedule and deadlines (technical coordination) (Accomplished and delivered in D12.1)
- To perform the financial and administrative tasks of the project (Accomplished and delivered through EC periodic and current EC final report)
Project Results:
The present section describes all project S & T results/foregrounds for the entire project duration, the latter being of 48 months starting on January 2015 until end of December 2018. The project lifecycle comprised of 12 distinct work packages has been divided in four periods each of them comprising of 12 months. All S & T results/foregrounds have been delivered in the context of the first nine technical work packages of the project, as the latter three are dealing with horizontal activities, namely dissemination, exploitation and project management.
First period (M1-M12) took place from January 2015 until December 2015 during which the work of WP1 was under full implementation while technical WPs i.e. WP2, WP3, WP4, WP5 and WP8 have commenced. The latter ones capitalise mainly on the end user requirements extracted - through questionnaires and a workshop - from a plethora of end users originating from different categories. Of equal importance input are considered the technical requirements, the INACHUS subsystems’/components’ specifications as well as the overall INACHUS architecture that set the basis for further developments and research activities. At the same time all WPs previously mentioned fuel with their outputs WP8 (started on M12) towards the very first INACHUS component functional and validation testing before integration.
Second period (M13-M24) has been realised from January 2016 until December 2016, during which the entirety of technical WPs, namely WP3, WP4, WP5, WP6, WP7, WP8 and WP9 have been running in parallel showcasing significant progress towards functional prototypes while WP1 and WP2 concluded. Noteworthy, the 2nd period includes the beginning of the integration activities (under WP8) towards the interfacing and interworking of the INACHUS integrated system. During the same period the individual INACHUS components have been preliminary evaluated under blast conditions over the execution of the component testing. Noteworthy, WP9 on Pilot Implementation started in the 2nd period providing an internal plan including Key Performance Indicators for the whole system, use cases as well as the description of the mock up exercises to validate the INACHUS system’s usability. The first pilot was realised during this period, providing valuable information about the INACHUS components by testing them in real conditions.
The third period of INACHUS (M25-M36) has been implemented during January 2017 until December 2017 providing almost finalised prototype of INACHUS solution. The main achievements for the aforementioned period was the establishment of the building library; the finalization and integration of the mapping and the cavity identification tool in the INACHUS system; a prototype validation of the tools to retrieve 3D data from the collapsed buildings using aerial and ground measurements, photogrammetric images from RGB cameras and 3D point clouds from Laser cameras;•an e-nose prototype including CO2, O2, NH3, CO, H2S, combustibles, and temperature and humidity sensors. Also the radar prototype and the thermal camera were completed and integrated in the INACHUS robot. For the robot, various parts were 3D printed and assembled into the final system. Furthermore, the SaR-ESS, was delivered while the COP was almost finalized. The integration interfaces between all the components and subsystems have been defined and the implementation and the compliance validation of the individual components with these interfaces have been tested and validated. Last, the second pilot of INACHUS project and the simulated exercises were successfully performed, whereas already the third pilot has been planned.
In the fourth and final period of INACHUS (M37-M48 i.e. January 2018 until December 2018) focus was given in the integration and testing of INACHUS system during the pilots. Some additional implementations in the mapping tool increasing the value of the tool for the end users and thereby simultaneously the probability of real usage of the tool in future were realised. In addition, 3D data exploitation using aerial laser cameras and image analysis by drones was finalised and SAR-ESS, a complex system composed by numerous modules and components developed to control, visualize and facilitate the search-and-rescue operations and protocols was finally delivered. Moreover, the final version of the INACHUS integrated system was finally deployed and tested during two pilots, on April 2018 in Weeze, Germany and on November 2018 in Roquebilliere, France. In both pilots the comments and feedback received by external participants were very positive for the features and user-friendliness of the INACHUS system.
The most important S & T results/foregrounds of the above mentioned WPs’ progress will be briefly but concisely described below for the entire INACHUS lifecycle:
INACHUS technical work commenced with WP1 which has initially focused on the State-of-the-Art analysis of current USaR ICT systems and on the extraction of the recommendations coming from the end-users. The information collected was analysed according to the technological aspects of navigation, communication platform and motion control of unmanned systems focusing on USaR operations. In parallel, special emphasis has been given on the formation of the INACHUS end-user community, as expected from T1.2 and to the validation of end-user requirements. This was realised by the preparation of a questionnaire and a series of workshops with end-users. The first end user workshop occurred on April 8 and 9, 2015 in Leiden, The Netherlands. The interactive, international workshop was attended by a diverse group of USaR workers and others related to the field, as well as INACHUS technical and end user partners. During the workshop, technical partners provided demonstrations of prototypes or mock ups of system components, and there were many opportunities for discussions and feedback. The goal of this first workshop was to collect user requirements. In addition, scenario elements, training needs and mission requirements were discussed. During the second workshop, hosted by EPLFM in Gardanne, France on June 10, 2015 the user requirements were validated. WP1 activities concluded in M15 with the delivery of D1.1 & D1.2 both related to end-user requirements as discussed and elaborated during the dedicated workshops (D1.1.) and as incorporated with the outcomes of the interviews and questionnaires (D1.2). Furthermore, list of EURs included specific operational end users requirements as collected by end-users in the context of T1.3 including operational reports from major disaster events such as the Haiti earthquake, Nepal earthquake and Japan tsunami. This task concluded by D1.3 a report including a detailed description of the USaR international procedures and guidelines, mainly elaborated by the INSARAG guidelines. Last but not least, T1.4 translated the operational needs into technological needs. Apart from EURs WP1 contributed in INACHUS project by generating the first use cases (UCs).
The work delivered in WP2, commenced in first period, includes the design and development of all the system components into a common Framework, and interoperability and integration matters were taken into consideration. The main outcome of T2.1 for the first period was a document entitled “Analysis of Regulatory Framework, Standardization and Interoperability Issues” incorporating a thorough review of the regulatory framework, the standardization and the interoperability issues through an investigation of existing standards and regulations regarding Information and Communication Security to ensure interoperability and interchange-ability of equipment and systems. These were mapped with respect to the applicable system components that comprise the INACHUS framework. The output of this work had the objective of feeding the INACHUS technical work packages (namely, WP3, WP4, WP5, WP6 and WP7) with inputs on standards and practices issues, on which the INAHCUS systems providers can base their work in order to ensure interoperability and security of the components as a working whole. WP2 produced 2 key deliverables for the success of the final INACHUS solution. D2.1 including the INACHUS System Architecture was produced under T2.2 and T2.3. In T2.2 conceptual and functional diagrammatic depiction of the INACHUS System (Conceptual Diagram and Functional Diagram), was created. Furthermore, T2.2 proceeded with an interactive process whereby input and feedback was provided by all the technical partners who were responsible for providing components or parts of components to the overall INACHUS System. The next step of this effort was the high-level conceptual diagram and the functional model. All delivered technical requirements, had been produced and updated, so that the technical requirements that were mapped from the end-user requirements could also be matched with the system’s specifications. Furthermore, the Technical Requirements were defined and mapped from corresponding End User Requirements. The process involved the categorisation of Technical Requirements according to System Component, and their type (“Functional”/“Non-Functional”). Additional information included the responsible partner, their subtype, the Use Case they cover, whether they are technically feasible, their feasibility constraints (if applicable), their interdependencies and their compliance with standards/regulations (if applicable). A careful cross-check of the System Specifications and Technical Requirements was conducted in order to ensure that all requirements deemed “mandatory” and “important” would be satisfied by the corresponding system components. In T2.3 the INACHUS architecture was designed. All partners active in the corresponding technical WPs contributed to the definition of the architecture of the corresponding parts of the INACHUS framework. The INACHUS architecture was complemented by the system specifications (D2.1 Annex 2) where information about the various component interfaces is presented. Note that the Annex 2 was a ‘live document’ until the end of T2.3 because the continuous development of the various INACHUS components was on going. Thus, the refinement and fine-tuning of the Technical Requirements, the updates on the system specifications and the cross checking of technical requirements to system specifications continued through the end of T2.3. Therefore, the technical team, having this information available, was able to continue the development of the INACHUS components and further deepen the analysis of the INACHUS framework contributing to the integrations activities, such as deployment diagram, interface control documents, interoperations among components, validation criteria, etc. Equally important, T2.4 was responsible for the development of a detailed plan for the integration process from the early stages of the project. This task was also responsible for defining the criteria for which the system components’ interconnection(s) would be validated. The D2.2 document presents a holistic view of the INACHUS platform integration that combines the general development procedures established in System Engineering with the development procedures established in the DOW and with the needs that the specific requirements and the specific design solutions generate for these pre-defined methodologies. This document first analyses these specific constrictions, and then it provides the specific solution procedures, the organization charts of responsibilities to execute them and the timing in order to optimize their execution including alternatives for contingency, and, finally, it defines criteria in order to validate the whole process.
WP3 provided simulation and realistic visualisation methods both on the scale of city quarters and for individual buildings. The methods provides USaR the capability to continuously improve the situational awareness of rescue teams starting from hour one after the incident occurs, up to a point in time in which the situation is fully assessed and responded to. More specifically, T3.1 aimed at the deployment of a software tool for vulnerability assessment of city quarters against blast scenarios enlarged by a module for evaluation of the expected damage under earthquake loading. The software tool included features not only based on a relative coordinate system set by the user but also by using real world coordinates that can be extracted directly form the mapping tool results. After the map is imported, the scale factor is automatically determined and each city object is referenced with real-world data. In addition, there is a feature of linking real-time earthquake data from shakemaps.org. Hence, it is possible to import ground accelerations directly, instead of estimating the intensity by the distance of the building to the epicentre. This means on the other hand that the effect of soil conditions on the earthquake’s intensity is now explicitly incorporated. T3.2 run in parallel with T3.1 aiming to conduct a literature review of available high-quality data sets for existing partly and totally collapsed buildings. Besides understanding the occurrence, reasons and the nature of building collapse, a further key objective of the literature survey was to identify suitable collapse cases, which can be used to judge the accuracy of the different simulation methods in predicting collapse for the single building assessment. The study of the collapse cases has started from the investigation of a database including 130 real collapse cases that occurred all around the world due to several causes as documented in D3.1. All of these events have been investigated evaluating the type of incident and the number of losses, the structural composition (materials and static scheme) and the dynamic trigger, the type of collapse (partial – total) and the collapse shape, the available official information and the potential use of simulation tools for supporting USaR teams. The review showed that many published cases are missing crucial information necessary for proper modelling. Relevant real cases from the formulated including relevant data were studied extensively. Moreover, three academic research examples were selected to be studied: Shirai et al. (2005) triaxial seismic testing, Peng (2015) progressive collapse experiment due to columns removal and Yu et al. (2014) blast testing of structural component. Additionally, several collapse cases taken from the ASI client database of controlled demolitions were examined. The data collected for each one of these case studies has guided WP3 to assess the selection criteria for the simulation cases according to the INACHUS objectives and to define the final validation strategy for the three available simulation methods (FEM, AEM and DEM). The validation was implemented in three different levels: a component level (Yu et al., 2014), a mid-size level (Peng, 2015) and a full-size level (simulation of the Pyne Gould Building, Christchurch). Two additional cases, namely an academic case (Shirai et. al.,2005) and the A.P Murrah building, was modelled to enrich the validation. The latter has been already modelled by ASI with a previous model of the software ELS but it was re-modelled and implemented specifically for the INACHUS project.
Chosen cases provided a variety in terms of the size of collapsed structures, the level of the damage to the structure, and the cause of collapse by earthquake or blast. The involved partners spent a great amount of efforts in D3.1. D3.1 gives an overview of relevant hazards and trends in literature, evaluates the different hazards with regards to their consideration within the INACHUS project and identifies different collapse shapes. One chapter is dedicated to the above described real collapse cases, discusses several cases and lays out the validation strategy as mentioned. The second part of the deliverable documents the predefined buildings for the mapping tool and for the detailed collapse simulations. The final work in this task consisted of the modelling of the chosen validation examples. At component level and mid-size level validation, applied element method and finite element method were used for the modelling while at full-size level validation applied element method was used. Details on the outcomes of modelling validation are included in D3.2. T3.3 typical building was defined including structural details and typical characteristics prone to collapse risks. A building library was prepared including building models that were later used for T3.4. Four buildings were chosen from the range of typical buildings defined in Task 3.3 as reported in D3.1. Chosen buildings are a multi-Family house, a high-rise building an office tower and a historic building. Artificial earthquake histories were generated, scaled and applied to the models in different directions. Explosion scenarios were modelled by sudden element removal, as further explained in D3.3. Next steps included simulations by using AEM and DEM in the four different types of buildings and are fully documented in D3.3 and in related annexes. Furthermore, an evaluation of speed enhancement of AEM and DEM models was implemented under T3.5 by a detailed study in order to assess a trade-off of accuracy, complexity and speed and thereby optimize the models towards fast on-site operations. Concerning AEM method, for the simplified and the refined model, three level of analysis were performed for an earthquake-loading scenario: “fast”, “medium” and “slow” depending on different parameter settings. Hence, a total amount of six simulations was analysed. Despite the geometrical and structural simplifications, the “slow” and the “medium” analysis show a similar final collapse shape. The medium analysis in particular matched the INACHUS request of building quickly a model (about three hours), which gives a realistic and usable debris pile within a reasonable calculation time (about 13 hours). The fast analysis provided different results for both the refined and the simplified model with a total collapse of the central core and can therefore be seen as not valid. With regards to the DEM method, despite the already mentioned pre-processing routine, which was tested and further improved within T3.5 the utilization of a customized blender release, the “Fracture modifier” (FM) was undertaken. More efficient object management in the FM compared to the Blender standard release led to a considerable speed improvement (by a factor of ca 35 to 40). The utilization of the FM can be seen as a major step towards “real-time” simulation. The modified, more effective object handling in the FM has been noticed favourably by Blender´s core developers. Details concerning the pre-processing routing and the usage of the FM are reported in D3.5. Next step in the development efforts under WP3 included the development of cavity identification and data processing (T3.6) and the creation of output and library for the Decision Support System (T3.7). A new software tool was developed able to identify cavities from simulation results delivered in specified data formats with features such as the automatic suggestions of rescue paths. This function sorts the cavities by size and connects the centre of gravity of the largest ten cavities by searching for the shortest distance between theses points. Finally, the shortest distance to the exterior is searched and indicated. T3.7 based on the analysis performed under T3.4 contributed by creating a database containing the initial buildings, the different collapse cases and the pre-simulated cavity files, each as .obj file and in the native file format. In addition to this, a short report was provided for each building.
WP4 commenced in the first period of the project focusing on 3D data mapping. The role of 3D mapping of collapsed areas in INACHUS was to provide the USaR personnel with accurate and timely 3D models to support situation assessment, localization of survival spaces and rescue paths and planning of actions. To be able to do this as efficiently as possible, a strategy is needed for how and when the respective 3D mapping systems should be used and how their respective strengths should be exploited, so as to offer maximal support in the search and rescue phase. T4.1 contained 2 distinct research focus area: work with the International Charter “Space and major disaster”, and the dasymetric population mapping. The research approach was initially given with a focus on Europe, yet aim at ultimate global applicability, not least because of relevant technical and organizational developments elsewhere that can be of benefit to Europe, but also because European strengths and assets can be fruitfully deployed in extra-European settings, enlarging the influence of Europe. Recent disaster events that took place outside of Europe (e.g. the 2015 Nepal earthquake) also constituted interesting cases to test some of the approaches being developed. The population mapping component of the INACHUS project has several different components to be considered. This is because remote sensing has been continuously used for population mapping in a dasymetric mapping approach, contributing through several proxies of population (e.g. built-up areas, nightlights in satellite imagery, land use land cover maps, among others). This approach having the target scale indicated at 50m spatial resolution, requires datasets with suitable resolution, in the case remote sensing data is to be considered. The tests carried out by ITC focused on the use of two main sets of data. A high-resolution urban land use land cover map from the Copernicus Programme, the Urban Atlas 2012 (UA12), available for 695 European cities; a building and household surface covering 3 Portuguese cities with the resolution almost matching the UA12, however with a general coarser resolution, especially in peri-urban areas. The experiments conducted considered three main different situations, where a dasymetric mapping was approached and used. This intended to simulate a situation where: 1) only urban LULC maps is available, 2) when there is information regarding the buildings which can be coupled with the LULC maps to determine the building function and 3) considering the households as a unit, which cannot be directly observed through remote sensing sensors. In this last situation the UA12 was also used to enable the comparison between all the approaches. As observed, to consider the household as proxy for residential population presence overall gives the best results. This shows the sensitivity of dasymetric mapping approaches to the used datasets as proxies of population, especially functional surface used. Even when considering an urban LULC map coupled with a building surface with the buildings divided by the number of floors still results in errors of up to 38%. To consider only the LULC maps as a proxy for population made the generated population map unusable. It seems that, at least for the considered cities, the household approach is the only one with results that may be considered. The function is of extreme importance in urban and high-resolution dasymetric mapping approaches, where an urban LULC map may not be sufficient to provide population maps at a resolution of 50x50 m².
The population maps obtained before must be coupled with early damage information. A thorough examination of the potential contributors of these graded damage maps to the INACHUS project was performed. The International Charter due to its copyright restrictions does not have the general authority to release of their reference and grading damage maps in GIS-ready format, as well as the used satellite imagery, to the public. Namely the vector information, since the imagery may be bounded to commercial agreements between these institutions and private satellite imagery providers such as IKONOS. The institutions and organizations contributing for the International Charter may decide, however, to publicly release their vector data, independently from the charter. UNOSAT, member of the International Charter, makes the vector information public through their website. However, UNOSAT is not involved in all the activations of the International Charter. The recently established Copernicus Emergency Management Service (EMS), also aims at disaster response, producing reference and damage maps for any area of the globe. Likewise the International Charter it may acquire private satellite imagery as base for their reference and grading damage maps, apart from the use of Sentinel series satellite imagery. In 2015, the Joint Research Centre of the European Commission, has contracted the Compagnia Generale Ripreseaeree (CGR) to acquire nadir and oblique images in a rapid mapping and response context, which are then assessed by the EMS. Contrasting with the International Charter, the EMS provides the vector information regarding all of their activations publicly, through their website, easing the incorporation of such information in GIS environments and consequently used in INACHUS.
Within T4.2 acting as the core 3D mapping task of the project, two different techniques were studied for collecting 3D data for disaster areas: 3D laser from airborne and ground measurement and 3D from airborne imaging. Digital elevation/surface models (DEM/DSM) of high resolution and high quality were fused from these two approaches (laser imagery and normal imagery). The images from the visual sensors can be draped onto the 3D data, thereby enriching the 3D models and making them easier to interpret for an operator. One main motivation to use two complementary data acquisition techniques (laser, images) is that both platforms can be deployed individually. The subsequent data interpretation is possible on a 24/7 basis if only one of these techniques is available. If, however, both are available and fused (see task 4.3) even better and more reliable analysis is possible. In order to provide enhanced visions to the USaR team in all weather conditions (rain, snow, fog, haze, dust wind etc.) two kinds of sensors to fulfil the target were distinguished: (i) large Field of View sensors (often passive visible/IR Camera or Radar systems) and, (ii) high spatial resolution and small FOV optical laser sensors prototypes. FOI was involved to obtain 3D data point clouds from the ground and ONERA collected 3D laser data from drones. Airborne sensors provide wide-area coverage while ground-based sensing produces dense high-accuracy data to complement the laser data from the UAV. First experiments were done in Toulouse, France in March 2016. The main objective was to share a common scenario and to identify the risks in using three different techniques on the same field test. During the third period of the project, the INACHUS consortium organized experiments to illustrate the concepts and to display first 3D data to the end-users in real-time. These experiments were shared with all the WP4 technical partners on a common scenario in Lyon, France (pilot 2 of the project). Real-time registrations were done on the 3D point clouds collected by the different systems. In Task 4.2.2 one of the indicated objectives was a view planning framework that allows a complete 3D reconstruction, which also covers the façades of the buildings. In spite of the fact that damage detection is not a part of this task, a view planning tailored for the Urban Search and Rescue (USaR) teams must consider early damage evidences (related with Task 4.5). The objective of USaR teams is the detection of buildings that may entrap victims. Therefore, the view planning must be able to discard non-damaged areas, while focusing on the partial and total collapsed ones. To achieve this, an incremental approach started being implemented within INACHUS, where earlier damage evidences are used to prioritize the following ones. The task was divided in two main sections:
• 1st: a nadir flight to have an overview over the scene to:
• 2nd: oblique views considering hot spots determined earlier
In March 2016, Toulouse, WP4 tested all of their sensors. Both nadir and oblique views were acquired. In the following link an online visualization of the point cloud is presented: December, 2016.
ICCS established workflows for fast and reliable image-based 3D modelling of disaster scenes using commercial (Agisoft PhotoScan) and open-source (MicMac and MeshLab) software. The objectives of the conducted research were (a) the a-priori establishment of a proper workflow for fast and reliable image-based 3D modelling and the a-priori establishment of the parameters that have to be set in each step of the photogrammetric process, so that a predefined plan can be applied in emergency situations, without wasting valuable time in the field for the determination of the pipeline and the parameter tuning; (b) the evaluation of commercial and open-source software implementing image-based 3D modelling. Several experiments for the establishment of 3D modelling workflows for USaR purposes via earthquake imagery were conducted and a thorough analysis on the parameters of the various modelling steps was made. Two UAV image datasets of a small region in Mirabello, Italy, around the Church of Saint Paul, acquired after the earthquake of 2012 were used in the experiments. PhotoScan and the combination of the open-source tools MicMac and MeshLab were used. The performance of the tests was evaluated in terms of time required for the creation of 3D textured models and number of generated dense cloud points and comparisons were made among the generated 3D models. Also, a study on multi-scale 3D modelling of damaged areas was performed. The generation of models at different levels of detail in an emergency scenario can be adopted in the case that a first draft 3D model of a scene is required to visualize the areas with damages (low resolution model) and detailed 3D models of areas with damages are needed to be used by USaR teams to locate possible victims. Based on previous tasks developments and achievements T4.3 focused in 3D data fusion. 3D data point clouds were collected during pilot 2 and were fused in a unique coordinate system. The outcomes of this task are detailed in D4.2 including the testing and refinements of 3D registration algorithm using multiple datasets;, final adjustments to the 3D data registration algorithm based on experience from tests during pilots, registration of the aerial laser scanner data, successful automatic fusion of ground-based laser scanning data with 3D photogrammetry models, and set up a tool for evaluating 3D registration accuracy. In addition, a tool for automatic construction of 3D occupancy grid from registered ground-based laser, as a means to high-light unexplored areas and segment possible voids was created. The dataset was exchanged with T4.5 for post-processing and data exploitation. T4.4 contributed in WP4 by developing a scenario filtering facilitating work under T4.5 where 3D data were exploited though semantic analysis and model matching. The design and implementation of the filtering approach was finished by the end of the second period. The first step was to analyze the models stored in the existing database (WP3). Then based on the format of these models a filter was implemented considering several parameters (soil, etc.).
In order to split up the tasks technical challenges (TC) have been defined and for all of those results were obtained:
TC 1: Initial scene annotation/simple features
TC 2: Model matching (link to WP3) and cavity identification based on WP3-model
TC 3: Semantic data analysis, without damage model from WP3
TC 4: Data exchange to COP
Part of the results achieved are described hereafter for each TCs.
Flight planning app – automatizing the light-weight (lw) UAV image capture and the damage detection routine
The app allows to automatically perform the lw-UAV flights and also the damage detection from the collected images in a near-real-time approach. Efforts on this period were on the testing and integration of the several components of the system such as the damage detection procedure and the communication between the UAV and a ground station for a faster processing. The developed procedure allows a flightplan to be executed, with real-time streaming of the image data and their processing, which means that an orthorectified image mosaic with initial damage indications is provided as soon as the lw UAV lands.
Multi-resolution feature fusion for the image classification of building damages
The INACHUS system takes advantage of remote sensing imagery at different geographical scales to derive the damaged state of a given region. This is performed with satellite imagery that allows for first responders to have an overview of the extent of the damage within a given region, delineating rescue efforts accordingly. However, these image data do not allow identifying damages on the façades or smaller signs of damage due to the low resolution. Aerial imagery captured from manned platforms enables the surveying of the façades and in higher resolution when compared with the satellite. Nonetheless there may be areas which are occluded from this aerial flight due to urban morphology for example. The unmanned aerial vehicle can then survey these occluded areas and at a higher resolution than the aerial manned ones. Currently, the image classification of building damages is performed considering separately each of the resolution levels using convolutional neural networks. In this approach we assess the impact of considering this multi-resolution image information in the image classification of building damages.
To accomplish this, a multi-resolution feature fusion using convolutional neural networks was used. Three different feature fusion approaches were tested. These were designed to capture different feature information from each of the resolutions network. Overall the multi-resolution feature fusion approach improved the accuracy of the image classification of building damages. This was also the case when considering a new geographical region as validation of the approach. The improvements in accuracy reached 2% considering the satellite and unmanned aerial vehicle resolution level; while the aerial manned imagery decreased the image classification accuracy when comparing with the standard approach.
Data Fusion Mediation server
The data exchange with the data fusion mediation server (linked with WP8 activities) is done through JSON files. The server needs the WP3 cavities and rescue paths. WP4 cavities and rescue paths are also needed when 1) there is no model matching between WP3 collapse-simulations and, 2) in the case the collapse simulation does not match the remote sensing derived models. The generation of the JSON files in the appropriate format were reported in the previous period. In the current period the focus was on how to feed these files to the Data Fusion Mediation Server. This file transfer, by indication of WP8, was performed using POST requests, a request method supported by HTTP.
Wide area surveillance is performed thanks to 3D laser scanners embedded on fixed wing UAVs that cover large areas. 3D point clouds are transmitted to the USaR ground-station and can be displayed on the Common Operational Picture (COP) to give a global view of the area. For smaller areas, such as a city block, 3D models of buildings, rubble and cavities can be retrieved from high resolution 3D data acquired with smaller laser imaging systems embedded on rotary wing UAVs by automatic or piloted navigation. 3D point cloud, fused visible and infrared videos are real-time transferred and displayed on the COP and USaR devices.
Experiments were organized in Roquebillière, France to validate the INACHUS tools. Partners, end users and USaR teams had were involved in a common scenario. A significant time reduction related to the USaR phase can be achieved by wide area situation awareness solutions. 3D laser imaging systems embedded on UAVs can operate in all weather conditions at day or night. The 3D laser scanner performs the strategic surveillance of the environment for various worldwide operations.
WP5 commenced in the first period of the project and concluded in the fourth period. Within INACHUS, a range of victim localisation solutions was developed. These solutions comprise surface sensors (sensor devices to be used and placed on and around the rubble) and the INACHUS snake robot, the latter being a versatile sensor deployment platform, being able to enter the pile of rubble enhancing the technical search capabilities of the end users. The surface sensors are:
• The mobile phone detector, which is a device that seeks to find the approximate location of victims through the signals emitted by their mobile phones. Four methods for victim localisation have been studied, with three effectively applicable on-site:
o The "network" method targets to locate a single phone by reading its list of the six neighbouring mobile network cells and using this information to triangulate its position if the base station location of the listed cells can be obtained. However, this information is not open data. Consequently, this method could not be investigated and tested further.
o The "triangulation" method is dedicated to locate multiple phones at once with a post-processing step: several measurements (complete scans of the environment) are sampled at different locations. The approximate victim location is subsequently inferred from these spatially distributed measurements of signal strength by means of a triangulation algorithm.
o The "proximity" method is dedicated to locate only one device at a time, directly on the field: a "silent" call is made by the prototype, thus entering in continuous communication with the device. By regularly sampling power measurements, the position of the signal source, i.e. the mobile device, can be discovered.
o The "ringing" strategy in which a mobile device can be forced to ring can be considered as an additional aid in finding victims simply by acoustics. This approach could be especially helpful when used in combination with the acoustic sensors (Task 5.4).
The prototype developed was tested successfully during the pilots organised within the duration of the project. Furthermore, a graphical user interface for location presentation in the COP as pinpoints as well as an integration of a density-map in the COP to visualise the concentration of mobile devices were developed. A detailed analysis on the personal handset positioning task is provided in D5.1.
• Ground-based seismic sensors forming a network of distributed vibrational sensors (ground sensors) that pick up vibrations at different points. With this system, vibrations generated by victims through knocking, hitting, etc. can be automatically detected and positioned. The interference of background noise is reduced through an adaptive background suppression technique that improves the usability of the system in real, noisy environments. The system was demonstrated at a field trial in June 15 2016 and was well received by the end users. The system presents detections as an image with colours indicating the location of the detected signals (a.k.a. “heat map”) gives the users a better overview and understanding of what is going on at the site. The system can continuously record data and then analyse them at a higher speed. In this way, a user can “fast forward” through previously data or reanalyse them more carefully or with other parameter settings. Knowing the position of the seismic sensors is needed for the heat map display but not for the background suppression and detection functionality of the system. Positioning of the individual sensors can be done by marking the position in a 3D model or a map (if available), using a positioning solution (e.g. Real-Time Location System, RTLS) or measured manually. With highly sensitive seismic sensors, very weak signals can be detected. However, it can be difficult to distinguish these signals from noise. Finding suitable threshold and parameter values requires extensive testing in a large variety of realistic scenarios, which is not possible to do within the scope of INACHUS. In practice a threshold has to be adjusted by the operator that governs the trade-offs between detections and false alarms in the scenario at hand. The signal detection module is still on a basic level and can be further developed to detect periodic patterns (e.g. repeated knocking) or signals reoccurring at the same location. As a spin-off of the fruitful discussions with end-users it was decided to design a plug-in hardware between the INACHUS seismic sensor system and the Delsar seismic/acoustic listening device in order to add automatic detection and noise cancellation functions to the Delsar system. This feature was successful demonstrated during the pilot 4 event. Moreover, an acoustic system has been built intended for mounting on the INACHUS mobile platform to enable communication with victims trapped in rubble. The system is made from standard audio components to facilitate integration on the platform and was successfully demonstrated to end users in a field trial.
• The Surface Radar, whose purpose is to detect trapped victims by their movements, e.g. the chest moving when breathing. The penetration capabilities of the radar enable it to 'see-thru' objects and therefore making it possible to detect movements of trapped victims behind obstacles. This helps Urban Search and Rescue (USaR) teams to increase the possibilities to find survivors as well as to speed up the whole procedure. The Surface Radar is an advanced UWB radar with nine transmitters and nine receivers with corresponding antennas, all simultaneously active and controlled in a timed manner. It is based on an earlier design that is further developed and improved by INACHUS. It provides information on whether there is a human or not in its scanning scope by measuring the slight movements from the trapped victim. It enhances the high range resolution of an impulse UWB radar with beamforming to get better location information of the target. The Surface Radar is a stand-alone system that is also integrated in the INACHUS environment to supply information about searched areas and detections. The Surface Radar is portable and of the size of a common middle-sized suitcase. It is wirelessly controlled from a ruggedized tablet computer by a single operator. The sensor is placed on top of the rubble pile. For best results a flat area with a minimum of air gaps between the sensor and the material on the pile should be found. The operator starts the measurement from some distance, 2-3 meters away, to avoid undesired influence on the sensitive measurement. The vibration environment on the site needs to be low in order to utilize the highest sensitivity. The Surface Radar is simple to operate and has only a switch for power and a connector that is used to charge the internal battery and if a wired data connection is desired. The device is ruggedized to endure rough environments and handling. In INACHUS deliverable D5.7 a more complete description of this topic is given.
The INACHUS Robot Snake carries the following sensors:
• A thermal long-wave infra-red (LWIR) camera that was used to detect the temperature variations due to heat produced by the victims. In order to detect and locate survived victims, several different sensors were used. These included mobile phone detectors, gas detectors (chemical sensors for human biomarkers), LWIR sensors, optical sensors, seismic and radar sensors. These sensors exploit different physical principles in order to maximize the detection probability and to narrow the identified volume of interest. In addition, a prototype of a snake robot mechanism is developed in the INACHUS project that is equipped with a miniaturized LWIR-camera. This thermal sensor in combination with robust and reliable detection algorithms developed is able to detect humans by thermal radiation. For efficient victim detection in uncontrolled emergency environments, the system exploits the thermal information captured by the LWIR sensor. In order to define with high accuracy which parts of the thermal image should be considered as foreground/background an area detection methodology is applied in order to identify regions with the temperature of a human body (34ºC-41ºC). This temperature range is adjustable according to the user needs and the specific targets to be detected. Objects with the desirable temperature appear in the thermal image with different pixel values. A detailed presentation of the infrared camera used in INACHUS is included in D5.3.
• The Electronic Nose Sensor (e-nose) is located in a compartment of the INACHUS robot snake. The e-nose can provide information about the concentration of gases originating from metabolic activity, as well as indicators of some specific hazardous conditions. E-nose was prototyped under T5.2. Three prototypes have been developed. The main objective for the e-nose, which is human presence detection, was successfully validated during the first pilot test with the second prototype device. The design progressed further with the third prototype including more, even non-mandatory, features while preserving the validated qualities of the second prototype. Tests performed with the third prototype show it has the same human presence detection capabilities which were validated during the third and fourth pilots. It also adds air related hazard detection such as toxic (CO, CO2, H2S, O2) and combustible air environment detection. The third prototype system is compatible for integration in the INACHUS robot and the INACHUS framework. The third prototype of the electronic nose, which is the final prototype developed for the INACHUS project and was the one planned to be integrated in the INACHUS robot (snake-like robot) in WP8, for Task 8.3. The device is designed to include all the technically feasible end user requirements in the constrained space of the INACHUS robot. For these reasons, a completely custom designed device was built using only a few off the shelf components. The electronics and mechanical design were made from scratch. The design of the electronics started before the mechanical design but the completion of both of them required constant exchange of information. The mechanical dimensions of the boards are a result of the allocated space and mounting positions assigned from the mechanical design. The allocated size for the boards in the mechanical design depends on the circuit complexity and the electronic component selection. Software design is relatively independent of the hardware design with the exception of the low-level firmware: its design requires a completed schematic diagram and its implementation is most favourable when at least the assembled PCBs are available. The control interface software of the electronic nose that was hosted on the control interface tablet of the INACHUS robot not planned to be available at the expected delivery date of deliverable D5.4. A standalone interface is available and used for testing purposes. Nevertheless, the design of the integrated interface is mature. Completion of its design and its implementation in the specific platform was successfully completed as part of WP8, Task 8.3. The 3rd prototype’s design is based on the sensor selection and system architecture of the successful 2nd prototype and is extended to implement the non-mandatory end user requirements that deal with safety aspects. The safety aspects that are implemented are the detection of some specific toxic and combustibles atmospheres and require the addition of three extra sensors, for CO, H2S and Combustibles. The CO and H2S sensors are to be in different packages so that there is the possibility, in the future, of building different configurations that can detect other gases. The implementation of the non-mandatory safety aspects is considered to be of added value for the electronic nose especially by the end users. This benefit comes with a significant cost that can be understood by the end users in many aspects of the tool: Size, weight, volume, training, operating, servicing and certification are affected. For the 3rdprototype, replacing the sensors in the field is not an option. The storage of the e-nose does not require the presence of a power source to keep the sensors biased. A combination of sensor selection and circuit design keeps the electrochemical sensors biased even without power. Little or no recovery time is needed for the sensors to be operational after storage. Some design features were included mainly in the electronics that would allow for the configuration of a standalone e-nose with the utilization of some additional modules. These features allow the connection of external devices to serve as the user interface. A tablet could be connected to one of the serial communication ports or a low-cost dedicated interface with vandal proof buttons, LED indicators and a buzzer could be connected to the exposed General Purpose Input/output pins available at an expansion header and the extra actuator output (speaker). The possibility of implementing a standalone e-nose sensor was investigated and a high-level solution was proposed but its implementation was not realized as there is no such obligation and the required resources are not available.
• The Robot Radar, a subsystem in the INACHUS robot, that detects movements of buried victims. The system is a CW-radar operating at 1.2 GHz and utilizes the Doppler effect. It detects movements but has no distance resolution due to the single-frequency that is used. The system has antennas in five different directions which the radar transmitter and receiver rapidly is switched among. This gives a direction estimation of the detection. One big challenge with this radar system is the requirement of the small physical size to fit into the INACHUS robot in combination with high penetration capability that generally means low frequencies which in turn requires large antennas. FOI had the task to design the antennas, each with a maximum size of 30x60 mm2. Each antenna consists of aperture coupled microstrip patches. The low frequency and the small size required a relatively high permittivity of the dielectric material used for the antenna design. TMM6 with epsilon ≈ 6 was therefore chosen as dielectric for the antenna. Cinside designed the RF- and controller electronics that were also restricted regarding the physical size but that did not pose any real problems. The power consumption of the radar system is quite low, but the voltages need to be high-quality and low-noise so they are generated internally from the 24V robot system voltage. Several versions of the radar hardware were developed and tested during the project. A spin-off product was the StickRadar that uses the same radar system but instead of being mounted in the robot it is mounted on a telescopic stick. With its own specially developed operator interface in a rugged wireless Android device, the StickRadar is a stand-alone system that also can communicate with the INACHUS environment or other systems.
• Apart from the aforementioned sensors, the INACHUS robot snake can also carry two video cameras for navigation (one at each end of the INACHUS robot snake) and microphone and speaker for communication with trapped victims.
In the last task of WP5 T5.6 the INACHUS robot design, locomotion mechanism and operator interface has been implemented. More specifically and as detailed, in D5.6 INACHUS robot acts as a mobile platform with the objective to carry several sensors inside the rubble and closer to the victims. Some of the sensors are standard (Video cameras, Audio Communication System), while others are more advanced (Robot Radar, IR-Camera, E-Nose). Audio Communication System was not planned in the description of work; however, it was clear from the first end-user workshops that this functionality is very important. Therefore, it was decided within WP5 that INACHUS robot had to include this device. All the robot parts were manufactured in the 3D CAD design software SOLIDWORKS. Most of the parts were manufactured by 3D-printing them in ABSplus material using the uPrint SE Plus 3D printer from Stratasys. This approach allowed to iteratively design parts, 3D print them, and then test them. When plastic material was not strong enough, the design was updated including metals parts to strengthen the mechanical properties. The design of the track module consists of the following four groups of components.
1. Rubber track and drive wheels.
2. Motor module.
3. Driveline.
4. Chassis.
The first link (module) of the INACHUS robot is the robot head. It is composed by four main parts: the yellow lid/box containing and protecting most of the sensors and electronics; the black track chassis connecting the two tracks to the lid/box; the tracks modules and the joint module. The central module has the largest available payload capacity among the three link modules. It is called standard module because it can be replicated to obtain a robot with more than three links. The electronic nose is the main payload of this module. The tail module and its joint mechanism carry the rear end video camera, the Moog multiplexer, the RTLS beacon, two LEDs, the power distribution board under the yellow cover. In addition, two electric motors are inside the joint and are driven by the embedded custom-made motor controller cards. The motor driver (VNH5019) is a board from Pololu (https://www.pololu.com/product/1451) and was bought from www.farnell.com with the product number 2475453. The module contains a fully integrated H-bridge that can be used for bidirectional speed control of a single brushed DC motor. It has a small size and is relatively low cost. It runs on a supply voltage in the range 5.5-24V. A custom-made circuit board utilizing a microcontroller (LPC1549) from NXP was made to run the low-level control algorithms based on commands from the CAN-bus. A power distribution board was designed and built by SINTEF. There are two identical power distribution boards in the INACHUS robot. The first board is in the robot head module and the second is in the robot tail module. Both communicate over the CAN-bus and supply 12V to cameras and LEDs and provide 5V power to raspberry pi to RTLS beacon and to Moog Multiplexer. The Operator Control Unit (OCU) is one of the main components of the INACHUS robot prototype. Inside the case we find a power supply, an industrial PC, specialized communication hardware and plug connectors. These hardware components are necessary to power the robot, run the control algorithms, connect the robot with its sensors, and communicate wirelessly to the TAUROB Tablet PC and the INACHUS infrastructure. The hardware is only one part of the system; proper software and a robot operator are the other parts necessary to operate the robot. Ever since the first interaction with the USaR teams, at the end-user workshops held in the INACHUS project, it was clear that an important part of the robot was going to be the human-machine interface. The end-users highlighted the importance to have:
• a solid and rugged user interface (Technical Requirement TR-N-RS/25-1), while simple to use (TR-N-RS/05-1, TR-N-RS/08-1);
• camera views, feedback and alarms from sensors visible in the GUI (TR-N-RS/18-1, TR-N-RS/16-1, TR-N-RS/17-1, respectively), both in bright and low light conditions (TR-N-RS/21-1);
• camera views showing a horizon to help with orientation (TR-N-RS/28-1);
• the possibility to operate the robot from several meters away from the entrance point (TR-N-RS/29-1).
To be able to fulfil the described requirements, different solutions were investigated, such as ordering off-the-shelf components, designing parts, and assembling at SINTEF. Finally, TAUROB Tablet PC was selected given that it is a provider of teleoperation solutions for Chemical, Biological, Radiological and Nuclear (CBRN) first responders, Explosive Ordnance Disposal (EOD) teams, fire-fighters and search and rescue teams. Further details on the INACHUS robot are provided within D5.6.
WP6 was the dedicated Work Package of INACHUS for decision-making and planning matters. WP6 commenced on M6 of the project included the following task activities resulting in two deliverables D6.1 on SaR-ESS interfaces with other systems and on Emergency Support System including Common Operational Picture and accompanying documentation. More specifically, The Search and Rescue Emergency Support System (SaR-ESS) represents the cerebral centre of the INACHUS project, and controls and operates the modules and the subcomponents that build up the INACHUS system. Furthermore, measurements and data from various sensors and legacy systems is collected by the SaR-ESS through secure data flow pipelines. The design of the SaR-ESS reflects the effort to pre-empt malfunctions and miscommunication of INACHUS as a whole system, but also to introduce innovative technological solutions, integration techniques and interfaces that come from the rich scientific and technical background of the project consortium in order to facilitate and contribute to future projects as well. The figure below illustrates the Search and Rescue Emergency Support System (SaR-ESS) architecture that has been drafted by the respective partners and which has been constantly updated as the system evolved to its final form. T6.1 and T6.2 are related to the development and deployment of the Multi Source Information Fusion Engine (MIFE) and its integration with the SaR-ESS. The Multi-source Information Fusion Engine (MIFE), is a data fusion module in which data from the different sensing elements coming from the field are combined, examined and processed. The MIFE includes the Data Fusion Mediation Server, which incorporates expert reasoning applied on the collected data from various sensors, using the INACHUS Ontological Model as a basis for the structure of the INACHUS data and their interrelations, and conforming to intelligent algorithms and pre-defined rules. The end result is the location of possible survivors. This result is published in the Fusion Backend, which in turn sends this information to the COP as an alert message. The data is also stored in database for possible future reference. The development of the MIFE API commenced based on the Java Spring Boot framework. The implementation of the MIFE connects to the RTC-Gateway and receives data from sensors. Listener classes have been implemented to receive the sensor data and loader classes to extract data from JSON files. In addition, loader classes have been implemented to extract cavities data from JSON files, which are sent to the Gateway to test initial rules on the Complex Event Processor (CEP). The Data Models for the Cavities and Alert messages was also defined, and the MIFE was updated to output alert information to RTC-Gateway to be displayed on the COP, which includes amongst other information, the possible location of survivors. SaR-ESS Portal consists of a group of components that together they form the administration dashboard of the INACHUS Platform. Alongside with the COP Dashboard, they shape the means of controlling and organizing a search and rescue operation. With the SaR-ESS Portal the Rescue operator can perform tasks, among others, such as sectorization, worksites’ management, teams’ management, monitor of the operation, manage the INSARAG Forms submitted at the field of operation etc. The SaR-ESS Portal is divided in two major parts the frontend and the backend. These two components communicate with the COP server, the authorization server and the mobile application. SaR-ESS Mobile application refers to an android (version 5.5.1 onwards) application oriented for tablet devices (with screen size 7” and up) which is focusing in supporting the work of the rescuer at the field of operation. The Environment Services (ES) encompasses the components that are necessary to serve map data and operational information to the instances of the Common Operational Picture (COP). This module consists of two components: The ES server, which provides GIS data and other geo-localized datasets, and the COP server, which provides the operational data and necessary metadata for the COPs. The Common Operational Picture (COP) is a map-centric supervision tool with advanced visualization features and annotation capabilities for collaborative management of crisis relief activities and real-time supervision of the crisis-affected region. The COP gives an overview about the scene in 2D and 3D, and is enriched with operational data in form of annotations such as text, icons, geo-localized photos, as well as the sectorization plan, worksite positions and team locations. Moreover, the datasets that are produced by WP3 and WP4 can be displayed and navigated directly on the map in the 3D view. The visualization of all this information in the most legible and user-friendly way possible is the challenge in the development of this component. The COP is available as a desktop application and a mobile application for Android.
Crisis management poses specific technical challenges that INACHUS addressed and they are related to:
• the large number of network subscribers that need to be served (crowd and first responders)
• the lack of prioritization schemes to support the first responders in most types of wireless communication (currently only TETRA claims to adopt prioritization schemes)
• the scarce nature of communication resources (i.e. the limited total wireless bandwidth available for a specific cell/access point) and the bottlenecks that appear in cases of congestion (e.g. in 3G mobile, signalling resources can be congested before the actual data bandwidth is consumed)
• the total time required for establishing a two-party or a multi-party call (normally for 3G mobile in the order of 5-8 sec, exceeds 20sec in cases of congestion with similar number in 4G band) and respectively the time required for one to register a wireless client to an access point (cases of WiFi/WiMax)
All these aspects were thoroughly analysed and tackled under WP7. All tasks in this WP aimed at the deployment of a reliable and stable communication platform. The communication platform that INACHUS developed plays a central role in the topology envisioned by the INACHUS project, and manages the seamless interoperation, the interconnection between other networks (cellular, etc.), as well as provide redundancy and recovery functionality. A multi-layer architecture is followed for the communication and the routing as well as the mobile gateway functionality. The mobile gateway can be accessed by the operator, from the control centre as well as locally by the first responders, when/if needed; as well as remotely controlled and maintained by specialized maintenance staff. For extending the capabilities and range of the crisis network in case of failures and/or congestion and to provide a distributed architecture, portable gateways and a long-range communication solution are envisaged. These gateways can be positioned according to relevant and appropriate network dimensioning, even deployed by personnel where needed (i.e. mobile gateways) and they can be automatically or manually switched on. The gateways support the functionality of any other mobile USaR center as an alternative and cost-effective solution in an integrated ad-hoc manner but with redundancy, extendibility and security capabilities to assure robustness, flexibility and reliability. The communication platform is complemented by a novel Real-Time Localization System (RTLS) - based on wireless communication and in particular focused on robustness and efficiency - which is crucial for USaR operations. Of special concern is the localization of targets under the rubble, which could be of irregular density and composed of heterogeneous materials (maybe metallic) that greatly increase the signal distortion and multi-path effect. Special focus was placed in technologies, protocols and algorithms resilient to multipath errors. Combined approaches were envisaged in order to compensate the weak points of the selected technologies. According to the momentary needs, the system should switch automatically between different technologies and adjust parameters (such as frequency bands) to give the best-effort localization.
The INACHUS localization system (RTLS) consists of clusters of devices with locating capabilities called beacons. Their purpose is to work in a cooperative way to relatively locate other beacons or tags. Absolute positions are given relatively to a known reference point. Beacons can be deployed as a separate device or be integrated within INACHUS mobile gateways.
In order to increase the accuracy and reliability of the system two localization domains were defined: surface location and underground location. Both domains work cooperatively to give relative and absolute positioning, although each of them targets specific challenges.
• Surface RTLS: probed state of the art RTLSs targeted to indoor environments were researched as the scenario expected on the surface are similar to an indoor environment. Integration with GPS was also explored to enable global positioning when environment changes from indoor-like to open-space/outdoor. The surface beacons were designed to allow a progressive, fast, in real time deployment and with little or no configuration at all. The system creates dynamic clusters and locate each element (beacon/tag) iteratively. Apart from the surface location, the Surface RTLS is the backbone and reference system for the positioning of the robot under the rubble.
• Underground RTLS: positioning an element in an environment full of irregular and heterogeneous obstacles is prone to high range distortions and multi-path effects. Technologies more suited for multipath-prone environments (such as UWB) in addition to error-aware algorithms were targeted in order to estimate and reduce the undesirable accuracy degradation. Additionally, a specific underground localization infrastructure was designed to bind the error. As the most effective approach, the underground nodes were attached to the power wire of the robot. This deployment is made at small intervals (≤1m) to minimize the ranging errors and increase the localization granularity.
WP8 deals with the INACHUS system integration activities. The integration process was defined for the INACHUS platform as a progressive and iterative process where technology components were first individually developed and verified (functional and inter-operational) and subsequently integrated to conform and validate the ready-to-deploy INACHUS framework platform. The general objective of WP8 was therefore to integrate all the hardware and software components defined for this INACHUS system framework. Partial objectives were the supervision of the execution of the Integration Plan, the definition and execution of the Integration protocol and the traceability and quality assurance of the Integration. The methodology of this integration process was structured in two steps: The first, and preliminary step, was focused into control that the individual components were ready for the integration. The second was the integration itself. For the first step, it was necessary to have functional validation traceability of the components and the compliance validation with interfaces (defined in the corresponding Interface Control Documents). The second step was the progressive integration of the components in a platform system as it was defined in the deliverables D8.2 and D8.3 following the Integration Protocol that was also defined as deliverable D8.1. The progressive integration was structured in partial integration levels that in the case of the Robot Snake and standalone sensors were supported in a specific component for integration: The Embedded Communication System. These partial integration Levels and finally the Integrated Platform was technically validated against technical requirements and use cases scenarios.
This methodology was structured around five tasks: T8.1 Functional and Validation tests before integration, T8.2 Embedded Communication system for the Mobile Platform., T8.3 Installation of sensors and communication module on the Mobile Platform, T8.4 Integration of the INACHUS System Framework and finally T8.5 Quality evaluation and continuous common connector update.
Task 8.1 concluded with the first definition of the interface and communication protocols compiled in the corresponding Interface Control Documents (ICDs) and with the individual components compliance validation according that defined interoperations.
T8.2 concluded with the development and deployment of a specific software component for integration: the Embedded Communication System (ECS) ,defined first for the mobile platform and finally also extended for the integration of the stand-alone sensors (Mobile Phone Detector, Seismic sensors and Surface Radar):. The ECS acts as an internal interface between the robot sensors and the robot OCU and as an external interface between the mobile platform (and the stand-alone sensors) with the ESS/COP via the Communication Infrastructure and the Real-time Communications Gateway (RTG). Additionally, it acts as a data broker (in this case without Data Base) to access the Robot Snake and sensor data. The ECS application provides a common API for all sensors and for the Robot Control System implemented as a library/interface that hides all the complexity of the real time communication processing and interoperation. The ECS contains two interfaces: Local and Remote. The Local interface is used by the Mobile Platform processors (Robot OCU and sensors) to interact with the ECS and to send data to any internal interested clients. On the other hand, the Remote interface is used by remote clients (ESS) to access data produced by the processors by means of a protocol structured in four layers where the two lower layers are standard and the two-upper layer are specific protocols for INACHUS. WebSockets protocol and STOMP protocol are the lower layers, the INACHUS ECS Policy layer and the JSON Models & Metamodels are the top levels. The higher layer of the remote API interface defined is then the INACHUS Data Model architecture that is the final information interchanged with the MIFE, through the RTC- Gateway also via the STOMP protocol.
Task 8.3 was focused in the specific physical integration (mechanical, electrical & communications) of the Mobile Platform. As indicated, in WP5, the INACHUS robot was designed and developed. In task 8.3 the e-Nose, LWIR (infrared camera) and Doppler radar units, all also developed in WP5, were integrated on the INACHUS robot with the on-board electronics, communications and also with the RTLS developed a WP7.
The INACHUS robot is structured into three modules (Head, Central and Tail) with joints between them, where each module has two tracks (self-contained, integrating the motor and electronics) for improved propulsion/performance. The modules are connected by active joints, meaning that the joints are actuated and therefore it is possible to change the relative position of two adjacent links. This modular snake-like body allows for carrying several sensors on the modules. Some of the sensors are Commercial Off-The-Shelf (Video cameras, Audio Communication System), while others are the sensors developed in WP5 (Robot Radar, IR-Camera, E-Nose) and WP7 (RTLS). Additionally, the robot provided components to support the integration: communications, power supply and processing units. All these components, physically integrated with functionally interoperated by means of the ECS, constitute the integrated Mobile Platform.
The task 8.3 concluded with this Mobile Platform fully integrated and ready to interoperate as a subsystem in the INACHUS platform. The mobile platform, defined to be deployed in the Worksites, provides real-time information related to the victim’s location to the INACHUS backend applications (ESS in Sector Command/Base of Operations) via the ECS and the INACHUS communications infrastructure (WP7).
As indicated, Tasks 8.4 and 8.5 involved the core activities of the INACHUS platform integration. Task 8.4 encompassed the work performed in the technical WPs to bring the INACHUS System Framework composed of heterogeneous technologies and layers with variable degree of complexity. So, in this task all the modules previously developed and independently validated were integrated together progressively to build the INACHUS platform meanwhile Task 8.5 was focused on quality assurance of that integration.
INACHUS platform is structured around a high number of complex components with also a high number of interoperations, all defined in the corresponding ICDs. Some of the interoperations are simple file exchange mechanisms, like file sharing, but others present high complexity interfaces, like the real-time interoperations conformed by means of the ECS. In order to manage these different levels of complexity, a progressive structure of five differentiate levels of integration were defined, from components to subsystems, with a Level 5 where the platform was validated as a complete system in a series of testing scenarios supported in the Use Cases defined for INACHUS.
This integral functional/interoperability validation was performed following a strict planning in order to leverage the possible interoperability errors of each element/layer interconnection as soon as possible. The traceability of technical quality of this process was covered by confirming that the individual developments were functional and compliant with the technical requirements, by confirming that the components were compliant with the interfaces and protocols defined in the Interface Control Documents and by confirming that the subsystems and the Platform as a whole were functional, with composability guarantee, and also compliant with the technical requirements.
The WP8 concluded with the Integration Protocol fully executed, in physical integration workshops complemented with virtual Integration Workshops, the physical Integration of the Mobile Platform finished and the complete INACHUS Integrated System fully defined and tested ready for demonstrations. Thist last technical WP9 of INACHUS started during the second period of the project and ended in the fourth period. It included integration activities related to pilots which occurred with the following details:
• Pilot 1, Sweden: Testing and validation of system components in realistic environments
o Victim localisation tools, coordination tool prototype
• Pilot 2, France: Small scale catastrophe for performance assessment
o Wide-area surveillance tools (drones, laser scanners) to monitor collapsed buildings; Simulation tools for structural damage analysis and casualty estimation
• Pilot 3, The Netherlands: Explosion in an industrial installation and buildings collapse due to sabotage, terrorist attack
o Initial demo of the integrated INACHUS system
• Pilot 4, French / Italian border: Final testing and validation of the international adaptability of the system
o Final demo of the entire integrated INACHUS system
Potential Impact:
Potential impact (including the socio-economic impact and the wider societal implications of the project so far) :
INACHUS main outcome, namely the delivery of a robust, resilient and secure platform for the provision of specialised ad-hoc services, facilities and support for first responders that operate at crises scenes.(the INACHUS tools, sensors, communication systems and building assessment software) reaches a wide applicability at an EU level and provides important enhancements at the service of Civil Protection and Crisis Management and more specifically in Urban Search and Rescue activities. The following high-level potential impacts evidence usability and crisis functions of INACHUS:
The increased effectiveness of FR operations through the employment of INACHUS leads to:
- Increased situation awareness
- Fast response of the communication infrastructure (e.g. priority services and QoS)
- Increased communications coverage with ad-hoc networking capabilities,
- reduction of injury and loss of life of the FRs and ultimately civilians by gaining faster, more accurate and safer localisation information and supporting guidance information.
- Preparedness may be enhanced, as the INACHUS system may act a simulation software
- A robust Situational Awareness mechanism that allows decision makers at both strategic and tactical levels to make well informed decisions has been developed.
- The CWA initiated by INACHUS consortium orientates the development of guidelines and consensus reaching process on the most efficient and harmonized ways to deploy secure
- INACHUS focused on the cooperative idea of sharing know-how in crises’ early stages, helping bring all EU countries into a coordinated Search and Rescue Operations’ handling framework. Emergency organizations in Europe were therefore able to harmonize their procedures through a common crisis database.
Since project conceptualisation, 11 high level objectives/Expected final results have been foreseen that drove the entire work implementation from idea to delivery throughout the 48 moths of project duration. As explained previously the entirety of project objectives have been accomplished and delivered, hence in the following we report on high level accomplishments and their potential impact (including the socio-economic impact and the wider societal implications of the project so far). By successfully delivering the high-level objectives, INACHUS may claim on achieving a wide site of associated potential impacts as per below.
1. INACHUS successfully developed simulation tools for estimating the number and locations of survival spaces created after a structural collapse
2. INACHUS deployed successfully Decision and Planning Components for Advanced Casualty and Damage Estimation.
Related potential impacts:
By successfully delivering the above high-level objectives, INACHUS may claim on achieving a wide site of associated potential impacts as per below.
- Stimulation of innovation in USaR field is achieved in the EU and beyond.
- Emergency response crews can be provided with critical and timely information on damage in monitored facilities so that danger can be pin-pointed and emergency response directed with precision
- Disaster results in less human victims
3. Successful integration of existing and novel sensors
Related potential impacts:
- INACHUS introduces a cost-effective and of high-performance mixture of sensors resulting in accurate, flexible and faster USaR activities in the field
- European leading position in this market which gains momentum over the last years in various application domains
4. INACHUS advanced the state of the art by developing a robust snake robot mechanism together with a snake
Related potential impacts:
- To result in accurate victim positioning without risking USaR team’s members lives
- To enhance methodologies of searching for entrapped victims
- To pave the way for alternative means of USaR activities
5. INACHUS successfully addressed interconnection issues between the developed devices
Related potential impacts:
- to decrease time of reaction
- to drastically increase the efficiency of the relevant actors
6. Successful enhancement in data fusion and analysis techniques
Related potential impacts
- To provide the rescue teams with a clearer picture of the situation at hand and eliminating redundant time to search
- To determine escape route, so that rescue teams can reach these potential survivors and extract them from the debris in the minimum possible time
7. Successful INACHUS system integration
Related potential impacts:
- To provide a reliable and stable system supporting USaR activities in the field
8. Successful contribution to standards/best practices and guidelines for the USaR operations through strong user presence
Related potential impacts:
- To provide a common approach from all stakeholders in USaR robotic platforms interoperability issues
- To enhance the technical and procedural interoperability in USaR offered technologies
9. Consideration of Societal Impact, Legal and Ethical issues of the proposed solution.
Related potential impacts (Points of Obj. 1 also apply):
- To provide a secure solution covering all Ethical and Legal issues respecting USaR crews and entrapped victims’ fundamental rights privacy
10. Arrangement of several field tests
Related potential impacts:
- To provide a stable and reliable solution tested in real conditions to USaR and CP entities and organization
- To provide a solution tested in different scenarios by different USaR teams.
11. Development of training package and extensive training courses
Related potential impacts:
- To increase the trust of USaR teams to the INACHUS solution
- To familiarise and create a user-friendly support tool for USaR activities
Main dissemination activities :
Public awareness activities aimed to spread the information about the project to the broad community to set the ground for exploitation of the results or possible technology transfer. Major awareness/communication and dissemination activities included: Project public website, Six-monthly newsletters, Social networks to communicate with larger audiences Videos showing the INACHUS system at work at YouTube, Press releases, INACHUS leaflet and poster, Presentation of the results to the public at large, Project workshops, Presentations/ scientific publications.
INACHUS Website:
A project website (www.inachus.eu) has been developed in order to disseminate the project outcomes; it includes project objectives and background, significant achievements, technology news, consortium contacts, scientific publications and presentations, etc.
Project Newsletters:
Four newsletters have been produced that depict the project's main outcomes together with dissemination activities and upcoming events related to the project itself.
Social Networks:
Facebook https://www.facebook.com/pages/Inachus-USaR-Research-Project/1649612961918245
Average of 3 posts per months (280 followers and 520 posts)
Twitter https://twitter.com/InachusUsar
4 tweets or retweets a week (a total of 1107 tweets, 185 followers)
LinkedIn https://www.linkedin.com/grp/home?gid=8385769
A total of 122 connections, 77 discussions launched, 84 members)
YouTube Channel https://www.youtube.com/channel/UCBz08Jf7tVT08x5LevztXcQ
20 videos from technical partners.
Press Releases and Radio/TV Interviews:
8 Press releases during the project: for Inachus Events, Achievements and Fields tests. During the second field test held in France a journalist from a National newspaper (Le Progrés) made interviews and a paper, and during the third field test of INACHUS project, held in Weeze, Germany, a journalist from a Dutch Fire and Rescue magazine Een-Een-Twee made a visit on the pilot site to interview partners and end users. Those interviews lead to a newspaper article, published during the summer.
Two more journalists made a visit to the pilot site:
- A journalist from a German National Radio (WDR5) made interviews of ends users and partners during the pilot. Those lead to a report of 2min30s on Friday 20 April morning on WDR5 in the "Morgenecho". Follow the link bellow to listen to the interview: https://www1.wdr.de/radio/wdr5/sendungen/morgenecho/index.html
- A journalist from RP online made a visit on the pilot site and carried out interviews and pictures report on the victim localization tools. This leads to a paper in German: http://www.rp-online.de/nrw/staedte/kevelaer/roboter-schlange-sucht-verschuettete-aid-1.7528750
- Two press releases about INACHUS final conference: one in French to inform local press and one in English for International Press. Link on our website:
- TV interview on National TV Channel (France 3) about the large USAR exercise where INACHUS final pilot was mentioned. http://www.entente-valabre.com/blog/262/48/sauvetage-deblaiement-interview-du-lt-col-p-tosello-a-france-3
- Newspaper article about the National USAR exercise in a Regional Newspaper (Nice-Matin), collocated with INACHUS final pilot (see above)
https://www.nicematin.com/vie-locale/photos-des-pompiers-de-la-france-entiere-ont-choisi-larriere-pays-pour-des-exercices-grandeur-nature-277464
- An article in “Air and Cosmos” (National reference on aeronautic, space and defence) where the pilot 4 is mentioned (above).
- Others papers on general support were also published during the project lifetime: concrete.org Nice Matin
INACHUS Leaflet and Poster:
INACHUS leaflet was first designed and printed on M3. Brochure was designed and printed on M13. Technical poster has been designed as well. In addition to the technical poster in the format of a standing roll-up banner (80cm × 200cm), a version in the format DIN A0 has been prepared with the intention to present it at conferences, where A0 is the common poster format for poster sessions. The logo poster has been reprinted in 2 copies in M34.
Clustering with other projects and initiatives
Clustering with ICARUS and DARIUS projects has been performed this first year. Furthermore, INACHUS has also identified RECONASS and EUCONCIP as relevant projects. In the second year, INACHUS considered the option to organize a common technical workshop with SPARTACUS with a focus on the communication and localization technologies, but this opportunity was cancelled by SPARTACUS partners. First exchanges have been started with DESTERIO, MOBNET, TRADR and eVACUATE. Liaison relationships have been established with the MOBNET and eVACUATE projects. MOBNET presented its developments in a special session during the 6th INACHUS plenary at Freiburg. The INACHUS Exploitation Cluster workshop occurred in Weeze, Germany, April 2018 was formed by the following projects:
DRIVER+, IN-PREP, TOXI-Triage, CAMELOT, beAWARE, ANYWHERE, STORM, GHOST, CIP-SEC, SMR, SAYSO.
INACHUS Workshops:
The Final Conference was collocated with the INSARAG Regional (Asia, Europe, Middle East and Africa) meeting in Valabre, Gardanne France and was an opportunity for EPLFM to present INACHUS project to deciders of INSARAG. INACHUS was present during the INSARAG meeting during the two days with a stand in the tent where the standing lunch was served. People were available and EPLFM had very good contacts with many INSARAG representants from different countries: Germany, France, Estonia, Czech Republic, Israel, Hungary, Morocco, Tunisia, Italy, Poland, England, Belgium, Jordania and OCHA, as well as a representant from the EC. All of them were willing to learn more about the project, took the envelope from the Final event as well as leaflet and newsletters. Most of them communicate their mail to stay in touch with the project. Two French industrials specialized in the USAR technologies were willing to take contact with the partners of some tools.
Presentations in Fora/Events/Conferences related to Urban Search and Rescue:
During the fourth reporting period several of the project partners contributed to disseminate information about the project and its results at international events and meetings. These are: the International Lidar Mapping Forum (ILMF) and ASPRS Annual Conference 2018 in Denver, Colorado, USA , the Security Counterterror Expo in London, UK , the European Robotic Forum 2018 in Finland, the Swedish Microwave Days in Sweden, the ISPRS Technical Commission II Symposium 2018 “Towards Photogrammetry 2018” in Riva del Garda, Italy, the 5th International Workshop on Emergency Networks for Public Protection and Disaster Relief 2018 (EN4PPDR ‘18) in Limassol, Cyprus and the Forum Innovation Defense 2018, in Paris, France, ASCE Structures Congress 2018, in Forth Worth, Texas.
Moreover, 4 papers were published in the IEEE Transactions on Network and Service Management journal, in the Een-een-Twee: Tijdshrift over Brandweer en Hulverlening, the Engineering Structures journal and the Remote Sensing journal.
Two workshops were organized by the INACHUS project consortium. A CEN/CENELEC Workshop in Brussels, Belgium and an INACHUS Exploitation Clustering Workshop in Weeze, Germany. Also, an interactive demonstration was held at a field trial in Revinge, Sweden.
Furthermore, during 2018 INACHUS presented its outcomes in two field tests, one in Weeze, Germany in April 2018 and another one, in Roquebillière, France in November 2018. A final event was organised in Valabre, France in October 2018.
More specifically the following dissemination activities took place:
• A paper entitled ‘’3D laser imaging techniques on UAVs: Detection and localization of collapsed buildings and trapped victims’’ was presented during the International Lidar Mapping Forum (ILMF) and ASPRS Annual Conference 2018 in Denver, Colorado, USA, 5-7 February 2018
• A keynote lecture and a project presentation entitled ‘’ A 3D Urban Planning Tool quantifying Urban Risk and Resilience’’ were performed at the Security Counterterror Expo in London, UK, 6-7 March 2018.
• A presentation entitled “Snake Robots for emergency response’’ was presented during the European Robotic Forum, in Finland, 15 March 2018.
• The 3rd field test was held in Weeze, Germany on 20 April 2018
• A CEN/CENELEC Workshop “Interoperability framework for functional components of a modular search and rescue robotic platform” was organised by the INACHUS project consortium in Brussels, Belgium, 9 May 2018.
• A poster presentation entitled “Low Cost Aperture Coupled Patch Antenna Design” was presented at the Swedish Microwave Days, in Sweden, 24-25 May 2018.
• A paper entitled ‘’Satellite image classification of building damages using satellite and airborne image samples in a deep learning approach’’ was presented at the ISPRS Technical Commission II Symposium 2018 “Towards Photogrammetry 2018” , in Riva del Garda, Italy, 3-7 June 2018.
• An interactive demonstration and a discussion organized by FOI, among the stakeholders was held in Revinge, Sweden, 26 June 2018.
• A paper entitled “INACHUS: Europese innovative voor Urban Search and Rescue” was published in the Een-een-Twee: Tijdshrift over Brandweer en Hulverlening (Dutch journal, professional USAR), June 2018
• The paper ‘’Reliability of Collapse Simulation-comparing Finite and Applied Element Method at different levels’’ was published at Engineering Structures journal, September 2018
• A paper under the title “A resilient, multi-access communication solution for USaR operations: the INACHUS approach” was presented at the 5th International Workshop on Emergency Networks for Public Protection and Disaster Relief 2018 (EN4PPDR ‘18) Limassol, Cyprus, 15-17 October 2018
• A paper under the title “Enhancing Information Resilience in Disruptive Information-Centric Networks” was published in the IEEE Transactions on Network and Service Management.
• A paper entitled “Multi-Resolution Feature Fusion for Image Classification of Building Damages with Convolutional Neural Networks” was published in Remote Sensing journal, October 2018
• The final event took place in Valabre, France on 16 October 2018
• INACHUS project was presented to INSARAG regional meeting participants in Valabre, (Oct 17 & 18) France, where INACHUS was invited.
• A paper entitled “PROGNOSEFÄHIGE NUMERISCHE SIMULATIONEN ALS NACHWEISMETHODE: VOM BAUTEIL ZUM BAUWERK“ was presented in the 8th workshop Bau-Protect 2018, Munich, Germany, 13-14 November 2018
• The final field test was held in Roquebillière, France on 14-15 November 2018
• The INACHUS project activities related to 3D laser imaging techniques on UAVs, detection and, localization of collapsed buildings and trapped victims were presented in the Forum Innovation Defence 2018, Paris, France, 22-24 November 2018.
• The Journal Paper “Reliability of Collapse Simulation- comparing Finite and Applied Element Method at different levels” has been published in Engineering Structures 176:265-278, December 2018.
INACHUS Stakeholder’s community
The number of members reached the 105 persons in the end of the project. The stakeholder community matrix, sorted out accordingly to country, type of stakeholder has been updated and the new file has been uploaded in the Redmine: https://redmine.iccs.gr/projects/inachus/dmsf?folder_id=2486
4.1.5 Exploitation of results
INACHUS showcases a wide set of exploitation activities, witnessing the great potential of the system’s commercialisation. INACHUS outcomes have been grouped into 3 sub-categories as following:
1. INACHUS Single Devices
Within this group, there are all the systems and devices developed in INACHUS including:
• Victim localization devices such as the Snake Robot, the e-Nose sensor, the ground-based seismic Sensor system, the surface radar, the miniaturized IR-Camera system for helmet, the Stick Radar,
• Emergency Support Systems such as the INACHUS Common Operational Picture (COP) and the INACHUS Mobile application,
• Secured Communication positioning systems such as the INACHUS Mobile Gateway
2. INACHUS Technologies, S/W and Simulations
Within this group, there are the different technologies, simulations and software developed in INACHUS including:
• Simulations such as Tool for Structural Damage Analysis and Casualty Estimation, Collapse simulation and cavity identification and 3D mapping simulations
• Technologies such as 3D mapping technology and Victim detection and recognition technology
• Software such as administration Software to support ESS-SAR and MIFE.
3. INACHUS platform
This group consists of the overall platform of INACHUS developed within the frames of this project including:
• USaR supportive systems and INACHUS Robot Localization (RTLS). IPRs of the Consortium Members:
The overall INACHUS solution considered in this plan is a solution that includes at least the top level organizational solutions that provide situation awareness, support and communication services and finally some of the bottom level solutions such as sensors and simulation tools. The core/backbone of the INACHUS solution is the COP, the SaR-ESS, ECS, MIFE and mobile gateway. The INACHUS solution is designed to be modular and expandable. It includes a variety of solutions for same or similar applications. Depending on the needs and size of the client organization or the needs of a specific deployment a configuration with different types and numbers of sensor systems, drones, simulation tools, communication gateways can be interconnected to the backbone as well as configure the backbone itself. Applications/organizations that do not require the core solutions could use a combination of the individual exploitable results. The USaR market for solutions of the INACHUS size, determined mainly by cost, has many potential user organizations but not that many buyers. The nature of the INACHUS solution which is in essence part of the humanitarian sector is also a valuable realization when seeking for buyers. The approach for performing the market analysis of the overall solution considers the potential user organizations. Ideas on how to address the buyer’s situation are available but reside on the establishment of some sort of end user recognition/ satisfaction and also should take into account the various specificities and differentiations existing in USaR teams coming from different countries in E.U.
During the project lifetime, several partners tried to exploit their outcomes through different channels, including the end-users already participating in INACHUS project. Below we are listing these tangible exploitation activities, as performed from each corresponding partner.
INACHUS short stories of exploitation activities as performed during project lifetime
DIGINEXT: DIGINEXT had been in direct exchange with the INSARAG working group on information management to investigate options how the INACHUS COP, possibly with the connection to the ESS, can be acquired by INSARAG USaR teams. Although the working group members are very enthusiastic about the adoption of the INACHUS COP and its benefits for real USaR operations, no funding could be secured so far by the information management working group to finance the licence fees and maintenance of this software.
Recently, DIGINEXT has made a partnership with the school of application of civil security of EPLFM, and is in advanced contact with the firefighter brigades of several French departments, for example SDIS57. EPFLM is now adopting a DIGINEXT product as operational and training solution, which is compatible with the INACHUS COP. It can be expected that the firefighters that are also USaR professionals will therefore opt to adopt the INACHUS COP to continue with this software also for their USaR operations
EXODUS: EXO has initiated discussions with USaR NL in order to investigate the possibilities of commercializing the ESS portal and mobile application through USaR NL an INSARAG Group member. A physical meeting was held in Athens (EXO premises) where EXO presented the mobile application and portal to USaR NL representative. Within this meeting and several remote discussions that followed-up this meeting, EXO had the opportunity to prepare and propose a high-level functionality matrix of the mobile and portal application for USaR Teams as well as a time estimation towards the finalization and productization of the SaR-ESS and communicate it the end-users. As INSARAG is now in a process of refining their operational procedures and specifically investing on new technologies and tools that will further strengthen and facilitate their day to day operations, EXO is expecting to follow-up these initial exploitation activities and by exploiting this transitional period that INSARAG group is currently running to invest on the ESS portal and mobile application and introduce it as an integral tool for USaR Teams towards the optimization of their tactical operations and field reporting procedures.
CINSIDE: Cinside has presented and demonstrated three different radar systems to localize victims for end-users and stakeholders during the INACHUS project, in pilots and exercises. During this period, we have collected feedback and general information that have been very important for the development and will be for the future marketing activities. Our goal is to bring at least one of the radars to the rescue market. The network of rescue professionals we have gained during the project is invaluable for tests and verifications, and hopefully also as the first reference customers, those are SBFF, EPLFM and USAR.NL within the project and numerous external e.g. UK-ISAR and MSB (Sweden). We have also found a potential reseller of our coming rescue products, LEADER France.
EMI: EMI stays in contact with VirtualCitySystems GmbH, a leading corporation for the generation and management of 3D city models. VirtualCitySystems GmbH is very interested to integrate the algorithms of the mapping software, a result of INACHUS WP3, into their 3D city engines and signed a letter of interest during the project period of INACHUS. First steps for the development of an interface to the mapping software and first workshops with German seismological services and institutions for crisis management are performed. Future steps intends an automated integration of a web based service to apply the mapping software via a 3D city model, which are currently available for many public authorities.
FOI: In 2018 FOI and the Swedish Civil Contingencies Agency (MSB) initiated a dialogue about possible routes towards a product based on the seismic system development in INACHUS. These discussions are likely to continue and deepen during 2019. It is expected that decisions will be taken regarding whether to aim for such a product and then – if yes – how to organize and support development towards it.
Research Highlights: Apart from the aforementioned activities, a core group of partners participating in INACHUS (ICCS, EXO, EPLFM, SINTEF, DXT, FOI and SBFF) joint their forces and by exploiting the outcomes/knowledge gained through their involvement in INACHUS project, they submitted two new proposals (CURSORS and INGENIOUS) under SEC Call 2018 in DRS-02 Topic that were finally funded by the European Commission, giving them as such the opportunity to further extend, exploit and improve their work as developed in INACHUS the previous years after its end.
Standardisation and Patenting:
INACHUS initiated a CEN Workshop Agreement (CWA) in order to provide guidance on the technical and procedural interoperability of urban search and rescue (USaR) robotic platforms. More specifically, the standardisation initiative taken by INACHUS focused on interoperability (hardware, software) between the USaR robotic platforms and the equipment, sensors and tools that are attached to them. It also provides guidance on the principles for enabling USaR robotic platforms (various types of them such as drones, snake-like, robots with wheels, legs, etc.) to operate in all ground search environments. In this way a generic platform can be adapted, designed and built for any possible search and rescue (SaR) scenario on the ground. The final outcome of this initiative was a submitted CWA to the CEN/CENELEC entitled “Urban search and rescue (UsaR) robotic platform technical and procedural interoperability - Guide and the components of them” with identification number CWA 17357. This CWA can act as the starting point in developing interoperable USaR robotic platform solutions and is for use by organizations responsible for designing, manufacturing, configuring, customizing and maintaining USaR robotic platforms, tools, equipment and sensors (e.g. drones, snake-like robots, mobile robots with wheels, cameras, lasers, environmental analysis sensors, etc.). The CWA is also for use by integrators and providers of SaR support systems in general, first responder organizations, operators of the USaR systems, public authorities and end-users in the field dealing with USaR mission organization and execution. Last but not least the CWA is also of interest in the procurement of USaR platforms.
Given that a CWA document is created and supported by an open list of organizations and individuals this specific CWA document will be circulated among all related actors in order to follow its recommendations (date of official publication to CEN/CENELEC website was 27 February 2019, link: https://standards.cen.eu/dyn/www/f?p=204:110:0::::FSP_PROJECT,FSP_ORG_ID:67489,2449074&cs=1BC7B9E467856890E384D9915FDABAB4E ). Furthermore, as a CWA can be an initial step in the development of a European Standard all partners will support it after the completion of the project by expanding the involved entities and organizations list and by promoting its recommendations within EU related projects and initiatives either ongoing or forthcoming.
List of Websites:
Address of project public website and relevant contact details :
As projected from the various INACHUS dissemination activities, a public website and other social groups have been created since the on-line presence of a project or initiative are nowadays considered to be a powerful tool towards diffusion of knowledge to all possible directions and audiences, permits interaction with individuals, groups, initiatives and projects that share similar interests and increases significantly the overall impact.
-Project Website: http://www.inachus.eu/
-Facebook https://www.facebook.com/pages/Inachus-USaR-Research .
-Twitter https://twitter.com/InachusUsar
-LinkedIn https://www.linkedin.com/grp/home?gid=8385769
-YouTube Channel https://www.youtube.com/channel/UCBz08Jf7tVT08x5LevztXcQ
The INACHUS Consortium and Contact details :
1. Coordinator: INSTITUTE OF COMMUNICATION AND COMPUTER SYSTEMS (ICCS), Angelos Amditis, a.amditis@iccs.gr, Greece
Partners:
2. EXODUS ANONYMOS ETAIREIA PLIROFORIKIS. (EXODUS S.A) Dimitris Petrantonakis, dpetr@exodussa.com, Greece
3. Totalforsvarets Forskningsinstitut (FOI), Gustav Tolt, gustav.tolt@foi.se, Andreas Gustafsson, andreas.gustafsson@foi.se, Sweden
4. Crisisplan B.V. (CBV), Maureen Weller, weller@crisisplan.nl, Werner Overdijk, overdijk@crisisplan.nl, The Netherlands
5. Office National D'etudes Et De Recherches Aerospatiales (ONERA), Nicolas RIVIERE, nicolas.riviere@onera.fr, France
6. TEKNIKER (TEK), Roberto Calvo, rcalvo@tekniker.es, Spain
7. Fraunhofer Institute for High-Speed Dynamics, Ernst-Mach-Institut, EMI (EMI),
Dr. Kai Fischer, kai.fischer@emi.fraunhofer.de, Germany
8. CINSIDE AB (CINSIDE), Dan Axelsson, danaxe@cinside.eu, Sweden
9. ASI Applied Science International Europe, SRL (ASI), Steven Scoba, sscoba@appliedscienceint.com, Italy
10. DIGINEXT (DXT), Stéphane Collins, stephane.collins@diginext.fr, France
11. Laurea University of Applied Sciences (LUAS), Seija Tiainen, seija.tiainen@laurea.fi, Finland
12. Entente Pour la Foret Mediterranee (EPLFM), Dr Frédérique Giroud, f.giroud@valabre.com, France
13. Stiftelsen SINTEF (SINTEF), Giancarlo Marafioti, giancarlo.marafioti@sintef.no, Norway
14. Universiteit TWENTE / Department of Earth Systems Analysis, Faculty of Geo-Information Science and Earth Observation (ITC), Norman Kerle, kerle@itc.nl, The Netherlands
15. Schussler-Plan Ingenieurgesellschaft Mbh (ScPl), Dr. Andreas Bach, abach@schuessler-plan.de , Germany
16. Södertörns brandförsvarsförbund (SBFF), Henrik Nyman, henrik.nyman@sbff.se, Sweden
17. TELINT RTD Consultancy Services LTD (TELINT), Effie Makri, emakri@telint.eu, United Kingdom
18. BYTE COMPUTER ANONYMI VIOMICHANIKIEMPORIKI ETAIREIA (BYTE), Nikolaos Bezerianos, bezerianos@byte.gr, Greece
19. Mikrosystimata Mikrorois Gia Genetikous Elegkous Kai Moriaki Diagnostiki Epe (M2G), Spyridon Blionas, sbli@micro2gen.com, Greece
20. INSTITUUT FYSIEKE VEILIGHEID (IFV), Marco Boulogne, Marco.boulogne@vrh.nl, The Netherlands