Periodic Reporting for period 2 - beAWARE (Enhancing decision support and management services in extreme weather climate events)
Reporting period: 2018-07-01 to 2019-12-31
beAWARE (Enhancing decision support and management services in extreme weather climate events.), funded by the European Union’s H2020 research and innovation programme (GA-700475), explored the combination of innovative technologies to result in a holistic approach to the realization of crisis management framework that support all the phases in an emergency call sequence.
The main goal of the project was to rely on platforms, theories and methodologies that are already used for disaster management and add the elements that are necessary to make them working efficiently under the same objective.
The key result is an integrated beAWARE platform that encompasses high-end technologies and machine learning capabilities for obtaining information from text messages, social media and voice calls; a classification mechanism to process weather and other multimodal data and to generate early warnings and real-time alerts; deep learning techniques to detect crisis events in visual content; automatic drone routing and piloting for receiving valuable information from aerial imagery; case-based reasoning and decision support algorithms for crisis management; and automatic generation of multilingual reports to transform all the above into linguistic information to the authorities creating a usable system for performing in real case scenarios.
Its usability and interoperability have been stress-tested in the lifetime of the project through 3 major field testings and will undergo a permanent stress-testing (demo) environment after the official end of the project.
The main goal of the project was to rely on platforms, theories and methodologies that are already used for disaster management and add the elements that are necessary to make them working efficiently under the same objective.
The key result is an integrated beAWARE platform that encompasses high-end technologies and machine learning capabilities for obtaining information from text messages, social media and voice calls; a classification mechanism to process weather and other multimodal data and to generate early warnings and real-time alerts; deep learning techniques to detect crisis events in visual content; automatic drone routing and piloting for receiving valuable information from aerial imagery; case-based reasoning and decision support algorithms for crisis management; and automatic generation of multilingual reports to transform all the above into linguistic information to the authorities creating a usable system for performing in real case scenarios.
Its usability and interoperability have been stress-tested in the lifetime of the project through 3 major field testings and will undergo a permanent stress-testing (demo) environment after the official end of the project.
During the course of the project, we have developed the beAWARE Platform. The Platform is a combination of different components offering a variety of tools and functionalities, some of which represent promising innovations to the current SoA technologies. The exploitable parts of the Platform are the following:
-Frost Server: Is a standard-based sensor data store, used in beAWARE to store and retrieve heterogeneous time series data and the metadata of the sensors in a uniform way.
-Crisis classification API: Identification and classification of crisis events with early warning and real-time risk assessment features
-Analysis Module: Semantic collection and integration of data. The module is supported by an ontological framework and is consisting of three analysis components: a) Multimedia Analysis sub-component to analyse visual and audio content and to automatically understand and detect crisis events in it. b) Textual Analysis sub-component to analyse and extract from text information such as the type of the crisis, the targets involved, the geolocation of an event etc. c) Social media Analysis sub-component to continuously collect and analyse tweets from twitter and classify them as relevant or not
-Drones platform and Dashboard: Service to connect drones service-related providers with customers to configure, launch, automatic piloting and monitoring drones missions;
-Knowledgebase for crisis management: A comprehensive system to access, aggregate and visualise different information sources to improve situation awareness;
-beAWARE PSAP Map and Dashboard: It involves visualisation and interaction techniques for enhanced situational awareness.
All the developed components have been created employing a participatory design approach, in a SCRUM development/evaluation spiral process, from the 1st prototype to the final system. The prototypes were demonstrated in three project pilots involving in a total of more than 600 participants. Ultimately, the Platform met its original goal and fulfilled users’ expectations. Furthermore, all project outcomes have been communicated and disseminated according to the original plan, developed in the first phase of the project and regularly updated.
-Frost Server: Is a standard-based sensor data store, used in beAWARE to store and retrieve heterogeneous time series data and the metadata of the sensors in a uniform way.
-Crisis classification API: Identification and classification of crisis events with early warning and real-time risk assessment features
-Analysis Module: Semantic collection and integration of data. The module is supported by an ontological framework and is consisting of three analysis components: a) Multimedia Analysis sub-component to analyse visual and audio content and to automatically understand and detect crisis events in it. b) Textual Analysis sub-component to analyse and extract from text information such as the type of the crisis, the targets involved, the geolocation of an event etc. c) Social media Analysis sub-component to continuously collect and analyse tweets from twitter and classify them as relevant or not
-Drones platform and Dashboard: Service to connect drones service-related providers with customers to configure, launch, automatic piloting and monitoring drones missions;
-Knowledgebase for crisis management: A comprehensive system to access, aggregate and visualise different information sources to improve situation awareness;
-beAWARE PSAP Map and Dashboard: It involves visualisation and interaction techniques for enhanced situational awareness.
All the developed components have been created employing a participatory design approach, in a SCRUM development/evaluation spiral process, from the 1st prototype to the final system. The prototypes were demonstrated in three project pilots involving in a total of more than 600 participants. Ultimately, the Platform met its original goal and fulfilled users’ expectations. Furthermore, all project outcomes have been communicated and disseminated according to the original plan, developed in the first phase of the project and regularly updated.
The progress beyond the state of the art can be summarised in the following:
Obj1: A set of approximately 100 user requirements was formulated which corresponds to 3 relevant use cases
Obj2: a) An adaptable speech recognition component implemented with the ability to transcribe calls from existing call centre solutions b) A wide coverage multilingual text analysis component to detect from text a wide range of incidents and impacted objects, locations and states associated with incidents
Obj3: beAWARE can collect, harmonise and store information from the following sources: Meteorological, Hydrological, risk maps, multimedia, tweets.
Obj4: The visual analysis components extract high-level information by extracting low-level features and translate them into high-level concepts based on DL techniques. The extracted concepts lead to a real-time global understanding of the situation from the visual information (people impacted, traffic conditions etc.) Moreover, novel methods proposed for combining multi-modal information in early and late fusion schemes along with dimensionality reduction algorithms.
Obj5: The backbone of the reasoning mechanisms of beAWARE consists in the use of an ontology for mapping the input, estimate the severity level of an incident, cluster relevant incidents, link the heterogeneous information and reveal hidden relations. Moreover, a validation mechanism was included to control system traffic and to minimise the likelihood of malicious data impeding system effectiveness.
Obj6: A multilingual report generator for the generation of multilingual reports that derive from beAWARE's ontological representations. Two types of multilingual reports can be issued by the system, situational and summary reports covering the occurrence of incidents in chronological order. Entities and concepts beyond those modelled by the ontology can be verbalised, thus facilitating the portability of the developed components to new emergency scenarios.
Obj7: A PSAP that involves visualisation and interaction techniques for enhanced situational awareness, including situational assessment, GIS analysis and mapping
Obj8: Three large scale pilots successfully carried out, involving in a total of more than 600 participants.
In terms of societal engagement, in addition to the participatory design, testing and evaluation stages, which resulted in the involvement of more than 600 individuals (FRs, and the general public), the project reaches several communities through participation to conferences, workshops, showcases and seminars. A focused Network of Interested was established to support the direct liaison with relevant stakeholders that reached 149 members. A detailed list of dissemination and societal engagement activities can be found in the D8.3(M36) available online at https://beaware-project.eu/results/public-deliverables
Despite the end of the project in December 2019, the consortium is now moving in several directions aiming at promoting the activities carried out during the project, further exploit the project outcomes and maximise future impact.
Obj1: A set of approximately 100 user requirements was formulated which corresponds to 3 relevant use cases
Obj2: a) An adaptable speech recognition component implemented with the ability to transcribe calls from existing call centre solutions b) A wide coverage multilingual text analysis component to detect from text a wide range of incidents and impacted objects, locations and states associated with incidents
Obj3: beAWARE can collect, harmonise and store information from the following sources: Meteorological, Hydrological, risk maps, multimedia, tweets.
Obj4: The visual analysis components extract high-level information by extracting low-level features and translate them into high-level concepts based on DL techniques. The extracted concepts lead to a real-time global understanding of the situation from the visual information (people impacted, traffic conditions etc.) Moreover, novel methods proposed for combining multi-modal information in early and late fusion schemes along with dimensionality reduction algorithms.
Obj5: The backbone of the reasoning mechanisms of beAWARE consists in the use of an ontology for mapping the input, estimate the severity level of an incident, cluster relevant incidents, link the heterogeneous information and reveal hidden relations. Moreover, a validation mechanism was included to control system traffic and to minimise the likelihood of malicious data impeding system effectiveness.
Obj6: A multilingual report generator for the generation of multilingual reports that derive from beAWARE's ontological representations. Two types of multilingual reports can be issued by the system, situational and summary reports covering the occurrence of incidents in chronological order. Entities and concepts beyond those modelled by the ontology can be verbalised, thus facilitating the portability of the developed components to new emergency scenarios.
Obj7: A PSAP that involves visualisation and interaction techniques for enhanced situational awareness, including situational assessment, GIS analysis and mapping
Obj8: Three large scale pilots successfully carried out, involving in a total of more than 600 participants.
In terms of societal engagement, in addition to the participatory design, testing and evaluation stages, which resulted in the involvement of more than 600 individuals (FRs, and the general public), the project reaches several communities through participation to conferences, workshops, showcases and seminars. A focused Network of Interested was established to support the direct liaison with relevant stakeholders that reached 149 members. A detailed list of dissemination and societal engagement activities can be found in the D8.3(M36) available online at https://beaware-project.eu/results/public-deliverables
Despite the end of the project in December 2019, the consortium is now moving in several directions aiming at promoting the activities carried out during the project, further exploit the project outcomes and maximise future impact.