Periodic Reporting for period 2 - SafeClouds.eu (Data-driven research addressing aviation safety intelligence)
Periodo di rendicontazione: 2018-04-01 al 2019-09-30
The SafeClouds consortium is comprised of a complete aviation stakeholders team, coordinated by Innaxis with the inclusion of five airlines (Air Europa, Iberia, Vueling, Norwegian, and Pegasus), IT infrastructure experts (Fraunhofer ITWM and Tadorea), ANSPs (LFV and ENAIRE-CRIDA), three universities (TU Münich, TU Delft, and Linköping University), Eurocontrol and authorities (EASA and AESA).
Aviation growth is challenging airport and airspace operations and introducing greater safety risks due to the increase in volume and complexity of operations. SafeClouds explores how artificial intelligence can improve operational safety by providing predictive analytics on a variety of scenarios. This allows better design of operations and raises awareness of hidden threats; this can, in turn, lead to new operational procedures that increase airports and airspace capacity while improving safety and accommodating traffic growth.
In order to do this, SafeClouds has developed a complete data cycle approach, which has involved improving data management skills, including data decoding, formatting and cleaning. The team has developed an ad-hoc AI data platform, DataBeacon, a complete AI development cloud environment capable of scaling up as computational requirements grow, which allows data analysts to work incredibly more efficiently than before.
DataBeacon provides analysts and data owners with Secure Data Frames (SDF) that accommodate data analysis needs while accounting for privacy and confidentiality. This has allowed for the unprecedented use of large amounts of flights, usually in the order of hundreds of thousands of flights. The SDF approach offers a great advantage to data owners who normally would only release limited samples.
The unparalleled diversity of stakeholders, under a single AI effort, has allowed SafeClouds to be the first project to merge different data sources securely, supporting four safety scenarios that are not addressable without collaboration across aviation actors. The main objective of SafeClouds is information-driven analysis of hazard identification in aviation.
Aviation growth is challenging airport and airspace operations and introducing greater safety risks due to the increase in volume and complexity of operations. SafeClouds explores how artificial intelligence can improve operational safety by providing predictive analytics on a variety of scenarios. This allows better design of operations and raises awareness of hidden threats; this can, in turn, lead to new operational procedures that increase airports and airspace capacity while improving safety and accommodating traffic growth.
In order to do this, SafeClouds has developed a complete data cycle approach, which has involved improving data management skills, including data decoding, formatting and cleaning. The team has developed an ad-hoc AI data platform, DataBeacon, a complete AI development cloud environment capable of scaling up as computational requirements grow, which allows data analysts to work incredibly more efficiently than before.
DataBeacon provides analysts and data owners with Secure Data Frames (SDF) that accommodate data analysis needs while accounting for privacy and confidentiality. This has allowed for the unprecedented use of large amounts of flights, usually in the order of hundreds of thousands of flights. The SDF approach offers a great advantage to data owners who normally would only release limited samples.
The unparalleled diversity of stakeholders, under a single AI effort, has allowed SafeClouds to be the first project to merge different data sources securely, supporting four safety scenarios that are not addressable without collaboration across aviation actors. The main objective of SafeClouds is information-driven analysis of hazard identification in aviation.
SafeClouds have contributed to aviation safety across five different threads:
- Precursor analysis: Existing data analytic methods facilitate historical analysis in monitoring relevant safety events (e.g. unstable approaches, ground proximity or airprox events). SafeClouds goes further by proposing a formal methodology to analyse precursors of those events. The precursor analysis, performed through machine learning techniques, has been matured to TRL5 for a number of safety-relevant scenarios, like unstable approaches or runway safety. The process has been automated, from data fusion and labelling to algorithm training and reporting, through a visual interface.
- Predictive analytics: Powerful predictive analytics has complemented historical analysis to help forecast the likelihood of a particular event in certain conditions. Current methodologies do not offer any predictive capability. SafeClouds has developed and presented a concept for understanding traffic evolution at different time horizons and the related safety implications. For instance, SafeClouds has proposed a concept to predict the likelihood of an unstable approach 30 seconds before the event, potentially allowing the design of new cabin crew procedures, or, with cooperation from ATCs, a concept to predict when an unstable approach is likely to happen. Predictive analytics can complement the current reporting systems.
- Data fusion for reporting: Data fusion is very limited in aviation. For instance, airline SMS departments can analyse their safety events from their perspective, using their sources of data. SafeClouds has opened the door to including meteorological reports or surrounding traffic data, a major breakthrough for the analytics of safety events. Similarly, runway safety, normally only analysed with airport data, can be improved through data fusion with airline data.
- Automatic safety monitoring: SafeClouds has presented DataBeacon, a data platform that automates the data pipeline including data ingestion, cleaning, de-identification, fusion, labelling for specific events, training, results and visualisation. This concept is much more advanced than existing data processes which require many manual steps and disconnected tools. This automatic safety monitoring paradigm is built-in in DataBeacon and contributes to future reporting systems.
- Anomaly detection: SafeClouds has made some very important steps in using neural network architectures for finding anomalous flights using FDM data. This is a novel approach with very few mentions in literature. SafeClouds has proved the value of deep learning by demonstrating the detection of rare safety events, such as Unstable Approaches.
- Precursor analysis: Existing data analytic methods facilitate historical analysis in monitoring relevant safety events (e.g. unstable approaches, ground proximity or airprox events). SafeClouds goes further by proposing a formal methodology to analyse precursors of those events. The precursor analysis, performed through machine learning techniques, has been matured to TRL5 for a number of safety-relevant scenarios, like unstable approaches or runway safety. The process has been automated, from data fusion and labelling to algorithm training and reporting, through a visual interface.
- Predictive analytics: Powerful predictive analytics has complemented historical analysis to help forecast the likelihood of a particular event in certain conditions. Current methodologies do not offer any predictive capability. SafeClouds has developed and presented a concept for understanding traffic evolution at different time horizons and the related safety implications. For instance, SafeClouds has proposed a concept to predict the likelihood of an unstable approach 30 seconds before the event, potentially allowing the design of new cabin crew procedures, or, with cooperation from ATCs, a concept to predict when an unstable approach is likely to happen. Predictive analytics can complement the current reporting systems.
- Data fusion for reporting: Data fusion is very limited in aviation. For instance, airline SMS departments can analyse their safety events from their perspective, using their sources of data. SafeClouds has opened the door to including meteorological reports or surrounding traffic data, a major breakthrough for the analytics of safety events. Similarly, runway safety, normally only analysed with airport data, can be improved through data fusion with airline data.
- Automatic safety monitoring: SafeClouds has presented DataBeacon, a data platform that automates the data pipeline including data ingestion, cleaning, de-identification, fusion, labelling for specific events, training, results and visualisation. This concept is much more advanced than existing data processes which require many manual steps and disconnected tools. This automatic safety monitoring paradigm is built-in in DataBeacon and contributes to future reporting systems.
- Anomaly detection: SafeClouds has made some very important steps in using neural network architectures for finding anomalous flights using FDM data. This is a novel approach with very few mentions in literature. SafeClouds has proved the value of deep learning by demonstrating the detection of rare safety events, such as Unstable Approaches.
SafeClouds progressed the state-of-the-art in different areas:
- Data availability: Several airlines, airports and ANPSs have shared datasets. Before SafeClouds, a repository that brought together data from different aviation sources did not exist; no other project has brought together such a variety of stakeholders under a single data infrastructure. This has meant overcoming technical and governance challenges and improving past data sharing solutions.
- Data Infrastructure - The concept of an AI data platform has been defined and matured and the analytic tasks have been run thanks to the computational and security features of DataBeacon. As a complete collaborative environment, DataBeacon has combined the functionalities from data ingestion and cleaning with the analytics and visualization of data.
- Data Analytics - Previous AI/ML research was limited as neither the variety nor the volume of the datasets were very significant. SafeClouds has pushed the aviation industry to adopt data analytic methodologies through strong communication with stakeholders. SafeClouds has developed multiple predictive algorithms that bring relevant advantages to concrete aviation related scenarios. By using machine learning and deep learning algorithms to detect the main precursors of critical events and provide accurate predictions, SafeClouds has innovated in the field of automatic data monitoring and analytics.
- Data Protection - Data was protected by deleting certain confidential fields rather than by encrypting them. No de-identification mechanisms allowed fusion after de-identification. Data owners often provided temporary access to limited datasets so that research teams could explore potential uses of data. However, due to confidentiality concerns, the use of the data was very restricted. SafeClouds has built a robust method for stakeholders to share de-identified data and guarantee privacy and ownership without compromising the use of the data.
- Data Visualisation - SafeClouds has defined a methodology for collaboration in data visualisation, integrating a visualisation framework within DataBeacon for easy consumption of ML results. This approach offers a consistent way of presenting predictive analytics results not possible with other visualisation tools and frameworks.
- Data availability: Several airlines, airports and ANPSs have shared datasets. Before SafeClouds, a repository that brought together data from different aviation sources did not exist; no other project has brought together such a variety of stakeholders under a single data infrastructure. This has meant overcoming technical and governance challenges and improving past data sharing solutions.
- Data Infrastructure - The concept of an AI data platform has been defined and matured and the analytic tasks have been run thanks to the computational and security features of DataBeacon. As a complete collaborative environment, DataBeacon has combined the functionalities from data ingestion and cleaning with the analytics and visualization of data.
- Data Analytics - Previous AI/ML research was limited as neither the variety nor the volume of the datasets were very significant. SafeClouds has pushed the aviation industry to adopt data analytic methodologies through strong communication with stakeholders. SafeClouds has developed multiple predictive algorithms that bring relevant advantages to concrete aviation related scenarios. By using machine learning and deep learning algorithms to detect the main precursors of critical events and provide accurate predictions, SafeClouds has innovated in the field of automatic data monitoring and analytics.
- Data Protection - Data was protected by deleting certain confidential fields rather than by encrypting them. No de-identification mechanisms allowed fusion after de-identification. Data owners often provided temporary access to limited datasets so that research teams could explore potential uses of data. However, due to confidentiality concerns, the use of the data was very restricted. SafeClouds has built a robust method for stakeholders to share de-identified data and guarantee privacy and ownership without compromising the use of the data.
- Data Visualisation - SafeClouds has defined a methodology for collaboration in data visualisation, integrating a visualisation framework within DataBeacon for easy consumption of ML results. This approach offers a consistent way of presenting predictive analytics results not possible with other visualisation tools and frameworks.