Periodic Reporting for period 2 - datACRON (Big Data Analytics for Time Critical Mobility Forecasting)
Berichtszeitraum: 2017-07-01 bis 2018-12-31
datAcron is a research and innovation collaborative project targeting at introducing novel methods to detect threats and abnormal activity of very large numbers of moving entities in large geographic areas.
Towards this target, the project aims to advance the management and integrated exploitation of voluminous and heterogeneous data-at-rest (archival data) and data-in-motion (streaming data) sources, so as to significantly advance the capacities of systems to promote safety and effectiveness of critical operations for large numbers of moving entities in large geographical areas.
Technological developments in datAcron are validated and evaluated in user-defined challenges that aim at increasing the safety, efficiency and economy of operations concerning moving entities in the air-traffic management (ATM) and maritime domains.
Specific objectives include:
- Distributed management and querying of integrated spatiotemporal RDF data-at-rest and data-in-motion in integrated manners: In-situ data processing and link discovery for data integration are critical technologies to those targets.
- Detection and prediction of trajectories of moving entities, aiming at efficient large-scale mobility data analytics.
- Recognition and Forecasting of Complex Events under uncertainty in noisy settings, aiming at optimization of complex events patterns’ structure and parameters and processing very large number of events/second with complex event definitions.
- Development of a general visual analytics infrastructure supporting all steps of analysis.
-Validation and Evaluation of Results in Critical Domains: Air Traffic Management and Maritime use cases.
The datAcron consortium combines expertise across the big data value chain: data management and semantic data integration, data, trajectory, events and visual analytics, data providers with domain-specific expertise targeting to big-data solutions.
Technical and scientific advances achieved are expected to have increased impact for safety and security regarding maritime activities and predictability in the aviation sector (which is crucial in the future context of trajectory-based operations), and of course to transportation in general, and logistics, to mention some of the most direct sectors of human activities. In addition, novelties and innovations are expected to be highly influential in the big data scientific community, regarding management and exploitation of mobility data, and going beyond.
Towards this target, the project aims to advance the management and integrated exploitation of voluminous and heterogeneous data-at-rest (archival data) and data-in-motion (streaming data) sources, so as to significantly advance the capacities of systems to promote safety and effectiveness of critical operations for large numbers of moving entities in large geographical areas.
Technological developments in datAcron are validated and evaluated in user-defined challenges that aim at increasing the safety, efficiency and economy of operations concerning moving entities in the air-traffic management (ATM) and maritime domains.
Specific objectives include:
- Distributed management and querying of integrated spatiotemporal RDF data-at-rest and data-in-motion in integrated manners: In-situ data processing and link discovery for data integration are critical technologies to those targets.
- Detection and prediction of trajectories of moving entities, aiming at efficient large-scale mobility data analytics.
- Recognition and Forecasting of Complex Events under uncertainty in noisy settings, aiming at optimization of complex events patterns’ structure and parameters and processing very large number of events/second with complex event definitions.
- Development of a general visual analytics infrastructure supporting all steps of analysis.
-Validation and Evaluation of Results in Critical Domains: Air Traffic Management and Maritime use cases.
The datAcron consortium combines expertise across the big data value chain: data management and semantic data integration, data, trajectory, events and visual analytics, data providers with domain-specific expertise targeting to big-data solutions.
Technical and scientific advances achieved are expected to have increased impact for safety and security regarding maritime activities and predictability in the aviation sector (which is crucial in the future context of trajectory-based operations), and of course to transportation in general, and logistics, to mention some of the most direct sectors of human activities. In addition, novelties and innovations are expected to be highly influential in the big data scientific community, regarding management and exploitation of mobility data, and going beyond.
datAcron has achieved progress according to the work plan by delivering methods for:
1. Scalable integration and management of data from disparate and heterogeneous data sources
• Scalable, fault-tolerant cross-streaming in-situ data integration, collection, processing
• Data annotation, integration, automatic link discovery
• Distributed management and querying of integrated spatio-temporal data
2. Trajectories detection and forecasting & Analytics for Trajectories
• Cross-streaming, real-time detection of moving entities’ trajectories
• Short- and long-term real-time forecasting of trajectories
• Data analytics over moving entities’ trajectories
3. Complex Event Recognition & Forecasting
• Real-time event recognition and forecasting algorithms that take full advantage of the data provided
• Methods for adapting event patterns in dynamic settings
• Resilient real-time event recognition and forecasting algorithms addressing lack of veracity of data
4. Interactive Analytics
• VA methods for data exploration and assessment of data quality considering data-in-motion and data-at-rest from multiple sources
• VA methods for interactive pattern extraction from data-in-motion and data-at-rest coming from multiple sources
• VA methods for user-guided model building and validation
• VA methods for building situation overview and situation monitoring
5. A coherent big data architecture and Integrated datAcron prototype
6. Validation and evaluation of the datAcron system and individual components on the surveillance of moving entities in the ATM and marine domains
Project partners achieved to have a notable record of dissemination activities.
1. Scalable integration and management of data from disparate and heterogeneous data sources
• Scalable, fault-tolerant cross-streaming in-situ data integration, collection, processing
• Data annotation, integration, automatic link discovery
• Distributed management and querying of integrated spatio-temporal data
2. Trajectories detection and forecasting & Analytics for Trajectories
• Cross-streaming, real-time detection of moving entities’ trajectories
• Short- and long-term real-time forecasting of trajectories
• Data analytics over moving entities’ trajectories
3. Complex Event Recognition & Forecasting
• Real-time event recognition and forecasting algorithms that take full advantage of the data provided
• Methods for adapting event patterns in dynamic settings
• Resilient real-time event recognition and forecasting algorithms addressing lack of veracity of data
4. Interactive Analytics
• VA methods for data exploration and assessment of data quality considering data-in-motion and data-at-rest from multiple sources
• VA methods for interactive pattern extraction from data-in-motion and data-at-rest coming from multiple sources
• VA methods for user-guided model building and validation
• VA methods for building situation overview and situation monitoring
5. A coherent big data architecture and Integrated datAcron prototype
6. Validation and evaluation of the datAcron system and individual components on the surveillance of moving entities in the ATM and marine domains
Project partners achieved to have a notable record of dissemination activities.
datAcron achievements contribute to the following impact objectives:
1. Track publicly and quantitatively progress in the performance and optimization of very large-scale data analytics technologies in a European ecosystem consisting of hundreds of companies.
This is achieved via advances in data management (WP1), integrated data processing for real-time analytics for detecting and forecasting trajectories (WP2) and important events (WP3), and advanced visualization and user experience (WP4).
Furthermore, datAcron involves European companies that have the capacity and aim to increase their business potential in their domains, together with partners that have important achievements in research areas/topics of focus.
2. Advanced real-time and predictive data analytics technologies thoroughly validated by means of rigorous experiments testing their scalability, accuracy, feasibility and ready to be turned over to thousands of innovators and large-scale system developers.
This is achieved via the delivery of novel real-time and predictive data analytics technologies (WPs 2 & 3), as well as for interactive visual analytics (WP4), in conjunction to novel scalable methods for the management of data from disparate sources (WP1). These methods are integrated in the datAcron prototype (WP1) and are evaluated / validated in accordance to the requirements specified in the use case scenarios (WP5 & WP6).
Data sources per use case and the methodologies for the evaluation/validation of novel methods in specific scenarios have been elaborated and specified in detail (WP5 & WP6).
3. Demonstrated ability of developed technologies to keep abreast of growth in data volumes and variety by validation experiments.
datAcron evaluated and validated all technologies developed in various scenarios of data growth, variety, velocity, veracity – quality (WPs 1,2,3 & 4). Also according to user-defined validation/evaluation cases.
4. Demonstration of the technological and value-generation potential of the European Open Data documenting improvements in the market position and job creations of hundreds of European data intensive companies.
5. Four innovation activities have been planned, one of which is already under development.
This is achieved by the datAcron policy to open datasets with respect to the IPR and legal issues related to the data available to the project, while keeping all publications online, also via widely accessed open access infrastructure.
1. Track publicly and quantitatively progress in the performance and optimization of very large-scale data analytics technologies in a European ecosystem consisting of hundreds of companies.
This is achieved via advances in data management (WP1), integrated data processing for real-time analytics for detecting and forecasting trajectories (WP2) and important events (WP3), and advanced visualization and user experience (WP4).
Furthermore, datAcron involves European companies that have the capacity and aim to increase their business potential in their domains, together with partners that have important achievements in research areas/topics of focus.
2. Advanced real-time and predictive data analytics technologies thoroughly validated by means of rigorous experiments testing their scalability, accuracy, feasibility and ready to be turned over to thousands of innovators and large-scale system developers.
This is achieved via the delivery of novel real-time and predictive data analytics technologies (WPs 2 & 3), as well as for interactive visual analytics (WP4), in conjunction to novel scalable methods for the management of data from disparate sources (WP1). These methods are integrated in the datAcron prototype (WP1) and are evaluated / validated in accordance to the requirements specified in the use case scenarios (WP5 & WP6).
Data sources per use case and the methodologies for the evaluation/validation of novel methods in specific scenarios have been elaborated and specified in detail (WP5 & WP6).
3. Demonstrated ability of developed technologies to keep abreast of growth in data volumes and variety by validation experiments.
datAcron evaluated and validated all technologies developed in various scenarios of data growth, variety, velocity, veracity – quality (WPs 1,2,3 & 4). Also according to user-defined validation/evaluation cases.
4. Demonstration of the technological and value-generation potential of the European Open Data documenting improvements in the market position and job creations of hundreds of European data intensive companies.
5. Four innovation activities have been planned, one of which is already under development.
This is achieved by the datAcron policy to open datasets with respect to the IPR and legal issues related to the data available to the project, while keeping all publications online, also via widely accessed open access infrastructure.