Periodic Reporting for period 2 - ELASTIC (A Software Architecture for Extreme-ScaLe Big-Data AnalyticS in Fog CompuTIng ECosystems)
Período documentado: 2020-06-01 hasta 2022-05-31
Cities are facing the challenge of extracting valuable knowledge from the vast amount of data generated by IoT-enabled city infrastructures, connected vehicles, etc. This challenge is complicated with the additional operation requirements, such as the need to process information in real-time, consuming as less energy as possible and guaranteeing a secure environment with changing communication conditions.
With total funding of € 5.9 million, the ELASTIC project has implemented a novel SW architecture (SA) for the development and execution of advanced mobility applications for smart cities. One of the key innovations of the ELASTIC SA is its capability of distributing extreme-scale big-data analytics workflows across the compute continuum (from edge to cloud), guaranteeing the city operational requirements. To do so, ELASTIC has defined two strategic goals:
1. Provide SW developers with the right level of abstraction to facilitate the development of complex extreme-scale big-data analytics workflows
2. Provide the required increase in data throughput and data analytics accuracy, while fulfilling the operational requirements of cities
The ELASTIC SA has been validated in a real environment in the city of Florence, enabling the generation a common knowledge base between the city infrastructure, the tram vehicles and private cars and upon which three advanced mobility applications have been developed:
- Next Generation Autonomous Positioning for the localization of tram vehicles, and Advanced Driving Assistant System, for obstacle detection capabilities
- Predictive maintenance, for monitoring the tramway infrastructure with the objective of identifying changes in equipment behaviour before the equipment starts to fail
- Public and private transport interaction to alert users and/or operators to identify critical situations (e.g. pedestrians crossing the rail tracks while the traffic light is red or the tram is approaching) and optimize local traffic regulation strategies (e.g. formation of vehicle queues in the intersection area)
With total funding of € 5.9 million, the ELASTIC project has implemented a novel SW architecture (SA) for the development and execution of advanced mobility applications for smart cities. One of the key innovations of the ELASTIC SA is its capability of distributing extreme-scale big-data analytics workflows across the compute continuum (from edge to cloud), guaranteeing the city operational requirements. To do so, ELASTIC has defined two strategic goals:
1. Provide SW developers with the right level of abstraction to facilitate the development of complex extreme-scale big-data analytics workflows
2. Provide the required increase in data throughput and data analytics accuracy, while fulfilling the operational requirements of cities
The ELASTIC SA has been validated in a real environment in the city of Florence, enabling the generation a common knowledge base between the city infrastructure, the tram vehicles and private cars and upon which three advanced mobility applications have been developed:
- Next Generation Autonomous Positioning for the localization of tram vehicles, and Advanced Driving Assistant System, for obstacle detection capabilities
- Predictive maintenance, for monitoring the tramway infrastructure with the objective of identifying changes in equipment behaviour before the equipment starts to fail
- Public and private transport interaction to alert users and/or operators to identify critical situations (e.g. pedestrians crossing the rail tracks while the traffic light is red or the tram is approaching) and optimize local traffic regulation strategies (e.g. formation of vehicle queues in the intersection area)
During the second period, ELASTIC focused on: (1) the refinement and integration of the required SW components into the ELASTIC SA; (2) the development of three advanced mobility applications extreme-scale big-data analytics requirements; and (3) the deployment of the necessary sensing, communication and computing infrastructure for the use cases.
The final release of the ELASTIC SA was delivered, prioritising SW components owned by the members of the ELASTIC consortium or offered as open-source components with a large community behind them, with the objective of reducing the time-to-market and maximize exploitation opportunities. The ELASTIC SA consists of four layers:
1. A Distributed Data Analytics Platform layer, for data accessibility across the compute continuum, supporting data-in-motion and data-at-rest analytics
2. An Orchestration layer, responsible for deploying and distributing extreme-scale big-data analytics workflows across the compute continuum, while guaranteeing their non-functional requirements
3. A Non-functional Requirement Analysis layer, for the continuous monitoring of the compute continuum and analytics execution, informing the Orchestrator about the operational requirements across the dimensions of time, energy, communication quality and security
4. A Fog Computing Platform layer that implements the compute continuum, including the monitoring, communication and data routing capabilities needed by the other layers
The ELASTIC SA has been used to develop, deploy and execute three advanced mobility applications, featuring advanced big-data analytics methods such as deep neural networks for the detection of objects, analytics methods for tracking and GPS object localization, data fusion methods, semantic models for the detection of events of interest, aggregation and learning methods for pattern extraction. The ELASTIC use cases have been deployed into the tram stops of Batoni, Arcipressi and Resistenza, equipped with cameras and distributed GPU-enabled edge computing nodes, connection to a traffic light manager module to receive the real-time status of the traffic lights in the area, and a V2X module for the transmission of real-time alerts to connected cars
For the extraction of data from the tramway network, three tram vehicles have been equipped with sensors (cameras, radars, LiDARs, etc.) for object detection and autonomous position estimation, whereas a maintenance vehicle has been equipped with cameras, GPS and laser scanner to monitor the status of the tracks. The trams have been also equipped with GPU-enabled edge computing platforms, supporting Wi-Fi and 4G connectivity for the transmission of data to the edge/cloud infrastructure
At the cloud, a private cloud server have been setup, for the aggregation of data from the trams (via 4G), the maintenance vehicle (via WiFi) and the edge infrastructure (via fiber)
The project results have been widely disseminated, with participation in a total of 26 public events, including a keynote talk, booths at international and industrial exhibitions (e.g. Smart City Expo), presentations at conferences, 6 scientific publications, over 110 press mentions, etc. A final full-day hybrid event was organized in Florence (combining in-person attendance and live streaming options), presenting the final ELASTIC outcomes to the public and the local authorities
In terms of exploitation, 33 foregrounds have been identified, 10 of those with a TRL above/equal to 6. Joint plans were also identified, mainly to distribute the open-source software foreground and potentially commercialize some software components via the Nuvla marketplace
The final release of the ELASTIC SA was delivered, prioritising SW components owned by the members of the ELASTIC consortium or offered as open-source components with a large community behind them, with the objective of reducing the time-to-market and maximize exploitation opportunities. The ELASTIC SA consists of four layers:
1. A Distributed Data Analytics Platform layer, for data accessibility across the compute continuum, supporting data-in-motion and data-at-rest analytics
2. An Orchestration layer, responsible for deploying and distributing extreme-scale big-data analytics workflows across the compute continuum, while guaranteeing their non-functional requirements
3. A Non-functional Requirement Analysis layer, for the continuous monitoring of the compute continuum and analytics execution, informing the Orchestrator about the operational requirements across the dimensions of time, energy, communication quality and security
4. A Fog Computing Platform layer that implements the compute continuum, including the monitoring, communication and data routing capabilities needed by the other layers
The ELASTIC SA has been used to develop, deploy and execute three advanced mobility applications, featuring advanced big-data analytics methods such as deep neural networks for the detection of objects, analytics methods for tracking and GPS object localization, data fusion methods, semantic models for the detection of events of interest, aggregation and learning methods for pattern extraction. The ELASTIC use cases have been deployed into the tram stops of Batoni, Arcipressi and Resistenza, equipped with cameras and distributed GPU-enabled edge computing nodes, connection to a traffic light manager module to receive the real-time status of the traffic lights in the area, and a V2X module for the transmission of real-time alerts to connected cars
For the extraction of data from the tramway network, three tram vehicles have been equipped with sensors (cameras, radars, LiDARs, etc.) for object detection and autonomous position estimation, whereas a maintenance vehicle has been equipped with cameras, GPS and laser scanner to monitor the status of the tracks. The trams have been also equipped with GPU-enabled edge computing platforms, supporting Wi-Fi and 4G connectivity for the transmission of data to the edge/cloud infrastructure
At the cloud, a private cloud server have been setup, for the aggregation of data from the trams (via 4G), the maintenance vehicle (via WiFi) and the edge infrastructure (via fiber)
The project results have been widely disseminated, with participation in a total of 26 public events, including a keynote talk, booths at international and industrial exhibitions (e.g. Smart City Expo), presentations at conferences, 6 scientific publications, over 110 press mentions, etc. A final full-day hybrid event was organized in Florence (combining in-person attendance and live streaming options), presenting the final ELASTIC outcomes to the public and the local authorities
In terms of exploitation, 33 foregrounds have been identified, 10 of those with a TRL above/equal to 6. Joint plans were also identified, mainly to distribute the open-source software foreground and potentially commercialize some software components via the Nuvla marketplace
ELASTIC is promoting a new set of technologies capable of efficiently distributing complex extreme-scale big-data analytics workflows across the compute continuum. This distribution leverages information provided by analytics tools that constantly monitor the execution of workflows to guarantee the fulfilment of requirements.
The ELASTIC SA, validated in three smart mobility use cases in the city of Florence, has achieved:
- Integration and optimization of advanced data analytics methods into a complex workflow for both real-time and offline analytics, executed across the edge/cloud continuum and collecting extreme data from multiple sources from both the tramway network and the city infrastructure
- Up to 50% reduction in SW development costs, bringing down the development time for the smart city use case from 2 months to 2 weeks
- Up to 38% reduction of the analytics response time, through advanced scheduling for distributed execution, taking into account data dependencies, the quality of communication links and real-time requirements
At a societal level, the solutions provided in ELASTIC can potentially:
- Reduce the number of accidents and provide a safer urban environment, by promptly alerting connected vehicles on hazardous situations (e.g. road users crossing the tracks with red, etc.)
- Collect and analyse offline datasets on real traffic (e.g. traffic queues) and road-user behaviour (e.g. detected violations), that can lead to more efficient and urban planning
- Provide fundamental functionalities towards the autonomous tram operation (advanced autonomous localization and real-time hazard detection)
- Offer predictive analytics methods to facilitate maintenance activities, reducing their cost
The ELASTIC SA, validated in three smart mobility use cases in the city of Florence, has achieved:
- Integration and optimization of advanced data analytics methods into a complex workflow for both real-time and offline analytics, executed across the edge/cloud continuum and collecting extreme data from multiple sources from both the tramway network and the city infrastructure
- Up to 50% reduction in SW development costs, bringing down the development time for the smart city use case from 2 months to 2 weeks
- Up to 38% reduction of the analytics response time, through advanced scheduling for distributed execution, taking into account data dependencies, the quality of communication links and real-time requirements
At a societal level, the solutions provided in ELASTIC can potentially:
- Reduce the number of accidents and provide a safer urban environment, by promptly alerting connected vehicles on hazardous situations (e.g. road users crossing the tracks with red, etc.)
- Collect and analyse offline datasets on real traffic (e.g. traffic queues) and road-user behaviour (e.g. detected violations), that can lead to more efficient and urban planning
- Provide fundamental functionalities towards the autonomous tram operation (advanced autonomous localization and real-time hazard detection)
- Offer predictive analytics methods to facilitate maintenance activities, reducing their cost