Periodic Reporting for period 2 - CLASS (Edge and CLoud Computation: A Highly Distributed Software Architecture for Big Data AnalyticS)
Reporting period: 2019-07-01 to 2021-06-30
The CLASS project is working towards sustainable, efficient and safe mobility applications for smart cities, connected cars and future autonomous vehicles. In that regard, a key technological challenge is the capability to process in real-time large amounts of data coming from geographically distributed sources such as vehicles, city infrastructures, IoT devices, etc.
With a € 3.9 million funding, CLASS has developed a novel software architecture (SA) to efficiently design, deploy and execute advanced mobility applications, incorporating advanced big data analytics workflows and distributing computation across the compute continuum from edge to cloud in a transparent way, while providing real-time guarantees. To do so, the CLASS project has worked on two strategic goals to:
1. develop a set of tools and methods with the right level of abstraction to facilitate the development of complex and distributed big-data analytics workflows
2. increase the real-time capabilities of big-data analytics systems, while providing the throughput speed-up and analytics accuracy to cope with current increasing trends of data volumes, variety and velocity.
In that context, three smart mobility applications have been implemented over the CLASS SA and validated in the Modena Automotive Smart Area (MASA), a real urban laboratory in Modena, Italy, featuring a heavy sensor, computing and networking infrastructure. Moreover, three connected cars prototypes, with the necessary sensing and connectivity capabilities, have been employed. The CLASS advanced mobility applications are:
1. The generation of a common knowledge in real-time, based on the fusion of the data collected by the city and the cars.
2. The collision detection application, alerting drivers in real-time for potential collision hazards, based on predictions of the trajectories of the detected road users included in the common knowledge.
3. The air pollution estimation application, estimating vehicle-related pollutants levels based on the type and acceleration of detected vehicles stored in real-time in the common knowledge, with granularity down to a few minutes.
With a € 3.9 million funding, CLASS has developed a novel software architecture (SA) to efficiently design, deploy and execute advanced mobility applications, incorporating advanced big data analytics workflows and distributing computation across the compute continuum from edge to cloud in a transparent way, while providing real-time guarantees. To do so, the CLASS project has worked on two strategic goals to:
1. develop a set of tools and methods with the right level of abstraction to facilitate the development of complex and distributed big-data analytics workflows
2. increase the real-time capabilities of big-data analytics systems, while providing the throughput speed-up and analytics accuracy to cope with current increasing trends of data volumes, variety and velocity.
In that context, three smart mobility applications have been implemented over the CLASS SA and validated in the Modena Automotive Smart Area (MASA), a real urban laboratory in Modena, Italy, featuring a heavy sensor, computing and networking infrastructure. Moreover, three connected cars prototypes, with the necessary sensing and connectivity capabilities, have been employed. The CLASS advanced mobility applications are:
1. The generation of a common knowledge in real-time, based on the fusion of the data collected by the city and the cars.
2. The collision detection application, alerting drivers in real-time for potential collision hazards, based on predictions of the trajectories of the detected road users included in the common knowledge.
3. The air pollution estimation application, estimating vehicle-related pollutants levels based on the type and acceleration of detected vehicles stored in real-time in the common knowledge, with granularity down to a few minutes.
CLASS has performed two main activities in the second period: (1) the integration of the CLASS SA components and their deployment at the City of Modena; and (2) the optimization of the data analytics workflows for the CLASS use-cases, and their validation at the MASA.
The CLASS SA considers four layers:
-The orchestration layer based on the COMPSs framework, handling the scheduling and distribution of data analytics workflows over the compute continuum, with real-time guarantees.
-The data analytics platform, providing a unified development environment supporting task-based and map-reduce analytics engines.
-The edge analytics platform, exploiting the GPU-accelerated libraries for object detection and tracking over live video streams.
-The cloud computing platform, employing Rotterdam, a Container-as-a-Service for the deployment and lifecycle management of containerized applications.
The software components have been developed with well-defined interfaces, providing a solid interoperable, portable and scalable infrastructure that exploits the distributed and heterogeneous environments considered in CLASS. The key data-analytics methods developed in CLASS include:
–object detection over live video, based on a convolutional deep neural network
–object positioning and tracking, based on homography and Kalman filters, to compute the GPS location of detected objects and their dynamics (e.g. trajectory, speed and acceleration)
–object deduplication, to fuse data from multiple cameras
–trajectory prediction of detected objects
–collision detection based on the predicted trajectory of objects
–pollution emission estimation, to determine vehicle-related pollutant particles
The performance of the CLASS SA has been evaluated in a real-life use-case in the MASA, from an analytics and a computational perspective. Some key results include: i) alerting drivers for potential collisions with a 2-seconds margin, while processing simultaneously 4 live video streams, ii) the 45% reduction of the edge analytics response time by applying advanced scheduling at the edge, while offering some degree of performance predictability, and iii) a valid estimation of several vehicle-related pollutants levels, with at different time scales.
The project results have been widely disseminated throughout the lifetime of the project, with participation in more than 22 international events during the second period. The key results and a live streaming of the CLASS use-cases in the City of Modena have been presented in a final dissemination event. In terms of exploitation, CLASS has developed to 11 exploitable assets (8 with a TRL above/equal to 6), one patent, the commercialization of 4 assets under consideration, and further exploitation in other ongoing R&D projects.
The CLASS SA considers four layers:
-The orchestration layer based on the COMPSs framework, handling the scheduling and distribution of data analytics workflows over the compute continuum, with real-time guarantees.
-The data analytics platform, providing a unified development environment supporting task-based and map-reduce analytics engines.
-The edge analytics platform, exploiting the GPU-accelerated libraries for object detection and tracking over live video streams.
-The cloud computing platform, employing Rotterdam, a Container-as-a-Service for the deployment and lifecycle management of containerized applications.
The software components have been developed with well-defined interfaces, providing a solid interoperable, portable and scalable infrastructure that exploits the distributed and heterogeneous environments considered in CLASS. The key data-analytics methods developed in CLASS include:
–object detection over live video, based on a convolutional deep neural network
–object positioning and tracking, based on homography and Kalman filters, to compute the GPS location of detected objects and their dynamics (e.g. trajectory, speed and acceleration)
–object deduplication, to fuse data from multiple cameras
–trajectory prediction of detected objects
–collision detection based on the predicted trajectory of objects
–pollution emission estimation, to determine vehicle-related pollutant particles
The performance of the CLASS SA has been evaluated in a real-life use-case in the MASA, from an analytics and a computational perspective. Some key results include: i) alerting drivers for potential collisions with a 2-seconds margin, while processing simultaneously 4 live video streams, ii) the 45% reduction of the edge analytics response time by applying advanced scheduling at the edge, while offering some degree of performance predictability, and iii) a valid estimation of several vehicle-related pollutants levels, with at different time scales.
The project results have been widely disseminated throughout the lifetime of the project, with participation in more than 22 international events during the second period. The key results and a live streaming of the CLASS use-cases in the City of Modena have been presented in a final dissemination event. In terms of exploitation, CLASS has developed to 11 exploitable assets (8 with a TRL above/equal to 6), one patent, the commercialization of 4 assets under consideration, and further exploitation in other ongoing R&D projects.
Current big-data systems are designed for either fast and reactive responses (data-in-motion), or computationally intensive analysis of a vast amount of data (data-at-rest). Instead of considering these options separately, the vision of CLASS is to provide an environment where data-in-motion and data-at-rest analytics can be combined into a single workflow, that can be efficiently distributed across the compute continuum, from the data sources and collocated edge devices, to the data servers at the cloud.
The CLASS SA environment, validated in a smart city use case, has achieved:
-Integration and optimization of advanced data analytics methods into a single workflow for collision detection and air pollution estimation, using both task-based and map-reduce analytics engines
-Up to 50% reduction in SW development costs, bringing down the development time for the smart city use case from 2 months to 3 weeks
-Up to 40% reduction of the analytics response time, through advanced scheduling for distributed execution, taking into account data dependencies, the quality of communication links and real-time requirements
-Support for concurrency at the cloud, through the execution of data analytics methods as serverless functions, as well as scaling capabilities through a predictive SLA management component
At a societal level, the solutions provided in CLASS can potentially:
-Reduce the number of accidents and provide a safer urban environment, by warning drivers for potential collisions, with up to 2 second margin
-Enable the estimation of vehicle-related air pollution levels in very small time scales (down to few minutes) based on real-time traffic observations, a feature not possible by current long-term statistical models, which enables the study of the impact of real-life traffic behaviour on the air quality, eventually leading to the identification of greener driving habits
-Enhance traffic management by incorporating smart vehicles able to respond in real-time to specific situations. For instance, through an enriched simulation framework developed in CLASS, a potential reduction of up to 36% of the response of an ambulance traveling within the MASA can be achieved by the use of smart vehicles
Overall, the CLASS ecosystem offers a smart, safe and sustainable transportation solution with faster, flexible and scalable software development and deployment capabilities, which can also be applied to a wide range of application domains with critical real-time requirements, such as smart factories, smart healthcare, etc.
The CLASS SA environment, validated in a smart city use case, has achieved:
-Integration and optimization of advanced data analytics methods into a single workflow for collision detection and air pollution estimation, using both task-based and map-reduce analytics engines
-Up to 50% reduction in SW development costs, bringing down the development time for the smart city use case from 2 months to 3 weeks
-Up to 40% reduction of the analytics response time, through advanced scheduling for distributed execution, taking into account data dependencies, the quality of communication links and real-time requirements
-Support for concurrency at the cloud, through the execution of data analytics methods as serverless functions, as well as scaling capabilities through a predictive SLA management component
At a societal level, the solutions provided in CLASS can potentially:
-Reduce the number of accidents and provide a safer urban environment, by warning drivers for potential collisions, with up to 2 second margin
-Enable the estimation of vehicle-related air pollution levels in very small time scales (down to few minutes) based on real-time traffic observations, a feature not possible by current long-term statistical models, which enables the study of the impact of real-life traffic behaviour on the air quality, eventually leading to the identification of greener driving habits
-Enhance traffic management by incorporating smart vehicles able to respond in real-time to specific situations. For instance, through an enriched simulation framework developed in CLASS, a potential reduction of up to 36% of the response of an ambulance traveling within the MASA can be achieved by the use of smart vehicles
Overall, the CLASS ecosystem offers a smart, safe and sustainable transportation solution with faster, flexible and scalable software development and deployment capabilities, which can also be applied to a wide range of application domains with critical real-time requirements, such as smart factories, smart healthcare, etc.