Periodic Reporting for period 2 - RapidAI4EO (RapidAI4EO - Advancing the State-of-the-Art for Rapid and Continuous Land Monitoring)
Reporting period: 2022-01-01 to 2023-03-31
Between January 2021 and March 2023 Planet Labs, VITO Remote Sensing, Vision Impulse, IIASA and Serco worked together in the H2020 project RapidAI4EO with the ambition to contribute enabling new and better ways of measuring and understanding the human footprint on our planet. The project idea was built on the convergence of two exciting trends in remote sensing and artificial intelligence: the explosion in satellite sensing constellations, both government and commercial, and incredible advances in Artificial Intelligence. The overarching goal of the project was to advance the state-of-the-art in deriving time-critical and location-specific insights into dynamic land surface processes by leveraging these trends, in particular, to enable faster and finer-grained classification and change detection of land use and land cover in Europe. To this aim we have created and released the RapidAI4EO corpus which is the largest and most comprehensive labelled, multi-source and multi-scale, spatiotemporal training dataset. This open dataset is expected to stimulate further development of new spatiotemporal land monitoring applications. We have implemented and benchmarked a number of novel Machine Learning (ML) architectures based on Deep Learning (DL) for patch-based change detection and pixel-based land cover mapping that can realise the full potential of new combined data sources. The project contributes to improving our understanding of land cover and has demonstrated a large scale, end-to-end process to continuously monitor and update LULC products.
We have created the RapidAI4EO corpus which is the most complete and dense, labelled, spatiotemporal, analysis-ready training dataset for modelling the dynamics of land covers (LC). It combines the higher spectral resolution of Sentinel-2 (12 bands) with the higher spatial (3m) and temporal resolution (5-daily observations) of cloud-free, harmonised Planet Fusion imagery at 500,000 patch locations. Locations were sampled to account for CLC classes and spatial distribution, as well as country representation, across the entire territory of the EU. The corpus is available to the community on Radiant Earth’s Source Cooperative (https://rapidai4eo.source.coop/) a key platform for ML datasets in the EO sector ensuring long-term access to this unique dataset. The corpus has the potential to improve various Copernicus Land Monitoring Service products, mainly because of its enhanced spatial and temporal resolution, and can be augmented with new labels, making it a versatile resource which can be used for other applications especially those which benefit from high cadence timeseries of imagery.
Furthermore, we have built and experimented with ML architectures based on the latest developments in Artificial Intelligence (DL) for patch-based change detection and pixel-based LC mapping that can realise the full potential of the combined data sources and temporal cadence of observations contained in the corpus. We have demonstrated our ability to detect change on a quarterly and land cover on a monthly basis. The introduction of the temporal dimension in both the change detection and LC classification was improving model performance. We experimented with both supervised and unsupervised methods. The developed methods were able to highlight LC changes at patch level, especially when combined with other techniques (e.g. Computer Vision methods, pixel-based filtering) and domain knowledge. They hold potential for integration in operational workflows as these support faster updates of existing LULC products but further work is required to facilitate wider adoption of these methods.
We presented the project and its results at multiple key conferences in the EO and ML domain (BiDS, IGARSS, ISPRS, ESA Living Planet) and released additional datasets to the community on open access platforms (reference dataset, LC maps). Furthermore, we released two demonstrations: (1) mimicking the human annotator workflow and (2) surfacing patch-level LULC maps and change detection heatmaps that resulted from the supervised ML work.
Furthermore, we have built and experimented with ML architectures based on the latest developments in Artificial Intelligence (DL) for patch-based change detection and pixel-based LC mapping that can realise the full potential of the combined data sources and temporal cadence of observations contained in the corpus. We have demonstrated our ability to detect change on a quarterly and land cover on a monthly basis. The introduction of the temporal dimension in both the change detection and LC classification was improving model performance. We experimented with both supervised and unsupervised methods. The developed methods were able to highlight LC changes at patch level, especially when combined with other techniques (e.g. Computer Vision methods, pixel-based filtering) and domain knowledge. They hold potential for integration in operational workflows as these support faster updates of existing LULC products but further work is required to facilitate wider adoption of these methods.
We presented the project and its results at multiple key conferences in the EO and ML domain (BiDS, IGARSS, ISPRS, ESA Living Planet) and released additional datasets to the community on open access platforms (reference dataset, LC maps). Furthermore, we released two demonstrations: (1) mimicking the human annotator workflow and (2) surfacing patch-level LULC maps and change detection heatmaps that resulted from the supervised ML work.
The RapidAI4EO dataset is revolutionary in both its spatiotemporal dimension and quality (multisource, harmonised, cloud-free, analysis ready). Before its advent, there were no available annotated large scale data sets that provided the high cadence, multi year time series necessary to drive the development of new AI architectures that can fully exploit the new data streams provided by multiple constellations.
The current CLC product has a Minimum Mapping Unit of 25 ha and 100m resolution. We have developed LC mapping and change detection solutions that can improve the spatial resolution of the product by 10-20x. The temporal updates of the current product are every 6 years and our solution can efficiently drive quarterly updates (quarterly heat maps of change, up to monthly land cover maps).
RapidAI4EO enables more accurate measurements from space in support of several of the SDGs thanks to the much higher temporal cadence and spatial resolution. It can provide change detection maps for the entire European continent which has an enormous potential for various sectors, enabling continuous environmental monitoring, monitoring of urban expansion, early alerts for deforestation, ploughing of protected permanent grasslands, and other abrupt or gradual environmental changes. Having an automated mapping approach to map different types of urban tissue or forest types, can be a game changer and help countries to automatically update their cadaster or better determine and monitor the amount of woody biomass and assess their carbon stock. Delivering continuous observation and mapping capabilities has an enormous potential for further scientific discoveries and to understand, anticipate and address the potential consequences of human activities on the planet and its climate. There is already evidence that high cadence, high resolution EO measurements have led to the discovery of previously unknown phenomenology.
The current CLC product has a Minimum Mapping Unit of 25 ha and 100m resolution. We have developed LC mapping and change detection solutions that can improve the spatial resolution of the product by 10-20x. The temporal updates of the current product are every 6 years and our solution can efficiently drive quarterly updates (quarterly heat maps of change, up to monthly land cover maps).
RapidAI4EO enables more accurate measurements from space in support of several of the SDGs thanks to the much higher temporal cadence and spatial resolution. It can provide change detection maps for the entire European continent which has an enormous potential for various sectors, enabling continuous environmental monitoring, monitoring of urban expansion, early alerts for deforestation, ploughing of protected permanent grasslands, and other abrupt or gradual environmental changes. Having an automated mapping approach to map different types of urban tissue or forest types, can be a game changer and help countries to automatically update their cadaster or better determine and monitor the amount of woody biomass and assess their carbon stock. Delivering continuous observation and mapping capabilities has an enormous potential for further scientific discoveries and to understand, anticipate and address the potential consequences of human activities on the planet and its climate. There is already evidence that high cadence, high resolution EO measurements have led to the discovery of previously unknown phenomenology.