Periodic Reporting for period 2 - PerceptiveSentinel (Perceptive Sentinel – BIG DATA knowledge extraction and re-creation platform)
Reporting period: 2019-04-01 to 2020-06-30
Free and open access to high temporal and high spatial resolution Copernicus’ satellite data is becoming a major game-changer in EO sector, delivering new opportunities as well as new challenges at the same time. How to ingest enormous amounts of data, how to unlock the hidden value of the data and how to deliver new value to end user community, were unanswered questions until recently.
Project results have been built upon an award-winning Sentinel Hub, world-first engine for archiving, processing and distribution of Sentinel data, delivering eo-learn, an open-source platform for prototyping, analysis and large scale machine learning-powered processing of EO data. Perceptive Sentinel – Big Data Knowledge Extraction and Re-creation Platform - has been built to challenge the current EO exploitation practices by delivering completely new, revolutionary EO ecosystem.
EO-learn provides a set of tools to design complex EO workflows as easy, fast and accessible as possible. It has grown into a remarkable piece of open-source software, ready to be put to use by anyone who is curious about EO data, to actually using them for data science and machine learning. Perceptive Sentinel tools have been used in several projects for various use-cases, such as land cover mapping, crop-type mapping, building footprint extraction, field delineation, and more. The open-source community has embraced eo-learn, ensuring it will evolve in the future, after successful completion of the project.
Project results have been built upon an award-winning Sentinel Hub, world-first engine for archiving, processing and distribution of Sentinel data, delivering eo-learn, an open-source platform for prototyping, analysis and large scale machine learning-powered processing of EO data. Perceptive Sentinel – Big Data Knowledge Extraction and Re-creation Platform - has been built to challenge the current EO exploitation practices by delivering completely new, revolutionary EO ecosystem.
EO-learn provides a set of tools to design complex EO workflows as easy, fast and accessible as possible. It has grown into a remarkable piece of open-source software, ready to be put to use by anyone who is curious about EO data, to actually using them for data science and machine learning. Perceptive Sentinel tools have been used in several projects for various use-cases, such as land cover mapping, crop-type mapping, building footprint extraction, field delineation, and more. The open-source community has embraced eo-learn, ensuring it will evolve in the future, after successful completion of the project.
In the first period of the project, eo-learn has grown into a remarkable piece of open-source software, ready to be put to use by anyone who is curious about EO data, to actually using them for data science and machine learning. We did a blog series on land use and land cover classification, using eo-learn, which have accumulated more than 28 thousand views so far. On GitHub , open-source repository, where eo-learn is available for download, the project has over 230 stars and 35 developers are subscribed to notifications about changes of eo-learn. More than 60 developers have created their own branch of eo-learn, which people do when they want to do local modifications. These numbers do not include people, who use eo-learn as a library installed like any other Python package. The eo-learn repository has more than 2.500 views in a bi-weekly period coming from more than 400 unique users.
During the second reporting period from April 2019 to June 2020 the focus of the project was the development of PerceptiveSentinel core libraries, which have been upgraded with the introduction of pre-processing algorithms incorporated into eo-learn library. The library illustrates high-level architecture and main components of PerceptiveSentinel. It is exposed as a rich variety of pre-processing and processing algorithms used for data pre-processing or creation of custom EO-processing chains such as data pre-processing, deep learning algorithms incorporated into processing library. These are integrated with EO-QMiner to support streaming machine learning, new data gateways and cases showing how to use eo-learn library with external services. Software components eo-learn library, QMiner, pre-processing library and Jupyter Notebook have been integrated with PerceptiveSentinel platform (eo-learn platform). EO-QMiner integration layer has been developed to support seamless communication with machine-learning engine. Its main tasks are to transform data into existing QMiner data formats for the ingestion by EO-QMiner and to provide labelled data. Workflow designer and workflow engine have been upgraded with the addition of new input modules to support new data gateways and new processing modules to support streaming machine learning. Platform provides learning and interpretation data; external services provide interpreted data.
During the second period of the project On PyPI, where eo-learn is available for download, the project has over 28.000 downloads. More than 175 developers have created their own branch of eo-learn, which people do when they want to do local modifications. These numbers do not include people, who use eo-learn as a library installed like any other Python package. The eo-learn repository has more than 400 downloads per week. The Sentinel Hub forum thread dedicated to our open-source libraries such as eo-learn has over 120 threads, which were viewed more than 40 thousand times. There are numerous Sentinel Hub users, Enterprise level subscribers, for whom we know they are using it due to eo-learn and they generate more than 80% of the volume of the processed requests, more than quarter of a billion requests per month. The Sentinel Hub blog has 100.000 views of blog posts related to eo-learn, e.g. Introducing eo-learn, Land Cover Classification with eo-learn, Land Cover Monitoring.
During the second reporting period from April 2019 to June 2020 the focus of the project was the development of PerceptiveSentinel core libraries, which have been upgraded with the introduction of pre-processing algorithms incorporated into eo-learn library. The library illustrates high-level architecture and main components of PerceptiveSentinel. It is exposed as a rich variety of pre-processing and processing algorithms used for data pre-processing or creation of custom EO-processing chains such as data pre-processing, deep learning algorithms incorporated into processing library. These are integrated with EO-QMiner to support streaming machine learning, new data gateways and cases showing how to use eo-learn library with external services. Software components eo-learn library, QMiner, pre-processing library and Jupyter Notebook have been integrated with PerceptiveSentinel platform (eo-learn platform). EO-QMiner integration layer has been developed to support seamless communication with machine-learning engine. Its main tasks are to transform data into existing QMiner data formats for the ingestion by EO-QMiner and to provide labelled data. Workflow designer and workflow engine have been upgraded with the addition of new input modules to support new data gateways and new processing modules to support streaming machine learning. Platform provides learning and interpretation data; external services provide interpreted data.
During the second period of the project On PyPI, where eo-learn is available for download, the project has over 28.000 downloads. More than 175 developers have created their own branch of eo-learn, which people do when they want to do local modifications. These numbers do not include people, who use eo-learn as a library installed like any other Python package. The eo-learn repository has more than 400 downloads per week. The Sentinel Hub forum thread dedicated to our open-source libraries such as eo-learn has over 120 threads, which were viewed more than 40 thousand times. There are numerous Sentinel Hub users, Enterprise level subscribers, for whom we know they are using it due to eo-learn and they generate more than 80% of the volume of the processed requests, more than quarter of a billion requests per month. The Sentinel Hub blog has 100.000 views of blog posts related to eo-learn, e.g. Introducing eo-learn, Land Cover Classification with eo-learn, Land Cover Monitoring.
Perceptive Sentinel library, eo-learn, makes extraction of valuable information from satellite imagery as easy as defining a sequence of operations to be performed on satellite imagery. It’s a Python library that acts as a bridge between Earth Observation/Remote Sensing and Python ecosystem for data science and machine learning. On one hand, it aims to make entry to the field of remote sensing for non-experts easier. On the other, brings the state-of-the-art tools for computer vision, machine learning, and deep learning existing in Python ecosystem to remote sensing experts. It makes it possible, for the first time, to design remote sensing data analysis processes as predictable and repeatable as data scientists have been used to with other datasets, abstracting the complexity of multi-temporal and multi-spectral nature of the satellite data. A plethora of use-cases and advanced ML techniques that accompany the eo-learn library make it possible for anyone to find a starting point for their challenges. The open-source nature of the project ensures that the community can improve it further, making it better and better.
We believe that the best measure for the impact is how much the tools, resulting from the project, are being used:
• more than 100.000 views for the blog posts related to eo-learn;
• over 28.000 downloads of the eo-learn library;
• more than 175 users branching eo-learn with the intention to develop it further;
• more than 7.000 external participants in knowledge exchange events;
• forum thread dedicated to eo-learn reaching 120 threads, being viewed more than 40.000 times.
We believe that the best measure for the impact is how much the tools, resulting from the project, are being used:
• more than 100.000 views for the blog posts related to eo-learn;
• over 28.000 downloads of the eo-learn library;
• more than 175 users branching eo-learn with the intention to develop it further;
• more than 7.000 external participants in knowledge exchange events;
• forum thread dedicated to eo-learn reaching 120 threads, being viewed more than 40.000 times.