Periodic Reporting for period 2 - EXPLORE (Innovative Scientific Data Exploration and Exploitation Applications for Space Sciences)
Okres sprawozdawczy: 2022-05-01 do 2023-12-31
The EXPLORE project has aimed to facilitate efficient, user-friendly exploitation of data from astrophysics and planetary space missions and supporting ground-based surveys.
Six novel Scientific Data Applications (SDAs) have been created that relate to three thematic areas: galactic science (G-Tomo and G-Arch), stellar characterisation (S-Phot and S-Disco) and lunar observation (L-Explo and L-Hex).
The SDAs draw primarily on datasets from recent lunar missions and the European Space Agency (ESA) cornerstone mission, Gaia. The tools deploy Machine Learning and advanced visual analytics to maximise scientific return-on-investment and to support scientific discovery by revealing previously unexploited aspects of the data.
G-Arch enables users to characterise large samples of stellar spectra from Gaia, in order to excavate the history of our galaxy through its chemical evolution. G-Tomo allows users to extract information on the absorption of light by interstellar dust grains and visualise the 3D distribution of dust in the Milky Way, supporting investigations of the relationship between interstellar clouds and populations of stars, and the effects of dust on observations of astrophysical objects.
S-Phot enables users to combine photometric and parallax data from Gaia and other relevant databases to reveal properties of stars in the Milky Way, such as their temperature, luminosity or evidence of circumstellar dust. S-Disco applies Machine Learning and Deep Learning to discover anomalies within large stellar datasets, such as unusual stars or unexpected populations or patterns.
L-Explo identifies subtle differences and anomalies within lunar orbital data to enhance geological mapping of the Moon. L-Hex supports human and robotic exploration of the Moon by providing access to historic and recent lunar data.
Each SDA uses the same software framework to ease development, integration and deployment on science platforms. The SDAs were tested and demonstrated on the dedicated, cloud-based analysis and exploitation platform (https://explore-platform.eu) and have been designed to be integrated and deployed on science platforms e.g. ESA Datalabs.
The SDAs are available as open-source software, in accordance with Findable, Accessible, Interoperable and Reusable (FAIR) principles. Targeted communication, dissemination and exploitation activities have not only helped develop a user base for the SDAs, but also stimulated space science communities to engage in further development and improvement of the tools. The legacy of EXPLORE is a flexible infrastructure for researchers to create and offer their own services on science platforms, with tools that are close to the input data and open to the community for direct, on-demand exploitation.
Six novel Scientific Data Applications (SDAs) have been created that relate to three thematic areas: galactic science (G-Tomo and G-Arch), stellar characterisation (S-Phot and S-Disco) and lunar observation (L-Explo and L-Hex).
The SDAs draw primarily on datasets from recent lunar missions and the European Space Agency (ESA) cornerstone mission, Gaia. The tools deploy Machine Learning and advanced visual analytics to maximise scientific return-on-investment and to support scientific discovery by revealing previously unexploited aspects of the data.
G-Arch enables users to characterise large samples of stellar spectra from Gaia, in order to excavate the history of our galaxy through its chemical evolution. G-Tomo allows users to extract information on the absorption of light by interstellar dust grains and visualise the 3D distribution of dust in the Milky Way, supporting investigations of the relationship between interstellar clouds and populations of stars, and the effects of dust on observations of astrophysical objects.
S-Phot enables users to combine photometric and parallax data from Gaia and other relevant databases to reveal properties of stars in the Milky Way, such as their temperature, luminosity or evidence of circumstellar dust. S-Disco applies Machine Learning and Deep Learning to discover anomalies within large stellar datasets, such as unusual stars or unexpected populations or patterns.
L-Explo identifies subtle differences and anomalies within lunar orbital data to enhance geological mapping of the Moon. L-Hex supports human and robotic exploration of the Moon by providing access to historic and recent lunar data.
Each SDA uses the same software framework to ease development, integration and deployment on science platforms. The SDAs were tested and demonstrated on the dedicated, cloud-based analysis and exploitation platform (https://explore-platform.eu) and have been designed to be integrated and deployed on science platforms e.g. ESA Datalabs.
The SDAs are available as open-source software, in accordance with Findable, Accessible, Interoperable and Reusable (FAIR) principles. Targeted communication, dissemination and exploitation activities have not only helped develop a user base for the SDAs, but also stimulated space science communities to engage in further development and improvement of the tools. The legacy of EXPLORE is a flexible infrastructure for researchers to create and offer their own services on science platforms, with tools that are close to the input data and open to the community for direct, on-demand exploitation.
At the end of the project, all six SDAs have been deployed. Prototypes were beta-tested and feedback was collected from early-adopters through the EXPLORE platform, user-workshops and demonstrations at conferences and other events. This formative evaluation guided development of the final project outputs. The SDAs have been presented at major international conferences, including the European Lunar Symposium, European Astronomical Society Annual Meeting and Artificial Intelligence for Science and Operations in Astronomy (SciOps) Meeting.
Multiple scientific articles have been published in Astronomy & Astrophysics (A&A) relating to G-Arch to date. Of particular note is Recio-Blanco et al. (2023), which describes the methodology used to extract the stellar chemo-physical parameters from Gaia Radial Velocity Spectrometer (RVS) spectra. This methodology has been pioneered through G-Arch, thus EXPLORE is the first attempt within the Gaia Data Processing and Analysis Consortium (DPAC) to bring an analysis pipeline closer to the users of the Gaia archive. Other papers relating to G-Arch include an analysis of stellar atmospheric parameters and chemical abundances, analysis of spatial and kinematical properties of diffuse interstellar bands and the development of a galactic chemical evolution model.
G-Tomo publications include a 3D map of interstellar dust in the Milky Way based on data from the Gaia early DR3 (eDR3) and 2MASS catalogue, and an intercalibration of data from complementary dust catalogues to improve 3D reconstruction and offer new maps at different spatial scales.
A paper describing S-Phot has been accepted in RAS Technology & Instrumentation (RASTI), and a further paper has been submitted to Open Research Europe. Several published papers have used the core science module of S-Phot, including one in Nature Astronomy. Two MSc theses are based on S-Phot, one accepted and one under minor revision. A paper describing the use of S-Disco for the Gaia RVS spectra is being prepared for publication in MNRAS, and two MSc thesis have been completed using S-Disco.
Two scientific projects with the stellar SDAs are in progress. The S-Phot prototype facilitates the location and retrieval of measurements of the brightness of stars from archives obtained from different telescopes and instruments, enabling accurate characterisation of stars and stellar groups in the Milky Way. The use of the S-Phot prototype to obtain stellar luminosities and temperatures for nearby mass-losing stars is described in a paper in preparation.
The lunar SDAs have been adopted by community mapping groups and have formed the basis of the Machine Learning Lunar Data Challenge to design a traverse on the Moon. A paper on L-Explo detection of landforms has been published in Earth and Space Sciences, with further papers submitted. A paper summarising lessons learned from the Lunar Data Challenge is also in preparation.
The Space Browser on the EXPLORE platform allows registered users to query planetary data collections through an interactive map browser. Users can constrain the number of results, view data products, browse images, add data as layers to the planetary surface basemap, and store products in the workspace for use with SDAs.
Multiple scientific articles have been published in Astronomy & Astrophysics (A&A) relating to G-Arch to date. Of particular note is Recio-Blanco et al. (2023), which describes the methodology used to extract the stellar chemo-physical parameters from Gaia Radial Velocity Spectrometer (RVS) spectra. This methodology has been pioneered through G-Arch, thus EXPLORE is the first attempt within the Gaia Data Processing and Analysis Consortium (DPAC) to bring an analysis pipeline closer to the users of the Gaia archive. Other papers relating to G-Arch include an analysis of stellar atmospheric parameters and chemical abundances, analysis of spatial and kinematical properties of diffuse interstellar bands and the development of a galactic chemical evolution model.
G-Tomo publications include a 3D map of interstellar dust in the Milky Way based on data from the Gaia early DR3 (eDR3) and 2MASS catalogue, and an intercalibration of data from complementary dust catalogues to improve 3D reconstruction and offer new maps at different spatial scales.
A paper describing S-Phot has been accepted in RAS Technology & Instrumentation (RASTI), and a further paper has been submitted to Open Research Europe. Several published papers have used the core science module of S-Phot, including one in Nature Astronomy. Two MSc theses are based on S-Phot, one accepted and one under minor revision. A paper describing the use of S-Disco for the Gaia RVS spectra is being prepared for publication in MNRAS, and two MSc thesis have been completed using S-Disco.
Two scientific projects with the stellar SDAs are in progress. The S-Phot prototype facilitates the location and retrieval of measurements of the brightness of stars from archives obtained from different telescopes and instruments, enabling accurate characterisation of stars and stellar groups in the Milky Way. The use of the S-Phot prototype to obtain stellar luminosities and temperatures for nearby mass-losing stars is described in a paper in preparation.
The lunar SDAs have been adopted by community mapping groups and have formed the basis of the Machine Learning Lunar Data Challenge to design a traverse on the Moon. A paper on L-Explo detection of landforms has been published in Earth and Space Sciences, with further papers submitted. A paper summarising lessons learned from the Lunar Data Challenge is also in preparation.
The Space Browser on the EXPLORE platform allows registered users to query planetary data collections through an interactive map browser. Users can constrain the number of results, view data products, browse images, add data as layers to the planetary surface basemap, and store products in the workspace for use with SDAs.
At the end of the project, the scientific impact of the EXPLORE SDAs is already evident through the 30+ papers published by the project team and by users. The majority of these have been published in high-impact journals (e.g. A&A, Nature Astronomy)
The EXPLORE platform’s Space Browser, SDA dashboard and administrator dashboard have each constituted major software developments. The EXPLORE platform has proved an important test environment for deploying prototype astronomy and space science applications in preparation for integration into cloud platforms. Work on EXPLORE’s SDAs has resulted in several new features and extensions to Visualizer, an open-source research framework that enables users to visualise, analyse and interact with data in multiple ways through their web-browser. Users can now collaboratively design a dashboard to explore data together in real-time, e.g. by mutually selecting interesting data fields, creating visualisations and annotating selected areas of interest.
EXPLORE has been presented as an exemplar of academic-industrial collaboration to policy makers and the academic community. Through the EXPLORE Data Challenges, the project has reached out to other fields of research, including AI, data analytics and geological mapping. The Public and Classroom Lunar Challenges have engaged participants from around the world, with a focus on communities in deprived areas.
The EXPLORE platform’s Space Browser, SDA dashboard and administrator dashboard have each constituted major software developments. The EXPLORE platform has proved an important test environment for deploying prototype astronomy and space science applications in preparation for integration into cloud platforms. Work on EXPLORE’s SDAs has resulted in several new features and extensions to Visualizer, an open-source research framework that enables users to visualise, analyse and interact with data in multiple ways through their web-browser. Users can now collaboratively design a dashboard to explore data together in real-time, e.g. by mutually selecting interesting data fields, creating visualisations and annotating selected areas of interest.
EXPLORE has been presented as an exemplar of academic-industrial collaboration to policy makers and the academic community. Through the EXPLORE Data Challenges, the project has reached out to other fields of research, including AI, data analytics and geological mapping. The Public and Classroom Lunar Challenges have engaged participants from around the world, with a focus on communities in deprived areas.