Periodic Reporting for period 3 - ASTERICS (Astronomy ESFRI and Research Infrastructure Cluster)
Reporting period: 2018-05-01 to 2019-04-30
The ASTERICS project (Astronomy ESFRI and Research Infrastructure Cluster) is a collaborative cluster for the next generation ESFRI telescope facilities (ELT, CTA, SKA, KM3NeT, EST, see Figure 1) and other relevant research infrastructure initiatives in the area of astronomy, astrophysics and astroparticle physics. The ASTERICS Consortium consists of 26 partners, universities and research institutes, linked to one or more of these ESFRI facilities.
Each of the facilities is a mega-scale Research Infrastructure, endorsed and prioritised by the European Strategy Forum on Research Infrastructures (ESFRI). These ESFRI projects and their pathfinders significantly extend our astronomical observational capabilities; they also open new windows on the universe through the detection of neutrinos, high-energy particle showers and gravitational waves. Moreover, multi-messenger astronomy provides an unprecedented chance to probe the combined observational parameter space.
The purpose of ASTERICS was to enhance the ESFRI facilities’ impact in the future. ASTERICS brought together developers of all facilities to more efficiently develop tools and standards and to lay the foundation for true multi-messenger astroparticle physics.
Ensuring that Europe, and the world, fully exploits the impact of the ESFRI facilities, demands close integration through operations, (data) processing, analysis and data interoperability. Before the ASTERICS project, interactions between the communities had been limited. As each of these new facilities will generate vast amounts of data, ASTERICS focused on aspects of data handling (generation, transport, preservation, retrieval and analysis), interoperability between facilities (linked to smart analysis, scheduling for simultaneous observations) and fast response.
The project realised close collaboration between the diverse astronomy and astroparticle physics domains. The project partners developed several tools for data handling including new algorithms and standards, with the objective of future use on data from large astronomical facilities. Since many of these tools can be used already on the precursor or pathfinder data, the use and further development is guaranteed and will have a huge impact on the data handling of these facilities.
Each of the facilities is a mega-scale Research Infrastructure, endorsed and prioritised by the European Strategy Forum on Research Infrastructures (ESFRI). These ESFRI projects and their pathfinders significantly extend our astronomical observational capabilities; they also open new windows on the universe through the detection of neutrinos, high-energy particle showers and gravitational waves. Moreover, multi-messenger astronomy provides an unprecedented chance to probe the combined observational parameter space.
The purpose of ASTERICS was to enhance the ESFRI facilities’ impact in the future. ASTERICS brought together developers of all facilities to more efficiently develop tools and standards and to lay the foundation for true multi-messenger astroparticle physics.
Ensuring that Europe, and the world, fully exploits the impact of the ESFRI facilities, demands close integration through operations, (data) processing, analysis and data interoperability. Before the ASTERICS project, interactions between the communities had been limited. As each of these new facilities will generate vast amounts of data, ASTERICS focused on aspects of data handling (generation, transport, preservation, retrieval and analysis), interoperability between facilities (linked to smart analysis, scheduling for simultaneous observations) and fast response.
The project realised close collaboration between the diverse astronomy and astroparticle physics domains. The project partners developed several tools for data handling including new algorithms and standards, with the objective of future use on data from large astronomical facilities. Since many of these tools can be used already on the precursor or pathfinder data, the use and further development is guaranteed and will have a huge impact on the data handling of these facilities.
Bringing together communities that have so far been independent requires bringing together the individuals. ASTERICS facilitated this by organising thematic meetings at all levels and facilitating participating to these meetings with targeted invitations and travel support. We offered participants a good reason and low threshold to join. The wide variety of themes led to a series of events that we spread conveniently over the years. We achieved not only bringing people together, but also making it good practice organising and visiting such events to share and learn.
The Policy Forum has studied science cases for the ASTERICS ESFRI facilities and identified challenges in delivering multi-messenger astrophysics as “who pays for the common challenges” and “who coordinates the matching of facility and user requirements”? The final report with recommendations was presented and discussed during the final ASTERICS meeting in Groningen.
ASTERICS developed a multi-messenger platform prototype. Combining existing data was already possible through IVOA-based analysis tools, but identifying new observation possibilities, accessing these and finding observations connected to alerts was almost impossible. The new platform provides a repository of tools. It also connects scientists and developers to create a fully functional platform, integrating tools developed all over the world.
ASTERICS contributed to solving critical (data) communication challenges faced by many scientific arrays of interconnected elements. Instruments like the CTA and SKA will contain even larger numbers of elements, which need to be synchronised to an extremely high degree of precision. To achieve this synchronisation, astronomy has adopted an open source technology, developed at CERN, called White Rabbit (WR).
We promote the ESFRI facilities and their data and software tools to the widest possible audiences. Three animation videos for promotional and educational activities were released (see Figure 3). Five Citizen Science experiments, “Muon Hunters”, “Muon Hunters 2.0: Return of the Ring”, “Galaxy Nurseries”, “Euclid – Challenge the Machines”, and “SuperWASP Variable Stars” have been set up and are being used widely.
Results of the project were shared widely by education, training, and interacting with ESFRI facility staff, industry, the wider user community, and, through exciting citizen science applications, with society and the general public.
The Final results of ASTERICS were presented and discussed during the Final ASTERICS Grand Event: “the new era of multi-messenger astrophysics symposium”. Results will be published in the proceedings on multi-messenger.asterics2020.eu.
The Policy Forum has studied science cases for the ASTERICS ESFRI facilities and identified challenges in delivering multi-messenger astrophysics as “who pays for the common challenges” and “who coordinates the matching of facility and user requirements”? The final report with recommendations was presented and discussed during the final ASTERICS meeting in Groningen.
ASTERICS developed a multi-messenger platform prototype. Combining existing data was already possible through IVOA-based analysis tools, but identifying new observation possibilities, accessing these and finding observations connected to alerts was almost impossible. The new platform provides a repository of tools. It also connects scientists and developers to create a fully functional platform, integrating tools developed all over the world.
ASTERICS contributed to solving critical (data) communication challenges faced by many scientific arrays of interconnected elements. Instruments like the CTA and SKA will contain even larger numbers of elements, which need to be synchronised to an extremely high degree of precision. To achieve this synchronisation, astronomy has adopted an open source technology, developed at CERN, called White Rabbit (WR).
We promote the ESFRI facilities and their data and software tools to the widest possible audiences. Three animation videos for promotional and educational activities were released (see Figure 3). Five Citizen Science experiments, “Muon Hunters”, “Muon Hunters 2.0: Return of the Ring”, “Galaxy Nurseries”, “Euclid – Challenge the Machines”, and “SuperWASP Variable Stars” have been set up and are being used widely.
Results of the project were shared widely by education, training, and interacting with ESFRI facility staff, industry, the wider user community, and, through exciting citizen science applications, with society and the general public.
The Final results of ASTERICS were presented and discussed during the Final ASTERICS Grand Event: “the new era of multi-messenger astrophysics symposium”. Results will be published in the proceedings on multi-messenger.asterics2020.eu.
The ASTERICS project succeeded in developing common tools linked to the Virtual Observatory and for general data handling of extremely large data sets.
ASTERICS contributed to the continuing development of a framework based on the International Virtual Observatory Alliance (IVOA) standards. This work has advanced the preparation for interoperable ESFRI data discovery and access. The ASTERICS all-sky multi-dimensional approach has led to spectacular new data visualisation capabilities, now implemented in observatory archive services and for gravitational wave tools (e.g. GWsky). Standards to describe data provenance, essential for data reusability and trustworthiness, have been developed. On-going technological work supports new developments for time domain interoperability.
ASTERICS created or improved several software packages to assess the quality of large datasets and execute automatic analysis to reduce their size. This resulted in a collection of statistically robust and domain independent open source software libraries for data analysis, integration, and data mining on Petascale datasets. We improved workflows for making software portable and scalable across a variety of software and hardware environments.
Software and tools are available through the ASTERICS repositories at www.asterics2020.eu/asterics-repositories; one repository for Virtual Observatory related tools, one for trainings and tutorials, and two for services and software tools for Petascale datasets.
ASTERICS organised schools and topical meetings to disseminate new and improved techniques and tools.
Virtual Observatory results will continue to be presented in the IVOA meetings.
Results are also published in peer-reviewed articles and presented at international seminars and conferences.
Many partners of ASTERICS are engaged with industries, and citizen science and will continue to do so after the project.
In collaboration with the worldwide White Rabbit community, ASTERICS has improved the accuracy of the synchronisation devices by several orders of magnitude. Calibration methods for large numbers of nodes were developed, and the synchronisation of astronomical instruments via operational networks, alongside commercial data traffic, was implemented. The technology has been implemented in the Dutch SURFnet8 network. A ten-fold increase of the transport capacity of the WR devices to 10GB was successfully developed and is a major success of ASTERICS.
ASTERICS contributed to the continuing development of a framework based on the International Virtual Observatory Alliance (IVOA) standards. This work has advanced the preparation for interoperable ESFRI data discovery and access. The ASTERICS all-sky multi-dimensional approach has led to spectacular new data visualisation capabilities, now implemented in observatory archive services and for gravitational wave tools (e.g. GWsky). Standards to describe data provenance, essential for data reusability and trustworthiness, have been developed. On-going technological work supports new developments for time domain interoperability.
ASTERICS created or improved several software packages to assess the quality of large datasets and execute automatic analysis to reduce their size. This resulted in a collection of statistically robust and domain independent open source software libraries for data analysis, integration, and data mining on Petascale datasets. We improved workflows for making software portable and scalable across a variety of software and hardware environments.
Software and tools are available through the ASTERICS repositories at www.asterics2020.eu/asterics-repositories; one repository for Virtual Observatory related tools, one for trainings and tutorials, and two for services and software tools for Petascale datasets.
ASTERICS organised schools and topical meetings to disseminate new and improved techniques and tools.
Virtual Observatory results will continue to be presented in the IVOA meetings.
Results are also published in peer-reviewed articles and presented at international seminars and conferences.
Many partners of ASTERICS are engaged with industries, and citizen science and will continue to do so after the project.
In collaboration with the worldwide White Rabbit community, ASTERICS has improved the accuracy of the synchronisation devices by several orders of magnitude. Calibration methods for large numbers of nodes were developed, and the synchronisation of astronomical instruments via operational networks, alongside commercial data traffic, was implemented. The technology has been implemented in the Dutch SURFnet8 network. A ten-fold increase of the transport capacity of the WR devices to 10GB was successfully developed and is a major success of ASTERICS.