Periodic Reporting for period 3 - SeaDataCloud (SeaDataCloud - Further developing the pan-European infrastructure for marine and ocean data management)
Période du rapport: 2019-11-01 au 2021-04-30
Marine data are collected by thousands of research institutes, governmental organisations and companies by various observing sensors and platforms and at considerable cost. These are of prime value for many applications, if well managed and available.
SeaDataNet operates an essential pan-European infrastructure managing, indexing and providing access to marine data sets and derived products, from research cruises, observing networks, projects, and others. Core partners are NODCs and major institutes from 34 states bordering European seas.
Main objectives of project:
1. To enhance and innovate the SeaDataNet standards, products and services offered to an expanded multi-disciplinary community:
2. To promote the adoption of the SeaDataNet protocols and standards
3. To present a long-term sustainable arrangement for the integrated SeaDataNet infrastructure and its network.
For uptake of cloud technologies and to seek ties with the European Open Science Cloud (EOSC) development, SeaDataNet developed a strategic cooperation with EUDAT network of computing infrastructures.
SeaDataNet operates an essential pan-European infrastructure managing, indexing and providing access to marine data sets and derived products, from research cruises, observing networks, projects, and others. Core partners are NODCs and major institutes from 34 states bordering European seas.
Main objectives of project:
1. To enhance and innovate the SeaDataNet standards, products and services offered to an expanded multi-disciplinary community:
2. To promote the adoption of the SeaDataNet protocols and standards
3. To present a long-term sustainable arrangement for the integrated SeaDataNet infrastructure and its network.
For uptake of cloud technologies and to seek ties with the European Open Science Cloud (EOSC) development, SeaDataNet developed a strategic cooperation with EUDAT network of computing infrastructures.
Project activities concerned considerably advancing services of the SeaDataNet infrastructure and increasing their contents, usage and performance. This included expanding the network of connected data centres, refining and expanding standards for data handling, and adopting cloud technology for upgrading core services and developing a Virtual Research Environment (VRE) as well as further networking with many data collectors, scientific user communities, and other marine data aggregators, like EMODnet and CMEMS.
The following results have been achieved:
• Upgraded set-up and look & feel of SeaDataNet portal
• Upgraded CDI Data Discovery & Access service, offering a very efficient query and shopping experience to a wealth of marine data sets for physics, chemistry, geology, biology, and bathymetry
• Expanded network of data centres connected to the CDI Data Discovery and Access service, from 103 to 114, and increased number of data sets from 1.87 to 2.62 million entries with 850 data originators
• Cloud data cache for all public data (91% of total CDI volume) adopted in CDI service and new tools for data replication from local to cloud cache deployed at all SeaDataNet data centres
• 33 new SeaDataNet products providing harmonised data collections of interest to many users (2 versions of regional and global aggregated datasets of T and S, resulting climatologies, and 11 new observational products)
• 6500 users from 113 countries registered as SeaDataNet user for downloading data and products, of which 1843 added during SeaDataCloud project. Every month, circa 50 downloading users. Last project year, 1,1 million CDIs downloaded by ‘normal users’. On top, regular major downloads harvested as input for SeaDataNet and EMODnet data products (Chemistry, Bathymetry, and Biology)
• Expanded capability for handling different data types with standards and guidelines for 1) High-Frequency Radar (HFR) data, 2) Flow Cytometer data, and 3) Glider data
• Upgraded and enriched SeaDataNet common vocabularies supporting various communities and greater semantic interoperability
• Multiple deliveries of updated SeaDataNet tools and services, supporting use and application of SeaDataNet standards, increasing functionality, and including new data types
• Delivery of prototype central transformation services for converting SeaDataNet data formats to relevant INSPIRE application schemas, next to INSPIRE compliance for the CDI metadata format
• Launch of prototype of SeaDataNet Virtual Research Environment (VRE) with advanced services, such as Web versions of ODV and DIVA analytical software
• Launch of brokerage service to discover datasets managed at major international portals: Australia Ocean Data Network (AODN), US – NOAA National Centers for Environmental Information (NCEI), and NOAA World Ocean Database (WOD)
• Delivery of Sensor Web Enablement (SWE) toolkit to facilitate operators of observing networks streamlining (N)RT data flow from observation sites to data centres to publishing
• Adoption of SEANOE (SEA scieNtific Open data Edition) service to enable scientists publishing their marine research data as citable resources with DOI. A selection is automatically duplicated to EMODnet Ingestion for wider publishing in European data portals feeding into EMODnet thematic portals
Moreover, activities have been directed towards exploitation with following results:
• Formulation of exploitation and business plan for the long-term sustainability: SeaDataNet should continue to play a role and perform activities as a major operational partner for well-funded EU initiatives such as EMODnet and CMEMS, and the planned EOOS initiative, gathering and providing delayed mode data from many data originators, and validation and long-term stewardship services to the wider oceanography community:
o Formalised cooperation with CMEMS by renewed Memorandum of Understanding (MoU)
o Assignment of SeaDataNet as the data repository of EMODnet Chemistry and Bathymetry
o Data exchange arranged with EMODnet Physics and Biology
o Assignment of SeaDataNet network of NODCs as data management experts for EMODnet Ingestion
o Close cooperation of SeaDataNet with EuroGOOS and CMEMS for European data exchange of operational oceanography data as relevant for EOOS
• Established SeaDataNet AISBL as legal entity under Belgian law (12th of February 2019): to improve quality and effectiveness of management, processing and dissemination of oceanographic data at national, regional and global levels. Currently, there are 30 members, representing nearly all European countries with a coastline and more applications underway.
• Ambition to strive for further research and development of the SeaDataNet standards, tools, and services, to manage and harness emerging technologies and requirements; Seeking Horizon Europe opportunities from EOSC and ESFRI seems most appropriate
• Agreement to secure the continued operation of SeaDataNet infrastructure over 3 years after the project’s end, while seeking EU opportunities for further FAIRification.
The following results have been achieved:
• Upgraded set-up and look & feel of SeaDataNet portal
• Upgraded CDI Data Discovery & Access service, offering a very efficient query and shopping experience to a wealth of marine data sets for physics, chemistry, geology, biology, and bathymetry
• Expanded network of data centres connected to the CDI Data Discovery and Access service, from 103 to 114, and increased number of data sets from 1.87 to 2.62 million entries with 850 data originators
• Cloud data cache for all public data (91% of total CDI volume) adopted in CDI service and new tools for data replication from local to cloud cache deployed at all SeaDataNet data centres
• 33 new SeaDataNet products providing harmonised data collections of interest to many users (2 versions of regional and global aggregated datasets of T and S, resulting climatologies, and 11 new observational products)
• 6500 users from 113 countries registered as SeaDataNet user for downloading data and products, of which 1843 added during SeaDataCloud project. Every month, circa 50 downloading users. Last project year, 1,1 million CDIs downloaded by ‘normal users’. On top, regular major downloads harvested as input for SeaDataNet and EMODnet data products (Chemistry, Bathymetry, and Biology)
• Expanded capability for handling different data types with standards and guidelines for 1) High-Frequency Radar (HFR) data, 2) Flow Cytometer data, and 3) Glider data
• Upgraded and enriched SeaDataNet common vocabularies supporting various communities and greater semantic interoperability
• Multiple deliveries of updated SeaDataNet tools and services, supporting use and application of SeaDataNet standards, increasing functionality, and including new data types
• Delivery of prototype central transformation services for converting SeaDataNet data formats to relevant INSPIRE application schemas, next to INSPIRE compliance for the CDI metadata format
• Launch of prototype of SeaDataNet Virtual Research Environment (VRE) with advanced services, such as Web versions of ODV and DIVA analytical software
• Launch of brokerage service to discover datasets managed at major international portals: Australia Ocean Data Network (AODN), US – NOAA National Centers for Environmental Information (NCEI), and NOAA World Ocean Database (WOD)
• Delivery of Sensor Web Enablement (SWE) toolkit to facilitate operators of observing networks streamlining (N)RT data flow from observation sites to data centres to publishing
• Adoption of SEANOE (SEA scieNtific Open data Edition) service to enable scientists publishing their marine research data as citable resources with DOI. A selection is automatically duplicated to EMODnet Ingestion for wider publishing in European data portals feeding into EMODnet thematic portals
Moreover, activities have been directed towards exploitation with following results:
• Formulation of exploitation and business plan for the long-term sustainability: SeaDataNet should continue to play a role and perform activities as a major operational partner for well-funded EU initiatives such as EMODnet and CMEMS, and the planned EOOS initiative, gathering and providing delayed mode data from many data originators, and validation and long-term stewardship services to the wider oceanography community:
o Formalised cooperation with CMEMS by renewed Memorandum of Understanding (MoU)
o Assignment of SeaDataNet as the data repository of EMODnet Chemistry and Bathymetry
o Data exchange arranged with EMODnet Physics and Biology
o Assignment of SeaDataNet network of NODCs as data management experts for EMODnet Ingestion
o Close cooperation of SeaDataNet with EuroGOOS and CMEMS for European data exchange of operational oceanography data as relevant for EOOS
• Established SeaDataNet AISBL as legal entity under Belgian law (12th of February 2019): to improve quality and effectiveness of management, processing and dissemination of oceanographic data at national, regional and global levels. Currently, there are 30 members, representing nearly all European countries with a coastline and more applications underway.
• Ambition to strive for further research and development of the SeaDataNet standards, tools, and services, to manage and harness emerging technologies and requirements; Seeking Horizon Europe opportunities from EOSC and ESFRI seems most appropriate
• Agreement to secure the continued operation of SeaDataNet infrastructure over 3 years after the project’s end, while seeking EU opportunities for further FAIRification.
• wider, simplified, and more efficient access to the SeaDataNet data: by upgraded and new services giving users better discovery and access to more data and abilities for doing science with these data
• synergies and complementary capabilities: new brokerage service and online SWE Ingestion service allow users to discover and retrieve complimentary data from major data portals and operational observing networks
• new generation of researchers educated to exploit all the essential tools for their research: upgraded infrastructure gives researchers easy access to an extensive treasurebox of marine data resources. SeaDataNet standards and supporting tools are adopted as models by other infrastructures and researchers
• better management of continuous flow of data: major progress with integration of more data centres, interoperability with other European and international data infrastructures, and ability to ingest major data collections and (near) real-time data streams
• Contribution to evidence-based policymaking: SeaDataNet is an important driver and component of several EMODnet portals, which generate data products in support of the MSFD implementation
• synergies and complementary capabilities: new brokerage service and online SWE Ingestion service allow users to discover and retrieve complimentary data from major data portals and operational observing networks
• new generation of researchers educated to exploit all the essential tools for their research: upgraded infrastructure gives researchers easy access to an extensive treasurebox of marine data resources. SeaDataNet standards and supporting tools are adopted as models by other infrastructures and researchers
• better management of continuous flow of data: major progress with integration of more data centres, interoperability with other European and international data infrastructures, and ability to ingest major data collections and (near) real-time data streams
• Contribution to evidence-based policymaking: SeaDataNet is an important driver and component of several EMODnet portals, which generate data products in support of the MSFD implementation