Periodic Reporting for period 2 - EarthServer-2 (Agile Analytics on Big Data Cubes)
Période du rapport: 2016-11-01 au 2018-04-30
EarthServer-2 makes Agile Analytics on Big Earth Data Cubes of sensor, image, simulation, and statistics data a commodity for non-experts and experts alike through. Built and upon its predecessor EarthServer Phase-1 (2011-2014) , it aims at exploiting large-scale Petabyte-sized data holdings thruogh open standards, in particular: the OGC Big Data standards and the forthcoming ISO SQL/MDA.
In the Joint Research Activity, the project is advancing the existing, world-leading rasdaman Array Database technology wrt. query functionality, integrated data/metadata queries, inter-federation data processing with automatic data and query distribution, tape archive integration, and 3D/4D visualization.
In the Services Activity, large data centers (ECMWF, PML, MEEO/ESA, ANU/NCI, JacobsUni) set up Ocean, Climate, Weather, Earth and Planetary Science data service on 3D & 4D data cubes reaching Petabyte-size with user-tailored clients based on NASA’s virtual globe technology (WebWoldWind, https://webworldwind.org). Cross-service data cube access and availability is present. Cross service queries even beyond those are being implemented.
In the Networking Activity, the project is actively advancing open Big Data standards in OGC, RDA, and ISO; in particular, the forthcoming ISO Array SQL standard is being written by project members. Further, a number of adequate information channels have been used for strong dissemination & exploitation, specifically with active conference and workshop participation and organization efforts; Earth science data user workshops for each domain addressed; actively contributing technology & experience to GEO / GEOSS and further bodies; establish standardized Big Geo Data benchmark and run it against EarthServer-2.
Altogether, EarthServer-2 is maintaining and extending the lead in Big Earth Data services established EarthServer-1 project.
More info on http://earthserver.eu/
In the Joint Research Activity, the project is advancing the existing, world-leading rasdaman Array Database technology wrt. query functionality, integrated data/metadata queries, inter-federation data processing with automatic data and query distribution, tape archive integration, and 3D/4D visualization.
In the Services Activity, large data centers (ECMWF, PML, MEEO/ESA, ANU/NCI, JacobsUni) set up Ocean, Climate, Weather, Earth and Planetary Science data service on 3D & 4D data cubes reaching Petabyte-size with user-tailored clients based on NASA’s virtual globe technology (WebWoldWind, https://webworldwind.org). Cross-service data cube access and availability is present. Cross service queries even beyond those are being implemented.
In the Networking Activity, the project is actively advancing open Big Data standards in OGC, RDA, and ISO; in particular, the forthcoming ISO Array SQL standard is being written by project members. Further, a number of adequate information channels have been used for strong dissemination & exploitation, specifically with active conference and workshop participation and organization efforts; Earth science data user workshops for each domain addressed; actively contributing technology & experience to GEO / GEOSS and further bodies; establish standardized Big Geo Data benchmark and run it against EarthServer-2.
Altogether, EarthServer-2 is maintaining and extending the lead in Big Earth Data services established EarthServer-1 project.
More info on http://earthserver.eu/
The EarthServer-2 project plans for the third milestone (M3) have been reached, initially implemented procedures for smooth, effective collaboration continue; the overall workplan is summarized on http://earthserver.eu/workplan. Management structure and procedures, including internal communication and reporting are put in place and being applied throughout the period leading to the project's third milestone.
The EarthServer-2 Consortium held project meetings regularly, in the premises of all partners.
The communities linked to the various service partners (Ocean, Climate, Weather, Earth and Planetary Science) have been kept involved and engaged throughout the developent of the project throughout the milestones, addressing each of them at topical or general scientific venues, such as INSPIRE (upcoming workshop) and EGU dedicated Splinter Sessions, dediated topical workshops which received excellent attendance and substantially raised interest on both the technology and the services already available from EarthServer-2.
The workflow across software development and adoption into Service operations has progressed steadily and at the third milestone, marked as Full, server and client instances allow to access, visualise and analyse a large - growing - amount of data, following the Description of Action.
More information on the technology powering EarthServer-2 is available on: http://earthserver.eu/technologies.
More information on the Services empowering EarthServer-2 is available on: http://earthserver.eu/services
More information on Publications derived from EarthServew-2 is available on: http://earthserver.eu/pub
You can join the EarthServer-2 community thorough its Social Media channels (e.g. https://twitter.com/earthserver_eu) , or address the discussion list: forum@earthserver.eu
With due consideration of these requirements as well as the contracted tasks as per the project's Description of Action, work items are progressing. A feedback loop between Service developers and operators, as well as their reference communities, exist, and needs are flowing back onto the underlying softare. Immersive 3-D visualisation technology from Open Source NASA/ESA World Wind also provides feedback to the needs of EarthServer-2 user and scientific users, in order to develop a seamless experience.
All partners have installed the latest released versions of service platform technology, rasdaman, and have proven operation based on operationally exploited datasets. Tutorials have been developed both from technology partners and service providers. They are reachable both from the project main page (http://earthserver.eu/webinars) and the project's window to the outside community on GitHub (https://github.com/earthserver-eu). What is developed by services is made available to the communities and cross-service querying empowers new trans-disciplinary ways to formulate - and get answers - scientific questions.
The final release of Rasdaman 9.5 both community and enterprise edition is upcoming, and Services already test release candidates, in addition to the stable production environments of the new stable version (9.4).
Immersive 3d visualization powered by NASA World Wind is now the standard web client delivery method for all partners, with a streamelined, consistent User Interface across the project, yet allowing for discipline-specific features (such as analytical tools, plotting and overall maps and data served, e.g. Solar System bodies for the Planetary Service - http://planetserver.eu/).
Visibility of EarthServer-2 in the Earth Science domains and communities has been achieved, with an outreach to currently altogether several thoudsand recipients across all social media. This activity will be sustained and expanded throughout and beyond the project.
Also, scientific talks as well as standardization continues intensely.
The EarthServer-2 Consortium held project meetings regularly, in the premises of all partners.
The communities linked to the various service partners (Ocean, Climate, Weather, Earth and Planetary Science) have been kept involved and engaged throughout the developent of the project throughout the milestones, addressing each of them at topical or general scientific venues, such as INSPIRE (upcoming workshop) and EGU dedicated Splinter Sessions, dediated topical workshops which received excellent attendance and substantially raised interest on both the technology and the services already available from EarthServer-2.
The workflow across software development and adoption into Service operations has progressed steadily and at the third milestone, marked as Full, server and client instances allow to access, visualise and analyse a large - growing - amount of data, following the Description of Action.
More information on the technology powering EarthServer-2 is available on: http://earthserver.eu/technologies.
More information on the Services empowering EarthServer-2 is available on: http://earthserver.eu/services
More information on Publications derived from EarthServew-2 is available on: http://earthserver.eu/pub
You can join the EarthServer-2 community thorough its Social Media channels (e.g. https://twitter.com/earthserver_eu) , or address the discussion list: forum@earthserver.eu
With due consideration of these requirements as well as the contracted tasks as per the project's Description of Action, work items are progressing. A feedback loop between Service developers and operators, as well as their reference communities, exist, and needs are flowing back onto the underlying softare. Immersive 3-D visualisation technology from Open Source NASA/ESA World Wind also provides feedback to the needs of EarthServer-2 user and scientific users, in order to develop a seamless experience.
All partners have installed the latest released versions of service platform technology, rasdaman, and have proven operation based on operationally exploited datasets. Tutorials have been developed both from technology partners and service providers. They are reachable both from the project main page (http://earthserver.eu/webinars) and the project's window to the outside community on GitHub (https://github.com/earthserver-eu). What is developed by services is made available to the communities and cross-service querying empowers new trans-disciplinary ways to formulate - and get answers - scientific questions.
The final release of Rasdaman 9.5 both community and enterprise edition is upcoming, and Services already test release candidates, in addition to the stable production environments of the new stable version (9.4).
Immersive 3d visualization powered by NASA World Wind is now the standard web client delivery method for all partners, with a streamelined, consistent User Interface across the project, yet allowing for discipline-specific features (such as analytical tools, plotting and overall maps and data served, e.g. Solar System bodies for the Planetary Service - http://planetserver.eu/).
Visibility of EarthServer-2 in the Earth Science domains and communities has been achieved, with an outreach to currently altogether several thoudsand recipients across all social media. This activity will be sustained and expanded throughout and beyond the project.
Also, scientific talks as well as standardization continues intensely.
EarthServer has been outstandingly influential in Big Earth Data demonstrating
new directions for flexible, scalable geo services based on innovative Array
Database technology. Researchers from Europe and the US have teamed up
materialize the datacube concept in six "Lighthouse Applications" together
covering all Earth Sciences. EarthServer has established client and server
technology for such spatio-temporal data-cubes. The underlying scalable Array
Database, rasdaman, has enabled direct interaction with databases of several hundred
TB. All services exclusively rely on the open OGC data and service standards for
"Big Geo Data", the Web Coverage Service (WCS) suite. In particular, the Web
Coverage Processing Service (WCPS) geo raster query language has proven
instrumental as a client data programming language which can be hidden behind
appealing visual interfaces, but available for additional clients and ways to access data through the same standards, being developed within the consortium.
Independent experts have characterized the outcome of EarthServer, based on
"proven evidence", that rasdaman will "significantly transform the way that
scientists in different areas of Earth Science will be able to access and use
data in a way that hitherto was not possible". And "with no doubt" this work
"has been shaping the Big Earth Data landscape through the standardization
activities within OGC, ISO and beyond".
In EarthServer-2 this is being pushed further:
Datacubes reached and crossed PetaByte-size built across borders to perform ad-hoc querying and merging.
new directions for flexible, scalable geo services based on innovative Array
Database technology. Researchers from Europe and the US have teamed up
materialize the datacube concept in six "Lighthouse Applications" together
covering all Earth Sciences. EarthServer has established client and server
technology for such spatio-temporal data-cubes. The underlying scalable Array
Database, rasdaman, has enabled direct interaction with databases of several hundred
TB. All services exclusively rely on the open OGC data and service standards for
"Big Geo Data", the Web Coverage Service (WCS) suite. In particular, the Web
Coverage Processing Service (WCPS) geo raster query language has proven
instrumental as a client data programming language which can be hidden behind
appealing visual interfaces, but available for additional clients and ways to access data through the same standards, being developed within the consortium.
Independent experts have characterized the outcome of EarthServer, based on
"proven evidence", that rasdaman will "significantly transform the way that
scientists in different areas of Earth Science will be able to access and use
data in a way that hitherto was not possible". And "with no doubt" this work
"has been shaping the Big Earth Data landscape through the standardization
activities within OGC, ISO and beyond".
In EarthServer-2 this is being pushed further:
Datacubes reached and crossed PetaByte-size built across borders to perform ad-hoc querying and merging.