Periodic Reporting for period 2 - EGI-Engage (Engaging the EGI Community towards an Open Science Commons)
Reporting period: 2016-03-01 to 2017-08-31
The EGI-Engage project brought together 43 partners with a mission to expand the capabilities of a backbone of federated services for compute, storage, data, communication, knowledge and expertise, complementing community-specific capabilities. Impact was delivered through a number of key exploitable results.
1. Update of the strategy, governance & procurement procedures. The EGI Federation now has a new strategy and a new governance model adapted to the recent evolution of the e-Infrastructure landscape in Europe. The project worked on an analysis of opportunities and barriers for cross-border procurement of e-Infrastructure services with input from major Research Infrastructures.
2. Integrated Management System and Certification. Following 18 months of work to define a system to plan, implement, monitor and continually improve all business processes under its responsibility, the EGI Foundation was awarded ISO 9001:2015 and ISO/IEC 20000-1:2011 certifications.
3. Security policies. The security team updated the EGI policy frameworks to follow the technical evolution of the EGI services and also to make them more general and re-usable by other initiatives.
4. Integrated thematic services. Throughout the project, the project worked with major European research communities and research infrastructures to co-design and co-develop new services. Most of these services are now offered as integrated scientific applications with EGI’s e-Infrastructure services.
5. Improved service portfolio. The EGI service portfolio was redesigned with improved service definitions.
6. Tools for federated service management. The project supported technological innovation and new services in the area of Service Registry and resource allocation. The EGI Accounting Portal was redeveloped and improved with a new user interface, new views and features. The operational tools were also continuously improved and adapted to satisfy new requirements from service providers and user communities.
7. Open Data platform. The project developed a platform designed to make data discoverable and available in an easy way across all EGI federated resources. The EGI DataHub will offer scalable data access and compute capabilities around scientific datasets for scientific groups at the large scale. Once in production, the DataHub will enable data processing in hybrid environments like public and private clouds.
8. Expanded Federated Cloud computing. The EGI Federated Cloud was expanded with new IaaS capabilities. It now integrates existing commercial and public IaaS Cloud deployments and e-Infrastructures with the current production infrastructures.
9. Marketplace. The project delivered a platform where services, delivered by EGI providers and many other partners, can be promoted, discovered, shared, ordered and accessed. It will include EGI services as well as discipline and community-specific tools and services. The platform is a building block of the future European Open Science Cloud.
10. Applications on Demand. A portfolio of scientific applications provided as a service was released. The service provides researchers with dedicated access to computational and storage resources, as well as other facilities needed to run scientific applications.
1. Update of the strategy, governance & procurement procedures. The EGI Federation now has a new strategy and a new governance model adapted to the recent evolution of the e-Infrastructure landscape in Europe. The project worked on an analysis of opportunities and barriers for cross-border procurement of e-Infrastructure services with input from major Research Infrastructures.
2. Integrated Management System and Certification. Following 18 months of work to define a system to plan, implement, monitor and continually improve all business processes under its responsibility, the EGI Foundation was awarded ISO 9001:2015 and ISO/IEC 20000-1:2011 certifications.
3. Security policies. The security team updated the EGI policy frameworks to follow the technical evolution of the EGI services and also to make them more general and re-usable by other initiatives.
4. Integrated thematic services. Throughout the project, the project worked with major European research communities and research infrastructures to co-design and co-develop new services. Most of these services are now offered as integrated scientific applications with EGI’s e-Infrastructure services.
5. Improved service portfolio. The EGI service portfolio was redesigned with improved service definitions.
6. Tools for federated service management. The project supported technological innovation and new services in the area of Service Registry and resource allocation. The EGI Accounting Portal was redeveloped and improved with a new user interface, new views and features. The operational tools were also continuously improved and adapted to satisfy new requirements from service providers and user communities.
7. Open Data platform. The project developed a platform designed to make data discoverable and available in an easy way across all EGI federated resources. The EGI DataHub will offer scalable data access and compute capabilities around scientific datasets for scientific groups at the large scale. Once in production, the DataHub will enable data processing in hybrid environments like public and private clouds.
8. Expanded Federated Cloud computing. The EGI Federated Cloud was expanded with new IaaS capabilities. It now integrates existing commercial and public IaaS Cloud deployments and e-Infrastructures with the current production infrastructures.
9. Marketplace. The project delivered a platform where services, delivered by EGI providers and many other partners, can be promoted, discovered, shared, ordered and accessed. It will include EGI services as well as discipline and community-specific tools and services. The platform is a building block of the future European Open Science Cloud.
10. Applications on Demand. A portfolio of scientific applications provided as a service was released. The service provides researchers with dedicated access to computational and storage resources, as well as other facilities needed to run scientific applications.
The project helped advancing the EGI Federation as distributed infrastructure of hundreds of computing and data centres spread across 40+ countries in Europe and worldwide that deliver advanced computing services to support scientists, international projects and research infrastructures. Activities focused on: increasing the number of research collaborations, research infrastructures and projects supported, on exploiting various key technologies in the form of production services disseminated through the EGI service catalogue and Marketplace, and on strengthening strategic activities of EGI including the contribution to the definition of the EOSC vision and implementation roadmap.
The continued support to science at all scales was demonstrated by an estimated production of more than 5,000 scientific papers in 2015 and 2016, and by the awarding of Physics Nobel Prize 2017 to the LIGO-Virgo research collaboration. EGI-Engage contributed to digital innovation in science with an increasing adoption of federated cloud services (+67.5% in 2016 and +40% in January-August 2017 with more than 10.4 Million CPU hours delivered) and a sustained major increase of use of High Throughput Computing (+23% and +20% in 2015 and 2016 respectively) under the push of an increasing data production rate by various large research experiments. The Competence Centres and other partnerships allows the extension of the portfolio of thematic services, which deliver scientific tools, data, pipelines and other digital artifacts necessary in community-specific research workflows. The project impacted society and economy by supporting research in various key areas (climate change, marine health, biodiversity conservation and environment, human health) and by completing 11 business collaborations with industry and SMEs.
The continued support to science at all scales was demonstrated by an estimated production of more than 5,000 scientific papers in 2015 and 2016, and by the awarding of Physics Nobel Prize 2017 to the LIGO-Virgo research collaboration. EGI-Engage contributed to digital innovation in science with an increasing adoption of federated cloud services (+67.5% in 2016 and +40% in January-August 2017 with more than 10.4 Million CPU hours delivered) and a sustained major increase of use of High Throughput Computing (+23% and +20% in 2015 and 2016 respectively) under the push of an increasing data production rate by various large research experiments. The Competence Centres and other partnerships allows the extension of the portfolio of thematic services, which deliver scientific tools, data, pipelines and other digital artifacts necessary in community-specific research workflows. The project impacted society and economy by supporting research in various key areas (climate change, marine health, biodiversity conservation and environment, human health) and by completing 11 business collaborations with industry and SMEs.
The scientific production enabled by EGI services and EGI-Engage key results, is estimated to amount to 5,000 peer-reviewed papers in 2015 and 2016; this was achieved thanks to an increasing number of researchers supported. During the project the registered users exceeded 30,500 units, and a similar increase was observed for users accessing EGI services via thematic services and community-specific portals, which offer data, data products, software and collaborative tools on top of generic EGI services. Thanks to the adoption of federated Cloud services in EGI, the total number of users of EGI services in 2017 amounts to 61,000 (+30% during EGI-Engage).
One of the impacts of the project is the increase of the adoption of advanced compute and storage services based on new technological solutions leveraging today’s megatrends, like Grid computing, Cloud computing and big data analytics tools.
Among the 31 large-scale research initiatives currently supported, those which reached maturity and full service adoption in 2015, have been largely increasing usage in 2016 and 2017. The number of active use cases that benefited from project technical support exceeded the defined annual thresholds defined in in the project KPIs, and reached the peak value of 63 business cases.
Scientific applications providing simulation, data processing and analysis functionalities, data visualization etc. are key enablers for a broad adoption of EGI Compute services. EGI-Engage supported the setup of 33 thematic services, as well as 17 scientific applications within the Applications on Demand Service.
The EGI Federation comprises to date about 730,000 cores, 300 PB of online storage and 346 PB of nearline storage. The compute capacity increased by +23% in the first reporting period, and by +12.3% in the latter one. EGI-Engage contributed to the advancement of the implementation of the ERA by strengthening the national systems – the national e-Infrastructures (NGIs), which were better connected to an increasing number of international research collaborations. Transnational cooperation was optimized by enabling access to national resources and services via an integrated service management system and an internal catalogue of services for providers of the EGI Federation.
One of the impacts of the project is the increase of the adoption of advanced compute and storage services based on new technological solutions leveraging today’s megatrends, like Grid computing, Cloud computing and big data analytics tools.
Among the 31 large-scale research initiatives currently supported, those which reached maturity and full service adoption in 2015, have been largely increasing usage in 2016 and 2017. The number of active use cases that benefited from project technical support exceeded the defined annual thresholds defined in in the project KPIs, and reached the peak value of 63 business cases.
Scientific applications providing simulation, data processing and analysis functionalities, data visualization etc. are key enablers for a broad adoption of EGI Compute services. EGI-Engage supported the setup of 33 thematic services, as well as 17 scientific applications within the Applications on Demand Service.
The EGI Federation comprises to date about 730,000 cores, 300 PB of online storage and 346 PB of nearline storage. The compute capacity increased by +23% in the first reporting period, and by +12.3% in the latter one. EGI-Engage contributed to the advancement of the implementation of the ERA by strengthening the national systems – the national e-Infrastructures (NGIs), which were better connected to an increasing number of international research collaborations. Transnational cooperation was optimized by enabling access to national resources and services via an integrated service management system and an internal catalogue of services for providers of the EGI Federation.