Next Generation I/O for Exascale

Risultati finali

Report giving the final assessment and evaluation of the NEXTGenIO solution. This report includes the results from benchmarking the I/O Workload Simulator and an I/O server solution on the NEXTGenIO architecture and we as well as evaluating the test-bed with different storage file systems.

The NEXTGenIO Architecture

Published report that gives a final description of the NEXTGenIO hardware, systemware, and data architecture.

Final report on energy-aware and data-aware scheduling

Final report on the job scheduler and data scheduler, with coordination and automatic guidance (using hints, metrics and tools). Evaluation of the benefits achieved using the prototype.

Final Public Project Report

Final Public Project Report.

Final report on NVRAM data storage

Dynamic management of the NVRAM layer, providing dynamic partition schemes and support for different applications. Analysis of different services to provide compression, resiliency, and de-duplication on the layer. Final software prototype released.

Final Dissemination and Exploitation Report

Final Dissemination and Exploitation Report. Public report. If any confidential information is included it will added as a separate confidential annex.

Final report on the NEXTGenIO systemware

Final report collating all the details on the NEXTGenIO systemware. This includes: the collaboration between the different schedulers developed and PyCOMPSs; a prototype of PyCOMPSs able to communicate I/O aspects of the jobs; details on the Unit Testing employed to validate and verify the hardware.

Basic Benchmark Evaluation Report

Report evaluating the basic performance of the NEXTGenIO prototype system.

Final Release of Performance Analysis and Debugging Tools

Release of performance analysis tools with support for NVRAM usage analysis and hardware topology mappings. Release of debugging tools that support the removal of NVRAM related application errors.

NVRAM Supported Tool Prototypes and Techniques

Prototype implementations and evaluations towards tool improvements that are enabled by NVRAM.

PyCOMPSs Release and Performance Integration

Final release of PyCOMPSs extensions and report on performance measurements with its profiling interface.

Energy-aware and data-aware job and data schedulers

Software implementation of the energy- and data-aware schedulers.

Final release of the I/O server software supporting NVRAM

Release of the updated version of the I/O server.

Final release of the I/O Workload Simulator

Release of the final version of the I/O Workload Simulator software.

Workflow Monitoring Final Release

Final release of an implementation for the design in D4.3.

PyCOMPSs for NEXTGenIO

Release of PyCOMPSs extended with support for the NEXTGenIO architecture.

Pubblicazioni

NORNS: Extending Slurm to Support Data-Driven Workflows through Asynchronous Data Staging

Autori: Alberto Miranda Adrian Jackson Tommaso Tocci Iakovos Panourgias Ramon Nou
Pubblicato in: 2019 IEEE International Conference on Cluster Computing (CLUSTER), 2019
Editore: IEEE

ECHOFS: A Scheduler-Guided Temporary Filesystem to Leverage Node-Local NVMS

Autori: Alberto Miranda, Ramon Nou, Toni Cortes
Pubblicato in: 2018 30th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD), 2018, Pagina/e 225-228, ISBN 978-1-5386-7769-8
Editore: IEEE
DOI: 10.1109/cahpc.2018.8645894

Visualization of Multi-layer I/O Performance in Vampir

Autori: Hartmut Mix, Christian Herold, Matthias Weber
Pubblicato in: 2018 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), 2018, Pagina/e 387-394, ISBN 978-1-5386-5555-9
Editore: IEEE
DOI: 10.1109/ipdpsw.2018.00073

GekkoFS - A Temporary Distributed File System for HPC Applications

Autori: Marc-Andre Vef, Nafiseh Moti, Tim SuB, Tommaso Tocci, Ramon Nou, Alberto Miranda, Toni Cortes, Andre Brinkmann
Pubblicato in: 2018 IEEE International Conference on Cluster Computing (CLUSTER), 2018, Pagina/e 319-324, ISBN 978-1-5386-8319-4
Editore: IEEE
DOI: 10.1109/cluster.2018.00049

A High-Performance Distributed Object-Store for Exascale Numerical Weather Prediction and Climate

Autori: Simon D. Smart, Tiago Quintino, Baudouin Raoult
Pubblicato in: Proceedings of the Platform for Advanced Scientific Computing Conference on - PASC '19, 2019, Pagina/e 1-11, ISBN 9781-450367707
Editore: ACM Press
DOI: 10.1145/3324989.3325726

A Scalable Object Store for Meteorological and Climate Data

Autori: Simon D. Smart, Tiago Quintino, Baudouin Raoult
Pubblicato in: Proceedings of the Platform for Advanced Scientific Computing Conference on - PASC '17, 2017, Pagina/e 1-8, ISBN 9781-450350624
Editore: ACM Press
DOI: 10.1145/3093172.3093238

High Performance Computing Enabled Simulation of the Food-Water-Energy System - Simulation of Intensively Managed Landscapes

Autori: Harold E.B. Dennis, Adam S. Ward, Tyler Balson, Yuwei Li, Robert Henschel, Shawn Slavin, Stephen Simms, Holger Brunst
Pubblicato in: Proceedings of the Practice and Experience in Advanced Research Computing 2017 on Sustainability, Success and Impact - PEARC17, 2017, Pagina/e 1-10, ISBN 9781-450352727
Editore: ACM Press
DOI: 10.1145/3093338.3093381

A Methodology for Performance Analysis of Applications Using Multi-layer I/O

Autori: Ronny Tschüter, Christian Herold, Bert Wesarg, Matthias Weber
Pubblicato in: Euro-Par 2018: Parallel Processing - 24th International Conference on Parallel and Distributed Computing, Turin, Italy, August 27 - 31, 2018, Proceedings, Numero 11014, 2018, Pagina/e 16-30, ISBN 978-3-319-96982-4
Editore: Springer International Publishing
DOI: 10.1007/978-3-319-96983-1_2

Risultati finali

Pubblicazioni

Condividi questa pagina

Scarica