DEEP – SOFTWARE FOR EXASCALE ARCHITECTURES | DEEP-SEA | Project | Results | H2020 | CORDIS

Resultado final

Definition of the quality control processes and templates for internal verification and document review for all project results and deliverables

Interoperability development

Detailed report on proven ability of GPISpace to support the execution of MPI or GASPI programs spanning over multiple nodes and detailed performance study of MPIGPU performance bottleneck

Application use cases and traces

Use cases integrated in JUBE profile and trace files provided to other WPs

Final evaluation of the system software stack

Optimisations, adjustments and bug fixing in accordance with the user experiences.This Deliverable is led by ParTec.

Initial application co-design input

Documents requirements of all applications for codesign includes all data analytics SW requirements compute memory communication performance footprints communication patterns etc

Final report on applications experience

Reports the results of the application experience using the DEEP-SEA developments. Performance and efficiency (both according to the most relevant metric for the application), scalability, and portability will be measured and compared with the code performances at the beginning of the project.

Applications use of DEEP-SEA software stack

Details the tools SW components and programming models that each application will use with further codesign input on the needed functionalities

Software specification

Specification of the complete SW stack based on the requirements collected from WPs 1 to 5 This comprises WP3internal interfaces and dependencies as well as the interplay with WPs 2 4 and 5

Documentation of last improvements of the tools

Documentation including bug fixes, lessons learned from deployment and next steps.

MSA-driven extension to system-wide programming and new memory integration

Detailed report and intermediate release on extension of MPI libraries, including collective communications, RMA communication, tuning, and the MPI memory management extension. Also includes details on NVM as a fast buffer for a workflow’s intermediate data, and GPI-2 extension to support persistent segments.

Report on standardisation activities

Summarising report covering all proposals made to the mentioned standardisation bodies based on work in DEEP-SEA as well as open standardisation potentials to create future roadmaps.

Final report

Description of the technical and scientific results of the project

Repository for training material

Design, access information and initial contents for the training repository.

Intermediate node-level programming environment

Intermediate release and detailed report of programming environment. Final release and SW stack for processing in memory (PIM). This SW release will provide the programming environment to the application partners for their work leading to D1.4 First evaluation results.

Complete system software implementation

Full system SW implementation of all components of the system SW stack meeting the requirements collected in D3.1 and matching the final specification developed in D3.2.This Deliverable is led by ParTec.

Final Node-level programming environment

Final release and detailed report of programming environment and basis for D1.5 Final report on applications experience. Any remaining integration and debugging work will continue in the integration task

Initial node-level programming environment

Initial release and detailed report of node-level programming environment. This SW release will enable early work by the application partners

Resiliency support

Intermediate release and detailed report on resiliency enhancement. It includes description on resiliency in MPI sessions, slum extensions, Support of persistent segments in the GPI-2 implementation.

Malleability concept and early prototype

Intermediate release and detailed report on malleability including concept and prototype version of MPIsessionbased malleability within MPI

Final system-level programming environment

Final release and detailed report on WP5 contributions to system-level programming environment. This report includes the malleability support (including MPI and OmpSs-2@cluster extensions), interoperability (including accelerator optimised communication, MPC+GPI interaction, ParaStation MPI communication layer support in GPI-2, GPI-Space programs that support MPI and GASPI at the same time, and both MPI+OmpSs-2@cluster and GPI+OmpSs-2@cluster), and resiliency capabilities (including support for persistent segments in in-memory checkpointing library, evaluation of Slurm extension, evaluation of the extension of the checkpoint restart capabilities, evaluation of Open MPI runtime fault tolerance). These evaluations will be conducted using micro-benchmark and applications from WP1.

Final release of the tools

Release with final feature set, tested and including updates to user documentation and installation guide.

Resource management and tool interface

Intermediate release and detailed report on the interfaces developed between MPI libraries and their collocated environment such as the resource manager and external tools.

Final outreach activity

Final brochure or project video, or an event or a combination of these. It will be decided during the project, which will be deemed most appropriate.

Communication and data management plan, toolkit and owned channels

Details of the communication and brand strategy including a toolkit with materials like logos and collateral templates and DEEPSEA communication channels It contains also the data management plan as part of the Open Research Data Pilot ORDP

Publicaciones

FaaSKeeper: Learning from Building Serverless Services with ZooKeeper as an Example

Autores: M. Copik, A. Calotoiu, K. Taranov, T. Hoefler
Publicado en: 2022
Editor: Arxiv
DOI: 10.48550/arxiv.2203.14859

Classification of Solar Flares using Data Analysis and Clustering of Active Regions

Autores: Hanne Baeke; Jorge Amaya; Giovanni Lapenta
Publicado en: Crossref, Edición 1, 2023
Editor: ESS Open Archive
DOI: 10.22541/essoar.167336864.46114556/v1

Processing in Memory: The Tipping Point

Autores: Petar Radojković, Paul Carpenter, Pouya Esmaili-Dokht, Rémy Cimadomo, Henri-Pierre Charles, Abu Sebastian, Paolo Amato
Publicado en: ETP4HPC White Paper, 2021
Editor: ETP4HPC
DOI: 10.5281/zenodo.4767489

Cppless: Productive and Performant Serverless Programming in C++

Autores: L. Möller, M. Copik, A. Calotoiu, T. Hoefler
Publicado en: 2024
Editor: Arxiv
DOI: 10.48550/arxiv.2401.10834

FTIO: Detecting I/O Periodicity Using Frequency Techniques

Autores: Tarraf, Ahmad; Bandet, Alexis; Zanon Boito, Francieli; Pallez, Guillaume; Wolf, Felix
Publicado en: https://inria.hal.science/hal-04382142, Edición 2, 2023
Editor: arxiv
DOI: 10.48550/arxiv.2306.08601

Modular Supercomputing Architecture

Autores: Suarez, Estela; Eicker, Norbert; Moschny, Thomas; Pickartz, Simon; Clauss, Carsten; Plugaru, Valentin; Herten, Andreas; Michielsen, Kristel; Lippert, Thomas
Publicado en: ETP4HPC White Papers, 2022
Editor: ETP4HPC
DOI: 10.5281/zenodo.6508394

Heterogeneous High Performance Computing

Autores: P. Carpenter, U.-U. Haus, E. Laure, S. Narasimhamurthy, E. Suarez
Publicado en: ETP4HPC White Paper, 2022
Editor: ETP4HPC
DOI: 10.5281/zenodo.6090425

Task-Based Performance Portability in HPC

Autores: Olivier Aumage, Paul Carpenter, Siegfried Benkner
Publicado en: ETP4HPC White Paper, 2021
Editor: ETP4HPC

HPC for Urgent Decision-Making

Autores: M. Marazakis, M.Duranton, D. Pleiter, G. Taffoni, and H.C. Hoppe
Publicado en: ETP4HPC White Paper, 2022
Editor: ETP4HPC
DOI: 10.5281/zenodo.6107362

A Compiler Approach to Automatic Multi-Pumping

Autores: Johnsen, Carl-Johannes; De Matteis, Tiziano; Ben-Nun, Tal; Licht, Johannes de Fine; Hoefler, Torsten
Publicado en: Proceedings of the 41st IEEE/ACM International Conference on Computer-Aided Design, Edición 2, 2022
Editor: ACM
DOI: 10.1145/3508352.3549374

An Emulation Layer for Dynamic Resources with MPI Sessions

Autores: Jan Fecht; Martin Schreiber; Martin Schulz; Howard Pritchard; Daniel J. Holmes
Publicado en: Lecture Notes in Computer Science, Edición 2, 2023, ISBN 9783031232190
Editor: Springer
DOI: 10.1007/978-3-031-23220-6_10

Arax: A Runtime Framework for Decoupling Applications from Heterogeneous Accelerators.

Autores: M. Pavlidakis, S. Mavridis, A. Chazapis, G. Vasiliadis, and A. Bilas
Publicado en: ACM Symposium on Cloud Computing (SoCC) 2022, 2022
Editor: ACM
DOI: 10.1145/3542929.3563467

Artifact and instructions to generate experimental results for Euro-Par 2022 paper: OmpSs-2@Cluster: Distributed memory execution of nested OpenMP-style tasks

Autores: Aguilar Mena, Jimmy; Shaaban, Omar; Beltran, Vicenç; Carpenter, Paul; Ayguadé, Eduard; Labarta Mancho, Jesus
Publicado en: Lecture Notes in Computer Science, Edición 2, 2022
Editor: Springer
DOI: 10.6084/m9.figshare.19960721.v1

NPBench: A Benchmarking Suite for High-Performance NumPy

Autores: Alexandros Nikolaos Ziogas, Tal Ben-Nun, Timo Schneider, and Torsten Hoefler
Publicado en: 2021
Editor: ICS'21

Maximum Flows in Parametric Graph Templates

Autores: T. Ben-Nun, L. Gianinazzi, T. Hoefler, Y. Oltchik
Publicado en: International Conference on Algorithms and Complexity, 2023
Editor: Springer
DOI: 10.48550/arxiv.2307.08420

rFaaS: Enabling High Performance Serverless with RDMA and Leases

Autores: M. Copik, K. Taranov, A. Calotoiu, T. Hoefler
Publicado en: 2023
Editor: IEEE
DOI: 10.1109/ipdps54959.2023.00094

OmpSs-2@Cluster: Distributed memory execution of nested OpenMP-style tasks.

Autores: J. Aguilar Mena, O. Shaaban, V. Beltran, P. Carpenter, E. Ayguade, and J. Labarta
Publicado en: Proceedings of Euro-Par 2022, 2022
Editor: Springer
DOI: 10.1007/978-3-031-12597-3_20

Towards Dynamic Resource Management with MPI Sessions and PMIx

Autores: Huber, Dominik; Streubel, Maximilian; Comprés, Isaías; Schulz, Martin; Schreiber, Martin; Pritchard, Howard
Publicado en: uroMPI/USA '22: Proceedings of the 29th European MPI Users' Group Meeting, Edición 2, 2022
Editor: ACM
DOI: 10.1145/3555819.3555856

A Data-Centric Optimization Framework for Machine Learning

Autores: Oliver Rausch; Tal Ben-Nun; Nikoli Dryden; Andrei Ivanov; Shigang Li; Torsten Hoefler
Publicado en: ICS '22: Proceedings of the 36th ACM International Conference on Supercomputing, 2022
Editor: Association for Computing Machinery
DOI: 10.48550/arxiv.2110.10802

Productive Performance Engineering for Weather and Climate Modeling with Python

Autores: T. Ben-Nun, L. Groner, F. Deconinck, T. Wicky, E. Davis, J. Dahm, O. Elbert, R. George, J. McGibbon, L. Trümper, E. Wu, O. Fuhrer, T. Schulthess, T. Hoefler
Publicado en: SC'22, 2022
Editor: -

Exploring the impact of node failures on the resource allocation for parallel jobs

Autores: Ioannis Vardas, Manolis Ploumidis, Manolis Marazakis
Publicado en: Proceedings of the 14th Resilience Workshop, held in conjunction with Euro-Par, 2021
Editor: JuSER

Bridging Control-Centric and Data-Centric Optimization

Autores: T. Ben-Nun, B. Ates, A. Calotoiu, T. Hoefler
Publicado en: IEEE/ACM International Symposium on Code Generation and Optimization (CGO), 2023
Editor: ACM
DOI: 10.1145/3579990.3580018

Building Blocks for Network-Accelerated Distributed File Systems

Autores: Salvatore Di Girolamo, Daniele De Sensi, Konstantin Taranov, Milos Malesevic, Maciej Besta, Timo Schneider, Severin Kistler, Torsten Hoefler
Publicado en: SC'22, 2022
Editor: SC'22
DOI: 10.48550/arxiv.2206.10007

Productivity, portability, performance: data-centric Python

Autores: A. N. Ziogas, T. Schneider, T. Ben-Nun, A. Calotoiu, T. De Matteis, J. de Fine Licht, L. Lavarini, T. Hoefler
Publicado en: 2021
Editor: ACM
DOI: 10.1145/3458817.3476176

Impact of Cache Coherence on the Performance of Shared-Memory based MPI Primitives: A Case Study for Broadcast on Intel Xeon Scalable Processors - Computational Artifacts

Autores: Katevenis, George; Ploumidis, Manolis; Marazakis, Manolis
Publicado en: 52nd International Conference on Parallel Processing (ICPP), Edición 2, 2023
Editor: ACM
DOI: 10.5281/zenodo.8074488

Accelerating Brain Simulations with the Fast Multipole Method

Autores: H. Nöttgen, F. Czappa, and F. Wolf
Publicado en: Proceedings of Euro-Par 2022, 2022
Editor: Springer
DOI: 10.1007/978-3-031-12597-3_24

Conquering Noise With Hardware Counters on HPC Systems

Autores: M. Ritter, A. Tarraf, A. Geiß, N. Daoud, B. Mohr, F. Wolf
Publicado en: 2022
Editor: IEEE
DOI: 10.1109/protools56701.2022.00007

A High-Fidelity Flow Solver for Unstructured Meshes on Field-Programmable Gate Arrays: Design, Evaluation, and Future Challenges.

Autores: Martin Karp, Artur Podobas, Tobias Kenter, Niclas Jansson, Christian Plessl, Philipp Schlatter, and Stefano Markidis
Publicado en: HPCAsia2022, 2022
Editor: ACM
DOI: 10.1145/3492805.3492808

Impact of Cache Coherence on the Performance of Shared-Memory based MPI Primitives: A Case Study for Broadcast on Intel Xeon Scalable Processors

Autores: G. Katevenis, M. Ploumidis, M. Marazakis
Publicado en: 52nd International Conference on Parallel Processing (ICPP), 2023
Editor: ACM
DOI: 10.1145/3605573.3605616

Maximum Flows in Parametric Graph Templates

Autores: Tal Ben-Nun; Lukas Gianinazzi; Torsten Hoefler; Yishai Oltchik
Publicado en: Lecture Notes in Computer Science, Edición vol 13898, 2023, Página(s) 97-111
Editor: Springer
DOI: 10.1007/978-3-031-30448-4_8

Boosting Performance Optimization with Interactive Data Movement Visualization

Autores: Philipp Schaad, Tal Ben-Nun, Torsten Hoefler
Publicado en: 2022
Editor: SC'22
DOI: 10.48550/arxiv.2207.07433

FuzzyFlow: Leveraging Dataflow To Find and Squash Program Optimization Bugs

Autores: P. Schaad, T. Schneider, T. Ben-Nun, A. Calotoiu, A. Nikolaos Ziogas, T. Hoefler
Publicado en: The International Conference for High Performance Computing, Networking, Storage and Analysis, 2023
Editor: ACM
DOI: 10.1145/3581784.3613214

Breaking Down the Parallel Performance of GROMACS, a High-Performance Molecular Dynamics Software

Autores: Andersson, Måns I.; Murugan, N. Arul; Podobas, Artur; Markidis, Stefano
Publicado en: PPAM22, Edición Lecture Notes in Computer Science, vol 13826, 2023, Página(s) 333–345, ISBN 978-3-031-30442-2
Editor: Springer
DOI: 10.48550/arxiv.2208.13658

Lifting C semantics for dataflow optimization

Autores: Calotoiu, Alexandru; Ben-Nun, Tal; Kwasniewski, Grzegorz; Licht, Johannes de Fine; Schneider, Timo; Schaad, Philipp; Hoefler, Torsten
Publicado en: ICS '22: Proceedings of the 36th ACM International Conference on Supercomputing, 2022
Editor: Association for Computing Machinery
DOI: 10.1145/3524059.3532389

ecoHMEM: Improving Object Placement Methodology for Hybrid Memory Systems in HPC

Autores: Jordà Peroliu, Marc; Rai, Siddharth; Ayguadé Parra, Eduard; Labarta Mancho, Jesús José; Peña Monferrer, Antonio José
Publicado en: Crossref, Edición 18, 2022
Editor: IEEE
DOI: 10.1109/cluster51413.2022.00040

Filtering and Ranking of Code Regions for Parallelization via Hotspot Detection and OpenMP Overhead Analysis

Autores: S.A. Mohammadi, L. Rothenberger, G. de Morais, B. Nico Görlich, E. Lille, H. Rüthers, F. Wolf
Publicado en: 2023
Editor: IEEE
DOI: 10.1145/3624062.3624206

PROGRAML: A Graph-based Program Representation for Data Flow Analysis and Compiler Optimizations

Autores: Chris Cummins and Zacharias V. Fisches and Tal Ben-Nun and Torsten Hoefler and Michael O’Boyle and Hugh Leather
Publicado en: ICML'21, 2022
Editor: -

Automatic aggregation of subtask accesses for nested OpenMP-style tasks

Autores: O. Shaaban, J. Aguilar Mena, V. Beltran, P. Carpenter, E. Ayguade and J. Labarta Mancho
Publicado en: SBAC-PAD 2022, 2022
Editor: IEEE
DOI: 10.1109/sbac-pad55451.2022.00042

Combining Dynamic Concurrency Throttling with Voltage and Frequency Scaling on Task-based Programming Models

Autores: Antoni Navarro Muñoz; Arthur F. Lorenzon; Eduard Ayguadé Parra; Vicenç Beltran Querol
Publicado en: ICPP, 2021
Editor: Association for Computing Machinery
DOI: 10.1145/3472456.3472471

Advanced synchronization techniques for task-based runtime systems.

Autores: D. Álvarez, K. Sala, M. Maroñas, A. Roca, V. Beltran
Publicado en: Proceedings of PPoPP 2021, 2021
Editor: ACM
DOI: 10.1145/3437801.3441601

FMI: Fast and Cheap Message Passing for Serverless Functions

Autores: Copik, Marcin; Böhringer, Roman; Calotoiu, Alexandru; Hoefler, Torsten
Publicado en: Proceedings of the 37th International Conference on Supercomputing, Edición 18, 2023
Editor: ACM
DOI: 10.1145/3577193.3593718

Deinsum: Practically I/O Optimal Multilinear Algebra

Autores: A. Nikolaos Ziogas, G. Kwasniewski, T. Ben-Nun, T. Schneider, T. Hoefler
Publicado en: 2022
Editor: SC'22
DOI: 10.48550/arxiv.2206.08301

a benchmarking suite for high-performance NumPy

Autores: Alexandros Nikolaos Ziogas; Tal Ben-Nun; Timo Schneider; Torsten Hoefler
Publicado en: Proceedings of the ISC 2021, 2021
Editor: Association for Computing Machinery
DOI: 10.1145/3447818.3460360

Influence of Network Performance Variability on Application Scalability

Autores: Daniele De Sensi; Tiziano De Matteis; Konstantin Taranov; Salvatore Di Girolamo; Tobias Rahn; Torsten Hoefler
Publicado en: Proceedings of the ACM on Measurement and Analysis of Computing Systems, 6 (3), Edición 2, 2022
Editor: ACM
DOI: 10.1145/3570609

Transparent load balancing of MPI programs using OmpSs-2@cluster and DLB

Autores: J. Aguilar Mena, O. Shaaban, V. Beltran, P. Carpenter, E. Ayguade, and J. Labarta
Publicado en: Proceedings of ICPP, 2022
Editor: ACM
DOI: 10.1145/3545008.3545045

A framework for hierarchical single-copy MPI collectives on multicore nodes

Autores: G. Katevenis-Bitzos, M. Ploumidis, and M. Marazakis
Publicado en: IEEE Cluster 2022, Edición Presented at conference, 2022
Editor: IEEE

Performance Embeddings: A Similarity-Based Transfer Tuning Approach to Performance Optimization

Autores: L Trümper, T Ben-Nun, P Schaad, A Calotoiu, T Hoefler
Publicado en: 2023
Editor: ACM
DOI: 10.1145/3577193.3593714

User-guided Page Merging for Memory Deduplication in Serverless Systems

Autores: Qiu, Wei; Copik, Marcin; Wang, Yun; Calotoiu, Alexandru; Hoefler, Torsten
Publicado en: 2023 IEEE International Conference on Big Data (Big Data), Edición 18, 2023
Editor: IEEE
DOI: 10.1109/bigdata59044.2023.10386487

Methodology for malleable applications on distributed memory systems

Autores: J. Aguilar Mena
Publicado en: 2022
Editor: Universitat Politècnica de Catalunya
DOI: 10.5821/dissertation-2117-380814

Critical Analysis of the Modular Supercomputing Architecture

Autores: E. Suarez, N. Eicker, Th.Moschny, Th. Lippert
Publicado en: Porting applications to a Modular Supercomputer - Experiences from the DEEP-EST project, 2021
Editor: FZJ Zentralbibliothek Verlag

An OpenMP free agent threads implementation

Autores: Lopez, Victor; Criado, Joel; Peñacoba, Raúl; Ferrer, Roger; Teruel, Xavier; Garcia-Gasulla, Marta
Publicado en: Lecture Notes in Computer Science - OpenMP: Enabling Massive Node-Level Parallelism, 2021
Editor: Springer
DOI: 10.1007/978-3-030-85262-7_15

Best practices guide

Autores: A. Kreuzer, J. Kreutz, B. Steinbusch
Publicado en: Porting applications to a Modular Supercomputer - Experiences from the DEEP-EST project, 2021
Editor: FZJ Zentralbibliothek Verlag

Space weather with DLMOS, xPic and GMM

Autores: J.Amaya
Publicado en: Porting applications to a Modular Supercomputer - Experiences from the DEEP-EST project, 2021
Editor: FZJ Zentralbibliothek Verlag

The DEEP-EST project

Autores: E. Suarez, A.Kreuzer, N. Eicker, Th. Lippert
Publicado en: Porting applications to a Modular Supercomputer - Experiences from the DEEP-EST project, 2021
Editor: FZJ Zentralbibliothek Verlag

Simulating Structural Plasticity of the Brain more Scalable than Expected

Autores: F. Czappa, A. Geiß, F. Wolf
Publicado en: Journal of Parallel and Distributed Computing, Edición 07437315, 2022, ISSN 0743-7315
Editor: Academic Press
DOI: 10.1016/j.jpdc.2022.09.001

Meshing strategies for 3d geo-electromagnetic modeling in the presence of metallic infrastructure

Autores: O. Castillo-Reyes, P. Rulff, E. Schankee Um, A. Amor-Martin
Publicado en: Computational geosciences, 2023, ISSN 1573-1499
Editor: Springer
DOI: 10.1007/s10596-023-10247-w

O(n) Key–Value Sort With Active Compute Memory

Autores: P. Esmaili-Dokht, M. Guiot, P. Radojkovic, X. Martorell, E. Ayguade, J. Labarta, J. Adlard, P. Amato, M. Sforzin
Publicado en: IEEE Transactions on Computers, 2024, ISSN 1557-9956
Editor: IEEE
DOI: 10.1109/tc.2024.3371773

Electromagnetic imaging and deep learning for transition to renewable energies: a technology review

Autores: O. Castillo-Reyes, X. Hu, B. Wang, Y. Wang, Z. Guo
Publicado en: Frontiers in Earth Science, 2023, ISSN 2296-6463
Editor: Frontiers
DOI: 10.3389/feart.2023.1159910

Mitigating the NUMA Effect on Task-Based Runtime Systems

Autores: M. Maroñas, A. Navarro, E. Ayguadé and V. Beltran
Publicado en: The Journal of Supercomputing, Edición 4, 2023, ISSN 0920-8542
Editor: Kluwer Academic Publishers
DOI: 10.1007/s11227-023-05164-9

Operational Data Analytics in practice: Experiences from design to deployment in production HPC environments

Autores: Alessio Netti, Michael Ott, Carla Guillen, Daniele Tafani, Martin Schulz
Publicado en: Parallel Computing (ParCo), Edición 01678191, 2022, ISSN 0167-8191
Editor: Elsevier BV
DOI: 10.1016/j.parco.2022.102950

Towards leveraging collective performance with the support of MPI 4.0 features in MPC

Autores: S. Bouhrour, T. Pepin, J. Jaeger
Publicado en: Journal on Parallel Computing (ParCo), Edición Volume 109, March 2022, 102860, 2022, ISSN 0167-8191
Editor: Elsevier BV
DOI: 10.1016/j.parco.2021.102860

Optimizing Iterative Data-flow Scientific Applications using Directed Cyclic Graphs

Autores: D. Álvarez and V. Beltran
Publicado en: IEEE Access, 2023, ISSN 2169-3536
Editor: Institute of Electrical and Electronics Engineers Inc.
DOI: 10.1109/access.2023.3269902

Porting applications to a Modular Supercomputer - Experiences from the DEEP-EST project

Autores: A. Kreuzer, E. Suarez, N. Eicker, Th. Lippert
Publicado en: Porting applications to a Modular Supercomputer - Experiences from the DEEP-EST project, 2021
Editor: FZJ Zentralbibliothek Verlag

Buscando datos de OpenAIRE...

DEEP – SOFTWARE FOR EXASCALE ARCHITECTURES

Resultado final

Publicaciones

Compartir esta página

Descargar