Periodic Reporting for period 1 - exaFOAM (Exploitation of Exascale Systems for Open-Source Computational Fluid Dynamics by Mainstream Industry)
Reporting period: 2021-04-01 to 2022-09-30
BACKGROUND:
The exaFOAM project aims to exploit new architectures and hybrid approaches is undertaken through the development and validation of algorithmic improvements, across the entire CFD process chain (preprocessing, simulation, I/O, post-processing). Effectiveness is demonstrated via a suite of HPC Grand Challenge and Industrial Application Challenge across all sectors of industry (transportation, power generation, disaster prevention, health and safety) to increase energy efficiency comfort and health/safety.
Stated AIMS:
- Demonstrate one or more orders-of-magnitude performance improvement on industry-based grand challenges
- Release technology advances realised during this project via opensource OpenFOAMvYYXX as respecting the terms of GPLv3
- Exploit performance gains among European Industry partners vested in exaFOAM as supporters/stakeholders in coordination with the OpenFOAM Governance Structure
Our collective Mission, Goals and Objectives are unchanged.
They are further consolidated through engagement with HPC-entities worldwide including AWS and Huawei (expect to be ratified as new Stakeholders shortly) - such are the direct means for technology end-users to gain access to the performance gains realised during this project.
Society IMPACT:
We restate that the Supporters and Stakeholders represent original equipment manufacturers whose design improvements realised by performance gains in this project benefits society directly in the form of energy-efficiencies, pollution-reduction and improved health/safety.
The exaFOAM project aims to exploit new architectures and hybrid approaches is undertaken through the development and validation of algorithmic improvements, across the entire CFD process chain (preprocessing, simulation, I/O, post-processing). Effectiveness is demonstrated via a suite of HPC Grand Challenge and Industrial Application Challenge across all sectors of industry (transportation, power generation, disaster prevention, health and safety) to increase energy efficiency comfort and health/safety.
Stated AIMS:
- Demonstrate one or more orders-of-magnitude performance improvement on industry-based grand challenges
- Release technology advances realised during this project via opensource OpenFOAMvYYXX as respecting the terms of GPLv3
- Exploit performance gains among European Industry partners vested in exaFOAM as supporters/stakeholders in coordination with the OpenFOAM Governance Structure
Our collective Mission, Goals and Objectives are unchanged.
They are further consolidated through engagement with HPC-entities worldwide including AWS and Huawei (expect to be ratified as new Stakeholders shortly) - such are the direct means for technology end-users to gain access to the performance gains realised during this project.
Society IMPACT:
We restate that the Supporters and Stakeholders represent original equipment manufacturers whose design improvements realised by performance gains in this project benefits society directly in the form of energy-efficiencies, pollution-reduction and improved health/safety.
Work performed during the first period of this project is summarised below per-workpackage:
WP1 — Management and Coordination: Addressing the challenges w.r.t. beneficiaries’ changes in management, resourcing and workplans, in particular challenges due to Covid in the period April-2021 to June 2022.
Regular meetings monitor the progress of the exaFOAM consortium members and report the progress to the Stakeholders
- Weekly touchpoint held every Wednesday @4.00CET via TEAMS virtual meetings
- Monthly Management Board meetings actions with minutes, held on the last Wednesday of every month via TEAMS virtual meetings
- Six-monthly General Assembly (GA) meetings, initially held virtually via TEAMS virtual meetings during Covid restrictions, and physically at Month-18.
- GA closely followed by TEAMS virtual meetings with global stakeholders to update them on progress
WP2 — Validation and Assessment: Grand Challenges, Industrial Benchmarks and Micro-benchmarks are all identified, and together with the Stakeholders, undergoing continuous monitoring, revision and updating to be current and progressive. These benchmarks are systematically made available to the end-user Stakeholders in order to quantify the performance gains realised during this project.
WP3 — Code Refactoring: Rearchitecting of software in order to realise HPC technology advanced. This mainly comprises the processing of sparse matrices (LUD) using matric compression (CSR) assessed on several state-of-the-art architectures based around Intel-Xeon, AMD-EPYC and Nvidia-GPU chip-sets. Performance gains of “percentage” and “factors” are realised which collectively increment towards the Objective of one-or-more orders of magnitude performance improvement. “Order-of-magnitude” performance improvements are addressed in the next workpackage.
WP4 — Evolution: Realisation of highly parallel solver gains based on coupled-solver techniques, I/O efficiencies, load-balancing especially for rotating-machinery and multi-physics applications, and external linear algebra solvers. The potential for orders-of-magnitude performance gains are clearly stated in respect of GPU or hybrid CPU-GPU architectures.
WP5 — Co-design, Profiling and Performance analysis: Profiling tools are in place and used to assess the benchmarks delivered in WP2, in respect of
- Parallel efficiency: The extent to which all resources in the system are kept active doing useful work
- Load Balance: The efficiency loss due to the global distribution of work among processes, in particular due to one process doing more computation than the others.
- Communication efficiency: The efficiency loss due to the communication of data.
- Transfer efficiency: Efficiency loss related to the non-instantaneous nature of communication mechanisms.
- Serialization efficiency: Inefficiencies caused by circular dependences or non-uniform imbalances.
- Computation scalability: How the time spent computing scales with respect to the reference case.
- Instructions scalability: How the number of instructions scales with respect to the reference case.
- IPC scalability: How the IPC scales with respect to the reference case.
- Frequency scalability: How the frequency scales with respect to the reference case.
WP6 — Integration: The several technologies under consideration in the previous work-packages are in pre-release format. The implementation of the final versions of these various technologies, validations and regression tests are in preparation to be release in the second half of this project.
WP7 — Dissemination, Impact and Exploitation: Partner exchange platform, external website for public visibility and dissemination programme are all in place, together with metrics of publications and presentations to conferences and workshops.
WP1 — Management and Coordination: Addressing the challenges w.r.t. beneficiaries’ changes in management, resourcing and workplans, in particular challenges due to Covid in the period April-2021 to June 2022.
Regular meetings monitor the progress of the exaFOAM consortium members and report the progress to the Stakeholders
- Weekly touchpoint held every Wednesday @4.00CET via TEAMS virtual meetings
- Monthly Management Board meetings actions with minutes, held on the last Wednesday of every month via TEAMS virtual meetings
- Six-monthly General Assembly (GA) meetings, initially held virtually via TEAMS virtual meetings during Covid restrictions, and physically at Month-18.
- GA closely followed by TEAMS virtual meetings with global stakeholders to update them on progress
WP2 — Validation and Assessment: Grand Challenges, Industrial Benchmarks and Micro-benchmarks are all identified, and together with the Stakeholders, undergoing continuous monitoring, revision and updating to be current and progressive. These benchmarks are systematically made available to the end-user Stakeholders in order to quantify the performance gains realised during this project.
WP3 — Code Refactoring: Rearchitecting of software in order to realise HPC technology advanced. This mainly comprises the processing of sparse matrices (LUD) using matric compression (CSR) assessed on several state-of-the-art architectures based around Intel-Xeon, AMD-EPYC and Nvidia-GPU chip-sets. Performance gains of “percentage” and “factors” are realised which collectively increment towards the Objective of one-or-more orders of magnitude performance improvement. “Order-of-magnitude” performance improvements are addressed in the next workpackage.
WP4 — Evolution: Realisation of highly parallel solver gains based on coupled-solver techniques, I/O efficiencies, load-balancing especially for rotating-machinery and multi-physics applications, and external linear algebra solvers. The potential for orders-of-magnitude performance gains are clearly stated in respect of GPU or hybrid CPU-GPU architectures.
WP5 — Co-design, Profiling and Performance analysis: Profiling tools are in place and used to assess the benchmarks delivered in WP2, in respect of
- Parallel efficiency: The extent to which all resources in the system are kept active doing useful work
- Load Balance: The efficiency loss due to the global distribution of work among processes, in particular due to one process doing more computation than the others.
- Communication efficiency: The efficiency loss due to the communication of data.
- Transfer efficiency: Efficiency loss related to the non-instantaneous nature of communication mechanisms.
- Serialization efficiency: Inefficiencies caused by circular dependences or non-uniform imbalances.
- Computation scalability: How the time spent computing scales with respect to the reference case.
- Instructions scalability: How the number of instructions scales with respect to the reference case.
- IPC scalability: How the IPC scales with respect to the reference case.
- Frequency scalability: How the frequency scales with respect to the reference case.
WP6 — Integration: The several technologies under consideration in the previous work-packages are in pre-release format. The implementation of the final versions of these various technologies, validations and regression tests are in preparation to be release in the second half of this project.
WP7 — Dissemination, Impact and Exploitation: Partner exchange platform, external website for public visibility and dissemination programme are all in place, together with metrics of publications and presentations to conferences and workshops.
STATE OF THE ART:
From the start of the project, the HPC performance of general-purpose Computational Fluid Dynamics algorithms used in mainstream is notably worse than idealised algorithms due to inherent bandwidth needs, three-dimensional and time-date handling with non-sparse matrix dependencies, spatial domain decomposition requirements, I/O challenges. The HPC-picture is constantly evolving, most notable in the development of GPU and hybrid CPU-GPU systems.
BEYOND THE STATE OF THE ART:
So far in this project the follow significant results have been achieved:
WP2 — Validation and Assessment: Benchmarks identified which are directly relevant to HPC profiling, repeatable and sharable to the Stakeholders
WP3 — Code Refactoring: Incremental gains which may collectively sum to order-of-magnitude performance gains
WP4 — Evolution: Significant potential for order-of-magnitude performance gains by adapting the algorithms to new architectures (GPU and hybrid GPU/CPU)
WP5 — Co-design, Profiling and Performance analysis: Profiling tools in place to quantify parallel efficiency, load-balance, communication efficiency, transfer efficiency, serialization efficiency, computation scalability, instructions scalability, IPC scalability and frequency scalability
WP7 — Dissemination, Impact and Exploitation: Partner exchange platform, external website for public visibility and dissemination programme are all in place
ONGOING OBJECTIVES:
We expect all three stated objectives to be achieved by the end of the project, namely; demonstration of order-of-magnitude performance improvements for general purpose CFD applications, visibility in the public domain via Quality Assured opensource releases and end-user engagement via dissemination to the key Stakeholders and European user community.
IMPACT:
European original equipment manufacturers (OEMs) will use the technologies realised in this project to accelerate the time-to-market and improve engineering design towards energy efficiency, comfort and health/safety.
From the start of the project, the HPC performance of general-purpose Computational Fluid Dynamics algorithms used in mainstream is notably worse than idealised algorithms due to inherent bandwidth needs, three-dimensional and time-date handling with non-sparse matrix dependencies, spatial domain decomposition requirements, I/O challenges. The HPC-picture is constantly evolving, most notable in the development of GPU and hybrid CPU-GPU systems.
BEYOND THE STATE OF THE ART:
So far in this project the follow significant results have been achieved:
WP2 — Validation and Assessment: Benchmarks identified which are directly relevant to HPC profiling, repeatable and sharable to the Stakeholders
WP3 — Code Refactoring: Incremental gains which may collectively sum to order-of-magnitude performance gains
WP4 — Evolution: Significant potential for order-of-magnitude performance gains by adapting the algorithms to new architectures (GPU and hybrid GPU/CPU)
WP5 — Co-design, Profiling and Performance analysis: Profiling tools in place to quantify parallel efficiency, load-balance, communication efficiency, transfer efficiency, serialization efficiency, computation scalability, instructions scalability, IPC scalability and frequency scalability
WP7 — Dissemination, Impact and Exploitation: Partner exchange platform, external website for public visibility and dissemination programme are all in place
ONGOING OBJECTIVES:
We expect all three stated objectives to be achieved by the end of the project, namely; demonstration of order-of-magnitude performance improvements for general purpose CFD applications, visibility in the public domain via Quality Assured opensource releases and end-user engagement via dissemination to the key Stakeholders and European user community.
IMPACT:
European original equipment manufacturers (OEMs) will use the technologies realised in this project to accelerate the time-to-market and improve engineering design towards energy efficiency, comfort and health/safety.