Predicting extreme weather events with artificial intelligence
Climate change is expected to reshape weather systems, potentially exacerbating threats such as extreme weather events. Improved weather and climate modelling will help us prepare for these future impacts. Researchers working on the MAELSTROM project have further developed Europe’s high-performance computing (HPC) architecture to enhance our ability to predict future climate events. The MAELSTROM team came together from across Europe to co-design bespoke computer systems to improve energy efficiency, new software able to incorporate machine learning, and new artificial intelligence applications to aid with weather prediction. These technological advances have already strengthened Europe’s HPC capabilities and boosted technical capabilities in weather and climate science. “This project helped the weather and climate community to better understand what machine learning can do to improve weather and climate predictions, how to make the best use of modern supercomputers, and how to enable the HPC industry to provide the best hardware for the application domain,” says Peter Dueben, MAELSTROM project coordinator and head of the Earth System Modelling section at the European Centre for Medium-Range Weather Forecasts (ECMWF) in the United Kingdom.
Co-designing new hardware and software
The MAELSTROM project is a collaborative effort in which new products and tools were created through a co-design cycle. The project was carried out with support from the European High Performance Computing Joint Undertaking (EuroHPC JU), an initiative set up to develop a world-class supercomputing ecosystem in Europe. Weather and climate scientists developed new applications in collaboration with machine learning researchers, while software engineers worked to enhance the software tools available to the scientists developing new applications in training, inference and benchmarking. Supercomputing scientists then took these applications and software and used them as benchmarks to test new hardware configurations and computer system designs. “Feedback from hardware developers to the application developers in turn is allowing us to build more efficient applications and to adjust parallelisation, energy consumption and numerical precision during training and inference,” explains Dueben.
Setting benchmarks for the machine learning community
The most important results from the project are the products and tools, says Dueben. These range from enhanced machine learning applications for weather and climate science, and meaningful benchmark data sets for the machine learning community which are published online, through to improved software for machine learning. The machine learning applications developed through MAELSTROM are able to transform climate data into more accurate predictions, by improved processing of observations, post-processing of model data and new forecasting abilities. The project also led to a deeper understanding of which computer system designs are most efficient for machine learning in physical applications. “In particular the application developments have resulted in tools that are already used at the participating weather forecast centres to improve weather predictions on a daily basis,” Dueben notes.
Revolutionising weather predictions
Machine learning is currently revolutionising how weather predictions are performed, with pure machine learning models outperforming conventional predictions in many aspects. The project was completed in March 2024, but with major weather centres such as project partner Met Norway developing tools using machine learning, work is ongoing. “I am convinced that all partners will continue their efforts to further improve the applications, software and hardware,” concludes Dueben.
Keywords
MAELSTROM, EuroHPC JU, climate, forecasts, weather, extreme, artificial intelligence, supercomputers, prediction, machine learning, HPC
