For optimal reading, please switch to desktop mode.
StackHPC's team attended the 21st ECMWF HPC Workshop, hosted at the Tecnopolo cluster in Bologna, Italy. The Tecnopolo centre brings together some of the most prominent Eurpoean and Italian research institutions, including ECMWF, INFN, CINECA and ENEA.
Happy 50th Birthday to ECMWF
ECMWF's 21st HPC workshop coincides with a significant 50th anniversary milestone for the organisation, founded in 1975. The organisation is established as a global leader in research into weather forecasting and climate modelling.
Visualising and exploring weather model data in augmented reality
AI and Machine Learning Setting a New Agenda
Findings on early prototypes of ECMWF's AI-powered weather forecasting system, AIFS, were presented two years ago at ECMWF's 20th HPC workshop.
An opening keynote by Mariana Clare on recent work on AI-powered forecasting and modelling
Since then the AIFS model has quickly advanced in accuracy and capability. It doesn't yet match the fine-grained resolution of the numerical weather models. However, by many measures the forecast accuracy of AIFS exceeds the numerical weather model.
Single and ensemble variants of the AIFS model are published on huggingface. The AIFS model can be used within a new python toolkit, Anemoi, co-developed by ECMWF and a number of national meteorological institutions. Anemoi is complemented by earthkit for supporting weather and climate science workflows more widely.
ECMWF is now working on a pan-European Horizon2020 project, WeatherGenerator to create a foundation model trained on five substantial climate datasets, with the objective of producing a model that is a digital twin of the Earth as part of the Extreme Event and Climate programme of Destination Earth.
EuroHPC and the AI Factories
The numerical weather prediction community has always been a community that builds and uses giant machines, in order to model Earth's weather systems with ever greater accuracy and detail. The HPC systems are typically designed around the specific requirements of one or more major applications.
The IT4LIA AI factory, hosted at CINECA, will integrate with CINECA's GAIA OpenStack cloud
Focus has shifted recently in the system requirements needed by the weather and climate community, with increased focus on GPU support, for both numerical weather models and AI models:
- Kathy Yelick (UC Berkeley): Can Climate Science Win the Hardware Lottery?
- Michael Lange (ECMWF): GPU Adaptation of the IFS
- Ji-Sun Kang (KISTI): Perspectives on HPC in Meteorology in hte Era of Heterogeneous Processar Architecture Based Supercomputer
- Stefano Ubbiali (ETHZ): GPU Porting of ECMWF Physical Paramterizations using a High-Level Programming Model
The JARVIS system planned at Jülich combines OpenStack, Kubernetes and a technology stack familiar to StackHPC's clients working at the convergence of HPC, AI and cloud.
The European Weather Cloud
StackHPC is proud to have implemented and to provide support for ECMWF's component of the European Weather Cloud, a federation of OpenStack infrastructures provided by ECMWF and EUMETSAT and harmonised to provide a consistent user experience between both sites.
At the European Weather Cloud User Workshop it was a delight to hear from some of the scientists and end users of the service about their work made possible with the support of the Eurpoean Weather Cloud service.
StackHPC's Poster Presentation
Finally, the StackHPC team were able to present a poster of the team's recent research into the combination of Slurm and Kubernetes, making opportunistic and preemptable use of valuable compute resources in a dynamic fashion. For more developments in this exciting area, watch this space!
Get in Touch
Reach out to us via BlueSky, LinkedIn or directly via our contact page.