ML for High-Performance Climate and Earth Virtualization Engines

Опубликовано 27 августа 2024, 16:12

Speaker: Torsten Hoefler
Host: Karin Strauss

Machine learning presents a great opportunity for Climate simulation and research. We will discuss some ideas from the Earth Virtualization Engines summit in Berlin and several research results ranging from ensemble prediction and bias correction of simulation output, extreme compression of high-resolution data, and a vision towards affordable km-scale ensemble simulations. We will also discuss programming framework research to improve simulation performance. Specifically, our ensemble spread prediction and bias correction network applied to global data, achieves a relative improvement in ensemble forecast skill (CRPS) of over 14%. Furthermore, we demonstrate that the improvement is larger for extreme weather events on select case studies. We also show that our post-processing can use fewer trajectories to achieve comparable results to the full ensemble. Our ML-based compression method achieves data reduction from 300x to more than 3,000x and outperforms the state-of-the-art compressor SZ3 in terms of weighted RMSE and MAE. It can faithfully preserve important large scale atmosphere structures and does not introduce artifacts. When using the resulting neural network as a 790x compressed data loader to train the WeatherBench forecasting model, its RMSE increases by less than 2%. The three orders of magnitude compression democratizes access to high-resolution climate data and enables numerous new research directions. We will close by discussing ongoing research directions and opportunities for using machine learning for ensemble simulations and combine several machine learning techniques. All those methods will contribute to enabling km-scale global climate simulations.

See more at microsoft.com/en-us/research/v...