Model-driven Machine Learning    Home    Research    People    ML Seminar    Github    Internal


We organize a biweekly seminar on machine learning, every second Tuesday at 3pm (GMT+2). We discuss papers on ML, often (but not always) with connections to Earth science, climate and weather and materials science.

The seminar also allows members of the Hamburg machine learning community to connect and present their ongoing work. We meet in person at HZG, but we also welcome remote online participants and stream the meeting live on our YouTube channel.

To get updates about each meeting or suggest a topic, please join our mailing list.

Future Topics

10. TBA 14.07.20

Past Topics

9. “The Best of All Possible Worlds” 30.06.20

We consider the critically important and monstrously difficult problem of tuning climate model parameters to match observations (reviewed in Hourdin et al., 2017).

This process is quite challenging, because:

We discuss several approaches to this problem:

8. “Uncharted History” 16.06.20

We discuss the paper “Artificial intelligence reconstructs missing climate information”, Kadow et al. 2020, Nat. Geosci. pdf code

We are very happy to have the first-author of the paper with us to present the study!

The computer vision field of image inpainting paper uses several techniques to reconstruct broken images, paintings, etc. In recent years, more and more diverse machine learning techniques have boosted the field. A major step was taken by Liu et al. 2018 paper video in using partial convolutions in a CNN. The study shown here will transfer the technology to climate research. The presentation will show the journey of changing and applying the NVIDIA technique to one of the big obstacles in climate research: missing climate information of the past. Therefore a transfer learning approach is set up using climate model data. After evaluating test-suites, a reconstruction of HadCRUT4 - one of the most important climate data sets - is shown and analyzed.

7. “Compressed Pressure”, 02.06.20

The main paper for this session will be Latent Space Physics: Towards Learning the Temporal Evolution of Fluid Flow, Wiewel et al, 2019. Also see their blog post.

We will also briefly discuss a follow-up from Wiewel et al. 2020, and a related paper on generative fluid modelling from the same group, Kim et al. 2019. The latter is nicely summarized in this video.

For those interested in the underlying ML methods, this session will be about autoencoders and sequence-to-sequence models:

6. “Minimalist Chaos”, 19.05.20

We’ll discus the Lorenz `96 model (L96) and its myriad uses. In “Predictability - a problem partly solved”, Edward Lorenz introduced a simple mathematical model exhibiting many of Earth science’s core computational challenges.

Challenging features of L96 include chaotic dynamics, nonlinearity, combination of dissipative and conservative aspects and coupling of vastly differing scales in space and time. Chaos means that small perturbations in the model state due to numerical errors or observation noise will, over time, lead to large deviations in the future model state.

L96 is a frequent test case for algorithms tackling many fundamental problems. We consider two of these: parameter tuning, and parameterizing sub-grid processes:

Finally, we’ll revisit the original paper and the issue of predictability, nearly 25 years later.

5. “Real Fake Clouds” 05.05.20

We discuss the paper “Modeling Cloud Reflectance Fields using Conditional Generative Adversarial Networks,” Schmidt et al. 2020, arXiv. pdf code

This paper uses generative adversarial networks, or GANs. In the GAN framework, a generator network learns to generate “fake” data points while a second discriminator network learns to tell real from fake data. Schmidt et al. use GANs to predict cloud reflectance fields from meteorological variables such as temperature and wind speed. Given these meteorological variables, it can produce multiple realistic output patterns instead of an ensemble average. That is, the network attempts to learn the conditional probability distribution of reflectance given the input variables.

We start with a very brief introduction of GANs. More background can be found in Diego Gomez Mosquera’s high accessible blog post or Ian Goodfellow’s extensive tutorial.

Importantly, this paper wasn’t able to get good results just by applying the GAN framework out of the box, and had to use some of the latest specialized tricks as well. So we’ll briefly go through some of these tricks:

Additional links from the discussion: Variational Dropout and and the Local Reparameterization Trick pdf Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning pdf

4. “Far into the Future”, 21.04.20

Lennard Schmidt from UFZ present on his work. He applies machine learning to do quality control for hydrological measurement data. He also uses a sophisticated convLSTM architecture to predict hydrological dynamics in an Elbe catchment basin. Code for a convLSTM layer in tensorflow/keras can be found here.

Eduardo Zorita presents “Deep learning for multi-year ENSO forecasts,” Ham et al. 2019, Nature. link This paper uses machine learning algorithms to predict the El Niño/Southern Oscillation 1.5 years into the future, farther than previous methods have achieved. Notably, it trains on a combination of simulations and historical data.

Additional references on the predictability paradox in climate science: “Do seasonal‐to‐decadal climate predictions underestimate the predictability of the real world?” Eade et al. 2014, Geophys. Research Letters. link

“Skilful predictions of the winter North Atlantic Oscillation one year ahead.” Dunstone et al. 2016, Nature. link

3. “MetNet, Convolutional-Recurrent Nets, and the Self-Attention Principle” 07.04.20

Linda von Garderen presents on her work.

We cover Google Research’s recent work on weather prediction: “MetNet: A Neural Weather Model for Precipitation Forecasting,” Sønderby et al., 2020, arXiv. paper, blog post

To understand the ML tools that went into this work, we briefly review some concepts from earlier works:

With these concepts in mind, we examine how MetNet combines them, and consider their results from the perspectives of both ML and weather prediction.

Relevant discussion links:

2. “Don’t Fear the Sphere” 31.03.20

We cover “Spherical CNNs on Unstructured Grids,” Jiang et al. 2019, ICLR. We also survey other ML approaches to spherical data (more links in the description on YouTube). With 5 minute presentations by Julianna Carvalho, Tobias Finn and Lennart Marien.

1. “Hidden Fluid Mechanics” 24.03.20

We discuss the paper “Hidden fluid mechanics: Learning velocity and pressure fields from flow visualizations,” Raissi et al. 2020, Science, and the more technical study from the same group, “Physics Informed Neural Networks,” Raissi et al., 2019, J. Computational Physics. Tobias Weigel from DKRZ explains the ML support team that forms part of the local Helmholtz AI unit.