Machine-Learning-Based Sampling in Lattice Field Theory and Quantum Chemistry

21 Oct 2024, 08:00 → 25 Oct 2024, 17:00 Europe/Berlin

Wegelerstr. 10 - Seminar Room 2.019 - 53115 Bonn (Bethe Center)

Poster_Funcke_Forum.jpg

Poster_Funcke_Forum.jpg

Poster_Funcke_Forum.pdf

Poster_Funcke_Forum.pdf

Poster_Funcke_Forum.pdf

reisekostenantrag_travel-expense-claim_01072018.pdf

Monday, 21 October

08:45 Registration and Coffee

After registration, all attendees will have some time until the welcome remarks to get to know each other and start interacting.

1 Welcome Remarks

WelcomeRemarks.pdf

2 Normalizing flows for lattice gauge theory in 3+1D

Normalizing flows have emerged over the past several years as a promising avenue to accelerating Monte Carlo sampling for a variety of lattice field theories. This talk will discuss recent progress and ongoing efforts in applying normalizing flows based on gauge-equivariant coupling layers to lattice gauge theories in 3+1d.

Speaker: Dr Gurtej Kanwar

Bonn_Workshop_Oct21_2024_PDF.pdf

10:40 Coffee Break

3 Practical applications of machine-learned flows on gauge fields

Normalizing flows are machine-learned maps between different lattice theories which can be used as components in exact sampling and inference schemes. Ongoing work yields increasingly expressive flows on gauge fields, but it remains an open question how flows can improve lattice QCD at state-of-the-art scales. I discuss progress of two strategies to employ flows for computational advantage. The first is applications of flows in replica exchange (parallel tempering) sampling, aimed at improving topological mixing, which are viable with iterative improvements upon presently available flows. The second is the use of flows to improve signal to noise in Feynman-Hellmann calculations and related approaches.

Speaker: Dr Daniel Hackett

hackett-ml4physchem.pdf

4 Stochastic Normalizing Flows for lattice gauge theory

Non-equilibrium Markov Chain Monte Carlo simulations based on Jarzynski's equality represent a powerful method to compute differences in free energy and also to sample from a target probability distribution without the need to thermalize the system under study.
If the target distribution suffers from long autocorrelation times, they provide a promising candidate to mitigate critical slowing down. These out-of-equilibrium simulations can be naturally combined with Normalizing Flows into a recently-developed architecture called Stochastic Normalizing Flows (SNF). In this talk we first outline our implementation of SNFs in the four-dimensional $SU(3)$ lattice gauge theory and then we focus on their promising scaling with the volume, both in terms of training and sampling. We discuss future systematic improvements and how a mitigation of topological freezing in simulations of lattice gauge theories at large volumes can be realistically achieved in the short term.

Speaker: Dr Alessandro Nada

Nada_Bonn.pdf

12:30 Lunch (On your own)

14:00 Coffee Break

5 Poster Flashtalks (1 min pp)

There will also be few words from our sponsors
- Prof. Dr. Carsten Urbach (NuMeriQS)
- Prof. Dr. Connie Lu and Prof. Dr. Sebastian Neubert (TRA)

15:45 Coffee Break

6 Poster Session

17:00 Reception

Tuesday, 22 October

08:45 Registration and Coffee

7 Keynote: Machine Learning and Physics

In this talk I will give an introduction to generative models for lattice field theory. Following the spirit of the workshop, I will try to give an accessible review of the basics of lattice field theory and discuss its close analogies to large molecule generation. I will also summarize the challenges of the field and the ongoing efforts to overcome them.

Speaker: Dr Pan Kessel

Keynote_bonn.pdf

10:40 Coffee Break

8 A dynamical systems perspective on measure transport and generative modeling

Generative modeling via measure transport can be effectively understood through the lens of dynamical systems that describe the evolution from a prior to the prescribed target measure. Specifically, this involves deterministic or stochastic evolutions described by ODEs or SDEs, respectively, that shall be learned in such a way that the respective process is distributed according to the target measure at terminal time. In this talk, we show that this principled framework naturally leads to underlying PDEs connected to the density evolution of the processes. On the computational side, those PDEs can then be approached via variational approaches, such as BSDEs or PINNs. Using the former, we can draw connections to optimal control theory and recover trajectory-based sampling methods, such as diffusion models or Schrödinger bridges - however, without relying on the concept of time reversal. PINNs, on the other hand, offer the appealing numerical property that no trajectories need to be simulated and no time discretization has to be considered, leading to efficient training and better mode coverage in the sampling task. We investigate different learning strategies (admitting either unique or infinitely many solutions) on multiple high-dimensional multimodal examples.

Speaker: Dr Lorenz Richter

A dynamical systems perspective on measure transport and generative modeling.pdf

9 NETS: A non-equilibrium transport sampler

We propose an algorithm, termed the Non-Equilibrium Transport Sampler (NETS), to sample from unnormalized probability distributions. NETS can be viewed as a variant of annealed importance sampling (AIS) based on Jarzynski's equality, in which the stochastic differential equation used to perform the non-equilibrium sampling is augmented with an additional learned drift term that lowers the impact of the unbiasing weights used in AIS. We show that this drift is the minimizer of a variety of objective functions, which can all be estimated in an unbiased fashion without backpropagating through solutions of the stochastic differential equations governing the sampling. We also prove that some these objectives control the Kullback-Leibler divergence of the estimated distribution from its target. NETS is shown to be unbiased and, in addition, has a tunable diffusion coefficient which can be adjusted post-training to maximize the effective sample size. We demonstrate the efficacy of the method on standard benchmarks, high-dimensional Gaussian mixture distributions, and a model from statistical lattice field theory, for which it surpasses the performances of related work and existing baselines.

Speaker: Dr Michael Albergo

Albergo_technicalTalk.pdf

12:30 Lunch (On your own)

14:00 Coffee Break

10 Panel Discussion: Research in Industry and Academia

Participants:
- Jan Gerken (Chalmers University)
- Pan Kessel (Prescient Design)
- Lorenz Richter (Zuse institute, dida)

15:45 Coffee Break

11 TRA Colloquium: Symmetries in AI4Science

Symmetries are of fundamental importance in all of science and therefore critical for the success of deep learning systems used in this domain.
In this talk, I will give an overview of the different forms in which symmetries appear in physics and chemistry and explain the theoretical background behind equivariant neural networks.
Then, I will discuss common ways of constructing equivariant networks in different settings and contrast manifestly equivariant networks with other techniques for reaching equivariant models.
Finally, I will report on recent results about the symmetry properties of deep ensembles trained with data augmentation.

Speaker: Prof. Jan Gerken

SymmetriesInAI4Science.pdf

19:00 Dinner - 60 seconds to Napoli

Wednesday, 23 October

08:45 Registration and Coffee

12 Neural Thermodynamic Integration: Free Energies from Energy-based Diffusion Models

Thermodynamic integration (TI) offers a rigorous method for estimating free-energy differences by integrating over a sequence of interpolating conformational ensembles. However, TI calculations are computationally expensive and typically limited to coupling a small number of degrees of freedom due to the need to sample numerous intermediate ensembles with sufficient conformational-space overlap. In this work, we propose to perform TI along an alchemical pathway represented by a trainable neural network, which we term Neural TI. Critically, we parametrize a time-dependent Hamiltonian interpolating between the interacting and non-interacting systems, and optimize its gradient using a denoising-diffusion objective. The ability of the resulting energy-based diffusion model to sample all intermediate ensembles allows us to perform TI from a single reference calculation. We apply our method to Lennard-Jones fluids, where we report accurate calculations of the excess chemical potential, demonstrating that Neural TI is capable of coupling hundreds of degrees of freedom at once.

Speaker: Prof. Tristan Bereau

202410_bonn_bereau.pdf

13 Thermodynamic Integration along interpolant ML-potentials

We propose to machine learn an interpolating family of potential functions between two Hamiltonians and to perform thermodynamic integration along it. The approach requires the functional forms of the Hamiltonians and samples from the corresponding Boltzmann distributions. We then define an interpolation between the distributions at the level of samples and optimize a neural network potential to match the corresponding equilibrium potential, i.e. the negative log-likelihood, at every intermediate time-step. Once the alignment between the interpolating samples and potentials is sufficiently accurate, the free energy difference between the two Hamiltonians can be estimated using (neural) thermodynamic integration. We experimentally validate the proposal on the estimation of solvation free energies.

Speaker: Mr Balint Mate

NeuralTI_stochastic interpolants.pdf

10:40 Coffee Break

14 Importance weights distribution in Neural Samplers

Neural samplers in general generate configurations from distribution that only approximates the desired target distribution. The importance weights, which are the quotients of the target probability of the sample to its actual probability, do account for this discrepancy and permit the correction of the distribution either by reweighting the samples or accepting/rejecting them using the Metropolis algorithm. Ideally, we would like those weights to be distributed around one with a small variance. It turns out however that, in the case of poorly trained sampler, this distribution can be long tailed, even to the point of having infinite variance. In my talk I will discuss possible causes of this behavior and its connection with mode collapse.

Speaker: Prof. Piotr Bialas

Bonn_2024_weights.pdf

15 Continuous Normalizing Flows in Lattice Gauge Theories

We have developed continuous normalizing flows for lattice gauge theories, building on Lüscher's perturbative framework of gradient flow and enhancing it through machine learning techniques. This approach offers a highly efficient means of trivializing the theory, as demonstrated in the 2-dimensional SU(3) gauge theory, where our method successfully trivializes the system using only a few hundred parameters. However, trivializing the 4-dimensional theory presents greater challenges due to critical slowing down at large values of $\beta$ and the inherently non-perturbative nature of strong interactions. While full trivialization of the 4-dimensional theory remains out of reach, I will present a promising application of our machine-learned gradient flow: the computation of gradients of physical observables with respect to action parameters. This method circumvents traditional difficulties and demonstrates the potential of machine learning to advance the study of lattice gauge theories, even in complex, higher-dimensional settings.

Speaker: Dr Simone Bacchio

bonn - cnf.pdf

12:30 Lunch (On your own)

14:00 Coffee Break

14:30 Social Activity (TBD)

Options Include:
- Boat tour
- City Walking Tour
- Beethoven Museum
- House of German History

Thursday, 24 October

08:45 Registration and Coffee

16 Keynote: Machine Learning and Generative Models

TBD

Speaker: Dr Michael Albergo

Albergo_keynote.pdf

10:40 Coffee Break

17 Continuous normalizing flows for lattice gauge theories

Continuous normalizing flows are known to be highly expressive and flexible, which allows for the easier incorporation of large symmetries and makes them a powerful tool for sampling in lattice field theories. Building on previous work, I will present a general continuous normalizing flow architecture for matrix Lie groups that are equivariant under group transformations. By applying it to lattice gauge theories in two dimensions as proof-of-principle experiments, I will show that it achieves competitive performance, making it plausibly a promising component in the toolbox for future lattice sampling tasks.

Speaker: Mr Pim de Haan

ml4physchem.pdf

18 Hands on NeuLat: a toolbox for neural samplers in lattice field theory

Normalizing flows have increasingly gained attention as a promising
choice for sampling in lattice field theory. However, there has been a
lack of software packages for applying powerful generative Machine
Learning models such as Normalizing flows specifically for lattice field
theory. To fill this gap, we present NeuLat: a fully customizable
software package that allows researchers in lattice field theory to
harness the recent advances in deep generative learning. In this
hands-on session, we explore how NeuLat can considerably simplify the
application and benchmarking of deep generative models for lattice
quantum field theory.

Speaker: Dr Christopher Anders

neulat.pdf

12:30 Lunch (On your own)

14:00 Coffee Break

19 Panel Discussion: Machine Learning and Physics

Participants:
- Piotr Bialas (Jagellonian University)
- Gurtej Kanwar (University of Edinburgh)
- Alessandro Nada (University of Turin)

15:45 Coffee Break

20 Bethe Colloquium: Generative flow models for lattice field theory

Lattice field theory simulations provide a non-perturbative understanding of strongly interacting field theories, such as quantum chromodynamics (QCD), and for example play an important role in providing inputs for a variety of high-energy experiments. State-of-the-art lattice calculations are, however, limited by the large computational cost of Monte Carlo simulation. Recently, significant progress has been made in applying a class of generative machine learning "flow models" to combat this issue. These generative samplers enable promising practical improvements in Monte Carlo sampling, such as fully parallelized configuration generation. In this talk, I will discuss the progress towards this goal and future prospects of the method.

Speaker: Dr Gurtej Kanwar

Bethe_Colloquium_Oct24_2024_PDF.pdf

Friday, 25 October

08:45 Registration and Coffee

21 Keynote: Machine Learning and Chemistry

TBD

Speaker: Mr Jonas Köhler

keynote_kohler.pdf

10:40 Coffee Break

22 Timewarp: Transferable Acceleration of Molecular Dynamics by Learning Time-Coarsened Dynamics

Timewarp is an enhanced sampling method which uses a normalizing flow as a proposal distribution in a Markov chain Monte Carlo method targeting the Boltzmann distribution. The flow is trained offline on Molecular Dynamics trajectories and learns to make large steps in time (up to one ns). Crucially, Timewarp is transferable between molecular systems: once trained, it generalizes to unseen small peptides (2-4 amino acids) at all-atom resolution, exploring their metastable states and providing wall-clock acceleration of sampling compared to standard Molecular Dynamics.

Speaker: Mr Leon Klein

klein_timewarp.pdf

23 Modeling Biomolecular Structures through Generative Models

Predicting the binding structure of a small molecule ligand to a protein -- a task known as molecular docking -- is critical to biological research and drug design. Framing molecular docking as a generative modeling problem we developed DiffDock, a diffusion model over the non-Euclidean manifold of ligand poses. Empirically, DiffDock was the first ML model to outperform traditional docking methods on blind docking and it has since been integrated in the drug discovery pipelines of many pharma and biotech companies. In this talk I will give an overview of DiffDock as well as recent improvements we made to its generalization across the proteome and its ability to model the protein flexibility.

Speaker: Mr Gabriele Corso

corso_bonn.pdf

12:30 Lunch (On your own)

14:00 Coffee Break

24 Panel Discussion: Machine Learning and Chemistry

Participants:
- Gabriele Corso (MIT)
- Jonas Köhler (cusp.ai)
- Lei Wang (Chinese Academy of Sciences)

15:45 Coffee Break

25 Closing Remarks