Diffusion, flow matching and the dynamical-systems view of generative modelling — how to fine-tune and steer a pretrained generator toward a reward, and what it takes to make any of this work for proteins, enzymes and molecular crystals.
Slides developed in part with Alex Tong.
Generative modelling is a coupling problem. GANs, VAEs, normalizing flows, diffusion and flow matching are all different ways to build and train a map ψθ that pushes an easy source distribution p onto the unknown data distribution q. The differences are training signal and inference cost, not the goal.
Flow and diffusion solve the same equation — and straightness is what buys fewer steps. A diffusion SDE and a flow ODE both push a density through the same continuity/Fokker–Planck dynamics. Flow matching regresses directly to a closed-form, single-pair velocity — the entire training loop is x_t = (1-t)x₀ + t·x₁; loss = ‖v(x_t,t) − (x₁−x₀)‖² — so it never needs to simulate an ODE to get a training signal. Because Euler error comes from path curvature, that straight-line path is also why flow matching needs 5–50 sampling steps against diffusion's 10–100+, and consistency/flow-map models push that down to 1–4.
Every alignment method targets the same reward-tilted distribution. Supervised fine-tuning, Best-of-N, DPO, RLHF and inference-time guidance are all ways to sample from p*(x) ∝ pref(x)·exp(R(x)/β) — they only differ in whether they retrain the model, need a differentiable reward, or run online. Retrain when you have a lot of data; steer at inference otherwise.
Classifier-free guidance is a useful hack; it doesn't reach the true target. Reward and classifier(-free) guidance don't actually converge to p*(x). Feynman–Kac steering — sampling N candidates and resampling them by an importance weight at every step, via Sequential Monte Carlo — provably converges to the reward-tilted target as N → ∞, without training anything new.
Joint multimodal generation breaks in practice, not in theory. Sampling sequence and structure from their true joint marginals is exact in theory, but naive joint discretization drifts badly. Fixing it takes explicit inference-time interventions: letting the model self-correct early wrong tokens, annealing its confidence upward over time, and conditioning each modality on a deliberately noisier version of the other.
New-to-nature enzymes no longer need a pre-existing catalytic motif. By generating protein sequence and structure simultaneously, a multimodal diffusion model can build new motifs directly onto a ligand, DNA or RNA target — using one to two orders of magnitude fewer samples than motif-scaffolding pipelines, and producing lab-validated designs (a Zn-metallohydrolase, a retroaldase, a Morita–Baylis–Hillman catalyst).
Crystal structure prediction is moving from search to sampling. Classical CSP enumerates tens of thousands of candidate packings and ranks them with DFT — accurate but computationally prohibitive, and blind to which minima kinetics actually favors. A diffusion model trained on ~600k crystal structures (OXtal) matches DFT-competitive accuracy at orders of magnitude lower inference cost — the first generative model shown to scale to molecular crystals.