Legacy Concept Lab

Diffusion, Score-Based Models & Flow Matching

Stable Diffusion: latent diffusion — DDPM in a VAE latent space

Concept 9 of 100Generative ModelsPhase 4

Open New Notebook All Foundations

#9DiffusionGenerative Models

key equation

\mathcal L = \mathbb E_{x_0,t,\epsilon} \big\|\epsilon - \epsilon_\theta(x_t,t)\big\|^2

Phase 4: Generative modeling familiesConcept 9 of 100

Why It Matters for Modern Models

Stable Diffusion: latent diffusion — DDPM in a VAE latent space
Sora: diffusion transformer over 3D spacetime patches
Flow-matching and rectified flows enable one-step or few-step generation

What Tutorials Skip

What is still poorly explained in textbooks and papers:

Intuitive explanation that denoising is learning ∇ₓ log pₜ(x) (scores), and how reverse-time SDE sampling corresponds to "walking uphill in log-density space"
Visual/interactive demonstrations of different probability paths (diffusion vs optimal transport)

Interactive Visualization

Core Math (Optional Deep Dive)

If you want intuition first, start with the key equation and the visualization. Come back here for the full walkthrough.

Key Equation

\mathcal L = \mathbb E_{x_0,t,\epsilon} \big\|\epsilon - \epsilon_\theta(x_t,t)\big\|^2

Forward diffusion adds noise:

q(x_t \mid x_0) = \mathcal N(\sqrt{\bar\alpha_t}\,x_0, (1-\bar\alpha_t)I)

Model learns to predict noise $\epsilon$ via MSE:

\mathcal L = \mathbb E_{x_0,t,\epsilon} \big\|\epsilon - \epsilon_\theta(x_t,t)\big\|^2

Score-based SDE view: forward SDE $dx_t = f(x_t,t)\,dt + g(t)\,dW_t$ . Reverse-time SDE uses score $\nabla_x \log p_t(x)$ .

Flow matching: train vector field $v_\theta(x,t)$ to match the "true" conditional field (often optimal transport / straight lines).

Canonical Papers

Deep Unsupervised Learning using Nonequilibrium Thermodynamics

Sohl-Dickstein et al.2015ICML

Denoising Diffusion Probabilistic Models

Ho et al.2020NeurIPS

Score-Based Generative Modeling through SDEs

Song et al.2021ICLR

Flow Matching for Generative Modeling

Lipman et al.2023ICLR

Connections

Prerequisites

ℒML/CE/KL ℤVAEs

Enables

⚡Efficiency 🖼️Multimodal VLP 🎲Decoding ∇pScore Matching ⚖️OT/Wasserstein 🎚️CFG 🎯Consistency 🌊Flow Match 🎬VideoWM

Next Moves

Explore this concept from different angles — like a mathematician would.

Semantic Connections

≈ Analogy

🔀Iterative → Recurrent→SSMs & Hybrids 🎲Guidance knob→Decoding

↔️ Mathematical Dual

ℒDenoising ↔ ELBO→ML/CE/KL ℤScore ↔ ELBO→VAEs ⚔Adversarial ↔ Denoising→GANs

🔧 Invented to Fix

⚡Accelerate sampling→Efficiency ℤBlurry samples→VAEs ⚔Fix mode collapse→GANs ℤFix blurry outputs→VAEs

🔄 Same Technique

⊗Cross-attention→Attention ℤReparameterized noise→VAEs 🖼️Text-conditional generation→Multimodal VLP