Sampling¶
Given a fixed target, how do you draw samples from it? Nothing is learned here: the model is known, and the question is one of dynamics and numerics.
- MCMC: Langevin dynamics and Hamiltonian Monte Carlo, plus the fact that chains are a batch dimension, so thousands of them cost one integer.
- Integrators: the numerical engines inside the samplers. Order of accuracy is measurable, so we measure it against an exact solution.
- Flow: continuous-time generation, where
FlowSamplerintegrates a velocity field as an ODE or an SDE using those same integrators.
Start with Langevin Dynamics 101; the integrator and flow examples show the numerics that both MCMC and generative sampling stand on.
Theory: Sampling and Integration.
Next: Training, where the target is learned rather than given.
| Example | Summary | Level |
|---|---|---|
| Langevin Dynamics 101 | Sample a 2D energy with Langevin; trade step size against noise. | intro |
| HMC 101 | Hamiltonian Monte Carlo on a correlated Gaussian; trade leapfrog steps against acceptance and decorrelation. | intro |
| Parallel Chains | 10,000 vectorized Langevin chains in one call; population statistics without long single-chain runs. | intro |
| Integrator Comparison | Euler, Heun, and RK4 against the exact solution of a harmonic oscillator; measure order of accuracy directly. | intermediate |
| FlowSampler ODE 101 | Integrate a closed-form velocity field from noise to a Gaussian target; study steps vs fidelity without training. | intermediate |