Training¶
The payoff: learn a target from data instead of sampling one you already have. Ordered from the most classical objective to the most recent.
- MCMC-based losses: contrastive divergence and its persistent variant, which buy a calibrated energy at the cost of sampling inside the training loop.
- Score matching: fit the score directly and drop the inner sampler entirely.
- Equilibrium matching: learn a time-invariant field, then generate by integrating it or by descending it as an energy.
- Energy matching: a single time-independent potential serving both transport and Boltzmann sampling.
- Couplings: which noise sample is paired with which datum, which decides how straight the transport paths are, and so how few steps generation needs.
Start with CD-k on Two Moons, then Equilibrium Matching in 2D to watch one model act as both a flow and an energy.
Theory: Learning Objectives for the losses, and Interpolants and Couplings for the paths and pairings they consume.
Next: Showcase.
| Example | Summary | Level |
|---|---|---|
| CD-k on Two Moons | Train an energy network with Contrastive Divergence and Langevin negatives. | intermediate |
| Persistent CD | CD with a replay buffer: negatives resume across updates and explore beyond k Langevin steps. | intermediate |
| Denoising Score Matching | Train an energy on two moons with no MCMC: perturb, match the denoising score, then sample the result with Langevin. | intermediate |
| Equilibrium Matching in 2D | Train a generative vector field with EquilibriumMatchingLoss and sample it with an ODE. | intermediate |
| Energy Matching in 2D | Reproduce the paper's two-moons experiment: a scalar potential trained with OT flow warm-up plus contrastive sharpening, sampled in one temperature-scheduled SDE sweep. | intermediate |
| Energy Matching: the paper's 2D suite | Reproduce the paper's toy experiments end to end: 8 Gaussians transported onto two moons, with trajectories, sample evolution, LID estimation, and repulsive diverse sampling. | advanced |
| Coupling Comparison | Independent, greedy, Sinkhorn, and exact OT pairings on the same batch; transport cost and per-pair weights. | intermediate |