Training¶
These examples learn a target from data, ordered from the most classical objective to the most recent.
- MCMC-based losses: contrastive divergence and its persistent variant, which fit a calibrated energy at the cost of sampling inside the training loop.
- Score matching: fit the score directly, with no sampler in the training loop.
- Equilibrium matching: learn a time-invariant field, then generate by integrating it as an ODE or by iterating it toward its attractors; the explicit variant learns a scalar energy.
- Energy matching: a single time-independent potential serving both transport and Boltzmann sampling.
- Couplings: which noise sample is paired with which datum, which decides how straight the transport paths are, and so how few steps generation needs.
- Flow maps: two-time models that jump from noise to data in one call, trained here with the self-consistency loss of shortcut models and with the MeanFlow identity.
Start with CD-k on Two Moons, then Equilibrium Matching in 2D to sample one time-invariant field two ways and an explicit energy with repulsion.
Theory: Learning Objectives for the losses, and Interpolants and Couplings for the paths and pairings they consume.
Next: Showcase.
| Example | Summary | Level |
|---|---|---|
| CD-k on Two Moons | Train an energy network with Contrastive Divergence and Langevin negatives. | intermediate |
| Persistent CD | CD with a replay buffer: negatives resume across updates and explore beyond k Langevin steps. | intermediate |
| Denoising Score Matching | Train an energy on two moons with no MCMC: perturb, match the denoising score, then sample the result with Langevin. | intermediate |
| Equilibrium Matching in 2D | Train a time-invariant field and an explicit potential with EquilibriumMatchingLoss; sample the field by ODE integration and by fixed-point iteration, and the potential by descent with optional repulsion. | intermediate |
| Flow Matching in 2D | Train a time-conditioned velocity field with FlowMatchingLoss and integrate it noise-to-data with FlowSampler, comparing the adaLN-Zero and concat conditioning of TimeConditionedMLP. | beginner |
| Energy Matching in 2D | Reproduce the paper's two-moons experiment: a scalar potential trained with OT flow warm-up plus contrastive sharpening, sampled in one temperature-scheduled SDE sweep. | intermediate |
| Shortcut Models in 2D | Train a two-time average-velocity model with ShortcutLoss and FlowMatchingLoss as in the reference code, then sample in one, two, four or 128 jumps with FlowMapSampler. | intermediate |
| Energy Matching: the paper's 2D suite | Reproduce the paper's toy experiments end to end: 8 Gaussians transported onto two moons, with trajectories, sample evolution, LID estimation, and repulsive diverse sampling. | advanced |
| MeanFlow in 2D | Train a two-time average-velocity model with MeanFlowLoss, using torch.func.jvp for the time derivative, then sample in one, two or four jumps with FlowMapSampler. | intermediate |
| Coupling Comparison | Independent, greedy, Sinkhorn, and exact OT pairings on the same batch; transport cost and per-pair weights. | intermediate |