The Energy-Based View¶
An energy function \(E_\theta : \mathbb{R}^d \to \mathbb{R}\) defines a probability density
where lower energy means higher probability. The normalizer \(Z_\theta\) is intractable in general, but differences of energies, gradients of energies, and ratios of densities do not depend on it.
For a model parameterized by an energy, two derived quantities do most of the work:
- the score \(s_\theta(x) = \nabla_x \log p_\theta(x) = -\nabla_x E_\theta(x)\), which is what score-matching objectives estimate, and
- the force \(-\nabla_x E_\theta(x)\) (the same vector), which is the drift every gradient-based sampler follows.
Models that output a vector field directly, such as flow-matching velocities and the implicit equilibrium-matching field, have no energy in general; Design and Scope places both kinds of model.
The model contract¶
Any differentiable map from a batch (N, d) to energies (N,) is a valid model. forward returns the energy; the call returns what prediction_type names: the energy (default), the score \(-\nabla_x E\) through autograd, or both from one pass. BaseModel adds device and dtype handling and the integration with every sampler and loss:
A model declares only what its training loss consumes; samplers read the attribute and differentiate an energy themselves.
Architecture guidance is the usual for EBMs: smooth activations (SiLU, GELU) keep the gradient field well behaved, and the output must remain a raw scalar per point (no final nonlinearity). For image-scale conditional models the library ships DiT (with the standard size presets dit_s_2 through dit_xl_8) and its components in torchebm.models.
Analytic energies¶
torchebm.core provides closed-form landscapes used throughout the examples, tests, and benchmarks (e.g. GaussianModel, DoubleWellModel). They make sampler behavior measurable, since means, covariances, and mode structure are known exactly. The shipped set, generated from the installed package:
graph TD
BaseModel(["BaseModel"])
BaseModel --> AckleyModel
BaseModel --> AnalyticModel
BaseModel --> BoothModel
BaseModel --> DoubleWellModel
BaseModel --> GaussianModel
BaseModel --> HarmonicModel
BaseModel --> HimmelblauModel
BaseModel --> RastriginModel
BaseModel --> RosenbrockModel
BaseModel --> StyblinskiTangModel They subclass AnalyticModel, so prediction_type="score" returns the closed-form score without autograd.
Scheduled parameters¶
Hyperparameters that should vary over a run (a step size annealed during sampling, a temperature ramp inside a loss) are Schedulable: any such parameter accepts a float or a BaseScheduler (linear, cosine, exponential-decay, warmup, and temperature schedules among the shipped ones), and the consuming component advances it once per step.