One Lens to Rule Them All: A Tutorial Argues Every ML Model Is Secretly an Energy-Based Model

Dai and Du recast GANs, diffusion, BERT, RL, and contrastive learning as special cases of one energy function. Not a new algorithm — a new way to see the ones you already use.

Imagine you're a locksmith with a universal key blank. Every lock you encounter — deadbolt, padlock, car ignition — looks different, but you know they all work by the same mechanism: shaped metal resists a spring until the right profile aligns the pins. If you understand the blank, you understand every lock; swap the bitting pattern and you open a different door. This paper argues that energy-based models are that universal key blank for machine learning. The committed claim: every major ML paradigm — discriminative classifiers, GANs, normalizing flows, diffusion models, autoregressive models, contrastive learning, word2vec, BERT, denoising autoencoders, RL policies, inverse RL, planning, and inference-time verifiers — is an energy-based model that differs only in three design choices: the energy function (what is represented), the inference procedure (how you sample or optimize), and the learning method (how you fit to data). The tutorial doesn't propose a new architecture. It proposes a new organizational principle, and argues that swapping one component while holding the others fixed is a principled recipe for generating new algorithms. The unifying thread is the partition function — the normalizing constant Z that makes $\exp(-E_\theta(x))$ sum to one. Most ML methods exist precisely because computing Z is intractable, and each method is a different strategy for dodging it. Maximum likelihood needs Z and approximates it via sampling. Noise contrastive estimation sidesteps Z by turning density estimation into classification. Score matching differentiates away Z entirely. Adversarial training replaces Z with a discriminator. The tutorial lays out these moves side by side and shows they share what the authors call a "contrastive structure" — every learning signal compares something the model likes against something it should like less. The ladder question is unusual for a tutorial paper. There is no SOTA to beat; the contribution is pedagogical and organizational. The closest comparators are the original Hinton (2002) and LeCun et al. (2006) EBM tutorials, and more recently the explosion of survey papers on diffusion models and score-based generative models (Song & Ermon 2019, Ho et al. 2020). What Dai and Du add is coverage breadth: pulling RL, representation learning, and inference-time computation into the same framework, which prior EBM tutorials did not attempt. Whether this breadth produces genuine insight or merely taxonomic tidiness is the real question the reader should bring to the text. Architecturally, the paper is framework-agnostic — it doesn't propose a specific parameterization but rather classifies existing parameterizations (transformers, U-Nets, MLPs, policy networks) by their energy-inference-learning triple. The most actionable claim is about composition: because energies are scalars, you can add them at inference time to combine independently trained models without retraining. This is the mechanism behind classifier-guided diffusion, and the tutorial argues it generalizes far beyond diffusion. Integrity is the dimension where tutorials always score oddly. There are no experiments, no benchmarks, no pre-registration — by design. The validation is conceptual: does the framework actually unify what it claims to unify, and does it produce correct derivations? The reader must evaluate coherence, not p-values. The authors are credible — Bo Dai (Google DeepMind) has published extensively on EBMs and MCMC, and Yilun Du (MIT/Harvard) is a leading voice in compositional energy models and diffusion representations. But credibility is not proof, and the paper's claims about "suggesting new algorithms" remain promissory until someone builds them. The milestone for a tutorial like this isn't a qubit count — it's adoption. The real test is whether the energy lens becomes a standard pedagogical frame in ML courses and whether the "swap one component" recipe produces algorithms that wouldn't have been found otherwise. Du's earlier work on compositional generation (2023) is one data point; the next would be a novel RL or representation learning method derived explicitly from the framework. The obvious experiment the authors didn't run: take the "swap" recipe, generate three candidate algorithms by substituting components, and benchmark them against existing methods. Honest read: they're saving it — this is the theoretical foundation paper, and the empirical follow-ups are coming.