03 · How Models Learn
Scaling Laws
Compute → capability, predictably
Empirical relationships showing that measures such as training loss often improve predictably as model scale, data, and compute increase under a given recipe. Specific capabilities and real-world usefulness do not always improve as smoothly.
Concrete example
Plot training loss against compute under a fixed recipe and the curve can be smooth enough to guide how model builders allocate data, parameters, and hardware.
Why it matters
Scaling laws make large training runs a measured bet, but they do not guarantee that every capability or product improves in lockstep.
Nearby in How Models Learn
Evidence
Sources for this definition
- 1Trends in Artificial Intelligence
Epoch AI · Research · checked 2026-07-13