03 · How Models Learn
Distillation
Teacher → student compression
Training a smaller “student” model to mimic a larger “teacher.” You get most of the capability at a fraction of the size and cost — how many fast, cheap production models are made.
Concrete example
A giant flagship model teaches a small fast one, which is what actually ships in the phone app.
Why it matters
How labs turn an expensive frontier model into something cheap enough to serve at scale.