03 · How Models Learn
Synthetic Data
AI-generated training data
Training data generated by AI rather than collected from the world — often used to teach or refine another model when real, high-quality data is scarce.
Concrete example
A strong model writes thousands of worked math solutions, which are then used to train a smaller, cheaper one.
Why it matters
Increasingly central as labs exhaust the easily available supply of human-made text.