02 · Model Architectures
Transformer
The architecture behind modern AI
The dominant architecture for modern language models. Its attention mechanism lets the network weigh relationships among tokens efficiently, and its design trains in parallel better than older recurrent approaches. Other architectures and hybrids still exist.
Concrete example
Attention helps connect “it” with the most relevant earlier noun in a sentence, while other layers transform that contextual representation.
Why it matters
The 2017 paper “Attention Is All You Need” introduced the Transformer architecture that underpins most modern LLMs.
Nearby in Model Architectures
Evidence
Sources for this definition
- 1Attention Is All You Need
Vaswani et al. · Research · checked 2026-07-13