09 · Hardware & Infrastructure
NVLink / InfiniBand
High-speed GPU interconnects
The wiring that lets thousands of GPUs in a cluster talk fast. Training a frontier model isn't one chip — it's tens of thousands working in concert, and the interconnect decides whether they cooperate efficiently or choke on overhead.
Concrete example
Poor interconnect can leave expensive GPUs idle, waiting on each other — directly dragging down MFU.
Why it matters
Often the hidden bottleneck that decides whether a giant cluster runs efficiently.