Follow the Money

There are two broad compute bills. Training creates or adapts model weights through large batches of computation. Inference runs the resulting model for users. Training may be a concentrated project cost; inference repeats with every request. Which dominates depends on the model, traffic, response length, and serving design.

Unit prices have fallen dramatically. Stanford's 2025 AI Index reported that the cost of querying a model at roughly GPT-3.5-level benchmark performance fell from about $20 per million tokens in November 2022 to $0.07 in October 2024—more than a 280-fold reduction for that comparison. It does not mean every model or workload became 280 times cheaper. 1

Cheaper units can still produce larger total bills. Products use longer contexts, more generated tokens, repeated tool loops, multiple models, retrieval, and media. Watch cost per successful task, not only price per token.

Scaling laws are part of the investment case: under a given training recipe, measures such as loss often improve predictably with more compute, data, and model capacity. Real capabilities are less smooth, and labs also invest in better data, algorithms, post-training, and inference-time computation.

Epoch AI's current trend page estimates the largest known training run, Grok 4, at around 5×10²⁶ FLOP and reports frontier training compute growth of roughly four to five times per year. These are research estimates built from incomplete public information, not audited disclosures from the model developer. 2

Infrastructure figures need labels: measured, disclosed, estimated, projected, or inferred.

The accelerator is only one part of the system. High-bandwidth memory feeds model weights and activations; interconnects move data among chips; storage and networking feed the cluster; cooling and power keep it operating. A bottleneck in any layer lowers throughput.

Model FLOPs utilization compares achieved model-training throughput with a chosen theoretical peak. A 35% MFU result does not mean two-thirds of the hardware is literally doing nothing; the gap can include communication, memory movement, pipeline bubbles, and arithmetic that the metric does not count. It is a useful efficiency signal only when the measurement and workload are comparable.

The spending headline is capital expenditure. Reporting based on first-quarter company guidance projected that Microsoft, Amazon, Alphabet, and Meta could spend up to about $725 billion in 2026, up from roughly $410 billion in 2025. That is a compiled forecast for total company capex heavily influenced by AI infrastructure—not an audited total of AI-only spending. 3

Power and construction can constrain expansion even when capital is available. Data centers need land, grid connections, generation, substations, cooling, equipment, permits, and skilled labor. A announced gigawatt is not the same as a completed, fully utilized facility.

The durable story is not that one number proves a boom or bubble. It is that model capability now depends on an unusually capital-intensive supply chain whose economics must be evaluated across utilization, demand, depreciation, power, and useful output.