The cost of machine intelligence is in freefall, and the evidence is stacking up across every layer of the stack simultaneously. GPU power efficiency is doubling roughly every two years — a logarithmic improvement rate not seen since Moore's Law in the 1960s. Model per-task costs (not per-token, which is a crucial distinction) dropped two orders of magnitude between 2025 and 2026, with the pareto frontier of cost-versus-quality shifting dramatically leftward. Inference engines like vLLM delivered 40% efficiency gains in 15 months, while NVIDIA's MLPerf stack showed up to 50% improvement between point releases. The architectural innovations compound the hardware gains. Mixture-of-Experts architectures achieve equivalent benchmark performance with models 7x smaller (6B down to 0.8B active parameters). Mamba-Transformer hybrids slash memory requirements — Nemotron-H-47B holds over a million tokens in 32 GB of VRAM where a comparable Llama-3.1 60B would need 120 GB. These aren't theoretical improvements; they directly determine what runs locally on commodity hardware versus requiring data center access. The most striking data point is at the specialized end. TypeSafe AI's Jev, a classifier that returns probabilities rather than generating text, prices input tokens at $0.042 per million — roughly $42 per billion tokens — with output tokens effectively free. At that rate, classifying five books costs three cents. The jgrep tool built on top of it returns results in 200 milliseconds for a thousandth of a cent, cheap and fast enough to sit in a Unix pipe alongside grep and awk. The structural story here is that AI is transitioning from product to infrastructure. When inference is cheap enough to embed in every pipe, every query, every CI check, the bottleneck shifts from cost to quality, data access, and integration. This is the pattern we saw with compute (cloud), storage (S3), and bandwidth (CDN) — the commodity layer disappears into the stack and the value migrates upward to what you do with it. The piece correctly distinguishes between different improvement vectors: GPU hardware affects everything, MoE helps hosted models more than local ones (experts still need to be in memory), and Mamba helps local inference disproportionately. These aren't interchangeable gains — they compound differently depending on deployment context. The per-token price of frontier models isn't consistently falling, but the per-task cost is, because smaller cheaper models can now accomplish what only large expensive models could a year ago. What the analysis doesn't fully address is who captures the surplus. When inference becomes near-free, the value concentrates in whoever controls the training runs, the proprietary data, and the integration points. The GPU efficiency gains benefit NVIDIA's monopoly position. The inference cost collapse benefits hyperscalers who can amortize fixed costs across massive scale. Open-weight models and local inference are genuine counterweights, but the capital requirements for frontier training remain enormous and rising. The 3-6 year timeline for frontier-quality local inference on commodity hardware is plausible given the compounding of GPU efficiency, architectural innovation, and quantization techniques — but it assumes continued exponential improvement in all three simultaneously, which historically is the point where one bottleneck or another reasserts itself.
