NVIDIA's NVL72 architecture connects all 72 GPUs in a rack via a unified NVLink fabric with 130 TB/s aggregate bandwidth, eliminating the historical bottleneck that forced multi-node jobs onto InfiniBand. The performance stakes are not abstract: Hugging Face measured a 43% drop in training throughput when tensor parallelism moved off NVLink onto InfiniBand. For MoE models with 64-way expert parallelism, unoptimized inter-node communication can consume up to 60% of training time. NVL72 keeps all 64 GPUs within a single NVLink domain. The catch is that making the rack the atomic unit of compute dramatically increases the fault blast radius. Meta measured a mean time between interruptions of roughly 490 days per 4-GPU node on its supercluster. An 18-node NVL72 rack divides that by 18, yielding an interruption roughly every 27 days per rack. On a 128-rack cluster, that means a rack becomes unusable for full-rack jobs approximately every 5 hours. The core insight from SkyPilot's operational data is counterintuitive: you get more usable capacity by deliberately leaving GPUs idle. At the 2% node unavailability rate measured on production NVL72 clusters, 16-node slices (reserving 2 spares per rack) deliver 2,037 usable nodes versus 1,602 for 18-node slices — a 27% improvement. The two spare nodes absorb the first two failures in any rack without taking the entire rack offline. The 16-node slice also beats 9-node slices by 4%, despite the seemingly tighter packing of two 9-node jobs per rack. With 9-node slices, every single failure costs half a rack; with 16-node slices, failures are absorbed by the spare capacity. Only when unavailability exceeds approximately 3.3% do 8-node slices begin to outperform. Sixteen nodes also maps cleanly to 64 GPUs — a power of two that matches standard parallelism group sizing. SkyPilot's platform implements this through topology-aware placement that keeps multi-node jobs rack-local, in-place recovery from hardware failures using spare nodes, and automatic defragmentation of shared clusters. The defragmentation problem is real: as jobs finish at different times and failures scatter free capacity across racks, naive scheduling steadily erodes rack-locality. This is fundamentally an operations paper dressed as an architecture post. The NVL72 hardware is NVIDIA's; the scheduling intelligence is where SkyPilot adds value. The argument is that the right scheduler can extract significantly more useful work from identical hardware — a claim backed by NVIDIA's own modeling and SkyPilot's production telemetry. The 27% figure is the headline, but the deeper point is that GPU cluster management is becoming a first-class engineering discipline, not an afterthought. The implications extend beyond any single vendor. As rack-scale interconnects become standard, every large training operator faces the same scheduling problem: how to maintain locality as hardware degrades. The operators who solve defragmentation and topology-aware placement will extract materially more training throughput from the same capital expenditure. Those who don't will leave roughly a quarter of their purchased compute stranded.
