Apple's machine learning team has demonstrated that a cluster of four Mac Studios with M5 Ultra chips can run Muse Spark Open — an 816-billion-parameter mixture-of-experts model — at over 36 tokens per second. That is above average human reading speed, achieved entirely on desktop hardware with no cloud dependency. The stack is MLX, Apple's open-source array framework, combined with a new RDMA-over-Thunderbolt 5 driver and communication collectives called jaccl. The single-machine story is already significant. A Mac Studio with M5 Ultra (512GB unified memory) runs Qwen 3.8 (27B parameters) and Mistral Medium 3.5 (128B parameters) with time-to-first-token up to 4.6× faster than the previous M3 Ultra generation. The GPU Neural Accelerator — dedicated matrix multiplication hardware — drives that compute-bound improvement directly. Generation throughput, which is memory-bandwidth bound, sees a 1.6× gain from the M5 Ultra's 1.2 TB/s unified memory bandwidth versus the M3 Ultra's 819 GB/s. The distributed story is where the architecture gets interesting. Muse Spark Open at BF16 would need 1.6TB of RAM for weights alone. Quantized to NVFP4, the footprint drops to ~493GB — still too large for one machine with overhead. MLX supports both pipeline parallelization (splitting the model's depth across machines) and tensor parallelization (splitting its width). Pipeline parallelism across two Mac Studios preserves single-machine decode speed while delivering up to 1.8× faster prefill at 65,536-token contexts through chunk overlapping. Tensor parallelism across four machines is what reaches the 36 tok/s headline number. Power consumption tells a quiet but important story. Peak draw is approximately 200W for M3 Ultra and 300W for M5 Ultra — less than a common toaster. A four-machine cluster serving a frontier-class model draws roughly 1.2kW. Compare that to a single NVIDIA H100 SXM at 700W, let alone the eight-GPU nodes typically used for inference at this scale. The energy economics are genuinely different. The strategic logic is transparent: Apple is building an alternative to renting NVIDIA GPU hours from hyperscalers. The unified memory architecture eliminates the CPU-GPU data transfer bottleneck that plagues discrete GPU systems. The RDMA-over-Thunderbolt driver eliminates the need for InfiniBand or specialized networking. The open-source MLX framework eliminates CUDA lock-in. Each layer removes a dependency on a different incumbent. The limitations are equally clear. Four Mac Studios with 512GB each at approximately $10,000 apiece represent a $40,000 capital outlay. That buys meaningful inference capability but not training. The NVFP4 quantization required to fit Muse Spark Open involves precision tradeoffs not discussed in the benchmarks. And the 36 tok/s figure is at 8,192 context — performance at longer contexts with tensor parallelism is not reported in the truncated post. These are engineering demonstrations, not peer-reviewed benchmarks. What matters is the direction: Apple is making local, private, multi-hundred-billion-parameter inference a desktop reality. For enterprises with data sovereignty requirements, for researchers who need to iterate without cloud bills, and for developers building agentic applications that touch sensitive code, this is a genuine new capability. The question is whether the software ecosystem — MLX adoption, model support, tooling maturity — can keep pace with the hardware story Apple is telling.
