A developer running a local LLM server on an M4 Pro Mac mini with 48GB of unified memory has documented a setup that handles agent backends, coding assistants, and mobile chat queries — all without touching a cloud API for roughly 80% of daily tasks. The stack centers on two models: a Qwen 3.6 35B mixture-of-experts model (only 3B parameters active per token, ~20GB in RAM) for reasoning-heavy work, and a 2.4GB Gemma 4 model for lightweight tasks. Tailscale meshes the mini with a MacBook and iPhone. Total setup time: 30 minutes. The economics are straightforward. The author was spending $400/month across two API subscriptions and hitting usage limits. The Mac mini costs roughly $800-1,600 once, draws minimal power, and every inference after purchase is free. At current subscription rates, the hardware pays for itself in two to four months. More importantly, the cost curve is flat — usage spikes don't generate surprise bills. Performance numbers are concrete: 325 tokens per second for prompt processing, 34 tokens per second for generation, powered by the M4 Pro's 273 GB/s unified memory bandwidth. The author notes this sits between the M3 Pro (200 GB/s) and M3 Max (400 GB/s). For context, 34 tok/s generation is faster than most people read. The 4-bit quantization from OptiQ loses only 1-2 benchmark points versus the uncompressed BF16 baseline — an acceptable tradeoff that shrinks memory requirements from ~70GB to ~20GB. The mixture-of-experts architecture is what makes this viable on consumer hardware. A dense 27B model needs ~14GB in 4-bit, leaving almost nothing for a 16GB machine. The Qwen 35B MoE model stores 35B parameters across 256 experts but activates only 3B per token, making its effective memory footprint comparable to a 6B dense model during inference. This is not a workaround — it is the architectural trend that makes local deployment practical. The sovereignty argument is the most structurally interesting. The author cites three risks of cloud dependence: pricing changes and model swaps without notice, data privacy exposure that cannot be undone, and government-imposed model restrictions that can halt workflows overnight. These are not hypothetical — US export controls have already limited model availability by geography, and multiple providers have modified model behavior between versions without user notification. The model-swapping workflow reveals something about the maturity of the local ecosystem. New models drop, the user downloads them into a directory, the oMLX server auto-discovers them, and the stack is updated in minutes. The KV cache persistence feature — caching context blocks to SSD so returning to earlier conversation context takes milliseconds instead of full recomputation — makes agent workflows genuinely practical rather than technically possible but painful. The author has already ordered a 128GB M5 Max Mac Studio, signaling that this is not a hobby experiment but a scaling infrastructure investment. The gap between local and cloud models is closing fast enough that the 80/20 split — local for routine work, cloud APIs for frontier capability — is becoming the rational default for anyone doing volume inference. The question is not whether local compute is good enough. It is whether cloud providers can justify their margins as the floor keeps rising.
