Imagine you're riding a bicycle with training wheels welded on. Weight decay and hard norm constraints are those training wheels — they keep you upright, but they also prevent you from leaning into turns. MuonGS is the move where you unbolt the training wheels but keep your center of gravity low by design, so the bike stays stable without clamping you to vertical. The paper's core mechanism is a Gram-Schmidt projection that strips the radial (outward-growing) component from each Muon update, letting weights drift in norm but only slowly — at most √t after t steps — rather than fixing them to a sphere. The committed claim: vanilla Muon accumulates effective learning rate only logarithmically over training, which limits how far the optimizer can explore the loss landscape. MuonGS guarantees Ω(√T) accumulation — strictly faster than log — without requiring bounded-norm control. This is not a new optimizer family; it's a surgical modification to Muon (itself a modification of Shampoo/SOAP-style second-order methods) that relaxes the constraint while preserving the benefit. The theoretical contribution is a formal link between norm-growth rate and accumulated effective learning rate, CT = Σ ηt / ||Wt||F, which the authors show governs how much the optimizer can displace weights relative to their scale. On the ladder: MuonGS is benchmarked against MuonH (Muon with hard norm constraint, itself the current competitive variant) across 160M-parameter and 400M-parameter dense transformers and a 135M-active/1B-total MoE model. The headline result is lower validation perplexity at long training horizons, with an estimated ~15% token-equivalent speedup for the dense model at matched validation loss. The comparison is honest — MuonGS and MuonH are close at short horizons, and the advantage emerges specifically at longer training. AdamW is included as a sanity baseline but is not the main competitor. Notably absent: any comparison to Shampoo or SOAP directly, or to other relaxed-constraint Muon variants that may exist in concurrent work. Architecturally, this lives in the second-order / spectral optimizer family. Muon uses Newton-like steepest-descent directions on the Stiefel manifold of weight matrices, and MuonGS modifies the update geometry by projecting out the radial direction via Gram-Schmidt orthogonalization. The key structural insight is that you don't need to constrain the manifold (keep weights on a sphere) — you just need to control the rate at which updates push weights off it. The compute overhead of the GS projection is negligible relative to Muon's existing SVD step. Integrity is mixed. The theoretical results — √t norm growth bound, Ω(√T) accumulation, O(T^{-1/2}) stationarity for scale-invariant objectives — are proved, not simulated. But the empirical validation is same-team, same-codebase, on models up to 1B total parameters. No pre-registration, no public code release mentioned, no independent replication. The benchmarks (validation perplexity on standard LM pretraining) are reasonable but not externally standardized in the way ImageNet or GLUE once were. The 9 figures across 21 pages suggest thorough ablations, but the core comparison set is narrow. The milestone to watch: MuonGS's advantage grows with training horizon. The paper demonstrates this at 160M-400M dense and 1B MoE scale. The natural next checkpoint is whether the ~15% token-equivalent speedup holds — or grows — at 1B-7B dense scale with 100B+ token training runs. That's the scale where pretraining compute savings translate to real dollars saved. If the advantage compresses or inverts at larger scale, the contribution shrinks to a theoretical curiosity. The obvious experiment not run is scaling to 1B+ dense models with full-length training. The authors likely ran out of compute — the MoE experiment at 1B total parameters with only 135M active is a classic compute-conscious workaround that lets you claim 'billion-scale' without billion-scale dense costs. This is not dishonest, but it leaves the load-bearing question unanswered.
