Imagine you're organizing a library. The dominant theory for the last few years said neural networks shelve books chaotically — cramming unrelated titles onto the same shelf because they ran out of space, creating a mess you'd need a second librarian to decode. Nick Cammarata's argument is simpler and more radical: the shelves were organized all along. What looked like 'cat faces and car fronts sharing a neuron' was actually a neuron generalizing across visually similar shapes — rounded forms with symmetrical features. The second librarian was never needed. The committed claim: superposition — the idea that neural networks fold unrelated concepts into the same neurons when capacity runs out — was the wrong explanation for polysemanticity. The real mechanism is recursive generalization, where neurons sprout related sub-concepts the way tree branches sprout sub-branches. A neuron detecting 'iy' grows to detect 'diy,' then 'do it yourself.' This is legible structure, not arbitrary packing. If true, it means the entire sparse autoencoder research program — training a second model to 'unfold' the first — is solving a problem that doesn't exist. The evidence spans both vision and language models. On the vision side, Cammarata retrained InceptionV1 with modern architectural improvements (no change to model size or training data) and found 99-100% of last-layer neurons interpretable, up from nearly zero. He trained versions across a 300× size range and found legibility at every scale — smaller models just use coarser-grained concepts (one car neuron instead of many). On the language side, preliminary audits of Qwen3-32B show roughly 90% of neuron poles have one main job, ~7% have two related jobs, and only 2-4% remain unexplained. Similar results held in models up to trillions of parameters, including Kimi K3. The canonical examples that convinced the field turn out to be errors. The famous car-detector-in-dog-neurons circuit from Zoom In (2020) — the founding evidence of superposition — wasn't the network hiding information in a high-dimensional space. Nothing downstream actually read that combination back out. It was ordinary generalization and composition. Cammarata notes that to his knowledge, no one has published a second circuit with the right shape for superposition. The field built its research agenda on a single unreplicated example. The architectural diagnosis is precise. InceptionV1's late layers weren't knotted by superposition — they were failing to learn properly due to architectural limitations. Modern improvements (better normalization, residual connections, training stability) fixed the interpretability problem as a side effect. This reframes the entire interpretability challenge: the obstacle wasn't a fundamental property of neural networks but an engineering bug in older architectures. The stakes are enormous for AI safety. If networks are 'already clean,' the field can return to direct mechanistic study — tracing circuits, watching data flow, running experiments — rather than routing everything through sparse autoencoders. Cammarata explicitly positions this against the SAE research program that Anthropic, OpenAI, and others have invested heavily in. The recursive generalization framing also means explanations can be fractal: start coarse, zoom in where needed. That's a fundamentally more tractable research program than 'extract every feature from a second model.' This is a bold, experience-dense argument from someone who co-authored the original superposition evidence. It's not a formal paper with controlled experiments and p-values — it's closer to a research memoir with embedded demonstrations. The browsable model is real, but the 90% interpretability claim for Qwen3-32B is described as a 'preliminary audit' with acknowledged threshold sensitivity. The strength is the depth of direct observation; the weakness is that much of the validation is the author's own judgment about what counts as 'interpretable.'
