Imagine you have a filing cabinet with 128 drawers, each holding a single index card. You can write a new card into any drawer by its number, and you can pull any card back out by that same number. Now imagine someone realized that a specific kind of neural network layer — Gated DeltaNet — works exactly like that filing cabinet. Each of its 128×128 state-matrix rows is a drawer. A one-hot key selects which row to write; a one-hot query reads it back. That is the entire trick. Once you see a transformer layer as addressable memory, you can stop training and start compiling. Nesformer, built by Drew Breunig on top of Divyansh Agrawal's Qwenputer project, does exactly that. It takes the NES's MOS 6502 CPU — all 151 official opcodes, every addressing mode, the flags register, the stack, the 2 KB of work RAM, and up to 32 KB of cartridge program ROM — and compiles the entire execution logic into the weights of a stock Qwen35ForCausalLM checkpoint. Nobody ran backpropagation. A compiler computed every one of the 9.3 million nonzero parameters (out of 466.6M total) across 7 Gated DeltaNet layers. The model then predicts the CPU's execution log one token at a time, greedily, while a small host program handles the rest of the console: drawing the picture, reading the controller, and firing the vblank interrupt. The architecture is deliberate. Seven DeltaNet layers divide the CPU's work into a pipeline: Layer 1 holds the program counter anchor, registers (A, X, Y, SP), and flags. Layer 2 computes PC advancement. Layer 3 fetches the opcode and operand bytes. Layer 4 resolves effective addresses. Layer 5 performs RAM and PRG data reads plus the ALU operations (ADC, SBC, compares, BIT). Layer 6 assembles the result byte, carry, overflow, and flags. Layer 7 emits the next token. An eighth all-zero attention layer exists only because Hugging Face's cache API demands it. The cartridge ROM is stored twice in the DeltaNet state — once for instruction fetch, once for data reads — because the 6502 can read data from the same address space it fetches instructions from. The validation is unusually rigorous for a project of this kind. The reference CPU implementation matches nestest's golden log for 3,348 instructions and the py65 emulator across 5.7 million instructions of game code. The compiled circuit, run through Qwenputer's evaluator with bf16 exactness checks, predicts every reference token across nestest plus 283,610 positions of two games' boot sequences — zero mismatches. The checkpoint running in stock Hugging Face transformers (bf16, CPU) predicts 2,651 positions with no errors. Free-running, the weights generate traces identical to the reference CPU for nestest, 700,000 tokens of Thwaite, and 2,645,885 tokens of Super Mario Bros. Full Game. Frame-by-frame comparison against a conventional emulator shows zero visual differences. Performance is honest about being slow. The numba-compiled engine hits about 1,650 tokens per second on one CPU core. A single NES frame takes 1-6 seconds, so 600 frames (10 seconds of NES time) require roughly 25 minutes. The plain numpy engine manages 210 tokens/s; stock transformers about 10. This is not competitive with any actual NES emulator. That is not the point. The point is a proof of concept for a specific thesis: that the DeltaNet state inside linear-attention transformers can serve as deterministic, compiler-addressable memory, making the transformer a programmable computer rather than a statistical pattern matcher. The 6502 is a stress test — a complex, real-world instruction set with indirect addressing, decimal mode, and DMA — and it passes. The weights zip to 3.3 MB. The checkpoint loads with a single frompretrained call. Any NROM (mapper 0) game runs without rebuilding the weights. The implications point toward a class of hybrid systems where compiled logic and learned behavior coexist in the same weight matrix, with the compiled portion providing guarantees that training cannot. The obvious limitation: mapper 0 only. The NES had dozens of memory mappers for bank-switching larger cartridges. Extending beyond NROM means compiling mapper logic into additional DeltaNet state, which scales the state requirements. The authors acknowledge this cleanly. The deeper question is whether this DeltaNet-as-memory paradigm extends to architectures without gated linear recurrence — standard softmax attention does not have the same addressable-row property, which means this trick is architecture-specific, not a general transformer result.
