You know how your browser has two ways to remember things? There's the cache — a fast, volatile scratchpad that gets wiped — and there's your bookmarks and history, persistent but slow to search. Every time you need an old page, you either hope it's cached or you go dig through your history. Modern LLMs have exactly this problem: their KV cache is the browser cache (fast, expensive, ephemeral), and retrieval-augmented generation (RAG) is the bookmark search (persistent, but with lookup overhead and context-window gymnastics). MemT-1.0's bet is that you can build a third option — memory that lives inside the model's weights themselves, updated token by token, persisting across turns without external storage. The committed claim: foundation models can internalize memory as a first-class computational primitive rather than bolting it on via retrieval, and this approach can match or beat RAG on memory tasks while being dramatically cheaper at inference time. This is not the first linear-recurrence memory model — it builds explicitly on Metis and Gated DeltaNet — but it is arguably the first to push the idea through a full training pipeline (continued pre-training + post-training on 34B tokens) and release a model family with comprehensive benchmarks against both parametric and retrieval-augmented baselines. Architecturally, MemT-1.0 belongs to the linear recurrence / state-space family, specifically using Gated DeltaNet-2 as its backbone. The key structural choice is what they call Mixture-of-Storage (MoS) — an analogue to Mixture-of-Experts but for memory capacity, where different storage 'slots' specialize in different types of persistent information. This is the move that lets them scale native memory without blowing up per-token compute. The model leans on the property that linear recurrence compresses history into a fixed-size state, avoiding the quadratic attention cost that makes long-context Transformers expensive. The two released variants — Nano and Flash — target different points on the size-capability tradeoff. On the ladder, the results are honest but mixed. MemT-1.0 outperforms parametric memory baselines (models that rely only on their trained weights with no retrieval) on average across memory benchmarks. Against RAG methods, it is 'competitive' — the paper's word — which in practice means it wins some benchmarks and loses others. The headline efficiency numbers are striking: at 128K token history, Nano delivers 1.86× end-to-end inference speedup and 66× reduction in persistent storage compared to full KV cache. But the general-capability preservation is described as 'largely' maintained, not fully maintained — a qualifier worth tracking. The agentic task improvements from internalizing procedural experience are demonstrated but not yet at a scale where you'd bet production systems on them. Integrity-wise, the evaluation is broad (66 pages, 15 figures, 16 tables), covering both memory-specific and general benchmarks. They compare against named baselines including RAG approaches and parametric memory models. The obvious concern: this is a self-evaluation by a large team (60+ authors) with no independent replication. The benchmark suite appears chosen after architecture development, not pre-registered. The 'competitive with RAG' framing leaves room for selective emphasis — you'd want to see which specific RAG configurations they tested and whether they used the strongest available retrieval pipelines. The milestone that matters is whether memory-native models can fully replace RAG at production scale while maintaining general capability parity. MemT-1.0 demonstrates viability at the Nano/Flash scale with 34B training tokens. The next number to watch: can this approach hold at 70B+ parameter scale with 500B+ training tokens without the general capability degradation widening? That's probably 12-18 months away given current scaling trajectories. If it holds, you get models that are simultaneously cheaper to serve (no retrieval infrastructure) and better at accumulating experience over time — the 'memory-native evolution' of the title. The experiment they didn't run — and this is telling — is a head-to-head against the strongest production RAG systems (e.g., well-tuned dense retrieval + reranking + Transformer reader) on the hardest long-context reasoning tasks. The honest read: probably (a) and (c). Building and fairly evaluating against a truly optimized RAG pipeline is expensive infrastructure work, and they're likely saving the 'MemT-1.0 vs. best-possible-RAG at scale' showdown for the next paper, once they have larger models. The absence of this comparison is the single biggest gap in an otherwise thorough evaluation.
