You know how your browser gets sluggish after a hundred tabs? Each one made sense when you opened it, but collectively they consume all your RAM, and eventually the machine is spending more effort managing its own state than doing useful work. That's the core mechanism here: LLM agents tasked with open-ended discovery accumulate history — past attempts, session context, candidate lists — until the weight of what they remember prevents them from exploring anything genuinely new. The fix is the same one you already know: close everything and start fresh from the best thing you had. The committed claim: periodic "forgetting" — wiping agent session state, communication history, and candidate logs, then restarting from only the best solution found so far — breaks through plateaus that more compute, more parallelism, and inter-agent communication cannot breach alone. This isn't a new architecture or a new model. It's a scheduling discipline layered on top of existing multi-agent setups. The results are concrete across four tasks spanning heuristic programming, kernel optimization, and model training: up to 15% improvement over parallel communicating agents, and up to 39% over a single iterating agent. The mechanism is simple — agents become anchored to their own edit history, making progressively smaller and fewer changes. Forgetting de-anchors them. The paper frames three scaling strategies (longer runs, parallel agents, parallel agents with communication) and shows all three eventually plateau for the same reason: accumulated state. Architecturally, this belongs to the family of search-with-restarts — a classical optimization idea (random restarts, simulated annealing reheating) applied to LLM agent workflows. The insight is that LLMs, despite their flexibility, exhibit the same anchoring pathology that gradient optimizers show near local minima. The team implements forgetting inside a multi-agent framework where agents communicate and share candidates, then periodically nuke the shared state. The compute property being exploited is that LLM inference is cheap enough to re-derive context from a single good candidate, rather than maintaining an ever-growing search tree. Integrity is mixed. Four tasks is a reasonable spread, but all evaluation is internal — the authors designed the tasks, ran the agents, and scored the results. There's no community benchmark for "open-ended LLM discovery" yet, which is both an honest limitation of the field and a weakness of this specific paper. The baselines (single agent, parallel agents, parallel agents with communication) are fair comparisons within the paper's own framework, but there's no comparison to external agent systems like those from Sakana AI, Microsoft Research, or similar automated-research pipelines. The milestone question is where this gets interesting. The paper doesn't give us a crisp "next number" — it gestures at a scaling law for forgetting frequency vs. budget allocation. The real unlock would be an adaptive forgetting schedule that decides when to wipe based on search-progress signals rather than a fixed period. That's conspicuously absent, and my read is (c): they're saving it for the next paper, because it's the obvious follow-on and the current fixed-schedule results already make a clean story. What this actually teaches you: for any agentic system doing open-ended search, memory management is a first-class design decision, not an afterthought. The default in most agent frameworks is "accumulate everything" because context feels like information. This paper argues context is also drag. The broader implication for automated research is that the architecture of forgetting — what to keep, what to discard, when to restart — may matter as much as the architecture of reasoning.
