Imagine you're training a sommelier. You could pour them Cabernet Sauvignon a thousand times, or you could make sure they taste one glass each of a hundred obscure varietals — Trousseau, Nerello Mascalese, Blaufränkisch. The repeated Cabernet teaches fluency. The one-shot tastings teach generalization: the ability to recognize a wine they've never encountered. This paper argues that LLMs work the same way. The facts a model sees exactly once during training — "monofacts" — are the critical ingredient for scientific discovery, not the volume of repeated facts or the raw diversity of the corpus. The committed claim: monofact rate in training data is a causal, manipulable lever for the scientific discovery capacity of language models. The authors develop a theoretical framing rooted in missing probability mass — the model's ability to assign nonzero probability to unseen combinations depends on how much of its training distribution was already rare. They then intervene directly: by surgically varying the proportion of monofacts in training sets for materials science and regulatory genomics tasks, they show that higher monofact rates consistently produce more discoveries. This holds across two open-weight model families, multiple sampling strategies, and different reasoning budgets. The ladder here is unusual because the paper isn't competing against a prior SOTA on a benchmark — it's proposing a new explanatory variable. The closest prior art is work on data diversity, memorization, and scaling laws (Chinchilla, D4, etc.), but those frameworks don't isolate the "seen once" regime. The authors explicitly test and reject the simpler hypothesis that training-set diversity alone explains their results. That's a meaningful separation. Architecturally, this is a controlled training-data ablation study, not a new model architecture. The authors use two open-weight model families (not named in the abstract, but the experimental design implies mid-scale causal LMs), train or fine-tune them on corpora with manipulated monofact rates, and evaluate downstream on scientific discovery tasks — specifically generating novel materials compositions and identifying regulatory genomic elements. The method leans on the ability to precisely control corpus composition, which is a data-engineering problem more than a compute problem. Integrity is where this gets interesting. The authors do the right thing by running interventionist experiments — they don't just correlate monofact rate with performance post-hoc, they manipulate it and measure the effect. They test across two scientific domains and two model families, which is above-average for an NLP paper making a training-data claim. The cross-domain transfer result (monofacts in one domain help discovery in another when paired with in-domain data) adds robustness. The main integrity question is whether the "discovery" evaluation metric itself is well-calibrated — how do you know the model genuinely discovered something vs. recombined memorized fragments in a way that looks novel? The abstract doesn't detail the evaluation protocol for what counts as a discovery. The milestone implied by this work is practical: if monofact rate is a design lever, then corpus curation for scientific LLMs should shift from "more data" or "more diverse data" to "more rare facts." The concrete next number would be something like: what monofact rate threshold maximizes discovery per training token? The paper presumably characterizes the relationship's shape, but the abstract doesn't give the number. The obvious experiment not run: scaling this to frontier-scale models (70B+ parameters) with web-scale corpora where you can't precisely control monofact rates. The authors likely used mid-scale models precisely because corpus control is tractable there. Whether the monofact effect survives when you can't surgically ablate training data — when monofacts are an emergent property of web scraping rather than an engineered variable — is the question that matters for OpenAI, Anthropic, and Google. My read: they ran out of compute for frontier-scale experiments, not that they tried and it failed.
