Imagine you're packing a suitcase for a trip, but you don't know where you're going yet. You can pack by rarity — things you can't buy at any destination — or by relevance — things that match the weather forecast. Both strategies sound reasonable. But if the forecast is wrong, relevance packing fills your bag with useless gear. And rare items might just be rare because nobody needs them. The only thing that actually matters is what's in the suitcase when you arrive, not the rule you used to pack it. That's the core mechanism of this paper. The committed claim: in multi-agent LLM discussions compressed into memory for a final decision-maker, neither rarity-based nor relevance-based memory selection reliably preserves the right evidence. Both heuristics look attractive on classical hidden-profile tasks (where shared vs. unshared information is confounded with useful vs. useless), but a controlled benchmark that disentangles ownership from usefulness shows both policies fail predictably. Accuracy tracks the balance of helpful-vs-harmful evidence in the memory, not which selection rule put it there. The authors introduce CrossProfile, a 60-template benchmark (480 tasks) that crosses fact ownership with fact usefulness — supporting, decoy, and irrelevant facts appear in identical text regardless of how many agents hold them. This is the key move. Classic hidden-profile setups confound rarity with helpfulness because supporting facts are always unshared. CrossProfile breaks that confound. Across Qwen2.5-7B, Llama-3.1-8B, and two DeepSeek models, relevance-based selection loses 23–31 accuracy points to rarest-first selection in the classic configuration specifically because decoys are on-topic — a perfect relevance filter still admits them. But rarest-first is no better than random on average, winning only when rare facts happen to be the useful ones. A third failure mode emerges from the decision model itself: social proof. The LLM gives extra weight to facts attributed to many agents, essentially treating popularity as evidence. Removing or randomizing attribution tags — while holding line order fixed — raises accuracy by 13–24 points in the classic configuration. This is not a memory problem; it's a reasoning problem baked into the model's priors about consensus. The most damaging finding may be the simplest: feeding the complete, uncompressed discussion transcript to the decision-maker produces lower accuracy than giving it no notes at all. The natural conversation format itself is the poison. Discussion dynamics — repetition, social signaling, tangential elaboration — actively degrade the evidence before any memory heuristic touches it. The compression step that researchers are trying to optimize may be treating a symptom while the disease is upstream. Methodologically, CrossProfile is a clean diagnostic tool. The automatic answer verification, template-based generation, and systematic crossing of variables make the benchmark reproducible and extensible. The four-model sweep (Qwen2.5-7B, Llama-3.1-8B, DeepSeek-R1-Distill-Qwen-7B, DeepSeek-R1-Distill-Llama-8B) stays within the 7–8B parameter range, which is both a strength (controlled comparison) and a limitation (no signal on whether larger models escape these traps). The paper lands a clean hit on a real problem in multi-agent LLM system design: memory compression is where practitioners are investing effort, and this work shows that effort may be misdirected. The failure isn't in the filter — it's in the evidence landscape the filter operates on, and in the decision model's own biases about social consensus. If you're building agentic systems with shared memory, this paper is telling you your memory module is the wrong thing to optimize.
