Imagine a murder mystery dinner party where each guest holds one clue card. No single person can solve the case alone — the answer only emerges when everyone pools exactly the right cards at the right time. Now imagine the guests are GPT-4 and Claude, and instead of clues, they hold medical records, legal filings, or code diffs. TeamGym asks: can AI agents actually do this? The answer, so far, is a pretty clear no. The paper's committed claim: distributing decision-critical facts across multiple LLM agents produces a 22.9 percentage-point drop in success rate compared to giving one agent all the same facts. That's not a marginal degradation — it's a structural failure of multi-agent communication. On top of that, when agents must extract their own pivotal information from raw source documents rather than receiving pre-packaged facts, performance drops another 19.9 points. The total gap from ideal (one model, clean facts) to realistic (team, raw sources) is roughly 43 points. The key methodological contribution is "pivotal information" — a formalism for identifying the minimal set of facts where each element is individually necessary and collectively sufficient for a correct decision. This is what lets TeamGym do something most multi-agent benchmarks cannot: precisely control who knows what, and measure whether agents successfully surface and integrate the right pieces. It's constructed from 12 real-world corpora spanning law, finance, medicine, and software engineering, which gives the benchmark ecological validity that toy coordination games lack. Eight language models were tested. The communication gap (22.9 pp) and retrieval gap (19.9 pp) are remarkably consistent across models and domains, suggesting this isn't a quirk of one architecture or one task type — it's a structural limitation of how current LLMs handle distributed information in conversational settings. The strongest models still fail at the same qualitative task: deciding what information is worth saying out loud to a teammate. The RL training results are the most interesting finding. When the authors train policies using reinforcement learning on team decision loss — rewarding only the final answer, not intermediate communication quality — agents spontaneously develop coordination behaviors: they learn when to speak, whom to address, and what to share. This emergent structure wasn't engineered; it fell out of the reward signal. Two policy-training experiments showed improved final-decision accuracy, though the paper doesn't quantify the absolute recovery against the 43-point deficit, which is a notable omission. The architecture sits squarely in the LLM-as-agent paradigm with RL fine-tuning on top — think RLHF but for multi-agent coordination rather than human preference alignment. The environment is text-in, text-out: agents communicate through natural language messages within a turn-based protocol. This is computationally cheaper than approaches requiring explicit graph-based communication channels, but it inherits all the opacity of language-based coordination. You can see that agents coordinate better after training, but the mechanism by which they do so remains a black box. The validation is internal — the authors designed the benchmark, ran the models, and reported the results. No independent replication exists, and the benchmark itself is new, so there's no community track record. The 12-corpus breadth is a strength, and the pivotal-information formalism is well-motivated, but the RL training experiments are small-scale and the absolute numbers after training aren't reported against the full communication+retrieval gap. The obvious next experiment — scaling RL training to larger teams (3+ agents) with heterogeneous model compositions — wasn't run. Most likely: compute constraints and the combinatorial explosion of multi-agent RL training made this impractical for the current submission.
