Imagine you want to compare a hundred chess players, but you can't make them all play each other. Instead, you show each player the same set of famous game positions and ask: 'What would you do here?' You never watch them play a full game. You just score their instincts against the same reference positions, and from those scores alone you build a map that tells you which players think alike, which ones are aggressive, which ones are cautious — and even predicts how well they'd do in a tournament. That's what Agent Behavior Maps do for LLM agents. The committed claim: you can represent any LLM-harness combination as a vector of centered action log-likelihoods computed over shared reference trajectories, and the distances in that space track real behavioral divergence (trajectory-level KL) in rank order — without ever running the target agent in the environment. The paper provides explicit error bounds relating squared coordinate distances to trajectory KL under stated assumptions, and validates empirically on eight cross-scored models. The scale is respectable: 100 LLMs across four environments, with ALFWorld as the primary testbed. The key result is that these coordinates support downstream performance prediction and behavioral clustering. Crucially, the behavioral similarities persist after controlling for both success rate and model size, and even hold across model families. Two agents can have identical success rates yet occupy very different positions on the map — the representation captures HOW agents succeed, not just WHETHER they do. Architecturally, this is an evaluation-regime paper, not a training paper. The method belongs to the family of probe-based behavioral characterization: you fix a set of reference trajectories (states and action sequences from existing rollouts), compute each target model's log-likelihood over actions in those trajectories, center the scores to remove model-level biases, and use the resulting vectors as coordinates. The key computational property is that you never run new rollouts — you only need forward passes through the target model, which is dramatically cheaper than generating full episodes. The theoretical grounding connects these coordinates to KL divergence between trajectory distributions, placing the approach in the information-geometry tradition. The ladder position is genuinely novel in framing. There isn't a direct prior-art 'Agent Behavior Map' to beat. The closest predecessors are agent evaluation frameworks that require full rollouts (like AgentBench) or behavioral probing methods from the LLM interpretability literature. The paper's contribution is showing that log-likelihood probes on shared trajectories are not just a cheap shortcut but a theoretically principled one, with error bounds. The eight-model cross-scoring experiment showing rank-order alignment with directly measured trajectory KL is the load-bearing validation. Integrity is mixed. The theoretical bounds are derived under explicit assumptions (which is good — they name what they need), and the empirical validation covers 100 models. But the primary testbed is ALFWorld, a text-based household environment that is relatively narrow. The four environments provide some breadth, but the compositionality results for harness-component displacements show accuracy 'varying across components and environments' — an honest but cautious framing. No code release is mentioned, no pre-registration, and no independent replication exists yet. The milestone question is about scale and generality. Right now this works on ALFWorld-scale environments with 100 models. The next concrete unlock would be demonstrating that behavior maps remain predictive and KL-aligned on more complex agent environments — think web browsing, coding, or multi-turn negotiation tasks — where the action space is larger and trajectories are longer. If the coordinates hold at that scale, this becomes a practical tool for agent selection and auditing. If they don't, it remains a theoretically interesting result scoped to narrow environments. The obvious next experiment the authors didn't run: testing on a genuinely open-ended environment like WebArena or SWE-bench, where action spaces are large and trajectories are long. The honest read is (a) — compute and logistics. Generating reference trajectories and computing log-likelihoods across 100 models in a complex environment is expensive, and ALFWorld was tractable enough to prove the concept. The compositionality extension (harness-component displacements) hints they're aware the method needs to scale, and the varying accuracy across environments suggests they've already seen the limits.
