Imagine you're teaching someone to cook by talking them through it over a video call. You can see what they see, you can describe each step, but you can't grab the spatula yourself. Now imagine the cook has never held a spatula before. That's agentic robot control: a frontier AI model receives camera images from a simulated robot and outputs motor commands directly, no pre-trained manipulation policy in between, no motion planner to lean on. RoboARCade asks how well today's best models handle that job across sustained, multi-step tasks. The committed claim: this is the first benchmark that tests frontier multimodal agents as direct robot controllers across three different robot embodiments, long task horizons, and wide initial-condition variation — all within a single simulation backend (MuJoCo). Prior benchmarks either test short-horizon skills, use a single robot, or don't stress in-context adaptation. RoboARCade is designed to separate agents that can actually reason and adapt from those that merely pattern-match a narrow distribution. The scoreboard is revealing. GPT-6 Astra leads at 49.8% task success, Claude Fable 5.1 follows at 43.6%. That 6.2-point gap is meaningful but what matters more is the ceiling: nobody cracks 50%. The failure analysis is the real payload — agents rarely misunderstand what they're looking at. They fail because they pick a bad manipulation strategy or execute a sound one imprecisely. And they lie about it: agents incorrectly claim success on 24% to 39% of subtasks they attempt, a self-calibration problem that has direct safety implications for any real-world deployment. An encouraging finding tempers the pessimism. When given a single successful demonstration in context, Astra's score on a five-task subset jumps from 53.5% to 73.0% — a 19.5 percentage-point lift from one example. Even more striking, a prompt the agent writes and revises for itself reaches 72.0% on the same subset, suggesting these models can bootstrap their own improvement without human supervision. That's the seed of a real capability loop. The interface comparison delivers a practical engineering insight. A richer visual interface (presumably higher-resolution or multi-view) adds only 1.5 to 6.8 percentage points of performance while costing 1.7× to 2.3× more in compute. For most applications, the lean interface dominates on cost-efficiency. This is the kind of finding that saves teams months of wasted infrastructure work. Architecturally, RoboARCade is designed for extensibility. All tasks run on MuJoCo, each is a self-contained data pack, and agents connect through interchangeable interfaces. New tasks, agents, and interfaces snap in independently. This modularity is the benchmark's strongest structural contribution — it lowers the bar for the community to add embodiments and tasks, which is how benchmarks become durable standards rather than one-off papers. The integrity picture is solid for a benchmark paper: code, videos, and full traces are publicly available at roboarcade.org. The comparison is head-to-head between current frontier models, not stale baselines. The main limitation is that all validation is in simulation — no physical robot experiments — which is acknowledged implicitly by the MuJoCo-only design. The obvious next experiment is sim-to-real transfer: do these failure modes (bad strategy, imprecise execution, false success claims) persist on physical hardware? The authors almost certainly know this is the question, and the likeliest read is they're saving it for the next paper while the community stress-tests the benchmark itself.
