Imagine you're teaching someone to diagnose why a car won't start. Approach A: you hand them a rigid 47-step checklist — check the battery first, then the alternator, then the starter motor, in that exact order, no deviations. Approach B: you hand them a voltmeter, a code reader, and the repair manual, then say "figure it out." This paper is a controlled experiment proving that Approach B wins decisively for LLM-based root cause analysis of microservice failures. The committed claim: simple LLM harnesses that prescribe none of the model's investigation calls — a single-call evidence package, a SQL agent, and the Codex coding agent — outperform five published LLM RCA agents and all non-LLM statistical/causal methods on a unified 90-case benchmark called RCAtlas. The simple harnesses hit 65-77% precision at rank 1 on mid-tier models, while published agents land at 16-53% and non-LLM methods at 6-39%. This isn't a marginal win. The gap is structural. The architecture insight is sharp: the published agents fix anywhere from a tenth to all of their model calls in the program — meaning they hardcode which APIs to call, which data to look at, and in what order. The paper's case study shows how this rigidity causes agents to skip the decisive evidence entirely. The failure mode isn't that the models can't reason; it's that the scaffolding prevents them from looking where they need to look. The simple harnesses belong to the "tool-augmented LLM" family — SQL agent for structured data queries, Codex for code-based investigation, and a zero-tool single-call over a pre-assembled evidence package. No role-agents, no fixed stages, no standard operating procedures. The ladder comparison is unusually thorough. Five named published agents are benchmarked alongside three non-LLM baselines (statistical and causal methods) across six models and three reasoning-effort levels on RCAtlas's 90 validated cases drawn from eight public datasets. The paper finds that a stronger model mainly improves reliability (fewer catastrophic failures), a multi-turn harness mainly improves aptitude (better peak accuracy), and cranking up reasoning effort changes accuracy by only a few points while multiplying wall-clock time by up to 7×. Every Pareto-optimal run in cost or time among runs beating the best non-LLM method uses a simple harness. Integrity is solid but not ironclad. RCAtlas aggregates eight public datasets into 90 validated cases — a genuine community benchmark rather than a bespoke evaluation. Six models and three reasoning levels give real breadth. The weakness: the authors built both the benchmark and the simple harnesses, so there's no fully independent validation. The published agents were re-implemented and run on the same data, which is fair but means the comparison depends on faithful reimplementation. No pre-registration, no code release mentioned. The milestone this points toward is clear: if simple harnesses at 65-77% PR@1 already dominate, the next threshold is 90%+ reliability on production-scale incident data with real-time latency constraints — the point where automated RCA becomes deployable without human verification. The gap between 77% and 90% is likely closed by better models and richer evidence packaging rather than more elaborate agent scaffolding. The experiment the authors did NOT run: testing on proprietary production incident data from a real microservice deployment at scale, with noise, incomplete telemetry, and cascading failures. The honest read is (a) — access and confidentiality prevent it, not methodological avoidance. The second missing experiment is ablation on evidence-package composition: which signals in the package actually drive the accuracy gap?
