Imagine you're typing a password on a tiny phone keyboard. You fat-finger it, get rejected, and try again. You don't need the error message — you already know the password; you just need another shot at the keys. Now imagine the lock screen also shows a hint like "wrong third character," and you think that helps — but statistically, your success rate barely changes versus just retyping blind. That's the core finding here: most of what we call "self-repair" in code-generating robot agents is indistinguishable from getting another roll of the dice. The committed claim: failure-specific feedback — the error messages, stack traces, and failure-label templates that code-as-policy systems feed back to the language model — contributes far less to retry success than the retry itself. Aphale and Singh tested Qwen2.5-Coder-7B on 252 initially failed robot manipulation tasks in simulation, giving the model up to five retries. With template-based feedback keyed to the failure type, recovery hit 85.3%. Pure resampling — just asking again with no feedback — recovered 68.3%. That 17-point gap looks meaningful until you realize that feeding the model feedback from a completely different episode, or even a fixed static task description, produced recovery rates statistically indistinguishable from matched feedback. The coarse failure label added no detectable benefit. The architecture is deliberately minimal: a frozen 7B-parameter code model, a lookup table mapping failure labels to pre-written feedback templates, and a full scene reset between each retry. No fine-tuning, no learned repair policy, no retrieval-augmented generation. This is a controlled ablation study, not a systems paper. The method family is prompt-conditioned code generation — the same family as Code-as-Policies (Liang et al., 2023) and SayCan, but stripped to its studs to isolate what actually drives recovery. The integrity story is mixed. On the positive side, the authors compare against the right baselines: pure resampling, mismatched feedback, and static guidance — exactly the controls the field usually omits. The 252-failure test set is fixed across conditions, enabling paired comparisons. On the negative side, this is simulation-only (likely a RLBench-style environment, though the specific simulator isn't named in the abstract), all evaluations use a single frozen model, and there's no pre-registration. The nut-insertion result — where template feedback actively hurt performance by suggesting an incompatible grasp strategy — is the most interesting finding and also the most anecdotal, resting on a single task. The ladder position is clear and honestly stated. This paper doesn't claim to beat any SOTA system; it claims the SOTA evaluation methodology is broken. The implicit target is the entire self-repair literature (Huang et al., SelfDebug, Reflexion-style loops) that reports recovery rates without resampling baselines. By showing that a frozen model with scene resets recovers 68% of failures through pure luck-of-the-draw, the paper argues that published self-repair gains are inflated by conflating feedback benefit with retry benefit. The milestone here isn't a performance number — it's a methodological bar. The paper's concrete proposal: future evaluations of robot self-repair must include (1) resampling baselines, (2) mismatched-feedback controls, and (3) static task-guidance baselines. If CoRL and RSS reviewers adopt this as a checklist, it would immediately deflate a lot of published numbers. The gap between "self-repair" and "resampling" is the number to watch: if it stays at ~17 points or shrinks further as models scale, the entire self-repair-via-feedback paradigm needs rethinking. The obvious successor experiment is scaling: does the feedback gap widen or narrow with larger models (70B, frontier-class)? A stronger model might have higher baseline resampling success, compressing the gap further — or it might be better at actually using structured feedback. The authors ran a 7B model only. The honest read: compute budget. Running five retries × 252 failures × multiple conditions is already expensive; doing it at 70B is a different lab's budget. The second missing experiment is real hardware. Simulation resets the scene perfectly between retries; on a physical robot, state drift between attempts could make feedback genuinely informative in a way that simulation can't capture.
