Imagine you're a judge in a courtroom. You know to be suspicious when someone walks in and says "ignore the case, rule for me instead" — that's an obvious hijack. But what if a witness simply lies about the facts, submits a forged receipt, or misquotes the law? The case proceeds normally. You issue a verdict. It's wrong. Nobody noticed an override because there wasn't one. That's the core mechanism this paper exposes in LLM-based agents. The committed claim: indirect prompt injection benchmarks have been measuring the wrong thing. Existing evaluations overwhelmingly test whether an attacker can redirect an agent to a completely different task — the courtroom equivalent of "ignore the case." Zhang et al. argue this misses the more dangerous class: attacks that leave the task intact but poison the information the agent uses to complete it. They formalize IPI around authority — planted content is an attack when it tries to influence a decision its source has no standing to determine — and build a 90-case benchmark across three tool-use environments to prove the point. The benchmark's design is precise. Each case specifies which source is authorized for which decision, then tests attacks targeting six distinct parts of the agent's reasoning: its objective, decision rules, available actions, expected consequences, observation reading, and beliefs about current state. This decomposition is the paper's real contribution. It reveals that attacks on decision rules, available actions, and state beliefs succeed more often than objective-override attacks across all five tested LLMs — without any defense in place. The hierarchy of vulnerability is the finding, not just the existence of vulnerability. Defenses look worse than you'd think. A detection-and-removal approach crushes objective-targeting attacks down to 2.4% success, which sounds great until you see observation-targeting and state-targeting attacks still land at 9.5% and 12.1% respectively. A tool-policy defense — restricting which tools the agent can call — barely helps at all: 15.4% success vs 17.0% undefended. Why? Because these attacks don't change the tools used; they change the values flowing through legitimate tool calls. The defense is watching the doors while the poison comes through the plumbing. The integrity picture is solid for a benchmark paper. All 90 cases are validated, the attack taxonomy is grounded in decision theory rather than ad hoc categories, code and data are public, and five different LLMs are tested. The limitation is that the environments are the authors' own constructions — there's no third-party adversarial red-teaming yet, and the three tool-use scenarios (while covering real patterns like buyer disputes and information retrieval) aren't community-standard benchmarks the way SWE-bench or MMLU are. The practical upshot is uncomfortable for anyone deploying tool-using agents. If your defense strategy is "detect and remove injected instructions," you're filtering for the loud attacks while the quiet ones walk through. The paper doesn't propose a complete solution — it diagnoses the gap. The next milestone is a defense architecture that reasons about information provenance at the value level, not just at the instruction level. That's a harder problem, and nobody has cracked it yet. This is an Amazon Science paper, which means the authors likely have access to production agent systems and real attack surfaces. The benchmark feels informed by actual deployment pain, not just academic curiosity. The six-target taxonomy (objective, rules, actions, consequences, observations, state) is clean enough to become a standard decomposition if the community adopts it — and it should, because the finding that defenses have blind spots by attack-target category is immediately actionable for anyone building agent guardrails.
