Researchers Show How to Poison the App Store Your AI Agent Shops At

StealthyPoison corrupts third-party skill procedures so LLM agents make wrong decisions while looking correct. Tested on 566 skills, it evades GPT-5.4-mini detection and transfers across models.

You know how a restaurant menu can technically list every ingredient but still mislead you about what you're eating — say, calling something "artisanal smoked protein" when it's gas-station jerky? The dish arrives looking right, tasting close enough, but the nutritional reality is quietly wrong. StealthyPoison does exactly this to LLM agents: it rewrites the procedural instructions inside a third-party skill so that the agent follows what looks like a correct recipe but arrives at the wrong answer. The menu hasn't changed; the cooking instructions have. The committed claim: you can construct reusable poisoned procedures for third-party skills in a public registry — without knowing what questions real users will ask — and those poisoned procedures will transfer across victim models, evade state-of-the-art defenses, and reliably produce incorrect task outcomes. This is not a toy demo. The authors tested 566 target skills against 5,244 user requests across a registry of more than 26,000 entries, and showed end-to-end attack success on Claude Code powered by Opus 4.8. The attack pipeline has three stages that are worth understanding separately because each exposes a different vulnerability. First, the attacker synthesizes plausible user requests from a target skill's public description and runs them through a clean surrogate agent to generate pseudo-reference outputs — ground truth without needing actual user data. Second, it iteratively rewrites the skill's decision-critical rules (not the surface description) so that execution diverges from correct outcomes while preserving advertised functionality. Third, and this is the supply-chain piece, it augments the poisoned skill's retrieval metadata to outrank benign alternatives in search — achieving 13.6% better top-1 placement than both HotFlip and GCG, the two standard adversarial text-generation baselines. The ladder here matters. HotFlip and GCG are character- and token-level perturbation attacks designed for different threat models — they craft adversarial triggers, not procedural corruptions. StealthyPoison argues that the right attack surface for agentic systems is the procedure itself, not the prompt or the retrieval query. The 13.6% improvement on retrieval placement is real but the more striking number is the transferability: payloads constructed with a single surrogate model work unmodified on three held-out victim models. That's the property that makes this practical rather than academic. Integrity-wise, the evaluation is thorough but self-contained. The 566-skill, 5,244-request, 26,000-entry registry setup is large for this kind of adversarial work, and testing against Claude Code / Opus 4.8 grounds the results in a real deployed system. The defense evaluation against SkillSpector and GPT-5.4-mini is the right move — showing evasion against the strongest available detector. But there's no independent replication, no pre-registration, and the surrogate-to-victim transfer is tested on a limited set of held-out models. The authors chose the benchmarks after designing the attack, which is standard for offensive security research but worth flagging. The milestone question is about defenses, not attacks. This paper demonstrates that procedural-level poisoning is viable and stealthy. The next number to watch is detection rate: can any defense achieve >90% detection of procedural poisoning payloads without unacceptable false-positive rates on clean skills? Right now, the answer appears to be no — both SkillSpector and GPT-5.4-mini fail. The field needs a procedural integrity benchmark with known-poisoned and known-clean skills before anyone can claim robust defense. The obvious experiment not run: deploying this in the wild against real user traffic on a real skill registry (even a sandboxed one with consenting users). The authors almost certainly didn't do this for ethical and legal reasons — offensive security papers rarely do. But the gap between synthetic requests and real user behavior is exactly where the attack's real-world success rate could diverge from the lab numbers. The second missing piece is scaling: what happens when the registry has 500,000 skills instead of 26,000, and the attacker is competing against more diverse benign alternatives?