Red-Teaming Mobile AI Agents: An Adaptive Framework That Follows the Agent's Path Before Attacking It

MobileProbe doesn't pin attacks to fixed steps — it watches where an agent actually goes, then injects context-aware payloads at discovered weak points. 83 attack instances, 12 apps, 3 agent architectures.

Imagine you're a building inspector who only checks the front door. You rattle the knob, declare the building secure, and move on — never noticing the side window that most tenants actually use to enter. That's roughly how existing security benchmarks evaluate mobile AI agents: they bolt a single malicious payload to a single predetermined step in a single task, then check whether the agent fell for it. If the agent happened to take a different route through the app, it never encounters the trap, and the benchmark records it as 'safe' — not because it's robust, but because it was never actually tested. MobileProbe replaces the fixed-door inspector with one who follows the tenant around the building. The committed claim: MobileProbe is the first trajectory-aware red-teaming framework for mobile GUI agents, adaptively discovering attack surfaces and injecting payloads along the agent's actual execution path rather than at pre-scripted locations. The framework operates in three stages — automated pentesting to discover attacker-controllable content surfaces (text fields, notifications, web views), trajectory analysis to localize vulnerable decision points as the agent navigates, and an evolutionary refinement loop that iterates payloads through a surrogate filter before deployment. The surrogate acts as a cheaper stand-in for the target agent, letting the attacker evolve effective injections without unlimited access to the victim. The threat model is environmental injection attack (EIA): an adversary embeds malicious instructions into ordinary UI content — a chat message, a search result, a notification — and the agent, which reads the screen and acts on what it sees, follows the injected instruction instead of the user's intent. The benchmark covers 83 long-horizon attack instances across 12 mobile applications, each annotated with a CIA (Confidentiality, Integrity, Availability) goal. This is a meaningful coverage upgrade: prior work like InjectAgent and MobileInjection tested narrower scopes with static placements. On the ladder: MobileProbe is compared against human-written payloads, zero-shot LLM-generated payloads, and prior automated red-teaming baselines adapted to mobile. It reports higher attack success rates across three representative agent architectures. The paper doesn't name exact percentage deltas in the abstract, but the claim of superiority over all four baseline categories — including human-crafted attacks — is strong if the numbers hold. The architecture family is evolutionary optimization with a surrogate filter, closer to adversarial machine learning's query-efficient attack literature than to classical pen-testing tools. Integrity has the shape you'd expect from a top-tier security venue (NDSS 2027 acceptance): a custom benchmark rather than a pre-existing community standard, but one that covers 12 apps and 83 instances with structured CIA annotation. The validation is self-benchmarked — the authors built both the framework and the evaluation suite. No independent replication yet. The evolutionary loop's surrogate filter introduces a question about transferability: payloads optimized against a surrogate may not transfer equally to all target architectures, and the paper's coverage of three architectures is a start but not exhaustive. The milestone to watch is whether this framework or its successors become standard pre-deployment auditing tools for commercial mobile agents. The current number is 83 attack instances across 12 apps. The threshold that matters is coverage of the top 50 consumer apps with validated attack taxonomies — roughly a 5× scale-up. The obvious experiment not run: testing against agents with explicit injection-defense mechanisms (system prompts, input sanitization, confirmation dialogs). The honest read is (a) and (c): defenses are still nascent in deployed agents, so testing against hardened targets wasn't feasible yet, and demonstrating attack capability first is the strategic play for a security paper. The defense paper is coming. For practitioners building or deploying mobile agents, the takeaway is concrete: static security benchmarks undercount your real attack surface. If your agent takes different paths through an app depending on context, a fixed-point test may never touch the vulnerable states. Trajectory-aware evaluation is the right frame, and MobileProbe provides the first structured implementation of it. Whether you adopt this specific tool or not, the principle — follow the agent, then attack — should shape how you audit.