An AI Agent That Knows How the Compiler Thinks: EngineAgent Doubles Success at Getting New LLMs Running on Phones

Separating reusable engine knowledge from model-specific code lets an agentic system hit 75% end-to-end success porting LLMs to mobile hardware — more than double the best coding-agent baseline.

Imagine you're a veteran car mechanic, and every month a new engine design lands on your lift. You don't re-learn physics each time — you know where the oil filter goes on this class of block, which torque specs the head bolts need, and which diagnostic port to plug into first. A fresh apprentice with a repair manual can eventually figure it out, but you're faster because your knowledge is organized by the ENGINE'S architecture, not by the car's brand name. EngineAgent applies exactly this insight to the problem of getting new large language models running on mobile devices. The committed claim: by pre-indexing the layered structure of an on-device inference engine — conversion, graph construction, operators, hardware backends — and pre-validating build/execution/debug workflows on already-supported models, an agentic system can coordinate cross-layer changes for unseen models and more than double end-to-end success rates over general-purpose coding agents. The paper reports 75% success across 12 enablement tasks (three mobile LLMs × four engine–backend pairs spanning CPUs, GPUs, and NPUs), compared to 33.3% for the strongest coding-agent baseline. Success is defined strictly: the model must compile, execute on-device, and produce logits that agree with a reference implementation. The architectural bet is a separation of concerns that most coding agents miss entirely. Existing agents treat engine code as flat — they'll patch one layer without understanding that a change to an operator kernel ripples into graph construction and then into backend-specific compilation. EngineAgent builds what the authors call a "hierarchy-aware engine index" before touching any new model. Think of it as a dependency graph of the engine itself, not the model. A builder agent uses this index to plan coordinated changes; an auditor agent then checks completeness and drives iterative debugging through actual on-device validation. This two-agent loop — builder plus auditor — is the core control flow. On the ladder: the 33.3% baseline is described as the "strongest coding-agent baseline," but the paper doesn't name a widely recognized community benchmark or a specific competing system beyond generic coding agents (likely Cursor-style or Devin-style tool-use agents). The 75% figure is real and measured end-to-end on hardware, which is unusually strict for a systems paper — most stop at compilation. But the task set is author-defined (12 tasks), and we don't know how representative those 12 are of the long tail of real-world enablement problems. The paper also notes that EngineAgent reduces average per-task time and model-API cost relative to baselines, though specific numbers for these secondary metrics aren't foregrounded in the abstract. Integrity is mixed. On the strong side: validation requires actual on-device execution and logit agreement, not just "it compiled." That's a high bar and it's the right bar. On the weak side: the 12 tasks are author-selected, no pre-registration is mentioned, and independent replication hasn't happened. The two underlying LLMs used to power EngineAgent both hit 75%, which is a useful robustness signal — it suggests the method isn't overfit to one particular foundation model's strengths. Code availability is not stated in the abstract. The milestone question: 75% on 12 tasks is the current mark. The obvious next number is 90%+ on a broader, community-defined task suite — say, 50+ enablement tasks covering the full zoo of attention variants, quantization schemes, and exotic hardware backends (RISC-V NPUs, custom DSPs). That's probably 1-2 years away if a community benchmark coalesces. The deeper milestone is whether this approach generalizes beyond LLMs to other model families (vision transformers, diffusion models, multimodal) and whether the hierarchy-aware index can be auto-generated rather than semi-manually constructed. The experiment not run: scaling to a genuinely novel architecture that shares almost no structural DNA with existing supported models — something like a state-space model on an NPU backend with no prior operator coverage. The honest read is (a) and (c): the current task set likely stays within families where partial operator support already exists, and testing on a truly alien architecture would be the natural next paper. If it failed, it would reveal whether the hierarchy-aware index is truly generalizable or just a sophisticated lookup table for known engine patterns.