ReTrace Distills LLM Agents Into Hardcoded Workflows — 130× Faster, 24% Cheaper, and Almost as Accurate

Extracts the reasoning behind agent tool calls and compiles it into deterministic code workflows. 93.8% completion vs 96.8% for agents, at a fraction of the latency and cost.

Imagine you've hired an expensive consultant to handle your insurance claims. She's brilliant — reads the situation, picks the right form, calls the right department, every time. But after watching her handle a hundred claims, you realize she follows the same decision tree 95% of the time. So you write down the tree, hand it to a junior clerk, and only call the consultant when the clerk gets stuck. That's ReTrace: watch the LLM agent run a Standard Operating Procedure a few times, extract its decision logic into a deterministic code workflow, and stop paying for an LLM call at every step. The committed claim: given a natural-language SOP and a handful of logged agent runs, ReTrace produces executable workflows that replace the LLM planner entirely — matching 94-88% of agent accuracy at 24% lower cost and up to 130× lower latency. This is not a new agent architecture. It is an agent compiler. The core insight is that the reasoning trace — the natural-language explanation the agent writes alongside each tool call — contains the conditional logic that raw tool-call logs do not. ReTrace extracts conditions from these traces across multiple runs, merges them into a compact 'rule document,' and then generates an executable workflow from that document. The rule document is platform-agnostic; compilation to different orchestration engines is a separate, final step. This separation is architecturally clean: the reasoning extraction and the code generation are decoupled. The ladder is honest and well-populated. On 79 production procedures, ReTrace reaches 93.8% completion versus 96.8% for the full LLM agent — a 3-point gap that buys you 24% cost savings. On τ²-Bench, a community benchmark for tool-using agents, it hits 75.8% task success (agent: 80.6%) but runs in 1.3 seconds versus 170 seconds. That's not a marginal speedup — it's a categorical shift from 'interactive wait' to 'instant.' Three rounds of repair on training failures push k=2 workflows from 66.7% to 81.2%, actually surpassing the agent. On SOP-Bench, repair lifts ReTrace to 87.8% at 0.40s per task versus 85.5% in 46.6s for the agent — beating the agent on both accuracy and speed. Integrity is mixed. The 79-procedure production evaluation is impressive scope but comes from the authors' own platform, meaning environment and SOP design may favor the approach. τ²-Bench and SOP-Bench provide external validation, which is good, though SOP-Bench has only eight procedures. The repair mechanism — iterating on training failures — is powerful but introduces a question: how much of the final accuracy comes from overfitting to the training failure distribution? The paper does not report variance across different training-run samples or failure seeds. The real significance here is economic, not algorithmic. LLM agents are expensive to run at scale because every step requires inference. ReTrace converts variable inference cost into a one-time compilation cost. For enterprises running the same SOP thousands of times — insurance claims, IT tickets, customer service workflows — the savings compound fast. The 3-point accuracy gap is the price of admission; the 130× latency reduction is what makes it production-viable. The obvious next experiment is scaling the repair loop: what happens with 10 rounds instead of 3? The authors stop at 3 and show diminishing returns (66.7% → 75.3% → 78.7% → 81.2% on τ²-Bench k=2), which suggests the curve is flattening but hasn't plateaued. The honest read is they're saving deeper repair analysis for a follow-up, or the cost of generating labeled training failures becomes prohibitive. The other missing piece is a head-to-head against hybrid approaches — workflows that fall back to an LLM agent on low-confidence branches. That's the natural production architecture, and its absence is conspicuous.