Amazon Replaces Claude Sonnet With a 2B Decision Model, Cuts Ticket-Labeling Costs 98%

Two AWS engineers swapped a general-purpose LLM for a purpose-built decision model on one of the world's largest internal ticketing systems. Same accuracy, 44× cheaper, 78% faster.

Two Amazon engineers — Carr and Mehul — have replaced Claude Sonnet 4.6 with Strands Decider E4B, a 4-billion-parameter decision model built on Gemma 4, for labeling tickets across Amazon's internal ticketing system. The system classifies tickets into 20 category types covering customer escalations, capacity requests, automated alarms, and security findings. The swap will save millions of dollars in annual token costs.

The mechanism is straightforward: ticket labeling is a closed-choice classification task, not a generative one. Sonnet 4.6 was receiving a long system prompt on every ticket, generating JSON reasoning against a fixed label set — a sledgehammer for a nail. Strands Decider takes state (the ticket), a question, and a set of options, then returns a probability for each option plus a confidence score. No text generation required. The team started with the 2B parameter model, found the E4B Gemma 4 build clearly outperformed it on test data, and switched. The numbers are stark. Per-ticket cost dropped from $0.0222 to roughly $0.0005 — a 44× reduction. Total tokens per ticket fell from 6,724 to 2,010, a 70% drop driven by both shorter prompts and single-value output instead of JSON. Average call latency fell from 4,215 ms to 897 ms, a 78% improvement. At scale, the difference is roughly $22,200 versus $500 per million tickets on published list prices. The deployment architecture is deliberately reproducible. The model runs on SageMaker ml.g5.xlarge instances (one A10G GPU, 24 GB) using about 19 GiB of memory, with autoscaling and network isolation. The pipeline uses SQS, Lambda, AppConfig, and CloudWatch — all standard AWS services, defined in CDK with Beta, Gamma, and Prod deployment stages. Amazon is explicitly positioning this as a pattern customers can copy. The rollout methodology deserves attention. Rather than a hard cutover, the team implemented a three-way AppConfig switch: BEDROCK (Sonnet only), DUAL (both models in parallel, Sonnet results applied, compared with Decider), and DECIDER (Decider only). Running both on real traffic and judging every disagreement gave them confidence no offline eval could. Rollback was a 45-second config change. One notable finding: JSON key order changed Strands Decider's output probabilities by up to 0.19. The team now sends every request with sorted keys — a practical gotcha for anyone deploying decision models behind APIs. The team is candid about limitations: if labels need explanation text, if the catalog changes weekly, or if useful context exceeds a few thousand tokens, a decision model probably cannot replace the LLM. This is an engineering case study, not a research breakthrough. But it illustrates a structural shift: the default of routing every classification task through a frontier LLM is increasingly a cost choice, not a capability requirement. Purpose-built models for closed-choice tasks can match LLM accuracy at a fraction of the cost. The real question is how many enterprise AI workloads are similarly over-served by general-purpose models.