A Bandit Algorithm That Picks Which LLM to Call — and Beats Always Using the Best One

Ballad routes LLM calls across cheap and expensive models using online bandits, matching or beating the best single model while cutting API costs up to 65.6%.

You know how a good basketball coach doesn't just play the star point guard for 48 minutes? Sometimes the backup catches fire in the second half, sometimes the matchup changes, and a rigid rotation loses games that a read-and-react coach wins. Ballad applies this logic to LLM-powered heuristic search: instead of always calling the most expensive model, it watches which model is hot right now and routes accordingly. The committed claim: given a pool of LLMs (frontier and open-weight), an online bandit algorithm can learn during a single optimization run which model to call at each step, producing heuristics that match or beat always calling the strongest model — while cutting API costs by up to 65.6%. The paper tests this across five NP-hard combinatorial optimization problems (TSP, bin packing, online bin packing, flow-shop scheduling, and an admissible heuristic problem) with a four-model pool spanning capability tiers. The deeper empirical finding that motivates the whole paper is genuinely interesting: bigger, costlier LLMs do NOT consistently produce better heuristics in automated heuristic discovery (AHD). Across the five problems, there is no single dominant model. This is surprising because the default assumption in AHD systems like FunSearch and ReEvo is that you should just throw GPT-4-class models at everything. The authors show this is wasteful. Different problems, and different phases of the same problem, favor different models. The architecture is a contextual multi-armed bandit with three key design choices that separate it from vanilla Thompson Sampling. First, the selection policy favors tail performance — it weights rare high-scoring heuristics over stable averages, because in combinatorial optimization one breakthrough mutation matters more than consistent mediocrity. Second, credit propagation runs through a DAG tracing the genealogy of the heuristic population: if Model A produced a mediocre heuristic that Model B later mutated into something excellent, Model A gets partial credit. Third, stale observations are discounted so the policy tracks each model's shifting utility rather than locking onto early impressions. Integrity-wise, the setup is reasonable but not airtight. The five problems are standard NP-hard benchmarks — TSP, bin packing, and scheduling are well-established testing grounds, not cherry-picked puzzles. The baseline comparison is against individual models run solo and against uniform-random model selection. However, the four-model pool (likely including something like GPT-4, GPT-3.5, and open-weight models, though exact names would strengthen the paper) is modest. The 65.6% cost reduction figure is relative to always calling the strongest model, which is the right comparison but also the most favorable framing. What matters more is whether Ballad consistently matches or beats the best-in-hindsight model — and the paper claims it does, including cases where it surpasses every individual model. The milestone question is clear: this works for four models across five problems. The next meaningful test is whether the approach scales to 10-20 model pools including fine-tuned specialists, and whether the bandit converges fast enough when AHD budgets are tight (say, 200 LLM calls instead of thousands). The non-stationarity of the reward signal — each model's output seeds the next call's context — is the hard technical wrinkle, and the authors' discounting and DAG credit mechanisms are their proposed solutions. Whether these hold under more aggressive non-stationarity is an open question. The obvious experiment not run: testing Ballad in a setting where one model is deliberately adversarial or degenerate, or scaling to a pool where most models are bad and only one or two are useful. The authors likely didn't run this because the current four-model pool is curated to be plausible — all models can in principle produce useful heuristics. A stress test with a noisy pool would reveal whether the bandit's exploration-exploitation tradeoff is actually robust or whether it benefits from a pre-screened pool. My read: they're saving this for the next paper, or possibly ran it and found the convergence was slower than they wanted to report.