Imagine you have five friends who each speak one foreign language fluently, and you want to build a single phrase book that captures all five. The brute-force approach: have each friend tutor a student for months. This paper's approach: record each friend answering a few hundred questions, measure the statistical shape of how they transform input into output at each layer, then solve for the single set of weights that best reproduces all five transformations simultaneously. It's a least-squares regression problem, not a training problem. The committed claim: when specialist LLMs share a base model, you can fuse them into a single generalist using only forward passes over a few hundred rollouts per specialist — no gradient computation, no teacher inference at training time — and the result outperforms naive task arithmetic (simple weight averaging) while approaching or matching each specialist on its own domain. The key insight is that each linear layer in the fused model must reproduce every specialist's output on the inputs that specialist actually sees while reading its own generated answers. This is a well-posed least-squares problem requiring only second-moment statistics of those inputs. The paper sits in the model merging lineage — task arithmetic, TIES, DARE, Fisher-weighted merging — but makes a sharper structural claim than any of these. Where task arithmetic treats weight deltas as vectors to be averaged, TFOD recognizes that the correct merge depends on the input distribution each specialist encounters during its own on-policy generation. That distribution-awareness is what separates this from prior art. The baseline ladder is honest: on OlMo 3 7B, task arithmetic scores are reported alongside TFOD across seven math and code benchmarks, and TFOD wins on aggregate while the paper acknowledges dimension-by-dimension variation. The most interesting practical finding is about data curation, not algebra. Among a math specialist's rollouts, answers that get stuck in repetition loops are rare by count but account for a disproportionate share of tokens. Removing these degenerate rollouts before computing input statistics lifts the fused model from task-arithmetic-level performance to near-specialist accuracy on AIME25. This suggests the bottleneck in model merging may be distribution quality, not solver sophistication — a finding that could generalize well beyond this specific method. Validation is simulation-only, using the authors' own codebase on one model family (OlMo 3 7B) with seven benchmarks. The integrity profile is mixed: benchmarks are public and well-known (AIME25, math/code standards), but there's no pre-registration, no independent replication, and only one base model tested. The few-GPU-hours compute claim is attractive but not independently verified. The method's linearity assumption — that layer-wise least-squares is the right objective — works when specialists share a base model but the paper doesn't test what happens when they don't. The milestone to watch is whether this scales to larger models and more diverse specialist sets. Seven benchmarks on a 7B model is a proof of concept. The real test is whether TFOD holds at 70B+ with 10+ specialists spanning reasoning, code, multilingual, and tool-use domains. If it does, this could become the default post-training consolidation step — dramatically cheaper than distillation. If it doesn't, the linear assumption breaks and this remains a clever trick for small-scale merges. The obvious experiment not run: applying TFOD to models that don't share a base. The paper's entire mathematical machinery depends on the shared-base assumption. Testing on independently trained specialists would either validate the approach's generality or reveal its fundamental limitation. The honest read is probably (a) — they know this is the hard extension and it's the next paper, not a result they're hiding.
