Imagine you're coaching a relay team. You could time the whole relay and yell "faster" at everyone equally — that's end-to-end reward. Or you could give the sprinter a split-time coach, the baton-passer a handoff coach, and the anchor a finish-line coach, each optimizing their specific job. This paper argues the second approach works better for search query understanding, and they have the numbers from Roblox's production system to prove it. The committed claim: a two-stage distill-then-RL pipeline where each query understanding component (intent classification, query expansion, etc.) receives its own reward signal derived from live search engine interaction outperforms both supervised fine-tuning alone (+8.9 NDCG@20) and single end-to-end RL reward (+3.5 NDCG@20). The key architectural insight is that different QU components have different operational roles in the search pipeline, and collapsing them into a single reward signal washes out the gradient information each component needs. The architecture sits squarely in the LLM-as-structured-generator family. A teacher model (presumably larger) distills into a student that produces schema-compliant search execution plans — intent labels, expanded queries, and other structured outputs. The SFT stage ensures the student doesn't hallucinate malformed outputs. Then RL fine-tuning treats the student as a policy, with the search engine itself as the environment. The novelty is in the reward decomposition: instead of one NDCG-based reward at the end, each component gets a reward tailored to its specific role in retrieval and ranking. The integrity picture is mixed. This is a production system evaluation on Roblox search, which means real user queries and a real search engine — that's better than a synthetic benchmark. But it's also entirely internal: the search engine, the query distribution, and the reward functions are all Roblox's. No external team can replicate this without access to Roblox's infrastructure. The baselines are the paper's own SFT policy and an end-to-end RL variant — reasonable ablations, but we don't see comparison to other production QU systems or published academic baselines on public datasets. The ladder position is honest but narrow. The paper compares against its own SFT baseline and its own single-reward RL variant. It doesn't name external competitors — no comparison to Google's QU stack, Bing's RLHF-based search improvements, or academic QU benchmarks. The +8.9 over SFT and +3.5 over end-to-end RL are meaningful deltas, but they're measured on a proprietary system. The generalizability question — does component-specific reward decomposition help on any multi-component search pipeline? — remains open. The milestone to watch is whether component-specific RL reward decomposition becomes a standard pattern in production search QU systems beyond Roblox. The current result is a single-system proof point. The real unlock would be reproduction on 2-3 other production search engines with different architectures, confirming that reward decomposition is the active ingredient rather than Roblox-specific pipeline properties. The obvious experiment not run: ablating which specific component rewards matter most. If you have N components each with their own reward, the combinatorial question is which decompositions drive the gains. Did intent classification reward contribute 80% of the lift? Or was it query expansion? The paper doesn't report per-component ablations at the reward level. Honest read: this is likely being saved for the next paper, as it's the natural follow-up and the data presumably exists.
