Imagine you're coaching a new employee who has to run a month-long science fair project. They make dozens of decisions — pick a dataset split, choose a model architecture, tune hyperparameters, debug errors — and the only feedback they get is a final score at the end. Now imagine you could rewind, chop that month into weekly windows, and give them a grade for each week based on whether their experiments improved. That's the core mechanism here: Windowed Interaction RL (WIRL), which breaks the credit-assignment problem in long-horizon ML engineering into manageable chunks. The committed claim: a 35B Mixture-of-Experts model, post-trained with a two-stage recipe (hierarchical supervised fine-tuning followed by WIRL), can autonomously run multi-step Kaggle-style ML experiments and achieve a 72.7% medal rate on MLE-Bench Lite — roughly doubling the 37.9% of its base model and matching much larger frontier systems like Kimi-K3 and GLM-5.3. The paper frames this as progress toward 'AI for AI' and recursive self-improvement, though the demonstrated capability is more precisely 'automated Kaggle competitor.' The training pipeline has two stages that matter. First, Hierarchical Agentic SFT takes existing tool-use and coding data and reshapes it into MLE-specific trajectories using what they call outcome-guided supervision — filtering training signal from long interaction logs to keep only the parts that led to measurable improvement. Second, WIRL treats the decisions between consecutive experiment submissions as a single 'execution window' and assigns a shared outcome-based reward to the whole window. This sidesteps the classic RL problem of assigning credit to individual actions in a 50-step trajectory where only the final result is observable. Asynchronous collection and policy correction let them update the policy before a full trajectory finishes, which is practically necessary when a single trajectory can involve hours of compute. The ladder comparison is both the paper's strength and its weakness. On MLE-Bench Lite, MLExpert at 35B active parameters matches Kimi-K3 and GLM-5.3, which are considerably larger frontier models. That's a genuine result. But the comparison set is narrow: they don't compare against open-weight models of similar size that have been fine-tuned for coding (like DeepSeek-Coder or Qwen2.5-Coder at 32B). The base model (likely an InternLM variant, given the team) starts at 37.9%, so the delta is real, but we don't know how much of it comes from WIRL specifically versus the SFT stage alone. An ablation exists but could be sharper. Integrity is mixed. MLE-Bench Lite is a community benchmark with known tasks, which is good. The Kaggle-MLE Factory for training data construction is described but the data itself isn't released — only the code and model weights are. The generalization experiments on NatureBench Lite, agent scaffold optimization, and algorithm optimization are encouraging but reported without the same rigor as the headline result. No pre-registration, no independent replication yet. The biggest integrity question: the authors built the training harness AND evaluated on it, which creates a subtle optimization-to-the-test-loop risk even with held-out tasks. The real contribution isn't the benchmark number — it's the training recipe for long-horizon agentic RL. WIRL is a practical answer to a real problem: how do you do RL when your trajectories are hours long, your actions are code blocks, and your reward is a leaderboard score that arrives after dozens of intermediate decisions? If this recipe generalizes, it's applicable to any domain where agents must plan over extended interaction sequences — not just ML engineering but code generation, scientific experimentation, and multi-step tool use. The obvious next experiment they didn't run: scaling WIRL to a larger base model (70B+) and testing on the full MLE-Bench rather than Lite. The Lite subset has 22 tasks; the full benchmark has 75. My read: compute budget. Training a 35B MoE model with multi-hour RL trajectories is already expensive; doing it at 70B on 75 tasks would be a significant infrastructure investment. They're likely saving it for the next paper, especially given the explicit framing toward recursive self-improvement.
