Imagine you're learning to cook a new dish, but someone has taped over half the ingredient labels. A master chef standing behind you can read every label — they have full information. The naive approach: just copy every move the chef makes. But the chef never hesitates, never peeks, never checks a taste — they already know what's in the jar. If you mimic them blindly, you'll never learn the crucial skill: how to figure out what's in the jar yourself. ASTEROID's insight is deceptively simple — let the student stumble around first, collecting their own confused partial observations, then ask the chef what the optimal move is given only what the student has seen so far. The student learns not just what to do, but when and how to investigate. The committed claim: supervised distillation from a privileged expert can teach test-time exploration behavior that is provably absent from standard imitation learning, achieving sample efficiency two orders of magnitude better than reinforcement learning while matching or exceeding RL returns. This is not a minor tweak — the paper argues that the entire RL-based exploration training pipeline is unnecessary for a wide class of partially observable problems, replaced by a student-teacher setup with on-policy context collection. The architecture sits squarely in the asymmetric actor-critic / privileged distillation family — think DAgger meets information-asymmetric training. The student is a history-conditioned policy (transformer or RNN consuming observation sequences), while the teacher is a state-conditioned expert with full access to the hidden task parameters. The critical structural choice is on-policy context: the student's own rollout trajectory becomes the conditioning input for the teacher's action labels. This breaks the classic distributional mismatch of offline imitation. No reward shaping, no value functions, no RL optimization loop — just supervised cross-entropy on the teacher's actions given the student's history. The ladder here is genuinely strong. The paper benchmarks across eleven environments spanning discrete exploration tasks (Tiger, Gridworld), continuous robotic manipulation (Meta-World, Franka kitchen), visual navigation, and Procgen games. Named baselines include RNN-based RL (PPO-RNN), DAgger, behavioral cloning, and — notably — frontier LLMs (GPT-4, Claude) on the discrete exploration tasks. ASTEROID achieves roughly 2× higher returns than RL under fixed exploration budgets, uses approximately 100× fewer environment interactions, and outperforms frontier LLMs on structured exploration. The real-robot transfer adds weight: +40% success rate over standard distillation on vision-denied manipulation, meaning the exploration behavior learned in simulation survives the sim-to-real gap. Integrity is solid but not bulletproof. The eleven environments provide breadth, and the sim-to-real transfer is a genuine physical experiment — not just another simulation. However, the teacher is an oracle with full state access, which means the ceiling is always set by how well the student can approximate Bayesian posterior inference from its history. There's no pre-registration, and while the environment suite is broad, most are established benchmarks rather than adversarially chosen. The comparison to frontier LLMs is a clever marketing move but somewhat apples-to-oranges — LLMs weren't designed for POMDP exploration. The code and environments appear reproducible. The milestone to watch is scaling ASTEROID to open-ended, high-dimensional partial observability — real-world autonomous systems where the hidden state space is vast and the teacher oracle is expensive or impossible to construct. Today: 11 environments, scripted or learned oracle teachers, moderate observation dimensionality. The threshold that matters: can this framework handle tasks where constructing a privileged expert is itself hard (e.g., autonomous driving with latent intent of other agents, or medical diagnosis with incomplete patient history)? That would move ASTEROID from a training trick to a foundational paradigm. The obvious experiment not run: scaling to environments where the privileged expert is approximate or noisy rather than an oracle. Every result in the paper assumes the teacher has perfect state information. Real deployment will require imperfect teachers — learned world models, human demonstrators with partial knowledge, or ensembles. The authors likely know this and are either (a) saving it for the next paper, or (c) the results with noisy teachers may degrade substantially and they want the clean story first. A second gap: no comparison to recent Bayesian RL or posterior-sampling methods that explicitly maintain belief states — the theoretical connection to Bayesian exploration is claimed but not empirically benchmarked against actual Bayesian methods like BAMDP solvers or variational Bayes RL.
