A Bimanual Robot That Knows When to Move Furniture Before Reaching for the Mug

Zero-shot VLM framework couples task planning with physical feasibility checking for two-armed humanoid manipulation, beating naive planners across 9 real-world scenarios.

Imagine you're moving into a new apartment and you need to grab a box from the back of a packed closet. You don't just reach — you first figure out which stuff to shift, whether you can carry two things at once, and what to do if a box is heavier than expected. This paper teaches a robot to do that same meta-reasoning, but with two arms and a vision-language model doing the thinking. The core claim: a zero-shot framework that jointly plans bimanual manipulation tasks AND reasons about physical execution constraints — obstacle clearance, kinematic reachability, grasp feasibility — outperforms a baseline that plans first and worries about physics later. The system runs on CENTAURO, a hybrid wheeled-legged humanoid from IIT, and was tested across nine language-conditioned tabletop scenarios with real clutter. Architecturally, this is a structured prompting pipeline over a vision-language model (the paper does not name which VLM, which matters). The scene is encoded as a semantic graph combining object relations with execution-relevant properties — reachability annotations, obstacle flags, arm-assignment constraints. The planner selects which obstacles to relocate and which arm handles what, then organizes compatible actions into parallel bimanual stages with explicit pre- and post-conditions. This is not learned end-to-end; it is structured reasoning on top of a frozen foundation model. The hierarchical validator is the genuinely interesting piece. It evaluates pre- and post-conditions at each stage under perceptual uncertainty, maintaining a stage-level execution history. When something fails — a grasp slips, an object isn't where expected — the validator attempts local recovery before escalating to full replanning. In dedicated failure scenarios, this hierarchical approach recovered with fewer VLM queries than an always-replan strategy, which matters because each query is expensive in latency and cost. The ladder is thin. The baseline is the authors' own non-execution-aware planner — not SayCan, not Inner Monologue, not Code-as-Policies, not any named community benchmark for language-conditioned manipulation. The paper reports higher task success across nine scenarios but does not provide exact success-rate numbers in the abstract or give comparison against published SOTA systems. This is a demonstration of a design principle, not a benchmark-beating result. The integrity picture is mixed. These are real-robot experiments on real hardware, which is genuinely harder than simulation — you get credit for that. But nine scenarios is a small evaluation set, the baseline is internal, there is no independent replication, and the specific VLM backbone and its version are not named in the available text. The paper is accepted at Humanoids 2026 (peer-reviewed venue), and a project webpage with videos is provided. The milestone question for this line of work: when does execution-aware bimanual planning reliably handle 50+ object scenes with deformable objects and multi-step tool use? This paper demonstrates the principle at tabletop scale with rigid objects. The gap between 9 curated scenarios and robust household deployment is measured in years and orders of magnitude of scene complexity.