Let the Neural Net Sketch, Let the Solver Build: A Clean Division of Labor Hits 97% on Multi-Robot Planning

Small VLM writes a plan sketch; a deterministic solver expands it into a collision-free schedule. 97% OOD accuracy at 7B parameters. Ordering errors vanish.

Imagine you're renovating a house and you hire an architect and a general contractor. The architect walks the site, decides which rooms need work, sketches the order of operations, and assigns trades — electrician here, plumber there. But she doesn't compute the exact schedule. She hands the sketch to the contractor, who resolves dependencies, avoids conflicts between crews, and produces a day-by-day timeline. You wouldn't want the architect computing pipe diameters, and you wouldn't want the contractor making aesthetic decisions. NSDL applies this exact division to multi-robot task planning from images and natural language. The committed claim: a small vision-language model (3B–8B parameters) trained with a deterministic solver in the loop can outperform both end-to-end neural planners and formalize-then-plan pipelines on heterogeneous multi-robot task planning, reaching 97.0% out-of-distribution accuracy on the VIKI-L2 benchmark at 7B scale. This is not the first paper to combine neural perception with symbolic planning, but it is a clean demonstration that the *interface* between the two — the plan sketch — can be learned rather than hand-engineered, and that training with the solver in the loop matters. The architecture is straightforward and that's its strength. A vision-language model (tested with Qwen2.5-VL-3B, LLaVA-OneVision-7B, and InternVL2.5-8B backbones) takes a scene image and a natural-language instruction, calls an object detector for grounding evidence, and outputs a plan sketch — a structured list of goals needing action, each assigned to a robot, in intended order. A deterministic solver then expands this sketch into a fully executable joint plan with correct prerequisites, handoffs, and timing. The key design choice: the solver runs during training, so the VLM learns to produce sketches the solver can actually expand. This closes the loop between perception-level judgment and combinatorial structure. On the ladder, NSDL beats both end-to-end planners and formalize-then-plan baselines across both benchmarks. The 7B model hits 97.0% OOD accuracy on VIKI-L2, and the paper introduces LOCUS-Bench specifically to disentangle perceptual errors from structural errors. The results show that once the solver handles ordering and scheduling, those error categories nearly vanish — the remaining failures are almost entirely misjudged scene states (goals the model either misidentifies or never lists). This is a useful diagnostic: it tells you exactly where to invest next. Integrity is solid for a robotics planning paper. Two benchmarks are used — the public VIKI-L2 and the newly introduced LOCUS-Bench — and three different VLM backbones are tested. Code, model weights, and LOCUS-Bench are publicly released. The concern is that LOCUS-Bench was designed by the same team, so the benchmark and the method co-evolved. VIKI-L2 is the independent check, and the 97% number there is strong. No physical robot experiments are reported; this is simulation-only. The milestone question is about scale: the current setup works for the task complexity in VIKI-L2 and LOCUS-Bench, but real warehouse or construction environments involve dozens of robots, partial observability, dynamic re-planning, and tasks that require continuous state estimation — not single-image snapshots. The next concrete test is whether the sketch-and-solve pattern holds when the number of robots exceeds ~10 and the planning horizon exceeds ~20 steps with real sensor noise. The gap between these benchmarks and deployable multi-robot coordination is significant but well-defined. The obvious experiment not run: physical robot deployment. The paper is simulation-only, and the authors clearly know this is the gap — the ICRA 2027 submission venue is a robotics conference that values hardware demonstrations. The honest read is (a) and (c) combined: physical experiments require hardware access and integration time that's a separate project, and a strong sim-only result at ICRA is a reasonable first paper with the physical follow-up planned. The other missing piece is scaling to larger robot teams and longer horizons, which would stress both the VLM's sketch quality and the solver's tractability.