A Four-Stage Pipeline That Warns You Before You Place the Wrong Tangram Piece

TanCRAFT chains scene tracking, assembly reasoning, motion forecasting, and intervention into a real-time system that catches assembly errors before they happen — validated in user study.

Imagine you're assembling flat-pack furniture and a friend is watching over your shoulder. They don't just say "wrong" after you've already hammered the nail — they see your hand reaching for the wrong shelf bracket and say "not that one" before contact. That's TanCRAFT's core mechanism: predict where a piece is heading, check whether that destination is compatible with the target assembly, and intervene in the gap between intention and action. The system operates as a four-stage pipeline. Stage one tracks the current scene state — what pieces are placed, where they sit. Stage two reasons about the partial assembly: given the pieces already down, what placements remain valid? Stage three forecasts the near-future trajectory of the piece currently being manipulated. Stage four fires an intervention if the predicted landing violates the target. The key architectural choice is separating "what's valid" from "what's predicted" — the system doesn't enforce a fixed construction order, which matters because real assembly tasks rarely have a single correct sequence. The authors instantiate TanCRAFT on tangram puzzles — geometric dissection puzzles where seven flat pieces must form a target shape. This is a deliberate simplification: tangrams have a closed solution space but variable construction order, making them a controlled proxy for the harder problem of freeform assembly guidance. The offline benchmarks evaluate each pipeline stage independently (detection, state tracking, trajectory forecasting, intervention accuracy), while the user study compares three conditions: unguided, reactive feedback (warning after a bad placement), and predictive feedback (TanCRAFT's anticipatory warning). The user study results are the load-bearing part. Predictive feedback reduced mistakes relative to both unguided and reactive conditions. It also produced the lowest mean perceived workload (NASA-TLX) and shortest mean completion time. The combination matters: you could trivially reduce errors by making the system so intrusive it slows people down, but TanCRAFT apparently threads the needle of intervening helpfully without creating friction. The within-subject design (each participant experiences all three conditions) controls for individual skill variation. The integrity picture is mixed but honest. The tangram domain is clean enough that the pipeline's correctness can be verified against ground truth placements, and the user study provides behavioral data beyond simulation. But the study is small-scale, the domain is toy-level relative to industrial assembly, and the paper is submitted to IEEE — not yet peer-reviewed. The authors acknowledge that extending to real manufacturing workcells with 3D parts, occlusion, and higher degrees of freedom is the actual hard problem. The field fight here is between reactive assistance (wait for an error, then correct) and anticipatory assistance (predict the error, intervene preemptively). Most deployed assembly guidance systems are reactive — step-by-step instructions or post-hoc error detection. TanCRAFT argues that the predictive approach is strictly better when you can forecast intention accurately enough. The open question is whether trajectory forecasting generalizes beyond planar puzzles to the spatially complex, occluded, high-DOF world of real parts. What makes this a useful step rather than a breakthrough is the gap between tangrams and industrial assembly. The pipeline architecture is sound and modular — you could swap in better detectors, better forecasters, better reasoners — but the paper doesn't demonstrate any of those harder instantiations. The contribution is the framework and the user-study evidence that predictive feedback works better than reactive feedback in a controlled setting. That's a real result, but the distance to deployment is measured in years and domain-specific engineering, not algorithmic novelty.