Treat Your LLM Like a Noisy Sensor: A Review of Classical Methods for Managing AI Fallibility in Autonomous Systems

Review from CMU argues LLMs in robotics should be treated like any other noisy component — with adaptive trust calibration borrowed from decades of sensor fusion and fault-tolerant design.

You already know how to think about this paper if you've ever driven with a GPS that confidently tells you to turn into a lake. You don't throw the GPS away — it's right 98% of the time. But you also don't follow it blindly. Instead, you build a mental model of when it's trustworthy (highways, cities) and when it's not (new construction, rural roads), and you adjust your reliance in real time. That's the entire thesis of this review: LLMs in autonomous systems should be managed the same way engineers have managed noisy sensors, flaky actuators, and unreliable software modules for decades. The committed claim is a synthesis, not a novel algorithm: classical methods from decision theory, estimation, and online learning can be systematically applied to calibrate how much an autonomous system trusts its LLM components. The paper frames this as "adaptive reliance" — the system accumulates evidence about the LLM's performance and dials its trust up or down accordingly. An LLM providing a plan gets weighted differently than one providing a prediction or a judgment, and each role demands a different reliability estimate. The architecture of the argument is deliberately meta. This is not proposing a new model or a benchmark result. It's a review paper from CMU Robotics (Samani and Atkeson) that taxonomizes the ways LLMs participate in autonomous systems — as planners, predictors, classifiers, reward-shapers — and maps each role to an existing classical toolkit. Kalman-style estimation for continuous predictions, Bayesian updating for discrete judgments, multi-armed bandit frameworks for choosing among LLM-generated plans. The key structural move is arguing that the LLM's role determines what you estimate, and the estimation method determines how you adapt reliance. The review identifies two tiers of safety. The first tier is statistical: adaptive reliance reduces the average rate at which LLM errors propagate into system behavior. The second tier is structural: for errors that are catastrophic even when rare, the paper argues you need an independently verified mechanism — a hard constraint, a physics-based check, a human-in-the-loop — that can veto LLM outputs before execution. Without this second tier, you're left with probabilistic bounds on risk, which the paper acknowledges is weaker but still better than uncalibrated trust. What's genuinely useful here is the framing, not the novelty. The paper doesn't claim to have invented adaptive reliance — it claims the robotics and autonomy communities are reinventing wheels that classical control, estimation, and decision theory already built. The reviewed empirical studies are described as "emerging but limited," which is honest. The comparisons that do exist suggest adaptive reliance improves system performance over fixed-trust baselines, but the evidence base is thin. The gap that jumps out is the absence of a unifying benchmark. The paper taxonomizes roles and methods but doesn't propose a standard evaluation protocol — no shared task where you can measure whether Method A calibrates LLM trust better than Method B across diverse failure modes. This is the obvious next step, and it likely wasn't done because it requires building the benchmark infrastructure, not just writing the review. The paper is a map, not a destination, and it's honest about that. For practitioners building LLM-augmented robots or autonomous systems today, the actionable takeaway is concrete: don't treat your LLM as either fully trusted or fully distrusted. Instrument it. Track its reliability per-task and per-context. Use the same estimation machinery you'd use for a sensor with known drift. And for safety-critical decisions, never rely on statistical calibration alone — build a hard veto layer that doesn't depend on the LLM at all.