Medical AI Models Say They're Confident Even When You Scramble the X-Ray

Four mammography VLMs tested for coherence between answers, confidence, and explanations. Confidence stays high even on perturbed images. Accuracy alone hides dangerous failure modes.

You know that friend who answers every trivia question with total confidence — wrong or right, same energy, same tone? Now imagine that friend is reading your mammogram. That's the core finding here: vision-language models applied to breast imaging maintain high stated confidence even when the input image is perturbed, shuffled, or degraded. The answer changes, but the swagger doesn't. The paper evaluates four VLMs on two mammography tasks — breast density classification and BI-RADS categorization — across two datasets. The key move is not measuring accuracy (the field already does that obsessively) but measuring *coherence* across three outputs: the model's answer, its stated confidence score, and its free-text explanation. The authors run controlled perturbations — changing the question phrasing, shuffling multiple-choice option order, and modifying the images themselves — then track whether the three outputs move together or drift apart. The results are uncomfortable. Confidence scores do not meaningfully decrease when images are altered, which means the model's self-reported certainty is decoupled from its actual input processing. Shuffling the order of multiple-choice options changes most models' answers — a classic positional bias that has been documented in text-only LLMs but is shown here to persist in the multimodal medical setting. And in some cases, the model's own written explanation directly contradicts its final answer selection, a failure mode invisible to any evaluation framework that only checks the label. This matters because the medical AI deployment pipeline is moving fast toward systems that produce not just classifications but justifications — the reasoning chain a clinician is supposed to review before acting. If the reasoning is incoherent with the answer, the justification is theater. A clinician who trusts the explanation is being actively misled in ways that a simple wrong-answer would not achieve. The study design is straightforward and well-motivated but limited in scale. Four models, two datasets, two tasks. The perturbation battery is sensible (question, option order, image) but not exhaustive — adversarial perturbations, subtle lesion removal, or contrast manipulation would push the analysis further. The authors do not name the four VLMs in the abstract (they may in the full paper), which makes external benchmarking harder to assess. The deeper contribution is methodological: the paper argues that *coherence* should be a first-class evaluation dimension alongside accuracy for medical VLMs. This is a real and underappreciated point. The FDA pathway for AI-as-medical-device currently focuses on sensitivity/specificity — if coherence metrics become standard, it changes what vendors must demonstrate before deployment. The paper is a diagnostic, not a solution, but the diagnostic is well-aimed. The honest read: this is a well-constructed evaluation paper that surfaces a genuine and important failure mode, but it is small in scale and does not propose fixes. Its value is in framing the right question — whether medical AI systems are internally consistent, not just statistically accurate — more than in delivering a definitive answer.