Imagine you hire a tutor who speaks with total authority, never hesitates, never says "I'm not sure." Unknown to you, a quarter of what they teach you is wrong. After the lesson, someone quizzes you. You confidently choose the tutor's wrong answers — and you'd bet money you're right. That's not a thought experiment. That's what happened in this study, and the mechanism it exposes — human deference to confident AI — is the mirror image of the sycophancy problem the AI safety community has been fixated on. The committed claim: when an LLM is consistently confident and wrong, humans absorb its falsehoods at a steady ~65% rate regardless of how much of the content is fabricated, hold those falsehoods with high confidence (8.4/10), and lose the ability to distinguish what they actually know from what they were fed. The authors call this "human-side sycophancy" and introduce a baseline-corrected Human Deference Index (HDI) to measure it. The paper's central provocation is that trustworthiness isn't a property of the model alone — it's a property of the human-AI pair. The experimental architecture is clean and well-controlled. Five GPT-5-mini assistants were given identical system prompts except for falsification level (0%, 25%, 50%, 75%, 100% of facts replaced by plausible falsehoods). Each of 32 participants chatted with all five (within-subjects), then took a surprise four-option quiz per topic with confidence ratings. The planted falsehood was always one of the four options. Across 1,040 answers, scores dropped from 7.4/10 (no falsification) to 2.9/10 (full falsification), and the planted option was chosen on 60% of falsified items vs 11% of correctly-taught ones — an odds ratio of 12.5. Crucially, the ~65% absorption rate per false claim was roughly constant across falsification levels, meaning participants weren't developing better filters as the assistant got worse. The confidence calibration collapse is the sharpest finding. For correctly-taught facts, participants' confidence discriminated right from wrong answers at an AUROC of 0.70 — reasonable metacognition. For absorbed falsehoods, that discrimination fell to chance. Participants weren't just learning wrong things; they had lost the internal signal that normally tells them "I'm guessing." Post-task trust ratings barely budged across conditions, confirming the calibration failure extended to explicit assessments of the source. A counterintuitive behavioral pattern emerged: participants returned to the chat more frequently when the assistant was worse, but returning correlated with more absorption, not less. Thirty percent of all turns involved pasting quiz questions back to the same assistant being evaluated — the fox guarding the henhouse. The assistant itself was almost never sycophantic and never conceded under pushback, ruling out the usual model-side explanation. The integrity picture is mixed. N=32 is small for the strength of the claims, though the within-subjects design and 1,040 total answers add statistical power. The use of a single model family (GPT-5-mini) limits generalizability. The science topics were chosen to be unfamiliar, which maximizes deference but may not reflect how users interact with AI on topics where they have prior knowledge. The authors are transparent about these limitations. No pre-registration is mentioned, and no code release is indicated, but the experimental protocol is described in enough detail for replication. The practical implications cut deep for anyone building learning tools, copilots, or agentic systems. If users cannot calibrate their own confidence when interacting with a confident AI, then reliability must be made legible at the claim level — not through post-hoc trust ratings or global confidence scores. The authors' prescription (per-claim reliability signals in learning interfaces) is the minimum viable response, but the harder question is whether the human capacity for deference is a feature of the interaction pattern that no interface design can fully fix. The next milestone is replication at scale with diverse model families and domain-expert users who bring strong priors.
