You know that friend who confidently tells you the answer to a trivia question, gets it right about 70% of the time, but if you ask them the same question tomorrow they give a different answer a quarter of the time? Now imagine that friend is making decisions about where your government allocates healthcare funding. That's the core problem this paper exposes. Louise Yorke tested eight configurations of GPT-4o classifying 1,200 UK Prime Minister's Questions into 21 policy topics — a real-world classification task with a human-coded ground truth. The headline accuracy numbers look respectable: 68.2% to 70.3% across configurations. But the instability numbers are the story. Run the same model on the same questions twice and 24.3% of items get different labels. Topic share rankings shift. The changes aren't random noise — they're systematic enough to change the policy picture you'd draw from the data. More damning is the agreement structure. The eight GPT-4o configurations agree more with each other than with human coders. This is the AI equivalent of everyone in a meeting nodding along — consensus without correctness. If you validated one AI output by checking it against another AI output (a practice the paper implies is tempting and common), you'd conclude everything was fine. The shared errors would be invisible. The paper's most operationally alarming finding concerns review routing: the proportion of items flagged for human review ranged from 0.9% to 41.0% depending on how score extraction was implemented. Same model, same data, same task — but one configuration sends almost nothing to humans while another sends nearly half. The 'human in the loop' safeguard is only as good as the threshold nobody is standardizing. Yorke then audits 37 UK and EU policy documents on AI governance. Both jurisdictions expect testing and oversight, but neither consistently requires checking for instability (same input, different output), shared errors across configurations, selective evaluation, or review routing sensitivity. The UK does better on stability — some guidance explicitly mentions it — but places the checking burden on public bodies themselves. The EU puts substantial testing duties on providers while leaving them to choose their own acceptance thresholds, which is a bit like telling students to grade their own exams with their own rubric. The methodological contribution here is framing AI outputs as evidence that must meet measurement-theory standards — validity, reliability, and fitness for purpose — rather than just hitting an accuracy threshold. This reframes the governance question from 'is the model good enough?' to 'has the organization justified reliance through appropriate checks?' That distinction matters because aggregate accuracy can mask systematic failures that change substantive conclusions. This is not a paper about whether GPT-4o is good at text classification. It's a paper about what 'good enough' means when AI evidence shapes public policy, and about the gap between what governance frameworks require and what reliability actually demands. The 24.3% instability rate and the 0.9%-to-41.0% review-routing swing are the numbers that should keep AI governance officers awake at night.
