You know how restaurant ratings on Yelp tell you the food is good but nothing about whether the kitchen passes health inspection, whether the staff are paid fairly, or whether the supply chain is sustainable? That's the state of LLM benchmarks for governments. MÖVE is a health-inspection framework — it checks the stuff that matters for public procurement but that no leaderboard measures. The committed claim: no existing benchmark evaluates LLMs along the governance dimensions that actually matter for public-sector deployment — energy consumption, provider transparency, and domain-specific political knowledge — and when you do measure these, the rankings scramble. No single model wins across all three. This is a framework paper, not a performance paper, and the distinction matters. The 60-fold energy variation is the headline number and it's genuinely striking. Two models doing roughly comparable work can differ by a factor of 60 in estimated energy consumption, and — crucially — model size alone doesn't explain it. That's a procurement-relevant finding: you can't just look at parameter counts and infer operational cost. The architecture, serving infrastructure, and provider all matter. For a government agency paying the electricity bill and reporting carbon footprint, this is actionable. On transparency, MÖVE finds that information disclosure varies systematically across providers. This is less surprising but still under-documented: some providers tell you what training data they used, others don't. Some disclose model architecture details, others treat it as trade secret. For public institutions bound by transparency and accountability requirements, this asymmetry is a procurement blocker that standard benchmarks completely ignore. The most counterintuitive finding: European models do not exhibit stronger knowledge of German party positions than non-European models. You'd expect a model trained with more European data to know more about European politics. The result suggests that training-data geography and political-knowledge performance don't map the way procurement officers might assume. This undermines the intuitive case for "buy European" on knowledge-quality grounds, though it says nothing about data sovereignty or regulatory alignment. Architecturally, MÖVE is a multi-dimensional evaluation framework — it's not proposing a new model or training method. It sits in the benchmarking-and-evaluation family alongside HELM, LMSYS Chatbot Arena, and the EU AI Act's conformity assessment requirements. The key structural choice is measuring governance dimensions orthogonal to task performance, which makes it more of a procurement decision tool than a research leaderboard. The paper's main limitation is that we only have the abstract and framing — the "first results" language signals early-stage work, accepted as a non-archival paper at Eval4SD (co-located with KONVENS 2026). The energy estimates are exactly that: estimates. The transparency scoring methodology and the political-knowledge test design are where the real methodological scrutiny needs to happen, and we don't have enough detail from this abstract to stress-test them. The obvious next experiment — running MÖVE on actual government tasks with real procurement decision-makers evaluating whether the framework changes their choices — hasn't been done, and the honest read is (a): this is a small team at an early stage, and that kind of field study requires institutional partnerships they're probably still building.
