Imagine you're a restaurant health inspector, but instead of walking into the kitchen yourself, you have to read raw lab reports — columns of bacterial counts, temperature logs, chemical residue numbers — and decide whether a restaurant passes. You'd kill for a dashboard with green and red lights. VERA is that dashboard, but for EU AI Act compliance of large language models. The committed claim: VERA is the first open-source, self-hosted dashboard that operationalizes COMPL-AI's technical interpretation of the EU AI Act, wrapping twelve measurable requirements into a point-and-click interface for non-engineer reviewers, and instrumenting energy consumption at run level. The paper does not claim new benchmarks or new evaluation science — it claims a delivery mechanism that makes existing benchmarks usable by the people who actually approve LLM deployments in regulated institutions. The prior-art ladder here is thin because the comparison class is narrow. COMPL-AI (Guldimann et al., 2024) defined the six principle groups and eighteen requirements. The benchmarks themselves — BBQ, TruthfulQA, BoolQ, and others — are community standard. VERA's contribution is the glue: a Plotly Dash web application that orchestrates benchmark runs, normalizes scores, and presents results visually. The 'baseline' is literally reading raw output files. In a within-subject study (N=11), reviewers completed assessments in 30 seconds via VERA versus 56 seconds from raw files — a 46% time reduction. Usefulness scored 4.36/5 in a post-task survey. These are real numbers, but they measure UX, not science. Architecturally, this is a software engineering contribution, not an ML one. VERA is a Python Plotly Dash app deployed behind the bank's firewall. It calls existing benchmark suites (lm-evaluation-harness and similar) against served open-weight models (Llama-3.1 8B, Mistral 7B v0.3, Qwen2.5 7B, Gemma-2 9B). The energy instrumentation uses CodeCarbon to track GPU consumption per evaluation run. The dashboard normalizes each benchmark score to a 0–1 scale per requirement. No new model training, no new benchmark design, no new evaluation methodology. Integrity deserves careful reading. The user study has N=11 participants from a single organization, within-subject design, no pre-registration. The benchmarks themselves are public and community-standard, which is good. The code and study materials are released publicly. But the core evaluation — 'is the dashboard useful?' — is a small internal UX study, not an independent validation. The 56-to-30-second improvement is averaged across participants reading four model results; variance isn't reported in the abstract. The energy tracking is a feature, not a validated measurement against a ground truth. The milestone question is where this paper reveals its true nature: it's an engineering tool, not a research frontier. The next concrete number isn't about VERA — it's about COMPL-AI. Today, 12 of 18 EU AI Act requirements have measurable benchmarks. The real milestone is getting from 12 to 18, which requires the community to develop benchmarks for the remaining six requirements (likely around robustness, human oversight, and documentation). VERA will grow as that coverage expands, but VERA itself doesn't push that frontier. The obvious experiment not run: deploying VERA in a second organization and measuring whether the UX gains replicate with different reviewers, different compliance cultures, and different model sets. The honest read is (a) — the authors are embedded in one bank and lacked access to a second deployment site. A multi-site study would have substantially strengthened the generalizability claim. The paper also doesn't compare VERA against other compliance dashboards (commercial or internal), which would test whether the specific UI choices matter or whether any dashboard beats raw files.
