Imagine you build a smoke detector and test it only in your kitchen. It works great — beeps at toast, ignores steam, passes every inspection. Then you install it in eleven different kitchens across four countries, and you discover something uncomfortable: it's not that individual kitchens are the problem, it's that kitchens with gas stoves behave fundamentally differently from kitchens with electric ones. The detector wasn't broken in your kitchen. Your testing methodology was broken by only having one kitchen. That's the structural mechanism of this paper. Nemecek, Zafar, Ganguly, Singh, Chaudhary, and Ayday argue that LLM watermarking — the practice of embedding statistical signals in generated text so you can later prove a model wrote it — has been evaluated almost exclusively on English, using each scheme's own detection threshold and a narrow quality-measurement regime. When you deploy these watermarks across eleven languages spanning four scripts and eight typological families, evaluation-design choices that are invisible on English suddenly determine whether you conclude the watermark works or fails. The committed claim: cross-lingual fairness gaps in watermarking are predominantly between-family on the typological partition, meaning they're structural to language properties rather than idiosyncratic to particular languages. This is not a 'Hindi is harder than English' finding — it's a 'agglutinative languages are systematically disadvantaged relative to analytic ones' finding. The difference matters enormously for how you'd fix the problem. The paper tests six watermarking schemes across three open-weight generators (base and instruction-tuned), proposing a four-component evaluation framework: empirically calibrated detection thresholds per deployment context, a threshold-independent companion metric that separates calibration failures from detection failures, three disjoint quality measurement paradigms (distributional, paired-semantic, and reference-perplexity), and a generalized-entropy decomposition of cross-language disparity over a typological family partition. The architecture here is evaluation methodology, not a new watermarking algorithm. The paper belongs to the fairness-auditing family — it builds measurement infrastructure rather than proposing a fix. The generalized-entropy decomposition is the key structural choice: by partitioning languages into typological families and decomposing total disparity into between-family and within-family components, the paper can distinguish 'this watermark fails on Turkish specifically' from 'this watermark fails on agglutinative languages generally.' That decomposition is what elevates this above a simple multilingual benchmark. On integrity, the design is solid for an evaluation paper. Six schemes, three generators, eleven languages, four scripts, eight families, two tuning regimes — that's a serious combinatorial test matrix. The three disjoint quality paradigms guard against the scenario where one measurement says quality is fine while another says it's degraded. The threshold-independent companion metric is a clever move: it prevents the authors from accidentally reporting 'watermark detected' when the real story is 'we set the threshold wrong for this language.' The main integrity question is whether eleven languages adequately represent eight typological families — some families likely get only one or two representatives, which limits statistical power for within-family claims. The field fight here is real and consequential. As LLM watermarking moves toward regulatory adoption — the EU AI Act and similar frameworks are pushing for provenance signals — the question of whether watermarks work equitably across languages isn't academic. If your watermark reliably detects AI-generated English but fails on Bengali or Turkish, you've built a system that effectively only governs English-language AI output. The paper lands squarely on the side that says current evaluation practices are inadequate and systematically mask cross-lingual harm. The obvious successor experiment: actually proposing a watermarking scheme that addresses the typological disparity. The authors built the diagnostic but not the cure. My read is option (c) — they're saving the intervention paper for the next submission. You don't build this much evaluation infrastructure without planning to use it to validate your own fix. The milestone to watch: watermarking parity across typological families at less than 10% relative disparity in detection rates. We're not there, and this paper doesn't tell us exactly how far away we are in absolute numbers, but it gives the field the ruler it needs to measure progress.
