Imagine you run a restaurant kitchen with two prep cooks — one meticulous but slow, one fast but sloppy. Every dish needs to be checked before it goes out. You could inspect every plate yourself, but that defeats the point of having two cooks. What you actually want is a system that catches the plates where both cooks agree (send those straight out), flags the plates where they obviously disagree (wrong ingredient entirely), and only calls you over for the genuinely ambiguous cases where a trained eye is needed. QuanReview is that system, built for structured text annotations rather than dinner plates. The committed claim: QuanReview is the first open-source, auditable reconciliation pipeline specifically designed for structured span annotations (quantities with units, uncertainty modifiers, event classes) produced by both humans and LLMs. It aligns two annotation streams at character level, applies logged policy decisions to unambiguous mismatches, and routes genuine conflicts to a browser-based adjudication UI. The novelty is not in any single component but in the end-to-end system design that keeps an audit trail. The numbers tell a story about triage efficiency. Applied to a 4,457-record humanitarian benchmark, the system fully auto-merged 8% of documents where annotations already agreed, applied automatic policy decisions to 1,513 more records, and concentrated human attention on 3,131 candidate conflicts — averaging 5.4 conflicts per reviewed document. That means roughly 65% of human review effort was on genuinely contested spans rather than on rubber-stamping agreement. The system didn't eliminate human review; it compressed it. Architecturally, this is a pipeline tool, not a model. Character-level alignment is the load-bearing mechanism — overlapping spans from two sources are matched by character offset, then categorized by an explicit decision policy (e.g., if one side has a span and the other doesn't, auto-accept the present one; if both have spans but disagree on a field, route to human). The campaign manager layer adds multi-annotator assignment with configurable redundancy and computes inter-annotator agreement at both document and span level. Unanimous documents get auto-merged. Everything exports in the original file format so the corrected layer is a drop-in replacement. The integrity picture is honest but limited. This is a system demonstration paper — 6 pages, 2 figures, 4 tables — not an empirical study with hypotheses and statistical tests. The validation is 'we ran it on a real dataset and here are the throughput numbers.' There's no comparison to alternative reconciliation tools (Prodigy, Label Studio's review modes, or custom scripts), no measurement of whether the auto-merge policy introduces systematic errors, and no inter-annotator agreement scores for the human adjudicators themselves. The code is open-source on GitHub, which is the strongest integrity signal here. The gap this fills is real. Anyone who has tried to fold LLM extractions into an existing annotation pipeline knows the pain: LLMs produce spans that are close-but-not-quite aligned with human spans, field values that partially overlap, and confident errors that look like correct annotations until you check. Doing this reconciliation in spreadsheets or custom scripts is fragile, unauditable, and demoralizing. QuanReview systematizes the workflow. It doesn't solve the hard problem (what's the correct annotation?), but it ensures the hard problem is the only one humans spend time on. The honest limitation: this is tooling, not methodology. The paper doesn't advance our understanding of when LLM annotations are trustworthy, what error patterns they exhibit, or how to improve them. It provides plumbing. Good plumbing matters — but the paper's contribution is bounded by the fact that the 8% auto-merge rate and 5.4 conflicts/document figures are specific to one dataset and one LLM extraction stream. Whether those numbers generalize to other domains, other LLMs, or other annotation schemas is an open question the paper doesn't address.
