Imagine you're a teacher who just assigned a proof-based exam to 500 students — except every student writes in a different language. Some submit essays in English, some write Python scripts, some turn in answers in a formal logic language you barely read. Now imagine you also can't trust your AI teaching assistant, because it marks wrong proofs as correct 23% of the time. That's the evaluation crisis at the heart of this paper. The Ramanujan Challenge is a benchmark of research-level number theory problems — concrete formulas for mathematical constants that can be verified numerically but whose proofs may require non-obvious mathematics. It includes unpublished results and open conjectures. The challenge attracted submissions spanning natural language proofs, Computer Algebra System (CAS) code, and Lean formalizations. That multi-format diversity is the whole point: real mathematical work doesn't arrive in a single tidy format. The headline result is a two-part contribution. First, the challenge itself produced genuine mathematical output: correct submissions conceptually distinct from the official solutions, and a proof of one of the published open conjectures. This isn't toy math — it's the kind of result that matters to working number theorists. Second and arguably more important, the authors built an AI-assisted evaluation pipeline to judge these diverse proof formats at scale, because one-shot LLM judgment simply couldn't handle it. The pipeline's numbers tell the story: false-positive rates on natural language proofs dropped from 23% to 6%, and judgment accuracy on supporting CAS code improved from 46% to 58%. Those baseline numbers are sobering — a naive LLM grader marks nearly a quarter of wrong natural-language proofs as correct and is basically coin-flipping on code. The pipeline doesn't solve the problem entirely (58% code accuracy is still painful), but it demonstrates that multi-stage, format-aware evaluation is a tractable engineering problem, not a theoretical impossibility. Architecturally, this is pipeline engineering over a heterogeneous proof corpus, not a single-model advance. The key design insight is decomposing evaluation into semantic and logical correctness stages, applied differently depending on whether the input is prose, code, or formal Lean. This is closer to a compiler toolchain than to a single neural evaluator — each format gets a different verification pathway. The integrity picture is mixed. The benchmark itself is well-designed: problems with numerical ground truth provide a clean correctness signal. But the evaluation pipeline is validated against human expert judgment on the same submissions it was designed for, creating some circularity. There's no independent test on a held-out challenge or cross-domain benchmark. The false-positive reduction is real and meaningful, but the 58% code accuracy number suggests the pipeline still has serious failure modes that aren't fully characterized. The bigger implication is methodological. As AI systems generate more research-level mathematics, the bottleneck shifts from generation to evaluation. This paper is an early attempt to industrialize proof checking across formats — and the fact that even their improved pipeline still fails substantially on code evaluation tells you how far the field has to go.
