Updated daily at 05:00 UTC
Benchmark
MathArena (ArXivMath, BrokenArXiv, ArXivLean)
Measures: Fresh research-level math from recent arXiv papers: ArXivMath asks for exact answers to results from new papers, BrokenArXiv tests whether a model refuses to 'prove' subtly false statements, ArXivLean requires Lean 4 proofs; ranked by expected performance, an IRT-adjusted average over all non-deprecated questions.
- Maintainer
- SRI Lab, ETH Zurich and INSAIT
- Introduced
- 2025
- Status
- Active
- Saturation
- Near-ceiling
- Typical provenance
- Independent
- In scoring set
- Yes
Leaderboard
Independent and third-party results only.
No independent or third-party results yet. Show all results