Updated daily at 05:00 UTC
Benchmark
HLE-Diamond
Measures: A cleaned 1,000-question subset of Humanity's Last Exam (500 reasoning, 500 knowledge) answered closed-book; scored as accuracy by an LLM judge, with per-split scores and calibration error.
- Maintainer
- Center for AI Safety (CAIS) and Scale AI
- Introduced
- 2026
- Status
- Active
- Saturation
- Headroom
- Typical provenance
- Independent
- In scoring set
- No
Not scored: Cleaned 1,000-question subset of humanitys-last-exam; scoring both would count HLE twice. Swap it in for HLE once independent coverage matches.
Leaderboard
Independent and third-party results only.
No independent or third-party results yet. Show all results