Updated daily at 05:00 UTC
Benchmark
Humanity's Last Exam (HLE)
Measures: 2,500 expert-written, closed-ended questions across 100+ subjects (about 14% need an image); a model gives one final answer plus a confidence, graded for exact-match accuracy by an LLM judge, with calibration error reported alongside.
- Maintainer
- Center for AI Safety (CAIS) and Scale AI
- Introduced
- 2025
- Status
- Active
- Saturation
- Headroom
- Typical provenance
- Mixed
- In scoring set
- Yes
Leaderboard
Independent and third-party results only.
No independent or third-party results yet. Show all results