HLE-Diamond

Measures: A cleaned 1,000-question subset of Humanity's Last Exam (500 reasoning, 500 knowledge) answered closed-book; scored as accuracy by an LLM judge, with per-split scores and calibration error.

Maintainer: Center for AI Safety (CAIS) and Scale AI
Introduced: 2026
Status: Active
Saturation: Headroom
Typical provenance: Independent
In scoring set: No

Not scored: Cleaned 1,000-question subset of humanitys-last-exam; scoring both would count HLE twice. Swap it in for HLE once independent coverage matches.

Paper Official leaderboard

Leaderboard

Show independent results only

Every result we hold, whoever ran it. Only independent and third-party rows count towards a score.

No results to show.