ZeroBench

Measures: 100 hand-built, adversarially filtered visual reasoning questions (plus 334 subquestions) that were designed to be impossible for multimodal models at release; each question is sampled 5 times and scored by pass@5, pass^5 (all 5 right) and mean pass@1.

Maintainer: Jonathan Roberts et al. (maintained by Jonathan Roberts)
Introduced: 2025
Status: Active
Saturation: Headroom
Typical provenance: Mixed
In scoring set: Yes

Paper Official leaderboard

Leaderboard

Show independent results only

Every result we hold, whoever ran it. Only independent and third-party rows count towards a score.

No results to show.