supermarket

Benchmark

FrontierMath Tier 4 (v1)

Measures: The first private set of 48 exceptionally difficult research-level math problems in FrontierMath's Tier 4; replaced by v2 on 2026-06-12.

Maintainer
Epoch AI
Introduced
2025
Status
Superseded
Saturation
Headroom
Typical provenance
Independent
In scoring set
No

PaperOfficial leaderboard

Leaderboard

Show independent results only

Every result we hold, whoever ran it. Only independent and third-party rows count towards a score.

Leaderboard
ModelAccuracyProvenanceSourceEvaluated
Claude 3.5 Sonnet
Anthropic
0.0%IndependentEpoch AI1 Jul 2025
Claude 3.5 Sonnet
Anthropic
0.0%IndependentEpoch AI1 Jul 2025
Claude Opus 4
Anthropic
0.0%IndependentEpoch AI1 Jul 2025
Claude Sonnet 4
Anthropic
0.0%IndependentEpoch AI1 Jul 2025
GLM-4.7
Z.ai (Zhipu AI)
0.0%IndependentEpoch AI30 Jan 2026
GPT-4.1
OpenAI
0.0%IndependentEpoch AI1 Jul 2025
GPT-5 nano
OpenAI · High effort
0.0%IndependentEpoch AI30 Oct 2025
Grok 3
SpaceXAI (xAI)
0.0%IndependentEpoch AI1 Jul 2025
Kimi K2 Thinking
Moonshot AI
0.0%IndependentEpoch AI4 Dec 2025
Qwen3-235B-A22B-Thinking-2507
Alibaba Group
0.0%IndependentEpoch AI11 Dec 2025
Qwen3.5-Flash
Alibaba Group
0.0%IndependentEpoch AI12 May 2026
Qwen3.6-Flash
Alibaba Group
0.0%IndependentEpoch AI12 May 2026
GPT-5.4 mini
OpenAI · High effort
2.1%IndependentEpoch AI15 Apr 2026
Claude Haiku 4.5
Anthropic · 32K effort
2.1%IndependentEpoch AI22 Oct 2025
Claude Opus 4.5
Anthropic · 16K effort
2.1%IndependentEpoch AI25 Nov 2025
Claude Sonnet 4.5
Anthropic
2.1%IndependentEpoch AI29 Sep 2025
Gemini 2.5 Pro
Google DeepMind
2.1%IndependentEpoch AI3 Jul 2025
GPT-5 nano
OpenAI · Medium effort
2.1%IndependentEpoch AI7 Aug 2025
Grok 4
SpaceXAI (xAI)
2.1%IndependentEpoch AI11 Aug 2025
o3
OpenAI · High effort
2.1%IndependentEpoch AI1 Jul 2025
o4-mini
OpenAI · Medium effort
2.1%IndependentEpoch AI7 Aug 2025
Qwen3.5-Plus
Alibaba Group
2.1%IndependentEpoch AI15 May 2026
GLM-5
Z.ai (Zhipu AI)
2.1%IndependentEpoch AI19 Feb 2026
GLM-4.6
Z.ai (Zhipu AI)
2.1%IndependentEpoch AI8 Dec 2025
Claude Opus 4
Anthropic · 27K effort
4.2%IndependentEpoch AI1 Jul 2025
Claude Opus 4.1
Anthropic · 27K effort
4.2%IndependentEpoch AI5 Aug 2025
Claude Opus 4.5
Anthropic
4.2%IndependentEpoch AI25 Nov 2025
Claude Opus 4.5
Anthropic · 32K effort
4.2%IndependentEpoch AI25 Nov 2025
Claude Sonnet 4.5
Anthropic · 32K effort
4.2%IndependentEpoch AI22 Oct 2025
Gemini 2.5 Flash
Google DeepMind
4.2%IndependentEpoch AI18 Dec 2025
Gemini 2.5 Pro
Google DeepMind
4.2%IndependentEpoch AI3 Jul 2025
Gemini 3 Flash
Google DeepMind
4.2%IndependentEpoch AI17 Dec 2025
GPT-5 mini
OpenAI · Medium effort
4.2%IndependentEpoch AI7 Aug 2025
GPT-5.1
OpenAI · Medium effort
4.2%IndependentEpoch AI17 Nov 2025
o3-mini
OpenAI · High effort
4.2%IndependentEpoch AI1 Jul 2025
Qwen3.6-Max-Preview
Alibaba Group
4.2%IndependentEpoch AI28 May 2026
GPT-5
OpenAI · Medium effort
6.3%IndependentEpoch AI7 Aug 2025
GPT-5 mini
OpenAI · High effort
6.3%IndependentEpoch AI30 Oct 2025
GPT-5.2
OpenAI · Low effort
6.3%IndependentEpoch AI11 Dec 2025
GPT-5.4 nano
OpenAI · High effort
6.3%IndependentEpoch AI15 Apr 2026
o4-mini
OpenAI · High effort
6.3%IndependentEpoch AI1 Jul 2025
Claude Sonnet 4.6
Anthropic · 16K effort
8.3%IndependentEpoch AI21 Feb 2026
Qwen3.6-Plus
Alibaba Group
8.3%IndependentEpoch AI12 May 2026
GLM-5.1
Z.ai (Zhipu AI)
12.5%IndependentEpoch AI12 May 2026
GPT-5
OpenAI · High effort
12.5%IndependentEpoch AI30 Oct 2025
GPT-5.1
OpenAI · High effort
12.5%IndependentEpoch AI13 Nov 2025
Kimi K2.6
Moonshot AI
14.6%IndependentEpoch AI8 May 2026
Claude Opus 4.6
Anthropic
14.6%IndependentEpoch AI5 Feb 2026
Gemini 3.5 Flash
Google DeepMind · High effort
14.6%IndependentEpoch AI25 May 2026
GPT-5 Pro
OpenAI · High effort
14.6%IndependentEpoch AI9 Oct 2025
Gemini 3.1 Pro
Google DeepMind
16.7%IndependentEpoch AI19 Feb 2026
GPT-5.2
OpenAI · Medium effort
16.7%IndependentEpoch AI11 Dec 2025
Gemini 3 Pro
Google DeepMind
18.8%IndependentEpoch AI21 Nov 2025
GPT-5.2
OpenAI · Xhigh effort
18.8%IndependentEpoch AI14 Dec 2025
GPT-5.2
OpenAI · High effort
18.8%IndependentEpoch AI11 Dec 2025
Claude Opus 4.6
Anthropic · 64K effort
20.8%IndependentEpoch AI6 Feb 2026
Claude Opus 4.6
Anthropic · 32K effort
20.8%IndependentEpoch AI6 Feb 2026
Claude Opus 4.6
Anthropic · Max effort
22.9%IndependentEpoch AI12 Feb 2026
Claude Opus 4.7
Anthropic · Xhigh effort
22.9%IndependentEpoch AI17 Apr 2026
GPT-5.4
OpenAI · Xhigh effort
27.1%IndependentEpoch AI6 Mar 2026
Claude Opus 4.8
Anthropic · Max effort
31.3%IndependentEpoch AI8 Jun 2026
  • Epoch AI, 'Capabilities & benchmarking', epoch.ai/benchmarks, CC BY 4.0.

Sources