supermarket

Benchmark

FrontierMath Tiers 1-3 (v1)

Measures: The first private set of 290 expert-written problems from advanced undergraduate to early research mathematics, each with an automatically checkable answer; replaced by v2 on 2026-06-12.

Maintainer
Epoch AI
Introduced
2024
Status
Superseded
Saturation
Saturated
Typical provenance
Independent
In scoring set
No

PaperOfficial leaderboard

Leaderboard

Show all results

Independent and third-party results only.

Leaderboard
ModelAccuracyProvenanceSourceEvaluated
Gemini 1.5 Flash
Google DeepMind
0.0%IndependentEpoch AI7 Mar 2025
Llama 4 Scout
Meta
0.0%IndependentEpoch AI8 Apr 2025
Claude 3.5 Haiku
Anthropic
0.3%IndependentEpoch AI7 Mar 2025
GPT-4o
OpenAI
0.3%IndependentEpoch AI7 Mar 2025
GPT-4o
OpenAI
0.3%IndependentEpoch AI6 Mar 2025
Mistral Large 2.1
Mistral AI
0.3%IndependentEpoch AI6 Mar 2025
Mistral Medium 3
Mistral AI
0.3%IndependentEpoch AI7 May 2025
Grok 2
SpaceXAI (xAI)
0.7%IndependentEpoch AI6 Mar 2025
Llama 4 Maverick
Meta
0.7%IndependentEpoch AI8 Apr 2025
Claude 3.5 Sonnet
Anthropic
1.0%IndependentEpoch AI7 Mar 2025
GPT-4.1 nano
OpenAI
1.0%IndependentEpoch AI14 Apr 2025
Qwen2.5-Max
Alibaba Group
1.0%IndependentEpoch AI2 Apr 2025
o1-mini
OpenAI · High effort
1.4%IndependentEpoch AI6 Mar 2025
DeepSeek-V3
DeepSeek
1.7%IndependentEpoch AI7 Mar 2025
Gemini 2.0 Flash
Google DeepMind
1.7%IndependentEpoch AI9 Mar 2025
o1-mini
OpenAI · Medium effort
1.7%IndependentEpoch AI6 Mar 2025
Qwen-Plus
Alibaba Group
1.7%IndependentEpoch AI14 May 2025
Claude 3.5 Sonnet
Anthropic
2.1%IndependentEpoch AI6 Mar 2025
GPT-5.1
OpenAI · None effort
2.1%IndependentEpoch AI25 Nov 2025
GLM-4.7
Z.ai (Zhipu AI)
2.4%IndependentEpoch AI30 Jan 2026
Grok 3 Mini
SpaceXAI (xAI) · Low effort
2.8%IndependentEpoch AI10 Apr 2025
Claude 3.7 Sonnet
Anthropic · 64K effort
3.1%IndependentEpoch AI13 Mar 2025
Claude 3.7 Sonnet
Anthropic
3.1%IndependentEpoch AI6 Mar 2025
Claude 3.7 Sonnet
Anthropic · 32K effort
3.4%IndependentEpoch AI13 Mar 2025
Grok 3
SpaceXAI (xAI)
3.8%IndependentEpoch AI10 Apr 2025
GLM-4.6
Z.ai (Zhipu AI)
3.8%IndependentEpoch AI8 Dec 2025
Claude 3.7 Sonnet
Anthropic · 16K effort
4.1%IndependentEpoch AI6 Mar 2025
Claude Haiku 4.5
Anthropic
4.1%IndependentEpoch AI16 Oct 2025
Claude Opus 4
Anthropic · 27K effort
4.1%IndependentEpoch AI5 Aug 2025
Claude Sonnet 4
Anthropic
4.1%IndependentEpoch AI4 Jul 2025
Claude Opus 4
Anthropic
4.5%IndependentEpoch AI4 Jul 2025
GPT-4.1 mini
OpenAI
4.5%IndependentEpoch AI14 Apr 2025
Gemini 2.5 Flash
Google DeepMind
4.8%IndependentEpoch AI18 Dec 2025
GPT-4.1
OpenAI
5.5%IndependentEpoch AI14 Apr 2025
Claude Opus 4.1
Anthropic
5.9%IndependentEpoch AI5 Aug 2025
Grok 3 Mini
SpaceXAI (xAI) · High effort
5.9%IndependentEpoch AI10 Apr 2025
Claude Haiku 4.5
Anthropic · 32K effort
5.9%IndependentEpoch AI22 Oct 2025
Qwen3.5-Flash
Alibaba Group
6.2%IndependentEpoch AI12 May 2026
Claude Opus 4.1
Anthropic · 27K effort
7.2%IndependentEpoch AI5 Aug 2025
GPT-5 nano
OpenAI · Medium effort
7.2%IndependentEpoch AI7 Aug 2025
o3-mini
OpenAI · Medium effort
8.1%IndependentEpoch AI6 Mar 2025
GPT-5 nano
OpenAI · High effort
8.3%IndependentEpoch AI30 Oct 2025
Qwen3-235B-A22B-Thinking-2507
Alibaba Group
8.5%IndependentEpoch AI11 Dec 2025
Claude Sonnet 4.5
Anthropic
9.3%IndependentEpoch AI16 Nov 2025
o1
OpenAI · High effort
9.3%IndependentEpoch AI7 Mar 2025
o3
OpenAI · Low effort
9.7%IndependentEpoch AI17 Nov 2025
Gemini 2.5 Pro
Google DeepMind
10.3%IndependentEpoch AI3 Jul 2025
Qwen3.6-Flash
Alibaba Group
10.3%IndependentEpoch AI12 May 2026
o4-mini
OpenAI · Low effort
10.7%IndependentEpoch AI16 Nov 2025
o3-mini
OpenAI · High effort
12.4%IndependentEpoch AI16 Nov 2025
Claude Sonnet 4.5
Anthropic · 59K effort
13.5%IndependentEpoch AI13 Nov 2025
Gemini 2.5 Pro
Google DeepMind
14.1%IndependentEpoch AI24 Nov 2025
Claude Sonnet 4.5
Anthropic · 32K effort
15.2%IndependentEpoch AI16 Nov 2025
GLM-5
Z.ai (Zhipu AI)
16.4%IndependentEpoch AI19 Feb 2026
o3
OpenAI · Medium effort
16.9%IndependentEpoch AI16 Nov 2025
GPT-5.1
OpenAI · Low effort
17.3%IndependentEpoch AI25 Nov 2025
o3
OpenAI · High effort
18.7%IndependentEpoch AI16 Nov 2025
o4-mini
OpenAI · Medium effort
19.0%IndependentEpoch AI13 Nov 2025
Grok 4
SpaceXAI (xAI)
19.7%IndependentEpoch AI13 Nov 2025
Claude Opus 4.5
Anthropic · 16K effort
20.3%IndependentEpoch AI25 Nov 2025
GPT-5 mini
OpenAI · Medium effort
20.3%IndependentEpoch AI13 Nov 2025
Claude Opus 4.5
Anthropic
20.7%IndependentEpoch AI25 Nov 2025
Claude Opus 4.5
Anthropic · 32K effort
20.7%IndependentEpoch AI25 Nov 2025
Qwen3.5-Plus
Alibaba Group
21.0%IndependentEpoch AI14 May 2026
Kimi K2 Thinking
Moonshot AI
21.4%IndependentEpoch AI5 Dec 2025
Qwen3.6-Max-Preview
Alibaba Group
23.1%IndependentEpoch AI27 May 2026
o4-mini
OpenAI · High effort
24.8%IndependentEpoch AI13 Nov 2025
GPT-5.4 nano
OpenAI · High effort
25.9%IndependentEpoch AI15 Apr 2026
Qwen3.6-Plus
Alibaba Group
26.2%IndependentEpoch AI12 May 2026
GPT-5.2
OpenAI · Low effort
26.6%IndependentEpoch AI11 Dec 2025
GPT-5.1
OpenAI · Medium effort
26.9%IndependentEpoch AI17 Nov 2025
GPT-5
OpenAI · Medium effort
27.2%IndependentEpoch AI13 Nov 2025
GPT-5 mini
OpenAI · High effort
27.2%IndependentEpoch AI13 Nov 2025
GPT-5.4 mini
OpenAI · High effort
28.3%IndependentEpoch AI15 Apr 2026
GPT-5.1
OpenAI · High effort
31.0%IndependentEpoch AI13 Nov 2025
Claude Sonnet 4.6
Anthropic · 16K effort
32.4%IndependentEpoch AI20 Feb 2026
GPT-5
OpenAI · High effort
32.4%IndependentEpoch AI13 Nov 2025
GLM-5.1
Z.ai (Zhipu AI)
33.4%IndependentEpoch AI11 May 2026
Gemini 3 Flash
Google DeepMind
35.6%IndependentEpoch AI17 Dec 2025
GPT-5.2
OpenAI · Medium effort
36.9%IndependentEpoch AI11 Dec 2025
Gemini 3.1 Pro
Google DeepMind
36.9%IndependentEpoch AI19 Feb 2026
Gemini 3 Pro
Google DeepMind
37.6%IndependentEpoch AI21 Nov 2025
Claude Opus 4.6
Anthropic
38.3%IndependentEpoch AI5 Feb 2026
Gemini 3.5 Flash
Google DeepMind · High effort
39.0%IndependentEpoch AI22 May 2026
Kimi K2.6
Moonshot AI
39.0%IndependentEpoch AI7 May 2026
Claude Opus 4.6
Anthropic · 64K effort
39.7%IndependentEpoch AI6 Feb 2026
Claude Opus 4.6
Anthropic · 32K effort
40.0%IndependentEpoch AI6 Feb 2026
GPT-5.2
OpenAI · High effort
40.3%IndependentEpoch AI11 Dec 2025
Claude Opus 4.6
Anthropic · Max effort
40.7%IndependentEpoch AI12 Feb 2026
GPT-5.2
OpenAI · Xhigh effort
40.7%IndependentEpoch AI13 Dec 2025
Claude Opus 4.7
Anthropic · Xhigh effort
43.8%IndependentEpoch AI17 Apr 2026
Claude Opus 4.8
Anthropic · Max effort
47.2%IndependentEpoch AI8 Jun 2026
GPT-5.4
OpenAI · Xhigh effort
47.6%IndependentEpoch AI6 Mar 2026
GPT-5.4 Pro
OpenAI · Xhigh effort
50.0%IndependentEpoch AI6 Mar 2026
  • Epoch AI, 'Capabilities & benchmarking', epoch.ai/benchmarks, CC BY 4.0.

Sources