Updated daily at 05:00 UTC
Benchmark
MATH Level 5
Measures: The 1,324 hardest (level 5) problems of the MATH test set, drawn from competitions such as the AMC 10, AMC 12 and AIME; scored on the final answer.
- Maintainer
- Epoch AI (dataset by Hendrycks et al.)
- Introduced
- 2021
- Status
- Active
- Saturation
- Saturated
- Typical provenance
- Independent
- In scoring set
- No
Leaderboard
Independent and third-party results only.
| Model | Accuracy | Provenance | Source | Evaluated |
|---|---|---|---|---|
| Llama 2 70B | 3.3% | Independent | Epoch AI | 27 Jan 2025 |
| Mistral 7B v0.3 | 3.6% | Independent | Epoch AI | 27 Jan 2025 |
| Mistral 7B v0.3 | 3.7% | Independent | Epoch AI | 27 Jan 2025 |
| Yi-34B-Chat | 5.1% | Independent | Epoch AI | 27 Jan 2025 |
| Llama 3 8B | 6.1% | Independent | Epoch AI | 27 Jan 2025 |
| DeepSeek LLM 67B Chat | 6.4% | Independent | Epoch AI | 27 Jan 2025 |
| Mixtral 8x7B | 9.3% | Independent | Epoch AI | 27 Jan 2025 |
| Mixtral 8x7B | 10.0% | Independent | Epoch AI | 27 Jan 2025 |
| Mistral NeMo 12B | 10.8% | Independent | Epoch AI | 27 Jan 2025 |
| Gemini 1.0 Pro | 11.2% | Independent | Epoch AI | 27 Jan 2025 |
| GPT-3.5 Turbo | 11.6% | Independent | Epoch AI | 27 Jan 2025 |
| DBRX Instruct | 11.7% | Independent | Epoch AI | 27 Jan 2025 |
| Claude 2 | 11.7% | Independent | Epoch AI | 27 Jan 2025 |
| Ministral 3B | 14.4% | Independent | Epoch AI | 27 Jan 2025 |
| Claude 3 Haiku | 14.9% | Independent | Epoch AI | 27 Jan 2025 |
| Ministral 8B | 14.9% | Independent | Epoch AI | 27 Jan 2025 |
| GPT-3.5 Turbo | 15.9% | Independent | Epoch AI | 27 Jan 2025 |
| Phi-3-medium | 17.6% | Independent | Epoch AI | 31 Jan 2025 |
| Claude 3 Sonnet | 18.2% | Independent | Epoch AI | 27 Jan 2025 |
| Gemma 2 9B | 21.0% | Independent | Epoch AI | 27 Jan 2025 |
| Llama 3 70B | 22.6% | Independent | Epoch AI | 27 Jan 2025 |
| Hermes 2 Theta Llama 3 70B | 22.7% | Independent | Epoch AI | 27 Jan 2025 |
| Llama 3.1 8B | 22.9% | Independent | Epoch AI | 27 Jan 2025 |
| GPT-4 | 23.0% | Independent | Epoch AI | 27 Jan 2025 |
| Mixtral 8x22B | 24.2% | Independent | Epoch AI | 27 Jan 2025 |
| Mistral Large 1.0 | 24.5% | Independent | Epoch AI | 27 Jan 2025 |
| Gemini 1.5 Flash | 25.1% | Independent | Epoch AI | 27 Jan 2025 |
| Yi-1.5-34B-Chat | 25.5% | Independent | Epoch AI | 27 Jan 2025 |
| WizardLM-2 8x22B | 25.7% | Independent | Epoch AI | 27 Jan 2025 |
| Gemma 2 27B | 27.9% | Independent | Epoch AI | 27 Jan 2025 |
| GPT-4 Turbo | 35.4% | Independent | Epoch AI | 27 Jan 2025 |
| Llama 3.1 70B | 36.7% | Independent | Epoch AI | 27 Jan 2025 |
| Claude 3 Opus | 37.5% | Independent | Epoch AI | 27 Jan 2025 |
| Qwen2-72B-Instruct | 39.1% | Independent | Epoch AI | 27 Jan 2025 |
| Llama 3.2 90B Vision | 39.4% | Independent | Epoch AI | 27 Jan 2025 |
| GPT-4 Turbo | 40.0% | Independent | Epoch AI | 27 Jan 2025 |
| Gemini 1.5 Pro | 40.7% | Independent | Epoch AI | 27 Jan 2025 |
| Llama 3.3 70B | 41.6% | Independent | Epoch AI | 27 Jan 2025 |
| Mistral Large 2.0 | 44.8% | Independent | Epoch AI | 27 Jan 2025 |
| Mistral Small 3.0 | 44.8% | Independent | Epoch AI | 30 Jan 2025 |
| Claude 3.5 Haiku | 46.4% | Independent | Epoch AI | 12 Mar 2025 |
| GPT-4 Turbo | 46.7% | Independent | Epoch AI | 27 Feb 2025 |
| Mistral Small 3.1 | 46.8% | Independent | Epoch AI | 18 Mar 2025 |
| GPT-4o | 49.8% | Independent | Epoch AI | 5 Feb 2025 |
| Llama 3.1 405B | 49.8% | Independent | Epoch AI | 27 Jan 2025 |
| Mistral Large 2.1 | 50.3% | Independent | Epoch AI | 25 Feb 2025 |
| GPT-4o | 51.0% | Independent | Epoch AI | 27 Jan 2025 |
| Claude 3.5 Sonnet | 51.7% | Independent | Epoch AI | 27 Jan 2025 |
| GPT-4o mini | 52.6% | Independent | Epoch AI | 27 Jan 2025 |
| GPT-4o | 53.3% | Independent | Epoch AI | 27 Jan 2025 |
| Qwen2.5-32B-Instruct | 56.1% | Independent | Epoch AI | 30 Jan 2025 |
| Qwen-Turbo | 56.2% | Independent | Epoch AI | 7 Apr 2025 |
| Claude 3.5 Sonnet | 56.9% | Independent | Epoch AI | 27 Jan 2025 |
| Gemini 1.5 Flash | 61.9% | Independent | Epoch AI | 27 Jan 2025 |
| Llama 4 Scout | 62.3% | Independent | Epoch AI | 8 Apr 2025 |
| Qwen2.5-72B-Instruct | 63.2% | Independent | Epoch AI | 27 Jan 2025 |
| Grok 2 | 63.5% | Independent | Epoch AI | 17 Feb 2025 |
| DeepSeek-V3 | 64.9% | Independent | Epoch AI | 27 Jan 2025 |
| Phi-4 | 64.9% | Independent | Epoch AI | 31 Jan 2025 |
| Qwen-Plus | 65.3% | Independent | Epoch AI | 7 Apr 2025 |
| Qwen2.5-Max | 67.2% | Independent | Epoch AI | 1 Apr 2025 |
| Claude 3.7 Sonnet | 68.2% | Independent | Epoch AI | 24 Feb 2025 |
| Qwen3-235B-A22B | 68.9% | Independent | Epoch AI | 3 Jun 2025 |
| GPT-4.1 nano | 70.0% | Independent | Epoch AI | 14 Apr 2025 |
| Gemini 1.5 Pro | 70.4% | Independent | Epoch AI | 27 Jan 2025 |
| Llama 4 Maverick | 73.0% | Independent | Epoch AI | 8 Apr 2025 |
| Gemma 3 27B | 74.0% | Independent | Epoch AI | 13 Mar 2025 |
| GPT-4.5 | 78.6% | Independent | Epoch AI | 28 Feb 2025 |
| Mistral Medium 3 | 81.6% | Independent | Epoch AI | 7 May 2025 |
| o1-preview | 81.6% | Independent | Epoch AI | 27 Jan 2025 |
| Gemini 2.0 Flash | 82.2% | Independent | Epoch AI | 6 Feb 2025 |
| GPT-4.1 | 83.0% | Independent | Epoch AI | 14 Apr 2025 |
| Gemini 2.0 Pro | 83.5% | Independent | Epoch AI | 6 Feb 2025 |
| o1-mini OpenAI · Medium effort | 84.3% | Independent | Epoch AI | 27 Jan 2025 |
| Claude Sonnet 4 | 84.4% | Independent | Epoch AI | 22 May 2025 |
| Claude Opus 4 | 85.0% | Independent | Epoch AI | 22 May 2025 |
| Claude 3.7 Sonnet Anthropic · 16K effort | 86.3% | Independent | Epoch AI | 26 Feb 2025 |
| Claude Haiku 4.5 | 86.9% | Independent | Epoch AI | 16 Oct 2025 |
| DeepSeek-R1-Distill-Qwen-14B | 87.1% | Independent | Epoch AI | 11 Mar 2025 |
| GPT-4.1 mini | 87.3% | Independent | Epoch AI | 14 Apr 2025 |
| Grok 3 Mini SpaceXAI (xAI) · High effort | 88.1% | Independent | Epoch AI | 10 Apr 2025 |
| Grok 3 | 88.7% | Independent | Epoch AI | 10 Apr 2025 |
| o1-mini OpenAI · High effort | 89.2% | Independent | Epoch AI | 13 Feb 2025 |
| DeepSeek-R1-Distill-Llama-70B | 89.9% | Independent | Epoch AI | 10 Mar 2025 |
| Claude 3.7 Sonnet Anthropic · 32K effort | 90.0% | Independent | Epoch AI | 12 Mar 2025 |
| Grok 3 Mini SpaceXAI (xAI) · Low effort | 90.9% | Independent | Epoch AI | 10 Apr 2025 |
| Claude 3.7 Sonnet Anthropic · 64K effort | 91.2% | Independent | Epoch AI | 13 Mar 2025 |
| DeepSeek-R1 | 93.1% | Independent | Epoch AI | 31 Jan 2025 |
| o1 OpenAI · Medium effort | 94.4% | Independent | Epoch AI | 27 Jan 2025 |
| o1 OpenAI · High effort | 94.7% | Independent | Epoch AI | 13 Feb 2025 |
| GPT-5 nano OpenAI · High effort | 94.9% | Independent | Epoch AI | 20 Aug 2025 |
| o3-mini OpenAI · Medium effort | 95.2% | Independent | Epoch AI | 31 Jan 2025 |
| GPT-5 nano OpenAI · Medium effort | 95.2% | Independent | Epoch AI | 20 Aug 2025 |
| Gemini 2.5 Pro | 95.6% | Independent | Epoch AI | 7 May 2025 |
| Gemini 2.5 Pro | 95.9% | Independent | Epoch AI | 8 May 2025 |
| Claude Haiku 4.5 Anthropic · 32K effort | 96.4% | Independent | Epoch AI | 22 Oct 2025 |
| o3-mini OpenAI · High effort | 96.5% | Independent | Epoch AI | 13 Feb 2025 |
| DeepSeek-R1 | 96.6% | Independent | Epoch AI | 29 May 2025 |
| GPT-5 mini OpenAI · Medium effort | 96.8% | Independent | Epoch AI | 20 Aug 2025 |
| Qwen3-Max | 97.1% | Independent | Epoch AI | 9 Oct 2025 |
| Claude Sonnet 4.5 Anthropic · 32K effort | 97.7% | Independent | Epoch AI | 21 Oct 2025 |
| o3 OpenAI · High effort | 97.8% | Independent | Epoch AI | 16 Apr 2025 |
| o4-mini OpenAI · High effort | 97.8% | Independent | Epoch AI | 16 Apr 2025 |
| GPT-5 mini OpenAI · High effort | 97.8% | Independent | Epoch AI | 30 Oct 2025 |
| GPT-5 OpenAI · Medium effort | 97.9% | Independent | Epoch AI | 20 Aug 2025 |
| GPT-5 OpenAI · High effort | 98.1% | Independent | Epoch AI | 29 Oct 2025 |
- Epoch AI, 'Capabilities & benchmarking', epoch.ai/benchmarks, CC BY 4.0.