Updated daily at 05:00 UTC
Benchmark
MMLU-Pro
Measures: Harder, reasoning-focused MMLU successor: multiple-choice questions across many academic subjects, with 10 answer options instead of 4; scored as accuracy.
- Maintainer
- TIGER-Lab
- Introduced
- 2024
- Status
- Unmaintained
- Saturation
- Saturated
- Typical provenance
- Mixed
- In scoring set
- No
Leaderboard
Every result we hold, whoever ran it. Only independent and third-party rows count towards a score.
| Model | Accuracy | Provenance | Source | Evaluated |
|---|---|---|---|---|
| Qwen2.5-VL-72B-Instruct | 0.6% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Qwen2-1.5B-Instruct | 22.6% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Yi-6B-Chat | 28.8% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Qwen1.5-7B-Chat | 29.1% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Mistral 7B v0.2 | 30.8% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Aya Expanse 8B | 33.7% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| DeepSeekMath 7B Instruct | 35.3% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Command R (03-2024) | 37.9% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Ministral 8B | 37.9% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Qwen1.5-14B-Chat | 38.0% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Yi-1.5-6B-Chat | 38.2% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| EXAONE-3.5-2.4B-Instruct | 39.1% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Llama 3 8B | 41.0% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| DeepSeek-Coder-V2-Lite | 41.6% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Mathstral 7B | 42.0% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Phi-3-mini | 43.9% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Aya Expanse 32B | 45.4% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Yi-1.5-9B-Chat | 46.0% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| EXAONE-3.5-7.8B-Instruct | 46.2% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Phi-3.5-mini | 47.9% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Mistral Small 2.0 | 48.4% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Jamba 1.5 Large | 49.5% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Granite 4.1 3B | 49.8% | Self-reported | Model Card via Hugging Face Community Evals | — |
| Phi-3-medium | 51.9% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Yi-1.5-34B-Chat | 52.3% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Phi-4-mini | 52.8% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Granite 4.1 8B | 56.0% | Self-reported | Model Card via Hugging Face Community Evals | — |
| InternLM3-8B-Instruct | 57.6% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| EXAONE-3.5-32B-Instruct | 58.9% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Hunyuan-Large | 60.2% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Granite 4.1 30B | 64.1% | Self-reported | Model Card via Hugging Face Community Evals | — |
| DeepSeek-V3 | 64.4% | Self-reported | Model Card via Hugging Face Community Evals | — |
| Mistral Large 2.0 | 65.9% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Hunyuan-A13B-Instruct | 67.3% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Mistral Large 2.1 | 67.9% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Qwen3-235B-A22B | 68.2% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| MiMo-V2.5-Pro | 68.5% | Self-reported | Model Card via Hugging Face Community Evals | — |
| Qwen3-4B-Instruct-2507 | 69.6% | Self-reported | Model Card via Hugging Face Community Evals | — |
| Phi-4 | 70.4% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| gpt-oss-20b | 73.6% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| MiniMax-Text-01 | 75.7% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| LongCat-Flash-Lite | 78.3% | Self-reported | Model Card via Hugging Face Community Evals | — |
| ERNIE-4.5-300B-A47B | 78.4% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| MiniMax-M2.5 | 80.1% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| gpt-oss-120b | 80.8% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Qwen3-30B-A3B-Thinking-2507 | 80.9% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| GLM-4.5-Air | 81.4% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| A.X K1 | 81.5% | Self-reported | A.X K1 model card via Hugging Face Community Evals | — |
| Nemotron 3.5 Lightning 30B-A3B | 81.9% | Self-reported | NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16 model card via Hugging Face Community Evals | — |
| MiniMax-M2 | 82.0% | Self-reported | Model Card via Hugging Face Community Evals | — |
| LongCat-Flash-Chat | 82.7% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Seed-OSS-36B-Instruct | 82.7% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Qwen3-235B-A22B-Instruct-2507 | 83.0% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| EXAONE-4.5-33B | 83.3% | Self-reported | Model Card via Hugging Face Community Evals | — |
| Intern-S1 | 83.5% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Nemotron 3 Super 120B-A12B | 83.7% | Self-reported | Model Card via Hugging Face Community Evals | — |
| K-EXAONE-236B-A23B | 83.8% | Self-reported | Model Card via Hugging Face Community Evals | — |
| DeepSeek-R1 | 84.0% | Self-reported | Model Card via Hugging Face Community Evals | — |
| Step 3.5 Flash | 84.4% | Self-reported | Step 3.5 Flash Paper via Hugging Face Community Evals | — |
| GLM-4.5 | 84.6% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Qwen3.6-35B-A3B | 85.2% | Self-reported | Model Card via Hugging Face Community Evals | — |
| GLM-5 | 86.0% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Qwen3.5-27B | 86.1% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Qwen3.6-27B | 86.2% | Self-reported | Model Card via Hugging Face Community Evals | — |
| Solar Open 2 (250B) | 86.2% | Self-reported | Model Card via Hugging Face Community Evals | — |
| Qwen3.5-122B-A10B | 86.7% | Self-reported | EvalEval via Hugging Face Community Evals | — |
| Nemotron 3 Ultra 550B-A55B | 86.8% | Self-reported | Model Card via Hugging Face Community Evals | — |
| Kimi K2.5 | 87.1% | Self-reported | Model Card via Hugging Face Community Evals | — |
| Qwen3.5-397B-A17B | 87.8% | Self-reported | Model Card via Hugging Face Community Evals | — |
| Intern-S2-Preview | 88.0% | Self-reported | Model Card via Hugging Face Community Evals | — |
| MiniMax-M2.1 | 88.0% | Self-reported | EvalEval via Hugging Face Community Evals | — |
- Reported on Hugging Face Community Evals; each result names its author and links its source.