Updated daily at 05:00 UTC
- Released
- 20 Jan 2025
- Status
- Legacy
- Context
- 131.1K
- Max output
- —
- Input per 1M tokens
- —
- Output per 1M tokens
- —
- Open weights
- Yes
- License
- mit
- Parameters
- 671B, 37B active
- Knowledge cutoff
- —
- Reasoning
- Yes
- Modalities
- text → text
Category scores
Out of 100, from independent and third-party results only, by method category-2026-09-29.1.
| Category | Score | Rank | Coverage |
|---|---|---|---|
| Knowledge & reasoning | 29.0 | Not ranked | 1 of 4 |
| Coding & software engineering | 71.4 | Not ranked | 1 of 5 |
| Agentic tool use | 35.0 | 9 | 2 of 2 |
| Human preference | 53.1 | Not ranked | 1 of 5 |
Benchmark results
Every result we hold for this model, with who ran it and where it was published.
Knowledge & reasoning
| Benchmark | Metric | Value | Provenance | Setting | Evaluated | Source |
|---|---|---|---|---|---|---|
| MMLU-Pro | Accuracy | 84.0% | Self-reported | — | — | Model Card via Hugging Face Community Evals |
| SimpleBench | Score (AVG@5) | 40.8% | Third party | — | — | SimpleBench Leaderboard via Epoch AI |
| SimpleBench | Score (AVG@5) | 30.9% | Third party | — | — | SimpleBench Leaderboard via Epoch AI |
Scientific reasoning
| Benchmark | Metric | Value | Provenance | Setting | Evaluated | Source |
|---|---|---|---|---|---|---|
| GPQA Diamond | Accuracy | 76.3% | Independent | — | 29 May 2025 | Epoch AI |
| GPQA Diamond | Accuracy | 71.7% | Independent | — | 26 May 2025 | Epoch AI |
| GPQA Diamond | Accuracy | 71.5% | Self-reported | — | — | Model Card via Hugging Face Community Evals |
Mathematics
| Benchmark | Metric | Value | Provenance | Setting | Evaluated | Source |
|---|---|---|---|---|---|---|
| MATH Level 5 | Accuracy | 96.6% | Independent | — | 29 May 2025 | Epoch AI |
| MATH Level 5 | Accuracy | 93.1% | Independent | — | 31 Jan 2025 | Epoch AI |
| OTIS Mock AIME 2024-2025 | Accuracy | 66.4% | Independent | — | 29 May 2025 | Epoch AI |
| OTIS Mock AIME 2024-2025 | Accuracy | 53.3% | Independent | — | 26 Feb 2025 | Epoch AI |
Coding & software engineering
| Benchmark | Metric | Value | Provenance | Setting | Evaluated | Source |
|---|---|---|---|---|---|---|
| Aider Polyglot | Percent correct | 71.4% | Third party | — | — | https://aider.chat/docs/leaderboards/ via Epoch AI |
| Aider Polyglot | Percent correct | 56.9% | Third party | — | — | https://aider.chat/docs/leaderboards/ via Epoch AI |
| METR Time Horizons | Average task success (Epoch's column) | 53.8% | Third party | — | — | METR - Measuring AI Ability to Complete Long Tasks via Epoch AI |
| METR Time Horizons | Average task success (Epoch's column) | 51.9% | Third party | — | — | METR - Measuring AI Ability to Complete Long Tasks via Epoch AI |
| WeirdML (v2) | Average accuracy | 41.6% | Third party | — | — | https://htihle.github.io/weirdml.html via Epoch AI |
| WeirdML (v2) | Average accuracy | 36.5% | Third party | — | — | https://htihle.github.io/weirdml.html via Epoch AI |
Agentic tool use
| Benchmark | Metric | Value | Provenance | Setting | Evaluated | Source |
|---|---|---|---|---|---|---|
| BALROG | Average progress | 34.9% | Third party | — | — | https://balrogai.com/ via Epoch AI |
| Deep Research Bench (FutureSearch) | Average score | 35.1% | Third party | — | — | DeepResearchBench Leaderboard via Epoch AI |
Long context
| Benchmark | Metric | Value | Provenance | Setting | Evaluated | Source |
|---|---|---|---|---|---|---|
| Fiction.LiveBench | % correct at 16K tokens (Epoch's column) | 75.0% | Third party | — | — | Fiction.live leaderboard via Epoch AI |
| Fiction.LiveBench | % correct at 16K tokens (Epoch's column) | 69.4% | Third party | — | — | Fiction.live leaderboard via Epoch AI |
Writing
| Benchmark | Metric | Value | Provenance | Setting | Evaluated | Source |
|---|---|---|---|---|---|---|
| LLM Creative Story-Writing Benchmark V1 (Lech Mazur) | Mean rubric score (percent of the 10-point scale) | 83.0% | Third party | — | — | lechmazur/writing Github repository via Epoch AI |
| LLM Creative Story-Writing Benchmark V1 (Lech Mazur) | Mean rubric score (percent of the 10-point scale) | 81.9% | Third party | — | — | lechmazur/writing Github repository via Epoch AI |
Human preference
| Benchmark | Metric | Value | Provenance | Setting | Evaluated | Source |
|---|---|---|---|---|---|---|
| Text Arena (arena.ai) | Arena score (Elo scale) | 1422 | Independent | — | 25 Sep 2026 | Arena |
| Text Arena (arena.ai) | Arena score (Elo scale) | 1398 | Independent | — | 25 Sep 2026 | Arena |
| Text Arena (arena.ai) | Votes | 18524 | Independent | — | 25 Sep 2026 | Arena |
| Text Arena (arena.ai) | Votes | 18065 | Independent | — | 25 Sep 2026 | Arena |
Other DeepSeek-R versions
- DeepSeek-R1-Zero20 Jan 2025
- DeepSeek-R1-Lite-Preview20 Nov 2024