Updated daily at 05:00 UTC
- Released
- 23 Jul 2024
- Status
- Legacy
- Context
- 131.1K
- Max output
- —
- Input per 1M tokens
- —
- Output per 1M tokens
- —
- Open weights
- Yes
- License
- llama-3.1
- Parameters
- 405.85B
- Knowledge cutoff
- Dec 2023
- Reasoning
- No
- Modalities
- text → text
Category scores
Out of 100, from independent and third-party results only, by method category-2026-09-29.1.
| Category | Score | Rank | Coverage |
|---|---|---|---|
| Knowledge & reasoning | 7.6 | Not ranked | 1 of 4 |
| Professional knowledge work | 7.4 | 9 | 1 of 1 |
Benchmark results
Every result we hold for this model, with who ran it and where it was published.
Knowledge & reasoning
| Benchmark | Metric | Value | Provenance | Setting | Evaluated | Source |
|---|---|---|---|---|---|---|
| AI2 Reasoning Challenge (ARC) | Challenge set accuracy | 95.3% | Self-reported | — | — | DeepSeek-V3 Technical Report via Epoch AI |
| BIG-Bench Hard (BBH) | Average accuracy | 82.9% | Self-reported | — | — | DeepSeek-V3 Technical Report via Epoch AI |
| BIG-Bench Hard (BBH) | Average accuracy | 82.9% | Self-reported | — | — | DeepSeek-V3 Technical Report via Epoch AI |
| DTBench (Decision Theory Benchmark) | Accuracy (capability questions) | 61.4% | Third party | — | — | https://conceptualreasoning.ai/dtbench via Epoch AI |
| HellaSwag | Accuracy | 89.2% | Self-reported | — | — | DeepSeek-V3 Technical Report via Epoch AI |
| MMLU | Accuracy | 84.5% | Third party | — | — | Stanford CRFM Leaderboard via Epoch AI |
| MMLU | Accuracy | 84.4% | Self-reported | — | — | DeepSeek-V3 Technical Report via Epoch AI |
| PIQA | Accuracy | 85.9% | Self-reported | — | — | DeepSeek-V3 Technical Report via Epoch AI |
| SimpleBench | Score (AVG@5) | 23.0% | Third party | — | — | SimpleBench Leaderboard via Epoch AI |
| WinoGrande | Accuracy | 89.2% | Self-reported | — | — | DeepSeek-V3 Technical Report via Epoch AI |
| WinoGrande | Accuracy | 82.2% | Self-reported | — | — | The Llama 3 Herd of Models via Epoch AI |
Scientific reasoning
| Benchmark | Metric | Value | Provenance | Setting | Evaluated | Source |
|---|---|---|---|---|---|---|
| GPQA Diamond | Accuracy | 50.9% | Independent | — | 27 Jan 2025 | Epoch AI |
Mathematics
| Benchmark | Metric | Value | Provenance | Setting | Evaluated | Source |
|---|---|---|---|---|---|---|
| MATH Level 5 | Accuracy | 49.8% | Independent | — | 27 Jan 2025 | Epoch AI |
| OTIS Mock AIME 2024-2025 | Accuracy | 9.7% | Independent | — | 25 Feb 2025 | Epoch AI |
Coding & software engineering
| Benchmark | Metric | Value | Provenance | Setting | Evaluated | Source |
|---|---|---|---|---|---|---|
| Cybench | Unguided challenges solved | 7.5% | Third party | — | — | Cybench leaderboard via Epoch AI |
| WeirdML (v2) | Average accuracy | 21.4% | Third party | — | — | https://htihle.github.io/weirdml.html via Epoch AI |
Professional knowledge work
| Benchmark | Metric | Value | Provenance | Setting | Evaluated | Source |
|---|---|---|---|---|---|---|
| TheAgentCompany | Tasks resolved | 7.4% | Third party | — | — | TheAgentCompany experiment results github via Epoch AI |
Factuality & hallucination
| Benchmark | Metric | Value | Provenance | Setting | Evaluated | Source |
|---|---|---|---|---|---|---|
| TriviaQA | Exact match | 82.7% | Self-reported | — | — | DeepSeek-V3 Technical Report via Epoch AI |
Other Llama versions
- Llama 4 Maverick5 Apr 2025
- Llama 4 Scout5 Apr 2025
- Llama 3.3 70B6 Dec 2024
- Llama 3.2 11B Vision25 Sep 2024
- Llama 3.2 1B25 Sep 2024
- Llama 3.2 3B25 Sep 2024
- Llama 3.2 90B Vision25 Sep 2024
- Llama 3.1 70B23 Jul 2024
- Llama 3.1 8B23 Jul 2024
- Llama 3 70B18 Apr 2024
- Llama 3 8B18 Apr 2024
- Llama 2 13B18 Jul 2023
- Llama 2 70B18 Jul 2023
- Llama 2 7B18 Jul 2023
- LLaMA 13B24 Feb 2023
- LLaMA 33B24 Feb 2023
- LLaMA 65B24 Feb 2023
- LLaMA 7B24 Feb 2023
- Llama 3.3 8B—