supermarket

Language

Llama 3 8B

By Meta, in the Llama family.

Released
18 Apr 2024
Status
Legacy
Context
8.2K
Max output
—
Input per 1M tokens
—
Output per 1M tokens
—
Open weights
Yes
License
llama-3
Parameters
8.03B
Knowledge cutoff
Mar 2023
Reasoning
No
Modalities
text → text

Category scores

Out of 100, from independent and third-party results only, by method category-2026-09-29.1.

Category scores
CategoryScoreRankCoverage
Knowledge & reasoning0.0Not ranked1 of 4

Benchmark results

Every result we hold for this model, with who ran it and where it was published.

Knowledge & reasoning

Knowledge & reasoning results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
Adversarial NLI (ANLI)Accuracy57.3%Self-reported——Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone via Epoch AI
AI2 Reasoning Challenge (ARC)Challenge set accuracy82.8%Self-reported——Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone via Epoch AI
Chess PuzzlesPuzzles solved0.0%Independent—28 Aug 2026Epoch AI
DTBench (Decision Theory Benchmark)Accuracy (capability questions)43.9%Third party——https://conceptualreasoning.ai/dtbench via Epoch AI
MMLUAccuracy68.8%Third party——Stanford CRFM Leaderboard via Epoch AI
MMLUAccuracy66.5%Self-reported——Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone via Epoch AI
MMLU-ProAccuracy41.0%Self-reported——EvalEval via Hugging Face Community Evals
OpenBookQAAccuracy82.6%Self-reported——Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone via Epoch AI
OpenBookQAAccuracy45.0%Self-reported——The Llama 3 Herd of Models via Epoch AI
WinoGrandeAccuracy65.0%Self-reported——Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone via Epoch AI

Scientific reasoning

Scientific reasoning results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
GPQA DiamondAccuracy26.1%Independent—27 Jan 2025Epoch AI

Mathematics

Mathematics results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
MATH Level 5Accuracy6.1%Independent—27 Jan 2025Epoch AI
OTIS Mock AIME 2024-2025Accuracy1.9%Independent—30 Aug 2026Epoch AI

Factuality & hallucination

Factuality & hallucination results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
TriviaQAExact match67.7%Self-reported——Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone via Epoch AI

Other Llama versions

Sources