supermarket

Language

Llama 2 7B

By Meta, in the Llama family.

Released
18 Jul 2023
Status
Legacy
Context
4.1K
Max output
—
Input per 1M tokens
—
Output per 1M tokens
—
Open weights
Yes
License
llama-2
Parameters
6.74B
Knowledge cutoff
Sep 2022
Reasoning
No
Modalities
text → text

Category scores

Out of 100, from independent and third-party results only, by method category-2026-09-29.1.

Not enough independent results yet

Only independent and third-party results rank a model; self-reported and unattributed numbers never do.

Benchmark results

Every result we hold for this model, with who ran it and where it was published.

Knowledge & reasoning

Knowledge & reasoning results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
AI2 Reasoning Challenge (ARC)Challenge set accuracy45.9%Third party——The Falcon Series of Open Language Models via Epoch AI
AI2 Reasoning Challenge (ARC)Challenge set accuracy45.9%Third party——Gemma: Open Models Based on Gemini Research and Technology via Epoch AI
AI2 Reasoning Challenge (ARC)Challenge set accuracy43.2%Third party——Mixtral of Experts via Epoch AI
AI2 Reasoning Challenge (ARC)Challenge set accuracy45.9%Self-reported——Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI
AI2 Reasoning Challenge (ARC)Challenge set accuracy45.9%Self-reported——Qwen Technical Report via Epoch AI
AI2 Reasoning Challenge (ARC)Challenge set accuracy43.4%Self-reported——Textbooks Are All You Need II: phi-1.5 technical report via Epoch AI
BIG-Bench Hard (BBH)Average accuracy39.2%Third party——Baichuan 2: Open Large-scale Language Models via Epoch AI
BIG-Bench Hard (BBH)Average accuracy32.6%Third party——Gemma: Open Models Based on Gemini Research and Technology via Epoch AI
BIG-Bench Hard (BBH)Average accuracy32.6%Self-reported——Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI
HellaSwagAccuracy77.2%Third party——Gemma: Open Models Based on Gemini Research and Technology via Epoch AI
HellaSwagAccuracy57.1%Self-reported——Textbooks Are All You Need II: phi-1.5 technical report via Epoch AI
LAMBADAAccuracy73.3%Self-reported——Qwen Technical Report via Epoch AI
LAMBADAAccuracy73.3%Self-reported——Qwen Technical Report via Epoch AI
MMLUAccuracy45.8%Third party——Stanford CRFM Leaderboard via Epoch AI
MMLUAccuracy45.7%Third party——Baichuan 2: Open Large-scale Language Models via Epoch AI
MMLUAccuracy45.3%Third party——Gemma: Open Models Based on Gemini Research and Technology via Epoch AI
MMLUAccuracy44.4%Third party——Mixtral of Experts via Epoch AI
MMLUAccuracy45.3%Self-reported——Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI
MMLUAccuracy45.3%Self-reported——Textbooks Are All You Need II: phi-1.5 technical report via Epoch AI
OpenBookQAAccuracy58.6%Self-reported——Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI
OpenBookQAAccuracy31.4%Self-reported——Textbooks Are All You Need II: phi-1.5 technical report via Epoch AI
PIQAAccuracy78.8%Third party——Gemma: Open Models Based on Gemini Research and Technology via Epoch AI
PIQAAccuracy78.8%Third party——The Falcon Series of Open Language Models via Epoch AI
PIQAAccuracy77.9%Third party——Mixtral of Experts via Epoch AI
PIQAAccuracy78.8%Self-reported——Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI
PIQAAccuracy78.8%Self-reported——Qwen Technical Report via Epoch AI
PIQAAccuracy78.1%Self-reported——Textbooks Are All You Need II: phi-1.5 technical report via Epoch AI
ScienceQAAccuracy43.1%Third party——ScienceQA leaderboard via Epoch AI
WinoGrandeAccuracy69.2%Third party——Gemma: Open Models Based on Gemini Research and Technology via Epoch AI
WinoGrandeAccuracy69.2%Self-reported——Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI
WinoGrandeAccuracy69.1%Self-reported——Textbooks Are All You Need II: phi-1.5 technical report via Epoch AI

Mathematics

Mathematics results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
GSM8KExact match16.7%Third party——Yi: Open Foundation Models by 01.AI via Epoch AI
GSM8KExact match16.2%Third party——Baichuan 2: Open Large-scale Language Models via Epoch AI
GSM8KExact match16.0%Third party——Mixtral of Experts via Epoch AI
GSM8KExact match14.6%Third party——Gemma: Open Models Based on Gemini Research and Technology via Epoch AI
GSM8KExact match16.7%Self-reported——Qwen Technical Report via Epoch AI
GSM8KExact match14.6%Self-reported——Textbooks Are All You Need II: phi-1.5 technical report via Epoch AI
GSM8KExact match14.6%Self-reported——Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI

Factuality & hallucination

Factuality & hallucination results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
TriviaQAExact match72.1%Third party——Gemma: Open Models Based on Gemini Research and Technology via Epoch AI
TriviaQAExact match73.7%Self-reported——Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI
TriviaQAExact match72.1%Self-reported——Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI
TriviaQAExact match68.9%Self-reported——Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI
TriviaQAExact match65.8%Self-reported——Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI

Other Llama versions

Sources