Language

MPT-7B (Instruct, Chat, StoryWriter)

Released
—
Status
Legacy
Context
—
Max output
—
Input per 1M tokens
—
Output per 1M tokens
—
Open weights
Yes
License
—
Parameters
6.7B
Knowledge cutoff
—
Reasoning
No
Modalities
text → text

Category scores

Out of 100, from independent and third-party results only, by method category-2026-09-29.1.

Not enough independent results yet

Only independent and third-party results rank a model; self-reported and unattributed numbers never do.

Benchmark results

Every result we hold for this model, with who ran it and where it was published.

Knowledge & reasoning

Knowledge & reasoning results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
AI2 Reasoning Challenge (ARC)Challenge set accuracy42.6%Third party——Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI
AI2 Reasoning Challenge (ARC)Challenge set accuracy42.6%Self-reported——Qwen Technical Report via Epoch AI
AI2 Reasoning Challenge (ARC)Challenge set accuracy41.7%Self-reported——XGen-7B Technical Report via Epoch AI
AI2 Reasoning Challenge (ARC)Challenge set accuracy40.5%Self-reported——Textbooks Are All You Need II: phi-1.5 technical report via Epoch AI
BIG-Bench Hard (BBH)Average accuracy35.2%Third party——Baichuan 2: Open Large-scale Language Models via Epoch AI
BIG-Bench Hard (BBH)Average accuracy31.0%Third party——Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI
BIG-Bench Hard (BBH)Average accuracy35.6%Self-reported——Qwen Technical Report via Epoch AI
HellaSwagAccuracy76.4%Self-reported——Qwen Technical Report via Epoch AI
HellaSwagAccuracy76.1%Self-reported——XGen-7B Technical Report via Epoch AI
HellaSwagAccuracy57.1%Self-reported——Textbooks Are All You Need II: phi-1.5 technical report via Epoch AI
LAMBADAAccuracy70.0%Self-reported——Qwen Technical Report via Epoch AI
LAMBADAAccuracy70.0%Self-reported——Qwen Technical Report via Epoch AI
MMLUAccuracy27.9%Third party——Baichuan 2: Open Large-scale Language Models via Epoch AI
MMLUAccuracy26.8%Third party——Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI
MMLUAccuracy30.8%Self-reported——Qwen Technical Report via Epoch AI
MMLUAccuracy27.4%Self-reported——XGen-7B Technical Report via Epoch AI
MMLUAccuracy26.8%Self-reported——Textbooks Are All You Need II: phi-1.5 technical report via Epoch AI
MMLUAccuracy26.7%Self-reported——XGen-7B Technical Report via Epoch AI
OpenBookQAAccuracy51.4%Third party——Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI
OpenBookQAAccuracy41.8%Self-reported——XGen-7B Technical Report via Epoch AI
OpenBookQAAccuracy31.4%Self-reported——Textbooks Are All You Need II: phi-1.5 technical report via Epoch AI
PIQAAccuracy80.6%Third party——Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI
PIQAAccuracy80.6%Self-reported——Qwen Technical Report via Epoch AI
PIQAAccuracy79.1%Self-reported——XGen-7B Technical Report via Epoch AI
PIQAAccuracy78.9%Self-reported——Textbooks Are All You Need II: phi-1.5 technical report via Epoch AI
WinoGrandeAccuracy68.3%Third party——Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI
WinoGrandeAccuracy68.6%Self-reported——XGen-7B Technical Report via Epoch AI
WinoGrandeAccuracy68.0%Self-reported——Textbooks Are All You Need II: phi-1.5 technical report via Epoch AI

Mathematics

Mathematics results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
GSM8KExact match8.6%Third party——Baichuan 2: Open Large-scale Language Models via Epoch AI
GSM8KExact match6.8%Third party——Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI
GSM8KExact match9.1%Self-reported——Qwen Technical Report via Epoch AI
GSM8KExact match6.8%Self-reported——Textbooks Are All You Need II: phi-1.5 technical report via Epoch AI

Factuality & hallucination

Factuality & hallucination results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
TriviaQAExact match61.6%Third party——Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI
TriviaQAExact match61.2%Third party——Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI
TriviaQAExact match59.6%Third party——Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI
TriviaQAExact match55.7%Third party——Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI

Other MPT (MosaicML Pretrained Transformer) versions

Sources