supermarket

Language

Qwen2.5-72B-Instruct

By Alibaba Group, in the Qwen family.

Released
19 Sep 2024
Status
Legacy
Context
131.1K
Max output
—
Input per 1M tokens
—
Output per 1M tokens
—
Open weights
Yes
License
qwen
Parameters
72B
Knowledge cutoff
—
Reasoning
No
Modalities
text → text

Category scores

Out of 100, from independent and third-party results only, by method category-2026-09-29.1.

Category scores
CategoryScoreRankCoverage
Knowledge & reasoning15.8Not ranked1 of 4
Agentic tool use16.2Not ranked1 of 2
Professional knowledge work5.7111 of 1
Human preference36.4Not ranked1 of 5

Benchmark results

Every result we hold for this model, with who ran it and where it was published.

Knowledge & reasoning

Knowledge & reasoning results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
DTBench (Decision Theory Benchmark)Accuracy (capability questions)62.9%Third party——https://conceptualreasoning.ai/dtbench via Epoch AI
LMCA (Language Model Conceptual Argumentation)Correlation with expert ratings (Pearson r)0.1342Third party——https://conceptualreasoning.ai/lmca via Epoch AI
MMLUAccuracy83.4%Third party——Stanford CRFM Leaderboard via Epoch AI
MMLUAccuracy85.3%Self-reported——Phi-4 Technical Report via Epoch AI

Scientific reasoning

Scientific reasoning results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
GPQA DiamondAccuracy49.1%Independent—27 Jan 2025Epoch AI

Mathematics

Mathematics results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
MATH Level 5Accuracy63.2%Independent—27 Jan 2025Epoch AI
OTIS Mock AIME 2024-2025Accuracy8.1%Independent—25 Feb 2025Epoch AI

Coding & software engineering

Coding & software engineering results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
METR Time HorizonsAverage task success (Epoch's column)35.8%Third party——METR - Measuring AI Ability to Complete Long Tasks via Epoch AI
WeirdML (v2)Average accuracy16.0%Third party——WeirdML Leaderboard via Epoch AI

Agentic tool use

Agentic tool use results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
BALROGAverage progress16.2%Third party——https://balrogai.com/ via Epoch AI

Professional knowledge work

Professional knowledge work results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
TheAgentCompanyTasks resolved5.7%Third party——TheAgentCompany experiment results github via Epoch AI

Human preference

Human preference results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
Text Arena (arena.ai)Arena score (Elo scale)1303Independent—25 Sep 2026Arena
Text Arena (arena.ai)Votes39406Independent—25 Sep 2026Arena

Other Qwen versions

Sources