Updated daily at 05:00 UTC

Vision-language

Claude Opus 4

By Anthropic, in the Claude Opus family.

Released: 22 May 2025
Status: Retired
Retired: 15 Jun 2026
Context: 200K
Max output: 32K
Input per 1M tokens: $15.00
Output per 1M tokens: $75.00
Open weights: No
License: proprietary
Parameters: —
Knowledge cutoff: Jan 2025
Reasoning: Yes
Modalities: text, image → text

Category scores

Out of 100, from independent and third-party results only, by method category-2026-09-29.1.

Category scores
Category	Score	Rank	Coverage
Knowledge & reasoning	47.3	22	2 of 4
Coding & software engineering	39.4	Not ranked	2 of 5
Agentic tool use	46.8	Not ranked	1 of 2
Vision & multimodal understanding	28.0	15	2 of 3
Human preference	37.4	Not ranked	2 of 5

Benchmark results

Every result we hold for this model, with who ran it and where it was published.

Knowledge & reasoning

Knowledge & reasoning results
Benchmark	Metric	Value	Provenance	Setting	Evaluated	Source
DTBench (Decision Theory Benchmark)	Accuracy (capability questions)	81.6%	Third party	unknown	—	https://conceptualreasoning.ai/dtbench via Epoch AI
LMCA (Language Model Conceptual Argumentation)	Correlation with expert ratings (Pearson r)	0.3743	Third party	unknown	—	https://conceptualreasoning.ai/lmca via Epoch AI
SimpleBench	Score (AVG@5)	58.8%	Third party	unknown	—	SimpleBench Leaderboard via Epoch AI
SimpleBench	Score (AVG@5)	58.8%	Third party	12K	—	SimpleBench Leaderboard via Epoch AI

Scientific reasoning

Scientific reasoning results
Benchmark	Metric	Value	Provenance	Setting	Evaluated	Source
GPQA Diamond	Accuracy	76.3%	Independent	16K	22 May 2025	Epoch AI
GPQA Diamond	Accuracy	69.2%	Independent	—	22 May 2025	Epoch AI

Mathematics

Mathematics results
Benchmark	Metric	Value	Provenance	Setting	Evaluated	Source
FrontierMath Tier 4 (v1)	Accuracy	4.2%	Independent	27K	1 Jul 2025	Epoch AI
FrontierMath Tier 4 (v1)	Accuracy	0.0%	Independent	—	1 Jul 2025	Epoch AI
FrontierMath Tiers 1-3 (v1)	Accuracy	4.5%	Independent	—	4 Jul 2025	Epoch AI
FrontierMath Tiers 1-3 (v1)	Accuracy	4.1%	Independent	27K	5 Aug 2025	Epoch AI
MATH Level 5	Accuracy	85.0%	Independent	—	22 May 2025	Epoch AI
OTIS Mock AIME 2024-2025	Accuracy	64.4%	Independent	27K	28 May 2025	Epoch AI
OTIS Mock AIME 2024-2025	Accuracy	60.0%	Independent	16K	22 May 2025	Epoch AI
OTIS Mock AIME 2024-2025	Accuracy	42.2%	Independent	—	22 May 2025	Epoch AI

Coding & software engineering

Coding & software engineering results
Benchmark	Metric	Value	Provenance	Setting	Evaluated	Source
Aider Polyglot	Percent correct	72.0%	Third party	32K	—	https://aider.chat/docs/leaderboards/ via Epoch AI
Aider Polyglot	Percent correct	70.7%	Third party	—	—	https://aider.chat/docs/leaderboards/ via Epoch AI
Cybench	Unguided challenges solved	38.0%	Third party	—	—	Cybench leaderboard via Epoch AI
GSO	OPT@1	6.9%	Third party	—	—	GSO Leaderboard via Epoch AI
GSO	OPT@1	6.9%	Third party	unknown	—	GSO Leaderboard via Epoch AI
METR Time Horizons	Average task success (Epoch's column)	63.9%	Third party	16K	—	METR - Measuring AI Ability to Complete Long Tasks via Epoch AI
METR Time Horizons	Average task success (Epoch's column)	61.5%	Source not stated	unknown	—	Epoch AI via Epoch AI
SWE-bench Verified	% resolved	70.7%	Independent	—	6 Feb 2026	Epoch AI
WeirdML (v2)	Average accuracy	43.7%	Third party	16K	—	https://htihle.github.io/weirdml.html via Epoch AI

Agentic tool use

Agentic tool use results
Benchmark	Metric	Value	Provenance	Setting	Evaluated	Source
Deep Research Bench (FutureSearch)	Average score	46.8%	Third party	2K	—	https://evals.futuresearch.ai/#drb via Epoch AI

Long context

Long context results
Benchmark	Metric	Value	Provenance	Setting	Evaluated	Source
Fiction.LiveBench	% correct at 16K tokens (Epoch's column)	61.1%	Third party	—	—	Fiction.live leaderboard via Epoch AI

Vision & multimodal understanding

Vision & multimodal understanding results
Benchmark	Metric	Value	Provenance	Setting	Evaluated	Source
GeoBench	Country accuracy	49.0%	Third party	32K	—	GeoBench leaderboard via Epoch AI
Visual Physics Comprehension Test (VPCT)	Correct	38.0%	Third party	16K	—	VPCT leaderboard via Epoch AI
Visual Physics Comprehension Test (VPCT)	Correct	33.0%	Third party	—	—	VPCT leaderboard via Epoch AI

Writing

Writing results
Benchmark	Metric	Value	Provenance	Setting	Evaluated	Source
LLM Creative Story-Writing Benchmark V1 (Lech Mazur)	Mean rubric score (percent of the 10-point scale)	83.6%	Third party	16K	—	lechmazur/writing Github repository via Epoch AI
LLM Creative Story-Writing Benchmark V1 (Lech Mazur)	Mean rubric score (percent of the 10-point scale)	83.1%	Third party	—	—	lechmazur/writing Github repository via Epoch AI

Human preference

Human preference results
Benchmark	Metric	Value	Provenance	Setting	Evaluated	Source
Text Arena (arena.ai)	Arena score (Elo scale)	1414	Independent	—	25 Sep 2026	Arena
Text Arena (arena.ai)	Votes	42789	Independent	—	25 Sep 2026	Arena
Vision Arena (arena.ai)	Arena score (Elo scale)	1188	Independent	—	28 Sep 2026	Arena
Vision Arena (arena.ai)	Votes	2177	Independent	—	28 Sep 2026	Arena

Other Claude Opus versions

Sources

Claude Opus 4: benchmarks, price and specs · supermarket.dev