supermarket

Vision-language

Claude 3.7 Sonnet

By Anthropic, in the Claude Sonnet family.

Released
24 Feb 2025
Status
Retired
Retired
19 Feb 2026
Context
200K
Max output
64K
Input per 1M tokens
$3.00
Output per 1M tokens
$15.00
Open weights
No
License
proprietary
Parameters
—
Knowledge cutoff
Oct 2024
Reasoning
Yes
Modalities
text, image → text

Category scores

Out of 100, from independent and third-party results only, by method category-2026-09-29.1.

Category scores
CategoryScoreRankCoverage
Knowledge & reasoning35.7Not ranked1 of 4
Coding & software engineering34.4Not ranked2 of 5
Agentic tool use43.6Not ranked1 of 2
Professional knowledge work30.941 of 1
Vision & multimodal understanding38.3112 of 3
Human preference33.8Not ranked2 of 5

Benchmark results

Every result we hold for this model, with who ran it and where it was published.

Knowledge & reasoning

Knowledge & reasoning results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
SimpleBenchScore (AVG@5)46.4%Third party12K—SimpleBench Leaderboard via Epoch AI
SimpleBenchScore (AVG@5)46.4%Third partyunknown—SimpleBench Leaderboard via Epoch AI
SimpleBenchScore (AVG@5)44.9%Third party——SimpleBench Leaderboard via Epoch AI

Scientific reasoning

Scientific reasoning results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
GPQA DiamondAccuracy79.7%Independent64K26 May 2025Epoch AI
GPQA DiamondAccuracy76.8%Independent16K26 Feb 2025Epoch AI
GPQA DiamondAccuracy76.8%Independent32K10 Mar 2025Epoch AI
GPQA DiamondAccuracy66.0%Independent—24 Feb 2025Epoch AI

Mathematics

Mathematics results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
FrontierMath Tiers 1-3 (v1)Accuracy4.1%Independent16K6 Mar 2025Epoch AI
FrontierMath Tiers 1-3 (v1)Accuracy3.4%Independent32K13 Mar 2025Epoch AI
FrontierMath Tiers 1-3 (v1)Accuracy3.1%Independent—6 Mar 2025Epoch AI
FrontierMath Tiers 1-3 (v1)Accuracy3.1%Independent64K13 Mar 2025Epoch AI
MATH Level 5Accuracy91.2%Independent64K13 Mar 2025Epoch AI
MATH Level 5Accuracy90.0%Independent32K12 Mar 2025Epoch AI
MATH Level 5Accuracy86.3%Independent16K26 Feb 2025Epoch AI
MATH Level 5Accuracy68.2%Independent—24 Feb 2025Epoch AI
OTIS Mock AIME 2024-2025Accuracy57.8%Independent64K13 Mar 2025Epoch AI
OTIS Mock AIME 2024-2025Accuracy53.3%Independent32K12 Mar 2025Epoch AI
OTIS Mock AIME 2024-2025Accuracy46.7%Independent16K26 Feb 2025Epoch AI
OTIS Mock AIME 2024-2025Accuracy21.9%Independent—25 Feb 2025Epoch AI

Coding & software engineering

Coding & software engineering results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
Aider PolyglotPercent correct64.9%Third party32K—https://aider.chat/docs/leaderboards/ via Epoch AI
Aider PolyglotPercent correct60.4%Third party——https://aider.chat/docs/leaderboards/ via Epoch AI
CadEvalOverall pass rate54.0%Third party——CadEval Dashboard via Epoch AI
CybenchUnguided challenges solved20.0%Third party——Cybench leaderboard via Epoch AI
GSOOPT@13.8%Third partyunknown—GSO Leaderboard via Epoch AI
GSOOPT@13.8%Third party——GSO Leaderboard via Epoch AI
METR Time HorizonsAverage task success (Epoch's column)60.0%Third party16K—METR - Measuring AI Ability to Complete Long Tasks via Epoch AI
METR Time HorizonsAverage task success (Epoch's column)55.8%Source not stated——Epoch AI via Epoch AI
SWE-bench Verified% resolved61.0%Independent—4 Feb 2026Epoch AI

Agentic tool use

Agentic tool use results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
Deep Research Bench (FutureSearch)Average score43.6%Third party2K—DeepResearchBench Leaderboard via Epoch AI

Computer & browser use

Computer & browser use results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
OSWorld-Verified% tasks completed35.8%Third party——OS World Website via Epoch AI
OSWorld-Verified% tasks completed35.6%Third party——OS World Website via Epoch AI
OSWorld-Verified% tasks completed27.1%Third party——OS World Website via Epoch AI

Professional knowledge work

Professional knowledge work results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
TheAgentCompanyTasks resolved30.9%Third party——TheAgentCompany leaderboard via Epoch AI
TheAgentCompanyTasks resolved26.3%Third party——TheAgentCompany leaderboard via Epoch AI

Long context

Long context results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
Fiction.LiveBench% correct at 16K tokens (Epoch's column)83.3%Third party8K—Fiction.live leaderboard via Epoch AI
Fiction.LiveBench% correct at 16K tokens (Epoch's column)50.0%Third party——Fiction.live leaderboard via Epoch AI

Vision & multimodal understanding

Vision & multimodal understanding results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
GeoBenchCountry accuracy68.0%Third party——GeoBench leaderboard via Epoch AI
GeoBenchCountry accuracy65.0%Third party15K—GeoBench leaderboard via Epoch AI
Visual Physics Comprehension Test (VPCT)Correct39.0%Third party——VPCT leaderboard via Epoch AI
Visual Physics Comprehension Test (VPCT)Correct35.0%Third party64K—VPCT leaderboard via Epoch AI

Writing

Writing results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
LLM Creative Story-Writing Benchmark V1 (Lech Mazur)Mean rubric score (percent of the 10-point scale)81.1%Third party16K—lechmazur/writing Github repository via Epoch AI
LLM Creative Story-Writing Benchmark V1 (Lech Mazur)Mean rubric score (percent of the 10-point scale)79.4%Third party——lechmazur/writing Github repository via Epoch AI

Human preference

Human preference results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
Text Arena (arena.ai)Arena score (Elo scale)1373Independent—25 Sep 2026Arena
Text Arena (arena.ai)Votes42408Independent—25 Sep 2026Arena
Vision Arena (arena.ai)Arena score (Elo scale)1175Independent—28 Sep 2026Arena
Vision Arena (arena.ai)Votes4519Independent—28 Sep 2026Arena

Other Claude Sonnet versions

Sources