supermarket

Vision-language

GPT-4o

By OpenAI, in the GPT-4o family.

Released
13 May 2024
Status
Legacy
Context
128K
Max output
16.4K
Input per 1M tokens
$2.50
Output per 1M tokens
$10.00
Open weights
No
License
proprietary
Parameters
—
Knowledge cutoff
Oct 2023
Reasoning
No
Modalities
text, image → text

Category scores

Out of 100, from independent and third-party results only, by method category-2026-09-29.1.

Category scores
CategoryScoreRankCoverage
Knowledge & reasoning9.8963 of 4
Coding & software engineering11.6Not ranked2 of 5
Agentic tool use32.3Not ranked1 of 2
Professional knowledge work8.681 of 1
Vision & multimodal understanding40.5102 of 3
Factuality & hallucination26.0551 of 1
Human preference31.3Not ranked2 of 5

Benchmark results

Every result we hold for this model, with who ran it and where it was published.

Knowledge & reasoning

Knowledge & reasoning results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
Chess PuzzlesPuzzles solved13.0%Independent—15 Jul 2026Epoch AI
DTBench (Decision Theory Benchmark)Accuracy (capability questions)64.5%Third party——https://conceptualreasoning.ai/dtbench via Epoch AI
LMCA (Language Model Conceptual Argumentation)Correlation with expert ratings (Pearson r)0.1659Third party——https://conceptualreasoning.ai/lmca via Epoch AI
MMLUAccuracy84.3%Third party——Stanford CRFM Leaderboard via Epoch AI
MMLUAccuracy84.2%Third party——Stanford CRFM Leaderboard via Epoch AI
MMLUAccuracy88.1%Self-reported——Phi-4 Technical Report via Epoch AI
ScienceQAAccuracy88.5%Self-reported——Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone via Epoch AI
SimpleBenchScore (AVG@5)17.8%Third party——SimpleBench Leaderboard via Epoch AI

Scientific reasoning

Scientific reasoning results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
GPQA DiamondAccuracy49.2%Independent—27 Jan 2025Epoch AI
GPQA DiamondAccuracy48.9%Independent—27 Jan 2025Epoch AI
GPQA DiamondAccuracy47.9%Independent—5 Feb 2025Epoch AI

Mathematics

Mathematics results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
FrontierMath Tiers 1-3Accuracy0.4%Independent—27 Aug 2026Epoch AI
FrontierMath Tiers 1-3 (v1)Accuracy0.3%Independent—7 Mar 2025Epoch AI
FrontierMath Tiers 1-3 (v1)Accuracy0.3%Independent—6 Mar 2025Epoch AI
MATH Level 5Accuracy53.3%Independent—27 Jan 2025Epoch AI
MATH Level 5Accuracy51.0%Independent—27 Jan 2025Epoch AI
MATH Level 5Accuracy49.8%Independent—5 Feb 2025Epoch AI
OTIS Mock AIME 2024-2025Accuracy6.4%Independent—25 Feb 2025Epoch AI
OTIS Mock AIME 2024-2025Accuracy6.3%Independent—25 Feb 2025Epoch AI
OTIS Mock AIME 2024-2025Accuracy6.3%Independent—25 Feb 2025Epoch AI

Coding & software engineering

Coding & software engineering results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
Aider PolyglotPercent correct23.1%Third party——https://aider.chat/docs/leaderboards/ via Epoch AI
Aider PolyglotPercent correct18.2%Third party——https://aider.chat/docs/leaderboards/ via Epoch AI
CadEvalOverall pass rate26.0%Third party——CadEval Dashboard via Epoch AI
CybenchUnguided challenges solved12.5%Third party——Cybench leaderboard via Epoch AI
GSOOPT@10.0%Third party——GSO Leaderboard via Epoch AI
METR Time HorizonsAverage task success (Epoch's column)40.8%Third party——METR - Measuring AI Ability to Complete Long Tasks via Epoch AI
METR Time HorizonsAverage task success (Epoch's column)33.8%Source not stated——Epoch AI via Epoch AI
SWE-bench Verified% resolved31.0%Independent—11 Feb 2026Epoch AI
WeirdML (v2)Average accuracy25.1%Third party——https://htihle.github.io/weirdml.html via Epoch AI

Agentic tool use

Agentic tool use results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
BALROGAverage progress32.3%Third party——https://balrogai.com/ via Epoch AI

Professional knowledge work

Professional knowledge work results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
GDPvalWins vs experts9.9%Source not stated——Epoch AI via Epoch AI
TheAgentCompanyTasks resolved8.6%Third party——TheAgentCompany experiment results github via Epoch AI

Vision & multimodal understanding

Vision & multimodal understanding results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
GeoBenchCountry accuracy71.0%Third party——GeoBench leaderboard via Epoch AI
Visual Physics Comprehension Test (VPCT)Correct40.0%Third party——VPCT leaderboard via Epoch AI

Factuality & hallucination

Factuality & hallucination results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
SimpleQA VerifiedAccuracy (% correct)26.0%Independent—31 Aug 2026Epoch AI

Writing

Writing results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
LLM Creative Story-Writing Benchmark V1 (Lech Mazur)Mean rubric score (percent of the 10-point scale)81.8%Third party——lechmazur/writing Github repository via Epoch AI

Human preference

Human preference results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
Text Arena (arena.ai)Arena score (Elo scale)1346Independent—25 Sep 2026Arena
Text Arena (arena.ai)Arena score (Elo scale)1336Independent—25 Sep 2026Arena
Text Arena (arena.ai)Votes112881Independent—25 Sep 2026Arena
Text Arena (arena.ai)Votes45499Independent—25 Sep 2026Arena
Vision Arena (arena.ai)Arena score (Elo scale)1162Independent—28 Sep 2026Arena
Vision Arena (arena.ai)Arena score (Elo scale)1118Independent—28 Sep 2026Arena
Vision Arena (arena.ai)Votes23273Independent—28 Sep 2026Arena
Vision Arena (arena.ai)Votes3376Independent—28 Sep 2026Arena

Other GPT-4o versions

Sources