Released
25 Mar 2025
Status
Legacy
Context
1M
Max output
65.5K
Input per 1M tokens
$1.25
Output per 1M tokens
$10.00
Open weights
No
License
proprietary
Parameters
—
Knowledge cutoff
Jan 2025
Reasoning
Yes
Modalities
text, image, video, audio, pdf → text

Category scores

Out of 100, from independent and third-party results only, by method category-2026-09-29.1.

Category scores
CategoryScoreRankCoverage
Knowledge & reasoning37.2403 of 4
Coding & software engineering43.5Not ranked2 of 5
Agentic tool use43.162 of 2
Professional knowledge work30.351 of 1
Vision & multimodal understanding54.042 of 3
Human preference41.8424 of 5

Benchmark results

Every result we hold for this model, with who ran it and where it was published.

Knowledge & reasoning

Knowledge & reasoning results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
Chess PuzzlesPuzzles solved20.0%Independent—8 Dec 2025Epoch AI
DTBench (Decision Theory Benchmark)Accuracy (capability questions)82.4%Third party——https://conceptualreasoning.ai/dtbench via Epoch AI
LMCA (Language Model Conceptual Argumentation)Correlation with expert ratings (Pearson r)0.3477Third party——https://conceptualreasoning.ai/lmca via Epoch AI
SimpleBenchScore (AVG@5)62.4%Third party——https://lmcouncil.ai/benchmarks via Epoch AI
SimpleBenchScore (AVG@5)51.6%Third party——SimpleBench Leaderboard via Epoch AI
SimpleBenchScore (AVG@5)51.6%Third party——SimpleBench Leaderboard via Epoch AI

Scientific reasoning

Scientific reasoning results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
GPQA DiamondAccuracy85.3%Independent—16 Nov 2025Epoch AI
GPQA DiamondAccuracy84.8%Independent—5 Jun 2025Epoch AI
GPQA DiamondAccuracy83.8%Independent—31 Mar 2025Epoch AI
GPQA DiamondAccuracy66.7%Independent—3 Jun 2025Epoch AI

Mathematics

Mathematics results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
FrontierMath Tier 4Accuracy0.0%Independent—11 Jun 2026Epoch AI
FrontierMath Tier 4 (v1)Accuracy4.2%Independent—3 Jul 2025Epoch AI
FrontierMath Tier 4 (v1)Accuracy2.1%Independent—3 Jul 2025Epoch AI
FrontierMath Tiers 1-3Accuracy24.6%Independent—11 Jun 2026Epoch AI
FrontierMath Tiers 1-3 (v1)Accuracy14.1%Independent—24 Nov 2025Epoch AI
FrontierMath Tiers 1-3 (v1)Accuracy10.3%Independent—3 Jul 2025Epoch AI
MATH Level 5Accuracy95.9%Independent—8 May 2025Epoch AI
MATH Level 5Accuracy95.6%Independent—7 May 2025Epoch AI
OTIS Mock AIME 2024-2025Accuracy84.7%Independent—16 Nov 2025Epoch AI

Coding & software engineering

Coding & software engineering results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
Aider PolyglotPercent correct83.1%Third party32K—https://aider.chat/docs/leaderboards/ via Epoch AI
Aider PolyglotPercent correct79.1%Third party——https://aider.chat/docs/leaderboards/ via Epoch AI
Aider PolyglotPercent correct76.9%Third party——https://aider.chat/docs/leaderboards/ via Epoch AI
Aider PolyglotPercent correct72.9%Third party——https://aider.chat/docs/leaderboards/ via Epoch AI
Aider PolyglotPercent correct72.9%Third party——Aider LLM Leaderboards via Epoch AI
CadEvalOverall pass rate64.0%Third party——CadEval Dashboard via Epoch AI
GSOOPT@13.9%Third party——GSO Leaderboard via Epoch AI
GSOOPT@13.9%Third party——https://gso-bench.github.io/index.html via Epoch AI
METR Time HorizonsAverage task success (Epoch's column)55.4%Third party——METR - Measuring AI Ability to Complete Long Tasks via Epoch AI
SWE-bench Verified% resolved57.6%Independent—13 Feb 2026Epoch AI
Terminal-Bench 2.0Accuracy (mean over trials)32.6%Third party——https://www.tbench.ai/leaderboard/terminal-bench/2.0 via Epoch AI
Terminal-Bench 2.0Accuracy (mean over trials)32.6%Third party——Terminal-Bench v2 Leaderboard via Epoch AI
Terminal-Bench 2.0Accuracy (mean over trials)26.1%Third party——Terminal-Bench v2 Leaderboard via Epoch AI
Terminal-Bench 2.0Accuracy (mean over trials)26.1%Third party——https://www.tbench.ai/leaderboard/terminal-bench/2.0 via Epoch AI
Terminal-Bench 2.0Accuracy (mean over trials)19.6%Third party——Terminal-Bench v2 Leaderboard via Epoch AI
Terminal-Bench 2.0Accuracy (mean over trials)19.6%Third party——https://www.tbench.ai/leaderboard/terminal-bench/2.0 via Epoch AI
Terminal-Bench 2.0Accuracy (mean over trials)16.4%Third party——https://www.tbench.ai/leaderboard/terminal-bench/2.0 via Epoch AI
Terminal-Bench 2.0Accuracy (mean over trials)16.4%Third party——Terminal-Bench v2 Leaderboard via Epoch AI
WeirdML (v2)Average accuracy54.0%Third party16K—WeirdML Leaderboard via Epoch AI

Agentic tool use

Agentic tool use results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
BALROGAverage progress43.3%Third party——https://balrogai.com/ via Epoch AI
Deep Research Bench (FutureSearch)Average score42.8%Third party——DeepResearchBench Leaderboard via Epoch AI
Deep Research Bench (FutureSearch)Average score41.5%Third party——https://evals.futuresearch.ai/#drb via Epoch AI

Professional knowledge work

Professional knowledge work results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
GDPvalWins vs experts23.3%Source not stated——Epoch AI via Epoch AI
TheAgentCompanyTasks resolved30.3%Third party——TheAgentCompany leaderboard via Epoch AI

Long context

Long context results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
Fiction.LiveBench% correct at 16K tokens (Epoch's column)91.7%Third party——Fiction.live leaderboard via Epoch AI
Fiction.LiveBench% correct at 16K tokens (Epoch's column)66.7%Third party——Fiction.live leaderboard via Epoch AI
Fiction.LiveBench% correct at 16K tokens (Epoch's column)66.7%Third party——Fiction.live leaderboard via Epoch AI
Fiction.LiveBench% correct at 16K tokens (Epoch's column)66.7%Third party——Fiction.live leaderboard via Epoch AI

Vision & multimodal understanding

Vision & multimodal understanding results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
GeoBenchCountry accuracy86.0%Third party——GeoBench leaderboard via Epoch AI
GeoBenchCountry accuracy81.0%Third party——GeoBench leaderboard via Epoch AI
GeoBenchCountry accuracy81.0%Third party——GeoBench leaderboard via Epoch AI
Visual Physics Comprehension Test (VPCT)Correct48.0%Third party——VPCT leaderboard via Epoch AI
Visual Physics Comprehension Test (VPCT)Correct46.4%Third party——VPCT leaderboard via Epoch AI
Visual Physics Comprehension Test (VPCT)Correct40.5%Third party——VPCT leaderboard via Epoch AI

Writing

Writing results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
LLM Creative Story-Writing Benchmark V1 (Lech Mazur)Mean rubric score (percent of the 10-point scale)83.8%Third party——lechmazur/writing Github repository via Epoch AI
LLM Creative Story-Writing Benchmark V1 (Lech Mazur)Mean rubric score (percent of the 10-point scale)80.9%Third party——lechmazur/writing Github repository via Epoch AI
LLM Creative Story-Writing Benchmark V1 (Lech Mazur)Mean rubric score (percent of the 10-point scale)80.5%Third party——lechmazur/writing Github repository via Epoch AI

Human preference

Human preference results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
Code Arena: WebDev (arena.ai)Arena score (Elo scale)1227Independent—25 Sep 2026Arena
Code Arena: WebDev (arena.ai)Votes2731Independent—25 Sep 2026Arena
Document Arena (arena.ai)Arena score (Elo scale)1431Independent—13 Sep 2026Arena
Document Arena (arena.ai)Votes25110Independent—13 Sep 2026Arena
Text Arena (arena.ai)Arena score (Elo scale)1446Independent—25 Sep 2026Arena
Text Arena (arena.ai)Votes124887Independent—25 Sep 2026Arena
Vision Arena (arena.ai)Arena score (Elo scale)1247Independent—28 Sep 2026Arena
Vision Arena (arena.ai)Votes88831Independent—28 Sep 2026Arena

Other Gemini Pro versions

Sources