Released
17 Apr 2025
Status
Legacy
Context
1M
Max output
65.5K
Input per 1M tokens
$0.30
Output per 1M tokens
$2.50
Open weights
No
License
proprietary
Parameters
—
Knowledge cutoff
Jan 2025
Reasoning
Yes
Modalities
text, image, audio, video → text

Category scores

Out of 100, from independent and third-party results only, by method category-2026-09-29.1.

Category scores
CategoryScoreRankCoverage
Knowledge & reasoning30.9572 of 4
Coding & software engineering55.1Not ranked1 of 5
Agentic tool use33.5Not ranked1 of 2
Professional knowledge work41.121 of 1
Vision & multimodal understanding47.772 of 3
Human preference39.4Not ranked2 of 5

Benchmark results

Every result we hold for this model, with who ran it and where it was published.

Knowledge & reasoning

Knowledge & reasoning results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
DTBench (Decision Theory Benchmark)Accuracy (capability questions)76.5%Third party——https://conceptualreasoning.ai/dtbench via Epoch AI
LMCA (Language Model Conceptual Argumentation)Correlation with expert ratings (Pearson r)0.2749Third party——https://conceptualreasoning.ai/lmca via Epoch AI
SimpleBenchScore (AVG@5)41.2%Third party——SimpleBench Leaderboard via Epoch AI

Mathematics

Mathematics results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
FrontierMath Tier 4 (v1)Accuracy4.2%Independent—18 Dec 2025Epoch AI
FrontierMath Tiers 1-3 (v1)Accuracy4.8%Independent—18 Dec 2025Epoch AI
OTIS Mock AIME 2024-2025Accuracy73.1%Independent—22 Apr 2025Epoch AI
OTIS Mock AIME 2024-2025Accuracy70.8%Independent—20 May 2025Epoch AI

Coding & software engineering

Coding & software engineering results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
Aider PolyglotPercent correct55.1%Third party23K—https://aider.chat/docs/leaderboards/ via Epoch AI
Aider PolyglotPercent correct47.1%Third party——https://aider.chat/docs/leaderboards/ via Epoch AI
Aider PolyglotPercent correct44.0%Third party——https://aider.chat/docs/leaderboards/ via Epoch AI
Terminal-Bench 2.0Accuracy (mean over trials)17.1%Third party——Terminal-Bench v2 Leaderboard via Epoch AI
Terminal-Bench 2.0Accuracy (mean over trials)17.1%Third party——https://www.tbench.ai/leaderboard/terminal-bench/2.0 via Epoch AI
Terminal-Bench 2.0Accuracy (mean over trials)16.9%Third party——Terminal-Bench v2 Leaderboard via Epoch AI
Terminal-Bench 2.0Accuracy (mean over trials)16.9%Third party——https://www.tbench.ai/leaderboard/terminal-bench/2.0 via Epoch AI
Terminal-Bench 2.0Accuracy (mean over trials)16.4%Third party——https://www.tbench.ai/leaderboard/terminal-bench/2.0 via Epoch AI
Terminal-Bench 2.0Accuracy (mean over trials)16.4%Third party——Terminal-Bench v2 Leaderboard via Epoch AI
Terminal-Bench 2.0Accuracy (mean over trials)15.4%Third party——https://www.tbench.ai/leaderboard/terminal-bench/2.0 via Epoch AI
Terminal-Bench 2.0Accuracy (mean over trials)15.4%Third party——Terminal-Bench v2 Leaderboard via Epoch AI
WeirdML (v2)Average accuracy41.9%Third party16k—https://htihle.github.io/weirdml.html via Epoch AI
WeirdML (v2)Average accuracy41.0%Third party16K—WeirdML Leaderboard via Epoch AI

Agentic tool use

Agentic tool use results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
BALROGAverage progress33.5%Third party——https://balrogai.com/ via Epoch AI

Professional knowledge work

Professional knowledge work results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
TheAgentCompanyTasks resolved41.1%Third party——TheAgentCompany leaderboard via Epoch AI

Long context

Long context results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
Fiction.LiveBench% correct at 16K tokens (Epoch's column)77.8%Third party——Fiction.live leaderboard via Epoch AI
Fiction.LiveBench% correct at 16K tokens (Epoch's column)47.2%Third party——Fiction.live leaderboard via Epoch AI

Vision & multimodal understanding

Vision & multimodal understanding results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
GeoBenchCountry accuracy76.0%Third party——GeoBench leaderboard via Epoch AI
GeoBenchCountry accuracy73.0%Third party——GeoBench leaderboard via Epoch AI
Visual Physics Comprehension Test (VPCT)Correct46.2%Third party——VPCT leaderboard via Epoch AI
Visual Physics Comprehension Test (VPCT)Correct38.0%Third party——VPCT leaderboard via Epoch AI

Human preference

Human preference results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
Text Arena (arena.ai)Arena score (Elo scale)1409Independent—25 Sep 2026Arena
Text Arena (arena.ai)Arena score (Elo scale)1403Independent—25 Sep 2026Arena
Text Arena (arena.ai)Votes125044Independent—25 Sep 2026Arena
Text Arena (arena.ai)Votes32853Independent—25 Sep 2026Arena
Vision Arena (arena.ai)Arena score (Elo scale)1230Independent—28 Sep 2026Arena
Vision Arena (arena.ai)Arena score (Elo scale)1215Independent—28 Sep 2026Arena
Vision Arena (arena.ai)Votes60141Independent—28 Sep 2026Arena
Vision Arena (arena.ai)Votes4793Independent—28 Sep 2026Arena

Other Gemini Flash versions

Sources