supermarket

Language

Grok 3

By SpaceXAI (xAI), in the Grok family.

Released
17 Feb 2025
Status
Retired
Retired
15 May 2026
Context
131.1K
Max output
8.2K
Input per 1M tokens
$3.00
Output per 1M tokens
$15.00
Open weights
No
License
proprietary
Parameters
—
Knowledge cutoff
Nov 2024
Reasoning
No
Modalities
text → text

Category scores

Out of 100, from independent and third-party results only, by method category-2026-09-29.1.

Category scores
CategoryScoreRankCoverage
Knowledge & reasoning23.3Not ranked1 of 4
Coding & software engineering53.3Not ranked1 of 5
Agentic tool use29.5Not ranked1 of 2

Benchmark results

Every result we hold for this model, with who ran it and where it was published.

Knowledge & reasoning

Knowledge & reasoning results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
SimpleBenchScore (AVG@5)36.1%Third party——SimpleBench Leaderboard via Epoch AI

Scientific reasoning

Scientific reasoning results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
GPQA DiamondAccuracy75.8%Independent—26 May 2025Epoch AI

Mathematics

Mathematics results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
FrontierMath Tier 4 (v1)Accuracy0.0%Independent—1 Jul 2025Epoch AI
FrontierMath Tiers 1-3 (v1)Accuracy3.8%Independent—10 Apr 2025Epoch AI
MATH Level 5Accuracy88.7%Independent—10 Apr 2025Epoch AI
OTIS Mock AIME 2024-2025Accuracy55.6%Independent—10 Apr 2025Epoch AI

Coding & software engineering

Coding & software engineering results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
Aider PolyglotPercent correct53.3%Third party——https://aider.chat/docs/leaderboards/ via Epoch AI
WeirdML (v2)Average accuracy37.2%Third party——https://htihle.github.io/weirdml.html via Epoch AI

Agentic tool use

Agentic tool use results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
BALROGAverage progress29.5%Third party——https://balrogai.com/ via Epoch AI

Long context

Long context results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
Fiction.LiveBench% correct at 16K tokens (Epoch's column)58.3%Third party——Fiction.live leaderboard via Epoch AI

Writing

Writing results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
LLM Creative Story-Writing Benchmark V1 (Lech Mazur)Mean rubric score (percent of the 10-point scale)76.4%Third party——lechmazur/writing Github repository via Epoch AI

Other Grok versions

Sources