supermarket

Vision-language

Gemma 3 12B

By Google DeepMind, in the Gemma family.

Released
10 Mar 2025
Status
Legacy
Context
131.1K
Max output
—
Input per 1M tokens
—
Output per 1M tokens
—
Open weights
Yes
License
gemma
Parameters
12.19B
Knowledge cutoff
Aug 2024
Reasoning
No
Modalities
text, image → text

Category scores

Out of 100, from independent and third-party results only, by method category-2026-09-29.1.

Category scores
CategoryScoreRankCoverage
Knowledge & reasoning2.61082 of 4
Human preference41.7Not ranked1 of 5

Benchmark results

Every result we hold for this model, with who ran it and where it was published.

Knowledge & reasoning

Knowledge & reasoning results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
Chess PuzzlesPuzzles solved0.0%Independent—28 Aug 2026Epoch AI
DTBench (Decision Theory Benchmark)Accuracy (capability questions)48.8%Third party——https://conceptualreasoning.ai/dtbench via Epoch AI
LMCA (Language Model Conceptual Argumentation)Correlation with expert ratings (Pearson r)0.0446Third party——https://conceptualreasoning.ai/lmca via Epoch AI

Scientific reasoning

Scientific reasoning results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
GPQA DiamondAccuracy39.5%Independent—28 Aug 2026Epoch AI

Mathematics

Mathematics results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
OTIS Mock AIME 2024-2025Accuracy16.7%Independent—28 Aug 2026Epoch AI

Human preference

Human preference results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
Text Arena (arena.ai)Arena score (Elo scale)1342Independent—25 Sep 2026Arena
Text Arena (arena.ai)Votes3829Independent—25 Sep 2026Arena

Other Gemma versions

Sources

Gemma 3 12B: benchmarks, price and specs · supermarket.dev