Updated daily at 05:00 UTC
- Released
- 21 Feb 2024
- Status
- Legacy
- Context
- 8.2K
- Max output
- —
- Input per 1M tokens
- —
- Output per 1M tokens
- —
- Open weights
- Yes
- License
- gemma
- Parameters
- 2.51B
- Knowledge cutoff
- —
- Reasoning
- No
- Modalities
- text → text
Category scores
Out of 100, from independent and third-party results only, by method category-2026-09-29.1.
| Category | Score | Rank | Coverage |
|---|---|---|---|
| Human preference | 14.6 | Not ranked | 1 of 5 |
Benchmark results
Every result we hold for this model, with who ran it and where it was published.
Knowledge & reasoning
| Benchmark | Metric | Value | Provenance | Setting | Evaluated | Source |
|---|---|---|---|---|---|---|
| AI2 Reasoning Challenge (ARC) | Challenge set accuracy | 42.1% | Self-reported | — | — | Gemma: Open Models Based on Gemini Research and Technology via Epoch AI |
| BIG-Bench Hard (BBH) | Average accuracy | 35.2% | Self-reported | — | — | Gemma: Open Models Based on Gemini Research and Technology via Epoch AI |
| HellaSwag | Accuracy | 71.4% | Self-reported | — | — | Gemma: Open Models Based on Gemini Research and Technology via Epoch AI |
| MMLU | Accuracy | 42.3% | Self-reported | — | — | Gemma: Open Models Based on Gemini Research and Technology via Epoch AI |
| PIQA | Accuracy | 77.3% | Self-reported | — | — | Gemma: Open Models Based on Gemini Research and Technology via Epoch AI |
| WinoGrande | Accuracy | 65.4% | Self-reported | — | — | Gemma: Open Models Based on Gemini Research and Technology via Epoch AI |
Mathematics
| Benchmark | Metric | Value | Provenance | Setting | Evaluated | Source |
|---|---|---|---|---|---|---|
| GSM8K | Exact match | 17.7% | Self-reported | — | — | Gemma: Open Models Based on Gemini Research and Technology via Epoch AI |
Factuality & hallucination
| Benchmark | Metric | Value | Provenance | Setting | Evaluated | Source |
|---|---|---|---|---|---|---|
| TriviaQA | Exact match | 53.2% | Self-reported | — | — | Gemma: Open Models Based on Gemini Research and Technology via Epoch AI |
Human preference
| Benchmark | Metric | Value | Provenance | Setting | Evaluated | Source |
|---|---|---|---|---|---|---|
| Text Arena (arena.ai) | Arena score (Elo scale) | 1094 | Independent | — | 25 Sep 2026 | Arena |
| Text Arena (arena.ai) | Votes | 4780 | Independent | — | 25 Sep 2026 | Arena |
Other Gemma versions
- Gemma 4 12B Unified3 Jun 2026
- Gemma 4 26B A4B31 Mar 2026
- Gemma 4 31B31 Mar 2026
- Gemma 4 E2B31 Mar 2026
- Gemma 4 E4B31 Mar 2026
- Gemma 3 270M14 Aug 2025
- Gemma 3n E2B26 Jun 2025
- Gemma 3n E4B26 Jun 2025
- Gemma 3 12B10 Mar 2025
- Gemma 3 1B10 Mar 2025
- Gemma 3 27B10 Mar 2025
- Gemma 3 4B10 Mar 2025
- Gemma 2 JPN 2B3 Oct 2024
- Gemma 2 2B31 Jul 2024
- Gemma 2 27B27 Jun 2024
- Gemma 2 9B27 Jun 2024
- Gemma 1.1 2B5 Apr 2024
- Gemma 1.1 7B5 Apr 2024
- Gemma 7B21 Feb 2024