Updated daily at 05:00 UTC
- Released
- 18 Jul 2024
- Status
- Retired
- Retired
- 31 Jul 2026
- Context
- 131.1K
- Max output
- —
- Input per 1M tokens
- $0.15
- Output per 1M tokens
- $0.15
- Open weights
- Yes
- License
- apache-2.0
- Parameters
- 12B
- Knowledge cutoff
- —
- Reasoning
- No
- Modalities
- text → text
Category scores
Out of 100, from independent and third-party results only, by method category-2026-09-29.1.
| Category | Score | Rank | Coverage |
|---|---|---|---|
| Agentic tool use | 17.6 | Not ranked | 1 of 2 |
Benchmark results
Every result we hold for this model, with who ran it and where it was published.
Knowledge & reasoning
| Benchmark | Metric | Value | Provenance | Setting | Evaluated | Source |
|---|---|---|---|---|---|---|
| DTBench (Decision Theory Benchmark) | Accuracy (capability questions) | 48.6% | Third party | — | — | https://conceptualreasoning.ai/dtbench via Epoch AI |
Scientific reasoning
| Benchmark | Metric | Value | Provenance | Setting | Evaluated | Source |
|---|---|---|---|---|---|---|
| GPQA Diamond | Accuracy | 29.9% | Independent | — | 27 Jan 2025 | Epoch AI |
Mathematics
| Benchmark | Metric | Value | Provenance | Setting | Evaluated | Source |
|---|---|---|---|---|---|---|
| MATH Level 5 | Accuracy | 10.8% | Independent | — | 27 Jan 2025 | Epoch AI |
Agentic tool use
| Benchmark | Metric | Value | Provenance | Setting | Evaluated | Source |
|---|---|---|---|---|---|---|
| BALROG | Average progress | 17.6% | Third party | — | — | https://balrogai.com/ via Epoch AI |