Updated daily at 05:00 UTC
Benchmark
Cybench
Measures: 40 professional capture-the-flag cybersecurity challenges that an agent must solve in a sandbox without hints (unguided); scored on the share solved.
- Maintainer
- Cybench team (Stanford)
- Introduced
- 2024
- Status
- Active
- Saturation
- Saturated
- Typical provenance
- Mixed
- In scoring set
- No
Leaderboard
Independent and third-party results only.
| Model | Unguided challenges solved | Provenance | Source | Evaluated |
|---|---|---|---|---|
| Claude Sonnet 4.5 | 55.0% | Third party | Cybench leaderboard via Epoch AI | — |
| Claude Opus 4.1 | 42.0% | Third party | Cybench leaderboard via Epoch AI | — |
| Claude Opus 4 | 38.0% | Third party | Cybench leaderboard via Epoch AI | — |
| Claude Sonnet 4 | 35.0% | Third party | Cybench leaderboard via Epoch AI | — |
| o3-mini OpenAI · Medium effort | 22.5% | Third party | Cybench leaderboard via Epoch AI | — |
| Claude 3.7 Sonnet | 20.0% | Third party | Cybench leaderboard via Epoch AI | — |
| Claude 3.5 Sonnet | 17.5% | Third party | Cybench leaderboard via Epoch AI | — |
| GPT-4.5 | 17.5% | Third party | Cybench leaderboard via Epoch AI | — |
| GPT-4o | 12.5% | Third party | Cybench leaderboard via Epoch AI | — |
| Claude 3 Opus | 10.0% | Third party | Cybench leaderboard via Epoch AI | — |
| o1-mini OpenAI · Medium effort | 10.0% | Third party | Cybench leaderboard via Epoch AI | — |
| o1-preview | 10.0% | Third party | Cybench leaderboard via Epoch AI | — |
| Llama 3.1 405B | 7.5% | Third party | Cybench leaderboard via Epoch AI | — |
| Mixtral 8x22B | 7.5% | Third party | Cybench leaderboard via Epoch AI | — |
| Llama 3 70B | 5.0% | Third party | Cybench leaderboard via Epoch AI | — |
- Epoch AI, 'Capabilities & benchmarking', epoch.ai/benchmarks, CC BY 4.0.