supermarket

Benchmark

GSO

Measures: Software performance optimisation: in real repositories an agent must make a program faster; OPT@1 is the share of tasks where one attempt reaches at least 95% of the speedup a human expert achieved.

Maintainer
GSO team
Introduced
2025
Status
Active
Saturation
Headroom
Typical provenance
Independent
In scoring set
Yes

PaperOfficial leaderboard

Leaderboard

Show all results

Independent and third-party results only.

Leaderboard
ModelOPT@1ProvenanceSourceEvaluated
GPT-4o
OpenAI
0.0%Third partyGSO Leaderboard via Epoch AI—
o3-mini
OpenAI · Low effort
1.3%Third partyGSO Leaderboard via Epoch AI—
o3-mini
OpenAI · High effort
1.3%Third partyGSO Leaderboard via Epoch AI—
GLM-4.5-Air
Z.ai (Zhipu AI)
2.9%Third partyGSO Leaderboard via Epoch AI—
o4-mini
OpenAI · High effort
3.6%Third partyGSO Leaderboard via Epoch AI—
Claude 3.7 Sonnet
Anthropic
3.8%Third partyGSO Leaderboard via Epoch AI—
Claude 3.7 Sonnet
Anthropic · Unknown effort
3.8%Third partyGSO Leaderboard via Epoch AI—
Gemini 2.5 Pro
Google DeepMind
3.9%Third partyhttps://gso-bench.github.io/index.html via Epoch AI—
Gemini 2.5 Pro
Google DeepMind
3.9%Third partyGSO Leaderboard via Epoch AI—
Claude 3.5 Sonnet
Anthropic
4.6%Third partyGSO Leaderboard via Epoch AI—
Claude Sonnet 4
Anthropic
4.9%Third partyGSO Leaderboard via Epoch AI—
Claude Sonnet 4
Anthropic · Unknown effort
4.9%Third partyGSO Leaderboard via Epoch AI—
Qwen3-Coder-480B-A35B-Instruct
Alibaba Group
4.9%Third partyhttps://gso-bench.github.io/index.html via Epoch AI—
Claude Opus 4
Anthropic · Unknown effort
6.9%Third partyGSO Leaderboard via Epoch AI—
GPT-5
OpenAI · High effort
6.9%Third partyhttps://gso-bench.github.io/index.html via Epoch AI—
Claude Opus 4
Anthropic
6.9%Third partyGSO Leaderboard via Epoch AI—
o3
OpenAI · High effort
8.8%Third partyGSO Leaderboard via Epoch AI—
Gemini 3 Flash
Google DeepMind
9.8%Third partyhttps://gso-bench.github.io/index.html via Epoch AI—
GPT-5.1
OpenAI · High effort
13.7%Third partyGSO Leaderboard via Epoch AI—
GPT-5.1
OpenAI · Unknown effort
13.7%Third partyhttps://gso-bench.github.io/index.html via Epoch AI—
Claude Sonnet 4.5
Anthropic
14.7%Third partyGSO Leaderboard via Epoch AI—
Claude Sonnet 4.5
Anthropic · Unknown effort
14.7%Third partyGSO Leaderboard via Epoch AI—
Gemini 3 Pro
Google DeepMind
18.6%Third partyhttps://gso-bench.github.io/index.html via Epoch AI—
Gemini 3.1 Pro
Google DeepMind
22.6%Third partyhttps://gso-bench.github.io/index.html via Epoch AI—
GPT-5.4
OpenAI · High effort
25.5%Third partyhttps://gso-bench.github.io/index.html via Epoch AI—
Claude Opus 4.5
Anthropic · Unknown effort
26.5%Third partyGSO Leaderboard via Epoch AI—
Claude Opus 4.5
Anthropic
26.5%Third partyGSO Leaderboard via Epoch AI—
GPT-5.2
OpenAI · High effort
27.4%Third partyhttps://gso-bench.github.io/index.html via Epoch AI—
GPT-5.4
OpenAI · Xhigh effort
31.4%Third partyhttps://gso-bench.github.io/index.html via Epoch AI—
Claude Opus 4.6
Anthropic · Unknown effort
33.3%Third partyGSO Leaderboard via Epoch AI—
Claude Opus 4.6
Anthropic
33.3%Third partyGSO Leaderboard via Epoch AI—
Claude Sonnet 5
Anthropic · Unknown effort
37.3%Third partyhttps://gso-bench.github.io/index.html via Epoch AI—
GPT-5.5
OpenAI · Xhigh effort
40.2%Third partyhttps://gso-bench.github.io/index.html via Epoch AI—
Claude Opus 4.6
Anthropic · High effort
41.2%Third partyGSO Leaderboard via Epoch AI—
Claude Opus 4.7
Anthropic · High effort
44.1%Third partyhttps://gso-bench.github.io/index.html via Epoch AI—
Claude Opus 4.7
Anthropic
44.1%Third partyGSO Leaderboard via Epoch AI—
Claude Opus 4.8
Anthropic · Unknown effort
47.1%Third partyhttps://gso-bench.github.io/index.html via Epoch AI—
  • Epoch AI, 'Capabilities & benchmarking', epoch.ai/benchmarks, CC BY 4.0.

Sources