Updated daily at 05:00 UTC
Benchmark
SWE-rebench
Measures: An agent must resolve fresh real GitHub issues collected in rolling time windows after models were released, which limits contamination; scored by mean % resolved over repeated runs (with SEM) plus pass@5.
- Maintainer
- Nebius (SWE-rebench team)
- Introduced
- 2025
- Status
- Active
- Saturation
- Headroom
- Typical provenance
- Independent
- In scoring set
- Yes
Leaderboard
Independent and third-party results only.
No independent or third-party results yet. Show all results