SWE-rebench

Measures: An agent must resolve fresh real GitHub issues collected in rolling time windows after models were released, which limits contamination; scored by mean % resolved over repeated runs (with SEM) plus pass@5.

Maintainer: Nebius (SWE-rebench team)
Introduced: 2025
Status: Active
Saturation: Headroom
Typical provenance: Independent
In scoring set: Yes

Paper Official leaderboard

Leaderboard

Show independent results only

Every result we hold, whoever ran it. Only independent and third-party rows count towards a score.

No results to show.