SciCode

Measures: 80 scientist-written research problems in 16 natural-science fields, split into subproblems that each need knowledge recall, reasoning and code; a model writes Python that must pass tests, scored at the subproblem level (pass@1) and as main-problem resolve rate.

Maintainer: SciCode authors (Tian et al.); actively re-run by Artificial Analysis
Introduced: 2024
Status: Active
Saturation: Headroom
Typical provenance: Mixed
In scoring set: Yes

Paper Official leaderboard

Leaderboard

Show independent results only

Every result we hold, whoever ran it. Only independent and third-party rows count towards a score.

No results to show.