Updated daily at 05:00 UTC
Benchmark
SciCode
Measures: 80 scientist-written research problems in 16 natural-science fields, split into subproblems that each need knowledge recall, reasoning and code; a model writes Python that must pass tests, scored at the subproblem level (pass@1) and as main-problem resolve rate.
- Maintainer
- SciCode authors (Tian et al.); actively re-run by Artificial Analysis
- Introduced
- 2024
- Status
- Active
- Saturation
- Headroom
- Typical provenance
- Mixed
- In scoring set
- Yes
Leaderboard
Independent and third-party results only.
No independent or third-party results yet. Show all results