MedHELM

Measures: A suite of 121 clinician-validated medical tasks (31 datasets, 12 private) across decision support, note generation, patient communication, research help and administration; open-ended outputs are rated by a 3-LLM jury and models are ranked by mean win rate.

Maintainer: Stanford CRFM (origin); now community-led with Pacific AI stewardship
Introduced: 2025
Status: Active
Saturation: —
Typical provenance: Independent
In scoring set: No

Not scored: Ranks by mean win rate, which depends on the set of models compared (HELM Capabilities dropped it for this reason); only 11 models.

Paper Official leaderboard

Leaderboard

Show independent results only

Every result we hold, whoever ran it. Only independent and third-party rows count towards a score.

No results to show.