Updated daily at 05:00 UTC
Benchmark
OpenAI MRCR v2 (8-needle)
Measures: The model reads a long synthetic multi-turn conversation that hides 8 near-identical requests and must reproduce the i-th matching response, prefixed by a required hash; scored by difflib SequenceMatcher ratio and reported per context-length bin from 4K up to 1M tokens.
- Maintainer
- OpenAI (openai/mrcr dataset); the MRCR task comes from Google DeepMind's Michelangelo paper, and Google DeepMind maintains its own separate MRCR v2 in eval_hub
- Introduced
- 2025
- Status
- Active
- Saturation
- Near-ceiling
- Typical provenance
- Self-reported
- In scoring set
- No
Leaderboard
Independent and third-party results only.
No independent or third-party results yet. Show all results