Updated daily at 05:00 UTC
Benchmark
LAMBADA
Measures: Predict the last word of a narrative passage, where the word can only be guessed from the whole passage and not from the final sentence alone.
- Maintainer
- Paperno et al.
- Introduced
- 2016
- Status
- Unmaintained
- Saturation
- Near-ceiling
- Typical provenance
- Self-reported
- In scoring set
- No
Leaderboard
Independent and third-party results only.
| Model | Accuracy | Provenance | Source | Evaluated |
|---|---|---|---|---|
| text-davinci-001 (InstructGPT) | 72.5% | Third party | GLaM: Efficient Scaling of Language Models with Mixture-of-Experts via Epoch AI | — |
| text-davinci-001 (InstructGPT) | 72.5% | Third party | Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model via Epoch AI | — |
| text-davinci-001 (InstructGPT) | 76.2% | Third party | Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model via Epoch AI | — |
| text-davinci-001 (InstructGPT) | 76.2% | Third party | The Falcon Series of Open Language Models via Epoch AI | — |
| Inflection-1 | 78.5% | Third party | The Falcon Series of Open Language Models via Epoch AI | — |
| text-davinci-001 (InstructGPT) | 86.4% | Third party | GLaM: Efficient Scaling of Language Models with Mixture-of-Experts via Epoch AI | — |
| text-davinci-001 (InstructGPT) | 86.4% | Third party | Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model via Epoch AI | — |
- Epoch AI, 'Capabilities & benchmarking', epoch.ai/benchmarks, CC BY 4.0.