Updated daily at 05:00 UTC
Benchmark
Adversarial NLI (ANLI)
Measures: Natural language inference examples written adversarially against strong models over three rounds; the model says whether a hypothesis follows from, contradicts or is neutral to a premise.
- Maintainer
- Nie et al. (Facebook AI and UNC)
- Introduced
- 2019
- Status
- Unmaintained
- Saturation
- Near-ceiling
- Typical provenance
- Self-reported
- In scoring set
- No
Leaderboard
Independent and third-party results only.
| Model | Accuracy | Provenance | Source | Evaluated |
|---|---|---|---|---|
| text-davinci-001 (InstructGPT) | 33.9% | Third party | Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model via Epoch AI | — |
| text-davinci-001 (InstructGPT) | 34.0% | Third party | Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model via Epoch AI | — |
| text-davinci-001 (InstructGPT) | 35.4% | Third party | Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model via Epoch AI | — |
- Epoch AI, 'Capabilities & benchmarking', epoch.ai/benchmarks, CC BY 4.0.