supermarket

Language

text-davinci-001 (InstructGPT)

By OpenAI, in the GPT-3 family.

Released
27 Jan 2022
Status
Retired
Retired
4 Jan 2024
Context
—
Max output
—
Input per 1M tokens
$20.00
Output per 1M tokens
$20.00
Open weights
No
License
proprietary
Parameters
175B
Knowledge cutoff
—
Reasoning
No
Modalities
text → text

Category scores

Out of 100, from independent and third-party results only, by method category-2026-09-29.1.

Not enough independent results yet

Only independent and third-party results rank a model; self-reported and unattributed numbers never do.

Benchmark results

Every result we hold for this model, with who ran it and where it was published.

Knowledge & reasoning

Knowledge & reasoning results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
Adversarial NLI (ANLI)Accuracy35.4%Third party——Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model via Epoch AI
Adversarial NLI (ANLI)Accuracy34.0%Third party——Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model via Epoch AI
Adversarial NLI (ANLI)Accuracy33.9%Third party——Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model via Epoch AI
AI2 Reasoning Challenge (ARC)Challenge set accuracy53.2%Third party——GLaM: Efficient Scaling of Language Models with Mixture-of-Experts via Epoch AI
AI2 Reasoning Challenge (ARC)Challenge set accuracy51.5%Third party——GLaM: Efficient Scaling of Language Models with Mixture-of-Experts via Epoch AI
AI2 Reasoning Challenge (ARC)Challenge set accuracy51.4%Third party——GLaM: Efficient Scaling of Language Models with Mixture-of-Experts via Epoch AI
AI2 Reasoning Challenge (ARC)Challenge set accuracy51.4%Third party——The Falcon Series of Open Language Models via Epoch AI
AI2 Reasoning Challenge (ARC)Challenge set accuracy51.4%Third party——LLaMA: Open and Efficient Foundation Language Models via Epoch AI
HellaSwagAccuracy79.3%Third party——Language Models are Few-Shot Learners via Epoch AI
HellaSwagAccuracy79.3%Third party——GLaM: Efficient Scaling of Language Models with Mixture-of-Experts via Epoch AI
HellaSwagAccuracy79.3%Third party——Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model via Epoch AI
HellaSwagAccuracy78.9%Third party——Language Models are Few-Shot Learners via Epoch AI
HellaSwagAccuracy78.9%Third party——Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model via Epoch AI
HellaSwagAccuracy78.9%Third party——GLaM: Efficient Scaling of Language Models with Mixture-of-Experts via Epoch AI
HellaSwagAccuracy78.1%Third party——GLaM: Efficient Scaling of Language Models with Mixture-of-Experts via Epoch AI
HellaSwagAccuracy78.1%Third party——Language Models are Few-Shot Learners via Epoch AI
HellaSwagAccuracy78.1%Third party——Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model via Epoch AI
LAMBADAAccuracy86.4%Third party——GLaM: Efficient Scaling of Language Models with Mixture-of-Experts via Epoch AI
LAMBADAAccuracy86.4%Third party——Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model via Epoch AI
LAMBADAAccuracy76.2%Third party——Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model via Epoch AI
LAMBADAAccuracy76.2%Third party——The Falcon Series of Open Language Models via Epoch AI
LAMBADAAccuracy72.5%Third party——Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model via Epoch AI
LAMBADAAccuracy72.5%Third party——GLaM: Efficient Scaling of Language Models with Mixture-of-Experts via Epoch AI
MMLUAccuracy43.9%Third party——Scaling Language Models: Methods, Analysis & Insights from Training Gopher via Epoch AI
MMLUAccuracy43.9%Third party——LLaMA: Open and Efficient Foundation Language Models via Epoch AI
OpenBookQAAccuracy65.4%Third party——GLaM: Efficient Scaling of Language Models with Mixture-of-Experts via Epoch AI
OpenBookQAAccuracy65.4%Third party——Language Models are Few-Shot Learners via Epoch AI
OpenBookQAAccuracy58.8%Third party——GLaM: Efficient Scaling of Language Models with Mixture-of-Experts via Epoch AI
OpenBookQAAccuracy58.8%Third party——Language Models are Few-Shot Learners via Epoch AI
OpenBookQAAccuracy57.6%Third party——GLaM: Efficient Scaling of Language Models with Mixture-of-Experts via Epoch AI
OpenBookQAAccuracy57.6%Third party——Language Models are Few-Shot Learners via Epoch AI
OpenBookQAAccuracy57.6%Third party——LLaMA: Open and Efficient Foundation Language Models via Epoch AI
PIQAAccuracy82.3%Third party——Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model via Epoch AI
PIQAAccuracy82.3%Third party——GLaM: Efficient Scaling of Language Models with Mixture-of-Experts via Epoch AI
PIQAAccuracy81.0%Third party——LLaMA: Open and Efficient Foundation Language Models via Epoch AI
PIQAAccuracy81.0%Third party——Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model via Epoch AI
PIQAAccuracy81.0%Third party——The Falcon Series of Open Language Models via Epoch AI
PIQAAccuracy80.5%Third party——GLaM: Efficient Scaling of Language Models with Mixture-of-Experts via Epoch AI
PIQAAccuracy80.5%Third party——Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model via Epoch AI
PIQAAccuracy80.4%Third party——GLaM: Efficient Scaling of Language Models with Mixture-of-Experts via Epoch AI
ScienceQAAccuracy74.0%Third party——ScienceQA leaderboard via Epoch AI
ScienceQAAccuracy74.0%Third party——ScienceQA leaderboard via Epoch AI
SuperGLUEAverage score71.8%Third party——Language Models are Few-Shot Learners via Epoch AI
WinoGrandeAccuracy77.7%Third party——Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model via Epoch AI
WinoGrandeAccuracy77.7%Third party——Language Models are Few-Shot Learners via Epoch AI
WinoGrandeAccuracy77.7%Third party——GLaM: Efficient Scaling of Language Models with Mixture-of-Experts via Epoch AI
WinoGrandeAccuracy77.7%Third party——Language Models are Few-Shot Learners via Epoch AI
WinoGrandeAccuracy73.2%Third party——Language Models are Few-Shot Learners via Epoch AI
WinoGrandeAccuracy73.2%Third party——Language Models are Few-Shot Learners via Epoch AI
WinoGrandeAccuracy73.2%Third party——Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model via Epoch AI
WinoGrandeAccuracy73.2%Third party——GLaM: Efficient Scaling of Language Models with Mixture-of-Experts via Epoch AI
WinoGrandeAccuracy70.2%Third party——Language Models are Few-Shot Learners via Epoch AI
WinoGrandeAccuracy70.2%Third party——LLaMA: Open and Efficient Foundation Language Models via Epoch AI
WinoGrandeAccuracy70.2%Third party——Language Models are Few-Shot Learners via Epoch AI
WinoGrandeAccuracy70.2%Third party——Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model via Epoch AI
WinoGrandeAccuracy70.2%Third party——GLaM: Efficient Scaling of Language Models with Mixture-of-Experts via Epoch AI

Factuality & hallucination

Factuality & hallucination results
BenchmarkMetricValueProvenanceSettingEvaluatedSource
TriviaQAExact match71.2%Third party——Language Models are Few-Shot Learners via Epoch AI
TriviaQAExact match71.2%Third party——Language Models are Few-Shot Learners via Epoch AI
TriviaQAExact match71.2%Third party——GLaM: Efficient Scaling of Language Models with Mixture-of-Experts via Epoch AI
TriviaQAExact match68.0%Third party——GLaM: Efficient Scaling of Language Models with Mixture-of-Experts via Epoch AI
TriviaQAExact match68.0%Third party——Language Models are Few-Shot Learners via Epoch AI
TriviaQAExact match68.0%Third party——Language Models are Few-Shot Learners via Epoch AI
TriviaQAExact match64.3%Third party——GLaM: Efficient Scaling of Language Models with Mixture-of-Experts via Epoch AI
TriviaQAExact match64.3%Third party——Language Models are Few-Shot Learners via Epoch AI
TriviaQAExact match64.3%Third party——Language Models are Few-Shot Learners via Epoch AI

Other GPT-3 versions

Sources

text-davinci-001 (InstructGPT): benchmarks, price and specs · supermarket.dev