PIQA

Measures: Physical commonsense: for an everyday goal, choose the more sensible of two solutions.

Maintainer: Bisk et al. (AI2 and University of Washington)
Introduced: 2019
Status: Unmaintained
Saturation: Near-ceiling
Typical provenance: Self-reported
In scoring set: No

Leaderboard

Independent and third-party results only.

Leaderboard
Model	Accuracy	Provenance	Source	Evaluated
Falcon-7B Instruct Technology Innovation Institute (TII)	76.7%	Third party	Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI	—
Llama 2 7B Meta	77.9%	Third party	Mixtral of Experts via Epoch AI	—
Llama 2 7B Meta	78.8%	Third party	Gemma: Open Models Based on Gemini Research and Technology via Epoch AI	—
Llama 2 7B Meta	78.8%	Third party	The Falcon Series of Open Language Models via Epoch AI	—
text-davinci-001 (InstructGPT) OpenAI	80.4%	Third party	GLaM: Efficient Scaling of Language Models with Mixture-of-Experts via Epoch AI	—
Llama 2 13B Meta	80.5%	Third party	Gemma: Open Models Based on Gemini Research and Technology via Epoch AI	—
Llama 2 13B Meta	80.5%	Third party	The Falcon Series of Open Language Models via Epoch AI	—
text-davinci-001 (InstructGPT) OpenAI	80.5%	Third party	GLaM: Efficient Scaling of Language Models with Mixture-of-Experts via Epoch AI	—
text-davinci-001 (InstructGPT) OpenAI	80.5%	Third party	Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model via Epoch AI	—
MPT-7B (Instruct, Chat, StoryWriter) Databricks	80.6%	Third party	Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI	—
Llama 2 13B Meta	80.8%	Third party	Mixtral of Experts via Epoch AI	—
text-davinci-001 (InstructGPT) OpenAI	81.0%	Third party	Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model via Epoch AI	—
text-davinci-001 (InstructGPT) OpenAI	81.0%	Third party	LLaMA: Open and Efficient Foundation Language Models via Epoch AI	—
text-davinci-001 (InstructGPT) OpenAI	81.0%	Third party	The Falcon Series of Open Language Models via Epoch AI	—
MPT-30B (Instruct, Chat) Databricks	81.9%	Third party	Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI	—
Mistral 7B v0.1 Mistral AI	82.2%	Third party	Mixtral of Experts via Epoch AI	—
Mistral 7B v0.2 Mistral AI	82.2%	Third party	Gemma: Open Models Based on Gemini Research and Technology via Epoch AI	—
text-davinci-001 (InstructGPT) OpenAI	82.3%	Third party	GLaM: Efficient Scaling of Language Models with Mixture-of-Experts via Epoch AI	—
text-davinci-001 (InstructGPT) OpenAI	82.3%	Third party	Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model via Epoch AI	—
Falcon-40B Instruct Technology Innovation Institute (TII)	82.4%	Third party	Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI	—
Inflection-1 Inflection AI	84.2%	Third party	The Falcon Series of Open Language Models via Epoch AI	—

Epoch AI, 'Capabilities & benchmarking', epoch.ai/benchmarks, CC BY 4.0.

Sources

Epoch AI