Benchmark

TriviaQA

Measures: Trivia questions written by enthusiasts, answered closed-book by the model; scored by exact match with the reference answer.

Maintainer
Joshi et al. (University of Washington)
Introduced
2017
Status
Unmaintained
Saturation
Near-ceiling
Typical provenance
Self-reported
In scoring set
No

Paper

Leaderboard

Independent and third-party results only.

Leaderboard
ModelExact matchProvenanceSourceEvaluated
Falcon-7B Instruct52.6%Third partyLlama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI—
MPT-7B (Instruct, Chat, StoryWriter)55.7%Third partyLlama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI—
Falcon-7B Instruct56.8%Third partyLlama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI—
MPT-7B (Instruct, Chat, StoryWriter)59.6%Third partyLlama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI—
Falcon-7B Instruct61.1%Third partyLlama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI—
MPT-7B (Instruct, Chat, StoryWriter)61.2%Third partyLlama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI—
MPT-7B (Instruct, Chat, StoryWriter)61.6%Third partyLlama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI—
Mistral 7B v0.162.5%Third partyGemma: Open Models Based on Gemini Research and Technology via Epoch AI—
text-davinci-001 (InstructGPT)64.3%Third partyLanguage Models are Few-Shot Learners via Epoch AI—
text-davinci-001 (InstructGPT)64.3%Third partyGLaM: Efficient Scaling of Language Models with Mixture-of-Experts via Epoch AI—
text-davinci-001 (InstructGPT)64.3%Third partyLanguage Models are Few-Shot Learners via Epoch AI—
Falcon-7B Instruct64.6%Third partyLlama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI—
MPT-30B (Instruct, Chat)68.0%Third partyLlama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI—
text-davinci-001 (InstructGPT)68.0%Third partyLanguage Models are Few-Shot Learners via Epoch AI—
text-davinci-001 (InstructGPT)68.0%Third partyGLaM: Efficient Scaling of Language Models with Mixture-of-Experts via Epoch AI—
text-davinci-001 (InstructGPT)68.0%Third partyLanguage Models are Few-Shot Learners via Epoch AI—
text-davinci-001 (InstructGPT)71.2%Third partyLanguage Models are Few-Shot Learners via Epoch AI—
text-davinci-001 (InstructGPT)71.2%Third partyGLaM: Efficient Scaling of Language Models with Mixture-of-Experts via Epoch AI—
text-davinci-001 (InstructGPT)71.2%Third partyLanguage Models are Few-Shot Learners via Epoch AI—
MPT-30B (Instruct, Chat)71.3%Third partyLlama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI—
Llama 2 7B72.1%Third partyGemma: Open Models Based on Gemini Research and Technology via Epoch AI—
MPT-30B (Instruct, Chat)73.3%Third partyLlama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI—
MPT-30B (Instruct, Chat)73.6%Third partyLlama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI—
Falcon-40B Instruct74.6%Third partyLlama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI—
Falcon-40B Instruct78.6%Third partyLlama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI—
Falcon-40B Instruct79.6%Third partyLlama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI—
Llama 2 13B79.6%Third partyGemma: Open Models Based on Gemini Research and Technology via Epoch AI—
Falcon-40B Instruct79.9%Third partyLlama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI—
GPT-484.8%Third partyPapersWithCode via Epoch AI—
  • Epoch AI, 'Capabilities & benchmarking', epoch.ai/benchmarks, CC BY 4.0.

Sources