Falcon-40B Instruct

By Technology Innovation Institute (TII), in the Falcon family.

Released: 25 May 2023
Status: Legacy
Context: 2K
Max output: —
Input per 1M tokens: —
Output per 1M tokens: —
Open weights: Yes
License: apache-2.0
Parameters: 41.84B
Knowledge cutoff: —
Reasoning: No
Modalities: text → text

Category scores

Out of 100, from independent and third-party results only, by method category-2026-09-29.1.

Not enough independent results yet

Only independent and third-party results rank a model; self-reported and unattributed numbers never do.

Benchmark results

Every result we hold for this model, with who ran it and where it was published.

Knowledge & reasoning

Knowledge & reasoning results
Benchmark	Metric	Value	Provenance	Setting	Evaluated	Source
AI2 Reasoning Challenge (ARC)	Challenge set accuracy	54.5%	Third party	—	—	Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI
AI2 Reasoning Challenge (ARC)	Challenge set accuracy	61.9%	Self-reported	—	—	Falcon2-11B Technical Report via Epoch AI
AI2 Reasoning Challenge (ARC)	Challenge set accuracy	56.7%	Self-reported	—	—	The Falcon Series of Open Language Models via Epoch AI
AI2 Reasoning Challenge (ARC)	Challenge set accuracy	54.7%	Self-reported	—	—	Falcon2-11B Technical Report via Epoch AI
BIG-Bench Hard (BBH)	Average accuracy	37.1%	Self-reported	—	—	Qwen Technical Report via Epoch AI
HellaSwag	Accuracy	85.3%	Self-reported	—	—	Falcon2-11B Technical Report via Epoch AI
HellaSwag	Accuracy	85.3%	Self-reported	—	—	Falcon2-11B Technical Report via Epoch AI
HellaSwag	Accuracy	82.8%	Self-reported	—	—	Falcon2-11B Technical Report via Epoch AI
HellaSwag	Accuracy	82.8%	Self-reported	—	—	Falcon2-11B Technical Report via Epoch AI
LAMBADA	Accuracy	77.3%	Self-reported	—	—	The Falcon Series of Open Language Models via Epoch AI
MMLU	Accuracy	55.4%	Third party	—	—	Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI
MMLU	Accuracy	56.9%	Self-reported	—	—	Falcon2-11B Technical ReportFalcon2-11B Technical Report via Epoch AI
MMLU	Accuracy	55.4%	Self-reported	—	—	Qwen Technical Report via Epoch AI
OpenBookQA	Accuracy	56.6%	Third party	—	—	Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI
PIQA	Accuracy	82.4%	Third party	—	—	Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI
PIQA	Accuracy	83.0%	Self-reported	—	—	The Falcon Series of Open Language Models via Epoch AI
WinoGrande	Accuracy	76.9%	Third party	—	—	Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI
WinoGrande	Accuracy	76.4%	Self-reported	—	—	Falcon2-11B Technical Report via Epoch AI
WinoGrande	Accuracy	76.4%	Self-reported	—	—	Falcon2-11B Technical Report via Epoch AI

Mathematics

Mathematics results
Benchmark	Metric	Value	Provenance	Setting	Evaluated	Source
GSM8K	Exact match	33.8%	Third party	—	—	Stanford HELM via Epoch AI
GSM8K	Exact match	25.0%	Third party	—	—	Stanford HELM via Epoch AI
GSM8K	Exact match	19.6%	Third party	—	—	Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI
GSM8K	Exact match	21.5%	Self-reported	—	—	Falcon2-11B Technical Report via Epoch AI
GSM8K	Exact match	19.6%	Self-reported	—	—	Qwen Technical Report via Epoch AI

Factuality & hallucination

Factuality & hallucination results
Benchmark	Metric	Value	Provenance	Setting	Evaluated	Source
TriviaQA	Exact match	79.9%	Third party	—	—	Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI
TriviaQA	Exact match	79.6%	Third party	—	—	Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI
TriviaQA	Exact match	78.6%	Third party	—	—	Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI
TriviaQA	Exact match	74.6%	Third party	—	—	Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI

Epoch AI, 'Capabilities & benchmarking', epoch.ai/benchmarks, CC BY 4.0.