DTBench (Decision Theory Benchmark)

Measures: 407 handcrafted multiple-choice questions with objectively correct answers on the decision theory of Newcomb-like problems.

Maintainer: Redwood Research (questions by Caspar Oesterheld)
Introduced: 2024
Status: Active
Saturation: Saturated
Typical provenance: Independent
In scoring set: No

Paper Official leaderboard

Leaderboard

Show independent results only

Every result we hold, whoever ran it. Only independent and third-party rows count towards a score.

Leaderboard
Model	Accuracy (capability questions)	Provenance	Source	Evaluated
Claude Opus 5.5 Anthropic · Max effort	98.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Fable 5 Anthropic · Max effort	98.4%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Opus 5 Anthropic · Xhigh effort	97.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Opus 5 Anthropic · High effort	97.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Fable 5.1 Anthropic · Unknown effort	97.6%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Fable 5.1 Anthropic · Xhigh effort	97.6%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Opus 5 Anthropic · Max effort	97.6%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Fable 5 Anthropic · Xhigh effort	97.3%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-6 Astra OpenAI · Xhigh effort	97.3%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-6 Sol OpenAI · Max effort	97.3%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Grok 4.6 SpaceXAI (xAI) · Xhigh effort	97.3%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Fable 5.1 Anthropic · Medium effort	97.1%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Gemini 3.1 Pro Google DeepMind · Medium effort	97.1%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-6 Astra OpenAI · Unknown effort	97.1%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-6 Astra OpenAI · Medium effort	97.1%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Gemini 3.7 Flash Google DeepMind · High effort	96.8%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Fable 5 Anthropic · High effort	96.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Fable 5.1 Anthropic · High effort	96.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-6 Astra OpenAI · High effort	96.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Grok 4.5 SpaceXAI (xAI) · High effort	96.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Muse Spark 1.3 Meta · Unknown effort	96.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-5.5 OpenAI · Xhigh effort	96.0%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-5.5 Pro OpenAI · Xhigh effort	96.0%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-5.6 Sol OpenAI · Promax effort	96.0%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Fable 5 Anthropic · Medium effort	95.7%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-5.6 Sol OpenAI · Xhigh effort	95.7%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Gemini 3.8 Flash Google DeepMind · Unknown effort	95.7%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Gemini 3.1 Pro Google DeepMind · High effort	95.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Gemini 3.6 Flash Google DeepMind · High effort	95.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-5.6 Sol OpenAI · Max effort	95.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-5.6 Sol OpenAI · High effort	95.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-5.6 Sol OpenAI · Medium effort	95.2%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Opus 4.8 Anthropic · Max effort	94.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Opus 5 Anthropic · Medium effort	94.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Gemini 3.1 Pro Google DeepMind · Low effort	94.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Gemini 3.8 Flash Google DeepMind · Medium effort	94.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Gemini 3.8 Flash Google DeepMind · Low effort	94.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-6 Astra OpenAI · Low effort	94.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Gemini 3.5 Flash Google DeepMind · High effort	94.7%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Gemini 3.7 Flash Google DeepMind · Medium effort	94.7%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Muse Spark 1.2 Meta · Xhigh effort	94.7%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Opus 4.7 Anthropic · Max effort	94.7%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-5.4 OpenAI · Xhigh effort	94.4%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Muse Spark 1.1 Meta · High effort	94.4%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Fable 5.1 Anthropic · Low effort	94.1%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
DeepSeek-V4-Pro DeepSeek · Unknown effort	93.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-5.6 Sol OpenAI · Low effort	93.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GLM-5.2 Z.ai (Zhipu AI) · Max effort	93.6%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Gemini 3.7 Flash Google DeepMind · Low effort	93.3%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-5.6 Terra OpenAI · Max effort	93.3%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Opus 5 Anthropic · Low effort	93.1%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-5.6 Terra OpenAI · Xhigh effort	92.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Sonnet 5 Anthropic · Max effort	92.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Qwen3.7-Max Alibaba Group	92.3%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Qwen3.8-Max Alibaba Group · Xhigh effort	92.0%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Fable 5 Anthropic · Low effort	91.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Opus 4.6 Anthropic · Max effort	91.2%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-5.6 Terra OpenAI · High effort	91.2%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Kimi K3 Moonshot AI · Max effort	91.2%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
DeepSeek-V4-Flash DeepSeek · Max effort	90.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-5.2 OpenAI · Xhigh effort	90.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Kimi K2.6 Moonshot AI	90.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
DeepSeek-V4-Pro DeepSeek · Max effort	90.7%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-5 OpenAI · High effort	90.7%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Grok 4.3 SpaceXAI (xAI) · High effort	90.7%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-5.1 OpenAI · High effort	90.1%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-5.6 Terra OpenAI · Medium effort	90.1%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-5.6 Terra OpenAI · Low effort	90.1%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-6 Luna OpenAI · Max effort	90.1%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Grok 4.20 SpaceXAI (xAI)	90.1%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Opus 4.5 Anthropic · High effort	89.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Sonnet 4.6 Anthropic · Max effort	89.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
DeepSeek-V4.1-Flash DeepSeek · Unknown effort	89.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Sonnet 5 Anthropic · Xhigh effort	89.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Gemini 3 Flash Google DeepMind · High effort	89.1%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-5.6 Luna OpenAI · Xhigh effort	89.1%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-5.6 Luna OpenAI · Max effort	88.8%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Qwen3.8-27B Alibaba Group	88.0%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
DeepSeek-V3.2-Exp DeepSeek · Thinking effort	87.7%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Grok 4.1 Fast SpaceXAI (xAI)	87.7%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GLM-5.3 Z.ai (Zhipu AI) · Unknown effort	87.7%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Inkling Thinking Machines Lab · Xhigh effort	87.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Qwen3.5-397B-A17B Alibaba Group	87.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Qwen3.6-Max-Preview Alibaba Group	87.2%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
o3-pro OpenAI · High effort	86.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-5.6 Luna OpenAI · High effort	86.7%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
DeepSeek-V4-Flash DeepSeek · Max effort	86.4%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
DeepSeek-V3.2 DeepSeek	85.6%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
o3 OpenAI · High effort	84.8%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Sonnet 5 Anthropic · High effort	84.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
MiMo-V2.5-Pro Xiaomi	84.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Qwen3.5-122B-A10B Alibaba Group · None effort	84.3%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Qwen3.7-Plus Alibaba Group	84.0%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-5.6 Sol OpenAI · None effort	83.7%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Gemini 3.5 Flash-Lite Google DeepMind · High effort	83.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Sonnet 4.5 Anthropic · Unknown effort	83.2%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-5.6 Luna OpenAI · Medium effort	83.2%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Qwen3.5-Flash Alibaba Group	82.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
DeepSeek-V3.1 DeepSeek · Thinking effort	82.7%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Gemma 4 31B Google DeepMind	82.7%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Grok 4 Fast SpaceXAI (xAI)	82.7%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Gemini 2.5 Pro Google DeepMind	82.4%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Qwen3.5-27B Alibaba Group	82.4%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Qwen3-Max Alibaba Group	82.1%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Qwen3.6-Plus Alibaba Group	81.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Opus 4 Anthropic · Unknown effort	81.6%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
DeepSeek-V3.1-Terminus DeepSeek	81.3%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Qwen-Plus Alibaba Group	81.1%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-5 mini OpenAI · High effort	80.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Qwen3.5-Plus Alibaba Group	80.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-5.4 nano OpenAI · Xhigh effort	80.3%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Qwen3-235B-A22B-Thinking-2507 Alibaba Group	80.3%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Opus 4.1 Anthropic · Unknown effort	80.0%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-5.4 mini OpenAI · Xhigh effort	80.0%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-5.6 Luna OpenAI · Low effort	80.0%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Qwen3.5-35B-A3B Alibaba Group	80.0%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Sonnet 5 Anthropic · Medium effort	79.2%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
MiniMax-M3 MiniMax	78.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-5.6 Terra OpenAI · None effort	78.7%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Qwen3-235B-A22B-Instruct-2507 Alibaba Group	78.4%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Qwen3.6-27B Alibaba Group	78.1%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
o4-mini OpenAI · High effort	77.6%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Sonnet 4 Anthropic · Unknown effort	77.1%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Qwen3.6-Flash Alibaba Group	77.1%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Gemini 3.1 Flash-Lite Google DeepMind · High effort	76.8%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Gemini 2.5 Flash Google DeepMind	76.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
gpt-oss-120b OpenAI · High effort	76.3%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Qwen3-235B-A22B Alibaba Group	75.7%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Mistral Medium 3.5 Mistral AI	75.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Gemma 4 26B A4B Google DeepMind	74.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
o1 OpenAI · High effort	74.7%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Qwen3.6-35B-A3B Alibaba Group	73.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Haiku 4.5 Anthropic · Unknown effort	73.6%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Sonnet 5 Anthropic · Low effort	73.6%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Qwen3.5-9B Alibaba Group	71.2%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Mistral Small 4 Mistral AI	70.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Qwen3-30B-A3B-Thinking-2507 Alibaba Group	69.3%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-5.6 Luna OpenAI · None effort	69.3%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-4.1 mini OpenAI	68.8%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
o3-mini OpenAI · High effort	68.8%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-4.1 OpenAI	68.3%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
gpt-oss-20b OpenAI · High effort	68.0%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude 3.5 Sonnet Anthropic	67.8%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude 3.5 Sonnet Anthropic	67.8%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Mistral Medium 3.1 Mistral AI	67.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Qwen3-32B Alibaba Group	67.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Qwen3-30B-A3B-Instruct-2507 Alibaba Group	67.2%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Grok 2 SpaceXAI (xAI)	65.2%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Mistral Large 3 Mistral AI	65.1%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-4o OpenAI	64.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Qwen3-14B Alibaba Group	64.0%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Gemini 2.0 Flash Google DeepMind	63.2%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Qwen2.5-72B-Instruct Alibaba Group	62.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
DeepSeek-V3.2 DeepSeek	62.7%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-4 OpenAI	62.7%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-5 nano OpenAI · High effort	62.7%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Mistral Medium 3 Mistral AI	62.3%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Llama 4 Maverick Meta	61.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude 3 Opus Anthropic	61.6%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-4 Turbo OpenAI	61.6%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Llama 3.1 405B Meta	61.4%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Command A Cohere	61.3%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Magistral Small 1.2 Mistral AI	61.3%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Mistral Large 2.0 Mistral AI	61.2%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Mistral Large 2.1 Mistral AI	60.8%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Qwen3-30B-A3B Alibaba Group	60.3%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Llama 3.1 70B Meta	60.0%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Mistral Small 3.2 Mistral AI	59.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Qwen3-8B Alibaba Group	59.7%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Llama 3.3 70B Meta	59.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Gemini 1.5 Pro Google DeepMind	59.0%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Mistral Small 3.1 Mistral AI	58.6%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Llama 4 Scout Meta	57.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude 3.5 Haiku Anthropic	56.7%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Gemini 1.5 Pro Google DeepMind	56.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Mistral Large 1.0 Mistral AI	55.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Mixtral 8x22B Mistral AI	55.1%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Command R+ (08-2024) Cohere	54.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-4o mini OpenAI	54.4%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Llama 3 70B Meta	54.2%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Gemini 1.5 Flash Google DeepMind	53.8%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude 3 Sonnet Anthropic	53.6%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Mistral Medium 1.0 Mistral AI	53.3%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Mistral Small 1.0 Mistral AI	53.0%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Gemini 2.0 Flash-Lite Google DeepMind	52.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Gemma 3 27B Google DeepMind	52.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-4.1 nano OpenAI	52.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude 2 Anthropic	51.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Ministral 3B Mistral AI	51.7%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Gemini 1.5 Flash Google DeepMind	51.4%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude 2.1 Anthropic	51.0%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Gemma 3 4B Google DeepMind	50.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Llama 3.1 8B Meta	50.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude 3 Haiku Anthropic	50.1%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Gemini 1.5 Flash-8B Google DeepMind	50.0%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Mixtral 8x7B Mistral AI	49.6%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Mistral Small 2.0 Mistral AI	49.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Gemma 3 12B Google DeepMind	48.8%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Mistral NeMo 12B Mistral AI	48.6%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
GPT-3.5 Turbo OpenAI	48.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Gemma 2 27B Google DeepMind	48.0%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Qwen2.5-7B-Instruct Alibaba Group	47.7%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Command R (08-2024) Cohere	46.4%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Gemini 1.0 Pro Google DeepMind	45.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Claude Instant 1.2 Anthropic	45.8%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Ministral 8B Mistral AI	45.7%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Llama 3 8B Meta	43.9%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—
Mistral 7B v0.3 Mistral AI	42.5%	Third party	https://conceptualreasoning.ai/dtbench via Epoch AI	—

Epoch AI, 'Capabilities & benchmarking', epoch.ai/benchmarks, CC BY 4.0.

Sources

Epoch AI