OTIS Mock AIME 2024-2025

Measures: 45 competition problems from three OTIS Mock AIME exams with integer answers from 0 to 999, harder than MATH Level 5 and easier than FrontierMath.

Maintainer: Epoch AI (problems by OTIS students)
Introduced: 2024
Status: Active
Saturation: Saturated
Typical provenance: Independent
In scoring set: No

Official leaderboard

Leaderboard

Show all results

Independent and third-party results only.

Leaderboard
Model	Accuracy	Provenance	Source	Evaluated
Claude Fable 5 Anthropic · High effort	100.0%	Independent	Epoch AI	6 Aug 2026
Claude Fable 5.1 Anthropic · Max effort	100.0%	Independent	Epoch AI	1 Sep 2026
Claude Opus 5.5 Anthropic · Max effort	100.0%	Independent	Epoch AI	22 Sep 2026
GPT-5.6 Sol OpenAI · Max effort	100.0%	Independent	Epoch AI	9 Jul 2026
GPT-6 Astra OpenAI · Max effort	100.0%	Independent	Epoch AI	30 Aug 2026
Qwen3.8-Max Alibaba Group · Xhigh effort	100.0%	Independent	Epoch AI	2 Sep 2026
Claude Fable 5 Anthropic · Max effort	99.7%	Independent	Epoch AI	10 Jun 2026
GPT-5.6 Terra OpenAI · Max effort	99.7%	Independent	Epoch AI	9 Jul 2026
Qwen3.8-Max Alibaba Group · Xhigh effort	99.4%	Independent	Epoch AI	4 Aug 2026
Grok 4.6 SpaceXAI (xAI) · Xhigh effort	99.2%	Independent	Epoch AI	14 Aug 2026
Muse Spark 1.3 Meta · Xhigh effort	99.2%	Independent	Epoch AI	17 Sep 2026
Claude Opus 5 Anthropic · Max effort	98.9%	Independent	Epoch AI	24 Jul 2026
Gemini 3.8 Flash Google DeepMind · High effort	98.9%	Independent	Epoch AI	2 Sep 2026
DeepSeek-V4-Pro DeepSeek · Max effort	98.6%	Independent	Epoch AI	18 Aug 2026
Claude Opus 4.8 Anthropic · Max effort	98.3%	Independent	Epoch AI	7 Jun 2026
GPT-5.6 Luna OpenAI · Max effort	98.3%	Independent	Epoch AI	9 Jul 2026
Claude Opus 4.7 Anthropic · Xhigh effort	97.8%	Independent	Epoch AI	17 Apr 2026
Claude Fable 5 Anthropic · Low effort	97.8%	Independent	Epoch AI	6 Aug 2026
Claude Opus 4.8 Anthropic · Low effort	97.8%	Independent	Epoch AI	6 Aug 2026
Claude Opus 5 Anthropic	97.8%	Independent	Epoch AI	6 Aug 2026
GPT-5.4 OpenAI · High effort	97.8%	Independent	Epoch AI	15 Jul 2026
Grok 4.5 SpaceXAI (xAI) · High effort	97.8%	Independent	Epoch AI	8 Jul 2026
Grok 4.6 SpaceXAI (xAI) · High effort	97.8%	Independent	Epoch AI	12 Aug 2026
Gemini 3.7 Flash Google DeepMind · High effort	97.2%	Independent	Epoch AI	14 Aug 2026
Kimi K3 Moonshot AI · Max effort	97.2%	Independent	Epoch AI	16 Jul 2026
DeepSeek-V4-Pro DeepSeek · Max effort	96.7%	Independent	Epoch AI	17 Jun 2026
GPT-5.2 OpenAI · High effort	96.1%	Independent	Epoch AI	11 Dec 2025
Kimi K2.6 Moonshot AI	96.1%	Independent	Epoch AI	2 May 2026
GPT-5.2 OpenAI · Xhigh effort	96.1%	Independent	Epoch AI	13 Dec 2025
Gemini 3.1 Pro Google DeepMind	95.6%	Independent	Epoch AI	20 Feb 2026
DeepSeek-V4-Pro DeepSeek · High effort	95.6%	Independent	Epoch AI	6 Aug 2026
Gemini 3 Flash Google DeepMind · High effort	95.6%	Independent	Epoch AI	6 Aug 2026
Gemini 3.1 Pro Google DeepMind · High effort	95.6%	Independent	Epoch AI	6 Aug 2026
Gemini 3.5 Flash Google DeepMind · High effort	95.6%	Independent	Epoch AI	25 May 2026
GPT-5.4 OpenAI · Medium effort	95.6%	Independent	Epoch AI	15 Jul 2026
GPT-5.6 Sol OpenAI · Low effort	95.6%	Independent	Epoch AI	7 Aug 2026
Kimi K2.7 Code Moonshot AI	95.6%	Independent	Epoch AI	7 Aug 2026
Qwen3.7-Max Alibaba Group	95.6%	Independent	Epoch AI	7 Aug 2026
GPT-5.4 OpenAI · Xhigh effort	95.3%	Independent	Epoch AI	6 Mar 2026
Claude Sonnet 5 Anthropic · Xhigh effort	94.7%	Independent	Epoch AI	1 Jul 2026
Claude Opus 4.6 Anthropic · 64K effort	94.4%	Independent	Epoch AI	6 Feb 2026
DeepSeek-V4-Flash DeepSeek · Max effort	94.4%	Independent	Epoch AI	2 Aug 2026
Gemini 3.6 Flash Google DeepMind · High effort	94.2%	Independent	Epoch AI	2 Aug 2026
GLM-5.3-Flash Z.ai (Zhipu AI) · Max effort	93.9%	Independent	Epoch AI	26 Aug 2026
GPT-5.2 OpenAI · Medium effort	93.9%	Independent	Epoch AI	11 Dec 2025
Claude Opus 5 Anthropic · Low effort	93.3%	Independent	Epoch AI	6 Aug 2026
GLM-5.1 Z.ai (Zhipu AI)	93.3%	Independent	Epoch AI	10 Aug 2026
Grok 4.3 SpaceXAI (xAI) · High effort	93.3%	Independent	Epoch AI	17 Jun 2026
Kimi K3 Moonshot AI · High effort	93.3%	Independent	Epoch AI	7 Aug 2026
Qwen3.6-Plus Alibaba Group	93.3%	Independent	Epoch AI	7 Aug 2026
Qwen3.7-Plus Alibaba Group	93.3%	Independent	Epoch AI	7 Aug 2026
Claude Opus 4.6 Anthropic · 32K effort	93.1%	Independent	Epoch AI	6 Feb 2026
Gemini 3 Flash Google DeepMind	92.8%	Independent	Epoch AI	17 Dec 2025
Grok 4.20 SpaceXAI (xAI)	92.2%	Independent	Epoch AI	13 Jul 2026
Gemini 3 Pro Google DeepMind	91.4%	Independent	Epoch AI	19 Nov 2025
GPT-5 OpenAI · High effort	91.4%	Independent	Epoch AI	29 Oct 2025
Claude Opus 4.6 Anthropic · Max effort	91.1%	Independent	Epoch AI	6 Aug 2026
GLM-5.3 Z.ai (Zhipu AI) · Max effort	91.1%	Independent	Epoch AI	24 Aug 2026
Qwen3.6-27B Alibaba Group	91.1%	Independent	Epoch AI	7 Aug 2026
Qwen3.6-Max-Preview Alibaba Group	91.1%	Independent	Epoch AI	7 Aug 2026
Inkling Small Thinking Machines Lab · Xhigh effort	90.0%	Independent	Epoch AI	14 Aug 2026
Gemini 3.5 Flash Google DeepMind · Low effort	88.9%	Independent	Epoch AI	6 Aug 2026
GPT-5.4 mini OpenAI · Xhigh effort	88.9%	Independent	Epoch AI	7 Aug 2026
GPT-5.6 Terra OpenAI · Low effort	88.9%	Independent	Epoch AI	7 Aug 2026
gpt-oss-120b OpenAI · High effort	88.9%	Independent	Epoch AI	11 Dec 2025
Inkling Thinking Machines Lab · Xhigh effort	88.9%	Independent	Epoch AI	5 Aug 2026
Qwen3.5-397B-A17B Alibaba Group	88.9%	Independent	Epoch AI	7 Aug 2026
GPT-5.1 OpenAI · High effort	88.6%	Independent	Epoch AI	13 Nov 2025
DeepSeek-V3.2 DeepSeek	87.8%	Independent	Epoch AI	16 Dec 2025
GPT-5.4 nano OpenAI · High effort	87.8%	Independent	Epoch AI	14 Apr 2026
GPT-5 OpenAI · Medium effort	87.2%	Independent	Epoch AI	7 Aug 2025
GPT-5.4 mini OpenAI · High effort	87.2%	Independent	Epoch AI	15 Apr 2026
Claude Opus 4.7 Anthropic · Max effort	86.7%	Independent	Epoch AI	6 Aug 2026
GPT-5 mini OpenAI · High effort	86.7%	Independent	Epoch AI	30 Oct 2025
Qwen3-235B-A22B-Thinking-2507 Alibaba Group	86.7%	Independent	Epoch AI	10 Dec 2025
Qwen3.5-Plus Alibaba Group	86.7%	Independent	Epoch AI	7 Aug 2026
Qwen3.6-35B-A3B Alibaba Group	86.7%	Independent	Epoch AI	7 Aug 2026
Qwen3.7-Flash Alibaba Group	86.7%	Independent	Epoch AI	7 Aug 2026
GLM-5.2 Z.ai (Zhipu AI) · Max effort	86.4%	Independent	Epoch AI	25 Jun 2026
Claude Opus 4.5 Anthropic · 32K effort	86.1%	Independent	Epoch AI	24 Nov 2025
Claude Sonnet 4.6 Anthropic · 32K effort	85.8%	Independent	Epoch AI	20 Feb 2026
GPT-5.1 OpenAI · Medium effort	85.6%	Independent	Epoch AI	17 Nov 2025
Gemini 2.5 Pro Google DeepMind	84.7%	Independent	Epoch AI	16 Nov 2025
Claude Opus 4.8 Anthropic · None effort	84.4%	Independent	Epoch AI	6 Aug 2026
GPT-5.4 OpenAI · Low effort	84.4%	Independent	Epoch AI	15 Jul 2026
GPT-5.5 OpenAI · Low effort	84.4%	Independent	Epoch AI	7 Aug 2026
o3 OpenAI · Medium effort	84.4%	Independent	Epoch AI	7 Aug 2026
Qwen3.5-Flash Alibaba Group	84.4%	Independent	Epoch AI	7 Aug 2026
Qwen3.6-Flash Alibaba Group	84.4%	Independent	Epoch AI	7 Aug 2026
Grok 4 SpaceXAI (xAI)	84.0%	Independent	Epoch AI	—
o3 OpenAI · High effort	83.9%	Independent	Epoch AI	16 Apr 2025
GLM-4.7 Z.ai (Zhipu AI)	83.3%	Independent	Epoch AI	29 Jan 2026
Kimi K2 Thinking Moonshot AI	83.1%	Independent	Epoch AI	13 Nov 2025
Claude Sonnet 4.6 Anthropic · Medium effort	82.2%	Independent	Epoch AI	13 Jul 2026
Gemini 3.6 Flash Google DeepMind · Low effort	82.2%	Independent	Epoch AI	7 Aug 2026
Gemma 4 26B A4B Google DeepMind · Minimal effort	82.2%	Independent	Epoch AI	28 Aug 2026
Qwen3.5-397B-A17B Alibaba Group · None effort	82.2%	Independent	Epoch AI	7 Aug 2026
Claude Opus 4.5 Anthropic · 16K effort	81.7%	Independent	Epoch AI	24 Nov 2025
o4-mini OpenAI · High effort	81.7%	Independent	Epoch AI	16 Apr 2025
GPT-5 nano OpenAI · High effort	81.1%	Independent	Epoch AI	31 Oct 2025
Claude Sonnet 5 Anthropic · Max effort	80.0%	Independent	Epoch AI	6 Aug 2026
Gemini 3.1 Flash-Lite Google DeepMind · High effort	80.0%	Independent	Epoch AI	6 Aug 2026
Gemini 3.5 Flash Google DeepMind · Minimal effort	80.0%	Independent	Epoch AI	15 Jul 2026
Gemini 3.6 Flash Google DeepMind · Minimal effort	80.0%	Independent	Epoch AI	7 Aug 2026
GLM-5 Z.ai (Zhipu AI)	80.0%	Independent	Epoch AI	12 Feb 2026
Qwen3.7-Plus Alibaba Group · None effort	80.0%	Independent	Epoch AI	7 Aug 2026
GPT-5.2 OpenAI · Low effort	78.9%	Independent	Epoch AI	11 Dec 2025
GPT-5 mini OpenAI · Medium effort	78.3%	Independent	Epoch AI	7 Aug 2025
Claude Sonnet 4.5 Anthropic · 59K effort	77.8%	Independent	Epoch AI	28 Oct 2025
Claude Sonnet 4.5 Anthropic · 32K effort	77.8%	Independent	Epoch AI	21 Oct 2025
Grok 3 Mini SpaceXAI (xAI) · High effort	77.8%	Independent	Epoch AI	10 Apr 2025
Qwen3.7-Flash Alibaba Group · None effort	77.8%	Independent	Epoch AI	7 Aug 2026
o3-mini OpenAI · High effort	76.9%	Independent	Epoch AI	27 Feb 2025
Claude Sonnet 4.6 Anthropic · High effort	75.6%	Independent	Epoch AI	13 Jul 2026
GLM-5.2 Z.ai (Zhipu AI) · Low effort	75.6%	Independent	Epoch AI	10 Aug 2026
GPT-5 nano OpenAI · Medium effort	74.2%	Independent	Epoch AI	7 Aug 2025
Gemma 4 31B Google DeepMind · Minimal effort	73.3%	Independent	Epoch AI	6 Aug 2026
o1 OpenAI · High effort	73.3%	Independent	Epoch AI	15 Jul 2026
o1 OpenAI · Medium effort	73.3%	Independent	Epoch AI	27 Feb 2025
o4-mini OpenAI · Medium effort	73.3%	Independent	Epoch AI	7 Aug 2026
Qwen3-Max Alibaba Group	73.3%	Independent	Epoch AI	6 Oct 2025
Gemini 2.5 Flash Google DeepMind	73.1%	Independent	Epoch AI	22 Apr 2025
Claude Sonnet 4 Anthropic · 32K effort	71.1%	Independent	Epoch AI	22 May 2025
Claude Sonnet 4.5 Anthropic · 16K effort	71.1%	Independent	Epoch AI	28 Oct 2025
Claude Sonnet 4.6 Anthropic · Max effort	71.1%	Independent	Epoch AI	6 Aug 2026
Gemini 3.5 Flash-Lite Google DeepMind · High effort	71.1%	Independent	Epoch AI	6 Aug 2026
MiniMax-M3 MiniMax	71.1%	Independent	Epoch AI	10 Aug 2026
Gemini 2.5 Flash Google DeepMind	70.8%	Independent	Epoch AI	20 May 2025
Qwen3-30B-A3B-Thinking-2507 Alibaba Group	70.3%	Independent	Epoch AI	30 Aug 2026
Qwen3.5-35B-A3B Alibaba Group · None effort	70.0%	Independent	Epoch AI	30 Aug 2026
Claude Opus 4.1 Anthropic · 27K effort	68.9%	Independent	Epoch AI	5 Aug 2025
Claude Sonnet 4 Anthropic · 59K effort	68.9%	Independent	Epoch AI	23 May 2025
GPT-5.4 nano OpenAI · Low effort	68.9%	Independent	Epoch AI	7 Aug 2026
GPT-5.6 Sol OpenAI · None effort	68.9%	Independent	Epoch AI	7 Aug 2026
Kimi K3 Moonshot AI · Low effort	68.9%	Independent	Epoch AI	7 Aug 2026
Qwen3.6-35B-A3B Alibaba Group · None effort	68.9%	Independent	Epoch AI	7 Aug 2026
Seed-OSS-36B-Instruct ByteDance	67.5%	Independent	Epoch AI	28 Aug 2026
Qwen3-32B Alibaba Group	66.9%	Independent	Epoch AI	30 Aug 2026
Claude Haiku 4.5 Anthropic · 32K effort	66.7%	Independent	Epoch AI	22 Oct 2025
GPT-5.6 Luna OpenAI · Low effort	66.7%	Independent	Epoch AI	7 Aug 2026
Qwen3.6-27B Alibaba Group · None effort	66.7%	Independent	Epoch AI	7 Aug 2026
DeepSeek-R1 DeepSeek	66.4%	Independent	Epoch AI	29 May 2025
Qwen3-14B Alibaba Group	66.4%	Independent	Epoch AI	28 Aug 2026
gpt-oss-20b OpenAI · Medium effort	65.3%	Independent	Epoch AI	27 Aug 2026
Claude Opus 4 Anthropic · 27K effort	64.4%	Independent	Epoch AI	28 May 2025
Claude Opus 4.1 Anthropic · 16K effort	64.4%	Independent	Epoch AI	5 Aug 2025
GPT-5.1 OpenAI · Low effort	63.9%	Independent	Epoch AI	25 Nov 2025
o3-mini OpenAI · Medium effort	63.9%	Independent	Epoch AI	25 Feb 2025
Qwen3-30B-A3B Alibaba Group	62.8%	Independent	Epoch AI	28 Aug 2026
GPT-5.2 OpenAI · None effort	62.2%	Independent	Epoch AI	13 Jul 2026
Grok 3 Mini SpaceXAI (xAI) · Low effort	62.2%	Independent	Epoch AI	10 Apr 2025
Qwen3-30B-A3B-Instruct-2507 Alibaba Group	62.2%	Independent	Epoch AI	30 Aug 2026
Qwen3.5-9B Alibaba Group · None effort	61.7%	Independent	Epoch AI	27 Aug 2026
Claude Opus 4 Anthropic · 16K effort	60.0%	Independent	Epoch AI	22 May 2025
Gemini 3.5 Flash-Lite Google DeepMind · Low effort	60.0%	Independent	Epoch AI	6 Aug 2026
o3 OpenAI · Low effort	60.0%	Independent	Epoch AI	15 Jul 2026
QwQ-32B Alibaba Group	59.2%	Independent	Epoch AI	28 Aug 2026
GLM-4.7-Flash Z.ai (Zhipu AI)	58.3%	Independent	Epoch AI	30 Aug 2026
Claude 3.7 Sonnet Anthropic · 64K effort	57.8%	Independent	Epoch AI	13 Mar 2025
Gemini 2.0 Flash Thinking Google DeepMind	57.8%	Independent	Epoch AI	7 Mar 2025
GPT-5.4 OpenAI · None effort	57.8%	Independent	Epoch AI	15 Jul 2026
GPT-5.5 OpenAI · None effort	57.8%	Independent	Epoch AI	7 Aug 2026
o4-mini OpenAI · Low effort	57.8%	Independent	Epoch AI	13 Jul 2026
Qwen3-8B Alibaba Group	56.1%	Independent	Epoch AI	27 Aug 2026
Qwen3.5-4B Alibaba Group · None effort	55.8%	Independent	Epoch AI	28 Aug 2026
DeepSeek-R1-Distill-Qwen-32B DeepSeek	55.6%	Independent	Epoch AI	28 Aug 2026
GPT-5 mini OpenAI · Minimal effort	55.6%	Independent	Epoch AI	7 Aug 2026
Grok 3 SpaceXAI (xAI)	55.6%	Independent	Epoch AI	10 Apr 2025
Qwen3.5-35B-A3B Alibaba Group	54.4%	Independent	Epoch AI	30 Aug 2026
Claude 3.7 Sonnet Anthropic · 32K effort	53.3%	Independent	Epoch AI	12 Mar 2025
Claude Sonnet 4 Anthropic · 16K effort	53.3%	Independent	Epoch AI	22 May 2025
DeepSeek-R1 DeepSeek	53.3%	Independent	Epoch AI	26 Feb 2025
GPT-5.6 Terra OpenAI · None effort	53.3%	Independent	Epoch AI	7 Aug 2026
o1 OpenAI · Low effort	53.3%	Independent	Epoch AI	15 Jul 2026
Qwen3-4B-Instruct-2507 Alibaba Group	52.2%	Independent	Epoch AI	28 Aug 2026
DeepSeek-R1-Distill-Llama-70B DeepSeek	51.4%	Independent	Epoch AI	7 Mar 2025
Gemini 3.5 Flash-Lite Google DeepMind · Minimal effort	51.1%	Independent	Epoch AI	6 Aug 2026
gpt-oss-20b OpenAI · High effort	50.8%	Independent	Epoch AI	27 Aug 2026
DeepSeek-R1-Distill-Qwen-14B DeepSeek	50.6%	Independent	Epoch AI	28 Aug 2026
DeepSeek-V3.2 DeepSeek	48.9%	Independent	Epoch AI	16 Jul 2026
Claude Opus 4.5 Anthropic	48.1%	Independent	Epoch AI	24 Nov 2025
o1-mini OpenAI · High effort	46.9%	Independent	Epoch AI	6 Mar 2025
Claude 3.7 Sonnet Anthropic · 16K effort	46.7%	Independent	Epoch AI	26 Feb 2025
DeepSeek-V4-Pro DeepSeek · None effort	46.7%	Independent	Epoch AI	6 Aug 2026
GPT-5 OpenAI · Minimal effort	46.7%	Independent	Epoch AI	20 Jul 2026
GPT-5 nano OpenAI · Low effort	46.7%	Independent	Epoch AI	13 Jul 2026
GPT-5.4 nano OpenAI · None effort	46.7%	Independent	Epoch AI	7 Aug 2026
GPT-4.1 mini OpenAI	44.7%	Independent	Epoch AI	14 Apr 2025
o1-mini OpenAI · Medium effort	44.7%	Independent	Epoch AI	6 Mar 2025
Gemini 3.1 Flash-Lite Google DeepMind · Low effort	44.4%	Independent	Epoch AI	6 Aug 2026
o3-mini OpenAI · Low effort	44.4%	Independent	Epoch AI	20 Jul 2026
DeepSeek-R1-0528-Qwen3-8B DeepSeek	43.9%	Independent	Epoch AI	28 Aug 2026
Qwen3.5-9B Alibaba Group	42.8%	Independent	Epoch AI	27 Aug 2026
Claude Opus 4 Anthropic	42.2%	Independent	Epoch AI	22 May 2025
gpt-oss-20b OpenAI · Low effort	40.3%	Independent	Epoch AI	27 Aug 2026
Claude Opus 4.1 Anthropic	40.0%	Independent	Epoch AI	5 Aug 2025
GPT-5.6 Luna OpenAI · None effort	40.0%	Independent	Epoch AI	7 Aug 2026
GPT-4.1 OpenAI	38.3%	Independent	Epoch AI	14 Apr 2025
Gemini 3.1 Flash-Lite Google DeepMind · Minimal effort	37.8%	Independent	Epoch AI	6 Aug 2026
GPT-4.5 OpenAI	37.8%	Independent	Epoch AI	28 Feb 2025
GPT-5.1 OpenAI · None effort	37.8%	Independent	Epoch AI	7 Aug 2026
Claude Haiku 4.5 Anthropic	35.8%	Independent	Epoch AI	16 Oct 2025
Claude Sonnet 4.5 Anthropic	35.6%	Independent	Epoch AI	29 Sep 2025
GPT-5 nano OpenAI · Minimal effort	35.6%	Independent	Epoch AI	13 Jul 2026
Mistral Medium 3 Mistral AI	32.2%	Independent	Epoch AI	7 May 2025
Gemini 2.0 Flash Google DeepMind	31.1%	Independent	Epoch AI	25 Feb 2025
o1-preview OpenAI	31.1%	Independent	Epoch AI	7 Mar 2025
Magistral Small 1.0 Mistral AI	30.0%	Independent	Epoch AI	6 Aug 2026
Claude Sonnet 4 Anthropic	28.9%	Independent	Epoch AI	22 May 2025
GLM-5.2 Z.ai (Zhipu AI) · None effort	28.9%	Independent	Epoch AI	10 Aug 2026
GPT-4.1 nano OpenAI	28.9%	Independent	Epoch AI	14 Apr 2025
Magistral Small 1.2 Mistral AI	28.1%	Independent	Epoch AI	30 Aug 2026
GPT-5.4 mini OpenAI · None effort	26.7%	Independent	Epoch AI	7 Aug 2026
MiniMax-M3 MiniMax · None effort	26.7%	Independent	Epoch AI	10 Aug 2026
Qwen3-14B Alibaba Group · None effort	25.8%	Independent	Epoch AI	30 Aug 2026
Qwen3-30B-A3B Alibaba Group · None effort	25.6%	Independent	Epoch AI	30 Aug 2026
GLM-4.7-Flash Z.ai (Zhipu AI) · None effort	25.0%	Independent	Epoch AI	28 Aug 2026
Gemini 1.5 Pro Google DeepMind	23.1%	Independent	Epoch AI	25 Feb 2025
Qwen3-32B Alibaba Group · None effort	23.1%	Independent	Epoch AI	30 Aug 2026
Gemma 3 27B Google DeepMind	22.5%	Independent	Epoch AI	27 Aug 2026
Qwen3-8B Alibaba Group · None effort	22.2%	Independent	Epoch AI	27 Aug 2026
Claude 3.7 Sonnet Anthropic	21.9%	Independent	Epoch AI	25 Feb 2025
DeepSeek-R1-Distill-Qwen-1.5B DeepSeek	21.4%	Independent	Epoch AI	28 Aug 2026
Llama 4 Maverick Meta	20.6%	Independent	Epoch AI	8 Apr 2025
Qwen-Plus Alibaba Group	17.8%	Independent	Epoch AI	7 Apr 2025
Gemma 3 12B Google DeepMind	16.7%	Independent	Epoch AI	28 Aug 2026
Gemini 1.5 Flash Google DeepMind	16.3%	Independent	Epoch AI	25 Feb 2025
Qwen2.5-Max Alibaba Group	16.1%	Independent	Epoch AI	1 Apr 2025
DeepSeek-V3 DeepSeek	15.8%	Independent	Epoch AI	25 Feb 2025
Phi-4 Microsoft	13.8%	Independent	Epoch AI	25 Feb 2025
Grok 2 SpaceXAI (xAI)	11.5%	Independent	Epoch AI	25 Feb 2025
Llama 3.1 405B Meta	9.7%	Independent	Epoch AI	25 Feb 2025
Claude 3.5 Sonnet Anthropic	8.5%	Independent	Epoch AI	25 Feb 2025
Mistral Large 2.0 Mistral AI	8.5%	Independent	Epoch AI	25 Feb 2025
Qwen2.5-72B-Instruct Alibaba Group	8.1%	Independent	Epoch AI	25 Feb 2025
Qwen3-1.7B Alibaba Group · None effort	8.1%	Independent	Epoch AI	28 Aug 2026
Llama 4 Scout Meta	7.8%	Independent	Epoch AI	8 Apr 2025
Mistral Large 2.1 Mistral AI	7.8%	Independent	Epoch AI	25 Feb 2025
Gemma 3 4B Google DeepMind	7.5%	Independent	Epoch AI	28 Aug 2026
Qwen2.5-32B-Instruct Alibaba Group	7.4%	Independent	Epoch AI	25 Feb 2025
GPT-4o mini OpenAI	6.9%	Independent	Epoch AI	30 Jul 2025
Gemini 1.5 Pro Google DeepMind	6.8%	Independent	Epoch AI	25 Feb 2025
GPT-4 Turbo OpenAI	6.7%	Independent	Epoch AI	27 Feb 2025
Claude 3.5 Sonnet Anthropic	6.5%	Independent	Epoch AI	25 Feb 2025
GPT-4o OpenAI	6.4%	Independent	Epoch AI	25 Feb 2025
GPT-4o OpenAI	6.3%	Independent	Epoch AI	25 Feb 2025
GPT-4o OpenAI	6.3%	Independent	Epoch AI	25 Feb 2025
Qwen-Turbo Alibaba Group	6.1%	Independent	Epoch AI	7 Apr 2025
Mistral Small 3.1 Mistral AI	5.8%	Independent	Epoch AI	18 Mar 2025
Mistral Small 3.0 Mistral AI	5.3%	Independent	Epoch AI	18 Mar 2025
Llama 3.3 70B Meta	5.1%	Independent	Epoch AI	25 Feb 2025
Claude 3 Opus Anthropic	4.7%	Independent	Epoch AI	25 Feb 2025
Gemini 1.5 Flash-8B Google DeepMind	4.6%	Independent	Epoch AI	25 Feb 2025
Claude 3.5 Haiku Anthropic	4.3%	Independent	Epoch AI	25 Feb 2025
Llama 3 70B Meta	4.3%	Independent	Epoch AI	25 Feb 2025
Gemini 1.5 Flash Google DeepMind	3.9%	Independent	Epoch AI	25 Feb 2025
Llama 3.1 70B Meta	3.6%	Independent	Epoch AI	25 Feb 2025
Granite 4.0 Micro (3B) IBM	2.8%	Independent	Epoch AI	30 Aug 2026
Llama 3.2 90B Vision Meta	2.6%	Independent	Epoch AI	25 Feb 2025
Claude 2 Anthropic	2.5%	Independent	Epoch AI	7 Mar 2025
Claude 3 Sonnet Anthropic	2.5%	Independent	Epoch AI	25 Feb 2025
Hermes 2 Theta Llama 3 70B Nous Research	2.5%	Independent	Epoch AI	1 Apr 2025
Qwen2.5-7B-Instruct Alibaba Group	2.5%	Independent	Epoch AI	30 Aug 2026
GPT-3.5 Turbo OpenAI	2.2%	Independent	Epoch AI	7 Aug 2026
Claude 2.1 Anthropic	1.9%	Independent	Epoch AI	7 Mar 2025
Llama 3 8B Meta	1.9%	Independent	Epoch AI	30 Aug 2026
Mistral Large 1.0 Mistral AI	1.9%	Independent	Epoch AI	25 Feb 2025
Claude 3 Haiku Anthropic	1.8%	Independent	Epoch AI	25 Feb 2025
Llama 3.1 8B Meta	1.7%	Independent	Epoch AI	27 Aug 2026
Gemma 2 27B Google DeepMind	1.4%	Independent	Epoch AI	7 Mar 2025
Gemini 1.0 Pro Google DeepMind	1.1%	Independent	Epoch AI	25 Feb 2025
Gemma 3 1B Google DeepMind	1.1%	Independent	Epoch AI	28 Aug 2026
GPT-4 OpenAI	1.1%	Independent	Epoch AI	23 Oct 2025
DeepSeek LLM 67B Chat DeepSeek	0.8%	Independent	Epoch AI	28 Aug 2026
Granite 4.0 1B (Nano) IBM	0.8%	Independent	Epoch AI	30 Aug 2026
Gemma 2 9B Google DeepMind	0.6%	Independent	Epoch AI	7 Mar 2025
GPT-4 OpenAI	0.6%	Independent	Epoch AI	22 Oct 2025
Llama 3.2 1B Meta	0.6%	Independent	Epoch AI	30 Aug 2026
Granite 4.0 350M (Nano) IBM	0.3%	Independent	Epoch AI	28 Aug 2026
Mistral 7B v0.3 Mistral AI	0.3%	Independent	Epoch AI	30 Aug 2026
Llama 2 70B Meta	0.0%	Independent	Epoch AI	25 Feb 2025

Epoch AI, 'Capabilities & benchmarking', epoch.ai/benchmarks, CC BY 4.0.

Sources

Epoch AI