LLM Creative Story-Writing Benchmark V1 (Lech Mazur)

Measures: Short stories built around required story elements, graded by LLM judges on a 0 to 10 rubric; this first version was frozen in August 2025 and replaced by a pairwise-comparison ranking.

Maintainer: Lech Mazur
Introduced: 2025
Status: Superseded
Saturation: Near-ceiling
Typical provenance: Independent
In scoring set: No

Official leaderboard

Leaderboard

Show all results

Independent and third-party results only.

Leaderboard
Model	Mean rubric score (percent of the 10-point scale)	Provenance	Source	Evaluated
Amazon Nova Pro Amazon	60.5%	Third party	lechmazur/writing Github repository via Epoch AI	—
o3-mini OpenAI · Medium effort	61.5%	Third party	lechmazur/writing Github repository via Epoch AI	—
o3-mini OpenAI · High effort	61.7%	Third party	lechmazur/writing Github repository via Epoch AI	—
Llama 4 Maverick Meta	62.0%	Third party	lechmazur/writing Github repository via Epoch AI	—
Phi-4 Microsoft	62.6%	Third party	lechmazur/writing Github repository via Epoch AI	—
Grok 2 SpaceXAI (xAI)	63.6%	Third party	lechmazur/writing Github repository via Epoch AI	—
o1-mini OpenAI · Medium effort	64.9%	Third party	lechmazur/writing Github repository via Epoch AI	—
GPT-4o mini OpenAI	67.2%	Third party	lechmazur/writing Github repository via Epoch AI	—
Mistral Large 2.0 Mistral AI	69.0%	Third party	lechmazur/writing Github repository via Epoch AI	—
o1 OpenAI · Medium effort	70.2%	Third party	lechmazur/writing Github repository via Epoch AI	—
Gemini 2.0 Flash Google DeepMind	71.5%	Third party	lechmazur/writing Github repository via Epoch AI	—
Qwen2.5-Max Alibaba Group	72.9%	Third party	lechmazur/writing Github repository via Epoch AI	—
GLM-4.5 Z.ai (Zhipu AI)	73.4%	Third party	lechmazur/writing Github repository via Epoch AI	—
Claude 3.5 Haiku Anthropic	73.5%	Third party	lechmazur/writing Github repository via Epoch AI	—
Grok 3 Mini SpaceXAI (xAI) · Low effort	73.5%	Third party	lechmazur/writing Github repository via Epoch AI	—
Gemini 2.0 Flash Thinking Google DeepMind	73.8%	Third party	lechmazur/writing Github repository via Epoch AI	—
o4-mini OpenAI · Medium effort	75.0%	Third party	lechmazur/writing Github repository via Epoch AI	—
Qwen3-30B-A3B Alibaba Group	75.3%	Third party	lechmazur/writing Github repository via Epoch AI	—
GPT-4.5 OpenAI	75.6%	Third party	lechmazur/writing Github repository via Epoch AI	—
Grok 3 SpaceXAI (xAI)	76.4%	Third party	lechmazur/writing Github repository via Epoch AI	—
Grok 4 SpaceXAI (xAI)	76.9%	Third party	lechmazur/writing Github repository via Epoch AI	—
gpt-oss-120b OpenAI	77.1%	Third party	lechmazur/writing Github repository via Epoch AI	—
Mistral Medium 3 Mistral AI	77.3%	Third party	lechmazur/writing Github repository via Epoch AI	—
Claude 3.7 Sonnet Anthropic	79.4%	Third party	lechmazur/writing Github repository via Epoch AI	—
Gemma 3 27B Google DeepMind	79.9%	Third party	lechmazur/writing Github repository via Epoch AI	—
Claude 3.5 Sonnet Anthropic	80.3%	Third party	lechmazur/writing Github repository via Epoch AI	—
Gemini 2.5 Pro Google DeepMind	80.5%	Third party	lechmazur/writing Github repository via Epoch AI	—
Claude Sonnet 4 Anthropic	80.9%	Third party	lechmazur/writing Github repository via Epoch AI	—
Gemini 2.5 Pro Google DeepMind	80.9%	Third party	lechmazur/writing Github repository via Epoch AI	—
Claude 3.7 Sonnet Anthropic · 16K effort	81.1%	Third party	lechmazur/writing Github repository via Epoch AI	—
Claude Sonnet 4 Anthropic · 16K effort	81.4%	Third party	lechmazur/writing Github repository via Epoch AI	—
GPT-4o OpenAI	81.8%	Third party	lechmazur/writing Github repository via Epoch AI	—
DeepSeek-R1 DeepSeek	81.9%	Third party	lechmazur/writing Github repository via Epoch AI	—
Qwen3-235B-A22B-Thinking-2507 Alibaba Group	82.4%	Third party	lechmazur/writing Github repository via Epoch AI	—
DeepSeek-R1 DeepSeek	83.0%	Third party	lechmazur/writing Github repository via Epoch AI	—
Qwen3-235B-A22B Alibaba Group	83.0%	Third party	lechmazur/writing Github repository via Epoch AI	—
Claude Opus 4 Anthropic	83.1%	Third party	lechmazur/writing Github repository via Epoch AI	—
GPT-5 mini OpenAI · Medium effort	83.1%	Third party	lechmazur/writing Github repository via Epoch AI	—
Claude Opus 4 Anthropic · 16K effort	83.6%	Third party	lechmazur/writing Github repository via Epoch AI	—
Gemini 2.5 Pro Google DeepMind	83.8%	Third party	lechmazur/writing Github repository via Epoch AI	—
o3 OpenAI · Medium effort	83.9%	Third party	lechmazur/writing Github repository via Epoch AI	—
o3-pro OpenAI · Medium effort	84.4%	Third party	lechmazur/writing Github repository via Epoch AI	—
Claude Opus 4.1 Anthropic · 16K effort	84.5%	Third party	lechmazur/writing Github repository via Epoch AI	—
Claude Opus 4.1 Anthropic	84.7%	Third party	lechmazur/writing Github repository via Epoch AI	—
GPT-5 OpenAI · Medium effort	86.0%	Third party	lechmazur/writing Github repository via Epoch AI	—

Epoch AI, 'Capabilities & benchmarking', epoch.ai/benchmarks, CC BY 4.0.

Sources

Epoch AI