BALROG

Measures: Agentic play in six game environments of widely varying difficulty (BabyAI, Crafter, TextWorld, Baba Is AI, MiniHack and NetHack); scored on average progress through each game.

Maintainer: BALROG team (Paglieri et al.)
Introduced: 2024
Status: Active
Saturation: Headroom
Typical provenance: Independent
In scoring set: Yes

Paper Official leaderboard

Leaderboard

Show all results

Independent and third-party results only.

Leaderboard
Model	Average progress	Provenance	Source	Evaluated
Qwen2-VL-7B-Instruct Alibaba Group	3.7%	Third party	https://balrogai.com/ via Epoch AI	—
Llama 3.2 1B Meta	6.6%	Third party	https://balrogai.com/ via Epoch AI	—
Qwen2.5-7B-Instruct Alibaba Group	7.8%	Third party	https://balrogai.com/ via Epoch AI	—
Llama 3.2 3B Meta	10.1%	Third party	https://balrogai.com/ via Epoch AI	—
Phi-4 Microsoft	11.6%	Third party	https://balrogai.com/ via Epoch AI	—
Qwen2-VL-72B-Instruct Alibaba Group	12.8%	Third party	https://balrogai.com/ via Epoch AI	—
Gemini 1.5 Flash Google DeepMind	14.6%	Third party	https://balrogai.com/ via Epoch AI	—
Llama 3.1 8B Meta	15.1%	Third party	https://balrogai.com/ via Epoch AI	—
Qwen2.5-72B-Instruct Alibaba Group	16.2%	Third party	https://balrogai.com/ via Epoch AI	—
Llama 3.2 11B Vision Meta	16.8%	Third party	https://balrogai.com/ via Epoch AI	—
GPT-4o mini OpenAI	17.4%	Third party	https://balrogai.com/ via Epoch AI	—
Mistral NeMo 12B Mistral AI	17.6%	Third party	https://balrogai.com/ via Epoch AI	—
Claude 3.5 Haiku Anthropic	19.3%	Third party	https://balrogai.com/ via Epoch AI	—
DeepSeek-R1-Distill-Qwen-32B DeepSeek	19.5%	Third party	https://balrogai.com/ via Epoch AI	—
Gemini 1.5 Pro Google DeepMind	21.0%	Third party	https://balrogai.com/ via Epoch AI	—
Llama 3.3 70B Meta	23.0%	Third party	https://balrogai.com/ via Epoch AI	—
Llama 3.2 90B Vision Meta	27.3%	Third party	https://balrogai.com/ via Epoch AI	—
Llama 3.1 70B Meta	27.9%	Third party	https://balrogai.com/ via Epoch AI	—
Reka Flash 3 Reka AI	29.2%	Third party	https://balrogai.com/ via Epoch AI	—
Grok 3 SpaceXAI (xAI)	29.5%	Third party	https://balrogai.com/ via Epoch AI	—
Claude Haiku 4.5 Anthropic	31.2%	Third party	Balrog Leaderboard via Epoch AI	—
Claude Haiku 4.5 Anthropic · 1K effort	31.2%	Third party	https://balrogai.com/ via Epoch AI	—
GPT-4o OpenAI	32.3%	Third party	https://balrogai.com/ via Epoch AI	—
Claude 3.5 Sonnet Anthropic	32.6%	Third party	https://balrogai.com/ via Epoch AI	—
GPT-5 OpenAI · Minimal effort	32.8%	Third party	https://balrogai.com/ via Epoch AI	—
Gemini 2.5 Flash Google DeepMind	33.5%	Third party	https://balrogai.com/ via Epoch AI	—
DeepSeek-R1 DeepSeek	34.9%	Third party	https://balrogai.com/ via Epoch AI	—
Claude Opus 4.5 Anthropic · 64K effort	43.0%	Third party	Balrog Leaderboard via Epoch AI	—
Claude Opus 4.5 Anthropic · Unknown effort	43.0%	Third party	https://balrogai.com/ via Epoch AI	—
Gemini 2.5 Pro Google DeepMind	43.3%	Third party	https://balrogai.com/ via Epoch AI	—
Claude Opus 4.5 Anthropic	43.5%	Third party	Balrog Leaderboard via Epoch AI	—
Grok 4 SpaceXAI (xAI)	43.6%	Third party	https://balrogai.com/ via Epoch AI	—
GPT-5.6 Luna OpenAI · Max effort	45.6%	Third party	https://balrogai.com/ via Epoch AI	—
Gemini 3 Flash Google DeepMind	48.1%	Third party	https://balrogai.com/ via Epoch AI	—
GPT-5.6 Terra OpenAI · Max effort	53.2%	Third party	https://balrogai.com/ via Epoch AI	—
Gemini 3.1 Pro Google DeepMind	56.9%	Third party	Balrog Leaderboard via Epoch AI	—
Gemini 3.1 Pro Google DeepMind	57.0%	Third party	Balrog Leaderboard via Epoch AI	—
Gemini 3 Pro Google DeepMind	58.1%	Third party	https://balrogai.com/ via Epoch AI	—
GPT-5.6 Sol OpenAI · Max effort	60.0%	Third party	https://balrogai.com/ via Epoch AI	—
Claude Opus 5 Anthropic · Max effort	63.4%	Third party	https://balrogai.com/ via Epoch AI	—
GPT-6 Astra OpenAI · Max effort	68.3%	Third party	https://balrogai.com/ via Epoch AI	—

Epoch AI, 'Capabilities & benchmarking', epoch.ai/benchmarks, CC BY 4.0.

Sources

Epoch AI