supermarket

Benchmark

LMCA (Language Model Conceptual Argumentation)

Measures: How well a model rates informal arguments in contested conceptual fields (decision theory, philosophy, AI safety), as the Pearson correlation of its ratings with 2,140 expert ratings.

Maintainer
Redwood Research
Introduced
2026
Status
Active
Saturation
Near-ceiling
Typical provenance
Independent
In scoring set
Yes

PaperOfficial leaderboard

Leaderboard

Show all results

Independent and third-party results only.

Leaderboard
ModelCorrelation with expert ratings (Pearson r)ProvenanceSourceEvaluated
Gemma 3 4B
Google DeepMind
0.0278Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Gemma 3 12B
Google DeepMind
0.0446Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Command R+ (08-2024)
Cohere
0.0501Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Llama 3.1 8B
Meta
0.0537Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-4.1 nano
OpenAI
0.0546Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Ministral 3B
Mistral AI
0.0548Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Qwen2.5-7B-Instruct
Alibaba Group
0.0642Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Gemma 2 27B
Google DeepMind
0.071Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-5 nano
OpenAI · High effort
0.0793Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Qwen3-8B
Alibaba Group
0.0882Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude 3 Haiku
Anthropic
0.0884Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Command R (08-2024)
Cohere
0.0919Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-3.5 Turbo
OpenAI
0.0965Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-4 Turbo
OpenAI
0.0978Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Command A
Cohere
0.1028Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-4o mini
OpenAI
0.1037Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Llama 4 Scout
Meta
0.1196Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Gemma 3 27B
Google DeepMind
0.1228Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Qwen2.5-72B-Instruct
Alibaba Group
0.1342Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
gpt-oss-20b
OpenAI · High effort
0.1451Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Llama 3.1 70B
Meta
0.1485Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
DeepSeek-V3.2
DeepSeek
0.1524Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Qwen3-30B-A3B
Alibaba Group
0.1582Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Llama 4 Maverick
Meta
0.1587Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-4o
OpenAI
0.1659Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Mistral Large 3
Mistral AI
0.167Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude 3 Opus
Anthropic
0.17Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-4
OpenAI
0.1706Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Qwen3-32B
Alibaba Group
0.1728Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Llama 3.3 70B
Meta
0.1752Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Qwen3-14B
Alibaba Group
0.1819Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
o3-mini
OpenAI · High effort
0.1897Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Mistral Medium 3.1
Mistral AI
0.1905Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Qwen3-30B-A3B-Thinking-2507
Alibaba Group
0.1953Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Mistral Small 4
Mistral AI
0.2061Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-4.1 mini
OpenAI
0.2112Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
gpt-oss-120b
OpenAI · High effort
0.2214Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
o1
OpenAI · High effort
0.2227Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Qwen3-30B-A3B-Instruct-2507
Alibaba Group
0.2237Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Qwen3-235B-A22B-Instruct-2507
Alibaba Group
0.2352Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Qwen-Plus
Alibaba Group
0.2396Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
DeepSeek-V3.1
DeepSeek · Thinking effort
0.2429Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Qwen3.5-9B
Alibaba Group
0.2447Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Qwen3-235B-A22B
Alibaba Group
0.25Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-4.1
OpenAI
0.2561Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Mistral Medium 3.5
Mistral AI
0.2606Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
o4-mini
OpenAI · High effort
0.265Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Gemini 2.5 Flash
Google DeepMind
0.2749Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Qwen3-Max
Alibaba Group
0.2827Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
DeepSeek-V3.1-Terminus
DeepSeek
0.2858Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
DeepSeek-V3.2
DeepSeek
0.2882Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Sonnet 4
Anthropic · Unknown effort
0.29Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
DeepSeek-V3.2-Exp
DeepSeek · Thinking effort
0.2911Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Qwen3.5-Flash
Alibaba Group
0.2914Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Qwen3-235B-A22B-Thinking-2507
Alibaba Group
0.2931Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Qwen3.5-35B-A3B
Alibaba Group
0.2947Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
MiMo-V2.5-Pro
Xiaomi
0.295Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Gemma 4 26B A4B
Google DeepMind
0.2968Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Qwen3.6-35B-A3B
Alibaba Group
0.297Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Haiku 4.5
Anthropic · Unknown effort
0.3094Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Qwen3.6-Flash
Alibaba Group
0.3098Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Qwen3.5-122B-A10B
Alibaba Group · None effort
0.3223Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Qwen3.6-Plus
Alibaba Group
0.3309Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
MiniMax-M3
MiniMax
0.337Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Qwen3.5-27B
Alibaba Group
0.3398Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-5 mini
OpenAI · High effort
0.342Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Qwen3.6-27B
Alibaba Group
0.3453Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Gemini 2.5 Pro
Google DeepMind
0.3477Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Gemini 3.1 Flash-Lite
Google DeepMind · High effort
0.3501Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
DeepSeek-V4-Flash
DeepSeek · Max effort
0.3588Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Qwen3.5-Plus
Alibaba Group
0.3636Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-5.4 nano
OpenAI · Xhigh effort
0.3686Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Opus 4.1
Anthropic · Unknown effort
0.3708Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Kimi K2.6
Moonshot AI
0.3726Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Opus 4
Anthropic · Unknown effort
0.3743Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Inkling
Thinking Machines Lab · Xhigh effort
0.3758Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Qwen3.7-Plus
Alibaba Group
0.3758Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Gemini 3.5 Flash-Lite
Google DeepMind · High effort
0.376Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Qwen3.5-397B-A17B
Alibaba Group
0.3789Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Grok 4.3
SpaceXAI (xAI) · High effort
0.3833Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
o3-pro
OpenAI · High effort
0.3848Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Grok 4.20
SpaceXAI (xAI)
0.3866Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Sonnet 4.5
Anthropic · Unknown effort
0.3876Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Gemma 4 31B
Google DeepMind
0.3925Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
o3
OpenAI · High effort
0.397Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-5
OpenAI · High effort
0.3999Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-5.4 mini
OpenAI · Xhigh effort
0.4079Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
DeepSeek-V4-Pro
DeepSeek · Max effort
0.412Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Qwen3.8-27B
Alibaba Group
0.4136Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-5.6 Luna
OpenAI · None effort
0.414Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
DeepSeek-V4-Flash
DeepSeek · Max effort
0.4173Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Qwen3.6-Max-Preview
Alibaba Group
0.4253Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Gemini 3 Flash
Google DeepMind · High effort
0.4313Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-5.6 Luna
OpenAI · Low effort
0.4379Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-5.1
OpenAI · High effort
0.4389Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-5.2
OpenAI · Xhigh effort
0.4391Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Qwen3.7-Max
Alibaba Group
0.4402Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-5.6 Luna
OpenAI · Medium effort
0.4425Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Opus 4.5
Anthropic · High effort
0.4446Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-6 Luna
OpenAI · Max effort
0.4449Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Gemini 3.6 Flash
Google DeepMind · High effort
0.4493Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Grok 4.5
SpaceXAI (xAI) · High effort
0.452Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
DeepSeek-V4-Pro
DeepSeek · Unknown effort
0.4545Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GLM-5.2
Z.ai (Zhipu AI) · Max effort
0.4582Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Qwen3.8-Max
Alibaba Group · Xhigh effort
0.4624Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Sonnet 4.6
Anthropic · Max effort
0.4651Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-5.6 Terra
OpenAI · None effort
0.4651Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
DeepSeek-V4.1-Flash
DeepSeek · Unknown effort
0.4697Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Gemini 3.7 Flash
Google DeepMind · Low effort
0.4709Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Gemini 3.5 Flash
Google DeepMind · High effort
0.4715Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-5.6 Luna
OpenAI · High effort
0.4738Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Muse Spark 1.2
Meta · Xhigh effort
0.4835Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Grok 4.6
SpaceXAI (xAI) · Xhigh effort
0.4845Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-5.6 Luna
OpenAI · Max effort
0.4854Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-5.6 Luna
OpenAI · Xhigh effort
0.4854Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Sonnet 5
Anthropic · Xhigh effort
0.4881Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Gemini 3.7 Flash
Google DeepMind · Medium effort
0.4922Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Sonnet 5
Anthropic · Max effort
0.493Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-5.6 Terra
OpenAI · Low effort
0.493Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-5.6 Terra
OpenAI · Medium effort
0.4964Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Sonnet 5
Anthropic · Low effort
0.4965Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Sonnet 5
Anthropic · Medium effort
0.4971Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Muse Spark 1.1
Meta · High effort
0.4991Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Sonnet 5
Anthropic · High effort
0.5002Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Gemini 3.7 Flash
Google DeepMind · High effort
0.5038Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Gemini 3.8 Flash
Google DeepMind · Low effort
0.5116Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Gemini 3.1 Pro
Google DeepMind · Low effort
0.5118Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-5.6 Terra
OpenAI · High effort
0.5137Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Gemini 3.8 Flash
Google DeepMind · Unknown effort
0.5155Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-5.4
OpenAI · Xhigh effort
0.5197Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Opus 4.7
Anthropic · Max effort
0.522Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-5.6 Terra
OpenAI · Xhigh effort
0.5223Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Kimi K3
Moonshot AI · Max effort
0.5274Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-5.6 Sol
OpenAI · None effort
0.5275Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Gemini 3.1 Pro
Google DeepMind · Medium effort
0.528Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Gemini 3.8 Flash
Google DeepMind · Medium effort
0.529Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Gemini 3.1 Pro
Google DeepMind · High effort
0.5382Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Muse Spark 1.3
Meta · Unknown effort
0.5387Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-5.5 Pro
OpenAI · Xhigh effort
0.5394Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-5.5
OpenAI · Xhigh effort
0.5427Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-5.6 Terra
OpenAI · Max effort
0.5495Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GLM-5.3
Z.ai (Zhipu AI) · Unknown effort
0.5551Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Opus 4.6
Anthropic · Max effort
0.5576Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-5.6 Sol
OpenAI · Low effort
0.5589Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-5.6 Sol
OpenAI · Medium effort
0.564Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-5.6 Sol
OpenAI · High effort
0.5694Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Opus 4.8
Anthropic · Max effort
0.5751Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-5.6 Sol
OpenAI · Max effort
0.5843Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-5.6 Sol
OpenAI · Xhigh effort
0.5849Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-6 Sol
OpenAI · Max effort
0.5908Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-5.6 Sol
OpenAI · Promax effort
0.5921Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Fable 5
Anthropic · Low effort
0.5959Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Fable 5
Anthropic · Medium effort
0.6032Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Fable 5
Anthropic · Max effort
0.6034Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Fable 5
Anthropic · Xhigh effort
0.6102Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Fable 5
Anthropic · High effort
0.6115Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-6 Astra
OpenAI · Low effort
0.6219Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Opus 5
Anthropic · Low effort
0.6258Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Opus 5
Anthropic · Medium effort
0.6293Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-6 Astra
OpenAI · Medium effort
0.6293Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Opus 5
Anthropic · Max effort
0.633Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Fable 5.1
Anthropic · Low effort
0.6346Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Opus 5
Anthropic · High effort
0.6367Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-6 Astra
OpenAI · High effort
0.6384Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-6 Astra
OpenAI · Unknown effort
0.6411Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
GPT-6 Astra
OpenAI · Xhigh effort
0.6437Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Opus 5
Anthropic · Xhigh effort
0.6445Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Fable 5.1
Anthropic · Xhigh effort
0.6506Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Fable 5.1
Anthropic · High effort
0.6509Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Fable 5.1
Anthropic · Medium effort
0.6532Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Fable 5.1
Anthropic · Unknown effort
0.6546Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
Claude Opus 5.5
Anthropic · Max effort
0.6823Third partyhttps://conceptualreasoning.ai/lmca via Epoch AI—
  • Epoch AI, 'Capabilities & benchmarking', epoch.ai/benchmarks, CC BY 4.0.

Sources

LMCA (Language Model Conceptual Argumentation): leaderboard and what it measures · supermarket.dev