| Qwen2.5-Coder-14B-Instruct | 94.2% | Self-reported | Qwen2.5-Coder Technical Report via Epoch AI | — |
| Qwen2.5-Coder-32B-Instruct | 93.0% | Self-reported | Qwen2.5-Coder Technical Report via Epoch AI | — |
| GPT-4 | 92.0% | Self-reported | GPT-4 Technical Report via Epoch AI | — |
| GPT-4o mini | 91.3% | Self-reported | Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone via Epoch AI | — |
| GPT-4 | 90.0% | Third party | Baichuan 2: Open Large-scale Language Models via Epoch AI | — |
| Phi-3.5-MoE | 88.7% | Self-reported | Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone via Epoch AI | — |
| DeepSeek-Coder-V2-Lite | 87.6% | Self-reported | Qwen2.5-Coder Technical Report via Epoch AI | — |
| Claude Instant 1.2 | 86.7% | Self-reported | Releasing Claude Instant 1.2 via Epoch AI | — |
| Qwen2.5-Coder-7B-Instruct | 86.7% | Self-reported | Qwen2.5-Coder Technical Report via Epoch AI | — |
| Phi-3.5-mini | 86.2% | Self-reported | Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone via Epoch AI | — |
| Gemma 2 9B | 84.9% | Self-reported | Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone via Epoch AI | — |
| Gemini 1.5 Flash | 82.4% | Self-reported | Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone via Epoch AI | — |
| Llama 3.1 8B | 82.4% | Self-reported | Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone via Epoch AI | — |
| Claude Instant 1 | 80.9% | Self-reported | Releasing Claude Instant 1.2 via Epoch AI | — |
| Qwen2.5-Coder-3B-Instruct | 80.7% | Self-reported | Qwen2.5-Coder Technical Report via Epoch AI | — |
| text-davinci-003 | 78.2% | Self-reported | Qwen Technical Report via Epoch AI | — |
| Yi-34B-Chat | 76.0% | Self-reported | Yi: Open Foundation Models by 01.AI via Epoch AI | — |
| Yi-34B-Chat | 71.7% | Self-reported | Yi: Open Foundation Models by 01.AI via Epoch AI | — |
| Stable Beluga 2 (Llama 2 70B fine-tune) | 69.6% | Self-reported | Qwen Technical Report via Epoch AI | — |
| Qwen-14B-Chat | 61.2% | Third party | Yi: Open Foundation Models by 01.AI via Epoch AI | — |
| Qwen-14B-Chat | 59.5% | Third party | Yi: Open Foundation Models by 01.AI via Epoch AI | — |
| GPT-3.5 Turbo | 57.8% | Third party | Baichuan 2: Open Large-scale Language Models via Epoch AI | — |
| text-davinci-003 | 57.1% | Third party | Yi: Open Foundation Models by 01.AI via Epoch AI | — |
| text-davinci-003 | 57.1% | Self-reported | GPT-4 Technical Report via Epoch AI | — |
| Codex (code-davinci-002) | 56.8% | Third party | Stanford HELM via Epoch AI | — |
| Falcon-180B Chat | 54.4% | Third party | Yi: Open Foundation Models by 01.AI via Epoch AI | — |
| Mistral 7B v0.1 | 54.4% | Self-reported | Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone via Epoch AI | — |
| Falcon 2 11B | 53.8% | Self-reported | Falcon2-11B Technical Report via Epoch AI | — |
| text-davinci-003 | 50.6% | Third party | Stanford HELM via Epoch AI | — |
| Mistral 7B v0.1 | 50.0% | Third party | Mixtral of Experts via Epoch AI | — |
| Mistral 7B v0.1 | 47.5% | Third party | Yi: Open Foundation Models by 01.AI via Epoch AI | — |
| GPT-3.5 Turbo | 46.9% | Third party | Stanford HELM via Epoch AI | — |
| Gemma 7B | 46.4% | Self-reported | Gemma: Open Models Based on Gemini Research and Technology via Epoch AI | — |
| Gemma 7B | 46.4% | Self-reported | Nemotron-4 15B Technical Report via Epoch AI | — |
| Baichuan2-13B-Chat | 45.7% | Third party | Yi: Open Foundation Models by 01.AI via Epoch AI | — |
| Yi-6B-Chat | 44.9% | Self-reported | Yi: Open Foundation Models by 01.AI via Epoch AI | — |
| text-davinci-002 | 41.5% | Third party | Stanford HELM via Epoch AI | — |
| Yi-6B-Chat | 38.4% | Self-reported | Yi: Open Foundation Models by 01.AI via Epoch AI | — |
| Mistral 7B v0.1 | 38.1% | Third party | Stanford HELM via Epoch AI | — |
| Mistral 7B v0.1 | 35.4% | Self-reported | Nemotron-4 15B Technical Report via Epoch AI | — |
| Mistral 7B v0.2 | 35.4% | Third party | Gemma: Open Models Based on Gemini Research and Technology via Epoch AI | — |
| Llama 2 13B | 34.3% | Third party | Mixtral of Experts via Epoch AI | — |
| Falcon-40B Instruct | 33.8% | Third party | Stanford HELM via Epoch AI | — |
| ChatGLM2-6B | 32.4% | Self-reported | Qwen Technical Report via Epoch AI | — |
| Llama 2 13B | 29.6% | Self-reported | Qwen Technical Report via Epoch AI | — |
| ChatGLM2-6B | 28.9% | Third party | Baichuan 2: Open Large-scale Language Models via Epoch AI | — |
| Llama 2 13B | 28.9% | Third party | Baichuan 2: Open Large-scale Language Models via Epoch AI | — |
| ChatGLM2-6B | 28.8% | Self-reported | Qwen Technical Report via Epoch AI | — |
| Llama 2 13B | 28.7% | Third party | Gemma: Open Models Based on Gemini Research and Technology via Epoch AI | — |
| Llama 2 13B | 28.7% | Self-reported | Nemotron-4 15B Technical Report via Epoch AI | — |
| Llama 2 13B | 28.7% | Self-reported | Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI | — |
| Falcon-40B Instruct | 25.0% | Third party | Stanford HELM via Epoch AI | — |
| Llama 2 13B | 24.5% | Third party | Stanford HELM via Epoch AI | — |
| Baichuan2-13B-Chat | 23.3% | Third party | Yi: Open Foundation Models by 01.AI via Epoch AI | — |
| Falcon-40B Instruct | 21.5% | Self-reported | Falcon2-11B Technical Report via Epoch AI | — |
| Falcon-40B Instruct | 19.6% | Third party | Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI | — |
| Falcon-40B Instruct | 19.6% | Self-reported | Qwen Technical Report via Epoch AI | — |
| Gemma 2B | 17.7% | Self-reported | Gemma: Open Models Based on Gemini Research and Technology via Epoch AI | — |
| Llama 2 7B | 16.7% | Third party | Yi: Open Foundation Models by 01.AI via Epoch AI | — |
| Llama 2 7B | 16.7% | Self-reported | Qwen Technical Report via Epoch AI | — |
| MPT-30B (Instruct, Chat) | 16.4% | Third party | Stanford HELM via Epoch AI | — |
| Llama 2 7B | 16.2% | Third party | Baichuan 2: Open Large-scale Language Models via Epoch AI | — |
| Llama 2 7B | 16.0% | Third party | Mixtral of Experts via Epoch AI | — |
| InternLM-Chat-20B | 15.7% | Third party | Yi: Open Foundation Models by 01.AI via Epoch AI | — |
| MPT-30B (Instruct, Chat) | 15.2% | Third party | Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI | — |
| MPT-30B (Instruct, Chat) | 15.2% | Self-reported | Qwen Technical Report via Epoch AI | — |
| Llama 2 7B | 14.6% | Third party | Gemma: Open Models Based on Gemini Research and Technology via Epoch AI | — |
| Llama 2 7B | 14.6% | Self-reported | Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI | — |
| Llama 2 7B | 14.6% | Self-reported | Textbooks Are All You Need II: phi-1.5 technical report via Epoch AI | — |
| Baichuan-7B (base) | 9.2% | Self-reported | Baichuan 2: Open Large-scale Language Models via Epoch AI | — |
| MPT-7B (Instruct, Chat, StoryWriter) | 9.1% | Self-reported | Qwen Technical Report via Epoch AI | — |
| GPT-3 davinci | 9.0% | Third party | Stanford HELM via Epoch AI | — |
| MPT-7B (Instruct, Chat, StoryWriter) | 8.6% | Third party | Baichuan 2: Open Large-scale Language Models via Epoch AI | — |
| Falcon-7B Instruct | 6.8% | Third party | Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI | — |
| Falcon-7B Instruct | 6.8% | Self-reported | Qwen Technical Report via Epoch AI | — |
| Falcon-7B Instruct | 6.8% | Self-reported | Textbooks Are All You Need II: phi-1.5 technical report via Epoch AI | — |
| MPT-7B (Instruct, Chat, StoryWriter) | 6.8% | Third party | Llama 2: Open Foundation and Fine-Tuned Chat Models via Epoch AI | — |
| MPT-7B (Instruct, Chat, StoryWriter) | 6.8% | Self-reported | Textbooks Are All You Need II: phi-1.5 technical report via Epoch AI | — |
| Falcon-7B Instruct | 5.5% | Third party | Baichuan 2: Open Large-scale Language Models via Epoch AI | — |
| Falcon-7B Instruct | 4.6% | Self-reported | Falcon2-11B Technical Report via Epoch AI | — |
| Falcon-7B Instruct | 4.0% | Third party | Stanford HELM via Epoch AI | — |
| OPT-66B | 1.8% | Third party | Stanford HELM via Epoch AI | — |
| GPT-3 curie | 1.6% | Third party | Stanford HELM via Epoch AI | — |
| GPT-3 babbage | 0.7% | Third party | Stanford HELM via Epoch AI | — |
| GPT-3 ada | 0.6% | Third party | Stanford HELM via Epoch AI | — |
| text-curie-001 (InstructGPT) | 0.6% | Third party | Stanford HELM via Epoch AI | — |
| text-ada-001 (InstructGPT) | 0.4% | Third party | Stanford HELM via Epoch AI | — |
| text-babbage-001 (InstructGPT) | 0.0% | Third party | Stanford HELM via Epoch AI | — |