OSWorld-Verified

Measures: A multimodal agent completes 369 open-ended tasks on real Ubuntu/Windows desktops (office apps, browser, file system, multi-app workflows), checked by execution-based evaluation scripts; scored as % of tasks completed within a step budget (15/50/100 steps).

Maintainer: XLANG Lab, University of Hong Kong
Introduced: 2024
Status: Superseded
Saturation: Saturated
Typical provenance: Mixed
In scoring set: No

Paper Official leaderboard

Leaderboard

Show all results

Independent and third-party results only.

Leaderboard
Model	% tasks completed	Provenance	Source	Evaluated
Claude Sonnet 4.6 Anthropic	72.1%	Third party	OS World Website via Epoch AI	—
Kimi K2.5 Moonshot AI	63.3%	Third party	OS World Website via Epoch AI	—
Claude Sonnet 4.5 Anthropic	62.9%	Third party	OS World Website via Epoch AI	—
Claude Sonnet 4.5 Anthropic	58.1%	Third party	OS World Website via Epoch AI	—
Claude Sonnet 4 Anthropic	43.9%	Third party	OS World Website via Epoch AI	—
Claude Sonnet 4.5 Anthropic	42.9%	Third party	OS World Website via Epoch AI	—
Claude Sonnet 4 Anthropic	41.4%	Third party	OS World Website via Epoch AI	—
Claude 3.7 Sonnet Anthropic	35.8%	Third party	OS World Website via Epoch AI	—
Claude 3.7 Sonnet Anthropic	35.6%	Third party	OS World Website via Epoch AI	—
Computer-Using Agent (computer-use-preview) OpenAI	31.3%	Third party	OS World Website via Epoch AI	—
Claude Sonnet 4 Anthropic	31.2%	Third party	OS World Website via Epoch AI	—
Computer-Using Agent (computer-use-preview) OpenAI	30.5%	Third party	OS World Website via Epoch AI	—
Claude 3.7 Sonnet Anthropic	27.1%	Third party	OS World Website via Epoch AI	—
Computer-Using Agent (computer-use-preview) OpenAI	26.0%	Third party	OS World Website via Epoch AI	—
o3 OpenAI · Medium effort	23.0%	Third party	OS World Website via Epoch AI	—
o3 OpenAI · Medium effort	17.2%	Third party	OS World Website via Epoch AI	—
o3 OpenAI · Medium effort	9.1%	Third party	OS World Website via Epoch AI	—
Qwen2.5-VL-72B-Instruct Alibaba Group	5.0%	Third party	OS World Website via Epoch AI	—
Qwen2.5-VL-72B-Instruct Alibaba Group	4.4%	Third party	OS World Website via Epoch AI	—

Epoch AI, 'Capabilities & benchmarking', epoch.ai/benchmarks, CC BY 4.0.

Sources

Epoch AI