Benchmark

Measures: An AI agent must deliver 240 real freelance projects end to end (game development, product design, architecture, data analysis, video animation and more; 23 domains, over 6,000 hours and $140,000 of human work); trained experts manually judge whether a reasonable client would accept each deliverable as at least as good as the human one, and the automation rate is the share that pass.