AutomationBench (and AutomationBench-AA)

Measures: An agent carries out realistic business workflows across about 40 simulated SaaS apps (Gmail, Sheets, Slack, Salesforce, Zendesk, Jira, HubSpot and others) in sales, marketing, operations, support, finance and HR, finding the right APIs itself and following business rules; a task passes only when every end-state assertion holds.

Maintainer: Zapier; Artificial Analysis runs the AutomationBench-AA variant
Introduced: 2026
Status: Active
Saturation: Headroom
Typical provenance: Independent
In scoring set: Yes

Paper Official leaderboard

Leaderboard

Show independent results only

Every result we hold, whoever ran it. Only independent and third-party rows count towards a score.

No results to show.