Updated daily at 05:00 UTC

Benchmark

Toolathlon (Tool Decathlon)

Measures: An agent completes 108 long, multi-app tasks (about 20 interactions each) using 32 MCP servers (604 tools) plus local toolkits, starting from realistic app states; a script checks the final state and the score is Pass@1.

Maintainer: HKUST NLP (hkust-nlp)
Introduced: 2025
Status: Active
Saturation: Headroom
Typical provenance: Independent
In scoring set: Yes

Paper Official leaderboard

Leaderboard

Show all results

Independent and third-party results only.

No independent or third-party results yet. Show all results