Updated daily at 05:00 UTC
Benchmark
Big Bench Audio
Measures: 1,000 spoken reasoning questions (250 each: formal fallacies, navigate, object counting, web of lies) adapted from BIG-Bench Hard; the model answers in audio and an LLM judge marks each answer right or wrong; accuracy is over all 1,000 questions.
- Maintainer
- Artificial Analysis
- Introduced
- 2024
- Status
- Active
- Saturation
- Saturated
- Typical provenance
- Independent
- In scoring set
- No
Leaderboard
Independent and third-party results only.
No independent or third-party results yet. Show all results