Today we're releasing the Avera Labs STT Leaderboard - a benchmark that finally measures what actually matters beyond WER.
For years, WER (Word Error Rate) has been the default ASR metric. It tells you whether the words match.But in real phone workflows, WER measures the text, not the conversation. It ignores structure, intent, timing, and the subtle cues that make transcripts usable in production.
We got something better.
The Avera Labs leaderboard adds four dimensions that matter for voice agents:
1. Speaker Diarization: Are words attributed to the right speaker, with clean turn boundaries?
2. Paralinguistic Events: Laughter, pauses, breaths, coughs… all the texture that humans use.
3. Disfluencies: Stutters, restarts, partial words — critical for intent and phrasing.
4. Filled Pauses: “Um,” “uh,” and hesitations: include them when they signal meaning, omit when noise.
WER tells you what was said, but these metrics tell you how it was said. And that’s what determines whether a transcript is actually useful, for QA, analytics, compliance, and downstream LLM reasoning.
Congrats on winning on STT
@elevenlabs!