We published how
@thinkymachines 's Inkling did on Voice Code Bench few weeks ago, and then we asked, what if we fine-tune Inkling? Inkling's main value prop is its small size and robustness, making it a good base model for domain specific fine-tuning. So we that's what we did.
We fine-tuned the same Inkling speech model on 1, 25, and 100 hours of our proprietary data containing alphanumeric entities. And the curve kept moving. 🍊
Not surprising, on the standard 300-item VoiceCodeBench evaluation, the 100-hour checkpoint delivered the strongest result.
Untrained Inkling → 100-hour post-trained Inkling:
📈 Task Success Rate: 56.33% → 79.00% (+22.67 pts)
📈 Entity recovery (CTEM): 86.84% → 94.80% (+7.96 pts)
📉 VoiceCodeBench WER: 2.3748% → 1.6107% (32.2% relative reduction)
🔧 139 misses fixed, 21 prior hits regressed: net +118 exact entities recovered
And the gains scaled as data scaled:
1h: 88.06% CTEM / 59.33% TSR / 2.8188% VCB WER
25h: 92.85% / 72.67% / 1.8517%
100h: 94.80% / 79.00% / 1.6107%
The 100-hour model recovered values the base model missed:
•--revert-last, instead of splitting one flag into “--revert --last”
• tests/auth/login.spec.ts, instead of test/auth/login.spec.ts
• ALLOWLIST_CIDR, instead of inserting an extra underscore
• SN-7KX-9042, instead of dropping the final digit
The largest entity-type gains were email addresses (+30.8 pts), postal addresses (+30.0), file paths (+23.5), environment variables (+22.9), and IP addresses (+20.0).
That’s the Besimple thesis: targeted human data can move the production metrics that matter for voice agents, even when the base model is already strong.
DM me if you want to evaluate your model on this benchmark or build the data that moves it. 🍊
#SpeechRecognition #VoiceAI #ASR #Transcription #DataQuality #PostTraining #Benchmarks
Checkout the full blog at:
besimple.ai/research/voice-c…