BuzzASR: A Swarm of 100+ Monolingual Speech Recognition Models
BuzzASR introduces 102 language-specialized fine-tuned Whisper models, scaling monolingual adaptation to a previously unexplored breadth Models outperform Whisper-large-v3 on 77 out of 102 languages, achieving an average character error rate (CER) reduction by a factor of over 2.8x A novel tokenizer replacement strategy yields an average 3.3x improvement in compression rate (characters per token), with gains up to 21.7x over Whisper's multilingual BPE State-of-the-art open-source CER results on
Analysis
TL;DR
- BuzzASR introduces 102 language-specialized fine-tuned Whisper models, scaling monolingual adaptation to a previously unexplored breadth
- Models outperform Whisper-large-v3 on 77 out of 102 languages, achieving an average character error rate (CER) reduction by a factor of over 2.8x
- A novel tokenizer replacement strategy yields an average 3.3x improvement in compression rate (characters per token), with gains up to 21.7x over Whisper's multilingual BPE
- State-of-the-art open-source CER results on 27 of 102 languages across combined FLEURS and Common Voice test sets
- All models, code, and detailed results are publicly released
Why It Matters
This work demonstrates that simple monolingual fine-tuning, when scaled massively across 102 languages, can dramatically outperform general-purpose multilingual models like Whisper-large-v3 — challenging the assumption that bigger multilingual models are always the best solution. For AI practitioners working with low-resource or underrepresented languages, BuzzASR provides a practical, open-source pathway to deploy high-quality ASR systems without relying on proprietary multilingual models.
Technical Details
- Architecture: Built on OpenAI's Whisper end-to-end Transformer-based ASR models, fine-tuned separately for each of 102 languages using monolingual data from the FLEURS dataset
- Tokenizer replacement: Replaced Whisper's multilingual BPE tokenizer with language-specific tokenizers, achieving an average 3.3x improvement in compression rate (characters per token), with maximum gains of 21.7x
- Data augmentation: Incorporated text-only fine-tuning as a data augmentation strategy to further improve language adaptation
- Evaluation: Benchmarked on the combined FLEURS and Common Voice test sets, measuring character error rate (CER) across all 102 languages
- Open-source release: All 102 models, training code, and detailed evaluation results are publicly available
Industry Insight
- The results strongly suggest that for production ASR systems targeting specific languages — especially low-resource ones — language-specific fine-tuning should be the default strategy rather than relying on multilingual foundation models
- Tokenizer customization is a high-leverage, low-cost intervention: the 3.3x average compression improvement indicates that multilingual tokenizers are a significant bottleneck, and replacing them can unlock substantial performance gains without architectural changes
- The massive scale-up to 102 languages validates that simple fine-tuning strategies generalize well across diverse language families, encouraging the community to adopt similar "swarm" approaches for other modalities beyond speech
Disclaimer: The above content is generated by AI and is for reference only.