--- title: "Overcoming Data Scarcity in Multi-Dialectal — Ozyilmaz 2025" description: "Commercial Arabic speech recognisers handle Modern Standard Arabic (MSA) reasonably well but perform poorly on the regional dialects people actually…" authors: ["Omer Tarik Ozyilmaz", "Matt Coler", "Matias Valdenegro-Toro"] year: 2025 venue: "arXiv:2506.02627" source: "https://arxiv.org/abs/2506.02627" doi: "10.21437/interspeech.2025-2260" arxiv: "2506.02627" licence: "CC0 1.0 (arXiv)" licence_url: "http://creativecommons.org/publicdomain/zero/1.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/ozyilmaz-2025-overcoming-data-scarcity-multi-dialectal/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ozyilmaz-2025-overcoming-data-scarcity-multi-dialectal/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/3/) / Overcoming Data Scarcity in Multi-Dialectal Arabic ASR via Whisper Fine-Tuning # Overcoming Data Scarcity in Multi-Dialectal Arabic ASR via Whisper Fine-Tuning **Authors:** Omer Tarik Ozyilmaz, Matt Coler, Matias Valdenegro-Toro · **Year:** 2025 · **Venue:** arXiv:2506.02627 · **Licence:** [CC0 1.0 (arXiv)](http://creativecommons.org/publicdomain/zero/1.0/) **Links:** [Publisher's page](https://doi.org/10.21437/interspeech.2025-2260) · [PDF](https://arxiv.org/pdf/2506.02627) · [DOI 10.21437/interspeech.2025-2260](https://doi.org/10.21437/interspeech.2025-2260) · [arXiv 2506.02627](https://arxiv.org/abs/2506.02627) · [OpenAlex](https://openalex.org/W4415133994) **Citations:** 0 citations (openalex-snapshot · 2026-10-14) · 4 in our library, of the 140884 papers whose bibliographies we read ## Summary Commercial Arabic speech recognisers handle Modern Standard Arabic (MSA) reasonably well but perform poorly on the regional dialects people actually speak, and dialectal training data is scarce and unevenly distributed. The authors fine-tune OpenAI's Whisper (the whisper-small checkpoint) using MSA speech from Mozilla Common Voice and dialectal speech from the MASC corpus. MASC covers five dialects: Gulf, Levantine, Iraqi, Egyptian and Maghrebi. They ask three questions: - How does the amount of MSA fine-tuning data affect accuracy? - Does an MSA training stage help before dialect fine-tuning? - Do one-dialect models beat a single model trained on all dialects? Even a modest amount of MSA data lifts the small model to roughly the level of an untuned whisper-large-v3, at much lower inference cost. The MSA stage brings little gain, which hints that MSA and the dialects share few useful features. A pooled multi-dialect model, built from balanced data, performs about as well as dialect-specific ones. This makes pooling a practical answer to data scarcity in low-resource speech recognition. Code, models and preprocessed data are publicly released. ## Key points - **Data:** - Common Voice 16.1 Arabic is treated as MSA. Its original train and validation parts (40 h) were merged and re-split 80:20, and the 13 h test part was held out. - MASC is crawled from YouTube and has over 1,000 h of speech, but it is heavily skewed: Egyptian 385 h and Levantine 148 h, versus Iraqi 13 h and Maghrebi 17 h. - **Balancing:** each dialect was capped at 20 h. Larger dialects were randomly undersampled and smaller ones were used whole. - **Model and runs:** - The model is whisper-small (244M parameters). - There are six scenarios: five single dialects plus one pooled set. Each was run with three split seeds, with and without the MSA stage, giving 36 models. - Five further models used 20–100% of the MSA data. - **Training:** - AdamW with a peak learning rate of 1e-5 after 500 warm-up steps, batch size 8, and at most 6,000 steps. - Early stopping fired in 33 of the 36 runs, on average after about 2,682 steps. - **Findings:** - Dialects that are geographically far apart, such as Maghrebi and Iraqi, diverge more in performance than neighbours such as Levantine and Gulf. - The authors list missing dialect-specific text normalisation as a limitation. ## Interesting ideas - Capping every dialect at the same number of hours lets low-resource varieties be pooled without the big ones dominating the model. — For Mova (tree): When training gparse on several English UD treebanks (web, spoken, academic genres), cap each source at a fixed token budget so that small treebanks still shape the parser. - Scores are reported as the mean over three random 80% subsets of the test set rather than a single number. This gives a cheap estimate of how stable a result is. — For Mova (learn): Evaluate each parser or reader variant on several seeded subsamples and report the spread, so tiny gains are not mistaken for real improvements. - Fine-tuning on growing fractions (20% to 100%) of the data traces how quickly returns diminish. — For Mova (learn): Draw such learning curves for silver data agreed by witness parsers, to decide when adding more silver stops paying off versus asking an LLM teacher tier. - Comparing per-variety models exposes which varieties sit linguistically close to each other, with performance gaps tracking geographic distance. — For Mova (tree): Use cross-evaluation gaps between treebanks or UD dialects as an explicit distance measure, guiding which sources need conversion rules before merging. - A "standard variety first, then dialect" training stage gave little benefit. This argues for testing staged training against plain pooling instead of assuming the staging helps. — For Mova (learn): Before building curricula, such as formal text first and then fables or school tests, run a pooled baseline as a control to check that the staging actually helps. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @misc{ozyilmaz2025overcoming, title = {{Overcoming Data Scarcity in Multi-Dialectal Arabic ASR via Whisper Fine-Tuning}}, author = {Ozyilmaz, Omer Tarik and Coler, Matt and Valdenegro-Toro, Matias}, year = {2025}, doi = {10.21437/interspeech.2025-2260}, eprint = {2506.02627}, archivePrefix = {arXiv}, url = {https://doi.org/10.21437/interspeech.2025-2260} } ``` - [← previous Optimizing RAG Pipelines for Arabic: A Systematic Analysis of Core Components](https://md.research.elamur.ai/free/en/papers/alsubhi-2025-optimizing-rag-pipelines-arabic-systematic/) - [next → Parsing the Switch: LLM-Based UD Annotation for Complex Code-Switched and Low-Resource Languages](https://md.research.elamur.ai/free/en/papers/kellert-2025-parsing-switch-llm-based-ud/) - [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/3/) ## References 17 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. H. A. Alsayadi, A. A. Abdelhamid, I. Hegazy, B. Alotaibi, and Z. T. Fayed, “Deep investigation of the recent advances in dialectal arabic speech recognition,” IEEE access, vol. 10, pp. 57 063–57 079, 2022. [find](https://scholar.google.com/scholar?q=H%2E+A%2E+Alsayadi%2C+A%2E+A%2E+Abdelhamid%2C+I%2E+Hegazy%2C+B%2E+Alotaibi%2C+and+Z%2E+T%2E+Fayed%2C+%E2%80%9CDeep+investigation+of+the+recent+advances+in+dialectal+arabic+speech+recognition%2C%E2%80%9D+IEEE+access%2C+vol%2E+10%2C+pp%2E+57+063%E2%80%9357+079%2C+) 2. D. Vergyri and K. Kirchhoff, “Automatic diacritization of arabic for acoustic modeling in speech recognition,” in Proceedings of the workshop on computational approaches to Arabic script-based languages, 2004, pp. 66– 73. [find](https://scholar.google.com/scholar?q=D%2E+Vergyri+and+K%2E+Kirchhoff%2C+%E2%80%9CAutomatic+diacritization+of+arabic+for+acoustic+modeling+in+speech+recognition%2C%E2%80%9D+in+Proceedings+of+the+workshop+on+computational+approaches+to+Arabic+script%2Dbased+languag) 3. A. Abdelali, H. Mubarak, S. Chowdhury, M. Hasanain, B. Mousi, S. Boughorbel, S. Abdaljalil, Y. El Kheir, D. Izham, F. Dalvi et al., “Larabench: Benchmarking arabic ai with large language models,” in Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), 2024, pp. 487–520. [find](https://scholar.google.com/scholar?q=A%2E+Abdelali%2C+H%2E+Mubarak%2C+S%2E+Chowdhury%2C+M%2E+Hasanain%2C+B%2E+Mousi%2C+S%2E+Boughorbel%2C+S%2E+Abdaljalil%2C+Y%2E+El+Kheir%2C+D%2E+Izham%2C+F%2E+Dalvi+et+al%2E%2C+%E2%80%9CLarabench%3A+Benchmarking+arabic+ai+with+large+language+models%2C%E2%80%9D+in+P) [our card](https://md.research.elamur.ai/free/en/papers/abdelali-2024-larabench-benchmarking-arabic-ai-large/) 4. A. Radford, J. W. Kim, T. Xu, G. Brockman, C. Mcleavey, and I. Sutskever, “Robust Speech Recognition via Large-Scale Weak Supervision,” in International Conference on Machine Learning. PMLR, Jul. 2023, pp. 28 492– 28 518, iSSN: 2640-3498. [Online]. Available: [https://proceedings.mlr.press/v202/radford23a.html](https://proceedings.mlr.press/v202/radford23a.html) [↗](https://proceedings.mlr.press/v202/radford23a.html) [our card](https://md.research.elamur.ai/free/en/papers/radford-2022-robust-speech-recognition-large-scale/) 5. Y. Zhang, W. Han, J. Qin, Y. Wang, A. Bapna, Z. Chen, N. Chen, B. Li, V. Axelrod, G. Wang et al., “Google usm: Scaling automatic speech recognition beyond 100 languages,” arXiv preprint arXiv:2303.01037, 2023. [↗](https://arxiv.org/abs/2303.01037) 6. D. Yu, L. Deng, and G. Dahl, “Roles of pre-training and fine-tuning in context-dependent dbn-hmms for real-world speech recognition,” in Proc. NIPS Workshop on Deep Learning and Unsupervised Feature Learning. sn, 2010. [find](https://scholar.google.com/scholar?q=D%2E+Yu%2C+L%2E+Deng%2C+and+G%2E+Dahl%2C+%E2%80%9CRoles+of+pre%2Dtraining+and+fine%2Dtuning+in+context%2Ddependent+dbn%2Dhmms+for+real%2Dworld+speech+recognition%2C%E2%80%9D+in+Proc%2E+NIPS+Workshop+on+Deep+Learning+and+Unsupervised+Feature+L) 7. E. Alsharhan and A. Ramsay, “Investigating the effects of gender, dialect, and training size on the performance of arabic speech recognition,” Language Resources and Evaluation, vol. 54, no. 4, pp. 975–998, 2020. [find](https://scholar.google.com/scholar?q=E%2E+Alsharhan+and+A%2E+Ramsay%2C+%E2%80%9CInvestigating+the+effects+of+gender%2C+dialect%2C+and+training+size+on+the+performance+of+arabic+speech+recognition%2C%E2%80%9D+Language+Resources+and+Evaluation%2C+vol%2E+54%2C+no%2E+4%2C+pp%2E+97) 8. S. Rijal, S. Adhikari, M. Dahal, M. Awale, and V. Ojha, “Whisper finetuning on nepali language,” arXiv preprint arXiv:2411.12587, 2024. [↗](https://arxiv.org/abs/2411.12587) 9. M. Torgbi, A. Clayman, J. J. Speight, and H. T. Madabushi, “Adapting whisper for regional dialects: Enhancing public services for vulnerable populations in the united kingdom,” arXiv preprint arXiv:2501.08502, 2025. [↗](https://arxiv.org/abs/2501.08502) 10. R. Ardila, M. Branson, K. Davis, M. Henretty, M. Kohler, J. Meyer, R. Morais, L. Saunders, F. M. Tyers, and G. Weber, “Common voice: A massively-multilingual speech corpus,” arXiv preprint arXiv:1912.06670, 2019. [↗](https://arxiv.org/abs/1912.06670) [our card](https://md.research.elamur.ai/free/en/papers/ardila-2019-common-voice-massively-multilingual-speech/) 11. M. Al-Fetyani, M. Al-Barham, G. Abandah, A. Alsharkawi, and M. Dawas, “Masc: Massive arabic speech corpus,” 2021. [Online]. Available: https://dx.doi.org/10.21227/e1qb-jv46 [↗](https://doi.org/10.21227/e1qb-jv46) 12. T. Wolf, “Huggingface’s transformers: State-ofthe-art natural language processing,” arXiv preprint arXiv:1910.03771, 2019. [↗](https://arxiv.org/abs/1910.03771) [our card](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/) 13. D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014. [↗](https://arxiv.org/abs/1412.6980) 14. I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” arXiv preprint arXiv:1711.05101, 2017. [↗](https://arxiv.org/abs/1711.05101) 15. A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, A. Desmaison, A. Kopf, E. Yang, Z. DeVito, M. Raison, A. Tejani, S. Chilamkurthy, B. Steiner, L. Fang, J. Bai, and S. Chintala, “Pytorch: An imperative style, high-performance deep learning library,” in Advances in Neural Information Processing Systems 32. Curran Associates, Inc., 2019, pp. 8024–8035. [find](https://scholar.google.com/scholar?q=A%2E+Paszke%2C+S%2E+Gross%2C+F%2E+Massa%2C+A%2E+Lerer%2C+J%2E+Bradbury%2C+G%2E+Chanan%2C+T%2E+Killeen%2C+Z%2E+Lin%2C+N%2E+Gimelshein%2C+L%2E+Antiga%2C+A%2E+Desmaison%2C+A%2E+Kopf%2C+E%2E+Yang%2C+Z%2E+DeVito%2C+M%2E+Raison%2C+A%2E+Tejani%2C+S%2E+Chilamkurthy%2C+B%2E+Stei) [our card](https://md.research.elamur.ai/free/en/papers/paszke-2019-pytorch-imperative-style-high-performance/) 16. S. Gandhi, “Fine-tune whisper for multilingual asr with transformers,” 2022. [Online]. Available: [https://huggingface.co/blog/fine-tune-whisper](https://huggingface.co/blog/fine-tune-whisper) [↗](https://huggingface.co/blog/fine-tune-whisper) 17. V. Srivastav, S. Majumdar, N. Koluguri, A. Moumen, S. Gandhi et al., “Open automatic speech recognition leaderboard,” [https://huggingface.co/spaces/hf-audio/](https://huggingface.co/spaces/hf-audio/) open asr leaderboard, 2023. [↗](https://huggingface.co/spaces/hf-audio/)