---
title: "Efficient ASR for Low-Resource Languages: Leveraging"
description: "The paper asks whether a compact speech recognizer can rival multi-billion-parameter systems for low-resource… 19 references."
authors: ["Srihari Bandarupalli", "Bhavana Akkiraju", "Sri Charan D", "Vamshi Raghu Simha Narasinga", "Anil Vuppala"]
year: 2025
venue: "Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics"
source: "https://aclanthology.org/2025.findings-ijcnlp.99/"
doi: "10.18653/v1/2025.findings-ijcnlp.99"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/bandarupalli-2025-efficient-asr-low-resource-languages/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/bandarupalli-2025-efficient-asr-low-resource-languages/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/17/) / Efficient ASR for Low-Resource Languages: Leveraging Cross-Lingual Unlabeled Data

# Efficient ASR for Low-Resource Languages: Leveraging Cross-Lingual Unlabeled Data

**Authors:** Srihari Bandarupalli, Bhavana Akkiraju, Sri Charan D, Vamshi Raghu Simha Narasinga and [Anil Vuppala](https://md.research.elamur.ai/free/en/authors/anil-vuppala/) · **Year:** 2025 · **Venue:** [Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2025](https://md.research.elamur.ai/free/en/venues/findings/2025/)) · **Publisher:** [The Asian Federation of Natural Language Processing and The Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/the-asian-federation-of-natural-language-processing-and-the-association-for/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.findings-ijcnlp.99/) · [PDF](https://aclanthology.org/2025.findings-ijcnlp.99.pdf) · [DOI 10.18653/v1/2025.findings-ijcnlp.99](https://doi.org/10.18653/v1/2025.findings-ijcnlp.99) · [OpenAlex](https://openalex.org/W7137976774)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper asks whether a compact speech recognizer can rival multi-billion-parameter systems for low-resource, morphologically rich languages. It uses Persian, Arabic and Urdu as a case study. The authors built a pipeline that collects public films, news and interviews, verifies the language, extracts the audio and filters it with Silero VAD. This yields roughly 3,000 hours of unlabeled multilingual speech. They also assembled about 60–74 hours of labeled training data per language. They compare three ways of starting self-supervised pretraining on this corpus: Wav2Vec 2.0 Base trained from scratch (CS), continued pretraining of XLS-R (CP1), and continued pretraining of the English-centric Wav2Vec 2.0 Large (CP2). Each model is then fine-tuned per language with a CTC head that predicts SentencePiece BPE subwords. The 300M-parameter CP2 model beats Whisper Large v3 on Persian and comes within a few WER points on Urdu and Arabic. The authors conclude that relevant data and a sound adaptation strategy can matter more than model scale. The code and model checkpoints are publicly released.

## Key points
- The unlabeled pretraining corpus contains 816 h of Urdu, 878 h of Persian and 1,310 h of Arabic. It was segmented into chunks of 3 to 32 seconds, and only segments with a speech probability above 70% were kept.
- Labeled data comes from Common Voice, IndicVoices, ManaTTS-Persian-Speech-Dataset, ParsiGoo, OpenSLR, MGB-2 and other sources. The test sets are about 10–11 h per language.
- Test WER for Urdu, Persian and Arabic:
  - CP2: 20.6 / 17.1 / 32.9
  - Whisper Large v3 (1.5B): 17.2 / 21.4 / 27.2
  - Seamless Large v2 (2.3B): 31.9 / 41.1 / 34.8
  - plain W2V Large fine-tuning: 39.3 / 33.2 / 48.7
- CS has only 95M parameters and is trained on just 3K hours, yet it still outperforms directly fine-tuned W2V Large. CP2, which starts from English-only pretraining, consistently beats CP1, which starts from XLS-R and its 128 languages.
- CS needed 200k pretraining steps, while CP1 and CP2 converged in about 40k. SentencePiece subwords lowered WER compared with character-level output.

## Interesting ideas
- Continuing to pretrain an existing encoder on in-domain unlabeled data is cheaper than training from scratch and can outperform much larger generic models.
- The authors add a capacity-matched control: the same architecture, fine-tuned without the extra pretraining. This cleanly separates the effect of the data from the effect of model size.
- Very broad pretraining on unrelated languages may cause negative transfer, so a narrower, cleaner starting point can work better.
- The data pipeline relies on simple gates: a confidence threshold of over 70% speech probability and fixed length bounds. These turn noisy web material into usable training data.
- Validation loss is reported alongside the main metric to show convergence and stability differences that WER alone hides.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): The gparse transformer encoder could be further pretrained on raw text from fables, school tests and word problems before fine-tuning it on treebanks.
- **I2** (learn): Every new silver-data or teacher tier should be compared against an identical parser trained without it, which works as a built-in negative control.
- **I3** (tree): When mixing treebanks or converted UD dialects for the parsers, test whether distant genres or dialects actually hurt English accuracy rather than assuming that more data always helps.
- **I4** (learn): Silver trees could likewise be admitted only when witness-parser agreement passes a threshold and sentence length falls within a set range, which filters noise before training.
- **I5** (learn): Parser training reports could log loss curves next to LAS, helping to spot unstable or slow-converging configurations early.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [International Institute of Information Technology, Hyderabad](https://md.research.elamur.ai/free/en/universities/international-institute-of-information-technology-hyderabad/) — University, Hyderabad, India · 42 works · 2,317 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (4): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 3,066 works · [🌍 Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) · 882 works · [🎙️ Speech & spoken language](https://md.research.elamur.ai/free/en/topics/speech/) · 519 works · [🌱 Low-resource languages](https://md.research.elamur.ai/free/en/topics/low-resource-languages/) · 287 works

## Cite

Cite as: Srihari Bandarupalli, Bhavana Akkiraju, Sri Charan D, Vamshi Raghu Simha Narasinga and Anil Vuppala. 2025. Efficient ASR for Low-Resource Languages: Leveraging Cross-Lingual Unlabeled Data. Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{bandarupalli2025efficient,
  title = {{Efficient ASR for Low-Resource Languages: Leveraging Cross-Lingual Unlabeled Data}},
  author = {Srihari Bandarupalli, Bhavana Akkiraju, Sri Charan D, Vamshi Raghu Simha Narasinga and Anil Vuppala},
  booktitle = {Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics},
  year = {2025},
  doi = {10.18653/v1/2025.findings-ijcnlp.99},
  url = {https://doi.org/10.18653/v1/2025.findings-ijcnlp.99}
}
```

- [← previous Efficient Architectures For Low-Resource Machine Translation](https://md.research.elamur.ai/free/en/papers/signoroni-2025-efficient-architectures-low-resource-machine/)
- [next → Embodied Intelligence Systems Based on Large Models: A Survey](https://md.research.elamur.ai/free/en/papers/wang-2025-embodied-intelligence-systems/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/17/)

## References

19 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ahmed Ali, Peter Bell, James Glass, Yacine Messaoui, Hamdy Mubarak, Steve Renals, and Yifan Zhang. 2016. [The mgb-2 challenge: Arabic multi-dialect broadcast media recognition](https://md.research.elamur.ai/free/en/papers/ali-2016-mgb-2-challenge-arabic-multi/). pages 279–284.
2. Rosana Ardila, Megan Branson, Kelly Davis, Michael Kohler, Josh Meyer, Michael Henretty, Reuben Morais, Lindsay Saunders, Francis Tyers, and Gregor Weber. 2020. [Common voice: A massively-multilingual speech corpus](https://md.research.elamur.ai/free/en/papers/ardila-2019-common-voice-massively-multilingual-speech/). In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 4218–4222, Marseille, France. European Language Resources Association.
3. Samee Arif, Aamina Jamal Khan, Mustafa Abbas, Agha Ali Raza, and Awais Athar. 2025. WER we stand: Benchmarking Urdu ASR models. In Proceedings of the 31st International Conference on Computational Linguistics, pages 5952–5961, Abu Dhabi, UAE. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Samee+Arif%2C+Aamina+Jamal+Khan%2C+Mustafa+Abbas%2C+Agha+Ali+Raza%2C+and+Awais+Athar%2E+2025%2E+WER+we+stand%3A+Benchmarking+Urdu+ASR+models%2E+In+Proceedings+of+the+31st+International+Conference+on+Computational+Lin)
4. Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. [wav2vec 2.0: A framework for self-supervised learning of speech representations](https://md.research.elamur.ai/free/en/papers/baevski-2020-wav2vec-2-0-framework-self/). In Advances in Neural Information Processing Systems, volume 33, pages 12449–12460. Curran Associates, Inc.
5. Srihari Bandarupalli, Bhavana Akkiraju, Sri Charan Devarakonda, Harinie Sivaramasethu, Vamshi-raghusimha Narasinga, and Anil Vuppala. 2025. Towards unified processing of Perso-Arabic scripts for ASR. In Proceedings of the 1st Workshop on NLP for Languages Using Arabic Script, pages 23–28, Abu Dhabi, UAE. Association for Computational Linguistics. Seamless Communication, Loïc Barrault, Yu-An Chung, Mariano Coria Meglioli, David Dale, Ning Dong, Mark Duppenthaler, Paul-Ambroise Duquenne, Brian Ellis, Hady Elsahar, Justin Haaheim, John Hoffman, Min-Jae Hwang, Hirofumi Inaguma, Christopher Klaiber, Ilia Kulikov, Pengwei Li, Daniel Licht, Jean Maillard, and 46 others. 2023. [Seamless: Multilingual expressive and streaming speech translation](https://md.research.elamur.ai/free/en/papers/communication-2023-seamless-multilingual-expressive-streaming-speech/). Preprint, arXiv:2312.05187. [↗](https://arxiv.org/abs/2312.05187)
6. Alexis Conneau, Alexei Baevski, Ronan Collobert, Abdelrahman Mohamed, and Michael Auli. 2021. Unsupervised cross-lingual representation learning for speech recognition. pages 2426–2430. [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Alexei+Baevski%2C+Ronan+Collobert%2C+Abdelrahman+Mohamed%2C+and+Michael+Auli%2E+2021%2E+Unsupervised+cross%2Dlingual+representation+learning+for+speech+recognition%2E+pages+2426%E2%80%932430%2E)
7. Jia Cui, Brian Kingsbury, Bhuvana Ramabhadran, Abhinav Sethy, Kartik Audhkhasi, Xiaodong Cui, Ellen Kislal, Lidia Mangu, Markus Nussbaum-Thom, Michael Picheny, Zoltan Tüske, Pavel Golik, Ralf Schlüter, Hermann Ney, Mark J. F. Gales, Kate M. Knill, Anton Ragni, Haipeng Wang, and Phil Woodland. 2015. Multilingual representations for low resource speech recognition and keyword search. In 2015 IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), pages 259–266. [find](https://scholar.google.com/scholar?q=Jia+Cui%2C+Brian+Kingsbury%2C+Bhuvana+Ramabhadran%2C+Abhinav+Sethy%2C+Kartik+Audhkhasi%2C+Xiaodong+Cui%2C+Ellen+Kislal%2C+Lidia+Mangu%2C+Markus+Nussbaum%2DThom%2C+Michael+Picheny%2C+Zoltan+T%C3%BCske%2C+Pavel+Golik%2C+Ralf+Schl%C3%BCter)
8. Yaroslav Getman, Tamas Grosz, and Mikko Kurimo. 2024. What happens in continued pre-training? analysis of self-supervised speech models with continued pre-training for colloquial finnish asr. In Interspeech 2024, pages 5043–5047. [find](https://scholar.google.com/scholar?q=Yaroslav+Getman%2C+Tamas+Grosz%2C+and+Mikko+Kurimo%2E+2024%2E+What+happens+in+continued+pre%2Dtraining%3F+analysis+of+self%2Dsupervised+speech+models+with+continued+pre%2Dtraining+for+colloquial+finnish+asr%2E+In+Inter)
9. Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. 2021. [Hubert: Self-supervised speech representation learning by masked prediction of hidden units](https://md.research.elamur.ai/free/en/papers/hsu-2021-hubert-self-supervised-speech-representation/). IEEE/ACM Transactions on Audio, Speech, and Language Processing, PP:1–1.
10. Tahir Javed, Janki Nawale, Eldho George, Sakshi Joshi, Kaushal Bhogale, Deovrat Mehendale, Ishvinder Sethi, Aparna Ananthanarayanan, Hafsah Faquih, Pratiti Palit, Sneha Ravishankar, Saranya Sukumaran, Tripura Panchagnula, Sunjay Murali, Kunal Gandhi, Ambujavalli R, Manickam M, C Vaijayanthi, Krishnan Karunganni, and 2 others. 2024. IndicVoices: Towards building an inclusive multilingual speech dataset for Indian languages. In Findings of the Association for Computational Linguistics: ACL 2024, pages 10740–10782, Bangkok, Thailand. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Tahir+Javed%2C+Janki+Nawale%2C+Eldho+George%2C+Sakshi+Joshi%2C+Kaushal+Bhogale%2C+Deovrat+Mehendale%2C+Ishvinder+Sethi%2C+Aparna+Ananthanarayanan%2C+Hafsah+Faquih%2C+Pratiti+Palit%2C+Sneha+Ravishankar%2C+Saranya+Sukumaran%2C)
11. Rostislav Kolobov, Olga Okhapkina, Olga Omelchishina, Andrey Platunov, Roman Bedyakin, Vyacheslav Moshkin, Dmitry Menshikov, and Nikolay Mikhaylovskiy. 2021. Mediaspeech: Multilanguage asr benchmark and dataset. Preprint, arXiv:2103.16193. [↗](https://arxiv.org/abs/2103.16193)
12. Nima Moradi. 2024. Persian text-to-speech audio. [find](https://scholar.google.com/scholar?q=Nima+Moradi%2E+2024%2E+Persian+text%2Dto%2Dspeech+audio%2E)
13. Vineel Pratap, Qiantong Xu, Anuroop Sriram, Gabriel Synnaeve, and Ronan Collobert. 2020. [Mls: A large-scale multilingual dataset for speech research](https://md.research.elamur.ai/free/en/papers/pratap-2020-mls-large-scale-multilingual-dataset/). In Interspeech 2020, pages 2757–2761.
14. Mahta Fetrat Qharabagh, Zahra Dehghanian, and Hamid R. Rabiee. 2024. Manatts persian: a recipe for creating tts datasets for lower resource languages. Preprint, arXiv:2409.07259. [↗](https://arxiv.org/abs/2409.07259)
15. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. [Robust speech recognition via large-scale weak supervision](https://md.research.elamur.ai/free/en/papers/radford-2022-robust-speech-recognition-large-scale/). ICML’23. JMLR.org.
16. Francis McCann Ramirez, Luka Chkhetiani, Andrew Ehrenberg, Robert McHardy, Rami Botros, Yash Khare, Andrea Vanzo, Taufiquzzaman Peyash, Gabriel Oexle, Michael Liang, Ilya Sklyar, Enver Fakhan, Ahmed Etefy, Daniel McCrystal, Sam Flamini, Domenic Donato, and Takuya Yoshioka. 2024. Anatomy of industrial scale multilingual ASR. CoRR, abs/2404.09841. [↗](https://arxiv.org/abs/2404.09841)
17. Jörgen Valk and Tanel Alumäe. 2020. Voxlingua107: A dataset for spoken language recognition. 2021 IEEE Spoken Language Technology Workshop (SLT), pages 652–658. [find](https://scholar.google.com/scholar?q=J%C3%B6rgen+Valk+and+Tanel+Alum%C3%A4e%2E+2020%2E+Voxlingua107%3A+A+dataset+for+spoken+language+recognition%2E+2021+IEEE+Spoken+Language+Technology+Workshop+%28SLT%29%2C+pages+652%E2%80%93658%2E)
18. Changhan Wang, Morgane Riviere, Ann Lee, Anne Wu, Chaitanya Talnikar, Daniel Haziza, Mary Williamson, Juan Pino, and Emmanuel Dupoux. 2021. [VoxPopuli: A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation](https://md.research.elamur.ai/free/en/papers/wang-2021-voxpopuli-large-scale-multilingual-speech/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 993–1003, Online. Association for Computational Linguistics.
19. Yingzhi Wang, Anas Alhmoud, and Muhammad Alqurishi. 2024. [Open universal arabic asr leaderboard](https://md.research.elamur.ai/free/en/papers/wang-2024-open-universal-arabic-asr-leaderboard/). Preprint, arXiv:2412.13788. [↗](https://arxiv.org/abs/2412.13788)

*Updated 2 November 2026*