---
title: "Leveraging Data Collection and Unsupervised — Abdallah 2023"
description: "Tunisian Arabic has no standard spelling, few annotated recordings, and speakers often mix in French and English."
authors: ["Ahmed Amine Ben Abdallah", "Ata Kabboudi", "Amir Kanoun", "Salah Zaiem"]
year: 2023
venue: "arXiv:2309.11327"
source: "https://arxiv.org/abs/2309.11327"
doi: "10.48550/arxiv.2309.11327"
arxiv: "2309.11327"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/abdallah-2023-leveraging-data-collection-unsupervised-learning/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/abdallah-2023-leveraging-data-collection-unsupervised-learning/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/4/) / Leveraging Data Collection and Unsupervised Learning for Code-switched Tunisian Arabic Automatic Speech Recognition

# Leveraging Data Collection and Unsupervised Learning for Code-switched Tunisian Arabic Automatic Speech Recognition

**Authors:** [Ahmed Amine Ben Abdallah](https://md.research.elamur.ai/free/en/authors/ahmed-amine-ben-abdallah/), Ata Kabboudi, Amir Kanoun, Salah Zaiem · **Year:** 2023 · **Venue:** arXiv:2309.11327 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2309.11327) · [PDF](https://arxiv.org/pdf/2309.11327) · [DOI 10.48550/arxiv.2309.11327](https://doi.org/10.48550/arxiv.2309.11327) · [arXiv 2309.11327](https://arxiv.org/abs/2309.11327) · [OpenAlex](https://openalex.org/W4392932033)

**Citations:** 12 citations (openalex-snapshot · 2026-10-14) · [18](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
Tunisian Arabic has no standard spelling, few annotated recordings, and speakers often mix in French and English. Large multilingual and Standard Arabic speech models do very badly on it. The authors build public resources and a baseline system to address this. They assembled text from the Tunisiya corpus and scraped code-switched text from the web. They recorded read speech through a crowd tool and hand-annotated radio and podcast audio that mixes three languages. They also gathered unlabeled TV audio.

Their recognizer fine-tunes a WavLM encoder with a CTC head and rescores outputs with a KenLM 4-gram model. Pseudo-labels on the unlabeled audio add self-training. For code-switched speech, a small mixer network combines frozen Tunisian, French and English recognizers.

The system clearly beats earlier Tunisian results, and a better language model gives the largest gains. A human evaluation suggests that automatic WER exaggerates errors because valid spellings vary. All data, models and code are released, giving the community a hard, realistic benchmark for three-language dialectal ASR.

## Key points
- Off-the-shelf models (MMS, Whisper Large v2, an Arabic wav2vec2.0) score roughly 74–139% WER on Tunisian test sets.
- New datasets released on Zenodo and Hugging Face:
  - **TunSwitch TO:** read speech from 89 contributors, 2,631 phrases.
  - **TunSwitch CS:** about 8h of training audio, roughly 13.9% French and 13.3% English words, about five times the mixing found in STAC.
  - **Unlabeled TV audio:** 317h, reduced to 153h after cleaning.
- The released text includes about 4.3M non-code-switched words and a much smaller code-switched set.
- The English-only WavLM encoder worked better than the 53-language wav2vec2.0 XLSR.
- Best WERs:
  - TARIC: 10.55, versus 22.6 in earlier work.
  - IWSLT: 38.86, versus 41.5.
  - TunSwitch TO: 25.54 with self-training and an out-of-domain LM, versus 41.45 with no LM.
- On the code-switched audio, the human annotators agreed only 80.4% of the time.

## Interesting ideas
- Annotators wrapped French and English words in inline language tags even though no model used them, keeping the data open to later code-switching work and usage statistics. — For Mova (tokens): Mark foreign or borrowed tokens (e.g., French phrases in English text) during annotation so the tagger and lemmatizer can treat them separately later.
- Rather than training one model on scarce mixed data, they froze three single-language experts and trained only a small BiLSTM "mixer" over their concatenated outputs on the little mixed data available. — For Mova (tree): A lightweight learned combiner over the frozen perceptron parser and gparse could merge their arc scores on hard or out-of-domain sentences without retraining either parser.
- When pseudo-labeled audio was added, retraining from scratch beat continuing to fine-tune the first model. — For Mova (learn): When silver trees from witness parsers are added, compare a fresh retrain against incremental fine-tuning instead of assuming continued training is better.
- They ran a human evaluation to separate true recognition errors from mismatches caused by inconsistent spelling in the reference transcripts. — For Mova (learn): Alongside exact-match metrics, have humans or a teacher judge whether disagreements with gold are real errors or acceptable variants (e.g., equally valid UD attachments), so that evaluation noise is not mistaken for model weakness.
- Sentences used as recording prompts were removed from the language-model corpus, preventing test leakage into the LM. — For Mova (learn): Systematically remove evaluation sentences (fables, school tests) from any corpus used to build compiled knowledge or LMs, as a built-in leakage control.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

## Cited by

Cited by

18 papers of our library cite it (of the 140884 whose bibliographies we read); 4 of them have a page here, 14 not yet.

- [Bulbul: A Dataset for Dialectal Arabic Speech Recognition](https://md.research.elamur.ai/free/en/papers/ashraf-2026-bulbul-dataset-dialectal-arabic-speech/)
- [ADI-20: Arabic Dialect Identification dataset and models](https://md.research.elamur.ai/free/en/papers/elleuch-2025-adi-20-arabic-dialect-identification/)
- [Ara-Best-RQ: Multi Dialectal Arabic SSL](https://md.research.elamur.ai/free/en/papers/elleuch-2026-ara-best-rq-multi-dialectal/)
- [LinTO Audio and Textual Datasets to Train and Evaluate Automatic Speech Recognition in Tunisian Arabic Dialect](https://md.research.elamur.ai/free/en/papers/naouara-2025-linto-audio-textual-datasets-train/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{abdallah2023leveraging,
  title = {{Leveraging Data Collection and Unsupervised Learning for Code-switched Tunisian Arabic Automatic Speech Recognition}},
  author = {Abdallah, Ahmed Amine Ben and Kabboudi, Ata and Kanoun, Amir and Zaiem, Salah},
  year = {2023},
  doi = {10.48550/arxiv.2309.11327},
  eprint = {2309.11327},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2309.11327}
}
```

- [← previous Leveraging BERT Language Model for Arabic Long Document Classification](https://md.research.elamur.ai/free/en/papers/al-2023-qurishi-leveraging-bert-language-model-arabic/)
- [next → Lexicalised and non-lexicalized multi-word expressions in WordNet: a cross-encoder approach](https://md.research.elamur.ai/free/en/papers/maziarz-2023-lexicalised-non-lexicalized-multi-word/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/4/)

## References

25 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever, “Robust speech recognition via large-scale weak supervision,” 2022. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jong+Wook+Kim%2C+Tao+Xu%2C+Greg+Brockman%2C+Christine+McLeavey%2C+and+Ilya+Sutskever%2C+%E2%80%9CRobust+speech+recognition+via+large%2Dscale+weak+supervision%2C%E2%80%9D+2022%2E) [our card](https://md.research.elamur.ai/free/en/papers/radford-2022-robust-speech-recognition-large-scale/)
2. Vineel Pratap, Andros Tjandra, Bowen Shi, Paden Tomasello, Arun Babu, Sayani Kundu, Ali Elkahky, Zhaoheng Ni, Apoorv Vyas, Maryam Fazel-Zarandi, Alexei Baevski, Yossi Adi, Xiaohui Zhang, Wei-Ning Hsu, Alexis Conneau, and Michael Auli, “Scaling speech technology to 1,000+ languages,” arXiv, 2023. [find](https://scholar.google.com/scholar?q=Vineel+Pratap%2C+Andros+Tjandra%2C+Bowen+Shi%2C+Paden+Tomasello%2C+Arun+Babu%2C+Sayani+Kundu%2C+Ali+Elkahky%2C+Zhaoheng+Ni%2C+Apoorv+Vyas%2C+Maryam+Fazel%2DZarandi%2C+Alexei+Baevski%2C+Yossi+Adi%2C+Xiaohui+Zhang%2C+Wei%2DNing+Hsu%2C) [our card](https://md.research.elamur.ai/free/en/papers/pratap-2023-scaling-speech-technology-1-000/)
3. Inès Zribi, Rahma Boujelbane, Abir Masmoudi, Mariem Ellouze, Lamia Hadrich Belguith, and Nizar Habash, “A conventional orthography for tunisian arabic.,” in LREC, 2014, pp. 2355–2361. [find](https://scholar.google.com/scholar?q=Ine%CC%80s+Zribi%2C+Rahma+Boujelbane%2C+Abir+Masmoudi%2C+Mariem+Ellouze%2C+Lamia+Hadrich+Belguith%2C+and+Nizar+Habash%2C+%E2%80%9CA+conventional+orthography+for+tunisian+arabic%2E%2C%E2%80%9D+in+LREC%2C+2014%2C+pp%2E+2355%E2%80%932361%2E)
4. Hala Mulki, Hatem Haddad, Chedi Bechikh Ali, and Ismail Babaoğlu, “Tunisian dialect sentiment analysis: a natural language processing-based approach,” Computación y Sistemas, vol. 22, no. 4, 2018. [find](https://scholar.google.com/scholar?q=Hala+Mulki%2C+Hatem+Haddad%2C+Chedi+Bechikh+Ali%2C+and+Ismail+Babaog%CC%86lu%2C+%E2%80%9CTunisian+dialect+sentiment+analysis%3A+a+natural+language+processing%2Dbased+approach%2C%E2%80%9D+Computacio%CC%81n+y+Sistemas%2C+vol%2E+22%2C+no%2E+4%2C+2018%2E)
5. Abir Masmoudi, Fethi Bougares, Mariem Ellouze, Yannick Estève, and Lamia Belguith, “Automatic speech recognition system for tunisian dialect,” Language Resources and Evaluation, vol. 52, pp. 249–267, 2018. [find](https://scholar.google.com/scholar?q=Abir+Masmoudi%2C+Fethi+Bougares%2C+Mariem+Ellouze%2C+Yannick+Este%CC%80ve%2C+and+Lamia+Belguith%2C+%E2%80%9CAutomatic+speech+recognition+system+for+tunisian+dialect%2C%E2%80%9D+Language+Resources+and+Evaluation%2C+vol%2E+52%2C+pp%2E+249%E2%80%93267%2C)
6. Abir Messaoudi, Hatem Haddad, Chayma Fourati, Moez Ben-Haj Hmida, Aymen Ben Elhaj Mabrouk, and Mohamed Graiet, “Tunisian dialectal end-to-end speech recognition based on deepspeech,” Procedia Computer Science, vol. 189, pp. 183– 190, 2021. [find](https://scholar.google.com/scholar?q=Abir+Messaoudi%2C+Hatem+Haddad%2C+Chayma+Fourati%2C+Moez+Ben%2DHaj+Hmida%2C+Aymen+Ben+Elhaj+Mabrouk%2C+and+Mohamed+Graiet%2C+%E2%80%9CTunisian+dialectal+end%2Dto%2Dend+speech+recognition+based+on+deepspeech%2C%E2%80%9D+Procedia+Computer)
7. Jonatas Grosman, “Fine-tuned XLSR-53 large model for speech recognition in Arabic,” 2021. [find](https://scholar.google.com/scholar?q=Jonatas+Grosman%2C+%E2%80%9CFine%2Dtuned+XLSR%2D53+large+model+for+speech+recognition+in+Arabic%2C%E2%80%9D+2021%2E)
8. Lotfi Sayahi, “Code-switching and language change in tunisia,” vol. 2011, no. 211, pp. 113–133, 2011. [find](https://scholar.google.com/scholar?q=Lotfi+Sayahi%2C+%E2%80%9CCode%2Dswitching+and+language+change+in+tunisia%2C%E2%80%9D+vol%2E+2011%2C+no%2E+211%2C+pp%2E+113%E2%80%93133%2C+2011%2E)
9. Xian Shi, Qiangze Feng, and Lei Xie, “The asru 2019 mandarin-english code-switching speech recognition challenge: Open datasets, tracks, methods and results,” arXiv preprint arXiv:2007.05916, 2020. [↗](https://arxiv.org/abs/2007.05916)
10. Djegdjiga Amazouz, Martine Adda-Decker, and Lori Lamel, “Addressing Code-Switching in French/Algerian Arabic Speech,” in Interspeech 2017, Stockholm, Sweden, Aug. 2017, pp. 62–66. [find](https://scholar.google.com/scholar?q=Djegdjiga+Amazouz%2C+Martine+Adda%2DDecker%2C+and+Lori+Lamel%2C+%E2%80%9CAddressing+Code%2DSwitching+in+French%2FAlgerian+Arabic+Speech%2C%E2%80%9D+in+Interspeech+2017%2C+Stockholm%2C+Sweden%2C+Aug%2E+2017%2C+pp%2E+62%E2%80%9366%2E)
11. Injy Hamed, Pavel Denisov, Chia-Yu Li, Mohamed Elmahdy, Slim Abdennadher, and Ngoc Thang Vu, “Investigations on speech recognition systems for low-resource dialectal arabic– english code-switching speech,” Computer Speech & Language, vol. 72, pp. 101278, 2022. [find](https://scholar.google.com/scholar?q=Injy+Hamed%2C+Pavel+Denisov%2C+Chia%2DYu+Li%2C+Mohamed+Elmahdy%2C+Slim+Abdennadher%2C+and+Ngoc+Thang+Vu%2C+%E2%80%9CInvestigations+on+speech+recognition+systems+for+low%2Dresource+dialectal+arabic%E2%80%93+english+code%2Dswitching+spe)
12. Sunayana Sitaram, Khyathi Raghavi Chandu, Sai Krishna Rallabandi, and Alan W Black, “A survey of code-switched speech and language processing,” 2019. [find](https://scholar.google.com/scholar?q=Sunayana+Sitaram%2C+Khyathi+Raghavi+Chandu%2C+Sai+Krishna+Rallabandi%2C+and+Alan+W+Black%2C+%E2%80%9CA+survey+of+code%2Dswitched+speech+and+language+processing%2C%E2%80%9D+2019%2E)
13. Karen McNeil, “Tunisian arabic corpus: Creating a written corpus of an ‘unwritten’language,” Arabic corpus linguistics, vol. 30, 2018. [find](https://scholar.google.com/scholar?q=Karen+McNeil%2C+%E2%80%9CTunisian+arabic+corpus%3A+Creating+a+written+corpus+of+an+%E2%80%98unwritten%E2%80%99language%2C%E2%80%9D+Arabic+corpus+linguistics%2C+vol%2E+30%2C+2018%2E)
14. Hiroki Nakayama, Takahiro Kubo, Junya Kamura, Yasufumi Taniguchi, and Xu Liang, “doccano: Text annotation tool for human,” 2018. [find](https://scholar.google.com/scholar?q=Hiroki+Nakayama%2C+Takahiro+Kubo%2C+Junya+Kamura%2C+Yasufumi+Taniguchi%2C+and+Xu+Liang%2C+%E2%80%9Cdoccano%3A+Text+annotation+tool+for+human%2C%E2%80%9D+2018%2E)
15. Hervé Bredin, Ruiqing Yin, Juan Manuel Coria, Gregory Gelly, Pavel Korshunov, Marvin Lavechin, Diego Fustes, Hadrien Titeux, Wassim Bouaziz, and Marie-Philippe Gill, “Pyannote. audio: neural building blocks for speaker diarization,” in ICASSP. IEEE, 2020. [find](https://scholar.google.com/scholar?q=Herve%CC%81+Bredin%2C+Ruiqing+Yin%2C+Juan+Manuel+Coria%2C+Gregory+Gelly%2C+Pavel+Korshunov%2C+Marvin+Lavechin%2C+Diego+Fustes%2C+Hadrien+Titeux%2C+Wassim+Bouaziz%2C+and+Marie%2DPhilippe+Gill%2C+%E2%80%9CPyannote%2E+audio%3A+neural+building)
16. Marcely Zanon Boito, John Ortega, Hugo Riguidel, Antoine Laurent, Loı̈c Barrault, Fethi Bougares, Firas Chaabani, Ha Nguyen, Florentin Barbier, Souhir Gahbiche, et al., “On-trac consortium systems for the iwslt 2022 dialect and low-resource speech translation tasks,” . [find](https://scholar.google.com/scholar?q=Marcely+Zanon+Boito%2C+John+Ortega%2C+Hugo+Riguidel%2C+Antoine+Laurent%2C+Lo%C4%B1%CC%88c+Barrault%2C+Fethi+Bougares%2C+Firas+Chaabani%2C+Ha+Nguyen%2C+Florentin+Barbier%2C+Souhir+Gahbiche%2C+et+al%2E%2C+%E2%80%9COn%2Dtrac+consortium+systems+for)
17. Anastasopoulos Antonios, Barrault Loc, Luisa Bentivogli, Marcely Zanon Boito, Bojar Ondřej, Roldano Cattoni, Currey Anna, Dinu Georgiana, Duh Kevin, Elbayad Maha, et al., “Findings of the iwslt 2022 evaluation campaign.,” in IWSLT 2022. ACL, 2022. [find](https://scholar.google.com/scholar?q=Anastasopoulos+Antonios%2C+Barrault+Loc%2C+Luisa+Bentivogli%2C+Marcely+Zanon+Boito%2C+Bojar+Ondr%CC%8Cej%2C+Roldano+Cattoni%2C+Currey+Anna%2C+Dinu+Georgiana%2C+Duh+Kevin%2C+Elbayad+Maha%2C+et+al%2E%2C+%E2%80%9CFindings+of+the+iwslt+2022+) [our card](https://md.research.elamur.ai/free/en/papers/anastasopoulos-2022-findings-iwslt-2022-evaluation-campaign/)
18. Salah Zaiem, Youcef Kemiche, Titouan Parcollet, Slim Essid, and Mirco Ravanelli, “Speech self-supervised representation benchmarking: Are we doing it right?,” arXiv preprint arXiv:2306.00452, 2023. [↗](https://arxiv.org/abs/2306.00452)
19. Arun Babu, Changhan Wang, Andros Tjandra, Kushal Lakhotia, Qiantong Xu, Naman Goyal, Kritika Singh, Patrick von Platen, Yatharth Saraf, Juan Pino, et al., “Xls-r: Self-supervised cross-lingual speech representation learning at scale,” 2021. [find](https://scholar.google.com/scholar?q=Arun+Babu%2C+Changhan+Wang%2C+Andros+Tjandra%2C+Kushal+Lakhotia%2C+Qiantong+Xu%2C+Naman+Goyal%2C+Kritika+Singh%2C+Patrick+von+Platen%2C+Yatharth+Saraf%2C+Juan+Pino%2C+et+al%2E%2C+%E2%80%9CXls%2Dr%3A+Self%2Dsupervised+cross%2Dlingual+speech+) [our card](https://md.research.elamur.ai/free/en/papers/babu-2021-xls-r-self-supervised-cross/)
20. Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, et al., “Wavlm: Large-scale self-supervised pre-training for full stack speech processing,” IEEE JSTSP, 2022. [find](https://scholar.google.com/scholar?q=Sanyuan+Chen%2C+Chengyi+Wang%2C+Zhengyang+Chen%2C+Yu+Wu%2C+Shujie+Liu%2C+Zhuo+Chen%2C+Jinyu+Li%2C+Naoyuki+Kanda%2C+Takuya+Yoshioka%2C+Xiong+Xiao%2C+et+al%2E%2C+%E2%80%9CWavlm%3A+Large%2Dscale+self%2Dsupervised+pre%2Dtraining+for+full+stack+)
21. Alex Graves, Abdel-rahman Mohamed, and Geoffrey Hinton, “Speech recognition with deep recurrent neural networks,” in ICASSP. IEEE, 2013, pp. 6645–6649. [find](https://scholar.google.com/scholar?q=Alex+Graves%2C+Abdel%2Drahman+Mohamed%2C+and+Geoffrey+Hinton%2C+%E2%80%9CSpeech+recognition+with+deep+recurrent+neural+networks%2C%E2%80%9D+in+ICASSP%2E+IEEE%2C+2013%2C+pp%2E+6645%E2%80%936649%2E)
22. Kenneth Heafield, “KenLM: Faster and smaller language model queries,” in Proceedings of the Sixth Workshop on Statistical Machine Translation, Edinburgh, Scotland, July 2011, pp. 187–197, ACL. [find](https://scholar.google.com/scholar?q=Kenneth+Heafield%2C+%E2%80%9CKenLM%3A+Faster+and+smaller+language+model+queries%2C%E2%80%9D+in+Proceedings+of+the+Sixth+Workshop+on+Statistical+Machine+Translation%2C+Edinburgh%2C+Scotland%2C+July+2011%2C+pp%2E+187%E2%80%93197%2C+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/heafield-2011-kenlm-faster-smaller-language-model/)
23. Dan Berrebbi, Ronan Collobert, Samy Bengio, Navdeep Jaitly, and Tatiana Likhomanenko, “Continuous pseudo-labeling from the start,” ICLR, 2023. [find](https://scholar.google.com/scholar?q=Dan+Berrebbi%2C+Ronan+Collobert%2C+Samy+Bengio%2C+Navdeep+Jaitly%2C+and+Tatiana+Likhomanenko%2C+%E2%80%9CContinuous+pseudo%2Dlabeling+from+the+start%2C%E2%80%9D+ICLR%2C+2023%2E)
24. Brian Yan, Matthew Wiesner, Ondřej Klejch, Preethi Jyothi, and Shinji Watanabe, “Towards zero-shot code-switched speech recognition,” in ICASSP, 2023. [find](https://scholar.google.com/scholar?q=Brian+Yan%2C+Matthew+Wiesner%2C+Ondr%CC%8Cej+Klejch%2C+Preethi+Jyothi%2C+and+Shinji+Watanabe%2C+%E2%80%9CTowards+zero%2Dshot+code%2Dswitched+speech+recognition%2C%E2%80%9D+in+ICASSP%2C+2023%2E)
25. Rosana Ardila, Megan Branson, Kelly Davis, Michael Henretty, Michael Kohler, Josh Meyer, Reuben Morais, Lindsay Saunders, Francis M. Tyers, and Gregor Weber, “Common voice: A massively-multilingual speech corpus,” 2020. [find](https://scholar.google.com/scholar?q=Rosana+Ardila%2C+Megan+Branson%2C+Kelly+Davis%2C+Michael+Henretty%2C+Michael+Kohler%2C+Josh+Meyer%2C+Reuben+Morais%2C+Lindsay+Saunders%2C+Francis+M%2E+Tyers%2C+and+Gregor+Weber%2C+%E2%80%9CCommon+voice%3A+A+massively%2Dmultilingual+sp) [our card](https://md.research.elamur.ai/free/en/papers/ardila-2019-common-voice-massively-multilingual-speech/)