---
title: "Almieyar: A Culturally Grounded Benchmark — Ghahroodi 2026"
description: "Arabic speech recognition has mostly been built and tested on Modern Standard Arabic."
authors: ["Omid Ghahroodi", "Anas Madkoor", "Dima Faris Al Saudi", "Fagr Tahir", "Malak Annan", "Talha shahid javad allah rakha", "Omar Al-Busaidi", "Zineb El Kahla", "Iheb Zouari", "Essa Ahmed Abou Jabal", "Ahmed Ezzat", "Hind AL-Merekhi", "Aisha Hamad M A Al-Naimi", "Hadi Wazni", "Bushra Alnajjar", "Omar Amin", "Haya Al-Thani", "Houssam Eddine-Othman Lachemat", "Marwa Elwakedy", "Sundus Abdulmalik Al Nahari", "Elahe Zahiri", "Osamah Sarraj", "Raghad Mousa", "Mckeen Assi", "Ahd Al Jumah", "Heyam Salman", "Alhanouf Abdulraqib", "Sara Benoumhani", "Alia Hamwi", "Ayaat Al-Yasseri", "Rim Ibrahim Ghazal", "Lamia Ben hiba", "Mohamed Eltabakh", "Fatima Al-Raisi", "Yassine El Kheir", "Mohammed Abdulrahman", "Hamdy Mubarak", "Ayah Hashem", "Lefkir Meriem", "Ehsaneddin Asgari"]
year: 2026
venue: "arXiv:2609.35564"
source: "https://arxiv.org/abs/2609.35564"
doi: "10.48550/arxiv.2609.35564"
arxiv: "2609.35564"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ghahroodi-2026-almieyar-culturally-grounded-benchmark-multi/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ghahroodi-2026-almieyar-culturally-grounded-benchmark-multi/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/) / Almieyar: A Culturally Grounded Benchmark for Multi-Dialect Arabic Speech Recognition

# Almieyar: A Culturally Grounded Benchmark for Multi-Dialect Arabic Speech Recognition

**Authors:** Omid Ghahroodi, Anas Madkoor, Dima Faris Al Saudi, Fagr Tahir, Malak Annan, Talha shahid javad allah rakha, Omar Al-Busaidi, Zineb El Kahla, Iheb Zouari, Essa Ahmed Abou Jabal, Ahmed Ezzat, Hind AL-Merekhi et al. (40 authors) · **Year:** 2026 · **Venue:** arXiv:2609.35564 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2609.35564) · [PDF](https://arxiv.org/pdf/2609.35564) · [DOI 10.48550/arxiv.2609.35564](https://doi.org/10.48550/arxiv.2609.35564) · [arXiv 2609.35564](https://arxiv.org/abs/2609.35564) · [OpenAlex](https://openalex.org/W7214764980)

**Citations:** 0 citations (openalex-snapshot · 2026-10-14)

## Summary
Arabic speech recognition has mostly been built and tested on Modern Standard Arabic. The dialects people speak at home and in daily life get little coverage. Existing dialect test sets also tend to come from broadcast or YouTube audio, which large models may already have seen during training. Almieyar addresses both problems with a newly recorded evaluation set covering 17 dialects from six families. Community coordinators picked culturally meaningful images across 10 themes. Native speakers then described each image following five fixed prompts, which gave about 50 minutes per dialect and 13.7 hours in total. Speakers corrected the transcripts and coordinators checked them. Twelve ASR systems were tested zero-shot. GPT-4o-transcribe came first with roughly 35% WER, ahead of Voxtral-Mini-4B (41.1%), Fanar-STT-LF (45.9%) and Whisper-Large-v3 (49.5%). The wav2vec2-based models stayed around 79–86%. No system wins in every dialect family, and CER often shows a different picture from WER. The resource offers a test bed that resists data leakage, and it is the first to publish ASR evaluation results for Ahwazi Arabic.

## Key points
- Coverage: 17 dialects in six families. The audio splits as Gulf 4.0 h, Levantine 3.0 h, Maghrebi 3.8 h, Iraqi+Ahwazi 1.6 h, Egyptian 0.7 h and Sudanese 0.7 h. Earlier benchmarks such as NADI 2025 and Casablanca separate at most eight varieties.
- Collection ran through a Telegram pipeline. An automatic transcript was used only as a draft for speakers to correct, and coordinators then confirmed the dialect was authentic and flagged code-switching.
- Scoring used WER, CER and MER after the same normalization for every system. Diacritics and tatweel were stripped, Alef/Hamza variants unified, and Ta Marbuta and Alef Maqsura normalized.
- Best overall was GPT-4o-transcribe at 34.9% WER and 16.5% CER (the abstract rounds this to 35.0%). It also had the lowest Iraqi/Ahwazi score at 28.3% WER. In Egyptian, however, SeamlessM4T-v2 (43.9%) beat GPT-4o-transcribe (49.7%).
- Dialects within a family can differ a lot: up to 16.8 WER points in Maghrebi and 12.8 in Gulf. The wav2vec2 models show large gaps between WER and CER, with errors concentrated in weak letters and short function morphemes.

## Interesting ideas
- Build the test set from speech elicited just for the benchmark rather than harvested from the web, so no model can have trained on it and scores reflect real generalisation. — For Mova (learn): write fresh held-out texts, such as new fables or word problems, that neither the LLM teacher tiers nor the witness parsers could have memorised, and use them as a contamination control.
- Prompt every contributor with the same fixed set of angles (context, subjects, background, mood, interpretation), which keeps content comparable across speakers while still varied. — For Mova (answer): generate questions about a text along fixed dimensions (who/what, setting, mood, interpretation) so each reader is probed at the same depth on every story.
- Report a coarse and a fine metric side by side, because the gap between them shows what kind of error dominates (here, near-miss words versus wholly wrong ones). — For Mova (tree): always publish LAS together with UAS and per-label accuracy, so attachment errors and labelling errors in gparse or the perceptron parser can be told apart.
- Break results down below the group average, since pooled family scores can hide a spread of many points between members. — For Mova (tree): evaluate per treebank and per genre instead of pooling, especially after UD dialect conversion, so a weak source is not masked by strong ones.
- Let a machine draft serve only as a starting point for human correction, with a second reviewer giving final approval. — For Mova (learn): show silver parses agreed by witness parsers to reviewers as drafts, and promote them to gold only after verification, keeping a record of who approved each tree.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{ghahroodi2026almieyar,
  title = {{Almieyar: A Culturally Grounded Benchmark for Multi-Dialect Arabic Speech Recognition}},
  author = {Ghahroodi, Omid and Madkoor, Anas and Saudi, Dima Faris Al and Tahir, Fagr and Annan, Malak and rakha, Talha shahid javad allah and Al-Busaidi, Omar and Kahla, Zineb El and Zouari, Iheb and Jabal, Essa Ahmed Abou and Ezzat, Ahmed and AL-Merekhi, Hind and Al-Naimi, Aisha Hamad M A and Wazni, Hadi and Alnajjar, Bushra and Amin, Omar and Al-Thani, Haya and Lachemat, Houssam Eddine-Othman and Elwakedy, Marwa and Nahari, Sundus Abdulmalik Al and Zahiri, Elahe and Sarraj, Osamah and Mousa, Raghad and Assi, Mckeen and Jumah, Ahd Al and Salman, Heyam and Abdulraqib, Alhanouf and Benoumhani, Sara and Hamwi, Alia and Al-Yasseri, Ayaat and Ghazal, Rim Ibrahim and hiba, Lamia Ben and Eltabakh, Mohamed and Al-Raisi, Fatima and Kheir, Yassine El and Abdulrahman, Mohammed and Mubarak, Hamdy and Hashem, Ayah and Meriem, Lefkir and Asgari, Ehsaneddin},
  year = {2026},
  doi = {10.48550/arxiv.2609.35564},
  eprint = {2609.35564},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2609.35564}
}
```

- [← previous Alma Brodersen, The End of the Psalter. Psalms 146–150 in the Masoretic Text, the Dead Sea Scrolls, and the Septuagint (Beihefte zur Zeitschrift fürdie alttestamentliche Wissenschaft 505; Berlin: De Gruyter 2017). ISBN 978-3-11-053476-4](https://md.research.elamur.ai/free/en/papers/watson-2026-alma-brodersen-end-psalter-psalms/)
- [next → AmazoniaNLP: A Survey of Extreme Low-Resource Languages in the Peruvian-Brazilian Amazon](https://md.research.elamur.ai/free/en/papers/zevallos-2026-amazonianlp-survey-extreme-low-resource/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/)

## References

16 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Arabic speech processing shared task,” in Proceedings of The Third Arabic Natural Language Processing Conference: Shared Tasks, Suzhou, China, Nov. 2025, pp. 720–733, Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Arabic+speech+processing+shared+task%2C%E2%80%9D+in+Proceedings+of+The+Third+Arabic+Natural+Language+Processing+Conference%3A+Shared+Tasks%2C+Suzhou%2C+China%2C+Nov%2E+2025%2C+pp%2E+720%E2%80%93733%2C+Association+for+Computational+Lin)
2. [7] Bashar Talafha, Karima Kadaoui, Samar Mohamed Magdy, Mariem Habiboullah, Chafei Mohamed Chafei, Ahmed Oumar El-Shangiti, Hiba Zayed, Mohamedou cheikh tourad, Rahaf Alhamouri, Rwaa Assi, Aisha Alraeesi, Hour Mohamed, Fakhraddin Alwajih, Abdelrahman Mohamed, Abdellah El Mekki, El Moatez Billah Nagoudi, Benelhadj Djelloul Mama Saadia, Hamzah A. Alsayadi, Walid Al-Dhabyani, Sara Shatnawi, Yasir Ech-Chammakhy, Amal Makouar, Yousra Berrachedi, Mustafa Jarrar, Shady Shehata, Ismail Berrada, and Muhammad Abdul-Mageed, “Casablanca: Data and models for multidialectal Arabic speech recognition,” 2024. [find](https://scholar.google.com/scholar?q=%5B7%5D+Bashar+Talafha%2C+Karima+Kadaoui%2C+Samar+Mohamed+Magdy%2C+Mariem+Habiboullah%2C+Chafei+Mohamed+Chafei%2C+Ahmed+Oumar+El%2DShangiti%2C+Hiba+Zayed%2C+Mohamedou+cheikh+tourad%2C+Rahaf+Alhamouri%2C+Rwaa+Assi%2C+Aisha+Alra)
3. [8] Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine Mcleavey, and Ilya Sutskever, “Robust speech recognition via large-scale weak supervision,” in Proceedings of the 40th International Conference on Machine Learning. 23–29 Jul 2023, vol. 202 of Proceedings of Machine Learning Research, pp. 28492–28518, PMLR. [find](https://scholar.google.com/scholar?q=%5B8%5D+Alec+Radford%2C+Jong+Wook+Kim%2C+Tao+Xu%2C+Greg+Brockman%2C+Christine+Mcleavey%2C+and+Ilya+Sutskever%2C+%E2%80%9CRobust+speech+recognition+via+large%2Dscale+weak+supervision%2C%E2%80%9D+in+Proceedings+of+the+40th+International+C) [our card](https://md.research.elamur.ai/free/en/papers/radford-2022-robust-speech-recognition-large-scale/)
4. [9] Mahmoud Salhab, Marwan Elghitany, Shameed Sait, Syed Sibghat Ullah, Mohammad Abusheikh, and Hasan Abusheikh, “Advancing arabic speech recognition through large-scale weakly supervised learning,” 2025. [find](https://scholar.google.com/scholar?q=%5B9%5D+Mahmoud+Salhab%2C+Marwan+Elghitany%2C+Shameed+Sait%2C+Syed+Sibghat+Ullah%2C+Mohammad+Abusheikh%2C+and+Hasan+Abusheikh%2C+%E2%80%9CAdvancing+arabic+speech+recognition+through+large%2Dscale+weakly+supervised+learning%2C%E2%80%9D+2) [our card](https://md.research.elamur.ai/free/en/papers/salhab-2025-advancing-arabic-speech-recognition-through/)
5. [10] Ahmed Ali, Stephan Vogel, and Steve Renals, “Speech recognition challenge in the wild: Arabic mgb-3,” in 2017 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). [find](https://scholar.google.com/scholar?q=%5B10%5D+Ahmed+Ali%2C+Stephan+Vogel%2C+and+Steve+Renals%2C+%E2%80%9CSpeech+recognition+challenge+in+the+wild%3A+Arabic+mgb%2D3%2C%E2%80%9D+in+2017+IEEE+Automatic+Speech+Recognition+and+Understanding+Workshop+%28ASRU%29%2E) [our card](https://md.research.elamur.ai/free/en/papers/ali-2017-speech-recognition-challenge-wild-arabic/)
6. [11] Ahmed Ali, Suwon Shon, Younes Samih, Hamdy Mubarak, Ahmed Abdelali, James Glass, Steve Renals, and Khalid Choukri, “The mgb-5 challenge: Recognition and dialect identification of dialectal arabic speech,” in 2019 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), 2019, pp. 1026–1033. [find](https://scholar.google.com/scholar?q=%5B11%5D+Ahmed+Ali%2C+Suwon+Shon%2C+Younes+Samih%2C+Hamdy+Mubarak%2C+Ahmed+Abdelali%2C+James+Glass%2C+Steve+Renals%2C+and+Khalid+Choukri%2C+%E2%80%9CThe+mgb%2D5+challenge%3A+Recognition+and+dialect+identification+of+dialectal+arabic)
7. [12] Rosana Ardila, Megan Branson, Kelly Davis, Michael Kohler, Josh Meyer, Michael Henretty, Reuben Morais, Lindsay Saunders, Francis Tyers, and Gregor Weber, “Common voice: A massively-multilingual speech corpus,” in Proceedings of the Twelfth Language Resources and Evaluation Conference (LREC 2020), Marseille, France, 2020, pp. 4218–4222, European Language Resources Association. [find](https://scholar.google.com/scholar?q=%5B12%5D+Rosana+Ardila%2C+Megan+Branson%2C+Kelly+Davis%2C+Michael+Kohler%2C+Josh+Meyer%2C+Michael+Henretty%2C+Reuben+Morais%2C+Lindsay+Saunders%2C+Francis+Tyers%2C+and+Gregor+Weber%2C+%E2%80%9CCommon+voice%3A+A+massively%2Dmultilingual+) [our card](https://md.research.elamur.ai/free/en/papers/ardila-2019-common-voice-massively-multilingual-speech/)
8. [13] Alexis Conneau, Min Ma, Simran Khanuja, Yu Zhang, Vera Axelrod, Siddharth Dalmia, Jason Riesa, Clara Rivera, and Ankur Bapna, “FLEURS: Few-shot learning evaluation of universal representations of speech,” in 2022 IEEE Spoken Language Technology Workshop (SLT). 2022, pp. 798–805, IEEE. [find](https://scholar.google.com/scholar?q=%5B13%5D+Alexis+Conneau%2C+Min+Ma%2C+Simran+Khanuja%2C+Yu+Zhang%2C+Vera+Axelrod%2C+Siddharth+Dalmia%2C+Jason+Riesa%2C+Clara+Rivera%2C+and+Ankur+Bapna%2C+%E2%80%9CFLEURS%3A+Few%2Dshot+learning+evaluation+of+universal+representations+of) [our card](https://md.research.elamur.ai/free/en/papers/conneau-2022-fleurs-few-shot-learning-evaluation/)
9. [14] Sadeen Alharbi, Areeb Alowisheq, Zoltán Tüske, Kareem Darwish, Abdullah Alrajeh, Abdulmajeed Alrowithi, Aljawharah Bin Tamran, Asma Ibrahim, Raghad Aloraini, Raneem Alnajim, Ranya Alkahtani, Renad Almuasaad, Sara Alrasheed, Shaykhah Alsubaie, and Yaser Alonaizan, “SADA: Saudi audio dataset for Arabic,” in ICASSP 2024 – 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2024, IEEE. [find](https://scholar.google.com/scholar?q=%5B14%5D+Sadeen+Alharbi%2C+Areeb+Alowisheq%2C+Zolt%C3%A1n+T%C3%BCske%2C+Kareem+Darwish%2C+Abdullah+Alrajeh%2C+Abdulmajeed+Alrowithi%2C+Aljawharah+Bin+Tamran%2C+Asma+Ibrahim%2C+Raghad+Aloraini%2C+Raneem+Alnajim%2C+Ranya+Alkahtani%2C+Rena)
10. [15] Ayman Mansour, “Doing more with less: Data augmentation for sudanese dialect automatic speech recognition,” 2026. [find](https://scholar.google.com/scholar?q=%5B15%5D+Ayman+Mansour%2C+%E2%80%9CDoing+more+with+less%3A+Data+augmentation+for+sudanese+dialect+automatic+speech+recognition%2C%E2%80%9D+2026%2E)
11. [16] Pardis Sadat Zahraei and Ehsaneddin Asgari, “I am aligned, but with whom? MENA values benchmark for evaluating cultural alignment and multilingual bias in LLMs,” 2025. [find](https://scholar.google.com/scholar?q=%5B16%5D+Pardis+Sadat+Zahraei+and+Ehsaneddin+Asgari%2C+%E2%80%9CI+am+aligned%2C+but+with+whom%3F+MENA+values+benchmark+for+evaluating+cultural+alignment+and+multilingual+bias+in+LLMs%2C%E2%80%9D+2025%2E)
12. [17] Mohammad Mahdi Abootorabi, Omid Ghahroodi, Anas Madkoor, Marzia Nouri, Doratossadat Dastgheib, and Ehsaneddin Asgari, “Almieyar-oryx-BloomBench: A bilingual multimodal benchmark for cognitively informed evaluation of vision-language models,” in Findings of the Association for Computational Linguistics: ACL 2026, Maria Liakata, Viviane P. Moreira, Jiajun Zhang, and David Jurgens, Eds., San Diego, California, United States, July 2026, pp. 28404–28436, Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=%5B17%5D+Mohammad+Mahdi+Abootorabi%2C+Omid+Ghahroodi%2C+Anas+Madkoor%2C+Marzia+Nouri%2C+Doratossadat+Dastgheib%2C+and+Ehsaneddin+Asgari%2C+%E2%80%9CAlmieyar%2Doryx%2DBloomBench%3A+A+bilingual+multimodal+benchmark+for+cognitively+i)
13. [18] Mistral AI, “Voxtral,” 2025. [find](https://scholar.google.com/scholar?q=%5B18%5D+Mistral+AI%2C+%E2%80%9CVoxtral%2C%E2%80%9D+2025%2E)
14. [19] Seamless Communication, Loïc Barrault, Yu-An Chung, Mariano Coria Meglioli, David Dale, Ning Dong, Mark Duppenthaler, Paul-Ambroise Duquenne, Brian Ellis, Hady Elsahar, Justin Haaheim, John Hoffman, Min-Jae Hwang, Hirofumi Inaguma, Christopher Klaiber, Ilia Kulikov, Pengwei Li, Daniel Licht, Jean Maillard, Ruslan Mavlyutov, Alice Rakotoarison, Kaushik Ram Sadagopan, Abinesh Ramakrishnan, Tuan Tran, Guillaume Wenzek, Yilin Yang, Ethan Ye, Ivan Evtimov, Pierre Fernandez, Cynthia Gao, Prangthip Hansanti, Elahe Kalbassi, Amanda Kallet, Artyom Kozhevnikov, Gabriel Mejia, Robin San Roman, Christophe Touret, Corinne Wong, Carleigh Wood, Bokai Yu, Pierre Andrews, Can Balioglu, Peng-Jen Chen, Marta R. Costa-jussà, Maha Elbayad, Hongyu Gong, Francisco Guzmán, Kevin Heffernan, Somya Jain, Justine Kao, Ann Lee, Xutai Ma, Alex Mourachko, Benjamin Peloquin, Juan Pino, Sravya Popuri, Christophe Ropers, Safiyyah Saleem, Holger Schwenk, Anna Sun, Paden Tomasello, Changhan Wang, Jeff Wang, Skyler Wang, and Mary Williamson, “Seamless: Multilingual expressive and streaming speech translation,” arXiv preprint arXiv:2312.05187, 2023. [↗](https://arxiv.org/abs/2312.05187) [our card](https://md.research.elamur.ai/free/en/papers/communication-2023-seamless-multilingual-expressive-streaming-speech/)
15. [20] Nat Jeffries, Evan King, Manjunath Kudlur, Guy Nicholson, James Wang, and Pete Warden, “Moonshine: Speech recognition for live transcription and voice commands,” 2024. [find](https://scholar.google.com/scholar?q=%5B20%5D+Nat+Jeffries%2C+Evan+King%2C+Manjunath+Kudlur%2C+Guy+Nicholson%2C+James+Wang%2C+and+Pete+Warden%2C+%E2%80%9CMoonshine%3A+Speech+recognition+for+live+transcription+and+voice+commands%2C%E2%80%9D+2024%2E)
16. [21] Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli, “wav2vec 2.0: A framework for self-supervised learning of speech representations,” Advances in neural information processing systems, vol. 33, pp. 12449–12460, 2020. [find](https://scholar.google.com/scholar?q=%5B21%5D+Alexei+Baevski%2C+Yuhao+Zhou%2C+Abdelrahman+Mohamed%2C+and+Michael+Auli%2C+%E2%80%9Cwav2vec+2%2E0%3A+A+framework+for+self%2Dsupervised+learning+of+speech+representations%2C%E2%80%9D+Advances+in+neural+information+processing+sys) [our card](https://md.research.elamur.ai/free/en/papers/baevski-2020-wav2vec-2-0-framework-self/)