---
title: "Gulf Arabic Diacritization: Guidelines, Initial Dataset, and Results"
description: "This paper builds a Gulf Arabic diacritization resource tied to the Dubai pronunciation. 6.7% WER. 19 references, cited by 2 papers here."
authors: ["Nouf Alabbasi", "Mohamed Al-Badrashiny", "Maryam Aldahmani", "Ahmed AlDhanhani", "Abdullah Saleh Alhashmi", "Fawaghy Ahmed Alhashmi", "Khalid Al Hashemi", "Rama Emad Alkhobbi", "Shamma T Al Maazmi", "Mohammed Ali Alyafeai", "Mariam M Alzaabi", "Mohamed Saqer Alzaabi", "Fatma Khalid Badri", "Kareem Darwish", "Ehab Mansour Diab", "Muhammad Morsy Elmallah", "Amira Ayman Elnashar", "Ashraf Hatim Elneima", "MHD Tameem Kabbani", "Nour Rabih", "Ahmad Saad", "Ammar Mamoun Sousou"]
year: 2022
venue: "Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP)"
source: "https://aclanthology.org/2022.wanlp-1.33/"
doi: "10.18653/v1/2022.wanlp-1.33"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/alabbasi-2022-gulf-arabic-diacritization-guidelines-initial/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/alabbasi-2022-gulf-arabic-diacritization-guidelines-initial/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/8/) / Gulf Arabic Diacritization: Guidelines, Initial Dataset, and Results

# Gulf Arabic Diacritization: Guidelines, Initial Dataset, and Results

**Authors:** Nouf Alabbasi, [Mohamed Al-Badrashiny](https://md.research.elamur.ai/free/en/authors/mohamed-al-badrashiny/), Maryam Aldahmani, Ahmed AlDhanhani, Abdullah Saleh Alhashmi, Fawaghy Ahmed Alhashmi, Khalid Al Hashemi, Rama Emad Alkhobbi, Shamma T Al Maazmi, [Mohammed Ali Alyafeai](https://md.research.elamur.ai/free/en/authors/mohammed-alyafeai/), Mariam M Alzaabi, Mohamed Saqer Alzaabi, Fatma Khalid Badri, [Kareem Darwish](https://md.research.elamur.ai/free/en/authors/kareem-darwish/), Ehab Mansour Diab, [Muhammad Morsy Elmallah](https://md.research.elamur.ai/free/en/authors/muhammad-morsy-elmallah/), Amira Ayman Elnashar, [Ashraf Hatim Elneima](https://md.research.elamur.ai/free/en/authors/ashraf-hatim-elneima/), MHD Tameem Kabbani, [Nour Rabih](https://md.research.elamur.ai/free/en/authors/nour-rabih/), Ahmad Saad and Ammar Mamoun Sousou · **Year:** 2022 · **Venue:** [Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP)](https://md.research.elamur.ai/free/en/venues/wanlp/) ([WANLP 2022](https://md.research.elamur.ai/free/en/venues/wanlp/2022/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2022.wanlp-1.33/) · [PDF](https://aclanthology.org/2022.wanlp-1.33.pdf) · [DOI 10.18653/v1/2022.wanlp-1.33](https://doi.org/10.18653/v1/2022.wanlp-1.33) · [OpenAlex](https://openalex.org/W4385572982)

**Citations:** 1 citations (OpenAlex, 2026-10-14) · [2](https://md.research.elamur.ai/free/en/papers/alabbasi-2022-gulf-arabic-diacritization-guidelines-initial/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Arabic writing usually leaves out short vowels, so applications such as text-to-speech and language learning need a step that restores these diacritic marks. Most diacritization resources cover Modern Standard Arabic and Classical Arabic. Dialects have little consistent annotated data, and pronunciation can differ even between nearby towns. This paper builds a Gulf Arabic diacritization resource tied to the Dubai pronunciation. The authors selected 19,850 words from the CODA-normalized, morphologically tagged Emirati portion of the Gumar corpus of internet novels. In a workshop, two senior computational linguists and 15 native Emirati speakers wrote explicit diacritization rules and then annotated the text in stages: automatic pre-filling from a list of frequent words, correction by groups, and expert review. Two baselines were trained on the data. A word-level HMM reached 6.7% WER, and a character seq2seq model with windowed voting reached 8.6%. The guidelines and dataset offer a reproducible starting point for scaling up dialectal diacritization.

## Key points
- Dataset: 2,953 sentences / 19,850 words from the Gumar corpus (Emirati subset, CODAfied), split into 2,700 sentences / 18,174 words for training and 253 sentences / 1,676 words for testing.
- Annotation pipeline:
  - The 1,300 most frequent words were diacritized out of context and applied automatically.
  - Four groups of native speakers, each handling about 5,000 words, then corrected the text and resolved disagreements together.
  - A senior Gulf-dialect linguist performed two review rounds.
- Guidelines fix many conventions:
  - Word-final sukun is assumed and dropped, and default diacritics are omitted.
  - Ambiguity is broken by a fixed priority: kasra before fatha before dammah, and plural before dual.
  - Only the existing MSA diacritic marks are used.
- HMM baseline: a KenLM word trigram model with A* lattice search, 6.7% WER. About 73.3% of its errors come from OOV words.
- Seq2seq baseline: a re-implementation of the Mubarak et al. (2019) setup in OpenNMT, 8.6% WER.
  - The model stacks two LSTM layers that run in one direction only, each with a state size of 512, and applies a dropout rate of 0.3.
  - Input is limited to 5-word windows, with voting across windows.
  - Hallucinations account for 34.6% of its errors.

## Interesting ideas
- Speed up annotation by labeling the most frequent word types once, without context, and projecting those labels onto the corpus so humans only fix exceptions; frequent types cover much of the text, so effort shrinks a lot.
- Write ambiguity tie-breakers into the guidelines as an explicit priority order, justified by frequency or set inclusion (plural is preferred over dual because dual is a subset of plural).
- Settle unclear cases with an operational probe: add a prefix to the word and see how the leading letter behaves (the "w test"), instead of relying on annotator intuition.
- Reduce seq2seq hallucination by feeding only short local windows and voting over the overlapping predictions for each token.
- Classify errors by cause (OOV versus hallucination) to judge whether the bottleneck is coverage, model choice, or data size.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Silver data for lemmas, senses or verb classes could start from a type-level lexicon pass over frequent words, with witness parsers or LLM teachers reviewing only the contextual deviations.
- **I2** (tokens): The tagger and morphology could carry documented default rankings for ambiguous analyses (e.g., number or tense ambiguity), so every fallback choice is explainable rather than arbitrary.
- **I3** (tree): Treebank conversion and annotation guidelines could encode substitution or insertion tests as executable checks for disputed attachments or categories, making disagreements resolvable by procedure.
- **I4** (back): When regenerating text from the meaning graph or translating, local chunked generation with cross-window voting could catch invented or garbled words before they reach the output.
- **I5** (learn): Evaluation reports could break failures down by cause (unknown words, spurious structure, etc.) to decide whether to add data, lexicon entries, or repair rules.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [New York University Abu Dhabi](https://md.research.elamur.ai/free/en/universities/new-york-university-abu-dhabi/) — University, Abu Dhabi, United Arab Emirates · 124 works · 4,709 citations
- [King's College London](https://md.research.elamur.ai/free/en/universities/king-s-college-london/) — University, London, United Kingdom · 46 works · 1,714 citations
- [University of Sharjah](https://md.research.elamur.ai/free/en/universities/university-of-sharjah/) — University, Sharjah, United Arab Emirates · 10 works · 209 citations
- [Khalifa University of Science and Technology](https://md.research.elamur.ai/free/en/universities/khalifa-university-of-science-and-technology/) — University, Abu Dhabi, United Arab Emirates · 3 works · 196 citations
- [United Arab Emirates University](https://md.research.elamur.ai/free/en/universities/united-arab-emirates-university/) — University, Al Ain, United Arab Emirates · 4 works · 89 citations
- [American University of Sharjah](https://md.research.elamur.ai/free/en/universities/american-university-of-sharjah/) — University, Sharjah, United Arab Emirates · 4 works · 81 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 3,066 works

## Cite

Cite as: Nouf Alabbasi, Mohamed Al-Badrashiny, Maryam Aldahmani, Ahmed AlDhanhani, Abdullah Saleh Alhashmi, Fawaghy Ahmed Alhashmi, Khalid Al Hashemi, Rama Emad Alkhobbi, Shamma T Al Maazmi, Mohammed Ali Alyafeai, Mariam M Alzaabi, Mohamed Saqer Alzaabi, Fatma Khalid Badri, Kareem Darwish, Ehab Mansour Diab, Muhammad Morsy Elmallah, Amira Ayman Elnashar, Ashraf Hatim Elneima, MHD Tameem Kabbani, Nour Rabih, Ahmad Saad and Ammar Mamoun Sousou. 2022. Gulf Arabic Diacritization: Guidelines, Initial Dataset, and Results. Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{alabbasi2022gulf,
  title = {{Gulf Arabic Diacritization: Guidelines, Initial Dataset, and Results}},
  author = {Nouf Alabbasi, Mohamed Al-Badrashiny, Maryam Aldahmani, Ahmed AlDhanhani, Abdullah Saleh Alhashmi, Fawaghy Ahmed Alhashmi, Khalid {Al Hashemi}, Rama Emad Alkhobbi, Shamma T {Al Maazmi}, Mohammed Ali Alyafeai, Mariam M Alzaabi, Mohamed Saqer Alzaabi, Fatma Khalid Badri, Kareem Darwish, Ehab Mansour Diab, Muhammad Morsy Elmallah, Amira Ayman Elnashar, Ashraf Hatim Elneima, MHD Tameem Kabbani, Nour Rabih, Ahmad Saad and Ammar Mamoun Sousou},
  booktitle = {Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP)},
  year = {2022},
  doi = {10.18653/v1/2022.wanlp-1.33},
  url = {https://doi.org/10.18653/v1/2022.wanlp-1.33}
}
```

- [← previous End-to-End Neural Bridging Resolution](https://md.research.elamur.ai/free/en/papers/kobayashi-2022-end-end-neural-bridging-resolution/)
- [next → I2C at SemEval-2022 Task 6: Intended Sarcasm in English using Deep Learning Techniques](https://md.research.elamur.ai/free/en/papers/monterde-2022-i2c-semeval-2022-task-6/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/8/)

## References

19 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Gheith A Abandah, Alex Graves, Balkees Al-Shagoor, Alaa Arabiyat, Fuad Jamour, and Majid Al-Taee. 2015. Automatic diacritization of arabic text using recurrent neural networks. International Journal on Document Analysis and Recognition (IJDAR), 18(2):183–197. [↗](https://doi.org/10.1007/s10032-015-0242-2)
2. Ahmed Abdelali, Mohammed Attia, Younes Samih, Kareem Darwish, and Hamdy Mubarak. 2018. [Diacritization of maghrebi arabic sub-dialects](https://md.research.elamur.ai/free/en/papers/abdelali-2018-diacritization-maghrebi-arabic-sub-dialects/). arXiv preprint arXiv:1810.06619. [↗](https://arxiv.org/abs/1810.06619)
3. Yonatan Belinkov and James Glass. 2015. [Arabic diacritization with recurrent neural networks](https://md.research.elamur.ai/free/en/papers/belinkov-2015-arabic-diacritization-recurrent-neural-networks/). In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2281– 2285, Lisbon, Portugal.
4. Kareem Darwish, Ahmed Abdelali, Hamdy Mubarak, Younes Samih, and Mohammed Attia. 2018. Diacritization of moroccan and tunisian arabic dialects: A crf approach. OSACT, 3:62. [find](https://scholar.google.com/scholar?q=Kareem+Darwish%2C+Ahmed+Abdelali%2C+Hamdy+Mubarak%2C+Younes+Samih%2C+and+Mohammed+Attia%2E+2018%2E+Diacritization+of+moroccan+and+tunisian+arabic+dialects%3A+A+crf+approach%2E+OSACT%2C+3%3A62%2E)
5. Kareem Darwish, Hamdy Mubarak, and Ahmed Abdelali. 2017. [Arabic diacritization: Stats, rules, and hacks](https://md.research.elamur.ai/free/en/papers/darwish-2017-arabic-diacritization-stats-rules-hacks/). In Proceedings of the Third Arabic Natural Language Processing Workshop, pages 9–17.
6. Ali Fadel, Ibraheem Tuffaha, Mahmoud Al-Ayyoub, et al. 2019. [Arabic text diacritization using deep neural networks](https://md.research.elamur.ai/free/en/papers/fadel-2019-arabic-text-diacritization-using-deep/). In 2019 2nd international conference on computer applications & information security (IC-CAIS), pages 1–7. IEEE.
7. Ya’akov Gal. 2002. [An hmm approach to vowel restoration in arabic and hebrew](https://md.research.elamur.ai/free/en/papers/gal-2002-hmm-approach-vowel-restoration-arabic/). In Proceedings of the ACL-02 workshop on Computational approaches to Semitic languages, pages 1–7. Association for Computational Linguistics.
8. Nizar Habash, Mona Diab, and Owen Rambow. 2012. [Conventional orthography for dialectal Arabic](https://md.research.elamur.ai/free/en/papers/habash-2012-conventional-orthography-dialectal-arabic/). In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), Istanbul, Turkey. European Language Resources Association (ELRA).
9. Mustafa Jarrar, Nizar Habash, Faeq Alrimawi, Diyam Akra, and Nasser Zalmout. 2017. Curras: an annotated corpus for the palestinian arabic dialect. Language Resources and Evaluation, 51(3):745–775. [↗](https://doi.org/10.1007/s10579-016-9370-7)
10. Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander Rush. 2017. [OpenNMT: Open-source toolkit for neural machine translation](https://md.research.elamur.ai/free/en/papers/klein-2017-opennmt-open-source-toolkit-neural/). In Proceedings of ACL 2017, System Demonstrations, pages 67–72, Vancouver, Canada. Association for Computational Linguistics.
11. Mohammed Maamouri, Ann Bies, Tim Buckwalter, and Wigdan Mekki. 2004. The penn arabic treebank: building a large-scale annotated arabic corpus. In NEMLAR Conference on Arabic Language Resources and Tools, pages 102—-109. [find](https://scholar.google.com/scholar?q=Mohammed+Maamouri%2C+Ann+Bies%2C+Tim+Buckwalter%2C+and+Wigdan+Mekki%2E+2004%2E+The+penn+arabic+treebank%3A+building+a+large%2Dscale+annotated+arabic+corpus%2E+In+NEMLAR+Conference+on+Arabic+Language+Resources+and+Too)
12. Hamdy Mubarak, Ahmed Abdelali, Kareem Darwish, Mohamed Eldesouki, Younes Samih, and Hassan Sajjad. 2019. [A system for diacritizing four varieties of arabic](https://md.research.elamur.ai/free/en/papers/mubarak-2019-system-diacritizing-four-varieties-arabic/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstrations, pages 217–222.
13. Rani Nelken and Stuart M Shieber. 2005. [Arabic diacritization using weighted finite-state transducers](https://md.research.elamur.ai/free/en/papers/nelken-2005-arabic-diacritization-weighted-finite-state/). In Proceedings of the ACL Workshop on Computational Approaches to Semitic Languages, pages 79–86. Association for Computational Linguistics.
14. Mohsen Rashwan, Ahmad Al Sallab, M. Raafat, and Ahmed Rafea. 2015. Deep learning framework with confused sub-set resolution architecture for automatic arabic diacritization. In IEEE Transactions on Audio, Speech, and Language Processing, pages 505–516. [↗](https://doi.org/10.1109/taslp.2015.2395255)
15. Nasser Zalmout and Nizar Habash. 2020. [Joint diacritization, lemmatization, normalization, and fine-grained morphological tagging](https://md.research.elamur.ai/free/en/papers/zalmout-2020-joint-diacritization-lemmatization-normalization-fine/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8297–8307.
16. Taha Zerrouki and Amar Balla. 2017. Tashkeela: Novel corpus of arabic vocalized texts, data for auto-diacritization systems. Data in brief, 11:147–151. [find](https://scholar.google.com/scholar?q=Taha+Zerrouki+and+Amar+Balla%2E+2017%2E+Tashkeela%3A+Novel+corpus+of+arabic+vocalized+texts%2C+data+for+auto%2Ddiacritization+systems%2E+Data+in+brief%2C+11%3A147%E2%80%93151%2E)
17. Imed Zitouni, Jeffrey S Sorensen, and Ruhi Sarikaya. 2006. [Maximum entropy based restoration of arabic diacritics](https://md.research.elamur.ai/free/en/papers/zitouni-2006-maximum-entropy-based-restoration-arabic/). In Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics, pages 577–584. Association for Computational Linguistics.
18. Salam Khalifa, Nizar Habash, Fadhl Eryani, Ossama Obeid, Dana Abdulrahim, and Meera Al Kaabi. 2018. [A morphologically annotated corpus of emirati arabic](https://md.research.elamur.ai/free/en/papers/khalifa-2018-morphologically-annotated-corpus-emirati-arabic/). In Proceedings of the eleventh international conference on language resources and evaluation (LREC 2018).
19. Salam Khalifa, Sara Hassan, and Nizar Habash. 2017. [A morphological analyzer for gulf arabic verbs](https://md.research.elamur.ai/free/en/papers/khalifa-2017-morphological-analyzer-gulf-arabic-verbs/). In Proceedings of the Third Arabic Natural Language Processing Workshop, pages 35–45.

*Updated 2 November 2026*