---
title: "Highly Effective Arabic Diacritization using Sequence to"
description: "Arabic is usually written without short vowels, but text-to-speech and similar applications… 4.49% WER. 29 references, cited by 23 papers here."
authors: ["Hamdy Mubarak", "Ahmed Abdelali", "Hassan Sajjad", "Younes Samih", "Kareem Darwish"]
year: 2019
venue: "Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)"
source: "https://aclanthology.org/N19-1248/"
doi: "10.18653/v1/n19-1248"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/mubarak-2019-highly-effective-arabic-diacritization-sequence/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/mubarak-2019-highly-effective-arabic-diacritization-sequence/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/5/) / Highly Effective Arabic Diacritization using Sequence to Sequence Modeling

# Highly Effective Arabic Diacritization using Sequence to Sequence Modeling

**Authors:** [Hamdy Mubarak](https://md.research.elamur.ai/free/en/authors/hamdy-mubarak/), [Ahmed Abdelali](https://md.research.elamur.ai/free/en/authors/ahmed-abdelali/), [Hassan Sajjad](https://md.research.elamur.ai/free/en/authors/hassan-sajjad/), [Younes Samih](https://md.research.elamur.ai/free/en/authors/younes-samih/) and [Kareem Darwish](https://md.research.elamur.ai/free/en/authors/kareem-darwish/) · **Year:** 2019 · **Venue:** [Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)](https://md.research.elamur.ai/free/en/venues/naacl/) ([NAACL 2019](https://md.research.elamur.ai/free/en/venues/naacl/2019/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/N19-1248/) · [PDF](https://aclanthology.org/N19-1248.pdf) · [DOI 10.18653/v1/n19-1248](https://doi.org/10.18653/v1/n19-1248) · [OpenAlex](https://openalex.org/W2945888757)

**Citations:** 28 citations (OpenAlex, 2026-10-14) · [23](https://md.research.elamur.ai/free/en/papers/mubarak-2019-highly-effective-arabic-diacritization-sequence/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Arabic is usually written without short vowels, but text-to-speech and similar applications need those marks restored. Two kinds of diacritics must be predicted. Core-word diacritics settle which lexical reading is meant, and case endings reflect a word's syntactic function. Earlier diacritizers depended on many hand-crafted morphological and syntactic features, especially for case endings. This paper instead casts restoration as character-level translation using ordinary NMT machinery (OpenNMT). The input is a stream of undiacritized letters. The output is a stream of letter-plus-diacritic units, so source and target always have equal length. Feeding whole sentences to the model produced hallucinated repetitions and lost words. The authors therefore cut each sentence into overlapping fixed-size word windows, so every word is predicted several times, and its final form is chosen by majority vote. They trained on a 4.5M-token diacritized corpus and tested on WikiNews. A 7-word window with voting across an attention LSTM and a Transformer reached 4.49% WER, against 12.25% for the best earlier system. This shows that one feature-free model can recover both diacritic types.

## Key points
- Best result: 4.49% WER and 1.21% DER on the WikiNews test set (about 18.3k words). Microsoft ATKS scored 12.25%, Farasa 12.76%, RDI 15.95%, MADAMIRA 19.02% and MIT 30.50%, so the error rate is 63.3% lower than the previous best.
- Running NMT directly on full sentences performed very poorly: 54.31% WER with word units and 48.31% with character units. Word-level windows also kept suffering from OOV words.
- Window size mattered. Character windows of 3, 5, 7 and 11 words with voting gave 7.79%, 5.49%, 4.77% and 6.40% WER, so 7 words was best.
- The LSTM seq2seq with attention and the Transformer both scored 4.77% WER on their own. Their errors were partly complementary, so voting over both lowered WER to 4.49%.
- Error analysis covered 100 errors of each type. For case endings, the main causes were long-range dependencies (24%), confusion between syntactic functions (22%), the wrong morphological reading (20%) and named entities (16%). For core-word errors, ambiguous words dominated (60%, mostly active vs. passive voice), followed by named entities (32%).

## Interesting ideas
- Each output unit is one letter together with its diacritics, so the decoder produces exactly as many tokens as it reads. This rules out inserted or dropped material and was preferred over BPE, which can break that alignment.
- Sliding overlapping windows show each word in several local contexts. A vote then picks the most frequent prediction, and ties go to the window where the word sits in the middle.
- Two architectures that are equally accurate overall can still make different mistakes, and a simple vote across them helps beyond either one alone.
- The error study sorts sampled errors into linguistic categories with percentages and explicitly blames the fixed window for the long-distance cases. That points straight at the next fix, such as mixing several window sizes.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tokens): Morphology or lemma tagging could be framed as one aligned label per input unit, so the tagger can never invent or lose tokens and each label stays traceable to its source.
- **I2** (tree): Very long sentences could be parsed in overlapping chunks, with head/label choices per token reconciled by vote and a preference for the most centered chunk, keeping each decision explainable as a tally.
- **I3** (learn): This supports building silver data from agreement among several witness parsers, with the perceptron parser and gparse voting per token rather than one being trusted.
- **I4** (learn): Evaluation reports could tag each sampled failure by cause (long-range attachment, named entity, voice ambiguity), so repair-rule work targets the largest bucket.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works

## Cite

Cite as: Hamdy Mubarak, Ahmed Abdelali, Hassan Sajjad, Younes Samih and Kareem Darwish. 2019. Highly Effective Arabic Diacritization using Sequence to Sequence Modeling. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{mubarak2019highly,
  title = {{Highly Effective Arabic Diacritization using Sequence to Sequence Modeling}},
  author = {Hamdy Mubarak, Ahmed Abdelali, Hassan Sajjad, Younes Samih and Kareem Darwish},
  booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)},
  year = {2019},
  doi = {10.18653/v1/n19-1248},
  url = {https://doi.org/10.18653/v1/n19-1248}
}
```

- [← previous Employing the Correspondence of Relations and Connectives to Identify Implicit Discourse Relations via Label Embeddings](https://md.research.elamur.ai/free/en/papers/nguyen-2019-employing-correspondence-relations-connectives-identify/)
- [next → No Army, No Navy: BERT Semi-Supervised Learning of Arabic Dialects](https://md.research.elamur.ai/free/en/papers/zhang-2019-no-army-no-navy-bert/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/5/)

## References

29 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Martı́n Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. 2016. [Tensorflow: a system for large-scale machine learning](https://md.research.elamur.ai/free/en/papers/abadi-2016-tensorflow-system-large-scale-machine/). In OSDI, volume 16, pages 265– 283. [↗](https://doi.org/10.48550/arxiv.1605.08695)
2. Gheith A Abandah, Alex Graves, Balkees Al-Shagoor, Alaa Arabiyat, Fuad Jamour, and Majid Al-Taee. 2015. Automatic diacritization of arabic text using recurrent neural networks. International Journal on Document Analysis and Recognition (IJDAR), 18(2):183–197. [↗](https://doi.org/10.1007/s10032-015-0242-2)
3. Aqil M Azmi and Reham S Almajed. 2015. A survey of automatic arabic diacritization techniques. Natural Language Engineering, 21(03):477–495. [find](https://scholar.google.com/scholar?q=Aqil+M+Azmi+and+Reham+S+Almajed%2E+2015%2E+A+survey+of+automatic+arabic+diacritization+techniques%2E+Natural+Language+Engineering%2C+21%2803%29%3A477%E2%80%93495%2E)
4. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. [Neural machine translation by jointly learning to align and translate](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/). arXiv:1409.0473. arXiv preprint Mohamed Bebah, Chennoufi Amine, Mazroui Azzeddine, and Lakhouaja Abdelhak. 2014. Hybrid approaches for automatic vowelization of arabic texts. arXiv preprint arXiv:1410.2646. [↗](https://arxiv.org/abs/1409.0473)
5. Yonatan Belinkov and James Glass. 2015. [Arabic diacritization with recurrent neural networks](https://md.research.elamur.ai/free/en/papers/belinkov-2015-arabic-diacritization-recurrent-neural-networks/). In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2281– 2285, Lisbon, Portugal.
6. Denny Britz, Anna Goldie, Thang Luong, and Quoc Le. 2017. [Massive Exploration of Neural Machine Translation Architectures](https://md.research.elamur.ai/free/en/papers/britz-2017-massive-exploration-neural-machine-translation/). ArXiv e-prints.
7. Tim Buckwalter. 2004. Buckwalter arabic morphological analyzer version 2.0. LDC catalog number LDC2004L02, ISBN 1-58563-324-0. [↗](https://doi.org/10.35111/050q-5r95)
8. Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. [Learning phrase representations using rnn encoder-decoder for statistical machine translation](https://md.research.elamur.ai/free/en/papers/cho-2014-learning-phrase-representations-rnn-encoder/). arXiv preprint arXiv:1406.1078. [↗](https://arxiv.org/abs/1406.1078)
9. Kareem Darwish, Hamdy Mubarak, and Ahmed Abdelali. 2017. [Arabic diacritization: Stats, rules, and hacks](https://md.research.elamur.ai/free/en/papers/darwish-2017-arabic-diacritization-stats-rules-hacks/). In Proceedings of the Third Arabic Natural Language Processing Workshop, pages 9–17.
10. Moustafa Elshafei, Husni Al-Muhtaseb, and Mansour Alghamdi. 2006. Statistical methods for automatic diacritization of arabic text. In The Saudi 18th National Computer Conference. Riyadh, volume 18, pages 301–306. [find](https://scholar.google.com/scholar?q=Moustafa+Elshafei%2C+Husni+Al%2DMuhtaseb%2C+and+Mansour+Alghamdi%2E+2006%2E+Statistical+methods+for+automatic+diacritization+of+arabic+text%2E+In+The+Saudi+18th+National+Computer+Conference%2E+Riyadh%2C+volume+18%2C+pa)
11. Ya’akov Gal. 2002. [An hmm approach to vowel restoration in arabic and hebrew](https://md.research.elamur.ai/free/en/papers/gal-2002-hmm-approach-vowel-restoration-arabic/). In Proceedings of the ACL-02 workshop on Computational approaches to Semitic languages, pages 1–7. Association for Computational Linguistics.
12. Nizar Habash and Owen Rambow. 2005. [Arabic tokenization, part-of-speech tagging and morphological disambiguation in one fell swoop](https://md.research.elamur.ai/free/en/papers/habash-2005-arabic-tokenization-part-speech-tagging/). In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, pages 573–580. Association for Computational Linguistics.
13. Nizar Habash and Owen Rambow. 2007. [Arabic diacritization through full morphological tagging](https://md.research.elamur.ai/free/en/papers/habash-2007-arabic-diacritization-through-full-morphological/). In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Companion Volume, Short Papers, pages 53–56. Association for Computational Linguistics. [↗](https://doi.org/10.3115/1614108.1614122)
14. G. Klein, Y. Kim, Y. Deng, J. Senellart, and A. M. Rush. 2017. [OpenNMT: Open-Source Toolkit for Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/klein-2017-opennmt-open-source-toolkit-neural/). ArXiv e-prints.
15. Oleksii Kuchaiev, Boris Ginsburg, Igor Gitman, Vitaly Lavrukhin, Carl Case, and Paulius Micikevicius. 2018. Openseq2seq: extensible toolkit for distributed and mixed precision training of sequence-to-sequence models. arXiv preprint arXiv:1805.10387. [↗](https://arxiv.org/abs/1805.10387)
16. Aya S Metwally, Mohsen A Rashwan, and Amir F Atiya. 2016. A multi-layered approach for arabic text diacritization. In Cloud Computing and Big Data Analysis (ICCCBDA), 2016 IEEE International Conference on, pages 389–393. IEEE. [find](https://scholar.google.com/scholar?q=Aya+S+Metwally%2C+Mohsen+A+Rashwan%2C+and+Amir+F+Atiya%2E+2016%2E+A+multi%2Dlayered+approach+for+arabic+text+diacritization%2E+In+Cloud+Computing+and+Big+Data+Analysis+%28ICCCBDA%29%2C+2016+IEEE+International+Conferenc)
17. Rani Nelken and Stuart M Shieber. 2005. [Arabic diacritization using weighted finite-state transducers](https://md.research.elamur.ai/free/en/papers/nelken-2005-arabic-diacritization-weighted-finite-state/). In Proceedings of the ACL Workshop on Computational Approaches to Semitic Languages, pages 79– 86. Association for Computational Linguistics.
18. Torsten Zesch Osama Hamed. 2017. A Survey and Comparative Study of Arabic Diacritization Tools. JLCL, 32(1):27–47. [↗](https://doi.org/10.21248/jlcl.32.2017.213)
19. Arfath Pasha, Mohamed Al-Badrashiny, Mona Diab, Ahmed El Kholy, Ramy Eskander, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan M Roth. 2014. [Madamira: A fast, comprehensive tool for morphological analysis and disambiguation of arabic](https://md.research.elamur.ai/free/en/papers/pasha-2014-madamira-fast-comprehensive-tool-morphological/). In LREC-2014, Reykjavik, Iceland.
20. Mohsen Rashwan, Ahmad Al Sallab, M. Raafat, and Ahmed Rafea. 2015. Deep learning framework with confused sub-set resolution architecture for automatic arabic diacritization. In IEEE Transactions on Audio, Speech, and Language Processing, pages 505–516. [↗](https://doi.org/10.1109/taslp.2015.2395255)
21. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. 2017. [Attention is All you Need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems 30, pages 5998–6008. Curran Associates, Inc.
22. Dimitra Vergyri and Katrin Kirchhoff. 2004. [Automatic diacritization of arabic for acoustic modeling in speech recognition](https://md.research.elamur.ai/free/en/papers/vergyri-2004-automatic-diacritization-arabic-acoustic-modeling/). In Proceedings of the workshop on computational approaches to Arabic script-based languages, COLING’04, pages 66– 73, Geneva, Switzerland. Association for Computational Linguistics. [↗](https://doi.org/10.21236/ada457846)
23. Ian H Witten, Eibe Frank, Mark A Hall, and Christopher J Pal. 2016. Data Mining: Practical machine learning tools and techniques. Morgan Kaufmann. [find](https://scholar.google.com/scholar?q=Ian+H+Witten%2C+Eibe+Frank%2C+Mark+A+Hall%2C+and+Christopher+J+Pal%2E+2016%2E+Data+Mining%3A+Practical+machine+learning+tools+and+techniques%2E+Morgan+Kaufmann%2E)
24. Haonan Yu, Jiang Wang, Zhiheng Huang, Yi Yang, and Wei Xu. 2016. Video paragraph captioning using hierarchical recurrent neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4584–4593. [find](https://scholar.google.com/scholar?q=Haonan+Yu%2C+Jiang+Wang%2C+Zhiheng+Huang%2C+Yi+Yang%2C+and+Wei+Xu%2E+2016%2E+Video+paragraph+captioning+using+hierarchical+recurrent+neural+networks%2E+In+Proceedings+of+the+IEEE+conference+on+computer+vision+and+p)
25. Imed Zitouni, Jeffrey S Sorensen, and Ruhi Sarikaya. 2006. [Maximum entropy based restoration of arabic diacritics](https://md.research.elamur.ai/free/en/papers/zitouni-2006-maximum-entropy-based-restoration-arabic/). In Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the Association for Computational Linguistics, pages 577–584. Association for Computational Linguistics.
26. Mohsen AA Rashwan, Mohamed ASAA Al-Badrashiny, Mohamed Attia, Sherif M Abdou, and Ahmed Rafea. 2011. A stochastic arabic diacritizer based on a hybrid of factorized and unfactorized textual features. IEEE Transactions on Audio, Speech, and Language Processing, 19(1):166–175. [↗](https://doi.org/10.1109/tasl.2010.2045240)
27. Ahmed Said, Mohamed El-Sharqwi, Achraf Chalabi, and Eslam Kamal. 2013. A hybrid approach for arabic diacritization. In Natural Language Processing and Information Systems, pages 53–64, Berlin, Heidelberg. Springer Berlin Heidelberg. [find](https://scholar.google.com/scholar?q=Ahmed+Said%2C+Mohamed+El%2DSharqwi%2C+Achraf+Chalabi%2C+and+Eslam+Kamal%2E+2013%2E+A+hybrid+approach+for+arabic+diacritization%2E+In+Natural+Language+Processing+and+Information+Systems%2C+pages+53%E2%80%9364%2C+Berlin%2C+Heidelb)
28. Rico Sennrich. 2017. [How grammatical is character-level neural machine translation? assessing mt quality with contrastive translation pairs](https://md.research.elamur.ai/free/en/papers/sennrich-2017-how-grammatical-is-character-level/). In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers.
29. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. [Neural Machine Translation of Rare Words with Subword Units](https://md.research.elamur.ai/free/en/papers/sennrich-2016-neural-machine-translation-rare-words/). In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715– 1725, Berlin, Germany. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/p16-1162)

*Updated 1 November 2026*