---
title: "LAMAD: A Linguistic Attentional Model for Arabic Text Diacritization"
description: "Written Arabic usually omits diacritics, the short-vowel and related marks that fix a word's pronunciation… 29 references, cited by 1 paper here."
authors: ["Raeed Al-Sabri", "Jianliang Gao"]
year: 2021
venue: "Findings of the Association for Computational Linguistics: EMNLP 2021"
source: "https://aclanthology.org/2021.findings-emnlp.317/"
doi: "10.18653/v1/2021.findings-emnlp.317"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/sabri-2021-lamad-linguistic-attentional-model-arabic/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/sabri-2021-lamad-linguistic-attentional-model-arabic/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/8/) / LAMAD: A Linguistic Attentional Model for Arabic Text Diacritization

# LAMAD: A Linguistic Attentional Model for Arabic Text Diacritization

**Authors:** Raeed Al-Sabri and Jianliang Gao · **Year:** 2021 · **Venue:** [Findings of the Association for Computational Linguistics: EMNLP 2021](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2021](https://md.research.elamur.ai/free/en/venues/findings/2021/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2021.findings-emnlp.317/) · [PDF](https://aclanthology.org/2021.findings-emnlp.317.pdf) · [DOI 10.18653/v1/2021.findings-emnlp.317](https://doi.org/10.18653/v1/2021.findings-emnlp.317) · [OpenAlex](https://openalex.org/W3213351479)

**Citations:** 4 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/sabri-2021-lamad-linguistic-attentional-model-arabic/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Written Arabic usually omits diacritics, the short-vowel and related marks that fix a word's pronunciation and meaning. Undiacritized text is highly ambiguous, and this hurts downstream tasks such as machine translation and speech synthesis. Most neural diacritizers work only on characters. The authors argue that this discards useful linguistic information such as part of speech, gender, number and stem.

LAMAD addresses this by attaching word-level and contextual linguistic features to every character. These features are embedded, encoded with a Bi-LSTM, and weighted by an attention layer that learns which features matter most for each prediction. The model was tested on three diacritized corpora of different sizes and genres: the Holy Quran, Sahih Al-Bukhary and Tashkeela. Ablations over feature combinations were run on the Quran data. LAMAD beat seven baselines, both neural and rule/statistical, on both DER and WER on all three corpora. The gains were largest on the small Quran corpus. The work shows that explicit morphological and syntactic features still help a character-level neural tagger.

## Key points
- **Best results:** the best feature set was CHAR-PRIOR-SEG-POS-STEM (surrounding characters, diacritic prior, position in segment, POS, stem).
  - Quran: DER 1.85 / WER 6.76
  - Sahih Al-Bukhary: DER 1.71 / WER 5.61
  - Tashkeela: DER 2.13 / WER 7.31
- **Feature ablations:** characters alone gave DER 9.22 on Quran. Removing the character context from a rich feature set made things much worse (DER 16.56). Linguistic features and character context are therefore complementary.
- **Baselines:** RNN-BNG, BiLSTM-CRF, BiLSTM, A-TCN, Multi-components, a rule-based hybrid system, and a BPE sub-word method (Hifny). On Quran, the strongest baseline reached DER 4.92 (BiLSTM).
- **Corpus sizes:** Quran has 6,236 lines and about 78K words. Tashkeela has 55,000 lines and about 2.65M words.
- **Preprocessing:** lines with under 80% diacritized characters were discarded. Compound diacritics were normalized so that Shaddah comes first.
- **Error analysis:** the authors sampled 100 erroneous words per corpus.
  - About 88 of those words had only one wrong diacritic.
  - Most errors fell at the end of the word (case endings), and most fell on nouns.
  - Excluding case endings, DER drops to 1.06–1.61.

## Interesting ideas
- Word-level properties (POS, stem, gender, number, named entity) are copied onto each character, so a character-level model receives lexical and syntactic context without switching to word units
- A binary "prior" feature, taken from the training set, marks whether a character can accept a diacritic at all, which restricts the label space before learning
- Metrics are reported both with and without the hardest, syntax-driven part (case endings) and with and without irrelevant characters, which separates local errors from structural ones
- The ablation table includes a variant that drops the raw character context and collapses, which acts as a negative control showing that the features alone are not enough
- Error analysis by position in the word and by POS on a random sample shows that remaining errors cluster at word endings, which are governed by syntactic role

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tokens): the en tagger and lemmatizer could feed word-level morphological features into character- or suffix-level decisions such as spelling variants and inflection choice, while staying explainable.
- **I2** (tokens): the tagger could precompute, per word form or suffix, the set of tags ever observed and use it as an explicit admissibility mask that can be logged.
- **I3** (learn): evaluations could report LAS or frame accuracy with and without the known-hard phenomena, so improvements in easy cases do not hide stagnation on difficult ones.
- **I4** (learn): when adding hand-crafted features to gparse or the perceptron, also train a variant without the base lexical signal to prove the features complement rather than replace it.
- **I5** (tree): this supports feeding UD dependency relations back into morphology decisions, and using an error breakdown by relation type to find where the parser hurts downstream stages.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Central South University](https://md.research.elamur.ai/free/en/universities/central-south-university/) — University, Changsha, China · 15 works · 582 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works

## Cite

Cite as: Raeed Al-Sabri and Jianliang Gao. 2021. LAMAD: A Linguistic Attentional Model for Arabic Text Diacritization. Findings of the Association for Computational Linguistics: EMNLP 2021.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{alsabri2021lamad,
  title = {{LAMAD: A Linguistic Attentional Model for Arabic Text Diacritization}},
  author = {Raeed Al-Sabri and Jianliang Gao},
  booktitle = {Findings of the Association for Computational Linguistics: EMNLP 2021},
  year = {2021},
  doi = {10.18653/v1/2021.findings-emnlp.317},
  url = {https://doi.org/10.18653/v1/2021.findings-emnlp.317}
}
```

- [← previous Drivers of English Syntactic Change in the Canadian Parliament](https://md.research.elamur.ai/free/en/papers/hou-2021-drivers-english-syntactic-change-canadian/)
- [next → Measuring the ‘I don't know' Problem through the Lens of Gricean Quantity](https://md.research.elamur.ai/free/en/papers/khayrallah-2021-measuring-i-don-t-know/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/8/)

## References

29 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Gheith A. Abandah, Alex Graves, Balkees Al-Shagoor, Alaa Arabiyat, Fuad Jamour, and Majid Al-Taee. 2015. Automatic diacritization of Arabic text using recurrent neural networks. International Journal on Document Analysis and Recognition, 18(2):183– 197. [↗](https://doi.org/10.1007/s10032-015-0242-2)
2. Hamza Abbad and Shengwu Xiong. 2020. Multi-components System for Automatic Arabic Diacritization. LNCS (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 12035 LNCS(April):341–355. [find](https://scholar.google.com/scholar?q=Hamza+Abbad+and+Shengwu+Xiong%2E+2020%2E+Multi%2Dcomponents+System+for+Automatic+Arabic+Diacritization%2E+LNCS+%28including+subseries+Lecture+Notes+in+Artificial+Intelligence+and+Lecture+Notes+in+Bioinformatics)
3. Sa’ed Abed, Mohammad Alshayeji, and Sari Sultan. 2019. Diacritics Effect on Arabic Speech Recognition. Arabian Journal for Science and Engineering, 44(11):9043–9056. [↗](https://doi.org/10.1007/s13369-019-04024-0)
4. Amine Chennoufi and Azzeddine Mazroui. 2017. Morphological, syntactic and diacritics rules for automatic diacritization of arabic sentences. J. King Saud Univ. Comput. Inf. Sci., 29(2):156–163. [↗](https://doi.org/10.1016/j.jksuci.2016.06.004)
5. Kareem Darwish, Ahmed Abdelali, Hamdy Mubarak, and Mohamed Eldesouki. 2021. [Arabic diacritic recovery using a feature-rich bilstm model](https://md.research.elamur.ai/free/en/papers/darwish-2020-arabic-diacritic-recovery-using-feature/). ACM Trans. Asian Low-Resour. Lang. Inf. Process., 20(2). [↗](https://doi.org/10.1145/3434235)
6. Kareem Darwish and Wei Gao. 2014. [Simple effective microblog named entity recognition: Arabic as an example](https://md.research.elamur.ai/free/en/papers/darwish-2014-simple-effective-microblog-named-entity/). In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 2513–2517, Reykjavik, Iceland. European Language Resources Association (ELRA).
7. Dia AbuZeina and Taqieddin Mostafa Abdalbaset. 2019. Exploring the Performance of Tagging for the Classical and the Modern Standard Arabic. Advances in Fuzzy Systems, 2019. [↗](https://doi.org/10.1155/2019/6254649)
8. Kareem Darwish and Hamdy Mubarak. 2016. [Farasa: A new fast and accurate Arabic word segmenter](https://md.research.elamur.ai/free/en/papers/darwish-2016-farasa-new-fast-accurate-arabic/). In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC’16), pages 1070–1074, Portorož, Slovenia. European Language Resources Association (ELRA).
9. Abdulmohsen Al-Thubaity, Atheer Alkhalifa, Abdulrahman Almuhareb, and Waleed Alsanie. 2020. Arabic diacritization using bidirectional long short-term memory neural networks with conditional random fields. IEEE Access, 8:154984–154996. [↗](https://doi.org/10.1109/access.2020.3018885)
10. Ali Fadel, Ibraheem Tuffaha, Mahmoud Al-Ayyoub, et al. 2019a. [Arabic text diacritization using deep neural networks](https://md.research.elamur.ai/free/en/papers/fadel-2019-arabic-text-diacritization-using-deep/). In 2019 2nd international conference on computer applications & information security (ICCAIS), pages 1–7. IEEE.
11. Abdulmohsen Al-Thubaity, Atheer Alkhalifa, Abdulrahman Almuhareb, and Waleed Alsanie. 2020. Arabic Diacritization Using Bidirectional Long Short-Term Memory Neural Networks with Conditional Random Fields. IEEE Access, 8:154984–154996. [↗](https://doi.org/10.1109/access.2020.3018885)
12. Ali Fadel, Ibraheem Tuffaha, Bara’ Al-Jawarneh, and Mahmoud Al-Ayyoub. 2019b. [Neural Arabic text diacritization: State of the art results and a novel approach for machine translation](https://md.research.elamur.ai/free/en/papers/fadel-2019-neural-arabic-text-diacritization-state/). In Proceedings of the 6th Workshop on Asian Translation, pages 215– 225, Hong Kong, China. Association for Computational Linguistics. [↗](https://doi.org/10.60692/wrdx8-hg164)
13. Sameh Alansary. 2018. Alserag: An automatic diacritization system for Arabic. Studies in Computational Intelligence, 740:523–543. [↗](https://doi.org/10.1007/978-3-319-67056-0_25)
14. Sawsan Alqahtani, Ajay Mishra, and Mona Diab. 2019. Efficient convolutional neural networks for diacritic restoration. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1442–1448, Hong Kong, China. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Sawsan+Alqahtani%2C+Ajay+Mishra%2C+and+Mona+Diab%2E+2019%2E+Efficient+convolutional+neural+networks+for+diacritic+restoration%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Pr)
15. Sawsan Alqahtani, Ajay Mishra, and Mona Diab. 2020. [A multitask learning approach for diacritic restoration](https://md.research.elamur.ai/free/en/papers/alqahtani-2020-multitask-learning-approach-diacritic-restoration/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8238–8247, Online. Association for Computational Linguistics.
16. Mohamed Seghir Hadj Ameur, Farid Meziane, and Ahmed Guessoum. 2020. Arabic Machine Translation: A survey of the latest trends and challenges. Computer Science Review, 38:100305. [↗](https://doi.org/10.1016/j.cosrev.2020.100305)
17. Tim Buckwalter. 2002. Buckwalter arabic morphological analyzer version 1.0. Linguistic Data Consortium, University of Pennsylvania. [↗](https://doi.org/10.35111/7vzm-mb15)
18. Ismail Hadjir, Mohamed Abbache, and Fatma Zohra Belkredim. 2019. An approach for arabic diacritization. In International Conference on Applications of Natural Language to Information Systems, pages 337–344. Springer. [find](https://scholar.google.com/scholar?q=Ismail+Hadjir%2C+Mohamed+Abbache%2C+and+Fatma+Zohra+Belkredim%2E+2019%2E+An+approach+for+arabic+diacritization%2E+In+International+Conference+on+Applications+of+Natural+Language+to+Information+Systems%2C+pages+33)
19. Osama Hamed and Torsten Zesch. 2017. A Survey and Comparative Study of Arabic Diacritization Tools. J. Lang. Technol. Comput. Linguistics, 32(1):27–47. [↗](https://doi.org/10.21248/jlcl.32.2017.213)
20. Yasser Hifny. 2019. Open vocabulary arabic diacritics restoration. IEEE Signal Processing Letters, 26(10):1421–1425. [↗](https://doi.org/10.1109/lsp.2019.2933721)
21. Tinghuai Ma, Raeed Al-Sabri, Lejun Zhang, Bockarie Marah, and Najla Al-Nabhan. 2020. The Impact of Weighting Schemes and Stemming Process on Topic Modeling of Arabic Long and Short Texts. ACM Trans. Asian Low-Resour. Lang. Inf. Process., 19(6). [↗](https://doi.org/10.1145/3405843)
22. Hamdy Mubarak, Ahmed Abdelali, Hassan Sajjad, Younes Samih, and Kareem Darwish. 2019. [Highly effective Arabic diacritization using sequence to sequence modeling](https://md.research.elamur.ai/free/en/papers/mubarak-2019-highly-effective-arabic-diacritization-sequence/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2390–2395, Minneapolis, Minnesota. Association for Computational Linguistics.
23. Jakub Náplava, Milan Straka, Pavel Straňák, and Jan Hajič. 2018. Diacritics restoration using neural networks. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Jakub+N%C3%A1plava%2C+Milan+Straka%2C+Pavel+Stran%CC%8C%C3%A1k%2C+and+Jan+Hajic%CC%8C%2E+2018%2E+Diacritics+restoration+using+neural+networks%2E+In+Proceedings+of+the+Eleventh+International+Conference+on+Language+Resources+and+Evalu)
24. Alexis Amid Neme and Sébastien Paumier. 2020. [Restoring Arabic vowels through omission-tolerant dictionary lookup](https://md.research.elamur.ai/free/en/papers/neme-2019-restoring-arabic-vowels-through-omission/). Language Resources and Evaluation, 54(2):487–551. [↗](https://doi.org/10.1007/s10579-019-09464-6)
25. Iroro Orife. 2018. Attentive Sequence-to-Sequence Learning for Diacritic Restoration of YorùBá Language Text. In Interspeech 2018, pages 2848–2852, ISCA. ISCA. [find](https://scholar.google.com/scholar?q=Iroro+Orife%2E+2018%2E+Attentive+Sequence%2Dto%2DSequence+Learning+for+Diacritic+Restoration+of+Yor%C3%B9B%C3%A1+Language+Text%2E+In+Interspeech+2018%2C+pages+2848%E2%80%932852%2C+ISCA%2E+ISCA%2E)
26. Arfath Pasha, Mohamed Al-Badrashiny, Mona Diab, Ahmed El Kholy, Ramy Eskander, Nizar Habash, Manoj Pooleery, Owen Rambow, and Ryan Roth. 2014. [MADAMIRA: A fast, comprehensive tool for morphological analysis and disambiguation of Arabic](https://md.research.elamur.ai/free/en/papers/pasha-2014-madamira-fast-comprehensive-tool-morphological/). In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 1094–1101, Reykjavik, Iceland. European Language Resources Association (ELRA).
27. Mohsen A.A. Rashwan, Ahmad A. Al Sallab, Hazem M. Raafat, and Ahmed Rafea. 2015. Deep learning framework with confused sub-set resolution architecture for automatic arabic diacritization. IEEE Transactions on Audio, Speech and Language Processing, 23(3):505–516. [↗](https://doi.org/10.1109/taslp.2015.2395255)
28. Nasser Zalmout and Nizar Habash. 2017. [Don’t throw those morphological analyzers away just yet: Neural morphological disambiguation for Arabic](https://md.research.elamur.ai/free/en/papers/zalmout-2017-don-t-throw-those-morphological/). In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 704– 713, Copenhagen, Denmark. Association for Computational Linguistics.
29. Yuan Zhang, Chengtao Li, Regina Barzilay, and Kareem Darwish. 2015. Randomized greedy inference for joint segmentation, POS tagging and dependency parsing. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 42–52, Denver, Colorado. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Yuan+Zhang%2C+Chengtao+Li%2C+Regina+Barzilay%2C+and+Kareem+Darwish%2E+2015%2E+Randomized+greedy+inference+for+joint+segmentation%2C+POS+tagging+and+dependency+parsing%2E+In+Proceedings+of+the+2015+Conference+of+the)

*Updated 1 November 2026*