---
title: "Arabic Text Diacritization In The Age Of — Skiredj 2024"
description: "Arabic is usually written without the short-vowel marks (diacritics) that fix pronunciation and meaning."
authors: ["Abderrahman Skiredj", "Ismail Berrada"]
year: 2024
venue: "arXiv:2401.04848"
source: "https://arxiv.org/abs/2401.04848"
doi: "10.48550/arxiv.2401.04848"
arxiv: "2401.04848"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/skiredj-2024-arabic-text-diacritization-age-transfer/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/skiredj-2024-arabic-text-diacritization-age-transfer/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/4/) / Arabic Text Diacritization In The Age Of Transfer Learning: Token Classification Is All You Need

# Arabic Text Diacritization In The Age Of Transfer Learning: Token Classification Is All You Need

**Authors:** [Abderrahman Skiredj](https://md.research.elamur.ai/free/en/authors/abderrahman-skiredj/), [Ismail Berrada](https://md.research.elamur.ai/free/en/authors/ismail-berrada/) · **Year:** 2024 · **Venue:** arXiv:2401.04848 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2401.04848) · [PDF](https://arxiv.org/pdf/2401.04848) · [DOI 10.48550/arxiv.2401.04848](https://doi.org/10.48550/arxiv.2401.04848) · [arXiv 2401.04848](https://arxiv.org/abs/2401.04848) · [OpenAlex](https://openalex.org/W6922660591)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [3](https://md.research.elamur.ai/free/en/papers/skiredj-2024-arabic-text-diacritization-age-transfer/cited-by/) in our library, of the 140884 papers whose bibliographies we read

## Summary
Arabic is usually written without the short-vowel marks (diacritics) that fix pronunciation and meaning. This leaves many words ambiguous and hurts downstream uses such as TTS, translation, POS tagging and search. The authors propose PTCAD (Pre-FineTuned Token Classification for Arabic Diacritization), which reframes Arabic Text Diacritization (ATD) as a token classification problem on top of a pre-trained BERT-like encoder. Training has two stages. First, a pre-finetuning stage trains the model jointly on several related tasks, all cast in masked-language-model form: adaptation to Classical Arabic text, POS tagging, segmentation and diacritization. Second, a finetuning stage trains a token classification head to predict diacritics.

PTCAD is evaluated on two cleaned Tashkeela benchmarks, from Abbad and Xiong (2020) and from Fadel et al. (2019a). On both, it lowers WER by about 20% relative to the previous state of the art. It also clearly beats GPT-4. An ablation shows that the multi-task pre-finetuning stage drives much of the gain. The work matters because it shows that a modest encoder plus careful staged training can beat a general LLM on a fine-grained linguistic labeling task.

## Key points
- PTCAD has two phases. Phase 1 is multi-task pre-finetuning, with CA text, POS tagging, segmentation and diacritization all expressed as MLM tasks. Phase 2 finetunes ATD as token classification.
- Two benchmarks were used: the cleaned Tashkeela of Abbad and Xiong (2020) and the Fadel et al. (2019a) version (about 55k sentences, 2.3M words). The earlier best result on the latter was AlKhamissi et al. (2020) at DER 2.09% / WER 5.08%.
- PTCAD cuts WER by roughly 20% versus prior SOTA on both datasets.
- GPT-4 reached only about 20% DER and 30% WER on the Fadel et al. (2019a) benchmark.
- The authors name several limitations:
  - the sequence-length limit of the encoder;
  - inconsistent diacritics inside a word, because each token is labeled independently;
  - weak sentence-level coherence.

  Their proposed fixes include an LSTM layer above the classification head, generative models such as LLama2, and better benchmarks.

## Interesting ideas
- Before training the target head, warm the encoder on several neighbouring linguistic tasks, all recast into one shared MLM format, so one model absorbs them without separate architectures; the ablation backs this up. — For Mova (tree): gparse could first be pre-finetuned on POS, morphology and lemma prediction in masked form, then trained as a dependency parser, checking the gain with an ablation.
- Predicting each sub-label independently can stitch together two individually valid readings into a hybrid that matches neither, so outputs need a consistency layer or joint decoding. — For Mova (tokens): the English tagger and morphology should choose a whole lexicon-licensed analysis per word, not separate feature slots, so impossible feature combinations cannot occur.
- Correct words do not guarantee a coherent sentence, so evaluation should also flag outputs that are locally right but globally contradictory. — For Mova (world): consistency checks on the text world, such as conflicting states or impossible event order, could catch errors that per-token scores miss.
- Score a general LLM on the same gold test as the specialized model instead of assuming it is competent; here GPT-4 trailed badly. — For Mova (learn): each LLM teacher tier should be measured against gold UD data per phenomenon before its outputs are allowed into silver data.
- Benchmarks with partial annotation, noisy gold labels or short isolated sentences distort the scores; full annotation and longer connected passages give a truer picture. — For Mova (answer): fable and school-test QA sets should be fully annotated and use whole connected texts, so the readers are tested on long-range consistency.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

Topics: [Part-of-speech tagging](https://md.research.elamur.ai/free/en/topics/part-of-speech-tagging/) · [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{skiredj2024arabic,
  title = {{Arabic Text Diacritization In The Age Of Transfer Learning: Token Classification Is All You Need}},
  author = {Skiredj, Abderrahman and Berrada, Ismail},
  year = {2024},
  doi = {10.48550/arxiv.2401.04848},
  eprint = {2401.04848},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2401.04848}
}
```

- [← previous Annotation Scheme for English Argument Structure Constructions Treebank](https://md.research.elamur.ai/free/en/papers/bank-2024-sung-annotation-scheme-english-argument-structure/)
- [next → Can ChatGPT capture swearing nuances? Evidence from translating Arabic oaths](https://md.research.elamur.ai/free/en/papers/shormani-2024-chatgpt-capture-swearing-nuances-evidence/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/4/)

## References

56 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Abandah, G. A., Khedher, M. Z., Abdel-Majeed, M. R., Mansour, H. M., Hulliel, S. F., and Bisharat, L. M. (2022a). Classifying and diacritizing arabic poems using deep recurrent neural networks. Journal of King Saud University-Computer and Information Sciences, 34(6):3775–3788. [find](https://scholar.google.com/scholar?q=Abandah%2C+G%2E+A%2E%2C+Khedher%2C+M%2E+Z%2E%2C+Abdel%2DMajeed%2C+M%2E+R%2E%2C+Mansour%2C+H%2E+M%2E%2C+Hulliel%2C+S%2E+F%2E%2C+and+Bisharat%2C+L%2E+M%2E+%282022a%29%2E+Classifying+and+diacritizing+arabic+poems+using+deep+recurrent+neural+networks%2E+Journa)
2. Abandah, G. A., Suyyagh, A. E., and Abdel-Majeed, M. R. (2022b). Transfer learning and multi-phase training for accurate diacritization of arabic poetry. Journal of King Saud University-Computer and Information Sciences, 34(6):3744–3757. [find](https://scholar.google.com/scholar?q=Abandah%2C+G%2E+A%2E%2C+Suyyagh%2C+A%2E+E%2E%2C+and+Abdel%2DMajeed%2C+M%2E+R%2E+%282022b%29%2E+Transfer+learning+and+multi%2Dphase+training+for+accurate+diacritization+of+arabic+poetry%2E+Journal+of+King+Saud+University%2DComputer+and+I)
3. Abbad, H. and Xiong, S. (2020). Multi-components system for automatic arabic diacritization. In Advances in Information Retrieval: 42nd European Conference on IR Research, ECIR 2020, Lisbon, Portugal, April 14–17, 2020, Proceedings, Part I 42, pages 341–355. Springer. [find](https://scholar.google.com/scholar?q=Abbad%2C+H%2E+and+Xiong%2C+S%2E+%282020%29%2E+Multi%2Dcomponents+system+for+automatic+arabic+diacritization%2E+In+Advances+in+Information+Retrieval%3A+42nd+European+Conference+on+IR+Research%2C+ECIR+2020%2C+Lisbon%2C+Portugal%2C)
4. Abbad, H. and Xiong, S. (2021). Simple extensible deep learning model for automatic arabic diacritization. Transactions on Asian and Low-Resource Language Information Processing, 21(2):1–16. [find](https://scholar.google.com/scholar?q=Abbad%2C+H%2E+and+Xiong%2C+S%2E+%282021%29%2E+Simple+extensible+deep+learning+model+for+automatic+arabic+diacritization%2E+Transactions+on+Asian+and+Low%2DResource+Language+Information+Processing%2C+21%282%29%3A1%E2%80%9316%2E)
5. Abdul-Mageed, M., Diab, M., and Rambow, O. (2012). Arabic diacritization using conditional random fields. Computational Linguistics, 38(3):585–612. [find](https://scholar.google.com/scholar?q=Abdul%2DMageed%2C+M%2E%2C+Diab%2C+M%2E%2C+and+Rambow%2C+O%2E+%282012%29%2E+Arabic+diacritization+using+conditional+random+fields%2E+Computational+Linguistics%2C+38%283%29%3A585%E2%80%93612%2E)
6. Abdul-Mageed, M., Elmadany, A., and Nagoudi, E. M. B. (2021). Arbert & marbert: Deep bidirectional transformers for arabic. [find](https://scholar.google.com/scholar?q=Abdul%2DMageed%2C+M%2E%2C+Elmadany%2C+A%2E%2C+and+Nagoudi%2C+E%2E+M%2E+B%2E+%282021%29%2E+Arbert+%26+marbert%3A+Deep+bidirectional+transformers+for+arabic%2E) [our card](https://md.research.elamur.ai/free/en/papers/abdul-2021-mageed-arbert-marbert-deep-bidirectional-transformers/)
7. Al, A., Rahman, M., Islam, M., and et al. (2022). Natural language processing in customer service: A systematic review. arXiv preprint arXiv:2212.09523. [↗](https://arxiv.org/abs/2212.09523)
8. Al-Hajj, S. and Al-Rawi, K. (2004). Arabic diacritization: A survey and some improvements. Journal of the American Society for Information Science and Technology, 55(6):495–505. [find](https://scholar.google.com/scholar?q=Al%2DHajj%2C+S%2E+and+Al%2DRawi%2C+K%2E+%282004%29%2E+Arabic+diacritization%3A+A+survey+and+some+improvements%2E+Journal+of+the+American+Society+for+Information+Science+and+Technology%2C+55%286%29%3A495%E2%80%93505%2E)
9. Al-Sabri, R. and Gao, J. (2021). Lamad: A linguistic attentional model for arabic text diacritization. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 3757–3764. [find](https://scholar.google.com/scholar?q=Al%2DSabri%2C+R%2E+and+Gao%2C+J%2E+%282021%29%2E+Lamad%3A+A+linguistic+attentional+model+for+arabic+text+diacritization%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+2021%2C+pages+3757%E2%80%933764%2E)
10. AlKhamissi, B., ElNokrashy, M. N., and Gabr, M. (2020). Deep diacritization: Efficient hierarchical recurrence for improved arabic diacritization. In Workshop on Arabic Natural Language Processing. [find](https://scholar.google.com/scholar?q=AlKhamissi%2C+B%2E%2C+ElNokrashy%2C+M%2E+N%2E%2C+and+Gabr%2C+M%2E+%282020%29%2E+Deep+diacritization%3A+Efficient+hierarchical+recurrence+for+improved+arabic+diacritization%2E+In+Workshop+on+Arabic+Natural+Language+Processing%2E)
11. Almanea, M. M. (2021). Automatic methods and neural networks in arabic texts diacritization: A comprehensive survey. IEEE Access, 9:145012–145032. [find](https://scholar.google.com/scholar?q=Almanea%2C+M%2E+M%2E+%282021%29%2E+Automatic+methods+and+neural+networks+in+arabic+texts+diacritization%3A+A+comprehensive+survey%2E+IEEE+Access%2C+9%3A145012%E2%80%93145032%2E)
12. Alqahtani, S., Mishra, A., and Diab, M. T. (2020). A multitask learning approach for diacritic restoration. In Jurafsky, D., Chai, J., Schluter, N., and Tetreault, J. R., editors, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 8238–8247. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Alqahtani%2C+S%2E%2C+Mishra%2C+A%2E%2C+and+Diab%2C+M%2E+T%2E+%282020%29%2E+A+multitask+learning+approach+for+diacritic+restoration%2E+In+Jurafsky%2C+D%2E%2C+Chai%2C+J%2E%2C+Schluter%2C+N%2E%2C+and+Tetreault%2C+J%2E+R%2E%2C+editors%2C+Proceedings+of+the+5) [our card](https://md.research.elamur.ai/free/en/papers/acl-2020-alqahtani-multitask-learning-approach-diacritic-restoration/)
13. Antoun, W., Baly, F., and Hajj, H. (2020). Arabert: Transformer-based model for arabic language understanding. In LREC 2020 Workshop Language Resources and Evaluation Conference 11–16 May 2020, page 9. [find](https://scholar.google.com/scholar?q=Antoun%2C+W%2E%2C+Baly%2C+F%2E%2C+and+Hajj%2C+H%2E+%282020%29%2E+Arabert%3A+Transformer%2Dbased+model+for+arabic+language+understanding%2E+In+LREC+2020+Workshop+Language+Resources+and+Evaluation+Conference+11%E2%80%9316+May+2020%2C+page+9) [our card](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/)
14. Anwar, M. (2018). Tashkeela-model. [https://github.com/Anwarvic/TashkeelaModel](https://github.com/Anwarvic/TashkeelaModel). [↗](https://github.com/Anwarvic/TashkeelaModel)
15. Barqawi, A. and Zerrouki, T. (2017). Shakkala, arabic text vocalization. Retrieved February, 20:2021. [find](https://scholar.google.com/scholar?q=Barqawi%2C+A%2E+and+Zerrouki%2C+T%2E+%282017%29%2E+Shakkala%2C+arabic+text+vocalization%2E+Retrieved+February%2C+20%3A2021%2E)
16. Belinkov, Y., Magidow, A., Barrón-Cedeño, A., Shmidman, A., and Romanov, M. (2019). Studying the history of the arabic language: language technology and a large-scale historical corpus. Language Resources and Evaluation, 53. [find](https://scholar.google.com/scholar?q=Belinkov%2C+Y%2E%2C+Magidow%2C+A%2E%2C+Barr%C3%B3n%2DCede%C3%B1o%2C+A%2E%2C+Shmidman%2C+A%2E%2C+and+Romanov%2C+M%2E+%282019%29%2E+Studying+the+history+of+the+arabic+language%3A+language+technology+and+a+large%2Dscale+historical+corpus%2E+Language+Resou)
17. Beltagy, I., Peters, M. E., and Cohan, A. (2020). Longformer: The long-document transformer. CoRR, abs/2004.05150. [↗](https://arxiv.org/abs/2004.05150)
18. Darwish, K., Abdelali, A., Mubarak, H., and Eldesouki, M. (2021). Arabic diacritic recovery using a feature-rich bilstm model. Transactions on Asian and Low-Resource Language Information Processing, 20(2):1–18. [find](https://scholar.google.com/scholar?q=Darwish%2C+K%2E%2C+Abdelali%2C+A%2E%2C+Mubarak%2C+H%2E%2C+and+Eldesouki%2C+M%2E+%282021%29%2E+Arabic+diacritic+recovery+using+a+feature%2Drich+bilstm+model%2E+Transactions+on+Asian+and+Low%2DResource+Language+Information+Processing%2C+2) [our card](https://md.research.elamur.ai/free/en/papers/darwish-2020-arabic-diacritic-recovery-using-feature/)
19. Darwish, K., Mubarak, H., and Abdelali, A. (2017). Arabic diacritization: Stats, rules, and hacks. In Proceedings of the Third Arabic Natural Language Processing Workshop, pages 9–17, Valencia, Spain. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Darwish%2C+K%2E%2C+Mubarak%2C+H%2E%2C+and+Abdelali%2C+A%2E+%282017%29%2E+Arabic+diacritization%3A+Stats%2C+rules%2C+and+hacks%2E+In+Proceedings+of+the+Third+Arabic+Natural+Language+Processing+Workshop%2C+pages+9%E2%80%9317%2C+Valencia%2C+Spain%2E) [our card](https://md.research.elamur.ai/free/en/papers/aclbulk-2017-darwish-arabic-diacritization-stats-rules-hacks/)
20. Diab, M., Habash, N., Rambow, O., and Roth, R. (2013). Ldc arabic treebanks and associated corpora: Data divisions manual. [find](https://scholar.google.com/scholar?q=Diab%2C+M%2E%2C+Habash%2C+N%2E%2C+Rambow%2C+O%2E%2C+and+Roth%2C+R%2E+%282013%29%2E+Ldc+arabic+treebanks+and+associated+corpora%3A+Data+divisions+manual%2E)
21. Ding, M., Zhou, C., Yang, H., and Tang, J. (2020). Cogltx: Applying bert to long texts. In Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Ding%2C+M%2E%2C+Zhou%2C+C%2E%2C+Yang%2C+H%2E%2C+and+Tang%2C+J%2E+%282020%29%2E+Cogltx%3A+Applying+bert+to+long+texts%2E+In+Neural+Information+Processing+Systems%2E)
22. El Mekki, A., Alami, A., Alami, H., Khoumsi, A., and Berrada, I. (2020). Weighted combination of bert and n-gram features for nuanced arabic dialect identification. In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 268–274. [find](https://scholar.google.com/scholar?q=El+Mekki%2C+A%2E%2C+Alami%2C+A%2E%2C+Alami%2C+H%2E%2C+Khoumsi%2C+A%2E%2C+and+Berrada%2C+I%2E+%282020%29%2E+Weighted+combination+of+bert+and+n%2Dgram+features+for+nuanced+arabic+dialect+identification%2E+In+Proceedings+of+the+Fifth+Arabic+)
23. El Mekki, A., El Mahdaouy, A., Berrada, I., and Khoumsi, A. (2021a). Domain adaptation for arabic cross-domain and cross-dialect sentiment analysis from contextualized word embedding. In Proceedings of the 2021 conference of the North American chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2824–2837. [find](https://scholar.google.com/scholar?q=El+Mekki%2C+A%2E%2C+El+Mahdaouy%2C+A%2E%2C+Berrada%2C+I%2E%2C+and+Khoumsi%2C+A%2E+%282021a%29%2E+Domain+adaptation+for+arabic+cross%2Ddomain+and+cross%2Ddialect+sentiment+analysis+from+contextualized+word+embedding%2E+In+Proceedings+o)
24. El Mekki, A., El Mahdaouy, A., Essefar, K., El Mamoun, N., Berrada, I., and Khoumsi, A. (2021b). BERT-based multi-task model for country and province level MSA and dialectal Arabic identification. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 271–275, Kyiv, Ukraine (Virtual). Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=El+Mekki%2C+A%2E%2C+El+Mahdaouy%2C+A%2E%2C+Essefar%2C+K%2E%2C+El+Mamoun%2C+N%2E%2C+Berrada%2C+I%2E%2C+and+Khoumsi%2C+A%2E+%282021b%29%2E+BERT%2Dbased+multi%2Dtask+model+for+country+and+province+level+MSA+and+dialectal+Arabic+identification%2E+In+)
25. European Language Resources Association (2001). An-nahar newspaper text corpus. [http://www.language-archives.org/item/oai:catalogue.elra.info:ELRA-W0027](http://www.language-archives.org/item/oai:catalogue.elra.info:ELRA-W0027). [↗](http://www.language-archives.org/item/oai:catalogue.elra.info:ELRA-W0027)
26. Face, H. (2023). Transformers training args. [https://github.com/huggingface/transformers/blob/main/src/transformers/training_args.py](https://github.com/huggingface/transformers/blob/main/src/transformers/training_args.py). [↗](https://github.com/huggingface/transformers/blob/main/src/transformers/training_args.py)
27. Fadel, A., Tuffaha, I., Al-Jawarneh, B., and Al-Ayyoub, M. (2019a). Arabic text diacritization using deep neural networks. In 2019 2nd International Conference on Computer Applications & Information Security (ICCAIS), pages 1–7. [find](https://scholar.google.com/scholar?q=Fadel%2C+A%2E%2C+Tuffaha%2C+I%2E%2C+Al%2DJawarneh%2C+B%2E%2C+and+Al%2DAyyoub%2C+M%2E+%282019a%29%2E+Arabic+text+diacritization+using+deep+neural+networks%2E+In+2019+2nd+International+Conference+on+Computer+Applications+%26+Information+S) [our card](https://md.research.elamur.ai/free/en/papers/fadel-2019-arabic-text-diacritization-using-deep/)
28. Fadel, A., Tuffaha, I., Al-Jawarneh, B., and Al-Ayyoub, M. (2019b). Neural Arabic text diacritization: State of the art results and a novel approach for machine translation. In Proceedings of the 6th Workshop on Asian Translation, pages 215–225, Hong Kong, China. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Fadel%2C+A%2E%2C+Tuffaha%2C+I%2E%2C+Al%2DJawarneh%2C+B%2E%2C+and+Al%2DAyyoub%2C+M%2E+%282019b%29%2E+Neural+Arabic+text+diacritization%3A+State+of+the+art+results+and+a+novel+approach+for+machine+translation%2E+In+Proceedings+of+the+6th+) [our card](https://md.research.elamur.ai/free/en/papers/aclbulk-2019-fadel-neural-arabic-text-diacritization-state/)
29. Gheith Abandah, A. A.-K. (2020). Accurate and fast recurrent neural network solution for the automatic diacritization of arabic text. Jordanian Journal of Computers and Information Technology (JJCIT), 06(02):103 – 121. [find](https://scholar.google.com/scholar?q=Gheith+Abandah%2C+A%2E+A%2E%2DK%2E+%282020%29%2E+Accurate+and+fast+recurrent+neural+network+solution+for+the+automatic+diacritization+of+arabic+text%2E+Jordanian+Journal+of+Computers+and+Information+Technology+%28JJCIT%29%2C)
30. Group, A. L. T. (2016). Farasa toolkit. [find](https://scholar.google.com/scholar?q=Group%2C+A%2E+L%2E+T%2E+%282016%29%2E+Farasa+toolkit%2E)
31. Habash, N., Rambow, O., and Roth, R. (2007). Arabic diacritization with and without a corpus. Arabian Journal for Science and Engineering, 32(1C):3–18. [find](https://scholar.google.com/scholar?q=Habash%2C+N%2E%2C+Rambow%2C+O%2E%2C+and+Roth%2C+R%2E+%282007%29%2E+Arabic+diacritization+with+and+without+a+corpus%2E+Arabian+Journal+for+Science+and+Engineering%2C+32%281C%29%3A3%E2%80%9318%2E)
32. Hassan, H. and Hassan, H. (2010). Sakhr arabic diacritization system. Proceedings of the International Conference on Language Resources and Evaluation (LREC), pages 3870–3876. [find](https://scholar.google.com/scholar?q=Hassan%2C+H%2E+and+Hassan%2C+H%2E+%282010%29%2E+Sakhr+arabic+diacritization+system%2E+Proceedings+of+the+International+Conference+on+Language+Resources+and+Evaluation+%28LREC%29%2C+pages+3870%E2%80%933876%2E)
33. Hifny, Y. (2021). Recent advances in arabic syntactic diacritics restoration. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 7768–7772. IEEE. [find](https://scholar.google.com/scholar?q=Hifny%2C+Y%2E+%282021%29%2E+Recent+advances+in+arabic+syntactic+diacritics+restoration%2E+In+ICASSP+2021%2D2021+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Processing+%28ICASSP%29%2C+pages+7768%E2%80%937772%2E+IE)
34. Howard, J. and Ruder, S. (2018). Universal language model fine-tuning for text classification. [find](https://scholar.google.com/scholar?q=Howard%2C+J%2E+and+Ruder%2C+S%2E+%282018%29%2E+Universal+language+model+fine%2Dtuning+for+text+classification%2E)
35. Inoue, G., Alhafni, B., Baimukan, N., Bouamor, H., and Habash, N. (2021). The interplay of variant, size, and task type in Arabic pre-trained language models. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, Kyiv, Ukraine (Online). Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Inoue%2C+G%2E%2C+Alhafni%2C+B%2E%2C+Baimukan%2C+N%2E%2C+Bouamor%2C+H%2E%2C+and+Habash%2C+N%2E+%282021%29%2E+The+interplay+of+variant%2C+size%2C+and+task+type+in+Arabic+pre%2Dtrained+language+models%2E+In+Proceedings+of+the+Sixth+Arabic+Natura) [our card](https://md.research.elamur.ai/free/en/papers/inoue-2021-interplay-variant-size-task-type/)
36. Khan, M., Khan, M., and Khan, M. (2023). Natural language processing and its use in education. International Journal of Advanced Computer Science and Applications, 5(12). [find](https://scholar.google.com/scholar?q=Khan%2C+M%2E%2C+Khan%2C+M%2E%2C+and+Khan%2C+M%2E+%282023%29%2E+Natural+language+processing+and+its+use+in+education%2E+International+Journal+of+Advanced+Computer+Science+and+Applications%2C+5%2812%29%2E)
37. Maamouri, M., Bies, A., Buckwalter, T., and Mekki, W. (2004). The penn arabic treebank: Building a large-scale annotated arabic corpus. In NEMLAR conference on Arabic language resources and tools, volume 27, pages 466–467. Cairo. [find](https://scholar.google.com/scholar?q=Maamouri%2C+M%2E%2C+Bies%2C+A%2E%2C+Buckwalter%2C+T%2E%2C+and+Mekki%2C+W%2E+%282004%29%2E+The+penn+arabic+treebank%3A+Building+a+large%2Dscale+annotated+arabic+corpus%2E+In+NEMLAR+conference+on+Arabic+language+resources+and+tools%2C+vol)
38. Maamouri, M., Krouna, S., Tabessi, D., Hamrouni, N., and Habash, N. (2012). Egyptian arabic morphological annotation guidelines. [find](https://scholar.google.com/scholar?q=Maamouri%2C+M%2E%2C+Krouna%2C+S%2E%2C+Tabessi%2C+D%2E%2C+Hamrouni%2C+N%2E%2C+and+Habash%2C+N%2E+%282012%29%2E+Egyptian+arabic+morphological+annotation+guidelines%2E)
39. Madhfar, M. A. H. and Qamar, A. M. (2020). Effective deep learning models for automatic diacritization of arabic text. IEEE Access, 9:273–288. [find](https://scholar.google.com/scholar?q=Madhfar%2C+M%2E+A%2E+H%2E+and+Qamar%2C+A%2E+M%2E+%282020%29%2E+Effective+deep+learning+models+for+automatic+diacritization+of+arabic+text%2E+IEEE+Access%2C+9%3A273%E2%80%93288%2E)
40. Mahdaouy, A. E., Mekki, A. E., Essefar, K., Mamoun, N. E., Berrada, I., and Khoumsi, A. (2021). Deep multi-task model for sarcasm detection and sentiment analysis in arabic language. arXiv preprint arXiv:2106.12488. [↗](https://arxiv.org/abs/2106.12488) [our card](https://md.research.elamur.ai/free/en/papers/mahdaouy-2021-deep-multi-task-model-sarcasm/)
41. Mubarak, H., Abdelali, A., Darwish, K., Eldesouki, M., Samih, Y., and Sajjad, H. (2019a). A system for diacritizing four varieties of arabic. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP): System Demonstrations, pages 217–222. [find](https://scholar.google.com/scholar?q=Mubarak%2C+H%2E%2C+Abdelali%2C+A%2E%2C+Darwish%2C+K%2E%2C+Eldesouki%2C+M%2E%2C+Samih%2C+Y%2E%2C+and+Sajjad%2C+H%2E+%282019a%29%2E+A+system+for+diacritizing+four+varieties+of+arabic%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods)
42. Mubarak, H., Abdelali, A., Sajjad, H., Samih, Y., and Darwish, K. (2019b). Highly effective arabic diacritization using sequence to sequence modeling. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2390–2395. [find](https://scholar.google.com/scholar?q=Mubarak%2C+H%2E%2C+Abdelali%2C+A%2E%2C+Sajjad%2C+H%2E%2C+Samih%2C+Y%2E%2C+and+Darwish%2C+K%2E+%282019b%29%2E+Highly+effective+arabic+diacritization+using+sequence+to+sequence+modeling%2E+In+Proceedings+of+the+2019+Conference+of+the+Nort)
43. OpenAI (2023). Gpt-4 technical report. arXiv. [find](https://scholar.google.com/scholar?q=OpenAI+%282023%29%2E+Gpt%2D4+technical+report%2E+arXiv%2E) [our card](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/)
44. Qamar, S. I. (2008). Shia library (a great collection of books from shia sect). Online Resource. [find](https://scholar.google.com/scholar?q=Qamar%2C+S%2E+I%2E+%282008%29%2E+Shia+library+%28a+great+collection+of+books+from+shia+sect%29%2E+Online+Resource%2E)
45. Qin, H., Chen, G., Tian, Y., and Song, Y. (2021). Improving arabic diacritization with regularized decoding and adversarial training. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 534–542. [find](https://scholar.google.com/scholar?q=Qin%2C+H%2E%2C+Chen%2C+G%2E%2C+Tian%2C+Y%2E%2C+and+Song%2C+Y%2E+%282021%29%2E+Improving+arabic+diacritization+with+regularized+decoding+and+adversarial+training%2E+In+Proceedings+of+the+59th+Annual+Meeting+of+the+Association+for+C) [our card](https://md.research.elamur.ai/free/en/papers/acl-2021-qin-improving-arabic-diacritization-regularized-decoding/)
46. Rashwan, M. A. A., Al Sallab, A. A., Raafat, H. M., and Rafea, A. (2015). Deep learning framework with confused sub-set resolution architecture for automatic arabic diacritization. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 23(3):505–516. [find](https://scholar.google.com/scholar?q=Rashwan%2C+M%2E+A%2E+A%2E%2C+Al+Sallab%2C+A%2E+A%2E%2C+Raafat%2C+H%2E+M%2E%2C+and+Rafea%2C+A%2E+%282015%29%2E+Deep+learning+framework+with+confused+sub%2Dset+resolution+architecture+for+automatic+arabic+diacritization%2E+IEEE%2FACM+Transactio)
47. Reimers, N. and Gurevych, I. (2019). Sentence-bert: Sentence embeddings using siamese bert-networks. [find](https://scholar.google.com/scholar?q=Reimers%2C+N%2E+and+Gurevych%2C+I%2E+%282019%29%2E+Sentence%2Dbert%3A+Sentence+embeddings+using+siamese+bert%2Dnetworks%2E)
48. Ryding, K. C. (2005). A Reference Grammar of Modern Standard Arabic. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Ryding%2C+K%2E+C%2E+%282005%29%2E+A+Reference+Grammar+of+Modern+Standard+Arabic%2E+Cambridge+University+Press%2E)
49. Taji, D., Habash, N., and Elkoury, R. (2014). The impact of arabic diacritization on part-of-speech tagging. In Proceedings of the International Conference on Language Resources and Evaluation (LREC), pages 1306–1311. [find](https://scholar.google.com/scholar?q=Taji%2C+D%2E%2C+Habash%2C+N%2E%2C+and+Elkoury%2C+R%2E+%282014%29%2E+The+impact+of+arabic+diacritization+on+part%2Dof%2Dspeech+tagging%2E+In+Proceedings+of+the+International+Conference+on+Language+Resources+and+Evaluation+%28LREC%29%2C)
50. Thompson, B. and Alshehri, A. (2022). Improving Arabic diacritization by learning to diacritize and translate. In Proceedings of the 19th International Conference on Spoken Language Translation (IWSLT 2022), pages 11–21, Dublin, Ireland (in-person and online). Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Thompson%2C+B%2E+and+Alshehri%2C+A%2E+%282022%29%2E+Improving+Arabic+diacritization+by+learning+to+diacritize+and+translate%2E+In+Proceedings+of+the+19th+International+Conference+on+Spoken+Language+Translation+%28IWSLT) [our card](https://md.research.elamur.ai/free/en/papers/thompson-2022-improving-arabic-diacritization-learning-diacritize/)
51. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. (2017). Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, page 6000–6010, Red Hook, NY, USA. Curran Associates Inc. [find](https://scholar.google.com/scholar?q=Vaswani%2C+A%2E%2C+Shazeer%2C+N%2E%2C+Parmar%2C+N%2E%2C+Uszkoreit%2C+J%2E%2C+Jones%2C+L%2E%2C+Gomez%2C+A%2E+N%2E%2C+Kaiser%2C+L%2E%2C+and+Polosukhin%2C+I%2E+%282017%29%2E+Attention+is+all+you+need%2E+In+Proceedings+of+the+31st+International+Conference+on+N)
52. Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., von Platen, P., Ma, C., Jernite, Y., Plu, J., Xu, C., Scao, T. L., Gugger, S., Drame, M., Lhoest, Q., and Rush, A. M. (2020). Huggingface’s transformers: State-of-the-art natural language processing. [find](https://scholar.google.com/scholar?q=Wolf%2C+T%2E%2C+Debut%2C+L%2E%2C+Sanh%2C+V%2E%2C+Chaumond%2C+J%2E%2C+Delangue%2C+C%2E%2C+Moi%2C+A%2E%2C+Cistac%2C+P%2E%2C+Rault%2C+T%2E%2C+Louf%2C+R%2E%2C+Funtowicz%2C+M%2E%2C+Davison%2C+J%2E%2C+Shleifer%2C+S%2E%2C+von+Platen%2C+P%2E%2C+Ma%2C+C%2E%2C+Jernite%2C+Y%2E%2C+Plu%2C+J%2E%2C+Xu%2C+C%2E%2C+Sca) [our card](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/)
53. Yousef, W. A., Ibrahime, O. M., Madbouly, T. M., and Mahmoud, M. (2019). Learning meters of arabic and english poems with recurrent neural networks: a step forward for language understanding and synthesis. ArXiv, abs/1905.05700. [↗](https://arxiv.org/abs/1905.05700)
54. Zaheer, M., Guruganesh, G., Dubey, A., Ainslie, J., Alberti, C., Ontanon, S., Pham, P., Ravula, A., Wang, Q., Yang, L., and Ahmed, A. (2021). Big bird: Transformers for longer sequences. [find](https://scholar.google.com/scholar?q=Zaheer%2C+M%2E%2C+Guruganesh%2C+G%2E%2C+Dubey%2C+A%2E%2C+Ainslie%2C+J%2E%2C+Alberti%2C+C%2E%2C+Ontanon%2C+S%2E%2C+Pham%2C+P%2E%2C+Ravula%2C+A%2E%2C+Wang%2C+Q%2E%2C+Yang%2C+L%2E%2C+and+Ahmed%2C+A%2E+%282021%29%2E+Big+bird%3A+Transformers+for+longer+sequences%2E) [our card](https://md.research.elamur.ai/free/en/papers/zaheer-2020-big-bird-transformers-longer-sequences/)
55. Zalmout, N. and Habash, N. (2020). Joint diacritization, lemmatization, normalization, and fine-grained morphological tagging. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8297–8307, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Zalmout%2C+N%2E+and+Habash%2C+N%2E+%282020%29%2E+Joint+diacritization%2C+lemmatization%2C+normalization%2C+and+fine%2Dgrained+morphological+tagging%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Association+for+Computat) [our card](https://md.research.elamur.ai/free/en/papers/acl-2020-zalmout-joint-diacritization-lemmatization-normalization-fine/)
56. Zerrouki, T. (2020). Towards An Open Platform For Arabic Language Processing. PhD thesis. [find](https://scholar.google.com/scholar?q=Zerrouki%2C+T%2E+%282020%29%2E+Towards+An+Open+Platform+For+Arabic+Language+Processing%2E+PhD+thesis%2E)