---
title: "Statistical-based System for Morphological Annotation of Arabic"
description: "The authors build AMAS (Arabic Morphological Annotation System), a Java tool with two steps. 89.01% recall. 19 references."
authors: ["Nabil Khoufi", "Manel Boudokhane"]
year: 2013
venue: "Proceedings of the Student Research Workshop associated with RANLP 2013"
source: "https://aclanthology.org/R13-2015/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/khoufi-2013-statistical-based-system-morphological-annotation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/khoufi-2013-statistical-based-system-morphological-annotation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2013](https://md.research.elamur.ai/free/en/papers/2013/page/3/) / Statistical-based System for Morphological Annotation of Arabic Texts

# Statistical-based System for Morphological Annotation of Arabic Texts

**Authors:** [Nabil Khoufi](https://md.research.elamur.ai/free/en/authors/nabil-khoufi/) and Manel Boudokhane · **Year:** 2013 · **Venue:** [Proceedings of the Student Research Workshop associated with RANLP 2013](https://md.research.elamur.ai/free/en/venues/ranlp/) ([RANLP 2013](https://md.research.elamur.ai/free/en/venues/ranlp/2013/)) · **Publisher:** [INCOMA Ltd.](https://md.research.elamur.ai/free/en/publishers/incoma-ltd/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/R13-2015/) · [PDF](https://aclanthology.org/R13-2015.pdf) · [OpenAlex](https://openalex.org/W2250903889)

**Citations:** 6 citations (OpenAlex, 2026-10-14)

## Summary
The paper tackles automatic morphological tagging of unvocalized Modern Standard Arabic. Missing short vowels and the shadda mark create many homographs, and clitics, prefixes and suffixes attach to words. The authors build AMAS (Arabic Morphological Annotation System), a Java tool with two steps. First, it splits text into sentences and tokens. Tokens not found in a stem lexicon taken from AraMorph are stripped of affixes using pre- and post-base lists. Second, it gathers every tag a word received in the Penn Arabic Treebank (ATB) and picks one. It uses a bigram probability conditioned on the previous tag, and falls back to plain frequency when no usable previous tag exists. A semi-automatic mode lets a user tag unknown words, and those choices then feed later decisions. Output is XML with a probability per word. Compared against a linguist on part of the EASC corpus, AMAS reaches an F-score of about 84%. That beats two rule-based analysers but stays below published statistical systems. The paper is a useful diagnosis of how segmentation and coverage errors cascade.

## Key points
- Training and knowledge source: the Penn Arabic Treebank (ATB), 599 unvowelled news texts from a Lebanese agency; AMAS uses the ATB tag set directly.
- Segmentation relies on the AraMorph (Buckwalter) stem base, cut down to bare stems, plus affix lists adapted from Abbes; only affixes seen in the training corpus are used.
- Disambiguation is a first-order tag model, P(t_i | t_i-1), with unigram frequency used for sentence-initial words and words after an unknown token.
- Evaluation covered 22 EASC texts (10,148 segmented words), averaged per domain: precision 89.01%, recall 80.24%, F-score 84.37%.
- For comparison, the knowledge-based analysers scored 76 and 82.14, while the statistical systems (Diab, Habash, Mansour, AlGahtani) scored 95.5–97.5.
- Reported error sources:
  - segmentation mistakes, such as article plus preposition, passed on to tagging;
  - unknown words breaking the tag context for the next word;
  - a weak choice for sentence-initial words;
  - gaps in ATB coverage;
  - spelling variants (different alef forms) treated as unknown.

## Interesting ideas
- When the previous token is unknown, the tagger deliberately drops to a context-free frequency estimate instead of conditioning on garbage, which limits how far one gap damages the sequence.
- Affix stripping is restricted to affixes actually attested in the training corpus, so the segmenter never proposes splits the tagger has no statistics for.
- Every output token carries the probability of its chosen tag, which exposes low-confidence decisions to downstream steps and to human checkers.
- A semi-automatic mode lets a human resolve unknown words mid-run, and those answers immediately inform the rest of the text.
- The error analysis shows that orthographic variants of the same word (different alef shapes) become false unknowns, so normalization before lexicon lookup matters.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tokens): the en tagger could mark OOV tokens explicitly and reset its contextual features after them, logging the fallback so the tag decision stays explainable.
- **I2** (tokens): suffix or contraction splitting in the tokenizer could be limited to patterns observed in the treebanks, keeping tokenizer and tagger vocabularies consistent.
- **I3** (learn): per-token confidence could flag which silver-data tokens need extra witness parsers or an LLM teacher tier, instead of trusting all tags equally.
- **I4** (learn): a similar loop could turn teacher corrections on OOV words into lexicon entries used within the same document, which also makes an easy evaluation of how much repair helps.
- **I5** (tokens): normalizing English spelling variants (British/American, curly quotes, hyphenation) before lemma lookup could cut spurious unknowns.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (2): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works · [🧩 Morphology](https://md.research.elamur.ai/free/en/topics/morphology/) · 405 works

## Cite

Cite as: Nabil Khoufi and Manel Boudokhane. 2013. Statistical-based System for Morphological Annotation of Arabic Texts. Proceedings of the Student Research Workshop associated with RANLP 2013.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{khoufi2013statistical,
  title = {{Statistical-based System for Morphological Annotation of Arabic Texts}},
  author = {Nabil Khoufi and Manel Boudokhane},
  booktitle = {Proceedings of the Student Research Workshop associated with RANLP 2013},
  year = {2013},
  url = {https://aclanthology.org/R13-2015/}
}
```

- [← previous Labeled Alignment for Recognizing Textual Entailment](https://md.research.elamur.ai/free/en/papers/wang-2013-labeled-alignment-recognizing-textual-entailment/)
- [next → Dudley North visits North London: Learning When to Transliterate to Arabic](https://md.research.elamur.ai/free/en/papers/azab-2013-dudley-north-visits-north-london/)
- [↑ back to papers of 2013](https://md.research.elamur.ai/free/en/papers/2013/page/3/)

## References

19 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ramzi Abbes. 2004. La conception et la réalisation d'un concordancier électronique pour l'arabe, Ph.D thesis, ENSSIB/INSA, Lyon, France. [find](https://scholar.google.com/scholar?q=Ramzi+Abbes%2E+2004%2E+La+conception+et+la+r%C3%A9alisation+d%27un+concordancier+%C3%A9lectronique+pour+l%27arabe%2C+Ph%2ED+thesis%2C+ENSSIB%2FINSA%2C+Lyon%2C+France%2E)
2. Lahsen Abouenour, Said El Hassani, Tawfiq Yazidy, Karim Bouzouba and Abdelfattah Hamdani. 2008. Building an Arabic Morphological Analyzer as part of an Open Arabic NLP Platform. In Workshop on HLT and NLP within the Arabic world: Arabic Language and local languages processing Status Updates and Prospects At the 6th Language Resources and Evaluation Conference (LREC’08), Marrakech, Morocco. [find](https://scholar.google.com/scholar?q=Lahsen+Abouenour%2C+Said+El+Hassani%2C+Tawfiq+Yazidy%2C+Karim+Bouzouba+and+Abdelfattah+Hamdani%2E+2008%2E+Building+an+Arabic+Morphological+Analyzer+as+part+of+an+Open+Arabic+NLP+Platform%2E+In+Workshop+on+HLT+and)
3. Shabib AlGahtani, William Black, and John Mc-Naught. 2009. Arabic part-of-speech-tagging using transformation-based learning. In Proceeedings of the 2nd International Conference on Arabic Language Resources and Tools, Cairo, Egypt. [find](https://scholar.google.com/scholar?q=Shabib+AlGahtani%2C+William+Black%2C+and+John+Mc%2DNaught%2E+2009%2E+Arabic+part%2Dof%2Dspeech%2Dtagging+using+transformation%2Dbased+learning%2E+In+Proceeedings+of+the+2nd+International+Conference+on+Arabic+Language+Res)
4. Mohammed Attia. 2006. An Ambiguity controlled Morphological Analyzer for Modern Standard Arabic Modelling Finite State Networks. In The challenge of Arabic for NLP/MT conference, the British Computer Society Conference, pages 48-67, London. [find](https://scholar.google.com/scholar?q=Mohammed+Attia%2E+2006%2E+An+Ambiguity+controlled+Morphological+Analyzer+for+Modern+Standard+Arabic+Modelling+Finite+State+Networks%2E+In+The+challenge+of+Arabic+for+NLP%2FMT+conference%2C+the+British+Computer+)
5. Kenneth Beesley. 2001. Finite-State Morphological Analysis and Generation of Arabic at Xerox Research: Status and Plans in 2001. In Proceedings of the Arabic Language Processing: Status and Prospect-39th Annual Meeting of the Association for Computational Linguistics, Toulouse, France. [find](https://scholar.google.com/scholar?q=Kenneth+Beesley%2E+2001%2E+Finite%2DState+Morphological+Analysis+and+Generation+of+Arabic+at+Xerox+Research%3A+Status+and+Plans+in+2001%2E+In+Proceedings+of+the+Arabic+Language+Processing%3A+Status+and+Prospect%2D3)
6. Abderrahim Boudlal, Rachid Belahbib, Abdelhak Lakhouaja, Azzeddine Mazroui, Abdelouafi Meziane and Mohamed Ould Abdallahi Ould Bebah. 2008. A Markovian Approach for Arabic Root Extraction. In The International Arab Conference on Information Technology, Hammamet, Tunisia. [find](https://scholar.google.com/scholar?q=Abderrahim+Boudlal%2C+Rachid+Belahbib%2C+Abdelhak+Lakhouaja%2C+Azzeddine+Mazroui%2C+Abdelouafi+Meziane+and+Mohamed+Ould+Abdallahi+Ould+Bebah%2E+2008%2E+A+Markovian+Approach+for+Arabic+Root+Extraction%2E+In+The+Inte)
7. Tim Buckwalter. 2004. [Issues in Arabic Orthography and Morphology Analysis](https://md.research.elamur.ai/free/en/papers/buckwalter-2004-issues-arabic-orthography-morphology-analysis/). In The Workshop on Computational Approaches to Arabic Script-based Languages, COLING, Geneva.
8. Alexander Clark. 2003. [Combining Distributional and Morphological Information for Part of Speech Induction](https://md.research.elamur.ai/free/en/papers/clark-2003-combining-distributional-morphological-information-part/). In Proceedings of the tenth Annual Meeting of the European Association for Computational Linguistics EACL 2003, pages 59-66.
9. Kareem Darwish. 2002. Building a Shallow Arabic Morphological Analyzer in One Day. In Proceedings of the ACL-02 workshop on Computational approaches to semitic languages, Stroudsburg, PA, USA. [find](https://scholar.google.com/scholar?q=Kareem+Darwish%2E+2002%2E+Building+a+Shallow+Arabic+Morphological+Analyzer+in+One+Day%2E+In+Proceedings+of+the+ACL%2D02+workshop+on+Computational+approaches+to+semitic+languages%2C+Stroudsburg%2C+PA%2C+USA%2E)
10. Mona Diab. 2009. Second Generation Tools (AMIRA 2.0): Fast and Robust Tokenization, POS tagging, and Base Phrase Chunking, MEDAR 2nd International Conference on Arabic Language Resources and Tools, Cairo, Egypt. [find](https://scholar.google.com/scholar?q=Mona+Diab%2E+2009%2E+Second+Generation+Tools+%28AMIRA+2%2E0%29%3A+Fast+and+Robust+Tokenization%2C+POS+tagging%2C+and+Base+Phrase+Chunking%2C+MEDAR+2nd+International+Conference+on+Arabic+Language+Resources+and+Tools%2C+Ca)
11. Mahmoud El-Haj, Udo Kruschwitz, and Chris Fox. 2010. Using Mechanical Turk to Create a Corpus of Arabic Summaries. In the Language Resources and Human Language Technologies for Semitic Languages workshop held in conjunction with the 7th International Language Resources and Evaluation Conference, pages 36-39, Valletta, Malta. [find](https://scholar.google.com/scholar?q=Mahmoud+El%2DHaj%2C+Udo+Kruschwitz%2C+and+Chris+Fox%2E+2010%2E+Using+Mechanical+Turk+to+Create+a+Corpus+of+Arabic+Summaries%2E+In+the+Language+Resources+and+Human+Language+Technologies+for+Semitic+Languages+works)
12. Nizar Habash, Owen Rambow and Ryan Roth. 2009. MADA+TOKAN: A Toolkit for Arabic Tokenization, Diacritization, Morphological Disambiguation, POS Tagging, Stemming and Lemmatization. In Proceedings of the 2nd International Conference on Arabic Language Resources and Tools (MEDAR), Cairo, Egypt. [find](https://scholar.google.com/scholar?q=Nizar+Habash%2C+Owen+Rambow+and+Ryan+Roth%2E+2009%2E+MADA%2BTOKAN%3A+A+Toolkit+for+Arabic+Tokenization%2C+Diacritization%2C+Morphological+Disambiguation%2C+POS+Tagging%2C+Stemming+and+Lemmatization%2E+In+Proceedings+of+t)
13. Mohamed Maamouri, Ann Bies, Tim Buckwalter, and Wigdan Mekki. 2004. The Penn Arabic Treebank: Building a Large-Scale Annotated Arabic Corpus. In NEMLAR International Conference on Arabic Language Resources and Tools. [find](https://scholar.google.com/scholar?q=Mohamed+Maamouri%2C+Ann+Bies%2C+Tim+Buckwalter%2C+and+Wigdan+Mekki%2E+2004%2E+The+Penn+Arabic+Treebank%3A+Building+a+Large%2DScale+Annotated+Arabic+Corpus%2E+In+NEMLAR+International+Conference+on+Arabic+Language+Reso)
14. Saib Mansour, Khalil Sima'an and Yoad Winter. 2007. [Smoothing a Lexicon-based POS tagger for Arabic and Hebrew](https://md.research.elamur.ai/free/en/papers/manour-2007-smoothing-lexicon-based-pos-tagger/). In proceedings of ACL 2007 Workshop on Computational Approaches to Semitic Languages: Common Issues and Resources. Prague, Czech Republic.
15. Paul McNamee, James Mayfield. 2004. Character N-Gram Tokenization for European Language Text Retrieval. In Journal Information Retrieval, volume 7, issue 1-2, pages 73 – 97. [↗](https://doi.org/10.1023/b:inrt.0000009441.78971.be)
16. Suleiman H. Mustafa and Qacem A. Al-Radaideh. 2004. Using N-grams for Arabic text searching. In Journal of the American Society for Information Science and Technology archive, Volume 55, Issue 11, Pages 1002-1007 John Wiley & Sons, Inc. New York, USA. [↗](https://doi.org/10.1002/asi.20051)
17. Riadh Ouersighni. 2002. La conception et la réalisation d'un système d'analyse morpho-syntaxique robuste pour l'arabe : utilisation pour la détection et le diagnostic des fautes d'accord. Ph.D. thesis, Lumiere-Lyon2 university, France. [find](https://scholar.google.com/scholar?q=Riadh+Ouersighni%2E+2002%2E+La+conception+et+la+r%C3%A9alisation+d%27un+syst%C3%A8me+d%27analyse+morpho%2Dsyntaxique+robuste+pour+l%27arabe+%3A+utilisation+pour+la+d%C3%A9tection+et+le+diagnostic+des+fautes+d%27accord%2E+Ph%2ED%2E+thesis)
18. Abdelhadi Soudi, Gunter Neumann and Antal Van den Bosch. 2007. Arabic Computational Morphology: Knowledge-based and Empirical Methods. In Arabic Computational Morphology, pages 3-14, Springer. [↗](https://doi.org/10.1007/978-1-4020-6046-5_1)
19. Otakar Smrz. 2007. [ElixirFM — Implementation of Functional Arabic Morphology](https://md.research.elamur.ai/free/en/papers/smrz-2007-elixirfm-implementation-functional-arabic-morphology/). In Proceedings of the 2007 Workshop on Computational Approaches to Semitic Languages: Common Issues and Resources, pages 1-8, Prague, Czech Republic.

*Updated 1 November 2026*