---
title: "Is Arabic Part of Speech Tagging Feasible Without Word"
description: "Mohamed and Kübler test whether raw orthographic words can be tagged directly. 8 references, cited by 4 papers here."
authors: ["Emad Mohamed", "Sandra Kübler"]
year: 2010
venue: "Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics"
source: "https://aclanthology.org/N10-1105/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/mohamed-2010-is-arabic-part-speech-tagging/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/mohamed-2010-is-arabic-part-speech-tagging/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2010](https://md.research.elamur.ai/free/en/papers/2010/page/2/) / Is Arabic Part of Speech Tagging Feasible Without Word Segmentation?

# Is Arabic Part of Speech Tagging Feasible Without Word Segmentation?

**Authors:** [Emad Mohamed](https://md.research.elamur.ai/free/en/authors/emad-mohamed/) and [Sandra Kübler](https://md.research.elamur.ai/free/en/authors/sandra-kubler/) · **Year:** 2010 · **Venue:** [Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/naacl/) ([NAACL 2010](https://md.research.elamur.ai/free/en/venues/naacl/2010/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/N10-1105/) · [PDF](https://aclanthology.org/N10-1105.pdf) · [OpenAlex](https://openalex.org/W2122118939)

**Citations:** 17 citations (OpenAlex, 2026-10-14) · [4](https://md.research.elamur.ai/free/en/papers/mohamed-2010-is-arabic-part-speech-tagging/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Arabic words often fuse a stem with clitics such as conjunctions, prepositions and pronouns. The Penn Arabic Treebank (ATB) therefore tags segments rather than whole words, and earlier taggers relied on the treebank's gold segmentation, which real text never provides. Mohamed and Kübler test whether raw orthographic words can be tagged directly. They took about 500,000 unvocalized words from ATB sections P1V3 and P3V1. To make the data resemble newswire, they re-attached split-off clitics and removed the null subjects the treebank inserts for pro-drop. Under 5-fold cross validation they compared three setups, all built on memory-based learners (TiMBL, MBT):
- tagging gold segments, as an upper bound;
- tagging the output of a learned character-level segmenter;
- tagging whole words with complex tags from the full tagset.

Whole-word tagging reached 94.74% word accuracy. That beats the segmenter pipeline (93.47%) and nearly matches gold segmentation (94.91%), even though the whole-word tagger had to choose among 993 complex tags. The authors note that this result may partly reflect the limited newswire vocabulary of the ATB. Still, it suggests that accurate Arabic tagging is possible without segmentation or a full morphological analysis.

## Key points
- The segmenter is TiMBL (IB1, k=1, gain-ratio weights) and reaches 98.23% word accuracy. Proper nouns cause 33.87% of its errors and common nouns 30.67%.
- Tagging accuracy on known vs. unknown words:
  - whole words: 96.61% vs. 74.64%;
  - segmenter pipeline: 95.57% vs. 71.06%;
  - gold segments: 95.90% vs. 84.25%.
- On average, 8.55% of test words are unknown. Of the pipeline's errors on unknown words, 59.68% trace back to bad segmentation, compared with only 6.24% of its errors on known words.
- The whole-word tagger faces 993 complex tags, 22.70% of them seen only once in training; the segment-level tagset has 139 tags.
- The whole-word output was mapped to the Habash and Rambow tokenization and a 15-tag reduced tagset for comparison. It scored 99.36% tokenization accuracy and 96.41% POS accuracy, close to their reported results.

## Interesting ideas
- Segmentation is treated as a per-character decision (does a segment end here?), using a window of 5 characters on each side, earlier decisions, and the whole-word tagger's guess as features. This makes splitting a simple, inspectable classifier informed by the tag.
- Before training and evaluating, the authors undo treebank conventions that do not exist in real text, such as split clitics and inserted null subjects, so that scores reflect performance on natural input.
- The "ambitag" feature summarizes all tags a word has been seen with into one ambiguity-class symbol. This gives the learner a compact lexical prior without hand-written rules.
- Errors are split into known vs. unknown words and traced to the upstream stage that caused them. This shows that a 98%-accurate segmenter still causes most downstream errors on unseen words.
- Comparing a pipeline against a joint, unsegmented model with the same learner and data split isolates whether a preprocessing step actually helps. In this paper, the step turned out to hurt.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tokens): Clitic splits like "don't", "can't" or possessive "'s" could become explicit per-character boundary decisions that use the tagger's hypothesis, so each split can be traced and explained.
- **I2** (tree): When converting between UD dialects, we could keep one test copy rendered exactly as raw text would appear, so that parser scores are not inflated by tokenization that gold annotation silently supplies.
- **I3** (tokens): The en tagger could use the attested tag set of each word form as a feature and show it in explanations, for example "this word is ambiguous between NOUN and VERB".
- **I4** (learn): Evaluation should report out-of-vocabulary slices separately and attribute each parse or frame error to the pipeline stage where it first appeared, so that small upstream error rates are not overlooked.
- **I5** (learn): Each pipeline stage could be ablated against a version that skips it under identical folds, a negative control to confirm the stage is worth its error propagation.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Indiana University Bloomington](https://md.research.elamur.ai/free/en/universities/indiana-university-bloomington/) — University, Bloomington, United States · 37 works · 3,685 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (3): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works · [🔤 Part-of-speech tagging](https://md.research.elamur.ai/free/en/topics/part-of-speech-tagging/) · 126 works · [✂️ Tokenization & segmentation](https://md.research.elamur.ai/free/en/topics/tokenization/) · 119 works

## Cite

Cite as: Emad Mohamed and Sandra Kübler. 2010. Is Arabic Part of Speech Tagging Feasible Without Word Segmentation?. Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{mohamed2010is,
  title = {{Is Arabic Part of Speech Tagging Feasible Without Word Segmentation?}},
  author = {Emad Mohamed and Sandra Kübler},
  booktitle = {Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2010},
  doi = {10.13140/2.1.3631.8402},
  url = {https://doi.org/10.13140/2.1.3631.8402}
}
```

- [← previous Exploiting Rich Features for Detecting Hedges and their Scope](https://md.research.elamur.ai/free/en/papers/li-2010-exploiting-rich-features-detecting-hedges/)
- [next → Modeling Morphosyntactic Agreement in Constituency-Based Parsing of Modern Hebrew](https://md.research.elamur.ai/free/en/papers/tsarfaty-2010-modeling-morphosyntactic-agreement-constituency-based/)
- [↑ back to papers of 2010](https://md.research.elamur.ai/free/en/papers/2010/page/2/)

## References

8 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Shahib AlGahtani, William Black, and John Mc-Naught. 2009. Arabic part-of-speech-tagging using transformation-based learning. In Proceeedings of the 2nd International Conference on Arabic Language Resources and Tools, Cairo, Egypt. [find](https://scholar.google.com/scholar?q=Shahib+AlGahtani%2C+William+Black%2C+and+John+Mc%2DNaught%2E+2009%2E+Arabic+part%2Dof%2Dspeech%2Dtagging+using+transformation%2Dbased+learning%2E+In+Proceeedings+of+the+2nd+International+Conference+on+Arabic+Language+Res)
2. Ann Bies and Mohamed Maamouri. 2003. Penn Arabic Treebank guidelines. Technical report, LDC, University of Pennsylvania. [find](https://scholar.google.com/scholar?q=Ann+Bies+and+Mohamed+Maamouri%2E+2003%2E+Penn+Arabic+Treebank+guidelines%2E+Technical+report%2C+LDC%2C+University+of+Pennsylvania%2E)
3. Walter Daelemans and Antal van den Bosch. 2005. Memory Based Language Processing. Cambridge University Press. [↗](https://doi.org/10.1017/cbo9780511486579)
4. Walter Daelemans, Jakub Zavrel, Peter Berck, and Steven Gillis. 1996. [MBT: A memory-based part of speech tagger-generator](https://md.research.elamur.ai/free/en/papers/daelemans-1996-mbt-memory-based-part-speech/). In Eva Ejerhed and Ido Dagan, editors, Proceedings of the 4th Workshop on Very Large Corpora, pages 14–27, Copenhagen, Denmark.
5. Mona Diab, Kadri Hacioglu, and Daniel Jurafsky. 2004. [Automatic tagging of Arabic text: From raw text to base phrase chunks](https://md.research.elamur.ai/free/en/papers/diab-2004-automatic-tagging-arabic-text-raw/). In Proceedings of HLT-NAACL, Boston, MA.
6. Nizar Habash and Owen Rambow. 2005. [Arabic tokenization, part-of-speech tagging and morphological disambiguation in one fell swoop](https://md.research.elamur.ai/free/en/papers/habash-2005-arabic-tokenization-part-speech-tagging/). In Proceedings of ACL-2005, pages 573—580, Ann Arbor, MI.
7. Emad Mohamed and Sandra Kübler. 2010. [Arabic part of speech tagging](https://md.research.elamur.ai/free/en/papers/mohamed-2010-arabic-part-speech-tagging/). In Proceedings of LREC, Valetta, Malta.
8. Antal van den Bosch, Erwin Marsi, and Abdelhadi Soudi. 2007. Memory-based morphological analysis and part-of-speech tagging of Arabic. In Abdelhadi Soudi, Antal van den Bosch, and Günter Neumann, editors, Arabic Computational Morphology. Springer. [↗](https://doi.org/10.1007/978-1-4020-6046-5_11)

*Updated 1 November 2026*