--- title: "The Effect of Automatic Tokenization, Vocalization, Stemming, and POS Tagging — Mohamed 2011" description: "Shared-task results for Arabic dependency parsing were usually reported on fully gold input. 71.56% LAS. 11 references, cited by 2 papers here." authors: ["Emad Mohamed"] year: 2011 venue: "Proceedings of the Fifteenth Conference on Computational Natural Language Learning" source: "https://aclanthology.org/W11-0302/" licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/mohamed-2011-effect-automatic-tokenization-vocalization-stemming/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/mohamed-2011-effect-automatic-tokenization-vocalization-stemming/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2011](https://md.research.elamur.ai/free/en/papers/2011/page/2/) / The Effect of Automatic Tokenization, Vocalization, Stemming, and POS Tagging on Arabic Dependency Parsing # The Effect of Automatic Tokenization, Vocalization, Stemming, and POS Tagging on Arabic Dependency Parsing **Authors:** [Emad Mohamed](https://md.research.elamur.ai/free/en/authors/emad-mohamed/) · **Year:** 2011 · **Venue:** Proceedings of the Fifteenth Conference on Computational Natural Language Learning · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/) **Links:** [Publisher's page](https://aclanthology.org/W11-0302/) · [PDF](https://aclanthology.org/W11-0302.pdf) · [OpenAlex](https://openalex.org/W1416891392) **Citations:** 3 citations (OpenAlex, 2026-10-14) · [2](https://md.research.elamur.ai/free/en/papers/mohamed-2011-effect-automatic-tokenization-vocalization-stemming/cited-by/) in our library, of the 140,884 papers whose bibliographies we read ## Summary Shared-task results for Arabic dependency parsing were usually reported on fully gold input. That means hand-annotated tokens, lemmas, POS tags, morphological features and vowel marks. This paper asks how much accuracy remains when every preprocessing step is automatic. The author keeps the CoNLL 2007 Arabic data (from the Prague Arabic Dependency Treebank), its train/test split, and the Single MaltParser configuration of Hall et al. (2007), so the only variable is the source of the input columns. The pipeline has four parts: - a TIMBL memory-based segmenter that labels each letter; - hand-written rules that turn the segments into tokens and stems; - the MBT tagger for POS; - an optional memory-based vocalizer. The morphological feature column is left empty. Every module scores well, yet LAS falls from 74.75% with gold input to 63.10% with automatic input. Nearly all of this loss comes from tokenization, even though tokenization is 99.34% accurate, because a wrong split creates or removes tree nodes. The paper concludes that for morphologically rich languages, the quality of segmentation limits how good parsing can be. ## Key points - Tokenization made 30 errors over 4,550 test words, but these touched 21 of the 132 test sentences. - 17 errors split too much and 13 split too little. - 13 of the over-splits were one word, blywn ("billion"), whose first letter was wrongly treated as a preposition. - On correctly tokenized sentences the automatic pipeline reaches 71.56% LAS. On sentences with a tokenization mistake it drops to 33.60%. - POS tagging accuracy is 96.39% on gold tokens and 95.70% on automatic tokens. Using the Penn Arabic Treebank tags as CPOS helps only without gold features (72.54% → 73.09% LAS). - Gold vocalization gives 74.77% LAS and unvocalized text gives 74.16%. The automatic vocalizer (93.8% accurate) still yields 74.43%. In the fully automatic setting, vocalization makes no difference (63.11% either way). - Most mis-tokenized words were absent from the training data, so the segmenter is nearly flawless on known words. One counted "error" was actually a mistake in the PADT gold annotation. ## Interesting ideas - Scoring the parser separately on sentences whose upstream segmentation was right versus wrong shows exactly how much damage the earlier stage passes on. - Swapping one gold input column at a time for its predicted version, while the parser and data stay fixed, gives a clean per-module ablation of a pipeline. - Segmentation is framed as a per-character boundary classification, and a small rule layer then derives both syntactic tokens and stems from the same boundaries. - Error analysis showed that errors cluster on words unseen in training, or seen only in a misleading form, and that a single frequent word can dominate the error count. - Some disagreements with the treebank were really annotation errors, where the system was right but was scored as wrong. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (learn): Report tree accuracy split by "tokenizer matched gold" vs "did not", so a regression can be traced to tokens or to the parser. - **I2** (tree): Run gparse and the perceptron parser with gold vs predicted tags, lemmas and tokens one at a time to rank which upstream module deserves effort. - **I3** (tokens): The crate en tokenizer could mark clitic boundaries ('s, n't, 'll) per character and derive both UD tokens and lemma stems from one shared decision. - **I4** (tokens): Break tokenizer and tagger errors down by in-vocabulary status and by word type, so one fix (e.g. a lexicon entry) can remove a whole error cluster. - **I5** (learn): Route witness-parser vs gold disagreements through an audit step before counting them, and feed confirmed gold errors into treebank cleanup. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/) Topics: [🌳 Dependency parsing](https://md.research.elamur.ai/free/en/topics/dependency-parsing/) · [🔤 Part-of-speech tagging](https://md.research.elamur.ai/free/en/topics/part-of-speech-tagging/) · [🧩 Morphology](https://md.research.elamur.ai/free/en/topics/morphology/) · [🐪 Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · [✂️ Tokenization & segmentation](https://md.research.elamur.ai/free/en/topics/tokenization/) ## Institutions - [Suez Canal University](https://md.research.elamur.ai/free/en/universities/suez-canal-university/) — University, Ismailia, Egypt · 2 works · 5 citations ## Cite Cite as: Emad Mohamed. 2011. The Effect of Automatic Tokenization, Vocalization, Stemming, and POS Tagging on Arabic Dependency Parsing. Proceedings of the Fifteenth Conference on Computational Natural Language Learning. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{mohamed2011effect, title = {{The Effect of Automatic Tokenization, Vocalization, Stemming, and POS Tagging on Arabic Dependency Parsing}}, author = {Emad Mohamed}, booktitle = {Proceedings of the Fifteenth Conference on Computational Natural Language Learning}, year = {2011}, url = {https://aclanthology.org/W11-0302/} } ``` - [← previous Exploring Difficulties in Parsing Imperatives and Questions](https://md.research.elamur.ai/free/en/papers/hara-2011-exploring-difficulties-parsing-imperatives-questions/) - [next → Measuring Language Development in Early Childhood Education: A Case Study of Grammar Checking in Child Language Transcripts](https://md.research.elamur.ai/free/en/papers/hassanali-2011-measuring-language-development-early-childhood/) - [↑ back to papers of 2011](https://md.research.elamur.ai/free/en/papers/2011/page/2/) ## References 11 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Buchholz, Sabine and Marsi, Erwin (2006). CoNLL-X shared task on multilingual dependency parsing. In Proceedings of the 10th Conference on Computational Natural Language Learning (CoNLL), pages 149–164. [our card](https://md.research.elamur.ai/free/en/papers/buchholz-2006-conll-x-shared-task-multilingual/) 2. Daelemans, Walter; Zavrel, Jakub; van der Sloot, Ko and van den Bosch, Antal (2007). TiMBL: Tilburg memory based learner – version 6.1 – reference guide. Technical Report ILK 07-09, Induction of Linguistic Knowledge, Computational Linguistics, Tilburg University. [find](https://scholar.google.com/scholar?q=Daelemans%2C+Walter%3B+Zavrel%2C+Jakub%3B+van+der+Sloot%2C+Ko+and+van+den+Bosch%2C+Antal+%282007%29%2E+TiMBL%3A+Tilburg+memory+based+learner+%E2%80%93+version+6%2E1+%E2%80%93+reference+guide%2E+Technical+Report+ILK+07%2D09%2C+Induction+of+Lingu) 3. Daelemans, Walter,; Zavrel, Jakub; an den Bosch, Antal, and van der Sloot, Ko (2007). MBT: Memory-Based Tagger- Version 3.1. Reference Guide. Technical Report ILK 07-09, Induction of Linguistic Knowledge, Computational Linguistics, Tilburg University. [find](https://scholar.google.com/scholar?q=Daelemans%2C+Walter%2C%3B+Zavrel%2C+Jakub%3B+an+den+Bosch%2C+Antal%2C+and+van+der+Sloot%2C+Ko+%282007%29%2E+MBT%3A+Memory%2DBased+Tagger%2D+Version+3%2E1%2E+Reference+Guide%2E+Technical+Report+ILK+07%2D09%2C+Induction+of+Linguistic+Knowle) 4. Hajič, Jan; Smrž, Otakar; Zemánek, Petr; Šnaidauf, Jan, and Beška, Emanuel (2004). Prague Arabic Dependency Treebank: Development in Data and Tools. In Proceedings of the EMLAR International Conference on Arabic Language Resources and Tools, pages 110-117, Cairo, Egypt, September 2004. [find](https://scholar.google.com/scholar?q=Haji%C4%8D%2C+Jan%3B+Smr%C5%BE%2C+Otakar%3B+Zem%C3%A1nek%2C+Petr%3B+%C5%A0naidauf%2C+Jan%2C+and+Be%C5%A1ka%2C+Emanuel+%282004%29%2E+Prague+Arabic+Dependency+Treebank%3A+Development+in+Data+and+Tools%2E+In+Proceedings+of+the+EMLAR+International+Conferenc) 5. Hall, Johan; Nilsson, Jens; Nivre, Joakim; Eryigit, Gülsen; Megyesi, Beáta; Nilsson, Mattias and Saers, Markus (2007). Single Malt or Blended? A Study in Multilingual Parser Optimization. In Proceedings of the CoNLL Shared Task Session of EMNLP-CoNLL 2007, 933-939. [find](https://scholar.google.com/scholar?q=Hall%2C+Johan%3B+Nilsson%2C+Jens%3B+Nivre%2C+Joakim%3B+Eryigit%2C+G%C3%BClsen%3B+Megyesi%2C+Be%C3%A1ta%3B+Nilsson%2C+Mattias+and+Saers%2C+Markus+%282007%29%2E+Single+Malt+or+Blended%3F+A+Study+in+Multilingual+Parser+Optimization%2E+In+Proceedin) 6. Maamouri, Mohamed and Bies, Ann (2004) Developing an Arabic Treebank: Methods, Guidelines, Procedures, and Tools. In Proceedings of the Workshop on Computational Approaches to Arabic Script-based Languages, COLING 2004, Geneva, August 28, 2004. [our card](https://md.research.elamur.ai/free/en/papers/maamouri-2004-developing-arabic-treebank-methods-guidelines/) 7. Marton, Yuval; Habash, Nizar; and Rambow, Owen (2010). Improving Arabic Dependency Parsing with Lexical and Inflectional Morphological Features. In Proceddings of The FirstWorkshop on Statistical Parsing of Morphologically Rich Languages (SPMRL 2010), LA, California. [our card](https://md.research.elamur.ai/free/en/papers/marton-2010-improving-arabic-dependency-parsing-lexical/) 8. McDonald, Ryan; Lerman, Kevin and Pereira, Fernando (2006). Multilingual dependency analysis with a two-stage discriminative parser. CoNLLX shared task on multilingual dependency parsing. In Proceedings of the 10th Conference on Computational Natural Language Learning Nivre, Joakim; Hall, Jonathan; Nilsson, Jens; Eryigit, Gülsen and Marinov, Svetsolav (2006). Labeled Pseudo-Projective Dependency Parsing with Support Vector Machines. In Proceedings of the Tenth Conference on Computational Natural Language Learning (CoNLL) [our card](https://md.research.elamur.ai/free/en/papers/mcdonald-2006-multilingual-dependency-analysis-two-stage/) 9. Nivre, Joakim; Hall, Johan; Kübler, Sandra; McDonald, Ryan; Nilsson, Jens; Riedel, Sebastian, and Yuret, Deniz. (2007). [find](https://scholar.google.com/scholar?q=Nivre%2C+Joakim%3B+Hall%2C+Johan%3B+K%C3%BCbler%2C+Sandra%3B+McDonald%2C+Ryan%3B+Nilsson%2C+Jens%3B+Riedel%2C+Sebastian%2C+and+Yuret%2C+Deniz%2E+%282007%29%2E) 10. The CoNLL 2007 shared task on dependency parsing. In Proceedings of the CoNLL Shared Task of EMNLP-CoNLL 2007, pages 915–932 [our card](https://md.research.elamur.ai/free/en/papers/nivre-2007-conll-2007-shared-task-dependency/) 11. Owen, Jonathan. The Arabic Grammatical Tradition. In Hetzron, Robert (ed.) (1997). The Semitic Languages. Routledge, London. [find](https://scholar.google.com/scholar?q=Owen%2C+Jonathan%2E+The+Arabic+Grammatical+Tradition%2E+In+Hetzron%2C+Robert+%28ed%2E%29+%281997%29%2E+The+Semitic+Languages%2E+Routledge%2C+London%2E)