---
title: "Developing and Using a Pilot Dialectal Arabic Treebank (2006)"
description: "Arabic dialects are spoken rather than written, so parsers lack dialect training data. 15 references, cited by 32 papers here."
authors: ["Mohamed Maamouri", "Ann Bies", "Tim Buckwalter", "Mona Diab", "Nizar Habash", "Owen Rambow", "Dalila Tabessi"]
year: 2006
venue: "Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC'06)"
source: "https://aclanthology.org/L06-1329/"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/maamouri-2006-developing-pilot-dialectal-arabic-treebank/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/maamouri-2006-developing-pilot-dialectal-arabic-treebank/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2006](https://md.research.elamur.ai/free/en/papers/2006/) / Developing and Using a Pilot Dialectal Arabic Treebank

# Developing and Using a Pilot Dialectal Arabic Treebank

**Authors:** [Mohamed Maamouri](https://md.research.elamur.ai/free/en/authors/mohamed-maamouri/), [Ann Bies](https://md.research.elamur.ai/free/en/authors/ann-bies/), [Tim Buckwalter](https://md.research.elamur.ai/free/en/authors/tim-buckwalter/), [Mona Diab](https://md.research.elamur.ai/free/en/authors/mona-diab/), [Nizar Habash](https://md.research.elamur.ai/free/en/authors/nizar-habash/), [Owen Rambow](https://md.research.elamur.ai/free/en/authors/owen-rambow/) and Dalila Tabessi · **Year:** 2006 · **Venue:** [Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC'06)](https://md.research.elamur.ai/free/en/venues/lrec/) ([LREC 2006](https://md.research.elamur.ai/free/en/venues/lrec/2006/)) · **Publisher:** [European Language Resources Association (ELRA)](https://md.research.elamur.ai/free/en/publishers/european-language-resources-association-elra/) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/L06-1329/) · [PDF](https://aclanthology.org/L06-1329.pdf) · [OpenAlex](https://openalex.org/W41492599)

**Citations:** 52 citations (OpenAlex, 2026-10-14) · [32](https://md.research.elamur.ai/free/en/papers/maamouri-2006-developing-pilot-dialectal-arabic-treebank/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Arabic dialects are spoken rather than written, so parsers lack dialect training data. For the 2005 JHU workshop on Parsing Arabic Dialects (PAD), the LDC quickly built a pilot Levantine Arabic Treebank (LATB). It covers about 26,000 words of Jordanian conversational telephone speech. The goal was a development and test resource, not a training set. Parsers were trained on Modern Standard Arabic (MSA) treebanks and then adapted to the dialect.

The MSA morphological analyzer could not handle the dialect, so annotation was done by hand. Annotators worked through a frequency-sorted wordlist. The existing MSA guidelines were adapted for disfluencies, verbal uses of active participles and word order.

In parallel, the team assembled Levantine–MSA dictionaries from four sources. In parsing tests, these dictionaries gave the largest gains. A small dictionary of closed-class and frequent words cut labeled constituent F1 error on the test set by over 10%.

The work shows that a small, quickly built dialect treebank plus a modest dialect-to-standard lexicon can guide tool development when a large treebank is out of reach.

## Key points
- LATB contains roughly 26,000 words of Levantine conversational telephone speech, collected at LDC in 2004 for the EARS project. It served PAD as development and test data.
- A modified Buckwalter MSA analyzer failed on the dialect, especially on verbal affixes and passive forms. MPG tagging (morphology, POS, English gloss) was therefore manual, with a few automatic consistency checks afterwards.
- Disfluencies were annotated in the style of the English Switchboard Treebank, using EDITED nodes for false starts.
- Active participles default to the adjective analysis, since they are negated with mi$ like adjectives, but they receive a verbal analysis when the context calls for it.
- VSO was kept as the underlying word order, matching MSA.
- Four sub-dictionaries were built: Automatic-Bridge, Egyptian-Cognate (2,500 lexeme pairs), Human-Checked and Simple-Modification.
  - Simple-Modification covers all 2,190 development types and took one lexicographer 8 hours.
  - 24.5% of those types needed changes. Most changes were morphological (46.1%), followed by orthographic, word-form and full translation changes.
  - When entries overlapped, Human-Checked was preferred over Egyptian-Cognate, which was preferred over English-Bridge.
- Two dictionary sizes were tested: a small one with 321 pairs and a large one with 1,560 extra pairs.
  - The large dictionary helped only when no POS tags were given in the input.
  - With gold POS tags, its extra gain disappeared, probably because it added translation ambiguity.

## Interesting ideas
- Annotate the most frequent surface forms first, then use pattern matching to propose prefix-stem-suffix splits for the remaining words. This gives maximum coverage early when no analyzer exists.
- Build a cheap bridge to a well-resourced variety by making minimal edits that push dialect word forms toward the standard form, so an existing parser can be reused.
- Merge lexical sources with an explicit trust ranking (human-checked beats cognate beats automatic bridge) instead of mixing them blindly.
- A small resource that covers closed-class and frequent words can beat a larger one, because extra entries bring extra ambiguity.
- Pick one underlying analysis (VSO here) and stay with it, while giving ambiguous categories such as participles a context-dependent dual label. This keeps the treebank consistent.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tokens): Unknown or rare English forms (slang, dialect spellings) could be handled by frequency-ordered review, with affix patterns proposing analyses for the long tail.
- **I2** (tree): Informal or dialectal English could be normalized toward standard spellings and morphology before the perceptron parser or gparse runs. Each rewrite would be logged so the step stays explainable.
- **I3** (learn): Silver data from different witness parsers and LLM teacher tiers could be combined with a declared precedence order for resolving conflicts.
- **I4** (world): Small, high-precision core mappings could be preferred when compiling WordNet/FrameNet/Wikidata knowledge, with larger noisy expansions tested under controls.
- **I5** (tree): UD dialect conversion could fix one canonical analysis per construction and mark participle-like words as adjectival or verbal by context, using diagnostics such as negation patterns.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Pennsylvania](https://md.research.elamur.ai/free/en/universities/university-of-pennsylvania/) — University, Philadelphia, United States · 221 works · 63,234 citations
- [Columbia University](https://md.research.elamur.ai/free/en/universities/columbia-university/) — University, New York, United States · 241 works · 54,053 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (2): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works · [🗂️ Treebanks](https://md.research.elamur.ai/free/en/topics/treebanks/) · 126 works

## Cite

Cite as: Mohamed Maamouri, Ann Bies, Tim Buckwalter, Mona Diab, Nizar Habash, Owen Rambow and Dalila Tabessi. 2006. Developing and Using a Pilot Dialectal Arabic Treebank. Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC'06).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{maamouri2006developing,
  title = {{Developing and Using a Pilot Dialectal Arabic Treebank}},
  author = {Mohamed Maamouri, Ann Bies, Tim Buckwalter, Mona Diab, Nizar Habash, Owen Rambow and Dalila Tabessi},
  booktitle = {Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC'06)},
  year = {2006},
  doi = {10.63317/3758yosch25d},
  url = {https://doi.org/10.63317/3758yosch25d}
}
```

- [← previous Cross Linguistic Name Matching in English and Arabic](https://md.research.elamur.ai/free/en/papers/freeman-2006-cross-linguistic-name-matching-english/)
- [next → Exploiting Syntactic Patterns as Clues in Zero-Anaphora Resolution](https://md.research.elamur.ai/free/en/papers/iida-2006-exploiting-syntactic-patterns-as-clues/)
- [↑ back to papers of 2006](https://md.research.elamur.ai/free/en/papers/2006/)

## References

15 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Bies, A. and Maamouri, M. (2003). Penn Arabic Treebank Guidelines. URL: [http://www.ircs.upenn.edu/arabic/Jan03release/guidelines-TB-1-28-03.pdf](http://www.ircs.upenn.edu/arabic/Jan03release/guidelines-TB-1-28-03.pdf) [↗](http://www.ircs.upenn.edu/arabic/Jan03release/guidelines-TB-1-28-03.pdf)
2. Bikel, D. (2004). On the Parameter Space of Generative Lexicalized Statistical Parsing Models. Ph.D. Dissertation. University of Pennsylvania. [find](https://scholar.google.com/scholar?q=Bikel%2C+D%2E+%282004%29%2E+On+the+Parameter+Space+of+Generative+Lexicalized+Statistical+Parsing+Models%2E+Ph%2ED%2E+Dissertation%2E+University+of+Pennsylvania%2E)
3. Brustad, K. (2000). The Syntax of Spoken Arabic: A comparative Study of Moroccan, Egyptian, Syrian, and Kuwaiti dialects. Washington, DC.: Georgetown University Press. [find](https://scholar.google.com/scholar?q=Brustad%2C+K%2E+%282000%29%2E+The+Syntax+of+Spoken+Arabic%3A+A+comparative+Study+of+Moroccan%2C+Egyptian%2C+Syrian%2C+and+Kuwaiti+dialects%2E+Washington%2C+DC%2E%3A+Georgetown+University+Press%2E)
4. Buckwalter, T. and Maamouri, M. (2004). Guidelines for the Transcription of Arabic Dialects (EARS). URL: [http://www.ldc.upenn.edu/Projects/EARS/Arabic/Guidelines_Levantine_MSA.htm](http://www.ldc.upenn.edu/Projects/EARS/Arabic/Guidelines_Levantine_MSA.htm). [↗](http://www.ldc.upenn.edu/Projects/EARS/Arabic/Guidelines_Levantine_MSA.htm)
5. Chiang, D., Diab, M., Habash, N., Rambow, O. and Shareef, S. (2006). Arabic Dialect Parsing. In Proceedings of the European chapter of the Association of Computational Linguistics EACL. [find](https://scholar.google.com/scholar?q=Chiang%2C+D%2E%2C+Diab%2C+M%2E%2C+Habash%2C+N%2E%2C+Rambow%2C+O%2E+and+Shareef%2C+S%2E+%282006%29%2E+Arabic+Dialect+Parsing%2E+In+Proceedings+of+the+European+chapter+of+the+Association+of+Computational+Linguistics+EACL%2E)
6. Diab, M., Hacioglu, K. and Jurafsky, D. (2004). [Automatic Tagging of Arabic Text: From raw text to Base Phrase Chunks](https://md.research.elamur.ai/free/en/papers/diab-2004-automatic-tagging-arabic-text-raw/). In Proceedings of HLT-NAACL 2004.
7. Eid, M. (1990). Perspectives on Arabic Linguistics I. Papers from the First Annual Symposium on Arabic Linguistics. Philadelphia: John Benjamins. [find](https://scholar.google.com/scholar?q=Eid%2C+M%2E+%281990%29%2E+Perspectives+on+Arabic+Linguistics+I%2E+Papers+from+the+First+Annual+Symposium+on+Arabic+Linguistics%2E+Philadelphia%3A+John+Benjamins%2E)
8. Habash, N. (2004). Large scale lexeme based Arabic morphological generation. In Proceedings of Traitement Automatique du Langage Naturel (TALN-04). Fez, Morocco. [find](https://scholar.google.com/scholar?q=Habash%2C+N%2E+%282004%29%2E+Large+scale+lexeme+based+Arabic+morphological+generation%2E+In+Proceedings+of+Traitement+Automatique+du+Langage+Naturel+%28TALN%2D04%29%2E+Fez%2C+Morocco%2E)
9. Habash, N. and Rambow, O. (2005). [Arabic Tokenization, Part-of-Speech Tagging and Morphological Disambiguation in One Fell Swoop](https://md.research.elamur.ai/free/en/papers/habash-2005-arabic-tokenization-part-speech-tagging/). In Proceedings of the Conference of American Association for Computational Linguistics (ACL’05). Ann Arbor, Michigan.
10. Levy, R. and Galen, A. (2006). Tsurgeon & Tregex. In Proceedings of LREC-2006. Genoa, Italy. [find](https://scholar.google.com/scholar?q=Levy%2C+R%2E+and+Galen%2C+A%2E+%282006%29%2E+Tsurgeon+%26+Tregex%2E+In+Proceedings+of+LREC%2D2006%2E+Genoa%2C+Italy%2E)
11. Maamouri, M. and Bies, A. (2004). [Developing an Arabic Treebank: Methods, Guidelines, Procedures, and Tools](https://md.research.elamur.ai/free/en/papers/maamouri-2004-developing-arabic-treebank-methods-guidelines/). In Proceedings of COLING 2004. Geneva, Switzerland.
12. Meteer, M. et al. (1995). Dysfluency Annotation Stylebook for the Switchboard Corpus. Revised by Ann Taylor, 1995. Ms., University of Pennsylvania. [find](https://scholar.google.com/scholar?q=Meteer%2C+M%2E+et+al%2E+%281995%29%2E+Dysfluency+Annotation+Stylebook+for+the+Switchboard+Corpus%2E+Revised+by+Ann+Taylor%2C+1995%2E+Ms%2E%2C+University+of+Pennsylvania%2E)
13. Mohammad, M. (2000). Word Order, Agreement and Pronominalization in Standard and Palestinian Arabic. Current Issues in Linguistic Theory. Philadelphia: John Benjamins. [↗](https://doi.org/10.1075/cilt.181)
14. Rambow, O., Chiang, D., Diab, M., Habash, N., Hwa, R., Lacey, V., Levy, R., Nicols, C., Shareef, S., Simaan, K. (2005). [Parsing Arabic Dialects](https://md.research.elamur.ai/free/en/papers/chiang-2006-parsing-arabic-dialects/). [↗](https://doi.org/10.7916/d8dj5q36)
15. Technical Report, The Johns-Hopkins University, 2005 Summer Research Workshop. [find](https://scholar.google.com/scholar?q=Technical+Report%2C+The+Johns%2DHopkins+University%2C+2005+Summer+Research+Workshop%2E)

*Updated 1 November 2026*