---
title: "A Multi-Dialect, Multi-Genre Corpus of Informal Written Arabic"
description: "The authors build a corpus of informal written Arabic that covers five dialect groups… 85% accuracy. 18 references, cited by 46 papers here."
authors: ["Ryan Cotterell", "Chris Callison-Burch"]
year: 2014
venue: "Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)"
source: "https://aclanthology.org/L14-1510/"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/cotterell-2014-multi-dialect-multi-genre-corpus/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/cotterell-2014-multi-dialect-multi-genre-corpus/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2014](https://md.research.elamur.ai/free/en/papers/2014/page/2/) / A Multi-Dialect, Multi-Genre Corpus of Informal Written Arabic

# A Multi-Dialect, Multi-Genre Corpus of Informal Written Arabic

**Authors:** [Ryan Cotterell](https://md.research.elamur.ai/free/en/authors/ryan-cotterell/) and [Chris Callison-Burch](https://md.research.elamur.ai/free/en/authors/chris-callison-burch/) · **Year:** 2014 · **Venue:** [Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)](https://md.research.elamur.ai/free/en/venues/lrec/) ([LREC 2014](https://md.research.elamur.ai/free/en/venues/lrec/2014/)) · **Publisher:** [European Language Resources Association (ELRA)](https://md.research.elamur.ai/free/en/publishers/european-language-resources-association-elra/) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/L14-1510/) · [PDF](https://aclanthology.org/L14-1510.pdf) · [OpenAlex](https://openalex.org/W2250278916)

**Citations:** 82 citations (OpenAlex, 2026-10-14) · [46](https://md.research.elamur.ai/free/en/papers/cotterell-2014-multi-dialect-multi-genre-corpus/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Most Arabic NLP tools and corpora target Modern Standard Arabic (MSA), but online user-generated text is largely dialectal. Because of diglossia, dialects differ from MSA at every linguistic level, so MSA-trained tools struggle with this text. The authors build a corpus of informal written Arabic that covers five dialect groups: Levantine, Gulf, Egyptian, Iraqi and Maghrebi. It draws on two genres, reader comments from newspaper websites and tweets. Unlike earlier work that labelled only a seed set and let a classifier tag the rest, every message here was judged by people on Amazon's Mechanical Turk. Annotators chose a dialect (or MSA) and a level of dialectness for each message, and hidden control items were used to check their quality. The paper also reports how individual workers performed. For dialect identification, it compares Naive Bayes and linear SVM classifiers built on word n-grams. Unigram Naive Bayes performed best and beat the earlier language-model approach on the Arabic Online Commentary data. The resource matters because dialect identification is a necessary first step before any dialect-specific processing.

## Key points
- Five dialect groups are covered. The newspaper sources are Al-Youm Al-Sabe' (Egypt), Al-Riyadh (Saudi Arabia), Al-Ghad (Jordan), Ech Chorouk El Youmi (Algeria) and Al-Wefaq (Iraq), and tweets were collected through the Twitter streaming API.
- There are two annotation groups of 100,000 messages each, one for commentary and one for Twitter. Each HIT held 10 sentences plus 2 controls taken from the Arabic Online Commentary Dataset.
- Workers had to pass a qualification "test" HIT: 230 tried it and 23 passed, all self-reported native Arabic speakers. These 23 finished every HIT over about 4 months.
- Full pay required 85% accuracy on the controls. Workers who scored above chance but below that threshold were paid in proportion to their correct answers.
- The classifiers were built in Scikit-learn and evaluated with 10-fold cross-validation, using unigram, bigram and trigram features. Unigrams generally won, and adding higher-order n-grams lowered accuracy.

## Interesting ideas
- Pre-labelled control items are hidden inside every batch, and pay is scaled by accuracy on them, which gives a continuous reliability score for each annotator.
- The authors warn that with one newspaper per dialect, a classifier may learn topic rather than dialect, and they suggest collecting several sources per class to expose this confound.
- Each message gets two labels: a category (which dialect) and a graded intensity (how dialectal), instead of a single hard choice.
- Higher-order n-grams hurt accuracy, probably because inconsistent informal spelling makes longer sequences too sparse to estimate reliably.
- Some short messages are ambiguous even for humans, so the authors argue for measuring a human error ceiling rather than expecting near-perfect accuracy.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Insert items with known answers into LLM teacher and witness-parser batches, so each source's trustworthiness is measured on the fly before its silver labels are accepted.
- **I2** (learn): Build negative controls that vary the source while holding the label fixed, to check that a learned rule or classifier is not exploiting a source artifact.
- **I3** (tokens): The tokenizer and tagger could attach a degree-of-nonstandardness score to each sentence, which would decide whether spelling normalization runs before tagging.
- **I4** (tokens): Normalize spelling variants to canonical lemmas first, and only then build contextual features, so that variation does not fragment the statistics.
- **I5** (answer): Allow readers to return an explicit "ambiguous" answer, and score them against measured human agreement rather than a single gold label.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Pennsylvania](https://md.research.elamur.ai/free/en/universities/university-of-pennsylvania/) — University, Philadelphia, United States · 221 works · 63,234 citations
- [Johns Hopkins University](https://md.research.elamur.ai/free/en/universities/johns-hopkins-university/) — University, Baltimore, United States · 240 works · 96,487 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works

## Cite

Cite as: Ryan Cotterell and Chris Callison-Burch. 2014. A Multi-Dialect, Multi-Genre Corpus of Informal Written Arabic. Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{cotterell2014multi,
  title = {{A Multi-Dialect, Multi-Genre Corpus of Informal Written Arabic}},
  author = {Ryan Cotterell and Chris Callison-Burch},
  booktitle = {Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC'14)},
  year = {2014},
  doi = {10.63317/3bqwpphzpx6n},
  url = {https://doi.org/10.63317/3bqwpphzpx6n}
}
```

- [← previous Major Life Event Extraction from Twitter based on Congratulations/Condolences Speech Acts](https://md.research.elamur.ai/free/en/papers/li-2014-major-life-event-extraction-twitter/)
- [next → Cancelling the Maxim of Quantity: Another challenge for a Gricean theory of Scalar Implicatures](https://md.research.elamur.ai/free/en/papers/fox-2014-cancelling-maxim-quantity-another-challenge/)
- [↑ back to papers of 2014](https://md.research.elamur.ai/free/en/papers/2014/page/2/)

## References

18 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Rania Al-Sabbagh and Roxana Girju. 2010. [Mining the web for the induction of a dialectical Arabic lexicon](https://md.research.elamur.ai/free/en/papers/sabbagh-2010-mining-web-induction-dialectical-arabic/). In LREC.
2. Rania Al-Sabbagh and Roxana Girju. 2012. [YADAC: Yet another dialectal Arabic corpus](https://md.research.elamur.ai/free/en/papers/sabbagh-2012-yadac-yet-another-dialectal-arabic/). In LREC, pages 2882– 2889.
3. Chris Callison-Burch and Mark Dredze. 2010. [Creating speech and language data with amazon’s mechanical turk](https://md.research.elamur.ai/free/en/papers/burch-2010-creating-speech-language-data-amazon/). In Proceedings of the NAACL HLT 2010 Workshop on Creating Speech and Language Data with Amazon’s Mechanical Turk, pages 1–12. Association for Computational Linguistics.
4. Bernard Comrie. 2013. The Major Languages of South Asia, the Middle East and Africa. Routledge. [find](https://scholar.google.com/scholar?q=Bernard+Comrie%2E+2013%2E+The+Major+Languages+of+South+Asia%2C+the+Middle+East+and+Africa%2E+Routledge%2E)
5. Mona Diab, Nizar Habash, Owen Rambow, Mohamed Altantawy, and Yassine Benajiba. 2010. COLABA: Arabic dialect annotation and processing. In LREC Workshop on Semitic Language Processing, pages 66–74. [find](https://scholar.google.com/scholar?q=Mona+Diab%2C+Nizar+Habash%2C+Owen+Rambow%2C+Mohamed+Altantawy%2C+and+Yassine+Benajiba%2E+2010%2E+COLABA%3A+Arabic+dialect+annotation+and+processing%2E+In+LREC+Workshop+on+Semitic+Language+Processing%2C+pages+66%E2%80%9374%2E)
6. Jacob Eisenstein. 2013. Phonological factors in social media writing. [find](https://scholar.google.com/scholar?q=Jacob+Eisenstein%2E+2013%2E+Phonological+factors+in+social+media+writing%2E)
7. Heba Elfardy and Mona Diab. 2012a. Token level identication of linguistic code switching. [find](https://scholar.google.com/scholar?q=Heba+Elfardy+and+Mona+Diab%2E+2012a%2E+Token+level+identication+of+linguistic+code+switching%2E)
8. Heba Elfardy and Mona T Diab. 2012b. [AIDA: Automatic identification and glossing of dialectal Arabic](https://md.research.elamur.ai/free/en/papers/elfardy-2012-aida-automatic-identification-glossing-dialectal/).
9. Heba Elfardy and Mona T Diab. 2012c. [Simplified guidelines for the creation of large scale dialectal Arabic annotations](https://md.research.elamur.ai/free/en/papers/elfardy-2012-simplified-guidelines-creation-large-scale/). In LREC, pages 371–378.
10. Heba Elfardy and Mona Diab. 2013. [Sentence level dialect identification in Arabic](https://md.research.elamur.ai/free/en/papers/elfardy-2013-sentence-level-dialect-identification-arabic/).
11. Heba Elfardy, Mohamed Al-Badrashiny, and Mona Diab. 2013. Code switch point detection in Arabic. In Natural Language Processing and Information Systems, pages 412–416. Springer. [find](https://scholar.google.com/scholar?q=Heba+Elfardy%2C+Mohamed+Al%2DBadrashiny%2C+and+Mona+Diab%2E+2013%2E+Code+switch+point+detection+in+Arabic%2E+In+Natural+Language+Processing+and+Information+Systems%2C+pages+412%E2%80%93416%2E+Springer%2E)
12. Paul McNamee. 2005. Language identification: a solved problem suitable for undergraduate instruction. Journal of Computing Sciences in Colleges, 20(3):94–101. [find](https://scholar.google.com/scholar?q=Paul+McNamee%2E+2005%2E+Language+identification%3A+a+solved+problem+suitable+for+undergraduate+instruction%2E+Journal+of+Computing+Sciences+in+Colleges%2C+20%283%29%3A94%E2%80%93101%2E)
13. Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. 2011. Scikit-learn: Machine learning in python. The Journal of Machine Learning Research, 12:2825–2830. [find](https://scholar.google.com/scholar?q=Fabian+Pedregosa%2C+Gae%CC%88l+Varoquaux%2C+Alexandre+Gramfort%2C+Vincent+Michel%2C+Bertrand+Thirion%2C+Olivier+Grisel%2C+Mathieu+Blondel%2C+Peter+Prettenhofer%2C+Ron+Weiss%2C+Vincent+Dubourg%2C+et+al%2E+2011%2E+Scikit%2Dlearn%3A+Mac)
14. Stephen Tratz, Douglas Briesch, Jamal Laoudi, and Clare Voss. 2013. [Tweet conversation annotation tool with a focus on an Arabic dialect, Moroccan Darija](https://md.research.elamur.ai/free/en/papers/tratz-2013-tweet-conversation-annotation-tool-focus/). LAW VII & ID, page 135.
15. Yi Yang and Jacob Eisenstein. 2013. A log-linear model for unsupervised text normalization. In Proc. of EMNLP. [find](https://scholar.google.com/scholar?q=Yi+Yang+and+Jacob+Eisenstein%2E+2013%2E+A+log%2Dlinear+model+for+unsupervised+text+normalization%2E+In+Proc%2E+of+EMNLP%2E)
16. Omar F Zaidan and Chris Callison-Burch. 2011a. [The Arabic online commentary dataset: an annotated dataset of informal Arabic with high dialectal content](https://md.research.elamur.ai/free/en/papers/zaidan-2011-arabic-online-commentary-dataset-annotated/). In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers-Volume 2, pages 37–41. Association for Computational Linguistics.
17. Omar F Zaidan and Chris Callison-Burch. 2011b. [Crowdsourcing translation: Professional quality from non-professionals](https://md.research.elamur.ai/free/en/papers/zaidan-2011-crowdsourcing-translation-professional-quality-non/). In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies-Volume 1, pages 1220– 1229. Association for Computational Linguistics.
18. Rabih Zbib, Erika Malchiodi, Jacob Devlin, David Stallard, Spyros Matsoukas, Richard Schwartz, John Makhoul, Omar F Zaidan, and Chris Callison-Burch. 2012. [Machine translation of Arabic dialects](https://md.research.elamur.ai/free/en/papers/zbib-2012-machine-translation-arabic-dialects/). In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 49–59. Association for Computational Linguistics.

*Updated 30 October 2026*