--- title: "The Arabic Online Commentary Dataset — Zaidan 2011" description: "Nearly all available Arabic text is Modern Standard Arabic (MSA), yet everyday speech uses regional dialects." authors: ["Omar F. Zaidan", "Chris Callison-Burch"] year: 2011 venue: "Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies" source: "https://aclanthology.org/P11-2007/" licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/zaidan-2011-arabic-online-commentary-dataset-annotated/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zaidan-2011-arabic-online-commentary-dataset-annotated/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2011](https://md.research.elamur.ai/free/en/papers/2011/) / The Arabic Online Commentary Dataset: an Annotated Dataset of Informal Arabic with High Dialectal Content # The Arabic Online Commentary Dataset: an Annotated Dataset of Informal Arabic with High Dialectal Content **Authors:** [Omar F. Zaidan](https://md.research.elamur.ai/free/en/authors/omar-f-zaidan/), [Chris Callison-Burch](https://md.research.elamur.ai/free/en/authors/chris-callison-burch/) · **Year:** 2011 · **Venue:** Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/) **Links:** [Publisher's page](https://aclanthology.org/P11-2007/) · [PDF](https://aclanthology.org/P11-2007.pdf) · [OpenAlex](https://openalex.org/W2109704865) **Citations:** 168 citations (openalex-snapshot · 2026-10-14) · [119](#cited-by) in our library, of the 140884 papers whose bibliographies we read ## Summary Nearly all available Arabic text is Modern Standard Arabic (MSA), yet everyday speech uses regional dialects. As a result, tools such as MT systems fail badly on dialectal input, largely because they lack dialectal training data. The authors build the Arabic Online Commentary (AOC) Dataset to address this. They harvested reader comments posted over about six months of 2010 on the websites of three newspapers: Al-Ghad (Jordan), Al-Riyadh (Saudi Arabia) and Al-Youm Al-Sabe' (Egypt). The result is a monolingual corpus of roughly 52M words in which dialectal content is common. They then used Amazon's Mechanical Turk to have native speakers label each sentence for how dialectal it is and which dialect it uses. Spam filtering relied on control sentences and expected per-source dialect distributions. About 108K sentences now carry agreed labels, and 41% of them are dialectal. As a baseline, per-class word trigram language models separate MSA from dialect with about 78% accuracy. The data and labels are public, giving dialect-focused NLP and MT work a substantial written resource. ## Key points - **Corpus scale:** the AOC contains about 1.4M comments drawn from 86.1K articles, totaling 52.1M words and roughly 3.1M sentences. - **Dialect coverage:** each source represents a different dialect: Levantine (Al-Ghad), Gulf (Al-Riyadh) and Egyptian (Al-Youm Al-Sabe'). - **Annotation setup:** - Workers saw screens of 10 sentences plus 2 MSA control sentences taken from article bodies. - Each screen paid $0.05 and was done by three different workers. - In total, 455 annotators took part, and 13.8% of assignments were rejected and reposted. - **Labeling outcome:** labels are assigned by majority vote, meaning at least 2 of 3 annotators agree. This yielded 63,555 MSA and 44,618 dialectal sentences, and about 2% of sentences without agreement were dropped. - **Classification results:** classifiers were SRILM word trigram models with modified Kneser-Ney smoothing, evaluated by 10-fold cross validation. - MSA vs. dialect: 77.8% accuracy. - Three-way dialect identification (LEV/GLF/EGY): 83.5% accuracy. - Four-way classification including MSA: 69.4% accuracy. ## Interesting ideas - **Comment sections as a source of informal text.** Reader comments are plentiful, publicly reachable and consistently structured, and their writing style is less formal. This makes them a cheap way to collect informal or low-resource language varieties, such as surzhyk or regional Ukrainian speech. - **Source origin as a geographic proxy.** Choosing outlets from particular countries gives a built-in signal of which dialect to expect. The same signal later doubles as a check on annotation quality. - **Two-part spammer detection.** Known-answer control items (formal article sentences) are mixed into each task. Each worker's label distribution is also compared against the prior expected for each source. Both checks transfer to any crowdsourcing task with predictable gold items. - **Per-class language models as classifiers.** Train one n-gram LM per class and assign each sentence to the class with the lowest perplexity. This is a simple, strong baseline for language or dialect identification that needs no feature engineering. - **Tuning for precision when mining a minority class.** If the aim is to collect the rare class, MSA can be favored by default, and a sentence is labeled dialectal only when its LM-score ratio passes a threshold. This raises precision at some cost to recall, which suits data-harvesting pipelines. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) ## Cited by Cited by 119 papers of our library cite it (of the 140884 whose bibliographies we read); 22 of them have a page here, 97 not yet. - [Arabic Dialect Identification in the Wild](https://md.research.elamur.ai/free/en/papers/abdelali-2020-arabic-dialect-identification-wild/) - [Camelira: An Arabic Multi-Dialect Morphological Disambiguator](https://md.research.elamur.ai/free/en/papers/aclbulk-2022-obeid-camelira-arabic-multi-dialect-morphological/) - [Parsing Arabic Dialects Revisited: New Benchmarks, Models, and Insights](https://md.research.elamur.ai/free/en/papers/aclbulk-2026-elshabrawy-parsing-arabic-dialects-revisited-new/) - [Leveraging Pretrained Word Embeddings for Part-of-Speech Tagging of Code Switching Data](https://md.research.elamur.ai/free/en/papers/alghamdi-2019-leveraging-pretrained-word-embeddings-part/) - [A Novel Dialect-Aware Framework for the Classification of Arabic Dialects and Emotions](https://md.research.elamur.ai/free/en/papers/alsadhan-2025-novel-dialect-aware-framework-classification/) - [Automatic Arabic Dialect Identification Systems for Written Texts: A Survey](https://md.research.elamur.ai/free/en/papers/althobaiti-2020-automatic-arabic-dialect-identification-systems/) - [Hierarchical Aggregation of Dialectal Data for Arabic Dialect Identification](https://md.research.elamur.ai/free/en/papers/baimukan-2022-hierarchical-aggregation-dialectal-data-arabic/) - [An Open Dataset and Model for Language Identification](https://md.research.elamur.ai/free/en/papers/burchell-2023-open-dataset-model-language-identification/) - [A Spelling Correction Corpus for Multiple Arabic Dialects](https://md.research.elamur.ai/free/en/papers/eryani-2020-spelling-correction-corpus-multiple-arabic/) - [The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models](https://md.research.elamur.ai/free/en/papers/inoue-2021-interplay-variant-size-task-type/) - [Automatic Language Identification in Texts: A Survey](https://md.research.elamur.ai/free/en/papers/jauhiainen-2019-automatic-language-identification-texts/) - [Optimizing the Size of Subword Vocabularies in Dialect Classification](https://md.research.elamur.ai/free/en/papers/kanjirangat-2023-optimizing-size-subword-vocabularies-dialect/) - [Exploring Data and Parameter Efficient Strategies for Arabic Dialect Identifications](https://md.research.elamur.ai/free/en/papers/kanjirangat-2025-exploring-data-parameter-efficient-strategies/) - [GlotLID: Language Identification for Low-Resource Languages](https://md.research.elamur.ai/free/en/papers/kargaran-2023-glotlid-language-identification-low-resource/) - [A Deep CNN Architecture with Novel Pooling Layer Applied to Two Sudanese Arabic Sentiment Datasets](https://md.research.elamur.ai/free/en/papers/mhamed-2022-deep-cnn-architecture-novel-pooling/) - [ASOS at NADI 2024 shared task: Bridging Dialectness Estimation and MSA Machine Translation for Arabic Language Enhancement](https://md.research.elamur.ai/free/en/papers/nacar-2024-asos-nadi-2024-shared-task/) - [CAMeL Tools: An Open Source Python Toolkit for Arabic Natural Language Processing](https://md.research.elamur.ai/free/en/papers/obeid-2020-camel-tools-open-source-python/) - [EgyBERT: A Large Language Model Pretrained on Egyptian Dialect Corpora](https://md.research.elamur.ai/free/en/papers/qarah-2024-egybert-large-language-model-pretrained/) - [Ramsa: A Large Sociolinguistically Rich Emirati Arabic Speech Corpus for ASR and TTS](https://md.research.elamur.ai/free/en/papers/sabbagh-2026-ramsa-large-sociolinguistically-rich-emirati/) - [Overview for the First Shared Task on Language Identification in Code-Switched Data](https://md.research.elamur.ai/free/en/papers/solorio-2014-overview-first-shared-task-language/) - [Arabic Dialect Identification](https://md.research.elamur.ai/free/en/papers/zaidan-2014-arabic-dialect-identification/) - [Machine Translation of Arabic Dialects](https://md.research.elamur.ai/free/en/papers/zbib-2012-machine-translation-arabic-dialects/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{zaidan2011arabic, title = {{The Arabic Online Commentary Dataset: an Annotated Dataset of Informal Arabic with High Dialectal Content}}, author = {Zaidan, Omar F. and Callison-Burch, Chris}, booktitle = {Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies}, year = {2011}, url = {https://aclanthology.org/P11-2007/} } ``` - [← previous Text Simplification using Typed Dependencies: A Comparision of the Robustness of Different Generation Strategies](https://md.research.elamur.ai/free/en/papers/siddharthan-2011-text-simplification-typed-dependencies-comparision/) - [next → The First Surface Realisation Shared Task: Overview and Evaluation Results](https://md.research.elamur.ai/free/en/papers/belz-2011-first-surface-realisation-shared-task/) - [↑ back to papers of 2011](https://md.research.elamur.ai/free/en/papers/2011/) ## References 7 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Fadi Biadsy, Julia Hirschberg, and Nizar Habash. 2009. Spoken arabic dialect identification using phonotactic modeling. In Proceedings of the EACL Workshop on Computational Approaches to Semitic Languages, pages 53–61. [find](https://scholar.google.com/scholar?q=Fadi+Biadsy%2C+Julia+Hirschberg%2C+and+Nizar+Habash%2E+2009%2E+Spoken+arabic+dialect+identification+using+phonotactic+modeling%2E+In+Proceedings+of+the+EACL+Workshop+on+Computational+Approaches+to+Semitic+Langu) 2. Mona Diab, Nizar Habash, Owen Rambow, Mohamed Altantawy, and Yassine Benajiba. 2010. COLABA: Arabic dialect annotation and processing. In LREC Workshop on Semitic Language Processing, pages 66– 74. [find](https://scholar.google.com/scholar?q=Mona+Diab%2C+Nizar+Habash%2C+Owen+Rambow%2C+Mohamed+Altantawy%2C+and+Yassine+Benajiba%2E+2010%2E+COLABA%3A+Arabic+dialect+annotation+and+processing%2E+In+LREC+Workshop+on+Semitic+Language+Processing%2C+pages+66%E2%80%93+74%2E) 3. Nizar Y. Habash. 2010. Introduction to Arabic Natural Language Processing. Morgan & Claypool. [find](https://scholar.google.com/scholar?q=Nizar+Y%2E+Habash%2E+2010%2E+Introduction+to+Arabic+Natural+Language+Processing%2E+Morgan+%26+Claypool%2E) 4. Ann Irvine and Alexandre Klementiev. 2010. Using Mechanical Turk to annotate lexicons for less commonly used languages. In Proceedings of the NAACL HLT Workshop on Creating Speech and Language Data With Amazon’s Mechanical Turk, pages 108–113. [find](https://scholar.google.com/scholar?q=Ann+Irvine+and+Alexandre+Klementiev%2E+2010%2E+Using+Mechanical+Turk+to+annotate+lexicons+for+less+commonly+used+languages%2E+In+Proceedings+of+the+NAACL+HLT+Workshop+on+Creating+Speech+and+Language+Data+Wi) 5. Yun Lei and John H. L. Hansen. 2011. Dialect classification via text-independent training and testing for arabic, spanish, and chinese. IEEE Transactions on Audio, Speech, and Language Processing, 19(1):85– 96. [find](https://scholar.google.com/scholar?q=Yun+Lei+and+John+H%2E+L%2E+Hansen%2E+2011%2E+Dialect+classification+via+text%2Dindependent+training+and+testing+for+arabic%2C+spanish%2C+and+chinese%2E+IEEE+Transactions+on+Audio%2C+Speech%2C+and+Language+Processing%2C+19%28) 6. Omar F. Zaidan and Chris Callison-Burch. 2011. Crowdsourcing translation: Professional quality from non-professionals. In Proceedings of ACL (this volume). [find](https://scholar.google.com/scholar?q=Omar+F%2E+Zaidan+and+Chris+Callison%2DBurch%2E+2011%2E+Crowdsourcing+translation%3A+Professional+quality+from+non%2Dprofessionals%2E+In+Proceedings+of+ACL+%28this+volume%29%2E) [our card](https://md.research.elamur.ai/free/en/papers/zaidan-2011-crowdsourcing-translation-professional-quality-non/) 7. Marc A. Zissman. 1996. Comparison of four approaches to automatic language identification of telephone speech. IEEE Transactions on Speech and Audio Processing, 4(1):31–44. [find](https://scholar.google.com/scholar?q=Marc+A%2E+Zissman%2E+1996%2E+Comparison+of+four+approaches+to+automatic+language+identification+of+telephone+speech%2E+IEEE+Transactions+on+Speech+and+Audio+Processing%2C+4%281%29%3A31%E2%80%9344%2E)