--- title: "YADAC: Yet another Dialectal Arabic Corpus (2012)" description: "The paper describes the first stage of YADAC, a written Dialectal Arabic corpus built from the Web. 11 references, cited by 33 papers here." authors: ["Rania Al-Sabbagh", "Roxana Girju"] year: 2012 venue: "Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12)" source: "https://aclanthology.org/L12-1387/" licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)" licence_url: "https://creativecommons.org/licenses/by-nc/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/sabbagh-2012-yadac-yet-another-dialectal-arabic/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/sabbagh-2012-yadac-yet-another-dialectal-arabic/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2012](https://md.research.elamur.ai/free/en/papers/2012/) / YADAC: Yet another Dialectal Arabic Corpus # YADAC: Yet another Dialectal Arabic Corpus **Authors:** [Rania Al-Sabbagh](https://md.research.elamur.ai/free/en/authors/rania-al-sabbagh/) and [Roxana Girju](https://md.research.elamur.ai/free/en/authors/roxana-girju/) · **Year:** 2012 · **Venue:** [Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12)](https://md.research.elamur.ai/free/en/venues/lrec/) ([LREC 2012](https://md.research.elamur.ai/free/en/venues/lrec/2012/)) · **Publisher:** [European Language Resources Association (ELRA)](https://md.research.elamur.ai/free/en/publishers/european-language-resources-association-elra/) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/) **Links:** [Publisher's page](https://aclanthology.org/L12-1387/) · [PDF](https://aclanthology.org/L12-1387.pdf) · [OpenAlex](https://openalex.org/W2251657882) **Citations:** 64 citations (OpenAlex, 2026-10-14) · [33](https://md.research.elamur.ai/free/en/papers/sabbagh-2012-yadac-yet-another-dialectal-arabic/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary The paper describes the first stage of YADAC, a written Dialectal Arabic corpus built from the Web. Existing dialect resources were small and mostly spoken. This stage covers Egyptian Arabic, with Levantine and Gulf varieties planned later. It adds two genres not used before in dialect corpora: Twitter posts and question–answer pairs from online knowledge market sites. Blogs and forums are also included. Harvesting used topic-neutral search queries built only from function words found exclusively in Egyptian Arabic. Filtering applied a threshold model based on those function words plus a dialect-specific affix. The authors argue this is more reliable than content words, which overlap heavily with Modern Standard Arabic. The paper also analyses spelling variation. It treats this variation as largely phonologically motivated, covering: - vowel-length spellings, - MSA-style versus speech-like consonant spellings, - hypercorrection. Finally, it reports early work on unsupervised POS tagging and base-phrase chunking, with results the authors call promising. After filtering, the corpus holds about 6M words, which the authors describe as larger than most comparable dialect corpora. ## Key points - **Query construction and crawl:** Queries combined at least three items drawn from a 1,527-entry list of Egyptian-exclusive function words. 15M permuted queries were sampled at random and crawled from May to November 2011, using two search engines per venue. - **Cleaning:** Removing markup, spam, media hits, broken links and cross-engine duplicates left about 11M words. - **Dialect filter evaluation:** A gold set of 1,000 items was rated 1–3 by two native speakers (Kappa 0.8), and 730 items were judged highly dialectal. - **Filter design and output:** The filter combines function words (threshold 0.20) with the progressive prefix b- (threshold 0.256). It favours precision over recall and leaves roughly 6M words. - **Spelling evidence and annotation:** - "very" appears 312 times with the MSA qaf spelling versus 1,557 times with the glottal-stop spelling. - "you (fem.)" appears 548 times with the short-vowel spelling versus 1,295 times with the long-vowel spelling. - The output uses compound morphological POS tags such as DT+SG_M_NN, along with base-phrase labels. ## Interesting ideas - Using closed-class words as harvesting queries makes retrieval independent of topic, and multi-word queries pull in longer stretches of real text rather than titles or captions. - Variety identification relies on function words and affixes rather than content words, because content vocabulary is shared across varieties while grammatical markers are not. - The authors deliberately tune the filter for high precision at the cost of recall, accepting data loss to keep the corpus clean. - Spelling variants are explained through their phonological source and through hypercorrection, instead of being treated as random noise. - Agreement between two annotators on a graded 1–3 dialectness scale provides a small but trusted gold set for validating an automatic filter. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (learn): Silver-data collection for informal or dialectal English could be seeded with queries built from distinctive function-word combinations, giving genre-diverse raw text that is not biased toward particular topics. - **I2** (tree): Before UD dialect conversion, a cheap and explainable detector could flag the source variety (for example AAVE or British forms) by counting dialect-specific function words and morphemes. - **I3** (learn): Witness-parser agreement thresholds could follow the same policy, discarding borderline sentences so that training silver stays reliable. - **I4** (tokens): Lemmatization of nonstandard English spellings (gonna, thru, should of) could map variants to lemmas via explicit sound-based rules, keeping each normalization traceable. - **I5** (learn): A few hundred doubly-annotated, graded items could serve as the fixed yardstick against which automatic filters are checked alongside negative controls. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [University of Illinois Urbana-Champaign](https://md.research.elamur.ai/free/en/universities/university-of-illinois-urbana-champaign/) — University, Urbana, United States · 184 works · 69,372 citations 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works 🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works ## Cite Cite as: Rania Al-Sabbagh and Roxana Girju. 2012. YADAC: Yet another Dialectal Arabic Corpus. Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{alsabbagh2012yadac, title = {{YADAC: Yet another Dialectal Arabic Corpus}}, author = {Rania Al-Sabbagh and Roxana Girju}, booktitle = {Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC'12)}, year = {2012}, doi = {10.63317/3rerzfv5rmys}, url = {https://doi.org/10.63317/3rerzfv5rmys} } ``` - [← previous Constrained Decoding for Text-Level Discourse Parsing](https://md.research.elamur.ai/free/en/papers/muller-2012-constrained-decoding-text-level-discourse/) - [next → Using Sense-labeled Discourse Connectives for Statistical Machine Translation](https://md.research.elamur.ai/free/en/papers/meyer-2012-sense-labeled-discourse-connectives-statistical/) - [↑ back to papers of 2012](https://md.research.elamur.ai/free/en/papers/2012/) ## References 11 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Alorifi, F. (2008). Automatic Identification of Arabic Dialects Using Hidden Markov Models. PhD Thesis, University of Pittsburg. [find](https://scholar.google.com/scholar?q=Alorifi%2C+F%2E+%282008%29%2E+Automatic+Identification+of+Arabic+Dialects+Using+Hidden+Markov+Models%2E+PhD+Thesis%2C+University+of+Pittsburg%2E) 2. Al-Sabbagh, R. and Girju, R. (2011). Finite-State Base-Phrases PSر €bر [VP]_rjE/ 3 _SG_M_VBD rys/SG_M_N #p%#52 PSM*ا [NP]_Alrys/DT+SG_M_N btAEnA/IN+1_PL_PP$ ع:J‚X*ا P(dM*ا [ADJP]_Alr}ys/DT+SG_M_NN AlmxlwE /DT+SG_M_JJ … Transducer Tagger for Egyptian Arabic. Technical Report: Semantics Frontiers Group, UIUC. [find](https://scholar.google.com/scholar?q=Al%2DSabbagh%2C+R%2E+and+Girju%2C+R%2E+%282011%29%2E+Finite%2DState+Base%2DPhrases+PS%D8%B1+%E2%82%ACb%D8%B1+%5BVP%5D%5FrjE%2F+3+%5FSG%5FM%5FVBD+rys%2FSG%5FM%5FN+%23p%25%2352+PSM%2A%D8%A7+%5BNP%5D%5FAlrys%2FDT%2BSG%5FM%5FN+btAEnA%2FIN%2B1%5FPL%5FPP%24+%D8%B9%3AJ%E2%80%9AX%2A%D8%A7+P%28dM%2A%D8%A7+%5BADJP%5D%5FAlr%7Dys%2FDT%2BSG%5FM%5FNN+Alm) 3. Biadsy,F., Hirschberg, J. and Habash, N. (2009). [Spoken Arabic Dialect Identification Using Phonotactic Modeling](https://md.research.elamur.ai/free/en/papers/biadsy-2009-spoken-arabic-dialect-identification-phonotactic/). EACL 2009 Workshop on Computational Approaches to Semitic Languages, Athens Buckwalter, T. (2002). Arabic Morphological Analyzer (AraMorph). Version 1.0.Linguistic Data Consortium, catalog No. LDC2002L49 and ISBN 1-58563-257-0 4. Canavan, A. and Zipperlen, G. (1996). [find](https://scholar.google.com/scholar?q=Canavan%2C+A%2E+and+Zipperlen%2C+G%2E+%281996%29%2E) 5. CALLFRIEND Egyptian Arabic, LDC, Philadelphia Canavan, A., Zipperlen, G. and Graff, D. (1997). CALLHOME Egyptian Arabic Speech. [find](https://scholar.google.com/scholar?q=CALLFRIEND+Egyptian+Arabic%2C+LDC%2C+Philadelphia+Canavan%2C+A%2E%2C+Zipperlen%2C+G%2E+and+Graff%2C+D%2E+%281997%29%2E+CALLHOME+Egyptian+Arabic+Speech%2E) 6. LDC, Philadelphia Dasigi, P. and Diab, M. (2011). [CODACT: Towards Identifying Orthographic Variants in Dialectal Arabic](https://md.research.elamur.ai/free/en/papers/dasigi-2011-codact-identifying-orthographic-variants-dialectal/). Proceedings of the 5 th International Joint Conference on Natural Language Processing, pages 318-326, Chaing Mai, Thailand, November 8-13, 2011 [↗](https://doi.org/10.7916/d8np2dqz) 7. Diab, M., Habash, N., Rambow, O., Altantawy, M. and Benajiba, Y. (2010). COLABA: Arabic Dialect Annotation and Processing. LREC Workshop on Semitic Language Processing, pages 66-74, Malta, May 2010 [find](https://scholar.google.com/scholar?q=Diab%2C+M%2E%2C+Habash%2C+N%2E%2C+Rambow%2C+O%2E%2C+Altantawy%2C+M%2E+and+Benajiba%2C+Y%2E+%282010%29%2E+COLABA%3A+Arabic+Dialect+Annotation+and+Processing%2E+LREC+Workshop+on+Semitic+Language+Processing%2C+pages+66%2D74%2C+Malta%2C+May+2010) 8. Duh, K. and Kirchhoff, K. (2005). [POS Tagging of Dialectal Arabic: A Minimally Supervised Approach](https://md.research.elamur.ai/free/en/papers/duh-2005-pos-tagging-dialectal-arabic-minimally/). Proceedings of the ACL Workshop on Computational Approaches to Semitic Languages, pages 55-62, Ann Arbor, June 2005 9. Habash, N., Diab, M. and Rambow, O. (2011). CODA: [Conventional Orthography for Dialectal Arabic](https://md.research.elamur.ai/free/en/papers/habash-2012-conventional-orthography-dialectal-arabic/) (CODA) Version 0.1 – July 2011. Technical Report: Center for Computational Learning Systems – Columbia University. 10. McNeil, K. and Faiza, M. (2011). Tunisian Arabic Corpus: Creating a Written Corpus of an Unwritten Language. Workshop on Arabic Corpus Linguistics (WACL), Lancaster University, 11-12 April 2011 [find](https://scholar.google.com/scholar?q=McNeil%2C+K%2E+and+Faiza%2C+M%2E+%282011%29%2E+Tunisian+Arabic+Corpus%3A+Creating+a+Written+Corpus+of+an+Unwritten+Language%2E+Workshop+on+Arabic+Corpus+Linguistics+%28WACL%29%2C+Lancaster+University%2C+11%2D12+April+2011) 11. Zaidan, O. and Callison-Burch, C. (2011). [The Arabic Online Commentary Dataset: An Annotated Dataset of Informal Arabic with High Dialectal Content](https://md.research.elamur.ai/free/en/papers/zaidan-2011-arabic-online-commentary-dataset-annotated/). Proceedings of the 49 th Annual Meeting of the ACL: Short Papers, pages 37-41, Oregon, June 2011 *Updated 1 November 2026*