---
title: "Arabic Dialect Identification under Scrutiny: Limitations of"
description: "They introduce Maximal Accuracy, the best score any model can reach on a dataset… 0.72. recall. 42 references, cited by 21 papers here."
authors: ["Amr Keleg", "Walid Magdy"]
year: 2023
venue: "Proceedings of ArabicNLP 2023"
source: "https://aclanthology.org/2023.arabicnlp-1.31/"
doi: "10.18653/v1/2023.arabicnlp-1.31"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/keleg-2023-arabic-dialect-identification-under-scrutiny/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/keleg-2023-arabic-dialect-identification-under-scrutiny/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/9/) / Arabic Dialect Identification under Scrutiny: Limitations of Single-label Classification

# Arabic Dialect Identification under Scrutiny: Limitations of Single-label Classification

**Authors:** [Amr Keleg](https://md.research.elamur.ai/free/en/authors/amr-keleg/) and [Walid Magdy](https://md.research.elamur.ai/free/en/authors/walid-magdy/) · **Year:** 2023 · **Venue:** [Proceedings of ArabicNLP 2023](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2023](https://md.research.elamur.ai/free/en/venues/arabicnlp/2023/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2023.arabicnlp-1.31/) · [PDF](https://aclanthology.org/2023.arabicnlp-1.31.pdf) · [DOI 10.18653/v1/2023.arabicnlp-1.31](https://doi.org/10.18653/v1/2023.arabicnlp-1.31) · [OpenAlex](https://openalex.org/W4389518426)

**Citations:** 3 citations (OpenAlex, 2026-10-14) · [21](https://md.research.elamur.ai/free/en/papers/keleg-2023-arabic-dialect-identification-under-scrutiny/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper argues that treating Arabic Dialect Identification (ADI) as single-label classification is a core reason why systems seem to fail at separating country- and city-level micro-dialects. Many sentences, especially short ones, are equally natural in several dialects. Forcing one label onto them makes the gold annotations incomplete and the evaluation unfair. The authors survey how existing ADI datasets were built: manual annotation, translation, lexical seed cues and geo-tagging. They show that each method assigns one dialect per sentence and also has its own sources of label noise. They introduce Maximal Accuracy, the best score any model can reach on a dataset whose labels are single but whose sentences are ambiguous. Seven native speakers of different dialects then reviewed the errors an ADI model made on the QADI test set. About two thirds of the checked false positives were actually valid in the predicted dialect. The paper therefore recommends a multi-label framing in which native speakers judge validity only in their own dialect. This matters because single-label metrics hide model quality and make error analysis misleading.

## Key points
- Of 490 validated false positives, 325 (≈66%) were also correct in the dialect the model predicted.
- Counting those valid predictions as true positives raised macro-averaged precision from 0.61 to 0.86 and F1 from 0.56 to 0.72. Recall moved only from 0.55 to 0.63.
- The authors surveyed datasets including AOC, VarDial, MADAR6/MADAR26, DART, QADI and NADI2020–2023, grouped by how they were labeled.
- In parallel corpora such as MADAR, the same translated sentence can appear under several country labels, so a classifier is asked to give different answers for an identical input.
- The analysis was limited by the difficulty of recruiting speakers from all 18 NADI 2023 countries. Only misclassified QADI test sentences were annotated.

## Interesting ideas
- Estimate a ceiling ("maximal accuracy") for any benchmark whose single gold answer hides legitimately ambiguous inputs, so that scores are read against a realistic upper bound and not against 100%.
- Re-audit false positives by asking whether the prediction is also acceptable, then recompute corrected precision, recall and F1 next to the official numbers.
- Collect labels as independent per-class yes/no judgments instead of one choice from a fixed list, which also lets new classes be added later without re-annotating the whole dataset.
- Ask each judge only about what they are natively competent in, because annotators tend to over-identify their own variety when choosing among all options.
- Turning a parallel corpus into a classification set can create identical inputs with conflicting labels, so check for such duplicates.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): We could estimate how much of the parser's "error" on treebank gold comes from structures that are truly ambiguous, such as attachment choices, and report scores relative to that ceiling.
- **I2** (learn): When gparse or the perceptron parser disagrees with gold, a reviewer (human or LLM teacher) could mark the alternative as valid or wrong, which separates real repair targets from noise before rule induction.
- **I3** (frames): Assigning a verb sense to VerbNet classes or frames as separate binary decisions would let a use belong to several frames, and adding a frame later would not invalidate earlier annotations.
- **I4** (learn): Each witness parser or teacher tier could vote only on phenomena where it is known to be reliable, which reduces bias in agreed silver data.
- **I5** (tree): During UD dialect conversion and merging of English treebanks, we could flag identical sentences or fragments that received different trees, and treat them as ambiguity or conversion artifacts.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Edinburgh](https://md.research.elamur.ai/free/en/universities/university-of-edinburgh/) — University, Edinburgh, United Kingdom · 359 works · 77,352 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works

## Cite

Cite as: Amr Keleg and Walid Magdy. 2023. Arabic Dialect Identification under Scrutiny: Limitations of Single-label Classification. Proceedings of ArabicNLP 2023.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{keleg2023arabic,
  title = {{Arabic Dialect Identification under Scrutiny: Limitations of Single-label Classification}},
  author = {Amr Keleg and Walid Magdy},
  booktitle = {Proceedings of ArabicNLP 2023},
  year = {2023},
  doi = {10.18653/v1/2023.arabicnlp-1.31},
  url = {https://doi.org/10.18653/v1/2023.arabicnlp-1.31}
}
```

- [← previous A Cross-Linguistic Pressure for Uniform Information Density in Word Order](https://md.research.elamur.ai/free/en/papers/clark-2023-cross-linguistic-pressure-uniform-information/)
- [next → Arabic Fine-Grained Entity Recognition](https://md.research.elamur.ai/free/en/papers/liqreina-2023-arabic-fine-grained-entity-recognition/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/9/)

## References

42 references, 28 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ahmed Abdelali, Hamdy Mubarak, Younes Samih, Sabit Hassan, and Kareem Darwish. 2021. [QADI: Arabic dialect identification in the wild](https://md.research.elamur.ai/free/en/papers/abdelali-2021-qadi-arabic-dialect-identification-wild/). In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 1–10, Kyiv, Ukraine (Virtual). Association for Computational Linguistics.
2. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021a. [ARBERT & MARBERT: Deep bidirectional transformers for Arabic](https://md.research.elamur.ai/free/en/papers/abdul-mageed-2021-arbert-marbert-deep-bidirectional-transformers/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105, Online. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2101.01785)
3. Muhammad Abdul-Mageed, AbdelRahim Elmadany, Chiyu Zhang, ElMoatez Billah Nagoudi, Houda Bouamor, and Nizar Habash. 2023. [NADI 2023: The Fourth Nuanced Arabic Dialect Identification Shared Task](https://md.research.elamur.ai/free/en/papers/mageed-2023-nadi-2023-fourth-nuanced-arabic/). In Proceedings of The First Arabic Natural Language Processing Conference (ArabicNLP 2023).
4. Muhammad Abdul-Mageed, Chiyu Zhang, Houda Bouamor, and Nizar Habash. 2020a. [NADI 2020: The first nuanced Arabic dialect identification shared task](https://md.research.elamur.ai/free/en/papers/mageed-2020-nadi-2020-first-nuanced-arabic/). In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 97–110, Barcelona, Spain (Online). Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2010.11334)
5. Muhammad Abdul-Mageed, Chiyu Zhang, AbdelRahim Elmadany, Houda Bouamor, and Nizar Habash. 2021b. [NADI 2021: The second nuanced Arabic dialect identification shared task](https://md.research.elamur.ai/free/en/papers/mageed-2021-nadi-2021-second-nuanced-arabic/). In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 244–259, Kyiv, Ukraine (Virtual). Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2103.08466)
6. Muhammad Abdul-Mageed, Chiyu Zhang, AbdelRahim Elmadany, Houda Bouamor, and Nizar Habash. 2022. [NADI 2022: The third nuanced Arabic dialect identification shared task](https://md.research.elamur.ai/free/en/papers/mageed-2022-nadi-2022-third-nuanced-arabic/). In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), pages 85–97, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
7. Muhammad Abdul-Mageed, Chiyu Zhang, AbdelRahim Elmadany, and Lyle Ungar. 2020b. Toward micro-dialect identification in diaglossic and code-switched environments. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5855–5876, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Muhammad+Abdul%2DMageed%2C+Chiyu+Zhang%2C+AbdelRahim+Elmadany%2C+and+Lyle+Ungar%2E+2020b%2E+Toward+micro%2Ddialect+identification+in+diaglossic+and+code%2Dswitched+environments%2E+In+Proceedings+of+the+2020+Conference+)
8. Ibrahim Abu Farha and Walid Magdy. 2022. [The effect of Arabic dialect familiarity on data annotation](https://md.research.elamur.ai/free/en/papers/farha-2022-effect-arabic-dialect-familiarity-data/). In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), pages 399– 408, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
9. Ibrahim Abu Farha, Wajdi Zaghouani, and Walid Magdy. 2021. [Overview of the WANLP 2021 shared task on sarcasm and sentiment detection in Arabic](https://md.research.elamur.ai/free/en/papers/farha-2021-overview-wanlp-2021-shared-task/). In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 296–305, Kyiv, Ukraine (Virtual). Association for Computational Linguistics.
10. Kathrein Abu Kwaik, Motaz Saad, Stergios Chatzikyriakidis, and Simon Dobnik. 2018. [Shami: A corpus of Levantine Arabic dialects](https://md.research.elamur.ai/free/en/papers/kwaik-2018-shami-corpus-levantine-arabic-dialects/). In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA).
11. Tareq Al-Moslmi, Mohammed Albared, Adel Al-Shabi, Nazlia Omar, and Salwani Abdullah. 2018. Arabic senti-lexicon: Constructing publicly available language resources for arabic sentiment analysis. Journal of Information Science, 44(3):345–362. [↗](https://doi.org/10.1177/0165551516683908)
12. Ahmed Y Al-Obaidi and Venus W Samawi. 2016. Opinion mining: Analysis of comments written in arabic colloquial. In Proceedings of the World Congress on Engineering and Computer Science, volume 1. [find](https://scholar.google.com/scholar?q=Ahmed+Y+Al%2DObaidi+and+Venus+W+Samawi%2E+2016%2E+Opinion+mining%3A+Analysis+of+comments+written+in+arabic+colloquial%2E+In+Proceedings+of+the+World+Congress+on+Engineering+and+Computer+Science%2C+volume+1%2E)
13. Israa Alsarsour, Esraa Mohamed, Reem Suwaileh, and Tamer Elsayed. 2018. [DART: A large dataset of dialectal Arabic tweets](https://md.research.elamur.ai/free/en/papers/alsarsour-2018-dart-large-dataset-dialectal-arabic/). In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA).
14. Maha J. Althobaiti. 2020. [Automatic arabic dialect identification systems for written texts: A survey](https://md.research.elamur.ai/free/en/papers/althobaiti-2020-automatic-arabic-dialect-identification-systems/). [↗](https://doi.org/10.48550/arxiv.2009.12622)
15. Maha J. Althobaiti. 2022. Creation of annotated country-level dialectal arabic resources: An unsupervised approach. Natural Language Engineering, 28(5):607–648. [↗](https://doi.org/10.1017/s135132492100019x)
16. Nurpeiis Baimukan, Houda Bouamor, and Nizar Habash. 2022. [Hierarchical aggregation of dialectal data for Arabic dialect identification](https://md.research.elamur.ai/free/en/papers/baimukan-2022-hierarchical-aggregation-dialectal-data-arabic/). In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 4586–4596, Marseille, France. European Language Resources Association.
17. Gabriel Bernier-colborne, Cyril Goutte, and Serge Leger. 2023. Dialect and variant identification as a multi-label classification task: A proposal based on near-duplicate analysis. In Tenth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2023), pages 142–151, Dubrovnik, Croatia. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Gabriel+Bernier%2Dcolborne%2C+Cyril+Goutte%2C+and+Serge+Leger%2E+2023%2E+Dialect+and+variant+identification+as+a+multi%2Dlabel+classification+task%3A+A+proposal+based+on+near%2Dduplicate+analysis%2E+In+Tenth+Workshop+o)
18. Houda Bouamor, Nizar Habash, and Kemal Oflazer. 2014. [A multidialectal parallel corpus of Arabic](https://md.research.elamur.ai/free/en/papers/bouamor-2014-multidialectal-parallel-corpus-arabic/). In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 1240–1245, Reykjavik, Iceland. European Language Resources Association (ELRA).
19. Houda Bouamor, Nizar Habash, Mohammad Salameh, Wajdi Zaghouani, Owen Rambow, Dana Abdulrahim, Ossama Obeid, Salam Khalifa, Fadhl Eryani, Alexander Erdmann, and Kemal Oflazer. 2018. The MADAR Arabic dialect corpus and lexicon. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Houda+Bouamor%2C+Nizar+Habash%2C+Mohammad+Salameh%2C+Wajdi+Zaghouani%2C+Owen+Rambow%2C+Dana+Abdulrahim%2C+Ossama+Obeid%2C+Salam+Khalifa%2C+Fadhl+Eryani%2C+Alexander+Erdmann%2C+and+Kemal+Oflazer%2E+2018%2E+The+MADAR+Arabic+di)
20. Houda Bouamor, Sabit Hassan, and Nizar Habash. 2019. [The MADAR shared task on Arabic fine-grained dialect identification](https://md.research.elamur.ai/free/en/papers/bouamor-2019-madar-shared-task-arabic-fine/). In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 199–207, Florence, Italy. Association for Computational Linguistics.
21. Anis Charfi, Wajdi Zaghouani, Syed Hassan Mehdi, and Esraa Mohamed. 2019. [A fine-grained annotated multi-dialectal Arabic corpus](https://md.research.elamur.ai/free/en/papers/charfi-2019-fine-grained-annotated-multi-dialectal/). In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019), pages 198–204, Varna, Bulgaria. INCOMA Ltd.
22. Ryan Cotterell and Chris Callison-Burch. 2014. [A multi-dialect, multi-genre corpus of informal written Arabic](https://md.research.elamur.ai/free/en/papers/cotterell-2014-multi-dialect-multi-genre-corpus/). In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 241–245, Reykjavik, Iceland. European Language Resources Association (ELRA).
23. Mahmoud El-Haj. 2020. [Habibi - a multi dialect multi national Arabic song lyrics corpus](https://md.research.elamur.ai/free/en/papers/haj-2020-habibi-multi-dialect-multi-national/). In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 1318–1326, Marseille, France. European Language Resources Association.
24. Mahmoud El-Haj, Paul Rayson, and Mariam Aboelezz. 2018. [Arabic dialect identification in the context of bivalency and code-switching](https://md.research.elamur.ai/free/en/papers/haj-2018-arabic-dialect-identification-context-bivalency/). In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA).
25. Mihaela Gaman, Adrian-Gabriel Chifu, William Domingues, and Radu Tudor Ionescu. 2022. Frecdo: A large corpus for french cross-domain dialect identification. [find](https://scholar.google.com/scholar?q=Mihaela+Gaman%2C+Adrian%2DGabriel+Chifu%2C+William+Domingues%2C+and+Radu+Tudor+Ionescu%2E+2022%2E+Frecdo%3A+A+large+corpus+for+french+cross%2Ddomain+dialect+identification%2E)
26. Salima Harrat, Karima Meftouh, and Kamel Smaïli. 2017. Creating Parallel Arabic Dialect Corpus: Pitfalls to Avoid. In 18th International Conference on Computational Linguistics and Intelligent Text Processing (CICLING), Budapest, Hungary. [find](https://scholar.google.com/scholar?q=Salima+Harrat%2C+Karima+Meftouh%2C+and+Kamel+Sma%C3%AFli%2E+2017%2E+Creating+Parallel+Arabic+Dialect+Corpus%3A+Pitfalls+to+Avoid%2E+In+18th+International+Conference+on+Computational+Linguistics+and+Intelligent+Text+Pr)
27. Trang Ho. 2006-. Tatoeba: Collection of sentences and translations. Available online, Accessed: 10 September 2023. [find](https://scholar.google.com/scholar?q=Trang+Ho%2E+2006%2D%2E+Tatoeba%3A+Collection+of+sentences+and+translations%2E+Available+online%2C+Accessed%3A+10+September+2023%2E)
28. Fei Huang. 2015. [Improved Arabic dialect classification with social media data](https://md.research.elamur.ai/free/en/papers/huang-2015-improved-arabic-dialect-classification-social/). In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2118–2126, Lisbon, Portugal. Association for Computational Linguistics.
29. Amr Keleg, Sharon Goldwater, and Walid Magdy. 2023. [ALDi: Quantifying the arabic level of dialectness of text](https://md.research.elamur.ai/free/en/papers/keleg-2023-aldi-quantifying-arabic-level-dialectness/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Singapore. Association for Computational Linguistics.
30. Shervin Malmasi, Marcos Zampieri, Nikola Ljubešić, Preslav Nakov, Ahmed Ali, and Jörg Tiedemann. 2016. [Discriminating between similar languages and Arabic dialect identification: A report on the third DSL shared task](https://md.research.elamur.ai/free/en/papers/malmasi-2016-discriminating-between-similar-languages/). In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial3), pages 1–14, Osaka, Japan.
31. The COLING 2016 Organizing Committee. [find](https://scholar.google.com/scholar?q=The+COLING+2016+Organizing+Committee%2E)
32. Karen McNeil and Miled Faiza. 2010-. Tunisian arabic corpus (tac): 895,000 words. Available online, Accessed: 10 September 2023. [find](https://scholar.google.com/scholar?q=Karen+McNeil+and+Miled+Faiza%2E+2010%2D%2E+Tunisian+arabic+corpus+%28tac%29%3A+895%2C000+words%2E+Available+online%2C+Accessed%3A+10+September+2023%2E)
33. Karima Meftouh, Salima Harrat, Salma Jamoussi, Mourad Abbas, and Kamel Smaili. 2015. [Machine translation experiments on PADIC: A parallel Arabic DIalect corpus](https://md.research.elamur.ai/free/en/papers/meftouh-2015-machine-translation-experiments-padic-parallel/). In Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation, pages 26–34, Shanghai, China.
34. Hamdy Mubarak. 2018. Dial2msa: A tweets corpus for converting dialectal arabic to modern standard arabic. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Paris, France. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Hamdy+Mubarak%2E+2018%2E+Dial2msa%3A+A+tweets+corpus+for+converting+dialectal+arabic+to+modern+standard+arabic%2E+In+Proceedings+of+the+Eleventh+International+Conference+on+Language+Resources+and+Evaluation+%28)
35. Hamdy Mubarak and Kareem Darwish. 2014. [Using Twitter to collect a multi-dialectal corpus of Arabic](https://md.research.elamur.ai/free/en/papers/mubarak-2014-twitter-collect-multi-dialectal-corpus/). In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP), pages 1–7, Doha, Qatar. Association for Computational Linguistics.
36. Ahmed Salama, Houda Bouamor, Behrang Mohit, and Kemal Oflazer. 2014. [YouDACC: the Youtube dialectal Arabic comment corpus](https://md.research.elamur.ai/free/en/papers/salama-2014-youdacc-youtube-dialectal-arabic-comment/). In Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14), pages 1246– 1251, Reykjavik, Iceland. European Language Resources Association (ELRA).
37. Wajdi Zaghouani and Anis Charfi. 2018. Arap-tweet: A large multi-dialect Twitter corpus for gender, age and language variety identification. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Wajdi+Zaghouani+and+Anis+Charfi%2E+2018%2E+Arap%2Dtweet%3A+A+large+multi%2Ddialect+Twitter+corpus+for+gender%2C+age+and+language+variety+identification%2E+In+Proceedings+of+the+Eleventh+International+Conference+on+)
38. Omar F. Zaidan and Chris Callison-Burch. 2011. [The Arabic online commentary dataset: an annotated dataset of informal Arabic with high dialectal content](https://md.research.elamur.ai/free/en/papers/zaidan-2011-arabic-online-commentary-dataset-annotated/). In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 37–41, Portland, Oregon, USA. Association for Computational Linguistics.
39. Omar F. Zaidan and Chris Callison-Burch. 2014. [Arabic dialect identification](https://md.research.elamur.ai/free/en/papers/zaidan-2014-arabic-dialect-identification/). Computational Linguistics, 40(1):171–202. [↗](https://doi.org/10.1162/coli_a_00169)
40. Marcos Zampieri, Shervin Malmasi, Nikola Ljubešić, Preslav Nakov, Ahmed Ali, Jörg Tiedemann, Yves Scherrer, and Noëmi Aepli. 2017. [Findings of the VarDial evaluation campaign 2017](https://md.research.elamur.ai/free/en/papers/zampieri-2017-findings-vardial-evaluation-campaign/). In Proceedings of the Fourth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial), pages 1–15, Valencia, Spain. Association for Computational Linguistics.
41. Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Ahmed Ali, Suwon Shon, James Glass, Yves Scherrer, Tanja Samardžić, Nikola Ljubešić, Jörg Tiedemann, Chris van der Lee, Stefan Grondelaers, Nelleke Oostdijk, Dirk Speelman, Antal van den Bosch, Ritesh Kumar, Bornini Lahiri, and Mayank Jain. 2018. [Language identification and morphosyntactic tagging: The second VarDial evaluation campaign](https://md.research.elamur.ai/free/en/papers/zampieri-2018-language-identification-morphosyntactic-tagging-second/). In Proceedings of the Fifth Workshop on NLP for Similar Languages, Varieties and Dialects (Var-Dial 2018), pages 1–17, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
42. Marcos Zampieri, Kai North, Tommi Jauhiainen, Mariano Felice, Neha Kumari, Nishant Nair, and Yash Bangera. 2023. Language variety identification with true labels. [find](https://scholar.google.com/scholar?q=Marcos+Zampieri%2C+Kai+North%2C+Tommi+Jauhiainen%2C+Mariano+Felice%2C+Neha+Kumari%2C+Nishant+Nair%2C+and+Yash+Bangera%2E+2023%2E+Language+variety+identification+with+true+labels%2E)

*Updated 1 November 2026*