---
title: "Country-level Arabic Dialect Identification Using Small Datasets"
description: "PMI is used to build per-dialect word lists automatically, with MSA vocabulary filtered out, and… 18.1 F1. 20 references, cited by 2 papers here."
authors: ["Maha J. Althobaiti"]
year: 2021
venue: "Proceedings of the Sixth Arabic Natural Language Processing Workshop"
source: "https://aclanthology.org/2021.wanlp-1.30/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/althobaiti-2021-country-level-arabic-dialect-identification/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/althobaiti-2021-country-level-arabic-dialect-identification/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/7/) / Country-level Arabic Dialect Identification Using Small Datasets with Integrated Machine Learning Techniques and Deep Learning Models

# Country-level Arabic Dialect Identification Using Small Datasets with Integrated Machine Learning Techniques and Deep Learning Models

**Authors:** [Maha J. Althobaiti](https://md.research.elamur.ai/free/en/authors/maha-althobaiti/) · **Year:** 2021 · **Venue:** [Proceedings of the Sixth Arabic Natural Language Processing Workshop](https://md.research.elamur.ai/free/en/venues/wanlp/) ([WANLP 2021](https://md.research.elamur.ai/free/en/venues/wanlp/2021/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2021.wanlp-1.30/) · [PDF](https://aclanthology.org/2021.wanlp-1.30.pdf) · [OpenAlex](https://openalex.org/W3156319105)

**Citations:** 5 citations (OpenAlex, 2026-10-14) · [2](https://md.research.elamur.ai/free/en/papers/althobaiti-2021-country-level-arabic-dialect-identification/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper tackles country-level Arabic dialect identification for the NADI 2021 shared task (subtask 1.2), where each tweet must be assigned to one of 21 countries using only a modest labelled set. The author compares classical learners (SVM, Logistic Regression, Random Forest), a stacked LR→SVM setup, a hard-voting ensemble, and two neural models (GRU, BiLSTM). Inputs include character and word n-grams, TFIDF, and features derived from Pointwise Mutual Information. PMI is used to build per-dialect word lists automatically, with MSA vocabulary filtered out, and to score each tweet against every dialect. Two ways of using the roughly 10M unlabelled tweets are also tested: co-training with three agreeing classifiers, and word2vec embeddings. The best system is an SVM with n-grams, dictionary flags and tweet-level PMI scores, reaching 18.94% macro-F1 on the test set. The stacked model has the highest precision (22.70%), and the neural models lag well behind. The work shows that cheap, interpretable lexical statistics outperform deep models when labelled data is scarce and unevenly distributed across classes.

## Key points
- Data: the NADI corpus contains 31,000 labelled tweets in 21 country dialects, split into 21,000 for training, 5,000 for development and 5,000 for test. It comes with about 10M unlabelled tweets, and six dialects have fewer than 250 training examples.
- Dictionaries: a word is added to a dialect's list when its PMI with that dialect exceeds 1.5. Words found in the subtask 1.1 MSA corpus are excluded, and each tweet also receives 21 summed PMI values, one per dialect.
- Results: the SVM with c2-5, w1-3, dict and PMI features scored 18.71 macro-F1 on dev and 18.94 on test. The integrated LR→SVM model scored about 18.1 F1 on test with higher accuracy (38.47), while BiLSTM and GRU with learned embeddings reached only about 13 F1 on dev.
- Co-training: the unanimous agreement of LR, SVM and RF was used to label 300K unlabelled tweets, producing the CoTraining labelled corpus. Training on this corpus (BiLSTM 12.63 F1) performed about the same as using embeddings pre-trained on it (12.85 F1).
- Error analysis: many errors were short tweets with no distinctive dialect words. Class imbalance also distorted PMI: a word shared by Egypt and Yemen received a high score only for Yemen, because Egypt had 4283 samples and Yemen 429.

## Interesting ideas
- Class-specific lexicons are built by keeping only words whose PMI passes a threshold, then removing words that also appear in a generic reference corpus (MSA), so each list keeps only the truly marked vocabulary. This is a cheap, inspectable alternative to learned cue detectors.
- An entire input is condensed into one summed association score per class, giving a very small, readable feature vector that sits alongside sparse n-grams. Each number can be traced back to the individual words that produced it.
- In the stacking design, one model's predictions become extra features for a second model, which raised precision while costing some recall. This is a simple knob for systems where a false positive is worse than a miss.
- Automatic labelling by unanimous agreement among three different classifiers was directly compared with simply pre-training embeddings on the same text, and the two came out level. This shows that agreement-based silver data should be benchmarked against a cheap alternative.
- The error analysis shows that association statistics computed on imbalanced sources favour the smaller class for shared words. This is a general warning for any corpus-derived lexicon.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): when inducing repair rules from silver data, cue words could be chosen by PMI with the error class and then filtered against a neutral corpus. This would keep each rule's trigger explainable.
- **I2** (frames): lemma-to-speech-act or lemma-to-irony PMI sums could act as transparent prior signals for prag. Each score could be justified by listing its contributing words.
- **I3** (tree): gparse's predicted heads and labels could be fed as features into the perceptron parser. The final, explainable parser could then decide when to trust the transformer.
- **I4** (learn): witness-parser silver data could be evaluated against a baseline that uses the same raw text without agreement. This would act as a negative control showing what the agreement filter actually contributes.
- **I5** (world): when compiling verb classes or value lexicons from uneven corpora, scores could be smoothed or class-balanced. This would stop sparsely covered sources from claiming shared words.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works

## Cite

Cite as: Maha J. Althobaiti. 2021. Country-level Arabic Dialect Identification Using Small Datasets with Integrated Machine Learning Techniques and Deep Learning Models. Proceedings of the Sixth Arabic Natural Language Processing Workshop.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{althobaiti2021country,
  title = {{Country-level Arabic Dialect Identification Using Small Datasets with Integrated Machine Learning Techniques and Deep Learning Models}},
  author = {Maha J. Althobaiti},
  booktitle = {Proceedings of the Sixth Arabic Natural Language Processing Workshop},
  year = {2021},
  url = {https://aclanthology.org/2021.wanlp-1.30/}
}
```

- [← previous Arabic Named Entity Recognition Using Transformer-based-CRF Model](https://md.research.elamur.ai/free/en/papers/qurishi-2021-arabic-named-entity-recognition-transformer/)
- [next → Decoding, Fast and Slow: A Case Study on Balancing Trade-Offs in Incremental, Character-level Pragmatic Reasoning](https://md.research.elamur.ai/free/en/papers/zarriess-2021-decoding-fast-slow-case-study/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/7/)

## References

20 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Muhammad Abdul-Mageed, Hassan Alhuzali, and Mohamed Elaraby. 2018. [You tweet what you speak: A city-level dataset of arabic dialects](https://md.research.elamur.ai/free/en/papers/mageed-2018-you-tweet-what-you-speak/). In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), pages 3653–3659.
2. Muhammad Abdul-Mageed, Chiyu Zhang, Houda Bouamor, and Nizar Habash. 2020. [NADI 2020: The First Nuanced Arabic Dialect Identification Shared Task](https://md.research.elamur.ai/free/en/papers/mageed-2020-nadi-2020-first-nuanced-arabic/). In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 97–110. [↗](https://doi.org/10.48550/arxiv.2010.11334)
3. Muhammad Abdul-Mageed, Chiyu Zhang, Abdel-Rahim Elmadany, Houda Bouamor, and Nizar Habash. 2021. [NADI 2021: The Second Nuanced Arabic Dialect Identification Shared Task](https://md.research.elamur.ai/free/en/papers/mageed-2021-nadi-2021-second-nuanced-arabic/). In Proceedings of the Sixth Arabic Natural Language Processing Workshop (WANLP 2021). [↗](https://doi.org/10.48550/arxiv.2103.08466)
4. Maha J Althobaiti. 2020a. [Automatic Arabic Dialect Identification Systems for Written Texts: A Survey](https://md.research.elamur.ai/free/en/papers/althobaiti-2020-automatic-arabic-dialect-identification-systems/). International Journal of Computational Linguistics (IJCL), 11(3):61–89. [↗](https://doi.org/10.48550/arxiv.2009.12622)
5. Maha J Althobaiti. 2020b. [Automatic Arabic Dialect Identification Systems for Written Texts: A Survey](https://md.research.elamur.ai/free/en/papers/althobaiti-2020-automatic-arabic-dialect-identification-systems/). arXiv preprint arXiv:2009.12622. [↗](https://arxiv.org/abs/2009.12622)
6. Avrim Blum and Tom Mitchell. 1998. [Combining labeled and unlabeled data with co-training](https://md.research.elamur.ai/free/en/papers/blum-1998-combining-labeled-unlabeled-data-co-training/). In Proceedings of the eleventh annual conference on Computational learning theory, pages 92–100. [↗](https://doi.org/10.1184/r1/6604187)
7. Houda Bouamor, Sabit Hassan, and Nizar Habash. 2019. [The MADAR shared task on Arabic fine-grained dialect identification](https://md.research.elamur.ai/free/en/papers/bouamor-2019-madar-shared-task-arabic-fine/). In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 199–207.
8. Karen Sparck Jones. 1972. A statistical interpretation of term specificity and its application in retrieval. Journal of documentation. [↗](https://doi.org/10.1108/eb026526)
9. Shervin Malmasi, Marcos Zampieri, Nikola Ljubešić, Preslav Nakov, Ahmed Ali, and Jörg Tiedemann. 2016. [Discriminating between similar languages and arabic dialect identification: A report on the third dsl shared task](https://md.research.elamur.ai/free/en/papers/malmasi-2016-discriminating-between-similar-languages/). In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial3), pages 1–14.
10. Christopher D Manning, Christopher D Manning, and Hinrich Schütze. 1999. Foundations of statistical natural language processing. MIT press. [find](https://scholar.google.com/scholar?q=Christopher+D+Manning%2C+Christopher+D+Manning%2C+and+Hinrich+Schu%CC%88tze%2E+1999%2E+Foundations+of+statistical+natural+language+processing%2E+MIT+press%2E)
11. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013a. [Efficient estimation of word representations in vector space](https://md.research.elamur.ai/free/en/papers/mikolov-2013-efficient-estimation-word-representations-vector/). In Proceedings of the International Conference on Learning Representations (ICLR 2013), pages 1–12. [↗](https://doi.org/10.48550/arxiv.1301.3781)
12. Tomáš Mikolov, Édouard Grave, Piotr Bojanowski, Christian Puhrsch, and Armand Joulin. 2018. [Advances in Pre-Training Distributed Word Representations](https://md.research.elamur.ai/free/en/papers/mikolov-2017-advances-pre-training-distributed-word/). In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), pages 52–55.
13. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013b. [Distributed Representations of Words and Phrases and their Compositionality](https://md.research.elamur.ai/free/en/papers/mikolov-2013-distributed-representations-words-phrases-their/). Advances in Neural Information Processing Systems, 26:3111–3119. [↗](https://doi.org/10.48550/arxiv.1310.4546)
14. Giovanni Molina, Fahad AlGhamdi, Mahmoud Ghoneim, Abdelati Hawwari, Nicolas Rey-Villamizar, Mona Diab, and Thamar Solorio. 2016. [Overview for the Second Shared Task on Language Identification in Code-Switched Data](https://md.research.elamur.ai/free/en/papers/molina-2016-overview-second-shared-task-language/). In Proceedings of the Second Workshop on Computational Approaches to Code Switching, pages 40–49.
15. Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. [Glove: Global vectors for word representation](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/). In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543. [↗](https://doi.org/10.3115/v1/d14-1162)
16. Radim Rehurek and Petr Sojka. 2010. Software framework for topic modelling with large corpora. In In Proceedings of the LREC 2010 workshop on new challenges for NLP frameworks, pages 46–50. [find](https://scholar.google.com/scholar?q=Radim+Rehurek+and+Petr+Sojka%2E+2010%2E+Software+framework+for+topic+modelling+with+large+corpora%2E+In+In+Proceedings+of+the+LREC+2010+workshop+on+new+challenges+for+NLP+frameworks%2C+pages+46%E2%80%9350%2E)
17. Hinrich Schütze, Christopher D Manning, and Prabhakar Raghavan. 2008. Introduction to information retrieval, volume 39. Cambridge University Press Cambridge. [find](https://scholar.google.com/scholar?q=Hinrich+Schu%CC%88tze%2C+Christopher+D+Manning%2C+and+Prabhakar+Raghavan%2E+2008%2E+Introduction+to+information+retrieval%2C+volume+39%2E+Cambridge+University+Press+Cambridge%2E)
18. Zhang Yun-tao, Gong Ling, and Wang Yong-cheng. 2005. An improved TF-IDF approach for text classification. Journal of Zhejiang University-Science A, 6(1):49–55. [↗](https://doi.org/10.1007/bf02842477)
19. Mohamed A Zahran, Ahmed Magooda, Ashraf Y Mahgoub, Hazem Raafat, Mohsen Rashwan, and Amir Atyia. 2015. Word representations in vector space and their applications for arabic. In International Conference on Intelligent Text Processing and Computational Linguistics, pages 430–443. Springer. [find](https://scholar.google.com/scholar?q=Mohamed+A+Zahran%2C+Ahmed+Magooda%2C+Ashraf+Y+Mahgoub%2C+Hazem+Raafat%2C+Mohsen+Rashwan%2C+and+Amir+Atyia%2E+2015%2E+Word+representations+in+vector+space+and+their+applications+for+arabic%2E+In+International+Conferen)
20. Marcos Zampieri, Shervin Malmasi, Nikola Ljubešić, Preslav Nakov, Ahmed Ali, Jörg Tiedemann, Yves Scherrer, and Noëmi Aepli. 2017. [Findings of the VarDial evaluation campaign 2017](https://md.research.elamur.ai/free/en/papers/zampieri-2017-findings-vardial-evaluation-campaign/). In Proceedings of the fourth workshop on NLP for similar languages, varieties and dialects.

*Updated 1 November 2026*