---
title: "Team Alexa at NADI Shared Task — paper by Younes et al. (2020)"
description: "The paper describes a system for country-level Arabic dialect identification in subtask 1… 35% accuracy. 16 references, cited by 3 papers here."
authors: ["Mutaz Younes", "Nour Al-khdour", "Mohammad AL-Smadi"]
year: 2020
venue: "Proceedings of the Fifth Arabic Natural Language Processing Workshop"
source: "https://aclanthology.org/2020.wanlp-1.22/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/younes-2020-team-alexa-nadi-shared-task/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/younes-2020-team-alexa-nadi-shared-task/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/9/) / Team Alexa at NADI Shared Task

# Team Alexa at NADI Shared Task

**Authors:** Mutaz Younes, Nour Al-khdour and [Mohammad AL-Smadi](https://md.research.elamur.ai/free/en/authors/mohammad-al-smadi/) · **Year:** 2020 · **Venue:** [Proceedings of the Fifth Arabic Natural Language Processing Workshop](https://md.research.elamur.ai/free/en/venues/wanlp/) ([WANLP 2020](https://md.research.elamur.ai/free/en/venues/wanlp/2020/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.wanlp-1.22/) · [PDF](https://aclanthology.org/2020.wanlp-1.22.pdf) · [OpenAlex](https://openalex.org/W3118759378)

**Citations:** 2 citations (OpenAlex, 2026-10-14) · [3](https://md.research.elamur.ai/free/en/papers/younes-2020-team-alexa-nadi-shared-task/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper describes a system for country-level Arabic dialect identification in subtask 1 of the NADI 2020 shared task. Each tweet must be assigned to one of 21 Arab countries, and the labelled data is small and heavily skewed toward a few countries. The official submission was deliberately simple. It combined word and character n-gram features (count and TF-IDF), weighted them by block, and fed them to a small ensemble of Multinomial Naive Bayes, a One-vs-Rest MNB and Logistic Regression. It reached 17% macro F1 and 35% accuracy, placing 12th of 18 teams. After the deadline, the authors fused the class probabilities of a fine-tuned bert-large-arabic (ArabicBERT) with MNB probabilities, using weights tuned on dev. This beat the task's best official score, with 27.73% F1 and 40.90% accuracy. The authors also audited the data with an MSA detector trained on MADAR. They estimate that over a fifth of the training and development tweets are actually MSA, a class the label set does not include. This points to label noise as a major ceiling on performance.

## Key points
- The NADI data has 21,000 train, 4,957 dev and 5,000 test tweets from 21 countries and 100 provinces. Egypt has 4,473 training tweets, while six countries have fewer than 240 each, and the model never predicted those six.
- The text-cleaning steps the authors tried all lowered scores, so they trained on raw tweets. The 10 million unlabeled tweets supplied by the organizers did not help in their experiments.
- The n-gram feature blocks were weighted as follows:
  - word unigrams and bigrams: 0.8
  - character 1–5-grams with word boundaries ("char wb"): 1.1
  - plain character n-grams ("char"): 1.0
- The post-evaluation fusion weighted bert-large-arabic probabilities at 1.4 and MNB probabilities at 0.35; BERT was trained for 2 epochs at a learning rate of 2e-5. Pre-trained embeddings from multilingual BERT and AraVec, used as input features, performed poorly.
- The MADAR-trained MSA detector flagged about 4,930 training and 1,074 development tweets as MSA.
  - Relabelling these as "MSA" raised MNB dev accuracy from about 35.2% to 43.8%.
  - F1 did not improve (14.9 in both settings). The authors explain that tweets mostly in MSA but containing a few dialect words get pulled into the new class.

## Interesting ideas
- Treat word-level and character-level n-gram blocks as separate feature groups with their own scaling factors; this lets subword evidence outweigh sparse word evidence without retraining anything else.
- Combine a strong neural model and a cheap sparse model by late fusion: multiply each model's class probabilities by a weight chosen on dev, then add them, instead of feeding embeddings in as features.
- Audit a labelled dataset with an independent classifier trained on a different corpus to find examples that belong to a class the label scheme forgot.
- Report accuracy and macro F1 side by side, because adding a missing "other" class can make one rise while the other stays flat, which shows what the change really does.
- A few hand-written keyword rules that nudge probabilities gave about +2% F1 but could misfire, so each rule needs its own error check.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tokens): The tagger and lemmatizer could give character-affix features their own tunable weight relative to word-identity features, which should help most on rare or misspelled words.
- **I2** (tree): Arc or label scores from gparse and the perceptron parser could be merged the same way, with interpretable dev-tuned weights, keeping each parser's contribution visible.
- **I3** (learn): A separate detector could flag silver or treebank sentences outside the expected genre or dialect before training, so they are excluded or labelled instead of silently adding noise.
- **I4** (learn): Evaluations of new repair rules or label additions should track both an overall and a per-class-averaged metric to catch fixes that only help the majority case.
- **I5** (learn): Self-induced repair rules should be kept only after measuring on held-out data both what they fix and what they break, as a built-in negative control.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works

## Cite

Cite as: Mutaz Younes, Nour Al-khdour and Mohammad AL-Smadi. 2020. Team Alexa at NADI Shared Task. Proceedings of the Fifth Arabic Natural Language Processing Workshop.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{younes2020team,
  title = {{Team Alexa at NADI Shared Task}},
  author = {Mutaz Younes, Nour Al-khdour and Mohammad AL-Smadi},
  booktitle = {Proceedings of the Fifth Arabic Natural Language Processing Workshop},
  year = {2020},
  url = {https://aclanthology.org/2020.wanlp-1.22/}
}
```

- [← previous Simple Data Augmentation with the Mask Token Improves Domain Adaptation for Dialog Act Tagging](https://md.research.elamur.ai/free/en/papers/yavuz-2020-simple-data-augmentation-mask-token/)
- [next → Toward Qualitative Evaluation of Embeddings for Arabic Sentiment Analysis](https://md.research.elamur.ai/free/en/papers/barhoumi-2020-qualitative-evaluation-embeddings-arabic-sentiment/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/9/)

## References

16 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Muhammad Abdul-Mageed, Chiyu Zhang, Houda Bouamor, and Nizar Habash. 2020. [NADI 2020: The First Nuanced Arabic Dialect Identification Shared Task](https://md.research.elamur.ai/free/en/papers/mageed-2020-nadi-2020-first-nuanced-arabic/). In Proceedings of the Fifth Arabic Natural Language Processing Workshop (WANLP 2020), Barcelona, Spain. [↗](https://doi.org/10.48550/arxiv.2010.11334)
2. Rania Al-Sabbagh and Roxana Girju. 2012. [Yadac: Yet another dialectal arabic corpus](https://md.research.elamur.ai/free/en/papers/sabbagh-2012-yadac-yet-another-dialectal-arabic/). In LREC, pages 2882– 2889.
3. Houda Bouamor, Nizar Habash, Mohammad Salameh, Wajdi Zaghouani, Owen Rambow, Dana Abdulrahim, Ossama Obeid, Salam Khalifa, Fadhl Eryani, Alexander Erdmann, and Kemal Oflazer. 2018. The MADAR Arabic dialect corpus and lexicon. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Miyazaki, Japan, May. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Houda+Bouamor%2C+Nizar+Habash%2C+Mohammad+Salameh%2C+Wajdi+Zaghouani%2C+Owen+Rambow%2C+Dana+Abdulrahim%2C+Ossama+Obeid%2C+Salam+Khalifa%2C+Fadhl+Eryani%2C+Alexander+Erdmann%2C+and+Kemal+Oflazer%2E+2018%2E+The+MADAR+Arabic+di)
4. Houda Bouamor, Sabit Hassan, and Nizar Habash. 2019. [The MADAR Shared Task on Arabic Fine-Grained Dialect Identification](https://md.research.elamur.ai/free/en/papers/bouamor-2019-madar-shared-task-arabic-fine/). In Proceedings of the Fourth Arabic Natural Language Processing Workshop (WANLP19), Florence, Italy.
5. Peter F Brown, Vincent J Della Pietra, Peter V Desouza, Jennifer C Lai, and Robert L Mercer. 1992. Class-based n-gram models of natural language. Computational linguistics, 18(4):467–480. [↗](https://doi.org/10.5555/176313.176316)
6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). arXiv preprint arXiv:1810.04805. [↗](https://arxiv.org/abs/1810.04805)
7. Mohamed Eldesouki, Fahim Dalvi, Hassan Sajjad, and Kareem Darwish. 2016. [Qcri@ dsl 2016: Spoken arabic dialect identification using textual features](https://md.research.elamur.ai/free/en/papers/eldesouki-2016-qcri-dsl-2016-spoken-arabic/). In Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial3), pages 221–226.
8. Heba Elfardy and Mona Diab. 2013. [Sentence level dialect identification in arabic](https://md.research.elamur.ai/free/en/papers/elfardy-2013-sentence-level-dialect-identification-arabic/). In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 456–461.
9. David G Kleinbaum, K Dietz, M Gail, Mitchel Klein, and Mitchell Klein. 2002. Logistic regression. Springer. [find](https://scholar.google.com/scholar?q=David+G+Kleinbaum%2C+K+Dietz%2C+M+Gail%2C+Mitchel+Klein%2C+and+Mitchell+Klein%2E+2002%2E+Logistic+regression%2E+Springer%2E)
10. Karima Meftouh, Karima Abidi, Salima Harrat, and Kamel Smaili. 2019. [The smart classifier for arabic fine-grained dialect identification](https://md.research.elamur.ai/free/en/papers/meftouh-2019-smart-classifier-arabic-fine-grained/).
11. Ali Safaya, Moutasem Abdullatif, and Deniz Yuret. 2020. [Kuisail at semeval-2020 task 12: Bert-cnn for offensive speech identification in social media](https://md.research.elamur.ai/free/en/papers/safaya-2020-kuisail-semeval-2020-task-12/).
12. Mohammad Salameh, Houda Bouamor, and Nizar Habash. 2018. [Fine-grained arabic dialect identification](https://md.research.elamur.ai/free/en/papers/salameh-2018-fine-grained-arabic-dialect-identification/). In Proceedings of the 27th International Conference on Computational Linguistics, pages 1332–1344.
13. Kenneth A Small and Cheng Hsiao. 1985. Multinomial logit specification tests. International economic review, pages 619–627. [↗](https://doi.org/10.2307/2526707)
14. Abu Bakr Soliman, Kareem Eissa, and Samhaa R El-Beltagy. 2017. Aravec: A set of arabic word embedding models for use in arabic nlp. Procedia Computer Science, 117:256–265. [find](https://scholar.google.com/scholar?q=Abu+Bakr+Soliman%2C+Kareem+Eissa%2C+and+Samhaa+R+El%2DBeltagy%2E+2017%2E+Aravec%3A+A+set+of+arabic+word+embedding+models+for+use+in+arabic+nlp%2E+Procedia+Computer+Science%2C+117%3A256%E2%80%93265%2E)
15. Bashar Talafha, Ali Fadel, Mahmoud Al-Ayyoub, Yaser Jararweh, AL-Smadi Mohammad, and Patrick Juola. 2019. [Team just at the madar shared task on arabic fine-grained dialect identification](https://md.research.elamur.ai/free/en/papers/talafha-2019-team-just-madar-shared-task/). In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 285–289. [↗](https://doi.org/10.60692/br9mc-4qp64)
16. Chiyu Zhang and Muhammad Abdul-Mageed. 2019. [No army, no navy: Bert semi-supervised learning of arabic dialects](https://md.research.elamur.ai/free/en/papers/zhang-2019-no-army-no-navy-bert/). In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 279–284.

*Updated 1 November 2026*