---
title: "NLP_DI at NADI 2024 shared task: Multi-label Arabic Dialect"
description: "The paper describes a team's entry for NADI 2024 Subtask 1, which asks systems to identify… 24 references, cited by 2 papers here."
authors: ["Vani Kanjirangat", "Tanja Samardzic", "Ljiljana Dolamic", "Fabio Rinaldi"]
year: 2024
venue: "Proceedings of the Second Arabic Natural Language Processing Conference"
source: "https://aclanthology.org/2024.arabicnlp-1.82/"
doi: "10.18653/v1/2024.arabicnlp-1.82"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/kanjirangat-2024-nlp-di-nadi-2024-shared/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/kanjirangat-2024-nlp-di-nadi-2024-shared/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/11/) / NLP\_DI at NADI 2024 shared task: Multi-label Arabic Dialect Classifications with an Unsupervised Cross-Encoder

# NLP\_DI at NADI 2024 shared task: Multi-label Arabic Dialect Classifications with an Unsupervised Cross-Encoder

**Authors:** [Vani Kanjirangat](https://md.research.elamur.ai/free/en/authors/vani-kanjirangat/), [Tanja Samardzic](https://md.research.elamur.ai/free/en/authors/tanja-samardzic/), [Ljiljana Dolamic](https://md.research.elamur.ai/free/en/authors/ljiljana-dolamic/) and [Fabio Rinaldi](https://md.research.elamur.ai/free/en/authors/fabio-rinaldi/) · **Year:** 2024 · **Venue:** [Proceedings of the Second Arabic Natural Language Processing Conference](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2024](https://md.research.elamur.ai/free/en/venues/arabicnlp/2024/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2024.arabicnlp-1.82/) · [PDF](https://aclanthology.org/2024.arabicnlp-1.82.pdf) · [DOI 10.18653/v1/2024.arabicnlp-1.82](https://doi.org/10.18653/v1/2024.arabicnlp-1.82) · [OpenAlex](https://openalex.org/W4402684042)

**Citations:** 2 citations (OpenAlex, 2026-10-14) · [2](https://md.research.elamur.ai/free/en/papers/kanjirangat-2024-nlp-di-nadi-2024-shared/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper describes a team's entry for NADI 2024 Subtask 1, which asks systems to identify country-level Arabic dialects in a multi-label way: one sentence may be valid in several dialects. The difficulty is that only single-label (multi-class) training data from earlier NADI editions exists. The development set has 100 sentences covering 8 dialects, and the 1000-sentence test set may involve up to 18.

The authors compared two families of approaches:
- **Supervised:** a MARBERTv2 classifier fine-tuned on NADI-2023, plus per-dialect binary classifiers.
- **Retrieval-based ("unsupervised"):** each input acts as a query against the labeled training set. Ten neighbours are retrieved using MARBERTv2 sentence embeddings and a multilingual mMARCO cross-encoder, and their labels are converted into a label set by counting rules.

Both families were then post-processed. Top-N filtering targets precision, while label expansion from dev-set co-occurrence or a confusion matrix targets recall.

Retrieval with filtering performed best, reaching 48.45% F on dev and 43.27% F on test (2nd place). The supervised variants lagged clearly behind. The result suggests that neighbour voting adapts single-label resources to multi-label tasks better than reshaping a classifier's output.

## Key points
- **Data:**
  - The training data is NADI-2023 (multi-class only).
  - In the dev set, 13 of the 100 items belong to none of the 8 dialects, about 12 belong to all 8, and most items carry at least two labels.
- **Retrieval pipeline:**
  - Sentence embeddings come from MARBERTv2 via Sentence Transformers.
  - A cross-encoder trained on mMARCO, the multilingual MS MARCO, is also used.
  - K=10 neighbours are retrieved, and the Un-Cross-LCT count heuristic converts their labels into a label set.
- **Dev results:**
  - Un-Cross + Post-FiltN (N=8) scored 48.45 F; the Un-Cross-LCT heuristic scored 45.57 F.
  - The supervised multi-class model scored only 8.45 F, and the best supervised variant (Sup-CM + FiltN) reached 37.37.
  - Lexical baselines did worse: Word_Jaccard 18.1, BPE_Jaccard 9.91, BM25 13.57.
- **Test results:**
  - The best F was 43.27 (Ranked 2), from Un-Cross + Post-FiltN (N=8).
  - Un-Cross-LCT reached the highest precision (64.0) but recall of only 21.4.
- **Supervised baseline quality:** the fine-tuned MARBERTv2 achieved 84% F on the NADI-2023 dev set, yet transferred poorly to the multi-label task.

## Interesting ideas
- **Labeling by retrieval:** a single-label annotated collection is treated as a searchable database, and a new item inherits the labels of its nearest examples, which yields multiple labels naturally.
- **Vote distribution decides label-set size:** a strong majority yields one label, while a flatter spread keeps several mid-count labels, so ambiguity is detected rather than forced into a single choice.
- **Two-sided post-processing:** label expansion (raising recall) and top-N filtering (raising precision) are separate steps, each tuned on dev, which makes the recall–precision trade-off explicit and adjustable.
- **Confusion matrix as a similarity map:** the errors of a single-label model reveal which classes genuinely overlap, and that overlap can be used to widen predictions.
- **Simple lexical baselines as controls:** Jaccard, BPE-Jaccard and BM25 were run alongside the neural methods to show that the gains come from meaning rather than surface overlap.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (frames): candidate VerbNet classes or frames for a clause could be proposed from the most similar annotated sentences, and the retrieved examples would serve as the explanation.
- **I2** (learn): when witness parsers disagree, the same rule could keep a single silver analysis only on clear majority and otherwise store a small ambiguous set instead of a false gold label.
- **I3** (tokens): ambiguous morphological or POS candidates could first be expanded from known confusable pairs and then pruned to the top N, giving a controlled ambiguity set for later stages.
- **I4** (tree): parser confusions such as obl vs nmod could define acceptable alternative relations when converting UD dialects or scoring against other treebanks.
- **I5** (learn): such shallow overlap scorers could serve as negative controls in evaluation, flagging tests that a non-structural method could already pass.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Zurich](https://md.research.elamur.ai/free/en/universities/university-of-zurich/) — University, Zurich, Switzerland · 78 works · 12,260 citations
- [Dalle Molle Institute for Artificial Intelligence Research](https://md.research.elamur.ai/free/en/labs/dalle-molle-institute-for-artificial-intelligence-research/) — Research institute, Lugano, Switzerland · 5 works · 18 citations
- [University of Applied Sciences and Arts of Southern Switzerland](https://md.research.elamur.ai/free/en/universities/university-of-applied-sciences-and-arts-of-southern-switzerland/) — University, Manno, Switzerland · 4 works · 17 citations
- [Bundesamt für Rüstung armasuisse](https://md.research.elamur.ai/free/en/government/bundesamt-fur-rustung-armasuisse/) — Government body, Bern, Switzerland · 3 works · 16 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works

## Cite

Cite as: Vani Kanjirangat, Tanja Samardzic, Ljiljana Dolamic and Fabio Rinaldi. 2024. NLP\_DI at NADI 2024 shared task: Multi-label Arabic Dialect Classifications with an Unsupervised Cross-Encoder. Proceedings of the Second Arabic Natural Language Processing Conference.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{kanjirangat2024nlp,
  title = {{NLP\_DI at NADI 2024 shared task: Multi-label Arabic Dialect Classifications with an Unsupervised Cross-Encoder}},
  author = {Vani Kanjirangat, Tanja Samardzic, Ljiljana Dolamic and Fabio Rinaldi},
  booktitle = {Proceedings of the Second Arabic Natural Language Processing Conference},
  year = {2024},
  doi = {10.18653/v1/2024.arabicnlp-1.82},
  url = {https://doi.org/10.18653/v1/2024.arabicnlp-1.82}
}
```

- [← previous Multilingual Nonce Dependency Treebanks: Understanding how Language Models Represent and Process Syntactic Structure](https://md.research.elamur.ai/free/en/papers/arps-2024-multilingual-nonce-dependency-treebanks-understanding/)
- [next → Normalized Orthography for Tunisian Arabic](https://md.research.elamur.ai/free/en/papers/turki-2024-normalized-orthography-tunisian-arabic/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/11/)

## References

24 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Luiz Bonifacio, Vitor Jeronymo, Hugo Queiroz Abonizio, Israel Campiotti, Marzieh Fadaee, Roberto Lotufo, and Rodrigo Nogueira. 2021. mmarco: A multilingual version of the ms marco passage ranking dataset. arXiv preprint arXiv:2108.13897. [↗](https://arxiv.org/abs/2108.13897)
2. Muhammad Abdul-Mageed, Amr Keleg, AbdelRahim Elmadany, Chiyu Zhang, Injy Hamed, Walid Magdy, Houda Bouamor, and Nizar Habash. 2024. [NADI 2024: The Fifth Nuanced Arabic Dialect Identification Shared Task](https://md.research.elamur.ai/free/en/papers/mageed-2024-nadi-2024-fifth-nuanced-arabic/). In Proceedings of The Second Arabic Natural Language Processing Conference (ArabicNLP 2024).
3. Muhammad Abdul-Mageed, Chiyu Zhang, Houda Bouamor, and Nizar Habash. 2020. [NADI 2020: The first nuanced Arabic dialect identification shared task](https://md.research.elamur.ai/free/en/papers/mageed-2020-nadi-2020-first-nuanced-arabic/). In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 97–110, Barcelona, Spain (Online). Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2010.11334)
4. Mohamed Ali. 2018. [Character level convolutional neural network for arabic dialect identification](https://md.research.elamur.ai/free/en/papers/ali-2018-character-level-convolutional-neural-network/). In Proceedings of the Fifth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2018), pages 122–127.
5. Badr AlKhamissi, Mohamed Gabr, Muhammad El-Nokrashy, and Khaled Essam. 2021. [Adapting marbert for improved arabic dialect identification: Submission to the nadi 2021 shared task](https://md.research.elamur.ai/free/en/papers/alkhamissi-2021-adapting-marbert-improved-arabic-dialect/). In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 260–264. [↗](https://doi.org/10.48550/arxiv.2103.01065)
6. Nouf AlShenaifi and Aqil Azmi. 2020. [Faheem at nadi shared task: Identifying the dialect of arabic tweet](https://md.research.elamur.ai/free/en/papers/alshenaifi-2020-faheem-nadi-shared-task-identifying/). In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 282–287. [↗](https://doi.org/10.48448/x2k4-g727)
7. Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, et al. 2016. [Ms marco: A human generated machine reading comprehension dataset](https://md.research.elamur.ai/free/en/papers/bajaj-2016-ms-marco-human-generated/). arXiv preprint arXiv:1611.09268. [↗](https://arxiv.org/abs/1611.09268)
8. Houda Bouamor, Sabit Hassan, and Nizar Habash. 2019. [The madar shared task on arabic fine-grained dialect identification](https://md.research.elamur.ai/free/en/papers/bouamor-2019-madar-shared-task-arabic-fine/). In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 199– 207.
9. Andrei Butnaru and Radu Tudor Ionescu. 2018. [Unibuckernel reloaded: First place in arabic dialect identification for the second year in a row](https://md.research.elamur.ai/free/en/papers/butnaru-2018-unibuckernel-reloaded-first-place-arabic/). In Proceedings of the Fifth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2018), pages 77–87. [↗](https://doi.org/10.48550/arxiv.1805.04876)
10. Çağrı Çöltekin, Taraka Rama, and Verena Blaschke. 2018. Tübingen-oslo team at the vardial 2018 evaluation campaign: An analysis of n-gram features in language variety identification. In Proceedings of the Fifth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2018), pages 55–65. [find](https://scholar.google.com/scholar?q=%C3%87ag%CC%86r%C4%B1+%C3%87%C3%B6ltekin%2C+Taraka+Rama%2C+and+Verena+Blaschke%2E+2018%2E+T%C3%BCbingen%2Doslo+team+at+the+vardial+2018+evaluation+campaign%3A+An+analysis+of+n%2Dgram+features+in+language+variety+identification%2E+In+Proceedings+o)
11. Abdellah El Mekki, Ahmed Alami, Hamza Alami, Ahmed Khoumsi, and Ismail Berrada. 2020. [Weighted combination of bert and n-gram features for nuanced arabic dialect identification](https://md.research.elamur.ai/free/en/papers/mekki-2020-weighted-combination-bert-n-gram/). In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 268–274.
12. Abdellah El Mekki, Abdelkader El Mahdaouy, Kabil Essefar, Nabil El Mamoun, Ismail Berrada, and Ahmed Khoumsi. 2021. [Bert-based multi-task model for country and province level msa and dialectal arabic identification](https://md.research.elamur.ai/free/en/papers/mekki-2021-bert-based-multi-task-model/). In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 271– 275.
13. Mohamed Elaraby and Muhammad Abdul-Mageed. 2018. [Deep models for arabic dialect identification on benchmarked data](https://md.research.elamur.ai/free/en/papers/elaraby-2018-deep-models-arabic-dialect-identification/). In Proceedings of the Fifth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2018), pages 263–274.
14. Salima Harrat, Karima Meftouh, Karima Abidi, and Kamel Smaïli. 2019. Automatic identification methods on a corpus of twenty five fine-grained arabic dialects. In International Conference on Arabic Language Processing, pages 79–92. Springer. [find](https://scholar.google.com/scholar?q=Salima+Harrat%2C+Karima+Meftouh%2C+Karima+Abidi%2C+and+Kamel+Sma%C3%AFli%2E+2019%2E+Automatic+identification+methods+on+a+corpus+of+twenty+five+fine%2Dgrained+arabic+dialects%2E+In+International+Conference+on+Arabic+Lan)
15. Mutaz Younes, Nour Al-Khdour, and AL-Smadi Mohammad. 2020. [Team alexa at nadi shared task](https://md.research.elamur.ai/free/en/papers/younes-2020-team-alexa-nadi-shared-task/). In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 237–242.
16. Omar Zaidan and Chris Callison-Burch. 2011. [The arabic online commentary dataset: an annotated dataset of informal arabic with high dialectal content](https://md.research.elamur.ai/free/en/papers/zaidan-2011-arabic-online-commentary-dataset-annotated/). In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 37–41.
17. Omar F Zaidan and Chris Callison-Burch. 2014. [Arabic dialect identification](https://md.research.elamur.ai/free/en/papers/zaidan-2014-arabic-dialect-identification/). Computational Linguistics, 40(1):171–202. [↗](https://doi.org/10.1162/coli_a_00169)
18. Marcos Zampieri, Shervin Malmasi, Nikola Ljubešić, Preslav Nakov, Ahmed Ali, Jörg Tiedemann, Yves Scherrer, and Noëmi Aepli. 2017. [Findings of the vardial evaluation campaign 2017](https://md.research.elamur.ai/free/en/papers/zampieri-2017-findings-vardial-evaluation-campaign/). In Proceedings of the fourth workshop on NLP for similar languages, varieties and dialects.
19. Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Ahmed Ali, Suwon Shon, James Glass, Yves Scherrer, Tanja Samardžić, Nikola Ljubešić, Jörg Tiedemann, et al. 2018. [Language identification and morphosyntactic tagging](https://md.research.elamur.ai/free/en/papers/zampieri-2018-language-identification-morphosyntactic-tagging-second/). the second vardial evaluation campaign.
20. Muhammad Abdul Mageed, Abdelrahim Elmadany, et al. 2021. [Arbert & marbert: Deep bidirectional transformers for arabic](https://md.research.elamur.ai/free/en/papers/abdul-mageed-2021-arbert-marbert-deep-bidirectional-transformers/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105. [↗](https://doi.org/10.48550/arxiv.2101.01785)
21. Shervin Malmasi, Marcos Zampieri, Nikola Ljubešić, Preslav Nakov, Ahmed Ali, and Jörg Tiedemann. 2016. [Discriminating between similar languages and arabic dialect identification: A report on the third dsl shared task](https://md.research.elamur.ai/free/en/papers/malmasi-2016-discriminating-between-similar-languages/). In Proceedings of the third workshop on NLP for similar languages, varieties and dialects (VarDial3), pages 1–14.
22. Nils Reimers and Iryna Gurevych. 2019. [Sentence-bert: Sentence embeddings using siamese bert-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992. [↗](https://doi.org/10.18653/v1/d19-1410)
23. Stephen E Robertson, Steve Walker, Micheline Hancock-Beaulieu, Aarron Gull, and Marianna Lau. 1992. Okapi at trec. In Text retrieval conference, pages 21–30. [find](https://scholar.google.com/scholar?q=Stephen+E+Robertson%2C+Steve+Walker%2C+Micheline+Hancock%2DBeaulieu%2C+Aarron+Gull%2C+and+Marianna+Lau%2E+1992%2E+Okapi+at+trec%2E+In+Text+retrieval+conference%2C+pages+21%E2%80%9330%2E)
24. Samia Touileb. 2020. [Ltg-st at nadi shared task 1: Arabic dialect identification using a stacking classifier](https://md.research.elamur.ai/free/en/papers/touileb-2020-ltg-st-nadi-shared-task/). In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 313–319.

*Updated 1 November 2026*