--- title: "iCompass Working Notes for the Nuanced Arabic Dialect" description: "This working-notes paper describes the iCompass entry for Subtask 1 of NADI 2022, the… 33.70 accuracy. 3 references, cited by 1 paper here." authors: ["Abir Messaoudi", "Chayma Fourati", "Hatem Haddad", "Moez BenHajhmida"] year: 2022 venue: "Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP)" source: "https://aclanthology.org/2022.wanlp-1.41/" doi: "10.18653/v1/2022.wanlp-1.41" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/messaoudi-2022-icompass-working-notes-nuanced-arabic/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/messaoudi-2022-icompass-working-notes-nuanced-arabic/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/7/) / iCompass Working Notes for the Nuanced Arabic Dialect Identification Shared task # iCompass Working Notes for the Nuanced Arabic Dialect Identification Shared task **Authors:** [Abir Messaoudi](https://md.research.elamur.ai/free/en/authors/abir-messaoudi/), [Chayma Fourati](https://md.research.elamur.ai/free/en/authors/chayma-fourati/), [Hatem Haddad](https://md.research.elamur.ai/free/en/authors/hatem-haddad/) and [Moez BenHajhmida](https://md.research.elamur.ai/free/en/authors/moez-benhajhmida/) · **Year:** 2022 · **Venue:** [Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP)](https://md.research.elamur.ai/free/en/venues/wanlp/) ([WANLP 2022](https://md.research.elamur.ai/free/en/venues/wanlp/2022/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2022.wanlp-1.41/) · [PDF](https://aclanthology.org/2022.wanlp-1.41.pdf) · [DOI 10.18653/v1/2022.wanlp-1.41](https://doi.org/10.18653/v1/2022.wanlp-1.41) · [OpenAlex](https://openalex.org/W4385573701) **Citations:** 5 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/messaoudi-2022-icompass-working-notes-nuanced-arabic/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary This working-notes paper describes the iCompass entry for Subtask 1 of NADI 2022, the shared task on country-level Arabic dialect identification. Given a tweet, a system must assign one of eighteen country labels. The organizers supplied training data of roughly 20k tweets plus a development set of about 4.9k. Class sizes are very uneven, with Egypt and Iraq far ahead of the other countries. The authors cleaned the text by stripping non-Arabic tokens, placeholders and emojis, splitting hashtags, and collapsing extra whitespace. They then compared four set-ups built on the dialect-oriented MARBERT model: - MARBERT v1 embeddings with a CNN head - MARBERT v2 embeddings with a CNN head - MARBERT v2 embeddings with a QRNN head - plain fine-tuning of MARBERT v2 Plain fine-tuning of MARBERT v2 performed best on the development data and became the single submitted run. It finished 4th on Test-A, 7th on Test-B, and 5th in the overall ranking, a few macro-F1 points below the winner. The paper shows that a dialect-focused pretrained model, fine-tuned directly, beats heavier classifier heads on this data. It also shows how class imbalance pulls macro-F1 down. ## Key points - The data covers 18 country dialects, with 20,398 training and 4,871 development sentences. Egypt and Iraq dominate the label distribution. - Development results (macro-F1 / accuracy) for each set-up: - MARBERT v1 + CNN: 0.12 / 0.39 - MARBERT v2 + CNN: 0.14 / 0.40 - MARBERT v2 + QRNN: 0.26 / 0.41 - fine-tuned MARBERT v2: 0.33 / 0.50 - The submitted model was MARBERT v2 trained for 4 epochs, with learning rate 2e-5, batch size 32 and maximum sequence length 64. Training ran in Google Colaboratory. - Official results: - Test-A (4,758 tweets, 18 known dialects): macro-F1 33.70, accuracy 51.91, rank 4. - Test-B (unknown country-level dialects): macro-F1 16.94, rank 7. - Average macro-F1: 25.32, rank 5. The winning team, rematchka, scored 27.06. - Confusion matrices show the classes with the most training data (Egypt, Iraq, KSA) are recognized best. Many Gulf tweets are misassigned to KSA, and Moroccan tweets are often labelled Algerian. ## Interesting ideas - The authors report macro-F1 next to accuracy and read the confusion matrices, which exposes a CNN head that keeps predicting the large classes despite decent accuracy. - The organizers added a second test set (Test-B) drawn from dialects not covered in training, and scores fell sharply there compared with the in-distribution set. - The authors compared a light head on top of pretrained embeddings against full fine-tuning, and direct fine-tuning won clearly on a small labelled set. - During preprocessing, hashtags are split into their component words instead of being dropped, so their content still reaches the model. - MARBERT's pretraining kept tweets that mix Arabic with foreign words, because dialect text is often code-switched, and the authors plan a model that also handles Latin-script dialect writing such as TUNIZI. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (learn): Mova's evaluation could report per-class or macro scores and confusion tables for tagger labels and relation types. This would catch a model that only gets frequent labels right. - **I2** (learn): Mova could keep a held-out out-of-domain test split, such as a different genre or English variety, as a standing check on the silver-trained parsers. - **I3** (tree): gparse experiments could include a fully fine-tuned baseline before adding extra layers on top of the transformer encoder. - **I4** (tokens): the en tokenizer could segment hashtags and camel-cased handles into words, so the tagger and lemmatizer see real tokens. - **I5** (tokens): the en pipeline could flag code-switched or non-standard-script spans explicitly instead of forcing them into English tags, which keeps later steps explainable. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works 🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 3,066 works ## Cite Cite as: Abir Messaoudi, Chayma Fourati, Hatem Haddad and Moez BenHajhmida. 2022. iCompass Working Notes for the Nuanced Arabic Dialect Identification Shared task. Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{messaoudi2022icompass, title = {{iCompass Working Notes for the Nuanced Arabic Dialect Identification Shared task}}, author = {Abir Messaoudi, Chayma Fourati, Hatem Haddad and Moez BenHajhmida}, booktitle = {Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP)}, year = {2022}, doi = {10.18653/v1/2022.wanlp-1.41}, url = {https://doi.org/10.18653/v1/2022.wanlp-1.41} } ``` - [← previous HeiMorph at SIGMORPHON 2022 Shared Task on Morphological Acquisition Trajectories](https://md.research.elamur.ai/free/en/papers/ramarao-2022-heimorph-sigmorphon-2022-shared-task/) - [next → IISERB Brains at SemEval-2022 Task 6: A Deep-learning Framework to Identify Intended Sarcasm in English](https://md.research.elamur.ai/free/en/papers/shekhawat-2022-iiserb-brains-semeval-2022-task/) - [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/7/) ## References 3 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Chayma Fourati, Hatem Haddad, Abir Messaoudi, Moez BenHajhmida, Aymen Ben Elhaj Mabrouk, and Malek Naski. 2021. [Introducing a large Tunisian Arabizi dialectal dataset for sentiment analysis](https://md.research.elamur.ai/free/en/papers/fourati-2021-introducing-large-tunisian-arabizi-dialectal/). In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 226–230, Kyiv, Ukraine (Virtual). Association for Computational Linguistics. 2. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2020a. [find](https://scholar.google.com/scholar?q=Muhammad+Abdul%2DMageed%2C+AbdelRahim+Elmadany%2C+and+El+Moatez+Billah+Nagoudi%2E+2020a%2E) 3. Arbert & Omar Zaidan and Chris Callison-Burch. 2011. [The arabic online commentary dataset: an annotated dataset of informal arabic with high dialectal content](https://md.research.elamur.ai/free/en/papers/zaidan-2011-arabic-online-commentary-dataset-annotated/). In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 37–41. *Updated 31 October 2026*