---
title: "SANA at NADI 2023 shared task: Ensemble of Layer-Wise"
description: "The paper tackles country-level identification of Arabic dialects in tweets. 85.43 F1. 22 references, cited by 2 papers here."
authors: ["Nada Almarwani", "Samah Aloufi"]
year: 2023
venue: "Proceedings of ArabicNLP 2023"
source: "https://aclanthology.org/2023.arabicnlp-1.65/"
doi: "10.18653/v1/2023.arabicnlp-1.65"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/almarwani-2023-sana-nadi-2023-shared-task/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/almarwani-2023-sana-nadi-2023-shared-task/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/15/) / SANA at NADI 2023 shared task: Ensemble of Layer-Wise BERT-based models for Dialectal Arabic Identification

# SANA at NADI 2023 shared task: Ensemble of Layer-Wise BERT-based models for Dialectal Arabic Identification

**Authors:** [Nada Almarwani](https://md.research.elamur.ai/free/en/authors/nada-almarwani/) and [Samah Aloufi](https://md.research.elamur.ai/free/en/authors/samah-aloufi/) · **Year:** 2023 · **Venue:** [Proceedings of ArabicNLP 2023](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2023](https://md.research.elamur.ai/free/en/venues/arabicnlp/2023/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2023.arabicnlp-1.65/) · [PDF](https://aclanthology.org/2023.arabicnlp-1.65.pdf) · [DOI 10.18653/v1/2023.arabicnlp-1.65](https://doi.org/10.18653/v1/2023.arabicnlp-1.65) · [OpenAlex](https://openalex.org/W4389518440)

**Citations:** 1 citations (OpenAlex, 2026-10-14) · [2](https://md.research.elamur.ai/free/en/papers/almarwani-2023-sana-nadi-2023-shared-task/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper tackles country-level identification of Arabic dialects in tweets. This is subtask 1 (closed track) of the NADI-2023 shared task. Dialectal Arabic now dominates informal online writing, and many downstream tools need to know which variety a text uses. The authors work with the TWT-23 dataset: 18 dialect classes, balanced splits of 18K training, 1,800 development and 3,600 test tweets.

They do not rely on the last layer of MARBERTv2. Instead they train one model per encoder layer. Each model mean-pools that layer's hidden states and passes the result to a linear classifier. Keeping the embedding layer frozen during fine-tuning worked better than updating it. The per-layer models are then combined by soft voting, and an ablation over layer ranges picks the best ensemble, which covers layers 6–11.

The submission reached 85.43 macro-F1 on the test set and placed fourth of sixteen. An error analysis traces the remaining mistakes to three causes: absent diacritics, the internal diversity of Saudi Arabic, and confusions between closely related dialect families.

## Key points
- **Data:** TWT-23 contains 23,400 tweets across 18 country dialects, with 1,000 training and 100 development examples per class.
- **Single layers:** Mean-pooled representations from higher MARBERTv2 layers did better. The strongest single model used layer 10 with frozen embeddings and scored 0.855 dev macro-F1.
- **Ensembles:** The soft-voting ensemble of layers 6–11 reached 0.874 dev F1 with frozen embeddings and 0.857 with full fine-tuning. Using all layers 1–11 or only the topmost few gave lower scores.
- **Training settings:** lr 2e-5, weight decay 1e-2, max length 512, batch size 8, 10 epochs, keeping the checkpoint that scored best on dev. MARBERTv2 outperformed AraBERT.
- **Test results:** 85.43 F1 and 85.39 accuracy, against the winner NLPeople at 87.27. Precision fell below 0.80 only for KSA (0.75) and UAE (0.76). Many wrong predictions were pulled toward the KSA label.

## Interesting ideas
- Give each encoder layer its own small mean-pooled classifier, which shows how useful each depth is for the task without any architecture changes.
- Combine layer-specific predictors by averaging their probabilities, and use ablation to choose the layer window. A mid-to-top window beat both "all layers" and "top layers only."
- Freezing the embedding layer while fine-tuning the rest gave consistent gains on a modest-sized dataset. This likely works by guarding pretrained lexical knowledge against overfitting.
- The error analysis found a "magnet" class (KSA) that soaks up false positives because it is internally diverse and over-represented. This explains low precision better than a single overall score does.
- Some errors come from spelling that drops disambiguating information, here diacritics. The authors treat this as genuine ambiguity, sometimes unresolvable even by humans, rather than as model failure.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tree): gparse could fit a cheap probe per layer to decide which transformer layers should feed the arc and label scorers, instead of defaulting to the final layer.
- **I2** (learn): the same ablation-driven subset choice could decide which witness parsers vote on silver trees, so that weak members are dropped rather than everything being averaged.
- **I3** (learn): when fine-tuning gparse on small treebanks or converted UD dialect data, freezing subword embeddings is a cheap regularizer worth trying.
- **I4** (learn): evaluation reports could flag labels with unusually high false-positive inflow, such as catch-all dependency relations or frame roles, as targets for repair rules.
- **I5** (tokens): the tagger and lemmatizer could mark forms whose surface spelling underdetermines the analysis (e.g., "read" past vs. present) and pass the ambiguity downstream instead of committing early.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Taibah University](https://md.research.elamur.ai/free/en/universities/taibah-university/) — University, Medina, Saudi Arabia · 12 works · 64 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 3,066 works

## Cite

Cite as: Nada Almarwani and Samah Aloufi. 2023. SANA at NADI 2023 shared task: Ensemble of Layer-Wise BERT-based models for Dialectal Arabic Identification. Proceedings of ArabicNLP 2023.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{almarwani2023sana,
  title = {{SANA at NADI 2023 shared task: Ensemble of Layer-Wise BERT-based models for Dialectal Arabic Identification}},
  author = {Nada Almarwani and Samah Aloufi},
  booktitle = {Proceedings of ArabicNLP 2023},
  year = {2023},
  doi = {10.18653/v1/2023.arabicnlp-1.65},
  url = {https://doi.org/10.18653/v1/2023.arabicnlp-1.65}
}
```

- [← previous rematchka at NADI 2023 shared task: Parameter Efficient tuning for Dialect Identification and Dialect Machine Translation](https://md.research.elamur.ai/free/en/papers/salam-2023-rematchka-nadi-2023-shared-task/)
- [next → Sentence-level Feedback Generation for English Language Learners: Does Data Augmentation Help?](https://md.research.elamur.ai/free/en/papers/behzad-2023-sentence-level-feedback-generation-english/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/15/)

## References

22 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Reem Abdel-Salam. 2022. Dialect & sentiment identification in nuanced Arabic tweets using an ensemble of prompt-based, fine-tuned, and multi-task BERT-based models. In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), pages 452–457, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Reem+Abdel%2DSalam%2E+2022%2E+Dialect+%26+sentiment+identification+in+nuanced+Arabic+tweets+using+an+ensemble+of+prompt%2Dbased%2C+fine%2Dtuned%2C+and+multi%2Dtask+BERT%2Dbased+models%2E+In+Proceedings+of+the+The+Seventh+A)
2. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021a. [ARBERT & MARBERT: Deep bidirectional transformers for Arabic](https://md.research.elamur.ai/free/en/papers/abdul-mageed-2021-arbert-marbert-deep-bidirectional-transformers/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105, Online. Association for Computational Linguistics.
3. Muhammad Abdul-Mageed, AbdelRahim Elmadany, Chiyu Zhang, ElMoatez Billah Nagoudi, Houda Bouamor, and Nizar Habash. 2023. [NADI 2023: The Fourth Nuanced Arabic Dialect Identification Shared Task](https://md.research.elamur.ai/free/en/papers/mageed-2023-nadi-2023-fourth-nuanced-arabic/). In Proceedings of The First Arabic Natural Language Processing Conference (ArabicNLP 2023).
4. Muhammad Abdul-Mageed, Chiyu Zhang, Houda Bouamor, and Nizar Habash. 2020. [NADI 2020: The first nuanced Arabic dialect identification shared task](https://md.research.elamur.ai/free/en/papers/mageed-2020-nadi-2020-first-nuanced-arabic/). In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 97–110, Barcelona, Spain (Online). Association for Computational Linguistics.
5. Muhammad Abdul-Mageed, Chiyu Zhang, Abdel-Rahim Elmadany, Houda Bouamor, and Nizar Habash. 2021b. [NADI 2021: The second nuanced Arabic dialect identification shared task](https://md.research.elamur.ai/free/en/papers/mageed-2021-nadi-2021-second-nuanced-arabic/). In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 244–259, Kyiv, Ukraine (Virtual). Association for Computational Linguistics.
6. Muhammad Abdul-Mageed, Chiyu Zhang, Abdel-Rahim Elmadany, Houda Bouamor, and Nizar Habash. 2022. [NADI 2022: The third nuanced Arabic dialect identification shared task](https://md.research.elamur.ai/free/en/papers/mageed-2022-nadi-2022-third-nuanced-arabic/). In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), pages 85–97, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
7. Latifah Almuqren and Alexandra I Cristea. 2016. Framework for sentiment analysis of arabic text. In Proceedings of the 27th ACM conference on hypertext and social media, pages 315–317. [find](https://scholar.google.com/scholar?q=Latifah+Almuqren+and+Alexandra+I+Cristea%2E+2016%2E+Framework+for+sentiment+analysis+of+arabic+text%2E+In+Proceedings+of+the+27th+ACM+conference+on+hypertext+and+social+media%2C+pages+315%E2%80%93317%2E)
8. Meshrif Alruily. 2020. Issues of dialectal saudi twitter corpus. Int. Arab J. Inf. Technol., 17(3):367–374. [find](https://scholar.google.com/scholar?q=Meshrif+Alruily%2E+2020%2E+Issues+of+dialectal+saudi+twitter+corpus%2E+Int%2E+Arab+J%2E+Inf%2E+Technol%2E%2C+17%283%29%3A367%E2%80%93374%2E)
9. Nouf AlShenaifi and Aqil Azmi. 2022. [Arabic dialect identification using machine learning and transformer-based models: Submission to the NADI 2022 shared task](https://md.research.elamur.ai/free/en/papers/alshenaifi-2022-arabic-dialect-identification-machine-learning/). In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), pages 464–467, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
10. Mahmoud Sobhy, Ahmed H. Abu El-Atta, Ahmed A. El-Sawy, and Hamada Nayel. 2022. [Word representation models for Arabic dialect identification](https://md.research.elamur.ai/free/en/papers/sobhy-2022-word-representation-models-arabic-dialect/). In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), pages 474–478, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
11. Maha J Althobaiti. 2020. [Automatic arabic dialect identification systems for written texts: A survey](https://md.research.elamur.ai/free/en/papers/althobaiti-2020-automatic-arabic-dialect-identification-systems/). arXiv preprint arXiv:2009.12622. [↗](https://arxiv.org/abs/2009.12622)
12. Mingyang Song, Yi Feng, and Liping Jing. 2022. Utilizing bert intermediate layers for unsupervised keyphrase extraction. In Proceedings of the 5th International Conference on Natural Language and Speech Processing (ICNLSP 2022), pages 277–281. [find](https://scholar.google.com/scholar?q=Mingyang+Song%2C+Yi+Feng%2C+and+Liping+Jing%2E+2022%2E+Utilizing+bert+intermediate+layers+for+unsupervised+keyphrase+extraction%2E+In+Proceedings+of+the+5th+International+Conference+on+Natural+Language+and+Spee)
13. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. [AraBERT: Transformer-based model for Arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/). In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15, Marseille, France. European Language Resource Association.
14. Wissam Antoun, Fady Baly, and Hazem Hajj. 2021. [AraGPT2: Pre-trained transformer for Arabic language generation](https://md.research.elamur.ai/free/en/papers/antoun-2021-aragpt2-pre-trained-transformer-arabic/). In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 196–207, Kyiv, Ukraine (Virtual). Association for Computational Linguistics.
15. Aqil M Azmi and Reham S Almajed. 2015. A survey of automatic arabic diacritization techniques. Natural Language Engineering, 21(3):477–495. [find](https://scholar.google.com/scholar?q=Aqil+M+Azmi+and+Reham+S+Almajed%2E+2015%2E+A+survey+of+automatic+arabic+diacritization+techniques%2E+Natural+Language+Engineering%2C+21%283%29%3A477%E2%80%93495%2E)
16. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. 2020. Transformers: State-ofthe-art natural language processing. In Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, pages 38–45. [find](https://scholar.google.com/scholar?q=Thomas+Wolf%2C+Lysandre+Debut%2C+Victor+Sanh%2C+Julien+Chaumond%2C+Clement+Delangue%2C+Anthony+Moi%2C+Pierric+Cistac%2C+Tim+Rault%2C+R%C3%A9mi+Louf%2C+Morgan+Funtowicz%2C+et+al%2E+2020%2E+Transformers%3A+State%2Dofthe%2Dart+natural+lan)
17. Patrick Zschech. 2022. Beyond descriptive taxonomies in data analytics: A systematic evaluation approach for data-driven method pipelines. Inf. Syst. E-Bus. Manag., 21(1):193–227. [find](https://scholar.google.com/scholar?q=Patrick+Zschech%2E+2022%2E+Beyond+descriptive+taxonomies+in+data+analytics%3A+A+systematic+evaluation+approach+for+data%2Ddriven+method+pipelines%2E+Inf%2E+Syst%2E+E%2DBus%2E+Manag%2E%2C+21%281%29%3A193%E2%80%93227%2E)
18. Afnan Bayazed, Ola Torabah, Redha AlSulami, Dimah Alahmadi, Amal Babour, and Kawther Saeedi. 2020. Sdct: Multi-dialects corpus classification for saudi tweets. International Journal of Advanced Computer Science and Applications, 11(11). [find](https://scholar.google.com/scholar?q=Afnan+Bayazed%2C+Ola+Torabah%2C+Redha+AlSulami%2C+Dimah+Alahmadi%2C+Amal+Babour%2C+and+Kawther+Saeedi%2E+2020%2E+Sdct%3A+Multi%2Ddialects+corpus+classification+for+saudi+tweets%2E+International+Journal+of+Advanced+Comput)
19. Ashraf Elnagar, Sane M. Yagi, Ali Bou Nassif, Ismail Shahin, and Said A. Salloum. 2021. Systematic literature review of dialectal arabic: Identification and detection. IEEE Access, 9:31010–31042. [find](https://scholar.google.com/scholar?q=Ashraf+Elnagar%2C+Sane+M%2E+Yagi%2C+Ali+Bou+Nassif%2C+Ismail+Shahin%2C+and+Said+A%2E+Salloum%2E+2021%2E+Systematic+literature+review+of+dialectal+arabic%3A+Identification+and+detection%2E+IEEE+Access%2C+9%3A31010%E2%80%9331042%2E)
20. Sofoklis Kakouros and Johannah O’Mahony. 2023. What does bert learn about prosody? In 20th International Congress of Phonetic Sciences ICPhS. International Phonetics Association. [find](https://scholar.google.com/scholar?q=Sofoklis+Kakouros+and+Johannah+O%E2%80%99Mahony%2E+2023%2E+What+does+bert+learn+about+prosody%3F+In+20th+International+Congress+of+Phonetic+Sciences+ICPhS%2E+International+Phonetics+Association%2E)
21. Yassir Matrane, Faouzia Benabbou, and Nawal Sael. 2023. A systematic literature review of arabic dialect sentiment analysis. Journal of King Saud University-Computer and Information Sciences, page 101570. [find](https://scholar.google.com/scholar?q=Yassir+Matrane%2C+Faouzia+Benabbou%2C+and+Nawal+Sael%2E+2023%2E+A+systematic+literature+review+of+arabic+dialect+sentiment+analysis%2E+Journal+of+King+Saud+University%2DComputer+and+Information+Sciences%2C+page+101)
22. Fouad Shammary, Yiyi Chen, Zsolt T Kardkovacs, Mehwish Alam, and Haithem Afli. 2022. [TF-IDF or transformers for Arabic dialect identification? ITFLOWS participation in the NADI 2022 shared task](https://md.research.elamur.ai/free/en/papers/shammary-2022-tf-idf-or-transformers-arabic/). In Proceedings of the The Seventh Arabic Natural Language Processing Workshop (WANLP), pages 420–424, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.

*Updated 2 November 2026*