---
title: "Dialect Identification in Nuanced Arabic Tweets Using Farasa"
description: "The paper describes one team's entry to the NADI 2021 shared task (EACL WANLP-2021, Shared… 19 references, cited by 3 papers here."
authors: ["Anshul Wadhawan"]
year: 2021
venue: "Proceedings of the Sixth Arabic Natural Language Processing Workshop"
source: "https://aclanthology.org/2021.wanlp-1.35/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wadhawan-2021-dialect-identification-nuanced-arabic-tweets/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wadhawan-2021-dialect-identification-nuanced-arabic-tweets/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/7/) / Dialect Identification in Nuanced Arabic Tweets Using Farasa Segmentation and AraBERT

# Dialect Identification in Nuanced Arabic Tweets Using Farasa Segmentation and AraBERT

**Authors:** [Anshul Wadhawan](https://md.research.elamur.ai/free/en/authors/anshul-wadhawan/) · **Year:** 2021 · **Venue:** [Proceedings of the Sixth Arabic Natural Language Processing Workshop](https://md.research.elamur.ai/free/en/venues/wanlp/) ([WANLP 2021](https://md.research.elamur.ai/free/en/venues/wanlp/2021/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2021.wanlp-1.35/) · [PDF](https://aclanthology.org/2021.wanlp-1.35.pdf) · [OpenAlex](https://openalex.org/W3152729570)

**Citations:** 4 citations (OpenAlex, 2026-10-14) · [3](https://md.research.elamur.ai/free/en/papers/wadhawan-2021-dialect-identification-nuanced-arabic-tweets/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper describes one team's entry to the NADI 2021 shared task (EACL WANLP-2021, Shared Task 1). The task is to guess where an Arabic tweet was written, at the level of 21 countries or 100 provinces, and it covers both Modern Standard Arabic (MSA) and dialectal Arabic (DA). The labeled data has 21,000 training tweets plus 5,000 each for validation and test. The system has two parts.

- **Cleaning step:** tweets are normalized so they look like the text the models saw during pre-training. URLs, emails and mentions become placeholder tokens, noise is stripped, spacing is fixed, and Farasa segmentation is applied for the models that expect it.
- **Model comparison:** six AraBERT variants and two AraELECTRA variants were fine-tuned. The best one per subtask was then retrained on train plus validation.

On the test set the final macro F1 scores were 0.216 and 0.235 at country level and 0.054 and 0.043 at province level. This placed the team second on MSA and fourth on DA. The work shows that careful input normalization and model selection go a long way, and that province-level dialect distinctions remain very hard.

## Key points
- **Data:** tweets from 100 provinces in 21 Arab countries. There are four subtasks: country-level MSA (1.1), country-level DA (1.2), province-level MSA (2.1) and province-level DA (2.2). Another 10M unlabeled tweets were offered. The official metric is macro-averaged F1.
- **Preprocessing:**
  - Farasa segmentation, used only for the models pre-trained on segmented text.
  - Arabic placeholder tokens for URLs, emails and user mentions.
  - Removal of HTML, emoticons, extra spaces and characters repeated more than twice.
  - Whitespace inserted between Arabic words and Latin letters, digits or brackets.
- **Models and settings:** AraBERTv0.1/v0.2/v1/v2 in base and large sizes, plus the AraELECTRA-base generator and discriminator, all loaded via Hugging Face. Fine-tuning used:
  - learning rate 1e-5
  - maximum length 256
  - 5 epochs
  - batch size 40 for base models and 4 for large models
- **Validation results:**
  - AraBERTv2-large was best at country level, with weighted F1 of 0.315 on 1.1 and 0.416 on 1.2.
  - AraBERTv0.2-base was best at province level, with 0.038 on 2.1 and 0.035 on 2.2. On these province tasks AraBERTv2-large scored only about 0.001.
  - Both AraELECTRA models trailed every AraBERT model.
- **Test results:** macro F1 of 0.216, 0.235, 0.054 and 0.043, with accuracy of 0.317, 0.433, 0.060 and 0.053.

## Interesting ideas
- Normalize fine-tuning input with the same pipeline the pretrained model was trained on, so the model never meets a distribution it has not seen; a mismatch quietly costs accuracy.
- Replacing URLs, emails and @-mentions with fixed placeholder tokens keeps their position in the sentence but drops content that carries no signal and would only bloat the vocabulary.
- Whether to pre-segment the input depends on the model: Farasa segmentation was applied only to models whose pre-training used it, so preprocessing becomes a property of each model.
- Choosing a separate model per subtask paid off: a large model won at the coarse level but collapsed on the 100-class tasks, where a base model was best.
- The paper reports weighted F1 on validation but macro F1 on test, which hides how badly rare classes do; the gap is a warning to always measure with the official metric.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tokens): Make the en tokenizer and normalizer a single shared function that runs both when treebanks are prepared and when live text arrives, so the tagger sees the same token shapes everywhere.
- **I2** (tree): Turn such items into typed placeholder tokens before parsing, so they attach as a single noun-like node and do not break the UD tree into odd fragments.
- **I3** (learn): When several witness parsers vote on silver data, store each parser's expected input form next to it and convert per witness, so disagreements reflect real parsing differences and not tokenization artifacts.
- **I4** (learn): Evaluate model size separately for coarse and fine-grained label sets, such as UD relation types versus VerbNet roles, instead of assuming the biggest model wins everywhere.
- **I5** (answer): Report school-test and QA scores both per class and averaged across classes, so poor performance on rare question types or rare frames cannot hide behind frequent ones.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works

## Cite

Cite as: Anshul Wadhawan. 2021. Dialect Identification in Nuanced Arabic Tweets Using Farasa Segmentation and AraBERT. Proceedings of the Sixth Arabic Natural Language Processing Workshop.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wadhawan2021dialect,
  title = {{Dialect Identification in Nuanced Arabic Tweets Using Farasa Segmentation and AraBERT}},
  author = {Anshul Wadhawan},
  booktitle = {Proceedings of the Sixth Arabic Natural Language Processing Workshop},
  year = {2021},
  url = {https://aclanthology.org/2021.wanlp-1.35/}
}
```

- [← previous Decoding, Fast and Slow: A Case Study on Balancing Trade-Offs in Incremental, Character-level Pragmatic Reasoning](https://md.research.elamur.ai/free/en/papers/zarriess-2021-decoding-fast-slow-case-study/)
- [next → Grapheme-to-Phoneme Transformer Model for Transfer Learning Dialects](https://md.research.elamur.ai/free/en/papers/engelhart-2021-grapheme-phoneme-transformer-model-transfer/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/7/)

## References

19 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ahmed Abdelali, Kareem Darwish, Nadir Durrani, and Hamdy Mubarak. 2016. [Farasa: A fast and furious segmenter for Arabic](https://md.research.elamur.ai/free/en/papers/abdelali-2016-farasa-fast-furious-segmenter-arabic/). In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, pages 11–16, San Diego, California. Association for Computational Linguistics.
2. Muhammad Abdul-Mageed, Chiyu Zhang, Houda Bouamor, and Nizar Habash. 2020. [NADI 2020: The First Nuanced Arabic Dialect Identification Shared Task](https://md.research.elamur.ai/free/en/papers/mageed-2020-nadi-2020-first-nuanced-arabic/). In Proceedings of the Fifth Arabic Natural Language Processing Workshop (WANLP 2020), Barcelona, Spain. [↗](https://doi.org/10.48550/arxiv.2010.11334)
3. Muhammad Abdul-Mageed, Chiyu Zhang, Abdel-Rahim Elmadany, Houda Bouamor, and Nizar Habash. 2021. [NADI 2021: The Second Nuanced Arabic Dialect Identification Shared Task](https://md.research.elamur.ai/free/en/papers/mageed-2021-nadi-2021-second-nuanced-arabic/). In Proceedings of the Sixth Arabic Natural Language Processing Workshop (WANLP 2021). [↗](https://doi.org/10.48550/arxiv.2103.08466)
4. Wissam Antoun, Fady Baly, and Hazem Hajj. [Arabert: Transformer-based model for arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/). In LREC 2020 Workshop Language Resources and Evaluation Conference 11–16 May 2020, page 9. [↗](https://doi.org/10.48550/arxiv.2003.00104)
5. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. [Araelectra: Pre-training text discriminators for arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2021-araelectra-pre-training-text-discriminators/). [↗](https://doi.org/10.48550/arxiv.2012.15516)
6. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Janvin. 2003. [A neural probabilistic language model](https://md.research.elamur.ai/free/en/papers/bengio-2003-neural-probabilistic-language-model/). J. Mach. Learn. Res., 3(null):1137–1155.
7. Houda Bouamor, Sabit Hassan, and Nizar Habash. 2019. [The madar shared task on arabic fine-grained dialect identification](https://md.research.elamur.ai/free/en/papers/bouamor-2019-madar-shared-task-arabic-fine/). In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 199–207.
8. Wei-Cheng Chang, Hsiang-Fu Yu, Kai Zhong, Yiming Yang, and Inderjit Dhillon. 2020. Taming pretrained transformers for extreme multi-label text classification. [find](https://scholar.google.com/scholar?q=Wei%2DCheng+Chang%2C+Hsiang%2DFu+Yu%2C+Kai+Zhong%2C+Yiming+Yang%2C+and+Inderjit+Dhillon%2E+2020%2E+Taming+pretrained+transformers+for+extreme+multi%2Dlabel+text+classification%2E)
9. Mohamed Elaraby and Muhammad Abdul-Mageed. 2018. [Deep models for Arabic dialect identification on benchmarked data](https://md.research.elamur.ai/free/en/papers/elaraby-2018-deep-models-arabic-dialect-identification/). In Proceedings of the Fifth Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial 2018), pages 263–274, Santa Fe, New Mexico, USA. Association for Computational Linguistics.
10. Heba Elfardy and Mona Diab. 2013. [Sentence level dialect identification in Arabic](https://md.research.elamur.ai/free/en/papers/elfardy-2013-sentence-level-dialect-identification-arabic/). In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 456–461, Sofia, Bulgaria. Association for Computational Linguistics.
11. Alex Graves, Abdel rahman Mohamed, and Geoffrey Hinton. 2013. Speech recognition with deep recurrent neural networks. [find](https://scholar.google.com/scholar?q=Alex+Graves%2C+Abdel+rahman+Mohamed%2C+and+Geoffrey+Hinton%2E+2013%2E+Speech+recognition+with+deep+recurrent+neural+networks%2E)
12. Imane Guellil, Houda Saâdane, Faical Azouaou, Billel Gueni, and Damien Nouvel. 2019. [Arabic natural language processing: An overview](https://md.research.elamur.ai/free/en/papers/guellil-2019-arabic-natural-language-processing-overview/). Journal of King Saud University - Computer and Information Sciences. [↗](https://doi.org/10.1016/j.jksuci.2019.02.006)
13. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. [Imagenet classification with deep convolutional neural networks](https://md.research.elamur.ai/free/en/papers/krizhevsky-2012-imagenet-classification-deep-convolutional/). In Proceedings of the 25th International Conference on Neural Information Processing Systems - Volume 1, NIPS’12, page 1097–1105, Red Hook, NY, USA. Curran Associates Inc.
14. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. [Distributed representations of words and phrases and their compositionality](https://md.research.elamur.ai/free/en/papers/mikolov-2013-distributed-representations-words-phrases-their/). In Proceedings of the 26th International Conference on Neural Information Processing Systems - Volume 2, NIPS’13, page 3111–3119, Red Hook, NY, USA. Curran Associates Inc. [↗](https://doi.org/10.48550/arxiv.1310.4546)
15. Mohammad Salameh, Houda Bouamor, and Nizar Habash. 2018. [Fine-grained arabic dialect identification](https://md.research.elamur.ai/free/en/papers/salameh-2018-fine-grained-arabic-dialect-identification/). In Proceedings of the 27th International Conference on Computational Linguistics, pages 1332– 1344.
16. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/).
17. Wen-tau Yih, Kristina Toutanova, John C. Platt, and Christopher Meek. 2011. [Learning discriminative projections for text similarity measures](https://md.research.elamur.ai/free/en/papers/yih-2011-learning-discriminative-projections-text-similarity/). In Proceedings of the Fifteenth Conference on Computational Natural Language Learning, pages 247–256, Portland, Oregon, USA. Association for Computational Linguistics.
18. Omar F. Zaidan and Chris Callison-Burch. 2014. [Arabic dialect identification](https://md.research.elamur.ai/free/en/papers/zaidan-2014-arabic-dialect-identification/). Computational Linguistics, 40(1):171–202. [↗](https://doi.org/10.1162/coli_a_00169)
19. Chiyu Zhang and Muhammad Abdul-Mageed. 2019. [No army, no navy: BERT semi-supervised learning of Arabic dialects](https://md.research.elamur.ai/free/en/papers/zhang-2019-no-army-no-navy-bert/). In Proceedings of the Fourth Arabic Natural Language Processing Workshop, pages 279–284, Florence, Italy. Association for Computational Linguistics.

*Updated 1 November 2026*