---
title: "ST MADAR 2019 Shared Task — paper by Abbas et al. (2019)"
description: "The paper presents the authors' entry in the MADAR 2019 shared task on fine-grained… 69.00 F1. 7 references, cited by 11 papers here."
authors: ["Mourad Abbas", "Mohamed Lichouri", "Abed Alhakim Freihat"]
year: 2019
venue: "Proceedings of the Fourth Arabic Natural Language Processing Workshop"
source: "https://aclanthology.org/W19-4635/"
doi: "10.18653/v1/w19-4635"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/abbas-2019-st-madar-2019-shared-task/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/abbas-2019-st-madar-2019-shared-task/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/6/) / ST MADAR 2019 Shared Task: Arabic Fine-Grained Dialect Identification

# ST MADAR 2019 Shared Task: Arabic Fine-Grained Dialect Identification

**Authors:** [Mourad Abbas](https://md.research.elamur.ai/free/en/authors/mourad-abbas/), [Mohamed Lichouri](https://md.research.elamur.ai/free/en/authors/mohamed-lichouri/) and [Abed Alhakim Freihat](https://md.research.elamur.ai/free/en/authors/abed-alhakim-freihat/) · **Year:** 2019 · **Venue:** [Proceedings of the Fourth Arabic Natural Language Processing Workshop](https://md.research.elamur.ai/free/en/venues/wanlp/) ([WANLP 2019](https://md.research.elamur.ai/free/en/venues/wanlp/2019/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/W19-4635/) · [PDF](https://aclanthology.org/W19-4635.pdf) · [DOI 10.18653/v1/w19-4635](https://doi.org/10.18653/v1/w19-4635) · [OpenAlex](https://openalex.org/W2970011044)

**Citations:** 11 citations (OpenAlex, 2026-10-14) · [11](https://md.research.elamur.ai/free/en/papers/abbas-2019-st-madar-2019-shared-task/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper presents the authors' entry in the MADAR 2019 shared task on fine-grained Arabic dialect identification. Each short sentence must be assigned to one of 26 classes: 25 city-level dialects plus MSA. The data is the MADAR Travel Domain corpus, built by having native speakers translate BTEC travel phrases into each variety. The authors used deliberately simple machinery. Preprocessing was minimal, consisting of whitespace tokenization and punctuation removal. Features were word n-grams, character n-grams and within-word character n-grams (char_wb), weighted with TF-IDF. These were fed to a range of scikit-learn classifiers: SVM, Naive Bayes variants, LR, SGD, Passive Aggressive and Perceptron. Multinomial Naive Bayes on word unigrams performed best, at about 62.9% macro-F1 on dev and 62.1% on test. This falls short of the official baseline (F1 69.00). The same setup reaches about 90.5% on a coarser 6-way version of the data, which shows how much harder the city-level distinction is. The authors attribute the difficulty to closely related dialects, non-standard spelling and the presence of MSA material.

## Key points
- The data contains 1,600 training sentences per variety and 200 each for dev and test, giving 41,600 / 5,200 / 5,200 sentences in total. Sentences are 4–15 words long.
- Classifiers and dev macro-F1 on Corpus-26:
  - MNB: 62.87
  - BNB: 62.72
  - linear SVM (SVM l): 61.94
  - Perceptron: weakest, at 55.89
- On test, MNB scored F1 62.12 and accuracy 62.17, below the baseline's 69.00 F1 and 67.90 accuracy.
- Preliminary experiments with POS-tagging and lemmatization features produced no significant improvement over surface n-grams.
- Results varied widely by dialect and region:
  - Mosul (MOS) was easiest, with F1 of 80 on dev and 78 on test.
  - AMM and MUS were hardest, at roughly 48–51 F1.
  - By region, Yemen averaged about 75% and the Maghreb over 67%.

## Interesting ideas
- The classification task was run on the same data at two levels of granularity (6-way and 26-way), turning the gap between them into a direct measure of how confusable the fine labels are.
- Character n-grams were restricted to word interiors with space padding at the edges (char_wb), which captures prefixes and suffixes while tolerating unstable spelling.
- The authors noted that richer linguistic features (POS, lemmas) failed to beat plain surface n-grams, an ablation worth reporting rather than hiding.
- Results were broken down per class and per regional group, exposing which neighbouring varieties collapse into each other.
- The corpus is parallel: the same content is expressed in every variety, so the classifier must rely on variety signal rather than topic.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Scoring evaluation at both coarse and fine label sets (e.g., UPOS vs. full feature bundles, frame vs. VerbNet subclass) would show whether errors are true confusions or near-misses.
- **I2** (tokens): Word-internal padded char n-grams could serve as backoff features in the tagger and lemmatizer for misspelled, informal or rare English words.
- **I3** (learn): Every added linguistic layer should be justified by an ablation against a surface-only control, so that complexity is kept only where it measurably pays off.
- **I4** (tree): Breaking down LAS by source treebank or converted UD dialect would reveal which conversions or genres are systematically confused by the perceptron parser and gparse.
- **I5** (tree): A small parallel set of identical sentences in different English registers or treebank conventions could test whether UD dialect conversion yields matching trees.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Trento](https://md.research.elamur.ai/free/en/universities/university-of-trento/) — University, Trento, Italy · 85 works · 15,524 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works

## Cite

Cite as: Mourad Abbas, Mohamed Lichouri and Abed Alhakim Freihat. 2019. ST MADAR 2019 Shared Task: Arabic Fine-Grained Dialect Identification. Proceedings of the Fourth Arabic Natural Language Processing Workshop.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{abbas2019st,
  title = {{ST MADAR 2019 Shared Task: Arabic Fine-Grained Dialect Identification}},
  author = {Mourad Abbas, Mohamed Lichouri and Abed Alhakim Freihat},
  booktitle = {Proceedings of the Fourth Arabic Natural Language Processing Workshop},
  year = {2019},
  doi = {10.18653/v1/w19-4635},
  url = {https://doi.org/10.18653/v1/w19-4635}
}
```

- [← previous Dialogue-Act Prediction of Future Responses Based on Conversation History](https://md.research.elamur.ai/free/en/papers/tanaka-2019-dialogue-act-prediction-future-responses/)
- [next → Natural Language Inference with Monotonicity](https://md.research.elamur.ai/free/en/papers/hu-2019-natural-language-inference-monotonicity/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/6/)

## References

7 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Houda Bouamor, Nizar Habash, Mohammad Salameh, Wajdi Zaghouani, Owen Rambow, Dana Abdulrahim, Ossama Obeid, Salam Khalifa, Fadhl Eryani, Alexander Erdmann, and Kemal Oflazer. 2018. The MADAR Arabic Dialect Corpus and Lexicon. In Proceedings of the Language Resources and Evaluation Conference (LREC), Miyazaki, Japan. [find](https://scholar.google.com/scholar?q=Houda+Bouamor%2C+Nizar+Habash%2C+Mohammad+Salameh%2C+Wajdi+Zaghouani%2C+Owen+Rambow%2C+Dana+Abdulrahim%2C+Ossama+Obeid%2C+Salam+Khalifa%2C+Fadhl+Eryani%2C+Alexander+Erdmann%2C+and+Kemal+Oflazer%2E+2018%2E+The+MADAR+Arabic+Di)
2. Mohammad Salameh, Houda Bouamor, and Nizar Habash. 2018. [Fine-grained Arabic dialect identification](https://md.research.elamur.ai/free/en/papers/salameh-2018-fine-grained-arabic-dialect-identification/). In Proceedings of the International Conference on Computational Linguistics (COLING), pages 1332–1344, Santa Fe, New Mexico, USA.
3. Toshiyuki Takezawa, Genichiro Kikui, Masahide Mizushima, and Eiichiro Sumita. 2007. Multilingual spoken language corpus development for communication research. International Journal of Computational Linguistics & Chinese Language Processing, Volume 12, Number 3, September 2007: Special Issue on Invited Papers from ISCSLP 2006, 12(3):303–324. [find](https://scholar.google.com/scholar?q=Toshiyuki+Takezawa%2C+Genichiro+Kikui%2C+Masahide+Mizushima%2C+and+Eiichiro+Sumita%2E+2007%2E+Multilingual+spoken+language+corpus+development+for+communication+research%2E+International+Journal+of+Computational+L)
4. Houda Bouamor, Sabit Hassan, and Nizar Habash. 2019. [The MADAR Shared Task on Arabic Fine-Grained Dialect Identification](https://md.research.elamur.ai/free/en/papers/bouamor-2019-madar-shared-task-arabic-fine/). In Proceedings of the Fourth Arabic Natural Language Processing Workshop (WANLP19), Florence, Italy.
5. Abed Alhakim Freihat, Mourad Abbas, Gábor Bella, and Fausto Giunchiglia. 2018a. Towards an optimal solution to lemmatization in arabic. Procedia computer science, 142:132–140. [find](https://scholar.google.com/scholar?q=Abed+Alhakim+Freihat%2C+Mourad+Abbas%2C+Ga%CC%81bor+Bella%2C+and+Fausto+Giunchiglia%2E+2018a%2E+Towards+an+optimal+solution+to+lemmatization+in+arabic%2E+Procedia+computer+science%2C+142%3A132%E2%80%93140%2E)
6. Omar F Zaidan and Chris Callison-Burch. 2014. [Arabic dialect identification](https://md.research.elamur.ai/free/en/papers/zaidan-2014-arabic-dialect-identification/). Computational Linguistics, 40(1):171–202. [↗](https://doi.org/10.1162/coli_a_00169)
7. Abed Alhakim Freihat, Gabor Bella, Hamdy Mubarak, and Fausto Giunchiglia. 2018b. A single-model approach for arabic segmentation, pos tagging, and [find](https://scholar.google.com/scholar?q=Abed+Alhakim+Freihat%2C+Gabor+Bella%2C+Hamdy+Mubarak%2C+and+Fausto+Giunchiglia%2E+2018b%2E+A+single%2Dmodel+approach+for+arabic+segmentation%2C+pos+tagging%2C+and)

*Updated 1 November 2026*