---
title: "Comparing Pipelined and Integrated Approaches to Dialectal Arabic Neural Machine — Shapiro 2019"
description: "The paper asks how to translate Arabic into English when the dialect of the input sentence… 36.42 BLEU on the MSA. 26 references, cited by 2 papers here."
authors: ["Pamela Shapiro", "Kevin Duh"]
year: 2019
venue: "Proceedings of the Sixth Workshop on NLP for Similar Languages, Varieties and Dialects"
source: "https://aclanthology.org/W19-1424/"
doi: "10.18653/v1/w19-1424"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/shapiro-2019-comparing-pipelined-integrated-approaches-dialectal/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/shapiro-2019-comparing-pipelined-integrated-approaches-dialectal/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/5/) / Comparing Pipelined and Integrated Approaches to Dialectal Arabic Neural Machine Translation

# Comparing Pipelined and Integrated Approaches to Dialectal Arabic Neural Machine Translation

**Authors:** [Pamela Shapiro](https://md.research.elamur.ai/free/en/authors/pamela-shapiro/) and [Kevin Duh](https://md.research.elamur.ai/free/en/authors/kevin-duh/) · **Year:** 2019 · **Venue:** Proceedings of the Sixth Workshop on NLP for Similar Languages, Varieties and Dialects · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/W19-1424/) · [PDF](https://aclanthology.org/W19-1424.pdf) · [DOI 10.18653/v1/w19-1424](https://doi.org/10.18653/v1/w19-1424) · [OpenAlex](https://openalex.org/W2964085300)

**Citations:** 3 citations (OpenAlex, 2026-10-14) · [2](https://md.research.elamur.ai/free/en/papers/shapiro-2019-comparing-pipelined-integrated-approaches-dialectal/cited-by/) in our library, of the 140,884 papers whose bibliographies we read

## Summary
The paper asks how to translate Arabic into English when the dialect of the input sentence is unknown. One option is a pipeline: identify the dialect first, then send the sentence to a translation model tuned for that dialect. The other is a single multi-dialectal Transformer used for every input. The authors use LDC bitexts for MSA, Egyptian and Levantine, with Levantine and Egyptian taken from LDC2012T09. The tuned models start from an MSA model and continue training on dialect data. All systems share one BPE vocabulary built from every variant. To see how much dialect-identification quality matters, they add growing amounts of random noise to the dialect labels and track BLEU. The integrated model works best for MSA. For dialects, the pipeline wins when three conditions hold: identification error stays below about 20%, the BPE merge setting is small, and the dialect has enough training data. Below that quality the shared model is the safer choice. This gives a practical guide for when routing inputs to specialists is worth its cost.

## Key points
- Three systems were compared: a model trained on MSA only, plus Egyptian- and Levantine-tuned models fine-tuned from MSA in the style of Zoph et al. (2016). These were set against one integrated multi-dialectal Transformer.
- The MSA-only model reaches 36.42 BLEU on the MSA test set but only about 10 BLEU on Egyptian and 8 on Levantine. The tuned models reach 22.79 BLEU on Egyptian and 23.78 on Levantine.
- The crossover in favour of the pipeline appears only below roughly 20% dialect-ID error, at a small BPE merge size, and with sufficient in-dialect data.
- Levantine has more data and gains consistently over the shared model. Egyptian has little data, fine-tuning converges quickly, and the gains are weaker.
- A plain langid.py classifier trained on the bitext is a poor dialect identifier:
  - with the original data proportions it labels almost everything MSA;
  - with upsampled dialect data it labels almost everything Egyptian.

## Interesting ideas
- Measure how good a router must be by corrupting its gold decisions with controlled random noise and plotting where specialist models stop beating one general model.
- Build specialists by continuing to train a model fitted on a large standard variety using a smaller variety, while keeping one shared subword vocabulary so nothing falls out of vocabulary.
- Smaller subword units, closer to characters, helped the dialects most, likely because they capture pieces shared across varied inflection.
- Include a naive identifier as a reference point, and check whether its accuracy simply reflects class imbalance, i.e. predicting the majority or upsampled class.
- Watch for confounds such as sentence length that differ systematically between varieties and affect both the identifier and the translator.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Gold genre or dialect labels in front of the parser could be randomly flipped at rising rates. This would find the accuracy at which routing to specialised parsers stops paying off.
- **I2** (tree): gparse could be trained on a large standard English treebank and then briefly tuned on smaller informal or dialectal UD data. Keeping the token inventory shared means the tuned parser stays comparable to the base one.
- **I3** (tokens): A test could check whether finer subword or character features in the tagger and lemmatizer generalise better to non-standard spellings and informal morphology.
- **I4** (learn): Evaluations of any classifier stage could add a majority-class negative control and report per-class results. That keeps a collapsed model from looking acceptable on aggregate scores.
- **I5** (answer): School tests and fable QA results could be broken down by sentence length. This would separate genre effects from plain length difficulty.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

Topics: [🔁 Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) · [🐪 Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/)

## Cite

Cite as: Pamela Shapiro and Kevin Duh. 2019. Comparing Pipelined and Integrated Approaches to Dialectal Arabic Neural Machine Translation. Proceedings of the Sixth Workshop on NLP for Similar Languages, Varieties and Dialects.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{shapiro2019comparing,
  title = {{Comparing Pipelined and Integrated Approaches to Dialectal Arabic Neural Machine Translation}},
  author = {Pamela Shapiro and Kevin Duh},
  booktitle = {Proceedings of the Sixth Workshop on NLP for Similar Languages, Varieties and Dialects},
  year = {2019},
  doi = {10.18653/v1/w19-1424},
  url = {https://doi.org/10.18653/v1/w19-1424}
}
```

- [← previous Subjective Sentiment Analysis for Arabic Newswire Comments](https://md.research.elamur.ai/free/en/papers/bessou-2019-subjective-sentiment-analysis-arabic-newswire/)
- [next → Detecting Syntactic Change Using a Neural Part-of-Speech Tagger](https://md.research.elamur.ai/free/en/papers/merrill-2019-detecting-syntactic-change-neural-part/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/5/)

## References

26 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ann Bies, Zhiyi Song, Mohamed Maamouri, Stephen Grimes, Haejoong Lee, Jonathan Wright, Stephanie Strassel, Nizar Habash, Ramy Eskander, and Owen Rambow. 2014. Transliteration of arabizi into arabic orthography: Developing a parallel annotated arabizi-arabic script sms/chat corpus. In Proceedings of the EMNLP 2014 Workshop on Arabic Natural Language Processing (ANLP), pages 93–103. [find](https://scholar.google.com/scholar?q=Ann+Bies%2C+Zhiyi+Song%2C+Mohamed+Maamouri%2C+Stephen+Grimes%2C+Haejoong+Lee%2C+Jonathan+Wright%2C+Stephanie+Strassel%2C+Nizar+Habash%2C+Ramy+Eskander%2C+and+Owen+Rambow%2E+2014%2E+Transliteration+of+arabizi+into+arabic+or)
2. Salima Harrat, Karima Meftouh, and Kamel Smaili. 2017. Machine translation for arabic dialects (survey). Information Processing & Management. [find](https://scholar.google.com/scholar?q=Salima+Harrat%2C+Karima+Meftouh%2C+and+Kamel+Smaili%2E+2017%2E+Machine+translation+for+arabic+dialects+%28survey%29%2E+Information+Processing+%26+Management%2E)
3. Hany Hassan, Mostafa Elaraby, and Ahmed Tawfik. 2017. Synthetic data for neural machine translation of spoken-dialects. arXiv preprint arXiv:1707.00079. [↗](https://arxiv.org/abs/1707.00079)
4. Felix Hieber, Tobias Domhan, Michael Denkowski, David Vilar, Artem Sokolov, Ann Clifton, and Matt Post. 2017. Sockeye: A toolkit for neural machine translation. arXiv preprint arXiv:1712.05690. [↗](https://arxiv.org/abs/1712.05690)
5. Wael Salloum and Nizar Habash. 2012. Elissa: A dialectal to standard arabic machine translation system. Proceedings of COLING 2012: Demonstration Papers, pages 385–392. [find](https://scholar.google.com/scholar?q=Wael+Salloum+and+Nizar+Habash%2E+2012%2E+Elissa%3A+A+dialectal+to+standard+arabic+machine+translation+system%2E+Proceedings+of+COLING+2012%3A+Demonstration+Papers%2C+pages+385%E2%80%93392%2E)
6. Melvin Johnson, Mike Schuster, Quoc V Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, et al. 2017. Googles multilingual neural machine translation system: Enabling zero-shot translation. Transactions of the Association for Computational Linguistics, 5:339–351. [find](https://scholar.google.com/scholar?q=Melvin+Johnson%2C+Mike+Schuster%2C+Quoc+V+Le%2C+Maxim+Krikun%2C+Yonghui+Wu%2C+Zhifeng+Chen%2C+Nikhil+Thorat%2C+Fernanda+Vie%CC%81gas%2C+Martin+Wattenberg%2C+Greg+Corrado%2C+et+al%2E+2017%2E+Googles+multilingual+neural+machine+tra)
7. Hassan Sawaf. 2010. Arabic dialect handling in hybrid machine translation. In Proceedings of the conference of the association for machine translation in the americas (amta), denver, colorado. [find](https://scholar.google.com/scholar?q=Hassan+Sawaf%2E+2010%2E+Arabic+dialect+handling+in+hybrid+machine+translation%2E+In+Proceedings+of+the+conference+of+the+association+for+machine+translation+in+the+americas+%28amta%29%2C+denver%2C+colorado%2E)
8. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1715–1725. [our card](https://md.research.elamur.ai/free/en/papers/sennrich-2016-neural-machine-translation-rare-words/)
9. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. [↗](https://arxiv.org/abs/1412.6980)
10. Surafel Melaku Lakew, Mauro Cettolo, and Marcello Federico. 2018a. A comparison of transformer and recurrent neural networks on multilingual neural machine translation. In Proceedings of the 27th International Conference on Computational Linguistics, pages 641–652. [find](https://scholar.google.com/scholar?q=Surafel+Melaku+Lakew%2C+Mauro+Cettolo%2C+and+Marcello+Federico%2E+2018a%2E+A+comparison+of+transformer+and+recurrent+neural+networks+on+multilingual+neural+machine+translation%2E+In+Proceedings+of+the+27th+Inte)
11. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pages 3104–3112. [find](https://scholar.google.com/scholar?q=Ilya+Sutskever%2C+Oriol+Vinyals%2C+and+Quoc+V+Le%2E+2014%2E+Sequence+to+sequence+learning+with+neural+networks%2E+In+Advances+in+neural+information+processing+systems%2C+pages+3104%E2%80%933112%2E)
12. Houcemeddine Turki, Emad Adel, Tariq Daouda, and Nassim Regragui. 2016. A conventional orthography for maghrebi arabic. In Proceedings of the International Conference on Language Resources and Evaluation (LREC). [find](https://scholar.google.com/scholar?q=Houcemeddine+Turki%2C+Emad+Adel%2C+Tariq+Daouda%2C+and+Nassim+Regragui%2E+2016%2E+A+conventional+orthography+for+maghrebi+arabic%2E+In+Proceedings+of+the+International+Conference+on+Language+Resources+and+Evaluat)
13. Surafel Melaku Lakew, Aliia Erofeeva, and Marcello Federico. 2018b. Neural machine translation into language varieties. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 156–164. [find](https://scholar.google.com/scholar?q=Surafel+Melaku+Lakew%2C+Aliia+Erofeeva%2C+and+Marcello+Federico%2E+2018b%2E+Neural+machine+translation+into+language+varieties%2E+In+Proceedings+of+the+Third+Conference+on+Machine+Translation%3A+Research+Papers%2C+)
14. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, pages 5998–6008. [our card](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/)
15. Jason Lee, Kyunghyun Cho, and Thomas Hofmann. 2017. Fully character-level neural machine translation without explicit segmentation. Transactions of the Association for Computational Linguistics, 5:365–378. [our card](https://md.research.elamur.ai/free/en/papers/lee-2017-fully-character-level-neural-machine/)
16. Kees Versteegh. 2014. Arabic Language. Edinburgh University Press. [find](https://scholar.google.com/scholar?q=Kees+Versteegh%2E+2014%2E+Arabic+Language%2E+Edinburgh+University+Press%2E)
17. Marco Lui and Timothy Baldwin. 2012. langid. py: An off-the-shelf language identification tool. In Proceedings of the ACL 2012 system demonstrations, pages 25–30. Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/lui-2012-langid-py-off-shelf-language/)
18. Janet CE Watson. 2007. The Phonology and Morphology of Arabic. OUP Oxford. [find](https://scholar.google.com/scholar?q=Janet+CE+Watson%2E+2007%2E+The+Phonology+and+Morphology+of+Arabic%2E+OUP+Oxford%2E)
19. Omar F Zaidan and Chris Callison-Burch. 2014. Arabic dialect identification. Computational Linguistics, 40(1):171–202. [our card](https://md.research.elamur.ai/free/en/papers/zaidan-2014-arabic-dialect-identification/)
20. Karima Meftouh, Salima Harrat, Salma Jamoussi, Mourad Abbas, and Kamel Smaili. 2015. Machine translation experiments on padic: A parallel arabic dialect corpus. In Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation, pages 26–34. [find](https://scholar.google.com/scholar?q=Karima+Meftouh%2C+Salima+Harrat%2C+Salma+Jamoussi%2C+Mourad+Abbas%2C+and+Kamel+Smaili%2E+2015%2E+Machine+translation+experiments+on+padic%3A+A+parallel+arabic+dialect+corpus%2E+In+Proceedings+of+the+29th+Pacific+Asia)
21. Rabih Zbib, Erika Malchiodi, Jacob Devlin, David Stallard, Spyros Matsoukas, Richard Schwartz, John Makhoul, Omar F Zaidan, and Chris Callison-Burch. 2012. Machine translation of arabic dialects. In Proceedings of the 2012 conference of the north american chapter of the association for computational linguistics: Human language technologies, pages 49–59. Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/zbib-2012-machine-translation-arabic-dialects/)
22. Hassan Sajjad, Kareem Darwish, and Yonatan Belinkov. 2013. Translating dialectal arabic to english. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), volume 2, pages 1–6. [find](https://scholar.google.com/scholar?q=Hassan+Sajjad%2C+Kareem+Darwish%2C+and+Yonatan+Belinkov%2E+2013%2E+Translating+dialectal+arabic+to+english%2E+In+Proceedings+of+the+51st+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volume+2)
23. Michal Ziemski, Marcin Junczys-Dowmunt, and Bruno Pouliquen. 2016. The united nations parallel corpus v1. 0. In Lrec. [our card](https://md.research.elamur.ai/free/en/papers/ziemski-2016-united-nations-parallel-corpus-v1/)
24. Mohammad Salameh and Houda Bouamor. 2018. Fine-grained arabic dialect identification. In Proceedings of the 27th International Conference on Computational Linguistics, pages 1332–1344. [our card](https://md.research.elamur.ai/free/en/papers/salameh-2018-fine-grained-arabic-dialect-identification/)
25. Barret Zoph, Deniz Yuret, Jonathan May, and Kevin Knight. 2016. Transfer learning for low-resource neural machine translation. arXiv preprint arXiv:1604.02201. [↗](https://arxiv.org/abs/1604.02201)
26. Wael Salloum, Heba Elfardy, Linda Alamir-Salloum, Nizar Habash, and Mona Diab. 2014. Sentence level dialect identification for machine translation system selection. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), volume 2, pages 772–778. [find](https://scholar.google.com/scholar?q=Wael+Salloum%2C+Heba+Elfardy%2C+Linda+Alamir%2DSalloum%2C+Nizar+Habash%2C+and+Mona+Diab%2E+2014%2E+Sentence+level+dialect+identification+for+machine+translation+system+selection%2E+In+Proceedings+of+the+52nd+Annual+M)