---
title: "First Result on Arabic Neural Machine Translation (2016)"
description: "The authors compare a plain attention-based encoder-decoder (the dl4mt implementation, with BPE subwords)… Cited by 14 papers here."
authors: ["Amjad Almahairi", "Kyunghyun Cho", "Nizar Habash", "Aaron Courville"]
year: 2016
venue: "arXiv:1606.02680"
source: "https://arxiv.org/abs/1606.02680"
doi: "10.48550/arxiv.1606.02680"
arxiv: "1606.02680"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/almahairi-2016-first-result-arabic-neural-machine/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/almahairi-2016-first-result-arabic-neural-machine/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2016](https://md.research.elamur.ai/free/en/papers/2016/page/4/) / First Result on Arabic Neural Machine Translation

# First Result on Arabic Neural Machine Translation

**Authors:** [Amjad Almahairi](https://md.research.elamur.ai/free/en/authors/amjad-almahairi/), [Kyunghyun Cho](https://md.research.elamur.ai/free/en/authors/kyunghyun-cho/), [Nizar Habash](https://md.research.elamur.ai/free/en/authors/nizar-habash/) and [Aaron Courville](https://md.research.elamur.ai/free/en/authors/aaron-courville/) · **Year:** 2016 · **Venue:** arXiv:1606.02680 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1606.02680) · [PDF](https://arxiv.org/pdf/1606.02680) · [DOI 10.48550/arxiv.1606.02680](https://doi.org/10.48550/arxiv.1606.02680) · [arXiv 1606.02680](https://arxiv.org/abs/1606.02680) · [OpenAlex](https://openalex.org/W2413534664)

**Citations:** 26 citations (OpenAlex, 2026-10-14) · [14](https://md.research.elamur.ai/free/en/papers/almahairi-2016-first-result-arabic-neural-machine/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
This short 2016 paper reports the first fully neural machine translation results for Arabic, in both directions (Ar→En and En→Ar). Earlier Arabic work used neural networks only as extra features inside phrase-based systems. The authors compare a plain attention-based encoder-decoder (the dl4mt implementation, with BPE subwords) against a standard Moses phrase-based system. Both are trained on about 1.2M mostly-news sentence pairs from LDC corpora.

They also vary Arabic preprocessing: simple tokenization, orthographic normalization, and morphology-aware ATB segmentation via MADAMIRA. English preprocessing varies too (tokenization, lowercasing or truecasing).

On in-domain news (MT05) the two systems end up close, with Moses slightly ahead. Preprocessing designed for phrase-based systems helps the neural system by a similar margin. On an out-of-domain climate-change web set (MEDAR), the neural system clearly wins in En→Ar.

This matters because it shows two things: established linguistic preprocessing still pays off for neural models, and neural MT appears more robust when the test text differs from the training domain.

## Key points
- **Training data:** LDC2004T18, LDC2004T17 and LDC2007T08 combined give about 1.2M sentence pairs, with about 33M tokens on the Arabic side. Pairs with more than 100 tokens on either side were dropped.
- **Evaluation sets:** MT04 was used as dev and MT05 as test. Arabic sentences that also appeared verbatim in the training data were removed (172 from MT04, 26 from MT05). MEDAR, with four Arabic references per sentence, served as the out-of-domain En→Ar test.
- **Domain distance:** average log-probabilities from a 5-gram LM trained on the training corpus confirm that MEDAR is far off-domain (MT04 −59.74, MT05 −55.97, MEDAR −75.03).
- **Systems:**
  - NMT: a 2-layer bidirectional GRU encoder (512×2), a 512-GRU decoder, Adadelta for about seven days, dropout plus L2 (1e−4), beam width 12, and a BPE vocabulary of up to 20k.
  - Phrase-based: Moses with GIZA++, phrases up to 8 words, and KenLM.
- **Best BLEU scores (Moses vs NMT):**
  - En→Ar MT05: 35.98 vs 33.62.
  - En→Ar MEDAR: 19.27 vs 24.46. Preprocessing added +10.58 for Moses and +13.37 for NMT over tokenization alone.
  - Ar→En MT05: 51.19 vs 49.7.

## Interesting ideas
- Before scoring, the authors searched the test sets for sentences copied verbatim from the training data and removed them, so the reported scores reflect real generalization rather than memorization.
- They measured domain shift directly by scoring each evaluation set with an n-gram language model trained on the training text, which turns "out-of-domain" into a number.
- To undo segmentation when generating Arabic, they first look up how each segmented form appeared in the training corpus and fall back to hand-written rules only when the lookup fails.
- Each preprocessing step was switched on cumulatively and reported as a gain over a minimal baseline, for two very different model types. This showed that linguistically motivated preprocessing helps both about equally.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Run the same overlap filter between silver training data and evaluation sets, and treat a high match rate as a negative control that flags inflated scores.
- **I2** (learn): Score each new test genre (fables, school texts, word problems) against the training data the same way, so parser or reader drops can be tied to measured distance.
- **I3** (back): When realizing text from the meaning graph, prefer surface forms already attested in the corpus for a lemma plus features, and use rule-based inflection only as a backoff.
- **I4** (tokens): Run the same layered comparison of normalization and morphology options against both the perceptron parser and gparse, to learn which steps are model-independent.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [New York University](https://md.research.elamur.ai/free/en/universities/new-york-university/) — University, New York, United States · 219 works · 103,487 citations
- [Université de Montréal](https://md.research.elamur.ai/free/en/universities/universite-de-montreal/) — University, Montreal, Canada · 87 works · 91,597 citations
- [Mila - Quebec Artificial Intelligence Institute](https://md.research.elamur.ai/free/en/labs/mila-quebec-artificial-intelligence-institute/) — Research institute, Montreal, Canada · 65 works · 14,425 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (2): [🔁 Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) · 1,105 works · [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 3,066 works

## Cite

Cite as: Amjad Almahairi, Kyunghyun Cho, Nizar Habash and Aaron Courville. 2016. First Result on Arabic Neural Machine Translation. arXiv:1606.02680.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{almahairi2016first,
  title = {{First Result on Arabic Neural Machine Translation}},
  author = {Amjad Almahairi, Kyunghyun Cho, Nizar Habash and Aaron Courville},
  year = {2016},
  doi = {10.48550/arxiv.1606.02680},
  eprint = {1606.02680},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1606.02680}
}
```

- [← previous Discourse Relation Sense Classification Using Cross-argument Semantic Similarity Based on Word Embeddings](https://md.research.elamur.ai/free/en/papers/mihaylov-2016-discourse-relation-sense-classification-cross/)
- [next → A puzzle for theories of redundancy: Exhaustification, incrementality, and the notion of local context](https://md.research.elamur.ai/free/en/papers/mayr-2016-puzzle-theories-redundancy-exhaustification-incrementality/)
- [↑ back to papers of 2016](https://md.research.elamur.ai/free/en/papers/2016/page/4/)

*Updated 29 October 2026*