---
title: "Domain Robustness in Neural Machine — Müller 2020"
description: "The paper asks whether modern neural machine translation still breaks down on text from domains it never saw in training, and why."
authors: ["Mathias Müller", "Annette Rios", "Rico Sennrich"]
year: 2020
venue: "Proceedings of the 14th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track)"
source: "https://aclanthology.org/2020.amta-research.14/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/muller-2020-domain-robustness-neural-machine-translation/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/muller-2020-domain-robustness-neural-machine-translation/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2020](https://md.research.elamur.ai/free/papers/2020/page/2/) / Domain Robustness in Neural Machine Translation

# Domain Robustness in Neural Machine Translation

**Authors:** [Mathias Müller](https://md.research.elamur.ai/free/authors/mathias-muller/), [Annette Rios](https://md.research.elamur.ai/free/authors/annette-rios/), [Rico Sennrich](https://md.research.elamur.ai/free/authors/rico-sennrich/) · **Year:** 2020 · **Venue:** Proceedings of the 14th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.amta-research.14/) · [PDF](https://aclanthology.org/2020.amta-research.14.pdf) · [OpenAlex](https://openalex.org/W3105718208)

**Citations:** 68 citations (openalex-snapshot · 2026-10-14) · [67](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
The paper asks whether modern neural machine translation still breaks down on text from domains it never saw in training, and why. The authors train Transformer models (Sockeye) and phrase-based Moses systems on a single domain and test them on unseen ones. The setups are German→English, trained on medical OPUS data and evaluated on IT, koran, law and subtitles, and low-resource German→Romansh, trained on law and press releases and evaluated on blogs.

Out-of-domain BLEU collapses for both system types, and their average scores end up nearly equal. The failures differ, though. SMT output stays faithful to the source but reads poorly. NMT output reads well but often has nothing to do with the input. Manual evaluation shows these hallucinations become much more frequent outside the training domain.

The authors then compare reconstruction, subword regularization, noisy channel reranking, defensive distillation and their combinations. These give moderate BLEU gains but improve adequacy only slightly. The work matters because it shows that BLEU alone can conceal very different error profiles. It also releases baselines for studying robustness when no domain adaptation is possible.

## Key points
- **Data:**
  - DE→EN uses the five OPUS domains from Koehn and Knowles (2017), with EMEA medical data (1.1m pairs) as the training domain.
  - DE→RM trains on about 100k pairs (the Allegra corpus plus in-house press releases) and tests on Convivenza blog posts.
- **DE→EN baselines:**
  - In-domain, NMT beats SMT (61.5 vs 58.4 BLEU).
  - Out-of-domain, both average only about 11.7–11.8 BLEU.
  - On IT, SMT is 4.3 BLEU ahead of NMT.
- **OOV words are not the main cause:** the koran and subtitles test sets score far lower than IT despite comparable OOV rates.
- **Reconstruction:**
  - It lowers the average share of hallucinated out-of-domain outputs from 35% to 29%, at some cost to fluency.
  - Only reconstruction combined with noisy channel reranking improved BLEU consistently across all domains and both language pairs.
- **Overall gains:** the best systems improve average out-of-domain BLEU by 1.5 (DE→EN) and 2.7 (DE→RM). Code and data are available at github.com/ZurichNLP/domain-robustness.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/) · [Machine translation](https://md.research.elamur.ai/free/shelves/machine-translation/)

Topics: [Machine translation](https://md.research.elamur.ai/free/topics/machine-translation/)

## Cited by

Cited by

67 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 61 not yet.

- [Unsupervised Domain Clusters in Pretrained Language Models](https://md.research.elamur.ai/free/papers/aharoni-2020-unsupervised-domain-clusters-pretrained-language/)
- [Hallucinations in Large Multilingual Translation Models](https://md.research.elamur.ai/free/papers/guerreiro-2023-hallucinations-large-multilingual-translation-models/)
- [Survey of Low-Resource Machine Translation](https://md.research.elamur.ai/free/papers/haddow-2022-survey-low-resource-machine-translation/)
- [Survey of Hallucination in Natural Language Generation](https://md.research.elamur.ai/free/papers/ji-2022-survey-hallucination-natural-language-generation/)
- [Context Volume Drives Performance: Tackling Domain Shift in Extremely Low-Resource Translation via RAG](https://md.research.elamur.ai/free/papers/lingbulk-2026-setiawan-context-volume-drives-performance-tackling/)
- [The Curious Case of Hallucinations in Neural Machine Translation](https://md.research.elamur.ai/free/papers/raunak-2021-curious-case-hallucinations-neural-machine/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{muller2020domain,
  title = {{Domain Robustness in Neural Machine Translation}},
  author = {Müller, Mathias and Rios, Annette and Sennrich, Rico},
  booktitle = {Proceedings of the 14th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track)},
  year = {2020},
  url = {https://aclanthology.org/2020.amta-research.14/}
}
```

- [← previous Does Syntax Need to Grow on Trees? Sources of Hierarchical Inductive Bias in Sequence-to-Sequence Networks](https://md.research.elamur.ai/free/papers/mccoy-2020-does-syntax-need-grow-trees/)
- [next → Domain-Specific Language Model Pretraining for Biomedical Natural Language Processing](https://md.research.elamur.ai/free/papers/gu-2020-domain-specific-language-model-pretraining/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/papers/2020/page/2/)

## References

35 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Arthur, P., Neubig, G., and Nakamura, S. (2016). Incorporating discrete translation lexicons into neural machine translation. In Proceedings of EMNLP 2016, pages 1557–1567. [find](https://scholar.google.com/scholar?q=Arthur%2C+P%2E%2C+Neubig%2C+G%2E%2C+and+Nakamura%2C+S%2E+%282016%29%2E+Incorporating+discrete+translation+lexicons+into+neural+machine+translation%2E+In+Proceedings+of+EMNLP+2016%2C+pages+1557%E2%80%931567%2E)
2. Ba, J. and Caruana, R. (2014). Do deep nets really need to be deep? In Advances in neural information processing systems, pages 2654–2662. [find](https://scholar.google.com/scholar?q=Ba%2C+J%2E+and+Caruana%2C+R%2E+%282014%29%2E+Do+deep+nets+really+need+to+be+deep%3F+In+Advances+in+neural+information+processing+systems%2C+pages+2654%E2%80%932662%2E)
3. Belinkov, Y. and Bisk, Y. (2018). Synthetic and natural noise both break neural machine translation. In Proceedings of the 6th International Conference on Learning Representations, Vancouver, Canada. [find](https://scholar.google.com/scholar?q=Belinkov%2C+Y%2E+and+Bisk%2C+Y%2E+%282018%29%2E+Synthetic+and+natural+noise+both+break+neural+machine+translation%2E+In+Proceedings+of+the+6th+International+Conference+on+Learning+Representations%2C+Vancouver%2C+Canada%2E)
4. Goodfellow, I., Shlens, J., and Szegedy, C. (2015). Explaining and harnessing adversarial examples. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Goodfellow%2C+I%2E%2C+Shlens%2C+J%2E%2C+and+Szegedy%2C+C%2E+%282015%29%2E+Explaining+and+harnessing+adversarial+examples%2E+In+International+Conference+on+Learning+Representations%2E) [our card](https://md.research.elamur.ai/free/papers/goodfellow-2014-explaining-harnessing-adversarial-examples/)
5. Goodfellow, I. J., Shlens, J., and Szegedy, C. (2014). Explaining and harnessing adversarial examples (2014). arXiv preprint arXiv:1412.6572. [find](https://scholar.google.com/scholar?q=Goodfellow%2C+I%2E+J%2E%2C+Shlens%2C+J%2E%2C+and+Szegedy%2C+C%2E+%282014%29%2E+Explaining+and+harnessing+adversarial+examples+%282014%29%2E+arXiv+preprint+arXiv%3A1412%2E6572%2E) [our card](https://md.research.elamur.ai/free/papers/goodfellow-2014-explaining-harnessing-adversarial-examples/)
6. Hieber, F., Domhan, T., Denkowski, M., Vilar, D., Sokolov, A., Clifton, A., and Post, M. (2018). The Sockeye Neural Machine Translation Toolkit at AMTA 2018. In Proceedings of the 13th Conference of the Association for Machine Translation in the Americas (AMTA), pages 200–207. [find](https://scholar.google.com/scholar?q=Hieber%2C+F%2E%2C+Domhan%2C+T%2E%2C+Denkowski%2C+M%2E%2C+Vilar%2C+D%2E%2C+Sokolov%2C+A%2E%2C+Clifton%2C+A%2E%2C+and+Post%2C+M%2E+%282018%29%2E+The+Sockeye+Neural+Machine+Translation+Toolkit+at+AMTA+2018%2E+In+Proceedings+of+the+13th+Conference+of+t)
7. Hinton, G., Vinyals, O., and Dean, J. (2015). Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531. [find](https://scholar.google.com/scholar?q=Hinton%2C+G%2E%2C+Vinyals%2C+O%2E%2C+and+Dean%2C+J%2E+%282015%29%2E+Distilling+the+knowledge+in+a+neural+network%2E+arXiv+preprint+arXiv%3A1503%2E02531%2E) [our card](https://md.research.elamur.ai/free/papers/hinton-2015-distilling-knowledge-neural-network/)
8. Jang, E., Gu, S., and Poole, B. (2017). Categorical reparametrization with gumbel-softmax. In Proceedings International Conference on Learning Representations 2017. OpenReviews.net. [find](https://scholar.google.com/scholar?q=Jang%2C+E%2E%2C+Gu%2C+S%2E%2C+and+Poole%2C+B%2E+%282017%29%2E+Categorical+reparametrization+with+gumbel%2Dsoftmax%2E+In+Proceedings+International+Conference+on+Learning+Representations+2017%2E+OpenReviews%2Enet%2E)
9. Johnson, M., Schuster, M., Le, Q. V., Krikun, M., Wu, Y., Chen, Z., Thorat, N., Viégas, F., Wattenberg, M., Corrado, G., Hughes, M., and Dean, J. (2017). Google’s multilingual neural machine translation system: Enabling zero-shot translation. Transactions of the Association for Computational Lingui… [find](https://scholar.google.com/scholar?q=Johnson%2C+M%2E%2C+Schuster%2C+M%2E%2C+Le%2C+Q%2E+V%2E%2C+Krikun%2C+M%2E%2C+Wu%2C+Y%2E%2C+Chen%2C+Z%2E%2C+Thorat%2C+N%2E%2C+Vie%CC%81gas%2C+F%2E%2C+Wattenberg%2C+M%2E%2C+Corrado%2C+G%2E%2C+Hughes%2C+M%2E%2C+and+Dean%2C+J%2E+%282017%29%2E+Google%E2%80%99s+multilingual+neural+machine+translat)
10. Kim, Y. and Rush, A. M. (2016). Sequence-level knowledge distillation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1317–1327, Austin, Texas. [find](https://scholar.google.com/scholar?q=Kim%2C+Y%2E+and+Rush%2C+A%2E+M%2E+%282016%29%2E+Sequence%2Dlevel+knowledge+distillation%2E+In+Proceedings+of+the+2016+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+1317%E2%80%931327%2C+Austin%2C+Texas%2E) [our card](https://md.research.elamur.ai/free/papers/kim-2016-sequence-level-knowledge-distillation/)
11. Kobus, C., Crego, J., and Senellart, J. (2017). Domain control for neural machine translation. In Proceedings of RANLP 2017, pages 372–378. INCOMA Ltd. [find](https://scholar.google.com/scholar?q=Kobus%2C+C%2E%2C+Crego%2C+J%2E%2C+and+Senellart%2C+J%2E+%282017%29%2E+Domain+control+for+neural+machine+translation%2E+In+Proceedings+of+RANLP+2017%2C+pages+372%E2%80%93378%2E+INCOMA+Ltd%2E)
12. Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi, N., Cowan, B., Shen, W., Moran, C., Zens, R., Dyer, C., Bojar, O., Constantin, A., and Herbst, E. (2007). Moses: Open Source Toolkit for Statistical Machine Translation. In Proceedings of the ACL-2007 Demo and Poster Sessio… [find](https://scholar.google.com/scholar?q=Koehn%2C+P%2E%2C+Hoang%2C+H%2E%2C+Birch%2C+A%2E%2C+Callison%2DBurch%2C+C%2E%2C+Federico%2C+M%2E%2C+Bertoldi%2C+N%2E%2C+Cowan%2C+B%2E%2C+Shen%2C+W%2E%2C+Moran%2C+C%2E%2C+Zens%2C+R%2E%2C+Dyer%2C+C%2E%2C+Bojar%2C+O%2E%2C+Constantin%2C+A%2E%2C+and+Herbst%2C+E%2E+%282007%29%2E+Moses%3A+Open+Sourc)
13. Koehn, P. and Knowles, R. (2017). Six Challenges for Neural Machine Translation. In Proceedings of the First Workshop on Neural Machine Translation, pages 28–39. [find](https://scholar.google.com/scholar?q=Koehn%2C+P%2E+and+Knowles%2C+R%2E+%282017%29%2E+Six+Challenges+for+Neural+Machine+Translation%2E+In+Proceedings+of+the+First+Workshop+on+Neural+Machine+Translation%2C+pages+28%E2%80%9339%2E)
14. Kudo, T. (2018). Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates. In Proceedings of ACL 2018, pages 66–75. [find](https://scholar.google.com/scholar?q=Kudo%2C+T%2E+%282018%29%2E+Subword+Regularization%3A+Improving+Neural+Network+Translation+Models+with+Multiple+Subword+Candidates%2E+In+Proceedings+of+ACL+2018%2C+pages+66%E2%80%9375%2E)
15. Läubli, S., Müller, M., Horat, B., and Volk, M. (2018). mtrain: A convenience tool for machine translation. [find](https://scholar.google.com/scholar?q=La%CC%88ubli%2C+S%2E%2C+Mu%CC%88ller%2C+M%2E%2C+Horat%2C+B%2E%2C+and+Volk%2C+M%2E+%282018%29%2E+mtrain%3A+A+convenience+tool+for+machine+translation%2E)
16. Lee, K., Firat, O., Agarwal, A., Fannjiang, C., and Sussillo, D. (2018). Hallucinations in neural machine translation. [find](https://scholar.google.com/scholar?q=Lee%2C+K%2E%2C+Firat%2C+O%2E%2C+Agarwal%2C+A%2E%2C+Fannjiang%2C+C%2E%2C+and+Sussillo%2C+D%2E+%282018%29%2E+Hallucinations+in+neural+machine+translation%2E)
17. Li, J. and Jurafsky, D. (2016). Mutual information and diverse decoding improve neural machine translation. arXiv preprint arXiv:1601.00372. — [arXiv:1601.00372](https://arxiv.org/abs/1601.00372) [find](https://scholar.google.com/scholar?q=Li%2C+J%2E+and+Jurafsky%2C+D%2E+%282016%29%2E+Mutual+information+and+diverse+decoding+improve+neural+machine+translation%2E+arXiv+preprint+arXiv%3A1601%2E00372%2E+%E2%80%94+%5BarXiv%3A1601%2E00372%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1601%2E00372%29)
18. Proceedings of the 14th Conference of the Association for Machine Translation in the Americas October 6 - 9, 2020, Volume 1: MT Research Track Li, X., Michel, P., Anastasopoulos, A., Belinkov, Y., Durrani, N., Firat, O., Koehn, P., Neubig, G., Pino, J., and Sajjad, H. (2019). Findings of the First S… [find](https://scholar.google.com/scholar?q=Proceedings+of+the+14th+Conference+of+the+Association+for+Machine+Translation+in+the+Americas+October+6+%2D+9%2C+2020%2C+Volume+1%3A+MT+Research+Track+Li%2C+X%2E%2C+Michel%2C+P%2E%2C+Anastasopoulos%2C+A%2E%2C+Belinkov%2C+Y%2E%2C+Dur)
19. Lison, P. and Tiedemann, J. (2016). OpenSubtitles2016: Extracting Large Parallel Corpora from Movie and TV Subtitles. In Proceedings of LREC 2016, pages 923–929. [find](https://scholar.google.com/scholar?q=Lison%2C+P%2E+and+Tiedemann%2C+J%2E+%282016%29%2E+OpenSubtitles2016%3A+Extracting+Large+Parallel+Corpora+from+Movie+and+TV+Subtitles%2E+In+Proceedings+of+LREC+2016%2C+pages+923%E2%80%93929%2E) [our card](https://md.research.elamur.ai/free/papers/lison-2016-opensubtitles2016-extracting-large-parallel-corpora/)
20. Luong, M.-T. and Manning, C. D. (2015). Stanford Neural Machine Translation Systems for Spoken Language Domains. In Proceedings of IWSLT 2015, Da Nang, Vietnam. [find](https://scholar.google.com/scholar?q=Luong%2C+M%2E%2DT%2E+and+Manning%2C+C%2E+D%2E+%282015%29%2E+Stanford+Neural+Machine+Translation+Systems+for+Spoken+Language+Domains%2E+In+Proceedings+of+IWSLT+2015%2C+Da+Nang%2C+Vietnam%2E)
21. Nguyen, T. and Chiang, D. (2018). Improving Lexical Choice in Neural Machine Translation. In Proceedings of NAACL-HLT 2018, pages 334–343. [find](https://scholar.google.com/scholar?q=Nguyen%2C+T%2E+and+Chiang%2C+D%2E+%282018%29%2E+Improving+Lexical+Choice+in+Neural+Machine+Translation%2E+In+Proceedings+of+NAACL%2DHLT+2018%2C+pages+334%E2%80%93343%2E)
22. Niu, X., Xu, W., and Carpuat, M. (2019). Bi-directional differentiable input reconstruction for low-resource neural machine translation. In Proceedings of NAACL-HLT 2019, pages 442–448. [find](https://scholar.google.com/scholar?q=Niu%2C+X%2E%2C+Xu%2C+W%2E%2C+and+Carpuat%2C+M%2E+%282019%29%2E+Bi%2Ddirectional+differentiable+input+reconstruction+for+low%2Dresource+neural+machine+translation%2E+In+Proceedings+of+NAACL%2DHLT+2019%2C+pages+442%E2%80%93448%2E)
23. Ott, M., Edunov, S., Baevski, A., Fan, A., Gross, S., Ng, N., Grangier, D., and Auli, M. (2019). fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of NAACL-HLT 2019: Demonstrations. [find](https://scholar.google.com/scholar?q=Ott%2C+M%2E%2C+Edunov%2C+S%2E%2C+Baevski%2C+A%2E%2C+Fan%2C+A%2E%2C+Gross%2C+S%2E%2C+Ng%2C+N%2E%2C+Grangier%2C+D%2E%2C+and+Auli%2C+M%2E+%282019%29%2E+fairseq%3A+A+fast%2C+extensible+toolkit+for+sequence+modeling%2E+In+Proceedings+of+NAACL%2DHLT+2019%3A+Demonstrat) [our card](https://md.research.elamur.ai/free/papers/ott-2019-fairseq-fast-extensible-toolkit-sequence/)
24. Papernot, N., McDaniel, P., Wu, X., Jha, S., and Swami, A. (2016). Distillation as a defense to adversarial perturbations against deep neural networks. In 2016 IEEE Symposium on Security and Privacy (SP), pages 582–597. IEEE. [find](https://scholar.google.com/scholar?q=Papernot%2C+N%2E%2C+McDaniel%2C+P%2E%2C+Wu%2C+X%2E%2C+Jha%2C+S%2E%2C+and+Swami%2C+A%2E+%282016%29%2E+Distillation+as+a+defense+to+adversarial+perturbations+against+deep+neural+networks%2E+In+2016+IEEE+Symposium+on+Security+and+Privacy+%28)
25. Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J. (2002). BLEU: A Method for Automatic Evaluation of Machine Translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pages 311–318, Philadelphia, PA. [find](https://scholar.google.com/scholar?q=Papineni%2C+K%2E%2C+Roukos%2C+S%2E%2C+Ward%2C+T%2E%2C+and+Zhu%2C+W%2E%2DJ%2E+%282002%29%2E+BLEU%3A+A+Method+for+Automatic+Evaluation+of+Machine+Translation%2E+In+Proceedings+of+the+40th+Annual+Meeting+on+Association+for+Computational+Li) [our card](https://md.research.elamur.ai/free/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
26. Post, M. (2018). A call for clarity in reporting bleu scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186–191, Belgium, Brussels. [find](https://scholar.google.com/scholar?q=Post%2C+M%2E+%282018%29%2E+A+call+for+clarity+in+reporting+bleu+scores%2E+In+Proceedings+of+the+Third+Conference+on+Machine+Translation%3A+Research+Papers%2C+pages+186%E2%80%93191%2C+Belgium%2C+Brussels%2E) [our card](https://md.research.elamur.ai/free/papers/post-2018-call-clarity-reporting-bleu-scores/)
27. Scherrer, Y. and Cartoni, B. (2012). The Trilingual ALLEGRA Corpus: Presentation and Possible Use for Lexicon Induction. In Proceedings of LREC-2012, pages 2890–2896, Istanbul, Turkey. [find](https://scholar.google.com/scholar?q=Scherrer%2C+Y%2E+and+Cartoni%2C+B%2E+%282012%29%2E+The+Trilingual+ALLEGRA+Corpus%3A+Presentation+and+Possible+Use+for+Lexicon+Induction%2E+In+Proceedings+of+LREC%2D2012%2C+pages+2890%E2%80%932896%2C+Istanbul%2C+Turkey%2E)
28. Sennrich, R., Haddow, B., and Birch, A. (2016a). Improving Neural Machine Translation Models with Monolingual Data. In Proceedings of ACL 2016, pages 86–96, Berlin, Germany. [find](https://scholar.google.com/scholar?q=Sennrich%2C+R%2E%2C+Haddow%2C+B%2E%2C+and+Birch%2C+A%2E+%282016a%29%2E+Improving+Neural+Machine+Translation+Models+with+Monolingual+Data%2E+In+Proceedings+of+ACL+2016%2C+pages+86%E2%80%9396%2C+Berlin%2C+Germany%2E)
29. Sennrich, R., Haddow, B., and Birch, A. (2016b). Neural Machine Translation of Rare Words with Subword Units. In Proceedings of ACL 2016, pages 1715–1725, Berlin, Germany. [find](https://scholar.google.com/scholar?q=Sennrich%2C+R%2E%2C+Haddow%2C+B%2E%2C+and+Birch%2C+A%2E+%282016b%29%2E+Neural+Machine+Translation+of+Rare+Words+with+Subword+Units%2E+In+Proceedings+of+ACL+2016%2C+pages+1715%E2%80%931725%2C+Berlin%2C+Germany%2E)
30. Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., and Fergus, R. (2013). Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199. [find](https://scholar.google.com/scholar?q=Szegedy%2C+C%2E%2C+Zaremba%2C+W%2E%2C+Sutskever%2C+I%2E%2C+Bruna%2C+J%2E%2C+Erhan%2C+D%2E%2C+Goodfellow%2C+I%2E%2C+and+Fergus%2C+R%2E+%282013%29%2E+Intriguing+properties+of+neural+networks%2E+arXiv+preprint+arXiv%3A1312%2E6199%2E) [our card](https://md.research.elamur.ai/free/papers/szegedy-2013-intriguing-properties-neural-networks/)
31. Tu, Z., Liu, Y., Shang, L., Liu, X., and Li, H. (2017). Neural Machine Translation with Reconstruction. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, pages 3097–3103. [find](https://scholar.google.com/scholar?q=Tu%2C+Z%2E%2C+Liu%2C+Y%2E%2C+Shang%2C+L%2E%2C+Liu%2C+X%2E%2C+and+Li%2C+H%2E+%282017%29%2E+Neural+Machine+Translation+with+Reconstruction%2E+In+Proceedings+of+the+Thirty%2DFirst+AAAI+Conference+on+Artificial+Intelligence%2C+pages+3097%E2%80%933103%2E)
32. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. (2017). Attention is All you Need. In Advances in Neural Information Processing Systems 30, pages 5998–6008. [find](https://scholar.google.com/scholar?q=Vaswani%2C+A%2E%2C+Shazeer%2C+N%2E%2C+Parmar%2C+N%2E%2C+Uszkoreit%2C+J%2E%2C+Jones%2C+L%2E%2C+Gomez%2C+A%2E+N%2E%2C+Kaiser%2C+%C5%81%2E%2C+and+Polosukhin%2C+I%2E+%282017%29%2E+Attention+is+All+you+Need%2E+In+Advances+in+Neural+Information+Processing+Systems+30%2C)
33. Yee, K., Dauphin, Y., and Auli, M. (2019). Simple and effective noisy channel modeling for neural machine translation. In Proceedings of EMNLP-IJCNLP 2019, pages 5696–5701, Hong Kong, China. [find](https://scholar.google.com/scholar?q=Yee%2C+K%2E%2C+Dauphin%2C+Y%2E%2C+and+Auli%2C+M%2E+%282019%29%2E+Simple+and+effective+noisy+channel+modeling+for+neural+machine+translation%2E+In+Proceedings+of+EMNLP%2DIJCNLP+2019%2C+pages+5696%E2%80%935701%2C+Hong+Kong%2C+China%2E)
34. Yu, L., Blunsom, P., Dyer, C., Grefenstette, E., and Kociský, T. (2017). The neural noisy channel. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France. [find](https://scholar.google.com/scholar?q=Yu%2C+L%2E%2C+Blunsom%2C+P%2E%2C+Dyer%2C+C%2E%2C+Grefenstette%2C+E%2E%2C+and+Kocisky%CC%81%2C+T%2E+%282017%29%2E+The+neural+noisy+channel%2E+In+5th+International+Conference+on+Learning+Representations%2C+ICLR+2017%2C+Toulon%2C+France%2E)
35. Proceedings of the 14th Conference of the Association for Machine Translation in the Americas October 6 - 9, 2020, Volume 1: MT Research Track [find](https://scholar.google.com/scholar?q=Proceedings+of+the+14th+Conference+of+the+Association+for+Machine+Translation+in+the+Americas+October+6+%2D+9%2C+2020%2C+Volume+1%3A+MT+Research+Track)