---
title: "On Exposure Bias, Hallucination and Domain Shift — Wang 2020"
description: "This paper asks whether exposure bias actually hurts neural machine translation in practice."
authors: ["Chaojun Wang", "Rico Sennrich"]
year: 2020
venue: "Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics"
source: "https://aclanthology.org/2020.acl-main.326/"
doi: "10.18653/v1/2020.acl-main.326"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wang-2020-exposure-bias-hallucination-domain-shift/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2020-exposure-bias-hallucination-domain-shift/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/3/) / On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translation

# On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translation

**Authors:** Chaojun Wang, [Rico Sennrich](https://md.research.elamur.ai/free/en/authors/rico-sennrich/) · **Year:** 2020 · **Venue:** Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.acl-main.326/) · [PDF](https://aclanthology.org/2020.acl-main.326.pdf) · [DOI 10.18653/v1/2020.acl-main.326](https://doi.org/10.18653/v1/2020.acl-main.326) · [OpenAlex](https://openalex.org/W3035214886)

**Citations:** 116 citations (openalex-snapshot · 2026-10-14) · [88](https://md.research.elamur.ai/free/en/papers/wang-2020-exposure-bias-hallucination-domain-shift/cited-by/) in our library, of the 140884 papers whose bibliographies we read

## Summary
This paper asks whether exposure bias actually hurts neural machine translation in practice. Exposure bias is the mismatch between teacher-forced MLE training and decoding from the model's own outputs. The authors argue it helps explain two known failures under domain shift: hallucinations (fluent output unrelated to the source) and worse quality at larger beam sizes. They fine-tune MLE-pretrained Transformer models in Nematus with Minimum Risk Training (MRT), a sequence-level objective that has no exposure bias. They test on IWSLT'14 German→English, an OPUS German→English multi-domain setup, and a low-resource German→Romansh task. MRT gives modest in-domain gains but clearly better out-of-domain BLEU, and manual evaluation shows fewer hallucinations. A per-token probability analysis explains why. After the first few tokens, the MLE model scores random distractor sentences about as highly as true references, so early mistakes are rarely penalised. MRT widens this gap. The work therefore supports sequence-level objectives for robustness, even when in-domain scores barely change.

## Key points
- **Data:**
  - IWSLT'14 DE→EN has 180,000 sentence pairs.
  - The OPUS DE→EN setup trains on the medical domain and tests out-of-domain on IT, law, koran and subtitles.
  - DE→RM trains on the Allegra corpus (law, 100,000 pairs) and tests on blog text from Convivenza.
- **MRT setup:** the loss is 1-BLEU smooth, and 4 randomly sampled candidate translations are used per source sentence. On IWSLT'14, MRT raises BLEU from 34.7 to 35.2.
- **Out-of-domain BLEU:** with label smoothing, MRT improves average OOD BLEU by 0.8 for DE→EN (11.2→12.0) and by 0.7 for DE→RM (18.0→18.7). In-domain scores stay roughly flat.
- **Hallucinations:** they occur in 33–35% of out-of-domain outputs versus 1–2% in-domain. MRT reduces them by up to 21% relative, for example from 33% to 26%.
- **Beam size:** at beam 50, MRT leads MLE by 0.6–1.5 BLEU. With label smoothing, the rise in hallucinations as beam size grows is smaller under MRT (+33%) than under MLE (+44%).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

Topics: [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wang2020exposure,
  title = {{On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translation}},
  author = {Wang, Chaojun and Sennrich, Rico},
  booktitle = {Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.acl-main.326},
  url = {https://doi.org/10.18653/v1/2020.acl-main.326}
}
```

- [← previous Entities as Experts: Sparse Memory Access with Entity Supervision](https://md.research.elamur.ai/free/en/papers/fevry-2020-entities-as-experts-sparse-memory/)
- [next → Contextualizing Hate Speech Classifiers with Post-hoc Explanation](https://md.research.elamur.ai/free/en/papers/kennedy-2020-contextualizing-hate-speech-classifiers-post/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/3/)

## References

34 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings. [find](https://scholar.google.com/scholar?q=Dzmitry+Bahdanau%2C+Kyunghyun+Cho%2C+and+Yoshua+Bengio%2E+2015%2E+Neural+machine+translation+by+jointly+learning+to+align+and+translate%2E+In+3rd+International+Conference+on+Learning+Representations%2C+ICLR+2015%2C) [our card](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
2. Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. 2015. Scheduled sampling for sequence prediction with recurrent neural networks. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems 28, pages 1171–1179. Curran… [find](https://scholar.google.com/scholar?q=Samy+Bengio%2C+Oriol+Vinyals%2C+Navdeep+Jaitly%2C+and+Noam+Shazeer%2E+2015%2E+Scheduled+sampling+for+sequence+prediction+with+recurrent+neural+networks%2E+In+C%2E+Cortes%2C+N%2E+D%2E+Lawrence%2C+D%2E+D%2E+Lee%2C+M%2E+Sugiyama%2C+and)
3. Jean Carletta. 1996. Assessing agreement on classification tasks: The kappa statistic. Computational Linguistics, 22(2):249–254. [find](https://scholar.google.com/scholar?q=Jean+Carletta%2E+1996%2E+Assessing+agreement+on+classification+tasks%3A+The+kappa+statistic%2E+Computational+Linguistics%2C+22%282%29%3A249%E2%80%93254%2E) [our card](https://md.research.elamur.ai/free/en/papers/carletta-1996-assessing-agreement-classification-tasks-kappa/)
4. Mauro Cettolo, Jan Niehues, Sebastian Stüker, Luisa Bentivogli, and Marcello Federico. 2014. Report on the 11th iwslt evaluation campaign, iwslt 2014. In Proceedings of the International Workshop on Spoken Language Translation, Hanoi, Vietnam, page 57. [find](https://scholar.google.com/scholar?q=Mauro+Cettolo%2C+Jan+Niehues%2C+Sebastian+Stu%CC%88ker%2C+Luisa+Bentivogli%2C+and+Marcello+Federico%2E+2014%2E+Report+on+the+11th+iwslt+evaluation+campaign%2C+iwslt+2014%2E+In+Proceedings+of+the+International+Workshop+on+) [our card](https://md.research.elamur.ai/free/en/papers/cettolo-2014-report-11th-iwslt-evaluation-campaign/)
5. Sergey Edunov, Myle Ott, Michael Auli, David Grangier, and Marc’Aurelio Ranzato. 2018. Classical structured prediction losses for sequence to sequence learning. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Techno… [find](https://scholar.google.com/scholar?q=Sergey+Edunov%2C+Myle+Ott%2C+Michael+Auli%2C+David+Grangier%2C+and+Marc%E2%80%99Aurelio+Ranzato%2E+2018%2E+Classical+structured+prediction+losses+for+sequence+to+sequence+learning%2E+In+Proceedings+of+the+2018+Conference+o) [our card](https://md.research.elamur.ai/free/en/papers/edunov-2018-classical-structured-prediction-losses-sequence/)
6. Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N. Dauphin. 2017. Convolutional sequence to sequence learning. In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML’17, pages 1243– 1252. JMLR.org. [find](https://scholar.google.com/scholar?q=Jonas+Gehring%2C+Michael+Auli%2C+David+Grangier%2C+Denis+Yarats%2C+and+Yann+N%2E+Dauphin%2E+2017%2E+Convolutional+sequence+to+sequence+learning%2E+In+Proceedings+of+the+34th+International+Conference+on+Machine+Learni) [our card](https://md.research.elamur.ai/free/en/papers/gehring-2017-convolutional-sequence-sequence-learning/)
7. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondřej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In Pro… [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2C+Hieu+Hoang%2C+Alexandra+Birch%2C+Chris+Callison%2DBurch%2C+Marcello+Federico%2C+Nicola+Bertoldi%2C+Brooke+Cowan%2C+Wade+Shen%2C+Christine+Moran%2C+Richard+Zens%2C+Chris+Dyer%2C+Ondr%CC%8Cej+Bojar%2C+Alexandra+Const)
8. Philipp Koehn and Rebecca Knowles. 2017. Six challenges for neural machine translation. In Proceedings of the First Workshop on Neural Machine Translation, pages 28–39, Vancouver. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Philipp+Koehn+and+Rebecca+Knowles%2E+2017%2E+Six+challenges+for+neural+machine+translation%2E+In+Proceedings+of+the+First+Workshop+on+Neural+Machine+Translation%2C+pages+28%E2%80%9339%2C+Vancouver%2E+Association+for+Comp)
9. Taku Kudo. 2018. Subword regularization: Improving neural network translation models with multiple subword candidates. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 66–75, Melbourne, Australia. Association for Computational… [find](https://scholar.google.com/scholar?q=Taku+Kudo%2E+2018%2E+Subword+regularization%3A+Improving+neural+network+translation+models+with+multiple+subword+candidates%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Computational+Li)
10. Jiwei Li and Dan Jurafsky. 2016. Mutual information and diverse decoding improve neural machine translation. [find](https://scholar.google.com/scholar?q=Jiwei+Li+and+Dan+Jurafsky%2E+2016%2E+Mutual+information+and+diverse+decoding+improve+neural+machine+translation%2E)
11. Chin-Yew Lin and Franz Josef Och. 2004. ORANGE: a method for evaluating automatic evaluation metrics for machine translation. In COLING 2004: Proceedings of the 20th International Conference on Computational Linguistics, pages 501–507, Geneva, Switzerland. COLING. [find](https://scholar.google.com/scholar?q=Chin%2DYew+Lin+and+Franz+Josef+Och%2E+2004%2E+ORANGE%3A+a+method+for+evaluating+automatic+evaluation+metrics+for+machine+translation%2E+In+COLING+2004%3A+Proceedings+of+the+20th+International+Conference+on+Comput) [our card](https://md.research.elamur.ai/free/en/papers/lin-2004-orange-method-evaluating-automatic-evaluation/)
12. Pierre Lison and Jörg Tiedemann. 2016. OpenSubtitles2016: Extracting large parallel corpora from movie and TV subtitles. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), pages 923–929, Portorož, Slovenia. European Language Resources Association… [find](https://scholar.google.com/scholar?q=Pierre+Lison+and+Jo%CC%88rg+Tiedemann%2E+2016%2E+OpenSubtitles2016%3A+Extracting+large+parallel+corpora+from+movie+and+TV+subtitles%2E+In+Proceedings+of+the+Tenth+International+Conference+on+Language+Resources+and) [our card](https://md.research.elamur.ai/free/en/papers/lison-2016-opensubtitles2016-extracting-large-parallel-corpora/)
13. Mathias Müller, Annette Rios, and Rico Sennrich. 2019. Domain robustness in neural machine translation. [find](https://scholar.google.com/scholar?q=Mathias+Mu%CC%88ller%2C+Annette+Rios%2C+and+Rico+Sennrich%2E+2019%2E+Domain+robustness+in+neural+machine+translation%2E) [our card](https://md.research.elamur.ai/free/en/papers/muller-2020-domain-robustness-neural-machine-translation/)
14. Kenton Murray and David Chiang. 2018. Correcting length bias in neural machine translation. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 212–223, Belgium, Brussels. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Kenton+Murray+and+David+Chiang%2E+2018%2E+Correcting+length+bias+in+neural+machine+translation%2E+In+Proceedings+of+the+Third+Conference+on+Machine+Translation%3A+Research+Papers%2C+pages+212%E2%80%93223%2C+Belgium%2C+Brus)
15. Xing Niu, Weijia Xu, and Marine Carpuat. 2019. Bidirectional differentiable input reconstruction for low-resource neural machine translation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (L… [find](https://scholar.google.com/scholar?q=Xing+Niu%2C+Weijia+Xu%2C+and+Marine+Carpuat%2E+2019%2E+Bidirectional+differentiable+input+reconstruction+for+low%2Dresource+neural+machine+translation%2E+In+Proceedings+of+the+2019+Conference+of+the+North+America)
16. Franz Josef Och. 2003. Minimum error rate training in statistical machine translation. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, pages 160–167, Sapporo, Japan. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Franz+Josef+Och%2E+2003%2E+Minimum+error+rate+training+in+statistical+machine+translation%2E+In+Proceedings+of+the+41st+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+pages+160%E2%80%93167%2C+Sappor)
17. Myle Ott, Michael Auli, David Grangier, and Marc’Aurelio Ranzato. 2018. Analyzing uncertainty in neural machine translation. [find](https://scholar.google.com/scholar?q=Myle+Ott%2C+Michael+Auli%2C+David+Grangier%2C+and+Marc%E2%80%99Aurelio+Ranzato%2E+2018%2E+Analyzing+uncertainty+in+neural+machine+translation%2E) [our card](https://md.research.elamur.ai/free/en/papers/ott-2018-analyzing-uncertainty-neural-machine-translation/)
18. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational… [find](https://scholar.google.com/scholar?q=Kishore+Papineni%2C+Salim+Roukos%2C+Todd+Ward%2C+and+Wei%2DJing+Zhu%2E+2002%2E+Bleu%3A+a+method+for+automatic+evaluation+of+machine+translation%2E+In+Proceedings+of+the+40th+Annual+Meeting+of+the+Association+for+Comp) [our card](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
19. Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186– 191, Belgium, Brussels. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Matt+Post%2E+2018%2E+A+call+for+clarity+in+reporting+BLEU+scores%2E+In+Proceedings+of+the+Third+Conference+on+Machine+Translation%3A+Research+Papers%2C+pages+186%E2%80%93+191%2C+Belgium%2C+Brussels%2E+Association+for+Computa) [our card](https://md.research.elamur.ai/free/en/papers/post-2018-call-clarity-reporting-bleu-scores/)
20. Marc’Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. 2016. Sequence level training with recurrent neural networks. In 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings. [find](https://scholar.google.com/scholar?q=Marc%E2%80%99Aurelio+Ranzato%2C+Sumit+Chopra%2C+Michael+Auli%2C+and+Wojciech+Zaremba%2E+2016%2E+Sequence+level+training+with+recurrent+neural+networks%2E+In+4th+International+Conference+on+Learning+Representations%2C+ICLR+) [our card](https://md.research.elamur.ai/free/en/papers/ranzato-2015-sequence-level-training-recurrent-neural/)
21. Yves Scherrer and Bruno Cartoni. 2012. The trilingual ALLEGRA corpus: Presentation and possible use for lexicon induction. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC-2012), pages 2890–2896, Istanbul, Turkey. European Languages Resources Associati… [find](https://scholar.google.com/scholar?q=Yves+Scherrer+and+Bruno+Cartoni%2E+2012%2E+The+trilingual+ALLEGRA+corpus%3A+Presentation+and+possible+use+for+lexicon+induction%2E+In+Proceedings+of+the+Eighth+International+Conference+on+Language+Resources+a)
22. Rico Sennrich, Orhan Firat, Kyunghyun Cho, Alexandra Birch, Barry Haddow, Julian Hitschler, Marcin Junczys-Dowmunt, Samuel Läubli, Antonio Valerio Miceli Barone, Jozef Mokry, and Maria Nădejde. 2017. Nematus: a toolkit for neural machine translation. In Proceedings of the Software Demonstrations o… [find](https://scholar.google.com/scholar?q=Rico+Sennrich%2C+Orhan+Firat%2C+Kyunghyun+Cho%2C+Alexandra+Birch%2C+Barry+Haddow%2C+Julian+Hitschler%2C+Marcin+Junczys%2DDowmunt%2C+Samuel+La%CC%88ubli%2C+Antonio+Valerio+Miceli+Barone%2C+Jozef+Mokry%2C+and+Maria+Na%CC%86dejde%2E+2017) [our card](https://md.research.elamur.ai/free/en/papers/sennrich-2017-nematus-toolkit-neural-machine-translation/)
23. Zhaopeng Tu, Yang Liu, Lifeng Shang, Xiaohua Liu, and Hang Li. 2017. Neural machine translation with reconstruction. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, AAAI’17, pages 3097–3103. AAAI Press. [find](https://scholar.google.com/scholar?q=Zhaopeng+Tu%2C+Yang+Liu%2C+Lifeng+Shang%2C+Xiaohua+Liu%2C+and+Hang+Li%2E+2017%2E+Neural+machine+translation+with+reconstruction%2E+In+Proceedings+of+the+Thirty%2DFirst+AAAI+Conference+on+Artificial+Intelligence%2C+AAAI)
24. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Proces… [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81+ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+In+I%2E+Guyon%2C+U%2E+V%2E+Luxburg%2C+S%2E+Bengio%2C+H%2E)
25. Sam Wiseman and Alexander M. Rush. 2016. Sequence-to-sequence learning as beam-search optimization. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1296–1306, Austin, Texas. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Sam+Wiseman+and+Alexander+M%2E+Rush%2E+2016%2E+Sequence%2Dto%2Dsequence+learning+as+beam%2Dsearch+optimization%2E+In+Proceedings+of+the+2016+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+129)
26. Felix Wu, Angela Fan, Alexei Baevski, Yann Dauphin, and Michael Auli. 2019. Pay less attention with lightweight and dynamic convolutions. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Felix+Wu%2C+Angela+Fan%2C+Alexei+Baevski%2C+Yann+Dauphin%2C+and+Michael+Auli%2E+2019%2E+Pay+less+attention+with+lightweight+and+dynamic+convolutions%2E+In+International+Conference+on+Learning+Representations%2E) [our card](https://md.research.elamur.ai/free/en/papers/wu-2019-pay-less-attention-lightweight-dynamic/)
27. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715– 1725, Berlin, Germany. Association for Computational Li… [find](https://scholar.google.com/scholar?q=Rico+Sennrich%2C+Barry+Haddow%2C+and+Alexandra+Birch%2E+2016%2E+Neural+machine+translation+of+rare+words+with+subword+units%2E+In+Proceedings+of+the+54th+Annual+Meeting+of+the+Association+for+Computational+Ling)
28. Lijun Wu, Xu Tan, Di He, Fei Tian, Tao Qin, Jianhuang Lai, and Tie-Yan Liu. 2018. Beyond error propagation in neural machine translation: Characteristics of language also matter. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3602–3611, Brussels, Bel… [find](https://scholar.google.com/scholar?q=Lijun+Wu%2C+Xu+Tan%2C+Di+He%2C+Fei+Tian%2C+Tao+Qin%2C+Jianhuang+Lai%2C+and+Tie%2DYan+Liu%2E+2018%2E+Beyond+error+propagation+in+neural+machine+translation%3A+Characteristics+of+language+also+matter%2E+In+Proceedings+of+the)
29. Shiqi Shen, Yong Cheng, Zhongjun He, Wei He, Hua Wu, Maosong Sun, and Yang Liu. 2016. Minimum risk training for neural machine translation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1683–1692, Berlin, Germany. Associatio… [find](https://scholar.google.com/scholar?q=Shiqi+Shen%2C+Yong+Cheng%2C+Zhongjun+He%2C+Wei+He%2C+Hua+Wu%2C+Maosong+Sun%2C+and+Yang+Liu%2E+2016%2E+Minimum+risk+training+for+neural+machine+translation%2E+In+Proceedings+of+the+54th+Annual+Meeting+of+the+Association) [our card](https://md.research.elamur.ai/free/en/papers/shen-2015-minimum-risk-training-neural-machine/)
30. Yilin Yang, Liang Huang, and Mingbo Ma. 2018. Breaking the beam search curse: A study of (re-)scoring methods and stopping criteria for neural machine translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3054–3059, Brussels, Belgium. Associati… [find](https://scholar.google.com/scholar?q=Yilin+Yang%2C+Liang+Huang%2C+and+Mingbo+Ma%2E+2018%2E+Breaking+the+beam+search+curse%3A+A+study+of+%28re%2D%29scoring+methods+and+stopping+criteria+for+neural+machine+translation%2E+In+Proceedings+of+the+2018+Conferenc)
31. Felix Stahlberg and Bill Byrne. 2019. On NMT search errors and model errors: Cat got your tongue? In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3347– 3353, Hong… [find](https://scholar.google.com/scholar?q=Felix+Stahlberg+and+Bill+Byrne%2E+2019%2E+On+NMT+search+errors+and+model+errors%3A+Cat+got+your+tongue%3F+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Processing+and+the+9th+) [our card](https://md.research.elamur.ai/free/en/papers/stahlberg-2019-nmt-search-errors-model-errors/)
32. Wen Zhang, Yang Feng, Fandong Meng, Di You, and Qun Liu. 2019. Bridging the gap between training and inference for neural machine translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4334– 4343, Florence, Italy. Association for Computational… [find](https://scholar.google.com/scholar?q=Wen+Zhang%2C+Yang+Feng%2C+Fandong+Meng%2C+Di+You%2C+and+Qun+Liu%2E+2019%2E+Bridging+the+gap+between+training+and+inference+for+neural+machine+translation%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Associat) [our card](https://md.research.elamur.ai/free/en/papers/zhang-2019-bridging-gap-between-training-inference/)
33. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 27, pages 3104–3112. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Ilya+Sutskever%2C+Oriol+Vinyals%2C+and+Quoc+V+Le%2E+2014%2E+Sequence+to+sequence+learning+with+neural+networks%2E+In+Z%2E+Ghahramani%2C+M%2E+Welling%2C+C%2E+Cortes%2C+N%2E+D%2E+Lawrence%2C+and+K%2E+Q%2E+Weinberger%2C+editors%2C+Advances)
34. C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna. 2016. Rethinking the inception architecture for computer vision. In 2016 IEEE Confer- [find](https://scholar.google.com/scholar?q=C%2E+Szegedy%2C+V%2E+Vanhoucke%2C+S%2E+Ioffe%2C+J%2E+Shlens%2C+and+Z%2E+Wojna%2E+2016%2E+Rethinking+the+inception+architecture+for+computer+vision%2E+In+2016+IEEE+Confer%2D)