---
title: "Breaking the Beam Search Curse — paper by Yang et al. (2018)"
description: "In neural machine translation, widening the beam beyond about 5 usually makes BLEU… 36.22 BLEU. 20 references, cited by 50 papers here."
authors: ["Yilin Yang", "Liang Huang", "Mingbo Ma"]
year: 2018
venue: "Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/D18-1342/"
doi: "10.18653/v1/d18-1342"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/yang-2018-breaking-beam-search-curse-study/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/yang-2018-breaking-beam-search-curse-study/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/4/) / Breaking the Beam Search Curse: A Study of (Re-)Scoring Methods and Stopping Criteria for Neural Machine Translation

# Breaking the Beam Search Curse: A Study of (Re-)Scoring Methods and Stopping Criteria for Neural Machine Translation

**Authors:** Yilin Yang, [Liang Huang](https://md.research.elamur.ai/free/en/authors/liang-huang/) and [Mingbo Ma](https://md.research.elamur.ai/free/en/authors/mingbo-ma/) · **Year:** 2018 · **Venue:** [Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2018](https://md.research.elamur.ai/free/en/venues/emnlp/2018/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/D18-1342/) · [PDF](https://aclanthology.org/D18-1342.pdf) · [DOI 10.18653/v1/d18-1342](https://doi.org/10.18653/v1/d18-1342) · [OpenAlex](https://openalex.org/W2963665552)

**Citations:** 106 citations (OpenAlex, 2026-10-14) · [50](https://md.research.elamur.ai/free/en/papers/yang-2018-breaking-beam-search-curse-study/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
In neural machine translation, widening the beam beyond about 5 usually makes BLEU worse. This is the "beam search curse." Yang, Huang and Ma trace it to output length. A wider beam reaches \</eos> earlier, and the model's scores favor short candidates. Translations therefore shrink, and the brevity penalty lowers BLEU sharply.

To fix this, the authors train a small MLP that predicts the target/source generation ratio from the encoder states. That gives an expected output length. They plug it into three rescoring schemes:
- Bounded Word-Reward with the predicted length
- Bounded Adaptive-Reward, which takes its reward from the current beam
- BP-Norm, which adds a log brevity penalty to the length-normalized score

The last two have no hyperparameters. The authors also derive optimal stopping rules for these scores.

On Chinese-to-English (NIST), the new methods keep length ratios near 1 and improve as the beam grows. The hyperparameter-free variants beat standard length normalization by about +2.0 BLEU. The authors recommend BP-Norm as the simplest choice that performs as well as the others.

## Key points
- **Data and model:** about 2M Chinese–English sentence pairs with BPE, giving 18k Chinese and 10k English vocabularies. NIST 06 is the dev set and NIST 08 the test set. The model is a 2-layer bidirectional LSTM in OpenNMT-py with 28.5M parameters.
- **Test results with large beams (b≈40):** BP-Norm scores 36.22 BLEU and Bounded Adaptive-Reward 36.23. Length normalization scores 34.26, and the default b=5 decoder 32.88.
- **Length ratio:** the proposed methods hold it around 0.97–0.99, compared with roughly 0.87–0.89 for the baselines. They help most on long input sentences.
- **Optimal stopping alone:** adding it to length normalization raises BLEU by about +0.9.
- **Baselines compared:** Moses, GNMT-style length and coverage penalties (α=β=0.3), and tuned Bounded Word-Reward.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Baidu](https://md.research.elamur.ai/free/en/companies/baidu/) — Company, Beijing, China · 40 works · 14,627 citations
- [Oregon State University](https://md.research.elamur.ai/free/en/universities/oregon-state-university/) — University, Corvallis, United States · 6 works · 886 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🔁 Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) · 958 works

## Cite

Cite as: Yilin Yang, Liang Huang and Mingbo Ma. 2018. Breaking the Beam Search Curse: A Study of (Re-)Scoring Methods and Stopping Criteria for Neural Machine Translation. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{yang2018breaking,
  title = {{Breaking the Beam Search Curse: A Study of (Re-)Scoring Methods and Stopping Criteria for Neural Machine Translation}},
  author = {Yilin Yang, Liang Huang and Mingbo Ma},
  booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
  doi = {10.18653/v1/d18-1342},
  url = {https://doi.org/10.18653/v1/d18-1342}
}
```

- [← previous Automatic Stance Detection Using End-to-End Memory Networks](https://md.research.elamur.ai/free/en/papers/mohtarami-2018-automatic-stance-detection-end-end/)
- [next → Revisiting Character-Based Neural Machine Translation with Capacity and Compression](https://md.research.elamur.ai/free/en/papers/cherry-2018-revisiting-character-based-neural-machine/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/4/)

## References

20 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. [Neural machine translation by jointly learning to align and translate](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/). arXiv preprint arXiv:1409.0473.
2. Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N Dauphin. 2017. [Convolutional sequence to sequence learning](https://md.research.elamur.ai/free/en/papers/gehring-2017-convolutional-sequence-sequence-learning/). In Proc. of ICML.
3. Wei He, Zhongjun He, Hua Wu, and Haifeng Wang. 2016. Improved neural machine translation with smt features. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, pages 151–157. AAAI Press. [find](https://scholar.google.com/scholar?q=Wei+He%2C+Zhongjun+He%2C+Hua+Wu%2C+and+Haifeng+Wang%2E+2016%2E+Improved+neural+machine+translation+with+smt+features%2E+In+Proceedings+of+the+Thirtieth+AAAI+Conference+on+Artificial+Intelligence%2C+pages+151%E2%80%93157%2E+A)
4. Liang Huang, Kai Zhao, and Mingbo Ma. 2017. When to finish? optimal beam search for neural text generation (modulo beam size). In EMNLP. [find](https://scholar.google.com/scholar?q=Liang+Huang%2C+Kai+Zhao%2C+and+Mingbo+Ma%2E+2017%2E+When+to+finish%3F+optimal+beam+search+for+neural+text+generation+%28modulo+beam+size%29%2E+In+EMNLP%2E)
5. Nal Kalchbrenner and Phil Blunsom. 2013. [Recurrent continuous translation models](https://md.research.elamur.ai/free/en/papers/kalchbrenner-2013-recurrent-continuous-translation-models/). In EMNLP, volume 3, page 413.
6. G. Klein, Y. Kim, Y. Deng, J. Senellart, and A. M. Rush. 2017. [OpenNMT: Open-Source Toolkit for Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/klein-2017-opennmt-open-source-toolkit-neural/). ArXiv e-prints.
7. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, et al. 2007. Moses: Open source toolkit for statistical machine translation. In Proceedings of the 45th annual meeting of the ACL on interacti… [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2C+Hieu+Hoang%2C+Alexandra+Birch%2C+Chris+Callison%2DBurch%2C+Marcello+Federico%2C+Nicola+Bertoldi%2C+Brooke+Cowan%2C+Wade+Shen%2C+Christine+Moran%2C+Richard+Zens%2C+et+al%2E+2007%2E+Moses%3A+Open+source+toolkit+fo)
8. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015. [Neural machine translation of rare words with subword units](https://md.research.elamur.ai/free/en/papers/sennrich-2016-neural-machine-translation-rare-words/). arXiv preprint arXiv:1508.07909.
9. Shiqi Shen, Yong Cheng, Zhongjun He, Wei He, Hua Wu, Maosong Sun, and Yang Liu. 2016. [Minimum risk training for neural machine translation](https://md.research.elamur.ai/free/en/papers/shen-2015-minimum-risk-training-neural-machine/). In Proceedings of ACL.
10. Xing Shi, Kevin Knight, and Deniz Yuret. 2016. Why neural translations are the right length. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2278–2282. [find](https://scholar.google.com/scholar?q=Xing+Shi%2C+Kevin+Knight%2C+and+Deniz+Yuret%2E+2016%2E+Why+neural+translations+are+the+right+length%2E+In+Proceedings+of+the+2016+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+2278%E2%80%932282%2E)
11. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pages 3104–3112. [find](https://scholar.google.com/scholar?q=Ilya+Sutskever%2C+Oriol+Vinyals%2C+and+Quoc+V+Le%2E+2014%2E+Sequence+to+sequence+learning+with+neural+networks%2E+In+Advances+in+neural+information+processing+systems%2C+pages+3104%E2%80%933112%2E)
12. Zhaopeng Tu, Yang Liu, Lifeng Shang, Xiaohua Liu, and Hang Li. 2017. Neural machine translation with reconstruction. In AAAI, pages 3097–3103. [find](https://scholar.google.com/scholar?q=Zhaopeng+Tu%2C+Yang+Liu%2C+Lifeng+Shang%2C+Xiaohua+Liu%2C+and+Hang+Li%2E+2017%2E+Neural+machine+translation+with+reconstruction%2E+In+AAAI%2C+pages+3097%E2%80%933103%2E)
13. Sam Wiseman and Alexander M Rush. 2016. Sequence-to-sequence learning as beam-search optimization. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Sam+Wiseman+and+Alexander+M+Rush%2E+2016%2E+Sequence%2Dto%2Dsequence+learning+as+beam%2Dsearch+optimization%2E+In+Proceedings+of+EMNLP%2E)
14. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144. [find](https://scholar.google.com/scholar?q=Yonghui+Wu%2C+Mike+Schuster%2C+Zhifeng+Chen%2C+Quoc+V+Le%2C+Mohammad+Norouzi%2C+Wolfgang+Macherey%2C+Maxim+Krikun%2C+Yuan+Cao%2C+Qin+Gao%2C+Klaus+Macherey%2C+et+al%2E+2016%2E+Google%E2%80%99s+neural+machine+translation+system%3A+Bridg)
15. Philipp Koehn and Rebecca Knowles. 2017. Six challenges for neural machine translation. CoRR, abs/1706.03872. [find](https://scholar.google.com/scholar?q=Philipp+Koehn+and+Rebecca+Knowles%2E+2017%2E+Six+challenges+for+neural+machine+translation%2E+CoRR%2C+abs%2F1706%2E03872%2E)
16. John Lafferty, Andrew McCallum, and Fernando Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In Proceedings of ICML. [find](https://scholar.google.com/scholar?q=John+Lafferty%2C+Andrew+McCallum%2C+and+Fernando+Pereira%2E+2001%2E+Conditional+random+fields%3A+Probabilistic+models+for+segmenting+and+labeling+sequence+data%2E+In+Proceedings+of+ICML%2E)
17. Kenton Murray and David Chiang. 2018. Correcting length bias in neural machine translation. In Proceedings of WMT 2018. [find](https://scholar.google.com/scholar?q=Kenton+Murray+and+David+Chiang%2E+2018%2E+Correcting+length+bias+in+neural+machine+translation%2E+In+Proceedings+of+WMT+2018%2E)
18. Myle Ott, Michael Auli, David Grangier, and Marc’Aurelio Ranzato. 2018. [Analyzing uncertainty in neural machine translation](https://md.research.elamur.ai/free/en/papers/ott-2018-analyzing-uncertainty-neural-machine-translation/). arXiv preprint arXiv:1803.00047.
19. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of ACL, pages 311–318, Philadephia, USA.
20. Marc’Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. 2016. [Sequence level training with recurrent neural networks](https://md.research.elamur.ai/free/en/papers/ranzato-2015-sequence-level-training-recurrent-neural/). ICLR.

*Updated 24 October 2026*