---
title: "MBR and QE Finetuning — Shah; 2023"
description: "Decoding strategies that rank candidate translations by estimated quality outperform standard MAP-style decoding, because model likelihood often…"
authors: ["Mara Finkelstein; Subhajit Naskar; Mehdi Mirzazadeh; Apurva Shah;", "Markus Freitag"]
year: 2023
venue: "arXiv:2309.10966"
source: "https://arxiv.org/abs/2309.10966"
doi: "10.48550/arxiv.2309.10966"
arxiv: "2309.10966"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/finkelstein-2023-mbr-qe-finetuning-training-time/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/finkelstein-2023-mbr-qe-finetuning-training-time/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2023](https://md.research.elamur.ai/free/papers/2023/page/4/) / MBR and QE Finetuning: Training-time Distillation of the Best and Most Expensive Decoding Methods

# MBR and QE Finetuning: Training-time Distillation of the Best and Most Expensive Decoding Methods

**Authors:** Mara Finkelstein; Subhajit Naskar; Mehdi Mirzazadeh; Apurva Shah;, [Markus Freitag](https://md.research.elamur.ai/free/authors/markus-freitag/) · **Year:** 2023 · **Venue:** arXiv:2309.10966 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2309.10966) · [PDF](https://arxiv.org/pdf/2309.10966) · [DOI 10.48550/arxiv.2309.10966](https://doi.org/10.48550/arxiv.2309.10966) · [arXiv 2309.10966](https://arxiv.org/abs/2309.10966) · [OpenAlex](https://openalex.org/W4386942720)

**Citations:** 0 citations (openalex-snapshot · 2026-10-14) · [30](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Decoding strategies that rank candidate translations by estimated quality outperform standard MAP-style decoding, because model likelihood often disagrees with what humans prefer. Minimum Bayes Risk (MBR) decoding and Quality Estimation (QE) reranking are two such strategies, but they are too costly to run at inference time. MBR needs pairwise metric comparisons, which grow quadratically with the number of candidates, and QE needs one metric call per candidate. The authors move this cost into training. They sample candidates using epsilon sampling, choose the best one with BLEURT-based MBR or with MetricX-XXL-QE, and finetune an NMT model on the selected outputs for monolingual source text. The finetuned model then translates with cheap beam or greedy search. The data can come from the model itself (self-training) or from an external LLM teacher. They tested this on English–German and English–Japanese. Both variants beat the base model, while finetuning on plain beam-search output hurt quality. With an LLM teacher, the gains even surpassed finetuning on human references. This makes monolingual data a practical substitute for scarce curated parallel data.

## Key points
- The two proposed methods are MBR finetuning (utility metric: BLEURT v0.2) and QE finetuning (utility metric: MetricX-XXL-QE, which shares the architecture of the WMT'22 Metrics winner MetricX-XXL). Candidates are generated with epsilon sampling at ϵ=0.02.
- The language pairs are en→de (base model trained on WMT'21 data, a high-resource pair) and en→ja (medium-resource). QE or MBR finetuning applied on top of a reference-finetuned model gives further gains.
- On en→de with self-training, COMET20 rises from 57.79 for the base model to 59.63 after QE finetuning and 58.02 after MBR finetuning. A balanced mix of MBR and QE data reaches 59.40.
- QE finetuning on 30k sentences roughly matches reference finetuning on 5k sentences, about a 6x data multiplier. The QE-finetuned student outperforms its QE-reranked teacher at every candidate size tested (32–256).
- Several ablations favor the simple setup:
  - Back-translated QE data helps less than forward-translated QE data.
  - Two rounds of iterative QE finetuning perform worse than a single round.
  - After finetuning, plain beam or greedy decoding is often competitive with MBR decoding or QE reranking.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Machine translation](https://md.research.elamur.ai/free/shelves/machine-translation/) · [Natural language processing](https://md.research.elamur.ai/free/shelves/nlp/) · [AI & machine learning](https://md.research.elamur.ai/free/shelves/ai/)

## Cited by

Cited by

30 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 28 not yet.

- [Beyond Correlation: Interpretable Evaluation of Machine Translation Metrics](https://md.research.elamur.ai/free/papers/perrella-2024-beyond-correlation-interpretable-evaluation-machine/)
- [Early-Exit and Instant Confidence Translation Quality Estimation](https://md.research.elamur.ai/free/papers/zouhar-2026-early-exit-instant-confidence-translation/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{shah2023mbr,
  title = {{MBR and QE Finetuning: Training-time Distillation of the Best and Most Expensive Decoding Methods}},
  author = {Shah;, Mara Finkelstein; Subhajit Naskar; Mehdi Mirzazadeh; Apurva and Freitag, Markus},
  year = {2023},
  doi = {10.48550/arxiv.2309.10966},
  eprint = {2309.10966},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2309.10966}
}
```

- [← previous MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://md.research.elamur.ai/free/papers/lu-2023-mathvista-evaluating-mathematical-reasoning-foundation/)
- [next → Meaning Representation of English Prepositional Phrase Roles: SNACS Supersenses vs. Tectogrammatical Functors](https://md.research.elamur.ai/free/papers/scivetti-2023-meaning-representation-english-prepositional-phrase/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/papers/2023/page/4/)

## References

32 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Farhad Akhbardeh, Arkady Arkhangorodsky, Magdalena Biesialska, Ondřej Bojar, Rajen Chatterjee, Vishrav Chaudhary, Marta R. Costa-jussa, Cristina España-Bonet, Angela Fan, Christian Federmann, Markus Freitag, Yvette Graham, Roman Grundkiewicz, Barry Haddow, Leonie Harter, Kenneth Heafield, Christopher Homan, Matthias Huck, Kwabena Amponsah-Kaakyire, Jungo Kasai, Daniel Khashabi, Kevin Knight, Tom Kocmi, Philipp Koehn, Nicholas Lourie, Christof Monz, Makoto Morishita, Masaaki Nagata, Ajay Nagesh, Toshiaki Nakazawa, Matteo Negri, Santanu Pal, Allahsera Auguste Tapo, Marco Turchi, Valentin Vydrin, and Marcos Zampieri. Findings of the 2021 conference on machine translation (WMT21). In Proceedings of the Sixth Conference on Machine Translation, pp. 1–88, Online, November 2021. Association for Computational Linguistics. URL https://aclanthology.org/2021.wmt-1.1. [↗](https://aclanthology.org/2021.wmt-1.1/) [our card](https://md.research.elamur.ai/free/papers/akhbardeh-2021-findings-2021-conference-machine-translation/)
2. Rohan Anil, Andrew M. Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, Eric Chu, Jonathan H. Clark, Laurent El Shafey, Yanping Huang, Kathy Meier-Hellstern, Gaurav Mishra, Erica Moreira, Mark Omernick, Kevin Robinson, Sebastian Ruder, Yi Tay, Kefan Xiao, Yuanzhong Xu, Yujing Zhang, Gustavo Hernandez Abrego, Junwhan Ahn, Jacob Austin, Paul Barham, Jan Botha, James Bradbury, Siddhartha Brahma, Kevin Brooks, Michele Catasta, Yong Cheng, Colin Cherry, Christopher A. Choquette-Choo, Aakanksha Chowdhery, Clément Crepy, Shachi Dave, Mostafa Dehghani, Sunipa Dev, Jacob Devlin, Mark Dı́az, Nan Du, Ethan Dyer, Vlad Feinberg, Fangxiaoyu Feng, Vlad Fienber, Markus Freitag, Xavier Garcia, Sebastian Gehrmann, Lucas Gonzalez, Guy Gur-Ari, Steven Hand, Hadi Hashemi, Le Hou, Joshua Howland, Andrea Hu, Jeffrey Hui, Jeremy Hurwitz, Michael Isard, Abe Ittycheriah, Matthew Jagielski, Wenhao Jia, Kathleen Kenealy, Maxim Krikun, Sneha Kudugunta, Chang Lan, Katherine Lee, Benjamin Lee, Eric Li, Music Li, Wei Li, YaGuang Li, Jian Li, Hyeontaek Lim, Hanzhao Lin, Zhongtao Liu, Frederick Liu, Marcello Maggioni, Aroma Mahendru, Joshua Maynez, Vedant Misra, Maysam Moussalem, Zachary Nado, John Nham, Eric Ni, Andrew Nystrom, Alicia Parrish, Marie Pellat, Martin Polacek, Alex Polozov, Reiner Pope, Siyuan Qiao, Emily Reif, Bryan Richter, Parker Riley, Alex Castro Ros, Aurko Roy, Brennan Saeta, Rajkumar Samuel, Renee Shelby, Ambrose Slone, Daniel Smilkov, David R. So, Daniel Sohn, Simon Tokumine, Dasha Valter, Vijay Vasudevan, Kiran Vodrahalli, Xuezhi Wang, Pidong Wang, Zirui Wang, Tao Wang, John Wieting, Yuhuai Wu, Kelvin Xu, Yunhan Xu, Linting Xue, Pengcheng Yin, Jiahui Yu, Qiao Zhang, Steven Zheng, Ce Zheng, Weikang Zhou, Denny Zhou, Slav Petrov, and Yonghui Wu. Palm 2 technical report, 2023. [find](https://scholar.google.com/scholar?q=Rohan+Anil%2C+Andrew+M%2E+Dai%2C+Orhan+Firat%2C+Melvin+Johnson%2C+Dmitry+Lepikhin%2C+Alexandre+Passos%2C+Siamak+Shakeri%2C+Emanuel+Taropa%2C+Paige+Bailey%2C+Zhifeng+Chen%2C+Eric+Chu%2C+Jonathan+H%2E+Clark%2C+Laurent+El+Shafey%2C+Y) [our card](https://md.research.elamur.ai/free/papers/anil-2023-palm-2-technical-report/)
3. Daniel Deutsch, George Foster, and Markus Freitag. Ties Matter: Modifying Kendall’s Tau for Modern Metric Meta-Evaluation, 2023. URL https://arxiv.org/abs/2305.14324. [↗](https://arxiv.org/abs/2305.14324)
4. Bryan Eikema and Wilker Aziz. Is map decoding all you need? the inadequacy of the mode in neural machine translation. arXiv preprint arXiv:2005.10283, 2020. [↗](https://arxiv.org/abs/2005.10283)
5. Bryan Eikema and Wilker Aziz. Sampling-based approximations to minimum bayes risk decoding for neural machine translation. arXiv preprint arXiv:2108.04718, 2021. [↗](https://arxiv.org/abs/2108.04718)
6. Patrick Fernandes, António Farinhas, Ricardo Rei, José GC de Souza, Perez Ogayo, Graham Neubig, and André FT Martins. Quality-aware decoding for neural machine translation. arXiv preprint arXiv:2205.00978, 2022. [↗](https://arxiv.org/abs/2205.00978)
7. Markus Freitag and Yaser Al-Onaizan. Fast domain adaptation for neural machine translation. arXiv preprint arXiv:1612.06897, 2016. [↗](https://arxiv.org/abs/1612.06897)
8. Markus Freitag, George Foster, David Grangier, Viresh Ratnakar, Qijun Tan, and Wolfgang Macherey. Experts, errors, and context: A large-scale study of human evaluation for machine translation. Transactions of the Association for Computational Linguistics, 9:1460–1474, 2021. doi: 10.1162/ tacl a 00437. URL https://aclanthology.org/2021.tacl-1.87. [↗](https://aclanthology.org/2021.tacl-1.87/) [our card](https://md.research.elamur.ai/free/papers/freitag-2021-experts-errors-context-large-scale/)
9. Markus Freitag, David Grangier, Qijun Tan, and Bowen Liang. High quality rather than high model probability: Minimum bayes risk decoding with neural metrics. Transactions of the Association for Computational Linguistics, 10:811–825, 2022a. [↗](https://doi.org/10.1162/tacl_a_00491)
10. Markus Freitag, Ricardo Rei, Nitika Mathur, Chi-kiu Lo, Craig Stewart, Eleftherios Avramidis, Tom Kocmi, George Foster, Alon Lavie, and André FT Martins. Results of wmt22 metrics shared task: Stop using bleu–neural metrics are better and more robust. In Proceedings of the Seventh Conference on Machine Translation (WMT), pp. 46–68, 2022b. [find](https://scholar.google.com/scholar?q=Markus+Freitag%2C+Ricardo+Rei%2C+Nitika+Mathur%2C+Chi%2Dkiu+Lo%2C+Craig+Stewart%2C+Eleftherios+Avramidis%2C+Tom+Kocmi%2C+George+Foster%2C+Alon+Lavie%2C+and+Andre%CC%81+FT+Martins%2E+Results+of+wmt22+metrics+shared+task%3A+Stop+us) [our card](https://md.research.elamur.ai/free/papers/freitag-2022-results-wmt22-metrics-shared-task/)
11. Markus Freitag, Behrooz Ghorbani, and Patrick Fernandes. Epsilon sampling rocks: Investigating sampling strategies for minimum bayes risk decoding for machine translation. arXiv preprint arXiv:2305.09860, 2023. [↗](https://arxiv.org/abs/2305.09860)
12. Caglar Gulcehre, Tom Le Paine, Srivatsan Srinivasan, Ksenia Konyushkova, Lotte Weerts, Abhishek Sharma, Aditya Siddhant, Alex Ahern, Miaosen Wang, Chenjie Gu, et al. Reinforced self-training (rest) for language modeling. arXiv preprint arXiv:2308.08998, 2023. [↗](https://arxiv.org/abs/2308.08998) [our card](https://md.research.elamur.ai/free/papers/gulcehre-2023-reinforced-self-training-rest-language/)
13. John Hewitt, Christopher D Manning, and Percy Liang. Truncation sampling as language model desmoothing. arXiv preprint arXiv:2210.15191, 2022. [↗](https://arxiv.org/abs/2210.15191)
14. Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015. [↗](https://arxiv.org/abs/1503.02531) [our card](https://md.research.elamur.ai/free/papers/hinton-2015-distilling-knowledge-neural-network/)
15. Norman P Jouppi, Doe Hyun Yoon, George Kurian, Sheng Li, Nishant Patil, James Laudon, Cliff Young, and David Patterson. A domain-specific supercomputer for training deep neural networks. Communications of the ACM, 63(7):67–78, 2020. [↗](https://doi.org/10.1145/3360307)
16. Juraj Juraska, Mara Finkelstein, Daniel Deutsch, Aditya Siddhant, Mehdi Mirzazadeh, and Markus Freitag. Metricx-23: The google submission to the wmt 2023 metrics shared task. In Proceedings of the Eighth Conference on Machine Translation, pp. 754–765, Singapore, December 2023. Association for Computational Linguistics. URL https://aclanthology.org/2023.wmt-1.63. [↗](https://aclanthology.org/2023.wmt-1.63/) [our card](https://md.research.elamur.ai/free/papers/juraska-2023-metricx-23-google-submission-wmt/)
17. Tom Kocmi, Rachel Bawden, Ondřej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Thamme Gowda, Yvette Graham, Roman Grundkiewicz, Barry Haddow, Rebecca Knowles, Philipp Koehn, Christof Monz, Makoto Morishita, Masaaki Nagata, Toshiaki Nakazawa, Michal Novák, Martin Popel, and Maja Popović. Findings of the 2022 conference on machine translation (WMT22). In Proceedings of the Seventh Conference on Machine Translation (WMT), pp. 1–45, Abu Dhabi, United Arab Emirates (Hybrid), December 2022. Association for Computational Linguistics. URL https://aclanthology.org/2022.wmt-1.1. [↗](https://aclanthology.org/2022.wmt-1.1/)
18. Taku Kudo and John Richardson. Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. arXiv preprint arXiv:1808.06226, 2018. [↗](https://arxiv.org/abs/1808.06226) [our card](https://md.research.elamur.ai/free/papers/kudo-2018-sentencepiece-simple-language-independent-subword/)
19. Arle Lommel, Hans Uszkoreit, and Aljoscha Burchardt. Multidimensional quality metrics (MQM): A framework for declaring and describing translation quality metrics. Revista Tradumàtica: tecnologies de la traducció, 2014. [↗](https://doi.org/10.5565/rev/tradumatica.77)
20. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp. 311–318, 2002. [find](https://scholar.google.com/scholar?q=Kishore+Papineni%2C+Salim+Roukos%2C+Todd+Ward%2C+and+Wei%2DJing+Zhu%2E+Bleu%3A+a+method+for+automatic+evaluation+of+machine+translation%2E+In+Proceedings+of+the+40th+annual+meeting+of+the+Association+for+Computatio) [our card](https://md.research.elamur.ai/free/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
21. Maja Popović. chrF: character n-gram F-score for automatic MT evaluation. In Ondřej Bojar, Rajan Chatterjee, Christian Federmann, Barry Haddow, Chris Hokamp, Matthias Huck, Varvara Logacheva, and Pavel Pecina (eds.), Proceedings of the Tenth Workshop on Statistical Machine Translation, pp. 392–395, Lisbon, Portugal, September 2015. Association for Computational Linguistics. doi: 10.18653/v1/W15-3049. URL https://aclanthology.org/W15-3049. [↗](https://doi.org/10.18653/v1/w15-3049) [our card](https://md.research.elamur.ai/free/papers/popovic-2015-chrf-character-n-gram-f/)
22. Ricardo Rei, Craig Stewart, Catarina Farinha, and Alon Lavie. Unbabel’s participation in the wmt20 metrics shared task. arXiv preprint arXiv:2010.15535, 2020. [↗](https://arxiv.org/abs/2010.15535) [our card](https://md.research.elamur.ai/free/papers/rei-2020-unbabel-s-participation-wmt20-metrics/)
23. Ricardo Rei, Ana C Farinha, Chrysoula Zerva, Daan van Stigt, Craig Stewart, Pedro Ramos, Taisiya Glushkova, André FT Martins, and Alon Lavie. Are references really needed? unbabel-ist 2021 submission for the metrics shared task. In Proceedings of the Sixth Conference on Machine Translation, pp. 1030–1040, 2021. [find](https://scholar.google.com/scholar?q=Ricardo+Rei%2C+Ana+C+Farinha%2C+Chrysoula+Zerva%2C+Daan+van+Stigt%2C+Craig+Stewart%2C+Pedro+Ramos%2C+Taisiya+Glushkova%2C+Andre%CC%81+FT+Martins%2C+and+Alon+Lavie%2E+Are+references+really+needed%3F+unbabel%2Dist+2021+submission) [our card](https://md.research.elamur.ai/free/papers/rei-2021-are-references-really-needed-unbabel/)
24. Ricardo Rei, José GC De Souza, Duarte Alves, Chrysoula Zerva, Ana C Farinha, Taisiya Glushkova, Alon Lavie, Luisa Coheur, and André FT Martins. Comet-22: Unbabel-ist 2022 submission for the metrics shared task. In Proceedings of the Seventh Conference on Machine Translation (WMT), pp. 578–585, 2022. [find](https://scholar.google.com/scholar?q=Ricardo+Rei%2C+Jose%CC%81+GC+De+Souza%2C+Duarte+Alves%2C+Chrysoula+Zerva%2C+Ana+C+Farinha%2C+Taisiya+Glushkova%2C+Alon+Lavie%2C+Luisa+Coheur%2C+and+Andre%CC%81+FT+Martins%2E+Comet%2D22%3A+Unbabel%2Dist+2022+submission+for+the+metrics+) [our card](https://md.research.elamur.ai/free/papers/rei-2022-comet-22-unbabel-ist-2022/)
25. Florian Schmidt. Generalization in generation: A closer look at exposure bias. arXiv preprint arXiv:1910.00292, 2019. [↗](https://arxiv.org/abs/1910.00292)
26. Thibault Sellam, Dipanjan Das, and Ankur P Parikh. Bleurt: Learning robust metrics for text generation. arXiv preprint arXiv:2004.04696, 2020. [↗](https://arxiv.org/abs/2004.04696)
27. Rico Sennrich, Barry Haddow, and Alexandra Birch. Improving neural machine translation models with monolingual data. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 86–96, Berlin, Germany, August 2016. Association for Computational Linguistics. doi: 10.18653/v1/P16-1009. URL https://aclanthology.org/P16-1009. [↗](https://doi.org/10.18653/v1/p16-1009)
28. Jonathan Shen, Patrick Nguyen, Yonghui Wu, Zhifeng Chen, Mia X Chen, Ye Jia, Anjuli Kannan, Tara Sainath, Yuan Cao, Chung-Cheng Chiu, et al. Lingvo: a modular and scalable framework for sequence-to-sequence modeling. arXiv preprint arXiv:1902.08295, 2019. [↗](https://arxiv.org/abs/1902.08295)
29. Shiqi Shen, Yong Cheng, Zhongjun He, Wei He, Hua Wu, Maosong Sun, and Yang Liu. Minimum risk training for neural machine translation. arXiv preprint arXiv:1512.02433, 2015. [↗](https://arxiv.org/abs/1512.02433) [our card](https://md.research.elamur.ai/free/papers/shen-2015-minimum-risk-training-neural-machine/)
30. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+Attention+is+all+you+need%2E+Advances+in+neural+information+processing+system)
31. Wei Wang, Taro Watanabe, Macduff Hughes, Tetsuji Nakagawa, and Ciprian Chelba. Denoising neural machine translation training with trusted data and online data selection. arXiv preprint arXiv:1809.00068, 2018. [↗](https://arxiv.org/abs/1809.00068) [our card](https://md.research.elamur.ai/free/papers/wang-2018-denoising-neural-machine-translation-training/)
32. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144, 2016. [↗](https://arxiv.org/abs/1609.08144)