---
title: "A Single Model Ensemble Framework for Neural — Oh 2026"
description: "Translation quality between low-resource or non-English language pairs is still weak."
authors: ["Seokjin Oh", "Keonwoong Noh", "Woohwan Jung"]
year: 2026
venue: "Proceedings of the Fifteenth Language Resources and Evaluation Conference"
source: "https://aclanthology.org/2026.lrec-1.672/"
doi: "10.63317/5jpaoar9p6cf"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/oh-2026-single-model-ensemble-framework-neural/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/oh-2026-single-model-ensemble-framework-neural/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/) / A Single Model Ensemble Framework for Neural Machine Translation Using Pivot Translation

# A Single Model Ensemble Framework for Neural Machine Translation Using Pivot Translation

**Authors:** Seokjin Oh, Keonwoong Noh, Woohwan Jung · **Year:** 2026 · **Venue:** Proceedings of the Fifteenth Language Resources and Evaluation Conference · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.lrec-1.672/) · [PDF](https://aclanthology.org/2026.lrec-1.672.pdf) · [DOI 10.63317/5jpaoar9p6cf](https://doi.org/10.63317/5jpaoar9p6cf) · [OpenAlex](https://openalex.org/W4407131704)

**Citations:** 0 citations (openalex-snapshot · 2026-10-14)

## Summary
Translation quality between low-resource or non-English language pairs is still weak. Classic ensembling helps, but it requires training several models and access to token-level probabilities, which black-box systems such as GPT-4o do not expose. The authors propose PIVOTE, a generation-based ensemble that runs on a single multilingual NMT model (e.g., NLLB).

The method has two steps. First, each source sentence is translated directly and also through several high-resource pivot languages, giving a pool of varied target-language candidates. Second, quality estimation ranks the candidates, and the top-k are fused into a new translation by a merging module (FiD, TRICE, or an LLM such as GPT-4).

Experiments use FLORES-200, news-commentary, WikiMatrix and TED 2020 data for distant and similar language pairs, including Ukrainian ones. PIVOTE is reported to consistently beat selection- and generation-based ensembles such as MBR, LLM-Blender and EVA. This matters because it delivers ensemble gains at lower cost and also works with closed models.

## Key points
- Language pairs include KO↔IT, AR↔PT, PT↔RU, NL↔RU, FR↔UK, ES↔PT and UK↔RU. Each has 2,000 dev and 2,000 test sentences.
  - FR↔UK training data comes from WikiMatrix v1 (166,063 sentences).
  - UK↔RU training data comes from TED 2020 v1 (197,978 sentences).
- Pivot paths are chosen on FLORES-200. English, Spanish, Portuguese and the direct path usually rank highest. Selecting paths by BLEU versus COMET changes results only marginally.
- Ukrainian (mid-resource) and Croatian (low-resource) pivots give weaker candidates for Korean→Italian with NLLB. Their BLEU is 11.95 and 12.25, compared with 13.87 for the Spanish pivot and 14.77 for the English pivot.
- On Korean→Italian with GPT-4 merging, the choice of pivots matters:
  - English+Spanish candidates: 16.27 BLEU / 86.50 COMET.
  - Ukrainian+Croatian candidates: 15.28 BLEU / 85.75 COMET.
- The MBR baseline performs best at temperature 0.0 (14.10 BLEU on Korean→Italian).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

Topics: [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{oh2026single,
  title = {{A Single Model Ensemble Framework for Neural Machine Translation Using Pivot Translation}},
  author = {Oh, Seokjin and Noh, Keonwoong and Jung, Woohwan},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/5jpaoar9p6cf},
  url = {https://doi.org/10.63317/5jpaoar9p6cf}
}
```

- [← previous A Severity-Based Curriculum Learning Strategy for Arabic Medical Text Generation](https://md.research.elamur.ai/free/en/papers/alansary-2026-severity-based-curriculum-learning-strategy/)
- [next → A Sociophonetic Analysis of Racial Bias in Commercial ASR Systems Using the Pacific Northwest English Corpus](https://md.research.elamur.ai/free/en/papers/scott-2026-sociophonetic-analysis-racial-bias-commercial/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/)

## References

68 references, 26 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. AI@Meta. 2024. Llama 3 model card. [find](https://scholar.google.com/scholar?q=AI%40Meta%2E+2024%2E+Llama+3+model+card%2E)
2. Alham Fikri Aji, Nikolay Bogoychev, Kenneth Heafield, and Rico Sennrich. 2020. In neural machine translation, what does transfer learning transfer? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7701–7710, Online. Association for Computational Lingu… [find](https://scholar.google.com/scholar?q=Alham+Fikri+Aji%2C+Nikolay+Bogoychev%2C+Kenneth+Heafield%2C+and+Rico+Sennrich%2E+2020%2E+In+neural+machine+translation%2C+what+does+transfer+learning+transfer%3F+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Ass)
3. Mikel Artetxe, Gorka Labaka, Eneko Agirre, and Kyunghyun Cho. 2018. Unsupervised neural machine translation. [find](https://scholar.google.com/scholar?q=Mikel+Artetxe%2C+Gorka+Labaka%2C+Eneko+Agirre%2C+and+Kyunghyun+Cho%2E+2018%2E+Unsupervised+neural+machine+translation%2E) [our card](https://md.research.elamur.ai/free/en/papers/artetxe-2017-unsupervised-neural-machine-translation/)
4. Baichuan. 2023. Baichuan 2: Open large-scale language models. [find](https://scholar.google.com/scholar?q=Baichuan%2E+2023%2E+Baichuan+2%3A+Open+large%2Dscale+language+models%2E) [our card](https://md.research.elamur.ai/free/en/papers/yang-2023-baichuan-2-open-large-scale/)
5. Ondřej Bojar, Christian Buck, Christian Federmann, Barry Haddow, Philipp Koehn, Johannes Leveling, Christof Monz, Pavel Pecina, Matt Post, Herve Saint-Amand, Radu Soricut, Lucia Specia, and Aleš Tamchyna. 2014. Findings of the 2014 workshop on statistical machine translation. In Proceedings of the… [find](https://scholar.google.com/scholar?q=Ondr%CC%8Cej+Bojar%2C+Christian+Buck%2C+Christian+Federmann%2C+Barry+Haddow%2C+Philipp+Koehn%2C+Johannes+Leveling%2C+Christof+Monz%2C+Pavel+Pecina%2C+Matt+Post%2C+Herve+Saint%2DAmand%2C+Radu+Soricut%2C+Lucia+Specia%2C+and+Ale%C5%A1+Tamc) [our card](https://md.research.elamur.ai/free/en/papers/bojar-2014-findings-2014-workshop-statistical-machine/)
6. Ye Chen, Wei Cai, Liangmin Wu, Xiaowei Li, Zhanxuan Xin, and Cong Fu. 2023. Tigerbot: An open multilingual multitask llm. [find](https://scholar.google.com/scholar?q=Ye+Chen%2C+Wei+Cai%2C+Liangmin+Wu%2C+Xiaowei+Li%2C+Zhanxuan+Xin%2C+and+Cong+Fu%2E+2023%2E+Tigerbot%3A+An+open+multilingual+multitask+llm%2E)
7. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. [find](https://scholar.google.com/scholar?q=Wei%2DLin+Chiang%2C+Zhuohan+Li%2C+Zi+Lin%2C+Ying+Sheng%2C+Zhanghao+Wu%2C+Hao+Zhang%2C+Lianmin+Zheng%2C+Siyuan+Zhuang%2C+Yonghao+Zhuang%2C+Joseph+E%2E+Gonzalez%2C+Ion+Stoica%2C+and+Eric+P%2E+Xing%2E+2023%2E+Vicuna%3A+An+open%2Dsource+cha)
8. Hyung Won Chung, Le Hou, S. Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Wei Yu, Vincent Zhao, Ya… [find](https://scholar.google.com/scholar?q=Hyung+Won+Chung%2C+Le+Hou%2C+S%2E+Longpre%2C+Barret+Zoph%2C+Yi+Tay%2C+William+Fedus%2C+Eric+Li%2C+Xuezhi+Wang%2C+Mostafa+Dehghani%2C+Siddhartha+Brahma%2C+Albert+Webson%2C+Shixiang+Shane+Gu%2C+Zhuyun+Dai%2C+Mirac+Suzgun%2C+Xinyun+C)
9. Mike Conover, Matt Hayes, Ankit Mathur, Xiangrui Meng, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. 2023. Free dolly: Introducing the world’s first truly open instruction-tuned llm. [find](https://scholar.google.com/scholar?q=Mike+Conover%2C+Matt+Hayes%2C+Ankit+Mathur%2C+Xiangrui+Meng%2C+Jianwei+Xie%2C+Jun+Wan%2C+Sam+Shah%2C+Ali+Ghodsi%2C+Patrick+Wendell%2C+Matei+Zaharia%2C+and+Reynold+Xin%2E+2023%2E+Free+dolly%3A+Introducing+the+world%E2%80%99s+first+trul)
10. Marta R Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, et al. 2022. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672. [find](https://scholar.google.com/scholar?q=Marta+R+Costa%2Djuss%C3%A0%2C+James+Cross%2C+Onur+%C3%87elebi%2C+Maha+Elbayad%2C+Kenneth+Heafield%2C+Kevin+Heffernan%2C+Elahe+Kalbassi%2C+Janice+Lam%2C+Daniel+Licht%2C+Jean+Maillard%2C+et+al%2E+2022%2E+No+language+left+behind%3A+Scaling+h)
11. Raj Dabre, Fabien Cromieres, Sadao Kurohashi, and Pushpak Bhattacharyya. 2015. Leveraging small multilingual corpora for SMT using many pivot languages. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… [find](https://scholar.google.com/scholar?q=Raj+Dabre%2C+Fabien+Cromieres%2C+Sadao+Kurohashi%2C+and+Pushpak+Bhattacharyya%2E+2015%2E+Leveraging+small+multilingual+corpora+for+SMT+using+many+pivot+languages%2E+In+Proceedings+of+the+2015+Conference+of+the+No)
12. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2024. Qlora: Efficient fine-tuning of quantized llms. Advances in Neural Information Processing Systems, 36. [find](https://scholar.google.com/scholar?q=Tim+Dettmers%2C+Artidoro+Pagnoni%2C+Ari+Holtzman%2C+and+Luke+Zettlemoyer%2E+2024%2E+Qlora%3A+Efficient+fine%2Dtuning+of+quantized+llms%2E+Advances+in+Neural+Information+Processing+Systems%2C+36%2E)
13. Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. 2022. GLM: General language model pretraining with autoregressive blank infilling. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 320–335… [find](https://scholar.google.com/scholar?q=Zhengxiao+Du%2C+Yujie+Qian%2C+Xiao+Liu%2C+Ming+Ding%2C+Jiezhong+Qiu%2C+Zhilin+Yang%2C+and+Jie+Tang%2E+2022%2E+GLM%3A+General+language+model+pretraining+with+autoregressive+blank+infilling%2E+In+Proceedings+of+the+60th+An) [our card](https://md.research.elamur.ai/free/en/papers/du-2022-glm-general-language-model-pretraining/)
14. Angela Fan, Shruti Bhosale, Holger Schwenk, Zhiyi Ma, Ahmed El-Kishky, Siddharth Goyal, Mandeep Baines, Onur Celebi, Guillaume Wenzek, Vishrav Chaudhary, Naman Goyal, Tom Birch, Vitaliy Liptchinsky, Sergey Edunov, Edouard Grave, Michael Auli, and Armand Joulin. 2020. Beyond english-centric multiling… [find](https://scholar.google.com/scholar?q=Angela+Fan%2C+Shruti+Bhosale%2C+Holger+Schwenk%2C+Zhiyi+Ma%2C+Ahmed+El%2DKishky%2C+Siddharth+Goyal%2C+Mandeep+Baines%2C+Onur+Celebi%2C+Guillaume+Wenzek%2C+Vishrav+Chaudhary%2C+Naman+Goyal%2C+Tom+Birch%2C+Vitaliy+Liptchinsky%2C+S)
15. António Farinhas, José de Souza, and Andre Martins. 2023. An empirical study of translation hypothesis ensembling with large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 11956–11970, Singapore. Association for Computational Linguis… [find](https://scholar.google.com/scholar?q=Ant%C3%B3nio+Farinhas%2C+Jos%C3%A9+de+Souza%2C+and+Andre+Martins%2E+2023%2E+An+empirical+study+of+translation+hypothesis+ensembling+with+large+language+models%2E+In+Proceedings+of+the+2023+Conference+on+Empirical+Methods)
16. Patrick Fernandes, António Farinhas, Ricardo Rei, José G. C. de Souza, Perez Ogayo, Graham Neubig, and Andre Martins. 2022. Quality-aware decoding for neural machine translation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Hum… [find](https://scholar.google.com/scholar?q=Patrick+Fernandes%2C+Ant%C3%B3nio+Farinhas%2C+Ricardo+Rei%2C+Jos%C3%A9+G%2E+C%2E+de+Souza%2C+Perez+Ogayo%2C+Graham+Neubig%2C+and+Andre+Martins%2E+2022%2E+Quality%2Daware+decoding+for+neural+machine+translation%2E+In+Proceedings+of+the) [our card](https://md.research.elamur.ai/free/en/papers/fernandes-2022-quality-aware-decoding-neural-machine/)
17. Orhan Firat, Baskaran Sankaran, Yaser Al-onaizan, Fatos T. Yarman Vural, and Kyunghyun Cho. 2016. Zero-resource translation with multilingual neural machine translation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 268–277, Austin, Texas. Associati… [find](https://scholar.google.com/scholar?q=Orhan+Firat%2C+Baskaran+Sankaran%2C+Yaser+Al%2Donaizan%2C+Fatos+T%2E+Yarman+Vural%2C+and+Kyunghyun+Cho%2E+2016%2E+Zero%2Dresource+translation+with+multilingual+neural+machine+translation%2E+In+Proceedings+of+the+2016+Con)
18. Xinyang Geng, Arnav Gudibande, Hao Liu, Eric Wallace, Pieter Abbeel, Sergey Levine, and Dawn Song. 2023. Koala: A dialogue model for academic research. Blog post. [find](https://scholar.google.com/scholar?q=Xinyang+Geng%2C+Arnav+Gudibande%2C+Hao+Liu%2C+Eric+Wallace%2C+Pieter+Abbeel%2C+Sergey+Levine%2C+and+Dawn+Song%2E+2023%2E+Koala%3A+A+dialogue+model+for+academic+research%2E+Blog+post%2E)
19. Vaibhava Goel and William J Byrne. 2000. Minimum bayes-risk automatic speech recognition. Computer Speech & Language, 14(2):115– 135. [find](https://scholar.google.com/scholar?q=Vaibhava+Goel+and+William+J+Byrne%2E+2000%2E+Minimum+bayes%2Drisk+automatic+speech+recognition%2E+Computer+Speech+%26+Language%2C+14%282%29%3A115%E2%80%93+135%2E)
20. Nuno M. Guerreiro, Duarte Alves, Jonas Waldendorf, Barry Haddow, Alexandra Birch, Pierre Colombo, and André F. T. Martins. 2023. Hallucinations in large multilingual translation models. [find](https://scholar.google.com/scholar?q=Nuno+M%2E+Guerreiro%2C+Duarte+Alves%2C+Jonas+Waldendorf%2C+Barry+Haddow%2C+Alexandra+Birch%2C+Pierre+Colombo%2C+and+Andr%C3%A9+F%2E+T%2E+Martins%2E+2023%2E+Hallucinations+in+large+multilingual+translation+models%2E) [our card](https://md.research.elamur.ai/free/en/papers/guerreiro-2023-hallucinations-large-multilingual-translation-models/)
21. Yinpeng Guo, Yi Liao, Xin Jiang, Qing Zhang, Yibo Zhang, and Qun Liu. 2019. Zero-shot paraphrase generation with multilingual language models. [find](https://scholar.google.com/scholar?q=Yinpeng+Guo%2C+Yi+Liao%2C+Xin+Jiang%2C+Qing+Zhang%2C+Yibo+Zhang%2C+and+Qun+Liu%2E+2019%2E+Zero%2Dshot+paraphrase+generation+with+multilingual+language+models%2E)
22. Zhiwei He, Xing Wang, Zhaopeng Tu, Shuming Shi, and Rui Wang. 2022. Tencent AI lab - shanghai jiao tong university low-resource translation system for the WMT22 translation task. In Proceedings of the Seventh Conference on Machine Translation (WMT), pages 260–267, Abu Dhabi, United Arab Emirates (Hy… [find](https://scholar.google.com/scholar?q=Zhiwei+He%2C+Xing+Wang%2C+Zhaopeng+Tu%2C+Shuming+Shi%2C+and+Rui+Wang%2E+2022%2E+Tencent+AI+lab+%2D+shanghai+jiao+tong+university+low%2Dresource+translation+system+for+the+WMT22+translation+task%2E+In+Proceedings+of+the)
23. Amr Hendy, Mohamed Abdelrehim, Amr Sharaf, Vikas Raunak, Mohamed Gabr, Hitokazu Matsushita, Young Jin Kim, Mohamed Afify, and Hany Hassan Awadalla. 2023. How good are gpt models at machine translation? a comprehensive evaluation. [find](https://scholar.google.com/scholar?q=Amr+Hendy%2C+Mohamed+Abdelrehim%2C+Amr+Sharaf%2C+Vikas+Raunak%2C+Mohamed+Gabr%2C+Hitokazu+Matsushita%2C+Young+Jin+Kim%2C+Mohamed+Afify%2C+and+Hany+Hassan+Awadalla%2E+2023%2E+How+good+are+gpt+models+at+machine+translation)
24. Gao Huang, Yixuan Li, Geoff Pleiss, Zhuang Liu, John E. Hopcroft, and Kilian Q. Weinberger. 2017. Snapshot ensembles: Train 1, get m for free. [find](https://scholar.google.com/scholar?q=Gao+Huang%2C+Yixuan+Li%2C+Geoff+Pleiss%2C+Zhuang+Liu%2C+John+E%2E+Hopcroft%2C+and+Kilian+Q%2E+Weinberger%2E+2017%2E+Snapshot+ensembles%3A+Train+1%2C+get+m+for+free%2E)
25. Xuancheng Huang, Jingfang Xu, Maosong Sun, and Yang Liu. 2021. Transfer learning for sequence generation: from single-source to multi-source. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Pro… [find](https://scholar.google.com/scholar?q=Xuancheng+Huang%2C+Jingfang+Xu%2C+Maosong+Sun%2C+and+Yang+Liu%2E+2021%2E+Transfer+learning+for+sequence+generation%3A+from+single%2Dsource+to+multi%2Dsource%2E+In+Proceedings+of+the+59th+Annual+Meeting+of+the+Associati)
26. Gautier Izacard and Edouard Grave. 2021. Leveraging passage retrieval with generative models for open domain question answering. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 874– 880, Online. Association for Comput… [find](https://scholar.google.com/scholar?q=Gautier+Izacard+and+Edouard+Grave%2E+2021%2E+Leveraging+passage+retrieval+with+generative+models+for+open+domain+question+answering%2E+In+Proceedings+of+the+16th+Conference+of+the+European+Chapter+of+the+As) [our card](https://md.research.elamur.ai/free/en/papers/izacard-2021-leveraging-passage-retrieval-generative-models/)
27. Dongfu Jiang, Xiang Ren, and Bill Yuchen Lin. 2023. LLM-blender: Ensembling large language models with pairwise ranking and generative fusion. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14165–14178, Toronto, Canada. Assoc… [find](https://scholar.google.com/scholar?q=Dongfu+Jiang%2C+Xiang+Ren%2C+and+Bill+Yuchen+Lin%2E+2023%2E+LLM%2Dblender%3A+Ensembling+large+language+models+with+pairwise+ranking+and+generative+fusion%2E+In+Proceedings+of+the+61st+Annual+Meeting+of+the+Associat) [our card](https://md.research.elamur.ai/free/en/papers/jiang-2023-llm-blender-ensembling-large-language/)
28. Yunsu Kim, Petre Petrov, Pavel Petrushkov, Shahram Khadivi, and Hermann Ney. 2019. Pivot-based transfer learning for neural machine translation between non-English languages. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Con… [find](https://scholar.google.com/scholar?q=Yunsu+Kim%2C+Petre+Petrov%2C+Pavel+Petrushkov%2C+Shahram+Khadivi%2C+and+Hermann+Ney%2E+2019%2E+Pivot%2Dbased+transfer+learning+for+neural+machine+translation+between+non%2DEnglish+languages%2E+In+Proceedings+of+the+201)
29. LAION-AI. 2023. Open assistant. [https://github.com/LAION-AI/Open-Assistant](https://github.com/LAION-AI/Open-Assistant). [↗](https://github.com/LAION-AI/Open-Assistant)
30. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Ilya+Loshchilov+and+Frank+Hutter%2E+2019%2E+Decoupled+weight+decay+regularization%2E+In+International+Conference+on+Learning+Representations%2E)
31. Jonathan Mallinson, Rico Sennrich, and Mirella Lapata. 2017. Paraphrasing revisited with neural machine translation. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers, pages 881–893, Valencia, Spain. Association for… [find](https://scholar.google.com/scholar?q=Jonathan+Mallinson%2C+Rico+Sennrich%2C+and+Mirella+Lapata%2E+2017%2E+Paraphrasing+revisited+with+neural+machine+translation%2E+In+Proceedings+of+the+15th+Conference+of+the+European+Chapter+of+the+Association+fo) [our card](https://md.research.elamur.ai/free/en/papers/mallinson-2017-paraphrasing-revisited-neural-machine-translation/)
32. Alireza Mohammadshahi, Jannis Vamvas, and Rico Sennrich. 2024. Investigating multi-pivot ensembling with massively multilingual machine translation models. In Proceedings of the Fifth Workshop on Insights from Negative Results in NLP, pages 169–180, Mexico City, Mexico. Association for Computational… [find](https://scholar.google.com/scholar?q=Alireza+Mohammadshahi%2C+Jannis+Vamvas%2C+and+Rico+Sennrich%2E+2024%2E+Investigating+multi%2Dpivot+ensembling+with+massively+multilingual+machine+translation+models%2E+In+Proceedings+of+the+Fifth+Workshop+on+Insi)
33. NLP Team MosaicML. 2023. Introducing mpt-7b: A new standard for open-source, ly usable llms. Accessed: 2023-05-23. [find](https://scholar.google.com/scholar?q=NLP+Team+MosaicML%2E+2023%2E+Introducing+mpt%2D7b%3A+A+new+standard+for+open%2Dsource%2C+ly+usable+llms%2E+Accessed%3A+2023%2D05%2D23%2E)
34. Yasmin Moslem, Rejwanul Haque, John D. Kelleher, and Andy Way. 2023. Adaptive machine translation with large language models. In Proceedings of the 24th Annual Conference of the European Association for Machine Translation, pages 227–237, Tampere, Finland. European Association for Machine Translatio… [find](https://scholar.google.com/scholar?q=Yasmin+Moslem%2C+Rejwanul+Haque%2C+John+D%2E+Kelleher%2C+and+Andy+Way%2E+2023%2E+Adaptive+machine+translation+with+large+language+models%2E+In+Proceedings+of+the+24th+Annual+Conference+of+the+European+Association+f) [our card](https://md.research.elamur.ai/free/en/papers/moslem-2023-adaptive-machine-translation-large-language/)
35. OpenAI. 2023. Gpt-4 technical report. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2023%2E+Gpt%2D4+technical+report%2E) [our card](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/)
36. OpenAI. 2024. Gpt-4o system card. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2024%2E+Gpt%2D4o+system+card%2E) [our card](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/)
37. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744. [find](https://scholar.google.com/scholar?q=Long+Ouyang%2C+Jeffrey+Wu%2C+Xu+Jiang%2C+Diogo+Almeida%2C+Carroll+Wainwright%2C+Pamela+Mishkin%2C+Chong+Zhang%2C+Sandhini+Agarwal%2C+Katarina+Slama%2C+Alex+Ray%2C+et+al%2E+2022%2E+Training+language+models+to+follow+instructi)
38. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational… [find](https://scholar.google.com/scholar?q=Kishore+Papineni%2C+Salim+Roukos%2C+Todd+Ward%2C+and+Wei%2DJing+Zhu%2E+2002%2E+Bleu%3A+a+method+for+automatic+evaluation+of+machine+translation%2E+In+Proceedings+of+the+40th+Annual+Meeting+of+the+Association+for+Comp) [our card](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
39. Michael Paul, Hirofumi Yamamoto, Eiichiro Sumita, and Satoshi Nakamura. 2009. On the importance of pivot language selection for statistical machine translation. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computationa… [find](https://scholar.google.com/scholar?q=Michael+Paul%2C+Hirofumi+Yamamoto%2C+Eiichiro+Sumita%2C+and+Satoshi+Nakamura%2E+2009%2E+On+the+importance+of+pivot+language+selection+for+statistical+machine+translation%2E+In+Proceedings+of+Human+Language+Techno)
40. Keqin Peng, Liang Ding, Qihuang Zhong, Li Shen, Xuebo Liu, Min Zhang, Yuanxin Ouyang, and Dacheng Tao. 2023. Towards making the most of chatgpt for machine translation. [find](https://scholar.google.com/scholar?q=Keqin+Peng%2C+Liang+Ding%2C+Qihuang+Zhong%2C+Li+Shen%2C+Xuebo+Liu%2C+Min+Zhang%2C+Yuanxin+Ouyang%2C+and+Dacheng+Tao%2E+2023%2E+Towards+making+the+most+of+chatgpt+for+machine+translation%2E) [our card](https://md.research.elamur.ai/free/en/papers/peng-2023-making-most-chatgpt-machine-translation/)
41. Maja Popović. 2017. chrF++: words helping character n-grams. In Proceedings of the Second Conference on Machine Translation, pages 612–618, Copenhagen, Denmark. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Maja+Popovic%CC%81%2E+2017%2E+chrF%2B%2B%3A+words+helping+character+n%2Dgrams%2E+In+Proceedings+of+the+Second+Conference+on+Machine+Translation%2C+pages+612%E2%80%93618%2C+Copenhagen%2C+Denmark%2E+Association+for+Computational+Linguist) [our card](https://md.research.elamur.ai/free/en/papers/popovic-2017-chrf-words-helping-character-n/)
42. Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186–191, Belgium, Brussels. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Matt+Post%2E+2018%2E+A+call+for+clarity+in+reporting+BLEU+scores%2E+In+Proceedings+of+the+Third+Conference+on+Machine+Translation%3A+Research+Papers%2C+pages+186%E2%80%93191%2C+Belgium%2C+Brussels%2E+Association+for+Computat) [our card](https://md.research.elamur.ai/free/en/papers/post-2018-call-clarity-reporting-bleu-scores/)
43. Ricardo Rei, José G. C. de Souza, Duarte Alves, Chrysoula Zerva, Ana C Farinha, Taisiya Glushkova, Alon Lavie, Luisa Coheur, and André F. T. Martins. 2022a. COMET-22: Unbabel-IST 2022 submission for the metrics shared task. In Proceedings of the Seventh Conference on Machine Translation (WMT), pages… [find](https://scholar.google.com/scholar?q=Ricardo+Rei%2C+Jos%C3%A9+G%2E+C%2E+de+Souza%2C+Duarte+Alves%2C+Chrysoula+Zerva%2C+Ana+C+Farinha%2C+Taisiya+Glushkova%2C+Alon+Lavie%2C+Luisa+Coheur%2C+and+Andr%C3%A9+F%2E+T%2E+Martins%2E+2022a%2E+COMET%2D22%3A+Unbabel%2DIST+2022+submission+for+t) [our card](https://md.research.elamur.ai/free/en/papers/rei-2022-comet-22-unbabel-ist-2022/)
44. Ricardo Rei, Marcos Treviso, Nuno M. Guerreiro, Chrysoula Zerva, Ana C. Farinha, Christine Maroti, José G. C. de Souza, Taisiya Glushkova, Duarte M. Alves, Alon Lavie, Luisa Coheur, and André F. T. Martins. 2022b. [find](https://scholar.google.com/scholar?q=Ricardo+Rei%2C+Marcos+Treviso%2C+Nuno+M%2E+Guerreiro%2C+Chrysoula+Zerva%2C+Ana+C%2E+Farinha%2C+Christine+Maroti%2C+Jos%C3%A9+G%2E+C%2E+de+Souza%2C+Taisiya+Glushkova%2C+Duarte+M%2E+Alves%2C+Alon+Lavie%2C+Luisa+Coheur%2C+and+Andr%C3%A9+F%2E+T%2E+Ma)
45. Cometkiwi: Ist-unbabel 2022 submission for the quality estimation shared task. [find](https://scholar.google.com/scholar?q=Cometkiwi%3A+Ist%2Dunbabel+2022+submission+for+the+quality+estimation+shared+task%2E) [our card](https://md.research.elamur.ai/free/en/papers/rei-2022-cometkiwi-ist-unbabel-2022-submission/)
46. Nathaniel Robinson, Perez Ogayo, David R. Mortensen, and Graham Neubig. 2023. Chat-GPT MT: Competitive for high- (but not low-) resource languages. In Proceedings of the Eighth Conference on Machine Translation, pages 392–418, Singapore. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Nathaniel+Robinson%2C+Perez+Ogayo%2C+David+R%2E+Mortensen%2C+and+Graham+Neubig%2E+2023%2E+Chat%2DGPT+MT%3A+Competitive+for+high%2D+%28but+not+low%2D%29+resource+languages%2E+In+Proceedings+of+the+Eighth+Conference+on+Machine+T)
47. Omer Sagi and Lior Rokach. 2018. Ensemble learning: A survey. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 8. [find](https://scholar.google.com/scholar?q=Omer+Sagi+and+Lior+Rokach%2E+2018%2E+Ensemble+learning%3A+A+survey%2E+Wiley+Interdisciplinary+Reviews%3A+Data+Mining+and+Knowledge+Discovery%2C+8%2E)
48. Stability-AI. 2023. Stablelm: Stability ai language models. [https://github.com/stability-AI/stableLM](https://github.com/stability-AI/stableLM). [↗](https://github.com/stability-AI/stableLM)
49. Tianxiang Sun and Xipeng Qiu. 2023. Moss. [https://github.com/OpenLMLab/MOSS](https://github.com/OpenLMLab/MOSS). [↗](https://github.com/OpenLMLab/MOSS)
50. Mirac Suzgun, Luke Melas-Kyriazi, and Dan Jurafsky. 2023. Follow the wisdom of the crowd: Effective text generation via minimum Bayes risk decoding. In Findings of the Association for Computational Linguistics: ACL 2023, pages 4265–4293, Toronto, Canada. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Mirac+Suzgun%2C+Luke+Melas%2DKyriazi%2C+and+Dan+Jurafsky%2E+2023%2E+Follow+the+wisdom+of+the+crowd%3A+Effective+text+generation+via+minimum+Bayes+risk+decoding%2E+In+Findings+of+the+Association+for+Computational+Li)
51. Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction-following llama model. [https://github.com/tatsu-lab/stanford_alpaca](https://github.com/tatsu-lab/stanford_alpaca). [↗](https://github.com/tatsu-lab/stanford_alpaca)
52. Gemini Team. 2023. Gemini: A family of highly capable multimodal models. [find](https://scholar.google.com/scholar?q=Gemini+Team%2E+2023%2E+Gemini%3A+A+family+of+highly+capable+multimodal+models%2E) [our card](https://md.research.elamur.ai/free/en/papers/gemini-2023-team-gemini-family-highly-capable-multimodal/)
53. Brian Thompson and Matt Post. 2020. Paraphrase generation as zero-shot multilingual translation: Disentangling semantic similarity from lexical and syntactic diversity. In Proceedings of the Fifth Conference on Machine Translation, pages 561–570, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Brian+Thompson+and+Matt+Post%2E+2020%2E+Paraphrase+generation+as+zero%2Dshot+multilingual+translation%3A+Disentangling+semantic+similarity+from+lexical+and+syntactic+diversity%2E+In+Proceedings+of+the+Fifth+Con) [our card](https://md.research.elamur.ai/free/en/papers/thompson-2020-paraphrase-generation-as-zero-shot/)
54. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,… [find](https://scholar.google.com/scholar?q=Hugo+Touvron%2C+Louis+Martin%2C+Kevin+Stone%2C+Peter+Albert%2C+Amjad+Almahairi%2C+Yasmine+Babaei%2C+Nikolay+Bashlykov%2C+Soumya+Batra%2C+Prajjwal+Bhargava%2C+Shruti+Bhosale%2C+Dan+Bikel%2C+Lukas+Blecher%2C+Cristian+Canton+Fe)
55. Masao Utiyama and Hitoshi Isahara. 2007. A comparison of pivot methods for phrase-based statistical machine translation. In Human Language Technologies 2007: The Conference of the North American Chapter of the Association for Computational Linguistics; Proceedings of the Main Conference, pages 484–4… [find](https://scholar.google.com/scholar?q=Masao+Utiyama+and+Hitoshi+Isahara%2E+2007%2E+A+comparison+of+pivot+methods+for+phrase%2Dbased+statistical+machine+translation%2E+In+Human+Language+Technologies+2007%3A+The+Conference+of+the+North+American+Chapt) [our card](https://md.research.elamur.ai/free/en/papers/utiyama-2007-comparison-pivot-methods-phrase-based/)
56. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, and Denny Zhou. 2022. Rationale-augmented ensembles in language models. [find](https://scholar.google.com/scholar?q=Xuezhi+Wang%2C+Jason+Wei%2C+Dale+Schuurmans%2C+Quoc+Le%2C+Ed+Chi%2C+and+Denny+Zhou%2E+2022%2E+Rationale%2Daugmented+ensembles+in+language+models%2E)
57. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoe… [find](https://scholar.google.com/scholar?q=Thomas+Wolf%2C+Lysandre+Debut%2C+Victor+Sanh%2C+Julien+Chaumond%2C+Clement+Delangue%2C+Anthony+Moi%2C+Pierric+Cistac%2C+Tim+Rault%2C+R%C3%A9mi+Louf%2C+Morgan+Funtowicz%2C+Joe+Davison%2C+Sam+Shleifer%2C+Patrick+von+Platen%2C+Clara+M) [our card](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/)
58. Hua Wu and Haifeng Wang. 2007. Pivot language approach for phrase-based statistical machine translation. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 856–863, Prague, Czech Republic. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Hua+Wu+and+Haifeng+Wang%2E+2007%2E+Pivot+language+approach+for+phrase%2Dbased+statistical+machine+translation%2E+In+Proceedings+of+the+45th+Annual+Meeting+of+the+Association+of+Computational+Linguistics%2C+page) [our card](https://md.research.elamur.ai/free/en/papers/wu-2007-pivot-language-approach-phrase-based/)
59. Canwen Xu, Daya Guo, Nan Duan, and Julian McAuley. 2023a. Baize: An open-source chat model with parameter-efficient tuning on self-chat data. ArXiv preprint, abs/2304.01196. [find](https://scholar.google.com/scholar?q=Canwen+Xu%2C+Daya+Guo%2C+Nan+Duan%2C+and+Julian+McAuley%2E+2023a%2E+Baize%3A+An+open%2Dsource+chat+model+with+parameter%2Defficient+tuning+on+self%2Dchat+data%2E+ArXiv+preprint%2C+abs%2F2304%2E01196%2E) [our card](https://md.research.elamur.ai/free/en/papers/xu-2023-baize-open-source-chat-model/)
60. Haoran Xu, Young Jin Kim, Amr Sharaf, and Hany Hassan Awadalla. 2023b. A paradigm shift in machine translation: Boosting translation performance of large language models. [find](https://scholar.google.com/scholar?q=Haoran+Xu%2C+Young+Jin+Kim%2C+Amr+Sharaf%2C+and+Hany+Hassan+Awadalla%2E+2023b%2E+A+paradigm+shift+in+machine+translation%3A+Boosting+translation+performance+of+large+language+models%2E)
61. Yangyifan Xu, Jinliang Lu, and Jiajun Zhang. 2024. Bridging the gap between different vocabularies for LLM ensemble. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 7140–7… [find](https://scholar.google.com/scholar?q=Yangyifan+Xu%2C+Jinliang+Lu%2C+and+Jiajun+Zhang%2E+2024%2E+Bridging+the+gap+between+different+vocabularies+for+LLM+ensemble%2E+In+Proceedings+of+the+2024+Conference+of+the+North+American+Chapter+of+the+Associat)
62. Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. mT5: A massively multilingual pre-trained text-to-text transformer. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Ling… [find](https://scholar.google.com/scholar?q=Linting+Xue%2C+Noah+Constant%2C+Adam+Roberts%2C+Mihir+Kale%2C+Rami+Al%2DRfou%2C+Aditya+Siddhant%2C+Aditya+Barua%2C+and+Colin+Raffel%2E+2021%2E+mT5%3A+A+massively+multilingual+pre%2Dtrained+text%2Dto%2Dtext+transformer%2E+In+Procee)
63. Zhangyue Yin, Qiushi Sun, Cheng Chang, Qipeng Guo, Junqi Dai, Xuanjing Huang, and Xipeng Qiu. 2023. Exchange-of-thought: Enhancing large language model capabilities through cross-model communication. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 151… [find](https://scholar.google.com/scholar?q=Zhangyue+Yin%2C+Qiushi+Sun%2C+Cheng+Chang%2C+Qipeng+Guo%2C+Junqi+Dai%2C+Xuanjing+Huang%2C+and+Xipeng+Qiu%2E+2023%2E+Exchange%2Dof%2Dthought%3A+Enhancing+large+language+model+capabilities+through+cross%2Dmodel+communication%2E+)
64. Meng Zhang, Liangyou Li, and Qun Liu. 2022. Triangular transfer: Freezing the pivot for triangular machine translation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 644–650, Dublin, Ireland. Association for Computational L… [find](https://scholar.google.com/scholar?q=Meng+Zhang%2C+Liangyou+Li%2C+and+Qun+Liu%2E+2022%2E+Triangular+transfer%3A+Freezing+the+pivot+for+triangular+machine+translation%2E+In+Proceedings+of+the+60th+Annual+Meeting+of+the+Association+for+Computational+L)
65. Barret Zoph, Deniz Yuret, Jonathan May, and Kevin Knight. 2016. Transfer learning for low-resource neural machine translation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1568–1575, Austin, Texas. Association for Computational Linguistics. 9. Lang… [find](https://scholar.google.com/scholar?q=Barret+Zoph%2C+Deniz+Yuret%2C+Jonathan+May%2C+and+Kevin+Knight%2E+2016%2E+Transfer+learning+for+low%2Dresource+neural+machine+translation%2E+In+Proceedings+of+the+2016+Conference+on+Empirical+Methods+in+Natural+Lan)
66. Reimers, Nils and Gurevych, Iryna. 2020. Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Reimers%2C+Nils+and+Gurevych%2C+Iryna%2E+2020%2E+Making+Monolingual+Sentence+Embeddings+Multilingual+using+Knowledge+Distillation%2E+Association+for+Computational+Linguistics%2E) [our card](https://md.research.elamur.ai/free/en/papers/reimers-2020-making-monolingual-sentence-embeddings-multilingual/)
67. Holger Schwenk and Vishrav Chaudhary and Shuo Sun and Hongyu Gong and Francisco Guzmán. 2019. WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipedia. [find](https://scholar.google.com/scholar?q=Holger+Schwenk+and+Vishrav+Chaudhary+and+Shuo+Sun+and+Hongyu+Gong+and+Francisco+Guzm%C3%A1n%2E+2019%2E+WikiMatrix%3A+Mining+135M+Parallel+Sentences+in+1620+Language+Pairs+from+Wikipedia%2E) [our card](https://md.research.elamur.ai/free/en/papers/schwenk-2019-wikimatrix-mining-135m-parallel-sentences/)
68. Jörg Tiedemann. 2012. Parallel Data, Tools and Interfaces in OPUS. European Language Resources Association (ELRA). 10. Appendices [find](https://scholar.google.com/scholar?q=J%C3%B6rg+Tiedemann%2E+2012%2E+Parallel+Data%2C+Tools+and+Interfaces+in+OPUS%2E+European+Language+Resources+Association+%28ELRA%29%2E+10%2E+Appendices)