---
title: "Quality or Quantity? On Data Scale and Diversity in Adapting"
description: "Large language models translate well between high-resource languages, but when the target is a… 58 references, cited by 13 papers here."
authors: ["Vivek Iyer; Bhavitvya Malik; Pavel Stepachev; Pinzhen Chen; Barry Haddow;", "Alexandra Birch"]
year: 2024
venue: "arXiv:2408.12780"
source: "https://arxiv.org/abs/2408.12780"
doi: "10.48550/arxiv.2408.12780"
arxiv: "2408.12780"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/iyer-2024-quality-quantity-data-scale-diversity/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/iyer-2024-quality-quantity-data-scale-diversity/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/8/) / Quality or Quantity? On Data Scale and Diversity in Adapting Large Language Models for Low-Resource Translation

# Quality or Quantity? On Data Scale and Diversity in Adapting Large Language Models for Low-Resource Translation

**Authors:** Vivek Iyer; Bhavitvya Malik; Pavel Stepachev; Pinzhen Chen; Barry Haddow; and [Alexandra Birch](https://md.research.elamur.ai/free/en/authors/alexandra-birch/) · **Year:** 2024 · **Venue:** [arXiv:2408.12780](https://md.research.elamur.ai/free/en/venues/wmt/) ([WMT 2024](https://md.research.elamur.ai/free/en/venues/wmt/2024/)) · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2408.12780) · [PDF](https://arxiv.org/pdf/2408.12780) · [DOI 10.48550/arxiv.2408.12780](https://doi.org/10.48550/arxiv.2408.12780) · [arXiv 2408.12780](https://arxiv.org/abs/2408.12780) · [OpenAlex](https://openalex.org/W4404781295)

**Citations:** 4 citations (OpenAlex, 2026-10-14) · [13](https://md.research.elamur.ai/free/en/papers/iyer-2024-quality-quantity-data-scale-diversity/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Large language models translate well between high-resource languages, but when the target is a low-resource language (LRL) they still fall well short of dedicated NMT systems. The authors ask what it takes to close this gap. They adapt the two-stage ALMA recipe, continued pre-training (CPT) followed by supervised fine-tuning (SFT), to two groups of languages: Indigenous American languages, which use Latin script, and North-East Indian languages, some of which use other scripts. The focus is on translation out of English or Spanish into these languages. During CPT they compare several ways of combining monolingual text with bitext. During SFT they vary how diverse the training data is in tasks, languages and instruction datasets. Their findings run against recent results for high-resource LLM-MT. Large amounts of parallel data matter a great deal in both stages, and adding diversity tends to interfere with translation rather than help it. The work suggests that bitext remains a key resource for extending LLM translation to under-served languages.

## Key points
- The experiments cover 11 Indigenous American and 4 North-East Indian languages. Three base models are tested (Gemma 2B, Mistral 7B, Llama 3 8B), and the trends are largely consistent across them.
- CPT uses LoRA adapters of rank 8 on the query and value matrices, and the input and output embeddings are also tuned. The setups include 1M–13M parallel sentences, roughly 50M–750M tokens.
- Three CPT data mixes are compared:
  - "All Mono" uses only the target side of the bitext.
  - "Mono + parallel (concat)" joins source and target with language codes.
  - "Mono + parallel (separate)" keeps the two sides as independent sentences.
- Mixing general instruction data into SFT (Alpaca, Aya, XQA, other MT directions) consistently lowers chrF++. Native-language instruction data for Assamese and Meitei hurts the most. Focused multilingual MT fine-tuning on about 500K spa-X examples, trained for more epochs, works best.
- The final recipe improves average chrF++ by 16.5 points over few-shot prompting. The largest gains are for 8 American languages that have only about 10K–50K parallel sentences each. A stated limitation is that evaluation relies on string-based metrics, mainly chrF++.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Edinburgh](https://md.research.elamur.ai/free/en/universities/university-of-edinburgh/) — University, Edinburgh, United Kingdom · 359 works · 77,352 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🌱 Low-resource languages](https://md.research.elamur.ai/free/en/topics/low-resource-languages/) · 267 works

## Cite

Cite as: Vivek Iyer; Bhavitvya Malik; Pavel Stepachev; Pinzhen Chen; Barry Haddow; and Alexandra Birch. 2024. Quality or Quantity? On Data Scale and Diversity in Adapting Large Language Models for Low-Resource Translation. arXiv:2408.12780.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{haddow2024quality,
  title = {{Quality or Quantity? On Data Scale and Diversity in Adapting Large Language Models for Low-Resource Translation}},
  author = {Vivek Iyer; Bhavitvya Malik; Pavel Stepachev; Pinzhen Chen; Barry Haddow; and Alexandra Birch},
  year = {2024},
  doi = {10.48550/arxiv.2408.12780},
  eprint = {2408.12780},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2408.12780}
}
```

- [← previous Orion-14B: Open-source Multilingual Large Language Models](https://md.research.elamur.ai/free/en/papers/chen-2024-orion-14b-open-source/)
- [next → Weak-to-Strong Search: Align Large Language Models via Searching over Small Language Models](https://md.research.elamur.ai/free/en/papers/zhou-2024-weak-strong-search-align-large/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/8/)

## References

58 references, 25 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Duarte M Alves, José Pombal, Nuno M Guerreiro, Pedro H Martins, João Alves, Amin Farajian, Ben Peters, Ricardo Rei, Patrick Fernandes, Sweta Agrawal, et al. 2024. [Tower: An open multilingual large language model for translation-related tasks](https://md.research.elamur.ai/free/en/papers/alves-2024-tower-open-multilingual-large-language/). arXiv preprint arXiv:2402.17733.
2. Marta R Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, et al. 2022. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672. [find](https://scholar.google.com/scholar?q=Marta+R+Costa%2Djuss%C3%A0%2C+James+Cross%2C+Onur+%C3%87elebi%2C+Maha+Elbayad%2C+Kenneth+Heafield%2C+Kevin+Heffernan%2C+Elahe+Kalbassi%2C+Janice+Lam%2C+Daniel+Licht%2C+Jean+Maillard%2C+et+al%2E+2022%2E+No+language+left+behind%3A+Scaling+h)
3. Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. 2023. [Palm 2 technical report](https://md.research.elamur.ai/free/en/papers/anil-2023-palm-2-technical-report/). arXiv preprint arXiv:2305.10403.
4. Ona De Gibert, Raúl Vázquez, Mikko Aulamo, Yves Scherrer, Sami Virpioja, and Jörg Tiedemann. 2023. Four approaches to low-resource multilingual NMT: The Helsinki submission to the AmericasNLP 2023 shared task. In Proceedings of the Workshop on Natural Language Processing for Indigenous Languages of… [find](https://scholar.google.com/scholar?q=Ona+De+Gibert%2C+Ra%C3%BAl+V%C3%A1zquez%2C+Mikko+Aulamo%2C+Yves+Scherrer%2C+Sami+Virpioja%2C+and+J%C3%B6rg+Tiedemann%2E+2023%2E+Four+approaches+to+low%2Dresource+multilingual+NMT%3A+The+Helsinki+submission+to+the+AmericasNLP+2023+sha)
5. Naveen Arivazhagan, Ankur Bapna, Orhan Firat, Dmitry Lepikhin, Melvin Johnson, Maxim Krikun, Mia Xu Chen, Yuan Cao, George Foster, Colin Cherry, et al. 2019. [Massively multilingual neural machine translation in the wild: Findings and challenges](https://md.research.elamur.ai/free/en/papers/arivazhagan-2019-massively-multilingual-neural-machine-translation/). arXiv preprint arXiv:1907.05019.
6. Joseph Attieh, Zachary Hopton, Yves Scherrer, and Tanja Samardzic. 2024. System description of the nordicsalps submission to the americasnlp 2024 machine translation shared task. In Proceedings of the Dan DeGenaro and Tom Lupicki. 2024. Experiments in mamba sequence modeling and nllb-200 fine-tuning… [find](https://scholar.google.com/scholar?q=Joseph+Attieh%2C+Zachary+Hopton%2C+Yves+Scherrer%2C+and+Tanja+Samardzic%2E+2024%2E+System+description+of+the+nordicsalps+submission+to+the+americasnlp+2024+machine+translation+shared+task%2E+In+Proceedings+of+the)
7. Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. [LoRA: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). In International Conference on Learning Representations.
8. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783.
9. Adam Ibrahim, Benjamin Thérien, Kshitij Gupta, Mats Leon Richter, Quentin Gregory Anthony, Eugene Belilovsky, Timothée Lesort, and Irina Rish. 2024. Simple and scalable strategies to continually pre-train large language models. Transactions on Machine Learning Research. [find](https://scholar.google.com/scholar?q=Adam+Ibrahim%2C+Benjamin+Th%C3%A9rien%2C+Kshitij+Gupta%2C+Mats+Leon+Richter%2C+Quentin+Gregory+Anthony%2C+Eugene+Belilovsky%2C+Timoth%C3%A9e+Lesort%2C+and+Irina+Rish%2E+2024%2E+Simple+and+scalable+strategies+to+continually+pre%2Dt)
10. Abteen Ebrahimi, Ona de Gibert, Raul Vazquez, Rolando Coto-Solano, Pavel Denisov, Robert Pugh, Manuel Mager, Arturo Oncevay, Luis Chiruzzo, Katharina von der Wense, and Shruti Rijhwani. 2024. Findings of the AmericasNLP 2024 shared task on machine translation into indigenous languages. In Proceeding… [find](https://scholar.google.com/scholar?q=Abteen+Ebrahimi%2C+Ona+de+Gibert%2C+Raul+Vazquez%2C+Rolando+Coto%2DSolano%2C+Pavel+Denisov%2C+Robert+Pugh%2C+Manuel+Mager%2C+Arturo+Oncevay%2C+Luis+Chiruzzo%2C+Katharina+von+der+Wense%2C+and+Shruti+Rijhwani%2E+2024%2E+Findings)
11. Ayyoob ImaniGooghari, Peiqin Lin, Amir Hossein Kargaran, Silvia Severini, Masoud Jalili Sabet, Nora Kassner, Chunlan Ma, Helmut Schmid, André Martins, François Yvon, and Hinrich Schütze. 2023. Glot500: Scaling multilingual corpora and language models to 500 languages. In Proceedings of the 61st Annu… [find](https://scholar.google.com/scholar?q=Ayyoob+ImaniGooghari%2C+Peiqin+Lin%2C+Amir+Hossein+Kargaran%2C+Silvia+Severini%2C+Masoud+Jalili+Sabet%2C+Nora+Kassner%2C+Chunlan+Ma%2C+Helmut+Schmid%2C+Andr%C3%A9+Martins%2C+Fran%C3%A7ois+Yvon%2C+and+Hinrich+Sch%C3%BCtze%2E+2023%2E+Glot500)
12. Kazuki Fujii, Taishi Nakamura, Mengsay Loem, Hiroki Iida, Masanari Ohi, Kakeru Hattori, Hirai Shota, Sakae Mizuki, Rio Yokota, and Naoaki Okazaki. 2024. [Continual pre-training for cross-lingual llm adaptation: Enhancing japanese language capabilities](https://md.research.elamur.ai/free/en/papers/fujii-2024-continual-pre-training-cross-lingual/). arXiv preprint arXiv:2404.17790.
13. Vivek Iyer, Pinzhen Chen, and Alexandra Birch. 2023. [Towards effective disambiguation for machine translation with large language models](https://md.research.elamur.ai/free/en/papers/iyer-2023-effective-disambiguation-machine-translation-large/). In Proceedings of the Eighth Conference on Machine Translation, pages 482–495, Singapore. Association for Computational Linguistics.
14. Xavier Garcia, Yamini Bansal, Colin Cherry, George Foster, Maxim Krikun, Melvin Johnson, and Orhan Firat. 2023. The unreasonable effectiveness of few-shot learning for machine translation. In International Conference on Machine Learning, pages 10867–10878. PMLR. [find](https://scholar.google.com/scholar?q=Xavier+Garcia%2C+Yamini+Bansal%2C+Colin+Cherry%2C+George+Foster%2C+Maxim+Krikun%2C+Melvin+Johnson%2C+and+Orhan+Firat%2E+2023%2E+The+unreasonable+effectiveness+of+few%2Dshot+learning+for+machine+translation%2E+In+Internat)
15. Vivek Iyer, Bhavitvya Malik, Wenhao Zhu, Pavel Stepachev, Pinzhen Chen, Barry Haddow, and Alexandra Birch. 2024. Exploring very low-resource translation with LLMs: The University of Edinburgh’s submission to AmericasNLP 2024 translation task. In Proceedings of the 4th Workshop on Natural Language Pr… [find](https://scholar.google.com/scholar?q=Vivek+Iyer%2C+Bhavitvya+Malik%2C+Wenhao+Zhu%2C+Pavel+Stepachev%2C+Pinzhen+Chen%2C+Barry+Haddow%2C+and+Alexandra+Birch%2E+2024%2E+Exploring+very+low%2Dresource+translation+with+LLMs%3A+The+University+of+Edinburgh%E2%80%99s+submis)
16. Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, et al. 2024. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295. [find](https://scholar.google.com/scholar?q=Gemma+Team%2C+Thomas+Mesnard%2C+Cassidy+Hardin%2C+Robert+Dadashi%2C+Surya+Bhupatiraju%2C+Shreya+Pathak%2C+Laurent+Sifre%2C+Morgane+Rivi%C3%A8re%2C+Mihir+Sanjay+Kale%2C+Juliette+Love%2C+et+al%2E+2024%2E+Gemma%3A+Open+models+based+on)
17. Jiaxin Guo, Hao Yang, Zongyao Li, Daimeng Wei, Hengchao Shang, and Xiaoyu Chen. 2024. A novel paradigm boosting translation capabilities of large language models. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 639–649, Mexico City, Mexico. Association for Computation… [find](https://scholar.google.com/scholar?q=Jiaxin+Guo%2C+Hao+Yang%2C+Zongyao+Li%2C+Daimeng+Wei%2C+Hengchao+Shang%2C+and+Xiaoyu+Chen%2E+2024%2E+A+novel+paradigm+boosting+translation+capabilities+of+large+language+models%2E+In+Findings+of+the+Association+for+Co)
18. Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825. [find](https://scholar.google.com/scholar?q=Albert+Q+Jiang%2C+Alexandre+Sablayrolles%2C+Arthur+Mensch%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guillaume+Lample%2C+Lucile+Saulnier%2C+et+al%2E+2023%2E+Mist)
19. Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. 2024. Mixtral of experts. arXiv preprint arXiv:2401.04088. [find](https://scholar.google.com/scholar?q=Albert+Q+Jiang%2C+Alexandre+Sablayrolles%2C+Antoine+Roux%2C+Arthur+Mensch%2C+Blanche+Savary%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Emma+Bou+Hanna%2C+Florian+Bressand%2C+et+al%2E+2024%2E+Mixtral+o)
20. Kevin Heffernan, Onur Çelebi, and Holger Schwenk. 2022. [Bitext mining using distilled sentence representations for low-resource languages](https://md.research.elamur.ai/free/en/papers/heffernan-2022-bitext-mining-distilled-sentence-representations/). In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 2101–2112, Abu Dhabi, United Arab Emirates. Association for Computational Linguis…
21. Tannon Kew, Florian Schottmann, and Rico Sennrich. 2023. Turning english-centric llms into polyglots: How much multilinguality is needed? arXiv preprint arXiv:2312.12683. [find](https://scholar.google.com/scholar?q=Tannon+Kew%2C+Florian+Schottmann%2C+and+Rico+Sennrich%2E+2023%2E+Turning+english%2Dcentric+llms+into+polyglots%3A+How+much+multilinguality+is+needed%3F+arXiv+preprint+arXiv%3A2312%2E12683%2E)
22. Amr Hendy, Mohamed Abdelrehim, Amr Sharaf, Vikas Raunak, Mohamed Gabr, Hitokazu Matsushita, Young Jin Kim, Mohamed Afify, and Hany Hassan Awadalla. 2023. How good are gpt models at machine translation? a comprehensive evaluation. arXiv preprint arXiv:2302.09210. [find](https://scholar.google.com/scholar?q=Amr+Hendy%2C+Mohamed+Abdelrehim%2C+Amr+Sharaf%2C+Vikas+Raunak%2C+Mohamed+Gabr%2C+Hitokazu+Matsushita%2C+Young+Jin+Kim%2C+Mohamed+Afify%2C+and+Hany+Hassan+Awadalla%2E+2023%2E+How+good+are+gpt+models+at+machine+translation)
23. Mohammed Safi Ur Rahman Khan, Priyam Mehta, Ananth Sankar, Umashankar Kumaravelan, Sumanth Doddapaneni, Suriyaprasaad G, Varun Balan G, Sparsh Jain, Anoop Kunchukuttan, Pratyush Kumar, Raj Dabre, and Mitesh M. Khapra. 2024. [Indicllmsuite: A blueprint for creating pre-training and fine-tuning dataset…](https://md.research.elamur.ai/free/en/papers/khan-2024-indicllmsuite-blueprint-creating-pre-training/)
24. M Saiful Bari, Sheng Shen, Zheng Xin Yong, Hailey Schoelkopf, Xiangru Tang, Dragomir Radev, Alham Fikri Aji, Khalid Almubarak, Samuel Albanie, Zaid Alyafeai, Albert Webson, Edward Raff, and Colin Raffel. 2023. [Crosslingual generalization through multitask finetuning](https://md.research.elamur.ai/free/en/papers/muennighoff-2022-crosslingual-generalization-through-multitask-finetuning/). In Proceedings of the 61st Annua…
25. Tom Kocmi, Eleftherios Avramidis, Rachel Bawden, Ondrej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Markus Freitag, Thamme Gowda, Roman Grundkiewicz, Barry Haddow, Marzena Karpinska, Philipp Koehn, Benjamin Marie, Kenton Murray, Masaaki Nagata, Martin Popel, Maja Popovic, Mariya Shmat… — [arXiv:2407.19884](https://arxiv.org/abs/2407.19884) [find](https://scholar.google.com/scholar?q=Tom+Kocmi%2C+Eleftherios+Avramidis%2C+Rachel+Bawden%2C+Ondrej+Bojar%2C+Anton+Dvorkovich%2C+Christian+Federmann%2C+Mark+Fishel%2C+Markus+Freitag%2C+Thamme+Gowda%2C+Roman+Grundkiewicz%2C+Barry+Haddow%2C+Marzena+Karpinska%2C+Ph)
26. Santanu Pal, Partha Pakray, Sahinur Rahman Laskar, Lenin Laitonjam, Vanlalmuansangi Khenglawt, Sunita Warjri, Pankaj Kundan Dadure, and Sandeep Kumar Dash. 2023. [Findings of the wmt 2023 shared task on low-resource indic language translation](https://md.research.elamur.ai/free/en/papers/pal-2023-findings-wmt-2023-shared-task/). In Proceedings of the Eighth Conference on Machine Transl…
27. Philipp Koehn. 2004. [Statistical significance tests for machine translation evaluation](https://md.research.elamur.ai/free/en/papers/koehn-2004-statistical-significance-tests-machine-translation/). In Proceedings of the 2004 conference on empirical methods in natural language processing, pages 388–395.
28. Maja Popović. 2017. [chrf++: words helping character n-grams](https://md.research.elamur.ai/free/en/papers/popovic-2017-chrf-words-helping-character-n/). In Proceedings of the second conference on machine translation, pages 612–618.
29. Sneha Kudugunta, Isaac Caswell, Biao Zhang, Xavier Garcia, Derrick Xin, Aditya Kusupati, Romi Stella, Ankur Bapna, and Orhan Firat. 2024. [Madlad-400: A multilingual and document-level large audited dataset](https://md.research.elamur.ai/free/en/papers/kudugunta-2023-madlad-400-multilingual-document-level/). Advances in Neural Information Processing Systems, 36.
30. Matt Post. 2018. [A call for clarity in reporting BLEU scores](https://md.research.elamur.ai/free/en/papers/post-2018-call-clarity-reporting-bleu-scores/). In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186– 191, Belgium, Brussels. Association for Computational Linguistics.
31. Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V Le, Barret Zoph, Jason Wei, and Adam Roberts. 2023. [The flan collection: Designing data and methods for effective instruction tuning](https://md.research.elamur.ai/free/en/papers/longpre-2023-flan-collection-designing-data-methods/). In Proceedings of the 40th International Conference on Machine Learning, vol…
32. Leonardo Ranaldi and Giulia Pucci. 2023. Does the English matter? elicit cross-lingual abilities of large language models. In Proceedings of the 3rd Workshop on Multi-lingual Representation Learning (MRL), pages 173–183, Singapore. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Leonardo+Ranaldi+and+Giulia+Pucci%2E+2023%2E+Does+the+English+matter%3F+elicit+cross%2Dlingual+abilities+of+large+language+models%2E+In+Proceedings+of+the+3rd+Workshop+on+Multi%2Dlingual+Representation+Learning+%28)
33. Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. 2020. Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD ’20, page 3505–3506,… [find](https://scholar.google.com/scholar?q=Jeff+Rasley%2C+Samyam+Rajbhandari%2C+Olatunji+Ruwase%2C+and+Yuxiong+He%2E+2020%2E+Deepspeed%3A+System+optimizations+enable+training+deep+learning+models+with+over+100+billion+parameters%2E+In+Proceedings+of+the+26t)
34. Hongyuan Lu, Haoyang Huang, Dongdong Zhang, Haoran Yang, Wai Lam, and Furu Wei. 2023. [Chain-of-dictionary prompting elicits translation in large language models](https://md.research.elamur.ai/free/en/papers/lu-2024-chain-dictionary-prompting-elicits-translation/). arXiv preprint arXiv:2305.06575.
35. Yinquan Lu, Wenhao Zhu, Lei Li, Yu Qiao, and Fei Yuan. 2024. Llamax: Scaling linguistic horizons of llm by enhancing translation capabilities beyond 100 languages. arXiv preprint arXiv:2407.05975. [find](https://scholar.google.com/scholar?q=Yinquan+Lu%2C+Wenhao+Zhu%2C+Lei+Li%2C+Yu+Qiao%2C+and+Fei+Yuan%2E+2024%2E+Llamax%3A+Scaling+linguistic+horizons+of+llm+by+enhancing+translation+capabilities+beyond+100+languages%2E+arXiv+preprint+arXiv%3A2407%2E05975%2E)
36. Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy Lillicrap, Jean-baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat, Julian Schrittwieser, et al. 2024. [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://md.research.elamur.ai/free/en/papers/gemini-team-2024-gemini-1-5-unlocking-multimodal/). arXiv preprint arXiv:2403.…
37. Zhuoyuan Mao and Yen Yu. 2024. Tuning LLMs with contrastive alignment instructions for machine translation in unseen, low-resource languages. In Proceedings of the The Seventh Workshop on Technologies for Machine Translation of Low-Resource Languages (LoResMT 2024), pages 1–25, Bangkok, Thailand. As… [find](https://scholar.google.com/scholar?q=Zhuoyuan+Mao+and+Yen+Yu%2E+2024%2E+Tuning+LLMs+with+contrastive+alignment+instructions+for+machine+translation+in+unseen%2C+low%2Dresource+languages%2E+In+Proceedings+of+the+The+Seventh+Workshop+on+Technologies)
38. Nathaniel Robinson, Perez Ogayo, David R. Mortensen, and Graham Neubig. 2023. [ChatGPT MT: Competitive for high- (but not low-) resource languages](https://md.research.elamur.ai/free/en/papers/robinson-2023-chatgpt-mt-competitive-high-but/). In Proceedings of the Eighth Conference on Machine Translation, pages 392–418, Singapore. Association for Computational Linguistics.
39. Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. 2022. Cross-task generalization via natural language crowdsourcing instructions. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3470–3487, Dublin, Irelan… [find](https://scholar.google.com/scholar?q=Swaroop+Mishra%2C+Daniel+Khashabi%2C+Chitta+Baral%2C+and+Hannaneh+Hajishirzi%2E+2022%2E+Cross%2Dtask+generalization+via+natural+language+crowdsourcing+instructions%2E+In+Proceedings+of+the+60th+Annual+Meeting+of+th)
40. Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, Debajyoti Niklas Muennighoff,… — [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://md.research.elamur.ai/free/en/papers/sanh-2021-multitask-prompted-training-enables-zero/)
41. Biao Zhang, Barry Haddow, and Alexandra Birch. 2023a. [Prompting large language model for machine translation: A case study](https://md.research.elamur.ai/free/en/papers/zhang-2023-prompting-large-language-model-machine/). In International Conference on Machine Learning, pages 41092–41110. PMLR.
42. Biao Zhang, Zhongtao Liu, Colin Cherry, and Orhan Firat. 2024a. When scaling meets LLM finetuning: The effect of data, model and finetuning method. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Biao+Zhang%2C+Zhongtao+Liu%2C+Colin+Cherry%2C+and+Orhan+Firat%2E+2024a%2E+When+scaling+meets+LLM+finetuning%3A+The+effect+of+data%2C+model+and+finetuning+method%2E+In+The+Twelfth+International+Conference+on+Learning+)
43. Uri Shaham, Jonathan Herzig, Roee Aharoni, Idan Szpektor, Reut Tsarfaty, and Matan Eyal. 2024. [Multilingual instruction tuning with just a pinch of multilinguality](https://md.research.elamur.ai/free/en/papers/shaham-2024-multilingual-instruction-tuning-just-pinch/). In Findings of the Association for Computational Linguistics ACL 2024, pages 2304–2317, Bangkok, Thailand and virtual meeting. Associat…
44. Kexun Zhang, Yee Choi, Zhenqiao Song, Taiqi He, William Yang Wang, and Lei Li. 2024b. Hire a linguist!: Learning endangered languages in LLMs with in-context linguistic descriptions. In Findings of the Association for Computational Linguistics ACL 2024, pages 15654–15669, Bangkok, Thailand and virtu… [find](https://scholar.google.com/scholar?q=Kexun+Zhang%2C+Yee+Choi%2C+Zhenqiao+Song%2C+Taiqi+He%2C+William+Yang+Wang%2C+and+Lei+Li%2E+2024b%2E+Hire+a+linguist%21%3A+Learning+endangered+languages+in+LLMs+with+in%2Dcontext+linguistic+descriptions%2E+In+Findings+of+th)
45. Shivalika Singh, Freddie Vargus, Daniel Dsouza, Börje F Karlsson, Abinaya Mahendiran, Wei-Yin Ko, Herumb Shandilya, Jay Patel, Deividas Mataciunas, Laura OMahony, et al. 2024. [Aya dataset: An open-access collection for multilingual instruction tuning](https://md.research.elamur.ai/free/en/papers/singh-2024-aya-dataset-open-access-collection/). arXiv preprint arXiv:2402.06619.
46. Shaolei Zhang, Qingkai Fang, Zhuocheng Zhang, Zhengrui Ma, Yan Zhou, Langlin Huang, Mengyu Bu, Shangtong Gui, Yunji Chen, Xilin Chen, et al. 2023b. Bayling: Bridging cross-lingual alignment and instruction following through interactive translation for large language models. arXiv preprint arXiv:2306… [find](https://scholar.google.com/scholar?q=Shaolei+Zhang%2C+Qingkai+Fang%2C+Zhuocheng+Zhang%2C+Zhengrui+Ma%2C+Yan+Zhou%2C+Langlin+Huang%2C+Mengyu+Bu%2C+Shangtong+Gui%2C+Yunji+Chen%2C+Xilin+Chen%2C+et+al%2E+2023b%2E+Bayling%3A+Bridging+cross%2Dlingual+alignment+and+instru)
47. Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction-following llama model. [https://github.com/tatsu-lab/stanford_alpaca](https://github.com/tatsu-lab/stanford_alpaca). [↗](https://github.com/tatsu-lab/stanford_alpaca)
48. Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. 2024. [Llamafactory: Unified efficient fine-tuning of 100+ language models](https://md.research.elamur.ai/free/en/papers/zheng-2024-llamafactory-unified-efficient-fine-tuning/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations),…
49. Jörg Tiedemann. 2009. News from opus-a collection of multilingual parallel corpora with tools and interfaces. In Recent advances in natural language processing, volume 5, pages 237–248. [find](https://scholar.google.com/scholar?q=J%C3%B6rg+Tiedemann%2E+2009%2E+News+from+opus%2Da+collection+of+multilingual+parallel+corpora+with+tools+and+interfaces%2E+In+Recent+advances+in+natural+language+processing%2C+volume+5%2C+pages+237%E2%80%93248%2E)
50. Jörg Tiedemann. 2012. Parallel data, tools and interfaces in opus. In Lrec, volume 2012, pages 2214– 2218. Citeseer. [find](https://scholar.google.com/scholar?q=J%C3%B6rg+Tiedemann%2E+2012%2E+Parallel+data%2C+tools+and+interfaces+in+opus%2E+In+Lrec%2C+volume+2012%2C+pages+2214%E2%80%93+2218%2E+Citeseer%2E)
51. Chunting Zhou, Pengfei Liu, Puxin Xu, Srinivasan Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, LILI YU, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, and Omer Levy. 2023. Lima: Less is more for alignment. In Advances in Neural Information Processing Systems, volume 36, pages… [find](https://scholar.google.com/scholar?q=Chunting+Zhou%2C+Pengfei+Liu%2C+Puxin+Xu%2C+Srinivasan+Iyer%2C+Jiao+Sun%2C+Yuning+Mao%2C+Xuezhe+Ma%2C+Avia+Efrat%2C+Ping+Yu%2C+LILI+YU%2C+Susan+Zhang%2C+Gargi+Ghosh%2C+Mike+Lewis%2C+Luke+Zettlemoyer%2C+and+Omer+Levy%2E+2023%2E+Lima%3A)
52. David Vilar, Markus Freitag, Colin Cherry, Jiaming Luo, Viresh Ratnakar, and George Foster. 2023. Prompting PaLM for translation: Assessing strategies and performance. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15406– 154… [find](https://scholar.google.com/scholar?q=David+Vilar%2C+Markus+Freitag%2C+Colin+Cherry%2C+Jiaming+Luo%2C+Viresh+Ratnakar%2C+and+George+Foster%2E+2023%2E+Prompting+PaLM+for+translation%3A+Assessing+strategies+and+performance%2E+In+Proceedings+of+the+61st+Annua)
53. Dawei Zhu, Pinzhen Chen, Miaoran Zhang, Barry Haddow, Xiaoyu Shen, and Dietrich Klakow. 2024a. Fine-tuning large language models to translate: Will a touch of noisy data in misaligned languages suffice? arXiv preprint arXiv:2404.14122. [find](https://scholar.google.com/scholar?q=Dawei+Zhu%2C+Pinzhen+Chen%2C+Miaoran+Zhang%2C+Barry+Haddow%2C+Xiaoyu+Shen%2C+and+Dietrich+Klakow%2E+2024a%2E+Fine%2Dtuning+large+language+models+to+translate%3A+Will+a+touch+of+noisy+data+in+misaligned+languages+suffic)
54. Xiangpeng Wei, Haoran Wei, Huan Lin, Tianhao Li, Pei Zhang, Xingzhang Ren, Mei Li, Yu Wan, Zhiwei Cao, Binbin Xie, et al. 2023. Polylm: An open source polyglot large language model. arXiv preprint arXiv:2307.06018. [find](https://scholar.google.com/scholar?q=Xiangpeng+Wei%2C+Haoran+Wei%2C+Huan+Lin%2C+Tianhao+Li%2C+Pei+Zhang%2C+Xingzhang+Ren%2C+Mei+Li%2C+Yu+Wan%2C+Zhiwei+Cao%2C+Binbin+Xie%2C+et+al%2E+2023%2E+Polylm%3A+An+open+source+polyglot+large+language+model%2E+arXiv+preprint+arX)
55. Wenhao Zhu, Hongyi Liu, Qingxiu Dong, Jingjing Xu, Shujian Huang, Lingpeng Kong, Jiajun Chen, and Lei Li. 2024b. [Multilingual machine translation with large language models: Empirical results and analysis](https://md.research.elamur.ai/free/en/papers/zhu-2024-multilingual-machine-translation-large-language/). In Findings of the Association for Computational Linguistics: NAACL 2024, pages 2765–2781, Mex…
56. Haoran Xu, Young Jin Kim, Amr Sharaf, and Hany Hassan Awadalla. 2024. A paradigm shift in machine translation: Boosting translation performance of large language models. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Haoran+Xu%2C+Young+Jin+Kim%2C+Amr+Sharaf%2C+and+Hany+Hassan+Awadalla%2E+2024%2E+A+paradigm+shift+in+machine+translation%3A+Boosting+translation+performance+of+large+language+models%2E+In+The+Twelfth+International+C)
57. Wenhao Zhu, Yunzhe Lv, Qingxiu Dong, Fei Yuan, Jingjing Xu, Shujian Huang, Lingpeng Kong, Jiajun Chen, and Lei Li. 2023. Extrapolating large language models to non-english by aligning languages. arXiv preprint arXiv:2308.04948. [find](https://scholar.google.com/scholar?q=Wenhao+Zhu%2C+Yunzhe+Lv%2C+Qingxiu+Dong%2C+Fei+Yuan%2C+Jingjing+Xu%2C+Shujian+Huang%2C+Lingpeng+Kong%2C+Jiajun+Chen%2C+and+Lei+Li%2E+2023%2E+Extrapolating+large+language+models+to+non%2Denglish+by+aligning+languages%2E+arXiv)
58. Wen Yang, Chong Li, Jiajun Zhang, and Chengqing Zong. 2023. Bigtranslate: Augmenting large language models with multilingual translation capability over 100 languages. arXiv preprint arXiv:2305.18098. — [arXiv:2305.18098](https://arxiv.org/abs/2305.18098) [find](https://scholar.google.com/scholar?q=Wen+Yang%2C+Chong+Li%2C+Jiajun+Zhang%2C+and+Chengqing+Zong%2E+2023%2E+Bigtranslate%3A+Augmenting+large+language+models+with+multilingual+translation+capability+over+100+languages%2E+arXiv+preprint+arXiv%3A2305%2E18098%2E)

*Updated 30 October 2026*