---
title: "Is Translation All You Need? A Study on Solving Multilingual"
description: "The paper examines whether translating non-English input into English is consistently the best way to… 41 references, cited by 47 papers here."
authors: ["Chaoqun Liu", "Wenxuan Zhang", "Yiran Zhao", "Anh Tuan Luu", "Lidong Bing"]
year: 2025
venue: "Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)"
source: "https://aclanthology.org/2025.naacl-long.485/"
doi: "10.18653/v1/2025.naacl-long.485"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/liu-2025-is-translation-all-you-need/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/liu-2025-is-translation-all-you-need/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/2/) / Is Translation All You Need? A Study on Solving Multilingual Tasks with Large Language Models

# Is Translation All You Need? A Study on Solving Multilingual Tasks with Large Language Models

**Authors:** [Chaoqun Liu](https://md.research.elamur.ai/free/en/authors/chaoqun-liu/), [Wenxuan Zhang](https://md.research.elamur.ai/free/en/authors/wenxuan-zhang/), [Yiran Zhao](https://md.research.elamur.ai/free/en/authors/yiran-zhao/), [Anh Tuan Luu](https://md.research.elamur.ai/free/en/authors/anh-tuan-luu/) and [Lidong Bing](https://md.research.elamur.ai/free/en/authors/lidong-bing/) · **Year:** 2025 · **Venue:** [Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/naacl/) ([NAACL 2025](https://md.research.elamur.ai/free/en/venues/naacl/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.naacl-long.485/) · [PDF](https://aclanthology.org/2025.naacl-long.485.pdf) · [DOI 10.18653/v1/2025.naacl-long.485](https://doi.org/10.18653/v1/2025.naacl-long.485) · [OpenAlex](https://openalex.org/W4411120534)

**Citations:** 14 citations (OpenAlex, 2026-10-14) · [47](https://md.research.elamur.ai/free/en/papers/liu-2025-is-translation-all-you-need/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper examines whether translating non-English input into English is consistently the best way to get good multilingual results from LLMs. Earlier work found translation helpful, but it mostly tested standard NLP benchmarks and English-centric models. The authors compare seven zero-shot prompting strategies: native or English instructions, native or English chain-of-thought, XLT, and translate-test via Google Translate or NLLB. They run these on six benchmarks spanning 24 languages, mainly with ChatGPT and Llama-2-70B-Chat. They then extend the comparison to real user queries, culture-heavy M3Exam questions, and non-English-centric (Chinese-optimized) models.

On classic NLP tasks, machine-translating the input into English is a strong and often winning baseline, especially for low-resource languages. The advantage shrinks or reverses in other settings: tasks needing cultural or language-specific knowledge, advanced models, certain languages, and models not centred on English. There, prompting in the native language tends to work better.

The broader message is that current multilingual evaluation over-relies on English-centric assumptions. The authors argue for broader benchmarks and genuinely multilingual LLMs.

## Key points
- **Benchmarks:** MGSM, XCOPA, XNLI, PAWS-X, MKQA and XL-Sum, with up to 500 test items per language. Languages above a 1% Common Crawl share count as high-resource, and English itself is excluded.
- **ChatGPT averages:** TRANS-GOOGLE scored 61.4 on high-resource and 59.0 on low-resource languages. NATIVE-BASIC scored 50.8 and 40.8.
- **Llama-2-70B-Chat averages:** TRANS-GOOGLE scored 56.4 and 54.4, versus 42.5 and 29.5 for NATIVE-BASIC. On low-resource MGSM, accuracy rose from 5.6 to 50.0.
- **Layer-wise language analysis (CLD3) on Chinese prompts:** Qwen1.5-7B-Chat keeps its hidden representations largely in Chinese, while Llama-2-7B-Chat drifts into other languages. The authors suggest this native processing avoids information loss.
- **Limitations and code:** The study uses only two Chinese-optimized LLMs and the multiple-choice M3Exam for culture tests. Code is released at github.com/DAMO-NLP-SG/translation-all-you-need.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [National University of Singapore](https://md.research.elamur.ai/free/en/universities/national-university-of-singapore/) — University, Singapore, Singapore · 180 works · 34,304 citations
- [Nanyang Technological University](https://md.research.elamur.ai/free/en/universities/nanyang-technological-university/) — University, Singapore, Singapore · 156 works · 25,644 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (1): [🌍 Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) · 828 works

## Cite

Cite as: Chaoqun Liu, Wenxuan Zhang, Yiran Zhao, Anh Tuan Luu and Lidong Bing. 2025. Is Translation All You Need? A Study on Solving Multilingual Tasks with Large Language Models. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{liu2025is,
  title = {{Is Translation All You Need? A Study on Solving Multilingual Tasks with Large Language Models}},
  author = {Chaoqun Liu, Wenxuan Zhang, Yiran Zhao, Anh Tuan Luu and Lidong Bing},
  booktitle = {Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.naacl-long.485},
  url = {https://doi.org/10.18653/v1/2025.naacl-long.485}
}
```

- [← previous Seed1.5-Thinking: Advancing Superb Reasoning Models with Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/bytedance-seed-2025-seed1-5-thinking-advancing/)
- [next → Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Models](https://md.research.elamur.ai/free/en/papers/wang-2025-astute-rag-overcoming-imperfect-retrieval/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/2/)

## References

41 references, 26 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Kabir Ahuja, Harshita Diddee, Rishav Hada, Millicent Ochieng, Krithika Ramesh, Prachi Jain, Akshay Nambi, Tanuja Ganu, Sameer Segal, Mohamed Ahmed, Kalika Bali, and Sunayana Sitaram. 2023. [MEGA: Multilingual Evaluation of Generative AI](https://md.research.elamur.ai/free/en/papers/ahuja-2023-mega-multilingual-evaluation-generative-ai/). In Proceedings of the 2023 Conference on Empirical Methods in N…
2. Mikel Artetxe, Vedanuj Goswami, Shruti Bhosale, Angela Fan, and Luke Zettlemoyer. 2023. Revisiting Machine Translation for Cross-lingual Classification. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6489–6499, Singapore. Association for Computationa… [find](https://scholar.google.com/scholar?q=Mikel+Artetxe%2C+Vedanuj+Goswami%2C+Shruti+Bhosale%2C+Angela+Fan%2C+and+Luke+Zettlemoyer%2E+2023%2E+Revisiting+Machine+Translation+for+Cross%2Dlingual+Classification%2E+In+Proceedings+of+the+2023+Conference+on+Empiri)
3. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, P… — [Qwen Technical Report](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-technical-report/)
4. Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, Quyet V. Do, Yan Xu, and Pascale Fung. 2023. [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://md.research.elamur.ai/free/en/papers/bang-2023-multitask-multilingual-multimodal-evaluation-chatgpt/). ArXiv:2302.04023 […
5. Patrick Bareiß, Roman Klinger, and Jeremy Barnes. 2024. [English Prompts are Better for NLI-based Zero-Shot Emotion Classification than Target-Language Prompts](https://md.research.elamur.ai/free/en/papers/barei-ss-2024-english-prompts-better-nli-based/). In Companion Proceedings of the ACM Web Conference 2024, WWW ’24, page 1318–1326, New York, NY, USA. Association for Computing Machinery.
6. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… — [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
7. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emi… — [PaLM: Scaling Language Modeling with Pathways](https://md.research.elamur.ai/free/en/papers/chowdhery-2022-palm-scaling-language-modeling-pathways/)
8. Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. XNLI: Evaluating Cross-lingual Sentence Representations. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2475–2485, Brussels, Bel… [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Ruty+Rinott%2C+Guillaume+Lample%2C+Adina+Williams%2C+Samuel+Bowman%2C+Holger+Schwenk%2C+and+Veselin+Stoyanov%2E+2018%2E+XNLI%3A+Evaluating+Cross%2Dlingual+Sentence+Representations%2E+In+Proceedings+of+the)
9. Julen Etxaniz, Gorka Azkune, Aitor Soroa, Oier Lopez de Lacalle, and Mikel Artetxe. 2023. [Do Multilingual Language Models Think Better in English?](https://md.research.elamur.ai/free/en/papers/etxaniz-2024-do-multilingual-language-models-think/) ArXiv:2308.01223 [cs].
10. Yi Fung, Ruining Zhao, Jae Doo, Chenkai Sun, and Heng Ji. 2024. [Massively Multi-Cultural Knowledge Acquisition & LM Benchmarking](https://md.research.elamur.ai/free/en/papers/fung-2024-massively-multi-cultural-knowledge-acquisition/). ArXiv:2402.09369 [cs].
11. Tahmid Hasan, Abhik Bhattacharjee, Md. Saiful Islam, Kazi Mubasshir, Yuan-Fang Li, Yong-Bin Kang, M. Sohel Rahman, and Rifat Shahriyar. 2021. XL-Sum: Large-Scale Multilingual Abstractive Summarization for 44 Languages. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pa… [find](https://scholar.google.com/scholar?q=Tahmid+Hasan%2C+Abhik+Bhattacharjee%2C+Md%2E+Saiful+Islam%2C+Kazi+Mubasshir%2C+Yuan%2DFang+Li%2C+Yong%2DBin+Kang%2C+M%2E+Sohel+Rahman%2C+and+Rifat+Shahriyar%2E+2021%2E+XL%2DSum%3A+Large%2DScale+Multilingual+Abstractive+Summarization)
12. Haoyang Huang, Tianyi Tang, Dongdong Zhang, Wayne Xin Zhao, Ting Song, Yan Xia, and Furu Wei. 2023. [Not All Languages Are Created Equal in LLMs: Improving Multilingual Capability by Cross-Lingual-Thought Prompting](https://md.research.elamur.ai/free/en/papers/huang-2023-all-languages-created-equal-llms/). ArXiv:2305.07004 [cs].
13. Lianzhe Huang, Shuming Ma, Dongdong Zhang, Furu Wei, and Houfeng Wang. 2022. Zero-shot Cross-lingual Transfer of Prompt-based Tuning with a Unified Multilingual Prompt. ArXiv:2202.11451 [cs]. [find](https://scholar.google.com/scholar?q=Lianzhe+Huang%2C+Shuming+Ma%2C+Dongdong+Zhang%2C+Furu+Wei%2C+and+Houfeng+Wang%2E+2022%2E+Zero%2Dshot+Cross%2Dlingual+Transfer+of+Prompt%2Dbased+Tuning+with+a+Unified+Multilingual+Prompt%2E+ArXiv%3A2202%2E11451+%5Bcs%5D%2E)
14. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… [find](https://scholar.google.com/scholar?q=Albert+Q%2E+Jiang%2C+Alexandre+Sablayrolles%2C+Arthur+Mensch%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guillaume+Lample%2C+Lucile+Saulnier%2C+L%C3%A9lio+Renard+Lav)
15. Viet Dac Lai, Nghia Trung Ngo, Amir Pouran Ben Veyseh, Hieu Man, Franck Dernoncourt, Trung Bui, and Thien Huu Nguyen. 2023. [ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning](https://md.research.elamur.ai/free/en/papers/lai-2023-chatgpt-beyond-english-comprehensive-evaluation/). ArXiv:2304.05613 [cs].
16. Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Xingyu Dang, Chuang Gan, and Song Han. 2023. Awq: Activation-aware weight quantization for llm compression and acceleration. [find](https://scholar.google.com/scholar?q=Ji+Lin%2C+Jiaming+Tang%2C+Haotian+Tang%2C+Shang+Yang%2C+Xingyu+Dang%2C+Chuang+Gan%2C+and+Song+Han%2E+2023%2E+Awq%3A+Activation%2Daware+weight+quantization+for+llm+compression+and+acceleration%2E)
17. Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, Ramakanth Pasunuru, Sam Shleifer, Punit Singh Koura, Vishrav Chaudhary, Brian O’Horo, Jeff Wang, Luke Zettlemoyer, Zornitsa Kozareva, Mona Diab, Veselin Stoyano… — [Few-shot Learning with Multilingual Language Models](https://md.research.elamur.ai/free/en/papers/lin-2021-few-shot-learning-multilingual-language/)
18. Patrick Littell, David R Mortensen, Ke Lin, Katherine Kairis, Carlisle Turner, and Lori Levin. 2017. [Uriel and lang2vec: Representing languages as typological, geographical, and phylogenetic vectors](https://md.research.elamur.ai/free/en/papers/littell-2017-uriel-lang2vec-representing-languages-as/). In Proceedings of the 15th Conference of the European Chapter of the Association for Computational L…
19. Shayne Longpre, Yi Lu, and Joachim Daiber. 2021. [MKQA: A Linguistically Diverse Benchmark for Multilingual Open Domain Question Answering](https://md.research.elamur.ai/free/en/papers/chen-2020-open-domain-question-answering/). Transactions of the Association for Computational Linguistics, 9:1389–1406. Place: Cambridge, MA Publisher: MIT Press.
20. Nikita Moghe, Tom Sherborne, Mark Steedman, and Alexandra Birch. 2023. Extrinsic Evaluation of Machine Translation Metrics. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13060– 13078, Toronto, Canada. Association for Computa… [find](https://scholar.google.com/scholar?q=Nikita+Moghe%2C+Tom+Sherborne%2C+Mark+Steedman%2C+and+Alexandra+Birch%2E+2023%2E+Extrinsic+Evaluation+of+Machine+Translation+Metrics%2E+In+Proceedings+of+the+61st+Annual+Meeting+of+the+Association+for+Computation)
21. Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng Xin Yong, Hailey Schoelkopf, Xiangru Tang, Dragomir Radev, Alham Fikri Aji, Khalid Almubarak, Samuel Albanie, Zaid Alyafeai, Albert Webson, Edward Raff, and Colin Raffel.… — [Crosslingual Generalization through Multitask Finetuning](https://md.research.elamur.ai/free/en/papers/muennighoff-2022-crosslingual-generalization-through-multitask-finetuning/)
22. Xuan-Phi Nguyen, Wenxuan Zhang, Xin Li, Mahani Aljunied, Qingyu Tan, Liying Cheng, Guanzheng Chen, Yue Deng, Sen Yang, Chaoqun Liu, Hang Zhang, and Lidong Bing. 2023. [SeaLLMs – Large Language Models for Southeast Asia](https://md.research.elamur.ai/free/en/papers/nguyen-2023-seallms-large-language-models-southeast/). ArXiv:2312.00738 [cs].
23. OpenAI. 2023. [GPT-4 ArXiv:2303.08774 cs](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/).
24. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [Bleu: a Method for Automatic Evaluation of Machine Translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational…
25. Fred Philippy, Siwen Guo, and Shohreh Haddadan. 2023. Identifying the Correlation Between Language Distance and Cross-Lingual Transfer in a Multilingual Representation Space. In Proceedings of the 5th Workshop on Research in Computational Linguistic Typology and Multilingual NLP, pages 22–29. ArXiv:… [find](https://scholar.google.com/scholar?q=Fred+Philippy%2C+Siwen+Guo%2C+and+Shohreh+Haddadan%2E+2023%2E+Identifying+the+Correlation+Between+Language+Distance+and+Cross%2DLingual+Transfer+in+a+Multilingual+Representation+Space%2E+In+Proceedings+of+the+5th)
26. Edoardo Maria Ponti, Goran Glavaš, Olga Majewska, Qianchu Liu, Ivan Vulić, and Anna Korhonen. 2020. [XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning](https://md.research.elamur.ai/free/en/papers/ponti-2020-xcopa-multilingual-dataset-causal-commonsense/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2362–2376, Online. Association…
27. Matt Post. 2018. [A Call for Clarity in Reporting BLEU Scores](https://md.research.elamur.ai/free/en/papers/post-2018-call-clarity-reporting-bleu-scores/). In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186– 191, Brussels, Belgium. Association for Computational Linguistics.
28. Kunxun Qi, Hai Wan, Jianfeng Du, and Haolan Chen. 2022. Enhancing Cross-lingual Natural Language Inference by Prompt-learning from Cross-lingual Templates. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1910–1923, Dublin, Ire… [find](https://scholar.google.com/scholar?q=Kunxun+Qi%2C+Hai+Wan%2C+Jianfeng+Du%2C+and+Haolan+Chen%2E+2022%2E+Enhancing+Cross%2Dlingual+Natural+Language+Inference+by+Prompt%2Dlearning+from+Cross%2Dlingual+Templates%2E+In+Proceedings+of+the+60th+Annual+Meeting+of)
29. Chengwei Qin, Aston Zhang, Zhuosheng Zhang, Jiaao Chen, Michihiro Yasunaga, and Diyi Yang. 2023a. [Is ChatGPT a General-Purpose Natural Language Processing Task Solver?](https://md.research.elamur.ai/free/en/papers/qin-2023-chatgpt-general-purpose-natural-language/) ArXiv:2302.06476 [cs].
30. Libo Qin, Qiguang Chen, Fuxuan Wei, Shijue Huang, and Wanxiang Che. 2023b. [Cross-lingual Prompting: Improving Zero-shot Chain-of-Thought Reasoning across Languages](https://md.research.elamur.ai/free/en/papers/qin-2023-cross-lingual-prompting-improving-zero/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2695–2709, Singapore. Association for…
31. Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, Dipanjan Das, and Jason Wei. 2022. [Language Models are Multilingual Chain-of-Thought Reasoners](https://md.research.elamur.ai/free/en/papers/shi-2022-language-models-multilingual-chain-thought/). ArXiv:2210.03057 [cs].
32. NLLB Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hof… [find](https://scholar.google.com/scholar?q=NLLB+Team%2C+Marta+R%2E+Costa%2Djuss%C3%A0%2C+James+Cross%2C+Onur+%C3%87elebi%2C+Maha+Elbayad%2C+Kenneth+Heafield%2C+Kevin+Heffernan%2C+Elahe+Kalbassi%2C+Janice+Lam%2C+Daniel+Licht%2C+Jean+Maillard%2C+Anna+Sun%2C+Skyler+Wang%2C+Guillaume+We)
33. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,… — [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/)
34. BigScience Workshop, Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al. 2023. BLOOM: A 176B-Parameter Open-Access Multilingual Language Model. ArXiv:2211.05100 [cs]. [find](https://scholar.google.com/scholar?q=BigScience+Workshop%2C+Teven+Le+Scao%2C+Angela+Fan%2C+Christopher+Akiki%2C+Ellie+Pavlick%2C+Suzana+Ilic%CC%81%2C+Daniel+Hesslow%2C+Roman+Castagn%C3%A9%2C+Alexandra+Sasha+Luccioni%2C+Fran%C3%A7ois+Yvon%2C+Matthias+Gall%C3%A9%2C+et+al%2E+2023%2E+BL)
35. Yinfei Yang, Yuan Zhang, Chris Tar, and Jason Baldridge. 2019. [PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identification](https://md.research.elamur.ai/free/en/papers/yang-2019-paws-x-cross-lingual-adversarial/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processin…
36. Wenxuan Zhang, Sharifah Mahani Aljunied, Chang Gao, Yew Ken Chia, and Lidong Bing. 2023a. M3Exam: A Multilingual, Multimodal, Multilevel Benchmark for Examining Large Language Models. ArXiv:2306.05179 [cs]. [find](https://scholar.google.com/scholar?q=Wenxuan+Zhang%2C+Sharifah+Mahani+Aljunied%2C+Chang+Gao%2C+Yew+Ken+Chia%2C+and+Lidong+Bing%2E+2023a%2E+M3Exam%3A+A+Multilingual%2C+Multimodal%2C+Multilevel+Benchmark+for+Examining+Large+Language+Models%2E+ArXiv%3A2306%2E05179)
37. Xiang Zhang, Senyu Li, Bradley Hauer, Ning Shi, and Grzegorz Kondrak. 2023b. Don’t Trust ChatGPT when your Question is not in English: A Study of Multilingual Abilities and Types of LLMs. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 7915– 7927, Sin… [find](https://scholar.google.com/scholar?q=Xiang+Zhang%2C+Senyu+Li%2C+Bradley+Hauer%2C+Ning+Shi%2C+and+Grzegorz+Kondrak%2E+2023b%2E+Don%E2%80%99t+Trust+ChatGPT+when+your+Question+is+not+in+English%3A+A+Study+of+Multilingual+Abilities+and+Types+of+LLMs%2E+In+Proceedin)
38. Yiran Zhao, Wenxuan Zhang, Guizhen Chen, Kenji Kawaguchi, and Lidong Bing. 2024. [How do Large Language Models Handle Multilingualism?](https://md.research.elamur.ai/free/en/papers/zhao-2024-large-language-models-handle-multilingualism/) ArXiv:2402.18815 [cs].
39. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). ArXiv:2306.05685 [cs].
40. A Natural Language Inference The XNLI (Conneau et al., 2018) benchmark includes one premise and one hypothesis. The model’s job is to determine if the hypothesis is entailed, contradicted, or neutral based on the premise. It covers 15 languages, and we evaluate it using the accuracy score. [find](https://scholar.google.com/scholar?q=A+Natural+Language+Inference+The+XNLI+%28Conneau+et+al%2E%2C+2018%29+benchmark+includes+one+premise+and+one+hypothesis%2E+The+model%E2%80%99s+job+is+to+determine+if+the+hypothesis+is+entailed%2C+contradicted%2C+or+neutral+)
41. Paraphrase Identification The PAWS-X (Yang et al., 2019) benchmark consists of two sentences and requires the model to judge whether they are paraphrases. It covers 7 languages, and we assess based on accuracy score. [find](https://scholar.google.com/scholar?q=Paraphrase+Identification+The+PAWS%2DX+%28Yang+et+al%2E%2C+2019%29+benchmark+consists+of+two+sentences+and+requires+the+model+to+judge+whether+they+are+paraphrases%2E+It+covers+7+languages%2C+and+we+assess+based+on)

*Updated 30 October 2026*