---
title: "Converging to a Lingua Franca — paper by Zeng et al. (2025)"
description: "They introduce three metrics: LRDS for language-specific clustering, SADS for meaning-based… 42 references, cited by 17 papers here."
authors: ["Hongchuan Zeng", "Senyu Han", "Lu Chen", "Kai Yu"]
year: 2025
venue: "Proceedings of the 31st International Conference on Computational Linguistics"
source: "https://aclanthology.org/2025.coling-main.707/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zeng-2025-converging-lingua-franca-evolution-linguistic/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zeng-2025-converging-lingua-franca-evolution-linguistic/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/5/) / Converging to a Lingua Franca: Evolution of Linguistic Regions and Semantics Alignment in Multilingual Large Language Models

# Converging to a Lingua Franca: Evolution of Linguistic Regions and Semantics Alignment in Multilingual Large Language Models

**Authors:** Hongchuan Zeng, Senyu Han, [Lu Chen](https://md.research.elamur.ai/free/en/authors/lu-chen/) and [Kai Yu](https://md.research.elamur.ai/free/en/authors/kai-yu/) · **Year:** 2025 · **Venue:** [Proceedings of the 31st International Conference on Computational Linguistics](https://md.research.elamur.ai/free/en/venues/coling/) ([COLING 2025](https://md.research.elamur.ai/free/en/venues/coling/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.coling-main.707/) · [PDF](https://aclanthology.org/2025.coling-main.707.pdf) · [OpenAlex](https://openalex.org/W4403575125)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [17](https://md.research.elamur.ai/free/en/papers/zeng-2025-converging-lingua-franca-evolution-linguistic/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Multilingual LLMs can carry skills across languages, but how they do this internally is poorly understood. This COLING 2025 study examines neuron activations in BLOOM and LLaMA2. The authors feed a parallel corpus to each model, mean-pool MLP neuron outputs over tokens, concatenate them across layers, and compare samples using cosine-similarity maps. They introduce three metrics: LRDS for language-specific clustering, SADS for meaning-based clustering, and SKLR for the total size of key linguistic regions. Released BLOOM training checkpoints let them follow these patterns over the course of training. Each language turns out to have its own set of key neurons, and switching them off damages that language while leaving the others mostly intact. Sentences that mean the same thing in different languages produce similar activations, which points to a shared semantic space the authors call a "Lingua Franca." This alignment grows stronger with more training and with larger models. These results help explain cross-lingual transfer and suggest where interventions could improve it.

## Key points
- Experiments cover the BLOOM family, using intermediate checkpoints and BLOOM-7b1, and the LLaMA2 family, including LLaMA2-7B. Code is available at github.com/X-LANCE/LinguaFranca.
- The analysis spans nine languages: English (en), Chinese (zh), French (fr), Spanish (es), Portuguese (pt), Arabic (ar), Vietnamese (vi), Hindi (hi) and Indonesian (id).
- Key linguistic neurons sit mainly in the first and last layers. With more training and larger scale, they become more concentrated in the early layers, while the overall linguistic regions shrink.
- In LLaMA2-7B, the key neurons for a language make up a few percent of all neurons, from 4.8% for en up to 16.5% for vi. Deactivating them raises that language's perplexity far more than deactivating a random 17% of neurons. In this condition the model's outputs become degenerate.
- LRDS in LLaMA2 fluctuates more than in BLOOM, which the authors link to LLaMA2's English-centric training data.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Shanghai Jiao Tong University](https://md.research.elamur.ai/free/en/universities/shanghai-jiao-tong-university/) — University, Shanghai, China · 149 works · 16,161 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🧠 Neuroscience](https://md.research.elamur.ai/free/en/shelves/neuroscience/) · 165 works · [🧠 Cognitive Neuroscience](https://md.research.elamur.ai/free/en/shelves/neuroscience/cognitive-neuroscience/) · 164 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🌍 Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) · 828 works

## Cite

Cite as: Hongchuan Zeng, Senyu Han, Lu Chen and Kai Yu. 2025. Converging to a Lingua Franca: Evolution of Linguistic Regions and Semantics Alignment in Multilingual Large Language Models. Proceedings of the 31st International Conference on Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zeng2025converging,
  title = {{Converging to a Lingua Franca: Evolution of Linguistic Regions and Semantics Alignment in Multilingual Large Language Models}},
  author = {Hongchuan Zeng, Senyu Han, Lu Chen and Kai Yu},
  booktitle = {Proceedings of the 31st International Conference on Computational Linguistics},
  year = {2025},
  url = {https://aclanthology.org/2025.coling-main.707/}
}
```

- [← previous Benchmarking Language Model Creativity: A Case Study on Code Generation](https://md.research.elamur.ai/free/en/papers/lu-2025-benchmarking-language-model-creativity-case/)
- [next → Dialectal Coverage And Generalization in Arabic Speech Recognition](https://md.research.elamur.ai/free/en/papers/djanibekov-2025-dialectal-coverage-generalization-arabic-speech/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/5/)

## References

42 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Terra Blevins and Luke Zettlemoyer. 2022. Language contamination helps explain the cross-lingual capabilities of english pretrained models. Preprint, arXiv:2204.08110. [find](https://scholar.google.com/scholar?q=Terra+Blevins+and+Luke+Zettlemoyer%2E+2022%2E+Language+contamination+helps+explain+the+cross%2Dlingual+capabilities+of+english+pretrained+models%2E+Preprint%2C+arXiv%3A2204%2E08110%2E)
2. Eleftheria Briakou, Colin Cherry, and George Foster. 2023. Searching for needles in a haystack: On the role of incidental bilingualism in palm’s translation capability. Preprint, arXiv:2305.10266. [find](https://scholar.google.com/scholar?q=Eleftheria+Briakou%2C+Colin+Cherry%2C+and+George+Foster%2E+2023%2E+Searching+for+needles+in+a+haystack%3A+On+the+role+of+incidental+bilingualism+in+palm%E2%80%99s+translation+capability%2E+Preprint%2C+arXiv%3A2305%2E10266%2E)
3. Nadezhda Chirkova and Vassilina Nikoulina. 2024. Zero-shot cross-lingual transfer in instruction tuning of large language models. Preprint, arXiv:2402.14778. [find](https://scholar.google.com/scholar?q=Nadezhda+Chirkova+and+Vassilina+Nikoulina%2E+2024%2E+Zero%2Dshot+cross%2Dlingual+transfer+in+instruction+tuning+of+large+language+models%2E+Preprint%2C+arXiv%3A2402%2E14778%2E)
4. Christos Christodouloupoulos and Mark Steedman. 2015. [A massively parallel corpus: the bible in 100 languages](https://md.research.elamur.ai/free/en/papers/christodouloupoulos-2014-massively-parallel-corpus-bible-100/). Language resources and evaluation, 49:375–395.
5. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. Preprint, arXiv:1911.02116. [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Kartikay+Khandelwal%2C+Naman+Goyal%2C+Vishrav+Chaudhary%2C+Guillaume+Wenzek%2C+Francisco+Guzm%C3%A1n%2C+Edouard+Grave%2C+Myle+Ott%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2020%2E+Unsupervised+cross%2Dlingu)
6. Yiming Cui, Ziqing Yang, and Xin Yao. 2024. [Efficient and effective text encoding for chinese llama and alpaca](https://md.research.elamur.ai/free/en/papers/cui-2023-efficient-effective-text-encoding-chinese/). Preprint, arXiv:2304.08177.
7. Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2022. Knowledge neurons in pretrained transformers. Preprint, arXiv:2104.08696. [find](https://scholar.google.com/scholar?q=Damai+Dai%2C+Li+Dong%2C+Yaru+Hao%2C+Zhifang+Sui%2C+Baobao+Chang%2C+and+Furu+Wei%2E+2022%2E+Knowledge+neurons+in+pretrained+transformers%2E+Preprint%2C+arXiv%3A2104%2E08696%2E)
8. Kunbo Ding, Weijie Liu, Yuejian Fang, Weiquan Mao, Zhe Zhao, Tao Zhu, Haoyan Liu, Rong Tian, and Yiren Chen. 2022. A simple and effective method to improve zero-shot cross-lingual transfer learning. Preprint, arXiv:2210.09934. [find](https://scholar.google.com/scholar?q=Kunbo+Ding%2C+Weijie+Liu%2C+Yuejian+Fang%2C+Weiquan+Mao%2C+Zhe+Zhao%2C+Tao+Zhu%2C+Haoyan+Liu%2C+Rong+Tian%2C+and+Yiren+Chen%2E+2022%2E+A+simple+and+effective+method+to+improve+zero%2Dshot+cross%2Dlingual+transfer+learning%2E+P)
9. Yi Dong, Zhilin Wang, Makesh Narsimhan Sreedhar, Xianchao Wu, and Oleksii Kuchaiev. 2023. Steerlm: Attribute conditioned sft as an (user-steerable) alternative to rlhf. Preprint, arXiv:2310.05344. [find](https://scholar.google.com/scholar?q=Yi+Dong%2C+Zhilin+Wang%2C+Makesh+Narsimhan+Sreedhar%2C+Xianchao+Wu%2C+and+Oleksii+Kuchaiev%2E+2023%2E+Steerlm%3A+Attribute+conditioned+sft+as+an+%28user%2Dsteerable%29+alternative+to+rlhf%2E+Preprint%2C+arXiv%3A2310%2E05344%2E)
10. Jinlan Fu, See-Kiong Ng, and Pengfei Liu. 2022. Polyglot prompt: Multilingual multitask promptraining. Preprint, arXiv:2204.14264. — [arXiv:2204.14264](https://arxiv.org/abs/2204.14264) [find](https://scholar.google.com/scholar?q=Jinlan+Fu%2C+See%2DKiong+Ng%2C+and+Pengfei+Liu%2E+2022%2E+Polyglot+prompt%3A+Multilingual+multitask+promptraining%2E+Preprint%2C+arXiv%3A2204%2E14264%2E+%E2%80%94+%5BarXiv%3A2204%2E14264%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2204%2E14264%29)
11. Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Ani… [find](https://scholar.google.com/scholar?q=Leo+Gao%2C+Jonathan+Tow%2C+Baber+Abbasi%2C+Stella+Biderman%2C+Sid+Black%2C+Anthony+DiPofi%2C+Charles+Foster%2C+Laurence+Golding%2C+Jeffrey+Hsu%2C+Alain+Le+Noac%E2%80%99h%2C+Haonan+Li%2C+Kyle+McDonell%2C+Niklas+Muennighoff%2C+Chris+Oci)
12. Pengzhi Gao, Zhongjun He, Hua Wu, and Haifeng Wang. 2024. Towards boosting many-to-many multilingual machine translation with large language models. Preprint, arXiv:2401.05861. — [arXiv:2401.05861](https://arxiv.org/abs/2401.05861) [find](https://scholar.google.com/scholar?q=Pengzhi+Gao%2C+Zhongjun+He%2C+Hua+Wu%2C+and+Haifeng+Wang%2E+2024%2E+Towards+boosting+many%2Dto%2Dmany+multilingual+machine+translation+with+large+language+models%2E+Preprint%2C+arXiv%3A2401%2E05861%2E+%E2%80%94+%5BarXiv%3A2401%2E05861%5D%28ht)
13. Tahmid Hasan, Abhik Bhattacharjee, Md. Saiful Islam, Kazi Mubasshir, Yuan-Fang Li, Yong-Bin Kang, M. Sohel Rahman, and Rifat Shahriyar. 2021. XL-sum: Large-scale multilingual abstractive summarization for 44 languages. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pa… [find](https://scholar.google.com/scholar?q=Tahmid+Hasan%2C+Abhik+Bhattacharjee%2C+Md%2E+Saiful+Islam%2C+Kazi+Mubasshir%2C+Yuan%2DFang+Li%2C+Yong%2DBin+Kang%2C+M%2E+Sohel+Rahman%2C+and+Rifat+Shahriyar%2E+2021%2E+XL%2Dsum%3A+Large%2Dscale+multilingual+abstractive+summarization)
14. Dan Hendrycks and Kevin Gimpel. 2023. Gaussian error linear units (gelus). Preprint, arXiv:1606.08415. [find](https://scholar.google.com/scholar?q=Dan+Hendrycks+and+Kevin+Gimpel%2E+2023%2E+Gaussian+error+linear+units+%28gelus%29%2E+Preprint%2C+arXiv%3A1606%2E08415%2E)
15. Oskar Holmström, Jenny Kunz, and Marco Kuhlmann. 2023. [Bridging the resource gap: Exploring the efficacy of English and multilingual LLMs for Swedish](https://md.research.elamur.ai/free/en/papers/holmstrom-2023-bridging-resource-gap-exploring-efficacy/). In Proceedings of the Second Workshop on Resources and Representations for Under-Resourced Languages and Domains (RESOURCEFUL-2023), pages 92–110, Tó…
16. Guanlin Li, Xuechen Zhao, Amir Jafari, Wenhao Shao, Reza Farahbakhsh, and Noel Crespi. 2024. Improving cross-lingual transfer with contrastive negative learning and self-training. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluatio… [find](https://scholar.google.com/scholar?q=Guanlin+Li%2C+Xuechen+Zhao%2C+Amir+Jafari%2C+Wenhao+Shao%2C+Reza+Farahbakhsh%2C+and+Noel+Crespi%2E+2024%2E+Improving+cross%2Dlingual+transfer+with+contrastive+negative+learning+and+self%2Dtraining%2E+In+Proceedings+of+th)
17. Huiling Li, Jing Qu, Chuansheng Chen, Yanjun Chen, Gui Xue, Lei Zhang, Chengrou Lu, and Leilei Mei. 2019. Lexical learning in a new language leads to neural pattern similarity with word reading in native language. Human Brain Mapping, 40(1):98–109. [find](https://scholar.google.com/scholar?q=Huiling+Li%2C+Jing+Qu%2C+Chuansheng+Chen%2C+Yanjun+Chen%2C+Gui+Xue%2C+Lei+Zhang%2C+Chengrou+Lu%2C+and+Leilei+Mei%2E+2019%2E+Lexical+learning+in+a+new+language+leads+to+neural+pattern+similarity+with+word+reading+in+nat)
18. Jindřich Libovický, Rudolf Rosa, and Alexander Fraser. 2020. [On the language neutrality of pre-trained multilingual representations](https://md.research.elamur.ai/free/en/papers/libovicky-2020-language-neutrality-pre-trained-multilingual/). In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1663–1674, Online. Association for Computational Linguistics.
19. Saima Malik-Moraleda, Olessia Jouravlev, Maya Taliaferro, Zachary Mineroff, Theodore Cucu, Kyle Mahowald, Idan A Blank, and Evelina Fedorenko. 2024. Functional characterization of the language network of polyglots and hyperpolyglots with precision fMRI. Cerebral Cortex, 34(3):bhae049. [find](https://scholar.google.com/scholar?q=Saima+Malik%2DMoraleda%2C+Olessia+Jouravlev%2C+Maya+Taliaferro%2C+Zachary+Mineroff%2C+Theodore+Cucu%2C+Kyle+Mahowald%2C+Idan+A+Blank%2C+and+Evelina+Fedorenko%2E+2024%2E+Functional+characterization+of+the+language+network)
20. Emily S Nichols, Yue Gao, Sofia Fregni, Li Liu, and Marc F Joanisse. 2021. Individual differences in representational similarity of first and second languages in the bilingual brain. Human Brain Mapping, 42(16):5433–5445. [find](https://scholar.google.com/scholar?q=Emily+S+Nichols%2C+Yue+Gao%2C+Sofia+Fregni%2C+Li+Liu%2C+and+Marc+F+Joanisse%2E+2021%2E+Individual+differences+in+representational+similarity+of+first+and+second+languages+in+the+bilingual+brain%2E+Human+Brain+Mappi)
21. OpenAI. 2024. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv:2303.08774.
22. Preprint, Michel Paradis. 1985. On the representation of two languages in one brain. Language Sciences, 7(1):1– 39. Towards a Neurology of Language. [find](https://scholar.google.com/scholar?q=Preprint%2C+Michel+Paradis%2E+1985%2E+On+the+representation+of+two+languages+in+one+brain%2E+Language+Sciences%2C+7%281%29%3A1%E2%80%93+39%2E+Towards+a+Neurology+of+Language%2E)
23. Michel Paradis. 2000. Generalizable outcomes of bilingual aphasia research. Folia phoniatrica et logopaed-ica : official organ of the International Association of Logopedics and Phoniatrics (IALP), 52:54–64. [find](https://scholar.google.com/scholar?q=Michel+Paradis%2E+2000%2E+Generalizable+outcomes+of+bilingual+aphasia+research%2E+Folia+phoniatrica+et+logopaed%2Dica+%3A+official+organ+of+the+International+Association+of+Logopedics+and+Phoniatrics+%28IALP%29%2C+52)
24. Telmo Pires, Eva Schlinger, and Dan Garrette. 2019. [How multilingual is multilingual BERT?](https://md.research.elamur.ai/free/en/papers/pires-2019-how-multilingual-is-multilingual-bert/) In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4996–5001, Florence, Italy. Association for Computational Linguistics.
25. Libo Qin, Qiguang Chen, Fuxuan Wei, Shijue Huang, and Wanxiang Che. 2023. [Cross-lingual prompting: Improving zero-shot chain-of-thought reasoning across languages](https://md.research.elamur.ai/free/en/papers/qin-2023-cross-lingual-prompting-improving-zero/). Preprint, arXiv:2310.14799.
26. Libo Qin, Qiguang Chen, Yuhang Zhou, Zhi Chen, Yinghui Li, Lizi Liao, Min Li, Wanxiang Che, and Philip S. Yu. 2024. Multilingual large language model: A survey of resources, taxonomy and frontiers. Preprint, arXiv:2404.04925. [find](https://scholar.google.com/scholar?q=Libo+Qin%2C+Qiguang+Chen%2C+Yuhang+Zhou%2C+Zhi+Chen%2C+Yinghui+Li%2C+Lizi+Liao%2C+Min+Li%2C+Wanxiang+Che%2C+and+Philip+S%2E+Yu%2E+2024%2E+Multilingual+large+language+model%3A+A+survey+of+resources%2C+taxonomy+and+frontiers%2E+Pr)
27. Prajit Ramachandran, Barret Zoph, and Quoc V. Le. 2017. Searching for activation functions. Preprint, arXiv:1710.05941. [find](https://scholar.google.com/scholar?q=Prajit+Ramachandran%2C+Barret+Zoph%2C+and+Quoc+V%2E+Le%2E+2017%2E+Searching+for+activation+functions%2E+Preprint%2C+arXiv%3A1710%2E05941%2E)
28. Nils Reimers and Iryna Gurevych. 2019. [Sentence-bert: Sentence embeddings using siamese bert-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). Preprint, arXiv:1908.10084.
29. Andy Rosenbaum, Saleh Soltan, Wael Hamza, Yannick Versley, and Markus Boese. 2022. LINGUIST: Language model instruction tuning to generate annotated utterances for intent classification and slot tagging. In Proceedings of the 29th International Conference on Computational Linguistics, pages 218–241,… [find](https://scholar.google.com/scholar?q=Andy+Rosenbaum%2C+Saleh+Soltan%2C+Wael+Hamza%2C+Yannick+Versley%2C+and+Markus+Boese%2E+2022%2E+LINGUIST%3A+Language+model+instruction+tuning+to+generate+annotated+utterances+for+intent+classification+and+slot+taggi)
30. Uri Shaham, Jonathan Herzig, Roee Aharoni, Idan Szpektor, Reut Tsarfaty, and Matan Eyal. 2024. [Multilingual instruction tuning with just a pinch of multilinguality](https://md.research.elamur.ai/free/en/papers/shaham-2024-multilingual-instruction-tuning-just-pinch/). Preprint, arXiv:2401.01854.
31. Zhiqing Sun, Yikang Shen, Hongxin Zhang, Qinhong Zhou, Zhenfang Chen, David Cox, Yiming Yang, and Chuang Gan. 2024. [Salmon: Self-alignment with instructable reward models](https://md.research.elamur.ai/free/en/papers/sun-2023-salmon-self-alignment-instructable-reward/). Preprint, arXiv:2310.05910.
32. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,… — [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/)
33. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2023. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). Preprint, arXiv:1706.03762.
34. Chris Wendler, Veniamin Veselovsky, Giovanni Monea, and Robert West. 2024. [Do llamas work in english? on the latent language of multilingual transformers](https://md.research.elamur.ai/free/en/papers/wendler-2024-do-llamas-work-english-latent/). Preprint, arXiv:2402.10588.
35. BigScience Workshop. 2023. Bloom: A 176b-parameter open-access multilingual language model. Preprint, arXiv:2211.05100. [find](https://scholar.google.com/scholar?q=BigScience+Workshop%2E+2023%2E+Bloom%3A+A+176b%2Dparameter+open%2Daccess+multilingual+language+model%2E+Preprint%2C+arXiv%3A2211%2E05100%2E)
36. Shijie Wu and Mark Dredze. 2019. [Beto, bentz, becas: The surprising cross-lingual effectiveness of BERT](https://md.research.elamur.ai/free/en/papers/wu-2019-beto-bentz-becas-surprising-cross/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 833–844,…
37. Min Xu, Duo Li, and Ping Li. 2021. Brain decoding in multiple languages: Can cross-language brain decoding work? Brain and Language, 215:104922. [find](https://scholar.google.com/scholar?q=Min+Xu%2C+Duo+Li%2C+and+Ping+Li%2E+2021%2E+Brain+decoding+in+multiple+languages%3A+Can+cross%2Dlanguage+brain+decoding+work%3F+Brain+and+Language%2C+215%3A104922%2E)
38. Dongkeun Yoon, Joel Jang, Sungdong Kim, Seungone Kim, Sheikh Shafayat, and Minjoon Seo. 2024. [Langbridge: Multilingual reasoning without multilingual supervision](https://md.research.elamur.ai/free/en/papers/yoon-2024-langbridge-multilingual-reasoning-without-multilingual/). Preprint, arXiv:2401.10695.
39. Hongchuan Zeng, Hongshen Xu, Lu Chen, and Kai Yu. 2024a. Multilingual brain surgeon: Large language models can be compressed leaving no language behind. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages… [find](https://scholar.google.com/scholar?q=Hongchuan+Zeng%2C+Hongshen+Xu%2C+Lu+Chen%2C+and+Kai+Yu%2E+2024a%2E+Multilingual+brain+surgeon%3A+Large+language+models+can+be+compressed+leaving+no+language+behind%2E+In+Proceedings+of+the+2024+Joint+International+)
40. Jiali Zeng, Fandong Meng, Yongjing Yin, and Jie Zhou. 2024b. Tim: Teaching large language models to translate with comparison. Preprint, arXiv:2307.04408. — [arXiv:2307.04408](https://arxiv.org/abs/2307.04408) [find](https://scholar.google.com/scholar?q=Jiali+Zeng%2C+Fandong+Meng%2C+Yongjing+Yin%2C+and+Jie+Zhou%2E+2024b%2E+Tim%3A+Teaching+large+language+models+to+translate+with+comparison%2E+Preprint%2C+arXiv%3A2307%2E04408%2E+%E2%80%94+%5BarXiv%3A2307%2E04408%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F23)
41. Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. 2022. [Opt: Open pre-trai…](https://md.research.elamur.ai/free/en/papers/zhang-2022-opt-open-pre-trained-transformer/)
42. Zhihao Zhang, Jun Zhao, Qi Zhang, Tao Gui, and Xuanjing Huang. 2024. [Unveiling linguistic regions in large language models](https://md.research.elamur.ai/free/en/papers/zhang-2024-unveiling-linguistic-regions-large-language/). Preprint, arXiv:2402.14700.

*Updated 30 October 2026*