---
title: "ShifCon — paper by Zhang et al. (2025) — summary and ideas"
description: "ShifCon thus offers a cheap way to narrow the language gap when supervised multilingual data is… 53 references, cited by 6 papers here."
authors: ["Hengyuan Zhang", "Chenming Shang", "Sizhe Wang", "Dongdong Zhang", "Yiyao Yu", "Feng Yao", "Renliang Sun", "Yujiu Yang", "Furu Wei"]
year: 2025
venue: "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2025.acl-long.239/"
doi: "10.18653/v1/2025.acl-long.239"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2025-shifcon-enhancing-non-dominant-language/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2025-shifcon-enhancing-non-dominant-language/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/11/) / ShifCon: Enhancing Non-Dominant Language Capabilities with a Shift-based Multilingual Contrastive Framework

# ShifCon: Enhancing Non-Dominant Language Capabilities with a Shift-based Multilingual Contrastive Framework

**Authors:** [Hengyuan Zhang](https://md.research.elamur.ai/free/en/authors/hengyuan-zhang/), [Chenming Shang](https://md.research.elamur.ai/free/en/authors/chenming-shang/), [Sizhe Wang](https://md.research.elamur.ai/free/en/authors/sizhe-wang/), [Dongdong Zhang](https://md.research.elamur.ai/free/en/authors/dongdong-zhang/), [Yiyao Yu](https://md.research.elamur.ai/free/en/authors/yiyao-yu/), Feng Yao, [Renliang Sun](https://md.research.elamur.ai/free/en/authors/renliang-sun/), [Yujiu Yang](https://md.research.elamur.ai/free/en/authors/yujiu-yang/) and [Furu Wei](https://md.research.elamur.ai/free/en/authors/furu-wei/) · **Year:** 2025 · **Venue:** [Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2025](https://md.research.elamur.ai/free/en/venues/acl/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.acl-long.239/) · [PDF](https://aclanthology.org/2025.acl-long.239.pdf) · [DOI 10.18653/v1/2025.acl-long.239](https://doi.org/10.18653/v1/2025.acl-long.239) · [OpenAlex](https://openalex.org/W4412886680)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [6](https://md.research.elamur.ai/free/en/papers/zhang-2025-shifcon-enhancing-non-dominant-language/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Multilingual LLMs fine-tuned on translated data (MSFT) still lag behind in non-dominant languages compared with English. Producing more high-quality translated supervision is costly and error-prone. ShifCon addresses this by changing how the model processes these languages internally rather than by adding data.

LDA visualisations show that even middle-layer representations keep distinct language-specific subspaces. Building on this, ShifCon works in two shifts:
- **Shift toward:** at a chosen layer, it subtracts a per-layer vector for the non-dominant language and adds the English one. These language vectors are averaged pooled sentence representations. The input is then handled as if it were English and can draw on knowledge stored in English form.
- **Shift back:** at a later layer, the shift is reversed so the model answers in the original language.

A subspace distance metric selects the layer range for shifting. It uses SVD-based affine subspaces and a Riemannian distance. Multilingual contrastive learning on translation pairs further tightens the alignment between the shifted and English representations.

Experiments on Llama-2, XGLM and BLOOM show consistent gains, largest for low-resource languages. ShifCon thus offers a cheap way to narrow the language gap when supervised multilingual data is scarce.

## Key points
- ShifCon reports an 18.9% improvement on MGSM for low-resource languages with Llama-2 7B.
- Language subspaces are sized to explain 90% of variance. Distances between subspaces of differing dimensionality are compared with a Riemannian metric on positive definite matrices.
- About 30% of layers, those with the lowest subspace distance, are used for shifting. These are middle layers, e.g. [9, 16] in XGLM 564M and [13, 22] in XGLM 7.5B. The authors speculate that these layers mainly aggregate information.
- Applying contrastive learning directly to the original (unshifted) representations can erase language-specific information and hurt generation in the target language.
- Limitations:
  - The method relies on parallel translation pairs.
  - Only decoder-only generative models of at most 8B parameters were tested.
  - Encoder-decoder architectures are left for future work.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/) — Company, Redmond, United States · 325 works · 142,931 citations
- [Tsinghua University](https://md.research.elamur.ai/free/en/universities/tsinghua-university/) — University, Beijing, China · 333 works · 87,019 citations
- [University of Southern California](https://md.research.elamur.ai/free/en/universities/university-of-southern-california/) — University, Los Angeles, United States · 162 works · 35,836 citations
- [University of California, Los Angeles](https://md.research.elamur.ai/free/en/universities/university-of-california-los-angeles/) — University, Los Angeles, United States · 87 works · 54,961 citations
- [University of California San Diego](https://md.research.elamur.ai/free/en/universities/university-of-california-san-diego/) — University, San Diego, United States · 73 works · 51,948 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🌍 Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) · 828 works

## Cite

Cite as: Hengyuan Zhang, Chenming Shang, Sizhe Wang, Dongdong Zhang, Yiyao Yu, Feng Yao, Renliang Sun, Yujiu Yang and Furu Wei. 2025. ShifCon: Enhancing Non-Dominant Language Capabilities with a Shift-based Multilingual Contrastive Framework. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhang2025shifcon,
  title = {{ShifCon: Enhancing Non-Dominant Language Capabilities with a Shift-based Multilingual Contrastive Framework}},
  author = {Hengyuan Zhang, Chenming Shang, Sizhe Wang, Dongdong Zhang, Yiyao Yu, Feng Yao, Renliang Sun, Yujiu Yang and Furu Wei},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.239},
  url = {https://doi.org/10.18653/v1/2025.acl-long.239}
}
```

- [← previous rStar2-Agent: Agentic Reasoning Technical Report](https://md.research.elamur.ai/free/en/papers/shang-2025-rstar2-agent-agentic-reasoning/)
- [next → Skywork-R1V3 Technical Report](https://md.research.elamur.ai/free/en/papers/shen-2025-skywork-r1v3-technical-report/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/11/)

## References

53 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774.
2. Ashish Agrawal, Barah Fazili, and Preethi Jyothi. 2024. Translation errors significantly impact low-resource languages in cross-lingual learning. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 2: Short Papers), pages 319–329, St… [find](https://scholar.google.com/scholar?q=Ashish+Agrawal%2C+Barah+Fazili%2C+and+Preethi+Jyothi%2E+2024%2E+Translation+errors+significantly+impact+low%2Dresource+languages+in+cross%2Dlingual+learning%2E+In+Proceedings+of+the+18th+Conference+of+the+European+)
3. Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. 2023. [Palm 2 technical report](https://md.research.elamur.ai/free/en/papers/anil-2023-palm-2-technical-report/). arXiv preprint arXiv:2305.10403.
4. Silvére Bonnabel and Rodolphe Sepulchre. 2009. Riemannian metric and geometric mean for positive semidefinite matrices of fixed rank. SIAM Journal on Matrix Analysis and Applications, 31:1055–1070. [find](https://scholar.google.com/scholar?q=Silv%C3%A9re+Bonnabel+and+Rodolphe+Sepulchre%2E+2009%2E+Riemannian+metric+and+geometric+mean+for+positive+semidefinite+matrices+of+fixed+rank%2E+SIAM+Journal+on+Matrix+Analysis+and+Applications%2C+31%3A1055%E2%80%931070%2E)
5. Tyler Chang, Zhuowen Tu, and Benjamin Bergen. 2022. [The geometry of multilingual language model representations](https://md.research.elamur.ai/free/en/papers/chang-2022-geometry-multilingual-language-model-representations/). In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 119–136, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
6. Nuo Chen, Zinan Zheng, Ning Wu, Linjun Shou, Ming Gong, Yangqiu Song, Dongmei Zhang, and Jia Li. 2023a. Breaking language barriers in multilingual mathematical reasoning: Insights and observations. arXiv preprint arXiv:2310.20246. [find](https://scholar.google.com/scholar?q=Nuo+Chen%2C+Zinan+Zheng%2C+Ning+Wu%2C+Linjun+Shou%2C+Ming+Gong%2C+Yangqiu+Song%2C+Dongmei+Zhang%2C+and+Jia+Li%2E+2023a%2E+Breaking+language+barriers+in+multilingual+mathematical+reasoning%3A+Insights+and+observations%2E+ar)
7. Nuo Chen, Zinan Zheng, Ning Wu, Linjun Shou, Ming Gong, Yangqiu Song, Dongmei Zhang, and Jia Li. 2023b. Breaking language barriers in multilingual mathematical reasoning: Insights and observations. arXiv preprint arXiv:2310.20246. [find](https://scholar.google.com/scholar?q=Nuo+Chen%2C+Zinan+Zheng%2C+Ning+Wu%2C+Linjun+Shou%2C+Ming+Gong%2C+Yangqiu+Song%2C+Dongmei+Zhang%2C+and+Jia+Li%2E+2023b%2E+Breaking+language+barriers+in+multilingual+mathematical+reasoning%3A+Insights+and+observations%2E+ar)
8. Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. XNLI: Evaluating cross-lingual sentence representations. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2475–2485, Brussels, Bel… [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Ruty+Rinott%2C+Guillaume+Lample%2C+Adina+Williams%2C+Samuel+Bowman%2C+Holger+Schwenk%2C+and+Veselin+Stoyanov%2E+2018%2E+XNLI%3A+Evaluating+cross%2Dlingual+sentence+representations%2E+In+Proceedings+of+the)
9. Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. 2023. Free dolly: Introducing the world’s first truly open instruction-tuned llm. databricks. [find](https://scholar.google.com/scholar?q=Mike+Conover%2C+Matt+Hayes%2C+Ankit+Mathur%2C+Jianwei+Xie%2C+Jun+Wan%2C+Sam+Shah%2C+Ali+Ghodsi%2C+Patrick+Wendell%2C+Matei+Zaharia%2C+and+Reynold+Xin%2E+2023%2E+Free+dolly%3A+Introducing+the+world%E2%80%99s+first+truly+open+instruct)
10. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv e-prints, pages arXiv–2407.
11. Daniil Gurgurov, Tanja Bäumel, and Tatiana Anikina. 2024. Multilingual large language models and curse of multilinguality. arXiv preprint arXiv:2406.10602. — [arXiv:2406.10602](https://arxiv.org/abs/2406.10602) [find](https://scholar.google.com/scholar?q=Daniil+Gurgurov%2C+Tanja+B%C3%A4umel%2C+and+Tatiana+Anikina%2E+2024%2E+Multilingual+large+language+models+and+curse+of+multilinguality%2E+arXiv+preprint+arXiv%3A2406%2E10602%2E+%E2%80%94+%5BarXiv%3A2406%2E10602%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2)
12. Haoyang Huang, Tianyi Tang, Dongdong Zhang, Wayne Xin Zhao, Ting Song, Yan Xia, and Furu Wei. 2023. [Not all languages are created equal in llms: Improving multilingual capability by cross-lingual-thought prompting](https://md.research.elamur.ai/free/en/papers/huang-2023-all-languages-created-equal-llms/). In Findings of the Association for Computational Linguistics: EMNLP 2023.
13. Nora Kassner, Philipp Dufter, and Hinrich Schütze. 2021. [Multilingual LAMA: Investigating knowledge in multilingual pretrained language models](https://md.research.elamur.ai/free/en/papers/kassner-2021-multilingual-lama-investigating-knowledge-multilingual/). In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 3250–3258, Online. Associ…
14. Nataliia Kholodna, Sahib Julka, Mohammad Khodadadi, Muhammed Nurullah Gumus, and Michael Granitzer. 2024. Llms in the loop: Leveraging large language model annotations for active learning in low-resource languages. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases… [find](https://scholar.google.com/scholar?q=Nataliia+Kholodna%2C+Sahib+Julka%2C+Mohammad+Khodadadi%2C+Muhammed+Nurullah+Gumus%2C+and+Michael+Granitzer%2E+2024%2E+Llms+in+the+loop%3A+Leveraging+large+language+model+annotations+for+active+learning+in+low%2Dresou)
15. Takeshi Kojima, Itsuki Okimura, Yusuke Iwasawa, Hitomi Yanaka, and Yutaka Matsuo. 2024. [On the multilingual ability of decoder-based pre-trained language models: Finding and controlling language-specific neurons](https://md.research.elamur.ai/free/en/papers/kojima-2024-multilingual-ability-decoder-based-pre/). In Proceedings of the 2024 Conference of the North American Chapter of the Association…
16. Cunliang Kong, Yun Chen, Hengyuan Zhang, Liner Yang, and Erhong Yang. 2022. Multitasking framework for unsupervised simple definition generation. arXiv preprint arXiv:2203.12926. [find](https://scholar.google.com/scholar?q=Cunliang+Kong%2C+Yun+Chen%2C+Hengyuan+Zhang%2C+Liner+Yang%2C+and+Erhong+Yang%2E+2022%2E+Multitasking+framework+for+unsupervised+simple+definition+generation%2E+arXiv+preprint+arXiv%3A2203%2E12926%2E)
17. Mikhail Kozhevnikov and Ivan Titov. 2014. Cross-lingual model transfer using feature representation projection. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 579–585. [find](https://scholar.google.com/scholar?q=Mikhail+Kozhevnikov+and+Ivan+Titov%2E+2014%2E+Cross%2Dlingual+model+transfer+using+feature+representation+projection%2E+In+Proceedings+of+the+52nd+Annual+Meeting+of+the+Association+for+Computational+Linguisti)
18. Chong Li, Shaonan Wang, Jiajun Zhang, and Chengqing Zong. 2024. Improving in-context learning of multilingual generative language models with cross-lingual alignment. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language… [find](https://scholar.google.com/scholar?q=Chong+Li%2C+Shaonan+Wang%2C+Jiajun+Zhang%2C+and+Chengqing+Zong%2E+2024%2E+Improving+in%2Dcontext+learning+of+multilingual+generative+language+models+with+cross%2Dlingual+alignment%2E+In+Proceedings+of+the+2024+Confer)
19. Dawei Li, Hengyuan Zhang, Yanran Li, and Shiping Yang. 2023a. Multi-level contrastive learning for script-based character understanding. arXiv preprint arXiv:2310.13231. [find](https://scholar.google.com/scholar?q=Dawei+Li%2C+Hengyuan+Zhang%2C+Yanran+Li%2C+and+Shiping+Yang%2E+2023a%2E+Multi%2Dlevel+contrastive+learning+for+script%2Dbased+character+understanding%2E+arXiv+preprint+arXiv%3A2310%2E13231%2E)
20. Haonan Li, Fajri Koto, Minghao Wu, Alham Fikri Aji, and Timothy Baldwin. 2023b. [Bactrian-x: Multilingual replicable instruction-following models with low-rank adaptation](https://md.research.elamur.ai/free/en/papers/li-2023-bactrian-x-multilingual-replicable-instruction/). arXiv preprint arXiv:2305.15011.
21. Tianjian Li and Kenton Murray. 2023. Why does zero-shot cross-lingual generation fail? an explanation and a solution. In Findings of the Association for Computational Linguistics: ACL 2023, pages 12461–12476, Toronto, Canada. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Tianjian+Li+and+Kenton+Murray%2E+2023%2E+Why+does+zero%2Dshot+cross%2Dlingual+generation+fail%3F+an+explanation+and+a+solution%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL+2023%2C+pages+1246)
22. Jindřich Libovický, Rudolf Rosa, and Alexander Fraser. 2020. [On the language neutrality of pre-trained multilingual representations](https://md.research.elamur.ai/free/en/papers/libovicky-2020-language-neutrality-pre-trained-multilingual/). In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1663–1674, Online. Association for Computational Linguistics.
23. Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, Ramakanth Pasunuru, Sam Shleifer, Punit Singh Koura, Vishrav Chaudhary, Brian O’Horo, Jeff Wang, Luke Zettlemoyer, Zornitsa Kozareva, Mona Diab, Veselin Stoyano… — [Few-shot Learning with Multilingual Language Models](https://md.research.elamur.ai/free/en/papers/lin-2021-few-shot-learning-multilingual-language/)
24. Yihong Liu, Chunlan Ma, Haotian Ye, and Hinrich Schuetze. 2024. TransliCo: A contrastive learning framework to address the script barrier in multilingual pretrained language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pa… [find](https://scholar.google.com/scholar?q=Yihong+Liu%2C+Chunlan+Ma%2C+Haotian+Ye%2C+and+Hinrich+Schuetze%2E+2024%2E+TransliCo%3A+A+contrastive+learning+framework+to+address+the+script+barrier+in+multilingual+pretrained+language+models%2E+In+Proceedings+of+)
25. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Ilya+Loshchilov+and+Frank+Hutter%2E+2019%2E+Decoupled+weight+decay+regularization%2E+In+International+Conference+on+Learning+Representations%2E)
26. Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng Xin Yong, Hailey Schoelkopf, Xiangru Tang, Dragomir Radev, Alham Fikri Aji, Khalid Almubarak, Samuel Albanie, Zaid Alyafeai, Albert Webson, Edward Raff, and Colin Raffel.… — [Crosslingual Generalization through Multitask Finetuning](https://md.research.elamur.ai/free/en/papers/muennighoff-2022-crosslingual-generalization-through-multitask-finetuning/)
27. Edoardo Maria Ponti, Goran Glavaš, Olga Majewska, Qianchu Liu, Ivan Vulić, and Anna Korhonen. 2020. [XCOPA: A multilingual dataset for causal commonsense reasoning](https://md.research.elamur.ai/free/en/papers/ponti-2020-xcopa-multilingual-dataset-causal-commonsense/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2362–2376, Online. Association…
28. Maja Popović. 2017. [chrF++: words helping character n-grams](https://md.research.elamur.ai/free/en/papers/popovic-2017-chrf-words-helping-character-n/). In Proceedings of the Second Conference on Machine Translation, pages 612–618, Copenhagen, Denmark. Association for Computational Linguistics.
29. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. [Learning transferable visual models from natural language supervision](https://md.research.elamur.ai/free/en/papers/radford-2021-learning-transferable-visual-models-natural/). In International Conference on Machine Learning, pages 8748–8763. PM…
30. Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al. 2022. Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100. [find](https://scholar.google.com/scholar?q=Teven+Le+Scao%2C+Angela+Fan%2C+Christopher+Akiki%2C+Ellie+Pavlick%2C+Suzana+Ilic%CC%81%2C+Daniel+Hesslow%2C+Roman+Castagn%C3%A9%2C+Alexandra+Sasha+Luccioni%2C+Fran%C3%A7ois+Yvon%2C+Matthias+Gall%C3%A9%2C+et+al%2E+2022%2E+Bloom%3A+A+176b%2Dparameter)
31. Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, et al. [Language models are multilingual chain-of-thought reasoners](https://md.research.elamur.ai/free/en/papers/shi-2022-language-models-multilingual-chain-thought/). In The Eleventh International Conference on Learning Representations.
32. Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, et al. 2022. [Language models are multilingual chain-of-thought reasoners](https://md.research.elamur.ai/free/en/papers/shi-2022-language-models-multilingual-chain-thought/). In International Conference on Learning Representations (ICLR).
33. Zhen Tan, Alimohammad Beigi, Song Wang, Ruocheng Guo, Amrita Bhattacharjee, Bohan Jiang, Mansooreh Karami, Jundong Li, Lu Cheng, and Huan Liu. 2024. Large language models for data annotation: A survey. arXiv preprint arXiv:2402.13446. — [arXiv:2402.13446](https://arxiv.org/abs/2402.13446) [find](https://scholar.google.com/scholar?q=Zhen+Tan%2C+Alimohammad+Beigi%2C+Song+Wang%2C+Ruocheng+Guo%2C+Amrita+Bhattacharjee%2C+Bohan+Jiang%2C+Mansooreh+Karami%2C+Jundong+Li%2C+Lu+Cheng%2C+and+Huan+Liu%2E+2024%2E+Large+language+models+for+data+annotation%3A+A+survey)
34. Tianyi Tang, Wenyang Luo, Haoyang Huang, Dongdong Zhang, Xiaolei Wang, Xin Zhao, Furu Wei, and Ji-Rong Wen. 2024. [Language-specific neurons: The key to multilingual capabilities in large language models](https://md.research.elamur.ai/free/en/papers/tang-2024-language-specific-neurons-key-multilingual/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volu…
35. Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction-following llama model. [https://github.com/tatsu-lab/stanford_alpaca](https://github.com/tatsu-lab/stanford_alpaca). [↗](https://github.com/tatsu-lab/stanford_alpaca)
36. "NLLB Team". 2022. No language left behind: Scaling human-centered machine translation. [find](https://scholar.google.com/scholar?q=%22NLLB+Team%22%2E+2022%2E+No+language+left+behind%3A+Scaling+human%2Dcentered+machine+translation%2E)
37. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,… — [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/)
38. Andrea W Wen-Yi and David Mimno. 2023. Hyperpoly-glot LLMs: Cross-lingual interpretability in token embeddings. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 1124–1131, Singapore. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Andrea+W+Wen%2DYi+and+David+Mimno%2E+2023%2E+Hyperpoly%2Dglot+LLMs%3A+Cross%2Dlingual+interpretability+in+token+embeddings%2E+In+Proceedings+of+the+2023+Conference+on+Empirical+Methods+in+Natural+Language+Processin)
39. Shaoyang Xu, Junzhuo Li, and Deyi Xiong. 2023. Language representation projection: Can we transfer factual knowledge across languages in multilingual language models? In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 3692–3702, Singapore. Association fo… [find](https://scholar.google.com/scholar?q=Shaoyang+Xu%2C+Junzhuo+Li%2C+and+Deyi+Xiong%2E+2023%2E+Language+representation+projection%3A+Can+we+transfer+factual+knowledge+across+languages+in+multilingual+language+models%3F+In+Proceedings+of+the+2023+Confer)
40. Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. [mT5: A massively multilingual pre-trained text-to-text transformer](https://md.research.elamur.ai/free/en/papers/xue-2021-mt5-massively-multilingual-pre-trained/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Ling…
41. Feng Yao, Yufan Zhuang, Zihao Sun, Sunan Xu, Animesh Kumar, and Jingbo Shang. 2024. Data contamination can cross language barriers. arXiv preprint arXiv:2406.13236. [find](https://scholar.google.com/scholar?q=Feng+Yao%2C+Yufan+Zhuang%2C+Zihao+Sun%2C+Sunan+Xu%2C+Animesh+Kumar%2C+and+Jingbo+Shang%2E+2024%2E+Data+contamination+can+cross+language+barriers%2E+arXiv+preprint+arXiv%3A2406%2E13236%2E)
42. Da Yin, Hritik Bansal, Masoud Monajatipoor, Liunian Harold Li, and Kai-Wei Chang. 2022. GeoM-LAMA: Geo-diverse commonsense probing on multilingual pre-trained language models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2039–2055, Abu Dhabi, Unite… [find](https://scholar.google.com/scholar?q=Da+Yin%2C+Hritik+Bansal%2C+Masoud+Monajatipoor%2C+Liunian+Harold+Li%2C+and+Kai%2DWei+Chang%2E+2022%2E+GeoM%2DLAMA%3A+Geo%2Ddiverse+commonsense+probing+on+multilingual+pre%2Dtrained+language+models%2E+In+Proceedings+of+the+20)
43. Dongkeun Yoon, Joel Jang, Sungdong Kim, Seungone Kim, Sheikh Shafayat, and Minjoon Seo. 2024. Lang-Bridge: Multilingual reasoning without multilingual supervision. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7502–7522, Ban… [find](https://scholar.google.com/scholar?q=Dongkeun+Yoon%2C+Joel+Jang%2C+Sungdong+Kim%2C+Seungone+Kim%2C+Sheikh+Shafayat%2C+and+Minjoon+Seo%2E+2024%2E+Lang%2DBridge%3A+Multilingual+reasoning+without+multilingual+supervision%2E+In+Proceedings+of+the+62nd+Annual+Me)
44. Hengyuan Zhang, Dawei Li, Yanran Li, Chenming Shang, Chufan Shi, and Yong Jiang. 2023a. Assisting language learners: Automated trans-lingual definition generation via contrastive prompt learning. arXiv preprint arXiv:2306.06058. [find](https://scholar.google.com/scholar?q=Hengyuan+Zhang%2C+Dawei+Li%2C+Yanran+Li%2C+Chenming+Shang%2C+Chufan+Shi%2C+and+Yong+Jiang%2E+2023a%2E+Assisting+language+learners%3A+Automated+trans%2Dlingual+definition+generation+via+contrastive+prompt+learning%2E+arXi)
45. Hengyuan Zhang, Dawei Li, Shiping Yang, and Yanran Li. 2022. Fine-grained contrastive learning for definition generation. [find](https://scholar.google.com/scholar?q=Hengyuan+Zhang%2C+Dawei+Li%2C+Shiping+Yang%2C+and+Yanran+Li%2E+2022%2E+Fine%2Dgrained+contrastive+learning+for+definition+generation%2E)
46. Hengyuan Zhang, Zitao Liu, Chenming Shang, Dawei Li, and Yong Jiang. 2025. A question-centric multi-experts contrastive learning framework for improving the accuracy and interpretability of deep sequential knowledge tracing models. ACM Transactions on Knowledge Discovery from Data, 19(2):1–25. [find](https://scholar.google.com/scholar?q=Hengyuan+Zhang%2C+Zitao+Liu%2C+Chenming+Shang%2C+Dawei+Li%2C+and+Yong+Jiang%2E+2025%2E+A+question%2Dcentric+multi%2Dexperts+contrastive+learning+framework+for+improving+the+accuracy+and+interpretability+of+deep+seque)
47. Hengyuan Zhang, Yanru Wu, Dawei Li, Sak Yang, Rui Zhao, Yong Jiang, and Fei Tan. 2024a. Balancing speciality and versatility: a coarse to fine framework for supervised fine-tuning large language model. In Findings of the Association for Computational Linguistics ACL 2024, pages 7467–7509. [find](https://scholar.google.com/scholar?q=Hengyuan+Zhang%2C+Yanru+Wu%2C+Dawei+Li%2C+Sak+Yang%2C+Rui+Zhao%2C+Yong+Jiang%2C+and+Fei+Tan%2E+2024a%2E+Balancing+speciality+and+versatility%3A+a+coarse+to+fine+framework+for+supervised+fine%2Dtuning+large+language+model)
48. Liang Zhang, Qin Jin, Haoyang Huang, Dongdong Zhang, and Furu Wei. 2024b. Respond in my language: Mitigating language inconsistency in response generation based on large language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers… [find](https://scholar.google.com/scholar?q=Liang+Zhang%2C+Qin+Jin%2C+Haoyang+Huang%2C+Dongdong+Zhang%2C+and+Furu+Wei%2E+2024b%2E+Respond+in+my+language%3A+Mitigating+language+inconsistency+in+response+generation+based+on+large+language+models%2E+In+Proceeding)
49. Miaoran Zhang, Vagrant Gautam, Mingyang Wang, Jesujoba Alabi, Xiaoyu Shen, Dietrich Klakow, and Marius Mosbach. 2024c. The impact of demonstrations on multilingual in-context learning: A multi-dimensional analysis. In Findings of the Association for Computational Linguistics ACL 2024, pages 7342–737… [find](https://scholar.google.com/scholar?q=Miaoran+Zhang%2C+Vagrant+Gautam%2C+Mingyang+Wang%2C+Jesujoba+Alabi%2C+Xiaoyu+Shen%2C+Dietrich+Klakow%2C+and+Marius+Mosbach%2E+2024c%2E+The+impact+of+demonstrations+on+multilingual+in%2Dcontext+learning%3A+A+multi%2Ddimensi)
50. Shaolei Zhang, Qingkai Fang, Zhuocheng Zhang, Zhengrui Ma, Yan Zhou, Langlin Huang, Mengyu Bu, Shangtong Gui, Yunji Chen, Xilin Chen, et al. 2023b. Bayling: Bridging cross-lingual alignment and instruction following through interactive translation for large language models. arXiv preprint arXiv:2306… [find](https://scholar.google.com/scholar?q=Shaolei+Zhang%2C+Qingkai+Fang%2C+Zhuocheng+Zhang%2C+Zhengrui+Ma%2C+Yan+Zhou%2C+Langlin+Huang%2C+Mengyu+Bu%2C+Shangtong+Gui%2C+Yunji+Chen%2C+Xilin+Chen%2C+et+al%2E+2023b%2E+Bayling%3A+Bridging+cross%2Dlingual+alignment+and+instru)
51. Xin Zhao, Naoki Yoshinaga, and Daisuke Oba. 2024. Tracing the roots of facts in multilingual language models: Independent, shared, and transferred knowledge. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages… [find](https://scholar.google.com/scholar?q=Xin+Zhao%2C+Naoki+Yoshinaga%2C+and+Daisuke+Oba%2E+2024%2E+Tracing+the+roots+of+facts+in+multilingual+language+models%3A+Independent%2C+shared%2C+and+transferred+knowledge%2E+In+Proceedings+of+the+18th+Conference+of+t)
52. Mu Zhu, Qingzhou Wu, Zhongli Bai, Yu Song, and Qiang Gao. 2024a. Eeg-eye movement based subject dependence, cross-subject, and cross-session emotion recognition with multidimensional homogeneous encoding space alignment. Expert Systems with Applications, 251:124001. [find](https://scholar.google.com/scholar?q=Mu+Zhu%2C+Qingzhou+Wu%2C+Zhongli+Bai%2C+Yu+Song%2C+and+Qiang+Gao%2E+2024a%2E+Eeg%2Deye+movement+based+subject+dependence%2C+cross%2Dsubject%2C+and+cross%2Dsession+emotion+recognition+with+multidimensional+homogeneous+encod)
53. Wenhao Zhu, Hongyi Liu, Qingxiu Dong, Jingjing Xu, Shujian Huang, Lingpeng Kong, Jiajun Chen, and Lei Li. 2024b. [Multilingual machine translation with large language models: Empirical results and analysis](https://md.research.elamur.ai/free/en/papers/zhu-2024-multilingual-machine-translation-large-language/). In Findings of the Association for Computational Linguistics: NAACL 2024.

*Updated 31 October 2026*