---
title: "Multilingual LLMs are Better Cross-lingual In-context Learners"
description: "They propose X-InSTA, a prompt construction strategy with two parts: Semantic alignment: it… 29 references, cited by 53 papers here."
authors: ["Eshaan Tanwar", "Subhabrata Dutta", "Manish Borthakur", "Tanmoy Chakraborty"]
year: 2023
venue: "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2023.acl-long.346/"
doi: "10.18653/v1/2023.acl-long.346"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/tanwar-2023-multilingual-llms-are-better-cross/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/tanwar-2023-multilingual-llms-are-better-cross/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/7/) / Multilingual LLMs are Better Cross-lingual In-context Learners with Alignment

# Multilingual LLMs are Better Cross-lingual In-context Learners with Alignment

**Authors:** Eshaan Tanwar, Subhabrata Dutta, Manish Borthakur and [Tanmoy Chakraborty](https://md.research.elamur.ai/free/en/authors/tanmoy-chakraborty/) · **Year:** 2023 · **Venue:** [Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2023](https://md.research.elamur.ai/free/en/venues/acl/2023/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2023.acl-long.346/) · [PDF](https://aclanthology.org/2023.acl-long.346.pdf) · [DOI 10.18653/v1/2023.acl-long.346](https://doi.org/10.18653/v1/2023.acl-long.346) · [OpenAlex](https://openalex.org/W4385572198)

**Citations:** 28 citations (OpenAlex, 2026-10-14) · [53](https://md.research.elamur.ai/free/en/papers/tanwar-2023-multilingual-llms-are-better-cross/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Large language models can perform in-context learning (ICL): they read a few labeled demonstrations in the prompt and predict a test label without any weight updates. This paper studies ICL in a cross-lingual setting, where the demonstrations come from a high-resource source language (English) and the test inputs are in another, often lower-resource, target language. The authors argue that the common practice of choosing random demonstrations works poorly here. Such prompts combine unrelated content with an abrupt language switch, so neither the inputs nor the label spaces line up.

They propose X-InSTA, a prompt construction strategy with two parts:
- **Semantic alignment:** it retrieves the source examples closest to the test input, using multilingual sentence-transformers embeddings and cosine similarity.
- **Task alignment:** it adds a manually written cue that maps source-language labels to their target-language equivalents.

Across three text classification tasks and 44 language pairs, X-InSTA clearly outperforms random selection. This matters for low-resource languages because label knowledge transfers better without new annotation or fine-tuning.

## Key points
- X-InSTA stands for Cross-lingual In-context Source-Target Alignment; the authors present it as the first prompt design for cross-lingual ICL that goes beyond random demonstration selection.
- Averaged over three classification tasks, X-InSTA gives about an 18% relative improvement over random prompt selection, with English as the source language. The tasks include sentiment on product reviews and hate-speech detection.
- Semantic alignment selects the top-k most similar source examples for each individual test input. This dynamic selection helps regardless of the language pair. Task alignment helps most for pairs whose label spaces are well aligned in the LLM's representations.
- Limitations noted by the authors:
  - The task aligners are static and hand-crafted.
  - Larger or commercial LLMs were not tested.
  - Cultural differences between languages still cause errors, which is especially sensitive for hate-speech classification.
- Code is released at github.com/EshaanT/X-InSTA.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Indian Institute of Technology Delhi](https://md.research.elamur.ai/free/en/universities/indian-institute-of-technology-delhi/) — University, New Delhi, India · 10 works · 915 citations
- [Delhi Technological University](https://md.research.elamur.ai/free/en/universities/delhi-technological-university/) — University, New Delhi, India · 2 works · 188 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 1,344 works · [🌍 Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) · 465 works

## Cite

Cite as: Eshaan Tanwar, Subhabrata Dutta, Manish Borthakur and Tanmoy Chakraborty. 2023. Multilingual LLMs are Better Cross-lingual In-context Learners with Alignment. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{tanwar2023multilingual,
  title = {{Multilingual LLMs are Better Cross-lingual In-context Learners with Alignment}},
  author = {Eshaan Tanwar, Subhabrata Dutta, Manish Borthakur and Tanmoy Chakraborty},
  booktitle = {Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2023},
  doi = {10.18653/v1/2023.acl-long.346},
  url = {https://doi.org/10.18653/v1/2023.acl-long.346}
}
```

- [← previous MasakhaNEWS: News Topic Classification for African languages](https://md.research.elamur.ai/free/en/papers/adelani-2023-masakhanews-news-topic-classification-african/)
- [next → Large Language Models Meet Harry Potter: A Dataset for Aligning Dialogue Agents with Characters](https://md.research.elamur.ai/free/en/papers/chen-2023-large-language-models-meet-harry/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/7/)

## References

29 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Sweta Agrawal, Chunting Zhou, Mike Lewis, Luke Zettlemoyer, and Marjan Ghazvininejad. 2022. [In-context examples selection for machine translation](https://md.research.elamur.ai/free/en/papers/agrawal-2023-context-examples-selection-machine-translation/).
2. Valerio Basile, Cristina Bosco, Elisabetta Fersini, Debora Nozza, Viviana Patti, Francisco Manuel Rangel Pardo, Paolo Rosso, and Manuela Sanguinetti. 2019. [SemEval-2019 task 5: Multilingual detection of hate speech against immigrants and women in Twitter](https://md.research.elamur.ai/free/en/papers/basile-2019-semeval-2019-task-5-multilingual/). In Proceedings of the 13th International Wor…
3. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems, 33:1877–1901.
4. Tyler A Chang, Zhuowen Tu, and Benjamin K Bergen. 2022. [The geometry of multilingual language model representations](https://md.research.elamur.ai/free/en/papers/chang-2022-geometry-multilingual-language-model-representations/). arXiv preprint arXiv:2205.10964.
5. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2022. [Palm: Scaling language modeling with pathways](https://md.research.elamur.ai/free/en/papers/chowdhery-2022-palm-scaling-language-modeling-pathways/). arXiv preprint arXiv:2204.02311.
6. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association fo… [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Kartikay+Khandelwal%2C+Naman+Goyal%2C+Vishrav+Chaudhary%2C+Guillaume+Wenzek%2C+Francisco+Guzm%C3%A1n%2C+Edouard+Grave%2C+Myle+Ott%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2020%2E+Unsupervised+cross%2Dlingu)
7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/).
8. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/).
9. Phillip Keung, Yichao Lu, György Szarvas, and Noah A. Smith. 2020. [The multilingual Amazon reviews corpus](https://md.research.elamur.ai/free/en/papers/keung-2020-multilingual-amazon-reviews-corpus/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4563–4568, Online. Association for Computational Linguistics.
10. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. [BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension](https://md.research.elamur.ai/free/en/papers/lewis-2020-bart-denoising-sequence-sequence-pre/). In Proceedings of the 58th Annual Meeting of the…
11. Yingcong Li, M Emrullah Ildiz, Dimitris Papailiopoulos, and Samet Oymak. 2023. Transformers as algorithms: Generalization and implicit model selection in in-context learning. arXiv preprint arXiv:2301.07067. — [arXiv:2301.07067](https://arxiv.org/abs/2301.07067) [find](https://scholar.google.com/scholar?q=Yingcong+Li%2C+M+Emrullah+Ildiz%2C+Dimitris+Papailiopoulos%2C+and+Samet+Oymak%2E+2023%2E+Transformers+as+algorithms%3A+Generalization+and+implicit+model+selection+in+in%2Dcontext+learning%2E+arXiv+preprint+arXiv%3A2301)
12. Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, Tianlu Wang, Shuohui Chen, Daniel Simig, Myle Ott, Naman Goyal, Shruti Bhosale, Jingfei Du, et al. 2021. [Few-shot learning with multilingual language models](https://md.research.elamur.ai/free/en/papers/lin-2021-few-shot-learning-multilingual-language/). arXiv preprint arXiv:2112.10668.
13. Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. 2022. [What makes good in-context examples for GPT-3?](https://md.research.elamur.ai/free/en/papers/liu-2022-what-makes-good-context-examples/) In Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, pages 100–114…
14. Yinhan Liu, Jiatao Gu, Naman Goyal, Xian Li, Sergey Edunov, Marjan Ghazvininejad, Mike Lewis, and Luke Zettlemoyer. 2020. Multilingual denoising pre-training for neural machine translation. [find](https://scholar.google.com/scholar?q=Yinhan+Liu%2C+Jiatao+Gu%2C+Naman+Goyal%2C+Xian+Li%2C+Sergey+Edunov%2C+Marjan+Ghazvininejad%2C+Mike+Lewis%2C+and+Luke+Zettlemoyer%2E+2020%2E+Multilingual+denoising+pre%2Dtraining+for+neural+machine+translation%2E)
15. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. [find](https://scholar.google.com/scholar?q=Yinhan+Liu%2C+Myle+Ott%2C+Naman+Goyal%2C+Jingfei+Du%2C+Mandar+Joshi%2C+Danqi+Chen%2C+Omer+Levy%2C+Mike+Lewis%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2019%2E+Roberta%3A+A+robustly+optimized+bert+pretraining+approach%2E)
16. Farhad Nooralahzadeh, Giannis Bekoulis, Johannes Bjerva, and Isabelle Augenstein. 2020. [Zero-shot cross-lingual transfer with meta learning](https://md.research.elamur.ai/free/en/papers/nooralahzadeh-2020-zero-shot-cross-lingual-transfer/). arXiv preprint arXiv:2003.02739.
17. Marinela Parović, Goran Glavaš, Ivan Vulić, and Anna Korhonen. 2022. BAD-X: Bilingual adapters improve zero-shot cross-lingual transfer. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1791–179… [find](https://scholar.google.com/scholar?q=Marinela+Parovic%CC%81%2C+Goran+Glava%C5%A1%2C+Ivan+Vulic%CC%81%2C+and+Anna+Korhonen%2E+2022%2E+BAD%2DX%3A+Bilingual+adapters+improve+zero%2Dshot+cross%2Dlingual+transfer%2E+In+Proceedings+of+the+2022+Conference+of+the+North+American+C)
18. Peter Prettenhofer and Benno Stein. 2010. [Cross-language text classification using structural correspondence learning](https://md.research.elamur.ai/free/en/papers/prettenhofer-2010-cross-language-text-classification-structural/). In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 1118–1127, Uppsala, Sweden. Association for Computational Linguistics.
19. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2019. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/).
20. Nils Reimers and Iryna Gurevych. 2020. [Making monolingual sentence embeddings multilingual using knowledge distillation](https://md.research.elamur.ai/free/en/papers/reimers-2020-making-monolingual-sentence-embeddings-multilingual/). arXiv preprint arXiv:2004.09813.
21. Peng Shi, Rui Zhang, He Bai, and Jimmy Lin. 2022. Xricl: Cross-lingual retrieval-augmented in-context learning for cross-lingual text-to-sql semantic parsing. arXiv preprint arXiv:2210.13693. [find](https://scholar.google.com/scholar?q=Peng+Shi%2C+Rui+Zhang%2C+He+Bai%2C+and+Jimmy+Lin%2E+2022%2E+Xricl%3A+Cross%2Dlingual+retrieval%2Daugmented+in%2Dcontext+learning+for+cross%2Dlingual+text%2Dto%2Dsql+semantic+parsing%2E+arXiv+preprint+arXiv%3A2210%2E13693%2E)
22. Johannes von Oswald, Eyvind Niklasson, Ettore Randazzo, João Sacramento, Alexander Mordvintsev, Andrey Zhmoginov, and Max Vladymyrov. 2022. [Transformers learn in-context by gradient descent](https://md.research.elamur.ai/free/en/papers/oswald-2022-transformers-learn-context-gradient-descent/). arXiv preprint arXiv:2212.07677.
23. Albert Webson and Ellie Pavlick. 2022. [Do prompt-based models really understand the meaning of their prompts?](https://md.research.elamur.ai/free/en/papers/webson-2022-do-prompt-based-models-really/) In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2300–2344, Seattle, United States. As…
24. Genta Indra Winata, Andrea Madotto, Zhaojiang Lin, Rosanne Liu, Jason Yosinski, and Pascale Fung. 2021. [Language models are few-shot multilingual learners](https://md.research.elamur.ai/free/en/papers/winata-2021-language-models-are-few-shot/). In Proceedings of the 1st Workshop on Multilingual Representation Learning, pages 1–15, Punta Cana, Dominican Republic. Association for Computat…
25. Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma. 2022. [An explanation of in-context learning as implicit bayesian inference](https://md.research.elamur.ai/free/en/papers/xie-2021-explanation-context-learning-implicit-bayesian/). In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net.
26. Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2020. [mt5: A massively multilingual pre-trained text-to-text transformer](https://md.research.elamur.ai/free/en/papers/xue-2021-mt5-massively-multilingual-pre-trained/).
27. Linting Xue, Noah Constant, Adam Roberts, Mihir Kale, Rami Al-Rfou, Aditya Siddhant, Aditya Barua, and Colin Raffel. 2021. [mT5: A massively multilingual pre-trained text-to-text transformer](https://md.research.elamur.ai/free/en/papers/xue-2021-mt5-massively-multilingual-pre-trained/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Ling…
28. Ningyu Zhang, Luoqiu Li, Xiang Chen, Shumin Deng, Zhen Bi, Chuanqi Tan, Fei Huang, and Huajun Chen. 2021. Differentiable prompt makes pre-trained language models better few-shot learners. arXiv preprint arXiv:2108.13161. — [arXiv:2108.13161](https://arxiv.org/abs/2108.13161) [find](https://scholar.google.com/scholar?q=Ningyu+Zhang%2C+Luoqiu+Li%2C+Xiang+Chen%2C+Shumin+Deng%2C+Zhen+Bi%2C+Chuanqi+Tan%2C+Fei+Huang%2C+and+Huajun+Chen%2E+2021%2E+Differentiable+prompt+makes+pre%2Dtrained+language+models+better+few%2Dshot+learners%2E+arXiv+prepri)
29. Tony Z. Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. [Calibrate before use: Improving few-shot performance of language models](https://md.research.elamur.ai/free/en/papers/zhao-2021-calibrate-before-use-improving-few/).

*Updated 24 October 2026*