---
title: "Code-Switching Information Retrieval: Benchmarks, Analysis,"
description: "This paper offers a systematic evaluation of retrieval under code-switching. 48 references."
authors: ["Qingcheng Zeng", "Yuheng Lu", "Zeqi Zhou", "Heli Qi", "Puxuan Yu", "Fuheng Zhao", "Hitomi Yanaka", "Weihao Xuan", "Naoto Yokoya"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.636/"
doi: "10.18653/v1/2026.findings-acl.636"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zeng-2026-code-switching-information-retrieval-benchmarks/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zeng-2026-code-switching-information-retrieval-benchmarks/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/11/) / Code-Switching Information Retrieval: Benchmarks, Analysis, and the Limits of Current Retrievers

# Code-Switching Information Retrieval: Benchmarks, Analysis, and the Limits of Current Retrievers

**Authors:** [Qingcheng Zeng](https://md.research.elamur.ai/free/en/authors/qingcheng-zeng/), [Yuheng Lu](https://md.research.elamur.ai/free/en/authors/yuheng-lu/), Zeqi Zhou, [Heli Qi](https://md.research.elamur.ai/free/en/authors/heli-qi/), [Puxuan Yu](https://md.research.elamur.ai/free/en/authors/puxuan-yu/), Fuheng Zhao, [Hitomi Yanaka](https://md.research.elamur.ai/free/en/authors/hitomi-yanaka/), [Weihao Xuan](https://md.research.elamur.ai/free/en/authors/weihao-xuan/) and [Naoto Yokoya](https://md.research.elamur.ai/free/en/authors/naoto-yokoya/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.636/) · [PDF](https://aclanthology.org/2026.findings-acl.636.pdf) · [DOI 10.18653/v1/2026.findings-acl.636](https://doi.org/10.18653/v1/2026.findings-acl.636) · [OpenAlex](https://openalex.org/W7166864223)

**Citations:** 1 citations (OpenAlex, 2026-10-14)

## Summary
Search engines and RAG pipelines are mostly built and tested on single-language queries, yet many bilingual users mix languages when they search. This paper offers a systematic evaluation of retrieval under code-switching. The authors built CSR-L, in which bilingual annotators rewrote English queries from four IR datasets into natural English–Chinese and English–Japanese mixtures, and a second annotator verified each rewrite. They tested lexical, bi-encoder, cross-encoder and late-interaction systems on CSR-L. Every family lost accuracy on mixed queries, including strong multilingual models. The losses coincided with a measurable gap between the embeddings of monolingual queries and their code-switched versions. To check whether the problem extends beyond retrieval, the authors created CS-MTEB, an LLM-assisted, MTEB-style suite of 11 tasks, where drops reached 27%. They also tried lexicon-based vocabulary expansion. It recovered part of the loss but did not restore monolingual performance. The authors conclude that code-switching is an open robustness problem that needs dedicated training data, not surface-level fixes.

## Key points
- **CSR-L** rewrites queries from Touché 2020, HumanEval, TRECCOVID and FollowIR. The Chinese version has 49, 158, 50 and 198 queries respectively. Scoring uses nDCG@10, except FollowIR, which uses p-MRR.
- **Models tested** include BM25, e5-large-v2, mE5-large, bge-m3, Arctic-Embed-m/l-v2.0, Qwen3-Embedding-0.6/4/8B, jina-reranker-v3, bge-reranker-v2-m3, Qwen3-Reranker and ColBERT v2. Cross-encoders scored the full corpus directly rather than reranking a first-stage candidate list.
- **CSR-L results (Chinese):** e5-large-v2 fell from 42.52 to 22.88 on Touché 2020 and its average fell from 47.22 to 35.32. Multilingual Qwen3-Embedding-8B was more stable, with its average dropping from 69.88 to 66.20.
- **CS-MTEB** covers 11 tasks across 7 task types. In the Japanese setting, Qwen3-Embedding-0.6B's reranking score fell from 63.09 to 37.33.
- **Vocabulary expansion** results (macro-average, before → after):

  | Model | CSR-L-Chinese | CSR-L-Japanese |
  |---|---|---|
  | all-MiniLM-L12-v2 | 30.09 → 37.73 | 29.94 → 34.34 |
  | e5-large-v2 | 35.32 → 43.50 | 34.70 → 39.80 |

  Most of the gains came from Touché 2020 and TRECCOVID.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Brown University](https://md.research.elamur.ai/free/en/universities/brown-university/) — University, Providence, United States · 76 works · 25,952 citations
- [The University of Tokyo](https://md.research.elamur.ai/free/en/universities/the-university-of-tokyo/) — University, Tokyo, Japan · 90 works · 9,156 citations
- [University of Utah](https://md.research.elamur.ai/free/en/universities/university-of-utah/) — University, Salt Lake City, United States · 38 works · 5,925 citations
- [Northwestern University](https://md.research.elamur.ai/free/en/universities/northwestern-university/) — University, Evanston, United States · 41 works · 2,194 citations
- [RIKEN Center for Advanced Intelligence Project](https://md.research.elamur.ai/free/en/labs/riken-center-for-advanced-intelligence-project/) — Research institute, Tokyo, Japan · 15 works · 957 citations
- [Waseda University](https://md.research.elamur.ai/free/en/universities/waseda-university/) — University, Tokyo, Japan · 20 works · 584 citations
- [RIKEN](https://md.research.elamur.ai/free/en/labs/riken/) — Research institute, Wako, Japan · 26 works · 526 citations
- [Snowflake Inc.](https://md.research.elamur.ai/free/en/companies/snowflake-inc/) — Company, Bozeman, United States · 3 works · 17 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works

🧭 Topics (2): [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,220 works · [🌍 Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) · 828 works

## Cite

Cite as: Qingcheng Zeng, Yuheng Lu, Zeqi Zhou, Heli Qi, Puxuan Yu, Fuheng Zhao, Hitomi Yanaka, Weihao Xuan and Naoto Yokoya. 2026. Code-Switching Information Retrieval: Benchmarks, Analysis, and the Limits of Current Retrievers. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zeng2026code,
  title = {{Code-Switching Information Retrieval: Benchmarks, Analysis, and the Limits of Current Retrievers}},
  author = {Qingcheng Zeng, Yuheng Lu, Zeqi Zhou, Heli Qi, Puxuan Yu, Fuheng Zhao, Hitomi Yanaka, Weihao Xuan and Naoto Yokoya},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.636},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.636}
}
```

- [← previous CoAct: Co-Active LLM Preference Learning with Human-AI Synergy](https://md.research.elamur.ai/free/en/papers/xu-2026-coact-co-active-llm-preference/)
- [next → CodeDet-NITS at SemEval-2026 Task 13: AI Code Authorship Detection Beyond Truncation](https://md.research.elamur.ai/free/en/papers/teja-2026-codedet-nits-semeval-2026-task/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/11/)

## References

48 references, 24 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Yusuf M. Ahmed. 2024. Code-switching in multilingual communities: Case studies from kenya, malaysia, and the UAE. Journal of International English Research Studies (JIERS), 2(4):13–21. [find](https://scholar.google.com/scholar?q=Yusuf+M%2E+Ahmed%2E+2024%2E+Code%2Dswitching+in+multilingual+communities%3A+Case+studies+from+kenya%2C+malaysia%2C+and+the+UAE%2E+Journal+of+International+English+Research+Studies+%28JIERS%29%2C+2%284%29%3A13%E2%80%9321%2E)
2. Alexander Bondarenko, Maik Fröbe, Meriem Beloucif, Lukas Gienapp, Yamen Ajjour, Alexander Panchenko, Chris Biemann, Benno Stein, Henning Wachsmuth, Martin Potthast, and Matthias Hagen. 2020. Overview of touché 2020: Argument retrieval. In Experimental IR Meets Multilinguality, Multimodality, and Int… [find](https://scholar.google.com/scholar?q=Alexander+Bondarenko%2C+Maik+Fr%C3%B6be%2C+Meriem+Beloucif%2C+Lukas+Gienapp%2C+Yamen+Ajjour%2C+Alexander+Panchenko%2C+Chris+Biemann%2C+Benno+Stein%2C+Henning+Wachsmuth%2C+Martin+Potthast%2C+and+Matthias+Hagen%2E+2020%2E+Overview+)
3. Supriya Chanda and Sukomal Pal. 2025. Overview of the shared task on code-mixed information retrieval from social media data. In Proceedings of the 16th Annual Meeting of the Forum for Information Retrieval Evaluation, FIRE ’24, page 29–31, New York, NY, USA. Association for Computing Machinery. [find](https://scholar.google.com/scholar?q=Supriya+Chanda+and+Sukomal+Pal%2E+2025%2E+Overview+of+the+shared+task+on+code%2Dmixed+information+retrieval+from+social+media+data%2E+In+Proceedings+of+the+16th+Annual+Meeting+of+the+Forum+for+Information+Ret)
4. Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. [Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation](https://md.research.elamur.ai/free/en/papers/chen-2024-m3-embedding-multi-linguality/). Preprint, arXiv:2402.03216.
5. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, and 39 others. 202… — [Evaluating Large Language Models Trained on Code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
6. Alexis Conneau, Guillaume Lample, Marc’Aurelio Ranzato, Ludovic Denoyer, and Hervé Jégou. 2018. Word translation without parallel data. Preprint, arXiv:1710.04087. [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Guillaume+Lample%2C+Marc%E2%80%99Aurelio+Ranzato%2C+Ludovic+Denoyer%2C+and+Herv%C3%A9+J%C3%A9gou%2E+2018%2E+Word+translation+without+parallel+data%2E+Preprint%2C+arXiv%3A1710%2E04087%2E)
7. Core Team, Bangjun Xiao, Bingquan Xia, Bo Yang, Bofei Gao, Bowen Shen, Chen Zhang, Chenhong He, Chiheng Lou, Fuli Luo, Gang Wang, Gang Xie, Hailin Zhang, Hanglong Lv, Hanyu Li, Heyu Chen, Hongshen Xu, Houbin Zhang, Huaqiu Liu, and 107 others. 2026. [Mimo-v2-flash technical report](https://md.research.elamur.ai/free/en/papers/team-2026-llm-mimo-v2-flash-technical/). Preprint, arXiv:260…
8. Thomas Diggelmann, Jordan Boyd-Graber, Jannis Bulian, Massimiliano Ciaramita, and Markus Leippold. 2021. [Climate-fever: A dataset for verification of real-world climate claims](https://md.research.elamur.ai/free/en/papers/diggelmann-2020-climate-fever-dataset-verification-real/). Preprint, arXiv:2012.00614.
9. Junggeun Do, Jaeseong Lee, and Seung-won Hwang. 2024. ContrastiveMix: Overcoming code-mixing dilemma in cross-lingual transfer for information retrieval. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies… [find](https://scholar.google.com/scholar?q=Junggeun+Do%2C+Jaeseong+Lee%2C+and+Seung%2Dwon+Hwang%2E+2024%2E+ContrastiveMix%3A+Overcoming+code%2Dmixing+dilemma+in+cross%2Dlingual+transfer+for+information+retrieval%2E+In+Proceedings+of+the+2024+Conference+of+the+N)
10. Kenneth Enevoldsen, Isaac Chung, Imene Kerboua, Márton Kardos, Ashwin Mathur, David Stap, Jay Gala, Wissam Siblini, Dominik Krzemiński, Genta Indra Winata, Saba Sturua, Saiteja Utpala, Mathieu Ciancone, Marion Schaeffer, Gabriel Sequeira, Diganta Misra, Shreeya Dhakal, Jonathan Rystrøm, Roman Solom… — [MMTEB: Massive Multilingual Text Embedding Benchmark](https://md.research.elamur.ai/free/en/papers/enevoldsen-2025-mmteb-massive-multilingual-text-embedding/)
11. Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. [SimCSE: Simple contrastive learning of sentence embeddings](https://md.research.elamur.ai/free/en/papers/gao-2021-simcse-simple-contrastive-learning-sentence/). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6894–6910, Online and Punta Cana, Dominican Republic. Association for Computational Linguistic…
12. Parth Gupta, Kalika Bali, Rafael E. Banchs, Monojit Choudhury, and Paolo Rosso. 2014. Query expansion for mixed-script information retrieval. In Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR ’14, page 677–686, New York, NY, USA.… [find](https://scholar.google.com/scholar?q=Parth+Gupta%2C+Kalika+Bali%2C+Rafael+E%2E+Banchs%2C+Monojit+Choudhury%2C+and+Paolo+Rosso%2E+2014%2E+Query+expansion+for+mixed%2Dscript+information+retrieval%2E+In+Proceedings+of+the+37th+International+ACM+SIGIR+Confere)
13. I-Hung Hsu, Avik Ray, Shubham Garg, Nanyun Peng, and Jing Huang. 2023. Code-switched text synthesis in unseen language pairs. In Findings of the Association for Computational Linguistics: ACL 2023, pages 5137–5151, Toronto, Canada. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=I%2DHung+Hsu%2C+Avik+Ray%2C+Shubham+Garg%2C+Nanyun+Peng%2C+and+Jing+Huang%2E+2023%2E+Code%2Dswitched+text+synthesis+in+unseen+language+pairs%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL+2023%2C+pa)
14. Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. 2025. [Search-r1: Training llms to reason and leverage search engines with reinforcement learning](https://md.research.elamur.ai/free/en/papers/jin-2025-search-r1-training-llms-reason/). Preprint, arXiv:2503.09516.
15. Omar Khattab and Matei Zaharia. 2020. [Colbert: Efficient and effective passage search via contextualized late interaction over bert](https://md.research.elamur.ai/free/en/papers/khattab-2020-colbert-efficient-effective-passage/). In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’20, page 39–48, New York, NY, USA. Associati…
16. Jonghwi Kim, Deokhyung Kang, Seonjeong Hwang, Yunsu Kim, Jungseul Ok, and Gary Lee. 2025. MiLQ: Benchmarking IR models for bilingual web search with mixed language queries. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 22643–22659, Suzhou, China. As… [find](https://scholar.google.com/scholar?q=Jonghwi+Kim%2C+Deokhyung+Kang%2C+Seonjeong+Hwang%2C+Yunsu+Kim%2C+Jungseul+Ok%2C+and+Gary+Lee%2E+2025%2E+MiLQ%3A+Benchmarking+IR+models+for+bilingual+web+search+with+mixed+language+queries%2E+In+Proceedings+of+the+2025+)
17. Tao Lei, Hrishikesh Joshi, Regina Barzilay, Tommi Jaakkola, Kateryna Tymoshenko, Alessandro Moschitti, and Lluís Màrquez. 2016. Semi-supervised question retrieval with gated convolutions. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguis… [find](https://scholar.google.com/scholar?q=Tao+Lei%2C+Hrishikesh+Joshi%2C+Regina+Barzilay%2C+Tommi+Jaakkola%2C+Kateryna+Tymoshenko%2C+Alessandro+Moschitti%2C+and+Llu%C3%ADs+M%C3%A0rquez%2E+2016%2E+Semi%2Dsupervised+question+retrieval+with+gated+convolutions%2E+In+Proceedin)
18. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2021. [Retrieval-augmented generation for knowledge-intensive nlp tasks](https://md.research.elamur.ai/free/en/papers/lewis-2020-retrieval-augmented-generation-knowledge-intensive/). Preprint, arXiv:2005.11401.
19. Wei Li. 2007. The bilingualism reader / edited by li wei. [find](https://scholar.google.com/scholar?q=Wei+Li%2E+2007%2E+The+bilingualism+reader+%2F+edited+by+li+wei%2E)
20. Robert Litschko, Ekaterina Artemova, and Barbara Plank. 2023. Boosting zero-shot cross-lingual retrieval by training on artificially code-switched data. In Findings of the Association for Computational Linguistics: ACL 2023, pages 3096–3108, Toronto, Canada. Association for Computational Linguistics… [find](https://scholar.google.com/scholar?q=Robert+Litschko%2C+Ekaterina+Artemova%2C+and+Barbara+Plank%2E+2023%2E+Boosting+zero%2Dshot+cross%2Dlingual+retrieval+by+training+on+artificially+code%2Dswitched+data%2E+In+Findings+of+the+Association+for+Computationa)
21. Robert Litschko, Oliver Kraus, Verena Blaschke, and Barbara Plank. 2025. Cross-dialect information retrieval: Information access in low-resource and high-variance languages. In Proceedings of the 31st International Conference on Computational Linguistics, pages 10158–10171, Abu Dhabi, UAE. Associati… [find](https://scholar.google.com/scholar?q=Robert+Litschko%2C+Oliver+Kraus%2C+Verena+Blaschke%2C+and+Barbara+Plank%2E+2025%2E+Cross%2Ddialect+information+retrieval%3A+Information+access+in+low%2Dresource+and+high%2Dvariance+languages%2E+In+Proceedings+of+the+31st)
22. Niklas Muennighoff, Nouamane Tazi, Loïc Magne, and Nils Reimers. 2023. [Mteb: Massive text embedding benchmark](https://md.research.elamur.ai/free/en/papers/muennighoff-2022-mteb-massive-text-embedding-benchmark/). Preprint, arXiv:2210.07316.
23. Carol Myers-Scotton. 1997. Duelling languages: Grammatical structure in codeswitching. Oxford University Press. [find](https://scholar.google.com/scholar?q=Carol+Myers%2DScotton%2E+1997%2E+Duelling+languages%3A+Grammatical+structure+in+codeswitching%2E+Oxford+University+Press%2E)
24. Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. [Representation learning with contrastive predictive coding](https://md.research.elamur.ai/free/en/papers/oord-2018-representation-learning-contrastive-predictive-coding/). arXiv preprint arXiv:1807.03748.
25. Shana Poplack. 2020. Sometimes i’ll start a sentence in spanish y termino en español: toward a typology of code-switching. In The bilingualism reader, pages 213–243. Routledge. [find](https://scholar.google.com/scholar?q=Shana+Poplack%2E+2020%2E+Sometimes+i%E2%80%99ll+start+a+sentence+in+spanish+y+termino+en+espa%C3%B1ol%3A+toward+a+typology+of+code%2Dswitching%2E+In+The+bilingualism+reader%2C+pages+213%E2%80%93243%2E+Routledge%2E)
26. Adithya Pratapa, Gayatri Bhat, Monojit Choudhury, Sunayana Sitaram, Sandipan Dandapat, and Kalika Bali. 2018. [Language modeling for code-mixing: The role of linguistic theory based synthetic data](https://md.research.elamur.ai/free/en/papers/pratapa-2018-language-modeling-code-mixing-role/). In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: L…
27. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, and 1 others. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+Ilya+Sutskever%2C+and+1+others%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E+OpenAI+blog%2C+1%288%29%3A9%2E)
28. Nils Reimers and Iryna Gurevych. 2019. [Sentence-BERT: Sentence embeddings using Siamese BERT-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,…
29. Kirk Roberts, Tasmeer Alam, Steven Bedrick, Dina Demner-Fushman, Kyle Lo, Ian Soboroff, Ellen Voorhees, Lucy Lu Wang, and William R Hersh. 2021. Searching for scientific evidence in a pandemic: An overview of trec-covid. Preprint, arXiv:2104.09632. — [arXiv:2104.09632](https://arxiv.org/abs/2104.09632) [find](https://scholar.google.com/scholar?q=Kirk+Roberts%2C+Tasmeer+Alam%2C+Steven+Bedrick%2C+Dina+Demner%2DFushman%2C+Kyle+Lo%2C+Ian+Soboroff%2C+Ellen+Voorhees%2C+Lucy+Lu+Wang%2C+and+William+R+Hersh%2E+2021%2E+Searching+for+scientific+evidence+in+a+pandemic%3A+An+ove)
30. Stephen Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: Bm25 and beyond. Found. Trends Inf. Retr., 3(4):333–389. [find](https://scholar.google.com/scholar?q=Stephen+Robertson+and+Hugo+Zaragoza%2E+2009%2E+The+probabilistic+relevance+framework%3A+Bm25+and+beyond%2E+Found%2E+Trends+Inf%2E+Retr%2E%2C+3%284%29%3A333%E2%80%93389%2E)
31. Andrew Rosenberg and Julia Hirschberg. 2007. [V-measure: A conditional entropy-based external cluster evaluation measure](https://md.research.elamur.ai/free/en/papers/rosenberg-2007-vmeasure-conditional-entropy-cluster-evaluation/). In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL), pages 410– 420, Prague,…
32. Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, and Matei Zaharia. 2022. [Colbertv2: Effective and efficient retrieval via lightweight late interaction](https://md.research.elamur.ai/free/en/papers/santhanam-2022-colbertv2-effective-efficient-retrieval-lightweight/). Preprint, arXiv:2112.01488.
33. Hongjin Su, Howard Yen, Mengzhou Xia, Weijia Shi, Niklas Muennighoff, Han yu Wang, Haisu Liu, Quan Shi, Zachary S. Siegel, Michael Tang, Ruoxi Sun, Jinsung Yoon, Sercan O. Arik, Danqi Chen, and Tao Yu. 2025. [Bright: A realistic and challenging benchmark for reasoning-intensive retrieval](https://md.research.elamur.ai/free/en/papers/su-2024-bright-realistic-challenging-benchmark-reasoning/). Preprint, a…
34. Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. [Beir: A heterogenous benchmark for zero-shot evaluation of information retrieval models](https://md.research.elamur.ai/free/en/papers/thakur-2021-beir-heterogenous-benchmark-zero/). Preprint, arXiv:2104.08663.
35. Henning Wachsmuth, Shahbaz Syed, and Benno Stein. 2018. [Retrieval of the best counterargument without prior topic knowledge](https://md.research.elamur.ai/free/en/papers/wachsmuth-2018-retrieval-best-counterargument-without-prior/). In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 241–251, Melbourne, Australia. Association for Comput…
36. Feng Wang, Yuqing Li, and Han Xiao. 2025. jina-reranker-v3: Last but not late interaction for listwise document reranking. Preprint, arXiv:2509.25085. — [arXiv:2509.25085](https://arxiv.org/abs/2509.25085) [find](https://scholar.google.com/scholar?q=Feng+Wang%2C+Yuqing+Li%2C+and+Han+Xiao%2E+2025%2E+jina%2Dreranker%2Dv3%3A+Last+but+not+late+interaction+for+listwise+document+reranking%2E+Preprint%2C+arXiv%3A2509%2E25085%2E+%E2%80%94+%5BarXiv%3A2509%2E25085%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2509%2E2)
37. Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2024a. [Text embeddings by weakly-supervised contrastive pre-training](https://md.research.elamur.ai/free/en/papers/wang-2022-text-embeddings-weakly-supervised-contrastive/). Preprint, arXiv:2212.03533.
38. Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024b. [Multilingual e5 text embeddings: A technical report](https://md.research.elamur.ai/free/en/papers/wang-2024-multilingual-e5-text-embeddings-technical/). arXiv preprint arXiv:2402.05672.
39. Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 11926–11942, Albuquerque, New Mexico. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Association+for+Computational+Linguistics%3A+Human+Language+Technologies+%28Volume+1%3A+Long+Papers%29%2C+pages+11926%E2%80%9311942%2C+Albuquerque%2C+New+Mexico%2E+Association+for+Computational+Linguistics%2E)
40. Genta Indra Winata, Ruochen Zhang, and David Ifeoluwa Adelani. 2024. MINERS: Multilingual language models as semantic retrievers. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 2742–2766, Miami, Florida, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Genta+Indra+Winata%2C+Ruochen+Zhang%2C+and+David+Ifeoluwa+Adelani%2E+2024%2E+MINERS%3A+Multilingual+language+models+as+semantic+retrievers%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+20)
41. Wei Xu, Chris Callison-Burch, and Bill Dolan. 2015. [SemEval-2015 task 1: Paraphrase and semantic similarity in Twitter (PIT)](https://md.research.elamur.ai/free/en/papers/xu-2015-semeval-2015-task-1-paraphrase/). In Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pages 1–11, Denver, Colorado. Association for Computational Linguistics.
42. Puxuan Yu, Luke Merrick, Gaurav Nuti, and Daniel Campos. 2024. Arctic-embed 2.0: Multilingual retrieval without compromise. Preprint, arXiv:2412.04506. — [arXiv:2412.04506](https://arxiv.org/abs/2412.04506) [find](https://scholar.google.com/scholar?q=Puxuan+Yu%2C+Luke+Merrick%2C+Gaurav+Nuti%2C+and+Daniel+Campos%2E+2024%2E+Arctic%2Dembed+2%2E0%3A+Multilingual+retrieval+without+compromise%2E+Preprint%2C+arXiv%3A2412%2E04506%2E+%E2%80%94+%5BarXiv%3A2412%2E04506%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2412%2E)
43. Qingcheng Zeng, Lucas Garay, Peilin Zhou, Dading Chong, Yining Hua, Jiageng Wu, Yikang Pan, Han Zhou, Rob Voigt, and Jie Yang. 2023. Greenplm: cross-lingual transfer of monolingual pre-trained language models at almost no cost. In Proceedings of the Thirty-Second International Joint Conference on Ar… [find](https://scholar.google.com/scholar?q=Qingcheng+Zeng%2C+Lucas+Garay%2C+Peilin+Zhou%2C+Dading+Chong%2C+Yining+Hua%2C+Jiageng+Wu%2C+Yikang+Pan%2C+Han+Zhou%2C+Rob+Voigt%2C+and+Jie+Yang%2E+2023%2E+Greenplm%3A+cross%2Dlingual+transfer+of+monolingual+pre%2Dtrained+languag)
44. Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou. 2025. [Qwen3 embedding: Advancing text embedding and reranking through foundation models](https://md.research.elamur.ai/free/en/papers/zhang-2025-qwen3-embedding-advancing-text/). Preprint, arXiv:2506.05176.
45. Fuheng Zhao, Jiayue Chen, Yiming Pan, Tahseen Rabbani, Divyakant Agrawal, and Amr El Abbadi. 2025. Access paths for efficient ordering with large language models. arXiv preprint arXiv:2509.00303. — [arXiv:2509.00303](https://arxiv.org/abs/2509.00303) [find](https://scholar.google.com/scholar?q=Fuheng+Zhao%2C+Jiayue+Chen%2C+Yiming+Pan%2C+Tahseen+Rabbani%2C+Divyakant+Agrawal%2C+and+Amr+El+Abbadi%2E+2025%2E+Access+paths+for+efficient+ordering+with+large+language+models%2E+arXiv+preprint+arXiv%3A2509%2E00303%2E+%E2%80%94+%5Ba)
46. Longfei Zuo, Pingjun Hong, Oliver Kraus, Barbara Plank, and Robert Litschko. 2025. Evaluating large language models for cross-lingual retrieval. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 11415–11429, Suzhou, China. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Longfei+Zuo%2C+Pingjun+Hong%2C+Oliver+Kraus%2C+Barbara+Plank%2C+and+Robert+Litschko%2E+2025%2E+Evaluating+large+language+models+for+cross%2Dlingual+retrieval%2E+In+Findings+of+the+Association+for+Computational+Lingui)
47. Xinyi Wang, Sebastian Ruder, and Graham Neubig. 2022. [Expanding pretrained models to thousands more languages via lexicon-based adaptation](https://md.research.elamur.ai/free/en/papers/wang-2022-expanding-pretrained-models-thousands-more/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 863–877, Dublin, Ireland. Association…
48. Orion Weller, Benjamin Chang, Sean MacAvaney, Kyle Lo, Arman Cohan, Benjamin Van Durme, Dawn Lawrie, and Luca Soldaini. 2025. [FollowIR: Evaluating and teaching information retrieval models to follow instructions](https://md.research.elamur.ai/free/en/papers/weller-2025-followir-evaluating-teaching-information-retrieval/). In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the

*Updated 1 November 2026*