---
title: "The Russian-focused embedders' exploration: ruMTEB"
description: "They release ru-en-RoSBERTa, a Russian-centred encoder also adapted to English so it can draw on… 67 references, cited by 13 papers here."
authors: ["Artem Snegirev", "Maria Tikhonova", "Maksimova Anna", "Alena Fenogenova", "Aleksandr Abramov"]
year: 2025
venue: "Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)"
source: "https://aclanthology.org/2025.naacl-long.12/"
doi: "10.18653/v1/2025.naacl-long.12"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/snegirev-2025-russian-focused-embedders-exploration-rumteb/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/snegirev-2025-russian-focused-embedders-exploration-rumteb/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/5/) / The Russian-focused embedders' exploration: ruMTEB benchmark and Russian embedding model design

# The Russian-focused embedders' exploration: ruMTEB benchmark and Russian embedding model design

**Authors:** [Artem Snegirev](https://md.research.elamur.ai/free/en/authors/artem-snegirev/), [Maria Tikhonova](https://md.research.elamur.ai/free/en/authors/maria-tikhonova/), Maksimova Anna, [Alena Fenogenova](https://md.research.elamur.ai/free/en/authors/alena-fenogenova/) and [Aleksandr Abramov](https://md.research.elamur.ai/free/en/authors/aleksandr-abramov/) · **Year:** 2025 · **Venue:** [Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/naacl/) ([NAACL 2025](https://md.research.elamur.ai/free/en/venues/naacl/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.naacl-long.12/) · [PDF](https://aclanthology.org/2025.naacl-long.12.pdf) · [DOI 10.18653/v1/2025.naacl-long.12](https://doi.org/10.18653/v1/2025.naacl-long.12) · [OpenAlex](https://openalex.org/W4411119637)

**Citations:** 6 citations (OpenAlex, 2026-10-14) · [13](https://md.research.elamur.ai/free/en/papers/snegirev-2025-russian-focused-embedders-exploration-rumteb/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary

Russian has lacked modern, dedicated text embedding models and a broad evaluation suite. Popular Russian encoders such as rubert-tiny2 and the SBERT large-nlu-ru variants are several years old, monolingual, and trained on dated corpora. The only Russian embedding benchmark, enkodechka, is small and has no retrieval tasks. The authors from SaluteDevices tackle both gaps. They release ru-en-RoSBERTa, a Russian-centred encoder also adapted to English so it can draw on cross-lingual knowledge transfer. They also build ruMTEB, a Russian extension of MTEB with 23 tasks in seven categories. Seventeen of these datasets are newly assembled from established Russian community resources, which were cleaned, deduplicated, spot-checked by hand and converted to MTEB format. The remaining six come from Russian subsets already present in multilingual MTEB. Across this suite, the new model performs comparably to leading multilingual and Russian embedders. The benchmark proves demanding enough to separate modern models. Code, datasets, the model and a public leaderboard integrated into MTEB are openly available, giving the community a shared standard for Russian embedding research.

## Key points

- ruMTEB covers Classification (9 datasets), Clustering (3), MultiLabel Classification (2), Pair Classification (1), Reranking (2), Retrieval (3) and STS (3).
- The metrics follow MTEB conventions:
  - Retrieval uses nDCG@10, as does MIRACLReranking.
  - Other reranking tasks use MAP@10.
  - STS uses Spearman correlation.
  - Clustering uses v-measure with K-means.
  - Classification uses few-shot logistic regression scored by accuracy.
- The six reused tasks include MassiveIntentClassification, MassiveScenarioClassification, MIRACLRetrieval, MIRACLReranking, RuParaphraserSTS and STS22.
- ru-en-RoSBERTa has a 512-token context. Contrastive pre-training was skipped because of compute limits, and the model was not evaluated on English.
- A full ruMTEB run for ru-en-RoSBERTa takes about 19 hours on one A100 80GB GPU. Training emitted roughly 3.66k kg of CO2.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,220 works

## Cite

Cite as: Artem Snegirev, Maria Tikhonova, Maksimova Anna, Alena Fenogenova and Aleksandr Abramov. 2025. The Russian-focused embedders' exploration: ruMTEB benchmark and Russian embedding model design. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{snegirev2025russian,
  title = {{The Russian-focused embedders' exploration: ruMTEB benchmark and Russian embedding model design}},
  author = {Artem Snegirev, Maria Tikhonova, Maksimova Anna, Alena Fenogenova and Aleksandr Abramov},
  booktitle = {Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.naacl-long.12},
  url = {https://doi.org/10.18653/v1/2025.naacl-long.12}
}
```

- [← previous Synthesizing Post-Training Data for LLMs through Multi-Agent Simulation](https://md.research.elamur.ai/free/en/papers/tang-2025-synthesizing-post-training-data-llms/)
- [next → Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chains](https://md.research.elamur.ai/free/en/papers/tan-2025-think-silently-think-fast-dynamic/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/5/)

## References

67 references, 34 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Eneko Agirre, Carmen Banea, Daniel Cer, Mona Diab, Aitor Gonzalez Agirre, Rada Mihalcea, German Rigau Claramunt, and Janyce Wiebe. 2016. [Semeval-2016 task 1: Semantic textual similarity, monolingual and cross-lingual evaluation](https://md.research.elamur.ai/free/en/papers/agirre-2016-semeval-2016-task-1-semantic/). In SemEval-2016. 10th International Workshop on Semantic Evaluation; 20…
2. Nikolay Babakov, Varvara Logacheva, Olga Kozlova, Nikita Semenov, and Alexander Panchenko. 2021. Detecting inappropriate messages on sensitive topics that could harm a company’s reputation. In Proceedings of the 8th Workshop on Balto-Slavic Natural Language Processing, pages 26–36, Kiyv, Ukraine. As… [find](https://scholar.google.com/scholar?q=Nikolay+Babakov%2C+Varvara+Logacheva%2C+Olga+Kozlova%2C+Nikita+Semenov%2C+and+Alexander+Panchenko%2E+2021%2E+Detecting+inappropriate+messages+on+sensitive+topics+that+could+harm+a+company%E2%80%99s+reputation%2E+In+Proceed)
3. Marta Bañón, Pinzhen Chen, Barry Haddow, Kenneth Heafield, Hieu Hoang, Miquel Esplà-Gomis, Mikel L. Forcada, Amir Kamran, Faheem Kirefu, Philipp Koehn, Sergio Ortiz Rojas, Leopoldo Pla Sempere, Gema Ramírez-Sánchez, Elsa Sarrías, Marek Strelec, Brian Thompson, William Waites, Dion Wiggins, and Jaume… — [ParaCrawl: Web-Scale Acquisition of Parallel Corpora](https://md.research.elamur.ai/free/en/papers/banon-2020-paracrawl-web-scale-acquisition-parallel/)
4. PD Blinov, Maria Klekovkina, Eugeny Kotelnikov, and Oleg Pestov. 2013. Research of lexical approach and machine learning methods for sentiment analysis. Computational Linguistics and Intellectual Technologies, 2(12):48–58. [find](https://scholar.google.com/scholar?q=PD+Blinov%2C+Maria+Klekovkina%2C+Eugeny+Kotelnikov%2C+and+Oleg+Pestov%2E+2013%2E+Research+of+lexical+approach+and+machine+learning+methods+for+sentiment+analysis%2E+Computational+Linguistics+and+Intellectual+Tech)
5. Samuel Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. 2015. [A large annotated corpus for learning natural language inference](https://md.research.elamur.ai/free/en/papers/bowman-2015-large-annotated-corpus-learning-natural/). In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632– 642.
6. Daniel Cer, Mona Diab, Eneko Agirre, Inigo Lopez-Gazpio, and Lucia Specia. 2017. [Semeval-2017 task 1: Semantic textual similarity-multilingual and cross-lingual focused evaluation](https://md.research.elamur.ai/free/en/papers/cer-2017-semeval-2017-task-1-semantic/). arXiv preprint arXiv:1708.00055.
7. Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. arXiv e-prints, pages arXiv–2402. [find](https://scholar.google.com/scholar?q=Jianlv+Chen%2C+Shitao+Xiao%2C+Peitian+Zhang%2C+Kun+Luo%2C+Defu+Lian%2C+and+Zheng+Liu%2E+2024%2E+Bge+m3%2Dembedding%3A+Multi%2Dlingual%2C+multi%2Dfunctionality%2C+multi%2Dgranularity+text+embeddings+through+self%2Dknowledge+distill)
8. Xi Chen, Ali Zeynali, Chico Q Camargo, Fabian Flöck, Devin Gaffney, Przemyslaw A Grabowicz, Scott A Hale, David Jurgens, and Mattia Samory. 2022. Semeval-2022 task 8: Multilingual news article similarity. [find](https://scholar.google.com/scholar?q=Xi+Chen%2C+Ali+Zeynali%2C+Chico+Q+Camargo%2C+Fabian+Fl%C3%B6ck%2C+Devin+Gaffney%2C+Przemyslaw+A+Grabowicz%2C+Scott+A+Hale%2C+David+Jurgens%2C+and+Mattia+Samory%2E+2022%2E+Semeval%2D2022+task+8%3A+Multilingual+news+article+similar)
9. Yihao Chen, Xianbiao Qi, Jianan Wang, and Lei Zhang. 2023. Disco-clip: A distributed contrastive loss for memory efficient clip training. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). [find](https://scholar.google.com/scholar?q=Yihao+Chen%2C+Xianbiao+Qi%2C+Jianan+Wang%2C+and+Lei+Zhang%2E+2023%2E+Disco%2Dclip%3A+A+distributed+contrastive+loss+for+memory+efficient+clip+training%2E+Proceedings+of+the+IEEE%2FCVF+Conference+on+Computer+Vision+and+)
10. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Unsupervised cross-lingual representation learning at scale. arXiv preprint arXiv:1911.02116. [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Kartikay+Khandelwal%2C+Naman+Goyal%2C+Vishrav+Chaudhary%2C+Guillaume+Wenzek%2C+Francisco+Guzm%C3%A1n%2C+Edouard+Grave%2C+Myle+Ott%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2019%2E+Unsupervised+cross%2Dlingu)
11. Alexis Conneau and Douwe Kiela. 2018. [Senteval: An evaluation toolkit for universal sentence representations](https://md.research.elamur.ai/free/en/papers/conneau-2018-senteval-evaluation-toolkit-universal-sentence/). In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018).
12. Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel R. Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. Xnli: Evaluating cross-lingual sentence representations. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computation… [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Ruty+Rinott%2C+Guillaume+Lample%2C+Adina+Williams%2C+Samuel+R%2E+Bowman%2C+Holger+Schwenk%2C+and+Veselin+Stoyanov%2E+2018%2E+Xnli%3A+Evaluating+cross%2Dlingual+sentence+representations%2E+In+Proceedings+of+)
13. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). arXiv preprint arXiv:1810.04805.
14. Pavel Efimov, Andrey Chertok, Leonid Boytsov, and Pavel Braslavski. 2020. Sberquad – russian reading comprehension dataset: Description and analysis. In Experimental IR Meets Multilinguality, Multimodality, and Interaction, pages 3–15. Springer International Publishing. [find](https://scholar.google.com/scholar?q=Pavel+Efimov%2C+Andrey+Chertok%2C+Leonid+Boytsov%2C+and+Pavel+Braslavski%2E+2020%2E+Sberquad+%E2%80%93+russian+reading+comprehension+dataset%3A+Description+and+analysis%2E+In+Experimental+IR+Meets+Multilinguality%2C+Multimod)
15. Kenneth Enevoldsen, Márton Kardos, Niklas Muennighoff, and Kristoffer Laigaard Nielbo. 2024. The scandinavian embedding benchmarks: Comprehensive assessment of multilingual and monolingual text embedding. arXiv preprint arXiv:2406.02396. [find](https://scholar.google.com/scholar?q=Kenneth+Enevoldsen%2C+M%C3%A1rton+Kardos%2C+Niklas+Muennighoff%2C+and+Kristoffer+Laigaard+Nielbo%2E+2024%2E+The+scandinavian+embedding+benchmarks%3A+Comprehensive+assessment+of+multilingual+and+monolingual+text+embedd)
16. Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2022. [Language-agnostic bert sentence embedding](https://md.research.elamur.ai/free/en/papers/feng-2020-language-agnostic-bert-sentence-embedding/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 878–891.
17. Daniil Gavrilov, Pavel Kalaidin, and Valentin Malykh. 2019. Self-attentive model for headline generation. In Proceedings of the 41st European Conference on Information Retrieval. [find](https://scholar.google.com/scholar?q=Daniil+Gavrilov%2C+Pavel+Kalaidin%2C+and+Valentin+Malykh%2E+2019%2E+Self%2Dattentive+model+for+headline+generation%2E+In+Proceedings+of+the+41st+European+Conference+on+Information+Retrieval%2E)
18. Vadim Gudkov, Olga Mitrofanova, and Elizaveta Fil-ippskikh. 2020. Automatically ranked russian paraphrase corpus for text generation. In Proceedings of the Fourth Workshop on Neural Generation and Translation, pages 54–59. [find](https://scholar.google.com/scholar?q=Vadim+Gudkov%2C+Olga+Mitrofanova%2C+and+Elizaveta+Fil%2Dippskikh%2E+2020%2E+Automatically+ranked+russian+paraphrase+corpus+for+text+generation%2E+In+Proceedings+of+the+Fourth+Workshop+on+Neural+Generation+and+Tra)
19. Neel Jain, Ping-yeh Chiang, Yuxin Wen, John Kirchenbauer, Hong-Min Chu, Gowthami Somepalli, Brian R Bartoldson, Bhavya Kailkhura, Avi Schwarzschild, Aniruddha Saha, et al. 2023. Neftune: Noisy embeddings improve instruction finetuning. arXiv preprint arXiv:2310.05914. — [arXiv:2310.05914](https://arxiv.org/abs/2310.05914) [find](https://scholar.google.com/scholar?q=Neel+Jain%2C+Ping%2Dyeh+Chiang%2C+Yuxin+Wen%2C+John+Kirchenbauer%2C+Hong%2DMin+Chu%2C+Gowthami+Somepalli%2C+Brian+R+Bartoldson%2C+Bhavya+Kailkhura%2C+Avi+Schwarzschild%2C+Aniruddha+Saha%2C+et+al%2E+2023%2E+Neftune%3A+Noisy+embeddi)
20. James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. 2017. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521… [find](https://scholar.google.com/scholar?q=James+Kirkpatrick%2C+Razvan+Pascanu%2C+Neil+Rabinowitz%2C+Joel+Veness%2C+Guillaume+Desjardins%2C+Andrei+A+Rusu%2C+Kieran+Milan%2C+John+Quan%2C+Tiago+Ramalho%2C+Agnieszka+Grabska%2DBarwinska%2C+et+al%2E+2017%2E+Overcoming+catas)
21. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. [Retrieval-augmented generation for knowledge-intensive nlp tasks](https://md.research.elamur.ai/free/en/papers/lewis-2020-retrieval-augmented-generation-knowledge-intensive/). Advances in Neural Information Processing Systems, 33:9459–9474.
22. Xianming Li and Jing Li. 2023. [Angle-optimized text embeddings](https://md.research.elamur.ai/free/en/papers/li-2023-angle-optimized-text-embeddings/). arXiv preprint arXiv:2309.12871.
23. Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. 2023. [Towards general text embeddings with multi-stage contrastive learning](https://md.research.elamur.ai/free/en/papers/li-2023-towards-general-text-embeddings-multi/). arXiv preprint arXiv:2308.03281.
24. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. [Roberta: A robustly optimized bert pretraining approach](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/). arXiv preprint arXiv:1907.11692.
25. Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101. [find](https://scholar.google.com/scholar?q=Ilya+Loshchilov+and+Frank+Hutter%2E+2017%2E+Decoupled+weight+decay+regularization%2E+arXiv+preprint+arXiv%3A1711%2E05101%2E)
26. Boris Malashenko, Anton Zemerov, and Egor Spirin. 2024a. Ruhnp: Russian hard-non-paraphrases. [find](https://scholar.google.com/scholar?q=Boris+Malashenko%2C+Anton+Zemerov%2C+and+Egor+Spirin%2E+2024a%2E+Ruhnp%3A+Russian+hard%2Dnon%2Dparaphrases%2E)
27. Boris Malashenko, Anton Zemerov, and Egor Spirin. 2024b. Ruwanli. [find](https://scholar.google.com/scholar?q=Boris+Malashenko%2C+Anton+Zemerov%2C+and+Egor+Spirin%2E+2024b%2E+Ruwanli%2E)
28. Nikita Martynov, Mark Baushenko, Anastasia Kozlova, Katerina Kolomeytseva, Aleksandr Abramov, and Alena Fenogenova. 2024. A methodology for generative spelling correction via natural spelling errors emulation across multiple domains and languages. In Findings of the Association for Computational Lin… [find](https://scholar.google.com/scholar?q=Nikita+Martynov%2C+Mark+Baushenko%2C+Anastasia+Kozlova%2C+Katerina+Kolomeytseva%2C+Aleksandr+Abramov%2C+and+Alena+Fenogenova%2E+2024%2E+A+methodology+for+generative+spelling+correction+via+natural+spelling+errors+e)
29. Luke Merrick, Danmei Xu, Gaurav Nuti, and Daniel Campos. 2024. Arctic-embed: Scalable, efficient, and accurate text embedding models. arXiv preprint arXiv:2405.05374. — [arXiv:2405.05374](https://arxiv.org/abs/2405.05374) [find](https://scholar.google.com/scholar?q=Luke+Merrick%2C+Danmei+Xu%2C+Gaurav+Nuti%2C+and+Daniel+Campos%2E+2024%2E+Arctic%2Dembed%3A+Scalable%2C+efficient%2C+and+accurate+text+embedding+models%2E+arXiv+preprint+arXiv%3A2405%2E05374%2E+%E2%80%94+%5BarXiv%3A2405%2E05374%5D%28https%3A%2F%2Farxi)
30. Vladislav Mikhailov, Tatiana Shamardina, Max Ryabinin, Alena Pestova, Ivan Smurov, and Ekaterina Artemova. 2022. Rucola: Russian corpus of linguistic acceptability. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Vladislav+Mikhailov%2C+Tatiana+Shamardina%2C+Max+Ryabinin%2C+Alena+Pestova%2C+Ivan+Smurov%2C+and+Ekaterina+Artemova%2E+2022%2E+Rucola%3A+Russian+corpus+of+linguistic+acceptability%2E+In+Proceedings+of+the+2022+Conferen)
31. Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. 2023. [Mteb: Massive text embedding benchmark](https://md.research.elamur.ai/free/en/papers/muennighoff-2022-mteb-massive-text-embedding-benchmark/). In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2014–2037.
32. Multi-Linguality Multi-Functionality Multi-Granularity. [M3-embedding: Multi-linguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation](https://md.research.elamur.ai/free/en/papers/chen-2024-m3-embedding-multi-linguality/).
33. Zach Nussbaum, John X. Morris, Brandon Duderstadt, and Andriy Mulyar. 2024. [Nomic embed: Training a reproducible long context text embedder](https://md.research.elamur.ai/free/en/papers/nussbaum-2024-nomic-embed-training-reproducible-long/). Preprint, arXiv:2402.01613.
34. Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. [Representation learning with contrastive predictive coding](https://md.research.elamur.ai/free/en/papers/oord-2018-representation-learning-contrastive-predictive-coding/). arXiv preprint arXiv:1807.03748.
35. Dina Pisarevskaya and Tatiana Shavrina. 2022. WikiOmnia: filtration and evaluation of the generated QA corpus on the whole Russian Wikipedia. In Proceedings of the 2nd Workshop on Natural Language Generation, Evaluation, and Metrics (GEM), pages 125–135, Abu Dhabi, United Arab Emirates (Hybrid). Ass… [find](https://scholar.google.com/scholar?q=Dina+Pisarevskaya+and+Tatiana+Shavrina%2E+2022%2E+WikiOmnia%3A+filtration+and+evaluation+of+the+generated+QA+corpus+on+the+whole+Russian+Wikipedia%2E+In+Proceedings+of+the+2nd+Workshop+on+Natural+Language+Gen)
36. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. [Learning transferable visual models from natural language supervision](https://md.research.elamur.ai/free/en/papers/radford-2021-learning-transferable-visual-models-natural/). In International conference on machine learning, pages 8748–8763. PM…
37. Nils Reimers, Philip Beyer, and Iryna Gurevych. 2016. Task-oriented intrinsic evaluation of semantic textual similarity. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 87–96. [find](https://scholar.google.com/scholar?q=Nils+Reimers%2C+Philip+Beyer%2C+and+Iryna+Gurevych%2E+2016%2E+Task%2Doriented+intrinsic+evaluation+of+semantic+textual+similarity%2E+In+Proceedings+of+COLING+2016%2C+the+26th+International+Conference+on+Computation)
38. Nils Reimers, Elliot Choi, Amr Kayid, Alekhya Nandula, Manoj Govindassamy, and Abdullah Elkady. 2023. Introducing embed v3. [find](https://scholar.google.com/scholar?q=Nils+Reimers%2C+Elliot+Choi%2C+Amr+Kayid%2C+Alekhya+Nandula%2C+Manoj+Govindassamy%2C+and+Abdullah+Elkady%2E+2023%2E+Introducing+embed+v3%2E)
39. Nils Reimers and Iryna Gurevych. 2019. [Sentence-bert: Sentence embeddings using siamese bert-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics.
40. Ruiyang Ren, Yingqi Qu, Jing Liu, Wayne Xin Zhao, Qiaoqiao She, Hua Wu, Haifeng Wang, and Ji-Rong Wen. 2021. [Rocketqav2: A joint training method for dense passage retrieval and passage re-ranking](https://md.research.elamur.ai/free/en/papers/ren-2021-rocketqav2-joint-training-method-dense/). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 2825–2…
41. Andrew Rosenberg and Julia Hirschberg. 2007. [V-measure: A conditional entropy-based external cluster evaluation measure](https://md.research.elamur.ai/free/en/papers/rosenberg-2007-vmeasure-conditional-entropy-cluster-evaluation/). In Proceedings of the 2007 joint conference on empirical methods in natural language processing and computational natural language learning (EMNLP-CoNLL), pages 410–420.
42. Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2019. [Superglue: A stickier benchmark for general-purpose language understanding systems](https://md.research.elamur.ai/free/en/papers/wang-2019-superglue-stickier-benchmark-general-purpose/). Advances in neural information processing systems, 32.
43. Ivan Rybin, Vladislav Korablinov, Pavel Efimov, and Pavel Braslavski. 2021. Rubq 2.0: an innovated russian question answering dataset. In The Semantic Web: 18th International Conference, ESWC 2021, Virtual Event, June 6–10, 2021, Proceedings 18, pages 532–547. Springer. [find](https://scholar.google.com/scholar?q=Ivan+Rybin%2C+Vladislav+Korablinov%2C+Pavel+Efimov%2C+and+Pavel+Braslavski%2E+2021%2E+Rubq+2%2E0%3A+an+innovated+russian+question+answering+dataset%2E+In+The+Semantic+Web%3A+18th+International+Conference%2C+ESWC+2021%2C+Vi)
44. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. 2018. [Glue: A multi-task benchmark and analysis platform for natural language understanding](https://md.research.elamur.ai/free/en/papers/wang-2018-glue-multi-task-benchmark-analysis/). arXiv preprint arXiv:1804.07461.
45. Alexander Sboev, Aleksandr Naumov, and Roman Rybka. 2021. Data-driven model for emotion detection in russian texts. Procedia Computer Science, 190:637–642. [find](https://scholar.google.com/scholar?q=Alexander+Sboev%2C+Aleksandr+Naumov%2C+and+Roman+Rybka%2E+2021%2E+Data%2Ddriven+model+for+emotion+detection+in+russian+texts%2E+Procedia+Computer+Science%2C+190%3A637%E2%80%93642%2E)
46. Kexin Wang, Nils Reimers, and Iryna Gurevych. 2021. Tsdae: Using transformer-based sequential denoising auto-encoderfor unsupervised sentence embedding learning. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 671–688. [find](https://scholar.google.com/scholar?q=Kexin+Wang%2C+Nils+Reimers%2C+and+Iryna+Gurevych%2E+2021%2E+Tsdae%3A+Using+transformer%2Dbased+sequential+denoising+auto%2Dencoderfor+unsupervised+sentence+embedding+learning%2E+In+Findings+of+the+Association+for+Com)
47. Timo Schick and Hinrich Schütze. 2021. [Generating datasets with pretrained language models](https://md.research.elamur.ai/free/en/papers/schick-2021-generating-datasets-pretrained-language-models/). Computing Research Repository, arXiv:2104.07540.
48. Tatiana Shavrina, Alena Fenogenova, Anton Emelyanov, Denis Shevelev, Ekaterina Artemova, Valentin Malykh, Vladislav Mikhailov, Maria Tikhonova, Andrey Chertok, and Andrey Evlampiev. 2020. Russiansuperglue: A russian language understanding evaluation benchmark. arXiv preprint arXiv:2010.15925. [find](https://scholar.google.com/scholar?q=Tatiana+Shavrina%2C+Alena+Fenogenova%2C+Anton+Emelyanov%2C+Denis+Shevelev%2C+Ekaterina+Artemova%2C+Valentin+Malykh%2C+Vladislav+Mikhailov%2C+Maria+Tikhonova%2C+Andrey+Chertok%2C+and+Andrey+Evlampiev%2E+2020%2E+Russiansuper)
49. Xiao Shitao, Liu Zheng, Shao Yingxia, and Zhao Cao. 2022. [Retromae: Pre-training retrieval-oriented language models via masked auto-encoder](https://md.research.elamur.ai/free/en/papers/xiao-2022-retromae-pre-training-retrieval-oriented/). In EMNLP.
50. Amanpreet Singh, Mike D’Arcy, Arman Cohan, Doug Downey, and Sergey Feldman. 2023. Scirepeval: A multi-format benchmark for scientific document representations. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5548–5566. [find](https://scholar.google.com/scholar?q=Amanpreet+Singh%2C+Mike+D%E2%80%99Arcy%2C+Arman+Cohan%2C+Doug+Downey%2C+and+Sergey+Feldman%2E+2023%2E+Scirepeval%3A+A+multi%2Dformat+benchmark+for+scientific+document+representations%2E+In+Proceedings+of+the+2023+Conference+on)
51. Emma Strubell, Ananya Ganesh, and Andrew McCallum. 2019. [Energy and policy considerations for deep learning in NLP](https://md.research.elamur.ai/free/en/papers/strubell-2019-energy-policy-considerations-deep-learning/). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3645–3650, Florence, Italy. Association for Computational Linguistics.
52. Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A. Smith, Luke Zettlemoyer, and Tao Yu. 2022. [One embedder, any task: Instruction-finetuned text embeddings](https://md.research.elamur.ai/free/en/papers/su-2022-one-embedder-any-task-instruction/).
53. Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2022a. [Text embeddings by weakly-supervised contrastive pre-training](https://md.research.elamur.ai/free/en/papers/wang-2022-text-embeddings-weakly-supervised-contrastive/). arXiv preprint arXiv:2212.03533.
54. Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2023a. [Improving text embeddings with large language models](https://md.research.elamur.ai/free/en/papers/wang-2024-improving-text-embeddings-large-language/). arXiv preprint arXiv:2401.00368.
55. Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024. [Multilingual e5 text embeddings: A technical report](https://md.research.elamur.ai/free/en/papers/wang-2024-multilingual-e5-text-embeddings-technical/). arXiv preprint arXiv:2402.05672.
56. Liang Wang, Nan Yang, and Furu Wei. 2023b. [Query2doc: Query expansion with large language models](https://md.research.elamur.ai/free/en/papers/wang-2023-query2doc-query-expansion-large-language/). arXiv preprint arXiv:2303.07678.
57. Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasekaran, Anjana Arunkumar, David Stap, et al. 2022b. Super-naturalinstructions: Generalization via declarative instructions on 1600+ nlp tasks. In Proceedings of the 202… [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Swaroop+Mishra%2C+Pegah+Alipoormolabashi%2C+Yeganeh+Kordi%2C+Amirreza+Mirzaei%2C+Atharva+Naik%2C+Arjun+Ashok%2C+Arut+Selvan+Dhanasekaran%2C+Anjana+Arunkumar%2C+David+Stap%2C+et+al%2E+2022b%2E+Super%2Dnaturalins)
58. Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighof. 2023a. [C-pack: Packaged resources to advance general chinese embedding](https://md.research.elamur.ai/free/en/papers/xiao-2023-c-pack-packed-resources/). arXiv preprint arXiv:2309.07597.
59. Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. [Beir: A heterogenous benchmark for zero-shot evaluation of information retrieval models](https://md.research.elamur.ai/free/en/papers/thakur-2021-beir-heterogenous-benchmark-zero/). arXiv preprint arXiv:2104.08663.
60. Shitao Xiao, Zheng Liu, Peitian Zhang, and Xingrun Xing. 2023b. Lm-cocktail: Resilient tuning of language models via model merging. arXiv preprint arXiv:2311.13534. [find](https://scholar.google.com/scholar?q=Shitao+Xiao%2C+Zheng+Liu%2C+Peitian+Zhang%2C+and+Xingrun+Xing%2E+2023b%2E+Lm%2Dcocktail%3A+Resilient+tuning+of+language+models+via+model+merging%2E+arXiv+preprint+arXiv%3A2311%2E13534%2E)
61. Jörg Tiedemann. 2012. Parallel data, tools and interfaces in OPUS. In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), pages 2214–2218, Istanbul, Turkey. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=J%C3%B6rg+Tiedemann%2E+2012%2E+Parallel+data%2C+tools+and+interfaces+in+OPUS%2E+In+Proceedings+of+the+Eighth+International+Conference+on+Language+Resources+and+Evaluation+%28LREC%E2%80%9912%29%2C+pages+2214%E2%80%932218%2C+Istanbul%2C+Turk)
62. Yinfei Yang, Gustavo Hernandez Abrego, Steve Yuan, Mandy Guo, Qinlan Shen, Daniel Cer, Yun-Hsuan Sung, Brian Strope, and Ray Kurzweil. 2019. Improving multilingual sentence embedding using bidirectional dual encoder with additive margin softmax. arXiv preprint arXiv:1902.08564. — [arXiv:1902.08564](https://arxiv.org/abs/1902.08564) [find](https://scholar.google.com/scholar?q=Yinfei+Yang%2C+Gustavo+Hernandez+Abrego%2C+Steve+Yuan%2C+Mandy+Guo%2C+Qinlan+Shen%2C+Daniel+Cer%2C+Yun%2DHsuan+Sung%2C+Brian+Strope%2C+and+Ray+Kurzweil%2E+2019%2E+Improving+multilingual+sentence+embedding+using+bidirection)
63. Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023. [Sigmoid loss for language image pre-training](https://md.research.elamur.ai/free/en/papers/zhai-2023-sigmoid-loss-language-image-pre/). In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 11941–11952. IEEE Computer Society.
64. Biao Zhang, Philip Williams, Ivan Titov, and Rico Sennrich. 2020. [Improving massively multilingual neural machine translation and zero-shot translation](https://md.research.elamur.ai/free/en/papers/zhang-2020-improving-massively-multilingual-neural-machine/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1628– 1639, Online. Association for Computational…
65. Xinyu Zhang, Xueguang Ma, Peng Shi, and Jimmy Lin. 2021. Mr. TyDi: A multi-lingual benchmark for dense retrieval. arXiv:2108.08787. [find](https://scholar.google.com/scholar?q=Xinyu+Zhang%2C+Xueguang+Ma%2C+Peng+Shi%2C+and+Jimmy+Lin%2E+2021%2E+Mr%2E+TyDi%3A+A+multi%2Dlingual+benchmark+for+dense+retrieval%2E+arXiv%3A2108%2E08787%2E)
66. Xinyu Zhang, Nandan Thakur, Odunayo Ogundepo, Ehsan Kamalloo, David Alfonso-Hermelo, Xiaoguang Li, Qun Liu, Mehdi Rezagholizadeh, and Jimmy Lin. 2022. Making a miracl: Multilingual information retrieval across a continuum of languages. arXiv preprint arXiv:2210.09984. — [arXiv:2210.09984](https://arxiv.org/abs/2210.09984) [find](https://scholar.google.com/scholar?q=Xinyu+Zhang%2C+Nandan+Thakur%2C+Odunayo+Ogundepo%2C+Ehsan+Kamalloo%2C+David+Alfonso%2DHermelo%2C+Xiaoguang+Li%2C+Qun+Liu%2C+Mehdi+Rezagholizadeh%2C+and+Jimmy+Lin%2E+2022%2E+Making+a+miracl%3A+Multilingual+information+retriev)
67. Dmitry Zmitrovich, Alexander Abramov, Andrey Kalmykov, Maria Tikhonova, Ekaterina Taktasheva, Danil Astafurov, Mark Baushenko, Artem Snegirev, Tatiana Shavrina, Sergey Markov, et al. 2023. [A family of pretrained transformer language models for russian](https://md.research.elamur.ai/free/en/papers/zmitrovich-2024-family-pretrained-transformer-language-models/). arXiv preprint arXiv:2309.10931.

*Updated 31 October 2026*