---
title: "Making Monolingual Sentence Embeddings — Reimers 2020"
description: "Most strong sentence embedding models exist only for English, because training data for other languages is scarce."
authors: ["Nils Reimers", "Iryna Gurevych"]
year: 2020
venue: "arXiv:2004.09813"
source: "https://arxiv.org/abs/2004.09813"
doi: "10.48550/arxiv.2004.09813"
arxiv: "2004.09813"
licence: "CC BY-SA 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by-sa/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/reimers-2020-making-monolingual-sentence-embeddings-multilingual/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/reimers-2020-making-monolingual-sentence-embeddings-multilingual/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/4/) / Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation

# Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation

**Authors:** [Nils Reimers](https://md.research.elamur.ai/free/en/authors/nils-reimers/), [Iryna Gurevych](https://md.research.elamur.ai/free/en/authors/iryna-gurevych/) · **Year:** 2020 · **Venue:** arXiv:2004.09813 · **Licence:** [CC BY-SA 4.0 (arXiv)](http://creativecommons.org/licenses/by-sa/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2004.09813) · [PDF](https://arxiv.org/pdf/2004.09813) · [DOI 10.48550/arxiv.2004.09813](https://doi.org/10.48550/arxiv.2004.09813) · [arXiv 2004.09813](https://arxiv.org/abs/2004.09813) · [OpenAlex](https://openalex.org/W3017604292)

**Citations:** 816 citations (openalex-snapshot · 2026-10-14) · [504](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
Most strong sentence embedding models exist only for English, because training data for other languages is scarce. Reimers and Gurevych propose multilingual knowledge distillation, a cheap way to carry an existing monolingual model over to new languages. An English teacher model (SBERT) produces vectors for source sentences. A student model, usually XLM-R, is then trained with an MSE loss so that its vectors for an English sentence and for its translation both land close to the teacher's vector. This aligns the languages in one vector space, and the student also inherits the properties of the teacher's space. The parallel training data comes from OPUS corpora and the authors' own TED2020 corpus; bilingual dictionaries are also tested. The method is evaluated on multilingual STS, bitext retrieval and cross-lingual similarity search. Compared with LASER, mUSE and LaBSE, the distilled models are better at judging similarity between sentences that are not exact translations, and they show almost no language bias. Because training a good English model is separated from adding languages, the approach needs far less compute and suits lower-resource languages such as Ukrainian.

## Key points
- **Training objective:** the student M̂ is trained so that M̂(s) ≈ M(s) and M̂(t) ≈ M(s) for each translation pair (s, t), using mean-squared loss. Teacher: SBERT-nli-stsb or SBERT-paraphrases. Students: XLM-R, mBERT or DistilmBERT. Hyperparameters: up to 20 epochs, batch size 64, learning rate 2e-5.
- **Language coverage:** one model was trained on parallel data for 50 languages, including uk, ru and pl. For low-resource languages it gains up to 40 accuracy points over LASER. The released code in UKPLab/sentence-transformers supports extension to more than 400 languages.
- **Training data:** GlobalVoices, TED2020 (subtitles from about 4,000 TED talks in over 100 languages, released by the authors), NewsCommentary, WikiMatrix (only pairs scoring above 1.05), Tatoeba, Europarl, JW300, OpenSubtitles2018 and UNPC. The MUSE and Wikititles bilingual dictionaries serve as low-resource alternatives.
- **Language bias on mixed-language STS:** pooling all language pairs into one STS set costs LaBSE −1.29 and LASER −0.92 Spearman points, both significant. mUSE (−0.19) and XLM-R ← SBERT-paraphrases (−0.11) are essentially unaffected; the latter reaches 83.9 on the pooled set.
- **Compute:** LASER needed 16 V100 GPUs for about 5 days, and mUSE and LaBSE require a complex translation-ranking setup with hard negatives. The distillation approach has much lower hardware demands.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Other](https://md.research.elamur.ai/free/en/shelves/other/)

## Cited by

Cited by

504 papers of our library cite it (of the 140884 whose bibliographies we read); 31 of them have a page here, 473 not yet.

- [A Corpus for Sentence-Level Subjectivity Detection on English News Articles](https://md.research.elamur.ai/free/en/papers/antici-2024-corpus-sentence-level-subjectivity-detection/)
- [Jais 2: A Family of Arabic-Centric Open Large Language Models](https://md.research.elamur.ai/free/en/papers/anwar-2026-jais-2-family-arabic-centric/)
- [LLM-Assisted Rule Based Machine Translation for Low/No-Resource Languages](https://md.research.elamur.ai/free/en/papers/coleman-2024-llm-assisted-rule-based-machine/)
- [SeamlessM4T: Massively Multilingual & Multimodal Machine Translation](https://md.research.elamur.ai/free/en/papers/communication-2023-seamlessm4t-massively-multilingual-multimodal-machine/)
- [Acquiring a Formality-Informed Lexical Resource for Style Analysis](https://md.research.elamur.ai/free/en/papers/eder-2021-acquiring-formality-informed-lexical-resource/)
- [Language-agnostic BERT Sentence Embedding](https://md.research.elamur.ai/free/en/papers/feng-2020-language-agnostic-bert-sentence-embedding/)
- [Results of the WMT21 Metrics Shared Task: Evaluating Metrics with Expert-based Human Evaluations on TED and News Domain](https://md.research.elamur.ai/free/en/papers/freitag-2021-results-wmt21-metrics-shared-task/)
- [Team_Zero at StanceEval2024: Frozen PLMs for Arabic Stance Detection](https://md.research.elamur.ai/free/en/papers/galal-2024-team-zero-stanceeval2024-frozen-plms/)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://md.research.elamur.ai/free/en/papers/grootendorst-2022-bertopic-neural-topic-modeling-class/)
- [Bitext Mining Using Distilled Sentence Representations for Low-Resource Languages](https://md.research.elamur.ai/free/en/papers/heffernan-2022-bitext-mining-distilled-sentence-representations/)
- [Bridging Cultures in the Kitchen: A Framework and Benchmark for Cross-Cultural Recipe Retrieval](https://md.research.elamur.ai/free/en/papers/hu-2024-bridging-cultures-kitchen-framework-benchmark/)
- [Mevaker: Conclusion Extraction and Allocation Resources for the Hebrew Language](https://md.research.elamur.ai/free/en/papers/langstream-2024-shalumov-mevaker-conclusion-extraction-allocation-resources/)
- [What Makes Good In-Context Examples for GPT-3?](https://md.research.elamur.ai/free/en/papers/liu-2022-what-makes-good-context-examples/)
- [Crossing the Threshold: Idiomatic Machine Translation through Retrieval Augmentation and Loss Weighting](https://md.research.elamur.ai/free/en/papers/liu-2023-crossing-threshold-idiomatic-machine-translation/)
- [Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models](https://md.research.elamur.ai/free/en/papers/ni-2021-sentence-t5-scalable-sentence-encoders/)
- [A Single Model Ensemble Framework for Neural Machine Translation Using Pivot Translation](https://md.research.elamur.ai/free/en/papers/oh-2026-single-model-ensemble-framework-neural/)
- [Controllable Sentence Simplification in Italian: Fine-Tuning Large Language Models on Automatically Generated Resources](https://md.research.elamur.ai/free/en/papers/papucci-2026-controllable-sentence-simplification-italian-fine/)
- [Harder than Finding the Lost Sheep? Towards Automatically Suggesting Deliberate Metaphor Annotations in German Sermons](https://md.research.elamur.ai/free/en/papers/quante-2026-harder-than-finding-lost-sheep/)
- [Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languages](https://md.research.elamur.ai/free/en/papers/ramesh-2022-samanantar-largest-publicly-available-parallel/)
- [Evaluating Latin and Ancient Greek Sentence Alignment through Parallel Sentence Mining](https://md.research.elamur.ai/free/en/papers/reichbauer-2026-evaluating-latin-ancient-greek-sentence/)
- [Modeling Cross-Cultural Pragmatic Inference with Codenames Duet](https://md.research.elamur.ai/free/en/papers/shaikh-2023-modeling-cross-cultural-pragmatic-inference/)
- [Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers](https://md.research.elamur.ai/free/en/papers/si-2024-llms-generate-novel-research/)
- [Advancing Arabic Reverse Dictionary Systems: A Transformer-Based Approach with Dataset Construction Guidelines](https://md.research.elamur.ai/free/en/papers/sibaee-2025-advancing-arabic-reverse-dictionary-systems/)
- [DarijaBanking: A New Resource for Overcoming Language Barriers in Banking Intent Detection for Moroccan Arabic Speakers](https://md.research.elamur.ai/free/en/papers/skiredj-2024-darijabanking-new-resource-overcoming-language/)
- [HOTATE: A Japanese Dialogue Corpus Annotated with Responses of Private Thoughts and Public Statements](https://md.research.elamur.ai/free/en/papers/toda-2026-hotate-japanese-dialogue-corpus-annotated/)
- [Gretino: A Greek and Latin Dataset to Benchmark Retrieval Systems in Classical Languages](https://md.research.elamur.ai/free/en/papers/toyin-2026-gretino-greek-latin-dataset-benchmark/)
- [A Unified Approach to Discourse Relation Classification in nine Languages](https://md.research.elamur.ai/free/en/papers/varachkina-2021-unified-approach-discourse-relation-classification/)
- [UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models](https://md.research.elamur.ai/free/en/papers/xie-2022-unifiedskg-unifying-multi-tasking-structured/)
- [Lost in Translation: Repurposing semantic similarity benchmarks for evaluating lexical-semantic consistency in LLM-based machine translation](https://md.research.elamur.ai/free/en/papers/ye-2026-lost-translation-repurposing-semantic-similarity/)
- [Terminology-Aware Retrieval-Augmented Knowledge Distillation for Biomedical Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/zafar-2026-terminology-aware-retrieval-augmented-knowledge/)
- [The DISRPT 2021 Shared Task on Elementary Discourse Unit Segmentation, Connective Detection, and Relation Classification](https://md.research.elamur.ai/free/en/papers/zeldes-2021-disrpt-2021-shared-task-elementary/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{reimers2020making,
  title = {{Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation}},
  author = {Reimers, Nils and Gurevych, Iryna},
  year = {2020},
  doi = {10.48550/arxiv.2004.09813},
  eprint = {2004.09813},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2004.09813}
}
```

- [← previous MAGPIE: A Large Corpus of Potentially Idiomatic Expressions](https://md.research.elamur.ai/free/en/papers/haagsma-2020-magpie-large-corpus-potentially-idiomatic/)
- [next → Many systems, one strategy: Acquiring ordinals in Dutch and English](https://md.research.elamur.ai/free/en/papers/meyer-2020-many-systems-one-strategy-acquiring/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/4/)

## References

35 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Željko Agić and Ivan Vulić. 2019. JW300: A Wide-Coverage Parallel Corpus for Low-Resource Languages. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3204–3210, Florence, Italy. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Z%CC%8Celjko+Agic%CC%81+and+Ivan+Vulic%CC%81%2E+2019%2E+JW300%3A+A+Wide%2DCoverage+Parallel+Corpus+for+Low%2DResource+Languages%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+page) [our card](https://md.research.elamur.ai/free/en/papers/agic-2019-jw300-wide-coverage-parallel-corpus/)
2. Mikel Artetxe, Gorka Labaka, and Eneko Agirre. 2018. Generalizing and improving bilingual word embedding mappings with a multi-step framework of linear transformations. AAAI Conference on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Mikel+Artetxe%2C+Gorka+Labaka%2C+and+Eneko+Agirre%2E+2018%2E+Generalizing+and+improving+bilingual+word+embedding+mappings+with+a+multi%2Dstep+framework+of+linear+transformations%2E+AAAI+Conference+on+Artificial+I)
3. Mikel Artetxe and Holger Schwenk. 2019a. Margin-based Parallel Corpus Mining with Multilingual Sentence Embeddings. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3197–3203, Florence, Italy. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Mikel+Artetxe+and+Holger+Schwenk%2E+2019a%2E+Margin%2Dbased+Parallel+Corpus+Mining+with+Multilingual+Sentence+Embeddings%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Computational+Lingu) [our card](https://md.research.elamur.ai/free/en/papers/artetxe-2019-margin-based-parallel-corpus-mining/)
4. Mikel Artetxe and Holger Schwenk. 2019b. Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond. Transactions of the Association for Computational Linguistics, 7(0):597–610. [find](https://scholar.google.com/scholar?q=Mikel+Artetxe+and+Holger+Schwenk%2E+2019b%2E+Massively+Multilingual+Sentence+Embeddings+for+Zero%2DShot+Cross%2DLingual+Transfer+and+Beyond%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+7%280%29) [our card](https://md.research.elamur.ai/free/en/papers/artetxe-2018-massively-multilingual-sentence-embeddings-zero/)
5. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 632–642, Lisbon, Portugal. Association for Computati… [find](https://scholar.google.com/scholar?q=Samuel+R%2E+Bowman%2C+Gabor+Angeli%2C+Christopher+Potts%2C+and+Christopher+D%2E+Manning%2E+2015%2E+A+large+annotated+corpus+for+learning+natural+language+inference%2E+In+Proceedings+of+the+2015+Conference+on+Empirica)
6. Daniel Cer, Mona Diab, Eneko Agirre, Iñigo Lopez-Gazpio, and Lucia Specia. 2017. SemEval-2017 Task 1: Semantic Textual Similarity Multilingual and Crosslingual Focused Evaluation. In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 1–14, Vancouver, Canada.… [find](https://scholar.google.com/scholar?q=Daniel+Cer%2C+Mona+Diab%2C+Eneko+Agirre%2C+In%CC%83igo+Lopez%2DGazpio%2C+and+Lucia+Specia%2E+2017%2E+SemEval%2D2017+Task+1%3A+Semantic+Textual+Similarity+Multilingual+and+Crosslingual+Focused+Evaluation%2E+In+Proceedings+of+t)
7. Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St. John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, Yun-Hsuan Sung, Brian Strope, and Ray Kurzweil. 2018. Universal Sentence Encoder. arXiv preprint arXiv:1803.11175. [find](https://scholar.google.com/scholar?q=Daniel+Cer%2C+Yinfei+Yang%2C+Sheng%2Dyi+Kong%2C+Nan+Hua%2C+Nicole+Limtiaco%2C+Rhomni+St%2E+John%2C+Noah+Constant%2C+Mario+Guajardo%2DCespedes%2C+Steve+Yuan%2C+Chris+Tar%2C+Yun%2DHsuan+Sung%2C+Brian+Strope%2C+and+Ray+Kurzweil%2E+2018%2E+) [our card](https://md.research.elamur.ai/free/en/papers/cer-2018-universal-sentence-encoder/)
8. Muthu Chidambaram, Yinfei Yang, Daniel Cer, Steve Yuan, Yunhsuan Sung, Brian Strope, and Ray Kurzweil. 2019. Learning Cross-Lingual Sentence Representations via a Multi-task Dual-Encoder Model. In Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019), pages 250–259, Flor… [find](https://scholar.google.com/scholar?q=Muthu+Chidambaram%2C+Yinfei+Yang%2C+Daniel+Cer%2C+Steve+Yuan%2C+Yunhsuan+Sung%2C+Brian+Strope%2C+and+Ray+Kurzweil%2E+2019%2E+Learning+Cross%2DLingual+Sentence+Representations+via+a+Multi%2Dtask+Dual%2DEncoder+Model%2E+In+Pro)
9. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Unsupervised Cross-lingual Representation Learning at Scale. arXiv preprint arXiv:1911.02116. [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Kartikay+Khandelwal%2C+Naman+Goyal%2C+Vishrav+Chaudhary%2C+Guillaume+Wenzek%2C+Francisco+Guzma%CC%81n%2C+Edouard+Grave%2C+Myle+Ott%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2019%2E+Unsupervised+Cross%2Dling)
10. Alexis Conneau, Douwe Kiela, Holger Schwenk, Loı̈c Barrault, and Antoine Bordes. 2017a. Supervised Learning of Universal Sentence Representations from Natural Language Inference Data. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 670–680, Copenhagen… [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Douwe+Kiela%2C+Holger+Schwenk%2C+Lo%C4%B1%CC%88c+Barrault%2C+and+Antoine+Bordes%2E+2017a%2E+Supervised+Learning+of+Universal+Sentence+Representations+from+Natural+Language+Inference+Data%2E+In+Proceedings+o) [our card](https://md.research.elamur.ai/free/en/papers/conneau-2017-supervised-learning-universal-sentence-representations/)
11. Alexis Conneau, Guillaume Lample, Marc’Aurelio Ranzato, Ludovic Denoyer, and Hervé Jégou. 2017b. Word Translation Without Parallel Data. arXiv preprint arXiv:1710.04087. [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Guillaume+Lample%2C+Marc%E2%80%99Aurelio+Ranzato%2C+Ludovic+Denoyer%2C+and+Herve%CC%81+Je%CC%81gou%2E+2017b%2E+Word+Translation+Without+Parallel+Data%2E+arXiv+preprint+arXiv%3A1710%2E04087%2E)
12. Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. XNLI: Evaluating Cross-lingual Sentence Representations. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2475– 2485, Brussels, Be… [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Ruty+Rinott%2C+Guillaume+Lample%2C+Adina+Williams%2C+Samuel+Bowman%2C+Holger+Schwenk%2C+and+Veselin+Stoyanov%2E+2018%2E+XNLI%3A+Evaluating+Cross%2Dlingual+Sentence+Representations%2E+In+Proceedings+of+the)
13. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv preprint arXiv:1810.04805. [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2018%2E+BERT%3A+Pre%2Dtraining+of+Deep+Bidirectional+Transformers+for+Language+Understanding%2E+arXiv+preprint+arXiv%3A1810%2E04805%2E)
14. Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2020. Language-agnostic BERT Sentence Embedding. arXiv preprint arXiv:2007.01852. [find](https://scholar.google.com/scholar?q=Fangxiaoyu+Feng%2C+Yinfei+Yang%2C+Daniel+Cer%2C+Naveen+Arivazhagan%2C+and+Wei+Wang%2E+2020%2E+Language%2Dagnostic+BERT+Sentence+Embedding%2E+arXiv+preprint+arXiv%3A2007%2E01852%2E) [our card](https://md.research.elamur.ai/free/en/papers/feng-2020-language-agnostic-bert-sentence-embedding/)
15. Mandy Guo, Qinlan Shen, Yinfei Yang, Heming Ge, Daniel Cer, Gustavo Hernandez Abrego, Keith Stevens, Noah Constant, Yun-Hsuan Sung, Brian Strope, and Ray Kurzweil. 2018. Effective Parallel Corpus Mining using Bilingual Sentence Embeddings. In Proceedings of the Third Conference on Machine Translatio… [find](https://scholar.google.com/scholar?q=Mandy+Guo%2C+Qinlan+Shen%2C+Yinfei+Yang%2C+Heming+Ge%2C+Daniel+Cer%2C+Gustavo+Hernandez+Abrego%2C+Keith+Stevens%2C+Noah+Constant%2C+Yun%2DHsuan+Sung%2C+Brian+Strope%2C+and+Ray+Kurzweil%2E+2018%2E+Effective+Parallel+Corpus+Mini) [our card](https://md.research.elamur.ai/free/en/papers/guo-2018-effective-parallel-corpus-mining-bilingual/)
16. Jiyeon Ham, Yo Joong Choe, Kyubyong Park, Ilji Choi, and Hyungjoon Soh. 2020. KorNLI and KorSTS: New Benchmark Datasets for Korean Natural Language Understanding. arXiv preprint arXiv:2004.03289. [find](https://scholar.google.com/scholar?q=Jiyeon+Ham%2C+Yo+Joong+Choe%2C+Kyubyong+Park%2C+Ilji+Choi%2C+and+Hyungjoon+Soh%2E+2020%2E+KorNLI+and+KorSTS%3A+New+Benchmark+Datasets+for+Korean+Natural+Language+Understanding%2E+arXiv+preprint+arXiv%3A2004%2E03289%2E)
17. Ryan Kiros, Yukun Zhu, Ruslan R Salakhutdinov, Richard Zemel, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015. Skip-Thought Vectors. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems 28, pages 3294–3302. Curran A… [find](https://scholar.google.com/scholar?q=Ryan+Kiros%2C+Yukun+Zhu%2C+Ruslan+R+Salakhutdinov%2C+Richard+Zemel%2C+Raquel+Urtasun%2C+Antonio+Torralba%2C+and+Sanja+Fidler%2E+2015%2E+Skip%2DThought+Vectors%2E+In+C%2E+Cortes%2C+N%2E+D%2E+Lawrence%2C+D%2E+D%2E+Lee%2C+M%2E+Sugiyama%2C+and+)
18. Philipp Koehn. 2005. Europarl: A Parallel Corpus for Statistical Machine Translation. In Conference Proceedings: the tenth Machine Translation Summit, pages 79–86, Phuket, Thailand. AAMT, AAMT. [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2E+2005%2E+Europarl%3A+A+Parallel+Corpus+for+Statistical+Machine+Translation%2E+In+Conference+Proceedings%3A+the+tenth+Machine+Translation+Summit%2C+pages+79%E2%80%9386%2C+Phuket%2C+Thailand%2E+AAMT%2C+AAMT%2E)
19. Guillaume Lample, Alexis Conneau, Ludovic Denoyer, and Marc’Aurelio Ranzato. 2018. Unsupervised Machine Translation Using Monolingual Corpora Only. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Guillaume+Lample%2C+Alexis+Conneau%2C+Ludovic+Denoyer%2C+and+Marc%E2%80%99Aurelio+Ranzato%2E+2018%2E+Unsupervised+Machine+Translation+Using+Monolingual+Corpora+Only%2E+In+International+Conference+on+Learning+Representati) [our card](https://md.research.elamur.ai/free/en/papers/lample-2017-unsupervised-machine-translation-using-monolingual/)
20. Pierre Lison and Jörg Tiedemann. 2016. OpenSubtitles2016: Extracting Large Parallel Corpora from Movie and TV Subtitles. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), Paris, France. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Pierre+Lison+and+Jo%CC%88rg+Tiedemann%2E+2016%2E+OpenSubtitles2016%3A+Extracting+Large+Parallel+Corpora+from+Movie+and+TV+Subtitles%2E+In+Proceedings+of+the+Tenth+International+Conference+on+Language+Resources+and) [our card](https://md.research.elamur.ai/free/en/papers/lison-2016-opensubtitles2016-extracting-large-parallel-corpora/)
21. Nils Reimers, Philip Beyer, and Iryna Gurevych. 2016. Task-Oriented Intrinsic Evaluation of Semantic Textual Similarity. In Proceedings of the 26th International Conference on Computational Linguistics (COLING), pages 87–96. [find](https://scholar.google.com/scholar?q=Nils+Reimers%2C+Philip+Beyer%2C+and+Iryna+Gurevych%2E+2016%2E+Task%2DOriented+Intrinsic+Evaluation+of+Semantic+Textual+Similarity%2E+In+Proceedings+of+the+26th+International+Conference+on+Computational+Linguistic)
22. Nils Reimers and Iryna Gurevych. 2017. Reporting score distributions makes a difference: Performance study of LSTM-networks for sequence tagging. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 338–348, Copenhagen, Denmark. Association for Computation… [find](https://scholar.google.com/scholar?q=Nils+Reimers+and+Iryna+Gurevych%2E+2017%2E+Reporting+score+distributions+makes+a+difference%3A+Performance+study+of+LSTM%2Dnetworks+for+sequence+tagging%2E+In+Proceedings+of+the+2017+Conference+on+Empirical+Met) [our card](https://md.research.elamur.ai/free/en/papers/reimers-2017-reporting-score-distributions-makes-difference/)
23. Nils Reimers and Iryna Gurevych. 2018. Why Comparing Single Performance Scores Does Not Allow to Draw Conclusions About Machine Learning Approaches. arXiv preprint arXiv:1803.09578, abs/1803.09578. — [arXiv:1803.09578](https://arxiv.org/abs/1803.09578) [find](https://scholar.google.com/scholar?q=Nils+Reimers+and+Iryna+Gurevych%2E+2018%2E+Why+Comparing+Single+Performance+Scores+Does+Not+Allow+to+Draw+Conclusions+About+Machine+Learning+Approaches%2E+arXiv+preprint+arXiv%3A1803%2E09578%2C+abs%2F1803%2E09578%2E+%E2%80%94+)
24. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,… [find](https://scholar.google.com/scholar?q=Nils+Reimers+and+Iryna+Gurevych%2E+2019%2E+Sentence%2DBERT%3A+Sentence+Embeddings+using+Siamese+BERT%2DNetworks%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Processing+and+the)
25. Uma Roy, Noah Constant, Rami Al-Rfou, Aditya Barua, Aaron Phillips, and Yinfei Yang. 2020. LAReQA: Language-agnostic answer retrieval from a multilingual pool. arXiv preprint arXiv:2004.05484. [find](https://scholar.google.com/scholar?q=Uma+Roy%2C+Noah+Constant%2C+Rami+Al%2DRfou%2C+Aditya+Barua%2C+Aaron+Phillips%2C+and+Yinfei+Yang%2E+2020%2E+LAReQA%3A+Language%2Dagnostic+answer+retrieval+from+a+multilingual+pool%2E+arXiv+preprint+arXiv%3A2004%2E05484%2E)
26. Sebastian Ruder. 2017. A survey of cross-lingual embedding models. arXiv preprint arXiv:1706.04902, abs/1706.04902. [find](https://scholar.google.com/scholar?q=Sebastian+Ruder%2E+2017%2E+A+survey+of+cross%2Dlingual+embedding+models%2E+arXiv+preprint+arXiv%3A1706%2E04902%2C+abs%2F1706%2E04902%2E)
27. Holger Schwenk, Vishrav Chaudhary, Shuo Sun, Hongyu Gong, and Francisco Guzmán. 2019. Wiki-Matrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipedia. arXiv preprint arXiv:11907.05791. [find](https://scholar.google.com/scholar?q=Holger+Schwenk%2C+Vishrav+Chaudhary%2C+Shuo+Sun%2C+Hongyu+Gong%2C+and+Francisco+Guzma%CC%81n%2E+2019%2E+Wiki%2DMatrix%3A+Mining+135M+Parallel+Sentences+in+1620+Language+Pairs+from+Wikipedia%2E+arXiv+preprint+arXiv%3A11907%2E057)
28. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to Sequence Learning with Neural Networks. In Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, editors, Advances in Neural Information Processing Systems 27, pages 3104–3112. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Ilya+Sutskever%2C+Oriol+Vinyals%2C+and+Quoc+V+Le%2E+2014%2E+Sequence+to+Sequence+Learning+with+Neural+Networks%2E+In+Z%2E+Ghahramani%2C+M%2E+Welling%2C+C%2E+Cortes%2C+N%2E+D%2E+Lawrence%2C+and+K%2E+Q%2E+Weinberger%2C+editors%2C+Advances)
29. Jörg Tiedemann. 2012. Parallel Data, Tools and Interfaces in OPUS. In Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC’12), Istanbul, Turkey. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Jo%CC%88rg+Tiedemann%2E+2012%2E+Parallel+Data%2C+Tools+and+Interfaces+in+OPUS%2E+In+Proceedings+of+the+Eight+International+Conference+on+Language+Resources+and+Evaluation+%28LREC%E2%80%9912%29%2C+Istanbul%2C+Turkey%2E+European+Lang)
30. Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Pa… [find](https://scholar.google.com/scholar?q=Adina+Williams%2C+Nikita+Nangia%2C+and+Samuel+Bowman%2E+2018%2E+A+Broad%2DCoverage+Challenge+Corpus+for+Sentence+Understanding+through+Inference%2E+In+Proceedings+of+the+2018+Conference+of+the+North+American+Chap)
31. Yinfei Yang, Daniel Cer, Amin Ahmad, Mandy Guo, Jax Law, Noah Constant, Gustavo Hernández Ábrego, Steve Yuan, Chris Tar, Yun-Hsuan Sung, Brian Strope, and Ray Kurzweil. 2019. Multilingual Universal Sentence Encoder for Semantic Retrieval. arXiv preprint arXiv:1907.04307, abs/1907.04307. [find](https://scholar.google.com/scholar?q=Yinfei+Yang%2C+Daniel+Cer%2C+Amin+Ahmad%2C+Mandy+Guo%2C+Jax+Law%2C+Noah+Constant%2C+Gustavo+Herna%CC%81ndez+A%CC%81brego%2C+Steve+Yuan%2C+Chris+Tar%2C+Yun%2DHsuan+Sung%2C+Brian+Strope%2C+and+Ray+Kurzweil%2E+2019%2E+Multilingual+Universal+) [our card](https://md.research.elamur.ai/free/en/papers/yang-2020-multilingual-universal-sentence-encoder-semantic/)
32. Yinfei Yang, Steve Yuan, Daniel Cer, Sheng-Yi Kong, Noah Constant, Petr Pilar, Heming Ge, Yun-hsuan Sung, Brian Strope, and Ray Kurzweil. 2018. Learning Semantic Textual Similarity from Conversations. In Proceedings of The Third Workshop on Representation Learning for NLP, pages 164–174, Melbourne,… [find](https://scholar.google.com/scholar?q=Yinfei+Yang%2C+Steve+Yuan%2C+Daniel+Cer%2C+Sheng%2DYi+Kong%2C+Noah+Constant%2C+Petr+Pilar%2C+Heming+Ge%2C+Yun%2Dhsuan+Sung%2C+Brian+Strope%2C+and+Ray+Kurzweil%2E+2018%2E+Learning+Semantic+Textual+Similarity+from+Conversations%2E)
33. Michał Ziemski, Marcin Junczys-Dowmunt, and Bruno Pouliquen. 2016. The United Nations Parallel Corpus v1.0. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), Paris, France. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Micha%C5%82+Ziemski%2C+Marcin+Junczys%2DDowmunt%2C+and+Bruno+Pouliquen%2E+2016%2E+The+United+Nations+Parallel+Corpus+v1%2E0%2E+In+Proceedings+of+the+Tenth+International+Conference+on+Language+Resources+and+Evaluation+%28L) [our card](https://md.research.elamur.ai/free/en/papers/ziemski-2016-united-nations-parallel-corpus-v1/)
34. Pierre Zweigenbaum, Serge Sharoff, and Reinhar Rapp. 2018. Overview of the Third BUCC Shared Task: Spotting Parallel Sentences in Comparable Corpora. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Paris, France. European Language Resources A… [find](https://scholar.google.com/scholar?q=Pierre+Zweigenbaum%2C+Serge+Sharoff%2C+and+Reinhar+Rapp%2E+2018%2E+Overview+of+the+Third+BUCC+Shared+Task%3A+Spotting+Parallel+Sentences+in+Comparable+Corpora%2E+In+Proceedings+of+the+Eleventh+International+Confe)
35. Pierre Zweigenbaum, Serge Sharoff, and Reinhard Rapp. 2017. Overview of the Second BUCC Shared Task: Spotting Parallel Sentences in Comparable Corpora. In Proceedings of the 10th Workshop on Building and Using Comparable Corpora, pages 60– 67, Vancouver, Canada. Association for Computational Linguis… [find](https://scholar.google.com/scholar?q=Pierre+Zweigenbaum%2C+Serge+Sharoff%2C+and+Reinhard+Rapp%2E+2017%2E+Overview+of+the+Second+BUCC+Shared+Task%3A+Spotting+Parallel+Sentences+in+Comparable+Corpora%2E+In+Proceedings+of+the+10th+Workshop+on+Building+)