---
title: "An Unsupervised Sentence Embedding Method by — Zhang 2020"
description: "Cross-encoding with BERT scales poorly for semantic search and clustering, because every candidate pair must pass through the network."
authors: ["Yan Zhang", "Ruidan He", "Zuozhu Liu", "Kwan Hui Lim", "Lidong Bing"]
year: 2020
venue: "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)"
source: "https://aclanthology.org/2020.emnlp-main.124/"
doi: "10.18653/v1/2020.emnlp-main.124"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2020-unsupervised-sentence-embedding-method-mutual/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2020-unsupervised-sentence-embedding-method-mutual/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/) / An Unsupervised Sentence Embedding Method by Mutual Information Maximization

# An Unsupervised Sentence Embedding Method by Mutual Information Maximization

**Authors:** [Yan Zhang](https://md.research.elamur.ai/free/en/authors/yan-zhang/), [Ruidan He](https://md.research.elamur.ai/free/en/authors/ruidan-he/), [Zuozhu Liu](https://md.research.elamur.ai/free/en/authors/zuozhu-liu/), Kwan Hui Lim, [Lidong Bing](https://md.research.elamur.ai/free/en/authors/lidong-bing/) · **Year:** 2020 · **Venue:** Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.emnlp-main.124/) · [PDF](https://aclanthology.org/2020.emnlp-main.124.pdf) · [DOI 10.18653/v1/2020.emnlp-main.124](https://doi.org/10.18653/v1/2020.emnlp-main.124) · [OpenAlex](https://openalex.org/W3100652389)

**Citations:** 140 citations (openalex-snapshot · 2026-10-14) · [73](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
Cross-encoding with BERT scales poorly for semantic search and clustering, because every candidate pair must pass through the network. SBERT avoids this by producing standalone sentence vectors, but it relies on labeled NLI pairs and transfers poorly to domains unlike NLI. The authors propose IS-BERT (Info-Sentence BERT), which learns sentence embeddings without labels. It adds a small module on top of BERT: 1-D CNNs with several window sizes turn token outputs into local n-gram features, and mean pooling compresses these into a single sentence vector. Training uses a contrastive objective inspired by DIM. It maximizes mutual information between each sentence vector and its own local features, with local features from other sentences serving as negatives. Because it needs no annotations, the model can be trained directly on a target-domain corpus. On STS and SentEval benchmarks it beats other unsupervised embedding methods and roughly matches supervised encoders like InferSent and USE. It also outperforms SBERT when no in-domain labels exist. This makes it useful for low-resource or specialized domains.

## Key points
- The motivating cost: finding the most similar pair among 10k sentences with BERT requires about 50 million inferences, which takes roughly 65 hours on a V100 GPU.
- On SentEval, IS-BERT-task averages 85.91 across seven tasks (MR, CR, SUBJ, MPQA, SST, TREC, MRPC) and leads the unsupervised baselines on 6 of them.
  - This average is slightly above InferSent-GloVe (85.59) and USE (85.10).
  - It remains below SBERT-NLI (87.41).
- Supervised STSb evaluation (8,628 pairs, split 5,749/1,500/1,379) reports Spearman ρ:
  - IS-BERT-STSb (ssl+ft): 84.84
  - SBERT-STSb: 84.67
  - BERT-STSb: 84.30
- Without the self-supervised stage, IS-BERT-STSb (ft) reaches only 74.25. Running MI pretraining before fine-tuning therefore gives a gain of more than 10 points, so the objective also works as a domain-adaptation step.
- The authors find that a more complex encoder alone does not guarantee better sentence embeddings; the self-supervised objective is what makes the difference.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

73 papers of our library cite it (of the 140884 whose bibliographies we read); 5 of them have a page here, 68 not yet.

- [DiffCSE: Difference-based Contrastive Learning for Sentence Embeddings](https://md.research.elamur.ai/free/en/papers/chuang-2022-diffcse-difference-based-contrastive-learning/)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://md.research.elamur.ai/free/en/papers/gao-2021-simcse-simple-contrastive-learning-sentence/)
- [Text and Code Embeddings by Contrastive Pre-Training](https://md.research.elamur.ai/free/en/papers/neelakantan-2022-text-code-embeddings-contrastive-pre/)
- [Generating Datasets with Pretrained Language Models](https://md.research.elamur.ai/free/en/papers/schick-2021-generating-datasets-pretrained-language-models/)
- [ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer](https://md.research.elamur.ai/free/en/papers/yan-2021-consert-contrastive-framework-self-supervised/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhang2020unsupervised,
  title = {{An Unsupervised Sentence Embedding Method by Mutual Information Maximization}},
  author = {Zhang, Yan and He, Ruidan and Liu, Zuozhu and Lim, Kwan Hui and Bing, Lidong},
  booktitle = {Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year = {2020},
  doi = {10.18653/v1/2020.emnlp-main.124},
  url = {https://doi.org/10.18653/v1/2020.emnlp-main.124}
}
```

- [← previous An Empirical Study of Contextual Data Augmentation for Japanese Zero Anaphora Resolution](https://md.research.elamur.ai/free/en/papers/konno-2020-empirical-study-contextual-data-augmentation/)
- [next → Analysing Lexical Semantic Change with Contextualised Word Representations](https://md.research.elamur.ai/free/en/papers/giulianelli-2020-analysing-lexical-semantic-change-contextualised/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/)

## References

44 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Eneko Agirre, Carmen Banea, Claire Cardie, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Weiwei Guo, Iñigo Lopez-Gazpio, Montse Maritxalar, Rada Mihalcea, German Rigau, Larraitz Uria, and Janyce Wiebe. 2015. Semeval-2015 task 2: Semantic textual similarity, english, spanish and pilot on interpretab… [find](https://scholar.google.com/scholar?q=Eneko+Agirre%2C+Carmen+Banea%2C+Claire+Cardie%2C+Daniel+Cer%2C+Mona+Diab%2C+Aitor+Gonzalez%2DAgirre%2C+Weiwei+Guo%2C+In%CC%83igo+Lopez%2DGazpio%2C+Montse+Maritxalar%2C+Rada+Mihalcea%2C+German+Rigau%2C+Larraitz+Uria%2C+and+Janyce+Wieb) [our card](https://md.research.elamur.ai/free/en/papers/agirre-2015-semeval-2015-task-2-semantic/)
2. Eneko Agirre, Carmen Banea, Claire Cardie, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Weiwei Guo, Rada Mihalcea, German Rigau, and Janyce Wiebe. 2014. Semeval-2014 task 10: Multilingual semantic textual similarity. In Proc. of SemEval@ACL. [find](https://scholar.google.com/scholar?q=Eneko+Agirre%2C+Carmen+Banea%2C+Claire+Cardie%2C+Daniel+Cer%2C+Mona+Diab%2C+Aitor+Gonzalez%2DAgirre%2C+Weiwei+Guo%2C+Rada+Mihalcea%2C+German+Rigau%2C+and+Janyce+Wiebe%2E+2014%2E+Semeval%2D2014+task+10%3A+Multilingual+semantic+te) [our card](https://md.research.elamur.ai/free/en/papers/agirre-2014-semeval-2014-task-10-multilingual/)
3. Eneko Agirre, Carmen Banea, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, Rada Mihalcea, German Rigau, and Janyce Wiebe. 2016. Semeval-2016 task 1: Semantic textual similarity, monolingual and cross-lingual evaluation. In Proc. of SemEval@ACL. [find](https://scholar.google.com/scholar?q=Eneko+Agirre%2C+Carmen+Banea%2C+Daniel+Cer%2C+Mona+Diab%2C+Aitor+Gonzalez%2DAgirre%2C+Rada+Mihalcea%2C+German+Rigau%2C+and+Janyce+Wiebe%2E+2016%2E+Semeval%2D2016+task+1%3A+Semantic+textual+similarity%2C+monolingual+and+cross%2Dl) [our card](https://md.research.elamur.ai/free/en/papers/agirre-2016-semeval-2016-task-1-semantic/)
4. Eneko Agirre, Daniel Cer, Mona Diab, and Aitor Gonzalez-Agirre. 2012. Semeval-2012 task 6: A pilot on semantic textual similarity. In SemEval@ACL. [find](https://scholar.google.com/scholar?q=Eneko+Agirre%2C+Daniel+Cer%2C+Mona+Diab%2C+and+Aitor+Gonzalez%2DAgirre%2E+2012%2E+Semeval%2D2012+task+6%3A+A+pilot+on+semantic+textual+similarity%2E+In+SemEval%40ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/agirre-2012-semeval-2012-task-6-pilot/)
5. Eneko Agirre, Daniel Cer, Mona Diab, Aitor Gonzalez-Agirre, and Weiwei Guo. 2013. *sem 2013 shared task: Semantic textual similarity. In The Second Joint Conference on Lexical and Computational Semantics. [find](https://scholar.google.com/scholar?q=Eneko+Agirre%2C+Daniel+Cer%2C+Mona+Diab%2C+Aitor+Gonzalez%2DAgirre%2C+and+Weiwei+Guo%2E+2013%2E+%2Asem+2013+shared+task%3A+Semantic+textual+similarity%2E+In+The+Second+Joint+Conference+on+Lexical+and+Computational+Semant) [our card](https://md.research.elamur.ai/free/en/papers/agirre-2013-sem-2013-shared-task-semantic/)
6. Mohamed Ishmael Belghazi, Aristide Baratin, Sai Rajeswar, Sherjil Ozair, Yoshua Bengio, Aaron Courville, and R Devon Hjelm. 2018. Mine: mutual information neural estimation. [find](https://scholar.google.com/scholar?q=Mohamed+Ishmael+Belghazi%2C+Aristide+Baratin%2C+Sai+Rajeswar%2C+Sherjil+Ozair%2C+Yoshua+Bengio%2C+Aaron+Courville%2C+and+R+Devon+Hjelm%2E+2018%2E+Mine%3A+mutual+information+neural+estimation%2E)
7. Alexis Conneau, Douwe Kiela, Holger Schwenk, Loı̈c Barrault, and Antoine Bordes. 2017. Supervised learning of universal sentence representations from natural language inference data. In Proc. of EMNLP. [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Douwe+Kiela%2C+Holger+Schwenk%2C+Lo%C4%B1%CC%88c+Barrault%2C+and+Antoine+Bordes%2E+2017%2E+Supervised+learning+of+universal+sentence+representations+from+natural+language+inference+data%2E+In+Proc%2E+of+EMNLP) [our card](https://md.research.elamur.ai/free/en/papers/conneau-2017-supervised-learning-universal-sentence-representations/)
8. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proc. of NAACL-HLT. [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+BERT%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+In+Proc%2E+of+NAACL%2DHLT%2E)
9. Bill Dolan, Chris Quirk, and Chris Brockett. 2004. Unsupervised construction of large paraphrase corpora: Exploiting massively parallel news sources. In Proc. of COLING. [find](https://scholar.google.com/scholar?q=Bill+Dolan%2C+Chris+Quirk%2C+and+Chris+Brockett%2E+2004%2E+Unsupervised+construction+of+large+paraphrase+corpora%3A+Exploiting+massively+parallel+news+sources%2E+In+Proc%2E+of+COLING%2E) [our card](https://md.research.elamur.ai/free/en/papers/dolan-2004-unsupervised-construction-large-paraphrase-corpora/)
10. Zhe Gan, Yunchen Pu, Ricardo Henao, Chunyuan Li, Xiaodong He, and Lawrence Carin. 2017. Learning generic sentence representations using convolutional neural networks. In Proc. of EMNLP. [find](https://scholar.google.com/scholar?q=Zhe+Gan%2C+Yunchen+Pu%2C+Ricardo+Henao%2C+Chunyuan+Li%2C+Xiaodong+He%2C+and+Lawrence+Carin%2E+2017%2E+Learning+generic+sentence+representations+using+convolutional+neural+networks%2E+In+Proc%2E+of+EMNLP%2E)
11. Michael U Gutmann and Aapo Hyvärinen. 2012. Noise-contrastive estimation of unnormalized statistical models, with applications to natural image statistics. Journal of Machine Learning Research, 13(Feb):307–361. [find](https://scholar.google.com/scholar?q=Michael+U+Gutmann+and+Aapo+Hyva%CC%88rinen%2E+2012%2E+Noise%2Dcontrastive+estimation+of+unnormalized+statistical+models%2C+with+applications+to+natural+image+statistics%2E+Journal+of+Machine+Learning+Research%2C+13%28Fe)
12. Felix Hill, Kyunghyun Cho, and Anna Korhonen. 2016. Learning distributed representations of sentences from unlabelled data. In Proc. of NAACL-HLT. [find](https://scholar.google.com/scholar?q=Felix+Hill%2C+Kyunghyun+Cho%2C+and+Anna+Korhonen%2E+2016%2E+Learning+distributed+representations+of+sentences+from+unlabelled+data%2E+In+Proc%2E+of+NAACL%2DHLT%2E) [our card](https://md.research.elamur.ai/free/en/papers/hill-2016-learning-distributed-representations-sentences-unlabelled/)
13. Devon Hjelm, Alex Fedorov, Samuel Lavoie-Marchildon, Karan Grewal, Philip Bachman, Adam Trischler, and Yoshua Bengio. 2019. Learning deep representations by mutual information estimation and maximization. In Proc. of ICLR. [find](https://scholar.google.com/scholar?q=Devon+Hjelm%2C+Alex+Fedorov%2C+Samuel+Lavoie%2DMarchildon%2C+Karan+Grewal%2C+Philip+Bachman%2C+Adam+Trischler%2C+and+Yoshua+Bengio%2E+2019%2E+Learning+deep+representations+by+mutual+information+estimation+and+maximizat)
14. Minqing Hu and Bing Liu. 2004. Mining and summarizing customer reviews. In Proc. of KDD. [find](https://scholar.google.com/scholar?q=Minqing+Hu+and+Bing+Liu%2E+2004%2E+Mining+and+summarizing+customer+reviews%2E+In+Proc%2E+of+KDD%2E)
15. Anthony J Bell and Terrence J Sejnowski. 1995. An information-maximization approach to blind separation and blind deconvolution. Neural computation, 7(6):1129–1159. [find](https://scholar.google.com/scholar?q=Anthony+J+Bell+and+Terrence+J+Sejnowski%2E+1995%2E+An+information%2Dmaximization+approach+to+blind+separation+and+blind+deconvolution%2E+Neural+computation%2C+7%286%29%3A1129%E2%80%931159%2E)
16. Aapo Hyvärinen and Erkki Oja. 2000. Independent component analysis: algorithms and applications. Neural networks, 13(4-5):411–430. [find](https://scholar.google.com/scholar?q=Aapo+Hyva%CC%88rinen+and+Erkki+Oja%2E+2000%2E+Independent+component+analysis%3A+algorithms+and+applications%2E+Neural+networks%2C+13%284%2D5%29%3A411%E2%80%93430%2E)
17. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In Proc. of EMNLP. [find](https://scholar.google.com/scholar?q=Samuel+R%2E+Bowman%2C+Gabor+Angeli%2C+Christopher+Potts%2C+and+Christopher+D%2E+Manning%2E+2015%2E+A+large+annotated+corpus+for+learning+natural+language+inference%2E+In+Proc%2E+of+EMNLP%2E)
18. Yacine Jernite, Samuel R. Bowman, and David A. Sontag. 2017. Discourse-based objectives for fast unsupervised sentence representation learning. arXiv preprint arXiv:1705.00557. — [arXiv:1705.00557](https://arxiv.org/abs/1705.00557) [find](https://scholar.google.com/scholar?q=Yacine+Jernite%2C+Samuel+R%2E+Bowman%2C+and+David+A%2E+Sontag%2E+2017%2E+Discourse%2Dbased+objectives+for+fast+unsupervised+sentence+representation+learning%2E+arXiv+preprint+arXiv%3A1705%2E00557%2E+%E2%80%94+%5BarXiv%3A1705%2E00557%5D%28ht)
19. Xu Ji, F. João Henriques, and Andrea Vedaldi. 2019. Invariant information clustering for unsupervised image classification and segmentation. arXiv preprint arXiv:1807.06653. — [arXiv:1807.06653](https://arxiv.org/abs/1807.06653) [find](https://scholar.google.com/scholar?q=Xu+Ji%2C+F%2E+Joa%CC%83o+Henriques%2C+and+Andrea+Vedaldi%2E+2019%2E+Invariant+information+clustering+for+unsupervised+image+classification+and+segmentation%2E+arXiv+preprint+arXiv%3A1807%2E06653%2E+%E2%80%94+%5BarXiv%3A1807%2E06653%5D%28http)
20. Yoon Kim. 2014. Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882. [find](https://scholar.google.com/scholar?q=Yoon+Kim%2E+2014%2E+Convolutional+neural+networks+for+sentence+classification%2E+arXiv+preprint+arXiv%3A1408%2E5882%2E) [our card](https://md.research.elamur.ai/free/en/papers/kim-2014-convolutional-neural-networks-sentence-classification/)
21. Ryan Kiros, Yukun Zhu, Russ R Salakhutdinov, Richard Zemel, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015. Skip-thought vectors. In Proc. of NIPS. [find](https://scholar.google.com/scholar?q=Ryan+Kiros%2C+Yukun+Zhu%2C+Russ+R+Salakhutdinov%2C+Richard+Zemel%2C+Raquel+Urtasun%2C+Antonio+Torralba%2C+and+Sanja+Fidler%2E+2015%2E+Skip%2Dthought+vectors%2E+In+Proc%2E+of+NIPS%2E)
22. Lingpeng Kong, Cyprien de Masson d’Autume, Lei Yu, Wang Ling, Zihang Dai, and Dani Yogatama. 2020. A mutual information maximization perspective of language representation learning. In Proc. of ICLR. [find](https://scholar.google.com/scholar?q=Lingpeng+Kong%2C+Cyprien+de+Masson+d%E2%80%99Autume%2C+Lei+Yu%2C+Wang+Ling%2C+Zihang+Dai%2C+and+Dani+Yogatama%2E+2020%2E+A+mutual+information+maximization+perspective+of+language+representation+learning%2E+In+Proc%2E+of+ICLR%2E)
23. Quoc V. Le and Tomas Mikolov. 2014. Distributed representations of sentences and documents. In Proc. of ICML. [find](https://scholar.google.com/scholar?q=Quoc+V%2E+Le+and+Tomas+Mikolov%2E+2014%2E+Distributed+representations+of+sentences+and+documents%2E+In+Proc%2E+of+ICML%2E) [our card](https://md.research.elamur.ai/free/en/papers/le-2014-distributed-representations-sentences-documents/)
24. Xin Li and Dan Roth. 2002. Learning question classifiers. In Proc. of COLING. [find](https://scholar.google.com/scholar?q=Xin+Li+and+Dan+Roth%2E+2002%2E+Learning+question+classifiers%2E+In+Proc%2E+of+COLING%2E) [our card](https://md.research.elamur.ai/free/en/papers/li-2002-learning-question-classifiers/)
25. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. [find](https://scholar.google.com/scholar?q=Yinhan+Liu%2C+Myle+Ott%2C+Naman+Goyal%2C+Jingfei+Du%2C+Mandar+Joshi%2C+Danqi+Chen%2C+Omer+Levy%2C+Mike+Lewis%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2019%2E+Roberta%3A+A+robustly+optimized+bert+pretraining+approach%2E+a)
26. Lajanugen Logeswaran and Honglak Lee. 2018. An efficient framework for learning sentence representations. In Proc. of ICLR. [find](https://scholar.google.com/scholar?q=Lajanugen+Logeswaran+and+Honglak+Lee%2E+2018%2E+An+efficient+framework+for+learning+sentence+representations%2E+In+Proc%2E+of+ICLR%2E) [our card](https://md.research.elamur.ai/free/en/papers/logeswaran-2018-efficient-framework-learning-sentence-representations/)
27. Marco Marelli, Stefano Menini, Marco Baroni, Luisa Bentivogli, Raffaella Bernardi, and Roberto Zamparelli. 2014. A SICK cure for the evaluation of compositional distributional semantic models. In Proc. of LREC. [find](https://scholar.google.com/scholar?q=Marco+Marelli%2C+Stefano+Menini%2C+Marco+Baroni%2C+Luisa+Bentivogli%2C+Raffaella+Bernardi%2C+and+Roberto+Zamparelli%2E+2014%2E+A+SICK+cure+for+the+evaluation+of+compositional+distributional+semantic+models%2E+In+Proc)
28. Amita Misra, Brian Ecker, and Marilyn Walker. 2016. Measuring the similarity of sentential arguments in dialogue. In Proc. of the 17th Annual Meeting of the Special Interest Group on Discourse and Dialogue. [find](https://scholar.google.com/scholar?q=Amita+Misra%2C+Brian+Ecker%2C+and+Marilyn+Walker%2E+2016%2E+Measuring+the+similarity+of+sentential+arguments+in+dialogue%2E+In+Proc%2E+of+the+17th+Annual+Meeting+of+the+Special+Interest+Group+on+Discourse+and+Dia)
29. Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. 2016. f-gan: Training generative neural samplers using variational divergence minimization. In Proc. of NIPS. [find](https://scholar.google.com/scholar?q=Sebastian+Nowozin%2C+Botond+Cseke%2C+and+Ryota+Tomioka%2E+2016%2E+f%2Dgan%3A+Training+generative+neural+samplers+using+variational+divergence+minimization%2E+In+Proc%2E+of+NIPS%2E)
30. Aäron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748. [find](https://scholar.google.com/scholar?q=Aa%CC%88ron+van+den+Oord%2C+Yazhe+Li%2C+and+Oriol+Vinyals%2E+2018%2E+Representation+learning+with+contrastive+predictive+coding%2E+arXiv+preprint+arXiv%3A1807%2E03748%2E) [our card](https://md.research.elamur.ai/free/en/papers/oord-2018-representation-learning-contrastive-predictive-coding/)
31. Matteo Pagliardini, Prakhar Gupta, and Martin Jaggi. 2017. Unsupervised learning of sentence embeddings using compositional n-gram features. arXiv preprint arXiv:1703.02507. [find](https://scholar.google.com/scholar?q=Matteo+Pagliardini%2C+Prakhar+Gupta%2C+and+Martin+Jaggi%2E+2017%2E+Unsupervised+learning+of+sentence+embeddings+using+compositional+n%2Dgram+features%2E+arXiv+preprint+arXiv%3A1703%2E02507%2E) [our card](https://md.research.elamur.ai/free/en/papers/pagliardini-2018-unsupervised-learning-sentence-embeddings-compositional/)
32. Bo Pang and Lillian Lee. 2005. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In Proc. of ACL. [find](https://scholar.google.com/scholar?q=Bo+Pang+and+Lillian+Lee%2E+2005%2E+Seeing+stars%3A+Exploiting+class+relationships+for+sentiment+categorization+with+respect+to+rating+scales%2E+In+Proc%2E+of+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/pang-2005-seeing-stars-exploiting-class-relationships/)
33. Nils Reimers, Philip Beyer, and Iryna Gurevych. 2016. Task-oriented intrinsic evaluation of semantic textual similarity. In Proc. of COLING. [find](https://scholar.google.com/scholar?q=Nils+Reimers%2C+Philip+Beyer%2C+and+Iryna+Gurevych%2E+2016%2E+Task%2Doriented+intrinsic+evaluation+of+semantic+textual+similarity%2E+In+Proc%2E+of+COLING%2E)
34. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proc. of EMNLP-IJCNLP. [find](https://scholar.google.com/scholar?q=Nils+Reimers+and+Iryna+Gurevych%2E+2019%2E+Sentence%2DBERT%3A+Sentence+embeddings+using+Siamese+BERT%2Dnetworks%2E+In+Proc%2E+of+EMNLP%2DIJCNLP%2E)
35. Nils Reimers, Benjamin Schiller, Tilman Beck, Johannes Daxenberger, Christian Stab, and Iryna Gurevych. 2019. Classification and clustering of arguments with contextualized word embeddings. In Proc. of ACL. [find](https://scholar.google.com/scholar?q=Nils+Reimers%2C+Benjamin+Schiller%2C+Tilman+Beck%2C+Johannes+Daxenberger%2C+Christian+Stab%2C+and+Iryna+Gurevych%2E+2019%2E+Classification+and+clustering+of+arguments+with+contextualized+word+embeddings%2E+In+Proc%2E+o) [our card](https://md.research.elamur.ai/free/en/papers/reimers-2019-classification-clustering-arguments-contextualized-word/)
36. Richard Socher, Eric H. Huang, Jeffrey Pennin, Christopher D Manning, and Andrew Y. Ng. 2011. Dynamic pooling and unfolding recursive autoencoders for paraphrase detection. In Proc. of NIPS. [find](https://scholar.google.com/scholar?q=Richard+Socher%2C+Eric+H%2E+Huang%2C+Jeffrey+Pennin%2C+Christopher+D+Manning%2C+and+Andrew+Y%2E+Ng%2E+2011%2E+Dynamic+pooling+and+unfolding+recursive+autoencoders+for+paraphrase+detection%2E+In+Proc%2E+of+NIPS%2E)
37. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proc. of EMNLP. [find](https://scholar.google.com/scholar?q=Richard+Socher%2C+Alex+Perelygin%2C+Jean+Wu%2C+Jason+Chuang%2C+Christopher+D%2E+Manning%2C+Andrew+Ng%2C+and+Christopher+Potts%2E+2013%2E+Recursive+deep+models+for+semantic+compositionality+over+a+sentiment+treebank%2E+In)
38. Fan-Yun Sun, Jordan Hoffmann, and Jian Tang. 2020. Infograph: Unsupervised and semi-supervised graph-level representation learning via mutual information maximization. In Proc. of ICLR. [find](https://scholar.google.com/scholar?q=Fan%2DYun+Sun%2C+Jordan+Hoffmann%2C+and+Jian+Tang%2E+2020%2E+Infograph%3A+Unsupervised+and+semi%2Dsupervised+graph%2Dlevel+representation+learning+via+mutual+information+maximization%2E+In+Proc%2E+of+ICLR%2E)
39. Petar Velickovic, William Fedus, William L. Hamilton, Pietro Liò, Yoshua Bengio, and R. Devon Hjelm. 2019. Deep graph infomax. In Proc. of ICLR. [find](https://scholar.google.com/scholar?q=Petar+Velickovic%2C+William+Fedus%2C+William+L%2E+Hamilton%2C+Pietro+Lio%CC%80%2C+Yoshua+Bengio%2C+and+R%2E+Devon+Hjelm%2E+2019%2E+Deep+graph+infomax%2E+In+Proc%2E+of+ICLR%2E)
40. Janyce Wiebe, Theresa Wilson, and Claire Cardie. 2005. Annotating expressions of opinions and emotions in language. Language Resources and Evaluation. [find](https://scholar.google.com/scholar?q=Janyce+Wiebe%2C+Theresa+Wilson%2C+and+Claire+Cardie%2E+2005%2E+Annotating+expressions+of+opinions+and+emotions+in+language%2E+Language+Resources+and+Evaluation%2E)
41. Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proc. of NAACL-HLT. [find](https://scholar.google.com/scholar?q=Adina+Williams%2C+Nikita+Nangia%2C+and+Samuel+Bowman%2E+2018%2E+A+broad%2Dcoverage+challenge+corpus+for+sentence+understanding+through+inference%2E+In+Proc%2E+of+NAACL%2DHLT%2E)
42. Yinfei Yang, Steve Yuan, Daniel Matthew Cer, Sheng yi Kong, Noah Constant, P. Pilar, Heming Ge, Yun-Hsuan Sung, B. Strope, and R. Kurzweil. 2018. Learning semantic textual similarity from conversations. In Proc. of RepL4NLP@ACL. [find](https://scholar.google.com/scholar?q=Yinfei+Yang%2C+Steve+Yuan%2C+Daniel+Matthew+Cer%2C+Sheng+yi+Kong%2C+Noah+Constant%2C+P%2E+Pilar%2C+Heming+Ge%2C+Yun%2DHsuan+Sung%2C+B%2E+Strope%2C+and+R%2E+Kurzweil%2E+2018%2E+Learning+semantic+textual+similarity+from+conversation)
43. Yukun Zhu, Ryan Kiros, S. Richard Zemel, Ruslan Salakhutdinov, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. 2015. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. In Proc. of ICCV. [find](https://scholar.google.com/scholar?q=Yukun+Zhu%2C+Ryan+Kiros%2C+S%2E+Richard+Zemel%2C+Ruslan+Salakhutdinov%2C+Raquel+Urtasun%2C+Antonio+Torralba%2C+and+Sanja+Fidler%2E+2015%2E+Aligning+books+and+movies%3A+Towards+story%2Dlike+visual+explanations+by+watching+m) [our card](https://md.research.elamur.ai/free/en/papers/zhu-2015-aligning-books-movies-towards-story/)
44. Bo Pang and Lillian Lee. 2004. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In Proc. of ACL. [find](https://scholar.google.com/scholar?q=Bo+Pang+and+Lillian+Lee%2E+2004%2E+A+sentimental+education%3A+Sentiment+analysis+using+subjectivity+summarization+based+on+minimum+cuts%2E+In+Proc%2E+of+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/pang-2004-sentimental-education-sentiment-analysis-subjectivity/)