---
title: "Distributed Representations of Sentences and — Le 2014"
description: "Classifiers and clustering algorithms typically need each text as a fixed-size vector."
authors: ["Quoc V. Le", "Tomas Mikolov"]
year: 2014
venue: "arXiv:1405.4053"
source: "https://arxiv.org/abs/1405.4053"
doi: "10.48550/arxiv.1405.4053"
arxiv: "1405.4053"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/le-2014-distributed-representations-sentences-documents/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/le-2014-distributed-representations-sentences-documents/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2014](https://md.research.elamur.ai/free/en/papers/2014/) / Distributed Representations of Sentences and Documents

# Distributed Representations of Sentences and Documents

**Authors:** [Quoc V. Le](https://md.research.elamur.ai/free/en/authors/quoc-v-le/), [Tomas Mikolov](https://md.research.elamur.ai/free/en/authors/tomas-mikolov/) · **Year:** 2014 · **Venue:** arXiv:1405.4053 · **Licence:** [arXiv — non-exclusive licence to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1405.4053) · [PDF](https://arxiv.org/pdf/1405.4053) · [DOI 10.48550/arxiv.1405.4053](https://doi.org/10.48550/arxiv.1405.4053) · [arXiv 1405.4053](https://arxiv.org/abs/1405.4053) · [OpenAlex](https://openalex.org/W2131744502)

**Citations:** 4934 citations (openalex-snapshot · 2026-10-14) · [811](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Classifiers and clustering algorithms typically need each text as a fixed-size vector. The standard bag-of-words or bag-of-n-grams encoding discards word order and treats all words as equally dissimilar. Le and Mikolov propose Paragraph Vector, an unsupervised method that gives every sentence, paragraph or document its own dense vector. That vector is trained together with shared word vectors to help predict the words of its text. In the PV-DM variant, the document vector is concatenated with a window of context words to predict the next word. In PV-DBOW, the document vector alone predicts words sampled from the text. For an unseen text, a vector is fitted by gradient descent while the word vectors and softmax weights stay frozen. The result then serves as input to ordinary classifiers such as logistic regression. On the Stanford Sentiment Treebank, IMDB reviews and a retrieval task built from search-result snippets, it beat bag-of-words baselines and set state-of-the-art sentiment results at the time. The approach, later popularised as doc2vec, learns its representations without a parser or labelled data.

## Key points
- **Two architectures:** PV-DM (Distributed Memory) and PV-DBOW (Distributed Bag of Words). PV-DM alone usually does better, but the authors recommend combining both. Within PV-DM, concatenating vectors works better than summing them (IMDB error 7.63% vs 8.06%).
- **IMDB:** the combined model reached 7.42% error. Inferring vectors for the 25,000 test reviews (about 230 words each on average) took roughly 30 minutes on a 16-core machine.
- **Stanford Sentiment Treebank:** about 12.2% error on the binary task and 51.3% on the fine-grained task. The authors report a relative error reduction of more than 16% over earlier methods, including parse-based recursive models.
- **Retrieval triplet task** (snippets from search results for popular queries): Paragraph Vector had 3.82% error, compared with 5.67% for weighted bag-of-bigrams, 7.28% for bag-of-bigrams, 8.10% for bag-of-words and 10.25% for averaged word vectors. That is roughly a 32% relative improvement.
- **Practical notes:** the context window size should be tuned by cross-validation, with 5–12 words a reasonable range. Training uses hierarchical softmax over a Huffman tree. The model has N×p + M×q parameters, but updates are sparse, so training stays efficient.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Other](https://md.research.elamur.ai/free/en/shelves/other/)

## Cited by

Cited by

811 papers of our library cite it (of the 140673 whose bibliographies we read); 53 of them have a page here, 758 not yet.

- [Automatic Text Scoring Using Neural Networks](https://md.research.elamur.ai/free/en/papers/alikaniotis-2016-automatic-text-scoring-neural-networks/)
- [Effect of Word Embedding Variable Parameters on Arabic Sentiment Analysis Performance](https://md.research.elamur.ai/free/en/papers/alnawas-2021-effect-word-embedding-variable-parameters/)
- [Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond](https://md.research.elamur.ai/free/en/papers/artetxe-2018-massively-multilingual-sentence-embeddings-zero/)
- [Findings of the WMT 2019 Biomedical Translation Shared Task: Evaluation for MEDLINE Abstracts and Biomedical Terminologies](https://md.research.elamur.ai/free/en/papers/bawden-2019-findings-wmt-2019-biomedical-translation/)
- [Generating Sentences from a Continuous Space](https://md.research.elamur.ai/free/en/papers/bowman-2016-generating-sentences-continuous-space/)
- [Long Short-Term Memory-Networks for Machine Reading](https://md.research.elamur.ai/free/en/papers/cheng-2016-long-short-term-memory-networks/)
- [SPECTER: Document-level Representation Learning using Citation-informed Transformers](https://md.research.elamur.ai/free/en/papers/cohan-2020-specter-document-level-representation-learning/)
- [GLaM: Efficient Scaling of Language Models with Mixture-of-Experts](https://md.research.elamur.ai/free/en/papers/du-2021-glam-efficient-scaling-language-models/)
- [DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations](https://md.research.elamur.ai/free/en/papers/giorgi-2021-declutr-deep-contrastive-learning-unsupervised/)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://md.research.elamur.ai/free/en/papers/grootendorst-2022-bertopic-neural-topic-modeling-class/)
- [Arabic natural language processing: An overview](https://md.research.elamur.ai/free/en/papers/guellil-2019-arabic-natural-language-processing-overview/)
- [Parameter-Efficient Transfer Learning with Diff Pruning](https://md.research.elamur.ai/free/en/papers/guo-2021-parameter-efficient-transfer-learning-diff/)
- [Argumentation Mining in User-Generated Web Discourse](https://md.research.elamur.ai/free/en/papers/habernal-2017-argumentation-mining-user-generated-web/)
- [CASCADE: Contextual Sarcasm Detection in Online Discussion Forums](https://md.research.elamur.ai/free/en/papers/hazarika-2018-cascade-contextual-sarcasm-detection-online/)
- [Learning Distributed Representations of Sentences from Unlabelled Data](https://md.research.elamur.ai/free/en/papers/hill-2016-learning-distributed-representations-sentences-unlabelled/)
- [Harnessing Deep Neural Networks with Logic Rules](https://md.research.elamur.ai/free/en/papers/hu-2016-harnessing-deep-neural-networks-logic/)
- [Deep Unordered Composition Rivals Syntactic Methods for Text Classification](https://md.research.elamur.ai/free/en/papers/iyyer-2015-deep-unordered-composition-rivals-syntactic/)
- [Effective Use of Word Order for Text Categorization with Convolutional Neural Networks](https://md.research.elamur.ai/free/en/papers/johnson-2015-effective-use-word-order-text/)
- [Representing Social Media Users for Sarcasm Detection](https://md.research.elamur.ai/free/en/papers/kolchinski-2018-representing-social-media-users-sarcasm/)
- [Ask Me Anything: Dynamic Memory Networks for Natural Language Processing](https://md.research.elamur.ai/free/en/papers/kumar-2015-ask-me-anything-dynamic-memory/)
- [A Structured Self-attentive Sentence Embedding](https://md.research.elamur.ai/free/en/papers/lin-2017-structured-self-attentive-sentence-embedding/)
- [Representation Learning Using Multi-Task Deep Neural Networks for Semantic Classification and Information Retrieval](https://md.research.elamur.ai/free/en/papers/liu-2015-representation-learning-multi-task-deep/)
- [An efficient framework for learning sentence representations](https://md.research.elamur.ai/free/en/papers/logeswaran-2018-efficient-framework-learning-sentence-representations/)
- [EmoNet: Fine-Grained Emotion Detection with Gated Recurrent Neural Networks](https://md.research.elamur.ai/free/en/papers/mageed-2017-emonet-fine-grained-emotion-detection/)
- [Siamese AraBERT-LSTM Model based Approach for Arabic Paraphrase Detection](https://md.research.elamur.ai/free/en/papers/mahmoud-2022-siamese-arabert-lstm-model-based/)
- [ILDC for CJPE: Indian Legal Documents Corpus for Court Judgment Prediction and Explanation](https://md.research.elamur.ai/free/en/papers/malik-2021-ildc-cjpe-indian-legal-documents/)
- [Adversarial Training Methods for Semi-Supervised Text Classification](https://md.research.elamur.ai/free/en/papers/miyato-2016-adversarial-training-methods-semi-supervised/)
- [Efficient Non-parametric Estimation of Multiple Embeddings per Word in Vector Space](https://md.research.elamur.ai/free/en/papers/neelakantan-2014-efficient-non-parametric-estimation-multiple/)
- [Representation Learning with Contrastive Predictive Coding](https://md.research.elamur.ai/free/en/papers/oord-2018-representation-learning-contrastive-predictive-coding/)
- [Exploring Author Context for Detecting Intended vs Perceived Sarcasm](https://md.research.elamur.ai/free/en/papers/oprea-2019-exploring-author-context-detecting-intended/)
- [iSarcasm: A Dataset of Intended Sarcasm](https://md.research.elamur.ai/free/en/papers/oprea-2020-isarcasm-dataset-intended-sarcasm/)
- [Semi-supervised sequence tagging with bidirectional language models](https://md.research.elamur.ai/free/en/papers/peters-2017-semi-supervised-sequence-tagging-bidirectional/)
- [When Numbers Tell Half the Story: Human-Metric Alignment in Topic Model Evaluation](https://md.research.elamur.ai/free/en/papers/prouteau-2026-when-numbers-tell-half-story/)
- [Pre-trained Models for Natural Language Processing: A Survey](https://md.research.elamur.ai/free/en/papers/qiu-2020-pre-trained-models-natural-language/)
- [Learning Transferable Visual Models From Natural Language Supervision](https://md.research.elamur.ai/free/en/papers/radford-2021-learning-transferable-visual-models-natural/)
- [Dialogue Act Classification with Context-Aware Self-Attention](https://md.research.elamur.ai/free/en/papers/raheja-2019-dialogue-act-classification-context-aware/)
- [A Systematic Study of Neural Discourse Models for Implicit Discourse Relation](https://md.research.elamur.ai/free/en/papers/rutherford-2017-systematic-study-neural-discourse-models/)
- [Generating Datasets with Pretrained Language Models](https://md.research.elamur.ai/free/en/papers/schick-2021-generating-datasets-pretrained-language-models/)
- [A Survey on Hate Speech Detection using Natural Language Processing](https://md.research.elamur.ai/free/en/papers/schmidt-2017-survey-hate-speech-detection-natural/)
- [Does String-Based Neural MT Learn Source Syntax?](https://md.research.elamur.ai/free/en/papers/shi-2016-does-string-based-neural-mt/)
- [Offensive Language and Hate Speech Detection for Danish](https://md.research.elamur.ai/free/en/papers/sigurbergsson-2020-offensive-language-hate-speech-detection/)
- [Offline bilingual word vectors, orthogonal transformations and the inverted softmax](https://md.research.elamur.ai/free/en/papers/smith-2017-offline-bilingual-word-vectors-orthogonal/)
- [MASS: Masked Sequence to Sequence Pre-training for Language Generation](https://md.research.elamur.ai/free/en/papers/song-2019-mass-masked-sequence-sequence-pre/)
- [Findings of the WMT 2018 Shared Task on Quality Estimation](https://md.research.elamur.ai/free/en/papers/specia-2018-findings-wmt-2018-shared-task/)
- [How to Fine-Tune BERT for Text Classification?](https://md.research.elamur.ai/free/en/papers/sun-2019-fine-tune-bert-text-classification/)
- [Document Modeling with Gated Recurrent Neural Network for Sentiment Classification](https://md.research.elamur.ai/free/en/papers/tang-2015-document-modeling-gated-recurrent-neural/)
- [Aspect Level Sentiment Classification with Deep Memory Network](https://md.research.elamur.ai/free/en/papers/tang-2016-aspect-level-sentiment-classification-deep/)
- [Effective LSTMs for Target-Dependent Sentiment Classification](https://md.research.elamur.ai/free/en/papers/tang-2016-effective-lstms-target-dependent-sentiment/)
- [Recursive Neural Conditional Random Fields for Aspect-based Sentiment Analysis](https://md.research.elamur.ai/free/en/papers/wang-2016-recursive-neural-conditional-random-fields/)
- [Beyond BLEU: Training Neural Machine Translation with Semantic Similarity](https://md.research.elamur.ai/free/en/papers/wieting-2019-beyond-bleu-training-neural-machine/)
- [The Creation and Analysis of a Website Privacy Policy Corpus](https://md.research.elamur.ai/free/en/papers/wilson-2016-creation-analysis-website-privacy-policy/)
- [WikiQA: A Challenge Dataset for Open-Domain Question Answering](https://md.research.elamur.ai/free/en/papers/yang-2015-wikiqa-challenge-dataset-open-domain/)
- [An Unsupervised Sentence Embedding Method by Mutual Information Maximization](https://md.research.elamur.ai/free/en/papers/zhang-2020-unsupervised-sentence-embedding-method-mutual/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{le2014distributed,
  title = {{Distributed Representations of Sentences and Documents}},
  author = {Le, Quoc V. and Mikolov, Tomas},
  year = {2014},
  doi = {10.48550/arxiv.1405.4053},
  eprint = {1405.4053},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1405.4053}
}
```

- [← previous Discriminative Reranking of Discourse Parses Using Tree Kernels](https://md.research.elamur.ai/free/en/papers/joty-2014-discriminative-reranking-discourse-parses-tree/)
- [next → Domain Adaptation with Active Learning for Coreference Resolution](https://md.research.elamur.ai/free/en/papers/zhao-2014-domain-adaptation-active-learning-coreference/)
- [↑ back to papers of 2014](https://md.research.elamur.ai/free/en/papers/2014/)

## References

38 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Bengio, Yoshua, Schwenk, Holger, Senécal, Jean-Sébastien, Morin, Fréderic, and Gauvain, Jean-Luc. Neural probabilistic language models. In Innovations in Machine Learning, pp. 137–186. Springer, 2006. [find](https://scholar.google.com/scholar?q=Bengio%2C+Yoshua%2C+Schwenk%2C+Holger%2C+Sene%CC%81cal%2C+Jean%2DSe%CC%81bastien%2C+Morin%2C+Fre%CC%81deric%2C+and+Gauvain%2C+Jean%2DLuc%2E+Neural+probabilistic+language+models%2E+In+Innovations+in+Machine+Learning%2C+pp%2E+137%E2%80%93186%2E+Springer%2C+20)
2. Collobert, Ronan and Weston, Jason. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th International Conference on Machine Learning, pp. 160– 167. ACM, 2008. [find](https://scholar.google.com/scholar?q=Collobert%2C+Ronan+and+Weston%2C+Jason%2E+A+unified+architecture+for+natural+language+processing%3A+Deep+neural+networks+with+multitask+learning%2E+In+Proceedings+of+the+25th+International+Conference+on+Machine)
3. Collobert, Ronan, Weston, Jason, Bottou, Léon, Karlen, Michael, Kavukcuoglu, Koray, and Kuksa, Pavel. Natural language processing (almost) from scratch. The Journal of Machine Learning Research, 12:2493–2537, 2011. [find](https://scholar.google.com/scholar?q=Collobert%2C+Ronan%2C+Weston%2C+Jason%2C+Bottou%2C+Le%CC%81on%2C+Karlen%2C+Michael%2C+Kavukcuoglu%2C+Koray%2C+and+Kuksa%2C+Pavel%2E+Natural+language+processing+%28almost%29+from+scratch%2E+The+Journal+of+Machine+Learning+Research%2C+12%3A2) [our card](https://md.research.elamur.ai/free/en/papers/collobert-2011-natural-language-processing-almost-scratch/)
4. Dahl, George E., Adams, Ryan P., and Larochelle, Hugo. Training Restricted Boltzmann Machines on word observations. In International Conference on Machine Learning, 2012. [find](https://scholar.google.com/scholar?q=Dahl%2C+George+E%2E%2C+Adams%2C+Ryan+P%2E%2C+and+Larochelle%2C+Hugo%2E+Training+Restricted+Boltzmann+Machines+on+word+observations%2E+In+International+Conference+on+Machine+Learning%2C+2012%2E)
5. Elman, Jeff. Finding structure in time. In Cognitive Science, pp. 179–211, 1990. [find](https://scholar.google.com/scholar?q=Elman%2C+Jeff%2E+Finding+structure+in+time%2E+In+Cognitive+Science%2C+pp%2E+179%E2%80%93211%2C+1990%2E)
6. Frome, Andrea, Corrado, Greg S., Shlens, Jonathon, Bengio, Samy, Dean, Jeffrey, Ranzato, Marc’Aurelio, and Mikolov, Tomas. DeViSE: A deep visual-semantic embedding model. In Advances in Neural Information Processing Systems, 2013. [find](https://scholar.google.com/scholar?q=Frome%2C+Andrea%2C+Corrado%2C+Greg+S%2E%2C+Shlens%2C+Jonathon%2C+Bengio%2C+Samy%2C+Dean%2C+Jeffrey%2C+Ranzato%2C+Marc%E2%80%99Aurelio%2C+and+Mikolov%2C+Tomas%2E+DeViSE%3A+A+deep+visual%2Dsemantic+embedding+model%2E+In+Advances+in+Neural+Informa)
7. Grefenstette, E., Dinu, G., Zhang, Y., Sadrzadeh, M., and Baroni, M. Multi-step regression learning for compositional distributional semantics. In Conference on Empirical Methods in Natural Language Processing, 2013. [find](https://scholar.google.com/scholar?q=Grefenstette%2C+E%2E%2C+Dinu%2C+G%2E%2C+Zhang%2C+Y%2E%2C+Sadrzadeh%2C+M%2E%2C+and+Baroni%2C+M%2E+Multi%2Dstep+regression+learning+for+compositional+distributional+semantics%2E+In+Conference+on+Empirical+Methods+in+Natural+Language+P)
8. Harris, Zellig. Distributional structure. Word, 1954. [find](https://scholar.google.com/scholar?q=Harris%2C+Zellig%2E+Distributional+structure%2E+Word%2C+1954%2E)
9. Huang, Eric, Socher, Richard, Manning, Christopher, and Ng, Andrew Y. Improving word representations via global context and multiple word prototypes. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Long Papers-Volume 1, pp. 873–882. Association for Computa… [find](https://scholar.google.com/scholar?q=Huang%2C+Eric%2C+Socher%2C+Richard%2C+Manning%2C+Christopher%2C+and+Ng%2C+Andrew+Y%2E+Improving+word+representations+via+global+context+and+multiple+word+prototypes%2E+In+Proceedings+of+the+50th+Annual+Meeting+of+the+A) [our card](https://md.research.elamur.ai/free/en/papers/huang-2012-improving-word-representations-global-context/)
10. Jaakkola, Tommi and Haussler, David. Exploiting generative models in discriminative classifiers. In Advances in Neural Information Processing Systems 11, pp. 487– 493, 1999. [find](https://scholar.google.com/scholar?q=Jaakkola%2C+Tommi+and+Haussler%2C+David%2E+Exploiting+generative+models+in+discriminative+classifiers%2E+In+Advances+in+Neural+Information+Processing+Systems+11%2C+pp%2E+487%E2%80%93+493%2C+1999%2E)
11. Klein, Dan and Manning, Chris D. Accurate unlexicalized parsing. In Proceedings of Association for Computational Linguistics, 2003. [find](https://scholar.google.com/scholar?q=Klein%2C+Dan+and+Manning%2C+Chris+D%2E+Accurate+unlexicalized+parsing%2E+In+Proceedings+of+Association+for+Computational+Linguistics%2C+2003%2E) [our card](https://md.research.elamur.ai/free/en/papers/aclbulk-2003-klein-accurate-unlexicalized-parsing/)
12. Larochelle, Hugo and Lauly, Stanislas. A neural autoregressive topic model. In Advances in Neural Information Processing Systems, 2012. [find](https://scholar.google.com/scholar?q=Larochelle%2C+Hugo+and+Lauly%2C+Stanislas%2E+A+neural+autoregressive+topic+model%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+2012%2E)
13. Maas, Andrew L., Daly, Raymond E., Pham, Peter T., Huang, Dan, Ng, Andrew Y., and Potts, Christopher. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, 2011. [find](https://scholar.google.com/scholar?q=Maas%2C+Andrew+L%2E%2C+Daly%2C+Raymond+E%2E%2C+Pham%2C+Peter+T%2E%2C+Huang%2C+Dan%2C+Ng%2C+Andrew+Y%2E%2C+and+Potts%2C+Christopher%2E+Learning+word+vectors+for+sentiment+analysis%2E+In+Proceedings+of+the+49th+Annual+Meeting+of+the+Ass) [our card](https://md.research.elamur.ai/free/en/papers/maas-2011-learning-word-vectors-sentiment-analysis/)
14. Mikolov, Tomas. Statistical Language Models based on Neural Networks. PhD thesis, Brno University of Technology, 2012. [find](https://scholar.google.com/scholar?q=Mikolov%2C+Tomas%2E+Statistical+Language+Models+based+on+Neural+Networks%2E+PhD+thesis%2C+Brno+University+of+Technology%2C+2012%2E)
15. Mikolov, Tomas, Chen, Kai, Corrado, Greg, and Dean, Jeffrey. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781, 2013a. [find](https://scholar.google.com/scholar?q=Mikolov%2C+Tomas%2C+Chen%2C+Kai%2C+Corrado%2C+Greg%2C+and+Dean%2C+Jeffrey%2E+Efficient+estimation+of+word+representations+in+vector+space%2E+arXiv+preprint+arXiv%3A1301%2E3781%2C+2013a%2E)
16. Mikolov, Tomas, Le, Quoc V., and Sutskever, Ilya. Exploiting similarities among languages for machine translation. CoRR, abs/1309.4168, 2013b. [find](https://scholar.google.com/scholar?q=Mikolov%2C+Tomas%2C+Le%2C+Quoc+V%2E%2C+and+Sutskever%2C+Ilya%2E+Exploiting+similarities+among+languages+for+machine+translation%2E+CoRR%2C+abs%2F1309%2E4168%2C+2013b%2E)
17. Mikolov, Tomas, Sutskever, Ilya, Chen, Kai, Corrado, Greg, and Dean, Jeffrey. Distributed representations of phrases and their compositionality. In Advances on Neural Information Processing Systems, 2013c. [find](https://scholar.google.com/scholar?q=Mikolov%2C+Tomas%2C+Sutskever%2C+Ilya%2C+Chen%2C+Kai%2C+Corrado%2C+Greg%2C+and+Dean%2C+Jeffrey%2E+Distributed+representations+of+phrases+and+their+compositionality%2E+In+Advances+on+Neural+Information+Processing+Systems%2C+2)
18. Mikolov, Tomas, Yih, Scott Wen-tau, and Zweig, Geoffrey. Linguistic regularities in continuous space word representations. In NAACL HLT, 2013d. [find](https://scholar.google.com/scholar?q=Mikolov%2C+Tomas%2C+Yih%2C+Scott+Wen%2Dtau%2C+and+Zweig%2C+Geoffrey%2E+Linguistic+regularities+in+continuous+space+word+representations%2E+In+NAACL+HLT%2C+2013d%2E) [our card](https://md.research.elamur.ai/free/en/papers/mikolov-2013-linguistic-regularities-continuous-space-word/)
19. Mitchell, Jeff and Lapata, Mirella. Composition in distributional models of semantics. Cognitive Science, 2010. [find](https://scholar.google.com/scholar?q=Mitchell%2C+Jeff+and+Lapata%2C+Mirella%2E+Composition+in+distributional+models+of+semantics%2E+Cognitive+Science%2C+2010%2E)
20. Mnih, Andriy and Hinton, Geoffrey E. A scalable hierarchical distributed language model. In Advances in Neural Information Processing Systems, pp. 1081–1088, 2008. [find](https://scholar.google.com/scholar?q=Mnih%2C+Andriy+and+Hinton%2C+Geoffrey+E%2E+A+scalable+hierarchical+distributed+language+model%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pp%2E+1081%E2%80%931088%2C+2008%2E)
21. Morin, Frederic and Bengio, Yoshua. Hierarchical probabilistic neural network language model. In Proceedings of the International Workshop on Artificial Intelligence and Statistics, pp. 246–252, 2005. [find](https://scholar.google.com/scholar?q=Morin%2C+Frederic+and+Bengio%2C+Yoshua%2E+Hierarchical+probabilistic+neural+network+language+model%2E+In+Proceedings+of+the+International+Workshop+on+Artificial+Intelligence+and+Statistics%2C+pp%2E+246%E2%80%93252%2C+2005%2E)
22. Pang, Bo and Lee, Lillian. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In Proceedings of Association for Computational Linguistics, pp. 115–124, 2005. [find](https://scholar.google.com/scholar?q=Pang%2C+Bo+and+Lee%2C+Lillian%2E+Seeing+stars%3A+Exploiting+class+relationships+for+sentiment+categorization+with+respect+to+rating+scales%2E+In+Proceedings+of+Association+for+Computational+Linguistics%2C+pp%2E+115) [our card](https://md.research.elamur.ai/free/en/papers/pang-2005-seeing-stars-exploiting-class-relationships/)
23. Perronnin, Florent and Dance, Christopher. Fisher kernels on visual vocabularies for image categorization. In IEEE Conference on Computer Vision and Pattern Recognition, 2007. [find](https://scholar.google.com/scholar?q=Perronnin%2C+Florent+and+Dance%2C+Christopher%2E+Fisher+kernels+on+visual+vocabularies+for+image+categorization%2E+In+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition%2C+2007%2E)
24. Perronnin, Florent, Liu, Yan, Sanchez, Jorge, and Poirier, Herve. Large-scale image retrieval with compressed fisher vectors. In IEEE Conference on Computer Vision and Pattern Recognition, 2010. [find](https://scholar.google.com/scholar?q=Perronnin%2C+Florent%2C+Liu%2C+Yan%2C+Sanchez%2C+Jorge%2C+and+Poirier%2C+Herve%2E+Large%2Dscale+image+retrieval+with+compressed+fisher+vectors%2E+In+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition%2C+2010%2E)
25. Rumelhart, David E, Hinton, Geoffrey E, and Williams, Ronald J. Learning representations by back-propagating errors. Nature, 323(6088):533–536, 1986. [find](https://scholar.google.com/scholar?q=Rumelhart%2C+David+E%2C+Hinton%2C+Geoffrey+E%2C+and+Williams%2C+Ronald+J%2E+Learning+representations+by+back%2Dpropagating+errors%2E+Nature%2C+323%286088%29%3A533%E2%80%93536%2C+1986%2E)
26. Socher, Richard, Huang, Eric H., Pennington, Jeffrey, Manning, Chris D., and Ng, Andrew Y. Dynamic pooling and unfolding recursive autoencoders for paraphrase detection. In Advances in Neural Information Processing Systems, 2011a. [find](https://scholar.google.com/scholar?q=Socher%2C+Richard%2C+Huang%2C+Eric+H%2E%2C+Pennington%2C+Jeffrey%2C+Manning%2C+Chris+D%2E%2C+and+Ng%2C+Andrew+Y%2E+Dynamic+pooling+and+unfolding+recursive+autoencoders+for+paraphrase+detection%2E+In+Advances+in+Neural+Informat)
27. Socher, Richard, Lin, Cliff C, Ng, Andrew, and Manning, Chris. Parsing natural scenes and natural language with recursive neural networks. In Proceedings of the 28th International Conference on Machine Learning (ICML-11), pp. 129–136, 2011b. [find](https://scholar.google.com/scholar?q=Socher%2C+Richard%2C+Lin%2C+Cliff+C%2C+Ng%2C+Andrew%2C+and+Manning%2C+Chris%2E+Parsing+natural+scenes+and+natural+language+with+recursive+neural+networks%2E+In+Proceedings+of+the+28th+International+Conference+on+Machin)
28. Socher, Richard, Pennington, Jeffrey, Huang, Eric H, Ng, Andrew Y, and Manning, Christopher D. Semi-supervised recursive autoencoders for predicting sentiment distributions. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, 2011c. [find](https://scholar.google.com/scholar?q=Socher%2C+Richard%2C+Pennington%2C+Jeffrey%2C+Huang%2C+Eric+H%2C+Ng%2C+Andrew+Y%2C+and+Manning%2C+Christopher+D%2E+Semi%2Dsupervised+recursive+autoencoders+for+predicting+sentiment+distributions%2E+In+Proceedings+of+the+Conf) [our card](https://md.research.elamur.ai/free/en/papers/socher-2011-semi-supervised-recursive-autoencoders-predicting/)
29. Socher, Richard, Chen, Danqi, Manning, Christopher D., and Ng, Andrew Y. Reasoning with neural tensor networks for knowledge base completion. In Advances in Neural Information Processing Systems, 2013a. [find](https://scholar.google.com/scholar?q=Socher%2C+Richard%2C+Chen%2C+Danqi%2C+Manning%2C+Christopher+D%2E%2C+and+Ng%2C+Andrew+Y%2E+Reasoning+with+neural+tensor+networks+for+knowledge+base+completion%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+2013)
30. Socher, Richard, Perelygin, Alex, Wu, Jean Y., Chuang, Jason, Manning, Christopher D., Ng, Andrew Y., and Potts, Christopher. Recursive deep models for semantic compositionality over a sentiment treebank. In Conference on Empirical Methods in Natural Language Processing, 2013b. [find](https://scholar.google.com/scholar?q=Socher%2C+Richard%2C+Perelygin%2C+Alex%2C+Wu%2C+Jean+Y%2E%2C+Chuang%2C+Jason%2C+Manning%2C+Christopher+D%2E%2C+Ng%2C+Andrew+Y%2E%2C+and+Potts%2C+Christopher%2E+Recursive+deep+models+for+semantic+compositionality+over+a+sentiment+treeb)
31. Srivastava, Nitish, Salakhutdinov, Ruslan, and Hinton, Geoffrey. Modeling documents with deep boltzmann machines. In Uncertainty in Artificial Intelligence, 2013. [find](https://scholar.google.com/scholar?q=Srivastava%2C+Nitish%2C+Salakhutdinov%2C+Ruslan%2C+and+Hinton%2C+Geoffrey%2E+Modeling+documents+with+deep+boltzmann+machines%2E+In+Uncertainty+in+Artificial+Intelligence%2C+2013%2E)
32. Turian, Joseph, Ratinov, Lev, and Bengio, Yoshua. Word representations: a simple and general method for semi-supervised learning. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pp. 384–394. Association for Computational Linguistics, 2010. [find](https://scholar.google.com/scholar?q=Turian%2C+Joseph%2C+Ratinov%2C+Lev%2C+and+Bengio%2C+Yoshua%2E+Word+representations%3A+a+simple+and+general+method+for+semi%2Dsupervised+learning%2E+In+Proceedings+of+the+48th+Annual+Meeting+of+the+Association+for+Compu) [our card](https://md.research.elamur.ai/free/en/papers/turian-2010-word-representations-simple-general-method/)
33. Turney, Peter D. and Pantel, Patrick. From frequency to meaning: Vector space models of semantics. Journal of Artificial Intelligence Research, 2010. [find](https://scholar.google.com/scholar?q=Turney%2C+Peter+D%2E+and+Pantel%2C+Patrick%2E+From+frequency+to+meaning%3A+Vector+space+models+of+semantics%2E+Journal+of+Artificial+Intelligence+Research%2C+2010%2E)
34. Wang, Sida and Manning, Chris D. Baselines and bigrams: Simple, good sentiment and text classification. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, 2012. [find](https://scholar.google.com/scholar?q=Wang%2C+Sida+and+Manning%2C+Chris+D%2E+Baselines+and+bigrams%3A+Simple%2C+good+sentiment+and+text+classification%2E+In+Proceedings+of+the+50th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+2012)
35. Yessenalina, Ainur and Cardie, Claire. Compositional matrix-space models for sentiment analysis. In Conference on Empirical Methods in Natural Language Processing, 2011. [find](https://scholar.google.com/scholar?q=Yessenalina%2C+Ainur+and+Cardie%2C+Claire%2E+Compositional+matrix%2Dspace+models+for+sentiment+analysis%2E+In+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+2011%2E)
36. Zanzotto, Fabio, Korkontzelos, Ioannis, Fallucchi, Francesca, and Manandhar, Suresh. Estimating linear models for compositional distributional semantics. In COLING, 2010. [find](https://scholar.google.com/scholar?q=Zanzotto%2C+Fabio%2C+Korkontzelos%2C+Ioannis%2C+Fallucchi%2C+Francesca%2C+and+Manandhar%2C+Suresh%2E+Estimating+linear+models+for+compositional+distributional+semantics%2E+In+COLING%2C+2010%2E)
37. Zhila, A., Yih, W.T., Meek, C., Zweig, G., and Mikolov, T. Combining heterogeneous models for measuring relational similarity. In NAACL HLT, 2013. [find](https://scholar.google.com/scholar?q=Zhila%2C+A%2E%2C+Yih%2C+W%2ET%2E%2C+Meek%2C+C%2E%2C+Zweig%2C+G%2E%2C+and+Mikolov%2C+T%2E+Combining+heterogeneous+models+for+measuring+relational+similarity%2E+In+NAACL+HLT%2C+2013%2E)
38. Zou, Will, Socher, Richard, Cer, Daniel, and Manning, Christopher. Bilingual word embeddings for phrase-based machine translation. In Conference on Empirical Methods in Natural Language Processing, 2013. [find](https://scholar.google.com/scholar?q=Zou%2C+Will%2C+Socher%2C+Richard%2C+Cer%2C+Daniel%2C+and+Manning%2C+Christopher%2E+Bilingual+word+embeddings+for+phrase%2Dbased+machine+translation%2E+In+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+20) [our card](https://md.research.elamur.ai/free/en/papers/zou-2013-bilingual-word-embeddings-phrase-based/)