---
title: "Don't count, predict! A systematic comparison — Baroni 2014"
description: "Distributional semantic models (DSMs) represent the meaning of a word as a vector built from the contexts in which that word occurs."
authors: ["Marco Baroni", "Georgiana Dinu", "Germán Kruszewski"]
year: 2014
venue: "Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/P14-1023/"
doi: "10.3115/v1/p14-1023"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/baroni-2014-don-t-count-predict-systematic/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/baroni-2014-don-t-count-predict-systematic/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2014](https://md.research.elamur.ai/free/en/papers/2014/) / Don't count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors

# Don't count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors

**Authors:** [Marco Baroni](https://md.research.elamur.ai/free/en/authors/marco-baroni/), [Georgiana Dinu](https://md.research.elamur.ai/free/en/authors/georgiana-dinu/), [Germán Kruszewski](https://md.research.elamur.ai/free/en/authors/german-kruszewski/) · **Year:** 2014 · **Venue:** Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/P14-1023/) · [PDF](https://aclanthology.org/P14-1023.pdf) · [DOI 10.3115/v1/p14-1023](https://doi.org/10.3115/v1/p14-1023) · [OpenAlex](https://openalex.org/W2251803266)

**Citations:** 1364 citations (openalex-snapshot · 2026-10-14) · [314](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary

Distributional semantic models (DSMs) represent the meaning of a word as a vector built from the contexts in which that word occurs. The authors compare two approaches to building such vectors. The classical ("count") approach counts word co-occurrences and then reweights and compresses these counts. The newer ("predict", or embeddings) approach trains vectors so that they predict a word's surrounding context as well as possible. Until then, there had been almost no systematic comparison of these approaches. All models are built on the same English corpus. The count vectors were obtained with DISSECT, and the predict vectors were trained with word2vec. Each approach was tested across many parameter settings on standard lexical-semantic benchmarks, including word relatedness rating and concept clustering. Two off-the-shelf models, dm and cw, were also evaluated. The authors expected the count models to win, or at least a mixed picture. Instead, the predict vectors won convincingly almost everywhere, and on some benchmarks they came close to the best known results. As a result, the paper became a strong empirical argument for moving to neural vector representations in semantics.

## Key points

- **Corpus:** about 2.8 billion tokens, combining ukWaC, the English Wikipedia and the British National Corpus. The target and context vocabulary is restricted to the 300K most frequent words.
- **Count models (36 variants, DISSECT):** context windows of 2 and 5 words, PPMI or LMI weighting. The vectors were used either in full or compressed via SVD or NMF to 200–500 dimensions. The best configuration used PMI without any compression, i.e. all 300K dimensions. Among the compression methods, SVD outperformed NMF.
- **Predict models (48 variants, word2vec CBOW):** windows of 2 and 5, 200–500 dimensions. The comparison covered hierarchical softmax versus negative sampling with k = 5 or 10, as well as training without subsampling and with subsampling at t = 1e-5. The best model has only 400 dimensions and uses negative sampling (k = 10) and subsampling of frequent words.
- **Benchmarks:** among them rg (65 noun pairs), WordSim353, and the ap (402 concepts, 21 categories) and esslli (44 concepts, 6 categories) clustering datasets.
- **Off-the-shelf models:** the word2vec vectors outperformed even the syntactically enriched count model dm (Distributional Memory). The off-the-shelf Collobert & Weston vectors (cw, 100 dimensions) performed poorly. In other words, predict models vary considerably in quality, and a comparison against cw alone would have led to the opposite conclusion.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

## Cited by

Cited by

314 papers of our library cite it (of the 140884 whose bibliographies we read); 18 of them have a page here, 296 not yet.

- [Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks](https://md.research.elamur.ai/free/en/papers/adi-2016-fine-grained-analysis-sentence-embeddings/)
- [SemEval-2015 Task 2: Semantic Textual Similarity, English, Spanish and Pilot on Interpretability](https://md.research.elamur.ai/free/en/papers/agirre-2015-semeval-2015-task-2-semantic/)
- [Event Extraction via Dynamic Multi-Pooling Convolutional Neural Networks](https://md.research.elamur.ai/free/en/papers/chen-2015-event-extraction-dynamic-multi-pooling/)
- [SimVerb-3500: A Large-Scale Evaluation Set of Verb Similarity](https://md.research.elamur.ai/free/en/papers/gerz-2016-simverb-3500-large-scale-evaluation/)
- [Analogy-based detection of morphological and semantic relations with word embeddings: what works and what doesn't.](https://md.research.elamur.ai/free/en/papers/gladkova-2016-analogy-based-detection-morphological-semantic/)
- [SimLex-999: Evaluating Semantic Models With (Genuine) Similarity Estimation](https://md.research.elamur.ai/free/en/papers/hill-2015-simlex-999-evaluating-semantic-models/)
- [Learning Distributed Representations of Sentences from Unlabelled Data](https://md.research.elamur.ai/free/en/papers/hill-2016-learning-distributed-representations-sentences-unlabelled/)
- [SensEmbed: Learning Sense Embeddings for Word and Relational Similarity](https://md.research.elamur.ai/free/en/papers/iacobacci-2015-sensembed-learning-sense-embeddings-word/)
- [One Vector is Not Enough: Entity-Augmented Distributed Semantics for Discourse Relations](https://md.research.elamur.ai/free/en/papers/ji-2015-one-vector-is-not-enough/)
- [Diachronic word embeddings and semantic shifts: a survey](https://md.research.elamur.ai/free/en/papers/kutuzov-2018-diachronic-word-embeddings-semantic-shifts/)
- [Combining Language and Vision with a Multimodal Skip-gram Model](https://md.research.elamur.ai/free/en/papers/lazaridou-2015-combining-language-vision-multimodal-skip/)
- [Linguistic Regularities in Sparse and Explicit Word Representations](https://md.research.elamur.ai/free/en/papers/levy-2014-linguistic-regularities-sparse-explicit-word/)
- [Representation Learning Using Multi-Task Deep Neural Networks for Semantic Classification and Information Retrieval](https://md.research.elamur.ai/free/en/papers/liu-2015-representation-learning-multi-task-deep/)
- [Language Models as Knowledge Bases?](https://md.research.elamur.ai/free/en/papers/petroni-2019-language-models-as-knowledge-bases/)
- [Learning How to Ask: Querying LMs with Mixtures of Soft Prompts](https://md.research.elamur.ai/free/en/papers/qin-2021-learning-ask-querying-lms-mixtures/)
- [Evaluation methods for unsupervised word embeddings](https://md.research.elamur.ai/free/en/papers/schnabel-2015-evaluation-methods-unsupervised-word-embeddings/)
- [Document Modeling with Gated Recurrent Neural Network for Sentiment Classification](https://md.research.elamur.ai/free/en/papers/tang-2015-document-modeling-gated-recurrent-neural/)
- [Transparent Semantic Change Detection with Dependency-Based Profiles](https://md.research.elamur.ai/free/en/papers/tat-2026-transparent-semantic-change-detection-dependency/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{baroni2014don,
  title = {{Don't count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors}},
  author = {Baroni, Marco and Dinu, Georgiana and Kruszewski, Germán},
  booktitle = {Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2014},
  doi = {10.3115/v1/p14-1023},
  url = {https://doi.org/10.3115/v1/p14-1023}
}
```

- [← previous Domain Adaptation with Active Learning for Coreference Resolution](https://md.research.elamur.ai/free/en/papers/zhao-2014-domain-adaptation-active-learning-coreference/)
- [next → Efficient Non-parametric Estimation of Multiple Embeddings per Word in Vector Space](https://md.research.elamur.ai/free/en/papers/neelakantan-2014-efficient-non-parametric-estimation-multiple/)
- [↑ back to papers of 2014](https://md.research.elamur.ai/free/en/papers/2014/)

## References

47 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Eneko Agirre, Enrique Alfonseca, Keith Hall, Jana Kravalova, Marius Pasça, and Aitor Soroa. 2009. [find](https://scholar.google.com/scholar?q=Eneko+Agirre%2C+Enrique+Alfonseca%2C+Keith+Hall%2C+Jana+Kravalova%2C+Marius+Pasc%CC%A7a%2C+and+Aitor+Soroa%2E+2009%2E)
2. A study on similarity and relatedness using distributional and WordNet-based approaches. In Proceedings of HLT-NAACL, pages 19–27, Boulder, CO. [find](https://scholar.google.com/scholar?q=A+study+on+similarity+and+relatedness+using+distributional+and+WordNet%2Dbased+approaches%2E+In+Proceedings+of+HLT%2DNAACL%2C+pages+19%E2%80%9327%2C+Boulder%2C+CO%2E) [our card](https://md.research.elamur.ai/free/en/papers/agirre-2009-study-similarity-relatedness-distributional-wordnet/)
3. Abdulrahman Almuhareb. 2006. Attributes in Lexical Acquisition. Phd thesis, University of Essex. [find](https://scholar.google.com/scholar?q=Abdulrahman+Almuhareb%2E+2006%2E+Attributes+in+Lexical+Acquisition%2E+Phd+thesis%2C+University+of+Essex%2E)
4. Marco Baroni and Alessandro Lenci. 2010. Distributional Memory: A general framework for corpus-based semantics. Computational Linguistics, 36(4):673–721. [find](https://scholar.google.com/scholar?q=Marco+Baroni+and+Alessandro+Lenci%2E+2010%2E+Distributional+Memory%3A+A+general+framework+for+corpus%2Dbased+semantics%2E+Computational+Linguistics%2C+36%284%29%3A673%E2%80%93721%2E) [our card](https://md.research.elamur.ai/free/en/papers/baroni-2010-distributional-memory-general-framework-corpus/)
5. Marco Baroni, Stefan Evert, and Alessandro Lenci, editors. 2008. Bridging the Gap between Semantic Theory and Computational Simulations: Proceedings of the ESSLLI Workshop on Distributional Lexical Semantic. FOLLI, Hamburg. [find](https://scholar.google.com/scholar?q=Marco+Baroni%2C+Stefan+Evert%2C+and+Alessandro+Lenci%2C+editors%2E+2008%2E+Bridging+the+Gap+between+Semantic+Theory+and+Computational+Simulations%3A+Proceedings+of+the+ESSLLI+Workshop+on+Distributional+Lexical+Se)
6. Marco Baroni, Eduard Barbu, Brian Murphy, and Massimo Poesio. 2010. Strudel: A distributional semantic model based on properties and types. Cognitive Science, 34(2):222–254. [find](https://scholar.google.com/scholar?q=Marco+Baroni%2C+Eduard+Barbu%2C+Brian+Murphy%2C+and+Massimo+Poesio%2E+2010%2E+Strudel%3A+A+distributional+semantic+model+based+on+properties+and+types%2E+Cognitive+Science%2C+34%282%29%3A222%E2%80%93254%2E)
7. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Janvin. 2003. A neural probabilistic language model. Journal of Machine Learning Research, 3:1137–1155. [find](https://scholar.google.com/scholar?q=Yoshua+Bengio%2C+Re%CC%81jean+Ducharme%2C+Pascal+Vincent%2C+and+Christian+Janvin%2E+2003%2E+A+neural+probabilistic+language+model%2E+Journal+of+Machine+Learning+Research%2C+3%3A1137%E2%80%931155%2E)
8. William Blacoe and Mirella Lapata. 2012. A comparison of vector-based representations for semantic composition. In Proceedings of EMNLP, pages 546–556, Jeju Island, Korea. [find](https://scholar.google.com/scholar?q=William+Blacoe+and+Mirella+Lapata%2E+2012%2E+A+comparison+of+vector%2Dbased+representations+for+semantic+composition%2E+In+Proceedings+of+EMNLP%2C+pages+546%E2%80%93556%2C+Jeju+Island%2C+Korea%2E) [our card](https://md.research.elamur.ai/free/en/papers/blacoe-2012-comparison-vector-based-representations-semantic/)
9. David Blei, Andrew Ng, and Michael Jordan. 2003. Latent Dirichlet allocation. Journal of Machine Learning Research, 3:993–1022. [find](https://scholar.google.com/scholar?q=David+Blei%2C+Andrew+Ng%2C+and+Michael+Jordan%2E+2003%2E+Latent+Dirichlet+allocation%2E+Journal+of+Machine+Learning+Research%2C+3%3A993%E2%80%931022%2E) [our card](https://md.research.elamur.ai/free/en/papers/blei-2003-latent-dirichlet-allocation/)
10. Elia Bruni, Nam Khanh Tran, and Marco Baroni. 2013. Multimodal distributional semantics. Journal of Artificial Intelligence Research. In press; [http://clic.cimec.unitn.it/marco/publications/mmds-jair.pdf](http://clic.cimec.unitn.it/marco/publications/mmds-jair.pdf). [↗](http://clic.cimec.unitn.it/marco/publications/mmds-jair.pdf)
11. John Bullinaria and Joseph Levy. 2007. Extracting semantic representations from word co-occurrence statistics: A computational study. Behavior Research Methods, 39:510–526. [find](https://scholar.google.com/scholar?q=John+Bullinaria+and+Joseph+Levy%2E+2007%2E+Extracting+semantic+representations+from+word+co%2Doccurrence+statistics%3A+A+computational+study%2E+Behavior+Research+Methods%2C+39%3A510%E2%80%93526%2E)
12. John Bullinaria and Joseph Levy. 2012. Extracting semantic representations from word co-occurrence statistics: Stop-lists, stemming and SVD. Behavior Research Methods, 44:890–907. [find](https://scholar.google.com/scholar?q=John+Bullinaria+and+Joseph+Levy%2E+2012%2E+Extracting+semantic+representations+from+word+co%2Doccurrence+statistics%3A+Stop%2Dlists%2C+stemming+and+SVD%2E+Behavior+Research+Methods%2C+44%3A890%E2%80%93907%2E)
13. Yanqing Chen, Bryan Perozzi, Rami Al-Rfou’, and Steven Skiena. 2013. The expressive power of word embeddings. In Proceedings of the ICML Workshop on Deep Learning for Audio, Speech and Language Processing, Atlanta, GA. Published online: [https://sites.google.com/site/deeplearningicml2013/accepted_pap…](https://sites.google.com/site/deeplearningicml2013/accepted_pap…) [↗](https://sites.google.com/site/deeplearningicml2013/accepted_pap…)
14. Stephen Clark. 2013. Vector space models of lexical meaning. In Shalom Lappin and Chris Fox, editors, Handbook of Contemporary Semantics, 2nd ed. Blackwell, Malden, MA. In press; [http://www.cl.cam.ac.uk/˜sc609/pubs/sem_handbook.pdf](http://www.cl.cam.ac.uk/˜sc609/pubs/sem_handbook.pdf). [↗](http://www.cl.cam.ac.uk/˜sc609/pubs/sem_handbook.pdf)
15. Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of ICML, pages 160–167, Helsinki, Finland. [find](https://scholar.google.com/scholar?q=Ronan+Collobert+and+Jason+Weston%2E+2008%2E+A+unified+architecture+for+natural+language+processing%3A+Deep+neural+networks+with+multitask+learning%2E+In+Proceedings+of+ICML%2C+pages+160%E2%80%93167%2C+Helsinki%2C+Finland%2E)
16. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research, 12:2493–2537. [find](https://scholar.google.com/scholar?q=Ronan+Collobert%2C+Jason+Weston%2C+Le%CC%81on+Bottou%2C+Michael+Karlen%2C+Koray+Kavukcuoglu%2C+and+Pavel+Kuksa%2E+2011%2E+Natural+language+processing+%28almost%29+from+scratch%2E+Journal+of+Machine+Learning+Research%2C+12%3A2493%E2%80%93) [our card](https://md.research.elamur.ai/free/en/papers/collobert-2011-natural-language-processing-almost-scratch/)
17. Katrin Erk. 2012. Vector space models of word meaning and phrase meaning: A survey. Language and Linguistics Compass, 6(10):635–653. [find](https://scholar.google.com/scholar?q=Katrin+Erk%2E+2012%2E+Vector+space+models+of+word+meaning+and+phrase+meaning%3A+A+survey%2E+Language+and+Linguistics+Compass%2C+6%2810%29%3A635%E2%80%93653%2E)
18. Stefan Evert. 2005. The Statistics of Word Cooccurrences. Ph.D dissertation, Stuttgart University. [find](https://scholar.google.com/scholar?q=Stefan+Evert%2E+2005%2E+The+Statistics+of+Word+Cooccurrences%2E+Ph%2ED+dissertation%2C+Stuttgart+University%2E)
19. Lev Finkelstein, Evgeniy Gabrilovich, Yossi Matias, Ehud Rivlin, Zach Solan, Gadi Wolfman, and Eytan Ruppin. 2002. Placing search in context: The concept revisited. ACM Transactions on Information Systems, 20(1):116–131. [find](https://scholar.google.com/scholar?q=Lev+Finkelstein%2C+Evgeniy+Gabrilovich%2C+Yossi+Matias%2C+Ehud+Rivlin%2C+Zach+Solan%2C+Gadi+Wolfman%2C+and+Eytan+Ruppin%2E+2002%2E+Placing+search+in+context%3A+The+concept+revisited%2E+ACM+Transactions+on+Information+Sys)
20. Andrea Frome, Greg Corrado, Jon Shlens, Samy Bengio, Jeff Dean, Marc’Aurelio Ranzato, and Tomas Mikolov. 2013. DeViSE: A deep visual-semantic embedding model. In Proceedings of NIPS, pages 2121–2129, Lake Tahoe, Nevada. [find](https://scholar.google.com/scholar?q=Andrea+Frome%2C+Greg+Corrado%2C+Jon+Shlens%2C+Samy+Bengio%2C+Jeff+Dean%2C+Marc%E2%80%99Aurelio+Ranzato%2C+and+Tomas+Mikolov%2E+2013%2E+DeViSE%3A+A+deep+visual%2Dsemantic+embedding+model%2E+In+Proceedings+of+NIPS%2C+pages+2121%E2%80%932129%2C+)
21. Gene Golub and Charles Van Loan. 1996. Matrix Computations (3rd ed.). JHU Press, Baltimore, MD. [find](https://scholar.google.com/scholar?q=Gene+Golub+and+Charles+Van+Loan%2E+1996%2E+Matrix+Computations+%283rd+ed%2E%29%2E+JHU+Press%2C+Baltimore%2C+MD%2E)
22. Tom Griffiths, Mark Steyvers, and Josh Tenenbaum. 2007. Topics in semantic representation. Psychological Review, 114:211–244. [find](https://scholar.google.com/scholar?q=Tom+Griffiths%2C+Mark+Steyvers%2C+and+Josh+Tenenbaum%2E+2007%2E+Topics+in+semantic+representation%2E+Psychological+Review%2C+114%3A211%E2%80%93244%2E)
23. Guy Halawi, Gideon Dror, Evgeniy Gabrilovich, and Yehuda Koren. 2012. Large-scale learning of word relatedness with constraints. In Proceedings of KDD, pages 1406–1414. [find](https://scholar.google.com/scholar?q=Guy+Halawi%2C+Gideon+Dror%2C+Evgeniy+Gabrilovich%2C+and+Yehuda+Koren%2E+2012%2E+Large%2Dscale+learning+of+word+relatedness+with+constraints%2E+In+Proceedings+of+KDD%2C+pages+1406%E2%80%931414%2E)
24. Samer Hassan and Rada Mihalcea. 2011. Semantic relatedness using salient semantic analysis. In Proceedings of AAAI, pages 884–889, San Francisco, CA. [find](https://scholar.google.com/scholar?q=Samer+Hassan+and+Rada+Mihalcea%2E+2011%2E+Semantic+relatedness+using+salient+semantic+analysis%2E+In+Proceedings+of+AAAI%2C+pages+884%E2%80%93889%2C+San+Francisco%2C+CA%2E)
25. Amaç Herdağdelen and Marco Baroni. 2009. Bag-Pack: A general framework to represent semantic relations. In Proceedings of GEMS, pages 33–40, Athens, Greece. [find](https://scholar.google.com/scholar?q=Amac%CC%A7+Herdag%CC%86delen+and+Marco+Baroni%2E+2009%2E+Bag%2DPack%3A+A+general+framework+to+represent+semantic+relations%2E+In+Proceedings+of+GEMS%2C+pages+33%E2%80%9340%2C+Athens%2C+Greece%2E)
26. Eric Huang, Richard Socher, Christopher Manning, and Andrew Ng. 2012. Improving word representations via global context and multiple word prototypes. In Proceedings of ACL, pages 873–882, Jeju Island, Korea. [find](https://scholar.google.com/scholar?q=Eric+Huang%2C+Richard+Socher%2C+Christopher+Manning%2C+and+Andrew+Ng%2E+2012%2E+Improving+word+representations+via+global+context+and+multiple+word+prototypes%2E+In+Proceedings+of+ACL%2C+pages+873%E2%80%93882%2C+Jeju+Island%2C) [our card](https://md.research.elamur.ai/free/en/papers/huang-2012-improving-word-representations-global-context/)
27. George Karypis. 2003. CLUTO: A clustering toolkit. Technical Report 02-017, University of Minnesota Department of Computer Science. [find](https://scholar.google.com/scholar?q=George+Karypis%2E+2003%2E+CLUTO%3A+A+clustering+toolkit%2E+Technical+Report+02%2D017%2C+University+of+Minnesota+Department+of+Computer+Science%2E)
28. Sophia Katrenko and Pieter Adriaans. 2008. Qualia structures and their impact on the concrete noun categorization task. In Proceedings of the ESS-LLI Workshop on Distributional Lexical Semantics, pages 17–24, Hamburg, Germany. [find](https://scholar.google.com/scholar?q=Sophia+Katrenko+and+Pieter+Adriaans%2E+2008%2E+Qualia+structures+and+their+impact+on+the+concrete+noun+categorization+task%2E+In+Proceedings+of+the+ESS%2DLLI+Workshop+on+Distributional+Lexical+Semantics%2C+page)
29. Thomas Landauer and Susan Dumais. 1997. A solution to Plato’s problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge. Psychological Review, 104(2):211– 240. [find](https://scholar.google.com/scholar?q=Thomas+Landauer+and+Susan+Dumais%2E+1997%2E+A+solution+to+Plato%E2%80%99s+problem%3A+The+latent+semantic+analysis+theory+of+acquisition%2C+induction%2C+and+representation+of+knowledge%2E+Psychological+Review%2C+104%282%29%3A211%E2%80%93)
30. Daniel Lee and Sebastian Seung. 2000. Algorithms for Non-negative Matrix Factorization. In Proceedings of NIPS, pages 556–562. [find](https://scholar.google.com/scholar?q=Daniel+Lee+and+Sebastian+Seung%2E+2000%2E+Algorithms+for+Non%2Dnegative+Matrix+Factorization%2E+In+Proceedings+of+NIPS%2C+pages+556%E2%80%93562%2E)
31. Chih-Jen Lin. 2007. Projected gradient methods for Nonnegative Matrix Factorization. Neural Computation, 19(10):2756–2779. [find](https://scholar.google.com/scholar?q=Chih%2DJen+Lin%2E+2007%2E+Projected+gradient+methods+for+Nonnegative+Matrix+Factorization%2E+Neural+Computation%2C+19%2810%29%3A2756%E2%80%932779%2E)
32. Ken McRae, Michael Spivey-Knowlton, and Michael Tanenhaus. 1998. Modeling the influence of thematic fit (and other constraints) in on-line sentence comprehension. Journal of Memory and Language, 38:283–312. [find](https://scholar.google.com/scholar?q=Ken+McRae%2C+Michael+Spivey%2DKnowlton%2C+and+Michael+Tanenhaus%2E+1998%2E+Modeling+the+influence+of+thematic+fit+%28and+other+constraints%29+in+on%2Dline+sentence+comprehension%2E+Journal+of+Memory+and+Language%2C+38%3A28)
33. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013a. Efficient estimation of word representations in vector space. http://arxiv.org/abs/1301.3781/. [↗](http://arxiv.org/abs/1301.3781/)
34. Tomas Mikolov, Quoc Le, and Ilya Sutskever. 2013b. Exploiting similarities among languages for Machine Translation. http://arxiv.org/abs/1309.4168. [↗](http://arxiv.org/abs/1309.4168)
35. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeff Dean. 2013c. Distributed representations of words and phrases and their compositionality. In Proceedings of NIPS, pages 3111–3119, Lake Tahoe, Nevada. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Ilya+Sutskever%2C+Kai+Chen%2C+Greg+Corrado%2C+and+Jeff+Dean%2E+2013c%2E+Distributed+representations+of+words+and+phrases+and+their+compositionality%2E+In+Proceedings+of+NIPS%2C+pages+3111%E2%80%933119%2C+Lake+)
36. Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. 2013d. Linguistic regularities in continuous space word representations. In Proceedings of NAACL, pages 746–751, Atlanta, Georgia. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Wen%2Dtau+Yih%2C+and+Geoffrey+Zweig%2E+2013d%2E+Linguistic+regularities+in+continuous+space+word+representations%2E+In+Proceedings+of+NAACL%2C+pages+746%E2%80%93751%2C+Atlanta%2C+Georgia%2E) [our card](https://md.research.elamur.ai/free/en/papers/mikolov-2013-linguistic-regularities-continuous-space-word/)
37. George Miller and Walter Charles. 1991. Contextual correlates of semantic similarity. Language and Cognitive Processes, 6(1):1–28. [find](https://scholar.google.com/scholar?q=George+Miller+and+Walter+Charles%2E+1991%2E+Contextual+correlates+of+semantic+similarity%2E+Language+and+Cognitive+Processes%2C+6%281%29%3A1%E2%80%9328%2E)
38. Sebastian Padó and Mirella Lapata. 2007. Dependency-based construction of semantic space models. Computational Linguistics, 33(2):161–199. [find](https://scholar.google.com/scholar?q=Sebastian+Pado%CC%81+and+Mirella+Lapata%2E+2007%2E+Dependency%2Dbased+construction+of+semantic+space+models%2E+Computational+Linguistics%2C+33%282%29%3A161%E2%80%93199%2E) [our card](https://md.research.elamur.ai/free/en/papers/pado-2007-dependency-based-construction-semantic-space/)
39. Ulrike Padó. 2007. The Integration of Syntax and Semantic Plausibility in a Wide-Coverage Model of Sentence Processing. Dissertation, Saarland University, Saarbrücken. [find](https://scholar.google.com/scholar?q=Ulrike+Pado%CC%81%2E+2007%2E+The+Integration+of+Syntax+and+Semantic+Plausibility+in+a+Wide%2DCoverage+Model+of+Sentence+Processing%2E+Dissertation%2C+Saarland+University%2C+Saarbru%CC%88cken%2E)
40. Klaus Rothenhäusler and Hinrich Schütze. 2009. Unsupervised classification with dependency based word spaces. In Proceedings of GEMS, pages 17– 24, Athens, Greece. [find](https://scholar.google.com/scholar?q=Klaus+Rothenha%CC%88usler+and+Hinrich+Schu%CC%88tze%2E+2009%2E+Unsupervised+classification+with+dependency+based+word+spaces%2E+In+Proceedings+of+GEMS%2C+pages+17%E2%80%93+24%2C+Athens%2C+Greece%2E)
41. Herbert Rubenstein and John Goodenough. 1965. Contextual correlates of synonymy. Communications of the ACM, 8(10):627–633. [find](https://scholar.google.com/scholar?q=Herbert+Rubenstein+and+John+Goodenough%2E+1965%2E+Contextual+correlates+of+synonymy%2E+Communications+of+the+ACM%2C+8%2810%29%3A627%E2%80%93633%2E)
42. Magnus Sahlgren. 2006. The Word-Space Model. Ph.D dissertation, Stockholm University. [find](https://scholar.google.com/scholar?q=Magnus+Sahlgren%2E+2006%2E+The+Word%2DSpace+Model%2E+Ph%2ED+dissertation%2C+Stockholm+University%2E)
43. Richard Socher, Brody Huval, Christopher Manning, and Andrew Ng. 2012. Semantic compositionality through recursive matrix-vector spaces. In Proceedings of EMNLP, pages 1201–1211, Jeju Island, Korea. [find](https://scholar.google.com/scholar?q=Richard+Socher%2C+Brody+Huval%2C+Christopher+Manning%2C+and+Andrew+Ng%2E+2012%2E+Semantic+compositionality+through+recursive+matrix%2Dvector+spaces%2E+In+Proceedings+of+EMNLP%2C+pages+1201%E2%80%931211%2C+Jeju+Island%2C+Korea%2E) [our card](https://md.research.elamur.ai/free/en/papers/socher-2012-semantic-compositionality-through-recursive-matrix/)
44. Richard Socher, Milind Ganjoo, Christopher Manning, and Andrew Ng. 2013. Zero-shot learning through cross-modal transfer. In Proceedings of NIPS, pages 935–943, Lake Tahoe, Nevada. [find](https://scholar.google.com/scholar?q=Richard+Socher%2C+Milind+Ganjoo%2C+Christopher+Manning%2C+and+Andrew+Ng%2E+2013%2E+Zero%2Dshot+learning+through+cross%2Dmodal+transfer%2E+In+Proceedings+of+NIPS%2C+pages+935%E2%80%93943%2C+Lake+Tahoe%2C+Nevada%2E)
45. Joseph Turian, Lev-Arie Ratinov, and Yoshua Bengio. 2010. Word representations: A simple and general method for semi-supervised learning. In Proceedings of ACL, pages 384–394, Uppsala, Sweden. [find](https://scholar.google.com/scholar?q=Joseph+Turian%2C+Lev%2DArie+Ratinov%2C+and+Yoshua+Bengio%2E+2010%2E+Word+representations%3A+A+simple+and+general+method+for+semi%2Dsupervised+learning%2E+In+Proceedings+of+ACL%2C+pages+384%E2%80%93394%2C+Uppsala%2C+Sweden%2E) [our card](https://md.research.elamur.ai/free/en/papers/turian-2010-word-representations-simple-general-method/)
46. Peter Turney and Patrick Pantel. 2010. From frequency to meaning: Vector space models of semantics. Journal of Artificial Intelligence Research, 37:141–188. [find](https://scholar.google.com/scholar?q=Peter+Turney+and+Patrick+Pantel%2E+2010%2E+From+frequency+to+meaning%3A+Vector+space+models+of+semantics%2E+Journal+of+Artificial+Intelligence+Research%2C+37%3A141%E2%80%93188%2E)
47. Peter Turney. 2012. Domain and function: A dual-space model of semantic relations and compositions. Journal of Artificial Intelligence Research, 44:533– 585. [find](https://scholar.google.com/scholar?q=Peter+Turney%2E+2012%2E+Domain+and+function%3A+A+dual%2Dspace+model+of+semantic+relations+and+compositions%2E+Journal+of+Artificial+Intelligence+Research%2C+44%3A533%E2%80%93+585%2E)