---
title: "Improving Distributional Similarity with Lessons — Levy 2015"
description: "The authors examine whether neural word embeddings (SGNS from word2vec, GloVe) really outperform classical count-based distributional approaches when…"
authors: ["Omer Levy", "Yoav Goldberg", "Ido Dagan"]
year: 2015
source: "https://aclanthology.org/Q15-1016/"
doi: "10.1162/tacl_a_00134"
licence: "CC BY-NC-SA 4.0 (TACL)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/levy-2015-improving-distributional-similarity-lessons-learned/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/levy-2015-improving-distributional-similarity-lessons-learned/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2015](https://md.research.elamur.ai/free/en/papers/2015/) / Improving Distributional Similarity with Lessons Learned from Word Embeddings

# Improving Distributional Similarity with Lessons Learned from Word Embeddings

**Authors:** [Omer Levy](https://md.research.elamur.ai/free/en/authors/omer-levy/), [Yoav Goldberg](https://md.research.elamur.ai/free/en/authors/yoav-goldberg/), [Ido Dagan](https://md.research.elamur.ai/free/en/authors/ido-dagan/) · **Year:** 2015 · **Licence:** [CC BY-NC-SA 4.0 (TACL)](https://creativecommons.org/licenses/by-nc-sa/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/Q15-1016/) · [PDF](https://aclanthology.org/Q15-1016.pdf) · [DOI 10.1162/tacl\_a\_00134](https://doi.org/10.1162/tacl_a_00134)

**Citations:** [313](https://md.research.elamur.ai/free/en/papers/levy-2015-improving-distributional-similarity-lessons-learned/cited-by/) in our library, of the 140884 papers whose bibliographies we read

## Summary
The authors examine whether neural word embeddings (SGNS from word2vec, GloVe) really outperform classical count-based distributional approaches when evaluated on tasks of similarity and analogy between words. Baroni et al. (2014) had previously claimed such an advantage. The researchers show that much of the gain does not come from the algorithms themselves. It comes from accompanying engineering choices and hyperparameters: a dynamic context window, subsampling of frequent words, the number of negative samples, smoothing of the context distribution, and adding context vectors. These settings are transferred to PPMI and SVD. In particular, the authors propose a smoothed variant of PMI. Four methods (PPMI, SVD, SGNS, GloVe) are compared in controlled experiments on several datasets. After identical tuning, the differences between the methods are mostly local or statistically insignificant, and no approach wins everywhere. Often, changing a single hyperparameter yields more than switching to a different algorithm or a larger corpus. The work is important as a methodological lesson: a fair comparison must account for hyperparameters and preprocessing.

## Key points
- Theoretical basis: SGNS implicitly factorizes the word–context PMI matrix shifted by log k (Levy & Goldberg, 2014c). That is, SGNS relies on the same information as the classical methods.
- Context distribution smoothing (cds = 0.75), borrowed from word2vec, is transferred to PMI. It yields on average more than 3 points per task. For example, on the MSR analogies, PPMI accuracy rose from .443 to .535.
- Evaluation datasets: WordSim (Similarity/Relatedness), MEN (Bruni et al.), M. Turk (Radinsky et al.), Rare Words (Luong et al.), SimLex (Hill et al.), as well as the Google and MSR analogies.
- Practical recommendations:
  - always use cds = 0.75;
  - for SVD, use symmetric variants instead of the classical eig = 1;
  - SGNS is a robust baseline: the fastest to train and the most economical in memory and disk usage;
  - for SGNS, it is better to use more negative samples;
  - for SGNS and GloVe, the w + c variant is worth trying.
- The code is released as hyperwords (bitbucket.org/omerlevy/hyperwords). Publication: TACL, vol. 3, 2015.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{levy2015improving,
  title = {{Improving Distributional Similarity with Lessons Learned from Word Embeddings}},
  author = {Levy, Omer and Goldberg, Yoav and Dagan, Ido},
  year = {2015},
  doi = {10.1162/tacl\_a\_00134},
  url = {https://doi.org/10.1162/tacl_a_00134}
}
```

- [← previous Question-Answer Driven Semantic Role Labeling: Using Natural Language to Annotate Natural Language](https://md.research.elamur.ai/free/en/papers/he-2015-question-answer-driven-semantic-role/)
- [next → Multi-task Sequence to Sequence Learning](https://md.research.elamur.ai/free/en/papers/luong-2015-multi-task-sequence-sequence-learning/)
- [↑ back to papers of 2015](https://md.research.elamur.ai/free/en/papers/2015/)

## References

34 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Eneko Agirre, Enrique Alfonseca, Keith Hall, Jana Kravalova, Marius Pasca, and Aitor Soroa. 2009. A study on similarity and relatedness using distributional and wordnet-based approaches. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the As… [find](https://scholar.google.com/scholar?q=Eneko+Agirre%2C+Enrique+Alfonseca%2C+Keith+Hall%2C+Jana+Kravalova%2C+Marius+Pasca%2C+and+Aitor+Soroa%2E+2009%2E+A+study+on+similarity+and+relatedness+using+distributional+and+wordnet%2Dbased+approaches%2E+In+Proceeding) [our card](https://md.research.elamur.ai/free/en/papers/agirre-2009-study-similarity-relatedness-distributional-wordnet/)
2. Marco Baroni and Alessandro Lenci. 2010. Distributional memory: A general framework for corpus-based semantics. Computational Linguistics, 36(4):673–721. [find](https://scholar.google.com/scholar?q=Marco+Baroni+and+Alessandro+Lenci%2E+2010%2E+Distributional+memory%3A+A+general+framework+for+corpus%2Dbased+semantics%2E+Computational+Linguistics%2C+36%284%29%3A673%E2%80%93721%2E) [our card](https://md.research.elamur.ai/free/en/papers/baroni-2010-distributional-memory-general-framework-corpus/)
3. Marco Baroni, Georgiana Dinu, and Germán Kruszewski. 2014. Dont count, predict! a systematic comparison of context-counting vs. context-predicting semantic vectors. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 238–247, Bal… [find](https://scholar.google.com/scholar?q=Marco+Baroni%2C+Georgiana+Dinu%2C+and+Germa%CC%81n+Kruszewski%2E+2014%2E+Dont+count%2C+predict%21+a+systematic+comparison+of+context%2Dcounting+vs%2E+context%2Dpredicting+semantic+vectors%2E+In+Proceedings+of+the+52nd+Annual+) [our card](https://md.research.elamur.ai/free/en/papers/baroni-2014-don-t-count-predict-systematic/)
4. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. 2003. A neural probabilistic language model. Journal of Machine Learning Research, 3:1137–1155. [find](https://scholar.google.com/scholar?q=Yoshua+Bengio%2C+Re%CC%81jean+Ducharme%2C+Pascal+Vincent%2C+and+Christian+Jauvin%2E+2003%2E+A+neural+probabilistic+language+model%2E+Journal+of+Machine+Learning+Research%2C+3%3A1137%E2%80%931155%2E) [our card](https://md.research.elamur.ai/free/en/papers/bengio-2003-neural-probabilistic-language-model/)
5. Elia Bruni, Gemma Boleda, Marco Baroni, and Nam Khanh Tran. 2012. Distributional semantics in technicolor. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 136–145, Jeju Island, Korea, July. Association for Computational Lingui… [find](https://scholar.google.com/scholar?q=Elia+Bruni%2C+Gemma+Boleda%2C+Marco+Baroni%2C+and+Nam+Khanh+Tran%2E+2012%2E+Distributional+semantics+in+technicolor%2E+In+Proceedings+of+the+50th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28V) [our card](https://md.research.elamur.ai/free/en/papers/bruni-2012-distributional-semantics-technicolor/)
6. John A Bullinaria and Joseph P Levy. 2007. Extracting semantic representations from word co-occurrence statistics: a computational study. Behavior Research Methods, 39(3):510–526. [find](https://scholar.google.com/scholar?q=John+A+Bullinaria+and+Joseph+P+Levy%2E+2007%2E+Extracting+semantic+representations+from+word+co%2Doccurrence+statistics%3A+a+computational+study%2E+Behavior+Research+Methods%2C+39%283%29%3A510%E2%80%93526%2E)
7. John A Bullinaria and Joseph P Levy. 2012. Extracting semantic representations from word co-occurrence statistics: Stop-lists, stemming, and SVD. Behavior Research Methods, 44(3):890–907. [find](https://scholar.google.com/scholar?q=John+A+Bullinaria+and+Joseph+P+Levy%2E+2012%2E+Extracting+semantic+representations+from+word+co%2Doccurrence+statistics%3A+Stop%2Dlists%2C+stemming%2C+and+SVD%2E+Behavior+Research+Methods%2C+44%283%29%3A890%E2%80%93907%2E)
8. John Caron. 2001. Experiments with LSA scoring: optimal rank and basis. In Proceedings of the SIAM Computational Information Retrieval Workshop, pages 157–169. [find](https://scholar.google.com/scholar?q=John+Caron%2E+2001%2E+Experiments+with+LSA+scoring%3A+optimal+rank+and+basis%2E+In+Proceedings+of+the+SIAM+Computational+Information+Retrieval+Workshop%2C+pages+157%E2%80%93169%2E)
9. Kenneth Ward Church and Patrick Hanks. 1990. Word association norms, mutual information, and lexicography. Computational Linguistics, 16(1):22–29. [find](https://scholar.google.com/scholar?q=Kenneth+Ward+Church+and+Patrick+Hanks%2E+1990%2E+Word+association+norms%2C+mutual+information%2C+and+lexicography%2E+Computational+Linguistics%2C+16%281%29%3A22%E2%80%9329%2E) [our card](https://md.research.elamur.ai/free/en/papers/church-1990-word-association-norms-mutual-information/)
10. Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of the 25th International Conference on Machine Learning, pages 160–167. [find](https://scholar.google.com/scholar?q=Ronan+Collobert+and+Jason+Weston%2E+2008%2E+A+unified+architecture+for+natural+language+processing%3A+Deep+neural+networks+with+multitask+learning%2E+In+Proceedings+of+the+25th+International+Conference+on+Mac)
11. Scott C. Deerwester, Susan T. Dumais, Thomas K. Landauer, George W. Furnas, and Richard A. Harshman. 1990. Indexing by latent semantic analysis. JASIS, 41(6):391–407. [find](https://scholar.google.com/scholar?q=Scott+C%2E+Deerwester%2C+Susan+T%2E+Dumais%2C+Thomas+K%2E+Landauer%2C+George+W%2E+Furnas%2C+and+Richard+A%2E+Harshman%2E+1990%2E+Indexing+by+latent+semantic+analysis%2E+JASIS%2C+41%286%29%3A391%E2%80%93407%2E)
12. C Eckart and G Young. 1936. The approximation of one matrix by another of lower rank. Psychometrika, 1:211–218. [find](https://scholar.google.com/scholar?q=C+Eckart+and+G+Young%2E+1936%2E+The+approximation+of+one+matrix+by+another+of+lower+rank%2E+Psychometrika%2C+1%3A211%E2%80%93218%2E)
13. Roi Reichart Felix Hill and Anna Korhonen. 2014. Simlex-999: Evaluating semantic models with (genuine) similarity estimation. arXiv preprint arXiv:1408.3456. [find](https://scholar.google.com/scholar?q=Roi+Reichart+Felix+Hill+and+Anna+Korhonen%2E+2014%2E+Simlex%2D999%3A+Evaluating+semantic+models+with+%28genuine%29+similarity+estimation%2E+arXiv+preprint+arXiv%3A1408%2E3456%2E) [our card](https://md.research.elamur.ai/free/en/papers/hill-2015-simlex-999-evaluating-semantic-models/)
14. Adriano Ferraresi, Eros Zanchetta, Marco Baroni, and Silvia Bernardini. 2008. Introducing and evaluating ukwac, a very large web-derived corpus of English. In Proceedings of the 4th Web as Corpus Workshop (WAC-4), pages 47–54. [find](https://scholar.google.com/scholar?q=Adriano+Ferraresi%2C+Eros+Zanchetta%2C+Marco+Baroni%2C+and+Silvia+Bernardini%2E+2008%2E+Introducing+and+evaluating+ukwac%2C+a+very+large+web%2Dderived+corpus+of+English%2E+In+Proceedings+of+the+4th+Web+as+Corpus+Work)
15. Lev Finkelstein, Evgeniy Gabrilovich, Yossi Matias, Ehud Rivlin, Zach Solan, Gadi Wolfman, and Eytan Ruppin. 2002. Placing search in context: The concept revisited. ACM Transactions on Information Systems, 20(1):116–131. [find](https://scholar.google.com/scholar?q=Lev+Finkelstein%2C+Evgeniy+Gabrilovich%2C+Yossi+Matias%2C+Ehud+Rivlin%2C+Zach+Solan%2C+Gadi+Wolfman%2C+and+Eytan+Ruppin%2E+2002%2E+Placing+search+in+context%3A+The+concept+revisited%2E+ACM+Transactions+on+Information+Sys)
16. Yoav Goldberg and Omer Levy. 2014. word2vec explained: deriving Mikolov et al.’s negative-sampling word-embedding method. arXiv preprint arXiv:1402.3722. — [arXiv:1402.3722](https://arxiv.org/abs/1402.3722) [find](https://scholar.google.com/scholar?q=Yoav+Goldberg+and+Omer+Levy%2E+2014%2E+word2vec+explained%3A+deriving+Mikolov+et+al%2E%E2%80%99s+negative%2Dsampling+word%2Dembedding+method%2E+arXiv+preprint+arXiv%3A1402%2E3722%2E+%E2%80%94+%5BarXiv%3A1402%2E3722%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1402)
17. Zellig Harris. 1954. Distributional structure. Word, 10(23):146–162. [find](https://scholar.google.com/scholar?q=Zellig+Harris%2E+1954%2E+Distributional+structure%2E+Word%2C+10%2823%29%3A146%E2%80%93162%2E)
18. Omer Levy and Yoav Goldberg. 2014a. Dependency-based word embeddings. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 302–308, Baltimore, Maryland. [find](https://scholar.google.com/scholar?q=Omer+Levy+and+Yoav+Goldberg%2E+2014a%2E+Dependency%2Dbased+word+embeddings%2E+In+Proceedings+of+the+52nd+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volume+2%3A+Short+Papers%29%2C+pages+302%E2%80%93308)
19. Omer Levy and Yoav Goldberg. 2014b. Linguistic regularities in sparse and explicit word representations. In Proceedings of the Eighteenth Conference on Computational Natural Language Learning, pages 171–180, Baltimore, Maryland. [find](https://scholar.google.com/scholar?q=Omer+Levy+and+Yoav+Goldberg%2E+2014b%2E+Linguistic+regularities+in+sparse+and+explicit+word+representations%2E+In+Proceedings+of+the+Eighteenth+Conference+on+Computational+Natural+Language+Learning%2C+pages+1) [our card](https://md.research.elamur.ai/free/en/papers/levy-2014-linguistic-regularities-sparse-explicit-word/)
20. Omer Levy and Yoav Goldberg. 2014c. Neural word embeddings as implicit matrix factorization. In Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Canada, pages 2177–2185. [find](https://scholar.google.com/scholar?q=Omer+Levy+and+Yoav+Goldberg%2E+2014c%2E+Neural+word+embeddings+as+implicit+matrix+factorization%2E+In+Advances+in+Neural+Information+Processing+Systems+27%3A+Annual+Conference+on+Neural+Information+Processing)
21. Minh-Thang Luong, Richard Socher, and Christopher D. Manning. 2013. Better word representations with recursive neural networks for morphology. In Proceedings of the Seventeenth Conference on Computational Natural Language Learning, pages 104–113, Sofia, Bulgaria, August. Association for Computationa… [find](https://scholar.google.com/scholar?q=Minh%2DThang+Luong%2C+Richard+Socher%2C+and+Christopher+D%2E+Manning%2E+2013%2E+Better+word+representations+with+recursive+neural+networks+for+morphology%2E+In+Proceedings+of+the+Seventeenth+Conference+on+Computati) [our card](https://md.research.elamur.ai/free/en/papers/luong-2013-better-word-representations-recursive-neural/)
22. Oren Melamud, Ido Dagan, Jacob Goldberger, Idan Szpektor, and Deniz Yuret. 2014. Probabilistic modeling of joint-context in distributional similarity. In Proceedings of the Eighteenth Conference on Computational Natural Language Learning, pages 181–190, Baltimore, Maryland, June. Association for Com… [find](https://scholar.google.com/scholar?q=Oren+Melamud%2C+Ido+Dagan%2C+Jacob+Goldberger%2C+Idan+Szpektor%2C+and+Deniz+Yuret%2E+2014%2E+Probabilistic+modeling+of+joint%2Dcontext+in+distributional+similarity%2E+In+Proceedings+of+the+Eighteenth+Conference+on+Co)
23. Tomas Mikolov, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013a. Efficient estimation of word representations in vector space. In Proceedings of the International Conference on Learning Representations (ICLR). [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Kai+Chen%2C+Gregory+S%2E+Corrado%2C+and+Jeffrey+Dean%2E+2013a%2E+Efficient+estimation+of+word+representations+in+vector+space%2E+In+Proceedings+of+the+International+Conference+on+Learning+Represent) [our card](https://md.research.elamur.ai/free/en/papers/mikolov-2013-efficient-estimation-word-representations-vector/)
24. Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013b. Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems, pages 3111–3119. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Ilya+Sutskever%2C+Kai+Chen%2C+Gregory+S%2E+Corrado%2C+and+Jeffrey+Dean%2E+2013b%2E+Distributed+representations+of+words+and+phrases+and+their+compositionality%2E+In+Advances+in+Neural+Information+Pro) [our card](https://md.research.elamur.ai/free/en/papers/mikolov-2013-distributed-representations-words-phrases-their/)
25. Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. 2013c. Linguistic regularities in continuous space word representations. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 746–751. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Wen%2Dtau+Yih%2C+and+Geoffrey+Zweig%2E+2013c%2E+Linguistic+regularities+in+continuous+space+word+representations%2E+In+Proceedings+of+the+2013+Conference+of+the+North+American+Chapter+of+the+Asso) [our card](https://md.research.elamur.ai/free/en/papers/mikolov-2013-linguistic-regularities-continuous-space-word/)
26. Sebastian Padó and Mirella Lapata. 2007. Dependency-based construction of semantic space models. Computational Linguistics, 33(2):161–199. [find](https://scholar.google.com/scholar?q=Sebastian+Pado%CC%81+and+Mirella+Lapata%2E+2007%2E+Dependency%2Dbased+construction+of+semantic+space+models%2E+Computational+Linguistics%2C+33%282%29%3A161%E2%80%93199%2E) [our card](https://md.research.elamur.ai/free/en/papers/pado-2007-dependency-based-construction-semantic-space/)
27. Patrick Pantel and Dekang Lin. 2002. Discovering word senses from text. In Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 613–619. ACM. [find](https://scholar.google.com/scholar?q=Patrick+Pantel+and+Dekang+Lin%2E+2002%2E+Discovering+word+senses+from+text%2E+In+Proceedings+of+the+eighth+ACM+SIGKDD+international+conference+on+Knowledge+discovery+and+data+mining%2C+pages+613%E2%80%93619%2E+ACM%2E)
28. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Doha, Qatar, October. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jeffrey+Pennington%2C+Richard+Socher%2C+and+Christopher+Manning%2E+2014%2E+Glove%3A+Global+vectors+for+word+representation%2E+In+Proceedings+of+the+2014+Conference+on+Empirical+Methods+in+Natural+Language+Process) [our card](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/)
29. Kira Radinsky, Eugene Agichtein, Evgeniy Gabrilovich, and Shaul Markovitch. 2011. A word at a time: Computing word relatedness using temporal semantic analysis. In Proceedings of the 20th international conference on World wide web, pages 337–346. ACM. [find](https://scholar.google.com/scholar?q=Kira+Radinsky%2C+Eugene+Agichtein%2C+Evgeniy+Gabrilovich%2C+and+Shaul+Markovitch%2E+2011%2E+A+word+at+a+time%3A+Computing+word+relatedness+using+temporal+semantic+analysis%2E+In+Proceedings+of+the+20th+internationa)
30. Magnus Sahlgren. 2006. The Word-Space Model. Ph.D. thesis, Stockholm University. [find](https://scholar.google.com/scholar?q=Magnus+Sahlgren%2E+2006%2E+The+Word%2DSpace+Model%2E+Ph%2ED%2E+thesis%2C+Stockholm+University%2E)
31. Peter D. Turney and Michael L. Littman. 2003. Measuring praise and criticism: Inference of semantic orientation from association. Transactions on Information Systems, 21(4):315–346. [find](https://scholar.google.com/scholar?q=Peter+D%2E+Turney+and+Michael+L%2E+Littman%2E+2003%2E+Measuring+praise+and+criticism%3A+Inference+of+semantic+orientation+from+association%2E+Transactions+on+Information+Systems%2C+21%284%29%3A315%E2%80%93346%2E)
32. Peter D. Turney and Patrick Pantel. 2010. From frequency to meaning: Vector space models of semantics. Journal of Artificial Intelligence Research, 37(1):141–188. [find](https://scholar.google.com/scholar?q=Peter+D%2E+Turney+and+Patrick+Pantel%2E+2010%2E+From+frequency+to+meaning%3A+Vector+space+models+of+semantics%2E+Journal+of+Artificial+Intelligence+Research%2C+37%281%29%3A141%E2%80%93188%2E) [our card](https://md.research.elamur.ai/free/en/papers/turney-2010-frequency-meaning-vector-space-models/)
33. Peter D. Turney. 2012. Domain and function: A dual-space model of semantic relations and compositions. Journal of Artificial Intelligence Research, 44:533– 585. [find](https://scholar.google.com/scholar?q=Peter+D%2E+Turney%2E+2012%2E+Domain+and+function%3A+A+dual%2Dspace+model+of+semantic+relations+and+compositions%2E+Journal+of+Artificial+Intelligence+Research%2C+44%3A533%E2%80%93+585%2E)
34. Torsten Zesch, Christof Müller, and Iryna Gurevych. 2008. Using wiktionary for computing semantic relatedness. In Proceedings of the 23rd National Conference on Artificial Intelligence - Volume 2, AAAI’08, pages 861–866. AAAI Press. [find](https://scholar.google.com/scholar?q=Torsten+Zesch%2C+Christof+Mu%CC%88ller%2C+and+Iryna+Gurevych%2E+2008%2E+Using+wiktionary+for+computing+semantic+relatedness%2E+In+Proceedings+of+the+23rd+National+Conference+on+Artificial+Intelligence+%2D+Volume+2%2C+AA)