---
title: "Problems With Evaluation of Word Embeddings — Faruqui 2016"
description: "Word similarity benchmarks are the most common intrinsic test for word embeddings."
authors: ["Manaal Faruqui", "Yulia Tsvetkov", "Pushpendre Rastogi", "Chris Dyer"]
year: 2016
venue: "Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP"
source: "https://aclanthology.org/W16-2506/"
doi: "10.18653/v1/w16-2506"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/faruqui-2016-problems-evaluation-word-embeddings-word/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/faruqui-2016-problems-evaluation-word-embeddings-word/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2016](https://md.research.elamur.ai/free/en/papers/2016/page/3/) / Problems With Evaluation of Word Embeddings Using Word Similarity Tasks

# Problems With Evaluation of Word Embeddings Using Word Similarity Tasks

**Authors:** [Manaal Faruqui](https://md.research.elamur.ai/free/en/authors/manaal-faruqui/), [Yulia Tsvetkov](https://md.research.elamur.ai/free/en/authors/yulia-tsvetkov/), [Pushpendre Rastogi](https://md.research.elamur.ai/free/en/authors/pushpendre-rastogi/), [Chris Dyer](https://md.research.elamur.ai/free/en/authors/chris-dyer/) · **Year:** 2016 · **Venue:** Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/W16-2506/) · [PDF](https://aclanthology.org/W16-2506.pdf) · [DOI 10.18653/v1/w16-2506](https://doi.org/10.18653/v1/w16-2506) · [OpenAlex](https://openalex.org/W2387546565)

**Citations:** 246 citations (openalex-snapshot · 2026-10-14) · [86](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
Word similarity benchmarks are the most common intrinsic test for word embeddings. Models are scored by the Spearman correlation between cosine similarities of word pairs and human ratings. They are cheap and fast, but the authors argue they are unreliable. This workshop paper is a position and survey piece rather than an experimental study. It lists the main weaknesses of this evaluation style and collects the remedies proposed in earlier work:

- unclear and subjective definitions of "similarity"
- a mismatch with task-trained embeddings
- no fixed train/dev/test splits, which encourages overfitting
- weak correlation with downstream results
- missing significance testing
- frequency and hubness effects in cosine space
- polysemy ignored in the gold ratings

The authors note that word analogy evaluation reduces to combinations of word similarities, so the same criticisms apply to it. They conclude that similarity benchmarks can lead researchers to wrong conclusions. Until better intrinsic methods exist, they recommend comparing embeddings on the downstream task of interest. This matters because much embedding research ranks models mainly by these small benchmarks.

## Key points
- The surveyed benchmarks are RG (65 pairs), MC (30), WS-353, YP-130, MTurk-287, MTurk-771, MEN (3000), RW (2034), Verb (144) and SimLex (999). Their average size is only about 781 pairs, which makes further splitting unstable.
- Similarity is often mixed up with relatedness. In WS-353, cup–coffee gets a higher rating than car–train, and MEN has similar anomalies. SimLex and the similarity/relatedness split of WS-353 by Agirre et al. try to address this.
- On significance testing, Shalaby and Zadrozny applied Steiger's test, but it needs the full model rankings. Rastogi et al. proposed MRDS, the smallest gap in correlation needed for a result to count as significant, and it can be computed from the correlation values alone. They also showed that many reported gains on small datasets were not significant.
- Schnabel et al. found a power law between a word's frequency rank and the average rank of its 1000 nearest neighbours (exponent ≈ 0.17). As a result, words of similar frequency look artificially similar under cosine.
- Most benchmarks assume one sense per word; for example, WS-353 rates bank–money 8.5/10. SCWS addresses this by rating word pairs inside their contexts.

## Interesting ideas
- **Cross-benchmark tuning:** tune hyperparameters on one benchmark and report results on all the others, following Faruqui and Dyer. This gives held-out evaluation even when no official splits exist, and works for any field with many small test sets.
- **Selecting for breadth:** pick the configuration with the best average across many tasks, or the one that beats baselines on most tasks (Lu et al.). This favours generalisation over leaderboard tuning on a single dataset.
- **Significance thresholds per dataset:** publish, for each dataset size, the minimum correlation gap that counts as significant (the MRDS idea). Readers can then judge claimed improvements without access to the raw model outputs.
- **Checking for frequency confounds:** before trusting nearest neighbours or similarity scores, test whether neighbours mostly share frequency rank rather than meaning. This check is useful for any embedding-based retrieval, not only for benchmarks.
- **Grading embeddings by their training objective:** do not judge embeddings learned for one task, such as POS tagging, by a semantic yardstick. Choose evaluation that matches what the vectors were trained to encode.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

86 papers of our library cite it (of the 140884 whose bibliographies we read); 7 of them have a page here, 79 not yet.

- [Morphological Word Embeddings for Arabic Neural Machine Translation in Low-Resource Settings](https://md.research.elamur.ai/free/en/papers/aclbulk-2018-shapiro-morphological-word-embeddings-arabic-neural/)
- [Analysis Methods in Neural Language Processing: A Survey](https://md.research.elamur.ai/free/en/papers/belinkov-2019-analysis-methods-neural-language-processing/)
- [Interpreting Pretrained Contextualized Representations via Reductions to Static Embeddings](https://md.research.elamur.ai/free/en/papers/bommasani-2020-interpreting-pretrained-contextualized-representations-reductions/)
- [SentEval: An Evaluation Toolkit for Universal Sentence Representations](https://md.research.elamur.ai/free/en/papers/conneau-2018-senteval-evaluation-toolkit-universal-sentence/)
- [SimVerb-3500: A Large-Scale Evaluation Set of Verb Similarity](https://md.research.elamur.ai/free/en/papers/gerz-2016-simverb-3500-large-scale-evaluation/)
- [Where New Words Are Born: Distributional Semantic Analysis of Neologisms and Their Semantic Neighborhoods](https://md.research.elamur.ai/free/en/papers/lex-2020-ryskina-neologisms-distributional/)
- [Intrinsic Bias Metrics Do Not Correlate with Application Bias](https://md.research.elamur.ai/free/en/papers/tarrant-2021-intrinsic-bias-metrics-do-not/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{faruqui2016problems,
  title = {{Problems With Evaluation of Word Embeddings Using Word Similarity Tasks}},
  author = {Faruqui, Manaal and Tsvetkov, Yulia and Rastogi, Pushpendre and Dyer, Chris},
  booktitle = {Proceedings of the 1st Workshop on Evaluating Vector-Space Representations for NLP},
  year = {2016},
  doi = {10.18653/v1/w16-2506},
  url = {https://doi.org/10.18653/v1/w16-2506}
}
```

- [← previous Probing for semantic evidence of composition by means of simple classification tasks](https://md.research.elamur.ai/free/en/papers/ettinger-2016-probing-semantic-evidence-composition-means/)
- [next → Putting Sarcasm Detection into Context: The Effects of Class Imbalance and Manual Labelling on Supervised Machine Classification of Twitter Conversations](https://md.research.elamur.ai/free/en/papers/abercrombie-2016-putting-sarcasm-detection-into-context/)
- [↑ back to papers of 2016](https://md.research.elamur.ai/free/en/papers/2016/page/3/)

## References

41 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Eneko Agirre, Enrique Alfonseca, Keith Hall, Jana Kravalova, Marius Paşca, and Aitor Soroa. 2009. A study on similarity and relatedness using distributional and wordnet-based approaches. In Proc. of NAACL. [find](https://scholar.google.com/scholar?q=Eneko+Agirre%2C+Enrique+Alfonseca%2C+Keith+Hall%2C+Jana+Kravalova%2C+Marius+Pas%CC%A7ca%2C+and+Aitor+Soroa%2E+2009%2E+A+study+on+similarity+and+relatedness+using+distributional+and+wordnet%2Dbased+approaches%2E+In+Proc%2E+of+) [our card](https://md.research.elamur.ai/free/en/papers/agirre-2009-study-similarity-relatedness-distributional-wordnet/)
2. Omer Levy and Yoav Goldberg. 2014a. Linguistic regularities in sparse and explicit word representations. In Proc. of CoNLL. [find](https://scholar.google.com/scholar?q=Omer+Levy+and+Yoav+Goldberg%2E+2014a%2E+Linguistic+regularities+in+sparse+and+explicit+word+representations%2E+In+Proc%2E+of+CoNLL%2E) [our card](https://md.research.elamur.ai/free/en/papers/levy-2014-linguistic-regularities-sparse-explicit-word/)
3. Omer Levy and Yoav Goldberg. 2014b. Neural word embedding as implicit matrix factorization. In Proc. of NIPS. [find](https://scholar.google.com/scholar?q=Omer+Levy+and+Yoav+Goldberg%2E+2014b%2E+Neural+word+embedding+as+implicit+matrix+factorization%2E+In+Proc%2E+of+NIPS%2E)
4. Simon Baker, Roi Reichart, and Anna Korhonen. 2014. An unsupervised model for instance level subcategorization acquisition. In Proc. of EMNLP. [find](https://scholar.google.com/scholar?q=Simon+Baker%2C+Roi+Reichart%2C+and+Anna+Korhonen%2E+2014%2E+An+unsupervised+model+for+instance+level+subcategorization+acquisition%2E+In+Proc%2E+of+EMNLP%2E)
5. Omer Levy, Yoav Goldberg, and Ido Dagan. 2015. Improving distributional similarity with lessons learned from word embeddings. TACL, 3:211–225. [find](https://scholar.google.com/scholar?q=Omer+Levy%2C+Yoav+Goldberg%2C+and+Ido+Dagan%2E+2015%2E+Improving+distributional+similarity+with+lessons+learned+from+word+embeddings%2E+TACL%2C+3%3A211%E2%80%93225%2E)
6. Taylor Berg-Kirkpatrick, David Burkett, and Dan Klein. 2012. An empirical investigation of statistical significance in nlp. In Proc. of EMNLP. [find](https://scholar.google.com/scholar?q=Taylor+Berg%2DKirkpatrick%2C+David+Burkett%2C+and+Dan+Klein%2E+2012%2E+An+empirical+investigation+of+statistical+significance+in+nlp%2E+In+Proc%2E+of+EMNLP%2E)
7. Elia Bruni, Gemma Boleda, Marco Baroni, and Nam-Khanh Tran. 2012. Distributional semantics in technicolor. In Proc. of ACL. [find](https://scholar.google.com/scholar?q=Elia+Bruni%2C+Gemma+Boleda%2C+Marco+Baroni%2C+and+Nam%2DKhanh+Tran%2E+2012%2E+Distributional+semantics+in+technicolor%2E+In+Proc%2E+of+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/bruni-2012-distributional-semantics-technicolor/)
8. Ang Lu, Weiran Wang, Mohit Bansal, Kevin Gimpel, and Karen Livescu. 2015. Deep multilingual correlation for improved word embeddings. In Proc. of NAACL. [find](https://scholar.google.com/scholar?q=Ang+Lu%2C+Weiran+Wang%2C+Mohit+Bansal%2C+Kevin+Gimpel%2C+and+Karen+Livescu%2E+2015%2E+Deep+multilingual+correlation+for+improved+word+embeddings%2E+In+Proc%2E+of+NAACL%2E)
9. Jonathan H Clark, Chris Dyer, Alon Lavie, and Noah A Smith. 2011. Better hypothesis testing for statistical machine translation: Controlling for optimizer instability. In Proc. of ACL. [find](https://scholar.google.com/scholar?q=Jonathan+H+Clark%2C+Chris+Dyer%2C+Alon+Lavie%2C+and+Noah+A+Smith%2E+2011%2E+Better+hypothesis+testing+for+statistical+machine+translation%3A+Controlling+for+optimizer+instability%2E+In+Proc%2E+of+ACL%2E)
10. Minh-Thang Luong, Richard Socher, and Christopher D. Manning. 2013. Better word representations with recursive neural networks for morphology. In Proc. of CoNLL. [find](https://scholar.google.com/scholar?q=Minh%2DThang+Luong%2C+Richard+Socher%2C+and+Christopher+D%2E+Manning%2E+2013%2E+Better+word+representations+with+recursive+neural+networks+for+morphology%2E+In+Proc%2E+of+CoNLL%2E) [our card](https://md.research.elamur.ai/free/en/papers/luong-2013-better-word-representations-recursive-neural/)
11. Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: deep neural networks with multitask learning. In Proc. of ICML. [find](https://scholar.google.com/scholar?q=Ronan+Collobert+and+Jason+Weston%2E+2008%2E+A+unified+architecture+for+natural+language+processing%3A+deep+neural+networks+with+multitask+learning%2E+In+Proc%2E+of+ICML%2E)
12. Christopher D. Manning and Hinrich Schütze. 1999. Foundations of statistical natural language processing. MIT Press, Cambridge, MA, USA. [find](https://scholar.google.com/scholar?q=Christopher+D%2E+Manning+and+Hinrich+Sch%C3%BCtze%2E+1999%2E+Foundations+of+statistical+natural+language+processing%2E+MIT+Press%2C+Cambridge%2C+MA%2C+USA%2E)
13. Georgiana Dinu, Angeliki Lazaridou, and Marco Baroni. 2014. Improving zero-shot learning by mitigating the hubness problem. arXiv preprint arXiv:1412.6568. — [arXiv:1412.6568](https://arxiv.org/abs/1412.6568) [find](https://scholar.google.com/scholar?q=Georgiana+Dinu%2C+Angeliki+Lazaridou%2C+and+Marco+Baroni%2E+2014%2E+Improving+zero%2Dshot+learning+by+mitigating+the+hubness+problem%2E+arXiv+preprint+arXiv%3A1412%2E6568%2E+%E2%80%94+%5BarXiv%3A1412%2E6568%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F14)
14. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013a. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Kai+Chen%2C+Greg+Corrado%2C+and+Jeffrey+Dean%2E+2013a%2E+Efficient+estimation+of+word+representations+in+vector+space%2E+arXiv+preprint+arXiv%3A1301%2E3781%2E)
15. Manaal Faruqui and Chris Dyer. 2014a. Community evaluation and exchange of word vectors at word-vectors.org. In Proc. of ACL: System Demo. [find](https://scholar.google.com/scholar?q=Manaal+Faruqui+and+Chris+Dyer%2E+2014a%2E+Community+evaluation+and+exchange+of+word+vectors+at+word%2Dvectors%2Eorg%2E+In+Proc%2E+of+ACL%3A+System+Demo%2E)
16. Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. 2013b. Linguistic regularities in continuous space word representations. In Proc. of NAACL. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Wen%2Dtau+Yih%2C+and+Geoffrey+Zweig%2E+2013b%2E+Linguistic+regularities+in+continuous+space+word+representations%2E+In+Proc%2E+of+NAACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/mikolov-2013-linguistic-regularities-continuous-space-word/)
17. Manaal Faruqui and Chris Dyer. 2014b. Improving vector space word representations using multilingual correlation. In Proc. of EACL. [find](https://scholar.google.com/scholar?q=Manaal+Faruqui+and+Chris+Dyer%2E+2014b%2E+Improving+vector+space+word+representations+using+multilingual+correlation%2E+In+Proc%2E+of+EACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/faruqui-2014-improving-vector-space-word-representations/)
18. George Miller and Walter Charles. 1991. Contextual correlates of semantic similarity. In Language and Cognitive Processes, pages 1–28. [find](https://scholar.google.com/scholar?q=George+Miller+and+Walter+Charles%2E+1991%2E+Contextual+correlates+of+semantic+similarity%2E+In+Language+and+Cognitive+Processes%2C+pages+1%E2%80%9328%2E)
19. Lev Finkelstein, Evgeniy Gabrilovich, Yossi Matias, Ehud Rivlin, Zach Solan, Gadi Wolfman, and Eytan Ruppin. 2002. Placing search in context: The concept revisited. ACM Transactions on Information Systems, 20(1). [find](https://scholar.google.com/scholar?q=Lev+Finkelstein%2C+Evgeniy+Gabrilovich%2C+Yossi+Matias%2C+Ehud+Rivlin%2C+Zach+Solan%2C+Gadi+Wolfman%2C+and+Eytan+Ruppin%2E+2002%2E+Placing+search+in+context%3A+The+concept+revisited%2E+ACM+Transactions+on+Information+Sys)
20. Jerome L. Myers and Arnold D. Well. 1995. Research Design & Statistical Analysis. Routledge. [find](https://scholar.google.com/scholar?q=Jerome+L%2E+Myers+and+Arnold+D%2E+Well%2E+1995%2E+Research+Design+%26+Statistical+Analysis%2E+Routledge%2E)
21. Arvind Neelakantan, Jeevan Shankar, Alexandre Passos, and Andrew McCallum. 2014. Efficient non-parametric estimation of multiple embeddings per word in vector space. In Proc. of EMNLP. [find](https://scholar.google.com/scholar?q=Arvind+Neelakantan%2C+Jeevan+Shankar%2C+Alexandre+Passos%2C+and+Andrew+McCallum%2E+2014%2E+Efficient+non%2Dparametric+estimation+of+multiple+embeddings+per+word+in+vector+space%2E+In+Proc%2E+of+EMNLP%2E) [our card](https://md.research.elamur.ai/free/en/papers/neelakantan-2014-efficient-non-parametric-estimation-multiple/)
22. Guy Halawi, Gideon Dror, Evgeniy Gabrilovich, and Yehuda Koren. 2012. Large-scale learning of word relatedness with constraints. In Proc. of SIGKDD. [find](https://scholar.google.com/scholar?q=Guy+Halawi%2C+Gideon+Dror%2C+Evgeniy+Gabrilovich%2C+and+Yehuda+Koren%2E+2012%2E+Large%2Dscale+learning+of+word+relatedness+with+constraints%2E+In+Proc%2E+of+SIGKDD%2E)
23. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. Glove: Global vectors for word representation. In Proc. of EMNLP. [find](https://scholar.google.com/scholar?q=Jeffrey+Pennington%2C+Richard+Socher%2C+and+Christopher+D%2E+Manning%2E+2014%2E+Glove%3A+Global+vectors+for+word+representation%2E+In+Proc%2E+of+EMNLP%2E)
24. Zellig Harris. 1954. Distributional structure. Word, 10(23):146–162. [find](https://scholar.google.com/scholar?q=Zellig+Harris%2E+1954%2E+Distributional+structure%2E+Word%2C+10%2823%29%3A146%E2%80%93162%2E)
25. Kira Radinsky, Eugene Agichtein, Evgeniy Gabrilovich, and Shaul Markovitch. 2011. A word at a time: computing word relatedness using temporal semantic analysis. In Proc. of WWW. [find](https://scholar.google.com/scholar?q=Kira+Radinsky%2C+Eugene+Agichtein%2C+Evgeniy+Gabrilovich%2C+and+Shaul+Markovitch%2E+2011%2E+A+word+at+a+time%3A+computing+word+relatedness+using+temporal+semantic+analysis%2E+In+Proc%2E+of+WWW%2E)
26. Felix Hill, Roi Reichart, and Anna Korhonen. 2014. Simlex-999: Evaluating semantic models with (genuine) similarity estimation. CoRR, abs/1408.3456. [find](https://scholar.google.com/scholar?q=Felix+Hill%2C+Roi+Reichart%2C+and+Anna+Korhonen%2E+2014%2E+Simlex%2D999%3A+Evaluating+semantic+models+with+%28genuine%29+similarity+estimation%2E+CoRR%2C+abs%2F1408%2E3456%2E) [our card](https://md.research.elamur.ai/free/en/papers/hill-2015-simlex-999-evaluating-semantic-models/)
27. Miloš Radovanović, Alexandros Nanopoulos, and Mirjana Ivanović. 2010. Hubs in space: Popular nearest neighbors in high-dimensional data. The Journal of Machine Learning Research, 11:2487–2531. [find](https://scholar.google.com/scholar?q=Milo%C5%A1+Radovanovic%CC%81%2C+Alexandros+Nanopoulos%2C+and+Mirjana+Ivanovic%CC%81%2E+2010%2E+Hubs+in+space%3A+Popular+nearest+neighbors+in+high%2Ddimensional+data%2E+The+Journal+of+Machine+Learning+Research%2C+11%3A2487%E2%80%932531%2E)
28. Pushpendre Rastogi, Benjamin Van Durme, and Raman Arora. 2015. Multiview lsa: Representation learning via generalized cca. In Proc. of NAACL. [find](https://scholar.google.com/scholar?q=Pushpendre+Rastogi%2C+Benjamin+Van+Durme%2C+and+Raman+Arora%2E+2015%2E+Multiview+lsa%3A+Representation+learning+via+generalized+cca%2E+In+Proc%2E+of+NAACL%2E)
29. Joseph Reisinger and Raymond J. Mooney. 2010. Multi-prototype vector-space models of word meaning. In Proc. of NAACL. [find](https://scholar.google.com/scholar?q=Joseph+Reisinger+and+Raymond+J%2E+Mooney%2E+2010%2E+Multi%2Dprototype+vector%2Dspace+models+of+word+meaning%2E+In+Proc%2E+of+NAACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/reisinger-2010-multi-prototype-vector-space-models/)
30. Sascha Rothe and Hinrich Schütze. 2015. Autoextend: Extending word embeddings to embeddings for synsets and lexemes. In Proc. of ACL. [find](https://scholar.google.com/scholar?q=Sascha+Rothe+and+Hinrich+Sch%C3%BCtze%2E+2015%2E+Autoextend%3A+Extending+word+embeddings+to+embeddings+for+synsets+and+lexemes%2E+In+Proc%2E+of+ACL%2E)
31. Herbert Rubenstein and John B. Goodenough. 1965. Contextual correlates of synonymy. Commun. ACM, 8(10). [find](https://scholar.google.com/scholar?q=Herbert+Rubenstein+and+John+B%2E+Goodenough%2E+1965%2E+Contextual+correlates+of+synonymy%2E+Commun%2E+ACM%2C+8%2810%29%2E)
32. Tobias Schnabel, Igor Labutov, David Mimno, and Thorsten Joachims. 2015. Evaluation methods for unsupervised word embeddings. In Proc. of EMNLP. [find](https://scholar.google.com/scholar?q=Tobias+Schnabel%2C+Igor+Labutov%2C+David+Mimno%2C+and+Thorsten+Joachims%2E+2015%2E+Evaluation+methods+for+unsupervised+word+embeddings%2E+In+Proc%2E+of+EMNLP%2E) [our card](https://md.research.elamur.ai/free/en/papers/schnabel-2015-evaluation-methods-unsupervised-word-embeddings/)
33. Walid Shalaby and Wlodek Zadrozny. 2015. Measuring semantic relatedness using mined semantic analysis. CoRR, abs/1512.03465. — [arXiv:1512.03465](https://arxiv.org/abs/1512.03465) [find](https://scholar.google.com/scholar?q=Walid+Shalaby+and+Wlodek+Zadrozny%2E+2015%2E+Measuring+semantic+relatedness+using+mined+semantic+analysis%2E+CoRR%2C+abs%2F1512%2E03465%2E+%E2%80%94+%5BarXiv%3A1512%2E03465%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1512%2E03465%29)
34. Anders Søgaard, Anders Johannsen, Barbara Plank, Dirk Hovy, and Héctor Martínez Alonso. 2014. What’s in a p-value in nlp? In Proc. of CoNLL. [find](https://scholar.google.com/scholar?q=Anders+S%C3%B8gaard%2C+Anders+Johannsen%2C+Barbara+Plank%2C+Dirk+Hovy%2C+and+H%C3%A9ctor+Mart%C3%ADnez+Alonso%2E+2014%2E+What%E2%80%99s+in+a+p%2Dvalue+in+nlp%3F+In+Proc%2E+of+CoNLL%2E)
35. James H Steiger. 1980. Tests for comparing elements of a correlation matrix. Psychological bulletin, 87(2):245. [find](https://scholar.google.com/scholar?q=James+H+Steiger%2E+1980%2E+Tests+for+comparing+elements+of+a+correlation+matrix%2E+Psychological+bulletin%2C+87%282%29%3A245%2E)
36. Karl Stratos, Michael Collins, and Daniel Hsu. 2015. Model-based word embeddings from decompositions of count matrices. In Proc. of ACL. [find](https://scholar.google.com/scholar?q=Karl+Stratos%2C+Michael+Collins%2C+and+Daniel+Hsu%2E+2015%2E+Model%2Dbased+word+embeddings+from+decompositions+of+count+matrices%2E+In+Proc%2E+of+ACL%2E)
37. Nenad Tomašev, Miloš Radovanovic, Dunja Mladenic, and Mirjana Ivanovic. 2011. A probabilistic approach to nearest-neighbor classification: Naive hubness bayesian knn. In Proc. of CIKM. [find](https://scholar.google.com/scholar?q=Nenad+Toma%C5%A1ev%2C+Milo%C5%A1+Radovanovic%2C+Dunja+Mladenic%2C+and+Mirjana+Ivanovic%2E+2011%2E+A+probabilistic+approach+to+nearest%2Dneighbor+classification%3A+Naive+hubness+bayesian+knn%2E+In+Proc%2E+of+CIKM%2E)
38. Yulia Tsvetkov, Manaal Faruqui, Wang Ling, Guillaume Lample, and Chris Dyer. 2015. Evaluation of word vector representations by subspace alignment. In Proc. of EMNLP. [find](https://scholar.google.com/scholar?q=Yulia+Tsvetkov%2C+Manaal+Faruqui%2C+Wang+Ling%2C+Guillaume+Lample%2C+and+Chris+Dyer%2E+2015%2E+Evaluation+of+word+vector+representations+by+subspace+alignment%2E+In+Proc%2E+of+EMNLP%2E)
39. Joseph Turian, Lev Ratinov, and Yoshua Bengio. 2010. Word representations: a simple and general method for semi-supervised learning. In Proc. of ACL. [find](https://scholar.google.com/scholar?q=Joseph+Turian%2C+Lev+Ratinov%2C+and+Yoshua+Bengio%2E+2010%2E+Word+representations%3A+a+simple+and+general+method+for+semi%2Dsupervised+learning%2E+In+Proc%2E+of+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/turian-2010-word-representations-simple-general-method/)
40. Peter D. Turney and Patrick Pantel. 2010. From frequency to meaning : Vector space models of semantics. JAIR, pages 141–188. [find](https://scholar.google.com/scholar?q=Peter+D%2E+Turney+and+Patrick+Pantel%2E+2010%2E+From+frequency+to+meaning+%3A+Vector+space+models+of+semantics%2E+JAIR%2C+pages+141%E2%80%93188%2E)
41. Dongqiang Yang and David M. W. Powers. 2006. Verb similarity on the taxonomy of wordnet. In 3rd International WordNet Conference. [find](https://scholar.google.com/scholar?q=Dongqiang+Yang+and+David+M%2E+W%2E+Powers%2E+2006%2E+Verb+similarity+on+the+taxonomy+of+wordnet%2E+In+3rd+International+WordNet+Conference%2E)