---
title: "Hubness and Pollution — paper by Lazaridou et al. (2015)"
description: "The authors analyse why a linear cross-space mapping fails and test this on two tasks. 34 references, cited by 61 papers here."
authors: ["Angeliki Lazaridou", "Georgiana Dinu", "Marco Baroni"]
year: 2015
venue: "Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)"
source: "https://aclanthology.org/P15-1027/"
doi: "10.3115/v1/p15-1027"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/lazaridou-2015-hubness-pollution-delving-into-cross/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/lazaridou-2015-hubness-pollution-delving-into-cross/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2015](https://md.research.elamur.ai/free/en/papers/2015/page/2/) / Hubness and Pollution: Delving into Cross-Space Mapping for Zero-Shot Learning

# Hubness and Pollution: Delving into Cross-Space Mapping for Zero-Shot Learning

**Authors:** [Angeliki Lazaridou](https://md.research.elamur.ai/free/en/authors/angeliki-lazaridou/), [Georgiana Dinu](https://md.research.elamur.ai/free/en/authors/georgiana-dinu/) and [Marco Baroni](https://md.research.elamur.ai/free/en/authors/marco-baroni/) · **Year:** 2015 · **Venue:** [Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2015](https://md.research.elamur.ai/free/en/venues/acl/2015/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/P15-1027/) · [PDF](https://aclanthology.org/P15-1027.pdf) · [DOI 10.3115/v1/p15-1027](https://doi.org/10.3115/v1/p15-1027) · [OpenAlex](https://openalex.org/W2250646737)

**Citations:** 210 citations (OpenAlex, 2026-10-14) · [61](https://md.research.elamur.ai/free/en/papers/lazaridou-2015-hubness-pollution-delving-into-cross/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Zero-shot labeling learns a function that projects items such as images or source-language words into a corpus-induced word space, where the nearest word becomes the label. Its accuracy has been too low for practical use. The authors analyse why a linear cross-space mapping fails and test this on two tasks. The first is English-to-Italian word translation, using word2vec CBOW vectors over a 200K-word search space. The second is image labeling, mapping CNN features of ImageNet pictures to English word vectors.

They find two problems:
- **Hubness:** least-squares (ridge) training makes some vectors appear as neighbours of many test items. Switching to a max-margin ranking loss largely fixes this.
- **Pollution:** in the visual setting, the neighbourhoods of mapped test items are dominated by target vectors seen in training, which indicates overfitting.

To fix pollution, they add synthetic "chimera" training examples. To improve learning further, they pick "intruder" negatives, which lie close to the mapped vector but far from the gold target. Together these changes clearly beat earlier results in both tasks. This shows that understanding how the mapping goes wrong can make zero-shot transfer considerably more usable.

## Key points
- **Cross-linguistic setup:** 300-dimensional CBOW vectors were trained on 2.8B English and 1.6B Italian tokens. Translation pairs come from a Europarl-derived dictionary, with 5K pairs for training and 1.5K test words spread over 5 frequency bins.
- **Cross-modal setup:** 5.1K concrete ImageNet labels with 100 images each, represented by fc7 features from the Krizhevsky et al. CNN (extracted with Caffe). Labels are split 75%/25% into train and test, and 1K test images are evaluated against all 5.1K labels.
- **Precision@1 for word translation:** 33.0 for the former state of the art, 29.7 for ridge, 39.4 for max-margin and 40.2 with intruder negatives.
- **Precision@1 for image labeling:** 0.5 for the former state of the art, 1.1 for ridge, 1.9 for max-margin, 3.7 with chimera augmentation and 5.6 with intruders.
- **Side effects of the new methods:** chimera augmentation lowered performance on the translation task, where pollution is weak. Intruder negatives cut cross-modal pollution from 71% to 63% and reached good accuracy within fewer training epochs.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Trento](https://md.research.elamur.ai/free/en/universities/university-of-trento/) — University, Trento, Italy · 62 works · 14,868 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

## Cite

Cite as: Angeliki Lazaridou, Georgiana Dinu and Marco Baroni. 2015. Hubness and Pollution: Delving into Cross-Space Mapping for Zero-Shot Learning. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{lazaridou2015hubness,
  title = {{Hubness and Pollution: Delving into Cross-Space Mapping for Zero-Shot Learning}},
  author = {Angeliki Lazaridou, Georgiana Dinu and Marco Baroni},
  booktitle = {Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)},
  year = {2015},
  doi = {10.3115/v1/p15-1027},
  url = {https://doi.org/10.3115/v1/p15-1027}
}
```

- [← previous Towards Universal Paraphrastic Sentence Embeddings](https://md.research.elamur.ai/free/en/papers/wieting-2015-towards-universal-paraphrastic-sentence-embeddings/)
- [next → Cross-lingual Dependency Parsing Based on Distributed Representations](https://md.research.elamur.ai/free/en/papers/guo-2015-cross-lingual-dependency-parsing-based/)
- [↑ back to papers of 2015](https://md.research.elamur.ai/free/en/papers/2015/page/2/)

## References

34 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Marco Baroni, Georgiana Dinu, and Germán Kruszewski. 2014. [Don’t count, predict!](https://md.research.elamur.ai/free/en/papers/baroni-2014-don-t-count-predict-systematic/) a systematic comparison of context-counting vs. context-predicting semantic vectors. In Proceedings of ACL, pages 238–247, Baltimore, MD.
2. Ken Chatfield, Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2014. Return of the devil in the details: Delving deep into convolutional nets. arXiv preprint arXiv:1405.3531. — [arXiv:1405.3531](https://arxiv.org/abs/1405.3531) [find](https://scholar.google.com/scholar?q=Ken+Chatfield%2C+Karen+Simonyan%2C+Andrea+Vedaldi%2C+and+Andrew+Zisserman%2E+2014%2E+Return+of+the+devil+in+the+details%3A+Delving+deep+into+convolutional+nets%2E+arXiv+preprint+arXiv%3A1405%2E3531%2E+%E2%80%94+%5BarXiv%3A1405%2E3531%5D)
3. Koby Crammer, Ofer Dekel, Joseph Keshet, Shai Shalev-Shwartz, and Yoram Singer. 2006. Online passive-aggressive algorithms. The Journal of Machine Learning Research, 7:551–585. [find](https://scholar.google.com/scholar?q=Koby+Crammer%2C+Ofer+Dekel%2C+Joseph+Keshet%2C+Shai+Shalev%2DShwartz%2C+and+Yoram+Singer%2E+2006%2E+Online+passive%2Daggressive+algorithms%2E+The+Journal+of+Machine+Learning+Research%2C+7%3A551%E2%80%93585%2E)
4. Jia Deng, Wei Dong, Richard Socher, Lia-Ji Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In Proceedings of CVPR, pages 248–255, Miami Beach, FL. [find](https://scholar.google.com/scholar?q=Jia+Deng%2C+Wei+Dong%2C+Richard+Socher%2C+Lia%2DJi+Li%2C+and+Li+Fei%2DFei%2E+2009%2E+Imagenet%3A+A+large%2Dscale+hierarchical+image+database%2E+In+Proceedings+of+CVPR%2C+pages+248%E2%80%93255%2C+Miami+Beach%2C+FL%2E)
5. Georgiana Dinu and Marco Baroni. 2014. How to make words with vectors: Phrase generation in distributional semantics. In Proceedings of ACL, pages 624–633, Baltimore, MD. [find](https://scholar.google.com/scholar?q=Georgiana+Dinu+and+Marco+Baroni%2E+2014%2E+How+to+make+words+with+vectors%3A+Phrase+generation+in+distributional+semantics%2E+In+Proceedings+of+ACL%2C+pages+624%E2%80%93633%2C+Baltimore%2C+MD%2E)
6. Georgiana Dinu, Angeliki Lazaridou, and Marco Baroni. 2015. Improving zero-shot learning by mitigating the hubness problem. In Proceedings of ICLR Workshop Track, San Diego, CA. Published online: [http://www.iclr.cc/doku.php?id=iclr2015:main](http://www.iclr.cc/doku.php?id=iclr2015:main). [↗](http://www.iclr.cc/doku.php?id=iclr2015:main)
7. Jeff Donahue, Judy Hoffman, Erik Rodner, Kate Saenko, and Trevor Darrell. 2013. Semi-supervised domain adaptation with instance constraints. In In Proceedings of CVPR, pages 668–675. [find](https://scholar.google.com/scholar?q=Jeff+Donahue%2C+Judy+Hoffman%2C+Erik+Rodner%2C+Kate+Saenko%2C+and+Trevor+Darrell%2E+2013%2E+Semi%2Dsupervised+domain+adaptation+with+instance+constraints%2E+In+In+Proceedings+of+CVPR%2C+pages+668%E2%80%93675%2E)
8. John Duchi, Elad Hazan, and Yoram Singer. 2011. Adaptive subgradient methods for online learning and stochastic optimization. The Journal of Machine Learning Research, 12:2121–2159. [find](https://scholar.google.com/scholar?q=John+Duchi%2C+Elad+Hazan%2C+and+Yoram+Singer%2E+2011%2E+Adaptive+subgradient+methods+for+online+learning+and+stochastic+optimization%2E+The+Journal+of+Machine+Learning+Research%2C+12%3A2121%E2%80%932159%2E)
9. Andrea Frome, Greg Corrado, Jon Shlens, Samy Bengio, Jeff Dean, Marc’Aurelio Ranzato, and Tomas Mikolov. 2013. DeViSE: A deep visual-semantic embedding model. In Proceedings of NIPS, pages 2121–2129, Lake Tahoe, NV. [find](https://scholar.google.com/scholar?q=Andrea+Frome%2C+Greg+Corrado%2C+Jon+Shlens%2C+Samy+Bengio%2C+Jeff+Dean%2C+Marc%E2%80%99Aurelio+Ranzato%2C+and+Tomas+Mikolov%2E+2013%2E+DeViSE%3A+A+deep+visual%2Dsemantic+embedding+model%2E+In+Proceedings+of+NIPS%2C+pages+2121%E2%80%932129%2C+)
10. Boqing Gong, Yuan Shi, Fei Sha, and Kristen Grauman. 2012. Geodesic flow kernel for unsupervised domain adaptation. In In Proceedings of CVPR, pages 2066–2073. [find](https://scholar.google.com/scholar?q=Boqing+Gong%2C+Yuan+Shi%2C+Fei+Sha%2C+and+Kristen+Grauman%2E+2012%2E+Geodesic+flow+kernel+for+unsupervised+domain+adaptation%2E+In+In+Proceedings+of+CVPR%2C+pages+2066%E2%80%932073%2E)
11. Kristen Grauman and Bastian Leibe. 2011. Visual Object Recognition. Morgan & Claypool, San Francisco. [find](https://scholar.google.com/scholar?q=Kristen+Grauman+and+Bastian+Leibe%2E+2011%2E+Visual+Object+Recognition%2E+Morgan+%26+Claypool%2C+San+Francisco%2E)
12. Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. 2014. [Caffe: Convolutional architecture for fast feature embedding](https://md.research.elamur.ai/free/en/papers/jia-2014-caffe-convolutional-architecture-fast-feature/). arXiv preprint arXiv:1408.5093.
13. Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton. 2012. ImageNet classification with deep convolutional neural networks. In Proceedings of NIPS, pages 1097–1105, Lake Tahoe, Nevada. [find](https://scholar.google.com/scholar?q=Alex+Krizhevsky%2C+Ilya+Sutskever%2C+and+Geoffrey+Hinton%2E+2012%2E+ImageNet+classification+with+deep+convolutional+neural+networks%2E+In+Proceedings+of+NIPS%2C+pages+1097%E2%80%931105%2C+Lake+Tahoe%2C+Nevada%2E)
14. Angeliki Lazaridou, Elia Bruni, and Marco Baroni. 2014. Is this a wampimuk? cross-modal mapping between distributional semantics and the visual world. In Proceedings of ACL, pages 1403–1414, Baltimore, MD. [find](https://scholar.google.com/scholar?q=Angeliki+Lazaridou%2C+Elia+Bruni%2C+and+Marco+Baroni%2E+2014%2E+Is+this+a+wampimuk%3F+cross%2Dmodal+mapping+between+distributional+semantics+and+the+visual+world%2E+In+Proceedings+of+ACL%2C+pages+1403%E2%80%931414%2C+Baltimore)
15. David McClosky, Eugene Charniak, and Mark Johnson. 2006. [Effective self-training for parsing](https://md.research.elamur.ai/free/en/papers/mcclosky-2006-effective-self-training-parsing/). In Proceedings of HLT-NAACL, pages 152–159.
16. Tomas Mikolov, Quoc V Le, and Ilya Sutskever. 2013a. Exploiting similarities among languages for machine translation. arXiv preprint arXiv:1309.4168. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Quoc+V+Le%2C+and+Ilya+Sutskever%2E+2013a%2E+Exploiting+similarities+among+languages+for+machine+translation%2E+arXiv+preprint+arXiv%3A1309%2E4168%2E)
17. Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. 2013b. [Linguistic regularities in continuous space word representations](https://md.research.elamur.ai/free/en/papers/mikolov-2013-linguistic-regularities-continuous-space-word/). In Proceedings of NAACL, pages 746–751, Atlanta, Georgia.
18. Tom Mitchell, Svetlana Shinkareva, Andrew Carlson, Kai-Min Chang, Vincente Malave, Robert Mason, and Marcel Just. 2008. Predicting human brain activity associated with the meanings of nouns. Science, 320:1191–1195. [find](https://scholar.google.com/scholar?q=Tom+Mitchell%2C+Svetlana+Shinkareva%2C+Andrew+Carlson%2C+Kai%2DMin+Chang%2C+Vincente+Malave%2C+Robert+Mason%2C+and+Marcel+Just%2E+2008%2E+Predicting+human+brain+activity+associated+with+the+meanings+of+nouns%2E+Science%2C+)
19. Mohammad Norouzi, Tomas Mikolov, Samy Bengio, Yoram Singer, Jonathon Shlens, Andrea Frome, Greg S Corrado, and Jeffrey Dean. 2014. Zero-shot learning by convex combination of semantic embeddings. In Proceedings of ICLR. [find](https://scholar.google.com/scholar?q=Mohammad+Norouzi%2C+Tomas+Mikolov%2C+Samy+Bengio%2C+Yoram+Singer%2C+Jonathon+Shlens%2C+Andrea+Frome%2C+Greg+S+Corrado%2C+and+Jeffrey+Dean%2E+2014%2E+Zero%2Dshot+learning+by+convex+combination+of+semantic+embeddings%2E+In+P)
20. Mark Palatucci, Dean Pomerleau, Geoffrey Hinton, and Tom Mitchell. 2009. Zero-shot learning with semantic output codes. In Proceedings of NIPS, pages 1410–1418, Vancouver, Canada. [find](https://scholar.google.com/scholar?q=Mark+Palatucci%2C+Dean+Pomerleau%2C+Geoffrey+Hinton%2C+and+Tom+Mitchell%2E+2009%2E+Zero%2Dshot+learning+with+semantic+output+codes%2E+In+Proceedings+of+NIPS%2C+pages+1410%E2%80%931418%2C+Vancouver%2C+Canada%2E)
21. Miloš Radovanović, Alexandros Nanopoulos, and Mirjana Ivanović. 2010a. Hubs in space: Popular nearest neighbors in high-dimensional data. Journal of Machine Learning Research, 11:2487–2531. [find](https://scholar.google.com/scholar?q=Milos%CC%8C+Radovanovic%CC%81%2C+Alexandros+Nanopoulos%2C+and+Mirjana+Ivanovic%CC%81%2E+2010a%2E+Hubs+in+space%3A+Popular+nearest+neighbors+in+high%2Ddimensional+data%2E+Journal+of+Machine+Learning+Research%2C+11%3A2487%E2%80%932531%2E)
22. Miloˇs Radovanović, Alexandros Nanopoulos, and Mirjana Ivanović. 2010b. On the existence of obstinate results in vector space models. In Proceedings of SIGIR, pages 186–193, Geneva, Switzerland. [find](https://scholar.google.com/scholar?q=Milo%CB%87s+Radovanovic%CC%81%2C+Alexandros+Nanopoulos%2C+and+Mirjana+Ivanovic%CC%81%2E+2010b%2E+On+the+existence+of+obstinate+results+in+vector+space+models%2E+In+Proceedings+of+SIGIR%2C+pages+186%E2%80%93193%2C+Geneva%2C+Switzerland%2E)
23. Roi Reichart and Ari Rappoport. 2007. Self-training for enhancement and domain adaptation of statistical parsers trained on small datasets. In In Proceedings of ACL, pages 616–623. [find](https://scholar.google.com/scholar?q=Roi+Reichart+and+Ari+Rappoport%2E+2007%2E+Self%2Dtraining+for+enhancement+and+domain+adaptation+of+statistical+parsers+trained+on+small+datasets%2E+In+In+Proceedings+of+ACL%2C+pages+616%E2%80%93623%2E)
24. Noah A Smith and Jason Eisner. 2005. [Contrastive estimation: Training log-linear models on unlabeled data](https://md.research.elamur.ai/free/en/papers/smith-2005-contrastive-estimation-training-log-linear/). In Proceedings of ACL, pages 354–362.
25. Richard Socher, Milind Ganjoo, Christopher Manning, and Andrew Ng. 2013. Zero-shot learning through cross-modal transfer. In Proceedings of NIPS, pages 935–943, Lake Tahoe, NV. [find](https://scholar.google.com/scholar?q=Richard+Socher%2C+Milind+Ganjoo%2C+Christopher+Manning%2C+and+Andrew+Ng%2E+2013%2E+Zero%2Dshot+learning+through+cross%2Dmodal+transfer%2E+In+Proceedings+of+NIPS%2C+pages+935%E2%80%93943%2C+Lake+Tahoe%2C+NV%2E)
26. Richard Socher, Quoc Le, Christopher Manning, and Andrew Ng. 2014. [Grounded compositional semantics for finding and describing images with sentences](https://md.research.elamur.ai/free/en/papers/socher-2014-grounded-compositional-semantics-finding-describing/). Transactions of the Association for Computational Linguistics, 2:207–218.
27. Gilbert Strang. 2003. Introduction to linear algebra, 3d edition. Wellesley-Cambridge Press, Wellesley, MA. [find](https://scholar.google.com/scholar?q=Gilbert+Strang%2E+2003%2E+Introduction+to+linear+algebra%2C+3d+edition%2E+Wellesley%2DCambridge+Press%2C+Wellesley%2C+MA%2E)
28. Jörg Tiedemann. 2012. Parallel data, tools and interfaces in OPUS. In Proceedings of LREC, pages 2214–2218. [find](https://scholar.google.com/scholar?q=Jo%CC%88rg+Tiedemann%2E+2012%2E+Parallel+data%2C+tools+and+interfaces+in+OPUS%2E+In+Proceedings+of+LREC%2C+pages+2214%E2%80%932218%2E)
29. Antonio Torralba and Alexei A Efros. 2011. Unbiased look at dataset bias. In In Proceedings of CVPR, pages 1521–1528. [find](https://scholar.google.com/scholar?q=Antonio+Torralba+and+Alexei+A+Efros%2E+2011%2E+Unbiased+look+at+dataset+bias%2E+In+In+Proceedings+of+CVPR%2C+pages+1521%E2%80%931528%2E)
30. Peter Turney and Patrick Pantel. 2010. [From frequency to meaning: Vector space models of semantics](https://md.research.elamur.ai/free/en/papers/turney-2010-frequency-meaning-vector-space-models/). Journal of Artificial Intelligence Research, 37:141–188.
31. Peter Turney, Yair Neuman, Dan Assaf, and Yohai Cohen. 2011. [Literal and metaphorical sense identification through concrete and abstract context](https://md.research.elamur.ai/free/en/papers/turney-2011-literal-metaphorical-sense-identification-through/). In Proceedings of EMNLP, pages 680–690, Edinburgh, UK.
32. Jason Weston, Samy Bengio, and Nicolas Usunier. 2011. Wsabie: Scaling up to large vocabulary image annotation. In Proceedings of IJCAI, pages 2764– 2770. [find](https://scholar.google.com/scholar?q=Jason+Weston%2C+Samy+Bengio%2C+and+Nicolas+Usunier%2E+2011%2E+Wsabie%3A+Scaling+up+to+large+vocabulary+image+annotation%2E+In+Proceedings+of+IJCAI%2C+pages+2764%E2%80%93+2770%2E)
33. Matthew Zeiler and Rob Fergus. 2014. [Visualizing and understanding convolutional networks](https://md.research.elamur.ai/free/en/papers/zeiler-2013-visualizing-understanding-convolutional-networks/). In Proceedings of ECCV (Part 1), pages 818–833, Zurich, Switzerland.
34. Xiang Zhang and Yann LeCun. 2015. Text understanding from scrath. arXiv preprint arXiv:1502.01710. [find](https://scholar.google.com/scholar?q=Xiang+Zhang+and+Yann+LeCun%2E+2015%2E+Text+understanding+from+scrath%2E+arXiv+preprint+arXiv%3A1502%2E01710%2E)

*Updated 24 October 2026*