---
title: "Combining Language and Vision with a — Lazaridou 2015"
description: "Purely text-based distributional models lack grounding in perception."
authors: ["Angeliki Lazaridou", "Nghia The Pham", "Marco Baroni"]
year: 2015
venue: "Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies"
source: "https://aclanthology.org/N15-1016/"
doi: "10.3115/v1/n15-1016"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/lazaridou-2015-combining-language-vision-multimodal-skip/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/lazaridou-2015-combining-language-vision-multimodal-skip/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2015](https://md.research.elamur.ai/free/en/papers/2015/) / Combining Language and Vision with a Multimodal Skip-gram Model

# Combining Language and Vision with a Multimodal Skip-gram Model

**Authors:** [Angeliki Lazaridou](https://md.research.elamur.ai/free/en/authors/angeliki-lazaridou/), Nghia The Pham, [Marco Baroni](https://md.research.elamur.ai/free/en/authors/marco-baroni/) · **Year:** 2015 · **Venue:** Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/N15-1016/) · [PDF](https://aclanthology.org/N15-1016.pdf) · [DOI 10.3115/v1/n15-1016](https://doi.org/10.3115/v1/n15-1016) · [OpenAlex](https://openalex.org/W1486723856)

**Citations:** 280 citations (openalex-snapshot · 2026-10-14) · [82](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
Purely text-based distributional models lack grounding in perception. Earlier multimodal models had three problems. They built text and image vectors separately and fused them afterwards. They needed visual data for every word. As a result, they could not reach words or images outside their training set. The authors extend SKIP-GRAM so that, while it predicts corpus contexts, some target words are also paired with fixed visual vectors taken from natural images. The objective adds a visual term, which is zero for words that have no images.

There are two variants:
- **MMSKIP-GRAM-A** pushes a word embedding toward its own image vector using a max-margin cosine loss.
- **MMSKIP-GRAM-B** learns a cross-modal mapping matrix jointly with the embeddings.

Because the joint training spreads visual information to words that never had images, the vectors do well on general semantic benchmarks. They also improve zero-shot image labeling and retrieval compared with text-only vectors. In addition, they give plausible visual associations for abstract words. This matters because a single incremental model handles both semantic and vision tasks.

## Key points
- The visual loss is a max-margin hinge on cosine similarity. Negative image vectors are sampled uniformly from the image dictionary, with their number set by hyperparameter k. Training uses hierarchical softmax.
- In MMSKIP-GRAM-B, an L2-regularized matrix M u→v maps linguistic vectors into visual space. This removes the requirement that both vectors have equal dimensionality.
- In a CrowdFlower study on the Kiela et al. (2014) word set, subjects preferred the model's nearest image over a random one in 48% of cases overall. The rate was 73% for concrete words and 23% for abstract words. Only two abstract words had direct visual input during training.
- On the 200 Kiela et al. (2014) words, the entropy of the vectors correlates with concreteness:

  | Vectors | Spearman ρ |
  |---|---|
  | MMSKIP-GRAM-A | −0.75 |
  | MMSKIP-GRAM-B mapped to visual space (B*) | −0.71 |
  | Kiela et al. image-dispersion measure | −0.65 |
  | SKIP-GRAM | 0.05 |
  | Unmapped MMSKIP-GRAM-B | 0.04 |

## Interesting ideas
- **Optional auxiliary loss.** Add a second-modality loss only for the tokens that have such data, and set it to zero elsewhere. The shared training then carries the signal to uncovered vocabulary. This fits any setting with partial annotations, such as a small lexicon or sparse metadata.
- **Learned mediating projection.** Instead of forcing the embedding to equal the external vector, learn a projection between them. This keeps the word space free in size and lets the same design take in other signals, for example feature norms or brain data.
- **Testing propagation honestly.** Report results separately for words that were never exposed to the extra modality during training. This shows whether knowledge truly propagated rather than being memorized.
- **Entropy as an unsupervised proxy for abstractness.** Shift vector values to be positive, unit-normalize, and compute entropy. Being cheap, this could help rank words by concreteness in languages that lack rating norms.
- **Human evaluation for abstract words.** Show annotators the model's nearest image next to a random foil. This gives a simple way to judge outputs that have no gold labels.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

82 papers of our library cite it (of the 140884 whose bibliographies we read); 5 of them have a page here, 77 not yet.

- [Microsoft COCO Captions: Data Collection and Evaluation Server](https://md.research.elamur.ai/free/en/papers/chen-2015-microsoft-coco-captions-data-collection/)
- [Massive Choice, Ample Tasks (MaChAmp): A Toolkit for Multi-task Learning in NLP](https://md.research.elamur.ai/free/en/papers/goot-2021-massive-choice-ample-tasks-machamp/)
- [The Importance of Modeling Social Factors of Language: Theory and Practice](https://md.research.elamur.ai/free/en/papers/hovy-2021-importance-modeling-social-factors-language/)
- [Probing Contextual Language Models for Common Ground with Visual Representations](https://md.research.elamur.ai/free/en/papers/ilharco-2021-probing-contextual-language-models-common/)
- [Multimodal Transformer for Unaligned Multimodal Language Sequences](https://md.research.elamur.ai/free/en/papers/tsai-2019-multimodal-transformer-unaligned-multimodal-language/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{lazaridou2015combining,
  title = {{Combining Language and Vision with a Multimodal Skip-gram Model}},
  author = {Lazaridou, Angeliki and Pham, Nghia The and Baroni, Marco},
  booktitle = {Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies},
  year = {2015},
  doi = {10.3115/v1/n15-1016},
  url = {https://doi.org/10.3115/v1/n15-1016}
}
```

- [← previous CODRA: A Novel Discriminative Framework for Rhetorical Analysis](https://md.research.elamur.ai/free/en/papers/joty-2015-codra-novel-discriminative-framework-rhetorical/)
- [next → Committed Belief Tagging on the Factbank and LU Corpora: A Comparative Study](https://md.research.elamur.ai/free/en/papers/werner-2015-committed-belief-tagging-factbank-lu/)
- [↑ back to papers of 2015](https://md.research.elamur.ai/free/en/papers/2015/)

## References

40 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Marco Baroni, Eduard Barbu, Brian Murphy, and Massimo Poesio. 2010. Strudel: A distributional semantic model based on properties and types. Cognitive Science, 34(2):222–254. [find](https://scholar.google.com/scholar?q=Marco+Baroni%2C+Eduard+Barbu%2C+Brian+Murphy%2C+and+Massimo+Poesio%2E+2010%2E+Strudel%3A+A+distributional+semantic+model+based+on+properties+and+types%2E+Cognitive+Science%2C+34%282%29%3A222%E2%80%93254%2E)
2. Marco Baroni, Georgiana Dinu, and Germán Kruszewski. 2014. Don’t count, predict! a systematic comparison of context-counting vs. context-predicting semantic vectors. In Proceedings of ACL, pages 238–247, Baltimore, MD. [find](https://scholar.google.com/scholar?q=Marco+Baroni%2C+Georgiana+Dinu%2C+and+Germa%CC%81n+Kruszewski%2E+2014%2E+Don%E2%80%99t+count%2C+predict%21+a+systematic+comparison+of+context%2Dcounting+vs%2E+context%2Dpredicting+semantic+vectors%2E+In+Proceedings+of+ACL%2C+pages+238%E2%80%93) [our card](https://md.research.elamur.ai/free/en/papers/baroni-2014-don-t-count-predict-systematic/)
3. Lawrence Barsalou and Katja Wiemer-Hastings. 2005. Situating abstract concepts. In D. Pecher and R. Zwaan, editors, Grounding Cognition: The Role of Perception and Action in Memory, Language, and Thought, pages 129–163. Cambridge University Press, Cambridge, UK. [find](https://scholar.google.com/scholar?q=Lawrence+Barsalou+and+Katja+Wiemer%2DHastings%2E+2005%2E+Situating+abstract+concepts%2E+In+D%2E+Pecher+and+R%2E+Zwaan%2C+editors%2C+Grounding+Cognition%3A+The+Role+of+Perception+and+Action+in+Memory%2C+Language%2C+and+Thou)
4. Lawrence Barsalou. 2008. Grounded cognition. Annual Review of Psychology, 59:617–645. [find](https://scholar.google.com/scholar?q=Lawrence+Barsalou%2E+2008%2E+Grounded+cognition%2E+Annual+Review+of+Psychology%2C+59%3A617%E2%80%93645%2E)
5. Elia Bruni, Gemma Boleda, Marco Baroni, and Nam Khanh Tran. 2012. Distributional semantics in Technicolor. In Proceedings of ACL, pages 136–145, Jeju Island, Korea. [find](https://scholar.google.com/scholar?q=Elia+Bruni%2C+Gemma+Boleda%2C+Marco+Baroni%2C+and+Nam+Khanh+Tran%2E+2012%2E+Distributional+semantics+in+Technicolor%2E+In+Proceedings+of+ACL%2C+pages+136%E2%80%93145%2C+Jeju+Island%2C+Korea%2E) [our card](https://md.research.elamur.ai/free/en/papers/bruni-2012-distributional-semantics-technicolor/)
6. Elia Bruni, Nam Khanh Tran, and Marco Baroni. 2014. Multimodal distributional semantics. Journal of Artificial Intelligence Research, 49:1–47. [find](https://scholar.google.com/scholar?q=Elia+Bruni%2C+Nam+Khanh+Tran%2C+and+Marco+Baroni%2E+2014%2E+Multimodal+distributional+semantics%2E+Journal+of+Artificial+Intelligence+Research%2C+49%3A1%E2%80%9347%2E)
7. Stephen Clark. 2015. Vector space models of lexical meaning. In Shalom Lappin and Chris Fox, editors, Handbook of Contemporary Semantics, 2nd ed. Blackwell, Malden, MA. In press; [http://www.cl.cam.ac.uk/˜sc609/pubs/sem_handbook.pdf](http://www.cl.cam.ac.uk/˜sc609/pubs/sem_handbook.pdf). [↗](http://www.cl.cam.ac.uk/˜sc609/pubs/sem_handbook.pdf)
8. Jia Deng, Wei Dong, Richard Socher, Lia-Ji Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In Proceedings of CVPR, pages 248–255, Miami Beach, FL. [find](https://scholar.google.com/scholar?q=Jia+Deng%2C+Wei+Dong%2C+Richard+Socher%2C+Lia%2DJi+Li%2C+and+Li+Fei%2DFei%2E+2009%2E+Imagenet%3A+A+large%2Dscale+hierarchical+image+database%2E+In+Proceedings+of+CVPR%2C+pages+248%E2%80%93255%2C+Miami+Beach%2C+FL%2E)
9. Jeffrey L Elman. 1990. Finding structure in time. Cognitive science, 14(2):179–211. [find](https://scholar.google.com/scholar?q=Jeffrey+L+Elman%2E+1990%2E+Finding+structure+in+time%2E+Cognitive+science%2C+14%282%29%3A179%E2%80%93211%2E)
10. Ali Farhadi, Ian Endres, Derek Hoiem, and David Forsyth. 2009. Describing objects by their attributes. In Proceedings of CVPR, pages 1778–1785, Miami Beach, FL. [find](https://scholar.google.com/scholar?q=Ali+Farhadi%2C+Ian+Endres%2C+Derek+Hoiem%2C+and+David+Forsyth%2E+2009%2E+Describing+objects+by+their+attributes%2E+In+Proceedings+of+CVPR%2C+pages+1778%E2%80%931785%2C+Miami+Beach%2C+FL%2E)
11. Yansong Feng and Mirella Lapata. 2010. Visual information in semantic representation. In Proceedings of HLT-NAACL, pages 91–99, Los Angeles, CA. [find](https://scholar.google.com/scholar?q=Yansong+Feng+and+Mirella+Lapata%2E+2010%2E+Visual+information+in+semantic+representation%2E+In+Proceedings+of+HLT%2DNAACL%2C+pages+91%E2%80%9399%2C+Los+Angeles%2C+CA%2E)
12. Andrea Frome, Greg Corrado, Jon Shlens, Samy Bengio, Jeff Dean, Marc’Aurelio Ranzato, and Tomas Mikolov. 2013. DeViSE: A deep visual-semantic embedding model. In Proceedings of NIPS, pages 2121– 2129, Lake Tahoe, NV. [find](https://scholar.google.com/scholar?q=Andrea+Frome%2C+Greg+Corrado%2C+Jon+Shlens%2C+Samy+Bengio%2C+Jeff+Dean%2C+Marc%E2%80%99Aurelio+Ranzato%2C+and+Tomas+Mikolov%2E+2013%2E+DeViSE%3A+A+deep+visual%2Dsemantic+embedding+model%2E+In+Proceedings+of+NIPS%2C+pages+2121%E2%80%93+2129%2C)
13. Alona Fyshe, Partha P Talukdar, Brian Murphy, and Tom M Mitchell. 2014. Interpretable semantic vectors from a joint model of brain-and text-based meaning. In In Proceedings of ACL, pages 489–499. [find](https://scholar.google.com/scholar?q=Alona+Fyshe%2C+Partha+P+Talukdar%2C+Brian+Murphy%2C+and+Tom+M+Mitchell%2E+2014%2E+Interpretable+semantic+vectors+from+a+joint+model+of+brain%2Dand+text%2Dbased+meaning%2E+In+In+Proceedings+of+ACL%2C+pages+489%E2%80%93499%2E)
14. Arthur Glenberg and David Robertson. 2000. Symbol grounding and meaning: A comparison of high-dimensional and embodied theories of meaning. Journal of Memory and Language, 3(43):379–401. [find](https://scholar.google.com/scholar?q=Arthur+Glenberg+and+David+Robertson%2E+2000%2E+Symbol+grounding+and+meaning%3A+A+comparison+of+high%2Ddimensional+and+embodied+theories+of+meaning%2E+Journal+of+Memory+and+Language%2C+3%2843%29%3A379%E2%80%93401%2E)
15. Stevan Harnad. 1990. The symbol grounding problem. Physica D: Nonlinear Phenomena, 42(1-3):335–346. [find](https://scholar.google.com/scholar?q=Stevan+Harnad%2E+1990%2E+The+symbol+grounding+problem%2E+Physica+D%3A+Nonlinear+Phenomena%2C+42%281%2D3%29%3A335%E2%80%93346%2E)
16. Felix Hill and Anna Korhonen. 2014. Learning abstract concept embeddings from multi-modal data: Since you probably can’t see what I mean. In Proceedings of EMNLP, pages 255–265, Doha, Qatar. [find](https://scholar.google.com/scholar?q=Felix+Hill+and+Anna+Korhonen%2E+2014%2E+Learning+abstract+concept+embeddings+from+multi%2Dmodal+data%3A+Since+you+probably+can%E2%80%99t+see+what+I+mean%2E+In+Proceedings+of+EMNLP%2C+pages+255%E2%80%93265%2C+Doha%2C+Qatar%2E)
17. Felix Hill, Roi Reichart, and Anna Korhonen. 2014. SimLex-999: Evaluating semantic models with (genuine) similarity estimation. http://arxiv.org/abs/arXiv:1408.3456. [↗](http://arxiv.org/abs/arXiv:1408.3456) [our card](https://md.research.elamur.ai/free/en/papers/hill-2015-simlex-999-evaluating-semantic-models/)
18. Steve Howell, Damian Jankowicz, and Suzanna Becker. 2005. A model of grounded language acquisition: Sensorimotor features improve lexical and grammatical learning. Journal of Memory and Language, 53:258–276. [find](https://scholar.google.com/scholar?q=Steve+Howell%2C+Damian+Jankowicz%2C+and+Suzanna+Becker%2E+2005%2E+A+model+of+grounded+language+acquisition%3A+Sensorimotor+features+improve+lexical+and+grammatical+learning%2E+Journal+of+Memory+and+Language%2C+53%3A2)
19. Yangqing Jia, Evan Shelhamer, Jeff Donahue, Sergey Karayev, Jonathan Long, Ross Girshick, Sergio Guadarrama, and Trevor Darrell. 2014. Caffe: Convolutional architecture for fast feature embedding. arXiv preprint arXiv:1408.5093. [find](https://scholar.google.com/scholar?q=Yangqing+Jia%2C+Evan+Shelhamer%2C+Jeff+Donahue%2C+Sergey+Karayev%2C+Jonathan+Long%2C+Ross+Girshick%2C+Sergio+Guadarrama%2C+and+Trevor+Darrell%2E+2014%2E+Caffe%3A+Convolutional+architecture+for+fast+feature+embedding%2E+arX) [our card](https://md.research.elamur.ai/free/en/papers/jia-2014-caffe-convolutional-architecture-fast-feature/)
20. Andrej Karpathy, Armand Joulin, and Li Fei-Fei. 2014. Deep fragment embeddings for bidirectional image sentence mapping. In Proceedings of NIPS, pages 1097–1105, Montreal, Canada. [find](https://scholar.google.com/scholar?q=Andrej+Karpathy%2C+Armand+Joulin%2C+and+Li+Fei%2DFei%2E+2014%2E+Deep+fragment+embeddings+for+bidirectional+image+sentence+mapping%2E+In+Proceedings+of+NIPS%2C+pages+1097%E2%80%931105%2C+Montreal%2C+Canada%2E)
21. Douwe Kiela and Léon Bottou. 2014. Learning image embeddings using convolutional neural networks for improved multi-modal semantics. In Proceedings of EMNLP, pages 36–45, Doha, Qatar. [find](https://scholar.google.com/scholar?q=Douwe+Kiela+and+Le%CC%81on+Bottou%2E+2014%2E+Learning+image+embeddings+using+convolutional+neural+networks+for+improved+multi%2Dmodal+semantics%2E+In+Proceedings+of+EMNLP%2C+pages+36%E2%80%9345%2C+Doha%2C+Qatar%2E)
22. Douwe Kiela, Felix Hill, Anna Korhonen, and Stephen Clark. 2014. Improving multi-modal representations using image dispersion: Why less is sometimes more. In Proceedings of ACL, pages 835–841, Baltimore, MD. [find](https://scholar.google.com/scholar?q=Douwe+Kiela%2C+Felix+Hill%2C+Anna+Korhonen%2C+and+Stephen+Clark%2E+2014%2E+Improving+multi%2Dmodal+representations+using+image+dispersion%3A+Why+less+is+sometimes+more%2E+In+Proceedings+of+ACL%2C+pages+835%E2%80%93841%2C+Baltimo)
23. Ryan Kiros, Ruslan Salakhutdinov, and Richard Zemel. 2014. Unifying visual-semantic embeddings with multimodal neural language models. In Proceedings of the NIPS Deep Learning and Representation Learning Workshop, Montreal, Canada. Published online: [http://www.dlworkshop.org/accepted-papers](http://www.dlworkshop.org/accepted-papers). [↗](http://www.dlworkshop.org/accepted-papers) [our card](https://md.research.elamur.ai/free/en/papers/kiros-2014-unifying-visual-semantic-embeddings-multimodal/)
24. Alex Krizhevsky, Ilya Sutskever, and Geoffrey Hinton. 2012. ImageNet classification with deep convolutional neural networks. In Proceedings of NIPS, pages 1097– 1105, Lake Tahoe, Nevada. [find](https://scholar.google.com/scholar?q=Alex+Krizhevsky%2C+Ilya+Sutskever%2C+and+Geoffrey+Hinton%2E+2012%2E+ImageNet+classification+with+deep+convolutional+neural+networks%2E+In+Proceedings+of+NIPS%2C+pages+1097%E2%80%93+1105%2C+Lake+Tahoe%2C+Nevada%2E)
25. George Lakoff and Mark Johnson. 1999. Philosophy in the Flesh: The Embodied Mind and Its Challenge to Western Thought. Basic Books, New York. [find](https://scholar.google.com/scholar?q=George+Lakoff+and+Mark+Johnson%2E+1999%2E+Philosophy+in+the+Flesh%3A+The+Embodied+Mind+and+Its+Challenge+to+Western+Thought%2E+Basic+Books%2C+New+York%2E)
26. Angeliki Lazaridou, Elia Bruni, and Marco Baroni. 2014. Is this a wampimuk? cross-modal mapping between distributional semantics and the visual world. In Proceedings of ACL, pages 1403–1414, Baltimore, MD. [find](https://scholar.google.com/scholar?q=Angeliki+Lazaridou%2C+Elia+Bruni%2C+and+Marco+Baroni%2E+2014%2E+Is+this+a+wampimuk%3F+cross%2Dmodal+mapping+between+distributional+semantics+and+the+visual+world%2E+In+Proceedings+of+ACL%2C+pages+1403%E2%80%931414%2C+Baltimore)
27. Junhua Mao, Wei Xu, Yi Yang, Jiang Wang, and Alan Yuille. 2014. Explain images with multimodal recurrent neural networks. In Proceedings of the NIPS Deep Learning and Representation Learning Workshop, Montreal, Canada. Published online: [http://www.dlworkshop.org/accepted-papers](http://www.dlworkshop.org/accepted-papers). [↗](http://www.dlworkshop.org/accepted-papers)
28. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeff Dean. 2013a. Distributed representations of words and phrases and their compositionality. In Proceedings of NIPS, pages 3111–3119, Lake Tahoe, NV. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Ilya+Sutskever%2C+Kai+Chen%2C+Greg+Corrado%2C+and+Jeff+Dean%2E+2013a%2E+Distributed+representations+of+words+and+phrases+and+their+compositionality%2E+In+Proceedings+of+NIPS%2C+pages+3111%E2%80%933119%2C+Lake+)
29. Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. 2013b. Linguistic regularities in continuous space word representations. In Proceedings of NAACL, pages 746–751, Atlanta, Georgia. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Wen%2Dtau+Yih%2C+and+Geoffrey+Zweig%2E+2013b%2E+Linguistic+regularities+in+continuous+space+word+representations%2E+In+Proceedings+of+NAACL%2C+pages+746%E2%80%93751%2C+Atlanta%2C+Georgia%2E) [our card](https://md.research.elamur.ai/free/en/papers/mikolov-2013-linguistic-regularities-continuous-space-word/)
30. Frederic Morin and Yoshua Bengio. 2005. Hierarchical probabilistic neural network language model. In Proceedings of AISTATS, pages 246–252, Barbados. [find](https://scholar.google.com/scholar?q=Frederic+Morin+and+Yoshua+Bengio%2E+2005%2E+Hierarchical+probabilistic+neural+network+language+model%2E+In+Proceedings+of+AISTATS%2C+pages+246%E2%80%93252%2C+Barbados%2E)
31. Maxime Oquab, Leon Bottou, Ivan Laptev, and Josef Sivic. 2014. Learning and transferring mid-level image representations using convolutional neural networks. In Proceedings of CVPR. [find](https://scholar.google.com/scholar?q=Maxime+Oquab%2C+Leon+Bottou%2C+Ivan+Laptev%2C+and+Josef+Sivic%2E+2014%2E+Learning+and+transferring+mid%2Dlevel+image+representations+using+convolutional+neural+networks%2E+In+Proceedings+of+CVPR%2E)
32. John Searle. 1984. Minds, Brains and Science. Harvard University Press, Cambridge, MA. [find](https://scholar.google.com/scholar?q=John+Searle%2E+1984%2E+Minds%2C+Brains+and+Science%2E+Harvard+University+Press%2C+Cambridge%2C+MA%2E)
33. Carina Silberer and Mirella Lapata. 2014. Learning grounded meaning representations with autoencoders. In Proceedings of ACL, pages 721–732, Baltimore, Maryland. [find](https://scholar.google.com/scholar?q=Carina+Silberer+and+Mirella+Lapata%2E+2014%2E+Learning+grounded+meaning+representations+with+autoencoders%2E+In+Proceedings+of+ACL%2C+pages+721%E2%80%93732%2C+Baltimore%2C+Maryland%2E)
34. Josef Sivic and Andrew Zisserman. 2003. Video Google: A text retrieval approach to object matching in videos. In Proceedings of ICCV, pages 1470–1477, Nice, France. [find](https://scholar.google.com/scholar?q=Josef+Sivic+and+Andrew+Zisserman%2E+2003%2E+Video+Google%3A+A+text+retrieval+approach+to+object+matching+in+videos%2E+In+Proceedings+of+ICCV%2C+pages+1470%E2%80%931477%2C+Nice%2C+France%2E)
35. Richard Socher, Milind Ganjoo, Christopher Manning, and Andrew Ng. 2013. Zero-shot learning through cross-modal transfer. In Proceedings of NIPS, pages 935–943, Lake Tahoe, NV. [find](https://scholar.google.com/scholar?q=Richard+Socher%2C+Milind+Ganjoo%2C+Christopher+Manning%2C+and+Andrew+Ng%2E+2013%2E+Zero%2Dshot+learning+through+cross%2Dmodal+transfer%2E+In+Proceedings+of+NIPS%2C+pages+935%E2%80%93943%2C+Lake+Tahoe%2C+NV%2E)
36. Richard Socher, Quoc Le, Christopher Manning, and Andrew Ng. 2014. Grounded compositional semantics for finding and describing images with sentences. Transactions of the Association for Computational Linguistics, 2:207–218. [find](https://scholar.google.com/scholar?q=Richard+Socher%2C+Quoc+Le%2C+Christopher+Manning%2C+and+Andrew+Ng%2E+2014%2E+Grounded+compositional+semantics+for+finding+and+describing+images+with+sentences%2E+Transactions+of+the+Association+for+Computational+) [our card](https://md.research.elamur.ai/free/en/papers/socher-2014-grounded-compositional-semantics-finding-describing/)
37. Peter Turney and Patrick Pantel. 2010. From frequency to meaning: Vector space models of semantics. Journal of Artificial Intelligence Research, 37:141–188. [find](https://scholar.google.com/scholar?q=Peter+Turney+and+Patrick+Pantel%2E+2010%2E+From+frequency+to+meaning%3A+Vector+space+models+of+semantics%2E+Journal+of+Artificial+Intelligence+Research%2C+37%3A141%E2%80%93188%2E)
38. Peter Turney, Yair Neuman, Dan Assaf, and Yohai Cohen. 2011. Literal and metaphorical sense identification through concrete and abstract context. In Proceedings of EMNLP, pages 680–690, Edinburgh, UK. [find](https://scholar.google.com/scholar?q=Peter+Turney%2C+Yair+Neuman%2C+Dan+Assaf%2C+and+Yohai+Cohen%2E+2011%2E+Literal+and+metaphorical+sense+identification+through+concrete+and+abstract+context%2E+In+Proceedings+of+EMNLP%2C+pages+680%E2%80%93690%2C+Edinburgh%2C+UK%2E) [our card](https://md.research.elamur.ai/free/en/papers/turney-2011-literal-metaphorical-sense-identification-through/)
39. Luis von Ahn and Laura Dabbish. 2004. Labeling images with a computer game. In Proceedings of CHI, pages 319–326, Vienna, Austria. [find](https://scholar.google.com/scholar?q=Luis+von+Ahn+and+Laura+Dabbish%2E+2004%2E+Labeling+images+with+a+computer+game%2E+In+Proceedings+of+CHI%2C+pages+319%E2%80%93326%2C+Vienna%2C+Austria%2E)
40. Jason Weston, Samy Bengio, and Nicolas Usunier. 2010. Large scale image annotation: learning to rank with joint word-image embeddings. Machine learning, 81(1):21–35. [find](https://scholar.google.com/scholar?q=Jason+Weston%2C+Samy+Bengio%2C+and+Nicolas+Usunier%2E+2010%2E+Large+scale+image+annotation%3A+learning+to+rank+with+joint+word%2Dimage+embeddings%2E+Machine+learning%2C+81%281%29%3A21%E2%80%9335%2E)