---
title: "AutoExtend — paper by Rothe and Schütze (2015)"
description: "AutoExtend tackles the lack of vector representations for the non-word units of lexical resources, such… 36 references, cited by 57 papers here."
authors: ["Sascha Rothe", "Hinrich Schütze"]
year: 2015
venue: "Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)"
source: "https://aclanthology.org/P15-1173/"
doi: "10.3115/v1/p15-1173"
arxiv: "1507.01127"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/rothe-2015-autoextend-extending-word-embeddings-embeddings/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/rothe-2015-autoextend-extending-word-embeddings-embeddings/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2015](https://md.research.elamur.ai/free/en/papers/2015/page/2/) / AutoExtend: Extending Word Embeddings to Embeddings for Synsets and Lexemes

# AutoExtend: Extending Word Embeddings to Embeddings for Synsets and Lexemes

**Authors:** [Sascha Rothe](https://md.research.elamur.ai/free/en/authors/sascha-rothe/) and [Hinrich Schütze](https://md.research.elamur.ai/free/en/authors/hinrich-schutze/) · **Year:** 2015 · **Venue:** [Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2015](https://md.research.elamur.ai/free/en/venues/acl/2015/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/P15-1173/) · [PDF](https://aclanthology.org/P15-1173.pdf) · [DOI 10.3115/v1/p15-1173](https://doi.org/10.3115/v1/p15-1173) · [arXiv 1507.01127](https://arxiv.org/abs/1507.01127) · [OpenAlex](https://openalex.org/W2125786288)

**Citations:** 110 citations (OpenAlex, 2026-10-14) · [57](https://md.research.elamur.ai/free/en/papers/rothe-2015-autoextend-extending-word-embeddings-embeddings/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
AutoExtend tackles the lack of vector representations for the non-word units of lexical resources, such as WordNet synsets and lexemes. It does this without a sense-annotated corpus, since such corpora are too costly to build at a useful scale. The method takes any pretrained word embeddings and maps them into synset and lexeme vectors that share the original vector space.

The key assumption is additive: a word vector is the sum of its lexeme vectors, and a synset vector is the sum of its member lexemes. These "synset constraints" are built into an autoencoder. Words are encoded into synsets (the hidden layer) and then decoded back into words. Lexeme constraints align the encoder-side and decoder-side lexeme vectors, and further WordNet relations add extra constraints. Because dimensions are treated independently and only existing lexemes carry weights, the problem splits into parallel per-dimension autoencoders.

The authors report state-of-the-art word similarity results on SCWS and strong word sense disambiguation (WSD) results. This matters because better word embeddings can immediately be transferred to resource entries.

## Key points
- The model ties words, lexemes and synsets together using diagonal distribution matrices. The full rank-4 tensor has about 10^15 entries, but sparsity leaves only about 10^7 learnable parameters.
- Training splits into one autoencoder per embedding dimension. Each one minimizes the error of reconstructing the word vectors through the synset layer.
- Three constraint types are weighted against each other: synsets, lexemes and WordNet relations. Their effect is analyzed on WSD-additional, WSD-alone and SCWS.
- The method is designed to carry over to Freebase, where entities play the role of synsets and aliases play the role of words. Via a learned linear translation matrix, it can also extend to multilingual resources like BabelNet.
- The authors release the AutoExtend code, the synset and lexeme embeddings, and the WSD evaluation scripts.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Ludwig-Maximilians-Universität München](https://md.research.elamur.ai/free/en/universities/ludwig-maximilians-universitat-munchen/) — University, Munich, Germany · 113 works · 56,817 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

💡 Concepts (1): [💡 Lemma](https://md.research.elamur.ai/free/en/concepts/lemma/) · 24 works

## Cite

Cite as: Sascha Rothe and Hinrich Schütze. 2015. AutoExtend: Extending Word Embeddings to Embeddings for Synsets and Lexemes. Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{rothe2015autoextend,
  title = {{AutoExtend: Extending Word Embeddings to Embeddings for Synsets and Lexemes}},
  author = {Sascha Rothe and Hinrich Schütze},
  booktitle = {Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)},
  year = {2015},
  doi = {10.3115/v1/p15-1173},
  eprint = {1507.01127},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.3115/v1/p15-1173}
}
```

- [← previous It Depends: Dependency Parser Comparison Using A Web-based Evaluation Tool](https://md.research.elamur.ai/free/en/papers/choi-2015-it-depends-dependency-parser-comparison/)
- [next → Tagging Performance Correlates with Author Age](https://md.research.elamur.ai/free/en/papers/hovy-2015-tagging-performance-correlates-author-age/)
- [↑ back to papers of 2015](https://md.research.elamur.ai/free/en/papers/2015/page/2/)

## References

36 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Yoshua Bengio, Rejean Ducharme, and Pascal Vincent. 2003. [A neural probabilistic language model](https://md.research.elamur.ai/free/en/papers/bengio-2003-neural-probabilistic-language-model/). Journal of Machine Learning Research, 3:1137–1155.
2. Jiang Bian, Bin Gao, and Tie-Yan Liu. 2014. Knowledge-powered deep learning for word embedding. In Proceedings of ECML PKDD. [find](https://scholar.google.com/scholar?q=Jiang+Bian%2C+Bin+Gao%2C+and+Tie%2DYan+Liu%2E+2014%2E+Knowledge%2Dpowered+deep+learning+for+word+embedding%2E+In+Proceedings+of+ECML+PKDD%2E)
3. Emmanuel Blanchard, Mounira Harzallah, Henri Briand, and Pascale Kuntz. 2005. A typology of ontology-based semantic measures. In Proceedings of EMOI - INTEROP. [find](https://scholar.google.com/scholar?q=Emmanuel+Blanchard%2C+Mounira+Harzallah%2C+Henri+Briand%2C+and+Pascale+Kuntz%2E+2005%2E+A+typology+of+ontology%2Dbased+semantic+measures%2E+In+Proceedings+of+EMOI+%2D+INTEROP%2E)
4. Kurt Bollacker, Colin Evans, Praveen Paritosh, Tim Sturge, and Jamie Taylor. 2008. Freebase: a collaboratively created graph database for structuring human knowledge. In Proceedings of ACM SIGMOD. [find](https://scholar.google.com/scholar?q=Kurt+Bollacker%2C+Colin+Evans%2C+Praveen+Paritosh%2C+Tim+Sturge%2C+and+Jamie+Taylor%2E+2008%2E+Freebase%3A+a+collaboratively+created+graph+database+for+structuring+human+knowledge%2E+In+Proceedings+of+ACM+SIGMOD%2E)
5. Antoine Bordes, Jason Weston, Ronan Collobert, Yoshua Bengio, et al. 2011. Learning structured embeddings of knowledge bases. In Proceedings of AAAI. [find](https://scholar.google.com/scholar?q=Antoine+Bordes%2C+Jason+Weston%2C+Ronan+Collobert%2C+Yoshua+Bengio%2C+et+al%2E+2011%2E+Learning+structured+embeddings+of+knowledge+bases%2E+In+Proceedings+of+AAAI%2E)
6. Antoine Bordes, Xavier Glorot, Jason Weston, and Yoshua Bengio. 2012. Joint learning of words and meaning representations for open-text semantic parsing. In Proceedings of AISTATS. [find](https://scholar.google.com/scholar?q=Antoine+Bordes%2C+Xavier+Glorot%2C+Jason+Weston%2C+and+Yoshua+Bengio%2E+2012%2E+Joint+learning+of+words+and+meaning+representations+for+open%2Dtext+semantic+parsing%2E+In+Proceedings+of+AISTATS%2E)
7. Elia Bruni, Nam Khanh Tran, and Marco Baroni. 2014. Multimodal distributional semantics. Journal of Artificial Intelligence Research, 49(1):1–47. [find](https://scholar.google.com/scholar?q=Elia+Bruni%2C+Nam+Khanh+Tran%2C+and+Marco+Baroni%2E+2014%2E+Multimodal+distributional+semantics%2E+Journal+of+Artificial+Intelligence+Research%2C+49%281%29%3A1%E2%80%9347%2E)
8. Xinxiong Chen, Zhiyuan Liu, and Maosong Sun. 2014. [A unified model for word sense representation and disambiguation](https://md.research.elamur.ai/free/en/papers/chen-2014-unified-model-word-sense-representation/). In Proceedings of EMNLP.
9. Ronan Collobert and Jason Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multitask learning. In Proceedings of ICML. [find](https://scholar.google.com/scholar?q=Ronan+Collobert+and+Jason+Weston%2E+2008%2E+A+unified+architecture+for+natural+language+processing%3A+Deep+neural+networks+with+multitask+learning%2E+In+Proceedings+of+ICML%2E)
10. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. [Natural language processing (almost) from scratch](https://md.research.elamur.ai/free/en/papers/collobert-2011-natural-language-processing-almost-scratch/). Journal of Machine Learning Research, 12:2493–2537.
11. Christiane Fellbaum. 1998. WordNet: An Electronic Lexical Database. Bradford Books. [find](https://scholar.google.com/scholar?q=Christiane+Fellbaum%2E+1998%2E+WordNet%3A+An+Electronic+Lexical+Database%2E+Bradford+Books%2E)
12. Lev Finkelstein, Evgeniy Gabrilovich, Yossi Matias, Ehud Rivlin, Zach Solan, Gadi Wolfman, and Eytan Ruppin. 2001. Placing search in context: The concept revisited. In Proceedings of WWW. [find](https://scholar.google.com/scholar?q=Lev+Finkelstein%2C+Evgeniy+Gabrilovich%2C+Yossi+Matias%2C+Ehud+Rivlin%2C+Zach+Solan%2C+Gadi+Wolfman%2C+and+Eytan+Ruppin%2E+2001%2E+Placing+search+in+context%3A+The+concept+revisited%2E+In+Proceedings+of+WWW%2E)
13. Jiang Guo, Wanxiang Che, Haifeng Wang, and Ting Liu. 2014. Learning sense-specific word embeddings by exploiting bilingual resources. In Proceedings of Coling, Technical Papers. [find](https://scholar.google.com/scholar?q=Jiang+Guo%2C+Wanxiang+Che%2C+Haifeng+Wang%2C+and+Ting+Liu%2E+2014%2E+Learning+sense%2Dspecific+word+embeddings+by+exploiting+bilingual+resources%2E+In+Proceedings+of+Coling%2C+Technical+Papers%2E)
14. Eric H Huang, Richard Socher, Christopher D Manning, and Andrew Y Ng. 2012. [Improving word representations via global context and multiple word prototypes](https://md.research.elamur.ai/free/en/papers/huang-2012-improving-word-representations-global-context/). In Proceedings of ACL.
15. Nal Kalchbrenner, Edward Grefenstette, and Phil Blunsom. 2014. [A convolutional neural network for modelling sentences](https://md.research.elamur.ai/free/en/papers/kalchbrenner-2014-convolutional-neural-network-modelling-sentences/). In Proceedings of ACL.
16. Adam Kilgarriff. 2001. [English lexical sample task description](https://md.research.elamur.ai/free/en/papers/kilgarriff-2001-english-lexical-sample-task-description/). In Proceedings of SENSEVAL-2.
17. Igor Labutov and Hod Lipson. 2013. Re-embedding words. In Proceedings of ACL. [find](https://scholar.google.com/scholar?q=Igor+Labutov+and+Hod+Lipson%2E+2013%2E+Re%2Dembedding+words%2E+In+Proceedings+of+ACL%2E)
18. Claudia Leacock and Martin Chodorow. 1998. Combining local context and wordnet similarity for word sense identification. WordNet: An electronic lexical database, 49(2):265–283. [find](https://scholar.google.com/scholar?q=Claudia+Leacock+and+Martin+Chodorow%2E+1998%2E+Combining+local+context+and+wordnet+similarity+for+word+sense+identification%2E+WordNet%3A+An+electronic+lexical+database%2C+49%282%29%3A265%E2%80%93283%2E)
19. Rada Mihalcea, Timothy Chklovski, and Adam Kilgarriff. 2004. [The senseval-3 english lexical sample task](https://md.research.elamur.ai/free/en/papers/mihalcea-2004-senseval-3-english-lexical-sample/). In Proceedings of SENSEVAL-3.
20. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013a. [Efficient estimation of word representations in vector space](https://md.research.elamur.ai/free/en/papers/mikolov-2013-efficient-estimation-word-representations-vector/). arXiv preprint arXiv:1301.3781.
21. Tomas Mikolov, Quoc V Le, and Ilya Sutskever. 2013b. Exploiting similarities among languages for machine translation. arXiv preprint arXiv:1309.4168. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Quoc+V+Le%2C+and+Ilya+Sutskever%2E+2013b%2E+Exploiting+similarities+among+languages+for+machine+translation%2E+arXiv+preprint+arXiv%3A1309%2E4168%2E)
22. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013c. [Distributed representations of words and phrases and their compositionality](https://md.research.elamur.ai/free/en/papers/mikolov-2013-distributed-representations-words-phrases-their/). In Proceedings of NIPS.
23. George A Miller and Walter G Charles. 1991. Contextual correlates of semantic similarity. Language and Cognitive Processes, 6(1):1–28. [find](https://scholar.google.com/scholar?q=George+A+Miller+and+Walter+G+Charles%2E+1991%2E+Contextual+correlates+of+semantic+similarity%2E+Language+and+Cognitive+Processes%2C+6%281%29%3A1%E2%80%9328%2E)
24. Andriy Mnih and Geoffrey Hinton. 2007. Three new graphical models for statistical language modelling. In Proceedings of ICML. [find](https://scholar.google.com/scholar?q=Andriy+Mnih+and+Geoffrey+Hinton%2E+2007%2E+Three+new+graphical+models+for+statistical+language+modelling%2E+In+Proceedings+of+ICML%2E)
25. Andriy Mnih and Geoffrey E Hinton. 2009. A scalable hierarchical distributed language model. In Proceedings of NIPS. [find](https://scholar.google.com/scholar?q=Andriy+Mnih+and+Geoffrey+E+Hinton%2E+2009%2E+A+scalable+hierarchical+distributed+language+model%2E+In+Proceedings+of+NIPS%2E)
26. Roberto Navigli and Simone Paolo Ponzetto. 2010. [Babelnet: Building a very large multilingual semantic network](https://md.research.elamur.ai/free/en/papers/navigli-2010-babelnet-building-very-large-multilingual/). In Proceedings of ACL.
27. Arvind Neelakantan, Jeevan Shankar, Alexandre Passos, and Andrew McCallum. 2014. [Efficient non-parametric estimation of multiple embeddings per word in vector space](https://md.research.elamur.ai/free/en/papers/neelakantan-2014-efficient-non-parametric-estimation-multiple/). In Proceedings of EMNLP.
28. Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. [Glove: Global vectors for word representation](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/). In Proceedings of EMNLP.
29. Joseph Reisinger and Raymond J Mooney. 2010. [Multi-prototype vector-space models of word meaning](https://md.research.elamur.ai/free/en/papers/reisinger-2010-multi-prototype-vector-space-models/). In Proceedings of NAACL.
30. Herbert Rubenstein and John B Goodenough. 1965. Contextual correlates of synonymy. Communications of the ACM, 8(10):627–633. [find](https://scholar.google.com/scholar?q=Herbert+Rubenstein+and+John+B+Goodenough%2E+1965%2E+Contextual+correlates+of+synonymy%2E+Communications+of+the+ACM%2C+8%2810%29%3A627%E2%80%93633%2E)
31. David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. 1988. Learning representations by backpropagating errors. Cognitive Modeling, 5:213–220. [find](https://scholar.google.com/scholar?q=David+E+Rumelhart%2C+Geoffrey+E+Hinton%2C+and+Ronald+J+Williams%2E+1988%2E+Learning+representations+by+backpropagating+errors%2E+Cognitive+Modeling%2C+5%3A213%E2%80%93220%2E)
32. Hinrich Schütze. 1998. [Automatic word sense discrimination](https://md.research.elamur.ai/free/en/papers/schutze-1998-automatic-word-sense-discrimination/). Computational Linguistics, 24(1):97– 123.
33. Fei Tian, Hanjun Dai, Jiang Bian, Bin Gao, Rui Zhang, Enhong Chen, and Tie-Yan Liu. 2014. A probabilistic model for learning multi-prototype word embeddings. In Proceedings of Coling, Technical Papers. [find](https://scholar.google.com/scholar?q=Fei+Tian%2C+Hanjun+Dai%2C+Jiang+Bian%2C+Bin+Gao%2C+Rui+Zhang%2C+Enhong+Chen%2C+and+Tie%2DYan+Liu%2E+2014%2E+A+probabilistic+model+for+learning+multi%2Dprototype+word+embeddings%2E+In+Proceedings+of+Coling%2C+Technical+Papers)
34. Zhibiao Wu and Martha Palmer. 1994. Verbs semantics and lexical selection. In Proceedings of ACL. [find](https://scholar.google.com/scholar?q=Zhibiao+Wu+and+Martha+Palmer%2E+1994%2E+Verbs+semantics+and+lexical+selection%2E+In+Proceedings+of+ACL%2E)
35. Mo Yu and Mark Dredze. 2014. [Improving lexical embeddings with semantic knowledge](https://md.research.elamur.ai/free/en/papers/yu-2014-improving-lexical-embeddings-semantic-knowledge/). In Proceedings of ACL.
36. Zhi Zhong and Hwee Tou Ng. 2010. It makes sense: A wide-coverage word sense disambiguation system for free text. In Proceedings of ACL, System Demonstrations. [find](https://scholar.google.com/scholar?q=Zhi+Zhong+and+Hwee+Tou+Ng%2E+2010%2E+It+makes+sense%3A+A+wide%2Dcoverage+word+sense+disambiguation+system+for+free+text%2E+In+Proceedings+of+ACL%2C+System+Demonstrations%2E)

*Updated 30 October 2026*