---
title: "Top2Vec: Distributed Representations of Topics (2020)"
description: "Angelov proposes top2vec, which treats a joint semantic space of document and word vectors (as… 37 references, cited by 67 papers here."
authors: ["Dimo Angelov"]
year: 2020
venue: "arXiv 2020"
source: "https://arxiv.org/abs/2008.09470"
arxiv: "2008.09470"
licence: "arXiv — non-exclusive distribution license"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/angelov-2020-top2vec-distributed-representations-topics/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/angelov-2020-top2vec-distributed-representations-topics/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/5/) / Top2Vec: Distributed Representations of Topics

# Top2Vec: Distributed Representations of Topics

**Authors:** Dimo Angelov · **Year:** 2020 · **Venue:** arXiv 2020 · **Licence:** [arXiv — non-exclusive distribution license](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2008.09470) · [PDF](https://arxiv.org/pdf/2008.09470) · [arXiv 2008.09470](https://arxiv.org/abs/2008.09470) · [OpenAlex](https://openalex.org/W3080951367)

**Citations:** 35 citations (OpenAlex, 2026-10-14) · [67](https://md.research.elamur.ai/free/en/papers/angelov-2020-top2vec-distributed-representations-topics/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Classic topic models such as LDA and PLSA need a preset number of topics. They also depend on preprocessing like stop-word lists, stemming and lemmatization, and they rely on bag-of-words input that ignores word meaning. Angelov proposes top2vec, which treats a joint semantic space of document and word vectors (as learned by doc2vec) as a continuous space of topics. In that space, dense regions of document vectors are taken as prominent topics, and the number of such regions sets the number of topics. Each topic vector is the centroid of its region, and the word vectors nearest to it serve as the topic's description. Topic quality is scored with a topic information gain measure, PWI(T), on the 20 News Groups and Yahoo Answers datasets. Top2vec topics score far higher than those from LDA and PLSA, even when the baselines have stop-words removed. The approach matters because it yields readable topics with almost no preprocessing and no prior guess about topic count.

## Key points
- Topic, document and word vectors share one embedding space, so distances between any of them reflect semantic similarity. The vector arithmetic familiar from word2vec carries over to topics.
- Neither stop-word removal nor stemming/lemmatization is required, and the number of topics is found automatically from dense document regions.
- On 20 News Groups with 20 topics, the top10-word PWI(T) totals are 996.6 for top2vec versus 360.9 for LDA after stop-word removal. Many LDA topics there consist of noise tokens such as "mk, mm, mp…".
- On Yahoo Answers with 10 topics, the totals are 837.6 for top2vec versus 153.3 for LDA with stop-words removed. The comparison figures also include PLSA, both with and without stop-word filtering.
- The 300-dimensional document vectors are projected to 2D with UMAP to compare top2vec topic labels against the true newsgroup and category labels.

## Interesting ideas
- **Topics as dense regions plus centroids:** Clustering document embeddings and labeling each centroid by its nearest words turns any joint doc–word embedding into a topic model. This could suit Ukrainian corpora, where good stop-word lists and lemmatizers are harder to come by.
- **Letting density choose the topic count:** Inferring the number of topics from the data avoids tuning *t* by hand. This helps with large or unfamiliar collections such as trade documents or regional news.
- **Scoring topics by informativeness, not reconstruction:** Rating topic words by information gain penalizes generic filler words. It works as a model-agnostic way to compare topic models, or keyword extractors, across methods.
- **Topic size from nearest topic vector:** Assigning each document to its closest topic vector gives topic proportions cheaply. The same vectors also support search, since a word or topic vector can retrieve similar documents.
- **Checking topics visually against gold labels:** Projecting document vectors to 2D with UMAP and coloring by true labels, then by model topics, gives a quick qualitative check. This plot also shows where finer sub-topics sit inside a coarse category.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,954 works · [🤝 General Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/general-social-sciences/) · 254 works

💡 Concepts (2): [💡 Topic modeling](https://md.research.elamur.ai/free/en/concepts/topic-modeling/) · 47 works · [💡 Stemming](https://md.research.elamur.ai/free/en/concepts/stemming/) · 17 works

## Cite

Cite as: Dimo Angelov. 2020. Top2Vec: Distributed Representations of Topics. arXiv 2020.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{angelov2020top2vec,
  title = {{Top2Vec: Distributed Representations of Topics}},
  author = {Dimo Angelov},
  year = {2020},
  eprint = {2008.09470},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2008.09470}
}
```

- [← previous NADI 2020: The First Nuanced Arabic Dialect Identification Shared Task](https://md.research.elamur.ai/free/en/papers/mageed-2020-nadi-2020-first-nuanced-arabic/)
- [next → Findings of the WMT 2020 Shared Task on Parallel Corpus Filtering and Alignment](https://md.research.elamur.ai/free/en/papers/koehn-2020-findings-wmt-2020-shared-task/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/5/)

## References

37 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. David M. Blei, Andrew Y. Ng, and Michael I. Jordan. [Latent dirichlet allocation](https://md.research.elamur.ai/free/en/papers/blei-2003-latent-dirichlet-allocation/). J. Mach. Learn. Res., 3:993–1022, March 2003.
2. Thomas Hofmann. Probabilistic latent semantic indexing. In Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, pages 50–57, 1999. [find](https://scholar.google.com/scholar?q=Thomas+Hofmann%2E+Probabilistic+latent+semantic+indexing%2E+In+Proceedings+of+the+22nd+annual+international+ACM+SIGIR+conference+on+Research+and+development+in+information+retrieval%2C+pages+50%E2%80%9357%2C+1999%2E)
3. Jordan L Boyd-Graber and David M Blei. Syntactic topic models. In Advances in neural information processing systems, pages 185–192, 2009. [find](https://scholar.google.com/scholar?q=Jordan+L+Boyd%2DGraber+and+David+M+Blei%2E+Syntactic+topic+models%2E+In+Advances+in+neural+information+processing+systems%2C+pages+185%E2%80%93192%2C+2009%2E)
4. S. Syed and M. Spruit. Full-text or abstract? examining topic coherence scores using latent dirichlet allocation. In 2017 IEEE International Conference on Data Science and Advanced Analytics (DSAA), pages 165–174, 2017. [find](https://scholar.google.com/scholar?q=S%2E+Syed+and+M%2E+Spruit%2E+Full%2Dtext+or+abstract%3F+examining+topic+coherence+scores+using+latent+dirichlet+allocation%2E+In+2017+IEEE+International+Conference+on+Data+Science+and+Advanced+Analytics+%28DSAA%29%2C+p)
5. Kai Yang, Yi Cai, Zhenhong Chen, Ho-fung Leung, and Raymond Lau. Exploring topic discriminating power of words in latent dirichlet allocation. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2238–2247, 2016. [find](https://scholar.google.com/scholar?q=Kai+Yang%2C+Yi+Cai%2C+Zhenhong+Chen%2C+Ho%2Dfung+Leung%2C+and+Raymond+Lau%2E+Exploring+topic+discriminating+power+of+words+in+latent+dirichlet+allocation%2E+In+Proceedings+of+COLING+2016%2C+the+26th+International+Con)
6. Geoffrey E Hinton et al. Learning distributed representations of concepts. In Proceedings of the eighth annual conference of the cognitive science society, volume 1, page 12. Amherst, MA, 1986. [find](https://scholar.google.com/scholar?q=Geoffrey+E+Hinton+et+al%2E+Learning+distributed+representations+of+concepts%2E+In+Proceedings+of+the+eighth+annual+conference+of+the+cognitive+science+society%2C+volume+1%2C+page+12%2E+Amherst%2C+MA%2C+1986%2E)
7. Henry Widdowson. J.R. Firth, 1957, papers in linguistics 1934–51. International Journal of Applied Linguistics, 17:402 – 413, 10 2007. [find](https://scholar.google.com/scholar?q=Henry+Widdowson%2E+J%2ER%2E+Firth%2C+1957%2C+papers+in+linguistics+1934%E2%80%9351%2E+International+Journal+of+Applied+Linguistics%2C+17%3A402+%E2%80%93+413%2C+10+2007%2E)
8. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. [Efficient estimation of word representations in vector space](https://md.research.elamur.ai/free/en/papers/mikolov-2013-efficient-estimation-word-representations-vector/), 2013.
9. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. [Distributed representations of words and phrases and their compositionality](https://md.research.elamur.ai/free/en/papers/mikolov-2013-distributed-representations-words-phrases-their/). In Advances in neural information processing systems, pages 3111–3119, 2013.
10. Marco Baroni, Georgiana Dinu, and Germán Kruszewski. Don’t count, predict! a systematic comparison of context-counting vs. context-predicting semantic vectors. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 238–247, Baltimore… — [Don't count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors](https://md.research.elamur.ai/free/en/papers/baroni-2014-don-t-count-predict-systematic/)
11. Zhuang Bairong, Wang Wenbo, Li Zhiyu, Zheng Chonghui, and Takahiro Shinozaki. Comparative analysis of word embedding methods for dstc6 end-to-end conversation modeling track. [find](https://scholar.google.com/scholar?q=Zhuang+Bairong%2C+Wang+Wenbo%2C+Li+Zhiyu%2C+Zheng+Chonghui%2C+and+Takahiro+Shinozaki%2E+Comparative+analysis+of+word+embedding+methods+for+dstc6+end%2Dto%2Dend+conversation+modeling+track%2E)
12. Omer Levy and Yoav Goldberg. Neural word embedding as implicit matrix factorization. In Advances in neural information processing systems, pages 2177–2185, 2014. [find](https://scholar.google.com/scholar?q=Omer+Levy+and+Yoav+Goldberg%2E+Neural+word+embedding+as+implicit+matrix+factorization%2E+In+Advances+in+neural+information+processing+systems%2C+pages+2177%E2%80%932185%2C+2014%2E)
13. Jeffrey Pennington, Richard Socher, and Christopher D Manning. [Glove: Global vectors for word representation](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/). In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543, 2014.
14. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. [A neural probabilistic language model](https://md.research.elamur.ai/free/en/papers/bengio-2003-neural-probabilistic-language-model/). Journal of machine learning research, 3(Feb):1137–1155, 2003.
15. Tomáš Mikolov, Wen-tau Yih, and Geoffrey Zweig. [Linguistic regularities in continuous space word representations](https://md.research.elamur.ai/free/en/papers/mikolov-2013-linguistic-regularities-continuous-space-word/). In Proceedings of the 2013 conference of the north american chapter of the association for computational linguistics: Human language technologies, pages 746–751, 2013.
16. Omer Levy, Yoav Goldberg, and Ido Dagan. [Improving distributional similarity with lessons learned from word embeddings](https://md.research.elamur.ai/free/en/papers/levy-2015-improving-distributional-similarity-lessons-learned/). Transactions of the Association for Computational Linguistics, 3:211–225, 2015.
17. Quoc V. Le and Tomas Mikolov. [Distributed representations of sentences and documents](https://md.research.elamur.ai/free/en/papers/le-2014-distributed-representations-sentences-documents/). ArXiv, abs/1405.4053, 2014.
18. Jey Han Lau and Timothy Baldwin. An empirical evaluation of doc2vec with practical insights into document embedding generation. In Proceedings of the 1st Workshop on Representation Learning for NLP, pages 78–86, Berlin, Germany, August 2016. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jey+Han+Lau+and+Timothy+Baldwin%2E+An+empirical+evaluation+of+doc2vec+with+practical+insights+into+document+embedding+generation%2E+In+Proceedings+of+the+1st+Workshop+on+Representation+Learning+for+NLP%2C+p)
19. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). arXiv preprint arXiv:1810.04805, 2018.
20. Kevin Clark, Minh-Thang Luong, Quoc V Le, and Christopher D Manning. Electra: Pre-training text encoders as discriminators rather than generators. arXiv preprint arXiv:2003.10555, 2020. [find](https://scholar.google.com/scholar?q=Kevin+Clark%2C+Minh%2DThang+Luong%2C+Quoc+V+Le%2C+and+Christopher+D+Manning%2E+Electra%3A+Pre%2Dtraining+text+encoders+as+discriminators+rather+than+generators%2E+arXiv+preprint+arXiv%3A2003%2E10555%2C+2020%2E)
21. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language understanding by generative pre-training. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Karthik+Narasimhan%2C+Tim+Salimans%2C+and+Ilya+Sutskever%2E+Improving+language+understanding+by+generative+pre%2Dtraining%2E)
22. Thomas L Griffiths, Mark Steyvers, and Joshua B Tenenbaum. Topics in semantic representation. Psychological review, 114(2):211, 2007. [find](https://scholar.google.com/scholar?q=Thomas+L+Griffiths%2C+Mark+Steyvers%2C+and+Joshua+B+Tenenbaum%2E+Topics+in+semantic+representation%2E+Psychological+review%2C+114%282%29%3A211%2C+2007%2E)
23. Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. [Linguistic regularities in continuous space word representations](https://md.research.elamur.ai/free/en/papers/mikolov-2013-linguistic-regularities-continuous-space-word/). In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 746–751, Atlanta, Georgia, June 2…
24. Radim Řehůřek and Petr Sojka. Software Framework for Topic Modelling with Large Corpora. In Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks, pages 45–50, Valletta, Malta, May 2010. ELRA. [http://is.muni.cz/publication/884893/en](http://is.muni.cz/publication/884893/en). [↗](http://is.muni.cz/publication/884893/en)
25. Ricardo JGB Campello, Davoud Moulavi, and Jörg Sander. Density-based clustering based on hierarchical density estimates. In Pacific-Asia conference on knowledge discovery and data mining, pages 160–172. Springer, 2013. [find](https://scholar.google.com/scholar?q=Ricardo+JGB+Campello%2C+Davoud+Moulavi%2C+and+J%C3%B6rg+Sander%2E+Density%2Dbased+clustering+based+on+hierarchical+density+estimates%2E+In+Pacific%2DAsia+conference+on+knowledge+discovery+and+data+mining%2C+pages+160%E2%80%9317)
26. Leland McInnes and John Healy. Accelerated hierarchical density based clustering. 2017 IEEE International Conference on Data Mining Workshops (ICDMW), Nov 2017. [find](https://scholar.google.com/scholar?q=Leland+McInnes+and+John+Healy%2E+Accelerated+hierarchical+density+based+clustering%2E+2017+IEEE+International+Conference+on+Data+Mining+Workshops+%28ICDMW%29%2C+Nov+2017%2E)
27. Leland McInnes, John Healy, and Steve Astels. [hdbscan: Hierarchical density based clustering](https://md.research.elamur.ai/free/en/papers/mcinnes-2017-hdbscan-hierarchical-density-clustering/). The Journal of Open Source Software, 2(11):205, 2017.
28. RB Marimont and MB Shapiro. Nearest neighbour searches and the curse of dimensionality. IMA Journal of Applied Mathematics, 24(1):59–70, 1979. [find](https://scholar.google.com/scholar?q=RB+Marimont+and+MB+Shapiro%2E+Nearest+neighbour+searches+and+the+curse+of+dimensionality%2E+IMA+Journal+of+Applied+Mathematics%2C+24%281%29%3A59%E2%80%9370%2C+1979%2E)
29. Leland McInnes, John Healy, and James Melville. [Umap: Uniform manifold approximation and projection for dimension reduction](https://md.research.elamur.ai/free/en/papers/mcinnes-2018-umap-uniform-manifold-approximation-projection/). arXiv preprint arXiv:1802.03426, 2018.
30. Leland McInnes, John Healy, Nathaniel Saul, and Lukas Grossberger. Umap: Uniform manifold approximation and projection. The Journal of Open Source Software, 3(29):861, 2018. [find](https://scholar.google.com/scholar?q=Leland+McInnes%2C+John+Healy%2C+Nathaniel+Saul%2C+and+Lukas+Grossberger%2E+Umap%3A+Uniform+manifold+approximation+and+projection%2E+The+Journal+of+Open+Source+Software%2C+3%2829%29%3A861%2C+2018%2E)
31. Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of machine learning research, 9(Nov):2579–2605, 2008. [find](https://scholar.google.com/scholar?q=Laurens+van+der+Maaten+and+Geoffrey+Hinton%2E+Visualizing+data+using+t%2Dsne%2E+Journal+of+machine+learning+research%2C+9%28Nov%29%3A2579%E2%80%932605%2C+2008%2E)
32. M Cover Thomas and A Thomas Joy. Elements of information theory. New York: Wiley, 3:37–38, 1991. [find](https://scholar.google.com/scholar?q=M+Cover+Thomas+and+A+Thomas+Joy%2E+Elements+of+information+theory%2E+New+York%3A+Wiley%2C+3%3A37%E2%80%9338%2C+1991%2E)
33. Akiko Aizawa. An information-theoretic perspective of tf–idf measures. Information Processing & Management, 39(1):45–65, 2003. [find](https://scholar.google.com/scholar?q=Akiko+Aizawa%2E+An+information%2Dtheoretic+perspective+of+tf%E2%80%93idf+measures%2E+Information+Processing+%26+Management%2C+39%281%29%3A45%E2%80%9365%2C+2003%2E)
34. Kenneth Ward Church and Patrick Hanks. [Word association norms, mutual information, and lexicography](https://md.research.elamur.ai/free/en/papers/church-1990-word-association-norms-mutual-information/). Computational Linguistics, 16(1):22–29, 1990.
35. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830… [find](https://scholar.google.com/scholar?q=F%2E+Pedregosa%2C+G%2E+Varoquaux%2C+A%2E+Gramfort%2C+V%2E+Michel%2C+B%2E+Thirion%2C+O%2E+Grisel%2C+M%2E+Blondel%2C+P%2E+Prettenhofer%2C+R%2E+Weiss%2C+V%2E+Dubourg%2C+J%2E+Vanderplas%2C+A%2E+Passos%2C+D%2E+Cournapeau%2C+M%2E+Brucher%2C+M%2E+Perrot%2C+and+E%2E+Duc)
36. Jamaal Hay Wenpeng Yin and Dan Roth. [Benchmarking zero-shot text classification: Datasets, evaluation and entailment approach](https://md.research.elamur.ai/free/en/papers/yin-2019-benchmarking-zero-shot-text-classification/). In EMNLP, 2019.
37. Xiang Zhang, Junbo Zhao, and Yann LeCun. [Character-level convolutional networks for text classification](https://md.research.elamur.ai/free/en/papers/zhang-2015-character-level-convolutional-networks-text/). In Advances in neural information processing systems, pages 649–657, 2015.

*Updated 30 October 2026*