---
title: "TopicGPT: A Prompt-based Topic Modeling Framework (2024)"
description: "Classic topic models such as LDA describe each theme as a list of probable words. 66 references, cited by 75 papers here."
authors: ["Chau Minh Pham", "Alexander Hoyle", "Simeng Sun", "Philip Resnik", "Mohit Iyyer"]
year: 2024
venue: "NAACL 2024"
source: "https://aclanthology.org/2024.naacl-long.164/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/pham-2024-topicgpt-prompt-based-topic-modeling/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/pham-2024-topicgpt-prompt-based-topic-modeling/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/2/) / TopicGPT: A Prompt-based Topic Modeling Framework

# TopicGPT: A Prompt-based Topic Modeling Framework

**Authors:** [Chau Minh Pham](https://md.research.elamur.ai/free/en/authors/chau-minh-pham/), Alexander Hoyle, [Simeng Sun](https://md.research.elamur.ai/free/en/authors/simeng-sun/), [Philip Resnik](https://md.research.elamur.ai/free/en/authors/philip-resnik/) and [Mohit Iyyer](https://md.research.elamur.ai/free/en/authors/mohit-iyyer/) · **Year:** 2024 · **Venue:** [NAACL 2024](https://md.research.elamur.ai/free/en/venues/naacl/) ([NAACL 2024](https://md.research.elamur.ai/free/en/venues/naacl/2024/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2024.naacl-long.164/) · [PDF](https://aclanthology.org/2024.naacl-long.164.pdf) · [OpenAlex](https://openalex.org/W4401043789)

**Citations:** 96 citations (OpenAlex, 2026-10-14) · [75](https://md.research.elamur.ai/free/en/papers/pham-2024-topicgpt-prompt-based-topic-modeling/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Classic topic models such as LDA describe each theme as a list of probable words. Users must interpret these lists themselves and have little say over how specific or how formatted the topics are. TopicGPT addresses this by having large language models do the topic modeling through prompting, in two stages. In the first stage, GPT-4 reads a sample of documents, starting from just two human-written seed topics, and proposes labeled topics with one-sentence descriptions. It then merges near-duplicates and drops rare topics. In the second stage, GPT-3.5-turbo assigns topics to each document and cites a supporting passage from it, with a self-correction step for invalid outputs. On Wikipedia articles and Congressional bills, TopicGPT's topics match human annotations more closely than LDA, SeededLDA and BERTopic do. Users can edit or constrain topics without retraining anything. This makes the approach a verifiable, human-centered tool for content analysis. Open-source models remain weak at the generation stage.

## Key points
- On Wikipedia, TopicGPT reaches a harmonic mean purity of 0.74, compared with 0.64 for the best baseline. It also improves on normalized mutual information and adjusted Rand index. Only 30.3% of its topics are misaligned with human labels, versus 62.4% for LDA.
- During refinement, Sentence-Transformer embeddings flag topic pairs with cosine similarity ≥ 0.5. The LLM then merges those that are near-duplicates, and topics that occur rarely are removed using a frequency threshold.
- Using GPT-4 for generation and GPT-3.5-turbo for assignment costs roughly $100 per dataset.
- Mistral-7B-Instruct assigns topics competently but fails to follow the generation instructions. As generators, Mistral produced 1,418 topics and GPT-3.5-turbo produced 151, mostly overly narrow ones.
- Code is available at github.com/chtmp223/topicGPT. Known limitations include truncating documents to fit the context window, testing on English only, and reliance on closed-source models.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Maryland, College Park](https://md.research.elamur.ai/free/en/universities/university-of-maryland-college-park/) — University, College Park, United States · 132 works · 25,380 citations
- [University of Massachusetts Amherst](https://md.research.elamur.ai/free/en/universities/university-of-massachusetts-amherst/) — University, Amherst Center, United States · 106 works · 66,692 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 General Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/general-social-sciences/) · 227 works

💡 Concepts (1): [💡 Topic modeling](https://md.research.elamur.ai/free/en/concepts/topic-modeling/) · 47 works

## Cite

Cite as: Chau Minh Pham, Alexander Hoyle, Simeng Sun, Philip Resnik and Mohit Iyyer. 2024. TopicGPT: A Prompt-based Topic Modeling Framework. NAACL 2024.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{pham2024topicgpt,
  title = {{TopicGPT: A Prompt-based Topic Modeling Framework}},
  author = {Chau Minh Pham, Alexander Hoyle, Simeng Sun, Philip Resnik and Mohit Iyyer},
  booktitle = {NAACL 2024},
  year = {2024},
  doi = {10.18653/v1/2024.naacl-long.164},
  url = {https://doi.org/10.18653/v1/2024.naacl-long.164}
}
```

- [← previous Scaling Synthetic Data Creation with 1,000,000,000 Personas](https://md.research.elamur.ai/free/en/papers/ge-2024-scaling-synthetic-data-creation-1/)
- [next → ShortGPT: Layers in Large Language Models are More Redundant Than You Expect](https://md.research.elamur.ai/free/en/papers/men-2024-shortgpt-layers-large-language-models/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/2/)

## References

66 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. E Scott Adler and John Wilkerson. 2018. Congressional bills project: 1995-2018. [find](https://scholar.google.com/scholar?q=E+Scott+Adler+and+John+Wilkerson%2E+2018%2E+Congressional+bills+project%3A+1995%2D2018%2E)
2. Enrique Amigó, Julio Gonzalo, Javier Artiles, and Felisa Verdejo. 2009. A comparison of extrinsic clustering evaluation metrics based on formal constraints. Information retrieval, 12:461–486. [find](https://scholar.google.com/scholar?q=Enrique+Amig%C3%B3%2C+Julio+Gonzalo%2C+Javier+Artiles%2C+and+Felisa+Verdejo%2E+2009%2E+A+comparison+of+extrinsic+clustering+evaluation+metrics+based+on+formal+constraints%2E+Information+retrieval%2C+12%3A461%E2%80%93486%2E)
3. David Andrzejewski and Xiaojin Zhu. 2009. Latent Dirichlet Allocation with Topic-in-Set Knowledge. In Proceedings of the NAACL HLT 2009 Workshop on Semi-supervised Learning for Natural Language Processing, pages 43–48, Boulder, Colorado. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=David+Andrzejewski+and+Xiaojin+Zhu%2E+2009%2E+Latent+Dirichlet+Allocation+with+Topic%2Din%2DSet+Knowledge%2E+In+Proceedings+of+the+NAACL+HLT+2009+Workshop+on+Semi%2Dsupervised+Learning+for+Natural+Language+Proces)
4. Christian Baden, Christian Pipal, Martijn Schoonvelde, and Mariken A.C.G. van der Velden. 2021. Three gaps in computational text analysis methods for social sciences: A research agenda. Communication Methods and Measures, 16:1 – 18. [find](https://scholar.google.com/scholar?q=Christian+Baden%2C+Christian+Pipal%2C+Martijn+Schoonvelde%2C+and+Mariken+A%2EC%2EG%2E+van+der+Velden%2E+2021%2E+Three+gaps+in+computational+text+analysis+methods+for+social+sciences%3A+A+research+agenda%2E+Communication+)
5. Federico Bianchi, Silvia Terragni, and Dirk Hovy. 2021. [Pre-training is a hot topic: Contextualized document embeddings improve topic coherence](https://md.research.elamur.ai/free/en/papers/bianchi-2021-pre-training-is-hot-topic/). 2964 9
6. David M Blei, Andrew Y Ng, and Michael I Jordan. 2003. [Latent dirichlet allocation](https://md.research.elamur.ai/free/en/papers/blei-2003-latent-dirichlet-allocation/). Journal of machine Learning research, 3(Jan):993–1022.
7. Hsiu-Fang Hsieh and Sarah E Shannon. 2005. Three approaches to qualitative content analysis. Qualitative health research, 15(9):1277–1288. [find](https://scholar.google.com/scholar?q=Hsiu%2DFang+Hsieh+and+Sarah+E+Shannon%2E+2005%2E+Three+approaches+to+qualitative+content+analysis%2E+Qualitative+health+research%2C+15%289%29%3A1277%E2%80%931288%2E)
8. Jonathan Chang, Sean Gerrish, Chong Wang, Jordan Boyd-Graber, and David Blei. 2009. [Reading tea leaves: How humans interpret topic models](https://md.research.elamur.ai/free/en/papers/chang-2009-reading-tea-leaves-humans-interpret-topic/). Advances in neural information processing systems, 22.
9. Yuening Hu, Jordan Boyd-Graber, Brianna Satinoff, and Alison Smith. 2014. Interactive topic modeling. Machine learning, 95:423–469. [find](https://scholar.google.com/scholar?q=Yuening+Hu%2C+Jordan+Boyd%2DGraber%2C+Brianna+Satinoff%2C+and+Alison+Smith%2E+2014%2E+Interactive+topic+modeling%2E+Machine+learning%2C+95%3A423%E2%80%93469%2E)
10. Robert Chew, John Bollenbacher, Michael Wenger, Jessica Speer, and Annice Kim. 2023. Llm-assisted content analysis: Using large language models to support deductive coding. [find](https://scholar.google.com/scholar?q=Robert+Chew%2C+John+Bollenbacher%2C+Michael+Wenger%2C+Jessica+Speer%2C+and+Annice+Kim%2E+2023%2E+Llm%2Dassisted+content+analysis%3A+Using+large+language+models+to+support+deductive+coding%2E)
11. Jason Chuang, Sonal Gupta, Christopher Manning, and Jeffrey Heer. 2013. Topic model diagnostics: Assessing domain relevance via topical alignment. In Proceedings of the 30th International Conference on Machine Learning, volume 28 of Proceedings of Machine Learning Research, pages 612–620, Atlanta, G… [find](https://scholar.google.com/scholar?q=Jason+Chuang%2C+Sonal+Gupta%2C+Christopher+Manning%2C+and+Jeffrey+Heer%2E+2013%2E+Topic+model+diagnostics%3A+Assessing+domain+relevance+via+topical+alignment%2E+In+Proceedings+of+the+30th+International+Conference+o)
12. Caitlin Doogan and Wray Buntine. 2021. Topic model or topic twaddle? re-evaluating semantic interpretability measures. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3824–3848, Online. Associati… [find](https://scholar.google.com/scholar?q=Caitlin+Doogan+and+Wray+Buntine%2E+2021%2E+Topic+model+or+topic+twaddle%3F+re%2Devaluating+semantic+interpretability+measures%2E+In+Proceedings+of+the+2021+Conference+of+the+North+American+Chapter+of+the+Associ)
13. Ryan J. Gallagher, Kyle Reing, David Kale, and Greg Ver Steeg. 2017. Anchored Correlation Explanation: Topic Modeling with Minimal Domain Knowledge. Transactions of the Association for Computational Linguistics, 5:529–542. [find](https://scholar.google.com/scholar?q=Ryan+J%2E+Gallagher%2C+Kyle+Reing%2C+David+Kale%2C+and+Greg+Ver+Steeg%2E+2017%2E+Anchored+Correlation+Explanation%3A+Topic+Modeling+with+Minimal+Domain+Knowledge%2E+Transactions+of+the+Association+for+Computational+L)
14. Thomas Griffiths, Michael Jordan, Joshua Tenenbaum, and David Blei. 2003. Hierarchical topic models and the nested chinese restaurant process. Advances in neural information processing systems, 16. [find](https://scholar.google.com/scholar?q=Thomas+Griffiths%2C+Michael+Jordan%2C+Joshua+Tenenbaum%2C+and+David+Blei%2E+2003%2E+Hierarchical+topic+models+and+the+nested+chinese+restaurant+process%2E+Advances+in+neural+information+processing+systems%2C+16%2E)
15. Tom Griffiths. 2002. Gibbs sampling in the generative model of latent dirichlet allocation. Standford University, 518(11):1–3. [find](https://scholar.google.com/scholar?q=Tom+Griffiths%2E+2002%2E+Gibbs+sampling+in+the+generative+model+of+latent+dirichlet+allocation%2E+Standford+University%2C+518%2811%29%3A1%E2%80%933%2E)
16. Maarten Grootendorst. 2022. [Bertopic: Neural topic modeling with a class-based tf-idf procedure](https://md.research.elamur.ai/free/en/papers/grootendorst-2022-bertopic-neural-topic-modeling-class/).
17. Alexander Hoyle, Pranav Goel, Andrew Hian-Cheong, Denis Peskov, Jordan Boyd-Graber, and Philip Resnik. 2021. Is Automated Topic Model Evaluation Broken? The Incoherence of Coherence. In Advances in Neural Information Processing Systems, volume 34, pages 2018–2033. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Alexander+Hoyle%2C+Pranav+Goel%2C+Andrew+Hian%2DCheong%2C+Denis+Peskov%2C+Jordan+Boyd%2DGraber%2C+and+Philip+Resnik%2E+2021%2E+Is+Automated+Topic+Model+Evaluation+Broken%3F+The+Incoherence+of+Coherence%2E+In+Advances+in+Ne)
18. Alexander Hoyle, Rupak Sarkar, Pranav Goel, and Philip Resnik. 2023. Natural language decompositions of implicit content enable better text representations. [find](https://scholar.google.com/scholar?q=Alexander+Hoyle%2C+Rupak+Sarkar%2C+Pranav+Goel%2C+and+Philip+Resnik%2E+2023%2E+Natural+language+decompositions+of+implicit+content+enable+better+text+representations%2E)
19. Alexander Miserlis Hoyle, Pranav Goel, Rupak Sarkar, and Philip Resnik. 2022. Are neural topic models broken? In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 5321–5344, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Alexander+Miserlis+Hoyle%2C+Pranav+Goel%2C+Rupak+Sarkar%2C+and+Philip+Resnik%2E+2022%2E+Are+neural+topic+models+broken%3F+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+2022%2C+pages+5321%E2%80%935344%2C)
20. Haoyang Huang, Tianyi Tang, Dongdong Zhang, Wayne Xin Zhao, Ting Song, Yan Xia, and Furu Wei. 2023. [Not all languages are created equal in llms: Improving multilingual capability by cross-lingual-thought prompting](https://md.research.elamur.ai/free/en/papers/huang-2023-all-languages-created-equal-llms/). In Findings of Empirical Methods in Natural Language Processing.
21. Lawrence Hubert and Phipps Arabie. 1985. Comparing partitions. Journal of classification, 2:193–218. [find](https://scholar.google.com/scholar?q=Lawrence+Hubert+and+Phipps+Arabie%2E+1985%2E+Comparing+partitions%2E+Journal+of+classification%2C+2%3A193%E2%80%93218%2E)
22. Jagadeesh Jagarlamudi, Hal Daumé III, and Raghavendra Udupa. 2012. Incorporating Lexical Priors into Topic Models. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pages 204–213, Avignon, France. Association for Computational Linguistics… [find](https://scholar.google.com/scholar?q=Jagadeesh+Jagarlamudi%2C+Hal+Daum%C3%A9+III%2C+and+Raghavendra+Udupa%2E+2012%2E+Incorporating+Lexical+Priors+into+Topic+Models%2E+In+Proceedings+of+the+13th+Conference+of+the+European+Chapter+of+the+Association+for+)
23. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… [find](https://scholar.google.com/scholar?q=Albert+Q%2E+Jiang%2C+Alexandre+Sablayrolles%2C+Arthur+Mensch%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guillaume+Lample%2C+Lucile+Saulnier%2C+L%C3%A9lio+Renard+Lav)
24. Damir Korenčić, Strahil Ristov, Jelena Repar, and Jan Šnajder. 2021. A Topic Coverage Approach to Evaluation of Topic Models. IEEE Access, 9:123280– 123312. ArXiv:2012.06274 [cs]. — [arXiv:2012.06274](https://arxiv.org/abs/2012.06274) [find](https://scholar.google.com/scholar?q=Damir+Korenc%CC%8Cic%CC%81%2C+Strahil+Ristov%2C+Jelena+Repar%2C+and+Jan+%C5%A0najder%2E+2021%2E+A+Topic+Coverage+Approach+to+Evaluation+of+Topic+Models%2E+IEEE+Access%2C+9%3A123280%E2%80%93+123312%2E+ArXiv%3A2012%2E06274+%5Bcs%5D%2E+%E2%80%94+%5BarXiv%3A2012%2E0627)
25. Helvi Kyngäs. 2020. Inductive content analysis. The application of content analysis in nursing science research, pages 13–21. [find](https://scholar.google.com/scholar?q=Helvi+Kyng%C3%A4s%2E+2020%2E+Inductive+content+analysis%2E+The+application+of+content+analysis+in+nursing+science+research%2C+pages+13%E2%80%9321%2E)
26. Jey Han Lau, Karl Grieser, David Newman, and Timothy Baldwin. 2011. Automatic labelling of topic models. In Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies, pages 1536–1545. [find](https://scholar.google.com/scholar?q=Jey+Han+Lau%2C+Karl+Grieser%2C+David+Newman%2C+and+Timothy+Baldwin%2E+2011%2E+Automatic+labelling+of+topic+models%2E+In+Proceedings+of+the+49th+annual+meeting+of+the+association+for+computational+linguistics%3A+hum)
27. Haonan Li, Fajri Koto, Minghao Wu, Alham Fikri Aji, and Timothy Baldwin. 2023. Bactrian-x : A multilingual replicable instruction-following model with low-rank adaptation. [find](https://scholar.google.com/scholar?q=Haonan+Li%2C+Fajri+Koto%2C+Minghao+Wu%2C+Alham+Fikri+Aji%2C+and+Timothy+Baldwin%2E+2023%2E+Bactrian%2Dx+%3A+A+multilingual+replicable+instruction%2Dfollowing+model+with+low%2Drank+adaptation%2E)
28. Marsha M Linehan, Judith L Goodstein, Stevan L Nielsen, and John A Chiles. 1983. Reasons for staying alive when you are thinking of killing yourself: the reasons for living inventory. Journal of consulting and clinical psychology, 51(2):276. [find](https://scholar.google.com/scholar?q=Marsha+M+Linehan%2C+Judith+L+Goodstein%2C+Stevan+L+Nielsen%2C+and+John+A+Chiles%2E+1983%2E+Reasons+for+staying+alive+when+you+are+thinking+of+killing+yourself%3A+the+reasons+for+living+inventory%2E+Journal+of+consu)
29. Sengjie Liu and Christopher G Healey. 2023. Abstractive summarization of large document collections using gpt. arXiv preprint arXiv:2310.05690. — [arXiv:2310.05690](https://arxiv.org/abs/2310.05690) [find](https://scholar.google.com/scholar?q=Sengjie+Liu+and+Christopher+G+Healey%2E+2023%2E+Abstractive+summarization+of+large+document+collections+using+gpt%2E+arXiv+preprint+arXiv%3A2310%2E05690%2E+%E2%80%94+%5BarXiv%3A2310%2E05690%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2310%2E05690%29)
30. Andrew Kachites McCallum. 2002. Mallet: A machine learning for language toolkit. Http://www.cs.umass.edu/ mccallum/mallet. 2965 10 [find](https://scholar.google.com/scholar?q=Andrew+Kachites+McCallum%2E+2002%2E+Mallet%3A+A+machine+learning+for+language+toolkit%2E+Http%3A%2F%2Fwww%2Ecs%2Eumass%2Eedu%2F+mccallum%2Fmallet%2E+2965+10)
31. Qiaozhu Mei, Xuehua Shen, and ChengXiang Zhai. 2007. Automatic labeling of multinomial topic models. In Knowledge Discovery and Data Mining. [find](https://scholar.google.com/scholar?q=Qiaozhu+Mei%2C+Xuehua+Shen%2C+and+ChengXiang+Zhai%2E+2007%2E+Automatic+labeling+of+multinomial+topic+models%2E+In+Knowledge+Discovery+and+Data+Mining%2E)
32. Marina Meilă. 2007. Comparing clusterings—an information based distance. Journal of multivariate analysis, 98(5):873–895. [find](https://scholar.google.com/scholar?q=Marina+Meila%CC%86%2E+2007%2E+Comparing+clusterings%E2%80%94an+information+based+distance%2E+Journal+of+multivariate+analysis%2C+98%285%29%3A873%E2%80%93895%2E)
33. Yu Meng, Jiaxin Huang, Guangyuan Wang, Zihan Wang, Chao Zhang, Yu Zhang, and Jiawei Han. 2020. Discriminative Topic Mining via Category-Name Guided Text Embedding. In Proceedings of The Web Conference 2020, pages 2121–2132. ArXiv:1908.07162 [cs]. — [arXiv:1908.07162](https://arxiv.org/abs/1908.07162) [find](https://scholar.google.com/scholar?q=Yu+Meng%2C+Jiaxin+Huang%2C+Guangyuan+Wang%2C+Zihan+Wang%2C+Chao+Zhang%2C+Yu+Zhang%2C+and+Jiawei+Han%2E+2020%2E+Discriminative+Topic+Mining+via+Category%2DName+Guided+Text+Embedding%2E+In+Proceedings+of+The+Web+Conference)
34. Stephen Merity, Nitish Shirish Keskar, and Richard Socher. 2018. [Regularizing and optimizing LSTM language models](https://md.research.elamur.ai/free/en/papers/merity-2017-regularizing-optimizing-lstm-language-models/). In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net.
35. David Mimno, Wei Li, and Andrew McCallum. 2007. Mixtures of hierarchical topics with pachinko allocation. In Proceedings of the 24th international conference on Machine learning, pages 633–640. [find](https://scholar.google.com/scholar?q=David+Mimno%2C+Wei+Li%2C+and+Andrew+McCallum%2E+2007%2E+Mixtures+of+hierarchical+topics+with+pachinko+allocation%2E+In+Proceedings+of+the+24th+international+conference+on+Machine+learning%2C+pages+633%E2%80%93640%2E)
36. David Newman, Jey Han Lau, Karl Grieser, and Timothy Baldwin. 2010. [Automatic evaluation of topic coherence](https://md.research.elamur.ai/free/en/papers/newman-2010-automatic-evaluation-topic-coherence/). In Human language technologies: The 2010 annual conference of the North American chapter of the association for computational linguistics, pages 100–108.
37. Sergey I Nikolenko, Sergei Koltcov, and Olessia Koltsova. 2017. Topic modelling for qualitative studies. Journal of Information Science, 43(1):88–102. [find](https://scholar.google.com/scholar?q=Sergey+I+Nikolenko%2C+Sergei+Koltcov%2C+and+Olessia+Koltsova%2E+2017%2E+Topic+modelling+for+qualitative+studies%2E+Journal+of+Information+Science%2C+43%281%29%3A88%E2%80%93102%2E)
38. OpenAI. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/).
39. John Paisley, Chong Wang, David M Blei, and Michael I Jordan. 2014. Nested hierarchical dirichlet processes. IEEE transactions on pattern analysis and machine intelligence, 37(2):256–270. [find](https://scholar.google.com/scholar?q=John+Paisley%2C+Chong+Wang%2C+David+M+Blei%2C+and+Michael+I+Jordan%2E+2014%2E+Nested+hierarchical+dirichlet+processes%2E+IEEE+transactions+on+pattern+analysis+and+machine+intelligence%2C+37%282%29%3A256%E2%80%93270%2E)
40. Forough Poursabzi-Sangdeh, Jordan Boyd-Graber, Leah Findlater, and Kevin Seppi. 2016. Alto: Active learning with topic overviews for speeding label induction and document labeling. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pag… [find](https://scholar.google.com/scholar?q=Forough+Poursabzi%2DSangdeh%2C+Jordan+Boyd%2DGraber%2C+Leah+Findlater%2C+and+Kevin+Seppi%2E+2016%2E+Alto%3A+Active+learning+with+topic+overviews+for+speeding+label+induction+and+document+labeling%2E+In+Proceedings+of+t)
41. Daniel Ramage, Christopher D. Manning, and Susan Dumais. 2011. Partially labeled topic models for interpretable text mining. In Proceedings of the 17th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 457–465, San Diego California USA. ACM. [find](https://scholar.google.com/scholar?q=Daniel+Ramage%2C+Christopher+D%2E+Manning%2C+and+Susan+Dumais%2E+2011%2E+Partially+labeled+topic+models+for+interpretable+text+mining%2E+In+Proceedings+of+the+17th+ACM+SIGKDD+international+conference+on+Knowledge)
42. William M Rand. 1971. Objective criteria for the evaluation of clustering methods. Journal of the American Statistical association, 66(336):846–850. [find](https://scholar.google.com/scholar?q=William+M+Rand%2E+1971%2E+Objective+criteria+for+the+evaluation+of+clustering+methods%2E+Journal+of+the+American+Statistical+association%2C+66%28336%29%3A846%E2%80%93850%2E)
43. Nils Reimers and Iryna Gurevych. 2019. [Sentence-BERT: Sentence embeddings using Siamese BERT-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,…
44. Philip Resnik, Pranav Goel, Alexander Hoyle, Rupak Sarkar, Josh Hagedorn, Maeve Gearing, and Carol Bruce. 2022. A step-by-step protocol for curation of topic models by subject matter experts. In New Directions in Analyzing Text as Data. [find](https://scholar.google.com/scholar?q=Philip+Resnik%2C+Pranav+Goel%2C+Alexander+Hoyle%2C+Rupak+Sarkar%2C+Josh+Hagedorn%2C+Maeve+Gearing%2C+and+Carol+Bruce%2E+2022%2E+A+step%2Dby%2Dstep+protocol+for+curation+of+topic+models+by+subject+matter+experts%2E+In+New+D)
45. Philip Resnik, Bolei Ma, Alexander Hoyle, Pranav Goel, Rupak Sarkar, Maeve Gearing, Anna-Carolina Haensch, and Frauke Kreuter. 2024a. A topic-oriented protocol for content analysis of text. Manuscript in preparation. [find](https://scholar.google.com/scholar?q=Philip+Resnik%2C+Bolei+Ma%2C+Alexander+Hoyle%2C+Pranav+Goel%2C+Rupak+Sarkar%2C+Maeve+Gearing%2C+Anna%2DCarolina+Haensch%2C+and+Frauke+Kreuter%2E+2024a%2E+A+topic%2Doriented+protocol+for+content+analysis+of+text%2E+Manuscript)
46. Philip Resnik, Katherine Musacchio Schafer, Rebecca Resnik, Josh Hagedorn, and Jonathan Singer. 2024b. Reasons to live instead of dying by suicide: New insights from a computer assisted content analysis. Manuscript in preparation. [find](https://scholar.google.com/scholar?q=Philip+Resnik%2C+Katherine+Musacchio+Schafer%2C+Rebecca+Resnik%2C+Josh+Hagedorn%2C+and+Jonathan+Singer%2E+2024b%2E+Reasons+to+live+instead+of+dying+by+suicide%3A+New+insights+from+a+computer+assisted+content+analys)
47. Emil Rijcken, Floortje Scheepers, Kalliopi Zervanou, Marco Spruit, Pablo Mosteiro, and Uzay Kaymak. 2023. Towards Interpreting Topic Models with Chat-GPT: The 20th World Congress of the International Fuzzy Systems Association. [find](https://scholar.google.com/scholar?q=Emil+Rijcken%2C+Floortje+Scheepers%2C+Kalliopi+Zervanou%2C+Marco+Spruit%2C+Pablo+Mosteiro%2C+and+Uzay+Kaymak%2E+2023%2E+Towards+Interpreting+Topic+Models+with+Chat%2DGPT%3A+The+20th+World+Congress+of+the+International+)
48. Claude Elwood Shannon. 1948. A mathematical theory of communication. The Bell system technical journal, 27(3):379–423. [find](https://scholar.google.com/scholar?q=Claude+Elwood+Shannon%2E+1948%2E+A+mathematical+theory+of+communication%2E+The+Bell+system+technical+journal%2C+27%283%29%3A379%E2%80%93423%2E)
49. Noah Shinn, Federico Cassano, Beck Labash, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. [Reflexion: Language agents with verbal reinforcement learning](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/).
50. Suzanna Sia, Ayush Dalmia, and Sabrina J. Mielke. 2020. Tired of topic models? clusters of pretrained word embeddings make for fast and good topics too! In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1728–1736, Online. Association for Computa… [find](https://scholar.google.com/scholar?q=Suzanna+Sia%2C+Ayush+Dalmia%2C+and+Sabrina+J%2E+Mielke%2E+2020%2E+Tired+of+topic+models%3F+clusters+of+pretrained+word+embeddings+make+for+fast+and+good+topics+too%21+In+Proceedings+of+the+2020+Conference+on+Empiri)
51. Akash Srivastava and Charles Sutton. 2017. Autoencoding variational inference for topic models. arXiv preprint arXiv:1703.01488. — [arXiv:1703.01488](https://arxiv.org/abs/1703.01488) [find](https://scholar.google.com/scholar?q=Akash+Srivastava+and+Charles+Sutton%2E+2017%2E+Autoencoding+variational+inference+for+topic+models%2E+arXiv+preprint+arXiv%3A1703%2E01488%2E+%E2%80%94+%5BarXiv%3A1703%2E01488%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1703%2E01488%29)
52. Dominik Stammbach, Vilém Zouhar, Alexander Hoyle, Mrinmaya Sachan, and Elliott Ash. 2023. Revisiting automated topic model evaluation with large language models. [find](https://scholar.google.com/scholar?q=Dominik+Stammbach%2C+Vil%C3%A9m+Zouhar%2C+Alexander+Hoyle%2C+Mrinmaya+Sachan%2C+and+Elliott+Ash%2E+2023%2E+Revisiting+automated+topic+model+evaluation+with+large+language+models%2E)
53. Alexander Strehl and Joydeep Ghosh. 2002. Cluster ensembles—a knowledge reuse framework for combining multiple partitions. Journal of machine learning research, 3(Dec):583–617. 2966 11 [find](https://scholar.google.com/scholar?q=Alexander+Strehl+and+Joydeep+Ghosh%2E+2002%2E+Cluster+ensembles%E2%80%94a+knowledge+reuse+framework+for+combining+multiple+partitions%2E+Journal+of+machine+learning+research%2C+3%28Dec%29%3A583%E2%80%93617%2E+2966+11)
54. Simeng Sun, Yang Liu, Shuohang Wang, Chenguang Zhu, and Mohit Iyyer. 2023. Pearl: Prompting large language models to plan and execute actions over long documents. [find](https://scholar.google.com/scholar?q=Simeng+Sun%2C+Yang+Liu%2C+Shuohang+Wang%2C+Chenguang+Zhu%2C+and+Mohit+Iyyer%2E+2023%2E+Pearl%3A+Prompting+large+language+models+to+plan+and+execute+actions+over+long+documents%2E)
55. Robert H Tai, Lillian R Bentley, Xin Xia, Jason M Sitt, Sarah C Fankhauser, Ana M Chicas-Mosier, and Barnas G Monteith. 2023. Use of large language models to aid analysis of textual data. bioRxiv, pages 2023–07. [find](https://scholar.google.com/scholar?q=Robert+H+Tai%2C+Lillian+R+Bentley%2C+Xin+Xia%2C+Jason+M+Sitt%2C+Sarah+C+Fankhauser%2C+Ana+M+Chicas%2DMosier%2C+and+Barnas+G+Monteith%2E+2023%2E+Use+of+large+language+models+to+aid+analysis+of+textual+data%2E+bioRxiv%2C+pag)
56. Yee Whye Teh, Michael I Jordan, Matthew J Beal, and David M Blei. 2006. Hierarchical Dirichlet Processes. Journal of the American Statistical Association, 101(476):1566–1581. [find](https://scholar.google.com/scholar?q=Yee+Whye+Teh%2C+Michael+I+Jordan%2C+Matthew+J+Beal%2C+and+David+M+Blei%2E+2006%2E+Hierarchical+Dirichlet+Processes%2E+Journal+of+the+American+Statistical+Association%2C+101%28476%29%3A1566%E2%80%931581%2E)
57. Laure Thompson and David Mimno. 2020. Topic Modeling with Contextualized Word Representation Clusters. ArXiv:2010.12626 [cs]. — [arXiv:2010.12626](https://arxiv.org/abs/2010.12626) [find](https://scholar.google.com/scholar?q=Laure+Thompson+and+David+Mimno%2E+2020%2E+Topic+Modeling+with+Contextualized+Word+Representation+Clusters%2E+ArXiv%3A2010%2E12626+%5Bcs%5D%2E+%E2%80%94+%5BarXiv%3A2010%2E12626%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2010%2E12626%29)
58. Danya F Vears and Lynn Gillam. 2022. Inductive content analysis: A guide for beginning qualitative researchers. Focus on Health Professional Education: A Multi-disciplinary Journal, 23(1):111–127. [find](https://scholar.google.com/scholar?q=Danya+F+Vears+and+Lynn+Gillam%2E+2022%2E+Inductive+content+analysis%3A+A+guide+for+beginning+qualitative+researchers%2E+Focus+on+Health+Professional+Education%3A+A+Multi%2Ddisciplinary+Journal%2C+23%281%29%3A111%E2%80%93127%2E)
59. Nguyen Xuan Vinh, Julien Epps, and James Bailey. 2009. Information theoretic measures for clusterings comparison: is a correction for chance necessary? In Proceedings of the 26th annual international conference on machine learning, pages 1073–1080. [find](https://scholar.google.com/scholar?q=Nguyen+Xuan+Vinh%2C+Julien+Epps%2C+and+James+Bailey%2E+2009%2E+Information+theoretic+measures+for+clusterings+comparison%3A+is+a+correction+for+chance+necessary%3F+In+Proceedings+of+the+26th+annual+international+)
60. Vijay Viswanathan, Kiril Gashteovski, Carolin Lawrence, Tongshuang Wu, and Graham Neubig. 2023. [Large language models enable few-shot clustering](https://md.research.elamur.ai/free/en/papers/viswanathan-2024-large-language-models-enable-few/).
61. Hanna Wallach, David Mimno, and Andrew McCallum. 2009. Rethinking lda: Why priors matter. Advances in neural information processing systems, 22. [find](https://scholar.google.com/scholar?q=Hanna+Wallach%2C+David+Mimno%2C+and+Andrew+McCallum%2E+2009%2E+Rethinking+lda%3A+Why+priors+matter%2E+Advances+in+neural+information+processing+systems%2C+22%2E)
62. Xiaojun Wan and Tianming Wang. 2016. Automatic labeling of topic models using text summaries. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2297–2305, Berlin, Germany. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Xiaojun+Wan+and+Tianming+Wang%2E+2016%2E+Automatic+labeling+of+topic+models+using+text+summaries%2E+In+Proceedings+of+the+54th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volume+1%3A+Long)
63. Zihan Wang, Jingbo Shang, and Ruiqi Zhong. 2023. Goal-driven explainable clustering via language descriptions. [find](https://scholar.google.com/scholar?q=Zihan+Wang%2C+Jingbo+Shang%2C+and+Ruiqi+Zhong%2E+2023%2E+Goal%2Ddriven+explainable+clustering+via+language+descriptions%2E)
64. Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, et al. 2021. [Ethical and social risks of harm from language models](https://md.research.elamur.ai/free/en/papers/weidinger-2021-ethical-social-risks-harm-language/). arXiv preprint arXiv:2112.04359.
65. Yuwei Zhang, Zihan Wang, and Jingbo Shang. 2023. ClusterLLM: Large Language Models as a Guide for Text Clustering. ArXiv:2305.14871 [cs]. [find](https://scholar.google.com/scholar?q=Yuwei+Zhang%2C+Zihan+Wang%2C+and+Jingbo+Shang%2E+2023%2E+ClusterLLM%3A+Large+Language+Models+as+a+Guide+for+Text+Clustering%2E+ArXiv%3A2305%2E14871+%5Bcs%5D%2E)
66. Ying Zhao. 2005. Criterion Functions for Document Clustering. Ph.D. thesis, University of Minnesota, USA. AAI3180039. 2967 12 [find](https://scholar.google.com/scholar?q=Ying+Zhao%2E+2005%2E+Criterion+Functions+for+Document+Clustering%2E+Ph%2ED%2E+thesis%2C+University+of+Minnesota%2C+USA%2E+AAI3180039%2E+2967+12)

*Updated 31 October 2026*