---
title: "Model Directions, Not Words — paper by Zheng et al. (2026)"
description: "The authors propose Mechanistic Topic Models (MTMs), which replace words with features from sparse autoencoders (SAEs)… 73 references."
authors: ["Carolina Zheng", "Nicolas Beltran-Velez", "Sweta Karlekar", "Claudia Shi", "Achille Nazaret", "Asif Mallik", "Amir Feder", "David M. Blei"]
year: 2026
venue: "Transactions of the Association for Computational Linguistics"
source: "https://aclanthology.org/2026.tacl-1.99/"
doi: "10.1162/tacl.a.797"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zheng-2026-model-directions-not-words-mechanistic/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zheng-2026-model-directions-not-words-mechanistic/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/42/) / Model Directions, Not Words: Mechanistic Topic Models Using Sparse Autoencoders

# Model Directions, Not Words: Mechanistic Topic Models Using Sparse Autoencoders

**Authors:** Carolina Zheng, Nicolas Beltran-Velez, Sweta Karlekar, [Claudia Shi](https://md.research.elamur.ai/free/en/authors/claudia-shi/), Achille Nazaret, Asif Mallik, [Amir Feder](https://md.research.elamur.ai/free/en/authors/amir-feder/) and [David M. Blei](https://md.research.elamur.ai/free/en/authors/david-m-blei/) · **Year:** 2026 · **Venue:** [Transactions of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/tacl/) ([Transactions of the Association for Computational Linguistics (TACL) 2026](https://md.research.elamur.ai/free/en/venues/tacl/2026/)) · **Publisher:** [The MIT Press](https://md.research.elamur.ai/free/en/publishers/the-mit-press/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.tacl-1.99/) · [PDF](https://aclanthology.org/2026.tacl-1.99.pdf) · [DOI 10.1162/tacl.a.797](https://doi.org/10.1162/tacl.a.797) · [OpenAlex](https://openalex.org/W6966649665)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Classic topic models such as LDA work on word counts, and most neural variants still present topics as ranked word lists. Both approaches have trouble expressing abstract themes like tone, style or social context. The authors propose Mechanistic Topic Models (MTMs), which replace words with features from sparse autoencoders (SAEs) trained on LLM activations. Each feature is an interpretable direction with an automatically generated natural-language description.

They adapt three existing models to this feature space: mLDA, mETM and mBERTopic. Because topics are now sets of activation directions, they also act as steering vectors for controlled text generation. Word-list and feature-based topics are hard to compare directly, so the authors introduce topic judge, a tournament-style evaluation in which an LLM makes pairwise comparisons.

On eight corpora, MTMs equal or beat traditional and neural baselines on coherence. Topic judge consistently prefers them, especially on abstract and short-text data, and steering keeps generations fluent. The work shows that interpretability tools can serve practical downstream tasks.

## Key points
- Three variants are built on SAE features: mLDA (from LDA), mETM (from Embedded Topic Model) and mBERTopic (from BERTopic). The code is at github.com/blei-lab/mechanistic-topic-models.
- Evaluation uses eight datasets: 20NG, Bills, Wiki, Yelp, AGNews, GoEmo, PoemSum and WritingPrompts.
- In mLDA steering, the steered text matched the target topic better than the comparison text in 84.4–98.9% of cases; the peak was on Bills. Most wins came from strengths λ ∈ {30, 40, 50}, which produced 80% of the wins for mLDA, 77% for mETM and 86% for mBERTopic.
- Fluency is measured as perplexity under the original model. Unsteered text scores about 6.2, which rises only slightly at λ=10–20 and reaches roughly 11–20 at λ=50. Steering vectors that use all of a topic's features work better than those using only the top 1, 5 or 10 features.
- The method has two main limitations. It depends on the few LLMs with pretrained SAEs and on accurate feature labels. It also adds a GPU-heavy featurization step, plus feature filtering with GPT-4o mini, which cost $0.33 in total across the eight datasets.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Google](https://md.research.elamur.ai/free/en/companies/google/) — Company, Mountain View, United States · 492 works · 440,462 citations
- [Columbia University](https://md.research.elamur.ai/free/en/universities/columbia-university/) — University, New York, United States · 254 works · 56,747 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,954 works · [🤝 General Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/general-social-sciences/) · 254 works

🧭 Topics (1): [🔬 Interpretability](https://md.research.elamur.ai/free/en/topics/interpretability/) · 149 works

💡 Concepts (1): [💡 Topic modeling](https://md.research.elamur.ai/free/en/concepts/topic-modeling/) · 47 works

## Cite

Cite as: Carolina Zheng, Nicolas Beltran-Velez, Sweta Karlekar, Claudia Shi, Achille Nazaret, Asif Mallik, Amir Feder and David M. Blei. 2026. Model Directions, Not Words: Mechanistic Topic Models Using Sparse Autoencoders. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@article{zheng2026model,
  title = {{Model Directions, Not Words: Mechanistic Topic Models Using Sparse Autoencoders}},
  author = {Carolina Zheng, Nicolas Beltran-Velez, Sweta Karlekar, Claudia Shi, Achille Nazaret, Asif Mallik, Amir Feder and David M. Blei},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2026},
  doi = {10.1162/tacl.a.797},
  url = {https://doi.org/10.1162/tacl.a.797}
}
```

- [← previous MODE-RAG: Manifold Outlier Diagnosis and Energy-based Retrieval-Augmented Generation Evaluation](https://md.research.elamur.ai/free/en/papers/wei-2026-mode-rag-manifold-outlier-diagnosis/)
- [next → Model-Agnostic Meta Learning for Class Imbalance Adaptation](https://md.research.elamur.ai/free/en/papers/rao-2026-model-agnostic-meta-learning-class/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/42/)

## References

73 references, 26 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Edward S. Adler and John D. Wilkerson. 2018. Congressional bills project: 1995-2018. [find](https://scholar.google.com/scholar?q=Edward+S%2E+Adler+and+John+D%2E+Wilkerson%2E+2018%2E+Congressional+bills+project%3A+1995%2D2018%2E)
2. Dimo Angelov. 2020. [Top2Vec: Distributed representations of topics](https://md.research.elamur.ai/free/en/papers/angelov-2020-top2vec-distributed-representations-topics/). arXiv preprint arXiv:2008.09470.
3. Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. 2024. [Refusal in language models is mediated by a single direction](https://md.research.elamur.ai/free/en/papers/arditi-2024-refusal-language-models-mediated-single/). In Advances in Neural Information Processing Systems, volume 37, pages 136037–136083.
4. Federico Bianchi, Silvia Terragni, and Dirk Hovy. 2021a. [Pre-training is a hot topic: Contextualized document embeddings improve topic coherence](https://md.research.elamur.ai/free/en/papers/bianchi-2021-pre-training-is-hot-topic/). In Proceedings of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing, pages 759–766.
5. Federico Bianchi, Silvia Terragni, Dirk Hovy, Debora Nozza, and Elisabetta Fersini. 2021b. [Cross-lingual contextualized topic models with zero-shot learning](https://md.research.elamur.ai/free/en/papers/bianchi-2021-cross-lingual-contextualized-topic-models/). In Proceedings of the European Chapter of the Association for Computational Linguistics, pages 1676–1683.
6. Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu, and William Saunders. 2023. Language models can explain neurons in language models. Technical blog post. [find](https://scholar.google.com/scholar?q=Steven+Bills%2C+Nick+Cammarata%2C+Dan+Mossing%2C+Henk+Tillman%2C+Leo+Gao%2C+Gabriel+Goh%2C+Ilya+Sutskever%2C+Jan+Leike%2C+Jeff+Wu%2C+and+William+Saunders%2E+2023%2E+Language+models+can+explain+neurons+in+language+models%2E+T)
7. David M. Blei. 2012. Probabilistic topic models. Communications of the ACM, 55(4):77–84. [find](https://scholar.google.com/scholar?q=David+M%2E+Blei%2E+2012%2E+Probabilistic+topic+models%2E+Communications+of+the+ACM%2C+55%284%29%3A77%E2%80%9384%2E)
8. David M. Blei, Andrew Y. Ng, and Michael I. Jordan. 2003. [Latent Dirichlet allocation](https://md.research.elamur.ai/free/en/papers/blei-2003-latent-dirichlet-allocation/). Journal of Machine Learning Research, 3:993–1022.
9. Ralph A. Bradley and Milton E. Terry. 1952. Rank analysis of incomplete block designs: I. the method of paired comparisons. 39(3/4):324–345. [find](https://scholar.google.com/scholar?q=Ralph+A%2E+Bradley+and+Milton+E%2E+Terry%2E+1952%2E+Rank+analysis+of+incomplete+block+designs%3A+I%2E+the+method+of+paired+comparisons%2E+39%283%2F4%29%3A324%E2%80%93345%2E)
10. Biometrika, Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Nguyen, Brayden Mc… [find](https://scholar.google.com/scholar?q=Biometrika%2C+Trenton+Bricken%2C+Adly+Templeton%2C+Joshua+Batson%2C+Brian+Chen%2C+Adam+Jermyn%2C+Tom+Conerly%2C+Nick+Turner%2C+Cem+Anil%2C+Carson+Denison%2C+Amanda+Askell%2C+Robert+Lasenby%2C+Yifan+Wu%2C+Shauna+Kravec%2C+Nichola)
11. Sophie Burkhardt and Stefan Kramer. 2019. Decoupling sparsity and smoothness in the Dirichlet variational autoencoder topic model. Journal of Machine Learning Research, 20(131):1–27. [find](https://scholar.google.com/scholar?q=Sophie+Burkhardt+and+Stefan+Kramer%2E+2019%2E+Decoupling+sparsity+and+smoothness+in+the+Dirichlet+variational+autoencoder+topic+model%2E+Journal+of+Machine+Learning+Research%2C+20%28131%29%3A1%E2%80%9327%2E)
12. Yuanpu Cao, Tianrong Zhang, Bochuan Cao, Ziyi Yin, Lu Lin, Fenglong Ma, and Jinghui Chen. 2024. Personalized steering of large language models: Versatile steering vectors through bidirectional preference optimization. In Advances in Neural Information Processing Systems, volume 37, pages 49519–49551… [find](https://scholar.google.com/scholar?q=Yuanpu+Cao%2C+Tianrong+Zhang%2C+Bochuan+Cao%2C+Ziyi+Yin%2C+Lu+Lin%2C+Fenglong+Ma%2C+and+Jinghui+Chen%2E+2024%2E+Personalized+steering+of+large+language+models%3A+Versatile+steering+vectors+through+bidirectional+prefere)
13. Jonathan D. Chang, Jordan L. Boyd-Graber, Sean Gerrish, Chong Wang, and David M. Blei. 2009. [Reading tea leaves: How humans interpret topic models](https://md.research.elamur.ai/free/en/papers/chang-2009-reading-tea-leaves-humans-interpret-topic/). In Advances in Neural Information Processing Systems, volume 22, pages 288–296.
14. Cheng-Han Chiang and Hung-yi Lee. 2023. Can large language models be an alternative to human evaluations? In Proceedings of the Association for Computational Linguistics, pages 15607–15631. [find](https://scholar.google.com/scholar?q=Cheng%2DHan+Chiang+and+Hung%2Dyi+Lee%2E+2023%2E+Can+large+language+models+be+an+alternative+to+human+evaluations%3F+In+Proceedings+of+the+Association+for+Computational+Linguistics%2C+pages+15607%E2%80%9315631%2E)
15. Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang, Michael Jordan, Joseph E. Gonzalez, and Ion Stoica. 2024. [Chatbot Arena: An open platform for evaluating LLMs by human preference](https://md.research.elamur.ai/free/en/papers/chiang-2024-chatbot-arena-open-platform-evaluating/). In Proceedings of the International Conference…
16. Valérie Costa, Thomas Fel, Ekdeep S. Lubana, Bahareh Tolooshams, and Demba Ba. 2025. From flat to hierarchical: Extracting sparse representations with matching pursuit. In Advances in Neural Information Processing Systems, volume 38, pages 34381–34424. [find](https://scholar.google.com/scholar?q=Val%C3%A9rie+Costa%2C+Thomas+Fel%2C+Ekdeep+S%2E+Lubana%2C+Bahareh+Tolooshams%2C+and+Demba+Ba%2E+2025%2E+From+flat+to+hierarchical%3A+Extracting+sparse+representations+with+matching+pursuit%2E+In+Advances+in+Neural+Informati)
17. Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. 2024. Sparse autoencoders find highly interpretable features in language models. In Proceedings of the International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Hoagy+Cunningham%2C+Aidan+Ewart%2C+Logan+Riggs%2C+Robert+Huben%2C+and+Lee+Sharkey%2E+2024%2E+Sparse+autoencoders+find+highly+interpretable+features+in+language+models%2E+In+Proceedings+of+the+International+Conferen)
18. Dorottya Demszky, Dana Movshovitz-Attias, Jeongwoo Ko, Alan Cowen, Gaurav Nemade, and Sujith Ravi. 2020. GoEmotions: A dataset of fine-grained emotions. In Proceedings of the Association for Computational Linguistics, pages 4040–4054. [find](https://scholar.google.com/scholar?q=Dorottya+Demszky%2C+Dana+Movshovitz%2DAttias%2C+Jeongwoo+Ko%2C+Alan+Cowen%2C+Gaurav+Nemade%2C+and+Sujith+Ravi%2E+2020%2E+GoEmotions%3A+A+dataset+of+fine%2Dgrained+emotions%2E+In+Proceedings+of+the+Association+for+Computati)
19. Adji B. Dieng, Francisco J. R. Ruiz, and David M. Blei. 2020. [Topic modeling in embedding spaces](https://md.research.elamur.ai/free/en/papers/dieng-2020-topic-modeling-embedding-spaces/). Transactions of the Association for Computational Linguistics, 8:439–453.
20. Caitlin Doogan. 2022. A Topic Is Not a Theme: Towards a Contextualised Approach to Topic Modelling. Ph.D. thesis, Monash University, Clayton, Vic, Australia. [find](https://scholar.google.com/scholar?q=Caitlin+Doogan%2E+2022%2E+A+Topic+Is+Not+a+Theme%3A+Towards+a+Contextualised+Approach+to+Topic+Modelling%2E+Ph%2ED%2E+thesis%2C+Monash+University%2C+Clayton%2C+Vic%2C+Australia%2E)
21. Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. 2022. Toy models of superposition. arXiv prepr… [find](https://scholar.google.com/scholar?q=Nelson+Elhage%2C+Tristan+Hume%2C+Catherine+Olsson%2C+Nicholas+Schiefer%2C+Tom+Henighan%2C+Shauna+Kravec%2C+Zac+Hatfield%2DDodds%2C+Robert+Lasenby%2C+Dawn+Drain%2C+Carol+Chen%2C+Roger+Grosse%2C+Sam+McCandlish%2C+Jared+Kaplan%2C+D)
22. Joshua Engels, Eric J. Michaud, Isaac Liao, Wes Gurnee, and Max Tegmark. 2025. Not all language model features are one-dimensionally linear. In Proceedings of the International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Joshua+Engels%2C+Eric+J%2E+Michaud%2C+Isaac+Liao%2C+Wes+Gurnee%2C+and+Max+Tegmark%2E+2025%2E+Not+all+language+model+features+are+one%2Ddimensionally+linear%2E+In+Proceedings+of+the+International+Conference+on+Learning+)
23. Angela Fan, Mike Lewis, and Yann Dauphin. 2018. [Hierarchical neural story generation](https://md.research.elamur.ai/free/en/papers/fan-2018-hierarchical-neural-story-generation/). In Proceedings of the Association for Computational Linguistics, pages 889–898.
24. Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. 2025a. Scaling and evaluating sparse autoencoders. In Proceedings of the International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Leo+Gao%2C+Tom+Dupr%C3%A9+la+Tour%2C+Henk+Tillman%2C+Gabriel+Goh%2C+Rajan+Troll%2C+Alec+Radford%2C+Ilya+Sutskever%2C+Jan+Leike%2C+and+Jeffrey+Wu%2E+2025a%2E+Scaling+and+evaluating+sparse+autoencoders%2E+In+Proceedings+of+the+In)
25. Mingqi Gao, Yixin Liu, Xinyu Hu, Xiaojun Wan, Jonathan Bragg, and Arman Cohan. 2025b. [find](https://scholar.google.com/scholar?q=Mingqi+Gao%2C+Yixin+Liu%2C+Xinyu+Hu%2C+Xiaojun+Wan%2C+Jonathan+Bragg%2C+and+Arman+Cohan%2E+2025b%2E)
26. Re-evaluating automatic LLM system ranking for alignment with human preference. In Findings of the Association for Computational Linguistics: North American Chapter of the Association for Computational Linguistics, pages 4605– 4629. Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Ca… [find](https://scholar.google.com/scholar?q=Re%2Devaluating+automatic+LLM+system+ranking+for+alignment+with+human+preference%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+North+American+Chapter+of+the+Association+for+Computationa)
27. Leander Girrbach and Zeynep Akata. 2026. Sparse autoencoders are topic models. In Proceedings of the International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Leander+Girrbach+and+Zeynep+Akata%2E+2026%2E+Sparse+autoencoders+are+topic+models%2E+In+Proceedings+of+the+International+Conference+on+Machine+Learning%2E)
28. Thomas L. Griffiths and Mark Steyvers. 2004. Finding scientific topics. Proceedings of the National Academy of Sciences of the United States of America, 101(Suppl 1):5228–5235. [find](https://scholar.google.com/scholar?q=Thomas+L%2E+Griffiths+and+Mark+Steyvers%2E+2004%2E+Finding+scientific+topics%2E+Proceedings+of+the+National+Academy+of+Sciences+of+the+United+States+of+America%2C+101%28Suppl+1%29%3A5228%E2%80%935235%2E)
29. Maarten Grootendorst. 2022. [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://md.research.elamur.ai/free/en/papers/grootendorst-2022-bertopic-neural-topic-modeling-class/). arXiv preprint arXiv:2203.05794.
30. Evan Hernandez, Belinda Z. Li, and Jacob Andreas. 2024. [Inspecting and editing knowledge representations in language models](https://md.research.elamur.ai/free/en/papers/hernandez-2023-inspecting-editing-knowledge-representations-language/). In Proceedings of the Conference on Language Modeling.
31. Matthew Honnibal, Ines Montani, Sofie Van Lan-deghem, and Adriane Boyd. 2020. spaCy: Industrial-strength natural language processing in Python. [find](https://scholar.google.com/scholar?q=Matthew+Honnibal%2C+Ines+Montani%2C+Sofie+Van+Lan%2Ddeghem%2C+and+Adriane+Boyd%2E+2020%2E+spaCy%3A+Industrial%2Dstrength+natural+language+processing+in+Python%2E)
32. Alexander M. Hoyle, Pranav Goel, Andrew Hian-Cheong, Denis Peskov, Jordan L. Boyd-Graber, and Philip Resnik. 2021. Is automated topic model evaluation broken? The incoherence of coherence. In Advances in Neural Information Processing Systems, volume 34, pages 2018– 2033. [find](https://scholar.google.com/scholar?q=Alexander+M%2E+Hoyle%2C+Pranav+Goel%2C+Andrew+Hian%2DCheong%2C+Denis+Peskov%2C+Jordan+L%2E+Boyd%2DGraber%2C+and+Philip+Resnik%2E+2021%2E+Is+automated+topic+model+evaluation+broken%3F+The+incoherence+of+coherence%2E+In+Advances)
33. Alexander M. Hoyle, Rupak Sarkar, Pranav Goel, and Philip Resnik. 2022. Are neural topic models broken? In Findings of the Association for Computational Linguistics: Empirical Methods in Natural Language Processing, pages 5321– 5344. [find](https://scholar.google.com/scholar?q=Alexander+M%2E+Hoyle%2C+Rupak+Sarkar%2C+Pranav+Goel%2C+and+Philip+Resnik%2E+2022%2E+Are+neural+topic+models+broken%3F+In+Findings+of+the+Association+for+Computational+Linguistics%3A+Empirical+Methods+in+Natural+Langu)
34. Ken Lang. 1995. NewsWeeder: Learning to filter netnews. In Proceedings of the International Conference on Machine Learning, pages 331– 339. [find](https://scholar.google.com/scholar?q=Ken+Lang%2E+1995%2E+NewsWeeder%3A+Learning+to+filter+netnews%2E+In+Proceedings+of+the+International+Conference+on+Machine+Learning%2C+pages+331%E2%80%93+339%2E)
35. Jey Han Lau, David Newman, and Timothy Baldwin. 2014. [Machine reading tea leaves: Automatically evaluating topic coherence and topic model quality](https://md.research.elamur.ai/free/en/papers/lau-2014-machine-reading-tea-leaves-automatically/). In Proceedings of the European Chapter of the Association for Computational Linguistics, pages 530–539.
36. Haitao Li, Qian Dong, Junjie Chen, Huixue Su, Yujia Zhou, Qingyao Ai, Ziyi Ye, and Yiqun Liu. 2024. [LLMs-as-judges: A comprehensive survey on LLM-based evaluation methods](https://md.research.elamur.ai/free/en/papers/li-2024-llms-judges-comprehensive-survey-llm/). arXiv preprint arXiv:2412.05579.
37. Qintong Li, Leyang Cui, Lingpeng Kong, and Wei Bi. 2025. [Exploring the reliability of large language models as customized evaluators for diverse NLP tasks](https://md.research.elamur.ai/free/en/papers/li-2025-exploring-reliability-large-language-models/). In Proceedings of the International Conference on Computational Linguistics, pages 10325–10344.
38. Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, János Kramár, Anca Dragan, Rohin Shah, and Neel Nanda. 2024. [Gemma Scope: Open sparse autoencoders everywhere all at once on Gemma 2](https://md.research.elamur.ai/free/en/papers/lieberum-2024-gemma-scope-open-sparse-autoencoders/). In Proceedings of the BlackboxNLP Workshop: Analyzing and Interpre…
39. Johnny Lin. 2023. Neuronpedia: Interactive reference and tooling for analyzing neural networks. [find](https://scholar.google.com/scholar?q=Johnny+Lin%2E+2023%2E+Neuronpedia%3A+Interactive+reference+and+tooling+for+analyzing+neural+networks%2E)
40. Yinhong Liu, Han Zhou, Zhijiang Guo, Ehsan Shareghi, Ivan Vulić, Anna Korhonen, and Nigel Collier. 2024. [Aligning with human judgement: The role of pairwise preference in large language model evaluators](https://md.research.elamur.ai/free/en/papers/liu-2024-aligning-human-judgement-role-pairwise/). In Proceedings of the Conference on Language Modeling, pages 1–20.
41. Ridwan Mahbub, Ifrad Khan, Samiha Anuva, Md Shihab Shahriar, Md Tahmid Rahman Laskar, and Sabbir Ahmed. 2023. Unveiling the essence of poetry: Introducing a comprehensive dataset and benchmark for poem summarization. In Empirical Methods in Natural Language Processing, pages 14878–14886. [find](https://scholar.google.com/scholar?q=Ridwan+Mahbub%2C+Ifrad+Khan%2C+Samiha+Anuva%2C+Md+Shihab+Shahriar%2C+Md+Tahmid+Rahman+Laskar%2C+and+Sabbir+Ahmed%2E+2023%2E+Unveiling+the+essence+of+poetry%3A+Introducing+a+comprehensive+dataset+and+benchmark+for+poe)
42. Andrew K. McCallum. 2002. MALLET: A machine learning for language toolkit. [find](https://scholar.google.com/scholar?q=Andrew+K%2E+McCallum%2E+2002%2E+MALLET%3A+A+machine+learning+for+language+toolkit%2E)
43. Leland McInnes, John Healy, and Steve Astels. 2017. [hdbscan: Hierarchical density based clustering](https://md.research.elamur.ai/free/en/papers/mcinnes-2017-hdbscan-hierarchical-density-clustering/). Journal of Open Source Software, 2(11):205.
44. Leland McInnes, John Healy, Nathaniel Saul, and Lukas Großberger. 2018. UMAP: Uniform manifold approximation and projection. Journal of Open Source Software, 3(29):861. [find](https://scholar.google.com/scholar?q=Leland+McInnes%2C+John+Healy%2C+Nathaniel+Saul%2C+and+Lukas+Gro%C3%9Fberger%2E+2018%2E+UMAP%3A+Uniform+manifold+approximation+and+projection%2E+Journal+of+Open+Source+Software%2C+3%2829%29%3A861%2E)
45. Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2017. [Pointer sentinel mixture models](https://md.research.elamur.ai/free/en/papers/merity-2016-pointer-sentinel-mixture-models/). In Proceedings of the International Conference on Learning Representations.
46. Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. 2013. [Linguistic regularities in continuous space word representations](https://md.research.elamur.ai/free/en/papers/mikolov-2013-linguistic-regularities-continuous-space-word/). In Proceedings of the North American Chapter of the Association for Computational Linguistics, pages 746–751.
47. Rajiv Movva, Kenny Peng, Nikhil Garg, Jon Kleinberg, and Emma Pierson. 2025. Sparse autoencoders for hypothesis generation. In Proceedings of the International Conference on Machine Learning, volume 267, pages 44997–45023. [find](https://scholar.google.com/scholar?q=Rajiv+Movva%2C+Kenny+Peng%2C+Nikhil+Garg%2C+Jon+Kleinberg%2C+and+Emma+Pierson%2E+2025%2E+Sparse+autoencoders+for+hypothesis+generation%2E+In+Proceedings+of+the+International+Conference+on+Machine+Learning%2C+volume+2)
48. Ben Naismith, Phoebe Mulcaire, and Jill Burstein. 2023. [Automated evaluation of written discourse coherence using GPT-4](https://md.research.elamur.ai/free/en/papers/naismith-2023-automated-evaluation-written-discourse-coherence/). In Proceedings of the Workshop on Innovative Use of NLP for Building Educational Applications, pages 394– 403.
49. David Newman, Jey Han Lau, Karl Grieser, and Timothy Baldwin. 2010. [Automatic evaluation of topic coherence](https://md.research.elamur.ai/free/en/papers/newman-2010-automatic-evaluation-topic-coherence/). In Proceedings of the North American Chapter of the Association for Computational Linguistics, pages 100–108.
50. Kyle O’Brien, David Majercak, Xavier Fernandes, Richard Edgar, Blake Bullwinkel, Jingya Chen, Harsha Nori, Dean Carignan, Eric Horvitz, and Forough Poursabzi-Sangde. 2025. Steering language model refusal with sparse autoencoders. In Proceedings of the Workshop on Reliable and Responsible Foundation… [find](https://scholar.google.com/scholar?q=Kyle+O%E2%80%99Brien%2C+David+Majercak%2C+Xavier+Fernandes%2C+Richard+Edgar%2C+Blake+Bullwinkel%2C+Jingya+Chen%2C+Harsha+Nori%2C+Dean+Carignan%2C+Eric+Horvitz%2C+and+Forough+Poursabzi%2DSangde%2E+2025%2E+Steering+language+model+refu)
51. OpenAI. 2024. GPT-4o mini: Advancing cost-efficient intelligence. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2024%2E+GPT%2D4o+mini%3A+Advancing+cost%2Defficient+intelligence%2E)
52. OpenAI. 2025. Introducing GPT-4.1 in the API. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2025%2E+Introducing+GPT%2D4%2E1+in+the+API%2E)
53. Kiho Park, Yo Joong Choe, and Victor Veitch. 2024. [The linear representation hypothesis and the geometry of large language models](https://md.research.elamur.ai/free/en/papers/park-2023-linear-representation-hypothesis-geometry-large/). In Proceedings of the International Conference on Machine Learning, volume 235, pages 39643–39666.
54. Gonçalo Santos Paulo, Alex Troy Mallen, Caden Juang, and Nora Belrose. 2025. Automatically interpreting millions of features in large language models. In Proceedings of the International Conference on Machine Learning, volume 267, pages 48393–48421. [find](https://scholar.google.com/scholar?q=Gon%C3%A7alo+Santos+Paulo%2C+Alex+Troy+Mallen%2C+Caden+Juang%2C+and+Nora+Belrose%2E+2025%2E+Automatically+interpreting+millions+of+features+in+large+language+models%2E+In+Proceedings+of+the+International+Conference+on)
55. Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake Vanderplas, Alexandre Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, and Édouard Duchesnay. 2011. Scikit-learn:… [find](https://scholar.google.com/scholar?q=Fabian+Pedregosa%2C+Ga%C3%ABl+Varoquaux%2C+Alexandre+Gramfort%2C+Vincent+Michel%2C+Bertrand+Thirion%2C+Olivier+Grisel%2C+Mathieu+Blondel%2C+Peter+Prettenhofer%2C+Ron+Weiss%2C+Vincent+Dubourg%2C+Jake+Vanderplas%2C+Alexandre+Pass)
56. Antônio Pereira, Felipe Viegas, Diego Roberto Colombo Dias, Elisa Tuler, Ana Cláudia Machado, Guilherme Fonseca, Marcos André Gonçalves, and Leonardo Rocha. 2025. “Are the current topic modeling evaluation metrics enough?” Mitigating the limitations of topic modeling evaluation metrics using a multi… [find](https://scholar.google.com/scholar?q=Ant%C3%B4nio+Pereira%2C+Felipe+Viegas%2C+Diego+Roberto+Colombo+Dias%2C+Elisa+Tuler%2C+Ana+Cl%C3%A1udia+Machado%2C+Guilherme+Fonseca%2C+Marcos+Andr%C3%A9+Gon%C3%A7alves%2C+and+Leonardo+Rocha%2E+2025%2E+%E2%80%9CAre+the+current+topic+modeling+evalu)
57. Chau Minh Pham, Alexander M. Hoyle, Simeng Sun, Philip Resnik, and Mohit Iyyer. 2024. [TopicGPT: A prompt-based topic modeling framework](https://md.research.elamur.ai/free/en/papers/pham-2024-topicgpt-prompt-based-topic-modeling/). In Proceedings of the North American Chapter of the Association for Computational Linguistics, pages 2956–2984.
58. Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Turner. 2024. Steering Llama 2 via contrastive activation addition. In Proceedings of the Association for Computational Linguistics, pages 15504– 15522. [find](https://scholar.google.com/scholar?q=Nina+Rimsky%2C+Nick+Gabrieli%2C+Julian+Schulz%2C+Meg+Tong%2C+Evan+Hubinger%2C+and+Alexander+Turner%2E+2024%2E+Steering+Llama+2+via+contrastive+activation+addition%2E+In+Proceedings+of+the+Association+for+Computationa)
59. Mansi Sakarvadia, Aswathy Ajith, Arham Khan, Daniel Grzenda, Nathaniel Hudson, André Bauer, Kyle Chard, and Ian Foster. 2023. Memory injections: Correcting multi-hop reasoning failures during inference in transformer-based language models. In Proceedings of the BlackboxNLP Workshop: Analyzing and In… [find](https://scholar.google.com/scholar?q=Mansi+Sakarvadia%2C+Aswathy+Ajith%2C+Arham+Khan%2C+Daniel+Grzenda%2C+Nathaniel+Hudson%2C+Andr%C3%A9+Bauer%2C+Kyle+Chard%2C+and+Ian+Foster%2E+2023%2E+Memory+injections%3A+Correcting+multi%2Dhop+reasoning+failures+during+inferenc)
60. Lewis Smith, Senthooran Rajamanoharan, Arthur Conmy, Callum McDougall, Tom Lieberum, János Kramár, Rohin Shah, and Neel Nanda. 2025. Negative results for SAEs on downstream tasks and deprioritising SAE research (GDM mech interp team progress update \#2). Technical blog post. [find](https://scholar.google.com/scholar?q=Lewis+Smith%2C+Senthooran+Rajamanoharan%2C+Arthur+Conmy%2C+Callum+McDougall%2C+Tom+Lieberum%2C+J%C3%A1nos+Kram%C3%A1r%2C+Rohin+Shah%2C+and+Neel+Nanda%2E+2025%2E+Negative+results+for+SAEs+on+downstream+tasks+and+deprioritising+SA)
61. Dominik Stammbach, Vilém Zouhar, Alexander Hoyle, Mrinmaya Sachan, and Elliott Ash. 2023. Revisiting automated topic model evaluation with large language models. In Empirical Methods in Natural Language Processing, pages 9348–9357. [find](https://scholar.google.com/scholar?q=Dominik+Stammbach%2C+Vil%C3%A9m+Zouhar%2C+Alexander+Hoyle%2C+Mrinmaya+Sachan%2C+and+Elliott+Ash%2E+2023%2E+Revisiting+automated+topic+model+evaluation+with+large+language+models%2E+In+Empirical+Methods+in+Natural+Langua)
62. Alex Tamkin, Mohammad Taufeeque, and Noah D. Goodman. 2024. Codebook features: Sparse and discrete interpretability for neural networks. In Proceedings of the International Conference on Machine Learning, volume 235, pages 47535– 47563. [find](https://scholar.google.com/scholar?q=Alex+Tamkin%2C+Mohammad+Taufeeque%2C+and+Noah+D%2E+Goodman%2E+2024%2E+Codebook+features%3A+Sparse+and+discrete+interpretability+for+neural+networks%2E+In+Proceedings+of+the+International+Conference+on+Machine+Learn)
63. Daniel Chee Hian Tan, David Chanin, Aengus Lynch, Brooks Paige, Dimitrios Kanoulas, Adrià Garriga-Alonso, and Robert Kirk. 2024. Analysing the generalisation and reliability of steering vectors. In Advances in Neural Information Processing Systems, volume 37, pages 139179–139212. [find](https://scholar.google.com/scholar?q=Daniel+Chee+Hian+Tan%2C+David+Chanin%2C+Aengus+Lynch%2C+Brooks+Paige%2C+Dimitrios+Kanoulas%2C+Adri%C3%A0+Garriga%2DAlonso%2C+and+Robert+Kirk%2E+2024%2E+Analysing+the+generalisation+and+reliability+of+steering+vectors%2E+In+Ad)
64. Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L. Turner, Callum McDougall, Monte MacDiarmid, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua Batson, Adam Jermyn, S… [find](https://scholar.google.com/scholar?q=Adly+Templeton%2C+Tom+Conerly%2C+Jonathan+Marcus%2C+Jack+Lindsey%2C+Trenton+Bricken%2C+Brian+Chen%2C+Adam+Pearce%2C+Craig+Citro%2C+Emmanuel+Ameisen%2C+Andy+Jones%2C+Hoagy+Cunningham%2C+Nicholas+L%2E+Turner%2C+Callum+McDougall%2C)
65. Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J. Vazquez, Ulisse Mini, and Monte MacDiarmid. 2023. [Steering language models with activation engineering](https://md.research.elamur.ai/free/en/papers/turner-2023-steering-language-models-activation-engineering/). arXiv preprint arXiv:2308.10248.
66. Felipe Viegas, Antonio Pereira, Washington Cunha, Celso França, Claudio Andrade, Elisa Tuler, Leonardo Rocha, and Marcos André Gonçalves. 2025. Exploiting contextual embeddings in hierarchical topic modeling and investigating the limits of the current evaluation metrics. Computational Linguistics, 5… [find](https://scholar.google.com/scholar?q=Felipe+Viegas%2C+Antonio+Pereira%2C+Washington+Cunha%2C+Celso+Fran%C3%A7a%2C+Claudio+Andrade%2C+Elisa+Tuler%2C+Leonardo+Rocha%2C+and+Marcos+Andr%C3%A9+Gon%C3%A7alves%2E+2025%2E+Exploiting+contextual+embeddings+in+hierarchical+topic+m)
67. Tianlong Wang, Xianfeng Jiao, Yinghao Zhu, Zhongzhi Chen, Yifan He, Xu Chu, Junyi Gao, Yasha Wang, and Liantao Ma. 2025. Adaptive activation steering: A tuning-free LLM truthfulness improvement method for diverse hallucinations categories. In Proceedings of the Association for Computing Machinery We… [find](https://scholar.google.com/scholar?q=Tianlong+Wang%2C+Xianfeng+Jiao%2C+Yinghao+Zhu%2C+Zhongzhi+Chen%2C+Yifan+He%2C+Xu+Chu%2C+Junyi+Gao%2C+Yasha+Wang%2C+and+Liantao+Ma%2E+2025%2E+Adaptive+activation+steering%3A+A+tuning%2Dfree+LLM+truthfulness+improvement+method)
68. Xiaobao Wu, Thong Nguyen, Delvin Ce Zhang, William Yang Wang, and Anh Tuan Luu. 2024. FASTopic: Pretrained transformer is a fast, adaptive, stable, and transferable topic model. In Advances in Neural Information Processing Systems, volume 37, pages 84447–84481. [find](https://scholar.google.com/scholar?q=Xiaobao+Wu%2C+Thong+Nguyen%2C+Delvin+Ce+Zhang%2C+William+Yang+Wang%2C+and+Anh+Tuan+Luu%2E+2024%2E+FASTopic%3A+Pretrained+transformer+is+a+fast%2C+adaptive%2C+stable%2C+and+transferable+topic+model%2E+In+Advances+in+Neural+)
69. Zhengxuan Wu, Aryaman Arora, Atticus Geiger, Zheng Wang, Jing Huang, Dan Jurafsky, Christopher D. Manning, and Christopher Potts. 2025. AxBench: Steering LLMs? Even simple baselines outperform sparse autoencoders. In Proceedings of the International Conference on Machine Learning, volume 267, pages… [find](https://scholar.google.com/scholar?q=Zhengxuan+Wu%2C+Aryaman+Arora%2C+Atticus+Geiger%2C+Zheng+Wang%2C+Jing+Huang%2C+Dan+Jurafsky%2C+Christopher+D%2E+Manning%2C+and+Christopher+Potts%2E+2025%2E+AxBench%3A+Steering+LLMs%3F+Even+simple+baselines+outperform+sparse+)
70. Zeyu Yun, Yubei Chen, Bruno Olshausen, and Yann LeCun. 2021. Transformer visualization via dictionary learning: Contextualized embedding as a linear superposition of transformer factors. In Proceedings of the Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, pages 1–1… [find](https://scholar.google.com/scholar?q=Zeyu+Yun%2C+Yubei+Chen%2C+Bruno+Olshausen%2C+and+Yann+LeCun%2E+2021%2E+Transformer+visualization+via+dictionary+learning%3A+Contextualized+embedding+as+a+linear+superposition+of+transformer+factors%2E+In+Proceeding)
71. Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. [Character-level convolutional networks for text classification](https://md.research.elamur.ai/free/en/papers/zhang-2015-character-level-convolutional-networks-text/). In Advances in Neural Information Processing Systems, volume 28, pages 649– 657.
72. Zihan Zhang, Meng Fang, Ling Chen, and Mohammad Reza Namazi Rad. 2022. Is neural topic modelling better than clustering? An empirical study on clustering with contextual embeddings for topics. In Proceedings of the North American Chapter of the Association for Computational Linguistics, pages 3886–3… [find](https://scholar.google.com/scholar?q=Zihan+Zhang%2C+Meng+Fang%2C+Ling+Chen%2C+and+Mohammad+Reza+Namazi+Rad%2E+2022%2E+Is+neural+topic+modelling+better+than+clustering%3F+An+empirical+study+on+clustering+with+contextual+embeddings+for+topics%2E+In+Proc)
73. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. [Judging LLM-as-a-judge with MT-Bench and Chatbot Arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). In Advances in Neural Information Processing Systems, volu…

*Updated 2 November 2026*