---
title: "AxBench — paper by Wu et al. (2025) — summary and ideas"
description: "The authors also propose ReFT-r1, a weakly-supervised rank-1 method that learns detection and… 69 references, cited by 59 papers here."
authors: ["Zhengxuan Wu", "Aryaman Arora", "Atticus Geiger", "Zheng Wang", "Jing Huang", "Dan Jurafsky", "Christopher D. Manning", "Christopher Potts"]
year: 2025
venue: "arXiv:2501.17148"
source: "https://arxiv.org/abs/2501.17148"
doi: "10.48550/arxiv.2501.17148"
arxiv: "2501.17148"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wu-2025-axbench-steering-llms-even-simple/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wu-2025-axbench-steering-llms-even-simple/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/2/) / AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders

# AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders

**Authors:** [Zhengxuan Wu](https://md.research.elamur.ai/free/en/authors/zhengxuan-wu/), [Aryaman Arora](https://md.research.elamur.ai/free/en/authors/aryaman-arora/), [Atticus Geiger](https://md.research.elamur.ai/free/en/authors/atticus-geiger/), [Zheng Wang](https://md.research.elamur.ai/free/en/authors/zheng-wang/), [Jing Huang](https://md.research.elamur.ai/free/en/authors/jing-huang/), [Dan Jurafsky](https://md.research.elamur.ai/free/en/authors/dan-jurafsky/), [Christopher D. Manning](https://md.research.elamur.ai/free/en/authors/christopher-d-manning/) and [Christopher Potts](https://md.research.elamur.ai/free/en/authors/christopher-e-potts/) · **Year:** 2025 · **Venue:** arXiv:2501.17148 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2501.17148) · [PDF](https://arxiv.org/pdf/2501.17148) · [DOI 10.48550/arxiv.2501.17148](https://doi.org/10.48550/arxiv.2501.17148) · [arXiv 2501.17148](https://arxiv.org/abs/2501.17148) · [OpenAlex](https://openalex.org/W4406961517)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [59](https://md.research.elamur.ai/free/en/papers/wu-2025-axbench-steering-llms-even-simple/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Representation-based steering methods, such as sparse autoencoders (SAEs), steering vectors and linear probes, are often proposed as interpretable alternatives to prompting and finetuning for controlling LLMs. Until now, no benchmark compared them at realistic scale. The authors built AxBench to fill this gap. Starting from natural-language concept descriptions taken from GemmaScope SAE labels, it uses gpt-4o-mini to generate synthetic labelled data: positive responses, negatives, and hard negatives based on polysemous words. Methods are scored on two axes:
- **Concept detection**, measured by AUROC.
- **Long-form steering**, rated by an LLM judge on fluency, instruction-following and concept presence.

Experiments on instruction-tuned Gemma-2-2B and Gemma-2-9B show that prompting steers best, followed by finetuning. For detection, simple difference-in-means leads. SAEs trail on both axes. The authors also propose ReFT-r1, a weakly-supervised rank-1 method that learns detection and steering jointly and is competitive on both. These findings temper current enthusiasm for SAE-based control. They also show that supervised dictionaries can be cheaper and stronger alternatives.

## Key points
- **Dataset and tasks:** The Concept500 dataset contains 500 concepts per site. The sites are Gemma-2-2B layers 10 and 20 and Gemma-2-9B layers 20 and 31, which gives 8 tasks (4 sites × 2 axes). Each concept has only 144 training examples, and each generated example costs under $0.00006.
- **Methods compared:** Baselines include prompting, full finetuning (SFT), LoRA and LoReFT. Representation methods include SAE, SAE-A, DiffMean, linear probe, SSV, PCA and LAT.
- **Win rates over SAEs:** On the steering axis, ReFT-r1 beats SAEs 88.0% of the time and DiffMean 61.6%. Most other supervised steering methods fail to beat SAEs.
- **Steering strength trade-off:** Larger steering factors consistently lower instruction-following. ReFT-r1 gives the best concept score at any given instruction-following level (Pareto-optimal).
- **Released resources:** The authors publish SAE-scale supervised dictionaries (SDL) for ReFT-r1 and DiffMean, plus all datasets, at huggingface.co/pyvene. Code is at github.com/stanfordnlp/axbench.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,959 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,789 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 4,145 works

## Cite

Cite as: Zhengxuan Wu, Aryaman Arora, Atticus Geiger, Zheng Wang, Jing Huang, Dan Jurafsky, Christopher D. Manning and Christopher Potts. 2025. AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders. arXiv:2501.17148.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{wu2025axbench,
  title = {{AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders}},
  author = {Zhengxuan Wu, Aryaman Arora, Atticus Geiger, Zheng Wang, Jing Huang, Dan Jurafsky, Christopher D. Manning and Christopher Potts},
  year = {2025},
  doi = {10.48550/arxiv.2501.17148},
  eprint = {2501.17148},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2501.17148}
}
```

- [← previous Astute RAG: Overcoming Imperfect Retrieval Augmentation and Knowledge Conflicts for Large Language Models](https://md.research.elamur.ai/free/en/papers/wang-2025-astute-rag-overcoming-imperfect-retrieval/)
- [next → Generalizing Verifiable Instruction Following](https://md.research.elamur.ai/free/en/papers/pyatkin-2025-generalizing-verifiable-instruction-following/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/2/)

## References

69 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Usman Anwar, Abulhair Saparov, Javier Rando, Daniel Paleka, Miles Turpin, Peter Hase, Ekdeep Singh Lubana, Erik Jenner, Stephen Casper, Oliver Sourbut, Benjamin L. Edelman, Zhaowei Zhang, Mario Günther, Anton Korinek, Jose Hernandez-Orallo, Lewis Hammond, Eric Bigelow, Alexander Pan, Lauro Langosco… — [Foundational Challenges in Assuring Alignment and Safety of Large Language Models](https://md.research.elamur.ai/free/en/papers/anwar-2024-foundational-challenges-assuring-alignment-safety/)
2. Yonatan Belinkov, Nadir Durrani, Fahim Dalvi, Hassan Sajjad, and James Glass. [What do neural machine translation models learn about morphology?](https://md.research.elamur.ai/free/en/papers/belinkov-2017-what-do-neural-machine-translation/) In Regina Barzilay and Min-Yen Kan, editors, Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)…
3. Usha Bhalla, Suraj Srinivas, Asma Ghandeharioun, and Himabindu Lakkaraju. Towards unifying interpretability and control: Evaluation via intervention. arXiv:2411.04430, 2025. URL [https://arxiv.org/](https://arxiv.org/) abs/2411.04430. — [arXiv:2411.04430](https://arxiv.org/abs/2411.04430) [↗](https://arxiv.org/)
4. Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu, and William Saunders. Language models can explain neurons in language models, 2023. URL [https://openaipublic.blob.core.windows.net/](https://openaipublic.blob.core.windows.net/) neuron-explainer/paper/index.html. [↗](https://openaipublic.blob.core.windows.net/)
5. Tolga Bolukbasi, Kai-Wei Chang, James Y. Zou, Venkatesh Saligrama, and Adam T. Kalai. [Man is to computer programmer as woman is to homemaker? Debiasing word embeddings](https://md.research.elamur.ai/free/en/papers/bolukbasi-2016-man-computer-programmer-woman-homemaker/). In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 29…
6. Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. Toy models of superposition. Transformer Circu… [find](https://scholar.google.com/scholar?q=Nelson+Elhage%2C+Tristan+Hume%2C+Catherine+Olsson%2C+Nicholas+Schiefer%2C+Tom+Henighan%2C+Shauna+Kravec%2C+Zac+Hatfield%2DDodds%2C+Robert+Lasenby%2C+Dawn+Drain%2C+Carol+Chen%2C+Roger+Grosse%2C+Sam+McCandlish%2C+Jared+Kaplan%2C+D)
7. Joschka Braun, Dmitrii Krasheninnikov, Usman Anwar, Robert Kirk, Daniel Tan, and David Scott Krueger. A sober look at steering vectors for LLMs. In Alignment Forum, 2024. URL [https://www.alignmentforum.org/posts/QQP4nq7TXg89CJGBh/](https://www.alignmentforum.org/posts/QQP4nq7TXg89CJGBh/) a-sober-look-at-steering-vectors-for-llms. [↗](https://www.alignmentforum.org/posts/QQP4nq7TXg89CJGBh/)
8. Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders, 2024. URL https://arxiv.org/abs/2406.04093. [↗](https://arxiv.org/abs/2406.04093)
9. Stephen Casper, Lennart Schulze, Oam Patel, and Dylan Hadfield-Menell. Defending against unforeseen failure modes with latent adversarial training, 2024. URL https://arxiv.org/abs/2403.05030. — [arXiv:2403.05030](https://arxiv.org/abs/2403.05030) [↗](https://arxiv.org/abs/2403.05030)
10. Sviatoslav Chalnev, Matthew Siu, and Arthur Conmy. Improving steering vectors by targeting sparse autoencoder features. arXiv:2411.02193, 2024. URL https://arxiv.org/abs/2411.02193. — [arXiv:2411.02193](https://arxiv.org/abs/2411.02193) [↗](https://arxiv.org/abs/2411.02193)
11. Lawrence Chan, Adrià Garriga-Alonso, Nicholas Goldowsky-Dill, Ryan Greenblatt, Jenny Nitishinskaya, Ansh Radhakrishnan, Buck Shlegeris, and Nate Thomas. Causal scrubbing: A method for rigorously testing interpretability hypotheses. In Alignment Forum, 2022. URL [https://shorturl.at/jZoHi](https://shorturl.at/jZoHi). [↗](https://shorturl.at/jZoHi)
12. Dami Choi, Vincent Huang, Kevin Meng, Daniel D. Johnson, Jacob Steinhardt, and Sarah Schwettmann. Scaling automatic neuron description, October 2024. URL [https://transluce.org/neuron-descriptions](https://transluce.org/neuron-descriptions). [↗](https://transluce.org/neuron-descriptions)
13. Alexis Conneau, German Kruszewski, Guillaume Lample, Loı̈c Barrault, and Marco Baroni. What you can cram into a single $&!\#* vector: Probing sentence embeddings for linguistic properties. In Iryna Gurevych and Yusuke Miyao, editors, Proceedings of the 56th Annual Meeting of the Association for Compu… — [What you can cram into a single \$&!\#* vector: Probing sentence embeddings for linguistic properties](https://md.research.elamur.ai/free/en/papers/conneau-2018-what-you-can-cram-into/)
14. Esin Durmus, Alex Tamkin, Jack Clark, Jerry Wei, Jonathan Marcus, Joshua Batson, Kunal Handa, Liane Lovitt, Meg Tong, Miles McCain, Oliver Rausch, Saffron Huang, Sam Bowman, Stuart Ritchie, Tom Henighan, and Deep Ganguli. Evaluating feature steering: A case study in mitigating social biases, 2024. U… [find](https://scholar.google.com/scholar?q=Esin+Durmus%2C+Alex+Tamkin%2C+Jack+Clark%2C+Jerry+Wei%2C+Jonathan+Marcus%2C+Joshua+Batson%2C+Kunal+Handa%2C+Liane+Lovitt%2C+Meg+Tong%2C+Miles+McCain%2C+Oliver+Rausch%2C+Saffron+Huang%2C+Sam+Bowman%2C+Stuart+Ritchie%2C+Tom+Henigh)
15. Atticus Geiger, Hanson Lu, Thomas Icard, and Christopher Potts. Causal abstractions of neural networks. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 9574–9586. Curran Associates, Inc., 2021… [find](https://scholar.google.com/scholar?q=Atticus+Geiger%2C+Hanson+Lu%2C+Thomas+Icard%2C+and+Christopher+Potts%2E+Causal+abstractions+of+neural+networks%2E+In+M%2E+Ranzato%2C+A%2E+Beygelzimer%2C+Y%2E+Dauphin%2C+P%2ES%2E+Liang%2C+and+J%2E+Wortman+Vaughan%2C+editors%2C+Advances)
16. Atticus Geiger, Zhengxuan Wu, Hanson Lu, Josh Rozner, Elisa Kreiss, Thomas Icard, Noah D. Goodman, and Christopher Potts. Inducing causal structure for interpretable neural networks. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvári, Gang Niu, and Sivan Sabato, editors, Internation… [find](https://scholar.google.com/scholar?q=Atticus+Geiger%2C+Zhengxuan+Wu%2C+Hanson+Lu%2C+Josh+Rozner%2C+Elisa+Kreiss%2C+Thomas+Icard%2C+Noah+D%2E+Goodman%2C+and+Christopher+Potts%2E+Inducing+causal+structure+for+interpretable+neural+networks%2E+In+Kamalika+Chaud)
17. Atticus Geiger, Duligur Ibeling, Amir Zur, Maheep Chaudhary, Sonakshi Chauhan, Jing Huang, Aryaman Arora, Zhengxuan Wu, Noah Goodman, Christopher Potts, and Thomas Icard. Causal abstraction: A theoretical foundation for mechanistic interpretability. arXiv:2301.04709, 2024. URL https://arxiv.org/abs/… [↗](https://arxiv.org/abs/…)
18. Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, Johan Ferret, Peter Liu, Pouya Tafti, Abe Friesen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Charline Le Lan, Sammy Jerome, Anton… [find](https://scholar.google.com/scholar?q=Gemma+Team%2C+Morgane+Riviere%2C+Shreya+Pathak%2C+Pier+Giuseppe+Sessa%2C+Cassidy+Hardin%2C+Surya+Bhupatiraju%2C+Le%CC%81onard+Hussenot%2C+Thomas+Mesnard%2C+Bobak+Shahriari%2C+Alexandre+Rame%CC%81%2C+Johan+Ferret%2C+Peter+Liu%2C+Pouya+)
19. Mario Giulianelli, Jack Harding, Florian Mohnert, Dieuwke Hupkes, and Willem Zuidema. Under the hood: Using diagnostic classifiers to investigate and improve how language models track agreement information. In Tal Linzen, Grzegorz Chrupała, and Afra Alishahi, editors, Proceedings of the 2018 EMNLP W… [find](https://scholar.google.com/scholar?q=Mario+Giulianelli%2C+Jack+Harding%2C+Florian+Mohnert%2C+Dieuwke+Hupkes%2C+and+Willem+Zuidema%2E+Under+the+hood%3A+Using+diagnostic+classifiers+to+investigate+and+improve+how+language+models+track+agreement+inform)
20. Gabriel Goh. Decoding the thought vector, 2017. URL [https://gabgoh.github.io/ThoughtVectors/](https://gabgoh.github.io/ThoughtVectors/). [↗](https://gabgoh.github.io/ThoughtVectors/)
21. Nicholas Goldowsky-Dill, Chris MacLeod, Lucas Sato, and Aryaman Arora. [Localizing model behavior with path patching](https://md.research.elamur.ai/free/en/papers/goldowsky-dill-2023-localizing-model-behavior-path-patching/). arXiv:2304.05969, 2023. URL https://arxiv.org/abs/2304.05969. [↗](https://arxiv.org/abs/2304.05969)
22. Clément Guerner, Anej Svete, Tianyu Liu, Alexander Warstadt, and Ryan Cotterell. A geometric notion of causal probing. arXiv:2307.15054, 2024. URL https://arxiv.org/abs/2307.15054. — [arXiv:2307.15054](https://arxiv.org/abs/2307.15054) [↗](https://arxiv.org/abs/2307.15054)
23. John Hewitt and Christopher D. Manning. [A structural probe for finding syntax in word representations](https://md.research.elamur.ai/free/en/papers/hewitt-2019-structural-probe-finding-syntax-word/). In Jill Burstein, Christy Doran, and Thamar Solorio, editors, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Techn…
24. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. [LoRA: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, 2022. URL [https://openreview.net/forum?](https://openreview.net/forum?) id=n… [↗](https://openreview.net/forum?)
25. Robert Huben, Hoagy Cunningham, Logan Riggs, Aidan Ewart, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, 2024. OpenReview.net. URL [https://openreview.net/foru…](https://openreview.net/foru…) [↗](https://openreview.net/foru…)
26. Caden Juang, Gonçalo Paulo, Jacob Drori, and Nora Belrose. Open source automated interpretability for sparse autoencoder features, 2024. URL [https://blog.eleuther.ai/autointerp/](https://blog.eleuther.ai/autointerp/). [↗](https://blog.eleuther.ai/autointerp/)
27. Dan Jurafsky and James H. Martin. Speech and Language Processing. Online, 2025. URL [https://web.stanford.edu/](https://web.stanford.edu/) ∼ jurafsky/slp3/. 3rd ed. draft. [↗](https://web.stanford.edu/)
28. Adam Karvonen, Dhruv Pai, Mason Wang, and Ben Keigwin. Sieve: SAEs beat baselines on a real-world task (a code generation case study). Tilde Research Blog, 2024. URL [https://www.tilderesearch.com/blog/sieve](https://www.tilderesearch.com/blog/sieve). Blog post. [↗](https://www.tilderesearch.com/blog/sieve)
29. Michael Lan, Philip Torr, Austin Meek, Ashkan Khakzar, David Krueger, and Fazl Barez. Sparse autoencoders reveal universal feature spaces across large language models. arXiv:2410.06981, 2024. URL https://arxiv.org/abs/2410.06981. — [arXiv:2410.06981](https://arxiv.org/abs/2410.06981) [↗](https://arxiv.org/abs/2410.06981)
30. Anders Boesen Lindbo Larsen, Søren Kaae Sønderby, Hugo Larochelle, and Ole Winther. Autoencoding beyond pixels using a learned similarity metric. In Maria-Florina Balcan and Kilian Q. Weinberger, editors, Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City,… [find](https://scholar.google.com/scholar?q=Anders+Boesen+Lindbo+Larsen%2C+S%C3%B8ren+Kaae+S%C3%B8nderby%2C+Hugo+Larochelle%2C+and+Ole+Winther%2E+Autoencoding+beyond+pixels+using+a+learned+similarity+metric%2E+In+Maria%2DFlorina+Balcan+and+Kilian+Q%2E+Weinberger%2C+edit)
31. JMLR.org, 2016. URL [http://proceedings.mlr.press/v48/larsen16.html](http://proceedings.mlr.press/v48/larsen16.html). [↗](http://proceedings.mlr.press/v48/larsen16.html)
32. Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. [Inference-time intervention: Eliciting truthful answers from a language model](https://md.research.elamur.ai/free/en/papers/li-2023-inference-time-intervention-eliciting-truthful/). Advances in Neural Information Processing Systems, 36, 2024. URL https://arxiv.org/abs/2306.03341. [↗](https://arxiv.org/abs/2306.03341)
33. Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. AlpacaEval: An automatic evaluator of instruction-following models. [https://github.com/tatsu-lab/alpaca](https://github.com/tatsu-lab/alpaca) eval, 5 2023. [↗](https://github.com/tatsu-lab/alpaca)
34. Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, Janos Kramar, Anca Dragan, Rohin Shah, and Neel Nanda. [Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2](https://md.research.elamur.ai/free/en/papers/lieberum-2024-gemma-scope-open-sparse-autoencoders/). In Yonatan Belinkov, Najoung Kim, Jaap Jumelet, Hosein Mohebbi, Aaron Mu…
35. Sheng Liu, Haotian Ye, Lei Xing, and James Y. Zou. [In-context vectors: Making in context learning more effective and controllable through latent space steering](https://md.research.elamur.ai/free/en/papers/liu-2023-context-vectors-making-context-learning/). In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024.
36. Open-Review.net, 2024. URL [https://openreview.net/](https://openreview.net/) forum?id=dJTChKgv3a. [↗](https://openreview.net/)
37. Aleksandar Makelov. Sparse autoencoders match supervised features for model steering on the IOI task. In ICML 2024 Workshop on Mechanistic Interpretability, 2024. URL [https://openreview.net/forum?id=JdrVuEQih5](https://openreview.net/forum?id=JdrVuEQih5). [↗](https://openreview.net/forum?id=JdrVuEQih5)
38. Samuel Marks and Max Tegmark. [The geometry of truth: Emergent linear structure in large language model representations of true/false datasets](https://md.research.elamur.ai/free/en/papers/marks-2023-geometry-truth-emergent-linear-structure/). arXiv:2310.06824, 2024. URL https://arxiv.org/abs/2310.06824. [↗](https://arxiv.org/abs/2310.06824)
39. Harry Mayne, Yushi Yang, and Adam Mahdi. Can sparse autoencoders be used to decompose and interpret steering vectors? arXiv:2411.08790, 2024. URL https://arxiv.org/abs/2411.08790. — [arXiv:2411.08790](https://arxiv.org/abs/2411.08790) [↗](https://arxiv.org/abs/2411.08790)
40. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. Locating and editing factual associations in GPT. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 17359–17372. Curran Associates, Inc., 2022.… [find](https://scholar.google.com/scholar?q=Kevin+Meng%2C+David+Bau%2C+Alex+Andonian%2C+and+Yonatan+Belinkov%2E+Locating+and+editing+factual+associations+in+GPT%2E+In+S%2E+Koyejo%2C+S%2E+Mohamed%2C+A%2E+Agarwal%2C+D%2E+Belgrave%2C+K%2E+Cho%2C+and+A%2E+Oh%2C+editors%2C+Advances+in)
41. Tomás Mikolov, Ilya Sutskever, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. [Distributed representations of words and phrases and their compositionality](https://md.research.elamur.ai/free/en/papers/mikolov-2013-distributed-representations-words-phrases-their/). In Christopher J. C. Burges, Léon Bottou, Zoubin Ghahramani, and Kilian Q. Weinberger, editors, Advances in Neural Information Processing Syst…
42. Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. [Linguistic regularities in continuous space word representations](https://md.research.elamur.ai/free/en/papers/mikolov-2013-linguistic-regularities-continuous-space-word/). In Lucy Vanderwende, Hal Daumé III, and Katrin Kirchhoff, editors, Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Hum…
43. Ulisse Mini, Peli Grietzer, Mrinank Sharma, Austin Meek, Monte MacDiarmid, and Alexander Matt Turner. Understanding and controlling a maze-solving policy network. arXiv:2310.08043, 2023. URL [https://arxiv.org/](https://arxiv.org/) abs/2310.08043. — [arXiv:2310.08043](https://arxiv.org/abs/2310.08043) [↗](https://arxiv.org/)
44. Neel Nanda, Andrew Lee, and Martin Wattenberg. [Emergent linear representations in world models of self-supervised sequence models](https://md.research.elamur.ai/free/en/papers/nanda-2023-emergent-linear-representations-world-models/). In Yonatan Belinkov, Sophie Hao, Jaap Jumelet, Najoung Kim, Arya McCarthy, and Hosein Mohebbi, editors, Proceedings of the 6th BlackboxNLP Workshop: Analyzing and Interp…
45. Kyle O’Brien, David Majercak, Xavier Fernandes, Richard Edgar, Jingya Chen, Harsha Nori, Dean Carignan, Eric Horvitz, and Forough Poursabzi-Sangde. Steering language model refusal with sparse autoencoders. arXiv:2411.11296, 2024. URL [https://arxiv.org/](https://arxiv.org/) abs/2411.11296. — [arXiv:2411.11296](https://arxiv.org/abs/2411.11296) [↗](https://arxiv.org/)
46. Kiho Park, Yo Joong Choe, and Victor Veitch. [The linear representation hypothesis and the geometry of large language models](https://md.research.elamur.ai/free/en/papers/park-2023-linear-representation-hypothesis-geometry-large/). arXiv:2311.03658, 2023. URL https://arxiv.org/abs/2311.03658. [↗](https://arxiv.org/abs/2311.03658)
47. Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake VanderPlas, Alexandre Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, and Edouard Duchesnay. Scikit-learn: Machi… [find](https://scholar.google.com/scholar?q=Fabian+Pedregosa%2C+Gae%CC%88l+Varoquaux%2C+Alexandre+Gramfort%2C+Vincent+Michel%2C+Bertrand+Thirion%2C+Olivier+Grisel%2C+Mathieu+Blondel%2C+Peter+Prettenhofer%2C+Ron+Weiss%2C+Vincent+Dubourg%2C+Jake+VanderPlas%2C+Alexandre+Pas)
48. Jeffrey Pennington, Richard Socher, and Christopher Manning. [GloVe: Global vectors for word representation](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/). In Alessandro Moschitti, Bo Pang, and Walter Daelemans, editors, Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1532–1543, Doha, Qatar, O…
49. Itamar Pres, Laura Ruis, Ekdeep Singh Lubana, and David Krueger. Towards reliable evaluation of behavior steering interventions in LLMs, 2024. URL https://arxiv.org/abs/2410.17245. — [arXiv:2410.17245](https://arxiv.org/abs/2410.17245) [↗](https://arxiv.org/abs/2410.17245)
50. Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Turner. Steering Llama 2 via contrastive activation addition. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1… [find](https://scholar.google.com/scholar?q=Nina+Rimsky%2C+Nick+Gabrieli%2C+Julian+Schulz%2C+Meg+Tong%2C+Evan+Hubinger%2C+and+Alexander+Turner%2E+Steering+Llama+2+via+contrastive+activation+addition%2E+In+Lun%2DWei+Ku%2C+Andre+Martins%2C+and+Vivek+Srikumar%2C+editor)
51. Naomi Saphra and Sarah Wiegreffe. Mechanistic? In Yonatan Belinkov, Najoung Kim, Jaap Jumelet, Hosein Mohebbi, Aaron Mueller, and Hanjie Chen, editors, Proceedings of the 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, pages 480– 498, Miami, Florida, US, November 2024.… [find](https://scholar.google.com/scholar?q=Naomi+Saphra+and+Sarah+Wiegreffe%2E+Mechanistic%3F+In+Yonatan+Belinkov%2C+Najoung+Kim%2C+Jaap+Jumelet%2C+Hosein+Mohebbi%2C+Aaron+Mueller%2C+and+Hanjie+Chen%2C+editors%2C+Proceedings+of+the+7th+BlackboxNLP+Workshop%3A+Ana)
52. Nishant Subramani, Nivedita Suresh, and Matthew Peters. [Extracting latent steering vectors from pretrained language models](https://md.research.elamur.ai/free/en/papers/subramani-2022-extracting-latent-steering-vectors-pretrained/). In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio, editors, Findings of the Association for Computational Linguistics: ACL 2022, pages 566–581, Dublin, Ireland, May 2…
53. Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine Learning -Volume 70, ICML’17, page 3319–3328. JMLR.org, 2017. [find](https://scholar.google.com/scholar?q=Mukund+Sundararajan%2C+Ankur+Taly%2C+and+Qiqi+Yan%2E+Axiomatic+attribution+for+deep+networks%2E+In+Proceedings+of+the+34th+International+Conference+on+Machine+Learning+%2DVolume+70%2C+ICML%E2%80%9917%2C+page+3319%E2%80%933328%2E+JML)
54. Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L Turner, Callum McDougall, Monte MacDiarmid, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua Batson, Adam Jermyn, Sh… [find](https://scholar.google.com/scholar?q=Adly+Templeton%2C+Tom+Conerly%2C+Jonathan+Marcus%2C+Jack+Lindsey%2C+Trenton+Bricken%2C+Brian+Chen%2C+Adam+Pearce%2C+Craig+Citro%2C+Emmanuel+Ameisen%2C+Andy+Jones%2C+Hoagy+Cunningham%2C+Nicholas+L+Turner%2C+Callum+McDougall%2C+)
55. Alexander Matt Turner, Peli Grietzer, Ulisse Mini, Monte M, and David Udell. Understanding and controlling a maze-solving policy network. Alignment Forum, March 2023a. URL [https://shorturl.at/XGtmh](https://shorturl.at/XGtmh). [↗](https://shorturl.at/XGtmh)
56. Alexander Matt Turner, Peli Grietzer, and Lisa Thiergart. Maze-solving agents: Add a top-right vector, make the agent go to the top-right. Alignment Forum, March 2023b. URL [https://shorturl.at/7Qdy9](https://shorturl.at/7Qdy9). [↗](https://shorturl.at/7Qdy9)
57. Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J. Vazquez, Ulisse Mini, and Monte Mac-Diarmid. [Steering language models with activation engineering](https://md.research.elamur.ai/free/en/papers/turner-2023-steering-language-models-activation-engineering/). arXiv:2308.10248, 2024. URL https://arxiv.org/abs/2308.10248. [↗](https://arxiv.org/abs/2308.10248)
58. Paul Upchurch, Jacob R. Gardner, Geoff Pleiss, Robert Pless, Noah Snavely, Kavita Bala, and Kilian Q. Weinberger. Deep feature interpolation for image content changes. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, pages 6090–6099. [find](https://scholar.google.com/scholar?q=Paul+Upchurch%2C+Jacob+R%2E+Gardner%2C+Geoff+Pleiss%2C+Robert+Pless%2C+Noah+Snavely%2C+Kavita+Bala%2C+and+Kilian+Q%2E+Weinberger%2E+Deep+feature+interpolation+for+image+content+changes%2E+In+2017+IEEE+Conference+on+Compu)
59. IEEE Computer Society, 2017. doi: 10.1109/CVPR.2017.645. URL https://doi.org/10.1109/CVPR.2017.645. [↗](https://doi.org/10.1109/CVPR.2017.645)
60. Teun van der Weij, Massimo Poesio, and Nandi Schoots. Extending activation steering to broad skills and multiple behaviours. arXiv:2403.05767, 2024. URL https://arxiv.org/abs/2403.05767. — [arXiv:2403.05767](https://arxiv.org/abs/2403.05767) [↗](https://arxiv.org/abs/2403.05767)
61. Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. Investigating gender bias in language models using causal mediation analysis. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Proces… [find](https://scholar.google.com/scholar?q=Jesse+Vig%2C+Sebastian+Gehrmann%2C+Yonatan+Belinkov%2C+Sharon+Qian%2C+Daniel+Nevo%2C+Yaron+Singer%2C+and+Stuart+Shieber%2E+Investigating+gender+bias+in+language+models+using+causal+mediation+analysis%2E+In+H%2E+Laroche)
62. Eric Wallace, Jens Tuyls, Junlin Wang, Sanjay Subramanian, Matt Gardner, and Sameer Singh. [AllenNLP interpret: A framework for explaining predictions of NLP models](https://md.research.elamur.ai/free/en/papers/wallace-2019-allennlp-interpret-framework-explaining-predictions/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on…
63. Kevin Ro Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. Interpretability in the wild: a circuit for indirect object identification in GPT-2 small. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, 2023. URL https://openr… [find](https://scholar.google.com/scholar?q=Kevin+Ro+Wang%2C+Alexandre+Variengien%2C+Arthur+Conmy%2C+Buck+Shlegeris%2C+and+Jacob+Steinhardt%2E+Interpretability+in+the+wild%3A+a+circuit+for+indirect+object+identification+in+GPT%2D2+small%2E+In+The+Eleventh+Inte)
64. Yulin Wang, Xuran Pan, Shiji Song, Hong Zhang, Gao Huang, and Cheng Wu. Implicit semantic data augmentation for deep networks. In Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily B. Fox, and Roman Garnett, editors, Advances in Neural Information Processing Systems 3… [find](https://scholar.google.com/scholar?q=Yulin+Wang%2C+Xuran+Pan%2C+Shiji+Song%2C+Hong+Zhang%2C+Gao+Huang%2C+and+Cheng+Wu%2E+Implicit+semantic+data+augmentation+for+deep+networks%2E+In+Hanna+M%2E+Wallach%2C+Hugo+Larochelle%2C+Alina+Beygelzimer%2C+Florence+d%E2%80%99Alche)
65. Tom White. Sampling generative networks. arXiv:1609.04468, 2016. URL https://arxiv.org/abs/1609.04468. — [arXiv:1609.04468](https://arxiv.org/abs/1609.04468) [↗](https://arxiv.org/abs/1609.04468)
66. Zhengxuan Wu, Aryaman Arora, Zheng Wang, Atticus Geiger, Dan Jurafsky, Christopher D. Manning, and Christopher Potts. [ReFT: Representation finetuning for language models](https://md.research.elamur.ai/free/en/papers/wu-2024-reft-representation-finetuning-language-models/). arXiv:2404.03592, 2024a. URL https://arxiv.org/abs/2404.03592. [↗](https://arxiv.org/abs/2404.03592)
67. Zhengxuan Wu, Atticus Geiger, Aryaman Arora, Jing Huang, Zheng Wang, Noah Goodman, Christopher Manning, and Christopher Potts. pyvene: A library for understanding and improving PyTorch models via interventions. In Kai-Wei Chang, Annie Lee, and Nazneen Rajani, editors, Proceedings of the 2024 Confere… [find](https://scholar.google.com/scholar?q=Zhengxuan+Wu%2C+Atticus+Geiger%2C+Aryaman+Arora%2C+Jing+Huang%2C+Zheng+Wang%2C+Noah+Goodman%2C+Christopher+Manning%2C+and+Christopher+Potts%2E+pyvene%3A+A+library+for+understanding+and+improving+PyTorch+models+via+inte)
68. Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J. Zico Kolter, and Dan Hendry… — [Representation Engineering: A Top-Down Approach to AI Transparency](https://md.research.elamur.ai/free/en/papers/zou-2023-representation-engineering-top-down-approach/)
69. Andy Zou, Long Phan, Justin Wang, Derek Duenas, Maxwell Lin, Maksym Andriushchenko, Rowan Wang, Zico Kolter, Matt Fredrikson, and Dan Hendrycks. [Improving alignment and robustness with circuit breakers](https://md.research.elamur.ai/free/en/papers/zou-2024-improving-alignment-robustness-circuit-breakers/), 2024. URL https://arxiv.org/abs/2406.04313. [↗](https://arxiv.org/abs/2406.04313)

*Updated 3 November 2026*