---
title: "Gemma Scope — paper by Lieberum et al. (2024)"
description: "The release removes a major compute barrier, enabling the community to attempt circuit-level… 42 references, cited by 113 papers here."
authors: ["Tom Lieberum", "Senthooran Rajamanoharan", "Arthur Conmy", "Lewis Smith", "Nicolas Sonnerat", "Vikrant Varma", "Janos Kramar", "Anca Dragan", "Rohin Shah", "Neel Nanda"]
year: 2024
venue: "arXiv:2408.05147"
source: "https://arxiv.org/abs/2408.05147"
doi: "10.48550/arxiv.2408.05147"
arxiv: "2408.05147"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/lieberum-2024-gemma-scope-open-sparse-autoencoders/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/lieberum-2024-gemma-scope-open-sparse-autoencoders/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/4/) / Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2

# Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2

**Authors:** [Tom Lieberum](https://md.research.elamur.ai/free/en/authors/tom-lieberum/), [Senthooran Rajamanoharan](https://md.research.elamur.ai/free/en/authors/senthooran-rajamanoharan/), [Arthur Conmy](https://md.research.elamur.ai/free/en/authors/arthur-conmy/), [Lewis Smith](https://md.research.elamur.ai/free/en/authors/lewis-smith/), Nicolas Sonnerat, [Vikrant Varma](https://md.research.elamur.ai/free/en/authors/vikrant-varma/), [Janos Kramar](https://md.research.elamur.ai/free/en/authors/janos-kramar/), [Anca Dragan](https://md.research.elamur.ai/free/en/authors/anca-dragan/), [Rohin Shah](https://md.research.elamur.ai/free/en/authors/rohin-shah/) and [Neel Nanda](https://md.research.elamur.ai/free/en/authors/neel-nanda/) · **Year:** 2024 · **Venue:** arXiv:2408.05147 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2408.05147) · [PDF](https://arxiv.org/pdf/2408.05147) · [DOI 10.48550/arxiv.2408.05147](https://doi.org/10.48550/arxiv.2408.05147) · [arXiv 2408.05147](https://arxiv.org/abs/2408.05147) · [OpenAlex](https://openalex.org/W4402502788)

**Citations:** 2 citations (OpenAlex, 2026-10-14) · [113](https://md.research.elamur.ai/free/en/papers/lieberum-2024-gemma-scope-open-sparse-autoencoders/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Sparse autoencoders (SAEs) break a language model's internal activations into many sparse, often human-interpretable directions. They are costly to train at scale, so most interpretability work outside large labs has been limited to small models or to a single residual-stream layer of proprietary systems. Google DeepMind addresses this gap with Gemma Scope, an openly licensed collection of JumpReLU SAEs. The collection covers every layer and sub-layer of Gemma 2 2B and 9B, plus selected layers of Gemma 2 27B. The SAEs sit at three points in each transformer block: attention output, MLP output and the residual stream. They are trained with a reconstruction loss plus a direct L0 sparsity penalty, using straight-through gradient estimates for the learned thresholds. Extras include SAEs for the instruction-tuned 9B model, a transcoder suite and multi-width SAEs for studying feature splitting. Every SAE is scored on standard quality metrics, and those results are published. The release removes a major compute barrier, enabling the community to attempt circuit-level analysis of modern models and to tackle a published list of open research questions.

## Key points
- The main release has more than 400 SAEs; counting variants at different sparsity levels, it exceeds 2,000. Together they hold over 30 million learned latents.
- Each SAE saw 4–16B tokens: 4B for 16.4K-wide SAEs, 16B for 1M-wide ones and 8B for the rest. The whole effort used more than 20% of GPT-3's training compute and about 20 PiB of stored activations.
- Shared hyperparameters across all runs:
  - KDE bandwidth ε = 0.001
  - learning rate 7×10⁻⁵
  - Adam optimizer
  - batch size 4,096
  - inputs normalized to unit mean squared norm, so settings transfer across layers
  - training mostly on TPUv3
- Weights are released under CC-BY-4.0 on HuggingFace (google/gemma-scope) with a tutorial, and there is an interactive demo on Neuronpedia. The Gemma 2 models themselves use a separate custom license.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

## Cite

Cite as: Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, Janos Kramar, Anca Dragan, Rohin Shah and Neel Nanda. 2024. Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2. arXiv:2408.05147.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{lieberum2024gemma,
  title = {{Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2}},
  author = {Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, Janos Kramar, Anca Dragan, Rohin Shah and Neel Nanda},
  year = {2024},
  doi = {10.48550/arxiv.2408.05147},
  eprint = {2408.05147},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2408.05147}
}
```

- [← previous From Persona to Personalization: A Survey on Role-Playing Language Agents](https://md.research.elamur.ai/free/en/papers/chen-2024-persona-personalization-survey-role-playing/)
- [next → Break the Sequential Dependency of LLM Inference Using Lookahead Decoding](https://md.research.elamur.ai/free/en/papers/fu-2024-break-sequential-dependency-llm-inference/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/4/)

## References

42 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. E. Anders, C. Neo, J. Hoelscher-Obermaier, and J. N. Howard. Sparse autoencoders find composed features in small toy models. LessWrong, 2024. [find](https://scholar.google.com/scholar?q=E%2E+Anders%2C+C%2E+Neo%2C+J%2E+Hoelscher%2DObermaier%2C+and+J%2E+N%2E+Howard%2E+Sparse+autoencoders+find+composed+features+in+small+toy+models%2E+LessWrong%2C+2024%2E)
2. Y. Belinkov. [Probing classifiers: Promises, shortcomings, and advances](https://md.research.elamur.ai/free/en/papers/belinkov-2022-probing-classifiers-promises-shortcomings-advances/). Computational Linguistics, 48(1):207–219, Mar. 2022. doi: 10.1162/coli_a_00422. URL https://aclanthology.org/2022.cl-1.7 . [↗](https://aclanthology.org/2022.cl-1.7)
3. S. Bills, N. Cammarata, D. Mossing, H. Tillman, L. Gao, G. Goh, I. Sutskever, J. Leike, J. Wu, and W. Saunders. Language models can explain neurons in language models. [https://openaipublic.blob.core.windows.net/neuron-explainer/](https://openaipublic.blob.core.windows.net/neuron-explainer/) paper/index.html , 2023. [↗](https://openaipublic.blob.core.windows.net/neuron-explainer/)
4. T. Bricken, A. Templeton, J. Batson, B. Chen, A. Jermyn, T. Conerly, N. Turner, C. Anil, C. Denison, A. Askell, R. Lasenby, Y. Wu, S. Kravec, N. Schiefer, T. Maxwell, N. Joseph, Z. Hatfield-Dodds, A. Tamkin, K. Nguyen, B. McLean, J. E. Burke, T. Hume, S. Carter, T. Henighan, and C. Olah. Towards mon… [find](https://scholar.google.com/scholar?q=T%2E+Bricken%2C+A%2E+Templeton%2C+J%2E+Batson%2C+B%2E+Chen%2C+A%2E+Jermyn%2C+T%2E+Conerly%2C+N%2E+Turner%2C+C%2E+Anil%2C+C%2E+Denison%2C+A%2E+Askell%2C+R%2E+Lasenby%2C+Y%2E+Wu%2C+S%2E+Kravec%2C+N%2E+Schiefer%2C+T%2E+Maxwell%2C+N%2E+Joseph%2C+Z%2E+Hatfield%2DDodds%2C+A%2E+)
5. T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Ber… — [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
6. S. McCandlish, J. Kaplan, D. Amodei, M. Wattenberg, and C. Olah. Toy models of superposition. Transformer Circuits Thread, 2022. [https://transformer-circuits.pub/2022/toy_model/index.html](https://transformer-circuits.pub/2022/toy_model/index.html). [↗](https://transformer-circuits.pub/2022/toy_model/index.html)
7. B. Bussmann, P. Leask, J. Bloom, C. Tigges, and N. Nanda. Stitching saes of different sizes, July J. Engels, I. Liao, E. J. Michaud, W. Gurnee, and 2024. URL [https://www.alignmentforum](https://www.alignmentforum). M. Tegmark. Not all language model features org/posts/baJyjpktzmcmRfosq/ are linear, 2024. URL https://arxiv.org/… — [arXiv:2405.14860](https://arxiv.org/abs/2405.14860) [↗](https://www.alignmentforum)
8. T. Conerly, A. Templeton, T. Bricken, J. Marcus, L. Gao, S. Biderman, S. Black, L. Golding, and T. Henighan. Update on how we train T. Hoppe, C. Foster, J. Phang, H. He, A. Thite, SAEs. Transformer Circuits Thread, 2024. N. Nabeshima, S. Presser, and C. Leahy. The URL https://transformer-circuits. p… [find](https://scholar.google.com/scholar?q=T%2E+Conerly%2C+A%2E+Templeton%2C+T%2E+Bricken%2C+J%2E+Marcus%2C+L%2E+Gao%2C+S%2E+Biderman%2C+S%2E+Black%2C+L%2E+Golding%2C+and+T%2E+Henighan%2E+Update+on+how+we+train+T%2E+Hoppe%2C+C%2E+Foster%2C+J%2E+Phang%2C+H%2E+He%2C+A%2E+Thite%2C+SAEs%2E+Transformer+Ci)
9. A. Conmy and N. Nanda. Activation steer- L. Gao, T. D. la Tour, H. Tillman, G. Goh, R. Troll, ing with SAEs. Alignment Forum, 2024. A. Radford, I. Sutskever, J. Leike, and J. Wu. URL — [Gemini: A Family of Highly Capable Multimodal Models](https://md.research.elamur.ai/free/en/papers/gemini-team-2023-gemini-family-highly-capable-multimodal/) [https://www.alignmentforum](https://www.alignmentforum). Scaling and evaluating sparse autoencoders, org/posts/C5KAZQib3bzzpeyrg/ 2024. URL https:… [↗](https://www.alignmentforum)
10. Trans-size, 2024b. URL [https://storage.former](https://storage.former) Circuits Thread, 2024. URL https://transformer-circuits. pub/2024/june-update/index.html\# topk-gated-comparison . [↗](https://storage.former)
11. H. Cunningham, A. Ewart, L. Riggs, R. Huben, and L. Sharkey. Sparse autoencoders find highly interpretable features in language models, 2023. [find](https://scholar.google.com/scholar?q=H%2E+Cunningham%2C+A%2E+Ewart%2C+L%2E+Riggs%2C+R%2E+Huben%2C+and+L%2E+Sharkey%2E+Sparse+autoencoders+find+highly+interpretable+features+in+language+models%2C+2023%2E)
12. J. Dunefsky, P. Chlenski, and N. Nanda. Transcoders find interpretable llm feature circuits, 2024. URL https://arxiv.org/abs/ 2406.11944 . [↗](https://arxiv.org/abs/)
13. N. Elhage, T. Hume, C. Olsson, N. Schiefer, T. Henighan, S. Kravec, Z. Hatfield-Dodds, R. Lasenby, D. Drain, C. Chen, R. Grosse, googleapis.com/deepmind-media/ gemma/gemma-2-report.pdf . W. Gurnee, N. Nanda, M. Pauly, K. Harvey, D. Troitskii, and D. Bertsimas. Finding neurons in a haystack: Case stu… — [Finding Neurons in a Haystack: Case Studies with Sparse Probing](https://md.research.elamur.ai/free/en/papers/gurnee-2023-finding-neurons-haystack-case-studies/)
14. W. Gurnee, T. Horsley, Z. C. Guo, T. R. Kheirkhah, Q. Sun, W. Hathaway, N. Nanda, and D. Bertsimas. Universal neurons in gpt2 language models, 2024. URL https://arxiv.org/abs/ 2401.12181 . [↗](https://arxiv.org/abs/)
15. M. Hanna, O. Liu, and A. Variengien. How C. Kissane, R. Krzyzanowski, J. I. Bloom, does gpt-2 compute greater-than?: Interpret-A. Conmy, and N. Nanda. Interpreting ating mathematical abilities in a pre-trained lan-tention layer outputs with sparse autoen-guage model, 2023. [URL https://arxiv](https://md.research.elamur.ai/free/en/papers/hanna-2023-gpt-2-compute-greater-than/). coders,…
16. K. He, X. Zhang, S. Ren, and J. Sun. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification, 2015. URL https://arxiv.org/abs/1502.01852 . [↗](https://arxiv.org/abs/1502.01852)
17. R. Hendel, M. Geva, and A. Globerson. [In-context learning creates task vectors](https://md.research.elamur.ai/free/en/papers/hendel-2023-context-learning-creates-task-vectors/). In The 2023 Conference on Empirical Methods in Natural Language Processing, 2023. URL [https://openreview.net/forum?id=](https://openreview.net/forum?id=) QYvFUlF19n . [↗](https://openreview.net/forum?id=)
18. J. Huang, A. Geiger, K. D’Oosterlinck, Z. Wu, and C. Potts. [Rigorously assessing natural language explanations of neurons](https://md.research.elamur.ai/free/en/papers/huang-2023-rigorously-assessing-natural-language-explanations/), 2023. URL https://arxiv.org/abs/2309.10312 . [↗](https://arxiv.org/abs/2309.10312)
19. E. Hubinger. A transparency and interpretability tech tree. Alignment Forum, 2022. [find](https://scholar.google.com/scholar?q=E%2E+Hubinger%2E+A+transparency+and+interpretability+tech+tree%2E+Alignment+Forum%2C+2022%2E)
20. C. Kissane, R. Krzyzanowski, A. Conmy, and N. Nanda. Saes (usually) transfer between base and chat models, 2024b. URL [https://www.alignmentforum.org/posts/fmwk6qxrpW8d4jvbd/](https://www.alignmentforum.org/posts/fmwk6qxrpW8d4jvbd/) saes-usually-transfer . [↗](https://www.alignmentforum.org/posts/fmwk6qxrpW8d4jvbd/)
21. P. Koehn. Europarl: A parallel corpus for statistical machine translation. In Proceedings of Machine Translation Summit X: Papers, pages 79–86, Phuket, Thailand, Sept. 13-15 2005. URL https://aclanthology.org/ 2005.mtsummit-papers.11 . [↗](https://aclanthology.org/)
22. K. Li, O. Patel, F. Viégas, H. Pfister, and M. Wattenberg. [Inference-time intervention: Eliciting truthful answers from a language model](https://md.research.elamur.ai/free/en/papers/li-2023-inference-time-intervention-eliciting-truthful/). In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL [https://openreview.net/forum?id=aLLuYpn83y](https://openreview.net/forum?id=aLLuYpn83y) . [↗](https://openreview.net/forum?id=aLLuYpn83y)
23. S. Jain, E. S. Lubana, K. Oksuz, T. Joy, P. H. S. Torr, A. Sanyal, and P. K. Dokania. What makes and breaks safety fine-tuning? a mechanistic study, J. Lin and J. Bloom. Analyzing neural networks 2024. [URL https://arxiv.org/abs/2407](https://md.research.elamur.ai/free/en/papers/mikolov-2013-efficient-estimation-word-representations-vector/). with dictionary learning, 2023. URL https://10264 . www.neuronped… [↗](https://arxiv.org/abs/2407)
24. N. Nanda. A longlist of theories of impact for interpretability. Alignment Forum, 2022. [find](https://scholar.google.com/scholar?q=N%2E+Nanda%2E+A+longlist+of+theories+of+impact+for+interpretability%2E+Alignment+Forum%2C+2022%2E)
25. N. Nanda and J. Bloom. Transformerlens. [https://github.com/TransformerLensOrg/](https://github.com/TransformerLensOrg/) TransformerLens , 2022. [↗](https://github.com/TransformerLensOrg/)
26. N. Nanda, A. Lee, and M. Wattenberg. [Emergent linear representations in world models of self-supervised sequence models](https://md.research.elamur.ai/free/en/papers/nanda-2023-emergent-linear-representations-world-models/). In Y. Belinkov, S. Hao, J. Jumelet, N. Kim, A. McCarthy, and H. Mohebbi, editors, Proceedings of the 6th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP,…
27. N. Nanda, S. Rajamanoharan, J. Kramar, and R. Shah. Fact finding: Attempting to reverse-engineer factual recall on the neuron level, Dec 2023b. [find](https://scholar.google.com/scholar?q=N%2E+Nanda%2C+S%2E+Rajamanoharan%2C+J%2E+Kramar%2C+and+R%2E+Shah%2E+Fact+finding%3A+Attempting+to+reverse%2Dengineer+factual+recall+on+the+neuron+level%2C+Dec+2023b%2E)
28. C. Olah. Interpretability. Alignment Forum, 2021. [find](https://scholar.google.com/scholar?q=C%2E+Olah%2E+Interpretability%2E+Alignment+Forum%2C+2021%2E)
29. C. Olah, N. Cammarata, L. Schubert, G. Goh, M. Petrov, and S. Carter. Zoom in: An introduction to circuits. Distill, 2020. doi: 10.23915/distill.00024.001. — [doi:10.23915/distill.00024.001](https://doi.org/10.23915/distill.00024.001) [find](https://scholar.google.com/scholar?q=C%2E+Olah%2C+N%2E+Cammarata%2C+L%2E+Schubert%2C+G%2E+Goh%2C+M%2E+Petrov%2C+and+S%2E+Carter%2E+Zoom+in%3A+An+introduction+to+circuits%2E+Distill%2C+2020%2E+doi%3A+10%2E23915%2Fdistill%2E00024%2E001%2E+%E2%80%94+%5Bdoi%3A10%2E23915%2Fdistill%2E00024%2E001%5D%28https%3A%2F%2Fd)
30. C. Olah, A. Templeton, T. Bricken, and A. Jermyn. Open Problem: Attribution Dictionary Learning. Transformer Circuits Thread, 2024. URL [https://transformer-circuits.pub/2024/](https://transformer-circuits.pub/2024/) april-update/index.html\#attr-dl . [↗](https://transformer-circuits.pub/2024/)
31. C. Olsson, N. Elhage, N. Nanda, N. Joseph, N. Das-Sarma, T. Henighan, B. Mann, A. Askell, Y. Bai, A. Chen, T. Conerly, D. Drain, D. Ganguli, Z. Hatfield-Dodds, D. Hernandez, S. Johnston, A. Jones, J. Kernion, L. Lovitt, K. Ndousse, D. Amodei, T. Brown, J. Clark, J. Kaplan, S. McCandlish, and C. Olah… — [Improving Dictionary Learning with Gated Sparse Autoencoders](https://md.research.elamur.ai/free/en/papers/rajamanoharan-2024-improving-dictionary-learning-gated-sparse/)
32. S. Rajamanoharan, T. Lieberum, N. Sonnerat, A. Conmy, V. Varma, J. Kramár, and N. Nanda. Jumping ahead: Improving reconstruction fidelity with jumprelu sparse autoencoders, 2024b. URL https://arxiv.org/abs/ 2407.14435 . [↗](https://arxiv.org/abs/)
33. D. Saxton, E. Grefenstette, F. Hill, and P. Kohli. [Analysing mathematical reasoning abilities of neural models](https://md.research.elamur.ai/free/en/papers/saxton-2019-analysing-mathematical-reasoning-abilities-neural/), 2019. URL https://arxiv.org/abs/1904.01557 . [↗](https://arxiv.org/abs/1904.01557)
34. M. Shoeybi, M. Patwary, R. Puri, P. LeGresley, J. Casper, and B. Catanzaro. Megatron-lm: Training multi-billion parameter language models using model parallelism, 2020. URL https://arxiv.org/abs/1909.08053 . [↗](https://arxiv.org/abs/1909.08053)
35. L. Smith. Replacing sae encoders with inference-time optimisation, 2024. URL [https://www.alignmentforum.org/](https://www.alignmentforum.org/) posts/C5KAZQib3bzzpeyrg\#Replacing_ SAE_Encoders_with_Inference_Time_ Optimisation . [↗](https://www.alignmentforum.org/)
36. A. Stolfo, Y. Belinkov, and M. Sachan. A mechanistic interpretation of arithmetic reasoning in language models using causal mediation analysis. In The 2023 Conference on Empirical Methods in Natural Language Processing, 2023. URL [https://openreview.net/](https://openreview.net/) forum?id=aB3Hwh4UzP . [↗](https://openreview.net/)
37. A. Templeton, T. Conerly, J. Marcus, J. Lindsey, T. Bricken, B. Chen, A. Pearce, C. Citro, E. Ameisen, A. Jones, H. Cunningham, N. L. Turner, C. McDougall, M. MacDiarmid, C. D. Freeman, T. R. Sumers, E. Rees, J. Batson, A. Jermyn, S. Carter, C. Olah, and T. Henighan. Scaling monosemanticity: Extract… [find](https://scholar.google.com/scholar?q=A%2E+Templeton%2C+T%2E+Conerly%2C+J%2E+Marcus%2C+J%2E+Lindsey%2C+T%2E+Bricken%2C+B%2E+Chen%2C+A%2E+Pearce%2C+C%2E+Citro%2C+E%2E+Ameisen%2C+A%2E+Jones%2C+H%2E+Cunningham%2C+N%2E+L%2E+Turner%2C+C%2E+McDougall%2C+M%2E+MacDiarmid%2C+C%2E+D%2E+Freeman%2C+T%2E+R%2E+Sumers%2C+)
38. S. Rajamanoharan, A. Conmy, L. Smith, T. Lieberum, V. Varma, J. Kramár, R. Shah, E. Todd, M. Li, A. S. Sharma, A. Mueller, B. C. Wallace, and D. Bau. Function vectors in large language models. In The Twelfth In- However, after training, we reparameterize the ternational Conference on Learning Repres… — [Steering Language Models With Activation Engineering](https://md.research.elamur.ai/free/en/papers/turner-2023-steering-language-models-activation-engineering/)
39. Distill, 2021. doi: 10.23915/distill.00024. x̂ raw (f) := 𝐶 · (W dec f + b dec ) . 008. [https://distill.pub/2020/circuits/branch-It](https://distill.pub/2020/circuits/branch-It) is straightforward to show that by defining the specialization. following rescaled and shifted parameters: K. Wang, A. Variengien, A. Conmy, B. Shlegeris, and J. Stein… [↗](https://distill.pub/2020/circuits/branch-It)
40. M. Wattenberg and F. Viégas. Relational composition in neural networks: A gentle survey and call to action. In ICML 2024 Workshop on Mechanistic Interpretability, 2024. URL [https://openreview.net/](https://openreview.net/) forum?id=zzCEiUIPk9 . [↗](https://openreview.net/)
41. B. Zhang and R. Sennrich. Root mean square layer normalization, 2019. URL https://arxiv.org/abs/1910.07467 . [↗](https://arxiv.org/abs/1910.07467)
42. D. Ziegler, S. Nix, L. Chan, T. Bauman, P. Schmidt-Nielsen, T. Lin, A. Scherlis, N. Nabeshima, B. Weinstein-Raun, D. de Haas, B. Shlegeris, and N. Thomas. Adversarial training for high-stakes reliability. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neur… [find](https://scholar.google.com/scholar?q=D%2E+Ziegler%2C+S%2E+Nix%2C+L%2E+Chan%2C+T%2E+Bauman%2C+P%2E+Schmidt%2DNielsen%2C+T%2E+Lin%2C+A%2E+Scherlis%2C+N%2E+Nabeshima%2C+B%2E+Weinstein%2DRaun%2C+D%2E+de+Haas%2C+B%2E+Shlegeris%2C+and+N%2E+Thomas%2E+Adversarial+training+for+high%2Dstakes+reliabil)

*Updated 2 November 2026*