---
title: "The Hydra Effect — paper by McGrath et al. (2023)"
description: "The second replaces the output via resample ablation and corresponds to the total effect. 48 references, cited by 46 papers here."
authors: ["Thomas McGrath", "Matthew Rahtz", "Janos Kramar", "Vladimir Mikulik", "Shane Legg"]
year: 2023
venue: "arXiv:2307.15771"
source: "https://arxiv.org/abs/2307.15771"
doi: "10.48550/arxiv.2307.15771"
arxiv: "2307.15771"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/mcgrath-2023-hydra-effect-emergent-self-repair/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/mcgrath-2023-hydra-effect-emergent-self-repair/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/11/) / The Hydra Effect: Emergent Self-repair in Language Model Computations

# The Hydra Effect: Emergent Self-repair in Language Model Computations

**Authors:** [Thomas McGrath](https://md.research.elamur.ai/free/en/authors/thomas-mcgrath/), [Matthew Rahtz](https://md.research.elamur.ai/free/en/authors/matthew-rahtz/), [Janos Kramar](https://md.research.elamur.ai/free/en/authors/janos-kramar/), [Vladimir Mikulik](https://md.research.elamur.ai/free/en/authors/vladimir-mikulik/) and [Shane Legg](https://md.research.elamur.ai/free/en/authors/shane-legg/) · **Year:** 2023 · **Venue:** arXiv:2307.15771 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2307.15771) · [PDF](https://arxiv.org/pdf/2307.15771) · [DOI 10.48550/arxiv.2307.15771](https://doi.org/10.48550/arxiv.2307.15771) · [arXiv 2307.15771](https://arxiv.org/abs/2307.15771) · [OpenAlex](https://openalex.org/W4385473442)

**Citations:** 3 citations (OpenAlex, 2026-10-14) · [46](https://md.research.elamur.ai/free/en/papers/mcgrath-2023-hydra-effect-emergent-self-repair/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper asks how far ablation can be trusted for working out which parts of a large language model drive its predictions. The authors study a 7-billion-parameter decoder-only model trained with the Chinchilla recipe. They prompt it with subject–relation pairs from the Counterfact dataset, so each answer requires recalling a fact. For every attention and MLP layer they compare two importance scores at the final token position. The first projects the layer's output through the model's own unembedding, in the logit lens style, and corresponds to the direct effect. The second replaces the output via resample ablation and corresponds to the total effect.

The two scores are weakly correlated, and ablation usually hurts less than the unembedding score predicts. Tracing downstream activations explains the gap. When an attention layer is removed, later attention layers increase their contribution, which the authors call the Hydra effect. At the same time, late MLPs that normally push down the top token suppress it less. This self-repair matters because circuit attribution, automated ablation pipelines and direct-logit readings of layers all assume components act independently.

## Key points
- Model and data: a Chinchilla-family 7B model trained without dropout or stochastic depth, evaluated on Counterfact prompts. Ablations use activation patches drawn from 15 alternative prompts.
- In middle layers, where compensation is strongest, the Hydra effect and the weakened MLP suppression together restore roughly 70% of the lost logit of the predicted token. The compensation is approximately linear.
- Late-layer "erasure" MLPs tend to lower the logit of the maximum-likelihood token, acting as a negative-feedback mechanism.
- Layers are fairly loosely coupled: knocking out one layer changes only a few downstream layers.
- Wang et al. (2022) observed similar "backup heads" in GPT-2 Small and attributed them to dropout. Because Chinchilla 7B used no dropout, these results refute that explanation. The authors instead speculate about a training-time "natural dropout" or superposition.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,959 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,789 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 4,145 works

## Cite

Cite as: Thomas McGrath, Matthew Rahtz, Janos Kramar, Vladimir Mikulik and Shane Legg. 2023. The Hydra Effect: Emergent Self-repair in Language Model Computations. arXiv:2307.15771.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{mcgrath2023hydra,
  title = {{The Hydra Effect: Emergent Self-repair in Language Model Computations}},
  author = {Thomas McGrath, Matthew Rahtz, Janos Kramar, Vladimir Mikulik and Shane Legg},
  year = {2023},
  doi = {10.48550/arxiv.2307.15771},
  eprint = {2307.15771},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2307.15771}
}
```

- [← previous MathPile: A Billion-Token-Scale Pretraining Corpus for Math](https://md.research.elamur.ai/free/en/papers/wang-2023-mathpile-billion-token-scale-pretraining/)
- [next → GrammarGPT: Exploring Open-Source LLMs for Native Chinese Grammatical Error Correction with Supervised Fine-Tuning](https://md.research.elamur.ai/free/en/papers/fan-2023-grammargpt-exploring-open-source-llms/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/11/)

## References

48 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. G. Alain and Y. Bengio. Understanding intermediate layers using linear classifier probes. arXiv preprint arXiv:1610.01644, 2016. [find](https://scholar.google.com/scholar?q=G%2E+Alain+and+Y%2E+Bengio%2E+Understanding+intermediate+layers+using+linear+classifier+probes%2E+arXiv+preprint+arXiv%3A1610%2E01644%2C+2016%2E)
2. J. L. Ba, J. R. Kiros, and G. E. Hinton. [Layer normalization](https://md.research.elamur.ai/free/en/papers/ba-2016-layer-normalization/). arXiv preprint arXiv:1607.06450, 2016.
3. D. Bahdanau, K. Cho, and Y. Bengio. [Neural machine translation by jointly learning to align and translate](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/). arXiv preprint arXiv:1409.0473, 2014.
4. D. Bau, J.-Y. Zhu, H. Strobelt, B. Zhou, J. B. Tenenbaum, W. T. Freeman, and A. Torralba. Gan dissection: Visualizing and understanding generative adversarial networks. arXiv preprint arXiv:1811.10597, 2018. — [arXiv:1811.10597](https://arxiv.org/abs/1811.10597) [find](https://scholar.google.com/scholar?q=D%2E+Bau%2C+J%2E%2DY%2E+Zhu%2C+H%2E+Strobelt%2C+B%2E+Zhou%2C+J%2E+B%2E+Tenenbaum%2C+W%2E+T%2E+Freeman%2C+and+A%2E+Torralba%2E+Gan+dissection%3A+Visualizing+and+understanding+generative+adversarial+networks%2E+arXiv+preprint+arXiv%3A1811%2E10597)
5. N. Belrose, Z. Furman, L. Smith, D. Halawi, I. Ostrovsky, L. McKinney, S. Biderman, and J. Steinhardt. [Eliciting latent predictions from transformers with the tuned lens](https://md.research.elamur.ai/free/en/papers/belrose-2023-eliciting-latent-predictions-transformers-tuned/). arXiv preprint arXiv:2303.08112, 2023.
6. N. Cammarata, G. Goh, S. Carter, L. Schubert, M. Petrov, and C. Olah. Curve detectors. Distill, 5(6): e00024–003, 2020. [find](https://scholar.google.com/scholar?q=N%2E+Cammarata%2C+G%2E+Goh%2C+S%2E+Carter%2C+L%2E+Schubert%2C+M%2E+Petrov%2C+and+C%2E+Olah%2E+Curve+detectors%2E+Distill%2C+5%286%29%3A+e00024%E2%80%93003%2C+2020%2E)
7. S. Carter, Z. Armstrong, L. Schubert, I. Johnson, and C. Olah. Activation atlas. Distill, 2019. doi: 10.23915/distill.00015. [https://distill.pub/2019/activation-atlas](https://distill.pub/2019/activation-atlas). — [doi:10.23915/distill.00015](https://doi.org/10.23915/distill.00015) [↗](https://distill.pub/2019/activation-atlas)
8. L. Chan, A. Garriga-Alonso, N. Goldwosky-Dill, R. Greenblatt, J. Nitishinskaya, A. Radhakrishnan, B. Shlegeris, and N. Thomas. Causal scrubbing, a method for rigorously testing interpretability hypotheses. AI Alignment Forum, 2022. [https://www.alignmentforum.org/posts/](https://www.alignmentforum.org/posts/) JvZhhzycHu2Yd57RN/causal-scrub… [↗](https://www.alignmentforum.org/posts/)
9. B. Chughtai, L. Chan, and N. Nanda. A toy model of universality: Reverse engineering how networks learn group operations. arXiv preprint arXiv:2302.03025, 2023. — [arXiv:2302.03025](https://arxiv.org/abs/2302.03025) [find](https://scholar.google.com/scholar?q=B%2E+Chughtai%2C+L%2E+Chan%2C+and+N%2E+Nanda%2E+A+toy+model+of+universality%3A+Reverse+engineering+how+networks+learn+group+operations%2E+arXiv+preprint+arXiv%3A2302%2E03025%2C+2023%2E+%E2%80%94+%5BarXiv%3A2302%2E03025%5D%28https%3A%2F%2Farxiv%2Eorg%2F)
10. A. Conmy, A. N. Mavor-Parker, A. Lynch, S. Heimersheim, and A. Garriga-Alonso. Towards automated circuit discovery for mechanistic interpretability. arXiv preprint arXiv:2304.14997, 2023. [find](https://scholar.google.com/scholar?q=A%2E+Conmy%2C+A%2E+N%2E+Mavor%2DParker%2C+A%2E+Lynch%2C+S%2E+Heimersheim%2C+and+A%2E+Garriga%2DAlonso%2E+Towards+automated+circuit+discovery+for+mechanistic+interpretability%2E+arXiv+preprint+arXiv%3A2304%2E14997%2C+2023%2E)
11. G. Dar, M. Geva, A. Gupta, and J. Berant. [Analyzing transformers in embedding space](https://md.research.elamur.ai/free/en/papers/dar-2023-analyzing-transformers-embedding-space/). arXiv preprint arXiv:2209.02535, 2022.
12. T. Dettmers, M. Lewis, Y. Belkada, and L. Zettlemoyer. Llm. int8 (): 8-bit matrix multiplication for transformers at scale. arXiv preprint arXiv:2208.07339, 2022. [find](https://scholar.google.com/scholar?q=T%2E+Dettmers%2C+M%2E+Lewis%2C+Y%2E+Belkada%2C+and+L%2E+Zettlemoyer%2E+Llm%2E+int8+%28%29%3A+8%2Dbit+matrix+multiplication+for+transformers+at+scale%2E+arXiv+preprint+arXiv%3A2208%2E07339%2C+2022%2E)
13. N. Elhage, N. Nanda, C. Olsson, T. Henighan, N. Joseph, B. Mann, A. Askell, Y. Bai, A. Chen, T. Conerly, N. DasSarma, D. Drain, D. Ganguli, Z. Hatfield-Dodds, D. Hernandez, A. Jones, J. Kernion, L. Lovitt, K. Ndousse, D. Amodei, T. Brown, J. Clark, J. Kaplan, S. McCandlish, and C. Olah. A mathematic… [find](https://scholar.google.com/scholar?q=N%2E+Elhage%2C+N%2E+Nanda%2C+C%2E+Olsson%2C+T%2E+Henighan%2C+N%2E+Joseph%2C+B%2E+Mann%2C+A%2E+Askell%2C+Y%2E+Bai%2C+A%2E+Chen%2C+T%2E+Conerly%2C+N%2E+DasSarma%2C+D%2E+Drain%2C+D%2E+Ganguli%2C+Z%2E+Hatfield%2DDodds%2C+D%2E+Hernandez%2C+A%2E+Jones%2C+J%2E+Kernion%2C+L%2E+Lo)
14. N. Elhage, T. Hume, C. Olsson, N. Schiefer, T. Henighan, S. Kravec, Z. Hatfield-Dodds, R. Lasenby, D. Drain, C. Chen, R. Grosse, S. McCandlish, J. Kaplan, D. Amodei, M. Wattenberg, and C. Olah. Toy models of superposition. Transformer Circuits Thread, 2022. [https://transformer-circuits.pub/2022/toy_…](https://transformer-circuits.pub/2022/toy_…) [↗](https://transformer-circuits.pub/2022/toy_…)
15. M. Finlayson, A. Mueller, S. Gehrmann, S. Shieber, T. Linzen, and Y. Belinkov. Causal analysis of syntactic agreement mechanisms in neural language models. arXiv preprint arXiv:2106.06087, 2021. [find](https://scholar.google.com/scholar?q=M%2E+Finlayson%2C+A%2E+Mueller%2C+S%2E+Gehrmann%2C+S%2E+Shieber%2C+T%2E+Linzen%2C+and+Y%2E+Belinkov%2E+Causal+analysis+of+syntactic+agreement+mechanisms+in+neural+language+models%2E+arXiv+preprint+arXiv%3A2106%2E06087%2C+2021%2E)
16. J. Z. Forde, C. Lovering, G. Konidaris, E. Pavlick, and M. L. Littman. Where, when & which concepts does alphazero learn? lessons from the game of hex. In AAAI Workshop on Reinforcement Learning in Games, volume 2, 2022. [find](https://scholar.google.com/scholar?q=J%2E+Z%2E+Forde%2C+C%2E+Lovering%2C+G%2E+Konidaris%2C+E%2E+Pavlick%2C+and+M%2E+L%2E+Littman%2E+Where%2C+when+%26+which+concepts+does+alphazero+learn%3F+lessons+from+the+game+of+hex%2E+In+AAAI+Workshop+on+Reinforcement+Learning+in+Ga)
17. A. Geiger, H. Lu, T. Icard, and C. Potts. Causal abstractions of neural networks. Advances in Neural Information Processing Systems, 34:9574–9586, 2021. [find](https://scholar.google.com/scholar?q=A%2E+Geiger%2C+H%2E+Lu%2C+T%2E+Icard%2C+and+C%2E+Potts%2E+Causal+abstractions+of+neural+networks%2E+Advances+in+Neural+Information+Processing+Systems%2C+34%3A9574%E2%80%939586%2C+2021%2E)
18. A. Geiger, Z. Wu, H. Lu, J. Rozner, E. Kreiss, T. Icard, N. Goodman, and C. Potts. Inducing causal structure for interpretable neural networks. In International Conference on Machine Learning, pages 7324–7338. PMLR, 2022. [find](https://scholar.google.com/scholar?q=A%2E+Geiger%2C+Z%2E+Wu%2C+H%2E+Lu%2C+J%2E+Rozner%2C+E%2E+Kreiss%2C+T%2E+Icard%2C+N%2E+Goodman%2C+and+C%2E+Potts%2E+Inducing+causal+structure+for+interpretable+neural+networks%2E+In+International+Conference+on+Machine+Learning%2C+pages+7)
19. A. Geiger, C. Potts, and T. Icard. Causal abstraction for faithful model interpretation. arXiv preprint arXiv:2301.04709, 2023a. [find](https://scholar.google.com/scholar?q=A%2E+Geiger%2C+C%2E+Potts%2C+and+T%2E+Icard%2E+Causal+abstraction+for+faithful+model+interpretation%2E+arXiv+preprint+arXiv%3A2301%2E04709%2C+2023a%2E)
20. A. Geiger, Z. Wu, C. Potts, T. Icard, and N. D. Goodman. [Finding alignments between interpretable causal variables and distributed neural representations](https://md.research.elamur.ai/free/en/papers/geiger-2023-finding-alignments-between-interpretable-causal/). arXiv preprint arXiv:2303.02536, 2023b.
21. M. Geva, A. Caciularu, G. Dar, P. Roit, S. Sadde, M. Shlain, B. Tamir, and Y. Goldberg. Lm-debugger: An interactive tool for inspection and intervention in transformer-based language models. arXiv preprint arXiv:2204.12130, 2022a. [find](https://scholar.google.com/scholar?q=M%2E+Geva%2C+A%2E+Caciularu%2C+G%2E+Dar%2C+P%2E+Roit%2C+S%2E+Sadde%2C+M%2E+Shlain%2C+B%2E+Tamir%2C+and+Y%2E+Goldberg%2E+Lm%2Ddebugger%3A+An+interactive+tool+for+inspection+and+intervention+in+transformer%2Dbased+language+models%2E+arXiv+pre)
22. M. Geva, A. Caciularu, K. R. Wang, and Y. Goldberg. [Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space](https://md.research.elamur.ai/free/en/papers/geva-2022-transformer-feed-forward-layers-build/). arXiv preprint arXiv:2203.14680, 2022b.
23. M. Glymour, J. Pearl, and N. P. Jewell. Causal inference in statistics: A primer. John Wiley & Sons, 2016. [find](https://scholar.google.com/scholar?q=M%2E+Glymour%2C+J%2E+Pearl%2C+and+N%2E+P%2E+Jewell%2E+Causal+inference+in+statistics%3A+A+primer%2E+John+Wiley+%26+Sons%2C+2016%2E)
24. G. Goh, N. C. †, C. V. †, S. Carter, M. Petrov, L. Schubert, A. Radford, and C. Olah. Multimodal neurons in artificial neural networks. Distill, 2021. doi: 10.23915/distill.00030. [https://distill.pub/2021/multimodal-neurons](https://distill.pub/2021/multimodal-neurons). [↗](https://distill.pub/2021/multimodal-neurons)
25. W. Gurnee, N. Nanda, M. Pauly, K. Harvey, D. Troitskii, and D. Bertsimas. [Finding neurons in a haystack: Case studies with sparse probing](https://md.research.elamur.ai/free/en/papers/gurnee-2023-finding-neurons-haystack-case-studies/). arXiv preprint arXiv:2305.01610, 2023.
26. J. Y. Halpern. Actual causality. MiT Press, 2016. [find](https://scholar.google.com/scholar?q=J%2E+Y%2E+Halpern%2E+Actual+causality%2E+MiT+Press%2C+2016%2E)
27. J. Hilton, N. Cammarata, S. Carter, G. Goh, and C. Olah. Understanding rl vision. Distill, 2020. doi: 10.23915/distill.00029. [https://distill.pub/2020/understanding-rl-vision](https://distill.pub/2020/understanding-rl-vision). — [doi:10.23915/distill.00029](https://doi.org/10.23915/distill.00029) [↗](https://distill.pub/2020/understanding-rl-vision)
28. J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. d. L. Casas, L. A. Hendricks, J. Welbl, A. Clark, et al. [Training compute-optimal large language models](https://md.research.elamur.ai/free/en/papers/hoffmann-2022-training-compute-optimal-large-language/). arXiv preprint arXiv:2203.15556, 2022.
29. M. L. Leavitt and A. Morcos. arXiv:2010.12016, 2020. — [arXiv:2010.12016](https://arxiv.org/abs/2010.12016) [find](https://scholar.google.com/scholar?q=M%2E+L%2E+Leavitt+and+A%2E+Morcos%2E+arXiv%3A2010%2E12016%2C+2020%2E+%E2%80%94+%5BarXiv%3A2010%2E12016%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2010%2E12016%29)
30. Towards falsifiable interpretability research. arXiv preprint R. Massidda, A. Geiger, T. Icard, and D. Bacciu. Causal abstraction with soft interventions. arXiv preprint arXiv:2211.12270, 2022. — [arXiv:2211.12270](https://arxiv.org/abs/2211.12270) [find](https://scholar.google.com/scholar?q=Towards+falsifiable+interpretability+research%2E+arXiv+preprint+R%2E+Massidda%2C+A%2E+Geiger%2C+T%2E+Icard%2C+and+D%2E+Bacciu%2E+Causal+abstraction+with+soft+interventions%2E+arXiv+preprint+arXiv%3A2211%2E12270%2C+2022%2E+%E2%80%94+%5BarX)
31. T. McGrath, A. Kapishnikov, N. Tomašev, A. Pearce, M. Wattenberg, D. Hassabis, B. Kim, U. Paquet, and V. Kramnik. Acquisition of chess knowledge in alphazero. Proceedings of the National Academy of Sciences, 119(47):e2206625119, 2022. [find](https://scholar.google.com/scholar?q=T%2E+McGrath%2C+A%2E+Kapishnikov%2C+N%2E+Toma%C5%A1ev%2C+A%2E+Pearce%2C+M%2E+Wattenberg%2C+D%2E+Hassabis%2C+B%2E+Kim%2C+U%2E+Paquet%2C+and+V%2E+Kramnik%2E+Acquisition+of+chess+knowledge+in+alphazero%2E+Proceedings+of+the+National+Academy+of+Sc)
32. K. Meng, D. Bau, A. Andonian, and Y. Belinkov. Locating and editing factual associations in gpt. Advances in Neural Information Processing Systems, 35:17359–17372, 2022. [find](https://scholar.google.com/scholar?q=K%2E+Meng%2C+D%2E+Bau%2C+A%2E+Andonian%2C+and+Y%2E+Belinkov%2E+Locating+and+editing+factual+associations+in+gpt%2E+Advances+in+Neural+Information+Processing+Systems%2C+35%3A17359%E2%80%9317372%2C+2022%2E)
33. A. S. Morcos, D. G. Barrett, N. C. Rabinowitz, and M. Botvinick. On the importance of single directions for generalization. arXiv preprint arXiv:1803.06959, 2018. — [arXiv:1803.06959](https://arxiv.org/abs/1803.06959) [find](https://scholar.google.com/scholar?q=A%2E+S%2E+Morcos%2C+D%2E+G%2E+Barrett%2C+N%2E+C%2E+Rabinowitz%2C+and+M%2E+Botvinick%2E+On+the+importance+of+single+directions+for+generalization%2E+arXiv+preprint+arXiv%3A1803%2E06959%2C+2018%2E+%E2%80%94+%5BarXiv%3A1803%2E06959%5D%28https%3A%2F%2Farxiv%2Eor)
34. N. Nanda, L. Chan, T. Liberum, J. Smith, and J. Steinhardt. [Progress measures for grokking via mechanistic interpretability](https://md.research.elamur.ai/free/en/papers/nanda-2023-progress-measures-grokking-mechanistic-interpretability/). arXiv preprint arXiv:2301.05217, 2023. nostalgebraist. interpreting gpt: the logit lens, 2020. URL [https://www.lesswrong.com/posts/](https://www.lesswrong.com/posts/) AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logi… [↗](https://www.lesswrong.com/posts/)
35. C. Olah, A. Satyanarayan, I. Johnson, S. Carter, L. Schubert, K. Ye, and A. Mordvintsev. The building blocks of interpretability. Distill, 3(3):e10, 2018. [find](https://scholar.google.com/scholar?q=C%2E+Olah%2C+A%2E+Satyanarayan%2C+I%2E+Johnson%2C+S%2E+Carter%2C+L%2E+Schubert%2C+K%2E+Ye%2C+and+A%2E+Mordvintsev%2E+The+building+blocks+of+interpretability%2E+Distill%2C+3%283%29%3Ae10%2C+2018%2E)
36. C. Olah, N. Cammarata, L. Schubert, G. Goh, M. Petrov, and S. Carter. Zoom in: An introduction to circuits. Distill, 2020. doi: 10.23915/distill.00024.001. [https://distill.pub/2020/circuits/zoom-in](https://distill.pub/2020/circuits/zoom-in). — [doi:10.23915/distill.00024.001](https://doi.org/10.23915/distill.00024.001) [↗](https://distill.pub/2020/circuits/zoom-in)
37. C. Olsson, N. Elhage, N. Nanda, N. Joseph, N. DasSarma, T. Henighan, B. Mann, A. Askell, Y. Bai, A. Chen, et al. [In-context learning and induction heads](https://md.research.elamur.ai/free/en/papers/olsson-2022-context-learning-induction-heads/). arXiv preprint arXiv:2209.11895, 2022.
38. J. Pearl. Causality. Cambridge university press, 2009. [find](https://scholar.google.com/scholar?q=J%2E+Pearl%2E+Causality%2E+Cambridge+university+press%2C+2009%2E)
39. M. Phuong and M. Hutter. Formal algorithms for transformers. arXiv preprint arXiv:2207.09238, 2022. — [arXiv:2207.09238](https://arxiv.org/abs/2207.09238) [find](https://scholar.google.com/scholar?q=M%2E+Phuong+and+M%2E+Hutter%2E+Formal+algorithms+for+transformers%2E+arXiv+preprint+arXiv%3A2207%2E09238%2C+2022%2E+%E2%80%94+%5BarXiv%3A2207%2E09238%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2207%2E09238%29)
40. A. Power, Y. Burda, H. Edwards, I. Babuschkin, and V. Misra. [Grokking: Generalization beyond overfitting on small algorithmic datasets](https://md.research.elamur.ai/free/en/papers/power-2022-grokking-generalization-overfitting-small-algorithmic/). arXiv preprint arXiv:2201.02177, 2022.
41. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. [Dropout: a simple way to prevent neural networks from overfitting](https://md.research.elamur.ai/free/en/papers/srivastava-2014-dropout-simple-way-prevent/). The journal of machine learning research, 15(1): 1929–1958, 2014.
42. N. Tinbergen. On aims and methods of ethology. Zeitschrift für tierpsychologie, 20(4):410–433, 1963. [find](https://scholar.google.com/scholar?q=N%2E+Tinbergen%2E+On+aims+and+methods+of+ethology%2E+Zeitschrift+f%C3%BCr+tierpsychologie%2C+20%284%29%3A410%E2%80%93433%2C+1963%2E)
43. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). Advances in neural information processing systems, 30, 2017.
44. A. Veit, M. J. Wilber, and S. Belongie. Residual networks behave like ensembles of relatively shallow networks. Advances in neural information processing systems, 29, 2016. [find](https://scholar.google.com/scholar?q=A%2E+Veit%2C+M%2E+J%2E+Wilber%2C+and+S%2E+Belongie%2E+Residual+networks+behave+like+ensembles+of+relatively+shallow+networks%2E+Advances+in+neural+information+processing+systems%2C+29%2C+2016%2E)
45. J. Vig, S. Gehrmann, Y. Belinkov, S. Qian, D. Nevo, S. Sakenis, J. Huang, Y. Singer, and S. Shieber. Causal mediation analysis for interpreting neural nlp: The case of gender bias. arXiv preprint arXiv:2004.12265, 2020a. — [arXiv:2004.12265](https://arxiv.org/abs/2004.12265) [find](https://scholar.google.com/scholar?q=J%2E+Vig%2C+S%2E+Gehrmann%2C+Y%2E+Belinkov%2C+S%2E+Qian%2C+D%2E+Nevo%2C+S%2E+Sakenis%2C+J%2E+Huang%2C+Y%2E+Singer%2C+and+S%2E+Shieber%2E+Causal+mediation+analysis+for+interpreting+neural+nlp%3A+The+case+of+gender+bias%2E+arXiv+preprint+arXi)
46. J. Vig, S. Gehrmann, Y. Belinkov, S. Qian, D. Nevo, Y. Singer, and S. Shieber. Investigating gender bias in language models using causal mediation analysis. Advances in neural information processing systems, 33:12388–12401, 2020b. [find](https://scholar.google.com/scholar?q=J%2E+Vig%2C+S%2E+Gehrmann%2C+Y%2E+Belinkov%2C+S%2E+Qian%2C+D%2E+Nevo%2C+Y%2E+Singer%2C+and+S%2E+Shieber%2E+Investigating+gender+bias+in+language+models+using+causal+mediation+analysis%2E+Advances+in+neural+information+processing+s)
47. K. Wang, A. Variengien, A. Conmy, B. Shlegeris, and J. Steinhardt. Interpretability in the wild: a circuit for indirect object identification in gpt-2 small. arXiv preprint arXiv:2211.00593, 2022. [find](https://scholar.google.com/scholar?q=K%2E+Wang%2C+A%2E+Variengien%2C+A%2E+Conmy%2C+B%2E+Shlegeris%2C+and+J%2E+Steinhardt%2E+Interpretability+in+the+wild%3A+a+circuit+for+indirect+object+identification+in+gpt%2D2+small%2E+arXiv+preprint+arXiv%3A2211%2E00593%2C+2022%2E)
48. B. Zhang and R. Sennrich. Root mean square layer normalization. Advances in Neural Information Processing Systems, 32, 2019. [find](https://scholar.google.com/scholar?q=B%2E+Zhang+and+R%2E+Sennrich%2E+Root+mean+square+layer+normalization%2E+Advances+in+Neural+Information+Processing+Systems%2C+32%2C+2019%2E)

*Updated 3 November 2026*