---
title: "Information Flow Routes — paper by Ferrando and Voita (2024)"
description: "Mechanistic interpretability usually finds circuits through activation patching. 36 references, cited by 37 papers here."
authors: ["Javier Ferrando", "Elena Voita"]
year: 2024
venue: "Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/2024.emnlp-main.965/"
doi: "10.18653/v1/2024.emnlp-main.965"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ferrando-2024-information-flow-routes-automatically-interpreting/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ferrando-2024-information-flow-routes-automatically-interpreting/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/8/) / Information Flow Routes: Automatically Interpreting Language Models at Scale

# Information Flow Routes: Automatically Interpreting Language Models at Scale

**Authors:** [Javier Ferrando](https://md.research.elamur.ai/free/en/authors/javier-ferrando/) and [Elena Voita](https://md.research.elamur.ai/free/en/authors/elena-voita/) · **Year:** 2024 · **Venue:** [Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2024](https://md.research.elamur.ai/free/en/venues/emnlp/2024/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2024.emnlp-main.965/) · [PDF](https://aclanthology.org/2024.emnlp-main.965.pdf) · [DOI 10.18653/v1/2024.emnlp-main.965](https://doi.org/10.18653/v1/2024.emnlp-main.965) · [OpenAlex](https://openalex.org/W4404782809)

**Citations:** 6 citations (OpenAlex, 2026-10-14) · [37](https://md.research.elamur.ai/free/en/papers/ferrando-2024-information-flow-routes-automatically-interpreting/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Mechanistic interpretability usually finds circuits through activation patching. That approach depends on hand-built prompt templates with contrastive counterparts and needs many forward passes, so it does not scale to large models or arbitrary inputs. Ferrando and Voita represent a Transformer's computation as a graph: token representations are nodes, and attention heads, FFN blocks and residual connections are edges. For each prediction they trace back from the top of the residual stream and keep only the edges whose importance exceeds a threshold τ. Edge importance comes from ALTI-style attribution, which compares each incoming vector with the sum it contributes to. This requires a single forward pass and is about 100 times faster than patching. On GPT2-Small, the extracted routes recover the known circuits for IOI and Greater-than. The method also works without templates, so the authors could study Llama 2 across many predictions and domains. Neither the general-importance analysis nor the domain-level analysis is feasible with patching. The approach therefore makes automatic, large-scale interpretation of language models practical.

## Key points
- Attention outputs are split into per-head, per-token sub-edges, with layer normalization linearized. Their importances are summed across heads to give a weight for each pair of connected nodes.
- The routes reproduce the IOI and Greater-than circuits in GPT2-Small. The method can also run in a contrastive, patching-like mode, and the authors argue that patching results are fragile because they depend on which contrastive template is chosen.
- In Llama 2, previous token heads and subword merging heads matter across predictions in general. Tokens with the same part of speech show similar flow patterns.
- Domain analysis of Llama 2-7B used 1000 sentences per domain:
  - C4
  - FLORES-200 devtest in English, Russian, Italian and Spanish
  - CodeParrot code
  - addition/subtraction data
  
  Heads that matter little in general became important for specific domains, and the important heads differed between domains. The last-layer FFN mattered less for non-English text.
- Projecting the SVD of each head's W_OV matrix onto the unembedding matrix showed interpretable behaviour. Code-specific heads promote technology tokens such as Xcode and iOS, while heads active on non-English text promote multilingual tokens rather than English ones. The experiments cover only the GPT-2, OPT and Llama 2 families.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Meta](https://md.research.elamur.ai/free/en/companies/meta/) — Company, Menlo Park, United States · 214 works · 241,492 citations
- [Universitat Politècnica de Catalunya](https://md.research.elamur.ai/free/en/universities/universitat-politecnica-de-catalunya/) — University, Barcelona, Spain · 31 works · 15,636 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works

## Cite

Cite as: Javier Ferrando and Elena Voita. 2024. Information Flow Routes: Automatically Interpreting Language Models at Scale. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{ferrando2024information,
  title = {{Information Flow Routes: Automatically Interpreting Language Models at Scale}},
  author = {Javier Ferrando and Elena Voita},
  booktitle = {Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  doi = {10.18653/v1/2024.emnlp-main.965},
  url = {https://doi.org/10.18653/v1/2024.emnlp-main.965}
}
```

- [← previous Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning](https://md.research.elamur.ai/free/en/papers/bi-2024-forest-thought-scaling-test-time/)
- [next → ValueBench: Towards Comprehensively Evaluating Value Orientations and Understanding of Large Language Models](https://md.research.elamur.ai/free/en/papers/ren-2024-valuebench-comprehensively-evaluating-value-orientations/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/8/)

## References

36 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jasmijn Bastings and Katja Filippova. 2020. [The elephant in the interpretability room: Why use attention as explanation when we have saliency methods?](https://md.research.elamur.ai/free/en/papers/bastings-2020-elephant-interpretability-room-why-use/) In Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pages 149–155, Online. Association for Comput…
2. Anthony Bau, Yonatan Belinkov, Hassan Sajjad, Nadir Durrani, Fahim Dalvi, and James Glass. 2019. [Identifying and controlling important neurons in neural machine translation](https://md.research.elamur.ai/free/en/papers/bau-2018-identifying-controlling-important-neurons-neural/). In International Conference on Learning Representations.
3. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chri… — [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
4. Arthur Conmy, Augustine Mavor-Parker, Aengus Lynch, Stefan Heimersheim, and Adrià Garriga-Alonso. 2023. Towards automated circuit discovery for mechanistic interpretability. In Advances in Neural Information Processing Systems, volume 36, pages 16318– 16352. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Arthur+Conmy%2C+Augustine+Mavor%2DParker%2C+Aengus+Lynch%2C+Stefan+Heimersheim%2C+and+Adri%C3%A0+Garriga%2DAlonso%2E+2023%2E+Towards+automated+circuit+discovery+for+mechanistic+interpretability%2E+In+Advances+in+Neural+Info)
5. Gonçalo M. Correia, Vlad Niculae, and André F. T. Martins. 2019. [Adaptively sparse transformers](https://md.research.elamur.ai/free/en/papers/correia-2019-adaptively-sparse-transformers/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2174– 2184, Hong…
6. David Dale, Elena Voita, Loic Barrault, and Marta R. Costa-jussà. 2023a. [Detecting and mitigating hallucinations in machine translation: Model internal workings alone do well, sentence similarity Even better](https://md.research.elamur.ai/free/en/papers/dale-2023-detecting-mitigating-hallucinations-machine-translation/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics…
7. David Dale, Elena Voita, Janice Lam, Prangthip Hansanti, Christophe Ropers, Elahe Kalbassi, Cynthia Gao, Loic Barrault, and Marta R. Costa-jussà. 2023b. Halomi: A manually annotated benchmark for multilingual hallucination and omission detection in machine translation. In Proceedings of the 2023 Con… [find](https://scholar.google.com/scholar?q=David+Dale%2C+Elena+Voita%2C+Janice+Lam%2C+Prangthip+Hansanti%2C+Christophe+Ropers%2C+Elahe+Kalbassi%2C+Cynthia+Gao%2C+Loic+Barrault%2C+and+Marta+R%2E+Costa%2Djuss%C3%A0%2E+2023b%2E+Halomi%3A+A+manually+annotated+benchmark+for+mult)
8. Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Ja… [find](https://scholar.google.com/scholar?q=Nelson+Elhage%2C+Neel+Nanda%2C+Catherine+Olsson%2C+Tom+Henighan%2C+Nicholas+Joseph%2C+Ben+Mann%2C+Amanda+Askell%2C+Yuntao+Bai%2C+Anna+Chen%2C+Tom+Conerly%2C+Nova+DasSarma%2C+Dawn+Drain%2C+Deep+Ganguli%2C+Zac+Hatfield%2DDodds%2C+Da)
9. Javier Ferrando, Gerard I. Gállego, and Marta R. Costa-jussà. 2022. Measuring the mixing of contextual information in the transformer. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 8698–8714, Abu Dhabi, United Arab Emirates. Association for Computat… [find](https://scholar.google.com/scholar?q=Javier+Ferrando%2C+Gerard+I%2E+G%C3%A1llego%2C+and+Marta+R%2E+Costa%2Djuss%C3%A0%2E+2022%2E+Measuring+the+mixing+of+contextual+information+in+the+transformer%2E+In+Proceedings+of+the+2022+Conference+on+Empirical+Methods+in+Nat)
10. Atticus Geiger, Kyle Richardson, and Christopher Potts. 2020. [Neural natural language inference models partially embed theories of lexical entailment and negation](https://md.research.elamur.ai/free/en/papers/geiger-2020-neural-natural-language-inference-models/). In Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pages 163–173, Online. Associati…
11. Naman Goyal, Cynthia Gao, Vishrav Chaudhary, Peng-Jen Chen, Guillaume Wenzek, Da Ju, Sanjana Krishnan, Marc’Aurelio Ranzato, Francisco Guzmán, and Angela Fan. 2022. The Flores-101 evaluation benchmark for low-resource and multilingual machine translation. Transactions of the Association for Computat… [find](https://scholar.google.com/scholar?q=Naman+Goyal%2C+Cynthia+Gao%2C+Vishrav+Chaudhary%2C+Peng%2DJen+Chen%2C+Guillaume+Wenzek%2C+Da+Ju%2C+Sanjana+Krishnan%2C+Marc%E2%80%99Aurelio+Ranzato%2C+Francisco+Guzm%C3%A1n%2C+and+Angela+Fan%2E+2022%2E+The+Flores%2D101+evaluation+benchmark)
12. Nuno M. Guerreiro, Duarte Alves, Jonas Waldendorf, Barry Haddow, Alexandra Birch, Pierre Colombo, and André F. T. Martins. 2023. [Hallucinations in large multilingual translation models](https://md.research.elamur.ai/free/en/papers/guerreiro-2023-hallucinations-large-multilingual-translation-models/). Preprint, arXiv:2303.16104.
13. Michael Hanna, Ollie Liu, and Alexandre Variengien. 2023. [How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model](https://md.research.elamur.ai/free/en/papers/hanna-2023-gpt-2-compute-greater-than/). In Advances in Neural Information Processing Systems, volume 36, pages 76033–76060. Curran Associates, Inc.
14. Michael Hanna, Sandro Pezzelle, and Yonatan Belinkov. 2024. Have faith in faithfulness: Going beyond circuit overlap when finding model mechanisms. Preprint, arXiv:2403.17806. [find](https://scholar.google.com/scholar?q=Michael+Hanna%2C+Sandro+Pezzelle%2C+and+Yonatan+Belinkov%2E+2024%2E+Have+faith+in+faithfulness%3A+Going+beyond+circuit+overlap+when+finding+model+mechanisms%2E+Preprint%2C+arXiv%3A2403%2E17806%2E)
15. Roee Hendel, Mor Geva, and Amir Globerson. 2023. [In-context learning creates task vectors](https://md.research.elamur.ai/free/en/papers/hendel-2023-context-learning-creates-task-vectors/). Preprint, arXiv:2310.15916.
16. Goro Kobayashi, Tatsuki Kuribayashi, Sho Yokoi, and Kentaro Inui. 2020. [Attention is not only a weight: Analyzing transformers with vector norms](https://md.research.elamur.ai/free/en/papers/kobayashi-2020-attention-is-not-only-weight/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7057–7075, Online. Association for Computational…
17. János Kramár, Tom Lieberum, Rohin Shah, and Neel Nanda. 2024. Atp*: An efficient and scalable method for localizing llm behaviour to components. Preprint, arXiv:2403.00745. [find](https://scholar.google.com/scholar?q=J%C3%A1nos+Kram%C3%A1r%2C+Tom+Lieberum%2C+Rohin+Shah%2C+and+Neel+Nanda%2E+2024%2E+Atp%2A%3A+An+efficient+and+scalable+method+for+localizing+llm+behaviour+to+components%2E+Preprint%2C+arXiv%3A2403%2E00745%2E)
18. Thomas McGrath, Matthew Rahtz, Janos Kramar, Vladimir Mikulik, and Shane Legg. 2023. The hydra effect: Emergent self-repair in language model computations. Preprint, arXiv:2307.15771. [find](https://scholar.google.com/scholar?q=Thomas+McGrath%2C+Matthew+Rahtz%2C+Janos+Kramar%2C+Vladimir+Mikulik%2C+and+Shane+Legg%2E+2023%2E+The+hydra+effect%3A+Emergent+self%2Drepair+in+language+model+computations%2E+Preprint%2C+arXiv%3A2307%2E15771%2E)
19. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in GPT. Advances in Neural Information Processing Systems, 36. [find](https://scholar.google.com/scholar?q=Kevin+Meng%2C+David+Bau%2C+Alex+Andonian%2C+and+Yonatan+Belinkov%2E+2022%2E+Locating+and+editing+factual+associations+in+GPT%2E+Advances+in+Neural+Information+Processing+Systems%2C+36%2E)
20. Beren Millidge and Sid Black. 2022. The singular value decompositions of transformer weight matrices are highly interpretable. [find](https://scholar.google.com/scholar?q=Beren+Millidge+and+Sid+Black%2E+2022%2E+The+singular+value+decompositions+of+transformer+weight+matrices+are+highly+interpretable%2E)
21. Raul Molina. 2023. Traveling words: A geometric interpretation of transformers. Preprint, arXiv:2309.07315. — [arXiv:2309.07315](https://arxiv.org/abs/2309.07315) [find](https://scholar.google.com/scholar?q=Raul+Molina%2E+2023%2E+Traveling+words%3A+A+geometric+interpretation+of+transformers%2E+Preprint%2C+arXiv%3A2309%2E07315%2E+%E2%80%94+%5BarXiv%3A2309%2E07315%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2309%2E07315%29)
22. Team NLLB, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hof… [find](https://scholar.google.com/scholar?q=Team+NLLB%2C+Marta+R%2E+Costa%2Djuss%C3%A0%2C+James+Cross%2C+Onur+%C3%87elebi%2C+Maha+Elbayad%2C+Kenneth+Heafield%2C+Kevin+Heffernan%2C+Elahe+Kalbassi%2C+Janice+Lam%2C+Daniel+Licht%2C+Jean+Maillard%2C+Anna+Sun%2C+Skyler+Wang%2C+Guillaume+We)
23. Judea Pearl. 2009. Causality, 2 edition. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Judea+Pearl%2E+2009%2E+Causality%2C+2+edition%2E+Cambridge+University+Press%2E)
24. Slav Petrov, Dipanjan Das, and Ryan McDonald. 2012. [A universal part-of-speech tagset](https://md.research.elamur.ai/free/en/papers/petrov-2012-universal-part-speech-tagset/). In Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), pages 2089– 2096, Istanbul, Turkey. European Language Resources Association (ELRA).
25. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). J. Mach. Learn. Res., 21(1).
26. Cody Rushing and Neel Nanda. 2024. Explorations of self-repair in language models. Preprint, arXiv:2402.15390. — [arXiv:2402.15390](https://arxiv.org/abs/2402.15390) [find](https://scholar.google.com/scholar?q=Cody+Rushing+and+Neel+Nanda%2E+2024%2E+Explorations+of+self%2Drepair+in+language+models%2E+Preprint%2C+arXiv%3A2402%2E15390%2E+%E2%80%94+%5BarXiv%3A2402%2E15390%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2402%2E15390%29)
27. Alessandro Stolfo, Yonatan Belinkov, and Mrinmaya Sachan. 2023. Understanding arithmetic reasoning in language models using causal mediation analysis. Preprint, arXiv:2305.15054. [find](https://scholar.google.com/scholar?q=Alessandro+Stolfo%2C+Yonatan+Belinkov%2C+and+Mrinmaya+Sachan%2E+2023%2E+Understanding+arithmetic+reasoning+in+language+models+using+causal+mediation+analysis%2E+Preprint%2C+arXiv%3A2305%2E15054%2E)
28. Aaquib Syed, Can Rager, and Arthur Conmy. 2023. [Attribution patching outperforms automated circuit discovery](https://md.research.elamur.ai/free/en/papers/syed-2023-attribution-patching-outperforms-automated-circuit/). Preprint, arXiv:2310.10348.
29. Eric Todd, Millicent L. Li, Arnab Sen Sharma, Aaron Mueller, Byron C. Wallace, and David Bau. 2023. [Function vectors in large language models](https://md.research.elamur.ai/free/en/papers/todd-2023-function-vectors-large-language-models/). Preprint, arXiv:2310.15213.
30. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023a. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). Preprin…
31. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,… — [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/)
32. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.
33. Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. 2020. Investigating gender bias in language models using causal mediation analysis. In Advances in Neural Information Processing Systems, volume 33, pages 12388–12401. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Jesse+Vig%2C+Sebastian+Gehrmann%2C+Yonatan+Belinkov%2C+Sharon+Qian%2C+Daniel+Nevo%2C+Yaron+Singer%2C+and+Stuart+Shieber%2E+2020%2E+Investigating+gender+bias+in+language+models+using+causal+mediation+analysis%2E+In+Adva)
34. Elena Voita, Javier Ferrando, and Christoforos Nalmpantis. 2023. Neurons in large language models: Dead, n-gram, positional. Preprint, arXiv:2309.04827. [find](https://scholar.google.com/scholar?q=Elena+Voita%2C+Javier+Ferrando%2C+and+Christoforos+Nalmpantis%2E+2023%2E+Neurons+in+large+language+models%3A+Dead%2C+n%2Dgram%2C+positional%2E+Preprint%2C+arXiv%3A2309%2E04827%2E)
35. Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov. 2019. [Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned](https://md.research.elamur.ai/free/en/papers/voita-2019-analyzing-multi-head-self-attention/). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5797–5808, Florence, I…
36. Kevin Ro Wang, Alexandre Variengien, Arthur Conmy, Buck Shlegeris, and Jacob Steinhardt. 2023. Interpretability in the wild: a circuit for indirect object identification in GPT-2 small. In The Eleventh International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Kevin+Ro+Wang%2C+Alexandre+Variengien%2C+Arthur+Conmy%2C+Buck+Shlegeris%2C+and+Jacob+Steinhardt%2E+2023%2E+Interpretability+in+the+wild%3A+a+circuit+for+indirect+object+identification+in+GPT%2D2+small%2E+In+The+Elevent)

*Updated 2 November 2026*