---
title: "Neuron to Graph: Interpreting Language Model Neurons at Scale"
description: "Foote et al. propose Neuron to Graph (N2G), a pipeline that converts each neuron into a compact… 33 references, cited by 12 papers here."
authors: ["Alex Foote", "Neel Nanda", "Esben Kran", "Ioannis Konstas", "Shay Cohen", "Fazl Barez"]
year: 2023
venue: "arXiv:2305.19911"
source: "https://arxiv.org/abs/2305.19911"
doi: "10.48550/arxiv.2305.19911"
arxiv: "2305.19911"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/foote-2023-neuron-graph-interpreting-language-model/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/foote-2023-neuron-graph-interpreting-language-model/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/14/) / Neuron to Graph: Interpreting Language Model Neurons at Scale

# Neuron to Graph: Interpreting Language Model Neurons at Scale

**Authors:** Alex Foote, [Neel Nanda](https://md.research.elamur.ai/free/en/authors/neel-nanda/), Esben Kran, [Ioannis Konstas](https://md.research.elamur.ai/free/en/authors/ioannis-konstas/), [Shay Cohen](https://md.research.elamur.ai/free/en/authors/shay-b-cohen/) and Fazl Barez · **Year:** 2023 · **Venue:** arXiv:2305.19911 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2305.19911) · [PDF](https://arxiv.org/pdf/2305.19911) · [DOI 10.48550/arxiv.2305.19911](https://doi.org/10.48550/arxiv.2305.19911) · [arXiv 2305.19911](https://arxiv.org/abs/2305.19911) · [OpenAlex](https://openalex.org/W4379089565)

**Citations:** 3 citations (OpenAlex, 2026-10-14) · [12](https://md.research.elamur.ai/free/en/papers/foote-2023-neuron-graph-interpreting-language-model/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Interpreting individual neurons in large language models usually means a researcher reads text snippets where a neuron fires strongly and guesses the shared pattern. This is slow, hard to verify, and can produce misleading conclusions. Foote et al. propose Neuron to Graph (N2G), a pipeline that converts each neuron into a compact graph of the token sequences that trigger it. N2G starts from the top activating training examples supplied by Neuroscope. It trims each example to the shortest preceding context that still drives the neuron, scores how much each remaining token matters, and creates extra variants by swapping key tokens for DistilBERT-suggested alternatives. These examples are merged into a lattice-like graph that people can inspect. The graph can also predict token-level activations, so its fidelity can be checked automatically against the real neuron. The authors applied N2G to every MLP neuron of a six-layer SoLU model trained on the Pile. It predicted activations better than two baselines and fit early layers well but later layers only partly. It also made neurons searchable and comparable, which is a step toward scalable, structured interpretability.

## Key points
- **Test model:** a SoLU Transformer whose activation function encourages monosemantic neurons. It has 6 blocks, each with one MLP layer of 3072 neurons (about 18.4k neurons in total), and was trained on the Pile.
- **Input data:** Neuroscope supplies the 20 most activating training examples per neuron, each 1024 tokens long. Pruning keeps adding left context until the pivot token reaches at least 50% of its original activation.
- **Scalability:** graphs for all neurons in the model were built on a single Tesla T4 GPU. The code is released at github.com/alexjfoote/Neuron2Graph.
- **Applications:**
  - Searching the graphs found an in-context learning neuron that reacts to repeated "ab...ab" patterns.
  - Pairwise graph comparison, using more than 90% overlap in both context and activating tokens, found 60 pairs of near-identical neurons, such as one in layer 1 and one in layer 4.
  - The authors link these duplicate pairs to superposition or dropout.
- **Limitations:**
  - Fidelity drops in later layers.
  - Polysemantic neurons in ordinary Transformers may be harder to capture.
  - Matching is based on exact tokens rather than abstract concepts.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Edinburgh](https://md.research.elamur.ai/free/en/universities/university-of-edinburgh/) — University, Edinburgh, United Kingdom · 396 works · 82,279 citations
- [Heriot-Watt University](https://md.research.elamur.ai/free/en/universities/heriot-watt-university/) — University, Edinburgh, United Kingdom · 18 works · 6,009 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,959 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,789 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 4,145 works

## Cite

Cite as: Alex Foote, Neel Nanda, Esben Kran, Ioannis Konstas, Shay Cohen and Fazl Barez. 2023. Neuron to Graph: Interpreting Language Model Neurons at Scale. arXiv:2305.19911.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{foote2023neuron,
  title = {{Neuron to Graph: Interpreting Language Model Neurons at Scale}},
  author = {Alex Foote, Neel Nanda, Esben Kran, Ioannis Konstas, Shay Cohen and Fazl Barez},
  year = {2023},
  doi = {10.48550/arxiv.2305.19911},
  eprint = {2305.19911},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2305.19911}
}
```

- [← previous Mapping AMR to UMR: Resources for Adapting Existing Corpora for Cross-Lingual Compatibility](https://md.research.elamur.ai/free/en/papers/bonn-2023-mapping-amr-umr-resources-adapting/)
- [next → TigerBot: An Open Multilingual Multitask LLM](https://md.research.elamur.ai/free/en/papers/chen-2023-tigerbot-open-multilingual-multitask/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/14/)

## References

33 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. [Concrete problems in ai safety](https://md.research.elamur.ai/free/en/papers/amodei-2016-concrete-problems-ai-safety/). arXiv preprint arXiv:1606.06565, 2016.
2. Fazl Barez, Hosein Hasanbeig, and Alessandro Abate. System III: Learning with domain knowledge for safety constraints. In NeurIPS ML Safety Workshop, 2022. [find](https://scholar.google.com/scholar?q=Fazl+Barez%2C+Hosein+Hasanbeig%2C+and+Alessandro+Abate%2E+System+III%3A+Learning+with+domain+knowledge+for+safety+constraints%2E+In+NeurIPS+ML+Safety+Workshop%2C+2022%2E)
3. Anthony Bau, Yonatan Belinkov, Hassan Sajjad, Nadir Durrani, Fahim Dalvi, and James R. Glass. [Identifying and controlling important neurons in neural machine translation](https://md.research.elamur.ai/free/en/papers/bau-2018-identifying-controlling-important-neurons-neural/). CoRR, abs/1811.01157, 2018.
4. Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu, and William Saunders. Language models can explain neurons in language models. 2023. [find](https://scholar.google.com/scholar?q=Steven+Bills%2C+Nick+Cammarata%2C+Dan+Mossing%2C+Henk+Tillman%2C+Leo+Gao%2C+Gabriel+Goh%2C+Ilya+Sutskever%2C+Jan+Leike%2C+Jeff+Wu%2C+and+William+Saunders%2E+Language+models+can+explain+neurons+in+language+models%2E+2023%2E)
5. Tolga Bolukbasi, Adam Pearce, Ann Yuan, Andy Coenen, Emily Reif, Fernanda Viégas, and Martin Wattenberg. An interpretability illusion for bert. arXiv preprint arXiv:2104.07143, 2021. — [arXiv:2104.07143](https://arxiv.org/abs/2104.07143) [find](https://scholar.google.com/scholar?q=Tolga+Bolukbasi%2C+Adam+Pearce%2C+Ann+Yuan%2C+Andy+Coenen%2C+Emily+Reif%2C+Fernanda+Vi%C3%A9gas%2C+and+Martin+Wattenberg%2E+An+interpretability+illusion+for+bert%2E+arXiv+preprint+arXiv%3A2104%2E07143%2C+2021%2E+%E2%80%94+%5BarXiv%3A2104%2E071)
6. Tolga Bolukbasi, Adam Pearce, Ann Yuan, Andy Coenen, Emily Reif, Fernanda B. Viégas, and Martin Wattenberg. An interpretability illusion for BERT. CoRR, abs/2104.07143, 2021. — [arXiv:2104.07143](https://arxiv.org/abs/2104.07143) [find](https://scholar.google.com/scholar?q=Tolga+Bolukbasi%2C+Adam+Pearce%2C+Ann+Yuan%2C+Andy+Coenen%2C+Emily+Reif%2C+Fernanda+B%2E+Vi%C3%A9gas%2C+and+Martin+Wattenberg%2E+An+interpretability+illusion+for+BERT%2E+CoRR%2C+abs%2F2104%2E07143%2C+2021%2E+%E2%80%94+%5BarXiv%3A2104%2E07143%5D%28http)
7. Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, and Furu Wei. Knowledge neurons in pretrained transformers. CoRR, abs/2104.08696, 2021. [find](https://scholar.google.com/scholar?q=Damai+Dai%2C+Li+Dong%2C+Yaru+Hao%2C+Zhifang+Sui%2C+and+Furu+Wei%2E+Knowledge+neurons+in+pretrained+transformers%2E+CoRR%2C+abs%2F2104%2E08696%2C+2021%2E)
8. Fahim Dalvi, Nadir Durrani, Hassan Sajjad, Yonatan Belinkov, Anthony Bau, and James Glass. What is one grain of sand in the desert? analyzing individual neurons in deep NLP models. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01):6309–6317, July 2019. [find](https://scholar.google.com/scholar?q=Fahim+Dalvi%2C+Nadir+Durrani%2C+Hassan+Sajjad%2C+Yonatan+Belinkov%2C+Anthony+Bau%2C+and+James+Glass%2E+What+is+one+grain+of+sand+in+the+desert%3F+analyzing+individual+neurons+in+deep+NLP+models%2E+Proceedings+of+the+)
9. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/), 2018. cite arxiv:1810.04805Comment: 13 pages.
10. Nadir Durrani, Hassan Sajjad, Fahim Dalvi, and Yonatan Belinkov. Analyzing individual neurons in pre-trained language models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4865–4880, Online, November 2020. Association for Computational Lingu… [find](https://scholar.google.com/scholar?q=Nadir+Durrani%2C+Hassan+Sajjad%2C+Fahim+Dalvi%2C+and+Yonatan+Belinkov%2E+Analyzing+individual+neurons+in+pre%2Dtrained+language+models%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natural+Lang)
11. Nelson Elhage, Tristan Hume, Catherine Olsson, Neel Nanda, Tom Henighan, Scott Johnston, Sheer ElShowk, Nicholas Joseph, Nova DasSarma, Ben Mann, Danny Hernandez, Amanda Askell, Kamal Ndousse, And Jones, Dawn Drain, Anna Chen, Yuntao Bai, Deep Ganguli, Liane Lovitt, Zac Hatfield-Dodds, Jackson Kerni… [find](https://scholar.google.com/scholar?q=Nelson+Elhage%2C+Tristan+Hume%2C+Catherine+Olsson%2C+Neel+Nanda%2C+Tom+Henighan%2C+Scott+Johnston%2C+Sheer+ElShowk%2C+Nicholas+Joseph%2C+Nova+DasSarma%2C+Ben+Mann%2C+Danny+Hernandez%2C+Amanda+Askell%2C+Kamal+Ndousse%2C+And+Jon)
12. Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. Toy models of superposition, 2022. [find](https://scholar.google.com/scholar?q=Nelson+Elhage%2C+Tristan+Hume%2C+Catherine+Olsson%2C+Nicholas+Schiefer%2C+Tom+Henighan%2C+Shauna+Kravec%2C+Zac+Hatfield%2DDodds%2C+Robert+Lasenby%2C+Dawn+Drain%2C+Carol+Chen%2C+Roger+Grosse%2C+Sam+McCandlish%2C+Jared+Kaplan%2C+D)
13. Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amodei, Tom Brown, Ja… [find](https://scholar.google.com/scholar?q=Nelson+Elhage%2C+Neel+Nanda%2C+Catherine+Olsson%2C+Tom+Henighan%2C+Nicholas+Joseph%2C+Ben+Mann%2C+Amanda+Askell%2C+Yuntao+Bai%2C+Anna+Chen%2C+Tom+Conerly%2C+Nova+DasSarma%2C+Dawn+Drain%2C+Deep+Ganguli%2C+Zac+Hatfield%2DDodds%2C+Da)
14. Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. The Pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027, 2020. [find](https://scholar.google.com/scholar?q=Leo+Gao%2C+Stella+Biderman%2C+Sid+Black%2C+Laurence+Golding%2C+Travis+Hoppe%2C+Charles+Foster%2C+Jason+Phang%2C+Horace+He%2C+Anish+Thite%2C+Noa+Nabeshima%2C+Shawn+Presser%2C+and+Connor+Leahy%2E+The+Pile%3A+An+800gb+dataset+of+)
15. Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. Transformer feed-forward layers are key-value memories. arXiv preprint arXiv:2012.14913, 2020. [find](https://scholar.google.com/scholar?q=Mor+Geva%2C+Roei+Schuster%2C+Jonathan+Berant%2C+and+Omer+Levy%2E+Transformer+feed%2Dforward+layers+are+key%2Dvalue+memories%2E+arXiv+preprint+arXiv%3A2012%2E14913%2C+2020%2E)
16. Gabriel Goh, Nick Cammarata, Chelsea Voss, Shan Carter, Michael Petrov, Ludwig Schubert, Alec Radford, and Chris Olah. Multimodal neurons in artificial neural networks. Distill, 6(3):e30, 2021. [find](https://scholar.google.com/scholar?q=Gabriel+Goh%2C+Nick+Cammarata%2C+Chelsea+Voss%2C+Shan+Carter%2C+Michael+Petrov%2C+Ludwig+Schubert%2C+Alec+Radford%2C+and+Chris+Olah%2E+Multimodal+neurons+in+artificial+neural+networks%2E+Distill%2C+6%283%29%3Ae30%2C+2021%2E)
17. Dan Hendrycks, Nicholas Carlini, John Schulman, and Jacob Steinhardt. [Unsolved problems in ml safety](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-unsolved-problems-ml-safety/). arXiv preprint arXiv:2109.13916, 2021.
18. Jesse Mu and Jacob Andreas. Compositional explanations of neurons. CoRR, abs/2006.14032, 2020. — [arXiv:2006.14032](https://arxiv.org/abs/2006.14032) [find](https://scholar.google.com/scholar?q=Jesse+Mu+and+Jacob+Andreas%2E+Compositional+explanations+of+neurons%2E+CoRR%2C+abs%2F2006%2E14032%2C+2020%2E+%E2%80%94+%5BarXiv%3A2006%2E14032%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2006%2E14032%29)
19. Neel Nanda. Neuroscope, 2022. [find](https://scholar.google.com/scholar?q=Neel+Nanda%2E+Neuroscope%2C+2022%2E)
20. Chris Olah. Mechanistic interpretability, variables, and the importance of interpretable bases. Transformer Circuits Thread(June 27). [http://www.transformer-circuits](http://www.transformer-circuits). pub/2022/mech-interp-essay/index. html, 2022. [↗](http://www.transformer-circuits)
21. Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. An overview of early vision in inceptionv1. Distill, 5(4):e00024–002, 2020. [find](https://scholar.google.com/scholar?q=Chris+Olah%2C+Nick+Cammarata%2C+Ludwig+Schubert%2C+Gabriel+Goh%2C+Michael+Petrov%2C+and+Shan+Carter%2E+An+overview+of+early+vision+in+inceptionv1%2E+Distill%2C+5%284%29%3Ae00024%E2%80%93002%2C+2020%2E)
22. Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. Zoom in: An introduction to circuits. Distill, 2020. [https://distill.pub/2020/circuits/zoom-in](https://distill.pub/2020/circuits/zoom-in). [↗](https://distill.pub/2020/circuits/zoom-in)
23. Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. Feature visualization. Distill, 2017. [https://distill.pub/2017/feature-visualization](https://distill.pub/2017/feature-visualization). [↗](https://distill.pub/2017/feature-visualization)
24. Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Scott Johnston, Andy Jones, Jackson Kernion, Liane Lovitt, Kamal Ndousse, Dario Amode… — [In-context Learning and Induction Heads](https://md.research.elamur.ai/free/en/papers/olsson-2022-context-learning-induction-heads/)
25. Nina Poerner, Benjamin Roth, and Hinrich Schütze. Interpretable textual neuron representations for NLP. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 325–327, Brussels, Belgium, November 2018. Association for Computational Linguistic… [find](https://scholar.google.com/scholar?q=Nina+Poerner%2C+Benjamin+Roth%2C+and+Hinrich+Sch%C3%BCtze%2E+Interpretable+textual+neuron+representations+for+NLP%2E+In+Proceedings+of+the+2018+EMNLP+Workshop+BlackboxNLP%3A+Analyzing+and+Interpreting+Neural+Network)
26. Alec Radford, Rafal Jozefowicz, and Ilya Sutskever. [Learning to generate reviews and discovering sentiment](https://md.research.elamur.ai/free/en/papers/radford-2017-learning-generate-reviews-discovering-sentiment/). arXiv preprint arXiv:1704.01444, 2017.
27. Alec Radford, Rafal Józefowicz, and Ilya Sutskever. [Learning to generate reviews and discovering sentiment](https://md.research.elamur.ai/free/en/papers/radford-2017-learning-generate-reviews-discovering-sentiment/). ArXiv, abs/1704.01444, 2017.
28. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. [Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter](https://md.research.elamur.ai/free/en/papers/sanh-2019-distilbert-distilled-version-bert-smaller/), 2020.
29. Ludwig Schubert, Chelsea Voss, Nick Cammarata, Gabriel Goh, and Chris Olah. High-low frequency detectors. Distill, 6(1):e00024–005, 2021. [find](https://scholar.google.com/scholar?q=Ludwig+Schubert%2C+Chelsea+Voss%2C+Nick+Cammarata%2C+Gabriel+Goh%2C+and+Chris+Olah%2E+High%2Dlow+frequency+detectors%2E+Distill%2C+6%281%29%3Ae00024%E2%80%93005%2C+2021%2E)
30. Esther Seyffarth, Younes Samih, Laura Kallmeyer, and Hassan Sajjad. Implicit representations of event properties within contextual language models: Searching for “causativity neurons”. In Proceedings of the 14th International Conference on Computational Semantics (IWCS), pages 110–120, Groningen, Th… [find](https://scholar.google.com/scholar?q=Esther+Seyffarth%2C+Younes+Samih%2C+Laura+Kallmeyer%2C+and+Hassan+Sajjad%2E+Implicit+representations+of+event+properties+within+contextual+language+models%3A+Searching+for+%E2%80%9Ccausativity+neurons%E2%80%9D%2E+In+Proceedings+)
31. Lucas Torroba Hennigen, Adina Williams, and Ryan Cotterell. Intrinsic probing through dimension selection. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 197–216, Online, November 2020. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Lucas+Torroba+Hennigen%2C+Adina+Williams%2C+and+Ryan+Cotterell%2E+Intrinsic+probing+through+dimension+selection%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EM)
32. Jesse Vig, Sebastian Gehrmann, Yonatan Belinkov, Sharon Qian, Daniel Nevo, Yaron Singer, and Stuart Shieber. Investigating Gender Bias in Language Models Using Causal Mediation Analysis. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin, editors, Advances in Neural Information Proce… [find](https://scholar.google.com/scholar?q=Jesse+Vig%2C+Sebastian+Gehrmann%2C+Yonatan+Belinkov%2C+Sharon+Qian%2C+Daniel+Nevo%2C+Yaron+Singer%2C+and+Stuart+Shieber%2E+Investigating+Gender+Bias+in+Language+Models+Using+Causal+Mediation+Analysis%2E+In+H%2E+Laroche)
33. John Wu, Yonatan Belinkov, Hassan Sajjad, Nadir Durrani, Fahim Dalvi, and James Glass. Similarity analysis of contextual word representation models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4638–4655, Online, July 2020. Association for Computa… [find](https://scholar.google.com/scholar?q=John+Wu%2C+Yonatan+Belinkov%2C+Hassan+Sajjad%2C+Nadir+Durrani%2C+Fahim+Dalvi%2C+and+James+Glass%2E+Similarity+analysis+of+contextual+word+representation+models%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+As)

*Updated 3 November 2026*