---
title: "Finding Skill Neurons in Pre-trained Transformer-based"
description: "The authors examine individual feed-forward (FFN) neurons in RoBERTa BASE. 69 references, cited by 58 papers here."
authors: ["Xiaozhi Wang", "Kaiyue Wen", "Zhengyan Zhang", "Lei Hou", "Zhiyuan Liu", "Juanzi Li"]
year: 2022
venue: "Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/2022.emnlp-main.765/"
doi: "10.18653/v1/2022.emnlp-main.765"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wang-2022-finding-skill-neurons-pre-trained/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2022-finding-skill-neurons-pre-trained/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/4/) / Finding Skill Neurons in Pre-trained Transformer-based Language Models

# Finding Skill Neurons in Pre-trained Transformer-based Language Models

**Authors:** [Xiaozhi Wang](https://md.research.elamur.ai/free/en/authors/xiaozhi-wang/), Kaiyue Wen, [Zhengyan Zhang](https://md.research.elamur.ai/free/en/authors/zhengyan-zhang/), [Lei Hou](https://md.research.elamur.ai/free/en/authors/lei-hou/), [Zhiyuan Liu](https://md.research.elamur.ai/free/en/authors/zhiyuan-liu/) and [Juanzi Li](https://md.research.elamur.ai/free/en/authors/juanzi-li/) · **Year:** 2022 · **Venue:** [Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2022](https://md.research.elamur.ai/free/en/venues/emnlp/2022/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2022.emnlp-main.765/) · [PDF](https://aclanthology.org/2022.emnlp-main.765.pdf) · [DOI 10.18653/v1/2022.emnlp-main.765](https://doi.org/10.18653/v1/2022.emnlp-main.765) · [OpenAlex](https://openalex.org/W4385573694)

**Citations:** 27 citations (OpenAlex, 2026-10-14) · [58](https://md.research.elamur.ai/free/en/papers/wang-2022-finding-skill-neurons-pre-trained/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Pre-trained Transformers perform well across many NLP tasks, but it is unclear where task-relevant abilities sit among their parameters. The authors examine individual feed-forward (FFN) neurons in RoBERTa BASE. They find that after prompt tuning, some neurons' activations on soft prompt tokens separate task labels very well. They call these "skill neurons." To find them, each neuron's activation is compared with its average over the training set, and the method measures how well this threshold predicts validation labels across several prompts. Multi-class tasks are split into binary subtasks.

On seven English classification tasks, skill neurons appear consistently. Adding noise to them hurts accuracy far more than perturbing random neurons. Related tasks share similar skill neurons, and the effect is not explained by keyword selectivity. The neurons also matter for adapter tuning and BitFit, and they stay predictive with random or untuned prompts. This suggests the skills originate in pre-training. The work aids interpretability and enables pruning and better prompt-transfer prediction.

## Key points
- **Setup:** RoBERTa BASE was tested on 7 tasks of 3 types, with 5 random trials each:
  - sentiment: SST-2, IMDB, TweetEval;
  - NLI: MNLI, QNLI;
  - topic classification: AG News, DBpedia.
- **Definition of neurons:** neurons are the columns of the FFN weight matrices, and their activations are the values after the FFN's first projection.
- **Evidence for pre-training origin:** skill neurons found through prompt tuning are also important for adapter-based tuning and BitFit, which likewise keep neuron weights frozen.
- **Pruning application:** keeping only the top 2% of skill neurons active in the top 9 layers cuts the model to 66.6% of its parameters. It gives about 1.4× faster inference with accuracy comparable to standard prompt tuning.
- **Transferability indicator:** restricting the overlap-of-activated-neurons metric from Su et al. (2021) to skill neurons predicts prompt transferability noticeably better. Code is available at github.com/THU-KEG/Skill-Neuron.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Tsinghua University](https://md.research.elamur.ai/free/en/universities/tsinghua-university/) — University, Beijing, China · 137 works · 83,262 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 1,344 works · [🔬 Interpretability](https://md.research.elamur.ai/free/en/topics/interpretability/) · 58 works

## Cite

Cite as: Xiaozhi Wang, Kaiyue Wen, Zhengyan Zhang, Lei Hou, Zhiyuan Liu and Juanzi Li. 2022. Finding Skill Neurons in Pre-trained Transformer-based Language Models. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wang2022finding,
  title = {{Finding Skill Neurons in Pre-trained Transformer-based Language Models}},
  author = {Xiaozhi Wang, Kaiyue Wen, Zhengyan Zhang, Lei Hou, Zhiyuan Liu and Juanzi Li},
  booktitle = {Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing},
  year = {2022},
  doi = {10.18653/v1/2022.emnlp-main.765},
  url = {https://doi.org/10.18653/v1/2022.emnlp-main.765}
}
```

- [← previous BlonDe: An Automatic Evaluation Metric for Document-level Machine Translation](https://md.research.elamur.ai/free/en/papers/jiang-2022-blonde-automatic-evaluation-metric-document/)
- [next → Neuron-level Interpretation of Deep NLP Models: A Survey](https://md.research.elamur.ai/free/en/papers/sajjad-2022-neuron-level-interpretation-deep-nlp/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/4/)

## References

69 references, 28 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Pulkit Agrawal, Ross B. Girshick, and Jitendra Malik. 2014. Analyzing the performance of multilayer neural networks for object recognition. In Proceedings of ECCV, pages 329–344. [find](https://scholar.google.com/scholar?q=Pulkit+Agrawal%2C+Ross+B%2E+Girshick%2C+and+Jitendra+Malik%2E+2014%2E+Analyzing+the+performance+of+multilayer+neural+networks+for+object+recognition%2E+In+Proceedings+of+ECCV%2C+pages+329%E2%80%93344%2E)
2. Omer Antverg and Yonatan Belinkov. 2022. On the pitfalls of analyzing individual neurons in language models. In Proceedings of ICLR. [find](https://scholar.google.com/scholar?q=Omer+Antverg+and+Yonatan+Belinkov%2E+2022%2E+On+the+pitfalls+of+analyzing+individual+neurons+in+language+models%2E+In+Proceedings+of+ICLR%2E)
3. Sajid Anwar, Kyuyeon Hwang, and Wonyong Sung. 2017. Structured pruning of deep convolutional neural networks. ACM Journal on Emerging Technologies in Computing Systems (JETC), 13(3):1–18. [find](https://scholar.google.com/scholar?q=Sajid+Anwar%2C+Kyuyeon+Hwang%2C+and+Wonyong+Sung%2E+2017%2E+Structured+pruning+of+deep+convolutional+neural+networks%2E+ACM+Journal+on+Emerging+Technologies+in+Computing+Systems+%28JETC%29%2C+13%283%29%3A1%E2%80%9318%2E)
4. Sanjeev Arora, Rong Ge, Behnam Neyshabur, and Yi Zhang. 2018. Stronger generalization bounds for deep nets via a compression approach. In Proceedings of ICML, pages 254–263. [find](https://scholar.google.com/scholar?q=Sanjeev+Arora%2C+Rong+Ge%2C+Behnam+Neyshabur%2C+and+Yi+Zhang%2E+2018%2E+Stronger+generalization+bounds+for+deep+nets+via+a+compression+approach%2E+In+Proceedings+of+ICML%2C+pages+254%E2%80%93263%2E)
5. Sören Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary Ives. 2007. DBpedia: A nucleus for a web of open data. In Proceedings of ISWC/ASWC, pages 722–735. [find](https://scholar.google.com/scholar?q=S%C3%B6ren+Auer%2C+Christian+Bizer%2C+Georgi+Kobilarov%2C+Jens+Lehmann%2C+Richard+Cyganiak%2C+and+Zachary+Ives%2E+2007%2E+DBpedia%3A+A+nucleus+for+a+web+of+open+data%2E+In+Proceedings+of+ISWC%2FASWC%2C+pages+722%E2%80%93735%2E)
6. Francesco Barbieri, Jose Camacho-Collados, Luis Espinosa Anke, and Leonardo Neves. 2020. [TweetEval: Unified benchmark and comparative evaluation for tweet classification](https://md.research.elamur.ai/free/en/papers/barbieri-2020-tweeteval-unified-benchmark-comparative-evaluation/). In Findings of EMNLP, pages 1644–1650.
7. Horace B Barlow. 1972. Single units and sensation: A neuron doctrine for perceptual psychology? Perception, 1(4):371–394. [find](https://scholar.google.com/scholar?q=Horace+B+Barlow%2E+1972%2E+Single+units+and+sensation%3A+A+neuron+doctrine+for+perceptual+psychology%3F+Perception%2C+1%284%29%3A371%E2%80%93394%2E)
8. Anthony Bau, Yonatan Belinkov, Hassan Sajjad, Nadir Durrani, Fahim Dalvi, and James Glass. 2018. Identifying and controlling important neurons in neural machine translation. In Proceedings of ICLR. [find](https://scholar.google.com/scholar?q=Anthony+Bau%2C+Yonatan+Belinkov%2C+Hassan+Sajjad%2C+Nadir+Durrani%2C+Fahim+Dalvi%2C+and+James+Glass%2E+2018%2E+Identifying+and+controlling+important+neurons+in+neural+machine+translation%2E+In+Proceedings+of+ICLR%2E)
9. David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. 2017. Network dissection: Quantifying interpretability of deep visual representations. Proceedings of CVPR, pages 3319–3327. [find](https://scholar.google.com/scholar?q=David+Bau%2C+Bolei+Zhou%2C+Aditya+Khosla%2C+Aude+Oliva%2C+and+Antonio+Torralba%2E+2017%2E+Network+dissection%3A+Quantifying+interpretability+of+deep+visual+representations%2E+Proceedings+of+CVPR%2C+pages+3319%E2%80%933327%2E)
10. David Bau, Jun-Yan Zhu, Hendrik Strobelt, Agata Lapedriza, Bolei Zhou, and Antonio Torralba. 2020. Understanding the role of individual units in a deep neural network. Proceedings of the National Academy of Sciences, 117(48):30071–30078. [find](https://scholar.google.com/scholar?q=David+Bau%2C+Jun%2DYan+Zhu%2C+Hendrik+Strobelt%2C+Agata+Lapedriza%2C+Bolei+Zhou%2C+and+Antonio+Torralba%2E+2020%2E+Understanding+the+role+of+individual+units+in+a+deep+neural+network%2E+Proceedings+of+the+National+Acad)
11. Elad Ben-Zaken, Shauli Ravfogel, and Yoav Goldberg. 2022. [BitFit: Simple parameter-efficient fine-tuning for transformer-based masked language-models](https://md.research.elamur.ai/free/en/papers/ben-zaken-2021-bitfit-simple-parameter-efficient-fine/). In Proceedings of ACL.
12. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… — [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
13. Xu Han, Zhengyan Zhang, Ning Ding, Yuxian Gu, Xiao Liu, Yuqi Huo, Jiezhong Qiu, Liang Zhang, Wentao Han, Minlie Huang, et al. 2021. Pre-trained models: Past, present and future. AI Open, pages 225–250. [find](https://scholar.google.com/scholar?q=Xu+Han%2C+Zhengyan+Zhang%2C+Ning+Ding%2C+Yuxian+Gu%2C+Xiao+Liu%2C+Yuqi+Huo%2C+Jiezhong+Qiu%2C+Liang+Zhang%2C+Wentao+Han%2C+Minlie+Huang%2C+et+al%2E+2021%2E+Pre%2Dtrained+models%3A+Past%2C+present+and+future%2E+AI+Open%2C+pages+225%E2%80%93250)
14. Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D. Manning. 2019. What does BERT look at? an analysis of BERT’s attention. In Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 276–286. [find](https://scholar.google.com/scholar?q=Kevin+Clark%2C+Urvashi+Khandelwal%2C+Omer+Levy%2C+and+Christopher+D%2E+Manning%2E+2019%2E+What+does+BERT+look+at%3F+an+analysis+of+BERT%E2%80%99s+attention%2E+In+Proceedings+of+the+2019+ACL+Workshop+BlackboxNLP%3A+Analyzing+an)
15. John Hewitt and Christopher D Manning. 2019. [A structural probe for finding syntax in word representations](https://md.research.elamur.ai/free/en/papers/hewitt-2019-structural-probe-finding-syntax-word/). In Proceedings of NACCL-HLT, pages 4129–4138.
16. Adam Coates, Andrej Karpathy, and A. Ng. 2012. Emergence of object-selective features in unsupervised feature learning. In Proceedings of NeurIPS, pages 2681–2689. [find](https://scholar.google.com/scholar?q=Adam+Coates%2C+Andrej+Karpathy%2C+and+A%2E+Ng%2E+2012%2E+Emergence+of+object%2Dselective+features+in+unsupervised+feature+learning%2E+In+Proceedings+of+NeurIPS%2C+pages+2681%E2%80%932689%2E)
17. Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, and Furu Wei. 2021. Knowledge neurons in pretrained transformers. arXiv preprint, arXiv:2104.08696. [find](https://scholar.google.com/scholar?q=Damai+Dai%2C+Li+Dong%2C+Yaru+Hao%2C+Zhifang+Sui%2C+and+Furu+Wei%2E+2021%2E+Knowledge+neurons+in+pretrained+transformers%2E+arXiv+preprint%2C+arXiv%3A2104%2E08696%2E)
18. Fahim Dalvi, Nadir Durrani, Hassan Sajjad, Yonatan Belinkov, Anthony Bau, and James Glass. 2019. What is one grain of sand in the desert? analyzing individual neurons in deep nlp models. In Proceedings of AAAI, pages 6309–6317. [find](https://scholar.google.com/scholar?q=Fahim+Dalvi%2C+Nadir+Durrani%2C+Hassan+Sajjad%2C+Yonatan+Belinkov%2C+Anthony+Bau%2C+and+James+Glass%2E+2019%2E+What+is+one+grain+of+sand+in+the+desert%3F+analyzing+individual+neurons+in+deep+nlp+models%2E+In+Proceeding)
19. Fahim Dalvi, Hassan Sajjad, Nadir Durrani, and Yonatan Belinkov. 2020. Analyzing redundancy in pretrained transformer models. In Proceedings of EMNLP, pages 4908–4926. [find](https://scholar.google.com/scholar?q=Fahim+Dalvi%2C+Hassan+Sajjad%2C+Nadir+Durrani%2C+and+Yonatan+Belinkov%2E+2020%2E+Analyzing+redundancy+in+pretrained+transformer+models%2E+In+Proceedings+of+EMNLP%2C+pages+4908%E2%80%934926%2E)
20. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of NAACL-HLT, pages 4171–4186.
21. Yihe Dong, Jean-Baptiste Cordonnier, and Andreas Loukas. 2021. Attention is not all you need: pure attention loses rank doubly exponentially with depth. In Proceedings of ICML, pages 2793–2803. [find](https://scholar.google.com/scholar?q=Yihe+Dong%2C+Jean%2DBaptiste+Cordonnier%2C+and+Andreas+Loukas%2E+2021%2E+Attention+is+not+all+you+need%3A+pure+attention+loses+rank+doubly+exponentially+with+depth%2E+In+Proceedings+of+ICML%2C+pages+2793%E2%80%932803%2E)
22. Nadir Durrani, Hassan Sajjad, Fahim Dalvi, and Yonatan Belinkov. 2020. Analyzing individual neurons in pre-trained language models. In Proceedings of EMNLP, pages 4865–4880. [find](https://scholar.google.com/scholar?q=Nadir+Durrani%2C+Hassan+Sajjad%2C+Fahim+Dalvi%2C+and+Yonatan+Belinkov%2E+2020%2E+Analyzing+individual+neurons+in+pre%2Dtrained+language+models%2E+In+Proceedings+of+EMNLP%2C+pages+4865%E2%80%934880%2E)
23. Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2021. Transformer feed-forward layers are key-value memories. In Proceedings of EMNLP, pages 5484–5495. [find](https://scholar.google.com/scholar?q=Mor+Geva%2C+Roei+Schuster%2C+Jonathan+Berant%2C+and+Omer+Levy%2E+2021%2E+Transformer+feed%2Dforward+layers+are+key%2Dvalue+memories%2E+In+Proceedings+of+EMNLP%2C+pages+5484%E2%80%935495%2E)
24. Mitchell A Gordon, Kevin Duh, and Nicholas Andrews. 2020. [Compressing bert: Studying the effects of weight pruning on transfer learning](https://md.research.elamur.ai/free/en/papers/gordon-2020-compressing-bert-studying-effects-weight/). arXiv preprint arXiv:2002.08307.
25. Song Han, Jeff Pool, John Tran, and William Dally. 2015. Learning both weights and connections for efficient neural network. In Proceedings of NeurIPS, pages 1135–1143. [find](https://scholar.google.com/scholar?q=Song+Han%2C+Jeff+Pool%2C+John+Tran%2C+and+William+Dally%2E+2015%2E+Learning+both+weights+and+connections+for+efficient+neural+network%2E+In+Proceedings+of+NeurIPS%2C+pages+1135%E2%80%931143%2E)
26. Lucas Torroba Hennigen, Adina Williams, and Ryan Cotterell. 2020. Intrinsic probing through dimension selection. In Proceedings of EMNLP, pages 197– 216. [find](https://scholar.google.com/scholar?q=Lucas+Torroba+Hennigen%2C+Adina+Williams%2C+and+Ryan+Cotterell%2E+2020%2E+Intrinsic+probing+through+dimension+selection%2E+In+Proceedings+of+EMNLP%2C+pages+197%E2%80%93+216%2E)
27. Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for NLP. In Proceedings of ICML, pages 2790–2799. [find](https://scholar.google.com/scholar?q=Neil+Houlsby%2C+Andrei+Giurgiu%2C+Stanislaw+Jastrzebski%2C+Bruna+Morrone%2C+Quentin+de+Laroussilhe%2C+Andrea+Gesmundo%2C+Mona+Attariyan%2C+and+Sylvain+Gelly%2E+2019%2E+Parameter%2Defficient+transfer+learning+for+NLP%2E+In+)
28. Zhengbao Jiang, Frank F. Xu, Jun Araki, and Graham Neubig. 2020. [How can we know what language models know?](https://md.research.elamur.ai/free/en/papers/jiang-2020-how-can-we-know-what/) Transactions of the Association for Computational Linguistics, 8:423–438.
29. Andrej Karpathy, Justin Johnson, and Li Fei-Fei. 2015. [Visualizing and understanding recurrent networks](https://md.research.elamur.ai/free/en/papers/karpathy-2015-visualizing-understanding-recurrent-networks/). arXiv preprint arXiv:1506.02078, pages 818–833.
30. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of ICLR. [find](https://scholar.google.com/scholar?q=Diederik+P%2E+Kingma+and+Jimmy+Ba%2E+2015%2E+Adam%3A+A+method+for+stochastic+optimization%2E+In+Proceedings+of+ICLR%2E)
31. Quoc V. Le, Marc’Aurelio Ranzato, Rajat Monga, Matthieu Devin, Gregory S. Corrado, Kai Chen, Jeffrey Dean, and A. Ng. 2013. Building high-level features using large scale unsupervised learning. In Proceedings of ICASSP, pages 8595–8598. [find](https://scholar.google.com/scholar?q=Quoc+V%2E+Le%2C+Marc%E2%80%99Aurelio+Ranzato%2C+Rajat+Monga%2C+Matthieu+Devin%2C+Gregory+S%2E+Corrado%2C+Kai+Chen%2C+Jeffrey+Dean%2C+and+A%2E+Ng%2E+2013%2E+Building+high%2Dlevel+features+using+large+scale+unsupervised+learning%2E+In+Pro)
32. Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning. In Proceedings of EMNLP, pages 3045– 3059. [find](https://scholar.google.com/scholar?q=Brian+Lester%2C+Rami+Al%2DRfou%2C+and+Noah+Constant%2E+2021%2E+The+power+of+scale+for+parameter%2Defficient+prompt+tuning%2E+In+Proceedings+of+EMNLP%2C+pages+3045%E2%80%93+3059%2E)
33. Quentin Lhoest, Albert Villanova del Moral, Yacine Jernite, Abhishek Thakur, Patrick von Platen, Suraj Patil, Julien Chaumond, Mariama Drame, Julien Plu, Lewis Tunstall, Joe Davison, Mario Šaško, Gunjan Chhablani, Bhavitvya Malik, Simon Brandeis, Teven Le Scao, Victor Sanh, Canwen Xu, Nicolas Patry,… — [Datasets: A Community Library for Natural Language Processing](https://md.research.elamur.ai/free/en/papers/lhoest-2021-datasets-community-library-natural-language/)
34. Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of ACL, pages 4582–4597. [find](https://scholar.google.com/scholar?q=Xiang+Lisa+Li+and+Percy+Liang%2E+2021%2E+Prefix%2Dtuning%3A+Optimizing+continuous+prompts+for+generation%2E+In+Proceedings+of+ACL%2C+pages+4582%E2%80%934597%2E)
35. Nelson F. Liu, Matt Gardner, Yonatan Belinkov, Matthew E. Peters, and Noah A. Smith. 2019a. [Linguistic knowledge and transferability of contextual representations](https://md.research.elamur.ai/free/en/papers/liu-2019-linguistic-knowledge-transferability-contextual-representations/). In Proceedings of NAACL-HLT, pages 1073–1094.
36. Xiao Liu, Kaixuan Ji, Yicheng Fu, Zhengxiao Du, Zhilin Yang, and Jie Tang. 2022. [P-Tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks](https://md.research.elamur.ai/free/en/papers/liu-2021-p-tuning-v2-prompt-tuning/). In Proceedings of ACL.
37. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019b. RoBERTa: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907,11692. [find](https://scholar.google.com/scholar?q=Yinhan+Liu%2C+Myle+Ott%2C+Naman+Goyal%2C+Jingfei+Du%2C+Mandar+Joshi%2C+Danqi+Chen%2C+Omer+Levy%2C+Mike+Lewis%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2019b%2E+RoBERTa%3A+A+robustly+optimized+bert+pretraining+approach%2E+)
38. Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. [Learning word vectors for sentiment analysis](https://md.research.elamur.ai/free/en/papers/maas-2011-learning-word-vectors-sentiment-analysis/). In Proceedings of ACL-HLT, pages 142–150.
39. Alec Radford, Rafal Józefowicz, and Ilya Sutskever. 2017. [Learning to generate reviews and discovering sentiment](https://md.research.elamur.ai/free/en/papers/radford-2017-learning-generate-reviews-discovering-sentiment/). arXiv preprint arXiv:1704.01444.
40. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). Journal of Machine Learning Research, 21:1– 67.
41. Sara Rosenthal, Noura Farra, and Preslav Nakov. 2017. [SemEval-2017 task 4: Sentiment analysis in Twitter](https://md.research.elamur.ai/free/en/papers/rosenthal-2017-semeval-2017-task-4-sentiment/). In Proceedings of SemEval, pages 502–518.
42. Bernardo Rudy, Gordon Fishell, SooHyun Lee, and Jens Hjerling-Leffler. 2011. Three groups of interneurons account for nearly 100% of neocortical gabaergic neurons. Developmental neurobiology, 71(1):45–61. [find](https://scholar.google.com/scholar?q=Bernardo+Rudy%2C+Gordon+Fishell%2C+SooHyun+Lee%2C+and+Jens+Hjerling%2DLeffler%2E+2011%2E+Three+groups+of+interneurons+account+for+nearly+100%25+of+neocortical+gabaergic+neurons%2E+Developmental+neurobiology%2C+71%281%29%3A45)
43. Eran Malach, Gilad Yehudai, Shai Shalev-shwartz, and Ohad Shamir. 2020. Proving the lottery ticket hypothesis: Pruning is all you need. In Proceedings of ICML. [find](https://scholar.google.com/scholar?q=Eran+Malach%2C+Gilad+Yehudai%2C+Shai+Shalev%2Dshwartz%2C+and+Ohad+Shamir%2E+2020%2E+Proving+the+lottery+ticket+hypothesis%3A+Pruning+is+all+you+need%2E+In+Proceedings+of+ICML%2E)
44. Timo Schick and Hinrich Schütze. 2021. [Exploiting cloze-questions for few-shot text classification and natural language inference](https://md.research.elamur.ai/free/en/papers/schick-2021-exploiting-cloze-questions-few-shot/). In Proceedings of EACL, pages 255–269.
45. Paul Michel, Omer Levy, and Graham Neubig. 2019. [Are sixteen heads really better than one?](https://md.research.elamur.ai/free/en/papers/michel-2019-sixteen-heads-really-better-than/) In Proceedings of NeurIPS, pages 14014–14024.
46. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of EMNLP, pages 1631–1642. [find](https://scholar.google.com/scholar?q=Richard+Socher%2C+Alex+Perelygin%2C+Jean+Wu%2C+Jason+Chuang%2C+Christopher+D%2E+Manning%2C+Andrew+Ng%2C+and+Christopher+Potts%2E+2013%2E+Recursive+deep+models+for+semantic+compositionality+over+a+sentiment+treebank%2E+In)
47. Eric Mitchell, Charles Lin, Antoine Bosselut, Chelsea Finn, and Christopher D Manning. 2021. Fast model editing at scale. In Proceedings of ICLR. [find](https://scholar.google.com/scholar?q=Eric+Mitchell%2C+Charles+Lin%2C+Antoine+Bosselut%2C+Chelsea+Finn%2C+and+Christopher+D+Manning%2E+2021%2E+Fast+model+editing+at+scale%2E+In+Proceedings+of+ICLR%2E)
48. Sparsh Mittal, Poonam Rajput, and Sreenivas Subra-money. 2021. A survey of deep learning on CPUs: opportunities and co-optimizations. IEEE Transactions on Neural Networks and Learning Systems, pages 1–21. [find](https://scholar.google.com/scholar?q=Sparsh+Mittal%2C+Poonam+Rajput%2C+and+Sreenivas+Subra%2Dmoney%2E+2021%2E+A+survey+of+deep+learning+on+CPUs%3A+opportunities+and+co%2Doptimizations%2E+IEEE+Transactions+on+Neural+Networks+and+Learning+Systems%2C+pages+1)
49. Ari S Morcos, David GT Barrett, Neil C Rabinowitz, and Matthew Botvinick. 2018. On the importance of single directions for generalization. In Proceedings of ICLR. [find](https://scholar.google.com/scholar?q=Ari+S+Morcos%2C+David+GT+Barrett%2C+Neil+C+Rabinowitz%2C+and+Matthew+Botvinick%2E+2018%2E+On+the+importance+of+single+directions+for+generalization%2E+In+Proceedings+of+ICLR%2E)
50. Jesse Mu and Jacob Andreas. 2020. Compositional explanations of neurons. In Proceedings of NeurIPS, pages 17153–17163. [find](https://scholar.google.com/scholar?q=Jesse+Mu+and+Jacob+Andreas%2E+2020%2E+Compositional+explanations+of+neurons%2E+In+Proceedings+of+NeurIPS%2C+pages+17153%E2%80%9317163%2E)
51. Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. [Language models as knowledge bases?](https://md.research.elamur.ai/free/en/papers/petroni-2019-language-models-as-knowledge-bases/) In Proceedings of EMNLP-IJCNLP, pages 2463–2473.
52. Ofir Press, Noah A. Smith, and Omer Levy. 2020. Improving transformer models by reordering their sublayers. In Proceedings of ACL, pages 2996–3005. [find](https://scholar.google.com/scholar?q=Ofir+Press%2C+Noah+A%2E+Smith%2C+and+Omer+Levy%2E+2020%2E+Improving+transformer+models+by+reordering+their+sublayers%2E+In+Proceedings+of+ACL%2C+pages+2996%E2%80%933005%2E)
53. Guanghui Qin and Jason Eisner. 2021. [Learning how to ask: Querying LMs with mixtures of soft prompts](https://md.research.elamur.ai/free/en/papers/qin-2021-learning-ask-querying-lms-mixtures/). In Proceedings of NAACL-HLT, pages 5203–5212.
54. R Quian Quiroga, Leila Reddy, Gabriel Kreiman, Christof Koch, and Itzhak Fried. 2005. Invariant visual representation by single neurons in the human brain. Nature, 435(7045):1102–1107. [find](https://scholar.google.com/scholar?q=R+Quian+Quiroga%2C+Leila+Reddy%2C+Gabriel+Kreiman%2C+Christof+Koch%2C+and+Itzhak+Fried%2E+2005%2E+Invariant+visual+representation+by+single+neurons+in+the+human+brain%2E+Nature%2C+435%287045%29%3A1102%E2%80%931107%2E)
55. Charles Spearman. 1987. The proof and measurement of association between two things. In Proceedings of AJP, 3/4, pages 441–471. [find](https://scholar.google.com/scholar?q=Charles+Spearman%2E+1987%2E+The+proof+and+measurement+of+association+between+two+things%2E+In+Proceedings+of+AJP%2C+3%2F4%2C+pages+441%E2%80%93471%2E)
56. Yusheng Su, Xiaozhi Wang, Yujia Qin, Chi-Min Chan, Yankai Lin, Zhiyuan Liu, Peng Li, Juanzi Li, Lei Hou, Maosong Sun, et al. 2021. On transferability of prompt tuning for natural language understanding. arXiv preprint arXiv:2111.06719. — [arXiv:2111.06719](https://arxiv.org/abs/2111.06719) [find](https://scholar.google.com/scholar?q=Yusheng+Su%2C+Xiaozhi+Wang%2C+Yujia+Qin%2C+Chi%2DMin+Chan%2C+Yankai+Lin%2C+Zhiyuan+Liu%2C+Peng+Li%2C+Juanzi+Li%2C+Lei+Hou%2C+Maosong+Sun%2C+et+al%2E+2021%2E+On+transferability+of+prompt+tuning+for+natural+language+understandin)
57. Xavier Suau, Luca Zappella, and Nicholas Apostoloff. 2020. Finding experts in transformer models. arXiv preprint arXiv:2005.07647. — [arXiv:2005.07647](https://arxiv.org/abs/2005.07647) [find](https://scholar.google.com/scholar?q=Xavier+Suau%2C+Luca+Zappella%2C+and+Nicholas+Apostoloff%2E+2020%2E+Finding+experts+in+transformer+models%2E+arXiv+preprint+arXiv%3A2005%2E07647%2E+%E2%80%94+%5BarXiv%3A2005%2E07647%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2005%2E07647%29)
58. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Proceedings of NeurIPS, pages 5998– 6008.
59. Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov. 2019. [Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned](https://md.research.elamur.ai/free/en/papers/voita-2019-analyzing-multi-head-self-attention/). In Proceedings of NAACL, pages 5797–5808.
60. Tu Vu, Brian Lester, Noah Constant, Rami Al-Rfou, and Daniel Cer. 2021. [Spot: Better frozen model adaptation through soft prompt transfer](https://md.research.elamur.ai/free/en/papers/vu-2022-spot-better-frozen-model-adaptation/). arXiv preprint arxiv:2110.07904.
61. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019. [GLUE: A multi-task benchmark and analysis platform for natural language understanding](https://md.research.elamur.ai/free/en/papers/wang-2018-glue-multi-task-benchmark-analysis/). In Proceedings of ICLR.
62. Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of NAACL-HLT, pages 1112–1122. [find](https://scholar.google.com/scholar?q=Adina+Williams%2C+Nikita+Nangia%2C+and+Samuel+Bowman%2E+2018%2E+A+broad%2Dcoverage+challenge+corpus+for+sentence+understanding+through+inference%2E+In+Proceedings+of+NAACL%2DHLT%2C+pages+1112%E2%80%931122%2E)
63. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. 2020. [Transformers: State-of-the-art natural language processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/). In Proceedings of EMNLP, pages 38–45.
64. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime G. Carbonell, Ruslan Salakhutdinov, and Quoc V. Le. 2019. [Xlnet: Generalized autoregressive pretraining for language understanding](https://md.research.elamur.ai/free/en/papers/yang-2019-xlnet-generalized-autoregressive-pretraining-language/). In Proceedings of NeurIPS, pages 5754–5764.
65. Matthew D. Zeiler and Rob Fergus. 2014. [Visualizing and understanding convolutional networks](https://md.research.elamur.ai/free/en/papers/zeiler-2013-visualizing-understanding-convolutional-networks/). In Proceedings of ECCV.
66. Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. [Character-level convolutional networks for text classification](https://md.research.elamur.ai/free/en/papers/zhang-2015-character-level-convolutional-networks-text/). In Proceedings of NeurIPS, pages 649–657.
67. Zhengyan Zhang, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, and Jie Zhou. 2021. Moefication: Conditional computation of transformer models for efficient inference. arXiv preprint arXiv:2110.01786. — [arXiv:2110.01786](https://arxiv.org/abs/2110.01786) [find](https://scholar.google.com/scholar?q=Zhengyan+Zhang%2C+Yankai+Lin%2C+Zhiyuan+Liu%2C+Peng+Li%2C+Maosong+Sun%2C+and+Jie+Zhou%2E+2021%2E+Moefication%3A+Conditional+computation+of+transformer+models+for+efficient+inference%2E+arXiv+preprint+arXiv%3A2110%2E01786%2E+)
68. Zexuan Zhong, Dan Friedman, and Danqi Chen. 2021. [Factual probing is MASK: Learning vs. learning to recall](https://md.research.elamur.ai/free/en/papers/zhong-2021-factual-probing-is-mask-learning/). In Proceedings of NAACL, pages 5017–5033.
69. Bolei Zhou, Aditya Khosla, Àgata Lapedriza, Aude Oliva, and Antonio Torralba. 2015. Object detectors emerge in deep scene cnns. In Proceedings of ICLR. [find](https://scholar.google.com/scholar?q=Bolei+Zhou%2C+Aditya+Khosla%2C+%C3%80gata+Lapedriza%2C+Aude+Oliva%2C+and+Antonio+Torralba%2E+2015%2E+Object+detectors+emerge+in+deep+scene+cnns%2E+In+Proceedings+of+ICLR%2E)

*Updated 24 October 2026*