---
title: "WARP: Word-level Adversarial ReProgramming (2021)"
description: "WARP addresses how to adapt one large pretrained language model to many classification tasks… 40 references, cited by 113 papers here."
authors: ["Karen Hambardzumyan", "Hrant Khachatrian", "Jonathan May"]
year: 2021
venue: "arXiv:2101.00121"
source: "https://arxiv.org/abs/2101.00121"
doi: "10.48550/arxiv.2101.00121"
arxiv: "2101.00121"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/hambardzumyan-2021-warp-word-level-adversarial-reprogramming/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/hambardzumyan-2021-warp-word-level-adversarial-reprogramming/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/2/) / WARP: Word-level Adversarial ReProgramming

# WARP: Word-level Adversarial ReProgramming

**Authors:** [Karen Hambardzumyan](https://md.research.elamur.ai/free/en/authors/karen-hambardzumyan/), [Hrant Khachatrian](https://md.research.elamur.ai/free/en/authors/hrant-khachatrian/) and [Jonathan May](https://md.research.elamur.ai/free/en/authors/jonathan-may/) · **Year:** 2021 · **Venue:** arXiv:2101.00121 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2101.00121) · [PDF](https://arxiv.org/pdf/2101.00121) · [DOI 10.48550/arxiv.2101.00121](https://doi.org/10.48550/arxiv.2101.00121) · [arXiv 2101.00121](https://arxiv.org/abs/2101.00121) · [OpenAlex](https://openalex.org/W3119555019)

**Citations:** 227 citations (OpenAlex, 2026-10-14) · [113](https://md.research.elamur.ai/free/en/papers/hambardzumyan-2021-warp-word-level-adversarial-reprogramming/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
WARP addresses how to adapt one large pretrained language model to many classification tasks without storing a separate fine-tuned copy for each task or adding adapter layers inside the network. The authors borrow adversarial reprogramming from computer vision, where padding around an image repurposes a frozen classifier. They insert a few trainable prompt vectors and a [MASK] position around the input text. A frozen masked language model then predicts the class through learned verbalizer embeddings. They used roberta-large for GLUE and albert-xxlarge-v2 for few-shot experiments. Only the prompt and verbalizer vectors are optimized, by gradient descent in the continuous embedding space. With roughly 25K task-specific parameters at most, WARP scores 81.6 on the GLUE test leaderboard, ahead of submissions that train up to a thousand times more parameters. When prompts are initialized from hand-written templates, it learns from only 32 labeled examples and beats GPT-3 on two SuperGLUE tasks. This matters in practice because one shared backbone can serve many tasks, even mixed in a single batch, with very little storage per task.

## Key points
- **Trainable parameters:** only the prompt tokens [P_1]…[P_K] and the class (verbalizer) embeddings [V_1]…[V_C] are learned. All Transformer layers and the original word embeddings stay frozen, so each task needs about 25K stored parameters.
- **GLUE:** WARP reaches 81.6 on the public GLUE leaderboard using roberta-large. It outperforms methods with up to 25M trainable parameters, including adapters, TinyBERT and linear probing.
- **Few-shot:** with manual prompt initialization and albert-xxlarge-v2, WARP beats GPT-3 on two SuperGLUE tasks (RTE and CB) using 32 training samples. Its few-shot results are compared directly with iPET.
- **Versus AutoPrompt:** on the SST-2 dev set, AutoPrompt searches a discrete vocabulary and gets 91.2% with 3–6 tokens. WARP gets 93.8% with a single continuous prompt token, and it also succeeds on RTE and CB, where AutoPrompt fails.
- **Storage and resources:** serving N tasks requires M + N·E(C+K) parameters, compared with M·N for full fine-tuning. The code is built on AllenNLP and huggingface transformers and is released at github.com/YerevaNN/WARP.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Southern California](https://md.research.elamur.ai/free/en/universities/university-of-southern-california/) — University, Los Angeles, United States · 175 works · 37,050 citations
- [Yerevan State University](https://md.research.elamur.ai/free/en/universities/yerevan-state-university/) — University, Yerevan, Armenia · 5 works · 350 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

## Cite

Cite as: Karen Hambardzumyan, Hrant Khachatrian and Jonathan May. 2021. WARP: Word-level Adversarial ReProgramming. arXiv:2101.00121.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{hambardzumyan2021warp,
  title = {{WARP: Word-level Adversarial ReProgramming}},
  author = {Karen Hambardzumyan, Hrant Khachatrian and Jonathan May},
  year = {2021},
  doi = {10.48550/arxiv.2101.00121},
  eprint = {2101.00121},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2101.00121}
}
```

- [← previous SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization](https://md.research.elamur.ai/free/en/papers/liu-2021-simcls-simple-framework-contrastive-learning/)
- [next → Efficient Attentions for Long Document Summarization](https://md.research.elamur.ai/free/en/papers/huang-2021-efficient-attentions-long-document-summarization/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/2/)

## References

40 references, 24 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Mikel Artetxe, Sebastian Ruder, and Dani Yogatama. 2020. [On the cross-lingual transferability of monolingual representations](https://md.research.elamur.ai/free/en/papers/artetxe-2020-cross-lingual-transferability-monolingual-representations/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4623–4637, Online. Association for Computational Linguistics.
2. Roy Bar Haim, Ido Dagan, Bill Dolan, Lisa Ferro, Danilo Giampiccolo, Bernardo Magnini, and Idan Szpektor. 2006. The second PASCAL recognising textual entailment challenge. [find](https://scholar.google.com/scholar?q=Roy+Bar+Haim%2C+Ido+Dagan%2C+Bill+Dolan%2C+Lisa+Ferro%2C+Danilo+Giampiccolo%2C+Bernardo+Magnini%2C+and+Idan+Szpektor%2E+2006%2E+The+second+PASCAL+recognising+textual+entailment+challenge%2E)
3. Luisa Bentivogli, Ido Dagan, Hoa Trang Dang, Danilo Giampiccolo, and Bernardo Magnini. 2009. The fifth PASCAL recognizing textual entailment challenge. [find](https://scholar.google.com/scholar?q=Luisa+Bentivogli%2C+Ido+Dagan%2C+Hoa+Trang+Dang%2C+Danilo+Giampiccolo%2C+and+Bernardo+Magnini%2E+2009%2E+The+fifth+PASCAL+recognizing+textual+entailment+challenge%2E)
4. T. Brown, B. Mann, Nick Ryder, Melanie Subbiah, J. Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, G. Krüger, T. Henighan, R. Child, Aditya Ramesh, D. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen,… — [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
5. Daniel Cer, Mona Diab, Eneko Agirre, Iñigo Lopez-Gazpio, and Lucia Specia. 2017. [SemEval-2017 task 1: Semantic textual similarity multilingual and crosslingual focused evaluation](https://md.research.elamur.ai/free/en/papers/cer-2017-semeval-2017-task-1-semantic/). In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 1–14, Vancouver, Canada.…
6. Kevin Clark, Minh-Thang Luong, Quoc Le, and Christopher D. Manning. 2020. Pre-training transformers as energy-based cloze models. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 285–294, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Kevin+Clark%2C+Minh%2DThang+Luong%2C+Quoc+Le%2C+and+Christopher+D%2E+Manning%2E+2020%2E+Pre%2Dtraining+transformers+as+energy%2Dbased+cloze+models%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natural+)
7. Ido Dagan, Oren Glickman, and Bernardo Magnini. 2006. The PASCAL recognising textual entailment challenge. In Machine learning challenges. evaluating predictive uncertainty, visual object classification, and recognising tectual entailment, pages 177– 190. Springer. [find](https://scholar.google.com/scholar?q=Ido+Dagan%2C+Oren+Glickman%2C+and+Bernardo+Magnini%2E+2006%2E+The+PASCAL+recognising+textual+entailment+challenge%2E+In+Machine+learning+challenges%2E+evaluating+predictive+uncertainty%2C+visual+object+classificati)
8. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
9. William B Dolan and Chris Brockett. 2005. [Automatically constructing a corpus of sentential paraphrases](https://md.research.elamur.ai/free/en/papers/dolan-2005-automatically-constructing-corpus-sentential-paraphrases/). In Proceedings of the International Workshop on Paraphrasing.
10. Gamaleldin F. Elsayed, Ian Goodfellow, and Jascha Sohl-Dickstein. 2019. Adversarial reprogramming of neural networks. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Gamaleldin+F%2E+Elsayed%2C+Ian+Goodfellow%2C+and+Jascha+Sohl%2DDickstein%2E+2019%2E+Adversarial+reprogramming+of+neural+networks%2E+In+International+Conference+on+Learning+Representations%2E)
11. Danilo Giampiccolo, Bernardo Magnini, Ido Dagan, and Bill Dolan. 2007. [The third PASCAL recognizing textual entailment challenge](https://md.research.elamur.ai/free/en/papers/giampiccolo-2007-third-pascal-recognizing-textual-entailment/). In Proceedings of the ACL-PASCAL workshop on textual entailment and paraphrasing, pages 1–9. Association for Computational Linguistics.
12. Thanh-Le Ha, Jan Niehues, and Alexander Waibel. 2016. [Toward multilingual neural machine translation with universal encoder and decoder](https://md.research.elamur.ai/free/en/papers/ha-2016-multilingual-neural-machine-translation-universal/).
13. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2020. [DeBERTa: Decoding-enhanced bert with disentangled attention](https://md.research.elamur.ai/free/en/papers/he-2020-deberta-decoding-enhanced-bert-disentangled/). arXiv preprint arXiv:2006.03654.
14. N. Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, and S. Gelly. 2019. Parameter-efficient transfer learning for nlp. In ICML. [find](https://scholar.google.com/scholar?q=N%2E+Houlsby%2C+Andrei+Giurgiu%2C+Stanislaw+Jastrzebski%2C+Bruna+Morrone%2C+Quentin+de+Laroussilhe%2C+Andrea+Gesmundo%2C+Mona+Attariyan%2C+and+S%2E+Gelly%2E+2019%2E+Parameter%2Defficient+transfer+learning+for+nlp%2E+In+ICML%2E)
15. Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, and Qun Liu. 2020. [TinyBERT: Distilling BERT for natural language understanding](https://md.research.elamur.ai/free/en/papers/jiao-2020-tinybert-distilling-bert-natural-language/). In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4163–4174, Online. Association for Computational Linguis…
16. Melvin Johnson, Mike Schuster, Quoc V. Le, Maxim Krikun, Yonghui Wu, Zhifeng Chen, Nikhil Thorat, Fernanda Viégas, Martin Wattenberg, Greg Corrado, Macduff Hughes, and Jeffrey Dean. 2017. [Google’s multilingual neural machine translation system: Enabling zero-shot translation](https://md.research.elamur.ai/free/en/papers/johnson-2017-google-s-multilingual-neural-machine/). Transactions of the As…
17. Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. [ALBERT: A lite BERT for self-supervised learning of language representations](https://md.research.elamur.ai/free/en/papers/lan-2019-albert-lite-bert-self-supervised/). In International Conference on Learning Representations.
18. Hector J Levesque, Ernest Davis, and Leora Morgenstern. 2011. The Winograd schema challenge. In AAAI Spring Symposium: Logical Formalizations of Commonsense Reasoning, volume 46, page 47. [find](https://scholar.google.com/scholar?q=Hector+J+Levesque%2C+Ernest+Davis%2C+and+Leora+Morgenstern%2E+2011%2E+The+Winograd+schema+challenge%2E+In+AAAI+Spring+Symposium%3A+Logical+Formalizations+of+Commonsense+Reasoning%2C+volume+46%2C+page+47%2E)
19. Xiang Lisa Li and Percy Liang. 2021. Prefix-Tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190. [find](https://scholar.google.com/scholar?q=Xiang+Lisa+Li+and+Percy+Liang%2E+2021%2E+Prefix%2DTuning%3A+Optimizing+continuous+prompts+for+generation%2E+arXiv+preprint+arXiv%3A2101%2E00190%2E)
20. Y. Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, M. Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. [find](https://scholar.google.com/scholar?q=Y%2E+Liu%2C+Myle+Ott%2C+Naman+Goyal%2C+Jingfei+Du%2C+Mandar+Joshi%2C+Danqi+Chen%2C+Omer+Levy%2C+M%2E+Lewis%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2019%2E)
21. [RoBERTa: A robustly optimized BERT pretraining approach](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/). ArXiv, abs/1907.11692.
22. Marie-Catherine de Marneffe, Mandy Simons, and Judith Tonhauser. 2019. [The CommitmentBank: Investigating projection in naturally occurring discourse](https://md.research.elamur.ai/free/en/papers/marneffe-2019-commitmentbank-investigating-projection-naturally/). Proceedings of Sinn und Bedeutung, 23(2):107–124.
23. Paarth Neekhara, Shehzeen Hussain, Shlomo Dubnov, and Farinaz Koushanfar. 2019. Adversarial reprogramming of text classification neural networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language P… [find](https://scholar.google.com/scholar?q=Paarth+Neekhara%2C+Shehzeen+Hussain%2C+Shlomo+Dubnov%2C+and+Farinaz+Koushanfar%2E+2019%2E+Adversarial+reprogramming+of+text+classification+neural+networks%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Met)
24. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. [Deep contextualized word representations](https://md.research.elamur.ai/free/en/papers/peters-2018-deep-contextualized-word-representations/). In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technolog…
25. Matthew E. Peters, Sebastian Ruder, and Noah A. Smith. 2019. To tune or not to tune? adapting pre-trained representations to diverse tasks. In Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019), pages 7–14, Florence, Italy. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Matthew+E%2E+Peters%2C+Sebastian+Ruder%2C+and+Noah+A%2E+Smith%2E+2019%2E+To+tune+or+not+to+tune%3F+adapting+pre%2Dtrained+representations+to+diverse+tasks%2E+In+Proceedings+of+the+4th+Workshop+on+Representation+Learnin)
26. Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. 2021. [AdapterFusion: Non-destructive task composition for transfer learning](https://md.research.elamur.ai/free/en/papers/pfeiffer-2020-adapterfusion-non-destructive-task-composition/). In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 487–5…
27. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Karthik+Narasimhan%2C+Tim+Salimans%2C+and+Ilya+Sutskever%2E+2018%2E+Improving+language+understanding+by+generative+pre%2Dtraining%2E)
28. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+and+Ilya+Sutskever%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E)
29. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. [SQuAD: 100,000+ questions for machine comprehension of text](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383–2392, Austin, Texas. Association for Computational Linguistics.
30. Timo Schick and Hinrich Schütze. 2021a. [Exploiting cloze-questions for few-shot text classification and natural language inference](https://md.research.elamur.ai/free/en/papers/schick-2021-exploiting-cloze-questions-few-shot/). In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 255–269, Online. Association for Com…
31. Timo Schick and Hinrich Schütze. 2021b. [It’s not just size that matters: Small language models are also few-shot learners](https://md.research.elamur.ai/free/en/papers/schick-2021-it-s-not-just-size/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2339–2352, Online. Asso…
32. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. [Controlling politeness in neural machine translation via side constraints](https://md.research.elamur.ai/free/en/papers/sennrich-2016-controlling-politeness-neural-machine-translation/). In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 35–40, San Diego…
33. Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh. 2020a. Auto-Prompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4222–4235, Onl… [find](https://scholar.google.com/scholar?q=Taylor+Shin%2C+Yasaman+Razeghi%2C+Robert+L%2E+Logan+IV%2C+Eric+Wallace%2C+and+Sameer+Singh%2E+2020a%2E+Auto%2DPrompt%3A+Eliciting+Knowledge+from+Language+Models+with+Automatically+Generated+Prompts%2E+In+Proceedings+of+t)
34. Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh. 2020b. Auto-Prompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4222–4235, Onl… [find](https://scholar.google.com/scholar?q=Taylor+Shin%2C+Yasaman+Razeghi%2C+Robert+L%2E+Logan+IV%2C+Eric+Wallace%2C+and+Sameer+Singh%2E+2020b%2E+Auto%2DPrompt%3A+Eliciting+Knowledge+from+Language+Models+with+Automatically+Generated+Prompts%2E+In+Proceedings+of+t)
35. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Ng, and Christopher Potts. 2013. [Recursive deep models for semantic compositionality over a sentiment treebank](https://md.research.elamur.ai/free/en/papers/socher-2013-recursive-deep-models-semantic-compositionality/). In Proceedings of EMNLP, pages 1631–1642.
36. Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019a. [SuperGLUE: A stickier benchmark for general-purpose language understanding systems](https://md.research.elamur.ai/free/en/papers/wang-2019-superglue-stickier-benchmark-general-purpose/). In NeurIPS.
37. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019b. [GLUE: A multi-task benchmark and analysis platform for natural language understanding](https://md.research.elamur.ai/free/en/papers/wang-2018-glue-multi-task-benchmark-analysis/). In International Conference on Learning Representations.
38. Alex Warstadt, Amanpreet Singh, and Samuel R. Bowman. 2019. Neural network acceptability judgments. Transactions of the Association for Computational Linguistics, 7:625–641. [find](https://scholar.google.com/scholar?q=Alex+Warstadt%2C+Amanpreet+Singh%2C+and+Samuel+R%2E+Bowman%2E+2019%2E+Neural+network+acceptability+judgments%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+7%3A625%E2%80%93641%2E)
39. Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. [A broad-coverage challenge corpus for sentence understanding through inference](https://md.research.elamur.ai/free/en/papers/williams-2018-broad-coverage-challenge-corpus-sentence/). In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Pa…
40. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lho… — [Transformers: State-of-the-Art Natural Language Processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/)

*Updated 2 November 2026*