---
title: "The Right Tool for the Job — paper by Schwartz et al. (2020)"
description: "Running large pretrained encoders such as BERT-large in production is costly, because every… 53 references, cited by 66 papers here."
authors: ["Roy Schwartz", "Gabriel Stanovsky", "Swabha Swayamdipta", "Jesse Dodge", "Noah A. Smith"]
year: 2020
venue: "Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics"
source: "https://aclanthology.org/2020.acl-main.593/"
doi: "10.18653/v1/2020.acl-main.593"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/schwartz-2020-right-tool-job-matching-model/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/schwartz-2020-right-tool-job-matching-model/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/4/) / The Right Tool for the Job: Matching Model and Instance Complexities

# The Right Tool for the Job: Matching Model and Instance Complexities

**Authors:** [Roy Schwartz](https://md.research.elamur.ai/free/en/authors/roy-schwartz/), [Gabriel Stanovsky](https://md.research.elamur.ai/free/en/authors/gabriel-stanovsky/), [Swabha Swayamdipta](https://md.research.elamur.ai/free/en/authors/swabha-swayamdipta/), [Jesse Dodge](https://md.research.elamur.ai/free/en/authors/jesse-dodge/) and [Noah A. Smith](https://md.research.elamur.ai/free/en/authors/noah-a-smith/) · **Year:** 2020 · **Venue:** [Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2020](https://md.research.elamur.ai/free/en/venues/acl/2020/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.acl-main.593/) · [PDF](https://aclanthology.org/2020.acl-main.593.pdf) · [DOI 10.18653/v1/2020.acl-main.593](https://doi.org/10.18653/v1/2020.acl-main.593) · [OpenAlex](https://openalex.org/W3016791490)

**Citations:** 128 citations (OpenAlex, 2026-10-14) · [66](https://md.research.elamur.ai/free/en/papers/schwartz-2020-right-tool-job-matching-model/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Running large pretrained encoders such as BERT-large in production is costly, because every incoming instance pays the full inference price. Schwartz et al. address this by noting that inputs differ in difficulty: many can be handled by shallow computation, while only some need the full network. During fine-tuning they attach several lightweight classifiers to intermediate layers of the 24-layer BERT-large and train them jointly by summing their losses. Each classifier sees a learned weighted mix of the layers beneath it. At inference, the classifiers run from shallow to deep. Processing stops at the first one whose confidence, made more reliable through temperature calibration, exceeds a user-chosen threshold. The authors evaluate on three text classification datasets (AG, IMDB, SST) and two NLI benchmarks (SNLI, MNLI). The method gives a better speed/accuracy tradeoff than fixed-depth baselines in nearly all cases and reaches up to fivefold speedups without losing accuracy. It matters because a single trained model, with almost no extra parameters or training cost, can be tuned to fit any inference budget.

## Key points
- The extra output layers add only about 6K–12K parameters, under 0.005% of BERT-large's 335M, and fine-tuning time stays the same as for the standard single-classifier setup.
- Calibration uses a single temperature T, fitted on development data. It does not change predictions and serves only to decide when to exit early.
- The speed/accuracy tradeoff is controlled by one confidence threshold at inference time, so no retraining is needed for each operating point (unlike distillation or pruning).
- The approach complements compression: applying it on top of a distilled BERT (Jiao et al., 2019) improved results further.
- Confidence differs across labels; for example, on AG the shallowest classifier averages 87.8% confidence for tech versus 96.8% for sports. This suggests that calibration holds overall but not for each label. Code: github.com/allenai/sledgehammer.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Carnegie Mellon University](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/) — University, Pittsburgh, United States · 365 works · 158,048 citations
- [University of Washington](https://md.research.elamur.ai/free/en/universities/university-of-washington/) — University, Seattle, United States · 263 works · 180,581 citations
- [Allen Institute for Artificial Intelligence](https://md.research.elamur.ai/free/en/labs/allen-institute-for-artificial-intelligence/) — Research institute, Seattle, United States · 219 works · 89,712 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 1,282 works

## Cite

Cite as: Roy Schwartz, Gabriel Stanovsky, Swabha Swayamdipta, Jesse Dodge and Noah A. Smith. 2020. The Right Tool for the Job: Matching Model and Instance Complexities. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{schwartz2020right,
  title = {{The Right Tool for the Job: Matching Model and Instance Complexities}},
  author = {Roy Schwartz, Gabriel Stanovsky, Swabha Swayamdipta, Jesse Dodge and Noah A. Smith},
  booktitle = {Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.acl-main.593},
  url = {https://doi.org/10.18653/v1/2020.acl-main.593}
}
```

- [← previous End-to-End Slot Alignment and Recognition for Cross-Lingual NLU](https://md.research.elamur.ai/free/en/papers/xu-2020-end-end-slot-alignment-recognition/)
- [next → That is a Known Lie: Detecting Previously Fact-Checked Claims](https://md.research.elamur.ai/free/en/papers/shaar-2020-that-is-known-lie-detecting/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/4/)

## References

53 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Dario Amodei and Danny Hernandez. 2018. AI and compute. Blog post. [find](https://scholar.google.com/scholar?q=Dario+Amodei+and+Danny+Hernandez%2E+2018%2E+AI+and+compute%2E+Blog+post%2E)
2. Ankur Bapna, Naveen Arivazhagan, and Orhan Firat. 2020. Controlling computation versus quality for neural sequence models. arXiv:2002.07106. — [arXiv:2002.07106](https://arxiv.org/abs/2002.07106) [find](https://scholar.google.com/scholar?q=Ankur+Bapna%2C+Naveen+Arivazhagan%2C+and+Orhan+Firat%2E+2020%2E+Controlling+computation+versus+quality+for+neural+sequence+models%2E+arXiv%3A2002%2E07106%2E+%E2%80%94+%5BarXiv%3A2002%2E07106%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2002%2E07106%29)
3. Tolga Bolukbasi, Joseph Wang, Ofer Dekel, and Venkatesh Saligrama. 2017. Adaptive neural networks for efficient inference. In Proc. of ICML. [find](https://scholar.google.com/scholar?q=Tolga+Bolukbasi%2C+Joseph+Wang%2C+Ofer+Dekel%2C+and+Venkatesh+Saligrama%2E+2017%2E+Adaptive+neural+networks+for+efficient+inference%2E+In+Proc%2E+of+ICML%2E)
4. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In Proc. of EMNLP. [find](https://scholar.google.com/scholar?q=Samuel+R%2E+Bowman%2C+Gabor+Angeli%2C+Christopher+Potts%2C+and+Christopher+D%2E+Manning%2E+2015%2E+A+large+annotated+corpus+for+learning+natural+language+inference%2E+In+Proc%2E+of+EMNLP%2E)
5. Raj Dabre, Raphael Rubino, and Atsushi Fujita. 2020. Balancing cost and benefit with tied-multi transformers. arXiv:2002.08614. [find](https://scholar.google.com/scholar?q=Raj+Dabre%2C+Raphael+Rubino%2C+and+Atsushi+Fujita%2E+2020%2E+Balancing+cost+and+benefit+with+tied%2Dmulti+transformers%2E+arXiv%3A2002%2E08614%2E)
6. Ido Dagan, Oren Glickman, and Bernardo Magnini. 2005. The PASCAL recognising textual entailment challenge. In Proc. of MLCW. [find](https://scholar.google.com/scholar?q=Ido+Dagan%2C+Oren+Glickman%2C+and+Bernardo+Magnini%2E+2005%2E+The+PASCAL+recognising+textual+entailment+challenge%2E+In+Proc%2E+of+MLCW%2E)
7. Morris H. DeGroot and Stephen E. Fienberg. 1983. The comparison and evaluation of forecasters. Journal of the Royal Statistical Society: Series D (The Statistician), 32(1-2):12–22. [find](https://scholar.google.com/scholar?q=Morris+H%2E+DeGroot+and+Stephen+E%2E+Fienberg%2E+1983%2E+The+comparison+and+evaluation+of+forecasters%2E+Journal+of+the+Royal+Statistical+Society%3A+Series+D+%28The+Statistician%29%2C+32%281%2D2%29%3A12%E2%80%9322%2E)
8. Yoav Goldberg and Michael Elhadad. 2010. An efficient algorithm for easy-first non-directional dependency parsing. In Proc. of NAACL. [find](https://scholar.google.com/scholar?q=Yoav+Goldberg+and+Michael+Elhadad%2E+2010%2E+An+efficient+algorithm+for+easy%2Dfirst+non%2Ddirectional+dependency+parsing%2E+In+Proc%2E+of+NAACL%2E)
9. Alex Graves. 2016. Adaptive computation time for recurrent neural networks. arXiv:1603.08983. [find](https://scholar.google.com/scholar?q=Alex+Graves%2E+2016%2E+Adaptive+computation+time+for+recurrent+neural+networks%2E+arXiv%3A1603%2E08983%2E)
10. Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. 2017. On calibration of modern neural networks. In Proc. of ICML. [find](https://scholar.google.com/scholar?q=Chuan+Guo%2C+Geoff+Pleiss%2C+Yu+Sun%2C+and+Kilian+Q%2E+Weinberger%2E+2017%2E+On+calibration+of+modern+neural+networks%2E+In+Proc%2E+of+ICML%2E)
11. Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith. 2018. Annotation artifacts in natural language inference data. In Proc. of NAACL. [find](https://scholar.google.com/scholar?q=Suchin+Gururangan%2C+Swabha+Swayamdipta%2C+Omer+Levy%2C+Roy+Schwartz%2C+Samuel+Bowman%2C+and+Noah+A%2E+Smith%2E+2018%2E+Annotation+artifacts+in+natural+language+inference+data%2E+In+Proc%2E+of+NAACL%2E)
12. Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2014. [Distilling the knowledge in a neural network](https://md.research.elamur.ai/free/en/papers/hinton-2015-distilling-knowledge-neural-network/). In Proc. of NeurIPS Deep Learning Workshop.
13. Gao Huang, Danlu Chen, Tianhong Li, Felix Wu, Laurens van der Maaten, and Kilian Q. Weinberger. 2018. Multi-scale dense networks for resource efficient image classification. In Proc. of ICLR. [find](https://scholar.google.com/scholar?q=Gao+Huang%2C+Danlu+Chen%2C+Tianhong+Li%2C+Felix+Wu%2C+Laurens+van+der+Maaten%2C+and+Kilian+Q%2E+Weinberger%2E+2018%2E+Multi%2Dscale+dense+networks+for+resource+efficient+image+classification%2E+In+Proc%2E+of+ICLR%2E)
14. Heinrich Jiang, Been Kim, Melody Y. Guan, and Maya Gupta. 2018. To trust or not to trust a classifier. In Proc. of NeurIPS. [find](https://scholar.google.com/scholar?q=Heinrich+Jiang%2C+Been+Kim%2C+Melody+Y%2E+Guan%2C+and+Maya+Gupta%2E+2018%2E+To+trust+or+not+to+trust+a+classifier%2E+In+Proc%2E+of+NeurIPS%2E)
15. Shrey Desai and Greg Durrett. 2020. Calibration of pre-trained transformers. arXiv:2003.07892. [find](https://scholar.google.com/scholar?q=Shrey+Desai+and+Greg+Durrett%2E+2020%2E+Calibration+of+pre%2Dtrained+transformers%2E+arXiv%3A2003%2E07892%2E)
16. Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, and Qun Liu. 2019. TinyBERT: Distilling BERT for natural language understanding. arXiv:1909.10351. [find](https://scholar.google.com/scholar?q=Xiaoqi+Jiao%2C+Yichun+Yin%2C+Lifeng+Shang%2C+Xin+Jiang%2C+Xiao+Chen%2C+Linlin+Li%2C+Fang+Wang%2C+and+Qun+Liu%2E+2019%2E+TinyBERT%3A+Distilling+BERT+for+natural+language+understanding%2E+arXiv%3A1909%2E10351%2E)
17. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proc. of NAACL.
18. Yoon Kim and Alexander M. Rush. 2016. [Sequence-level knowledge distillation](https://md.research.elamur.ai/free/en/papers/kim-2016-sequence-level-knowledge-distillation/). In Proc. of EMNLP.
19. Jesse Dodge, Suchin Gururangan, Dallas Card, Roy Schwartz, and Noah A. Smith. 2019a. Show your work: Improved reporting of experimental results. In Proc. of EMNLP. [find](https://scholar.google.com/scholar?q=Jesse+Dodge%2C+Suchin+Gururangan%2C+Dallas+Card%2C+Roy+Schwartz%2C+and+Noah+A%2E+Smith%2E+2019a%2E+Show+your+work%3A+Improved+reporting+of+experimental+results%2E+In+Proc%2E+of+EMNLP%2E)
20. Jesse Dodge, Roy Schwartz, Hao Peng, and Noah A. Smith. 2019b. RNN architecture learning with sparse regularization. In Proc. of EMNLP. [find](https://scholar.google.com/scholar?q=Jesse+Dodge%2C+Roy+Schwartz%2C+Hao+Peng%2C+and+Noah+A%2E+Smith%2E+2019b%2E+RNN+architecture+learning+with+sparse+regularization%2E+In+Proc%2E+of+EMNLP%2E)
21. Jason Eisner and Giorgio Satta. 1999. [Efficient parsing for bilexical context-free grammars and head automaton grammars](https://md.research.elamur.ai/free/en/papers/eisner-1999-efficient-parsing-bilexical-context-free/). In Proc. of ACL.
22. Maha Elbayad, , Jiatao Gu, Edouard Grave, and Michael Auli. 2020. Depth-adaptive transformer. In Proc. of ICLR. [find](https://scholar.google.com/scholar?q=Maha+Elbayad%2C+%2C+Jiatao+Gu%2C+Edouard+Grave%2C+and+Michael+Auli%2E+2020%2E+Depth%2Dadaptive+transformer%2E+In+Proc%2E+of+ICLR%2E)
23. Angela Fan, Edouard Grave, and Armand Joulin. 2020. [Reducing transformer depth on demand with structured dropout](https://md.research.elamur.ai/free/en/papers/fan-2019-reducing-transformer-depth-demand-structured/). In Proc. of ICLR.
24. Trevor Gale, Erich Elsen, and Sara Hooker. 2019. The state of sparsity in deep neural networks. arXiv:1902.09574. — [arXiv:1902.09574](https://arxiv.org/abs/1902.09574) [find](https://scholar.google.com/scholar?q=Trevor+Gale%2C+Erich+Elsen%2C+and+Sara+Hooker%2E+2019%2E+The+state+of+sparsity+in+deep+neural+networks%2E+arXiv%3A1902%2E09574%2E+%E2%80%94+%5BarXiv%3A1902%2E09574%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1902%2E09574%29)
25. Matt Gardner, Joel Grus, Mark Neumann, Oyvind Tafjord, Pradeep Dasigi, Nelson F. Liu, Matthew Peters, Michael Schmitz, and Luke Zettlemoyer. 2018. [AllenNLP: A deep semantic natural language processing platform](https://md.research.elamur.ai/free/en/papers/gardner-2018-allennlp-deep-semantic-natural-language/). In Proc. of NLP-OSS.
26. Ludmila I. Kuncheva and Christopher J. Whitaker. 2003. Measures of diversity in classifier ensembles and their relationship with the ensemble accuracy. Machine learning, 51(2):181–207. [find](https://scholar.google.com/scholar?q=Ludmila+I%2E+Kuncheva+and+Christopher+J%2E+Whitaker%2E+2003%2E+Measures+of+diversity+in+classifier+ensembles+and+their+relationship+with+the+ensemble+accuracy%2E+Machine+learning%2C+51%282%29%3A181%E2%80%93207%2E)
27. Yann LeCun, John S. Denker, and Sara A. Solla. 1990. Optimal brain damage. In Proc. of NeurIPS. [find](https://scholar.google.com/scholar?q=Yann+LeCun%2C+John+S%2E+Denker%2C+and+Sara+A%2E+Solla%2E+1990%2E+Optimal+brain+damage%2E+In+Proc%2E+of+NeurIPS%2E)
28. Liyuan Liu, Xiang Ren, Jingbo Shang, Xiaotao Gu, Jian Peng, and Jiawei Han. 2018. Efficient contextualized representation: Language model pruning for sequence labeling. In Proc. of EMNLP. [find](https://scholar.google.com/scholar?q=Liyuan+Liu%2C+Xiang+Ren%2C+Jingbo+Shang%2C+Xiaotao+Gu%2C+Jian+Peng%2C+and+Jiawei+Han%2E+2018%2E+Efficient+contextualized+representation%3A+Language+model+pruning+for+sequence+labeling%2E+In+Proc%2E+of+EMNLP%2E)
29. Weijie Liu, Peng Zhou, Zhe Zhao, Zhiruo Wang, Haotang Deng, and Qi Ju. 2020. [FastBERT: a self-distilling BERT with adaptive inference time](https://md.research.elamur.ai/free/en/papers/liu-2020-fastbert-self-distilling-bert-adaptive/). In Proc. of ACL.
30. Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. [Learning word vectors for sentiment analysis](https://md.research.elamur.ai/free/en/papers/maas-2011-learning-word-vectors-sentiment-analysis/). In Proc. of ACL.
31. Paul Michel, Omer Levy, and Graham Neubig. 2019. [Are sixteen heads really better than one?](https://md.research.elamur.ai/free/en/papers/michel-2019-sixteen-heads-really-better-than/) In Proc. of NeurIPS.
32. Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. [Deep contextualized word representations](https://md.research.elamur.ai/free/en/papers/peters-2018-deep-contextualized-word-representations/). In Proc. of NAACL.
33. Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, and Kilian Q. Weinberger. 2017. On fairness and calibration. In Proc. of NeurIPS. [find](https://scholar.google.com/scholar?q=Geoff+Pleiss%2C+Manish+Raghavan%2C+Felix+Wu%2C+Jon+Kleinberg%2C+and+Kilian+Q%2E+Weinberger%2E+2017%2E+On+fairness+and+calibration%2E+In+Proc%2E+of+NeurIPS%2E)
34. Adam Poliak, Jason Naradowsky, Aparajita Haldar, Rachel Rudinger, and Benjamin Van Durme. 2018. Hypothesis Only Baselines in Natural Language Inference. In Proc. of ∗ SEM. [find](https://scholar.google.com/scholar?q=Adam+Poliak%2C+Jason+Naradowsky%2C+Aparajita+Haldar%2C+Rachel+Rudinger%2C+and+Benjamin+Van+Durme%2E+2018%2E+Hypothesis+Only+Baselines+in+Natural+Language+Inference%2E+In+Proc%2E+of+%E2%88%97+SEM%2E)
35. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+and+Ilya+Sutskever%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E+OpenAI+Blog%2E)
36. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2019. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). arXiv:1910.10683.
37. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2019. WinoGrande: An adversarial winograd schema challenge at scale. arXiv:1907.10641. [find](https://scholar.google.com/scholar?q=Keisuke+Sakaguchi%2C+Ronan+Le+Bras%2C+Chandra+Bhagavatula%2C+and+Yejin+Choi%2E+2019%2E+WinoGrande%3A+An+adversarial+winograd+schema+challenge+at+scale%2E+arXiv%3A1907%2E10641%2E)
38. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. In Proc. of EMC2. [find](https://scholar.google.com/scholar?q=Victor+Sanh%2C+Lysandre+Debut%2C+Julien+Chaumond%2C+and+Thomas+Wolf%2E+2019%2E+DistilBERT%2C+a+distilled+version+of+BERT%3A+smaller%2C+faster%2C+cheaper+and+lighter%2E+In+Proc%2E+of+EMC2%2E)
39. Roy Schwartz, Jesse Dodge, Noah A. Smith, and Oren Etzioni. 2019. [Green AI](https://md.research.elamur.ai/free/en/papers/schwartz-2019-green-ai/). arXiv:1907.10597.
40. Minjoon Seo, Sewon Min, Ali Farhadi, and Hannaneh Hajishirzi. 2018. Neural speed reading via skim-RNN. In Proc. of ICLR. [find](https://scholar.google.com/scholar?q=Minjoon+Seo%2C+Sewon+Min%2C+Ali+Farhadi%2C+and+Hannaneh+Hajishirzi%2E+2018%2E+Neural+speed+reading+via+skim%2DRNN%2E+In+Proc%2E+of+ICLR%2E)
41. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Proc. of NeurIPS.
42. Elena Voita, David Talbot, Fedor Moiseev, Rico Sennrich, and Ivan Titov. 2019. [Analyzing multi-head self-attention: Specialized heads do the heavy lifting, the rest can be pruned](https://md.research.elamur.ai/free/en/papers/voita-2019-analyzing-multi-head-self-attention/). In Proc. of ACL.
43. Xin Wang, Fisher Yu, Zi-Yi Dou, Trevor Darrell, and Joseph E. Gonzalez. 2018. SkipNet: Learning dynamic routing in convolutional networks. In Proc. of ECCV. [find](https://scholar.google.com/scholar?q=Xin+Wang%2C+Fisher+Yu%2C+Zi%2DYi+Dou%2C+Trevor+Darrell%2C+and+Joseph+E%2E+Gonzalez%2E+2018%2E+SkipNet%3A+Learning+dynamic+routing+in+convolutional+networks%2E+In+Proc%2E+of+ECCV%2E)
44. Adina Williams, Nikita Nangia, and Samuel R Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proc. of NAACL. [find](https://scholar.google.com/scholar?q=Adina+Williams%2C+Nikita+Nangia%2C+and+Samuel+R+Bowman%2E+2018%2E+A+broad%2Dcoverage+challenge+corpus+for+sentence+understanding+through+inference%2E+In+Proc%2E+of+NAACL%2E)
45. David H. Wolpert. 1992. Stacked generalization. Neural Networks, 5:241–259. [find](https://scholar.google.com/scholar?q=David+H%2E+Wolpert%2E+1992%2E+Stacked+generalization%2E+Neural+Networks%2C+5%3A241%E2%80%93259%2E)
46. Krzysztof Wróbel, Marcin Pietroń, Maciej Wielgosz, Michał Karwatowski, and Kazimierz Wiatr. 2018. Convolutional neural network compression for natural language processing. arXiv:1805.10796. — [arXiv:1805.10796](https://arxiv.org/abs/1805.10796) [find](https://scholar.google.com/scholar?q=Krzysztof+Wro%CC%81bel%2C+Marcin+Pietron%CC%81%2C+Maciej+Wielgosz%2C+Micha%C5%82+Karwatowski%2C+and+Kazimierz+Wiatr%2E+2018%2E+Convolutional+neural+network+compression+for+natural+language+processing%2E+arXiv%3A1805%2E10796%2E+%E2%80%94+%5BarXiv)
47. Ofir Zafrir, Guy Boudoukh, Peter Izsak, and Moshe Wasserblat. 2019. Q8BERT: Quantized 8bit BERT. In Proc. of EMC2. [find](https://scholar.google.com/scholar?q=Ofir+Zafrir%2C+Guy+Boudoukh%2C+Peter+Izsak%2C+and+Moshe+Wasserblat%2E+2019%2E+Q8BERT%3A+Quantized+8bit+BERT%2E+In+Proc%2E+of+EMC2%2E)
48. Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. [Character-level convolutional networks for text classification](https://md.research.elamur.ai/free/en/papers/zhang-2015-character-level-convolutional-networks-text/). In Proc. of NeurIPS.
49. Sheng Shen, Zhen Dong, Jiayu Ye, Linjian Ma, Zhewei Yao, Amir Gholami, Michael W. Mahoney, and Kurt Keutzer. 2019. Q-BERT: Hessian based ultra low precision quantization of BERT. arXiv:1909.05840. — [arXiv:1909.05840](https://arxiv.org/abs/1909.05840) [find](https://scholar.google.com/scholar?q=Sheng+Shen%2C+Zhen+Dong%2C+Jiayu+Ye%2C+Linjian+Ma%2C+Zhewei+Yao%2C+Amir+Gholami%2C+Michael+W%2E+Mahoney%2C+and+Kurt+Keutzer%2E+2019%2E+Q%2DBERT%3A+Hessian+based+ultra+low+precision+quantization+of+BERT%2E+arXiv%3A1909%2E05840%2E+%E2%80%94+%5B)
50. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proc. of EMNLP. [find](https://scholar.google.com/scholar?q=Richard+Socher%2C+Alex+Perelygin%2C+Jean+Wu%2C+Jason+Chuang%2C+Christopher+D%2E+Manning%2C+Andrew+Ng%2C+and+Christopher+Potts%2E+2013%2E+Recursive+deep+models+for+semantic+compositionality+over+a+sentiment+treebank%2E+In)
51. Swabha Swayamdipta, Ankur P. Parikh, and Tom Kwiatkowski. 2018. Multi-mention learning for reading comprehension with neural cascades. In Proc. of ICLR. [find](https://scholar.google.com/scholar?q=Swabha+Swayamdipta%2C+Ankur+P%2E+Parikh%2C+and+Tom+Kwiatkowski%2E+2018%2E+Multi%2Dmention+learning+for+reading+comprehension+with+neural+cascades%2E+In+Proc%2E+of+ICLR%2E)
52. Raphael Tang, Yao Lu, Linqing Liu, Lili Mou, Olga Vechtomova, and Jimmy Lin. 2019. Distilling task-specific knowledge from BERT into simple neural networks. arXiv:1903.12136. [find](https://scholar.google.com/scholar?q=Raphael+Tang%2C+Yao+Lu%2C+Linqing+Liu%2C+Lili+Mou%2C+Olga+Vechtomova%2C+and+Jimmy+Lin%2E+2019%2E+Distilling+task%2Dspecific+knowledge+from+BERT+into+simple+neural+networks%2E+arXiv%3A1903%2E12136%2E)
53. Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, and Geoffrey J. Gordon. 2019. An empirical study of example forgetting during deep neural network learning. In Proc. of ICLR. [find](https://scholar.google.com/scholar?q=Mariya+Toneva%2C+Alessandro+Sordoni%2C+Remi+Tachet+des+Combes%2C+Adam+Trischler%2C+Yoshua+Bengio%2C+and+Geoffrey+J%2E+Gordon%2E+2019%2E+An+empirical+study+of+example+forgetting+during+deep+neural+network+learning%2E+In)

*Updated 24 October 2026*