---
title: "ZeroQuant: Efficient and Affordable Post-Training — Yao 2022"
description: "Serving very large Transformer language models takes a lot of memory and compute."
authors: ["Zhewei Yao", "Reza Yazdani Aminabadi", "Minjia Zhang", "Xiaoxia Wu", "Conglong Li", "Yuxiong He"]
year: 2022
venue: "arXiv:2206.01861"
source: "https://arxiv.org/abs/2206.01861"
doi: "10.48550/arxiv.2206.01861"
arxiv: "2206.01861"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/yao-2022-zeroquant-efficient-affordable-post-training/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/yao-2022-zeroquant-efficient-affordable-post-training/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/5/) / ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers

# ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers

**Authors:** [Zhewei Yao](https://md.research.elamur.ai/free/en/authors/zhewei-yao/), [Reza Yazdani Aminabadi](https://md.research.elamur.ai/free/en/authors/reza-yazdani-aminabadi/), [Minjia Zhang](https://md.research.elamur.ai/free/en/authors/minjia-zhang/), Xiaoxia Wu, [Conglong Li](https://md.research.elamur.ai/free/en/authors/conglong-li/), [Yuxiong He](https://md.research.elamur.ai/free/en/authors/yuxiong-he/) · **Year:** 2022 · **Venue:** arXiv:2206.01861 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2206.01861) · [PDF](https://arxiv.org/pdf/2206.01861) · [DOI 10.48550/arxiv.2206.01861](https://doi.org/10.48550/arxiv.2206.01861) · [arXiv 2206.01861](https://arxiv.org/abs/2206.01861) · [OpenAlex](https://openalex.org/W7133231030)

**Citations:** 104 citations (openalex-snapshot · 2026-10-14) · [197](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Serving very large Transformer language models takes a lot of memory and compute. Quantization-aware training needs the original data and a full training run, which is often impractical. ZeroQuant, from Microsoft's DeepSpeed team, is a post-training quantization pipeline for BERT and GPT-3-style models. It has three parts:

- **Fine-grained, GPU-friendly quantization.** Weights use group-wise scaling and activations use per-token scaling. This responds to the large range differences the authors observed across tokens and weight rows.
- **Layer-by-layer knowledge distillation (LKD).** Each quantized layer is trained separately against the full-precision teacher. Only one extra layer occupies device memory, and no labels are needed.
- **An optimized inference backend.** It removes the overhead of quantize/dequantize operations on INT8 Tensor cores.

With INT8 weights and activations and no retraining, accuracy stays close to FP16 while latency drops severalfold. With LKD, a mixed INT4/INT8 setup reduces the memory footprint about threefold. The work shows that compressing multi-billion-parameter models can be affordable, even without the original training data.

## Key points
- INT8 ZeroQuant runs up to 5.19x faster than FP16 on BERT base and 4.16x faster on GPT-3 350M (A100 GPUs). GPT-J 6B gets a 3.67x speedup with near-identical perplexity.
- For GPT-NeoX 20B, the self-attention input is kept in FP16 and the rest is INT8 (W8A8/16). This cuts hardware from 2 A100s to 1 and latency from 65 ms to 25 ms, about 5.2x better efficiency.
- In the W4/8A8 scheme, attention weights are INT8 and feed-forward weights are INT4. LKD finishes in about 33 s for BERT base and about 10 minutes for BERT large.
- Ablation on GLUE (BERT large, W4/8A8), average score:
  - plain PTQ: 35.85
  - adding group-wise weights: 66.52
  - adding token-wise activations: 81.06
  - adding LKD: 81.62
- LKD does not need the original data. For GPT-3 350M, Wikitext-2 perplexity falls from 92.1 (no LKD) to:
  - 40.6 with random tokens
  - 30.4 with Huggingface Wikipedia
  - 31.1 with the original PILE

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Computing & GPUs](https://md.research.elamur.ai/free/en/shelves/computing/)

## Cited by

Cited by

197 papers of our library cite it (of the 140673 whose bibliographies we read); 8 of them have a page here, 189 not yet.

- [Challenges and Applications of Large Language Models](https://md.research.elamur.ai/free/en/papers/kaddour-2023-challenges-applications-large-language-models/)
- [SqueezeLLM: Dense-and-Sparse Quantization](https://md.research.elamur.ai/free/en/papers/kim-2023-squeezellm-dense-sparse-quantization/)
- [LLM-QAT: Data-Free Quantization Aware Training for Large Language Models](https://md.research.elamur.ai/free/en/papers/liu-2024-llm-qat-data-free-quantization/)
- [LLM-Pruner: On the Structural Pruning of Large Language Models](https://md.research.elamur.ai/free/en/papers/ma-2023-llm-pruner-structural-pruning-large/)
- [FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU](https://md.research.elamur.ai/free/en/papers/sheng-2023-flexgen-high-throughput-generative-inference/)
- [Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding](https://md.research.elamur.ai/free/en/papers/zhang-2024-draft-verify-lossless-large-language/)
- [A Survey of Large Language Models](https://md.research.elamur.ai/free/en/papers/zhao-2023-survey-large-language-models/)
- [A Survey on Model Compression for Large Language Models](https://md.research.elamur.ai/free/en/papers/zhu-2024-survey-model-compression-large-language/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{yao2022zeroquant,
  title = {{ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers}},
  author = {Yao, Zhewei and Aminabadi, Reza Yazdani and Zhang, Minjia and Wu, Xiaoxia and Li, Conglong and He, Yuxiong},
  year = {2022},
  doi = {10.48550/arxiv.2206.01861},
  eprint = {2206.01861},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2206.01861}
}
```

- [← previous ZeroGen: Efficient Zero-shot Learning via Dataset Generation](https://md.research.elamur.ai/free/en/papers/ye-2022-zerogen-efficient-zero-shot-learning/)
- [next → ÚFAL CorPipe at CRAC 2022: Effectivity of Multilingual Models for Coreference Resolution](https://md.research.elamur.ai/free/en/papers/straka-2022-fal-corpipe-crac-2022-effectivity/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/5/)

## References

79 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ardavan Afshar, Ioakeim Perros, Evangelos E Papalexakis, Elizabeth Searles, Joyce Ho, and Jimeng Sun. Copa: Constrained parafac2 for sparse & large datasets. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management, pages 793–802, 2018. [find](https://scholar.google.com/scholar?q=Ardavan+Afshar%2C+Ioakeim+Perros%2C+Evangelos+E+Papalexakis%2C+Elizabeth+Searles%2C+Joyce+Ho%2C+and+Jimeng+Sun%2E+Copa%3A+Constrained+parafac2+for+sparse+%26+large+datasets%2E+In+Proceedings+of+the+27th+ACM+Internation)
2. Haoli Bai, Wei Zhang, Lu Hou, Lifeng Shang, Jing Jin, Xin Jiang, Qun Liu, Michael Lyu, and Irwin King. Binarybert: Pushing the limit of bert quantization. arXiv preprint arXiv:2012.15701, 2020. [find](https://scholar.google.com/scholar?q=Haoli+Bai%2C+Wei+Zhang%2C+Lu+Hou%2C+Lifeng+Shang%2C+Jing+Jin%2C+Xin+Jiang%2C+Qun+Liu%2C+Michael+Lyu%2C+and+Irwin+King%2E+Binarybert%3A+Pushing+the+limit+of+bert+quantization%2E+arXiv+preprint+arXiv%3A2012%2E15701%2C+2020%2E)
3. Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. Semantic parsing on Freebase from question-answer pairs. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1533–1544, Seattle, Washington, USA, October 2013. Association for Computational Lingu… [find](https://scholar.google.com/scholar?q=Jonathan+Berant%2C+Andrew+Chou%2C+Roy+Frostig%2C+and+Percy+Liang%2E+Semantic+parsing+on+Freebase+from+question%2Danswer+pairs%2E+In+Proceedings+of+the+2013+Conference+on+Empirical+Methods+in+Natural+Language+Proc)
4. Sid Black, Stella Biderman, Alex Andonian, Quentin Anthony, Preetham Gali, Leo Gao, Eric Hallahan, Josh Levy-Kramer, Connor Leahy, Lucas Nestler, Kip Parker, Jason Phang, Michael Pieler, Shivanshu Purohit, Tri Songz, Phil Wang, and Samuel Weinbach. GPT-NeoX: Large scale autoregressive language model… [find](https://scholar.google.com/scholar?q=Sid+Black%2C+Stella+Biderman%2C+Alex+Andonian%2C+Quentin+Anthony%2C+Preetham+Gali%2C+Leo+Gao%2C+Eric+Hallahan%2C+Josh+Levy%2DKramer%2C+Connor+Leahy%2C+Lucas+Nestler%2C+Kip+Parker%2C+Jason+Phang%2C+Michael+Pieler%2C+Shivanshu+Pur)
5. NVIDIA blog. CUTLASS: Fast Linear Algebra in CUDA C++. [https://developer.nvidia.com/blo](https://developer.nvidia.com/blo) g/cutlass-linear-algebra-cuda/, December 2017. [↗](https://developer.nvidia.com/blo)
6. Yelysei Bondarenko, Markus Nagel, and Tijmen Blankevoort. Understanding and overcoming the challenges of efficient transformer quantization. arXiv preprint arXiv:2109.12948, 2021. [find](https://scholar.google.com/scholar?q=Yelysei+Bondarenko%2C+Markus+Nagel%2C+and+Tijmen+Blankevoort%2E+Understanding+and+overcoming+the+challenges+of+efficient+transformer+quantization%2E+arXiv+preprint+arXiv%3A2109%2E12948%2C+2021%2E)
7. Michael Boratko, Harshit Padigela, Divyendra Mikkilineni, Pritish Yuvraj, Rajarshi Das, Andrew McCallum, Maria Chang, Achille Fokoue-Nkoutche, Pavan Kapanipathi, Nicholas Mattei, et al. A systematic classification of knowledge, reasoning, and context within the arc dataset. arXiv preprint arXiv:1806… [find](https://scholar.google.com/scholar?q=Michael+Boratko%2C+Harshit+Padigela%2C+Divyendra+Mikkilineni%2C+Pritish+Yuvraj%2C+Rajarshi+Das%2C+Andrew+McCallum%2C+Maria+Chang%2C+Achille+Fokoue%2DNkoutche%2C+Pavan+Kapanipathi%2C+Nicholas+Mattei%2C+et+al%2E+A+systematic+c)
8. Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. arXiv preprint arXiv:2005.14165, 2020. [find](https://scholar.google.com/scholar?q=Tom+B+Brown%2C+Benjamin+Mann%2C+Nick+Ryder%2C+Melanie+Subbiah%2C+Jared+Kaplan%2C+Prafulla+Dhariwal%2C+Arvind+Neelakantan%2C+Pranav+Shyam%2C+Girish+Sastry%2C+Amanda+Askell%2C+et+al%2E+Language+models+are+few%2Dshot+learners%2E+)
9. Yaohui Cai, Zhewei Yao, Zhen Dong, Amir Gholami, Michael W Mahoney, and Kurt Keutzer. Zeroq: A novel zero shot quantization framework. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13169–13178, 2020. [find](https://scholar.google.com/scholar?q=Yaohui+Cai%2C+Zhewei+Yao%2C+Zhen+Dong%2C+Amir+Gholami%2C+Michael+W+Mahoney%2C+and+Kurt+Keutzer%2E+Zeroq%3A+A+novel+zero+shot+quantization+framework%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Computer+Vision+and+)
10. Daniel Cer, Mona Diab, Eneko Agirre, Inigo Lopez-Gazpio, and Lucia Specia. Semeval-2017 task 1: Semantic textual similarity-multilingual and cross-lingual focused evaluation. arXiv preprint arXiv:1708.00055, 2017. [find](https://scholar.google.com/scholar?q=Daniel+Cer%2C+Mona+Diab%2C+Eneko+Agirre%2C+Inigo+Lopez%2DGazpio%2C+and+Lucia+Specia%2E+Semeval%2D2017+task+1%3A+Semantic+textual+similarity%2Dmultilingual+and+cross%2Dlingual+focused+evaluation%2E+arXiv+preprint+arXiv%3A1708)
11. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311, 2022. [find](https://scholar.google.com/scholar?q=Aakanksha+Chowdhery%2C+Sharan+Narang%2C+Jacob+Devlin%2C+Maarten+Bosma%2C+Gaurav+Mishra%2C+Adam+Roberts%2C+Paul+Barham%2C+Hyung+Won+Chung%2C+Charles+Sutton%2C+Sebastian+Gehrmann%2C+et+al%2E+Palm%3A+Scaling+language+modeling+w)
12. Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. Boolq: Exploring the surprising difficulty of natural yes/no questions. arXiv preprint arXiv:1905.10044, 2019. [find](https://scholar.google.com/scholar?q=Christopher+Clark%2C+Kenton+Lee%2C+Ming%2DWei+Chang%2C+Tom+Kwiatkowski%2C+Michael+Collins%2C+and+Kristina+Toutanova%2E+Boolq%3A+Exploring+the+surprising+difficulty+of+natural+yes%2Fno+questions%2E+arXiv+preprint+arXiv%3A19) [our card](https://md.research.elamur.ai/free/en/papers/clark-2019-boolq-exploring-surprising-difficulty-natural/)
13. Ido Dagan, Dan Roth, Mark Sammons, and Fabio Massimo Zanzotto. Recognizing textual entailment: Models and applications. Synthesis Lectures on Human Language Technologies, 6(4):1–220, 2013. [find](https://scholar.google.com/scholar?q=Ido+Dagan%2C+Dan+Roth%2C+Mark+Sammons%2C+and+Fabio+Massimo+Zanzotto%2E+Recognizing+textual+entailment%3A+Models+and+applications%2E+Synthesis+Lectures+on+Human+Language+Technologies%2C+6%284%29%3A1%E2%80%93220%2C+2013%2E)
14. Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and Łukasz Kaiser. Universal transformers. arXiv preprint arXiv:1807.03819, 2018. [find](https://scholar.google.com/scholar?q=Mostafa+Dehghani%2C+Stephan+Gouws%2C+Oriol+Vinyals%2C+Jakob+Uszkoreit%2C+and+%C5%81ukasz+Kaiser%2E+Universal+transformers%2E+arXiv+preprint+arXiv%3A1807%2E03819%2C+2018%2E)
15. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+BERT%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+arXiv+preprint+arXiv%3A1810%2E04805%2C+2018%2E)
16. Jesse Dodge, Gabriel Ilharco, Roy Schwartz, Ali Farhadi, Hannaneh Hajishirzi, and Noah Smith. Fine-tuning pretrained language models: Weight initializations, data orders, and early stopping. arXiv preprint arXiv:2002.06305, 2020. [find](https://scholar.google.com/scholar?q=Jesse+Dodge%2C+Gabriel+Ilharco%2C+Roy+Schwartz%2C+Ali+Farhadi%2C+Hannaneh+Hajishirzi%2C+and+Noah+Smith%2E+Fine%2Dtuning+pretrained+language+models%3A+Weight+initializations%2C+data+orders%2C+and+early+stopping%2E+arXiv+pre)
17. William B Dolan and Chris Brockett. Automatically constructing a corpus of sentential paraphrases. In Proceedings of the Third International Workshop on Paraphrasing (IWP2005), 2005. [find](https://scholar.google.com/scholar?q=William+B+Dolan+and+Chris+Brockett%2E+Automatically+constructing+a+corpus+of+sentential+paraphrases%2E+In+Proceedings+of+the+Third+International+Workshop+on+Paraphrasing+%28IWP2005%29%2C+2005%2E) [our card](https://md.research.elamur.ai/free/en/papers/dolan-2005-automatically-constructing-corpus-sentential-paraphrases/)
18. Zhen Dong, Zhewei Yao, Amir Gholami, Michael W Mahoney, and Kurt Keutzer. HAWQ: Hessian aware quantization of neural networks with mixed-precision. In Proceedings of the IEEE International Conference on Computer Vision, pages 293–302, 2019. [find](https://scholar.google.com/scholar?q=Zhen+Dong%2C+Zhewei+Yao%2C+Amir+Gholami%2C+Michael+W+Mahoney%2C+and+Kurt+Keutzer%2E+HAWQ%3A+Hessian+aware+quantization+of+neural+networks+with+mixed%2Dprecision%2E+In+Proceedings+of+the+IEEE+International+Conference+)
19. Steven K Esser, Jeffrey L McKinstry, Deepika Bablani, Rathinakumar Appuswamy, and Dharmendra S Modha. Learned step size quantization. arXiv preprint arXiv:1902.08153, 2019. — [arXiv:1902.08153](https://arxiv.org/abs/1902.08153) [find](https://scholar.google.com/scholar?q=Steven+K+Esser%2C+Jeffrey+L+McKinstry%2C+Deepika+Bablani%2C+Rathinakumar+Appuswamy%2C+and+Dharmendra+S+Modha%2E+Learned+step+size+quantization%2E+arXiv+preprint+arXiv%3A1902%2E08153%2C+2019%2E+%E2%80%94+%5BarXiv%3A1902%2E08153%5D%28https%3A)
20. Angela Fan, Edouard Grave, and Armand Joulin. Reducing transformer depth on demand with structured dropout. arXiv preprint arXiv:1909.11556, 2019. [find](https://scholar.google.com/scholar?q=Angela+Fan%2C+Edouard+Grave%2C+and+Armand+Joulin%2E+Reducing+transformer+depth+on+demand+with+structured+dropout%2E+arXiv+preprint+arXiv%3A1909%2E11556%2C+2019%2E) [our card](https://md.research.elamur.ai/free/en/papers/fan-2019-reducing-transformer-depth-demand-structured/)
21. Angela Fan, Pierre Stock, Benjamin Graham, Edouard Grave, Remi Gribonval, Herve Jegou, and Armand Joulin. Training with quantization noise for extreme fixed-point compression. arXiv preprint arXiv:2004.07320, 2020. — [arXiv:2004.07320](https://arxiv.org/abs/2004.07320) [find](https://scholar.google.com/scholar?q=Angela+Fan%2C+Pierre+Stock%2C+Benjamin+Graham%2C+Edouard+Grave%2C+Remi+Gribonval%2C+Herve+Jegou%2C+and+Armand+Joulin%2E+Training+with+quantization+noise+for+extreme+fixed%2Dpoint+compression%2E+arXiv+preprint+arXiv%3A200)
22. Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, et al. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027, 2020. [find](https://scholar.google.com/scholar?q=Leo+Gao%2C+Stella+Biderman%2C+Sid+Black%2C+Laurence+Golding%2C+Travis+Hoppe%2C+Charles+Foster%2C+Jason+Phang%2C+Horace+He%2C+Anish+Thite%2C+Noa+Nabeshima%2C+et+al%2E+The+pile%3A+An+800gb+dataset+of+diverse+text+for+language+)
23. Amir Gholami, Sehoon Kim, Zhen Dong, Zhewei Yao, Michael W Mahoney, and Kurt Keutzer. A survey of quantization methods for efficient neural network inference. arXiv preprint arXiv:2103.13630, 2021. — [arXiv:2103.13630](https://arxiv.org/abs/2103.13630) [find](https://scholar.google.com/scholar?q=Amir+Gholami%2C+Sehoon+Kim%2C+Zhen+Dong%2C+Zhewei+Yao%2C+Michael+W+Mahoney%2C+and+Kurt+Keutzer%2E+A+survey+of+quantization+methods+for+efficient+neural+network+inference%2E+arXiv+preprint+arXiv%3A2103%2E13630%2C+2021%2E+%E2%80%94+)
24. Mitchell A Gordon, Kevin Duh, and Nicholas Andrews. Compressing bert: Studying the effects of weight pruning on transfer learning. arXiv preprint arXiv:2002.08307, 2020. [find](https://scholar.google.com/scholar?q=Mitchell+A+Gordon%2C+Kevin+Duh%2C+and+Nicholas+Andrews%2E+Compressing+bert%3A+Studying+the+effects+of+weight+pruning+on+transfer+learning%2E+arXiv+preprint+arXiv%3A2002%2E08307%2C+2020%2E) [our card](https://md.research.elamur.ai/free/en/papers/gordon-2020-compressing-bert-studying-effects-weight/)
25. Song Han, Jeff Pool, John Tran, and William Dally. Learning both weights and connections for efficient neural network. In Advances in neural information processing systems, pages 1135–1143, 2015. [find](https://scholar.google.com/scholar?q=Song+Han%2C+Jeff+Pool%2C+John+Tran%2C+and+William+Dally%2E+Learning+both+weights+and+connections+for+efficient+neural+network%2E+In+Advances+in+neural+information+processing+systems%2C+pages+1135%E2%80%931143%2C+2015%2E)
26. Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. Workshop paper in NIPS, 2014. [find](https://scholar.google.com/scholar?q=Geoffrey+Hinton%2C+Oriol+Vinyals%2C+and+Jeff+Dean%2E+Distilling+the+knowledge+in+a+neural+network%2E+Workshop+paper+in+NIPS%2C+2014%2E) [our card](https://md.research.elamur.ai/free/en/papers/hinton-2015-distilling-knowledge-neural-network/)
27. Shankar Iyer, Nikhil Dandekar, and Kornl Csernai. First quora dataset release: Question pairs.(2017). URL [https://data.quora](https://data.quora). com/First-Quora-Dataset-Release-Question-Pairs, 2017. [↗](https://data.quora)
28. Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, and Qun Liu. Tinybert: Distilling bert for natural language understanding. arXiv preprint arXiv:1909.10351, 2019. [find](https://scholar.google.com/scholar?q=Xiaoqi+Jiao%2C+Yichun+Yin%2C+Lifeng+Shang%2C+Xin+Jiang%2C+Xiao+Chen%2C+Linlin+Li%2C+Fang+Wang%2C+and+Qun+Liu%2E+Tinybert%3A+Distilling+bert+for+natural+language+understanding%2E+arXiv+preprint+arXiv%3A1909%2E10351%2C+2019%2E)
29. Jing Jin, Cai Liang, Tiancheng Wu, Liqin Zou, and Zhiliang Gan. Kdlsq-bert: A quantized bert combining knowledge distillation with learned step size quantization. arXiv preprint arXiv:2101.05938, 2021. — [arXiv:2101.05938](https://arxiv.org/abs/2101.05938) [find](https://scholar.google.com/scholar?q=Jing+Jin%2C+Cai+Liang%2C+Tiancheng+Wu%2C+Liqin+Zou%2C+and+Zhiliang+Gan%2E+Kdlsq%2Dbert%3A+A+quantized+bert+combining+knowledge+distillation+with+learned+step+size+quantization%2E+arXiv+preprint+arXiv%3A2101%2E05938%2C+2021)
30. Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. arXiv preprint arXiv:1705.03551, 2017. [find](https://scholar.google.com/scholar?q=Mandar+Joshi%2C+Eunsol+Choi%2C+Daniel+S+Weld%2C+and+Luke+Zettlemoyer%2E+Triviaqa%3A+A+large+scale+distantly+supervised+challenge+dataset+for+reading+comprehension%2E+arXiv+preprint+arXiv%3A1705%2E03551%2C+2017%2E) [our card](https://md.research.elamur.ai/free/en/papers/joshi-2017-triviaqa-large-scale-distantly-supervised/)
31. Sehoon Kim, Amir Gholami, Zhewei Yao, Michael W Mahoney, and Kurt Keutzer. I-bert: Integer-only bert quantization. In International conference on machine learning, pages 5506–5518. PMLR, 2021. [find](https://scholar.google.com/scholar?q=Sehoon+Kim%2C+Amir+Gholami%2C+Zhewei+Yao%2C+Michael+W+Mahoney%2C+and+Kurt+Keutzer%2E+I%2Dbert%3A+Integer%2Donly+bert+quantization%2E+In+International+conference+on+machine+learning%2C+pages+5506%E2%80%935518%2E+PMLR%2C+2021%2E) [our card](https://md.research.elamur.ai/free/en/papers/kim-2021-i-bert-integer-only-bert/)
32. Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. Race: Large-scale reading comprehension dataset from examinations. arXiv preprint arXiv:1704.04683, 2017. [find](https://scholar.google.com/scholar?q=Guokun+Lai%2C+Qizhe+Xie%2C+Hanxiao+Liu%2C+Yiming+Yang%2C+and+Eduard+Hovy%2E+Race%3A+Large%2Dscale+reading+comprehension+dataset+from+examinations%2E+arXiv+preprint+arXiv%3A1704%2E04683%2C+2017%2E) [our card](https://md.research.elamur.ai/free/en/papers/lai-2017-race-large-scale-reading-comprehension/)
33. Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942, 2019. [find](https://scholar.google.com/scholar?q=Zhenzhong+Lan%2C+Mingda+Chen%2C+Sebastian+Goodman%2C+Kevin+Gimpel%2C+Piyush+Sharma%2C+and+Radu+Soricut%2E+Albert%3A+A+lite+bert+for+self%2Dsupervised+learning+of+language+representations%2E+arXiv+preprint+arXiv%3A1909%2E11)
34. Yann LeCun, John S Denker, and Sara A Solla. Optimal brain damage. In Advances in neural information processing systems, pages 598–605, 1990. [find](https://scholar.google.com/scholar?q=Yann+LeCun%2C+John+S+Denker%2C+and+Sara+A+Solla%2E+Optimal+brain+damage%2E+In+Advances+in+neural+information+processing+systems%2C+pages+598%E2%80%93605%2C+1990%2E)
35. Hector Levesque, Ernest Davis, and Leora Morgenstern. The winograd schema challenge. In Thirteenth International Conference on the Principles of Knowledge Representation and Reasoning. Citeseer, 2012. [find](https://scholar.google.com/scholar?q=Hector+Levesque%2C+Ernest+Davis%2C+and+Leora+Morgenstern%2E+The+winograd+schema+challenge%2E+In+Thirteenth+International+Conference+on+the+Principles+of+Knowledge+Representation+and+Reasoning%2E+Citeseer%2C+2012%2E)
36. Fengfu Li, Bo Zhang, and Bin Liu. Ternary weight networks. arXiv preprint arXiv:1605.04711, 2016. — [arXiv:1605.04711](https://arxiv.org/abs/1605.04711) [find](https://scholar.google.com/scholar?q=Fengfu+Li%2C+Bo+Zhang%2C+and+Bin+Liu%2E+Ternary+weight+networks%2E+arXiv+preprint+arXiv%3A1605%2E04711%2C+2016%2E+%E2%80%94+%5BarXiv%3A1605%2E04711%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1605%2E04711%29)
37. Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf. Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710, 2016. — [arXiv:1608.08710](https://arxiv.org/abs/1608.08710) [find](https://scholar.google.com/scholar?q=Hao+Li%2C+Asim+Kadav%2C+Igor+Durdanovic%2C+Hanan+Samet%2C+and+Hans+Peter+Graf%2E+Pruning+filters+for+efficient+convnets%2E+arXiv+preprint+arXiv%3A1608%2E08710%2C+2016%2E+%E2%80%94+%5BarXiv%3A1608%2E08710%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1608%2E08)
38. Zhenhua Liu, Yunhe Wang, Kai Han, Wei Zhang, Siwei Ma, and Wen Gao. Post-training quantization for vision transformer. Advances in Neural Information Processing Systems, 34, 2021. [find](https://scholar.google.com/scholar?q=Zhenhua+Liu%2C+Yunhe+Wang%2C+Kai+Han%2C+Wei+Zhang%2C+Siwei+Ma%2C+and+Wen+Gao%2E+Post%2Dtraining+quantization+for+vision+transformer%2E+Advances+in+Neural+Information+Processing+Systems%2C+34%2C+2021%2E)
39. Huizi Mao, Song Han, Jeff Pool, Wenshuo Li, Xingyu Liu, Yu Wang, and William J Dally. Exploring the regularity of sparse structure in convolutional neural networks. Workshop paper in CVPR, 2017. [find](https://scholar.google.com/scholar?q=Huizi+Mao%2C+Song+Han%2C+Jeff+Pool%2C+Wenshuo+Li%2C+Xingyu+Liu%2C+Yu+Wang%2C+and+William+J+Dally%2E+Exploring+the+regularity+of+sparse+structure+in+convolutional+neural+networks%2E+Workshop+paper+in+CVPR%2C+2017%2E)
40. Yihuan Mao, Yujing Wang, Chufan Wu, Chen Zhang, Yang Wang, Yaming Yang, Quanlu Zhang, Yunhai Tong, and Jing Bai. Ladabert: Lightweight adaptation of bert through hybrid model compression. arXiv preprint arXiv:2004.04124, 2020. [find](https://scholar.google.com/scholar?q=Yihuan+Mao%2C+Yujing+Wang%2C+Chufan+Wu%2C+Chen+Zhang%2C+Yang+Wang%2C+Yaming+Yang%2C+Quanlu+Zhang%2C+Yunhai+Tong%2C+and+Jing+Bai%2E+Ladabert%3A+Lightweight+adaptation+of+bert+through+hybrid+model+compression%2E+arXiv+prepri)
41. Mary Ann Marcinkiewicz. Building a large annotated corpus of english: The penn treebank. Using Large Corpora, page 273, 1994. [find](https://scholar.google.com/scholar?q=Mary+Ann+Marcinkiewicz%2E+Building+a+large+annotated+corpus+of+english%3A+The+penn+treebank%2E+Using+Large+Corpora%2C+page+273%2C+1994%2E) [our card](https://md.research.elamur.ai/free/en/papers/bank-1993-marcus-building-large-annotated-corpus-english/)
42. Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. In International Conference on Learning Representations, 2017. [find](https://scholar.google.com/scholar?q=Stephen+Merity%2C+Caiming+Xiong%2C+James+Bradbury%2C+and+Richard+Socher%2E+Pointer+sentinel+mixture+models%2E+In+International+Conference+on+Learning+Representations%2C+2017%2E) [our card](https://md.research.elamur.ai/free/en/papers/merity-2016-pointer-sentinel-mixture-models/)
43. Paul Michel, Omer Levy, and Graham Neubig. Are sixteen heads really better than one? arXiv preprint arXiv:1905.10650, 2019. [find](https://scholar.google.com/scholar?q=Paul+Michel%2C+Omer+Levy%2C+and+Graham+Neubig%2E+Are+sixteen+heads+really+better+than+one%3F+arXiv+preprint+arXiv%3A1905%2E10650%2C+2019%2E) [our card](https://md.research.elamur.ai/free/en/papers/michel-2019-sixteen-heads-really-better-than/)
44. Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? a new dataset for open book question answering. arXiv preprint arXiv:1809.02789, 2018. [find](https://scholar.google.com/scholar?q=Todor+Mihaylov%2C+Peter+Clark%2C+Tushar+Khot%2C+and+Ashish+Sabharwal%2E+Can+a+suit+of+armor+conduct+electricity%3F+a+new+dataset+for+open+book+question+answering%2E+arXiv+preprint+arXiv%3A1809%2E02789%2C+2018%2E)
45. Markus Nagel, Rana Ali Amjad, Mart Van Baalen, Christos Louizos, and Tijmen Blankevoort. Up or down? adaptive rounding for post-training quantization. In International Conference on Machine Learning, pages 7197–7206. PMLR, 2020. [find](https://scholar.google.com/scholar?q=Markus+Nagel%2C+Rana+Ali+Amjad%2C+Mart+Van+Baalen%2C+Christos+Louizos%2C+and+Tijmen+Blankevoort%2E+Up+or+down%3F+adaptive+rounding+for+post%2Dtraining+quantization%2E+In+International+Conference+on+Machine+Learning%2C+)
46. Markus Nagel, Mart van Baalen, Tijmen Blankevoort, and Max Welling. Data-free quantization through weight equalization and bias correction. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1325–1334, 2019. [find](https://scholar.google.com/scholar?q=Markus+Nagel%2C+Mart+van+Baalen%2C+Tijmen+Blankevoort%2C+and+Max+Welling%2E+Data%2Dfree+quantization+through+weight+equalization+and+bias+correction%2E+In+Proceedings+of+the+IEEE%2FCVF+International+Conference+on+C)
47. Denis Paperno, Germán Kruszewski, Angeliki Lazaridou, Quan Ngoc Pham, Raffaella Bernardi, Sandro Pezzelle, Marco Baroni, Gemma Boleda, and Raquel Fernández. The lambada dataset: Word prediction requiring a broad discourse context. arXiv preprint arXiv:1606.06031, 2016. [find](https://scholar.google.com/scholar?q=Denis+Paperno%2C+Germ%C3%A1n+Kruszewski%2C+Angeliki+Lazaridou%2C+Quan+Ngoc+Pham%2C+Raffaella+Bernardi%2C+Sandro+Pezzelle%2C+Marco+Baroni%2C+Gemma+Boleda%2C+and+Raquel+Fern%C3%A1ndez%2E+The+lambada+dataset%3A+Word+prediction+requir) [our card](https://md.research.elamur.ai/free/en/papers/paperno-2016-lambada-dataset-word-prediction-requiring/)
48. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. 2019. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeff+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+and+Ilya+Sutskever%2E+Language+models+are+unsupervised+multitask+learners%2E+2019%2E)
49. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. Exploring the limits of transfer learning with a unified text-to-text transformer, 2019. [find](https://scholar.google.com/scholar?q=Colin+Raffel%2C+Noam+Shazeer%2C+Adam+Roberts%2C+Katherine+Lee%2C+Sharan+Narang%2C+Michael+Matena%2C+Yanqi+Zhou%2C+Wei+Li%2C+and+Peter+J%2E+Liu%2E+Exploring+the+limits+of+transfer+learning+with+a+unified+text%2Dto%2Dtext+tran)
50. Alessandro Raganato, Yves Scherrer, and Jörg Tiedemann. Fixed encoder self-attention patterns in transformer-based machine translation. arXiv preprint arXiv:2002.10260, 2020. [find](https://scholar.google.com/scholar?q=Alessandro+Raganato%2C+Yves+Scherrer%2C+and+J%C3%B6rg+Tiedemann%2E+Fixed+encoder+self%2Dattention+patterns+in+transformer%2Dbased+machine+translation%2E+arXiv+preprint+arXiv%3A2002%2E10260%2C+2020%2E)
51. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. SQuAD: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250, 2016. [find](https://scholar.google.com/scholar?q=Pranav+Rajpurkar%2C+Jian+Zhang%2C+Konstantin+Lopyrev%2C+and+Percy+Liang%2E+SQuAD%3A+100%2C000%2B+questions+for+machine+comprehension+of+text%2E+arXiv+preprint+arXiv%3A1606%2E05250%2C+2016%2E) [our card](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/)
52. Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pages 3505–3506, 2020. [find](https://scholar.google.com/scholar?q=Jeff+Rasley%2C+Samyam+Rajbhandari%2C+Olatunji+Ruwase%2C+and+Yuxiong+He%2E+Deepspeed%3A+System+optimizations+enable+training+deep+learning+models+with+over+100+billion+parameters%2E+In+Proceedings+of+the+26th+ACM+)
53. Greg Ruetsch. Using tensor cores in cuda fortran. Nvidia Blog, 2021. [find](https://scholar.google.com/scholar?q=Greg+Ruetsch%2E+Using+tensor+cores+in+cuda+fortran%2E+Nvidia+Blog%2C+2021%2E)
54. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 8732–8740, 2020. [find](https://scholar.google.com/scholar?q=Keisuke+Sakaguchi%2C+Ronan+Le+Bras%2C+Chandra+Bhagavatula%2C+and+Yejin+Choi%2E+Winogrande%3A+An+adversarial+winograd+schema+challenge+at+scale%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+)
55. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108, 2019. [find](https://scholar.google.com/scholar?q=Victor+Sanh%2C+Lysandre+Debut%2C+Julien+Chaumond%2C+and+Thomas+Wolf%2E+Distilbert%2C+a+distilled+version+of+bert%3A+smaller%2C+faster%2C+cheaper+and+lighter%2E+arXiv+preprint+arXiv%3A1910%2E01108%2C+2019%2E)
56. Sheng Shen, Zhen Dong, Jiayu Ye, Linjian Ma, Zhewei Yao, Amir Gholami, Michael W Mahoney, and Kurt Keutzer. Q-BERT: Hessian based ultra low precision quantization of bert. In AAAI, pages 8815–8821, 2020. [find](https://scholar.google.com/scholar?q=Sheng+Shen%2C+Zhen+Dong%2C+Jiayu+Ye%2C+Linjian+Ma%2C+Zhewei+Yao%2C+Amir+Gholami%2C+Michael+W+Mahoney%2C+and+Kurt+Keutzer%2E+Q%2DBERT%3A+Hessian+based+ultra+low+precision+quantization+of+bert%2E+In+AAAI%2C+pages+8815%E2%80%938821%2C+20)
57. Shaden Smith, Mostofa Patwary, Brandon Norick, Patrick LeGresley, Samyam Rajbhandari, Jared Casper, Zhun Liu, Shrimai Prabhumoye, George Zerveas, Vijay Korthikanti, et al. Using deepspeed and megatron to train megatron-turing nlg 530b, a large-scale generative language model. arXiv preprint arXiv:22… [find](https://scholar.google.com/scholar?q=Shaden+Smith%2C+Mostofa+Patwary%2C+Brandon+Norick%2C+Patrick+LeGresley%2C+Samyam+Rajbhandari%2C+Jared+Casper%2C+Zhun+Liu%2C+Shrimai+Prabhumoye%2C+George+Zerveas%2C+Vijay+Korthikanti%2C+et+al%2E+Using+deepspeed+and+megatron) [our card](https://md.research.elamur.ai/free/en/papers/smith-2022-using-deepspeed-megatron-train-megatron/)
58. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language processing, pages 1631–1642,… [find](https://scholar.google.com/scholar?q=Richard+Socher%2C+Alex+Perelygin%2C+Jean+Wu%2C+Jason+Chuang%2C+Christopher+D+Manning%2C+Andrew+Y+Ng%2C+and+Christopher+Potts%2E+Recursive+deep+models+for+semantic+compositionality+over+a+sentiment+treebank%2E+In+Proc)
59. Siqi Sun, Yu Cheng, Zhe Gan, and Jingjing Liu. Patient knowledge distillation for bert model compression. arXiv preprint arXiv:1908.09355, 2019. [find](https://scholar.google.com/scholar?q=Siqi+Sun%2C+Yu+Cheng%2C+Zhe+Gan%2C+and+Jingjing+Liu%2E+Patient+knowledge+distillation+for+bert+model+compression%2E+arXiv+preprint+arXiv%3A1908%2E09355%2C+2019%2E) [our card](https://md.research.elamur.ai/free/en/papers/sun-2019-patient-knowledge-distillation-bert-model/)
60. Zhiqing Sun, Hongkun Yu, Xiaodan Song, Renjie Liu, Yiming Yang, and Denny Zhou. Mobilebert: a compact task-agnostic bert for resource-limited devices. arXiv preprint arXiv:2004.02984, 2020. [find](https://scholar.google.com/scholar?q=Zhiqing+Sun%2C+Hongkun+Yu%2C+Xiaodan+Song%2C+Renjie+Liu%2C+Yiming+Yang%2C+and+Denny+Zhou%2E+Mobilebert%3A+a+compact+task%2Dagnostic+bert+for+resource%2Dlimited+devices%2E+arXiv+preprint+arXiv%3A2004%2E02984%2C+2020%2E) [our card](https://md.research.elamur.ai/free/en/papers/sun-2020-mobilebert-compact-task-agnostic-bert/)
61. Chaofan Tao, Lu Hou, Wei Zhang, Lifeng Shang, Xin Jiang, Qun Liu, Ping Luo, and Ngai Wong. Compression of generative pre-trained language models via quantization. arXiv preprint arXiv:2203.10705, 2022. [find](https://scholar.google.com/scholar?q=Chaofan+Tao%2C+Lu+Hou%2C+Wei+Zhang%2C+Lifeng+Shang%2C+Xin+Jiang%2C+Qun+Liu%2C+Ping+Luo%2C+and+Ngai+Wong%2E+Compression+of+generative+pre%2Dtrained+language+models+via+quantization%2E+arXiv+preprint+arXiv%3A2203%2E10705%2C+2022)
62. Sandeep Tata and Jignesh M Patel. Piqa: An algebra for querying protein data sets. In 15th International Conference on Scientific and Statistical Database Management, 2003., pages 141–150. IEEE, 2003. [find](https://scholar.google.com/scholar?q=Sandeep+Tata+and+Jignesh+M+Patel%2E+Piqa%3A+An+algebra+for+querying+protein+data+sets%2E+In+15th+International+Conference+on+Scientific+and+Statistical+Database+Management%2C+2003%2E%2C+pages+141%E2%80%93150%2E+IEEE%2C+2003%2E)
63. Ian Tenney, Dipanjan Das, and Ellie Pavlick. Bert rediscovers the classical nlp pipeline. arXiv:1905.05950, 2019. [find](https://scholar.google.com/scholar?q=Ian+Tenney%2C+Dipanjan+Das%2C+and+Ellie+Pavlick%2E+Bert+rediscovers+the+classical+nlp+pipeline%2E+arXiv%3A1905%2E05950%2C+2019%2E) [our card](https://md.research.elamur.ai/free/en/papers/tenney-2019-bert-rediscovers-classical-nlp-pipeline/)
64. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pages 5998–6008, 2017. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+Attention+is+all+you+need%2E+In+Advances+in+neural+information+processing+sys)
65. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. GLUE: A multi-task benchmark and analysis platform for natural language understanding. arXiv preprint arXiv:1804.07461, 2018. [find](https://scholar.google.com/scholar?q=Alex+Wang%2C+Amanpreet+Singh%2C+Julian+Michael%2C+Felix+Hill%2C+Omer+Levy%2C+and+Samuel+R+Bowman%2E+GLUE%3A+A+multi%2Dtask+benchmark+and+analysis+platform+for+natural+language+understanding%2E+arXiv+preprint+arXiv%3A1804) [our card](https://md.research.elamur.ai/free/en/papers/wang-2018-glue-multi-task-benchmark-analysis/)
66. Ben Wang and Aran Komatsuzaki. GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model. [https://github.com/kingoflolz/mesh-transformer-jax](https://github.com/kingoflolz/mesh-transformer-jax), May 2021. [↗](https://github.com/kingoflolz/mesh-transformer-jax)
67. Guibin Wang, Yisong Lin, and Wei Yi. Kernel fusion: An effective method for better power efficiency on multithreaded GPU. In Peidong Zhu, Lizhe Wang, Feng Xia, Huajun Chen, Ian McLoughlin, Shiao-Li Tsao, Mitsuhisa Sato, Sun-Ki Chai, and Irwin King, editors, 2010 IEEE/ACM Int’l Conference on Green Co… [find](https://scholar.google.com/scholar?q=Guibin+Wang%2C+Yisong+Lin%2C+and+Wei+Yi%2E+Kernel+fusion%3A+An+effective+method+for+better+power+efficiency+on+multithreaded+GPU%2E+In+Peidong+Zhu%2C+Lizhe+Wang%2C+Feng+Xia%2C+Huajun+Chen%2C+Ian+McLoughlin%2C+Shiao%2DLi+Ts)
68. Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou. Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers. arXiv preprint arXiv:2002.10957, 2020. [find](https://scholar.google.com/scholar?q=Wenhui+Wang%2C+Furu+Wei%2C+Li+Dong%2C+Hangbo+Bao%2C+Nan+Yang%2C+and+Ming+Zhou%2E+Minilm%3A+Deep+self%2Dattention+distillation+for+task%2Dagnostic+compression+of+pre%2Dtrained+transformers%2E+arXiv+preprint+arXiv%3A2002%2E10957)
69. Alex Warstadt, Amanpreet Singh, and Samuel R Bowman. Neural network acceptability judgments. arXiv preprint arXiv:1805.12471, 2018. [find](https://scholar.google.com/scholar?q=Alex+Warstadt%2C+Amanpreet+Singh%2C+and+Samuel+R+Bowman%2E+Neural+network+acceptability+judgments%2E+arXiv+preprint+arXiv%3A1805%2E12471%2C+2018%2E)
70. Adina Williams, Nikita Nangia, and Samuel Bowman. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers),… [find](https://scholar.google.com/scholar?q=Adina+Williams%2C+Nikita+Nangia%2C+and+Samuel+Bowman%2E+A+broad%2Dcoverage+challenge+corpus+for+sentence+understanding+through+inference%2E+In+Proceedings+of+the+2018+Conference+of+the+North+American+Chapter+of)
71. Adina Williams, Tristan Thrush, and Douwe Kiela. Anlizing the adversarial natural language inference dataset. arXiv preprint arXiv:2010.12729, 2020. [find](https://scholar.google.com/scholar?q=Adina+Williams%2C+Tristan+Thrush%2C+and+Douwe+Kiela%2E+Anlizing+the+adversarial+natural+language+inference+dataset%2E+arXiv+preprint+arXiv%3A2010%2E12729%2C+2020%2E)
72. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. HuggingFace’s Transformers: State-of-the-art natural language processing. ArXiv, pages arXiv–1910, 2019. [find](https://scholar.google.com/scholar?q=Thomas+Wolf%2C+Lysandre+Debut%2C+Victor+Sanh%2C+Julien+Chaumond%2C+Clement+Delangue%2C+Anthony+Moi%2C+Pierric+Cistac%2C+Tim+Rault%2C+R%C3%A9mi+Louf%2C+Morgan+Funtowicz%2C+et+al%2E+HuggingFace%E2%80%99s+Transformers%3A+State%2Dof%2Dthe%2Dart+na) [our card](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/)
73. Vikas Yadav, Steven Bethard, and Mihai Surdeanu. Quick and (not so) dirty: Unsupervised selection of justification sentences for multi-hop question answering. arXiv preprint arXiv:1911.07176, 2019. [find](https://scholar.google.com/scholar?q=Vikas+Yadav%2C+Steven+Bethard%2C+and+Mihai+Surdeanu%2E+Quick+and+%28not+so%29+dirty%3A+Unsupervised+selection+of+justification+sentences+for+multi%2Dhop+question+answering%2E+arXiv+preprint+arXiv%3A1911%2E07176%2C+2019%2E)
74. Zhewei Yao, Linjian Ma, Sheng Shen, Kurt Keutzer, and Michael W Mahoney. Mlpruning: A multilevel structured pruning framework for transformer-based models. arXiv preprint arXiv:2105.14636, 2021. — [arXiv:2105.14636](https://arxiv.org/abs/2105.14636) [find](https://scholar.google.com/scholar?q=Zhewei+Yao%2C+Linjian+Ma%2C+Sheng+Shen%2C+Kurt+Keutzer%2C+and+Michael+W+Mahoney%2E+Mlpruning%3A+A+multilevel+structured+pruning+framework+for+transformer%2Dbased+models%2E+arXiv+preprint+arXiv%3A2105%2E14636%2C+2021%2E+%E2%80%94+%5Bar)
75. Ali Hadi Zadeh, Isak Edo, Omar Mohamed Awad, and Andreas Moshovos. Gobo: Quantizing attention-based nlp models for low latency and energy efficient inference. In 2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), pages 811–824. IEEE, 2020. [find](https://scholar.google.com/scholar?q=Ali+Hadi+Zadeh%2C+Isak+Edo%2C+Omar+Mohamed+Awad%2C+and+Andreas+Moshovos%2E+Gobo%3A+Quantizing+attention%2Dbased+nlp+models+for+low+latency+and+energy+efficient+inference%2E+In+2020+53rd+Annual+IEEE%2FACM+Internationa)
76. Ofir Zafrir, Guy Boudoukh, Peter Izsak, and Moshe Wasserblat. Q8BERT: Quantized 8bit bert. arXiv preprint arXiv:1910.06188, 2019. — [arXiv:1910.06188](https://arxiv.org/abs/1910.06188) [find](https://scholar.google.com/scholar?q=Ofir+Zafrir%2C+Guy+Boudoukh%2C+Peter+Izsak%2C+and+Moshe+Wasserblat%2E+Q8BERT%3A+Quantized+8bit+bert%2E+arXiv+preprint+arXiv%3A1910%2E06188%2C+2019%2E+%E2%80%94+%5BarXiv%3A1910%2E06188%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1910%2E06188%29)
77. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830, 2019. [find](https://scholar.google.com/scholar?q=Rowan+Zellers%2C+Ari+Holtzman%2C+Yonatan+Bisk%2C+Ali+Farhadi%2C+and+Yejin+Choi%2E+Hellaswag%3A+Can+a+machine+really+finish+your+sentence%3F+arXiv+preprint+arXiv%3A1905%2E07830%2C+2019%2E)
78. Sheng Zhang, Xiaodong Liu, Jingjing Liu, Jianfeng Gao, Kevin Duh, and Benjamin Van Durme. Record: Bridging the gap between human and machine commonsense reading comprehension. arXiv preprint arXiv:1810.12885, 2018. [find](https://scholar.google.com/scholar?q=Sheng+Zhang%2C+Xiaodong+Liu%2C+Jingjing+Liu%2C+Jianfeng+Gao%2C+Kevin+Duh%2C+and+Benjamin+Van+Durme%2E+Record%3A+Bridging+the+gap+between+human+and+machine+commonsense+reading+comprehension%2E+arXiv+preprint+arXiv%3A181) [our card](https://md.research.elamur.ai/free/en/papers/zhang-2018-record-bridging-gap-between-human/)
79. Wei Zhang, Lu Hou, Yichun Yin, Lifeng Shang, Xiao Chen, Xin Jiang, and Qun Liu. Ternarybert: Distillation-aware ultra-low bit bert. arXiv preprint arXiv:2009.12812, 2020. [find](https://scholar.google.com/scholar?q=Wei+Zhang%2C+Lu+Hou%2C+Yichun+Yin%2C+Lifeng+Shang%2C+Xiao+Chen%2C+Xin+Jiang%2C+and+Qun+Liu%2E+Ternarybert%3A+Distillation%2Daware+ultra%2Dlow+bit+bert%2E+arXiv+preprint+arXiv%3A2009%2E12812%2C+2020%2E)