---
title: "Who cites DistilBERT, a distilled version — 2,404 citing papers"
description: "2,404 papers of the Elamur Research library cite «DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter», found in their…"
canonical: "https://research.elamur.ai/free/en/papers/sanh-2019-distilbert-distilled-version-bert-smaller/cited-by/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/sanh-2019-distilbert-distilled-version-bert-smaller/cited-by/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [DistilBERT, a distilled version of BERT: smaller…](https://md.research.elamur.ai/free/en/papers/sanh-2019-distilbert-distilled-version-bert-smaller/) / cited by

# Who cites «DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter»

2,404 papers of our library cite it (of the 149,439 whose bibliographies we read) · 284 with a page here, the most cited first · page 1 of 3 · 1–100 · [← back to «DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter»](https://md.research.elamur.ai/free/en/papers/sanh-2019-distilbert-distilled-version-bert-smaller/)

**1** [2](https://md.research.elamur.ai/free/en/papers/sanh-2019-distilbert-distilled-version-bert-smaller/cited-by/page/2/) [3](https://md.research.elamur.ai/free/en/papers/sanh-2019-distilbert-distilled-version-bert-smaller/cited-by/page/3/) [next →](https://md.research.elamur.ai/free/en/papers/sanh-2019-distilbert-distilled-version-bert-smaller/cited-by/page/2/)

1. **[Transformers: State-of-the-Art Natural Language Processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Thomas Wolf, Lysandre Debut, Victor Sanh, et al.*2020 · 15,755 citations*
2. [**Language Models are Few-Shot Learners arXiv:2005.14165**Tom B. Brown, Benjamin Mann, Nick Ryder, et al.*2020 · 14,178 citations*](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
3. [**Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer arXiv:1910.10683**Colin Raffel, Noam Shazeer, Adam Roberts, et al.*2019 · 11,325 citations*](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/)
4. [**The Llama 3 Herd of Models arXiv:2407.21783**Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, et al.*2024 · 7,682 citations*](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/)
5. [**On the Opportunities and Risks of Foundation Models arXiv:2108.07258**Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, et al.*2021 · 3,160 citations*](https://md.research.elamur.ai/free/en/papers/bommasani-2021-opportunities-risks-foundation-models/)
6. **[TinyBERT: Distilling BERT for Natural Language Understanding](https://md.research.elamur.ai/free/en/papers/jiao-2020-tinybert-distilling-bert-natural-language/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Xiaoqi Jiao, Yichun Yin, Lifeng Shang, et al.*2020 · 2,378 citations*
7. [**COMET: A Neural Framework for MT Evaluation**Ricardo Rei, Craig Stewart, Ana C Farinha, et al.*2020 · 1,792 citations*](https://md.research.elamur.ai/free/en/papers/rei-2020-comet-neural-framework-mt-evaluation/)
8. [**Pre-trained Models for Natural Language Processing: A Survey arXiv:2003.08271 · SCIENCE CHINA Technological Sciences,2020, 63, 1872-1897**Xipeng Qiu, Tianxiang Sun, Yige Xu, et al.*2020 · 1,759 citations*](https://md.research.elamur.ai/free/en/papers/qiu-2020-pre-trained-models-natural-language/)
9. **[Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://md.research.elamur.ai/free/en/papers/min-2021-recent-advances-natural-language-processing/) [ACM Computing Surveys](https://md.research.elamur.ai/free/en/venues/acm-computing-surveys/)**Bonan Min, Hayley Ross, Elior Sulem, et al.*2021 · 1,556 citations*
10. **[Efficient Transformers: A Survey](https://md.research.elamur.ai/free/en/papers/tay-2020-efficient-transformers-survey/) [ACM Computing Surveys](https://md.research.elamur.ai/free/en/venues/acm-computing-surveys/)**Yi Tay, Mostafa Dehghani, Dara Bahri, et al.*2020 · 1,539 citations*
11. **[It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/schick-2021-it-s-not-just-size/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Timo Schick, Hinrich Schütze*2021 · 1,132 citations*
12. **[Quantifying Attention Flow in Transformers](https://md.research.elamur.ai/free/en/papers/abnar-2020-quantifying-attention-flow-transformers/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Samira Abnar, Willem Zuidema*2020 · 1,064 citations*
13. **[MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices](https://md.research.elamur.ai/free/en/papers/sun-2020-mobilebert-compact-task-agnostic-bert/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Zhiqing Sun, Hongkun Yu, Xiaodan Song, et al.*2020 · 1,016 citations*
14. **[Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond](https://md.research.elamur.ai/free/en/papers/yang-2023-harnessing-power-llms-practice-survey/) [ACM Transactions on Knowledge Discovery from Data](https://md.research.elamur.ai/free/en/venues/acm-transactions-on-knowledge-discovery-from-data/)**Jingfeng Yang, Hongye Jin, Ruixiang Tang, et al.*2023 · 814 citations*
15. [**GLM-130B: An Open Bilingual Pre-trained Model arXiv:2210.02414**Aohan Zeng, Xiao Liu, Zhengxiao Du, et al.*2022 · 786 citations*](https://md.research.elamur.ai/free/en/papers/zeng-2022-glm-130b-open-bilingual-pre/)
16. **[Efficient Intent Detection with Dual Sentence Encoders](https://md.research.elamur.ai/free/en/papers/casanueva-2020-efficient-intent-detection-dual-sentence/) [Workshop on Natural Language Processing for Conversational AI](https://md.research.elamur.ai/free/en/venues/nlp4convai/)**Iñigo Casanueva, Tadas Temčinas, Daniela Gerz, et al.*2020 · 628 citations*
17. **[FNet: Mixing Tokens with Fourier Transforms](https://md.research.elamur.ai/free/en/papers/thorp-2022-fnet-mixing-tokens-fourier-transforms/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**James Lee-Thorp, Joshua Ainslie, Ilya Eckstein, et al.*2022 · 597 citations*
18. **[DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations](https://md.research.elamur.ai/free/en/papers/giorgi-2021-declutr-deep-contrastive-learning-unsupervised/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**John Giorgi, Osvald Nitski, Bo Wang, et al.*2021 · 516 citations*
19. **[DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference](https://md.research.elamur.ai/free/en/papers/xin-2020-deebert-dynamic-early-exiting-accelerating/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Ji Xin, Raphael Tang, Jaejun Lee, et al.*2020 · 487 citations*
20. **[Pretrained Transformers Improve Out-of-Distribution Robustness](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-pretrained-transformers-improve-out-distribution/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Dan Hendrycks, Xiaoyuan Liu, Eric Wallace, et al.*2020 · 452 citations*
21. **[Efficiently Teaching an Effective Dense Retriever with Balanced Topic Aware Sampling](https://md.research.elamur.ai/free/en/papers/hofstatter-2021-efficiently-teaching-effective-dense-retriever/) [International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR)](https://md.research.elamur.ai/free/en/venues/international-acm-sigir-conference-on-research-and-development-in-information/)**Sebastian Hofstatter, Sheng-Chieh Lin, Jheng-Hong Yang, et al.*2021 · 425 citations*
22. [**True Few-Shot Learning with Language Models arXiv:2105.11447**Ethan Perez, Douwe Kiela, Kyunghyun Cho*2021 · 412 citations*](https://md.research.elamur.ai/free/en/papers/perez-2021-true-few-shot-learning-language/)
23. **[Do Prompt-Based Models Really Understand the Meaning of Their Prompts?](https://md.research.elamur.ai/free/en/papers/webson-2022-do-prompt-based-models-really/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Albert Webson, Ellie Pavlick*2022 · 407 citations*
24. [**A Simple Language Model for Task-Oriented Dialogue arXiv:2005.00796**Ehsan Hosseini-Asl, Bryan McCann, Chien-Sheng Wu, et al.*2020 · 399 citations*](https://md.research.elamur.ai/free/en/papers/hosseini-asl-2020-simple-language-model-task-oriented/)
25. [**Aligning AI With Shared Human Values arXiv:2008.02275**Dan Hendrycks, Collin Burns, Steven Basart, et al.*2020 · 395 citations*](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-aligning-ai-shared-human-values/)
26. **[FastBERT: a Self-distilling BERT with Adaptive Inference Time](https://md.research.elamur.ai/free/en/papers/liu-2020-fastbert-self-distilling-bert-adaptive/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Weijie Liu, Peng Zhou, Zhiruo Wang, et al.*2020 · 383 citations*
27. **[Parameter-Efficient Transfer Learning with Diff Pruning](https://md.research.elamur.ai/free/en/papers/guo-2021-parameter-efficient-transfer-learning-diff/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Demi Guo, Alexander Rush, Yoon Kim*2021 · 367 citations*
28. **[Symbolic Knowledge Distillation: from General Language Models to Commonsense Models](https://md.research.elamur.ai/free/en/papers/west-2022-symbolic-knowledge-distillation-general-language/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Peter West, Chandra Bhagavatula, Jack Hessel, et al.*2022 · 349 citations*
29. [**UL2: Unifying Language Learning Paradigms arXiv:2205.05131**Yi Tay, Mostafa Dehghani, Vinh Q. Tran, et al.*2022 · 347 citations*](https://md.research.elamur.ai/free/en/papers/tay-2022-ul2-unifying-language-learning-paradigms/)
30. **[Unsupervised Domain Clusters in Pretrained Language Models](https://md.research.elamur.ai/free/en/papers/aharoni-2020-unsupervised-domain-clusters-pretrained-language/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Roee Aharoni, Yoav Goldberg*2020 · 330 citations*
31. [**Carbon Emissions and Large Neural Network Training arXiv:2104.10350**David Patterson, Joseph Gonzalez, Quoc Le, et al.*2021 · 323 citations*](https://md.research.elamur.ai/free/en/papers/patterson-2021-carbon-emissions-large-neural-network/)
32. [**ALFWorld: Aligning Text and Embodied Environments for Interactive Learning arXiv:2010.03768**Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, et al.*2020 · 322 citations*](https://md.research.elamur.ai/free/en/papers/shridhar-2020-alfworld-aligning-text-embodied-environments/)
33. **[From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models](https://md.research.elamur.ai/free/en/papers/feng-2023-pretraining-data-language-models-downstream/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Shangbin Feng, Chan Young Park, Yuhan Liu, et al.*2023 · 311 citations*
34. **[AugGPT: Leveraging ChatGPT for Text Data Augmentation](https://md.research.elamur.ai/free/en/papers/dai-2023-auggpt-leveraging-chatgpt-text-data/) [IEEE Transactions on Big Data](https://md.research.elamur.ai/free/en/venues/ieee-transactions-on-big-data/)**Haixing Dai; Zhengliang Liu; Wenxiong Liao; Xiaoke Huang; Yihan Cao; Zihao Wu; Lin Zhao; Shaochen Xu; Wei Liu; Ninghao Liu; Sheng Li; Dajiang Zhu; Hongmin Cai; Lichao Sun; Quanzheng Li; Dinggang Shen; Tianming Liu;, Xiang Li*2023 · 306 citations*
35. **[SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer](https://md.research.elamur.ai/free/en/papers/vu-2022-spot-better-frozen-model-adaptation/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Tu Vu, Brian Lester, Noah Constant, et al.*2022 · 306 citations*
36. **[ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers](https://md.research.elamur.ai/free/en/papers/yao-2022-zeroquant-efficient-affordable-post-training/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Zhewei Yao, Reza Yazdani Aminabadi, Minjia Zhang, et al.*2022 · 304 citations*
37. [**On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes arXiv:2306.13649**Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, et al.*2023 · 299 citations*](https://md.research.elamur.ai/free/en/papers/agarwal-2023-policy-distillation-language-models-learning/)
38. **[AdapterDrop: On the Efficiency of Adapters in Transformers](https://md.research.elamur.ai/free/en/papers/ruckle-2021-adapterdrop-efficiency-adapters-transformers/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Andreas Rücklé, Gregor Geigle, Max Glockner, et al.*2021 · 296 citations*
39. **[AlignScore: Evaluating Factual Consistency with A Unified Alignment Function](https://md.research.elamur.ai/free/en/papers/zha-2023-alignscore-evaluating-factual-consistency-unified/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Yuheng Zha, Yichi Yang, Ruichen Li, et al.*2023 · 284 citations*
40. **[Contextualized Perturbation for Textual Adversarial Attack](https://md.research.elamur.ai/free/en/papers/li-2021-contextualized-perturbation-textual-adversarial-attack/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Dianqi Li, Yizhe Zhang, Hao Peng, et al.*2021 · 283 citations*
41. **[GPT3Mix: Leveraging Large-scale Language Models for Text Augmentation](https://md.research.elamur.ai/free/en/papers/yoo-2021-gpt3mix-leveraging-large-scale-language/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Kang Min Yoo, Dongju Park, Jaewook Kang, et al.*2021 · 270 citations*
42. **[DiffCSE: Difference-based Contrastive Learning for Sentence Embeddings](https://md.research.elamur.ai/free/en/papers/chuang-2022-diffcse-difference-based-contrastive-learning/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Yung-Sung Chuang, Rumen Dangovski, Hongyin Luo, et al.*2022 · 269 citations*
43. **[ConveRT: Efficient and Accurate Conversational Representations from Transformers](https://md.research.elamur.ai/free/en/papers/henderson-2020-convert-efficient-accurate-conversational-representations/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Matthew Henderson, Iñigo Casanueva, Nikola Mrkšić, et al.*2020 · 269 citations*
44. [**BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models NeurIPS 2021 Datasets and Benchmarks**Nandan Thakur, Nils Reimers, Andreas Rücklé, et al.*2021 · 265 citations*](https://md.research.elamur.ai/free/en/papers/thakur-2021-beir-heterogenous-benchmark-zero/)
45. [**DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale arXiv:2201.05596**Samyam Rajbhandari, Conglong Li, Zhewei Yao, et al.*2022 · 261 citations*](https://md.research.elamur.ai/free/en/papers/rajbhandari-2022-deepspeed-moe-advancing-mixture-experts/)
46. [**Multilingual is not enough: BERT for Finnish arXiv:1912.07076**Antti Virtanen, Jenna Kanerva, Rami Ilo, et al.*2019 · 258 citations*](https://md.research.elamur.ai/free/en/papers/virtanen-2019-multilingual-enough-bert-finnish/)
47. **[ILDC for CJPE: Indian Legal Documents Corpus for Court Judgment Prediction and Explanation](https://md.research.elamur.ai/free/en/papers/malik-2021-ildc-cjpe-indian-legal-documents/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Vijit Malik, Rishabh Sanjay, Shubham Kumar Nigam, et al.*2021 · 251 citations*
48. **[Interpreting Pretrained Contextualized Representations via Reductions to Static Embeddings](https://md.research.elamur.ai/free/en/papers/bommasani-2020-interpreting-pretrained-contextualized-representations-reductions/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Rishi Bommasani, Kelly Davis, Claire Cardie*2020 · 250 citations*
49. [**A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models arXiv:2401.01313**S.M Towhidul Islam Tonmoy, S M Mehedi Zaman, Vinija Jain, et al.*2024 · 249 citations*](https://md.research.elamur.ai/free/en/papers/tonmoy-2024-comprehensive-survey-hallucination-mitigation-techniques/)
50. **[ZeroGen: Efficient Zero-shot Learning via Dataset Generation](https://md.research.elamur.ai/free/en/papers/ye-2022-zerogen-efficient-zero-shot-learning/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Jiacheng Ye, Jiahui Gao, Qintong Li, et al.*2022 · 245 citations*
51. [**FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance arXiv:2305.05176**Lingjiao Chen, Matei Zaharia, James Zou*2023 · 241 citations*](https://md.research.elamur.ai/free/en/papers/chen-2023-frugalgpt-use-large-language-models/)
52. **[Pretrained Language Models for Biomedical and Clinical Tasks: Understanding and Extending the State-of-the-Art](https://md.research.elamur.ai/free/en/papers/lewis-2020-pretrained-language-models-biomedical-clinical/) [Workshop on Clinical Natural Language Processing (Clinical NLP) @ LREC](https://md.research.elamur.ai/free/en/venues/clinicalnlp/)**Patrick Lewis, Myle Ott, Jingfei Du, et al.*2020 · 241 citations*
53. **[BERT-of-Theseus: Compressing BERT by Progressive Module Replacing](https://md.research.elamur.ai/free/en/papers/xu-2020-bert-theseus-compressing-bert-progressive/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Canwen Xu, Wangchunshu Zhou, Tao Ge, et al.*2020 · 240 citations*
54. **[Structured Pruning Learns Compact and Accurate Models](https://md.research.elamur.ai/free/en/papers/xia-2022-structured-pruning-learns-compact-accurate/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Mengzhou Xia, Zexuan Zhong, Danqi Chen*2022 · 221 citations*
55. **[Generating Fact Checking Explanations](https://md.research.elamur.ai/free/en/papers/atanasova-2020-generating-fact-checking-explanations/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Pepa Atanasova, Jakob Grue Simonsen, Christina Lioma, et al.*2020 · 220 citations*
56. [**TinyStories: How Small Can Language Models Be and Still Speak Coherent English?**Ronen Eldan, Yuanzhi Li*2023 · 213 citations*](https://md.research.elamur.ai/free/en/papers/eldan-2023-tinystories-small-language-models-still/)
57. **[TernaryBERT: Distillation-aware Ultra-low Bit BERT](https://md.research.elamur.ai/free/en/papers/zhang-2020-ternarybert-distillation-aware-ultra-low/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Wei Zhang, Lu Hou, Yichun Yin, et al.*2020 · 213 citations*
58. **[Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias](https://md.research.elamur.ai/free/en/papers/yu-2023-large-language-model-attributed-training/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Yue Yu, Yuchen Zhuang, Jieyu Zhang, et al.*2023 · 202 citations*
59. **[Structured Pruning of Large Language Models](https://md.research.elamur.ai/free/en/papers/wang-2020-structured-pruning-large-language-models/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Ziheng Wang, Jeremy Wohlwend, Tao Lei*2020 · 202 citations*
60. [**RRHF: Rank Responses to Align Language Models with Human Feedback without tears arXiv:2304.05302**Zheng Yuan, Hongyi Yuan, Chuanqi Tan, et al.*2023 · 201 citations*](https://md.research.elamur.ai/free/en/papers/yuan-2023-rrhf-rank-responses-align-language/)
61. **[A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets](https://md.research.elamur.ai/free/en/papers/laskar-2023-systematic-study-comprehensive-evaluation-chatgpt/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Md Tahmid Rahman Laskar, M Saiful Bari, Mizanur Rahman, et al.*2023 · 200 citations*
62. **[Block Pruning For Faster Transformers](https://md.research.elamur.ai/free/en/papers/lagunas-2021-block-pruning-faster-transformers/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**François Lagunas, Ella Charlaix, Victor Sanh, et al.*2021 · 199 citations*
63. **[The Right Tool for the Job: Matching Model and Instance Complexities](https://md.research.elamur.ai/free/en/papers/schwartz-2020-right-tool-job-matching-model/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Roy Schwartz, Gabriel Stanovsky, Swabha Swayamdipta, et al.*2020 · 196 citations*
64. **[What Can We Learn from Collective Human Opinions on Natural Language Inference Data?](https://md.research.elamur.ai/free/en/papers/nie-2020-what-can-we-learn-collective/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Yixin Nie, Xiang Zhou, Mohit Bansal*2020 · 196 citations*
65. **[Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuning](https://md.research.elamur.ai/free/en/papers/alabi-2022-adapting-pre-trained-language-models/) [International Conference on Computational Linguistics (COLING)](https://md.research.elamur.ai/free/en/venues/coling/)**Jesujoba O. Alabi, David Ifeoluwa Adelani, Marius Mosbach, et al.*2022 · 195 citations*
66. **[HONEST: Measuring Hurtful Sentence Completion in Language Models](https://md.research.elamur.ai/free/en/papers/nozza-2021-honest-measuring-hurtful-sentence-completion/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Debora Nozza, Federico Bianchi, Dirk Hovy*2021 · 191 citations*
67. **[SemEval-2022 Task 11: Multilingual Complex Named Entity Recognition (MultiCoNER)](https://md.research.elamur.ai/free/en/papers/malmasi-2022-semeval-2022-task-11-multilingual/) [International Workshop on Semantic Evaluation (SemEval)](https://md.research.elamur.ai/free/en/venues/semeval/)**Shervin Malmasi, Anjie Fang, Besnik Fetahu, et al.*2022 · 190 citations*
68. **[Compressing Large-Scale Transformer-Based Models: A Case Study on BERT](https://md.research.elamur.ai/free/en/papers/ganesh-2021-compressing-large-scale-transformer-based/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Prakhar Ganesh, Yao Chen, Xin Lou, et al.*2021 · 190 citations*
69. **[Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfer](https://md.research.elamur.ai/free/en/papers/qi-2021-mind-style-text-adversarial-backdoor/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Fanchao Qi, Yangyi Chen, Xurui Zhang, et al.*2021 · 189 citations*
70. [**Eliciting Latent Predictions from Transformers with the Tuned Lens arXiv:2303.08112**Nora Belrose, Igor Ostrovsky, Lev McKinney, et al.*2023 · 188 citations*](https://md.research.elamur.ai/free/en/papers/belrose-2023-eliciting-latent-predictions-transformers-tuned/)
71. **[SemEval-2023 Task 10: Explainable Detection of Online Sexism](https://md.research.elamur.ai/free/en/papers/kirk-2023-semeval-2023-task-10-explainable/) [International Workshop on Semantic Evaluation (SemEval)](https://md.research.elamur.ai/free/en/venues/semeval/)**Hannah Kirk, Wenjie Yin, Bertie Vidgen, et al.*2023 · 185 citations*
72. **[Debiasing Pre-trained Contextualised Embeddings](https://md.research.elamur.ai/free/en/papers/kaneko-2021-debiasing-pre-trained-contextualised-embeddings/) [Conference of the European Chapter of the Association for Computational Linguistics (EACL)](https://md.research.elamur.ai/free/en/venues/eacl/)**Masahiro Kaneko, Danushka Bollegala*2021 · 175 citations*
73. [**Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization arXiv:2210.01241**Rajkumar Ramamurthy, Prithviraj Ammanabrolu, Kiante Brantley, et al.*2022 · 174 citations*](https://md.research.elamur.ai/free/en/papers/ramamurthy-2022-reinforcement-learning-natural-language-processing/)
74. **[The Troubling Emergence of Hallucination in Large Language Models - An Extensive Definition, Quantification, and Prescriptive Remediations](https://md.research.elamur.ai/free/en/papers/rawte-2023-troubling-emergence-hallucination-large-language/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Vipula Rawte, Swagata Chakraborty, Agnibh Pathak, et al.*2023 · 171 citations*
75. [**AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving OSDI 2023**Zhuohan Li, Lianmin Zheng, Yinmin Zhong, et al.*2023 · 170 citations*](https://md.research.elamur.ai/free/en/papers/li-2023-alpaserve-statistical-multiplexing-model-parallelism/)
76. **[Quark: Controllable Text Generation with Reinforced Unlearning](https://md.research.elamur.ai/free/en/papers/lu-2022-quark-controllable-text-generation-reinforced/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Ximing Lu, Sean Welleck, Jack Hessel, et al.*2022 · 170 citations*
77. **[BinaryBERT: Pushing the Limit of BERT Quantization](https://md.research.elamur.ai/free/en/papers/bai-2021-binarybert-pushing-limit-bert-quantization/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Haoli Bai, Wei Zhang, Lu Hou, et al.*2021 · 166 citations*
78. [**CLIPort: What and Where Pathways for Robotic Manipulation arXiv:2109.12098**Mohit Shridhar, Lucas Manuelli, Dieter Fox*2021 · 159 citations*](https://md.research.elamur.ai/free/en/papers/shridhar-2021-cliport-where-pathways-robotic-manipulation/)
79. **[GPL: Generative Pseudo Labeling for Unsupervised Domain Adaptation of Dense Retrieval](https://md.research.elamur.ai/free/en/papers/wang-2022-gpl-generative-pseudo-labeling-unsupervised/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Kexin Wang, Nandan Thakur, Nils Reimers, et al.*2022 · 155 citations*
80. [**AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators arXiv:2303.16854**Xingwei He, Zhenghao Lin, Yeyun Gong, et al.*2023 · 152 citations*](https://md.research.elamur.ai/free/en/papers/he-2023-annollm-making-large-language-models/)
81. **[Exploring Versatile Generative Language Model Via Parameter-Efficient Transfer Learning](https://md.research.elamur.ai/free/en/papers/lin-2020-exploring-versatile-generative-language-model/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Zhaojiang Lin, Andrea Madotto, Pascale Fung*2020 · 152 citations*
82. **[CPM: A Large-scale Generative Chinese Pre-trained Language Model](https://md.research.elamur.ai/free/en/papers/zhang-2020-cpm-large-scale-generative/) [AI Open](https://md.research.elamur.ai/free/en/venues/ai-open/)**Zhengyan Zhang, Xu Han, Hao Zhou, et al.*2020 · 150 citations*
83. **[Introducing CAD: the Contextual Abuse Dataset](https://md.research.elamur.ai/free/en/papers/vidgen-2021-introducing-cad-contextual-abuse-dataset/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Bertie Vidgen, Dong Nguyen, Helen Margetts, et al.*2021 · 147 citations*
84. **[Efficient Methods for Natural Language Processing: A Survey](https://md.research.elamur.ai/free/en/papers/treviso-2023-efficient-methods-natural-language-processing/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Marcos Treviso, Ji-Ung Lee, Tianchu Ji, et al.*2023 · 145 citations*
85. **[UNQOVERing Stereotyping Biases via Underspecified Questions](https://md.research.elamur.ai/free/en/papers/li-2020-unqovering-stereotyping-biases-underspecified-questions/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Tao Li, Daniel Khashabi, Tushar Khot, et al.*2020 · 145 citations*
86. [**MiniLLM: On-Policy Distillation of Large Language Models arXiv:2306.08543**Yuxian Gu, Li Dong, Furu Wei, et al.*2023 · 144 citations*](https://md.research.elamur.ai/free/en/papers/gu-2023-minillm-policy-distillation-large-language/)
87. [**Identifying and Measuring Annotator Bias Based on Annotators' Demographic Characteristics Proceedings of the Fourth Workshop on Online Abuse and Harms**Hala Al Kuwatly, Maximilian Wich, Georg Groh*2020 · 144 citations*](https://md.research.elamur.ai/free/en/papers/kuwatly-2020-identifying-measuring-annotator-bias-based/)
88. [**Rethinking embedding coupling in pre-trained language models arXiv:2010.12821**Hyung Won Chung, Thibault Févry, Henry Tsai, et al.*2020 · 142 citations*](https://md.research.elamur.ai/free/en/papers/chung-2020-rethinking-embedding-coupling-pre-trained/)
89. **[On the Language Neutrality of Pre-trained Multilingual Representations](https://md.research.elamur.ai/free/en/papers/libovicky-2020-language-neutrality-pre-trained-multilingual/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Jindřich Libovický, Rudolf Rosa, Alexander Fraser*2020 · 140 citations*
90. [**Fantastic Questions and Where to Find Them: FairytaleQA – An Authentic Dataset for Narrative Comprehension**Ying Xu, Dakuo Wang, Mo Yu, et al.*2022 · 137 citations*](https://md.research.elamur.ai/free/en/papers/xu-2022-fantastic-questions-where-find-them/)
91. **[Grounded Adaptation for Zero-shot Executable Semantic Parsing](https://md.research.elamur.ai/free/en/papers/zhong-2020-grounded-adaptation-zero-shot-executable/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Victor Zhong, Mike Lewis, Sida I. Wang, et al.*2020 · 136 citations*
92. **[UnNatural Language Inference](https://md.research.elamur.ai/free/en/papers/sinha-2021-unnatural-language-inference/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Koustuv Sinha, Prasanna Parthasarathi, Joelle Pineau, et al.*2021 · 135 citations*
93. [**Recent Advances in Natural Language Inference: A Survey of Benchmarks, Resources, and Approaches arXiv:1904.01172**Shane Storks, Qiaozi Gao, Joyce Y. Chai*2019 · 135 citations*](https://md.research.elamur.ai/free/en/papers/storks-2019-recent-advances-natural-language-inference/)
94. **[Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Dataset](https://md.research.elamur.ai/free/en/papers/henderson-2022-pile-law-learning-responsible-data/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Peter Henderson, Mark S. Krass, Lucia Zheng, et al.*2022 · 133 citations*
95. **[Learning Compact Metrics for MT](https://md.research.elamur.ai/free/en/papers/pu-2021-learning-compact-metrics-mt/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Amy Pu, Hyung Won Chung, Ankur Parikh, et al.*2021 · 130 citations*
96. **[LEXTREME: A Multi-Lingual and Multi-Task Benchmark for the Legal Domain](https://md.research.elamur.ai/free/en/papers/niklaus-2023-lextreme-multi-lingual-multi-task/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Joel Niklaus, Veton Matoshi, Pooja Rani, et al.*2023 · 123 citations*
97. **[I-BERT: Integer-only BERT Quantization](https://md.research.elamur.ai/free/en/papers/kim-2021-i-bert-integer-only-bert/) [International Conference on Machine Learning (ICML)](https://md.research.elamur.ai/free/en/venues/international-conference-on-machine-learning/)**Sehoon Kim, Amir Gholami, Zhewei Yao, et al.*2021 · 122 citations*
98. **[Logical Fallacy Detection](https://md.research.elamur.ai/free/en/papers/jin-2022-logical-fallacy-detection/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Zhijing Jin, Abhinav Lalwani, Tejas Vaidhya, et al.*2022 · 119 citations*
99. **[Unmasking Contextual Stereotypes: Measuring and Mitigating BERT's Gender Bias](https://md.research.elamur.ai/free/en/papers/bartl-2020-unmasking-contextual-stereotypes-measuring-mitigating/) [Workshop on Gender Bias in Natural Language Processing](https://md.research.elamur.ai/free/en/venues/gebnlp/)**Marion Bartl, Malvina Nissim, Albert Gatt*2020 · 116 citations*
100. **[MoEfication: Transformer Feed-forward Layers are Mixtures of Experts](https://md.research.elamur.ai/free/en/papers/zhang-2022-moefication-transformer-feed-forward-layers/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Zhengyan Zhang, Yankai Lin, Zhiyuan Liu, et al.*2022 · 115 citations*

**1** [2](https://md.research.elamur.ai/free/en/papers/sanh-2019-distilbert-distilled-version-bert-smaller/cited-by/page/2/) [3](https://md.research.elamur.ai/free/en/papers/sanh-2019-distilbert-distilled-version-bert-smaller/cited-by/page/3/) [next →](https://md.research.elamur.ai/free/en/papers/sanh-2019-distilbert-distilled-version-bert-smaller/cited-by/page/2/)

- [← back to](https://md.research.elamur.ai/free/en/papers/sanh-2019-distilbert-distilled-version-bert-smaller/) — DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter