---
title: "Who cites DistilBERT, a — page 2 — 2,404 citing papers"
description: "2,404 papers of the Elamur Research library cite «DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter», found in their…"
canonical: "https://research.elamur.ai/free/en/papers/sanh-2019-distilbert-distilled-version-bert-smaller/cited-by/page/2/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/sanh-2019-distilbert-distilled-version-bert-smaller/cited-by/page/2/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [DistilBERT, a distilled version of BERT: smaller…](https://md.research.elamur.ai/free/en/papers/sanh-2019-distilbert-distilled-version-bert-smaller/) / [cited by](https://md.research.elamur.ai/free/en/papers/sanh-2019-distilbert-distilled-version-bert-smaller/cited-by/) / page 2

# Who cites «DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter»

2,404 papers of our library cite it (of the 149,439 whose bibliographies we read) · 284 with a page here, the most cited first · page 2 of 3 · 101–200 · [← back to «DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter»](https://md.research.elamur.ai/free/en/papers/sanh-2019-distilbert-distilled-version-bert-smaller/)

[← prev](https://md.research.elamur.ai/free/en/papers/sanh-2019-distilbert-distilled-version-bert-smaller/cited-by/) [1](https://md.research.elamur.ai/free/en/papers/sanh-2019-distilbert-distilled-version-bert-smaller/cited-by/) **2** [3](https://md.research.elamur.ai/free/en/papers/sanh-2019-distilbert-distilled-version-bert-smaller/cited-by/page/3/) [next →](https://md.research.elamur.ai/free/en/papers/sanh-2019-distilbert-distilled-version-bert-smaller/cited-by/page/3/)

101. **[Aspect-Controllable Opinion Summarization](https://md.research.elamur.ai/free/en/papers/amplayo-2021-aspect-controllable-opinion-summarization/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Reinald Kim Amplayo, Stefanos Angelidis, Mirella Lapata*2021 · 114 citations*
102. [**Certifying LLM Safety against Adversarial Prompting arXiv:2309.02705**Aounon Kumar, Chirag Agarwal, Suraj Srinivas, et al.*2023 · 113 citations*](https://md.research.elamur.ai/free/en/papers/kumar-2023-certifying-llm-safety-against-adversarial/)
103. [**Efficient Large Scale Language Modeling with Mixtures of Experts arXiv:2112.10684**Mikel Artetxe, Shruti Bhosale, Naman Goyal, et al.*2021 · 113 citations*](https://md.research.elamur.ai/free/en/papers/artetxe-2021-efficient-large-scale-language-modeling/)
104. [**DistiLLM: Towards Streamlined Distillation for Large Language Models arXiv:2402.03898**Jongwoo Ko, Sungnyun Kim, Tianyi Chen, et al.*2024 · 112 citations*](https://md.research.elamur.ai/free/en/papers/ko-2024-distillm-towards-streamlined-distillation-large/)
105. **[Large Language Models Enable Few-Shot Clustering](https://md.research.elamur.ai/free/en/papers/viswanathan-2024-large-language-models-enable-few/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Vijay Viswanathan, Kiril Gashteovski, Carolin Lawrence, et al.*2024 · 111 citations*
106. **[Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems](https://md.research.elamur.ai/free/en/papers/miao-2023-towards-efficient-generative-large-language/) [ACM Computing Surveys](https://md.research.elamur.ai/free/en/venues/acm-computing-surveys/)**Xupeng Miao, Gabriele Oliaro, Zhihao Zhang, et al.*2023 · 110 citations*
107. **[Common Sense Beyond English: Evaluating and Improving Multilingual Language Models for Commonsense Reasoning](https://md.research.elamur.ai/free/en/papers/lin-2021-common-sense-beyond-english-evaluating/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Bill Yuchen Lin, Seyeon Lee, Xiaoyang Qiao, et al.*2021 · 107 citations*
108. **[LEDGAR: A Large-Scale Multi-label Corpus for Text Classification of Legal Provisions in Contracts](https://md.research.elamur.ai/free/en/papers/tuggener-2020-ledgar-large-scale-multi-label/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Don Tuggener, Pius von Däniken, Thomas Peetz, et al.*2020 · 107 citations*
109. [**LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions arXiv:2304.14402**Minghao Wu, Abdul Waheed, Chiyu Zhang, et al.*2023 · 105 citations*](https://md.research.elamur.ai/free/en/papers/wu-2023-lamini-lm-diverse-herd-distilled/)
110. [**minicons: Enabling Flexible Behavioral and Representational Analyses of Transformer Language Models arXiv:2203.13112**Kanishka Misra*2022 · 102 citations*](https://md.research.elamur.ai/free/en/papers/misra-2022-minicons-enabling-flexible-behavioral-representational/)
111. **[On the Intrinsic and Extrinsic Fairness Evaluation Metrics for Contextualized Language Representations](https://md.research.elamur.ai/free/en/papers/cao-2022-intrinsic-extrinsic-fairness-evaluation-metrics/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Yang Trista Cao, Yada Pruksachatkun, Kai-Wei Chang, et al.*2022 · 100 citations*
112. [**Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models arXiv:2601.18734**Siyan Zhao, Zhihui Xie, Mengchen Liu, et al.*2026 · 97 citations*](https://md.research.elamur.ai/free/en/papers/zhao-2026-self-distilled-reasoner-policy-self/)
113. **[CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologies](https://md.research.elamur.ai/free/en/papers/shi-2024-culturebank-online-community-driven-knowledge/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Weiyan Shi, Ryan Li, Yutong Zhang, et al.*2024 · 93 citations*
114. **[TransQuest: Translation Quality Estimation with Cross-lingual Transformers](https://md.research.elamur.ai/free/en/papers/ranasinghe-2020-transquest-translation-quality-estimation-cross/) [International Conference on Computational Linguistics (COLING)](https://md.research.elamur.ai/free/en/venues/coling/)**Tharindu Ranasinghe, Constantin Orasan, Ruslan Mitkov*2020 · 91 citations*
115. **[Probing for idiomaticity in vector space models](https://md.research.elamur.ai/free/en/papers/garcia-2021-probing-idiomaticity-vector-space-models/) [Conference of the European Chapter of the Association for Computational Linguistics (EACL)](https://md.research.elamur.ai/free/en/venues/eacl/)**Marcos Garcia, Tiago Kramer Vieira, Carolina Scarton, et al.*2021 · 89 citations*
116. [**Do Membership Inference Attacks Work on Large Language Models? arXiv:2402.07841**Michael Duan, Anshuman Suri, Niloofar Mireshghallah, et al.*2024 · 87 citations*](https://md.research.elamur.ai/free/en/papers/duan-2024-membership-inference-attacks-work-large/)
117. [**Foundational Challenges in Assuring Alignment and Safety of Large Language Models arXiv:2404.09932**Usman Anwar, Abulhair Saparov, Javier Rando, et al.*2024 · 87 citations*](https://md.research.elamur.ai/free/en/papers/anwar-2024-foundational-challenges-assuring-alignment-safety/)
118. [**REST: Retrieval-Based Speculative Decoding arXiv:2311.08252**Zhenyu He; Zexuan Zhong; Tianle Cai; Jason D. Lee;, Di He*2023 · 87 citations*](https://md.research.elamur.ai/free/en/papers/he-2023-rest-retrieval-based-speculative-decoding/)
119. **[Retrieval-Augmented Generation for Natural Language Processing: A Survey](https://md.research.elamur.ai/free/en/papers/wu-2024-retrieval-augmented-generation-natural-language/) [Artificial Intelligence Review](https://md.research.elamur.ai/free/en/venues/artificial-intelligence-review/)**Shangyu Wu, Ying Xiong, Yufei Cui, et al.*2024 · 86 citations*
120. [**ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewing arXiv:2306.00622**Ryan Liu, Nihar B. Shah*2023 · 84 citations*](https://md.research.elamur.ai/free/en/papers/liu-2023-reviewergpt-exploratory-study-using/)
121. **[ORCA: A Challenging Benchmark for Arabic Language Understanding](https://md.research.elamur.ai/free/en/papers/elmadany-2023-orca-challenging-benchmark-arabic-language/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**AbdelRahim Elmadany, ElMoatez Billah Nagoudi, Muhammad Abdul-Mageed*2023 · 83 citations*
122. [**Knowledge Fusion of Large Language Models arXiv:2401.10491**Fanqi Wan, Xinting Huang, Deng Cai, et al.*2024 · 82 citations*](https://md.research.elamur.ai/free/en/papers/wan-2024-knowledge-fusion-large-language-models/)
123. [**Reinforcement Learning via Self-Distillation arXiv:2601.20802**Jonas Hubotter, Frederike Lubeck, Lejs Behric, et al.*2026 · 74 citations*](https://md.research.elamur.ai/free/en/papers/hubotter-2026-reinforcement-learning-self-distillation/)
124. [**QuRating: Selecting High-Quality Data for Training Language Models arXiv:2402.09739**Alexander Wettig, Aatmik Gupta, Saumya Malik, et al.*2024 · 74 citations*](https://md.research.elamur.ai/free/en/papers/wettig-2024-qurating-selecting-high-quality-data/)
125. **[Structural Persistence in Language Models: Priming as a Window into Abstract Language Representations](https://md.research.elamur.ai/free/en/papers/sinclair-2022-structural-persistence-language-models-priming/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Arabella Sinclair, Jaap Jumelet, Willem Zuidema, et al.*2022 · 74 citations*
126. [**Annotation Error Detection: Analyzing the Past and Present for a More Coherent Future**Jan-Christoph Klie, Bonnie Webber, Iryna Gurevych*2023 · 73 citations*](https://md.research.elamur.ai/free/en/papers/klie-2023-annotation-error-detection-analyzing-past/)
127. **[ArAIEval Shared Task: Persuasion Techniques and Disinformation Detection in Arabic Text](https://md.research.elamur.ai/free/en/papers/hasanain-2023-araieval-shared-task-persuasion-techniques/) [Arabic Natural Language Processing Conference (ArabicNLP)](https://md.research.elamur.ai/free/en/venues/arabicnlp/)**Maram Hasanain, Firoj Alam, Hamdy Mubarak, et al.*2023 · 67 citations*
128. **[FORK: A Bite-Sized Test Set for Probing Culinary Cultural Biases in Commonsense Reasoning Models](https://md.research.elamur.ai/free/en/papers/palta-2023-fork-bite-sized-test-set/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Shramay Palta, Rachel Rudinger*2023 · 67 citations*
129. **[A Causal Framework to Quantify the Robustness of Mathematical Reasoning with Language Models](https://md.research.elamur.ai/free/en/papers/stolfo-2023-causal-framework-quantify-robustness-mathematical/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Alessandro Stolfo, Zhijing Jin, Kumar Shridhar, et al.*2023 · 65 citations*
130. [**It is AI's Turn to Ask Humans a Question: Question-Answer Pair Generation for Children's Story Books**Bingsheng Yao, Dakuo Wang, Tongshuang Wu, et al.*2022 · 64 citations*](https://md.research.elamur.ai/free/en/papers/yao-2022-it-is-ai-s-turn/)
131. [**Building a Bridge: A Method for Image-Text Sarcasm Detection Without Pretraining on Image-Text Data Proceedings of the First International Workshop on Natural Language Processing Beyond Text**Xinyu Wang, Xiaowen Sun, Tan Yang, et al.*2020 · 62 citations*](https://md.research.elamur.ai/free/en/papers/wang-2020-building-bridge-method-image-text/)
132. [**mmBERT: A Modern Multilingual Encoder with Annealed Language Learning arXiv:2509.06888**Marc Marone, Orion Weller, William Fleshman, et al.*2025 · 59 citations*](https://md.research.elamur.ai/free/en/papers/marone-2025-mmbert-modern-multilingual-encoder-annealed/)
133. **[Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketching](https://md.research.elamur.ai/free/en/papers/aytes-2025-sketch-thought-efficient-llm-reasoning/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Simon A. Aytes, Jinheon Baek, Sung Ju Hwang*2025 · 51 citations*
134. [**From Decoding to Meta-Generation: Inference-time Algorithms for Large Language Models arXiv:2406.16838**Sean Welleck, Amanda Bertsch, Matthew Finlayson, et al.*2024 · 51 citations*](https://md.research.elamur.ai/free/en/papers/welleck-2024-decoding-meta-generation-inference-time/)
135. **[VariErr NLI: Separating Annotation Error from Human Label Variation](https://md.research.elamur.ai/free/en/papers/genzel-2024-varierr-nli-separating-annotation-error/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Leon Weber-Genzel, Siyao Peng, Marie-Catherine De Marneffe, et al.*2024 · 51 citations*
136. [**Towards a Decomposable Metric for Explainable Evaluation of Text Generation from AMR**Juri Opitz, Anette Frank*2021 · 51 citations*](https://md.research.elamur.ai/free/en/papers/opitz-2021-decomposable-metric-explainable-evaluation-text/)
137. [**Language models scale reliably with over-training and on downstream tasks arXiv:2403.08540**Samir Yitzhak Gadre, Georgios Smyrnis, Vaishaal Shankar, et al.*2024 · 46 citations*](https://md.research.elamur.ai/free/en/papers/gadre-2024-language-models-scale-reliably-over/)
138. [**Improving Cross-lingual Information Retrieval on Low-Resource Languages via Optimal Transport Distillation arXiv:2301.12566**Zhiqi Huang, Puxuan Yu, James Allan*2023 · 46 citations*](https://md.research.elamur.ai/free/en/papers/huang-2023-improving-cross-lingual-information-retrieval/)
139. **[F-coref: Fast, Accurate and Easy to Use Coreference Resolution](https://md.research.elamur.ai/free/en/papers/otmazgin-2022-f-coref-fast-accurate-easy/) [Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (AACL)](https://md.research.elamur.ai/free/en/venues/aacl/)**Shon Otmazgin, Arie Cattan, Yoav Goldberg*2022 · 45 citations*
140. [**Knowledge Distillation in Automated Annotation: Supervised Text Classification with LLM-Generated Training Labels arXiv:2406.17633**Nicholas Pangakis, Samuel Wolken*2024 · 42 citations*](https://md.research.elamur.ai/free/en/papers/pangakis-2024-knowledge-distillation-automated-annotation-supervised/)
141. [**DeepSeek: Paradigm Shifts and Technical Evolution in Large AI Models arXiv:2507.09955**Luolin Xiong, Haofen Wang, Xi Chen, et al.*2025 · 41 citations*](https://md.research.elamur.ai/free/en/papers/xiong-2025-deepseek-paradigm-shifts-technical/)
142. [**Speculative Knowledge Distillation: Bridging the Teacher-Student Gap Through Interleaved Sampling arXiv:2410.11325**Wenda Xu, Rujun Han, Zifeng Wang, et al.*2024 · 41 citations*](https://md.research.elamur.ai/free/en/papers/xu-2024-speculative-knowledge-distillation-bridging-teacher/)
143. **[Dual-Space Knowledge Distillation for Large Language Models](https://md.research.elamur.ai/free/en/papers/zhang-2024-dual-space-knowledge-distillation-large/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Songming Zhang, Xue Zhang, Zengkui Sun, et al.*2024 · 39 citations*
144. [**Towards Cross-Tokenizer Distillation: the Universal Logit Distillation Loss for LLMs arXiv:2402.12030 · Transactions on Machine Learning Research, January 2025**Nicolas Boizard, Kevin El Haddad, Celine Hudelot, et al.*2024 · 35 citations*](https://md.research.elamur.ai/free/en/papers/boizard-2024-towards-cross-tokenizer-distillation-universal/)
145. **[Computational Politeness in Natural Language Processing: A Survey](https://md.research.elamur.ai/free/en/papers/priya-2024-computational-politeness-natural-language-processing/) [ACM Computing Surveys](https://md.research.elamur.ai/free/en/venues/acm-computing-surveys/)**Priyanshu Priya, Mauajama Firdaus, Asif Ekbal*2024 · 34 citations*
146. **[Controllable Summarization with Constrained Markov Decision Process](https://md.research.elamur.ai/free/en/papers/chan-2021-controllable-summarization-constrained-markov-decision/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Hou Pong Chan, Lu Wang, Irwin King*2021 · 34 citations*
147. [**A Survey of Small Language Models arXiv:2410.20011**Chien Van Nguyen, Xuan Shen, Ryan Aponte, et al.*2024 · 31 citations*](https://md.research.elamur.ai/free/en/papers/nguyen-2024-survey-small-language-models/)
148. **[Let's be explicit about that: Distant supervision for implicit discourse relation classification via connective prediction](https://md.research.elamur.ai/free/en/papers/kurfali-2021-let-s-be-explicit-about/) [Workshop on Understanding Implicit and Underspecified Language](https://md.research.elamur.ai/free/en/venues/unimplicit/)**Murathan Kurfalı, Robert Östling*2021 · 31 citations*
149. **[Click: Controllable Text Generation with Sequence Likelihood Contrastive Learning](https://md.research.elamur.ai/free/en/papers/zheng-2023-click-controllable-text-generation-sequence/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Chujie Zheng, Pei Ke, Zheng Zhang, et al.*2023 · 29 citations*
150. **[How Abstract Is Linguistic Generalization in Large Language Models? Experiments with Argument Structure](https://md.research.elamur.ai/free/en/papers/wilson-2023-how-abstract-is-linguistic-generalization/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Michael Wilson, Jackson Petty, Robert Frank*2023 · 29 citations*
151. [**One Law, Many Languages: Benchmarking Multilingual Legal Reasoning for Judicial Support arXiv:2306.09237**Ronja Stern, Vishvaksenan Rasiah, Veton Matoshi, et al.*2023 · 29 citations*](https://md.research.elamur.ai/free/en/papers/stern-2023-one-law-many-languages-benchmarking/)
152. **[A Survey on Asking Clarification Questions Datasets in Conversational Systems](https://md.research.elamur.ai/free/en/papers/rahmani-2023-survey-asking-clarification-questions-datasets/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Hossein A. Rahmani, Xi Wang, Yue Feng, et al.*2023 · 28 citations*
153. **[Context Matters: A Pragmatic Study of PLMs' Negation Understanding](https://md.research.elamur.ai/free/en/papers/gubelmann-2022-context-matters-pragmatic-study-plms/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Reto Gubelmann, Siegfried Handschuh*2022 · 28 citations*
154. **[Towards the Law of Capacity Gap in Distilling Language Models](https://md.research.elamur.ai/free/en/papers/zhang-2025-law-capacity-gap-distilling-language/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Chen Zhang, Qiuchi Li, Dawei Song, et al.*2025 · 27 citations*
155. **[FuseChat: Knowledge Fusion of Chat Models](https://md.research.elamur.ai/free/en/papers/wan-2025-fusechat-knowledge-fusion-chat-models/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Fanqi Wan, Longguang Zhong, Ziyi Yang, et al.*2025 · 25 citations*
156. **[What is the Role of Small Models in the LLM Era: A Survey](https://md.research.elamur.ai/free/en/papers/chen-2024-role-small-models-llm-era/) [Computational Linguistics](https://md.research.elamur.ai/free/en/venues/cl/)**Lihu Chen, Gael Varoquaux*2024 · 23 citations*
157. **[Joint Prompt Optimization of Stacked LLMs using Variational Inference](https://md.research.elamur.ai/free/en/papers/sordoni-2023-joint-prompt-optimization-stacked-llms/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Alessandro Sordoni, Xingdi Yuan, Marc-Alexandre Cote, et al.*2023 · 23 citations*
158. **[Exploring Retrieval Augmented Generation in Arabic](https://md.research.elamur.ai/free/en/papers/el-beltagy-2024-exploring-retrieval-augmented-generation-arabic/) [Procedia Computer Science](https://md.research.elamur.ai/free/en/venues/procedia-computer-science/)**Samhaa R. El-Beltagy, Mohamed A. Abdallah*2024 · 22 citations*
159. **[A Large-Scale English Multi-Label Twitter Dataset for Cyberbullying and Online Abuse Detection](https://md.research.elamur.ai/free/en/papers/salawu-2021-large-scale-english-multi-label/) [Workshop on Online Abuse and Harms](https://md.research.elamur.ai/free/en/venues/woah/)**Semiu Salawu, Jo Lumsden, Yulan He*2021 · 22 citations*
160. **[Semantics of Multiword Expressions in Transformer-Based Models: A Survey](https://md.research.elamur.ai/free/en/papers/miletic-2024-semantics-multiword-expressions-transformer-based/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Filip Miletić, Sabine Schulte im Walde*2024 · 20 citations*
161. **[Few-Shot Anaphora Resolution in Scientific Protocols via Mixtures of In-Context Experts](https://md.research.elamur.ai/free/en/papers/le-2022-few-shot-anaphora-resolution-scientific/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Nghia T. Le, Fan Bai, Alan Ritter*2022 · 20 citations*
162. **[Unleashing the Power of Neural Discourse Parsers - A Context and Structure Aware Approach Using Large Scale Pretraining](https://md.research.elamur.ai/free/en/papers/guz-2020-unleashing-power-neural-discourse-parsers/) [International Conference on Computational Linguistics (COLING)](https://md.research.elamur.ai/free/en/venues/coling/)**Grigorii Guz, Patrick Huber, Giuseppe Carenini*2020 · 19 citations*
163. **[Does Commonsense help in detecting Sarcasm?](https://md.research.elamur.ai/free/en/papers/chowdhury-2021-does-commonsense-help-detecting-sarcasm/) [Workshop on Insights from Negative Results in NLP](https://md.research.elamur.ai/free/en/venues/insights/)**Somnath Basu Roy Chowdhury, Snigdha Chaturvedi*2021 · 17 citations*
164. [**AraCOVID19-MFH: Arabic COVID-19 Multi-label Fake News and Hate Speech Detection Dataset arXiv:2105.03143**Mohamed Seghir Hadj Ameur, Hassina Aliane*2021 · 16 citations*](https://md.research.elamur.ai/free/en/papers/ameur-2021-aracovid19-mfh-arabic-covid-19/)
165. **[When Truth Matters - Addressing Pragmatic Categories in Natural Language Inference (NLI) by Large Language Models (LLMs)](https://md.research.elamur.ai/free/en/papers/gubelmann-2023-when-truth-matters-addressing-pragmatic/) [Joint Conference on Lexical and Computational Semantics (*SEM)](https://md.research.elamur.ai/free/en/venues/starsem/)**Reto Gubelmann, Aikaterini-lida Kalouli, Christina Niklaus, et al.*2023 · 15 citations*
166. **[Pre-training Distillation for Large Language Models: A Design Space Exploration](https://md.research.elamur.ai/free/en/papers/peng-2025-pre-training-distillation-large-language/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Hao Peng, Xin Lv, Yushi Bai, et al.*2025 · 14 citations*
167. [**MiniPLM: Knowledge Distillation for Pre-Training Language Models arXiv:2410.17215**Yuxian Gu, Hao Zhou, Fandong Meng, et al.*2024 · 12 citations*](https://md.research.elamur.ai/free/en/papers/gu-2024-miniplm-knowledge-distillation-pre-training/)
168. **[Parsing Argumentative Structure in English-as-Foreign-Language Essays](https://md.research.elamur.ai/free/en/papers/putra-2021-parsing-argumentative-structure-english-as/) [Workshop on Innovative Use of NLP for Building Educational Applications (BEA)](https://md.research.elamur.ai/free/en/venues/bea/)**Jan Wira Gotama Putra, Simone Teufel, Takenobu Tokunaga*2021 · 12 citations*
169. **[How Are Idioms Processed Inside Transformer Language Models?](https://md.research.elamur.ai/free/en/papers/tian-2023-how-are-idioms-processed-inside/) [Joint Conference on Lexical and Computational Semantics (*SEM)](https://md.research.elamur.ai/free/en/venues/starsem/)**Ye Tian, Isobel James, Hye Son*2023 · 11 citations*
170. **[Speciesist Language and Nonhuman Animal Bias in English Masked Language Models](https://md.research.elamur.ai/free/en/papers/takeshita-2022-speciesist-language-nonhuman-animal-bias/) [Information Processing & Management](https://md.research.elamur.ai/free/en/venues/information-processing-management/)**Masashi Takeshita, Rafal Rzepka, Kenji Araki*2022 · 11 citations*
171. [**T-STAR: Truthful Style Transfer using AMR Graph as Intermediate Representation**Anubhav Jangra, Preksha Nema, Aravindan Raghuveer*2022 · 11 citations*](https://md.research.elamur.ai/free/en/papers/jangra-2022-t-star-truthful-style-transfer/)
172. **[EMO: Embedding Model Distillation via Intra-Model Relation and Optimal Transport Alignments](https://md.research.elamur.ai/free/en/papers/truong-2025-emo-embedding-model-distillation-intra/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Minh-Phuc Truong, Hai An Vu, Tu Vu, et al.*2025 · 10 citations*
173. [**Knowledge Distillation of Black-Box Large Language Models arXiv:2401.07013**Hongzhan Chen, Ruijun Chen, Yuqi Yi, et al.*2024 · 10 citations*](https://md.research.elamur.ai/free/en/papers/chen-2024-knowledge-distillation-black-box-large/)
174. [**Investigating Forgetting in Pre-Trained Representations Through Continual Learning arXiv:2305.05968**Yun Luo, Zhen Yang, Xuefeng Bai, et al.*2023 · 10 citations*](https://md.research.elamur.ai/free/en/papers/luo-2023-investigating-forgetting-pre-trained-representations/)
175. **[RTQA : Recursive Thinking for Complex Temporal Knowledge Graph Question Answering with Large Language Models](https://md.research.elamur.ai/free/en/papers/gong-2025-rtqa-recursive-thinking-complex-temporal/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Zhaoyan Gong, Juan Li, Zhiqiang Liu, et al.*2025 · 9 citations*
176. **[To Distill or Not to Distill? On the Robustness of Robust Knowledge Distillation](https://md.research.elamur.ai/free/en/papers/waheed-2024-distill-or-not-distill-robustness/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Abdul Waheed, Karima Kadaoui, Muhammad Abdul-Mageed*2024 · 8 citations*
177. **[Cross-lingual transfer for low-resource Arabic language understanding](https://md.research.elamur.ai/free/en/papers/abboud-2022-cross-lingual-transfer-low-resource/) [Arabic Natural Language Processing Workshop (WANLP)](https://md.research.elamur.ai/free/en/venues/wanlp/)**Khadige Abboud, Olga Golovneva, Christopher DiPersio*2022 · 8 citations*
178. [**SemEval-2022 Task 3: PreTENS — Evaluating Neural Networks on Presuppositional Semantic Knowledge**Roberto Zamparelli, Shammur Chowdhury, Dominique Brunato, et al.*2022 · 8 citations*](https://md.research.elamur.ai/free/en/papers/zamparelli-2022-semeval-2022-task-3-pretens/)
179. **[ArabicTransformer: Efficient Large Arabic Language Model with Funnel Transformer and ELECTRA Objective](https://md.research.elamur.ai/free/en/papers/alrowili-2021-arabictransformer-efficient-large-arabic-language/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Sultan Alrowili, K. Vijay-Shanker*2021 · 8 citations*
180. [**End-to-end Neural Coreference Resolution Revisited: A Simple yet Effective Baseline arXiv:2107.01700**Tuan Manh Lai, Trung Bui, Doo Soon Kim*2021 · 8 citations*](https://md.research.elamur.ai/free/en/papers/lai-2021-end-end-neural-coreference-resolution/)
181. **[Beyond Logits: Aligning Feature Dynamics for Effective Knowledge Distillation](https://md.research.elamur.ai/free/en/papers/gong-2025-beyond-logits-aligning-feature-dynamics/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Guoqiang Gong, Jiaxing Wang, Jin Xu, et al.*2025 · 7 citations*
182. **[Why Supervised Fine-Tuning Fails to Learn: A Systematic Study of Incomplete Learning in Large Language Models](https://md.research.elamur.ai/free/en/papers/xue-2026-why-supervised-fine-tuning-fails/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Chao Xue, Yao Wang, Mengqiao Liu, et al.*2026 · 6 citations*
183. **[Sparse Logit Sampling: Accelerating Knowledge Distillation in LLMs](https://md.research.elamur.ai/free/en/papers/anshumann-2025-sparse-logit-sampling-accelerating-knowledge/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Anshumann, Mohd Abbas Zaidi, Akhil Kedia, et al.*2025 · 6 citations*
184. [**Detecting Anti-Semitic Hate Speech using Transformer-based Large Language Models arXiv:2405.03794**Dengyi Liu, Minghao Wang, Andrew G. Catlin*2024 · 6 citations*](https://md.research.elamur.ai/free/en/papers/liu-2024-detecting-anti-semitic-hate-speech/)
185. **[PTUK-HULAT at ArAIEval Shared Task Fine-tuned Distilbert to Predict Disinformative Tweets](https://md.research.elamur.ai/free/en/papers/jaber-2023-ptuk-hulat-araieval-shared-task/) [Arabic Natural Language Processing Conference (ArabicNLP)](https://md.research.elamur.ai/free/en/venues/arabicnlp/)**Areej Jaber, Paloma Martinez*2023 · 6 citations*
186. **[IISERB Brains at SemEval-2022 Task 6: A Deep-learning Framework to Identify Intended Sarcasm in English](https://md.research.elamur.ai/free/en/papers/shekhawat-2022-iiserb-brains-semeval-2022-task/) [International Workshop on Semantic Evaluation (SemEval)](https://md.research.elamur.ai/free/en/venues/semeval/)**Tanuj Shekhawat, Manoj Kumar, Udaybhan Rathore, et al.*2022 · 6 citations*
187. **[Vulnerability of LLMs to Vertically Aligned Text Manipulations](https://md.research.elamur.ai/free/en/papers/li-2025-vulnerability-llms-vertically-aligned-text/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Zhecheng Li, Yiwei Wang, Bryan Hooi, et al.*2025 · 5 citations*
188. **[NovBench: Evaluating Large Language Models on Academic Paper Novelty Assessment](https://md.research.elamur.ai/free/en/papers/wu-2026-novbench-evaluating-large-language-models/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Wenqing Wu, Yi Zhao, Yuzhuo Wang, et al.*2026 · 4 citations*
189. **[EHSAN: Leveraging ChatGPT in a Hybrid Framework for Arabic Aspect-Based Sentiment Analysis in Healthcare](https://md.research.elamur.ai/free/en/papers/alamoudi-2025-ehsan-leveraging-chatgpt-hybrid-framework/) [Lecture notes in computer science](https://md.research.elamur.ai/free/en/venues/lecture-notes-in-computer-science/)**Eman Alamoudi, Ellis Solaiman*2025 · 4 citations*
190. **[MultiCoPIE: A Multilingual Corpus of Potentially Idiomatic Expressions for Cross-lingual PIE Disambiguation](https://md.research.elamur.ai/free/en/papers/sentsova-2025-multicopie-multilingual-corpus-potentially-idiomatic/) [Workshop on Multiword Expressions (MWE)](https://md.research.elamur.ai/free/en/venues/mwe/)**Uliana Sentsova, Debora Ciminari, Josef van Genabith, et al.*2025 · 4 citations*
191. **[ReDASPersuasion at ArAIEval Shared Task: Multilingual and Monolingual Models For Arabic Persuasion Detection](https://md.research.elamur.ai/free/en/papers/qachfar-2023-redaspersuasion-araieval-shared-task-multilingual/) [Arabic Natural Language Processing Conference (ArabicNLP)](https://md.research.elamur.ai/free/en/venues/arabicnlp/)**Fatima Zahra Qachfar, Rakesh Verma*2023 · 4 citations*
192. **[Agree, Disagree, Explain: Decomposing Human Label Variation in NLI through the Lens of Explanations](https://md.research.elamur.ai/free/en/papers/hong-2026-agree-disagree-explain-decomposing-human/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Pingjun Hong, Beiduo Chen, Siyao Peng, et al.*2026 · 3 citations*
193. [**Modeling the Evolution of English Noun Compounds with Feature-Rich Diachronic Compositionality Prediction**Filip Miletić, Sabine Schulte im Walde*2025 · 3 citations*](https://md.research.elamur.ai/free/en/papers/miletic-2025-modeling-evolution-english-noun-compounds/)
194. [**TafBERTa: Learning Grammatical Rules from Small-Scale Language Acquisition Data in Hebrew**Anita Gelboim, Elior Sulem*2025 · 3 citations*](https://md.research.elamur.ai/free/en/papers/gelboim-2025-tafberta-learning-grammatical-rules-small/)
195. [**Embible: Reconstruction of Ancient Hebrew and Aramaic Texts Using Transformers**Niv Fono, Harel Moshayof, Eldar Karol, et al.*2024 · 3 citations*](https://md.research.elamur.ai/free/en/papers/fono-2024-embible-reconstruction-ancient-hebrew-aramaic/)
196. **[Evaluating Customized vs. Generalist Transformer-based Models for Legal Contract Classification](https://md.research.elamur.ai/free/en/papers/singh-2026-evaluating-customized-vs-generalist-transformer/) [Workshop on Customizable NLP: Progress and Challenges in Customizing NLP for a Domain, Application, Group, or Individual](https://md.research.elamur.ai/free/en/venues/customnlp4u/)**Amrita Singh, H. Suhan Karaca, Aditya Joshi, et al.*2026 · 2 citations*
197. **[LEAF: Knowledge Distillation of Text Embedding Models with Teacher-Aligned Representations](https://md.research.elamur.ai/free/en/papers/vujanic-2026-leaf-knowledge-distillation-text-embedding/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Robin Vujanic, Thomas Rückstieß*2026 · 2 citations*
198. **[MALicious INTent Dataset and Inoculating LLMs for Enhanced Disinformation Detection](https://md.research.elamur.ai/free/en/papers/modzelewski-2026-malicious-intent-dataset-inoculating-llms/) [Conference of the European Chapter of the Association for Computational Linguistics (EACL)](https://md.research.elamur.ai/free/en/venues/eacl/)**Arkadiusz Modzelewski, Witold Sosnowski, Eleni Papadopulos, et al.*2026 · 2 citations*
199. **[Maximizing Local Entropy Where It Matters: Prefix-Aware Localized LLM Unlearning](https://md.research.elamur.ai/free/en/papers/zhai-2026-maximizing-local-entropy-where-it/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Naixin Zhai, Pengyang Shao, Binbin Zheng, et al.*2026 · 2 citations*
200. **[Multi-Granularity Semantic Revision for Large Language Model Distillation](https://md.research.elamur.ai/free/en/papers/liu-2026-multi-granularity-semantic-revision-large/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Xiaoyu Liu, Yun Zhang, Wei Li, et al.*2026 · 2 citations*

[← prev](https://md.research.elamur.ai/free/en/papers/sanh-2019-distilbert-distilled-version-bert-smaller/cited-by/) [1](https://md.research.elamur.ai/free/en/papers/sanh-2019-distilbert-distilled-version-bert-smaller/cited-by/) **2** [3](https://md.research.elamur.ai/free/en/papers/sanh-2019-distilbert-distilled-version-bert-smaller/cited-by/page/3/) [next →](https://md.research.elamur.ai/free/en/papers/sanh-2019-distilbert-distilled-version-bert-smaller/cited-by/page/3/)

- [← back to](https://md.research.elamur.ai/free/en/papers/sanh-2019-distilbert-distilled-version-bert-smaller/) — DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter