---
title: "Who cites RoBERTa: A — page 2 — 9,837 citing papers"
description: "9,837 papers of the Elamur Research library cite «RoBERTa: A Robustly Optimized BERT Pretraining Approach», found in their…"
canonical: "https://research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/cited-by/page/2/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/cited-by/page/2/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [RoBERTa: A Robustly Optimized BERT Pretraining…](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/) / [cited by](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/cited-by/) / page 2

# Who cites «RoBERTa: A Robustly Optimized BERT Pretraining Approach»

9,837 papers of our library cite it (of the 149,439 whose bibliographies we read) · 1406 with a page here, the most cited first · page 2 of 15 · 101–200 · [← back to «RoBERTa: A Robustly Optimized BERT Pretraining Approach»](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/)

[← prev](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/cited-by/) [1](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/cited-by/) **2** [3](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/cited-by/page/3/) … [10](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/cited-by/page/10/) … [15](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/cited-by/page/15/) [next →](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/cited-by/page/3/)

101. **[TaBERT: Pretraining for Joint Understanding of Textual and Tabular Data](https://md.research.elamur.ai/free/en/papers/yin-2020-tabert-pretraining-joint-understanding-textual/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Pengcheng Yin, Graham Neubig, Wen-tau Yih, et al.*2020 · 692 citations*
102. [**Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval arXiv:2007.00808**Lee Xiong, Chenyan Xiong, Ye Li, et al.*2020 · 688 citations*](https://md.research.elamur.ai/free/en/papers/xiong-2020-approximate-nearest-neighbor-negative-contrastive/)
103. **[KILT: a Benchmark for Knowledge Intensive Language Tasks](https://md.research.elamur.ai/free/en/papers/petroni-2020-kilt-benchmark-knowledge-intensive-language/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Fabio Petroni, Aleksandra Piktus, Angela Fan, et al.*2020 · 678 citations*
104. [**Well-Read Students Learn Better: On the Importance of Pre-training Compact Models arXiv:1908.08962**Iulia Turc, Ming-Wei Chang, Kenton Lee, et al.*2019 · 678 citations*](https://md.research.elamur.ai/free/en/papers/turc-2019-well-read-students-learn-better/)
105. **[Improving Multi-hop Question Answering over Knowledge Graphs using Knowledge Base Embeddings](https://md.research.elamur.ai/free/en/papers/saxena-2020-improving-multi-hop-question-answering/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Apoorv Saxena, Aditay Tripathi, Partha Talukdar*2020 · 671 citations*
106. **[CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data](https://md.research.elamur.ai/free/en/papers/wenzek-2020-ccnet-extracting-high-quality-monolingual/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Guillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau, et al.*2020 · 666 citations*
107. [**K-Adapter: Infusing Knowledge into Pre-Trained Models with Adapters arXiv:2002.01808**Ruize Wang, Duyu Tang, Nan Duan, et al.*2020 · 666 citations*](https://md.research.elamur.ai/free/en/papers/wang-2020-k-adapter-infusing-knowledge-pre/)
108. **[ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer](https://md.research.elamur.ai/free/en/papers/yan-2021-consert-contrastive-framework-self-supervised/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Yuanmeng Yan, Rumei Li, Sirui Wang, et al.*2021 · 647 citations*
109. **[Efficient Intent Detection with Dual Sentence Encoders](https://md.research.elamur.ai/free/en/papers/casanueva-2020-efficient-intent-detection-dual-sentence/) [Workshop on Natural Language Processing for Conversational AI](https://md.research.elamur.ai/free/en/venues/nlp4convai/)**Iñigo Casanueva, Tadas Temčinas, Daniela Gerz, et al.*2020 · 628 citations*
110. [**Towards a Unified View of Parameter-Efficient Transfer Learning arXiv:2110.04366**Junxian He, Chunting Zhou, Xuezhe Ma, et al.*2021 · 619 citations*](https://md.research.elamur.ai/free/en/papers/he-2021-towards-unified-view-parameter-efficient/)
111. [**Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model arXiv:2201.11990**Shaden Smith, Mostofa Patwary, Brandon Norick, et al.*2022 · 615 citations*](https://md.research.elamur.ai/free/en/papers/smith-2022-using-deepspeed-megatron-train-megatron/)
112. [**Text Embeddings by Weakly-Supervised Contrastive Pre-training arXiv:2212.03533**Liang Wang, Nan Yang, Xiaolong Huang, et al.*2022 · 598 citations*](https://md.research.elamur.ai/free/en/papers/wang-2022-text-embeddings-weakly-supervised-contrastive/)
113. **[A Robustly Optimized BERT Pre-training Approach with Post-training](https://md.research.elamur.ai/free/en/papers/zhuang-2021-robustly-optimized-bert-pre-training/) [Chinese National Conference on Computational Linguistics](https://md.research.elamur.ai/free/en/venues/ccl/)**Liu Zhuang, Lin Wayne, Shi Ya, et al.*2021 · 587 citations*
114. **[Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models](https://md.research.elamur.ai/free/en/papers/ni-2021-sentence-t5-scalable-sentence-encoders/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Jianmo Ni, Gustavo Hernandez Abrego, Noah Constant, et al.*2021 · 584 citations*
115. **[AudioLM: a Language Modeling Approach to Audio Generation](https://md.research.elamur.ai/free/en/papers/borsos-2022-audiolm-language-modeling-approach-audio/) [IEEE/ACM Transactions on Audio Speech and Language Processing](https://md.research.elamur.ai/free/en/venues/ieee-acm-transactions-on-audio-speech-and-language-processing/)**Zalan Borsos, Raphael Marinier, Damien Vincent, et al.*2022 · 581 citations*
116. **[Learning How to Ask: Querying LMs with Mixtures of Soft Prompts](https://md.research.elamur.ai/free/en/papers/qin-2021-learning-ask-querying-lms-mixtures/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Guanghui Qin, Jason Eisner*2021 · 573 citations*
117. **[PhoBERT: Pre-trained language models for Vietnamese](https://md.research.elamur.ai/free/en/papers/nguyen-2020-phobert-pre-trained-language-models/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Dat Quoc Nguyen, Anh Tuan Nguyen*2020 · 571 citations*
118. **[Extractive Summarization as Text Matching](https://md.research.elamur.ai/free/en/papers/zhong-2020-extractive-summarization-as-text-matching/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Ming Zhong, Pengfei Liu, Yiran Chen, et al.*2020 · 570 citations*
119. [**What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams arXiv:2009.13081**Di Jin, Eileen Pan, Nassim Oufattole, et al.*2020 · 567 citations*](https://md.research.elamur.ai/free/en/papers/jin-2020-disease-this-patient-have-large/)
120. [**MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models arXiv:2309.12284**Longhui Yu, Weisen Jiang, Han Shi, et al.*2023 · 565 citations*](https://md.research.elamur.ai/free/en/papers/yu-2023-metamath-bootstrap-your-own-mathematical/)
121. **[Leveraging Pre-trained Checkpoints for Sequence Generation Tasks](https://md.research.elamur.ai/free/en/papers/rothe-2019-leveraging-pre-trained-checkpoints-sequence/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Sascha Rothe, Shashi Narayan, Aliaksei Severyn*2019 · 564 citations*
122. [**Reducing Transformer Depth on Demand with Structured Dropout arXiv:1909.11556**Angela Fan, Edouard Grave, Armand Joulin*2019 · 560 citations*](https://md.research.elamur.ai/free/en/papers/fan-2019-reducing-transformer-depth-demand-structured/)
123. [**How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection arXiv:2301.07597**Biyang Guo, Xin Zhang, Ziyuan Wang, et al.*2023 · 555 citations*](https://md.research.elamur.ai/free/en/papers/guo-2023-close-chatgpt-human-experts-comparison/)
124. **[Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning](https://md.research.elamur.ai/free/en/papers/aghajanyan-2021-intrinsic-dimensionality-explains-effectiveness-language/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Armen Aghajanyan, Sonal Gupta, Luke Zettlemoyer*2021 · 542 citations*
125. [**Evaluating Models' Local Decision Boundaries via Contrast Sets**Matt Gardner, Yoav Artzi, Victoria Basmov, et al.*2020 · 531 citations*](https://md.research.elamur.ai/free/en/papers/gardner-2020-evaluating-models-local-decision-boundaries/)
126. **[How Does NLP Benefit Legal System: A Summary of Legal Artificial Intelligence](https://md.research.elamur.ai/free/en/papers/zhong-2020-how-does-nlp-benefit-legal/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Haoxi Zhong, Chaojun Xiao, Cunchao Tu, et al.*2020 · 518 citations*
127. **[DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations](https://md.research.elamur.ai/free/en/papers/giorgi-2021-declutr-deep-contrastive-learning-unsupervised/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**John Giorgi, Osvald Nitski, Bo Wang, et al.*2021 · 516 citations*
128. [**TabFact: A Large-scale Dataset for Table-based Fact Verification arXiv:1909.02164**Wenhu Chen, Hongmin Wang, Jianshu Chen, et al.*2019 · 515 citations*](https://md.research.elamur.ai/free/en/papers/chen-2019-tabfact-large-scale-dataset-table/)
129. **[FinQA: A Dataset of Numerical Reasoning over Financial Data](https://md.research.elamur.ai/free/en/papers/chen-2021-finqa-dataset-numerical-reasoning-over/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Zhiyu Chen, Wenhu Chen, Charese Smiley, et al.*2021 · 513 citations*
130. **[A-OKVQA: A Benchmark for Visual Question Answering using World Knowledge](https://md.research.elamur.ai/free/en/papers/schwenk-2022-okvqa-benchmark-visual-question-answering/) [Lecture notes in computer science](https://md.research.elamur.ai/free/en/venues/lecture-notes-in-computer-science/)**Dustin Schwenk, Apoorv Khandelwal, Christopher Clark, et al.*2022 · 509 citations*
131. [**Few-shot Learning with Multilingual Language Models arXiv:2112.10668**Xi Victoria Lin, Todor Mihaylov, Mikel Artetxe, et al.*2021 · 500 citations*](https://md.research.elamur.ai/free/en/papers/lin-2021-few-shot-learning-multilingual-language/)
132. **[BBQ: A hand-built bias benchmark for question answering](https://md.research.elamur.ai/free/en/papers/parrish-2022-bbq-hand-built-bias-benchmark/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Alicia Parrish, Angelica Chen, Nikita Nangia, et al.*2022 · 497 citations*
133. [**GLaM: Efficient Scaling of Language Models with Mixture-of-Experts arXiv:2112.06905**Nan Du, Yanping Huang, Andrew M. Dai, et al.*2021 · 496 citations*](https://md.research.elamur.ai/free/en/papers/du-2021-glam-efficient-scaling-language-models/)
134. **[DeeBERT: Dynamic Early Exiting for Accelerating BERT Inference](https://md.research.elamur.ai/free/en/papers/xin-2020-deebert-dynamic-early-exiting-accelerating/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Ji Xin, Raphael Tang, Jaejun Lee, et al.*2020 · 487 citations*
135. [**DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature arXiv:2301.11305**Eric Mitchell, Yoonho Lee, Alexander Khazatsky, et al.*2023 · 481 citations*](https://md.research.elamur.ai/free/en/papers/mitchell-2023-detectgpt-zero-shot-machine-generated/)
136. [**SimVLM: Simple Visual Language Model Pretraining with Weak Supervision arXiv:2108.10904**Zirui Wang, Jiahui Yu, Adams Wei Yu, et al.*2021 · 481 citations*](https://md.research.elamur.ai/free/en/papers/wang-2021-simvlm-simple-visual-language-model/)
137. **[HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training](https://md.research.elamur.ai/free/en/papers/li-2020-hero-hierarchical-encoder-video-language/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Linjie Li, Yen-Chun Chen, Yu Cheng, et al.*2020 · 481 citations*
138. [**ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic arXiv:2101.01785 · ACL-2021 camera ready version**Muhammad Abdul-Mageed, AbdelRahim Elmadany, El Moatez Billah Nagoudi*2021 · 478 citations*](https://md.research.elamur.ai/free/en/papers/abdul-mageed-2021-arbert-marbert-deep-bidirectional-transformers/)
139. [**Data Augmentation using Pre-trained Transformer Models arXiv:2003.02245**Varun Kumar, Ashutosh Choudhary, Eunah Cho*2020 · 474 citations*](https://md.research.elamur.ai/free/en/papers/kumar-2020-data-augmentation-using-pre-trained/)
140. **[PPT: Pre-trained Prompt Tuning for Few-shot Learning](https://md.research.elamur.ai/free/en/papers/gu-2022-ppt-pre-trained-prompt-tuning/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Yuxian Gu, Xu Han, Zhiyuan Liu, et al.*2022 · 471 citations*
141. [**P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks arXiv:2110.07602**Xiao Liu, Kaixuan Ji, Yicheng Fu, et al.*2021 · 469 citations*](https://md.research.elamur.ai/free/en/papers/liu-2021-p-tuning-v2-prompt-tuning/)
142. [**OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Framework arXiv:2202.03052**Peng Wang, An Yang, Rui Men, et al.*2022 · 467 citations*](https://md.research.elamur.ai/free/en/papers/wang-2022-ofa-unifying-architectures-tasks-modalities/)
143. **[From Zero to Hero: On the Limitations of Zero-Shot Language Transfer with Multilingual Transformers](https://md.research.elamur.ai/free/en/papers/lauscher-2020-zero-hero-limitations-zero-shot/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Anne Lauscher, Vinit Ravishankar, Ivan Vulić, et al.*2020 · 463 citations*
144. **[Are All Languages Created Equal in Multilingual BERT?](https://md.research.elamur.ai/free/en/papers/wu-2020-are-all-languages-created-equal/) [Workshop on Representation Learning for NLP](https://md.research.elamur.ai/free/en/venues/repl4nlp/)**Shijie Wu, Mark Dredze*2020 · 460 citations*
145. **[LinkBERT: Pretraining Language Models with Document Links](https://md.research.elamur.ai/free/en/papers/yasunaga-2022-linkbert-pretraining-language-models-document/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Michihiro Yasunaga, Jure Leskovec, Percy Liang*2022 · 458 citations*
146. [**Ethical and social risks of harm from Language Models arXiv:2112.04359**Laura Weidinger, John Mellor, Maribeth Rauh, et al.*2021 · 456 citations*](https://md.research.elamur.ai/free/en/papers/weidinger-2021-ethical-social-risks-harm-language/)
147. **[LLM-Pruner: On the Structural Pruning of Large Language Models](https://md.research.elamur.ai/free/en/papers/ma-2023-llm-pruner-structural-pruning-large/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Xinyin Ma, Gongfan Fang, Xinchao Wang*2023 · 452 citations*
148. **[Pretrained Transformers Improve Out-of-Distribution Robustness](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-pretrained-transformers-improve-out-distribution/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Dan Hendrycks, Xiaoyuan Liu, Eric Wallace, et al.*2020 · 452 citations*
149. [**FreeLB: Enhanced Adversarial Training for Natural Language Understanding arXiv:1909.11764**Chen Zhu, Yu Cheng, Zhe Gan, et al.*2019 · 452 citations*](https://md.research.elamur.ai/free/en/papers/zhu-2019-freelb-enhanced-adversarial-training-natural/)
150. **[How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Models](https://md.research.elamur.ai/free/en/papers/rust-2021-how-good-is-your-tokenizer/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Phillip Rust, Jonas Pfeiffer, Ivan Vulić, et al.*2021 · 449 citations*
151. **[Masked Language Model Scoring](https://md.research.elamur.ai/free/en/papers/salazar-2020-masked-language-model-scoring/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Julian Salazar, Davis Liang, Toan Q. Nguyen, et al.*2020 · 447 citations*
152. [**Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents arXiv:2201.07207**Wenlong Huang, Pieter Abbeel, Deepak Pathak, et al.*2022 · 443 citations*](https://md.research.elamur.ai/free/en/papers/huang-2022-language-models-zero-shot-planners/)
153. **[oLMpics — On what Language Model Pre-training Captures](https://md.research.elamur.ai/free/en/papers/talmor-2019-olmpics-language-model-pre-training/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Alon Talmor, Yanai Elazar, Yoav Goldberg, et al.*2019 · 440 citations*
154. **[Self-Alignment Pretraining for Biomedical Entity Representations](https://md.research.elamur.ai/free/en/papers/liu-2021-self-alignment-pretraining-biomedical-entity/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Fangyu Liu, Ehsan Shareghi, Zaiqiao Meng, et al.*2021 · 437 citations*
155. **[GPT-NER: Named Entity Recognition via Large Language Models](https://md.research.elamur.ai/free/en/papers/wang-2025-gpt-ner-named-entity-recognition/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Shuhe Wang, Xiaofei Sun, Xiaoya Li, et al.*2025 · 436 citations*
156. [**InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks arXiv:2312.14238**Zhe Chen, Jiannan Wu, Wenhai Wang, et al.*2023 · 435 citations*](https://md.research.elamur.ai/free/en/papers/chen-2023-internvl-scaling-up-vision/)
157. **[Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly](https://md.research.elamur.ai/free/en/papers/kassner-2020-negated-misprimed-probes-pretrained-language/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Nora Kassner, Hinrich Schütze*2020 · 435 citations*
158. **[Security and Privacy Challenges of Large Language Models: A Survey](https://md.research.elamur.ai/free/en/papers/das-2024-security-privacy-challenges-large-language/) [ACM Computing Surveys](https://md.research.elamur.ai/free/en/venues/acm-computing-surveys/)**Badhan Chandra Das, M. Hadi Amini, Yanzhao Wu*2024 · 424 citations*
159. [**Tulu 3: Pushing Frontiers in Open Language Model Post-Training arXiv:2411.15124**Nathan Lambert, Jacob Morrison, Valentina Pyatkin, et al.*2024 · 423 citations*](https://md.research.elamur.ai/free/en/papers/lambert-2024-tulu-3-pushing-frontiers-open/)
160. **[MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classification](https://md.research.elamur.ai/free/en/papers/chen-2020-mixtext-linguistically-informed-interpolation-hidden/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Jiaao Chen, Zichao Yang, Diyi Yang*2020 · 417 citations*
161. **[Factual Probing Is MASK: Learning vs. Learning to Recall](https://md.research.elamur.ai/free/en/papers/zhong-2021-factual-probing-is-mask-learning/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Zexuan Zhong, Dan Friedman, Danqi Chen*2021 · 415 citations*
162. **[A Computational Approach to Understanding Empathy Expressed in Text-Based Mental Health Support](https://md.research.elamur.ai/free/en/papers/sharma-2020-computational-approach-understanding-empathy-expressed/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Ashish Sharma, Adam Miner, David Atkins, et al.*2020 · 412 citations*
163. **[Do Prompt-Based Models Really Understand the Meaning of Their Prompts?](https://md.research.elamur.ai/free/en/papers/webson-2022-do-prompt-based-models-really/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Albert Webson, Ellie Pavlick*2022 · 407 citations*
164. **[Compressing BERT: Studying the Effects of Weight Pruning on Transfer Learning](https://md.research.elamur.ai/free/en/papers/gordon-2020-compressing-bert-studying-effects-weight/) [Workshop on Representation Learning for NLP](https://md.research.elamur.ai/free/en/venues/repl4nlp/)**Mitchell Gordon, Kevin Duh, Nicholas Andrews*2020 · 407 citations*
165. [**BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models arXiv:2106.10199**Elad Ben-Zaken, Shauli Ravfogel, Yoav Goldberg*2021 · 406 citations*](https://md.research.elamur.ai/free/en/papers/ben-zaken-2021-bitfit-simple-parameter-efficient-fine/)
166. **[LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning](https://md.research.elamur.ai/free/en/papers/liu-2020-logiqa-challenge-dataset-machine-reading/) [International Joint Conference on Artificial Intelligence](https://md.research.elamur.ai/free/en/venues/international-joint-conference-on-artificial-intelligence/)**Jian Liu, Leyang Cui, Hanmeng Liu, et al.*2020 · 404 citations*
167. **[A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios](https://md.research.elamur.ai/free/en/papers/hedderich-2021-survey-recent-approaches-natural-language/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Michael A. Hedderich, Lukas Lange, Heike Adel, et al.*2021 · 401 citations*
168. **[LongT5: Efficient Text-To-Text Transformer for Long Sequences](https://md.research.elamur.ai/free/en/papers/guo-2022-longt5-efficient-text-text-transformer/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Mandy Guo, Joshua Ainslie, David Uthus, et al.*2022 · 399 citations*
169. **[TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Finance](https://md.research.elamur.ai/free/en/papers/zhu-2021-tat-qa-question-answering-benchmark/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Fengbin Zhu, Wenqiang Lei, Youcheng Huang, et al.*2021 · 397 citations*
170. **[Lawformer: A Pre-trained Language Model for Chinese Legal Long Documents](https://md.research.elamur.ai/free/en/papers/xiao-2021-lawformer-pre-trained-language-model/) [AI Open](https://md.research.elamur.ai/free/en/venues/ai-open/)**Chaojun Xiao, Xueyu Hu, Zhiyuan Liu, et al.*2021 · 396 citations*
171. [**Aligning AI With Shared Human Values arXiv:2008.02275**Dan Hendrycks, Collin Burns, Steven Basart, et al.*2020 · 395 citations*](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-aligning-ai-shared-human-values/)
172. [**UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training arXiv:2002.12804**Hangbo Bao, Li Dong, Furu Wei, et al.*2020 · 395 citations*](https://md.research.elamur.ai/free/en/papers/bao-2020-unilmv2-pseudo-masked-language-models/)
173. [**Green AI arXiv:1907.10597**Roy Schwartz, Jesse Dodge, Noah A. Smith, et al.*2019 · 389 citations*](https://md.research.elamur.ai/free/en/papers/schwartz-2019-green-ai/)
174. [**Evaluation of Text Generation: A Survey arXiv:2006.14799**Asli Celikyilmaz, Elizabeth Clark, Jianfeng Gao*2020 · 388 citations*](https://md.research.elamur.ai/free/en/papers/celikyilmaz-2020-evaluation-text-generation-survey/)
175. **[ETC: Encoding Long and Structured Inputs in Transformers](https://md.research.elamur.ai/free/en/papers/ainslie-2020-etc-encoding-long-structured-inputs/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Joshua Ainslie, Santiago Ontañón, Chris Alberti, et al.*2020 · 386 citations*
176. **[German's Next Language Model](https://md.research.elamur.ai/free/en/papers/chan-2020-german-s-next-language-model/) [International Conference on Computational Linguistics (COLING)](https://md.research.elamur.ai/free/en/venues/coling/)**Branden Chan, Stefan Schweter, Timo Möller*2020 · 385 citations*
177. **[FastBERT: a Self-distilling BERT with Adaptive Inference Time](https://md.research.elamur.ai/free/en/papers/liu-2020-fastbert-self-distilling-bert-adaptive/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Weijie Liu, Peng Zhou, Zhiruo Wang, et al.*2020 · 383 citations*
178. **[Transformers as Soft Reasoners over Language](https://md.research.elamur.ai/free/en/papers/clark-2020-transformers-soft-reasoners-over-language/) [International Joint Conference on Artificial Intelligence](https://md.research.elamur.ai/free/en/venues/international-joint-conference-on-artificial-intelligence/)**Peter Clark, Oyvind Tafjord, Kyle Richardson*2020 · 382 citations*
179. [**Splitwise: Efficient generative LLM inference using phase splitting ISCA 2024**Pratyush Patel, Esha Choukse, Chaojie Zhang, et al.*2023 · 379 citations*](https://md.research.elamur.ai/free/en/papers/patel-2023-splitwise-efficient-generative-llm-inference/)
180. **[The GEM Benchmark: Natural Language Generation, its Evaluation and Metrics](https://md.research.elamur.ai/free/en/papers/gehrmann-2021-gem-benchmark-natural-language-generation/) [Workshop on Generation, Evaluation and Metrics (GEM)](https://md.research.elamur.ai/free/en/venues/gem/)**Sebastian Gehrmann, Tosin Adewumi, Karmanya Aggarwal, et al.*2021 · 369 citations*
181. [**Revisiting Few-sample BERT Fine-tuning arXiv:2006.05987**Tianyi Zhang, Felix Wu, Arzoo Katiyar, et al.*2020 · 368 citations*](https://md.research.elamur.ai/free/en/papers/zhang-2020-revisiting-few-sample-bert-fine/)
182. **[Parameter-Efficient Transfer Learning with Diff Pruning](https://md.research.elamur.ai/free/en/papers/guo-2021-parameter-efficient-transfer-learning-diff/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Demi Guo, Alexander Rush, Yoon Kim*2021 · 367 citations*
183. [**The Interplay of Variant, Size, and Task Type in Arabic Pre-trained Language Models arXiv:2103.06678**Go Inoue, Bashar Alhafni, Nurpeiis Baimukan, et al.*2021 · 365 citations*](https://md.research.elamur.ai/free/en/papers/inoue-2021-interplay-variant-size-task-type/)
184. **[Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification](https://md.research.elamur.ai/free/en/papers/hu-2022-knowledgeable-prompt-tuning-incorporating-knowledge/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Shengding Hu, Ning Ding, Huadong Wang, et al.*2022 · 362 citations*
185. **[TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue](https://md.research.elamur.ai/free/en/papers/wu-2020-tod-bert-pre-trained-natural/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Chien-Sheng Wu, Steven C.H. Hoi, Richard Socher, et al.*2020 · 361 citations*
186. [**BERTje: A Dutch BERT Model arXiv:1912.09582**Wietse de Vries, Andreas van Cranenburgh, Arianna Bisazza, et al.*2019 · 361 citations*](https://md.research.elamur.ai/free/en/papers/de-vries-2019-bertje-dutch-bert-model/)
187. **[BRIO: Bringing Order to Abstractive Summarization](https://md.research.elamur.ai/free/en/papers/liu-2022-brio-bringing-order-abstractive-summarization/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Yixin Liu, Pengfei Liu, Dragomir Radev, et al.*2022 · 360 citations*
188. [**Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference arXiv:2412.13663**Benjamin Warner, Antoine Chaffin, Benjamin Clavié, et al.*2024 · 358 citations*](https://md.research.elamur.ai/free/en/papers/warner-2024-smarter-better-faster-longer-modern/)
189. **[XGLUE: A New Benchmark Dataset for Cross-lingual Pre-training, Understanding and Generation](https://md.research.elamur.ai/free/en/papers/liang-2020-xglue-new-benchmark-dataset-cross/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Yaobo Liang, Nan Duan, Yeyun Gong, et al.*2020 · 358 citations*
190. [**Reformulating Unsupervised Style Transfer as Paraphrase Generation**Kalpesh Krishna, John Wieting, Mohit Iyyer*2020 · 350 citations*](https://md.research.elamur.ai/free/en/papers/krishna-2020-reformulating-unsupervised-style-transfer-as/)
191. **[Symbolic Knowledge Distillation: from General Language Models to Commonsense Models](https://md.research.elamur.ai/free/en/papers/west-2022-symbolic-knowledge-distillation-general-language/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Peter West, Chandra Bhagavatula, Jack Hessel, et al.*2022 · 349 citations*
192. **[Time-Aware Language Models as Temporal Knowledge Bases](https://md.research.elamur.ai/free/en/papers/dhingra-2022-time-aware-language-models-as/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Bhuwan Dhingra, Jeremy R. Cole, Julian Martin Eisenschlos, et al.*2022 · 349 citations*
193. [**RobBERT: a Dutch RoBERTa-based Language Model arXiv:2001.06286**Pieter Delobelle, Thomas Winters, Bettina Berendt*2020 · 349 citations*](https://md.research.elamur.ai/free/en/papers/delobelle-2020-robbert-dutch-roberta-based-language/)
194. [**UL2: Unifying Language Learning Paradigms arXiv:2205.05131**Yi Tay, Mostafa Dehghani, Vinh Q. Tran, et al.*2022 · 347 citations*](https://md.research.elamur.ai/free/en/papers/tay-2022-ul2-unifying-language-learning-paradigms/)
195. **[Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space](https://md.research.elamur.ai/free/en/papers/geva-2022-transformer-feed-forward-layers-build/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Mor Geva, Avi Caciularu, Kevin Ro Wang, et al.*2022 · 346 citations*
196. **[Directed Acyclic Graph Network for Conversational Emotion Recognition](https://md.research.elamur.ai/free/en/papers/shen-2021-directed-acyclic-graph-network-conversational/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Weizhou Shen, Siyue Wu, Yunyi Yang, et al.*2021 · 346 citations*
197. **[SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization](https://md.research.elamur.ai/free/en/papers/liu-2021-simcls-simple-framework-contrastive-learning/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Yixin Liu, Pengfei Liu*2021 · 341 citations*
198. [**WARP: Word-level Adversarial ReProgramming arXiv:2101.00121**Karen Hambardzumyan, Hrant Khachatrian, Jonathan May*2021 · 340 citations*](https://md.research.elamur.ai/free/en/papers/hambardzumyan-2021-warp-word-level-adversarial-reprogramming/)
199. **[RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/deng-2022-rlprompt-optimizing-discrete-text-prompts/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Mingkai Deng, Jianyu Wang, Cheng-Ping Hsieh, et al.*2022 · 338 citations*
200. [**Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers arXiv:2301.02111**Chengyi Wang, Sanyuan Chen, Yu Wu, et al.*2023 · 335 citations*](https://md.research.elamur.ai/free/en/papers/wang-2023-neural-codec-language-models-zero/)

[← prev](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/cited-by/) [1](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/cited-by/) **2** [3](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/cited-by/page/3/) … [10](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/cited-by/page/10/) … [15](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/cited-by/page/15/) [next →](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/cited-by/page/3/)

- [← back to](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/) — RoBERTa: A Robustly Optimized BERT Pretraining Approach