---
title: "Who cites ALBERT: A Lite BERT for — 1,861 citing papers"
description: "1,861 papers of the Elamur Research library cite «ALBERT: A Lite BERT for Self-supervised Learning of Language Representations», found…"
canonical: "https://research.elamur.ai/free/en/papers/lan-2019-albert-lite-bert-self-supervised/cited-by/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/lan-2019-albert-lite-bert-self-supervised/cited-by/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [ALBERT: A Lite BERT for Self-supervised Learning…](https://md.research.elamur.ai/free/en/papers/lan-2019-albert-lite-bert-self-supervised/) / cited by

# Who cites «ALBERT: A Lite BERT for Self-supervised Learning of Language Representations»

1,861 papers of our library cite it (of the 149,439 whose bibliographies we read) · 262 with a page here, the most cited first · page 1 of 3 · 1–100 · [← back to «ALBERT: A Lite BERT for Self-supervised Learning of Language Representations»](https://md.research.elamur.ai/free/en/papers/lan-2019-albert-lite-bert-self-supervised/)

**1** [2](https://md.research.elamur.ai/free/en/papers/lan-2019-albert-lite-bert-self-supervised/cited-by/page/2/) [3](https://md.research.elamur.ai/free/en/papers/lan-2019-albert-lite-bert-self-supervised/cited-by/page/3/) [next →](https://md.research.elamur.ai/free/en/papers/lan-2019-albert-lite-bert-self-supervised/cited-by/page/2/)

1. **[Transformers: State-of-the-Art Natural Language Processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Thomas Wolf, Lysandre Debut, Victor Sanh, et al.*2020 · 15,755 citations*
2. [**Language Models are Few-Shot Learners arXiv:2005.14165**Tom B. Brown, Benjamin Mann, Nick Ryder, et al.*2020 · 14,178 citations*](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
3. **[BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://md.research.elamur.ai/free/en/papers/lewis-2020-bart-denoising-sequence-sequence-pre/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Mike Lewis, Yinhan Liu, Naman Goyal, et al.*2020 · 12,856 citations*
4. [**Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer arXiv:1910.10683**Colin Raffel, Noam Shazeer, Adam Roberts, et al.*2019 · 11,325 citations*](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/)
5. **[Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://md.research.elamur.ai/free/en/papers/liu-2021-pre-train-prompt-predict-systematic/) [ACM Computing Surveys](https://md.research.elamur.ai/free/en/venues/acm-computing-surveys/)**Pengfei Liu, Weizhe Yuan, Jinlan Fu, et al.*2021 · 5,423 citations*
6. [**XLNet: Generalized Autoregressive Pretraining for Language Understanding arXiv:1906.08237**Zhilin Yang, Zihang Dai, Yiming Yang, et al.*2019 · 3,857 citations*](https://md.research.elamur.ai/free/en/papers/yang-2019-xlnet-generalized-autoregressive-pretraining-language/)
7. [**Measuring Massive Multitask Language Understanding arXiv:2009.03300**Dan Hendrycks, Collin Burns, Steven Basart, et al.*2020 · 3,620 citations*](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/)
8. [**Scaling Laws for Neural Language Models arXiv:2001.08361**Jared Kaplan, Sam McCandlish, Tom Henighan, et al.*2020 · 3,434 citations*](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/)
9. [**On the Opportunities and Risks of Foundation Models arXiv:2108.07258**Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, et al.*2021 · 3,160 citations*](https://md.research.elamur.ai/free/en/papers/bommasani-2021-opportunities-risks-foundation-models/)
10. [**RoFormer: Enhanced Transformer with Rotary Position Embedding arXiv:2104.09864**Jianlin Su, Yu Lu, Shengfeng Pan, et al.*2021 · 3,028 citations*](https://md.research.elamur.ai/free/en/papers/su-2021-roformer-enhanced-transformer-rotary-position/)
11. **[UNITER: UNiversal Image-TExt Representation Learning](https://md.research.elamur.ai/free/en/papers/chen-2019-uniter-universal-image-text-representation/) [Lecture notes in computer science](https://md.research.elamur.ai/free/en/venues/lecture-notes-in-computer-science/)**Yen-Chun Chen, Linjie Li, Licheng Yu, et al.*2019 · 2,418 citations*
12. **[TinyBERT: Distilling BERT for Natural Language Understanding](https://md.research.elamur.ai/free/en/papers/jiao-2020-tinybert-distilling-bert-natural-language/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Xiaoqi Jiao, Yichun Yin, Lifeng Shang, et al.*2020 · 2,378 citations*
13. [**DeBERTa: Decoding-enhanced BERT with Disentangled Attention arXiv:2006.03654**Pengcheng He, Xiaodong Liu, Jianfeng Gao, et al.*2020 · 1,828 citations*](https://md.research.elamur.ai/free/en/papers/he-2020-deberta-decoding-enhanced-bert-disentangled/)
14. [**Pre-trained Models for Natural Language Processing: A Survey arXiv:2003.08271 · SCIENCE CHINA Technological Sciences,2020, 63, 1872-1897**Xipeng Qiu, Tianxiang Sun, Yige Xu, et al.*2020 · 1,759 citations*](https://md.research.elamur.ai/free/en/papers/qiu-2020-pre-trained-models-natural-language/)
15. [**Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models arXiv:2206.04615 · Transactions on Machine Learning Research, May/2022, https://openreview.net/forum?id=uyTL5Bvosj**Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, et al.*2022 · 1,756 citations*](https://md.research.elamur.ai/free/en/papers/srivastava-2022-imitation-game-quantifying-extrapolating-capabilities/)
16. [**BERTopic: Neural topic modeling with a class-based TF-IDF procedure arXiv:2203.05794**Maarten Grootendorst*2022 · 1,737 citations*](https://md.research.elamur.ai/free/en/papers/grootendorst-2022-bertopic-neural-topic-modeling-class/)
17. **[Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://md.research.elamur.ai/free/en/papers/min-2021-recent-advances-natural-language-processing/) [ACM Computing Surveys](https://md.research.elamur.ai/free/en/venues/acm-computing-surveys/)**Bonan Min, Hayley Ross, Elior Sulem, et al.*2021 · 1,556 citations*
18. **[Unifying Large Language Models and Knowledge Graphs: A Roadmap](https://md.research.elamur.ai/free/en/papers/pan-2023-unifying-large-language-models-knowledge/) [IEEE Transactions on Knowledge and Data Engineering](https://md.research.elamur.ai/free/en/venues/ieee-transactions-on-knowledge-and-data-engineering/)**Shirui Pan, Linhao Luo, Yufei Wang, et al.*2023 · 1,551 citations*
19. **[Efficient Transformers: A Survey](https://md.research.elamur.ai/free/en/papers/tay-2020-efficient-transformers-survey/) [ACM Computing Surveys](https://md.research.elamur.ai/free/en/venues/acm-computing-surveys/)**Yi Tay, Mostafa Dehghani, Dara Bahri, et al.*2020 · 1,539 citations*
20. **[LEGAL-BERT: The Muppets straight out of Law School](https://md.research.elamur.ai/free/en/papers/chalkidis-2020-legal-bert-muppets-straight-out/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Ilias Chalkidis, Manos Fergadiotis, Prodromos Malakasiotis, et al.*2020 · 1,493 citations*
21. **[Pre-Training with Whole Word Masking for Chinese BERT](https://md.research.elamur.ai/free/en/papers/cui-2019-pre-training-whole-word-masking/) [IEEE/ACM Transactions on Audio Speech and Language Processing](https://md.research.elamur.ai/free/en/venues/ieee-acm-transactions-on-audio-speech-and-language-processing/)**Yiming Cui, Wanxiang Che, Ting Liu, et al.*2019 · 1,490 citations*
22. **[GLM: General Language Model Pretraining with Autoregressive Blank Infilling](https://md.research.elamur.ai/free/en/papers/du-2022-glm-general-language-model-pretraining/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Zhengxiao Du, Yujie Qian, Xiao Liu, et al.*2022 · 1,455 citations*
23. [**LaMDA: Language Models for Dialog Applications arXiv:2201.08239**Romal Thoppilan, Daniel De Freitas, Jamie Hall, et al.*2022 · 1,344 citations*](https://md.research.elamur.ai/free/en/papers/thoppilan-2022-lamda-language-models-dialog-applications/)
24. [**DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing arXiv:2111.09543**Pengcheng He, Jianfeng Gao, Weizhu Chen*2021 · 1,320 citations*](https://md.research.elamur.ai/free/en/papers/he-2021-debertav3-improving-deberta-using-electra/)
25. [**AraBERT: Transformer-based Model for Arabic Language Understanding arXiv:2003.00104**Wissam Antoun, Fady Baly, Hazem Hajj*2020 · 1,304 citations*](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/)
26. **[CamemBERT: a Tasty French Language Model](https://md.research.elamur.ai/free/en/papers/martin-2019-camembert-tasty-french-language-model/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Louis Martin, Benjamin Muller, Pedro Javier Ortiz Suárez, et al.*2019 · 1,269 citations*
27. **[It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/schick-2021-it-s-not-just-size/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Timo Schick, Hinrich Schütze*2021 · 1,132 citations*
28. **[Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models](https://md.research.elamur.ai/free/en/papers/zhang-2023-siren-s-song-ai-ocean/) [Computational Linguistics](https://md.research.elamur.ai/free/en/venues/cl/)**Yue Zhang, Yafu Li, Leyang Cui, et al.*2023 · 977 citations*
29. **[CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models](https://md.research.elamur.ai/free/en/papers/nangia-2020-crows-pairs-challenge-dataset-measuring/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Nikita Nangia, Clara Vania, Rasika Bhalerao, et al.*2020 · 897 citations*
30. **[UNIFIEDQA: Crossing Format Boundaries with a Single QA System](https://md.research.elamur.ai/free/en/papers/khashabi-2020-unifiedqa-crossing-format-boundaries-single/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Daniel Khashabi, Sewon Min, Tushar Khot, et al.*2020 · 857 citations*
31. [**Unsupervised Dense Information Retrieval with Contrastive Learning arXiv:2112.09118**Gautier Izacard, Mathilde Caron, Lucas Hosseini, et al.*2021 · 836 citations*](https://md.research.elamur.ai/free/en/papers/izacard-2021-unsupervised-dense-information-retrieval-contrastive/)
32. **[Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond](https://md.research.elamur.ai/free/en/papers/yang-2023-harnessing-power-llms-practice-survey/) [ACM Transactions on Knowledge Discovery from Data](https://md.research.elamur.ai/free/en/venues/acm-transactions-on-knowledge-discovery-from-data/)**Jingfeng Yang, Hongye Jin, Ruixiang Tang, et al.*2023 · 814 citations*
33. [**Revisiting Pre-Trained Models for Chinese Natural Language Processing arXiv:2004.13922**Yiming Cui, Wanxiang Che, Ting Liu, et al.*2020 · 790 citations*](https://md.research.elamur.ai/free/en/papers/cui-2020-revisiting-pre-trained-models-chinese/)
34. **[IndicNLPSuite: Monolingual Corpora, Evaluation Benchmarks and Pre-trained Multilingual Language Models for Indian Languages](https://md.research.elamur.ai/free/en/papers/kakwani-2020-indicnlpsuite-monolingual-corpora-evaluation-benchmarks/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Divyanshu Kakwani, Anoop Kunchukuttan, Satish Golla, et al.*2020 · 732 citations*
35. **[CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data](https://md.research.elamur.ai/free/en/papers/wenzek-2020-ccnet-extracting-high-quality-monolingual/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Guillaume Wenzek, Marie-Anne Lachaux, Alexis Conneau, et al.*2020 · 666 citations*
36. **[A Frustratingly Easy Approach for Entity and Relation Extraction](https://md.research.elamur.ai/free/en/papers/zhong-2020-frustratingly-easy-approach-entity-relation/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Zexuan Zhong, Danqi Chen*2020 · 631 citations*
37. **[A Robustly Optimized BERT Pre-training Approach with Post-training](https://md.research.elamur.ai/free/en/papers/zhuang-2021-robustly-optimized-bert-pre-training/) [Chinese National Conference on Computational Linguistics](https://md.research.elamur.ai/free/en/venues/ccl/)**Liu Zhuang, Lin Wayne, Shi Ya, et al.*2021 · 587 citations*
38. [**What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams arXiv:2009.13081**Di Jin, Eileen Pan, Nassim Oufattole, et al.*2020 · 567 citations*](https://md.research.elamur.ai/free/en/papers/jin-2020-disease-this-patient-have-large/)
39. **[Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning](https://md.research.elamur.ai/free/en/papers/aghajanyan-2021-intrinsic-dimensionality-explains-effectiveness-language/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Armen Aghajanyan, Sonal Gupta, Luke Zettlemoyer*2021 · 542 citations*
40. **[DeCLUTR: Deep Contrastive Learning for Unsupervised Textual Representations](https://md.research.elamur.ai/free/en/papers/giorgi-2021-declutr-deep-contrastive-learning-unsupervised/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**John Giorgi, Osvald Nitski, Bo Wang, et al.*2021 · 516 citations*
41. **[HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training](https://md.research.elamur.ai/free/en/papers/li-2020-hero-hierarchical-encoder-video-language/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Linjie Li, Yen-Chun Chen, Yu Cheng, et al.*2020 · 481 citations*
42. [**ARBERT & MARBERT: Deep Bidirectional Transformers for Arabic arXiv:2101.01785 · ACL-2021 camera ready version**Muhammad Abdul-Mageed, AbdelRahim Elmadany, El Moatez Billah Nagoudi*2021 · 478 citations*](https://md.research.elamur.ai/free/en/papers/abdul-mageed-2021-arbert-marbert-deep-bidirectional-transformers/)
43. [**Large Language Models: A Survey arXiv:2402.06196**Shervin Minaee, Tomas Mikolov, Narjes Nikzad, et al.*2024 · 471 citations*](https://md.research.elamur.ai/free/en/papers/minaee-2024-large-language-models-survey/)
44. **[LLM-Pruner: On the Structural Pruning of Large Language Models](https://md.research.elamur.ai/free/en/papers/ma-2023-llm-pruner-structural-pruning-large/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Xinyin Ma, Gongfan Fang, Xinchao Wang*2023 · 452 citations*
45. **[Pretrained Transformers Improve Out-of-Distribution Robustness](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-pretrained-transformers-improve-out-distribution/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Dan Hendrycks, Xiaoyuan Liu, Eric Wallace, et al.*2020 · 452 citations*
46. [**FreeLB: Enhanced Adversarial Training for Natural Language Understanding arXiv:1909.11764**Chen Zhu, Yu Cheng, Zhe Gan, et al.*2019 · 452 citations*](https://md.research.elamur.ai/free/en/papers/zhu-2019-freelb-enhanced-adversarial-training-natural/)
47. **[Experience Grounds Language](https://md.research.elamur.ai/free/en/papers/bisk-2020-experience-grounds-language/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Yonatan Bisk, Ari Holtzman, Jesse Thomason, et al.*2020 · 439 citations*
48. **[GPT-NER: Named Entity Recognition via Large Language Models](https://md.research.elamur.ai/free/en/papers/wang-2025-gpt-ner-named-entity-recognition/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Shuhe Wang, Xiaofei Sun, Xiaoya Li, et al.*2025 · 436 citations*
49. **[Factual Probing Is MASK: Learning vs. Learning to Recall](https://md.research.elamur.ai/free/en/papers/zhong-2021-factual-probing-is-mask-learning/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Zexuan Zhong, Dan Friedman, Danqi Chen*2021 · 415 citations*
50. [**True Few-Shot Learning with Language Models arXiv:2105.11447**Ethan Perez, Douwe Kiela, Kyunghyun Cho*2021 · 412 citations*](https://md.research.elamur.ai/free/en/papers/perez-2021-true-few-shot-learning-language/)
51. **[Do Prompt-Based Models Really Understand the Meaning of Their Prompts?](https://md.research.elamur.ai/free/en/papers/webson-2022-do-prompt-based-models-really/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Albert Webson, Ellie Pavlick*2022 · 407 citations*
52. [**Aligning AI With Shared Human Values arXiv:2008.02275**Dan Hendrycks, Collin Burns, Steven Basart, et al.*2020 · 395 citations*](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-aligning-ai-shared-human-values/)
53. [**UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training arXiv:2002.12804**Hangbo Bao, Li Dong, Furu Wei, et al.*2020 · 395 citations*](https://md.research.elamur.ai/free/en/papers/bao-2020-unilmv2-pseudo-masked-language-models/)
54. **[ETC: Encoding Long and Structured Inputs in Transformers](https://md.research.elamur.ai/free/en/papers/ainslie-2020-etc-encoding-long-structured-inputs/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Joshua Ainslie, Santiago Ontañón, Chris Alberti, et al.*2020 · 386 citations*
55. **[German's Next Language Model](https://md.research.elamur.ai/free/en/papers/chan-2020-german-s-next-language-model/) [International Conference on Computational Linguistics (COLING)](https://md.research.elamur.ai/free/en/venues/coling/)**Branden Chan, Stefan Schweter, Timo Möller*2020 · 385 citations*
56. **[FastBERT: a Self-distilling BERT with Adaptive Inference Time](https://md.research.elamur.ai/free/en/papers/liu-2020-fastbert-self-distilling-bert-adaptive/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Weijie Liu, Peng Zhou, Zhiruo Wang, et al.*2020 · 383 citations*
57. [**Revisiting Few-sample BERT Fine-tuning arXiv:2006.05987**Tianyi Zhang, Felix Wu, Arzoo Katiyar, et al.*2020 · 368 citations*](https://md.research.elamur.ai/free/en/papers/zhang-2020-revisiting-few-sample-bert-fine/)
58. **[Parameter-Efficient Transfer Learning with Diff Pruning](https://md.research.elamur.ai/free/en/papers/guo-2021-parameter-efficient-transfer-learning-diff/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Demi Guo, Alexander Rush, Yoon Kim*2021 · 367 citations*
59. **[Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification](https://md.research.elamur.ai/free/en/papers/hu-2022-knowledgeable-prompt-tuning-incorporating-knowledge/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Shengding Hu, Ning Ding, Huadong Wang, et al.*2022 · 362 citations*
60. [**BERTje: A Dutch BERT Model arXiv:1912.09582**Wietse de Vries, Andreas van Cranenburgh, Arianna Bisazza, et al.*2019 · 361 citations*](https://md.research.elamur.ai/free/en/papers/de-vries-2019-bertje-dutch-bert-model/)
61. [**RobBERT: a Dutch RoBERTa-based Language Model arXiv:2001.06286**Pieter Delobelle, Thomas Winters, Bettina Berendt*2020 · 349 citations*](https://md.research.elamur.ai/free/en/papers/delobelle-2020-robbert-dutch-roberta-based-language/)
62. [**ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation arXiv:2107.02137**Yu Sun, Shuohuan Wang, Shikun Feng, et al.*2021 · 332 citations*](https://md.research.elamur.ai/free/en/papers/sun-2021-ernie-3-0-large/)
63. **[Unsupervised Corpus Aware Language Model Pre-training for Dense Passage Retrieval](https://md.research.elamur.ai/free/en/papers/gao-2022-unsupervised-corpus-aware-language-model/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Luyu Gao, Jamie Callan*2022 · 328 citations*
64. **[How many data points is a prompt worth?](https://md.research.elamur.ai/free/en/papers/scao-2021-how-many-data-points-is/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Teven Le Scao, Alexander Rush*2021 · 320 citations*
65. **[From Pretraining Data to Language Models to Downstream Tasks: Tracking the Trails of Political Biases Leading to Unfair NLP Models](https://md.research.elamur.ai/free/en/papers/feng-2023-pretraining-data-language-models-downstream/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Shangbin Feng, Chan Young Park, Yuhan Liu, et al.*2023 · 311 citations*
66. **[Byte Pair Encoding is Suboptimal for Language Model Pretraining](https://md.research.elamur.ai/free/en/papers/bostrom-2020-byte-pair-encoding-is-suboptimal/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Kaj Bostrom, Greg Durrett*2020 · 310 citations*
67. **[SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer](https://md.research.elamur.ai/free/en/papers/vu-2022-spot-better-frozen-model-adaptation/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Tu Vu, Brian Lester, Noah Constant, et al.*2022 · 306 citations*
68. **[ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers](https://md.research.elamur.ai/free/en/papers/yao-2022-zeroquant-efficient-affordable-post-training/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Zhewei Yao, Reza Yazdani Aminabadi, Minjia Zhang, et al.*2022 · 304 citations*
69. **[SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization](https://md.research.elamur.ai/free/en/papers/jiang-2020-smart-robust-efficient-fine-tuning/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Haoming Jiang, Pengcheng He, Weizhu Chen, et al.*2020 · 304 citations*
70. **[Scalable Multi-Hop Relational Reasoning for Knowledge-Aware Question Answering](https://md.research.elamur.ai/free/en/papers/feng-2020-scalable-multi-hop-relational-reasoning/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Yanlin Feng, Xinyue Chen, Bill Yuchen Lin, et al.*2020 · 300 citations*
71. [**FlauBERT: Unsupervised Language Model Pre-training for French arXiv:1912.05372**Hang Le, Loïc Vial, Jibril Frej, et al.*2019 · 299 citations*](https://md.research.elamur.ai/free/en/papers/le-2019-flaubert-unsupervised-language-model-pre/)
72. **[BanglaBERT: Language Model Pretraining and Benchmarks for Low-Resource Language Understanding Evaluation in Bangla](https://md.research.elamur.ai/free/en/papers/bhattacharjee-2022-banglabert-language-model-pretraining-benchmarks/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Abhik Bhattacharjee, Tahmid Hasan, Wasi Ahmad, et al.*2022 · 296 citations*
73. **[AdapterDrop: On the Efficiency of Adapters in Transformers](https://md.research.elamur.ai/free/en/papers/ruckle-2021-adapterdrop-efficiency-adapters-transformers/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Andreas Rücklé, Gregor Geigle, Max Glockner, et al.*2021 · 296 citations*
74. **[PAQ: 65 Million Probably-Asked Questions and What You Can Do With Them](https://md.research.elamur.ai/free/en/papers/lewis-2021-paq-65-million-probably-asked/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Patrick Lewis, Yuxiang Wu, Linqing Liu, et al.*2021 · 288 citations*
75. **[Two are Better than One: Joint Entity and Relation Extraction with Table-Sequence Encoders](https://md.research.elamur.ai/free/en/papers/wang-2020-two-are-better-than-one/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Jue Wang, Wei Lu*2020 · 288 citations*
76. **[Spelling Error Correction with Soft-Masked BERT](https://md.research.elamur.ai/free/en/papers/zhang-2020-spelling-error-correction-soft-masked/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Shaohua Zhang, Haoran Huang, Jicong Liu, et al.*2020 · 284 citations*
77. **[TRUE: Re-evaluating Factual Consistency Evaluation](https://md.research.elamur.ai/free/en/papers/honovich-2022-true-re-evaluating-factual-consistency/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Or Honovich, Roee Aharoni, Jonathan Herzig, et al.*2022 · 279 citations*
78. **[Text Classification Using Label Names Only: A Language Model Self-Training Approach](https://md.research.elamur.ai/free/en/papers/meng-2020-text-classification-label-names-only/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Yu Meng, Yunyi Zhang, Jiaxin Huang, et al.*2020 · 274 citations*
79. **[Attention is Not Only a Weight: Analyzing Transformers with Vector Norms](https://md.research.elamur.ai/free/en/papers/kobayashi-2020-attention-is-not-only-weight/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Goro Kobayashi, Tatsuki Kuribayashi, Sho Yokoi, et al.*2020 · 270 citations*
80. [**Perceiver: General Perception with Iterative Attention arXiv:2103.03206**Andrew Jaegle, Felix Gimeno, Andrew Brock, et al.*2021 · 269 citations*](https://md.research.elamur.ai/free/en/papers/jaegle-2021-perceiver-general-perception-iterative-attention/)
81. **[ConveRT: Efficient and Accurate Conversational Representations from Transformers](https://md.research.elamur.ai/free/en/papers/henderson-2020-convert-efficient-accurate-conversational-representations/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Matthew Henderson, Iñigo Casanueva, Nikola Mrkšić, et al.*2020 · 269 citations*
82. **[Injecting Numerical Reasoning Skills into Language Models](https://md.research.elamur.ai/free/en/papers/geva-2020-injecting-numerical-reasoning-skills-into/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Mor Geva, Ankit Gupta, Jonathan Berant*2020 · 269 citations*
83. [**CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review arXiv:2103.06268**Dan Hendrycks, Collin Burns, Anya Chen, et al.*2021 · 268 citations*](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-cuad-expert-annotated-nlp-dataset/)
84. **[IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understanding](https://md.research.elamur.ai/free/en/papers/wilie-2020-indonlu-benchmark-resources-evaluating-indonesian/) [Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (AACL)](https://md.research.elamur.ai/free/en/venues/aacl/)**Bryan Wilie, Karissa Vincentio, Genta Indra Winata, et al.*2020 · 266 citations*
85. [**DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale arXiv:2201.05596**Samyam Rajbhandari, Conglong Li, Zhewei Yao, et al.*2022 · 261 citations*](https://md.research.elamur.ai/free/en/papers/rajbhandari-2022-deepspeed-moe-advancing-mixture-experts/)
86. **[Condenser: a Pre-training Architecture for Dense Retrieval](https://md.research.elamur.ai/free/en/papers/gao-2021-condenser-pre-training-architecture-dense/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Luyu Gao, Jamie Callan*2021 · 261 citations*
87. **[SemEval-2020 Task 8: Memotion Analysis- the Visuo-Lingual Metaphor!](https://md.research.elamur.ai/free/en/papers/sharma-2020-semeval-2020-task-8-memotion/) [International Workshop on Semantic Evaluation (SemEval)](https://md.research.elamur.ai/free/en/venues/semeval/)**Chhavi Sharma, Deepesh Bhageria, William Scott, et al.*2020 · 261 citations*
88. **[CLIFF: Contrastive Learning for Improving Faithfulness and Factuality in Abstractive Summarization](https://md.research.elamur.ai/free/en/papers/cao-2021-cliff-contrastive-learning-improving-faithfulness/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Shuyang Cao, Lu Wang*2021 · 260 citations*
89. [**Measuring and Reducing Gendered Correlations in Pre-trained Models arXiv:2010.06032**Kellie Webster, Xuezhi Wang, Ian Tenney, et al.*2020 · 254 citations*](https://md.research.elamur.ai/free/en/papers/webster-2020-measuring-reducing-gendered-correlations-pre/)
90. **[GPT-RE: In-context Learning for Relation Extraction using Large Language Models](https://md.research.elamur.ai/free/en/papers/wan-2023-gpt-re-context-learning-relation/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Zhen Wan, Fei Cheng, Zhuoyuan Mao, et al.*2023 · 253 citations*
91. [**A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models arXiv:2401.01313**S.M Towhidul Islam Tonmoy, S M Mehedi Zaman, Vinija Jain, et al.*2024 · 249 citations*](https://md.research.elamur.ai/free/en/papers/tonmoy-2024-comprehensive-survey-hallucination-mitigation-techniques/)
92. **[Learning Span-Level Interactions for Aspect Sentiment Triplet Extraction](https://md.research.elamur.ai/free/en/papers/xu-2021-learning-span-level-interactions-aspect/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Lu Xu, Yew Ken Chia, Lidong Bing*2021 · 246 citations*
93. **[ZeroGen: Efficient Zero-shot Learning via Dataset Generation](https://md.research.elamur.ai/free/en/papers/ye-2022-zerogen-efficient-zero-shot-learning/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Jiacheng Ye, Jiahui Gao, Qintong Li, et al.*2022 · 245 citations*
94. [**CLUE: A Chinese Language Understanding Evaluation Benchmark arXiv:2004.05986**Liang Xu, Hai Hu, Xuanwei Zhang, et al.*2020 · 244 citations*](https://md.research.elamur.ai/free/en/papers/xu-2020-clue-chinese-language-understanding/)
95. **[Hierarchical Graph Network for Multi-hop Question Answering](https://md.research.elamur.ai/free/en/papers/fang-2020-hierarchical-graph-network-multi-hop/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Yuwei Fang, Siqi Sun, Zhe Gan, et al.*2020 · 243 citations*
96. **[BERT-of-Theseus: Compressing BERT by Progressive Module Replacing](https://md.research.elamur.ai/free/en/papers/xu-2020-bert-theseus-compressing-bert-progressive/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Canwen Xu, Wangchunshu Zhou, Tao Ge, et al.*2020 · 240 citations*
97. [**How Context Affects Language Models' Factual Predictions arXiv:2005.04611**Fabio Petroni, Patrick Lewis, Aleksandra Piktus, et al.*2020 · 234 citations*](https://md.research.elamur.ai/free/en/papers/petroni-2020-context-affects-language-models-factual/)
98. **[A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages](https://md.research.elamur.ai/free/en/papers/suarez-2020-monolingual-approach-contextualized-word-embeddings/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Pedro Javier Ortiz Suárez, Laurent Romary, Benoît Sagot*2020 · 222 citations*
99. **[What Happens To BERT Embeddings During Fine-tuning?](https://md.research.elamur.ai/free/en/papers/merchant-2020-what-happens-bert-embeddings-during/) [BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP](https://md.research.elamur.ai/free/en/venues/blackboxnlp/)**Amil Merchant, Elahe Rahimtoroghi, Ellie Pavlick, et al.*2020 · 220 citations*
100. **[Cutting Down on Prompts and Parameters: Simple Few-Shot Learning with Language Models](https://md.research.elamur.ai/free/en/papers/iv-2022-cutting-down-prompts-parameters-simple/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Robert Logan IV, Ivana Balazevic, Eric Wallace, et al.*2022 · 216 citations*

**1** [2](https://md.research.elamur.ai/free/en/papers/lan-2019-albert-lite-bert-self-supervised/cited-by/page/2/) [3](https://md.research.elamur.ai/free/en/papers/lan-2019-albert-lite-bert-self-supervised/cited-by/page/3/) [next →](https://md.research.elamur.ai/free/en/papers/lan-2019-albert-lite-bert-self-supervised/cited-by/page/2/)

- [← back to](https://md.research.elamur.ai/free/en/papers/lan-2019-albert-lite-bert-self-supervised/) — ALBERT: A Lite BERT for Self-supervised Learning of Language Representations