---
title: "Language models & pretraining — papers, page 2"
description: "Language models & pretraining: 1044 papers of the Elamur Research library · page 2 of 12 · 26–125, the most cited first."
canonical: "https://research.elamur.ai/free/en/topics/language-models/page/2/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/topics/language-models/page/2/)

[home](https://md.research.elamur.ai/free/en/) / [topics](https://md.research.elamur.ai/free/en/topics/) / [Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) / page 2

# 🧠 Language models & pretraining

1044 papers · page 2 of 12 · 26–125 · the most cited first · [all topics](https://md.research.elamur.ai/free/en/topics/)

Models that learn a language from raw text, and how they are pretrained, tuned and prompted.

[← prev](https://md.research.elamur.ai/free/en/topics/language-models/) [1](https://md.research.elamur.ai/free/en/topics/language-models/) **2** [3](https://md.research.elamur.ai/free/en/topics/language-models/page/3/) … [12](https://md.research.elamur.ai/free/en/topics/language-models/page/12/) [next →](https://md.research.elamur.ai/free/en/topics/language-models/page/3/)

26. [**PaLM: Scaling Language Modeling with Pathways**Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, et al.*2022 · 2 603 citations*](https://md.research.elamur.ai/free/en/papers/chowdhery-2022-palm-scaling-language-modeling-pathways/)
27. [**SpanBERT: Improving Pre-training by Representing and Predicting Spans**Mandar Joshi, Danqi Chen, Yinhan Liu, et al.*2019 · 2 580 citations*](https://md.research.elamur.ai/free/en/papers/joshi-2019-spanbert-improving-pre-training-representing/)
28. [**Improved Baselines with Visual Instruction Tuning**Haotian Liu, Chunyuan Li, Yuheng Li, et al.*2023 · 2 557 citations*](https://md.research.elamur.ai/free/en/papers/liu-2023-improved-baselines-visual-instruction-tuning/)
29. [**Lost in the Middle: How Language Models Use Long Contexts**Nelson F. Liu, Kevin Lin, John Hewitt, et al.*2023 · 2 540 citations*](https://md.research.elamur.ai/free/en/papers/liu-2023-lost-middle-language-models-use/)
30. [**Evaluating Large Language Models Trained on Code**Mark Chen, Jerry Tworek, Heewoo Jun, et al.*2021 · 2 529 citations*](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
31. [**ReAct: Synergizing Reasoning and Acting in Language Models**Shunyu Yao, Jeffrey Zhao, Dian Yu, et al.*2022 · 2 493 citations*](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/)
32. [**Visual Instruction Tuning**Haotian Liu, Chunyuan Li, Qingyang Wu, et al.*2023 · 2 487 citations*](https://md.research.elamur.ai/free/en/papers/liu-2023-visual-instruction-tuning/)
33. [**Direct Preference Optimization: Your Language Model is Secretly a Reward Model**Rafael Rafailov, Archit Sharma, Eric Mitchell, et al.*2023 · 2 478 citations*](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/)
34. [**Sparks of Artificial General Intelligence: Early experiments with GPT-4**Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, et al.*2023 · 2 389 citations*](https://md.research.elamur.ai/free/en/papers/bubeck-2023-sparks-artificial-general-intelligence-early/)
35. [**GPT-4o System Card**OpenAI, Aaron Hurst, Adam Lerer, et al.*2024 · 2 221 citations*](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/)
36. [**ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks**Jiasen Lu, Dhruv Batra, Devi Parikh, et al.*2019 · 2 217 citations*](https://md.research.elamur.ai/free/en/papers/lu-2019-vilbert-pretraining-task-agnostic-visiolinguistic/)
37. [**Emergent Abilities of Large Language Models**Jason Wei, Yi Tay, Rishi Bommasani, et al.*2022 · 2 215 citations*](https://md.research.elamur.ai/free/en/papers/wei-2022-emergent-abilities-large-language-models/)
38. [**Scaling Instruction-Finetuned Language Models**Hyung Won Chung, Le Hou, Shayne Longpre, et al.*2022 · 2 176 citations*](https://md.research.elamur.ai/free/en/papers/chung-2022-scaling-instruction-finetuned-language-models/)
39. [**BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models**Junnan Li, Dongxu Li, Silvio Savarese, et al.*2023 · 2 114 citations*](https://md.research.elamur.ai/free/en/papers/li-2023-blip-2-bootstrapping-language-image/)
40. [**Flamingo: a Visual Language Model for Few-Shot Learning**Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, et al.*2022 · 2 108 citations*](https://md.research.elamur.ai/free/en/papers/alayrac-2022-flamingo-visual-language-model-few/)
41. [**Toolformer: Language Models Can Teach Themselves to Use Tools**Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, et al.*2023 · 2 020 citations*](https://md.research.elamur.ai/free/en/papers/schick-2023-toolformer-language-models-can-teach/)
42. [**DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models**Zhihong Shao, Peiyi Wang, Qihao Zhu, et al.*2024 · 2 013 citations*](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/)
43. [**Making Pre-trained Language Models Better Few-shot Learners**Tianyu Gao, Adam Fisch, Danqi Chen*2021 · 1 952 citations*](https://md.research.elamur.ai/free/en/papers/gao-2021-making-pre-trained-language-models/)
44. [**AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts**Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, et al.*2020 · 1 879 citations*](https://md.research.elamur.ai/free/en/papers/shin-2020-autoprompt-eliciting-knowledge-language-models/)
45. [**A Language Modeling Approach to Information Retrieval**Jay M. Ponte, W. Bruce Croft*1998 · 1 867 citations*](https://md.research.elamur.ai/free/en/papers/ponte-1998-language-modeling-approach-information/)
46. [**Character-Aware Neural Language Models**Yoon Kim, Yacine Jernite, David Sontag, et al.*2015 · 1 852 citations*](https://md.research.elamur.ai/free/en/papers/kim-2015-character-aware-neural-language-models/)
47. [**Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks**Xiujun Li, Xi Yin, Chunyuan Li, et al.*2020 · 1 827 citations*](https://md.research.elamur.ai/free/en/papers/li-2020-oscar-object-semantics-aligned-pre/)
48. [**Scaling Laws for Neural Language Models**Jared Kaplan, Sam McCandlish, Tom Henighan, et al.*2020 · 1 814 citations*](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/)
49. [**PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization**Jingqing Zhang, Yao Zhao, Mohammad Saleh, et al.*2019 · 1 806 citations*](https://md.research.elamur.ai/free/en/papers/zhang-2019-pegasus-pre-training-extracted-gap/)
50. [**KenLM: Faster and Smaller Language Model Queries**Kenneth Heafield*2011 · 1 759 citations*](https://md.research.elamur.ai/free/en/papers/heafield-2011-kenlm-faster-smaller-language-model/)
51. [**OPT: Open Pre-trained Transformer Language Models**Susan Zhang, Stephen Roller, Naman Goyal, et al.*2022 · 1 709 citations*](https://md.research.elamur.ai/free/en/papers/zhang-2022-opt-open-pre-trained-transformer/)
52. [**Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models**Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, et al.*2022 · 1 695 citations*](https://md.research.elamur.ai/free/en/papers/srivastava-2022-imitation-game-quantifying-extrapolating-capabilities/)
53. [**Retrieval-Augmented Generation for Large Language Models: A Survey**Yunfan Gao, Yun Xiong, Xinyu Gao, et al.*2023 · 1 647 citations*](https://md.research.elamur.ai/free/en/papers/gao-2023-retrieval-augmented-generation-large-language/)
54. [**Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey**Bonan Min, Hayley Ross, Elior Sulem, et al.*2021 · 1 548 citations*](https://md.research.elamur.ai/free/en/papers/min-2021-recent-advances-natural-language-processing/)
55. [**Unifying Large Language Models and Knowledge Graphs: A Roadmap**Shirui Pan, Linhao Luo, Yufei Wang, et al.*2023 · 1 545 citations*](https://md.research.elamur.ai/free/en/papers/pan-2023-unifying-large-language-models-knowledge/)
56. [**How Can We Know What Language Models Know?**Zhengbao Jiang, Frank F. Xu, Jun Araki, et al.*2020 · 1 493 citations*](https://md.research.elamur.ai/free/en/papers/jiang-2020-how-can-we-know-what/)
57. [**Pre-Training with Whole Word Masking for Chinese BERT**Yiming Cui, Wanxiang Che, Ting Liu, et al.*2019 · 1 485 citations*](https://md.research.elamur.ai/free/en/papers/cui-2019-pre-training-whole-word-masking/)
58. [**BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation**Junnan Li, Dongxu Li, Caiming Xiong, et al.*2022 · 1 463 citations*](https://md.research.elamur.ai/free/en/papers/li-2022-blip-bootstrapping-language-image-pre/)
59. [**GLM: General Language Model Pretraining with Autoregressive Blank Infilling**Zhengxiao Du, Yujie Qian, Xiao Liu, et al.*2022 · 1 449 citations*](https://md.research.elamur.ai/free/en/papers/du-2022-glm-general-language-model-pretraining/)
60. [**Sigmoid Loss for Language Image Pre-Training**Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, et al.*2023 · 1 427 citations*](https://md.research.elamur.ai/free/en/papers/zhai-2023-sigmoid-loss-language-image-pre/)
61. [**Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?**Sewon Min, Xinxi Lyu, Ari Holtzman, et al.*2022 · 1 423 citations*](https://md.research.elamur.ai/free/en/papers/min-2022-rethinking-role-demonstrations-what-makes/)
62. [**Code Llama: Open Foundation Models for Code**Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, et al.*2024 · 1 399 citations*](https://md.research.elamur.ai/free/en/papers/roziere-2024-code-llama-open-foundation-models/)
63. [**Unified Language Model Pre-training for Natural Language Understanding and Generation**Li Dong, Nan Yang, Wenhui Wang, et al.*2019 · 1 389 citations*](https://md.research.elamur.ai/free/en/papers/dong-2019-unified-language-model-pre-training/)
64. [**Parameter-efficient fine-tuning of large-scale pre-trained language models**Ning Ding, Yujia Qin, Guang You Yang, et al.*2023 · 1 324 citations*](https://md.research.elamur.ai/free/en/papers/ding-2023-parameter-efficient-fine-tuning-large/)
65. [**DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing**Pengcheng He, Jianfeng Gao, Weizhu Chen*2021 · 1 316 citations*](https://md.research.elamur.ai/free/en/papers/he-2021-debertav3-improving-deberta-using-electra/)
66. [**Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models**Ryan Kiros, Ruslan Salakhutdinov, Richard S. Zemel*2014 · 1 315 citations*](https://md.research.elamur.ai/free/en/papers/kiros-2014-unifying-visual-semantic-embeddings-multimodal/)
67. [**LaMDA: Language Models for Dialog Applications**Romal Thoppilan, Daniel De Freitas, Jamie Hall, et al.*2022 · 1 314 citations*](https://md.research.elamur.ai/free/en/papers/thoppilan-2022-lamda-language-models-dialog-applications/)
68. [**Language Modeling with Gated Convolutional Networks**Yann N. Dauphin, Angela Fan, Michael Auli, et al.*2016 · 1 305 citations*](https://md.research.elamur.ai/free/en/papers/dauphin-2016-language-modeling-gated-convolutional-networks/)
69. [**A Comprehensive Overview of Large Language Models**Humza Naveed, Asad Ullah Khan, Shi Qiu, et al.*2023 · 1 286 citations*](https://md.research.elamur.ai/free/en/papers/naveed-2023-comprehensive-overview-large-language-models/)
70. [**RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models**Samuel Gehman, Suchin Gururangan, Maarten Sap, et al.*2020 · 1 277 citations*](https://md.research.elamur.ai/free/en/papers/gehman-2020-realtoxicityprompts-evaluating-neural-toxic-degeneration/)
71. [**CTRL: A Conditional Transformer Language Model for Controllable Generation**Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, et al.*2019 · 1 273 citations*](https://md.research.elamur.ai/free/en/papers/keskar-2019-ctrl-conditional-transformer-language-model/)
72. [**CamemBERT: a Tasty French Language Model**Louis Martin, Benjamin Muller, Pedro Javier Ortiz Suárez, et al.*2019 · 1 262 citations*](https://md.research.elamur.ai/free/en/papers/martin-2019-camembert-tasty-french-language-model/)
73. [**What Makes Good In-Context Examples for GPT-3?**Jiachang Liu, Dinghan Shen, Yizhe Zhang, et al.*2022 · 1 235 citations*](https://md.research.elamur.ai/free/en/papers/liu-2022-what-makes-good-context-examples/)
74. [**An Empirical Study of Smoothing Techniques for Language Modeling**Stanley F. Chen, Joshua Goodman*1996 · 1 209 citations*](https://md.research.elamur.ai/free/en/papers/chen-1996-empirical-study-smoothing-techniques-language/)
75. [**A Survey on Large Language Model (LLM) Security and Privacy: The Good, the Bad, and the Ugly**Yifan Yao, Jinhao Duan, Kaidi Xu, et al.*2023 · 1 181 citations*](https://md.research.elamur.ai/free/en/papers/yao-2023-survey-large-language-model-llm/)
76. [**Least-to-most prompting enables complex reasoning in large language models***2022 · 1 143 citations*](https://md.research.elamur.ai/free/en/papers/al-2022-least-most-prompting-enables-complex/)
77. [**BERTweet: A pre-trained language model for English Tweets**Dat Quoc Nguyen, Thanh Vu, Anh Tuan Nguyen*2020 · 1 139 citations*](https://md.research.elamur.ai/free/en/papers/nguyen-2020-bertweet-pre-trained-language-model/)
78. [**StereoSet: Measuring stereotypical bias in pretrained language models**Moin Nadeem, Anna Bethke, Siva Reddy*2021 · 1 131 citations*](https://md.research.elamur.ai/free/en/papers/nadeem-2021-stereoset-measuring-stereotypical-bias-pretrained/)
79. [**It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners**Timo Schick, Hinrich Schütze*2021 · 1 118 citations*](https://md.research.elamur.ai/free/en/papers/schick-2021-it-s-not-just-size/)
80. [**Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone**Marah Abdin, Jyoti Aneja, Hany Awadalla, et al.*2024 · 1 113 citations*](https://md.research.elamur.ai/free/en/papers/abdin-2024-phi-3-technical-report-highly/)
81. [**Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm**Laria Reynolds, Kyle McDonell*2021 · 1 111 citations*](https://md.research.elamur.ai/free/en/papers/reynolds-2021-prompt-programming-large-language-models/)
82. [**Training Compute-Optimal Large Language Models**Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, et al.*2022 · 1 110 citations*](https://md.research.elamur.ai/free/en/papers/hoffmann-2022-training-compute-optimal-large-language/)
83. [**MASS: Masked Sequence to Sequence Pre-training for Language Generation**Kaitao Song, Xu Tan, Tao Qin, et al.*2019 · 1 109 citations*](https://md.research.elamur.ai/free/en/papers/song-2019-mass-masked-sequence-sequence-pre/)
84. [**MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models**Deyao Zhu, Jun Chen, Xiaoqian Shen, et al.*2023 · 1 105 citations*](https://md.research.elamur.ai/free/en/papers/zhu-2023-minigpt-4-enhancing-vision-language/)
85. [**Bias and Fairness in Large Language Models: A Survey**Isabel O. Gallegos, Ryan A. Rossi, Joe Barrow, et al.*2023 · 1 072 citations*](https://md.research.elamur.ai/free/en/papers/gallegos-2023-bias-fairness-large-language-models/)
86. [**A Survey on Multimodal Large Language Models**Shukang Yin, Chaoyou Fu, Sirui Zhao, et al.*2023 · 1 060 citations*](https://md.research.elamur.ai/free/en/papers/yin-2023-survey-multimodal-large-language-models/)
87. [**InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning**Wenliang Dai, Junnan Li, Dongxu Li, et al.*2023 · 1 051 citations*](https://md.research.elamur.ai/free/en/papers/dai-2023-instructblip-towards-general-purpose-vision/)
88. [**GPT Understands, Too**Xiao Liu, Yanan Zheng, Zhengxiao Du, et al.*2021 · 1 039 citations*](https://md.research.elamur.ai/free/en/papers/liu-2021-gpt-understands-too/)
89. [**LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models**Yaowei Zheng, Richong Zhang, Junhao Zhang, et al.*2024 · 1 035 citations*](https://md.research.elamur.ai/free/en/papers/zheng-2024-llamafactory-unified-efficient-fine-tuning/)
90. [**VL-BERT: Pre-training of Generic Visual-Linguistic Representations**Weijie Su, Xizhou Zhu, Yue Cao, et al.*2019 · 1 008 citations*](https://md.research.elamur.ai/free/en/papers/su-2019-vl-bert-pre-training-generic/)
91. [**A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models**Wenqi Fan; Yujuan Ding; Liangbo Ning; Shijie Wang; Hengyun Li; Dawei Yin; Tat-Seng Chua;, Qing Li*2024 · 1 006 citations*](https://md.research.elamur.ai/free/en/papers/fan-2024-survey-rag-meeting-llms-towards/)
92. [**Evaluating Object Hallucination in Large Vision-Language Models**Yifan Li, Yifan Du, Kun Zhou, et al.*2023 · 986 citations*](https://md.research.elamur.ai/free/en/papers/li-2023-evaluating-object-hallucination-large-vision/)
93. [**ERNIE 2.0: A Continual Pre-training Framework for Language Understanding**Yu Sun, Shuohuan Wang, Yukun Li, et al.*2019 · 976 citations*](https://md.research.elamur.ai/free/en/papers/sun-2019-ernie-2-0-continual/)
94. [**Solving Quantitative Reasoning Problems with Language Models**Aitor Lewkowycz, Anders Andreassen, David Dohan, et al.*2022 · 973 citations*](https://md.research.elamur.ai/free/en/papers/lewkowycz-2022-solving-quantitative-reasoning-problems-language/)
95. [**Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Models**Yue Zhang, Yafu Li, Leyang Cui, et al.*2023 · 962 citations*](https://md.research.elamur.ai/free/en/papers/zhang-2023-siren-s-song-ai-ocean/)
96. [**A Survey on In-context Learning**Qingxiu Dong, Lei Li, Damai Dai, et al.*2023 · 954 citations*](https://md.research.elamur.ai/free/en/papers/dong-2023-survey-context-learning/)
97. [**Universal and Transferable Adversarial Attacks on Aligned Language Models**Andy Zou, Zifan Wang, Nicholas Carlini, et al.*2023 · 951 citations*](https://md.research.elamur.ai/free/en/papers/zou-2023-universal-transferable-adversarial-attacks-aligned/)
98. [**ERNIE: Enhanced Representation through Knowledge Integration**Yu Sun, Shuohuan Wang, Yukun Li, et al.*2019 · 949 citations*](https://md.research.elamur.ai/free/en/papers/sun-2019-ernie-enhanced-representation-through-knowledge/)
99. [**MPNet: Masked and Permuted Pre-training for Language Understanding**Kaitao Song, Xu Tan, Tao Qin, et al.*2020 · 944 citations*](https://md.research.elamur.ai/free/en/papers/song-2020-mpnet-masked-permuted-pre-training/)
100. [**Out of One, Many: Using Language Models to Simulate Human Samples**Lisa P. Argyle, Ethan C. Busby, Nancy Fulda, et al.*2022 · 942 citations*](https://md.research.elamur.ai/free/en/papers/argyle-2022-out-one-many-using-language/)
101. [**One Billion Word Benchmark for Measuring Progress in Statistical Language Modeling**Ciprian Chelba, Tomas Mikolov, Mike Schuster, et al.*2013 · 940 citations*](https://md.research.elamur.ai/free/en/papers/chelba-2013-one-billion-word-benchmark-measuring/)
102. [**Summary of ChatGPT-Related Research and Perspective Towards the Future of Large Language Models**Yiheng Liu, Tianle Han, Siyuan Ma, et al.*2023 · 936 citations*](https://md.research.elamur.ai/free/en/papers/liu-2023-summary-chatgpt-related-research-perspective/)
103. [**Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond**Jinze Bai, Shuai Bai, Shusheng Yang, et al.*2023 · 913 citations*](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-vl-versatile-vision-language/)
104. [**Extracting Training Data from Large Language Models**Nicholas Carlini, Florian Tramer, Eric Wallace, et al.*2020 · 901 citations*](https://md.research.elamur.ai/free/en/papers/carlini-2020-extracting-training-data-large-language/)
105. [**A Neural Probabilistic Language Model**Yoshua Bengio, Réjean Ducharme, Pascal Vincent, et al.*2003 · 901 citations*](https://md.research.elamur.ai/free/en/papers/bengio-2003-neural-probabilistic-language-model/)
106. [**Using the Output Embedding to Improve Language Models**Ofir Press, Lior Wolf*2017 · 891 citations*](https://md.research.elamur.ai/free/en/papers/press-2017-output-embedding-improve-language-models/)
107. [**LayoutLM: Pre-training of Text and Layout for Document Image Understanding**Yiheng Xu, Minghao Li, Lei Cui, et al.*2019 · 889 citations*](https://md.research.elamur.ai/free/en/papers/xu-2019-layoutlm-pre-training-text-layout/)
108. [**CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models**Nikita Nangia, Clara Vania, Rasika Bhalerao, et al.*2020 · 886 citations*](https://md.research.elamur.ai/free/en/papers/nangia-2020-crows-pairs-challenge-dataset-measuring/)
109. [**Intelligent Selection of Language Model Training Data**Robert C. Moore, William Lewis*2010 · 848 citations*](https://md.research.elamur.ai/free/en/papers/moore-2010-intelligent-selection-language-model-training/)
110. [**TaPas: Weakly Supervised Table Parsing via Pre-training**Jonathan Herzig, Pawel Krzysztof Nowak, Thomas Müller, et al.*2020 · 844 citations*](https://md.research.elamur.ai/free/en/papers/herzig-2020-tapas-weakly-supervised-table-parsing/)
111. [**What BERT Is Not: Lessons from a New Suite of Psycholinguistic Diagnostics for Language Models**Allyson Ettinger*2020 · 843 citations*](https://md.research.elamur.ai/free/en/papers/ettinger-2020-what-bert-is-not-lessons/)
112. [**On the Sentence Embeddings from Pre-trained Language Models**Bohan Li, Hao Zhou, Junxian He, et al.*2020 · 830 citations*](https://md.research.elamur.ai/free/en/papers/li-2020-sentence-embeddings-pre-trained-language/)
113. [**Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution**Peng Wang, Shuai Bai, Sinan Tan, et al.*2024 · 819 citations*](https://md.research.elamur.ai/free/en/papers/wang-2024-qwen2-vl-enhancing-vision/)
114. [**GPT-NeoX-20B: An Open-Source Autoregressive Language Model**Sidney Black, Stella Biderman, Eric Hallahan, et al.*2022 · 818 citations*](https://md.research.elamur.ai/free/en/papers/black-2022-gpt-neox-20b-open-source/)
115. [**Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond**Jingfeng Yang, Hongye Jin, Ruixiang Tang, et al.*2023 · 806 citations*](https://md.research.elamur.ai/free/en/papers/yang-2023-harnessing-power-llms-practice-survey/)
116. [**Calibrate Before Use: Improving Few-Shot Performance of Language Models**Tony Z. Zhao, Eric Wallace, Shi Feng, et al.*2021 · 791 citations*](https://md.research.elamur.ai/free/en/papers/zhao-2021-calibrate-before-use-improving-few/)
117. [**Semi-supervised sequence tagging with bidirectional language models**Matthew E. Peters, Waleed Ammar, Chandra Bhagavatula, et al.*2017 · 786 citations*](https://md.research.elamur.ai/free/en/papers/peters-2017-semi-supervised-sequence-tagging-bidirectional/)
118. [**Explainability for Large Language Models: A Survey**Haiyan Zhao, Hanjie Chen, Fan Yang, et al.*2023 · 785 citations*](https://md.research.elamur.ai/free/en/papers/zhao-2023-explainability-large-language-models-survey/)
119. [**Can Large Language Models Transform Computational Social Science?**Caleb Ziems, William Held, Omar Shaikh, et al.*2023 · 776 citations*](https://md.research.elamur.ai/free/en/papers/ziems-2023-large-language-models-transform-computational/)
120. [**Towards Reasoning in Large Language Models: A Survey**Jie Huang, Kevin Chen-Chuan Chang*2023 · 775 citations*](https://md.research.elamur.ai/free/en/papers/huang-2023-reasoning-large-language-models-survey/)
121. [**LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking**Yupan Huang, Tengchao Lv, Lei Cui, et al.*2022 · 774 citations*](https://md.research.elamur.ai/free/en/papers/huang-2022-layoutlmv3-pre-training-document-ai/)
122. [**UniXcoder: Unified Cross-Modal Pre-training for Code Representation**Daya Guo, Shuai Lu, Nan Duan, et al.*2022 · 768 citations*](https://md.research.elamur.ai/free/en/papers/guo-2022-unixcoder-unified-cross-modal-pre/)
123. [**Unified Pre-training for Program Understanding and Generation**Wasi Ahmad, Saikat Chakraborty, Baishakhi Ray, et al.*2021 · 746 citations*](https://md.research.elamur.ai/free/en/papers/ahmad-2021-unified-pre-training-program-understanding/)
124. [**Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning**Haokun Liu, Derek Tam, Mohammed Muqeeth, et al.*2022 · 735 citations*](https://md.research.elamur.ai/free/en/papers/liu-2022-few-shot-parameter-efficient-fine/)
125. [**IndicNLPSuite: Monolingual Corpora, Evaluation Benchmarks and Pre-trained Multilingual Language Models for Indian Languages**Divyanshu Kakwani, Anoop Kunchukuttan, Satish Golla, et al.*2020 · 730 citations*](https://md.research.elamur.ai/free/en/papers/kakwani-2020-indicnlpsuite-monolingual-corpora-evaluation-benchmarks/)

[← prev](https://md.research.elamur.ai/free/en/topics/language-models/) [1](https://md.research.elamur.ai/free/en/topics/language-models/) **2** [3](https://md.research.elamur.ai/free/en/topics/language-models/page/3/) … [12](https://md.research.elamur.ai/free/en/topics/language-models/page/12/) [next →](https://md.research.elamur.ai/free/en/topics/language-models/page/3/)