---
title: "Language models & pretraining — papers, page 7"
description: "Language models & pretraining: 1,079 papers of the Elamur Research library · page 7 of 12 · 526–625, the most cited first."
canonical: "https://research.elamur.ai/free/en/topics/language-models/page/7/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/topics/language-models/page/7/)

[home](https://md.research.elamur.ai/free/en/) / [topics](https://md.research.elamur.ai/free/en/topics/) / [Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) / page 7

# 🧠 Language models & pretraining

1,079 papers · page 7 of 12 · 526–625 · the most cited first · [all topics](https://md.research.elamur.ai/free/en/topics/)

Models that learn a language from raw text, and how they are pretrained, tuned and prompted.

[← prev](https://md.research.elamur.ai/free/en/topics/language-models/page/6/) [1](https://md.research.elamur.ai/free/en/topics/language-models/) … [6](https://md.research.elamur.ai/free/en/topics/language-models/page/6/) **7** [8](https://md.research.elamur.ai/free/en/topics/language-models/page/8/) … [12](https://md.research.elamur.ai/free/en/topics/language-models/page/12/) [next →](https://md.research.elamur.ai/free/en/topics/language-models/page/8/)

526. [**LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Provers**Theo X. Olausson, Alex Gu, Benjamin Lipkin, et al.*2023 · 171 citations*](https://md.research.elamur.ai/free/en/papers/olausson-2023-linc-neurosymbolic-approach-logical-reasoning/)
527. [**OpenChat: Advancing Open-source Language Models with Mixed-Quality Data**Guan Wang, Sijie Cheng, Xianyuan Zhan, et al.*2023 · 171 citations*](https://md.research.elamur.ai/free/en/papers/wang-2023-openchat-advancing-open-source-language/)
528. [**KernelBench: Can LLMs Write Efficient GPU Kernels?**Anne Ouyang, Simon Guo, Simran Arora, et al.*2025 · 170 citations*](https://md.research.elamur.ai/free/en/papers/ouyang-2025-kernelbench-llms-write-efficient-gpu/)
529. [**Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs**Simone Balloccu, Patrícia Schmidtová, Mateusz Lango, et al.*2024 · 170 citations*](https://md.research.elamur.ai/free/en/papers/balloccu-2024-leak-cheat-repeat-data-contamination/)
530. [**LESS: Selecting Influential Data for Targeted Instruction Tuning**Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, et al.*2024 · 170 citations*](https://md.research.elamur.ai/free/en/papers/xia-2024-less-selecting-influential-data-targeted/)
531. [**AudioPaLM: A Large Language Model That Can Speak and Listen**Paul K. Rubenstein, Chulayuth Asawaroengchai, Duc Dung Nguyen, et al.*2023 · 169 citations*](https://md.research.elamur.ai/free/en/papers/rubenstein-2023-audiopalm-large-language-model-that/)
532. [**Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity**Cunxiang Wang, Xiaoze Liu, Yuanhao Yue, et al.*2023 · 169 citations*](https://md.research.elamur.ai/free/en/papers/wang-2023-survey-factuality-large-language-models/)
533. [**Low-Resource Languages Jailbreak GPT-4**Zheng-Xin Yong, Cristina Menghini, Stephen H. Bach*2023 · 168 citations*](https://md.research.elamur.ai/free/en/papers/yong-2023-low-resource-languages-jailbreak-gpt/)
534. [**ShortGPT: Layers in Large Language Models are More Redundant Than You Expect**Xin Men, Mingyu Xu, Qingyu Zhang, et al.*2024 · 167 citations*](https://md.research.elamur.ai/free/en/papers/men-2024-shortgpt-layers-large-language-models/)
535. [**The Capacity for Moral Self-Correction in Large Language Models**Deep Ganguli, Amanda Askell, Nicholas Schiefer, et al.*2023 · 167 citations*](https://md.research.elamur.ai/free/en/papers/ganguli-2023-capacity-moral-self-correction-large/)
536. [**Speak, Memory: An Archaeology of Books Known to ChatGPT/GPT-4**Kent K. Chang, Mackenzie Cramer, Sandeep Soni, et al.*2023 · 166 citations*](https://md.research.elamur.ai/free/en/papers/chang-2023-speak-memory-archaeology-books-known/)
537. [**Deduplicating Training Data Mitigates Privacy Risks in Language Models**Nikhil Kandpal, Eric Wallace, Colin Raffel*2022 · 166 citations*](https://md.research.elamur.ai/free/en/papers/kandpal-2022-deduplicating-training-data-mitigates-privacy/)
538. [**Structure-Grounded Pretraining for Text-to-SQL**Xiang Deng, Ahmed Hassan Awadallah, Christopher Meek, et al.*2021 · 166 citations*](https://md.research.elamur.ai/free/en/papers/deng-2021-structure-grounded-pretraining-text-sql/)
539. [**Generative Representational Instruction Tuning**Niklas Muennighoff, Hongjin Su, Liang Wang, et al.*2024 · 165 citations*](https://md.research.elamur.ai/free/en/papers/muennighoff-2024-generative-representational-instruction-tuning/)
540. [**Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts**Jian Xie, Kai Zhang, Jiangjie Chen, et al.*2023 · 165 citations*](https://md.research.elamur.ai/free/en/papers/xie-2023-adaptive-chameleon-stubborn-sloth-revealing/)
541. [**Large Language Model Unlearning**Yuanshun Yao, Xiaojun Xu, Yang Liu*2023 · 165 citations*](https://md.research.elamur.ai/free/en/papers/yao-2023-large-language-model-unlearning/)
542. [**Binding Language Models in Symbolic Languages**Zhoujun Cheng, Tianbao Xie, Peng Shi, et al.*2022 · 165 citations*](https://md.research.elamur.ai/free/en/papers/cheng-2022-binding-language-models-symbolic-languages/)
543. [**“I'm sorry to hear that”: Finding New Biases in Language Models with a Holistic Descriptor Dataset**Eric Michael Smith, Melissa Hall, Melanie Kambadur, et al.*2022 · 165 citations*](https://md.research.elamur.ai/free/en/papers/smith-2022-i-m-sorry-hear-that/)
544. [**On Generative Spoken Language Modeling from Raw Audio**Kushal Lakhotia, Eugene Kharitonov, Wei-Ning Hsu, et al.*2021 · 165 citations*](https://md.research.elamur.ai/free/en/papers/lakhotia-2021-generative-spoken-language-modeling-raw/)
545. [**Sustainable Modular Debiasing of Language Models**Anne Lauscher, Tobias Lueken, Goran Glavaš*2021 · 165 citations*](https://md.research.elamur.ai/free/en/papers/lauscher-2021-sustainable-modular-debiasing-language-models/)
546. [**Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models**Orevaoghene Ahia, Sachin Kumar, Hila Gonen, et al.*2023 · 164 citations*](https://md.research.elamur.ai/free/en/papers/ahia-2023-do-all-languages-cost-same/)
547. [**From System 1 to System 2: A Survey of Reasoning Large Language Models**Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, et al.*2025 · 163 citations*](https://md.research.elamur.ai/free/en/papers/li-2025-system-1-system-2-survey/)
548. [**Improve Mathematical Reasoning in Language Models by Automated Process Supervision**Liangchen Luo, Yinxiao Liu, Rosanne Liu, et al.*2024 · 163 citations*](https://md.research.elamur.ai/free/en/papers/luo-2024-improve-mathematical-reasoning-language-models/)
549. [**AmericasNLI: Evaluating Zero-shot Natural Language Understanding of Pretrained Multilingual Models in Truly Low-resource Languages**Abteen Ebrahimi, Manuel Mager, Arturo Oncevay, et al.*2022 · 163 citations*](https://md.research.elamur.ai/free/en/papers/ebrahimi-2022-americasnli-evaluating-zero-shot-natural/)
550. [**Long-form factuality in large language models**Jerry Wei, Chengrun Yang, Xinying Song, et al.*2024 · 162 citations*](https://md.research.elamur.ai/free/en/papers/wei-2024-long-form-factuality-large-language/)
551. [**Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation**Yangsibo Huang, Samyak Gupta, Mengzhou Xia, et al.*2023 · 162 citations*](https://md.research.elamur.ai/free/en/papers/huang-2023-catastrophic-jailbreak-open-source-llms/)
552. [**GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Prompts**Jiahao Yu, Xingwei Lin, Zheng Yu, et al.*2023 · 162 citations*](https://md.research.elamur.ai/free/en/papers/yu-2023-gptfuzzer-red-teaming-large-language/)
553. [**LooGLE: Can Long-Context Language Models Understand Long Contexts?**Jiaqi Li, Mengmeng Wang, Zilong Zheng, et al.*2023 · 162 citations*](https://md.research.elamur.ai/free/en/papers/li-2023-loogle-long-context-language-models/)
554. [**Impact of Pretraining Term Frequencies on Few-Shot Numerical Reasoning**Yasaman Razeghi, Robert L Logan IV, Matt Gardner, et al.*2022 · 162 citations*](https://md.research.elamur.ai/free/en/papers/razeghi-2022-impact-pretraining-term-frequencies-few/)
555. [**Probing What Different NLP Tasks Teach Machines about Function Word Comprehension**Najoung Kim, Roma Patel, Adam Poliak, et al.*2019 · 161 citations*](https://md.research.elamur.ai/free/en/papers/kim-2019-probing-what-different-nlp-tasks/)
556. [**CultureLLM: Incorporating Cultural Differences into Large Language Models**Cheng Li, Mengzhou Chen, Jindong Wang, et al.*2024 · 160 citations*](https://md.research.elamur.ai/free/en/papers/li-2024-culturellm-incorporating-cultural-differences-large/)
557. [**Orca 2: Teaching Small Language Models How to Reason**Arindam Mitra, Luciano Del Corro, Shweti Mahajan, et al.*2023 · 160 citations*](https://md.research.elamur.ai/free/en/papers/mitra-2023-orca-2-teaching-small-language/)
558. [**The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets**Samuel Marks, Max Tegmark*2023 · 160 citations*](https://md.research.elamur.ai/free/en/papers/marks-2023-geometry-truth-emergent-linear-structure/)
559. [**Reasoning with Language Model Prompting: A Survey**Shuofei Qiao, Yixin Ou, Ningyu Zhang, et al.*2022 · 160 citations*](https://md.research.elamur.ai/free/en/papers/qiao-2022-reasoning-language-model-prompting-survey/)
560. [**Larger-Scale Transformers for Multilingual Masked Language Modeling**Naman Goyal, Jingfei Du, Myle Ott, et al.*2021 · 160 citations*](https://md.research.elamur.ai/free/en/papers/goyal-2021-larger-scale-transformers-multilingual-masked/)
561. [**Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Models**Bofei Gao, Feifan Song, Zhe Yang, et al.*2024 · 159 citations*](https://md.research.elamur.ai/free/en/papers/gao-2024-omni-math-universal-olympiad-level/)
562. [**When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMs**Ryo Kamoi, Yusen Zhang, Nan Zhang, et al.*2024 · 159 citations*](https://md.research.elamur.ai/free/en/papers/kamoi-2024-when-can-llms-actually-correct/)
563. [**L-Eval: Instituting Standardized Evaluation for Long Context Language Models**Chenxin An, Shansan Gong, Ming Zhong, et al.*2023 · 159 citations*](https://md.research.elamur.ai/free/en/papers/an-2023-l-eval-instituting-standardized-evaluation/)
564. [**PanGu-α: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation**Wei Zeng, Xiaozhe Ren, Teng Su, et al.*2021 · 159 citations*](https://md.research.elamur.ai/free/en/papers/zeng-2021-pangu-alpha-large-scale/)
565. [**GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher**Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, et al.*2023 · 158 citations*](https://md.research.elamur.ai/free/en/papers/yuan-2023-gpt-4-too-smart-safe/)
566. [**LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression**Huiqiang Jiang, Qianhui Wu, Xufang Luo, et al.*2023 · 158 citations*](https://md.research.elamur.ai/free/en/papers/jiang-2023-longllmlingua-accelerating-enhancing-llms-long/)
567. [**GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models**Aohan Zeng, Xin Lv, Qinkai Zheng, et al.*2025 · 157 citations*](https://md.research.elamur.ai/free/en/papers/team-2025-4-glm-4-5-agentic/)
568. [**From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuning**Ming Li, Yong Zhang, Zhitao Li, et al.*2023 · 157 citations*](https://md.research.elamur.ai/free/en/papers/li-2023-quantity-quality-boosting-llm-performance/)
569. [**SmoothLLM: Defending Large Language Models Against Jailbreaking Attacks**Alexander Robey, Eric Wong, Hamed Hassani, et al.*2023 · 157 citations*](https://md.research.elamur.ai/free/en/papers/robey-2023-smoothllm-defending-large-language-models/)
570. [**The Linear Representation Hypothesis and the Geometry of Large Language Models**Kiho Park, Yo Joong Choe, Victor Veitch*2023 · 157 citations*](https://md.research.elamur.ai/free/en/papers/park-2023-linear-representation-hypothesis-geometry-large/)
571. [**Unsupervised Text Style Transfer using Language Models as Discriminators**Zichao Yang, Zhiting Hu, Chris Dyer, et al.*2018 · 157 citations*](https://md.research.elamur.ai/free/en/papers/yang-2018-unsupervised-text-style-transfer-using/)
572. [**DataComp-LM: In search of the next generation of training sets for language models**Jeffrey Li, Alex Fang, Georgios Smyrnis, et al.*2024 · 156 citations*](https://md.research.elamur.ai/free/en/papers/li-2024-datacomp-lm-search-next-generation/)
573. [**Canine: Pre-training an Efficient Tokenization-Free Encoder for Language Representation**Jonathan H. Clark, Dan Garrette, Iulia Turc, et al.*2022 · 156 citations*](https://md.research.elamur.ai/free/en/papers/clark-2022-canine-pre-training-efficient-tokenization/)
574. [**LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding**Mostafa Elhoushi, Akshat Shrivastava, Diana Liskovich, et al.*2024 · 155 citations*](https://md.research.elamur.ai/free/en/papers/elhoushi-2024-layerskip-enabling-early-exit-inference/)
575. [**WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs**Seungju Han, Kavel Rao, Allyson Ettinger, et al.*2024 · 155 citations*](https://md.research.elamur.ai/free/en/papers/han-2024-wildguard-open-one-stop-moderation/)
576. [**Probing Pre-Trained Language Models for Cross-Cultural Differences in Values**Arnav Arora, Lucie-aimée Kaffee, Isabelle Augenstein*2023 · 155 citations*](https://md.research.elamur.ai/free/en/papers/arora-2023-probing-pre-trained-language-models/)
577. [**Moshi: a speech-text foundation model for real-time dialogue**Alexandre Defossez, Laurent Mazare, Manu Orsini, et al.*2024 · 154 citations*](https://md.research.elamur.ai/free/en/papers/defossez-2024-moshi-speech-text-foundation-model/)
578. [**ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs**Justin Chen, Swarnadeep Saha, Mohit Bansal*2024 · 154 citations*](https://md.research.elamur.ai/free/en/papers/chen-2024-reconcile-round-table-conference-improves/)
579. [**Large Language Models with Controllable Working Memory**Daliang Li, Ankit Singh Rawat, Manzil Zaheer, et al.*2023 · 154 citations*](https://md.research.elamur.ai/free/en/papers/li-2023-large-language-models-controllable-working/)
580. [**Leveraging Large Language Models for Multiple Choice Question Answering**Joshua Robinson, Christopher Michael Rytting, David Wingate*2022 · 154 citations*](https://md.research.elamur.ai/free/en/papers/robinson-2022-leveraging-large-language-models-multiple/)
581. [**Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models**Yang Sui, Yu-Neng Chuang, Guanchu Wang, et al.*2025 · 153 citations*](https://md.research.elamur.ai/free/en/papers/sui-2025-stop-overthinking-survey-efficient-reasoning/)
582. [**Can GPT-3 Perform Statutory Reasoning?**Andrew Blair-Stanek, Nils Holzenberger, Benjamin Van Durme*2023 · 153 citations*](https://md.research.elamur.ai/free/en/papers/blair-stanek-2023-gpt-3-perform-statutory-reasoning/)
583. [**Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning**Lean Wang, Lei Li, Damai Dai, et al.*2023 · 153 citations*](https://md.research.elamur.ai/free/en/papers/wang-2023-label-words-anchors-information-flow/)
584. [**Graph Pre-training for AMR Parsing and Generation**Xuefeng Bai, Yulong Chen, Yue Zhang*2022 · 153 citations*](https://md.research.elamur.ai/free/en/papers/bai-2022-graph-pre-training-amr-parsing/)
585. [**SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training**Tianzhe Chu, Yuexiang Zhai, Jihan Yang, et al.*2025 · 151 citations*](https://md.research.elamur.ai/free/en/papers/chu-2025-sft-memorizes-rl-generalizes-comparative/)
586. [**Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Models**Tianyi Tang, Wenyang Luo, Haoyang Huang, et al.*2024 · 151 citations*](https://md.research.elamur.ai/free/en/papers/tang-2024-language-specific-neurons-key-multilingual/)
587. [**Training Language Models to Self-Correct via Reinforcement Learning**Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, et al.*2024 · 151 citations*](https://md.research.elamur.ai/free/en/papers/kumar-2024-training-language-models-self-correct/)
588. [**The Dawn After the Dark: An Empirical Study on Factuality Hallucination in Large Language Models**Junyi Li, Jie Chen, Ruiyang Ren, et al.*2024 · 150 citations*](https://md.research.elamur.ai/free/en/papers/li-2024-dawn-after-dark-empirical-study/)
589. [**AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators**Xingwei He, Zhenghao Lin, Yeyun Gong, et al.*2023 · 150 citations*](https://md.research.elamur.ai/free/en/papers/he-2023-annollm-making-large-language-models/)
590. [**Pre-Training BERT on Arabic Tweets: Practical Considerations**Ahmed Abdelali, Sabit Hassan, Hamdy Mubarak, et al.*2021 · 150 citations*](https://md.research.elamur.ai/free/en/papers/abdelali-2021-pre-training-bert-arabic-tweets/)
591. [**Chain-of-Note: Enhancing Robustness in Retrieval-Augmented Language Models**Wenhao Yu, Hongming Zhang, Xiaoman Pan, et al.*2024 · 149 citations*](https://md.research.elamur.ai/free/en/papers/yu-2024-chain-note-enhancing-robustness-retrieval/)
592. [**Language Model Tokenizers Introduce Unfairness Between Languages**Aleksandar Petrov, Emanuele La Malfa, Philip H. S. Torr, et al.*2023 · 149 citations*](https://md.research.elamur.ai/free/en/papers/petrov-2023-language-model-tokenizers-introduce-unfairness/)
593. [**Investigating Pretrained Language Models for Graph-to-Text Generation**Leonardo F. R. Ribeiro, Martin Schmitt, Hinrich Schütze, et al.*2021 · 149 citations*](https://md.research.elamur.ai/free/en/papers/ribeiro-2021-investigating-pretrained-language-models-graph/)
594. [**SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models**Xiaoxuan Wang, Ziniu Hu, Pan Lu, et al.*2023 · 148 citations*](https://md.research.elamur.ai/free/en/papers/wang-2023-scibench-evaluating-college-level-scientific/)
595. [**BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languages**Junho Myung, Nayeon Lee, Yi Zhou, et al.*2024 · 147 citations*](https://md.research.elamur.ai/free/en/papers/myung-2024-blend-benchmark-llms-everyday-knowledge/)
596. [**Measuring short-form factuality in large language models**Jason Wei, Nguyen Karina, Hyung Won Chung, et al.*2024 · 147 citations*](https://md.research.elamur.ai/free/en/papers/wei-2024-measuring-short-form-factuality-large/)
597. [**rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking**Xinyu Guan, Li Lyna Zhang, Yifei Liu, et al.*2025 · 146 citations*](https://md.research.elamur.ai/free/en/papers/guan-2025-rstar-math-small-llms-can/)
598. [**Draft & Verify: Lossless Large Language Model Acceleration via Self-Speculative Decoding**Jun Zhang, Jue Wang, Huan Li, et al.*2024 · 146 citations*](https://md.research.elamur.ai/free/en/papers/zhang-2024-draft-verify-lossless-large-language/)
599. [**Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAs**Microsoft, Abdelrahman Abouelenin, Atabak Ashfaq, et al.*2025 · 145 citations*](https://md.research.elamur.ai/free/en/papers/microsoft-2025-phi-4-mini-technical-report/)
600. [**How Abilities in Large Language Models are Affected by Supervised Fine-tuning Data Composition**Guanting Dong, Hongyi Yuan, Keming Lu, et al.*2024 · 145 citations*](https://md.research.elamur.ai/free/en/papers/dong-2024-how-abilities-large-language-models/)
601. [**Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding**Heming Xia, Zhe Yang, Qingxiu Dong, et al.*2024 · 145 citations*](https://md.research.elamur.ai/free/en/papers/xia-2024-unlocking-efficiency-large-language-model/)
602. [**Context-faithful Prompting for Large Language Models**Wenxuan Zhou, Sheng Zhang, Hoifung Poon, et al.*2023 · 145 citations*](https://md.research.elamur.ai/free/en/papers/zhou-2023-context-faithful-prompting-large-language/)
603. [**RA-DIT: Retrieval-Augmented Dual Instruction Tuning**Xi Victoria Lin, Xilun Chen, Mingda Chen, et al.*2023 · 145 citations*](https://md.research.elamur.ai/free/en/papers/lin-2023-ra-dit-retrieval-augmented-dual/)
604. [**SyntaxGym: An Online Platform for Targeted Evaluation of Language Models**Jon Gauthier, Jennifer Hu, Ethan Wilcox, et al.*2020 · 145 citations*](https://md.research.elamur.ai/free/en/papers/gauthier-2020-syntaxgym-online-platform-targeted-evaluation/)
605. [**Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing**Zhangchen Xu, Fengqing Jiang, Luyao Niu, et al.*2024 · 144 citations*](https://md.research.elamur.ai/free/en/papers/xu-2024-magpie-alignment-data-synthesis-scratch/)
606. [**Unified Demonstration Retriever for In-Context Learning**Xiaonan Li, Kai Lv, Hang Yan, et al.*2023 · 144 citations*](https://md.research.elamur.ai/free/en/papers/li-2023-unified-demonstration-retriever-context-learning/)
607. [**In-Context Learning for Few-Shot Dialogue State Tracking**Yushi Hu, Chia-Hsuan Lee, Tianbao Xie, et al.*2022 · 144 citations*](https://md.research.elamur.ai/free/en/papers/hu-2022-context-learning-few-shot-dialogue/)
608. [**Is GPT-3 a Good Data Annotator?**Bosheng Ding, Chengwei Qin, Linlin Liu, et al.*2022 · 144 citations*](https://md.research.elamur.ai/free/en/papers/ding-2022-gpt-3-good-data-annotator/)
609. [**MiniLLM: On-Policy Distillation of Large Language Models**Yuxian Gu, Li Dong, Furu Wei, et al.*2023 · 143 citations*](https://md.research.elamur.ai/free/en/papers/gu-2023-minillm-policy-distillation-large-language/)
610. [**Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning**Shenzhi Wang, Le Yu, Chang Gao, et al.*2025 · 142 citations*](https://md.research.elamur.ai/free/en/papers/wang-2025-80-20-rule-high-entropy/)
611. [**Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge**Jiayi Ye, Yanbo Wang, Yue Huang, et al.*2024 · 142 citations*](https://md.research.elamur.ai/free/en/papers/ye-2024-justice-prejudice-quantifying-biases-llm/)
612. [**In-Context Learning Creates Task Vectors**Roee Hendel, Mor Geva, Amir Globerson*2023 · 142 citations*](https://md.research.elamur.ai/free/en/papers/hendel-2023-context-learning-creates-task-vectors/)
613. [**PMC-LLaMA: Towards Building Open-source Language Models for Medicine**Chaoyi Wu, Weixiong Lin, Xiaoman Zhang, et al.*2023 · 142 citations*](https://md.research.elamur.ai/free/en/papers/wu-2023-pmc-llama-towards-building-open/)
614. [**ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embeddings**Shibo Hao, Tianyang Liu, Zhen Wang, et al.*2023 · 142 citations*](https://md.research.elamur.ai/free/en/papers/hao-2023-toolkengpt-augmenting-frozen-language-models/)
615. [**TALM: Tool Augmented Language Models**Aaron Parisi, Yao Zhao, Noah Fiedel*2022 · 142 citations*](https://md.research.elamur.ai/free/en/papers/parisi-2022-talm-tool-augmented-language-models/)
616. [**Rethinking embedding coupling in pre-trained language models**Hyung Won Chung, Thibault Févry, Henry Tsai, et al.*2020 · 142 citations*](https://md.research.elamur.ai/free/en/papers/chung-2020-rethinking-embedding-coupling-pre-trained/)
617. [**JudgeLM: Fine-tuned Large Language Models are Scalable Judges**Lianghui Zhu, Xinggang Wang, Xinlong Wang*2023 · 141 citations*](https://md.research.elamur.ai/free/en/papers/zhu-2023-judgelm-fine-tuned-large-language/)
618. [**How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model**Michael Hanna, Ollie Liu, Alexandre Variengien*2023 · 140 citations*](https://md.research.elamur.ai/free/en/papers/hanna-2023-gpt-2-compute-greater-than/)
619. [**A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generation**Neeraj Varshney; Wenlin Yao; Hongming Zhang; Jianshu Chen;, Dong Yu*2023 · 139 citations*](https://md.research.elamur.ai/free/en/papers/varshney-2023-stitch-time-saves-nine-detecting/)
620. [**The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning**Bill Yuchen Lin, Abhilasha Ravichander, Ximing Lu, et al.*2023 · 139 citations*](https://md.research.elamur.ai/free/en/papers/lin-2023-unlocking-spell-base-llms-rethinking/)
621. [**LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens**Yiran Ding, Li Lyna Zhang, Chengruidong Zhang, et al.*2024 · 138 citations*](https://md.research.elamur.ai/free/en/papers/ding-2024-longrope-extending-llm-context/)
622. [**DeepInception: Hypnotize Large Language Model to Be Jailbreaker**Xuan Li, Zhanke Zhou, Jianing Zhu, et al.*2023 · 138 citations*](https://md.research.elamur.ai/free/en/papers/li-2023-deepinception-hypnotize-large-language-model/)
623. [**Exploring Length Generalization in Large Language Models**Cem Anil, Yuhuai Wu, Anders Andreassen, et al.*2022 · 138 citations*](https://md.research.elamur.ai/free/en/papers/anil-2022-exploring-length-generalization-large-language/)
624. [**R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning**Huatong Song, Jinhao Jiang, Yingqian Min, et al.*2025 · 137 citations*](https://md.research.elamur.ai/free/en/papers/song-2025-r1-searcher-incentivizing-search/)
625. [**Investigating Data Contamination in Modern Benchmarks for Large Language Models**Chunyuan Deng, Yilun Zhao, Xiangru Tang, et al.*2024 · 137 citations*](https://md.research.elamur.ai/free/en/papers/deng-2024-investigating-data-contamination-modern-benchmarks/)

[← prev](https://md.research.elamur.ai/free/en/topics/language-models/page/6/) [1](https://md.research.elamur.ai/free/en/topics/language-models/) … [6](https://md.research.elamur.ai/free/en/topics/language-models/page/6/) **7** [8](https://md.research.elamur.ai/free/en/topics/language-models/page/8/) … [12](https://md.research.elamur.ai/free/en/topics/language-models/page/12/) [next →](https://md.research.elamur.ai/free/en/topics/language-models/page/8/)