---
title: "Efficient models & compression — papers, page 2"
description: "Efficient models & compression: 60 papers of the Elamur Research library · page 2 of 2 · 26–60, the most cited first."
canonical: "https://research.elamur.ai/free/en/topics/efficient-models/page/2/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/topics/efficient-models/page/2/)

[home](https://md.research.elamur.ai/free/en/) / [topics](https://md.research.elamur.ai/free/en/topics/) / [Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/) / page 2

# 🪶 Efficient models & compression

60 papers · page 2 of 2 · 26–60 · the most cited first · [all topics](https://md.research.elamur.ai/free/en/topics/)

Smaller and faster models: quantization, pruning, distillation and sparse computation.

[← prev](https://md.research.elamur.ai/free/en/topics/efficient-models/) [1](https://md.research.elamur.ai/free/en/topics/efficient-models/) **2**

26. [**Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs**Suyu Ge, Yunan Zhang, Liyuan Liu, et al.*2023 · 198 citations*](https://md.research.elamur.ai/free/en/papers/ge-2023-model-tells-discard-adaptive-kv/)
27. [**Structured Pruning of Large Language Models**Ziheng Wang, Jeremy Wohlwend, Tao Lei*2020 · 196 citations*](https://md.research.elamur.ai/free/en/papers/wang-2020-structured-pruning-large-language-models/)
28. [**OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models**Wenqi Shao, Mengzhao Chen, Zhaoyang Zhang, et al.*2023 · 193 citations*](https://md.research.elamur.ai/free/en/papers/shao-2023-omniquant-omnidirectionally-calibrated-quantization-large/)
29. [**SqueezeLLM: Dense-and-Sparse Quantization**Sehoon Kim, Coleman Hooper, Amir Gholami, et al.*2023 · 168 citations*](https://md.research.elamur.ai/free/en/papers/kim-2023-squeezellm-dense-sparse-quantization/)
30. [**LLMLingua-2: Data Distillation for Efficient and Faithful Task-Agnostic Prompt Compression**Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, et al.*2024 · 159 citations*](https://md.research.elamur.ai/free/en/papers/pan-2024-llmlingua-2-data-distillation-efficient/)
31. [**LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding**Mostafa Elhoushi, Akshat Shrivastava, Diana Liskovich, et al.*2024 · 155 citations*](https://md.research.elamur.ai/free/en/papers/elhoushi-2024-layerskip-enabling-early-exit-inference/)
32. [**PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling**Zefan Cai, Yichi Zhang, Bofei Gao, et al.*2024 · 151 citations*](https://md.research.elamur.ai/free/en/papers/cai-2024-pyramidkv-dynamic-kv-cache-compression/)
33. [**Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding**Heming Xia, Zhe Yang, Qingxiu Dong, et al.*2024 · 145 citations*](https://md.research.elamur.ai/free/en/papers/xia-2024-unlocking-efficiency-large-language-model/)
34. [**SCOTT: Self-Consistent Chain-of-Thought Distillation**Peifeng Wang, Zhengyang Wang, Zheng Li, et al.*2023 · 145 citations*](https://md.research.elamur.ai/free/en/papers/wang-2023-scott-self-consistent-chain-thought/)
35. [**MiniLLM: On-Policy Distillation of Large Language Models**Yuxian Gu, Li Dong, Furu Wei, et al.*2023 · 143 citations*](https://md.research.elamur.ai/free/en/papers/gu-2023-minillm-policy-distillation-large-language/)
36. [**Symbolic Chain-of-Thought Distillation: Small Models Can Also “Think” Step-by-Step**Liunian Harold Li, Jack Hessel, Youngjae Yu, et al.*2023 · 139 citations*](https://md.research.elamur.ai/free/en/papers/li-2023-symbolic-chain-thought-distillation-small/)
37. [**Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention**Jingyang Yuan, Huazuo Gao, Damai Dai, et al.*2025 · 135 citations*](https://md.research.elamur.ai/free/en/papers/yuan-2025-native-sparse-attention-hardware/)
38. [**The case for 4-bit precision: k-bit Inference Scaling Laws**Tim Dettmers, Luke Zettlemoyer*2022 · 119 citations*](https://md.research.elamur.ai/free/en/papers/dettmers-2022-case-4-bit-precision-k/)
39. [**I-BERT: Integer-only BERT Quantization**Sehoon Kim, Amir Gholami, Zhewei Yao, et al.*2021 · 118 citations*](https://md.research.elamur.ai/free/en/papers/kim-2021-i-bert-integer-only-bert/)
40. [**DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence**DeepSeek-AI, Anyi Xu, Bangcai Lin, et al.*2026 · 117 citations*](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2026-deepseek-v4-highly-efficient/)
41. [**QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks**Albert Tseng, Jerry Chee, Qingyao Sun, et al.*2024 · 116 citations*](https://md.research.elamur.ai/free/en/papers/tseng-2024-quip-even-better-llm-quantization/)
42. [**DistiLLM: Towards Streamlined Distillation for Large Language Models**Jongwoo Ko, Sungnyun Kim, Tianyi Chen, et al.*2024 · 111 citations*](https://md.research.elamur.ai/free/en/papers/ko-2024-distillm-towards-streamlined-distillation-large/)
43. [**DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads**Guangxuan Xiao, Jiaming Tang, Jingwei Zuo, et al.*2024 · 109 citations*](https://md.research.elamur.ai/free/en/papers/xiao-2024-duoattention-efficient-long-context-llm/)
44. [**TokenSkip: Controllable Chain-of-Thought Compression in LLMs**Heming Xia, Chak Tou Leong, Wenjie Wang, et al.*2025 · 105 citations*](https://md.research.elamur.ai/free/en/papers/xia-2025-tokenskip-controllable-chain-thought-compression/)
45. [**Language Modeling Is Compression**Gregoire Deletang, Anian Ruoss, Paul-Ambroise Duquenne, et al.*2023 · 89 citations*](https://md.research.elamur.ai/free/en/papers/deletang-2023-language-modeling-compression/)
46. [**Tokenization Is More Than Compression**Craig W Schmidt, Varshini Reddy, Haoran Zhang, et al.*2024 · 66 citations*](https://md.research.elamur.ai/free/en/papers/schmidt-2024-tokenization-is-more-than-compression/)
47. [**Improving Stance Detection with Multi-Dataset Learning and Knowledge Distillation**Yingjie Li, Chenye Zhao, Cornelia Caragea*2021 · 39 citations*](https://md.research.elamur.ai/free/en/papers/li-2021-improving-stance-detection-multi-dataset/)
48. [**Towards Cross-Tokenizer Distillation: the Universal Logit Distillation Loss for LLMs**Nicolas Boizard, Kevin El Haddad, Celine Hudelot, et al.*2024 · 34 citations*](https://md.research.elamur.ai/free/en/papers/boizard-2024-towards-cross-tokenizer-distillation-universal/)
49. [**MBR and QE Finetuning: Training-time Distillation of the Best and Most Expensive Decoding Methods**Mara Finkelstein; Subhajit Naskar; Mehdi Mirzazadeh; Apurva Shah;, Markus Freitag*2023 · 30 citations*](https://md.research.elamur.ai/free/en/papers/finkelstein-2023-mbr-qe-finetuning-training-time/)
50. [**The Quantization Model of Neural Scaling**Eric J. Michaud, Ziming Liu, Uzay Girit, et al.*2023 · 27 citations*](https://md.research.elamur.ai/free/en/papers/michaud-2023-quantization-model-neural-scaling/)
51. [**Distillation Scaling Laws**Dan Busbridge, Amitis Shidani, Floris Weers, et al.*2025 · 23 citations*](https://md.research.elamur.ai/free/en/papers/busbridge-2025-distillation-scaling-laws/)
52. [**Okay, Let's Do This! Modeling Event Coreference with Generated Rationales and Knowledge Distillation**Abhijnan Nath, Shadi Manafi Avari, Avyakta Chelle, et al.*2024 · 16 citations*](https://md.research.elamur.ai/free/en/papers/nath-2024-okay-let-s-do-this/)
53. [**Selected-Layer Codec Compression for Compact Speech Translation Models: An IWSLT 2026 English-to-Chinese Submission**Alonso Palomino*2026 · 1 citation*](https://md.research.elamur.ai/free/en/papers/palomino-2026-selected-layer-codec-compression-compact/)
54. [**Rationale-Guided Distillation for E-Commerce Relevance Classification: Bridging Large Language Models and Lightweight Cross-Encoders**Sanjay Agrawal, Faizan Ahemad, Vivek Varadarajan Sembium*2025 · 1 citation*](https://md.research.elamur.ai/free/en/papers/agrawal-2025-rationale-guided-distillation-e-commerce/)
55. [**Evaluating Small Language Models for English-to-Portuguese Translation: Impact of Model Scale and Quantization**Gustavo Lopes Tamiosso, Rafael Oleques Nunes, Dennis Giovani Balreira*2026*](https://md.research.elamur.ai/free/en/papers/tamiosso-2026-evaluating-small-language-models-english/)
56. [**Reason-to-Learn (R2L): Multi-Agent Knowledge Distillation for Lightweight LLMs in Sentiment Analysis**Le-Huy Tu, Quan Nguyen, Vincent NGUYEN, et al.*2026*](https://md.research.elamur.ai/free/en/papers/tu-2026-reason-learn-r2l-multi-agent/)
57. [**SPQ: An Ensemble Technique for Large Language Model Compression**Jiamin Yao, Eren Gultepe*2026*](https://md.research.elamur.ai/free/en/papers/yao-2026-spq-ensemble-technique-large-language/)
58. [**Survey on KV Cache Compression Methods for Large Language Model Inference (大语言模型推理中的键值缓存压缩方法综述)**Shipeng Hu (胡世鹏), Guangyan Zhang (张广艳), Weimin Zheng (郑纬民)*2026*](https://md.research.elamur.ai/free/en/papers/hu-2026-kv-cache-compression/)
59. [**Terminology-Aware Retrieval-Augmented Knowledge Distillation for Biomedical Neural Machine Translation**Maria Zafar, Souhail Bakkali, Rejwanul Haque*2026*](https://md.research.elamur.ai/free/en/papers/zafar-2026-terminology-aware-retrieval-augmented-knowledge/)
60. [**Survey of Storage Optimization Techniques in Large Language Model Inference (大语言模型推理中的存储优化技术综述)**Xuran Ge (葛旭冉), Yang Ou (欧洋), Bo Wang (王博), et al.*2025*](https://md.research.elamur.ai/free/en/papers/ge-2025-storage-optimization-techniques/)

[← prev](https://md.research.elamur.ai/free/en/topics/efficient-models/) [1](https://md.research.elamur.ai/free/en/topics/efficient-models/) **2**