--- title: "大语言模型推理中的键值缓存压缩方法综述 — 胡世鹏 2026" description: "This survey from Tsinghua University examines KV cache compression for large language model inference." authors: ["胡世鹏", "张广艳", "郑纬民"] year: 2026 venue: "计算机研究与发展 (Journal of Computer Research and Development)" source: "https://crad.ict.ac.cn/article/doi/10.7544/issn1000-1239.202550461" doi: "10.7544/issn1000-1239.202550461" licence: "Free access (crad.ict.ac.cn), no open licence stated — readonly; © 计算机研究与发展" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/papers/hu-2026-kv-cache-compression/" --- [Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/hu-2026-kv-cache-compression/) [home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2026](https://md.research.elamur.ai/free/papers/2026/page/5/) / 大语言模型推理中的键值缓存压缩方法综述 # 大语言模型推理中的键值缓存压缩方法综述 **Authors:** 胡世鹏, 张广艳, 郑纬民 · **Year:** 2026 · **Venue:** 计算机研究与发展 (Journal of Computer Research and Development) · **Licence:** Free access (crad.ict.ac.cn), no open licence stated — readonly; © 计算机研究与发展 **Links:** [Publisher's page](https://doi.org/10.7544/issn1000-1239.202550461) · [PDF](https://crad.ict.ac.cn/cn/article/pdf/preview/10.7544/issn1000-1239.202550461.pdf) · [DOI 10.7544/issn1000-1239.202550461](https://doi.org/10.7544/issn1000-1239.202550461) ## Summary This survey from Tsinghua University examines KV cache compression for large language model inference. During decoding, the keys and values of previous tokens are stored so they do not have to be recomputed. That cache grows linearly with context length, model depth and the number of concurrent sessions. It causes two problems. GPU memory runs out, which limits how many requests can be served in parallel. Long-term conversational memory also forces providers to keep huge caches on SSDs, which costs money and slows loading. The authors set out three design challenges: preserving answer quality, keeping compression overhead low, and generalizing across tasks, models and context lengths. They group existing techniques into five families: merging, quantization, token eviction, sharing, and attention-head reduction. For each family they describe how representative systems work, and they compare them by compression dimension, extra computation, architectural changes and how the compression ratio is tuned. Their outlook calls for: * better generalization across tasks and scenarios, * cheaper compression, * automatic method selection with adaptive parameters. The work is a practical map for anyone building efficient LLM serving. ## Key points - The survey appeared in *Journal of Computer Research and Development* (计算机研究与发展), vol. 63(4), pp. 1021–1038, 2026. - The taxonomy's representative methods are: - **Merging:** KVMerger, MiniCache - **Quantization:** KVQuant, CacheGen, Ecco - **Token eviction:** Scissorhands, Impress, ClusterKV - **Sharing:** KVSharer, LCKV - **Head reduction:** MQA (2019), GQA (2023) - Scale example: a prompt of roughly 2,000 tokens can produce up to about 5 GB of KV cache. Even 40/80 GB NVIDIA A100 GPUs can therefore hold only a few concurrent requests, and Claude supports contexts of up to 200K tokens. - KVMerger was evaluated on LLaMa2-7b-chat with SynthWiki (average length about 4,000 tokens). Key vectors of neighbouring tokens showed strong locality, with some cosine similarities above 90%. Tokens are merged with Gaussian-kernel weighting around a pivot token chosen by attention score. - Cited evaluations show weak generalization. Compressed caches perform poorly on arithmetic reasoning such as GSM8K and in long-context QA. Answer quality on instruction-tuned models can fall by as much as 25.53%. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/) · [Natural language processing](https://md.research.elamur.ai/free/shelves/nlp/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @article{2026, title = {{大语言模型推理中的键值缓存压缩方法综述}}, author = {胡世鹏 and 张广艳 and 郑纬民}, journal = {计算机研究与发展 (Journal of Computer Research and Development)}, year = {2026}, doi = {10.7544/issn1000-1239.202550461}, url = {https://doi.org/10.7544/issn1000-1239.202550461} } ``` - [← previous 大语言模型幻觉检测方法综述](https://md.research.elamur.ai/free/papers/li-2026-hallucination-detection-methods/) - [next → 推理型大语言模型角色扮演能力评估与优化](https://md.research.elamur.ai/free/papers/feng-2026-enhancing-role-playing/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/papers/2026/page/5/) ## References 4 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. ［47］ Gao Wei, Zhou Xinyu, Sun Peng, et al. Rethinking key-value cache compression techniques for large language model serving[J]. arXiv preprint, arXiv: 2503.24000, 2025 — [arXiv:2503.24000](https://arxiv.org/abs/2503.24000) [find](https://scholar.google.com/scholar?q=%EF%BC%BB47%EF%BC%BD+Gao+Wei%2C+Zhou+Xinyu%2C+Sun+Peng%2C+et+al%2E+Rethinking+key%2Dvalue+cache+compression+techniques+for+large+language+model+serving%5BJ%5D%2E+arXiv+preprint%2C+arXiv%3A+2503%2E24000%2C+2025+%E2%80%94+%5BarXiv%3A2503%2E24000%5D%28https%3A%2F%2Fa) 2. ［48］ Liu Xiang, Tang Zhenheng, Chen Hong, et al. Can LLMs maintain fundamental abilities under KV cache compression?[J]. arXiv preprint, arXiv: 2502.01941, 2025 — [arXiv:2502.01941](https://arxiv.org/abs/2502.01941) [find](https://scholar.google.com/scholar?q=%EF%BC%BB48%EF%BC%BD+Liu+Xiang%2C+Tang+Zhenheng%2C+Chen+Hong%2C+et+al%2E+Can+LLMs+maintain+fundamental+abilities+under+KV+cache+compression%3F%5BJ%5D%2E+arXiv+preprint%2C+arXiv%3A+2502%2E01941%2C+2025+%E2%80%94+%5BarXiv%3A2502%2E01941%5D%28https%3A%2F%2Farxiv%2Eorg%2F) 3. ［49］ Cobbe K, Kosaraju V, Bavarian M, et al. Training verifiers to solve math word problems[J]. arXiv preprint, arXiv: 2110.14168, 2021 [find](https://scholar.google.com/scholar?q=%EF%BC%BB49%EF%BC%BD+Cobbe+K%2C+Kosaraju+V%2C+Bavarian+M%2C+et+al%2E+Training+verifiers+to+solve+math+word+problems%5BJ%5D%2E+arXiv+preprint%2C+arXiv%3A+2110%2E14168%2C+2021) [our card](https://md.research.elamur.ai/free/papers/cobbe-2021-training-verifiers-solve-math-word/) 4. ［50］ Liu Haotian, Li Chunyuan, Wu Qingyang, et al. Visual instruction tuning[C]//Proc of the 37th Conf on Advances in Neural Information Processing Systems. Red Hook, NY: Curran Associates Inc, 2023: [find](https://scholar.google.com/scholar?q=%EF%BC%BB50%EF%BC%BD+Liu+Haotian%2C+Li+Chunyuan%2C+Wu+Qingyang%2C+et+al%2E+Visual+instruction+tuning%5BC%5D%2F%2FProc+of+the+37th+Conf+on+Advances+in+Neural+Information+Processing+Systems%2E+Red+Hook%2C+NY%3A+Curran+Associates+Inc%2C+2023%3A)