---
title: "HqeKV — paper by Wang et al. (2026) — summary and ideas"
description: "They offer few granularity options and judge token importance mainly from keys. 31 references, cited by 1 paper here."
authors: ["He Wang", "Yu Gu", "Fangfang Li", "Zhigang Wang", "Zhenghao Liu", "Ning Wang", "Xiaohua Li", "Ge Yu"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.201/"
doi: "10.18653/v1/2026.findings-acl.201"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wang-2026-hqekv-hybrid-quantization-eviction-kv/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2026-hqekv-hybrid-quantization-eviction-kv/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/13/) / HqeKV: Towards Hybrid Quantization and Eviction for KV Cache in Long-Context LLM Inference

# HqeKV: Towards Hybrid Quantization and Eviction for KV Cache in Long-Context LLM Inference

**Authors:** [He Wang](https://md.research.elamur.ai/free/en/authors/he-wang/), [Yu Gu](https://md.research.elamur.ai/free/en/authors/yu-gu/), Fangfang Li, Zhigang Wang, [Zhenghao Liu](https://md.research.elamur.ai/free/en/authors/zhenghao-liu/), [Ning Wang](https://md.research.elamur.ai/free/en/authors/ning-wang/), Xiaohua Li and [Ge Yu](https://md.research.elamur.ai/free/en/authors/ge-yu/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.201/) · [PDF](https://aclanthology.org/2026.findings-acl.201.pdf) · [DOI 10.18653/v1/2026.findings-acl.201](https://doi.org/10.18653/v1/2026.findings-acl.201) · [OpenAlex](https://openalex.org/W7166850384)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 149,439 papers whose bibliographies we read

## Summary
Storing key-value (KV) pairs during autoregressive decoding avoids recomputation, but the cache grows with sequence length and can outweigh the model's own memory in long-context serving. Existing compressors mostly choose one tool, either quantization or eviction. They offer few granularity options and judge token importance mainly from keys. HqeKV merges both tools. Cached pairs are ranked by importance and each receives one of five treatments: keep in FP16, quantize to 4, 2 or 1 bit, or drop. An offline search picks the share of each tier for a given memory budget. Importance is scored by a new joint K–V metric that adds the numeric range of value vectors to cumulative attention weights. The authors back this choice with a derivation and empirical evidence that quantization error rises with range. Uniform quantization serves the higher bit-widths and normalized quantization the lowest ones. On LongBench with several LLMs, HqeKV beats prior methods at equal memory. This matters for running long-context inference on limited GPU memory.

## Key points
- **Five compression actions per cached element:** FP16, 4-bit, 2-bit, 1-bit, or eviction. The cache is re-quantized periodically during decoding, after every T new tokens.
- **Range as an error signal:** the correlation (PCC) between conversion loss and value-vector range is 0.96 for 4-bit V Cache tokens, versus 0.88 for random normal vectors. This motivates the joint metric over prior K-only or VATP-style scoring.
- **Headline result:** on LongBench under the same memory limit, quality rises from 40.53 for the best competitor to 49.98. Baselines include eviction methods (H2O, SnapKV, PyramidKV, CAKE), quantization methods (KIVI, KVTuner) and hybrids (MiniKV, EVICPRESS).
- **Offline ratio search:** for a 3.2-bit average budget it took about 7 min 46 s. The allocation was roughly 0.65 / 0.28 / 0.06 / 0.02 for 4-bit / 2-bit / 1-bit / eviction, and 200 versus 300 iterations made little difference.
- **Best metric hyperparameters:** α = 2 and β = 4 scored 56.75 on HotpotQA with Llama-3.1-8B-Instruct. Code is published at github.com/skywclouds/HqeKV.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Northeastern University](https://md.research.elamur.ai/free/en/universities/northeastern-university/) — University, Boston, United States · 57 works · 6,533 citations
- [Northeastern University](https://md.research.elamur.ai/free/en/universities/northeastern-university-cn/) — University, Shenyang, China · 42 works · 1,940 citations
- [Guangzhou University](https://md.research.elamur.ai/free/en/universities/guangzhou-university/) — University, Guangzhou, China · 4 works · 7 citations
- [Ocean University of China](https://md.research.elamur.ai/free/en/universities/ocean-university-of-china/) — University, Qingdao, China · 2 works · 2 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🪶 Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/) · 242 works

## Cite

Cite as: He Wang, Yu Gu, Fangfang Li, Zhigang Wang, Zhenghao Liu, Ning Wang, Xiaohua Li and Ge Yu. 2026. HqeKV: Towards Hybrid Quantization and Eviction for KV Cache in Long-Context LLM Inference. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wang2026hqekv,
  title = {{HqeKV: Towards Hybrid Quantization and Eviction for KV Cache in Long-Context LLM Inference}},
  author = {He Wang, Yu Gu, Fangfang Li, Zhigang Wang, Zhenghao Liu, Ning Wang, Xiaohua Li and Ge Yu},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.201},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.201}
}
```

- [← previous HPLT 3.0: Very Large-Scale Multilingual Resources for LLMs and MT. Mono- and Bi-lingual Data, Multilingual Evaluation, and Pre-Trained Models](https://md.research.elamur.ai/free/en/papers/oepen-2026-hplt-3-0-very-large/)
- [next → HSE NLP TEAM at MEDIQA-SYNUR 2026: Consensus Adjudication Ensemble (ACE): Balancing Precision and Recall for Schema-Bystander Clinical Extraction](https://md.research.elamur.ai/free/en/papers/valiev-2026-hse-nlp-team-mediqa-synur/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/13/)

## References

31 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Mislav Balunović, Jasper Dekoninck, Ivo Petrov, Nikola Jovanović, and Martin Vechev. 2025. Matharena: Evaluating llms on uncontaminated math competitions. [find](https://scholar.google.com/scholar?q=Mislav+Balunovic%CC%81%2C+Jasper+Dekoninck%2C+Ivo+Petrov%2C+Nikola+Jovanovic%CC%81%2C+and+Martin+Vechev%2E+2025%2E+Matharena%3A+Evaluating+llms+on+uncontaminated+math+competitions%2E)
2. James Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. 2011. Algorithms for hyper-parameter optimization. Advances in neural information processing systems, 24. [find](https://scholar.google.com/scholar?q=James+Bergstra%2C+R%C3%A9mi+Bardenet%2C+Yoshua+Bengio%2C+and+Bal%C3%A1zs+K%C3%A9gl%2E+2011%2E+Algorithms+for+hyper%2Dparameter+optimization%2E+Advances+in+neural+information+processing+systems%2C+24%2E)
3. Zefan Cai, Yichi Zhang, Bofei Gao, Yuliang Liu, Tianyu Liu, Keming Lu, Wayne Xiong, Yue Dong, Baobao Chang, Junjie Hu, and Wen Xiao. 2024. [Pyramidkv: Dynamic KV cache compression based on pyramidal information funneling](https://md.research.elamur.ai/free/en/papers/cai-2024-pyramidkv-dynamic-kv-cache-compression/). CoRR, abs/2406.02069.
4. Zhihang Cai, Xingjun Zhang, Zhendong Tan, and Zheng Wei. 2025. Nqkv: A kv cache quantization scheme based on normal distribution characteristics. arXiv preprint arXiv:2505.16210. — [arXiv:2505.16210](https://arxiv.org/abs/2505.16210) [find](https://scholar.google.com/scholar?q=Zhihang+Cai%2C+Xingjun+Zhang%2C+Zhendong+Tan%2C+and+Zheng+Wei%2E+2025%2E+Nqkv%3A+A+kv+cache+quantization+scheme+based+on+normal+distribution+characteristics%2E+arXiv+preprint+arXiv%3A2505%2E16210%2E+%E2%80%94+%5BarXiv%3A2505%2E16210%5D%28)
5. Wen Cheng, Shichen Dong, Jiayu Qin, and Wei Wang. 2025. Qaq: Quality adaptive quantization for llm kv cache. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2542– 2550. [find](https://scholar.google.com/scholar?q=Wen+Cheng%2C+Shichen+Dong%2C+Jiayu+Qin%2C+and+Wei+Wang%2E+2025%2E+Qaq%3A+Quality+adaptive+quantization+for+llm+kv+cache%2E+In+Proceedings+of+the+IEEE%2FCVF+International+Conference+on+Computer+Vision%2C+pages+2542%E2%80%93+255)
6. Haojie Duanmu, Zhihang Yuan, Xiuhong Li, Jiangfei Duan, Xingcheng Zhang, and Dahua Lin. 2024. Skvq: Sliding-window key and value cache quantization for large language models. arXiv preprint arXiv:2405.06219. [find](https://scholar.google.com/scholar?q=Haojie+Duanmu%2C+Zhihang+Yuan%2C+Xiuhong+Li%2C+Jiangfei+Duan%2C+Xingcheng+Zhang%2C+and+Dahua+Lin%2E+2024%2E+Skvq%3A+Sliding%2Dwindow+key+and+value+cache+quantization+for+large+language+models%2E+arXiv+preprint+arXiv%3A2405)
7. Shaoting Feng, Yuhan Liu, Hanchen Li, Xiaokun Chen, Samuel Shen, Kuntai Du, Zhuohan Gu, Rui Zhang, Yuyang Huang, Yihua Cheng, and 1 others. 2025. Evicpress: Joint kv-cache compression and eviction for efficient llm serving. arXiv preprint arXiv:2512.14946. [find](https://scholar.google.com/scholar?q=Shaoting+Feng%2C+Yuhan+Liu%2C+Hanchen+Li%2C+Xiaokun+Chen%2C+Samuel+Shen%2C+Kuntai+Du%2C+Zhuohan+Gu%2C+Rui+Zhang%2C+Yuyang+Huang%2C+Yihua+Cheng%2C+and+1+others%2E+2025%2E+Evicpress%3A+Joint+kv%2Dcache+compression+and+eviction+for)
8. Ravi Ghadia, Avinash Kumar, Gaurav Jain, Prashant J. Nair, and Poulami Das. 2025. Dialogue without limits: Constant-sized KV caches for extended response in LLMs. In Forty-second International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Ravi+Ghadia%2C+Avinash+Kumar%2C+Gaurav+Jain%2C+Prashant+J%2E+Nair%2C+and+Poulami+Das%2E+2025%2E+Dialogue+without+limits%3A+Constant%2Dsized+KV+caches+for+extended+response+in+LLMs%2E+In+Forty%2Dsecond+International+Confere)
9. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783.
10. Zhiyu Guo, Hidetaka Kamigaito, and Taro Watanabe. 2024. Attention score is not all you need for token importance indicator in KV cache reduction: Value also matters. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 21158–21166, Miami, Florida, USA. Ass… [find](https://scholar.google.com/scholar?q=Zhiyu+Guo%2C+Hidetaka+Kamigaito%2C+and+Taro+Watanabe%2E+2024%2E+Attention+score+is+not+all+you+need+for+token+importance+indicator+in+KV+cache+reduction%3A+Value+also+matters%2E+In+Proceedings+of+the+2024+Confere)
11. Yefei He, Luoming Zhang, Weijia Wu, Jing Liu, Hong Zhou, and Bohan Zhuang. 2024. Zipcache: Accurate and efficient kv cache quantization with salient token identification. In Advances in Neural Information Processing Systems, volume 37, pages 68287–68307. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Yefei+He%2C+Luoming+Zhang%2C+Weijia+Wu%2C+Jing+Liu%2C+Hong+Zhou%2C+and+Bohan+Zhuang%2E+2024%2E+Zipcache%3A+Accurate+and+efficient+kv+cache+quantization+with+salient+token+identification%2E+In+Advances+in+Neural+Informa)
12. Coleman Hooper, Sehoon Kim, Hiva Mohammadzadeh, Michael W. Mahoney, Yakun Sophia Shao, Kurt Keutzer, and Amir Gholami. 2024. [Kvquant: Towards 10 million context length llm inference with kv cache quantization](https://md.research.elamur.ai/free/en/papers/hooper-2024-kvquant-towards-10-million-context/). In Advances in Neural Information Processing Systems, volume 37, pages 1270– 1303. Curran…
13. Minsu Kim, Seongmin Hong, RyeoWook Ko, Soongyu Choi, Hunjong Lee, Junsoo Kim, Joo-Young Kim, and Jongse Park. 2025. Oaken: Fast and efficient llm serving with online-offline hybrid kv cache quantization. In Proceedings of the 52nd Annual International Symposium on Computer Architecture, ISCA ’25, pa… [find](https://scholar.google.com/scholar?q=Minsu+Kim%2C+Seongmin+Hong%2C+RyeoWook+Ko%2C+Soongyu+Choi%2C+Hunjong+Lee%2C+Junsoo+Kim%2C+Joo%2DYoung+Kim%2C+and+Jongse+Park%2E+2025%2E+Oaken%3A+Fast+and+efficient+llm+serving+with+online%2Doffline+hybrid+kv+cache+quantizati)
14. Xing Li, Zeyu XING, Yiming Li, Linping Qu, Hui-Ling Zhen, Yiwu Yao, Wulong Liu, Sinno Jialin Pan, and Mingxuan Yuan. 2025. KVTuner: Sensitivity-aware layer-wise mixed-precision KV cache quantization for efficient and nearly lossless LLM inference. In Forty-second International Conference on Machine… [find](https://scholar.google.com/scholar?q=Xing+Li%2C+Zeyu+XING%2C+Yiming+Li%2C+Linping+Qu%2C+Hui%2DLing+Zhen%2C+Yiwu+Yao%2C+Wulong+Liu%2C+Sinno+Jialin+Pan%2C+and+Mingxuan+Yuan%2E+2025%2E+KVTuner%3A+Sensitivity%2Daware+layer%2Dwise+mixed%2Dprecision+KV+cache+quantization+f)
15. Yuhong Li, Yingbing Huang, Bowen Yang, Bharat Venkitesh, Acyr Locatelli, Hanchen Ye, Tianle Cai, Patrick Lewis, and Deming Chen. 2024. [SnapKV: LLM knows what you are looking for before generation](https://md.research.elamur.ai/free/en/papers/li-2024-snapkv-llm-knows-looking-before/). In The Thirty-eighth Annual Conference on Neural Information Processing Systems.
16. Zechun Liu, Changsheng Zhao, Igor Fedorov, Bilge Soran, Dhruv Choudhary, Raghuraman Krishnamoorthi, Vikas Chandra, Yuandong Tian, and Tijmen Blankevoort. 2025. Spinquant: LLM quantization with learned rotations. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Zechun+Liu%2C+Changsheng+Zhao%2C+Igor+Fedorov%2C+Bilge+Soran%2C+Dhruv+Choudhary%2C+Raghuraman+Krishnamoorthi%2C+Vikas+Chandra%2C+Yuandong+Tian%2C+and+Tijmen+Blankevoort%2E+2025%2E+Spinquant%3A+LLM+quantization+with+learned)
17. Zichang Liu, Aditya Desai, Fangshuo Liao, Weitao Wang, Victor Xie, Zhaozhuo Xu, Anastasios Kyrillidis, and Anshumali Shrivastava. 2023. Scissorhands: Exploiting the persistence of importance hypothesis for llm kv cache compression at test time. Advances in Neural Information Processing Systems, 36:5… [find](https://scholar.google.com/scholar?q=Zichang+Liu%2C+Aditya+Desai%2C+Fangshuo+Liao%2C+Weitao+Wang%2C+Victor+Xie%2C+Zhaozhuo+Xu%2C+Anastasios+Kyrillidis%2C+and+Anshumali+Shrivastava%2E+2023%2E+Scissorhands%3A+Exploiting+the+persistence+of+importance+hypothesi)
18. Zirui Liu, Jiayi Yuan, Hongye Jin, Shaochen (Henry) Zhong, Zhaozhuo Xu, Vladimir Braverman, Beidi Chen, and Xia Hu. 2024. [Kivi: a tuning-free asymmetric 2bit quantization for kv cache](https://md.research.elamur.ai/free/en/papers/liu-2024-kivi-tuning-free-asymmetric-2bit/). In Proceedings of the 41st International Conference on Machine Learning, ICML’24. JMLR.org.
19. Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2016. [Pointer sentinel mixture models](https://md.research.elamur.ai/free/en/papers/merity-2016-pointer-sentinel-mixture-models/). Preprint, arXiv:1609.07843.
20. Ziran Qin, Yuchen Cao, Mingbao Lin, Wen Hu, Shixuan Fan, Ke Cheng, Weiyao Lin, and Jianguo Li. 2025. CAKE: Cascading and adaptive KV cache eviction with layer preferences. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Ziran+Qin%2C+Yuchen+Cao%2C+Mingbao+Lin%2C+Wen+Hu%2C+Shixuan+Fan%2C+Ke+Cheng%2C+Weiyao+Lin%2C+and+Jianguo+Li%2E+2025%2E+CAKE%3A+Cascading+and+adaptive+KV+cache+eviction+with+layer+preferences%2E+In+The+Thirteenth+Internatio)
21. Akshat Sharma, Hangliang Ding, Jianping Li, Neel Dani, and Minjia Zhang. 2025. MiniKV: Pushing the limits of 2-bit KV cache via compression and system co-design for efficient long context inference. In Findings of the Association for Computational Linguistics: ACL 2025, pages 18506–18523, Vienna, Au… [find](https://scholar.google.com/scholar?q=Akshat+Sharma%2C+Hangliang+Ding%2C+Jianping+Li%2C+Neel+Dani%2C+and+Minjia+Zhang%2E+2025%2E+MiniKV%3A+Pushing+the+limits+of+2%2Dbit+KV+cache+via+compression+and+system+co%2Ddesign+for+efficient+long+context+inference%2E+I)
22. Yi Su, Yuechi Zhou, Quantong Qiu, Juntao Li, Qingrong Xia, Ping Li, Xinyu Duan, Zhefeng Wang, and Min Zhang. 2025. Accurate KV cache quantization with outlier tokens tracing. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 128… [find](https://scholar.google.com/scholar?q=Yi+Su%2C+Yuechi+Zhou%2C+Quantong+Qiu%2C+Juntao+Li%2C+Qingrong+Xia%2C+Ping+Li%2C+Xinyu+Duan%2C+Zhefeng+Wang%2C+and+Min+Zhang%2E+2025%2E+Accurate+KV+cache+quantization+with+outlier+tokens+tracing%2E+In+Proceedings+of+the+63r)
23. Wei Tao, Bin Zhang, Xiaoyang Qu, Jiguang Wan, and Jianzong Wang. 2025. Cocktail: Chunk-adaptive mixed-precision quantization for long-context llm inference. In 2025 Design, Automation & Test in Europe Conference (DATE), pages 1–7. IEEE. [find](https://scholar.google.com/scholar?q=Wei+Tao%2C+Bin+Zhang%2C+Xiaoyang+Qu%2C+Jiguang+Wan%2C+and+Jianzong+Wang%2E+2025%2E+Cocktail%3A+Chunk%2Dadaptive+mixed%2Dprecision+quantization+for+long%2Dcontext+llm+inference%2E+In+2025+Design%2C+Automation+%26+Test+in+Europe)
24. Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. 2024. Efficient streaming language models with attention sinks. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net. [find](https://scholar.google.com/scholar?q=Guangxuan+Xiao%2C+Yuandong+Tian%2C+Beidi+Chen%2C+Song+Han%2C+and+Mike+Lewis%2E+2024%2E+Efficient+streaming+language+models+with+attention+sinks%2E+In+The+Twelfth+International+Conference+on+Learning+Representations)
25. Yuhui Xu, Zhanming Jie, Hanze Dong, Lei Wang, Xudong Lu, Aojun Zhou, Amrita Saha, Caiming Xiong, and Doyen Sahoo. 2025. Think: Thinner key cache by query-driven pruning. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Yuhui+Xu%2C+Zhanming+Jie%2C+Hanze+Dong%2C+Lei+Wang%2C+Xudong+Lu%2C+Aojun+Zhou%2C+Amrita+Saha%2C+Caiming+Xiong%2C+and+Doyen+Sahoo%2E+2025%2E+Think%3A+Thinner+key+cache+by+query%2Ddriven+pruning%2E+In+The+Thirteenth+Internationa)
26. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). arXiv preprint arXiv:2505.09388.
27. June Yong Yang, Byeongwook Kim, Jeongin Bae, Beomseok Kwon, Gunho Park, Eunho Yang, Se Jung Kwon, and Dongsoo Lee. 2024. No token left behind: Reliable kv cache compression via importance-aware mixed precision quantization. CoRR, abs/2402.18096. — [arXiv:2402.18096](https://arxiv.org/abs/2402.18096) [find](https://scholar.google.com/scholar?q=June+Yong+Yang%2C+Byeongwook+Kim%2C+Jeongin+Bae%2C+Beomseok+Kwon%2C+Gunho+Park%2C+Eunho+Yang%2C+Se+Jung+Kwon%2C+and+Dongsoo+Lee%2E+2024%2E+No+token+left+behind%3A+Reliable+kv+cache+compression+via+importance%2Daware+mixed+)
28. Amir Zandieh, Majid Daliri, and Insu Han. 2025. Qjl: 1-bit quantized jl transform for kv cache quantization with zero overhead. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 25805–25813. [find](https://scholar.google.com/scholar?q=Amir+Zandieh%2C+Majid+Daliri%2C+and+Insu+Han%2E+2025%2E+Qjl%3A+1%2Dbit+quantized+jl+transform+for+kv+cache+quantization+with+zero+overhead%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+volume)
29. Hailin Zhang, Xiaodong Ji, Yilin Chen, Fangcheng Fu, Xupeng Miao, Xiaonan Nie, Weipeng Chen, and Bin Cui. 2025. Pqcache: Product quantization-based kvcache for long context llm inference. Proceedings of the ACM on Management of Data, 3(3):1–30. [find](https://scholar.google.com/scholar?q=Hailin+Zhang%2C+Xiaodong+Ji%2C+Yilin+Chen%2C+Fangcheng+Fu%2C+Xupeng+Miao%2C+Xiaonan+Nie%2C+Weipeng+Chen%2C+and+Bin+Cui%2E+2025%2E+Pqcache%3A+Product+quantization%2Dbased+kvcache+for+long+context+llm+inference%2E+Proceedings+)
30. Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark Barrett, and 1 others. 2023. H2o: Heavy-hitter oracle for efficient generative inference of large language models. Advances in Neural Information Processing Systems, 36:346… [find](https://scholar.google.com/scholar?q=Zhenyu+Zhang%2C+Ying+Sheng%2C+Tianyi+Zhou%2C+Tianlong+Chen%2C+Lianmin+Zheng%2C+Ruisi+Cai%2C+Zhao+Song%2C+Yuandong+Tian%2C+Christopher+R%C3%A9%2C+Clark+Barrett%2C+and+1+others%2E+2023%2E+H2o%3A+Heavy%2Dhitter+oracle+for+efficient+gene)
31. Yilong Zhao, Chien-Yu Lin, Kan Zhu, Zihao Ye, Lequn Chen, Size Zheng, Luis Ceze, Arvind Krishnamurthy, Tianqi Chen, and Baris Kasikci. 2024. Atom: Low-bit quantization for efficient and accurate llm serving. Proceedings of Machine Learning and Systems, 6:196–209. [find](https://scholar.google.com/scholar?q=Yilong+Zhao%2C+Chien%2DYu+Lin%2C+Kan+Zhu%2C+Zihao+Ye%2C+Lequn+Chen%2C+Size+Zheng%2C+Luis+Ceze%2C+Arvind+Krishnamurthy%2C+Tianqi+Chen%2C+and+Baris+Kasikci%2E+2024%2E+Atom%3A+Low%2Dbit+quantization+for+efficient+and+accurate+llm+s)

*Updated 1 November 2026*