---
title: "TurboRAG — paper by Lu et al. (2025) — summary and ideas"
description: "The authors base this design on two observations. 32 references, cited by 20 papers here."
authors: ["Songshuo Lu", "Hua Wang", "Yutian Rong", "Zhi Chen", "Yaohua Tang"]
year: 2025
venue: "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/2025.emnlp-main.334/"
doi: "10.18653/v1/2025.emnlp-main.334"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/lu-2025-turborag-accelerating-retrieval-augmented-generation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/lu-2025-turborag-accelerating-retrieval-augmented-generation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/3/) / TurboRAG: Accelerating Retrieval-Augmented Generation with Precomputed KV Caches for Chunked Text

# TurboRAG: Accelerating Retrieval-Augmented Generation with Precomputed KV Caches for Chunked Text

**Authors:** Songshuo Lu, Hua Wang, Yutian Rong, [Zhi Chen](https://md.research.elamur.ai/free/en/authors/zhi-chen/) and Yaohua Tang · **Year:** 2025 · **Venue:** [Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2025](https://md.research.elamur.ai/free/en/venues/emnlp/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.emnlp-main.334/) · [PDF](https://aclanthology.org/2025.emnlp-main.334.pdf) · [DOI 10.18653/v1/2025.emnlp-main.334](https://doi.org/10.18653/v1/2025.emnlp-main.334) · [OpenAlex](https://openalex.org/W4416037017)

**Citations:** 12 citations (OpenAlex, 2026-10-14) · [20](https://md.research.elamur.ai/free/en/papers/lu-2025-turborag-accelerating-retrieval-augmented-generation/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Standard RAG pipelines concatenate every retrieved chunk with the query and run prefill on the whole sequence for each request. This repeats work for passages that are retrieved often and makes time-to-first-token (TTFT) grow quickly as context gets longer. TurboRAG moves most of that work offline. Each document chunk gets its own KV cache, computed ahead of time and kept in storage. When a query arrives, the system loads the caches for the retrieved chunks and joins them together.

Two techniques keep this joining consistent:
- an independent-attention mask, so a chunk attends only to itself;
- reordered RoPE position IDs, so the combined context still has monotonically increasing positions.

The authors base this design on two observations. Attention between chunks is sparse in typical RAG inputs, and RoPE depends only on relative token distances. The base LLM is lightly fine-tuned so it can read the new cache layout. Its architecture stays unchanged.

On LongBench multi-document QA, accuracy stays close to standard RAG while TTFT falls substantially. This matters most for latency-sensitive customer-support bots and for on-device assistants with limited GPU capacity.

## Key points
- **Speed:** TTFT drops by up to 9.4× (8.6× on average) on LongBench multi-document QA, with accuracy comparable to standard RAG. Online compute (TFLOPs) falls by about 98.5%, which also allows larger batch sizes.
- **Scaling and transfer cost:** The speedup holds across the Qwen-2 family from 1.5B to 72B parameters. On the 72B model (4 GPUs), it ranges from roughly 9× to 11×. Even when caches must be copied from host to GPU, TurboRAG is still about 4× faster than naïve RAG.
- **Fine-tuning requirement:** The abstract says no model modification is needed, yet the method relies on supervised fine-tuning. The authors list this dependency as a limitation.
- **Storage cost:** For Qwen2-7B, one 512-token FP16 chunk cache takes about 28 MB, so one million chunks need about 28 TB. The authors argue that cheap disks and 4/8-bit quantization make this cost acceptable.
- **Related work:** The approach is close to Block-Attention, which was developed independently. It is described as complementary to KV-cache compression and speculative decoding.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works

🧭 Topics (2): [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) · 473 works · [🪶 Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/) · 242 works

## Cite

Cite as: Songshuo Lu, Hua Wang, Yutian Rong, Zhi Chen and Yaohua Tang. 2025. TurboRAG: Accelerating Retrieval-Augmented Generation with Precomputed KV Caches for Chunked Text. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{lu2025turborag,
  title = {{TurboRAG: Accelerating Retrieval-Augmented Generation with Precomputed KV Caches for Chunked Text}},
  author = {Songshuo Lu, Hua Wang, Yutian Rong, Zhi Chen and Yaohua Tang},
  booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  doi = {10.18653/v1/2025.emnlp-main.334},
  url = {https://doi.org/10.18653/v1/2025.emnlp-main.334}
}
```

- [← previous The Hidden Attention of Mamba Models](https://md.research.elamur.ai/free/en/papers/ali-2025-hidden-attention-mamba-models/)
- [next → VLSBench: Unveiling Visual Leakage in Multimodal Safety](https://md.research.elamur.ai/free/en/papers/hu-2025-vlsbench-unveiling-visual-leakage-multimodal/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/3/)

## References

32 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, et al. 2023. Longbench: A 16 TB Exos X18 enterprise drive lists for $210 on Amazon ($13/TB), while an 18 TB WD Red Pro sells for about $350 ($19/TB). [find](https://scholar.google.com/scholar?q=Yushi+Bai%2C+Xin+Lv%2C+Jiajie+Zhang%2C+Hongchang+Lyu%2C+Jiankai+Tang%2C+Zhidian+Huang%2C+Zhengxiao+Du%2C+Xiao+Liu%2C+Aohan+Zeng%2C+Lei+Hou%2C+et+al%2E+2023%2E+Longbench%3A+A+16+TB+Exos+X18+enterprise+drive+lists+for+%24210+on+Am)
2. A bilingual, multitask benchmark for long context understanding. arXiv preprint arXiv:2308.14508. — [LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding](https://md.research.elamur.ai/free/en/papers/bai-2023-longbench-bilingual-multitask-benchmark/)
3. Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, et al. 2022. Improving language models by retrieving from trillions of tokens. In International conference on machine learn… [find](https://scholar.google.com/scholar?q=Sebastian+Borgeaud%2C+Arthur+Mensch%2C+Jordan+Hoffmann%2C+Trevor+Cai%2C+Eliza+Rutherford%2C+Katie+Millican%2C+George+Bm+Van+Den+Driessche%2C+Jean%2DBaptiste+Lespiau%2C+Bogdan+Damoc%2C+Aidan+Clark%2C+et+al%2E+2022%2E+Improving+)
4. Jiawei Chen, Hongyu Lin, Xianpei Han, and Le Sun. 2024. [Benchmarking large language models in retrieval-augmented generation](https://md.research.elamur.ai/free/en/papers/chen-2023-benchmarking-large-language-models-retrieval/). In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 17754–17762.
5. Shouyuan Chen, Sherman Wong, Liangjian Chen, and Yuandong Tian. 2023. Extending context window of large language models via positional interpolation. arXiv preprint arXiv:2306.15595. [find](https://scholar.google.com/scholar?q=Shouyuan+Chen%2C+Sherman+Wong%2C+Liangjian+Chen%2C+and+Yuandong+Tian%2E+2023%2E+Extending+context+window+of+large+language+models+via+positional+interpolation%2E+arXiv+preprint+arXiv%3A2306%2E15595%2E)
6. Krzysztof Choromanski, Valerii Likhosherstov, David Dohan, Xingyou Song, Andreea Gane, Tamas Sarlos, Peter Hawkins, Jared Davis, Afroz Mohiuddin, Lukasz Kaiser, et al. 2020. [Rethinking attention with performers](https://md.research.elamur.ai/free/en/papers/choromanski-2020-rethinking-attention-performers/). arXiv preprint arXiv:2009.14794.
7. Tri Dao. 2023. [Flashattention-2: Faster attention with better parallelism and work partitioning](https://md.research.elamur.ai/free/en/papers/dao-2023-flashattention-2-faster-attention-better/). arXiv preprint arXiv:2307.08691.
8. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). Preprint, arXiv:1810.04805.
9. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783.
10. Gautier Izacard and Edouard Grave. 2020. [Leveraging passage retrieval with generative models for open domain question answering](https://md.research.elamur.ai/free/en/papers/izacard-2021-leveraging-passage-retrieval-generative-models/). arXiv preprint arXiv:2007.01282.
11. Wenqi Jiang, Shuai Zhang, Boran Han, Jie Wang, Bernie Wang, and Tim Kraska. 2024. Piperag: Fast retrieval-augmented generation via algorithm-system co-design. arXiv preprint arXiv:2403.05676. — [arXiv:2403.05676](https://arxiv.org/abs/2403.05676) [find](https://scholar.google.com/scholar?q=Wenqi+Jiang%2C+Shuai+Zhang%2C+Boran+Han%2C+Jie+Wang%2C+Bernie+Wang%2C+and+Tim+Kraska%2E+2024%2E+Piperag%3A+Fast+retrieval%2Daugmented+generation+via+algorithm%2Dsystem+co%2Ddesign%2E+arXiv+preprint+arXiv%3A2403%2E05676%2E+%E2%80%94+%5BarXiv)
12. Chao Jin, Zili Zhang, Xuanlin Jiang, Fangyue Liu, Xin Liu, Xuanzhe Liu, and Xin Jin. 2024. Ragcache: Efficient knowledge caching for retrieval-augmented generation. arXiv preprint arXiv:2404.12457. [find](https://scholar.google.com/scholar?q=Chao+Jin%2C+Zili+Zhang%2C+Xuanlin+Jiang%2C+Fangyue+Liu%2C+Xin+Liu%2C+Xuanzhe+Liu%2C+and+Xin+Jin%2E+2024%2E+Ragcache%3A+Efficient+knowledge+caching+for+retrieval%2Daugmented+generation%2E+arXiv+preprint+arXiv%3A2404%2E12457%2E)
13. Nikita Kitaev, Łukasz Kaiser, and Anselm Levskaya. 2020. [Reformer: The efficient transformer](https://md.research.elamur.ai/free/en/papers/kitaev-2020-reformer-efficient-transformer/). arXiv preprint arXiv:2001.04451.
14. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. [Retrieval-augmented generation for knowledge-intensive nlp tasks](https://md.research.elamur.ai/free/en/papers/lewis-2020-retrieval-augmented-generation-knowledge-intensive/). Advances in Neural Information Processing Systems, 33:9459–9474.
15. Xiang Liu, Zhenheng Tang, Peijie Dong, Zeyu Li, Bo Li, Xuming Hu, and Xiaowen Chu. 2025. Chunkkv: Semantic-preserving kv cache compression for efficient long-context llm inference. arXiv preprint arXiv:2502.00299. [find](https://scholar.google.com/scholar?q=Xiang+Liu%2C+Zhenheng+Tang%2C+Peijie+Dong%2C+Zeyu+Li%2C+Bo+Li%2C+Xuming+Hu%2C+and+Xiaowen+Chu%2E+2025%2E+Chunkkv%3A+Semantic%2Dpreserving+kv+cache+compression+for+efficient+long%2Dcontext+llm+inference%2E+arXiv+preprint+arXi)
16. Yuhan Liu, Hanchen Li, Yihua Cheng, Siddhant Ray, Yuyang Huang, Qizheng Zhang, Kuntai Du, Jiayi Yao, Shan Lu, Ganesh Ananthanarayanan, et al. 2024. Cachegen: Kv cache compression and streaming for fast large language model serving. In Proceedings of the ACM SIGCOMM 2024 Conference, pages 38–56. [find](https://scholar.google.com/scholar?q=Yuhan+Liu%2C+Hanchen+Li%2C+Yihua+Cheng%2C+Siddhant+Ray%2C+Yuyang+Huang%2C+Qizheng+Zhang%2C+Kuntai+Du%2C+Jiayi+Yao%2C+Shan+Lu%2C+Ganesh+Ananthanarayanan%2C+et+al%2E+2024%2E+Cachegen%3A+Kv+cache+compression+and+streaming+for+fas)
17. I Loshchilov. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101. [find](https://scholar.google.com/scholar?q=I+Loshchilov%2E+2017%2E+Decoupled+weight+decay+regularization%2E+arXiv+preprint+arXiv%3A1711%2E05101%2E)
18. Dongyang Ma, Yan Wang, and Lan Tian. 2024. Block-attention for efficient prefilling. arXiv preprint arXiv:2409.15355. — [arXiv:2409.15355](https://arxiv.org/abs/2409.15355) [find](https://scholar.google.com/scholar?q=Dongyang+Ma%2C+Yan+Wang%2C+and+Lan+Tian%2E+2024%2E+Block%2Dattention+for+efficient+prefilling%2E+arXiv+preprint+arXiv%3A2409%2E15355%2E+%E2%80%94+%5BarXiv%3A2409%2E15355%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2409%2E15355%29)
19. Rui Pan, Zhuang Wang, Zhen Jia, Can Karakus, Luca Zancato, Tri Dao, Yida Wang, and Ravi Netravali. 2024. Marconi: Prefix caching for the era of hybrid llms. arXiv preprint arXiv:2411.19379. — [arXiv:2411.19379](https://arxiv.org/abs/2411.19379) [find](https://scholar.google.com/scholar?q=Rui+Pan%2C+Zhuang+Wang%2C+Zhen+Jia%2C+Can+Karakus%2C+Luca+Zancato%2C+Tri+Dao%2C+Yida+Wang%2C+and+Ravi+Netravali%2E+2024%2E+Marconi%3A+Prefix+caching+for+the+era+of+hybrid+llms%2E+arXiv+preprint+arXiv%3A2411%2E19379%2E+%E2%80%94+%5BarXiv%3A2)
20. Guofeng Quan, Wenfeng Feng, Chuzhan Hao, Guochao Jiang, Yuewei Zhang, and Hao Wang. 2025. Rasd: Retrieval-augmented speculative decoding. arXiv preprint arXiv:2503.03434. [find](https://scholar.google.com/scholar?q=Guofeng+Quan%2C+Wenfeng+Feng%2C+Chuzhan+Hao%2C+Guochao+Jiang%2C+Yuewei+Zhang%2C+and+Hao+Wang%2E+2025%2E+Rasd%3A+Retrieval%2Daugmented+speculative+decoding%2E+arXiv+preprint+arXiv%3A2503%2E03434%2E)
21. Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023. [In-context retrieval-augmented language models](https://md.research.elamur.ai/free/en/papers/ram-2023-context-retrieval-augmented-language-models/). Transactions of the Association for Computational Linguistics, 11:1316–1331.
22. Zhibin Wang, Rui Ning, Chao Fang, Zhonghui Zhang, Xi Lin, Shaobo Ma, Mo Zhou, Xue Li, Zhongfeng Wang, Chengying Huan, et al. 2025. Flashforge: Ultra-efficient prefix-aware attention for llm decoding. arXiv preprint arXiv:2505.17694. — [arXiv:2505.17694](https://arxiv.org/abs/2505.17694) [find](https://scholar.google.com/scholar?q=Zhibin+Wang%2C+Rui+Ning%2C+Chao+Fang%2C+Zhonghui+Zhang%2C+Xi+Lin%2C+Shaobo+Ma%2C+Mo+Zhou%2C+Xue+Li%2C+Zhongfeng+Wang%2C+Chengying+Huan%2C+et+al%2E+2025%2E+Flashforge%3A+Ultra%2Defficient+prefix%2Daware+attention+for+llm+decoding%2E+)
23. Zilong Wang, Zifeng Wang, Long Le, Huaixiu Steven Zheng, Swaroop Mishra, Vincent Perot, Yuwei Zhang, Anush Mattapalli, Ankur Taly, Jingbo Shang, et al. 2024b. Speculative rag: Enhancing retrieval augmented generation through drafting. arXiv preprint arXiv:2407.08223. — [arXiv:2407.08223](https://arxiv.org/abs/2407.08223) [find](https://scholar.google.com/scholar?q=Zilong+Wang%2C+Zifeng+Wang%2C+Long+Le%2C+Huaixiu+Steven+Zheng%2C+Swaroop+Mishra%2C+Vincent+Perot%2C+Yuwei+Zhang%2C+Anush+Mattapalli%2C+Ankur+Taly%2C+Jingbo+Shang%2C+et+al%2E+2024b%2E+Speculative+rag%3A+Enhancing+retrieval+augm)
24. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671. [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chang+Zhou%2C+Chengpeng+Li%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+et+al%2E+2024%2E+Qwen2+technical+report%2E+arXiv+preprint+arXiv%3A2407%2E10671%2E)
25. Jiayi Yao, Hanchen Li, Yuhan Liu, Siddhant Ray, Yihua Cheng, Qizheng Zhang, Kuntai Du, Shan Lu, and Junchen Jiang. 2025. Cacheblend: Fast large language model serving for rag with cached knowledge fusion. In Proceedings of the Twentieth European Conference on Computer Systems, pages 94–109. [find](https://scholar.google.com/scholar?q=Jiayi+Yao%2C+Hanchen+Li%2C+Yuhan+Liu%2C+Siddhant+Ray%2C+Yihua+Cheng%2C+Qizheng+Zhang%2C+Kuntai+Du%2C+Shan+Lu%2C+and+Junchen+Jiang%2E+2025%2E+Cacheblend%3A+Fast+large+language+model+serving+for+rag+with+cached+knowledge+fus)
26. Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark Barrett, et al. 2024. H2o: Heavy-hitter oracle for efficient generative inference of large language models. Advances in Neural Information Processing Systems, 36. [find](https://scholar.google.com/scholar?q=Zhenyu+Zhang%2C+Ying+Sheng%2C+Tianyi+Zhou%2C+Tianlong+Chen%2C+Lianmin+Zheng%2C+Ruisi+Cai%2C+Zhao+Song%2C+Yuandong+Tian%2C+Christopher+R%C3%A9%2C+Clark+Barrett%2C+et+al%2E+2024%2E+H2o%3A+Heavy%2Dhitter+oracle+for+efficient+generative+)
27. Yun Zhu, Jia-Chen Gu, Caitlin Sikora, Ho Ko, Yinxiao Liu, Chu-Cheng Lin, Lei Shu, Liangchen Luo, Lei Meng, Bang Liu, et al. 2024. Accelerating inference of retrieval-augmented generation via sparse context selection. arXiv preprint arXiv:2405.16178. — [arXiv:2405.16178](https://arxiv.org/abs/2405.16178) [find](https://scholar.google.com/scholar?q=Yun+Zhu%2C+Jia%2DChen+Gu%2C+Caitlin+Sikora%2C+Ho+Ko%2C+Yinxiao+Liu%2C+Chu%2DCheng+Lin%2C+Lei+Shu%2C+Liangchen+Luo%2C+Lei+Meng%2C+Bang+Liu%2C+et+al%2E+2024%2E+Accelerating+inference+of+retrieval%2Daugmented+generation+via+sparse+co)
28. Nir Ratner, Yoav Levine, Yonatan Belinkov, Ori Ram, Inbal Magar, Omri Abend, Ehud Karpas, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2022. [Parallel context windows for large language models](https://md.research.elamur.ai/free/en/papers/ratner-2023-parallel-context-windows-large-language/). arXiv preprint arXiv:2212.10947.
29. Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. 2024. [Roformer: Enhanced transformer with rotary position embedding](https://md.research.elamur.ai/free/en/papers/su-2021-roformer-enhanced-transformer-rotary-position/). Neurocomputing, 568:127063.
30. Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. [Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions](https://md.research.elamur.ai/free/en/papers/trivedi-2022-interleaving-retrieval-chain-thought-reasoning/). arXiv preprint arXiv:2212.10509.
31. Sinong Wang, Belinda Z Li, Madian Khabsa, Han Fang, and Hao Ma. 2020. Linformer: Self-attention with linear complexity. arXiv preprint arXiv:2006.04768. [find](https://scholar.google.com/scholar?q=Sinong+Wang%2C+Belinda+Z+Li%2C+Madian+Khabsa%2C+Han+Fang%2C+and+Hao+Ma%2E+2020%2E+Linformer%3A+Self%2Dattention+with+linear+complexity%2E+arXiv+preprint+arXiv%3A2006%2E04768%2E)
32. Zheng Wang, Boxiao Jin, Zhongzhi Yu, and Minjia Zhang. 2024a. Model tells you where to merge: Adaptive kv cache merging for llms on long-context tasks. arXiv preprint arXiv:2407.08454. — [arXiv:2407.08454](https://arxiv.org/abs/2407.08454) [find](https://scholar.google.com/scholar?q=Zheng+Wang%2C+Boxiao+Jin%2C+Zhongzhi+Yu%2C+and+Minjia+Zhang%2E+2024a%2E+Model+tells+you+where+to+merge%3A+Adaptive+kv+cache+merging+for+llms+on+long%2Dcontext+tasks%2E+arXiv+preprint+arXiv%3A2407%2E08454%2E+%E2%80%94+%5BarXiv%3A2407%2E0)

*Updated 31 October 2026*