---
title: "EQUIP — paper by Ramachandran et al. (2026)"
description: "KV-cache token pruning methods such as StreamingLLM and Heavy Hitters (H2O) keep the cache small, but each eviction… 40 references."
authors: ["Arun Ramachandran", "R Govindarajan", "Murali Annavaram", "Prakash Raghavendra"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.1210/"
doi: "10.18653/v1/2026.acl-long.1210"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ramachandran-2026-equip-equivariant-preserving-place-updates/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ramachandran-2026-equip-equivariant-preserving-place-updates/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/29/) / EQUIP: EQUivariant preserving In-Place updates for Efficient Token Pruning

# EQUIP: EQUivariant preserving In-Place updates for Efficient Token Pruning

**Authors:** Arun Ramachandran, R Govindarajan, [Murali Annavaram](https://md.research.elamur.ai/free/en/authors/murali-annavaram/) and Prakash Raghavendra · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.1210/) · [PDF](https://aclanthology.org/2026.acl-long.1210.pdf) · [DOI 10.18653/v1/2026.acl-long.1210](https://doi.org/10.18653/v1/2026.acl-long.1210) · [OpenAlex](https://openalex.org/W7166798098)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary

KV-cache token pruning methods such as StreamingLLM and Heavy Hitters (H2O) keep the cache small, but each eviction leaves gaps. Closing those gaps means shifting and copying whole K/V tensors. Evictions also change token positions, which forces RoPE to be recomputed. EQUIP, from AMD, IISc and USC, removes both costs. It relies on the fact that the attention pipeline (batched matmul, softmax, batched matmul) gives the same result regardless of where a token sits in the cache. Because of this, a new token can be written directly into the slot of the evicted one, with no shifting. For RoPE, EQUIP stores the rotated-key tensors, updates them in the same in-place way, and uses a cheap position reindexing scheme.

The authors tested EQUIP in Hugging Face and vLLM (including Paged Attention) on Intel CPUs and AMD Instinct MI210 GPUs. It gives clear end-to-end speedups and produces exactly the same outputs as the pruning method it accelerates. It also makes token pruning practical inside custom kernels like Paged Attention, which normally struggle with a non-contiguous cache.

## Key points

- **Speedups over existing pruning methods (geometric mean):**
  - Over StreamingLLM: 1.62× on CPU and 1.47× on GPU.
  - Over H2O: 3.45× on CPU and 1.86× on GPU.
- **Speedups with Paged Attention:** combined with vLLM's Paged Attention, EQUIP is 4.18× faster on CPU and 2.61× faster on GPU than autoregressive baselines.
- **Accuracy is unchanged:**
  - Perplexity on Llama 2-7B matches StreamingLLM exactly.
  - ARC-Easy accuracy in a streaming setup is 74.90% for both.
  - Layer-level deviations stay below 10⁻⁹ at SDPA and below 10⁻⁵ at RoPE.
- **Multi-token eviction and speculative decoding:** EQUIP handles both. With speculative decoding (γ=8) it is 1.47×–1.67× faster than StreamingLLM.
- **Gains depend on batch size and sequence length:**
  - At batch size 1, gains are small (about 1.04–1.07×), because MLP computation dominates the runtime.
  - Gains grow with batch size and sequence length, reaching 2.45× at BS=8 with 4k-token sequences on Llama 2-7B.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Southern California](https://md.research.elamur.ai/free/en/universities/university-of-southern-california/) — University, Los Angeles, United States · 162 works · 35,836 citations
- [Indian Institute of Science Bangalore](https://md.research.elamur.ai/free/en/universities/indian-institute-of-science-bangalore/) — University, Bengaluru, India · 10 works · 1,142 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (1): [🪶 Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/) · 242 works

## Cite

Cite as: Arun Ramachandran, R Govindarajan, Murali Annavaram and Prakash Raghavendra. 2026. EQUIP: EQUivariant preserving In-Place updates for Efficient Token Pruning. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{ramachandran2026equip,
  title = {{EQUIP: EQUivariant preserving In-Place updates for Efficient Token Pruning}},
  author = {Arun Ramachandran, R Govindarajan, Murali Annavaram and Prakash Raghavendra},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1210},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1210}
}
```

- [← previous EpiGator: An Event-based Surveillance System for Infectious Disease Outbreaks](https://md.research.elamur.ai/free/en/papers/wu-2026-epigator-event-based-surveillance-system/)
- [next → Eraserhead at OSACT7 Shared Task: ASR Consistency Filtering and Speaker-Adaptive Post-Processing for Arabic Speech Diacritization](https://md.research.elamur.ai/free/en/papers/horaira-2026-eraserhead-osact7-shared-task-asr/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/29/)

## References

40 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Yuan Feng, Junlin Lv, Yukun Cao, Xike Xie, and S. Kevin Zhou. 2025. Identify critical kv cache in llm inference from an output perturbation perspective. Preprint, arXiv:2502.03805. Submitted on 6 Feb 2025. — [arXiv:2502.03805](https://arxiv.org/abs/2502.03805) [find](https://scholar.google.com/scholar?q=Yuan+Feng%2C+Junlin+Lv%2C+Yukun+Cao%2C+Xike+Xie%2C+and+S%2E+Kevin+Zhou%2E+2025%2E+Identify+critical+kv+cache+in+llm+inference+from+an+output+perturbation+perspective%2E+Preprint%2C+arXiv%3A2502%2E03805%2E+Submitted+on+6+Feb+)
2. Suyu Ge, Yunan Zhang, Liyuan Liu, Minjia Zhang, Jiawei Han, and Jianfeng Gao. 2024. [Model tells you what to discard: Adaptive KV cache compression for LLMs](https://md.research.elamur.ai/free/en/papers/ge-2023-model-tells-discard-adaptive-kv/). In International Conference on Learning Representations (ICLR).
3. Gemini Team and Google. 2023. [Gemini: A family of highly capable multimodal models](https://md.research.elamur.ai/free/en/papers/gemini-team-2023-gemini-family-highly-capable-multimodal/).
4. Nathan Godey, Alessio Devoto, Yu Zhao, Simone Scardapane, Pasquale Minervini, Éric de la Clergerie, and Benoît Sagot. 2025. Q-filters: Leveraging qk geometry for efficient kv cache compression. Preprint, arXiv:2503.02812. Submitted on 4 Mar 2025. — [arXiv:2503.02812](https://arxiv.org/abs/2503.02812) [find](https://scholar.google.com/scholar?q=Nathan+Godey%2C+Alessio+Devoto%2C+Yu+Zhao%2C+Simone+Scardapane%2C+Pasquale+Minervini%2C+%C3%89ric+de+la+Clergerie%2C+and+Beno%C3%AEt+Sagot%2E+2025%2E+Q%2Dfilters%3A+Leveraging+qk+geometry+for+efficient+kv+cache+compression%2E+Prepri)
5. HuggingFace. 2024. Transformers: State-of-the-art natural language processing for pytorch, tensorflow, and jax. Accessed: 2024-07-06. [find](https://scholar.google.com/scholar?q=HuggingFace%2E+2024%2E+Transformers%3A+State%2Dof%2Dthe%2Dart+natural+language+processing+for+pytorch%2C+tensorflow%2C+and+jax%2E+Accessed%3A+2024%2D07%2D06%2E)
6. A. H. Hunter, Chris Kennelly, Paul Turner, Darryl Gove, Tipp Moseley, and Parthasarathy Ranganathan. 2021. Beyond malloc efficiency to fleet efficiency: a hugepage-aware memory allocator. In 15th USENIX Symposium on Operating Systems Design and Implementation (OSDI 21), pages 257–273. [find](https://scholar.google.com/scholar?q=A%2E+H%2E+Hunter%2C+Chris+Kennelly%2C+Paul+Turner%2C+Darryl+Gove%2C+Tipp+Moseley%2C+and+Parthasarathy+Ranganathan%2E+2021%2E+Beyond+malloc+efficiency+to+fleet+efficiency%3A+a+hugepage%2Daware+memory+allocator%2E+In+15th+USEN)
7. Intel. 2024a. The AI PC opportunity. Intel White Paper. Originally available at [https://www.intel.com/content/dam/www/central-libraries/](https://www.intel.com/content/dam/www/central-libraries/) us/en/documents/2024-01/ the-ai-pc-opportunity-white-paper.pdf; accessed January 2024. [↗](https://www.intel.com/content/dam/www/central-libraries/)
8. Intel. 2024b. Mkl: Improved small matrix performance using just-in-time (jit) code. Accessed: 2024-08-01. [find](https://scholar.google.com/scholar?q=Intel%2E+2024b%2E+Mkl%3A+Improved+small+matrix+performance+using+just%2Din%2Dtime+%28jit%29+code%2E+Accessed%3A+2024%2D08%2D01%2E)
9. Intel Corporation. 2024. oneDNN API Documentation. Accessed: 2024-10-31. [find](https://scholar.google.com/scholar?q=Intel+Corporation%2E+2024%2E+oneDNN+API+Documentation%2E+Accessed%3A+2024%2D10%2D31%2E)
10. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… [find](https://scholar.google.com/scholar?q=Albert+Q%2E+Jiang%2C+Alexandre+Sablayrolles%2C+Arthur+Mensch%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guillaume+Lample%2C+Lucile+Saulnier%2C+L%C3%A9lio+Renard+Lav)
11. Risi Kondor and Shubhendu Trivedi. 2018. On the generalization of equivariance and convolution in neural networks to the action of compact groups. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 2747–2755. [find](https://scholar.google.com/scholar?q=Risi+Kondor+and+Shubhendu+Trivedi%2E+2018%2E+On+the+generalization+of+equivariance+and+convolution+in+neural+networks+to+the+action+of+compact+groups%2E+In+Proceedings+of+the+35th+International+Conference+o)
12. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. [Efficient memory management for large language model serving with PagedAttention](https://md.research.elamur.ai/free/en/papers/kwon-2023-efficient-memory-management-large-language/). In Proceedings of the 29th ACM Symposium on Operating Systems Principles (SOSP). ACM.
13. Yaniv Leviathan, Matan Kalman, and Yossi Matias. 2023. Fast inference from transformers via speculative decoding. In International Conference on Machine Learning (ICML), pages 19274–19286. [find](https://scholar.google.com/scholar?q=Yaniv+Leviathan%2C+Matan+Kalman%2C+and+Yossi+Matias%2E+2023%2E+Fast+inference+from+transformers+via+speculative+decoding%2E+In+International+Conference+on+Machine+Learning+%28ICML%29%2C+pages+19274%E2%80%9319286%2E)
14. Yuhong Li, Yingbing Huang, Bowen Yang, Bharat Venkitesh, Acyr Locatelli, Hanchen Ye, Tianle Cai, Patrick Lewis, and Deming Chen. 2024a. [SnapKV: LLM knows what you are looking for before generation](https://md.research.elamur.ai/free/en/papers/li-2024-snapkv-llm-knows-looking-before/). In Advances in Neural Information Processing Systems (NeurIPS).
15. Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. 2024b. [EAGLE-2: Faster inference of language models with dynamic draft trees](https://md.research.elamur.ai/free/en/papers/li-2024-eagle-2-faster-inference-language/). In Empirical Methods in Natural Language Processing (EMNLP).
16. Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. 2024c. [EAGLE: Speculative sampling requires rethinking feature uncertainty](https://md.research.elamur.ai/free/en/papers/li-2024-eagle-speculative-sampling-requires-rethinking/). In International Conference on Machine Learning (ICML).
17. Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. 2025. EAGLE-3: Scaling up inference acceleration of large language models via training-time test. [find](https://scholar.google.com/scholar?q=Yuhui+Li%2C+Fangyun+Wei%2C+Chao+Zhang%2C+and+Hongyang+Zhang%2E+2025%2E+EAGLE%2D3%3A+Scaling+up+inference+acceleration+of+large+language+models+via+training%2Dtime+test%2E)
18. Manlai Liang, JiaMing Zhang, Xiong Li, and Jinlong Li. 2025. LagKV: Lag-relative information of the KV cache tells which tokens are important. Preprint, arXiv:2504.04704. — [arXiv:2504.04704](https://arxiv.org/abs/2504.04704) [find](https://scholar.google.com/scholar?q=Manlai+Liang%2C+JiaMing+Zhang%2C+Xiong+Li%2C+and+Jinlong+Li%2E+2025%2E+LagKV%3A+Lag%2Drelative+information+of+the+KV+cache+tells+which+tokens+are+important%2E+Preprint%2C+arXiv%3A2504%2E04704%2E+%E2%80%94+%5BarXiv%3A2504%2E04704%5D%28https%3A%2F%2F)
19. Linux Kernel Community. 2024. Transparent hugepages – Linux kernel documentation. [https://docs.kernel.org/admin-guide/mm/transhuge.html](https://docs.kernel.org/admin-guide/mm/transhuge.html). Accessed: 2024-07-21. [↗](https://docs.kernel.org/admin-guide/mm/transhuge.html)
20. Zirui Liu, Jiayi Yuan, Hongye Jin, Shaochen Zhong, Zhaozhuo Xu, Vladimir Braverman, Beidi Chen, and Xia Hu. 2024. [KIVI: A tuning-free asymmetric 2bit quantization for KV cache](https://md.research.elamur.ai/free/en/papers/liu-2024-kivi-tuning-free-asymmetric-2bit/). In International Conference on Machine Learning (ICML). llama. 2023. Llama-7b: A large language model, hugging face and hug…
21. Xinyin Ma, Gongfan Fang, and Xinchao Wang. 2023. [Llm-pruner: On the structural pruning of large language models](https://md.research.elamur.ai/free/en/papers/ma-2023-llm-pruner-structural-pruning-large/). In Advances in Neural Information Processing Systems.
22. Meta. 2024. Introducing meta llama 3: The most capable openly available llm to date. [find](https://scholar.google.com/scholar?q=Meta%2E+2024%2E+Introducing+meta+llama+3%3A+The+most+capable+openly+available+llm+to+date%2E)
23. Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. Can a suit of armor conduct electricity? A new dataset for open book question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP). OpenBookQA dataset: [https://huggingface.co…](https://huggingface.co…) [↗](https://huggingface.co…)
24. MLCommons. 2024. Inference data center benchmarks. Accessed: 2024-08-01. [find](https://scholar.google.com/scholar?q=MLCommons%2E+2024%2E+Inference+data+center+benchmarks%2E+Accessed%3A+2024%2D08%2D01%2E)
25. NVIDIA. 2022. Fastertransformer. [https://Ac-github.com/NVIDIA/FasterTransformer](https://Ac-github.com/NVIDIA/FasterTransformer). cessed: 2024-08-01. [↗](https://Ac-github.com/NVIDIA/FasterTransformer)
26. NVIDIA. 2023. Tensorrt-llm. Accessed: 2024-08-02. [find](https://scholar.google.com/scholar?q=NVIDIA%2E+2023%2E+Tensorrt%2Dllm%2E+Accessed%3A+2024%2D08%2D02%2E)
27. NVIDIA. 2025a. Chunkkvpress. [https://github.com/NVIDIA/kvpress/blob/main/](https://github.com/NVIDIA/kvpress/blob/main/) kvpress/presses/chunkkv_press.py. [↗](https://github.com/NVIDIA/kvpress/blob/main/)
28. NVIDIA. 2025b. Chunkpress. [https://github.com/NVIDIA/kvpress/blob/main/](https://github.com/NVIDIA/kvpress/blob/main/) kvpress/presses/chunk_press.py. [↗](https://github.com/NVIDIA/kvpress/blob/main/)
29. NVIDIA. 2025c. Expectedattention. [https://github.com/NVIDIA/kvpress/blob/main/kvpress/](https://github.com/NVIDIA/kvpress/blob/main/kvpress/) presses/expected_attention_press.py. [↗](https://github.com/NVIDIA/kvpress/blob/main/kvpress/)
30. NVIDIA. 2025d. Finchpress. [https://github.com/](https://github.com/) NVIDIA/kvpress/blob/main/kvpress/presses/ finch_press.py. [↗](https://github.com/)
31. NVIDIA. 2025e. Keyrerotationpress. [https://github.com/NVIDIA/kvpress/blob/main/](https://github.com/NVIDIA/kvpress/blob/main/) kvpress/presses/key_rerotation_press.py. [↗](https://github.com/NVIDIA/kvpress/blob/main/)
32. NVIDIA. 2025f. Observedattention. [https://github.com/NVIDIA/kvpress/blob/main/kvpress/](https://github.com/NVIDIA/kvpress/blob/main/kvpress/) presses/observed_attention_press.py. [↗](https://github.com/NVIDIA/kvpress/blob/main/kvpress/)
33. NVIDIA. 2025g. Randompress. [https://github.com/NVIDIA/kvpress/blob/main/](https://github.com/NVIDIA/kvpress/blob/main/) kvpress/presses/random_press.py. [↗](https://github.com/NVIDIA/kvpress/blob/main/)
34. NVIDIA. 2025h. Scorerpress. [https://github.com/NVIDIA/kvpress/blob/main/](https://github.com/NVIDIA/kvpress/blob/main/) kvpress/presses/scorer_press.py. [↗](https://github.com/NVIDIA/kvpress/blob/main/)
35. NVIDIA. 2025i. Tova. [https://github.com/](https://github.com/) NVIDIA/kvpress/blob/main/kvpress/presses/ tova_press.py. [↗](https://github.com/)
36. S. Park, J. Choi, S. Lee, and U. Kang. 2024. A comprehensive survey of compression algorithms for language models. arXiv preprint. [find](https://scholar.google.com/scholar?q=S%2E+Park%2C+J%2E+Choi%2C+S%2E+Lee%2C+and+U%2E+Kang%2E+2024%2E+A+comprehensive+survey+of+compression+algorithms+for+language+models%2E+arXiv+preprint%2E)
37. Aaron Pham, Chaoyu Yang, Sean Sheng, Shenyang Zhao, Sauyon Lee, Bo Jiang, Fog Dong, Xipeng Guan, and Frost Ming. 2023. Openllm: Operating llms in production. [find](https://scholar.google.com/scholar?q=Aaron+Pham%2C+Chaoyu+Yang%2C+Sean+Sheng%2C+Shenyang+Zhao%2C+Sauyon+Lee%2C+Bo+Jiang%2C+Fog+Dong%2C+Xipeng+Guan%2C+and+Frost+Ming%2E+2023%2E+Openllm%3A+Operating+llms+in+production%2E)
38. Ramya Prabhu, Ajay Nayak, Jayashree Mohan, Ramachandran Ramjee, and Ashish Panwar. 2025. vAttention: Dynamic memory management for serving LLMs without PagedAttention. In Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (AS… [find](https://scholar.google.com/scholar?q=Ramya+Prabhu%2C+Ajay+Nayak%2C+Jayashree+Mohan%2C+Ramachandran+Ramjee%2C+and+Ashish+Panwar%2E+2025%2E+vAttention%3A+Dynamic+memory+management+for+serving+LLMs+without+PagedAttention%2E+In+Proceedings+of+the+30th+ACM+I)
39. PyTorch. 2024. Pytorch serve. Accessed: 2024-08-01. [find](https://scholar.google.com/scholar?q=PyTorch%2E+2024%2E+Pytorch+serve%2E+Accessed%3A+2024%2D08%2D01%2E)
40. Ranajoy Sadhukhan, Jian Chen, Zhuoming Chen, Vashisth Tiwari, Ruihang Lai, Jinyuan Shi, Ian En-Hsu Yen, Avner May, Tianqi Chen, and Beidi Chen. 2025. MagicDec: Breaking the latency-throughput tradeoff for long context generation with speculative decoding. In International Conference on Learning Repr… [find](https://scholar.google.com/scholar?q=Ranajoy+Sadhukhan%2C+Jian+Chen%2C+Zhuoming+Chen%2C+Vashisth+Tiwari%2C+Ruihang+Lai%2C+Jinyuan+Shi%2C+Ian+En%2DHsu+Yen%2C+Avner+May%2C+Tianqi+Chen%2C+and+Beidi+Chen%2E+2025%2E+MagicDec%3A+Breaking+the+latency%2Dthroughput+tradeoff)

*Updated 1 November 2026*