---
title: "500xCompressor — paper by Li et al. (2025)"
description: "The authors present 500xCompressor, a soft-prompt approach that compresses a context of roughly… 22 references, cited by 26 papers here."
authors: ["Zongqian Li", "Yixuan Su", "Nigel Collier"]
year: 2025
venue: "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2025.acl-long.1219/"
doi: "10.18653/v1/2025.acl-long.1219"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/li-2025-500xcompressor-generalized-prompt-compression-large/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/li-2025-500xcompressor-generalized-prompt-compression-large/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/3/) / 500xCompressor: Generalized Prompt Compression for Large Language Models

# 500xCompressor: Generalized Prompt Compression for Large Language Models

**Authors:** Zongqian Li, [Yixuan Su](https://md.research.elamur.ai/free/en/authors/yixuan-su/) and [Nigel Collier](https://md.research.elamur.ai/free/en/authors/nigel-collier/) · **Year:** 2025 · **Venue:** [Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2025](https://md.research.elamur.ai/free/en/venues/acl/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.acl-long.1219/) · [PDF](https://aclanthology.org/2025.acl-long.1219.pdf) · [DOI 10.18653/v1/2025.acl-long.1219](https://doi.org/10.18653/v1/2025.acl-long.1219) · [OpenAlex](https://openalex.org/W4412889950)

**Citations:** 6 citations (OpenAlex, 2026-10-14) · [26](https://md.research.elamur.ai/free/en/papers/li-2025-500xcompressor-generalized-prompt-compression-large/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Long prompts slow down LLM inference, increase cost, and run into context limits. Existing compression methods tend to have modest ratios (ICAE reaches about 15x at most), and their evaluations may be contaminated by texts the base model already saw during pretraining. The authors present 500xCompressor, a soft-prompt approach that compresses a context of roughly 500 tokens into as few as one special token. A LoRA-tuned copy of a frozen LLaMA-3-8B-Instruct serves as the encoder and writes the context into compression tokens. The KV values of those tokens are then passed to the unchanged base model, which can either reconstruct the text or answer questions about it. Training has two stages: pretraining on ArxivCorpus and fine-tuning on the new ArxivQA set. Testing uses only abstracts published after the model's knowledge cutoff, along with several cross-domain QA benchmarks. Even at extreme ratios, the method retains much of the uncompressed model's QA ability. The results also show that KV values preserve more information than output embeddings. This makes very compact, reusable prompts practical without any fine-tuning of the LLM itself.

## Key points
- The method adds only about 0.3% extra parameters and achieves compression ratios from 6x to 500x. At 500x, it preserves roughly 70–74% of F1 and 77–84% of Exact Match relative to full prompts.
- At 500x compression and output lengths of 100–400 tokens, computation falls by about 27–90% and KV-cache memory by about 55–83%. The largest gains occur when a compressed prompt is reused.
- ArxivCorpus splits arXiv abstracts by date: papers before July 2023 form the training set, and papers after January 2024 form the development and test sets. ArxivQA contains more than 250k extractive QA pairs generated with LLaMa-3-70b-Instruct.
- Evaluation covers ArxivQA, TriviaQA, RelationExtraction, NaturalQuestions, TextbookQA, and RACE, with LLMLingua-2 and ICAE as baselines. Text regeneration is scored with Rouge-2-F and BLEU, and QA with F1 and EM.
- Code is available at github.com/ZongqianLi/500xCompressor.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Cambridge](https://md.research.elamur.ai/free/en/universities/university-of-cambridge/) — University, Cambridge, United Kingdom · 256 works · 123,373 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [🪶 Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/) · 297 works

## Cite

Cite as: Zongqian Li, Yixuan Su and Nigel Collier. 2025. 500xCompressor: Generalized Prompt Compression for Large Language Models. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{li2025500xcompressor,
  title = {{500xCompressor: Generalized Prompt Compression for Large Language Models}},
  author = {Zongqian Li, Yixuan Su and Nigel Collier},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.1219},
  url = {https://doi.org/10.18653/v1/2025.acl-long.1219}
}
```

- [← previous Unveiling Privacy Risks in LLM Agent Memory](https://md.research.elamur.ai/free/en/papers/wang-2025-unveiling-privacy-risks-llm-agent/)
- [next → Attacking Vision-Language Computer Agents via Pop-ups](https://md.research.elamur.ai/free/en/papers/zhang-2025-attacking-vision-language-computer-agents/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/3/)

## References

22 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Xin Cheng, Xun Wang, Xingxing Zhang, Tao Ge, Si-Qing Chen, Furu Wei, Huishuai Zhang, and Dongyan Zhao. 2024. xrag: Extreme context compression for retrieval-augmented generation with one token. arXiv preprint arXiv:2405.13792. — [arXiv:2405.13792](https://arxiv.org/abs/2405.13792) [find](https://scholar.google.com/scholar?q=Xin+Cheng%2C+Xun+Wang%2C+Xingxing+Zhang%2C+Tao+Ge%2C+Si%2DQing+Chen%2C+Furu+Wei%2C+Huishuai+Zhang%2C+and+Dongyan+Zhao%2E+2024%2E+xrag%3A+Extreme+context+compression+for+retrieval%2Daugmented+generation+with+one+token%2E+arXiv+)
2. Alexis Chevalier, Alexander Wettig, Anirudh Ajith, and Danqi Chen. 2023. [Adapting language models to compress contexts](https://md.research.elamur.ai/free/en/papers/chevalier-2023-adapting-language-models-compress-contexts/). In The 2023 Conference on Empirical Methods in Natural Language Processing.
3. Adam Fisch, Alon Talmor, Robin Jia, Minjoon Seo, Eunsol Choi, and Danqi Chen. 2019. [find](https://scholar.google.com/scholar?q=Adam+Fisch%2C+Alon+Talmor%2C+Robin+Jia%2C+Minjoon+Seo%2C+Eunsol+Choi%2C+and+Danqi+Chen%2E+2019%2E)
4. [MRQA 2019 shared task: Evaluating generalization in reading comprehension](https://md.research.elamur.ai/free/en/papers/fisch-2019-mrqa-2019-shared-task-evaluating/). In Proceedings of 2nd Machine Reading for Reading Comprehension (MRQA) Workshop at EMNLP.
5. Tao Ge, Hu Jing, Lei Wang, Xun Wang, Si-Qing Chen, and Furu Wei. 2024. [In-context autoencoder for context compression in a large language model](https://md.research.elamur.ai/free/en/papers/ge-2023-context-autoencoder-context-compression-large/). In The Twelfth International Conference on Learning Representations.
6. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, ..., and Zhiyu Ma. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). Preprint, arXiv:2407.21783.
7. Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. [LoRA: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). In International Conference on Learning Representations.
8. Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. 2023a. [LLMLingua: Compressing prompts for accelerated inference of large language models](https://md.research.elamur.ai/free/en/papers/jiang-2023-llmlingua-compressing-prompts-accelerated-inference/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 13358–13376, Singapore. Association for Comp…
9. Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. 2023b. [Longllmlingua: Accelerating and enhancing llms in long context scenarios via prompt compression](https://md.research.elamur.ai/free/en/papers/jiang-2023-longllmlingua-accelerating-enhancing-llms-long/). arXiv preprint arXiv:2310.06839.
10. Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. [TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension](https://md.research.elamur.ai/free/en/papers/joshi-2017-triviaqa-large-scale-distantly-supervised/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1601–1611, Vancouver…
11. Aniruddha Kembhavi, Minjoon Seo, Dustin Schwenk, Jonghyun Choi, Ali Farhadi, and Hannaneh Hajishirzi. 2017. Are you smarter than a sixth grader? textbook question answering for multimodal machine comprehension. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5376–538… [find](https://scholar.google.com/scholar?q=Aniruddha+Kembhavi%2C+Minjoon+Seo%2C+Dustin+Schwenk%2C+Jonghyun+Choi%2C+Ali+Farhadi%2C+and+Hannaneh+Hajishirzi%2E+2017%2E+Are+you+smarter+than+a+sixth+grader%3F+textbook+question+answering+for+multimodal+machine+comp)
12. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. [Natural…](https://md.research.elamur.ai/free/en/papers/kwiatkowski-2019-natural-questions-benchmark-question-answering/)
13. Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. 2017. [RACE: Large-scale ReAding comprehension dataset from examinations](https://md.research.elamur.ai/free/en/papers/lai-2017-race-large-scale-reading-comprehension/). In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 785– 794, Copenhagen, Denmark. Association for Computational Lin…
14. Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. [The power of scale for parameter-efficient prompt tuning](https://md.research.elamur.ai/free/en/papers/lester-2021-power-scale-parameter-efficient-prompt/). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3045–3059, Online and Punta Cana, Dominican Republic. Association for Computational Linguist…
15. Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. 2017. [Zero-shot relation extraction via reading comprehension](https://md.research.elamur.ai/free/en/papers/levy-2017-zero-shot-relation-extraction-reading/). In Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pages 333–342, Vancouver, Canada. Association for Computational Linguistics.
16. Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4582–… [find](https://scholar.google.com/scholar?q=Xiang+Lisa+Li+and+Percy+Liang%2E+2021%2E+Prefix%2Dtuning%3A+Optimizing+continuous+prompts+for+generation%2E+In+Proceedings+of+the+59th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+and+the+11t)
17. Yucheng Li, Bo Dong, Frank Guerin, and Chenghua Lin. 2023. [Compressing context to enhance inference efficiency of large language models](https://md.research.elamur.ai/free/en/papers/li-2023-compressing-context-enhance-inference-efficiency/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 6342–6353, Singapore. Association for Computational Linguistics.
18. Jesse Mu, Xiang Lisa Li, and Noah Goodman. 2024. [Learning to compress prompts with gist tokens](https://md.research.elamur.ai/free/en/papers/mu-2023-learning-compress-prompts-gist-tokens/). In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Red Hook, NY, USA. Curran Associates Inc.
19. Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin, Victor Rühle, Yuqing Yang, Chin-Yew Lin, H. Vicky Zhao, Lili Qiu, and Dongmei Zhang. 2024. [LLMLingua-2: Data distillation for efficient and faithful task-agnostic prompt compression](https://md.research.elamur.ai/free/en/papers/pan-2024-llmlingua-2-data-distillation-efficient/). In Findings of the Associat…
20. David Rau, Shuai Wang, Hervé Déjean, and Stéphane Clinchant. 2024. Context embeddings for efficient answer generation in rag. arXiv preprint arXiv:2407.09252. — [arXiv:2407.09252](https://arxiv.org/abs/2407.09252) [find](https://scholar.google.com/scholar?q=David+Rau%2C+Shuai+Wang%2C+Herv%C3%A9+D%C3%A9jean%2C+and+St%C3%A9phane+Clinchant%2E+2024%2E+Context+embeddings+for+efficient+answer+generation+in+rag%2E+arXiv+preprint+arXiv%3A2407%2E09252%2E+%E2%80%94+%5BarXiv%3A2407%2E09252%5D%28https%3A%2F%2Farxiv%2Eorg%2Fab)
21. Siyu Ren, Qi Jia, and Kenny Q. Zhu. 2023. Context compression for auto-regressive transformers with sentinel tokens. In The 2023 Conference on Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Siyu+Ren%2C+Qi+Jia%2C+and+Kenny+Q%2E+Zhu%2E+2023%2E+Context+compression+for+auto%2Dregressive+transformers+with+sentinel+tokens%2E+In+The+2023+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2E)
22. David Wingate, Mohammad Shoeybi, and Taylor Sorensen. 2022. Prompt compression and contrastive conditioning for controllability and toxicity reduction in language models. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 5621–5634, Abu Dhabi, United Arab Emirates. Assoc… [find](https://scholar.google.com/scholar?q=David+Wingate%2C+Mohammad+Shoeybi%2C+and+Taylor+Sorensen%2E+2022%2E+Prompt+compression+and+contrastive+conditioning+for+controllability+and+toxicity+reduction+in+language+models%2E+In+Findings+of+the+Associatio)

*Updated 2 November 2026*