---
title: "大语言模型推理中的存储优化技术综述 — 葛旭冉 2025"
description: "This survey, published in the Journal of Computer Research and Development (2025), examines why serving large language models is increasingly limited by…"
authors: ["葛旭冉", "欧洋", "王博", "赵宇", "吴利舟", "王子聪", "陈志广", "肖侬"]
year: 2025
venue: "计算机研究与发展 (Journal of Computer Research and Development)"
source: "https://crad.ict.ac.cn/article/doi/10.7544/issn1000-1239.202440628"
doi: "10.7544/issn1000-1239.202440628"
licence: "Free access (crad.ict.ac.cn), no open licence stated — readonly; © 计算机研究与发展"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/ge-2025-storage-optimization-techniques/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/ge-2025-storage-optimization-techniques/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2025](https://md.research.elamur.ai/free/papers/2025/page/3/) / 大语言模型推理中的存储优化技术综述

# 大语言模型推理中的存储优化技术综述

**Authors:** 葛旭冉, 欧洋, 王博, 赵宇, 吴利舟, 王子聪, 陈志广, 肖侬 · **Year:** 2025 · **Venue:** 计算机研究与发展 (Journal of Computer Research and Development) · **Licence:** Free access (crad.ict.ac.cn), no open licence stated — readonly; © 计算机研究与发展

**Links:** [Publisher's page](https://doi.org/10.7544/issn1000-1239.202440628) · [PDF](https://crad.ict.ac.cn/cn/article/pdf/preview/10.7544/issn1000-1239.202440628.pdf) · [DOI 10.7544/issn1000-1239.202440628](https://doi.org/10.7544/issn1000-1239.202440628)

## Summary
This survey, published in the Journal of Computer Research and Development (2025), examines why serving large language models is increasingly limited by storage rather than compute. Model parameters keep growing, and users want long-context generation. Both the weights and the key-value (KV) cache compete for scarce GPU memory, and distributed deployments often waste cluster resources and recover poorly from failures. The authors first explain the prefill and decode stages of Transformer inference: prefill is compute-bound, while decoding is limited by memory bandwidth. They then sort the literature into three families:
- **Memory optimization:** KV cache management, KV cache compression, and attention-operator redesign.
- **Heterogeneous storage:** GPU memory, CPU RAM and SSD combined through tensor placement, asynchronous transfers that overlap compute, and smart allocation and prefetching.
- **Distributed storage:** batching, multi-level scheduling, and redundant replication across machines.

The survey ends with open problems and future directions. It matters as a structured map for anyone who needs to deploy LLMs on limited hardware.

## Key points
- **Worked memory example:** LLaMA2-70B in FP16 needs about 140 GB for its weights alone. With batch size 8, sequence length 4,096, hidden size 8,192 and 80 layers, the KV cache adds roughly 80 GB. A single NVIDIA A100 offers at most 80 GB.
- **KV cache size formula:** the cache scales as 2 × batch × sequence length × hidden size × layers × precision.
- **ORCA's fragmentation problem:** ORCA reserves space for the maximum token count. In the paper's example, 2,048 token slots are reserved but only 512 are used, leaving 1,536 slots as internal fragmentation.
- **vLLM and Infinite-LLM:** vLLM's PagedAttention stores the KV cache in non-contiguous, fixed-size blocks, much like OS paging, so waste per request is limited to one block. Infinite-LLM extends this idea with DistAttention: it uses rBlocks, a per-instance rManager, and a global gManager that lends memory between instances.
- **Other systems cited:** Splitwise, DéjàVu (fault-tolerant KV-cache streaming) and ServerlessLLM. The keywords also flag fault recovery and serverless inference as areas the survey covers.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/) · [Natural language processing](https://md.research.elamur.ai/free/shelves/nlp/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@article{2025,
  title = {{大语言模型推理中的存储优化技术综述}},
  author = {葛旭冉 and 欧洋 and 王博 and 赵宇 and 吴利舟 and 王子聪 and 陈志广 and 肖侬},
  journal = {计算机研究与发展 (Journal of Computer Research and Development)},
  year = {2025},
  doi = {10.7544/issn1000-1239.202440628},
  url = {https://doi.org/10.7544/issn1000-1239.202440628}
}
```

- [← previous 大语言模型微调方法研究综述](https://md.research.elamur.ai/free/papers/wu-2025-fine-tuning-methods/)
- [next → 大语言模型的中文多轮指令遵循能力评测研究](https://md.research.elamur.ai/free/papers/zhu-2025-evaluation-chinese-multiturn/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/papers/2025/page/3/)