---
title: "FLAT-LLM — paper by Tian et al. (2026) — summary and ideas"
description: "Large language models are costly to run on limited hardware. 27 references. Among them: 2."
authors: ["Jiayi Tian", "Ryan Solgi", "Jinming Lu", "Yifan Yang", "Hai Li", "Zheng Zhang"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: EACL 2026"
source: "https://aclanthology.org/2026.findings-eacl.156/"
doi: "10.18653/v1/2026.findings-eacl.156"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/tian-2026-flat-llm-fine-grained-low/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/tian-2026-flat-llm-fine-grained-low/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/7/) / FLAT-LLM: Fine-grained Low-rank Activation Space Transformation for Large Language Model Compression

# FLAT-LLM: Fine-grained Low-rank Activation Space Transformation for Large Language Model Compression

**Authors:** Jiayi Tian, Ryan Solgi, Jinming Lu, [Yifan Yang](https://md.research.elamur.ai/free/en/authors/yifan-yang/), Hai Li and [Zheng Zhang](https://md.research.elamur.ai/free/en/authors/zheng-zhang/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: EACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-eacl.156/) · [PDF](https://aclanthology.org/2026.findings-eacl.156.pdf) · [DOI 10.18653/v1/2026.findings-eacl.156](https://doi.org/10.18653/v1/2026.findings-eacl.156) · [OpenAlex](https://openalex.org/W7140123693)

**Citations:** 2 citations (OpenAlex, 2026-10-14)

## Summary
Large language models are costly to run on limited hardware. Low-rank compression helps, but SVD-based variants often lose accuracy, need recovery fine-tuning, and split layers into two smaller matrix multiplications that run poorly on GPUs. SliceGPT avoids SVD but adds adapters on residual paths, which costs memory. FLAT-LLM is a training-free structural compression method that works in the activation space. For each attention head, it runs PCA on the value-layer outputs using calibration samples and keeps only the leading eigenvectors. Those truncated bases are then folded into the value and output projection weights, which shrinks both with no extra modules. Query and key projections are compressed separately with the same per-head procedure. A greedy, tuning-free scheme called Importance-Preserving Rank Selection (IPRS) shifts rank budgets between decoder layers. Calibration takes only minutes. Across 5 models (LLaMA and Mistral families) and 11 datasets, FLAT-LLM beats structural pruning baselines and runs faster than low-rank competitors. This makes it a practical option for deploying compressed models.

## Key points
- The core idea is head-wise PCA on value outputs, with eigenvectors absorbed jointly into the value and output weights. It adds no memory overhead, the compression ratio equals the retained rank ratio, and it is compatible with GQA.
- IPRS completes in seconds, is over 100× faster than the hypernetwork-based Adaptive SVD, and outperforms MoDeGPT's allocation. On LLaMA-2 13B at 40% compression, it beats uniform ranks by 8.0% (WikiText-2 calibration) and 5.4% (Alpaca calibration).
- On LLaMA-2 7B, throughput is at least 1.50× that of the original model at every ratio tested, and 1.68× even at 10% compression. Against SliceGPT and SVD-LLM, throughput is up to 1.63× and 1.93× higher, with 19% and 20% less memory.
- Combining 30% FLAT-LLM compression with GPTQ 3/4-bit gives lower WikiText-2 perplexity than GPTQ 2/3-bit alone at a similar or smaller size. Overall compression reaches about 5–8×.
- The authors note limitations: real speedups still need CUDA kernel–level optimization, and the rank allocation is not hardware-aware. Code is available at github.com/TTTTTTris/FLAT-LLM.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of California, Santa Barbara](https://md.research.elamur.ai/free/en/universities/university-of-california-santa-barbara/) — University, Santa Barbara, United States · 57 works · 9,752 citations
- [Intel](https://md.research.elamur.ai/free/en/companies/intel/) — Company, Santa Clara, United States · 15 works · 41,257 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🪶 Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/) · 242 works

💡 Concepts (1): [💡 Latent semantic indexing](https://md.research.elamur.ai/free/en/concepts/latent-semantic-indexing/) · 29 works

## Cite

Cite as: Jiayi Tian, Ryan Solgi, Jinming Lu, Yifan Yang, Hai Li and Zheng Zhang. 2026. FLAT-LLM: Fine-grained Low-rank Activation Space Transformation for Large Language Model Compression. Findings of the Association for Computational Linguistics: EACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{tian2026flat,
  title = {{FLAT-LLM: Fine-grained Low-rank Activation Space Transformation for Large Language Model Compression}},
  author = {Jiayi Tian, Ryan Solgi, Jinming Lu, Yifan Yang, Hai Li and Zheng Zhang},
  booktitle = {Findings of the Association for Computational Linguistics: EACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-eacl.156},
  url = {https://doi.org/10.18653/v1/2026.findings-eacl.156}
}
```

- [← previous First Shared Task on UMR Parsing](https://md.research.elamur.ai/free/en/papers/stepanek-2026-first-shared-task-umr-parsing/)
- [next → Focus-dLLM: Accelerating Long-Context Diffusion LLM Inference via Confidence-Guided Context Focusing](https://md.research.elamur.ai/free/en/papers/long-2026-focus-dllm-accelerating-long-context/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/7/)

## References

27 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebron, and Sumit Sanghai. 2023. [Gqa: Training generalized multi-query transformer models from multi-head checkpoints](https://md.research.elamur.ai/free/en/papers/ainslie-2023-gqa-training-generalized-multi-query/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 4895– 4…
2. Yongqi An, Xu Zhao, Tao Yu, Ming Tang, and Jinqiao Wang. 2024. Fluctuation-based adaptive structured pruning for large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 10865–10873. [find](https://scholar.google.com/scholar?q=Yongqi+An%2C+Xu+Zhao%2C+Tao+Yu%2C+Ming+Tang%2C+and+Jinqiao+Wang%2E+2024%2E+Fluctuation%2Dbased+adaptive+structured+pruning+for+large+language+models%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence)
3. Saleh Ashkboos, Maximilian L Croci, Marcelo Gennari do Nascimento, Torsten Hoefler, and James Hensman. 2024. Slicegpt: Compress large language models by deleting rows and columns. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Saleh+Ashkboos%2C+Maximilian+L+Croci%2C+Marcelo+Gennari+do+Nascimento%2C+Torsten+Hoefler%2C+and+James+Hensman%2E+2024%2E+Slicegpt%3A+Compress+large+language+models+by+deleting+rows+and+columns%2E+In+The+Twelfth+Inter)
4. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, and 1 others. 2023. [Qwen technical report](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-technical-report/). arXiv preprint arXiv:2309.16609.
5. Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, and 5 others.… [find](https://scholar.google.com/scholar?q=Leo+Gao%2C+Jonathan+Tow%2C+Baber+Abbasi%2C+Stella+Biderman%2C+Sid+Black%2C+Anthony+DiPofi%2C+Charles+Foster%2C+Laurence+Golding%2C+Jeffrey+Hsu%2C+Alain+Le+Noac%E2%80%99h%2C+Haonan+Li%2C+Kyle+McDonell%2C+Niklas+Muennighoff%2C+Chris+Oci)
6. Shangqian Gao, Ting Hua, Yen-Chang Hsu, Yilin Shen, and Hongxia Jin. 2024b. Adaptive rank selections for low-rank approximation of language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volum… [find](https://scholar.google.com/scholar?q=Shangqian+Gao%2C+Ting+Hua%2C+Yen%2DChang+Hsu%2C+Yilin+Shen%2C+and+Hongxia+Jin%2E+2024b%2E+Adaptive+rank+selections+for+low%2Drank+approximation+of+language+models%2E+In+Proceedings+of+the+2024+Conference+of+the+North+A)
7. Wei Huang, Yangdong Liu, Haotong Qin, Ying Li, Shiming Zhang, Xianglong Liu, Michele Magno, and Xiaojuan Qi. 2024. Billm: pushing the limit of post-training quantization for llms. In Proceedings of the 41st International Conference on Machine Learning, pages 20023–20042. [find](https://scholar.google.com/scholar?q=Wei+Huang%2C+Yangdong+Liu%2C+Haotong+Qin%2C+Ying+Li%2C+Shiming+Zhang%2C+Xianglong+Liu%2C+Michele+Magno%2C+and+Xiaojuan+Qi%2E+2024%2E+Billm%3A+pushing+the+limit+of+post%2Dtraining+quantization+for+llms%2E+In+Proceedings+of+th)
8. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… [find](https://scholar.google.com/scholar?q=Albert+Q%2E+Jiang%2C+Alexandre+Sablayrolles%2C+Arthur+Mensch%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guillaume+Lample%2C+Lucile+Saulnier%2C+L%C3%A9lio+Renard+Lav)
9. Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, and Qun Liu. 2020. [Tinybert: Distilling bert for natural language understanding](https://md.research.elamur.ai/free/en/papers/jiao-2020-tinybert-distilling-bert-natural-language/). In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4163–4174.
10. Chi-Heng Lin, Shangqian Gao, James Seale Smith, Abhishek Patel, Shikhar Tuli, Yilin Shen, Hongxia Jin, and Yen-Chang Hsu. 2024. Modegpt: Modular decomposition for large language model compression. arXiv preprint arXiv:2408.09632. [find](https://scholar.google.com/scholar?q=Chi%2DHeng+Lin%2C+Shangqian+Gao%2C+James+Seale+Smith%2C+Abhishek+Patel%2C+Shikhar+Tuli%2C+Yilin+Shen%2C+Hongxia+Jin%2C+and+Yen%2DChang+Hsu%2E+2024%2E+Modegpt%3A+Modular+decomposition+for+large+language+model+compression%2E+arX)
11. Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, and 1 others. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437. [find](https://scholar.google.com/scholar?q=Aixin+Liu%2C+Bei+Feng%2C+Bing+Xue%2C+Bingxuan+Wang%2C+Bochao+Wu%2C+Chengda+Lu%2C+Chenggang+Zhao%2C+Chengqi+Deng%2C+Chenyu+Zhang%2C+Chong+Ruan%2C+and+1+others%2E+2024%2E+Deepseek%2Dv3+technical+report%2E+arXiv+preprint+arXiv%3A2412)
12. Xinyin Ma, Gongfan Fang, and Xinchao Wang. 2023. [Llm-pruner: On the structural pruning of large language models](https://md.research.elamur.ai/free/en/papers/ma-2023-llm-pruner-structural-pruning-large/). Advances in neural information processing systems, 36:21702–21720.
13. Xin Men, Mingyu Xu, Qingyu Zhang, Bingning Wang, Hongyu Lin, Yaojie Lu, Xianpei Han, and Weipeng Chen. 2024. [Shortgpt: Layers in large language models are more redundant than you expect](https://md.research.elamur.ai/free/en/papers/men-2024-shortgpt-layers-large-language-models/). arXiv preprint arXiv:2403.03853.
14. Mingjie Sun, Zhuang Liu, Anna Bair, and J Zico Kolter. 2024. A simple and effective pruning approach for large language models. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Mingjie+Sun%2C+Zhuang+Liu%2C+Anna+Bair%2C+and+J+Zico+Kolter%2E+2024%2E+A+simple+and+effective+pruning+approach+for+large+language+models%2E+In+The+Twelfth+International+Conference+on+Learning+Representations%2E)
15. Zhiqing Sun, Hongkun Yu, Xiaodan Song, Renjie Liu, Yiming Yang, and Denny Zhou. 2020. [Mobilebert: a compact task-agnostic bert for resource-limited devices](https://md.research.elamur.ai/free/en/papers/sun-2020-mobilebert-compact-task-agnostic-bert/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2158–2170.
16. Jiayi Tian, Chao Fang, Haonan Wang, and Zhongfeng Wang. 2023. Bebert: Efficient and robust binary ensemble bert. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE. [find](https://scholar.google.com/scholar?q=Jiayi+Tian%2C+Chao+Fang%2C+Haonan+Wang%2C+and+Zhongfeng+Wang%2E+2023%2E+Bebert%3A+Efficient+and+robust+binary+ensemble+bert%2E+In+ICASSP+2023%2D2023+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Proce)
17. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, and 1 others. 2023. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). arXiv preprint arXiv:2302.13971.
18. Tycho FA van der Ouderaa, Markus Nagel, Mart Van Baalen, and Tijmen Blankevoort. 2024. The llm surgeon. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Tycho+FA+van+der+Ouderaa%2C+Markus+Nagel%2C+Mart+Van+Baalen%2C+and+Tijmen+Blankevoort%2E+2024%2E+The+llm+surgeon%2E+In+The+Twelfth+International+Conference+on+Learning+Representations%2E)
19. Xin Wang, Yu Zheng, Zhongwei Wan, and Mi Zhang. 2024. Svd-llm: Truncation-aware singular value decomposition for large language model compression. arXiv preprint arXiv:2403.07378. — [arXiv:2403.07378](https://arxiv.org/abs/2403.07378) [find](https://scholar.google.com/scholar?q=Xin+Wang%2C+Yu+Zheng%2C+Zhongwei+Wan%2C+and+Mi+Zhang%2E+2024%2E+Svd%2Dllm%3A+Truncation%2Daware+singular+value+decomposition+for+large+language+model+compression%2E+arXiv+preprint+arXiv%3A2403%2E07378%2E+%E2%80%94+%5BarXiv%3A2403%2E07378%5D)
20. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, and 1 others. 2019. Huggingface’s [transformers: State-of-the-art natural language processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/). arXiv preprint arXiv:1910.03771.
21. Yifan Yang, Kai Zhen, Bhavana Ganesh, Aram Galstyan, Goeric Huybrechts, Markus Müller, Jonas M Kübler, Rupak Vignesh Swaminathan, Athanasios Mouchtaris, Sravan Babu Bodapati, and 1 others. 2025. Wanda++: Pruning large language models via regional gradients. arXiv preprint arXiv:2503.04992. [find](https://scholar.google.com/scholar?q=Yifan+Yang%2C+Kai+Zhen%2C+Bhavana+Ganesh%2C+Aram+Galstyan%2C+Goeric+Huybrechts%2C+Markus+M%C3%BCller%2C+Jonas+M+K%C3%BCbler%2C+Rupak+Vignesh+Swaminathan%2C+Athanasios+Mouchtaris%2C+Sravan+Babu+Bodapati%2C+and+1+others%2E+2025%2E+Wanda)
22. Yifan Yang, Jiajun Zhou, Ngai Wong, and Zheng Zhang. 2024a. Loretta: Low-rank economic tensor-train adaptation for ultra-low-parameter fine-tuning of large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lan… [find](https://scholar.google.com/scholar?q=Yifan+Yang%2C+Jiajun+Zhou%2C+Ngai+Wong%2C+and+Zheng+Zhang%2E+2024a%2E+Loretta%3A+Low%2Drank+economic+tensor%2Dtrain+adaptation+for+ultra%2Dlow%2Dparameter+fine%2Dtuning+of+large+language+models%2E+In+Proceedings+of+the+2024+)
23. Yifei Yang, Zouying Cao, and Hai Zhao. 2024b. Laco: Large language model pruning via layer collapse. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 6401–6417. [find](https://scholar.google.com/scholar?q=Yifei+Yang%2C+Zouying+Cao%2C+and+Hai+Zhao%2E+2024b%2E+Laco%3A+Large+language+model+pruning+via+layer+collapse%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+2024%2C+pages+6401%E2%80%936417%2E)
24. Hao Yu and Jianxin Wu. 2023. Compressing transformers: features are low-rank, but weights are not! In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 11007–11015. [find](https://scholar.google.com/scholar?q=Hao+Yu+and+Jianxin+Wu%2E+2023%2E+Compressing+transformers%3A+features+are+low%2Drank%2C+but+weights+are+not%21+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+volume+37%2C+pages+11007%E2%80%9311015%2E)
25. Zhihang Yuan, Yuzhang Shang, Yue Song, Qiang Wu, Yan Yan, and Guangyu Sun. 2023. Asvd: Activation-aware singular value decomposition for compressing large language models. arXiv preprint arXiv:2312.05821. — [arXiv:2312.05821](https://arxiv.org/abs/2312.05821) [find](https://scholar.google.com/scholar?q=Zhihang+Yuan%2C+Yuzhang+Shang%2C+Yue+Song%2C+Qiang+Wu%2C+Yan+Yan%2C+and+Guangyu+Sun%2E+2023%2E+Asvd%3A+Activation%2Daware+singular+value+decomposition+for+compressing+large+language+models%2E+arXiv+preprint+arXiv%3A2312%2E05)
26. Jialin Zhao, Yingtao Zhang, and Carlo Vittorio Cannistraci. 2025. Pivoting factorization: A compact meta low-rank representation of sparsity for efficient inference in large language models. In Forty-second International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Jialin+Zhao%2C+Yingtao+Zhang%2C+and+Carlo+Vittorio+Cannistraci%2E+2025%2E+Pivoting+factorization%3A+A+compact+meta+low%2Drank+representation+of+sparsity+for+efficient+inference+in+large+language+models%2E+In+Forty%2D)
27. Longguang Zhong, Fanqi Wan, Ruijun Chen, Xiaojun Quan, and Liangzhi Li. 2024. Blockpruner: Fine-grained pruning for large language models. arXiv preprint arXiv:2406.10594. [find](https://scholar.google.com/scholar?q=Longguang+Zhong%2C+Fanqi+Wan%2C+Ruijun+Chen%2C+Xiaojun+Quan%2C+and+Liangzhi+Li%2E+2024%2E+Blockpruner%3A+Fine%2Dgrained+pruning+for+large+language+models%2E+arXiv+preprint+arXiv%3A2406%2E10594%2E)

*Updated 29 October 2026*