---
title: "Iterative Structured Pruning for Large Language Models with"
description: "Unstructured pruning creates irregular sparsity that standard hardware cannot exploit well. 38 references."
authors: ["Guangxin Wu", "Hao Zhang", "Zhang Zhibin", "Jiafeng Guo", "Xueqi Cheng"]
year: 2026
venue: "Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track)"
source: "https://aclanthology.org/2026.eacl-industry.1/"
doi: "10.18653/v1/2026.eacl-industry.1"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wu-2026-iterative-structured-pruning-large-language/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wu-2026-iterative-structured-pruning-large-language/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/14/) / Iterative Structured Pruning for Large Language Models with Multi-Domain Calibration

# Iterative Structured Pruning for Large Language Models with Multi-Domain Calibration

**Authors:** Guangxin Wu, [Hao Zhang](https://md.research.elamur.ai/free/en/authors/hao-zhang/), Zhang Zhibin, [Jiafeng Guo](https://md.research.elamur.ai/free/en/authors/jiafeng-guo/) and [Xueqi Cheng](https://md.research.elamur.ai/free/en/authors/xueqi-cheng/) · **Year:** 2026 · **Venue:** [Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track)](https://md.research.elamur.ai/free/en/venues/eacl/) ([EACL 2026](https://md.research.elamur.ai/free/en/venues/eacl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.eacl-industry.1/) · [PDF](https://aclanthology.org/2026.eacl-industry.1.pdf) · [DOI 10.18653/v1/2026.eacl-industry.1](https://doi.org/10.18653/v1/2026.eacl-industry.1) · [OpenAlex](https://openalex.org/W7140107856)

**Citations:** 1 citations (OpenAlex, 2026-10-14)

## Summary
Large language models are expensive to deploy because of their memory use, compute cost and latency. Unstructured pruning creates irregular sparsity that standard hardware cannot exploit well. The authors therefore focus on structured pruning, which removes whole channels and attention heads. They build on FLAP, which scores channels by activation variance weighted by weight magnitude, prunes in one pass and offsets the removed output with a bias term. They add two components:

1. **Mixed-domain calibration set.** It combines Wikipedia, Common Crawl, source code and mathematical text, so that activation statistics are not skewed toward a single domain.
2. **Iterative calibration.** Statistics are recomputed after each pruning round, because earlier removals change downstream activations.

Experiments on the Qwen2.5 family compare the method against Wanda-sp and FLAP. The paper reports better accuracy retention at equal compression, although the detailed baseline comparison was not in the excerpt provided. Gradual pruning matters most at high sparsity. The approach needs no retraining and produces models that run on ordinary hardware.

## Key points
- **Setup:** Baselines are Wanda-sp and FLAP. Models are Qwen2.5-7B, 14B and 32B. Evaluation uses zero-shot ARC-c, ARC-e, HellaSwag, OBQA, PIQA and Winogrande, plus WikiText2 perplexity.
- **Iterative pruning results:**
  - At 25% pruning, perplexity barely changes with the number of steps.
  - At 50% pruning, most of the gain comes in the first 3–4 iterations, and 4–6 steps are usually enough.
  - For Qwen2.5-14B at 50%, one-shot pruning degrades the model severely, while six steps bring perplexity down to roughly 44.
- **Mixed-domain calibration on Qwen2.5-14B** (average zero-shot accuracy; the dense model scores 65.42):

  | Pruning ratio | Single-domain | Mixed-domain |
  |---|---|---|
  | 25% | 53.96 | 53.94 |
  | 50% | 35.90 | 37.35 |

  The benefit therefore shows up mainly at high sparsity; at 25% the two settings are essentially tied.
- **Method details:** Low-fluctuation channels are removed, and a bias computed from mean activations compensates for them. Statistics from the different domains are combined as a weighted mixture.
- **Limitation:** Because of compute limits, the method was not tested on models of around 70B parameters.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Chinese Academy of Sciences](https://md.research.elamur.ai/free/en/government/chinese-academy-of-sciences/) — Government body, Beijing, China · 194 works · 23,906 citations
- [University of Chinese Academy of Sciences](https://md.research.elamur.ai/free/en/universities/university-of-chinese-academy-of-sciences/) — University, Beijing, China · 147 works · 8,480 citations
- [Institute of Computing Technology](https://md.research.elamur.ai/free/en/labs/institute-of-computing-technology/) — Research institute, Beijing, China · 51 works · 3,411 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🪶 Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/) · 242 works

## Cite

Cite as: Guangxin Wu, Hao Zhang, Zhang Zhibin, Jiafeng Guo and Xueqi Cheng. 2026. Iterative Structured Pruning for Large Language Models with Multi-Domain Calibration. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wu2026iterative,
  title = {{Iterative Structured Pruning for Large Language Models with Multi-Domain Calibration}},
  author = {Guangxin Wu, Hao Zhang, Zhang Zhibin, Jiafeng Guo and Xueqi Cheng},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-industry.1},
  url = {https://doi.org/10.18653/v1/2026.eacl-industry.1}
}
```

- [← previous Ita-Lib at SVELA: Detecting the Forgotten –- Representation-Based Approach for Verifying Machine Unlearning](https://md.research.elamur.ai/free/en/papers/yassine-2026-ita-lib-svela-detecting-forgotten/)
- [next → ITLC at SemEval-2026 Task 11: Normalization and Deterministic Parsing for Formal Reasoning in LLMs](https://md.research.elamur.ai/free/en/papers/muhamad-2026-itlc-semeval-2026-task-11/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/14/)

## References

38 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Yongqi An, Xu Zhao, Tao Yu, Ming Tang, and Jinqiao Wang. 2024. Fluctuation-based adaptive structured pruning for large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 10865–10873. [find](https://scholar.google.com/scholar?q=Yongqi+An%2C+Xu+Zhao%2C+Tao+Yu%2C+Ming+Tang%2C+and+Jinqiao+Wang%2E+2024%2E+Fluctuation%2Dbased+adaptive+structured+pruning+for+large+language+models%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence)
2. C. Li and 1 others. 2023a. Fine-tuning techniques for efficient model adaptation. AI Research Journal. [find](https://scholar.google.com/scholar?q=C%2E+Li+and+1+others%2E+2023a%2E+Fine%2Dtuning+techniques+for+efficient+model+adaptation%2E+AI+Research+Journal%2E)
3. Yong Li, Wei Du, Liquan Han, Zhenjian Zhang, and Tongtong Liu. 2023b. A communication-efficient, privacy-preserving federated learning algorithm based on two-stage gradient pruning and differentiated differential privacy. Sensors, 23(23):9305. [find](https://scholar.google.com/scholar?q=Yong+Li%2C+Wei+Du%2C+Liquan+Han%2C+Zhenjian+Zhang%2C+and+Tongtong+Liu%2E+2023b%2E+A+communication%2Defficient%2C+privacy%2Dpreserving+federated+learning+algorithm+based+on+two%2Dstage+gradient+pruning+and+differentiated+)
4. Anonymous. 2024. Unstructured pruning and low rank factorisation of self-supervised pre-trained speech models. IEEE Transactions on Audio, Speech, and Language Processing, pages 1046–1058. [find](https://scholar.google.com/scholar?q=Anonymous%2E+2024%2E+Unstructured+pruning+and+low+rank+factorisation+of+self%2Dsupervised+pre%2Dtrained+speech+models%2E+IEEE+Transactions+on+Audio%2C+Speech%2C+and+Language+Processing%2C+pages+1046%E2%80%931058%2E)
5. Sheng Liao and 1 others. 2023. Can unstructured pruning reduce the depth in deep neural networks? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. [find](https://scholar.google.com/scholar?q=Sheng+Liao+and+1+others%2E+2023%2E+Can+unstructured+pruning+reduce+the+depth+in+deep+neural+networks%3F+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Computer+Vision+and+Pattern+Recognition+Workshops%2E)
6. Saleh Ashkboos, Maximilian L Croci, Marcelo Gennari do Nascimento, Torsten Hoefler, and James Hensman. 2024. Slicegpt: Compress large language models by deleting rows and columns. arXiv preprint arXiv:2401.15024. [find](https://scholar.google.com/scholar?q=Saleh+Ashkboos%2C+Maximilian+L+Croci%2C+Marcelo+Gennari+do+Nascimento%2C+Torsten+Hoefler%2C+and+James+Hensman%2E+2024%2E+Slicegpt%3A+Compress+large+language+models+by+deleting+rows+and+columns%2E+arXiv+preprint+arXiv)
7. Yang Luo, Shiru Wang, Jun Liu, Jiaxuan Xiao, Rundong Xue, Zeyu Zhang, Hao Zhang, Yu Lu, Yang Zhao, and Yutong Xie. 2025. Pathohr: Breast cancer survival prediction on high-resolution pathological images. arXiv preprint arXiv:2503.17970. — [arXiv:2503.17970](https://arxiv.org/abs/2503.17970) [find](https://scholar.google.com/scholar?q=Yang+Luo%2C+Shiru+Wang%2C+Jun+Liu%2C+Jiaxuan+Xiao%2C+Rundong+Xue%2C+Zeyu+Zhang%2C+Hao+Zhang%2C+Yu+Lu%2C+Yang+Zhao%2C+and+Yutong+Xie%2E+2025%2E+Pathohr%3A+Breast+cancer+survival+prediction+on+high%2Dresolution+pathological+imag)
8. Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, and 1 others. 2020. [Piqa: Reasoning about physical commonsense in natural language](https://md.research.elamur.ai/free/en/papers/bisk-2019-piqa-reasoning-about-physical-commonsense/). In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 7432–7439.
9. Chenrui Ma, Rongchang Zhao, Xi Xiao, Hongyang Xie, Tianyang Wang, Xiao Wang, Hao Zhang, and Yanning Shen. 2025. Cad-vae: Leveraging correlation-aware latents for comprehensive fair disentanglement. arXiv preprint arXiv:2503.07938. — [arXiv:2503.07938](https://arxiv.org/abs/2503.07938) [find](https://scholar.google.com/scholar?q=Chenrui+Ma%2C+Rongchang+Zhao%2C+Xi+Xiao%2C+Hongyang+Xie%2C+Tianyang+Wang%2C+Xiao+Wang%2C+Hao+Zhang%2C+and+Yanning+Shen%2E+2025%2E+Cad%2Dvae%3A+Leveraging+correlation%2Daware+latents+for+comprehensive+fair+disentanglement%2E+ar)
10. Yuchen Cai, Zhen Wang, Yujun Li, Sheng Wang, Zhiyuan Liu, and Maosong Sun. 2023. Gptq: Accurate post-training quantization for generative pre-trained transformers. arXiv preprint arXiv:2302.06557. [find](https://scholar.google.com/scholar?q=Yuchen+Cai%2C+Zhen+Wang%2C+Yujun+Li%2C+Sheng+Wang%2C+Zhiyuan+Liu%2C+and+Maosong+Sun%2E+2023%2E+Gptq%3A+Accurate+post%2Dtraining+quantization+for+generative+pre%2Dtrained+transformers%2E+arXiv+preprint+arXiv%3A2302%2E06557%2E)
11. Xinyin Ma, Gongfan Fang, and Xinchao Wang. 2023. [Llm-pruner: On the structural pruning of large language models](https://md.research.elamur.ai/free/en/papers/ma-2023-llm-pruner-structural-pruning-large/). Advances in neural information processing systems, 36:21702–21720.
12. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. [Think you have solved question answering? try arc, the ai2 reasoning challenge](https://md.research.elamur.ai/free/en/papers/clark-2018-think-have-solved-question-answering/). arXiv preprint arXiv:1803.05457.
13. Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. Can a suit of armor conduct electricity? a new dataset for open book question answering. arXiv preprint arXiv:1809.02789. [find](https://scholar.google.com/scholar?q=Todor+Mihaylov%2C+Peter+Clark%2C+Tushar+Khot%2C+and+Ashish+Sabharwal%2E+2018%2E+Can+a+suit+of+armor+conduct+electricity%3F+a+new+dataset+for+open+book+question+answering%2E+arXiv+preprint+arXiv%3A1809%2E02789%2E)
14. Zhaoyang Cong, Ziyang Wang, Hao Zhang, Guowei Zheng, Keming Cao, Lina Zhao, Ruipeng Song, Jianqing Li, and Chengyu Liu. 2025. Hierarchical multi-scale feature fusion network for multi-center major depressive disorder classification with t1-weighted mri. In Annual International Conference of the IEEE… [find](https://scholar.google.com/scholar?q=Zhaoyang+Cong%2C+Ziyang+Wang%2C+Hao+Zhang%2C+Guowei+Zheng%2C+Keming+Cao%2C+Lina+Zhao%2C+Ruipeng+Song%2C+Jianqing+Li%2C+and+Chengyu+Liu%2E+2025%2E+Hierarchical+multi%2Dscale+feature+fusion+network+for+multi%2Dcenter+major+dep)
15. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, and 1 others. 2019. [Pytorch: An imperative style, high-performance deep learning library](https://md.research.elamur.ai/free/en/papers/paszke-2019-pytorch-imperative-style-high-performance/). Advances in neural information processing systems, 32.
16. G. Ding and 1 others. 2022. Efficient fine-tuning for resource-constrained systems. Proceedings of the Machine Learning Conference. [find](https://scholar.google.com/scholar?q=G%2E+Ding+and+1+others%2E+2022%2E+Efficient+fine%2Dtuning+for+resource%2Dconstrained+systems%2E+Proceedings+of+the+Machine+Learning+Conference%2E)
17. Xuyin Qi, Zeyu Zhang, Canxuan Gang, Hao Zhang, Lei Zhang, Zhiwei Zhang, and Yang Zhao. 2025a. [find](https://scholar.google.com/scholar?q=Xuyin+Qi%2C+Zeyu+Zhang%2C+Canxuan+Gang%2C+Hao+Zhang%2C+Lei+Zhang%2C+Zhiwei+Zhang%2C+and+Yang+Zhao%2E+2025a%2E)
18. Mediaug: Exploring visual augmentation in medical imaging. In Annual Conference on Medical Image Understanding and Analysis, pages 218–232. Springer. [find](https://scholar.google.com/scholar?q=Mediaug%3A+Exploring+visual+augmentation+in+medical+imaging%2E+In+Annual+Conference+on+Medical+Image+Understanding+and+Analysis%2C+pages+218%E2%80%93232%2E+Springer%2E)
19. Heng Zhang, Tianyi Zhang, Yuling Shi, Xiaodong Gu, Yaomin Shen, Zijian Zhang, Yilei Yuan, Hao Zhang, and Jin Huang. 2025b. Can representation gaps be the key to enhancing robustness in graph-text alignment? arXiv preprint arXiv:2510.12087. — [arXiv:2510.12087](https://arxiv.org/abs/2510.12087) [find](https://scholar.google.com/scholar?q=Heng+Zhang%2C+Tianyi+Zhang%2C+Yuling+Shi%2C+Xiaodong+Gu%2C+Yaomin+Shen%2C+Zijian+Zhang%2C+Yilei+Yuan%2C+Hao+Zhang%2C+and+Jin+Huang%2E+2025b%2E+Can+representation+gaps+be+the+key+to+enhancing+robustness+in+graph%2Dtext+alig)
20. Xuyin Qi, Zeyu Zhang, Huazhan Zheng, Mingxi Chen, Numan Kutaiba, Ruth Lim, Cherie Chiang, Zi En Tham, Xuan Ren, Wenxin Zhang, and 1 others. 2025b. Medconv: Convolutions beat transformers on long-tailed bone density prediction. IJCNN2025. [find](https://scholar.google.com/scholar?q=Xuyin+Qi%2C+Zeyu+Zhang%2C+Huazhan+Zheng%2C+Mingxi+Chen%2C+Numan+Kutaiba%2C+Ruth+Lim%2C+Cherie+Chiang%2C+Zi+En+Tham%2C+Xuan+Ren%2C+Wenxin+Zhang%2C+and+1+others%2E+2025b%2E+Medconv%3A+Convolutions+beat+transformers+on+long%2Dtaile)
21. Qifan Zhang, Yunhui Guo, and Yu Xiang. 2024. Continual distillation learning: Knowledge distillation in prompt-based continual learning. Preprint, arXiv:2407.13911. — [arXiv:2407.13911](https://arxiv.org/abs/2407.13911) [find](https://scholar.google.com/scholar?q=Qifan+Zhang%2C+Yunhui+Guo%2C+and+Yu+Xiang%2E+2024%2E+Continual+distillation+learning%3A+Knowledge+distillation+in+prompt%2Dbased+continual+learning%2E+Preprint%2C+arXiv%3A2407%2E13911%2E+%E2%80%94+%5BarXiv%3A2407%2E13911%5D%28https%3A%2F%2Farxiv%2E)
22. A. Qin and 1 others. 2023. Advances in state-of-the-art natural language processing. Journal of NLP Research. [find](https://scholar.google.com/scholar?q=A%2E+Qin+and+1+others%2E+2023%2E+Advances+in+state%2Dof%2Dthe%2Dart+natural+language+processing%2E+Journal+of+NLP+Research%2E)
23. Heng Zheng, Yuling Shi, Xiaodong Gu, Haochen You, Zijian Zhang, Lubin Gan, Hao Zhang, Wenjun Huang, and Jin Huang. 2025a. Graphgeo: Multi-agent debate framework for visual geo-localization with heterogeneous graph neural networks. arXiv preprint arXiv:2511.00908. — [arXiv:2511.00908](https://arxiv.org/abs/2511.00908) [find](https://scholar.google.com/scholar?q=Heng+Zheng%2C+Yuling+Shi%2C+Xiaodong+Gu%2C+Haochen+You%2C+Zijian+Zhang%2C+Lubin+Gan%2C+Hao+Zhang%2C+Wenjun+Huang%2C+and+Jin+Huang%2E+2025a%2E+Graphgeo%3A+Multi%2Dagent+debate+framework+for+visual+geo%2Dlocalization+with+hetero)
24. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99–106. [find](https://scholar.google.com/scholar?q=Keisuke+Sakaguchi%2C+Ronan+Le+Bras%2C+Chandra+Bhagavatula%2C+and+Yejin+Choi%2E+2021%2E+Winogrande%3A+An+adversarial+winograd+schema+challenge+at+scale%2E+Communications+of+the+ACM%2C+64%289%29%3A99%E2%80%93106%2E)
25. Heng Zheng, Haochen You, Zijun Liu, Zijian Zhang, Lubin Gan, Hao Zhang, Wenjun Huang, and Jin Huang. 2025b. G2rammar: Bilingual grammar modeling for enhanced text-attributed graph learning. arXiv preprint arXiv:2511.00911. — [arXiv:2511.00911](https://arxiv.org/abs/2511.00911) [find](https://scholar.google.com/scholar?q=Heng+Zheng%2C+Haochen+You%2C+Zijun+Liu%2C+Zijian+Zhang%2C+Lubin+Gan%2C+Hao+Zhang%2C+Wenjun+Huang%2C+and+Jin+Huang%2E+2025b%2E+G2rammar%3A+Bilingual+grammar+modeling+for+enhanced+text%2Dattributed+graph+learning%2E+arXiv+prep)
26. Mingjie Sun, Zhuang Liu, Anna Bair, and J Zico Kolter. 2023. A simple and effective pruning approach for large language models. arXiv preprint arXiv:2306.11695. [find](https://scholar.google.com/scholar?q=Mingjie+Sun%2C+Zhuang+Liu%2C+Anna+Bair%2C+and+J+Zico+Kolter%2E+2023%2E+A+simple+and+effective+pruning+approach+for+large+language+models%2E+arXiv+preprint+arXiv%3A2306%2E11695%2E)
27. Yuxiao Zhou, Zhen Wang, Yujun Li, Sheng Wang, Zhiyuan Liu, and Maosong Sun. 2023. Smoothquant: Accurate and efficient post-training quantization for large language models. arXiv preprint arXiv:2302.06557. [find](https://scholar.google.com/scholar?q=Yuxiao+Zhou%2C+Zhen+Wang%2C+Yujun+Li%2C+Sheng+Wang%2C+Zhiyuan+Liu%2C+and+Maosong+Sun%2E+2023%2E+Smoothquant%3A+Accurate+and+efficient+post%2Dtraining+quantization+for+large+language+models%2E+arXiv+preprint+arXiv%3A2302%2E06)
28. F. Wang and 1 others. 2023. Practical applications of llms in specialized domains. Specialized AI Applications. [find](https://scholar.google.com/scholar?q=F%2E+Wang+and+1+others%2E+2023%2E+Practical+applications+of+llms+in+specialized+domains%2E+Specialized+AI+Applications%2E)
29. Thomas Wolf. 2020. [Transformers: State-of-the-art natural language processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/). arXiv preprint arXiv:1910.03771.
30. Yuxiao Zhou, Zhen Wang, Yujun Li, Sheng Wang, Zhiyuan Liu, and Maosong Sun. 2024. Framequant: Flexible low-bit quantization for transformers. arXiv preprint arXiv:2402.06557. — [arXiv:2402.06557](https://arxiv.org/abs/2402.06557) [find](https://scholar.google.com/scholar?q=Yuxiao+Zhou%2C+Zhen+Wang%2C+Yujun+Li%2C+Sheng+Wang%2C+Zhiyuan+Liu%2C+and+Maosong+Sun%2E+2024%2E+Framequant%3A+Flexible+low%2Dbit+quantization+for+transformers%2E+arXiv+preprint+arXiv%3A2402%2E06557%2E+%E2%80%94+%5BarXiv%3A2402%2E06557%5D%28http)
31. Yu-Hang Wu, Yu-Jie Xiong, Hao Zhang, Jia-Chen Zhang, and Zheng Zhou. 2025. Sugar-coated poison: Benign generation unlocks llm jailbreaking. [find](https://scholar.google.com/scholar?q=Yu%2DHang+Wu%2C+Yu%2DJie+Xiong%2C+Hao+Zhang%2C+Jia%2DChen+Zhang%2C+and+Zheng+Zhou%2E+2025%2E+Sugar%2Dcoated+poison%3A+Benign+generation+unlocks+llm+jailbreaking%2E)
32. B. Zhu and 1 others. 2023. Large language models: Progress and applications. Advances in NLP. [find](https://scholar.google.com/scholar?q=B%2E+Zhu+and+1+others%2E+2023%2E+Large+language+models%3A+Progress+and+applications%2E+Advances+in+NLP%2E)
33. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, and 1 others. 2024. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115. [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Haoran+Wei%2C+and+1+others%2E+2024%2E+Qwen2%2E+5+technical+report%2E+arXiv+preprint+arXiv%3A2412%2E15115)
34. Zhen Yang, Zilun Zhang, Sheng Wang, Jie Li, Meishan Zhang, Zhiyuan Liu, and Maosong Sun. 2021. Knowledge distillation: A survey. arXiv preprint arXiv:2106.05860. — [arXiv:2106.05860](https://arxiv.org/abs/2106.05860) [find](https://scholar.google.com/scholar?q=Zhen+Yang%2C+Zilun+Zhang%2C+Sheng+Wang%2C+Jie+Li%2C+Meishan+Zhang%2C+Zhiyuan+Liu%2C+and+Maosong+Sun%2E+2021%2E+Knowledge+distillation%3A+A+survey%2E+arXiv+preprint+arXiv%3A2106%2E05860%2E+%E2%80%94+%5BarXiv%3A2106%2E05860%5D%28https%3A%2F%2Farxiv%2Eorg)
35. Zhengwu Yang and Han Zhang. 2022. Comparative analysis of structured pruning and unstructured pruning. In Frontier Computing, page 112. Springer. [find](https://scholar.google.com/scholar?q=Zhengwu+Yang+and+Han+Zhang%2E+2022%2E+Comparative+analysis+of+structured+pruning+and+unstructured+pruning%2E+In+Frontier+Computing%2C+page+112%2E+Springer%2E)
36. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830. [find](https://scholar.google.com/scholar?q=Rowan+Zellers%2C+Ari+Holtzman%2C+Yonatan+Bisk%2C+Ali+Farhadi%2C+and+Yejin+Choi%2E+2019%2E+Hellaswag%3A+Can+a+machine+really+finish+your+sentence%3F+arXiv+preprint+arXiv%3A1905%2E07830%2E)
37. D. Zhang and 1 others. 2023. Parameter-efficient fine-tuning methods for llms. Journal of Machine Learning Research. [find](https://scholar.google.com/scholar?q=D%2E+Zhang+and+1+others%2E+2023%2E+Parameter%2Defficient+fine%2Dtuning+methods+for+llms%2E+Journal+of+Machine+Learning+Research%2E)
38. Heng Zhang, Haichuan Hu, Yaomin Shen, Weihao Yu, Yilei Yuan, Haochen You, Guo Cheng, Zijian Zhang, Lubin Gan, Huihui Wei, and 1 others. 2025a. Asy-moe: Leveraging modal asymmetry for enhanced expert specialization in large vision-language models. arXiv preprint arXiv:2509.12715. — [arXiv:2509.12715](https://arxiv.org/abs/2509.12715) [find](https://scholar.google.com/scholar?q=Heng+Zhang%2C+Haichuan+Hu%2C+Yaomin+Shen%2C+Weihao+Yu%2C+Yilei+Yuan%2C+Haochen+You%2C+Guo+Cheng%2C+Zijian+Zhang%2C+Lubin+Gan%2C+Huihui+Wei%2C+and+1+others%2E+2025a%2E+Asy%2Dmoe%3A+Leveraging+modal+asymmetry+for+enhanced+expert+s)

*Updated 1 November 2026*