---
title: "Faster MoE LLM Inference for Extremely Large Models (2026)"
description: "To address this, they propose ASET (Adaptive Skipping with Entropy-Penalized Thresholding). 35 references, cited by 8 papers here."
authors: ["Haoqi Yang", "Luohe Shi", "Qiwei Li", "Zuchao Li", "Ping Wang", "Hao Huang", "Hai Zhao"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.2140/"
doi: "10.18653/v1/2026.findings-acl.2140"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/yang-2026-faster-moe-llm-inference-extremely/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/yang-2026-faster-moe-llm-inference-extremely/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/2/) / Faster MoE LLM Inference for Extremely Large Models

# Faster MoE LLM Inference for Extremely Large Models

**Authors:** Haoqi Yang, Luohe Shi, Qiwei Li, [Zuchao Li](https://md.research.elamur.ai/free/en/authors/zuchao-li/), [Ping Wang](https://md.research.elamur.ai/free/en/authors/ping-wang/), Hao Huang and [Hai Zhao](https://md.research.elamur.ai/free/en/authors/hai-zhao/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.2140/) · [PDF](https://aclanthology.org/2026.findings-acl.2140.pdf) · [DOI 10.18653/v1/2026.findings-acl.2140](https://doi.org/10.18653/v1/2026.findings-acl.2140) · [OpenAlex](https://openalex.org/W4415248342)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [8](https://md.research.elamur.ai/free/en/papers/yang-2026-faster-moe-llm-inference-extremely/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Fine-grained Mixture-of-Experts models such as DeepSeek-V3 route each token through many small experts. Their inference cost therefore depends heavily on how many experts are activated, yet these models usually apply the same Top-k to every token. The authors study training-free expert skipping at inference time, meaning they cut the activated experts below the router's original choice. They compare a uniform reduction of k with layer-wise Top-k schedules found by search. Static skipping alone raises throughput substantially. However, the best layer-wise pattern depends on the model and its routing function (softmax vs. sigmoid), so no schedule works universally. To address this, they propose ASET (Adaptive Skipping with Entropy-Penalized Thresholding). For each token, ASET uses router confidence and entropy to set a threshold, then keeps the smallest expert subset whose confidence mass reaches that threshold, never exceeding the original budget. On OLMoE, ASET offers a smooth range of trade-offs between speed and quality. This matters because it lets operators speed up very large MoE deployments without retraining.

## Key points
- Static skipping strategies raised throughput by 10–78% across the studied fine-grained MoEs with little quality loss. On DeepSeek-V3, gains of at least 10% came with no measurable degradation.
- Optimal layer-wise schedules reverse between models. Descending reductions suit DeepSeek-V2-Lite (softmax router), while ascending reductions suit DeepSeek-V3 (sigmoid router).
- ASET is tuned through three hyperparameters (θ_min, θ_max, γ). The average number of active experts is set mainly by θ_min and θ_max, and γ has only a minor shaping effect.
- On OLMoE, ASET produces a Pareto frontier in which task score rises monotonically, with diminishing returns, as the average activation k̄ increases.
- The evaluation has two main limits: throughput is measured only under controlled decoding, and evidence for ASET comes mostly from OLMoE. Code is released at github.com/brinenick511/ASET.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Shanghai Jiao Tong University](https://md.research.elamur.ai/free/en/universities/shanghai-jiao-tong-university/) — University, Shanghai, China · 212 works · 19,544 citations
- [Wuhan University](https://md.research.elamur.ai/free/en/universities/wuhan-university/) — University, Wuhan, China · 47 works · 1,246 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works · [🔌 Hardware and Architecture](https://md.research.elamur.ai/free/en/shelves/computer-science/hardware-and-architecture/) · 167 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works

## Cite

Cite as: Haoqi Yang, Luohe Shi, Qiwei Li, Zuchao Li, Ping Wang, Hao Huang and Hai Zhao. 2026. Faster MoE LLM Inference for Extremely Large Models. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{yang2026faster,
  title = {{Faster MoE LLM Inference for Extremely Large Models}},
  author = {Haoqi Yang, Luohe Shi, Qiwei Li, Zuchao Li, Ping Wang, Hao Huang and Hai Zhao},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.2140},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.2140}
}
```

- [← previous FadeIT at EVALITA 2026: Overview of the Fallacy Detection in Italian Social Media Texts Task](https://md.research.elamur.ai/free/en/papers/ramponi-2026-fadeit-evalita-2026-overview-fallacy/)
- [next → FinCARDS: Card-Based Analyst Reranking for Financial Document Question Answering](https://md.research.elamur.ai/free/en/papers/zhou-2026-fincards-card-based-analyst-reranking/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/2/)

## References

35 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Luisa Bentivogli, Bernardo Magnini, Ido Dagan, Hoa Trang Dang, and Danilo Giampiccolo. 2009. The fifth PASCAL recognizing textual entailment challenge. In Proceedings of the Second Text Analysis Conference, TAC 2009, Gaithersburg, Maryland, USA, November 16-17, 2009. NIST. [find](https://scholar.google.com/scholar?q=Luisa+Bentivogli%2C+Bernardo+Magnini%2C+Ido+Dagan%2C+Hoa+Trang+Dang%2C+and+Danilo+Giampiccolo%2E+2009%2E+The+fifth+PASCAL+recognizing+textual+entailment+challenge%2E+In+Proceedings+of+the+Second+Text+Analysis+Confe)
2. I-Chun Chen, Hsu-Shen Liu, Wei-Fang Sun, Chen-Hao Chao, Yen-Chang Hsu, and Chun-Yi Lee. 2025. Retraining-free merging of sparse mixture-of-experts via hierarchical clustering. [find](https://scholar.google.com/scholar?q=I%2DChun+Chen%2C+Hsu%2DShen+Liu%2C+Wei%2DFang+Sun%2C+Chen%2DHao+Chao%2C+Yen%2DChang+Hsu%2C+and+Chun%2DYi+Lee%2E+2025%2E+Retraining%2Dfree+merging+of+sparse+mixture%2Dof%2Dexperts+via+hierarchical+clustering%2E)
3. Tianyu Chen, Shaohan Huang, Yuan Xie, Binxing Jiao, Daxin Jiang, Haoyi Zhou, Jianxin Li, and Furu Wei. 2022. Task-specific expert pruning for sparse mixture-of-experts. CoRR, abs/2206.00277. — [arXiv:2206.00277](https://arxiv.org/abs/2206.00277) [find](https://scholar.google.com/scholar?q=Tianyu+Chen%2C+Shaohan+Huang%2C+Yuan+Xie%2C+Binxing+Jiao%2C+Daxin+Jiang%2C+Haoyi+Zhou%2C+Jianxin+Li%2C+and+Furu+Wei%2E+2022%2E+Task%2Dspecific+expert+pruning+for+sparse+mixture%2Dof%2Dexperts%2E+CoRR%2C+abs%2F2206%2E00277%2E+%E2%80%94+%5BarXiv%3A)
4. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, and… — [PaLM: Scaling Language Modeling with Pathways](https://md.research.elamur.ai/free/en/papers/chowdhery-2022-palm-scaling-language-modeling-pathways/)
5. Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019. [BoolQ: Exploring the surprising difficulty of natural yes/no questions](https://md.research.elamur.ai/free/en/papers/clark-2019-boolq-exploring-surprising-difficulty-natural/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:…
6. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. [Think you have solved question answering? try arc, the AI2 reasoning challenge](https://md.research.elamur.ai/free/en/papers/clark-2018-think-have-solved-question-answering/). CoRR, abs/1803.05457.
7. Damai Dai, Chengqi Deng, Chenggang Zhao, R.x. Xu, Huazuo Gao, Deli Chen, Jiashi Li, Wangding Zeng, Xingkai Yu, Y. Wu, Zhenda Xie, Y.k. Li, Panpan Huang, Fuli Luo, Chong Ruan, Zhifang Sui, and Wenfeng Liang. 2024. DeepSeekMoE: Towards ultimate expert specialization in mixture-of-experts language mode… [find](https://scholar.google.com/scholar?q=Damai+Dai%2C+Chengqi+Deng%2C+Chenggang+Zhao%2C+R%2Ex%2E+Xu%2C+Huazuo+Gao%2C+Deli+Chen%2C+Jiashi+Li%2C+Wangding+Zeng%2C+Xingkai+Yu%2C+Y%2E+Wu%2C+Zhenda+Xie%2C+Y%2Ek%2E+Li%2C+Panpan+Huang%2C+Fuli+Luo%2C+Chong+Ruan%2C+Zhifang+Sui%2C+and+Wenfeng+)
8. William Fedus, Barret Zoph, and Noam Shazeer. 2022. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. J. Mach. Learn. Res., 23:120:1–120:39. [find](https://scholar.google.com/scholar?q=William+Fedus%2C+Barret+Zoph%2C+and+Noam+Shazeer%2E+2022%2E+Switch+transformers%3A+Scaling+to+trillion+parameter+models+with+simple+and+efficient+sparsity%2E+J%2E+Mach%2E+Learn%2E+Res%2E%2C+23%3A120%3A1%E2%80%93120%3A39%2E)
9. Elias Frantar and Dan Alistarh. 2023. Sparsegpt: Massive language models can be accurately pruned in one-shot. In International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pages 10323–10337. PMLR. [find](https://scholar.google.com/scholar?q=Elias+Frantar+and+Dan+Alistarh%2E+2023%2E+Sparsegpt%3A+Massive+language+models+can+be+accurately+pruned+in+one%2Dshot%2E+In+International+Conference+on+Machine+Learning%2C+ICML+2023%2C+23%2D29+July+2023%2C+Honolulu%2C+Ha)
10. Andrey Gromov, Kushal Tirumala, Hassan Shapourian, Paolo Glorioso, and Dan Roberts. 2025. [The unreasonable ineffectiveness of the deeper layers](https://md.research.elamur.ai/free/en/papers/gromov-2024-unreasonable-ineffectiveness-deeper-layers/). In The Thirteenth International Conference on Learning Representations.
11. Junhui He, Shangyu Wu, Weidong Wen, Chun Jason Xue, and Qingan Li. 2024. CHESS: Optimizing LLM inference via channel-wise thresholding and selective sparsification. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 18658–18668, Miami, Florida, USA. Asso… [find](https://scholar.google.com/scholar?q=Junhui+He%2C+Shangyu+Wu%2C+Weidong+Wen%2C+Chun+Jason+Xue%2C+and+Qingan+Li%2E+2024%2E+CHESS%3A+Optimizing+LLM+inference+via+channel%2Dwise+thresholding+and+selective+sparsification%2E+In+Proceedings+of+the+2024+Conferen)
12. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. [Efficient memory management for large language model serving with pagedattention](https://md.research.elamur.ai/free/en/papers/kwon-2023-efficient-memory-management-large-language/). In Proceedings of the 29th Symposium on Operating Systems Principles, SOSP ’23, pages 61…
13. Jaeseong Lee, Seung-won Hwang, Aurick Qiao, Daniel F. Campos, Zhewei Yao, and Yuxiong He. 2024. STUN: structured-then-unstructured pruning for scalable moe pruning. CoRR, abs/2409.06211. [find](https://scholar.google.com/scholar?q=Jaeseong+Lee%2C+Seung%2Dwon+Hwang%2C+Aurick+Qiao%2C+Daniel+F%2E+Campos%2C+Zhewei+Yao%2C+and+Yuxiong+He%2E+2024%2E+STUN%3A+structured%2Dthen%2Dunstructured+pruning+for+scalable+moe+pruning%2E+CoRR%2C+abs%2F2409%2E06211%2E)
14. Pingzhi Li, Zhenyu Zhang, Prateek Yadav, Yi-Lin Sung, Yu Cheng, Mohit Bansal, and Tianlong Chen. 2024. Merge, then compress: Demystify efficient smoe with hints from its routing policy. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. O… [find](https://scholar.google.com/scholar?q=Pingzhi+Li%2C+Zhenyu+Zhang%2C+Prateek+Yadav%2C+Yi%2DLin+Sung%2C+Yu+Cheng%2C+Mohit+Bansal%2C+and+Tianlong+Chen%2E+2024%2E+Merge%2C+then+compress%3A+Demystify+efficient+smoe+with+hints+from+its+routing+policy%2E+In+The+Twelfth)
15. Xudong Lu, Qi Liu, Yuhui Xu, Aojun Zhou, Siyuan Huang, Bo Zhang, Junchi Yan, and Hongsheng Li. 2024. [Not all experts are equal: Efficient expert pruning and skipping for mixture-of-experts large language models](https://md.research.elamur.ai/free/en/papers/lu-2024-not-all-experts-are-equal/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguisti…
16. Shi Luohe, Zuchao Li, Lefei Zhang, Baoyuan Qi, Liu Guoming, and Hai Zhao. 2025. [KV-latent: Dimensional-level KV cache reduction with frequency-aware rotary positional embedding](https://md.research.elamur.ai/free/en/papers/luohe-2025-kv-latent-dimensional-level-kv/). In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages…
17. Ziyang Ma, Zuchao Li, Lefei Zhang, Gui-Song Xia, Bo Du, Liangpei Zhang, and Dacheng Tao. 2026. Phase transitions in large language model compression. npj Artificial Intelligence, 2(1):21. [find](https://scholar.google.com/scholar?q=Ziyang+Ma%2C+Zuchao+Li%2C+Lefei+Zhang%2C+Gui%2DSong+Xia%2C+Bo+Du%2C+Liangpei+Zhang%2C+and+Dacheng+Tao%2E+2026%2E+Phase+transitions+in+large+language+model+compression%2E+npj+Artificial+Intelligence%2C+2%281%29%3A21%2E)
18. Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. Can a suit of armor conduct electricity? a new dataset for open book question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2381–2391, Brussels, Belgium. Association fo… [find](https://scholar.google.com/scholar?q=Todor+Mihaylov%2C+Peter+Clark%2C+Tushar+Khot%2C+and+Ashish+Sabharwal%2E+2018%2E+Can+a+suit+of+armor+conduct+electricity%3F+a+new+dataset+for+open+book+question+answering%2E+In+Proceedings+of+the+2018+Conference+on+)
19. Niklas Muennighoff, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Jacob Morrison, Sewon Min, Weijia Shi, Pete Walsh, Oyvind Tafjord, Nathan Lambert, Yuling Gu, Shane Arora, Akshita Bhagia, Dustin Schwenk, David Wadden, Alexander Wettig, Binyuan Hui, Tim Dettmers, Douwe Kiela, and 5 others. 2024. Olmoe: O… [find](https://scholar.google.com/scholar?q=Niklas+Muennighoff%2C+Luca+Soldaini%2C+Dirk+Groeneveld%2C+Kyle+Lo%2C+Jacob+Morrison%2C+Sewon+Min%2C+Weijia+Shi%2C+Pete+Walsh%2C+Oyvind+Tafjord%2C+Nathan+Lambert%2C+Yuling+Gu%2C+Shane+Arora%2C+Akshita+Bhagia%2C+Dustin+Schwenk%2C+)
20. Alexandre Muzio, Alex Sun, and Churan He. 2024. Seer-moe: Sparse expert efficiency through regularization for mixture-of-experts. CoRR, abs/2404.05089. — [arXiv:2404.05089](https://arxiv.org/abs/2404.05089) [find](https://scholar.google.com/scholar?q=Alexandre+Muzio%2C+Alex+Sun%2C+and+Churan+He%2E+2024%2E+Seer%2Dmoe%3A+Sparse+expert+efficiency+through+regularization+for+mixture%2Dof%2Dexperts%2E+CoRR%2C+abs%2F2404%2E05089%2E+%E2%80%94+%5BarXiv%3A2404%2E05089%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2404%2E)
21. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. Winogrande: an adversarial winograd schema challenge at scale. Commun. ACM, 64(9):99–106. [find](https://scholar.google.com/scholar?q=Keisuke+Sakaguchi%2C+Ronan+Le+Bras%2C+Chandra+Bhagavatula%2C+and+Yejin+Choi%2E+2021%2E+Winogrande%3A+an+adversarial+winograd+schema+challenge+at+scale%2E+Commun%2E+ACM%2C+64%289%29%3A99%E2%80%93106%2E)
22. Noam Shazeer. 2020. GLU variants improve transformer. CoRR, abs/2002.05202. [find](https://scholar.google.com/scholar?q=Noam+Shazeer%2E+2020%2E+GLU+variants+improve+transformer%2E+CoRR%2C+abs%2F2002%2E05202%2E)
23. Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc V. Le, Geoffrey E. Hinton, and Jeff Dean. 2017. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2… [find](https://scholar.google.com/scholar?q=Noam+Shazeer%2C+Azalia+Mirhoseini%2C+Krzysztof+Maziarz%2C+Andy+Davis%2C+Quoc+V%2E+Le%2C+Geoffrey+E%2E+Hinton%2C+and+Jeff+Dean%2E+2017%2E+Outrageously+large+neural+networks%3A+The+sparsely%2Dgated+mixture%2Dof%2Dexperts+layer%2E+In)
24. Luohe Shi, Zuchao Li, Lefei Zhang, Baoyuan Qi, Guoming Liu, and Hai Zhao. 2026. Scaling LLM speculative decoding: Non-autoregressive forecasting in large-batch scenarios. In Fortieth AAAI Conference on Artificial Intelligence, Thirty-Eighth Conference on Innovative Applications of Artificial Intelli… [find](https://scholar.google.com/scholar?q=Luohe+Shi%2C+Zuchao+Li%2C+Lefei+Zhang%2C+Baoyuan+Qi%2C+Guoming+Liu%2C+and+Hai+Zhao%2E+2026%2E+Scaling+LLM+speculative+decoding%3A+Non%2Dautoregressive+forecasting+in+large%2Dbatch+scenarios%2E+In+Fortieth+AAAI+Conference+o)
25. Zhibin Wang, Shipeng Li, Yuhang Zhou, Xue Li, Rong Gu, Nguyen Cam-Tu, Chen Tian, and Sheng Zhong. 2024. Revisiting SLO and goodput metrics in LLM serving. CoRR, abs/2410.14257. — [arXiv:2410.14257](https://arxiv.org/abs/2410.14257) [find](https://scholar.google.com/scholar?q=Zhibin+Wang%2C+Shipeng+Li%2C+Yuhang+Zhou%2C+Xue+Li%2C+Rong+Gu%2C+Nguyen+Cam%2DTu%2C+Chen+Tian%2C+and+Sheng+Zhong%2E+2024%2E+Revisiting+SLO+and+goodput+metrics+in+LLM+serving%2E+CoRR%2C+abs%2F2410%2E14257%2E+%E2%80%94+%5BarXiv%3A2410%2E14257%5D%28ht)
26. Samuel Williams, Andrew Waterman, and David Patterson. 2009. [Roofline: an insightful visual performance model for multicore architectures](https://md.research.elamur.ai/free/en/papers/williams-2009-roofline-insightful-visual-performance-model/). Commun. ACM, 52(4):65–76.
27. [Transformers: State-of-the-art natural language processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics.
28. Yanyue Xie, Zhi Zhang, Ding Zhou, Cong Xie, Ziang Song, Xin Liu, Yanzhi Wang, Xue Lin, and An Xu. 2024. Moe-pruner: Pruning mixture-of-experts large language model using the hints from its router. CoRR, abs/2410.12013. — [arXiv:2410.12013](https://arxiv.org/abs/2410.12013) [find](https://scholar.google.com/scholar?q=Yanyue+Xie%2C+Zhi+Zhang%2C+Ding+Zhou%2C+Cong+Xie%2C+Ziang+Song%2C+Xin+Liu%2C+Yanzhi+Wang%2C+Xue+Lin%2C+and+An+Xu%2E+2024%2E+Moe%2Dpruner%3A+Pruning+mixture%2Dof%2Dexperts+large+language+model+using+the+hints+from+its+router%2E+CoR)
29. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, and 22 others. 2024a. Qwen2.5 technical report. CoRR, abs/2412.151… [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Haoran+Wei%2C+Huan+Lin%2C+Jian+Yang%2C+Jianhong+Tu%2C+Jianwei+Zhang%2C+Jianxin+Yang%2C+Jiaxi+Yang%2C+Jin)
30. Cheng Yang, Yang Sui, Jinqi Xiao, Lingyi Huang, Yu Gong, Yuanlin Duan, Wenqi Jia, Miao Yin, Yu Cheng, and Bo Yuan. 2024b. Moe-i2 : Compressing mixture of experts models through inter-expert pruning and intra-expert low-rank decomposition. CoRR, abs/2411.01016. — [arXiv:2411.01016](https://arxiv.org/abs/2411.01016) [find](https://scholar.google.com/scholar?q=Cheng+Yang%2C+Yang+Sui%2C+Jinqi+Xiao%2C+Lingyi+Huang%2C+Yu+Gong%2C+Yuanlin+Duan%2C+Wenqi+Jia%2C+Miao+Yin%2C+Yu+Cheng%2C+and+Bo+Yuan%2E+2024b%2E+Moe%2Di2+%3A+Compressing+mixture+of+experts+models+through+inter%2Dexpert+pruning+an)
31. Haoqi Yang, Yao Yao, Zuchao Li, Baoyuan Qi, Liu Guoming, and Hai Zhao. 2025. XQuant: Achieving ultra-low bit KV cache quantization with cross-layer compression. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 9785–9800, Suzhou, China. Association for… [find](https://scholar.google.com/scholar?q=Haoqi+Yang%2C+Yao+Yao%2C+Zuchao+Li%2C+Baoyuan+Qi%2C+Liu+Guoming%2C+and+Hai+Zhao%2E+2025%2E+XQuant%3A+Achieving+ultra%2Dlow+bit+KV+cache+quantization+with+cross%2Dlayer+compression%2E+In+Proceedings+of+the+2025+Conference+o)
32. Lu Ye, Ze Tao, Yong Huang, and Yang Li. 2024. ChunkAttention: Efficient self-attention with prefix-aware KV cache and two-phase partition. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11608–11620, Bangkok, Thailand. Associa… [find](https://scholar.google.com/scholar?q=Lu+Ye%2C+Ze+Tao%2C+Yong+Huang%2C+and+Yang+Li%2E+2024%2E+ChunkAttention%3A+Efficient+self%2Dattention+with+prefix%2Daware+KV+cache+and+two%2Dphase+partition%2E+In+Proceedings+of+the+62nd+Annual+Meeting+of+the+Association+)
33. Gyeong-In Yu, Joo Seong Jeong, Geon-Woo Kim, Soojeong Kim, and Byung-Gon Chun. 2022. [Orca: A distributed serving system for Transformer-Based generative models](https://md.research.elamur.ai/free/en/papers/yu-2022-orca-distributed-serving-system-transformer/). In 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22), pages 521–538, Carlsbad, CA. USENIX Association.
34. Zihao Zeng, Yibo Miao, Hongcheng Gao, Hao Zhang, and Zhijie Deng. 2024. AdaMoE: Token-adaptive routing with null experts for mixture-of-experts language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 6223–6235, Miami, Florida, USA. Association for Computation… [find](https://scholar.google.com/scholar?q=Zihao+Zeng%2C+Yibo+Miao%2C+Hongcheng+Gao%2C+Hao+Zhang%2C+and+Zhijie+Deng%2E+2024%2E+AdaMoE%3A+Token%2Dadaptive+routing+with+null+experts+for+mixture%2Dof%2Dexperts+language+models%2E+In+Findings+of+the+Association+for+Comp)
35. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, and 3 others. 2020. [find](https://scholar.google.com/scholar?q=Thomas+Wolf%2C+Lysandre+Debut%2C+Victor+Sanh%2C+Julien+Chaumond%2C+Clement+Delangue%2C+Anthony+Moi%2C+Pier%2Dric+Cistac%2C+Tim+Rault%2C+Remi+Louf%2C+Morgan+Funtowicz%2C+Joe+Davison%2C+Sam+Shleifer%2C+Patrick+von+Platen%2C+Clara+)

*Updated 2 November 2026*