---
title: "Expert Calibration Lens for Pruning Mixture of Experts (2026)"
description: "The authors present Expert Calibration Lens, a demo system built for gpt-oss-20b. 18 references."
authors: ["Luis Frentzen Salim", "Chia-Chun Wu", "Tran Van Nhiem", "Lun-Wei Ku", "Yung-Hui Li"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)"
source: "https://aclanthology.org/2026.acl-demo.72/"
doi: "10.18653/v1/2026.acl-demo.72"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/salim-2026-expert-calibration-lens-pruning-mixture/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/salim-2026-expert-calibration-lens-pruning-mixture/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/32/) / Expert Calibration Lens for Pruning Mixture of Experts

# Expert Calibration Lens for Pruning Mixture of Experts

**Authors:** [Luis Frentzen Salim](https://md.research.elamur.ai/free/en/authors/luis-frentzen-salim/), Chia-Chun Wu, Tran Van Nhiem, [Lun-Wei Ku](https://md.research.elamur.ai/free/en/authors/lun-wei-ku/) and [Yung-Hui Li](https://md.research.elamur.ai/free/en/authors/yung-hui-li/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-demo.72/) · [PDF](https://aclanthology.org/2026.acl-demo.72.pdf) · [DOI 10.18653/v1/2026.acl-demo.72](https://doi.org/10.18653/v1/2026.acl-demo.72) · [OpenAlex](https://openalex.org/W7166860051)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Expert pruning makes Mixture-of-Experts (MoE) models cheaper to deploy, but its results depend heavily on the calibration set. The effects of changing that set are hard to predict: adding more samples may not help, and data in another language can work about as well. Normally each such question requires a full pruning run. The authors present Expert Calibration Lens, a demo system built for gpt-oss-20b. It hooks into each MoE layer and records dense router probabilities, plus an optional "feature energy" estimate of how much each expert contributes. It then compares datasets through these activation statistics in a web interface, so no pruning is needed. Case studies use C4, MMLU ProX and a proprietary Traditional Chinese FoxBrain corpus. Evaluation runs on MMLU and MathQA through LM Evaluation Harness. The analysis shows that the set of selected experts stabilizes after a few dozen sequences. It also shows that Chinese and English calibration choose largely the same experts, and the pruned models score similarly downstream. Practitioners can therefore check calibration choices cheaply before committing to expensive pruning runs.

## Key points
- **Metrics:** The tool computes metrics for single datasets (sparsity, variance, effective number of experts via inverse participation ratio, Shannon entropy). It also compares pairs of datasets (Jensen-Shannon divergence, Jaccard overlap of top-r experts, Pearson correlation of feature energy, and asymmetric retained mass A→B).
- **Calibration size:** Jaccard similarity of the retained experts converges after roughly 32–64 sequences. Larger calibration sets brought no consistent gains (tested with 24 retained experts).
- **Cross-lingual calibration:** Across 7 MMLU ProX languages (ar, de, en, fr, ja, ru, zh), calibrating in another language performed comparably to calibrating in the evaluation language.
- **Downstream scores (Table 1):** The unpruned baseline scores 0.365 MathQA / 0.563 MMLU. After pruning, C4 calibration gives 0.3889 / 0.617, the private Chinese set 0.405 / 0.611, and a MATH-derived set 0.408 / 0.585.
- **Availability and scope:** The system is part of the FoxBrain project (Hon Hai) and is currently proprietary. It supports only the GPT-OSS architecture, and the public demo is limited to three precomputed activation sets.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Institute of Information Science, Academia Sinica](https://md.research.elamur.ai/free/en/labs/institute-of-information-science-academia-sinica/) — Research institute, Taipei, Taiwan · 8 works · 51 citations
- [National Taiwan University of Science and Technology](https://md.research.elamur.ai/free/en/universities/national-taiwan-university-of-science-and-technology/) — University, Taipei, Taiwan · 4 works · 6 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

🧭 Topics (1): [🪶 Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/) · 297 works

## Cite

Cite as: Luis Frentzen Salim, Chia-Chun Wu, Tran Van Nhiem, Lun-Wei Ku and Yung-Hui Li. 2026. Expert Calibration Lens for Pruning Mixture of Experts. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{salim2026expert,
  title = {{Expert Calibration Lens for Pruning Mixture of Experts}},
  author = {Luis Frentzen Salim, Chia-Chun Wu, Tran {Van Nhiem}, Lun-Wei Ku and Yung-Hui Li},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-demo.72},
  url = {https://doi.org/10.18653/v1/2026.acl-demo.72}
}
```

- [← previous Experiments or Outcomes? Probing Scientific Feasibility in Large Language Models](https://md.research.elamur.ai/free/en/papers/mohammadi-2026-experiments-or-outcomes-probing-scientific/)
- [next → Expert-Guided Schema-Based Structured Extraction from CONSORT Diagrams Using Vision-Language Models](https://md.research.elamur.ai/free/en/papers/stachura-2026-expert-guided-schema-based-structured/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/32/)

## References

18 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Aida Amini, Saadia Gabriel, Shanchuan Lin, Rik Koncel-Kedziorski, Yejin Choi, and Hannaneh Hajishirzi. 2019. [MathQA: Towards interpretable math word problem solving with operation-based formalisms](https://md.research.elamur.ai/free/en/papers/amini-2019-mathqa-interpretable-math-word-problem/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computation…
2. Lucas Bandarkar, Chenyuan Yang, Mohsen Fayyaz, Junlin Hu, and Nanyun Peng. 2025. Multilingual routing in mixture-of-experts. In The Fourteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Lucas+Bandarkar%2C+Chenyuan+Yang%2C+Mohsen+Fayyaz%2C+Junlin+Hu%2C+and+Nanyun+Peng%2E+2025%2E+Multilingual+routing+in+mixture%2Dof%2Dexperts%2E+In+The+Fourteenth+International+Conference+on+Learning+Representations%2E)
3. Tianyu Chen, Shaohan Huang, Yuan Xie, Binxing Jiao, Daxin Jiang, Haoyi Zhou, Jianxin Li, and Furu Wei. 2022. Task-specific expert pruning for sparse mixture-of-experts. Preprint, arXiv:2206.00277. — [arXiv:2206.00277](https://arxiv.org/abs/2206.00277) [find](https://scholar.google.com/scholar?q=Tianyu+Chen%2C+Shaohan+Huang%2C+Yuan+Xie%2C+Binxing+Jiao%2C+Daxin+Jiang%2C+Haoyi+Zhou%2C+Jianxin+Li%2C+and+Furu+Wei%2E+2022%2E+Task%2Dspecific+expert+pruning+for+sparse+mixture%2Dof%2Dexperts%2E+Preprint%2C+arXiv%3A2206%2E00277%2E+%E2%80%94+%5B)
4. Damai Dai, Chengqi Deng, Chenggang Zhao, R. X. Xu, Huazuo Gao, Deli Chen, Jiashi Li, Wangding Zeng, Xingkai Yu, Y. Wu, Zhenda Xie, Y. K. Li, Panpan Huang, Fuli Luo, Chong Ruan, Zhifang Sui, and Wenfeng Liang. 2024. Deepseekmoe: Towards ultimate expert specialization in mixture-of-experts language mo… [find](https://scholar.google.com/scholar?q=Damai+Dai%2C+Chengqi+Deng%2C+Chenggang+Zhao%2C+R%2E+X%2E+Xu%2C+Huazuo+Gao%2C+Deli+Chen%2C+Jiashi+Li%2C+Wangding+Zeng%2C+Xingkai+Yu%2C+Y%2E+Wu%2C+Zhenda+Xie%2C+Y%2E+K%2E+Li%2C+Panpan+Huang%2C+Fuli+Luo%2C+Chong+Ruan%2C+Zhifang+Sui%2C+and+Wenfen)
5. Jesse Dodge, Maarten Sap, Ana Marasović, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Margaret Mitchell, and Matt Gardner. 2021. Documenting large webtext corpora: A case study on the colossal clean crawled corpus. Preprint, arXiv:2104.08758. [find](https://scholar.google.com/scholar?q=Jesse+Dodge%2C+Maarten+Sap%2C+Ana+Marasovic%CC%81%2C+William+Agnew%2C+Gabriel+Ilharco%2C+Dirk+Groeneveld%2C+Margaret+Mitchell%2C+and+Matt+Gardner%2E+2021%2E+Documenting+large+webtext+corpora%3A+A+case+study+on+the+colossal+cl)
6. Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, and 5 others.… [find](https://scholar.google.com/scholar?q=Leo+Gao%2C+Jonathan+Tow%2C+Baber+Abbasi%2C+Stella+Biderman%2C+Sid+Black%2C+Anthony+DiPofi%2C+Charles+Foster%2C+Laurence+Golding%2C+Jeffrey+Hsu%2C+Alain+Le+Noac%E2%80%99h%2C+Haonan+Li%2C+Kyle+McDonell%2C+Niklas+Muennighoff%2C+Chris+Oci)
7. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. [Measuring Massive Multitask Language Understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). In The Ninth International Conference on Learning Representations (ICLR). ArXiv:2009.03300.
8. Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock,… [find](https://scholar.google.com/scholar?q=Albert+Q%2E+Jiang%2C+Alexandre+Sablayrolles%2C+Antoine+Roux%2C+Arthur+Mensch%2C+Blanche+Savary%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Emma+Bou+Hanna%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guill)
9. Slimmoe: Structured compression of large moe models via expert slimming and distillation. Preprint, arXiv:2506.18349. [find](https://scholar.google.com/scholar?q=Slimmoe%3A+Structured+compression+of+large+moe+models+via+expert+slimming+and+distillation%2E+Preprint%2C+arXiv%3A2506%2E18349%2E)
10. Neel Nanda and Joseph Bloom. 2022. Transformerlens. [https://github.com/TransformerLensOrg/](https://github.com/TransformerLensOrg/) TransformerLens. [↗](https://github.com/TransformerLensOrg/)
11. Samyam Rajbhandari, Conglong Li, Zhewei Yao, Minjia Zhang, Reza Yazdani Aminabadi, Ammar Ahmad Awan, Jeff Rasley, and Yuxiong He. 2022. [Deepspeed-moe: Advancing mixture-of-experts inference and training to power next-generation ai scale](https://md.research.elamur.ai/free/en/papers/rajbhandari-2022-deepspeed-moe-advancing-mixture-experts/). Preprint, arXiv:2201.05596.
12. Carlos Riquelme, Joan Puigcerver, Basil Mustafa, Maxim Neumann, Rodolphe Jenatton, André Susano Pinto, Daniel Keysers, and Neil Houlsby. 2021. [Scaling vision with sparse mixture of experts](https://md.research.elamur.ai/free/en/papers/riquelme-2021-scaling-vision-sparse-mixture-experts/). Preprint, arXiv:2106.05974.
13. Luis Frentzen Salim, Lun-Wei Ku, and Hsing-Kuo Kenneth Pao. 2026. Positional cognitive specialization: Where do llms learn to comprehend and speak your language? Proceedings of the AAAI Conference on Artificial Intelligence, 40(39):32875–32883. [find](https://scholar.google.com/scholar?q=Luis+Frentzen+Salim%2C+Lun%2DWei+Ku%2C+and+Hsing%2DKuo+Kenneth+Pao%2E+2026%2E+Positional+cognitive+specialization%3A+Where+do+llms+learn+to+comprehend+and+speak+your+language%3F+Proceedings+of+the+AAAI+Conference+on+)
14. Ian Tenney, James Wexler, Jasmijn Bastings, Tolga Bolukbasi, Andy Coenen, Sebastian Gehrmann, Ellen Jiang, Mahima Pushkarna, Carey Radebaugh, Emily Reif, and Ann Yuan. 2020. The language interpretability tool: Extensible, interactive visualizations and analysis for nlp models. Preprint, arXiv:2008.0… [find](https://scholar.google.com/scholar?q=Ian+Tenney%2C+James+Wexler%2C+Jasmijn+Bastings%2C+Tolga+Bolukbasi%2C+Andy+Coenen%2C+Sebastian+Gehrmann%2C+Ellen+Jiang%2C+Mahima+Pushkarna%2C+Carey+Radebaugh%2C+Emily+Reif%2C+and+Ann+Yuan%2E+2020%2E+The+language+interpretabil)
15. Igor Tufanov, Karen Hambardzumyan, Javier Ferrando, and Elena Voita. 2024. LM transparency tool: Interactive tool for analyzing transformer language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), pages 51–60, Bang… [find](https://scholar.google.com/scholar?q=Igor+Tufanov%2C+Karen+Hambardzumyan%2C+Javier+Ferrando%2C+and+Elena+Voita%2E+2024%2E+LM+transparency+tool%3A+Interactive+tool+for+analyzing+transformer+language+models%2E+In+Proceedings+of+the+62nd+Annual+Meeting+o)
16. Yanyue Xie, Zhi Zhang, Ding Zhou, Cong Xie, Ziang Song, Xin Liu, Yanzhi Wang, Xue Lin, and An Xu. 2024. Moe-pruner: Pruning mixture-of-experts large language model using the hints from its router. Preprint, arXiv:2410.12013. — [arXiv:2410.12013](https://arxiv.org/abs/2410.12013) [find](https://scholar.google.com/scholar?q=Yanyue+Xie%2C+Zhi+Zhang%2C+Ding+Zhou%2C+Cong+Xie%2C+Ziang+Song%2C+Xin+Liu%2C+Yanzhi+Wang%2C+Xue+Lin%2C+and+An+Xu%2E+2024%2E+Moe%2Dpruner%3A+Pruning+mixture%2Dof%2Dexperts+large+language+model+using+the+hints+from+its+router%2E+Pre)
17. Weihao Xuan, Rui Yang, Heli Qi, Qingcheng Zeng, Yunze Xiao, Aosong Feng, Dairui Liu, Yun Xing, Junjue Wang, Fan Gao, Jinghui Lu, Yuang Jiang, Huitao Li, Xin Li, Kunyu Yu, Ruihai Dong, Shangding Gu, Yuekang Li, Xiaofei Xie, and 13 others. 2025. [Mmlu-prox: A multilingual benchmark for advanced large l…](https://md.research.elamur.ai/free/en/papers/xuan-2025-mmlu-prox-multilingual-benchmark-advanced/)
18. Zichong Li, Chen Liang, Zixuan Zhang, Ilgee Hong, Young Jin Kim, Weizhu Chen, and Tuo Zhao. 2025. [find](https://scholar.google.com/scholar?q=Zichong+Li%2C+Chen+Liang%2C+Zixuan+Zhang%2C+Ilgee+Hong%2C+Young+Jin+Kim%2C+Weizhu+Chen%2C+and+Tuo+Zhao%2E+2025%2E)

*Updated 2 November 2026*