---
title: "Benchmarking Post-Training Quantization of Large Language"
description: "Most post-training quantization (PTQ) methods for LLMs were built for integer formats. 60 references."
authors: ["Manyi Zhang", "Ji-Fu Li", "Zhongao Sun", "Haoli Bai", "Hui-Ling Zhen", "Zhenhua Dong", "Xianzhi Yu"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.1854/"
doi: "10.18653/v1/2026.acl-long.1854"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2026-benchmarking-post-training-quantization-large/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2026-benchmarking-post-training-quantization-large/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/24/) / Benchmarking Post-Training Quantization of Large Language Models under Microscaling Floating Point Formats

# Benchmarking Post-Training Quantization of Large Language Models under Microscaling Floating Point Formats

**Authors:** Manyi Zhang, Ji-Fu Li, Zhongao Sun, [Haoli Bai](https://md.research.elamur.ai/free/en/authors/haoli-bai/), [Hui-Ling Zhen](https://md.research.elamur.ai/free/en/authors/hui-ling-zhen/), [Zhenhua Dong](https://md.research.elamur.ai/free/en/authors/zhenhua-dong/) and [Xianzhi Yu](https://md.research.elamur.ai/free/en/authors/xianzhi-yu/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.1854/) · [PDF](https://aclanthology.org/2026.acl-long.1854.pdf) · [DOI 10.18653/v1/2026.acl-long.1854](https://doi.org/10.18653/v1/2026.acl-long.1854) · [OpenAlex](https://openalex.org/W7166884434)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary

Most post-training quantization (PTQ) methods for LLMs were built for integer formats. It is unclear how they behave under Microscaling Floating-Point (MXFP), a block-scaled format with growing hardware support. The authors benchmark PTQ under MXFP8 and MXFP4, simulated with Microsoft's microxcaling library. They group the methods into four families:

- channel-wise scaling
- error compensation
- rotation
- learnable affine transforms

They test weight-only, weight-activation and KV cache settings on text and multimodal models. Evaluation covers perplexity, commonsense, math reasoning and vision-language benchmarks.

8-bit MXFP turns out to be essentially lossless, while 4-bit still causes clear degradation. Error-compensation and affine methods suit MXFP best, especially when combined, yet plain RTN stays surprisingly competitive. Method rankings transfer well across models. In multimodal models, the language backbone, not the vision encoder, drives sensitivity to quantization.

The authors trace much of the MXFP4 error to power-of-two block scales and show that a simple input pre-scaling step helps. The work argues that MXFP needs format-aware PTQ design rather than reused integer-era assumptions.

## Key points

- **Scope:** more than 7 PTQ algorithms (RTN, SmoothQuant, AWQ, GPTQ, MR-GPTQ, QuaRot, SpinQuant, FlatQuant) and 15 benchmarks.
  - Text benchmarks: WikiText2, PIQA, ARC, MATH-500, AIME24/25.
  - Multimodal benchmarks: OCRBench, MMBench, ChartQA, MMMU and others.
  - Models: Llama-3.1-8B-Instruct, openPangu-Embedded-7B-V1.1, Qwen2.5-VL-7B and openPangu-VL-7B.
- **Formats:** MXFP uses blocks of 32 values sharing a UE8M0 scale. MXFP8 is used in its E4M3 variant, and MXFP4 is E2M1.
- **Degradation regimes:** results are classified as lossless (≤1% drop vs. BF16), fair (1–3%) or risky (≥3%).
  - W8A8 is mostly lossless.
  - W4A8 needs PTQ algorithms to recover accuracy.
  - W4A4 remains hard.
- **Consistency across models:** PTQ recovery trends correlate strongly between models, with an average Pearson correlation of about 0.917.
- **Pre-scaling fix:** multiplying inputs by 3/4 before MXFP4 quantization helps openPangu-Embedded-7B-V1.1 under W4A4.
  - Average accuracy rises from 52.39 to 56.76.
  - WikiText2 PPL falls from 104.42 to 49.33.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Huawei Technologies](https://md.research.elamur.ai/free/en/companies/huawei-technologies/) — Company, Shenzhen, China · 96 works · 15,432 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works · [🧮 Computational Theory and Mathematics](https://md.research.elamur.ai/free/en/shelves/computer-science/computational-theory-and-mathematics/) · 321 works

🧭 Topics (1): [🪶 Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/) · 297 works

## Cite

Cite as: Manyi Zhang, Ji-Fu Li, Zhongao Sun, Haoli Bai, Hui-Ling Zhen, Zhenhua Dong and Xianzhi Yu. 2026. Benchmarking Post-Training Quantization of Large Language Models under Microscaling Floating Point Formats. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhang2026benchmarking,
  title = {{Benchmarking Post-Training Quantization of Large Language Models under Microscaling Floating Point Formats}},
  author = {Manyi Zhang, Ji-Fu Li, Zhongao Sun, Haoli Bai, Hui-Ling Zhen, Zhenhua Dong and Xianzhi Yu},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1854},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1854}
}
```

- [← previous Benchmarking Portuguese Open Information Extraction](https://md.research.elamur.ai/free/en/papers/silva-2026-benchmarking-portuguese-open-information-extraction/)
- [next → Benchmarking Retrieval-Augmented Generation for Scientific Knowledge QA in European Portuguese](https://md.research.elamur.ai/free/en/papers/matos-2026-benchmarking-retrieval-augmented-generation-scientific/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/24/)

## References

60 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Mengzhao Chen, Meng Wu, Hui Jin, Zhihang Yuan, Jing Liu, Chaoyi Zhang, Yunshui Li, Jie Huang, Jin Ma, Zeyue Xue, and 1 others. 2025b. Int vs fp: A comprehensive study of fine-grained low-bit quantization formats. arXiv preprint arXiv:2510.25602. — [arXiv:2510.25602](https://arxiv.org/abs/2510.25602) [find](https://scholar.google.com/scholar?q=Mengzhao+Chen%2C+Meng+Wu%2C+Hui+Jin%2C+Zhihang+Yuan%2C+Jing+Liu%2C+Chaoyi+Zhang%2C+Yunshui+Li%2C+Jie+Huang%2C+Jin+Ma%2C+Zeyue+Xue%2C+and+1+others%2E+2025b%2E+Int+vs+fp%3A+A+comprehensive+study+of+fine%2Dgrained+low%2Dbit+quantizat)
2. Yuxiang Chen, Haocheng Xi, Jun Zhu, and Jianfei Chen. 2025c. Oscillation-reduced mxfp4 training for vision transformers. arXiv preprint arXiv:2502.20853. [find](https://scholar.google.com/scholar?q=Yuxiang+Chen%2C+Haocheng+Xi%2C+Jun+Zhu%2C+and+Jianfei+Chen%2E+2025c%2E+Oscillation%2Dreduced+mxfp4+training+for+vision+transformers%2E+arXiv+preprint+arXiv%3A2502%2E20853%2E)
3. Jack Choquette. 2023. Nvidia hopper h100 gpu: Scaling performance. IEEE Micro, 43(3):9–17. [find](https://scholar.google.com/scholar?q=Jack+Choquette%2E+2023%2E+Nvidia+hopper+h100+gpu%3A+Scaling+performance%2E+IEEE+Micro%2C+43%283%29%3A9%E2%80%9317%2E)
4. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. [Think you have solved question answering? try arc, the ai2 reasoning challenge](https://md.research.elamur.ai/free/en/papers/clark-2018-think-have-solved-question-answering/). arXiv preprint arXiv:1803.05457.
5. Jack Cook, Junxian Guo, Guangxuan Xiao, Yujun Lin, and Song Han. 2025. Four over six: More accurate nvfp4 quantization with adaptive block scaling. arXiv preprint arXiv:2512.02010. — [arXiv:2512.02010](https://arxiv.org/abs/2512.02010) [find](https://scholar.google.com/scholar?q=Jack+Cook%2C+Junxian+Guo%2C+Guangxuan+Xiao%2C+Yujun+Lin%2C+and+Song+Han%2E+2025%2E+Four+over+six%3A+More+accurate+nvfp4+quantization+with+adaptive+block+scaling%2E+arXiv+preprint+arXiv%3A2512%2E02010%2E+%E2%80%94+%5BarXiv%3A2512%2E02010)
6. Pierre V Dantas, Lucas C Cordeiro, and Waldir SS Junior. 2025. A review of state-of-the-art techniques for large language model compression. Complex & Intelligent Systems, 11(9):407. [find](https://scholar.google.com/scholar?q=Pierre+V+Dantas%2C+Lucas+C+Cordeiro%2C+and+Waldir+SS+Junior%2E+2025%2E+A+review+of+state%2Dof%2Dthe%2Dart+techniques+for+large+language+model+compression%2E+Complex+%26+Intelligent+Systems%2C+11%289%29%3A407%2E)
7. Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022. Gpt3. int8 (): 8-bit matrix multiplication for transformers at scale. In NeurIPS, volume 35, pages 30318–30332. [find](https://scholar.google.com/scholar?q=Tim+Dettmers%2C+Mike+Lewis%2C+Younes+Belkada%2C+and+Luke+Zettlemoyer%2E+2022%2E+Gpt3%2E+int8+%28%29%3A+8%2Dbit+matrix+multiplication+for+transformers+at+scale%2E+In+NeurIPS%2C+volume+35%2C+pages+30318%E2%80%9330332%2E)
8. Vage Egiazarian, Roberto L Castro, Denis Kuznedelev, Andrei Panferov, Eldar Kurtic, Shubhra Pandit, Alexandre Marques, Mark Kurtz, Saleh Ashkboos, Torsten Hoefler, and 1 others. 2025. Bridging the gap between promise and performance for microscaling fp4 quantization. arXiv preprint arXiv:2509.23202. [find](https://scholar.google.com/scholar?q=Vage+Egiazarian%2C+Roberto+L+Castro%2C+Denis+Kuznedelev%2C+Andrei+Panferov%2C+Eldar+Kurtic%2C+Shubhra+Pandit%2C+Alexandre+Marques%2C+Mark+Kurtz%2C+Saleh+Ashkboos%2C+Torsten+Hoefler%2C+and+1+others%2E+2025%2E+Bridging+the+gap)
9. Advanced Micro Devices, Inc. 2025. AMD CDNA TM 4 Architecture Whitepaper. White paper, Advanced Micro Devices, Inc. Accessed: 2025-09-24. [find](https://scholar.google.com/scholar?q=Advanced+Micro+Devices%2C+Inc%2E+2025%2E+AMD+CDNA+TM+4+Architecture+Whitepaper%2E+White+paper%2C+Advanced+Micro+Devices%2C+Inc%2E+Accessed%3A+2025%2D09%2D24%2E)
10. Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. 2022. Gptq: Accurate post-training quantization for generative pre-trained transformers. arXiv preprint arXiv:2210.17323. [find](https://scholar.google.com/scholar?q=Elias+Frantar%2C+Saleh+Ashkboos%2C+Torsten+Hoefler%2C+and+Dan+Alistarh%2E+2022%2E+Gptq%3A+Accurate+post%2Dtraining+quantization+for+generative+pre%2Dtrained+transformers%2E+arXiv+preprint+arXiv%3A2210%2E17323%2E)
11. Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman. 2024. [Quarot: Outlier-free 4-bit inference in rotated llms](https://md.research.elamur.ai/free/en/papers/ashkboos-2024-quarot-outlier-free-4-bit/). In NeurIPS, pages 100213–100240.
12. Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, and 1 others. 2025. [Mme: A comprehensive evaluation benchmark for multimodal large language models](https://md.research.elamur.ai/free/en/papers/fu-2023-mme-comprehensive-evaluation-benchmark-multimodal/). In NeurIPS Datasets and Benchmarks Track.
13. Jacob Benesty, Jingdong Chen, Yiteng Huang, and Israel Cohen. 2009. Pearson correlation coefficient. In Noise Reduction in Speech Processing, pages 1–4. [find](https://scholar.google.com/scholar?q=Jacob+Benesty%2C+Jingdong+Chen%2C+Yiteng+Huang%2C+and+Israel+Cohen%2E+2009%2E+Pearson+correlation+coefficient%2E+In+Noise+Reduction+in+Speech+Processing%2C+pages+1%E2%80%934%2E)
14. Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, and 1 others. 2020. [Piqa: Reasoning about physical commonsense in natural language](https://md.research.elamur.ai/free/en/papers/bisk-2019-piqa-reasoning-about-physical-commonsense/). In AAAI, volume 34, pages 7432–7439.
15. Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, and 5 others.… [find](https://scholar.google.com/scholar?q=Leo+Gao%2C+Jonathan+Tow%2C+Baber+Abbasi%2C+Stella+Biderman%2C+Sid+Black%2C+Anthony+DiPofi%2C+Charles+Foster%2C+Laurence+Golding%2C+Jeffrey+Hsu%2C+Alain+Le+Noac%E2%80%99h%2C+Haonan+Li%2C+Kyle+McDonell%2C+Niklas+Muennighoff%2C+Chris+Oci)
16. Hanting Chen, Yasheng Wang, Kai Han, Dong Li, Lin Li, Zhenni Bi, Jinpeng Li, Haoyu Wang, Fei Mi, Mingjian Zhu, and 1 others. 2025a. [Pangu embedded: An efficient dual-system llm reasoner with metacognition](https://md.research.elamur.ai/free/en/papers/chen-2025-pangu-embedded-efficient-dual/). arXiv preprint arXiv:2505.22375.
17. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and 1 others. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783.
18. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. [Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). arXiv preprint arXiv:2501.12948.
19. Ruikang Liu, Yuxuan Sun, Manyi Zhang, Haoli Bai, Xianzhi Yu, Tiezheng Yu, Chun Yuan, and Lu Hou. 2025a. Quantization hurts reasoning? an empirical study on quantized reasoning models. arXiv preprint arXiv:2504.04823. — [arXiv:2504.04823](https://arxiv.org/abs/2504.04823) [find](https://scholar.google.com/scholar?q=Ruikang+Liu%2C+Yuxuan+Sun%2C+Manyi+Zhang%2C+Haoli+Bai%2C+Xianzhi+Yu%2C+Tiezheng+Yu%2C+Chun+Yuan%2C+and+Lu+Hou%2E+2025a%2E+Quantization+hurts+reasoning%3F+an+empirical+study+on+quantized+reasoning+models%2E+arXiv+preprint+a)
20. Nathan Habib, Clémentine Fourrier, Hynek Kydlíček, Thomas Wolf, and Lewis Tunstall. 2023. Lighteval: A lightweight framework for llm evaluation. [find](https://scholar.google.com/scholar?q=Nathan+Habib%2C+Cl%C3%A9mentine+Fourrier%2C+Hynek+Kydl%C3%ADc%CC%8Cek%2C+Thomas+Wolf%2C+and+Lewis+Tunstall%2E+2023%2E+Lighteval%3A+A+lightweight+framework+for+llm+evaluation%2E)
21. Xiaohao Liu, Xiaobo Xia, Manyi Zhang, Ji-Fu Li, Xianzhi Yu, Fei Shen, Xiu Su, See-Kiong Ng, and Tat-Seng Chua. 2026. Freeact: Freeing activations for llm quantization. arXiv preprint arXiv:2603.01776. — [arXiv:2603.01776](https://arxiv.org/abs/2603.01776) [find](https://scholar.google.com/scholar?q=Xiaohao+Liu%2C+Xiaobo+Xia%2C+Manyi+Zhang%2C+Ji%2DFu+Li%2C+Xianzhi+Yu%2C+Fei+Shen%2C+Xiu+Su%2C+See%2DKiong+Ng%2C+and+Tat%2DSeng+Chua%2E+2026%2E+Freeact%3A+Freeing+activations+for+llm+quantization%2E+arXiv+preprint+arXiv%3A2603%2E01776%2E)
22. Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, and Boris Ginsburg. 2024. [Ruler: What’s the real context size of your long-context language models?](https://md.research.elamur.ai/free/en/papers/hsieh-2024-ruler-s-real-context-size/) arXiv preprint arXiv:2404.06654.
23. Xiaohao Liu, Xiaobo Xia, Weixiang Zhao, Manyi Zhang, Xianzhi Yu, Xiu Su, Shuo Yang, See-Kiong Ng, and Tat-Seng Chua. 2025b. L-mtp: Leap multi-token prediction beyond adjacent context for large language models. arXiv preprint arXiv:2505.17505. — [arXiv:2505.17505](https://arxiv.org/abs/2505.17505) [find](https://scholar.google.com/scholar?q=Xiaohao+Liu%2C+Xiaobo+Xia%2C+Weixiang+Zhao%2C+Manyi+Zhang%2C+Xianzhi+Yu%2C+Xiu+Su%2C+Shuo+Yang%2C+See%2DKiong+Ng%2C+and+Tat%2DSeng+Chua%2E+2025b%2E+L%2Dmtp%3A+Leap+multi%2Dtoken+prediction+beyond+adjacent+context+for+large+languag)
24. Wei Huang, Xudong Ma, Haotong Qin, Xingyu Zheng, Chengtao Lv, Hong Chen, Jie Luo, Xiaojuan Qi, Xianglong Liu, and Michele Magno. 2024. How good are low-bit quantized llama3 models? an empirical study. CoRR. [find](https://scholar.google.com/scholar?q=Wei+Huang%2C+Xudong+Ma%2C+Haotong+Qin%2C+Xingyu+Zheng%2C+Chengtao+Lv%2C+Hong+Chen%2C+Jie+Luo%2C+Xiaojuan+Qi%2C+Xianglong+Liu%2C+and+Michele+Magno%2E+2024%2E+How+good+are+low%2Dbit+quantized+llama3+models%3F+an+empirical+study%2E)
25. Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, and 1 others. 2024a. [Mmbench: Is your multi-modal model an all-around player?](https://md.research.elamur.ai/free/en/papers/liu-2023-mmbench-multi-modal-model-all/) In ECCV, pages 216–233.
26. Eldar Kurtic, Alexandre Noll Marques, Shubhra Pandit, Mark Kurtz, and Dan Alistarh. 2025. [“give me bf16 or give me death”?](https://md.research.elamur.ai/free/en/papers/kurtic-2025-give-me-bf16-or-give/) accuracy-performance trade-offs in llm quantization. In ACL, pages 26872–26886.
27. Yuliang Liu, Zhang Li, Mingxin Huang, Biao Yang, Wenwen Yu, Chunyuan Li, Xu-Cheng Yin, Cheng-Lin Liu, Lianwen Jin, and Xiang Bai. 2024b. [Ocrbench: on the hidden mystery of ocr in large multimodal models](https://md.research.elamur.ai/free/en/papers/liu-2023-ocrbench-hidden-mystery-ocr-large/). Science China Information Sciences, 67(12):220102.
28. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. [Efficient memory management for large language model serving with pagedattention](https://md.research.elamur.ai/free/en/papers/kwon-2023-efficient-memory-management-large-language/). In SOSP, pages 611–626.
29. Ji-Fu Li, Manyi Zhang, Xiaobo Xia, Han Bao, Haoli Bai, Zhenhua Dong, and Xianzhi Yu. 2026. Batquant: Outlier-resilient mxfp4 quantization via learnable block-wise optimization. arXiv preprint arXiv:2603.16590. — [arXiv:2603.16590](https://arxiv.org/abs/2603.16590) [find](https://scholar.google.com/scholar?q=Ji%2DFu+Li%2C+Manyi+Zhang%2C+Xiaobo+Xia%2C+Han+Bao%2C+Haoli+Bai%2C+Zhenhua+Dong%2C+and+Xianzhi+Yu%2E+2026%2E+Batquant%3A+Outlier%2Dresilient+mxfp4+quantization+via+learnable+block%2Dwise+optimization%2E+arXiv+preprint+arXiv%3A26)
30. Muyang Li, Yujun Lin, Zhekai Zhang, Tianle Cai, Xiuyu Li, Junxian Guo, Enze Xie, Chenlin Meng, Jun-Yan Zhu, and Song Han. 2024. Svdquant: Absorbing outliers by low-rank components for 4-bit diffusion models. arXiv preprint arXiv:2411.05007. [find](https://scholar.google.com/scholar?q=Muyang+Li%2C+Yujun+Lin%2C+Zhekai+Zhang%2C+Tianle+Cai%2C+Xiuyu+Li%2C+Junxian+Guo%2C+Enze+Xie%2C+Chenlin+Meng%2C+Jun%2DYan+Zhu%2C+and+Song+Han%2E+2024%2E+Svdquant%3A+Absorbing+outliers+by+low%2Drank+components+for+4%2Dbit+diffusion+)
31. Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. [Let’s verify step by step](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/). In ICLR.
32. Zechun Liu, Changsheng Zhao, Igor Fedorov, Bilge Soran, Dhruv Choudhary, Raghuraman Krishnamoorthi, Vikas Chandra, Yuandong Tian, and Tijmen Blankevoort. 2024c. Spinquant: Llm quantization with learned rotations. arXiv preprint arXiv:2405.16406. — [arXiv:2405.16406](https://arxiv.org/abs/2405.16406) [find](https://scholar.google.com/scholar?q=Zechun+Liu%2C+Changsheng+Zhao%2C+Igor+Fedorov%2C+Bilge+Soran%2C+Dhruv+Choudhary%2C+Raghuraman+Krishnamoorthi%2C+Vikas+Chandra%2C+Yuandong+Tian%2C+and+Tijmen+Blankevoort%2E+2024c%2E+Spinquant%3A+Llm+quantization+with+learne)
33. Keyu Lv, Manyi Zhang, Xiaobo Xia, Jingchen Ni, Shannan Yan, Xianzhi Yu, Lu Hou, Chun Yuan, and Haoli Bai. 2026. What makes low-bit quantization-aware training work for reasoning llms? a systematic study. arXiv preprint arXiv:2601.14888. — [arXiv:2601.14888](https://arxiv.org/abs/2601.14888) [find](https://scholar.google.com/scholar?q=Keyu+Lv%2C+Manyi+Zhang%2C+Xiaobo+Xia%2C+Jingchen+Ni%2C+Shannan+Yan%2C+Xianzhi+Yu%2C+Lu+Hou%2C+Chun+Yuan%2C+and+Haoli+Bai%2E+2026%2E+What+makes+low%2Dbit+quantization%2Daware+training+work+for+reasoning+llms%3F+a+systematic+stu)
34. Yuexiao Ma, Huixia Li, Xiawu Zheng, Feng Ling, Xuefeng Xiao, Rui Wang, Shilei Wen, Fei Chao, and Rongrong Ji. 2024. Affinequant: Affine transformation quantization for large language models. arXiv preprint arXiv:2403.12544. — [arXiv:2403.12544](https://arxiv.org/abs/2403.12544) [find](https://scholar.google.com/scholar?q=Yuexiao+Ma%2C+Huixia+Li%2C+Xiawu+Zheng%2C+Feng+Ling%2C+Xuefeng+Xiao%2C+Rui+Wang%2C+Shilei+Wen%2C+Fei+Chao%2C+and+Rongrong+Ji%2E+2024%2E+Affinequant%3A+Affine+transformation+quantization+for+large+language+models%2E+arXiv+pre)
35. Peter Markstein. 2008. The new ieee-754 standard for floating point arithmetic. [find](https://scholar.google.com/scholar?q=Peter+Markstein%2E+2008%2E+The+new+ieee%2D754+standard+for+floating+point+arithmetic%2E)
36. Haokun Lin, Haobo Xu, Yichen Wu, Jingzhi Cui, Yingtao Zhang, Linzhan Mou, Linqi Song, Zhenan Sun, and Ying Wei. 2024a. Duquant: Distributing outliers via dual transformation makes stronger quantized llms. In NeurIPS, pages 87766–87800. [find](https://scholar.google.com/scholar?q=Haokun+Lin%2C+Haobo+Xu%2C+Yichen+Wu%2C+Jingzhi+Cui%2C+Yingtao+Zhang%2C+Linzhan+Mou%2C+Linqi+Song%2C+Zhenan+Sun%2C+and+Ying+Wei%2E+2024a%2E+Duquant%3A+Distributing+outliers+via+dual+transformation+makes+stronger+quantized+l)
37. Ahmed Masry, Xuan Long Do, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. 2022. [Chartqa: A benchmark for question answering about charts with visual and logical reasoning](https://md.research.elamur.ai/free/en/papers/masry-2022-chartqa-benchmark-question-answering-about/). In Findings of ACL, pages 2263– 2279.
38. Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. 2024b. Awq: Activation-aware weight quantization for on-device llm compression and acceleration. Proceedings of Machine Learning and Systems, 6:87–100. [find](https://scholar.google.com/scholar?q=Ji+Lin%2C+Jiaming+Tang%2C+Haotian+Tang%2C+Shang+Yang%2C+Wei%2DMing+Chen%2C+Wei%2DChen+Wang%2C+Guangxuan+Xiao%2C+Xingyu+Dang%2C+Chuang+Gan%2C+and+Song+Han%2E+2024b%2E+Awq%3A+Activation%2Daware+weight+quantization+for+on%2Ddevice+llm+)
39. Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2016. [Pointer sentinel mixture models](https://md.research.elamur.ai/free/en/papers/merity-2016-pointer-sentinel-mixture-models/). arXiv preprint arXiv:1609.07843.
40. Asit Mishra, Dusan Stosic, Simon Layton, and Paulius Micikevicius. 2025. Recipes for pre-training llms with mxfp8. arXiv preprint arXiv:2506.08027. [find](https://scholar.google.com/scholar?q=Asit+Mishra%2C+Dusan+Stosic%2C+Simon+Layton%2C+and+Paulius+Micikevicius%2E+2025%2E+Recipes+for+pre%2Dtraining+llms+with+mxfp8%2E+arXiv+preprint+arXiv%3A2506%2E08027%2E)
41. Markus Nagel, Marios Fournarakis, Rana Ali Amjad, Yelysei Bondarenko, Mart Van Baalen, and Tijmen Blankevoort. 2021. A white paper on neural network quantization. arXiv preprint arXiv:2106.08295. openPangu Team. 2025a. openpangu-embedded-7b-v1.1. — [arXiv:2106.08295](https://arxiv.org/abs/2106.08295) [find](https://scholar.google.com/scholar?q=Markus+Nagel%2C+Marios+Fournarakis%2C+Rana+Ali+Amjad%2C+Yelysei+Bondarenko%2C+Mart+Van+Baalen%2C+and+Tijmen+Blankevoort%2E+2021%2E+A+white+paper+on+neural+network+quantization%2E+arXiv+preprint+arXiv%3A2106%2E08295%2E+open)
42. Huawei openPangu Team. 2025b. openpangu-vl-7b: A multi-model large language model designed and optimized for ascend npus. [find](https://scholar.google.com/scholar?q=Huawei+openPangu+Team%2E+2025b%2E+openpangu%2Dvl%2D7b%3A+A+multi%2Dmodel+large+language+model+designed+and+optimized+for+ascend+npus%2E)
43. Bita Darvish Rouhani, Ritchie Zhao, Ankit More, Mathew Hall, Alireza Khodamoradi, Summer Deng, Dhruv Choudhary, Marius Cornea, Eric Dellinger, Kristof Denolf, and 1 others. 2023. [Microscaling data formats for deep learning](https://md.research.elamur.ai/free/en/papers/rouhani-2023-microscaling-data-formats-deep-learning/). arXiv preprint arXiv:2310.10537.
44. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99–106. [find](https://scholar.google.com/scholar?q=Keisuke+Sakaguchi%2C+Ronan+Le+Bras%2C+Chandra+Bhagavatula%2C+and+Yejin+Choi%2E+2021%2E+Winogrande%3A+An+adversarial+winograd+schema+challenge+at+scale%2E+Communications+of+the+ACM%2C+64%289%29%3A99%E2%80%93106%2E)
45. Wenqi Shao, Mengzhao Chen, Zhaoyang Zhang, Peng Xu, Lirui Zhao, Zhiqian Li, Kaipeng Zhang, Peng Gao, Yu Qiao, and Ping Luo. 2023. [Omniquant: Omnidirectionally calibrated quantization for large language models](https://md.research.elamur.ai/free/en/papers/shao-2023-omniquant-omnidirectionally-calibrated-quantization-large/). arXiv preprint arXiv:2308.13137.
46. Yuantian Shao, Peisong Wang, Yuanteng Chen, Chang Xu, Zhihui Wei, and Jian Cheng. 2025. Block rotation is all you need for mxfp4 quantization. arXiv preprint arXiv:2511.04214. — [arXiv:2511.04214](https://arxiv.org/abs/2511.04214) [find](https://scholar.google.com/scholar?q=Yuantian+Shao%2C+Peisong+Wang%2C+Yuanteng+Chen%2C+Chang+Xu%2C+Zhihui+Wei%2C+and+Jian+Cheng%2E+2025%2E+Block+rotation+is+all+you+need+for+mxfp4+quantization%2E+arXiv+preprint+arXiv%3A2511%2E04214%2E+%E2%80%94+%5BarXiv%3A2511%2E04214%5D%28htt)
47. Amanpreet Singh, Vivek Natarjan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. 2019. Towards vqa models that can read. In CVPR, pages 8317–8326. [find](https://scholar.google.com/scholar?q=Amanpreet+Singh%2C+Vivek+Natarjan%2C+Meet+Shah%2C+Yu+Jiang%2C+Xinlei+Chen%2C+Dhruv+Batra%2C+Devi+Parikh%2C+and+Marcus+Rohrbach%2E+2019%2E+Towards+vqa+models+that+can+read%2E+In+CVPR%2C+pages+8317%E2%80%938326%2E)
48. Yuxuan Sun, Ruikang Liu, Haoli Bai, Han Bao, Kang Zhao, Yuening Li, Jiaxin Hu, Xianzhi Yu, Lu Hou, Chun Yuan, and 1 others. 2024. Flatquant: Flatness matters for llm quantization. arXiv preprint arXiv:2410.09426. — [arXiv:2410.09426](https://arxiv.org/abs/2410.09426) [find](https://scholar.google.com/scholar?q=Yuxuan+Sun%2C+Ruikang+Liu%2C+Haoli+Bai%2C+Han+Bao%2C+Kang+Zhao%2C+Yuening+Li%2C+Jiaxin+Hu%2C+Xianzhi+Yu%2C+Lu+Hou%2C+Chun+Yuan%2C+and+1+others%2E+2024%2E+Flatquant%3A+Flatness+matters+for+llm+quantization%2E+arXiv+preprint+arXiv)
49. Yehui Tang, Yichun Yin, Yaoyuan Wang, Hang Zhou, Yu Pan, Wei Guo, Ziyang Zhang, Miao Rang, Fangcheng Liu, Naifu Zhang, and 1 others. 2025. [Pangu ultra moe: How to train your big moe on ascend npus](https://md.research.elamur.ai/free/en/papers/tang-2025-pangu-ultra-moe-train/). arXiv preprint arXiv:2505.04519.
50. Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. 2024. [Qwen2-vl: Enhancing vision-language model’s perception of the w…](https://md.research.elamur.ai/free/en/papers/wang-2024-qwen2-vl-enhancing-vision/)
51. Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. 2023. Smoothquant: Accurate and efficient post-training quantization for large language models. In ICML, pages 38087– 38099. [find](https://scholar.google.com/scholar?q=Guangxuan+Xiao%2C+Ji+Lin%2C+Mickael+Seznec%2C+Hao+Wu%2C+Julien+Demouth%2C+and+Song+Han%2E+2023%2E+Smoothquant%3A+Accurate+and+efficient+post%2Dtraining+quantization+for+large+language+models%2E+In+ICML%2C+pages+38087%E2%80%93+3809)
52. Yufei Xue, Yushi Huang, Jiawei Shao, and Jun Zhang. 2025. Vlmq: Efficient post-training quantization for large vision-language models via hessian augmentation. arXiv preprint arXiv:2508.03351. — [arXiv:2508.03351](https://arxiv.org/abs/2508.03351) [find](https://scholar.google.com/scholar?q=Yufei+Xue%2C+Yushi+Huang%2C+Jiawei+Shao%2C+and+Jun+Zhang%2E+2025%2E+Vlmq%3A+Efficient+post%2Dtraining+quantization+for+large+vision%2Dlanguage+models+via+hessian+augmentation%2E+arXiv+preprint+arXiv%3A2508%2E03351%2E+%E2%80%94+%5BarXi)
53. JiangYong Yu, Sifan Zhou, Dawei Yang, Shuo Wang, Shuoyu Li, Xing Hu, Chen Xu, Zukang Xu, Changyong Shu, and Zhihang Yuan. 2025. Mquant: Unleashing the inference potential of multimodal large language models via full static quantization. arXiv preprint arXiv:2502.00425. — [arXiv:2502.00425](https://arxiv.org/abs/2502.00425) [find](https://scholar.google.com/scholar?q=JiangYong+Yu%2C+Sifan+Zhou%2C+Dawei+Yang%2C+Shuo+Wang%2C+Shuoyu+Li%2C+Xing+Hu%2C+Chen+Xu%2C+Zukang+Xu%2C+Changyong+Shu%2C+and+Zhihang+Yuan%2E+2025%2E+Mquant%3A+Unleashing+the+inference+potential+of+multimodal+large+language+)
54. Zhihang Yuan, Yuzhang Shang, Yang Zhou, Zhen Dong, Zhe Zhou, Chenhao Xue, Bingzhe Wu, Zhikai Li, Qingyi Gu, Yong Jae Lee, and 1 others. 2024. [Llm inference unveiled: Survey and roofline model insights](https://md.research.elamur.ai/free/en/papers/yuan-2024-llm-inference-unveiled-survey-roofline/). arXiv preprint arXiv:2402.16363.
55. Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, and 1 others. 2024. [Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi](https://md.research.elamur.ai/free/en/papers/yue-2023-mmmu-massive-multi-discipline-multimodal/). In CVPR, pages 9556–9567.
56. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830. [find](https://scholar.google.com/scholar?q=Rowan+Zellers%2C+Ari+Holtzman%2C+Yonatan+Bisk%2C+Ali+Farhadi%2C+and+Yejin+Choi%2E+2019%2E+Hellaswag%3A+Can+a+machine+really+finish+your+sentence%3F+arXiv+preprint+arXiv%3A1905%2E07830%2E)
57. Kaichen Zhang, Bo Li, Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Shuai Liu, Yuanhan Zhang, Jingkang Yang, Chunyuan Li, and 1 others. 2025. [Lmms-eval: Reality check on the evaluation of large multimodal models](https://md.research.elamur.ai/free/en/papers/zhang-2025-lmms-eval-reality-check-evaluation/). In Findings of NAACL, pages 881–916.
58. Ajay Tirumala and Raymond Wong. 2024. Nvidia blackwell platform: Advancing generative ai and accelerated computing. In HCS, pages 1–33. [find](https://scholar.google.com/scholar?q=Ajay+Tirumala+and+Raymond+Wong%2E+2024%2E+Nvidia+blackwell+platform%3A+Advancing+generative+ai+and+accelerated+computing%2E+In+HCS%2C+pages+1%E2%80%9333%2E)
59. Albert Tseng, Tao Yu, and Youngsuk Park. 2025. Training LLMs with MXFP4. In AISTATS. [find](https://scholar.google.com/scholar?q=Albert+Tseng%2C+Tao+Yu%2C+and+Youngsuk+Park%2E+2025%2E+Training+LLMs+with+MXFP4%2E+In+AISTATS%2E)
60. Hongyu Wang, Shuming Ma, Li Dong, Shaohan Huang, Huaijie Wang, Lingxiao Ma, Fan Yang, Ruiping Wang, Yi Wu, and Furu Wei. 2023. Bitnet: Scaling 1-bit transformers for large language models. arXiv preprint arXiv:2310.11453. [find](https://scholar.google.com/scholar?q=Hongyu+Wang%2C+Shuming+Ma%2C+Li+Dong%2C+Shaohan+Huang%2C+Huaijie+Wang%2C+Lingxiao+Ma%2C+Fan+Yang%2C+Ruiping+Wang%2C+Yi+Wu%2C+and+Furu+Wei%2E+2023%2E+Bitnet%3A+Scaling+1%2Dbit+transformers+for+large+language+models%2E+arXiv+prepr)

*Updated 2 November 2026*