---
title: "LBLLM — paper by Song et al. (2026) — summary and ideas"
description: "Running LLMs on limited hardware requires compressing both weights and activations, but going below 4 bits usually costs a… 38 references."
authors: ["Siqing Song", "Chuang Wang", "Yong Lang", "Yi Yang", "Xu-Yao Zhang"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.1640/"
doi: "10.18653/v1/2026.acl-long.1640"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/song-2026-lbllm-lightweight-binarization-large-language/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/song-2026-lbllm-lightweight-binarization-large-language/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/34/) / LBLLM: Lightweight Binarization of Large Language Models via Three-Stage Distillation

# LBLLM: Lightweight Binarization of Large Language Models via Three-Stage Distillation

**Authors:** Siqing Song, Chuang Wang, Yong Lang, [Yi Yang](https://md.research.elamur.ai/free/en/authors/yi-yang/) and [Xu-Yao Zhang](https://md.research.elamur.ai/free/en/authors/xu-yao-zhang/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.1640/) · [PDF](https://aclanthology.org/2026.acl-long.1640.pdf) · [DOI 10.18653/v1/2026.acl-long.1640](https://doi.org/10.18653/v1/2026.acl-long.1640) · [OpenAlex](https://openalex.org/W7166841334)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Running LLMs on limited hardware requires compressing both weights and activations, but going below 4 bits usually costs a lot of accuracy. Existing approaches have drawbacks. Binarization via QAT needs huge GPU budgets. PTQ binarizers lose much quality, and outlier handling often depends on mixed-precision channels or rotation matrices. LBLLM merges the PTQ and QAT approaches into a three-stage pipeline:

1. A PTQ binarizer (BWA) provides the starting model.
2. Layer-wise distillation tunes the binary weights, the group bitmaps and their quantization parameters, while activations stay in full precision.
3. Learnable clipping factors are trained to quantize activations to 4 bits.

Splitting weight and activation training keeps their quantization errors from interfering during straight-through gradient estimation. The result is a W(1+1)A4 model, roughly equivalent to W2A4, with no extra high-precision channels or rotations. The method uses only 0.016B training tokens and a single GPU. It beats prior binarization methods on perplexity, commonsense QA and MMLU, which suggests that extreme low-bit LLMs can be built cheaply.

## Key points
- On LLAMA-2-7B, WikiText2 perplexity is 9.18 for W(1+1)A4 versus 5.47 for FP16. LLAMA-1-7B reaches 9.08 (FP16: 5.68), and LLAMA-3-8B degrades more, to 17.56 (FP16: 6.14).
- The authors claim an improvement of more than 10 perplexity points over earlier PTQ binarizers. They report accuracy close to full-QAT methods at a cost of a few dozen GPU hours for a 7B model.
- Average zero-shot commonsense accuracy is 54.91 on LLAMA-2-7B, against 63.71 for FP16.
- Ablations show two failure modes. Jointly quantizing weights and activations with direct distillation makes perplexity blow up. Initializing with RTN instead of BWA also causes divergence.
- Code is promised at github.com/JimmyCrave/LBLLM after internal approval. The authors note that real speedups still require custom kernels or dedicated hardware.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Chinese Academy of Sciences](https://md.research.elamur.ai/free/en/government/chinese-academy-of-sciences/) — Government body, Beijing, China · 194 works · 23,906 citations
- [Huawei Technologies](https://md.research.elamur.ai/free/en/companies/huawei-technologies/) — Company, Shenzhen, China · 75 works · 12,975 citations
- [Institute of Automation](https://md.research.elamur.ai/free/en/labs/institute-of-automation/) — Research institute, Beijing, China · 70 works · 14,251 citations
- [University of Chinese Academy of Sciences](https://md.research.elamur.ai/free/en/universities/university-of-chinese-academy-of-sciences/) — University, Beijing, China · 147 works · 8,480 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works

## Cite

Cite as: Siqing Song, Chuang Wang, Yong Lang, Yi Yang and Xu-Yao Zhang. 2026. LBLLM: Lightweight Binarization of Large Language Models via Three-Stage Distillation. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{song2026lbllm,
  title = {{LBLLM: Lightweight Binarization of Large Language Models via Three-Stage Distillation}},
  author = {Siqing Song, Chuang Wang, Yong Lang, Yi Yang and Xu-Yao Zhang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1640},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1640}
}
```

- [← previous Layer-wise Probing of wav2vec 2.0 and Whisper for Consonant Cluster Reduction in African American English](https://md.research.elamur.ai/free/en/papers/mojarad-2026-layer-wise-probing-wav2vec-2/)
- [next → LCO: LLM-based Constraint Optimization for Safer Agentic LLMs in Real-world Tasks](https://md.research.elamur.ai/free/en/papers/wan-2026-lco-llm-based-constraint-optimization/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/34/)

## References

38 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman. 2024. [Quarot: outlier-free 4-bit inference in rotated llms](https://md.research.elamur.ai/free/en/papers/ashkboos-2024-quarot-outlier-free-4-bit/). In Proceedings of the 38th International Conference on Neural Information Processing Systems…
2. Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, and 1 others. 2020. [Piqa: Reasoning about physical commonsense in natural language](https://md.research.elamur.ai/free/en/papers/bisk-2019-piqa-reasoning-about-physical-commonsense/). In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 7432–7439.
3. Mengzhao Chen, Wenqi Shao, Peng Xu, Jiahao Wang, Peng Gao, Kaipeng Zhang, and Ping Luo. 2025. [EfficientQAT: Efficient quantization-aware training for large language models](https://md.research.elamur.ai/free/en/papers/chen-2025-efficientqat-efficient-quantization-aware-training/). In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics, pages 10081–10100.
4. Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019. [Boolq: Exploring the surprising difficulty of natural yes/no questions](https://md.research.elamur.ai/free/en/papers/clark-2019-boolq-exploring-surprising-difficulty-natural/). In Proceedings of NAACL-HLT, pages 2924–2936.
5. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. [Think you have solved question answering? try arc, the ai2 reasoning challenge](https://md.research.elamur.ai/free/en/papers/clark-2018-think-have-solved-question-answering/). arXiv preprint arXiv:1803.05457.
6. Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022. Gpt3. int8 (): 8-bit matrix multiplication for transformers at scale. Advances in Neural Information Processing Systems, 35:30318– 30332. [find](https://scholar.google.com/scholar?q=Tim+Dettmers%2C+Mike+Lewis%2C+Younes+Belkada%2C+and+Luke+Zettlemoyer%2E+2022%2E+Gpt3%2E+int8+%28%29%3A+8%2Dbit+matrix+multiplication+for+transformers+at+scale%2E+Advances+in+Neural+Information+Processing+Systems%2C+35%3A30318%E2%80%93)
7. Xin Ding, Xiaoyu Liu, Zhijun Tu, Yun Zhang, Wei Li, Jie Hu, Hanting Chen, Yehui Tang, Zhiwei Xiong, Baoqun Yin, and 1 others. 2024. Cbq: Cross-block quantization for large language models. In Proceedings of the 38th International Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Xin+Ding%2C+Xiaoyu+Liu%2C+Zhijun+Tu%2C+Yun+Zhang%2C+Wei+Li%2C+Jie+Hu%2C+Hanting+Chen%2C+Yehui+Tang%2C+Zhiwei+Xiong%2C+Baoqun+Yin%2C+and+1+others%2E+2024%2E+Cbq%3A+Cross%2Dblock+quantization+for+large+language+models%2E+In+Proceedi)
8. Peijie Dong, Lujun Li, Yuedong Zhong, Dayou Du, Ruibo Fan, Yuhan Chen, Zhenheng Tang, Qiang Wang, Wei Xue, Yike Guo, and 1 others. 2024. Stbllm: Breaking the 1-bit barrier with structured binary llms. In Proceedings of the 38th International Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Peijie+Dong%2C+Lujun+Li%2C+Yuedong+Zhong%2C+Dayou+Du%2C+Ruibo+Fan%2C+Yuhan+Chen%2C+Zhenheng+Tang%2C+Qiang+Wang%2C+Wei+Xue%2C+Yike+Guo%2C+and+1+others%2E+2024%2E+Stbllm%3A+Breaking+the+1%2Dbit+barrier+with+structured+binary+llms%2E)
9. Dayou Du, Yijia Zhang, Shijie Cao, Jiaqi Guo, Ting Cao, Xiaowen Chu, and Ningyi Xu. 2024. Bitdistiller: Unleashing the potential of sub-4-bit llms via self-distillation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 102–116. [find](https://scholar.google.com/scholar?q=Dayou+Du%2C+Yijia+Zhang%2C+Shijie+Cao%2C+Jiaqi+Guo%2C+Ting+Cao%2C+Xiaowen+Chu%2C+and+Ningyi+Xu%2E+2024%2E+Bitdistiller%3A+Unleashing+the+potential+of+sub%2D4%2Dbit+llms+via+self%2Ddistillation%2E+In+Proceedings+of+the+62nd+Ann)
10. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). arXiv preprint arXiv:2009.03300.
11. Wei Huang, Yangdong Liu, Haotong Qin, Ying Li, Shiming Zhang, Xianglong Liu, Michele Magno, and Xiaojuan Qi. 2024. Billm: pushing the limit of post-training quantization for llms. In Proceedings of the 41st International Conference on Machine Learning, pages 20023–20042. [find](https://scholar.google.com/scholar?q=Wei+Huang%2C+Yangdong+Liu%2C+Haotong+Qin%2C+Ying+Li%2C+Shiming+Zhang%2C+Xianglong+Liu%2C+Michele+Magno%2C+and+Xiaojuan+Qi%2E+2024%2E+Billm%3A+pushing+the+limit+of+post%2Dtraining+quantization+for+llms%2E+In+Proceedings+of+th)
12. Hyesung Jeon, Yulhwa Kim, and Jae-Joon Kim. 2025. [L4q: parameter efficient quantization-aware fine-tuning on large language models](https://md.research.elamur.ai/free/en/papers/jeon-2025-l4q-parameter-efficient-quantization-aware/). In Proceedings of the 63rd Annual Meeting of the Association for 35478 9
13. Computational Linguistics (Volume 1: Long Papers), pages 2002–2024. [find](https://scholar.google.com/scholar?q=Computational+Linguistics+%28Volume+1%3A+Long+Papers%29%2C+pages+2002%E2%80%932024%2E)
14. Eldar Kurtic, Alexandre Noll Marques, Shubhra Pandit, Mark Kurtz, and Dan Alistarh. 2025. [“give me bf16 or give me death”?](https://md.research.elamur.ai/free/en/papers/kurtic-2025-give-me-bf16-or-give/) accuracy-performance trade-offs in llm quantization. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 26…
15. Zhiteng Li, Xianglong Yan, Tianao Zhang, Haotong Qin, Dong Xie, Jiang Tian, Linghe Kong, Yulun Zhang, Xiaokang Yang, and 1 others. 2024. Arb-llm: Alternating refined binarizations for large language models. In Proceedings of the 38th International Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Zhiteng+Li%2C+Xianglong+Yan%2C+Tianao+Zhang%2C+Haotong+Qin%2C+Dong+Xie%2C+Jiang+Tian%2C+Linghe+Kong%2C+Yulun+Zhang%2C+Xiaokang+Yang%2C+and+1+others%2E+2024%2E+Arb%2Dllm%3A+Alternating+refined+binarizations+for+large+language+m)
16. Haokun Lin, Haobo Xu, Yichen Wu, Jingzhi Cui, Yingtao Zhang, Linzhan Mou, Linqi Song, Zhenan Sun, and Ying Wei. 2024a. Duquant: Distributing outliers via dual transformation makes stronger quantized llms. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Haokun+Lin%2C+Haobo+Xu%2C+Yichen+Wu%2C+Jingzhi+Cui%2C+Yingtao+Zhang%2C+Linzhan+Mou%2C+Linqi+Song%2C+Zhenan+Sun%2C+and+Ying+Wei%2E+2024a%2E+Duquant%3A+Distributing+outliers+via+dual+transformation+makes+stronger+quantized+l)
17. Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. 2024b. Awq: Activation-aware weight quantization for on-device llm compression and acceleration. Proceedings of Machine Learning and Systems, pages 87– 100. [find](https://scholar.google.com/scholar?q=Ji+Lin%2C+Jiaming+Tang%2C+Haotian+Tang%2C+Shang+Yang%2C+Wei%2DMing+Chen%2C+Wei%2DChen+Wang%2C+Guangxuan+Xiao%2C+Xingyu+Dang%2C+Chuang+Gan%2C+and+Song+Han%2E+2024b%2E+Awq%3A+Activation%2Daware+weight+quantization+for+on%2Ddevice+llm+)
18. Zechun Liu, Barlas Oguz, Changsheng Zhao, Ernie Chang, Pierre Stock, Yashar Mehdad, Yangyang Shi, Raghuraman Krishnamoorthi, and Vikas Chandra. 2024a. [Llm-qat: Data-free quantization aware training for large language models](https://md.research.elamur.ai/free/en/papers/liu-2024-llm-qat-data-free-quantization/). In Findings of the Association for Computational Linguistics: ACL 2024, pag…
19. Zechun Liu, Changsheng Zhao, Igor Fedorov, Bilge Soran, Dhruv Choudhary, Raghuraman Krishnamoorthi, Vikas Chandra, Yuandong Tian, and Tijmen Blankevoort. 2024b. Spinquant–llm quantization with learned rotations. In Proceedings of the 38th International Conference on Neural Information Processing Sys… [find](https://scholar.google.com/scholar?q=Zechun+Liu%2C+Changsheng+Zhao%2C+Igor+Fedorov%2C+Bilge+Soran%2C+Dhruv+Choudhary%2C+Raghuraman+Krishnamoorthi%2C+Vikas+Chandra%2C+Yuandong+Tian%2C+and+Tijmen+Blankevoort%2E+2024b%2E+Spinquant%E2%80%93llm+quantization+with+learned)
20. Liqun Ma, Mingjie Sun, and Zhiqiang Shen. 2024. [Fbi-llm: Scaling up fully binarized llms from scratch via autoregressive distillation](https://md.research.elamur.ai/free/en/papers/ma-2024-fbi-llm-scaling-up-fully/). arXiv preprint arXiv:2407.07093.
21. Mitch Marcus, Grace Kim, Mary Ann Marcinkiewicz, Robert MacIntyre, Ann Bies, Mark Ferguson, Karen Katz, and Britta Schasberger. 1994. [The penn treebank: Annotating predicate argument structure](https://md.research.elamur.ai/free/en/papers/marcus-1994-penn-treebank-annotating-predicate-argument/). In Human Language Technology: Proceedings of a Workshop held at Plainsboro, New Jersey, March 8-11, 1994.
22. Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2016. [Pointer sentinel mixture models](https://md.research.elamur.ai/free/en/papers/merity-2016-pointer-sentinel-mixture-models/). In Proceedings of the 30th International Conference on Neural Information Processing Systems.
23. Xu Ouyang, Tao Ge, Thomas Hartvigsen, Zhisong Zhang, Haitao Mi, and Dong Yu. 2024. Low-bit quantization favors undertrained llms: Scaling laws for quantized llms with 100t training tokens. arXiv preprint arXiv:2411.17691. [find](https://scholar.google.com/scholar?q=Xu+Ouyang%2C+Tao+Ge%2C+Thomas+Hartvigsen%2C+Zhisong+Zhang%2C+Haitao+Mi%2C+and+Dong+Yu%2E+2024%2E+Low%2Dbit+quantization+favors+undertrained+llms%3A+Scaling+laws+for+quantized+llms+with+100t+training+tokens%2E+arXiv+prepr)
24. Jungwoo Park, Taewhoo Lee, Chanwoong Yoon, Hyeon Hwang, and Jaewoo Kang. 2025a. Outlier-safe pre-training for robust 4-bit quantization of large language models. arXiv preprint arXiv:2506.19697. [find](https://scholar.google.com/scholar?q=Jungwoo+Park%2C+Taewhoo+Lee%2C+Chanwoong+Yoon%2C+Hyeon+Hwang%2C+and+Jaewoo+Kang%2E+2025a%2E+Outlier%2Dsafe+pre%2Dtraining+for+robust+4%2Dbit+quantization+of+large+language+models%2E+arXiv+preprint+arXiv%3A2506%2E19697%2E)
25. Seungcheol Park, Jeongin Bae, Beomseok Kwon, Minjun Kim, Byeongwook Kim, Se Jung Kwon, U Kang, and Dongsoo Lee. 2025b. Unifying uniform and binary-coding quantization for accurate compression of large language models. arXiv preprint arXiv:2506.03781. [find](https://scholar.google.com/scholar?q=Seungcheol+Park%2C+Jeongin+Bae%2C+Beomseok+Kwon%2C+Minjun+Kim%2C+Byeongwook+Kim%2C+Se+Jung+Kwon%2C+U+Kang%2C+and+Dongsoo+Lee%2E+2025b%2E+Unifying+uniform+and+binary%2Dcoding+quantization+for+accurate+compression+of+large)
26. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). Journal of machine learning research, 21(140):1–67.
27. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99–106. [find](https://scholar.google.com/scholar?q=Keisuke+Sakaguchi%2C+Ronan+Le+Bras%2C+Chandra+Bhagavatula%2C+and+Yejin+Choi%2E+2021%2E+Winogrande%3A+An+adversarial+winograd+schema+challenge+at+scale%2E+Communications+of+the+ACM%2C+64%289%29%3A99%E2%80%93106%2E)
28. Wenqi Shao, Mengzhao Chen, Zhaoyang Zhang, Peng Xu, Lirui Zhao, Zhiqian Li, Kaipeng Zhang, Peng Gao, Yu Qiao, and Ping Luo. 2016. Omniquant: Omnidirectionally calibrated quantization for large language models. In Proceedings of the 30th International Conference on Neural Information Processing Syste… [find](https://scholar.google.com/scholar?q=Wenqi+Shao%2C+Mengzhao+Chen%2C+Zhaoyang+Zhang%2C+Peng+Xu%2C+Lirui+Zhao%2C+Zhiqian+Li%2C+Kaipeng+Zhang%2C+Peng+Gao%2C+Yu+Qiao%2C+and+Ping+Luo%2E+2016%2E+Omniquant%3A+Omnidirectionally+calibrated+quantization+for+large+languag)
29. Siqing Song, Chuang Wang, Rui-Qi Wang, Yi Yang, and Xu-Yao Zhang. 2025. Achieving binary weight and activation for llms using post-training quantization. In Findings of the Association for Computational Linguistics: ACL 2025, pages 8782–8795. [find](https://scholar.google.com/scholar?q=Siqing+Song%2C+Chuang+Wang%2C+Rui%2DQi+Wang%2C+Yi+Yang%2C+and+Xu%2DYao+Zhang%2E+2025%2E+Achieving+binary+weight+and+activation+for+llms+using+post%2Dtraining+quantization%2E+In+Findings+of+the+Association+for+Computation)
30. Yi Su, Yuechi Zhou, Quantong Qiu, Juntao Li, Qingrong Xia, Ping Li, Xinyu Duan, Zhefeng Wang, and Min Zhang. 2025. Accurate kv cache quantization with outlier tokens tracing. arXiv preprint arXiv:2505.10938. [find](https://scholar.google.com/scholar?q=Yi+Su%2C+Yuechi+Zhou%2C+Quantong+Qiu%2C+Juntao+Li%2C+Qingrong+Xia%2C+Ping+Li%2C+Xinyu+Duan%2C+Zhefeng+Wang%2C+and+Min+Zhang%2E+2025%2E+Accurate+kv+cache+quantization+with+outlier+tokens+tracing%2E+arXiv+preprint+arXiv%3A2505)
31. Wei Tao, Haocheng Lu, Xiaoyang Qu, Bin Zhang, Kai Lu, Jiguang Wan, and Jianzong Wang. 2025. Moqae: Mixed-precision quantization for long-context llm inference via mixture of quantization-aware experts. arXiv preprint arXiv:2506.07533. [find](https://scholar.google.com/scholar?q=Wei+Tao%2C+Haocheng+Lu%2C+Xiaoyang+Qu%2C+Bin+Zhang%2C+Kai+Lu%2C+Jiguang+Wan%2C+and+Jianzong+Wang%2E+2025%2E+Moqae%3A+Mixed%2Dprecision+quantization+for+long%2Dcontext+llm+inference+via+mixture+of+quantization%2Daware+experts)
32. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, and 1 others. 2023a. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). arXiv preprint arXiv:2302.13971.
33. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti 35479 10 Bhosale, and 1 others. 2023b. [Llama 2: Open foundation and fine-tuned chat models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/). arXiv preprint arXiv:2307.09288.
34. Hongyu Wang, Shuming Ma, Li Dong, Shaohan Huang, Huaijie Wang, Lingxiao Ma, Fan Yang, Ruiping Wang, Yi Wu, and Furu Wei. 2023. Bitnet: Scaling 1-bit transformers for large language models. arXiv preprint arXiv:2310.11453. [find](https://scholar.google.com/scholar?q=Hongyu+Wang%2C+Shuming+Ma%2C+Li+Dong%2C+Shaohan+Huang%2C+Huaijie+Wang%2C+Lingxiao+Ma%2C+Fan+Yang%2C+Ruiping+Wang%2C+Yi+Wu%2C+and+Furu+Wei%2E+2023%2E+Bitnet%3A+Scaling+1%2Dbit+transformers+for+large+language+models%2E+arXiv+prepr)
35. Hongyu Wang, Shuming Ma, and Furu Wei. 2024. Bitnet a4. 8: 4-bit activations for 1-bit llms. arXiv preprint arXiv:2411.04965. — [arXiv:2411.04965](https://arxiv.org/abs/2411.04965) [find](https://scholar.google.com/scholar?q=Hongyu+Wang%2C+Shuming+Ma%2C+and+Furu+Wei%2E+2024%2E+Bitnet+a4%2E+8%3A+4%2Dbit+activations+for+1%2Dbit+llms%2E+arXiv+preprint+arXiv%3A2411%2E04965%2E+%E2%80%94+%5BarXiv%3A2411%2E04965%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2411%2E04965%29)
36. Maurice Weber, Daniel Y. Fu, Quentin Anthony, Yonatan Oren, Shane Adams, Anton Alexandrov, Xiaozhong Lyu, Huu Nguyen, Xiaozhe Yao, Virginia Adams, Ben Athiwaratkun, Rahul Chalamala, Kezhen Chen, Max Ryabinin, Tri Dao, Percy Liang, Christopher Ré, Irina Rish, and Ce Zhang. 2024. [Redpajama: an open da…](https://md.research.elamur.ai/free/en/papers/weber-2024-redpajama-open-dataset-training-large/)
37. Ziwei Xiang, Fanhu Zeng, Hongjian Fang, Rui-Qi Wang, Renxing Chen, Yanan Zhu, Yi Chen, Peipei Yang, and Xu-Yao Zhang. 2026. Fine-grained post-training quantization for large vision language models with quantization-aware integrated gradients. arXiv preprint arXiv:2603.17809. — [arXiv:2603.17809](https://arxiv.org/abs/2603.17809) [find](https://scholar.google.com/scholar?q=Ziwei+Xiang%2C+Fanhu+Zeng%2C+Hongjian+Fang%2C+Rui%2DQi+Wang%2C+Renxing+Chen%2C+Yanan+Zhu%2C+Yi+Chen%2C+Peipei+Yang%2C+and+Xu%2DYao+Zhang%2E+2026%2E+Fine%2Dgrained+post%2Dtraining+quantization+for+large+vision+language+models+wit)
38. Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. 2023. Smoothquant: Accurate and efficient post-training quantization for large language models. In International Conference on Machine Learning, pages 38087–38099. PMLR. [find](https://scholar.google.com/scholar?q=Guangxuan+Xiao%2C+Ji+Lin%2C+Mickael+Seznec%2C+Hao+Wu%2C+Julien+Demouth%2C+and+Song+Han%2E+2023%2E+Smoothquant%3A+Accurate+and+efficient+post%2Dtraining+quantization+for+large+language+models%2E+In+International+Conferenc)

*Updated 1 November 2026*