---
title: "pQuant — paper by Zhang and Liu (2026) — summary and ideas"
description: "Very low-bit LLMs (below 2 bits per weight) suit edge devices, but models trained from scratch at 1 bit still trail FP16 accuracy… 80 references."
authors: ["Wenzheng Zhang", "Bingzheng Liu"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.1566/"
doi: "10.18653/v1/2026.findings-acl.1566"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2026-pquant-effective-low-bit-language/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2026-pquant-effective-low-bit-language/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/45/) / pQuant: Towards Effective Low-Bit Language Models via Decoupled Linear Quantization-Aware Training

# pQuant: Towards Effective Low-Bit Language Models via Decoupled Linear Quantization-Aware Training

**Authors:** [Wenzheng Zhang](https://md.research.elamur.ai/free/en/authors/wenzheng-zhang/) and Bingzheng Liu · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.1566/) · [PDF](https://aclanthology.org/2026.findings-acl.1566.pdf) · [DOI 10.18653/v1/2026.findings-acl.1566](https://doi.org/10.18653/v1/2026.findings-acl.1566) · [OpenAlex](https://openalex.org/W7166802776)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Very low-bit LLMs (below 2 bits per weight) suit edge devices, but models trained from scratch at 1 bit still trail FP16 accuracy and scale poorly. The authors trace this to what they call parameter democratization. They measure weight sensitivity with an Optimal Brain Surgeon–style metric. In FP16 LLaMA-3, a few weights stand out as highly sensitive, but in 1-bit BitNet the sensitivity map is almost flat.

pQuant, a QAT-Scratch method, restores this differentiation. Attention projections stay fully 1-bit. Each FFN linear layer is split into a large 1-bit branch and a small 8-bit branch. Learnable feature scaling pushes the most important weights into the 8-bit branch. That branch can be extended into several 8-bit experts, with a top-1 router choosing one per token, so capacity grows while inference cost stays nearly flat.

At 1.3B parameters, pQuant reaches lower WikiText2 perplexity than both 1-bit and 2-bit baselines. It also runs faster than 2-bit and FP16 models. This makes sub-2-bit LLMs more practical for deployment.

## Key points
- On a 1.3B model (WikiText2 PPL), pQuant scores 17.2 at about 1.35 bits per weight. The baselines are:
  - BitNet: 21.8 (1-bit)
  - PTQ1.61: 39.6
  - OmniQuant: 42.4 (2-bit)
  - BiLLM: 69.9
- The reported gains are:
  - 32.0% lower perplexity than the strongest 1-bit baseline.
  - 18.2% higher throughput than 2-bit models while beating them in accuracy.
  - More than 2× the throughput of FP16 models at comparable quality.
- At a matched 1.3B total size, pQuant uses N=8 experts with 926M activated parameters. It reaches PPL 16.8 versus 16.8–16.9 for BitNet1.58 and runs 1.6× faster. FP16 LLaMA-2 gets 14.4 but needs 2.64G of memory, compared with 0.98G for pQuant.
- In the ablations:
  - Perplexity improves steadily as the number of 8-bit branches N grows, and N=4 already beats the 2-bit baseline.
  - Simply keeping 8% of BitNet weights in high precision does worse than pQuant's roughly 5%.
- The authors note two limitations: training cost is higher than for PTQ or standard QAT, and memory use rises when N≥4.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Peking University](https://md.research.elamur.ai/free/en/universities/peking-university/) — University, Beijing, China · 315 works · 43,267 citations
- [Fudan University](https://md.research.elamur.ai/free/en/universities/fudan-university/) — University, Shanghai, China · 202 works · 22,509 citations

📚 Shelves (7): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works · [⚙️ Engineering](https://md.research.elamur.ai/free/en/shelves/engineering/) · 188 works · [⚙️ Electrical and Electronic Engineering](https://md.research.elamur.ai/free/en/shelves/engineering/electrical-and-electronic-engineering/) · 33 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works

## Cite

Cite as: Wenzheng Zhang and Bingzheng Liu. 2026. pQuant: Towards Effective Low-Bit Language Models via Decoupled Linear Quantization-Aware Training. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhang2026pquant,
  title = {{pQuant: Towards Effective Low-Bit Language Models via Decoupled Linear Quantization-Aware Training}},
  author = {Wenzheng Zhang and Bingzheng Liu},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1566},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1566}
}
```

- [← previous Post Hoc Agentic Refinement for Improving Precision in Multilingual Clinical Text De-identification](https://md.research.elamur.ai/free/en/papers/xu-2026-post-hoc-agentic-refinement-improving/)
- [next → PRA-RAG: Provably Robust Aggregation in Retrieval-Augmented Generation against Retrieval Corruption](https://md.research.elamur.ai/free/en/papers/tan-2026-pra-rag-provably-robust-aggregation/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/45/)

## References

80 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/).
2. Saleh Ashkboos, Amirkeivan Mohtashami, Maximilian L Croci, Bo Li, Pashmina Cameron, Martin Jaggi, Dan Alistarh, Torsten Hoefler, and James Hensman. 2024. [Quarot: Outlier-free 4-bit inference in rotated llms](https://md.research.elamur.ai/free/en/papers/ashkboos-2024-quarot-outlier-free-4-bit/). Advances in Neural Information Processing Systems, 37:100213–100240.
3. Yoshua Bengio, Nicholas Léonard, and Aaron Courville. 2013. Estimating or propagating gradients through stochastic neurons for conditional computation. [find](https://scholar.google.com/scholar?q=Yoshua+Bengio%2C+Nicholas+L%C3%A9onard%2C+and+Aaron+Courville%2E+2013%2E+Estimating+or+propagating+gradients+through+stochastic+neurons+for+conditional+computation%2E)
4. Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, and 1 others. 2020. [Piqa: Reasoning about physical commonsense in natural language](https://md.research.elamur.ai/free/en/papers/bisk-2019-piqa-reasoning-about-physical-commonsense/).
5. Yelysei Bondarenko, Markus Nagel, and Tijmen Blankevoort. 2021. Understanding and overcoming the challenges of efficient transformer quantization. [find](https://scholar.google.com/scholar?q=Yelysei+Bondarenko%2C+Markus+Nagel%2C+and+Tijmen+Blankevoort%2E+2021%2E+Understanding+and+overcoming+the+challenges+of+efficient+transformer+quantization%2E)
6. Jerry Chee, Yaohui Cai, Volodymyr Kuleshov, and Christopher M De Sa. 2024. Quip: 2-bit quantization of large language models with guarantees. [find](https://scholar.google.com/scholar?q=Jerry+Chee%2C+Yaohui+Cai%2C+Volodymyr+Kuleshov%2C+and+Christopher+M+De+Sa%2E+2024%2E+Quip%3A+2%2Dbit+quantization+of+large+language+models+with+guarantees%2E)
7. Mengzhao Chen, Wenqi Shao, Peng Xu, Jiahao Wang, Peng Gao, Kaipeng Zhang, and Ping Luo. 2024. [Efficientqat: Efficient quantization-aware training for large language models](https://md.research.elamur.ai/free/en/papers/chen-2025-efficientqat-efficient-quantization-aware-training/).
8. Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019. [Boolq: Exploring the surprising difficulty of natural yes/no questions](https://md.research.elamur.ai/free/en/papers/clark-2019-boolq-exploring-surprising-difficulty-natural/).
9. Arman Cohan, Franck Dernoncourt, Doo Soon Kim, Trung Bui, Seokhwan Kim, Walter Chang, and Nazli Goharian. 2018. [A discourse-aware attention model for abstractive summarization of long documents](https://md.research.elamur.ai/free/en/papers/cohan-2018-discourse-aware-attention-model-abstractive/). In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational…
10. Damai Dai, Chengqi Deng, Chenggang Zhao, RX Xu, Huazuo Gao, Deli Chen, Jiashi Li, Wangding Zeng, Xingkai Yu, Y Wu, and 1 others. 2024. Deepseekmoe: Towards ultimate expert specialization in mixture-of-experts language models. [find](https://scholar.google.com/scholar?q=Damai+Dai%2C+Chengqi+Deng%2C+Chenggang+Zhao%2C+RX+Xu%2C+Huazuo+Gao%2C+Deli+Chen%2C+Jiashi+Li%2C+Wangding+Zeng%2C+Xingkai+Yu%2C+Y+Wu%2C+and+1+others%2E+2024%2E+Deepseekmoe%3A+Towards+ultimate+expert+specialization+in+mixture%2Dof)
11. Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022. Gpt3. int8 (): 8-bit matrix multiplication for transformers at scale. Advances in Neural Information Processing Systems, 35:30318– 30332. [find](https://scholar.google.com/scholar?q=Tim+Dettmers%2C+Mike+Lewis%2C+Younes+Belkada%2C+and+Luke+Zettlemoyer%2E+2022%2E+Gpt3%2E+int8+%28%29%3A+8%2Dbit+matrix+multiplication+for+transformers+at+scale%2E+Advances+in+Neural+Information+Processing+Systems%2C+35%3A30318%E2%80%93)
12. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2024. [Qlora: Efficient finetuning of quantized llms](https://md.research.elamur.ai/free/en/papers/dettmers-2023-qlora-efficient-finetuning-quantized-llms/).
13. Tim Dettmers, Ruslan Svirschevski, Vage Egiazarian, Denis Kuznedelev, Elias Frantar, Saleh Ashkboos, Alexander Borzunov, Torsten Hoefler, and Dan Alistarh. 2023. Spqr: A sparse-quantized representation for near-lossless llm weight compression. [find](https://scholar.google.com/scholar?q=Tim+Dettmers%2C+Ruslan+Svirschevski%2C+Vage+Egiazarian%2C+Denis+Kuznedelev%2C+Elias+Frantar%2C+Saleh+Ashkboos%2C+Alexander+Borzunov%2C+Torsten+Hoefler%2C+and+Dan+Alistarh%2E+2023%2E+Spqr%3A+A+sparse%2Dquantized+representatio)
14. Dayou Du, Yijia Zhang, Shijie Cao, Jiaqi Guo, Ting Cao, Xiaowen Chu, and Ningyi Xu. 2024. Bitdistiller: Unleashing the potential of sub-4-bit llms via self-distillation. [find](https://scholar.google.com/scholar?q=Dayou+Du%2C+Yijia+Zhang%2C+Shijie+Cao%2C+Jiaqi+Guo%2C+Ting+Cao%2C+Xiaowen+Chu%2C+and+Ningyi+Xu%2E+2024%2E+Bitdistiller%3A+Unleashing+the+potential+of+sub%2D4%2Dbit+llms+via+self%2Ddistillation%2E)
15. Wikimedia Foundation. 2020. Wikimedia downloads. [find](https://scholar.google.com/scholar?q=Wikimedia+Foundation%2E+2020%2E+Wikimedia+downloads%2E)
16. Elias Frantar and Dan Alistarh. 2022. Optimal brain compression: A framework for accurate post-training quantization and pruning. Advances in Neural Information Processing Systems, 35:4475–4488. [find](https://scholar.google.com/scholar?q=Elias+Frantar+and+Dan+Alistarh%2E+2022%2E+Optimal+brain+compression%3A+A+framework+for+accurate+post%2Dtraining+quantization+and+pruning%2E+Advances+in+Neural+Information+Processing+Systems%2C+35%3A4475%E2%80%934488%2E)
17. Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. 2022. Gptq: Accurate post-training quantization for generative pre-trained transformers. [find](https://scholar.google.com/scholar?q=Elias+Frantar%2C+Saleh+Ashkboos%2C+Torsten+Hoefler%2C+and+Dan+Alistarh%2E+2022%2E+Gptq%3A+Accurate+post%2Dtraining+quantization+for+generative+pre%2Dtrained+transformers%2E)
18. Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, and 5 others.… [find](https://scholar.google.com/scholar?q=Leo+Gao%2C+Jonathan+Tow%2C+Baber+Abbasi%2C+Stella+Biderman%2C+Sid+Black%2C+Anthony+DiPofi%2C+Charles+Foster%2C+Laurence+Golding%2C+Jeffrey+Hsu%2C+Alain+Le+Noac%E2%80%99h%2C+Haonan+Li%2C+Kyle+McDonell%2C+Niklas+Muennighoff%2C+Chris+Oci)
19. Wei Huang, Yangdong Liu, Haotong Qin, Ying Li, Shiming Zhang, Xianglong Liu, Michele Magno, and Xiaojuan Qi. 2024. Billm: Pushing the limit of post-training quantization for llms. [find](https://scholar.google.com/scholar?q=Wei+Huang%2C+Yangdong+Liu%2C+Haotong+Qin%2C+Ying+Li%2C+Shiming+Zhang%2C+Xianglong+Liu%2C+Michele+Magno%2C+and+Xiaojuan+Qi%2E+2024%2E+Billm%3A+Pushing+the+limit+of+post%2Dtraining+quantization+for+llms%2E)
20. Itay Hubara, Matthieu Courbariaux, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. 2016. Binarized neural networks. Advances in neural information processing systems, 29. [find](https://scholar.google.com/scholar?q=Itay+Hubara%2C+Matthieu+Courbariaux%2C+Daniel+Soudry%2C+Ran+El%2DYaniv%2C+and+Yoshua+Bengio%2E+2016%2E+Binarized+neural+networks%2E+Advances+in+neural+information+processing+systems%2C+29%2E)
21. Dongwon Jo, Taesu Kim, Yulhwa Kim, and 1 others. 2024. Mixture of scales: Memory-efficient token-adaptive binarization for large language models. Advances in Neural Information Processing Systems, 37:137474–137494. [find](https://scholar.google.com/scholar?q=Dongwon+Jo%2C+Taesu+Kim%2C+Yulhwa+Kim%2C+and+1+others%2E+2024%2E+Mixture+of+scales%3A+Memory%2Defficient+token%2Dadaptive+binarization+for+large+language+models%2E+Advances+in+Neural+Information+Processing+Systems%2C+37%3A)
22. Damjan Kalajdzievski. 2023. A rank stabilization scaling factor for fine-tuning with lora. arXiv preprint arXiv:2312.03732. — [arXiv:2312.03732](https://arxiv.org/abs/2312.03732) [find](https://scholar.google.com/scholar?q=Damjan+Kalajdzievski%2E+2023%2E+A+rank+stabilization+scaling+factor+for+fine%2Dtuning+with+lora%2E+arXiv+preprint+arXiv%3A2312%2E03732%2E+%E2%80%94+%5BarXiv%3A2312%2E03732%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2312%2E03732%29)
23. Changhun Lee, Jungyu Jin, Taesu Kim, Hyungjun Kim, and Eunhyeok Park. 2024. [Owq: Outlier-aware weight quantization for efficient fine-tuning and inference of large language models](https://md.research.elamur.ai/free/en/papers/lee-2023-owq-outlier-aware-weight-quantization/). Proceedings of the AAAI Conference on Artificial Intelligence, 38:13355– 13364.
24. Guoyu Li, Shengyu Ye, Chunyun Chen, Yang Wang, Fan Yang, Ting Cao, Cheng Liu, Mohamed M Sabry Aly, and Mao Yang. 2025a. Lut-dla: Lookup table as efficient extreme low-bit deep learning accelerator. [find](https://scholar.google.com/scholar?q=Guoyu+Li%2C+Shengyu+Ye%2C+Chunyun+Chen%2C+Yang+Wang%2C+Fan+Yang%2C+Ting+Cao%2C+Cheng+Liu%2C+Mohamed+M+Sabry+Aly%2C+and+Mao+Yang%2E+2025a%2E+Lut%2Ddla%3A+Lookup+table+as+efficient+extreme+low%2Dbit+deep+learning+accelerator%2E)
25. He Li, Jianhang Hong, Yuanzhuo Wu, Snehal Adbol, and Zonglin Li. 2024. Continuous approximations for improving quantization aware training of llms. [find](https://scholar.google.com/scholar?q=He+Li%2C+Jianhang+Hong%2C+Yuanzhuo+Wu%2C+Snehal+Adbol%2C+and+Zonglin+Li%2E+2024%2E+Continuous+approximations+for+improving+quantization+aware+training+of+llms%2E)
26. Shiyao Li, Yingchun Hu, Xuefei Ning, Xihui Liu, Ke Hong, Xiaotao Jia, Xiuhong Li, Yaqi Yan, Pei Ran, Guohao Dai, and 1 others. 2025b. Mbq: Modality-balanced quantization for large vision-language models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 4167–4177. [find](https://scholar.google.com/scholar?q=Shiyao+Li%2C+Yingchun+Hu%2C+Xuefei+Ning%2C+Xihui+Liu%2C+Ke+Hong%2C+Xiaotao+Jia%2C+Xiuhong+Li%2C+Yaqi+Yan%2C+Pei+Ran%2C+Guohao+Dai%2C+and+1+others%2E+2025b%2E+Mbq%3A+Modality%2Dbalanced+quantization+for+large+vision%2Dlanguage+mode)
27. Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. Can a suit of armor conduct electricity? a new dataset for open book question answering. [find](https://scholar.google.com/scholar?q=Todor+Mihaylov%2C+Peter+Clark%2C+Tushar+Khot%2C+and+Ashish+Sabharwal%2E+2018%2E+Can+a+suit+of+armor+conduct+electricity%3F+a+new+dataset+for+open+book+question+answering%2E)
28. Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. 2024. Awq: Activation-aware weight quantization for on-device llm compression and acceleration. Proceedings of Machine Learning and Systems, 6:87–100. [find](https://scholar.google.com/scholar?q=Ji+Lin%2C+Jiaming+Tang%2C+Haotian+Tang%2C+Shang+Yang%2C+Wei%2DMing+Chen%2C+Wei%2DChen+Wang%2C+Guangxuan+Xiao%2C+Xingyu+Dang%2C+Chuang+Gan%2C+and+Song+Han%2E+2024%2E+Awq%3A+Activation%2Daware+weight+quantization+for+on%2Ddevice+llm+c)
29. Gunho Park, Baeseong Park, Minsub Kim, Sungjae Lee, Jeonghoon Kim, Beomseok Kwon, Se Jung Kwon, Byeongwook Kim, Youngjoo Lee, and Dongsoo Lee. 2024. [Lut-gemm: Quantized matrix multiplication based on luts for efficient inference in large-scale generative language models](https://md.research.elamur.ai/free/en/papers/park-2022-lut-gemm-quantized-matrix-multiplication/).
30. Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, and 1 others. 2024a. Deepseek-v3 technical report. [find](https://scholar.google.com/scholar?q=Aixin+Liu%2C+Bei+Feng%2C+Bing+Xue%2C+Bingxuan+Wang%2C+Bochao+Wu%2C+Chengda+Lu%2C+Chenggang+Zhao%2C+Chengqi+Deng%2C+Chenyu+Zhang%2C+Chong+Ruan%2C+and+1+others%2E+2024a%2E+Deepseek%2Dv3+technical+report%2E)
31. Houwen Peng, Kan Wu, Yixuan Wei, Guoshuai Zhao, Yuxiang Yang, Ze Liu, Yifan Xiong, Ziyue Yang, Bolin Ni, Jingcheng Hu, and 1 others. 2023. Fp8-lm: Training fp8 large language models. [find](https://scholar.google.com/scholar?q=Houwen+Peng%2C+Kan+Wu%2C+Yixuan+Wei%2C+Guoshuai+Zhao%2C+Yuxiang+Yang%2C+Ze+Liu%2C+Yifan+Xiong%2C+Ziyue+Yang%2C+Bolin+Ni%2C+Jingcheng+Hu%2C+and+1+others%2E+2023%2E+Fp8%2Dlm%3A+Training+fp8+large+language+models%2E)
32. Shih-Yang Liu, Zechun Liu, and Kwang-Ting Cheng. 2023a. Oscillation-free quantization for low-bit vision transformers. [find](https://scholar.google.com/scholar?q=Shih%2DYang+Liu%2C+Zechun+Liu%2C+and+Kwang%2DTing+Cheng%2E+2023a%2E+Oscillation%2Dfree+quantization+for+low%2Dbit+vision+transformers%2E)
33. Haotong Qin, Yifu Ding, Mingyuan Zhang, Qinghua Yan, Aishan Liu, Qingqing Dang, Ziwei Liu, and Xianglong Liu. 2022. Bibert: Accurate fully binarized bert. [find](https://scholar.google.com/scholar?q=Haotong+Qin%2C+Yifu+Ding%2C+Mingyuan+Zhang%2C+Qinghua+Yan%2C+Aishan+Liu%2C+Qingqing+Dang%2C+Ziwei+Liu%2C+and+Xianglong+Liu%2E+2022%2E+Bibert%3A+Accurate+fully+binarized+bert%2E)
34. Zechun Liu, Barlas Oguz, Aasish Pappu, Lin Xiao, Scott Yih, Meng Li, Raghuraman Krishnamoorthi, and Yashar Mehdad. 2022. Bit: Robustly binarized multi-distilled transformer. Advances in neural information processing systems, 35:14303–14316. [find](https://scholar.google.com/scholar?q=Zechun+Liu%2C+Barlas+Oguz%2C+Aasish+Pappu%2C+Lin+Xiao%2C+Scott+Yih%2C+Meng+Li%2C+Raghuraman+Krishnamoorthi%2C+and+Yashar+Mehdad%2E+2022%2E+Bit%3A+Robustly+binarized+multi%2Ddistilled+transformer%2E+Advances+in+neural+informa)
35. Zechun Liu, Barlas Oguz, Changsheng Zhao, Ernie Chang, Pierre Stock, Yashar Mehdad, Yangyang Shi, Raghuraman Krishnamoorthi, and Vikas Chandra. 2023b. [Llm-qat: Data-free quantization aware training for large language models](https://md.research.elamur.ai/free/en/papers/liu-2024-llm-qat-data-free-quantization/).
36. Zechun Liu, Changsheng Zhao, Igor Fedorov, Bilge Soran, Dhruv Choudhary, Raghuraman Krishnamoorthi, Vikas Chandra, Yuandong Tian, and Tijmen Blankevoort. 2024b. Spinquant: Llm quantization with learned rotations. [find](https://scholar.google.com/scholar?q=Zechun+Liu%2C+Changsheng+Zhao%2C+Igor+Fedorov%2C+Bilge+Soran%2C+Dhruv+Choudhary%2C+Raghuraman+Krishnamoorthi%2C+Vikas+Chandra%2C+Yuandong+Tian%2C+and+Tijmen+Blankevoort%2E+2024b%2E+Spinquant%3A+Llm+quantization+with+learne)
37. Dan Luo, Kangfeng Zheng, Chunhua Wu, and Xiujuan Wang. 2025. Mslora: Meta-learned scaling for adaptive fine-tuning of lora. Neurocomputing, page 130374. [find](https://scholar.google.com/scholar?q=Dan+Luo%2C+Kangfeng+Zheng%2C+Chunhua+Wu%2C+and+Xiujuan+Wang%2E+2025%2E+Mslora%3A+Meta%2Dlearned+scaling+for+adaptive+fine%2Dtuning+of+lora%2E+Neurocomputing%2C+page+130374%2E)
38. Ziyang Luo, Artur Kulmizev, and Xiaoxi Mao. 2020. Positional artefacts propagate through masked language model embeddings. [find](https://scholar.google.com/scholar?q=Ziyang+Luo%2C+Artur+Kulmizev%2C+and+Xiaoxi+Mao%2E+2020%2E+Positional+artefacts+propagate+through+masked+language+model+embeddings%2E)
39. Liqun Ma, Mingjie Sun, and Zhiqiang Shen. 2024a. [Fbi-llm: Scaling up fully binarized llms from scratch via autoregressive distillation](https://md.research.elamur.ai/free/en/papers/ma-2024-fbi-llm-scaling-up-fully/).
40. Shuming Ma, Hongyu Wang, Shaohan Huang, Xingxing Zhang, Ying Hu, Ting Song, Yan Xia, and Furu Wei. 2025. Bitnet b1. 58 2b4t technical report. [find](https://scholar.google.com/scholar?q=Shuming+Ma%2C+Hongyu+Wang%2C+Shaohan+Huang%2C+Xingxing+Zhang%2C+Ying+Hu%2C+Ting+Song%2C+Yan+Xia%2C+and+Furu+Wei%2E+2025%2E+Bitnet+b1%2E+58+2b4t+technical+report%2E)
41. Shuming Ma, Hongyu Wang, Lingxiao Ma, Lei Wang, Wenhui Wang, Shaohan Huang, Li Dong, Ruiping Wang, Jilong Xue, and Furu Wei. 2024b. The era of 1-bit llms: All large language models are in 1.58 bits. [find](https://scholar.google.com/scholar?q=Shuming+Ma%2C+Hongyu+Wang%2C+Lingxiao+Ma%2C+Lei+Wang%2C+Wenhui+Wang%2C+Shaohan+Huang%2C+Li+Dong%2C+Ruiping+Wang%2C+Jilong+Xue%2C+and+Furu+Wei%2E+2024b%2E+The+era+of+1%2Dbit+llms%3A+All+large+language+models+are+in+1%2E58+bits%2E)
42. Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2016. [Pointer sentinel mixture models](https://md.research.elamur.ai/free/en/papers/merity-2016-pointer-sentinel-mixture-models/).
43. Asit Mishra, Dusan Stosic, and Simon Layton. 2025. Recipes for pre-training llms with mxfp8. [find](https://scholar.google.com/scholar?q=Asit+Mishra%2C+Dusan+Stosic%2C+and+Simon+Layton%2E+2025%2E+Recipes+for+pre%2Dtraining+llms+with+mxfp8%2E)
44. Xiaoyi Qu, David Aponte, Colby Banbury, Daniel P Robinson, Tianyu Ding, Kazuhito Koishida, Ilya Zharkov, and Tianyi Chen. 2025. Automatic joint structured pruning and quantization for efficient neural network training and compression. In Proceedings of the Computer Vision and Pattern Recognition Con… [find](https://scholar.google.com/scholar?q=Xiaoyi+Qu%2C+David+Aponte%2C+Colby+Banbury%2C+Daniel+P+Robinson%2C+Tianyu+Ding%2C+Kazuhito+Koishida%2C+Ilya+Zharkov%2C+and+Tianyi+Chen%2E+2025%2E+Automatic+joint+structured+pruning+and+quantization+for+efficient+neural)
45. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). Journal of machine learning research, 21(140):1–67.
46. Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. 2020. Deepspeed: System optimizations enable training deep learning models with over 100 billion parameters. [find](https://scholar.google.com/scholar?q=Jeff+Rasley%2C+Samyam+Rajbhandari%2C+Olatunji+Ruwase%2C+and+Yuxiong+He%2E+2020%2E+Deepspeed%3A+System+optimizations+enable+training+deep+learning+models+with+over+100+billion+parameters%2E)
47. Mohammad Rastegari, Vicente Ordonez, Joseph Redmon, and Ali Farhadi. 2016. [Xnor-net: Imagenet classification using binary convolutional neural networks](https://md.research.elamur.ai/free/en/papers/rastegari-2016-xnor-net-imagenet-classification-using/). In European conference on computer vision, pages 525–542. Springer.
48. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99–106. [find](https://scholar.google.com/scholar?q=Keisuke+Sakaguchi%2C+Ronan+Le+Bras%2C+Chandra+Bhagavatula%2C+and+Yejin+Choi%2E+2021%2E+Winogrande%3A+An+adversarial+winograd+schema+challenge+at+scale%2E+Communications+of+the+ACM%2C+64%289%29%3A99%E2%80%93106%2E)
49. Yuzhang Shang, Zhihang Yuan, Qiang Wu, and Zhen Dong. 2023. Pb-llm: Partially binarized large language models. [find](https://scholar.google.com/scholar?q=Yuzhang+Shang%2C+Zhihang+Yuan%2C+Qiang+Wu%2C+and+Zhen+Dong%2E+2023%2E+Pb%2Dllm%3A+Partially+binarized+large+language+models%2E)
50. Wenqi Shao, Mengzhao Chen, Zhaoyang Zhang, Peng Xu, Lirui Zhao, Zhiqian Li, Kaipeng Zhang, Peng Gao, Yu Qiao, and Ping Luo. 2023. [Omniquant: Omnidirectionally calibrated quantization for large language models](https://md.research.elamur.ai/free/en/papers/shao-2023-omniquant-omnidirectionally-calibrated-quantization-large/).
51. Noam Shazeer. 2020. Glu variants improve transformer. [find](https://scholar.google.com/scholar?q=Noam+Shazeer%2E+2020%2E+Glu+variants+improve+transformer%2E)
52. Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. 2017. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. [find](https://scholar.google.com/scholar?q=Noam+Shazeer%2C+Azalia+Mirhoseini%2C+Krzysztof+Maziarz%2C+Andy+Davis%2C+Quoc+Le%2C+Geoffrey+Hinton%2C+and+Jeff+Dean%2E+2017%2E+Outrageously+large+neural+networks%3A+The+sparsely%2Dgated+mixture%2Dof%2Dexperts+layer%2E)
53. Mingzhu Shen, Feng Liang, Ruihao Gong, Yuhang Li, Chuming Li, Chen Lin, Fengwei Yu, Junjie Yan, and Wanli Ouyang. 2021. Once quantization-aware training: High performance extremely low-bit architecture search. [find](https://scholar.google.com/scholar?q=Mingzhu+Shen%2C+Feng+Liang%2C+Ruihao+Gong%2C+Yuhang+Li%2C+Chuming+Li%2C+Chen+Lin%2C+Fengwei+Yu%2C+Junjie+Yan%2C+and+Wanli+Ouyang%2E+2021%2E+Once+quantization%2Daware+training%3A+High+performance+extremely+low%2Dbit+architectur)
54. Ma Shuming, Wang Hongyu, and Furu Wei. 2024. The era of 1 bit llms training tips code faq. [find](https://scholar.google.com/scholar?q=Ma+Shuming%2C+Wang+Hongyu%2C+and+Furu+Wei%2E+2024%2E+The+era+of+1+bit+llms+training+tips+code+faq%2E)
55. Falcon-LLM Team. 2025. Falcon-e, a series of powerful, universal and fine-tunable 1.58bit language models. [find](https://scholar.google.com/scholar?q=Falcon%2DLLM+Team%2E+2025%2E+Falcon%2De%2C+a+series+of+powerful%2C+universal+and+fine%2Dtunable+1%2E58bit+language+models%2E)
56. Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, and 1 others. 2023. [Gemini: a family of highly capable multimodal models](https://md.research.elamur.ai/free/en/papers/gemini-team-2023-gemini-family-highly-capable-multimodal/).
57. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, and 1 others. 2023a. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/).
58. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and 1 others. 2023b. [Llama 2: Open foundation and fine-tuned chat models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/).
59. Jianyu Wei, Shijie Cao, Ting Cao, Lingxiao Ma, Lei Wang, Yanyong Zhang, and Mao Yang. 2024. T-mac: Cpu renaissance via table lookup for low-bit llm deployment on edge. Preprint, arXiv:2407.00088. — [arXiv:2407.00088](https://arxiv.org/abs/2407.00088) [find](https://scholar.google.com/scholar?q=Jianyu+Wei%2C+Shijie+Cao%2C+Ting+Cao%2C+Lingxiao+Ma%2C+Lei+Wang%2C+Yanyong+Zhang%2C+and+Mao+Yang%2E+2024%2E+T%2Dmac%3A+Cpu+renaissance+via+table+lookup+for+low%2Dbit+llm+deployment+on+edge%2E+Preprint%2C+arXiv%3A2407%2E00088%2E+%E2%80%94+%5Ba)
60. Xiuying Wei, Yunchen Zhang, Yuhang Li, Xiangguo Zhang, Ruihao Gong, Jinyang Guo, and Xianglong Liu. 2023. Outlier suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling. [find](https://scholar.google.com/scholar?q=Xiuying+Wei%2C+Yunchen+Zhang%2C+Yuhang+Li%2C+Xiangguo+Zhang%2C+Ruihao+Gong%2C+Jinyang+Guo%2C+and+Xianglong+Liu%2E+2023%2E+Outlier+suppression%2B%3A+Accurate+quantization+of+large+language+models+by+equivalent+and+optimal)
61. Xiuying Wei, Yunchen Zhang, Xiangguo Zhang, Ruihao Gong, Shanghang Zhang, Qi Zhang, Fengwei Yu, and Xianglong Liu. 2022. Outlier suppression: Pushing the limit of low-bit transformer language models. Advances in Neural Information Processing Systems, 35:17402–17414. [find](https://scholar.google.com/scholar?q=Xiuying+Wei%2C+Yunchen+Zhang%2C+Xiangguo+Zhang%2C+Ruihao+Gong%2C+Shanghang+Zhang%2C+Qi+Zhang%2C+Fengwei+Yu%2C+and+Xianglong+Liu%2E+2022%2E+Outlier+suppression%3A+Pushing+the+limit+of+low%2Dbit+transformer+language+models%2E+)
62. Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. 2023. Smoothquant: Accurate and efficient post-training quantization for large language models. [find](https://scholar.google.com/scholar?q=Guangxuan+Xiao%2C+Ji+Lin%2C+Mickael+Seznec%2C+Hao+Wu%2C+Julien+Demouth%2C+and+Song+Han%2E+2023%2E+Smoothquant%3A+Accurate+and+efficient+post%2Dtraining+quantization+for+large+language+models%2E)
63. Yuzhuang Xu, Xu Han, Zonghan Yang, Shuo Wang, Qingfu Zhu, Zhiyuan Liu, Weidong Liu, and Wanxiang Che. 2024. Onebit: Towards extremely low-bit large language models. [find](https://scholar.google.com/scholar?q=Yuzhuang+Xu%2C+Xu+Han%2C+Zonghan+Yang%2C+Shuo+Wang%2C+Qingfu+Zhu%2C+Zhiyuan+Liu%2C+Weidong+Liu%2C+and+Wanxiang+Che%2E+2024%2E+Onebit%3A+Towards+extremely+low%2Dbit+large+language+models%2E)
64. Fuzhao Xue, Zian Zheng, Yao Fu, Jinjie Ni, Zangwei Zheng, Wangchunshu Zhou, and Yang You. 2024. [Openmoe: An early effort on open mixture-of-experts language models](https://md.research.elamur.ai/free/en/papers/xue-2024-openmoe-early-effort-open-mixture/).
65. Vikas Yadav, Steven Bethard, and Mihai Surdeanu. 2019. Quick and (not so) dirty: Unsupervised selection of justification sentences for multi-hop question answering. [find](https://scholar.google.com/scholar?q=Vikas+Yadav%2C+Steven+Bethard%2C+and+Mihai+Surdeanu%2E+2019%2E+Quick+and+%28not+so%29+dirty%3A+Unsupervised+selection+of+justification+sentences+for+multi%2Dhop+question+answering%2E)
66. Mart Van Baalen, Andrey Kuzmin, Suparna S Nair, Yuwei Ren, Eric Mahurin, Chirag Patel, Sundar Subramanian, Sanghyuk Lee, Markus Nagel, Joseph So-riaga, and 1 others. 2023. Fp8 versus int8 for efficient deep learning inference. [find](https://scholar.google.com/scholar?q=Mart+Van+Baalen%2C+Andrey+Kuzmin%2C+Suparna+S+Nair%2C+Yuwei+Ren%2C+Eric+Mahurin%2C+Chirag+Patel%2C+Sundar+Subramanian%2C+Sanghyuk+Lee%2C+Markus+Nagel%2C+Joseph+So%2Driaga%2C+and+1+others%2E+2023%2E+Fp8+versus+int8+for+efficien)
67. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, and 1 others. 2024. Qwen2. 5 technical report. [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Haoran+Wei%2C+and+1+others%2E+2024%2E+Qwen2%2E+5+technical+report%2E)
68. Feiyu Wang, Guoan Wang, Yihao Zhang, Shengfan Wang, Weitao Li, Bokai Huang, Shimao Chen, Zihan Jiang, Rui Xu, and Tong Yang. 2025. ifairy: The first 2-bit complex llm with all parameters in {+1, -1, +i, -i}. arXiv preprint arXiv:2508.05571. — [arXiv:2508.05571](https://arxiv.org/abs/2508.05571) [find](https://scholar.google.com/scholar?q=Feiyu+Wang%2C+Guoan+Wang%2C+Yihao+Zhang%2C+Shengfan+Wang%2C+Weitao+Li%2C+Bokai+Huang%2C+Shimao+Chen%2C+Zihan+Jiang%2C+Rui+Xu%2C+and+Tong+Yang%2E+2025%2E+ifairy%3A+The+first+2%2Dbit+complex+llm+with+all+parameters+in+%7B%2B1%2C+%2D1%2C+%2B)
69. Zhewei Yao, Zhen Dong, Zhangcheng Zheng, Amir Gholami, Jiali Yu, Eric Tan, Leyuan Wang, Qijing Huang, Yida Wang, Michael Mahoney, and 1 others. 2021. Hawq-v3: Dyadic neural network quantization. [find](https://scholar.google.com/scholar?q=Zhewei+Yao%2C+Zhen+Dong%2C+Zhangcheng+Zheng%2C+Amir+Gholami%2C+Jiali+Yu%2C+Eric+Tan%2C+Leyuan+Wang%2C+Qijing+Huang%2C+Yida+Wang%2C+Michael+Mahoney%2C+and+1+others%2E+2021%2E+Hawq%2Dv3%3A+Dyadic+neural+network+quantization%2E)
70. Hongyu Wang, Shuming Ma, Li Dong, Shaohan Huang, Huaijie Wang, Lingxiao Ma, Fan Yang, Ruiping Wang, Yi Wu, and Furu Wei. 2023. Bitnet: Scaling 1-bit transformers for large language models. [find](https://scholar.google.com/scholar?q=Hongyu+Wang%2C+Shuming+Ma%2C+Li+Dong%2C+Shaohan+Huang%2C+Huaijie+Wang%2C+Lingxiao+Ma%2C+Fan+Yang%2C+Ruiping+Wang%2C+Yi+Wu%2C+and+Furu+Wei%2E+2023%2E+Bitnet%3A+Scaling+1%2Dbit+transformers+for+large+language+models%2E)
71. Zhewei Yao, Reza Yazdani Aminabadi, Minjia Zhang, Xiaoxia Wu, Conglong Li, and Yuxiong He. 2022. [Zeroquant: Efficient and affordable post-training quantization for large-scale transformers](https://md.research.elamur.ai/free/en/papers/yao-2022-zeroquant-efficient-affordable-post-training/). Advances in Neural Information Processing Systems, 35:27168– 27183.
72. Lei Wang, Lingxiao Ma, Shijie Cao, Quanlu Zhang, Jilong Xue, Yining Shi, Ningxin Zheng, Ziming Miao, Fan Yang, Ting Cao, Yuqing Yang, and Mao Yang. 2024. Ladder: Enabling efficient Low-Precision deep learning computing through hardware-aware tensor transformation. In 18th USENIX Symposium on Operati… [find](https://scholar.google.com/scholar?q=Lei+Wang%2C+Lingxiao+Ma%2C+Shijie+Cao%2C+Quanlu+Zhang%2C+Jilong+Xue%2C+Yining+Shi%2C+Ningxin+Zheng%2C+Ziming+Miao%2C+Fan+Yang%2C+Ting+Cao%2C+Yuqing+Yang%2C+and+Mao+Yang%2E+2024%2E+Ladder%3A+Enabling+efficient+Low%2DPrecision+deep+)
73. Penghang Yin, Jiancheng Lyu, Shuai Zhang, Stanley Osher, Yingyong Qi, and Jack Xin. 2019. Understanding straight-through estimator in training activation quantized neural nets. [find](https://scholar.google.com/scholar?q=Penghang+Yin%2C+Jiancheng+Lyu%2C+Shuai+Zhang%2C+Stanley+Osher%2C+Yingyong+Qi%2C+and+Jack+Xin%2E+2019%2E+Understanding+straight%2Dthrough+estimator+in+training+activation+quantized+neural+nets%2E)
74. Mengxia Yu, De Wang, Qi Shan, and Alvin Wan. 2024. The super weight in large language models. [find](https://scholar.google.com/scholar?q=Mengxia+Yu%2C+De+Wang%2C+Qi+Shan%2C+and+Alvin+Wan%2E+2024%2E+The+super+weight+in+large+language+models%2E)
75. Zhihang Yuan, Lin Niu, Jiawei Liu, Wenyu Liu, Xinggang Wang, Yuzhang Shang, Guangyu Sun, Qiang Wu, Jiaxiang Wu, and Bingzhe Wu. 2023. Rptq: Reorder-based post-training quantization for large language models. arXiv preprint arXiv:2304.01089. — [arXiv:2304.01089](https://arxiv.org/abs/2304.01089) [find](https://scholar.google.com/scholar?q=Zhihang+Yuan%2C+Lin+Niu%2C+Jiawei+Liu%2C+Wenyu+Liu%2C+Xinggang+Wang%2C+Yuzhang+Shang%2C+Guangyu+Sun%2C+Qiang+Wu%2C+Jiaxiang+Wu%2C+and+Bingzhe+Wu%2E+2023%2E+Rptq%3A+Reorder%2Dbased+post%2Dtraining+quantization+for+large+language+)
76. Zhihang Yuan, Yuzhang Shang, Yang Zhou, Zhen Dong, Zhe Zhou, Chenhao Xue, Bingzhe Wu, Zhikai Li, Qingyi Gu, Yong Jae Lee, and 1 others. 2024. [Llm inference unveiled: Survey and roofline model insights](https://md.research.elamur.ai/free/en/papers/yuan-2024-llm-inference-unveiled-survey-roofline/).
77. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? [find](https://scholar.google.com/scholar?q=Rowan+Zellers%2C+Ari+Holtzman%2C+Yonatan+Bisk%2C+Ali+Farhadi%2C+and+Yejin+Choi%2E+2019%2E+Hellaswag%3A+Can+a+machine+really+finish+your+sentence%3F)
78. Hailin Zhang, Xiaodong Ji, Yilin Chen, Fangcheng Fu, Xupeng Miao, Xiaonan Nie, Weipeng Chen, and Bin Cui. 2025. Pqcache: Product quantization-based kvcache for long context llm inference. Proceedings of the ACM on Management of Data, 3(3):1–30. [find](https://scholar.google.com/scholar?q=Hailin+Zhang%2C+Xiaodong+Ji%2C+Yilin+Chen%2C+Fangcheng+Fu%2C+Xupeng+Miao%2C+Xiaonan+Nie%2C+Weipeng+Chen%2C+and+Bin+Cui%2E+2025%2E+Pqcache%3A+Product+quantization%2Dbased+kvcache+for+long+context+llm+inference%2E+Proceedings+)
79. Jiaqi Zhao, Miao Zhang, Ming Wang, Yuzhang Shang, Kaihao Zhang, Weili Guan, Yaowei Wang, and Min Zhang. 2025. Ptq1. 61: Push the real limit of extremely low-bit post-training quantization methods for large language models. arXiv preprint arXiv:2502.13179. [find](https://scholar.google.com/scholar?q=Jiaqi+Zhao%2C+Miao+Zhang%2C+Ming+Wang%2C+Yuzhang+Shang%2C+Kaihao+Zhang%2C+Weili+Guan%2C+Yaowei+Wang%2C+and+Min+Zhang%2E+2025%2E+Ptq1%2E+61%3A+Push+the+real+limit+of+extremely+low%2Dbit+post%2Dtraining+quantization+methods+for+)
80. Rui-Jie Zhu, Yu Zhang, Ethan Sifferman, Tyler Sheaves, Yiqiao Wang, Dustin Richmond, Peng Zhou, and Jason K Eshraghian. 2024. Scalable matmul-free language modeling. [find](https://scholar.google.com/scholar?q=Rui%2DJie+Zhu%2C+Yu+Zhang%2C+Ethan+Sifferman%2C+Tyler+Sheaves%2C+Yiqiao+Wang%2C+Dustin+Richmond%2C+Peng+Zhou%2C+and+Jason+K+Eshraghian%2E+2024%2E+Scalable+matmul%2Dfree+language+modeling%2E)

*Updated 2 November 2026*