---
title: "How Quantization Shapes Bias in Large Language Models (2026)"
description: "The authors examine how post-training quantization alters bias across three tiers: beliefs, where they… 51 references, cited by 2 papers here."
authors: ["Federico Marcuzzi", "Xuefei Ning", "Roy Schwartz", "Iryna Gurevych"]
year: 2026
venue: "Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.eacl-long.17/"
doi: "10.18653/v1/2026.eacl-long.17"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/marcuzzi-2026-how-quantization-shapes-bias-large/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/marcuzzi-2026-how-quantization-shapes-bias-large/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/5/) / How Quantization Shapes Bias in Large Language Models

# How Quantization Shapes Bias in Large Language Models

**Authors:** Federico Marcuzzi, [Xuefei Ning](https://md.research.elamur.ai/free/en/authors/xuefei-ning/), [Roy Schwartz](https://md.research.elamur.ai/free/en/authors/roy-schwartz/) and [Iryna Gurevych](https://md.research.elamur.ai/free/en/authors/iryna-gurevych/) · **Year:** 2026 · **Venue:** [Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/eacl/) ([EACL 2026](https://md.research.elamur.ai/free/en/venues/eacl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.eacl-long.17/) · [PDF](https://aclanthology.org/2026.eacl-long.17.pdf) · [DOI 10.18653/v1/2026.eacl-long.17](https://doi.org/10.18653/v1/2026.eacl-long.17) · [OpenAlex](https://openalex.org/W7140099977)

**Citations:** 1 citations (OpenAlex, 2026-10-14) · 2 in our library, of the 149,439 papers whose bibliographies we read

## Summary
Quantization is a common way to shrink LLMs for deployment, but its effect on social bias has received little systematic study. The authors examine how post-training quantization alters bias across three tiers: beliefs, where they gauge stereotyping; decisions, where they assess fairness; and language, where they compare toxicity and sentiment between subgroups. They cover gender, race, and religion and look at both whole categories and individual subgroups. Four instruction-tuned models are compressed with weight-only methods (GPTQ, AWQ) and a weight-activation method (SmoothQuant) at several bit-widths. The models differ in architecture family (LLaMA vs. Qwen) and in whether they are reasoning-oriented. Evaluation uses 13 benchmarks with both probability-based and generation-based metrics.

Quantization usually lowers raw toxicity and barely changes sentiment, apart from a mild drift toward neutral. However, it somewhat increases stereotyping and, in generative settings, unfairness. These effects grow stronger with more aggressive compression, which also hurts capabilities and raises refusal or no-answer rates. Reasoning models start out less biased, but this advantage does not cancel the harm from quantization. The work matters because it shows practitioners that efficiency gains can carry hidden fairness costs. Bias checks should therefore be fine-grained and use multiple metrics.

## Key points
- Seven quantization settings were tested: GPTQ W4/W8, AWQ W3/W4/W8, and SmoothQuant W4A8/W8A8. Together with the unquantized originals, this gives 32 evaluated model variants.
- The four base models are LLaMA-3.1-8B-Instruct, Qwen2.5-14B-Instruct, DeepSeek-R1-Distill-LLaMA-8B, and DeepSeek-R1-Distill-Qwen-14B. The authors present this as the first bias study of quantized reasoning models.
- Benchmarks include:
  - Capabilities: MMLU
  - Stereotypes: StereoSet, RedditBias, WinoBias, BBQ
  - Fairness: DiscrimEval, DiscrimEvalGen, DT-Fairness
  - Language-level bias: DT-Toxicity
- Bias gaps across categories and subgroups mostly stay close to those of the original model. The exception is fairness under generated-text metrics, and in BBQ quantization widens disparities for religion (and also gender in non-reasoning models). Architecture family makes little difference.
- Limitations: results come from single runs with greedy decoding, checked with an approximate randomization test for significance. The study covers English only and excludes QAT and mixed-precision quantization.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 543 works

🧭 Topics (3): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🛡️ Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/) · 517 works · [🪶 Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/) · 242 works

## Cite

Cite as: Federico Marcuzzi, Xuefei Ning, Roy Schwartz and Iryna Gurevych. 2026. How Quantization Shapes Bias in Large Language Models. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{marcuzzi2026how,
  title = {{How Quantization Shapes Bias in Large Language Models}},
  author = {Federico Marcuzzi, Xuefei Ning, Roy Schwartz and Iryna Gurevych},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.17},
  url = {https://doi.org/10.18653/v1/2026.eacl-long.17}
}
```

- [← previous How Long Reasoning Chains Influence LLMs' Judgment of Answer Factuality](https://md.research.elamur.ai/free/en/papers/tu-2026-how-long-reasoning-chains-influence/)
- [next → Imbalanced Gradients in RL Post-Training of Multi-Task LLMs](https://md.research.elamur.ai/free/en/papers/wu-2026-imbalanced-gradients-rl-post-training/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/5/)

## References

51 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Simone Balloccu, Patrícia Schmidtová, Mateusz Lango, and Ondrej Dusek. 2024. [Leak, cheat, repeat: Data contamination and evaluation malpractices in closed-source llms](https://md.research.elamur.ai/free/en/papers/balloccu-2024-leak-cheat-repeat-data-contamination/). In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2024 - Volume 1…
2. Mislav Balunović, Jasper Dekoninck, Ivo Petrov, Nikola Jovanović, and Martin Vechev. 2025. Matharena: Evaluating llms on uncontaminated math competitions. Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmark. [find](https://scholar.google.com/scholar?q=Mislav+Balunovic%CC%81%2C+Jasper+Dekoninck%2C+Ivo+Petrov%2C+Nikola+Jovanovic%CC%81%2C+and+Martin+Vechev%2E+2025%2E+Matharena%3A+Evaluating+llms+on+uncontaminated+math+competitions%2E+Proceedings+of+the+Neural+Information+Proce)
3. Soumya Barikeri, Anne Lauscher, Ivan Vulic, and Goran Glavas. 2021. Redditbias: A real-world resource for bias evaluation and debiasing of conversational language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confe… [find](https://scholar.google.com/scholar?q=Soumya+Barikeri%2C+Anne+Lauscher%2C+Ivan+Vulic%2C+and+Goran+Glavas%2E+2021%2E+Redditbias%3A+A+real%2Dworld+resource+for+bias+evaluation+and+debiasing+of+conversational+language+models%2E+In+Proceedings+of+the+59th+An)
4. Barry Becker and Ronny Kohavi. 1996. UCI Machine Learning Repository. https://doi.org/10.24432/C5XW20. — [doi:10.24432/c5xw20](https://doi.org/10.24432/c5xw20) [↗](https://doi.org/10.24432/C5XW20)
5. Adult. DOI: J. Cohen. 1988. Statistical Power Analysis for the Behavioral Sciences. Lawrence Erlbaum Associates. [find](https://scholar.google.com/scholar?q=Adult%2E+DOI%3A+J%2E+Cohen%2E+1988%2E+Statistical+Power+Analysis+for+the+Behavioral+Sciences%2E+Lawrence+Erlbaum+Associates%2E)
6. DeepSeek AI. 2025a. DeepSeek-R1-Distill-Llama-8B. [https://huggingface.co/deepseek-ai/DeepSe](https://huggingface.co/deepseek-ai/DeepSe) ek-R1-Distill-Llama-8B. Model card; released January 2025. [↗](https://huggingface.co/deepseek-ai/DeepSe)
7. DeepSeek AI. 2025b. DeepSeek-R1-Distill-Qwen-14B. [https://huggingface.co/deepseek-ai/DeepSe](https://huggingface.co/deepseek-ai/DeepSe) ek-R1-Distill-Qwen-14B. Model card; released January 2025. [↗](https://huggingface.co/deepseek-ai/DeepSe)
8. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 81 others. 2025. [Deepseek-r1: Incentivizing reasoning capability in llm…](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/)
9. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
10. Jwala Dhamala, Tony Sun, Varun Kumar, Satyapriya Krishna, Yada Pruksachatkun, Kai-Wei Chang, and Rahul Gupta. 2021. BOLD: dataset and metrics for measuring biases in open-ended language generation. In FAccT ’21: 2021 ACM Conference on Fairness, Accountability, and Transparency, Virtual Event / Toron… [find](https://scholar.google.com/scholar?q=Jwala+Dhamala%2C+Tony+Sun%2C+Varun+Kumar%2C+Satyapriya+Krishna%2C+Yada+Pruksachatkun%2C+Kai%2DWei+Chang%2C+and+Rahul+Gupta%2E+2021%2E+BOLD%3A+dataset+and+metrics+for+measuring+biases+in+open%2Dended+language+generation%2E+In)
11. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, and 82 others. 2024… — [The Llama 3 Herd of Models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/)
12. Gustavo Gonçalves and Emma Strubell. 2023. Understanding the effect of model compression on social bias in large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pages 2663–2675. Association for Co… [find](https://scholar.google.com/scholar?q=Gustavo+Gon%C3%A7alves+and+Emma+Strubell%2E+2023%2E+Understanding+the+effect+of+model+compression+on+social+bias+in+large+language+models%2E+In+Proceedings+of+the+2023+Conference+on+Empirical+Methods+in+Natural+)
13. Ruihao Gong, Yang Yong, Shiqiao Gu, Yushi Huang, Chengtao Lv, Yunchen Zhang, Dacheng Tao, and Xianglong Liu. 2024. LLMC: benchmarking large language model quantization with a versatile compression toolkit. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: EMN… [find](https://scholar.google.com/scholar?q=Ruihao+Gong%2C+Yang+Yong%2C+Shiqiao+Gu%2C+Yushi+Huang%2C+Chengtao+Lv%2C+Yunchen+Zhang%2C+Dacheng+Tao%2C+and+Xianglong+Liu%2E+2024%2E+LLMC%3A+benchmarking+large+language+model+quantization+with+a+versatile+compression+too)
14. William Sealy Gosset. 1908. The probable error of a mean. Biometrika, 6(1):1–25. Originally published under the pseudonym “Student”. [find](https://scholar.google.com/scholar?q=William+Sealy+Gosset%2E+1908%2E+The+probable+error+of+a+mean%2E+Biometrika%2C+6%281%29%3A1%E2%80%9325%2E+Originally+published+under+the+pseudonym+%E2%80%9CStudent%E2%80%9D%2E)
15. Philipp Guldimann, Alexander Spiridonov, Robin Staab, Nikola Jovanovic, Mark Vero, Velko Vechev, Anna Gueorguieva, Mislav Balunovic, Nikola Konstantinov, Pavol Bielik, Petar Tsankov, and Martin T. Vechev. 2024. COMPL-AI framework: A technical interpretation and LLM benchmarking suite for the EU arti… — [arXiv:2410.07959](https://arxiv.org/abs/2410.07959) [find](https://scholar.google.com/scholar?q=Philipp+Guldimann%2C+Alexander+Spiridonov%2C+Robin+Staab%2C+Nikola+Jovanovic%2C+Mark+Vero%2C+Velko+Vechev%2C+Anna+Gueorguieva%2C+Mislav+Balunovic%2C+Nikola+Konstantinov%2C+Pavol+Bielik%2C+Petar+Tsankov%2C+and+Martin+T%2E+Vec)
16. Laura Hanu and Unitary team. 2020. Detoxify. Github. [https://github.com/unitaryai/detoxify](https://github.com/unitaryai/detoxify). [↗](https://github.com/unitaryai/detoxify)
17. Moritz Hardt, Eric Price, and Nati Srebro. 2016. [Equality of opportunity in supervised learning](https://md.research.elamur.ai/free/en/papers/hardt-2016-equality-opportunity-supervised-learning/). In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5-10, 2016, Barcelona, Spain, pages 3315–3323.
18. Larry V. Hedges. 1981. Distribution Theory for Glass’s Estimator of Effect size and Related Estimators. Journal of Educational and Behavioral Statistics, 6(2):107–128. [find](https://scholar.google.com/scholar?q=Larry+V%2E+Hedges%2E+1981%2E+Distribution+Theory+for+Glass%E2%80%99s+Estimator+of+Effect+size+and+Related+Estimators%2E+Journal+of+Educational+and+Behavioral+Statistics%2C+6%282%29%3A107%E2%80%93128%2E)
19. Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard S. Zemel. 2012. Fairness through awareness. In Innovations in Theoretical Computer Science 2012, Cambridge, MA, USA, January 8-10, 2012, pages 214–226. ACM. [find](https://scholar.google.com/scholar?q=Cynthia+Dwork%2C+Moritz+Hardt%2C+Toniann+Pitassi%2C+Omer+Reingold%2C+and+Richard+S%2E+Zemel%2E+2012%2E+Fairness+through+awareness%2E+In+Innovations+in+Theoretical+Computer+Science+2012%2C+Cambridge%2C+MA%2C+USA%2C+January+8%2D)
20. Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. 2023. OPTQ: accurate quantization for generative pre-trained transformers. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net. [find](https://scholar.google.com/scholar?q=Elias+Frantar%2C+Saleh+Ashkboos%2C+Torsten+Hoefler%2C+and+Dan+Alistarh%2E+2023%2E+OPTQ%3A+accurate+quantization+for+generative+pre%2Dtrained+transformers%2E+In+The+Eleventh+International+Conference+on+Learning+Repres)
21. Isabel O. Gallegos, Ryan A. Rossi, Joe Barrow, Md. Mehrab Tanjim, Sungchul Kim, Franck Dernoncourt, Tong Yu, Ruiyi Zhang, and Nesreen K. Ahmed. 2024. [Bias and fairness in large language models: A survey](https://md.research.elamur.ai/free/en/papers/gallegos-2023-bias-fairness-large-language-models/). Comput. Linguistics, 50(3):1097–1179.
22. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net.
23. Junyuan Hong, Jinhao Duan, Chenhui Zhang, Zhangheng Li, Chulin Xie, Kelsey Lieberman, James Diffenderfer, Brian R. Bartoldson, Ajay Kumar Jaiswal, Kaidi Xu, Bhavya Kailkhura, Dan Hendrycks, Dawn Song, Zhangyang Wang, and Bo Li. 2024. Decoding compressed trust: Scrutinizing the trustworthiness of eff… [find](https://scholar.google.com/scholar?q=Junyuan+Hong%2C+Jinhao+Duan%2C+Chenhui+Zhang%2C+Zhangheng+Li%2C+Chulin+Xie%2C+Kelsey+Lieberman%2C+James+Diffenderfer%2C+Brian+R%2E+Bartoldson%2C+Ajay+Kumar+Jaiswal%2C+Kaidi+Xu%2C+Bhavya+Kailkhura%2C+Dan+Hendrycks%2C+Dawn+Song%2C)
24. Clayton J. Hutto and Eric Gilbert. 2014. VADER: A parsimonious rule-based model for sentiment analysis of social media text. In Proceedings of the Eighth International Conference on Weblogs and Social Media, ICWSM 2014, Ann Arbor, Michigan, USA, June 1-4, 2014. The AAAI Press. [find](https://scholar.google.com/scholar?q=Clayton+J%2E+Hutto+and+Eric+Gilbert%2E+2014%2E+VADER%3A+A+parsimonious+rule%2Dbased+model+for+sentiment+analysis+of+social+media+text%2E+In+Proceedings+of+the+Eighth+International+Conference+on+Weblogs+and+Social)
25. Meta. 2025. LLaMA-3.3-70B-Instruct. https://hugg ingface.co/meta-llama/Llama-3.3-70B-Instr uct. Model card; released June 2025. [find](https://scholar.google.com/scholar?q=Meta%2E+2025%2E+LLaMA%2D3%2E3%2D70B%2DInstruct%2E+https%3A%2F%2Fhugg+ingface%2Eco%2Fmeta%2Dllama%2FLlama%2D3%2E3%2D70B%2DInstr+uct%2E+Model+card%3B+released+June+2025%2E)
26. Ajay Kumar Jaiswal, Zhe Gan, Xianzhi Du, Bowen Zhang, Zhangyang Wang, and Yinfei Yang. 2024. Compressing llms: The truth is rarely pure and never simple. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net. [find](https://scholar.google.com/scholar?q=Ajay+Kumar+Jaiswal%2C+Zhe+Gan%2C+Xianzhi+Du%2C+Bowen+Zhang%2C+Zhangyang+Wang%2C+and+Yinfei+Yang%2E+2024%2E+Compressing+llms%3A+The+truth+is+rarely+pure+and+never+simple%2E+In+The+Twelfth+International+Conference+on+Lea)
27. Lingbo Mo, Boshi Wang, Muhao Chen, and Huan Sun. 2024. How trustworthy are open-source llms? an assessment under malicious demonstrations shows their vulnerabilities. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language… [find](https://scholar.google.com/scholar?q=Lingbo+Mo%2C+Boshi+Wang%2C+Muhao+Chen%2C+and+Huan+Sun%2E+2024%2E+How+trustworthy+are+open%2Dsource+llms%3F+an+assessment+under+malicious+demonstrations+shows+their+vulnerabilities%2E+In+Proceedings+of+the+2024+Confer)
28. Elisabeth Kirsten, Ivan Habernal, Vedant Nanda, and Muhammad Bilal Zafar. 2025. The impact of inference acceleration on bias of llms. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2025… [find](https://scholar.google.com/scholar?q=Elisabeth+Kirsten%2C+Ivan+Habernal%2C+Vedant+Nanda%2C+and+Muhammad+Bilal+Zafar%2E+2025%2E+The+impact+of+inference+acceleration+on+bias+of+llms%2E+In+Proceedings+of+the+2025+Conference+of+the+Nations+of+the+Americ)
29. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. [Efficient memory management for large language model serving with pagedattention](https://md.research.elamur.ai/free/en/papers/kwon-2023-efficient-memory-management-large-language/). In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles.
30. Qun Li, Yuan Meng, Chen Tang, Jiacheng Jiang, and Zhi Wang. 2024a. Investigating the impact of quantization on adversarial robustness. CoRR, abs/2404.05639. — [arXiv:2404.05639](https://arxiv.org/abs/2404.05639) [find](https://scholar.google.com/scholar?q=Qun+Li%2C+Yuan+Meng%2C+Chen+Tang%2C+Jiacheng+Jiang%2C+and+Zhi+Wang%2E+2024a%2E+Investigating+the+impact+of+quantization+on+adversarial+robustness%2E+CoRR%2C+abs%2F2404%2E05639%2E+%E2%80%94+%5BarXiv%3A2404%2E05639%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F)
31. Shiyao Li, Xuefei Ning, Luning Wang, Tengxuan Liu, Xiangsheng Shi, Shengen Yan, Guohao Dai, Huazhong Yang, and Yu Wang. 2024b. Evaluating quantized large language models. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net. [find](https://scholar.google.com/scholar?q=Shiyao+Li%2C+Xuefei+Ning%2C+Luning+Wang%2C+Tengxuan+Liu%2C+Xiangsheng+Shi%2C+Shengen+Yan%2C+Guohao+Dai%2C+Huazhong+Yang%2C+and+Yu+Wang%2E+2024b%2E+Evaluating+quantized+large+language+models%2E+In+Forty%2Dfirst+International+)
32. Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Cosgrove, Christopher D. Manning, Christopher Ré, Diana Acosta-Navas, Drew A. Hudson, and 3… [find](https://scholar.google.com/scholar?q=Percy+Liang%2C+Rishi+Bommasani%2C+Tony+Lee%2C+Dimitris+Tsipras%2C+Dilara+Soylu%2C+Michihiro+Yasunaga%2C+Yian+Zhang%2C+Deepak+Narayanan%2C+Yuhuai+Wu%2C+Ananya+Kumar%2C+Benjamin+Newman%2C+Binhang+Yuan%2C+Bobby+Yan%2C+Ce+Zhang%2C+C)
33. Trans. Mach. Learn. Res., 2023. [find](https://scholar.google.com/scholar?q=Trans%2E+Mach%2E+Learn%2E+Res%2E%2C+2023%2E)
34. Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Guangxuan Xiao, and Song Han. 2024. AWQ: activation-aware weight quantization for on-device LLM compression and acceleration. GetMobile Mob. Comput. Commun., 28(4):12–17. [find](https://scholar.google.com/scholar?q=Ji+Lin%2C+Jiaming+Tang%2C+Haotian+Tang%2C+Shang+Yang%2C+Guangxuan+Xiao%2C+and+Song+Han%2E+2024%2E+AWQ%3A+activation%2Daware+weight+quantization+for+on%2Ddevice+LLM+compression+and+acceleration%2E+GetMobile+Mob%2E+Comput%2E+Com)
35. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. [Roberta: A robustly optimized BERT pretraining approach](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/). CoRR, abs/1907.11692.
36. Meta. 2024. LLaMA-3.1-8B-Instruct. https://hugg ingface.co/meta-llama/Llama-3.1-8B-Instr uct. Model card; released July 2024. [find](https://scholar.google.com/scholar?q=Meta%2E+2024%2E+LLaMA%2D3%2E1%2D8B%2DInstruct%2E+https%3A%2F%2Fhugg+ingface%2Eco%2Fmeta%2Dllama%2FLlama%2D3%2E1%2D8B%2DInstr+uct%2E+Model+card%3B+released+July+2024%2E)
37. Moin Nadeem, Anna Bethke, and Siva Reddy. 2021. [Stereoset: Measuring stereotypical bias in pretrained language models](https://md.research.elamur.ai/free/en/papers/nadeem-2021-stereoset-measuring-stereotypical-bias-pretrained/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 202…
38. OpenAI. 2023. [GPT-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). abs/2303.08774.
39. CoRR, Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Jana Thompson, Phu Mon Htut, and Samuel R. Bowman. 2022. [BBQ: A hand-built bias benchmark for question answering](https://md.research.elamur.ai/free/en/papers/parrish-2022-bbq-hand-built-bias-benchmark/). In Findings of the Association for Computational Linguistics: ACL 2022, Dublin, Ireland, May 22-27, 2…
40. Qwen Team. 2024. Qwen2.5-14B-Instruct. [https://huggingface.co/Qwen/Qwen2.5-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-14B-Instruct). Model card; released September 2024. [↗](https://huggingface.co/Qwen/Qwen2.5-14B-Instruct)
41. Stefan Riezler and John T. Maxwell III. 2005. [On some pitfalls in automatic evaluation and significance testing for MT](https://md.research.elamur.ai/free/en/papers/riezler-2005-some-pitfalls-automatic-evaluation-significance/). In Proceedings of the Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization@ACL 2005, Ann Arbor, Michigan, USA, June 29, 2005, pages…
42. Alex Tamkin, Amanda Askell, Liane Lovitt, Esin Durmus, Nicholas Joseph, Shauna Kravec, Karina Nguyen, Jared Kaplan, and Deep Ganguli. 2023. Evaluating and mitigating discrimination in language model decisions. CoRR, abs/2312.03689. [find](https://scholar.google.com/scholar?q=Alex+Tamkin%2C+Amanda+Askell%2C+Liane+Lovitt%2C+Esin+Durmus%2C+Nicholas+Joseph%2C+Shauna+Kravec%2C+Karina+Nguyen%2C+Jared+Kaplan%2C+and+Deep+Ganguli%2E+2023%2E+Evaluating+and+mitigating+discrimination+in+language+model+d)
43. Boxin Wang, Weixin Chen, Hengzhi Pei, Chulin Xie, Mintong Kang, Chenhui Zhang, Chejian Xu, Zidi Xiong, Ritik Dutta, Rylan Schaeffer, Sang T. Truong, Simran Arora, Mantas Mazeika, Dan Hendrycks, Zinan Lin, Yu Cheng, Sanmi Koyejo, Dawn Song, and Bo Li. 2023. [Decodingtrust: A comprehensive assessment o…](https://md.research.elamur.ai/free/en/papers/wang-2023-decodingtrust-comprehensive-assessment-trustworthiness-gpt/)
44. Xinpeng Wang, Bolei Ma, Chengzhi Hu, Leon Weber-Genzel, Paul Röttger, Frauke Kreuter, Dirk Hovy, and Barbara Plank. 2024. ["my answer is c": First-token probabilities do not match text answers in instruction-tuned language models](https://md.research.elamur.ai/free/en/papers/wang-2024-my-answer-is-c-first/). In Findings of the Association for Computational Linguistics, ACL 2024…
45. Guangxuan Xiao, Ji Lin, Mickaël Seznec, Hao Wu, Julien Demouth, and Song Han. 2023. Smoothquant: Accurate and efficient post-training quantization for large language models. In International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings… [find](https://scholar.google.com/scholar?q=Guangxuan+Xiao%2C+Ji+Lin%2C+Micka%C3%ABl+Seznec%2C+Hao+Wu%2C+Julien+Demouth%2C+and+Song+Han%2E+2023%2E+Smoothquant%3A+Accurate+and+efficient+post%2Dtraining+quantization+for+large+language+models%2E+In+International+Conferenc)
46. Guangxuan Xu and Qingyuan Hu. 2022. Can model compression improve NLP fairness. CoRR, abs/2201.08542. — [arXiv:2201.08542](https://arxiv.org/abs/2201.08542) [find](https://scholar.google.com/scholar?q=Guangxuan+Xu+and+Qingyuan+Hu%2E+2022%2E+Can+model+compression+improve+NLP+fairness%2E+CoRR%2C+abs%2F2201%2E08542%2E+%E2%80%94+%5BarXiv%3A2201%2E08542%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2201%2E08542%29)
47. Zhichao Xu, Ashim Gupta, Tao Li, Oliver Bentham, and Vivek Srikumar. 2024. Beyond perplexity: Multi-dimensional safety evaluation of LLM compression. In Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12-16, 2024, pages 15359–15396. Association fo… [find](https://scholar.google.com/scholar?q=Zhichao+Xu%2C+Ashim+Gupta%2C+Tao+Li%2C+Oliver+Bentham%2C+and+Vivek+Srikumar%2E+2024%2E+Beyond+perplexity%3A+Multi%2Ddimensional+safety+evaluation+of+LLM+compression%2E+In+Findings+of+the+Association+for+Computational+L)
48. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, and 22 others. 2024. Qwen2.5 technical report. CoRR, abs/2412.1511… [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Haoran+Wei%2C+Huan+Lin%2C+Jian+Yang%2C+Jianhong+Tu%2C+Jianwei+Zhang%2C+Jianxin+Yang%2C+Jiaxi+Yang%2C+Jin)
49. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018. [Gender bias in coreference resolution: Evaluation and debiasing methods](https://md.research.elamur.ai/free/en/papers/zhao-2018-gender-bias-coreference-resolution-evaluation/). In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
50. Kun Zhou, Yutao Zhu, Zhipeng Chen, Wentong Chen, Wayne Xin Zhao, Xu Chen, Yankai Lin, Ji-Rong Wen, and Jiawei Han. 2023. Don’t make your LLM an evaluation benchmark cheater. CoRR, abs/2311.01964. [find](https://scholar.google.com/scholar?q=Kun+Zhou%2C+Yutao+Zhu%2C+Zhipeng+Chen%2C+Wentong+Chen%2C+Wayne+Xin+Zhao%2C+Xu+Chen%2C+Yankai+Lin%2C+Ji%2DRong+Wen%2C+and+Jiawei+Han%2E+2023%2E+Don%E2%80%99t+make+your+LLM+an+evaluation+benchmark+cheater%2E+CoRR%2C+abs%2F2311%2E01964%2E)
51. Zixuan Zhou, Xuefei Ning, Ke Hong, Tianyu Fu, Jiaming Xu, Shiyao Li, Yuming Lou, Luning Wang, Zhihang Yuan, Xiuhong Li, Shengen Yan, Guohao Dai, Xiao-Ping Zhang, Yuhan Dong, and Yu Wang. 2024. A survey on efficient inference for large language models. CoRR, abs/2404.14294. [find](https://scholar.google.com/scholar?q=Zixuan+Zhou%2C+Xuefei+Ning%2C+Ke+Hong%2C+Tianyu+Fu%2C+Jiaming+Xu%2C+Shiyao+Li%2C+Yuming+Lou%2C+Luning+Wang%2C+Zhihang+Yuan%2C+Xiuhong+Li%2C+Shengen+Yan%2C+Guohao+Dai%2C+Xiao%2DPing+Zhang%2C+Yuhan+Dong%2C+and+Yu+Wang%2E+2024%2E+A+surve)

*Updated 31 October 2026*