--- title: "Lost in Quantization — paper by Silva et al. (2026)" description: "The authors test Meta-Llama-3-8B on English and Brazilian Portuguese (PT-BR) under three regimes: a BF16 reference… 6 references." authors: ["Guilherme Silva", "Pedro Silva", "Matheus Peixoto", "Gladston Moreira", "Eduardo Luz"] year: 2026 venue: "Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1" source: "https://aclanthology.org/2026.propor-1.108/" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/silva-2026-lost-quantization-activation-outliers-explain/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/silva-2026-lost-quantization-activation-outliers-explain/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/35/) / Lost in Quantization: Activation Outliers Explain Language-Specific FP8 Sensitivity in Llama-3 # Lost in Quantization: Activation Outliers Explain Language-Specific FP8 Sensitivity in Llama-3 **Authors:** [Guilherme Silva](https://md.research.elamur.ai/free/en/authors/guilherme-silva/), [Pedro Silva](https://md.research.elamur.ai/free/en/authors/pedro-silva/), [Matheus Peixoto](https://md.research.elamur.ai/free/en/authors/matheus-peixoto/), [Gladston Moreira](https://md.research.elamur.ai/free/en/authors/gladston-moreira/) and [Eduardo Luz](https://md.research.elamur.ai/free/en/authors/eduardo-luz/) · **Year:** 2026 · **Venue:** [Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1](https://md.research.elamur.ai/free/en/venues/propor/) ([International Conference on Computational Processing of Portuguese 2026](https://md.research.elamur.ai/free/en/venues/propor/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.propor-1.108/) · [PDF](https://aclanthology.org/2026.propor-1.108.pdf) ## Summary Most evidence that 8-bit inference is nearly lossless comes from English evaluations, so it is unclear whether low-precision formats affect languages equally. The authors test Meta-Llama-3-8B on English and Brazilian Portuguese (PT-BR) under three regimes: - a BF16 reference; - bitsandbytes INT8, which keeps outlier dimensions in higher precision (LLM.int8()-style); - a plain cast of the weights to FP8 E4M3 with no calibration or scaling. This is meant as a stress test, not a production setup. The test texts come from Project Gutenberg: Hamlet for English and Dom Casmurro for PT-BR. Each is cut to 200,000 characters and split deterministically into 64 segments of 2048 tokens. The authors compare perplexity within each language. They also log the largest absolute attention activation per batch using forward hooks. INT8 leaves perplexity unchanged for both languages. Naive FP8 raises it by about 18% for English but only 3.9% for PT-BR. The authors attribute this to English producing rarer, higher activation peaks, while PT-BR peaks cluster in a narrower band. The work suggests FP8 scaling should account for language- and domain-specific outlier profiles. ## Key points - Perplexity (BF16 → INT8 → FP8): English 3.45 → 3.45 → 4.08; PT-BR 13.25 → 13.25 → 13.77. Absolute values are not compared across languages, only relative degradation. - Attention-activation maxima, English vs. PT-BR: mean 6.97 vs. 7.23, P99 21.88 vs. 22.38, max 35.50 vs. 31.88. PT-BR has slightly more high values on average, while English has the single largest spikes. - The activation histograms are bimodal: most values fall around 1–3, with a second cluster at 15–20. English has a sparse tail extending past 35. - The setup ran inference only, with no fine-tuning, on NVIDIA H200-class GPUs. A KL-divergence metric is described as optional but its results are not reported. Code is available at github.com/silvagal/llama3_fp8_language_sensitivity. - Limitations noted by the authors: one model, two languages, and literary texts that differ in style and era. The FP8 configuration is intentionally uncalibrated, and calibrated, outlier-aware FP8 is left for future work. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* 📚 Shelves (2): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works 🧭 Topics (1): [🪶 Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/) · 242 works ## Cite Cite as: Guilherme Silva, Pedro Silva, Matheus Peixoto, Gladston Moreira and Eduardo Luz. 2026. Lost in Quantization: Activation Outliers Explain Language-Specific FP8 Sensitivity in Llama-3. Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{silva2026lost, title = {{Lost in Quantization: Activation Outliers Explain Language-Specific FP8 Sensitivity in Llama-3}}, author = {Guilherme Silva, Pedro Silva, Matheus Peixoto, Gladston Moreira and Eduardo Luz}, booktitle = {Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1}, year = {2026}, url = {https://aclanthology.org/2026.propor-1.108/} } ``` - [← previous Lost in Expression: Diagnosing Systemic Challenges with Non-Manual Generalization in Sign Language Understanding Tasks](https://md.research.elamur.ai/free/en/papers/sazonov-2026-lost-expression-diagnosing-systemic-challenges/) - [next → Lost in the Prompt Order: Revealing the Limitations of Causal Attention in Language Models](https://md.research.elamur.ai/free/en/papers/ok-2026-lost-prompt-order-revealing-limitations/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/35/) ## References 6 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022. Gpt3.int8(): 8-bit matrix multiplication for transformers at scale. Advances in Neural Information Processing Systems, 35:30318–30332. [find](https://scholar.google.com/scholar?q=Tim+Dettmers%2C+Mike+Lewis%2C+Younes+Belkada%2C+and+Luke+Zettlemoyer%2E+2022%2E+Gpt3%2Eint8%28%29%3A+8%2Dbit+matrix+multiplication+for+transformers+at+scale%2E+Advances+in+Neural+Information+Processing+Systems%2C+35%3A30318%E2%80%9330) 2. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783. 3. Amir Gholami, Sehoon Kim, Zhen Dong, Zhewei Yao, Michael W Mahoney, and Kurt Keutzer. 2022. A survey of quantization methods for efficient neural network inference. In Low-Power Computer Vision, pages 291–326. [find](https://scholar.google.com/scholar?q=Amir+Gholami%2C+Sehoon+Kim%2C+Zhen+Dong%2C+Zhewei+Yao%2C+Michael+W+Mahoney%2C+and+Kurt+Keutzer%2E+2022%2E+A+survey+of+quantization+methods+for+efficient+neural+network+inference%2E+In+Low%2DPower+Computer+Vision%2C+pages) 4. Pratik Joshi, Sebastin Santy, Amar Budhiraja, Kalika Bali, and Monojit Choudhury. 2020. [The state and fate of linguistic diversity and inclusion in the NLP world](https://md.research.elamur.ai/free/en/papers/joshi-2020-state-fate-linguistic-diversity-inclusion/). Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6282–6293. 5. Olga Kovaleva, Alexey Romanov, Anna Rogers, and Anna Rumshisky. 2019. [Revealing the dark secrets of BERT](https://md.research.elamur.ai/free/en/papers/kovaleva-2019-revealing-dark-secrets-bert/). Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4365–4374. 6. Paulius Micikevicius, Dusan Stosic, Neil Burgess, Marius Cornea, Pradeep Dubey, Richard Grisenthwaite, Sangwon Ha, Alexander Heinecke, Patrick Judd, John Kamalu, and 1 others. 2022. Fp8 formats for deep learning. arXiv preprint arXiv:2209.05433. [find](https://scholar.google.com/scholar?q=Paulius+Micikevicius%2C+Dusan+Stosic%2C+Neil+Burgess%2C+Marius+Cornea%2C+Pradeep+Dubey%2C+Richard+Grisenthwaite%2C+Sangwon+Ha%2C+Alexander+Heinecke%2C+Patrick+Judd%2C+John+Kamalu%2C+and+1+others%2E+2022%2E+Fp8+formats+for+de) *Updated 1 November 2026*