---
title: "Hallucinations as Orthogonal Noise — paper by Zhao et al. (2026)"
description: "They argue that some attention heads inject components lying outside the subspace defined by the context, and that this… 41 references."
authors: ["Mingkuan Zhao", "Wentao Hu", "Tianchen Huang", "Yuheng Min", "Suquan Chen", "Yide Gao", "Yanbo Zhai", "Shuangyong Song", "Xuelong Li"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.1822/"
doi: "10.18653/v1/2026.findings-acl.1822"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhao-2026-hallucinations-as-orthogonal-noise-inference/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhao-2026-hallucinations-as-orthogonal-noise-inference/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/35/) / Hallucinations as Orthogonal Noise: Inference-Time Manifold Alignment via Dynamic Contextual Orthogonalization

# Hallucinations as Orthogonal Noise: Inference-Time Manifold Alignment via Dynamic Contextual Orthogonalization

**Authors:** [Mingkuan Zhao](https://md.research.elamur.ai/free/en/authors/mingkuan-zhao/), [Wentao Hu](https://md.research.elamur.ai/free/en/authors/wentao-hu/), [Tianchen Huang](https://md.research.elamur.ai/free/en/authors/tianchen-huang/), [Yuheng Min](https://md.research.elamur.ai/free/en/authors/yuheng-min/), [Suquan Chen](https://md.research.elamur.ai/free/en/authors/suquan-chen/), [Yide Gao](https://md.research.elamur.ai/free/en/authors/yide-gao/), Yanbo Zhai, [Shuangyong Song](https://md.research.elamur.ai/free/en/authors/shuangyong-song/) and [Xuelong Li](https://md.research.elamur.ai/free/en/authors/xuelong-li/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.1822/) · [PDF](https://aclanthology.org/2026.findings-acl.1822.pdf) · [DOI 10.18653/v1/2026.findings-acl.1822](https://doi.org/10.18653/v1/2026.findings-acl.1822) · [OpenAlex](https://openalex.org/W7166859649)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper addresses LLM hallucinations, where outputs contradict the given context or logical constraints. The authors frame this geometrically, using the Linear Representation Hypothesis. They argue that some attention heads inject components lying outside the subspace defined by the context, and that this "orthogonal noise" pushes the residual stream off its semantic manifold. Their method, Dynamic Contextual Orthogonalization (DCO), needs no training and runs at inference time. At each layer, the normalized input residual stream serves as a context anchor. Each head's output is split into a part parallel to this anchor and a perpendicular part. A layer-wise Z-score rule then dampens only those perpendicular parts that are statistical outliers. On Llama-3-8B and 70B, DCO is reported to beat intervention baselines such as ITI and DoLa on contextual faithfulness, while keeping knowledge-intensive accuracy intact. This matters because it targets the usual trade-off between suppressing hallucination and losing parametric knowledge, and it does so cheaply within a single forward pass.

## Key points
- **Models and benchmarks:**
  - Models: Llama-3-8B and Llama-3-70B.
  - Faithfulness benchmarks: XSum, NQ-Swap and IFEval.
  - Knowledge retention benchmarks: TriviaQA, TruthfulQA and MuSiQue.
- **Baselines:** static steering (ITI) and logit-contrast decoding (DoLa, CAD). DeCoRe and CD are discussed as related work.
- **Method steps:** the context anchor is the RMSNorm-normalized input residual of the layer, rescaled to unit length. Each head's output is projected onto this anchor, and outlier orthogonal components are attenuated using Z-scores computed per layer.
- **Efficiency:** the added cost is linear in d_model and fits in one inference pass. Contrastive decoding methods, by comparison, need extra passes.
- **Qualitative case:** on a counterfactual "Banana Inc." context, DCO stops the model from reverting to "Apple"/"iPhone".
- **Code:** github.com/Harry-Miral/DCO.
- **Limitation:** the notion of orthogonality is less clear in fully context-free, closed-book generation.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Tsinghua University](https://md.research.elamur.ai/free/en/universities/tsinghua-university/) — University, Beijing, China · 415 works · 94,028 citations
- [University of Science and Technology of China](https://md.research.elamur.ai/free/en/universities/university-of-science-and-technology-of-china/) — University, Hefei, China · 145 works · 114,409 citations
- [Xi'an Jiaotong University](https://md.research.elamur.ai/free/en/universities/xi-an-jiaotong-university/) — University, Xi'an, China · 42 works · 36,431 citations
- [China Telecom](https://md.research.elamur.ai/free/en/companies/china-telecom-cn/) — Company, Beijing, China · 9 works · 16 citations
- [China Telecom (China)](https://md.research.elamur.ai/free/en/companies/china-telecom/) — Company, Changsha, China · 8 works · 15 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

## Cite

Cite as: Mingkuan Zhao, Wentao Hu, Tianchen Huang, Yuheng Min, Suquan Chen, Yide Gao, Yanbo Zhai, Shuangyong Song and Xuelong Li. 2026. Hallucinations as Orthogonal Noise: Inference-Time Manifold Alignment via Dynamic Contextual Orthogonalization. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhao2026hallucinations,
  title = {{Hallucinations as Orthogonal Noise: Inference-Time Manifold Alignment via Dynamic Contextual Orthogonalization}},
  author = {Mingkuan Zhao, Wentao Hu, Tianchen Huang, Yuheng Min, Suquan Chen, Yide Gao, Yanbo Zhai, Shuangyong Song and Xuelong Li},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1822},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1822}
}
```

- [← previous HalluAudio: A Comprehensive Benchmark for Hallucination Detection in Large Audio-Language Models](https://md.research.elamur.ai/free/en/papers/zhao-2026-halluaudio-comprehensive-benchmark-hallucination-detection/)
- [next → Halluscoring 2026: The first shared task on llms hallucination detection and answer verification](https://md.research.elamur.ai/free/en/papers/alansari-2026-halluscoring-2026-first-shared-task/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/35/)

## References

41 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Shiqi Chen, Miao Xiong, Junteng Liu, Zhengxuan Wu, Teng Xiao, Siyang Gao, and Junxian He. 2024. In-context sharpness as alerts: An inner representation perspective for hallucination mitigation. arXiv preprint arXiv:2403.01548. — [arXiv:2403.01548](https://arxiv.org/abs/2403.01548) [find](https://scholar.google.com/scholar?q=Shiqi+Chen%2C+Miao+Xiong%2C+Junteng+Liu%2C+Zhengxuan+Wu%2C+Teng+Xiao%2C+Siyang+Gao%2C+and+Junxian+He%2E+2024%2E+In%2Dcontext+sharpness+as+alerts%3A+An+inner+representation+perspective+for+hallucination+mitigation%2E+arXiv+)
2. Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James Glass, and Pengcheng He. 2023. [Dola: Decoding by contrasting layers improves factuality in large language models](https://md.research.elamur.ai/free/en/papers/chuang-2023-dola-decoding-contrasting-layers-improves/). arXiv preprint arXiv:2309.03883.
3. Guy Dar, Mor Geva, Ankit Gupta, and Jonathan Berant. 2023. [Analyzing transformers in embedding space](https://md.research.elamur.ai/free/en/papers/dar-2023-analyzing-transformers-embedding-space/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 16124–16170, Toronto, Canada. Association for Computational Linguistics.
4. Shangbin Feng, Vidhisha Balachandran, Yuyang Bai, and Yulia Tsvetkov. 2023. FactKB: Generalizable factuality evaluation using language models enhanced with factual knowledge. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 933–952, Singapore. Associat… [find](https://scholar.google.com/scholar?q=Shangbin+Feng%2C+Vidhisha+Balachandran%2C+Yuyang+Bai%2C+and+Yulia+Tsvetkov%2E+2023%2E+FactKB%3A+Generalizable+factuality+evaluation+using+language+models+enhanced+with+factual+knowledge%2E+In+Proceedings+of+the+202)
5. Aryo Pradipta Gema, Chen Jin, Ahmed Abdulaal, Tom Diethe, Philip Teare, Beatrice Alex, Pasquale Minervini, and Amrutha Saseendran. 2024. Decore: Decoding by contrasting retrieval heads to mitigate hallucinations. arXiv preprint arXiv:2410.18860. [find](https://scholar.google.com/scholar?q=Aryo+Pradipta+Gema%2C+Chen+Jin%2C+Ahmed+Abdulaal%2C+Tom+Diethe%2C+Philip+Teare%2C+Beatrice+Alex%2C+Pasquale+Minervini%2C+and+Amrutha+Saseendran%2E+2024%2E+Decore%3A+Decoding+by+contrasting+retrieval+heads+to+mitigate+hal)
6. Ariel Gera, Roni Friedman, Ofir Arviv, Chulaka Gunasekara, Benjamin Sznajder, Noam Slonim, and Eyal Shnarch. 2023. The benefits of bad advice: Autocontrastive decoding across model layers. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pape… [find](https://scholar.google.com/scholar?q=Ariel+Gera%2C+Roni+Friedman%2C+Ofir+Arviv%2C+Chulaka+Gunasekara%2C+Benjamin+Sznajder%2C+Noam+Slonim%2C+and+Eyal+Shnarch%2E+2023%2E+The+benefits+of+bad+advice%3A+Autocontrastive+decoding+across+model+layers%2E+In+Proceedi)
7. Mor Geva, Jasmijn Bastings, Katja Filippova, and Amir Globerson. 2023. Dissecting recall of factual associations in auto-regressive language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 12216–12235, Singapore. Association for Computational… [find](https://scholar.google.com/scholar?q=Mor+Geva%2C+Jasmijn+Bastings%2C+Katja+Filippova%2C+and+Amir+Globerson%2E+2023%2E+Dissecting+recall+of+factual+associations+in+auto%2Dregressive+language+models%2E+In+Proceedings+of+the+2023+Conference+on+Empirical+)
8. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 o… — [The Llama 3 Herd of Models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/)
9. Wes Gurnee and Max Tegmark. 2024. models represent space and time. arXiv:2310.02207. [find](https://scholar.google.com/scholar?q=Wes+Gurnee+and+Max+Tegmark%2E+2024%2E+models+represent+space+and+time%2E+arXiv%3A2310%2E02207%2E)
10. Language Preprint, Zhongjiang He, Zihan Wang, Xinzhang Liu, Shixuan Liu, Yitong Yao, Yuyao Huang, Xuelong Li, Yongxiang Li, Zhonghao Che, Zhaoxi Zhang, Yan Wang, Xin Wang, Luwen Pu, Huinan Xu, Ruiyu Fang, Yu Zhao, Jie Zhang, Xiaomeng Huang, Zhilong Lu, and 17 others. 2024. [Telechat technical report](https://md.research.elamur.ai/free/en/papers/he-2024-telechat-technical-report/).…
11. Evan Hernandez, Arnab Sen Sharma, Tal Haklay, Kevin Meng, Martin Wattenberg, Jacob Andreas, Yonatan Belinkov, and David Bau. 2024. [Linearity of relation decoding in transformer language models](https://md.research.elamur.ai/free/en/papers/hernandez-2023-linearity-relation-decoding-transformer-language/). Preprint, arXiv:2308.09124.
12. Wentao Hu, Mingkuan Zhao, Shuangyong Song, Xiaoyan Zhu, Xin Lai, and Jiayin Wang. 2025. Mosaic pruning: A hierarchical framework for generalizable pruning of mixture-of-experts models. Preprint, arXiv:2511.19822. — [arXiv:2511.19822](https://arxiv.org/abs/2511.19822) [find](https://scholar.google.com/scholar?q=Wentao+Hu%2C+Mingkuan+Zhao%2C+Shuangyong+Song%2C+Xiaoyan+Zhu%2C+Xin+Lai%2C+and+Jiayin+Wang%2E+2025%2E+Mosaic+pruning%3A+A+hierarchical+framework+for+generalizable+pruning+of+mixture%2Dof%2Dexperts+models%2E+Preprint%2C+arXiv)
13. Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. 2017. [Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension](https://md.research.elamur.ai/free/en/papers/joshi-2017-triviaqa-large-scale-distantly-supervised/). Preprint, arXiv:1705.03551.
14. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. [Natural…](https://md.research.elamur.ai/free/en/papers/kwiatkowski-2019-natural-questions-benchmark-question-answering/)
15. Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. 2024a. [Inference-time intervention: Eliciting truthful answers from a language model](https://md.research.elamur.ai/free/en/papers/li-2023-inference-time-intervention-eliciting-truthful/). Preprint, arXiv:2306.03341.
16. Xiang Li, Yiqun Yao, Xin Jiang, Xuezhi Fang, Chao Wang, Xinzhang Liu, Zihan Wang, Yu Zhao, Xin Wang, Yuyao Huang, Shuangyong Song, Yongxiang Li, Zheng Zhang, Bo Zhao, Aixin Sun, Yequan Wang, Zhongjiang He, Zhongyuan Wang, Xuelong Li, and Tiejun Huang. 2024b. 52b to 1t: Lessons learned via tele-flm s… — [arXiv:2407.02783](https://arxiv.org/abs/2407.02783) [find](https://scholar.google.com/scholar?q=Xiang+Li%2C+Yiqun+Yao%2C+Xin+Jiang%2C+Xuezhi+Fang%2C+Chao+Wang%2C+Xinzhang+Liu%2C+Zihan+Wang%2C+Yu+Zhao%2C+Xin+Wang%2C+Yuyao+Huang%2C+Shuangyong+Song%2C+Yongxiang+Li%2C+Zheng+Zhang%2C+Bo+Zhao%2C+Aixin+Sun%2C+Yequan+Wang%2C+Zhongjian)
17. Xiang Li, Yiqun Yao, Xin Jiang, Xuezhi Fang, Chao Wang, Xinzhang Liu, Zihan Wang, Yu Zhao, Xin Wang, Yuyao Huang, Shuangyong Song, Yongxiang Li, Zheng Zhang, Bo Zhao, Aixin Sun, Yequan Wang, Zhongjiang He, Zhongyuan Wang, Xuelong Li, and Tiejun Huang. 2024c. Tele-flm technical report. Preprint, arXi… — [arXiv:2404.16645](https://arxiv.org/abs/2404.16645) [find](https://scholar.google.com/scholar?q=Xiang+Li%2C+Yiqun+Yao%2C+Xin+Jiang%2C+Xuezhi+Fang%2C+Chao+Wang%2C+Xinzhang+Liu%2C+Zihan+Wang%2C+Yu+Zhao%2C+Xin+Wang%2C+Yuyao+Huang%2C+Shuangyong+Song%2C+Yongxiang+Li%2C+Zheng+Zhang%2C+Bo+Zhao%2C+Aixin+Sun%2C+Yequan+Wang%2C+Zhongjian)
18. Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, and Mike Lewis. 2023. [Contrastive decoding: Open-ended text generation as optimization](https://md.research.elamur.ai/free/en/papers/li-2022-contrastive-decoding-open-ended-text/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: L…
19. Zhongqiu Li, Shiquan Wang, Ruiyu Fang, Mengjiao Bao, Zhenhe Wu, Shuangyong Song, Yongxiang Li, and Zhongjiang He. 2025. Mr-uie: Multi-perspective reasoning with reinforcement learning for universal information extraction. Preprint, arXiv:2509.09082. — [arXiv:2509.09082](https://arxiv.org/abs/2509.09082) [find](https://scholar.google.com/scholar?q=Zhongqiu+Li%2C+Shiquan+Wang%2C+Ruiyu+Fang%2C+Mengjiao+Bao%2C+Zhenhe+Wu%2C+Shuangyong+Song%2C+Yongxiang+Li%2C+and+Zhongjiang+He%2E+2025%2E+Mr%2Duie%3A+Multi%2Dperspective+reasoning+with+reinforcement+learning+for+universal+in)
20. Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. [Truthfulqa: Measuring how models mimic human falsehoods](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/). Preprint, arXiv:2109.07958.
21. Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. [Lost in the middle: How language models use long contexts](https://md.research.elamur.ai/free/en/papers/liu-2023-lost-middle-language-models-use/). Transactions of the Association for Computational Linguistics, 12:157–173.
22. Xinzhang Liu, Chao Wang, Zhihao Yang, Zhuo Jiang, Xuncheng Zhao, Haoran Wang, Lei Li, Dongdong He, Luobin Liu, Kaizhe Yuan, Han Gao, Zihan Wang, Yitong Yao, Sishi Xiong, Wenmin Deng, Haowei He, Kaidong Yu, Yu Zhao, Ruiyu Fang, and 35 others. 2025. Training report of telechat3-moe. Preprint, arXiv:25… — [arXiv:2512.24157](https://arxiv.org/abs/2512.24157) [find](https://scholar.google.com/scholar?q=Xinzhang+Liu%2C+Chao+Wang%2C+Zhihao+Yang%2C+Zhuo+Jiang%2C+Xuncheng+Zhao%2C+Haoran+Wang%2C+Lei+Li%2C+Dongdong+He%2C+Luobin+Liu%2C+Kaizhe+Yuan%2C+Han+Gao%2C+Zihan+Wang%2C+Yitong+Yao%2C+Sishi+Xiong%2C+Wenmin+Deng%2C+Haowei+He%2C+Kaidon)
23. Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh. 2021. [Entity-based knowledge conflicts in question answering](https://md.research.elamur.ai/free/en/papers/longpre-2021-entity-based-knowledge-conflicts-question/). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7052–7063, Online and Punta Cana, Dominican Re…
24. Kevin Meng, David Bau, Alex Andonian, and Yonatan Belinkov. 2022. Locating and editing factual associations in gpt. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22, Red Hook, NY, USA. Curran Associates Inc. [find](https://scholar.google.com/scholar?q=Kevin+Meng%2C+David+Bau%2C+Alex+Andonian%2C+and+Yonatan+Belinkov%2E+2022%2E+Locating+and+editing+factual+associations+in+gpt%2E+In+Proceedings+of+the+36th+International+Conference+on+Neural+Information+Processing)
25. Neel Nanda, Lawrence Chan, Tom Lieberum, Jess Smith, and Jacob Steinhardt. 2023. [Progress measures for grokking via mechanistic interpretability](https://md.research.elamur.ai/free/en/papers/nanda-2023-progress-measures-grokking-mechanistic-interpretability/). Preprint, arXiv:2301.05217.
26. Shashi Narayan, Shay B. Cohen, and Mirella Lapata. 2018. [Don’t give me the details, just the summary!](https://md.research.elamur.ai/free/en/papers/narayan-2018-don-t-give-me-details/) topic-aware convolutional neural networks for extreme summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1797–1807, Brussels, Belgium. As…
27. Catherine Olsson, Nelson Elhage, Neel Nanda, Nicholas Joseph, Nova DasSarma, Tom Henighan, Ben Mann, Amanda Askell, Yuntao Bai, Anna Chen, Tom Conerly, Dawn Drain, Deep Ganguli, Zac Hatfield-Dodds, Danny Hernandez, Scott Johnston, Andy Jones, Jackson Kernion, Liane Lovitt, and 7 others. 2022. [In-con…](https://md.research.elamur.ai/free/en/papers/olsson-2022-context-learning-induction-heads/)
28. Kiho Park, Yo Joong Choe, and Victor Veitch. 2024. [The linear representation hypothesis and the geometry of large language models](https://md.research.elamur.ai/free/en/papers/park-2023-linear-representation-hypothesis-geometry-large/). Preprint, arXiv:2311.03658.
29. Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, and Wen-tau Yih. 2024. [Trusting your evidence: Hallucinate less with context-aware decoding](https://md.research.elamur.ai/free/en/papers/shi-2024-trusting-your-evidence-hallucinate-less/). In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language…
30. Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. Musique: Multi-hop questions via single-hop question composition. Preprint, arXiv:2108.00573. — [arXiv:2108.00573](https://arxiv.org/abs/2108.00573) [find](https://scholar.google.com/scholar?q=Harsh+Trivedi%2C+Niranjan+Balasubramanian%2C+Tushar+Khot%2C+and+Ashish+Sabharwal%2E+2022%2E+Musique%3A+Multi%2Dhop+questions+via+single%2Dhop+question+composition%2E+Preprint%2C+arXiv%3A2108%2E00573%2E+%E2%80%94+%5BarXiv%3A2108%2E00573%5D%28htt)
31. Zihan Wang, XinZhang Liu, Shixuan Liu, Yitong Yao, Yunyao Huang, Mengxiang Li, Zhongjiang He, Yongxian Li, Luwen Pu, Huinan Xu, Chao Wang, and Shuangyong Song. 2024. TeleChat: An open-source billingual large language model. In Proceedings of the 10th SIGHAN Workshop on Chinese Language Processing (S… [find](https://scholar.google.com/scholar?q=Zihan+Wang%2C+XinZhang+Liu%2C+Shixuan+Liu%2C+Yitong+Yao%2C+Yunyao+Huang%2C+Mengxiang+Li%2C+Zhongjiang+He%2C+Yongxian+Li%2C+Luwen+Pu%2C+Huinan+Xu%2C+Chao+Wang%2C+and+Shuangyong+Song%2E+2024%2E+TeleChat%3A+An+open%2Dsource+billingua)
32. Zihan Wang, Xinzhang Liu, Yitong Yao, Chao Wang, Yu Zhao, Zhihao Yang, Wenmin Deng, Kaipeng Jia, Jiaxin Peng, Yuyao Huang, Sishi Xiong, Zhuo Jiang, Kaidong Yu, Xiaohui Hu, Fubei Yao, Ruiyu Fang, Zhuoru Jiang, Ruiting Song, Qiyi Xie, and 19 others. 2025. Technical report of telechat2, telechat2.5 and… — [arXiv:2507.18013](https://arxiv.org/abs/2507.18013) [find](https://scholar.google.com/scholar?q=Zihan+Wang%2C+Xinzhang+Liu%2C+Yitong+Yao%2C+Chao+Wang%2C+Yu+Zhao%2C+Zhihao+Yang%2C+Wenmin+Deng%2C+Kaipeng+Jia%2C+Jiaxin+Peng%2C+Yuyao+Huang%2C+Sishi+Xiong%2C+Zhuo+Jiang%2C+Kaidong+Yu%2C+Xiaohui+Hu%2C+Fubei+Yao%2C+Ruiyu+Fang%2C+Zhuor)
33. Wenhao Wu, Yizhong Wang, Guangxuan Xiao, Hao Peng, and Yao Fu. 2024. [Retrieval head mechanistically explains long-context factuality](https://md.research.elamur.ai/free/en/papers/wu-2024-retrieval-head-mechanistically-explains-long/). Preprint, arXiv:2404.15574.
34. Hongrui Xing, Xinzhang Liu, Zhuo Jiang, Zhihao Yang, Yitong Yao, Zihan Wang, Wenmin Deng, Chao Wang, Shuangyong Song, Wang Yang, Zhongjiang He, and Yongxiang Li. 2025. LLMSR@XLLM25: A language model-based pipeline for structured reasoning data construction. In Proceedings of the 1st Joint Workshop o… [find](https://scholar.google.com/scholar?q=Hongrui+Xing%2C+Xinzhang+Liu%2C+Zhuo+Jiang%2C+Zhihao+Yang%2C+Yitong+Yao%2C+Zihan+Wang%2C+Wenmin+Deng%2C+Chao+Wang%2C+Shuangyong+Song%2C+Wang+Yang%2C+Zhongjiang+He%2C+and+Yongxiang+Li%2E+2025%2E+LLMSR%40XLLM25%3A+A+language+model%2Db)
35. Sishi Xiong, Dakai Wang, Yu Zhao, Jie Zhang, Changzai Pan, Haowei He, Xiangyu Li, Wenhan Chang, Zhongjiang He, Shuangyong Song, and Yongxiang Li. 2025. Tablereasoner: Advancing table reasoning framework with large language models. Preprint, arXiv:2507.08046. — [arXiv:2507.08046](https://arxiv.org/abs/2507.08046) [find](https://scholar.google.com/scholar?q=Sishi+Xiong%2C+Dakai+Wang%2C+Yu+Zhao%2C+Jie+Zhang%2C+Changzai+Pan%2C+Haowei+He%2C+Xiangyu+Li%2C+Wenhan+Chang%2C+Zhongjiang+He%2C+Shuangyong+Song%2C+and+Yongxiang+Li%2E+2025%2E+Tablereasoner%3A+Advancing+table+reasoning+framewo)
36. Lei Yu, Meng Cao, Jackie CK Cheung, and Yue Dong. 2024. Mechanistic understanding and mitigation of language model non-factual hallucinations. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 7943–7956, Miami, Florida, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Lei+Yu%2C+Meng+Cao%2C+Jackie+CK+Cheung%2C+and+Yue+Dong%2E+2024%2E+Mechanistic+understanding+and+mitigation+of+language+model+non%2Dfactual+hallucinations%2E+In+Findings+of+the+Association+for+Computational+Linguist)
37. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Evaluating text generation with bert. Preprint, arXiv:1904.09675. [find](https://scholar.google.com/scholar?q=Tianyi+Zhang%2C+Varsha+Kishore%2C+Felix+Wu%2C+Kilian+Q%2E+Weinberger%2C+and+Yoav+Artzi%2E+2020%2E+Bertscore%3A+Evaluating+text+generation+with+bert%2E+Preprint%2C+arXiv%3A1904%2E09675%2E)
38. Deji Zhao, Donghong Han, Jia Wu, Zhongjiang He, Bo Ning, Ye Yuan, Yongxiang Li, Chao Wang, and Shuangyong Song. 2025a. Enhancing math reasoning ability of large language models via computation logic graphs. Knowledge-Based Systems, 325:113905. [find](https://scholar.google.com/scholar?q=Deji+Zhao%2C+Donghong+Han%2C+Jia+Wu%2C+Zhongjiang+He%2C+Bo+Ning%2C+Ye+Yuan%2C+Yongxiang+Li%2C+Chao+Wang%2C+and+Shuangyong+Song%2E+2025a%2E+Enhancing+math+reasoning+ability+of+large+language+models+via+computation+logic+g)
39. Mingkuan Zhao, Wentao Hu, Jiayin Wang, Xin Lai, Tianchen Huang, Yuheng Min, Rui Yan, and Xiaoyan Zhu. 2025b. Making every head count: Sparse attention without the speed-performance trade-off. Preprint, arXiv:2511.09596. — [arXiv:2511.09596](https://arxiv.org/abs/2511.09596) [find](https://scholar.google.com/scholar?q=Mingkuan+Zhao%2C+Wentao+Hu%2C+Jiayin+Wang%2C+Xin+Lai%2C+Tianchen+Huang%2C+Yuheng+Min%2C+Rui+Yan%2C+and+Xiaoyan+Zhu%2E+2025b%2E+Making+every+head+count%3A+Sparse+attention+without+the+speed%2Dperformance+trade%2Doff%2E+Preprint)
40. Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. 2023. [Instruction-following evaluation for large language models](https://md.research.elamur.ai/free/en/papers/zhou-2023-instruction-following-evaluation-large-language/). ArXiv, abs/2311.07911.
41. Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, and 2 others. 2025. [Representa…](https://md.research.elamur.ai/free/en/papers/zou-2023-representation-engineering-top-down-approach/)

*Updated 2 November 2026*