---
title: "KnowRL — paper by Ren et al. (2026) — summary and ideas"
description: "Slow-thinking LLMs such as DeepSeek-R1 distillations reason well but… 6.64% accuracy on SimpleQA. 105 references, cited by 5 papers here."
authors: ["Baochang Ren", "Shuofei Qiao", "Ningyu Zhang", "Da Zheng", "Huajun Chen"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.1840/"
doi: "10.18653/v1/2026.acl-long.1840"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ren-2026-knowrl-exploring-knowledgeable-reinforcement-learning/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ren-2026-knowrl-exploring-knowledgeable-reinforcement-learning/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/3/) / KnowRL: Exploring Knowledgeable Reinforcement Learning for Factuality

# KnowRL: Exploring Knowledgeable Reinforcement Learning for Factuality

**Authors:** Baochang Ren, [Shuofei Qiao](https://md.research.elamur.ai/free/en/authors/shuofei-qiao/), [Ningyu Zhang](https://md.research.elamur.ai/free/en/authors/ningyu-zhang/), [Da Zheng](https://md.research.elamur.ai/free/en/authors/da-zheng/) and [Huajun Chen](https://md.research.elamur.ai/free/en/authors/huajun-chen/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.1840/) · [PDF](https://aclanthology.org/2026.acl-long.1840.pdf) · [DOI 10.18653/v1/2026.acl-long.1840](https://doi.org/10.18653/v1/2026.acl-long.1840) · [OpenAlex](https://openalex.org/W4414685036)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [5](https://md.research.elamur.ai/free/en/papers/ren-2026-knowrl-exploring-knowledgeable-reinforcement-learning/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Slow-thinking LLMs such as DeepSeek-R1 distillations reason well but hallucinate heavily. One reason is that standard RL rewards only the final answer, so invented reasoning steps still get reinforced whenever the answer happens to be right. KnowRL addresses this by adding factual supervision to the reasoning trace itself. Each chain of thought is split into atomic facts, and each fact is checked against retrieved ground-truth knowledge. The share of supported facts becomes a dense reward, which is combined with format and answer-correctness rewards inside GRPO. Because an explicit refusal earns a positive but smaller reward than a correct answer, the model learns to admit ignorance instead of guessing. On SimpleQA, the paper reports a 20.3% reduction in Incorrect Rate for 7B distillation-based models. Reasoning performance on GPQA and AIME 2025 is preserved. The gains also carry over to ChineseSimpleQA, although the knowledge source is English-only. The work matters because it improves factual reliability without the reasoning loss that SFT-based knowledge injection tends to cause.

## Key points
- **Reward design:**
  - Format: +1 if valid, −1 otherwise.
  - Correctness: +2 for a correct answer, +1 for an explicit refusal, −1 for a wrong answer.
  - Fact: fraction of atomic facts supported by the knowledge base, inspired by FactScore.
- **Tooling:** GPT-4o-mini handles fact decomposition, verification and answer evaluation. Relevant knowledge is retrieved with sentence-transformers/gtr-t5-large.
- **Motivation:** DeepSeek-R1-Distill-Qwen-32B reaches only 6.64% accuracy on SimpleQA. Larger distilled models gain on GPQA but not on hallucination resistance.
- **Evaluation:** Base models are Skywork-OR1-7B-Preview and DeepSeek-R1-Distill-Qwen-7B. Baselines are DPO, FactTune-FS, SFT and TruthRL. Results are multi-run (Avg@5, temperature 0.6).
- **Availability and future work:** Code is at github.com/zjunlp/KnowRL. The authors list multimodal (VLM) extension and the mechanism behind cross-lingual transfer as open directions.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Zhejiang University](https://md.research.elamur.ai/free/en/universities/zhejiang-university/) — University, Hangzhou, China · 172 works · 12,988 citations
- [Ant Group](https://md.research.elamur.ai/free/en/companies/ant-group/) — Company, Hangzhou, China · 42 works · 1,131 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,426 works

## Cite

Cite as: Baochang Ren, Shuofei Qiao, Ningyu Zhang, Da Zheng and Huajun Chen. 2026. KnowRL: Exploring Knowledgeable Reinforcement Learning for Factuality. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{ren2026knowrl,
  title = {{KnowRL: Exploring Knowledgeable Reinforcement Learning for Factuality}},
  author = {Baochang Ren, Shuofei Qiao, Ningyu Zhang, Da Zheng and Huajun Chen},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1840},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1840}
}
```

- [← previous KNN-SSD: Enabling Dynamic Self-Speculative Decoding via Nearest Neighbor Layer Set Optimization](https://md.research.elamur.ai/free/en/papers/song-2026-knn-ssd-enabling-dynamic-self/)
- [next → Language-Coupled Reinforcement Learning for Multilingual Retrieval-Augmented Generation](https://md.research.elamur.ai/free/en/papers/qi-2026-language-coupled-reinforcement-learning-multilingual/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/3/)

## References

105 references, 35 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. computational linguistics: technical papers, pages 2503–2514. [find](https://scholar.google.com/scholar?q=computational+linguistics%3A+technical+papers%2C+pages+2503%E2%80%932514%2E)
2. Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. [Semantic parsing on freebase from question-answer pairs](https://md.research.elamur.ai/free/en/papers/berant-2013-semantic-parsing-freebase-question-answer/). In Proceedings of the 2013 conference on empirical methods in natural language processing, pages 1533–1544.
3. Lukas Berglund, Meg Tong, Max Kaufmann, Mikita Balesni, Asa Cooper Stickland, Tomasz Korbak, and Owain Evans. 2023. The reversal curse: Llms trained on" a is b" fail to learn" b is a". arXiv preprint arXiv:2309.12288. [find](https://scholar.google.com/scholar?q=Lukas+Berglund%2C+Meg+Tong%2C+Max+Kaufmann%2C+Mikita+Balesni%2C+Asa+Cooper+Stickland%2C+Tomasz+Korbak%2C+and+Owain+Evans%2E+2023%2E+The+reversal+curse%3A+Llms+trained+on%22+a+is+b%22+fail+to+learn%22+b+is+a%22%2E+arXiv+preprint+)
4. Yuzheng Cai, Siqi Cai, Yuchen Shi, Zihan Xu, Lichao Chen, Yulei Qin, Xiaoyu Tan, Gang Li, Zongyi Li, Haojia Lin, et al. 2025. Training-free group relative policy optimization. arXiv preprint arXiv:2510.08191. — [arXiv:2510.08191](https://arxiv.org/abs/2510.08191) [find](https://scholar.google.com/scholar?q=Yuzheng+Cai%2C+Siqi+Cai%2C+Yuchen+Shi%2C+Zihan+Xu%2C+Lichao+Chen%2C+Yulei+Qin%2C+Xiaoyu+Tan%2C+Gang+Li%2C+Zongyi+Li%2C+Haojia+Lin%2C+et+al%2E+2025%2E+Training%2Dfree+group+relative+policy+optimization%2E+arXiv+preprint+arXiv%3A251)
5. Shicong Cen, Jincheng Mei, Katayoon Goshvadi, Hanjun Dai, Tong Yang, Sherry Yang, Dale Schuurmans, Yuejie Chi, and Bo Dai. 2024. Value-incentivized preference optimization: A unified approach to online and offline rlhf. arXiv preprint arXiv:2405.19320. — [arXiv:2405.19320](https://arxiv.org/abs/2405.19320) [find](https://scholar.google.com/scholar?q=Shicong+Cen%2C+Jincheng+Mei%2C+Katayoon+Goshvadi%2C+Hanjun+Dai%2C+Tong+Yang%2C+Sherry+Yang%2C+Dale+Schuurmans%2C+Yuejie+Chi%2C+and+Bo+Dai%2E+2024%2E+Value%2Dincentivized+preference+optimization%3A+A+unified+approach+to+onlin)
6. Guoxin Chen, Minpeng Liao, Chengxi Li, and Kai Fan. 2024a. Step-level value preference optimization for mathematical reasoning. arXiv preprint arXiv:2406.10858. [find](https://scholar.google.com/scholar?q=Guoxin+Chen%2C+Minpeng+Liao%2C+Chengxi+Li%2C+and+Kai+Fan%2E+2024a%2E+Step%2Dlevel+value+preference+optimization+for+mathematical+reasoning%2E+arXiv+preprint+arXiv%3A2406%2E10858%2E)
7. Howard Chen, Noam Razin, Karthik Narasimhan, and Danqi Chen. 2025a. Retaining by doing: The role of on-policy data in mitigating forgetting. arXiv preprint arXiv:2510.18874. [find](https://scholar.google.com/scholar?q=Howard+Chen%2C+Noam+Razin%2C+Karthik+Narasimhan%2C+and+Danqi+Chen%2E+2025a%2E+Retaining+by+doing%3A+The+role+of+on%2Dpolicy+data+in+mitigating+forgetting%2E+arXiv+preprint+arXiv%3A2510%2E18874%2E)
8. Huajun Chen. 2023. Large knowledge model: Perspectives and challenges. arXiv preprint arXiv:2312.02706. [find](https://scholar.google.com/scholar?q=Huajun+Chen%2E+2023%2E+Large+knowledge+model%3A+Perspectives+and+challenges%2E+arXiv+preprint+arXiv%3A2312%2E02706%2E)
9. Peter Chen, Xiaopeng Li, Ziniu Li, Xi Chen, and Tianyi Lin. 2025b. Spectral policy optimization: Coloring your incorrect reasoning in grpo. In 2nd AI for Math Workshop@ ICML 2025. [find](https://scholar.google.com/scholar?q=Peter+Chen%2C+Xiaopeng+Li%2C+Ziniu+Li%2C+Xi+Chen%2C+and+Tianyi+Lin%2E+2025b%2E+Spectral+policy+optimization%3A+Coloring+your+incorrect+reasoning+in+grpo%2E+In+2nd+AI+for+Math+Workshop%40+ICML+2025%2E)
10. Weizhe Chen, Sven Koenig, and Bistra Dilkina. 2025c. Lspo: Length-aware dynamic sampling for policy optimization in llm reasoning. arXiv preprint arXiv:2510.01459. [find](https://scholar.google.com/scholar?q=Weizhe+Chen%2C+Sven+Koenig%2C+and+Bistra+Dilkina%2E+2025c%2E+Lspo%3A+Length%2Daware+dynamic+sampling+for+policy+optimization+in+llm+reasoning%2E+arXiv+preprint+arXiv%3A2510%2E01459%2E)
11. Iván Arcuschin, Jett Janiak, Robert Krzyzanowski, Senthooran Rajamanoharan, Neel Nanda, and Arthur Conmy. 2025. Chain-of-thought reasoning in the wild is not always faithful. arXiv preprint arXiv:2503.08679. [find](https://scholar.google.com/scholar?q=Iv%C3%A1n+Arcuschin%2C+Jett+Janiak%2C+Robert+Krzyzanowski%2C+Senthooran+Rajamanoharan%2C+Neel+Nanda%2C+and+Arthur+Conmy%2E+2025%2E+Chain%2Dof%2Dthought+reasoning+in+the+wild+is+not+always+faithful%2E+arXiv+preprint+arXiv%3A2503)
12. Xiang Chen, Ningyu Zhang, Xin Xie, Shumin Deng, Yunzhi Yao, Chuanqi Tan, Fei Huang, Luo Si, and Huajun Chen. 2022. Knowprompt: Knowledge-aware prompt-tuning with synergistic optimization for relation extraction. In Proceedings of the ACM Web conference 2022, pages 2778–2788. [find](https://scholar.google.com/scholar?q=Xiang+Chen%2C+Ningyu+Zhang%2C+Xin+Xie%2C+Shumin+Deng%2C+Yunzhi+Yao%2C+Chuanqi+Tan%2C+Fei+Huang%2C+Luo+Si%2C+and+Huajun+Chen%2E+2022%2E+Knowprompt%3A+Knowledge%2Daware+prompt%2Dtuning+with+synergistic+optimization+for+relation+)
13. Junwei Bao, Nan Duan, Zhao Yan, Ming Zhou, and Tiejun Zhao. 2016. [Constraint-based question answering with knowledge graph](https://md.research.elamur.ai/free/en/papers/bao-2016-constraint-based-question-answering-knowledge/). In Proceedings of COLING 2016, the 26th international conference on Xinxi Chen, Li Wang, Wei Wu, Qi Tang, and Yiyao Liu. 2024b. Honest ai: Fine-tuning" small" language models t…
14. Zhenfang Chen, Delin Chen, Rui Sun, Wenjun Liu, and Chuang Gan. 2025d. Scaling autonomous agents via automatic reward modeling and planning. arXiv preprint arXiv:2502.12130. — [arXiv:2502.12130](https://arxiv.org/abs/2502.12130) [find](https://scholar.google.com/scholar?q=Zhenfang+Chen%2C+Delin+Chen%2C+Rui+Sun%2C+Wenjun+Liu%2C+and+Chuang+Gan%2E+2025d%2E+Scaling+autonomous+agents+via+automatic+reward+modeling+and+planning%2E+arXiv+preprint+arXiv%3A2502%2E12130%2E+%E2%80%94+%5BarXiv%3A2502%2E12130%5D%28https)
15. Seungwook Han, Idan Shenfeld, Akash Srivastava, Yoon Kim, and Pulkit Agrawal. 2024. Value augmented sampling for language model alignment and personalization. arXiv preprint arXiv:2405.06639. — [arXiv:2405.06639](https://arxiv.org/abs/2405.06639) [find](https://scholar.google.com/scholar?q=Seungwook+Han%2C+Idan+Shenfeld%2C+Akash+Srivastava%2C+Yoon+Kim%2C+and+Pulkit+Agrawal%2E+2024%2E+Value+augmented+sampling+for+language+model+alignment+and+personalization%2E+arXiv+preprint+arXiv%3A2405%2E06639%2E+%E2%80%94+%5BarXiv)
16. Ruoxi Cheng, Haoxuan Ma, Weixin Wang, Ranjie Duan, Jiexi Liu, Xiaoshuang Jia, Simeng Qin, Xiaochun Cao, Yang Liu, and Xiaojun Jia. 2025a. Inverse reinforcement learning with dynamic reward scaling for llm alignment. arXiv preprint arXiv:2503.18991. — [arXiv:2503.18991](https://arxiv.org/abs/2503.18991) [find](https://scholar.google.com/scholar?q=Ruoxi+Cheng%2C+Haoxuan+Ma%2C+Weixin+Wang%2C+Ranjie+Duan%2C+Jiexi+Liu%2C+Xiaoshuang+Jia%2C+Simeng+Qin%2C+Xiaochun+Cao%2C+Yang+Liu%2C+and+Xiaojun+Jia%2E+2025a%2E+Inverse+reinforcement+learning+with+dynamic+reward+scaling+for)
17. Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, et al. 2024a. [Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems](https://md.research.elamur.ai/free/en/papers/he-2024-olympiadbench-challenging-benchmark-promoting-agi/). arXiv preprint arXiv:2402.14008.
18. Xiaoxue Cheng, Junyi Li, Wayne Xin Zhao, and Ji-Rong Wen. 2025b. Think more, hallucinate less: Mitigating hallucinations via dual process of fast and slow thinking. arXiv preprint arXiv:2501.01306. [find](https://scholar.google.com/scholar?q=Xiaoxue+Cheng%2C+Junyi+Li%2C+Wayne+Xin+Zhao%2C+and+Ji%2DRong+Wen%2E+2025b%2E+Think+more%2C+hallucinate+less%3A+Mitigating+hallucinations+via+dual+process+of+fast+and+slow+thinking%2E+arXiv+preprint+arXiv%3A2501%2E01306%2E)
19. Jujie He, Jiacai Liu, Chris Yuhao Liu, Rui Yan, Chaojie Wang, Peng Cheng, Xiaoyu Zhang, Fuxiang Zhang, Jiacheng Xu, Wei Shen, Siyuan Li, Liang Zeng, Tianwen Wei, Cheng Cheng, Yang Liu, and Yahui Zhou. 2025a. Skywork open reasoner series. Notion Blog. [find](https://scholar.google.com/scholar?q=Jujie+He%2C+Jiacai+Liu%2C+Chris+Yuhao+Liu%2C+Rui+Yan%2C+Chaojie+Wang%2C+Peng+Cheng%2C+Xiaoyu+Zhang%2C+Fuxiang+Zhang%2C+Jiacheng+Xu%2C+Wei+Shen%2C+Siyuan+Li%2C+Liang+Zeng%2C+Tianwen+Wei%2C+Cheng+Cheng%2C+Yang+Liu%2C+and+Yahui+Zhou%2E)
20. Shehzaad Dhuliawala, Mojtaba Komeili, Jing Xu, Roberta Raileanu, Xian Li, Asli Celikyilmaz, and Jason Weston. 2024. [Chain-of-verification reduces hallucination in large language models](https://md.research.elamur.ai/free/en/papers/dhuliawala-2023-chain-verification-reduces-hallucination-large/). In Findings of the Association for Computational Linguistics: ACL 2024, pages 3563–3578.
21. Fei Ding, Baiqiao Wang, Zijian Zeng, and Youwei Wang. 2025. Multi-layer grpo: Enhancing reasoning and self-correction in large language models. arXiv preprint arXiv:2506.04746. — [arXiv:2506.04746](https://arxiv.org/abs/2506.04746) [find](https://scholar.google.com/scholar?q=Fei+Ding%2C+Baiqiao+Wang%2C+Zijian+Zeng%2C+and+Youwei+Wang%2E+2025%2E+Multi%2Dlayer+grpo%3A+Enhancing+reasoning+and+self%2Dcorrection+in+large+language+models%2E+arXiv+preprint+arXiv%3A2506%2E04746%2E+%E2%80%94+%5BarXiv%3A2506%2E04746%5D%28ht)
22. Mucong Ding, Souradip Chakraborty, Vibhu Agrawal, Zora Che, Alec Koppel, Mengdi Wang, Amrit Bedi, and Furong Huang. 2024. Sail: Self-improving efficient online alignment of large language models. arXiv preprint arXiv:2406.15567. — [arXiv:2406.15567](https://arxiv.org/abs/2406.15567) [find](https://scholar.google.com/scholar?q=Mucong+Ding%2C+Souradip+Chakraborty%2C+Vibhu+Agrawal%2C+Zora+Che%2C+Alec+Koppel%2C+Mengdi+Wang%2C+Amrit+Bedi%2C+and+Furong+Huang%2E+2024%2E+Sail%3A+Self%2Dimproving+efficient+online+alignment+of+large+language+models%2E+arXi)
23. Hanze Dong, Wei Xiong, Bo Pang, Haoxiang Wang, Han Zhao, Yingbo Zhou, Nan Jiang, Doyen Sahoo, Caiming Xiong, and Tong Zhang. 2024. Rlhf workflow: From reward modeling to online rlhf. arXiv preprint arXiv:2405.07863. [find](https://scholar.google.com/scholar?q=Hanze+Dong%2C+Wei+Xiong%2C+Bo+Pang%2C+Haoxiang+Wang%2C+Han+Zhao%2C+Yingbo+Zhou%2C+Nan+Jiang%2C+Doyen+Sahoo%2C+Caiming+Xiong%2C+and+Tong+Zhang%2E+2024%2E+Rlhf+workflow%3A+From+reward+modeling+to+online+rlhf%2E+arXiv+preprint+ar)
24. Shangbin Feng, Weijia Shi, Yike Wang, Wenxuan Ding, Vidhisha Balachandran, and Yulia Tsvetkov. 2024. Don’t hallucinate, abstain: Identifying llm knowledge gaps via multi-llm collaboration. arXiv preprint arXiv:2402.00367. [find](https://scholar.google.com/scholar?q=Shangbin+Feng%2C+Weijia+Shi%2C+Yike+Wang%2C+Wenxuan+Ding%2C+Vidhisha+Balachandran%2C+and+Yulia+Tsvetkov%2E+2024%2E+Don%E2%80%99t+hallucinate%2C+abstain%3A+Identifying+llm+knowledge+gaps+via+multi%2Dllm+collaboration%2E+arXiv+prepr)
25. Qianxi He, Qingyu Ren, Shanzhe Lei, Xuhong Wang, and Yingchun Wang. 2025b. Beyond correctness: Confidence-aware reward modeling for enhancing large language model reasoning. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 27215–27231. [find](https://scholar.google.com/scholar?q=Qianxi+He%2C+Qingyu+Ren%2C+Shanzhe+Lei%2C+Xuhong+Wang%2C+and+Yingchun+Wang%2E+2025b%2E+Beyond+correctness%3A+Confidence%2Daware+reward+modeling+for+enhancing+large+language+model+reasoning%2E+In+Proceedings+of+the+2025)
26. Yancheng He, Shilong Li, Jiaheng Liu, Yingshui Tan, Weixun Wang, Hui Huang, Xingyuan Bu, Hangyu Guo, Chengwei Hu, Boren Zheng, et al. 2024b. [Chinese simpleqa: A chinese factuality evaluation for large language models](https://md.research.elamur.ai/free/en/papers/he-2024-chinese-simpleqa-chinese-factuality/). arXiv preprint arXiv:2411.07140.
27. Alex Heyman and Joel Zylberberg. 2025. Reasoning large language model errors arise from hallucinating critical problem features. arXiv preprint arXiv:2505.12151. — [arXiv:2505.12151](https://arxiv.org/abs/2505.12151) [find](https://scholar.google.com/scholar?q=Alex+Heyman+and+Joel+Zylberberg%2E+2025%2E+Reasoning+large+language+model+errors+arise+from+hallucinating+critical+problem+features%2E+arXiv+preprint+arXiv%3A2505%2E12151%2E+%E2%80%94+%5BarXiv%3A2505%2E12151%5D%28https%3A%2F%2Farxiv%2Eorg)
28. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. 2022. [Lora: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). ICLR, 1(2):3.
29. Jian Hu, Jason Klein Liu, Haotian Xu, and Wei Shen. 2025. Reinforce++: An efficient rlhf algorithm with robustness to both prompt and reward models. arXiv preprint arXiv:2501.03262. [find](https://scholar.google.com/scholar?q=Jian+Hu%2C+Jason+Klein+Liu%2C+Haotian+Xu%2C+and+Wei+Shen%2E+2025%2E+Reinforce%2B%2B%3A+An+efficient+rlhf+algorithm+with+robustness+to+both+prompt+and+reward+models%2E+arXiv+preprint+arXiv%3A2501%2E03262%2E)
30. Xidong Feng, Ziyu Wan, Muning Wen, Stephen Marcus McAleer, Ying Wen, Weinan Zhang, and Jun Wang. 2023. Alphazero-like tree-search can guide large language model decoding and training. arXiv preprint arXiv:2309.17179. [find](https://scholar.google.com/scholar?q=Xidong+Feng%2C+Ziyu+Wan%2C+Muning+Wen%2C+Stephen+Marcus+McAleer%2C+Ying+Wen%2C+Weinan+Zhang%2C+and+Jun+Wang%2E+2023%2E+Alphazero%2Dlike+tree%2Dsearch+can+guide+large+language+model+decoding+and+training%2E+arXiv+preprint+a)
31. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, et al. 2025. [A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions](https://md.research.elamur.ai/free/en/papers/huang-2023-survey-hallucination-large-language-models/). ACM Transactions on Information Systems, 43(2):…
32. Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D Goodman. 2025. [Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars](https://md.research.elamur.ai/free/en/papers/gandhi-2025-cognitive-behaviors-that-enable-self/). arXiv preprint arXiv:2503.01307.
33. Yuki Ichihara and Yuu Jinnai. 2025. Auto-weighted group relative preference optimization for multi-objective text generation tasks. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 1134–1147. [find](https://scholar.google.com/scholar?q=Yuki+Ichihara+and+Yuu+Jinnai%2E+2025%2E+Auto%2Dweighted+group+relative+preference+optimization+for+multi%2Dobjective+text+generation+tasks%2E+In+Proceedings+of+the+2025+Conference+on+Empirical+Methods+in+Natura)
34. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025. [Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). arXiv preprint arXiv:2501.12948.
35. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. [Survey of hallucination in natural language generation](https://md.research.elamur.ai/free/en/papers/ji-2022-survey-hallucination-natural-language-generation/). ACM computing surveys, 55(12):1–38.
36. Bowen Jin, Jinsung Yoon, Priyanka Kargupta, Sercan O Arik, and Jiawei Han. 2025a. An empirical study on reinforcement learning for reasoning-search interleaved llm agents. arXiv preprint arXiv:2505.15117. — [arXiv:2505.15117](https://arxiv.org/abs/2505.15117) [find](https://scholar.google.com/scholar?q=Bowen+Jin%2C+Jinsung+Yoon%2C+Priyanka+Kargupta%2C+Sercan+O+Arik%2C+and+Jiawei+Han%2E+2025a%2E+An+empirical+study+on+reinforcement+learning+for+reasoning%2Dsearch+interleaved+llm+agents%2E+arXiv+preprint+arXiv%3A2505%2E15)
37. Junzhe Li, Yutao Cui, Tao Huang, Yinping Ma, Chun Fan, Miles Yang, and Zhao Zhong. 2025b. Mixgrpo: Unlocking flow-based grpo efficiency with mixed ode-sde. arXiv preprint arXiv:2507.21802. — [arXiv:2507.21802](https://arxiv.org/abs/2507.21802) [find](https://scholar.google.com/scholar?q=Junzhe+Li%2C+Yutao+Cui%2C+Tao+Huang%2C+Yinping+Ma%2C+Chun+Fan%2C+Miles+Yang%2C+and+Zhao+Zhong%2E+2025b%2E+Mixgrpo%3A+Unlocking+flow%2Dbased+grpo+efficiency+with+mixed+ode%2Dsde%2E+arXiv+preprint+arXiv%3A2507%2E21802%2E+%E2%80%94+%5BarXiv%3A25)
38. Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. 2025b. [Search-r1: Training llms to reason and leverage search engines with reinforcement learning](https://md.research.elamur.ai/free/en/papers/jin-2025-search-r1-training-llms-reason/). arXiv preprint arXiv:2503.09516.
39. Long-Fei Li, Yu-Yang Qian, Peng Zhao, and Zhi-Hua Zhou. 2025c. Provably efficient online rlhf with one-pass reward modeling. arXiv preprint arXiv:2502.07193. — [arXiv:2502.07193](https://arxiv.org/abs/2502.07193) [find](https://scholar.google.com/scholar?q=Long%2DFei+Li%2C+Yu%2DYang+Qian%2C+Peng+Zhao%2C+and+Zhi%2DHua+Zhou%2E+2025c%2E+Provably+efficient+online+rlhf+with+one%2Dpass+reward+modeling%2E+arXiv+preprint+arXiv%3A2502%2E07193%2E+%E2%80%94+%5BarXiv%3A2502%2E07193%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs)
40. Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, et al. 2022. Language models (mostly) know what they know. arXiv preprint arXiv:2207.05221. [find](https://scholar.google.com/scholar?q=Saurav+Kadavath%2C+Tom+Conerly%2C+Amanda+Askell%2C+Tom+Henighan%2C+Dawn+Drain%2C+Ethan+Perez%2C+Nicholas+Schiefer%2C+Zac+Hatfield%2DDodds%2C+Nova+DasSarma%2C+Eli+Tran%2DJohnson%2C+et+al%2E+2022%2E+Language+models+%28mostly%29+know+w)
41. Katie Kang, Eric Wallace, Claire Tomlin, Aviral Kumar, and Sergey Levine. 2025. [Unfamiliar finetuning examples control how language models hallucinate](https://md.research.elamur.ai/free/en/papers/kang-2025-unfamiliar-finetuning-examples-control-how/). In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Techno…
42. Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. 2023. Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation. arXiv preprint arXiv:2302.09664. [find](https://scholar.google.com/scholar?q=Lorenz+Kuhn%2C+Yarin+Gal%2C+and+Sebastian+Farquhar%2E+2023%2E+Semantic+uncertainty%3A+Linguistic+invariances+for+uncertainty+estimation+in+natural+language+generation%2E+arXiv+preprint+arXiv%3A2302%2E09664%2E)
43. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. 2019. [Natural questions: a benchmark for question answering research](https://md.research.elamur.ai/free/en/papers/kwiatkowski-2019-natural-questions-benchmark-question-answering/). Transactions of the Association for Computational Linguistic…
44. Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. 2019. [Latent retrieval for weakly supervised open domain question answering](https://md.research.elamur.ai/free/en/papers/lee-2019-latent-retrieval-weakly-supervised-open/). arXiv preprint arXiv:1906.00300.
45. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. [Retrieval-augmented generation for knowledge-intensive nlp tasks](https://md.research.elamur.ai/free/en/papers/lewis-2020-retrieval-augmented-generation-knowledge-intensive/). Advances in neural information processing systems, 33:9459–9474.
46. Gang Li, Ming Lin, Tomer Galanti, Zhengzhong Tu, and Tianbao Yang. 2025a. Disco: Reinforcing large reasoning models with discriminative constrained optimization. arXiv preprint arXiv:2505.12366. — [arXiv:2505.12366](https://arxiv.org/abs/2505.12366) [find](https://scholar.google.com/scholar?q=Gang+Li%2C+Ming+Lin%2C+Tomer+Galanti%2C+Zhengzhong+Tu%2C+and+Tianbao+Yang%2E+2025a%2E+Disco%3A+Reinforcing+large+reasoning+models+with+discriminative+constrained+optimization%2E+arXiv+preprint+arXiv%3A2505%2E12366%2E+%E2%80%94+%5Bar)
47. Gen Li and Yuling Yan. 2025. Towards efficient online exploration for reinforcement learning with human feedback. arXiv preprint arXiv:2509.22633. — [arXiv:2509.22633](https://arxiv.org/abs/2509.22633) [find](https://scholar.google.com/scholar?q=Gen+Li+and+Yuling+Yan%2E+2025%2E+Towards+efficient+online+exploration+for+reinforcement+learning+with+human+feedback%2E+arXiv+preprint+arXiv%3A2509%2E22633%2E+%E2%80%94+%5BarXiv%3A2509%2E22633%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2509%2E22633)
48. Junyi Li and Hwee Tou Ng. Reasoning models hallucinate more: Factuality-aware reinforcement learning for large reasoning models. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Junyi+Li+and+Hwee+Tou+Ng%2E+Reasoning+models+hallucinate+more%3A+Factuality%2Daware+reinforcement+learning+for+large+reasoning+models%2E+In+The+Thirty%2Dninth+Annual+Conference+on+Neural+Information+Processing+)
49. Yuming Li, Yikai Wang, Yuying Zhu, Zhongyu Zhao, Ming Lu, Qi She, and Shanghang Zhang. 2025d. Branchgrpo: Stable and efficient grpo with structured branching in diffusion models. arXiv preprint arXiv:2509.06040. — [arXiv:2509.06040](https://arxiv.org/abs/2509.06040) [find](https://scholar.google.com/scholar?q=Yuming+Li%2C+Yikai+Wang%2C+Yuying+Zhu%2C+Zhongyu+Zhao%2C+Ming+Lu%2C+Qi+She%2C+and+Shanghang+Zhang%2E+2025d%2E+Branchgrpo%3A+Stable+and+efficient+grpo+with+structured+branching+in+diffusion+models%2E+arXiv+preprint+arXiv%3A)
50. Yuxin Liang, Zhuoyang Song, Hao Wang, and Jiaxing Zhang. 2024. Learning to trust your feelings: Leveraging self-awareness in llms for hallucination mitigation. arXiv preprint arXiv:2401.15449. [find](https://scholar.google.com/scholar?q=Yuxin+Liang%2C+Zhuoyang+Song%2C+Hao+Wang%2C+and+Jiaxing+Zhang%2E+2024%2E+Learning+to+trust+your+feelings%3A+Leveraging+self%2Dawareness+in+llms+for+hallucination+mitigation%2E+arXiv+preprint+arXiv%3A2401%2E15449%2E)
51. Sheng-Chieh Lin, Luyu Gao, Barlas Oguz, Wenhan Xiong, Jimmy Lin, Wen-tau Yih, and Xilun Chen. 2024. Flame: Factuality-aware alignment for large language models. Advances in Neural Information Processing Systems, 37:115588–115614. [find](https://scholar.google.com/scholar?q=Sheng%2DChieh+Lin%2C+Luyu+Gao%2C+Barlas+Oguz%2C+Wenhan+Xiong%2C+Jimmy+Lin%2C+Wen%2Dtau+Yih%2C+and+Xilun+Chen%2E+2024%2E+Flame%3A+Factuality%2Daware+alignment+for+large+language+models%2E+Advances+in+Neural+Information+Processi)
52. Stephanie Lin, Jacob Hilton, and Owain Evans. 2021. [Truthfulqa: Measuring how models mimic human falsehoods](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/). arXiv preprint arXiv:2109.07958.
53. Xukai Liu, Ye Liu, Shiwen Wu, Yanghai Zhang, Yihao Yuan, Kai Zhang, and Qi Liu. 2025a. Know3-rag: A knowledge-aware rag framework with adaptive retrieval, generation, and filtering. arXiv preprint arXiv:2505.12662. — [arXiv:2505.12662](https://arxiv.org/abs/2505.12662) [find](https://scholar.google.com/scholar?q=Xukai+Liu%2C+Ye+Liu%2C+Shiwen+Wu%2C+Yanghai+Zhang%2C+Yihao+Yuan%2C+Kai+Zhang%2C+and+Qi+Liu%2E+2025a%2E+Know3%2Drag%3A+A+knowledge%2Daware+rag+framework+with+adaptive+retrieval%2C+generation%2C+and+filtering%2E+arXiv+preprint+arX)
54. Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. 2025b. [Understanding r1-zero-like training: A critical perspective](https://md.research.elamur.ai/free/en/papers/liu-2025-understanding-r1-zero-like-training/). arXiv preprint arXiv:2503.20783.
55. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. 2023. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36:46534–46594. [find](https://scholar.google.com/scholar?q=Aman+Madaan%2C+Niket+Tandon%2C+Prakhar+Gupta%2C+Skyler+Hallinan%2C+Luyu+Gao%2C+Sarah+Wiegreffe%2C+Uri+Alon%2C+Nouha+Dziri%2C+Shrimai+Prabhumoye%2C+Yiming+Yang%2C+et+al%2E+2023%2E+Self%2Drefine%3A+Iterative+refinement+with+self%2Df)
56. Potsawee Manakul, Adian Liusie, and Mark JF Gales. 2023. Selfcheckgpt: Zero-resource black-box hallucination detection for generative large language models. arXiv preprint arXiv:2303.08896. [find](https://scholar.google.com/scholar?q=Potsawee+Manakul%2C+Adian+Liusie%2C+and+Mark+JF+Gales%2E+2023%2E+Selfcheckgpt%3A+Zero%2Dresource+black%2Dbox+hallucination+detection+for+generative+large+language+models%2E+arXiv+preprint+arXiv%3A2303%2E08896%2E)
57. Ariana Martino, Michael Iannelli, and Coleen Truong. 2023. Knowledge injection to counter large language model (llm) hallucination. In European Semantic Web Conference, pages 182–185. Springer. [find](https://scholar.google.com/scholar?q=Ariana+Martino%2C+Michael+Iannelli%2C+and+Coleen+Truong%2E+2023%2E+Knowledge+injection+to+counter+large+language+model+%28llm%29+hallucination%2E+In+European+Semantic+Web+Conference%2C+pages+182%E2%80%93185%2E+Springer%2E)
58. Jianbiao Mei, Tao Hu, Daocheng Fu, Licheng Wen, Xuemeng Yang, Rong Wu, Pinlong Cai, Xing Gao, Yu Yang, Chengjun Xie, et al. 2025. o 2 -searcher: A searching-based agent model for open-domain open-ended question answering. arXiv preprint arXiv:2505.16582. — [arXiv:2505.16582](https://arxiv.org/abs/2505.16582) [find](https://scholar.google.com/scholar?q=Jianbiao+Mei%2C+Tao+Hu%2C+Daocheng+Fu%2C+Licheng+Wen%2C+Xuemeng+Yang%2C+Rong+Wu%2C+Pinlong+Cai%2C+Xing+Gao%2C+Yu+Yang%2C+Chengjun+Xie%2C+et+al%2E+2025%2E+o+2+%2Dsearcher%3A+A+searching%2Dbased+agent+model+for+open%2Ddomain+open%2Dende)
59. Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Wei Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. [Factscore: Fine-grained atomic evaluation of factual precision in long form text generation](https://md.research.elamur.ai/free/en/papers/min-2023-factscore-fine-grained-atomic-evaluation/). arXiv preprint arXiv:2305.14251.
60. Youssef Mroueh, Nicolas Dupuis, Brian Belgodere, Apoorva Nitsure, Mattia Rigotti, Kristjan Greenewald, Jiri Navratil, Jerret Ross, and Jesus Rios. 2025. Revisiting group relative policy optimization: Insights into on-policy and off-policy training. arXiv preprint arXiv:2505.22257. — [arXiv:2505.22257](https://arxiv.org/abs/2505.22257) [find](https://scholar.google.com/scholar?q=Youssef+Mroueh%2C+Nicolas+Dupuis%2C+Brian+Belgodere%2C+Apoorva+Nitsure%2C+Mattia+Rigotti%2C+Kristjan+Greenewald%2C+Jiri+Navratil%2C+Jerret+Ross%2C+and+Jesus+Rios%2E+2025%2E+Revisiting+group+relative+policy+optimization%3A+)
61. Niels Mündler, Jingxuan He, Slobodan Jenko, and Martin Vechev. 2023. [Self-contradictory hallucinations of large language models: Evaluation, detection and mitigation](https://md.research.elamur.ai/free/en/papers/mundler-2023-self-contradictory-hallucinations-large-language/). arXiv preprint arXiv:2305.15852.
62. Gongrui Nan, Siye Chen, Jing Huang, Mengyu Lu, Dexun Wang, Chunmei Xie, Weiqi Xiong, Xianzhou Zeng, Qixuan Zhou, Yadong Li, et al. 2025. Ngrpo: Negative-enhanced group relative policy optimization. arXiv preprint arXiv:2509.18851. — [arXiv:2509.18851](https://arxiv.org/abs/2509.18851) [find](https://scholar.google.com/scholar?q=Gongrui+Nan%2C+Siye+Chen%2C+Jing+Huang%2C+Mengyu+Lu%2C+Dexun+Wang%2C+Chunmei+Xie%2C+Weiqi+Xiong%2C+Xianzhou+Zeng%2C+Qixuan+Zhou%2C+Yadong+Li%2C+et+al%2E+2025%2E+Ngrpo%3A+Negative%2Denhanced+group+relative+policy+optimization%2E+ar)
63. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in neural information processing systems, 35:27730–27744.
64. Lei Pang and Ruinan Jin. 2025. On the theory and practice of grpo: A trajectory-corrected approach with fast convergence. arXiv preprint arXiv:2508.02833. — [arXiv:2508.02833](https://arxiv.org/abs/2508.02833) [find](https://scholar.google.com/scholar?q=Lei+Pang+and+Ruinan+Jin%2E+2025%2E+On+the+theory+and+practice+of+grpo%3A+A+trajectory%2Dcorrected+approach+with+fast+convergence%2E+arXiv+preprint+arXiv%3A2508%2E02833%2E+%E2%80%94+%5BarXiv%3A2508%2E02833%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F25)
65. Nisarg Patel, Mohith Kulkarni, Mihir Parmar, Aashna Budhiraja, Mutsumi Nakamura, Neeraj Varshney, and Chitta Baral. 2024. [Multi-logieval: Towards evaluating multi-step logical reasoning ability of large language models](https://md.research.elamur.ai/free/en/papers/patel-2024-multi-logieval-evaluating-multi-step/). arXiv preprint arXiv:2406.17169.
66. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. 2024. [Gpqa: A graduate-level google-proof q&a benchmark](https://md.research.elamur.ai/free/en/papers/rein-2023-gpqa-graduate-level-google-proof/). In First Conference on Language Modeling.
67. Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, and Ji-Rong Wen. 2025a. [R1-searcher: Incentivizing the search capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/song-2025-r1-searcher-incentivizing-search/). arXiv preprint arXiv:2503.05592.
68. Linxin Song, Taiwei Shi, and Jieyu Zhao. 2025b. The hallucination tax of reinforcement finetuning. arXiv preprint arXiv:2505.13988. [find](https://scholar.google.com/scholar?q=Linxin+Song%2C+Taiwei+Shi%2C+and+Jieyu+Zhao%2E+2025b%2E+The+hallucination+tax+of+reinforcement+finetuning%2E+arXiv+preprint+arXiv%3A2505%2E13988%2E)
69. Zhongxiang Sun, Qipeng Wang, Haoyu Wang, Xiao Zhang, and Jun Xu. 2025. Detection and mitigation of hallucination in large reasoning models: A mechanistic perspective. arXiv preprint arXiv:2505.12886. — [arXiv:2505.12886](https://arxiv.org/abs/2505.12886) [find](https://scholar.google.com/scholar?q=Zhongxiang+Sun%2C+Qipeng+Wang%2C+Haoyu+Wang%2C+Xiao+Zhang%2C+and+Jun+Xu%2E+2025%2E+Detection+and+mitigation+of+hallucination+in+large+reasoning+models%3A+A+mechanistic+perspective%2E+arXiv+preprint+arXiv%3A2505%2E12886%2E+)
70. Yunhao Tang, Daniel Zhaohan Guo, Zeyu Zheng, Daniele Calandriello, Yuan Cao, Eugene Tarassov, Rémi Munos, Bernardo Ávila Pires, Michal Valko, Yong Cheng, et al. 2024. Understanding the performance gap between online and offline alignment algorithms. arXiv preprint arXiv:2405.08448. — [arXiv:2405.08448](https://arxiv.org/abs/2405.08448) [find](https://scholar.google.com/scholar?q=Yunhao+Tang%2C+Daniel+Zhaohan+Guo%2C+Zeyu+Zheng%2C+Daniele+Calandriello%2C+Yuan+Cao%2C+Eugene+Tarassov%2C+R%C3%A9mi+Munos%2C+Bernardo+%C3%81vila+Pires%2C+Michal+Valko%2C+Yong+Cheng%2C+et+al%2E+2024%2E+Understanding+the+performance+gap)
71. Qwen Team. 2024. Qwen2.5: A party of foundation models. [find](https://scholar.google.com/scholar?q=Qwen+Team%2E+2024%2E+Qwen2%2E5%3A+A+party+of+foundation+models%2E)
72. Katherine Tian, Eric Mitchell, Huaxiu Yao, Christopher D Manning, and Chelsea Finn. 2023. [Fine-tuning language models for factuality](https://md.research.elamur.ai/free/en/papers/tian-2023-fine-tuning-language-models-factuality/). In The Twelfth International Conference on Learning Representations.
73. SM Tonmoy, SM Zaman, Vinija Jain, Anku Rani, Vipula Rawte, Aman Chadha, and Amitava Das. 2024. [A comprehensive survey of hallucination mitigation techniques in large language models](https://md.research.elamur.ai/free/en/papers/tonmoy-2024-comprehensive-survey-hallucination-mitigation-techniques/). arXiv preprint arXiv:2401.01313, 6.
74. Jing Wang, Jiajun Liang, Jie Liu, Henglin Liu, Gongye Liu, Jun Zheng, Wanyuan Pang, Ao Ma, Zhenyu Xie, Xintao Wang, et al. 2025. Grpo-guard: Mitigating implicit over-optimization in flow matching via regulated clipping. arXiv preprint arXiv:2510.22319. — [arXiv:2510.22319](https://arxiv.org/abs/2510.22319) [find](https://scholar.google.com/scholar?q=Jing+Wang%2C+Jiajun+Liang%2C+Jie+Liu%2C+Henglin+Liu%2C+Gongye+Liu%2C+Jun+Zheng%2C+Wanyuan+Pang%2C+Ao+Ma%2C+Zhenyu+Xie%2C+Xintao+Wang%2C+et+al%2E+2025%2E+Grpo%2Dguard%3A+Mitigating+implicit+over%2Doptimization+in+flow+matching+via+)
75. Jason Wei, Nguyen Karina, Hyung Won Chung, Yunxin Joy Jiao, Spencer Papay, Amelia Glaese, John Schulman, and William Fedus. 2024. [Measuring short-form factuality in large language models](https://md.research.elamur.ai/free/en/papers/wei-2024-measuring-short-form-factuality-large/). arXiv preprint arXiv:2411.04368.
76. Zhepei Wei, Xiao Yang, Kai Sun, Jiaqi Wang, Rulin Shao, Sean Chen, Mohammad Kachuee, Teja Gollapudi, Tony Liao, Nicolas Scheffer, et al. 2025. Truthrl: Incentivizing truthful llms via reinforcement learning. arXiv preprint arXiv:2509.25760. — [arXiv:2509.25760](https://arxiv.org/abs/2509.25760) [find](https://scholar.google.com/scholar?q=Zhepei+Wei%2C+Xiao+Yang%2C+Kai+Sun%2C+Jiaqi+Wang%2C+Rulin+Shao%2C+Sean+Chen%2C+Mohammad+Kachuee%2C+Teja+Gollapudi%2C+Tony+Liao%2C+Nicolas+Scheffer%2C+et+al%2E+2025%2E+Truthrl%3A+Incentivizing+truthful+llms+via+reinforcement+le)
77. Xueru Wen, Jie Lou, Xinyu Lu, Yuqiu Ji, Xinyan Guan, Yaojie Lu, Hongyu Lin, Ben He, Xianpei Han, Debing Zhang, et al. 2025. On-policy self-alignment with fine-grained knowledge feedback for hallucination mitigation. In Findings of the Association for Computational Linguistics: ACL 2025, pages 5215–… [find](https://scholar.google.com/scholar?q=Xueru+Wen%2C+Jie+Lou%2C+Xinyu+Lu%2C+Yuqiu+Ji%2C+Xinyan+Guan%2C+Yaojie+Lu%2C+Hongyu+Lin%2C+Ben+He%2C+Xianpei+Han%2C+Debing+Zhang%2C+et+al%2E+2025%2E+On%2Dpolicy+self%2Dalignment+with+fine%2Dgrained+knowledge+feedback+for+hallucinat)
78. Zhiheng Xi, Xin Guo, Yang Nan, Enyu Zhou, Junrui Shen, Wenxiang Chen, Jiaqi Liu, Jixuan Huang, Zhihao Zhang, Honglin Guo, et al. 2025. Bapo: Stabilizing off-policy reinforcement learning for llms via balanced policy optimization with adaptive clipping. arXiv preprint arXiv:2510.18927. — [arXiv:2510.18927](https://arxiv.org/abs/2510.18927) [find](https://scholar.google.com/scholar?q=Zhiheng+Xi%2C+Xin+Guo%2C+Yang+Nan%2C+Enyu+Zhou%2C+Junrui+Shen%2C+Wenxiang+Chen%2C+Jiaqi+Liu%2C+Jixuan+Huang%2C+Zhihao+Zhang%2C+Honglin+Guo%2C+et+al%2E+2025%2E+Bapo%3A+Stabilizing+off%2Dpolicy+reinforcement+learning+for+llms+via+)
79. Changyi Xiao, Mengdi Zhang, and Yixin Cao. 2025. Bnpo: Beta normalization policy optimization. arXiv preprint arXiv:2506.02864. — [arXiv:2506.02864](https://arxiv.org/abs/2506.02864) [find](https://scholar.google.com/scholar?q=Changyi+Xiao%2C+Mengdi+Zhang%2C+and+Yixin+Cao%2E+2025%2E+Bnpo%3A+Beta+normalization+policy+optimization%2E+arXiv+preprint+arXiv%3A2506%2E02864%2E+%E2%80%94+%5BarXiv%3A2506%2E02864%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2506%2E02864%29)
80. Tengyang Xie, Dylan J Foster, Akshay Krishnamurthy, Corby Rosset, Ahmed Awadallah, and Alexander Rakhlin. 2024. Exploratory preference optimization: Harnessing implicit q*-approximation for sample-efficient rlhf. arXiv preprint arXiv:2405.21046. — [arXiv:2405.21046](https://arxiv.org/abs/2405.21046) [find](https://scholar.google.com/scholar?q=Tengyang+Xie%2C+Dylan+J+Foster%2C+Akshay+Krishnamurthy%2C+Corby+Rosset%2C+Ahmed+Awadallah%2C+and+Alexander+Rakhlin%2E+2024%2E+Exploratory+preference+optimization%3A+Harnessing+implicit+q%2A%2Dapproximation+for+sample%2Deff)
81. Tian Xie, Zitian Gao, Qingnan Ren, Haoming Luo, Yuqian Hong, Bryan Dai, Joey Zhou, Kai Qiu, Zhirong Wu, and Chong Luo. 2025. [Logic-rl: Unleashing llm reasoning with rule-based reinforcement learning](https://md.research.elamur.ai/free/en/papers/xie-2025-logic-rl-unleashing-llm/). arXiv preprint arXiv:2502.14768.
82. Erhan Xu, Kai Ye, Hongyi Zhou, Luhan Zhu, Francesco Quinzan, and Chengchun Shi. 2025. Doubly robust alignment for large language models. arXiv preprint arXiv:2506.01183. — [arXiv:2506.01183](https://arxiv.org/abs/2506.01183) [find](https://scholar.google.com/scholar?q=Erhan+Xu%2C+Kai+Ye%2C+Hongyi+Zhou%2C+Luhan+Zhu%2C+Francesco+Quinzan%2C+and+Chengchun+Shi%2E+2025%2E+Doubly+robust+alignment+for+large+language+models%2E+arXiv+preprint+arXiv%3A2506%2E01183%2E+%E2%80%94+%5BarXiv%3A2506%2E01183%5D%28https%3A%2F%2Fa)
83. Shaolei Zhang, Tian Yu, and Yang Feng. 2024a. [Truthx: Alleviating hallucinations by editing large language models in truthful space](https://md.research.elamur.ai/free/en/papers/zhang-2024-truthx-alleviating-hallucinations-editing-large/). arXiv preprint arXiv:2402.17811.
84. Hongshen Xu, Zichen Zhu, Situo Zhang, Da Ma, Shuai Fan, Lu Chen, and Kai Yu. 2024a. Rejection improves reliability: Training llms to refuse unknown questions using rl from knowledge feedback. arXiv preprint arXiv:2403.18349. — [arXiv:2403.18349](https://arxiv.org/abs/2403.18349) [find](https://scholar.google.com/scholar?q=Hongshen+Xu%2C+Zichen+Zhu%2C+Situo+Zhang%2C+Da+Ma%2C+Shuai+Fan%2C+Lu+Chen%2C+and+Kai+Yu%2E+2024a%2E+Rejection+improves+reliability%3A+Training+llms+to+refuse+unknown+questions+using+rl+from+knowledge+feedback%2E+arXiv+pr)
85. Xingjian Zhang, Siwei Wen, Wenjun Wu, and Lei Huang. 2025b. Edge-grpo: Entropy-driven grpo with guided error correction for advantage diversity. arXiv preprint arXiv:2507.21848. — [arXiv:2507.21848](https://arxiv.org/abs/2507.21848) [find](https://scholar.google.com/scholar?q=Xingjian+Zhang%2C+Siwei+Wen%2C+Wenjun+Wu%2C+and+Lei+Huang%2E+2025b%2E+Edge%2Dgrpo%3A+Entropy%2Ddriven+grpo+with+guided+error+correction+for+advantage+diversity%2E+arXiv+preprint+arXiv%3A2507%2E21848%2E+%E2%80%94+%5BarXiv%3A2507%2E21848%5D%28h)
86. Shusheng Xu, Wei Fu, Jiaxuan Gao, Wenjie Ye, Weilin Liu, Zhiyu Mei, Guangju Wang, Chao Yu, and Yi Wu. 2024b. Is dpo superior to ppo for llm alignment? a comprehensive study. arXiv preprint arXiv:2404.10719. — [arXiv:2404.10719](https://arxiv.org/abs/2404.10719) [find](https://scholar.google.com/scholar?q=Shusheng+Xu%2C+Wei+Fu%2C+Jiaxuan+Gao%2C+Wenjie+Ye%2C+Weilin+Liu%2C+Zhiyu+Mei%2C+Guangju+Wang%2C+Chao+Yu%2C+and+Yi+Wu%2E+2024b%2E+Is+dpo+superior+to+ppo+for+llm+alignment%3F+a+comprehensive+study%2E+arXiv+preprint+arXiv%3A2404%2E)
87. Yuqing Yang, Ethan Chern, Xipeng Qiu, Graham Neubig, and Pengfei Liu. 2024. Alignment for honesty. Advances in Neural Information Processing Systems, 37:63565–63598. [find](https://scholar.google.com/scholar?q=Yuqing+Yang%2C+Ethan+Chern%2C+Xipeng+Qiu%2C+Graham+Neubig%2C+and+Pengfei+Liu%2E+2024%2E+Alignment+for+honesty%2E+Advances+in+Neural+Information+Processing+Systems%2C+37%3A63565%E2%80%9363598%2E)
88. Zijun Yao, Yantao Liu, Yanxu Chen, Jianhui Chen, Junfeng Fang, Lei Hou, Juanzi Li, and Tat-Seng Chua. 2025. Are reasoning models more prone to hallucination? arXiv preprint arXiv:2505.23646. — [arXiv:2505.23646](https://arxiv.org/abs/2505.23646) [find](https://scholar.google.com/scholar?q=Zijun+Yao%2C+Yantao+Liu%2C+Yanxu+Chen%2C+Jianhui+Chen%2C+Junfeng+Fang%2C+Lei+Hou%2C+Juanzi+Li%2C+and+Tat%2DSeng+Chua%2E+2025%2E+Are+reasoning+models+more+prone+to+hallucination%3F+arXiv+preprint+arXiv%3A2505%2E23646%2E+%E2%80%94+%5BarXiv%3A)
89. Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. 2025. [Demystifying long chain-of-thought reasoning in llms](https://md.research.elamur.ai/free/en/papers/yeo-2025-demystifying-long-chain-thought-reasoning/). arXiv preprint arXiv:2502.03373.
90. Chenlong Yin, Zeyang Sha, Shiwen Cui, and Changhua Meng. 2025. [The reasoning trap: How enhancing llm reasoning amplifies tool hallucination](https://md.research.elamur.ai/free/en/papers/yin-2026-reasoning-trap-how-enhancing-llm/). arXiv preprint arXiv:2510.22977.
91. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. 2025. [Dapo: An open-source llm reinforcement learning system at scale](https://md.research.elamur.ai/free/en/papers/yu-2025-dapo-open-source-llm-reinforcement/). arXiv preprint arXiv:2503.14476.
92. Zheng Yuan, Hongyi Yuan, Chuanqi Tan, Wei Wang, Songfang Huang, and Fei Huang. 2023. [Rrhf: Rank responses to align language models with human feedback without tears](https://md.research.elamur.ai/free/en/papers/yuan-2023-rrhf-rank-responses-align-language/). arXiv preprint arXiv:2304.05302.
93. Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Shiji Song, and Gao Huang. 2025. [Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?](https://md.research.elamur.ai/free/en/papers/yue-2025-reinforcement-learning-really-incentivize-reasoning/) arXiv preprint arXiv:2504.13837.
94. Chen Zhang. 2023. User-controlled knowledge fusion in large language models: Balancing creativity and hallucination. arXiv preprint arXiv:2307.16139. — [arXiv:2307.16139](https://arxiv.org/abs/2307.16139) [find](https://scholar.google.com/scholar?q=Chen+Zhang%2E+2023%2E+User%2Dcontrolled+knowledge+fusion+in+large+language+models%3A+Balancing+creativity+and+hallucination%2E+arXiv+preprint+arXiv%3A2307%2E16139%2E+%E2%80%94+%5BarXiv%3A2307%2E16139%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2307%2E16)
95. Mike Zhang, Johannes Bjerva, and Russa Biswas. 2025a. Scaling reasoning can improve factuality in large language models. arXiv preprint arXiv:2505.11140. — [arXiv:2505.11140](https://arxiv.org/abs/2505.11140) [find](https://scholar.google.com/scholar?q=Mike+Zhang%2C+Johannes+Bjerva%2C+and+Russa+Biswas%2E+2025a%2E+Scaling+reasoning+can+improve+factuality+in+large+language+models%2E+arXiv+preprint+arXiv%3A2505%2E11140%2E+%E2%80%94+%5BarXiv%3A2505%2E11140%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F250)
96. Yiming Zhang, Harshita Diddee, Susan Holm, Hanchen Liu, Xinyue Liu, Vinay Samuel, Barry Wang, and Daphne Ippolito. 2025c. Noveltybench: Evaluating language models for humanlike diversity. arXiv preprint arXiv:2504.05228. — [arXiv:2504.05228](https://arxiv.org/abs/2504.05228) [find](https://scholar.google.com/scholar?q=Yiming+Zhang%2C+Harshita+Diddee%2C+Susan+Holm%2C+Hanchen+Liu%2C+Xinyue+Liu%2C+Vinay+Samuel%2C+Barry+Wang%2C+and+Daphne+Ippolito%2E+2025c%2E+Noveltybench%3A+Evaluating+language+models+for+humanlike+diversity%2E+arXiv+prepri)
97. Yuji Zhang, Sha Li, Jiateng Liu, Pengfei Yu, Yi R Fung, Jing Li, Manling Li, and Heng Ji. 2024b. Knowledge overshadowing causes amalgamated hallucination in large language models. arXiv preprint arXiv:2407.08039. — [arXiv:2407.08039](https://arxiv.org/abs/2407.08039) [find](https://scholar.google.com/scholar?q=Yuji+Zhang%2C+Sha+Li%2C+Jiateng+Liu%2C+Pengfei+Yu%2C+Yi+R+Fung%2C+Jing+Li%2C+Manling+Li%2C+and+Heng+Ji%2E+2024b%2E+Knowledge+overshadowing+causes+amalgamated+hallucination+in+large+language+models%2E+arXiv+preprint+arXiv)
98. Yuji Zhang, Sha Li, Cheng Qian, Jiateng Liu, Pengfei Yu, Chi Han, Yi R Fung, Kathleen McKeown, Chengxiang Zhai, Manling Li, et al. 2025d. The law of knowledge overshadowing: Towards understanding, predicting, and preventing llm hallucination. arXiv preprint arXiv:2502.16143. [find](https://scholar.google.com/scholar?q=Yuji+Zhang%2C+Sha+Li%2C+Cheng+Qian%2C+Jiateng+Liu%2C+Pengfei+Yu%2C+Chi+Han%2C+Yi+R+Fung%2C+Kathleen+McKeown%2C+Chengxiang+Zhai%2C+Manling+Li%2C+et+al%2E+2025d%2E+The+law+of+knowledge+overshadowing%3A+Towards+understanding%2C+pre)
99. Jian Zhao, Runze Liu, Kaiyan Zhang, Zhimu Zhou, Junqi Gao, Dong Li, Jiafei Lyu, Zhouyi Qian, Biqing Qi, Xiu Li, et al. 2025. Genprm: Scaling test-time compute of process reward models via generative reasoning. arXiv preprint arXiv:2504.00891. — [arXiv:2504.00891](https://arxiv.org/abs/2504.00891) [find](https://scholar.google.com/scholar?q=Jian+Zhao%2C+Runze+Liu%2C+Kaiyan+Zhang%2C+Zhimu+Zhou%2C+Junqi+Gao%2C+Dong+Li%2C+Jiafei+Lyu%2C+Zhouyi+Qian%2C+Biqing+Qi%2C+Xiu+Li%2C+et+al%2E+2025%2E+Genprm%3A+Scaling+test%2Dtime+compute+of+process+reward+models+via+generative+r)
100. Suifeng Zhao, Tong Zhou, Zhuoran Jin, Hongbang Yuan, Yubo Chen, Kang Liu, and Sujian Li. 2024. Awecita: Generating answer with appropriate and well-grained citations using llms. Data Intelligence, 6(4):1134– 1157. [find](https://scholar.google.com/scholar?q=Suifeng+Zhao%2C+Tong+Zhou%2C+Zhuoran+Jin%2C+Hongbang+Yuan%2C+Yubo+Chen%2C+Kang+Liu%2C+and+Sujian+Li%2E+2024%2E+Awecita%3A+Generating+answer+with+appropriate+and+well%2Dgrained+citations+using+llms%2E+Data+Intelligence%2C+6%284)
101. Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, et al. 2025a. [Group sequence policy optimization](https://md.research.elamur.ai/free/en/papers/zheng-2025-group-sequence-policy-optimization/). arXiv preprint arXiv:2507.18071.
102. Hang Zheng, Hongshen Xu, Yuncong Liu, Lu Chen, Pascale Fung, and Kai Yu. 2025b. Enhancing llm reliability via explicit knowledge boundary modeling. arXiv preprint arXiv:2503.02233. — [arXiv:2503.02233](https://arxiv.org/abs/2503.02233) [find](https://scholar.google.com/scholar?q=Hang+Zheng%2C+Hongshen+Xu%2C+Yuncong+Liu%2C+Lu+Chen%2C+Pascale+Fung%2C+and+Kai+Yu%2E+2025b%2E+Enhancing+llm+reliability+via+explicit+knowledge+boundary+modeling%2E+arXiv+preprint+arXiv%3A2503%2E02233%2E+%E2%80%94+%5BarXiv%3A2503%2E02233)
103. Banghua Zhu, Michael I Jordan, and Jiantao Jiao. 2024. Iterative data smoothing: Mitigating reward overfitting and overoptimization in rlhf. arXiv preprint arXiv:2401.16335. — [arXiv:2401.16335](https://arxiv.org/abs/2401.16335) [find](https://scholar.google.com/scholar?q=Banghua+Zhu%2C+Michael+I+Jordan%2C+and+Jiantao+Jiao%2E+2024%2E+Iterative+data+smoothing%3A+Mitigating+reward+overfitting+and+overoptimization+in+rlhf%2E+arXiv+preprint+arXiv%3A2401%2E16335%2E+%E2%80%94+%5BarXiv%3A2401%2E16335%5D%28https)
104. Nana Zhu, Zixian Feng, Hang Wang, Xing Gao, Xinyi Wang, and Yuanxing Liu. 2025. Clarifying user preference with maximum entropy based recommendation. Data Intelligence, 7(2):527–548. [find](https://scholar.google.com/scholar?q=Nana+Zhu%2C+Zixian+Feng%2C+Hang+Wang%2C+Xing+Gao%2C+Xinyi+Wang%2C+and+Yuanxing+Liu%2E+2025%2E+Clarifying+user+preference+with+maximum+entropy+based+recommendation%2E+Data+Intelligence%2C+7%282%29%3A527%E2%80%93548%2E)
105. Muru Zhang, Ofir Press, William Merrill, Alisa Liu, and Noah A Smith. 2023. [How language model hallucinations can snowball](https://md.research.elamur.ai/free/en/papers/zhang-2023-language-model-hallucinations-snowball/). arXiv preprint arXiv:2305.13534.

*Updated 31 October 2026*