---
title: "Abstain-R1 — paper by Zhai et al. (2026) — summary and ideas"
description: "They argue a model should decline to answer and also name what is missing. 45 references, cited by 1 paper here."
authors: ["Haotian Zhai", "Jingcheng Liang", "Dongyeop Kang"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.985/"
doi: "10.18653/v1/2026.findings-acl.985"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhai-2026-abstain-r1-calibrated-abstention-post/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhai-2026-abstain-r1-calibrated-abstention-post/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/6/) / Abstain-R1: Calibrated Abstention and Post-Refusal Clarification via Verifiable RL

# Abstain-R1: Calibrated Abstention and Post-Refusal Clarification via Verifiable RL

**Authors:** Haotian Zhai, Jingcheng Liang and [Dongyeop Kang](https://md.research.elamur.ai/free/en/authors/dongyeop-kang/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.985/) · [PDF](https://aclanthology.org/2026.findings-acl.985.pdf) · [DOI 10.18653/v1/2026.findings-acl.985](https://doi.org/10.18653/v1/2026.findings-acl.985) · [OpenAlex](https://openalex.org/W7166850325)

**Citations:** 1 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 149,439 papers whose bibliographies we read

## Summary
Reinforcement fine-tuning can push LLMs to guess or invent missing details when a question cannot actually be resolved. The authors target queries whose meaning is clear but whose answer cannot be determined from the given information, such as missing conditions, false premises, contradictions, or objectively unknown facts. They argue a model should decline to answer and also name what is missing. Their approach uses standard GRPO with a clarification-aware RLVR reward. Correct answers on answerable questions are rewarded. On unanswerable ones, a rule-checked abstention format earns a base reward, and a clarification that matches the reference earns a bonus. A cold-start SFT set, Abstain-CoT, is derived from AbstentionBench with DeepSeek-V3 annotations. The resulting 3B model, Abstain-R1, improves greatly over its base model on Abstain-Test, Abstain-QA and SelfAware. Its handling of unanswerable queries is competitive with much larger systems such as DeepSeek-R1. This suggests useful abstention can be trained with verifiable rewards rather than relying on scale.

## Key points
- New resources: the Abstain-CoT SFT dataset, the Abstain-Test benchmark and its metric suite, and the Abstain-R1 model. Abstain-Test measures answerable accuracy, false refusals, refusal rate (U-Ref) and clarification quality (U-Clar).
- On Abstain-Test, the 3B base model scores 9.4 U-Ref and 0.6 U-Clar. Abstain-R1 reaches 68.1 and 55.1, and its answerable accuracy also rises from 48.8 to 57.2.
- Abstain-R1's U-Ref exceeds the 5-shot ICL result of Qwen2.5 32B (66.2).
- SFT-ALL is an SFT variant using DeepSeek-V3 CoT traces generated on the RL training data. It refuses more often (78.4 U-Ref) but has lower answerable accuracy (47.1) and the most false refusals among trained variants. It also requires an external strong model.
- Ablations show the penalty for abstaining on answerable queries is crucial: −1 works better than −0.5. Limitations include English-only QA evaluation and reliance on LLM judges.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Minnesota](https://md.research.elamur.ai/free/en/universities/university-of-minnesota/) — University, Minneapolis, United States · 28 works · 2,184 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

## Cite

Cite as: Haotian Zhai, Jingcheng Liang and Dongyeop Kang. 2026. Abstain-R1: Calibrated Abstention and Post-Refusal Clarification via Verifiable RL. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhai2026abstain,
  title = {{Abstain-R1: Calibrated Abstention and Post-Refusal Clarification via Verifiable RL}},
  author = {Haotian Zhai, Jingcheng Liang and Dongyeop Kang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.985},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.985}
}
```

- [← previous a1: Steep Test-time Scaling Law via Environment Augmented Generation](https://md.research.elamur.ai/free/en/papers/mei-2026-a1-steep-test-time-scaling/)
- [next → Accurate Legal Reasoning at Scale: Neuro-Symbolic Offloading and Structural Auditability for Robust Legal Adjudication](https://md.research.elamur.ai/free/en/papers/sojka-2026-accurate-legal-reasoning-scale-neuro/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/6/)

## References

45 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alfonso Amayuelas, Kyle Wong, Liangming Pan, Wenhu Chen, and William Yang Wang. 2024. [Knowledge of knowledge: Exploring known-unknowns uncertainty with large language models](https://md.research.elamur.ai/free/en/papers/amayuelas-2024-knowledge-knowledge-exploring-known-unknowns/). In Findings of the Association for Computational Linguistics: ACL 2024, pages 6416–6432.
2. Youssef Benchekroun, Megi Dervishi, Mark Ibrahim, Jean-Baptiste Gaya, Xavier Martinet, Grégoire Mialon, Thomas Scialom, Emmanuel Dupoux, Dieuwke Hupkes, and Pascal Vincent. 2023. [Worldsense: A synthetic benchmark for grounded reasoning in large language models](https://md.research.elamur.ai/free/en/papers/benchekroun-2023-worldsense-synthetic-benchmark-grounded-reasoning/). arXiv preprint arXiv:2311.15930.
3. Faeze Brahman, Sachin Kumar, Vidhisha Balachandran, Pradeep Dasigi, Valentina Pyatkin, Abhilasha Ravichander, Sarah Wiegreffe, Nouha Dziri, Khyathi Chandu, Jack Hessel, et al. 2024. [The art of saying no: Contextual noncompliance in language models](https://md.research.elamur.ai/free/en/papers/brahman-2024-art-saying-no-contextual-noncompliance/). Advances in Neural Information Processing Systems,…
4. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… — [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
5. Ding Chen, Qingchen Yu, Pengyuan Wang, Wentao Zhang, Bo Tang, Feiyu Xiong, Xinchi Li, Minchuan Yang, and Zhiyu Li. 2025. xverify: Efficient answer verifier for reasoning model evaluations. arXiv preprint arXiv:2504.10481. — [arXiv:2504.10481](https://arxiv.org/abs/2504.10481) [find](https://scholar.google.com/scholar?q=Ding+Chen%2C+Qingchen+Yu%2C+Pengyuan+Wang%2C+Wentao+Zhang%2C+Bo+Tang%2C+Feiyu+Xiong%2C+Xinchi+Li%2C+Minchuan+Yang%2C+and+Zhiyu+Li%2E+2025%2E+xverify%3A+Efficient+answer+verifier+for+reasoning+model+evaluations%2E+arXiv+prepr)
6. Qinyuan Cheng, Tianxiang Sun, Xiangyang Liu, Wenwei Zhang, Zhangyue Yin, Shimin Li, Linyang Li, Zhengfu He, Kai Chen, and Xipeng Qiu. 2024. Can ai assistants know what they don’t know? arXiv preprint arXiv:2401.13275. — [arXiv:2401.13275](https://arxiv.org/abs/2401.13275) [find](https://scholar.google.com/scholar?q=Qinyuan+Cheng%2C+Tianxiang+Sun%2C+Xiangyang+Liu%2C+Wenwei+Zhang%2C+Zhangyue+Yin%2C+Shimin+Li%2C+Linyang+Li%2C+Zhengfu+He%2C+Kai+Chen%2C+and+Xipeng+Qiu%2E+2024%2E+Can+ai+assistants+know+what+they+don%E2%80%99t+know%3F+arXiv+preprint+)
7. Xilin Dang, Kexin Chen, Xiaorui Su, Ayush Noori, Iñaki Arango, Lucas Vittor, Xinyi Long, Yuyang Du, Marinka Zitnik, and Pheng Ann Heng. 2025. Knowguard: Knowledge-driven abstention for multi-round clinical reasoning. arXiv preprint arXiv:2509.24816. — [arXiv:2509.24816](https://arxiv.org/abs/2509.24816) [find](https://scholar.google.com/scholar?q=Xilin+Dang%2C+Kexin+Chen%2C+Xiaorui+Su%2C+Ayush+Noori%2C+I%C3%B1aki+Arango%2C+Lucas+Vittor%2C+Xinyi+Long%2C+Yuyang+Du%2C+Marinka+Zitnik%2C+and+Pheng+Ann+Heng%2E+2025%2E+Knowguard%3A+Knowledge%2Ddriven+abstention+for+multi%2Dround+cli)
8. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chengg… — [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/)
9. Xuwei Ding, Skylar Zhai, Linxin Song, Jiate Li, Taiwei Shi, Nicholas Meade, Siva Reddy, Jian Kang, and Jieyu Zhao. 2026. The blind spot of agent safety: How benign user instructions expose critical vulnerabilities in computer-use agents. arXiv preprint arXiv:2604.10577. — [arXiv:2604.10577](https://arxiv.org/abs/2604.10577) [find](https://scholar.google.com/scholar?q=Xuwei+Ding%2C+Skylar+Zhai%2C+Linxin+Song%2C+Jiate+Li%2C+Taiwei+Shi%2C+Nicholas+Meade%2C+Siva+Reddy%2C+Jian+Kang%2C+and+Jieyu+Zhao%2E+2026%2E+The+blind+spot+of+agent+safety%3A+How+benign+user+instructions+expose+critical+vu)
10. Polina Kirichenko, Mark Ibrahim, Kamalika Chaudhuri, and Samuel J Bell. 2025. Abstentionbench: Reasoning llms fail on unanswerable questions. arXiv preprint arXiv:2506.09038. [find](https://scholar.google.com/scholar?q=Polina+Kirichenko%2C+Mark+Ibrahim%2C+Kamalika+Chaudhuri%2C+and+Samuel+J+Bell%2E+2025%2E+Abstentionbench%3A+Reasoning+llms+fail+on+unanswerable+questions%2E+arXiv+preprint+arXiv%3A2506%2E09038%2E)
11. Stella Li, Vidhisha Balachandran, Shangbin Feng, Jonathan Ilgen, Emma Pierson, Pang Wei W Koh, and Yulia Tsvetkov. 2024. [Mediq: Question-asking llms and a benchmark for reliable interactive clinical reasoning](https://md.research.elamur.ai/free/en/papers/li-2024-mediq-question-asking-llms-benchmark/). Advances in Neural Information Processing Systems, 37:28858–28888.
12. Shangbin Feng, Weijia Shi, Yike Wang, Wenxuan Ding, Vidhisha Balachandran, and Yulia Tsvetkov. 2024. Don’t hallucinate, abstain: Identifying llm knowledge gaps via multi-llm collaboration. arXiv preprint arXiv:2402.00367. [find](https://scholar.google.com/scholar?q=Shangbin+Feng%2C+Weijia+Shi%2C+Yike+Wang%2C+Wenxuan+Ding%2C+Vidhisha+Balachandran%2C+and+Yulia+Tsvetkov%2E+2024%2E+Don%E2%80%99t+hallucinate%2C+abstain%3A+Identifying+llm+knowledge+gaps+via+multi%2Dllm+collaboration%2E+arXiv+prepr)
13. Zhaowei Liu, Xin Guo, Zhi Yang, Fangqi Lou, Lingfeng Zeng, Mengping Li, Qi Qi, Zhiqiang Liu, Yiyang Han, Dongpo Cheng, Xingdong Feng, Huixia Judy Wang, Chengchun Shi, and Liwen Zhang. 2025. Fin-r1: A large language model for financial reasoning through reinforcement learning. arXiv e-prints, page ar… — [arXiv:2503.16252](https://arxiv.org/abs/2503.16252) [find](https://scholar.google.com/scholar?q=Zhaowei+Liu%2C+Xin+Guo%2C+Zhi+Yang%2C+Fangqi+Lou%2C+Lingfeng+Zeng%2C+Mengping+Li%2C+Qi+Qi%2C+Zhiqiang+Liu%2C+Yiyang+Han%2C+Dongpo+Cheng%2C+Xingdong+Feng%2C+Huixia+Judy+Wang%2C+Chengchun+Shi%2C+and+Liwen+Zhang%2E+2025%2E+Fin%2Dr1%3A+A+)
14. Cheng Gao, Huimin Chen, Chaojun Xiao, Zhiyi Chen, Zhiyuan Liu, and Maosong Sun. 2025. H-neurons: On the existence, impact, and origin of hallucination-associated neurons. arXiv preprint arXiv:2512.01797. — [arXiv:2512.01797](https://arxiv.org/abs/2512.01797) [find](https://scholar.google.com/scholar?q=Cheng+Gao%2C+Huimin+Chen%2C+Chaojun+Xiao%2C+Zhiyi+Chen%2C+Zhiyuan+Liu%2C+and+Maosong+Sun%2E+2025%2E+H%2Dneurons%3A+On+the+existence%2C+impact%2C+and+origin+of+hallucination%2Dassociated+neurons%2E+arXiv+preprint+arXiv%3A2512%2E017)
15. Run Luo, Lu Wang, Wanwei He, Longze Chen, Jiaming Li, and Xiaobo Xia. 2025. Gui-r1 : A generalist r1-style vision-language action model for gui agents. arXiv e-prints, page arXiv:2504.10458. — [arXiv:2504.10458](https://arxiv.org/abs/2504.10458) [find](https://scholar.google.com/scholar?q=Run+Luo%2C+Lu+Wang%2C+Wanwei+He%2C+Longze+Chen%2C+Jiaming+Li%2C+and+Xiaobo+Xia%2E+2025%2E+Gui%2Dr1+%3A+A+generalist+r1%2Dstyle+vision%2Dlanguage+action+model+for+gui+agents%2E+arXiv+e%2Dprints%2C+page+arXiv%3A2504%2E10458%2E+%E2%80%94+%5BarXiv%3A)
16. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783.
17. Anxin Guo and Jingwei Li. 2026. Hallucination is a consequence of space-optimality: A rate-distortion theorem for membership testing. arXiv preprint arXiv:2602.00906. — [arXiv:2602.00906](https://arxiv.org/abs/2602.00906) [find](https://scholar.google.com/scholar?q=Anxin+Guo+and+Jingwei+Li%2E+2026%2E+Hallucination+is+a+consequence+of+space%2Doptimality%3A+A+rate%2Ddistortion+theorem+for+membership+testing%2E+arXiv+preprint+arXiv%3A2602%2E00906%2E+%E2%80%94+%5BarXiv%3A2602%2E00906%5D%28https%3A%2F%2Farxi)
18. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). arXiv preprint arXiv:2009.03300.
19. Shengding Hu, Yifan Luo, Huadong Wang, Xingyi Cheng, Zhiyuan Liu, and Maosong Sun. 2023. Won’t get fooled again: Answering questions with false premises. arXiv preprint arXiv:2307.02394. [find](https://scholar.google.com/scholar?q=Shengding+Hu%2C+Yifan+Luo%2C+Huadong+Wang%2C+Xingyi+Cheng%2C+Zhiyuan+Liu%2C+and+Maosong+Sun%2E+2023%2E+Won%E2%80%99t+get+fooled+again%3A+Answering+questions+with+false+premises%2E+arXiv+preprint+arXiv%3A2307%2E02394%2E)
20. Hugging Face. 2025. Math-verify: A robust mathematical expression evaluation library. GitHub repository. Commit and version may vary; accessed 2025-12-12. [find](https://scholar.google.com/scholar?q=Hugging+Face%2E+2025%2E+Math%2Dverify%3A+A+robust+mathematical+expression+evaluation+library%2E+GitHub+repository%2E+Commit+and+version+may+vary%3B+accessed+2025%2D12%2D12%2E)
21. Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. 2025. [Search-r1: Training llms to reason and leverage search engines with reinforcement learning](https://md.research.elamur.ai/free/en/papers/jin-2025-search-r1-training-llms-reason/). arXiv e-prints, page arXiv:2503.09516.
22. Adam Tauman Kalai, Ofir Nachum, Santosh S Vempala, and Edwin Zhang. 2025. Why language models hallucinate. arXiv preprint arXiv:2509.04664. — [arXiv:2509.04664](https://arxiv.org/abs/2509.04664) [find](https://scholar.google.com/scholar?q=Adam+Tauman+Kalai%2C+Ofir+Nachum%2C+Santosh+S+Vempala%2C+and+Edwin+Zhang%2E+2025%2E+Why+language+models+hallucinate%2E+arXiv+preprint+arXiv%3A2509%2E04664%2E+%E2%80%94+%5BarXiv%3A2509%2E04664%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2509%2E04664%29)
23. Najoung Kim, Phu Mon Htut, Samuel Bowman, and Jackson Petty. 2023. 2: Question answering with questionable assumptions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8466– 8487. [find](https://scholar.google.com/scholar?q=Najoung+Kim%2C+Phu+Mon+Htut%2C+Samuel+Bowman%2C+and+Jackson+Petty%2E+2023%2E+2%3A+Question+answering+with+questionable+assumptions%2E+In+Proceedings+of+the+61st+Annual+Meeting+of+the+Association+for+Computational+L)
24. Peixian Ma, Xialie Zhuang, Chengjin Xu, Xuhui Jiang, Ran Chen, and Jian Guo. 2025. Sql-r1: Training natural language to sql reasoning model by reinforcement learning. arXiv e-prints, page arXiv:2504.08600. — [arXiv:2504.08600](https://arxiv.org/abs/2504.08600) [find](https://scholar.google.com/scholar?q=Peixian+Ma%2C+Xialie+Zhuang%2C+Chengjin+Xu%2C+Xuhui+Jiang%2C+Ran+Chen%2C+and+Jian+Guo%2E+2025%2E+Sql%2Dr1%3A+Training+natural+language+to+sql+reasoning+model+by+reinforcement+learning%2E+arXiv+e%2Dprints%2C+page+arXiv%3A2504%2E0)
25. Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (V… [find](https://scholar.google.com/scholar?q=Alex+Mallen%2C+Akari+Asai%2C+Victor+Zhong%2C+Rajarshi+Das%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+2023%2E+When+not+to+trust+language+models%3A+Investigating+effectiveness+of+parametric+and+non%2Dparametric+mem)
26. OpenAI. 2025. o4-mini. [Large language model]. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2025%2E+o4%2Dmini%2E+%5BLarge+language+model%5D%2E)
27. Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Jana Thompson, Phu Mon Htut, and Samuel Bowman. 2022. [Bbq: A hand-built bias benchmark for question answering](https://md.research.elamur.ai/free/en/papers/parrish-2022-bbq-hand-built-bias-benchmark/). In Findings of the Association for Computational Linguistics: ACL 2022, pages 2086–2105.
28. Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. [Know what you don’t know: Unanswerable questions for squad](https://md.research.elamur.ai/free/en/papers/rajpurkar-2018-know-what-you-don-t/). arXiv preprint arXiv:1806.03822.
29. Nino Scherrer, Claudia Shi, Amir Feder, and David Blei. 2023. Evaluating the moral beliefs encoded in llms. Advances in Neural Information Processing Systems, 36:51778–51809. [find](https://scholar.google.com/scholar?q=Nino+Scherrer%2C+Claudia+Shi%2C+Amir+Feder%2C+and+David+Blei%2E+2023%2E+Evaluating+the+moral+beliefs+encoded+in+llms%2E+Advances+in+Neural+Information+Processing+Systems%2C+36%3A51778%E2%80%9351809%2E)
30. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Filip+Wolski%2C+Prafulla+Dhariwal%2C+Alec+Radford%2C+and+Oleg+Klimov%2E+2017%2E+Proximal+policy+optimization+algorithms%2E+arXiv+preprint+arXiv%3A1707%2E06347%2E)
31. Taiwei Shi, Sihao Chen, Bowen Jiang, Linxin Song, Longqi Yang, and Jieyu Zhao. 2026. Experiential reinforcement learning. arXiv preprint arXiv:2602.13949. — [arXiv:2602.13949](https://arxiv.org/abs/2602.13949) [find](https://scholar.google.com/scholar?q=Taiwei+Shi%2C+Sihao+Chen%2C+Bowen+Jiang%2C+Linxin+Song%2C+Longqi+Yang%2C+and+Jieyu+Zhao%2E+2026%2E+Experiential+reinforcement+learning%2E+arXiv+preprint+arXiv%3A2602%2E13949%2E+%E2%80%94+%5BarXiv%3A2602%2E13949%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F26)
32. Aviv Slobodkin, Omer Goldman, Avi Caciularu, Ido Dagan, and Shauli Ravfogel. 2023. The curious case of hallucinatory (un) answerability: Finding truths in the hidden states of over-confident large language models. arXiv preprint arXiv:2310.11877. [find](https://scholar.google.com/scholar?q=Aviv+Slobodkin%2C+Omer+Goldman%2C+Avi+Caciularu%2C+Ido+Dagan%2C+and+Shauli+Ravfogel%2E+2023%2E+The+curious+case+of+hallucinatory+%28un%29+answerability%3A+Finding+truths+in+the+hidden+states+of+over%2Dconfident+large+lan)
33. Linxin Song, Taiwei Shi, and Jieyu Zhao. 2025. The hallucination tax of reinforcement finetuning. arXiv preprint arXiv:2505.13988. [find](https://scholar.google.com/scholar?q=Linxin+Song%2C+Taiwei+Shi%2C+and+Jieyu+Zhao%2E+2025%2E+The+hallucination+tax+of+reinforcement+finetuning%2E+arXiv+preprint+arXiv%3A2505%2E13988%2E)
34. Yuhong Sun, Zhangyue Yin, Qipeng Guo, Jiawen Wu, Xipeng Qiu, and Hui Zhao. 2024. Benchmarking hallucination in large language models based on unanswerable math word problem. arXiv preprint arXiv:2403.03558. [find](https://scholar.google.com/scholar?q=Yuhong+Sun%2C+Zhangyue+Yin%2C+Qipeng+Guo%2C+Jiawen+Wu%2C+Xipeng+Qiu%2C+and+Hui+Zhao%2E+2024%2E+Benchmarking+hallucination+in+large+language+models+based+on+unanswerable+math+word+problem%2E+arXiv+preprint+arXiv%3A2403%2E)
35. Qwen Team. 2024. Qwen2.5: A party of foundation models. [find](https://scholar.google.com/scholar?q=Qwen+Team%2E+2024%2E+Qwen2%2E5%3A+A+party+of+foundation+models%2E)
36. Guiyao Tie, Zeli Zhao, Dingjie Song, Fuyang Wei, Rong Zhou, Yurou Dai, Wen Yin, Zhejian Yang, Jiangyue Yan, Yao Su, Zhenhan Dai, Yifeng Xie, Yihan Cao, Lichao Sun, Pan Zhou, Lifang He, Hechang Chen, Yu Zhang, Qingsong Wen, Tianming Liu, Neil Zhenqiang Gong, Jiliang Tang, Caiming Xiong, Heng Ji, Phil… [find](https://scholar.google.com/scholar?q=Guiyao+Tie%2C+Zeli+Zhao%2C+Dingjie+Song%2C+Fuyang+Wei%2C+Rong+Zhou%2C+Yurou+Dai%2C+Wen+Yin%2C+Zhejian+Yang%2C+Jiangyue+Yan%2C+Yao+Su%2C+Zhenhan+Dai%2C+Yifeng+Xie%2C+Yihan+Cao%2C+Lichao+Sun%2C+Pan+Zhou%2C+Lifang+He%2C+Hechang+Chen%2C+Y)
37. Ante Wang, Yujie Lin, Jingyao Liu, Suhang Wu, Hao Liu, Xinyan Xiao, and Jinsong Su. 2025. Beyond passive critical thinking: Fostering proactive questioning to enhance human-ai collaboration. arXiv preprint arXiv:2507.23407. — [arXiv:2507.23407](https://arxiv.org/abs/2507.23407) [find](https://scholar.google.com/scholar?q=Ante+Wang%2C+Yujie+Lin%2C+Jingyao+Liu%2C+Suhang+Wu%2C+Hao+Liu%2C+Xinyan+Xiao%2C+and+Jinsong+Su%2E+2025%2E+Beyond+passive+critical+thinking%3A+Fostering+proactive+questioning+to+enhance+human%2Dai+collaboration%2E+arXiv+pre)
38. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). Advances in neural information processing systems, 35:24824–24837.
39. Zhepei Wei, Wenlin Yao, Yao Liu, Weizhi Zhang, Qin Lu, Liang Qiu, Changlong Yu, Puyang Xu, Chao Zhang, Bing Yin, Hyokun Yun, and Lihong Li. 2025. [Webagent-r1: Training web agents via end-to-end multi-turn reinforcement learning](https://md.research.elamur.ai/free/en/papers/wei-2025-webagent-r1-training-web-agents/). In Proceedings of the 2025 Conference on Empirical Methods in Natural L…
40. Yuqing Yang, Ethan Chern, Xipeng Qiu, Graham Neubig, and Pengfei Liu. 2024. Alignment for honesty. Advances in Neural Information Processing Systems, 37:63565–63598. [find](https://scholar.google.com/scholar?q=Yuqing+Yang%2C+Ethan+Chern%2C+Xipeng+Qiu%2C+Graham+Neubig%2C+and+Pengfei+Liu%2E+2024%2E+Alignment+for+honesty%2E+Advances+in+Neural+Information+Processing+Systems%2C+37%3A63565%E2%80%9363598%2E)
41. Zhewei Yao, Guoheng Sun, Lukasz Borchmann, Zheyu Shen, Minghang Deng, Bohan Zhai, Hao Zhang, Ang Li, and Yuxiong He. 2025a. [Arctic-text2sql-r1: Simple rewards, strong reasoning in text-to-sql](https://md.research.elamur.ai/free/en/papers/yao-2026-arctic-text2sql-r1-simple-rewards/). arXiv e-prints, page arXiv:2505.20315.
42. Zijun Yao, Yantao Liu, Yanxu Chen, Jianhui Chen, Junfeng Fang, Lei Hou, Juanzi Li, and Tat-Seng Chua. 2025b. Are reasoning models more prone to hallucination? arXiv preprint arXiv:2505.23646. — [arXiv:2505.23646](https://arxiv.org/abs/2505.23646) [find](https://scholar.google.com/scholar?q=Zijun+Yao%2C+Yantao+Liu%2C+Yanxu+Chen%2C+Jianhui+Chen%2C+Junfeng+Fang%2C+Lei+Hou%2C+Juanzi+Li%2C+and+Tat%2DSeng+Chua%2E+2025b%2E+Are+reasoning+models+more+prone+to+hallucination%3F+arXiv+preprint+arXiv%3A2505%2E23646%2E+%E2%80%94+%5BarXiv)
43. Xunjian Yin, Baizhou Huang, and Xiaojun Wan. 2023a. Alcuna: Large language models meet new knowledge. arXiv preprint arXiv:2310.14820. [find](https://scholar.google.com/scholar?q=Xunjian+Yin%2C+Baizhou+Huang%2C+and+Xiaojun+Wan%2E+2023a%2E+Alcuna%3A+Large+language+models+meet+new+knowledge%2E+arXiv+preprint+arXiv%3A2310%2E14820%2E)
44. Zhangyue Yin, Qiushi Sun, Qipeng Guo, Jiawen Wu, Xipeng Qiu, and Xuanjing Huang. 2023b. Do large language models know what they don’t know? In Findings of the Association for Computational Linguistics: ACL 2023, pages 8653–8665, Toronto, Canada. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Zhangyue+Yin%2C+Qiushi+Sun%2C+Qipeng+Guo%2C+Jiawen+Wu%2C+Xipeng+Qiu%2C+and+Xuanjing+Huang%2E+2023b%2E+Do+large+language+models+know+what+they+don%E2%80%99t+know%3F+In+Findings+of+the+Association+for+Computational+Linguistics)
45. Yanjun Zheng, Xiyang Du, Longfei Liao, Xiaoke Zhao, Zhaowen Zhou, Jingze Song, Bo Zhang, Jiawei Liu, Xiang Qi, Zhe Li, Zhiqiang Zhang, Wei Wang, and Peng Zhang. 2025. Agentar-fin-r1: Enhancing financial intelligence through domain expertise, training efficiency, and advanced reasoning. arXiv e-print… — [arXiv:2507.16802](https://arxiv.org/abs/2507.16802) [find](https://scholar.google.com/scholar?q=Yanjun+Zheng%2C+Xiyang+Du%2C+Longfei+Liao%2C+Xiaoke+Zhao%2C+Zhaowen+Zhou%2C+Jingze+Song%2C+Bo+Zhang%2C+Jiawei+Liu%2C+Xiang+Qi%2C+Zhe+Li%2C+Zhiqiang+Zhang%2C+Wei+Wang%2C+and+Peng+Zhang%2E+2025%2E+Agentar%2Dfin%2Dr1%3A+Enhancing+financi)

*Updated 31 October 2026*