---
title: "R-Search — paper by Zhao et al. (2026) — summary and ideas"
description: "LLMs with strong reasoning skills still hallucinate on knowledge-heavy questions. 47 references, cited by 7 papers here."
authors: ["Qingfei Zhao", "Ruobing Wang", "Dingling Xu", "Daren Zha", "Ma Bowen", "Zhichun Wang", "Shijie Jia", "Limin Liu", "Xin Wang"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.1896/"
doi: "10.18653/v1/2026.findings-acl.1896"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhao-2026-r-search-empowering-llm-reasoning/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhao-2026-r-search-empowering-llm-reasoning/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/2/) / R-Search: Empowering LLM Reasoning with Search via Multi-Reward Reinforcement Learning

# R-Search: Empowering LLM Reasoning with Search via Multi-Reward Reinforcement Learning

**Authors:** Qingfei Zhao, [Ruobing Wang](https://md.research.elamur.ai/free/en/authors/ruobing-wang/), [Dingling Xu](https://md.research.elamur.ai/free/en/authors/dingling-xu/), Daren Zha, Ma Bowen, [Zhichun Wang](https://md.research.elamur.ai/free/en/authors/zhichun-wang/), Shijie Jia, Limin Liu and [Xin Wang](https://md.research.elamur.ai/free/en/authors/xin-wang/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.1896/) · [PDF](https://aclanthology.org/2026.findings-acl.1896.pdf) · [DOI 10.18653/v1/2026.findings-acl.1896](https://doi.org/10.18653/v1/2026.findings-acl.1896) · [OpenAlex](https://openalex.org/W4416075279)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [7](https://md.research.elamur.ai/free/en/papers/zhao-2026-r-search-empowering-llm-reasoning/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
LLMs with strong reasoning skills still hallucinate on knowledge-heavy questions. Prompt-driven multi-step RAG pipelines also tend to choose poorly when and what to retrieve, and they keep reasoning and retrieval only loosely connected.

R-Search addresses this with a reinforcement learning framework in which the model interleaves reasoning and search on its own. It can issue a search query at any token of its chain of thought and receives retrieved passages as observations. After the interaction, it condenses the findings into an explicit evidence block before answering.

Training combines three rewards: answer correctness, the quality of the distilled evidence (judged by a separate evidence-based check model), and correct output format. These are optimized with GRPO or PPO.

The model was trained only on 2WikiMQA and tested on seven multi-hop and single-hop QA datasets. R-Search outperforms both vanilla and advanced RAG baselines, with the clearest gains on multi-hop questions. The work shows that process-level rewards make search-augmented reasoning more reliable than rewards based on the final answer alone.

## Key points
- **Reward design:** the total reward sums an answer reward, an evidence reward and a format reward.
  - Removing the evidence step, or only its rewards, noticeably lowers results, especially on multi-hop datasets.
  - Several different LLMs work as the evidence checker, and the best results generally come from using a checker distinct from the policy model.
- **RL algorithms:** experiments use Qwen2.5 3B and 7B with GRPO and PPO.
  - Both algorithms work, but GRPO reaches a higher reward ceiling and converges faster.
  - The authors attribute this to GRPO's group-mean baseline versus PPO's learned critic.
- **Search behaviour:** as training progresses, models perform more valid searches, meaning more rounds of reasoning-search interaction.
- **Training data:** training used only the 2WikiMQA training set, yet gains hold on both in-domain and out-of-domain benchmarks. The authors name broader training data as future work.
- **R-Search-Tool:** this companion component supplies distilled evidence snippets to other downstream generation models. Code and data are released at github.com/QingFei1/R-Search.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Chinese Academy of Sciences](https://md.research.elamur.ai/free/en/government/chinese-academy-of-sciences/) — Government body, Beijing, China · 194 works · 23,906 citations
- [University of Chinese Academy of Sciences](https://md.research.elamur.ai/free/en/universities/university-of-chinese-academy-of-sciences/) — University, Beijing, China · 147 works · 8,480 citations
- [Beijing Normal University](https://md.research.elamur.ai/free/en/universities/beijing-normal-university/) — University, Beijing, China · 15 works · 3,177 citations
- [Institute of Information Engineering](https://md.research.elamur.ai/free/en/labs/institute-of-information-engineering/) — Research institute, Beijing, China · 30 works · 1,411 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,426 works

## Cite

Cite as: Qingfei Zhao, Ruobing Wang, Dingling Xu, Daren Zha, Ma Bowen, Zhichun Wang, Shijie Jia, Limin Liu and Xin Wang. 2026. R-Search: Empowering LLM Reasoning with Search via Multi-Reward Reinforcement Learning. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhao2026r,
  title = {{R-Search: Empowering LLM Reasoning with Search via Multi-Reward Reinforcement Learning}},
  author = {Qingfei Zhao, Ruobing Wang, Dingling Xu, Daren Zha, Ma Bowen, Zhichun Wang, Shijie Jia, Limin Liu and Xin Wang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1896},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1896}
}
```

- [← previous ProgressLM: Towards Progress Reasoning in Vision-Language Models](https://md.research.elamur.ai/free/en/papers/zhang-2026-progresslm-progress-reasoning-vision-language/)
- [next → Red Teaming Large Reasoning Models](https://md.research.elamur.ai/free/en/papers/chen-2026-red-teaming-large-reasoning-models/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/2/)

## References

47 references, 30 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Akari Asai and Hannaneh Hajishirzi. 2020. [Logic-guided data augmentation and regularization for consistent question answering](https://md.research.elamur.ai/free/en/papers/asai-2020-logic-guided-data-augmentation-regularization/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 5642–5650. Association for Computational…
2. Akari Asai, Zeqiu Wu, Yizhong Wang, Avirup Sil, and Hannaneh Hajishirzi. 2024. [Self-rag: Learning to retrieve, generate, and critique through self-reflection](https://md.research.elamur.ai/free/en/papers/asai-2023-self-rag-learning-retrieve-generate/). In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net.
3. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… — [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
4. Qinyuan Cheng, Xiaonan Li, Shimin Li, Qin Zhu, Zhangyue Yin, Yunfan Shao, Linyang Li, Tianxiang Sun, Hang Yan, and Xipeng Qiu. 2024. Unified active retrieval for retrieval augmented generation. In Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12… [find](https://scholar.google.com/scholar?q=Qinyuan+Cheng%2C+Xiaonan+Li%2C+Shimin+Li%2C+Qin+Zhu%2C+Zhangyue+Yin%2C+Yunfan+Shao%2C+Linyang+Li%2C+Tianxiang+Sun%2C+Hang+Yan%2C+and+Xipeng+Qiu%2E+2024%2E+Unified+active+retrieval+for+retrieval+augmented+generation%2E+In+Fin)
5. Christoph Dann, Yishay Mansour, and Mehryar Mohri. 2023. Reinforcement learning can be more efficient with multiple rewards. In International Conference on Machine Learning, pages 6948–6967. PMLR. [find](https://scholar.google.com/scholar?q=Christoph+Dann%2C+Yishay+Mansour%2C+and+Mehryar+Mohri%2E+2023%2E+Reinforcement+learning+can+be+more+efficient+with+multiple+rewards%2E+In+International+Conference+on+Machine+Learning%2C+pages+6948%E2%80%936967%2E+PMLR%2E)
6. DeepSeek-AI. 2025. [Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). CoRR, abs/2501.12948.
7. Wenqi Fan, Yujuan Ding, Liangbo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. 2024. [A survey on RAG meeting llms: Towards retrieval-augmented large language models](https://md.research.elamur.ai/free/en/papers/fan-2024-survey-rag-meeting-llms-towards/). In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2024, Barcelona, S…
8. Xinyan Guan, Jiali Zeng, Fandong Meng, Chunlei Xin, Yaojie Lu, Hongyu Lin, Xianpei Han, Le Sun, and Jie Zhou. 2025. Deeprag: Thinking to retrieval step by step for large language models. CoRR, abs/2502.01142. [find](https://scholar.google.com/scholar?q=Xinyan+Guan%2C+Jiali+Zeng%2C+Fandong+Meng%2C+Chunlei+Xin%2C+Yaojie+Lu%2C+Hongyu+Lin%2C+Xianpei+Han%2C+Le+Sun%2C+and+Jie+Zhou%2E+2025%2E+Deeprag%3A+Thinking+to+retrieval+step+by+step+for+large+language+models%2E+CoRR%2C+abs%2F250)
9. Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. 2020. [Retrieval augmented language model pre-training](https://md.research.elamur.ai/free/en/papers/guu-2020-realm-retrieval-augmented-language-model/). In International conference on machine learning, pages 3929–3938. PMLR.
10. Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020. [Constructing A multi-hop QA dataset for comprehensive evaluation of reasoning steps](https://md.research.elamur.ai/free/en/papers/ho-2020-constructing-multi-hop-qa-dataset/). In Proceedings of the 28th International Conference on Computational Linguistics, COLING 2020, Barcelona, Spain (Online), December 8-13, 2020, p…
11. Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Allison Tam, Ally Bennett, Ananya Kumar, and 80 others. 20… — [OpenAI o1 System Card](https://md.research.elamur.ai/free/en/papers/openai-2024-openai-o1-system-card/)
12. Soyeong Jeong, Jinheon Baek, Sukmin Cho, Sung Ju Hwang, and Jong Park. 2024. [Adaptive-rag: Learning to adapt retrieval-augmented large language models through question complexity](https://md.research.elamur.ai/free/en/papers/jeong-2024-adaptive-rag-learning-adapt-retrieval/). In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Hu…
13. Zhengbao Jiang, Frank F. Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi-Yu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. [Active retrieval augmented generation](https://md.research.elamur.ai/free/en/papers/jiang-2023-active-retrieval-augmented-generation/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2…
14. Bowen Jin, Hansi Zeng, Zhenrui Yue, Dong Wang, Hamed Zamani, and Jiawei Han. 2025a. [Search-r1: Training llms to reason and leverage search engines with reinforcement learning](https://md.research.elamur.ai/free/en/papers/jin-2025-search-r1-training-llms-reason/). CoRR, abs/2503.09516.
15. Jiajie Jin, Yutao Zhu, Zhicheng Dou, Guanting Dong, Xinyu Yang, Chenghao Zhang, Tong Zhao, Zhao Yang, and Ji-Rong Wen. 2025b. Flashrag: A modular toolkit for efficient retrieval-augmented generation research. In Companion Proceedings of the ACM on Web Conference 2025, WWW 2025, Sydney, NSW, Australi… [find](https://scholar.google.com/scholar?q=Jiajie+Jin%2C+Yutao+Zhu%2C+Zhicheng+Dou%2C+Guanting+Dong%2C+Xinyu+Yang%2C+Chenghao+Zhang%2C+Tong+Zhao%2C+Zhao+Yang%2C+and+Ji%2DRong+Wen%2E+2025b%2E+Flashrag%3A+A+modular+toolkit+for+efficient+retrieval%2Daugmented+generation+r)
16. Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. 2017. [Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension](https://md.research.elamur.ai/free/en/papers/joshi-2017-triviaqa-large-scale-distantly-supervised/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4…
17. Leslie Pack Kaelbling, Michael L Littman, and Andrew W Moore. 1996. Reinforcement learning: A survey. Journal of artificial intelligence research, 4:237–285. [find](https://scholar.google.com/scholar?q=Leslie+Pack+Kaelbling%2C+Michael+L+Littman%2C+and+Andrew+W+Moore%2E+1996%2E+Reinforcement+learning%3A+A+survey%2E+Journal+of+artificial+intelligence+research%2C+4%3A237%E2%80%93285%2E)
18. Jaehyung Kim, Jaehyun Nam, Sangwoo Mo, Jongjin Park, Sang-Woo Lee, Minjoon Seo, Jung-Woo Ha, and Jinwoo Shin. 2024. Sure: Summarizing retrievals using answer candidates for open-domain QA of llms. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-… [find](https://scholar.google.com/scholar?q=Jaehyung+Kim%2C+Jaehyun+Nam%2C+Sangwoo+Mo%2C+Jongjin+Park%2C+Sang%2DWoo+Lee%2C+Minjoon+Seo%2C+Jung%2DWoo+Ha%2C+and+Jinwoo+Shin%2E+2024%2E+Sure%3A+Summarizing+retrievals+using+answer+candidates+for+open%2Ddomain+QA+of+llms%2E+In+)
19. Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Yi Su, John D. Co-Reyes, Avi Singh, Kate Baumli, Shariq Iqbal, Colton Bishop, Rebecca Roelofs, Lei M. Zhang, Kay McKinney, Disha Shrivastava, Cosmin Paduraru, George Tucker, Doina Precup, Feryal M. P. Behbahani, and Aleksandra Faust. 2025. [Training lang…](https://md.research.elamur.ai/free/en/papers/kumar-2024-training-language-models-self-correct/)
20. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur P. Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. [Natur…](https://md.research.elamur.ai/free/en/papers/kwiatkowski-2019-natural-questions-benchmark-question-answering/)
21. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. [Efficient memory management for large language model serving with pagedattention](https://md.research.elamur.ai/free/en/papers/kwon-2023-efficient-memory-management-large-language/). In Proceedings of the 29th Symposium on Operating Systems Principles, SOSP 2023, Koblenz…
22. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. [Retrieval-augmented generation for knowledge-intensive NLP tasks](https://md.research.elamur.ai/free/en/papers/lewis-2020-retrieval-augmented-generation-knowledge-intensive/). In Advances in Neural Information Proc…
23. Xiaoxi Li, Guanting Dong, Jiajie Jin, Yuyao Zhang, Yujia Zhou, Yutao Zhu, Peitian Zhang, and Zhicheng Dou. 2025. [Search-o1: Agentic search-enhanced large reasoning models](https://md.research.elamur.ai/free/en/papers/li-2025-search-o1-agentic-search-enhanced/). In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 5420–5438, Suzhou, China. Assoc…
24. Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (V… [find](https://scholar.google.com/scholar?q=Alex+Mallen%2C+Akari+Asai%2C+Victor+Zhong%2C+Rajarshi+Das%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+2023%2E+When+not+to+trust+language+models%3A+Investigating+effectiveness+of+parametric+and+non%2Dparametric+mem)
25. Meta. 2024. Llama 3.2: Revolutionizing edge ai and vision with open, customizable models. [find](https://scholar.google.com/scholar?q=Meta%2E+2024%2E+Llama+3%2E2%3A+Revolutionizing+edge+ai+and+vision+with+open%2C+customizable+models%2E)
26. Tong Mu, Alec Helyar, Johannes Heidecke, Joshua Achiam, Andrea Vallone, Ian Kivlichan, Molly Lin, Alex Beutel, John Schulman, and Lilian Weng. 2024. Rule based rewards for language model safety. In Advances in Neural Information Processing Systems, volume 37, pages 108877–108901. Curran Associates,… [find](https://scholar.google.com/scholar?q=Tong+Mu%2C+Alec+Helyar%2C+Johannes+Heidecke%2C+Joshua+Achiam%2C+Andrea+Vallone%2C+Ian+Kivlichan%2C+Molly+Lin%2C+Alex+Beutel%2C+John+Schulman%2C+and+Lilian+Weng%2E+2024%2E+Rule+based+rewards+for+language+model+safety%2E+In+Ad)
27. OpenAI. 2023. [GPT-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). abs/2303.08774.
28. CoRR, Aske Plaat, Annie Wong, Suzan Verberne, Joost Broekens, Niki van Stein, and Thomas Bäck. 2026. Multi-step reasoning with large language models, a survey. ACM Comput. Surv., 58(6):160:1–160:35. [find](https://scholar.google.com/scholar?q=CoRR%2C+Aske+Plaat%2C+Annie+Wong%2C+Suzan+Verberne%2C+Joost+Broekens%2C+Niki+van+Stein%2C+and+Thomas+B%C3%A4ck%2E+2026%2E+Multi%2Dstep+reasoning+with+large+language+models%2C+a+survey%2E+ACM+Comput%2E+Surv%2E%2C+58%286%29%3A160%3A1%E2%80%93160%3A35%2E)
29. Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A. Smith, and Mike Lewis. 2023. Measuring and narrowing the compositionality gap in language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023, pages 5687–5711. Association for Comp… [find](https://scholar.google.com/scholar?q=Ofir+Press%2C+Muru+Zhang%2C+Sewon+Min%2C+Ludwig+Schmidt%2C+Noah+A%2E+Smith%2C+and+Mike+Lewis%2E+2023%2E+Measuring+and+narrowing+the+compositionality+gap+in+language+models%2E+In+Findings+of+the+Association+for+Computat)
30. Ori Ram, Yoav Levine, Itay Dalmedigos, Dor Muhlgay, Amnon Shashua, Kevin Leyton-Brown, and Yoav Shoham. 2023. [In-context retrieval-augmented language models](https://md.research.elamur.ai/free/en/papers/ram-2023-context-retrieval-augmented-language-models/). Trans. Assoc. Comput. Linguistics, 11:1316–1331.
31. Devendra Singh Sachan, Siva Reddy, William L. Hamilton, Chris Dyer, and Dani Yogatama. 2021. End-to-end training of multi-document reader and retriever for [open-domain question answering](https://md.research.elamur.ai/free/en/papers/chen-2020-open-domain-question-answering/). In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Syst…
32. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. CoRR, abs/1707.06347. [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Filip+Wolski%2C+Prafulla+Dhariwal%2C+Alec+Radford%2C+and+Oleg+Klimov%2E+2017%2E+Proximal+policy+optimization+algorithms%2E+CoRR%2C+abs%2F1707%2E06347%2E)
33. Zhihong Shao, Yeyun Gong, Yelong Shen, Minlie Huang, Nan Duan, and Weizhu Chen. 2023. [Enhancing retrieval-augmented large language models with iterative retrieval-generation synergy](https://md.research.elamur.ai/free/en/papers/shao-2023-enhancing-retrieval-augmented-large-language/). In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023, pages 9248–…
34. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). CoRR, abs/2402.03300.
35. Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, and Ji-Rong Wen. 2025. [R1-searcher: Incentivizing the search capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/song-2025-r1-searcher-incentivizing-search/). CoRR, abs/2503.05592.
36. Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. Musique: Multi-hop questions via single-hop question composition. Trans. Assoc. Comput. Linguistics, 10:539–554. [find](https://scholar.google.com/scholar?q=Harsh+Trivedi%2C+Niranjan+Balasubramanian%2C+Tushar+Khot%2C+and+Ashish+Sabharwal%2E+2022%2E+Musique%3A+Multi%2Dhop+questions+via+single%2Dhop+question+composition%2E+Trans%2E+Assoc%2E+Comput%2E+Linguistics%2C+10%3A539%E2%80%93554%2E)
37. Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2023. [Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions](https://md.research.elamur.ai/free/en/papers/trivedi-2022-interleaving-retrieval-chain-thought-reasoning/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), A…
38. Ruobing Wang, Qingfei Zhao, Yukun Yan, Daren Zha, Yuxuan Chen, Shi Yu, Zhenghao Liu, Yixuan Wang, Shuo Wang, Xu Han, Zhiyuan Liu, and Maosong Sun. 2025. Deepnote: Note-centric deep retrieval-augmented generation. In Findings of the Association for Computational Linguistics: EMNLP 2025, Suzhou, China… [find](https://scholar.google.com/scholar?q=Ruobing+Wang%2C+Qingfei+Zhao%2C+Yukun+Yan%2C+Daren+Zha%2C+Yuxuan+Chen%2C+Shi+Yu%2C+Zhenghao+Liu%2C+Yixuan+Wang%2C+Shuo+Wang%2C+Xu+Han%2C+Zhiyuan+Liu%2C+and+Maosong+Sun%2E+2025%2E+Deepnote%3A+Note%2Dcentric+deep+retrieval%2Daugmented)
39. Yixuan Weng, Minjun Zhu, Fei Xia, Bin Li, Shizhu He, Shengping Liu, Bin Sun, Kang Liu, and Jun Zhao. 2023. [Large language models are better reasoners with self-verification](https://md.research.elamur.ai/free/en/papers/weng-2023-large-language-models-are-better/). In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023, Findings of ACL, pag…
40. Marco A Wiering and Martijn Van Otterlo. 2012. Reinforcement learning. Adaptation, learning, and optimization, 12(3):729. [find](https://scholar.google.com/scholar?q=Marco+A+Wiering+and+Martijn+Van+Otterlo%2E+2012%2E+Reinforcement+learning%2E+Adaptation%2C+learning%2C+and+optimization%2C+12%283%29%3A729%2E)
41. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, and 22 others. 2024. Qwen2.5 technical report. CoRR, abs/2412.1511… [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Haoran+Wei%2C+Huan+Lin%2C+Jian+Yang%2C+Jianhong+Tu%2C+Jianwei+Zhang%2C+Jianxin+Yang%2C+Jiaxi+Yang%2C+Jin)
42. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. [Hotpotqa: A dataset for diverse, explainable multi-hop question answering](https://md.research.elamur.ai/free/en/papers/yang-2018-hotpotqa-dataset-diverse-explainable-multi/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussel…
43. Ori Yoran, Tomer Wolfson, Ori Ram, and Jonathan Berant. 2024. [Making retrieval-augmented language models robust to irrelevant context](https://md.research.elamur.ai/free/en/papers/yoran-2023-making-retrieval-augmented-language-models/). In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net.
44. Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Diego Rojas, Guanyu Feng, Hanlin Zhao, Hanyu Lai, Hao Yu, Hongning Wang, Jiadai Sun, Jiajie Zhang, Jiale Cheng, Jiayi Gui, Jie Tang, Jing Zhang, Juanzi Li, Lei Zhao, and 36 others. 2024. [Chatglm: A family of large language models from GLM-130B t…](https://md.research.elamur.ai/free/en/papers/glm-2024-chatglm-family-large-language-models/)
45. Jixiao Zhang and Chunsheng Zuo. 2025. [GRPO-LEAD: A difficulty-aware reinforcement learning approach for concise mathematical reasoning in language models](https://md.research.elamur.ai/free/en/papers/zhang-2025-grpo-lead-difficulty-aware-reinforcement/). In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, EMNLP 2025, Suzhou, China, November 4-9, 2025, pages 5…
46. Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, Longyue Wang, Anh Tuan Luu, Wei Bi, Freda Shi, and Shuming Shi. 2023. [Siren’s song in the AI ocean: A survey on hallucination in large language models](https://md.research.elamur.ai/free/en/papers/zhang-2023-siren-s-song-ai-ocean/). CoRR, abs/2309.01219.
47. Qingfei Zhao, Ruobing Wang, Yukuo Cen, Daren Zha, Shicheng Tan, Yuxiao Dong, and Jie Tang. 2024. Longrag: A dual-perspective retrieval-augmented generation paradigm for long-context question answering. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, EMNLP 2… [find](https://scholar.google.com/scholar?q=Qingfei+Zhao%2C+Ruobing+Wang%2C+Yukuo+Cen%2C+Daren+Zha%2C+Shicheng+Tan%2C+Yuxiao+Dong%2C+and+Jie+Tang%2E+2024%2E+Longrag%3A+A+dual%2Dperspective+retrieval%2Daugmented+generation+paradigm+for+long%2Dcontext+question+answering)

*Updated 1 November 2026*