---
title: "ChatR1 — paper by Lupart et al. (2026) — summary and ideas"
description: "Conversational question answering (CQA) is hard because later user turns often leave things… 11.8 F1. 48 references, cited by 4 papers here."
authors: ["Simon Lupart", "Mohammad Aliannejadi", "Evangelos Kanoulas"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.103/"
doi: "10.18653/v1/2026.acl-long.103"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/lupart-2026-chatr1-reinforcement-learning-conversational-reasoning/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/lupart-2026-chatr1-reinforcement-learning-conversational-reasoning/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/3/) / ChatR1: Reinforcement Learning for Conversational Reasoning and Retrieval Augmented Question Answering

# ChatR1: Reinforcement Learning for Conversational Reasoning and Retrieval Augmented Question Answering

**Authors:** [Simon Lupart](https://md.research.elamur.ai/free/en/authors/simon-lupart/), [Mohammad Aliannejadi](https://md.research.elamur.ai/free/en/authors/mohammad-aliannejadi/) and [Evangelos Kanoulas](https://md.research.elamur.ai/free/en/authors/evangelos-kanoulas/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.103/) · [PDF](https://aclanthology.org/2026.acl-long.103.pdf) · [DOI 10.18653/v1/2026.acl-long.103](https://doi.org/10.18653/v1/2026.acl-long.103) · [OpenAlex](https://openalex.org/W4415275737)

**Citations:** 1 citations (OpenAlex, 2026-10-14) · [4](https://md.research.elamur.ai/free/en/papers/lupart-2026-chatr1-reinforcement-learning-conversational-reasoning/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Conversational question answering (CQA) is hard because later user turns often leave things unstated and rely on earlier dialogue. Existing reasoning-with-search models trained by reinforcement learning handle only single, self-contained questions. Conversational systems, meanwhile, mostly use fixed supervised pipelines that rewrite, retrieve and then generate. ChatR1 closes this gap with a policy LLM that interleaves thinking, search calls and answering within each turn, trained end-to-end with PPO. Outcome-only rewards are sparse, so the authors add an intent-aware reward. It scores each issued search query against human-annotated query rewrites, which serve as canonical user intent and are used only during training. Experiments with 3B and 7B backbones cover five CQA datasets spanning topic shifts, mixed initiative and multi-document grounding. ChatR1 beats competitive baselines, including SFT, on F1, BERTScore and LLM-as-judge, both in-domain and out-of-domain. Plain RL alone barely improves on SFT; the intent reward is what makes the difference. The work shows RL can learn adaptive, context-aware search behaviour in dialogue.

## Key points
- Evaluation spans five datasets (TopiOCQA, QReCC, plus INS, MD2D, FD in the tables). Models and data are released on Hugging Face (collection `slupart/chatr1`).
- PPO trains more stably than GRPO, which tends to collapse after 100–200 steps. Even so, GRPO reached better final scores on two datasets.
- TopiOCQA lacks human rewrites, so silver intent labels were tested. Rewrites from T5, Mistral and GPT-4.1 gave F1 of 24.7, 27.7 and 29.4 respectively, meaning better rewrites produce better answers.
- The models were trained with dense retrieval. Switching to BM25 at inference costs 11.8 F1 for 3B but only 7.8 F1 for 7B, while adding reranking helps; for example, ChatR1-7B on QReCC rises from 31.0 to 33.1 F1.
- Stated limitations include:
  - dialogues of only moderate length (10–20 turns);
  - no personalization;
  - extra compute cost from RL.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Amsterdam](https://md.research.elamur.ai/free/en/universities/university-of-amsterdam/) — University, Amsterdam, Netherlands · 144 works · 173,906 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works

🧭 Topics (4): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,426 works · [🗨️ Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) · 624 works · [❓ Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/) · 441 works · [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) · 473 works

## Cite

Cite as: Simon Lupart, Mohammad Aliannejadi and Evangelos Kanoulas. 2026. ChatR1: Reinforcement Learning for Conversational Reasoning and Retrieval Augmented Question Answering. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{lupart2026chatr1,
  title = {{ChatR1: Reinforcement Learning for Conversational Reasoning and Retrieval Augmented Question Answering}},
  author = {Simon Lupart, Mohammad Aliannejadi and Evangelos Kanoulas},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.103},
  url = {https://doi.org/10.18653/v1/2026.acl-long.103}
}
```

- [← previous ChartVerse: Scaling Chart Reasoning via Reliable Programmatic Synthesis from Scratch](https://md.research.elamur.ai/free/en/papers/liu-2026-chartverse-scaling-chart-reasoning-reliable/)
- [next → CODERL+: Improving Code Generation via Reinforcement with Execution Semantics Alignment](https://md.research.elamur.ai/free/en/papers/jiang-2026-coderl-improving-code-generation-reinforcement/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/3/)

## References

48 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Zahra Abbasiantaeb, Simon Lupart, and Mohammad Aliannejadi. 2026. Generating multi-aspect queries for conversational search. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8201–8217, Rabat, Morocco. Associ… [find](https://scholar.google.com/scholar?q=Zahra+Abbasiantaeb%2C+Simon+Lupart%2C+and+Mohammad+Aliannejadi%2E+2026%2E+Generating+multi%2Daspect+queries+for+conversational+search%2E+In+Proceedings+of+the+19th+Conference+of+the+European+Chapter+of+the+Associ)
2. Zahra Abbasiantaeb, Simon Lupart, Leif Azzopardi, Jeffrey Dalton, and Mohammad Aliannejadi. 2025. Conversational gold: Evaluating personalized conversational search system using gold nuggets. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Ret… [find](https://scholar.google.com/scholar?q=Zahra+Abbasiantaeb%2C+Simon+Lupart%2C+Leif+Azzopardi%2C+Jeffrey+Dalton%2C+and+Mohammad+Aliannejadi%2E+2025%2E+Conversational+gold%3A+Evaluating+personalized+conversational+search+system+using+gold+nuggets%2E+In+Proce)
3. Emre Can Acikgoz, Jeremiah Greer, Akul Datta, Ze Yang, William Zeng, Oussama Elachqar, Emmanouil Koukoumidis, Dilek Hakkani-Tür, and Gokhan Tur. 2025. [Can a single model master both multi-turn conversations and tool use? CoALM: A unified conversational agentic language model](https://md.research.elamur.ai/free/en/papers/acikgoz-2025-can-single-model-master-both/). In Proceedings of the 6…
4. Vaibhav Adlakha, Shehzaad Dhuliawala, Kaheer Suleman, Harm de Vries, and Siva Reddy. 2022. [TopiOCQA: Open-domain conversational question answering with topic switching](https://md.research.elamur.ai/free/en/papers/adlakha-2022-topiocqa-open-domain-conversational-question/). Transactions of the Association for Computational Linguistics, 10:468– 483.
5. Mohammad Aliannejadi, Zahra Abbasiantaeb, Shubham Chatterjee, Jeffrey Dalton, and Leif Azzopardi. 2024. Trec ikat 2023: A test collection for evaluating conversational and interactive knowledge assistants. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in I… [find](https://scholar.google.com/scholar?q=Mohammad+Aliannejadi%2C+Zahra+Abbasiantaeb%2C+Shubham+Chatterjee%2C+Jeffrey+Dalton%2C+and+Leif+Azzopardi%2E+2024%2E+Trec+ikat+2023%3A+A+test+collection+for+evaluating+conversational+and+interactive+knowledge+assist)
6. Raviteja Anantha, Svitlana Vakulenko, Zhucheng Tu, Shayne Longpre, Stephen Pulman, and Srinivas Chappidi. 2021. [Open-domain question answering goes conversational via question rewriting](https://md.research.elamur.ai/free/en/papers/anantha-2021-open-domain-question-answering-goes/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguist…
7. Mingyang Chen, Tianpeng Li, Haoze Sun, Yijie Zhou, Chenzheng Zhu, Haofen Wang, Jeff Z Pan, Wen Zhang, Huajun Chen, Fan Yang, and 1 others. 2025. Learning to reason with search for llms via reinforcement learning. arXiv preprint arXiv:2503.19470. [find](https://scholar.google.com/scholar?q=Mingyang+Chen%2C+Tianpeng+Li%2C+Haoze+Sun%2C+Yijie+Zhou%2C+Chenzheng+Zhu%2C+Haofen+Wang%2C+Jeff+Z+Pan%2C+Wen+Zhang%2C+Huajun+Chen%2C+Fan+Yang%2C+and+1+others%2E+2025%2E+Learning+to+reason+with+search+for+llms+via+reinforceme)
8. Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V Le, Sergey Levine, and Yi Ma. 2025. [Sft memorizes, rl generalizes: A comparative study of foundation model post-training](https://md.research.elamur.ai/free/en/papers/chu-2025-sft-memorizes-rl-generalizes-comparative/). arXiv preprint arXiv:2501.17161.
9. Jeffrey Dalton, Sophie Fischer, Paul Owoicho, Filip Radlinski, Federico Rossetto, Johanne R. Trippas, and Hamed Zamani. 2022. Conversational information seeking: Theory and application. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval… [find](https://scholar.google.com/scholar?q=Jeffrey+Dalton%2C+Sophie+Fischer%2C+Paul+Owoicho%2C+Filip+Radlinski%2C+Federico+Rossetto%2C+Johanne+R%2E+Trippas%2C+and+Hamed+Zamani%2E+2022%2E+Conversational+information+seeking%3A+Theory+and+application%2E+In+Proceedings)
10. Jeffrey Dalton, Chenyan Xiong, Vaibhav Kumar, and Jamie Callan. 2020. Cast-19: A dataset for conversational information seeking. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’20, page 1985–1988, New York, NY, USA. Associati… [find](https://scholar.google.com/scholar?q=Jeffrey+Dalton%2C+Chenyan+Xiong%2C+Vaibhav+Kumar%2C+and+Jamie+Callan%2E+2020%2E+Cast%2D19%3A+A+dataset+for+conversational+information+seeking%2E+In+Proceedings+of+the+43rd+International+ACM+SIGIR+Conference+on+Resear)
11. Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. 2018. [Wizard of wikipedia: Knowledge-powered conversational agents](https://md.research.elamur.ai/free/en/papers/dinan-2018-wizard-wikipedia-knowledge-powered-conversational/). arXiv preprint arXiv:1811.01241.
12. Nouha Dziri, Ehsan Kamalloo, Sivan Milton, Osmar Zaiane, Mo Yu, Edoardo M. Ponti, and Siva Reddy. 2022. [FaithDial: A faithful benchmark for information-seeking dialogue](https://md.research.elamur.ai/free/en/papers/dziri-2022-faithdial-faithful-benchmark-information-seeking/). Transactions of the Association for Computational Linguistics, 10:1473– 1490.
13. Ahmed Elgohary, Denis Peskov, and Jordan Boyd-Graber. 2019. [Can you unpack that? learning to rewrite questions-in-context](https://md.research.elamur.ai/free/en/papers/elgohary-2019-can-you-unpack-that-learning/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP…
14. Song Feng, Siva Sankalp Patel, Hui Wan, and Sachindra Joshi. 2021. MultiDoc2Dial: Modeling dialogues grounded in multiple documents. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6162–6176, Online and Punta Cana, Dominican Republic. Association for… [find](https://scholar.google.com/scholar?q=Song+Feng%2C+Siva+Sankalp+Patel%2C+Hui+Wan%2C+and+Sachindra+Joshi%2E+2021%2E+MultiDoc2Dial%3A+Modeling+dialogues+grounded+in+multiple+documents%2E+In+Proceedings+of+the+2021+Conference+on+Empirical+Methods+in+Natur)
15. Marcel Gohsen, Zahra Abbasiantaeb, Mohammad Aliannejadi, Krisztian Balog, Timo Breuer, Jeffrey Dalton, Maik Fröbe, Christin Katharina Kreutz, Andreas Kruff, Simon Lupart, and 1 others. 2025. User simulation in practice: Lessons learned from three shared tasks. In ACM SIGIR Forum, volume 59, pages 1–… [find](https://scholar.google.com/scholar?q=Marcel+Gohsen%2C+Zahra+Abbasiantaeb%2C+Mohammad+Aliannejadi%2C+Krisztian+Balog%2C+Timo+Breuer%2C+Jeffrey+Dalton%2C+Maik+Fr%C3%B6be%2C+Christin+Katharina+Kreutz%2C+Andreas+Kruff%2C+Simon+Lupart%2C+and+1+others%2E+2025%2E+User+simu)
16. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. [Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). arXiv preprint arXiv:2501.12948.
17. Kalervo Järvelin and Jaana Kekäläinen. 2000. Ir evaluation methods for retrieving highly relevant documents. In Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’00, pages 41–48, New York, NY, USA. Association for Computing… [find](https://scholar.google.com/scholar?q=Kalervo+J%C3%A4rvelin+and+Jaana+Kek%C3%A4l%C3%A4inen%2E+2000%2E+Ir+evaluation+methods+for+retrieving+highly+relevant+documents%2E+In+Proceedings+of+the+23rd+Annual+International+ACM+SIGIR+Conference+on+Research+and+Develo)
18. Bowen Jin, Jinsung Yoon, Priyanka Kargupta, Sercan O Arik, and Jiawei Han. 2025a. An empirical study on reinforcement learning for reasoning-search interleaved llm agents. arXiv preprint arXiv:2505.15117. — [arXiv:2505.15117](https://arxiv.org/abs/2505.15117) [find](https://scholar.google.com/scholar?q=Bowen+Jin%2C+Jinsung+Yoon%2C+Priyanka+Kargupta%2C+Sercan+O+Arik%2C+and+Jiawei+Han%2E+2025a%2E+An+empirical+study+on+reinforcement+learning+for+reasoning%2Dsearch+interleaved+llm+agents%2E+arXiv+preprint+arXiv%3A2505%2E15)
19. Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. 2025b. [Search-r1: Training llms to reason and leverage search engines with reinforcement learning](https://md.research.elamur.ai/free/en/papers/jin-2025-search-r1-training-llms-reason/). arXiv preprint arXiv:2503.09516.
20. Leslie Pack Kaelbling, Michael L. Littman, and Andrew W. Moore. 1996. Reinforcement learning: a survey. J. Artif. Int. Res., 4(1):237–285. [find](https://scholar.google.com/scholar?q=Leslie+Pack+Kaelbling%2C+Michael+L%2E+Littman%2C+and+Andrew+W%2E+Moore%2E+1996%2E+Reinforcement+learning%3A+a+survey%2E+J%2E+Artif%2E+Int%2E+Res%2E%2C+4%281%29%3A237%E2%80%93285%2E)
21. Wenjun Li, Zhi Chen, Jingru Lin, Hannan Cao, Wei Han, Sheng Liang, Zhi Zhang, Kuicai Dong, Dexun Li, Chen Zhang, and 1 others. 2025. Reinforcement learning foundations for deep research systems: A survey. arXiv preprint arXiv:2509.06733. — [arXiv:2509.06733](https://arxiv.org/abs/2509.06733) [find](https://scholar.google.com/scholar?q=Wenjun+Li%2C+Zhi+Chen%2C+Jingru+Lin%2C+Hannan+Cao%2C+Wei+Han%2C+Sheng+Liang%2C+Zhi+Zhang%2C+Kuicai+Dong%2C+Dexun+Li%2C+Chen+Zhang%2C+and+1+others%2E+2025%2E+Reinforcement+learning+foundations+for+deep+research+systems%3A+A+sur)
22. Zihan Liu, Wei Ping, Rajarshi Roy, Peng Xu, Chankyu Lee, Mohammad Shoeybi, and Bryan Catanzaro. 2025. [Chatqa: surpassing gpt-4 on conversational qa and rag](https://md.research.elamur.ai/free/en/papers/liu-2024-chatqa-surpassing-gpt-4-conversational/). In Proceedings of the 38th International Conference on Neural Information Processing Systems, NIPS ’24, Red Hook, NY, USA. Curran Associates In…
23. Simon Lupart, Mohammad Aliannejadi, and Evangelos Kanoulas. 2025a. Disco: Llm knowledge distillation for efficient sparse retrieval in conversational search. In Proceedings of the 48th International ACM SI-GIR Conference on Research and Development in Information Retrieval, pages 9–19. [find](https://scholar.google.com/scholar?q=Simon+Lupart%2C+Mohammad+Aliannejadi%2C+and+Evangelos+Kanoulas%2E+2025a%2E+Disco%3A+Llm+knowledge+distillation+for+efficient+sparse+retrieval+in+conversational+search%2E+In+Proceedings+of+the+48th+International+A)
24. Simon Lupart, Daniël van Dijk, Eric Langezaal, Ian van Dort, and Mohammad Aliannejadi. 2025b. Investigating llm variability in personalized conversational information retrieval. In Proceedings of the 2025 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval… [find](https://scholar.google.com/scholar?q=Simon+Lupart%2C+Dani%C3%ABl+van+Dijk%2C+Eric+Langezaal%2C+Ian+van+Dort%2C+and+Mohammad+Aliannejadi%2E+2025b%2E+Investigating+llm+variability+in+personalized+conversational+information+retrieval%2E+In+Proceedings+of+the+)
25. Kelong Mao, Chenlong Deng, Haonan Chen, Fengran Mo, Zheng Liu, Tetsuya Sakai, and Zhicheng Dou. 2024. ChatRetriever: Adapting large language models for generalized and robust conversational dense retrieval. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pa… [find](https://scholar.google.com/scholar?q=Kelong+Mao%2C+Chenlong+Deng%2C+Haonan+Chen%2C+Fengran+Mo%2C+Zheng+Liu%2C+Tetsuya+Sakai%2C+and+Zhicheng+Dou%2E+2024%2E+ChatRetriever%3A+Adapting+large+language+models+for+generalized+and+robust+conversational+dense+retr)
26. Mihran Miroyan, Tsung-Han Wu, Logan King, Tianle Li, Jiayi Pan, Xinyan Hu, Wei-Lin Chiang, Anastasios N Angelopoulos, Trevor Darrell, Narges Norouzi, and 1 others. 2025. Search arena: Analyzing search-augmented llms. arXiv preprint arXiv:2506.05334. [find](https://scholar.google.com/scholar?q=Mihran+Miroyan%2C+Tsung%2DHan+Wu%2C+Logan+King%2C+Tianle+Li%2C+Jiayi+Pan%2C+Xinyan+Hu%2C+Wei%2DLin+Chiang%2C+Anastasios+N+Angelopoulos%2C+Trevor+Darrell%2C+Narges+Norouzi%2C+and+1+others%2E+2025%2E+Search+arena%3A+Analyzing+search)
27. Fengran Mo, Yifan Gao, Chuan Meng, Xin Liu, Zhuofeng Wu, Kelong Mao, Zhengyang Wang, Pei Chen, Zheng Li, Xian Li, Bing Yin, and Meng Jiang. 2025a. UniConv: Unifying retrieval and response generation for large language models in conversations. In Proceedings of the 63rd Annual Meeting of the Associat… [find](https://scholar.google.com/scholar?q=Fengran+Mo%2C+Yifan+Gao%2C+Chuan+Meng%2C+Xin+Liu%2C+Zhuofeng+Wu%2C+Kelong+Mao%2C+Zhengyang+Wang%2C+Pei+Chen%2C+Zheng+Li%2C+Xian+Li%2C+Bing+Yin%2C+and+Meng+Jiang%2E+2025a%2E+UniConv%3A+Unifying+retrieval+and+response+generation+f)
28. Fengran Mo, Kelong Mao, Ziliang Zhao, Hongjin Qian, Haonan Chen, Yiruo Cheng, Xiaoxi Li, Yutao Zhu, Zhicheng Dou, and Jian-Yun Nie. 2025b. A survey of conversational search. ACM Trans. Inf. Syst., 43(6). [find](https://scholar.google.com/scholar?q=Fengran+Mo%2C+Kelong+Mao%2C+Ziliang+Zhao%2C+Hongjin+Qian%2C+Haonan+Chen%2C+Yiruo+Cheng%2C+Xiaoxi+Li%2C+Yutao+Zhu%2C+Zhicheng+Dou%2C+and+Jian%2DYun+Nie%2E+2025b%2E+A+survey+of+conversational+search%2E+ACM+Trans%2E+Inf%2E+Syst%2E%2C+43%28)
29. Chen Qu, Liu Yang, Cen Chen, Minghui Qiu, W. Bruce Croft, and Mohit Iyyer. 2020. Open-retrieval conversational question answering. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’20, page 539–548, New York, NY, USA. Associati… [find](https://scholar.google.com/scholar?q=Chen+Qu%2C+Liu+Yang%2C+Cen+Chen%2C+Minghui+Qiu%2C+W%2E+Bruce+Croft%2C+and+Mohit+Iyyer%2E+2020%2E+Open%2Dretrieval+conversational+question+answering%2E+In+Proceedings+of+the+43rd+International+ACM+SIGIR+Conference+on+Rese)
30. Filip Radlinski and Nick Craswell. 2017. A theoretical framework for conversational search. In Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval, CHIIR ’17, page 117–126, New York, NY, USA. Association for Computing Machinery. [find](https://scholar.google.com/scholar?q=Filip+Radlinski+and+Nick+Craswell%2E+2017%2E+A+theoretical+framework+for+conversational+search%2E+In+Proceedings+of+the+2017+Conference+on+Conference+Human+Information+Interaction+and+Retrieval%2C+CHIIR+%E2%80%9917%2C+)
31. David Rau, Hervé Déjean, Nadezhda Chirkova, Thibault Formal, Shuai Wang, Stéphane Clinchant, and Vassilina Nikoulina. 2024. BERGEN: A benchmarking library for retrieval-augmented generation. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 7640–7663, Miami, Florida, US… [find](https://scholar.google.com/scholar?q=David+Rau%2C+Herv%C3%A9+D%C3%A9jean%2C+Nadezhda+Chirkova%2C+Thibault+Formal%2C+Shuai+Wang%2C+St%C3%A9phane+Clinchant%2C+and+Vassilina+Nikoulina%2E+2024%2E+BERGEN%3A+A+benchmarking+library+for+retrieval%2Daugmented+generation%2E+In+Findin)
32. Timo Schick, Jane Dwivedi-Yu, Roberto Dessí, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. [Toolformer: language models can teach themselves to use tools](https://md.research.elamur.ai/free/en/papers/schick-2023-toolformer-language-models-can-teach/). In Proceedings of the 37th International Conference on Neural Information Processing S…
33. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). arXiv preprint arXiv:2505.09388.
34. John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel. 2015. High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438. [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Philipp+Moritz%2C+Sergey+Levine%2C+Michael+Jordan%2C+and+Pieter+Abbeel%2E+2015%2E+High%2Ddimensional+continuous+control+using+generalized+advantage+estimation%2E+arXiv+preprint+arXiv%3A1506%2E02438%2E)
35. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Filip+Wolski%2C+Prafulla+Dhariwal%2C+Alec+Radford%2C+and+Oleg+Klimov%2E+2017%2E+Proximal+policy+optimization+algorithms%2E+arXiv+preprint+arXiv%3A1707%2E06347%2E)
36. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and 1 others. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). arXiv preprint arXiv:2402.03300.
37. Joykirat Singh, Raghav Magazine, Yash Pandya, and Akshay Nambi. 2025. Agentic reasoning and tool integration for llms via reinforcement learning. arXiv preprint arXiv:2505.01441. — [arXiv:2505.01441](https://arxiv.org/abs/2505.01441) [find](https://scholar.google.com/scholar?q=Joykirat+Singh%2C+Raghav+Magazine%2C+Yash+Pandya%2C+and+Akshay+Nambi%2E+2025%2E+Agentic+reasoning+and+tool+integration+for+llms+via+reinforcement+learning%2E+arXiv+preprint+arXiv%3A2505%2E01441%2E+%E2%80%94+%5BarXiv%3A2505%2E01441%5D%28)
38. Shi Yu, Zhenghao Liu, Chenyan Xiong, Tao Feng, and Zhiyuan Liu. 2021. Few-shot conversational dense retrieval. In Proceedings of the 44th International ACM SIGIR Conference on research and development in information retrieval, pages 829–838. [find](https://scholar.google.com/scholar?q=Shi+Yu%2C+Zhenghao+Liu%2C+Chenyan+Xiong%2C+Tao+Feng%2C+and+Zhiyuan+Liu%2E+2021%2E+Few%2Dshot+conversational+dense+retrieval%2E+In+Proceedings+of+the+44th+International+ACM+SIGIR+Conference+on+research+and+development)
39. Kaiyan Zhang, Yuxin Zuo, Bingxiang He, Youbang Sun, Runze Liu, Che Jiang, Yuchen Fan, Kai Tian, Guoli Jia, Pengfei Li, and 1 others. 2025. A survey of reinforcement learning for large reasoning models. arXiv preprint arXiv:2509.08827. — [arXiv:2509.08827](https://arxiv.org/abs/2509.08827) [find](https://scholar.google.com/scholar?q=Kaiyan+Zhang%2C+Yuxin+Zuo%2C+Bingxiang+He%2C+Youbang+Sun%2C+Runze+Liu%2C+Che+Jiang%2C+Yuchen+Fan%2C+Kai+Tian%2C+Guoli+Jia%2C+Pengfei+Li%2C+and+1+others%2E+2025%2E+A+survey+of+reinforcement+learning+for+large+reasoning+models)
40. Yuxiang Zheng, Dayuan Fu, Xiangkun Hu, Xiaojie Cai, Lyumanshan Ye, Pengrui Lu, and Pengfei Liu. 2025. [Deepresearcher: Scaling deep research via reinforcement learning in real-world environments](https://md.research.elamur.ai/free/en/papers/zheng-2025-deepresearcher-scaling-deep-research-reinforcement/). arXiv preprint arXiv:2504.03160.
41. Changtai Zhu, Siyin Wang, Ruijun Feng, Kai Song, and Xipeng Qiu. 2025. [Convsearch-r1: Enhancing query reformulation for conversational search with reasoning via reinforcement learning](https://md.research.elamur.ai/free/en/papers/zhu-2025-convsearch-r1-enhancing-query-reformulation/). arXiv preprint arXiv:2505.15776.
42. Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, and Ji-Rong Wen. 2025. [R1-searcher: Incentivizing the search capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/song-2025-r1-searcher-incentivizing-search/). arXiv preprint arXiv:2503.05592.
43. Richard S. Sutton and Andrew G. Barto. 2018. Reinforcement Learning: An Introduction. A Bradford Book, Cambridge, MA, USA. [find](https://scholar.google.com/scholar?q=Richard+S%2E+Sutton+and+Andrew+G%2E+Barto%2E+2018%2E+Reinforcement+Learning%3A+An+Introduction%2E+A+Bradford+Book%2C+Cambridge%2C+MA%2C+USA%2E)
44. Ziliang Wang, Xuhui Zheng, Kang An, Cijun Ouyang, Jialu Cai, Yuhang Wang, and Yichao Wu. 2025. Stepsearch: Igniting llms search ability via stepwise proximal policy optimization. arXiv preprint arXiv:2505.15107. — [arXiv:2505.15107](https://arxiv.org/abs/2505.15107) [find](https://scholar.google.com/scholar?q=Ziliang+Wang%2C+Xuhui+Zheng%2C+Kang+An%2C+Cijun+Ouyang%2C+Jialu+Cai%2C+Yuhang+Wang%2C+and+Yichao+Wu%2E+2025%2E+Stepsearch%3A+Igniting+llms+search+ability+via+stepwise+proximal+policy+optimization%2E+arXiv+preprint+arXiv%3A)
45. Zeqiu Wu, Yi Luan, Hannah Rashkin, D. Reitter, and Gaurav Singh Tomar. 2021. [Conqrr: Conversational query rewriting for retrieval with reinforcement learning](https://md.research.elamur.ai/free/en/papers/wu-2022-conqrr-conversational-query-rewriting-retrieval/). In Conference on Empirical Methods in Natural Language Processing.
46. Zeqiu Wu, Ryu Parish, Hao Cheng, Sewon Min, Prithviraj Ammanabrolu, Mari Ostendorf, and Hannaneh Hajishirzi. 2023. [Inscit: Information-seeking conversations with mixed-initiative interactions](https://md.research.elamur.ai/free/en/papers/wu-2023-inscit-information-seeking-conversations-mixed/). Transactions of the Association for Computational Linguistics, 11:453–468.
47. Peng Xu, Wei Ping, Xianchao Wu, Chejian Xu, Zihan Liu, Mohammad Shoeybi, and Bryan Catanzaro. 2024. Chatqa 2: Bridging the gap to proprietary llms *Table 9: Prompt used for LLM-as-judge evaluation.* Given a Question and its Golden Answer, verify whether the Predicted Answer is correct. The predictio… [find](https://scholar.google.com/scholar?q=Peng+Xu%2C+Wei+Ping%2C+Xianchao+Wu%2C+Chejian+Xu%2C+Zihan+Liu%2C+Mohammad+Shoeybi%2C+and+Bryan+Catanzaro%2E+2024%2E+Chatqa+2%3A+Bridging+the+gap+to+proprietary+llms+%2ATable+9%3A+Prompt+used+for+LLM%2Das%2Djudge+evaluation%2E%2A+G)
48. Question: {question} Golden Answer: {golden_answer} Predicted Answer: {predicted_answer} [find](https://scholar.google.com/scholar?q=Question%3A+%7Bquestion%7D+Golden+Answer%3A+%7Bgolden%5Fanswer%7D+Predicted+Answer%3A+%7Bpredicted%5Fanswer%7D)

*Updated 31 October 2026*