---
title: "NOVER — paper by Liu et al. (2025) — summary and ideas"
description: "Incentive training rewards a model only on its final answer, which pushes it to produce intermediate… 71 references, cited by 7 papers here."
authors: ["Wei Liu", "Siya Qi", "Xinyu Wang", "Chen Qian", "Yali Du", "Yulan He"]
year: 2025
venue: "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/2025.emnlp-main.378/"
doi: "10.18653/v1/2025.emnlp-main.378"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/liu-2025-nover-incentive-training-language-models/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/liu-2025-nover-incentive-training-language-models/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/11/) / NOVER: Incentive Training for Language Models via Verifier-Free Reinforcement Learning

# NOVER: Incentive Training for Language Models via Verifier-Free Reinforcement Learning

**Authors:** [Wei Liu](https://md.research.elamur.ai/free/en/authors/wei-liu/), Siya Qi, [Xinyu Wang](https://md.research.elamur.ai/free/en/authors/xinyu-wang/), [Chen Qian](https://md.research.elamur.ai/free/en/authors/chen-qian/), [Yali Du](https://md.research.elamur.ai/free/en/authors/yali-du/) and [Yulan He](https://md.research.elamur.ai/free/en/authors/yulan-he/) · **Year:** 2025 · **Venue:** [Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2025](https://md.research.elamur.ai/free/en/venues/emnlp/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.emnlp-main.378/) · [PDF](https://aclanthology.org/2025.emnlp-main.378.pdf) · [DOI 10.18653/v1/2025.emnlp-main.378](https://doi.org/10.18653/v1/2025.emnlp-main.378) · [OpenAlex](https://openalex.org/W4416036525)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [7](https://md.research.elamur.ai/free/en/papers/liu-2025-nover-incentive-training-language-models/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Incentive training rewards a model only on its final answer, which pushes it to produce intermediate reasoning on its own. Approaches like DeepSeek R1-Zero rely on external verifiers, so they work mainly for math and code, where correctness is easy to check. Trained reward or verifier models could fill the gap, but they are expensive to build and to run during RL.

NOVER removes the verifier entirely and needs only ordinary SFT data (prompt plus reference answer). Its reward signal comes from a proxy model that is periodically synchronized with the policy. That proxy scores how likely the ground-truth answer is, given the policy's generated reasoning; this "reasoning perplexity" is length-normalized. Efficiency and format rewards are added, and the policy is optimized with GRPO.

This makes incentive training possible for free-form text-to-text tasks such as open-ended QA, social reasoning, creative writing and translation. NOVER outperforms a same-size model distilled from DeepSeek R1 671B by 7.7%. Its flexibility also enables "inverse incentive training", in which the intermediate tokens themselves become the desired output.

## Key points
- **Reward without decoding:** The reward is the perplexity of the reference answer conditioned on the model's reasoning, divided by a length factor N(|t|) = max(1, 1 + log|t|). Because it is computed with teacher forcing, it costs only about 5% of total training time.
- **Main result:** NOVER beats a model of the same size distilled from DeepSeek R1 671B by 7.7%. The authors also report more stable training than model-as-verifier setups and reduced reward hacking.
- **Base models:** Experiments use Qwen models (e.g., Qwen2.5-7B). The authors argue that free-form reasoning requires a base model that already has some CoT and instruction-following ability.
- **Reasoning patterns:** Gemini-2.0-flash classified reasoning traces into six pattern types. Over training, task decomposition gave way to more direct, efficient reasoning, encouraged by the efficiency reward.
- **Inverse incentive training (NOVER_RUBRIC):** On the SS-GEN story dataset, the story is treated as the intermediate step and rubric-based self-assessment as the answer. This raised Qwen2.5-7B accuracy from 50.79% to 64.37%, and human judges preferred the resulting stories on every rubric. A demo is available at huggingface.co/spaces/thinkwee/NOVER.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Shanghai Jiao Tong University](https://md.research.elamur.ai/free/en/universities/shanghai-jiao-tong-university/) — University, Shanghai, China · 212 works · 19,544 citations
- [The Alan Turing Institute](https://md.research.elamur.ai/free/en/labs/the-alan-turing-institute/) — Research institute, London, United Kingdom · 31 works · 4,535 citations
- [King's College London](https://md.research.elamur.ai/free/en/universities/king-s-college-london/) — University, London, United Kingdom · 46 works · 1,714 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,685 works

## Cite

Cite as: Wei Liu, Siya Qi, Xinyu Wang, Chen Qian, Yali Du and Yulan He. 2025. NOVER: Incentive Training for Language Models via Verifier-Free Reinforcement Learning. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{liu2025nover,
  title = {{NOVER: Incentive Training for Language Models via Verifier-Free Reinforcement Learning}},
  author = {Wei Liu, Siya Qi, Xinyu Wang, Chen Qian, Yali Du and Yulan He},
  booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  doi = {10.18653/v1/2025.emnlp-main.378},
  url = {https://doi.org/10.18653/v1/2025.emnlp-main.378}
}
```

- [← previous NorEval: A Norwegian Language Understanding and Generation Evaluation Benchmark](https://md.research.elamur.ai/free/en/papers/mikhailov-2025-noreval-norwegian-language-understanding-generation/)
- [next → NYUAD at MAHED Shared Task: Detecting Hope, Hate, and Emotion in Arabic Textual Speech and Multi-modal Memes Using Large Language Models](https://md.research.elamur.ai/free/en/papers/aldahoul-2025-nyuad-mahed-shared-task-detecting/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/11/)

## References

71 references, 39 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Dario Amodei, Chris Olah, Jacob Steinhardt, Paul F. Christiano, John Schulman, and Dan Mané. 2016. [Concrete Problems in AI Safety](https://md.research.elamur.ai/free/en/papers/amodei-2016-concrete-problems-ai-safety/). CoRR, abs/1606.06565.
2. Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M. Dai, Anja Hauth, Katie Millican, David Silver, Slav Petrov, Melvin Johnson, Ioannis Antonoglou, Julian Schrittwieser, Amelia Glaese, Jilin Chen, Emily Pitler, Timothy P. Lillicrap,… — [Gemini: A Family of Highly Capable Multimodal Models](https://md.research.elamur.ai/free/en/papers/gemini-team-2023-gemini-family-highly-capable-multimodal/)
3. Yi Feng, Mingyang Song, Jiaqi Wang, Zhuang Chen, Guanqun Bi, Minlie Huang, Liping Jing, and Jian Yu. 2025c. SS-GEN: A Social Story Generation Framework with Large Language Models. In AAAI-25, Sponsored by the Association for the Advancement of Artificial Intelligence, February 25 - March 4, 2025, Ph… [find](https://scholar.google.com/scholar?q=Yi+Feng%2C+Mingyang+Song%2C+Jiaqi+Wang%2C+Zhuang+Chen%2C+Guanqun+Bi%2C+Minlie+Huang%2C+Liping+Jing%2C+and+Jian+Yu%2E+2025c%2E+SS%2DGEN%3A+A+Social+Story+Generation+Framework+with+Large+Language+Models%2E+In+AAAI%2D25%2C+Sponsore)
4. Tom Foster, Eshaan Ahmed, James Cook, Shaked Lifshitz, Tim Rocktäschel, and Jakob N. Foerster. 2024. On Reward Functions For Self-improving Chain-of-thought Reasoning Without Supervised Datasets (Abridged Version). In Language Gamification -NeurIPS 2024 Workshop. [find](https://scholar.google.com/scholar?q=Tom+Foster%2C+Eshaan+Ahmed%2C+James+Cook%2C+Shaked+Lifshitz%2C+Tim+Rockt%C3%A4schel%2C+and+Jakob+N%2E+Foerster%2E+2024%2E+On+Reward+Functions+For+Self%2Dimproving+Chain%2Dof%2Dthought+Reasoning+Without+Supervised+Datasets+%28Abri)
5. Zhuang Chen, Jincenzi Wu, Jinfeng Zhou, Bosi Wen, Guanqun Bi, Gongyao Jiang, Yaru Cao, Mengting Hu, Yunghwei Lai, Zexuan Xiong, and Minlie Huang. 2024. [ToMBench: Benchmarking Theory of Mind in Large Language Models](https://md.research.elamur.ai/free/en/papers/chen-2024-tombench-benchmarking-theory-mind-large/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Lingu…
6. Gemini, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, and 1 others. 2024. [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://md.research.elamur.ai/free/en/papers/gemini-team-2024-gemini-1-5-unlocking-multimodal/). arXiv preprint arXiv:2403.05530.
7. Paul F. Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep Reinforcement Learning from Human Preferences. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach… [find](https://scholar.google.com/scholar?q=Paul+F%2E+Christiano%2C+Jan+Leike%2C+Tom+B%2E+Brown%2C+Miljan+Martic%2C+Shane+Legg%2C+and+Dario+Amodei%2E+2017%2E+Deep+Reinforcement+Learning+from+Human+Preferences%2E+In+Advances+in+Neural+Information+Processing+Systems)
8. Alexey Gorbatovski, Boris Shaposhnikov, Alexey Malakhov, Nikita Surnachev, Yaroslav Aksenov, Ian Maksimov, Nikita Balagansky, and Daniil Gavrilov. 2025. Learn Your Reference Model for Real Good Alignment. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, A… [find](https://scholar.google.com/scholar?q=Alexey+Gorbatovski%2C+Boris+Shaposhnikov%2C+Alexey+Malakhov%2C+Nikita+Surnachev%2C+Yaroslav+Aksenov%2C+Ian+Maksimov%2C+Nikita+Balagansky%2C+and+Daniil+Gavrilov%2E+2025%2E+Learn+Your+Reference+Model+for+Real+Good+Alignm)
9. Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V. Le, Sergey Levine, and Yi Ma. 2025. [SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training](https://md.research.elamur.ai/free/en/papers/chu-2025-sft-memorizes-rl-generalizes-comparative/). CoRR, abs/2501.17161.
10. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, and 80 others. 2025. [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs…](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/)
11. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. [Training Verifiers to Solve Math Word Problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). CoRR, abs/2110.14168.
12. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv e-prints, pages arXiv–2407.
13. Jiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang, Yujia Qin, Baoquan Zhong, Chengquan Jiang, Jinxin Chi, and Wanjun Zhong. 2025a. [Retool: Reinforcement learning for strategic tool use in llms](https://md.research.elamur.ai/free/en/papers/feng-2025-retool-reinforcement-learning-strategic-tool/). arXiv preprint arXiv:2504.11536.
14. Kaituo Feng, Kaixiong Gong, Bohao Li, Zonghao Guo, Yibing Wang, Tianshuo Peng, Benyou Wang, and Xiangyu Yue. 2025b. Video-R1: Reinforcing Video Reasoning in MLLMs. CoRR, abs/2503.21776. — [arXiv:2503.21776](https://arxiv.org/abs/2503.21776) [find](https://scholar.google.com/scholar?q=Kaituo+Feng%2C+Kaixiong+Gong%2C+Bohao+Li%2C+Zonghao+Guo%2C+Yibing+Wang%2C+Tianshuo+Peng%2C+Benyou+Wang%2C+and+Xiangyu+Yue%2E+2025b%2E+Video%2DR1%3A+Reinforcing+Video+Reasoning+in+MLLMs%2E+CoRR%2C+abs%2F2503%2E21776%2E+%E2%80%94+%5BarXiv%3A2503%2E)
15. General Reasoning. 2025. GeneralThought-430K: Open Reasoning Dataset. [https://huggingface.co/datasets/GeneralReasoning/](https://huggingface.co/datasets/GeneralReasoning/) GeneralThought-430K. [↗](https://huggingface.co/datasets/GeneralReasoning/)
16. Aashish Gurung and Mirella Lapata. 2025. Learning to Reason for Long-form Story Generation. arXiv preprint arXiv:2503.22828. — [arXiv:2503.22828](https://arxiv.org/abs/2503.22828) [find](https://scholar.google.com/scholar?q=Aashish+Gurung+and+Mirella+Lapata%2E+2025%2E+Learning+to+Reason+for+Long%2Dform+Story+Generation%2E+arXiv+preprint+arXiv%3A2503%2E22828%2E+%E2%80%94+%5BarXiv%3A2503%2E22828%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2503%2E22828%29)
17. Minggui He, Yilun Liu, Shimin Tao, Yuanchang Luo, Hongyong Zeng, Chang Su, Li Zhang, Hongxia Ma, Daimeng Wei, Weibin Meng, Hao Yang, Boxing Chen, and Osamu Yoshie. 2025. [R1-T1: Fully Incentivizing Translation Capability in LLMs via Reasoning Learning](https://md.research.elamur.ai/free/en/papers/he-2026-r1-t1-fully-incentivizing-translation/). CoRR, abs/2502.19735.
18. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. [LoRA: Low-rank Adaptation of Large Language Models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net.
19. Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, Xiangyu Zhang, and Heung-Yeung Shum. 2025. [Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model](https://md.research.elamur.ai/free/en/papers/hu-2025-open-reasoner-zero-open-source/). CoRR, abs/2503.24290.
20. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2025. [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://md.research.elamur.ai/free/en/papers/huang-2023-survey-hallucination-large-language-models/). ACM Trans. Inf. Syst., 43(2):42:1–42:55.
21. Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Allison Tam, Ally Bennett, Ananya Kumar, and 80 others. 20… — [OpenAI o1 System Card](https://md.research.elamur.ai/free/en/papers/openai-2024-openai-o1-system-card/)
22. AQ Jiang, A Sablayrolles, A Mensch, C Bamford, DS Chaplot, Ddl Casas, F Bressand, G Lengyel, G Lample, L Saulnier, and 1 others. 2024. Mistral 7b. arXiv preprint arXiv:2310.06825. [find](https://scholar.google.com/scholar?q=AQ+Jiang%2C+A+Sablayrolles%2C+A+Mensch%2C+C+Bamford%2C+DS+Chaplot%2C+Ddl+Casas%2C+F+Bressand%2C+G+Lengyel%2C+G+Lample%2C+L+Saulnier%2C+and+1+others%2E+2024%2E+Mistral+7b%2E+arXiv+preprint+arXiv%3A2310%2E06825%2E)
23. Bowen Jin, Hansi Zeng, Zhenrui Yue, Dong Wang, Hamed Zamani, and Jiawei Han. 2025. [Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/jin-2025-search-r1-training-llms-reason/). CoRR, abs/2503.09516.
24. Hyunwoo Kim, Melanie Sclar, Xuhui Zhou, Ronan Le Bras, Gunhee Kim, Yejin Choi, and Maarten Sap. 2023. FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactions. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, Decem… [find](https://scholar.google.com/scholar?q=Hyunwoo+Kim%2C+Melanie+Sclar%2C+Xuhui+Zhou%2C+Ronan+Le+Bras%2C+Gunhee+Kim%2C+Yejin+Choi%2C+and+Maarten+Sap%2E+2023%2E+FANToM%3A+A+Benchmark+for+Stress%2Dtesting+Machine+Theory+of+Mind+in+Interactions%2E+In+Proceedings+of+t)
25. Levente Kocsis and Csaba Szepesvári. 2006. [find](https://scholar.google.com/scholar?q=Levente+Kocsis+and+Csaba+Szepesv%C3%A1ri%2E+2006%2E)
26. Bandit Based Monte-carlo Planning. In Machine Learning: ECML 2006, 17th European Conference on Machine Learning, Berlin, Germany, September 18-22, 2006, Proceedings, volume 4212 of Lecture Notes in Computer Science, pages 282–293. Springer. [find](https://scholar.google.com/scholar?q=Bandit+Based+Monte%2Dcarlo+Planning%2E+In+Machine+Learning%3A+ECML+2006%2C+17th+European+Conference+on+Machine+Learning%2C+Berlin%2C+Germany%2C+September+18%2D22%2C+2006%2C+Proceedings%2C+volume+4212+of+Lecture+Notes+in+Co)
27. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. [Large Language Models are Zero-shot Reasoners](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/). In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, Novemb…
28. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. [Efficient memory management for large language model serving with pagedattention](https://md.research.elamur.ai/free/en/papers/kwon-2023-efficient-memory-management-large-language/). In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles.
29. Bangzheng Li, Ben Zhou, Fei Wang, Xingyu Fu, Dan Roth, and Muhao Chen. 2024. Deceptive Semantic Shortcuts on Reasoning Chains: How Far Can Models Go without Hallucination? In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Lang… [find](https://scholar.google.com/scholar?q=Bangzheng+Li%2C+Ben+Zhou%2C+Fei+Wang%2C+Xingyu+Fu%2C+Dan+Roth%2C+and+Muhao+Chen%2E+2024%2E+Deceptive+Semantic+Shortcuts+on+Reasoning+Chains%3A+How+Far+Can+Models+Go+without+Hallucination%3F+In+Proceedings+of+the+2024+C)
30. Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2024. [Let’s Verify Step by Step](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/). In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenRevi…
31. Wei Liu, Chenxi Wang, YiFei Wang, Zihao Xie, Rennai Qiu, Yufan Dang, Zhuoyun Du, Weize Chen, Cheng Yang, and Chen Qian. 2024. Autonomous agents for collaborative task under information asymmetry. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Wei+Liu%2C+Chenxi+Wang%2C+YiFei+Wang%2C+Zihao+Xie%2C+Rennai+Qiu%2C+Yufan+Dang%2C+Zhuoyun+Du%2C+Weize+Chen%2C+Cheng+Yang%2C+and+Chen+Qian%2E+2024%2E+Autonomous+agents+for+collaborative+task+under+information+asymmetry%2E+In+T)
32. Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. 2025a. [Understanding R1-Zero-like Training: A Critical Perspective](https://md.research.elamur.ai/free/en/papers/liu-2025-understanding-r1-zero-like-training/). CoRR, abs/2503.20783.
33. Zijun Liu, Peiyi Wang, Runxin Xu, Shirong Ma, Chong Ruan, Peng Li, Yang Liu, and Yu Wu. 2025b. [Inference-time scaling for generalist reward modeling](https://md.research.elamur.ai/free/en/papers/liu-2025-inference-time-scaling-generalist/). arXiv preprint arXiv:2504.02495.
34. Yilun Lu, Cheng Zhang, Jiaming Song, Linxi Fan, and William Wang. 2025. ToM-RL: Reinforcement Learning Unlocks Theory of Mind in Small LLMs. arXiv preprint arXiv:2504.01698. — [arXiv:2504.01698](https://arxiv.org/abs/2504.01698) [find](https://scholar.google.com/scholar?q=Yilun+Lu%2C+Cheng+Zhang%2C+Jiaming+Song%2C+Linxi+Fan%2C+and+William+Wang%2E+2025%2E+ToM%2DRL%3A+Reinforcement+Learning+Unlocks+Theory+of+Mind+in+Small+LLMs%2E+arXiv+preprint+arXiv%3A2504%2E01698%2E+%E2%80%94+%5BarXiv%3A2504%2E01698%5D%28https)
35. Xueguang Ma, Qian Liu, Dongfu Jiang, Ge Zhang, Zejun Ma, and Wenhu Chen. 2025. [General-Reasoner: Advancing LLM Reasoning Across All Domains](https://md.research.elamur.ai/free/en/papers/ma-2025-general-reasoner-advancing-llm-reasoning/). [https://github.com/TIGER-AI-Lab/](https://github.com/TIGER-AI-Lab/) General-Reasoner/blob/main/General_ Reasoner.pdf. [↗](https://github.com/TIGER-AI-Lab/)
36. Andrew Y Ng and Stuart J Russell. 2000. Algorithms for Inverse Reinforcement Learning. In Proceedings of the Seventeenth International Conference on Machine Learning, pages 663–670. [find](https://scholar.google.com/scholar?q=Andrew+Y+Ng+and+Stuart+J+Russell%2E+2000%2E+Algorithms+for+Inverse+Reinforcement+Learning%2E+In+Proceedings+of+the+Seventeenth+International+Conference+on+Machine+Learning%2C+pages+663%E2%80%93670%2E)
37. Liangming Pan, Alon Albalak, Xinyi Wang, and William Yang Wang. 2023. [Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning](https://md.research.elamur.ai/free/en/papers/pan-2023-logic-lm-empowering-large-language/). In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, December 6-10, 2023, pages 3806– 3824. Associati…
38. Siya Qi, Yulan He, and Zheng Yuan. 2024. Can we catch the elephant? the evolvement of hallucination evaluation on natural language generation: A survey. arXiv e-prints, pages arXiv–2404. [find](https://scholar.google.com/scholar?q=Siya+Qi%2C+Yulan+He%2C+and+Zheng+Yuan%2E+2024%2E+Can+we+catch+the+elephant%3F+the+evolvement+of+hallucination+evaluation+on+natural+language+generation%3A+A+survey%2E+arXiv+e%2Dprints%2C+pages+arXiv%E2%80%932404%2E)
39. Sahand Sabour, Siyang Liu, Zheyuan Zhang, June Liu, Jinfeng Zhou, Alvionna Sunaryo, Tatia Lee, Rada Mihalcea, and Minlie Huang. 2024. [EmoBench: Evaluating the Emotional Intelligence of Large Language Models](https://md.research.elamur.ai/free/en/papers/sabour-2024-emobench-evaluating-emotional-intelligence-large/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (…
40. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal Policy Optimization Algorithms. CoRR, abs/1707.06347. [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Filip+Wolski%2C+Prafulla+Dhariwal%2C+Alec+Radford%2C+and+Oleg+Klimov%2E+2017%2E+Proximal+Policy+Optimization+Algorithms%2E+CoRR%2C+abs%2F1707%2E06347%2E)
41. ByteDance Seed, Yufeng Yuan, Yu Yue, Mingxuan Wang, Xiaochen Zuo, Jiaze Chen, Lin Yan, Wenyuan Xu, Chi Zhang, Xin Liu, and 1 others. 2025. [Seed-thinking-v1](https://md.research.elamur.ai/free/en/papers/bytedance-seed-2025-seed1-5-thinking-advancing/). 5: Advancing superb reasoning models with reinforcement learning. arXiv preprint arXiv:2504.13914.
42. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). CoRR, abs/2402.03300.
43. Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, and 1 others. 2025. [Vlm-r1: A stable and generalizable r1-style large vision-language model](https://md.research.elamur.ai/free/en/papers/shen-2025-vlm-r1-stable-generalizable-r1/). arXiv preprint arXiv:2504.07615.
44. Huatong Song, Jinhao Jiang, Yingqian Min, Jie Chen, Zhipeng Chen, Wayne Xin Zhao, Lei Fang, and Ji-Rong Wen. 2025. [R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/song-2025-r1-searcher-incentivizing-search/). CoRR, abs/2503.05592.
45. Yi Su, Dian Yu, Linfeng Song, Juntao Li, Haitao Mi, Zhaopeng Tu, Min Zhang, and Dong Yu. 2025. Crossing the Reward Bridge: Expanding RL with Verifiable Rewards Across Diverse Domains. CoRR, abs/2503.23829. — [arXiv:2503.23829](https://arxiv.org/abs/2503.23829) [find](https://scholar.google.com/scholar?q=Yi+Su%2C+Dian+Yu%2C+Linfeng+Song%2C+Juntao+Li%2C+Haitao+Mi%2C+Zhaopeng+Tu%2C+Min+Zhang%2C+and+Dong+Yu%2E+2025%2E+Crossing+the+Reward+Bridge%3A+Expanding+RL+with+Verifiable+Rewards+Across+Diverse+Domains%2E+CoRR%2C+abs%2F2503%2E2)
46. TIGER-Lab. 2025. WebInstruct-verified Dataset. [https://huggingface.co/datasets/](https://huggingface.co/datasets/) TIGER-Lab/WebInstruct-verified. [↗](https://huggingface.co/datasets/)
47. Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beeching, Tristan Thrush, Nathan Lambert, Shengyi Huang, Kashif Rasul, and Quentin Gallouédec. 2020. TRL: Transformer Reinforcement Learning. [https://github.com/huggingface/](https://github.com/huggingface/) trl. [↗](https://github.com/huggingface/)
48. Lei Wang, Wanyu Xu, Yihuai Lan, Zhiqiang Hu, Yunshi Lan, Roy Ka-Wei Lee, and Ee-Peng Lim. 2023a. Plan-and-solve Prompting: Improving Zero-shot Chain-of-thought Reasoning by Large Language Models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Lo… [find](https://scholar.google.com/scholar?q=Lei+Wang%2C+Wanyu+Xu%2C+Yihuai+Lan%2C+Zhiqiang+Hu%2C+Yunshi+Lan%2C+Roy+Ka%2DWei+Lee%2C+and+Ee%2DPeng+Lim%2E+2023a%2E+Plan%2Dand%2Dsolve+Prompting%3A+Improving+Zero%2Dshot+Chain%2Dof%2Dthought+Reasoning+by+Large+Language+Models%2E+In+P)
49. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023b. [Self-consistency Improves Chain of Thought Reasoning in Language Models](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/). In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5…
50. Zihan Wang, Kangrui Wang, Qineng Wang, Pingyue Zhang, Linjie Li, Zhengyuan Yang, Kefan Yu, Minh Nhat Nguyen, Licheng Liu, Eli Gottlieb, and 1 others. 2025. [Ragen: Understanding self-evolution in llm agents via multi-turn reinforcement learning](https://md.research.elamur.ai/free/en/papers/wang-2025-ragen-understanding-self-evolution-llm/). arXiv preprint arXiv:2504.20073.
51. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. [Chain-of-thought Prompting Elicits Reasoning in Large Language Models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processi…
52. Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Hanjie Chen, and Xia Ben Hu. 2025. [Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models](https://md.research.elamur.ai/free/en/papers/sui-2025-stop-overthinking-survey-efficient-reasoning/). CoRR, abs/2503.16419.
53. Lilian Weng. 2025. weng.github.io. [find](https://scholar.google.com/scholar?q=Lilian+Weng%2E+2025%2E+weng%2Egithub%2Eio%2E)
54. M.-A-P. Team, Xinrun Du, Yifan Yao, Kaijing Ma, Bingli Wang, Tianyu Zheng, Kang Zhu, Minghao Liu, Yiming Liang, Xiaolong Jin, Zhenlin Wei, Chujie Zheng, Kaixin Deng, Shian Jia, Sichao Jiang, Yiyan Liao, Rui Li, Qinrui Li, Sirun Li, and 77 others. 2025. [SuperGPQA: Scaling LLM Evaluation across 285 Gr…](https://md.research.elamur.ai/free/en/papers/team-2025-supergpqa-scaling-llm-evaluation-across/)
55. Tian Xie, Zitian Gao, Qingnan Ren, Haoming Luo, Yuqian Hong, Bryan Dai, Joey Zhou, Kai Qiu, Zhirong Wu, and Chong Luo. 2025. [Logic-RL: Unleashing LLM Reasoning with Rule-based Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/xie-2025-logic-rl-unleashing-llm/). CoRR, abs/2502.14768.
56. Jörg Tiedemann. 2012. Parallel Data, Tools and Interfaces in OPUS. In Proceedings of the Eighth International Conference on Language Resources and Evaluation, LREC 2012, Istanbul, Turkey, May 23-25, 2012, pages 2214–2218. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=J%C3%B6rg+Tiedemann%2E+2012%2E+Parallel+Data%2C+Tools+and+Interfaces+in+OPUS%2E+In+Proceedings+of+the+Eighth+International+Conference+on+Language+Resources+and+Evaluation%2C+LREC+2012%2C+Istanbul%2C+Turkey%2C+May+23%2D25%2C+2)
57. Why we think. lilian-Xiaobo Xia and Run Luo. 2025. Gui-r1: A generalist r1-style vision-language action model for gui agents. arXiv preprint arXiv:2504.10458. — [arXiv:2504.10458](https://arxiv.org/abs/2504.10458) [find](https://scholar.google.com/scholar?q=Why+we+think%2E+lilian%2DXiaobo+Xia+and+Run+Luo%2E+2025%2E+Gui%2Dr1%3A+A+generalist+r1%2Dstyle+vision%2Dlanguage+action+model+for+gui+agents%2E+arXiv+preprint+arXiv%3A2504%2E10458%2E+%E2%80%94+%5BarXiv%3A2504%2E10458%5D%28https%3A%2F%2Farxiv%2Eorg%2Fab)
58. Hanqi Yan, Qinglin Zhu, Xinyu Wang, Lin Gui, and Yulan He. 2024. Mirror: Multiple-perspective Self-reflection Method for Knowledge-rich Reasoning. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Thailand, August 1… [find](https://scholar.google.com/scholar?q=Hanqi+Yan%2C+Qinglin+Zhu%2C+Xinyu+Wang%2C+Lin+Gui%2C+and+Yulan+He%2E+2024%2E+Mirror%3A+Multiple%2Dperspective+Self%2Dreflection+Method+for+Knowledge%2Drich+Reasoning%2E+In+Proceedings+of+the+62nd+Annual+Meeting+of+the+Asso)
59. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, and 22 others. 2024. Qwen2.5 Technical Report. CoRR, abs/2412.1511… [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Haoran+Wei%2C+Huan+Lin%2C+Jian+Yang%2C+Jianhong+Tu%2C+Jianwei+Zhang%2C+Jianxin+Yang%2C+Jiaxi+Yang%2C+Jin)
60. Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, Bo Zhang, and Wei Chen. 2025. R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-modal Formalization. CoRR, abs/2503.10615. [find](https://scholar.google.com/scholar?q=Yi+Yang%2C+Xiaoxuan+He%2C+Hongkun+Pan%2C+Xiyan+Jiang%2C+Yan+Deng%2C+Xingtao+Yang%2C+Haoyu+Lu%2C+Dacheng+Yin%2C+Fengyun+Rao%2C+Minfeng+Zhu%2C+Bo+Zhang%2C+and+Wei+Chen%2E+2025%2E+R1%2DOnevision%3A+Advancing+Generalized+Multimodal+Re)
61. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023a. Tree of Thoughts: Deliberate Problem Solving with Large Language Models. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023,… [find](https://scholar.google.com/scholar?q=Shunyu+Yao%2C+Dian+Yu%2C+Jeffrey+Zhao%2C+Izhak+Shafran%2C+Tom+Griffiths%2C+Yuan+Cao%2C+and+Karthik+Narasimhan%2E+2023a%2E+Tree+of+Thoughts%3A+Deliberate+Problem+Solving+with+Large+Language+Models%2E+In+Advances+in+Neural)
62. Capacity in LLMs Beyond the Base Model? arXiv preprint arXiv:2504.13837. — [Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?](https://md.research.elamur.ai/free/en/papers/yue-2025-reinforcement-learning-really-incentivize-reasoning/)
63. Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. 2025. [SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild](https://md.research.elamur.ai/free/en/papers/zeng-2025-simplerl-zoo-investigating-taming-zero/). CoRR, abs/2503.18892.
64. Jingyi Zhang, Jiaxing Huang, Huanjin Yao, Shunyu Liu, Xikun Zhang, Shijian Lu, and Dacheng Tao. 2025. R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimization. CoRR, abs/2503.12937. — [arXiv:2503.12937](https://arxiv.org/abs/2503.12937) [find](https://scholar.google.com/scholar?q=Jingyi+Zhang%2C+Jiaxing+Huang%2C+Huanjin+Yao%2C+Shunyu+Liu%2C+Xikun+Zhang%2C+Shijian+Lu%2C+and+Dacheng+Tao%2E+2025%2E+R1%2DVL%3A+Learning+to+Reason+with+Multimodal+Large+Language+Models+via+Step%2Dwise+Group+Relative+Polic)
65. Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc V. Le, and Ed H. Chi. 2023. [Least-to-most Prompting Enables Complex Reasoning in Large Language Models](https://md.research.elamur.ai/free/en/papers/al-2022-least-most-prompting-enables-complex/). In The Eleventh International Conference on Learning Representatio…
66. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R. Narasimhan, and Yuan Cao. 2023b. [ReAct: Synergizing Reasoning and Acting in Language Models](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/). In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net.
67. Hongbin Ye, Tong Liu, Aijia Zhang, Wei Hua, and Weiqiang Jia. 2024. Cognitive Mirage: A Review of Hallucinations in Large Language Models. In Proceedings of the First International OpenKG Workshop: Large Knowledge-Enhanced Models co-locacted with The International Joint Conference on Artificial Inte… [find](https://scholar.google.com/scholar?q=Hongbin+Ye%2C+Tong+Liu%2C+Aijia+Zhang%2C+Wei+Hua%2C+and+Weiqiang+Jia%2E+2024%2E+Cognitive+Mirage%3A+A+Review+of+Hallucinations+in+Large+Language+Models%2E+In+Proceedings+of+the+First+International+OpenKG+Workshop%3A+La)
68. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, and 16 others. 2025. [DAPO: An Open-source LLM Reinforcement Learning Sys…](https://md.research.elamur.ai/free/en/papers/yu-2025-dapo-open-source-llm-reinforcement/)
69. Weizhe Yuan, Jane Yu, Song Jiang, Karthik Padthe, Yang Li, Dong Wang, Ilia Kulikov, Kyunghyun Cho, Yuandong Tian, Jason E Weston, and 1 others. 2025a. Naturalreasoning: Reasoning in the wild with 2.8 m challenging questions. arXiv preprint arXiv:2502.13124. [find](https://scholar.google.com/scholar?q=Weizhe+Yuan%2C+Jane+Yu%2C+Song+Jiang%2C+Karthik+Padthe%2C+Yang+Li%2C+Dong+Wang%2C+Ilia+Kulikov%2C+Kyunghyun+Cho%2C+Yuandong+Tian%2C+Jason+E+Weston%2C+and+1+others%2E+2025a%2E+Naturalreasoning%3A+Reasoning+in+the+wild+with+2%2E8+)
70. Yufeng Yuan, Qiying Yu, Xiaochen Zuo, Ruofei Zhu, Wenyuan Xu, Jiaze Chen, Chengyi Wang, TianTian Fan, Zhengyin Du, Xiangpeng Wei, and 1 others. 2025b. [VAPO: Efficient and reliable reinforcement learning for advanced reasoning tasks](https://md.research.elamur.ai/free/en/papers/yue-2025-vapo-efficient-reliable-reinforcement-learning/). arXiv preprint arXiv:2504.05118.
71. Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Shiji Song, and Gao Huang. 2025. Does Reinforcement Learning Really Incentivize Reasoning [find](https://scholar.google.com/scholar?q=Yang+Yue%2C+Zhiqi+Chen%2C+Rui+Lu%2C+Andrew+Zhao%2C+Zhaokai+Wang%2C+Shiji+Song%2C+and+Gao+Huang%2E+2025%2E+Does+Reinforcement+Learning+Really+Incentivize+Reasoning)

*Updated 2 November 2026*