---
title: "Incentivizing In-depth Reasoning over Long Contexts with"
description: "Reinforcement learning with verifiable rewards (RLVR) boosts LLM reasoning on short inputs but… 45 references, cited by 2 papers here."
authors: ["Miao Peng", "Weizhou Shen", "Nuo Chen", "Chenliang Li", "Ming Yan", "Jia Li"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.1306/"
doi: "10.18653/v1/2026.findings-acl.1306"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/peng-2026-incentivizing-depth-reasoning-over-long/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/peng-2026-incentivizing-depth-reasoning-over-long/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/5/) / Incentivizing In-depth Reasoning over Long Contexts with Process Advantage Shaping

# Incentivizing In-depth Reasoning over Long Contexts with Process Advantage Shaping

**Authors:** [Miao Peng](https://md.research.elamur.ai/free/en/authors/miao-peng/), [Weizhou Shen](https://md.research.elamur.ai/free/en/authors/weizhou-shen/), [Nuo Chen](https://md.research.elamur.ai/free/en/authors/nuo-chen/), [Chenliang Li](https://md.research.elamur.ai/free/en/authors/chenliang-li/), [Ming Yan](https://md.research.elamur.ai/free/en/authors/ming-yan/) and [Jia Li](https://md.research.elamur.ai/free/en/authors/jia-li/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.1306/) · [PDF](https://aclanthology.org/2026.findings-acl.1306.pdf) · [DOI 10.18653/v1/2026.findings-acl.1306](https://doi.org/10.18653/v1/2026.findings-acl.1306) · [OpenAlex](https://openalex.org/W7166870774)

**Citations:** 1 citations (OpenAlex, 2026-10-14) · [2](https://md.research.elamur.ai/free/en/papers/peng-2026-incentivizing-depth-reasoning-over-long/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Reinforcement learning with verifiable rewards (RLVR) boosts LLM reasoning on short inputs but works poorly on long documents. The authors identify "almost-there" rollouts as a key cause. These trajectories get most intermediate steps right but end with a wrong answer, and outcome-only rewards penalize them in full. Two factors make this worse: long-context QA data rarely forces dense multi-step reasoning, and sparse rewards cannot tell which step actually failed.

The paper makes two contributions:
- **DeepReasonQA**: a dataset synthesized by building knowledge graphs from documents, sampling subgraphs, and turning reasoning paths into obfuscated, hard multi-hop questions. Each question comes with an explicit reference reasoning chain.
- **LongPAS**: a method that scores each step of a policy rollout against that chain for Validity and Relevance. These scores reweight GRPO advantages at the token level, so useful parts of failed rollouts still contribute learning signal.

On three long-context reasoning benchmarks, the approach beats RLVR baselines and performs on par with much larger frontier LLMs. Training also stays stable. This suggests that process-level credit assignment is a practical route to deeper reasoning over long documents.

## Key points
- **Diagnostic study:** On FRAMES with Qwen3-4B, ground-truth chains were annotated using Gemini-2.5-Pro. Failed rollouts still covered many correct entities and reasoning triplets.
  - Entity coverage was consistently higher than triplet coverage, indicating good grounding but weak logical chaining.
  - Coverage peaked for questions solved about half the time.
- **Data synthesis:** The DeepReasonQA pipeline builds a KG from Wikipedia documents. It produces questions covering multi-hop, temporal, causal, and hypothetical reasoning.
- **Method:** LongPAS builds on GRPO. It adds step-level Validity/Relevance signals as token-level advantage reweighting coefficients. Rewards combine rule-based checks with an LLM-as-a-judge.
- **Training dynamics (vs. GRPO/DAPO):**
  - LongPAS keeps response length under control, whereas DAPO's length grows unchecked until generation degenerates.
  - Its reward curve eventually overtakes GRPO's.
  - Triplet coverage on training data stabilizes at about 0.37–0.38, clearly above GRPO.
- **Availability:** Code is released at github.com/GKNL/LongPAS. Stated limitations are reliance on Wikipedia as the sole source and dependence on reference reasoning chains.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Alibaba Group](https://md.research.elamur.ai/free/en/companies/alibaba-group/) — Company, Hangzhou, China · 168 works · 15,386 citations
- [Alibaba Group (Cayman Islands)](https://md.research.elamur.ai/free/en/companies/alibaba-group-cayman-islands/) — Company, George Town, Cayman Islands · 32 works · 3,389 citations
- [The Hong Kong University of Science and Technology (Guangzhou)](https://md.research.elamur.ai/free/en/universities/the-hong-kong-university-of-science-and-technology-guangzhou/) — University, Guangzhou, China · 53 works · 1,367 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,426 works

## Cite

Cite as: Miao Peng, Weizhou Shen, Nuo Chen, Chenliang Li, Ming Yan and Jia Li. 2026. Incentivizing In-depth Reasoning over Long Contexts with Process Advantage Shaping. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{peng2026incentivizing,
  title = {{Incentivizing In-depth Reasoning over Long Contexts with Process Advantage Shaping}},
  author = {Miao Peng, Weizhou Shen, Nuo Chen, Chenliang Li, Ming Yan and Jia Li},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1306},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1306}
}
```

- [← previous Imbalanced Gradients in RL Post-Training of Multi-Task LLMs](https://md.research.elamur.ai/free/en/papers/wu-2026-imbalanced-gradients-rl-post-training/)
- [next → IndicJR: A Judge-Free Benchmark of Jailbreak Robustness in South Asian Languages](https://md.research.elamur.ai/free/en/papers/pattnayak-2026-indicjr-judge-free-benchmark-jailbreak/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/5/)

## References

45 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, and 1 others. 2025. [gpt-oss-120b & gpt-oss-20b model card](https://md.research.elamur.ai/free/en/papers/agarwal-2025-gpt-oss-120b-gpt-oss/). arXiv preprint arXiv:2508.10925.
2. Sanghwan Bae, Jiwoo Hong, Min Young Lee, Hanbyul Kim, JeongYeon Nam, and Donghyun Kwak. 2025. [Online difficulty filtering for reasoning oriented reinforcement learning](https://md.research.elamur.ai/free/en/papers/bae-2026-online-difficulty-filtering-reasoning-oriented/). CoRR, abs/2504.03380.
3. Yushi Bai, Xin Lv, Jiajie Zhang, Yuze He, Ji Qi, Lei Hou, Jie Tang, Yuxiao Dong, and Juanzi Li. 2024a. [Longalign: A recipe for long context alignment of large language models](https://md.research.elamur.ai/free/en/papers/bai-2024-longalign-recipe-long-context/). In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 1376–1395.
4. Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. 2024b. [Longbench: A bilingual, multitask benchmark for long context understanding](https://md.research.elamur.ai/free/en/papers/bai-2023-longbench-bilingual-multitask-benchmark/). In ACL (1), pages 3119–3137. Association for Computationa…
5. Yushi Bai, Shangqing Tu, Jiajie Zhang, Hao Peng, Xiaozhi Wang, Xin Lv, Shulin Cao, Jiazheng Xu, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. 2025. [Longbench v2: Towards deeper understanding and reasoning on realistic long-context multitasks](https://md.research.elamur.ai/free/en/papers/bai-2024-longbench-v2-deeper-understanding/). In ACL (1), pages 3639–3664. Association for Computation…
6. Guanzheng Chen, Xin Li, Michael Shieh, and Lidong Bing. a. Longpo: Long context self-evolution of large language models through short-to-long preference optimization. In The Thirteenth International Conference on Learning Representations. Guanzheng Chen, Michael Qizhe Shieh, and Lidong Bing. b. Long… [find](https://scholar.google.com/scholar?q=Guanzheng+Chen%2C+Xin+Li%2C+Michael+Shieh%2C+and+Lidong+Bing%2E+a%2E+Longpo%3A+Long+context+self%2Devolution+of+large+language+models+through+short%2Dto%2Dlong+preference+optimization%2E+In+The+Thirteenth+International+C)
7. Nuo Chen, Yuhan Li, Jianheng Tang, and Jia Li. 2024. Graphwiz: An instruction-following language model for graph computational problems. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 353–364. [find](https://scholar.google.com/scholar?q=Nuo+Chen%2C+Yuhan+Li%2C+Jianheng+Tang%2C+and+Jia+Li%2E+2024%2E+Graphwiz%3A+An+instruction%2Dfollowing+language+model+for+graph+computational+problems%2E+In+Proceedings+of+the+30th+ACM+SIGKDD+Conference+on+Knowledge+D)
8. Daixuan Cheng, Shaohan Huang, Xuekai Zhu, Bo Dai, Wayne Xin Zhao, Zhenliang Zhang, and Furu Wei. 2025. Reasoning with exploration: An entropy perspective. arXiv preprint arXiv:2506.14758. [find](https://scholar.google.com/scholar?q=Daixuan+Cheng%2C+Shaohan+Huang%2C+Xuekai+Zhu%2C+Bo+Dai%2C+Wayne+Xin+Zhao%2C+Zhenliang+Zhang%2C+and+Furu+Wei%2E+2025%2E+Reasoning+with+exploration%3A+An+entropy+perspective%2E+arXiv+preprint+arXiv%3A2506%2E14758%2E)
9. Deepmind. 2025. Gemini 2.5: Our most intelligent ai model. Technical report, Deepmind. [find](https://scholar.google.com/scholar?q=Deepmind%2E+2025%2E+Gemini+2%2E5%3A+Our+most+intelligent+ai+model%2E+Technical+report%2C+Deepmind%2E)
10. Yihe Deng, I Hsu, Jun Yan, Zifeng Wang, Rujun Han, Gufeng Zhang, Yanfei Chen, Wei Wang, Tomas Pfister, Chen-Yu Lee, and 1 others. 2025. Supervised reinforcement learning: From expert trajectories to step-wise reasoning. arXiv preprint arXiv:2510.25992. — [arXiv:2510.25992](https://arxiv.org/abs/2510.25992) [find](https://scholar.google.com/scholar?q=Yihe+Deng%2C+I+Hsu%2C+Jun+Yan%2C+Zifeng+Wang%2C+Rujun+Han%2C+Gufeng+Zhang%2C+Yanfei+Chen%2C+Wei+Wang%2C+Tomas+Pfister%2C+Chen%2DYu+Lee%2C+and+1+others%2E+2025%2E+Supervised+reinforcement+learning%3A+From+expert+trajectories+to+s)
11. Chaochen Gao, W Xing, Qi Fu, and Songlin Hu. a. Quest: Query-centric data synthesis approach for long-context scaling of large language model. In The Thirteenth International Conference on Learning Representations. Chaochen Gao, W Xing, Zijia Lin, Debing Zhang, and Songlin Hu. b. Nextlong: Toward ef… — [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/)
12. Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. 2020. [Constructing a multi-hop qa dataset for comprehensive evaluation of reasoning steps](https://md.research.elamur.ai/free/en/papers/ho-2020-constructing-multi-hop-qa-dataset/). In Proceedings of the 28th International Conference on Computational Linguistics, pages 6609– 6625.
13. Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, and Boris Ginsburg. Ruler: What’s the real context size of your long-context language models? In First Conference on Language Modeling. — [RULER: What's the Real Context Size of Your Long-Context Language Models?](https://md.research.elamur.ai/free/en/papers/hsieh-2024-ruler-s-real-context-size/)
14. Zihao Jiang, Ben Liu, Miao Peng, Wenjie Xu, Yao Xiao, Zhenyan Shan, and Min Peng. 2025. Towards explainable temporal reasoning in large language models: A structure-aware generative framework. In Findings of the Association for Computational Linguistics: ACL 2025, pages 7232–7251. [find](https://scholar.google.com/scholar?q=Zihao+Jiang%2C+Ben+Liu%2C+Miao+Peng%2C+Wenjie+Xu%2C+Yao+Xiao%2C+Zhenyan+Shan%2C+and+Min+Peng%2E+2025%2E+Towards+explainable+temporal+reasoning+in+large+language+models%3A+A+structure%2Daware+generative+framework%2E+In+Find)
15. Bowen Jin, Hansi Zeng, Zhenrui Yue, Jinsung Yoon, Sercan Arik, Dong Wang, Hamed Zamani, and Jiawei Han. 2025. [Search-r1: Training llms to reason and leverage search engines with reinforcement learning](https://md.research.elamur.ai/free/en/papers/jin-2025-search-r1-training-llms-reason/). arXiv preprint arXiv:2503.09516.
16. Satyapriya Krishna, Kalpesh Krishna, Anhad Mohananey, Steven Schwarcz, Adam Stambler, Shyam Upadhyay, and Manaal Faruqui. 2025. Fact, fetch, and reason: A unified evaluation of retrieval-augmented generation. In NAACL (Long Papers), pages 4745–4759. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Satyapriya+Krishna%2C+Kalpesh+Krishna%2C+Anhad+Mohananey%2C+Steven+Schwarcz%2C+Adam+Stambler%2C+Shyam+Upadhyay%2C+and+Manaal+Faruqui%2E+2025%2E+Fact%2C+fetch%2C+and+reason%3A+A+unified+evaluation+of+retrieval%2Daugmented+gen)
17. Huayang Li, Pat Verga, Priyanka Sen, Bowen Yang, Vijay Viswanathan, Patrick Lewis, Taro Watanabe, and Yixuan Su. 2024a. Alr2: A retrieve-then-reason framework for long-context question answering. arXiv preprint arXiv:2410.03227. — [arXiv:2410.03227](https://arxiv.org/abs/2410.03227) [find](https://scholar.google.com/scholar?q=Huayang+Li%2C+Pat+Verga%2C+Priyanka+Sen%2C+Bowen+Yang%2C+Vijay+Viswanathan%2C+Patrick+Lewis%2C+Taro+Watanabe%2C+and+Yixuan+Su%2E+2024a%2E+Alr2%3A+A+retrieve%2Dthen%2Dreason+framework+for+long%2Dcontext+question+answering%2E+arXi)
18. Jiaxi Li, Xingxing Zhang, Xun Wang, Xiaolong Huang, Li Dong, Liang Wang, Si-Qing Chen, Wei Lu, and Furu Wei. 2025. Wildlong: Synthesizing realistic long-context instruction data at scale. arXiv preprint arXiv:2502.16684. — [arXiv:2502.16684](https://arxiv.org/abs/2502.16684) [find](https://scholar.google.com/scholar?q=Jiaxi+Li%2C+Xingxing+Zhang%2C+Xun+Wang%2C+Xiaolong+Huang%2C+Li+Dong%2C+Liang+Wang%2C+Si%2DQing+Chen%2C+Wei+Lu%2C+and+Furu+Wei%2E+2025%2E+Wildlong%3A+Synthesizing+realistic+long%2Dcontext+instruction+data+at+scale%2E+arXiv+prepri)
19. Siheng Li, Cheng Yang, Zesen Cheng, Lemao Liu, Mo Yu, Yujiu Yang, and Wai Lam. 2024b. Large language models can self-improve in long-context reasoning. arXiv preprint arXiv:2411.08147. — [arXiv:2411.08147](https://arxiv.org/abs/2411.08147) [find](https://scholar.google.com/scholar?q=Siheng+Li%2C+Cheng+Yang%2C+Zesen+Cheng%2C+Lemao+Liu%2C+Mo+Yu%2C+Yujiu+Yang%2C+and+Wai+Lam%2E+2024b%2E+Large+language+models+can+self%2Dimprove+in+long%2Dcontext+reasoning%2E+arXiv+preprint+arXiv%3A2411%2E08147%2E+%E2%80%94+%5BarXiv%3A2411%2E0)
20. Yanyang Li, Shuo Liang, Michael Lyu, and Liwei Wang. 2024c. Making long-context language models better multi-hop reasoners. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2462– 2475. [find](https://scholar.google.com/scholar?q=Yanyang+Li%2C+Shuo+Liang%2C+Michael+Lyu%2C+and+Liwei+Wang%2E+2024c%2E+Making+long%2Dcontext+language+models+better+multi%2Dhop+reasoners%2E+In+Proceedings+of+the+62nd+Annual+Meeting+of+the+Association+for+Computation)
21. Zhan Ling, Kang Liu, Kai Yan, Yifan Yang, Weijian Lin, Ting-Han Fan, Lingfeng Shen, Zhengyin Du, and Jiecao Chen. 2025. Longreason: A synthetic long-context reasoning benchmark via context expansion. arXiv preprint arXiv:2501.15089. — [arXiv:2501.15089](https://arxiv.org/abs/2501.15089) [find](https://scholar.google.com/scholar?q=Zhan+Ling%2C+Kang+Liu%2C+Kai+Yan%2C+Yifan+Yang%2C+Weijian+Lin%2C+Ting%2DHan+Fan%2C+Lingfeng+Shen%2C+Zhengyin+Du%2C+and+Jiecao+Chen%2E+2025%2E+Longreason%3A+A+synthetic+long%2Dcontext+reasoning+benchmark+via+context+expansion%2E+)
22. Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, and 1 others. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437. [find](https://scholar.google.com/scholar?q=Aixin+Liu%2C+Bei+Feng%2C+Bing+Xue%2C+Bingxuan+Wang%2C+Bochao+Wu%2C+Chengda+Lu%2C+Chenggang+Zhao%2C+Chengqi+Deng%2C+Chenyu+Zhang%2C+Chong+Ruan%2C+and+1+others%2E+2024%2E+Deepseek%2Dv3+technical+report%2E+arXiv+preprint+arXiv%3A2412)
23. Belinda Mo, Kyssen Yu, Joshua Kazdan, Joan Cabezas, Proud Mpala, Lisa Yu, Chris Cundy, Charilaos Kanatsoulis, and Sanmi Koyejo. 2025. Kggen: Extracting knowledge graphs from plain text with language models. Preprint, arXiv:2502.09956. — [arXiv:2502.09956](https://arxiv.org/abs/2502.09956) [find](https://scholar.google.com/scholar?q=Belinda+Mo%2C+Kyssen+Yu%2C+Joshua+Kazdan%2C+Joan+Cabezas%2C+Proud+Mpala%2C+Lisa+Yu%2C+Chris+Cundy%2C+Charilaos+Kanatsoulis%2C+and+Sanmi+Koyejo%2E+2025%2E+Kggen%3A+Extracting+knowledge+graphs+from+plain+text+with+language+m)
24. OpenAI. 2025. Introducing gpt-5. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2025%2E+Introducing+gpt%2D5%2E)
25. Bowen Peng, Jeffrey Quesnelle, Honglu Fan, and Enrico Shippole. 2023. Yarn: Efficient context window extension of large language models. arXiv preprint arXiv:2309.00071. [find](https://scholar.google.com/scholar?q=Bowen+Peng%2C+Jeffrey+Quesnelle%2C+Honglu+Fan%2C+and+Enrico+Shippole%2E+2023%2E+Yarn%3A+Efficient+context+window+extension+of+large+language+models%2E+arXiv+preprint+arXiv%3A2309%2E00071%2E)
26. Miao Peng, Nuo Chen, Zongrui Suo, and Jia Li. 2025. Rewarding graph reasoning process makes llms more generalized reasoners. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, pages 2257–2268. [find](https://scholar.google.com/scholar?q=Miao+Peng%2C+Nuo+Chen%2C+Zongrui+Suo%2C+and+Jia+Li%2E+2025%2E+Rewarding+graph+reasoning+process+makes+llms+more+generalized+reasoners%2E+In+Proceedings+of+the+31st+ACM+SIGKDD+Conference+on+Knowledge+Discovery+and)
27. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). CoRR, abs/2402.03300.
28. Weizhou Shen, Ziyi Yang, Chenliang Li, Zhiyuan Lu, Miao Peng, Huashan Sun, Yingcheng Shi, Shengyi Liao, Shaopeng Lai, Bo Zhang, and 1 others. 2025. [Qwenlong-l1](https://md.research.elamur.ai/free/en/papers/shen-2025-qwenlong-l1-5-post/). 5: Post-training recipe for long-context reasoning and memory management. arXiv preprint arXiv:2512.12967.
29. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2024a. [Hybridflow: A flexible and efficient rlhf framework](https://md.research.elamur.ai/free/en/papers/sheng-2024-hybridflow-flexible-efficient-rlhf-framework/). arXiv preprint arXiv: 2409.19256.
30. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2024b. [Hybridflow: A flexible and efficient rlhf framework](https://md.research.elamur.ai/free/en/papers/sheng-2024-hybridflow-flexible-efficient-rlhf-framework/). arXiv preprint arXiv: 2409.19256.
31. Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Shiji Song, and Gao Huang. 2025. [Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model?](https://md.research.elamur.ai/free/en/papers/yue-2025-reinforcement-learning-really-incentivize-reasoning/) arXiv preprint arXiv:2504.13837.
32. Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. 2024. [Roformer: Enhanced transformer with rotary position embedding](https://md.research.elamur.ai/free/en/papers/su-2021-roformer-enhanced-transformer-rotary-position/). Neurocomputing, 568:127063.
33. Qifan Zhang, Nuo Chen, Zehua Li, Miao Peng, Jing Tang, and Jia Li. Improving llms ‘generalized reasoning abilities by graph problems. In Second Conference on Language Modeling. Huashan Sun, Shengyi Liao, Yansen Han, Yu Bai, Yang Gao, Cheng Fu, Weizhou Shen, Fanqi Wan, Ming Yan, Ji Zhang, and 1 other… [find](https://scholar.google.com/scholar?q=Qifan+Zhang%2C+Nuo+Chen%2C+Zehua+Li%2C+Miao+Peng%2C+Jing+Tang%2C+and+Jia+Li%2E+Improving+llms+%E2%80%98generalized+reasoning+abilities+by+graph+problems%2E+In+Second+Conference+on+Language+Modeling%2E+Huashan+Sun%2C+Shengyi+Li)
34. Zhiyang Zhang, Ziqiang Liu, Huiming Wang, Renke Shan, Li Kuang, Lu Wang, and De Wen Soh. 2025. Re3syn: A dependency-based data synthesis framework for long-context post-training. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages… [find](https://scholar.google.com/scholar?q=Zhiyang+Zhang%2C+Ziqiang+Liu%2C+Huiming+Wang%2C+Renke+Shan%2C+Li+Kuang%2C+Lu+Wang%2C+and+De+Wen+Soh%2E+2025%2E+Re3syn%3A+A+dependency%2Dbased+data+synthesis+framework+for+long%2Dcontext+post%2Dtraining%2E+In+Proceedings+of+the)
35. Tongyi DeepResearch Team, Baixuan Li, Bo Zhang, Dingchu Zhang, Fei Huang, Guangyu Li, Guoxin Chen, Huifeng Yin, Jialong Wu, Jingren Zhou, and 1 others. 2025. Tongyi deepresearch technical report. arXiv preprint arXiv:2510.24701. — [arXiv:2510.24701](https://arxiv.org/abs/2510.24701) [find](https://scholar.google.com/scholar?q=Tongyi+DeepResearch+Team%2C+Baixuan+Li%2C+Bo+Zhang%2C+Dingchu+Zhang%2C+Fei+Huang%2C+Guangyu+Li%2C+Guoxin+Chen%2C+Huifeng+Yin%2C+Jialong+Wu%2C+Jingren+Zhou%2C+and+1+others%2E+2025%2E+Tongyi+deepresearch+technical+report%2E+arXi)
36. Yida Zhao, Kuan Li, Xixi Wu, Liwen Zhang, Dingchu Zhang, Baixuan Li, Maojia Song, Zhuo Chen, Chenxi Wang, Xinyu Wang, and 1 others. 2025. Repurposing synthetic data for fine-grained search agent supervision. arXiv preprint arXiv:2510.24694. — [arXiv:2510.24694](https://arxiv.org/abs/2510.24694) [find](https://scholar.google.com/scholar?q=Yida+Zhao%2C+Kuan+Li%2C+Xixi+Wu%2C+Liwen+Zhang%2C+Dingchu+Zhang%2C+Baixuan+Li%2C+Maojia+Song%2C+Zhuo+Chen%2C+Chenxi+Wang%2C+Xinyu+Wang%2C+and+1+others%2E+2025%2E+Repurposing+synthetic+data+for+fine%2Dgrained+search+agent+super)
37. Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. Musique: Multi-hop questions via single-hop question composition. Transactions of the Association for Computational Linguistics, 10:539–554. [find](https://scholar.google.com/scholar?q=Harsh+Trivedi%2C+Niranjan+Balasubramanian%2C+Tushar+Khot%2C+and+Ashish+Sabharwal%2E+2022%2E+Musique%3A+Multi%2Dhop+questions+via+single%2Dhop+question+composition%2E+Transactions+of+the+Association+for+Computational+Li)
38. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, and 1 others. 2023. [Judging llm-as-a-judge with mt-bench and chatbot arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). Advances in neural information processing systems, 36:46595–46623.
39. Fanqi Wan, Weizhou Shen, Shengyi Liao, Yingcheng Shi, Chenliang Li, Ziyi Yang, Ji Zhang, Fei Huang, Jingren Zhou, and Ming Yan. 2025. [Qwenlong-l1: Towards long-context large reasoning models with reinforcement learning](https://md.research.elamur.ai/free/en/papers/wan-2025-qwenlong-l1-long-context/). arXiv preprint arXiv:2505.17667.
40. Siyuan Wang, Gaokai Zhang, Li Lyna Zhang, Ning Shang, Fan Yang, Dongyao Chen, and Mao Yang. 2025. Loongrl: Reinforcement learning for advanced reasoning over long contexts. arXiv preprint arXiv:2510.19363. — [arXiv:2510.19363](https://arxiv.org/abs/2510.19363) [find](https://scholar.google.com/scholar?q=Siyuan+Wang%2C+Gaokai+Zhang%2C+Li+Lyna+Zhang%2C+Ning+Shang%2C+Fan+Yang%2C+Dongyao+Chen%2C+and+Mao+Yang%2E+2025%2E+Loongrl%3A+Reinforcement+learning+for+advanced+reasoning+over+long+contexts%2E+arXiv+preprint+arXiv%3A2510%2E1)
41. Wenhao Zhu, Pinzhen Chen, Hanxu Hu, Shujian Huang, Fei Yuan, Jiajun Chen, and Alexandra Birch. 2025a. Generalizing from short to long: Effective data synthesis for long-context instruction tuning. arXiv preprint arXiv:2502.15592. — [arXiv:2502.15592](https://arxiv.org/abs/2502.15592) [find](https://scholar.google.com/scholar?q=Wenhao+Zhu%2C+Pinzhen+Chen%2C+Hanxu+Hu%2C+Shujian+Huang%2C+Fei+Yuan%2C+Jiajun+Chen%2C+and+Alexandra+Birch%2E+2025a%2E+Generalizing+from+short+to+long%3A+Effective+data+synthesis+for+long%2Dcontext+instruction+tuning%2E+arX)
42. Wenqiao Zhu, Ji Liu, Rongjunchen Zhang, Haipang Wu, and Yulun Zhang. 2025b. Carft: Boosting llm reasoning via contrastive learning with annotated chain-of-thought-based reinforced fine-tuning. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 5933–5948. [find](https://scholar.google.com/scholar?q=Wenqiao+Zhu%2C+Ji+Liu%2C+Rongjunchen+Zhang%2C+Haipang+Wu%2C+and+Yulun+Zhang%2E+2025b%2E+Carft%3A+Boosting+llm+reasoning+via+contrastive+learning+with+annotated+chain%2Dof%2Dthought%2Dbased+reinforced+fine%2Dtuning%2E+In+Proc)
43. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025a. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). arXiv preprint arXiv:2505.09388.
44. Cehao Yang, Xueyuan Lin, Chengjin Xu, Xuhui Jiang, Shengjie Ma, Aofan Liu, Hui Xiong, and Jian Guo. 2025b. LongFaith: Enhancing long-context reasoning in LLMs with faithful synthetic data. In Findings of the Association for Computational Linguistics: ACL 2025, Vienna, Austria. Association for Comput… [find](https://scholar.google.com/scholar?q=Cehao+Yang%2C+Xueyuan+Lin%2C+Chengjin+Xu%2C+Xuhui+Jiang%2C+Shengjie+Ma%2C+Aofan+Liu%2C+Hui+Xiong%2C+and+Jian+Guo%2E+2025b%2E+LongFaith%3A+Enhancing+long%2Dcontext+reasoning+in+LLMs+with+faithful+synthetic+data%2E+In+Findings)
45. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018. [Hotpotqa: A dataset for diverse, explainable multi-hop question answering](https://md.research.elamur.ai/free/en/papers/yang-2018-hotpotqa-dataset-diverse-explainable-multi/). In Proceedings of the 2018 conference on empirical methods in natural language processing, pages 2369–…

*Updated 1 November 2026*