---
title: "Learning from Cognition — paper by Sun et al. (2026)"
description: "Zero-RL training with verifiable rewards wastes much of its compute on weaker base LLMs. 82 references."
authors: ["Zexu Sun", "Yongcheng Zeng", "Erxue Min", "Heyang Gao", "Bokai Ji", "Dugang Liu", "Xing Tang", "Xiuqiang He", "Xu Chen"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.1022/"
doi: "10.18653/v1/2026.acl-long.1022"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/sun-2026-learning-cognition-enhancing-rl-efficiency/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/sun-2026-learning-cognition-enhancing-rl-efficiency/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/34/) / Learning from Cognition: Enhancing RL Efficiency for LLM Reasoning via Hierarchical Metacognitive Decomposition and Refinement

# Learning from Cognition: Enhancing RL Efficiency for LLM Reasoning via Hierarchical Metacognitive Decomposition and Refinement

**Authors:** Zexu Sun, Yongcheng Zeng, [Erxue Min](https://md.research.elamur.ai/free/en/authors/erxue-min/), Heyang Gao, Bokai Ji, [Dugang Liu](https://md.research.elamur.ai/free/en/authors/dugang-liu/), Xing Tang, Xiuqiang He and [Xu Chen](https://md.research.elamur.ai/free/en/authors/xu-chen/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.1022/) · [PDF](https://aclanthology.org/2026.acl-long.1022.pdf) · [DOI 10.18653/v1/2026.acl-long.1022](https://doi.org/10.18653/v1/2026.acl-long.1022) · [OpenAlex](https://openalex.org/W7166822968)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Zero-RL training with verifiable rewards wastes much of its compute on weaker base LLMs. With a fixed prompt template, many problems yield no correct rollout, so accuracy-based filtering discards them. Cog-Rethinker tackles this by adding two cognition-inspired rollout stages on top of a DAPO-style pipeline. Problems that receive zero accuracy in the direct rollout are first re-attempted by having the policy split them into subproblems. This step is guided by demonstrations drawn from a metacognitive buffer, which stores the policy's own successful decompositions and is searched by problem similarity. Problems that still fail are then retried with a reflection template that shows the model its earlier incorrect answers. Correct outputs from both stages are mapped back to the original prompt format and used for SFT. This supports cold-start and keeps training and testing consistent. On Qwen2.5-1.5B-Base and Qwen2.5-7B-Base, the method mostly outperforms GRPO, DAPO and Reinforce++ on math and science benchmarks. It also produces more usable samples early on and converges faster, which makes RL more practical for smaller models.

## Key points
- With Qwen2.5-7B-Base, Cog-Rethinker scores 93.32% on GSM8K, 80.60% on MATH-500, 26.67% on both AIME 24 and AIME 25, 73.50% on AMC 2023, 65.52% on Gaokao 2023en, 36.42% on GPQA-diamond and 44.22% on Olympiad.
- At 1.5B scale, it is the only method with a nonzero AIME 25 score (all others reach 0.00%), though it slightly trails DAPO on GSM8K.
- In ablations, removing SFT hurts most (1.5B GSM8K falls to 72.25%). Removing the buffer drops 7B MATH-500 to 78.00%, and removing reflection drops 1.5B Olympiad to 18.67.
- Compared with DAPO, it yields more valid training batches with less generation overhead, gives shorter stabilized responses, and shows better accuracy per cumulative batch on MATH-500.
- Limitations: results depend on base-model quality and on the buffer demonstrations, the binary reward gives no credit for intermediate steps, and only math reasoning was tested. Code is released as Cog-Rethinker on GitHub.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Baidu](https://md.research.elamur.ai/free/en/companies/baidu/) — Company, Beijing, China · 74 works · 15,263 citations
- [Renmin University of China](https://md.research.elamur.ai/free/en/universities/renmin-university-of-china/) — University, Beijing, China · 114 works · 16,118 citations
- [Chinese Academy of Sciences](https://md.research.elamur.ai/free/en/government/chinese-academy-of-sciences/) — Government body, Beijing, China · 194 works · 23,906 citations
- [Institute of Automation](https://md.research.elamur.ai/free/en/labs/institute-of-automation/) — Research institute, Beijing, China · 70 works · 14,251 citations
- [Shenzhen University](https://md.research.elamur.ai/free/en/universities/shenzhen-university/) — University, Shenzhen, China · 16 works · 95 citations
- [Shenzhen Technology University](https://md.research.elamur.ai/free/en/universities/shenzhen-technology-university/) — University, Shenzhen, China · 3 works · 59 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,426 works

## Cite

Cite as: Zexu Sun, Yongcheng Zeng, Erxue Min, Heyang Gao, Bokai Ji, Dugang Liu, Xing Tang, Xiuqiang He and Xu Chen. 2026. Learning from Cognition: Enhancing RL Efficiency for LLM Reasoning via Hierarchical Metacognitive Decomposition and Refinement. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{sun2026learning,
  title = {{Learning from Cognition: Enhancing RL Efficiency for LLM Reasoning via Hierarchical Metacognitive Decomposition and Refinement}},
  author = {Zexu Sun, Yongcheng Zeng, Erxue Min, Heyang Gao, Bokai Ji, Dugang Liu, Xing Tang, Xiuqiang He and Xu Chen},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1022},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1022}
}
```

- [← previous LearnerCoMPASS: Intelligent Tutoring System with Dynamic Cognitive Diagnosis and Multi-Model Path Planning](https://md.research.elamur.ai/free/en/papers/sheng-2026-learnercompass-intelligent-tutoring-system-dynamic/)
- [next → Learning from Emptiness: De-biasing Listwise Rerankers with Content-Agnostic Probability Calibration](https://md.research.elamur.ai/free/en/papers/lv-2026-learning-emptiness-de-biasing-listwise/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/34/)

## References

82 references, 36 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Russell L Ackoff. 1989. From data to wisdom. Journal of applied systems analysis, 16(1):3–9. [find](https://scholar.google.com/scholar?q=Russell+L+Ackoff%2E+1989%2E+From+data+to+wisdom%2E+Journal+of+applied+systems+analysis%2C+16%281%29%3A3%E2%80%939%2E)
2. Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. 2016. [Concrete problems in ai safety](https://md.research.elamur.ai/free/en/papers/amodei-2016-concrete-problems-ai-safety/). arXiv preprint arXiv:1606.06565.
3. Sanghwan Bae, Jiwoo Hong, Min Young Lee, Hanbyul Kim, JeongYeon Nam, and Donghyun Kwak. 2025. [Online difficulty filtering for reasoning oriented reinforcement learning](https://md.research.elamur.ai/free/en/papers/bae-2026-online-difficulty-filtering-reasoning-oriented/). arXiv preprint arXiv:2504.03380.
4. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, and 1 others. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862. [find](https://scholar.google.com/scholar?q=Yuntao+Bai%2C+Andy+Jones%2C+Kamal+Ndousse%2C+Amanda+Askell%2C+Anna+Chen%2C+Nova+DasSarma%2C+Dawn+Drain%2C+Stanislav+Fort%2C+Deep+Ganguli%2C+Tom+Henighan%2C+and+1+others%2E+2022%2E+Training+a+helpful+and+harmless+assistant+wi)
5. Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, and 1 others. 2024. Graph of thoughts: Solving elaborate problems with large language models. In Proceedings of the AAAI Conference on A… [find](https://scholar.google.com/scholar?q=Maciej+Besta%2C+Nils+Blach%2C+Ales+Kubicek%2C+Robert+Gerstenberger%2C+Michal+Podstawski%2C+Lukas+Gianinazzi%2C+Joanna+Gajda%2C+Tomasz+Lehmann%2C+Hubert+Niewiadomski%2C+Piotr+Nyczyk%2C+and+1+others%2E+2024%2E+Graph+of+thought)
6. Sehyun Choi, Tianqing Fang, Zhaowei Wang, and Yangqiu Song. 2023. Kcts: knowledge-constrained tree search decoding with token-level hallucination detection. arXiv preprint arXiv:2310.09044. [find](https://scholar.google.com/scholar?q=Sehyun+Choi%2C+Tianqing+Fang%2C+Zhaowei+Wang%2C+and+Yangqiu+Song%2E+2023%2E+Kcts%3A+knowledge%2Dconstrained+tree+search+decoding+with+token%2Dlevel+hallucination+detection%2E+arXiv+preprint+arXiv%3A2310%2E09044%2E)
7. Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30. [find](https://scholar.google.com/scholar?q=Paul+F+Christiano%2C+Jan+Leike%2C+Tom+Brown%2C+Miljan+Martic%2C+Shane+Legg%2C+and+Dario+Amodei%2E+2017%2E+Deep+reinforcement+learning+from+human+preferences%2E+Advances+in+neural+information+processing+systems%2C+30%2E)
8. Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V Le, Sergey Levine, and Yi Ma. 2025. [Sft memorizes, rl generalizes: A comparative study of foundation model post-training](https://md.research.elamur.ai/free/en/papers/chu-2025-sft-memorizes-rl-generalizes-comparative/). arXiv preprint arXiv:2501.17161.
9. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, and 1 others. 2021. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168.
10. Ganqu Cui, Lifan Yuan, Zefan Wang, Hanbin Wang, Wendi Li, Bingxiang He, Yuchen Fan, Tianyu Yu, Qixin Xu, Weize Chen, and 1 others. 2025. [Process reinforcement through implicit rewards](https://md.research.elamur.ai/free/en/papers/cui-2025-process-reinforcement-through-implicit-rewards/). arXiv preprint arXiv:2502.01456.
11. Mz Dai, Chenxu Yang, and Qingyi Si. 2025. S-GRPO: Early exit via reinforcement learning in reasoning models. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Mz+Dai%2C+Chenxu+Yang%2C+and+Qingyi+Si%2E+2025%2E+S%2DGRPO%3A+Early+exit+via+reinforcement+learning+in+reasoning+models%2E+In+The+Thirty%2Dninth+Annual+Conference+on+Neural+Information+Processing+Systems%2E)
12. Mica R Endsley, Robert Hoffman, David Kaber, and Emilie Roth. 2007. Cognitive engineering and decision making: An overview and future course. Journal of Cognitive Engineering and Decision Making, 1(1):1–21. [find](https://scholar.google.com/scholar?q=Mica+R+Endsley%2C+Robert+Hoffman%2C+David+Kaber%2C+and+Emilie+Roth%2E+2007%2E+Cognitive+engineering+and+decision+making%3A+An+overview+and+future+course%2E+Journal+of+Cognitive+Engineering+and+Decision+Making%2C+1%281%29)
13. Tom Everitt, Marcus Hutter, Ramana Kumar, and Victoria Krakovna. 2021. Reward tampering problems and solutions in reinforcement learning: A causal influence diagram perspective. Synthese, 198(Suppl 27):6435–6467. [find](https://scholar.google.com/scholar?q=Tom+Everitt%2C+Marcus+Hutter%2C+Ramana+Kumar%2C+and+Victoria+Krakovna%2E+2021%2E+Reward+tampering+problems+and+solutions+in+reinforcement+learning%3A+A+causal+influence+diagram+perspective%2E+Synthese%2C+198%28Suppl+27)
14. Tom Everitt, Victoria Krakovna, Laurent Orseau, Marcus Hutter, and Shane Legg. 2017. Reinforcement learning with a corrupted reward channel. arXiv preprint arXiv:1705.08417. — [arXiv:1705.08417](https://arxiv.org/abs/1705.08417) [find](https://scholar.google.com/scholar?q=Tom+Everitt%2C+Victoria+Krakovna%2C+Laurent+Orseau%2C+Marcus+Hutter%2C+and+Shane+Legg%2E+2017%2E+Reinforcement+learning+with+a+corrupted+reward+channel%2E+arXiv+preprint+arXiv%3A1705%2E08417%2E+%E2%80%94+%5BarXiv%3A1705%2E08417%5D%28https)
15. Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D Goodman. 2025. [Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars](https://md.research.elamur.ai/free/en/papers/gandhi-2025-cognitive-behaviors-that-enable-self/). arXiv preprint arXiv:2503.01307.
16. Heyang Gao, Zexu Sun, Erxue Min, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Xu Chen. 2025. Solving the granularity mismatch: Hierarchical preference learning for long-horizon llm agents. arXiv preprint arXiv:2510.03253. — [arXiv:2510.03253](https://arxiv.org/abs/2510.03253) [find](https://scholar.google.com/scholar?q=Heyang+Gao%2C+Zexu+Sun%2C+Erxue+Min%2C+Hengyi+Cai%2C+Shuaiqiang+Wang%2C+Dawei+Yin%2C+and+Xu+Chen%2E+2025%2E+Solving+the+granularity+mismatch%3A+Hierarchical+preference+learning+for+long%2Dhorizon+llm+agents%2E+arXiv+prepri)
17. Leo Gao, John Schulman, and Jacob Hilton. 2023. [Scaling laws for reward model overoptimization](https://md.research.elamur.ai/free/en/papers/gao-2022-scaling-laws-reward-model-overoptimization/). In International Conference on Machine Learning, pages 10835–10866. PMLR.
18. Google. 2024. Introducing gemini 2.0: our new ai model for the agentic era. [https://blog.google/technology/google-deepmind/google-gemini-ai-update-december-2024/](https://blog.google/technology/google-deepmind/google-gemini-ai-update-december-2024/). [↗](https://blog.google/technology/google-deepmind/google-gemini-ai-update-december-2024/)
19. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. [Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). arXiv preprint arXiv:2501.12948.
20. Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, and 1 others. 2024. [Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems](https://md.research.elamur.ai/free/en/papers/he-2024-olympiadbench-challenging-benchmark-promoting-agi/). arXiv preprint arXiv:2402.14008.
21. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. [Measuring mathematical problem solving with the math dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). arXiv preprint arXiv:2103.03874.
22. Jian Hu. 2025. Reinforce++: A simple and efficient approach for aligning large language models. arXiv preprint arXiv:2501.03262. [find](https://scholar.google.com/scholar?q=Jian+Hu%2E+2025%2E+Reinforce%2B%2B%3A+A+simple+and+efficient+approach+for+aligning+large+language+models%2E+arXiv+preprint+arXiv%3A2501%2E03262%2E)
23. Jian Hu, Jason Klein Liu, Haotian Xu, and Wei Shen. 2025a. Reinforce++: An efficient rlhf algorithm with robustness to both prompt and reward models. arXiv preprint arXiv:2501.03262. [find](https://scholar.google.com/scholar?q=Jian+Hu%2C+Jason+Klein+Liu%2C+Haotian+Xu%2C+and+Wei+Shen%2E+2025a%2E+Reinforce%2B%2B%3A+An+efficient+rlhf+algorithm+with+robustness+to+both+prompt+and+reward+models%2E+arXiv+preprint+arXiv%3A2501%2E03262%2E)
24. Jingcheng Hu, Yinmin Zhang, Qi Han, Daxin Jiang, Xiangyu Zhang, and Heung-Yeung Shum. 2025b. [Open-reasoner-zero: An open source approach to scaling up reinforcement learning on the base model](https://md.research.elamur.ai/free/en/papers/hu-2025-open-reasoner-zero-open-source/). arXiv preprint arXiv:2503.24290.
25. Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, and 1 others. 2024. [Openai o1 system card](https://md.research.elamur.ai/free/en/papers/openai-2024-openai-o1-system-card/). arXiv preprint arXiv:2412.16720.
26. Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. 2024. [Livecodebench: Holistic and contamination free evaluation of large language models for code](https://md.research.elamur.ai/free/en/papers/jain-2024-livecodebench-holistic-contamination-free-evaluation/). arXiv preprint arXiv:2403.07974.
27. Albert Q Jiang, Sean Welleck, Jin Peng Zhou, Wenda Li, Jiacheng Liu, Mateja Jamnik, Timothée Lacroix, Yuhuai Wu, and Guillaume Lample. 2022. [Draft, sketch, and prove: Guiding formal theorem provers with informal proofs](https://md.research.elamur.ai/free/en/papers/jiang-2022-draft-sketch-prove-guiding-formal/). arXiv preprint arXiv:2210.12283.
28. Amirhossein Kazemnejad, Milad Aghajohari, Eva Portelance, Alessandro Sordoni, Siva Reddy, Aaron Courville, and Nicolas Le Roux. 2024. Vineppo: Unlocking rl potential for llm reasoning through refined credit assignment. arXiv preprint arXiv:2410.01679. [find](https://scholar.google.com/scholar?q=Amirhossein+Kazemnejad%2C+Milad+Aghajohari%2C+Eva+Portelance%2C+Alessandro+Sordoni%2C+Siva+Reddy%2C+Aaron+Courville%2C+and+Nicolas+Le+Roux%2E+2024%2E+Vineppo%3A+Unlocking+rl+potential+for+llm+reasoning+through+refined+)
29. Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, and 1 others. 2024. [T\\” ulu 3: Pushing frontiers in open language model post-training](https://md.research.elamur.ai/free/en/papers/lambert-2024-tulu-3-pushing-frontiers-open/). arXiv preprint arXiv:2411.15124.
30. Thomas Landauer, Darrell Laham, and Peter Foltz. 1997. Learning human-like knowledge by singular value decomposition: A progress report. Advances in neural information processing systems, 10. [find](https://scholar.google.com/scholar?q=Thomas+Landauer%2C+Darrell+Laham%2C+and+Peter+Foltz%2E+1997%2E+Learning+human%2Dlike+knowledge+by+singular+value+decomposition%3A+A+progress+report%2E+Advances+in+neural+information+processing+systems%2C+10%2E)
31. Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, and 1 others. 2022. [Solving quantitative reasoning problems with language models](https://md.research.elamur.ai/free/en/papers/lewkowycz-2022-solving-quantitative-reasoning-problems-language/). Advances in Neural Information Processing Systems, 35:3843–3857…
32. Jia Li, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Huang, Kashif Rasul, Longhui Yu, Albert Q Jiang, Ziju Shen, and 1 others. 2024. Numinamath: The largest public dataset in ai4maths with 860k pairs of competition math problems and solutions. Hugging Face repository, 13:9. [find](https://scholar.google.com/scholar?q=Jia+Li%2C+Edward+Beeching%2C+Lewis+Tunstall%2C+Ben+Lipkin%2C+Roman+Soletskyi%2C+Shengyi+Huang%2C+Kashif+Rasul%2C+Longhui+Yu%2C+Albert+Q+Jiang%2C+Ziju+Shen%2C+and+1+others%2E+2024%2E+Numinamath%3A+The+largest+public+dataset+in+)
33. Yifei Li, Zeqi Lin, Shizhuo Zhang, Qiang Fu, Bei Chen, Jian-Guang Lou, and Weizhu Chen. 2023. [Making language models better reasoners with step-aware verifier](https://md.research.elamur.ai/free/en/papers/li-2023-making-language-models-better-reasoners/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5315–5333.
34. Minpeng Liao, Wei Luo, Chengxi Li, Jing Wu, and Kai Fan. 2024. Mario: Math reasoning with code interpreter output–a reproducible pipeline. arXiv preprint arXiv:2401.08190. [find](https://scholar.google.com/scholar?q=Minpeng+Liao%2C+Wei+Luo%2C+Chengxi+Li%2C+Jing+Wu%2C+and+Kai+Fan%2E+2024%2E+Mario%3A+Math+reasoning+with+code+interpreter+output%E2%80%93a+reproducible+pipeline%2E+arXiv+preprint+arXiv%3A2401%2E08190%2E)
35. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. [Let’s verify step by step](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/). arXiv preprint arXiv:2305.20050.
36. Zhihang Lin, Mingbao Lin, Yuan Xie, and Rongrong Ji. 2025. Cppo: Accelerating the training of group relative policy optimization-based reasoning models. arXiv preprint arXiv:2503.22342. [find](https://scholar.google.com/scholar?q=Zhihang+Lin%2C+Mingbao+Lin%2C+Yuan+Xie%2C+and+Rongrong+Ji%2E+2025%2E+Cppo%3A+Accelerating+the+training+of+group+relative+policy+optimization%2Dbased+reasoning+models%2E+arXiv+preprint+arXiv%3A2503%2E22342%2E)
37. Jiacheng Liu, Andrew Cohen, Ramakanth Pasunuru, Yejin Choi, Hannaneh Hajishirzi, and Asli Celikyilmaz. 2024. [Don’t throw away your value model!](https://md.research.elamur.ai/free/en/papers/liu-2023-don-t-throw-away-value/) generating more preferable text with value-guided monte-carlo tree search decoding. In First Conference on Language Modeling.
38. Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. 2025. [Understanding r1-zero-like training: A critical perspective](https://md.research.elamur.ai/free/en/papers/liu-2025-understanding-r1-zero-like-training/). arXiv preprint arXiv:2503.20783.
39. Meta AI Team. 2024. Llama 3.2: Revolutionizing edge AI and vision with open, customizable models. [https://ai.meta.com/blog/](https://ai.meta.com/blog/) llama-3-2-connect-2024-vision-edge -mobile-devices/. [↗](https://ai.meta.com/blog/)
40. OpenAI, :, Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Allison Tam, and 244 others. 2024. [Openai o1 sy…](https://md.research.elamur.ai/free/en/papers/openai-2024-openai-o1-system-card/)
41. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in Neural Information Processing Systems, 35:27730–27744.
42. Qwen-Team. 2024. Qwq: Reflect deeply on the boundaries of the unknown. [find](https://scholar.google.com/scholar?q=Qwen%2DTeam%2E+2024%2E+Qwq%3A+Reflect+deeply+on+the+boundaries+of+the+unknown%2E)
43. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. 2024. [Gpqa: A graduate-level google-proof q&a benchmark](https://md.research.elamur.ai/free/en/papers/rein-2023-gpqa-graduate-level-google-proof/). In First Conference on Language Modeling.
44. Stephen Robertson, Hugo Zaragoza, and 1 others. 2009. The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends® in Information Retrieval, 3(4):333–389. [find](https://scholar.google.com/scholar?q=Stephen+Robertson%2C+Hugo+Zaragoza%2C+and+1+others%2E+2009%2E+The+probabilistic+relevance+framework%3A+Bm25+and+beyond%2E+Foundations+and+Trends%C2%AE+in+Information+Retrieval%2C+3%284%29%3A333%E2%80%93389%2E)
45. Sneheel Sarangi, Maha Elgarf, and Hanan Salam. 2025. [Decompose-tom: Enhancing theory of mind reasoning in large language models through simulation and task decomposition](https://md.research.elamur.ai/free/en/papers/sarangi-2025-decompose-tom-enhancing-theory-mind/). arXiv preprint arXiv:2501.09056.
46. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Filip+Wolski%2C+Prafulla+Dhariwal%2C+Alec+Radford%2C+and+Oleg+Klimov%2E+2017%2E+Proximal+policy+optimization+algorithms%2E+arXiv+preprint+arXiv%3A1707%2E06347%2E)
47. Amrith Setlur, Chirag Nagpal, Adam Fisch, Xinyang Geng, Jacob Eisenstein, Rishabh Agarwal, Alekh Agarwal, Jonathan Berant, and Aviral Kumar. 2024. Rewarding progress: Scaling automated process verifiers for llm reasoning. arXiv preprint arXiv:2410.08146. [find](https://scholar.google.com/scholar?q=Amrith+Setlur%2C+Chirag+Nagpal%2C+Adam+Fisch%2C+Xinyang+Geng%2C+Jacob+Eisenstein%2C+Rishabh+Agarwal%2C+Alekh+Agarwal%2C+Jonathan+Berant%2C+and+Aviral+Kumar%2E+2024%2E+Rewarding+progress%3A+Scaling+automated+process+verifie)
48. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, and 1 others. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). arXiv preprint arXiv:2402.03300.
49. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2024. [Hybridflow: A flexible and efficient rlhf framework](https://md.research.elamur.ai/free/en/papers/sheng-2024-hybridflow-flexible-efficient-rlhf-framework/). arXiv preprint arXiv:2409.19256.
50. Jingwei Shi, Xinxiang Yin, Jing Huang, Jinman Zhao, and Shengyu Tao. 2026. [Codehacker: Automated test case generation for detecting vulnerabilities in competitive programming solutions](https://md.research.elamur.ai/free/en/papers/shi-2026-codehacker-automated-test-case-generation/). arXiv preprint arXiv:2602.20213.
51. Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2024. [Scaling llm test-time compute optimally can be more effective than scaling model parameters](https://md.research.elamur.ai/free/en/papers/snell-2024-scaling-llm-test-time-compute/). arXiv preprint arXiv:2408.03314.
52. Feifan Song, Bowen Yu, Minghao Li, Haiyang Yu, Fei Huang, Yongbin Li, and Houfeng Wang. 2023. [Preference ranking optimization for human alignment](https://md.research.elamur.ai/free/en/papers/song-2023-preference-ranking-optimization-human-alignment/). arXiv preprint arXiv:2306.17492.
53. K Spärck Jones, S Walker, and SE Robertson. 1998. A probabilistic model of information and retrieval: development and status. Technical report, University of Cambridge, Computer Laboratory. [find](https://scholar.google.com/scholar?q=K+Spa%CC%88rck+Jones%2C+S+Walker%2C+and+SE+Robertson%2E+1998%2E+A+probabilistic+model+of+information+and+retrieval%3A+development+and+status%2E+Technical+report%2C+University+of+Cambridge%2C+Computer+Laboratory%2E)
54. Zexu Sun, Bokai Ji, Hengyi Cai, Shuaiqiang Wang, Lei Wang, Guangxia Li, and Xu Chen. 2026. Agentskiller: Scaling generalist agent intelligence through semantically integrated cross-domain data synthesis. arXiv preprint arXiv:2602.09372. — [arXiv:2602.09372](https://arxiv.org/abs/2602.09372) [find](https://scholar.google.com/scholar?q=Zexu+Sun%2C+Bokai+Ji%2C+Hengyi+Cai%2C+Shuaiqiang+Wang%2C+Lei+Wang%2C+Guangxia+Li%2C+and+Xu+Chen%2E+2026%2E+Agentskiller%3A+Scaling+generalist+agent+intelligence+through+semantically+integrated+cross%2Ddomain+data+synthes)
55. Richard S Sutton, David McAllester, Satinder Singh, and Yishay Mansour. 1999. Policy gradient methods for reinforcement learning with function approximation. Advances in neural information processing systems, 12. [find](https://scholar.google.com/scholar?q=Richard+S+Sutton%2C+David+McAllester%2C+Satinder+Singh%2C+and+Yishay+Mansour%2E+1999%2E+Policy+gradient+methods+for+reinforcement+learning+with+function+approximation%2E+Advances+in+neural+information+processing+)
56. Hao Tang, Keya Hu, Jin Peng Zhou, Sicheng Zhong, Wei-Long Zheng, Xujie Si, and Kevin Ellis. 2024. Code repair with llms gives an exploration-exploitation tradeoff. arXiv preprint arXiv:2405.17503. — [arXiv:2405.17503](https://arxiv.org/abs/2405.17503) [find](https://scholar.google.com/scholar?q=Hao+Tang%2C+Keya+Hu%2C+Jin+Peng+Zhou%2C+Sicheng+Zhong%2C+Wei%2DLong+Zheng%2C+Xujie+Si%2C+and+Kevin+Ellis%2E+2024%2E+Code+repair+with+llms+gives+an+exploration%2Dexploitation+tradeoff%2E+arXiv+preprint+arXiv%3A2405%2E17503%2E+%E2%80%94+%5B)
57. Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, and 1 others. 2025. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599. — [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://md.research.elamur.ai/free/en/papers/kimi-team-2025-kimi-k1-5-scaling-reinforcement/)
58. Paul Thagard. 2013. Cognitive science. In The Routledge companion to philosophy of science, pages 597– 608. Routledge. [find](https://scholar.google.com/scholar?q=Paul+Thagard%2E+2013%2E+Cognitive+science%2E+In+The+Routledge+companion+to+philosophy+of+science%2C+pages+597%E2%80%93+608%2E+Routledge%2E)
59. Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. 2024. [Math-shepherd: Verify and reinforce llms step-by-step without human annotations](https://md.research.elamur.ai/free/en/papers/wang-2024-math-shepherd-verify-reinforce-llms/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Paper…
60. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2022. Self-instruct: Aligning language models with self-generated instructions. arXiv preprint arXiv:2212.10560. [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Yeganeh+Kordi%2C+Swaroop+Mishra%2C+Alisa+Liu%2C+Noah+A+Smith%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+2022%2E+Self%2Dinstruct%3A+Aligning+language+models+with+self%2Dgenerated+instructions%2E+arXiv+pr)
61. Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, and 1 others. 2025. [Dapo: An open-source llm reinforcement learning system at scale](https://md.research.elamur.ai/free/en/papers/yu-2025-dapo-open-source-llm-reinforcement/). arXiv preprint arXiv:2503.14476.
62. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). Advances in neural information processing systems, 35:24824– 24837.
63. Yufeng Yuan, Yu Yue, Ruofei Zhu, Tiantian Fan, and Lin Yan. 2025. What’s behind ppo’s collapse in long-cot? value optimization holds the secret. arXiv preprint arXiv:2503.01491. — [arXiv:2503.01491](https://arxiv.org/abs/2503.01491) [find](https://scholar.google.com/scholar?q=Yufeng+Yuan%2C+Yu+Yue%2C+Ruofei+Zhu%2C+Tiantian+Fan%2C+and+Lin+Yan%2E+2025%2E+What%E2%80%99s+behind+ppo%E2%80%99s+collapse+in+long%2Dcot%3F+value+optimization+holds+the+secret%2E+arXiv+preprint+arXiv%3A2503%2E01491%2E+%E2%80%94+%5BarXiv%3A2503%2E01491%5D%28h)
64. Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. 2024. [Inference scaling laws: An empirical analysis of compute-optimal inference for problem-solving with language models](https://md.research.elamur.ai/free/en/papers/wu-2024-inference-scaling-laws-empirical-analysis/). arXiv preprint arXiv:2408.00724.
65. Shijie Xia, Yiwei Qin, Xuefeng Li, Yan Ma, Run-Ze Fan, Steffi Chern, Haoyang Zou, Fan Zhou, Xiangkun Hu, Jiahe Jin, and 1 others. 2025. Generative ai act ii: Test time scaling drives cognition engineering. arXiv preprint arXiv:2504.13828. — [arXiv:2504.13828](https://arxiv.org/abs/2504.13828) [find](https://scholar.google.com/scholar?q=Shijie+Xia%2C+Yiwei+Qin%2C+Xuefeng+Li%2C+Yan+Ma%2C+Run%2DZe+Fan%2C+Steffi+Chern%2C+Haoyang+Zou%2C+Fan+Zhou%2C+Xiangkun+Hu%2C+Jiahe+Jin%2C+and+1+others%2E+2025%2E+Generative+ai+act+ii%3A+Test+time+scaling+drives+cognition+enginee)
66. Tian Xie, Zitian Gao, Qingnan Ren, Haoming Luo, Yuqian Hong, Bryan Dai, Joey Zhou, Kai Qiu, Zhirong Wu, and Chong Luo. 2025. [Logic-rl: Unleashing llm reasoning with rule-based reinforcement learning](https://md.research.elamur.ai/free/en/papers/xie-2025-logic-rl-unleashing-llm/). arXiv preprint arXiv:2502.14768.
67. Wei Xiong, Jiarui Yao, Yuhui Xu, Bo Pang, Lei Wang, Doyen Sahoo, Junnan Li, Nan Jiang, Tong Zhang, Caiming Xiong, and 1 others. 2025. A minimalist approach to llm reasoning: from rejection sampling to reinforce. arXiv preprint arXiv:2504.11343. — [arXiv:2504.11343](https://arxiv.org/abs/2504.11343) [find](https://scholar.google.com/scholar?q=Wei+Xiong%2C+Jiarui+Yao%2C+Yuhui+Xu%2C+Bo+Pang%2C+Lei+Wang%2C+Doyen+Sahoo%2C+Junnan+Li%2C+Nan+Jiang%2C+Tong+Zhang%2C+Caiming+Xiong%2C+and+1+others%2E+2025%2E+A+minimalist+approach+to+llm+reasoning%3A+from+rejection+sampling+to)
68. Shangzi Xue, Zhenya Huang, Jiayu Liu, Xin Lin, Yuting Ning, Binbin Jin, Xin Li, and Qi Liu. 2024. Decompose, analyze and rethink: Solving intricate problems with human-like reasoning cycle. Advances in Neural Information Processing Systems, 37:357–385. [find](https://scholar.google.com/scholar?q=Shangzi+Xue%2C+Zhenya+Huang%2C+Jiayu+Liu%2C+Xin+Lin%2C+Yuting+Ning%2C+Binbin+Jin%2C+Xin+Li%2C+and+Qi+Liu%2E+2024%2E+Decompose%2C+analyze+and+rethink%3A+Solving+intricate+problems+with+human%2Dlike+reasoning+cycle%2E+Advances+i)
69. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, and 1 others. 2024a. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115. [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Haoran+Wei%2C+and+1+others%2E+2024a%2E+Qwen2%2E+5+technical+report%2E+arXiv+preprint+arXiv%3A2412%2E1511)
70. Milan Zeleny. 1987. Management support systems: Towards integrated knowledge management. Human systems management, 7(1):59–70. [find](https://scholar.google.com/scholar?q=Milan+Zeleny%2E+1987%2E+Management+support+systems%3A+Towards+integrated+knowledge+management%2E+Human+systems+management%2C+7%281%29%3A59%E2%80%9370%2E)
71. Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. 2025. [Simplerl-zoo: Investigating and taming zero reinforcement learning for open base models in the wild](https://md.research.elamur.ai/free/en/papers/zeng-2025-simplerl-zoo-investigating-taming-zero/). arXiv preprint arXiv:2503.18892.
72. Qiyuan Zhang, Fuyuan Lyu, Zexu Sun, Lei Wang, Weixu Zhang, Zhihan Guo, Yufei Wang, Irwin King, Xue Liu, and Chen Ma. 2025a. What, how, where, and how well? a survey on test-time scaling in large language models. arXiv preprint arXiv:2503.24235. [find](https://scholar.google.com/scholar?q=Qiyuan+Zhang%2C+Fuyuan+Lyu%2C+Zexu+Sun%2C+Lei+Wang%2C+Weixu+Zhang%2C+Zhihan+Guo%2C+Yufei+Wang%2C+Irwin+King%2C+Xue+Liu%2C+and+Chen+Ma%2E+2025a%2E+What%2C+how%2C+where%2C+and+how+well%3F+a+survey+on+test%2Dtime+scaling+in+large+langu)
73. Qiyuan Zhang, Fuyuan Lyu, Zexu Sun, Lei Wang, Weixu Zhang, Wenyue Hua, Haolun Wu, Zhihan Guo, Yufei Wang, Niklas Muennighoff, and 1 others. 2025b. A survey on test-time scaling in large language models: What, how, where, and how well? arXiv preprint arXiv:2503.24235. [find](https://scholar.google.com/scholar?q=Qiyuan+Zhang%2C+Fuyuan+Lyu%2C+Zexu+Sun%2C+Lei+Wang%2C+Weixu+Zhang%2C+Wenyue+Hua%2C+Haolun+Wu%2C+Zhihan+Guo%2C+Yufei+Wang%2C+Niklas+Muennighoff%2C+and+1+others%2E+2025b%2E+A+survey+on+test%2Dtime+scaling+in+large+language+model)
74. Shun Zhang, Zhenfang Chen, Yikang Shen, Mingyu Ding, Joshua B Tenenbaum, and Chuang Gan. 2024. Planning with large language models for code generation. International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Shun+Zhang%2C+Zhenfang+Chen%2C+Yikang+Shen%2C+Mingyu+Ding%2C+Joshua+B+Tenenbaum%2C+and+Chuang+Gan%2E+2024%2E+Planning+with+large+language+models+for+code+generation%2E+International+Conference+on+Machine+Learning%2E)
75. Jinman Zhao, Erxue Min, Hui Wu, Ziheng Li, Zexu Sun, Hengyi Cai, Shuaiqiang Wang, Xu Chen, and Gerald Penn. 2026. Beyond step pruning: Information theory based step-level optimization for self-refining large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40… [find](https://scholar.google.com/scholar?q=Jinman+Zhao%2C+Erxue+Min%2C+Hui+Wu%2C+Ziheng+Li%2C+Zexu+Sun%2C+Hengyi+Cai%2C+Shuaiqiang+Wang%2C+Xu+Chen%2C+and+Gerald+Penn%2E+2026%2E+Beyond+step+pruning%3A+Information+theory+based+step%2Dlevel+optimization+for+self%2Drefinin)
76. Chenxu Yang, Qingyi Si, Mz Dai, Dingyu Yao, Mingyu Zheng, Minghui Chen, Zheng Lin, and Weiping Wang. 2025. Test-time prompt intervention. Preprint, arXiv:2508.02511. — [arXiv:2508.02511](https://arxiv.org/abs/2508.02511) [find](https://scholar.google.com/scholar?q=Chenxu+Yang%2C+Qingyi+Si%2C+Mz+Dai%2C+Dingyu+Yao%2C+Mingyu+Zheng%2C+Minghui+Chen%2C+Zheng+Lin%2C+and+Weiping+Wang%2E+2025%2E+Test%2Dtime+prompt+intervention%2E+Preprint%2C+arXiv%3A2508%2E02511%2E+%E2%80%94+%5BarXiv%3A2508%2E02511%5D%28https%3A%2F%2Farxiv)
77. Rosie Zhao, Alexandru Meterez, Sham Kakade, Cengiz Pehlevan, Samy Jelassi, and Eran Malach. 2025. Echo chamber: Rl post-training amplifies behaviors learned in pretraining. arXiv preprint arXiv:2504.07912. [find](https://scholar.google.com/scholar?q=Rosie+Zhao%2C+Alexandru+Meterez%2C+Sham+Kakade%2C+Cengiz+Pehlevan%2C+Samy+Jelassi%2C+and+Eran+Malach%2E+2025%2E+Echo+chamber%3A+Rl+post%2Dtraining+amplifies+behaviors+learned+in+pretraining%2E+arXiv+preprint+arXiv%3A2504%2E0)
78. Ling Yang, Zhaochen Yu, Tianjun Zhang, Shiyi Cao, Minkai Xu, Wentao Zhang, Joseph E Gonzalez, and Bin Cui. 2024b. Buffer of thoughts: Thought-augmented reasoning with large language models. Advances in Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Ling+Yang%2C+Zhaochen+Yu%2C+Tianjun+Zhang%2C+Shiyi+Cao%2C+Minkai+Xu%2C+Wentao+Zhang%2C+Joseph+E+Gonzalez%2C+and+Bin+Cui%2E+2024b%2E+Buffer+of+thoughts%3A+Thought%2Daugmented+reasoning+with+large+language+models%2E+Advances+i)
79. Xueliang Zhao, Wenda Li, and Lingpeng Kong. 2023. Decomposing the enigma: Subgoal-based demonstration learning for formal theorem proving. arXiv preprint arXiv:2305.16366. [find](https://scholar.google.com/scholar?q=Xueliang+Zhao%2C+Wenda+Li%2C+and+Lingpeng+Kong%2E+2023%2E+Decomposing+the+enigma%3A+Subgoal%2Dbased+demonstration+learning+for+formal+theorem+proving%2E+arXiv+preprint+arXiv%3A2305%2E16366%2E)
80. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2024. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36. [find](https://scholar.google.com/scholar?q=Shunyu+Yao%2C+Dian+Yu%2C+Jeffrey+Zhao%2C+Izhak+Shafran%2C+Tom+Griffiths%2C+Yuan+Cao%2C+and+Karthik+Narasimhan%2E+2024%2E+Tree+of+thoughts%3A+Deliberate+problem+solving+with+large+language+models%2E+Advances+in+Neural+Inf)
81. Andy Zhou, Kai Yan, Michal Shlapentokh-Rothman, Haohan Wang, and Yu-Xiong Wang. 2023. Language agent tree search unifies reasoning acting and planning in language models. arXiv preprint arXiv:2310.04406. [find](https://scholar.google.com/scholar?q=Andy+Zhou%2C+Kai+Yan%2C+Michal+Shlapentokh%2DRothman%2C+Haohan+Wang%2C+and+Yu%2DXiong+Wang%2E+2023%2E+Language+agent+tree+search+unifies+reasoning+acting+and+planning+in+language+models%2E+arXiv+preprint+arXiv%3A2310%2E044)
82. Yichi Zhou, Jianqiu Zhao, Yongxin Zhang, Bohan Wang, Siran Wang, Luoxin Chen, Jiahui Wang, Haowei Chen, Allan Jie, Xinbo Zhang, and 1 others. 2025. Solving formal math problems by decomposition and iterative reflection. arXiv preprint arXiv:2507.15225. [find](https://scholar.google.com/scholar?q=Yichi+Zhou%2C+Jianqiu+Zhao%2C+Yongxin+Zhang%2C+Bohan+Wang%2C+Siran+Wang%2C+Luoxin+Chen%2C+Jiahui+Wang%2C+Haowei+Chen%2C+Allan+Jie%2C+Xinbo+Zhang%2C+and+1+others%2E+2025%2E+Solving+formal+math+problems+by+decomposition+and+it)

*Updated 1 November 2026*