--- title: "Rethinking Reflection in Pre-Training — paper by Shah et al. (2025)" description: "This Essential AI study asks whether that skill already develops during pre-training. 67 references, cited by 15 papers here." authors: ["Essential AI: Darsh J Shah", "Peter Rushton", "Somanshu Singla", "Mohit Parmar", "Kurt Smith", "Yash Vanjani", "Ashish Vaswani", "Adarsh Chaluvaraju", "Andrew Hojel", "Andrew Ma", "Anil Thomas", "Anthony Polloreno", "Ashish Tanwer", "Burhan Drak Sibai", "Divya S Mansingka", "Divya Shivaprasad", "Ishaan Shah", "Karl Stratos", "Khoi Nguyen", "Michael Callahan", "Michael Pust", "Mrinal Iyer", "Philip Monk", "Platon Mazarakis", "Ritvik Kapila", "Saurabh Srivastava", "Tim Romanski"] year: 2025 venue: "arXiv:2504.04022" source: "https://arxiv.org/abs/2504.04022" doi: "10.48550/arxiv.2504.04022" arxiv: "2504.04022" licence: "CC BY 4.0 (arXiv)" licence_url: "http://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/shah-2025-rethinking-reflection-pre-training/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/shah-2025-rethinking-reflection-pre-training/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/6/) / Rethinking Reflection in Pre-Training # Rethinking Reflection in Pre-Training **Authors:** Essential AI: Darsh J Shah, Peter Rushton, [Somanshu Singla](https://md.research.elamur.ai/free/en/authors/somanshu-singla/), Mohit Parmar, Kurt Smith, Yash Vanjani, [Ashish Vaswani](https://md.research.elamur.ai/free/en/authors/ashish-vaswani/), Adarsh Chaluvaraju, [Andrew Hojel](https://md.research.elamur.ai/free/en/authors/andrew-hojel/), Andrew Ma, Anil Thomas, Anthony Polloreno, Ashish Tanwer, Burhan Drak Sibai, Divya S Mansingka, Divya Shivaprasad, Ishaan Shah, [Karl Stratos](https://md.research.elamur.ai/free/en/authors/karl-stratos/), Khoi Nguyen, Michael Callahan, Michael Pust, Mrinal Iyer, Philip Monk, Platon Mazarakis, Ritvik Kapila, [Saurabh Srivastava](https://md.research.elamur.ai/free/en/authors/saurabh-srivastava/) and Tim Romanski · **Year:** 2025 · **Venue:** arXiv:2504.04022 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://arxiv.org/abs/2504.04022) · [PDF](https://arxiv.org/pdf/2504.04022) · [DOI 10.48550/arxiv.2504.04022](https://doi.org/10.48550/arxiv.2504.04022) · [arXiv 2504.04022](https://arxiv.org/abs/2504.04022) · [OpenAlex](https://openalex.org/W4416125058) **Citations:** 0 citations (OpenAlex, 2026-10-14) · [15](https://md.research.elamur.ai/free/en/papers/shah-2025-rethinking-reflection-pre-training/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary Reasoning models are often said to acquire reflection, meaning the ability to notice and repair flaws in a line of reasoning, through reinforcement learning. This Essential AI study asks whether that skill already develops during pre-training. The authors adapt existing benchmarks into adversarial tasks. For situational-reflection, they corrupt correct chains-of-thought from frontier models with human-like slips such as arithmetic mistakes or logical missteps. For self-reflection, they reuse the model's own failed attempts. They then append a trigger such as "Wait," and check whether the model still reaches the right answer. They also separate explicit reflection, flagged by a prompt-based LLM classifier, from implicit reflection, where the model answers correctly without saying it found an error. Across many OLMo-2 pre-training checkpoints and Qwen2.5 models, reflection appears early and strengthens steadily as pre-training compute grows. The findings challenge the view that reflection belongs only to post-training. They also suggest that extra pre-training can lower the test-time compute needed for a given accuracy. ## Key points - The authors release six adversarial datasets covering Mathematics, Coding, Logical Reasoning and Knowledge Acquisition. They are built from GSM8K, GSM8K-Platinum, CruxEval (cruxeval-i, cruxeval-o), BBH and TriviaQA. Code and data are at github.com/Essential-AI/reflection. - Situational-reflection occurred at least once in 231 of 240 dataset–checkpoint pairs, and self-reflection in 154 of 240. Across tasks, accuracy correlates with log(pre-training compute) at an average Pearson r of 0.76. - Early checkpoints already reflect. OLMo-2-7B shows reflection after only 198B tokens and self-corrects on all six self-reflection tasks at 4T tokens. On cruxeval-i, self-reflection emerges before successful self-correction. - The authors estimate training compute as 6nt and inference compute as 2nw. For OLMo-2-32B checkpoints on GSM8K-Platinum, more training compute means less test-time compute is needed to solve the same number of problems. - Qwen2.5 models show the same pattern: adversarial-task performance rises as model size increases. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* 📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works 🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works ## Cite Cite as: Essential AI: Darsh J Shah, Peter Rushton, Somanshu Singla, Mohit Parmar, Kurt Smith, Yash Vanjani, Ashish Vaswani, Adarsh Chaluvaraju, Andrew Hojel, Andrew Ma, Anil Thomas, Anthony Polloreno, Ashish Tanwer, Burhan Drak Sibai, Divya S Mansingka, Divya Shivaprasad, Ishaan Shah, Karl Stratos, Khoi Nguyen, Michael Callahan, Michael Pust, Mrinal Iyer, Philip Monk, Platon Mazarakis, Ritvik Kapila, Saurabh Srivastava and Tim Romanski. 2025. Rethinking Reflection in Pre-Training. arXiv:2504.04022. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @misc{shah2025rethinking, title = {{Rethinking Reflection in Pre-Training}}, author = {Essential AI: Darsh J Shah, Peter Rushton, Somanshu Singla, Mohit Parmar, Kurt Smith, Yash Vanjani, Ashish Vaswani, Adarsh Chaluvaraju, Andrew Hojel, Andrew Ma, Anil Thomas, Anthony Polloreno, Ashish Tanwer, Burhan Drak Sibai, Divya S Mansingka, Divya Shivaprasad, Ishaan Shah, Karl Stratos, Khoi Nguyen, Michael Callahan, Michael Pust, Mrinal Iyer, Philip Monk, Platon Mazarakis, Ritvik Kapila, Saurabh Srivastava and Tim Romanski}, year = {2025}, doi = {10.48550/arxiv.2504.04022}, eprint = {2504.04022}, archivePrefix = {arXiv}, url = {https://doi.org/10.48550/arxiv.2504.04022} } ``` - [← previous PRACTIQ: A Practical Conversational Text-to-SQL dataset with Ambiguous and Unanswerable Queries](https://md.research.elamur.ai/free/en/papers/dong-2025-practiq-practical-conversational-text-sql/) - [next → Rewarding the Unlikely: Lifting GRPO Beyond Distribution Sharpening](https://md.research.elamur.ai/free/en/papers/he-2025-rewarding-unlikely-lifting-grpo-beyond/) - [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/6/) ## References 67 references, 39 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. BIG bench authors. [Beyond the imitation game: Quantifying and extrapolating the capabilities of language models](https://md.research.elamur.ai/free/en/papers/srivastava-2022-imitation-game-quantifying-extrapolating-capabilities/). Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL [https://openreview.net/forum?id=uyTL5Bvosj](https://openreview.net/forum?id=uyTL5Bvosj). [↗](https://openreview.net/forum?id=uyTL5Bvosj) 2. Leonardo Bertolazzi, Davide Mazzaccara, Filippo Merlo, and Raffaella Bernardi. ChatGPT‘s information seeking strategy: Insights from the 20-questions game. In C. Maria Keet, Hung-Yi Lee, and Sina Zarrieß, editors, Proceedings of the 16th International Natural Language Generation Conference, pages 15… [find](https://scholar.google.com/scholar?q=Leonardo+Bertolazzi%2C+Davide+Mazzaccara%2C+Filippo+Merlo%2C+and+Raffaella+Bernardi%2E+ChatGPT%E2%80%98s+information+seeking+strategy%3A+Insights+from+the+20%2Dquestions+game%2E+In+C%2E+Maria+Keet%2C+Hung%2DYi+Lee%2C+and+Sina+Zarr) 3. Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šrndić, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. In Hendrik Blockeel, Kristian Kersting, Siegfried Nijssen, and Filip Železný, editors, Machine Learning and Knowledge Di… — [doi:10.1007/978-3-642-40994-3_25](https://doi.org/10.1007/978-3-642-40994-3_25) [find](https://scholar.google.com/scholar?q=Battista+Biggio%2C+Igino+Corona%2C+Davide+Maiorca%2C+Blaine+Nelson%2C+Nedim+%C5%A0rndic%CC%81%2C+Pavel+Laskov%2C+Giorgio+Giacinto%2C+and+Fabio+Roli%2E+Evasion+attacks+against+machine+learning+at+test+time%2E+In+Hendrik+Blockeel%2C) 4. Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, and Yi Zhang. [Sparks of artificial general intelligence: Early experiments with gpt-4](https://md.research.elamur.ai/free/en/papers/bubeck-2023-sparks-artificial-general-intelligence-early/), 2023. URL https:/… 5. Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Úlfar Erlingsson, Alina Oprea, and Colin Raffel. [Extracting training data from large language models](https://md.research.elamur.ai/free/en/papers/carlini-2020-extracting-training-data-large-language/). In 30th USENIX Security Symposium (USENIX Security 21), page… 6. Feng Chen, Allan Raventos, Nan Cheng, Surya Ganguli, and Shaul Druckmann. Rethinking fine-tuning when scaling test-time compute: Limiting confidence improves mathematical reasoning, 2025. URL https://arxiv.org/abs/2502.07154. [↗](https://arxiv.org/abs/2502.07154) 7. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/), 2021. URL https://arxiv.org/abs/2110.14168. [↗](https://arxiv.org/abs/2110.14168) 8. Israel Cohen, Yiteng Huang, Jingdong Chen, Jacob Benesty, Jacob Benesty, Jingdong Chen, Yiteng Huang, and Israel Cohen. Pearson correlation coefficient. Noise reduction in speech processing, pages 1–4, 2009. URL [https://link.springer.com/content/pdf/10.1007/978-3-642-00296-0_5.pdf](https://link.springer.com/content/pdf/10.1007/978-3-642-00296-0_5.pdf). — [doi:10.1007/978-3-642-00296-0_5.pdf](https://doi.org/10.1007/978-3-642-00296-0_5.pdf) [↗](https://link.springer.com/content/pdf/10.1007/978-3-642-00296-0_5.pdf) 9. DeepSeek-AI. [Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/), 2025a. URL https://arxiv.org/abs/2501.12948. [↗](https://arxiv.org/abs/2501.12948) 10. DeepSeek-AI. Deepseek-v3 technical report, 2025b. URL https://arxiv.org/abs/2412.19437. [↗](https://arxiv.org/abs/2412.19437) 11. Mingkai Deng, Jianyu Wang, Cheng-Ping Hsieh, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric Xing, and Zhiting Hu. [RLPrompt: Optimizing discrete text prompts with reinforcement learning](https://md.research.elamur.ai/free/en/papers/deng-2022-rlprompt-optimizing-discrete-text-prompts/). In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang, editors, Proceedings of the 2022 Conference on Empirical Me… 12. Subhabrata Dutta, Joykirat Singh, Soumen Chakrabarti, and Tanmoy Chakraborty. How to think step-by-step: A mechanistic understanding of chain-of-thought reasoning. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL [https://openreview.net/forum?id=uHLDkQVtyC](https://openreview.net/forum?id=uHLDkQVtyC). [↗](https://openreview.net/forum?id=uHLDkQVtyC) 13. Nouha Dziri, Ximing Lu, Melanie Sclar, Xiang (Lorraine) Li, Liwei Jiang, Bill Yuchen Lin, Sean Welleck, Peter West, Chandra Bhagavatula, Ronan Le Bras, Jena Hwang, Soumya Sanyal, Xiang Ren, Allyson Ettinger, Zaid Harchaoui, and Yejin Choi. [Faith and fate: Limits of transformers on compositionality](https://md.research.elamur.ai/free/en/papers/dziri-2023-faith-fate-limits-transformers-compositionality/).… 14. Yao Fu, Litu Ou, Mingyu Chen, Yuhao Wan, Hao Peng, and Tushar Khot. Chain-of-thought hub: A continuous effort to measure large language models’ reasoning performance, 2023a. [URL https://arxiv.org/abs/ 2305.17306](https://md.research.elamur.ai/free/en/papers/fu-2023-chain-thought-hub-continuous-effort/). [↗](https://arxiv.org/abs/) 15. Yao Fu, Hao Peng, Litu Ou, Ashish Sabharwal, and Tushar Khot. [Specializing smaller language models towards multi-step reasoning](https://md.research.elamur.ai/free/en/papers/fu-2023-specializing-smaller-language-models-towards/). In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Mac… 16. Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D. Goodman. [Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars](https://md.research.elamur.ai/free/en/papers/gandhi-2025-cognitive-behaviors-that-enable-self/), 2025. URL https://arxiv.org/abs/2503.01307. [↗](https://arxiv.org/abs/2503.01307) 17. Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac’h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Ani… [find](https://scholar.google.com/scholar?q=Leo+Gao%2C+Jonathan+Tow%2C+Baber+Abbasi%2C+Stella+Biderman%2C+Sid+Black%2C+Anthony+DiPofi%2C+Charles+Foster%2C+Laurence+Golding%2C+Jeffrey+Hsu%2C+Alain+Le+Noac%E2%80%99h%2C+Haonan+Li%2C+Kyle+McDonell%2C+Niklas+Muennighoff%2C+Chris+Oci) 18. Alex Gu, Baptiste Roziere, Hugh James Leather, Armando Solar-Lezama, Gabriel Synnaeve, and Sida Wang. [CRUXEval: A benchmark for code reasoning, understanding and execution](https://md.research.elamur.ai/free/en/papers/gu-2024-cruxeval-benchmark-code-reasoning-understanding/). In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, e… 19. Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Thomas Hennigan, Eric Noland, Katherine Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karén Simo… [find](https://scholar.google.com/scholar?q=Jordan+Hoffmann%2C+Sebastian+Borgeaud%2C+Arthur+Mensch%2C+Elena+Buchatskaya%2C+Trevor+Cai%2C+Eliza+Rutherford%2C+Diego+de+Las+Casas%2C+Lisa+Anne+Hendricks%2C+Johannes+Welbl%2C+Aidan+Clark%2C+Thomas+Hennigan%2C+Eric+Noland%2C) 20. Yifan Hou, Jiaoda Li, Yu Fei, Alessandro Stolfo, Wangchunshu Zhou, Guangtao Zeng, Antoine Bosselut, and Mrinmaya Sachan. Towards a mechanistic interpretation of multi-step reasoning capabilities of language models. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Confer… [find](https://scholar.google.com/scholar?q=Yifan+Hou%2C+Jiaoda+Li%2C+Yu+Fei%2C+Alessandro+Stolfo%2C+Wangchunshu+Zhou%2C+Guangtao+Zeng%2C+Antoine+Bosselut%2C+and+Mrinmaya+Sachan%2E+Towards+a+mechanistic+interpretation+of+multi%2Dstep+reasoning+capabilities+of+la) 21. Sandy H. Huang, Nicolas Papernot, Ian J. Goodfellow, Yan Duan, and Pieter Abbeel. Adversarial attacks on neural network policies. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Workshop Track Proceedings. [find](https://scholar.google.com/scholar?q=Sandy+H%2E+Huang%2C+Nicolas+Papernot%2C+Ian+J%2E+Goodfellow%2C+Yan+Duan%2C+and+Pieter+Abbeel%2E+Adversarial+attacks+on+neural+network+policies%2E+In+5th+International+Conference+on+Learning+Representations%2C+ICLR+2017) 22. OpenReview.net, 2017. URL [https://openreview.net/forum?id=ryvlRyBKl](https://openreview.net/forum?id=ryvlRyBKl). [↗](https://openreview.net/forum?id=ryvlRyBKl) 23. Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, and Tuo Zhao. [SMART: Robust and efficient fine-tuning for pre-trained natural language models through principled regularized optimization](https://md.research.elamur.ai/free/en/papers/jiang-2020-smart-robust-efficient-fine-tuning/). In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault, editors, Proceedings of… 24. Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. [TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension](https://md.research.elamur.ai/free/en/papers/joshi-2017-triviaqa-large-scale-distantly-supervised/). In Regina Barzilay and Min-Yen Kan, editors, Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long… 25. Daniel Kang, Xuechen Li, Ion Stoica, Carlos Guestrin, Matei Zaharia, and Tatsunori Hashimoto. Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attacks . In 2024 IEEE Security and Privacy Workshops (SPW), pages 132–143, Los Alamitos, CA, USA, May 2024. IEEE Computer Societ… [find](https://scholar.google.com/scholar?q=Daniel+Kang%2C+Xuechen+Li%2C+Ion+Stoica%2C+Carlos+Guestrin%2C+Matei+Zaharia%2C+and+Tatsunori+Hashimoto%2E+Exploiting+Programmatic+Behavior+of+LLMs%3A+Dual%2DUse+Through+Standard+Security+Attacks+%2E+In+2024+IEEE+Securi) 26. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. [Scaling laws for neural language models](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/), 2020. URL https://arxiv.org/abs/2001.08361. [↗](https://arxiv.org/abs/2001.08361) 27. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. [Large language models are zero-shot reasoners](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/). In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22, Red Hook, NY, USA, 2022. Curran Associates Inc. ISBN 9781713871088… 28. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. [Efficient memory management for large language model serving with pagedattention](https://md.research.elamur.ai/free/en/papers/kwon-2023-efficient-memory-management-large-language/). In Proceedings of the 29th Symposium on Operating Systems Principles, SOSP ’23, page 611–626,… 29. Dacheng Li, Shiyi Cao, Tyler Griggs, Shu Liu, Xiangxi Mo, Eric Tang, Sumanth Hegde, Kourosh Hakhamaneshi, Shishir G. Patil, Matei Zaharia, Joseph E. Gonzalez, and Ion Stoica. Llms can easily learn to reason from demonstrations structure, not content, is what matters!, 2025. URL [https://arxiv.org/abs…](https://arxiv.org/abs…) [↗](https://arxiv.org/abs…) 30. Jeffrey Li, Alex Fang, Georgios Smyrnis, Maor Ivgi, Matt Jordan, Samir Yitzhak Gadre, Hritik Bansal, Etash Guha, Sedrick Scott Keh, Kushal Arora, et al. [Datacomp-lm: In search of the next generation of training sets for language models](https://md.research.elamur.ai/free/en/papers/li-2024-datacomp-lm-search-next-generation/). Advances in Neural Information Processing Systems, 37:14200–142… 31. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step, 2023. [URL https://arxiv.org/abs/ 2305.20050](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/). [↗](https://arxiv.org/abs/) 32. Hanmeng Liu, Ruoxi Ning, Zhiyang Teng, Jian Liu, Qiji Zhou, and Yue Zhang. [Evaluating the logical reasoning ability of chatgpt and gpt-4](https://md.research.elamur.ai/free/en/papers/liu-2023-evaluating-logical-reasoning-ability-chatgpt/), 2023. URL https://arxiv.org/abs/2304.03439. [↗](https://arxiv.org/abs/2304.03439) 33. Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. [Understanding r1-zero-like training: A critical perspective](https://md.research.elamur.ai/free/en/papers/liu-2025-understanding-r1-zero-like-training/), 2025. URL https://arxiv.org/abs/2503. 20783. [↗](https://arxiv.org/abs/2503) 34. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterative refinement with self-… [find](https://scholar.google.com/scholar?q=Aman+Madaan%2C+Niket+Tandon%2C+Prakhar+Gupta%2C+Skyler+Hallinan%2C+Luyu+Gao%2C+Sarah+Wiegreffe%2C+Uri+Alon%2C+Nouha+Dziri%2C+Shrimai+Prabhumoye%2C+Yiming+Yang%2C+Shashank+Gupta%2C+Bodhisattwa+Prasad+Majumder%2C+Katherine+Her) 35. Philipp Mondorf and Barbara Plank. Beyond accuracy: Evaluating the reasoning behavior of large language models - a survey. In First Conference on Language Modeling, 2024. URL — [Beyond Accuracy: Evaluating the Reasoning Behavior of Large Language Models -- A Survey](https://md.research.elamur.ai/free/en/papers/mondorf-2024-accuracy-evaluating-reasoning-behavior-large/) [https://openreview.net/](https://openreview.net/) forum?id=Lmjgl2n11u. [↗](https://openreview.net/) 36. Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. [s1: Simple test-time scaling](https://md.research.elamur.ai/free/en/papers/muennighoff-2025-s1-simple-test-time-scaling/), 2025. URL https://arxiv.org/abs/2501.19393. [↗](https://arxiv.org/abs/2501.19393) 37. Team OLMo, Pete Walsh, Luca Soldaini, Dirk Groeneveld, Kyle Lo, Shane Arora, Akshita Bhagia, Yuling Gu, Shengyi Huang, Matt Jordan, Nathan Lambert, Dustin Schwenk, Oyvind Tafjord, Taira Anderson, David Atkinson, Faeze Brahman, Christopher Clark, Pradeep Dasigi, Nouha Dziri, Michal Guerquin, Hamish I… [find](https://scholar.google.com/scholar?q=Team+OLMo%2C+Pete+Walsh%2C+Luca+Soldaini%2C+Dirk+Groeneveld%2C+Kyle+Lo%2C+Shane+Arora%2C+Akshita+Bhagia%2C+Yuling+Gu%2C+Shengyi+Huang%2C+Matt+Jordan%2C+Nathan+Lambert%2C+Dustin+Schwenk%2C+Oyvind+Tafjord%2C+Taira+Anderson%2C+Davi) 38. OpenAI. [Gpt-4o system card](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/), 2024a. URL https://arxiv.org/abs/2410.21276. [↗](https://arxiv.org/abs/2410.21276) 39. OpenAI. [Openai o1 system card](https://md.research.elamur.ai/free/en/papers/openai-2024-openai-o1-system-card/), 2024b. URL https://arxiv.org/abs/2412.16720. [↗](https://arxiv.org/abs/2412.16720) 40. Paulo Pirozelli, Marcos M. José, Paulo de Tarso P. Filho, Anarosa A. F. Brandão, and Fabio G. Cozman. Assessing logical reasoning capabilities of encoder-only transformer models. In Tarek R. Besold, Artur d’Avila Garcez, Ernesto Jimenez-Ruiz, Roberto Confalonieri, Pranava Madhyastha, and Benedikt Wa… — [doi:10.1007/978-3-031-71167-1_2](https://doi.org/10.1007/978-3-031-71167-1_2) [find](https://scholar.google.com/scholar?q=Paulo+Pirozelli%2C+Marcos+M%2E+Jos%C3%A9%2C+Paulo+de+Tarso+P%2E+Filho%2C+Anarosa+A%2E+F%2E+Brand%C3%A3o%2C+and+Fabio+G%2E+Cozman%2E+Assessing+logical+reasoning+capabilities+of+encoder%2Donly+transformer+models%2E+In+Tarek+R%2E+Besold%2C+A) 41. Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng… [find](https://scholar.google.com/scholar?q=Qwen%2C+%3A%2C+An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Haoran+Wei%2C+Huan+Lin%2C+Jian+Yang%2C+Jianhong+Tu%2C+Jianwei+Zhang%2C+Jianxin+Yang%2C+Jiaxi+) 42. Nils Reimers and Iryna Gurevych. [Making monolingual sentence embeddings multilingual using knowledge distillation](https://md.research.elamur.ai/free/en/papers/reimers-2020-making-monolingual-sentence-embeddings-multilingual/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 11 2020. URL https://arxiv.org/abs/2004.09813. [↗](https://arxiv.org/abs/2004.09813) 43. Laura Ruis, Maximilian Mozes, Juhan Bae, Siddhartha Rao Kamalakara, Dwarak Talupuru, Acyr Locatelli, Robert Kirk, Tim Rocktäschel, Edward Grefenstette, and Max Bartolo. Procedural knowledge in pretraining drives reasoning in large language models, 2025. URL https://arxiv.org/abs/2411.12580. [↗](https://arxiv.org/abs/2411.12580) 44. Abulhair Saparov and He He. [Language models are greedy reasoners: A systematic formal analysis of chain-of-thought](https://md.research.elamur.ai/free/en/papers/saparov-2022-language-models-greedy-reasoners-systematic/). In The Eleventh International Conference on Learning Representations, 2023. URL [https://openreview.net/forum?id=qFVVBzXxR2V](https://openreview.net/forum?id=qFVVBzXxR2V). [↗](https://openreview.net/forum?id=qFVVBzXxR2V) 45. Tal Schuster, Darsh Shah, Yun Jie Serene Yeo, Daniel Roberto Filizzola Ortiz, Enrico Santus, and Regina Barzilay. [Towards debiasing fact verification models](https://md.research.elamur.ai/free/en/papers/schuster-2019-debiasing-fact-verification-models/). In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan, editors, Proceedings of the 2019 Conference on Empirical Methods in Natural Language… 46. Tal Schuster, Roei Schuster, Darsh J. Shah, and Regina Barzilay. [The limitations of stylometry for detecting machine-generated fake news](https://md.research.elamur.ai/free/en/papers/schuster-2020-limitations-stylometry-detecting-machine-generated/). Computational Linguistics, 46(2):499–510, June 2020. doi: 10.1162/coli_a_ 00380. URL https://aclanthology.org/2020.cl-2.8/. [↗](https://aclanthology.org/2020.cl-2.8/) 47. Leo Schwinn, David Dobre, Stephan Günnemann, and Gauthier Gidel. Adversarial attacks and defenses in large language models: Old and new threats. In Javier Antorán, Arno Blaas, Kelly Buchanan, Fan Feng, Vincent Fortuin, Sahra Ghalebikesabi, Andreas Kriegler, Ian Mason, David Rohde, Francisco J. R. Ru… [find](https://scholar.google.com/scholar?q=Leo+Schwinn%2C+David+Dobre%2C+Stephan+G%C3%BCnnemann%2C+and+Gauthier+Gidel%2E+Adversarial+attacks+and+defenses+in+large+language+models%3A+Old+and+new+threats%2E+In+Javier+Antor%C3%A1n%2C+Arno+Blaas%2C+Kelly+Buchanan%2C+Fan+Feng) 48. Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. ["do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models](https://md.research.elamur.ai/free/en/papers/shen-2023-anything-now-characterizing-evaluating-wild/). In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, CCS ’24, page 1671–1685, New York,… 49. Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. [Scaling llm test-time compute optimally can be more effective than scaling model parameters](https://md.research.elamur.ai/free/en/papers/snell-2024-scaling-llm-test-time-compute/), 2024. URL https://arxiv.org/abs/2408.03314. [↗](https://arxiv.org/abs/2408.03314) 50. Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian J. Goodfellow, and Rob Fergus. [Intriguing properties of neural networks](https://md.research.elamur.ai/free/en/papers/szegedy-2013-intriguing-properties-neural-networks/). In Yoshua Bengio and Yann LeCun, editors, 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada, April 14-16,… 51. Joshua Vendrow, Edward Vendrow, Sara Beery, and Aleksander Madry. Do large language model benchmarks test reliability?, 2025. URL https://arxiv.org/abs/2502.03461. [↗](https://arxiv.org/abs/2502.03461) 52. Boshi Wang, Xiang Yue, and Huan Sun. Can ChatGPT defend its belief in truth? evaluating LLM reasoning via debate. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Findings of the Association for Computational Linguistics: EMNLP 2023, pages 11865–11881, Singapore, December 2023a. Association fo… [find](https://scholar.google.com/scholar?q=Boshi+Wang%2C+Xiang+Yue%2C+and+Huan+Sun%2E+Can+ChatGPT+defend+its+belief+in+truth%3F+evaluating+LLM+reasoning+via+debate%2E+In+Houda+Bouamor%2C+Juan+Pino%2C+and+Kalika+Bali%2C+editors%2C+Findings+of+the+Association+for) 53. Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. [Math-shepherd: Verify and reinforce LLMs step-by-step without human annotations](https://md.research.elamur.ai/free/en/papers/wang-2024-math-shepherd-verify-reinforce-llms/). In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association f… 54. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. [Self-consistency improves chain of thought reasoning in language models](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/). In The Eleventh International Conference on Learning Representations, 2023b. URL [https://openreview.net/forum?](https://openreview.net/forum?) i… [↗](https://openreview.net/forum?) 55. Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. [Jailbroken: How does llm safety training fail?](https://md.research.elamur.ai/free/en/papers/wei-2023-jailbroken-llm-safety-training-fail/) In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 80079–80110. Curran Associates, Inc., 2023. URL htt… 56. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS ’22, Red Hoo… 57. Moritz Willig, Matej Zečević, Devendra Singh Dhami, and Kristian Kersting. Can foundation models talk causality? In UAI 2022 Workshop on Causal Representation Learning, 2022. URL [https://openreview.net/forum?id=DbJXEqU0kaM](https://openreview.net/forum?id=DbJXEqU0kaM). [↗](https://openreview.net/forum?id=DbJXEqU0kaM) 58. Larry Wos, Ross Overbeek, Rusty Lusk, and Jim Boyle. Automated Reasoning: Introduction and Applications (2nd edition). McGraw-Hill, 2nd edition, 1992. URL [https://www.abebooks.com/](https://www.abebooks.com/) Automated-Reasoning-Introduction-Applications-2nd-Edition/3444415211/bd. [↗](https://www.abebooks.com/) 59. Sophie Xhonneux, Alessandro Sordoni, Stephan Günnemann, Gauthier Gidel, and Leo Schwinn. Efficient adversarial training in llms with continuous attacks. arXiv preprint arXiv:2405.15589, 2024. URL [https://arxiv.org/pdf/2405.15589](https://arxiv.org/pdf/2405.15589). — [arXiv:2405.15589](https://arxiv.org/abs/2405.15589) [↗](https://arxiv.org/pdf/2405.15589) 60. Xilie Xu, Keyi Kong, Ning Liu, Lizhen Cui, Di Wang, Jingfeng Zhang, and Mohan S. Kankanhalli. An LLM can fool itself: A prompt-based adversarial attack. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. [find](https://scholar.google.com/scholar?q=Xilie+Xu%2C+Keyi+Kong%2C+Ning+Liu%2C+Lizhen+Cui%2C+Di+Wang%2C+Jingfeng+Zhang%2C+and+Mohan+S%2E+Kankanhalli%2E+An+LLM+can+fool+itself%3A+A+prompt%2Dbased+adversarial+attack%2E+In+The+Twelfth+International+Conference+on+Lear) 61. OpenReview.net, 2024. URL [https://openreview.net/forum?id=VVgGbB9TNV](https://openreview.net/forum?id=VVgGbB9TNV). [↗](https://openreview.net/forum?id=VVgGbB9TNV) 62. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Syste… [find](https://scholar.google.com/scholar?q=Shunyu+Yao%2C+Dian+Yu%2C+Jeffrey+Zhao%2C+Izhak+Shafran%2C+Tom+Griffiths%2C+Yuan+Cao%2C+and+Karthik+Narasimhan%2E+Tree+of+thoughts%3A+Deliberate+problem+solving+with+large+language+models%2E+In+A%2E+Oh%2C+T%2E+Naumann%2C+A%2E+Glo) 63. Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. [Demystifying long chain-of-thought reasoning in llms](https://md.research.elamur.ai/free/en/papers/yeo-2025-demystifying-long-chain-thought-reasoning/), 2025. URL https://arxiv.org/abs/2502.03373. [↗](https://arxiv.org/abs/2502.03373) 64. Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark Barrett, and Ying Sheng. [Sglang: Efficient execution of structured language model programs](https://md.research.elamur.ai/free/en/papers/zheng-2023-sglang-efficient-execution-structured-language/), 2024. URL https://arxiv.org/abs/2312.07104. [↗](https://arxiv.org/abs/2312.07104) 65. Chen Zhu, Yu Cheng, Zhe Gan, S. Sun, Tom Goldstein, and Jingjing Liu. [Freelb: Enhanced adversarial training for natural language understanding](https://md.research.elamur.ai/free/en/papers/zhu-2019-freelb-enhanced-adversarial-training-natural/). arXiv: Computation and Language, 2019. URL [https://api.semanticscholar.org/CorpusID:209475786](https://api.semanticscholar.org/CorpusID:209475786). [↗](https://api.semanticscholar.org/CorpusID:209475786) 66. Yan Zhuang, Qi Liu, Yuting Ning, Weizhe Huang, Rui Lv, Zhenya Huang, GuanHao Zhao, Zheng Zhang, Qingyang Mao, Shijin Wang, and Enhong Chen. Efficiently measuring the cognitive ability of LLMs: An adaptive testing perspective, 2024. URL [https://openreview.net/forum?id=s6X3s3rBPW](https://openreview.net/forum?id=s6X3s3rBPW). [↗](https://openreview.net/forum?id=s6X3s3rBPW) 67. Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. [Universal and transferable adversarial attacks on aligned language models](https://md.research.elamur.ai/free/en/papers/zou-2023-universal-transferable-adversarial-attacks-aligned/), 2023. URL https://arxiv.org/abs/2307. 15043. [↗](https://arxiv.org/abs/2307) *Updated 31 October 2026*