---
title: "The Unreasonable Effectiveness of Entropy Minimization in LLM"
description: "The authors from UIUC test entropy minimization (EM) as the sole objective in three settings. 99 references, cited by 44 papers here."
authors: ["Shivam Agarwal", "Zimin Zhang", "Lifan Yuan", "Jiawei Han", "Hao Peng"]
year: 2025
venue: "arXiv:2505.15134"
source: "https://arxiv.org/abs/2505.15134"
doi: "10.48550/arxiv.2505.15134"
arxiv: "2505.15134"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/agarwal-2025-unreasonable-effectiveness-entropy-minimization-llm/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/agarwal-2025-unreasonable-effectiveness-entropy-minimization-llm/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/3/) / The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning

# The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning

**Authors:** Shivam Agarwal, Zimin Zhang, [Lifan Yuan](https://md.research.elamur.ai/free/en/authors/lifan-yuan/), [Jiawei Han](https://md.research.elamur.ai/free/en/authors/jiawei-han/) and [Hao Peng](https://md.research.elamur.ai/free/en/authors/hao-peng/) · **Year:** 2025 · **Venue:** [arXiv:2505.15134](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/) ([NeurIPS 2025](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/2025/)) · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2505.15134) · [PDF](https://arxiv.org/pdf/2505.15134) · [DOI 10.48550/arxiv.2505.15134](https://doi.org/10.48550/arxiv.2505.15134) · [arXiv 2505.15134](https://arxiv.org/abs/2505.15134) · [OpenAlex](https://openalex.org/W4415328503)

**Citations:** 3 citations (OpenAlex, 2026-10-14) · [44](https://md.research.elamur.ai/free/en/papers/agarwal-2025-unreasonable-effectiveness-entropy-minimization-llm/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper asks whether pretrained LLMs can improve at reasoning with no labels, only by becoming more certain of their own outputs. The authors from UIUC test entropy minimization (EM) as the sole objective in three settings. EM-FT is a finetuning variant that lowers token-level entropy on the model's own sampled answers. EM-RL is a reinforcement-learning setup whose only reward is negative entropy, computed at the token or sequence level. EM-INF is a decoding-time method that adjusts logits at each step to sharpen the output distribution, with no change to the weights. They evaluate these on math, physics and coding benchmarks. Qwen-2.5-7B trained without supervision performs about as well as GRPO and RLOO trained on labeled data. EM-INF alone helps Qwen-2.5-32B compete with proprietary models on scientific coding. The gains largely disappear when the base model is weak or when confidence does not track correctness. The authors therefore argue that EM should become a standard baseline for separating algorithmic contributions from abilities the model already has.

## Key points
- On Qwen-7B, label-free EM-RL matches or exceeds GRPO and RLOO baselines that use 60K labeled examples. EM-FT and EM-RL outperform them on LeetCode, Minerva and (for EM-RL) AMC.
- On SciCode, EM-INF lets Qwen-2.5-32B match or outperform GPT-4o, Gemini 1.5 Pro and Claude 3 Opus. It needs roughly a third of the compute of self-consistency or sequential refinement because it decodes only one trajectory.
- EM-INF helps most on hard, high-uncertainty tasks such as AIME, UGPhysics and SciCode.
- **Limitation (base model):** on Llama-3.1-8B-Instruct, EM brings little benefit for math. Math accuracy rises from 40.6 to 42.2 with EM-FT, compared with 52.0 for RLOO, and EM sometimes lowers scores.
- **Limitation (task):** on individualistic value reasoning (IndVal), EM leaves Qwen2.5-7B-Instruct essentially flat at about 66%. Code is available at github.com/shivamag125/EM_PT.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Illinois Urbana-Champaign](https://md.research.elamur.ai/free/en/universities/university-of-illinois-urbana-champaign/) — University, Urbana, United States · 228 works · 74,591 citations
- [Beihang University](https://md.research.elamur.ai/free/en/universities/beihang-university/) — University, Beijing, China · 89 works · 9,227 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,959 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,789 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 4,145 works · [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,948 works

## Cite

Cite as: Shivam Agarwal, Zimin Zhang, Lifan Yuan, Jiawei Han and Hao Peng. 2025. The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning. arXiv:2505.15134.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{agarwal2025unreasonable,
  title = {{The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning}},
  author = {Shivam Agarwal, Zimin Zhang, Lifan Yuan, Jiawei Han and Hao Peng},
  year = {2025},
  doi = {10.48550/arxiv.2505.15134},
  eprint = {2505.15134},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2505.15134}
}
```

- [← previous Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Models](https://md.research.elamur.ai/free/en/papers/wu-2025-rethinking-kullback-leibler-divergence-knowledge/)
- [next → Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought](https://md.research.elamur.ai/free/en/papers/xiang-2025-towards-system-2-reasoning-llms/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/3/)

## References

99 references, 46 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Arash Ahmadian, Chris Cremer, Matthias Gallé, Marzieh Fadaee, Julia Kreutzer, Olivier Pietquin, Ahmet Üstün, and Sara Hooker. Back to basics: Revisiting reinforce style optimization for learning from human feedback in llms. arXiv preprint arXiv:2402.14740, 2024. [find](https://scholar.google.com/scholar?q=Arash+Ahmadian%2C+Chris+Cremer%2C+Matthias+Gall%C3%A9%2C+Marzieh+Fadaee%2C+Julia+Kreutzer%2C+Olivier+Pietquin%2C+Ahmet+%C3%9Cst%C3%BCn%2C+and+Sara+Hooker%2E+Back+to+basics%3A+Revisiting+reinforce+style+optimization+for+learning+from+)
2. Jihyun Janice Ahn and Wenpeng Yin. Prompt-reverse inconsistency: Llm self-inconsistency beyond generative randomness and prompt paraphrasing. arXiv preprint arXiv:2504.01282, 2025. [find](https://scholar.google.com/scholar?q=Jihyun+Janice+Ahn+and+Wenpeng+Yin%2E+Prompt%2Dreverse+inconsistency%3A+Llm+self%2Dinconsistency+beyond+generative+randomness+and+prompt+paraphrasing%2E+arXiv+preprint+arXiv%3A2504%2E01282%2C+2025%2E)
3. Vamsi Aribandi, Yi Tay, Tal Schuster, Jinfeng Rao, Huaixiu Steven Zheng, Sanket Vaibhav Mehta, Honglei Zhuang, Vinh Q Tran, Dara Bahri, Jianmo Ni, et al. [Ext5: Towards extreme multi-task scaling for transfer learning](https://md.research.elamur.ai/free/en/papers/aribandi-2021-ext5-towards-extreme-multi-task/). arXiv preprint arXiv:2111.10952, 2021.
4. Berk Atil, Alexa Chittams, Liseng Fu, Ferhan Ture, Lixinyu Xu, and Breck Baldwin. Llm stability: A detailed analysis with some surprises. arXiv preprint arXiv:2408.04667, 2024. — [arXiv:2408.04667](https://arxiv.org/abs/2408.04667) [find](https://scholar.google.com/scholar?q=Berk+Atil%2C+Alexa+Chittams%2C+Liseng+Fu%2C+Ferhan+Ture%2C+Lixinyu+Xu%2C+and+Breck+Baldwin%2E+Llm+stability%3A+A+detailed+analysis+with+some+surprises%2E+arXiv+preprint+arXiv%3A2408%2E04667%2C+2024%2E+%E2%80%94+%5BarXiv%3A2408%2E04667%5D%28ht)
5. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. [Constitutional ai: Harmlessness from ai feedback](https://md.research.elamur.ai/free/en/papers/bai-2022-constitutional-ai-harmlessness-ai-feedback/). arXiv preprint arXiv:2212.08073, 2022.
6. Vidhisha Balachandran, Jingya Chen, Lingjiao Chen, Shivam Garg, Neel Joshi, Yash Lara, John Langford, Besmira Nushi, Vibhav Vineet, Yue Wu, et al. Inference-time scaling for complex tasks: Where we stand and what lies ahead. arXiv preprint arXiv:2504.00294, 2025. [find](https://scholar.google.com/scholar?q=Vidhisha+Balachandran%2C+Jingya+Chen%2C+Lingjiao+Chen%2C+Shivam+Garg%2C+Neel+Joshi%2C+Yash+Lara%2C+John+Langford%2C+Besmira+Nushi%2C+Vibhav+Vineet%2C+Yue+Wu%2C+et+al%2E+Inference%2Dtime+scaling+for+complex+tasks%3A+Where+we+st)
7. Ahmad Beirami, Alekh Agarwal, Jonathan Berant, Alexander D’Amour, Jacob Eisenstein, Chirag Nagpal, and Ananda Theertha Suresh. Theoretical guarantees on the best-of-n alignment policy. arXiv preprint arXiv:2401.01879, 2024. — [arXiv:2401.01879](https://arxiv.org/abs/2401.01879) [find](https://scholar.google.com/scholar?q=Ahmad+Beirami%2C+Alekh+Agarwal%2C+Jonathan+Berant%2C+Alexander+D%E2%80%99Amour%2C+Jacob+Eisenstein%2C+Chirag+Nagpal%2C+and+Ananda+Theertha+Suresh%2E+Theoretical+guarantees+on+the+best%2Dof%2Dn+alignment+policy%2E+arXiv+preprint+)
8. David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A Raffel. Mixmatch: A holistic approach to semi-supervised learning. Advances in neural information processing systems, 32, 2019. [find](https://scholar.google.com/scholar?q=David+Berthelot%2C+Nicholas+Carlini%2C+Ian+Goodfellow%2C+Nicolas+Papernot%2C+Avital+Oliver%2C+and+Colin+A+Raffel%2E+Mixmatch%3A+A+holistic+approach+to+semi%2Dsupervised+learning%2E+Advances+in+neural+information+proces)
9. Malik Boudiaf, Romain Mueller, Ismail Ben Ayed, and Luca Bertinetto. Parameter-free online test-time adaptation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8344–8353, 2022. [find](https://scholar.google.com/scholar?q=Malik+Boudiaf%2C+Romain+Mueller%2C+Ismail+Ben+Ayed%2C+and+Luca+Bertinetto%2E+Parameter%2Dfree+online+test%2Dtime+adaptation%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Computer+Vision+and+Pattern+Recognition%2C+p)
10. Sébastien Bubeck, Varun Chadrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. [Sparks of artificial general intelligence: Early experiments with gpt-4](https://md.research.elamur.ai/free/en/papers/bubeck-2023-sparks-artificial-general-intelligence-early/), 2023.
11. Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. Teaching large language models to self-debug. In The Twelfth International Conference on Learning Representations, 2024. URL [https://openreview.net/forum?id=KuPixIqPiq](https://openreview.net/forum?id=KuPixIqPiq). [↗](https://openreview.net/forum?id=KuPixIqPiq)
12. Thomas M Cover. Elements of information theory. John Wiley & Sons, 1999. [find](https://scholar.google.com/scholar?q=Thomas+M+Cover%2E+Elements+of+information+theory%2E+John+Wiley+%26+Sons%2C+1999%2E)
13. Ganqu Cui, Lifan Yuan, Zefan Wang, Hanbin Wang, Wendi Li, Bingxiang He, Yuchen Fan, Tianyu Yu, Qixin Xu, Weize Chen, et al. [Process reinforcement through implicit rewards](https://md.research.elamur.ai/free/en/papers/cui-2025-process-reinforcement-through-implicit-rewards/). arXiv preprint arXiv:2502.01456, 2025.
14. Yuhao Ding, Junzi Zhang, Hyunin Lee, and Javad Lavaei. Beyond exact gradients: Convergence of stochastic soft-max policy gradient methods with entropy regularization. arXiv preprint arXiv:2110.10117, 2021. — [arXiv:2110.10117](https://arxiv.org/abs/2110.10117) [find](https://scholar.google.com/scholar?q=Yuhao+Ding%2C+Junzi+Zhang%2C+Hyunin+Lee%2C+and+Javad+Lavaei%2E+Beyond+exact+gradients%3A+Convergence+of+stochastic+soft%2Dmax+policy+gradient+methods+with+entropy+regularization%2E+arXiv+preprint+arXiv%3A2110%2E10117%2C+)
15. Mario Döbler, Florian Marencke, Robert A Marsden, and Bin Yang. Diversity-aware buffer for coping with temporally correlated data streams in online test-time adaptation. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 7665–7669. IEEE, 2024… [find](https://scholar.google.com/scholar?q=Mario+D%C3%B6bler%2C+Florian+Marencke%2C+Robert+A+Marsden%2C+and+Bin+Yang%2E+Diversity%2Daware+buffer+for+coping+with+temporally+correlated+data+streams+in+online+test%2Dtime+adaptation%2E+In+ICASSP+2024%2D2024+IEEE+Inter)
16. Hanze Dong, Wei Xiong, Deepanshu Goyal, Yihan Zhang, Winnie Chow, Rui Pan, Shizhe Diao, Jipeng Zhang, KaShun SHUM, and Tong Zhang. [RAFT: Reward ranked finetuning for generative foundation model alignment](https://md.research.elamur.ai/free/en/papers/dong-2023-raft-reward-ranked-finetuning-generative/). Transactions on Machine Learning Research, 2023. ISSN 2835-8856. URL [https://openreview.net/for…](https://openreview.net/for…) [↗](https://openreview.net/for…)
17. Laura Ekroot and Thomas M Cover. The entropy of markov trajectories. IEEE Transactions on Information Theory, 39(4):1418–1421, 2002. [find](https://scholar.google.com/scholar?q=Laura+Ekroot+and+Thomas+M+Cover%2E+The+entropy+of+markov+trajectories%2E+IEEE+Transactions+on+Information+Theory%2C+39%284%29%3A1418%E2%80%931421%2C+2002%2E)
18. Benjamin Eysenbach and Sergey Levine. Maximum entropy rl (provably) solves some robust rl problems. arXiv preprint arXiv:2103.06257, 2021. — [arXiv:2103.06257](https://arxiv.org/abs/2103.06257) [find](https://scholar.google.com/scholar?q=Benjamin+Eysenbach+and+Sergey+Levine%2E+Maximum+entropy+rl+%28provably%29+solves+some+robust+rl+problems%2E+arXiv+preprint+arXiv%3A2103%2E06257%2C+2021%2E+%E2%80%94+%5BarXiv%3A2103%2E06257%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2103%2E06257%29)
19. Angela Fan, Mike Lewis, and Yann Dauphin. [Hierarchical neural story generation](https://md.research.elamur.ai/free/en/papers/fan-2018-hierarchical-neural-story-generation/). In Annual Meeting of the Association for Computational Linguistics, 2018. URL [https://api.semanticscholar.org/CorpusID:44134226](https://api.semanticscholar.org/CorpusID:44134226). [↗](https://api.semanticscholar.org/CorpusID:44134226)
20. Chelsea Finn, Tianhe Yu, Justin Fu, Pieter Abbeel, and Sergey Levine. Generalizing skills with semi-supervised reinforcement learning. arXiv preprint arXiv:1612.00429, 2016. — [arXiv:1612.00429](https://arxiv.org/abs/1612.00429) [find](https://scholar.google.com/scholar?q=Chelsea+Finn%2C+Tianhe+Yu%2C+Justin+Fu%2C+Pieter+Abbeel%2C+and+Sergey+Levine%2E+Generalizing+skills+with+semi%2Dsupervised+reinforcement+learning%2E+arXiv+preprint+arXiv%3A1612%2E00429%2C+2016%2E+%E2%80%94+%5BarXiv%3A1612%2E00429%5D%28https)
21. Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D Goodman. [Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars](https://md.research.elamur.ai/free/en/papers/gandhi-2025-cognitive-behaviors-that-enable-self/). arXiv preprint arXiv:2503.01307, 2025.
22. Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, and Weizhu Chen. [Critic: Large language models can self-correct with tool-interactive critiquing](https://md.research.elamur.ai/free/en/papers/gou-2023-critic-large-language-models-self/). arXiv preprint arXiv:2305.11738, 2023.
23. Yves Grandvalet and Yoshua Bengio. Semi-supervised learning by entropy minimization. Advances in neural information processing systems, 17, 2004. [find](https://scholar.google.com/scholar?q=Yves+Grandvalet+and+Yoshua+Bengio%2E+Semi%2Dsupervised+learning+by+entropy+minimization%2E+Advances+in+neural+information+processing+systems%2C+17%2C+2004%2E)
24. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783, 2024.
25. S Dhillon Guneet, Chaudhari Pratik, Ravichandran Avinash, and S Stefano. A baseline for few-shot image classification. In International Conference on Learning Representations (ICLR), volume 10, 2020. [find](https://scholar.google.com/scholar?q=S+Dhillon+Guneet%2C+Chaudhari+Pratik%2C+Ravichandran+Avinash%2C+and+S+Stefano%2E+A+baseline+for+few%2Dshot+image+classification%2E+In+International+Conference+on+Learning+Representations+%28ICLR%29%2C+volume+10%2C+2020%2E)
26. Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, et al. Deepseek-coder: When the large language model meets programming–the rise of code intelligence. arXiv preprint arXiv:2401.14196, 2024. [find](https://scholar.google.com/scholar?q=Daya+Guo%2C+Qihao+Zhu%2C+Dejian+Yang%2C+Zhenda+Xie%2C+Kai+Dong%2C+Wentao+Zhang%2C+Guanting+Chen%2C+Xiao+Bi%2C+Yu+Wu%2C+YK+Li%2C+et+al%2E+Deepseek%2Dcoder%3A+When+the+large+language+model+meets+programming%E2%80%93the+rise+of+code+inte)
27. Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning, pages 1861–1870. Pmlr, 2018. [find](https://scholar.google.com/scholar?q=Tuomas+Haarnoja%2C+Aurick+Zhou%2C+Pieter+Abbeel%2C+and+Sergey+Levine%2E+Soft+actor%2Dcritic%3A+Off%2Dpolicy+maximum+entropy+deep+reinforcement+learning+with+a+stochastic+actor%2E+In+International+conference+on+machin)
28. Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, and Maosong Sun. [OlympiadBench: A challenging benchmark for promoting AGI with olympiad-level bilingual multimodal scientific problems](https://md.research.elamur.ai/free/en/papers/he-2024-olympiadbench-challenging-benchmark-promoting-agi/). In Lun-Wei Ku,…
29. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. [Measuring mathematical problem solving with the math dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). NeurIPS, 2021.
30. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. [The curious case of neural text degeneration](https://md.research.elamur.ai/free/en/papers/holtzman-2019-curious-case-neural-text-degeneration/). arXiv preprint arXiv:1904.09751, 2019.
31. Audrey Huang, Adam Block, Dylan J Foster, Dhruv Rohatgi, Cyril Zhang, Max Simchowitz, Jordan T Ash, and Akshay Krishnamurthy. [Self-improvement in language models: The sharpening mechanism](https://md.research.elamur.ai/free/en/papers/huang-2024-self-improvement-language-models-sharpening/). arXiv preprint arXiv:2412.01951, 2024.
32. Jiaxin Huang, Shixiang Shane Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. [Large language models can self-improve](https://md.research.elamur.ai/free/en/papers/huang-2023-large-language-models-can-self/). arXiv preprint arXiv:2210.11610, 2022.
33. Shiyu Huang, Hang Su, Jun Zhu, and Ting Chen. Svqn: Sequential variational soft q-learning networks. In International Conference on Learning Representations, 2020. URL [https://openreview.net/forum?id=r1xPh2VtPB](https://openreview.net/forum?id=r1xPh2VtPB). [↗](https://openreview.net/forum?id=r1xPh2VtPB)
34. Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. [Livecodebench: Holistic and contamination free evaluation of large language models for code](https://md.research.elamur.ai/free/en/papers/jain-2024-livecodebench-holistic-contamination-free-evaluation/). arXiv preprint arXiv:2403.07974, 2024.
35. Liwei Jiang, Taylor Sorensen, Sydney Levine, and Yejin Choi. [Can language models reason about individualistic human values and preferences?](https://md.research.elamur.ai/free/en/papers/jiang-2025-can-language-models-reason-about/) arXiv preprint arXiv:2410.03868, 2024.
36. Liwei Jiang, Taylor Sorensen, Sydney Levine, and Yejin Choi. [Can language models reason about individualistic human values and preferences?](https://md.research.elamur.ai/free/en/papers/jiang-2025-can-language-models-reason-about/), 2024. URL https://arxiv.org/abs/ 2410.03868. [↗](https://arxiv.org/abs/)
37. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. [Scaling laws for neural language models](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/). arXiv preprint arXiv:2001.08361, 2020.
38. Hilbert J Kappen, Vicenç Gómez, and Manfred Opper. Optimal control as a graphical model inference problem. Machine learning, 87:159–182, 2012. [find](https://scholar.google.com/scholar?q=Hilbert+J+Kappen%2C+Vicen%C3%A7+G%C3%B3mez%2C+and+Manfred+Opper%2E+Optimal+control+as+a+graphical+model+inference+problem%2E+Machine+learning%2C+87%3A159%E2%80%93182%2C+2012%2E)
39. Mijeong Kim, Seonguk Seo, and Bohyung Han. Infonerf: Ray entropy minimization for few-shot neural volume rendering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12912–12921, 2022. [find](https://scholar.google.com/scholar?q=Mijeong+Kim%2C+Seonguk+Seo%2C+and+Bohyung+Han%2E+Infonerf%3A+Ray+entropy+minimization+for+few%2Dshot+neural+volume+rendering%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Computer+Vision+and+Pattern+Recognition)
40. Dong-Hyun Lee. Pseudo-label : The simple and efficient semi-supervised learning method for deep neural networks. 2013. URL [https://api.semanticscholar.org/CorpusID:](https://api.semanticscholar.org/CorpusID:) 18507866. [↗](https://api.semanticscholar.org/CorpusID:)
41. Sergey Levine. Reinforcement learning and control as probabilistic inference: Tutorial and review. arXiv preprint arXiv:1805.00909, 2018. — [arXiv:1805.00909](https://arxiv.org/abs/1805.00909) [find](https://scholar.google.com/scholar?q=Sergey+Levine%2E+Reinforcement+learning+and+control+as+probabilistic+inference%3A+Tutorial+and+review%2E+arXiv+preprint+arXiv%3A1805%2E00909%2C+2018%2E+%E2%80%94+%5BarXiv%3A1805%2E00909%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1805%2E00909%29)
42. Aitor Lewkowycz, Anders Johan Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Venkatesh Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, Yuhuai Wu, Behnam Neyshabur, Guy Gur-Ari, and Vedant Misra. [Solving quantitative reasoning problems with language models](https://md.research.elamur.ai/free/en/papers/lewkowycz-2022-solving-quantitative-reasoning-problems-language/). In Alic…
43. Jia LI, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Costa Huang, Kashif Rasul, Longhui Yu, Albert Jiang, Ziju Shen, Zihan Qin, Bin Dong, Li Zhou, Yann Fleureau, Guillaume Lample, and Stanislas Polu. Numinamath. [https://huggingface.co/AI-MO/NuminaMath-CoT](https://github.co… [find](https://scholar.google.com/scholar?q=Jia+LI%2C+Edward+Beeching%2C+Lewis+Tunstall%2C+Ben+Lipkin%2C+Roman+Soletskyi%2C+Shengyi+Costa+Huang%2C+Kashif+Rasul%2C+Longhui+Yu%2C+Albert+Jiang%2C+Ziju+Shen%2C+Zihan+Qin%2C+Bin+Dong%2C+Li+Zhou%2C+Yann+Fleureau%2C+Guillaume+Lam)
44. Yiwei Li, Ji Zhang, Shaoxiong Feng, Peiwen Yuan, Xinglin Wang, Jiayi Shi, Yueqi Zhang, Chuyi Tan, Boyuan Pan, Yao Hu, et al. Revisiting self-consistency from dynamic distributional alignment perspective on answer aggregation. arXiv preprint arXiv:2502.19830, 2025. [find](https://scholar.google.com/scholar?q=Yiwei+Li%2C+Ji+Zhang%2C+Shaoxiong+Feng%2C+Peiwen+Yuan%2C+Xinglin+Wang%2C+Jiayi+Shi%2C+Yueqi+Zhang%2C+Chuyi+Tan%2C+Boyuan+Pan%2C+Yao+Hu%2C+et+al%2E+Revisiting+self%2Dconsistency+from+dynamic+distributional+alignment+perspecti)
45. Zijun Liu, Peiyi Wang, Runxin Xu, Shirong Ma, Chong Ruan, Peng Li, Yang Liu, and Yu Wu. [Inference-time scaling for generalist reward modeling](https://md.research.elamur.ai/free/en/papers/liu-2025-inference-time-scaling-generalist/). arXiv preprint arXiv:2504.02495, 2025.
46. Trung Quoc Luong, Xinbo Zhang, Zhanming Jie, Peng Sun, Xiaoran Jin, and Hang Li. [Reft: Reasoning with reinforced fine-tuning](https://md.research.elamur.ai/free/en/papers/trung-2024-reft-reasoning-reinforced-fine-tuning/). arXiv preprint arXiv:2401.08967, 3, 2024.
47. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36:46534–46594, 2023. [find](https://scholar.google.com/scholar?q=Aman+Madaan%2C+Niket+Tandon%2C+Prakhar+Gupta%2C+Skyler+Hallinan%2C+Luyu+Gao%2C+Sarah+Wiegreffe%2C+Uri+Alon%2C+Nouha+Dziri%2C+Shrimai+Prabhumoye%2C+Yiming+Yang%2C+et+al%2E+Self%2Drefine%3A+Iterative+refinement+with+self%2Dfeedbac)
48. Robert A Marsden, Mario Döbler, and Bin Yang. Universal test-time adaptation through weight ensembling, diversity weighting, and prior correction. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 2555–2565, 2024. [find](https://scholar.google.com/scholar?q=Robert+A+Marsden%2C+Mario+D%C3%B6bler%2C+and+Bin+Yang%2E+Universal+test%2Dtime+adaptation+through+weight+ensembling%2C+diversity+weighting%2C+and+prior+correction%2E+In+Proceedings+of+the+IEEE%2FCVF+Winter+Conference+on+A)
49. Jincheng Mei, Wesley Chung, Valentin Thomas, Bo Dai, Csaba Szepesvari, and Dale Schuurmans. The role of baselines in policy gradient optimization. Advances in Neural Information Processing Systems, 35:17818–17830, 2022. [find](https://scholar.google.com/scholar?q=Jincheng+Mei%2C+Wesley+Chung%2C+Valentin+Thomas%2C+Bo+Dai%2C+Csaba+Szepesvari%2C+and+Dale+Schuurmans%2E+The+role+of+baselines+in+policy+gradient+optimization%2E+Advances+in+Neural+Information+Processing+Systems%2C+35)
50. Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. [s1: Simple test-time scaling](https://md.research.elamur.ai/free/en/papers/muennighoff-2025-s1-simple-test-time-scaling/). arXiv preprint arXiv:2501.19393, 2025.
51. Gang Niu, Bo Dai, Makoto Yamada, and Masashi Sugiyama. Information-theoretic semi-supervised metric learning via entropy regularization. Neural Computation, 26:1717–1762, 2012. URL [https://api.semanticscholar.org/CorpusID:15064396](https://api.semanticscholar.org/CorpusID:15064396). [↗](https://api.semanticscholar.org/CorpusID:15064396)
52. Avital Oliver, Augustus Odena, Colin A Raffel, Ekin Dogus Cubuk, and Ian Goodfellow. Realistic evaluation of deep semi-supervised learning algorithms. Advances in neural information processing systems, 31, 2018. [find](https://scholar.google.com/scholar?q=Avital+Oliver%2C+Augustus+Odena%2C+Colin+A+Raffel%2C+Ekin+Dogus+Cubuk%2C+and+Ian+Goodfellow%2E+Realistic+evaluation+of+deep+semi%2Dsupervised+learning+algorithms%2E+Advances+in+neural+information+processing+systems)
53. Daniel Jarne Ornia, Giannis Delimpaltadakis, Jens Kober, and Javier Alonso-Mora. Predictable reinforcement learning dynamics through entropy rate minimization. arXiv preprint arXiv:2311.18703, 2023. — [arXiv:2311.18703](https://arxiv.org/abs/2311.18703) [find](https://scholar.google.com/scholar?q=Daniel+Jarne+Ornia%2C+Giannis+Delimpaltadakis%2C+Jens+Kober%2C+and+Javier+Alonso%2DMora%2E+Predictable+reinforcement+learning+dynamics+through+entropy+rate+minimization%2E+arXiv+preprint+arXiv%3A2311%2E18703%2C+2023%2E+%E2%80%94)
54. Jing-Cheng Pang, Pengyuan Wang, Kaiyuan Li, Xiong-Hui Chen, Jiacheng Xu, Zongzhang Zhang, and Yang Yu. [Language model self-improvement by reinforcement learning contemplation](https://md.research.elamur.ai/free/en/papers/pang-2023-language-model-self-improvement-reinforcement/). In The Twelfth International Conference on Learning Representations, 2024. URL [https://openreview.net/forum?id=38E4yUbrgr](https://openreview.net/forum?id=38E4yUbrgr). [↗](https://openreview.net/forum?id=38E4yUbrgr)
55. Jongjin Park, Younggyo Seo, Jinwoo Shin, Honglak Lee, Pieter Abbeel, and Kimin Lee. Surf: Semi-supervised reward learning with data augmentation for feedback-efficient preference-based reinforcement learning. arXiv preprint arXiv:2203.10050, 2022. — [arXiv:2203.10050](https://arxiv.org/abs/2203.10050) [find](https://scholar.google.com/scholar?q=Jongjin+Park%2C+Younggyo+Seo%2C+Jinwoo+Shin%2C+Honglak+Lee%2C+Pieter+Abbeel%2C+and+Kimin+Lee%2E+Surf%3A+Semi%2Dsupervised+reward+learning+with+data+augmentation+for+feedback%2Defficient+preference%2Dbased+reinforcement+l)
56. Ori Press, Ravid Shwartz-Ziv, Yann LeCun, and Matthias Bethge. The entropy enigma: Success and failure of entropy minimization. arXiv preprint arXiv:2405.05012, 2024. — [arXiv:2405.05012](https://arxiv.org/abs/2405.05012) [find](https://scholar.google.com/scholar?q=Ori+Press%2C+Ravid+Shwartz%2DZiv%2C+Yann+LeCun%2C+and+Matthias+Bethge%2E+The+entropy+enigma%3A+Success+and+failure+of+entropy+minimization%2E+arXiv+preprint+arXiv%3A2405%2E05012%2C+2024%2E+%E2%80%94+%5BarXiv%3A2405%2E05012%5D%28https%3A%2F%2Farxi)
57. Yuxiao Qu, Tianjun Zhang, Naman Garg, and Aviral Kumar. [Recursive introspection: Teaching language model agents how to self-improve](https://md.research.elamur.ai/free/en/papers/qu-2024-recursive-introspection-teaching-language-model/). Advances in Neural Information Processing Systems, 37:55249–55285, 2024.
58. Alec Radford and Karthik Narasimhan. Improving language understanding by generative pre-training. 2018. URL [https://api.semanticscholar.org/CorpusID:49313245](https://api.semanticscholar.org/CorpusID:49313245). [↗](https://api.semanticscholar.org/CorpusID:49313245)
59. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). Journal of machine learning research, 21(140):1–67, 2020.
60. Subhankar Roy, Aliaksandr Siarohin, Enver Sangineto, Samuel Rota Bulo, Nicu Sebe, and Elisa Ricci. Unsupervised domain adaptation using feature-whitening and consensus loss. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9471–9480, 2019. [find](https://scholar.google.com/scholar?q=Subhankar+Roy%2C+Aliaksandr+Siarohin%2C+Enver+Sangineto%2C+Samuel+Rota+Bulo%2C+Nicu+Sebe%2C+and+Elisa+Ricci%2E+Unsupervised+domain+adaptation+using+feature%2Dwhitening+and+consensus+loss%2E+In+Proceedings+of+the+IEEE)
61. Evgenia Rusak, Steffen Schneider, George Pachitariu, Luisa Eck, Peter Gehler, Oliver Bringmann, Wieland Brendel, and Matthias Bethge. If your data distribution shifts, use self-learning. arXiv preprint arXiv:2104.12928, 2021. — [arXiv:2104.12928](https://arxiv.org/abs/2104.12928) [find](https://scholar.google.com/scholar?q=Evgenia+Rusak%2C+Steffen+Schneider%2C+George+Pachitariu%2C+Luisa+Eck%2C+Peter+Gehler%2C+Oliver+Bringmann%2C+Wieland+Brendel%2C+and+Matthias+Bethge%2E+If+your+data+distribution+shifts%2C+use+self%2Dlearning%2E+arXiv+preprin)
62. Kuniaki Saito, Donghyun Kim, Stan Sclaroff, Trevor Darrell, and Kate Saenko. Semi-supervised domain adaptation via minimax entropy. In Proceedings of the IEEE/CVF international conference on computer vision, pages 8050–8058, 2019. [find](https://scholar.google.com/scholar?q=Kuniaki+Saito%2C+Donghyun+Kim%2C+Stan+Sclaroff%2C+Trevor+Darrell%2C+and+Kate+Saenko%2E+Semi%2Dsupervised+domain+adaptation+via+minimax+entropy%2E+In+Proceedings+of+the+IEEE%2FCVF+international+conference+on+computer+)
63. Nikhil Sardana, Jacob Portes, Sasha Doubov, and Jonathan Frankle. [Beyond chinchilla-optimal: Accounting for inference in language model scaling laws](https://md.research.elamur.ai/free/en/papers/sardana-2024-chinchilla-optimal-accounting-inference-language/). arXiv preprint arXiv:2401.00448, 2023.
64. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017. [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Filip+Wolski%2C+Prafulla+Dhariwal%2C+Alec+Radford%2C+and+Oleg+Klimov%2E+Proximal+policy+optimization+algorithms%2E+arXiv+preprint+arXiv%3A1707%2E06347%2C+2017%2E)
65. Amrith Setlur, Chirag Nagpal, Adam Fisch, Xinyang Geng, Jacob Eisenstein, Rishabh Agarwal, Alekh Agarwal, Jonathan Berant, and Aviral Kumar. [Rewarding progress: Scaling automated process verifiers for llm reasoning](https://md.research.elamur.ai/free/en/papers/setlur-2024-rewarding-progress-scaling-automated-process/). arXiv preprint arXiv:2410.08146, 2024.
66. Amrith Setlur, Nived Rajaraman, Sergey Levine, and Aviral Kumar. [Scaling test-time compute without verification or rl is suboptimal](https://md.research.elamur.ai/free/en/papers/setlur-2025-scaling-test-time-compute-verification/). arXiv preprint arXiv:2502.12118, 2025.
67. Claude E Shannon. A mathematical theory of communication. The Bell system technical journal, 27(3):379–423, 1948. [find](https://scholar.google.com/scholar?q=Claude+E+Shannon%2E+A+mathematical+theory+of+communication%2E+The+Bell+system+technical+journal%2C+27%283%29%3A379%E2%80%93423%2C+1948%2E)
68. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). arXiv preprint arXiv:2402.03300, 2024.
69. Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. [Hybridflow: A flexible and efficient rlhf framework](https://md.research.elamur.ai/free/en/papers/sheng-2024-hybridflow-flexible-efficient-rlhf-framework/). arXiv preprint arXiv: 2409.19256, 2024.
70. Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. [Reflexion: Language agents with verbal reinforcement learning](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/). Advances in Neural Information Processing Systems, 36:8634–8652, 2023.
71. Manli Shu, Weili Nie, De-An Huang, Zhiding Yu, Tom Goldstein, Anima Anandkumar, and Chaowei Xiao. Test-time prompt tuning for zero-shot generalization in vision-language models. Advances in Neural Information Processing Systems, 35:14274–14289, 2022. [find](https://scholar.google.com/scholar?q=Manli+Shu%2C+Weili+Nie%2C+De%2DAn+Huang%2C+Zhiding+Yu%2C+Tom+Goldstein%2C+Anima+Anandkumar%2C+and+Chaowei+Xiao%2E+Test%2Dtime+prompt+tuning+for+zero%2Dshot+generalization+in+vision%2Dlanguage+models%2E+Advances+in+Neural+Inf)
72. Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. [Scaling llm test-time compute optimally can be more effective than scaling model parameters](https://md.research.elamur.ai/free/en/papers/snell-2024-scaling-llm-test-time-compute/), 2024. URL https://arxiv.org/abs/2408.03314, 11, 2024. [↗](https://arxiv.org/abs/2408.03314)
73. Kaya Stechly, Karthik Valmeekam, and Subbarao Kambhampati. [On the self-verification limitations of large language models on reasoning and planning tasks](https://md.research.elamur.ai/free/en/papers/stechly-2024-self-verification-limitations-large-language/). arXiv preprint arXiv:2402.08115, 2024.
74. Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul Christiano. Learning to summarize from human feedback, 2022. URL https://arxiv.org/abs/2009.01325. [↗](https://arxiv.org/abs/2009.01325)
75. Richard S Sutton, David McAllester, Satinder Singh, and Yishay Mansour. Policy gradient methods for reinforcement learning with function approximation. In S. Solla, T. Leen, and K. Müller, editors, Advances in Neural Information Processing Systems, volume 12. MIT Press, 1999. URL https://proceedings… [find](https://scholar.google.com/scholar?q=Richard+S+Sutton%2C+David+McAllester%2C+Satinder+Singh%2C+and+Yishay+Mansour%2E+Policy+gradient+methods+for+reinforcement+learning+with+function+approximation%2E+In+S%2E+Solla%2C+T%2E+Leen%2C+and+K%2E+M%C3%BCller%2C+editors%2C+Ad)
76. Richard S Sutton, David McAllester, Satinder Singh, and Yishay Mansour. Policy gradient methods for reinforcement learning with function approximation. Advances in neural information processing systems, 12, 1999. [find](https://scholar.google.com/scholar?q=Richard+S+Sutton%2C+David+McAllester%2C+Satinder+Singh%2C+and+Yishay+Mansour%2E+Policy+gradient+methods+for+reinforcement+learning+with+function+approximation%2E+Advances+in+neural+information+processing+system)
77. Qwen Team. Qwen2.5: A party of foundation models, September 2024. URL [https://qwenlm.github.io/blog/qwen2.5/](https://qwenlm.github.io/blog/qwen2.5/). [↗](https://qwenlm.github.io/blog/qwen2.5/)
78. Minyang Tian, Luyu Gao, Shizhuo Dylan Zhang, Xinan Chen, Cunwei Fan, Xuefei Guo, Roland Haas, Pan Ji, Kittithat Krongchon, Yao Li, et al. [Scicode: A research coding benchmark curated by scientists](https://md.research.elamur.ai/free/en/papers/tian-2024-scicode-research-coding-benchmark/), 2024. URL https://arxiv.org/abs/2407.13168. Friedrichs, Ralf Helbig, Behnoush Hajian, Dawid K. Fiejte… [↗](https://arxiv.org/abs/2407.13168)
79. Daniil Tiapkin, Denis Belomestny, Daniele Calandriello, Eric Moulines, Remi Munos, Alexey Naumov, Pierre Perrault, Yunhao Tang, Michal Valko, and Pierre Menard. Fast rates for maximum entropy exploration. In International Conference on Machine Learning, pages 34161–34221. PMLR, 2023. [find](https://scholar.google.com/scholar?q=Daniil+Tiapkin%2C+Denis+Belomestny%2C+Daniele+Calandriello%2C+Eric+Moulines%2C+Remi+Munos%2C+Alexey+Naumov%2C+Pierre+Perrault%2C+Yunhao+Tang%2C+Michal+Valko%2C+and+Pierre+Menard%2E+Fast+rates+for+maximum+entropy+explorat)
80. Tuan-Hung Vu, Himalaya Jain, Maxime Bucher, Matthieu Cord, and Patrick Pérez. Advent: Adversarial entropy minimization for domain adaptation in semantic segmentation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2517–2526, 2019. [find](https://scholar.google.com/scholar?q=Tuan%2DHung+Vu%2C+Himalaya+Jain%2C+Maxime+Bucher%2C+Matthieu+Cord%2C+and+Patrick+P%C3%A9rez%2E+Advent%3A+Adversarial+entropy+minimization+for+domain+adaptation+in+semantic+segmentation%2E+In+Proceedings+of+the+IEEE%2FCVF+co)
81. Dequan Wang, Evan Shelhamer, Shaoteng Liu, Bruno Olshausen, and Trevor Darrell. Tent: Fully test-time adaptation by entropy minimization. arXiv preprint arXiv:2006.10726, 2020. [find](https://scholar.google.com/scholar?q=Dequan+Wang%2C+Evan+Shelhamer%2C+Shaoteng+Liu%2C+Bruno+Olshausen%2C+and+Trevor+Darrell%2E+Tent%3A+Fully+test%2Dtime+adaptation+by+entropy+minimization%2E+arXiv+preprint+arXiv%3A2006%2E10726%2C+2020%2E)
82. Julian Junyan Wang and Victor Xiaoqi Wang. Assessing consistency and reproducibility in the outputs of large language models: Evidence across diverse finance and accounting tasks. arXiv preprint arXiv:2503.16974, 2025. — [arXiv:2503.16974](https://arxiv.org/abs/2503.16974) [find](https://scholar.google.com/scholar?q=Julian+Junyan+Wang+and+Victor+Xiaoqi+Wang%2E+Assessing+consistency+and+reproducibility+in+the+outputs+of+large+language+models%3A+Evidence+across+diverse+finance+and+accounting+tasks%2E+arXiv+preprint+arXiv)
83. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. [Self-consistency improves chain of thought reasoning in language models](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/). arXiv preprint arXiv:2203.11171, 2022.
84. Yiping Wang, Qing Yang, Zhiyuan Zeng, Liliang Ren, Lucas Liu, Baolin Peng, Hao Cheng, Xuehai He, Kuan Wang, Jianfeng Gao, et al. [Reinforcement learning for reasoning in large language models with one training example](https://md.research.elamur.ai/free/en/papers/wang-2025-reinforcement-learning-reasoning-large-language/). arXiv preprint arXiv:2504.20571, 2025.
85. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language models with self-generated instructions. arXiv preprint arXiv:2212.10560, 2022. [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Yeganeh+Kordi%2C+Swaroop+Mishra%2C+Alisa+Liu%2C+Noah+A+Smith%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+Self%2Dinstruct%3A+Aligning+language+models+with+self%2Dgenerated+instructions%2E+arXiv+preprint)
86. Sean Welleck, Amanda Bertsch, Matthew Finlayson, Hailey Schoelkopf, Alex Xie, Graham Neubig, Ilia Kulikov, and Zaid Harchaoui. [From decoding to meta-generation: Inference-time algorithms for large language models](https://md.research.elamur.ai/free/en/papers/welleck-2024-decoding-meta-generation-inference-time/). arXiv preprint arXiv:2406.16838, 2024.
87. Cathy Wu, Aravind Rajeswaran, Yan Duan, Vikash Kumar, Alexandre M Bayen, Sham Kakade, Igor Mordatch, and Pieter Abbeel. Variance reduction for policy gradient with action-dependent factorized baselines. arXiv preprint arXiv:1803.07246, 2018. — [arXiv:1803.07246](https://arxiv.org/abs/1803.07246) [find](https://scholar.google.com/scholar?q=Cathy+Wu%2C+Aravind+Rajeswaran%2C+Yan+Duan%2C+Vikash+Kumar%2C+Alexandre+M+Bayen%2C+Sham+Kakade%2C+Igor+Mordatch%2C+and+Pieter+Abbeel%2E+Variance+reduction+for+policy+gradient+with+action%2Ddependent+factorized+baseline)
88. Xin Xu, Qiyun Xu, Tong Xiao, Tianhao Chen, Yuchen Yan, Jiaxin Zhang, Shizhe Diao, Can Yang, and Yang Wang. [Ugphysics: A comprehensive benchmark for undergraduate physics reasoning with large language models](https://md.research.elamur.ai/free/en/papers/xu-2025-ugphysics-comprehensive-benchmark-undergraduate-physics/). arXiv preprint arXiv:2502.00334, 2025.
89. An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, et al. Qwen2. 5-math technical report: Toward mathematical expert model via self-improvement. arXiv preprint arXiv:2409.[12122, 2024](https://md.research.elamur.ai/free/en/papers/yang-2024-qwen2-5-math-technical-report/).
90. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jin… — [Qwen3 Technical Report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/)
91. Zheng Yuan, Hongyi Yuan, Chengpeng Li, Guanting Dong, Keming Lu, Chuanqi Tan, Chang Zhou, and Jingren Zhou. [Scaling relationship on learning mathematical reasoning with large language models](https://md.research.elamur.ai/free/en/papers/yuan-2023-scaling-relationship-learning-mathematical-reasoning/). arXiv preprint arXiv:2308.01825, 2023.
92. Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, et al. [Glm-130b: An open bilingual pre-trained model](https://md.research.elamur.ai/free/en/papers/zeng-2022-glm-130b-open-bilingual-pre/). arXiv preprint arXiv:2210.02414, 2022.
93. Aohan Zeng, Mingdao Liu, Rui Lu, Bowen Wang, Xiao Liu, Yuxiao Dong, and Jie Tang. [Agenttuning: Enabling generalized agent abilities for llms](https://md.research.elamur.ai/free/en/papers/zeng-2023-agenttuning-enabling-generalized-agent/). arXiv preprint arXiv:2310.12823, 2023.
94. Qingyang Zhang, Yatao Bian, Xinke Kong, Peilin Zhao, and Changqing Zhang. COME: Test-time adaption by conservatively minimizing entropy. In The Thirteenth International Conference on Learning Representations, 2025. URL [https://openreview.net/forum?](https://openreview.net/forum?) id=506BjJ1ziZ. [↗](https://openreview.net/forum?)
95. Qingyang Zhang, Haitao Wu, Changqing Zhang, Peilin Zhao, and Yatao Bian. [Right question is already half the answer: Fully unsupervised llm reasoning incentivization](https://md.research.elamur.ai/free/en/papers/zhang-2025-right-question-already-half-answer/). arXiv preprint arXiv:2504.05812, 2025.
96. Tianjun Zhang, Xuezhi Wang, Denny Zhou, Dale Schuurmans, and Joseph E Gonzalez. Tempera: Test-time prompting via reinforcement learning. arXiv preprint arXiv:2211.11890, 2022. — [arXiv:2211.11890](https://arxiv.org/abs/2211.11890) [find](https://scholar.google.com/scholar?q=Tianjun+Zhang%2C+Xuezhi+Wang%2C+Denny+Zhou%2C+Dale+Schuurmans%2C+and+Joseph+E+Gonzalez%2E+Tempera%3A+Test%2Dtime+prompting+via+reinforcement+learning%2E+arXiv+preprint+arXiv%3A2211%2E11890%2C+2022%2E+%E2%80%94+%5BarXiv%3A2211%2E11890%5D%28htt)
97. Minjun Zhu, Yixuan Weng, Linyi Yang, and Yue Zhang. [Personality alignment of large language models](https://md.research.elamur.ai/free/en/papers/zhu-2024-personality-alignment-large-language-models/). arXiv preprint arXiv:2408.11779, 2024.
98. Brian D Ziebart, Andrew L Maas, J Andrew Bagnell, Anind K Dey, et al. Maximum entropy inverse reinforcement learning. In Aaai, volume 8, pages 1433–1438. Chicago, IL, USA, 2008. [find](https://scholar.google.com/scholar?q=Brian+D+Ziebart%2C+Andrew+L+Maas%2C+J+Andrew+Bagnell%2C+Anind+K+Dey%2C+et+al%2E+Maximum+entropy+inverse+reinforcement+learning%2E+In+Aaai%2C+volume+8%2C+pages+1433%E2%80%931438%2E+Chicago%2C+IL%2C+USA%2C+2008%2E)
99. Yuxin Zuo, Kaiyan Zhang, Shang Qu, Li Sheng, Xuekai Zhu, Biqing Qi, Youbang Sun, Ganqu Cui, Ning Ding, and Bowen Zhou. [Ttrl: Test-time reinforcement learning](https://md.research.elamur.ai/free/en/papers/zuo-2025-ttrl-test-time-reinforcement-learning/). arXiv preprint arXiv:2504.16084, 2025.

*Updated 3 November 2026*