---
title: "Easy-to-Hard Generalization — paper by Sun et al. (2024)"
description: "Current alignment methods learn from human demonstrations or human judgments, so a model's skill… 97 references, cited by 49 papers here."
authors: ["Zhiqing Sun", "Longhui Yu", "Yikang Shen", "Weiyang Liu", "Yiming Yang", "Sean Welleck", "Chuang Gan"]
year: 2024
venue: "arXiv:2403.09472"
source: "https://arxiv.org/abs/2403.09472"
doi: "10.48550/arxiv.2403.09472"
arxiv: "2403.09472"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/sun-2024-easy-hard-generalization-scalable-alignment/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/sun-2024-easy-hard-generalization-scalable-alignment/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/8/) / Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervision

# Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervision

**Authors:** [Zhiqing Sun](https://md.research.elamur.ai/free/en/authors/zhiqing-sun/), [Longhui Yu](https://md.research.elamur.ai/free/en/authors/longhui-yu/), [Yikang Shen](https://md.research.elamur.ai/free/en/authors/yikang-shen/), [Weiyang Liu](https://md.research.elamur.ai/free/en/authors/weiyang-liu/), [Yiming Yang](https://md.research.elamur.ai/free/en/authors/yiming-yang/), [Sean Welleck](https://md.research.elamur.ai/free/en/authors/sean-welleck/) and [Chuang Gan](https://md.research.elamur.ai/free/en/authors/chuang-gan/) · **Year:** 2024 · **Venue:** [arXiv:2403.09472](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/) ([NeurIPS 2024](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/2024/)) · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2403.09472) · [PDF](https://arxiv.org/pdf/2403.09472) · [DOI 10.48550/arxiv.2403.09472](https://doi.org/10.48550/arxiv.2403.09472) · [arXiv 2403.09472](https://arxiv.org/abs/2403.09472) · [OpenAlex](https://openalex.org/W4415797700)

**Citations:** 2 citations (OpenAlex, 2026-10-14) · [49](https://md.research.elamur.ai/free/en/papers/sun-2024-easy-hard-generalization-scalable-alignment/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Current alignment methods learn from human demonstrations or human judgments, so a model's skill may be capped at what people can supervise. This paper asks whether supervision limited to easier problems can still produce models that solve harder ones. The authors call this setting easy-to-hard generalization. Using the MATH benchmark, they treat levels 1–3 as supervised "easy" data and levels 4–5 as "hard" problems given only as questions.

They find that generators trained with SFT or ICL on easy problems transfer poorly to hard ones. Evaluators transfer much better. These are outcome and process reward models (ORMs, PRMs, plus a combined OPRM) trained on easy-level data from PRM800K, MetaMath and Math-Shepherd. The evaluators can then improve the generator in two ways: by re-ranking sampled solutions, or by serving as the reward in RL (ReST-EM, DPO, PPO).

With this setup, a 7b policy trained by RL slightly beats a prior state-of-the-art model that used supervision on all levels. A similar effect appears for code on APPS. The work points to a practical route for improving models past the limits of human supervision.

## Key points
- Easy/hard split on MATH: levels 1–3 are supervised, levels 4–5 are unsupervised. The level distribution is roughly 1:2:2:3:3, so the two halves are about equal in size. Evaluation uses MATH500.
- Main results on MATH500, using human supervision only on easy problems:
  - A process-supervised 7b RL model reaches 34.0%, versus 33.0% for the previous best model trained on all levels.
  - A 34b model with re-ranking over 1024 samples reaches 52.5%.
- Comparing RL algorithms:
  - PPO performs best on PRM800K.
  - DPO matches PPO when the SFT starting point is strong (MetaMath).
  - ReST-EM gives only small gains.
  - For hard tasks, PRM rewards help more than rewards based only on final answers.
- OPRM is a new reward model that judges both individual reasoning steps and the final answer.
- Coding transfer on APPS (test set of 500 sampled problems):
  - The ORM is trained only on Introductory problems, yet it improves best-of-N on Interview and Competition problems.
  - For Code Llama-34B, overall strict accuracy rises from 7.8% (greedy decoding, SFT on all data) to 13.0% (best-of-16 with the easy-trained ORM).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,959 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,789 works

## Cite

Cite as: Zhiqing Sun, Longhui Yu, Yikang Shen, Weiyang Liu, Yiming Yang, Sean Welleck and Chuang Gan. 2024. Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervision. arXiv:2403.09472.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{sun2024easy,
  title = {{Easy-to-Hard Generalization: Scalable Alignment Beyond Human Supervision}},
  author = {Zhiqing Sun, Longhui Yu, Yikang Shen, Weiyang Liu, Yiming Yang, Sean Welleck and Chuang Gan},
  year = {2024},
  doi = {10.48550/arxiv.2403.09472},
  eprint = {2403.09472},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2403.09472}
}
```

- [← previous Arithmetic Without Algorithms: Language Models Solve Math With a Bag of Heuristics](https://md.research.elamur.ai/free/en/papers/nikankin-2024-arithmetic-algorithms-language-models-solve/)
- [next → FanOutQA: A Multi-Hop, Multi-Document Question Answering Benchmark for Large Language Models](https://md.research.elamur.ai/free/en/papers/zhu-2024-fanoutqa-multi-hop-multi-document/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/8/)

## References

97 references, 68 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. [Concrete problems in ai safety](https://md.research.elamur.ai/free/en/papers/amodei-2016-concrete-problems-ai-safety/). arXiv preprint arXiv:1606.06565, 2016. 3
2. Shengnan An, Zexiong Ma, Zeqi Lin, Nanning Zheng, Jian-Guang Lou, and Weizhu Chen. [Learning from mistakes makes llm better reasoner](https://md.research.elamur.ai/free/en/papers/an-2023-learning-mistakes-makes-llm-better/). arXiv preprint arXiv:2310.20689, 2023. 18
3. Jacob Andreas. [Language models as agent models](https://md.research.elamur.ai/free/en/papers/andreas-2022-language-models-as-agent-models/). In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 5769–5779, 2022. 3
4. Thomas Anthony, Zheng Tian, and David Barber. [Thinking fast and slow with deep learning and tree search](https://md.research.elamur.ai/free/en/papers/anthony-2017-thinking-fast-slow-deep-learning/). Advances in neural information processing systems, 30, 2017. 18
5. Mohammad Gheshlaghi Azar, Mark Rowland, Bilal Piot, Daniel Guo, Daniele Calandriello, Michal Valko, and Rémi Munos. [A general theoretical paradigm to understand learning from human preferences](https://md.research.elamur.ai/free/en/papers/azar-2023-general-theoretical-paradigm-understand-learning/). arXiv preprint arXiv:2310.12036, 2023. 18
6. Zhangir Azerbayev, Hailey Schoelkopf, Keiran Paster, Marco Dos Santos, Stephen McAleer, Albert Q Jiang, Jia Deng, Stella Biderman, and Sean Welleck. [Llemma: An open language model for mathematics](https://md.research.elamur.ai/free/en/papers/azerbayev-2023-llemma-open-language-model-mathematics/). arXiv preprint arXiv:2310.10631, 2023. 6, 8, 18, 24
7. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022. 18 [find](https://scholar.google.com/scholar?q=Yuntao+Bai%2C+Andy+Jones%2C+Kamal+Ndousse%2C+Amanda+Askell%2C+Anna+Chen%2C+Nova+DasSarma%2C+Dawn+Drain%2C+Stanislav+Fort%2C+Deep+Ganguli%2C+Tom+Henighan%2C+et+al%2E+Training+a+helpful+and+harmless+assistant+with+reinforcem)
8. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jamie Kerr, Jared… — [Constitutional AI: Harmlessness from AI Feedback](https://md.research.elamur.ai/free/en/papers/bai-2022-constitutional-ai-harmlessness-ai-feedback/)
9. Samuel R Bowman, Jeeyoon Hyun, Ethan Perez, Edwin Chen, Craig Pettit, Scott Heiner, Kamile Lukosuite, Amanda Askell, Andy Jones, Anna Chen, et al. [Measuring progress on scalable oversight for large language models](https://md.research.elamur.ai/free/en/papers/bowman-2022-measuring-progress-scalable-oversight-large/). arXiv preprint arXiv:2211.03540, 2022. 1, 2, 3
10. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D. Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in Neural Information Processing Systems, 33:1877–1901, 2020. 2, 4
11. Collin Burns, Pavel Izmailov, Jan Hendrik Kirchner, Bowen Baker, Leo Gao, Leopold Aschenbrenner, Yining Chen, Adrien Ecoffet, Manas Joglekar, Jan Leike, et al. [Weak-to-strong generalization: Eliciting strong capabilities with weak supervision](https://md.research.elamur.ai/free/en/papers/burns-2023-weak-strong-generalization-eliciting-strong/). arXiv preprint arXiv:2312.09390, 2023. 2, 3, 5, 6
12. James A Carlson, Arthur Jaffe, and Andrew Wiles. The millennium prize problems. American Mathematical Soc., 2006. 2 [find](https://scholar.google.com/scholar?q=James+A+Carlson%2C+Arthur+Jaffe%2C+and+Andrew+Wiles%2E+The+millennium+prize+problems%2E+American+Mathematical+Soc%2E%2C+2006%2E+2)
13. Noam Chomsky. On the representation of form and function. 1981. 4 [find](https://scholar.google.com/scholar?q=Noam+Chomsky%2E+On+the+representation+of+form+and+function%2E+1981%2E+4)
14. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, Sharan… [find](https://scholar.google.com/scholar?q=Hyung+Won+Chung%2C+Le+Hou%2C+Shayne+Longpre%2C+Barret+Zoph%2C+Yi+Tay%2C+William+Fedus%2C+Yunxuan+Li%2C+Xuezhi+Wang%2C+Mostafa+Dehghani%2C+Siddhartha+Brahma%2C+Albert+Webson%2C+Shixiang+Shane+Gu%2C+Zhuyun+Dai%2C+Mirac+Suzgun%2C+X)
15. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. [Think you have solved question answering? try arc, the ai2 reasoning challenge](https://md.research.elamur.ai/free/en/papers/clark-2018-think-have-solved-question-answering/). arXiv preprint arXiv:1803.05457, 2018. 4
16. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168, 2021. 4, 6, 18
17. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168, 2021. 4
18. Ajeya Cotra. The case for aligning narrowly superhuman models. In AI Alignment Forum, 2021. 3 [find](https://scholar.google.com/scholar?q=Ajeya+Cotra%2E+The+case+for+aligning+narrowly+superhuman+models%2E+In+AI+Alignment+Forum%2C+2021%2E+3)
19. Jesse Dodge, Gabriel Ilharco, Roy Schwartz, Ali Farhadi, Hannaneh Hajishirzi, and Noah Smith. Fine-tuning pretrained language models: Weight initializations, data orders, and early stopping. arXiv preprint arXiv:2002.06305, 2020. 7 [find](https://scholar.google.com/scholar?q=Jesse+Dodge%2C+Gabriel+Ilharco%2C+Roy+Schwartz%2C+Ali+Farhadi%2C+Hannaneh+Hajishirzi%2C+and+Noah+Smith%2E+Fine%2Dtuning+pretrained+language+models%3A+Weight+initializations%2C+data+orders%2C+and+early+stopping%2E+arXiv+pre)
20. Andrew Drozdov, Nathanael Schärli, Ekin Akyürek, Nathan Scales, Xinying Song, Xinyun Chen, Olivier Bousquet, and Denny Zhou. Compositional semantic parsing with large language models. In The Eleventh International Conference on Learning Representations, 2022. 4 [find](https://scholar.google.com/scholar?q=Andrew+Drozdov%2C+Nathanael+Sch%C3%A4rli%2C+Ekin+Aky%C3%BCrek%2C+Nathan+Scales%2C+Xinying+Song%2C+Xinyun+Chen%2C+Olivier+Bousquet%2C+and+Denny+Zhou%2E+Compositional+semantic+parsing+with+large+language+models%2E+In+The+Eleventh+)
21. Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. [DROP: A reading comprehension benchmark requiring discrete reasoning over paragraphs](https://md.research.elamur.ai/free/en/papers/dua-2019-drop-reading-comprehension-benchmark-requiring/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Hum…
22. Yann Dubois, Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. [Alpacafarm: A simulation framework for methods that learn from human feedback](https://md.research.elamur.ai/free/en/papers/dubois-2023-alpacafarm-simulation-framework-methods-that/). arXiv preprint arXiv:2305.14387, 2023. 20
23. Jerry A Fodor and Ernest Lepore. The compositionality papers. Oxford University Press, 2002. 4 [find](https://scholar.google.com/scholar?q=Jerry+A+Fodor+and+Ernest+Lepore%2E+The+compositionality+papers%2E+Oxford+University+Press%2C+2002%2E+4)
24. Yao Fu, Hao Peng, Ashish Sabharwal, Peter Clark, and Tushar Khot. [Complexity-based prompting for multi-step reasoning](https://md.research.elamur.ai/free/en/papers/fu-2022-complexity-based-prompting-multi-step/). In The Eleventh International Conference on Learning Representations, 2022. 2, 4, 5, 18
25. Leo Gao, John Schulman, and Jacob Hilton. [Scaling laws for reward model overoptimization](https://md.research.elamur.ai/free/en/papers/gao-2022-scaling-laws-reward-model-overoptimization/). In International Conference on Machine Learning, pages 10835–10866. PMLR, 2023. 8
26. Zhibin Gou, Zhihong Shao, Yeyun Gong, Yujiu Yang, Minlie Huang, Nan Duan, Weizhu Chen, et al. [Tora: A tool-integrated reasoning agent for mathematical problem solving](https://md.research.elamur.ai/free/en/papers/gou-2023-tora-tool-integrated-reasoning-agent/). arXiv preprint arXiv:2309.17452, 2023. 18
27. Arnav Gudibande, Eric Wallace, Charlie Snell, Xinyang Geng, Hao Liu, Pieter Abbeel, Sergey Levine, and Dawn Song. [The false promise of imitating proprietary llms](https://md.research.elamur.ai/free/en/papers/gudibande-2023-false-promise-imitating-proprietary-llms/). arXiv preprint arXiv:2305.15717, 2023. 7
28. Caglar Gulcehre, Tom Le Paine, Srivatsan Srinivasan, Ksenia Konyushkova, Lotte Weerts, Abhishek Sharma, Aditya Siddhant, Alex Ahern, Miaosen Wang, Chenjie Gu, et al. [Reinforced self-training (rest) for language modeling](https://md.research.elamur.ai/free/en/papers/gulcehre-2023-reinforced-self-training-rest-language/). arXiv preprint arXiv:2308.08998, 2023. 6, 18
29. Peter Hase, Mohit Bansal, Peter Clark, and Sarah Wiegreffe. [The unreasonable effectiveness of easy training data for hard tasks](https://md.research.elamur.ai/free/en/papers/hase-2024-unreasonable-effectiveness-easy-training-data/). arXiv preprint arXiv:2401.06751, 2024. 2, 4, 5, 6
30. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). In International Conference on Learning Representations, 2020. 4
31. Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, et al. [Measuring coding challenge competence with apps](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-coding-challenge-competence-apps/). arXiv preprint arXiv:2105.09938, 2021. 10
32. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. [Measuring mathematical problem solving with the math dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). arXiv preprint arXiv:2103.03874, 2021. 4, 5, 8, 18, 24
33. Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. [Swe-bench: Can language models resolve real-world github issues?](https://md.research.elamur.ai/free/en/papers/jimenez-2024-swe-bench-can-language-models/) arXiv preprint arXiv:2310.06770, 2023. 18
34. Richard M Karp. On the computational complexity of combinatorial problems. Networks, 5(1): 45–68, 1975. 2 [find](https://scholar.google.com/scholar?q=Richard+M+Karp%2E+On+the+computational+complexity+of+combinatorial+problems%2E+Networks%2C+5%281%29%3A+45%E2%80%9368%2C+1975%2E+2)
35. Daniel Keysers, Nathanael Schärli, Nathan Scales, Hylke Buisman, Daniel Furrer, Sergii Kashubin, Nikola Momchev, Danila Sinopalnikov, Lukasz Stafiniak, Tibor Tihon, et al. Measuring compositional generalization: A comprehensive method on realistic data. In International Conference on Learning Repres… [find](https://scholar.google.com/scholar?q=Daniel+Keysers%2C+Nathanael+Sch%C3%A4rli%2C+Nathan+Scales%2C+Hylke+Buisman%2C+Daniel+Furrer%2C+Sergii+Kashubin%2C+Nikola+Momchev%2C+Danila+Sinopalnikov%2C+Lukasz+Stafiniak%2C+Tibor+Tihon%2C+et+al%2E+Measuring+compositional+gene)
36. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. [Large language models are zero-shot reasoners](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/). arXiv preprint arXiv:2205.11916, 2022. 18
37. Brenden Lake and Marco Baroni. Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks. In International conference on machine learning, pages 2873–2882. PMLR, 2018. 4 [find](https://scholar.google.com/scholar?q=Brenden+Lake+and+Marco+Baroni%2E+Generalization+without+systematicity%3A+On+the+compositional+skills+of+sequence%2Dto%2Dsequence+recurrent+networks%2E+In+International+conference+on+machine+learning%2C+pages+2873)
38. Jan Leike, David Krueger, Tom Everitt, Miljan Martic, Vishal Maini, and Shane Legg. [Scalable agent alignment via reward modeling: a research direction](https://md.research.elamur.ai/free/en/papers/leike-2018-scalable-agent-alignment-reward-modeling/). arXiv preprint arXiv:1811.07871, 2018. 3
39. Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, et al. [Solving quantitative reasoning problems with language models](https://md.research.elamur.ai/free/en/papers/lewkowycz-2022-solving-quantitative-reasoning-problems-language/). arXiv preprint arXiv:2206.14858, 2022. 5, 18
40. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. arXiv preprint arXiv:2305.[20050, 2023](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/). 2, 5, 6, 8, 10, 18, 19, 30
41. Wang Ling, Dani Yogatama, Chris Dyer, and Phil Blunsom. Program induction by rationale generation: Learning to solve and explain algebraic word problems. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 158–167, 2017. 18 [find](https://scholar.google.com/scholar?q=Wang+Ling%2C+Dani+Yogatama%2C+Chris+Dyer%2C+and+Phil+Blunsom%2E+Program+induction+by+rationale+generation%3A+Learning+to+solve+and+explain+algebraic+word+problems%2E+In+Proceedings+of+the+55th+Annual+Meeting+of+t)
42. Bingbin Liu, Sebastien Bubeck, Ronen Eldan, Janardhan Kulkarni, Yuanzhi Li, Anh Nguyen, Rachel Ward, and Yi Zhang. Tinygsm: achieving\> 80% on gsm8k with small language models. arXiv preprint arXiv:2312.[09241, 2023](https://md.research.elamur.ai/free/en/papers/liu-2023-tinygsm-achieving-80-gsm8k-small/). 18
43. Weiyang Liu, Bo Dai, Ahmad Humayun, Charlene Tay, Chen Yu, Linda B Smith, James M Rehg, and Le Song. Iterative machine teaching. In International Conference on Machine Learning, pages 2149–2158. PMLR, 2017. 4 [find](https://scholar.google.com/scholar?q=Weiyang+Liu%2C+Bo+Dai%2C+Ahmad+Humayun%2C+Charlene+Tay%2C+Chen+Yu%2C+Linda+B+Smith%2C+James+M+Rehg%2C+and+Le+Song%2E+Iterative+machine+teaching%2E+In+International+Conference+on+Machine+Learning%2C+pages+2149%E2%80%932158%2E+PMLR%2C)
44. Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang. [Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct](https://md.research.elamur.ai/free/en/papers/luo-2023-wizardmath-empowering-mathematical-reasoning-large/). arXiv preprint arXiv:2308.09583, 2023. 18
45. Rémi Munos, Michal Valko, Daniele Calandriello, Mohammad Gheshlaghi Azar, Mark Rowland, Zhaohan Daniel Guo, Yunhao Tang, Matthieu Geist, Thomas Mesnard, Andrea Michi, et al. [Nash learning from human feedback](https://md.research.elamur.ai/free/en/papers/munos-2023-nash-learning-human-feedback/). arXiv preprint arXiv:2312.00886, 2023. 18
46. Moni Naor. Evaluation may be easier than generation. In Proceedings of the twenty-eighth annual ACM symposium on Theory of computing, pages 74–83, 1996. 2 [find](https://scholar.google.com/scholar?q=Moni+Naor%2E+Evaluation+may+be+easier+than+generation%2E+In+Proceedings+of+the+twenty%2Deighth+annual+ACM+symposium+on+Theory+of+computing%2C+pages+74%E2%80%9383%2C+1996%2E+2)
47. OpenAI. OpenAI: Introducing ChatGPT, 2022. URL [https://openai.com/blog/chatgpt](https://openai.com/blog/chatgpt). 1, 5 [↗](https://openai.com/blog/chatgpt)
48. OpenAI. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/), 2023. 5, 18
49. OpenAI. OpenAI: GPT-4, 2023. URL [https://openai.com/research/gpt-4](https://openai.com/research/gpt-4). 1 [↗](https://openai.com/research/gpt-4)
50. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). arXiv preprint arXiv:2203.02155, 2022. 1, 2, 5, 7, 9, 18, 20
51. Ethan Perez, Patrick Lewis, Wen-tau Yih, Kyunghyun Cho, and Douwe Kiela. [Unsupervised question decomposition for question answering](https://md.research.elamur.ai/free/en/papers/perez-2020-unsupervised-question-decomposition-question-answering/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8864–8880, 2020. 4
52. Ethan Perez, Douwe Kiela, and Kyunghyun Cho. [True few-shot learning with language models](https://md.research.elamur.ai/free/en/papers/perez-2021-true-few-shot-learning-language/). Advances in neural information processing systems, 34:11054–11070, 2021. 5
53. Ethan Perez, Sam Ringer, Kamilė Lukošiūtė, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, et al. [Discovering language model behaviors with model-written evaluations](https://md.research.elamur.ai/free/en/papers/perez-2022-discovering-language-model-behaviors-model/). arXiv preprint arXiv:2212.09251, 2022. 3
54. Stanislas Polu and Ilya Sutskever. [Generative language modeling for automated theorem proving](https://md.research.elamur.ai/free/en/papers/polu-2020-generative-language-modeling-automated-theorem/). arXiv preprint arXiv:2009.03393, 2020. 18
55. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language understanding by generative pre-training. 2018. 2, 4 [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Karthik+Narasimhan%2C+Tim+Salimans%2C+and+Ilya+Sutskever%2E+Improving+language+understanding+by+generative+pre%2Dtraining%2E+2018%2E+2%2C+4)
56. Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D Manning, and Chelsea Finn. [Direct preference optimization: Your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/). arXiv preprint arXiv:2305.18290, 2023. 6, 9, 18, 20
57. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. [Gpqa: A graduate-level google-proof q&a benchmark](https://md.research.elamur.ai/free/en/papers/rein-2023-gpqa-graduate-level-google-proof/). arXiv preprint arXiv:2311.12022, 2023. 2, 3
58. Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, Jérémy Rapin, et al. [Code llama: Open foundation models for code](https://md.research.elamur.ai/free/en/papers/roziere-2024-code-llama-open-foundation-models/). arXiv preprint arXiv:2308.12950, 2023. 6, 10
59. Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, et al. [Multitask prompted training enables zero-shot task generalization](https://md.research.elamur.ai/free/en/papers/sanh-2021-multitask-prompted-training-enables-zero/). In International Conference on Learning Representations, 2021. 1, 7
60. William Saunders, Catherine Yeh, Jeff Wu, Steven Bills, Long Ouyang, Jonathan Ward, and Jan Leike. [Self-critiquing models for assisting human evaluators](https://md.research.elamur.ai/free/en/papers/saunders-2022-self-critiquing-models-assisting-human/). arXiv preprint arXiv:2206.05802, 2022. 2, 3
61. John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel. High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.02438, 2015. 20 [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Philipp+Moritz%2C+Sergey+Levine%2C+Michael+Jordan%2C+and+Pieter+Abbeel%2E+High%2Ddimensional+continuous+control+using+generalized+advantage+estimation%2E+arXiv+preprint+arXiv%3A1506%2E02438%2C+2015%2E+20)
62. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017. 6, 18 [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Filip+Wolski%2C+Prafulla+Dhariwal%2C+Alec+Radford%2C+and+Oleg+Klimov%2E+Proximal+policy+optimization+algorithms%2E+arXiv+preprint+arXiv%3A1707%2E06347%2C+2017%2E+6%2C+18)
63. Avi Schwarzschild, Eitan Borgnia, Arjun Gupta, Furong Huang, Uzi Vishkin, Micah Goldblum, and Tom Goldstein. Can you learn an algorithm? generalizing from easy to hard problems with recurrent networks. Advances in Neural Information Processing Systems, 34:6695–6706, 2021. 2 [find](https://scholar.google.com/scholar?q=Avi+Schwarzschild%2C+Eitan+Borgnia%2C+Arjun+Gupta%2C+Furong+Huang%2C+Uzi+Vishkin%2C+Micah+Goldblum%2C+and+Tom+Goldstein%2E+Can+you+learn+an+algorithm%3F+generalizing+from+easy+to+hard+problems+with+recurrent+networks)
64. Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R Bowman, Newton Cheng, Esin Durmus, Zac Hatfield-Dodds, Scott R Johnston, et al. [Towards understanding sycophancy in language models](https://md.research.elamur.ai/free/en/papers/sharma-2023-towards-understanding-sycophancy-language-models/). arXiv preprint arXiv:2310.13548, 2023. 1, 3
65. David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587):484–489, 2016. 6 [find](https://scholar.google.com/scholar?q=David+Silver%2C+Aja+Huang%2C+Chris+J+Maddison%2C+Arthur+Guez%2C+Laurent+Sifre%2C+George+Van+Den+Driessche%2C+Julian+Schrittwieser%2C+Ioannis+Antonoglou%2C+Veda+Panneershelvam%2C+Marc+Lanctot%2C+et+al%2E+Mastering+the+game+)
66. David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, et al. Mastering the game of go without human knowledge. nature, 550(7676):354–359, 2017. 18 [find](https://scholar.google.com/scholar?q=David+Silver%2C+Julian+Schrittwieser%2C+Karen+Simonyan%2C+Ioannis+Antonoglou%2C+Aja+Huang%2C+Arthur+Guez%2C+Thomas+Hubert%2C+Lucas+Baker%2C+Matthew+Lai%2C+Adrian+Bolton%2C+et+al%2E+Mastering+the+game+of+go+without+human+kn)
67. Avi Singh, John D Co-Reyes, Rishabh Agarwal, Ankesh Anand, Piyush Patil, Peter J Liu, James Harrison, Jaehoon Lee, Kelvin Xu, Aaron Parisi, et al. [Beyond human data: Scaling self-training for problem-solving with language models](https://md.research.elamur.ai/free/en/papers/singh-2023-human-data-scaling-self-training/). arXiv preprint arXiv:2312.06585, 2023. 6, 18, 19, 20
68. Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 33:3008–3021, 2020. 1, 5, 7, 18 [find](https://scholar.google.com/scholar?q=Nisan+Stiennon%2C+Long+Ouyang%2C+Jeffrey+Wu%2C+Daniel+Ziegler%2C+Ryan+Lowe%2C+Chelsea+Voss%2C+Alec+Radford%2C+Dario+Amodei%2C+and+Paul+F+Christiano%2E+Learning+to+summarize+with+human+feedback%2E+Advances+in+Neural+Infor)
69. Zhiqing Sun, Yikang Shen, Hongxin Zhang, Qinhong Zhou, Zhenfang Chen, David Cox, Yiming Yang, and Chuang Gan. Salmon: Self-alignment with principle-following reward models. arXiv preprint arXiv:2310.[05910, 2023](https://md.research.elamur.ai/free/en/papers/sun-2023-salmon-self-alignment-instructable-reward/). 18
70. Zhiqing Sun, Yikang Shen, Qinhong Zhou, Hongxin Zhang, Zhenfang Chen, David Cox, Yiming Yang, and Chuang Gan. [Principle-driven self-alignment of language models from scratch with minimal human supervision](https://md.research.elamur.ai/free/en/papers/sun-2023-principle-driven-self-alignment-language/). arXiv preprint arXiv:2305.03047, 2023. 7
71. Swabha Swayamdipta, Roy Schwartz, Nicholas Lourie, Yizhong Wang, Hannaneh Hajishirzi, Noah A Smith, and Yejin Choi. Dataset cartography: Mapping and diagnosing datasets with training dynamics. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 92… [find](https://scholar.google.com/scholar?q=Swabha+Swayamdipta%2C+Roy+Schwartz%2C+Nicholas+Lourie%2C+Yizhong+Wang%2C+Hannaneh+Hajishirzi%2C+Noah+A+Smith%2C+and+Yejin+Choi%2E+Dataset+cartography%3A+Mapping+and+diagnosing+datasets+with+training+dynamics%2E+In+Proc)
72. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. [Llama 2: Open foundation and fine-tuned chat models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/). arXiv preprint arXiv:2307.09288, 2023. 6, 18
73. Trieu H Trinh, Yuhuai Wu, Quoc V Le, He He, and Thang Luong. [Solving olympiad geometry without human demonstrations](https://md.research.elamur.ai/free/en/papers/trinh-2024-solving-olympiad-geometry-without-human/). Nature, 625(7995):476–482, 2024. 2
74. Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. [Solving math word problems with process- and outcome-based feedback](https://md.research.elamur.ai/free/en/papers/uesato-2022-solving-math-word-problems-process/). arXiv preprint arXiv:2211.14275, 2022. 2, 6, 7, 8, 18, 29
75. Peiyi Wang, Lei Li, Zhihong Shao, RX Xu, Damai Dai, Yifei Li, Deli Chen, Y Wu, and Zhifang Sui. [Math-shepherd: Verify and reinforce llms step-by-step without human annotations](https://md.research.elamur.ai/free/en/papers/wang-2024-math-shepherd-verify-reinforce-llms/). CoRR, abs/2312.08935, 2023. 2, 3, 5, 6, 9, 19, 20, 26
76. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language model with self generated instructions. arXiv preprint arXiv:2212.10560, 2022. 5, 6, 7 [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Yeganeh+Kordi%2C+Swaroop+Mishra%2C+Alisa+Liu%2C+Noah+A+Smith%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+Self%2Dinstruct%3A+Aligning+language+model+with+self+generated+instructions%2E+arXiv+preprint+)
77. Zihan Wang, Yunxuan Li, Yuexin Wu, Liangchen Luo, Le Hou, Hongkun Yu, and Jingbo Shang. [Multi-step problem solving through a verifier: An empirical analysis on model-induced process supervision](https://md.research.elamur.ai/free/en/papers/wang-2024-multi-step-problem-solving-through/). arXiv preprint arXiv:2402.02658, 2024. 30
78. Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. [Finetuned language models are zero-shot learners](https://md.research.elamur.ai/free/en/papers/wei-2021-finetuned-language-models-zero-shot/). In International Conference on Learning Representations, 2021. 1, 7
79. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). NeurIPS, 2022. 5, 18
80. Jerry Wei, Da Huang, Yifeng Lu, Denny Zhou, and Quoc V Le. [Simple synthetic data reduces sycophancy in large language models](https://md.research.elamur.ai/free/en/papers/wei-2023-simple-synthetic-data-reduces-sycophancy/). arXiv preprint arXiv:2308.03958, 2023. 3
81. Jeff Wu, Long Ouyang, Daniel M Ziegler, Nisan Stiennon, Ryan Lowe, Jan Leike, and Paul Christiano. [Recursively summarizing books with human feedback](https://md.research.elamur.ai/free/en/papers/wu-2021-recursively-summarizing-books-human-feedback/). arXiv preprint arXiv:2109.10862, 2021. 2
82. Jing Xu, Andrew Lee, Sainbayar Sukhbaatar, and Jason Weston. Some things are more cringe than others: Preference optimization with the pairwise cringe loss. arXiv preprint arXiv:2312.16682, 2023. 18 — [arXiv:2312.16682](https://arxiv.org/abs/2312.16682) [find](https://scholar.google.com/scholar?q=Jing+Xu%2C+Andrew+Lee%2C+Sainbayar+Sukhbaatar%2C+and+Jason+Weston%2E+Some+things+are+more+cringe+than+others%3A+Preference+optimization+with+the+pairwise+cringe+loss%2E+arXiv+preprint+arXiv%3A2312%2E16682%2C+2023%2E+18+%E2%80%94)
83. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. [React: Synergizing reasoning and acting in language models](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/). arXiv preprint arXiv:2210.03629, 2022. 18
84. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. arXiv preprint arXiv:2305.10601, 2023. 18 [find](https://scholar.google.com/scholar?q=Shunyu+Yao%2C+Dian+Yu%2C+Jeffrey+Zhao%2C+Izhak+Shafran%2C+Thomas+L+Griffiths%2C+Yuan+Cao%2C+and+Karthik+Narasimhan%2E+Tree+of+thoughts%3A+Deliberate+problem+solving+with+large+language+models%2E+arXiv+preprint+arXiv%3A23)
85. Fei Yu, Anningzhe Gao, and Benyou Wang. Outcome-supervised verifiers for planning in mathematical reasoning. arXiv preprint arXiv:2311.[09724, 2023](https://md.research.elamur.ai/free/en/papers/yu-2023-ovm-outcome-supervised-value-models/). 2, 6
86. Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. [Metamath: Bootstrap your own mathematical questions for large language models](https://md.research.elamur.ai/free/en/papers/yu-2023-metamath-bootstrap-your-own-mathematical/). arXiv preprint arXiv:2309.12284, 2023. 3, 5, 18, 19, 20, 26
87. Zheng Yuan, Hongyi Yuan, Chengpeng Li, Guanting Dong, Chuanqi Tan, and Chang Zhou. [Scaling relationship on learning mathematical reasoning with large language models](https://md.research.elamur.ai/free/en/papers/yuan-2023-scaling-relationship-learning-mathematical-reasoning/). arXiv preprint arXiv:2308.01825, 2023. 18
88. Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. [Mammoth: Building math generalist models through hybrid instruction tuning](https://md.research.elamur.ai/free/en/papers/yue-2023-mammoth-building-math-generalist-models/). arXiv preprint arXiv:2309.05653, 2023. 18
89. Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. [Star: Bootstrapping reasoning with reasoning](https://md.research.elamur.ai/free/en/papers/zelikman-2022-star-bootstrapping-reasoning-reasoning/). Advances in Neural Information Processing Systems, 35:15476–15488, 2022. 18, 19
90. Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. [Automatic chain of thought prompting in large language models](https://md.research.elamur.ai/free/en/papers/zhang-2022-automatic-chain-thought-prompting-large/). arXiv preprint arXiv:2210.03493, 2022. 18
91. Yao Zhao, Mikhail Khalman, Rishabh Joshi, Shashi Narayan, Mohammad Saleh, and Peter J Liu. Calibrating sequence likelihood improves conditional language generation. In The Eleventh International Conference on Learning Representations, 2022. 18 [find](https://scholar.google.com/scholar?q=Yao+Zhao%2C+Mikhail+Khalman%2C+Rishabh+Joshi%2C+Shashi+Narayan%2C+Mohammad+Saleh%2C+and+Peter+J+Liu%2E+Calibrating+sequence+likelihood+improves+conditional+language+generation%2E+In+The+Eleventh+International+Confe)
92. Yao Zhao, Rishabh Joshi, Tianqi Liu, Misha Khalman, Mohammad Saleh, and Peter J Liu. [Slic-hf: Sequence likelihood calibration with human feedback](https://md.research.elamur.ai/free/en/papers/zhao-2023-slic-hf-sequence-likelihood-calibration/). arXiv preprint arXiv:2305.10425, 2023. 18
93. Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. [Calibrate before use: Improving few-shot performance of language models](https://md.research.elamur.ai/free/en/papers/zhao-2021-calibrate-before-use-improving-few/). In International Conference on Machine Learning, pages 12697–12706. PMLR, 2021. 7
94. Chuanyang Zheng, Zhengying Liu, Enze Xie, Zhenguo Li, and Yu Li. [Progressive-hint prompting improves reasoning in large language models](https://md.research.elamur.ai/free/en/papers/zheng-2023-progressive-hint-prompting-improves-reasoning/). arXiv preprint arXiv:2304.09797, 2023. 18
95. Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc V Le, et al. [Least-to-most prompting enables complex reasoning in large language models](https://md.research.elamur.ai/free/en/papers/al-2022-least-most-prompting-enables-complex/). In The Eleventh International Conference on Learning Representations, 2022. 2, 4,…
96. Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. [Large language models are human-level prompt engineers](https://md.research.elamur.ai/free/en/papers/zhou-2022-large-language-models-human-level/). arXiv preprint arXiv:2211.01910, 2022. 18
97. Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593, 2019. 1 [find](https://scholar.google.com/scholar?q=Daniel+M+Ziegler%2C+Nisan+Stiennon%2C+Jeffrey+Wu%2C+Tom+B+Brown%2C+Alec+Radford%2C+Dario+Amodei%2C+Paul+Christiano%2C+and+Geoffrey+Irving%2E+Fine%2Dtuning+language+models+from+human+preferences%2E+arXiv+preprint+arXiv%3A19)

*Updated 3 November 2026*