---
title: "Learning From Mistakes Makes LLM Better Reasoner — An 2023"
description: "This paper proposes LEMA (LEarning from MistAkes), which borrows the way students learn from… 70 references, cited by 59 papers here."
authors: ["Shengnan An", "Zexiong Ma", "Zeqi Lin", "Nanning Zheng", "Jian-Guang Lou", "Weizhu Chen"]
year: 2023
venue: "arXiv:2310.20689"
source: "https://arxiv.org/abs/2310.20689"
doi: "10.48550/arxiv.2310.20689"
arxiv: "2310.20689"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/an-2023-learning-mistakes-makes-llm-better/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/an-2023-learning-mistakes-makes-llm-better/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/10/) / Learning From Mistakes Makes LLM Better Reasoner

# Learning From Mistakes Makes LLM Better Reasoner

**Authors:** [Shengnan An](https://md.research.elamur.ai/free/en/authors/shengnan-an/), [Zexiong Ma](https://md.research.elamur.ai/free/en/authors/zexiong-ma/), [Zeqi Lin](https://md.research.elamur.ai/free/en/authors/zeqi-lin/), [Nanning Zheng](https://md.research.elamur.ai/free/en/authors/nanning-zheng/), [Jian-Guang Lou](https://md.research.elamur.ai/free/en/authors/jian-guang-lou/) and [Weizhu Chen](https://md.research.elamur.ai/free/en/authors/weizhu-chen/) · **Year:** 2023 · **Venue:** arXiv:2310.20689 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2310.20689) · [PDF](https://arxiv.org/pdf/2310.20689) · [DOI 10.48550/arxiv.2310.20689](https://doi.org/10.48550/arxiv.2310.20689) · [arXiv 2310.20689](https://arxiv.org/abs/2310.20689) · [OpenAlex](https://openalex.org/W4388185044)

**Citations:** 5 citations (OpenAlex, 2026-10-14) · [59](https://md.research.elamur.ai/free/en/papers/an-2023-learning-mistakes-makes-llm-better/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Open-source LLMs such as LLaMA-2 still lag behind proprietary systems on multi-step math reasoning. They are usually improved by fine-tuning on question–rationale pairs (CoT data). This paper proposes LEMA (LEarning from MistAkes), which borrows the way students learn from their own errors. Several LLaMA and GPT models produce reasoning paths, and the ones with wrong final answers are kept. GPT-4 then acts as a corrector. Guided by a few annotated examples, it marks the faulty step, explains why it is wrong, and rewrites the solution from that step. Only corrections that reach the gold answer are kept. To scale this data, the authors use a correction-centric evolution strategy, which generates new questions mainly from moderately difficult seeds. The correction data is mixed with CoT data for fine-tuning. Across five open-source LLMs and five reasoning benchmarks, LEMA consistently beats fine-tuning on CoT data alone, including for math-specialized models. The work shows that error-correction data is a useful training signal of its own, not just more of the same.

## Key points
- With LLaMA-2-70B, LEMA scores 83.5% on GSM8K and 25.0% on MATH, compared with 81.4% and 23.6% for CoT-only fine-tuning. Correction-centric evolution raises MATH further to 29.3%.
- The method also helps outside math: LLaMA-2-70B on CSQA improves from 84.2% to 85.3%. It additionally boosts the specialized models WizardMath and MetaMath.
- A manual check of 50 GPT-4 corrections on GSM8K problems rated 35 excellent, 11 good and 4 poor.
- Ablations with matched data size and token budget show that mixing CoT and correction data beats using either source alone. This suggests the two data types contribute in different ways.
- In the evolution experiments (10K seed questions per strategy), full fine-tuning keeps improving as the question set grows. QLoRA gains only slightly, which implies limited capacity to absorb large amounts of correction data.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/) — Company, Redmond, United States · 378 works · 161,630 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works

## Cite

Cite as: Shengnan An, Zexiong Ma, Zeqi Lin, Nanning Zheng, Jian-Guang Lou and Weizhu Chen. 2023. Learning From Mistakes Makes LLM Better Reasoner. arXiv:2310.20689.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{an2023learning,
  title = {{Learning From Mistakes Makes LLM Better Reasoner}},
  author = {Shengnan An, Zexiong Ma, Zeqi Lin, Nanning Zheng, Jian-Guang Lou and Weizhu Chen},
  year = {2023},
  doi = {10.48550/arxiv.2310.20689},
  eprint = {2310.20689},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2310.20689}
}
```

- [← previous Fairness-guided Few-shot Prompting for Large Language Models](https://md.research.elamur.ai/free/en/papers/ma-2023-fairness-guided-few-shot-prompting/)
- [next → Linear Representations of Sentiment in Large Language Models](https://md.research.elamur.ai/free/en/papers/tigges-2023-linear-representations-sentiment-large-language/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/10/)

## References

70 references, 43 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Chuyuan Fu, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, Daniel Ho, Jasmine Hsu, Julian Ibarz, Brian Ichter, Alex Irpan, Eric Jang, Rosario Jauregui Ruano, Kyle Jeffrey, Sally Jesmonth, Nikhil… — [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://md.research.elamur.ai/free/en/papers/ahn-2022-i-i-say-grounding-language/)
2. Alibaba. Alibaba open sources qwen, a 7b parameter ai model, 2023. [https://www.maginative.com/article/alibaba-open-sources-qwen-a-7b-\\](https://www.maginative.com/article/alibaba-open-sources-qwen-a-7b-\\) parameter-ai-model/. [↗](https://www.maginative.com/article/alibaba-open-sources-qwen-a-7b-\\)
3. Shengnan An, Yifei Li, Zeqi Lin, Qian Liu, Bei Chen, Qiang Fu, Weizhu Chen, Nanning Zheng, and Jian-Guang Lou. Input-tuning: Adapting unfamiliar inputs to frozen pre-trained models, 2022. [find](https://scholar.google.com/scholar?q=Shengnan+An%2C+Yifei+Li%2C+Zeqi+Lin%2C+Qian+Liu%2C+Bei+Chen%2C+Qiang+Fu%2C+Weizhu+Chen%2C+Nanning+Zheng%2C+and+Jian%2DGuang+Lou%2E+Input%2Dtuning%3A+Adapting+unfamiliar+inputs+to+frozen+pre%2Dtrained+models%2C+2022%2E)
4. Rohan Anil, Andrew M. Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, Eric Chu, Jonathan H. Clark, Laurent El Shafey, Yanping Huang, Kathy Meier-Hellstern, Gaurav Mishra, Erica Moreira, Mark Omernick, Kevin Robinson, Se… — [PaLM 2 Technical Report](https://md.research.elamur.ai/free/en/papers/anil-2023-palm-2-technical-report/)
5. Anthropic. Model card and evaluations for claude models, 2023. URL [https://www-files.anthropic.com/production/images/Model-Card-Claude-2.pdf](https://www-files.anthropic.com/production/images/Model-Card-Claude-2.pdf). [↗](https://www-files.anthropic.com/production/images/Model-Card-Claude-2.pdf)
6. Kourosh Hakhamaneshi Artur Niederfahrenhorst and Rehaan Ahmad. Fine-tuning llms: Lora or full-parameter? an in-depth analysis with llama 2, 2023. URL [https://www.anyscale.com/blog/fine-tuning-llms-lora-or-full-\\](https://www.anyscale.com/blog/fine-tuning-llms-lora-or-full-\\) parameter-an-in-depth-analysis-\\with-llama-2. [↗](https://www.anyscale.com/blog/fine-tuning-llms-lora-or-full-\\)
7. Zhangir Azerbayev, Hailey Schoelkopf, Keiran Paster, Marco Dos Santos, Stephen McAleer, Albert Q. Jiang, Jia Deng, Stella Biderman, and Sean Welleck. [Llemma: An open language model for mathematics](https://md.research.elamur.ai/free/en/papers/azerbayev-2023-llemma-open-language-model-mathematics/), 2023.
8. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems, 33:1877–1901, 2020.
9. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168, 2021.
10. Antonia Creswell and Murray Shanahan. [Faithful reasoning using large language models](https://md.research.elamur.ai/free/en/papers/creswell-2022-faithful-reasoning-using-large-language/). arXiv preprint arXiv:2208.14271, 2022.
11. Antonia Creswell, Murray Shanahan, and Irina Higgins. Selection-inference: Exploiting large language models for interpretable logical reasoning. In The Eleventh International Conference on Learning Representations, 2022. [find](https://scholar.google.com/scholar?q=Antonia+Creswell%2C+Murray+Shanahan%2C+and+Irina+Higgins%2E+Selection%2Dinference%3A+Exploiting+large+language+models+for+interpretable+logical+reasoning%2E+In+The+Eleventh+International+Conference+on+Learning+Re)
12. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. [Qlora: Efficient finetuning of quantized llms](https://md.research.elamur.ai/free/en/papers/dettmers-2023-qlora-efficient-finetuning-quantized-llms/). arXiv preprint arXiv:2305.14314, 2023.
13. Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies. Transactions of the Association for Computational Linguistics, 9:346–361, 04 2021. ISSN 2307-387X. doi: 10.1162/tacl a 003… [find](https://scholar.google.com/scholar?q=Mor+Geva%2C+Daniel+Khashabi%2C+Elad+Segal%2C+Tushar+Khot%2C+Dan+Roth%2C+and+Jonathan+Berant%2E+Did+Aristotle+Use+a+Laptop%3F+A+Question+Answering+Benchmark+with+Implicit+Reasoning+Strategies%2E+Transactions+of+the+As)
14. Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Minlie Huang, Nan Duan, and Weizhu Chen. [Tora: A tool-integrated reasoning agent for mathematical problem solving](https://md.research.elamur.ai/free/en/papers/gou-2023-tora-tool-integrated-reasoning-agent/), 2023.
15. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. [Measuring mathematical problem solving with the math dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021.
16. Namgyu Ho, Laura Schmid, and Se-Young Yun. [Large language models are reasoning teachers](https://md.research.elamur.ai/free/en/papers/ho-2023-large-language-models-are-reasoning/). arXiv preprint arXiv:2212.10071, 2022.
17. Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. [Training compute-optimal large language models](https://md.research.elamur.ai/free/en/papers/hoffmann-2022-training-compute-optimal-large-language/). arXiv preprint arXiv:2203.15556, 2022.
18. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. [LoRA: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). In International Conference on Learning Representations, 2022. URL [https://openreview.net/forum?id=nZeVKeeFYf9](https://openreview.net/forum?id=nZeVKeeFYf9). [↗](https://openreview.net/forum?id=nZeVKeeFYf9)
19. Jiaxin Huang, Shixiang Shane Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. [Large language models can self-improve](https://md.research.elamur.ai/free/en/papers/huang-2023-large-language-models-can-self/). arXiv preprint arXiv:2210.11610, 2022.
20. Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. [Large language models cannot self-correct reasoning yet](https://md.research.elamur.ai/free/en/papers/huang-2023-large-language-models-cannot-self/), 2023.
21. Zhanming Jie and Wei Lu. Leveraging training data in few-shot prompting for numerical reasoning. arXiv preprint arXiv:2305.18170, 2023. [find](https://scholar.google.com/scholar?q=Zhanming+Jie+and+Wei+Lu%2E+Leveraging+training+data+in+few%2Dshot+prompting+for+numerical+reasoning%2E+arXiv+preprint+arXiv%3A2305%2E18170%2C+2023%2E)
22. Rik Koncel-Kedziorski, Subhro Roy, Aida Amini, Nate Kushman, and Hannaneh Hajishirzi. MAWPS: A math word problem repository. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1152–1157, San Diego, Ca… [find](https://scholar.google.com/scholar?q=Rik+Koncel%2DKedziorski%2C+Subhro+Roy%2C+Aida+Amini%2C+Nate+Kushman%2C+and+Hannaneh+Hajishirzi%2E+MAWPS%3A+A+math+word+problem+repository%2E+In+Proceedings+of+the+2016+Conference+of+the+North+American+Chapter+of+the+)
23. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. [Efficient memory management for large language model serving with pagedattention](https://md.research.elamur.ai/free/en/papers/kwon-2023-efficient-memory-management-large-language/). In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023.
24. Harrison Lee, Samrat Phatale, Hassan Mansoor, Thomas Mesnard, Johan Ferret, Kellie Lu, Colton Bishop, Ethan Hall, Victor Carbune, Abhinav Rastogi, and Sushant Prakash. Rlaif: Scaling reinforcement learning from human feedback with ai feedback, 2023. [find](https://scholar.google.com/scholar?q=Harrison+Lee%2C+Samrat+Phatale%2C+Hassan+Mansoor%2C+Thomas+Mesnard%2C+Johan+Ferret%2C+Kellie+Lu%2C+Colton+Bishop%2C+Ethan+Hall%2C+Victor+Carbune%2C+Abhinav+Rastogi%2C+and+Sushant+Prakash%2E+Rlaif%3A+Scaling+reinforcement+lea)
25. Bin Lei, Chunhua Liao, Caiwen Ding, et al. Boosting logical reasoning in large language models through a new framework: The graph of thought. arXiv preprint arXiv:2308.08614, 2023. — [arXiv:2308.08614](https://arxiv.org/abs/2308.08614) [find](https://scholar.google.com/scholar?q=Bin+Lei%2C+Chunhua+Liao%2C+Caiwen+Ding%2C+et+al%2E+Boosting+logical+reasoning+in+large+language+models+through+a+new+framework%3A+The+graph+of+thought%2E+arXiv+preprint+arXiv%3A2308%2E08614%2C+2023%2E+%E2%80%94+%5BarXiv%3A2308%2E08614)
26. Brian Lester, Rami Al-Rfou, and Noah Constant. [The power of scale for parameter-efficient prompt tuning](https://md.research.elamur.ai/free/en/papers/lester-2021-power-scale-parameter-efficient-prompt/). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 3045–3059, 2021.
27. Chengpeng Li, Zheng Yuan, Guanting Dong, Keming Lu, Jiancan Wu, Chuanqi Tan, Xiang Wang, and Chang Zhou. Query and response augmentation cannot help out-of-domain math reasoning generalization. arXiv preprint arXiv:2310.05506, 2023a. [find](https://scholar.google.com/scholar?q=Chengpeng+Li%2C+Zheng+Yuan%2C+Guanting+Dong%2C+Keming+Lu%2C+Jiancan+Wu%2C+Chuanqi+Tan%2C+Xiang+Wang%2C+and+Chang+Zhou%2E+Query+and+response+augmentation+cannot+help+out%2Dof%2Ddomain+math+reasoning+generalization%2E+arXiv+)
28. Shiyang Li, Jianshu Chen, Yelong Shen, Zhiyu Chen, Xinlu Zhang, Zekun Li, Hong Wang, Jing Qian, Baolin Peng, Yi Mao, et al. [Explanations from large language models make small reasoners better](https://md.research.elamur.ai/free/en/papers/li-2022-explanations-large-language-models-make/). arXiv preprint arXiv:2210.06726, 2022.
29. Yifei Li, Zeqi Lin, Shizhuo Zhang, Qiang Fu, Bei Chen, Jian-Guang Lou, and Weizhu Chen. [Making language models better reasoners with step-aware verifier](https://md.research.elamur.ai/free/en/papers/li-2023-making-language-models-better-reasoners/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 5315–5333, 2023b.
30. Zhenwen Liang, Wenhao Yu, Tanmay Rajpurohit, Peter Clark, Xiangliang Zhang, and Ashwin Kaylan. Let gpt be a math tutor: Teaching math word problem solvers with customized exercise generation. arXiv preprint arXiv:2305.14386, 2023. [find](https://scholar.google.com/scholar?q=Zhenwen+Liang%2C+Wenhao+Yu%2C+Tanmay+Rajpurohit%2C+Peter+Clark%2C+Xiangliang+Zhang%2C+and+Ashwin+Kaylan%2E+Let+gpt+be+a+math+tutor%3A+Teaching+math+word+problem+solvers+with+customized+exercise+generation%2E+arXiv+pr)
31. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step, 2023. — [Let's Verify Step by Step](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/)
32. Bingbin Liu, Sebastien Bubeck, Ronen Eldan, Janardhan Kulkarni, Yuanzhi Li, Anh Nguyen, Rachel Ward, and Yi Zhang. Tinygsm: achieving ¿80 Yixin Liu, Avi Singh, C. Daniel Freeman, John D. Co-Reyes, and Peter J. Liu. Improving large language model fine-tuning for solving math problems, 2023b. [find](https://scholar.google.com/scholar?q=Bingbin+Liu%2C+Sebastien+Bubeck%2C+Ronen+Eldan%2C+Janardhan+Kulkarni%2C+Yuanzhi+Li%2C+Anh+Nguyen%2C+Rachel+Ward%2C+and+Yi+Zhang%2E+Tinygsm%3A+achieving+%C2%BF80+Yixin+Liu%2C+Avi+Singh%2C+C%2E+Daniel+Freeman%2C+John+D%2E+Co%2DReyes%2C+and)
33. Ximing Lu, Sean Welleck, Jack Hessel, Liwei Jiang, Lianhui Qin, Peter West, Prithviraj Ammanabrolu, and Yejin Choi. [Quark: Controllable text generation with reinforced unlearning](https://md.research.elamur.ai/free/en/papers/lu-2022-quark-controllable-text-generation-reinforced/). In Advances in Neural Information Processing Systems, 2022.
34. Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang. [Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct](https://md.research.elamur.ai/free/en/papers/luo-2023-wizardmath-empowering-mathematical-reasoning-large/). arXiv preprint arXiv:2308.09583, 2023.
35. Lucie Charlotte Magister, Jonathan Mallinson, Jakub Adamek, Eric Malmi, and Aliaksei Severyn. [Teaching small language models to reason](https://md.research.elamur.ai/free/en/papers/magister-2023-teaching-small-language-models-reason/). arXiv preprint arXiv:2212.08410, 2022.
36. Shen-yun Miao, Chao-Chun Liang, and Keh-Yih Su. A diverse corpus for evaluating and developing English math word problem solvers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 975–984, Online, July 2020. Association for Computational Linguistics. doi… [find](https://scholar.google.com/scholar?q=Shen%2Dyun+Miao%2C+Chao%2DChun+Liang%2C+and+Keh%2DYih+Su%2E+A+diverse+corpus+for+evaluating+and+developing+English+math+word+problem+solvers%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Association+for+Compu)
37. OpenAI. Gpt-3.5 turbo fine-tuning and api updates, 2023a. URL [https://openai.com/blog/gpt-3-5-turbo-fine-tuning-and-\\api-updates](https://openai.com/blog/gpt-3-5-turbo-fine-tuning-and-\\api-updates). [↗](https://openai.com/blog/gpt-3-5-turbo-fine-tuning-and-\\api-updates)
38. OpenAI. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/), 2023b.
39. OpenAI. Openai documentation: Models, 2023c. URL [https://platform.openai.com/docs/models/gpt-3-5](https://platform.openai.com/docs/models/gpt-3-5). [↗](https://platform.openai.com/docs/models/gpt-3-5)
40. Liangming Pan, Alon Albalak, Xinyi Wang, and William Yang Wang. [Logic-lm: Empowering large language models with symbolic solvers for faithful logical reasoning](https://md.research.elamur.ai/free/en/papers/pan-2023-logic-lm-empowering-large-language/). arXiv preprint arXiv:2305.12295, 2023.
41. Arkil Patel, Satwik Bhattamishra, and Navin Goyal. [Are NLP models really able to solve simple math word problems?](https://md.research.elamur.ai/free/en/papers/patel-2021-are-nlp-models-really-able/) In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2080–2094, Online, June 2021. Assoc…
42. Chengwei Qin, Aston Zhang, Zhuosheng Zhang, Jiaao Chen, Michihiro Yasunaga, and Diyi Yang. [Is chatgpt a general-purpose natural language processing task solver?](https://md.research.elamur.ai/free/en/papers/qin-2023-chatgpt-general-purpose-natural-language/) arXiv preprint arXiv:2302.06476, 2023.
43. Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. [Direct preference optimization: Your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/), 2023.
44. Leonardo Ranaldi and Andre Freitas. Aligning large and small language models via chain-of-thought reasoning. In Yvette Graham and Matthew Purver (eds.), Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1812–1827,… [find](https://scholar.google.com/scholar?q=Leonardo+Ranaldi+and+Andre+Freitas%2E+Aligning+large+and+small+language+models+via+chain%2Dof%2Dthought+reasoning%2E+In+Yvette+Graham+and+Matthew+Purver+%28eds%2E%29%2C+Proceedings+of+the+18th+Conference+of+the+Europ)
45. Julian’s, Malta, March 2024. [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/papers/ranaldi-2024-aligning-large-small-language-models/). URL https://aclanthology.org/2024.eacl-long.109. [↗](https://aclanthology.org/2024.eacl-long.109)
46. Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, Jérémy Rapin, Artyom Kozhevnikov, Ivan Evtimov, Joanna Bitton, Manish Bhatt, Cristian Canton Ferrer, Aaron Grattafiori, Wenhan Xiong, Alexandre Défossez, Jade Copet, Fai… — [Code Llama: Open Foundation Models for Code](https://md.research.elamur.ai/free/en/papers/roziere-2024-code-llama-open-foundation-models/)
47. Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, Dipanjan Das, and Jason Wei. [Language models are multilingual chain-of-thought reasoners](https://md.research.elamur.ai/free/en/papers/shi-2022-language-models-multilingual-chain-thought/). In The Eleventh International Conference on Learning Representations,…
48. Shaden Smith, Mostofa Patwary, Brandon Norick, Patrick LeGresley, Samyam Rajbhandari, Jared Casper, Zhun Liu, Shrimai Prabhumoye, George Zerveas, Vijay Korthikanti, et al. [Using deepspeed and megatron to train megatron-turing nlg 530b, a large-scale generative language model](https://md.research.elamur.ai/free/en/papers/smith-2022-using-deepspeed-megatron-train-megatron/). arXiv preprint arXiv:22…
49. Kaya Stechly, Matthew Marquez, and Subbarao Kambhampati. Gpt-4 doesn’t know it’s wrong: An analysis of iterative prompting for reasoning problems, 2023. [find](https://scholar.google.com/scholar?q=Kaya+Stechly%2C+Matthew+Marquez%2C+and+Subbarao+Kambhampati%2E+Gpt%2D4+doesn%E2%80%99t+know+it%E2%80%99s+wrong%3A+An+analysis+of+iterative+prompting+for+reasoning+problems%2C+2023%2E)
50. Yusheng Su, Chi-Min Chan, Jiali Cheng, Yujia Qin, Yankai Lin, Shengding Hu, Zonghan Yang, Ning Ding, Xingzhi Sun, Guotong Xie, Zhiyuan Liu, and Maosong Sun. Exploring the impact of model scaling on parameter-efficient tuning, 2023. [find](https://scholar.google.com/scholar?q=Yusheng+Su%2C+Chi%2DMin+Chan%2C+Jiali+Cheng%2C+Yujia+Qin%2C+Yankai+Lin%2C+Shengding+Hu%2C+Zonghan+Yang%2C+Ning+Ding%2C+Xingzhi+Sun%2C+Guotong+Xie%2C+Zhiyuan+Liu%2C+and+Maosong+Sun%2E+Exploring+the+impact+of+model+scaling+on+pa)
51. Xianghui Sun, Yunjie Ji, Baochang Ma, and Xiangang Li. A comparative study between full-parameter and lora-based fine-tuning on chinese instruction data for instruction following large language model, 2023. [find](https://scholar.google.com/scholar?q=Xianghui+Sun%2C+Yunjie+Ji%2C+Baochang+Ma%2C+and+Xiangang+Li%2E+A+comparative+study+between+full%2Dparameter+and+lora%2Dbased+fine%2Dtuning+on+chinese+instruction+data+for+instruction+following+large+language+model%2C)
52. Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. [CommonsenseQA: A question answering challenge targeting commonsense knowledge](https://md.research.elamur.ai/free/en/papers/talmor-2019-commonsenseqa-question-answering-challenge-targeting/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume…
53. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/), 2023a.
54. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. [Llama 2: Open foundation and fine-tuned chat models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/). arXiv preprint arXiv:2307.09288, 2023b.
55. Karthik Valmeekam, Matthew Marquez, and Subbarao Kambhampati. Can large language models really improve by self-critiquing their own plans?, 2023. [find](https://scholar.google.com/scholar?q=Karthik+Valmeekam%2C+Matthew+Marquez%2C+and+Subbarao+Kambhampati%2E+Can+large+language+models+really+improve+by+self%2Dcritiquing+their+own+plans%3F%2C+2023%2E)
56. Ke Wang, Houxing Ren, Aojun Zhou, Zimu Lu, Sichun Luo, Weikang Shi, Renrui Zhang, Linqi Song, Mingjie Zhan, and Hongsheng Li. [Mathcoder: Seamless code integration in llms for enhanced mathematical reasoning](https://md.research.elamur.ai/free/en/papers/wang-2023-mathcoder-seamless-code-integration-llms/), 2023a.
57. Ruida Wang, Wangchunshu Zhou, and Mrinmaya Sachan. Let’s synthesize step by step: Iterative dataset synthesis with large language models by extrapolating errors from small models, 2023b. [find](https://scholar.google.com/scholar?q=Ruida+Wang%2C+Wangchunshu+Zhou%2C+and+Mrinmaya+Sachan%2E+Let%E2%80%99s+synthesize+step+by+step%3A+Iterative+dataset+synthesis+with+large+language+models+by+extrapolating+errors+from+small+models%2C+2023b%2E)
58. Xinyi Wang, Lucas Caccia, Oleksiy Ostapenko, Xingdi Yuan, and Alessandro Sordoni. Guiding language model reasoning with planning tokens. arXiv preprint arXiv:2310.05707, 2023c. — [arXiv:2310.05707](https://arxiv.org/abs/2310.05707) [find](https://scholar.google.com/scholar?q=Xinyi+Wang%2C+Lucas+Caccia%2C+Oleksiy+Ostapenko%2C+Xingdi+Yuan%2C+and+Alessandro+Sordoni%2E+Guiding+language+model+reasoning+with+planning+tokens%2E+arXiv+preprint+arXiv%3A2310%2E05707%2C+2023c%2E+%E2%80%94+%5BarXiv%3A2310%2E05707%5D%28ht)
59. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. [Self-consistency improves chain of thought reasoning in language models](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/). In The Eleventh International Conference on Learning Representations, 2022.
60. Zengzhi Wang, Rui Xia, and Pengfei Liu. Generative ai for math: Part i – mathpile: A billion-token-scale pretraining corpus for math, 2023d. [find](https://scholar.google.com/scholar?q=Zengzhi+Wang%2C+Rui+Xia%2C+and+Pengfei+Liu%2E+Generative+ai+for+math%3A+Part+i+%E2%80%93+mathpile%3A+A+billion%2Dtoken%2Dscale+pretraining+corpus+for+math%2C+2023d%2E)
61. Zhaoyang Wang, Shaohan Huang, Yuxuan Liu, Jiahai Wang, Minghui Song, Zihan Zhang, Haizhen Huang, Furu Wei, Weiwei Deng, Feng Sun, and Qi Zhang. Democratizing reasoning ability: Tailored learning from large language model, 2023e. [find](https://scholar.google.com/scholar?q=Zhaoyang+Wang%2C+Shaohan+Huang%2C+Yuxuan+Liu%2C+Jiahai+Wang%2C+Minghui+Song%2C+Zihan+Zhang%2C+Haizhen+Huang%2C+Furu+Wei%2C+Weiwei+Deng%2C+Feng+Sun%2C+and+Qi+Zhang%2E+Democratizing+reasoning+ability%3A+Tailored+learning+from+)
62. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). Advances in Neural Information Processing Systems, 35:24824–24837, 2022.
63. Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. Wizardlm: Empowering large language models to follow complex instructions, 2023. [find](https://scholar.google.com/scholar?q=Can+Xu%2C+Qingfeng+Sun%2C+Kai+Zheng%2C+Xiubo+Geng%2C+Pu+Zhao%2C+Jiazhan+Feng%2C+Chongyang+Tao%2C+and+Daxin+Jiang%2E+Wizardlm%3A+Empowering+large+language+models+to+follow+complex+instructions%2C+2023%2E)
64. Aiyuan Yang, Bin Xiao, Bingning Wang, Borong Zhang, Chao Yin, Chenxu Lv, Da Pan, Dian Wang, Dong Yan, Fan Yang, et al. [Baichuan 2: Open large-scale language models](https://md.research.elamur.ai/free/en/papers/yang-2023-baichuan-2-open-large-scale/). arXiv preprint arXiv:2309.10305, 2023.
65. Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. [Metamath: Bootstrap your own mathematical questions for large language models](https://md.research.elamur.ai/free/en/papers/yu-2023-metamath-bootstrap-your-own-mathematical/). arXiv preprint arXiv:2309.12284, 2023.
66. Zheng Yuan, Hongyi Yuan, Chengpeng Li, Guanting Dong, Chuanqi Tan, and Chang Zhou. [Scaling relationship on learning mathematical reasoning with large language models](https://md.research.elamur.ai/free/en/papers/yuan-2023-scaling-relationship-learning-mathematical-reasoning/). arXiv preprint arXiv:2308.01825, 2023.
67. Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. [Mammoth: Building math generalist models through hybrid instruction tuning](https://md.research.elamur.ai/free/en/papers/yue-2023-mammoth-building-math-generalist-models/), 2023.
68. Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. [Opt: Open pre-trained transformer language models](https://md.research.elamur.ai/free/en/papers/zhang-2022-opt-open-pre-trained-transformer/). arXiv preprint arXiv:2205.01068, 2022.
69. Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc V Le, et al. [Least-to-most prompting enables complex reasoning in large language models](https://md.research.elamur.ai/free/en/papers/al-2022-least-most-prompting-enables-complex/). In The Eleventh International Conference on Learning Representations, 2022.
70. This is the Appendix of the paper: Learning From Mistakes Makes LLM Better Reasoner. [find](https://scholar.google.com/scholar?q=This+is+the+Appendix+of+the+paper%3A+Learning+From+Mistakes+Makes+LLM+Better+Reasoner%2E)

*Updated 2 November 2026*