---
title: "Learning Math Reasoning from Self-Sampled Correct and"
description: "Language models fine-tuned for grade-school math tend to overfit because most training sets offer… 28 references, cited by 37 papers here."
authors: ["Ansong Ni", "Jeevana Priya Inala", "Chenglong Wang", "Oleksandr Polozov", "Christopher Meek", "Dragomir Radev", "Jianfeng Gao"]
year: 2022
venue: "arXiv:2205.14318"
source: "https://arxiv.org/abs/2205.14318"
doi: "10.48550/arxiv.2205.14318"
arxiv: "2205.14318"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ni-2022-learning-math-reasoning-self-sampled/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ni-2022-learning-math-reasoning-self-sampled/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/5/) / Learning Math Reasoning from Self-Sampled Correct and Partially-Correct Solutions

# Learning Math Reasoning from Self-Sampled Correct and Partially-Correct Solutions

**Authors:** [Ansong Ni](https://md.research.elamur.ai/free/en/authors/ansong-ni/), [Jeevana Priya Inala](https://md.research.elamur.ai/free/en/authors/jeevana-priya-inala/), [Chenglong Wang](https://md.research.elamur.ai/free/en/authors/chenglong-wang/), [Oleksandr Polozov](https://md.research.elamur.ai/free/en/authors/oleksandr-polozov/), [Christopher Meek](https://md.research.elamur.ai/free/en/authors/christopher-meek/), [Dragomir Radev](https://md.research.elamur.ai/free/en/authors/dragomir-radev/) and [Jianfeng Gao](https://md.research.elamur.ai/free/en/authors/jianfeng-gao/) · **Year:** 2022 · **Venue:** arXiv:2205.14318 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2205.14318) · [PDF](https://arxiv.org/pdf/2205.14318) · [DOI 10.48550/arxiv.2205.14318](https://doi.org/10.48550/arxiv.2205.14318) · [arXiv 2205.14318](https://arxiv.org/abs/2205.14318) · [OpenAlex](https://openalex.org/W4281994620)

**Citations:** 2 citations (OpenAlex, 2026-10-14) · [37](https://md.research.elamur.ai/free/en/papers/ni-2022-learning-math-reasoning-self-sampled/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Language models fine-tuned for grade-school math tend to overfit because most training sets offer only one reference solution per problem, even though many valid reasoning paths usually exist. The authors treat word problems as a task of generating Python programs whose execution must give the gold answer. They fine-tune GPT-Neo models on MathQA-Python and GSM8K.

Instead of learning only from the given reference, the model draws its own samples during training and executes them. Programs that reach the right answer go into a per-question buffer that persists across epochs, and trivial variants are discarded. The method also keeps partially-correct program prefixes, meaning prefixes whose intermediate execution state matches a state of an already known correct solution. These prefixes steer sampling toward more alternative solutions. Several objectives for learning from multiple targets are compared.

Training on self-sampled solutions clearly raises PASS@k over standard MLE, and the gains hold across model sizes. This shows that a model can enrich its own supervision without additional human-written solutions.

## Key points
- PASS@100 on GSM8K rises from 35.5% to 44.5%, and PASS@80 on a filtered MathQA-Python rises from 27.6% to 36.2%, compared with single-reference MLE.
- Adding partially-correct solutions on top of fully-correct ones yields about +3.0% PASS@100 on GSM8K. It also leads to more fully-correct solutions accumulating in the buffer, and partial solutions mainly help early in training.
- Among the losses MLE-Aug, MML and β-MML, MLE-Aug works best. MML barely reduces overfitting, because it pushes probability mass toward a single solution.
- Duplicate filtering compares linearized ASTs and caps program length at the line count of the reference. Without this filtering, performance degrades sharply.
- Checkpoints are selected by best PASS@1. The method does not improve the peak PASS@50/100 reached early in training. Code is released at github.com/microsoft/TraceCodegen.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Google](https://md.research.elamur.ai/free/en/companies/google/) — Company, Mountain View, United States · 533 works · 443,885 citations
- [Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/) — Company, Redmond, United States · 403 works · 162,838 citations
- [University of Washington](https://md.research.elamur.ai/free/en/universities/university-of-washington/) — University, Seattle, United States · 422 works · 213,539 citations
- [Yale University](https://md.research.elamur.ai/free/en/universities/yale-university/) — University, New Haven, United States · 112 works · 14,009 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,959 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,789 works · [🧮 Computational Theory and Mathematics](https://md.research.elamur.ai/free/en/shelves/computer-science/computational-theory-and-mathematics/) · 496 works

🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,948 works

## Cite

Cite as: Ansong Ni, Jeevana Priya Inala, Chenglong Wang, Oleksandr Polozov, Christopher Meek, Dragomir Radev and Jianfeng Gao. 2022. Learning Math Reasoning from Self-Sampled Correct and Partially-Correct Solutions. arXiv:2205.14318.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{ni2022learning,
  title = {{Learning Math Reasoning from Self-Sampled Correct and Partially-Correct Solutions}},
  author = {Ansong Ni, Jeevana Priya Inala, Chenglong Wang, Oleksandr Polozov, Christopher Meek, Dragomir Radev and Jianfeng Gao},
  year = {2022},
  doi = {10.48550/arxiv.2205.14318},
  eprint = {2205.14318},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2205.14318}
}
```

- [← previous Fault-Aware Neural Code Rankers](https://md.research.elamur.ai/free/en/papers/inala-2022-fault-aware-neural-code-rankers/)
- [next → SEQZERO: Few-shot Compositional Semantic Parsing with Sequential Prompts and Zero-shot Models](https://md.research.elamur.ai/free/en/papers/yang-2022-seqzero-few-shot-compositional-semantic/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/5/)

## References

28 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Rishabh Agarwal, Chen Liang, Dale Schuurmans, and Mohammad Norouzi. Learning to generalize from sparse and underspecified rewards. In International Conference on Machine Learning, pp. 130–140. PMLR, 2019. [find](https://scholar.google.com/scholar?q=Rishabh+Agarwal%2C+Chen+Liang%2C+Dale+Schuurmans%2C+and+Mohammad+Norouzi%2E+Learning+to+generalize+from+sparse+and+underspecified+rewards%2E+In+International+Conference+on+Machine+Learning%2C+pp%2E+130%E2%80%93140%2E+PMLR%2C+2)
2. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. [Program synthesis with large language models](https://md.research.elamur.ai/free/en/papers/austin-2021-program-synthesis-large-language-models/). arXiv preprint arXiv:2108.07732, 2021.
3. Sid Black, Leo Gao, Phil Wang, Connor Leahy, and Stella Biderman. GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh-Tensorflow, March 2021. [find](https://scholar.google.com/scholar?q=Sid+Black%2C+Leo+Gao%2C+Phil+Wang%2C+Connor+Leahy%2C+and+Stella+Biderman%2E+GPT%2DNeo%3A+Large+Scale+Autoregressive+Language+Modeling+with+Mesh%2DTensorflow%2C+March+2021%2E)
4. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems, 33:1877–1901, 2020.
5. Rudy Bunel, Matthew Hausknecht, Jacob Devlin, Rishabh Singh, and Pushmeet Kohli. [Leveraging grammar and reinforcement learning for neural program synthesis](https://md.research.elamur.ai/free/en/papers/bunel-2018-leveraging-grammar-reinforcement-learning-neural/). In International Conference on Learning Representations, 2018.
6. Bei Chen, Fengji Zhang, Anh Nguyen, Daoguang Zan, Zeqi Lin, Jian-Guang Lou, and Weizhu Chen. Codet: Code generation with generated tests. arXiv preprint arXiv:2207.10397, 2022. [find](https://scholar.google.com/scholar?q=Bei+Chen%2C+Fengji+Zhang%2C+Anh+Nguyen%2C+Daoguang+Zan%2C+Zeqi+Lin%2C+Jian%2DGuang+Lou%2C+and+Weizhu+Chen%2E+Codet%3A+Code+generation+with+generated+tests%2E+arXiv+preprint+arXiv%3A2207%2E10397%2C+2022%2E)
7. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. [Evaluating large language models trained on code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/). arXiv preprint arXiv:2107.03374, 2021a.
8. Xinyun Chen, Chang Liu, and Dawn Song. Execution-guided neural program synthesis. In International Conference on Learning Representations, 2018. [find](https://scholar.google.com/scholar?q=Xinyun+Chen%2C+Chang+Liu%2C+and+Dawn+Song%2E+Execution%2Dguided+neural+program+synthesis%2E+In+International+Conference+on+Learning+Representations%2C+2018%2E)
9. Xinyun Chen, Dawn Song, and Yuandong Tian. Latent execution for neural program synthesis beyond domain-specific languages. Advances in Neural Information Processing Systems, 34, 2021b. [find](https://scholar.google.com/scholar?q=Xinyun+Chen%2C+Dawn+Song%2C+and+Yuandong+Tian%2E+Latent+execution+for+neural+program+synthesis+beyond+domain%2Dspecific+languages%2E+Advances+in+Neural+Information+Processing+Systems%2C+34%2C+2021b%2E)
10. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. [Palm: Scaling language modeling with pathways](https://md.research.elamur.ai/free/en/papers/chowdhery-2022-palm-scaling-language-modeling-pathways/). arXiv preprint arXiv:2204.02311, 2022.
11. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168, 2021.
12. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volum…
13. Kevin Ellis, Maxwell Nye, Yewen Pu, Felix Sosa, Josh Tenenbaum, and Armando Solar-Lezama. [Write, execute, assess: Program synthesis with a repl](https://md.research.elamur.ai/free/en/papers/ellis-2019-write-execute-assess-program-synthesis/). Advances in Neural Information Processing Systems, 32, 2019.
14. Catherine Finegan-Dollak, Jonathan K Kummerfeld, Li Zhang, Karthik Ramanathan, Sesh Sadasivam, Rui Zhang, and Dragomir R Radev. [Improving text-to-sql evaluation methodology](https://md.research.elamur.ai/free/en/papers/dollak-2018-improving-text-sql-evaluation-methodology/). In ACL, 2018.
15. Kelvin Guu, Panupong Pasupat, Evan Zheran Liu, and Percy Liang. [From language to programs: Bridging reinforcement learning and maximum marginal likelihood](https://md.research.elamur.ai/free/en/papers/guu-2017-language-programs-bridging-reinforcement-learning/). In ACL, 2017.
16. Jayant Krishnamurthy, Pradeep Dasigi, and Matt Gardner. Neural semantic parsing with type constraints for semi-structured tables. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 1516–1526, 2017. [find](https://scholar.google.com/scholar?q=Jayant+Krishnamurthy%2C+Pradeep+Dasigi%2C+and+Matt+Gardner%2E+Neural+semantic+parsing+with+type+constraints+for+semi%2Dstructured+tables%2E+In+Proceedings+of+the+2017+Conference+on+Empirical+Methods+in+Natural+)
17. Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, et al. [Competition-level code generation with alphacode](https://md.research.elamur.ai/free/en/papers/li-2022-competition-level-code-generation-alphacode/). arXiv preprint arXiv:2203.07814, 2022.
18. Chen Liang, Jonathan Berant, Quoc Le, Kenneth Forbus, and Ni Lao. Neural symbolic machines: Learning semantic parsers on freebase with weak supervision. In ACL, pp. 23–33, 2017. [find](https://scholar.google.com/scholar?q=Chen+Liang%2C+Jonathan+Berant%2C+Quoc+Le%2C+Kenneth+Forbus%2C+and+Ni+Lao%2E+Neural+symbolic+machines%3A+Learning+semantic+parsers+on+freebase+with+weak+supervision%2E+In+ACL%2C+pp%2E+23%E2%80%9333%2C+2017%2E)
19. Chen Liang, Mohammad Norouzi, Jonathan Berant, Quoc V Le, and Ni Lao. Memory augmented policy optimization for program synthesis and semantic parsing. Advances in Neural Information Processing Systems, 31, 2018. [find](https://scholar.google.com/scholar?q=Chen+Liang%2C+Mohammad+Norouzi%2C+Jonathan+Berant%2C+Quoc+V+Le%2C+and+Ni+Lao%2E+Memory+augmented+policy+optimization+for+program+synthesis+and+semantic+parsing%2E+Advances+in+Neural+Information+Processing+Systems)
20. Sewon Min, Danqi Chen, Hannaneh Hajishirzi, and Luke Zettlemoyer. [A discrete hard em approach for weakly supervised question answering](https://md.research.elamur.ai/free/en/papers/min-2019-discrete-hard-em-approach-weakly/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing…
21. Ansong Ni, Pengcheng Yin, and Graham Neubig. Merging weak and active supervision for semantic parsing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pp. 8536– 8543, 2020. [find](https://scholar.google.com/scholar?q=Ansong+Ni%2C+Pengcheng+Yin%2C+and+Graham+Neubig%2E+Merging+weak+and+active+supervision+for+semantic+parsing%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+volume+34%2C+pp%2E+8536%E2%80%93+8543%2C+2020)
22. Maxwell Nye, Yewen Pu, Matthew Bowers, Jacob Andreas, Joshua B Tenenbaum, and Armando Solar-Lezama. Representing partial programs with blended abstract semantics. In International Conference on Learning Representations, 2020. [find](https://scholar.google.com/scholar?q=Maxwell+Nye%2C+Yewen+Pu%2C+Matthew+Bowers%2C+Jacob+Andreas%2C+Joshua+B+Tenenbaum%2C+and+Armando+Solar%2DLezama%2E+Representing+partial+programs+with+blended+abstract+semantics%2E+In+International+Conference+on+Learni)
23. Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, et al. [Show your work: Scratchpads for intermediate computation with language models](https://md.research.elamur.ai/free/en/papers/al-2021-show-your-work-scratchpads-intermediate/). arXiv preprint arXiv:2112.00114, 2021.
24. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu, et al. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). J. Mach. Learn. Res., 21(140):1–67, 2020.
25. Tal Schuster, Ashwin Kalyan, Alex Polozov, and Adam Tauman Kalai. Programming puzzles. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1), 2021. [find](https://scholar.google.com/scholar?q=Tal+Schuster%2C+Ashwin+Kalyan%2C+Alex+Polozov%2C+and+Adam+Tauman+Kalai%2E+Programming+puzzles%2E+In+Thirty%2Dfifth+Conference+on+Neural+Information+Processing+Systems+Datasets+and+Benchmarks+Track+%28Round+1%29%2C+2021)
26. Riley Simmons-Edler, Anders Miltner, and Sebastian Seung. Program synthesis through reinforcement learning guided tree search. arXiv preprint arXiv:1806.02932, 2018. — [arXiv:1806.02932](https://arxiv.org/abs/1806.02932) [find](https://scholar.google.com/scholar?q=Riley+Simmons%2DEdler%2C+Anders+Miltner%2C+and+Sebastian+Seung%2E+Program+synthesis+through+reinforcement+learning+guided+tree+search%2E+arXiv+preprint+arXiv%3A1806%2E02932%2C+2018%2E+%E2%80%94+%5BarXiv%3A1806%2E02932%5D%28https%3A%2F%2Farxiv)
27. Chenglong Wang, Kedar Tatwawadi, Marc Brockschmidt, Po-Sen Huang, Yi Mao, Oleksandr Polozov, and Rishabh Singh. Robust text-to-sql generation with execution-guided decoding. arXiv preprint arXiv:1807.03100, 2018. [find](https://scholar.google.com/scholar?q=Chenglong+Wang%2C+Kedar+Tatwawadi%2C+Marc+Brockschmidt%2C+Po%2DSen+Huang%2C+Yi+Mao%2C+Oleksandr+Polozov%2C+and+Rishabh+Singh%2E+Robust+text%2Dto%2Dsql+generation+with+execution%2Dguided+decoding%2E+arXiv+preprint+arXiv%3A1807%2E)
28. Ruiqi Zhong, Tao Yu, and Dan Klein. Semantic evaluation for text-to-sql with distilled test suites. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 396–411, 2020. [find](https://scholar.google.com/scholar?q=Ruiqi+Zhong%2C+Tao+Yu%2C+and+Dan+Klein%2E+Semantic+evaluation+for+text%2Dto%2Dsql+with+distilled+test+suites%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP%29%2C+p)

*Updated 3 November 2026*