---
title: "Augmenting Math Word Problems via Iterative Question"
description: "Open-source LLMs still do poorly on competition-level mathematics such as the MATH… 8.2 points. 38 references, cited by 39 papers here."
authors: ["Haoxiong Liu", "Yifan Zhang", "Yifan Luo", "Andrew Chi-Chih Yao"]
year: 2024
venue: "arXiv:2401.09003"
source: "https://arxiv.org/abs/2401.09003"
doi: "10.48550/arxiv.2401.09003"
arxiv: "2401.09003"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/liu-2024-augmenting-math-word-problems-iterative/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/liu-2024-augmenting-math-word-problems-iterative/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/6/) / Augmenting Math Word Problems via Iterative Question Composing

# Augmenting Math Word Problems via Iterative Question Composing

**Authors:** Haoxiong Liu, [Yifan Zhang](https://md.research.elamur.ai/free/en/authors/yifan-zhang/), [Yifan Luo](https://md.research.elamur.ai/free/en/authors/yifan-luo/) and Andrew Chi-Chih Yao · **Year:** 2024 · **Venue:** [arXiv:2401.09003](https://md.research.elamur.ai/free/en/venues/proceedings-of-the-aaai-conference-on-artificial-intelligence/) ([Proceedings of the AAAI Conference on Artificial Intelligence 2024](https://md.research.elamur.ai/free/en/venues/proceedings-of-the-aaai-conference-on-artificial-intelligence/2024/)) · **Publisher:** [Association for the Advancement of Artificial Intelligence](https://md.research.elamur.ai/free/en/publishers/association-for-the-advancement-of-artificial-intelligence/) · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2401.09003) · [PDF](https://arxiv.org/pdf/2401.09003) · [DOI 10.48550/arxiv.2401.09003](https://doi.org/10.48550/arxiv.2401.09003) · [arXiv 2401.09003](https://arxiv.org/abs/2401.09003) · [OpenAlex](https://openalex.org/W4409348030)

**Citations:** 5 citations (OpenAlex, 2026-10-14) · [39](https://md.research.elamur.ai/free/en/papers/liu-2024-augmenting-math-word-problems-iterative/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Open-source LLMs still do poorly on competition-level mathematics such as the MATH benchmark when they cannot call external tools. The authors combine two common strategies: continued training on web math text and supervised fine-tuning on synthetic question-answer pairs. Their result is MMIQC, a training mixture with two parts:
- roughly 1.2M filtered Math StackExchange pairs taken from RedPajama;
- several kinds of GPT-4-generated data, checked by rejection sampling with GPT-3.5-Turbo.

Their new augmentation technique, Iterative Question Composing (IQC), asks GPT-4 to build progressively harder problems from seed questions over several rounds. A new problem is kept only if GPT-3.5-Turbo's answers match the composed answer. Models fine-tuned on MMIQC beat the same base models fine-tuned on MetaMathQA at every size tested. Qwen-72B-MMIQC reaches 45.0% on MATH, ahead of the earlier open-source best and the original 2023 GPT-4. Results on the 2023 Hungarian national math finals indicate the gains carry over to unseen problems. The work shows that mixing pre-training-style corpora with diverse synthetic data is an effective recipe.

## Key points
- MMIQC components:
  - about 1203.6K Math StackExchange pairs;
  - answer augmentation and question bootstrapping data;
  - similar-problem generation from MATH training seeds;
  - four IQC iterations;
  - a 203.7K subset of MetaMathQA.
- MATH pass@1 without tools:
  - Mistral-7B-MMIQC: 36.0%;
  - Llemma-34B-MMIQC: 38.6%;
  - DeepSeek-67B-MMIQC: 41.0%;
  - Qwen-72B-MMIQC: 45.0%, which is 8.2 points above the earlier open-source best (DeepSeek-67B-MetaMathQA, 36.8%).
- Ablation on Mistral-7B: the IQC rounds add about 55.1K samples and raise accuracy from 31.5% to 34.4%. Later rounds still help. Halving MetaMathQA costs only 1.8 points, which suggests the original set is redundant.
- Contamination check:
  - A 30-gram check found 44 overlaps with the MATH test solutions, compared with 168 between the MATH train and test sets.
  - Manual review showed 43 of the 44 overlaps were shared intermediate steps, not memorized test problems.
- Dataset and code are public: HuggingFace `Vivacem/MMIQC`, GitHub `iiis-ai/IterativeQuestionComposing`.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Tsinghua University](https://md.research.elamur.ai/free/en/universities/tsinghua-university/) — University, Beijing, China · 333 works · 87,019 citations
- [Shanghai Qi Zhi Institute](https://md.research.elamur.ai/free/en/labs/shanghai-qi-zhi-institute/) — Research institute, Shanghai, China · 6 works · 3,008 citations

📚 Shelves (7): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [➗ Mathematics](https://md.research.elamur.ai/free/en/shelves/mathematics/) · 177 works · [🧮 Computational Theory and Mathematics](https://md.research.elamur.ai/free/en/shelves/computer-science/computational-theory-and-mathematics/) · 252 works · [➗ Applied Mathematics](https://md.research.elamur.ai/free/en/shelves/mathematics/applied-mathematics/) · 46 works

🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,426 works

## Cite

Cite as: Haoxiong Liu, Yifan Zhang, Yifan Luo and Andrew Chi-Chih Yao. 2024. Augmenting Math Word Problems via Iterative Question Composing. arXiv:2401.09003.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{liu2024augmenting,
  title = {{Augmenting Math Word Problems via Iterative Question Composing}},
  author = {Haoxiong Liu, Yifan Zhang, Yifan Luo and Andrew Chi-Chih Yao},
  year = {2024},
  doi = {10.48550/arxiv.2401.09003},
  eprint = {2401.09003},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2401.09003}
}
```

- [← previous Reliable, Adaptable, and Attributable Language Models with Retrieval](https://md.research.elamur.ai/free/en/papers/asai-2024-reliable-adaptable-attributable-language-models/)
- [next → Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization](https://md.research.elamur.ai/free/en/papers/wang-2024-enhancing-reasoning-ability-multimodal-large/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/6/)

## References

38 references, 28 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Azerbayev, Z.; Schoelkopf, H.; Paster, K.; Santos, M. D.; McAleer, S.; Jiang, A. Q.; Deng, J.; Biderman, S.; and Welleck, S. 2023. [Llemma: An open language model for mathematics](https://md.research.elamur.ai/free/en/papers/azerbayev-2023-llemma-open-language-model-mathematics/). arXiv preprint arXiv:2310.10631.
2. Bai, J.; Bai, S.; Chu, Y.; Cui, Z.; Dang, K.; Deng, X.; Fan, Y.; Ge, W.; Han, Y.; Huang, F.; et al. 2023. [Qwen technical report](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-technical-report/). arXiv preprint arXiv:2309.16609.
3. Bi, X.; Chen, D.; Chen, G.; Chen, S.; Dai, D.; Deng, C.; Ding, H.; Dong, K.; Du, Q.; Fu, Z.; et al. 2024. DeepSeek LLM: Scaling Open-Source Language Models with Longtermism. arXiv preprint arXiv:2401.02954. [find](https://scholar.google.com/scholar?q=Bi%2C+X%2E%3B+Chen%2C+D%2E%3B+Chen%2C+G%2E%3B+Chen%2C+S%2E%3B+Dai%2C+D%2E%3B+Deng%2C+C%2E%3B+Ding%2C+H%2E%3B+Dong%2C+K%2E%3B+Du%2C+Q%2E%3B+Fu%2C+Z%2E%3B+et+al%2E+2024%2E+DeepSeek+LLM%3A+Scaling+Open%2DSource+Language+Models+with+Longtermism%2E+arXiv+preprint+arXiv%3A2401%2E)
4. Brown, T. B.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; Agarwal, S.; Herbert-Voss, A.; Krueger, G.; Henighan, T. J.; Child, R.; Ramesh, A.; Ziegler, D. M.; Wu, J.; Winter, C.; Hesse, C.; Chen, M.; Sigler, E.; Litwin, M.; Gray, S.;… — [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
5. Bubeck, S.; Chandrasekaran, V.; Eldan, R.; Gehrke, J.; Horvitz, E.; Kamar, E.; Lee, P.; Lee, Y. T.; Li, Y.; Lundberg, S.; et al. 2023. [Sparks of artificial general intelligence: Early experiments with gpt-4](https://md.research.elamur.ai/free/en/papers/bubeck-2023-sparks-artificial-general-intelligence-early/). arXiv preprint arXiv:2303.12712.
6. Chen, M.; Tworek, J.; Jun, H.; Yuan, Q.; de Oliveira Pinto, H. P.; Kaplan, J.; Edwards, H.; Burda, Y.; Joseph, N.; Brockman, G.; Ray, A.; Puri, R.; Krueger, G.; Petrov, M.; Khlaaf, H.; Sastry, G.; Mishkin, P.; Chan, B.; Gray, S.; Ryder, N.; Pavlov, M.; Power, A.; Kaiser, L.; Bavarian, M.; Winter, C.… — [Evaluating Large Language Models Trained on Code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
7. Cobbe, K.; Kosaraju, V.; Bavarian, M.; Chen, M.; Jun, H.; Kaiser, L.; Plappert, M.; Tworek, J.; Hilton, J.; Nakano, R.; Hesse, C.; and Schulman, J. 2021. [Training Verifiers to Solve Math Word Problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168.
8. Computer, T. 2023. RedPajama: An Open Source Recipe to Reproduce LLaMA training dataset. [find](https://scholar.google.com/scholar?q=Computer%2C+T%2E+2023%2E+RedPajama%3A+An+Open+Source+Recipe+to+Reproduce+LLaMA+training+dataset%2E)
9. Cossu, A.; Tuytelaars, T.; Carta, A.; Passaro, L.; Lomonaco, V.; and Bacciu, D. 2022. Continual pre-training mitigates forgetting in language and vision. arXiv preprint arXiv:2205.09357. — [arXiv:2205.09357](https://arxiv.org/abs/2205.09357) [find](https://scholar.google.com/scholar?q=Cossu%2C+A%2E%3B+Tuytelaars%2C+T%2E%3B+Carta%2C+A%2E%3B+Passaro%2C+L%2E%3B+Lomonaco%2C+V%2E%3B+and+Bacciu%2C+D%2E+2022%2E+Continual+pre%2Dtraining+mitigates+forgetting+in+language+and+vision%2E+arXiv+preprint+arXiv%3A2205%2E09357%2E+%E2%80%94+%5BarXiv%3A2205)
10. Fu, Y.; Peng, H.; Sabharwal, A.; Clark, P.; and Khot, T. 2022. [Complexity-based prompting for multi-step reasoning](https://md.research.elamur.ai/free/en/papers/fu-2022-complexity-based-prompting-multi-step/). arXiv preprint arXiv:2210.00720.
11. Gao, L.; Madaan, A.; Zhou, S.; Alon, U.; Liu, P.; Yang, Y.; Callan, J.; and Neubig, G. 2022. PAL: Program-aided Language Models. arXiv preprint arXiv:2211.10435. [find](https://scholar.google.com/scholar?q=Gao%2C+L%2E%3B+Madaan%2C+A%2E%3B+Zhou%2C+S%2E%3B+Alon%2C+U%2E%3B+Liu%2C+P%2E%3B+Yang%2C+Y%2E%3B+Callan%2C+J%2E%3B+and+Neubig%2C+G%2E+2022%2E+PAL%3A+Program%2Daided+Language+Models%2E+arXiv+preprint+arXiv%3A2211%2E10435%2E)
12. Gou, Z.; Shao, Z.; Gong, Y.; Yang, Y.; Huang, M.; Duan, N.; Chen, W.; et al. 2023. [ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving](https://md.research.elamur.ai/free/en/papers/gou-2023-tora-tool-integrated-reasoning-agent/). arXiv preprint arXiv:2309.17452.
13. Hendrycks, D.; Burns, C.; Basart, S.; Zou, A.; Mazeika, M.; Song, D.; and Steinhardt, J. 2021a. [Measuring Massive Multitask Language Understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). arXiv preprint arXiv:2009.03300.
14. Hendrycks, D.; Burns, C.; Kadavath, S.; Arora, A.; Basart, S.; Tang, E.; Song, D.; and Steinhardt, J. 2021b. [Measuring Mathematical Problem Solving With the MATH Dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). NeurIPS.
15. Huang, J.; Gu, S. S.; Hou, L.; Wu, Y.; Wang, X.; Yu, H.; and Han, J. 2022a. [Large language models can self-improve](https://md.research.elamur.ai/free/en/papers/huang-2023-large-language-models-can-self/). arXiv preprint arXiv:2210.11610.
16. Huang, W.; Abbeel, P.; Pathak, D.; and Mordatch, I. 2022b. [Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents](https://md.research.elamur.ai/free/en/papers/huang-2022-language-models-zero-shot-planners/). ArXiv, abs/2201.07207. Jiang, A. Q.; Sablayrolles, A.; Mensch, A.; Bamford, C.; Chaplot, D. S.; Casas, D. d. l.; Bressand, F.; Lengyel, G.; Lample, G…
17. Lewkowycz, A.; Andreassen, A. J.; Dohan, D.; Dyer, E.; Michalewski, H.; Ramasesh, V. V.; Slone, A.; Anil, C.; Schlag, I.; Gutman-Solo, T.; Wu, Y.; Neyshabur, B.; Gur-Ari, G.; and Misra, V. 2022. [Solving Quantitative Reasoning Problems with Language Models](https://md.research.elamur.ai/free/en/papers/lewkowycz-2022-solving-quantitative-reasoning-problems-language/). In Oh, A. H.; Agarwal, A.; Belgrave, D.; an…
18. Lightman, H.; Kosaraju, V.; Burda, Y.; Edwards, H.; Baker, B.; Lee, T.; Leike, J.; Schulman, J.; Sutskever, I.; and Cobbe, K. 2023. [Let’s Verify Step by Step](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/). arXiv preprint arXiv:2305.20050.
19. Liu, B.; Bubeck, S.; Eldan, R.; Kulkarni, J.; Li, Y.; Nguyen, A.; Ward, R.; and Zhang, Y. 2023. TinyGSM: achieving¿ 80% on GSM8k with small language models. arXiv preprint arXiv:2312.09241. Meurer, A.; Smith, C. P.; Paprocki, M.; Čertı́k, O.; Kirpichev, S. B.; Rocklin, M.; Kumar, A.; Ivanov, S.; Mo… [find](https://scholar.google.com/scholar?q=Liu%2C+B%2E%3B+Bubeck%2C+S%2E%3B+Eldan%2C+R%2E%3B+Kulkarni%2C+J%2E%3B+Li%2C+Y%2E%3B+Nguyen%2C+A%2E%3B+Ward%2C+R%2E%3B+and+Zhang%2C+Y%2E+2023%2E+TinyGSM%3A+achieving%C2%BF+80%25+on+GSM8k+with+small+language+models%2E+arXiv+preprint+arXiv%3A2312%2E09241%2E+Meurer%2C+A%2E)
20. OpenAI. 2023. [GPT-4 Technical Report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). ArXiv, abs/2303.08774.
21. Ouyang, L.; Wu, J.; Jiang, X.; Almeida, D.; Wainwright, C. L.; Mishkin, P.; Zhang, C.; Agarwal, S.; Slama, K.; Ray, A.; Schulman, J.; Hilton, J.; Kelton, F.; Miller, L.; Simens, M.; Askell, A.; Welinder, P.; Christiano, P.; Leike, J.; and Lowe, R. 2022. [Training language models to follow instruction…](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/)
22. Park, J. S.; O’Brien, J.; Cai, C. J.; Morris, M. R.; Liang, P.; and Bernstein, M. S. 2023. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, 1–22. [find](https://scholar.google.com/scholar?q=Park%2C+J%2E+S%2E%3B+O%E2%80%99Brien%2C+J%2E%3B+Cai%2C+C%2E+J%2E%3B+Morris%2C+M%2E+R%2E%3B+Liang%2C+P%2E%3B+and+Bernstein%2C+M%2E+S%2E+2023%2E+Generative+agents%3A+Interactive+simulacra+of+human+behavior%2E+In+Proceedings+of+the+36th+Annual+ACM+Symposium+o)
23. Paster, K. 2023a. Testing Language Models on a Held-Out High School National Finals Exam. [https://huggingface.co/](https://huggingface.co/) datasets/keirp/hungarian national hs finals exam. [↗](https://huggingface.co/)
24. Paster, K. 2023b. [Testing Language Models on a Held-Out High School National Finals Exam](https://md.research.elamur.ai/free/en/papers/paster-2023-openwebmath-open-dataset-high-quality/). [https://huggingface.co/](https://huggingface.co/) datasets/keirp/hungarian national hs finals exam. Paster, K.; Dos Santos, . M.; Azerbayev, Z.; and Ba, . J. 2023. OpenWebMath: An Open Dataset of High-Quality Mathematical Web Text. arXi… [↗](https://huggingface.co/)
25. Radford, A.; Wu, J.; Child, R.; Luan, D.; Amodei, D.; and Sutskever, I. 2019. Language Models are Unsupervised Multitask Learners. [find](https://scholar.google.com/scholar?q=Radford%2C+A%2E%3B+Wu%2C+J%2E%3B+Child%2C+R%2E%3B+Luan%2C+D%2E%3B+Amodei%2C+D%2E%3B+and+Sutskever%2C+I%2E+2019%2E+Language+Models+are+Unsupervised+Multitask+Learners%2E)
26. OpenAI Blog. Rajbhandari, S.; Rasley, J.; Ruwase, O.; and He, Y. 2020. Zero: Memory optimizations toward training trillion parameter models. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, 1–16. [find](https://scholar.google.com/scholar?q=OpenAI+Blog%2E+Rajbhandari%2C+S%2E%3B+Rasley%2C+J%2E%3B+Ruwase%2C+O%2E%3B+and+He%2C+Y%2E+2020%2E+Zero%3A+Memory+optimizations+toward+training+trillion+parameter+models%2E+In+SC20%3A+International+Conference+for+High+Performance+Comp)
27. IEEE. Taori, R.; Gulrajani, I.; Zhang, T.; Dubois, Y.; Li, X.; Guestrin, C.; Liang, P.; and Hashimoto, T. B. 2023. Stanford Alpaca: An Instruction-following LLaMA model. [https://github.com/tatsu-lab/stanford](https://github.com/tatsu-lab/stanford) alpaca. [↗](https://github.com/tatsu-lab/stanford)
28. Team, G.; Anil, R.; Borgeaud, S.; Wu, Y.; Alayrac, J.-B.; Yu, J.; Soricut, R.; Schalkwyk, J.; Dai, A. M.; Hauth, A.; et al. 2023. [Gemini: a family of highly capable multimodal models](https://md.research.elamur.ai/free/en/papers/gemini-team-2023-gemini-family-highly-capable-multimodal/). arXiv preprint arXiv:2312.11805.
29. Touvron, H.; Martin, L.; Stone, K.; Albert, P.; Almahairi, A.; Babaei, Y.; Bashlykov, N.; Batra, S.; Bhargava, P.; Bhosale, S.; Bikel, D.; Blecher, L.; Ferrer, C. C.; Chen, M.; Cucurull, G.; Esiobu, D.; Fernandes, J.; Fu, J.; Fu, W.; Fuller, B.; Gao, C.; Goswami, V.; Goyal, N.; Hartshorn, A.; Hossei… — [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/)
30. Touvron, H.; Martin, L.; Stone, K. R.; Albert, P.; Almahairi, A.; Babaei, Y.; Bashlykov, N.; Batra, S.; Bhargava, P.; Bhosale, S.; Bikel, D. M.; Blecher, L.; Ferrer, C. C.; Chen, M.; Cucurull, G.; Esiobu, D.; Fernandes, J.; Fu, J.; Fu, W.; Fuller, B.; Gao, C.; Goswami, V.; Goyal, N.; Hartshorn, A. S… — [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/)
31. Uesato, J.; Kushman, N.; Kumar, R.; Song, F.; Siegel, N.; Wang, L.; Creswell, A.; Irving, G.; and Higgins, I. 2022. [Solving math word problems with process- and outcome-based feedback](https://md.research.elamur.ai/free/en/papers/uesato-2022-solving-math-word-problems-process/). arXiv:2211.14275.
32. Wang, P.; Li, L.; Shao, Z.; Xu, R.; Dai, D.; Li, Y.; Chen, D.; Wu, Y.; and Sui, Z. 2023a. [Math-Shepherd: A Label-Free Step-by-Step Verifier for LLMs in Mathematical Reasoning](https://md.research.elamur.ai/free/en/papers/wang-2024-math-shepherd-verify-reinforce-llms/). arXiv preprint arXiv:2312.08935.
33. Wang, X.; Wei, J.; Schuurmans, D.; Le, Q. V.; Chi, E. H.; Narang, S.; Chowdhery, A.; and Zhou, D. 2023b. [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/). In The Eleventh International Conference on Learning Representations.
34. Wei, J.; Wang, X.; Schuurmans, D.; Bosma, M.; Ichter, B.; Xia, F.; Chi, E.; Le, Q.; and Zhou, D. 2023. [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). arXiv:2201.11903.
35. Wolf, T.; Debut, L.; Sanh, V.; Chaumond, J.; Delangue, C.; Moi, A.; Cistac, P.; Rault, T.; Louf, R.; Funtowicz, M.; et al. 2019. Huggingface’s [transformers: State-of-the-art natural language processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/). arXiv preprint arXiv:1910.03771.
36. Yu, L.; Jiang, W.; Shi, H.; Yu, J.; Liu, Z.; Zhang, Y.; Kwok, J. T.; Li, Z.; Weller, A.; and Liu, W. 2023. [MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models](https://md.research.elamur.ai/free/en/papers/yu-2023-metamath-bootstrap-your-own-mathematical/). arXiv preprint arXiv:2309.12284.
37. Yuan, Z.; Yuan, H.; Li, C.; Dong, G.; Tan, C.; and Zhou, C. 2023. [Scaling relationship on learning mathematical reasoning with large language models](https://md.research.elamur.ai/free/en/papers/yuan-2023-scaling-relationship-learning-mathematical-reasoning/). arXiv preprint arXiv:2308.01825.
38. Yue, X.; Qu, X.; Zhang, G.; Fu, Y.; Huang, W.; Sun, H.; Su, Y.; and Chen, W. 2023. [Mammoth: Building math generalist models through hybrid instruction tuning](https://md.research.elamur.ai/free/en/papers/yue-2023-mammoth-building-math-generalist-models/). arXiv preprint arXiv:2309.05653.

*Updated 30 October 2026*