---
title: "MetaMath: Bootstrap Your Own Mathematical — Yu 2023"
description: "Open-source LLMs such as LLaMA-2 still trail proprietary systems on multi-step math word problems, and finetuning can only improve them as far as the…"
authors: ["Longhui Yu", "Weisen Jiang", "Han Shi", "Jincheng Yu", "Zhengying Liu", "Yu Zhang", "James T. Kwok", "Zhenguo Li", "Adrian Weller", "Weiyang Liu"]
year: 2023
venue: "arXiv:2309.12284"
source: "https://arxiv.org/abs/2309.12284"
doi: "10.48550/arxiv.2309.12284"
arxiv: "2309.12284"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "https://arxiv.org/licenses/nonexclusive-distrib/1.0/license.html"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/yu-2023-metamath-bootstrap-your-own-mathematical/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/yu-2023-metamath-bootstrap-your-own-mathematical/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/4/) / MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models

# MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models

**Authors:** Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, [Yu Zhang](https://md.research.elamur.ai/free/en/authors/yu-zhang/), James T. Kwok, Zhenguo Li, Adrian Weller, Weiyang Liu · **Year:** 2023 · **Venue:** arXiv:2309.12284 · **Licence:** [arXiv — non-exclusive licence to distribute](https://arxiv.org/licenses/nonexclusive-distrib/1.0/license.html)

**Links:** [Publisher's page](https://arxiv.org/abs/2309.12284) · [PDF](https://arxiv.org/pdf/2309.12284v4) · [DOI 10.48550/arxiv.2309.12284](https://doi.org/10.48550/arxiv.2309.12284) · [arXiv 2309.12284](https://arxiv.org/abs/2309.12284) · [OpenAlex](https://openalex.org/W7103374186)

**Citations:** 27 citations (openalex-snapshot · 2026-10-14) · [529](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Open-source LLMs such as LLaMA-2 still trail proprietary systems on multi-step math word problems, and finetuning can only improve them as far as the training data allows. The authors expand the GSM8K and MATH training sets by generating several variants of each original ("meta") question. First, they sample extra chain-of-thought solutions from a stronger LLM and keep only those that reach the correct answer (AnsAug). They also have the LLM reword each question. Finally, they build backward-reasoning items: one number in the problem is replaced by an unknown x, and the model has to recover it from the known final answer (Self-Verification and FOBAR formats). The combined corpus, MetaMathQA, is used to finetune LLaMA-2 into the MetaMath family (7B, 13B, 70B). MetaMath-7B reaches 66.5% on GSM8K and 19.8% on MATH, which is 11.5 and 8.7 points above the best open models of the same size. MetaMath-70B slightly outperforms GPT-3.5-Turbo on GSM8K. The study argues that varying how questions are posed, not just adding more solutions, keeps accuracy rising. The dataset, models and code are public.

## Key points
- MetaMathQA uses four augmentation types: AnsAug, rephrased questions, SV and FOBAR. Earlier work used backward reasoning only to check answers at inference time; here it becomes training data.
- Headline scores: MetaMath-7B gets 66.5% on GSM8K and 19.8% on MATH; MetaMath-70B, trained with QLoRA, gets 82.3% on GSM8K.
- Ablation on LLaMA-2-13B: GSM8K rises from 50.9% with plain SFT to 72.3% with all augmentations, and MATH from 4.5% to 7.2%. With AnsAug-only data, accuracy stops improving even at 240K examples, while bootstrapped questions keep raising it.
- The dataset also helps other base models: Mistral-7B goes from 52.2% to 77.7% on GSM8K, and Llemma-7B from 18.0% to 30.0% on MATH.
- Out of distribution, on zero-shot DROP questions with numeric answers, MetaMath-7B scores 37.1% exact match against 31.5% for WizardMath-7B. This suggests the gains do not come from overfitting the two benchmarks.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Mathematics & statistics](https://md.research.elamur.ai/free/en/shelves/mathematics/)

## Cited by

Cited by

529 papers of our library cite it (of the 140673 whose bibliographies we read); 33 of them have a page here, 496 not yet.

- [Large Language Models for Mathematical Reasoning: Progresses and Challenges](https://md.research.elamur.ai/free/en/papers/ahn-2024-large-language-models-mathematical-reasoning/)
- [SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model](https://md.research.elamur.ai/free/en/papers/allal-2025-smollm2-smol-goes-big-data/)
- [Llemma: An Open Language Model For Mathematics](https://md.research.elamur.ai/free/en/papers/azerbayev-2023-llemma-open-language-model-mathematics/)
- [Bridging the Culture Gap: A Framework for LLM-Driven Socio-Cultural Localization of Math Word Problems in Low-Resource Languages](https://md.research.elamur.ai/free/en/papers/azime-2026-bridging-culture-gap-framework-llm/)
- [Qwen Technical Report](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-technical-report/)
- [A Survey on Evaluation of Large Language Models](https://md.research.elamur.ai/free/en/papers/chang-2023-survey-evaluation-large-language-models/)
- [AlphaMath Almost Zero: Process Supervision without Process](https://md.research.elamur.ai/free/en/papers/chen-2024-alphamath-almost-zero-process-supervision/)
- [Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling](https://md.research.elamur.ai/free/en/papers/chen-2024-expanding-performance-boundaries-open/)
- [How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites](https://md.research.elamur.ai/free/en/papers/chen-2024-far-gpt-4v-closing/)
- [Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models](https://md.research.elamur.ai/free/en/papers/chen-2025-towards-reasoning-era-survey-long/)
- [Scaling Synthetic Data Creation with 1,000,000,000 Personas](https://md.research.elamur.ai/free/en/papers/ge-2024-scaling-synthetic-data-creation-1/)
- [rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking](https://md.research.elamur.ai/free/en/papers/guan-2025-rstar-math-small-llms-can/)
- [What matters when building vision-language models?](https://md.research.elamur.ai/free/en/papers/lauren-2024-ccon-matters-building-vision-language-models/)
- [DeepInception: Hypnotize Large Language Model to Be Jailbreaker](https://md.research.elamur.ai/free/en/papers/li-2023-deepinception-hypnotize-large-language-model/)
- [CultureLLM: Incorporating Cultural Differences into Large Language Models](https://md.research.elamur.ai/free/en/papers/li-2024-culturellm-incorporating-cultural-differences-large/)
- [GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solvers](https://md.research.elamur.ai/free/en/papers/li-2024-gsm-plus-comprehensive-benchmark-evaluating/)
- [On LLMs-Driven Synthetic Data Generation, Curation, and Evaluation: A Survey](https://md.research.elamur.ai/free/en/papers/long-2024-llms-driven-synthetic-data-generation/)
- [WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct](https://md.research.elamur.ai/free/en/papers/luo-2023-wizardmath-empowering-mathematical-reasoning-large/)
- [Improve Mathematical Reasoning in Language Models by Automated Process Supervision](https://md.research.elamur.ai/free/en/papers/luo-2024-improve-mathematical-reasoning-language-models/)
- [Orca-Math: Unlocking the potential of SLMs in Grade School Math](https://md.research.elamur.ai/free/en/papers/mitra-2024-orca-math-unlocking-potential-slms/)
- [s1: Simple test-time scaling](https://md.research.elamur.ai/free/en/papers/muennighoff-2025-s1-simple-test-time-scaling/)
- [Iterative Reasoning Preference Optimization](https://md.research.elamur.ai/free/en/papers/pang-2024-iterative-reasoning-preference-optimization/)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/)
- [ReFT: Reasoning with Reinforced Fine-Tuning](https://md.research.elamur.ai/free/en/papers/trung-2024-reft-reasoning-reinforced-fine-tuning/)
- [Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotations](https://md.research.elamur.ai/free/en/papers/wang-2024-math-shepherd-verify-reinforce-llms/)
- [DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding](https://md.research.elamur.ai/free/en/papers/wu-2024-deepseek-vl2-mixture-experts/)
- [Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models](https://md.research.elamur.ai/free/en/papers/wu-2024-inference-scaling-laws-empirical-analysis/)
- [LIMO: Less is More for Reasoning](https://md.research.elamur.ai/free/en/papers/ye-2025-limo-less-more-reasoning/)
- [Demystifying Long Chain-of-Thought Reasoning in LLMs](https://md.research.elamur.ai/free/en/papers/yeo-2025-demystifying-long-chain-thought-reasoning/)
- [MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning](https://md.research.elamur.ai/free/en/papers/yue-2023-mammoth-building-math-generalist-models/)
- [MAmmoTH2: Scaling Instructions from the Web](https://md.research.elamur.ai/free/en/papers/yue-2024-mammoth2-scaling-instructions-web/)
- [ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Search](https://md.research.elamur.ai/free/en/papers/zhang-2024-rest-mcts-llm-self-training/)
- [JudgeLM: Fine-tuned Large Language Models are Scalable Judges](https://md.research.elamur.ai/free/en/papers/zhu-2023-judgelm-fine-tuned-large-language/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{yu2023metamath,
  title = {{MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models}},
  author = {Yu, Longhui and Jiang, Weisen and Shi, Han and Yu, Jincheng and Liu, Zhengying and Zhang, Yu and Kwok, James T. and Li, Zhenguo and Weller, Adrian and Liu, Weiyang},
  year = {2023},
  doi = {10.48550/arxiv.2309.12284},
  eprint = {2309.12284},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2309.12284}
}
```

- [← previous MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://md.research.elamur.ai/free/en/papers/hong-2023-metagpt-meta-programming-multi/)
- [next → MetricX-23: The Google Submission to the WMT 2023 Metrics Shared Task](https://md.research.elamur.ai/free/en/papers/juraska-2023-metricx-23-google-submission-wmt/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/4/)

## References

85 references, 31 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alibaba. Qwen-7b. Technical Report, 2023. [find](https://scholar.google.com/scholar?q=Alibaba%2E+Qwen%2D7b%2E+Technical+Report%2C+2023%2E)
2. R. Anil, A. Dai, O. Firat, M. Johnson, D. Lepikhin, A. Passos, S. Shakeri, E. Taropa, P. Bailey, Z. Chen, E. Chu, J. Clark, L. Shafey, Y. Huang, K. Meier-Hellstern, G. Mishra, E. Moreira, M. Omernick, K. Robinson, S. Ruder, Y. Tay, K. Xiao, Y. Xu, Y. Zhang, G. Abrego, J. Ahn, J. Austin, P. Barham, J… [find](https://scholar.google.com/scholar?q=R%2E+Anil%2C+A%2E+Dai%2C+O%2E+Firat%2C+M%2E+Johnson%2C+D%2E+Lepikhin%2C+A%2E+Passos%2C+S%2E+Shakeri%2C+E%2E+Taropa%2C+P%2E+Bailey%2C+Z%2E+Chen%2C+E%2E+Chu%2C+J%2E+Clark%2C+L%2E+Shafey%2C+Y%2E+Huang%2C+K%2E+Meier%2DHellstern%2C+G%2E+Mishra%2C+E%2E+Moreira%2C+M%2E+Omernick%2C) [our card](https://md.research.elamur.ai/free/en/papers/anil-2023-palm-2-technical-report/)
3. Z. Azerbayev, H. Schoelkopf, K. Paster, M. Dos, S. McAleer, A. Jiang, J. Deng, S. Biderman, and S. Welleck. Llemma: An Open Language Model For Mathematics. In International Conference on Learning Representations, 2024. [find](https://scholar.google.com/scholar?q=Z%2E+Azerbayev%2C+H%2E+Schoelkopf%2C+K%2E+Paster%2C+M%2E+Dos%2C+S%2E+McAleer%2C+A%2E+Jiang%2C+J%2E+Deng%2C+S%2E+Biderman%2C+and+S%2E+Welleck%2E+Llemma%3A+An+Open+Language+Model+For+Mathematics%2E+In+International+Conference+on+Learning+Repr) [our card](https://md.research.elamur.ai/free/en/papers/azerbayev-2023-llemma-open-language-model-mathematics/)
4. BaichuanInc. Baichuan 2. Technical Report, 2023. [find](https://scholar.google.com/scholar?q=BaichuanInc%2E+Baichuan+2%2E+Technical+Report%2C+2023%2E)
5. L. Berglund, M. Tong, M. Kaufmann, M. Balesni, A. Stickland, T. Korbak, and O. Evans. The Reversal Curse: LLMs Trained on “A is B” Fail to Learn “B is A”. In International Conference on Learning Representations, 2024. [find](https://scholar.google.com/scholar?q=L%2E+Berglund%2C+M%2E+Tong%2C+M%2E+Kaufmann%2C+M%2E+Balesni%2C+A%2E+Stickland%2C+T%2E+Korbak%2C+and+O%2E+Evans%2E+The+Reversal+Curse%3A+LLMs+Trained+on+%E2%80%9CA+is+B%E2%80%9D+Fail+to+Learn+%E2%80%9CB+is+A%E2%80%9D%2E+In+International+Conference+on+Learning+Repre)
6. J. Bilmes. Submodularity In Machine Learning and Artificial Intelligence. arXiv:2202.00132, 2022. — [arXiv:2202.00132](https://arxiv.org/abs/2202.00132) [find](https://scholar.google.com/scholar?q=J%2E+Bilmes%2E+Submodularity+In+Machine+Learning+and+Artificial+Intelligence%2E+arXiv%3A2202%2E00132%2C+2022%2E+%E2%80%94+%5BarXiv%3A2202%2E00132%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2202%2E00132%29)
7. T. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S… [find](https://scholar.google.com/scholar?q=T%2E+Brown%2C+B%2E+Mann%2C+N%2E+Ryder%2C+M%2E+Subbiah%2C+J%2E+Kaplan%2C+P%2E+Dhariwal%2C+A%2E+Neelakantan%2C+P%2E+Shyam%2C+G%2E+Sastry%2C+A%2E+Askell%2C+S%2E+Agarwal%2C+A%2E+Herbert%2DVoss%2C+G%2E+Krueger%2C+T%2E+Henighan%2C+R%2E+Child%2C+A%2E+Ramesh%2C+D%2E+Ziegler%2C+)
8. M. Chen, J. Tworek, H. Jun, Q. Yuan, H. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. Such, D. Cummings, M. Pl… [find](https://scholar.google.com/scholar?q=M%2E+Chen%2C+J%2E+Tworek%2C+H%2E+Jun%2C+Q%2E+Yuan%2C+H%2E+Pinto%2C+J%2E+Kaplan%2C+H%2E+Edwards%2C+Y%2E+Burda%2C+N%2E+Joseph%2C+G%2E+Brockman%2C+A%2E+Ray%2C+R%2E+Puri%2C+G%2E+Krueger%2C+M%2E+Petrov%2C+H%2E+Khlaaf%2C+G%2E+Sastry%2C+P%2E+Mishkin%2C+B%2E+Chan%2C+S%2E+Gray%2C+N%2E+R) [our card](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
9. W. Chen, X. Ma, X. Wang, and W. Cohen. Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks. Preprint arXiv:2211.12588, 2022. Preprint [find](https://scholar.google.com/scholar?q=W%2E+Chen%2C+X%2E+Ma%2C+X%2E+Wang%2C+and+W%2E+Cohen%2E+Program+of+Thoughts+Prompting%3A+Disentangling+Computation+from+Reasoning+for+Numerical+Reasoning+Tasks%2E+Preprint+arXiv%3A2211%2E12588%2C+2022%2E+Preprint)
10. Y. Chen, R. Zhong, S. Zha, G. Karypis, and H. He. Meta-learning via Language Model In-context Tuning. In Annual Meeting of the Association for Computational Linguistics, 2022. [find](https://scholar.google.com/scholar?q=Y%2E+Chen%2C+R%2E+Zhong%2C+S%2E+Zha%2C+G%2E+Karypis%2C+and+H%2E+He%2E+Meta%2Dlearning+via+Language+Model+In%2Dcontext+Tuning%2E+In+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+2022%2E)
11. W. Chiang, Z. Li, Z. Lin, Y. Sheng, Z. Wu, H. Zhang, L. Zheng, S. Zhuang, Y. Zhuang, J. Gonzalez, I. Stoica, and E. Xing. Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90% ChatGPT Quality. Technical Report, 2023. [find](https://scholar.google.com/scholar?q=W%2E+Chiang%2C+Z%2E+Li%2C+Z%2E+Lin%2C+Y%2E+Sheng%2C+Z%2E+Wu%2C+H%2E+Zhang%2C+L%2E+Zheng%2C+S%2E+Zhuang%2C+Y%2E+Zhuang%2C+J%2E+Gonzalez%2C+I%2E+Stoica%2C+and+E%2E+Xing%2E+Vicuna%3A+An+Open%2DSource+Chatbot+Impressing+GPT%2D4+with+90%25+ChatGPT+Quality%2E+Tech)
12. A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. Chung, C. Sutton, S. Gehrmann, P. Schuh, K. Shi, S. Tsvyashchenko, J. Maynez, A. Rao, P. Barnes, Y. Tay, N. Shazeer, V. Prabhakaran, E. Reif, N. Du, B. Hutchinson, R. Pope, J. Bradbury, J. Austin, M. Isard, G. Gur-Ari… [find](https://scholar.google.com/scholar?q=A%2E+Chowdhery%2C+S%2E+Narang%2C+J%2E+Devlin%2C+M%2E+Bosma%2C+G%2E+Mishra%2C+A%2E+Roberts%2C+P%2E+Barham%2C+H%2E+Chung%2C+C%2E+Sutton%2C+S%2E+Gehrmann%2C+P%2E+Schuh%2C+K%2E+Shi%2C+S%2E+Tsvyashchenko%2C+J%2E+Maynez%2C+A%2E+Rao%2C+P%2E+Barnes%2C+Y%2E+Tay%2C+N%2E+Shazeer%2C+)
13. K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman. Training Verifiers to Solve Math Word Problems. Preprint arXiv:2110.14168, 2021. [find](https://scholar.google.com/scholar?q=K%2E+Cobbe%2C+V%2E+Kosaraju%2C+M%2E+Bavarian%2C+M%2E+Chen%2C+H%2E+Jun%2C+L%2E+Kaiser%2C+M%2E+Plappert%2C+J%2E+Tworek%2C+J%2E+Hilton%2C+R%2E+Nakano%2C+C%2E+Hesse%2C+and+J%2E+Schulman%2E+Training+Verifiers+to+Solve+Math+Word+Problems%2E+Preprint+arXiv%3A) [our card](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/)
14. K. Collins, A. Jiang, S. Frieder, L. Wong, M. Zilka, U. Bhatt, T. Lukasiewicz, Y. Wu, J. Tenenbaum, W. Hart, T. Gowers, W. Li, A. Weller, and M. Jamnik. Evaluating Language Models for Mathematics through Interactions. Preprint arXiv:2306.01694, 2023. [find](https://scholar.google.com/scholar?q=K%2E+Collins%2C+A%2E+Jiang%2C+S%2E+Frieder%2C+L%2E+Wong%2C+M%2E+Zilka%2C+U%2E+Bhatt%2C+T%2E+Lukasiewicz%2C+Y%2E+Wu%2C+J%2E+Tenenbaum%2C+W%2E+Hart%2C+T%2E+Gowers%2C+W%2E+Li%2C+A%2E+Weller%2C+and+M%2E+Jamnik%2E+Evaluating+Language+Models+for+Mathematics+thro)
15. T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer. QLoRA: Efficient finetuning of quantized llms. Preprint arXiv:2305.14314, 2023. [find](https://scholar.google.com/scholar?q=T%2E+Dettmers%2C+A%2E+Pagnoni%2C+A%2E+Holtzman%2C+and+L%2E+Zettlemoyer%2E+QLoRA%3A+Efficient+finetuning+of+quantized+llms%2E+Preprint+arXiv%3A2305%2E14314%2C+2023%2E)
16. J. Devlin, M. Chang, K. Lee, and K. Toutanova. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In North American Chapter of the Association for Computational Linguistics, 2019. [find](https://scholar.google.com/scholar?q=J%2E+Devlin%2C+M%2E+Chang%2C+K%2E+Lee%2C+and+K%2E+Toutanova%2E+BERT%3A+Pre%2Dtraining+of+Deep+Bidirectional+Transformers+for+Language+Understanding%2E+In+North+American+Chapter+of+the+Association+for+Computational+Linguist)
17. D. Dua, Y. Wang, P. Dasigi, G. Stanovsky, S. Singh, and M. Gardner. DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs. In North American Chapter of the Association for Computational Linguistics, 2019. [find](https://scholar.google.com/scholar?q=D%2E+Dua%2C+Y%2E+Wang%2C+P%2E+Dasigi%2C+G%2E+Stanovsky%2C+S%2E+Singh%2C+and+M%2E+Gardner%2E+DROP%3A+A+Reading+Comprehension+Benchmark+Requiring+Discrete+Reasoning+Over+Paragraphs%2E+In+North+American+Chapter+of+the+Association+f) [our card](https://md.research.elamur.ai/free/en/papers/dua-2019-drop-reading-comprehension-benchmark-requiring/)
18. R. Eldan and Y. Li. TinyStories: How Small Can Language Models Be and Still Speak Coherent English? Preprint arXiv:2305.07759, 2023. [find](https://scholar.google.com/scholar?q=R%2E+Eldan+and+Y%2E+Li%2E+TinyStories%3A+How+Small+Can+Language+Models+Be+and+Still+Speak+Coherent+English%3F+Preprint+arXiv%3A2305%2E07759%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/eldan-2023-tinystories-small-language-models-still/)
19. Y. Fu, H. Peng, L. Ou, A. Sabharwal, and T. Khot. Specializing Smaller Language Models towards Multi-Step Reasoning. In International Conference on Machine Learning, 2023. [find](https://scholar.google.com/scholar?q=Y%2E+Fu%2C+H%2E+Peng%2C+L%2E+Ou%2C+A%2E+Sabharwal%2C+and+T%2E+Khot%2E+Specializing+Smaller+Language+Models+towards+Multi%2DStep+Reasoning%2E+In+International+Conference+on+Machine+Learning%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/fu-2023-specializing-smaller-language-models-towards/)
20. Y. Fu, H. Peng, A. Sabharwal, P. Clark, and T. Khot. Complexity-Based Prompting for Multi-step Reasoning. In International Conference on Learning Representations, 2023. [find](https://scholar.google.com/scholar?q=Y%2E+Fu%2C+H%2E+Peng%2C+A%2E+Sabharwal%2C+P%2E+Clark%2C+and+T%2E+Khot%2E+Complexity%2DBased+Prompting+for+Multi%2Dstep+Reasoning%2E+In+International+Conference+on+Learning+Representations%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/fu-2022-complexity-based-prompting-multi-step/)
21. J. Gou, B. Yu, S. Maybank, and D. Tao. Knowledge Distillation: A Survey. International Journal of Computer Vision, 2021. [find](https://scholar.google.com/scholar?q=J%2E+Gou%2C+B%2E+Yu%2C+S%2E+Maybank%2C+and+D%2E+Tao%2E+Knowledge+Distillation%3A+A+Survey%2E+International+Journal+of+Computer+Vision%2C+2021%2E)
22. T. He, C. Shen, Z. Tian, D. Gong, C. Sun, and Y. Yan. Knowledge Adaptation for Efficient Semantic Segmentation. In Computer Vision and Pattern Recognition, 2019. [find](https://scholar.google.com/scholar?q=T%2E+He%2C+C%2E+Shen%2C+Z%2E+Tian%2C+D%2E+Gong%2C+C%2E+Sun%2C+and+Y%2E+Yan%2E+Knowledge+Adaptation+for+Efficient+Semantic+Segmentation%2E+In+Computer+Vision+and+Pattern+Recognition%2C+2019%2E)
23. D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt. Measuring Mathematical Problem Solving With the MATH Dataset. In Neural Information Processing Systems: Datasets and Benchmarks, 2021. [find](https://scholar.google.com/scholar?q=D%2E+Hendrycks%2C+C%2E+Burns%2C+S%2E+Kadavath%2C+A%2E+Arora%2C+S%2E+Basart%2C+E%2E+Tang%2C+D%2E+Song%2C+and+J%2E+Steinhardt%2E+Measuring+Mathematical+Problem+Solving+With+the+MATH+Dataset%2E+In+Neural+Information+Processing+Systems%3A+D) [our card](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/)
24. G. Hinton, O. Vinyals, and J. Dean. Distilling the Knowledge in a Neural Network. Preprint arXiv:1503.02531, 2015. [find](https://scholar.google.com/scholar?q=G%2E+Hinton%2C+O%2E+Vinyals%2C+and+J%2E+Dean%2E+Distilling+the+Knowledge+in+a+Neural+Network%2E+Preprint+arXiv%3A1503%2E02531%2C+2015%2E) [our card](https://md.research.elamur.ai/free/en/papers/hinton-2015-distilling-knowledge-neural-network/)
25. N. Ho, L. Schmid, and S. Yun. Large Language Models Are Reasoning Teachers. In Annual Meeting of the Association for Computational Linguistics, 2023. [find](https://scholar.google.com/scholar?q=N%2E+Ho%2C+L%2E+Schmid%2C+and+S%2E+Yun%2E+Large+Language+Models+Are+Reasoning+Teachers%2E+In+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/ho-2023-large-language-models-are-reasoning/)
26. C. Hsieh, C. Li, C. Yeh, H. Nakhost, Y. Fujii, A. Ratner, R. Krishna, C. Lee, and T. Pfister. Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes. In Annual Meeting of the Association for Computational Linguistics, 2023. [find](https://scholar.google.com/scholar?q=C%2E+Hsieh%2C+C%2E+Li%2C+C%2E+Yeh%2C+H%2E+Nakhost%2C+Y%2E+Fujii%2C+A%2E+Ratner%2C+R%2E+Krishna%2C+C%2E+Lee%2C+and+T%2E+Pfister%2E+Distilling+Step%2Dby%2DStep%21+Outperforming+Larger+Language+Models+with+Less+Training+Data+and+Smaller+Model+Si) [our card](https://md.research.elamur.ai/free/en/papers/hsieh-2023-distilling-step-step-outperforming-larger/)
27. J. Huang, S. Gu, L. Hou, Y. Wu, X. Wang, H. Yu, and J. Han. Large Language Models Can Self-Improve. Preprint arXiv:2210.11610, 2022. [find](https://scholar.google.com/scholar?q=J%2E+Huang%2C+S%2E+Gu%2C+L%2E+Hou%2C+Y%2E+Wu%2C+X%2E+Wang%2C+H%2E+Yu%2C+and+J%2E+Han%2E+Large+Language+Models+Can+Self%2DImprove%2E+Preprint+arXiv%3A2210%2E11610%2C+2022%2E) [our card](https://md.research.elamur.ai/free/en/papers/huang-2023-large-language-models-can-self/)
28. S. Imani, L. Du, and H. Shrivastava. MathPrompter: Mathematical Reasoning using Large Language Models. In Annual Meeting of the Association for Computational Linguistics, 2023. [find](https://scholar.google.com/scholar?q=S%2E+Imani%2C+L%2E+Du%2C+and+H%2E+Shrivastava%2E+MathPrompter%3A+Mathematical+Reasoning+using+Large+Language+Models%2E+In+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/imani-2023-mathprompter-mathematical-reasoning-large-language/)
29. InternLM. InternLM: A Multilingual Language Model with Progressively Enhanced Capabilities. Technical Report, 2023. [find](https://scholar.google.com/scholar?q=InternLM%2E+InternLM%3A+A+Multilingual+Language+Model+with+Progressively+Enhanced+Capabilities%2E+Technical+Report%2C+2023%2E)
30. A. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. Chaplot, F. Bressand D. Casas, G. Lengyel, G. Lample, L. Saulnier, L. Lavaud, M. Lachaux, P. Stock, T. Scao, T. Lavril, T. Wang, and T. Lacroixand W. Sayed. Mistral 7B. Preprint arXiv:2310.06825, 2023. [find](https://scholar.google.com/scholar?q=A%2E+Jiang%2C+A%2E+Sablayrolles%2C+A%2E+Mensch%2C+C%2E+Bamford%2C+D%2E+Chaplot%2C+F%2E+Bressand+D%2E+Casas%2C+G%2E+Lengyel%2C+G%2E+Lample%2C+L%2E+Saulnier%2C+L%2E+Lavaud%2C+M%2E+Lachaux%2C+P%2E+Stock%2C+T%2E+Scao%2C+T%2E+Lavril%2C+T%2E+Wang%2C+and+T%2E+Lacroixand+)
31. W. Jiang, B. Lin, H. Shi, Y. Zhang, Z. Li, and J. Kwok. BYOM: Building Your Own Multi-Task Model for Free. Preprint arXiv:2310.01886, 2023. — [arXiv:2310.01886](https://arxiv.org/abs/2310.01886) [find](https://scholar.google.com/scholar?q=W%2E+Jiang%2C+B%2E+Lin%2C+H%2E+Shi%2C+Y%2E+Zhang%2C+Z%2E+Li%2C+and+J%2E+Kwok%2E+BYOM%3A+Building+Your+Own+Multi%2DTask+Model+for+Free%2E+Preprint+arXiv%3A2310%2E01886%2C+2023%2E+%E2%80%94+%5BarXiv%3A2310%2E01886%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2310%2E01886%29)
32. W. Jiang, H. Shi, L. Yu, Z. Liu, Y. Zhang, Z. Li, and J. Kwok. Forward-Backward Reasoning in Large Language Models for Mathematical Verification. Preprint arXiv:2308.07758, 2023. [find](https://scholar.google.com/scholar?q=W%2E+Jiang%2C+H%2E+Shi%2C+L%2E+Yu%2C+Z%2E+Liu%2C+Y%2E+Zhang%2C+Z%2E+Li%2C+and+J%2E+Kwok%2E+Forward%2DBackward+Reasoning+in+Large+Language+Models+for+Mathematical+Verification%2E+Preprint+arXiv%3A2308%2E07758%2C+2023%2E)
33. W. Jiang, Y. Zhang, and J. Kwok. Effective Structured-Prompting by Meta-Learning and Representitive Verbalizer. In International Conference on Machine Learning, 2023. [find](https://scholar.google.com/scholar?q=W%2E+Jiang%2C+Y%2E+Zhang%2C+and+J%2E+Kwok%2E+Effective+Structured%2DPrompting+by+Meta%2DLearning+and+Representitive+Verbalizer%2E+In+International+Conference+on+Machine+Learning%2C+2023%2E)
34. N. Kilbertus, G. Parascandolo, and B. Schölkopf. Generalization in anti-causal learning. Preprint arXiv:1812.00524, 2018. — [arXiv:1812.00524](https://arxiv.org/abs/1812.00524) [find](https://scholar.google.com/scholar?q=N%2E+Kilbertus%2C+G%2E+Parascandolo%2C+and+B%2E+Scho%CC%88lkopf%2E+Generalization+in+anti%2Dcausal+learning%2E+Preprint+arXiv%3A1812%2E00524%2C+2018%2E+%E2%80%94+%5BarXiv%3A1812%2E00524%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1812%2E00524%29)
35. A. Lewkowycz, A. Andreassen, D. Dohan, E. Dyer, H. Michalewski, V. Ramasesh, A. Slone, C. Anil, I. Schlag, T. Gutman-Solo, Y. Wu, B. Neyshabur, G. Gur-Ari, and V. Misra. Solving Quantitative Reasoning Problems with Language Models. In Neural Information Processing Systems, 2022. [find](https://scholar.google.com/scholar?q=A%2E+Lewkowycz%2C+A%2E+Andreassen%2C+D%2E+Dohan%2C+E%2E+Dyer%2C+H%2E+Michalewski%2C+V%2E+Ramasesh%2C+A%2E+Slone%2C+C%2E+Anil%2C+I%2E+Schlag%2C+T%2E+Gutman%2DSolo%2C+Y%2E+Wu%2C+B%2E+Neyshabur%2C+G%2E+Gur%2DAri%2C+and+V%2E+Misra%2E+Solving+Quantitative+Reasoning) [our card](https://md.research.elamur.ai/free/en/papers/lewkowycz-2022-solving-quantitative-reasoning-problems-language/)
36. R. Li, L. Allal, Y. Zi, N. Muennighoff, D. Kocetkov, C. Mou, M. Marone, C. Akiki, J. Li, J. Chim, Q. Liu, E. Zheltonozhskii, T. Zhuo, T. Wang, O. Dehaene, M. Davaadorj, J. Lamy-Poirier, J. Monteiro, O. Shliazhko, N. Gontier, N. Meade, A. Zebaze, M. Yee, L. Umapathi, J. Zhu, B. Lipkin, M. Oblokulov,… [find](https://scholar.google.com/scholar?q=R%2E+Li%2C+L%2E+Allal%2C+Y%2E+Zi%2C+N%2E+Muennighoff%2C+D%2E+Kocetkov%2C+C%2E+Mou%2C+M%2E+Marone%2C+C%2E+Akiki%2C+J%2E+Li%2C+J%2E+Chim%2C+Q%2E+Liu%2C+E%2E+Zheltonozhskii%2C+T%2E+Zhuo%2C+T%2E+Wang%2C+O%2E+Dehaene%2C+M%2E+Davaadorj%2C+J%2E+Lamy%2DPoirier%2C+J%2E+Monteiro%2C+O)
37. S. Li, J. Chen, Y. Shen, Z. Chen, X. Zhang, Z. Li, H. Wang, J. Qian, B. Peng, Y. Mao, W. Chen, and X. Yan. Explanations from Large Language Models Make Small Reasoners Better. Preprint arXiv:2210.06726, 2022. [find](https://scholar.google.com/scholar?q=S%2E+Li%2C+J%2E+Chen%2C+Y%2E+Shen%2C+Z%2E+Chen%2C+X%2E+Zhang%2C+Z%2E+Li%2C+H%2E+Wang%2C+J%2E+Qian%2C+B%2E+Peng%2C+Y%2E+Mao%2C+W%2E+Chen%2C+and+X%2E+Yan%2E+Explanations+from+Large+Language+Models+Make+Small+Reasoners+Better%2E+Preprint+arXiv%3A2210%2E0672)
38. X. Li, Z. Zhou, J. Zhu, J. Yao, T. Liu, and B. Han. DeepInception: Hypnotize Large Language Model to be Jailbreaker. Preprint arXiv:2311.03191, 2023. [find](https://scholar.google.com/scholar?q=X%2E+Li%2C+Z%2E+Zhou%2C+J%2E+Zhu%2C+J%2E+Yao%2C+T%2E+Liu%2C+and+B%2E+Han%2E+DeepInception%3A+Hypnotize+Large+Language+Model+to+be+Jailbreaker%2E+Preprint+arXiv%3A2311%2E03191%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/li-2023-deepinception-hypnotize-large-language-model/)
39. H. Lightman, V. Kosaraju, Y. Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, and K. Cobbe. Let’s Verify Step by Step. In International Conference on Learning Representations, 2024. [find](https://scholar.google.com/scholar?q=H%2E+Lightman%2C+V%2E+Kosaraju%2C+Y%2E+Burda%2C+H%2E+Edwards%2C+B%2E+Baker%2C+T%2E+Lee%2C+J%2E+Leike%2C+J%2E+Schulman%2C+I%2E+Sutskever%2C+and+K%2E+Cobbe%2E+Let%E2%80%99s+Verify+Step+by+Step%2E+In+International+Conference+on+Learning+Representations%2C) [our card](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/)
40. W. Liu, B. Dai, A. Humayun, C. Tay, C. Yu, L. Smith, J. Rehg, and L. Song. Iterative Machine Teaching. In International Conference on Machine Learning, 2017. [find](https://scholar.google.com/scholar?q=W%2E+Liu%2C+B%2E+Dai%2C+A%2E+Humayun%2C+C%2E+Tay%2C+C%2E+Yu%2C+L%2E+Smith%2C+J%2E+Rehg%2C+and+L%2E+Song%2E+Iterative+Machine+Teaching%2E+In+International+Conference+on+Machine+Learning%2C+2017%2E)
41. W. Liu, Z. Liu, H. Wang, L. Paull, B. Schölkopf, and A. Weller. Iterative Teaching by Label Synthesis. In Neural Information Processing Systems, 2021. [find](https://scholar.google.com/scholar?q=W%2E+Liu%2C+Z%2E+Liu%2C+H%2E+Wang%2C+L%2E+Paull%2C+B%2E+Scho%CC%88lkopf%2C+and+A%2E+Weller%2E+Iterative+Teaching+by+Label+Synthesis%2E+In+Neural+Information+Processing+Systems%2C+2021%2E)
42. Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov. RoBERTa: A Robustly Optimized BERT Pretraining Approach. Preprint arXiv:1907.11692, 2019. [find](https://scholar.google.com/scholar?q=Y%2E+Liu%2C+M%2E+Ott%2C+N%2E+Goyal%2C+J%2E+Du%2C+M%2E+Joshi%2C+D%2E+Chen%2C+O%2E+Levy%2C+M%2E+Lewis%2C+L%2E+Zettlemoyer%2C+and+V%2E+Stoyanov%2E+RoBERTa%3A+A+Robustly+Optimized+BERT+Pretraining+Approach%2E+Preprint+arXiv%3A1907%2E11692%2C+2019%2E)
43. H. Luo, Q. Sun, C. Xu, P. Zhao, J. Lou, C. Tao, X. Geng, Q. Lin, S. Chen, and D. Zhang. WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct. Preprint arXiv:2308.09583, 2023. [find](https://scholar.google.com/scholar?q=H%2E+Luo%2C+Q%2E+Sun%2C+C%2E+Xu%2C+P%2E+Zhao%2C+J%2E+Lou%2C+C%2E+Tao%2C+X%2E+Geng%2C+Q%2E+Lin%2C+S%2E+Chen%2C+and+D%2E+Zhang%2E+WizardMath%3A+Empowering+Mathematical+Reasoning+for+Large+Language+Models+via+Reinforced+Evol%2DInstruct%2E+Preprint+a) [our card](https://md.research.elamur.ai/free/en/papers/luo-2023-wizardmath-empowering-mathematical-reasoning-large/)
44. Z. Luo, C. Xu, P. Zhao, Q. Sun, X. Geng, W. Hu, C. Tao, J. Ma, Q. Lin, and D. Jiang. WizardCoder: Empowering Code Large Language Models with Evol-Instruct. In International Conference on Learning Representations, 2024. [find](https://scholar.google.com/scholar?q=Z%2E+Luo%2C+C%2E+Xu%2C+P%2E+Zhao%2C+Q%2E+Sun%2C+X%2E+Geng%2C+W%2E+Hu%2C+C%2E+Tao%2C+J%2E+Ma%2C+Q%2E+Lin%2C+and+D%2E+Jiang%2E+WizardCoder%3A+Empowering+Code+Large+Language+Models+with+Evol%2DInstruct%2E+In+International+Conference+on+Learning+Repr)
45. L. Magister, J. Mallinson, J. Adamek, E. Malmi, and A. Severyn. Teaching Small Language Models to Reason. In Annual Meeting of the Association for Computational Linguistics, 2023. [find](https://scholar.google.com/scholar?q=L%2E+Magister%2C+J%2E+Mallinson%2C+J%2E+Adamek%2C+E%2E+Malmi%2C+and+A%2E+Severyn%2E+Teaching+Small+Language+Models+to+Reason%2E+In+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/magister-2023-teaching-small-language-models-reason/)
46. M. Marion, A. Üstün, L. Pozzobon, A. Wang, M. Fadaee, and S. Hooker. When Less is More: Investigating Data Pruning for Pretraining LLMs at Scale. Preprint arXiv:2309.04564, 2023. [find](https://scholar.google.com/scholar?q=M%2E+Marion%2C+A%2E+U%CC%88stu%CC%88n%2C+L%2E+Pozzobon%2C+A%2E+Wang%2C+M%2E+Fadaee%2C+and+S%2E+Hooker%2E+When+Less+is+More%3A+Investigating+Data+Pruning+for+Pretraining+LLMs+at+Scale%2E+Preprint+arXiv%3A2309%2E04564%2C+2023%2E)
47. S. Min, M. Lewis, L. Zettlemoyer, and H. Hajishirzi. MetaICL: Learning to Learn In Context. In North American Chapter of the Association for Computational Linguistics, 2022. [find](https://scholar.google.com/scholar?q=S%2E+Min%2C+M%2E+Lewis%2C+L%2E+Zettlemoyer%2C+and+H%2E+Hajishirzi%2E+MetaICL%3A+Learning+to+Learn+In+Context%2E+In+North+American+Chapter+of+the+Association+for+Computational+Linguistics%2C+2022%2E)
48. S. Mirzadeh, M. Farajtabar, A. Li, N. Levine, A. Matsukawa, and H. Ghasemzadeh. Improved Knowledge Distillation via Teacher Assistant. In AAAI Conference on Artificial Intelligence, 2020. [find](https://scholar.google.com/scholar?q=S%2E+Mirzadeh%2C+M%2E+Farajtabar%2C+A%2E+Li%2C+N%2E+Levine%2C+A%2E+Matsukawa%2C+and+H%2E+Ghasemzadeh%2E+Improved+Knowledge+Distillation+via+Teacher+Assistant%2E+In+AAAI+Conference+on+Artificial+Intelligence%2C+2020%2E) [our card](https://md.research.elamur.ai/free/en/papers/mirzadeh-2019-improved-knowledge-distillation-teacher-assistant/)
49. MosaicML. Introducing MPT-7B: A New Standard for Open-Source, Commercially Usable LLMs. Technical Report, 2023. [find](https://scholar.google.com/scholar?q=MosaicML%2E+Introducing+MPT%2D7B%3A+A+New+Standard+for+Open%2DSource%2C+Commercially+Usable+LLMs%2E+Technical+Report%2C+2023%2E)
50. Ariel N., Cole J., and Nataniel R. Platypus: Quick, Cheap, and Powerful Refinement of LLMs. Preprint arXiv:2308.07317, 2023. [find](https://scholar.google.com/scholar?q=Ariel+N%2E%2C+Cole+J%2E%2C+and+Nataniel+R%2E+Platypus%3A+Quick%2C+Cheap%2C+and+Powerful+Refinement+of+LLMs%2E+Preprint+arXiv%3A2308%2E07317%2C+2023%2E)
51. E. Nijkamp, B. Pang, H. Hayashi, L. Tu, H. Wang, Y. Zhou, S. Savarese, and C. Xiong. CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis. Preprint arXiv:2203.13474, 2022. [find](https://scholar.google.com/scholar?q=E%2E+Nijkamp%2C+B%2E+Pang%2C+H%2E+Hayashi%2C+L%2E+Tu%2C+H%2E+Wang%2C+Y%2E+Zhou%2C+S%2E+Savarese%2C+and+C%2E+Xiong%2E+CodeGen%3A+An+Open+Large+Language+Model+for+Code+with+Multi%2DTurn+Program+Synthesis%2E+Preprint+arXiv%3A2203%2E13474%2C+2022%2E)
52. OpenAI. GPT-3.5. Technical Report, 2022. [find](https://scholar.google.com/scholar?q=OpenAI%2E+GPT%2D3%2E5%2E+Technical+Report%2C+2022%2E)
53. OpenAI. GPT-3.5-Turbo. Technical Report, 2022. [find](https://scholar.google.com/scholar?q=OpenAI%2E+GPT%2D3%2E5%2DTurbo%2E+Technical+Report%2C+2022%2E)
54. OpenAI. GPT-4. Technical Report, 2023. [find](https://scholar.google.com/scholar?q=OpenAI%2E+GPT%2D4%2E+Technical+Report%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/)
55. L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, and R. Lowe. Training Language Models to Follow Instructions with Human Feedback. In Ne… [find](https://scholar.google.com/scholar?q=L%2E+Ouyang%2C+J%2E+Wu%2C+X%2E+Jiang%2C+D%2E+Almeida%2C+C%2E+Wainwright%2C+P%2E+Mishkin%2C+C%2E+Zhang%2C+S%2E+Agarwal%2C+K%2E+Slama%2C+A%2E+Ray%2C+J%2E+Schulman%2C+J%2E+Hilton%2C+F%2E+Kelton%2C+L%2E+Miller%2C+M%2E+Simens%2C+A%2E+Askell%2C+P%2E+Welinder%2C+P%2E+Christian)
56. W. Park, D. Kim, Y. Lu, and M. Cho. Relational Knowledge Distillation. In Computer Vision and Pattern Recognition, 2019. [find](https://scholar.google.com/scholar?q=W%2E+Park%2C+D%2E+Kim%2C+Y%2E+Lu%2C+and+M%2E+Cho%2E+Relational+Knowledge+Distillation%2E+In+Computer+Vision+and+Pattern+Recognition%2C+2019%2E)
57. G. Penedo, Q. Malartic, D. Hesslow, R. Cojocaru, A. Cappelli, H. Alobeidli, B. Pannier, E. Almazrouei, and J. Launay. The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Only. Preprint arXiv:2306.01116, 2023. [find](https://scholar.google.com/scholar?q=G%2E+Penedo%2C+Q%2E+Malartic%2C+D%2E+Hesslow%2C+R%2E+Cojocaru%2C+A%2E+Cappelli%2C+H%2E+Alobeidli%2C+B%2E+Pannier%2C+E%2E+Almazrouei%2C+and+J%2E+Launay%2E+The+RefinedWeb+Dataset+for+Falcon+LLM%3A+Outperforming+Curated+Corpora+with+Web+Data)
58. Z. Qiu, W. Liu, T. Xiao, Z. Liu, U. Bhatt, Y. Luo, A. Weller, and B. Schölkopf. Iterative Teaching by Data Hallucination. In Artificial Intelligence and Statistics, 2023. [find](https://scholar.google.com/scholar?q=Z%2E+Qiu%2C+W%2E+Liu%2C+T%2E+Xiao%2C+Z%2E+Liu%2C+U%2E+Bhatt%2C+Y%2E+Luo%2C+A%2E+Weller%2C+and+B%2E+Scho%CC%88lkopf%2E+Iterative+Teaching+by+Data+Hallucination%2E+In+Artificial+Intelligence+and+Statistics%2C+2023%2E)
59. A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever. Language Models are Unsupervised Multitask Learners. Technical Report, 2019. [find](https://scholar.google.com/scholar?q=A%2E+Radford%2C+J%2E+Wu%2C+R%2E+Child%2C+D%2E+Luan%2C+D%2E+Amodei%2C+and+I%2E+Sutskever%2E+Language+Models+are+Unsupervised+Multitask+Learners%2E+Technical+Report%2C+2019%2E)
60. C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. Liu. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. Journal of Machine Learning Research, 2020. [find](https://scholar.google.com/scholar?q=C%2E+Raffel%2C+N%2E+Shazeer%2C+A%2E+Roberts%2C+K%2E+Lee%2C+S%2E+Narang%2C+M%2E+Matena%2C+Y%2E+Zhou%2C+W%2E+Li%2C+and+P%2E+Liu%2E+Exploring+the+Limits+of+Transfer+Learning+with+a+Unified+Text%2Dto%2DText+Transformer%2E+Journal+of+Machine+Learn)
61. B. Rozière, J. Gehring, F. Gloeckle, S. Sootla, I. Gat, X. Tan, Y. Adi, J. Liu, T. Remez, J. Rapin, A. Kozhevnikov, I. Evtimov, J. Bitton, M. Bhatt, C. Ferrer, A. Grattafiori, W. Xiong, A. Défossez, J. Copet, F. Azhar, H. Touvron, L. Martin, N. Usunier, T. Scialom, and G. Synnaeve. Code Llama: Ope… [find](https://scholar.google.com/scholar?q=B%2E+Rozie%CC%80re%2C+J%2E+Gehring%2C+F%2E+Gloeckle%2C+S%2E+Sootla%2C+I%2E+Gat%2C+X%2E+Tan%2C+Y%2E+Adi%2C+J%2E+Liu%2C+T%2E+Remez%2C+J%2E+Rapin%2C+A%2E+Kozhevnikov%2C+I%2E+Evtimov%2C+J%2E+Bitton%2C+M%2E+Bhatt%2C+C%2E+Ferrer%2C+A%2E+Grattafiori%2C+W%2E+Xiong%2C+A%2E+De%CC%81fossez%2C) [our card](https://md.research.elamur.ai/free/en/papers/roziere-2024-code-llama-open-foundation-models/)
62. J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal Policy Optimization Algorithms. Preprint arXiv:1707.06347, 2017. [find](https://scholar.google.com/scholar?q=J%2E+Schulman%2C+F%2E+Wolski%2C+P%2E+Dhariwal%2C+A%2E+Radford%2C+and+O%2E+Klimov%2E+Proximal+Policy+Optimization+Algorithms%2E+Preprint+arXiv%3A1707%2E06347%2C+2017%2E)
63. P. Shen, X. Lu, S. Li, and H. Kawai. Feature Representation of Short Utterances Based on Knowledge Distillation for Spoken Language Identification. In International Speech Communication Association, 2018. [find](https://scholar.google.com/scholar?q=P%2E+Shen%2C+X%2E+Lu%2C+S%2E+Li%2C+and+H%2E+Kawai%2E+Feature+Representation+of+Short+Utterances+Based+on+Knowledge+Distillation+for+Spoken+Language+Identification%2E+In+International+Speech+Communication+Association%2C+2)
64. K. Shridhar, A. Stolfo, and M. Sachan. Distilling Reasoning Capabilities into Smaller Language Models. In Findings of the Association for Computational Linguistics, 2023. [find](https://scholar.google.com/scholar?q=K%2E+Shridhar%2C+A%2E+Stolfo%2C+and+M%2E+Sachan%2E+Distilling+Reasoning+Capabilities+into+Smaller+Language+Models%2E+In+Findings+of+the+Association+for+Computational+Linguistics%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/shridhar-2023-distilling-reasoning-capabilities-into-smaller/)
65. J. Sun, C. Zheng, E. Xie, Z. Liu, R. Chu, J. Qiu, et al. A Survey of Reasoning with Foundation Models. Preprint arXiv:2312.11562, 2023. — [arXiv:2312.11562](https://arxiv.org/abs/2312.11562) [find](https://scholar.google.com/scholar?q=J%2E+Sun%2C+C%2E+Zheng%2C+E%2E+Xie%2C+Z%2E+Liu%2C+R%2E+Chu%2C+J%2E+Qiu%2C+et+al%2E+A+Survey+of+Reasoning+with+Foundation+Models%2E+Preprint+arXiv%3A2312%2E11562%2C+2023%2E+%E2%80%94+%5BarXiv%3A2312%2E11562%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2312%2E11562%29)
66. A. Talmor, J. Herzig, N. Lourie, and J. Berant. CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge. In North American Chapter of the Association for Computational Linguistics, 2019. [find](https://scholar.google.com/scholar?q=A%2E+Talmor%2C+J%2E+Herzig%2C+N%2E+Lourie%2C+and+J%2E+Berant%2E+CommonsenseQA%3A+A+Question+Answering+Challenge+Targeting+Commonsense+Knowledge%2E+In+North+American+Chapter+of+the+Association+for+Computational+Linguistic)
67. R. Taori, I. Gulrajani, T. Zhang, Y. Dubois, X. Li, C. Guestrin, P. Liang, and T. Hashimoto. Stanford Alpaca: An Instruction-following LLaMA Model. Technical report, 2023. [find](https://scholar.google.com/scholar?q=R%2E+Taori%2C+I%2E+Gulrajani%2C+T%2E+Zhang%2C+Y%2E+Dubois%2C+X%2E+Li%2C+C%2E+Guestrin%2C+P%2E+Liang%2C+and+T%2E+Hashimoto%2E+Stanford+Alpaca%3A+An+Instruction%2Dfollowing+LLaMA+Model%2E+Technical+report%2C+2023%2E)
68. R. Taylor, M. Kardas, G. Cucurull, T. Scialom, A. Hartshorn, E. Saravia, A. Poulton, V. Kerkez, and R. Stojnic. Galactica: A Large Language Model for Science. Preprint arXiv:2211.09085, 2022. [find](https://scholar.google.com/scholar?q=R%2E+Taylor%2C+M%2E+Kardas%2C+G%2E+Cucurull%2C+T%2E+Scialom%2C+A%2E+Hartshorn%2C+E%2E+Saravia%2C+A%2E+Poulton%2C+V%2E+Kerkez%2C+and+R%2E+Stojnic%2E+Galactica%3A+A+Large+Language+Model+for+Science%2E+Preprint+arXiv%3A2211%2E09085%2C+2022%2E) [our card](https://md.research.elamur.ai/free/en/papers/taylor-2022-galactica-large-language-model-science/)
69. H. Touvron, T. Lavril, G. Izacard, X. Martinet, M. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, and G. Lample. LLaMA: Open and Efficient Foundation Language Models. Preprint arXiv:2302.13971, 2023. [find](https://scholar.google.com/scholar?q=H%2E+Touvron%2C+T%2E+Lavril%2C+G%2E+Izacard%2C+X%2E+Martinet%2C+M%2E+Lachaux%2C+T%2E+Lacroix%2C+B%2E+Rozie%CC%80re%2C+N%2E+Goyal%2C+E%2E+Hambro%2C+F%2E+Azhar%2C+A%2E+Rodriguez%2C+A%2E+Joulin%2C+E%2E+Grave%2C+and+G%2E+Lample%2E+LLaMA%3A+Open+and+Efficient+Foundati)
70. H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, D. Bikel, L. Blecher, C. Ferrer, M. Chen, G. Cucurull, D. Esiobu, J. Fernandes, J. Fu, W. Fu, B. Fuller, C. Gao, V. Goswami, N. Goyal, A. Hartshorn, S. Hosseini, R. Hou, H. Inan, M.… [find](https://scholar.google.com/scholar?q=H%2E+Touvron%2C+L%2E+Martin%2C+K%2E+Stone%2C+P%2E+Albert%2C+A%2E+Almahairi%2C+Y%2E+Babaei%2C+N%2E+Bashlykov%2C+S%2E+Batra%2C+P%2E+Bhargava%2C+S%2E+Bhosale%2C+D%2E+Bikel%2C+L%2E+Blecher%2C+C%2E+Ferrer%2C+M%2E+Chen%2C+G%2E+Cucurull%2C+D%2E+Esiobu%2C+J%2E+Fernandes%2C+J%2E)
71. B. Wang and A. Komatsuzaki. GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model. Technical Report, 2021. [find](https://scholar.google.com/scholar?q=B%2E+Wang+and+A%2E+Komatsuzaki%2E+GPT%2DJ%2D6B%3A+A+6+Billion+Parameter+Autoregressive+Language+Model%2E+Technical+Report%2C+2021%2E)
72. P. Wang, L. Li, L. Chen, F. Song, B. Lin, Y. Cao, T. Liu, and Z. Sui. Making Large Language Models Better Reasoners with Alignment. Preprint arXiv:2309.02144, 2023. — [arXiv:2309.02144](https://arxiv.org/abs/2309.02144) [find](https://scholar.google.com/scholar?q=P%2E+Wang%2C+L%2E+Li%2C+L%2E+Chen%2C+F%2E+Song%2C+B%2E+Lin%2C+Y%2E+Cao%2C+T%2E+Liu%2C+and+Z%2E+Sui%2E+Making+Large+Language+Models+Better+Reasoners+with+Alignment%2E+Preprint+arXiv%3A2309%2E02144%2C+2023%2E+%E2%80%94+%5BarXiv%3A2309%2E02144%5D%28https%3A%2F%2Farxiv%2E)
73. T. Wang, J. Zhu, A. Torralba, and A. Efros. Dataset Distillation. Preprint arXiv:1811.10959, 2018. [find](https://scholar.google.com/scholar?q=T%2E+Wang%2C+J%2E+Zhu%2C+A%2E+Torralba%2C+and+A%2E+Efros%2E+Dataset+Distillation%2E+Preprint+arXiv%3A1811%2E10959%2C+2018%2E)
74. X. Wang, J. Wei, D. Schuurmans, Q. Le, E. Chi, S. Narang, A. Chowdhery, and D. Zhou. Self-Consistency Improves Chain of Thought Reasoning in Language Models. In International Conference on Learning Representations, 2023. [find](https://scholar.google.com/scholar?q=X%2E+Wang%2C+J%2E+Wei%2C+D%2E+Schuurmans%2C+Q%2E+Le%2C+E%2E+Chi%2C+S%2E+Narang%2C+A%2E+Chowdhery%2C+and+D%2E+Zhou%2E+Self%2DConsistency+Improves+Chain+of+Thought+Reasoning+in+Language+Models%2E+In+International+Conference+on+Learning+Re) [our card](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/)
75. J. Wei, X. Wang, D. Schuurmans, Maarten Bosma, B. Ichter, F. Xia, E. Chi, Q. Le, and D. Zhou. Chain of Thought Prompting Elicits Reasoning in Large Language Models. In Neural Information Processing Systems, 2022. [find](https://scholar.google.com/scholar?q=J%2E+Wei%2C+X%2E+Wang%2C+D%2E+Schuurmans%2C+Maarten+Bosma%2C+B%2E+Ichter%2C+F%2E+Xia%2C+E%2E+Chi%2C+Q%2E+Le%2C+and+D%2E+Zhou%2E+Chain+of+Thought+Prompting+Elicits+Reasoning+in+Large+Language+Models%2E+In+Neural+Information+Processing+Sy) [our card](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/)
76. Y. Weng, M. Zhu, F. Xia, B. Li, S. He, K. Liu, and J. Zhao. Large Language Models are Better Reasoners with Self-Verification. In Conference on Empirical Methods in Natural Language Processing, 2023. [find](https://scholar.google.com/scholar?q=Y%2E+Weng%2C+M%2E+Zhu%2C+F%2E+Xia%2C+B%2E+Li%2C+S%2E+He%2C+K%2E+Liu%2C+and+J%2E+Zhao%2E+Large+Language+Models+are+Better+Reasoners+with+Self%2DVerification%2E+In+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/weng-2023-large-language-models-are-better/)
77. H. Xin, H. Wang, C. Zheng, L. Li, Z. Liu, Q. Cao, Y. Huang, J. Xiong, H. Shi, E. Xie, J. Yin, Z. Li, H. Liao, and X. Liang. Lego-Prover: Neural theorem proving with growing libraries. In International Conference on Learning Representations, 2024. [find](https://scholar.google.com/scholar?q=H%2E+Xin%2C+H%2E+Wang%2C+C%2E+Zheng%2C+L%2E+Li%2C+Z%2E+Liu%2C+Q%2E+Cao%2C+Y%2E+Huang%2C+J%2E+Xiong%2C+H%2E+Shi%2C+E%2E+Xie%2C+J%2E+Yin%2C+Z%2E+Li%2C+H%2E+Liao%2C+and+X%2E+Liang%2E+Lego%2DProver%3A+Neural+theorem+proving+with+growing+libraries%2E+In+International)
78. J. Xiong, Z. Li, C. Zheng, Z. Guo, Y. Yin, E. Xie, Z. Yang, Q. Cao, H. Wang, X. Han, J. Tang, C. Li, and X. Liang. DQ-LoRE: Dual queries with low rank approximation re-ranking for in-context learning. In International Conference on Learning Representations, 2024. [find](https://scholar.google.com/scholar?q=J%2E+Xiong%2C+Z%2E+Li%2C+C%2E+Zheng%2C+Z%2E+Guo%2C+Y%2E+Yin%2C+E%2E+Xie%2C+Z%2E+Yang%2C+Q%2E+Cao%2C+H%2E+Wang%2C+X%2E+Han%2C+J%2E+Tang%2C+C%2E+Li%2C+and+X%2E+Liang%2E+DQ%2DLoRE%3A+Dual+queries+with+low+rank+approximation+re%2Dranking+for+in%2Dcontext+learning%2E)
79. Z. Yuan, H. Yuan, C. Li, G. Dong, C. Tan, and C. Zhou. Scaling Relationship on Learning Mathematical Reasoning with Large Language Models. Preprint arXiv:2308.01825, 2023. [find](https://scholar.google.com/scholar?q=Z%2E+Yuan%2C+H%2E+Yuan%2C+C%2E+Li%2C+G%2E+Dong%2C+C%2E+Tan%2C+and+C%2E+Zhou%2E+Scaling+Relationship+on+Learning+Mathematical+Reasoning+with+Large+Language+Models%2E+Preprint+arXiv%3A2308%2E01825%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/yuan-2023-scaling-relationship-learning-mathematical-reasoning/)
80. X. Yue, X. Qu, G. Zhang, Y. Fu, W. Huang, H. Sun, Y. Su, and W. Chen. MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning. In International Conference on Learning Representations, 2024. [find](https://scholar.google.com/scholar?q=X%2E+Yue%2C+X%2E+Qu%2C+G%2E+Zhang%2C+Y%2E+Fu%2C+W%2E+Huang%2C+H%2E+Sun%2C+Y%2E+Su%2C+and+W%2E+Chen%2E+MAmmoTH%3A+Building+Math+Generalist+Models+through+Hybrid+Instruction+Tuning%2E+In+International+Conference+on+Learning+Representation) [our card](https://md.research.elamur.ai/free/en/papers/yue-2023-mammoth-building-math-generalist-models/)
81. A. Zeng, X. Liu, Z. Du, Z. Wang, H. Lai, M. Ding, Z. Yang, Y. Xu, W. Zheng, X. Xia, W. Tam, Z. Ma, Y. Xue, J. Zhai, W. Chen, P. Zhang, Y. Dong, and J. Tang. GLM-130B: An Open Bilingual Pre-trained Model. Preprint arXiv:2210.02414, 2022. [find](https://scholar.google.com/scholar?q=A%2E+Zeng%2C+X%2E+Liu%2C+Z%2E+Du%2C+Z%2E+Wang%2C+H%2E+Lai%2C+M%2E+Ding%2C+Z%2E+Yang%2C+Y%2E+Xu%2C+W%2E+Zheng%2C+X%2E+Xia%2C+W%2E+Tam%2C+Z%2E+Ma%2C+Y%2E+Xue%2C+J%2E+Zhai%2C+W%2E+Chen%2C+P%2E+Zhang%2C+Y%2E+Dong%2C+and+J%2E+Tang%2E+GLM%2D130B%3A+An+Open+Bilingual+Pre%2Dtrained+Mod) [our card](https://md.research.elamur.ai/free/en/papers/zeng-2022-glm-130b-open-bilingual-pre/)
82. B. Zhao, K. Mopuri, and H. Bilen. Dataset Condensation with Gradient Matching. In International Conference on Learning Representations, 2021. [find](https://scholar.google.com/scholar?q=B%2E+Zhao%2C+K%2E+Mopuri%2C+and+H%2E+Bilen%2E+Dataset+Condensation+with+Gradient+Matching%2E+In+International+Conference+on+Learning+Representations%2C+2021%2E)
83. C. Zhou, P. Liu, P. Xu, S. Iyer, J. Sun, Y. Mao, X. Ma, A. Efrat, P. Yu, L. Yu, S. Zhang, G. Ghosh, M. Lewis, L. Zettlemoyer, and O. Levy. LIMA: Less Is More for Alignment. In Neural Information Processing Systems, 2023. [find](https://scholar.google.com/scholar?q=C%2E+Zhou%2C+P%2E+Liu%2C+P%2E+Xu%2C+S%2E+Iyer%2C+J%2E+Sun%2C+Y%2E+Mao%2C+X%2E+Ma%2C+A%2E+Efrat%2C+P%2E+Yu%2C+L%2E+Yu%2C+S%2E+Zhang%2C+G%2E+Ghosh%2C+M%2E+Lewis%2C+L%2E+Zettlemoyer%2C+and+O%2E+Levy%2E+LIMA%3A+Less+Is+More+for+Alignment%2E+In+Neural+Information+Proce)
84. D. Zhou, N. Schärli, L. Hou, J. Wei, N. Scales, X. Wang, D. Schuurmans, C. Cui, O. Bousquet, Q. Le, and E. Chi. Least-to-Most Prompting Enables Complex Reasoning in Large Language Models. In International Conference on Learning Representations, 2023. [find](https://scholar.google.com/scholar?q=D%2E+Zhou%2C+N%2E+Scha%CC%88rli%2C+L%2E+Hou%2C+J%2E+Wei%2C+N%2E+Scales%2C+X%2E+Wang%2C+D%2E+Schuurmans%2C+C%2E+Cui%2C+O%2E+Bousquet%2C+Q%2E+Le%2C+and+E%2E+Chi%2E+Least%2Dto%2DMost+Prompting+Enables+Complex+Reasoning+in+Large+Language+Models%2E+In+Internat) [our card](https://md.research.elamur.ai/free/en/papers/al-2022-least-most-prompting-enables-complex/)
85. X. Zhu. Machine Teaching: An Inverse Problem to Machine Learning and an Approach Toward Optimal Education. In AAAI Conference on Artificial Intelligence, 2015. Table of Contents [find](https://scholar.google.com/scholar?q=X%2E+Zhu%2E+Machine+Teaching%3A+An+Inverse+Problem+to+Machine+Learning+and+an+Approach+Toward+Optimal+Education%2E+In+AAAI+Conference+on+Artificial+Intelligence%2C+2015%2E+Table+of+Contents)