--- title: "MathPrompter — Imani 2023" description: "Large language models often get multi-step arithmetic word problems wrong, and they give users no sign of how far an answer can be trusted." authors: ["Shima Imani", "Liang Du", "Harsh Shrivastava"] year: 2023 venue: "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 5: Industry Track)" source: "https://aclanthology.org/2023.acl-industry.4/" doi: "10.18653/v1/2023.acl-industry.4" arxiv: "2303.05398" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/imani-2023-mathprompter-mathematical-reasoning-large-language/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/imani-2023-mathprompter-mathematical-reasoning-large-language/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/5/) / MathPrompter: Mathematical Reasoning using Large Language Models # MathPrompter: Mathematical Reasoning using Large Language Models **Authors:** Shima Imani, Liang Du, Harsh Shrivastava · **Year:** 2023 · **Venue:** Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 5: Industry Track) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2023.acl-industry.4/) · [PDF](https://aclanthology.org/2023.acl-industry.4.pdf) · [DOI 10.18653/v1/2023.acl-industry.4](https://doi.org/10.18653/v1/2023.acl-industry.4) · [arXiv 2303.05398](https://arxiv.org/abs/2303.05398) · [OpenAlex](https://openalex.org/W4385572162) **Citations:** 151 citations (openalex-snapshot · 2026-10-14) · [146](#cited-by) in our library, of the 140884 papers whose bibliographies we read ## Summary Large language models often get multi-step arithmetic word problems wrong, and they give users no sign of how far an answer can be trusted. Earlier Zero-shot-CoT prompting also never checks whether the intermediate reasoning is correct. MathPrompter addresses both problems by copying the way students double-check their work. First, the numbers in a question are replaced with variables to produce an algebraic template. Next, the LLM is prompted to solve that template in several independent ways, here as an algebraic expression and as a Python function. These candidate solutions are run with Python's eval() on random variable values to see whether they agree. When they match, the original numbers are plugged back in. The whole cycle runs about five times, and the most frequent answer is kept. With GPT-3 DaVinci (175B) on the MultiArith dataset, accuracy rises from 78.7% to 92.5% compared with Zero-shot-CoT. That is comparable to 540B-parameter models and the best Few-shot-CoT setups, even though MathPrompter needs no worked examples. How well the solutions agree also gives a rough measure of confidence. ## Key points - The pipeline has four stages: (I) build an algebraic template by mapping numbers to variables, e.g. {A:5, B:15, C:8}; (II) Math-prompts that ask for an algebraic expression and a Python function; (III) compute verification with eval() on random inputs; (IV) a majority vote over N ≈ 5 repetitions. - MultiArith accuracy, mostly 175B models: Zero-shot 17.7%, Zero-shot-CoT (Kojima et al., 2022) 78.7%, MathPrompter 92.5%. - Larger and few-shot reference points: Zero-shot-CoT + self-consistency with PaLM 540B reaches 89.0%, and Few-Shot-CoT with 8 examples reaches 93.0%. - Compared with CoT text, the Python and algebraic outputs need fewer tokens. Running the arithmetic in code avoids cases where the reasoning is right but the final calculation is wrong. - Limitations: both solution types can agree on the same wrong answer, as in a commonsense error on a candy-box problem. The method was tested on MultiArith only. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Mathematics & statistics](https://md.research.elamur.ai/free/en/shelves/mathematics/) ## Cited by Cited by 146 papers of our library cite it (of the 140884 whose bibliographies we read); 13 of them have a page here, 133 not yet. - [Large Language Models for Mathematical Reasoning: Progresses and Challenges](https://md.research.elamur.ai/free/en/papers/ahn-2024-large-language-models-mathematical-reasoning/) - [Cutting Through the Noise: Boosting LLM Performance on Math Word Problems](https://md.research.elamur.ai/free/en/papers/anantheswaran-2024-cutting-through-noise-boosting-llm/) - [Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models](https://md.research.elamur.ai/free/en/papers/chen-2025-towards-reasoning-era-survey-long/) - [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://md.research.elamur.ai/free/en/papers/li-2023-camel-communicative-agents-mind/) - [Chameleon: Plug-and-Play Compositional Reasoning with Large Language Models](https://md.research.elamur.ai/free/en/papers/lu-2023-chameleon-plug-play-compositional-reasoning/) - [A Comprehensive Overview of Large Language Models](https://md.research.elamur.ai/free/en/papers/naveed-2023-comprehensive-overview-large-language-models/) - [Reasoning with Language Model Prompting: A Survey](https://md.research.elamur.ai/free/en/papers/qiao-2022-reasoning-language-model-prompting-survey/) - [Consistency of LLMs to Comparative Statements in Mathematical Reasoning Tasks](https://md.research.elamur.ai/free/en/papers/san-2026-consistency-llms-comparative-statements-mathematical/) - [ReFT: Reasoning with Reinforced Fine-Tuning](https://md.research.elamur.ai/free/en/papers/trung-2024-reft-reasoning-reinforced-fine-tuning/) - [Faithful Logical Reasoning via Symbolic Chain-of-Thought](https://md.research.elamur.ai/free/en/papers/xu-2024-faithful-logical-reasoning-symbolic-chain/) - [MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models](https://md.research.elamur.ai/free/en/papers/yu-2023-metamath-bootstrap-your-own-mathematical/) - [JudgeLM: Fine-tuned Large Language Models are Scalable Judges](https://md.research.elamur.ai/free/en/papers/zhu-2023-judgelm-fine-tuned-large-language/) - [ToolQA: A Dataset for LLM Question Answering with External Tools](https://md.research.elamur.ai/free/en/papers/zhuang-2023-toolqa-dataset-llm-question-answering/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{imani2023mathprompter, title = {{MathPrompter: Mathematical Reasoning using Large Language Models}}, author = {Imani, Shima and Du, Liang and Shrivastava, Harsh}, booktitle = {Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 5: Industry Track)}, year = {2023}, doi = {10.18653/v1/2023.acl-industry.4}, eprint = {2303.05398}, archivePrefix = {arXiv}, url = {https://doi.org/10.18653/v1/2023.acl-industry.4} } ``` - [← previous Mathematical Discoveries from Program Search with Large Language Models](https://md.research.elamur.ai/free/en/papers/paredes-2023-mathematical-discoveries-program-search-large/) - [next → MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://md.research.elamur.ai/free/en/papers/lu-2023-mathvista-evaluating-mathematical-reasoning-foundation/) - [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/5/) ## References 1 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. MathPrompter. Mapping = {A:14, B:5, C:6} Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33… [find](https://scholar.google.com/scholar?q=MathPrompter%2E+Mapping+%3D+%7BA%3A14%2C+B%3A5%2C+C%3A6%7D+Tom+Brown%2C+Benjamin+Mann%2C+Nick+Ryder%2C+Melanie+Subbiah%2C+Jared+D+Kaplan%2C+Prafulla+Dhariwal%2C+Arvind+Neelakantan%2C+Pranav+Shyam%2C+Girish+Sastry%2C+Amanda+Askell%2C+et+al)