---
title: "Solving Math Word Problems via Cooperative Reasoning induced"
description: "Inspired by dual-process theory from cognitive science, the authors propose Cooperative Reasoning… 30 references, cited by 67 papers here."
authors: ["Xinyu Zhu", "Junjie Wang", "Lin Zhang", "Yuxiang Zhang", "Yongfeng Huang", "Ruyi Gan", "Jiaxing Zhang", "Yujiu Yang"]
year: 2023
venue: "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2023.acl-long.245/"
doi: "10.18653/v1/2023.acl-long.245"
arxiv: "2210.16257"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhu-2023-solving-math-word-problems-cooperative/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhu-2023-solving-math-word-problems-cooperative/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/6/) / Solving Math Word Problems via Cooperative Reasoning induced Language Models

# Solving Math Word Problems via Cooperative Reasoning induced Language Models

**Authors:** Xinyu Zhu, Junjie Wang, Lin Zhang, [Yuxiang Zhang](https://md.research.elamur.ai/free/en/authors/yuxiang-zhang/), [Yongfeng Huang](https://md.research.elamur.ai/free/en/authors/yongfeng-huang/), Ruyi Gan, Jiaxing Zhang and [Yujiu Yang](https://md.research.elamur.ai/free/en/authors/yujiu-yang/) · **Year:** 2023 · **Venue:** [Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2023](https://md.research.elamur.ai/free/en/venues/acl/2023/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2023.acl-long.245/) · [PDF](https://aclanthology.org/2023.acl-long.245.pdf) · [DOI 10.18653/v1/2023.acl-long.245](https://doi.org/10.18653/v1/2023.acl-long.245) · [arXiv 2210.16257](https://arxiv.org/abs/2210.16257) · [OpenAlex](https://openalex.org/W4385570246)

**Citations:** 18 citations (OpenAlex, 2026-10-14) · [67](https://md.research.elamur.ai/free/en/papers/zhu-2023-solving-math-word-problems-cooperative/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Pre-trained language models often fail on math word problems because they generate text without step-by-step supervision. Prompting methods such as Chain-of-Thought mostly help only very large models. Inspired by dual-process theory from cognitive science, the authors propose Cooperative Reasoning (CoRe). A generator acts as the fast, intuitive System 1, and verifiers act as the deliberate System 2. The pipeline has three stages. First, the generator is fine-tuned on GSM8K questions with annotated rationales. It then samples many candidate solutions, which are used to train two verifiers. Second, at inference time, Monte Carlo Tree Search lets the verifier scores steer decoding while a solution is still being built, rather than only ranking finished answers. Third, a self-thinking loop keeps high-scoring generated solutions and adds them back to the training data. CoRe was tested in zero-shot and fine-tuning settings on several arithmetic benchmarks. It outperforms baselines dozens of times larger, gaining up to 9.6% on MultiArith. This shows that feedback-guided search can partly replace sheer model scale.

## Key points
- **Generator and step verifier.** The generator (System 1) is trained with a standard language-modeling loss. The step verifier V_step is a token-level scorer trained jointly on language modeling and per-token MSE regression toward answer correctness.
- **Path verifier.** V_path is a BERT-like model that scores a whole reasoning path from its [CLS] representation. The overall verifier objective combines the step loss, the path loss and the language-modeling loss.
- **Guided search.** Cooperative inference uses MCTS with a modified PUCT selection rule. This gives feedback on partial reasoning paths during generation, not just on final answers.
- **Training data.** Fine-tuning and self-thinking rely only on GSM8K, which has about 7.5K training examples. Zero-shot results on other arithmetic datasets suggest that reasoning skill transfers within the domain.
- **Limitations and code.** The method works with models of only a few billion parameters, but it was not tested on GPT-3 or PaLM. Code is available at github.com/TianHongZXY/CoRe.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Tsinghua University](https://md.research.elamur.ai/free/en/universities/tsinghua-university/) — University, Beijing, China · 137 works · 83,262 citations
- [Chinese University of Hong Kong](https://md.research.elamur.ai/free/en/universities/chinese-university-of-hong-kong/) — University, Hong Kong, China · 53 works · 12,921 citations
- [Waseda University](https://md.research.elamur.ai/free/en/universities/waseda-university/) — University, Tokyo, Japan · 4 works · 531 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works · [➗ Mathematics](https://md.research.elamur.ai/free/en/shelves/mathematics/) · 140 works · [➗ Applied Mathematics](https://md.research.elamur.ai/free/en/shelves/mathematics/applied-mathematics/) · 21 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 1,343 works · [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 510 works

## Cite

Cite as: Xinyu Zhu, Junjie Wang, Lin Zhang, Yuxiang Zhang, Yongfeng Huang, Ruyi Gan, Jiaxing Zhang and Yujiu Yang. 2023. Solving Math Word Problems via Cooperative Reasoning induced Language Models. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhu2023solving,
  title = {{Solving Math Word Problems via Cooperative Reasoning induced Language Models}},
  author = {Xinyu Zhu, Junjie Wang, Lin Zhang, Yuxiang Zhang, Yongfeng Huang, Ruyi Gan, Jiaxing Zhang and Yujiu Yang},
  booktitle = {Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2023},
  doi = {10.18653/v1/2023.acl-long.245},
  eprint = {2210.16257},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2023.acl-long.245}
}
```

- [← previous NL-Augmenter: A Framework for Task-Sensitive Natural Language Augmentation](https://md.research.elamur.ai/free/en/papers/dhole-2023-nl-augmenter-framework-task-sensitive/)
- [next → Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural Language Generation](https://md.research.elamur.ai/free/en/papers/fernandes-2023-bridging-gap-survey-integrating-human/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/6/)

## References

30 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Akhter Al Amin and Kazi Sinthia Kabir. 2022. A disability lens towards biases in GPT-3 generated open-ended languages. CoRR, abs/2206.11993. — [arXiv:2206.11993](https://arxiv.org/abs/2206.11993) [find](https://scholar.google.com/scholar?q=Akhter+Al+Amin+and+Kazi+Sinthia+Kabir%2E+2022%2E+A+disability+lens+towards+biases+in+GPT%2D3+generated+open%2Dended+languages%2E+CoRR%2C+abs%2F2206%2E11993%2E+%E2%80%94+%5BarXiv%3A2206%2E11993%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2206%2E11993%29)
2. [Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,…](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
3. [Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emi…](https://md.research.elamur.ai/free/en/papers/chowdhery-2022-palm-scaling-language-modeling-pathways/)
4. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). CoRR, abs/2110.14168.
5. Johannes Czech, Patrick Korus, and Kristian Kersting. 2021. Improving alphazero using monte-carlo graph search. In ICAPS, pages 103–111. AAAI Press. [find](https://scholar.google.com/scholar?q=Johannes+Czech%2C+Patrick+Korus%2C+and+Kristian+Kersting%2E+2021%2E+Improving+alphazero+using+monte%2Dcarlo+graph+search%2E+In+ICAPS%2C+pages+103%E2%80%93111%2E+AAAI+Press%2E)
6. Jonathan St.B.T. Evans. 2003. In two minds: dual-process accounts of reasoning. Trends in Cognitive Sciences, 7(10):454–459. [find](https://scholar.google.com/scholar?q=Jonathan+St%2EB%2ET%2E+Evans%2E+2003%2E+In+two+minds%3A+dual%2Dprocess+accounts+of+reasoning%2E+Trends+in+Cognitive+Sciences%2C+7%2810%29%3A454%E2%80%93459%2E)
7. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. [Deberta: decoding-enhanced bert with disentangled attention](https://md.research.elamur.ai/free/en/papers/he-2020-deberta-decoding-enhanced-bert-disentangled/). In ICLR. OpenReview.net.
8. Daniel Kahneman. 2011. Thinking, fast and slow. Farrar, Straus and Giroux. [find](https://scholar.google.com/scholar?q=Daniel+Kahneman%2E+2011%2E+Thinking%2C+fast+and+slow%2E+Farrar%2C+Straus+and+Giroux%2E)
9. Levente Kocsis and Csaba Szepesvári. 2006. Bandit based monte-carlo planning. In ECML, volume 4212 of Lecture Notes in Computer Science, pages 282–293. Springer. [find](https://scholar.google.com/scholar?q=Levente+Kocsis+and+Csaba+Szepesv%C3%A1ri%2E+2006%2E+Bandit+based+monte%2Dcarlo+planning%2E+In+ECML%2C+volume+4212+of+Lecture+Notes+in+Computer+Science%2C+pages+282%E2%80%93293%2E+Springer%2E)
10. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. [Large language models are zero-shot reasoners](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/). CoRR, abs/2205.11916.
11. Rik Koncel-Kedziorski, Hannaneh Hajishirzi, Ashish Sabharwal, Oren Etzioni, and Siena Dumas Ang. 2015. Parsing algebraic word problems into equations. Transactions of the Association for Computational Linguistics, 3:585–597. [find](https://scholar.google.com/scholar?q=Rik+Koncel%2DKedziorski%2C+Hannaneh+Hajishirzi%2C+Ashish+Sabharwal%2C+Oren+Etzioni%2C+and+Siena+Dumas+Ang%2E+2015%2E+Parsing+algebraic+word+problems+into+equations%2E+Transactions+of+the+Association+for+Computational)
12. Sanjay Krishnan, Jiannan Wang, Michael J. Franklin, Ken Goldberg, and Tim Kraska. 2016. Privateclean: Data cleaning and differential privacy. In SIGMOD Conference, pages 937–951. ACM. [find](https://scholar.google.com/scholar?q=Sanjay+Krishnan%2C+Jiannan+Wang%2C+Michael+J%2E+Franklin%2C+Ken+Goldberg%2C+and+Tim+Kraska%2E+2016%2E+Privateclean%3A+Data+cleaning+and+differential+privacy%2E+In+SIGMOD+Conference%2C+pages+937%E2%80%93951%2E+ACM%2E)
13. Yihuai Lan, Lei Wang, Qiyuan Zhang, Yunshi Lan, Bing Tian Dai, Yan Wang, Dongxiang Zhang, and Ee-Peng Lim. 2022. Mwptoolkit: An open-source framework for deep learning-based math word problem solvers. In AAAI, pages 13188–13190. AAAI Press. [find](https://scholar.google.com/scholar?q=Yihuai+Lan%2C+Lei+Wang%2C+Qiyuan+Zhang%2C+Yunshi+Lan%2C+Bing+Tian+Dai%2C+Yan+Wang%2C+Dongxiang+Zhang%2C+and+Ee%2DPeng+Lim%2E+2022%2E+Mwptoolkit%3A+An+open%2Dsource+framework+for+deep+learning%2Dbased+math+word+problem+solvers%2E)
14. Yifei Li, Zeqi Lin, Shizhuo Zhang, Qiang Fu, Bei Chen, Jian-Guang Lou, and Weizhu Chen. 2022. On the advance of making language models better reasoners. CoRR, abs/2206.02336. — [arXiv:2206.02336](https://arxiv.org/abs/2206.02336) [find](https://scholar.google.com/scholar?q=Yifei+Li%2C+Zeqi+Lin%2C+Shizhuo+Zhang%2C+Qiang+Fu%2C+Bei+Chen%2C+Jian%2DGuang+Lou%2C+and+Weizhu+Chen%2E+2022%2E+On+the+advance+of+making+language+models+better+reasoners%2E+CoRR%2C+abs%2F2206%2E02336%2E+%E2%80%94+%5BarXiv%3A2206%2E02336%5D%28http)
15. Chao-Chun Liang, Kuang-Yi Hsu, Chien-Tsung Huang, Chung-Min Li, Shen-Yu Miao, and Keh-Yih Su. 2016. A tag-based statistical english math word problem solver with understanding, reasoning and explanation. In IJCAI, pages 4254–4255. IJCAI/AAAI Press. [find](https://scholar.google.com/scholar?q=Chao%2DChun+Liang%2C+Kuang%2DYi+Hsu%2C+Chien%2DTsung+Huang%2C+Chung%2DMin+Li%2C+Shen%2DYu+Miao%2C+and+Keh%2DYih+Su%2E+2016%2E+A+tag%2Dbased+statistical+english+math+word+problem+solver+with+understanding%2C+reasoning+and+explanati)
16. Li Lucy and David Bamman. 2021. [Gender and representation bias in GPT-3 generated stories](https://md.research.elamur.ai/free/en/papers/lucy-2021-gender-representation-bias-gpt-3/). In Proceedings of the Third Workshop on Narrative Understanding, pages 48–55, Virtual. Association for Computational Linguistics.
17. Shen-yun Miao, Chao-Chun Liang, and Keh-Yih Su. 2020. A diverse corpus for evaluating and developing English math word problem solvers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 975–984, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Shen%2Dyun+Miao%2C+Chao%2DChun+Liang%2C+and+Keh%2DYih+Su%2E+2020%2E+A+diverse+corpus+for+evaluating+and+developing+English+math+word+problem+solvers%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Association+for)
18. Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, Charles Sutton, and Augustus Odena. 2022. [Show your work: Scratchpads for intermediate computation with language models](https://md.research.elamur.ai/free/en/papers/al-2021-show-your-work-scratchpads-intermediate/). In Deep Learning for Co…
19. Maxwell I. Nye, Michael Henry Tessler, Joshua B. Tenenbaum, and Brenden M. Lake. 2021. Improving coherence and consistency in neural sequence models with dual-system, neuro-symbolic reasoning. In NeurIPS, pages 25192–25204. [find](https://scholar.google.com/scholar?q=Maxwell+I%2E+Nye%2C+Michael+Henry+Tessler%2C+Joshua+B%2E+Tenenbaum%2C+and+Brenden+M%2E+Lake%2E+2021%2E+Improving+coherence+and+consistency+in+neural+sequence+models+with+dual%2Dsystem%2C+neuro%2Dsymbolic+reasoning%2E+In+Neur)
20. Subhro Roy and Dan Roth. 2015. [Solving general arithmetic word problems](https://md.research.elamur.ai/free/en/papers/roy-2015-solving-general-arithmetic-word-problems/). In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, September 17-21, 2015, pages 1743–1752. The Association for Computational Linguistics.
21. Subhro Roy and Dan Roth. 2017. [Unit dependency graph and its application to arithmetic word problem solving](https://md.research.elamur.ai/free/en/papers/roy-2017-unit-dependency-graph-arithmetic/). In AAAI, pages 3082–3088. AAAI Press.
22. Subhro Roy, Tim Vieira, and Dan Roth. 2015. Reasoning about quantities in natural language. Trans. Assoc. Comput. Linguistics, 3:1–13. [find](https://scholar.google.com/scholar?q=Subhro+Roy%2C+Tim+Vieira%2C+and+Dan+Roth%2E+2015%2E+Reasoning+about+quantities+in+natural+language%2E+Trans%2E+Assoc%2E+Comput%2E+Linguistics%2C+3%3A1%E2%80%9313%2E)
23. [Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal V. Nayak, Debajyoti Datta, Jonathan C…](https://md.research.elamur.ai/free/en/papers/sanh-2021-multitask-prompted-training-enables-zero/)
24. Thomas Scialom, Paul-Alexis Dray, Jacopo Staiano, Sylvain Lamprier, and Benjamin Piwowarski. 2021. To beam or not to beam: That is a question of cooperation for language gans. In NeurIPS, pages 26585– 26597. [find](https://scholar.google.com/scholar?q=Thomas+Scialom%2C+Paul%2DAlexis+Dray%2C+Jacopo+Staiano%2C+Sylvain+Lamprier%2C+and+Benjamin+Piwowarski%2E+2021%2E+To+beam+or+not+to+beam%3A+That+is+a+question+of+cooperation+for+language+gans%2E+In+NeurIPS%2C+pages+26585%E2%80%93)
25. [Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, YaGuang Li, Hongrae Lee, Huaixiu Steven Zheng, Amin Ghafouri, Marcelo Menegali, Yanping Huang, Maxim Krikun, Dmitry Lepikhin, James Qin, Dehao Chen, Yuanzho…](https://md.research.elamur.ai/free/en/papers/thoppilan-2022-lamda-language-models-dialog-applications/)
26. Ben Wang and Aran Komatsuzaki. 2021. GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model. [https://github.com/kingoflolz/](https://github.com/kingoflolz/) mesh-transformer-jax. [↗](https://github.com/kingoflolz/)
27. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, and Denny Zhou. 2022. [Self-consistency improves chain of thought reasoning in language models](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/). CoRR, abs/2203.11171.
28. Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. 2022a. Finetuned language models are zero-shot learners. In ICLR. OpenReview.net. [find](https://scholar.google.com/scholar?q=Jason+Wei%2C+Maarten+Bosma%2C+Vincent+Y%2E+Zhao%2C+Kelvin+Guu%2C+Adams+Wei+Yu%2C+Brian+Lester%2C+Nan+Du%2C+Andrew+M%2E+Dai%2C+and+Quoc+V%2E+Le%2E+2022a%2E+Finetuned+language+models+are+zero%2Dshot+learners%2E+In+ICLR%2E+OpenReview%2En)
29. Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. 2022b. [Emergent abilities of large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-emergent-abilities-large-language-models/). CoRR, abs/2206.07…
30. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed H. Chi, Quoc Le, and Denny Zhou. 2022c. [Chain of thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). CoRR, abs/2201.11903.

*Updated 24 October 2026*