---
title: "MAPO — paper by She et al. (2024) — summary and ideas"
description: "Large language models tend to solve math word problems much better in English than in other… 19 references, cited by 52 papers here."
authors: ["Shuaijie She", "Wei Zou", "Shujian Huang", "Wenhao Zhu", "Xiang Liu", "Xiang Geng", "Jiajun Chen"]
year: 2024
venue: "Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2024.acl-long.539/"
doi: "10.18653/v1/2024.acl-long.539"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/she-2024-mapo-advancing-multilingual-reasoning-through/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/she-2024-mapo-advancing-multilingual-reasoning-through/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/5/) / MAPO: Advancing Multilingual Reasoning through Multilingual-Alignment-as-Preference Optimization

# MAPO: Advancing Multilingual Reasoning through Multilingual-Alignment-as-Preference Optimization

**Authors:** [Shuaijie She](https://md.research.elamur.ai/free/en/authors/shuaijie-she/), Wei Zou, [Shujian Huang](https://md.research.elamur.ai/free/en/authors/shujian-huang/), [Wenhao Zhu](https://md.research.elamur.ai/free/en/authors/wenhao-zhu/), [Xiang Liu](https://md.research.elamur.ai/free/en/authors/xiang-liu/), Xiang Geng and [Jiajun Chen](https://md.research.elamur.ai/free/en/authors/jiajun-chen/) · **Year:** 2024 · **Venue:** [Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2024](https://md.research.elamur.ai/free/en/venues/acl/2024/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2024.acl-long.539/) · [PDF](https://aclanthology.org/2024.acl-long.539.pdf) · [DOI 10.18653/v1/2024.acl-long.539](https://doi.org/10.18653/v1/2024.acl-long.539) · [OpenAlex](https://openalex.org/W4402671600)

**Citations:** 7 citations (OpenAlex, 2026-10-14) · [52](https://md.research.elamur.ai/free/en/papers/she-2024-mapo-advancing-multilingual-reasoning-through/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Large language models tend to solve math word problems much better in English than in other languages, mainly because their training data is skewed toward English. Earlier fixes relied on supervised fine-tuning with translated reasoning chains, which are costly, often noisy, and generalize poorly. MAPO avoids new reasoning annotation. It treats the model's own English reasoning as a reference and rewards non-English reasoning that agrees with it. For each question, the model samples answers in English and in a non-dominant language. An off-the-shelf translator (NLLB-600M-distilled) then scores how likely the English answer is as a translation of each non-English answer. These alignment scores serve as preferences for PPO, where the translator acts as the reward model, or for DPO, which is also run iteratively. Across MSVAMP, MGSM and MNumGLUESub, each covering 10 languages, MAPO consistently raises accuracy and cross-language consistency. Its 7B models reach state-of-the-art results, including the largest gain on the out-of-domain MSVAMP set. This suggests a cheap, annotation-free way to improve reasoning in lower-resource languages.

## Key points
- Average accuracy gains reach +16.2% on MSVAMP (out-of-domain, 10,000 test items), +6.1% on MGSM (2,500 items) and +13.3% on MNumGLUESub (5,530 items).
- MNumGLUESub is a new resource built by the authors. They took tasks 1, 4 and 8 of NumGLUE and translated them into the nine non-English languages used in MGSM8KInstruct.
- Training is efficient. With DPO, MathOctopus 7B overtook both ChatGPT and MathOctopus 13B on MSVAMP in about 100 steps. PPO needed roughly 1,000 steps to pass the 13B model and about 1,600 to beat ChatGPT.
- English does not have to be the dominant language. With Spanish as the reference, MetaMathOctopus7B rose from 53.0 to 59.9 on MSVAMP and from 45.5 to 47.7 on MGSM, versus 61.1 and 48.9 with English.
- MAPO requires a policy model that already has some multilingual reasoning ability from SFT. Experiments were limited to 7B and 13B models.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Nanjing University](https://md.research.elamur.ai/free/en/universities/nanjing-university/) — University, Nanjing, China · 74 works · 7,814 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (2): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,426 works · [🌍 Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) · 828 works

## Cite

Cite as: Shuaijie She, Wei Zou, Shujian Huang, Wenhao Zhu, Xiang Liu, Xiang Geng and Jiajun Chen. 2024. MAPO: Advancing Multilingual Reasoning through Multilingual-Alignment-as-Preference Optimization. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{she2024mapo,
  title = {{MAPO: Advancing Multilingual Reasoning through Multilingual-Alignment-as-Preference Optimization}},
  author = {Shuaijie She, Wei Zou, Shujian Huang, Wenhao Zhu, Xiang Liu, Xiang Geng and Jiajun Chen},
  booktitle = {Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2024},
  doi = {10.18653/v1/2024.acl-long.539},
  url = {https://doi.org/10.18653/v1/2024.acl-long.539}
}
```

- [← previous Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmetic](https://md.research.elamur.ai/free/en/papers/bhardwaj-2024-language-models-are-homer-simpson/)
- [next → NADI 2024: The Fifth Nuanced Arabic Dialect Identification Shared Task](https://md.research.elamur.ai/free/en/papers/mageed-2024-nadi-2024-fifth-nuanced-arabic/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/5/)

## References

19 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Sh… [find](https://scholar.google.com/scholar?q=Yuntao+Bai%2C+Andy+Jones%2C+Kamal+Ndousse%2C+Amanda+Askell%2C+Anna+Chen%2C+Nova+DasSarma%2C+Dawn+Drain%2C+Stanislav+Fort%2C+Deep+Ganguli%2C+Tom+Henighan%2C+Nicholas+Joseph%2C+Saurav+Kadavath%2C+Jackson+Kernion%2C+Tom+Conerly%2C+)
2. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Sh… [find](https://scholar.google.com/scholar?q=Yuntao+Bai%2C+Andy+Jones%2C+Kamal+Ndousse%2C+Amanda+Askell%2C+Anna+Chen%2C+Nova+DasSarma%2C+Dawn+Drain%2C+Stanislav+Fort%2C+Deep+Ganguli%2C+Tom+Henighan%2C+Nicholas+Joseph%2C+Saurav+Kadavath%2C+Jackson+Kernion%2C+Tom+Conerly%2C+)
3. Nuo Chen, Zinan Zheng, Ning Wu, Linjun Shou, Ming Gong, Yangqiu Song, Dongmei Zhang, and Jia Li. 2023. Breaking language barriers in multilingual mathematical reasoning: Insights and observations. arXiv preprint arXiv:2310.20246. [find](https://scholar.google.com/scholar?q=Nuo+Chen%2C+Zinan+Zheng%2C+Ning+Wu%2C+Linjun+Shou%2C+Ming+Gong%2C+Yangqiu+Song%2C+Dongmei+Zhang%2C+and+Jia+Li%2E+2023%2E+Breaking+language+barriers+in+multilingual+mathematical+reasoning%3A+Insights+and+observations%2E+arX)
4. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). CoRR, abs/2110.14168.
5. Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jian-guang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang. 2023. [Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct](https://md.research.elamur.ai/free/en/papers/luo-2023-wizardmath-empowering-mathematical-reasoning-large/). CoRR, abs/2308.09583.
6. Swaroop Mishra, Arindam Mitra, Neeraj Varshney, Bhavdeep Sachdeva, Peter Clark, Chitta Baral, and Ashwin Kalyan. 2022. Numglue: A suite of fundamental yet challenging mathematical reasoning tasks. arXiv preprint arXiv:2204.05660. [find](https://scholar.google.com/scholar?q=Swaroop+Mishra%2C+Arindam+Mitra%2C+Neeraj+Varshney%2C+Bhavdeep+Sachdeva%2C+Peter+Clark%2C+Chitta+Baral%2C+and+Ashwin+Kalyan%2E+2022%2E+Numglue%3A+A+suite+of+fundamental+yet+challenging+mathematical+reasoning+tasks%2E+arX)
7. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. [Train…](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/)
8. Arkil Patel, Satwik Bhattamishra, and Navin Goyal. 2021. [Are nlp models really able to solve simple math word problems?](https://md.research.elamur.ai/free/en/papers/patel-2021-are-nlp-models-really-able/)
9. Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. 2023. [Direct preference optimization: Your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/). CoRR, abs/2305.18290.
10. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. CoRR, abs/1707.06347. [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Filip+Wolski%2C+Prafulla+Dhariwal%2C+Alec+Radford%2C+and+Oleg+Klimov%2E+2017%2E+Proximal+policy+optimization+algorithms%2E+CoRR%2C+abs%2F1707%2E06347%2E)
11. Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, Dipanjan Das, and Jason Wei. 2022. [Language models are multilingual chain-of-thought reasoners](https://md.research.elamur.ai/free/en/papers/shi-2022-language-models-multilingual-chain-thought/).
12. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,… — [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/)
13. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. [Self-consistency improves chain of thought reasoning in language models](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/). arXiv preprint arXiv:2203.11171.
14. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/).
15. Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T. Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. 2023. [Metamath: Bootstrap your own mathematical questions for large language models](https://md.research.elamur.ai/free/en/papers/yu-2023-metamath-bootstrap-your-own-mathematical/). CoRR, abs/2309.12284.
16. Zheng Yuan, Hongyi Yuan, Chengpeng Li, Guanting Dong, Chuanqi Tan, and Chang Zhou. 2023. [Scaling relationship on learning mathematical reasoning with large language models](https://md.research.elamur.ai/free/en/papers/yuan-2023-scaling-relationship-learning-mathematical-reasoning/). arXiv preprint arXiv:2308.01825.
17. Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. 2023. [Mammoth: Building math generalist models through hybrid instruction tuning](https://md.research.elamur.ai/free/en/papers/yue-2023-mammoth-building-math-generalist-models/). CoRR, abs/2309.05653.
18. Rui Zheng, Shihan Dou, Songyang Gao, Yuan Hua, Wei Shen, Binghai Wang, Yan Liu, Senjie Jin, Qin Liu, Yuhao Zhou, Limao Xiong, Lu Chen, Zhiheng Xi, Nuo Xu, Wenbin Lai, Minghao Zhu, Cheng Chang, Zhangyue Yin, Rongxiang Weng, Wensen Cheng, Haoran Huang, Tianxiang Sun, Hang Yan, Tao Gui, Qi Zhang, Xipen… — [Secrets of RLHF in Large Language Models Part I: PPO](https://md.research.elamur.ai/free/en/papers/zheng-2023-secrets-rlhf-large-language-models/)
19. Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. 2018. [Texygen: A benchmarking platform for text generation models](https://md.research.elamur.ai/free/en/papers/zhu-2018-texygen-benchmarking-platform-text-generation/). CoRR, abs/1802.01886.

*Updated 31 October 2026*