--- title: "Marco-o1 — paper by Zhao et al. (2024) — summary and ideas" description: "Marco-o1, from Alibaba's MarcoPolo Team, is an early open attempt to reproduce the behavior of… 7 references, cited by 77 papers here." authors: ["Yu Zhao", "Huifeng Yin", "Bo Zeng", "Hao Wang", "Tianqi Shi", "Chenyang Lyu", "Longyue Wang", "Weihua Luo", "Kaifu Zhang"] year: 2024 venue: "arXiv:2411.14405" source: "https://arxiv.org/abs/2411.14405" doi: "10.48550/arxiv.2411.14405" arxiv: "2411.14405" licence: "arXiv — non-exclusive licence to distribute" licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/zhao-2024-marco-o1-towards-open-reasoning/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhao-2024-marco-o1-towards-open-reasoning/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/5/) / Marco-o1: Towards Open Reasoning Models for Open-Ended Solutions # Marco-o1: Towards Open Reasoning Models for Open-Ended Solutions **Authors:** [Yu Zhao](https://md.research.elamur.ai/free/en/authors/yu-zhao/), [Huifeng Yin](https://md.research.elamur.ai/free/en/authors/huifeng-yin/), [Bo Zeng](https://md.research.elamur.ai/free/en/authors/bo-zeng/), [Hao Wang](https://md.research.elamur.ai/free/en/authors/hao-wang/), [Tianqi Shi](https://md.research.elamur.ai/free/en/authors/tianqi-shi/), [Chenyang Lyu](https://md.research.elamur.ai/free/en/authors/chenyang-lyu/), [Longyue Wang](https://md.research.elamur.ai/free/en/authors/longyue-wang/), [Weihua Luo](https://md.research.elamur.ai/free/en/authors/weihua-luo/) and [Kaifu Zhang](https://md.research.elamur.ai/free/en/authors/kaifu-zhang/) · **Year:** 2024 · **Venue:** arXiv:2411.14405 · **Licence:** [arXiv — non-exclusive licence to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/) **Links:** [Publisher's page](https://arxiv.org/abs/2411.14405) · [PDF](https://arxiv.org/pdf/2411.14405) · [DOI 10.48550/arxiv.2411.14405](https://doi.org/10.48550/arxiv.2411.14405) · [arXiv 2411.14405](https://arxiv.org/abs/2411.14405) · [OpenAlex](https://openalex.org/W4404649409) **Citations:** 5 citations (OpenAlex, 2026-10-14) · [77](https://md.research.elamur.ai/free/en/papers/zhao-2024-marco-o1-towards-open-reasoning/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary Marco-o1, from Alibaba's MarcoPolo Team, is an early open attempt to reproduce the behavior of OpenAI's o1-style large reasoning models (LRM). It also asks whether such reasoning extends to tasks without clear correct answers or easily measured rewards. The authors fully fine-tune Qwen2-7B-Instruct on about 60K examples: a filtered Open-O1 CoT dataset, their own MCTS-generated CoT data, and instruction-following data. At inference they add Monte Carlo Tree Search. Each node is a partial reasoning state, and a path is scored by the model's own confidence: a softmax over the top-5 token log probabilities, averaged across the rollout. They compare whole reasoning steps with smaller 64- or 32-token "mini-steps" as search actions and add a self-reflection prompt. On the English and Chinese MGSM subsets, the MCTS variants outperform both the base model and the CoT-only model, though no single granularity wins in both languages. Qualitative examples suggest better handling of slang in Chinese–English translation. The work is useful as a transparent sketch of an o1-like pipeline, and the authors explicitly label it a work in progress. ## Key points - **Training data:** 60,266 SFT samples in total. - 45,125 from the filtered Open-O1 CoT dataset. - 10,000 from the synthetic Marco-o1 CoT dataset, generated via MCTS. - 5,141 from the Marco Instruction Dataset. - **Accuracy (Test@1):** - MGSM-En: Qwen2-7B-Instruct scores 84.00%, Marco-o1-CoT 85.60%, and the best result is 90.40% (MCTS, step as action). - MGSM-Zh: Qwen2-7B-Instruct scores 76.80%. Marco-o1-CoT drops to 71.20%, which the authors attribute to English-only CoT data. The best result is 82.40% (MCTS, 32-token mini-step). - **Test@8 / Test@32 results:** with multiple attempts, all fine-tuned variants reach roughly 93–99.6%. MCTS gives its clearest advantage at Test@1. - **Reflection prompt:** appending a "rethink from scratch" style prompt reportedly fixes about half of the hard problems the model initially got wrong. - **Limitations and future work:** using confidence as the reward makes search results noticeably random. Planned work includes ORM/PRM reward models and RL. The authors also claim the first study of LRMs on machine translation, comparing against Google Translate in case studies only. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Alibaba Group](https://md.research.elamur.ai/free/en/companies/alibaba-group/) — Company, Hangzhou, China · 201 works · 17,173 citations 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works 🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,685 works ## Cite Cite as: Yu Zhao, Huifeng Yin, Bo Zeng, Hao Wang, Tianqi Shi, Chenyang Lyu, Longyue Wang, Weihua Luo and Kaifu Zhang. 2024. Marco-o1: Towards Open Reasoning Models for Open-Ended Solutions. arXiv:2411.14405. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @misc{zhao2024marco, title = {{Marco-o1: Towards Open Reasoning Models for Open-Ended Solutions}}, author = {Yu Zhao, Huifeng Yin, Bo Zeng, Hao Wang, Tianqi Shi, Chenyang Lyu, Longyue Wang, Weihua Luo and Kaifu Zhang}, year = {2024}, doi = {10.48550/arxiv.2411.14405}, eprint = {2411.14405}, archivePrefix = {arXiv}, url = {https://doi.org/10.48550/arxiv.2411.14405} } ``` - [← previous Knowledge Fusion of Large Language Models](https://md.research.elamur.ai/free/en/papers/wan-2024-knowledge-fusion-large-language-models/) - [next → MART: Improving LLM Safety with Multi-round Automatic Red-Teaming](https://md.research.elamur.ai/free/en/papers/ge-2024-mart-improving-llm-safety-multi/) - [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/5/) ## References 7 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/), 2021. URL https://arxiv.org/abs/2110.14168. [↗](https://arxiv.org/abs/2110.14168) 2. MGSM-En Test@1 Acc. Test@8 Acc. Test@32 Acc. 84.00% 85.60% 90.40% 88.40% 87.60% 89.60% 97.60% 99.20% 98.40% 98.80% 96.00% 99.20% 99.20% 99.60% 99.20% MGSM-Zh Test@1 Acc. Test@8 Acc. Test@32 Acc. 76.80% 71.20% 80.00% 80.40% 82.40% 80.80% 93.60% 93.60% 92.80% 93.20% 92.40% 96.40% 96.00% 95.20% 96.80%… [find](https://scholar.google.com/scholar?q=MGSM%2DEn+Test%401+Acc%2E+Test%408+Acc%2E+Test%4032+Acc%2E+84%2E00%25+85%2E60%25+90%2E40%25+88%2E40%25+87%2E60%25+89%2E60%25+97%2E60%25+99%2E20%25+98%2E40%25+98%2E80%25+96%2E00%25+99%2E20%25+99%2E20%25+99%2E60%25+99%2E20%25+MGSM%2DZh+Test%401+Acc%2E+Test%408+Acc%2E+Test%4032+Acc%2E+76%2E80) 3. J. Huang, S. S. Gu, L. Hou, Y. Wu, X. Wang, H. Yu, and J. Han. [Large language models can self-improve](https://md.research.elamur.ai/free/en/papers/huang-2023-large-language-models-can-self/). arXiv preprint arXiv:2210.11610, 2022. 4. Z. Li, B. Peng, P. He, M. Galley, J. Gao, and X. Yan. Guiding large language models via directional stimulus prompting. Advances in Neural Information Processing Systems, 36, 2024. [find](https://scholar.google.com/scholar?q=Z%2E+Li%2C+B%2E+Peng%2C+P%2E+He%2C+M%2E+Galley%2C+J%2E+Gao%2C+and+X%2E+Yan%2E+Guiding+large+language+models+via+directional+stimulus+prompting%2E+Advances+in+Neural+Information+Processing+Systems%2C+36%2C+2024%2E) 5. H. Lightman, V. Kosaraju, Y. Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, and K. Cobbe. Let’s verify step by step. arXiv preprint arXiv:2305.[20050, 2023](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/). 6. A. Madaan, N. Tandon, P. Gupta, S. Hallinan, L. Gao, S. Wiegreffe, U. Alon, N. Dziri, S. Prabhumoye, Y. Yang, et al. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36, 2024. [find](https://scholar.google.com/scholar?q=A%2E+Madaan%2C+N%2E+Tandon%2C+P%2E+Gupta%2C+S%2E+Hallinan%2C+L%2E+Gao%2C+S%2E+Wiegreffe%2C+U%2E+Alon%2C+N%2E+Dziri%2C+S%2E+Prabhumoye%2C+Y%2E+Yang%2C+et+al%2E+Self%2Drefine%3A+Iterative+refinement+with+self%2Dfeedback%2E+Advances+in+Neural+Informatio) 7. OpenAI. Learning to reason with llms. learning-to-reason-with-llms/, 2024. [Accessed 19-09-2024]. [https://openai.com/index/](https://openai.com/index/) [↗](https://openai.com/index/) *Updated 2 November 2026*