---
title: "MathArena — paper by Balunović et al. (2025)"
description: "Popular math benchmarks for LLMs, such as AIME 2024, are freely available online. 40 references, cited by 94 papers here."
authors: ["Mislav Balunović", "Jasper Dekoninck", "Ivo Petrov", "Nikola Jovanović", "Martin Vechev"]
year: 2025
venue: "arXiv:2505.23281"
source: "https://arxiv.org/abs/2505.23281"
doi: "10.48550/arxiv.2505.23281"
arxiv: "2505.23281"
licence: "CC BY 4.0 (arXiv)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/balunovic-2025-matharena-evaluating-llms-uncontaminated-math/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/balunovic-2025-matharena-evaluating-llms-uncontaminated-math/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/) / MathArena: Evaluating LLMs on Uncontaminated Math Competitions

# MathArena: Evaluating LLMs on Uncontaminated Math Competitions

**Authors:** [Mislav Balunović](https://md.research.elamur.ai/free/en/authors/mislav-balunovic/), [Jasper Dekoninck](https://md.research.elamur.ai/free/en/authors/jasper-dekoninck/), [Ivo Petrov](https://md.research.elamur.ai/free/en/authors/ivo-petrov/), Nikola Jovanović and [Martin Vechev](https://md.research.elamur.ai/free/en/authors/martin-vechev/) · **Year:** 2025 · **Venue:** [arXiv:2505.23281](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/) ([NeurIPS 2025](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/2025/)) · **Licence:** [CC BY 4.0 (arXiv)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2505.23281) · [PDF](https://arxiv.org/pdf/2505.23281v3) · [DOI 10.48550/arxiv.2505.23281](https://doi.org/10.48550/arxiv.2505.23281) · [arXiv 2505.23281](https://arxiv.org/abs/2505.23281) · [OpenAlex](https://openalex.org/W4416610439)

**Citations:** 5 citations (OpenAlex, 2026-10-14) · [94](https://md.research.elamur.ai/free/en/papers/balunovic-2025-matharena-evaluating-llms-uncontaminated-math/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Popular math benchmarks for LLMs, such as AIME 2024, are freely available online. High scores on them may therefore reflect memorization rather than reasoning, and most of them check only final answers, not proofs. Private alternatives like FrontierMath and HLE avoid leakage, but they are expensive, opaque and hard to reproduce. MathArena takes a different route. It evaluates models on recurring human math competitions as soon as new problems are published, so the problems cannot have been in the training data. An open pipeline extracts the problems, queries the models, and then grades the responses. Final-answer tasks are scored by a rule-based parser, while proof tasks are scored by human judges. Across 2025 competitions, the authors find clear evidence that AIME 2024 is contaminated. Leading models such as GPT-5, Grok 4 and Gemini-2.5-Pro beat nearly all human contestants on answer-based contests, but they remain weak at writing rigorous proofs. Because the benchmark is transparent and continuously updated, it offers a reproducible, contamination-resistant way to track progress in mathematical reasoning.

## Key points
- Scale: more than 50 models were evaluated on 162 problems from seven competitions: AIME, HMMT Feb., USAMO, CMIMC, BRUMO, IMO and Project Euler. Code, data and model outputs are open-sourced on GitHub (eth-sri/matharena).
- First benchmark to grade natural-language proofs. On IMO 2025, the best model, GPT-5 (high), averaged 16.0 of 42 points, which is just under 40%. Gemini-2.5-Pro scored 13.3, o3 (high) 7.0 and Grok 4 5.0.
- Best-of-n selection over 32 raw responses was essential for the proof scores. A judge estimated that unfiltered scores would fall below 10%.
- Qualitative issues: Grok 4 often gave terse, unjustified answers, and Gemini-2.5-Pro sometimes cited theorems that do not exist.
- Limitations: the small number of problems leads to wide confidence intervals, and answer-based contests may saturate around 2026. Evaluations run within hours to days of each contest, which keeps contamination risk low.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [ETH Zurich](https://md.research.elamur.ai/free/en/universities/eth-zurich/) — University, Zurich, Switzerland · 84 works · 9,181 citations
- [Sofia University "St. Kliment Ohridski"](https://md.research.elamur.ai/free/en/universities/sofia-university-st-kliment-ohridski/) — University, Sofia, Bulgaria · 23 works · 2,694 citations

📚 Shelves (7): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [➗ Mathematics](https://md.research.elamur.ai/free/en/shelves/mathematics/) · 189 works · [🧮 Computational Theory and Mathematics](https://md.research.elamur.ai/free/en/shelves/computer-science/computational-theory-and-mathematics/) · 321 works · [➗ Applied Mathematics](https://md.research.elamur.ai/free/en/shelves/mathematics/applied-mathematics/) · 52 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,685 works

## Cite

Cite as: Mislav Balunović, Jasper Dekoninck, Ivo Petrov, Nikola Jovanović and Martin Vechev. 2025. MathArena: Evaluating LLMs on Uncontaminated Math Competitions. arXiv:2505.23281.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{balunovic2025matharena,
  title = {{MathArena: Evaluating LLMs on Uncontaminated Math Competitions}},
  author = {Mislav Balunović, Jasper Dekoninck, Ivo Petrov, Nikola Jovanović and Martin Vechev},
  year = {2025},
  doi = {10.48550/arxiv.2505.23281},
  eprint = {2505.23281},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2505.23281}
}
```

- [← previous FineWeb2: One Pipeline to Scale Them All — Adapting Pre-Training Data Processing to Every Language](https://md.research.elamur.ai/free/en/papers/penedo-2025-fineweb2-one-pipeline-scale-them/)
- [next → ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/chen-2025-research-learning-reason-search-llms/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/)

## References

40 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Marah Abdin, Sahaj Agarwal, Ahmed Awadallah, Vidhisha Balachandran, Harkirat Behl, Lingjiao Chen, Gustavo de Rosa, Suriya Gunasekar, Mojan Javaheripi, Neel Joshi, Piero Kauffmann, Yash Lara, Caio César Teodoro Mendes, Arindam Mitra, Besmira Nushi, Dimitris Papailiopoulos, Olli Saarikivi, Shital Shah… — [Phi-4-reasoning Technical Report](https://md.research.elamur.ai/free/en/papers/abdin-2025-phi-4-reasoning-technical-report/)
2. Art of Problem Solving. 2025 aime i. Art of Problem Solving Wiki, 2025. URL [https://artofproblemsolving.com/wiki/index.php/2025_AIME_I](https://artofproblemsolving.com/wiki/index.php/2025_AIME_I) . Accessed: 2025. [↗](https://artofproblemsolving.com/wiki/index.php/2025_AIME_I)
3. Art of Problem Solving. 2025 aime ii. Art of Problem Solving Wiki, 2025. URL [https://artofproblemsolving.com/wiki/index.php/2025_AIME_II](https://artofproblemsolving.com/wiki/index.php/2025_AIME_II) . Accessed: 2025. [↗](https://artofproblemsolving.com/wiki/index.php/2025_AIME_II)
4. BRUMO. Brown university math olympiad 2025, 2025. URL [https://www.brumo.org/](https://www.brumo.org/) . Accessed: 2025. [↗](https://www.brumo.org/)
5. CMIMC. Cmimc 2025, 2025. URL [https://cmimc.math.cmu.edu/](https://cmimc.math.cmu.edu/) . Accessed: 2025. [↗](https://cmimc.math.cmu.edu/)
6. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). CoRR, abs/2110.14168, 2021.
7. DeepSeek-AI. [Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). arXiv preprint arXiv:2501.12948, 2025.
8. Jasper Dekoninck, Mislav Balunovic, Nikola Jovanović, Ivo Petrov, and Martin Vechev. Mathconstruct: Challenging llm reasoning with constructive proofs. In ICLR 2025 Workshop: VerifAI: AI Verification in the Wild. [find](https://scholar.google.com/scholar?q=Jasper+Dekoninck%2C+Mislav+Balunovic%2C+Nikola+Jovanovic%CC%81%2C+Ivo+Petrov%2C+and+Martin+Vechev%2E+Mathconstruct%3A+Challenging+llm+reasoning+with+constructive+proofs%2E+In+ICLR+2025+Workshop%3A+VerifAI%3A+AI+Verification)
9. Jasper Dekoninck, Ivo Petrov, Kristian Minchev, Mislav Balunovic, Martin T. Vechev, Miroslav Marinov, Maria Drencheva, Lyuba Konova, Milen Shumanov, Kaloyan Tsvetkov, Nikolay Drenchev, Lazar Todorov, Kalina Nikolova, Nikolay Georgiev, Vanesa Kalinkova, and Margulan Ismoldayev. The open proof corpus:… — [The Open Proof Corpus: A Large-Scale Study of LLM-Generated Mathematical Proofs](https://md.research.elamur.ai/free/en/papers/dekoninck-2025-open-proof-corpus-large-scale/)
10. Epoch. Openai and frontiermath. Epoch AI Blog, 2024. URL [https://epoch.ai/blog/](https://epoch.ai/blog/) openai-and-frontiermath . [↗](https://epoch.ai/blog/)
11. Project Euler. Project euler, 2025. URL [https://projecteuler.net/](https://projecteuler.net/) . Accessed: 2025. [↗](https://projecteuler.net/)
12. IMO Foundation. International mathematical olympiad, 2025. URL [https://www.imo-official.org/](https://www.imo-official.org/) . Accessed: 2025. [↗](https://www.imo-official.org/)
13. Simon Frieder, Luca Pinchetti, Alexis Chevalier, Ryan-Rhys Griffiths, Tommaso Salvatori, Thomas Lukasiewicz, Philipp Petersen, and Julius Berner. [Mathematical capabilities of chatgpt](https://md.research.elamur.ai/free/en/papers/frieder-2023-mathematical-capabilities-chatgpt/). In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neu…
14. Bofei Gao, Feifan Song, Zhe Yang, Zefan Cai, Yibo Miao, Qingxiu Dong, Lei Li, Chenghao Ma, Liang Chen, Runxin Xu, Zhengyang Tang, Benyou Wang, Daoguang Zan, Shanghaoran Quan, Ge Zhang, Lei Sha, Yichang Zhang, Xuancheng Ren, Tianyu Liu, and Baobao Chang. Omni-math: A universal olympiad level mathemat… — [Omni-MATH: A Universal Olympiad Level Mathematic Benchmark For Large Language Models](https://md.research.elamur.ai/free/en/papers/gao-2024-omni-math-universal-olympiad-level/)
15. Elliot Glazer, Ege Erdil, Tamay Besiroglu, Diego Chicharro, Evan Chen, Alex Gunning, Caroline Falkman Olsson, Jean-Stanislas Denain, Anson Ho, Emily de Oliveira Santos, Olli Järviniemi, Matthew Barnett, Robert Sandler, Matej Vrzala, Jaime Sevilla, Qiuyu Ren, Elizabeth Pratt, Lionel Levine, Grant Bar… — [FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI](https://md.research.elamur.ai/free/en/papers/glazer-2024-frontiermath-benchmark-evaluating-advanced-mathematical/)
16. Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, and Maosong Sun. [Olympiadbench: A challenging benchmark for promoting AGI with olympiad-level bilingual multimodal scientific problems](https://md.research.elamur.ai/free/en/papers/he-2024-olympiadbench-challenging-benchmark-promoting-agi/). In ACL (1…
17. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. [Measuring mathematical problem solving with the MATH dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). In NeurIPS Datasets and Benchmarks, 2021.
18. HMMT. Hmmt 2025, 2025. URL [https://www.hmmt.org/](https://www.hmmt.org/) . Accessed: 2025. [↗](https://www.hmmt.org/)
19. Kaixuan Huang, Jiacheng Guo, Zihao Li, Xiang Ji, Jiawei Ge, Wenzhe Li, Yingqing Guo, Tianle Cai, Hui Yuan, Runzhe Wang, Yue Wu, Ming Yin, Shange Tang, Yangsibo Huang, Chi Jin, Xinyun Chen, Chiyuan Zhang, and Mengdi Wang. Math-perturb: Benchmarking llms’ math reasoning abilities against hard perturba… [find](https://scholar.google.com/scholar?q=Kaixuan+Huang%2C+Jiacheng+Guo%2C+Zihao+Li%2C+Xiang+Ji%2C+Jiawei+Ge%2C+Wenzhe+Li%2C+Yingqing+Guo%2C+Tianle+Cai%2C+Hui+Yuan%2C+Runzhe+Wang%2C+Yue+Wu%2C+Ming+Yin%2C+Shange+Tang%2C+Yangsibo+Huang%2C+Chi+Jin%2C+Xinyun+Chen%2C+Chiyuan+Zha)
20. Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. [Openai o1 system card](https://md.research.elamur.ai/free/en/papers/openai-2024-openai-o1-system-card/). arXiv preprint arXiv:2412.16720, 2024.
21. Chengwu Liu, Jianhao Shen, Huajian Xin, Zhengying Liu, Ye Yuan, Haiming Wang, Wei Ju, Chuanyang Zheng, Yichun Yin, Lin Li, Ming Zhang, and Qun Liu. [FIMO: A challenge formal dataset for automated theorem proving](https://md.research.elamur.ai/free/en/papers/liu-2023-fimo-challenge-formal-dataset-automated/). CoRR, abs/2309.04295, 2023.
22. Hamed Mahdavi, Alireza Hashemi, Majid Daliri, Pegah Mohammadipour, Alireza Farhadi, Samira Malek, Yekta Yazdanifard, Amir Khasahmadi, and Vasant Honavar. Brains vs. bytes: Evaluating llm proficiency in olympiad mathematics. arXiv preprint arXiv:2504.01995, 2025. [find](https://scholar.google.com/scholar?q=Hamed+Mahdavi%2C+Alireza+Hashemi%2C+Majid+Daliri%2C+Pegah+Mohammadipour%2C+Alireza+Farhadi%2C+Samira+Malek%2C+Yekta+Yazdanifard%2C+Amir+Khasahmadi%2C+and+Vasant+Honavar%2E+Brains+vs%2E+bytes%3A+Evaluating+llm+proficiency+i)
23. Sadegh Mahdavi, Muchen Li, Kaiwen Liu, Christos Thrampoulidis, Leonid Sigal, and Renjie Liao. [Leveraging online olympiad-level math problems for llms training and contamination-resistant evaluation](https://md.research.elamur.ai/free/en/papers/mahdavi-2025-leveraging-online-olympiad-level-math/). CoRR, abs/2501.14275, 2025.
24. Iman Mirzadeh, Keivan Alizadeh, Hooman Shahrokhi, Oncel Tuzel, Samy Bengio, and Mehrdad Farajtabar. [Gsm-symbolic: Understanding the limitations of mathematical reasoning in large language models](https://md.research.elamur.ai/free/en/papers/mirzadeh-2024-gsm-symbolic-understanding-limitations-mathematical/). In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-2…
25. Mathematical Association of America. 2025 putnam mathematical competition, 2025. URL [https://maa.org/putnam/](https://maa.org/putnam/) . Accessed: 2025. [↗](https://maa.org/putnam/)
26. Art of Problem Solving. 2025 usa math olympiad, 2025. URL [https://artofproblemsolving.com/wiki/index.php/2025_USAMO](https://artofproblemsolving.com/wiki/index.php/2025_USAMO) . Accessed: 2025. [↗](https://artofproblemsolving.com/wiki/index.php/2025_USAMO)
27. OpenAI. Deep research, 2025. URL [https://openai.com/index/introducing-deep-research/](https://openai.com/index/introducing-deep-research/) . [↗](https://openai.com/index/introducing-deep-research/)
28. Ivo Petrov, Jasper Dekoninck, Lyuben Baltadzhiev, Maria Drencheva, Kristian Minchev, Mislav Balunović, Nikola Jovanović, and Martin Vechev. Proof or bluff? evaluating llms on 2025 usa math olympiad. arXiv preprint arXiv:2503.21934, 2025. [find](https://scholar.google.com/scholar?q=Ivo+Petrov%2C+Jasper+Dekoninck%2C+Lyuben+Baltadzhiev%2C+Maria+Drencheva%2C+Kristian+Minchev%2C+Mislav+Balunovic%CC%81%2C+Nikola+Jovanovic%CC%81%2C+and+Martin+Vechev%2E+Proof+or+bluff%3F+evaluating+llms+on+2025+usa+math+olympiad%2E)
29. Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Sean Shi, Michael Choi, Anish Agrawal, Arnav Chopra, Adam Khoja, Ryan Kim, Jason Hausenloy, Oliver Zhang, et al. Humanity’s last exam. arXiv, 2025. — [Humanity's Last Exam](https://md.research.elamur.ai/free/en/papers/phan-2025-humanity-s-last-exam/)
30. Haoxiang Sun, Yingqian Min, Zhipeng Chen, Wayne Xin Zhao, Zheng Liu, Zhongyuan Wang, Lei Fang, and Ji-Rong Wen. [Challenging the boundaries of reasoning: An olympiad-level math benchmark for large language models](https://md.research.elamur.ai/free/en/papers/sun-2026-challenging-boundaries-reasoning-olympiad-level/). CoRR, abs/2503.21380, 2025.
31. Gemini Team. [Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities](https://md.research.elamur.ai/free/en/papers/comanici-2025-gemini-2-5-pushing-frontier/). CoRR, abs/2507.06261, 2025. doi: 10.48550/ARXIV.2507.06261. URL https://doi.org/10.48550/arXiv.2507.06261 . [↗](https://doi.org/10.48550/arXiv.2507.06261)
32. George Tsoukalas, Jasper Lee, John Jennings, Jimmy Xin, Michelle Ding, Michael Jennings, Amitayush Thakur, and Swarat Chaudhuri. [Putnambench: Evaluating neural theorem-provers on the putnam mathematical competition](https://md.research.elamur.ai/free/en/papers/tsoukalas-2024-putnambench-evaluating-neural-theorem-provers/). CoRR, abs/2407.11214, 2024.
33. Colin White, Samuel Dooley, Manley Roberts, Arka Pal, Ben Feuer, Siddhartha Jain, Ravid Shwartz-Ziv, Neel Jain, Khalid Saifullah, Siddartha Naidu, et al. Livebench: A challenging, contamination-free llm benchmark. arXiv preprint arXiv:2406.19314, 2024. [find](https://scholar.google.com/scholar?q=Colin+White%2C+Samuel+Dooley%2C+Manley+Roberts%2C+Arka+Pal%2C+Ben+Feuer%2C+Siddhartha+Jain%2C+Ravid+Shwartz%2DZiv%2C+Neel+Jain%2C+Khalid+Saifullah%2C+Siddartha+Naidu%2C+et+al%2E+Livebench%3A+A+challenging%2C+contamination%2Dfree+l)
34. xAI Team. Grok 3 beta — the age of reasoning agents, February 2025. URL [https://x.ai/](https://x.ai/) news/grok-3 . News post. [↗](https://x.ai/)
35. Huaiyuan Ying, Zijian Wu, Yihan Geng, Jiayu Wang, Dahua Lin, and Kai Chen. [Lean workbook: A large-scale lean problem set formalized from natural language math problems](https://md.research.elamur.ai/free/en/papers/ying-2024-lean-workbook-large-scale-lean/). arXiv preprint arXiv:2406.03847, 2024.
36. Albert S. Yue, Lovish Madaan, Ted Moskovitz, DJ Strouse, and Aaditya K. Singh. HARP: A challenging human-annotated math reasoning benchmark. CoRR, abs/2412.08819, 2024. — [arXiv:2412.08819](https://arxiv.org/abs/2412.08819) [find](https://scholar.google.com/scholar?q=Albert+S%2E+Yue%2C+Lovish+Madaan%2C+Ted+Moskovitz%2C+DJ+Strouse%2C+and+Aaditya+K%2E+Singh%2E+HARP%3A+A+challenging+human%2Dannotated+math+reasoning+benchmark%2E+CoRR%2C+abs%2F2412%2E08819%2C+2024%2E+%E2%80%94+%5BarXiv%3A2412%2E08819%5D%28https%3A%2F%2Far)
37. Hugh Zhang, Jeff Da, Dean Lee, Vaughn Robinson, Catherine Wu, William Song, Tiffany Zhao, Pranav Raja, Charlotte Zhuang, Dylan Slack, et al. [A careful examination of large language model performance on grade school arithmetic](https://md.research.elamur.ai/free/en/papers/zhang-2024-careful-examination-large-language-model/). Advances in Neural Information Processing Systems, 37:46819–46836, 2024.
38. Jun Zhao, Jingqi Tong, Yurong Mou, Ming Zhang, Qi Zhang, and Xuanjing Huang. Exploring the compositional deficiency of large language models in mathematical reasoning. arXiv preprint arXiv:2405.06680, 2024. — [arXiv:2405.06680](https://arxiv.org/abs/2405.06680) [find](https://scholar.google.com/scholar?q=Jun+Zhao%2C+Jingqi+Tong%2C+Yurong+Mou%2C+Ming+Zhang%2C+Qi+Zhang%2C+and+Xuanjing+Huang%2E+Exploring+the+compositional+deficiency+of+large+language+models+in+mathematical+reasoning%2E+arXiv+preprint+arXiv%3A2405%2E06680%2C)
39. Kunhao Zheng, Jesse Michael Han, and Stanislas Polu. [minif2f: a cross-system benchmark for formal olympiad-level mathematics](https://md.research.elamur.ai/free/en/papers/zheng-2021-minif2f-cross-system-benchmark-formal/). In ICLR. OpenReview.net, 2022.
40. Chengke Zou, Xingang Guo, Rui Yang, Junyu Zhang, Bin Hu, and Huan Zhang. Dynamath: A dynamic visual benchmark for evaluating mathematical reasoning robustness of vision language models. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. O… [find](https://scholar.google.com/scholar?q=Chengke+Zou%2C+Xingang+Guo%2C+Rui+Yang%2C+Junyu+Zhang%2C+Bin+Hu%2C+and+Huan+Zhang%2E+Dynamath%3A+A+dynamic+visual+benchmark+for+evaluating+mathematical+reasoning+robustness+of+vision+language+models%2E+In+The+Thirtee)

*Updated 2 November 2026*