---
title: "MavenCoder — paper by Xu et al. (2026) — summary and ideas"
description: "The paper addresses competition-level code generation with LLMs. 55 references. Among them: 0."
authors: ["ZhenChun Xu", "Yi Cai", "Dajun Zheng", "li Yuan", "Mengchen Zhao", "Qixiang Wang", "Jiexin Wang"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.1403/"
doi: "10.18653/v1/2026.acl-long.1403"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/xu-2026-mavencoder-competitive-code-generation-model/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/xu-2026-mavencoder-competitive-code-generation-model/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/40/) / MavenCoder: Competitive Code Generation via Model Adaptive Planning Strategies and Multi-Perspective Verification Enhancement

# MavenCoder: Competitive Code Generation via Model Adaptive Planning Strategies and Multi-Perspective Verification Enhancement

**Authors:** ZhenChun Xu, [Yi Cai](https://md.research.elamur.ai/free/en/authors/yi-cai/), Dajun Zheng, [li Yuan](https://md.research.elamur.ai/free/en/authors/li-yuan/), [Mengchen Zhao](https://md.research.elamur.ai/free/en/authors/mengchen-zhao/), Qixiang Wang and [Jiexin Wang](https://md.research.elamur.ai/free/en/authors/jiexin-wang/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.1403/) · [PDF](https://aclanthology.org/2026.acl-long.1403.pdf) · [DOI 10.18653/v1/2026.acl-long.1403](https://doi.org/10.18653/v1/2026.acl-long.1403) · [OpenAlex](https://openalex.org/W7166887141)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper addresses competition-level code generation with LLMs. Current multi-agent planning–coding–debugging pipelines have two weaknesses: they use one fixed planning recipe for every task, and they rarely check intermediate reasoning, so early mistakes carry into the code. MavenCoder addresses both with three components:
- **Difficulty assessment:** it estimates how hard a problem is *for the specific model*. It does this from token-level confidence, while filtering out formatting ("template") tokens that would inflate certainty. It then picks a direct, moderate or deeper planning strategy.
- **Plan verification:** a multi-perspective verification module checks the plan and fixes errors before coding begins.
- **Coding and debugging:** a final stage writes the code and repairs it against public tests.

Across six benchmarks and several backbone LLMs, the framework improves pass@1 over prior agent-based systems. It also typically uses less time and fewer tokens than those systems. The work matters because it shows that matching reasoning effort to the model's own perceived difficulty helps. Early checking of plans also makes LLM program synthesis more accurate and cheaper.

## Key points
- **Headline pass@1:** the framework reaches 87.5% for LiveCodeBench, 93.9% for HumanEval+, 81.7% for MBPP+, and 46.1% for CodeContests. The reported gains over recent agent frameworks range from about 3% to 40%.
- **Confidence measure:**
  - Each token's confidence is the negative mean of its top-k log-probabilities.
  - Windows that contain template tokens (Markdown symbols, filler phrases) are skipped during sliding-window aggregation.
  - k-means then splits the scores into confident and unconfident groups.
- **Efficiency with Qwen3-Coder+:**
  - On MBPP+, MavenCoder took 1h17m and 8.2M tokens, versus 2h and 22.6M tokens for LPW.
  - On LiveCodeBench, it used 5.8M tokens, against over 200M for LPW.
- **Iterations:** more global, verification and debugging rounds generally raise accuracy. Global rounds help most, because they explore alternative reasoning paths.
- **Limitations:**
  - The method needs token log-probabilities, which some closed or reasoning models do not expose.
  - Gains on easy tasks are small.
  - The code is available at github.com/XZC888/MavenCoder.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works · [🖥️ Computer Science Applications](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-science-applications/) · 131 works

🧭 Topics (2): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,685 works · [💻 Code & programming languages](https://md.research.elamur.ai/free/en/topics/code-and-programming/) · 168 works

## Cite

Cite as: ZhenChun Xu, Yi Cai, Dajun Zheng, li Yuan, Mengchen Zhao, Qixiang Wang and Jiexin Wang. 2026. MavenCoder: Competitive Code Generation via Model Adaptive Planning Strategies and Multi-Perspective Verification Enhancement. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{xu2026mavencoder,
  title = {{MavenCoder: Competitive Code Generation via Model Adaptive Planning Strategies and Multi-Perspective Verification Enhancement}},
  author = {ZhenChun Xu, Yi Cai, Dajun Zheng, li Yuan, Mengchen Zhao, Qixiang Wang and Jiexin Wang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1403},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1403}
}
```

- [← previous MaTIAS – Machine Translation to Inform Asylum Seekers: final results](https://md.research.elamur.ai/free/en/papers/wilde-2026-matias-machine-translation-inform-asylum/)
- [next → MAX-EVAL-11: A Large Scale Benchmark for Evaluating Large Language Models on Full-Spectrum ICD-11 Medical Coding](https://md.research.elamur.ai/free/en/papers/singh-2026-max-eval-11-large-scale/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/40/)

## References

55 references, 28 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. David Arthur and Sergei Vassilvitskii. 2007. kmeans++: The advantages of careful seeding. In Proceedings of the eighteenth annual ACM-SIAM symposium on Discrete algorithms, pages 1027–1035. Society for Industrial and Applied Mathematics. [find](https://scholar.google.com/scholar?q=David+Arthur+and+Sergei+Vassilvitskii%2E+2007%2E+kmeans%2B%2B%3A+The+advantages+of+careful+seeding%2E+In+Proceedings+of+the+eighteenth+annual+ACM%2DSIAM+symposium+on+Discrete+algorithms%2C+pages+1027%E2%80%931035%2E+Society+fo)
2. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Charles Sutton. 2021. [Program synthesis with large language models](https://md.research.elamur.ai/free/en/papers/austin-2021-program-synthesis-large-language-models/). Preprint, arXiv:2108.07732.
3. Ian Nery Bandeira, Thiago Veras Machado, Vitor F Dul-lens, and Edna Dias Canedo. 2019. Competitive programming: A teaching methodology analysis applied to first-year programming classes. In 2019 IEEE Frontiers in Education Conference (FIE), pages 1–8. IEEE. [find](https://scholar.google.com/scholar?q=Ian+Nery+Bandeira%2C+Thiago+Veras+Machado%2C+Vitor+F+Dul%2Dlens%2C+and+Edna+Dias+Canedo%2E+2019%2E+Competitive+programming%3A+A+teaching+methodology+analysis+applied+to+first%2Dyear+programming+classes%2E+In+2019+IEEE+)
4. Mark Chen. 2021. [Evaluating large language models trained on code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/). arXiv preprint arXiv:2107.03374.
5. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, and 1 others. 2025. [Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities](https://md.research.elamur.ai/free/en/papers/comanici-2025-gemini-2-5-pushing-frontier/). arX…
6. Dejian Yang Zhenda Xie Kai Dong Wentao Zhang Guanting Chen Xiao Bi Y. Wu Y.K. Li Fuli Luo Yingfei Xiong Wenfeng Liang Daya Guo, Qihao Zhu. 2024. Deepseek-coder: When the large language model meets programming – the rise of code intelligence. [find](https://scholar.google.com/scholar?q=Dejian+Yang+Zhenda+Xie+Kai+Dong+Wentao+Zhang+Guanting+Chen+Xiao+Bi+Y%2E+Wu+Y%2EK%2E+Li+Fuli+Luo+Yingfei+Xiong+Wenfeng+Liang+Daya+Guo%2C+Qihao+Zhu%2E+2024%2E+Deepseek%2Dcoder%3A+When+the+large+language+model+meets+pro)
7. Dr S Rama Devi, Ommi U CH BhagyaSri, R Sravanthi, SL Chaitrika, MN Priyanka, M Swarna, and M Srilekha. 2024. Ai-enhanced cursor navigator. R. and Chaitrika, SL and Priyanka, MN and Swarna, M. and Srilekha, M., AI-Enhanced Cursor Navigator (May 10, 2024). [find](https://scholar.google.com/scholar?q=Dr+S+Rama+Devi%2C+Ommi+U+CH+BhagyaSri%2C+R+Sravanthi%2C+SL+Chaitrika%2C+MN+Priyanka%2C+M+Swarna%2C+and+M+Srilekha%2E+2024%2E+Ai%2Denhanced+cursor+navigator%2E+R%2E+and+Chaitrika%2C+SL+and+Priyanka%2C+MN+and+Swarna%2C+M%2E+and+Sril)
8. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv e-prints, pages arXiv–2407.
9. Ahmed El-Kishky, Alexander Wei, Andre Saraiva, Borys Minaiev, Daniel Selsam, David Dohan, Francis Song, Hunter Lightman, Ignasi Clavera, Jakub Pachocki, and 1 others. 2025. [Competitive programming with large reasoning models](https://md.research.elamur.ai/free/en/papers/el-kishky-2025-competitive-programming-large-reasoning-models/). arXiv preprint arXiv:2502.06807.
10. Angela Fan, Beliz Gokkaya, Mark Harman, Mitya Lyubarskiy, Shubho Sengupta, Shin Yoo, and Jie M Zhang. 2023. Large language models for software engineering: Survey and open problems. In 2023 IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE), pages 3… [find](https://scholar.google.com/scholar?q=Angela+Fan%2C+Beliz+Gokkaya%2C+Mark+Harman%2C+Mitya+Lyubarskiy%2C+Shubho+Sengupta%2C+Shin+Yoo%2C+and+Jie+M+Zhang%2E+2023%2E+Large+language+models+for+software+engineering%3A+Survey+and+open+problems%2E+In+2023+IEEE%2FACM+I)
11. Yichao Fu, Xuewei Wang, Yuandong Tian, and Jiawei Zhao. 2025. [Deep think with confidence](https://md.research.elamur.ai/free/en/papers/fu-2025-deep-think-confidence/). arXiv preprint arXiv:2508.15260.
12. Team Glm, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Dan Zhang, Diego Rojas, Guanyu Feng, Hanlin Zhao, and 1 others. 2024. [Chatglm: A family of large language models from glm-130b to glm-4 all tools](https://md.research.elamur.ai/free/en/papers/glm-2024-chatglm-family-large-language-models/). arXiv preprint arXiv:2406.12793.
13. Sumit Gulwani, Oleksandr Polozov, Rishabh Singh, and 1 others. 2017. [Program synthesis](https://md.research.elamur.ai/free/en/papers/gulwani-2017-program-synthesis/). Foundations and Trends® in Programming Languages, 4(1-2):1–119.
14. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2019. [The curious case of neural text degeneration](https://md.research.elamur.ai/free/en/papers/holtzman-2019-curious-case-neural-text-degeneration/). arXiv preprint arXiv:1904.09751.
15. Dong Huang, Jie M Zhang, Michael Luck, Qingwen Bu, Yuhao Qing, and Heming Cui. 2023. [Agentcoder: Multi-agent-based code generation with iterative testing and optimisation](https://md.research.elamur.ai/free/en/papers/huang-2023-agentcoder-multi-agent-based/). arXiv preprint arXiv:2312.13010.
16. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. [Gpt-4o system card](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/). arXiv preprint arXiv:2410.21276.
17. Md Ashraful Islam, Mohammed Eunus Ali, and Md Rizwan Parvez. 2024. [Mapcoder: Multi-agent code generation for competitive problem solving](https://md.research.elamur.ai/free/en/papers/islam-2024-mapcoder-multi-agent-code/). arXiv preprint arXiv:2405.11403.
18. Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. 2024. [Livecodebench: Holistic and contamination free evaluation of large language models for code](https://md.research.elamur.ai/free/en/papers/jain-2024-livecodebench-holistic-contamination-free-evaluation/). arXiv preprint arXiv:2403.07974.
19. Xue Jiang, Yihong Dong, Lecheng Wang, Zheng Fang, Qiwei Shang, Ge Li, Zhi Jin, and Wenpin Jiao. 2024. [Self-planning code generation with large language models](https://md.research.elamur.ai/free/en/papers/jiang-2023-self-planning-code-generation-large/). ACM Transactions on Software Engineering and Methodology, 33(7):1–30.
20. Zhewei Kang, Xuandong Zhao, and Dawn Song. 2025. Scalable best-of-n selection for large language models via self-certainty. arXiv preprint arXiv:2502.18581. [find](https://scholar.google.com/scholar?q=Zhewei+Kang%2C+Xuandong+Zhao%2C+and+Dawn+Song%2E+2025%2E+Scalable+best%2Dof%2Dn+selection+for+large+language+models+via+self%2Dcertainty%2E+arXiv+preprint+arXiv%3A2502%2E18581%2E)
21. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. [Large language models are zero-shot reasoners](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/). Advances in neural information processing systems, 35:22199– 22213.
22. Chao Lei, Yanchuan Chang, Nir Lipovetzky, and Krista A Ehinger. 2025. Planning-driven programming: A large language model programming workflow. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12647–12684. [find](https://scholar.google.com/scholar?q=Chao+Lei%2C+Yanchuan+Chang%2C+Nir+Lipovetzky%2C+and+Krista+A+Ehinger%2E+2025%2E+Planning%2Ddriven+programming%3A+A+large+language+model+programming+workflow%2E+In+Proceedings+of+the+63rd+Annual+Meeting+of+the+Associa)
23. Jia Li, Yuqi Zhu, Yongmin Li, Ge Li, and Zhi Jin. 2024. Showing llm-generated code selectively based on confidence of llms. arXiv preprint arXiv:2410.03234. — [arXiv:2410.03234](https://arxiv.org/abs/2410.03234) [find](https://scholar.google.com/scholar?q=Jia+Li%2C+Yuqi+Zhu%2C+Yongmin+Li%2C+Ge+Li%2C+and+Zhi+Jin%2E+2024%2E+Showing+llm%2Dgenerated+code+selectively+based+on+confidence+of+llms%2E+arXiv+preprint+arXiv%3A2410%2E03234%2E+%E2%80%94+%5BarXiv%3A2410%2E03234%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F)
24. Jierui Li, Hung Le, Yingbo Zhou, Caiming Xiong, Silvio Savarese, and Doyen Sahoo. 2025. [Codetree: Agent-guided tree search for code generation with large language models](https://md.research.elamur.ai/free/en/papers/li-2025-codetree-agent-guided-tree-search/). In Proceedings of the 2025
25. Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 3711–3726. [find](https://scholar.google.com/scholar?q=Conference+of+the+Nations+of+the+Americas+Chapter+of+the+Association+for+Computational+Linguistics%3A+Human+Language+Technologies+%28Volume+1%3A+Long+Papers%29%2C+pages+3711%E2%80%933726%2E)
26. Jingyao Li, Pengguang Chen, Bin Xia, Hong Xu, and Jiaya Jia. 2023a. Motcoder: Elevating large language models with modular of thought for challenging programming tasks. arXiv preprint arXiv:2312.15960. — [arXiv:2312.15960](https://arxiv.org/abs/2312.15960) [find](https://scholar.google.com/scholar?q=Jingyao+Li%2C+Pengguang+Chen%2C+Bin+Xia%2C+Hong+Xu%2C+and+Jiaya+Jia%2E+2023a%2E+Motcoder%3A+Elevating+large+language+models+with+modular+of+thought+for+challenging+programming+tasks%2E+arXiv+preprint+arXiv%3A2312%2E15960)
27. Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, and 1 others. 2023b. Starcoder: may the source be with you! arXiv preprint arXiv:2305.06161. [find](https://scholar.google.com/scholar?q=Raymond+Li%2C+Loubna+Ben+Allal%2C+Yangtian+Zi%2C+Niklas+Muennighoff%2C+Denis+Kocetkov%2C+Chenghao+Mou%2C+Marc+Marone%2C+Christopher+Akiki%2C+Jia+Li%2C+Jenny+Chim%2C+and+1+others%2E+2023b%2E+Starcoder%3A+may+the+source+be+with+)
28. Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, Thomas Hubert, Peter Choy, Cyprien de Masson d’Autume, Igor Babuschkin, Xinyun Chen, Po-Sen Huang, Johannes Welbl, Sven Gowal, Alexey Cherepanov, and 7 o… — [Competition-Level Code Generation with AlphaCode](https://md.research.elamur.ai/free/en/papers/li-2022-competition-level-code-generation-alphacode/)
29. Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, and 1 others. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437. [find](https://scholar.google.com/scholar?q=Aixin+Liu%2C+Bei+Feng%2C+Bing+Xue%2C+Bingxuan+Wang%2C+Bochao+Wu%2C+Chengda+Lu%2C+Chenggang+Zhao%2C+Chengqi+Deng%2C+Chenyu+Zhang%2C+Chong+Ruan%2C+and+1+others%2E+2024%2E+Deepseek%2Dv3+technical+report%2E+arXiv+preprint+arXiv%3A2412)
30. Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. 2023. [Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation](https://md.research.elamur.ai/free/en/papers/liu-2023-your-code-generated-chatgpt-really/). Advances in Neural Information Processing Systems, 36:21558–21572.
31. Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. 2023. Wizardcoder: Empowering code large language models with evol-instruct. arXiv preprint arXiv:2306.08568. [find](https://scholar.google.com/scholar?q=Ziyang+Luo%2C+Can+Xu%2C+Pu+Zhao%2C+Qingfeng+Sun%2C+Xiubo+Geng%2C+Wenxiang+Hu%2C+Chongyang+Tao%2C+Jing+Ma%2C+Qingwei+Lin%2C+and+Daxin+Jiang%2E+2023%2E+Wizardcoder%3A+Empowering+code+large+language+models+with+evol%2Dinstruct%2E+a)
32. Qing Lyu, Shreya Havaldar, Adam Stein, Li Zhang, Delip Rao, Eric Wong, Marianna Apidianaki, and Chris Callison-Burch. 2023. [Faithful chain-of-thought reasoning](https://md.research.elamur.ai/free/en/papers/lyu-2023-faithful-chain-thought-reasoning/). In The 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Associa…
33. Zohar Manna and Richard Waldinger. 1980. A deductive approach to program synthesis. ACM Transactions on Programming Languages and Systems (TOPLAS), 2(1):90–121. [find](https://scholar.google.com/scholar?q=Zohar+Manna+and+Richard+Waldinger%2E+1980%2E+A+deductive+approach+to+program+synthesis%2E+ACM+Transactions+on+Programming+Languages+and+Systems+%28TOPLAS%29%2C+2%281%29%3A90%E2%80%93121%2E)
34. Zohar Manna and Richard J Waldinger. 1971. Toward automatic program synthesis. Communications of the ACM, 14(3):151–165. [find](https://scholar.google.com/scholar?q=Zohar+Manna+and+Richard+J+Waldinger%2E+1971%2E+Toward+automatic+program+synthesis%2E+Communications+of+the+ACM%2C+14%283%29%3A151%E2%80%93165%2E)
35. Ibomoiye Domor Mienye, Nobert Jere, George Obaido, Oyindamola Omolara Ogunruku, Ebenezer Esenogho, and Cameron Modisane. 2025. Large language models: an overview of foundational architectures, recent trends, and a new taxonomy. Discover Applied Sciences, 7(9):1027. [find](https://scholar.google.com/scholar?q=Ibomoiye+Domor+Mienye%2C+Nobert+Jere%2C+George+Obaido%2C+Oyindamola+Omolara+Ogunruku%2C+Ebenezer+Esenogho%2C+and+Cameron+Modisane%2E+2025%2E+Large+language+models%3A+an+overview+of+foundational+architectures%2C+recent+)
36. Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. 2022. Codegen: An open large language model for code with multi-turn program synthesis. arXiv preprint arXiv:2203.13474. [find](https://scholar.google.com/scholar?q=Erik+Nijkamp%2C+Bo+Pang%2C+Hiroaki+Hayashi%2C+Lifu+Tu%2C+Huan+Wang%2C+Yingbo+Zhou%2C+Silvio+Savarese%2C+and+Caiming+Xiong%2E+2022%2E+Codegen%3A+An+open+large+language+model+for+code+with+multi%2Dturn+program+synthesis%2E+arX)
37. OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, and 262 others.… — [GPT-4 Technical Report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/)
38. Pejman Peykani, Fatemeh Ramezanlou, Cristina Tanas-escu, and Sanly Ghanidel. 2025. Large language models: A structured taxonomy and review of challenges, limitations, solutions, and future directions. Applied Sciences, 15(14):8103. [find](https://scholar.google.com/scholar?q=Pejman+Peykani%2C+Fatemeh+Ramezanlou%2C+Cristina+Tanas%2Descu%2C+and+Sanly+Ghanidel%2E+2025%2E+Large+language+models%3A+A+structured+taxonomy+and+review+of+challenges%2C+limitations%2C+solutions%2C+and+future+directions%2E)
39. Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A Smith, and Mike Lewis. 2023. Measuring and narrowing the compositionality gap in language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5687–5711. [find](https://scholar.google.com/scholar?q=Ofir+Press%2C+Muru+Zhang%2C+Sewon+Min%2C+Ludwig+Schmidt%2C+Noah+A+Smith%2C+and+Mike+Lewis%2E+2023%2E+Measuring+and+narrowing+the+compositionality+gap+in+language+models%2E+In+Findings+of+the+Association+for+Computati)
40. Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, and 1 others. 2023. [Code llama: Open foundation models for code](https://md.research.elamur.ai/free/en/papers/roziere-2024-code-llama-open-foundation-models/). arXiv preprint arXiv:2308.12950.
41. Cody Rushing and Neel Nanda. 2024. Explorations of self-repair in language models. arXiv preprint arXiv:2402.15390. — [arXiv:2402.15390](https://arxiv.org/abs/2402.15390) [find](https://scholar.google.com/scholar?q=Cody+Rushing+and+Neel+Nanda%2E+2024%2E+Explorations+of+self%2Drepair+in+language+models%2E+arXiv+preprint+arXiv%3A2402%2E15390%2E+%E2%80%94+%5BarXiv%3A2402%2E15390%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2402%2E15390%29)
42. Jiho Shin, Jaechang Nam, and 1 others. 2021. A survey of automatic code generation from natural language. Journal of Information Processing Systems, 17(3):537–555. [find](https://scholar.google.com/scholar?q=Jiho+Shin%2C+Jaechang+Nam%2C+and+1+others%2E+2021%2E+A+survey+of+automatic+code+generation+from+natural+language%2E+Journal+of+Information+Processing+Systems%2C+17%283%29%3A537%E2%80%93555%2E)
43. Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. [Reflexion: Language agents with verbal reinforcement learning](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/). Advances in Neural Information Processing Systems, 36:8634–8652.
44. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. [Sequence to sequence learning with neural networks](https://md.research.elamur.ai/free/en/papers/sutskever-2014-sequence-sequence-learning-neural-networks/). Preprint, arXiv:1409.3215.
45. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). Advances in neural information processing systems, 35:24824– 24837.
46. Tianci Xue, Ziqi Wang, Zhenhailong Wang, Chi Han, Pengfei Yu, and Heng Ji. 2023. Rcot: Detecting and rectifying factual inconsistency in reasoning by reversing chain-of-thought. Preprint, arXiv:2305.11499. — [arXiv:2305.11499](https://arxiv.org/abs/2305.11499) [find](https://scholar.google.com/scholar?q=Tianci+Xue%2C+Ziqi+Wang%2C+Zhenhailong+Wang%2C+Chi+Han%2C+Pengfei+Yu%2C+and+Heng+Ji%2E+2023%2E+Rcot%3A+Detecting+and+rectifying+factual+inconsistency+in+reasoning+by+reversing+chain%2Dof%2Dthought%2E+Preprint%2C+arXiv%3A2305%2E1)
47. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). arXiv preprint arXiv:2505.09388.
48. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems, 36:11809–11822. [find](https://scholar.google.com/scholar?q=Shunyu+Yao%2C+Dian+Yu%2C+Jeffrey+Zhao%2C+Izhak+Shafran%2C+Tom+Griffiths%2C+Yuan+Cao%2C+and+Karthik+Narasimhan%2E+2023%2E+Tree+of+thoughts%3A+Deliberate+problem+solving+with+large+language+models%2E+Advances+in+neural+inf)
49. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2022. [React: Synergizing reasoning and acting in language models](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/). In The eleventh international conference on learning representations.
50. Kevin KF Yuen, Dennis YW Liu, and Hong Va Leong. 2023. Competitive programming in computational thinking and problem solving education. Computer Applications in Engineering Education, 31(4):850– 866. [find](https://scholar.google.com/scholar?q=Kevin+KF+Yuen%2C+Dennis+YW+Liu%2C+and+Hong+Va+Leong%2E+2023%2E+Competitive+programming+in+computational+thinking+and+problem+solving+education%2E+Computer+Applications+in+Engineering+Education%2C+31%284%29%3A850%E2%80%93+866%2E)
51. Huan Zhang, Wei Cheng, Yuhan Wu, and Wei Hu. 2024. A pair programming framework for code generation via multi-plan exploration and feedback-driven refinement. In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering, pages 1319–1331. [find](https://scholar.google.com/scholar?q=Huan+Zhang%2C+Wei+Cheng%2C+Yuhan+Wu%2C+and+Wei+Hu%2E+2024%2E+A+pair+programming+framework+for+code+generation+via+multi%2Dplan+exploration+and+feedback%2Ddriven+refinement%2E+In+Proceedings+of+the+39th+IEEE%2FACM+Inter)
52. Jian Zhang, Zhangqi Wang, Haiping Zhu, Kangda Cheng, Kai He, Bo Li, Qika Lin, Jun Liu, and Erik Cambria. 2026a. Mars: Multi-agent adaptive reasoning with socratic guidance for automated prompt optimization. Proceedings of the AAAI Conference on Artificial Intelligence, 40(19):16307–16315. [find](https://scholar.google.com/scholar?q=Jian+Zhang%2C+Zhangqi+Wang%2C+Haiping+Zhu%2C+Kangda+Cheng%2C+Kai+He%2C+Bo+Li%2C+Qika+Lin%2C+Jun+Liu%2C+and+Erik+Cambria%2E+2026a%2E+Mars%3A+Multi%2Dagent+adaptive+reasoning+with+socratic+guidance+for+automated+prompt+optimiz)
53. Jian Zhang, Zhiyuan Wang, Zhangqi Wang, Fangzhi Xu, Qika Lin, Lingling Zhang, Rui Mao, Erik Cambria, and Jun Liu. 2026b. Maps: Multi-agent personality shaping for collaborative reasoning. Proceedings of the AAAI Conference on Artificial Intelligence, 40(19):16316–16324. [find](https://scholar.google.com/scholar?q=Jian+Zhang%2C+Zhiyuan+Wang%2C+Zhangqi+Wang%2C+Fangzhi+Xu%2C+Qika+Lin%2C+Lingling+Zhang%2C+Rui+Mao%2C+Erik+Cambria%2C+and+Jun+Liu%2E+2026b%2E+Maps%3A+Multi%2Dagent+personality+shaping+for+collaborative+reasoning%2E+Proceedings+)
54. Shun Zhang, Zhenfang Chen, Yikang Shen, Mingyu Ding, Joshua B Tenenbaum, and Chuang Gan. 2023. [Planning with large language models for code generation](https://md.research.elamur.ai/free/en/papers/zhang-2023-planning-large-language-models-code/). arXiv preprint arXiv:2303.05510.
55. Li Zhong, Zilong Wang, and Jingbo Shang. 2024. [Debug like a human: A large language model debugger via verifying runtime execution step-by-step](https://md.research.elamur.ai/free/en/papers/zhong-2024-debug-like-human-large-language/). arXiv preprint arXiv:2402.16906.

*Updated 2 November 2026*