---
title: "V-GameGym — paper by Zhang et al. (2026)"
description: "Existing code benchmarks mostly check whether generated programs compile and pass tests. 52 references, cited by 1 paper here."
authors: ["Wei Zhang", "Jian Yang", "Renshuai Tao", "Linzheng Chai", "Shuyue Guo", "Jiajun Wu", "Xiaoming Chen", "Ganqu Cui", "Ning Ding", "Xander Xu", "HU Wei", "Bowen Zhou"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.276/"
doi: "10.18653/v1/2026.findings-acl.276"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2026-v-gamegym-visual-game-generation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2026-v-gamegym-visual-game-generation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/19/) / V-GameGym: Visual Game Generation for Code Large Language Models

# V-GameGym: Visual Game Generation for Code Large Language Models

**Authors:** [Wei Zhang](https://md.research.elamur.ai/free/en/authors/wei-zhang/), [Jian Yang](https://md.research.elamur.ai/free/en/authors/jian-yang/), Renshuai Tao, [Linzheng Chai](https://md.research.elamur.ai/free/en/authors/linzheng-chai/), [Shuyue Guo](https://md.research.elamur.ai/free/en/authors/shuyue-guo/), [Jiajun Wu](https://md.research.elamur.ai/free/en/authors/jiajun-wu/), Xiaoming Chen, [Ganqu Cui](https://md.research.elamur.ai/free/en/authors/ganqu-cui/), [Ning Ding](https://md.research.elamur.ai/free/en/authors/ning-ding/), Xander Xu, HU Wei and [Bowen Zhou](https://md.research.elamur.ai/free/en/authors/bowen-zhou/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.276/) · [PDF](https://aclanthology.org/2026.findings-acl.276.pdf) · [DOI 10.18653/v1/2026.findings-acl.276](https://doi.org/10.18653/v1/2026.findings-acl.276) · [OpenAlex](https://openalex.org/W7166860548)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/zhang-2026-v-gamegym-visual-game-generation/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Existing code benchmarks mostly check whether generated programs compile and pass tests. They largely ignore what matters for games: whether the result is playable, looks right and behaves coherently over time. V-GameGym fills this gap with a benchmark for visual game generation in Python/Pygame. The authors mined Pygame-related files from OpenCoder and The Stack v2 and grouped them by feature vectors (size, structure, API usage, semantics) using MiniBatchKMeans. From each group they kept the most structurally complete program. An LLM-driven pipeline then turned these seeds into self-contained, autonomously running demos, fixed execution errors in a sandbox, and wrote natural-language requirements. Eight graduate students verified the results by hand. Evaluation runs each generated program, records screenshots and video, and has LLM/MLLM judges score the code, static visuals and gameplay. Testing 70 models showed a clear lead for proprietary systems, but even the best succeed on only about 45% of tasks. Most outputs land in low quality bands, so the benchmark exposes real weaknesses in visual and interactive code generation.

## Key points
- The dataset has 2,219 human-checked samples from 2,190 distinct repositories, spread across 100 clusters. Reference programs average about 257 lines of code, about 9.9 screenshots and roughly 10 s of recorded video.
- The final score is a weighted combination of three judge-based assessments (code, image, video). Scores fall into four bands: Excellent 80–100, Good 60–80, Fair 40–60 and Poor 0–40.
- Performance grows roughly with the logarithm of model size (R² = 0.545). The authors report averages of about 20 points for models under 10B and over 40 points for 400B+ models.
- Code, screenshot and video scores are positively correlated. Some frontier models, such as GPT-5, achieve near-perfect code scores but weaker visual scores, which the authors read as a possible trade-off.
- Stated limitations:
  - the benchmark covers Python only;
  - it relies on LLM judges, which may not capture playability or aesthetics well;
  - tasks are mostly single-file games.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Alibaba Group](https://md.research.elamur.ai/free/en/companies/alibaba-group/) — Company, Hangzhou, China · 201 works · 17,173 citations
- [Shanghai Artificial Intelligence Laboratory](https://md.research.elamur.ai/free/en/labs/shanghai-artificial-intelligence-laboratory/) — Research institute, Shanghai, China · 152 works · 13,166 citations
- [Beijing Jiaotong University](https://md.research.elamur.ai/free/en/universities/beijing-jiaotong-university/) — University, Beijing, China · 28 works · 1,754 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works

## Cite

Cite as: Wei Zhang, Jian Yang, Renshuai Tao, Linzheng Chai, Shuyue Guo, Jiajun Wu, Xiaoming Chen, Ganqu Cui, Ning Ding, Xander Xu, HU Wei and Bowen Zhou. 2026. V-GameGym: Visual Game Generation for Code Large Language Models. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhang2026v,
  title = {{V-GameGym: Visual Game Generation for Code Large Language Models}},
  author = {Wei Zhang, Jian Yang, Renshuai Tao, Linzheng Chai, Shuyue Guo, Jiajun Wu, Xiaoming Chen, Ganqu Cui, Ning Ding, Xander Xu, HU Wei and Bowen Zhou},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.276},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.276}
}
```

- [← previous uva-irlab-conv at SemEval-2026 Task 8: Multi-Turn RAG with Learned Sparse Retrieval and Listwise Reranking](https://md.research.elamur.ai/free/en/papers/lupart-2026-uva-irlab-conv-semeval-2026/)
- [next → VARH-AI at SemEval-2026 Task 10: Exploiting Architectural Diversity with Transformer-SSM Ensembles and Confidence-Based Iterative Refinement for Conspiracy Detection](https://md.research.elamur.ai/free/en/papers/solanki-2026-varh-ai-semeval-2026-task/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/19/)

## References

52 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Anthropic. 2025. Claude 3.7 sonnet and claude code. [find](https://scholar.google.com/scholar?q=Anthropic%2E+2025%2E+Claude+3%2E7+sonnet+and+claude+code%2E)
2. Federico Cassano, John Gouwar, Daniel Nguyen, Sydney Nguyen, Luna Phipps-Costin, Donald Pinckney, Ming-Ho Yee, Yangtian Zi, Carolyn Jane Anderson, Molly Q Feldman, and 1 others. 2023. Multipl-e: a scalable and polyglot approach to benchmarking neural code generation. IEEE Transactions on Software En… [find](https://scholar.google.com/scholar?q=Federico+Cassano%2C+John+Gouwar%2C+Daniel+Nguyen%2C+Sydney+Nguyen%2C+Luna+Phipps%2DCostin%2C+Donald+Pinckney%2C+Ming%2DHo+Yee%2C+Yangtian+Zi%2C+Carolyn+Jane+Anderson%2C+Molly+Q+Feldman%2C+and+1+others%2E+2023%2E+Multipl%2De%3A+a+sca)
3. Linzheng Chai, Shukai Liu, Jian Yang, Yuwei Yin, Ke Jin, Jiaheng Liu, Tao Sun, Ge Zhang, Changyu Ren, Hongcheng Guo, and 1 others. 2024. Mceval: Massively multilingual code evaluation. arXiv preprint arXiv:2406.07436. — [arXiv:2406.07436](https://arxiv.org/abs/2406.07436) [find](https://scholar.google.com/scholar?q=Linzheng+Chai%2C+Shukai+Liu%2C+Jian+Yang%2C+Yuwei+Yin%2C+Ke+Jin%2C+Jiaheng+Liu%2C+Tao+Sun%2C+Ge+Zhang%2C+Changyu+Ren%2C+Hongcheng+Guo%2C+and+1+others%2E+2024%2E+Mceval%3A+Massively+multilingual+code+evaluation%2E+arXiv+preprint+)
4. Sahil Chaudhary. 2023. Code alpaca: An instruction-following llama model for code generation. [https://github.com/sahil280114/codealpaca](https://github.com/sahil280114/codealpaca). [↗](https://github.com/sahil280114/codealpaca)
5. Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, and 39 others. 2021. [Evaluating large language models trained on code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/). abs/2107.03374.
6. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, and 181 others. 2025a. [Deepseek-r1: Incentivizing reasoning capability in l…](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/)
7. DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, and 181 others. 2025b. Deepseek-v3 technical report. Preprint, arXiv:… [find](https://scholar.google.com/scholar?q=DeepSeek%2DAI%2C+Aixin+Liu%2C+Bei+Feng%2C+Bing+Xue%2C+Bingxuan+Wang%2C+Bochao+Wu%2C+Chengda+Lu%2C+Chenggang+Zhao%2C+Chengqi+Deng%2C+Chenyu+Zhang%2C+Chong+Ruan%2C+Damai+Dai%2C+Daya+Guo%2C+Dejian+Yang%2C+Deli+Chen%2C+Dongjie+Ji%2C+Erhan)
8. Ran Gong, Qiuyuan Huang, Xiaojian Ma, Yusuke Noda, Zane Durante, Zilong Zheng, Demetri Terzopoulos, Li Fei-Fei, Jianfeng Gao, and Hoi Vo. 2024. Mindagent: Emergent gaming interaction. In Findings of the Association for Computational Linguistics: NAACL 2024, Mexico City, Mexico, June 16-21, 2024, pag… [find](https://scholar.google.com/scholar?q=Ran+Gong%2C+Qiuyuan+Huang%2C+Xiaojian+Ma%2C+Yusuke+Noda%2C+Zane+Durante%2C+Zilong+Zheng%2C+Demetri+Terzopoulos%2C+Li+Fei%2DFei%2C+Jianfeng+Gao%2C+and+Hoi+Vo%2E+2024%2E+Mindagent%3A+Emergent+gaming+interaction%2E+In+Findings+of+t)
9. Google. 2025a. gemini 2.5-flash. [find](https://scholar.google.com/scholar?q=Google%2E+2025a%2E+gemini+2%2E5%2Dflash%2E)
10. Google. 2025b. gemini 2.5-pro. [find](https://scholar.google.com/scholar?q=Google%2E+2025b%2E+gemini+2%2E5%2Dpro%2E)
11. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 o… — [The Llama 3 Herd of Models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/)
12. Leon Guertler, Bobby Cheng, Simon Yu, Bo Liu, Leshem Choshen, and Cheston Tan. 2025. Textarena. arXiv preprint arXiv:2504.11442. — [arXiv:2504.11442](https://arxiv.org/abs/2504.11442) [find](https://scholar.google.com/scholar?q=Leon+Guertler%2C+Bobby+Cheng%2C+Simon+Yu%2C+Bo+Liu%2C+Leshem+Choshen%2C+and+Cheston+Tan%2E+2025%2E+Textarena%2E+arXiv+preprint+arXiv%3A2504%2E11442%2E+%E2%80%94+%5BarXiv%3A2504%2E11442%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2504%2E11442%29)
13. Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Y. Wu, Y. K. Li, Fuli Luo, Yingfei Xiong, and Wenfeng Liang. 2024a. Deepseek-coder: When the large language model meets programming – the rise of code intelligence. Preprint, arXiv:2401.14196. [find](https://scholar.google.com/scholar?q=Daya+Guo%2C+Qihao+Zhu%2C+Dejian+Yang%2C+Zhenda+Xie%2C+Kai+Dong%2C+Wentao+Zhang%2C+Guanting+Chen%2C+Xiao+Bi%2C+Y%2E+Wu%2C+Y%2E+K%2E+Li%2C+Fuli+Luo%2C+Yingfei+Xiong%2C+and+Wenfeng+Liang%2E+2024a%2E+Deepseek%2Dcoder%3A+When+the+large+languag)
14. Dake Chen, Hanbin Wang, Yunhao Huo, Yuzhao Li, and Haoyang Zhang. 2023. Gamegpt: Multi-agent collaborative framework for game development. arXiv preprint arXiv:2310.08067. [find](https://scholar.google.com/scholar?q=Dake+Chen%2C+Hanbin+Wang%2C+Yunhao+Huo%2C+Yuzhao+Li%2C+and+Haoyang+Zhang%2E+2023%2E+Gamegpt%3A+Multi%2Dagent+collaborative+framework+for+game+development%2E+arXiv+preprint+arXiv%3A2310%2E08067%2E)
15. Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Y Wu, YK Li, and 1 others. 2024b. Deepseek-coder: When the large language model meets programming–the rise of code intelligence. arXiv preprint arXiv:2401.14196. [find](https://scholar.google.com/scholar?q=Daya+Guo%2C+Qihao+Zhu%2C+Dejian+Yang%2C+Zhenda+Xie%2C+Kai+Dong%2C+Wentao+Zhang%2C+Guanting+Chen%2C+Xiao+Bi%2C+Y+Wu%2C+YK+Li%2C+and+1+others%2E+2024b%2E+Deepseek%2Dcoder%3A+When+the+large+language+model+meets+programming%E2%80%93the+rise)
16. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pondé de Oliveira Pinto, Jared Kaplan, Harrison Edwards, Yuri Burda, Nicholas Joseph, Greg Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V. Chawla, Olaf Wiest, and Xiangliang Zhang. 2024c. [Large language model based…](https://md.research.elamur.ai/free/en/papers/guo-2024-large-language-model-based/)
17. Siming Huang, Tianhao Cheng, J. K. Liu, Jiaran Hao, Liuyihan Song, Yang Xu, J. Yang, Jiaheng Liu, Chenchen Zhang, Linzheng Chai, Ruifeng Yuan, Zhaoxiang Zhang, Jie Fu, Qian Liu, Ge Zhang, Zili Wang, Yuan Qi, Yinghui Xu, and Wei Chu. 2025. [Opencoder: The open cookbook for top-tier code large language…](https://md.research.elamur.ai/free/en/papers/huang-2025-opencoder-open-cookbook-top-tier/)
18. Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Kai Dang, and 1 others. 2024. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186. — [Qwen2.5-Coder Technical Report](https://md.research.elamur.ai/free/en/papers/hui-2024-qwen2-5-coder-technical-report/)
19. Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, Qian Liu, Evgenii Zheltonozhskii, Terry Yue Zhuo, Thomas Wang, Olivier Dehaene, Mishig Davaadorj, Joel Lamy-Poirier, João Monteiro, Oleh Shliazhko, and 48… [find](https://scholar.google.com/scholar?q=Raymond+Li%2C+Loubna+Ben+Allal%2C+Yangtian+Zi%2C+Niklas+Muennighoff%2C+Denis+Kocetkov%2C+Chenghao+Mou%2C+Marc+Marone%2C+Christopher+Akiki%2C+Jia+Li%2C+Jenny+Chim%2C+Qian+Liu%2C+Evgenii+Zheltonozhskii%2C+Terry+Yue+Zhuo%2C+Thoma)
20. Jonathan Light, Min Cai, Sheng Shen, and Ziniu Hu. 2023. Avalonbench: Evaluating llms playing the game of avalon. arXiv preprint arXiv:2310.05036. — [arXiv:2310.05036](https://arxiv.org/abs/2310.05036) [find](https://scholar.google.com/scholar?q=Jonathan+Light%2C+Min+Cai%2C+Sheng+Shen%2C+and+Ziniu+Hu%2E+2023%2E+Avalonbench%3A+Evaluating+llms+playing+the+game+of+avalon%2E+arXiv+preprint+arXiv%3A2310%2E05036%2E+%E2%80%94+%5BarXiv%3A2310%2E05036%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2310%2E05036)
21. Shukai Liu, Linzheng Chai, Jian Yang, Jiajun Shi, He Zhu, Liran Wang, Ke Jin, Wei Zhang, Hualei Zhu, Shuyue Guo, and 1 others. 2024a. [Mdeval: Massively multilingual code debugging](https://md.research.elamur.ai/free/en/papers/liu-2026-mdeval-massively-multilingual-code-debugging/). arXiv preprint arXiv:2411.02310.
22. Siyao Liu, He Zhu, Jerry Liu, Shulin Xin, Aoyan Li, Rui Long, Li Chen, Jack Yang, Jinxiang Xia, ZY Peng, and 1 others. 2024b. Fullstack bench: Evaluating llms as full stack coder. arXiv preprint arXiv:2412.00535. — [arXiv:2412.00535](https://arxiv.org/abs/2412.00535) [find](https://scholar.google.com/scholar?q=Siyao+Liu%2C+He+Zhu%2C+Jerry+Liu%2C+Shulin+Xin%2C+Aoyan+Li%2C+Rui+Long%2C+Li+Chen%2C+Jack+Yang%2C+Jinxiang+Xia%2C+ZY+Peng%2C+and+1+others%2E+2024b%2E+Fullstack+bench%3A+Evaluating+llms+as+full+stack+coder%2E+arXiv+preprint+arXiv)
23. Anton Lozhkov, Raymond Li, Loubna Ben Allal, Federico Cassano, Joel Lamy-Poirier, Nouamane Tazi, Ao Tang, Dmytro Pykhtar, Jiawei Liu, Yuxiang Wei, Tianyang Liu, Max Tian, Denis Kocetkov, Arthur Zucker, Younes Belkada, Zijian Wang, Qian Liu, Dmitry Abulkhanov, Indraneil Paul, and 47 others. 2024a. St… [find](https://scholar.google.com/scholar?q=Anton+Lozhkov%2C+Raymond+Li%2C+Loubna+Ben+Allal%2C+Federico+Cassano%2C+Joel+Lamy%2DPoirier%2C+Nouamane+Tazi%2C+Ao+Tang%2C+Dmytro+Pykhtar%2C+Jiawei+Liu%2C+Yuxiang+Wei%2C+Tianyang+Liu%2C+Max+Tian%2C+Denis+Kocetkov%2C+Arthur+Zucker)
24. Anton Lozhkov, Raymond Li, Loubna Ben Allal, Federico Cassano, Joel Lamy-Poirier, Nouamane Tazi, Ao Tang, Dmytro Pykhtar, Jiawei Liu, Yuxiang Wei, and 1 others. 2024b. Starcoder 2 and the stack v2: The next generation. arXiv preprint arXiv:2402.19173. [find](https://scholar.google.com/scholar?q=Anton+Lozhkov%2C+Raymond+Li%2C+Loubna+Ben+Allal%2C+Federico+Cassano%2C+Joel+Lamy%2DPoirier%2C+Nouamane+Tazi%2C+Ao+Tang%2C+Dmytro+Pykhtar%2C+Jiawei+Liu%2C+Yuxiang+Wei%2C+and+1+others%2E+2024b%2E+Starcoder+2+and+the+stack+v2%3A+Th)
25. Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin B. Clement, Dawn Drain, Daxin Jiang, Duyu Tang, Ge Li, Lidong Zhou, Linjun Shou, Long Zhou, Michele Tufano, Ming Gong, Ming Zhou, Nan Duan, Neel Sundaresan, and 3 others. 2021. Codexglue: A machine learning bench… [find](https://scholar.google.com/scholar?q=Shuai+Lu%2C+Daya+Guo%2C+Shuo+Ren%2C+Junjie+Huang%2C+Alexey+Svyatkovskiy%2C+Ambrosio+Blanco%2C+Colin+B%2E+Clement%2C+Dawn+Drain%2C+Daxin+Jiang%2C+Duyu+Tang%2C+Ge+Li%2C+Lidong+Zhou%2C+Linjun+Shou%2C+Long+Zhou%2C+Michele+Tufano%2C+Ming)
26. Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. 2023. Wizardcoder: Empowering code large language models with evol-instruct. CoRR, abs/2306.08568. [find](https://scholar.google.com/scholar?q=Ziyang+Luo%2C+Can+Xu%2C+Pu+Zhao%2C+Qingfeng+Sun%2C+Xiubo+Geng%2C+Wenxiang+Hu%2C+Chongyang+Tao%2C+Jing+Ma%2C+Qingwei+Lin%2C+and+Daxin+Jiang%2E+2023%2E+Wizardcoder%3A+Empowering+code+large+language+models+with+evol%2Dinstruct%2E+C)
27. OpenAI. 2023. [GPT-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). abs/2303.08774.
28. CoRR, OpenAI. 2025a. Introducing gpt-4.5. [find](https://scholar.google.com/scholar?q=CoRR%2C+OpenAI%2E+2025a%2E+Introducing+gpt%2D4%2E5%2E)
29. OpenAI. 2025b. Introducing gpt-oss. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2025b%2E+Introducing+gpt%2Doss%2E)
30. OpenAI. 2025c. Introducing o3-and-o4-mini. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2025c%2E+Introducing+o3%2Dand%2Do4%2Dmini%2E)
31. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. 2022.… — [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/)
32. Qwen. 2025. Qwq-32b: Embracing the power of reinforcement learning. [find](https://scholar.google.com/scholar?q=Qwen%2E+2025%2E+Qwq%2D32b%3A+Embracing+the+power+of+reinforcement+learning%2E)
33. Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, Jérémy Rapin, Artyom Kozhevnikov, Ivan Evtimov, Joanna Bitton, Manish Bhatt, Cristian Canton-Ferrer, Aaron Grattafiori, Wenhan Xiong, Alexandre Défossez, Jade Copet, and 6 o… — [Code Llama: Open Foundation Models for Code](https://md.research.elamur.ai/free/en/papers/roziere-2024-code-llama-open-foundation-models/)
34. ByteDance Seed. 2025. Seed-oss open-source models release. [find](https://scholar.google.com/scholar?q=ByteDance+Seed%2E+2025%2E+Seed%2Doss+open%2Dsource+models+release%2E)
35. ByteDance Seed, Yuyu Zhang, Jing Su, Yifan Sun, Chenguang Xi, Xia Xiao, Shen Zheng, Anxiang Zhang, Kaibo Liu, Daoguang Zan, and 1 others. 2025. [Seed-coder: Let the code model curate data for itself](https://md.research.elamur.ai/free/en/papers/bytedance-seed-2025-seed-coder-let-code/). arXiv preprint arXiv:2506.03524.
36. Sglang Team. Sglang project. Jiajun Shi, Jian Yang, Jiaheng Liu, Xingyuan Bu, Jiangjie Chen, Junting Zhou, Kaijing Ma, Zhoufutu Wen, Bingli Wang, Yancheng He, and 1 others. 2025. Korgym: A dynamic game platform for llm reasoning evaluation. arXiv preprint arXiv:2505.14552. — [arXiv:2505.14552](https://arxiv.org/abs/2505.14552) [find](https://scholar.google.com/scholar?q=Sglang+Team%2E+Sglang+project%2E+Jiajun+Shi%2C+Jian+Yang%2C+Jiaheng+Liu%2C+Xingyuan+Bu%2C+Jiangjie+Chen%2C+Junting+Zhou%2C+Kaijing+Ma%2C+Zhoufutu+Wen%2C+Bingli+Wang%2C+Yancheng+He%2C+and+1+others%2E+2025%2E+Korgym%3A+A+dynamic+gam)
37. Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, and 1 others. 2025. [Kimi k2: Open agentic intelligence](https://md.research.elamur.ai/free/en/papers/kimi-team-2025-kimi-k2-open-agentic/). arXiv preprint arXiv:2507.20534.
38. Jingqi Tong, Jixin Tang, Hangcheng Li, Yurong Mou, Ming Zhang, Jun Zhao, Yanbo Wen, Fan Song, Jiahao Zhan, Yuyang Lu, Chaoran Tao, and 1 others. 2025. Game-rl: Synthesizing verifiable game tasks at scale to boost vlms general reasoning. arXiv preprint arXiv:2505.13886. — [arXiv:2505.13886](https://arxiv.org/abs/2505.13886) [find](https://scholar.google.com/scholar?q=Jingqi+Tong%2C+Jixin+Tang%2C+Hangcheng+Li%2C+Yurong+Mou%2C+Ming+Zhang%2C+Jun+Zhao%2C+Yanbo+Wen%2C+Fan+Song%2C+Jiahao+Zhan%2C+Yuyang+Lu%2C+Chaoran+Tao%2C+and+1+others%2E+2025%2E+Game%2Drl%3A+Synthesizing+verifiable+game+tasks+at+sc)
39. Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, Wayne Xin Zhao, Zhewei Wei, and Ji-Rong Wen. 2023a. [A survey on large language model based autonomous agents](https://md.research.elamur.ai/free/en/papers/wang-2023-survey-large-language-model-based/). CoRR, abs/2308.11432.
40. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2023b. Self-instruct: Aligning language models with self-generated instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long… [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Yeganeh+Kordi%2C+Swaroop+Mishra%2C+Alisa+Liu%2C+Noah+A%2E+Smith%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+2023b%2E+Self%2Dinstruct%3A+Aligning+language+models+with+self%2Dgenerated+instructions%2E+In+Pro)
41. Yuxiang Wei, Zhe Wang, Jiawei Liu, Yifeng Ding, and Lingming Zhang. 2023. Magicoder: Source code is all you need. CoRR, abs/2312.02120. [find](https://scholar.google.com/scholar?q=Yuxiang+Wei%2C+Zhe+Wang%2C+Jiawei+Liu%2C+Yifeng+Ding%2C+and+Lingming+Zhang%2E+2023%2E+Magicoder%3A+Source+code+is+all+you+need%2E+CoRR%2C+abs%2F2312%2E02120%2E)
42. Shuang Wu, Liwen Zhu, Tao Yang, Shiwei Xu, Qiang Fu, Yang Wei, and Haobo Fu. 2024. Enhance reasoning for large language models in the game werewolf. arXiv preprint arXiv:2402.02330. — [arXiv:2402.02330](https://arxiv.org/abs/2402.02330) [find](https://scholar.google.com/scholar?q=Shuang+Wu%2C+Liwen+Zhu%2C+Tao+Yang%2C+Shiwei+Xu%2C+Qiang+Fu%2C+Yang+Wei%2C+and+Haobo+Fu%2E+2024%2E+Enhance+reasoning+for+large+language+models+in+the+game+werewolf%2E+arXiv+preprint+arXiv%3A2402%2E02330%2E+%E2%80%94+%5BarXiv%3A2402%2E0233)
43. Aohan Zeng, Xin Lv, Qinkai Zheng, Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang, Da Yin, Hao Zeng, Jiajie Zhang, and 1 others. 2025. [Glm-4.5: Agentic, reasoning, and coding (arc) foundation models](https://md.research.elamur.ai/free/en/papers/team-2025-4-glm-4-5-agentic/). arXiv preprint arXiv:2508.06471.
44. Renrui Zhang, Jiaming Han, Aojun Zhou, Xiangfei Hu, Shilin Yan, Pan Lu, Hongsheng Li, Peng Gao, and Yu Qiao. 2023. [Llama-adapter: Efficient fine-tuning of language models with zero-init attention](https://md.research.elamur.ai/free/en/papers/zhang-2023-llama-adapter-efficient-fine-tuning/). CoRR, abs/2303.16199.
45. Wei Zhang, Jian Yang, Jiaxi Yang, Ya Wang, Zhoujun Li, Zeyu Cui, Binyuan Hui, and Junyang Lin. 2025a. Turning the tide: Repository-based code reflection. Preprint, arXiv:2507.09866. [find](https://scholar.google.com/scholar?q=Wei+Zhang%2C+Jian+Yang%2C+Jiaxi+Yang%2C+Ya+Wang%2C+Zhoujun+Li%2C+Zeyu+Cui%2C+Binyuan+Hui%2C+and+Junyang+Lin%2E+2025a%2E+Turning+the+tide%3A+Repository%2Dbased+code+reflection%2E+Preprint%2C+arXiv%3A2507%2E09866%2E)
46. Wei Zhang, Yi Zhang, Li Zhu, Qianghuai Jia, Feijun Jiang, Hongcheng Guo, Zhoujun Li, and Mengping Zhou. 2025b. Adc: Enhancing function calling via adversarial datasets and code line-level feedback. In ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP… [find](https://scholar.google.com/scholar?q=Wei+Zhang%2C+Yi+Zhang%2C+Li+Zhu%2C+Qianghuai+Jia%2C+Feijun+Jiang%2C+Hongcheng+Guo%2C+Zhoujun+Li%2C+and+Mengping+Zhou%2E+2025b%2E+Adc%3A+Enhancing+function+calling+via+adversarial+datasets+and+code+line%2Dlevel+feedback%2E+In)
47. Terry Yue Zhuo, Minh Chien Vu, Jenny Chim, Han Hu, Wenhao Yu, Ratnadira Widyasari, Imam Nur Bani Yusuf, Haolan Zhan, Junda He, Indraneil Paul, and 1 others. 2024. [Bigcodebench: Benchmarking code generation with diverse function calls and complex instructions](https://md.research.elamur.ai/free/en/papers/zhuo-2024-bigcodebench-benchmarking-code-generation-diverse/). arXiv preprint arXiv:2406.15877. xAI. 20…
48. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, and 41 others. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). Preprint, arXiv:2505.09388.
49. Jian Yang, Jiaxi Yang, Ke Jin, Yibo Miao, Lei Zhang, Liqun Yang, Zeyu Cui, Yichang Zhang, Binyuan Hui, and Junyang Lin. 2024a. Evaluating and aligning codellms on human preference. arXiv preprint arXiv:2412.05210. — [arXiv:2412.05210](https://arxiv.org/abs/2412.05210) [find](https://scholar.google.com/scholar?q=Jian+Yang%2C+Jiaxi+Yang%2C+Ke+Jin%2C+Yibo+Miao%2C+Lei+Zhang%2C+Liqun+Yang%2C+Zeyu+Cui%2C+Yichang+Zhang%2C+Binyuan+Hui%2C+and+Junyang+Lin%2E+2024a%2E+Evaluating+and+aligning+codellms+on+human+preference%2E+arXiv+preprint+arXi)
50. Jian Yang, Jiajun Zhang, Jiaxi Yang, Ke Jin, Lei Zhang, Qiyao Peng, Ken Deng, Yibo Miao, Tianyu Liu, Zeyu Cui, and 1 others. 2024b. Execrepobench: Multi-level executable code completion evaluation. arXiv preprint arXiv:2412.11990. — [arXiv:2412.11990](https://arxiv.org/abs/2412.11990) [find](https://scholar.google.com/scholar?q=Jian+Yang%2C+Jiajun+Zhang%2C+Jiaxi+Yang%2C+Ke+Jin%2C+Lei+Zhang%2C+Qiyao+Peng%2C+Ken+Deng%2C+Yibo+Miao%2C+Tianyu+Liu%2C+Zeyu+Cui%2C+and+1+others%2E+2024b%2E+Execrepobench%3A+Multi%2Dlevel+executable+code+completion+evaluation%2E+ar)
51. Jianzhu Yao, Kevin Wang, Ryan Hsieh, Haisu Zhou, Tianqing Zou, Zerui Cheng, Zhangyang Wang, and Pramod Viswanath. 2025. Spin-bench: How well do llms plan strategically and reason socially? arXiv preprint arXiv:2503.12349. — [arXiv:2503.12349](https://arxiv.org/abs/2503.12349) [find](https://scholar.google.com/scholar?q=Jianzhu+Yao%2C+Kevin+Wang%2C+Ryan+Hsieh%2C+Haisu+Zhou%2C+Tianqing+Zou%2C+Zerui+Cheng%2C+Zhangyang+Wang%2C+and+Pramod+Viswanath%2E+2025%2E+Spin%2Dbench%3A+How+well+do+llms+plan+strategically+and+reason+socially%3F+arXiv+prepr)
52. Zhaojian Yu, Xin Zhang, Ning Shang, Yangyu Huang, Can Xu, Yishujie Zhao, Wenxiang Hu, and Qiufeng Yin. 2023. Wavecoder: Widespread and versatile enhanced instruction tuning with refined data generation. CoRR, abs/2312.14187. — [arXiv:2312.14187](https://arxiv.org/abs/2312.14187) [find](https://scholar.google.com/scholar?q=Zhaojian+Yu%2C+Xin+Zhang%2C+Ning+Shang%2C+Yangyu+Huang%2C+Can+Xu%2C+Yishujie+Zhao%2C+Wenxiang+Hu%2C+and+Qiufeng+Yin%2E+2023%2E+Wavecoder%3A+Widespread+and+versatile+enhanced+instruction+tuning+with+refined+data+generatio)

*Updated 31 October 2026*