---
title: "Generative Gamer — paper by Zhang et al. (2026)"
description: "LLMs play strategic games poorly because finding equilibrium moves usually means exploring a game tree that grows… 28 references."
authors: ["Yadong Zhang", "Xinshu Shen", "Yupei Ren", "Shangqing Zhao", "Man Lan"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.574/"
doi: "10.18653/v1/2026.acl-long.574"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2026-generative-gamer-learning-equilibrium-strategy/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2026-generative-gamer-learning-equilibrium-strategy/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/35/) / Generative Gamer: Learning Equilibrium Strategy by LLM-driven Dynamic Deduction

# Generative Gamer: Learning Equilibrium Strategy by LLM-driven Dynamic Deduction

**Authors:** [Yadong Zhang](https://md.research.elamur.ai/free/en/authors/yadong-zhang/), [Xinshu Shen](https://md.research.elamur.ai/free/en/authors/xinshu-shen/), Yupei Ren, Shangqing Zhao and [Man Lan](https://md.research.elamur.ai/free/en/authors/man-lan/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.574/) · [PDF](https://aclanthology.org/2026.acl-long.574.pdf) · [DOI 10.18653/v1/2026.acl-long.574](https://doi.org/10.18653/v1/2026.acl-long.574) · [OpenAlex](https://openalex.org/W7166878695)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
LLMs play strategic games poorly because finding equilibrium moves usually means exploring a game tree that grows explosively with depth. Running classic search such as MCTS through slow autoregressive decoding is impractical. GenGamer, from East China Normal University, instead trains a model to write out a compact textual search tree, called a Dynamic Deduction Tree, in a single generation pass. The model then picks its move from that tree using minimax backup. Three pruning mechanisms keep the tree small:
- keeping only high-probability actions;
- halting expansion of states that are too deep, clearly decided, or stable in value;
- applying alpha-beta cutoffs.

Training has two stages. First, the model imitates trees produced by an expert search such as MCTS. Second, reinforcement learning uses the Deduction Tree Reward (DTR), which scores the action choice and value estimate at every node rather than only the final game result. On Tic-Tac-Toe, Connect Four, Nim and Leduc Poker, the authors report stronger play than state-of-the-art LLMs and outcome-supervised fine-tuning, along with steadier training. The work shows that search procedures can be distilled into a model's own reasoning text.

## Key points
- **Pipeline:** Training runs in two stages. "Expert Deduction Imitation" is SFT on serialized trees built by an oracle (MCTS or an AlphaGo-style solver). "Deduction Process Optimization" then applies the process-level DTR, which includes value and chance components for stochastic nodes in poker.
- **Pruning parameters:** Action pruning samples actions until their cumulative probability passes a top_p threshold. State pruning relies on max_depth, certainty_threshold and stability_threshold. Alpha-beta propagation discards branches that are provably worse.
- **Training behavior:** Compared with a Deduction Imitation + GRPO baseline that uses outcome rewards, GenGamer trains more stably and reaches higher peak accuracy. The baseline's value-estimation accuracy collapses late in training on Leduc Hold'em. Shallow variants (depth=1) converge fast but plateau lower.
- **Limitations:** Each move costs hundreds to thousands of generated tokens. Context-window limits make scaling to Go or Chess difficult. The method depends on an expert oracle to produce supervision data.
- **Availability:** Code and data are published at github.com/cubenlp/gengamer (ACL 2026, Long Papers, pp. 12604–12617).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [East China Normal University](https://md.research.elamur.ai/free/en/universities/east-china-normal-university/) — University, Shanghai, China · 69 works · 2,490 citations

📚 Shelves (2): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works

## Cite

Cite as: Yadong Zhang, Xinshu Shen, Yupei Ren, Shangqing Zhao and Man Lan. 2026. Generative Gamer: Learning Equilibrium Strategy by LLM-driven Dynamic Deduction. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhang2026generative,
  title = {{Generative Gamer: Learning Equilibrium Strategy by LLM-driven Dynamic Deduction}},
  author = {Yadong Zhang, Xinshu Shen, Yupei Ren, Shangqing Zhao and Man Lan},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.574},
  url = {https://doi.org/10.18653/v1/2026.acl-long.574}
}
```

- [← previous Generation of Instruction and Preference Dataset for Improving Japanese Instruction Following in LLMs](https://md.research.elamur.ai/free/en/papers/moriyama-2026-generation-instruction-preference-dataset-improving/)
- [next → Generative Information Extraction from Biographical Sources](https://md.research.elamur.ai/free/en/papers/winkle-2026-generative-information-extraction-biographical-sources/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/35/)

## References

28 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774.
2. Noam Brown, Anton Bakhtin, Adam Lerer, and Qucheng Gong. 2020. Combining deep reinforcement learning and search for imperfect-information games. Advances in neural information processing systems, 33:17057–17069. [find](https://scholar.google.com/scholar?q=Noam+Brown%2C+Anton+Bakhtin%2C+Adam+Lerer%2C+and+Qucheng+Gong%2E+2020%2E+Combining+deep+reinforcement+learning+and+search+for+imperfect%2Dinformation+games%2E+Advances+in+neural+information+processing+systems%2C+33%3A1)
3. Cameron B Browne, Edward Powley, Daniel Whitehouse, Simon M Lucas, Peter I Cowling, Philipp Rohlfshagen, Stephen Tavener, Diego Perez, Spyridon Samothrakis, and Simon Colton. 2012. A survey of monte carlo tree search methods. IEEE Transactions on Computational Intelligence and AI in games, 4(1):1–43… [find](https://scholar.google.com/scholar?q=Cameron+B+Browne%2C+Edward+Powley%2C+Daniel+Whitehouse%2C+Simon+M+Lucas%2C+Peter+I+Cowling%2C+Philipp+Rohlfshagen%2C+Stephen+Tavener%2C+Diego+Perez%2C+Spyridon+Samothrakis%2C+and+Simon+Colton%2E+2012%2E+A+survey+of+monte+c)
4. Jinhao Duan, Renming Zhang, James Diffenderfer, Bhavya Kailkhura, Lichao Sun, Elias Stengel-Eskin, Mohit Bansal, Tianlong Chen, and Kaidi Xu. 2024. Gtbench: Uncovering the strategic reasoning limitations of llms via game-theoretic evaluations. arXiv preprint arXiv:2402.12348. [find](https://scholar.google.com/scholar?q=Jinhao+Duan%2C+Renming+Zhang%2C+James+Diffenderfer%2C+Bhavya+Kailkhura%2C+Lichao+Sun%2C+Elias+Stengel%2DEskin%2C+Mohit+Bansal%2C+Tianlong+Chen%2C+and+Kaidi+Xu%2E+2024%2E+Gtbench%3A+Uncovering+the+strategic+reasoning+limitati)
5. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv e-prints, pages arXiv–2407.
6. Meta Fundamental AI Research Diplomacy Team (FAIR)†, Anton Bakhtin, Noam Brown, Emily Dinan, Gabriele Farina, Colin Flaherty, Daniel Fried, Andrew Goff, Jonathan Gray, Hengyuan Hu, and 1 others. 2022. Human-level play in the game of diplomacy by combining language models with strategic reasoning. Sc… [find](https://scholar.google.com/scholar?q=Meta+Fundamental+AI+Research+Diplomacy+Team+%28FAIR%29%E2%80%A0%2C+Anton+Bakhtin%2C+Noam+Brown%2C+Emily+Dinan%2C+Gabriele+Farina%2C+Colin+Flaherty%2C+Daniel+Fried%2C+Andrew+Goff%2C+Jonathan+Gray%2C+Hengyuan+Hu%2C+and+1+others%2E+2022%2E)
7. Johannes Heinrich and David Silver. 2016. Deep reinforcement learning from self-play in imperfect-information games. arXiv preprint arXiv:1603.01121. — [arXiv:1603.01121](https://arxiv.org/abs/1603.01121) [find](https://scholar.google.com/scholar?q=Johannes+Heinrich+and+David+Silver%2E+2016%2E+Deep+reinforcement+learning+from+self%2Dplay+in+imperfect%2Dinformation+games%2E+arXiv+preprint+arXiv%3A1603%2E01121%2E+%E2%80%94+%5BarXiv%3A1603%2E01121%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1603%2E01)
8. Dom Huh and Prasant Mohapatra. 2023. Multi-agent reinforcement learning: A comprehensive survey. arXiv preprint arXiv:2312.10256. — [arXiv:2312.10256](https://arxiv.org/abs/2312.10256) [find](https://scholar.google.com/scholar?q=Dom+Huh+and+Prasant+Mohapatra%2E+2023%2E+Multi%2Dagent+reinforcement+learning%3A+A+comprehensive+survey%2E+arXiv+preprint+arXiv%3A2312%2E10256%2E+%E2%80%94+%5BarXiv%3A2312%2E10256%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2312%2E10256%29)
9. Jingru Jia, Zehua Yuan, Junhao Pan, Paul E McNamara, and Deming Chen. 2025. Large language model strategic reasoning evaluation through behavioral game theory. arXiv preprint arXiv:2502.20432. — [arXiv:2502.20432](https://arxiv.org/abs/2502.20432) [find](https://scholar.google.com/scholar?q=Jingru+Jia%2C+Zehua+Yuan%2C+Junhao+Pan%2C+Paul+E+McNamara%2C+and+Deming+Chen%2E+2025%2E+Large+language+model+strategic+reasoning+evaluation+through+behavioral+game+theory%2E+arXiv+preprint+arXiv%3A2502%2E20432%2E+%E2%80%94+%5BarXi)
10. Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. 2023. [Swe-bench: Can language models resolve real-world github issues?](https://md.research.elamur.ai/free/en/papers/jimenez-2024-swe-bench-can-language-models/) arXiv preprint arXiv:2310.06770.
11. Donald E Knuth and Ronald W Moore. 1975. An analysis of alpha-beta pruning. Artificial intelligence, 6(4):293–326. [find](https://scholar.google.com/scholar?q=Donald+E+Knuth+and+Ronald+W+Moore%2E+1975%2E+An+analysis+of+alpha%2Dbeta+pruning%2E+Artificial+intelligence%2C+6%284%29%3A293%E2%80%93326%2E)
12. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. [Large language models are zero-shot reasoners](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/). Advances in neural information processing systems, 35:22199– 22213.
13. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. [Efficient memory management for large language model serving with pagedattention](https://md.research.elamur.ai/free/en/papers/kwon-2023-efficient-memory-management-large-language/). In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles.
14. Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. [Let’s verify step by step](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/). In The Twelfth International Conference on Learning Representations.
15. Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Chen Bo Calvin Zhang, Mohamed Shaaban, John Ling, Sean Shi, and 1 others. 2025. [Humanity’s last exam](https://md.research.elamur.ai/free/en/papers/phan-2025-humanity-s-last-exam/). arXiv preprint arXiv:2501.14249.
16. Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, and 25 others. 2025. Qwen2.5 technical report. Preprint, arXiv:2412.15115. [find](https://scholar.google.com/scholar?q=Qwen%2C+%3A%2C+An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Haoran+Wei%2C+Huan+Lin%2C+Jian+Yang%2C+Jianhong+Tu%2C+Jianwei+Zhang%2C+Jianxin+Yang%2C+Jiaxi+)
17. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Filip+Wolski%2C+Prafulla+Dhariwal%2C+Alec+Radford%2C+and+Oleg+Klimov%2E+2017%2E+Proximal+policy+optimization+algorithms%2E+arXiv+preprint+arXiv%3A1707%2E06347%2E)
18. David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, and 1 others. 2016. Mastering the game of go with deep neural networks and tree search. nature, 529(7587):484–489. [find](https://scholar.google.com/scholar?q=David+Silver%2C+Aja+Huang%2C+Chris+J+Maddison%2C+Arthur+Guez%2C+Laurent+Sifre%2C+George+Van+Den+Driessche%2C+Julian+Schrittwieser%2C+Ioannis+Antonoglou%2C+Veda+Panneershelvam%2C+Marc+Lanctot%2C+and+1+others%2E+2016%2E+Master)
19. David Silver, Julian Schrittwieser, Karen Simonyan, Ioannis Antonoglou, Aja Huang, Arthur Guez, Thomas Hubert, Lucas Baker, Matthew Lai, Adrian Bolton, and 1 others. 2017. Mastering the game of go without human knowledge. nature, 550(7676):354– 359. [find](https://scholar.google.com/scholar?q=David+Silver%2C+Julian+Schrittwieser%2C+Karen+Simonyan%2C+Ioannis+Antonoglou%2C+Aja+Huang%2C+Arthur+Guez%2C+Thomas+Hubert%2C+Lucas+Baker%2C+Matthew+Lai%2C+Adrian+Bolton%2C+and+1+others%2E+2017%2E+Mastering+the+game+of+go+wit)
20. Finnegan Southey, Michael P Bowling, Bryce Larson, Carmelo Piccione, Neil Burch, Darse Billings, and Chris Rayner. 2012. Bayes’ bluff: Opponent modelling in poker. arXiv preprint arXiv:1207.1411. — [arXiv:1207.1411](https://arxiv.org/abs/1207.1411) [find](https://scholar.google.com/scholar?q=Finnegan+Southey%2C+Michael+P+Bowling%2C+Bryce+Larson%2C+Carmelo+Piccione%2C+Neil+Burch%2C+Darse+Billings%2C+and+Chris+Rayner%2E+2012%2E+Bayes%E2%80%99+bluff%3A+Opponent+modelling+in+poker%2E+arXiv+preprint+arXiv%3A1207%2E1411%2E+%E2%80%94+%5Ba)
21. Vivienne Sze, Yu-Hsin Chen, Tien-Ju Yang, and Joel S Emer. 2017. [Efficient processing of deep neural networks: A tutorial and survey](https://md.research.elamur.ai/free/en/papers/sze-2017-efficient-processing-deep-neural-networks/). Proceedings of the IEEE, 105(12):2295–2329.
22. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). Advances in neural information processing systems, 35:24824– 24837.
23. Jian Xie, Kai Zhang, Jiangjie Chen, Tinghui Zhu, Renze Lou, Yuandong Tian, Yanghua Xiao, and Yu Su. 2024. [Travelplanner: A benchmark for real-world planning with language agents](https://md.research.elamur.ai/free/en/papers/xie-2024-travelplanner-benchmark-real-world-planning/). arXiv preprint arXiv:2402.01622.
24. Kaixuan Xu, Jiajun Chai, Sicheng Li, Yuqian Fu, Yuanheng Zhu, and Dongbin Zhao. 2025. Dipllm: Fine-tuning llm for strategic decision-making in diplomacy. arXiv preprint arXiv:2506.09655. — [arXiv:2506.09655](https://arxiv.org/abs/2506.09655) [find](https://scholar.google.com/scholar?q=Kaixuan+Xu%2C+Jiajun+Chai%2C+Sicheng+Li%2C+Yuqian+Fu%2C+Yuanheng+Zhu%2C+and+Dongbin+Zhao%2E+2025%2E+Dipllm%3A+Fine%2Dtuning+llm+for+strategic+decision%2Dmaking+in+diplomacy%2E+arXiv+preprint+arXiv%3A2506%2E09655%2E+%E2%80%94+%5BarXiv%3A2506)
25. Yadong Zhang, Shaoguang Mao, Tao Ge, Xun Wang, Adrian de Wynter, Yan Xia, Wenshan Wu, Ting Song, Man Lan, and Furu Wei. 2024a. Llm as a mastermind: A survey of strategic reasoning with large language models. arXiv preprint arXiv:2404.01230. [find](https://scholar.google.com/scholar?q=Yadong+Zhang%2C+Shaoguang+Mao%2C+Tao+Ge%2C+Xun+Wang%2C+Adrian+de+Wynter%2C+Yan+Xia%2C+Wenshan+Wu%2C+Ting+Song%2C+Man+Lan%2C+and+Furu+Wei%2E+2024a%2E+Llm+as+a+mastermind%3A+A+survey+of+strategic+reasoning+with+large+language+)
26. Yadong Zhang, Shaoguang Mao, Tao Ge, Xun Wang, Yan Xia, Man Lan, and Furu Wei. 2024b. K-level reasoning with large language models. arXiv preprint arXiv:2402.01521. — [arXiv:2402.01521](https://arxiv.org/abs/2402.01521) [find](https://scholar.google.com/scholar?q=Yadong+Zhang%2C+Shaoguang+Mao%2C+Tao+Ge%2C+Xun+Wang%2C+Yan+Xia%2C+Man+Lan%2C+and+Furu+Wei%2E+2024b%2E+K%2Dlevel+reasoning+with+large+language+models%2E+arXiv+preprint+arXiv%3A2402%2E01521%2E+%E2%80%94+%5BarXiv%3A2402%2E01521%5D%28https%3A%2F%2Farxiv%2E)
27. Claude E Shannon. 1950. Xxii. programming a computer for playing chess. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 41(314):256–275. [find](https://scholar.google.com/scholar?q=Claude+E+Shannon%2E+1950%2E+Xxii%2E+programming+a+computer+for+playing+chess%2E+The+London%2C+Edinburgh%2C+and+Dublin+Philosophical+Magazine+and+Journal+of+Science%2C+41%28314%29%3A256%E2%80%93275%2E)
28. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and 1 others. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). arXiv preprint arXiv:2402.03300.

*Updated 2 November 2026*