---
title: "Planning with Large Language Models for Code Generation (2023)"
description: "The authors propose Planning-Guided Transformer Decoding (PG-TD). 49 references, cited by 75 papers here."
authors: ["Shun Zhang", "Zhenfang Chen", "Yikang Shen", "Mingyu Ding", "Joshua B. Tenenbaum", "Chuang Gan"]
year: 2023
venue: "arXiv:2303.05510"
source: "https://arxiv.org/abs/2303.05510"
doi: "10.48550/arxiv.2303.05510"
arxiv: "2303.05510"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "https://arxiv.org/licenses/nonexclusive-distrib/1.0/license.html"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2023-planning-large-language-models-code/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2023-planning-large-language-models-code/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/8/) / Planning with Large Language Models for Code Generation

# Planning with Large Language Models for Code Generation

**Authors:** Shun Zhang, [Zhenfang Chen](https://md.research.elamur.ai/free/en/authors/zhenfang-chen/), [Yikang Shen](https://md.research.elamur.ai/free/en/authors/yikang-shen/), Mingyu Ding, [Joshua B. Tenenbaum](https://md.research.elamur.ai/free/en/authors/joshua-b-tenenbaum/) and [Chuang Gan](https://md.research.elamur.ai/free/en/authors/chuang-gan/) · **Year:** 2023 · **Venue:** arXiv:2303.05510 · **Licence:** [arXiv — non-exclusive licence to distribute](https://arxiv.org/licenses/nonexclusive-distrib/1.0/license.html)

**Links:** [Publisher's page](https://arxiv.org/abs/2303.05510) · [PDF](https://arxiv.org/pdf/2303.05510) · [DOI 10.48550/arxiv.2303.05510](https://doi.org/10.48550/arxiv.2303.05510) · [arXiv 2303.05510](https://arxiv.org/abs/2303.05510) · [OpenAlex](https://openalex.org/W4323927478)

**Citations:** 15 citations (OpenAlex, 2026-10-14) · [75](https://md.research.elamur.ai/free/en/papers/zhang-2023-planning-large-language-models-code/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Code generators built on large language models usually decode with beam search or sampling. Their outputs may resemble reference code at the token level yet still fail to compile or give wrong answers. The common sampling + filtering approach, used by systems like AlphaCode, checks candidates against test cases only after full programs are produced. This wastes many samples. The authors propose Planning-Guided Transformer Decoding (PG-TD). It is a tree-search planner, based on Monte-Carlo tree search, that runs inside the decoding loop. The planner completes partial programs, executes them on public test cases, and uses the pass rate as a reward to steer which token comes next. The Transformer's next-token probabilities and beam search act as heuristics within the search. The authors also add mechanisms that let the planner and the model share information, which avoids redundant computation. PG-TD needs no finetuning and works with any backbone model. On competitive programming benchmarks it beats beam search and sampling + filtering, which shows that test-aware search can strengthen frozen code models.

## Key points
- PG-TD treats code generation as sequential decision-making. Test cases are split into public ones, which the planner uses for reward, and private ones, which are held out to measure generalization.
- The method is model-agnostic. It was demonstrated with GPT-2 (1.5B), GPT-Neo (2.7B) and Codex (code-davinci-002) as backbones, without retraining any of them.
- Changing the planner's reward enables controllable generation, such as shorter programs or more heavily commented code.
- Programs found by PG-TD can also serve as extra training data to finetune a code-generation Transformer.
- In one illustrative example, PG-TD reached a pass rate of 1.00, compared with 0.125 for beam search and 0.625 for sampling + filtering. The project page is https://codeaimcts.github.io.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Massachusetts Institute of Technology](https://md.research.elamur.ai/free/en/universities/massachusetts-institute-of-technology/) — University, Cambridge, United States · 243 works · 76,506 citations
- [IBM](https://md.research.elamur.ai/free/en/companies/ibm/) — Company, Armonk, United States · 154 works · 72,606 citations
- [MIT Computer Science and Artificial Intelligence Laboratory](https://md.research.elamur.ai/free/en/labs/mit-computer-science-and-artificial-intelligence-laboratory/) — Research institute, Cambridge, United States · 76 works · 32,207 citations
- [University of Hong Kong](https://md.research.elamur.ai/free/en/universities/university-of-hong-kong/) — University, Pok Fu Lam, China · 118 works · 13,496 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works · [🖥️ Computer Science Applications](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-science-applications/) · 131 works

🧭 Topics (1): [💻 Code & programming languages](https://md.research.elamur.ai/free/en/topics/code-and-programming/) · 168 works

## Cite

Cite as: Shun Zhang, Zhenfang Chen, Yikang Shen, Mingyu Ding, Joshua B. Tenenbaum and Chuang Gan. 2023. Planning with Large Language Models for Code Generation. arXiv:2303.05510.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{zhang2023planning,
  title = {{Planning with Large Language Models for Code Generation}},
  author = {Shun Zhang, Zhenfang Chen, Yikang Shen, Mingyu Ding, Joshua B. Tenenbaum and Chuang Gan},
  year = {2023},
  doi = {10.48550/arxiv.2303.05510},
  eprint = {2303.05510},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2303.05510}
}
```

- [← previous PLACES: Prompting Language Models for Social Conversation Synthesis](https://md.research.elamur.ai/free/en/papers/chen-2023-places-prompting-language-models-social/)
- [next → Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards](https://md.research.elamur.ai/free/en/papers/rame-2023-rewarded-soups-towards-pareto-optimal/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/8/)

## References

49 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Wasi Ahmad, Saikat Chakraborty, Baishakhi Ray, and Kai-Wei Chang. [Unified pre-training for program understanding and generation](https://md.research.elamur.ai/free/en/papers/ahmad-2021-unified-pre-training-program-understanding/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2655–2668, Online, J…
2. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Charles Sutton. [Program Synthesis with Large Language Models](https://md.research.elamur.ai/free/en/papers/austin-2021-program-synthesis-large-language-models/). arXiv:2108.07732 [cs], August 2021. URL http://arxiv.org/abs/2108.07732. arXiv: 2108.07732. 2 [↗](http://arxiv.org/abs/2108.07732)
3. Rudy Bunel, Matthew Hausknecht, Jacob Devlin, Rishabh Singh, and Pushmeet Kohli. Leveraging Grammar and Reinforcement Learning for Neural Program Synthesis. ICLR, May 2018. URL http://arxiv.org/abs/1805.04276. arXiv: 1805.04276. 3, 4 [↗](http://arxiv.org/abs/1805.04276)
4. Antoine Chaffin, Vincent Claveau, and Ewa Kijak. Ppl-mcts: Constrained textual generation through discriminator-guided mcts decoding. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2953–2967, 2022… [find](https://scholar.google.com/scholar?q=Antoine+Chaffin%2C+Vincent+Claveau%2C+and+Ewa+Kijak%2E+Ppl%2Dmcts%3A+Constrained+textual+generation+through+discriminator%2Dguided+mcts+decoding%2E+In+Proceedings+of+the+2022+Conference+of+the+North+American+Chapte)
5. Guillaume MJ-B Chaslot, Mark HM Winands, and HJVD Herik. Parallel monte-carlo tree search. In International Conference on Computers and Games, pp. 60–71. Springer, 2008. 9 [find](https://scholar.google.com/scholar?q=Guillaume+MJ%2DB+Chaslot%2C+Mark+HM+Winands%2C+and+HJVD+Herik%2E+Parallel+monte%2Dcarlo+tree+search%2E+In+International+Conference+on+Computers+and+Games%2C+pp%2E+60%E2%80%9371%2E+Springer%2C+2008%2E+9)
6. Bei Chen, Fengji Zhang, Anh Nguyen, Daoguang Zan, Zeqi Lin, Jian-Guang Lou, and Weizhu Chen. CodeT: Code Generation with Generated Tests, July 2022. URL http://arxiv.org/abs/2207.10397. arXiv:2207.10397 [cs]. 9, 17 [↗](http://arxiv.org/abs/2207.10397)
7. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhai… — [Evaluating Large Language Models Trained on Code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
8. Xinyun Chen, Chang Liu, and Dawn Song. Execution-guided neural program synthesis. In International Conference on Learning Representations, 2018. 3, 9 [find](https://scholar.google.com/scholar?q=Xinyun+Chen%2C+Chang+Liu%2C+and+Dawn+Song%2E+Execution%2Dguided+neural+program+synthesis%2E+In+International+Conference+on+Learning+Representations%2C+2018%2E+3%2C+9)
9. Xinyun Chen, Dawn Song, and Yuandong Tian. Latent execution for neural program synthesis beyond domain-specific languages. Advances in Neural Information Processing Systems, 2021b. 3 [find](https://scholar.google.com/scholar?q=Xinyun+Chen%2C+Dawn+Song%2C+and+Yuandong+Tian%2E+Latent+execution+for+neural+program+synthesis+beyond+domain%2Dspecific+languages%2E+Advances+in+Neural+Information+Processing+Systems%2C+2021b%2E+3)
10. Raj Dabre and Atsushi Fujita. Softmax tempering for training neural machine translation models. arXiv, 2020. 1, 4 [find](https://scholar.google.com/scholar?q=Raj+Dabre+and+Atsushi+Fujita%2E+Softmax+tempering+for+training+neural+machine+translation+models%2E+arXiv%2C+2020%2E+1%2C+4)
11. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volum…
12. Kevin Ellis, Catherine Wong, Maxwell Nye, Mathias Sablé-Meyer, Lucas Morales, Luke Hewitt, Luc Cary, Armando Solar-Lezama, and Joshua B Tenenbaum. Dreamcoder: Bootstrapping inductive program synthesis with wake-sleep library learning. In Proceedings of the 42nd ACM SIGPLAN International Conference… [find](https://scholar.google.com/scholar?q=Kevin+Ellis%2C+Catherine+Wong%2C+Maxwell+Nye%2C+Mathias+Sable%CC%81%2DMeyer%2C+Lucas+Morales%2C+Luke+Hewitt%2C+Luc+Cary%2C+Armando+Solar%2DLezama%2C+and+Joshua+B+Tenenbaum%2E+Dreamcoder%3A+Bootstrapping+inductive+program+synthesi)
13. Kevin M. Ellis, Maxwell Nye, Yewen Pu, Felix Sosa, Joshua Tenenbaum, and Armando Solar-Lezama. Write, execute, assess: Program synthesis with a repl. Neural Information Processing Systems, 2019. 3, 4, 7 [find](https://scholar.google.com/scholar?q=Kevin+M%2E+Ellis%2C+Maxwell+Nye%2C+Yewen+Pu%2C+Felix+Sosa%2C+Joshua+Tenenbaum%2C+and+Armando+Solar%2DLezama%2E+Write%2C+execute%2C+assess%3A+Program+synthesis+with+a+repl%2E+Neural+Information+Processing+Systems%2C+2019%2E+3%2C+4%2C)
14. Angela Fan, Mike Lewis, and Yann Dauphin. [Hierarchical neural story generation](https://md.research.elamur.ai/free/en/papers/fan-2018-hierarchical-neural-story-generation/). In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 889–898, 2018. 1, 3, 4
15. Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, et al. [Codebert: A pre-trained model for programming and natural languages](https://md.research.elamur.ai/free/en/papers/feng-2020-codebert-pre-trained-model-programming/). In Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 1536–1547, 2020. 2
16. Sylvain Gelly and David Silver. Monte-carlo tree search and rapid action value estimation in computer go. Artificial Intelligence, 175(11):1856–1875, 2011. 3 [find](https://scholar.google.com/scholar?q=Sylvain+Gelly+and+David+Silver%2E+Monte%2Dcarlo+tree+search+and+rapid+action+value+estimation+in+computer+go%2E+Artificial+Intelligence%2C+175%2811%29%3A1856%E2%80%931875%2C+2011%2E+3)
17. Alex Graves. [Sequence transduction with recurrent neural networks](https://md.research.elamur.ai/free/en/papers/graves-2012-sequence-transduction-recurrent-neural-networks/). In International Conference on Machine Learning: Representation Learning Workshop, 2012. 3, 4
18. Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, LIU Shujie, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, et al. [Graphcodebert: Pre-training code representations with data flow](https://md.research.elamur.ai/free/en/papers/guo-2021-graphcodebert-pre-training-code-representations/). In International Conference on Learning Representations, 2020. 2
19. Kavi Gupta, Peter Ebert Christensen, Xinyun Chen, and Dawn Song. Synthesize, execute and debug: Learning to repair for neural program synthesis. Advances in Neural Information Processing Systems, 2020. 3 [find](https://scholar.google.com/scholar?q=Kavi+Gupta%2C+Peter+Ebert+Christensen%2C+Xinyun+Chen%2C+and+Dawn+Song%2E+Synthesize%2C+execute+and+debug%3A+Learning+to+repair+for+neural+program+synthesis%2E+Advances+in+Neural+Information+Processing+Systems%2C+2020)
20. Peter E Hart, Nils J Nilsson, and Bertram Raphael. A formal basis for the heuristic determination of minimum cost paths. IEEE transactions on Systems Science and Cybernetics, 4(2):100–107, 1968. 3 [find](https://scholar.google.com/scholar?q=Peter+E+Hart%2C+Nils+J+Nilsson%2C+and+Bertram+Raphael%2E+A+formal+basis+for+the+heuristic+determination+of+minimum+cost+paths%2E+IEEE+transactions+on+Systems+Science+and+Cybernetics%2C+4%282%29%3A100%E2%80%93107%2C+1968%2E+3)
21. Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, and Jacob Steinhardt. [Measuring coding challenge competence with APPS](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-coding-challenge-competence-apps/). NeurIPS, 2021. 1, 4, 7, 8, 14, 17
22. Aditya Kanade, Petros Maniatis, Gogul Balakrishnan, and Kensen Shi. Learning and evaluating contextual embedding of source code. In International Conference on Machine Learning, pp. 5110–5121. PMLR, 2020. 2 [find](https://scholar.google.com/scholar?q=Aditya+Kanade%2C+Petros+Maniatis%2C+Gogul+Balakrishnan%2C+and+Kensen+Shi%2E+Learning+and+evaluating+contextual+embedding+of+source+code%2E+In+International+Conference+on+Machine+Learning%2C+pp%2E+5110%E2%80%935121%2E+PMLR%2C+2)
23. Levente Kocsis and Csaba Szepesvári. Bandit based monte-carlo planning. In European conference on machine learning, pp. 282–293. Springer, 2006. 21 [find](https://scholar.google.com/scholar?q=Levente+Kocsis+and+Csaba+Szepesva%CC%81ri%2E+Bandit+based+monte%2Dcarlo+planning%2E+In+European+conference+on+machine+learning%2C+pp%2E+282%E2%80%93293%2E+Springer%2C+2006%2E+21)
24. Hung Le, Yue Wang, Akhilesh Deepak Gotmare, Silvio Savarese, and Steven C. H. Hoi. [CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/le-2022-coderl-mastering-code-generation-through/). arXiv, July 2022. doi: 10.48550/arXiv.2207.01780. URL http://arxiv.org/abs/2207.01780. arXiv:2207.01780 [cs]. 3, 24 [↗](http://arxiv.org/abs/2207.01780)
25. Rémi Leblond, Jean-Baptiste Alayrac, Laurent Sifre, Miruna Pislar, Jean-Baptiste Lespiau, Ioannis Antonoglou, Karen Simonyan, and Oriol Vinyals. Machine translation decoding beyond beam search. arXiv preprint arXiv:2104.05336, 2021. 3 [find](https://scholar.google.com/scholar?q=Re%CC%81mi+Leblond%2C+Jean%2DBaptiste+Alayrac%2C+Laurent+Sifre%2C+Miruna+Pislar%2C+Jean%2DBaptiste+Lespiau%2C+Ioannis+Antonoglou%2C+Karen+Simonyan%2C+and+Oriol+Vinyals%2E+Machine+translation+decoding+beyond+beam+search%2E+arXiv)
26. Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, and Agustin Dal Lago. [Competition-Level Code Generation with AlphaCode](https://md.research.elamur.ai/free/en/papers/li-2022-competition-level-code-generation-alphacode/). arXiv preprint arXiv:2203.07814, 2022. 1, 2, 3, 4, 7, 8, 14, 17
27. Jinsuk Lim and Shin Yoo. Field report: Applying monte carlo tree search for program synthesis. In International Symposium on Search Based Software Engineering, pp. 304–310. Springer, 2016. 3 [find](https://scholar.google.com/scholar?q=Jinsuk+Lim+and+Shin+Yoo%2E+Field+report%3A+Applying+monte+carlo+tree+search+for+program+synthesis%2E+In+International+Symposium+on+Search+Based+Software+Engineering%2C+pp%2E+304%E2%80%93310%2E+Springer%2C+2016%2E+3)
28. Ximing Lu, Sean Welleck, Peter West, Liwei Jiang, Jungo Kasai, Daniel Khashabi, Ronan Le Bras, Lianhui Qin, Youngjae Yu, Rowan Zellers, Noah A. Smith, and Yejin Choi. [NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristics](https://md.research.elamur.ai/free/en/papers/lu-2022-neurologic-esque-decoding-constrained-text/), December 2021. URL http://arxiv.org/abs/2112.0872… [↗](http://arxiv.org/abs/2112.0872…)
29. Nadia Matulewicz. Inductive program synthesis through using monte carlo tree search guided by a heuristic-based loss function. 2022. 3 [find](https://scholar.google.com/scholar?q=Nadia+Matulewicz%2E+Inductive+program+synthesis+through+using+monte+carlo+tree+search+guided+by+a+heuristic%2Dbased+loss+function%2E+2022%2E+3)
30. Glenford J Myers. The art of software testing. 1979. 3 [find](https://scholar.google.com/scholar?q=Glenford+J+Myers%2E+The+art+of+software+testing%2E+1979%2E+3)
31. Maxwell Nye, Armando Solar-Lezama, Josh Tenenbaum, and Brenden M Lake. Learning compositional rules via neural program synthesis. Advances in Neural Information Processing Systems, 33:10832–10842, 2020. 2 [find](https://scholar.google.com/scholar?q=Maxwell+Nye%2C+Armando+Solar%2DLezama%2C+Josh+Tenenbaum%2C+and+Brenden+M+Lake%2E+Learning+compositional+rules+via+neural+program+synthesis%2E+Advances+in+Neural+Information+Processing+Systems%2C+33%3A10832%E2%80%9310842%2C+202)
32. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. 2019. 2, 28 [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeff+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+and+Ilya+Sutskever%2E+Language+models+are+unsupervised+multitask+learners%2E+2019%2E+2%2C+28)
33. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). Journal of Machine Learning Research, 2020. URL [http://jmlr.org/papers/v21/20-074.html](http://jmlr.org/papers/v21/20-074.html). 2 [↗](http://jmlr.org/papers/v21/20-074.html)
34. Baptiste Roziere, Marie-Anne Lachaux, Lowik Chanussot, and Guillaume Lample. [Unsupervised translation of programming languages](https://md.research.elamur.ai/free/en/papers/lachaux-2020-unsupervised-translation-programming-languages/). Advances in Neural Information Processing Systems, 33, 2020. 2
35. Baptiste Roziere, Jie M Zhang, Francois Charton, Mark Harman, Gabriel Synnaeve, and Guillaume Lample. Leveraging automated unit tests for unsupervised code translation. In International Conference on Learning Representations, 2022. 1, 3 [find](https://scholar.google.com/scholar?q=Baptiste+Roziere%2C+Jie+M+Zhang%2C+Francois+Charton%2C+Mark+Harman%2C+Gabriel+Synnaeve%2C+and+Guillaume+Lample%2E+Leveraging+automated+unit+tests+for+unsupervised+code+translation%2E+In+International+Conference+on+)
36. Thomas Scialom, Paul-Alexis Dray, Jacopo Staiano, Sylvain Lamprier, and Benjamin Piwowarski. To beam or not to beam: That is a question of cooperation for language gans. Advances in neural information processing systems, 34:26585–26597, 2021. 3 [find](https://scholar.google.com/scholar?q=Thomas+Scialom%2C+Paul%2DAlexis+Dray%2C+Jacopo+Staiano%2C+Sylvain+Lamprier%2C+and+Benjamin+Piwowarski%2E+To+beam+or+not+to+beam%3A+That+is+a+question+of+cooperation+for+language+gans%2E+Advances+in+neural+information)
37. David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, and Marc Lanctot. Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587):484, 2016. Publisher: Nature Publis… [find](https://scholar.google.com/scholar?q=David+Silver%2C+Aja+Huang%2C+Chris+J%2E+Maddison%2C+Arthur+Guez%2C+Laurent+Sifre%2C+George+Van+Den+Driessche%2C+Julian+Schrittwieser%2C+Ioannis+Antonoglou%2C+Veda+Panneershelvam%2C+and+Marc+Lanctot%2E+Mastering+the+game+of)
38. David Silver, Thomas Hubert, Julian Schrittwieser, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, and Demis Hassabis. [Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm](https://md.research.elamur.ai/free/en/papers/al-2017-mastering-chess-shogi-self-play/)…
39. Riley Simmons-Edler, Anders Miltner, and Sebastian Seung. Program synthesis through reinforcement learning guided tree search. arXiv preprint arXiv:1806.02932, 2018. 3 — [arXiv:1806.02932](https://arxiv.org/abs/1806.02932) [find](https://scholar.google.com/scholar?q=Riley+Simmons%2DEdler%2C+Anders+Miltner%2C+and+Sebastian+Seung%2E+Program+synthesis+through+reinforcement+learning+guided+tree+search%2E+arXiv+preprint+arXiv%3A1806%2E02932%2C+2018%2E+3+%E2%80%94+%5BarXiv%3A1806%2E02932%5D%28https%3A%2F%2Farx)
40. Richard S. Sutton and Andrew G. Barto. Reinforcement Learning: An Introduction. MIT Press, October 2018. ISBN 978-0-262-35270-3. 4 [find](https://scholar.google.com/scholar?q=Richard+S%2E+Sutton+and+Andrew+G%2E+Barto%2E+Reinforcement+Learning%3A+An+Introduction%2E+MIT+Press%2C+October+2018%2E+ISBN+978%2D0%2D262%2D35270%2D3%2E+4)
41. Michele Tufano, Dawn Drain, Alexey Svyatkovskiy, Shao Kun Deng, and Neel Sundaresan. Unit test case generation with transformers and focal context. arXiv, 2020a. 3 [find](https://scholar.google.com/scholar?q=Michele+Tufano%2C+Dawn+Drain%2C+Alexey+Svyatkovskiy%2C+Shao+Kun+Deng%2C+and+Neel+Sundaresan%2E+Unit+test+case+generation+with+transformers+and+focal+context%2E+arXiv%2C+2020a%2E+3)
42. Michele Tufano, Dawn Drain, Alexey Svyatkovskiy, and Neel Sundaresan. Generating accurate assert statements for unit test cases using pretrained transformers. arXiv, 2020b. 3 [find](https://scholar.google.com/scholar?q=Michele+Tufano%2C+Dawn+Drain%2C+Alexey+Svyatkovskiy%2C+and+Neel+Sundaresan%2E+Generating+accurate+assert+statements+for+unit+test+cases+using+pretrained+transformers%2E+arXiv%2C+2020b%2E+3)
43. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). Advances in neural information processing systems, 2017a. 1, 3
44. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. [Attention Is All You Need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). arXiv:1706.03762 [cs], December 2017b. URL http://arxiv.org/abs/1706.03762. arXiv: 1706.03762. 2 [↗](http://arxiv.org/abs/1706.03762)
45. Yue Wang, Weishi Wang, Shafiq Joty, and Steven C.H. Hoi. CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 8696–8708, Online and Punta Cana, Dominic… [find](https://scholar.google.com/scholar?q=Yue+Wang%2C+Weishi+Wang%2C+Shafiq+Joty%2C+and+Steven+C%2EH%2E+Hoi%2E+CodeT5%3A+Identifier%2Daware+Unified+Pre%2Dtrained+Encoder%2DDecoder+Models+for+Code+Understanding+and+Generation%2E+In+Proceedings+of+the+2021+Conferenc)
46. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, and Morgan Funtowicz. Huggingface’s [transformers: State-of-the-art natural language processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/). arXiv preprint arXiv:1910.03771, 2019. 17
47. Yifan Xu, Lu Dai, Udaikaran Singh, Kening Zhang, and Zhuowen Tu. Neural program synthesis by self-learning. arXiv preprint arXiv:1910.05865, 2019a. 3 — [arXiv:1910.05865](https://arxiv.org/abs/1910.05865) [find](https://scholar.google.com/scholar?q=Yifan+Xu%2C+Lu+Dai%2C+Udaikaran+Singh%2C+Kening+Zhang%2C+and+Zhuowen+Tu%2E+Neural+program+synthesis+by+self%2Dlearning%2E+arXiv+preprint+arXiv%3A1910%2E05865%2C+2019a%2E+3+%E2%80%94+%5BarXiv%3A1910%2E05865%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1910%2E05)
48. Yifan Xu, Lu Dai, Udaikaran Singh, Kening Zhang, and Zhuowen Tu. Neural Program Synthesis By Self-Learning. October 2019b. doi: 10.48550/arXiv.1910.05865. URL https://arxiv.org/abs/1910.05865v1. 3 — [doi:10.48550/arxiv.1910.05865](https://doi.org/10.48550/arxiv.1910.05865) [↗](https://arxiv.org/abs/1910.05865v1)
49. A PPENDIX In this appendix, we supplement the main paper by providing more thorough empirical evaluations to back up our claims and more detailed descriptions of the algorithms to help readers better understand our paper. This appendix is organized as follows. • In Sec. A, we provide more comprehens… [find](https://scholar.google.com/scholar?q=A+PPENDIX+In+this+appendix%2C+we+supplement+the+main+paper+by+providing+more+thorough+empirical+evaluations+to+back+up+our+claims+and+more+detailed+descriptions+of+the+algorithms+to+help+readers+better+)

*Updated 2 November 2026*