---
title: "PaT: Planning-after-Trial for Efficient Test-Time Code Generation"
description: "The authors propose Planning-after-Trial (PaT), which reverses this order. 29 references. Among them: 0."
authors: ["Youngsik Yoon", "Sungjae Lee", "Seockbean Song", "Siwei Wang", "Wei Chen", "Jungseul Ok"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.1703/"
doi: "10.18653/v1/2026.acl-long.1703"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/yoon-2026-pat-planning-after-trial-efficient/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/yoon-2026-pat-planning-after-trial-efficient/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/44/) / PaT: Planning-after-Trial for Efficient Test-Time Code Generation

# PaT: Planning-after-Trial for Efficient Test-Time Code Generation

**Authors:** Youngsik Yoon, [Sungjae Lee](https://md.research.elamur.ai/free/en/authors/sungjae-lee/), Seockbean Song, Siwei Wang, [Wei Chen](https://md.research.elamur.ai/free/en/authors/wei-chen/) and [Jungseul Ok](https://md.research.elamur.ai/free/en/authors/jungseul-ok/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.1703/) · [PDF](https://aclanthology.org/2026.acl-long.1703.pdf) · [DOI 10.18653/v1/2026.acl-long.1703](https://doi.org/10.18653/v1/2026.acl-long.1703) · [OpenAlex](https://openalex.org/W7166861490)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Test-time scaling methods for LLM code generation often plan before trying any code (Planning-before-Trial, PbT). Planners such as FunCoder therefore spend compute decomposing every task, including easy ones a model could solve directly. The authors propose Planning-after-Trial (PaT), which reverses this order.

PaT works in stages:
- A generator first makes a Best-of-N attempt.
- Candidates are checked in a sandboxed Python runtime against self-generated tests, since benchmarks like MBPP lack usable tests.
- Only if every candidate fails is a planner called to split the problem into subproblems.
- Each subproblem is solved recursively with the same try-first rule, then composed and re-verified.

Because planning is rare, a cheap model can generate code while a strong model handles planning. Experiments used Qwen3 models (4B–32B) on HumanEval, MBPP and their EvalPlus variants, plus other benchmarks and model families. PaT beats FunCoder at every model size and costs less. This matters because it shows test-time compute can be spent selectively, without training any extra routing policy.

## Key points
- PaT outperforms the PbT baseline FunCoder at all Qwen3 sizes while using about 60% of its inference cost on average.
- In the mixed setup, a small generator paired with a Qwen3 32B planner comes within 1% of the all-32B system at roughly 31% of the cost (about 69% savings).
- Upgrading only the planner gives large gains for little extra cost, because the planner is invoked infrequently.
- The system generates about 6.7 test cases per problem and uses consensus-based scoring to cope with noisy tests. Qwen3 4B alone already reaches 76% Pass@1, which shows many tasks need no planning.
- Stated limitations:
  - Noisy self-made tests can trigger needless planning.
  - Hosting two models raises static memory use.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Microsoft Research Asia](https://md.research.elamur.ai/free/en/companies/microsoft-research-asia/) — Company, Beijing, China · 82 works · 61,484 citations
- [Pohang University of Science and Technology](https://md.research.elamur.ai/free/en/universities/pohang-university-of-science-and-technology/) — University, Pohang, South Korea · 8 works · 464 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works

🧭 Topics (1): [💻 Code & programming languages](https://md.research.elamur.ai/free/en/topics/code-and-programming/) · 168 works

## Cite

Cite as: Youngsik Yoon, Sungjae Lee, Seockbean Song, Siwei Wang, Wei Chen and Jungseul Ok. 2026. PaT: Planning-after-Trial for Efficient Test-Time Code Generation. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{yoon2026pat,
  title = {{PaT: Planning-after-Trial for Efficient Test-Time Code Generation}},
  author = {Youngsik Yoon, Sungjae Lee, Seockbean Song, Siwei Wang, Wei Chen and Jungseul Ok},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1703},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1703}
}
```

- [← previous PartisanLens: A Multilingual Dataset of Hyperpartisan and Conspiratorial Immigration Narratives in European Media](https://md.research.elamur.ai/free/en/papers/maggini-2026-partisanlens-multilingual-dataset-hyperpartisan-conspiratorial/)
- [next → Pattern Enhanced Multi-Turn Jailbreaking: Exploiting Structural Vulnerabilities in Large Language Models](https://md.research.elamur.ai/free/en/papers/nihal-2026-pattern-enhanced-multi-turn-jailbreaking/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/44/)

## References

29 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Gurusha Juneja, Subhabrata Dutta, and Tanmoy Chakraborty. 2024. LM 2 : A simple society of language models solves complex reasoning. arXiv preprint arXiv:2404.02255. — [arXiv:2404.02255](https://arxiv.org/abs/2404.02255) [find](https://scholar.google.com/scholar?q=Gurusha+Juneja%2C+Subhabrata+Dutta%2C+and+Tanmoy+Chakraborty%2E+2024%2E+LM+2+%3A+A+simple+society+of+language+models+solves+complex+reasoning%2E+arXiv+preprint+arXiv%3A2404%2E02255%2E+%E2%80%94+%5BarXiv%3A2404%2E02255%5D%28https%3A%2F%2Farxiv)
2. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and 1 others. 2021. [Program synthesis with large language models](https://md.research.elamur.ai/free/en/papers/austin-2021-program-synthesis-large-language-models/). arXiv preprint arXiv:2108.07732.
3. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. [Scaling laws for neural language models](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/). arXiv preprint arXiv:2001.08361.
4. Mohammad Abdullah Matin Khan, M Saiful Bari, Xuan Long Do, Weishi Wang, Md Rizwan Parvez, and Shafiq Joty. 2023. [xcodeeval: A large scale multilingual multitask benchmark for code understanding, generation, translation and retrieval](https://md.research.elamur.ai/free/en/papers/khan-2023-xcodeeval-large-scale-multilingual-multitask/). arXiv preprint arXiv:2303.03004.
5. Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Yi Su, John D Co-Reyes, Avi Singh, Kate Baumli, Shariq Iqbal, Colton Bishop, Rebecca Roelofs, and 1 others. 2024. [Training language models to self-correct via reinforcement learning](https://md.research.elamur.ai/free/en/papers/kumar-2024-training-language-models-self-correct/). arXiv preprint arXiv:2409.12917.
6. Hung Le, Hailin Chen, Amrita Saha, Akash Gokul, Doyen Sahoo, and Shafiq Joty. 2024. Codechain: Towards modular code generation through chain of self-revisions with representative sub-modules. In ICLR. [find](https://scholar.google.com/scholar?q=Hung+Le%2C+Hailin+Chen%2C+Amrita+Saha%2C+Akash+Gokul%2C+Doyen+Sahoo%2C+and+Shafiq+Joty%2E+2024%2E+Codechain%3A+Towards+modular+code+generation+through+chain+of+self%2Drevisions+with+representative+sub%2Dmodules%2E+In+ICLR%2E)
7. Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, R’emi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, and 1 others. 2022. [Competition-level code generation with alphacode](https://md.research.elamur.ai/free/en/papers/li-2022-competition-level-code-generation-alphacode/). Science, 378(6624):1092–1097.
8. Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. 2023. [Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation](https://md.research.elamur.ai/free/en/papers/liu-2023-your-code-generated-chatgpt-really/). Advances in Neural Information Processing Systems, 36:21558–21572.
9. Yifei Liu, Li Lyna Zhang, Yi Zhu, Bingcheng Dong, Xudong Zhou, Ning Shang, Fan Yang, and Mao Yang. 2025. [rstar-coder: Scaling competitive code reasoning with a large-scale verified dataset](https://md.research.elamur.ai/free/en/papers/liu-2025-rstar-coder-scaling-competitive/). arXiv preprint arXiv:2505.21297.
10. Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. 2023. Wizardcoder: Empowering code large language models with evol-instruct. arXiv preprint arXiv:2306.08568. [find](https://scholar.google.com/scholar?q=Ziyang+Luo%2C+Can+Xu%2C+Pu+Zhao%2C+Qingfeng+Sun%2C+Xiubo+Geng%2C+Wenxiang+Hu%2C+Chongyang+Tao%2C+Jing+Ma%2C+Qingwei+Lin%2C+and+Daxin+Jiang%2E+2023%2E+Wizardcoder%3A+Empowering+code+large+language+models+with+evol%2Dinstruct%2E+a)
11. OpenAI. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774.
12. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in neural information processing systems, 35:27730–27744.
13. Davide Paglieri, Bartłomiej Cupiał, Jonathan Cook, Ulyana Piterbarg, Jens Tuyls, Edward Grefenstette, Jakob Nicolaus Foerster, Jack Parker-Holder, and Tim Rocktäschel. 2025. Learning when to plan: Efficiently allocating test-time compute for llm agents. arXiv preprint arXiv:2509.03581. — [arXiv:2509.03581](https://arxiv.org/abs/2509.03581) [find](https://scholar.google.com/scholar?q=Davide+Paglieri%2C+Bart%C5%82omiej+Cupia%C5%82%2C+Jonathan+Cook%2C+Ulyana+Piterbarg%2C+Jens+Tuyls%2C+Edward+Grefenstette%2C+Jakob+Nicolaus+Foerster%2C+Jack+Parker%2DHolder%2C+and+Tim+Rockt%C3%A4schel%2E+2025%2E+Learning+when+to+plan%3A+Eff)
14. Archiki Prasad, Elias Stengel-Eskin, Justin Chih-Yao Chen, Zaid Khan, and Mohit Bansal. 2025. Learning to generate unit tests for automated debugging. arXiv preprint arXiv:2502.01619. [find](https://scholar.google.com/scholar?q=Archiki+Prasad%2C+Elias+Stengel%2DEskin%2C+Justin+Chih%2DYao+Chen%2C+Zaid+Khan%2C+and+Mohit+Bansal%2E+2025%2E+Learning+to+generate+unit+tests+for+automated+debugging%2E+arXiv+preprint+arXiv%3A2502%2E01619%2E)
15. Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, and 1 others. 2023. [Code llama: Open foundation models for code](https://md.research.elamur.ai/free/en/papers/roziere-2024-code-llama-open-foundation-models/). arXiv preprint arXiv:2308.12950.
16. Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. [Reflexion: Language agents with verbal reinforcement learning](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/). Advances in Neural Information Processing Systems, 36:8634–8652.
17. Charlie Victor Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2025. Scaling llm test-time compute optimally can be more effective than scaling parameters for reasoning. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Charlie+Victor+Snell%2C+Jaehoon+Lee%2C+Kelvin+Xu%2C+and+Aviral+Kumar%2E+2025%2E+Scaling+llm+test%2Dtime+compute+optimally+can+be+more+effective+than+scaling+parameters+for+reasoning%2E+In+The+Thirteenth+Internation)
18. Evan Z Wang, Federico Cassano, Catherine Wu, Yunfeng Bai, William Song, Vaskar Nath, Ziwen Han, Sean M Hendryx, Summer Yue, and Hugh Zhang. 2025a. Planning in natural language improves llm search for code generation. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Evan+Z+Wang%2C+Federico+Cassano%2C+Catherine+Wu%2C+Yunfeng+Bai%2C+William+Song%2C+Vaskar+Nath%2C+Ziwen+Han%2C+Sean+M+Hendryx%2C+Summer+Yue%2C+and+Hugh+Zhang%2E+2025a%2E+Planning+in+natural+language+improves+llm+search+for+)
19. Hanbin Wang, Zhenghao Liu, Shuo Wang, Ganqu Cui, Ning Ding, Zhiyuan Liu, and Ge Yu. 2024. Intervenor: Prompting the coding ability of large language models with the interactive chain of repair. In ACL (Findings). [find](https://scholar.google.com/scholar?q=Hanbin+Wang%2C+Zhenghao+Liu%2C+Shuo+Wang%2C+Ganqu+Cui%2C+Ning+Ding%2C+Zhiyuan+Liu%2C+and+Ge+Yu%2E+2024%2E+Intervenor%3A+Prompting+the+coding+ability+of+large+language+models+with+the+interactive+chain+of+repair%2E+In+ACL)
20. Wenhan Wang, Chenyuan Yang, Zhijie Wang, Yuheng Huang, Zhaoyang Chu, Da Song, Lingming Zhang, An Ran Chen, and Lei Ma. 2025b. Testeval: Benchmarking large language models for test case generation. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 3547–3562. [find](https://scholar.google.com/scholar?q=Wenhan+Wang%2C+Chenyuan+Yang%2C+Zhijie+Wang%2C+Yuheng+Huang%2C+Zhaoyang+Chu%2C+Da+Song%2C+Lingming+Zhang%2C+An+Ran+Chen%2C+and+Lei+Ma%2E+2025b%2E+Testeval%3A+Benchmarking+large+language+models+for+test+case+generation%2E+In+)
21. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). Advances in neural information processing systems, 35:24824– 24837.
22. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). arXiv preprint arXiv:2505.09388.
23. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems, 36:11809–11822. [find](https://scholar.google.com/scholar?q=Shunyu+Yao%2C+Dian+Yu%2C+Jeffrey+Zhao%2C+Izhak+Shafran%2C+Tom+Griffiths%2C+Yuan+Cao%2C+and+Karthik+Narasimhan%2E+2023%2E+Tree+of+thoughts%3A+Deliberate+problem+solving+with+large+language+models%2E+Advances+in+neural+inf)
24. Fengji Zhang, Bei Chen, Yue Zhang, Jacky Keung, Jin Liu, Daoguang Zan, Yi Mao, Jian-Guang Lou, and Weizhu Chen. 2023. [Repocoder: Repository-level code completion through iterative retrieval and generation](https://md.research.elamur.ai/free/en/papers/zhang-2023-repocoder-repository-level-code-completion/). arXiv preprint arXiv:2303.12570.
25. Huan Zhang, Wei Cheng, Yuhan Wu, and Wei Hu. 2024. A pair programming framework for code generation via multi-plan exploration and feedback-driven refinement. In ASE. [find](https://scholar.google.com/scholar?q=Huan+Zhang%2C+Wei+Cheng%2C+Yuhan+Wu%2C+and+Wei+Hu%2E+2024%2E+A+pair+programming+framework+for+code+generation+via+multi%2Dplan+exploration+and+feedback%2Ddriven+refinement%2E+In+ASE%2E)
26. Li Zhong, Zilong Wang, and Jingbo Shang. 2024. [Debug like a human: A large language model debugger via verifying runtime execution step by step](https://md.research.elamur.ai/free/en/papers/zhong-2024-debug-like-human-large-language/). In Findings of the Association for Computational Linguistics ACL 2024, pages 851–870.
27. Assumption 2 (Planning cost). A Planner-produced n-way plan maps a problem of size k to n independent subproblems of size n k and incurs a fixed per-subproblem overhead D M P , so that Denny Zhou, Nathanael Sch"arli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier… — [Least-to-most prompting enables complex reasoning in large language models](https://md.research.elamur.ai/free/en/papers/al-2022-least-most-prompting-enables-complex/)
28. Cost Planning = nD M P . Qihao Zhu, Daya Guo, Zhihong Shao, Dejian Yang, Peiyi Wang, Runxin Xu, Y Wu, Yukun Li, Huazuo Gao, Shirong Ma, and 1 others. 2024. [Deepseek-coder-v2: Breaking the barrier of closed-source models in code intelligence](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2024-deepseek-coder-v2-breaking-barrier/). arXiv preprint arXiv:2406.11931.
29. Yueheng Zhu, Chao Liu, Xuan He, Xiaoxue Ren, Zhongxin Liu, Ruwei Pan, and Hongyu Zhang. 2025. Adacoder: An adaptive planning and multi-agent framework for function-level code generation. arXiv preprint arXiv:2504.04220. — [arXiv:2504.04220](https://arxiv.org/abs/2504.04220) [find](https://scholar.google.com/scholar?q=Yueheng+Zhu%2C+Chao+Liu%2C+Xuan+He%2C+Xiaoxue+Ren%2C+Zhongxin+Liu%2C+Ruwei+Pan%2C+and+Hongyu+Zhang%2E+2025%2E+Adacoder%3A+An+adaptive+planning+and+multi%2Dagent+framework+for+function%2Dlevel+code+generation%2E+arXiv+preprin)

*Updated 2 November 2026*