---
title: "Process Supervision-Guided Policy Optimization for Code"
description: "The authors (ByteDance and Oregon State University) propose Process Supervision-Guided Policy… 34 references, cited by 12 papers here."
authors: ["Ning Dai", "Zheng Wu", "Renjie Zheng", "Ziyun Wei", "Wenlei Shi", "Xing Jin", "Guanlin Liu", "Chen Dun", "Liang Huang", "Lin Yan"]
year: 2024
venue: "arXiv:2410.17621"
source: "https://arxiv.org/abs/2410.17621"
doi: "10.48550/arxiv.2410.17621"
arxiv: "2410.17621"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/dai-2024-process-supervision-guided-policy-optimization/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/dai-2024-process-supervision-guided-policy-optimization/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/13/) / Process Supervision-Guided Policy Optimization for Code Generation

# Process Supervision-Guided Policy Optimization for Code Generation

**Authors:** [Ning Dai](https://md.research.elamur.ai/free/en/authors/ning-dai/), [Zheng Wu](https://md.research.elamur.ai/free/en/authors/zheng-wu/), [Renjie Zheng](https://md.research.elamur.ai/free/en/authors/renjie-zheng/), Ziyun Wei, [Wenlei Shi](https://md.research.elamur.ai/free/en/authors/wenlei-shi/), [Xing Jin](https://md.research.elamur.ai/free/en/authors/xing-jin/), [Guanlin Liu](https://md.research.elamur.ai/free/en/authors/guanlin-liu/), Chen Dun, [Liang Huang](https://md.research.elamur.ai/free/en/authors/liang-huang/) and [Lin Yan](https://md.research.elamur.ai/free/en/authors/lin-yan/) · **Year:** 2024 · **Venue:** arXiv:2410.17621 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2410.17621) · [PDF](https://arxiv.org/pdf/2410.17621) · [DOI 10.48550/arxiv.2410.17621](https://doi.org/10.48550/arxiv.2410.17621) · [arXiv 2410.17621](https://arxiv.org/abs/2410.17621) · [OpenAlex](https://openalex.org/W4404305217)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [12](https://md.research.elamur.ai/free/en/papers/dai-2024-process-supervision-guided-policy-optimization/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
RL from unit-test feedback helps LLMs write better code, but the reward is sparse. A single pass/fail signal arrives only after the whole program runs, and a program that fails every test gives the model nothing to learn from. The authors (ByteDance and Oregon State University) propose Process Supervision-Guided Policy Optimization (PSGPO), which adds a Process Reward Model (PRM) that scores each generated line of code.

PRM training labels are produced automatically. A policy response is split into line-level prefixes, best-of-K completions are sampled from each prefix and executed against unit tests, and a binary search finds the point after which no completion succeeds. In RL, the PRM plays two roles at once: it supplies dense per-line rewards, and it initializes the value function. This combination worked best and raised pass rates on HumanEval, MBPP and LiveCodeBench for two base LLMs, with the clearest gains on long outputs. The work matters because it brings process supervision, previously explored mostly for math reasoning, to code generation and offers a practical recipe for using it.

## Key points
- PRM labels need no human annotation. A prefix counts as correct (+1) if any of K sampled completions passes all unit tests, and incorrect (−1) otherwise. Binary search over lines keeps the number of checks low.
- Using the PRM both as a dense reward and as the value-model initialization gave a stronger synergy than either use alone. The authors also report better exploration and learning efficiency.
- In the response-length analysis, PRM-trained models improved Pass@1 by about 9% over the baseline. Gains were consistent for responses longer than 100 tokens, while shorter responses saw neutral or slightly negative effects.
- PRM data selection affects results. On LiveCodeBench with Doubao-Lite-RL, the scores were:
  - Full data: 26.9
  - Remove Hard: 27.8
  - Medium Only: 26.9
  - Revised Only: 29.8 (best)
- Policy quality kept improving as more responses per prompt were collected for PRM data. The main limitations are:
  - noisy automatic labels;
  - high compute cost;
  - reliance on unit tests, which limits use in tasks without clear correctness checks.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,888 works · [🖥️ Computer Science Applications](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-science-applications/) · 184 works

🧭 Topics (1): [💻 Code & programming languages](https://md.research.elamur.ai/free/en/topics/code-and-programming/) · 194 works

## Cite

Cite as: Ning Dai, Zheng Wu, Renjie Zheng, Ziyun Wei, Wenlei Shi, Xing Jin, Guanlin Liu, Chen Dun, Liang Huang and Lin Yan. 2024. Process Supervision-Guided Policy Optimization for Code Generation. arXiv:2410.17621.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{dai2024process,
  title = {{Process Supervision-Guided Policy Optimization for Code Generation}},
  author = {Ning Dai, Zheng Wu, Renjie Zheng, Ziyun Wei, Wenlei Shi, Xing Jin, Guanlin Liu, Chen Dun, Liang Huang and Lin Yan},
  year = {2024},
  doi = {10.48550/arxiv.2410.17621},
  eprint = {2410.17621},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2410.17621}
}
```

- [← previous Performance Analysis of Speech Encoders for Low-Resource SLU and ASR in Tunisian Dialect](https://md.research.elamur.ai/free/en/papers/mdhaffar-2024-performance-analysis-speech-encoders-low/)
- [next → Proving Olympiad Algebraic Inequalities without Human Demonstrations](https://md.research.elamur.ai/free/en/papers/wei-2024-proving-olympiad-algebraic-inequalities-without/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/13/)

## References

34 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. trained on code. CoRR, abs/2107.[03374, 2021b](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/). URL https://arxiv.org/abs/2107.03374. [↗](https://arxiv.org/abs/2107.03374)
2. Allal, L. B., Li, R., Kocetkov, D., Mou, C., Akiki, C., Ferrandis, C. M., Muennighoff, N., Mishra, M., Gu, A., Dey, M., Umapathi, L. K., Anderson, C. J., Zi, Y., Poirier, J. L., Schoelkopf, H., Troshin, S., Abulkhanov, D., Romero, M., Lappert, M., Toni, F. D., del Rı́o, B. G., Liu, Q., Bose, S., Bha… — [SantaCoder: don't reach for the stars!](https://md.research.elamur.ai/free/en/papers/allal-2023-santacoder-don-t-reach-stars/)
3. Christopoulou, F., Lampouras, G., Gritta, M., Zhang, G., Guo, Y., Li, Z., Zhang, Q., Xiao, M., Shen, B., Li, L., Yu, H., Yan, L., Zhou, P., Wang, X., Ma, Y., Iacobacci, I., Wang, Y., Liang, G., Wei, J., Jiang, X., Wang, Q., and Liu, Q. Pangu-coder: Program synthesis with function-level language mode… — [PanGu-Coder: Program Synthesis with Function-Level Language Modeling](https://md.research.elamur.ai/free/en/papers/christopoulou-2022-pangu-coder-program-synthesis/)
4. Dou, S., Liu, Y., Jia, H., Xiong, L., Zhou, E., Shan, J., Huang, C., Shen, W., Fan, X., Xi, Z., et al. Stepcoder: Improve code generation with reinforcement learning from compiler feedback. arXiv preprint arXiv:2402.01391, 2024. — [arXiv:2402.01391](https://arxiv.org/abs/2402.01391) [find](https://scholar.google.com/scholar?q=Dou%2C+S%2E%2C+Liu%2C+Y%2E%2C+Jia%2C+H%2E%2C+Xiong%2C+L%2E%2C+Zhou%2C+E%2E%2C+Shan%2C+J%2E%2C+Huang%2C+C%2E%2C+Shen%2C+W%2E%2C+Fan%2C+X%2E%2C+Xi%2C+Z%2E%2C+et+al%2E+Stepcoder%3A+Improve+code+generation+with+reinforcement+learning+from+compiler+feedback%2E+arXiv+prep)
5. Austin, J., Odena, A., Nye, M. I., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C. J., Terry, M., Le, Q. V., and Sutton, C. [Program synthesis with large language models](https://md.research.elamur.ai/free/en/papers/austin-2021-program-synthesis-large-language-models/). CoRR, abs/2108.07732, 2021. URL https://arxiv.org/abs/2108.07732. [↗](https://arxiv.org/abs/2108.07732)
6. Gunasekar, S., Zhang, Y., Aneja, J., Mendes, C. C. T., Giorno, A. D., Gopi, S., Javaheripi, M., Kauffmann, P., de Rosa, G., Saarikivi, O., Salim, A., Shah, S., Behl, H. S., Wang, X., Bubeck, S., Eldan, R., Kalai, A. T., Lee, Y. T., and Li, Y. Textbooks are all you need. CoRR, abs/2306.11644, 2023. d… [find](https://scholar.google.com/scholar?q=Gunasekar%2C+S%2E%2C+Zhang%2C+Y%2E%2C+Aneja%2C+J%2E%2C+Mendes%2C+C%2E+C%2E+T%2E%2C+Giorno%2C+A%2E+D%2E%2C+Gopi%2C+S%2E%2C+Javaheripi%2C+M%2E%2C+Kauffmann%2C+P%2E%2C+de+Rosa%2C+G%2E%2C+Saarikivi%2C+O%2E%2C+Salim%2C+A%2E%2C+Shah%2C+S%2E%2C+Behl%2C+H%2E+S%2E%2C+Wang%2C+X%2E%2C+Bubeck%2C+S%2E%2C+Eldan)
7. Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., Huang, F., et al. [Qwen technical report](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-technical-report/). arXiv preprint arXiv:2309.16609, 2023.
8. Chen, M., Tworek, J., Jun, H., Yuan, Q., de Oliveira Pinto, H. P., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., Ray, A., Puri, R., Krueger, G., Petrov, M., Khlaaf, H., Sastry, G., Mishkin, P., Chan, B., Gray, S., Ryder, N., Pavlov, M., Power, A., Kaiser, L., Bavarian, M., Winter, C.… — [Evaluating Large Language Models Trained on Code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
9. Guo, D., Zhu, Q., Yang, D., Xie, Z., Dong, K., Zhang, W., Chen, G., Bi, X., Wu, Y., Li, Y. K., Luo, F., Xiong, Y., and Liang, W. Deepseek-coder: When the large language model meets programming – the rise of code intelligence, 2024. URL https://arxiv.org/abs/ 2401.14196. [↗](https://arxiv.org/abs/)
10. Jain, N., Han, K., Gu, A., Li, W.-D., Yan, F., Zhang, T., Wang, S., Solar-Lezama, A., Sen, K., and Stoica, I. [Livecodebench: Holistic and contamination free evaluation of large language models for code](https://md.research.elamur.ai/free/en/papers/jain-2024-livecodebench-holistic-contamination-free-evaluation/). arXiv preprint arXiv:2403.07974, 2024.
11. Jiao, F., Qin, C., Liu, Z., Chen, N. F., and Joty, S. [Learning planning-based reasoning by trajectories collection and process reward synthesizing](https://md.research.elamur.ai/free/en/papers/jiao-2024-learning-planning-based-reasoning-trajectories/). CoRR, abs/2402.00658, 2024. doi: 10.48550/ARXIV.2402.00658. URL https://doi.org/10.48550/arXiv.2402.00658. [↗](https://doi.org/10.48550/arXiv.2402.00658)
12. Chen, M., Tworek, J., Jun, H., Yuan, Q., de Oliveira Pinto, H. P., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., Ray, A., Puri, R., Krueger, G., Petrov, M., Khlaaf, H., Sastry, G., Mishkin, P., Chan, B., Gray, S., Ryder, N., Pavlov, M., Power, A., Kaiser, L., Bavarian, M., Winter, C.… — [SPoC: Search-based Pseudocode to Code](https://md.research.elamur.ai/free/en/papers/kulal-2019-spoc-search-based-pseudocode-code/)
13. Le, H., La Cava, W., and Moore, J. H. Coderl: Mastering code generation through pretrained models and deep reinforcement learning. arXiv preprint arXiv:2207.[01780, 2022](https://md.research.elamur.ai/free/en/papers/luo-2024-improve-mathematical-reasoning-language-models/). 2024. doi: 10.48550/ARXIV.2406.06592. URL https://doi.org/10.48550/arXiv.2406.06592. [↗](https://doi.org/10.48550/arXiv.2406.06592)
14. Luo, Z., Xu, C., Zhao, P., Sun, Q., Geng, X., Hu, W., Tao, C., Ma, J., Lin, Q., and Jiang, D. Wizardcoder: Empowering code large language models with evol-instruct, 2023. URL https://arxiv.org/abs/2306.08568. [↗](https://arxiv.org/abs/2306.08568)
15. Li, R., Allal, L. B., Zi, Y., Muennighoff, N., Kocetkov, D., Mou, C., Marone, M., Akiki, C., Li, J., Chim, J., Liu, Q., Zheltonozhskii, E., Zhuo, T. Y., Wang, T., Dehaene, O., Davaadorj, M., Lamy-Poirier, J., Monteiro, J., Shliazhko, O., Gontier, N., Meade, N., Zebaze, A., Yee, M.-H., Umapathi, L. K… [find](https://scholar.google.com/scholar?q=Li%2C+R%2E%2C+Allal%2C+L%2E+B%2E%2C+Zi%2C+Y%2E%2C+Muennighoff%2C+N%2E%2C+Kocetkov%2C+D%2E%2C+Mou%2C+C%2E%2C+Marone%2C+M%2E%2C+Akiki%2C+C%2E%2C+Li%2C+J%2E%2C+Chim%2C+J%2E%2C+Liu%2C+Q%2E%2C+Zheltonozhskii%2C+E%2E%2C+Zhuo%2C+T%2E+Y%2E%2C+Wang%2C+T%2E%2C+Dehaene%2C+O%2E%2C+Davaadorj%2C+M%2E%2C+Lamy%2DPoir)
16. OpenAI. [GPT-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). CoRR, abs/2303.08774, 2023. doi: 10.48550/ARXIV.2303.08774. URL https://doi.org/10.48550/arXiv.2303.08774. [↗](https://doi.org/10.48550/arXiv.2303.08774)
17. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., Schulman, J., Hilton, J., Kelton, F., Miller, L., Simens, M., Askell, A., Welinder, P., Christiano, P. F., Leike, J., and Lowe, R. Training language models to follow instructions w… — [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/)
18. QwenTeam. Qwen2.5: A party of foundation models, September 2024. URL [https://qwenlm.github.io/blog/qwen2.5/](https://qwenlm.github.io/blog/qwen2.5/). [↗](https://qwenlm.github.io/blog/qwen2.5/)
19. Li, Y., Choi, D., Chung, J., Guo, A., Yang, T.-Y., et al. [Competition-level code generation with alphacode](https://md.research.elamur.ai/free/en/papers/li-2022-competition-level-code-generation-alphacode/). Science, 378(6624):1092–1097, 2022.
20. Rafailov, R., Chittepu, Y., Park, R., Sikchi, H., Hejna, J., Knox, B., Finn, C., and Niekum, S. Scaling laws for reward model overoptimization in direct alignment algorithms. arXiv preprint arXiv:2406.02900, 2024. — [arXiv:2406.02900](https://arxiv.org/abs/2406.02900) [find](https://scholar.google.com/scholar?q=Rafailov%2C+R%2E%2C+Chittepu%2C+Y%2E%2C+Park%2C+R%2E%2C+Sikchi%2C+H%2E%2C+Hejna%2C+J%2E%2C+Knox%2C+B%2E%2C+Finn%2C+C%2E%2C+and+Niekum%2C+S%2E+Scaling+laws+for+reward+model+overoptimization+in+direct+alignment+algorithms%2E+arXiv+preprint+arXiv%3A2406)
21. Li, Y., Bubeck, S., Eldan, R., Giorno, A. D., Gunasekar, S., and Lee, Y. T. [Textbooks are all you need II: phi-1.5 technical report](https://md.research.elamur.ai/free/en/papers/li-2023-textbooks-all-need-ii-phi/). CoRR, abs/2309.05463, 2023b. doi: 10.48550/ARXIV.2309.05463. URL https://doi.org/10.48550/arXiv.2309.05463. [↗](https://doi.org/10.48550/arXiv.2309.05463)
22. Lightman, H., Kosaraju, V., Burda, Y., Edwards, H., Baker, B., Lee, T., Leike, J., Schulman, J., Sutskever, I., and Cobbe, K. Let’s verify step by step. CoRR, abs/2305.[20050, 2023](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/). doi: 10.48550/ ARXIV.2305.20050. URL https://doi.org/ 10.48550/arXiv.2305.20050.
23. Rozière, B., Gehring, J., Gloeckle, F., Sootla, S., Gat, I., Tan, X. E., Adi, Y., Liu, J., Sauvestre, R., Remez, T., Rapin, J., Kozhevnikov, A., Evtimov, I., Bitton, J., Bhatt, M., Ferrer, C. C., Grattafiori, A., Xiong, W., Défossez, A., Copet, J., Azhar, F., Touvron, H., Martin, L., Usunier, N.,… — [Code Llama: Open Foundation Models for Code](https://md.research.elamur.ai/free/en/papers/roziere-2024-code-llama-open-foundation-models/)
24. Liu, J., Zhu, Y., Xiao, K., Fu, Q., Han, X., Yang, W., and Ye, D. RLTF: reinforcement learning from unit test feedback. Trans. Mach. Learn. Res., 2023, 2023. URL [https://openreview.net/forum?id=hjYmsV6nXZ](https://openreview.net/forum?id=hjYmsV6nXZ). [↗](https://openreview.net/forum?id=hjYmsV6nXZ)
25. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. Proximal policy optimization algorithms. CoRR, abs/1707.06347, 2017. URL http://arxiv.org/abs/1707.06347. [↗](http://arxiv.org/abs/1707.06347)
26. Lu, S., Guo, D., Ren, S., Huang, J., Svyatkovskiy, A., Blanco, A., Clement, C., Drain, D., Jiang, D., Tang, D., Li, G., Zhou, L., Shou, L., Zhou, L., Tufano, M., Gong, M., Zhou, M., Duan, N., Sundaresan, N., Deng, S. K., Fu, S., and Liu, S. Codexglue: A machine learning benchmark dataset for code un… [find](https://scholar.google.com/scholar?q=Lu%2C+S%2E%2C+Guo%2C+D%2E%2C+Ren%2C+S%2E%2C+Huang%2C+J%2E%2C+Svyatkovskiy%2C+A%2E%2C+Blanco%2C+A%2E%2C+Clement%2C+C%2E%2C+Drain%2C+D%2E%2C+Jiang%2C+D%2E%2C+Tang%2C+D%2E%2C+Li%2C+G%2E%2C+Zhou%2C+L%2E%2C+Shou%2C+L%2E%2C+Zhou%2C+L%2E%2C+Tufano%2C+M%2E%2C+Gong%2C+M%2E%2C+Zhou%2C+M%2E%2C+Duan%2C+N%2E%2C+Sundares)
27. Shojaee, P., Jain, A., Tipirneni, S., and Reddy, C. K. [Execution-based code generation using deep reinforcement learning](https://md.research.elamur.ai/free/en/papers/shojaee-2023-execution-based-code-generation-using/). arXiv preprint arXiv:2301.13816, 2023.
28. Skalse, J., Howe, N., Krasheninnikov, D., and Krueger, D. Defining and characterizing reward gaming. Advances in Neural Information Processing Systems, 35:9460–9471, 2022. [find](https://scholar.google.com/scholar?q=Skalse%2C+J%2E%2C+Howe%2C+N%2E%2C+Krasheninnikov%2C+D%2E%2C+and+Krueger%2C+D%2E+Defining+and+characterizing+reward+gaming%2E+Advances+in+Neural+Information+Processing+Systems%2C+35%3A9460%E2%80%939471%2C+2022%2E)
29. Luo, L., Liu, Y., Liu, R., Phatale, S., Lara, H., Li, Y., Shu, L., Zhu, Y., Meng, L., Sun, J., and Rastogi, A. Improve mathematical reasoning in language models by automated process supervision. CoRR, abs/2406.[06592, Wang, P., Li, L., Shao, Z., Xu, R., Dai, D., Li, Y., Chen, D., Wu, Y., and Sui, Z.…](https://md.research.elamur.ai/free/en/papers/wang-2024-math-shepherd-verify-reinforce-llms/)
30. Wang, Z., Li, Y., Wu, Y., Luo, L., Hou, L., Yu, H., and Shang, J. [Multi-step problem solving through a verifier: An empirical analysis on model-induced process supervision](https://md.research.elamur.ai/free/en/papers/wang-2024-multi-step-problem-solving-through/). CoRR, abs/2402.02658, 2024b. doi: 10.48550/ARXIV.2402.02658. URL https://doi.org/10.48550/arXiv.2402.02658. [↗](https://doi.org/10.48550/arXiv.2402.02658)
31. Williams, R. J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8:229–256, 1992. [find](https://scholar.google.com/scholar?q=Williams%2C+R%2E+J%2E+Simple+statistical+gradient%2Dfollowing+algorithms+for+connectionist+reinforcement+learning%2E+Machine+learning%2C+8%3A229%E2%80%93256%2C+1992%2E)
32. Yang, A., Yang, B., Hui, B., Zheng, B., Yu, B., Zhou, C., Li, C., Li, C., Liu, D., Huang, F., Dong, G., Wei, H., Lin, H., Tang, J., Wang, J., Yang, J., Tu, J., Zhang, J., Ma, J., Xu, J., Zhou, J., Bai, J., He, J., Lin, J., Dang, K., Lu, K., Chen, K., Yang, K., Li, M., Xue, M., Ni, N., Zhang, P., Wan… [find](https://scholar.google.com/scholar?q=Yang%2C+A%2E%2C+Yang%2C+B%2E%2C+Hui%2C+B%2E%2C+Zheng%2C+B%2E%2C+Yu%2C+B%2E%2C+Zhou%2C+C%2E%2C+Li%2C+C%2E%2C+Li%2C+C%2E%2C+Liu%2C+D%2E%2C+Huang%2C+F%2E%2C+Dong%2C+G%2E%2C+Wei%2C+H%2E%2C+Lin%2C+H%2E%2C+Tang%2C+J%2E%2C+Wang%2C+J%2E%2C+Yang%2C+J%2E%2C+Tu%2C+J%2E%2C+Zhang%2C+J%2E%2C+Ma%2C+J%2E%2C+Xu%2C+J%2E%2C+Zhou%2C+J%2E%2C+Bai)
33. Zheng, Q., Xia, X., Zou, X., Dong, Y., Wang, S., Xue, Y., Wang, Z., Shen, L., Wang, A., Li, Y., Su, T., Yang, Z., and Tang, J. [Codegeex: A pre-trained model for code generation with multilingual benchmarking on humaneval-x](https://md.research.elamur.ai/free/en/papers/zheng-2023-codegeex-pre-trained-model-code/), 2024. URL [https://arxiv.org/](https://arxiv.org/) abs/2303.17568. [↗](https://arxiv.org/)
34. A. A Typical Example of the Learned Line-wise Rewards [find](https://scholar.google.com/scholar?q=A%2E+A+Typical+Example+of+the+Learned+Line%2Dwise+Rewards)

*Updated 3 November 2026*