---
title: "Advancing Reasoning in Diffusion Language Models with"
description: "The authors therefore propose the Denoising Process Reward (DPR). 56.3 accuracy. 37 references."
authors: ["Shaoan Xie", "Lingjing Kong", "Xiangchen Song", "Xinshuai Dong", "Guangyi Chen", "Eric P. Xing", "Kun Zhang"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.1978/"
doi: "10.18653/v1/2026.acl-long.1978"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/xie-2026-advancing-reasoning-diffusion-language-models/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/xie-2026-advancing-reasoning-diffusion-language-models/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/21/) / Advancing Reasoning in Diffusion Language Models with Denoising Process Rewards

# Advancing Reasoning in Diffusion Language Models with Denoising Process Rewards

**Authors:** Shaoan Xie, [Lingjing Kong](https://md.research.elamur.ai/free/en/authors/lingjing-kong/), [Xiangchen Song](https://md.research.elamur.ai/free/en/authors/xiangchen-song/), [Xinshuai Dong](https://md.research.elamur.ai/free/en/authors/xinshuai-dong/), Guangyi Chen, [Eric P. Xing](https://md.research.elamur.ai/free/en/authors/eric-p-xing/) and [Kun Zhang](https://md.research.elamur.ai/free/en/authors/kun-zhang/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.1978/) · [PDF](https://aclanthology.org/2026.acl-long.1978.pdf) · [DOI 10.18653/v1/2026.acl-long.1978](https://doi.org/10.18653/v1/2026.acl-long.1978) · [OpenAlex](https://openalex.org/W4414817567)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Masked diffusion language models (MdLLMs) such as LLaDA generate text by gradually unmasking a sequence. Reinforcement learning methods such as diffu-GRPO reward only the correctness of the final answer, so they do not supervise the denoising steps themselves. As a result, models often waste intermediate steps on repeated or empty tokens and settle the answer only at the end, which makes their reasoning hard to interpret and unstable.

Process Reward Models built for autoregressive LLMs are a poor fit here. They struggle with partially masked intermediate states, they are open to reward hacking, and they are costly to run during training. The authors therefore propose the Denoising Process Reward (DPR). DPR estimates how much a denoising interval contributes to the outcome by comparing the correctness of rollouts started from different timesteps. It needs no human annotation and no external reward model. An efficient stochastic estimator reuses ordinary training rollouts, so DPR fits into GRPO-style pipelines at little extra cost.

On reasoning benchmarks, DPR improves stability, interpretability and accuracy over LLaDA and diffu-GRPO, and code and checkpoints are released.

## Key points
- DPR is a process-level signal computed from the model's own denoising trajectory, built on top of GRPO and diffu-GRPO.
- On COUNTDOWN, the full method reaches 51.6, 52.0 and 56.3 accuracy at sequence lengths 128, 256 and 512. A variant without upweighting (Ours-NoUpweight) scores 41.0, 41.4 and 50.4.
- The efficient estimator runs at 3.42 sec/it while performing comparably to Ours-Cover (6.23 sec/it) and Ours-Random (4.76 sec/it). The authors describe its training speed as close to diffu-GRPO.
- In an ablation over the number of samples N ∈ {1, 3, 6, 9}, N = 1 gives noisy reward estimates, while N ≥ 3 is stable. The main experiments use N = 3.
- The method has two acknowledged limitations. It relies on diffu-GRPO's mean-field likelihood approximation. It also uses a simplified estimator (t₂ = T), which may introduce bias.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Carnegie Mellon University](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/) — University, Pittsburgh, United States · 598 works · 191,226 citations
- [Mohamed bin Zayed University of Artificial Intelligence](https://md.research.elamur.ai/free/en/universities/mohamed-bin-zayed-university-of-artificial-intelligence/) — University, Abu Dhabi, United Arab Emirates · 138 works · 6,590 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,685 works

## Cite

Cite as: Shaoan Xie, Lingjing Kong, Xiangchen Song, Xinshuai Dong, Guangyi Chen, Eric P. Xing and Kun Zhang. 2026. Advancing Reasoning in Diffusion Language Models with Denoising Process Rewards. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{xie2026advancing,
  title = {{Advancing Reasoning in Diffusion Language Models with Denoising Process Rewards}},
  author = {Shaoan Xie, Lingjing Kong, Xiangchen Song, Xinshuai Dong, Guangyi Chen, Eric P. Xing and Kun Zhang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1978},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1978}
}
```

- [← previous ADHD-Lang: A Large-Scale Social Media Dataset for Verbal Behavior and Digital Phenotyping in Adult ADHD](https://md.research.elamur.ai/free/en/papers/wiechmann-2026-adhd-lang-large-scale-social/)
- [next → Adversarial Yet Cooperative: Multi-Perspective Reasoning in Retrieved-Augmented Language Models](https://md.research.elamur.ai/free/en/papers/xu-2026-adversarial-yet-cooperative-multi-perspective/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/21/)

## References

37 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Black-Phoenix. 2024. 4x4-sudoku-dataset: 1 million 4×4 sudoku boards. [https://github.com/](https://github.com/) Black-Phoenix/4x4-Sudoku-Dataset. [↗](https://github.com/)
2. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. [Think you have solved question answering? try arc, the ai2 reasoning challenge](https://md.research.elamur.ai/free/en/papers/clark-2018-think-have-solved-question-answering/). arXiv preprint arXiv:1803.05457.
3. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, and 1 others. 2021. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168.
4. Ganqu Cui, Lifan Yuan, Zefan Wang, Hanbin Wang, Wendi Li, Bingxiang He, Yuchen Fan, Tianyu Yu, Qixin Xu, Weize Chen, and 1 others. 2025. [Process reinforcement through implicit rewards](https://md.research.elamur.ai/free/en/papers/cui-2025-process-reinforcement-through-implicit-rewards/). arXiv preprint arXiv:2502.01456.
5. Shansan Gong, Shivam Agarwal, Yizhe Zhang, Jiacheng Ye, Lin Zheng, Mukai Li, Chenxin An, Peilin Zhao, Wei Bi, Jiawei Han, and 1 others. 2024. Scaling diffusion language models via adaptation from autoregressive models. arXiv preprint arXiv:2410.17891. — [arXiv:2410.17891](https://arxiv.org/abs/2410.17891) [find](https://scholar.google.com/scholar?q=Shansan+Gong%2C+Shivam+Agarwal%2C+Yizhe+Zhang%2C+Jiacheng+Ye%2C+Lin+Zheng%2C+Mukai+Li%2C+Chenxin+An%2C+Peilin+Zhao%2C+Wei+Bi%2C+Jiawei+Han%2C+and+1+others%2E+2024%2E+Scaling+diffusion+language+models+via+adaptation+from+auto)
6. Shansan Gong, Ruixiang Zhang, Huangjie Zheng, Jiatao Gu, Navdeep Jaitly, Lingpeng Kong, and Yizhe Zhang. 2025. Diffucoder: Understanding and improving masked diffusion models for code generation. arXiv preprint arXiv:2506.20639. — [arXiv:2506.20639](https://arxiv.org/abs/2506.20639) [find](https://scholar.google.com/scholar?q=Shansan+Gong%2C+Ruixiang+Zhang%2C+Huangjie+Zheng%2C+Jiatao+Gu%2C+Navdeep+Jaitly%2C+Lingpeng+Kong%2C+and+Yizhe+Zhang%2E+2025%2E+Diffucoder%3A+Understanding+and+improving+masked+diffusion+models+for+code+generation%2E+arXi)
7. Haoyu He, Katrin Renz, Yong Cao, and Andreas Geiger. 2025. Mdpo: Overcoming the training-inference divide of masked diffusion language models. arXiv preprint arXiv:2508.13148. — [arXiv:2508.13148](https://arxiv.org/abs/2508.13148) [find](https://scholar.google.com/scholar?q=Haoyu+He%2C+Katrin+Renz%2C+Yong+Cao%2C+and+Andreas+Geiger%2E+2025%2E+Mdpo%3A+Overcoming+the+training%2Dinference+divide+of+masked+diffusion+language+models%2E+arXiv+preprint+arXiv%3A2508%2E13148%2E+%E2%80%94+%5BarXiv%3A2508%2E13148%5D%28htt)
8. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Filip+Wolski%2C+Prafulla+Dhariwal%2C+Alec+Radford%2C+and+Oleg+Klimov%2E+2017%2E+Proximal+policy+optimization+algorithms%2E+arXiv+preprint+arXiv%3A1707%2E06347%2E)
9. Feng Hong, Geng Yu, Yushi Ye, Haicheng Huang, Huangjie Zheng, Ya Zhang, Yanfeng Wang, and Jiangchao Yao. 2025. Wide-in, narrow-out: Revok-able decoding for efficient and effective dllms. arXiv preprint arXiv:2507.18578. — [arXiv:2507.18578](https://arxiv.org/abs/2507.18578) [find](https://scholar.google.com/scholar?q=Feng+Hong%2C+Geng+Yu%2C+Yushi+Ye%2C+Haicheng+Huang%2C+Huangjie+Zheng%2C+Ya+Zhang%2C+Yanfeng+Wang%2C+and+Jiangchao+Yao%2E+2025%2E+Wide%2Din%2C+narrow%2Dout%3A+Revok%2Dable+decoding+for+efficient+and+effective+dllms%2E+arXiv+preprin)
10. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and 1 others. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). arXiv preprint arXiv:2402.03300.
11. Pengxiang Li, Yefan Zhou, Dilxat Muhtar, Lu Yin, Shilin Yan, Li Shen, Yi Liang, Soroush Vosoughi, and Shiwei Liu. 2025. Diffusion language models know the answer before decoding. arXiv preprint arXiv:2508.19982. — [arXiv:2508.19982](https://arxiv.org/abs/2508.19982) [find](https://scholar.google.com/scholar?q=Pengxiang+Li%2C+Yefan+Zhou%2C+Dilxat+Muhtar%2C+Lu+Yin%2C+Shilin+Yan%2C+Li+Shen%2C+Yi+Liang%2C+Soroush+Vosoughi%2C+and+Shiwei+Liu%2E+2025%2E+Diffusion+language+models+know+the+answer+before+decoding%2E+arXiv+preprint+arXiv%3A)
12. Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. [Let’s verify step by step](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/). In The Twelfth International Conference on Learning Representations.
13. Yuerong Song, Xiaoran Liu, Ruixiao Li, Zhigeng Liu, Zengfeng Huang, Qipeng Guo, Ziwei He, and Xipeng Qiu. 2025. Sparse-dllm: Accelerating diffusion llms with dynamic cache eviction. arXiv preprint arXiv:2508.02558. — [arXiv:2508.02558](https://arxiv.org/abs/2508.02558) [find](https://scholar.google.com/scholar?q=Yuerong+Song%2C+Xiaoran+Liu%2C+Ruixiao+Li%2C+Zhigeng+Liu%2C+Zengfeng+Huang%2C+Qipeng+Guo%2C+Ziwei+He%2C+and+Xipeng+Qiu%2E+2025%2E+Sparse%2Ddllm%3A+Accelerating+diffusion+llms+with+dynamic+cache+eviction%2E+arXiv+preprint+arX)
14. Xiaohang Tang, Rares Dolga, Sangwoong Yoon, and Ilija Bogunovic. 2025. wd1: Weighted policy optimization for reasoning in diffusion language models. arXiv preprint arXiv:2507.08838. — [arXiv:2507.08838](https://arxiv.org/abs/2507.08838) [find](https://scholar.google.com/scholar?q=Xiaohang+Tang%2C+Rares+Dolga%2C+Sangwoong+Yoon%2C+and+Ilija+Bogunovic%2E+2025%2E+wd1%3A+Weighted+policy+optimization+for+reasoning+in+diffusion+language+models%2E+arXiv+preprint+arXiv%3A2507%2E08838%2E+%E2%80%94+%5BarXiv%3A2507%2E0883)
15. Xiaoran Liu, Zhigeng Liu, Zengfeng Huang, Qipeng Guo, Ziwei He, and Xipeng Qiu. 2025a. Longllada: Unlocking long context capabilities in diffusion llms. arXiv preprint arXiv:2506.14429. — [arXiv:2506.14429](https://arxiv.org/abs/2506.14429) [find](https://scholar.google.com/scholar?q=Xiaoran+Liu%2C+Zhigeng+Liu%2C+Zengfeng+Huang%2C+Qipeng+Guo%2C+Ziwei+He%2C+and+Xipeng+Qiu%2E+2025a%2E+Longllada%3A+Unlocking+long+context+capabilities+in+diffusion+llms%2E+arXiv+preprint+arXiv%3A2506%2E14429%2E+%E2%80%94+%5BarXiv%3A2506%2E)
16. Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. 2022. [Solving math word problems with process-and outcome-based feedback](https://md.research.elamur.ai/free/en/papers/uesato-2022-solving-math-word-problems-process/). arXiv preprint arXiv:2211.14275.
17. Zhiyuan Liu, Yicun Yang, Yaojie Zhang, Junjie Chen, Chang Zou, Qingyuan Wei, Shaobo Wang, and Linfeng Zhang. 2025b. dllm-cache: Accelerating diffusion large language models with adaptive caching. arXiv preprint arXiv:2506.06295. — [arXiv:2506.06295](https://arxiv.org/abs/2506.06295) [find](https://scholar.google.com/scholar?q=Zhiyuan+Liu%2C+Yicun+Yang%2C+Yaojie+Zhang%2C+Junjie+Chen%2C+Chang+Zou%2C+Qingyuan+Wei%2C+Shaobo+Wang%2C+and+Linfeng+Zhang%2E+2025b%2E+dllm%2Dcache%3A+Accelerating+diffusion+large+language+models+with+adaptive+caching%2E+arXi)
18. Guanghan Wang, Yair Schiff, Subham Sekhar Sahoo, and Volodymyr Kuleshov. 2025a. Remasking discrete diffusion models with inference-time scaling. arXiv preprint arXiv:2503.00307. — [arXiv:2503.00307](https://arxiv.org/abs/2503.00307) [find](https://scholar.google.com/scholar?q=Guanghan+Wang%2C+Yair+Schiff%2C+Subham+Sekhar+Sahoo%2C+and+Volodymyr+Kuleshov%2E+2025a%2E+Remasking+discrete+diffusion+models+with+inference%2Dtime+scaling%2E+arXiv+preprint+arXiv%3A2503%2E00307%2E+%E2%80%94+%5BarXiv%3A2503%2E00307%5D%28h)
19. Liangchen Luo, Yinxiao Liu, Rosanne Liu, Samrat Phatale, Meiqi Guo, Harsh Lara, Yunxuan Li, Lei Shu, Yun Zhu, Lei Meng, and 1 others. 2024. [Improve mathematical reasoning in language models by automated process supervision](https://md.research.elamur.ai/free/en/papers/luo-2024-improve-mathematical-reasoning-language-models/). arXiv preprint arXiv:2406.06592.
20. Xinyin Ma, Runpeng Yu, Gongfan Fang, and Xinchao Wang. 2025. dkv-cache: The cache for diffusion language models. arXiv preprint arXiv:2505.15781. — [arXiv:2505.15781](https://arxiv.org/abs/2505.15781) [find](https://scholar.google.com/scholar?q=Xinyin+Ma%2C+Runpeng+Yu%2C+Gongfan+Fang%2C+and+Xinchao+Wang%2E+2025%2E+dkv%2Dcache%3A+The+cache+for+diffusion+language+models%2E+arXiv+preprint+arXiv%3A2505%2E15781%2E+%E2%80%94+%5BarXiv%3A2505%2E15781%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2505%2E15781%29)
21. Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. 2025. [Large language diffusion models](https://md.research.elamur.ai/free/en/papers/nie-2025-large-language-diffusion-models/). arXiv preprint arXiv:2502.09992.
22. Jiayi Pan, Junjie Zhang, Xingyao Wang, Lifan Yuan, Hao Peng, and Alane Suhr. 2025. Tinyzero. [https://github.com/Jiayi-Pan/TinyZero](https://github.com/Jiayi-Pan/TinyZero). Accessed: 2025-01-24. [↗](https://github.com/Jiayi-Pan/TinyZero)
23. Arkil Patel, Satwik Bhattamishra, and Navin Goyal. 2021. [Are nlp models really able to solve simple math word problems?](https://md.research.elamur.ai/free/en/papers/patel-2021-are-nlp-models-really-able/) arXiv preprint arXiv:2103.07191.
24. Subham Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin Chiu, Alexander Rush, and Volodymyr Kuleshov. 2024. [Simple and effective masked diffusion language models](https://md.research.elamur.ai/free/en/papers/sahoo-2024-simple-effective-masked-diffusion-language/). Advances in Neural Information Processing Systems, 37:130136–130184.
25. Peiyi Wang, Lei Li, Zhihong Shao, RX Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. 2023. [Math-shepherd: Verify and reinforce llms step-by-step without human annotations](https://md.research.elamur.ai/free/en/papers/wang-2024-math-shepherd-verify-reinforce-llms/). arXiv preprint arXiv:2312.08935.
26. Weiyun Wang, Zhangwei Gao, Lianjie Chen, Zhe Chen, Jinguo Zhu, Xiangyu Zhao, Yangzhou Liu, Yue Cao, Shenglong Ye, Xizhou Zhu, and 1 others. 2025b. Visualprm: An effective process reward model for multimodal reasoning. arXiv preprint arXiv:2503.10291. — [arXiv:2503.10291](https://arxiv.org/abs/2503.10291) [find](https://scholar.google.com/scholar?q=Weiyun+Wang%2C+Zhangwei+Gao%2C+Lianjie+Chen%2C+Zhe+Chen%2C+Jinguo+Zhu%2C+Xiangyu+Zhao%2C+Yangzhou+Liu%2C+Yue+Cao%2C+Shenglong+Ye%2C+Xizhou+Zhu%2C+and+1+others%2E+2025b%2E+Visualprm%3A+An+effective+process+reward+model+for+mult)
27. Wen Wang, Bozhen Fang, Chenchen Jing, Yongliang Shen, Yangyi Shen, Qiuyu Wang, Hao Ouyang, Hao Chen, and Chunhua Shen. 2025c. Time is a feature: Exploiting temporal dynamics in diffusion language models. arXiv preprint arXiv:2508.09138. — [arXiv:2508.09138](https://arxiv.org/abs/2508.09138) [find](https://scholar.google.com/scholar?q=Wen+Wang%2C+Bozhen+Fang%2C+Chenchen+Jing%2C+Yongliang+Shen%2C+Yangyi+Shen%2C+Qiuyu+Wang%2C+Hao+Ouyang%2C+Hao+Chen%2C+and+Chunhua+Shen%2E+2025c%2E+Time+is+a+feature%3A+Exploiting+temporal+dynamics+in+diffusion+language+mode)
28. Yinjie Wang, Ling Yang, Bowen Li, Ye Tian, Ke Shen, and Mengdi Wang. 2025d. Revolutionizing reinforcement learning framework for diffusion large language models. arXiv preprint arXiv:2509.06949. — [arXiv:2509.06949](https://arxiv.org/abs/2509.06949) [find](https://scholar.google.com/scholar?q=Yinjie+Wang%2C+Ling+Yang%2C+Bowen+Li%2C+Ye+Tian%2C+Ke+Shen%2C+and+Mengdi+Wang%2E+2025d%2E+Revolutionizing+reinforcement+learning+framework+for+diffusion+large+language+models%2E+arXiv+preprint+arXiv%3A2509%2E06949%2E+%E2%80%94+%5Bar)
29. Chengyue Wu, Hao Zhang, Shuchen Xue, Zhijian Liu, Shizhe Diao, Ligeng Zhu, Ping Luo, Song Han, and Enze Xie. 2025. Fast-dllm: Training-free acceleration of diffusion llm by enabling kv cache and parallel decoding. arXiv preprint arXiv:2505.22618. — [arXiv:2505.22618](https://arxiv.org/abs/2505.22618) [find](https://scholar.google.com/scholar?q=Chengyue+Wu%2C+Hao+Zhang%2C+Shuchen+Xue%2C+Zhijian+Liu%2C+Shizhe+Diao%2C+Ligeng+Zhu%2C+Ping+Luo%2C+Song+Han%2C+and+Enze+Xie%2E+2025%2E+Fast%2Ddllm%3A+Training%2Dfree+acceleration+of+diffusion+llm+by+enabling+kv+cache+and+paral)
30. Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong. 2025. [Dream 7b: Diffusion large language models](https://md.research.elamur.ai/free/en/papers/ye-2025-dream-7b-diffusion-large-language/). arXiv preprint arXiv:2508.15487.
31. Fei Yu, Anningzhe Gao, and Benyou Wang. 2023. [Ovm, outcome-supervised value models for planning in mathematical reasoning](https://md.research.elamur.ai/free/en/papers/yu-2023-ovm-outcome-supervised-value-models/). arXiv preprint arXiv:2311.09724.
32. Lifan Yuan, Wendi Li, Huayu Chen, Ganqu Cui, Ning Ding, Kaiyan Zhang, Bowen Zhou, Zhiyuan Liu, and Hao Peng. 2024. [Free process rewards without process labels](https://md.research.elamur.ai/free/en/papers/yuan-2024-free-process-rewards-process-labels/). arXiv preprint arXiv:2412.01981.
33. Dan Zhang, Sining Zhoubian, Ziniu Hu, Yisong Yue, Yuxiao Dong, and Jie Tang. 2024a. [Rest-mcts*: Llm self-training via process reward guided tree search](https://md.research.elamur.ai/free/en/papers/zhang-2024-rest-mcts-llm-self-training/). Advances in Neural Information Processing Systems, 37:64735–64772.
34. Hanning Zhang, Pengcheng Wang, Shizhe Diao, Yong Lin, Rui Pan, Hanze Dong, Dylan Zhang, Pavlo Molchanov, and Tong Zhang. 2024b. Entropy-regularized process reward model. arXiv preprint arXiv:2412.11006. [find](https://scholar.google.com/scholar?q=Hanning+Zhang%2C+Pengcheng+Wang%2C+Shizhe+Diao%2C+Yong+Lin%2C+Rui+Pan%2C+Hanze+Dong%2C+Dylan+Zhang%2C+Pavlo+Molchanov%2C+and+Tong+Zhang%2E+2024b%2E+Entropy%2Dregularized+process+reward+model%2E+arXiv+preprint+arXiv%3A2412%2E1100)
35. Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. 2025. [The lessons of developing process reward models in mathematical reasoning](https://md.research.elamur.ai/free/en/papers/zhang-2025-lessons-developing-process-reward-models/). arXiv preprint arXiv:2501.07301.
36. Siyan Zhao, Devaansh Gupta, Qinqing Zheng, and Aditya Grover. 2025a. d1: Scaling reasoning in diffusion large language models via reinforcement learning. arXiv preprint arXiv:2504.12216. — [arXiv:2504.12216](https://arxiv.org/abs/2504.12216) [find](https://scholar.google.com/scholar?q=Siyan+Zhao%2C+Devaansh+Gupta%2C+Qinqing+Zheng%2C+and+Aditya+Grover%2E+2025a%2E+d1%3A+Scaling+reasoning+in+diffusion+large+language+models+via+reinforcement+learning%2E+arXiv+preprint+arXiv%3A2504%2E12216%2E+%E2%80%94+%5BarXiv%3A2504)
37. Siyan Zhao, Mengchen Liu, Jing Huang, Miao Liu, Chenyu Wang, Bo Liu, Yuandong Tian, Guan Pang, Sean Bell, Aditya Grover, and 1 others. 2025b. Inpainting-guided policy optimization for diffusion large language models. arXiv preprint arXiv:2509.10396. — [arXiv:2509.10396](https://arxiv.org/abs/2509.10396) [find](https://scholar.google.com/scholar?q=Siyan+Zhao%2C+Mengchen+Liu%2C+Jing+Huang%2C+Miao+Liu%2C+Chenyu+Wang%2C+Bo+Liu%2C+Yuandong+Tian%2C+Guan+Pang%2C+Sean+Bell%2C+Aditya+Grover%2C+and+1+others%2E+2025b%2E+Inpainting%2Dguided+policy+optimization+for+diffusion+large+)

*Updated 2 November 2026*