---
title: "Improving the Throughput of Diffusion-based Large Language"
description: "Based on this, they propose CadLLM, a training-free controller. 10 references, cited by 2 papers here."
authors: ["Jucheng Shen", "Gaurav Sarkar", "Yeonju Ro", "Sharath Nittur Sridhar", "Zhangyang Wang", "Aditya Akella", "Souvik Kundu"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.478/"
doi: "10.18653/v1/2026.findings-acl.478"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/shen-2026-improving-throughput-diffusion-based-large/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/shen-2026-improving-throughput-diffusion-based-large/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/7/) / Improving the Throughput of Diffusion-based Large Language Models via a Training-Free Confidence-Aware Calibration

# Improving the Throughput of Diffusion-based Large Language Models via a Training-Free Confidence-Aware Calibration

**Authors:** Jucheng Shen, Gaurav Sarkar, Yeonju Ro, Sharath Nittur Sridhar, [Zhangyang Wang](https://md.research.elamur.ai/free/en/authors/zhangyang-wang/), Aditya Akella and Souvik Kundu · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.478/) · [PDF](https://aclanthology.org/2026.findings-acl.478.pdf) · [DOI 10.18653/v1/2026.findings-acl.478](https://doi.org/10.18653/v1/2026.findings-acl.478) · [OpenAlex](https://openalex.org/W7166854811)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [2](https://md.research.elamur.ai/free/en/papers/shen-2026-improving-throughput-diffusion-based-large/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Diffusion-based LLMs (dLLMs) such as LLaDA and DREAM generate text by repeatedly denoising masked positions, which makes inference slow. The prior accelerator, Fast-dLLM, decodes tokens in parallel using fixed block sizes, a fixed number of steps per block, a fixed commit threshold, and full-vocabulary sampling. The authors show that token confidence varies considerably both across blocks and across denoising steps. Based on this, they propose CadLLM, a training-free controller. It uses the average confidence of unmasked tokens to set four parameters on the fly:

- block size
- number of refinement steps
- confidence threshold for committing tokens
- size of the vocabulary subset used for softmax, with a repetition guard

Each policy is a simple clipped linear mapping, so the controller adds almost no overhead. It works with any model and with both prefix and dual KV caching.

The method was tested on a single H100 with GSM8K, MATH, MBPP, and HumanEval at generation lengths 256 and 512. Compared with Fast-dLLM, CadLLM raises throughput by 1.1–2.28× on LLaDA and 1.1–1.4× on DREAM, with comparable accuracy. This makes dLLM inference noticeably cheaper without any retraining.

## Key points
- **Speedup on LLaDA-Instruct:** throughput rises 1.1–2.28× over the Fast-dLLM threshold baseline. The largest gain is on HumanEval at g=256: 220.81 vs 96.84 tokens/s, with accuracy of 35.97% vs 37.19%.
- **Softmax cost:** running softmax over the full ~50K-token vocabulary is roughly ten times slower than over a small subset. CadLLM therefore restricts softmax to the top-logit tokens, typically 35–1,000 of them.
- **Ablation of the threshold (GSM8K, g=256):** removing the adaptive threshold cuts throughput by 71.6% and increases the number of function evaluations (NFE) by 289%.
- **Ablation of the vocabulary policy:** dropping the adaptive vocabulary size lowers accuracy by 4.6 percentage points.
- **Limitations:** results remain sensitive to hyperparameter tuning. The authors suggest learned schedules, such as RL-based or Bayesian optimization, as future work.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [The University of Texas at Austin](https://md.research.elamur.ai/free/en/universities/the-university-of-texas-at-austin/) — University, Austin, United States · 109 works · 56,942 citations
- [Intel](https://md.research.elamur.ai/free/en/companies/intel/) — Company, Santa Clara, United States · 15 works · 41,257 citations
- [Rice University](https://md.research.elamur.ai/free/en/universities/rice-university/) — University, Houston, United States · 11 works · 1,850 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works

## Cite

Cite as: Jucheng Shen, Gaurav Sarkar, Yeonju Ro, Sharath Nittur Sridhar, Zhangyang Wang, Aditya Akella and Souvik Kundu. 2026. Improving the Throughput of Diffusion-based Large Language Models via a Training-Free Confidence-Aware Calibration. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{shen2026improving,
  title = {{Improving the Throughput of Diffusion-based Large Language Models via a Training-Free Confidence-Aware Calibration}},
  author = {Jucheng Shen, Gaurav Sarkar, Yeonju Ro, Sharath Nittur Sridhar, Zhangyang Wang, Aditya Akella and Souvik Kundu},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.478},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.478}
}
```

- [← previous Improving Romanian LLM Pretraining Data using Diversity and Quality Filtering](https://md.research.elamur.ai/free/en/papers/negoita-2026-improving-romanian-llm-pretraining-data/)
- [next → In-Context Representation Hijacking](https://md.research.elamur.ai/free/en/papers/yona-2026-context-representation-hijacking/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/7/)

## References

10 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jacob Austin, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. 2021. [Structured denoising diffusion models in discrete state-spaces](https://md.research.elamur.ai/free/en/papers/austin-2021-structured-denoising-diffusion-models-discrete/). Preprint, arXiv:2107.03006.
2. Andrew Campbell, Joe Benton, Valentin De Bortoli, Tom Rainforth, George Deligiannidis, and Arnaud Doucet. 2022. A continuous time framework for discrete denoising models. Preprint, arXiv:2205.14987. — [arXiv:2205.14987](https://arxiv.org/abs/2205.14987) [find](https://scholar.google.com/scholar?q=Andrew+Campbell%2C+Joe+Benton%2C+Valentin+De+Bortoli%2C+Tom+Rainforth%2C+George+Deligiannidis%2C+and+Arnaud+Doucet%2E+2022%2E+A+continuous+time+framework+for+discrete+denoising+models%2E+Preprint%2C+arXiv%3A2205%2E14987%2E+%E2%80%94)
3. Angela Fan, Mike Lewis, and Yann Dauphin. 2018. [Hierarchical neural story generation](https://md.research.elamur.ai/free/en/papers/fan-2018-hierarchical-neural-story-generation/). arXiv preprint arXiv:1805.04833.
4. Ari Holtzman, Jan Buys, Maxwell Forbes, Antoine Bosselut, David Golub, and Yejin Choi. 2018. [Learning to write with cooperative discriminators](https://md.research.elamur.ai/free/en/papers/holtzman-2018-learning-write-cooperative-discriminators/). arXiv preprint arXiv:1805.06087.
5. Pengxiang Li, Yefan Zhou, Dilxat Muhtar, Lu Yin, Shilin Yan, Li Shen, Yi Liang, Soroush Vosoughi, and Shiwei Liu. 2025. Diffusion language models know the answer before decoding. Preprint, arXiv:2508.19982. — [arXiv:2508.19982](https://arxiv.org/abs/2508.19982) [find](https://scholar.google.com/scholar?q=Pengxiang+Li%2C+Yefan+Zhou%2C+Dilxat+Muhtar%2C+Lu+Yin%2C+Shilin+Yan%2C+Li+Shen%2C+Yi+Liang%2C+Soroush+Vosoughi%2C+and+Shiwei+Liu%2E+2025%2E+Diffusion+language+models+know+the+answer+before+decoding%2E+Preprint%2C+arXiv%3A2508%2E)
6. Guanxi Lu, Hao Mark Chen, Yuto Karashima, Zhican Wang, Daichi Fujiki, and Hongxiang Fan. 2026. Adablock-dllm: Semantic-aware diffusion llm inference via adaptive block size. Preprint, arXiv:2509.26432. — [arXiv:2509.26432](https://arxiv.org/abs/2509.26432) [find](https://scholar.google.com/scholar?q=Guanxi+Lu%2C+Hao+Mark+Chen%2C+Yuto+Karashima%2C+Zhican+Wang%2C+Daichi+Fujiki%2C+and+Hongxiang+Fan%2E+2026%2E+Adablock%2Ddllm%3A+Semantic%2Daware+diffusion+llm+inference+via+adaptive+block+size%2E+Preprint%2C+arXiv%3A2509%2E26432)
7. Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. 2025. [Large language diffusion models](https://md.research.elamur.ai/free/en/papers/nie-2025-large-language-diffusion-models/). Preprint, arXiv:2502.09992.
8. Chengyue Wu, Hao Zhang, Shuchen Xue, Zhijian Liu, Shizhe Diao, Ligeng Zhu, Ping Luo, Song Han, and Enze Xie. 2025. Fast-dllm: Training-free acceleration of diffusion llm by enabling kv cache and parallel decoding. Preprint, arXiv:2505.22618. — [arXiv:2505.22618](https://arxiv.org/abs/2505.22618) [find](https://scholar.google.com/scholar?q=Chengyue+Wu%2C+Hao+Zhang%2C+Shuchen+Xue%2C+Zhijian+Liu%2C+Shizhe+Diao%2C+Ligeng+Zhu%2C+Ping+Luo%2C+Song+Han%2C+and+Enze+Xie%2E+2025%2E+Fast%2Ddllm%3A+Training%2Dfree+acceleration+of+diffusion+llm+by+enabling+kv+cache+and+paral)
9. Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong. 2025a. Dream 7b: Diffusion large language models. Preprint, arXiv:2508.15487. [find](https://scholar.google.com/scholar?q=Jiacheng+Ye%2C+Zhihui+Xie%2C+Lin+Zheng%2C+Jiahui+Gao%2C+Zirui+Wu%2C+Xin+Jiang%2C+Zhenguo+Li%2C+and+Lingpeng+Kong%2E+2025a%2E+Dream+7b%3A+Diffusion+large+language+models%2E+Preprint%2C+arXiv%3A2508%2E15487%2E)
10. Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong. 2025b. Dream 7b: Diffusion large language models. arXiv preprint arXiv:2508.15487. [find](https://scholar.google.com/scholar?q=Jiacheng+Ye%2C+Zhihui+Xie%2C+Lin+Zheng%2C+Jiahui+Gao%2C+Zirui+Wu%2C+Xin+Jiang%2C+Zhenguo+Li%2C+and+Lingpeng+Kong%2E+2025b%2E+Dream+7b%3A+Diffusion+large+language+models%2E+arXiv+preprint+arXiv%3A2508%2E15487%2E)

*Updated 31 October 2026*