---
title: "Diffusion with Truncated Blocks — paper by Zhou et al. (2026)"
description: "The paper examines a weakness of diffusion-based large language models (dLLMs), which produce text by repeatedly filling in… 31 references."
authors: ["Yuyan Zhou", "Weiyu Chen", "James Kwok"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.212/"
doi: "10.18653/v1/2026.findings-acl.212"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhou-2026-diffusion-truncated-blocks-fast-high/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhou-2026-diffusion-truncated-blocks-fast-high/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/30/) / Diffusion with Truncated Blocks: Fast and High-Quality Text Generation using Truncated Block Generation

# Diffusion with Truncated Blocks: Fast and High-Quality Text Generation using Truncated Block Generation

**Authors:** [Yuyan Zhou](https://md.research.elamur.ai/free/en/authors/yuyan-zhou/), Weiyu Chen and [James Kwok](https://md.research.elamur.ai/free/en/authors/james-tin-yau-kwok/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.212/) · [PDF](https://aclanthology.org/2026.findings-acl.212.pdf) · [DOI 10.18653/v1/2026.findings-acl.212](https://doi.org/10.18653/v1/2026.findings-acl.212) · [OpenAlex](https://openalex.org/W7166850100)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper examines a weakness of diffusion-based large language models (dLLMs), which produce text by repeatedly filling in masked positions rather than writing left to right. The authors find that when many [MASK] tokens are appended to a prompt, much of the model's attention goes to these placeholders, which carry little information, and less goes to the prompt. They call this "attention dilution." They support it with theoretical arguments and attention measurements, and show it is strongest in models trained with token-level noise (Dream). Models trained with sequence-level noise (LLaDA) are affected less. Their fix is Truncated Block Generation (TBG), a sampling procedure that works in rounds. Each round appends and denoises only a short block of masks. The resolved block is then cut back and reused as context, so the model always sees a dense, informative context. TBG improves accuracy over standard block generation on code and math benchmarks. The authors also report faster inference and support for outputs of flexible length. Because it is a sampling-time change, it can be applied to existing dLLMs without retraining.

## Key points
- **Attention dilution in Dream:** The first mask token's attention to the prompt word "You" falls from 2.1×10⁻⁵ with 64 appended masks to 5.3×10⁻⁶ with 1024. Contextual attention declines steadily on MBPP samples as more masks are added.
- **Explanation:** In both Dream and LLaDA, mask tokens copy a prompt token as their query in later layers. Sequence-level noise in LLaDA trains every mask to carry useful information, which weakens the dilution effect.
- **TBG vs. block generation (Table 5):**

  | Benchmark | Block generation | TBG |
  |---|---|---|
  | HumanEval | 48.1 | 52.4 |
  | MBPP | 58.0 | 60.4 |
  | Math | 43.6 | 44.0 |
  | GSM8K | 66.6 | 78.9 |

- **Dream-7B results:** MBPP rises from 58.0 to 60.9 and HumanEval from 43.9 to 53.1 with 3–4 TBG blocks.
- **Ablation and limitations:** A truncated length of 30 worked best on both validation and test sets. The method depends on hyperparameters (generation threshold, truncation length), and the authors leave automatic tuning of these to future work.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Hong Kong University of Science and Technology](https://md.research.elamur.ai/free/en/universities/hong-kong-university-of-science-and-technology/) — University, Hong Kong, China · 170 works · 26,341 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

🧭 Topics (1): [✍️ Text generation](https://md.research.elamur.ai/free/en/topics/text-generation/) · 214 works

## Cite

Cite as: Yuyan Zhou, Weiyu Chen and James Kwok. 2026. Diffusion with Truncated Blocks: Fast and High-Quality Text Generation using Truncated Block Generation. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhou2026diffusion,
  title = {{Diffusion with Truncated Blocks: Fast and High-Quality Text Generation using Truncated Block Generation}},
  author = {Yuyan Zhou, Weiyu Chen and James Kwok},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.212},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.212}
}
```

- [← previous Differentially Private De-identification of Dutch Clinical Notes: A Comparative Evaluation](https://md.research.elamur.ai/free/en/papers/miranda-2026-differentially-private-de-identification-dutch/)
- [next → Digital Preservation of Aromanian Through Knowledge Management and Automatic Speech Recognition Evaluation](https://md.research.elamur.ai/free/en/papers/pendevska-2026-digital-preservation-aromanian-through-knowledge/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/30/)

## References

31 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). Techreport, arXiv:2303.08774.
2. Marianne Arriola, Aaron Gokaslan, Justin T Chiu, Zhihan Yang, Zhixuan Qi, Jiaqi Han, Subham Sekhar Sahoo, and Volodymyr Kuleshov. 2025. Block diffusion: Interpolating between autoregressive and diffusion language models. Techreport, arXiv:2503.09573. — [arXiv:2503.09573](https://arxiv.org/abs/2503.09573) [find](https://scholar.google.com/scholar?q=Marianne+Arriola%2C+Aaron+Gokaslan%2C+Justin+T+Chiu%2C+Zhihan+Yang%2C+Zhixuan+Qi%2C+Jiaqi+Han%2C+Subham+Sekhar+Sahoo%2C+and+Volodymyr+Kuleshov%2E+2025%2E+Block+diffusion%3A+Interpolating+between+autoregressive+and+diffus)
3. Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. 2021a. [Structured denoising diffusion models in discrete state-spaces](https://md.research.elamur.ai/free/en/papers/austin-2021-structured-denoising-diffusion-models-discrete/). NeurIPS.
4. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Charles Sutton. 2021b. [Program synthesis with large language models](https://md.research.elamur.ai/free/en/papers/austin-2021-program-synthesis-large-language-models/). Techreport, arXiv:2108.07732.
5. Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T Freeman. 2022. Maskgit: Masked generative image transformer. In CVPR. [find](https://scholar.google.com/scholar?q=Huiwen+Chang%2C+Han+Zhang%2C+Lu+Jiang%2C+Ce+Liu%2C+and+William+T+Freeman%2E+2022%2E+Maskgit%3A+Masked+generative+image+transformer%2E+In+CVPR%2E)
6. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, and 39 others. 202… — [Evaluating Large Language Models Trained on Code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
7. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). Techreport, arXiv:2110.14168.
8. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). Techreport, pages arXiv–2407.
9. Xiaochuang Han, Sachin Kumar, and Yulia Tsvetkov. 2022. Ssd-lm: Semi-autoregressive simplex-based diffusion language model for text generation and modular control. Techreport, arXiv:2210.17432. [find](https://scholar.google.com/scholar?q=Xiaochuang+Han%2C+Sachin+Kumar%2C+and+Yulia+Tsvetkov%2E+2022%2E+Ssd%2Dlm%3A+Semi%2Dautoregressive+simplex%2Dbased+diffusion+language+model+for+text+generation+and+modular+control%2E+Techreport%2C+arXiv%3A2210%2E17432%2E)
10. Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. NeurIPS. [find](https://scholar.google.com/scholar?q=Jonathan+Ho%2C+Ajay+Jain%2C+and+Pieter+Abbeel%2E+2020%2E+Denoising+diffusion+probabilistic+models%2E+NeurIPS%2E)
11. Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. 2022. Elucidating the design space of diffusion-based generative models. NeurIPS. [find](https://scholar.google.com/scholar?q=Tero+Karras%2C+Miika+Aittala%2C+Timo+Aila%2C+and+Samuli+Laine%2E+2022%2E+Elucidating+the+design+space+of+diffusion%2Dbased+generative+models%2E+NeurIPS%2E)
12. Samar Khanna, Siddhant Kharbanda, Shufan Li, Harshit Varma, Eric Wang, Sawyer Birnbaum, Ziyang Luo, Yanis Miraoui, Akash Palrecha, Stefano Ermon, and 1 others. 2025. Mercury: Ultra-fast language models based on diffusion. Techreport, arXiv:2506.17298. [find](https://scholar.google.com/scholar?q=Samar+Khanna%2C+Siddhant+Kharbanda%2C+Shufan+Li%2C+Harshit+Varma%2C+Eric+Wang%2C+Sawyer+Birnbaum%2C+Ziyang+Luo%2C+Yanis+Miraoui%2C+Akash+Palrecha%2C+Stefano+Ermon%2C+and+1+others%2E+2025%2E+Mercury%3A+Ultra%2Dfast+language+model)
13. Jaeyeon Kim, Kulin Shah, Vasilis Kontonis, Sham Kakade, and Sitan Chen. 2025. Train for the worst, plan for the best: Understanding token ordering in masked diffusions. Techreport, arXiv:2502.06768. — [arXiv:2502.06768](https://arxiv.org/abs/2502.06768) [find](https://scholar.google.com/scholar?q=Jaeyeon+Kim%2C+Kulin+Shah%2C+Vasilis+Kontonis%2C+Sham+Kakade%2C+and+Sitan+Chen%2E+2025%2E+Train+for+the+worst%2C+plan+for+the+best%3A+Understanding+token+ordering+in+masked+diffusions%2E+Techreport%2C+arXiv%3A2502%2E06768%2E+%E2%80%94)
14. Xiang Li, John Thickstun, Ishaan Gulrajani, Percy S Liang, and Tatsunori B Hashimoto. 2022. [Diffusion-lm improves controllable text generation](https://md.research.elamur.ai/free/en/papers/li-2022-diffusion-lm-improves-controllable-text/). NeurIPS.
15. Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, and 1 others. 2024a. Deepseek-v3 technical report. Techreport, arXiv:2412.19437. [find](https://scholar.google.com/scholar?q=Aixin+Liu%2C+Bei+Feng%2C+Bing+Xue%2C+Bingxuan+Wang%2C+Bochao+Wu%2C+Chengda+Lu%2C+Chenggang+Zhao%2C+Chengqi+Deng%2C+Chenyu+Zhang%2C+Chong+Ruan%2C+and+1+others%2E+2024a%2E+Deepseek%2Dv3+technical+report%2E+Techreport%2C+arXiv%3A2412%2E1)
16. Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024b. [Lost in the middle: How language models use long contexts](https://md.research.elamur.ai/free/en/papers/liu-2023-lost-middle-language-models-use/). Transactions of the association for computational linguistics, 12:157–173.
17. Aaron Lou, Chenlin Meng, and Stefano Ermon. 2023. [Discrete diffusion modeling by estimating the ratios of the data distribution](https://md.research.elamur.ai/free/en/papers/lou-2023-discrete-diffusion-modeling-estimating-ratios/). Techreport, arXiv:2310.16834.
18. Rabeeh Karimi Mahabadi, Hamish Ivison, Jaesung Tae, James Henderson, Iz Beltagy, Matthew E Peters, and Arman Cohan. 2023. Tess: Text-to-text self-conditioned simplex diffusion. Techreport, arXiv:2305.08379. [find](https://scholar.google.com/scholar?q=Rabeeh+Karimi+Mahabadi%2C+Hamish+Ivison%2C+Jaesung+Tae%2C+James+Henderson%2C+Iz+Beltagy%2C+Matthew+E+Peters%2C+and+Arman+Cohan%2E+2023%2E+Tess%3A+Text%2Dto%2Dtext+self%2Dconditioned+simplex+diffusion%2E+Techreport%2C+arXiv%3A2305%2E)
19. Chenlin Meng, Kristy Choi, Jiaming Song, and Stefano Ermon. 2022. Concrete score matching: Generalized score matching for discrete data. NeurIPS. [find](https://scholar.google.com/scholar?q=Chenlin+Meng%2C+Kristy+Choi%2C+Jiaming+Song%2C+and+Stefano+Ermon%2E+2022%2E+Concrete+score+matching%3A+Generalized+score+matching+for+discrete+data%2E+NeurIPS%2E)
20. Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Le, Ed Chi, Denny Zhou, and 1 others. 2023. [Challenging big-bench tasks and whether chain-of-thought can solve them](https://md.research.elamur.ai/free/en/papers/suzgun-2023-challenging-big-bench-tasks-whether/). In Findings of the Association for Computational Linguistics: AC…
21. Bingning Wang, Haizhou Zhao, Huozhi Zhou, Liang Song, Mingyu Xu, Wei Cheng, Xiangrong Zeng, Yupeng Zhang, Yuqi Huo, Zecheng Wang, and 1 others. 2025a. [Baichuan-m1: Pushing the medical capability of large language models](https://md.research.elamur.ai/free/en/papers/wang-2025-baichuan-m1-pushing-medical/). Techreport, arXiv:2502.12671.
22. Pinzheng Wang, Zecheng Tang, Keyan Zhou, Juntao Li, Qiaoming Zhu, and Min Zhang. 2025b. Revealing and mitigating over-attention in knowledge editing. Techreport, arXiv:2502.14838. — [arXiv:2502.14838](https://arxiv.org/abs/2502.14838) [find](https://scholar.google.com/scholar?q=Pinzheng+Wang%2C+Zecheng+Tang%2C+Keyan+Zhou%2C+Juntao+Li%2C+Qiaoming+Zhu%2C+and+Min+Zhang%2E+2025b%2E+Revealing+and+mitigating+over%2Dattention+in+knowledge+editing%2E+Techreport%2C+arXiv%3A2502%2E14838%2E+%E2%80%94+%5BarXiv%3A2502%2E14838%5D)
23. Chengyue Wu, Hao Zhang, Shuchen Xue, Zhijian Liu, Shizhe Diao, Ligeng Zhu, Ping Luo, Song Han, and Enze Xie. 2025. Fast-dllm: Training-free acceleration of diffusion llm by enabling kv cache and parallel decoding. Techreport, arXiv:2505.22618. — [arXiv:2505.22618](https://arxiv.org/abs/2505.22618) [find](https://scholar.google.com/scholar?q=Chengyue+Wu%2C+Hao+Zhang%2C+Shuchen+Xue%2C+Zhijian+Liu%2C+Shizhe+Diao%2C+Ligeng+Zhu%2C+Ping+Luo%2C+Song+Han%2C+and+Enze+Xie%2E+2025%2E+Fast%2Ddllm%3A+Training%2Dfree+acceleration+of+diffusion+llm+by+enabling+kv+cache+and+paral)
24. Zhihui Xie, Jiacheng Ye, Lin Zheng, Jiahui Gao, Jingwei Dong, Zirui Wu, Xueliang Zhao, Shansan Gong, Xin Jiang, Zhenguo Li, and 1 others. 2025. Dreamcoder 7b: An open diffusion language model for code. Techreport, arXiv:2509.01142. — [arXiv:2509.01142](https://arxiv.org/abs/2509.01142) [find](https://scholar.google.com/scholar?q=Zhihui+Xie%2C+Jiacheng+Ye%2C+Lin+Zheng%2C+Jiahui+Gao%2C+Jingwei+Dong%2C+Zirui+Wu%2C+Xueliang+Zhao%2C+Shansan+Gong%2C+Xin+Jiang%2C+Zhenguo+Li%2C+and+1+others%2E+2025%2E+Dreamcoder+7b%3A+An+open+diffusion+language+model+for+code)
25. Jiacheng Ye, Zhihui Xie, Lin Zheng, Jiahui Gao, Zirui Wu, Xin Jiang, Zhenguo Li, and Lingpeng Kong. 2025. [Dream 7b: Diffusion large language models](https://md.research.elamur.ai/free/en/papers/ye-2025-dream-7b-diffusion-large-language/). Techreport, arXiv:2508.15487.
26. Yuyan Zhou, Kai Syun Hou, Weiyu Chen, and James Kwok. 2026. Attention-based sampler for diffusion language models. Techreport, arXiv:2604.08564. — [arXiv:2604.08564](https://arxiv.org/abs/2604.08564) [find](https://scholar.google.com/scholar?q=Yuyan+Zhou%2C+Kai+Syun+Hou%2C+Weiyu+Chen%2C+and+James+Kwok%2E+2026%2E+Attention%2Dbased+sampler+for+diffusion+language+models%2E+Techreport%2C+arXiv%3A2604%2E08564%2E+%E2%80%94+%5BarXiv%3A2604%2E08564%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2604%2E08564%29)
27. Yuyan Zhou, Liang Song, Bingning Wang, and Weipeng Chen. 2024. Metagpt: Merging large language models using model exclusive task arithmetic. In EMNLP. [find](https://scholar.google.com/scholar?q=Yuyan+Zhou%2C+Liang+Song%2C+Bingning+Wang%2C+and+Weipeng+Chen%2E+2024%2E+Metagpt%3A+Merging+large+language+models+using+model+exclusive+task+arithmetic%2E+In+EMNLP%2E)
28. Fengqi Zhu, Rongzhen Wang, Shen Nie, Xiaolu Zhang, Chunwei Wu, Jun Hu, Jun Zhou, Jianfei Chen, Yankai Lin, Ji-Rong Wen, and 1 others. 2025. [Llada 1.5: Variance-reduced preference optimization for large language diffusion models](https://md.research.elamur.ai/free/en/papers/zhu-2026-llada-1-5-variance-reduced/). Techreport, arXiv:2505.19223.
29. Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. 2025. [Large language diffusion models](https://md.research.elamur.ai/free/en/papers/nie-2025-large-language-diffusion-models/). Techreport, arXiv:2502.09992.
30. Grefenstette Saxton and Kohli Hill. 2019. [Analysing mathematical reasoning abilities of neural models](https://md.research.elamur.ai/free/en/papers/saxton-2019-analysing-mathematical-reasoning-abilities-neural/). arXiv:1904.01557.
31. Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. 2015. [Deep unsupervised learning using nonequilibrium thermodynamics](https://md.research.elamur.ai/free/en/papers/sohl-dickstein-2015-deep-unsupervised-learning-using-nonequilibrium/). In ICML.

*Updated 2 November 2026*