---
title: "RACC — paper by Zeng et al. (2026) — summary and ideas"
description: "The authors propose RACC, a training-free decoding add-on. 1.00 points. 32 references. Among them: 0."
authors: ["Qinglin Zeng", "Jusheng Zhang", "Jing Yang", "Ningyuan Liu", "Keze Wang"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.1138/"
doi: "10.18653/v1/2026.findings-acl.1138"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zeng-2026-racc-regret-aware-confidence-calibration/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zeng-2026-racc-regret-aware-confidence-calibration/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/46/) / RACC: Regret-Aware Confidence Calibration for Consistent Masked Discrete Diffusion Decoding

# RACC: Regret-Aware Confidence Calibration for Consistent Masked Discrete Diffusion Decoding

**Authors:** Qinglin Zeng, Jusheng Zhang, [Jing Yang](https://md.research.elamur.ai/free/en/authors/jing-yang/), Ningyuan Liu and [Keze Wang](https://md.research.elamur.ai/free/en/authors/keze-wang/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.1138/) · [PDF](https://aclanthology.org/2026.findings-acl.1138.pdf) · [DOI 10.18653/v1/2026.findings-acl.1138](https://doi.org/10.18653/v1/2026.findings-acl.1138) · [OpenAlex](https://openalex.org/W7166799922)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Masked Discrete Diffusion Models (MDMs) generate text by gradually unmasking tokens, and their bidirectional attention re-scores every position at each step. The authors argue that standard samplers, such as confidence-based Top-K filling and PC-Sampler, waste this property. These samplers use only the current step's probabilities and ignore how a position's confidence evolves over time. The authors propose RACC, a training-free decoding add-on. For each position, RACC keeps a momentum anchor: an exponential moving average of the top candidate's probability. When the current probability falls clearly below this anchor (beyond a tolerance ϵ), RACC treats the drop as a "regret" signal. It then penalizes that candidate with a cosine-scheduled strength, so unstable tokens are deferred during Top-K selection. RACC reuses logits that each forward pass already computes, so it adds no inference cost, unlike Re-MDM remasking or LookUM lookahead. On reasoning and code benchmarks (e.g., HumanEval, GSM8K), it improves accuracy and consistency, and it also strengthens Temporal Self-Consistency Voting (TSCV). This offers a cheap route to more reliable diffusion LLM reasoning.

## Key points
- On LLaDA-1.5 (HumanEval), RACC with β = 0.8 reaches 45.73% Pass@1. The local-only variant (β = 0) reaches 43.9%, and the reported standard baseline reaches 23.7%.
- Paired with TSCV, RACC raises vote accuracy on GSM8K (69.60→70.20), SVAMP (83.33→84.67) and MATH (28.60→29.40). Final-answer accuracy gains range from 0.53 to 1.00 points.
- In the sensitivity analysis, a moderate penalty range [1.0, 5.0] works best, while an aggressive range [3.0, 10.0] drops Pass@1 to 45.12%. Tolerance values of ϵ ≈ 0.08–0.10 beat ϵ ≈ 0.
- The authors note three limitations. RACC needs the backbone to actually show confidence drops. Its hyperparameters were tuned empirically. Regret detected in the final steps leaves little budget for re-planning, and the authors suggest combining RACC with re-masking to address this.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Sun Yat-sen University](https://md.research.elamur.ai/free/en/universities/sun-yat-sen-university/) — University, Guangzhou, China · 72 works · 9,502 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

## Cite

Cite as: Qinglin Zeng, Jusheng Zhang, Jing Yang, Ningyuan Liu and Keze Wang. 2026. RACC: Regret-Aware Confidence Calibration for Consistent Masked Discrete Diffusion Decoding. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zeng2026racc,
  title = {{RACC: Regret-Aware Confidence Calibration for Consistent Masked Discrete Diffusion Decoding}},
  author = {Qinglin Zeng, Jusheng Zhang, Jing Yang, Ningyuan Liu and Keze Wang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1138},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1138}
}
```

- [← previous RABIT: Rationale-Based Distillation Towards Interpretable Automatic Speaking Assessment via a Small Language Model](https://md.research.elamur.ai/free/en/papers/yan-2026-rabit-rationale-based-distillation-interpretable/)
- [next → RacismoBR: A Manually Annotated Dataset for Racist Discourse Detection in Brazilian Portuguese](https://md.research.elamur.ai/free/en/papers/vaz-2026-racismobr-manually-annotated-dataset-racist/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/46/)

## References

32 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774.
2. Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne Van Den Berg. 2021a. [Structured denoising diffusion models in discrete state-spaces](https://md.research.elamur.ai/free/en/papers/austin-2021-structured-denoising-diffusion-models-discrete/). Advances in neural information processing systems, 34:17981–17993.
3. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and 1 others. 2021b. [Program synthesis with large language models](https://md.research.elamur.ai/free/en/papers/austin-2021-program-synthesis-large-language-models/). arXiv preprint arXiv:2108.07732.
4. Kecheng Chen, Ziru Liu, Xijia Tao, Hui Liu, Xinyu Fu, Suiyun Zhang, Dandan Tu, Lingpeng Kong, Rui Liu, and Haoliang Li. 2025. Beyond confidence: Adaptive and coherent decoding for diffusion language models. arXiv preprint arXiv:2512.02044. — [arXiv:2512.02044](https://arxiv.org/abs/2512.02044) [find](https://scholar.google.com/scholar?q=Kecheng+Chen%2C+Ziru+Liu%2C+Xijia+Tao%2C+Hui+Liu%2C+Xinyu+Fu%2C+Suiyun+Zhang%2C+Dandan+Tu%2C+Lingpeng+Kong%2C+Rui+Liu%2C+and+Haoliang+Li%2E+2025%2E+Beyond+confidence%3A+Adaptive+and+coherent+decoding+for+diffusion+language+m)
5. Mark Chen. 2021. [Evaluating large language models trained on code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/). arXiv preprint arXiv:2107.03374.
6. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, and 1 others. 2021. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168.
7. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, and 1 others. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783.
8. Zhengcong Fei, Mingyuan Fan, Changqian Yu, De-bang Li, Youqiang Zhang, and Junshi Huang. 2024. Dimba: Transformer-mamba diffusion models. arXiv preprint arXiv:2406.01159. — [arXiv:2406.01159](https://arxiv.org/abs/2406.01159) [find](https://scholar.google.com/scholar?q=Zhengcong+Fei%2C+Mingyuan+Fan%2C+Changqian+Yu%2C+De%2Dbang+Li%2C+Youqiang+Zhang%2C+and+Junshi+Huang%2E+2024%2E+Dimba%3A+Transformer%2Dmamba+diffusion+models%2E+arXiv+preprint+arXiv%3A2406%2E01159%2E+%E2%80%94+%5BarXiv%3A2406%2E01159%5D%28https%3A%2F%2F)
9. Yichao Fu, Xuewei Wang, Yuandong Tian, and Jiawei Zhao. 2025. [Deep think with confidence](https://md.research.elamur.ai/free/en/papers/fu-2025-deep-think-confidence/). arXiv preprint arXiv:2508.15260.
10. Marjan Ghazvininejad, Omer Levy, Yinhan Liu, and Luke Zettlemoyer. 2019. [Mask-predict: Parallel decoding of conditional masked language models](https://md.research.elamur.ai/free/en/papers/ghazvininejad-2019-mask-predict-parallel-decoding-conditional/). arXiv preprint arXiv:1904.09324.
11. Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, and Weizhu Chen. 2023. [Critic: Large language models can self-correct with tool-interactive critiquing](https://md.research.elamur.ai/free/en/papers/gou-2023-critic-large-language-models-self/). arXiv preprint arXiv:2305.11738.
12. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. [Measuring mathematical problem solving with the math dataset](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/). arXiv preprint arXiv:2103.03874.
13. Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. 2023. [Large language models cannot self-correct reasoning yet](https://md.research.elamur.ai/free/en/papers/huang-2023-large-language-models-cannot-self/). arXiv preprint arXiv:2310.01798.
14. Pengcheng Huang, Shuhao Liu, Zhenghao Liu, Yukun Yan, Shuo Wang, Zulong Chen, and Tong Xiao. 2025. Pc-sampler: Position-aware calibration of decoding bias in masked diffusion models. arXiv preprint arXiv:2508.13021. — [arXiv:2508.13021](https://arxiv.org/abs/2508.13021) [find](https://scholar.google.com/scholar?q=Pengcheng+Huang%2C+Shuhao+Liu%2C+Zhenghao+Liu%2C+Yukun+Yan%2C+Shuo+Wang%2C+Zulong+Chen%2C+and+Tong+Xiao%2E+2025%2E+Pc%2Dsampler%3A+Position%2Daware+calibration+of+decoding+bias+in+masked+diffusion+models%2E+arXiv+preprint+ar)
15. Adam Tauman Kalai and Santosh S Vempala. 2024. Calibrated language models must hallucinate. In Proceedings of the 56th Annual ACM Symposium on Theory of Computing, pages 160–171. [find](https://scholar.google.com/scholar?q=Adam+Tauman+Kalai+and+Santosh+S+Vempala%2E+2024%2E+Calibrated+language+models+must+hallucinate%2E+In+Proceedings+of+the+56th+Annual+ACM+Symposium+on+Theory+of+Computing%2C+pages+160%E2%80%93171%2E)
16. Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. 2023. [Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation](https://md.research.elamur.ai/free/en/papers/kuhn-2023-semantic-uncertainty-linguistic-invariances-uncertainty/). arXiv preprint arXiv:2302.09664.
17. Sanghyun Lee, Seungryong Kim, Jongho Park, and Dongmin Park. 2025. Lookahead unmasking elicits accurate decoding in diffusion language models. arXiv preprint arXiv:2511.05563. and 1 others. 2022. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). Advances in neural information pr… — [arXiv:2511.05563](https://arxiv.org/abs/2511.05563)
18. Aaron Lou, Chenlin Meng, and Stefano Ermon. 2023. [Discrete diffusion modeling by estimating the ratios of the data distribution](https://md.research.elamur.ai/free/en/papers/lou-2023-discrete-diffusion-modeling-estimating-ratios/). arXiv preprint arXiv:2310.16834.
19. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems, 36:11809–11822. [find](https://scholar.google.com/scholar?q=Shunyu+Yao%2C+Dian+Yu%2C+Jeffrey+Zhao%2C+Izhak+Shafran%2C+Tom+Griffiths%2C+Yuan+Cao%2C+and+Karthik+Narasimhan%2E+2023%2E+Tree+of+thoughts%3A+Deliberate+problem+solving+with+large+language+models%2E+Advances+in+neural+inf)
20. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, and 1 others. 2023. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36:46534–46594. [find](https://scholar.google.com/scholar?q=Aman+Madaan%2C+Niket+Tandon%2C+Prakhar+Gupta%2C+Skyler+Hallinan%2C+Luyu+Gao%2C+Sarah+Wiegreffe%2C+Uri+Alon%2C+Nouha+Dziri%2C+Shrimai+Prabhumoye%2C+Yiming+Yang%2C+and+1+others%2E+2023%2E+Self%2Drefine%3A+Iterative+refinement+with)
21. Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022. [Rethinking the role of demonstrations: What makes in-context learning work?](https://md.research.elamur.ai/free/en/papers/min-2022-rethinking-role-demonstrations-what-makes/) arXiv preprint arXiv:2202.12837.
22. Shen Nie, Fengqi Zhu, Chao Du, Tianyu Pang, Qian Liu, Guangtao Zeng, Min Lin, and Chongxuan Li. 2024. Scaling up masked diffusion models on text. arXiv preprint arXiv:2410.18514. — [arXiv:2410.18514](https://arxiv.org/abs/2410.18514) [find](https://scholar.google.com/scholar?q=Shen+Nie%2C+Fengqi+Zhu%2C+Chao+Du%2C+Tianyu+Pang%2C+Qian+Liu%2C+Guangtao+Zeng%2C+Min+Lin%2C+and+Chongxuan+Li%2E+2024%2E+Scaling+up+masked+diffusion+models+on+text%2E+arXiv+preprint+arXiv%3A2410%2E18514%2E+%E2%80%94+%5BarXiv%3A2410%2E18514%5D%28)
23. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2022. [React: Synergizing reasoning and acting in language models](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/). In The eleventh international conference on learning representations.
24. Qinglin Zeng, Jing Yang, and Keze Wang. 2025. Reflective confidence: Correcting reasoning flaws via online self-correction. arXiv preprint arXiv:2512.18605. — [arXiv:2512.18605](https://arxiv.org/abs/2512.18605) [find](https://scholar.google.com/scholar?q=Qinglin+Zeng%2C+Jing+Yang%2C+and+Keze+Wang%2E+2025%2E+Reflective+confidence%3A+Correcting+reasoning+flaws+via+online+self%2Dcorrection%2E+arXiv+preprint+arXiv%3A2512%2E18605%2E+%E2%80%94+%5BarXiv%3A2512%2E18605%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F)
25. Fengqi Zhu, Rongzhen Wang, Shen Nie, Xiaolu Zhang, Chunwei Wu, Jun Hu, Jun Zhou, Jianfei Chen, Yankai Lin, Ji-Rong Wen, and 1 others. 2025. [Llada 1.5: Variance-reduced preference optimization for large language diffusion models](https://md.research.elamur.ai/free/en/papers/zhu-2026-llada-1-5-variance-reduced/). arXiv preprint arXiv:2505.19223.
26. Shen Nie, Fengqi Zhu, Zebin You, Xiaolu Zhang, Jingyang Ou, Jun Hu, Jun Zhou, Yankai Lin, Ji-Rong Wen, and Chongxuan Li. 2025. [Large language diffusion models](https://md.research.elamur.ai/free/en/papers/nie-2025-large-language-diffusion-models/). arXiv preprint arXiv:2502.09992.
27. Subham Sahoo, Marianne Arriola, Yair Schiff, Aaron Gokaslan, Edgar Marroquin, Justin Chiu, Alexander Rush, and Volodymyr Kuleshov. 2024. [Simple and effective masked diffusion language models](https://md.research.elamur.ai/free/en/papers/sahoo-2024-simple-effective-masked-diffusion-language/). Advances in Neural Information Processing Systems, 37:130136–130184.
28. Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. [Reflexion: Language agents with verbal reinforcement learning](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/). Advances in Neural Information Processing Systems, 36:8634–8652.
29. Guanghan Wang, Yair Schiff, Subham Sekhar Sahoo, and Volodymyr Kuleshov. 2025a. Remasking discrete diffusion models with inference-time scaling. arXiv preprint arXiv:2503.00307. — [arXiv:2503.00307](https://arxiv.org/abs/2503.00307) [find](https://scholar.google.com/scholar?q=Guanghan+Wang%2C+Yair+Schiff%2C+Subham+Sekhar+Sahoo%2C+and+Volodymyr+Kuleshov%2E+2025a%2E+Remasking+discrete+diffusion+models+with+inference%2Dtime+scaling%2E+arXiv+preprint+arXiv%3A2503%2E00307%2E+%E2%80%94+%5BarXiv%3A2503%2E00307%5D%28h)
30. Wen Wang, Bozhen Fang, Chenchen Jing, Yongliang Shen, Yangyi Shen, Qiuyu Wang, Hao Ouyang, Hao Chen, and Chunhua Shen. 2025b. Time is a feature: Exploiting temporal dynamics in diffusion language models. arXiv preprint arXiv:2508.09138. — [arXiv:2508.09138](https://arxiv.org/abs/2508.09138) [find](https://scholar.google.com/scholar?q=Wen+Wang%2C+Bozhen+Fang%2C+Chenchen+Jing%2C+Yongliang+Shen%2C+Yangyi+Shen%2C+Qiuyu+Wang%2C+Hao+Ouyang%2C+Hao+Chen%2C+and+Chunhua+Shen%2E+2025b%2E+Time+is+a+feature%3A+Exploiting+temporal+dynamics+in+diffusion+language+mode)
31. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. [Self-consistency improves chain of thought reasoning in language models](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/). arXiv preprint arXiv:2203.11171.
32. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, Supplementary Material for RACC: Regret-Aware Confidence Calibration for Consistent Masked Discrete Diffusion Decoding [find](https://scholar.google.com/scholar?q=Jason+Wei%2C+Xuezhi+Wang%2C+Dale+Schuurmans%2C+Maarten+Bosma%2C+Fei+Xia%2C+Ed+Chi%2C+Quoc+V+Le%2C+Denny+Zhou%2C+Supplementary+Material+for+RACC%3A+Regret%2DAware+Confidence+Calibration+for+Consistent+Masked+Discrete+Diff)

*Updated 2 November 2026*