---
title: "Can Diffusion Model Achieve Better Performance — Tang 2023"
description: "Diffusion models for text embed discrete tokens into a continuous space and then denoise them."
authors: ["Zecheng Tang", "Pinzheng Wang", "Keyan Zhou", "Juntao Li", "Ziqiang Cao", "Min Zhang"]
year: 2023
venue: "Findings of the Association for Computational Linguistics: ACL 2023"
source: "https://aclanthology.org/2023.findings-acl.721/"
doi: "10.18653/v1/2023.findings-acl.721"
arxiv: "2305.04465"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/tang-2023-can-diffusion-model-achieve-better/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/tang-2023-can-diffusion-model-achieve-better/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2023](https://md.research.elamur.ai/free/papers/2023/) / Can Diffusion Model Achieve Better Performance in Text Generation ? Bridging the Gap between Training and Inference !

# Can Diffusion Model Achieve Better Performance in Text Generation ? Bridging the Gap between Training and Inference !

**Authors:** [Zecheng Tang](https://md.research.elamur.ai/free/authors/zecheng-tang/), Pinzheng Wang, Keyan Zhou, Juntao Li, [Ziqiang Cao](https://md.research.elamur.ai/free/authors/ziqiang-cao/), [Min Zhang](https://md.research.elamur.ai/free/authors/min-zhang/) · **Year:** 2023 · **Venue:** Findings of the Association for Computational Linguistics: ACL 2023 · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2023.findings-acl.721/) · [PDF](https://aclanthology.org/2023.findings-acl.721.pdf) · [DOI 10.18653/v1/2023.findings-acl.721](https://doi.org/10.18653/v1/2023.findings-acl.721) · [arXiv 2305.04465](https://arxiv.org/abs/2305.04465) · [OpenAlex](https://openalex.org/W4385570527)

**Citations:** 5 citations (openalex-snapshot · 2026-10-14) · [13](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Diffusion models for text embed discrete tokens into a continuous space and then denoise them. Their training and inference behave differently, and this hurts output quality. During training, each noisy state is computed directly from the clean data. At inference, the model must build on its own earlier predictions, a form of exposure bias. A second mismatch comes from speedups: uniformly skipping steps at inference produces a trajectory unlike the full one seen in training. The authors first ran a diagnostic study on story generation with ROC Stories, using a DiffuSeq-style model by Gong et al. They found that mixing self-predicted noise into training improves results. They also found that allocating more sampled steps to the early part of the reverse process beats uniform downsampling. From these findings they derived two techniques. Distance Penalty is a training-side fix, and Adaptive Decay Sampling is a non-uniform step schedule for inference. Neither adds parameters or alters the architecture. Across six generation tasks, the methods give roughly 100× faster decoding with better quality, or about 200× with comparable quality. This makes diffusion text generation far more practical.

## Key points
- Two gaps are targeted: reliance on self-generated noise at inference, and the mismatch between downsampled inference steps and the full training trajectory.
- The motivating cost example: a diffusion model needs about 12 hours on one A100 GPU to generate 10K sentences of length 128. A CMLM non-autoregressive model needs only minutes for the same job.
- The pilot study used the ROC Stories corpus with B-2, LR-2, R-2 and PPL metrics. Weighted injection of predicted noise outperformed both vanilla training and full noise replacement. Front-loaded non-uniform sampling even exceeded running all reverse steps.
- The evaluation covers 6 tasks in three settings: directed, open-ended and controllable generation. Human evaluation showed substantial inter-annotator agreement, and the gains over vanilla diffusion grew as fewer reverse steps were used.
- The authors note two limitations: output diversity is lower than that of LMs because of clamp sampling, and performance on open-ended generation is weak. Code is available at github.com/CODINNLG/Bridge_Gap_Diffusion.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

## Cited by

Cited by

13 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 12 not yet.

- [DiffuSeq-v2: Bridging Discrete and Continuous Text Spaces for Accelerated Seq2Seq Diffusion Models](https://md.research.elamur.ai/free/papers/gong-2023-diffuseq-v2-bridging-discrete-continuous/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{tang2023can,
  title = {{Can Diffusion Model Achieve Better Performance in Text Generation ? Bridging the Gap between Training and Inference !}},
  author = {Tang, Zecheng and Wang, Pinzheng and Zhou, Keyan and Li, Juntao and Cao, Ziqiang and Zhang, Min},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2023},
  year = {2023},
  doi = {10.18653/v1/2023.findings-acl.721},
  eprint = {2305.04465},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2023.findings-acl.721}
}
```

- [← previous Can AI-Generated Text be Reliably Detected?](https://md.research.elamur.ai/free/papers/sadasivan-2023-ai-generated-text-reliably-detected/)
- [next → Can GPT-3 Perform Statutory Reasoning?](https://md.research.elamur.ai/free/papers/blair-2023-stanek-gpt-3-perform-statutory-reasoning/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/papers/2023/)

## References

59 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. 2021. Structured denoising diffusion models in discrete state-spaces. Advances in Neural Information Processing Systems, 34:17981–17993. [find](https://scholar.google.com/scholar?q=Jacob+Austin%2C+Daniel+D+Johnson%2C+Jonathan+Ho%2C+Daniel+Tarlow%2C+and+Rianne+van+den+Berg%2E+2021%2E+Structured+denoising+diffusion+models+in+discrete+state%2Dspaces%2E+Advances+in+Neural+Information+Processing+Sys) [our card](https://md.research.elamur.ai/free/papers/austin-2021-structured-denoising-diffusion-models-discrete/)
2. Dzmitry Bahdanau, Philemon Brakel, Kelvin Xu, Anirudh Goyal, Ryan Lowe, Joelle Pineau, Aaron Courville, and Yoshua Bengio. 2016. An actor-critic algorithm for sequence prediction. arXiv preprint arXiv:1607.07086. [find](https://scholar.google.com/scholar?q=Dzmitry+Bahdanau%2C+Philemon+Brakel%2C+Kelvin+Xu%2C+Anirudh+Goyal%2C+Ryan+Lowe%2C+Joelle+Pineau%2C+Aaron+Courville%2C+and+Yoshua+Bengio%2E+2016%2E+An+actor%2Dcritic+algorithm+for+sequence+prediction%2E+arXiv+preprint+arXiv) [our card](https://md.research.elamur.ai/free/papers/bahdanau-2016-actor-critic-algorithm-sequence-prediction/)
3. Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. 2015. Scheduled sampling for sequence prediction with recurrent neural networks. Advances in neural information processing systems, 28. [find](https://scholar.google.com/scholar?q=Samy+Bengio%2C+Oriol+Vinyals%2C+Navdeep+Jaitly%2C+and+Noam+Shazeer%2E+2015%2E+Scheduled+sampling+for+sequence+prediction+with+recurrent+neural+networks%2E+Advances+in+neural+information+processing+systems%2C+28%2E)
4. Dzmitry Bahdanau Philemon Brakel, Kelvin Xu Anirudh Goyal, RL PINEAU, Aaron Courville, and Yoshua Bengio. 2017. An actor-critic algorithm for sequence prediction. In Open Review. net. International Conference on Learning Representations, pages 1–17. [find](https://scholar.google.com/scholar?q=Dzmitry+Bahdanau+Philemon+Brakel%2C+Kelvin+Xu+Anirudh+Goyal%2C+RL+PINEAU%2C+Aaron+Courville%2C+and+Yoshua+Bengio%2E+2017%2E+An+actor%2Dcritic+algorithm+for+sequence+prediction%2E+In+Open+Review%2E+net%2E+International+Co) [our card](https://md.research.elamur.ai/free/papers/bahdanau-2016-actor-critic-algorithm-sequence-prediction/)
5. Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2019. Plug and play language models: A simple approach to controlled text generation. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Sumanth+Dathathri%2C+Andrea+Madotto%2C+Janice+Lan%2C+Jane+Hung%2C+Eric+Frank%2C+Piero+Molino%2C+Jason+Yosinski%2C+and+Rosanne+Liu%2E+2019%2E+Plug+and+play+language+models%3A+A+simple+approach+to+controlled+text+generatio) [our card](https://md.research.elamur.ai/free/papers/dathathri-2019-plug-play-language-models-simple/)
6. Bhuwan Dhingra, Kathryn Mazaitis, and William W Cohen. 2017. Quasar: Datasets for question answering by search and reading. arXiv preprint arXiv:1707.03904. — [arXiv:1707.03904](https://arxiv.org/abs/1707.03904) [find](https://scholar.google.com/scholar?q=Bhuwan+Dhingra%2C+Kathryn+Mazaitis%2C+and+William+W+Cohen%2E+2017%2E+Quasar%3A+Datasets+for+question+answering+by+search+and+reading%2E+arXiv+preprint+arXiv%3A1707%2E03904%2E+%E2%80%94+%5BarXiv%3A1707%2E03904%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F)
7. Joseph L Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological bulletin, 76(5):378. [find](https://scholar.google.com/scholar?q=Joseph+L+Fleiss%2E+1971%2E+Measuring+nominal+scale+agreement+among+many+raters%2E+Psychological+bulletin%2C+76%285%29%3A378%2E)
8. Zhujin Gao, Junliang Guo, Xu Tan, Yongxin Zhu, Fang Zhang, Jiang Bian, and Linli Xu. 2022. Difformer: Empowering diffusion model on embedding space for text generation. arXiv preprint arXiv:2212.09412. — [arXiv:2212.09412](https://arxiv.org/abs/2212.09412) [find](https://scholar.google.com/scholar?q=Zhujin+Gao%2C+Junliang+Guo%2C+Xu+Tan%2C+Yongxin+Zhu%2C+Fang+Zhang%2C+Jiang+Bian%2C+and+Linli+Xu%2E+2022%2E+Difformer%3A+Empowering+diffusion+model+on+embedding+space+for+text+generation%2E+arXiv+preprint+arXiv%3A2212%2E09412)
9. Marjan Ghazvininejad, Omer Levy, Yinhan Liu, and Luke Zettlemoyer. 2019. Mask-predict: Parallel decoding of conditional masked language models. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Pro… [find](https://scholar.google.com/scholar?q=Marjan+Ghazvininejad%2C+Omer+Levy%2C+Yinhan+Liu%2C+and+Luke+Zettlemoyer%2E+2019%2E+Mask%2Dpredict%3A+Parallel+decoding+of+conditional+masked+language+models%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Metho) [our card](https://md.research.elamur.ai/free/papers/ghazvininejad-2019-mask-predict-parallel-decoding-conditional/)
10. Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, and LingPeng Kong. 2022. Diffuseq: Sequence to sequence text generation with diffusion models. arXiv preprint arXiv:2210.08933. [find](https://scholar.google.com/scholar?q=Shansan+Gong%2C+Mukai+Li%2C+Jiangtao+Feng%2C+Zhiyong+Wu%2C+and+LingPeng+Kong%2E+2022%2E+Diffuseq%3A+Sequence+to+sequence+text+generation+with+diffusion+models%2E+arXiv+preprint+arXiv%3A2210%2E08933%2E)
11. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2020. Generative adversarial networks. Communications of the ACM, 63(11):139–144. [find](https://scholar.google.com/scholar?q=Ian+Goodfellow%2C+Jean+Pouget%2DAbadie%2C+Mehdi+Mirza%2C+Bing+Xu%2C+David+Warde%2DFarley%2C+Sherjil+Ozair%2C+Aaron+Courville%2C+and+Yoshua+Bengio%2E+2020%2E+Generative+adversarial+networks%2E+Communications+of+the+ACM%2C+63%2811)
12. Jiatao Gu, Changhan Wang, and Junbo Zhao. 2019. Levenshtein transformer. Advances in Neural Information Processing Systems, 32. [find](https://scholar.google.com/scholar?q=Jiatao+Gu%2C+Changhan+Wang%2C+and+Junbo+Zhao%2E+2019%2E+Levenshtein+transformer%2E+Advances+in+Neural+Information+Processing+Systems%2C+32%2E)
13. Jian Guan, Xiaoxi Mao, Changjie Fan, Zitao Liu, Wenbiao Ding, and Minlie Huang. 2021. Long text generation by modeling sentence-level and discourse-level coherence. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference… [find](https://scholar.google.com/scholar?q=Jian+Guan%2C+Xiaoxi+Mao%2C+Changjie+Fan%2C+Zitao+Liu%2C+Wenbiao+Ding%2C+and+Minlie+Huang%2E+2021%2E+Long+text+generation+by+modeling+sentence%2Dlevel+and+discourse%2Dlevel+coherence%2E+In+Proceedings+of+the+59th+Annual+M)
14. Xiaochuang Han, Sachin Kumar, and Yulia Tsvetkov. 2022. Ssd-lm: Semi-autoregressive simplex-based diffusion language model for text generation and modular control. arXiv preprint arXiv:2210.17432. [find](https://scholar.google.com/scholar?q=Xiaochuang+Han%2C+Sachin+Kumar%2C+and+Yulia+Tsvetkov%2E+2022%2E+Ssd%2Dlm%3A+Semi%2Dautoregressive+simplex%2Dbased+diffusion+language+model+for+text+generation+and+modular+control%2E+arXiv+preprint+arXiv%3A2210%2E17432%2E)
15. Tianxing He, Jingzhao Zhang, Zhiming Zhou, and James Glass. 2021. Exposure bias versus self-recovery: Are distortions really incremental for autoregressive text generation? In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5087–5102. [find](https://scholar.google.com/scholar?q=Tianxing+He%2C+Jingzhao+Zhang%2C+Zhiming+Zhou%2C+and+James+Glass%2E+2021%2E+Exposure+bias+versus+self%2Drecovery%3A+Are+distortions+really+incremental+for+autoregressive+text+generation%3F+In+Proceedings+of+the+2021+)
16. Zhengfu He, Tianxiang Sun, Kuanning Wang, Xuanjing Huang, and Xipeng Qiu. 2022. Diffusionbert: Improving generative masked language models with diffusion models. arXiv preprint arXiv:2211.15029. [find](https://scholar.google.com/scholar?q=Zhengfu+He%2C+Tianxiang+Sun%2C+Kuanning+Wang%2C+Xuanjing+Huang%2C+and+Xipeng+Qiu%2E+2022%2E+Diffusionbert%3A+Improving+generative+masked+language+models+with+diffusion+models%2E+arXiv+preprint+arXiv%3A2211%2E15029%2E)
17. Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840– 6851. [find](https://scholar.google.com/scholar?q=Jonathan+Ho%2C+Ajay+Jain%2C+and+Pieter+Abbeel%2E+2020%2E+Denoising+diffusion+probabilistic+models%2E+Advances+in+Neural+Information+Processing+Systems%2C+33%3A6840%E2%80%93+6851%2E)
18. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2019. The curious case of neural text degeneration. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Ari+Holtzman%2C+Jan+Buys%2C+Li+Du%2C+Maxwell+Forbes%2C+and+Yejin+Choi%2E+2019%2E+The+curious+case+of+neural+text+degeneration%2E+In+International+Conference+on+Learning+Representations%2E)
19. Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Forré, and Max Welling. 2021. Argmax flows and multinomial diffusion: Learning categorical distributions. Advances in Neural Information Processing Systems, 34:12454–12465. [find](https://scholar.google.com/scholar?q=Emiel+Hoogeboom%2C+Didrik+Nielsen%2C+Priyank+Jaini%2C+Patrick+Forr%C3%A9%2C+and+Max+Welling%2E+2021%2E+Argmax+flows+and+multinomial+diffusion%3A+Learning+categorical+distributions%2E+Advances+in+Neural+Information+Process)
20. Emiel Hoogeboom, Vıctor Garcia Satorras, Clément Vignac, and Max Welling. 2022. Equivariant diffusion for molecule generation in 3d. In International Conference on Machine Learning, pages 8867–8887. PMLR. [find](https://scholar.google.com/scholar?q=Emiel+Hoogeboom%2C+V%C4%B1ctor+Garcia+Satorras%2C+Cl%C3%A9ment+Vignac%2C+and+Max+Welling%2E+2022%2E+Equivariant+diffusion+for+molecule+generation+in+3d%2E+In+International+Conference+on+Machine+Learning%2C+pages+8867%E2%80%938887%2E+P)
21. Ferenc Huszár. 2015. How (not) to train your generative model: Scheduled sampling, likelihood, adversary? arXiv preprint arXiv:1511.05101. — [arXiv:1511.05101](https://arxiv.org/abs/1511.05101) [find](https://scholar.google.com/scholar?q=Ferenc+Husz%C3%A1r%2E+2015%2E+How+%28not%29+to+train+your+generative+model%3A+Scheduled+sampling%2C+likelihood%2C+adversary%3F+arXiv+preprint+arXiv%3A1511%2E05101%2E+%E2%80%94+%5BarXiv%3A1511%2E05101%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1511%2E05101%29)
22. Chao Jiang, Mounica Maddela, Wuwei Lan, Yang Zhong, and Wei Xu. 2020. Neural crf model for sentence alignment in text simplification. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7943–7960. [find](https://scholar.google.com/scholar?q=Chao+Jiang%2C+Mounica+Maddela%2C+Wuwei+Lan%2C+Yang+Zhong%2C+and+Wei+Xu%2E+2020%2E+Neural+crf+model+for+sentence+alignment+in+text+simplification%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Association+for+C) [our card](https://md.research.elamur.ai/free/papers/jiang-2020-neural-crf-model-sentence-alignment/)
23. Alexia Jolicoeur-Martineau, Ke Li, Rémi Piché-Taillefer, Tal Kachman, and Ioannis Mitliagkas. 2021. Gotta go fast when generating data with score-based models. arXiv preprint arXiv:2105.14080. — [arXiv:2105.14080](https://arxiv.org/abs/2105.14080) [find](https://scholar.google.com/scholar?q=Alexia+Jolicoeur%2DMartineau%2C+Ke+Li%2C+R%C3%A9mi+Pich%C3%A9%2DTaillefer%2C+Tal+Kachman%2C+and+Ioannis+Mitliagkas%2E+2021%2E+Gotta+go+fast+when+generating+data+with+score%2Dbased+models%2E+arXiv+preprint+arXiv%3A2105%2E14080%2E+%E2%80%94+%5BarXi)
24. Beomsu Kim and Jong Chul Ye. 2022. Denoising mcmc for accelerating diffusion-based generative models. arXiv preprint arXiv:2209.14593. — [arXiv:2209.14593](https://arxiv.org/abs/2209.14593) [find](https://scholar.google.com/scholar?q=Beomsu+Kim+and+Jong+Chul+Ye%2E+2022%2E+Denoising+mcmc+for+accelerating+diffusion%2Dbased+generative+models%2E+arXiv+preprint+arXiv%3A2209%2E14593%2E+%E2%80%94+%5BarXiv%3A2209%2E14593%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2209%2E14593%29)
25. Jekaterina Novikova, Ondřej Dušek, and Verena Rieser. 2017. The e2e dataset: New challenges for end-to-end generation. arXiv preprint arXiv:1706.09254. [find](https://scholar.google.com/scholar?q=Jekaterina+Novikova%2C+Ondr%CC%8Cej+Du%C5%A1ek%2C+and+Verena+Rieser%2E+2017%2E+The+e2e+dataset%3A+New+challenges+for+end%2Dto%2Dend+generation%2E+arXiv+preprint+arXiv%3A1706%2E09254%2E) [our card](https://md.research.elamur.ai/free/papers/novikova-2017-e2e-dataset-new-challenges-end/)
26. Diederik P Kingma and Max Welling. 2013. Autoencoding variational bayes. arXiv preprint arXiv:1312.6114. [find](https://scholar.google.com/scholar?q=Diederik+P+Kingma+and+Max+Welling%2E+2013%2E+Autoencoding+variational+bayes%2E+arXiv+preprint+arXiv%3A1312%2E6114%2E) [our card](https://md.research.elamur.ai/free/papers/kingma-2013-auto-encoding-variational-bayes/)
27. Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstr… [find](https://scholar.google.com/scholar?q=Myle+Ott%2C+Sergey+Edunov%2C+Alexei+Baevski%2C+Angela+Fan%2C+Sam+Gross%2C+Nathan+Ng%2C+David+Grangier%2C+and+Michael+Auli%2E+2019%2E+fairseq%3A+A+fast%2C+extensible+toolkit+for+sequence+modeling%2E+In+Proceedings+of+the+2019) [our card](https://md.research.elamur.ai/free/papers/ott-2019-fairseq-fast-extensible-toolkit-sequence/)
28. Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. 2020. Diffwave: A versatile diffusion model for audio synthesis. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Zhifeng+Kong%2C+Wei+Ping%2C+Jiaji+Huang%2C+Kexin+Zhao%2C+and+Bryan+Catanzaro%2E+2020%2E+Diffwave%3A+A+versatile+diffusion+model+for+audio+synthesis%2E+In+International+Conference+on+Learning+Representations%2E)
29. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the… [find](https://scholar.google.com/scholar?q=Mike+Lewis%2C+Yinhan+Liu%2C+Naman+Goyal%2C+Marjan+Ghazvininejad%2C+Abdelrahman+Mohamed%2C+Omer+Levy%2C+Veselin+Stoyanov%2C+and+Luke+Zettlemoyer%2E+2020%2E+Bart%3A+Denoising+sequence%2Dto%2Dsequence+pre%2Dtraining+for+natural+l) [our card](https://md.research.elamur.ai/free/papers/lewis-2020-bart-denoising-sequence-sequence-pre/)
30. Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and William B Dolan. 2016. A diversity-promoting objective function for neural conversation models. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies… [find](https://scholar.google.com/scholar?q=Jiwei+Li%2C+Michel+Galley%2C+Chris+Brockett%2C+Jianfeng+Gao%2C+and+William+B+Dolan%2E+2016%2E+A+diversity%2Dpromoting+objective+function+for+neural+conversation+models%2E+In+Proceedings+of+the+2016+Conference+of+the+) [our card](https://md.research.elamur.ai/free/papers/li-2016-diversity-promoting-objective-function-neural/)
31. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318. [find](https://scholar.google.com/scholar?q=Kishore+Papineni%2C+Salim+Roukos%2C+Todd+Ward%2C+and+Wei%2DJing+Zhu%2E+2002%2E+Bleu%3A+a+method+for+automatic+evaluation+of+machine+translation%2E+In+Proceedings+of+the+40th+annual+meeting+of+the+Association+for+Comp) [our card](https://md.research.elamur.ai/free/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
32. Krishna Pillutla, Swabha Swayamdipta, Rowan Zellers, John Thickstun, Sean Welleck, Yejin Choi, and Zaid Harchaoui. 2021. Mauve: Measuring the gap between neural text and human text using divergence frontiers. Advances in Neural Information Processing Systems, 34:4816–4828. [find](https://scholar.google.com/scholar?q=Krishna+Pillutla%2C+Swabha+Swayamdipta%2C+Rowan+Zellers%2C+John+Thickstun%2C+Sean+Welleck%2C+Yejin+Choi%2C+and+Zaid+Harchaoui%2E+2021%2E+Mauve%3A+Measuring+the+gap+between+neural+text+and+human+text+using+divergence+fr) [our card](https://md.research.elamur.ai/free/papers/pillutla-2021-mauve-measuring-gap-between-neural/)
33. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+Ilya+Sutskever%2C+et+al%2E+Language+models+are+unsupervised+multitask+learners%2E)
34. Xiang Lisa Li, John Thickstun, Ishaan Gulrajani, Percy Liang, and Tatsunori B Hashimoto. 2022. Diffusion-lm improves controllable text generation. arXiv preprint arXiv:2205.14217. [find](https://scholar.google.com/scholar?q=Xiang+Lisa+Li%2C+John+Thickstun%2C+Ishaan+Gulrajani%2C+Percy+Liang%2C+and+Tatsunori+B+Hashimoto%2E+2022%2E+Diffusion%2Dlm+improves+controllable+text+generation%2E+arXiv+preprint+arXiv%3A2205%2E14217%2E) [our card](https://md.research.elamur.ai/free/papers/li-2022-diffusion-lm-improves-controllable-text/)
35. Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. 2022. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125. [find](https://scholar.google.com/scholar?q=Aditya+Ramesh%2C+Prafulla+Dhariwal%2C+Alex+Nichol%2C+Casey+Chu%2C+and+Mark+Chen%2E+2022%2E+Hierarchical+text%2Dconditional+image+generation+with+clip+latents%2E+arXiv+preprint+arXiv%3A2204%2E06125%2E) [our card](https://md.research.elamur.ai/free/papers/ramesh-2022-hierarchical-text-conditional-image-generation/)
36. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81. [find](https://scholar.google.com/scholar?q=Chin%2DYew+Lin%2E+2004%2E+Rouge%3A+A+package+for+automatic+evaluation+of+summaries%2E+In+Text+summarization+branches+out%2C+pages+74%E2%80%9381%2E)
37. Marc’Aurelio Ranzato, Sumit Chopra, Michael Auli, and Wojciech Zaremba. 2015. Sequence level training with recurrent neural networks. arXiv preprint arXiv:1511.06732. [find](https://scholar.google.com/scholar?q=Marc%E2%80%99Aurelio+Ranzato%2C+Sumit+Chopra%2C+Michael+Auli%2C+and+Wojciech+Zaremba%2E+2015%2E+Sequence+level+training+with+recurrent+neural+networks%2E+arXiv+preprint+arXiv%3A1511%2E06732%2E) [our card](https://md.research.elamur.ai/free/papers/ranzato-2015-sequence-level-training-recurrent-neural/)
38. Yankai Lin, Haozhe Ji, Zhiyuan Liu, and Maosong Sun. 2018. Denoising distantly supervised open-domain question answering. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1736– 1745. [find](https://scholar.google.com/scholar?q=Yankai+Lin%2C+Haozhe+Ji%2C+Zhiyuan+Liu%2C+and+Maosong+Sun%2E+2018%2E+Denoising+distantly+supervised+open%2Ddomain+question+answering%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Computational)
39. Zhenghao Lin, Yeyun Gong, Yelong Shen, Tong Wu, Zhihao Fan, Chen Lin, Weizhu Chen, and Nan Duan. 2022. Genie: Large scale pre-training for text generation with diffusion model. arXiv preprint arXiv:2212.11685. — [arXiv:2212.11685](https://arxiv.org/abs/2212.11685) [find](https://scholar.google.com/scholar?q=Zhenghao+Lin%2C+Yeyun+Gong%2C+Yelong+Shen%2C+Tong+Wu%2C+Zhihao+Fan%2C+Chen+Lin%2C+Weizhu+Chen%2C+and+Nan+Duan%2E+2022%2E+Genie%3A+Large+scale+pre%2Dtraining+for+text+generation+with+diffusion+model%2E+arXiv+preprint+arXiv%3A22)
40. Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, and James Allen. 2016. A corpus and cloze evaluation for deeper understanding of commonsense stories. In Proceedings of the 2016 Conference of the North American Chapter of the Associati… [find](https://scholar.google.com/scholar?q=Nasrin+Mostafazadeh%2C+Nathanael+Chambers%2C+Xiaodong+He%2C+Devi+Parikh%2C+Dhruv+Batra%2C+Lucy+Vanderwende%2C+Pushmeet+Kohli%2C+and+James+Allen%2E+2016%2E+A+corpus+and+cloze+evaluation+for+deeper+understanding+of+commo)
41. Alexander Quinn Nichol and Prafulla Dhariwal. 2021. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning, pages 8162–8171. PMLR. [find](https://scholar.google.com/scholar?q=Alexander+Quinn+Nichol+and+Prafulla+Dhariwal%2E+2021%2E+Improved+denoising+diffusion+probabilistic+models%2E+In+International+Conference+on+Machine+Learning%2C+pages+8162%E2%80%938171%2E+PMLR%2E)
42. Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684–10695. [find](https://scholar.google.com/scholar?q=Robin+Rombach%2C+Andreas+Blattmann%2C+Dominik+Lorenz%2C+Patrick+Esser%2C+and+Bj%C3%B6rn+Ommer%2E+2022%2E+High%2Dresolution+image+synthesis+with+latent+diffusion+models%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Compu)
43. Tim Salimans and Jonathan Ho. 2021. Progressive distillation for fast sampling of diffusion models. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Tim+Salimans+and+Jonathan+Ho%2E+2021%2E+Progressive+distillation+for+fast+sampling+of+diffusion+models%2E+In+International+Conference+on+Learning+Representations%2E)
44. Florian Schmidt. 2019. Generalization in generation: A closer look at exposure bias. [find](https://scholar.google.com/scholar?q=Florian+Schmidt%2E+2019%2E+Generalization+in+generation%3A+A+closer+look+at+exposure+bias%2E)
45. EMNLP-IJCNLP 2019, page 157. [find](https://scholar.google.com/scholar?q=EMNLP%2DIJCNLP+2019%2C+page+157%2E)
46. Zhihong Shao, Minlie Huang, Jiangtao Wen, Wenfei Xu, and Xiaoyan Zhu. 2019. Long and diverse text generation with planning-based hierarchical variational model. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Nat… [find](https://scholar.google.com/scholar?q=Zhihong+Shao%2C+Minlie+Huang%2C+Jiangtao+Wen%2C+Wenfei+Xu%2C+and+Xiaoyan+Zhu%2E+2019%2E+Long+and+diverse+text+generation+with+planning%2Dbased+hierarchical+variational+model%2E+In+Proceedings+of+the+2019+Conference+o)
47. Shiqi Shen, Yong Cheng, Zhongjun He, Wei He, Hua Wu, Maosong Sun, and Yang Liu. 2016. Minimum risk training for neural machine translation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1683–1692. [find](https://scholar.google.com/scholar?q=Shiqi+Shen%2C+Yong+Cheng%2C+Zhongjun+He%2C+Wei+He%2C+Hua+Wu%2C+Maosong+Sun%2C+and+Yang+Liu%2E+2016%2E+Minimum+risk+training+for+neural+machine+translation%2E+In+Proceedings+of+the+54th+Annual+Meeting+of+the+Association) [our card](https://md.research.elamur.ai/free/papers/shen-2015-minimum-risk-training-neural-machine/)
48. Jascha Sohl-Dickstein, Eric Weiss, Niru Mah-eswaranathan, and Surya Ganguli. 2015. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning, pages 2256–2265. PMLR. [find](https://scholar.google.com/scholar?q=Jascha+Sohl%2DDickstein%2C+Eric+Weiss%2C+Niru+Mah%2Deswaranathan%2C+and+Surya+Ganguli%2E+2015%2E+Deep+unsupervised+learning+using+nonequilibrium+thermodynamics%2E+In+International+Conference+on+Machine+Learning%2C+page) [our card](https://md.research.elamur.ai/free/papers/sohl-2015-dickstein-deep-unsupervised-learning-using-nonequilibrium/)
49. Jiaming Song, Chenlin Meng, and Stefano Ermon. 2020. Denoising diffusion implicit models. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Jiaming+Song%2C+Chenlin+Meng%2C+and+Stefano+Ermon%2E+2020%2E+Denoising+diffusion+implicit+models%2E+In+International+Conference+on+Learning+Representations%2E) [our card](https://md.research.elamur.ai/free/papers/song-2020-denoising-diffusion-implicit-models/)
50. Robin Strudel, Corentin Tallec, Florent Altché, Yilun Du, Yaroslav Ganin, Arthur Mensch, Will Grathwohl, Nikolay Savinov, Sander Dieleman, Laurent Sifre, et al. 2022. Self-conditioned embedding diffusion for text generation. arXiv preprint arXiv:2211.04236. — [arXiv:2211.04236](https://arxiv.org/abs/2211.04236) [find](https://scholar.google.com/scholar?q=Robin+Strudel%2C+Corentin+Tallec%2C+Florent+Altch%C3%A9%2C+Yilun+Du%2C+Yaroslav+Ganin%2C+Arthur+Mensch%2C+Will+Grathwohl%2C+Nikolay+Savinov%2C+Sander+Dieleman%2C+Laurent+Sifre%2C+et+al%2E+2022%2E+Self%2Dconditioned+embedding+diffus)
51. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+Advances+in+neural+information+processing+)
52. Chunqi Wang, Ji Zhang, and Haiqing Chen. 2018. Semi-autoregressive neural machine translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 479–488. [find](https://scholar.google.com/scholar?q=Chunqi+Wang%2C+Ji+Zhang%2C+and+Haiqing+Chen%2E+2018%2E+Semi%2Dautoregressive+neural+machine+translation%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+479%E2%80%93488%2E)
53. Sam Wiseman and Alexander M Rush. 2016. Sequence-to-sequence learning as beam-search optimization. arXiv preprint arXiv:1606.02960. [find](https://scholar.google.com/scholar?q=Sam+Wiseman+and+Alexander+M+Rush%2E+2016%2E+Sequence%2Dto%2Dsequence+learning+as+beam%2Dsearch+optimization%2E+arXiv+preprint+arXiv%3A1606%2E02960%2E)
54. Kevin Yang and Dan Klein. 2021. Fudge: Controlled text generation with future discriminators. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 3511–3535. [find](https://scholar.google.com/scholar?q=Kevin+Yang+and+Dan+Klein%2E+2021%2E+Fudge%3A+Controlled+text+generation+with+future+discriminators%2E+In+Proceedings+of+the+2021+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+L) [our card](https://md.research.elamur.ai/free/papers/yang-2021-fudge-controlled-text-generation-future/)
55. Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Runsheng Xu, Yue Zhao, Yingxia Shao, Wentao Zhang, Bin Cui, and Ming-Hsuan Yang. 2022. Diffusion models: A comprehensive survey of methods and applications. arXiv preprint arXiv:2209.00796. — [arXiv:2209.00796](https://arxiv.org/abs/2209.00796) [find](https://scholar.google.com/scholar?q=Ling+Yang%2C+Zhilong+Zhang%2C+Yang+Song%2C+Shenda+Hong%2C+Runsheng+Xu%2C+Yue+Zhao%2C+Yingxia+Shao%2C+Wentao+Zhang%2C+Bin+Cui%2C+and+Ming%2DHsuan+Yang%2E+2022%2E+Diffusion+models%3A+A+comprehensive+survey+of+methods+and+applica)
56. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. 2019a. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Tianyi+Zhang%2C+Varsha+Kishore%2C+Felix+Wu%2C+Kilian+Q+Weinberger%2C+and+Yoav+Artzi%2E+2019a%2E+Bertscore%3A+Evaluating+text+generation+with+bert%2E+In+International+Conference+on+Learning+Representations%2E)
57. Wen Zhang, Yang Feng, Fandong Meng, Di You, and Qun Liu. 2019b. Bridging the gap between training and inference for neural machine translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4334– 4343. [find](https://scholar.google.com/scholar?q=Wen+Zhang%2C+Yang+Feng%2C+Fandong+Meng%2C+Di+You%2C+and+Qun+Liu%2E+2019b%2E+Bridging+the+gap+between+training+and+inference+for+neural+machine+translation%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Associa) [our card](https://md.research.elamur.ai/free/papers/zhang-2019-bridging-gap-between-training-inference/)
58. Zhirui Zhang, Shuangzhi Wu, Shujie Liu, Mu Li, Ming Zhou, and Tong Xu. 2019c. Regularizing neural machine translation by target-bidirectional agreement. [find](https://scholar.google.com/scholar?q=Zhirui+Zhang%2C+Shuangzhi+Wu%2C+Shujie+Liu%2C+Mu+Li%2C+Ming+Zhou%2C+and+Tong+Xu%2E+2019c%2E+Regularizing+neural+machine+translation+by+target%2Dbidirectional+agreement%2E)
59. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pages 443–450. [find](https://scholar.google.com/scholar?q=In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+volume+33%2C+pages+443%E2%80%93450%2E)