---
title: "DiffuSeq — paper by Gong et al. (2022) — summary and ideas"
description: "The authors propose DiffuSeq, which they describe as the first diffusion model for SEQ2SEQ text… 51 references, cited by 98 papers here."
authors: ["Shansan Gong", "Mukai Li", "Jiangtao Feng", "Zhiyong Wu", "Lingpeng Kong"]
year: 2022
venue: "arXiv:2210.08933"
source: "https://arxiv.org/abs/2210.08933"
doi: "10.48550/arxiv.2210.08933"
arxiv: "2210.08933"
licence: "CC BY-NC-SA 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by-nc-sa/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/gong-2022-diffuseq-sequence-sequence-text-generation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/gong-2022-diffuseq-sequence-sequence-text-generation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/2/) / DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models

# DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models

**Authors:** [Shansan Gong](https://md.research.elamur.ai/free/en/authors/shansan-gong/), [Mukai Li](https://md.research.elamur.ai/free/en/authors/mukai-li/), [Jiangtao Feng](https://md.research.elamur.ai/free/en/authors/jiangtao-feng/), [Zhiyong Wu](https://md.research.elamur.ai/free/en/authors/zhiyong-wu/) and [Lingpeng Kong](https://md.research.elamur.ai/free/en/authors/lingpeng-kong/) · **Year:** 2022 · **Venue:** arXiv:2210.08933 · **Licence:** [CC BY-NC-SA 4.0 (arXiv)](http://creativecommons.org/licenses/by-nc-sa/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2210.08933) · [PDF](https://arxiv.org/pdf/2210.08933) · [DOI 10.48550/arxiv.2210.08933](https://doi.org/10.48550/arxiv.2210.08933) · [arXiv 2210.08933](https://arxiv.org/abs/2210.08933) · [OpenAlex](https://openalex.org/W4306802991)

**Citations:** 91 citations (OpenAlex, 2026-10-14) · [98](https://md.research.elamur.ai/free/en/papers/gong-2022-diffuseq-sequence-sequence-text-generation/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Diffusion models perform well on continuous signals such as images and audio, but text is discrete. Earlier text diffusion work, such as Diffusion-LM, mostly handled unconditional generation or steered outputs with separately trained classifiers. That approach is impractical when the condition is an entire input sentence. The authors propose DiffuSeq, which they describe as the first diffusion model for SEQ2SEQ text generation. Source and target tokens are embedded into one shared continuous space and concatenated. In the forward process only the target part receives Gaussian noise ("partial noising"). A single Transformer then learns to reconstruct the target while attending to the intact source, with no external classifier. Training samples diffusion steps by importance, and decoding restores the source segment at every step. Final outputs are chosen from several samples via MBR. On four SEQ2SEQ tasks, including dialogue, question generation and text simplification, DiffuSeq matches or beats six baselines, among them fine-tuned GPT2, and yields clearly more diverse outputs. The authors also analyse it theoretically as an extension of iterative non-autoregressive models. The work shows that diffusion is a viable approach to conditional language generation.

## Key points
- **Core method:** the conditioning is classifier-free and comes from noising only the target half of the concatenated embedding. Embeddings are shared between source and target. At inference, an anchoring step rounds states back to word embeddings and resets the source part to its original value.
- **Training and decoding:** importance sampling over diffusion steps (following Nichol & Dhariwal) addresses insufficient training. MBR decoding uses negative BLEU as the risk.
- **Baselines:** GRU-attention (65M), Transformer-base (80M), GPT2-base FT (117M), GPT2-large FT (774M), GPVAE-T5 (220M) and NAR-LevT (80M). DiffuSeq has 91M parameters.
- **Evaluation:** metrics are sentence-level BLEU, ROUGE-L and BERTScore (with microsoft/deberta-xlarge-mnli). The question generation data come from Quasar-T, giving 119K document–question training pairs.
- **Trade-offs and availability:** outputs are more diverse than those of the baselines but sometimes less fluent than fine-tuned GPT2. Code is released at github.com/Shark-NLP/DiffuSeq.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Hong Kong](https://md.research.elamur.ai/free/en/universities/university-of-hong-kong/) — University, Pok Fu Lam, China · 118 works · 13,496 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

🧭 Topics (1): [✍️ Text generation](https://md.research.elamur.ai/free/en/topics/text-generation/) · 214 works

## Cite

Cite as: Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu and Lingpeng Kong. 2022. DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models. arXiv:2210.08933.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{gong2022diffuseq,
  title = {{DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models}},
  author = {Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu and Lingpeng Kong},
  year = {2022},
  doi = {10.48550/arxiv.2210.08933},
  eprint = {2210.08933},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2210.08933}
}
```

- [← previous SemEval-2022 Task 11: Multilingual Complex Named Entity Recognition (MultiCoNER)](https://md.research.elamur.ai/free/en/papers/malmasi-2022-semeval-2022-task-11-multilingual/)
- [next → Packed Levitated Marker for Entity and Relation Extraction](https://md.research.elamur.ai/free/en/papers/ye-2022-packed-levitated-marker-entity-relation/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/2/)

## References

51 references, 30 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jacob Austin, Daniel D Johnson, Jonathan Ho, Daniel Tarlow, and Rianne van den Berg. [Structured denoising diffusion models in discrete state-spaces](https://md.research.elamur.ai/free/en/papers/austin-2021-structured-denoising-diffusion-models-discrete/). Advances in Neural Information Processing Systems, 2021.
2. Nanxin Chen, Yu Zhang, Heiga Zen, Ron J Weiss, Mohammad Norouzi, and William Chan. Wavegrad: Estimating gradients for waveform generation. In International Conference on Learning Representations, ICLR, 2021. [find](https://scholar.google.com/scholar?q=Nanxin+Chen%2C+Yu+Zhang%2C+Heiga+Zen%2C+Ron+J+Weiss%2C+Mohammad+Norouzi%2C+and+William+Chan%2E+Wavegrad%3A+Estimating+gradients+for+waveform+generation%2E+In+International+Conference+on+Learning+Representations%2C+ICLR)
3. Ting Chen, Ruixiang Zhang, and Geoffrey Hinton. Analog bits: Generating discrete data using diffusion models with self-conditioning. arXiv preprint arXiv:2208.04202, 2022. — [arXiv:2208.04202](https://arxiv.org/abs/2208.04202) [find](https://scholar.google.com/scholar?q=Ting+Chen%2C+Ruixiang+Zhang%2C+and+Geoffrey+Hinton%2E+Analog+bits%3A+Generating+discrete+data+using+diffusion+models+with+self%2Dconditioning%2E+arXiv+preprint+arXiv%3A2208%2E04202%2C+2022%2E+%E2%80%94+%5BarXiv%3A2208%2E04202%5D%28https%3A%2F)
4. Kyunghyun Cho, Bart van Merrienboer, Çaglar Gülçehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder-decoder for statistical machine translation. In Conference on Empirical Methods in Natural Language Processing, EMNLP, 2014. — [Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation](https://md.research.elamur.ai/free/en/papers/cho-2014-learning-phrase-representations-rnn-encoder/)
5. Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. [Plug and play language models: A simple approach to controlled text generation](https://md.research.elamur.ai/free/en/papers/dathathri-2019-plug-play-language-models-simple/). In 8th International Conference on Learning Representations, ICLR, 2020.
6. Aditya Deshpande, Jyoti Aneja, Liwei Wang, Alexander G Schwing, and David Forsyth. Fast, diverse and accurate image captioning guided by part-of-speech. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10695–10704, 2019. [find](https://scholar.google.com/scholar?q=Aditya+Deshpande%2C+Jyoti+Aneja%2C+Liwei+Wang%2C+Alexander+G+Schwing%2C+and+David+Forsyth%2E+Fast%2C+diverse+and+accurate+image+captioning+guided+by+part%2Dof%2Dspeech%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Co)
7. Bhuwan Dhingra, Kathryn Mazaitis, and William W Cohen. Quasar: Datasets for question answering by search and reading. arXiv preprint arXiv:1707.03904, 2017. [find](https://scholar.google.com/scholar?q=Bhuwan+Dhingra%2C+Kathryn+Mazaitis%2C+and+William+W+Cohen%2E+Quasar%3A+Datasets+for+question+answering+by+search+and+reading%2E+arXiv+preprint+arXiv%3A1707%2E03904%2C+2017%2E)
8. Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using real nvp. In International Conference on Learning Representations, ICLR, 2017. [find](https://scholar.google.com/scholar?q=Laurent+Dinh%2C+Jascha+Sohl%2DDickstein%2C+and+Samy+Bengio%2E+Density+estimation+using+real+nvp%2E+In+International+Conference+on+Learning+Representations%2C+ICLR%2C+2017%2E)
9. Wanyu Du, Jianqiao Zhao, Liwei Wang, and Yangfeng Ji. Diverse text generation via variational encoder-decoder models with gaussian process priors. In Proceedings of 6th Workshop on Structured Prediction for NLP of the Association for Computational Linguistics, ACL, 2022. [find](https://scholar.google.com/scholar?q=Wanyu+Du%2C+Jianqiao+Zhao%2C+Liwei+Wang%2C+and+Yangfeng+Ji%2E+Diverse+text+generation+via+variational+encoder%2Ddecoder+models+with+gaussian+process+priors%2E+In+Proceedings+of+6th+Workshop+on+Structured+Predicti)
10. Marjan Ghazvininejad, Omer Levy, Yinhan Liu, and Luke Zettlemoyer. [Mask-predict: Parallel decoding of conditional masked language models](https://md.research.elamur.ai/free/en/papers/ghazvininejad-2019-mask-predict-parallel-decoding-conditional/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processin…
11. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. [Generative adversarial nets](https://md.research.elamur.ai/free/en/papers/goodfellow-2014-generative-adversarial-nets/). In Advances in Neural Information Processing Systems, 2014.
12. Jiatao Gu, James Bradbury, Caiming Xiong, Victor OK Li, and Richard Socher. [Non-autoregressive neural machine translation](https://md.research.elamur.ai/free/en/papers/gu-2017-non-autoregressive-neural-machine-translation/). In International Conference on Learning Representations, ICLR, 2018.
13. Jiatao Gu, Changhan Wang, and Junbo Zhao. Levenshtein transformer. Advances in Neural Information Processing Systems, 2019. [find](https://scholar.google.com/scholar?q=Jiatao+Gu%2C+Changhan+Wang%2C+and+Junbo+Zhao%2E+Levenshtein+transformer%2E+Advances+in+Neural+Information+Processing+Systems%2C+2019%2E)
14. Jonathan Ho and Tim Salimans. arXiv:2207.[12598, 2022](https://md.research.elamur.ai/free/en/papers/ho-2022-classifier-free-diffusion-guidance/).
15. Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 2020. [find](https://scholar.google.com/scholar?q=Jonathan+Ho%2C+Ajay+Jain%2C+and+Pieter+Abbeel%2E+Denoising+diffusion+probabilistic+models%2E+Advances+in+Neural+Information+Processing+Systems%2C+2020%2E)
16. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. [The curious case of neural text degeneration](https://md.research.elamur.ai/free/en/papers/holtzman-2019-curious-case-neural-text-degeneration/). In International Conference on Learning Representations, ICLR, 2019.
17. Emiel Hoogeboom, Didrik Nielsen, Priyank Jaini, Patrick Forré, and Max Welling. Argmax flows and multinomial diffusion: Learning categorical distributions. Advances in Neural Information Processing Systems, 2021. [find](https://scholar.google.com/scholar?q=Emiel+Hoogeboom%2C+Didrik+Nielsen%2C+Priyank+Jaini%2C+Patrick+Forre%CC%81%2C+and+Max+Welling%2E+Argmax+flows+and+multinomial+diffusion%3A+Learning+categorical+distributions%2E+Advances+in+Neural+Information+Processing+S)
18. Fei Huang, Tianhua Tao, Hao Zhou, Lei Li, and Minlie Huang. On the learning of non-autoregressive transformers. In International Conference on Machine Learning, ICML, 2022. [find](https://scholar.google.com/scholar?q=Fei+Huang%2C+Tianhua+Tao%2C+Hao+Zhou%2C+Lei+Li%2C+and+Minlie+Huang%2E+On+the+learning+of+non%2Dautoregressive+transformers%2E+In+International+Conference+on+Machine+Learning%2C+ICML%2C+2022%2E)
19. Chao Jiang, Mounica Maddela, Wuwei Lan, Yang Zhong, and Wei Xu. [Neural crf model for sentence alignment in text simplification](https://md.research.elamur.ai/free/en/papers/jiang-2020-neural-crf-model-sentence-alignment/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL, 2020.
20. Diederik P. Kingma and Max Welling. [Auto-Encoding Variational Bayes](https://md.research.elamur.ai/free/en/papers/kingma-2013-auto-encoding-variational-bayes/). In International Conference on Learning Representations, ICLR, 2014.
21. Philipp Koehn. [Statistical significance tests for machine translation evaluation](https://md.research.elamur.ai/free/en/papers/koehn-2004-statistical-significance-tests-machine-translation/). In Proceedings of the 2004 conference on empirical methods in natural language processing, pp. 388–395, 2004.
22. Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis. In International Conference on Learning Representations, ICLR, 2020. [find](https://scholar.google.com/scholar?q=Zhifeng+Kong%2C+Wei+Ping%2C+Jiaji+Huang%2C+Kexin+Zhao%2C+and+Bryan+Catanzaro%2E+Diffwave%3A+A+versatile+diffusion+model+for+audio+synthesis%2E+In+International+Conference+on+Learning+Representations%2C+ICLR%2C+2020%2E)
23. Xiang Lisa Li, John Thickstun, Ishaan Gulrajani, Percy Liang, and Tatsunori B Hashimoto. [Diffusion-lm improves controllable text generation](https://md.research.elamur.ai/free/en/papers/li-2022-diffusion-lm-improves-controllable-text/). arXiv preprint arXiv:2205.14217, 2022.
24. Chin-Yew Lin. [Rouge: A package for automatic evaluation of summaries](https://md.research.elamur.ai/free/en/papers/lin-2004-rouge-package-automatic-evaluation-summaries/). In Text summarization branches out, 2004.
25. Yankai Lin, Haozhe Ji, Zhiyuan Liu, and Maosong Sun. [Denoising distantly supervised open-domain question answering](https://md.research.elamur.ai/free/en/papers/lin-2018-denoising-distantly-supervised-open-domain/). In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL, 2018.
26. Luke Metz, Ben Poole, David Pfau, and Jascha Sohl-Dickstein. Unrolled generative adversarial networks. In International Conference on Learning Representations, ICLR, 2017. [find](https://scholar.google.com/scholar?q=Luke+Metz%2C+Ben+Poole%2C+David+Pfau%2C+and+Jascha+Sohl%2DDickstein%2E+Unrolled+generative+adversarial+networks%2E+In+International+Conference+on+Learning+Representations%2C+ICLR%2C+2017%2E)
27. Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. In International Conference on Machine Learning, ICML, 2022. [find](https://scholar.google.com/scholar?q=Alex+Nichol%2C+Prafulla+Dhariwal%2C+Aditya+Ramesh%2C+Pranav+Shyam%2C+Pamela+Mishkin%2C+Bob+McGrew%2C+Ilya+Sutskever%2C+and+Mark+Chen%2E+Glide%3A+Towards+photorealistic+image+generation+and+editing+with+text%2Dguided+diff)
28. Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning, ICML, 2021. [find](https://scholar.google.com/scholar?q=Alexander+Quinn+Nichol+and+Prafulla+Dhariwal%2E+Improved+denoising+diffusion+probabilistic+models%2E+In+International+Conference+on+Machine+Learning%2C+ICML%2C+2021%2E)
29. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, ACL, 2002.
30. Lihua Qian, Hao Zhou, Yu Bao, Mingxuan Wang, Lin Qiu, Weinan Zhang, Yong Yu, and Lei Li. [Glancing transformer for non-autoregressive neural machine translation](https://md.research.elamur.ai/free/en/papers/qian-2021-glancing-transformer-non-autoregressive-neural/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on N…
31. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. 2019. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeff+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+and+Ilya+Sutskever%2E+Language+models+are+unsupervised+multitask+learners%2E+2019%2E)
32. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). Journal of Machine Learning Research, 21(140):1–67, 2020.
33. Classifier-free diffusion guidance. arXiv preprint Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. [Hierarchical text-conditional image generation with clip latents](https://md.research.elamur.ai/free/en/papers/ramesh-2022-hierarchical-text-conditional-image-generation/). arXiv preprint arXiv:2204.06125, 2022.
34. Chitwan Saharia, William Chan, Huiwen Chang, Chris Lee, Jonathan Ho, Tim Salimans, David Fleet, and Mohammad Norouzi. Palette: Image-to-image diffusion models. In ACM SIGGRAPH 2022 Conference Proceedings, 2022a. [find](https://scholar.google.com/scholar?q=Chitwan+Saharia%2C+William+Chan%2C+Huiwen+Chang%2C+Chris+Lee%2C+Jonathan+Ho%2C+Tim+Salimans%2C+David+Fleet%2C+and+Mohammad+Norouzi%2E+Palette%3A+Image%2Dto%2Dimage+diffusion+models%2E+In+ACM+SIGGRAPH+2022+Conference+Proceedi)
35. Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S Sara Mahdavi, Rapha Gontijo Lopes, et al. [Photorealistic text-to-image diffusion models with deep language understanding](https://md.research.elamur.ai/free/en/papers/saharia-2022-photorealistic-text-image-diffusion-models/). arXiv preprint arXiv:2205.11487, 2022b.
36. Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, Xi Chen, and Xi Chen. [Improved techniques for training gans](https://md.research.elamur.ai/free/en/papers/salimans-2016-improved-techniques-training-gans/). In Advances in Neural Information Processing Systems, 2016.
37. Nikolay Savinov, Junyoung Chung, Mikolaj Binkowski, Erich Elsen, and Aaron van den Oord. Step-unrolled denoising autoencoders for text generation. In International Conference on Learning Representations, ICLR, 2021. [find](https://scholar.google.com/scholar?q=Nikolay+Savinov%2C+Junyoung+Chung%2C+Mikolaj+Binkowski%2C+Erich+Elsen%2C+and+Aaron+van+den+Oord%2E+Step%2Dunrolled+denoising+autoencoders+for+text+generation%2E+In+International+Conference+on+Learning+Representatio)
38. Rico Sennrich, Barry Haddow, and Alexandra Birch. [Neural machine translation of rare words with subword units](https://md.research.elamur.ai/free/en/papers/sennrich-2016-neural-machine-translation-rare-words/). In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1715–1725. Association for Computational Linguistics, 2016.
39. Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. [Deep unsupervised learning using nonequilibrium thermodynamics](https://md.research.elamur.ai/free/en/papers/sohl-dickstein-2015-deep-unsupervised-learning-using-nonequilibrium/). In International Conference on Machine Learning, ICML, 2015.
40. Jiaming Song, Chenlin Meng, and Stefano Ermon. [Denoising diffusion implicit models](https://md.research.elamur.ai/free/en/papers/song-2020-denoising-diffusion-implicit-models/). In International Conference on Learning Representations, ICLR, 2020.
41. Yang Song and Stefano Ermon. [Generative modeling by estimating gradients of the data distribution](https://md.research.elamur.ai/free/en/papers/song-2019-generative-modeling-estimating-gradients-data/). Advances in Neural Information Processing Systems, 32, 2019.
42. Yusuke Tashiro, Jiaming Song, Yang Song, and Stefano Ermon. Csdi: Conditional score-based diffusion models for probabilistic time series imputation. Advances in Neural Information Processing Systems, 2021. [find](https://scholar.google.com/scholar?q=Yusuke+Tashiro%2C+Jiaming+Song%2C+Yang+Song%2C+and+Stefano+Ermon%2E+Csdi%3A+Conditional+score%2Dbased+diffusion+models+for+probabilistic+time+series+imputation%2E+Advances+in+Neural+Information+Processing+Systems%2C+)
43. Iulia Turc, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Well-read students learn better: The impact of student initialization on knowledge distillation. CoRR, abs/1908.[08962, 2019](https://md.research.elamur.ai/free/en/papers/turc-2019-well-read-students-learn-better/).
44. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). Advances in neural information processing systems, 2017.
45. Ashwin K Vijayakumar, Michael Cogswell, Ramprasath R Selvaraju, Qing Sun, Stefan Lee, David Crandall, and Dhruv Batra. [Diverse beam search: Decoding diverse solutions from neural sequence models](https://md.research.elamur.ai/free/en/papers/vijayakumar-2016-diverse-beam-search-decoding-diverse/). arXiv preprint arXiv:1610.02424, 2016.
46. P Yu, S Xie, X Ma, B Jia, B Pang, R Gao, Y Zhu, S-C Zhu, and YN Wu. Latent diffusion energy-based model for interpretable text modeling. In International Conference on Machine Learning, ICML, 2022. [find](https://scholar.google.com/scholar?q=P+Yu%2C+S+Xie%2C+X+Ma%2C+B+Jia%2C+B+Pang%2C+R+Gao%2C+Y+Zhu%2C+S%2DC+Zhu%2C+and+YN+Wu%2E+Latent+diffusion+energy%2Dbased+model+for+interpretable+text+modeling%2E+In+International+Conference+on+Machine+Learning%2C+ICML%2C+2022%2E)
47. Hugh Zhang, Daniel Duckworth, Daphne Ippolito, and Arvind Neelakantan. [Trading off diversity and quality in natural language generation](https://md.research.elamur.ai/free/en/papers/zhang-2021-trading-off-diversity-quality-natural/). In Proceedings of the Workshop on Human Evaluation of NLP Systems (HumEval), pp. 25–33. Association for Computational Linguistics, 2021.
48. Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q Weinberger, and Yoav Artzi. Bertscore: Evaluating text generation with bert. In International Conference on Learning Representations, ICLR, 2019. [find](https://scholar.google.com/scholar?q=Tianyi+Zhang%2C+Varsha+Kishore%2C+Felix+Wu%2C+Kilian+Q+Weinberger%2C+and+Yoav+Artzi%2E+Bertscore%3A+Evaluating+text+generation+with+bert%2E+In+International+Conference+on+Learning+Representations%2C+ICLR%2C+2019%2E)
49. Tiancheng Zhao, Ran Zhao, and Maxine Eskenazi. [Learning discourse-level diversity for neural dialog models using conditional variational autoencoders](https://md.research.elamur.ai/free/en/papers/zhao-2017-learning-discourse-level-diversity-neural/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 2017.
50. Hao Zhou, Tom Young, Minlie Huang, Haizhou Zhao, Jingfang Xu, and Xiaoyan Zhu. Commonsense knowledge aware conversation generation with graph attention. In International Joint Conference on Artificial Intelligence, IJCAI, 2018. [find](https://scholar.google.com/scholar?q=Hao+Zhou%2C+Tom+Young%2C+Minlie+Huang%2C+Haizhou+Zhao%2C+Jingfang+Xu%2C+and+Xiaoyan+Zhu%2E+Commonsense+knowledge+aware+conversation+generation+with+graph+attention%2E+In+International+Joint+Conference+on+Artificial)
51. Yaoming Zhu, Sidi Lu, Lei Zheng, Jiaxian Guo, Weinan Zhang, Jun Wang, and Yong Yu. [Texygen: A benchmarking platform for text generation models](https://md.research.elamur.ai/free/en/papers/zhu-2018-texygen-benchmarking-platform-text-generation/). In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pp. 1097–1100, 2018.

*Updated 2 November 2026*