---
title: "Score-Based Generative Modeling through — Song 2020"
description: "Two families of generative models, SMLD and DDPM, add noise to data in a finite series of steps and learn to undo it."
authors: ["Yang Song", "Jascha Sohl-Dickstein", "Diederik P. Kingma", "Abhishek Kumar", "Stefano Ermon", "Ben Poole"]
year: 2020
venue: "arXiv:2011.13456"
source: "https://arxiv.org/abs/2011.13456"
doi: "10.48550/arxiv.2011.13456"
arxiv: "2011.13456"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/song-2020-score-based-generative-modeling-through/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/song-2020-score-based-generative-modeling-through/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/5/) / Score-Based Generative Modeling through Stochastic Differential Equations

# Score-Based Generative Modeling through Stochastic Differential Equations

**Authors:** [Yang Song](https://md.research.elamur.ai/free/en/authors/yang-song/), [Jascha Sohl-Dickstein](https://md.research.elamur.ai/free/en/authors/jascha-sohl-dickstein/), [Diederik P. Kingma](https://md.research.elamur.ai/free/en/authors/diederik-p-kingma/), [Abhishek Kumar](https://md.research.elamur.ai/free/en/authors/abhishek-kumar/), [Stefano Ermon](https://md.research.elamur.ai/free/en/authors/stefano-ermon/), [Ben Poole](https://md.research.elamur.ai/free/en/authors/ben-poole/) · **Year:** 2020 · **Venue:** arXiv:2011.13456 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2011.13456) · [PDF](https://arxiv.org/pdf/2011.13456) · [DOI 10.48550/arxiv.2011.13456](https://doi.org/10.48550/arxiv.2011.13456) · [arXiv 2011.13456](https://arxiv.org/abs/2011.13456) · [OpenAlex](https://openalex.org/W3110257065)

**Citations:** 1216 citations (openalex-snapshot · 2026-10-14) · [312](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Two families of generative models, SMLD and DDPM, add noise to data in a finite series of steps and learn to undo it. This paper treats both as special cases of a single continuous-time framework. A fixed SDE with no trainable parameters gradually turns data into a simple Gaussian prior. Generation runs the corresponding reverse-time SDE (Anderson, 1982) back toward the data. The only unknown in that reverse process is the score: the gradient of the log density of the noised data at each moment. The authors train one time-conditioned network with denoising score matching to estimate this score, then produce samples with numerical solvers.

The framework supports three new capabilities:
- **Predictor-Corrector samplers**, which follow each solver step with an MCMC refinement step.
- **A deterministic probability flow ODE**, which gives exact likelihoods and fast adaptive sampling.
- **Conditional tasks** (class-conditional generation, inpainting, colorization) using an unconditional model without retraining.

Combined with architectural upgrades, the approach set new CIFAR-10 records and scaled score-based generation to megapixel images. It became a shared theoretical foundation for later diffusion research.

## Key points
- SMLD and DDPM correspond to discretizations of two processes, the VE SDE and the VP SDE. The authors also propose a new sub-VP SDE, which produced their best likelihood result.
- With the NCSN++ architecture and PC sampling, unconditional CIFAR-10 generation reached an Inception score of 9.89 and an FID of 2.20, a new state of the art.
- The probability flow ODE reached 2.99 bits/dim on uniformly dequantized CIFAR-10, a record likelihood for that setting.
- This was the first score-based model to generate 1024×1024 images at high fidelity.
- Controllable generation was demonstrated in two ways:
  - Class-conditional CIFAR-10 sampling, guided by a noise-conditional classifier and using PC with 2000 discretization steps.
  - Inpainting and colorization of 256×256 bedroom and church images. Colorization works by applying an orthogonal transform that separates color channels.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Other](https://md.research.elamur.ai/free/en/shelves/other/)

## Cited by

Cited by

312 papers of our library cite it (of the 140673 whose bibliographies we read); 11 of them have a page here, 301 not yet.

- [Structured Denoising Diffusion Models in Discrete State-Spaces](https://md.research.elamur.ai/free/en/papers/austin-2021-structured-denoising-diffusion-models-discrete/)
- [RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment](https://md.research.elamur.ai/free/en/papers/dong-2023-raft-reward-ranked-finetuning-generative/)
- [Classifier-Free Diffusion Guidance](https://md.research.elamur.ai/free/en/papers/ho-2022-classifier-free-diffusion-guidance/)
- [Diffusion-LM Improves Controllable Text Generation](https://md.research.elamur.ai/free/en/papers/li-2022-diffusion-lm-improves-controllable-text/)
- [Flow Matching for Generative Modeling](https://md.research.elamur.ai/free/en/papers/lipman-2022-flow-matching-generative-modeling/)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://md.research.elamur.ai/free/en/papers/liu-2022-flow-straight-fast-learning-generate/)
- [Large Language Diffusion Models](https://md.research.elamur.ai/free/en/papers/nie-2025-large-language-diffusion-models/)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://md.research.elamur.ai/free/en/papers/podell-2023-sdxl-improving-latent-diffusion-models/)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://md.research.elamur.ai/free/en/papers/saharia-2022-photorealistic-text-image-diffusion-models/)
- [Simple and Effective Masked Diffusion Language Models](https://md.research.elamur.ai/free/en/papers/sahoo-2024-simple-effective-masked-diffusion-language/)
- [Denoising Diffusion Implicit Models](https://md.research.elamur.ai/free/en/papers/song-2020-denoising-diffusion-implicit-models/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{song2020score,
  title = {{Score-Based Generative Modeling through Stochastic Differential Equations}},
  author = {Song, Yang and Sohl-Dickstein, Jascha and Kingma, Diederik P. and Kumar, Abhishek and Ermon, Stefano and Poole, Ben},
  year = {2020},
  doi = {10.48550/arxiv.2011.13456},
  eprint = {2011.13456},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2011.13456}
}
```

- [← previous SciREX: A Challenge Dataset for Document-Level Information Extraction](https://md.research.elamur.ai/free/en/papers/jain-2020-scirex-challenge-dataset-document-level/)
- [next → Semantic differences in translation: Exploring the field of inchoativity](https://md.research.elamur.ai/free/en/papers/vandevoorde-2020-semantic-differences-translation-exploring/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/5/)

## References

51 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Eugene L Allgower and Kurt Georg. Numerical continuation methods: an introduction, volume 13. Springer Science & Business Media, 2012. [find](https://scholar.google.com/scholar?q=Eugene+L+Allgower+and+Kurt+Georg%2E+Numerical+continuation+methods%3A+an+introduction%2C+volume+13%2E+Springer+Science+%26+Business+Media%2C+2012%2E)
2. Brian D O Anderson. Reverse-time diffusion equation models. Stochastic Process. Appl., 12(3): 313–326, May 1982. [find](https://scholar.google.com/scholar?q=Brian+D+O+Anderson%2E+Reverse%2Dtime+diffusion+equation+models%2E+Stochastic+Process%2E+Appl%2E%2C+12%283%29%3A+313%E2%80%93326%2C+May+1982%2E)
3. Jens Behrmann, Will Grathwohl, Ricky TQ Chen, David Duvenaud, and Jörn-Henrik Jacobsen. Invertible residual networks. In International Conference on Machine Learning, pp. 573–582, 2019. [find](https://scholar.google.com/scholar?q=Jens+Behrmann%2C+Will+Grathwohl%2C+Ricky+TQ+Chen%2C+David+Duvenaud%2C+and+Jo%CC%88rn%2DHenrik+Jacobsen%2E+Invertible+residual+networks%2E+In+International+Conference+on+Machine+Learning%2C+pp%2E+573%E2%80%93582%2C+2019%2E)
4. Florian Bordes, Sina Honari, and Pascal Vincent. Learning to generate samples from noise through infusion training. arXiv preprint arXiv:1703.06975, 2017. — [arXiv:1703.06975](https://arxiv.org/abs/1703.06975) [find](https://scholar.google.com/scholar?q=Florian+Bordes%2C+Sina+Honari%2C+and+Pascal+Vincent%2E+Learning+to+generate+samples+from+noise+through+infusion+training%2E+arXiv+preprint+arXiv%3A1703%2E06975%2C+2017%2E+%E2%80%94+%5BarXiv%3A1703%2E06975%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F17)
5. Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high fidelity natural image synthesis. In International Conference on Learning Representations, 2018. [find](https://scholar.google.com/scholar?q=Andrew+Brock%2C+Jeff+Donahue%2C+and+Karen+Simonyan%2E+Large+scale+gan+training+for+high+fidelity+natural+image+synthesis%2E+In+International+Conference+on+Learning+Representations%2C+2018%2E)
6. Ruojin Cai, Guandao Yang, Hadar Averbuch-Elor, Zekun Hao, Serge Belongie, Noah Snavely, and Bharath Hariharan. Learning gradient fields for shape generation. In Proceedings of the European Conference on Computer Vision (ECCV), 2020. [find](https://scholar.google.com/scholar?q=Ruojin+Cai%2C+Guandao+Yang%2C+Hadar+Averbuch%2DElor%2C+Zekun+Hao%2C+Serge+Belongie%2C+Noah+Snavely%2C+and+Bharath+Hariharan%2E+Learning+gradient+fields+for+shape+generation%2E+In+Proceedings+of+the+European+Conference+)
7. Nanxin Chen, Yu Zhang, Heiga Zen, Ron J Weiss, Mohammad Norouzi, and William Chan. Wavegrad: Estimating gradients for waveform generation. arXiv preprint arXiv:2009.00713, 2020. — [arXiv:2009.00713](https://arxiv.org/abs/2009.00713) [find](https://scholar.google.com/scholar?q=Nanxin+Chen%2C+Yu+Zhang%2C+Heiga+Zen%2C+Ron+J+Weiss%2C+Mohammad+Norouzi%2C+and+William+Chan%2E+Wavegrad%3A+Estimating+gradients+for+waveform+generation%2E+arXiv+preprint+arXiv%3A2009%2E00713%2C+2020%2E+%E2%80%94+%5BarXiv%3A2009%2E00713%5D%28h)
8. Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differential equations. In Advances in neural information processing systems, pp. 6571–6583, 2018. [find](https://scholar.google.com/scholar?q=Ricky+TQ+Chen%2C+Yulia+Rubanova%2C+Jesse+Bettencourt%2C+and+David+K+Duvenaud%2E+Neural+ordinary+differential+equations%2E+In+Advances+in+neural+information+processing+systems%2C+pp%2E+6571%E2%80%936583%2C+2018%2E)
9. Ricky TQ Chen, Jens Behrmann, David K Duvenaud, and Jörn-Henrik Jacobsen. Residual flows for invertible generative modeling. In Advances in Neural Information Processing Systems, pp. 9916–9926, 2019. [find](https://scholar.google.com/scholar?q=Ricky+TQ+Chen%2C+Jens+Behrmann%2C+David+K+Duvenaud%2C+and+Jo%CC%88rn%2DHenrik+Jacobsen%2E+Residual+flows+for+invertible+generative+modeling%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pp%2E+9916%E2%80%939926%2C+2019%2E)
10. Laurent Dinh, Jascha Sohl-Dickstein, and Samy Bengio. Density estimation using real nvp. arXiv preprint arXiv:1605.08803, 2016. — [arXiv:1605.08803](https://arxiv.org/abs/1605.08803) [find](https://scholar.google.com/scholar?q=Laurent+Dinh%2C+Jascha+Sohl%2DDickstein%2C+and+Samy+Bengio%2E+Density+estimation+using+real+nvp%2E+arXiv+preprint+arXiv%3A1605%2E08803%2C+2016%2E+%E2%80%94+%5BarXiv%3A1605%2E08803%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1605%2E08803%29)
11. John R Dormand and Peter J Prince. A family of embedded runge-kutta formulae. Journal of computational and applied mathematics, 6(1):19–26, 1980. [find](https://scholar.google.com/scholar?q=John+R+Dormand+and+Peter+J+Prince%2E+A+family+of+embedded+runge%2Dkutta+formulae%2E+Journal+of+computational+and+applied+mathematics%2C+6%281%29%3A19%E2%80%9326%2C+1980%2E)
12. Yilun Du and Igor Mordatch. Implicit generation and modeling with energy based models. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 32, pp. 3608–3618. Curran Associates, Inc., 2019. [find](https://scholar.google.com/scholar?q=Yilun+Du+and+Igor+Mordatch%2E+Implicit+generation+and+modeling+with+energy+based+models%2E+In+H%2E+Wallach%2C+H%2E+Larochelle%2C+A%2E+Beygelzimer%2C+F%2E+d%27Alche%CC%81%2DBuc%2C+E%2E+Fox%2C+and+R%2E+Garnett+%28eds%2E%29%2C+Advances+in+Neural+)
13. Bradley Efron. Tweedie’s formula and selection bias. Journal of the American Statistical Association, 106(496):1602–1614, 2011. [find](https://scholar.google.com/scholar?q=Bradley+Efron%2E+Tweedie%E2%80%99s+formula+and+selection+bias%2E+Journal+of+the+American+Statistical+Association%2C+106%28496%29%3A1602%E2%80%931614%2C+2011%2E)
14. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural information processing systems, pp. 2672–2680, 2014. [find](https://scholar.google.com/scholar?q=Ian+Goodfellow%2C+Jean+Pouget%2DAbadie%2C+Mehdi+Mirza%2C+Bing+Xu%2C+David+Warde%2DFarley%2C+Sherjil+Ozair%2C+Aaron+Courville%2C+and+Yoshua+Bengio%2E+Generative+adversarial+nets%2E+In+Advances+in+neural+information+processi)
15. Anirudh Goyal Alias Parth Goyal, Nan Rosemary Ke, Surya Ganguli, and Yoshua Bengio. Variational walkback: Learning a transition operator as a stochastic recurrent net. In Advances in Neural Information Processing Systems, pp. 4392–4402, 2017. [find](https://scholar.google.com/scholar?q=Anirudh+Goyal+Alias+Parth+Goyal%2C+Nan+Rosemary+Ke%2C+Surya+Ganguli%2C+and+Yoshua+Bengio%2E+Variational+walkback%3A+Learning+a+transition+operator+as+a+stochastic+recurrent+net%2E+In+Advances+in+Neural+Informatio)
16. Will Grathwohl, Ricky TQ Chen, Jesse Bettencourt, Ilya Sutskever, and David Duvenaud. Ffjord: Free-form continuous dynamics for scalable reversible generative models. In International Conference on Learning Representations, 2018. [find](https://scholar.google.com/scholar?q=Will+Grathwohl%2C+Ricky+TQ+Chen%2C+Jesse+Bettencourt%2C+Ilya+Sutskever%2C+and+David+Duvenaud%2E+Ffjord%3A+Free%2Dform+continuous+dynamics+for+scalable+reversible+generative+models%2E+In+International+Conference+on+Le)
17. Ulf Grenander and Michael I Miller. Representations of knowledge in complex systems. Journal of the Royal Statistical Society: Series B (Methodological), 56(4):549–581, 1994. [find](https://scholar.google.com/scholar?q=Ulf+Grenander+and+Michael+I+Miller%2E+Representations+of+knowledge+in+complex+systems%2E+Journal+of+the+Royal+Statistical+Society%3A+Series+B+%28Methodological%29%2C+56%284%29%3A549%E2%80%93581%2C+1994%2E)
18. Jonathan Ho, Xi Chen, Aravind Srinivas, Yan Duan, and Pieter Abbeel. Flow++: Improving flow-based generative models with variational dequantization and architecture design. In International Conference on Machine Learning, pp. 2722–2730, 2019. [find](https://scholar.google.com/scholar?q=Jonathan+Ho%2C+Xi+Chen%2C+Aravind+Srinivas%2C+Yan+Duan%2C+and+Pieter+Abbeel%2E+Flow%2B%2B%3A+Improving+flow%2Dbased+generative+models+with+variational+dequantization+and+architecture+design%2E+In+International+Conference)
19. Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33, 2020. [find](https://scholar.google.com/scholar?q=Jonathan+Ho%2C+Ajay+Jain%2C+and+Pieter+Abbeel%2E+Denoising+diffusion+probabilistic+models%2E+Advances+in+Neural+Information+Processing+Systems%2C+33%2C+2020%2E)
20. Michael F Hutchinson. A stochastic estimator of the trace of the influence matrix for Laplacian smoothing splines. Communications in Statistics-Simulation and Computation, 19(2):433–450, 1990. [find](https://scholar.google.com/scholar?q=Michael+F+Hutchinson%2E+A+stochastic+estimator+of+the+trace+of+the+influence+matrix+for+Laplacian+smoothing+splines%2E+Communications+in+Statistics%2DSimulation+and+Computation%2C+19%282%29%3A433%E2%80%93450%2C+1990%2E)
21. Aapo Hyvärinen. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 6(Apr):695–709, 2005. [find](https://scholar.google.com/scholar?q=Aapo+Hyva%CC%88rinen%2E+Estimation+of+non%2Dnormalized+statistical+models+by+score+matching%2E+Journal+of+Machine+Learning+Research%2C+6%28Apr%29%3A695%E2%80%93709%2C+2005%2E)
22. Alexia Jolicoeur-Martineau, Rémi Piché-Taillefer, Rémi Tachet des Combes, and Ioannis Mitliagkas. Adversarial score matching and improved sampling for image generation. arXiv preprint arXiv:2009.05475, 2020. — [arXiv:2009.05475](https://arxiv.org/abs/2009.05475) [find](https://scholar.google.com/scholar?q=Alexia+Jolicoeur%2DMartineau%2C+Re%CC%81mi+Piche%CC%81%2DTaillefer%2C+Re%CC%81mi+Tachet+des+Combes%2C+and+Ioannis+Mitliagkas%2E+Adversarial+score+matching+and+improved+sampling+for+image+generation%2E+arXiv+preprint+arXiv%3A2009%2E05)
23. Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation. In International Conference on Learning Representations, 2018. [find](https://scholar.google.com/scholar?q=Tero+Karras%2C+Timo+Aila%2C+Samuli+Laine%2C+and+Jaakko+Lehtinen%2E+Progressive+growing+of+gans+for+improved+quality%2C+stability%2C+and+variation%2E+In+International+Conference+on+Learning+Representations%2C+2018%2E)
24. Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4401–4410, 2019. [find](https://scholar.google.com/scholar?q=Tero+Karras%2C+Samuli+Laine%2C+and+Timo+Aila%2E+A+style%2Dbased+generator+architecture+for+generative+adversarial+networks%2E+In+Proceedings+of+the+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition%2C+pp)
25. Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Training generative adversarial networks with limited data. Advances in Neural Information Processing Systems, 33, 2020a. [find](https://scholar.google.com/scholar?q=Tero+Karras%2C+Miika+Aittala%2C+Janne+Hellsten%2C+Samuli+Laine%2C+Jaakko+Lehtinen%2C+and+Timo+Aila%2E+Training+generative+adversarial+networks+with+limited+data%2E+Advances+in+Neural+Information+Processing+Systems%2C)
26. Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of StyleGAN. In Proc. CVPR, 2020b. [find](https://scholar.google.com/scholar?q=Tero+Karras%2C+Samuli+Laine%2C+Miika+Aittala%2C+Janne+Hellsten%2C+Jaakko+Lehtinen%2C+and+Timo+Aila%2E+Analyzing+and+improving+the+image+quality+of+StyleGAN%2E+In+Proc%2E+CVPR%2C+2020b%2E)
27. Durk P Kingma and Prafulla Dhariwal. Glow: Generative flow with invertible 1x1 convolutions. In Advances in Neural Information Processing Systems, pp. 10215–10224, 2018. [find](https://scholar.google.com/scholar?q=Durk+P+Kingma+and+Prafulla+Dhariwal%2E+Glow%3A+Generative+flow+with+invertible+1x1+convolutions%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pp%2E+10215%E2%80%9310224%2C+2018%2E)
28. Peter E Kloeden and Eckhard Platen. Numerical solution of stochastic differential equations, volume 23. Springer Science & Business Media, 2013. [find](https://scholar.google.com/scholar?q=Peter+E+Kloeden+and+Eckhard+Platen%2E+Numerical+solution+of+stochastic+differential+equations%2C+volume+23%2E+Springer+Science+%26+Business+Media%2C+2013%2E)
29. Zhifeng Kong, Wei Ping, Jiaji Huang, Kexin Zhao, and Bryan Catanzaro. Diffwave: A versatile diffusion model for audio synthesis. arXiv preprint arXiv:2009.09761, 2020. — [arXiv:2009.09761](https://arxiv.org/abs/2009.09761) [find](https://scholar.google.com/scholar?q=Zhifeng+Kong%2C+Wei+Ping%2C+Jiaji+Huang%2C+Kexin+Zhao%2C+and+Bryan+Catanzaro%2E+Diffwave%3A+A+versatile+diffusion+model+for+audio+synthesis%2E+arXiv+preprint+arXiv%3A2009%2E09761%2C+2020%2E+%E2%80%94+%5BarXiv%3A2009%2E09761%5D%28https%3A%2F%2Farx)
30. Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009. [find](https://scholar.google.com/scholar?q=Alex+Krizhevsky%2C+Geoffrey+Hinton%2C+et+al%2E+Learning+multiple+layers+of+features+from+tiny+images%2E+2009%2E)
31. Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015. [find](https://scholar.google.com/scholar?q=Ziwei+Liu%2C+Ping+Luo%2C+Xiaogang+Wang%2C+and+Xiaoou+Tang%2E+Deep+learning+face+attributes+in+the+wild%2E+In+Proceedings+of+International+Conference+on+Computer+Vision+%28ICCV%29%2C+December+2015%2E)
32. Dimitra Maoutsa, Sebastian Reich, and Manfred Opper. Interacting particle solutions of fokker-planck equations through gradient-log-density estimation. arXiv preprint arXiv:2006.00702, 2020. — [arXiv:2006.00702](https://arxiv.org/abs/2006.00702) [find](https://scholar.google.com/scholar?q=Dimitra+Maoutsa%2C+Sebastian+Reich%2C+and+Manfred+Opper%2E+Interacting+particle+solutions+of+fokker%2Dplanck+equations+through+gradient%2Dlog%2Ddensity+estimation%2E+arXiv+preprint+arXiv%3A2006%2E00702%2C+2020%2E+%E2%80%94+%5BarXiv%3A)
33. Radford M Neal et al. Mcmc using hamiltonian dynamics. Handbook of markov chain monte carlo, 2(11):2, 2011. [find](https://scholar.google.com/scholar?q=Radford+M+Neal+et+al%2E+Mcmc+using+hamiltonian+dynamics%2E+Handbook+of+markov+chain+monte+carlo%2C+2%2811%29%3A2%2C+2011%2E)
34. Chenhao Niu, Yang Song, Jiaming Song, Shengjia Zhao, Aditya Grover, and Stefano Ermon. Permutation invariant graph generation via score-based generative modeling. volume 108 of Proceedings of Machine Learning Research, pp. 4474–4484, Online, 26–28 Aug 2020. PMLR. [find](https://scholar.google.com/scholar?q=Chenhao+Niu%2C+Yang+Song%2C+Jiaming+Song%2C+Shengjia+Zhao%2C+Aditya+Grover%2C+and+Stefano+Ermon%2E+Permutation+invariant+graph+generation+via+score%2Dbased+generative+modeling%2E+volume+108+of+Proceedings+of+Machine+)
35. Bernt Øksendal. Stochastic differential equations. In Stochastic differential equations, pp. 65–84. Springer, 2003. [find](https://scholar.google.com/scholar?q=Bernt+%C3%98ksendal%2E+Stochastic+differential+equations%2E+In+Stochastic+differential+equations%2C+pp%2E+65%E2%80%9384%2E+Springer%2C+2003%2E)
36. Tianyu Pang, Kun Xu, Chongxuan Li, Yang Song, Stefano Ermon, and Jun Zhu. Efficient learning of generative models via finite-difference score matching. arXiv preprint arXiv:2007.03317, 2020. — [arXiv:2007.03317](https://arxiv.org/abs/2007.03317) [find](https://scholar.google.com/scholar?q=Tianyu+Pang%2C+Kun+Xu%2C+Chongxuan+Li%2C+Yang+Song%2C+Stefano+Ermon%2C+and+Jun+Zhu%2E+Efficient+learning+of+generative+models+via+finite%2Ddifference+score+matching%2E+arXiv+preprint+arXiv%3A2007%2E03317%2C+2020%2E+%E2%80%94+%5BarXiv%3A)
37. Giorgio Parisi. Correlation functions and computer simulations. Nuclear Physics B, 180(3):378–384, 1981. [find](https://scholar.google.com/scholar?q=Giorgio+Parisi%2E+Correlation+functions+and+computer+simulations%2E+Nuclear+Physics+B%2C+180%283%29%3A378%E2%80%93384%2C+1981%2E)
38. Ali Razavi, Aaron van den Oord, and Oriol Vinyals. Generating diverse high-fidelity images with vq-vae-2. In Advances in Neural Information Processing Systems, pp. 14837–14847, 2019. [find](https://scholar.google.com/scholar?q=Ali+Razavi%2C+Aaron+van+den+Oord%2C+and+Oriol+Vinyals%2E+Generating+diverse+high%2Dfidelity+images+with+vq%2Dvae%2D2%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pp%2E+14837%E2%80%9314847%2C+2019%2E)
39. Geoffrey Roeder, Luke Metz, and Diederik P Kingma. On linear identifiability of learned representations. arXiv preprint arXiv:2007.00810, 2020. — [arXiv:2007.00810](https://arxiv.org/abs/2007.00810) [find](https://scholar.google.com/scholar?q=Geoffrey+Roeder%2C+Luke+Metz%2C+and+Diederik+P+Kingma%2E+On+linear+identifiability+of+learned+representations%2E+arXiv+preprint+arXiv%3A2007%2E00810%2C+2020%2E+%E2%80%94+%5BarXiv%3A2007%2E00810%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2007%2E00810%29)
40. Simo Särkkä and Arno Solin. Applied stochastic differential equations, volume 10. Cambridge University Press, 2019. [find](https://scholar.google.com/scholar?q=Simo+Sa%CC%88rkka%CC%88+and+Arno+Solin%2E+Applied+stochastic+differential+equations%2C+volume+10%2E+Cambridge+University+Press%2C+2019%2E)
41. John Skilling. The eigenvalues of mega-dimensional matrices. In Maximum Entropy and Bayesian Methods, pp. 455–466. Springer, 1989. [find](https://scholar.google.com/scholar?q=John+Skilling%2E+The+eigenvalues+of+mega%2Ddimensional+matrices%2E+In+Maximum+Entropy+and+Bayesian+Methods%2C+pp%2E+455%E2%80%93466%2E+Springer%2C+1989%2E)
42. Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning, pp. 2256–2265, 2015. [find](https://scholar.google.com/scholar?q=Jascha+Sohl%2DDickstein%2C+Eric+Weiss%2C+Niru+Maheswaranathan%2C+and+Surya+Ganguli%2E+Deep+unsupervised+learning+using+nonequilibrium+thermodynamics%2E+In+International+Conference+on+Machine+Learning%2C+pp%2E+2256%E2%80%9322) [our card](https://md.research.elamur.ai/free/en/papers/sohl-2015-dickstein-deep-unsupervised-learning-using-nonequilibrium/)
43. Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In Advances in Neural Information Processing Systems, pp. 11895–11907, 2019. [find](https://scholar.google.com/scholar?q=Yang+Song+and+Stefano+Ermon%2E+Generative+modeling+by+estimating+gradients+of+the+data+distribution%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pp%2E+11895%E2%80%9311907%2C+2019%2E) [our card](https://md.research.elamur.ai/free/en/papers/song-2019-generative-modeling-estimating-gradients-data/)
44. Yang Song and Stefano Ermon. Improved techniques for training score-based generative models. Advances in Neural Information Processing Systems, 33, 2020. [find](https://scholar.google.com/scholar?q=Yang+Song+and+Stefano+Ermon%2E+Improved+techniques+for+training+score%2Dbased+generative+models%2E+Advances+in+Neural+Information+Processing+Systems%2C+33%2C+2020%2E)
45. Yang Song, Sahaj Garg, Jiaxin Shi, and Stefano Ermon. Sliced score matching: A scalable approach to density and score estimation. In Proceedings of the Thirty-Fifth Conference on Uncertainty in Artificial Intelligence, UAI 2019, Tel Aviv, Israel, July 22-25, 2019, pp. 204, 2019a. [find](https://scholar.google.com/scholar?q=Yang+Song%2C+Sahaj+Garg%2C+Jiaxin+Shi%2C+and+Stefano+Ermon%2E+Sliced+score+matching%3A+A+scalable+approach+to+density+and+score+estimation%2E+In+Proceedings+of+the+Thirty%2DFifth+Conference+on+Uncertainty+in+Artifi)
46. Yang Song, Chenlin Meng, and Stefano Ermon. Mintnet: Building invertible neural networks with masked convolutions. In Advances in Neural Information Processing Systems, pp. 11002–11012, 2019b. [find](https://scholar.google.com/scholar?q=Yang+Song%2C+Chenlin+Meng%2C+and+Stefano+Ermon%2E+Mintnet%3A+Building+invertible+neural+networks+with+masked+convolutions%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pp%2E+11002%E2%80%9311012%2C+2019b%2E)
47. Matthew Tancik, Pratul P. Srinivasan, Ben Mildenhall, Sara Fridovich-Keil, Nithin Raghavan, Utkarsh Singhal, Ravi Ramamoorthi, Jonathan T. Barron, and Ren Ng. Fourier features let networks learn high frequency functions in low dimensional domains. NeurIPS, 2020. [find](https://scholar.google.com/scholar?q=Matthew+Tancik%2C+Pratul+P%2E+Srinivasan%2C+Ben+Mildenhall%2C+Sara+Fridovich%2DKeil%2C+Nithin+Raghavan%2C+Utkarsh+Singhal%2C+Ravi+Ramamoorthi%2C+Jonathan+T%2E+Barron%2C+and+Ren+Ng%2E+Fourier+features+let+networks+learn+high+)
48. Pascal Vincent. A connection between score matching and denoising autoencoders. Neural computation, 23(7):1661–1674, 2011. [find](https://scholar.google.com/scholar?q=Pascal+Vincent%2E+A+connection+between+score+matching+and+denoising+autoencoders%2E+Neural+computation%2C+23%287%29%3A1661%E2%80%931674%2C+2011%2E)
49. Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao. Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365, 2015. — [arXiv:1506.03365](https://arxiv.org/abs/1506.03365) [find](https://scholar.google.com/scholar?q=Fisher+Yu%2C+Ari+Seff%2C+Yinda+Zhang%2C+Shuran+Song%2C+Thomas+Funkhouser%2C+and+Jianxiong+Xiao%2E+Lsun%3A+Construction+of+a+large%2Dscale+image+dataset+using+deep+learning+with+humans+in+the+loop%2E+arXiv+preprint+arXi)
50. Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. arXiv preprint arXiv:1605.07146, 2016. — [arXiv:1605.07146](https://arxiv.org/abs/1605.07146) [find](https://scholar.google.com/scholar?q=Sergey+Zagoruyko+and+Nikos+Komodakis%2E+Wide+residual+networks%2E+arXiv+preprint+arXiv%3A1605%2E07146%2C+2016%2E+%E2%80%94+%5BarXiv%3A1605%2E07146%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1605%2E07146%29)
51. Richard Zhang. Making convolutional networks shift-invariant again. In ICML, 2019. [find](https://scholar.google.com/scholar?q=Richard+Zhang%2E+Making+convolutional+networks+shift%2Dinvariant+again%2E+In+ICML%2C+2019%2E)