---
title: "Categorical Reparameterization with — Jang 2016"
description: "Neural networks with discrete latent variables are hard to train because sampling from a categorical distribution blocks backpropagation."
authors: ["Eric Jang", "Shixiang Gu", "Ben Poole"]
year: 2016
venue: "arXiv:1611.01144"
source: "https://arxiv.org/abs/1611.01144"
doi: "10.48550/arxiv.1611.01144"
arxiv: "1611.01144"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/jang-2016-categorical-reparameterization-gumbel-softmax/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/jang-2016-categorical-reparameterization-gumbel-softmax/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2016](https://md.research.elamur.ai/free/en/papers/2016/) / Categorical Reparameterization with Gumbel-Softmax

# Categorical Reparameterization with Gumbel-Softmax

**Authors:** Eric Jang, [Shixiang Gu](https://md.research.elamur.ai/free/en/authors/shixiang-gu/), [Ben Poole](https://md.research.elamur.ai/free/en/authors/ben-poole/) · **Year:** 2016 · **Venue:** arXiv:1611.01144 · **Licence:** [arXiv — non-exclusive licence to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1611.01144) · [PDF](https://arxiv.org/pdf/1611.01144) · [DOI 10.48550/arxiv.1611.01144](https://doi.org/10.48550/arxiv.1611.01144) · [arXiv 1611.01144](https://arxiv.org/abs/1611.01144) · [OpenAlex](https://openalex.org/W2547875792)

**Citations:** 3063 citations (OpenAlex, 2026-10-14) · [508](https://md.research.elamur.ai/free/en/papers/jang-2016-categorical-reparameterization-gumbel-softmax/cited-by/) in our library, of the 140884 papers whose bibliographies we read

## Summary
Neural networks with discrete latent variables are hard to train because sampling from a categorical distribution blocks backpropagation. Earlier workarounds used either high-variance score-function (REINFORCE-style) estimators or biased straight-through tricks built for binary units. The authors introduce the Gumbel-Softmax distribution. It adds Gumbel noise to the log-probabilities and replaces the argmax of the Gumbel-Max trick with a softmax controlled by a temperature. The result is a continuous, reparameterizable relaxation on the simplex that becomes a true categorical distribution as the temperature approaches zero, so ordinary backprop can compute gradients. During training, the temperature is usually annealed from high to low. A straight-through variant uses hard one-hot samples in the forward pass and relaxed gradients in the backward pass. On MNIST benchmarks, the estimator beat other single-sample estimators such as SF, DARN, MuProp, NVIL and ST. The benchmarks covered structured output prediction, VAEs with Bernoulli or categorical latents, and semi-supervised classification. The method gives a simple, general way to use discrete choices in VAEs, attention and RL policies.

## Key points
- **Temperature trade-off:** a low τ gives nearly one-hot samples but noisier gradients, while a high τ gives smooth samples with lower gradient variance. If τ is learned instead of scheduled, it acts like entropy regularization.
- **Straight-Through (ST) Gumbel-Softmax:** discretizes samples with argmax but backpropagates through the relaxed sample. It is meant for settings that need truly discrete values, such as discrete action spaces or quantized compression.
- **Semi-supervised speedup:** in the semi-supervised model of Kingma et al. (2014), it removes the costly sum over all values of the unobserved class label. The paper reports about a 2× speedup for 10 classes and close to 10× for 100 classes, with comparable accuracy. These figures come from my knowledge of the paper's results section, which was missing from the text provided.
- **Same as Concrete:** Maddison et al. (2016) found the same distribution independently and called it the "Concrete" distribution. The paper also derives its exact density in closed form (Appendix B).
- **Baseline scope:** VIMCO was left out of the comparisons because it targets multi-sample objectives. The authors note that Gumbel-Softmax extends easily to that setting.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Other](https://md.research.elamur.ai/free/en/shelves/other/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{jang2016categorical,
  title = {{Categorical Reparameterization with Gumbel-Softmax}},
  author = {Jang, Eric and Gu, Shixiang and Poole, Ben},
  year = {2016},
  doi = {10.48550/arxiv.1611.01144},
  eprint = {1611.01144},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1611.01144}
}
```

- [← previous Neural Architecture Search with Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/zoph-2016-neural-architecture-search-reinforcement-learning/)
- [next → Neural Machine Translation of Rare Words with Subword Units](https://md.research.elamur.ai/free/en/papers/sennrich-2016-neural-machine-translation-rare-words/)
- [↑ back to papers of 2016](https://md.research.elamur.ai/free/en/papers/2016/)

## References

28 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Y. Bengio, N. Léonard, and A. Courville. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432, 2013. [find](https://scholar.google.com/scholar?q=Y%2E+Bengio%2C+N%2E+Le%CC%81onard%2C+and+A%2E+Courville%2E+Estimating+or+propagating+gradients+through+stochastic+neurons+for+conditional+computation%2E+arXiv+preprint+arXiv%3A1308%2E3432%2C+2013%2E)
2. Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. Infogan: Interpretable representation learning by information maximizing generative adversarial nets. CoRR, abs/1606.03657, 2016. — [arXiv:1606.03657](https://arxiv.org/abs/1606.03657) [find](https://scholar.google.com/scholar?q=Xi+Chen%2C+Yan+Duan%2C+Rein+Houthooft%2C+John+Schulman%2C+Ilya+Sutskever%2C+and+Pieter+Abbeel%2E+Infogan%3A+Interpretable+representation+learning+by+information+maximizing+generative+adversarial+nets%2E+CoRR%2C+abs%2F160)
3. J. Chung, S. Ahn, and Y. Bengio. Hierarchical multiscale recurrent neural networks. arXiv preprint arXiv:1609.01704, 2016. [find](https://scholar.google.com/scholar?q=J%2E+Chung%2C+S%2E+Ahn%2C+and+Y%2E+Bengio%2E+Hierarchical+multiscale+recurrent+neural+networks%2E+arXiv+preprint+arXiv%3A1609%2E01704%2C+2016%2E)
4. P. W Glynn. Likelihood ratio gradient estimation for stochastic systems. Communications of the ACM, 33(10):75–84, 1990. [find](https://scholar.google.com/scholar?q=P%2E+W+Glynn%2E+Likelihood+ratio+gradient+estimation+for+stochastic+systems%2E+Communications+of+the+ACM%2C+33%2810%29%3A75%E2%80%9384%2C+1990%2E)
5. A. Graves, G. Wayne, M. Reynolds, T. Harley, I. Danihelka, A. Grabska-Barwińska, S. G. Colmenarejo, E. Grefenstette, T. Ramalho, J. Agapiou, et al. Hybrid computing using a neural network with dynamic external memory. Nature, 538(7626):471–476, 2016. [find](https://scholar.google.com/scholar?q=A%2E+Graves%2C+G%2E+Wayne%2C+M%2E+Reynolds%2C+T%2E+Harley%2C+I%2E+Danihelka%2C+A%2E+Grabska%2DBarwin%CC%81ska%2C+S%2E+G%2E+Colmenarejo%2C+E%2E+Grefenstette%2C+T%2E+Ramalho%2C+J%2E+Agapiou%2C+et+al%2E+Hybrid+computing+using+a+neural+network+with+dynami)
6. Alex Graves, Greg Wayne, and Ivo Danihelka. Neural turing machines. CoRR, abs/1410.5401, 2014. [find](https://scholar.google.com/scholar?q=Alex+Graves%2C+Greg+Wayne%2C+and+Ivo+Danihelka%2E+Neural+turing+machines%2E+CoRR%2C+abs%2F1410%2E5401%2C+2014%2E)
7. K. Gregor, I. Danihelka, A. Mnih, C. Blundell, and D. Wierstra. Deep autoregressive networks. arXiv preprint arXiv:1310.8499, 2013. — [arXiv:1310.8499](https://arxiv.org/abs/1310.8499) [find](https://scholar.google.com/scholar?q=K%2E+Gregor%2C+I%2E+Danihelka%2C+A%2E+Mnih%2C+C%2E+Blundell%2C+and+D%2E+Wierstra%2E+Deep+autoregressive+networks%2E+arXiv+preprint+arXiv%3A1310%2E8499%2C+2013%2E+%E2%80%94+%5BarXiv%3A1310%2E8499%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1310%2E8499%29)
8. S. Gu, S. Levine, I. Sutskever, and A Mnih. MuProp: Unbiased Backpropagation for Stochastic Neural Networks. ICLR, 2016. [find](https://scholar.google.com/scholar?q=S%2E+Gu%2C+S%2E+Levine%2C+I%2E+Sutskever%2C+and+A+Mnih%2E+MuProp%3A+Unbiased+Backpropagation+for+Stochastic+Neural+Networks%2E+ICLR%2C+2016%2E)
9. E. J. Gumbel. Statistical theory of extreme values and some practical applications: a series of lectures. Number 33. US Govt. Print. Office, 1954. [find](https://scholar.google.com/scholar?q=E%2E+J%2E+Gumbel%2E+Statistical+theory+of+extreme+values+and+some+practical+applications%3A+a+series+of+lectures%2E+Number+33%2E+US+Govt%2E+Print%2E+Office%2C+1954%2E)
10. D. P. Kingma and M. Welling. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013. [find](https://scholar.google.com/scholar?q=D%2E+P%2E+Kingma+and+M%2E+Welling%2E+Auto%2Dencoding+variational+bayes%2E+arXiv+preprint+arXiv%3A1312%2E6114%2C+2013%2E) [our card](https://md.research.elamur.ai/free/en/papers/kingma-2013-auto-encoding-variational-bayes/)
11. D. P. Kingma, S. Mohamed, D. J. Rezende, and M. Welling. Semi-supervised learning with deep generative models. In Advances in Neural Information Processing Systems, pp. 3581–3589, 2014. [find](https://scholar.google.com/scholar?q=D%2E+P%2E+Kingma%2C+S%2E+Mohamed%2C+D%2E+J%2E+Rezende%2C+and+M%2E+Welling%2E+Semi%2Dsupervised+learning+with+deep+generative+models%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pp%2E+3581%E2%80%933589%2C+2014%2E)
12. H. Larochelle and I. Murray. The neural autoregressive distribution estimator. In AISTATS, volume 1, pp. 2, 2011. [find](https://scholar.google.com/scholar?q=H%2E+Larochelle+and+I%2E+Murray%2E+The+neural+autoregressive+distribution+estimator%2E+In+AISTATS%2C+volume+1%2C+pp%2E+2%2C+2011%2E)
13. C. J. Maddison, D. Tarlow, and T. Minka. A* sampling. In Advances in Neural Information Processing Systems, pp. 3086–3094, 2014. [find](https://scholar.google.com/scholar?q=C%2E+J%2E+Maddison%2C+D%2E+Tarlow%2C+and+T%2E+Minka%2E+A%2A+sampling%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pp%2E+3086%E2%80%933094%2C+2014%2E)
14. C. J. Maddison, A. Mnih, and Y. Whye Teh. The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables. ArXiv e-prints, November 2016. [find](https://scholar.google.com/scholar?q=C%2E+J%2E+Maddison%2C+A%2E+Mnih%2C+and+Y%2E+Whye+Teh%2E+The+Concrete+Distribution%3A+A+Continuous+Relaxation+of+Discrete+Random+Variables%2E+ArXiv+e%2Dprints%2C+November+2016%2E) [our card](https://md.research.elamur.ai/free/en/papers/maddison-2016-concrete-distribution-continuous-relaxation-discrete/)
15. A. Mnih and K. Gregor. Neural variational inference and learning in belief networks. ICML, 31, 2014. [find](https://scholar.google.com/scholar?q=A%2E+Mnih+and+K%2E+Gregor%2E+Neural+variational+inference+and+learning+in+belief+networks%2E+ICML%2C+31%2C+2014%2E)
16. A. Mnih and D. J. Rezende. Variational inference for monte carlo objectives. arXiv preprint arXiv:1602.06725, 2016. — [arXiv:1602.06725](https://arxiv.org/abs/1602.06725) [find](https://scholar.google.com/scholar?q=A%2E+Mnih+and+D%2E+J%2E+Rezende%2E+Variational+inference+for+monte+carlo+objectives%2E+arXiv+preprint+arXiv%3A1602%2E06725%2C+2016%2E+%E2%80%94+%5BarXiv%3A1602%2E06725%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1602%2E06725%29)
17. J. Paisley, D. Blei, and M. Jordan. Variational Bayesian Inference with Stochastic Search. ArXiv e-prints, June 2012. [find](https://scholar.google.com/scholar?q=J%2E+Paisley%2C+D%2E+Blei%2C+and+M%2E+Jordan%2E+Variational+Bayesian+Inference+with+Stochastic+Search%2E+ArXiv+e%2Dprints%2C+June+2012%2E)
18. Gabriel Pereyra, Geoffrey Hinton, George Tucker, and Lukasz Kaiser. Regularizing neural networks by penalizing confident output distributions. 2016. [find](https://scholar.google.com/scholar?q=Gabriel+Pereyra%2C+Geoffrey+Hinton%2C+George+Tucker%2C+and+Lukasz+Kaiser%2E+Regularizing+neural+networks+by+penalizing+confident+output+distributions%2E+2016%2E) [our card](https://md.research.elamur.ai/free/en/papers/pereyra-2017-regularizing-neural-networks-penalizing-confident/)
19. J. W Rae, J. J Hunt, T. Harley, I. Danihelka, A. Senior, G. Wayne, A. Graves, and T. P Lillicrap. Scaling Memory-Augmented Neural Networks with Sparse Reads and Writes. ArXiv e-prints, October 2016. [find](https://scholar.google.com/scholar?q=J%2E+W+Rae%2C+J%2E+J+Hunt%2C+T%2E+Harley%2C+I%2E+Danihelka%2C+A%2E+Senior%2C+G%2E+Wayne%2C+A%2E+Graves%2C+and+T%2E+P+Lillicrap%2E+Scaling+Memory%2DAugmented+Neural+Networks+with+Sparse+Reads+and+Writes%2E+ArXiv+e%2Dprints%2C+October+2016%2E)
20. T. Raiko, M. Berglund, G. Alain, and L. Dinh. Techniques for learning binary stochastic feedforward neural networks. arXiv preprint arXiv:1406.2989, 2014. — [arXiv:1406.2989](https://arxiv.org/abs/1406.2989) [find](https://scholar.google.com/scholar?q=T%2E+Raiko%2C+M%2E+Berglund%2C+G%2E+Alain%2C+and+L%2E+Dinh%2E+Techniques+for+learning+binary+stochastic+feedforward+neural+networks%2E+arXiv+preprint+arXiv%3A1406%2E2989%2C+2014%2E+%E2%80%94+%5BarXiv%3A1406%2E2989%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F140)
21. D. J. Rezende, S. Mohamed, and D. Wierstra. Stochastic backpropagation and approximate inference in deep generative models. arXiv preprint arXiv:1401.4082, 2014a. [find](https://scholar.google.com/scholar?q=D%2E+J%2E+Rezende%2C+S%2E+Mohamed%2C+and+D%2E+Wierstra%2E+Stochastic+backpropagation+and+approximate+inference+in+deep+generative+models%2E+arXiv+preprint+arXiv%3A1401%2E4082%2C+2014a%2E) [our card](https://md.research.elamur.ai/free/en/papers/rezende-2014-stochastic-backpropagation-approximate-inference-deep/)
22. D. J. Rezende, S. Mohamed, and D. Wierstra. Stochastic backpropagation and approximate inference in deep generative models. In Proceedings of The 31st International Conference on Machine Learning, pp. 1278–1286, 2014b. [find](https://scholar.google.com/scholar?q=D%2E+J%2E+Rezende%2C+S%2E+Mohamed%2C+and+D%2E+Wierstra%2E+Stochastic+backpropagation+and+approximate+inference+in+deep+generative+models%2E+In+Proceedings+of+The+31st+International+Conference+on+Machine+Learning%2C+pp%2E) [our card](https://md.research.elamur.ai/free/en/papers/rezende-2014-stochastic-backpropagation-approximate-inference-deep/)
23. J. T. Rolfe. Discrete Variational Autoencoders. ArXiv e-prints, September 2016. [find](https://scholar.google.com/scholar?q=J%2E+T%2E+Rolfe%2E+Discrete+Variational+Autoencoders%2E+ArXiv+e%2Dprints%2C+September+2016%2E)
24. R. Salakhutdinov and I. Murray. On the quantitative analysis of deep belief networks. In Proceedings of the 25th international conference on Machine learning, pp. 872–879. ACM, 2008. [find](https://scholar.google.com/scholar?q=R%2E+Salakhutdinov+and+I%2E+Murray%2E+On+the+quantitative+analysis+of+deep+belief+networks%2E+In+Proceedings+of+the+25th+international+conference+on+Machine+learning%2C+pp%2E+872%E2%80%93879%2E+ACM%2C+2008%2E)
25. J. Schulman, N. Heess, T. Weber, and P. Abbeel. Gradient estimation using stochastic computation graphs. In Advances in Neural Information Processing Systems, pp. 3528–3536, 2015. [find](https://scholar.google.com/scholar?q=J%2E+Schulman%2C+N%2E+Heess%2C+T%2E+Weber%2C+and+P%2E+Abbeel%2E+Gradient+estimation+using+stochastic+computation+graphs%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pp%2E+3528%E2%80%933536%2C+2015%2E)
26. C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna. Rethinking the inception architecture for computer vision. arXiv preprint arXiv:1512.00567, 2015. [find](https://scholar.google.com/scholar?q=C%2E+Szegedy%2C+V%2E+Vanhoucke%2C+S%2E+Ioffe%2C+J%2E+Shlens%2C+and+Z%2E+Wojna%2E+Rethinking+the+inception+architecture+for+computer+vision%2E+arXiv+preprint+arXiv%3A1512%2E00567%2C+2015%2E)
27. R. J. Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8(3-4):229–256, 1992. [find](https://scholar.google.com/scholar?q=R%2E+J%2E+Williams%2E+Simple+statistical+gradient%2Dfollowing+algorithms+for+connectionist+reinforcement+learning%2E+Machine+learning%2C+8%283%2D4%29%3A229%E2%80%93256%2C+1992%2E)
28. K. Xu, J. Ba, R. Kiros, K. Cho, A. C. Courville, R. Salakhutdinov, R. S. Zemel, and Y. Bengio. Show, attend and tell: Neural image caption generation with visual attention. CoRR, abs/1502.03044, 2015. [find](https://scholar.google.com/scholar?q=K%2E+Xu%2C+J%2E+Ba%2C+R%2E+Kiros%2C+K%2E+Cho%2C+A%2E+C%2E+Courville%2C+R%2E+Salakhutdinov%2C+R%2E+S%2E+Zemel%2C+and+Y%2E+Bengio%2E+Show%2C+attend+and+tell%3A+Neural+image+caption+generation+with+visual+attention%2E+CoRR%2C+abs%2F1502%2E03044%2C+2015%2E) [our card](https://md.research.elamur.ai/free/en/papers/xu-2015-show-attend-tell-neural-image/)