---
title: "Overcoming Catastrophic Forgetting by Incremental Moment"
description: "The paper offers two variants: Mean-IMM takes a weighted average of the networks' weights. 34 references, cited by 43 papers here."
authors: ["Sang-Woo Lee", "Jin-Hwa Kim", "Jaehyun Jun", "Jung-Woo Ha", "Byoung-Tak Zhang"]
year: 2017
venue: "arXiv:1703.08475"
source: "https://arxiv.org/abs/1703.08475"
doi: "10.48550/arxiv.1703.08475"
arxiv: "1703.08475"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/lee-2017-overcoming-catastrophic-forgetting-incremental-moment/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/lee-2017-overcoming-catastrophic-forgetting-incremental-moment/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2017](https://md.research.elamur.ai/free/en/papers/2017/page/2/) / Overcoming Catastrophic Forgetting by Incremental Moment Matching

# Overcoming Catastrophic Forgetting by Incremental Moment Matching

**Authors:** [Sang-Woo Lee](https://md.research.elamur.ai/free/en/authors/sang-woo-lee/), [Jin-Hwa Kim](https://md.research.elamur.ai/free/en/authors/jin-hwa-kim/), Jaehyun Jun, [Jung-Woo Ha](https://md.research.elamur.ai/free/en/authors/jung-woo-ha/) and [Byoung-Tak Zhang](https://md.research.elamur.ai/free/en/authors/byoung-tak-zhang/) · **Year:** 2017 · **Venue:** arXiv:1703.08475 · **Licence:** [arXiv — non-exclusive licence to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1703.08475) · [PDF](https://arxiv.org/pdf/1703.08475) · [DOI 10.48550/arxiv.1703.08475](https://doi.org/10.48550/arxiv.1703.08475) · [arXiv 1703.08475](https://arxiv.org/abs/1703.08475) · [OpenAlex](https://openalex.org/W2605043629)

**Citations:** 255 citations (OpenAlex, 2026-10-14) · [43](https://md.research.elamur.ai/free/en/papers/lee-2017-overcoming-catastrophic-forgetting-incremental-moment/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
When a neural network is trained on tasks one after another, it tends to lose earlier knowledge once it fits a new task. This is known as catastrophic forgetting. Lee et al. (NIPS 2017) address it with incremental moment matching (IMM). IMM treats each task's trained network as a Gaussian posterior over the parameters and then merges these posteriors into one Gaussian that covers all tasks.

The paper offers two variants:
- **Mean-IMM** takes a weighted average of the networks' weights.
- **Mode-IMM** also scales each parameter by its precision, estimated with a diagonal Fisher information matrix, to approximate the peak of the Gaussian mixture.

Merging only works if the loss landscape between the two solutions has no high barriers. To achieve this, the authors combine IMM with weight-transfer, L2-transfer and a new dropout variant called drop-transfer.

Experiments on MNIST variants, CIFAR-10, Caltech-UCSD-Birds and the Lifelog dataset give state-of-the-art or competitive results with a single network. A practical benefit is that the balance between old and new tasks can be adjusted after training by changing the mixing ratios.

## Key points
- On the ImageNet→CUB transfer with AlexNet, mode-IMM (α = 0.99) scores 55.10 on ImageNet and 59.12 on CUB. This is +0.02 and +1.35 over the previous best result, which was an LwF model.
- Lifelog contains 660,000 egocentric video instances recorded with Google Glass by three participants over 46 days. Frames are labeled for location, sub-location and activity. Both IMM variants rival the dual-memory ensemble while using only one network.
- The diagonal covariance assumption cuts storage from O(D²) to O(D). For example, an MLP of [784-800-800-800-10] has D = 1,917,610 parameters.
- Weight-transfer was critical to performance. Transfer techniques push the optimal α above 1/k, and mode-IMM proved more robust to this shift than mean-IMM.
- The authors attribute EWC's failure on disjoint MNIST to its reliance on a diagonal Fisher matrix.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Seoul National University](https://md.research.elamur.ai/free/en/universities/seoul-national-university/) — University, Seoul, South Korea · 43 works · 45,969 citations
- [Naver](https://md.research.elamur.ai/free/en/companies/naver/) — Company, Seongnam-si, South Korea · 20 works · 12,146 citations
- [NAVER Cloud](https://md.research.elamur.ai/free/en/companies/naver-cloud/) — Company, Seongnam-si, South Korea · 16 works · 11,005 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works · [🧠 Neuroscience](https://md.research.elamur.ai/free/en/shelves/neuroscience/) · 186 works · [🧠 Cognitive Neuroscience](https://md.research.elamur.ai/free/en/shelves/neuroscience/cognitive-neuroscience/) · 185 works

## Cite

Cite as: Sang-Woo Lee, Jin-Hwa Kim, Jaehyun Jun, Jung-Woo Ha and Byoung-Tak Zhang. 2017. Overcoming Catastrophic Forgetting by Incremental Moment Matching. arXiv:1703.08475.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{lee2017overcoming,
  title = {{Overcoming Catastrophic Forgetting by Incremental Moment Matching}},
  author = {Sang-Woo Lee, Jin-Hwa Kim, Jaehyun Jun, Jung-Woo Ha and Byoung-Tak Zhang},
  year = {2017},
  doi = {10.48550/arxiv.1703.08475},
  eprint = {1703.08475},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1703.08475}
}
```

- [← previous Copied Monolingual Data Improves Low-Resource Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/currey-2017-copied-monolingual-data-improves-low/)
- [next → Survey: Multiword Expression Processing: A Survey](https://md.research.elamur.ai/free/en/papers/constant-2017-survey-multiword-expression-processing-survey/)
- [↑ back to papers of 2017](https://md.research.elamur.ai/free/en/papers/2017/page/2/)

## References

34 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Michael McCloskey and Neal J Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. Psychology of learning and motivation, 24:109–165, 1989. [find](https://scholar.google.com/scholar?q=Michael+McCloskey+and+Neal+J+Cohen%2E+Catastrophic+interference+in+connectionist+networks%3A+The+sequential+learning+problem%2E+Psychology+of+learning+and+motivation%2C+24%3A109%E2%80%93165%2C+1989%2E)
2. Robert M French. Catastrophic forgetting in connectionist networks. Trends in cognitive sciences, 3(4):128–135, 1999. [find](https://scholar.google.com/scholar?q=Robert+M+French%2E+Catastrophic+forgetting+in+connectionist+networks%2E+Trends+in+cognitive+sciences%2C+3%284%29%3A128%E2%80%93135%2C+1999%2E)
3. Ian J Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio. An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211, 2013. [find](https://scholar.google.com/scholar?q=Ian+J+Goodfellow%2C+Mehdi+Mirza%2C+Da+Xiao%2C+Aaron+Courville%2C+and+Yoshua+Bengio%2E+An+empirical+investigation+of+catastrophic+forgetting+in+gradient%2Dbased+neural+networks%2E+arXiv+preprint+arXiv%3A1312%2E6211%2C+201)
4. Rupesh K Srivastava, Jonathan Masci, Sohrob Kazerounian, Faustino Gomez, and Jürgen Schmidhuber. Compete to compute. In Advances in neural information processing systems, pages 2310–2318, 2013. [find](https://scholar.google.com/scholar?q=Rupesh+K+Srivastava%2C+Jonathan+Masci%2C+Sohrob+Kazerounian%2C+Faustino+Gomez%2C+and+J%C3%BCrgen+Schmidhuber%2E+Compete+to+compute%2E+In+Advances+in+neural+information+processing+systems%2C+pages+2310%E2%80%932318%2C+2013%2E)
5. Zoubin Ghahramani. Online variational bayesian learning. In NIPS workshop on Online Learning, 2000. [find](https://scholar.google.com/scholar?q=Zoubin+Ghahramani%2E+Online+variational+bayesian+learning%2E+In+NIPS+workshop+on+Online+Learning%2C+2000%2E)
6. Tamara Broderick, Nicholas Boyd, Andre Wibisono, Ashia C Wilson, and Michael I Jordan. Streaming variational bayes. In Advances in Neural Information Processing Systems, pages 1727–1735, 2013. [find](https://scholar.google.com/scholar?q=Tamara+Broderick%2C+Nicholas+Boyd%2C+Andre+Wibisono%2C+Ashia+C+Wilson%2C+and+Michael+I+Jordan%2E+Streaming+variational+bayes%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pages+1727%E2%80%931735%2C+2013%2E)
7. Zhizhong Li and Derek Hoiem. Learning without forgetting. In European Conference on Computer Vision, pages 614–629. Springer, 2016. [find](https://scholar.google.com/scholar?q=Zhizhong+Li+and+Derek+Hoiem%2E+Learning+without+forgetting%2E+In+European+Conference+on+Computer+Vision%2C+pages+614%E2%80%93629%2E+Springer%2C+2016%2E)
8. James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 2017. [find](https://scholar.google.com/scholar?q=James+Kirkpatrick%2C+Razvan+Pascanu%2C+Neil+Rabinowitz%2C+Joel+Veness%2C+Guillaume+Desjardins%2C+Andrei+A+Rusu%2C+Kieran+Milan%2C+John+Quan%2C+Tiago+Ramalho%2C+Agnieszka+Grabska%2DBarwinska%2C+et+al%2E+Overcoming+catastrophi)
9. David JC MacKay. A practical bayesian framework for backpropagation networks. Neural computation, 4(3):448–472, 1992. [find](https://scholar.google.com/scholar?q=David+JC+MacKay%2E+A+practical+bayesian+framework+for+backpropagation+networks%2E+Neural+computation%2C+4%283%29%3A448%E2%80%93472%2C+1992%2E)
10. Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural network. In Proceedings of the 32nd International Conference on Machine Learning (ICML-15), pages 1613–1622, 2015. [find](https://scholar.google.com/scholar?q=Charles+Blundell%2C+Julien+Cornebise%2C+Koray+Kavukcuoglu%2C+and+Daan+Wierstra%2E+Weight+uncertainty+in+neural+network%2E+In+Proceedings+of+the+32nd+International+Conference+on+Machine+Learning+%28ICML%2D15%29%2C+pages)
11. Jacob Goldberger and Sam T Roweis. Hierarchical clustering of a mixture model. In Advances in Neural Information Processing Systems, pages 505–512, 2005. [find](https://scholar.google.com/scholar?q=Jacob+Goldberger+and+Sam+T+Roweis%2E+Hierarchical+clustering+of+a+mixture+model%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pages+505%E2%80%93512%2C+2005%2E)
12. Sang-Woo Lee, Chung-Yeon Lee, Dong Hyun Kwak, Jiwon Kim, Jeonghee Kim, and Byoung-Tak Zhang. Dual-memory deep learning architectures for lifelong learning of everyday human behaviors. In Twenty-Fifth International Joint Conference on Artificial Intelligencee, pages 1669–1675, 2016. [find](https://scholar.google.com/scholar?q=Sang%2DWoo+Lee%2C+Chung%2DYeon+Lee%2C+Dong+Hyun+Kwak%2C+Jiwon+Kim%2C+Jeonghee+Kim%2C+and+Byoung%2DTak+Zhang%2E+Dual%2Dmemory+deep+learning+architectures+for+lifelong+learning+of+everyday+human+behaviors%2E+In+Twenty%2DFifth+)
13. Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. Progressive neural networks. arXiv preprint arXiv:1606.04671, 2016. [find](https://scholar.google.com/scholar?q=Andrei+A+Rusu%2C+Neil+C+Rabinowitz%2C+Guillaume+Desjardins%2C+Hubert+Soyer%2C+James+Kirkpatrick%2C+Koray+Kavukcuoglu%2C+Razvan+Pascanu%2C+and+Raia+Hadsell%2E+Progressive+neural+networks%2E+arXiv+preprint+arXiv%3A1606%2E046)
14. Chrisantha Fernando, Dylan Banarse, Charles Blundell, Yori Zwols, David Ha, Andrei A Rusu, Alexander Pritzel, and Daan Wierstra. [Pathnet: Evolution channels gradient descent in super neural networks](https://md.research.elamur.ai/free/en/papers/fernando-2017-pathnet-evolution-channels-gradient-descent/). arXiv preprint arXiv:1701.08734, 2017.
15. Zhen Huang, Jinyu Li, Sabato Marco Siniscalchi, I-Fan Chen, Chao Weng, and Chin-Hui Lee. Feature space maximum a posteriori linear regression for adaptation of deep neural networks. In Fifteenth Annual Conference of the International Speech Communication Association, 2014. [find](https://scholar.google.com/scholar?q=Zhen+Huang%2C+Jinyu+Li%2C+Sabato+Marco+Siniscalchi%2C+I%2DFan+Chen%2C+Chao+Weng%2C+and+Chin%2DHui+Lee%2E+Feature+space+maximum+a+posteriori+linear+regression+for+adaptation+of+deep+neural+networks%2E+In+Fifteenth+Annua)
16. Zhen Huang, Sabato Marco Siniscalchi, I-Fan Chen, Jinyu Li, Jiadong Wu, and Chin-Hui Lee. Maximum a posteriori adaptation of network parameters in deep models. In Sixteenth Annual Conference of the International Speech Communication Association, 2015. [find](https://scholar.google.com/scholar?q=Zhen+Huang%2C+Sabato+Marco+Siniscalchi%2C+I%2DFan+Chen%2C+Jinyu+Li%2C+Jiadong+Wu%2C+and+Chin%2DHui+Lee%2E+Maximum+a+posteriori+adaptation+of+network+parameters+in+deep+models%2E+In+Sixteenth+Annual+Conference+of+the+In)
17. Abdullah Rashwan, Han Zhao, and Pascal Poupart. Online and distributed bayesian moment matching for parameter learning in sum-product networks. In Proceedings of the 19th International Conference on Artificial Intelligence and Statistics, pages 1469–1477, 2016. [find](https://scholar.google.com/scholar?q=Abdullah+Rashwan%2C+Han+Zhao%2C+and+Pascal+Poupart%2E+Online+and+distributed+bayesian+moment+matching+for+parameter+learning+in+sum%2Dproduct+networks%2E+In+Proceedings+of+the+19th+International+Conference+on+A)
18. Kai Zhang and James T Kwok. Simplifying mixture models through function approximation. Neural Networks, IEEE Transactions on, 21(4):644–658, 2010. [find](https://scholar.google.com/scholar?q=Kai+Zhang+and+James+T+Kwok%2E+Simplifying+mixture+models+through+function+approximation%2E+Neural+Networks%2C+IEEE+Transactions+on%2C+21%284%29%3A644%E2%80%93658%2C+2010%2E)
19. Manas Pathak, Shantanu Rane, and Bhiksha Raj. Multiparty differential privacy via aggregation of locally trained classifiers. In Advances in Neural Information Processing Systems, pages 1876–1884, 2010. [find](https://scholar.google.com/scholar?q=Manas+Pathak%2C+Shantanu+Rane%2C+and+Bhiksha+Raj%2E+Multiparty+differential+privacy+via+aggregation+of+locally+trained+classifiers%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pages+1876%E2%80%931884%2C+201)
20. Pierre Baldi and Peter J Sadowski. Understanding dropout. In Advances in Neural Information Processing Systems, pages 2814–2822, 2013. [find](https://scholar.google.com/scholar?q=Pierre+Baldi+and+Peter+J+Sadowski%2E+Understanding+dropout%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pages+2814%E2%80%932822%2C+2013%2E)
21. Surajit Ray and Bruce G Lindsay. The topography of multivariate normal mixtures. Annals of Statistics, pages 2042–2065, 2005. [find](https://scholar.google.com/scholar?q=Surajit+Ray+and+Bruce+G+Lindsay%2E+The+topography+of+multivariate+normal+mixtures%2E+Annals+of+Statistics%2C+pages+2042%E2%80%932065%2C+2005%2E)
22. Razvan Pascanu and Yoshua Bengio. Revisiting natural gradient for deep networks. arXiv preprint arXiv:1301.3584, 2013. — [arXiv:1301.3584](https://arxiv.org/abs/1301.3584) [find](https://scholar.google.com/scholar?q=Razvan+Pascanu+and+Yoshua+Bengio%2E+Revisiting+natural+gradient+for+deep+networks%2E+arXiv+preprint+arXiv%3A1301%2E3584%2C+2013%2E+%E2%80%94+%5BarXiv%3A1301%2E3584%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1301%2E3584%29)
23. Ian J Goodfellow, Oriol Vinyals, and Andrew M Saxe. Qualitatively characterizing neural network optimization problems. arXiv preprint arXiv:1412.6544, 2014. [find](https://scholar.google.com/scholar?q=Ian+J+Goodfellow%2C+Oriol+Vinyals%2C+and+Andrew+M+Saxe%2E+Qualitatively+characterizing+neural+network+optimization+problems%2E+arXiv+preprint+arXiv%3A1412%2E6544%2C+2014%2E)
24. Jason Yosinski, Jeff Clune, Yoshua Bengio, and Hod Lipson. How transferable are features in deep neural networks? In Advances in neural information processing systems, pages 3320– 3328, 2014. [find](https://scholar.google.com/scholar?q=Jason+Yosinski%2C+Jeff+Clune%2C+Yoshua+Bengio%2C+and+Hod+Lipson%2E+How+transferable+are+features+in+deep+neural+networks%3F+In+Advances+in+neural+information+processing+systems%2C+pages+3320%E2%80%93+3328%2C+2014%2E)
25. Theodoros Evgeniou and Massimiliano Pontil. Regularized multi–task learning. In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 109–117. ACM, 2004. [find](https://scholar.google.com/scholar?q=Theodoros+Evgeniou+and+Massimiliano+Pontil%2E+Regularized+multi%E2%80%93task+learning%2E+In+Proceedings+of+the+tenth+ACM+SIGKDD+international+conference+on+Knowledge+discovery+and+data+mining%2C+pages+109%E2%80%93117%2E+ACM%2C)
26. Wolf Kienzle and Kumar Chellapilla. Personalized handwriting recognition via biased regularization. In Proceedings of the 23rd international conference on Machine learning, pages 457–464. ACM, 2006. [find](https://scholar.google.com/scholar?q=Wolf+Kienzle+and+Kumar+Chellapilla%2E+Personalized+handwriting+recognition+via+biased+regularization%2E+In+Proceedings+of+the+23rd+international+conference+on+Machine+learning%2C+pages+457%E2%80%93464%2E+ACM%2C+2006%2E)
27. Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. [Dropout: a simple way to prevent neural networks from overfitting](https://md.research.elamur.ai/free/en/papers/srivastava-2014-dropout-simple-way-prevent/). Journal of Machine Learning Research, 15(1):1929–1958, 2014.
28. Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The caltech-ucsd birds-200-2011 dataset. Tech. Rep. CNS-TR-2011-001, 2011. [find](https://scholar.google.com/scholar?q=Catherine+Wah%2C+Steve+Branson%2C+Peter+Welinder%2C+Pietro+Perona%2C+and+Serge+Belongie%2E+The+caltech%2Ducsd+birds%2D200%2D2011+dataset%2E+Tech%2E+Rep%2E+CNS%2DTR%2D2011%2D001%2C+2011%2E)
29. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. [Imagenet classification with deep convolutional neural networks](https://md.research.elamur.ai/free/en/papers/krizhevsky-2012-imagenet-classification-deep-convolutional/). In Advances in neural information processing systems, pages 1097–1105, 2012.
30. Sang-Woo Lee, Chung-Yeon Lee, Dong-Hyun Kwak, Jung-Woo Ha, Jeonghee Kim, and Byoung-Tak Zhang. Dual-memory neural networks for modeling cognitive activities of humans via wearable sensors. Neural Networks, 2017. [find](https://scholar.google.com/scholar?q=Sang%2DWoo+Lee%2C+Chung%2DYeon+Lee%2C+Dong%2DHyun+Kwak%2C+Jung%2DWoo+Ha%2C+Jeonghee+Kim%2C+and+Byoung%2DTak+Zhang%2E+Dual%2Dmemory+neural+networks+for+modeling+cognitive+activities+of+humans+via+wearable+sensors%2E+Neural+Netw)
31. Christos Louizos and Max Welling. Structured and efficient variational deep learning with matrix gaussian posteriors. arXiv preprint arXiv:1603.04733, 2016. [find](https://scholar.google.com/scholar?q=Christos+Louizos+and+Max+Welling%2E+Structured+and+efficient+variational+deep+learning+with+matrix+gaussian+posteriors%2E+arXiv+preprint+arXiv%3A1603%2E04733%2C+2016%2E)
32. Surajit Ray and Dan Ren. On the upper bound of the number of modes of a multivariate normal mixture. Journal of Multivariate Analysis, 108:41–52, 2012. [find](https://scholar.google.com/scholar?q=Surajit+Ray+and+Dan+Ren%2E+On+the+upper+bound+of+the+number+of+modes+of+a+multivariate+normal+mixture%2E+Journal+of+Multivariate+Analysis%2C+108%3A41%E2%80%9352%2C+2012%2E)
33. Carlos Améndola, Alexander Engström, and Christian Haase. Maximum number of modes of gaussian mixtures. arXiv preprint arXiv:1702.05066, 2017. — [arXiv:1702.05066](https://arxiv.org/abs/1702.05066) [find](https://scholar.google.com/scholar?q=Carlos+Am%C3%A9ndola%2C+Alexander+Engstr%C3%B6m%2C+and+Christian+Haase%2E+Maximum+number+of+modes+of+gaussian+mixtures%2E+arXiv+preprint+arXiv%3A1702%2E05066%2C+2017%2E+%E2%80%94+%5BarXiv%3A1702%2E05066%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1702%2E05066%29)
34. Diederik P Kingma and Max Welling. [Auto-encoding variational bayes](https://md.research.elamur.ai/free/en/papers/kingma-2013-auto-encoding-variational-bayes/). arXiv preprint arXiv:1312.6114, 2013.

*Updated 2 November 2026*