---
title: "Memory Aware Synapses: Learning what (not) to forget (2017)"
description: "To this end, they propose Memory Aware Synapses (MAS). 39 references, cited by 106 papers here."
authors: ["Rahaf Aljundi", "Francesca Babiloni", "Mohamed Elhoseiny", "Marcus Rohrbach", "Tinne Tuytelaars"]
year: 2017
venue: "arXiv:1711.09601"
source: "https://arxiv.org/abs/1711.09601"
doi: "10.48550/arxiv.1711.09601"
arxiv: "1711.09601"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/aljundi-2017-memory-aware-synapses-learning-forget/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/aljundi-2017-memory-aware-synapses-learning-forget/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2017](https://md.research.elamur.ai/free/en/papers/2017/) / Memory Aware Synapses: Learning what (not) to forget

# Memory Aware Synapses: Learning what (not) to forget

**Authors:** [Rahaf Aljundi](https://md.research.elamur.ai/free/en/authors/rahaf-aljundi/), Francesca Babiloni, [Mohamed Elhoseiny](https://md.research.elamur.ai/free/en/authors/mohamed-elhoseiny/), [Marcus Rohrbach](https://md.research.elamur.ai/free/en/authors/marcus-rohrbach/) and [Tinne Tuytelaars](https://md.research.elamur.ai/free/en/authors/tinne-tuytelaars/) · **Year:** 2017 · **Venue:** [arXiv:1711.09601](https://md.research.elamur.ai/free/en/venues/lecture-notes-in-computer-science/) ([Lecture notes in computer science 2017](https://md.research.elamur.ai/free/en/venues/lecture-notes-in-computer-science/2017/)) · **Publisher:** [Springer Science+Business Media](https://md.research.elamur.ai/free/en/publishers/springer-science-business-media/) · **Licence:** [arXiv — non-exclusive licence to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1711.09601) · [PDF](https://arxiv.org/pdf/1711.09601) · [DOI 10.48550/arxiv.1711.09601](https://doi.org/10.48550/arxiv.1711.09601) · [arXiv 1711.09601](https://arxiv.org/abs/1711.09601) · [OpenAlex](https://openalex.org/W2768412495)

**Citations:** 1,502 citations (OpenAlex, 2026-10-14) · [106](https://md.research.elamur.ai/free/en/papers/aljundi-2017-memory-aware-synapses-learning-forget/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper addresses lifelong learning (LLL), where a single neural network learns tasks one after another and cannot access data from earlier tasks. As a result, it tends to suffer from catastrophic forgetting. The authors, from KU Leuven and Facebook AI Research, note that a model has finite capacity, so it cannot retain everything. The model should itself decide what is worth remembering and what can be let go. To this end, they propose Memory Aware Synapses (MAS). For each parameter, MAS accumulates an importance weight: how strongly the network's output function changes when the parameter is slightly perturbed. Loss values are not used, so no labels are needed, and importance can be updated online on unlabeled data from the real environment. When a new task is learned, a regularizer penalizes changes to important parameters. The method was tested on sequences of object recognition tasks and on learning embeddings for \<subject, predicate, object> triplets. The authors report state-of-the-art results. The paper's value lies in letting the model focus its "memory" on what actually comes up during use.

## Key points
- MAS is compared with data-based methods (LwF, EBLL) and model-based methods (EWC, IMM, SI). Unlike EWC (the diagonal of the Fisher matrix) and SI (changes in the loss during training), MAS estimates the sensitivity of the output function itself. In this way, the method avoids near-zero gradients at a local minimum of the loss.
- The method requires a constant amount of memory, is not tied to a particular type of task (not only classification), works with pretrained models, and adapts to test conditions.
- A local variant of MAS is theoretically connected to Hebb's rule: connections between neurons that fire together are strengthened.
- Importance was estimated separately on training data and on test data. For the 1000 most important parameters, the two estimates almost coincide, which means the important parameters can be identified without labels.
- In an experiment with four fact learning tasks, MAS was adapted to a subset of sports facts from the first task that occurred frequently at test time. The adaptive variant best preserved both accuracy and the embedding structure for this subset. Plain finetuning, by contrast, eventually "collapsed" all of these examples into a single point.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Meta](https://md.research.elamur.ai/free/en/companies/meta/) — Company, Menlo Park, United States · 214 works · 241,492 citations
- [KU Leuven](https://md.research.elamur.ai/free/en/universities/ku-leuven/) — University, Leuven, Belgium · 46 works · 7,833 citations
- [IMEC](https://md.research.elamur.ai/free/en/ngos/imec/) — Nonprofit, Leuven, Belgium · 4 works · 1,823 citations

📚 Shelves (3): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

## Cite

Cite as: Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach and Tinne Tuytelaars. 2017. Memory Aware Synapses: Learning what (not) to forget. arXiv:1711.09601.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{aljundi2017memory,
  title = {{Memory Aware Synapses: Learning what (not) to forget}},
  author = {Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach and Tinne Tuytelaars},
  year = {2017},
  doi = {10.48550/arxiv.1711.09601},
  eprint = {1711.09601},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1711.09601}
}
```

- [← previous RACE: Large-scale ReAding Comprehension Dataset From Examinations](https://md.research.elamur.ai/free/en/papers/lai-2017-race-large-scale-reading-comprehension/)
- [next → Understanding Black-box Predictions via Influence Functions](https://md.research.elamur.ai/free/en/papers/koh-2017-understanding-black-box-predictions-influence/)
- [↑ back to papers of 2017](https://md.research.elamur.ai/free/en/papers/2017/)

## References

39 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Aljundi, R., Chakravarty, P., Tuytelaars, T.: Expert gate: Lifelong learning with a network of experts. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016) [find](https://scholar.google.com/scholar?q=Aljundi%2C+R%2E%2C+Chakravarty%2C+P%2E%2C+Tuytelaars%2C+T%2E%3A+Expert+gate%3A+Lifelong+learning+with+a+network+of+experts%2E+In%3A+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition+%28CVPR%29+%282016%29)
2. Andrychowicz, M., Denil, M., Gómez, S., Hoffman, M.W., Pfau, D., Schaul, T., de Freitas, N.: Learning to learn by gradient descent by gradient descent. In: Lee, D.D., Sugiyama, M., Luxburg, U.V., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems 29, pp. 3981–3989. Curr… [find](https://scholar.google.com/scholar?q=Andrychowicz%2C+M%2E%2C+Denil%2C+M%2E%2C+Go%CC%81mez%2C+S%2E%2C+Hoffman%2C+M%2EW%2E%2C+Pfau%2C+D%2E%2C+Schaul%2C+T%2E%2C+de+Freitas%2C+N%2E%3A+Learning+to+learn+by+gradient+descent+by+gradient+descent%2E+In%3A+Lee%2C+D%2ED%2E%2C+Sugiyama%2C+M%2E%2C+Luxburg%2C+U%2EV%2E%2C+Guy)
3. de Campos, T.E., Babu, B.R., Varma, M.: Character recognition in natural images. In: Proceedings of the International Conference on Computer Vision Theory and Applications, Lisbon, Portugal (February 2009) [find](https://scholar.google.com/scholar?q=de+Campos%2C+T%2EE%2E%2C+Babu%2C+B%2ER%2E%2C+Varma%2C+M%2E%3A+Character+recognition+in+natural+images%2E+In%3A+Proceedings+of+the+International+Conference+on+Computer+Vision+Theory+and+Applications%2C+Lisbon%2C+Portugal+%28February+)
4. Elhoseiny, M., Cohen, S., Chang, W., Price, B.L., Elgammal, A.M.: Sherlock: Scalable fact learning in images. In: AAAI. pp. 4016–4024 (2017) [find](https://scholar.google.com/scholar?q=Elhoseiny%2C+M%2E%2C+Cohen%2C+S%2E%2C+Chang%2C+W%2E%2C+Price%2C+B%2EL%2E%2C+Elgammal%2C+A%2EM%2E%3A+Sherlock%3A+Scalable+fact+learning+in+images%2E+In%3A+AAAI%2E+pp%2E+4016%E2%80%934024+%282017%29)
5. Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PAS-CAL Visual Object Classes Challenge 2012 (VOC2012) Results. [http://www.pascal-network.org/challenges/VOC/voc2012/workshop/index.html](http://www.pascal-network.org/challenges/VOC/voc2012/workshop/index.html) [↗](http://www.pascal-network.org/challenges/VOC/voc2012/workshop/index.html)
6. Fernando, C., Banarse, D., Blundell, C., Zwols, Y., Ha, D., Rusu, A.A., Pritzel, A., Wierstra, D.: [Pathnet: Evolution channels gradient descent in super neural networks](https://md.research.elamur.ai/free/en/papers/fernando-2017-pathnet-evolution-channels-gradient-descent/). arXiv preprint arXiv:1701.08734 (2017)
7. Finn, C., Abbeel, P., Levine, S.: [Model-agnostic meta-learning for fast adaptation of deep networks](https://md.research.elamur.ai/free/en/papers/finn-2017-model-agnostic-meta-learning-fast/). In: Proceedings of the International Conference on Machine Learning (ICML) (2017)
8. French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) [find](https://scholar.google.com/scholar?q=French%2C+R%2EM%2E%3A+Catastrophic+forgetting+in+connectionist+networks%2E+Trends+in+cognitive+sciences+3%284%29%2C+128%E2%80%93135+%281999%29)
9. Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) [find](https://scholar.google.com/scholar?q=Goodfellow%2C+I%2EJ%2E%2C+Mirza%2C+M%2E%2C+Xiao%2C+D%2E%2C+Courville%2C+A%2E%2C+Bengio%2C+Y%2E%3A+An+empirical+investigation+of+catastrophic+forgetting+in+gradient%2Dbased+neural+networks%2E+arXiv+preprint+arXiv%3A1312%2E6211+%282013%29)
10. Hebb, D.: The organization of behavior. 1949. New York Wiely (2002) [find](https://scholar.google.com/scholar?q=Hebb%2C+D%2E%3A+The+organization+of+behavior%2E+1949%2E+New+York+Wiely+%282002%29)
11. Huszár, F.: Note on the quadratic penalties in elastic weight consolidation. Proceedings of the National Academy of Sciences (2018). https://doi.org/10.1073/pnas.1717042115, http://www.pnas.org/content/early/2018/02/16/1717042115 — [doi:10.1073/pnas.1717042115](https://doi.org/10.1073/pnas.1717042115) [↗](https://doi.org/10.1073/pnas.1717042115)
12. Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. arXiv preprint arXiv:1612.00796 (2016) [find](https://scholar.google.com/scholar?q=Kirkpatrick%2C+J%2E%2C+Pascanu%2C+R%2E%2C+Rabinowitz%2C+N%2E%2C+Veness%2C+J%2E%2C+Desjardins%2C+G%2E%2C+Rusu%2C+A%2EA%2E%2C+Milan%2C+K%2E%2C+Quan%2C+J%2E%2C+Ramalho%2C+T%2E%2C+Grabska%2DBarwinska%2C+A%2E%2C+et+al%2E%3A+Overcoming+catastrophic+forgetting+in+neural+netw)
13. Krause, J., Stark, M., Deng, J., Fei-Fei, L.: 3d object representations for fine-grained categorization. In: Proceedings of the IEEE International Conference on Computer Vision Workshops. pp. 554–561 (2013) [find](https://scholar.google.com/scholar?q=Krause%2C+J%2E%2C+Stark%2C+M%2E%2C+Deng%2C+J%2E%2C+Fei%2DFei%2C+L%2E%3A+3d+object+representations+for+fine%2Dgrained+categorization%2E+In%3A+Proceedings+of+the+IEEE+International+Conference+on+Computer+Vision+Workshops%2E+pp%2E+554%E2%80%93561+)
14. Krizhevsky, A.: [One weird trick for parallelizing convolutional neural networks](https://md.research.elamur.ai/free/en/papers/krizhevsky-2014-one-weird-trick-parallelizing-convolutional/). arXiv preprint arXiv:1404.5997 (2014)
15. Krizhevsky, A., Sutskever, I., Hinton, G.E.: [Imagenet classification with deep convolutional neural networks](https://md.research.elamur.ai/free/en/papers/krizhevsky-2012-imagenet-classification-deep-convolutional/). In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 25, pp. 1097–1105. Curran Associates, Inc. (2012)
16. Lee, S.W., Kim, J.H., Ha, J.W., Zhang, B.T.: [Overcoming catastrophic forgetting by incremental moment matching](https://md.research.elamur.ai/free/en/papers/lee-2017-overcoming-catastrophic-forgetting-incremental-moment/). arXiv preprint arXiv:1703.08475 (2017)
17. Li, Z., Hoiem, D.: Learning without forgetting. In: European Conference on Computer Vision. pp. 614–629. Springer (2016) [find](https://scholar.google.com/scholar?q=Li%2C+Z%2E%2C+Hoiem%2C+D%2E%3A+Learning+without+forgetting%2E+In%3A+European+Conference+on+Computer+Vision%2E+pp%2E+614%E2%80%93629%2E+Springer+%282016%29)
18. Lopez-Paz, D., Ranzato, M.: Gradient episodic memory for continual learning. In: Advances in Neural Information Processing Systems. pp. 6470–6479 (2017) [find](https://scholar.google.com/scholar?q=Lopez%2DPaz%2C+D%2E%2C+Ranzato%2C+M%2E%3A+Gradient+episodic+memory+for+continual+learning%2E+In%3A+Advances+in+Neural+Information+Processing+Systems%2E+pp%2E+6470%E2%80%936479+%282017%29)
19. Maji, S., Kannala, J., Rahtu, E., Blaschko, M., Vedaldi, A.: Fine-grained visual classification of aircraft. Tech. rep. (2013) [find](https://scholar.google.com/scholar?q=Maji%2C+S%2E%2C+Kannala%2C+J%2E%2C+Rahtu%2C+E%2E%2C+Blaschko%2C+M%2E%2C+Vedaldi%2C+A%2E%3A+Fine%2Dgrained+visual+classification+of+aircraft%2E+Tech%2E+rep%2E+%282013%29)
20. McCloskey, M., Cohen, N.J.: Catastrophic interference in connectionist networks: The sequential learning problem. Psychology of learning and motivation 24, 109–165 (1989) [find](https://scholar.google.com/scholar?q=McCloskey%2C+M%2E%2C+Cohen%2C+N%2EJ%2E%3A+Catastrophic+interference+in+connectionist+networks%3A+The+sequential+learning+problem%2E+Psychology+of+learning+and+motivation+24%2C+109%E2%80%93165+%281989%29)
21. Mikolov, T., Chen, K., Corrado, G., Dean, J.: [Efficient estimation of word representations in vector space](https://md.research.elamur.ai/free/en/papers/mikolov-2013-efficient-estimation-word-representations-vector/). arXiv preprint arXiv:1301.3781 (2013)
22. Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng, A.Y.: Reading digits in natural images with unsupervised feature learning (2011) [find](https://scholar.google.com/scholar?q=Netzer%2C+Y%2E%2C+Wang%2C+T%2E%2C+Coates%2C+A%2E%2C+Bissacco%2C+A%2E%2C+Wu%2C+B%2E%2C+Ng%2C+A%2EY%2E%3A+Reading+digits+in+natural+images+with+unsupervised+feature+learning+%282011%29)
23. Nilsback, M.E., Zisserman, A.: Automated flower classification over a large number of classes. In: Proceedings of the Indian Conference on Computer Vision, Graphics and Image Processing (Dec 2008) [find](https://scholar.google.com/scholar?q=Nilsback%2C+M%2EE%2E%2C+Zisserman%2C+A%2E%3A+Automated+flower+classification+over+a+large+number+of+classes%2E+In%3A+Proceedings+of+the+Indian+Conference+on+Computer+Vision%2C+Graphics+and+Image+Processing+%28Dec+2008%29)
24. Pentina, A., Lampert, C.H.: Lifelong learning with non-i.i.d. tasks. In: Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., Garnett, R. (eds.) Advances in Neural Information Processing Systems 28, pp. 1540–1548 (2015) [find](https://scholar.google.com/scholar?q=Pentina%2C+A%2E%2C+Lampert%2C+C%2EH%2E%3A+Lifelong+learning+with+non%2Di%2Ei%2Ed%2E+tasks%2E+In%3A+Cortes%2C+C%2E%2C+Lawrence%2C+N%2ED%2E%2C+Lee%2C+D%2ED%2E%2C+Sugiyama%2C+M%2E%2C+Garnett%2C+R%2E+%28eds%2E%29+Advances+in+Neural+Information+Processing+Systems+28%2C+p)
25. Pentina, A., Lampert, C.H.: Lifelong learning with non-iid tasks. In: Advances in Neural Information Processing Systems. pp. 1540–1548 (2015) [find](https://scholar.google.com/scholar?q=Pentina%2C+A%2E%2C+Lampert%2C+C%2EH%2E%3A+Lifelong+learning+with+non%2Diid+tasks%2E+In%3A+Advances+in+Neural+Information+Processing+Systems%2E+pp%2E+1540%E2%80%931548+%282015%29)
26. Quadrianto, N., Petterson, J., Smola, A.J.: Distribution matching for transduction. In: Advances in Neural Information Processing Systems. pp. 1500–1508 (2009) [find](https://scholar.google.com/scholar?q=Quadrianto%2C+N%2E%2C+Petterson%2C+J%2E%2C+Smola%2C+A%2EJ%2E%3A+Distribution+matching+for+transduction%2E+In%3A+Advances+in+Neural+Information+Processing+Systems%2E+pp%2E+1500%E2%80%931508+%282009%29)
27. Quattoni, A., Torralba, A.: Recognizing indoor scenes. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 413–420. IEEE (2009) [find](https://scholar.google.com/scholar?q=Quattoni%2C+A%2E%2C+Torralba%2C+A%2E%3A+Recognizing+indoor+scenes%2E+In%3A+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition+%28CVPR%29%2E+pp%2E+413%E2%80%93420%2E+IEEE+%282009%29)
28. Rannen, A., Aljundi, R., Blaschko, M.B., Tuytelaars, T.: Encoder based lifelong learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 1320–1328 (2017) [find](https://scholar.google.com/scholar?q=Rannen%2C+A%2E%2C+Aljundi%2C+R%2E%2C+Blaschko%2C+M%2EB%2E%2C+Tuytelaars%2C+T%2E%3A+Encoder+based+lifelong+learning%2E+In%3A+Proceedings+of+the+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition%2E+pp%2E+1320%E2%80%931328+%282017%29)
29. Rebuffi, S.A., Kolesnikov, A., Lampert, C.H.: icarl: Incremental classifier and representation learning. arXiv preprint arXiv:1611.07725 (2016) [find](https://scholar.google.com/scholar?q=Rebuffi%2C+S%2EA%2E%2C+Kolesnikov%2C+A%2E%2C+Lampert%2C+C%2EH%2E%3A+icarl%3A+Incremental+classifier+and+representation+learning%2E+arXiv+preprint+arXiv%3A1611%2E07725+%282016%29)
30. Ring, M.B.: Child: A first step towards continual learning. Machine Learning 28(1), 77–104 (1997) [find](https://scholar.google.com/scholar?q=Ring%2C+M%2EB%2E%3A+Child%3A+A+first+step+towards+continual+learning%2E+Machine+Learning+28%281%29%2C+77%E2%80%93104+%281997%29)
31. Royer, A., Lampert, C.H.: Classifier adaptation at prediction time. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pp. 1401–1409 (2015) [find](https://scholar.google.com/scholar?q=Royer%2C+A%2E%2C+Lampert%2C+C%2EH%2E%3A+Classifier+adaptation+at+prediction+time%2E+In%3A+Proceedings+of+the+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition%2E+pp%2E+1401%E2%80%931409+%282015%29)
32. Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A.C., Fei-Fei, L.: [ImageNet Large Scale Visual Recognition Challenge](https://md.research.elamur.ai/free/en/papers/russakovsky-2014-imagenet-large-scale-visual-recognition/). International Journal of Computer Vision (IJCV) 115(3), 211–252 (2015). https://doi.org/10.1007/s11263-… [↗](https://doi.org/10.1007/s11263-…)
33. Rusu, A.A., Rabinowitz, N.C., Desjardins, G., Soyer, H., Kirkpatrick, J., Kavukcuoglu, K., Pascanu, R., Hadsell, R.: Progressive neural networks. arXiv preprint arXiv:1606.04671 (2016) [find](https://scholar.google.com/scholar?q=Rusu%2C+A%2EA%2E%2C+Rabinowitz%2C+N%2EC%2E%2C+Desjardins%2C+G%2E%2C+Soyer%2C+H%2E%2C+Kirkpatrick%2C+J%2E%2C+Kavukcuoglu%2C+K%2E%2C+Pascanu%2C+R%2E%2C+Hadsell%2C+R%2E%3A+Progressive+neural+networks%2E+arXiv+preprint+arXiv%3A1606%2E04671+%282016%29)
34. Shmelkov, K., Schmid, C., Alahari, K.: Incremental learning of object detectors without catastrophic forgetting. In: The IEEE International Conference on Computer Vision (ICCV) (2017) [find](https://scholar.google.com/scholar?q=Shmelkov%2C+K%2E%2C+Schmid%2C+C%2E%2C+Alahari%2C+K%2E%3A+Incremental+learning+of+object+detectors+without+catastrophic+forgetting%2E+In%3A+The+IEEE+International+Conference+on+Computer+Vision+%28ICCV%29+%282017%29)
35. Silver, D.L., Yang, Q., Li, L.: Lifelong machine learning systems: Beyond learning algorithms. In: AAAI Spring Symposium: Lifelong Machine Learning. pp. 49–55. Citeseer (2013) [find](https://scholar.google.com/scholar?q=Silver%2C+D%2EL%2E%2C+Yang%2C+Q%2E%2C+Li%2C+L%2E%3A+Lifelong+machine+learning+systems%3A+Beyond+learning+algorithms%2E+In%3A+AAAI+Spring+Symposium%3A+Lifelong+Machine+Learning%2E+pp%2E+49%E2%80%9355%2E+Citeseer+%282013%29)
36. Simonyan, K., Zisserman, A.: [Very deep convolutional networks for large-scale image recognition](https://md.research.elamur.ai/free/en/papers/simonyan-2014-very-deep-convolutional-networks-large/). arXiv preprint arXiv:1409.1556 (2014)
37. Thrun, S., Mitchell, T.M.: Lifelong robot learning. Robotics and autonomous systems 15(1-2), 25–46 (1995) [find](https://scholar.google.com/scholar?q=Thrun%2C+S%2E%2C+Mitchell%2C+T%2EM%2E%3A+Lifelong+robot+learning%2E+Robotics+and+autonomous+systems+15%281%2D2%29%2C+25%E2%80%9346+%281995%29)
38. Welinder, P., Branson, S., Mita, T., Wah, C., Schroff, F., Belongie, S., Perona, P.: Caltech-UCSD Birds 200. Tech. Rep. CNS-TR-2010-001, California Institute of Technology (2010) [find](https://scholar.google.com/scholar?q=Welinder%2C+P%2E%2C+Branson%2C+S%2E%2C+Mita%2C+T%2E%2C+Wah%2C+C%2E%2C+Schroff%2C+F%2E%2C+Belongie%2C+S%2E%2C+Perona%2C+P%2E%3A+Caltech%2DUCSD+Birds+200%2E+Tech%2E+Rep%2E+CNS%2DTR%2D2010%2D001%2C+California+Institute+of+Technology+%282010%29)
39. Zenke, F., Poole, B., Ganguli, S.: Improved multitask learning through synaptic intelligence. In: Proceedings of the International Conference on Machine Learning (ICML) (2017) [find](https://scholar.google.com/scholar?q=Zenke%2C+F%2E%2C+Poole%2C+B%2E%2C+Ganguli%2C+S%2E%3A+Improved+multitask+learning+through+synaptic+intelligence%2E+In%3A+Proceedings+of+the+International+Conference+on+Machine+Learning+%28ICML%29+%282017%29)

*Updated 2 November 2026*