---
title: "SLoRA — paper by Yang et al. (2026) — summary and ideas"
description: "Based on this, they propose SLoRA (Subspace-Denoised Low-Rank Adaptation). 40 references."
authors: ["Lina Yang", "Yusheng Liao", "Yanfeng Wang", "Yu Wang"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.247/"
doi: "10.18653/v1/2026.acl-long.247"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/yang-2026-slora-balancing-plasticity-forgetting-large/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/yang-2026-slora-balancing-plasticity-forgetting-large/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/50/) / SLoRA: Balancing Plasticity and Forgetting in Large Language Models for Continual Learning

# SLoRA: Balancing Plasticity and Forgetting in Large Language Models for Continual Learning

**Authors:** Lina Yang, Yusheng Liao, [Yanfeng Wang](https://md.research.elamur.ai/free/en/authors/yanfeng-wang/) and [Yu Wang](https://md.research.elamur.ai/free/en/authors/yu-wang/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.247/) · [PDF](https://aclanthology.org/2026.acl-long.247.pdf) · [DOI 10.18653/v1/2026.acl-long.247](https://doi.org/10.18653/v1/2026.acl-long.247) · [OpenAlex](https://openalex.org/W7166799902)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper addresses catastrophic forgetting when a large language model is fine-tuned with LoRA on a sequence of tasks. The authors argue that forgetting is driven largely by noise accumulating in LoRA updates. In a pilot study, they trained a model on Amazon Reviews and then on Yahoo Answers. Pruning low-importance parts of the second task's adapter improved accuracy on the first task, while pruning the first task's adapter hurt it. Based on this, they propose SLoRA (Subspace-Denoised Low-Rank Adaptation). It decomposes each LoRA update with SVD, keeps the directions that align with the base model's subspace and discards weakly aligned ones as noise. The method needs no regularization, no replay data, no stored gradients and no change to training. SLoRA-Pre denoises after every task for online settings, and SLoRA-Post denoises once at the end for offline settings. On TRACE, SLoRA beats regularization-based and subspace-constrained baselines and keeps the ability to learn new tasks, which addresses the stability–plasticity trade-off.

## Key points
- On the TRACE benchmark, SLoRA raises final accuracy by up to 12% and cuts forgetting by up to 29%, while pruning over 30% of LoRA parameters as noise.
- Experiments use Llama3-8B and Qwen2.5-7B, with Seq-LoRA, O-LoRA, SD-LoRA and classical CL methods as comparison points.
- Gains hold for LoRA ranks from r = 16 to r = 256. The method also transfers to DoRA, rsLoRA and Full-FT, where SLoRA-Pre gives a 15.82% improvement under full fine-tuning.
- The authors list three limitations:
  - SLoRA requires white-box access to the base weights.
  - Randomized SVD adds a small computational cost.
  - SLoRA-Post must temporarily buffer all raw task updates.
- Code is released at github.com/alina1031/SLoRA.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Shanghai Jiao Tong University](https://md.research.elamur.ai/free/en/universities/shanghai-jiao-tong-university/) — University, Shanghai, China · 212 works · 19,544 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works

## Cite

Cite as: Lina Yang, Yusheng Liao, Yanfeng Wang and Yu Wang. 2026. SLoRA: Balancing Plasticity and Forgetting in Large Language Models for Continual Learning. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{yang2026slora,
  title = {{SLoRA: Balancing Plasticity and Forgetting in Large Language Models for Continual Learning}},
  author = {Lina Yang, Yusheng Liao, Yanfeng Wang and Yu Wang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.247},
  url = {https://doi.org/10.18653/v1/2026.acl-long.247}
}
```

- [← previous SLIP: Soft Label Mechanism and Key-Extraction-Guided CoT-based Defense Against Instruction Backdoor in APIs](https://md.research.elamur.ai/free/en/papers/zhengxian-2026-slip-soft-label-mechanism-key/)
- [next → Slovene Morphological and Word Formation Segmentation: A Novel Dataset and Evaluation](https://md.research.elamur.ai/free/en/papers/pranjic-2026-slovene-morphological-word-formation-segmentation/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/50/)

## References

40 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774.
2. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems, 33:1877–1901.
3. Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zonghan Yang, Yusheng Su, Shengding Hu, Yulin Chen, Chi-Min Chan, Weize Chen, and 1 others. 2023. [Parameter-efficient fine-tuning of large-scale pre-trained language models](https://md.research.elamur.ai/free/en/papers/ding-2023-parameter-efficient-fine-tuning-large/). Nature Machine Intelligence, 5(3):220–235.
4. Mehrdad Farajtabar, Navid Azizan, Alex Mott, and Ang Li. 2020. Orthogonal gradient descent for continual learning. In International conference on artificial intelligence and statistics, pages 3762–3773. PMLR. [find](https://scholar.google.com/scholar?q=Mehrdad+Farajtabar%2C+Navid+Azizan%2C+Alex+Mott%2C+and+Ang+Li%2E+2020%2E+Orthogonal+gradient+descent+for+continual+learning%2E+In+International+conference+on+artificial+intelligence+and+statistics%2C+pages+3762%E2%80%93377)
5. Xu Han, Yi Dai, Tianyu Gao, Yankai Lin, Zhiyuan Liu, Peng Li, Maosong Sun, and Jie Zhou. 2020. [Continual relation learning via episodic memory activation and reconsolidation](https://md.research.elamur.ai/free/en/papers/han-2020-continual-relation-learning-episodic-memory/). In Proceedings of the 58th annual meeting of the association for computational linguistics, pages 6429–6440.
6. Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for nlp. In International conference on machine learning, pages 2790–2799. PMLR. [find](https://scholar.google.com/scholar?q=Neil+Houlsby%2C+Andrei+Giurgiu%2C+Stanislaw+Jastrzebski%2C+Bruna+Morrone%2C+Quentin+De+Laroussilhe%2C+Andrea+Gesmundo%2C+Mona+Attariyan%2C+and+Sylvain+Gelly%2E+2019%2E+Parameter%2Defficient+transfer+learning+for+nlp%2E+In+)
7. Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, and 1 others. 2021. [Lora: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). In International Conference on Learning Representations.
8. Joel Jang, Seonghyeon Ye, Sohee Yang, Joongbo Shin, Janghoon Han, KIM Gyeonghun, Stanley Jungkyu Choi, and Minjoon Seo. 2022. Towards continual knowledge learning of language models. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Joel+Jang%2C+Seonghyeon+Ye%2C+Sohee+Yang%2C+Joongbo+Shin%2C+Janghoon+Han%2C+KIM+Gyeonghun%2C+Stanley+Jungkyu+Choi%2C+and+Minjoon+Seo%2E+2022%2E+Towards+continual+knowledge+learning+of+language+models%2E+In+International+)
9. Shuyang Jiang, Yusheng Liao, Ya Zhang, Yanfeng Wang, and Yu Wang. 2024. Taia: Large language models are out-of-distribution data learners. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Shuyang+Jiang%2C+Yusheng+Liao%2C+Ya+Zhang%2C+Yanfeng+Wang%2C+and+Yu+Wang%2E+2024%2E+Taia%3A+Large+language+models+are+out%2Dof%2Ddistribution+data+learners%2E+In+The+Thirty%2Deighth+Annual+Conference+on+Neural+Information+)
10. Shuyang Jiang, Yusheng Liao, Ya Zhang, Yanfeng Wang, and Yu Wang. 2025. Fine-tuning with reserved majority for noise reduction. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Shuyang+Jiang%2C+Yusheng+Liao%2C+Ya+Zhang%2C+Yanfeng+Wang%2C+and+Yu+Wang%2E+2025%2E+Fine%2Dtuning+with+reserved+majority+for+noise+reduction%2E+In+The+Thirteenth+International+Conference+on+Learning+Representations%2E)
11. Damjan Kalajdzievski. 2023. A rank stabilization scaling factor for fine-tuning with lora. arXiv preprint arXiv:2312.03732. — [arXiv:2312.03732](https://arxiv.org/abs/2312.03732) [find](https://scholar.google.com/scholar?q=Damjan+Kalajdzievski%2E+2023%2E+A+rank+stabilization+scaling+factor+for+fine%2Dtuning+with+lora%2E+arXiv+preprint+arXiv%3A2312%2E03732%2E+%E2%80%94+%5BarXiv%3A2312%2E03732%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2312%2E03732%29)
12. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, and 1 others. 2023. [Palm: Scaling language modeling with pathways](https://md.research.elamur.ai/free/en/papers/chowdhery-2022-palm-scaling-language-modeling-pathways/). Journal of Machine Learning Research, 24(240):1–113.
13. James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, and 1 others. 2017. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(1… [find](https://scholar.google.com/scholar?q=James+Kirkpatrick%2C+Razvan+Pascanu%2C+Neil+Rabinowitz%2C+Joel+Veness%2C+Guillaume+Desjardins%2C+Andrei+A+Rusu%2C+Kieran+Milan%2C+John+Quan%2C+Tiago+Ramalho%2C+Agnieszka+Grabska%2DBarwinska%2C+and+1+others%2E+2017%2E+Overcomin)
14. Bhuwan Dhingra, Jeremy R Cole, Julian Martin Eisenschlos, Dan Gillick, Jacob Eisenstein, and William Cohen. 2022. [Time-aware language models as temporal knowledge bases](https://md.research.elamur.ai/free/en/papers/dhingra-2022-time-aware-language-models-as/). Transactions of the Association for Computational Linguistics, 10:257–273.
15. Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. [The power of scale for parameter-efficient prompt tuning](https://md.research.elamur.ai/free/en/papers/lester-2021-power-scale-parameter-efficient-prompt/). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3045–3059.
16. Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4582–… [find](https://scholar.google.com/scholar?q=Xiang+Lisa+Li+and+Percy+Liang%2E+2021%2E+Prefix%2Dtuning%3A+Optimizing+continuous+prompts+for+generation%2E+In+Proceedings+of+the+59th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+and+the+11t)
17. Xilai Li, Yingbo Zhou, Tianfu Wu, Richard Socher, and Caiming Xiong. 2019. Learn to grow: A continual structure learning framework for overcoming catastrophic forgetting. In International conference on machine learning, pages 3925–3934. PMLR. [find](https://scholar.google.com/scholar?q=Xilai+Li%2C+Yingbo+Zhou%2C+Tianfu+Wu%2C+Richard+Socher%2C+and+Caiming+Xiong%2E+2019%2E+Learn+to+grow%3A+A+continual+structure+learning+framework+for+overcoming+catastrophic+forgetting%2E+In+International+conference+o)
18. Zhizhong Li and Derek Hoiem. 2017. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12):2935–2947. [find](https://scholar.google.com/scholar?q=Zhizhong+Li+and+Derek+Hoiem%2E+2017%2E+Learning+without+forgetting%2E+IEEE+transactions+on+pattern+analysis+and+machine+intelligence%2C+40%2812%29%3A2935%E2%80%932947%2E)
19. Yan-Shuo Liang and Wu-Jun Li. 2024. Inflora: Interference-free low-rank adaptation for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23638–23647. [find](https://scholar.google.com/scholar?q=Yan%2DShuo+Liang+and+Wu%2DJun+Li%2E+2024%2E+Inflora%3A+Interference%2Dfree+low%2Drank+adaptation+for+continual+learning%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Computer+Vision+and+Pattern+Recognition%2C+pages+2)
20. Chin-Yew Lin. 2004. [ROUGE: A package for automatic evaluation of summaries](https://md.research.elamur.ai/free/en/papers/lin-2004-rouge-package-automatic-evaluation-summaries/). In Text Summarization Branches Out, pages 74–81. Association for Computational Linguistics.
21. Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, and Min-Hung Chen. 2024. Dora: Weight-decomposed low-rank adaptation. In Forty-first International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Shih%2DYang+Liu%2C+Chien%2DYi+Wang%2C+Hongxu+Yin%2C+Pavlo+Molchanov%2C+Yu%2DChiang+Frank+Wang%2C+Kwang%2DTing+Cheng%2C+and+Min%2DHung+Chen%2E+2024%2E+Dora%3A+Weight%2Ddecomposed+low%2Drank+adaptation%2E+In+Forty%2Dfirst+International+Co)
22. David Lopez-Paz and Marc’Aurelio Ranzato. 2017. Gradient episodic memory for continual learning. Advances in neural information processing systems, 30. [find](https://scholar.google.com/scholar?q=David+Lopez%2DPaz+and+Marc%E2%80%99Aurelio+Ranzato%2E+2017%2E+Gradient+episodic+memory+for+continual+learning%2E+Advances+in+neural+information+processing+systems%2C+30%2E)
23. Michael McCloskey and Neal J Cohen. 1989. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of learning and motivation, volume 24, pages 109–165. Elsevier. [find](https://scholar.google.com/scholar?q=Michael+McCloskey+and+Neal+J+Cohen%2E+1989%2E+Catastrophic+interference+in+connectionist+networks%3A+The+sequential+learning+problem%2E+In+Psychology+of+learning+and+motivation%2C+volume+24%2C+pages+109%E2%80%93165%2E+Else)
24. David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne. 2019. Experience replay for continual learning. Advances in neural information processing systems, 32. [find](https://scholar.google.com/scholar?q=David+Rolnick%2C+Arun+Ahuja%2C+Jonathan+Schwarz%2C+Timothy+Lillicrap%2C+and+Gregory+Wayne%2E+2019%2E+Experience+replay+for+continual+learning%2E+Advances+in+neural+information+processing+systems%2C+32%2E)
25. Andrei A Rusu, Neil C Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell. 2016. Progressive neural networks. arXiv preprint arXiv:1606.04671. [find](https://scholar.google.com/scholar?q=Andrei+A+Rusu%2C+Neil+C+Rabinowitz%2C+Guillaume+Desjardins%2C+Hubert+Soyer%2C+James+Kirkpatrick%2C+Koray+Kavukcuoglu%2C+Razvan+Pascanu%2C+and+Raia+Hadsell%2E+2016%2E+Progressive+neural+networks%2E+arXiv+preprint+arXiv%3A16)
26. Chenyang Song, Xu Han, Zheni Zeng, Kuai Li, Chen Chen, Zhiyuan Liu, Maosong Sun, and Tao Yang. 2023. [Conpet: Continual parameter-efficient tuning for large language models](https://md.research.elamur.ai/free/en/papers/song-2023-conpet-continual-parameter-efficient-tuning/). arXiv preprint arXiv:2309.14763.
27. Hanqing Wang, Yixia Li, Shuo Wang, Guanhua Chen, and Yun Chen. 2024. Milora: Harnessing minor singular components for parameter-efficient llm fine-tuning. arXiv preprint arXiv:2406.09044. — [arXiv:2406.09044](https://arxiv.org/abs/2406.09044) [find](https://scholar.google.com/scholar?q=Hanqing+Wang%2C+Yixia+Li%2C+Shuo+Wang%2C+Guanhua+Chen%2C+and+Yun+Chen%2E+2024%2E+Milora%3A+Harnessing+minor+singular+components+for+parameter%2Defficient+llm+fine%2Dtuning%2E+arXiv+preprint+arXiv%3A2406%2E09044%2E+%E2%80%94+%5BarXiv%3A240)
28. Xiao Wang, Tianze Chen, Qiming Ge, Han Xia, Rong Bao, Rui Zheng, Qi Zhang, Tao Gui, and Xuan-Jing Huang. 2023a. Orthogonal subspace learning for language model continual learning. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 10658–10671. [find](https://scholar.google.com/scholar?q=Xiao+Wang%2C+Tianze+Chen%2C+Qiming+Ge%2C+Han+Xia%2C+Rong+Bao%2C+Rui+Zheng%2C+Qi+Zhang%2C+Tao+Gui%2C+and+Xuan%2DJing+Huang%2E+2023a%2E+Orthogonal+subspace+learning+for+language+model+continual+learning%2E+In+Findings+of+the+A)
29. Xiao Wang, Yuansen Zhang, Tianze Chen, Songyang Gao, Senjie Jin, Xianjun Yang, Zhiheng Xi, Rui Zheng, Yicheng Zou, Tao Gui, and 1 others. 2023b. Trace: A comprehensive benchmark for continual learning in large language models. arXiv preprint arXiv:2310.06762. — [arXiv:2310.06762](https://arxiv.org/abs/2310.06762) [find](https://scholar.google.com/scholar?q=Xiao+Wang%2C+Yuansen+Zhang%2C+Tianze+Chen%2C+Songyang+Gao%2C+Senjie+Jin%2C+Xianjun+Yang%2C+Zhiheng+Xi%2C+Rui+Zheng%2C+Yicheng+Zou%2C+Tao+Gui%2C+and+1+others%2E+2023b%2E+Trace%3A+A+comprehensive+benchmark+for+continual+learning)
30. Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. 2022. Learning to prompt for continual learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 139–149. [find](https://scholar.google.com/scholar?q=Zifeng+Wang%2C+Zizhao+Zhang%2C+Chen%2DYu+Lee%2C+Han+Zhang%2C+Ruoxi+Sun%2C+Xiaoqi+Ren%2C+Guolong+Su%2C+Vincent+Perot%2C+Jennifer+Dy%2C+and+Tomas+Pfister%2E+2022%2E+Learning+to+prompt+for+continual+learning%2E+In+Proceedings+of+)
31. Fanxu Meng, Zhaohui Wang, and Muhan Zhang. 2024. [Pissa: Principal singular values and singular vectors adaptation of large language models](https://md.research.elamur.ai/free/en/papers/meng-2024-pissa-principal-singular-values-singular/). Advances in Neural Information Processing Systems, 37:121038– 121072.
32. Genta Winata, Lingjue Xie, Karthik Radhakrishnan, Shijie Wu, Xisen Jin, Pengxiang Cheng, Mayank Kulkarni, and Daniel Preoţiuc-Pietro. 2023. Overcoming catastrophic forgetting in massively multilingual continual learning. In Findings of the Association for Computational Linguistics: ACL 2023, pages… [find](https://scholar.google.com/scholar?q=Genta+Winata%2C+Lingjue+Xie%2C+Karthik+Radhakrishnan%2C+Shijie+Wu%2C+Xisen+Jin%2C+Pengxiang+Cheng%2C+Mayank+Kulkarni%2C+and+Daniel+Preot%CC%A7iuc%2DPietro%2E+2023%2E+Overcoming+catastrophic+forgetting+in+massively+multilingua)
33. Fuli Qiao and Mehrdad Mahdavi. 2024. Learn more, but bother less: parameter efficient continual learning. Advances in Neural Information Processing Systems, 37:97476–97498. [find](https://scholar.google.com/scholar?q=Fuli+Qiao+and+Mehrdad+Mahdavi%2E+2024%2E+Learn+more%2C+but+bother+less%3A+parameter+efficient+continual+learning%2E+Advances+in+Neural+Information+Processing+Systems%2C+37%3A97476%E2%80%9397498%2E)
34. Tongtong Wu, Linhao Luo, Yuan-Fang Li, Shirui Pan, Thuy-Trang Vu, and Gholamreza Haffari. 2024a. Continual learning for [large language models: A survey](https://md.research.elamur.ai/free/en/papers/minaee-2024-large-language-models-survey/). arXiv preprint arXiv:2402.01364.
35. Roger Ratcliff. 1990. Connectionist models of recognition memory: constraints imposed by learning and forgetting functions. Psychological review, 97(2):285. [find](https://scholar.google.com/scholar?q=Roger+Ratcliff%2E+1990%2E+Connectionist+models+of+recognition+memory%3A+constraints+imposed+by+learning+and+forgetting+functions%2E+Psychological+review%2C+97%282%29%3A285%2E)
36. Yichen Wu, Hongming Piao, Long-Kai Huang, Renzhen Wang, Wanhua Li, Hanspeter Pfister, Deyu Meng, Kede Ma, and Ying Wei. 2025. Sd-lora: Scalable decoupled low-rank adaptation for class incremental learning. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Yichen+Wu%2C+Hongming+Piao%2C+Long%2DKai+Huang%2C+Renzhen+Wang%2C+Wanhua+Li%2C+Hanspeter+Pfister%2C+Deyu+Meng%2C+Kede+Ma%2C+and+Ying+Wei%2E+2025%2E+Sd%2Dlora%3A+Scalable+decoupled+low%2Drank+adaptation+for+class+incremental+lear)
37. Anastasia Razdaibiedina, Yuning Mao, Rui Hou, Madian Khabsa, Mike Lewis, and Amjad Almahairi. 2023. Progressive prompts: Continual learning for language models. In The Eleventh International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Anastasia+Razdaibiedina%2C+Yuning+Mao%2C+Rui+Hou%2C+Madian+Khabsa%2C+Mike+Lewis%2C+and+Amjad+Almahairi%2E+2023%2E+Progressive+prompts%3A+Continual+learning+for+language+models%2E+In+The+Eleventh+International+Conferenc)
38. Yichen Wu, Hong Wang, Peilin Zhao, Yefeng Zheng, Ying Wei, and Long-Kai Huang. 2024b. Mitigating catastrophic forgetting in online continual learning by modeling previous task interrelations via pareto optimization. In Forty-first International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Yichen+Wu%2C+Hong+Wang%2C+Peilin+Zhao%2C+Yefeng+Zheng%2C+Ying+Wei%2C+and+Long%2DKai+Huang%2E+2024b%2E+Mitigating+catastrophic+forgetting+in+online+continual+learning+by+modeling+previous+task+interrelations+via+paret)
39. Wei Xu, Courtney Napoles, Ellie Pavlick, Quanze Chen, and Chris Callison-Burch. 2016. [Optimizing statistical machine translation for text simplification](https://md.research.elamur.ai/free/en/papers/xu-2016-optimizing-statistical-machine-translation-text/). In Transactions of the Association for Computational Linguistics, volume 4, pages 401–415.
40. Jaehong Yoon, Eunho Yang, Jeongtae Lee, and Sung Ju Hwang. 2018. Lifelong learning with dynamically expandable networks. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Jaehong+Yoon%2C+Eunho+Yang%2C+Jeongtae+Lee%2C+and+Sung+Ju+Hwang%2E+2018%2E+Lifelong+learning+with+dynamically+expandable+networks%2E+In+International+Conference+on+Learning+Representations%2E)

*Updated 2 November 2026*