---
title: "Sparse Adapter Fusion for Continual Learning in NLP — Zeng 2026"
description: "The authors propose SAFM (Sparse Adapter Fusion Method), which operates on every transformer… 37 references, cited by 1 paper here."
authors: ["Min Zeng", "Xi Chen", "Haiqin Yang", "Yike Guo"]
year: 2026
venue: "Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.eacl-long.37/"
doi: "10.18653/v1/2026.eacl-long.37"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zeng-2026-sparse-adapter-fusion-continual-learning/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zeng-2026-sparse-adapter-fusion-continual-learning/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/17/) / Sparse Adapter Fusion for Continual Learning in NLP

# Sparse Adapter Fusion for Continual Learning in NLP

**Authors:** [Min Zeng](https://md.research.elamur.ai/free/en/authors/min-zeng/), [Xi Chen](https://md.research.elamur.ai/free/en/authors/xi-chen/), Haiqin Yang and [Yike Guo](https://md.research.elamur.ai/free/en/authors/yike-guo/) · **Year:** 2026 · **Venue:** [Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/eacl/) ([EACL 2026](https://md.research.elamur.ai/free/en/venues/eacl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.eacl-long.37/) · [PDF](https://aclanthology.org/2026.eacl-long.37.pdf) · [DOI 10.18653/v1/2026.eacl-long.37](https://doi.org/10.18653/v1/2026.eacl-long.37) · [OpenAlex](https://openalex.org/W7140129212)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 149,439 papers whose bibliographies we read

## Summary
Continual learning systems must take on a sequence of new tasks without losing what they learned on earlier ones, a failure known as catastrophic forgetting. Adapter-based methods such as AdapterCL and ACM protect old tasks by attaching extra modules, but their size grows with every task, and related tasks share little of what they learn. The authors propose SAFM (Sparse Adapter Fusion Method), which operates on every transformer layer of a decoder-only language model. In the first phase, called the decision stage, a warmup step is followed by a layer-by-layer architecture search. At each layer, the search chooses between inserting no adapter, reusing an earlier one, or creating a new one, and it favours the two cheaper options. In the second phase, called the tuning stage, the selected architecture is trained with pseudo-replay of earlier tasks. A layer-wise cosine-similarity loss also keeps the adapters in neighbouring layers distinct. Across several task orders, including E2ENLG and WikiSQL, SAFM matches or exceeds state-of-the-art baselines while using less than 60% of their parameters. This makes continual learning cheaper and more scalable.

## Key points
- **Candidate adapters:** At each layer, SAFM considers k+2 candidates: an empty adapter, the k distinct adapters already learned at that layer, and a newly initialized adapter. The layer's output is a weighted combination of these candidates, and the candidate with the highest weight is kept.
- **Backbone details:** The main backbone is GPT-2, with 12 layers and a hidden size of 768. The authors also scale up to GPT-2-medium, GPT-2-large and Llama3-8B.
- **GPT-2 (124M) results, Order 1, after task 8:** SAFM scores 56.55 with 6.7M learnable parameters, while ACM scores 56.11 with 14.3M.
- **Llama3-8B results, Order 1, after task 8:** SAFM scores 42.78 with 1031.1M learnable parameters, while ACM scores 20.95 with 5279.3M. Both methods perform worse on Llama3-8B than on GPT-2, which the authors attribute to limited training data.
- **Further analysis:** Leaving one adapter layer out of the architecture search improves results, and the gain is larger for higher layers.
- **Case study:** On E2ENLG and WikiSQL examples, ACM drops input fields or produces wrong SQL, while SAFM's outputs match the reference.
- **Limitation:** SAFM handles new information at the task level but not at the sample level, so it may perform worse on outlier samples.
- **Code:** Available at github.com/OzymandiasChen/SAFM.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Hong Kong University of Science and Technology](https://md.research.elamur.ai/free/en/universities/hong-kong-university-of-science-and-technology/) — University, Hong Kong, China · 142 works · 25,136 citations
- [Shenzhen Technology University](https://md.research.elamur.ai/free/en/universities/shenzhen-technology-university/) — University, Shenzhen, China · 3 works · 59 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

## Cite

Cite as: Min Zeng, Xi Chen, Haiqin Yang and Yike Guo. 2026. Sparse Adapter Fusion for Continual Learning in NLP. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zeng2026sparse,
  title = {{Sparse Adapter Fusion for Continual Learning in NLP}},
  author = {Min Zeng, Xi Chen, Haiqin Yang and Yike Guo},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.37},
  url = {https://doi.org/10.18653/v1/2026.eacl-long.37}
}
```

- [← previous SPARD: Self-Paced Curriculum for RL Alignment via Integrating Reward Dynamics and Data Utility](https://md.research.elamur.ai/free/en/papers/zhi-2026-spard-self-paced-curriculum-rl/)
- [next → SPD-Faith Bench: Diagnosing and Improving Faithfulness in Chain-of-Thought for Multimodal Large Language Models](https://md.research.elamur.ai/free/en/papers/lv-2026-spd-faith-bench-diagnosing-improving/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/17/)

## References

37 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. 2018. Memory aware synapses: Learning what (not) to forget. In Proceedings of the European conference on computer vision (ECCV), pages 139–154. [find](https://scholar.google.com/scholar?q=Rahaf+Aljundi%2C+Francesca+Babiloni%2C+Mohamed+Elhoseiny%2C+Marcus+Rohrbach%2C+and+Tinne+Tuytelaars%2E+2018%2E+Memory+aware+synapses%3A+Learning+what+%28not%29+to+forget%2E+In+Proceedings+of+the+European+conference+on+co)
2. Magdalena Biesialska, Katarzyna Biesialska, and Marta R. Costa-jussà. 2020. Continual lifelong learning in natural language processing: A survey. In Proceedings of the 28th International Conference on Computational Linguistics, COLING 2020, Barcelona, Spain (Online), December 8-13, 2020, pages 6523–… [find](https://scholar.google.com/scholar?q=Magdalena+Biesialska%2C+Katarzyna+Biesialska%2C+and+Marta+R%2E+Costa%2Djuss%C3%A0%2E+2020%2E+Continual+lifelong+learning+in+natural+language+processing%3A+A+survey%2E+In+Proceedings+of+the+28th+International+Conference+on)
3. Bill Byrne, Karthik Krishnamoorthi, Chinnadhurai Sankar, Arvind Neelakantan, Daniel Duckworth, Semih Yavuz, Ben Goodrich, Amit Dubey, Andy Cedilnik, and Kyu-Young Kim. 2019. [Taskmaster-1: Toward a realistic and diverse dialog dataset](https://md.research.elamur.ai/free/en/papers/byrne-2019-taskmaster-1-realistic-diverse-dialog/). arXiv preprint arXiv:1909.05358. the Americas Chapter of the Asso…
4. Dumitru Erhan, Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2009. Visualizing higher-layer features of a deep network. University of Montreal, 1341(3):1. [find](https://scholar.google.com/scholar?q=Dumitru+Erhan%2C+Yoshua+Bengio%2C+Aaron+Courville%2C+and+Pascal+Vincent%2E+2009%2E+Visualizing+higher%2Dlayer+features+of+a+deep+network%2E+University+of+Montreal%2C+1341%283%29%3A1%2E)
5. Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. 2022. [Training compute-optimal large language models](https://md.research.elamur.ai/free/en/papers/hoffmann-2022-training-compute-optimal-large-language/). arxiv. arXiv preprint arXiv:2203.15556.
6. Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for nlp. In ICML, pages 2790–2799. [find](https://scholar.google.com/scholar?q=Neil+Houlsby%2C+Andrei+Giurgiu%2C+Stanislaw+Jastrzebski%2C+Bruna+Morrone%2C+Quentin+De+Laroussilhe%2C+Andrea+Gesmundo%2C+Mona+Attariyan%2C+and+Sylvain+Gelly%2E+2019%2E+Parameter%2Defficient+transfer+learning+for+nlp%2E+In+)
7. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. [Scaling laws for neural language models](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/). arXiv preprint arXiv:2001.08361.
8. James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. 2017. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521… [find](https://scholar.google.com/scholar?q=James+Kirkpatrick%2C+Razvan+Pascanu%2C+Neil+Rabinowitz%2C+Joel+Veness%2C+Guillaume+Desjardins%2C+Andrei+A+Rusu%2C+Kieran+Milan%2C+John+Quan%2C+Tiago+Ramalho%2C+Agnieszka+Grabska%2DBarwinska%2C+et+al%2E+2017%2E+Overcoming+catas)
9. David Lopez-Paz and Marc’Aurelio Ranzato. 2017. Gradient episodic memory for continual learning. In NIPS, volume 30. [find](https://scholar.google.com/scholar?q=David+Lopez%2DPaz+and+Marc%E2%80%99Aurelio+Ranzato%2E+2017%2E+Gradient+episodic+memory+for+continual+learning%2E+In+NIPS%2C+volume+30%2E)
10. Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101. [find](https://scholar.google.com/scholar?q=Ilya+Loshchilov+and+Frank+Hutter%2E+2017%2E+Decoupled+weight+decay+regularization%2E+arXiv+preprint+arXiv%3A1711%2E05101%2E)
11. Andrea Madotto, Zhaojiang Lin, Zhenpeng Zhou, Seungwhan Moon, Paul Crook, Bing Liu, Zhou Yu, Eunjoon Cho, Pascale Fung, and Zhiguang Wang. 2021. Continual learning in task-oriented dialogue systems. In EMNLP. [find](https://scholar.google.com/scholar?q=Andrea+Madotto%2C+Zhaojiang+Lin%2C+Zhenpeng+Zhou%2C+Seungwhan+Moon%2C+Paul+Crook%2C+Bing+Liu%2C+Zhou+Yu%2C+Eunjoon+Cho%2C+Pascale+Fung%2C+and+Zhiguang+Wang%2E+2021%2E+Continual+learning+in+task%2Doriented+dialogue+systems%2E+I)
12. Jorge A. Mendez and Eric Eaton. 2023. How to reuse and compose knowledge for a lifetime of tasks: A survey on continual learning and functional composition. [find](https://scholar.google.com/scholar?q=Jorge+A%2E+Mendez+and+Eric+Eaton%2E+2023%2E+How+to+reuse+and+compose+knowledge+for+a+lifetime+of+tasks%3A+A+survey+on+continual+learning+and+functional+composition%2E)
13. Trans. Mach. Learn. Res., 2023. [find](https://scholar.google.com/scholar?q=Trans%2E+Mach%2E+Learn%2E+Res%2E%2C+2023%2E)
14. Rich Caruana. 1997. Multitask learning. Machine learning, 28:41–75. [find](https://scholar.google.com/scholar?q=Rich+Caruana%2E+1997%2E+Multitask+learning%2E+Machine+learning%2C+28%3A41%E2%80%9375%2E)
15. Fei Mi, Liangwei Chen, Mengjie Zhao, Minlie Huang, and Boi Faltings. 2020. Continual learning for natural language generation in task-oriented dialog systems. In EMNLP, pages 3461–3474. [find](https://scholar.google.com/scholar?q=Fei+Mi%2C+Liangwei+Chen%2C+Mengjie+Zhao%2C+Minlie+Huang%2C+and+Boi+Faltings%2E+2020%2E+Continual+learning+for+natural+language+generation+in+task%2Doriented+dialog+systems%2E+In+EMNLP%2C+pages+3461%E2%80%933474%2E)
16. Xi Chen and Min Zeng. 2025. Prototype conditioned generative replay for continual learning in nlp. In Proceedings of the 2025 Conference of the Nations of Jekaterina Novikova, Ondřej Dušek, and Verena Rieser. 2017. [The e2e dataset: New challenges for end-to-end generation](https://md.research.elamur.ai/free/en/papers/novikova-2017-e2e-dataset-new-challenges-end/). arXiv preprint arXiv:1706…
17. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318.
18. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+Ilya+Sutskever%2C+et+al%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E+OpenAI+blog%2C+1%288%29%3A9%2E)
19. (Volume 1: Long Papers), pages 663–677, Mexico City, Mexico. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=%28Volume+1%3A+Long+Papers%29%2C+pages+663%E2%80%93677%2C+Mexico+City%2C+Mexico%2E+Association+for+Computational+Linguistics%2E)
20. Tsung-Hsien Wen, Milica Gasic, Nikola Mrksic, Pei-Hao Su, David Vandyke, and Steve Young. 2015. [Semantically conditioned lstm-based natural language generation for spoken dialogue systems](https://md.research.elamur.ai/free/en/papers/wen-2015-semantically-conditioned-lstm-based-natural/). arXiv preprint arXiv:1508.01745.
21. Chien-Sheng Wu, Andrea Madotto, Ehsan Hosseini-Asl, Caiming Xiong, Richard Socher, and Pascale Fung. 2019. [Transferable multi-domain state generator for task-oriented dialogue systems](https://md.research.elamur.ai/free/en/papers/wu-2019-transferable-multi-domain-state-generator/). arXiv preprint arXiv:1905.08743.
22. Abhinav Rastogi, Xiaoxue Zang, Srinivas Sunkara, Raghav Gupta, and Pranav Khaitan. 2020. [Towards scalable multi-domain conversational agents: The schema-guided dialogue dataset](https://md.research.elamur.ai/free/en/papers/rastogi-2019-towards-scalable-multi-domain-conversational/). In Proceedings of the AAAI conference on artificial intelligence, 05, pages 8689–8696.
23. Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H Lampert. 2017. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, pages 2001–2010. [find](https://scholar.google.com/scholar?q=Sylvestre%2DAlvise+Rebuffi%2C+Alexander+Kolesnikov%2C+Georg+Sperl%2C+and+Christoph+H+Lampert%2E+2017%2E+icarl%3A+Incremental+classifier+and+representation+learning%2E+In+Proceedings+of+the+IEEE+conference+on+Computer)
24. Jonathan Schwarz, Wojciech Czarnecki, Jelena Luketina, Agnieszka Grabska-Barwinska, Yee Whye Teh, Razvan Pascanu, and Raia Hadsell. 2018. Progress & compress: A scalable framework for continual learning. In ICML, pages 4528–4537. [find](https://scholar.google.com/scholar?q=Jonathan+Schwarz%2C+Wojciech+Czarnecki%2C+Jelena+Luketina%2C+Agnieszka+Grabska%2DBarwinska%2C+Yee+Whye+Teh%2C+Razvan+Pascanu%2C+and+Raia+Hadsell%2E+2018%2E+Progress+%26+compress%3A+A+scalable+framework+for+continual+learni)
25. Abigail See, Peter J Liu, and Christopher D Manning. 2017. Get to the point: Summarization with pointer-generator networks. arXiv preprint arXiv:1704.04368. [find](https://scholar.google.com/scholar?q=Abigail+See%2C+Peter+J+Liu%2C+and+Christopher+D+Manning%2E+2017%2E+Get+to+the+point%3A+Summarization+with+pointer%2Dgenerator+networks%2E+arXiv+preprint+arXiv%3A1704%2E04368%2E)
26. Fan-Keng Sun, Cheng-Hao Ho, and Hung-Yi Lee. 2019. Lamol: Language modeling for lifelong language learning. arXiv preprint arXiv:1909.03329. [find](https://scholar.google.com/scholar?q=Fan%2DKeng+Sun%2C+Cheng%2DHao+Ho%2C+and+Hung%2DYi+Lee%2E+2019%2E+Lamol%3A+Language+modeling+for+lifelong+language+learning%2E+arXiv+preprint+arXiv%3A1909%2E03329%2E)
27. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). arXiv preprint arXiv:2302.13971.
28. Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. 2024a. A comprehensive survey of continual learning: Theory, method and application. IEEE Trans. Pattern Anal. Mach. Intell., 46(8):5362–5383. [find](https://scholar.google.com/scholar?q=Liyuan+Wang%2C+Xingxing+Zhang%2C+Hang+Su%2C+and+Jun+Zhu%2E+2024a%2E+A+comprehensive+survey+of+continual+learning%3A+Theory%2C+method+and+application%2E+IEEE+Trans%2E+Pattern+Anal%2E+Mach%2E+Intell%2E%2C+46%288%29%3A5362%E2%80%935383%2E)
29. Xiao Wang, Tianze Chen, Qiming Ge, Han Xia, Rong Bao, Rui Zheng, Qi Zhang, Tao Gui, and Xuanjing Huang. 2023. Orthogonal subspace learning for language model continual learning. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 10658–10671, Singapore. Association for Co… [find](https://scholar.google.com/scholar?q=Xiao+Wang%2C+Tianze+Chen%2C+Qiming+Ge%2C+Han+Xia%2C+Rong+Bao%2C+Rui+Zheng%2C+Qi+Zhang%2C+Tao+Gui%2C+and+Xuanjing+Huang%2E+2023%2E+Orthogonal+subspace+learning+for+language+model+continual+learning%2E+In+Findings+of+the+Ass)
30. Yifan Wang, Yafei Liu, Chufan Shi, Haoling Li, Chen Chen, Haonan Lu, and Yujiu Yang. 2024b. InsCL: A data-efficient continual learning paradigm for fine-tuning large language models with instructions. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computat… [find](https://scholar.google.com/scholar?q=Yifan+Wang%2C+Yafei+Liu%2C+Chufan+Shi%2C+Haoling+Li%2C+Chen+Chen%2C+Haonan+Lu%2C+and+Yujiu+Yang%2E+2024b%2E+InsCL%3A+A+data%2Defficient+continual+learning+paradigm+for+fine%2Dtuning+large+language+models+with+instructions%2E)
31. Min Zeng, Haiqin Yang, Xi Chen, and Yike Guo. 2025. Task-wrapped continual learning in task-oriented dialogue systems. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 3173–3183. [find](https://scholar.google.com/scholar?q=Min+Zeng%2C+Haiqin+Yang%2C+Xi+Chen%2C+and+Yike+Guo%2E+2025%2E+Task%2Dwrapped+continual+learning+in+task%2Doriented+dialogue+systems%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+NAACL+2025%2C+pages+3)
32. Min Zeng, Haiqin Yang, Wei Xue, Qifeng Liu, and Yike Guo. 2024. Dirichlet continual learning: Tackling catastrophic forgetting in nlp. In The 40th Conference on Uncertainty in Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Min+Zeng%2C+Haiqin+Yang%2C+Wei+Xue%2C+Qifeng+Liu%2C+and+Yike+Guo%2E+2024%2E+Dirichlet+continual+learning%3A+Tackling+catastrophic+forgetting+in+nlp%2E+In+The+40th+Conference+on+Uncertainty+in+Artificial+Intelligence%2E)
33. Friedemann Zenke, Ben Poole, and Surya Ganguli. 2017. Continual learning through synaptic intelligence. In ICML, pages 3987–3995. [find](https://scholar.google.com/scholar?q=Friedemann+Zenke%2C+Ben+Poole%2C+and+Surya+Ganguli%2E+2017%2E+Continual+learning+through+synaptic+intelligence%2E+In+ICML%2C+pages+3987%E2%80%933995%2E)
34. Yanzhe Zhang, Xuezhi Wang, and Diyi Yang. 2022. Continual sequence generation with adaptive compositional modules. In ACL. [find](https://scholar.google.com/scholar?q=Yanzhe+Zhang%2C+Xuezhi+Wang%2C+and+Diyi+Yang%2E+2022%2E+Continual+sequence+generation+with+adaptive+compositional+modules%2E+In+ACL%2E)
35. Weixiang Zhao, Shilong Wang, Yulin Hu, Yanyan Zhao, Bing Qin, Xuanyu Zhang, Qing Yang, Dongliang Xu, and Wanxiang Che. 2024. SAPT: A shared attention framework for parameter-efficient continual learning of large language models. In Proceedings of the 62nd Annual Meeting of the Association for Comput… [find](https://scholar.google.com/scholar?q=Weixiang+Zhao%2C+Shilong+Wang%2C+Yulin+Hu%2C+Yanyan+Zhao%2C+Bing+Qin%2C+Xuanyu+Zhang%2C+Qing+Yang%2C+Dongliang+Xu%2C+and+Wanxiang+Che%2E+2024%2E+SAPT%3A+A+shared+attention+framework+for+parameter%2Defficient+continual+learni)
36. Victor Zhong, Caiming Xiong, and Richard Socher. 2017. [Seq2sql: Generating structured queries from natural language using reinforcement learning](https://md.research.elamur.ai/free/en/papers/zhong-2017-seq2sql-generating-structured-queries-natural/). arXiv preprint arXiv:1709.00103.
37. Qi Zhu, Bing Li, Fei Mi, Xiaoyan Zhu, and Minlie Huang. 2022. Continual prompt tuning for dialog state tracking. In ACL, pages 1124–1137. [find](https://scholar.google.com/scholar?q=Qi+Zhu%2C+Bing+Li%2C+Fei+Mi%2C+Xiaoyan+Zhu%2C+and+Minlie+Huang%2E+2022%2E+Continual+prompt+tuning+for+dialog+state+tracking%2E+In+ACL%2C+pages+1124%E2%80%931137%2E)

*Updated 31 October 2026*