---
title: "Safety Alignment via Constrained Knowledge Unlearning (2025)"
description: "They propose Constrained Knowledge Unlearning (CKU), a safety alignment method with three parts. 71 references, cited by 9 papers here."
authors: ["Zesheng Shi", "Yucheng Zhou", "Jing Li", "Yuxin Jin", "Yu Li", "Daojing He", "Fangming Liu", "Saleh Alharbi", "Jun Yu", "Min Zhang"]
year: 2025
venue: "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2025.acl-long.1240/"
doi: "10.18653/v1/2025.acl-long.1240"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/shi-2025-safety-alignment-constrained-knowledge-unlearning/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/shi-2025-safety-alignment-constrained-knowledge-unlearning/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/7/) / Safety Alignment via Constrained Knowledge Unlearning

# Safety Alignment via Constrained Knowledge Unlearning

**Authors:** [Zesheng Shi](https://md.research.elamur.ai/free/en/authors/zesheng-shi/), [Yucheng Zhou](https://md.research.elamur.ai/free/en/authors/yucheng-zhou/), [Jing Li](https://md.research.elamur.ai/free/en/authors/jing-li/), Yuxin Jin, [Yu Li](https://md.research.elamur.ai/free/en/authors/yu-li/), [Daojing He](https://md.research.elamur.ai/free/en/authors/daojing-he/), [Fangming Liu](https://md.research.elamur.ai/free/en/authors/fangming-liu/), Saleh Alharbi, [Jun Yu](https://md.research.elamur.ai/free/en/authors/jun-yu/) and [Min Zhang](https://md.research.elamur.ai/free/en/authors/min-zhang/) · **Year:** 2025 · **Venue:** [Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2025](https://md.research.elamur.ai/free/en/venues/acl/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.acl-long.1240/) · [PDF](https://aclanthology.org/2025.acl-long.1240.pdf) · [DOI 10.18653/v1/2025.acl-long.1240](https://doi.org/10.18653/v1/2025.acl-long.1240) · [OpenAlex](https://openalex.org/W4412889911)

**Citations:** 4 citations (OpenAlex, 2026-10-14) · [9](https://md.research.elamur.ai/free/en/papers/shi-2025-safety-alignment-constrained-knowledge-unlearning/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Aligned large language models still produce harmful content under jailbreak attacks. The authors argue this happens because standard defenses, such as continued safety training, teach models to refuse without removing the dangerous knowledge itself, and they can also erode general skills. They propose Constrained Knowledge Unlearning (CKU), a safety alignment method with three parts.

- **Knowledge localization and retention:** neurons in selected MLP layers are scored on an identification dataset to find a set U that carries useful knowledge.
- **Harmful knowledge unlearning:** harmful prompts are forgotten through gradient ascent, while the gradients of neurons in U are pruned so that knowledge stays intact.
- **Unlearning regularization:** a term weighted by λ limits how far the loss can be pushed.

Experiments on Llama2-7B-Chat, Llama-3-8B-Instruct and Qwen2.5-7B-Instruct show large safety gains with only a small loss of utility, a better trade-off than existing methods. The study also analyzes how sensitive neurons in different MLP layers are to knowledge. These findings matter for jailbreak defense and are also relevant to knowledge editing and model pruning.

## Key points
- CKU freezes the top-scoring neurons, giving a Neuron Locking Rate (NLR) of 0.8, which noticeably improves safety. Applying unlearning to only some MLP layers yields clear safety gains at little utility cost.
- The regularizer λ was tested at 0, 0.2, 1.0, 1.5, 2.0 and 2.5 on Llama2-7B-Chat. As λ grows, both GCG attack success rate (ASR) and average accuracy fall, and values above 1 noticeably hurt utility. λ = 1.5 gave the best balance.
- On Arena-Hard, CKU models score almost the same as their base models, within 95% confidence intervals:
  - Llama-2-7b-chat: 4.7 → 4.8
  - Llama-3-8B-Instruct: 32.3 → 32.1
  - Qwen2.5-7B-Instruct: 71.4 → 71.2
- Average response length also stays nearly unchanged on Arena-Hard (for example, 572 vs 565 tokens for Llama2).
- The authors list two limitations: effectiveness differs across domains and datasets, and the models may give less nuanced answers in complex or ambiguous contexts.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Harbin Institute of Technology](https://md.research.elamur.ai/free/en/universities/harbin-institute-of-technology/) — University, Harbin, China · 158 works · 28,841 citations
- [Zhejiang University](https://md.research.elamur.ai/free/en/universities/zhejiang-university/) — University, Hangzhou, China · 172 works · 12,988 citations
- [University of Macau](https://md.research.elamur.ai/free/en/universities/university-of-macau/) — University, Macao, China · 33 works · 2,223 citations
- [Peng Cheng Laboratory](https://md.research.elamur.ai/free/en/labs/peng-cheng-laboratory/) — Research institute, Shenzhen, China · 38 works · 2,504 citations
- [Nankai University](https://md.research.elamur.ai/free/en/universities/nankai-university/) — University, Tianjin, China · 24 works · 762 citations
- [Shaqra University](https://md.research.elamur.ai/free/en/universities/shaqra-university/) — University, Riyadh, Saudi Arabia · 2 works · 14 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🛡️ Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/) · 517 works

## Cite

Cite as: Zesheng Shi, Yucheng Zhou, Jing Li, Yuxin Jin, Yu Li, Daojing He, Fangming Liu, Saleh Alharbi, Jun Yu and Min Zhang. 2025. Safety Alignment via Constrained Knowledge Unlearning. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{shi2025safety,
  title = {{Safety Alignment via Constrained Knowledge Unlearning}},
  author = {Zesheng Shi, Yucheng Zhou, Jing Li, Yuxin Jin, Yu Li, Daojing He, Fangming Liu, Saleh Alharbi, Jun Yu and Min Zhang},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.1240},
  url = {https://doi.org/10.18653/v1/2025.acl-long.1240}
}
```

- [← previous Safer or Luckier? LLMs as Safety Evaluators Are Not Robust to Artifacts](https://md.research.elamur.ai/free/en/papers/chen-2025-safer-or-luckier-llms-as/)
- [next → SSA-COMET: Do LLMs Outperform Learned Metrics in Evaluating MT for Under-Resourced African Languages?](https://md.research.elamur.ai/free/en/papers/li-2025-ssa-comet-do-llms-outperform/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/7/)

## References

71 references, 27 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, and Dawn Drain. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. [find](https://scholar.google.com/scholar?q=Yuntao+Bai%2C+Andy+Jones%2C+Kamal+Ndousse%2C+Amanda+Askell%2C+Anna+Chen%2C+Nova+DasSarma%2C+and+Dawn+Drain%2E+2022%2E+Training+a+helpful+and+harmless+assistant+with+reinforcement+learning+from+human+feedback%2E)
2. Lucas Bourtoule, Varun Chandrasekaran, Christopher A. Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. 2021. Machine unlearning. In In Proceedings of the Conference on 42nd IEEE Symposium on Security and Privacy (SP), pages 141–159. [find](https://scholar.google.com/scholar?q=Lucas+Bourtoule%2C+Varun+Chandrasekaran%2C+Christopher+A%2E+Choquette%2DChoo%2C+Hengrui+Jia%2C+Adelin+Travers%2C+Baiwu+Zhang%2C+David+Lie%2C+and+Nicolas+Papernot%2E+2021%2E+Machine+unlearning%2E+In+In+Proceedings+of+the+Conf)
3. Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, and Eric Wong. 2024. [Jailbreaking black box large language models in twenty queries](https://md.research.elamur.ai/free/en/papers/chao-2023-jailbreaking-black-box-large-language/).
4. Jiaao Chen and Diyi Yang. 2023. [Unlearn what you want to forget: Efficient unlearning for llms](https://md.research.elamur.ai/free/en/papers/chen-2023-unlearn-what-you-want-forget/). In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 12041–12052.
5. Damai Dai, Li Dong, Yaru Hao, Zhifang Sui, Baobao Chang, and Furu Wei. 2022. Knowledge neurons in pretrained transformers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL), pages 8493–8502. [find](https://scholar.google.com/scholar?q=Damai+Dai%2C+Li+Dong%2C+Yaru+Hao%2C+Zhifang+Sui%2C+Baobao+Chang%2C+and+Furu+Wei%2E+2022%2E+Knowledge+neurons+in+pretrained+transformers%2E+In+Proceedings+of+the+60th+Annual+Meeting+of+the+Association+for+Computationa)
6. Josef Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji, Xinbo Xu, and Mickel Liu. 2024. [Safe RLHF: safe reinforcement learning from human feedback](https://md.research.elamur.ai/free/en/papers/dai-2023-safe-rlhf-safe-reinforcement-learning/). In Proceedings of The Twelfth International Conference on Learning Representations (ICLR).
7. Boyi Deng, Wenjie Wang, Fuli Feng, Yang Deng, Qifan Wang, and Xiangnan He. 2023. Attack prompt generation for red teaming and defending large language models. In Findings of the Association for Computational Linguistics (EMNLP), pages 2176–2189. [find](https://scholar.google.com/scholar?q=Boyi+Deng%2C+Wenjie+Wang%2C+Fuli+Feng%2C+Yang+Deng%2C+Qifan+Wang%2C+and+Xiangnan+He%2E+2023%2E+Attack+prompt+generation+for+red+teaming+and+defending+large+language+models%2E+In+Findings+of+the+Association+for+Comput)
8. Yue Deng, Wenxuan Zhang, Sinno Jialin Pan, and Lidong Bing. 2024. [Multilingual jailbreak challenges in large language models](https://md.research.elamur.ai/free/en/papers/deng-2023-multilingual-jailbreak-challenges-large-language/). In Proceedings of The Twelfth International Conference on Learning Representations (ICLR).
9. Guodong Du, Zitao Fang, Jing Li, Junlin Li, Runhua Jiang, Shuyang Yu, Yifei Guo, Yangneng Chen, Sim Kuan Goh, Ho-Kin Tang, Daojing He, Honghai Liu, and Min Zhang. 2025a. [Neural parameter search for slimmer fine-tuned models and better transfer](https://md.research.elamur.ai/free/en/papers/du-2025-neural-parameter-search-slimmer-fine/). arXiv preprint arXiv:2505.18713.
10. Guodong Du, Runhua Jiang, Senqiao Yang, Haoyang Li, Wei Chen, Keren Li, Sim Kuan Goh, and Ho-Kin Tang. 2024a. Impacts of darwinian evolution on pre-trained deep neural networks. In 2024 IEEE International Conference on Systems, Man, and Cybernetics (SMC). [find](https://scholar.google.com/scholar?q=Guodong+Du%2C+Runhua+Jiang%2C+Senqiao+Yang%2C+Haoyang+Li%2C+Wei+Chen%2C+Keren+Li%2C+Sim+Kuan+Goh%2C+and+Ho%2DKin+Tang%2E+2024a%2E+Impacts+of+darwinian+evolution+on+pre%2Dtrained+deep+neural+networks%2E+In+2024+IEEE+Internati)
11. Guodong Du, Junlin Lee, Jing Li, Runhua Jiang, Yifei Guo, Shuyang Yu, Hanting Liu, Sim Kuan Goh, Ho-Kin Tang, Daojing He, and Min Zhang. 2024b. Parameter competition balancing for model merging. In The Thirty-eighth Annual Conference on Neural Information Processing Systems (NeurIPS). [find](https://scholar.google.com/scholar?q=Guodong+Du%2C+Junlin+Lee%2C+Jing+Li%2C+Runhua+Jiang%2C+Yifei+Guo%2C+Shuyang+Yu%2C+Hanting+Liu%2C+Sim+Kuan+Goh%2C+Ho%2DKin+Tang%2C+Daojing+He%2C+and+Min+Zhang%2E+2024b%2E+Parameter+competition+balancing+for+model+merging%2E+In+Th)
12. Guodong Du, Jing Li, Hanting Liu, Runhua Jiang, Shuyang Yu, Yifei Guo, Sim Kuan Goh, and Ho-Kin Tang. 2024c. Knowledge fusion by evolving weights of language models. In Findings of the Association for Computational Linguistics: ACL 2024. [find](https://scholar.google.com/scholar?q=Guodong+Du%2C+Jing+Li%2C+Hanting+Liu%2C+Runhua+Jiang%2C+Shuyang+Yu%2C+Yifei+Guo%2C+Sim+Kuan+Goh%2C+and+Ho%2DKin+Tang%2E+2024c%2E+Knowledge+fusion+by+evolving+weights+of+language+models%2E+In+Findings+of+the+Association+for)
13. Guodong Du, Xuanning Zhou, Junlin Li, Zhuo Li, Zesheng Shi, Wanyu Lin, Ho-Kin Tang, Xiucheng Li, Fangming Liu, Wenya Wang, Min Zhang, and Jing Li. 2025b. Knowledge grafting of large language models. arXiv preprint arXiv:2505.18502. — [arXiv:2505.18502](https://arxiv.org/abs/2505.18502) [find](https://scholar.google.com/scholar?q=Guodong+Du%2C+Xuanning+Zhou%2C+Junlin+Li%2C+Zhuo+Li%2C+Zesheng+Shi%2C+Wanyu+Lin%2C+Ho%2DKin+Tang%2C+Xiucheng+Li%2C+Fangming+Liu%2C+Wenya+Wang%2C+Min+Zhang%2C+and+Jing+Li%2E+2025b%2E+Knowledge+grafting+of+large+language+models%2E+a)
14. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, and et al. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). CoRR.
15. Myles Foley, Ambrish Rawat, Taesung Lee, Yufang Hou, Gabriele Picco, and Giulio Zizzo. 2023. Matching pairs: Attributing fine-tuned models to their pre-trained large language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL), pages 7423–7442. [find](https://scholar.google.com/scholar?q=Myles+Foley%2C+Ambrish+Rawat%2C+Taesung+Lee%2C+Yufang+Hou%2C+Gabriele+Picco%2C+and+Giulio+Zizzo%2E+2023%2E+Matching+pairs%3A+Attributing+fine%2Dtuned+models+to+their+pre%2Dtrained+large+language+models%2E+In+Proceedings+of)
16. Simon Geisler, Tom Wollschläger, M. H. I. Abdalla, Johannes Gasteiger, and Stephan Günnemann. 2024. Attacking large language models with projected gradient descent. [find](https://scholar.google.com/scholar?q=Simon+Geisler%2C+Tom+Wollschl%C3%A4ger%2C+M%2E+H%2E+I%2E+Abdalla%2C+Johannes+Gasteiger%2C+and+Stephan+G%C3%BCnnemann%2E+2024%2E+Attacking+large+language+models+with+projected+gradient+descent%2E)
17. Mor Geva, Roei Schuster, Jonathan Berant, and Omer Levy. 2021. Transformer feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5484–5495. [find](https://scholar.google.com/scholar?q=Mor+Geva%2C+Roei+Schuster%2C+Jonathan+Berant%2C+and+Omer+Levy%2E+2021%2E+Transformer+feed%2Dforward+layers+are+key%2Dvalue+memories%2E+In+Proceedings+of+the+2021+Conference+on+Empirical+Methods+in+Natural+Language+Pr)
18. Gustavo Gonçalves and Emma Strubell. 2023. Understanding the effect of model compression on social bias in large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2663–2675. [find](https://scholar.google.com/scholar?q=Gustavo+Gon%C3%A7alves+and+Emma+Strubell%2E+2023%2E+Understanding+the+effect+of+model+compression+on+social+bias+in+large+language+models%2E+In+Proceedings+of+the+2023+Conference+on+Empirical+Methods+in+Natural+)
19. Saaketh Koundinya Gundavarapu, Shreya Agarwal, Arushi Arora, and Chandana Thimmalapura Jagadee-shaiah. 2024. Machine unlearning in large language models. [find](https://scholar.google.com/scholar?q=Saaketh+Koundinya+Gundavarapu%2C+Shreya+Agarwal%2C+Arushi+Arora%2C+and+Chandana+Thimmalapura+Jagadee%2Dshaiah%2E+2024%2E+Machine+unlearning+in+large+language+models%2E)
20. Zhiyuan Hu, Yuliang Liu, Jinman Zhao, Suyuchen Wang, Yan Wang, Wei Shen, Qing Gu, Anh Tuan Luu, See-Kiong Ng, Zhiwei Jiang, and Bryan Hooi. 2024. [Longrecipe: Recipe for efficient long context generalization in large language models](https://md.research.elamur.ai/free/en/papers/hu-2025-longrecipe-recipe-efficient-long-context/).
21. Xiaowei Huang, Wenjie Ruan, Wei Huang, Gaojie Jin, Yi Dong, and Changshun Wu. 2023. A survey of safety and trustworthiness of large language models through the lens of verification and validation. [find](https://scholar.google.com/scholar?q=Xiaowei+Huang%2C+Wenjie+Ruan%2C+Wei+Huang%2C+Gaojie+Jin%2C+Yi+Dong%2C+and+Changshun+Wu%2E+2023%2E+A+survey+of+safety+and+trustworthiness+of+large+language+models+through+the+lens+of+verification+and+validation%2E)
22. Yangsibo Huang, Samyak Gupta, Mengzhou Xia, Kai Li, and Danqi Chen. 2024. [Catastrophic jailbreak of open-source llms via exploiting generation](https://md.research.elamur.ai/free/en/papers/huang-2023-catastrophic-jailbreak-open-source-llms/). In Proceedings of The Twelfth International Conference on Learning Representations (ICLR).
23. Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Minjoon Seo. 2023. [Knowledge unlearning for mitigating privacy risks in language models](https://md.research.elamur.ai/free/en/papers/jang-2022-knowledge-unlearning-mitigating-privacy-risks/). In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pages 14389–14408.
24. Jiaming Ji, Mickel Liu, Josef Dai, Xuehai Pan, Chi Zhang, and Ce Bian. 2023. [Beavertails: Towards improved safety alignment of LLM via a human-preference dataset](https://md.research.elamur.ai/free/en/papers/ji-2023-beavertails-towards-improved-safety-alignment/). In Proceedings of Advances in Neural Information Processing Systems (NeurIPS).
25. Xiaojun Jia, Tianyu Pang, Chao Du, Yihao Huang, Jindong Gu, Yang Liu, Xiaochun Cao, and Min Lin. 2024. Improved techniques for optimization-based jailbreaking on large language models. [find](https://scholar.google.com/scholar?q=Xiaojun+Jia%2C+Tianyu+Pang%2C+Chao+Du%2C+Yihao+Huang%2C+Jindong+Gu%2C+Yang+Liu%2C+Xiaochun+Cao%2C+and+Min+Lin%2E+2024%2E+Improved+techniques+for+optimization%2Dbased+jailbreaking+on+large+language+models%2E)
26. Masahiro Kaneko, Danushka Bollegala, and Naoaki Okazaki. 2022. Debiasing isn’t enough! - on the effectiveness of debiasing mlms and their social biases in downstream tasks. In Proceedings of the 29th International Conference on Computational Linguistics (COLING), pages 1299–1310. [find](https://scholar.google.com/scholar?q=Masahiro+Kaneko%2C+Danushka+Bollegala%2C+and+Naoaki+Okazaki%2E+2022%2E+Debiasing+isn%E2%80%99t+enough%21+%2D+on+the+effectiveness+of+debiasing+mlms+and+their+social+biases+in+downstream+tasks%2E+In+Proceedings+of+the+29th+)
27. Robert Kirk, Ishita Mediratta, Christoforos Nalmpantis, Jelena Luketina, and Eric Hambro. 2024. [Understanding the effects of RLHF on LLM generalisation and diversity](https://md.research.elamur.ai/free/en/papers/kirk-2023-understanding-effects-rlhf-llm-generalisation/). In roceedings of The Twelfth International Conference on Learning Representations (ICLR).
28. Harrison Lee, Samrat Phatale, Hassan Mansoor, Thomas Mesnard, and Johan Ferret. 2023. Rlaif: Scaling reinforcement learning from human feedback with ai feedback. [find](https://scholar.google.com/scholar?q=Harrison+Lee%2C+Samrat+Phatale%2C+Hassan+Mansoor%2C+Thomas+Mesnard%2C+and+Johan+Ferret%2E+2023%2E+Rlaif%3A+Scaling+reinforcement+learning+from+human+feedback+with+ai+feedback%2E)
29. Junlin Lee, Yequan Wang, Jing Li, and Min Zhang. 2024. Multimodal reasoning with multimodal knowledge graph. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). [find](https://scholar.google.com/scholar?q=Junlin+Lee%2C+Yequan+Wang%2C+Jing+Li%2C+and+Min+Zhang%2E+2024%2E+Multimodal+reasoning+with+multimodal+knowledge+graph%2E+In+Proceedings+of+the+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28ACL%29)
30. Namhoon Lee, Thalaiyasingam Ajanthan, and Philip H. S. Torr. 2019. Snip: single-shot network pruning based on connection sensitivity. In Proceedings of 7th International Conference on Learning Representations (ICLR). [find](https://scholar.google.com/scholar?q=Namhoon+Lee%2C+Thalaiyasingam+Ajanthan%2C+and+Philip+H%2E+S%2E+Torr%2E+2019%2E+Snip%3A+single%2Dshot+network+pruning+based+on+connection+sensitivity%2E+In+Proceedings+of+7th+International+Conference+on+Learning+Represe)
31. Sheng-Chieh Lin, Luyu Gao, Barlas Oguz, Wenhan Xiong, Jimmy Lin, Wen tau Yih, and Xilun Chen. 2024. Flame: Factuality-aware alignment for large language models. [find](https://scholar.google.com/scholar?q=Sheng%2DChieh+Lin%2C+Luyu+Gao%2C+Barlas+Oguz%2C+Wenhan+Xiong%2C+Jimmy+Lin%2C+Wen+tau+Yih%2C+and+Xilun+Chen%2E+2024%2E+Flame%3A+Factuality%2Daware+alignment+for+large+language+models%2E)
32. Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. 2024. [Autodan: Generating stealthy jailbreak prompts on aligned large language models](https://md.research.elamur.ai/free/en/papers/liu-2023-autodan-generating-stealthy-jailbreak-prompts/). In Proceedings of The Twelfth International Conference on Learning Representations (ICLR).
33. Weikai Lu, Ziqian Zeng, Jianwei Wang, Zhengdong Lu, Zelin Chen, Huiping Zhuang, and Cen Chen. 2024. Eraser: Jailbreaking defense in large language models via unlearning harmful knowledge. [find](https://scholar.google.com/scholar?q=Weikai+Lu%2C+Ziqian+Zeng%2C+Jianwei+Wang%2C+Zhengdong+Lu%2C+Zelin+Chen%2C+Huiping+Zhuang%2C+and+Cen+Chen%2E+2024%2E+Eraser%3A+Jailbreaking+defense+in+large+language+models+via+unlearning+harmful+knowledge%2E)
34. Ximing Lu, Sean Welleck, Jack Hessel, Liwei Jiang, Lianhui Qin, and Peter West. 2022. [QUARK: controllable text generation with reinforced unlearning](https://md.research.elamur.ai/free/en/papers/lu-2022-quark-controllable-text-generation-reinforced/). In Proceedings of Advances in Neural Information Processing Systems (NeurIPS).
35. Yifan Lu, Yigeng Zhou, Jing Li, Yequan Wang, Xuebo Liu, Daojing He, Fangming Liu, and Min Zhang. 2025. Knowledge editing with dynamic knowledge graphs for multi-hop question answering. In Proceedings of the AAAI Conference on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Yifan+Lu%2C+Yigeng+Zhou%2C+Jing+Li%2C+Yequan+Wang%2C+Xuebo+Liu%2C+Daojing+He%2C+Fangming+Liu%2C+and+Min+Zhang%2E+2025%2E+Knowledge+editing+with+dynamic+knowledge+graphs+for+multi%2Dhop+question+answering%2E+In+Proceedings+)
36. Todor Markov, Chong Zhang, Sandhini Agarwal, Florentine Eloundou Nekoul, Theodore Lee, and Steven Adler. 2023. [A holistic approach to undesired content detection in the real world](https://md.research.elamur.ai/free/en/papers/markov-2022-holistic-approach-undesired-content-detection/). In Proceedings of Conference on Innovative Applications of Artificial Intelligence (AAAI), pages 15009–15018.
37. Paul Michel, Omer Levy, and Graham Neubig. 2019. [Are sixteen heads really better than one?](https://md.research.elamur.ai/free/en/papers/michel-2019-sixteen-heads-really-better-than/) In Proceedings of Advances in Neural Information Processing Systems (NeurIPS), pages 14014–14024.
38. Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. Can a suit of armor conduct electricity? A new dataset for open book question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2381–2391. [find](https://scholar.google.com/scholar?q=Todor+Mihaylov%2C+Peter+Clark%2C+Tushar+Khot%2C+and+Ashish+Sabharwal%2E+2018%2E+Can+a+suit+of+armor+conduct+electricity%3F+A+new+dataset+for+open+book+question+answering%2E+In+Proceedings+of+the+2018+Conference+on+)
39. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, and Pamela Mishkin. 2022. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). In Proceedings of Advances in Neural Information Processing Systems (NeurIPS).
40. Anselm Paulus, Arman Zharmagambetov, Chuan Guo, Brandon Amos, and Yuandong Tian. 2024. Advprompter: Fast adaptive adversarial prompting for llms. [find](https://scholar.google.com/scholar?q=Anselm+Paulus%2C+Arman+Zharmagambetov%2C+Chuan+Guo%2C+Brandon+Amos%2C+and+Yuandong+Tian%2E+2024%2E+Advprompter%3A+Fast+adaptive+adversarial+prompting+for+llms%2E)
41. Mansi Phute, Alec Helbling, Matthew Hull, Shengyun Peng, Sebastian Szyller, Cory Cornelius, and Duen Horng Chau. 2024. LLM self defense: By self examination, llms know they are being tricked. In Proceedings of International Conference on Learning Representations (ICLR). [find](https://scholar.google.com/scholar?q=Mansi+Phute%2C+Alec+Helbling%2C+Matthew+Hull%2C+Shengyun+Peng%2C+Sebastian+Szyller%2C+Cory+Cornelius%2C+and+Duen+Horng+Chau%2E+2024%2E+LLM+self+defense%3A+By+self+examination%2C+llms+know+they+are+being+tricked%2E+In+Proce)
42. Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. 2024. [Fine-tuning aligned language models compromises safety, even when users do not intend to!](https://md.research.elamur.ai/free/en/papers/qi-2023-fine-tuning-aligned-language-models/) In Proceedings of The Twelfth International Conference on Learning Representations (ICLR).
43. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. 2023. [Direct preference optimization: Your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/). In Proceedings of Advances in Neural Information Processing Systems (NeurIPS).
44. Feiliang Ren, Longhui Zhang, Shujuan Yin, Xiaofeng Zhao, Shilei Liu, Bochao Li, and Yaduo Liu. 2021. A novel global feature-oriented relational triple extraction model based on table filling. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP). [find](https://scholar.google.com/scholar?q=Feiliang+Ren%2C+Longhui+Zhang%2C+Shujuan+Yin%2C+Xiaofeng+Zhao%2C+Shilei+Liu%2C+Bochao+Li%2C+and+Yaduo+Liu%2E+2021%2E+A+novel+global+feature%2Doriented+relational+triple+extraction+model+based+on+table+filling%2E+In+Proce)
45. Feiliang Ren, Longhui Zhang, Xiaofeng Zhao, Shujuan Yin, Shilei Liu, and Bochao Li. 2022. A simple but effective bidirectional framework for relational triple extraction. In Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining. [find](https://scholar.google.com/scholar?q=Feiliang+Ren%2C+Longhui+Zhang%2C+Xiaofeng+Zhao%2C+Shujuan+Yin%2C+Shilei+Liu%2C+and+Bochao+Li%2E+2022%2E+A+simple+but+effective+bidirectional+framework+for+relational+triple+extraction%2E+In+Proceedings+of+the+Fifteen)
46. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. Winogrande: an adversarial winograd schema challenge at scale. Commun. ACM, 64(9):99–106. [find](https://scholar.google.com/scholar?q=Keisuke+Sakaguchi%2C+Ronan+Le+Bras%2C+Chandra+Bhagavatula%2C+and+Yejin+Choi%2E+2021%2E+Winogrande%3A+an+adversarial+winograd+schema+challenge+at+scale%2E+Commun%2E+ACM%2C+64%289%29%3A99%E2%80%93106%2E)
47. Erfan Shayegani, Yue Dong, and Nael B. Abu-Ghazaleh. 2024. Jailbreak in pieces: Compositional adversarial attacks on multi-modal language models. In Proceedings of The Twelfth International Conference on Learning Representations (ICLR). [find](https://scholar.google.com/scholar?q=Erfan+Shayegani%2C+Yue+Dong%2C+and+Nael+B%2E+Abu%2DGhazaleh%2E+2024%2E+Jailbreak+in+pieces%3A+Compositional+adversarial+attacks+on+multi%2Dmodal+language+models%2E+In+Proceedings+of+The+Twelfth+International+Conference)
48. Zesheng Shi and Yucheng Zhou. 2023. Topic-selective graph network for topic-focused summarization. In Advances in Knowledge Discovery and Data Mining - 27th Pacific-Asia Conference on Knowledge Discovery and Data Mining (PAKDD). [find](https://scholar.google.com/scholar?q=Zesheng+Shi+and+Yucheng+Zhou%2E+2023%2E+Topic%2Dselective+graph+network+for+topic%2Dfocused+summarization%2E+In+Advances+in+Knowledge+Discovery+and+Data+Mining+%2D+27th+Pacific%2DAsia+Conference+on+Knowledge+Discov)
49. Megan Su and Yuwei Bao. 2024. User modeling challenges in interactive AI assistant systems. In Proceedings of Conference on Learning Representations (ICLR). [find](https://scholar.google.com/scholar?q=Megan+Su+and+Yuwei+Bao%2E+2024%2E+User+modeling+challenges+in+interactive+AI+assistant+systems%2E+In+Proceedings+of+Conference+on+Learning+Representations+%28ICLR%29%2E)
50. Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. [Commonsenseqa: A question answering challenge targeting commonsense knowledge](https://md.research.elamur.ai/free/en/papers/talmor-2019-commonsenseqa-question-answering-challenge-targeting/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (…
51. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, and Amjad Almahairi. 2023. [Llama 2: Open foundation and fine-tuned chat models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/).
52. Fanqi Wan, Xinting Huang, Tao Yang, Xiaojun Quan, Wei Bi, and Shuming Shi. 2023. Explore-instruct: Enhancing domain-specific instruction coverage through active exploration. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP). [find](https://scholar.google.com/scholar?q=Fanqi+Wan%2C+Xinting+Huang%2C+Tao+Yang%2C+Xiaojun+Quan%2C+Wei+Bi%2C+and+Shuming+Shi%2E+2023%2E+Explore%2Dinstruct%3A+Enhancing+domain%2Dspecific+instruction+coverage+through+active+exploration%2E+In+Proceedings+of+the+2023)
53. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019. [GLUE: A multi-task benchmark and analysis platform for natural language understanding](https://md.research.elamur.ai/free/en/papers/wang-2018-glue-multi-task-benchmark-analysis/). In Proceedings of 7th International Conference on Learning Representations (ICLR).
54. Feng Wang, Zesheng Shi, Bo Wang, Nan Wang, and Han Xiao. 2025. Readerlm-v2: Small language model for HTML to markdown and JSON. CoRR. [find](https://scholar.google.com/scholar?q=Feng+Wang%2C+Zesheng+Shi%2C+Bo+Wang%2C+Nan+Wang%2C+and+Han+Xiao%2E+2025%2E+Readerlm%2Dv2%3A+Small+language+model+for+HTML+to+markdown+and+JSON%2E+CoRR%2E)
55. Yixu Wang, Yan Teng, Kexin Huang, Chengqi Lyu, Songyang Zhang, Wenwei Zhang, Xingjun Ma, Yu-Gang Jiang, Yu Qiao, and Yingchun Wang. 2024a. Fake alignment: Are llms really aligned well? [find](https://scholar.google.com/scholar?q=Yixu+Wang%2C+Yan+Teng%2C+Kexin+Huang%2C+Chengqi+Lyu%2C+Songyang+Zhang%2C+Wenwei+Zhang%2C+Xingjun+Ma%2C+Yu%2DGang+Jiang%2C+Yu+Qiao%2C+and+Yingchun+Wang%2E+2024a%2E+Fake+alignment%3A+Are+llms+really+aligned+well%3F)
56. Yu Wang, Ruihan Wu, Zexue He, Xiusi Chen, and Julian McAuley. 2024b. Large scale knowledge washing. [find](https://scholar.google.com/scholar?q=Yu+Wang%2C+Ruihan+Wu%2C+Zexue+He%2C+Xiusi+Chen%2C+and+Julian+McAuley%2E+2024b%2E+Large+scale+knowledge+washing%2E)
57. Boyi Wei, Kaixuan Huang, Yangsibo Huang, Tinghao Xie, Xiangyu Qi, and Mengzhou Xia. 2024. Assessing the brittleness of safety alignment via pruning and low-rank modifications. [find](https://scholar.google.com/scholar?q=Boyi+Wei%2C+Kaixuan+Huang%2C+Yangsibo+Huang%2C+Tinghao+Xie%2C+Xiangyu+Qi%2C+and+Mengzhou+Xia%2E+2024%2E+Assessing+the+brittleness+of+safety+alignment+via+pruning+and+low%2Drank+modifications%2E)
58. Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, and Nan Du. 2022. [Finetuned language models are zero-shot learners](https://md.research.elamur.ai/free/en/papers/wei-2021-finetuned-language-models-zero-shot/). In Proceedings of International Conference on Learning Representations (ICLR)).
59. Yuanshun Yao, Xiaojun Xu, and Yang Liu. 2024. [Large language model unlearning](https://md.research.elamur.ai/free/en/papers/yao-2023-large-language-model-unlearning/).
60. Sibo Yi, Yule Liu, Zhen Sun, Tianshuo Cong, Xinlei He, Jiaxing Song, Ke Xu, and Qi Li. 2024. Jailbreak attacks and defenses against [large language models: A survey](https://md.research.elamur.ai/free/en/papers/minaee-2024-large-language-models-survey/).
61. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? In Proceedings of the 57th Conference of the Association for Computational Linguistics (ACL), pages 4791–4800. [find](https://scholar.google.com/scholar?q=Rowan+Zellers%2C+Ari+Holtzman%2C+Yonatan+Bisk%2C+Ali+Farhadi%2C+and+Yejin+Choi%2E+2019%2E+Hellaswag%3A+Can+a+machine+really+finish+your+sentence%3F+In+Proceedings+of+the+57th+Conference+of+the+Association+for+Computa)
62. Yi Zeng, Hongpeng Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia, and Weiyan Shi. 2024. [How johnny can persuade llms to jailbreak them: Rethinking persuasion to challenge ai safety by humanizing llms](https://md.research.elamur.ai/free/en/papers/zeng-2024-how-johnny-can-persuade-llms/).
63. Longhui Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, Meishan Zhang, and Min Zhang. 2024a. A two-stage adaptation of large language models for text ranking. In Findings of the Association for Computational Linguistics: ACL 2024. [find](https://scholar.google.com/scholar?q=Longhui+Zhang%2C+Yanzhao+Zhang%2C+Dingkun+Long%2C+Pengjun+Xie%2C+Meishan+Zhang%2C+and+Min+Zhang%2E+2024a%2E+A+two%2Dstage+adaptation+of+large+language+models+for+text+ranking%2E+In+Findings+of+the+Association+for+Compu)
64. Zhexin Zhang, Junxiao Yang, Pei Ke, Shiyao Cui, Chujie Zheng, Hongning Wang, and Minlie Huang. 2024b. Safe unlearning: A surprisingly effective and generalizable solution to defend against jailbreak attacks. [find](https://scholar.google.com/scholar?q=Zhexin+Zhang%2C+Junxiao+Yang%2C+Pei+Ke%2C+Shiyao+Cui%2C+Chujie+Zheng%2C+Hongning+Wang%2C+and+Minlie+Huang%2E+2024b%2E+Safe+unlearning%3A+A+surprisingly+effective+and+generalizable+solution+to+defend+against+jailbreak+a)
65. Jinman Zhao and Xueyan Zhang. 2024. Large language model is not a (multilingual) compositional relation reasoner. In First Conference on Language Modeling. [find](https://scholar.google.com/scholar?q=Jinman+Zhao+and+Xueyan+Zhang%2E+2024%2E+Large+language+model+is+not+a+%28multilingual%29+compositional+relation+reasoner%2E+In+First+Conference+on+Language+Modeling%2E)
66. Lei Zhao, Junlin Li, Lianli Gao, Yunbo Rao, Jingkuan Song, and Heng Tao Shen. 2022. Heterogeneous knowledge network for visual dialog. IEEE Transactions on Circuits and Systems for Video Technology. [find](https://scholar.google.com/scholar?q=Lei+Zhao%2C+Junlin+Li%2C+Lianli+Gao%2C+Yunbo+Rao%2C+Jingkuan+Song%2C+and+Heng+Tao+Shen%2E+2022%2E+Heterogeneous+knowledge+network+for+visual+dialog%2E+IEEE+Transactions+on+Circuits+and+Systems+for+Video+Technology%2E)
67. Yukun Zhao, Lingyong Yan, Weiwei Sun, Guoliang Xing, Shuaiqiang Wang, and Chong Meng. 2024. Improving the robustness of large language models via consistency alignment. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC/COL… [find](https://scholar.google.com/scholar?q=Yukun+Zhao%2C+Lingyong+Yan%2C+Weiwei+Sun%2C+Guoliang+Xing%2C+Shuaiqiang+Wang%2C+and+Chong+Meng%2E+2024%2E+Improving+the+robustness+of+large+language+models+via+consistency+alignment%2E+In+Proceedings+of+the+2024+Join)
68. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. [Judging llm-as-a-judge with mt-bench and chatbot arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). In Advances in Neural Information Processing Systems 36: A…
69. Xin Zhou, Yi Lu, Ruotian Ma, Tao Gui, Qi Zhang, and Xuanjing Huang. 2023. Making harmful behaviors unlearnable for large language models. [find](https://scholar.google.com/scholar?q=Xin+Zhou%2C+Yi+Lu%2C+Ruotian+Ma%2C+Tao+Gui%2C+Qi+Zhang%2C+and+Xuanjing+Huang%2E+2023%2E+Making+harmful+behaviors+unlearnable+for+large+language+models%2E)
70. Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, and Richard Ren. 2023a. [Representation engineering: A top-down approach to ai transparency](https://md.research.elamur.ai/free/en/papers/zou-2023-representation-engineering-top-down-approach/).
71. Andy Zou, Long Phan, Justin Wang, Derek Duenas, Maxwell Lin, Maksym Andriushchenko, Rowan Wang, Zico Kolter, Matt Fredrikson, and Dan Hendrycks. 2024. Improving alignment and robustness with circuit breakers. [find](https://scholar.google.com/scholar?q=Andy+Zou%2C+Long+Phan%2C+Justin+Wang%2C+Derek+Duenas%2C+Maxwell+Lin%2C+Maksym+Andriushchenko%2C+Rowan+Wang%2C+Zico+Kolter%2C+Matt+Fredrikson%2C+and+Dan+Hendrycks%2E+2024%2E+Improving+alignment+and+robustness+with+circuit+b)

*Updated 31 October 2026*