--- title: "Defending LLMs against Jailbreak Attacks via Template-Based" description: "The authors propose a lightweight plug-in defense (code: DSICL) with two parts. 2 references." authors: ["Ruiyang Ni", "Changlong Li", "Shuaibiao Han", "Zhiyu Yi", "Perley Xu", "Wenjie Ruan"] year: 2026 venue: "Findings of the Association for Computational Linguistics: ACL 2026" source: "https://aclanthology.org/2026.findings-acl.2123/" doi: "10.18653/v1/2026.findings-acl.2123" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/ni-2026-defending-llms-against-jailbreak-attacks/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ni-2026-defending-llms-against-jailbreak-attacks/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/27/) / Defending LLMs against Jailbreak Attacks via Template-Based ICL with a Defensive Suffix # Defending LLMs against Jailbreak Attacks via Template-Based ICL with a Defensive Suffix **Authors:** Ruiyang Ni, Changlong Li, Shuaibiao Han, Zhiyu Yi, Perley Xu and Wenjie Ruan · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.2123/) · [PDF](https://aclanthology.org/2026.findings-acl.2123.pdf) · [DOI 10.18653/v1/2026.findings-acl.2123](https://doi.org/10.18653/v1/2026.findings-acl.2123) · [OpenAlex](https://openalex.org/W7166834889) **Citations:** 0 citations (OpenAlex, 2026-10-14) ## Summary Jailbreak prompts can push aligned LLMs into producing harmful content. Existing defenses have drawbacks: fine-tuning is costly and needs access to model weights, and input or output filters often generalize poorly. The authors propose a lightweight plug-in defense (code: DSICL) with two parts. - **Offline:** an iterative algorithm optimizes a defensive suffix. The objective increases the likelihood of refusals on attack prompts. A weighted utility term, controlled by α, keeps answers on benign prompt–response pairs intact. - **At inference:** a stochastic random search picks in-context demonstrations. A fixed template then combines the demonstrations, the user request and the suffix. The design follows two observations. First, GCG-style adversarial suffixes break when a few extra tokens are appended. Second, template-based ICL helps against attacks that rewrite queries, such as PAIR and TAP. Experiments cover Vicuna, Llama-2 and GPT-3.5/GPT-4, against white-box and black-box attacks. The method cuts attack success to near zero on several models and generally outperforms the PAT baseline. Costs are small: a minor drop in benign utility and slightly more over-refusal. This makes it practical for API-level deployment. ## Key points - Motivating finding: on Llama-2-7b with AdvBench, appending a few random tokens to I-GCG suffixes lowers ASR from 96% to roughly zero. - Transfer attacks on JailbreakBench (DSN, GCG, JBC, PAIR, RS), average ASR: - Vicuna-7b: 66.2% → 1.8% (PAT: 19%) - Llama-2-7b: 25.2% → 0.2% - GPT-4: 25.4% → 2.75% - Vicuna-13b remains hardest: 79% → 38% - Adaptive white-box attacks on Vicuna-7b: I-GCG ASR falls from 100% to 42%, AutoDAN from 100% to 46%, and PAIR from 86% to 8%. - Utility and over-refusal: - MMLU barely changes (Vicuna-7b: 49.8 → 49.4). - MT-bench drops slightly (6.41 → 6.13). - On XSTest, full refusals for Llama-2-7b rise from 39.2% to 41.6%. - Ablation on Vicuna-7b (AdvBench subset), I-GCG ASR: - suffix only: 40% - ICL only: 60% - both combined: 2% *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [University of Science and Technology of China](https://md.research.elamur.ai/free/en/universities/university-of-science-and-technology-of-china/) — University, Hefei, China · 108 works · 112,698 citations - [University of Dundee](https://md.research.elamur.ai/free/en/universities/university-of-dundee/) — University, Dundee, United Kingdom · 4 works · 644 citations 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works 🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🛡️ Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/) · 517 works ## Cite Cite as: Ruiyang Ni, Changlong Li, Shuaibiao Han, Zhiyu Yi, Perley Xu and Wenjie Ruan. 2026. Defending LLMs against Jailbreak Attacks via Template-Based ICL with a Defensive Suffix. Findings of the Association for Computational Linguistics: ACL 2026. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{ni2026defending, title = {{Defending LLMs against Jailbreak Attacks via Template-Based ICL with a Defensive Suffix}}, author = {Ruiyang Ni, Changlong Li, Shuaibiao Han, Zhiyu Yi, Perley Xu and Wenjie Ruan}, booktitle = {Findings of the Association for Computational Linguistics: ACL 2026}, year = {2026}, doi = {10.18653/v1/2026.findings-acl.2123}, url = {https://doi.org/10.18653/v1/2026.findings-acl.2123} } ``` - [← previous DeepPavlov Strikes Back: A Toolkit for Improving LLM Reliability and Trustworthiness](https://md.research.elamur.ai/free/en/papers/nikolaev-2026-deeppavlov-strikes-back-toolkit-improving/) - [next → Defense Against Knowledge Poisoning Attack on GraphRAG](https://md.research.elamur.ai/free/en/papers/noughabi-2026-defense-against-knowledge-poisoning-attack/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/27/) ## References 2 references, 2 of them with a card of ours. ↗ leads to the work. 1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, and Shyamal Anadkat. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). OpenAI. 2. Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang. 2016. [Ms marco: A human generated machine reading comprehension dataset](https://md.research.elamur.ai/free/en/papers/bajaj-2016-ms-marco-human-generated/). NeurIPS. *Updated 1 November 2026*