--- title: "Jailbreak Attacks and Defenses Against Large Language Models:" description: "Large language models such as ChatGPT and Gemini undergo safety alignment so they refuse… 2 references, cited by 90 papers here." authors: ["Sibo Yi", "Yule Liu", "Zhen Sun", "Tianshuo Cong", "Xinlei He", "Jiaxing Song", "Ke Xu", "Qi Li"] year: 2024 venue: "arXiv:2407.04295" source: "https://arxiv.org/abs/2407.04295" doi: "10.48550/arxiv.2407.04295" arxiv: "2407.04295" licence: "CC BY 4.0 (arXiv)" licence_url: "http://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/yi-2024-jailbreak-attacks-defenses-against-large/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/yi-2024-jailbreak-attacks-defenses-against-large/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/4/) / Jailbreak Attacks and Defenses Against Large Language Models: A Survey # Jailbreak Attacks and Defenses Against Large Language Models: A Survey **Authors:** Sibo Yi, Yule Liu, [Zhen Sun](https://md.research.elamur.ai/free/en/authors/zhen-sun/), [Tianshuo Cong](https://md.research.elamur.ai/free/en/authors/tianshuo-cong/), Xinlei He, Jiaxing Song, [Ke Xu](https://md.research.elamur.ai/free/en/authors/ke-xu/) and [Qi Li](https://md.research.elamur.ai/free/en/authors/qi-li/) · **Year:** 2024 · **Venue:** arXiv:2407.04295 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://arxiv.org/abs/2407.04295) · [PDF](https://arxiv.org/pdf/2407.04295) · [DOI 10.48550/arxiv.2407.04295](https://doi.org/10.48550/arxiv.2407.04295) · [arXiv 2407.04295](https://arxiv.org/abs/2407.04295) · [OpenAlex](https://openalex.org/W4400434624) **Citations:** 7 citations (OpenAlex, 2026-10-14) · [90](https://md.research.elamur.ai/free/en/papers/yi-2024-jailbreak-attacks-defenses-against-large/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary Large language models such as ChatGPT and Gemini undergo safety alignment so they refuse harmful requests. Carefully engineered adversarial prompts, known as jailbreaks, can still push them into producing content that violates usage policies. Earlier surveys mostly catalogued attacks and paid little attention to the matching countermeasures. This survey addresses that gap with a single taxonomy that covers both sides. Attacks are grouped by how much the attacker can see of the target model: - **White-box methods** exploit gradients, output logits, or fine-tuning access. - **Black-box methods** rely on templates, rewritten prompts, or a helper LLM that writes attack prompts. Defenses are split by whether they leave the protected model unchanged (prompt-level) or modify or extend it (model-level). The authors chart which defenses also work against attack families they were not built for. They also review evaluation metrics, datasets and frameworks. Their central observation is that attacks keep getting stronger while needing less knowledge of the target, which makes them more practical. This raises the urgency of robust defenses. The paper is a useful map for anyone securing deployed LLMs. ## Key points - **White-box attacks** fall into three families: gradient-based, logits-based and fine-tuning-based. - The gradient-based family includes GCG (Zou et al.), whose optimized adversarial suffix transferred to ChatGPT, Bard and Claude. - Follow-ups such as ARCA, AutoDAN and ASETF aim for more readable suffixes. - **Black-box attacks** fall into three families: - template completion (scenario nesting, context-based, code injection); - prompt rewriting (cipher, low-resource languages, genetic-algorithm-based); - LLM-based generation. - **Prompt-level defenses** include perplexity-based prompt detection, prompt perturbation and system prompt safeguards. - **Model-level defenses** include SFT-based and RLHF-based training, gradient and logit analysis, refinement, and proxy defense (a separate secure LLM that filters outputs). - **Evaluation** uses attack success rate (ASR) and Perplexity as metrics. - Surveyed datasets include AdvBench, HarmBench, JailbreakBench, StrongREJECT, Do-not-Answer, XSTEST, SafetyBench, plus the Chinese-focused SC-Safety and Safety-Prompts. - Surveyed frameworks include EasyJailbreak, JailbreakBench and JailbreakEval. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Tsinghua University](https://md.research.elamur.ai/free/en/universities/tsinghua-university/) — University, Beijing, China · 415 works · 94,028 citations - [The Hong Kong University of Science and Technology (Guangzhou)](https://md.research.elamur.ai/free/en/universities/the-hong-kong-university-of-science-and-technology-guangzhou/) — University, Guangzhou, China · 67 works · 1,765 citations 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works 🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [🛡️ Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/) · 587 works ## Cite Cite as: Sibo Yi, Yule Liu, Zhen Sun, Tianshuo Cong, Xinlei He, Jiaxing Song, Ke Xu and Qi Li. 2024. Jailbreak Attacks and Defenses Against Large Language Models: A Survey. arXiv:2407.04295. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @misc{yi2024jailbreak, title = {{Jailbreak Attacks and Defenses Against Large Language Models: A Survey}}, author = {Sibo Yi, Yule Liu, Zhen Sun, Tianshuo Cong, Xinlei He, Jiaxing Song, Ke Xu and Qi Li}, year = {2024}, doi = {10.48550/arxiv.2407.04295}, eprint = {2407.04295}, archivePrefix = {arXiv}, url = {https://doi.org/10.48550/arxiv.2407.04295} } ``` - [← previous FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI](https://md.research.elamur.ai/free/en/papers/glazer-2024-frontiermath-benchmark-evaluating-advanced-mathematical/) - [next → LoRA+: Efficient Low Rank Adaptation of Large Models](https://md.research.elamur.ai/free/en/papers/hayou-2024-lora-efficient-low-rank-adaptation/) - [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/4/) ## References 2 references, 2 of them with a card of ours. ↗ leads to the work. 1. [1] Gabriel Alon and Michael Kamfonas. [Detecting Language Model Attacks with Perplexity](https://md.research.elamur.ai/free/en/papers/alon-2023-detecting-language-model-attacks-perplexity/). CoRR abs/2308.14132, 2023. 12, 16 2. [2] Maksym Andriushchenko, Francesco Croce, and Nicolas Flammarion. [Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks](https://md.research.elamur.ai/free/en/papers/andriushchenko-2024-jailbreaking-leading-safety-aligned-llms/). CoRR abs/2404.02151, 2024. 4 *Updated 2 November 2026*