--- title: "面向大语言模型的越狱攻击综述 — 李南 2024" description: "This survey from Shanghai Jiao Tong University covers jailbreak attacks: prompts that get safety-aligned large language models (aligned via SFT and RLHF)…" authors: ["李南", "丁益东", "江浩宇", "牛佳飞", "易平"] year: 2024 venue: "计算机研究与发展 (Journal of Computer Research and Development)" source: "https://crad.ict.ac.cn/article/doi/10.7544/issn1000-1239.202330962" doi: "10.7544/issn1000-1239.202330962" licence: "Free access (crad.ict.ac.cn), no open licence stated — readonly; © 计算机研究与发展" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/papers/li-2024-jailbreak-attack-large/" --- [Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/li-2024-jailbreak-attack-large/) [home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2024](https://md.research.elamur.ai/free/papers/2024/page/4/) / 面向大语言模型的越狱攻击综述 # 面向大语言模型的越狱攻击综述 **Authors:** 李南, 丁益东, 江浩宇, 牛佳飞, 易平 · **Year:** 2024 · **Venue:** 计算机研究与发展 (Journal of Computer Research and Development) · **Licence:** Free access (crad.ict.ac.cn), no open licence stated — readonly; © 计算机研究与发展 **Links:** [Publisher's page](https://doi.org/10.7544/issn1000-1239.202330962) · [PDF](https://crad.ict.ac.cn/cn/article/pdf/preview/10.7544/issn1000-1239.202330962.pdf) · [DOI 10.7544/issn1000-1239.202330962](https://doi.org/10.7544/issn1000-1239.202330962) ## Summary This survey from Shanghai Jiao Tong University covers jailbreak attacks: prompts that get safety-aligned large language models (aligned via SFT and RLHF) to produce content they would normally refuse. The authors argue these attacks are a growing threat for three reasons. They need no access to model weights. Non-experts can write them. Automated methods can now mass-produce variants faster than providers can re-align their models. The paper sorts the literature by how the jailbreak prompt is built: * **Manually designed attacks**, such as early internet prompts like DAN * **LLM-generated attacks**, where a helper model writes or refines prompts * **Adversarial optimization attacks**, which search over perturbed characters or tokens For each group, the authors explain the underlying principles, methods and findings. They then trace how the field has moved toward automation. Defenses are briefly reviewed in two groups, internal and external to the model, with their pros and cons. The survey ends with open problems and possible future directions, namely multimodality, model editing and multi-agent approaches. It serves as a structured entry point to LLM safety research. ## Key points - Bibliographic details: *Journal of Computer Research and Development* 61(5), pp. 1156–1181, 2024; DOI 10.7544/issn1000-1239.202330962; about 137 references. - Comparison of the three attack classes: - Manual attacks: black-box, readable prompts, not automated. - Model-generated attacks: black-box, readable, automated. - Optimization attacks: white- or black-box, usually unreadable, automated. - White-box access is realistic mainly for open models such as LLaMA and Vicuna, not for API-only models like ChatGPT, Claude or PaLM. - Attack mechanisms follow Wei et al. and fall into two groups: - *Competing objectives* (language modeling, instruction following and safety pulling against each other): prefix injection ("Absolutely! Here's"), refusal suppression, style injection and DAN role-play. - *Mismatched generalization*: for example, base64-encoded requests that pretraining-acquired capabilities can decode but safety training does not cover. - The survey links this to model scale: larger, more capable models may expose more exploitable gaps unless defenses grow with them. The example given is GPT-3.5 struggling to understand base64-encoded input. - Defenses are split into two groups: - Internal: the pre-training, alignment and inference stages. - External: for example, detection-based filters. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/) · [Natural language processing](https://md.research.elamur.ai/free/shelves/nlp/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @article{2024, title = {{面向大语言模型的越狱攻击综述}}, author = {李南 and 丁益东 and 江浩宇 and 牛佳飞 and 易平}, journal = {计算机研究与发展 (Journal of Computer Research and Development)}, year = {2024}, doi = {10.7544/issn1000-1239.202330962}, url = {https://doi.org/10.7544/issn1000-1239.202330962} } ``` - [← previous 非自回归神经机器翻译综述](https://md.research.elamur.ai/free/papers/cao-2024-non-autoregressive-neural/) - [next → 面向语言学习者的跨语言反馈评语生成方法(Cross-Lingual Feedback Comment Generation for Language Learners)](https://md.research.elamur.ai/free/papers/an-2024-cross-lingual-feedback-comment-generation/) - [↑ back to papers of 2024](https://md.research.elamur.ai/free/papers/2024/page/4/) ## References 4 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. ［49］ Li Haoran, Guo Dadi, Fan Wei, et al. Multi-step jailbreaking privacy attacks on ChatGPT[J]. arXiv preprint, arXiv: 2304.05197, 2023 [find](https://scholar.google.com/scholar?q=%EF%BC%BB49%EF%BC%BD+Li+Haoran%2C+Guo+Dadi%2C+Fan+Wei%2C+et+al%2E+Multi%2Dstep+jailbreaking+privacy+attacks+on+ChatGPT%5BJ%5D%2E+arXiv+preprint%2C+arXiv%3A+2304%2E05197%2C+2023) [our card](https://md.research.elamur.ai/free/papers/li-2023-multi-step-jailbreaking-privacy-attacks/) 2. ［50］ Huang Yangsibo, Gupta S, Xia Mengzhou, et al. Catastrophic jailbreak of open-source LLMs via exploiting generation[J]. arXiv preprint, arXiv: 2310.06987, 2023 [find](https://scholar.google.com/scholar?q=%EF%BC%BB50%EF%BC%BD+Huang+Yangsibo%2C+Gupta+S%2C+Xia+Mengzhou%2C+et+al%2E+Catastrophic+jailbreak+of+open%2Dsource+LLMs+via+exploiting+generation%5BJ%5D%2E+arXiv+preprint%2C+arXiv%3A+2310%2E06987%2C+2023) [our card](https://md.research.elamur.ai/free/papers/huang-2023-catastrophic-jailbreak-open-source-llms/) 3. ［51］ Yong Z X, Menghini C, Bach S H. Low-resource languages jailbreak GPT-4[J]. arXiv preprint, arXiv: 2310.02446, 2023 [find](https://scholar.google.com/scholar?q=%EF%BC%BB51%EF%BC%BD+Yong+Z+X%2C+Menghini+C%2C+Bach+S+H%2E+Low%2Dresource+languages+jailbreak+GPT%2D4%5BJ%5D%2E+arXiv+preprint%2C+arXiv%3A+2310%2E02446%2C+2023) [our card](https://md.research.elamur.ai/free/papers/yong-2023-low-resource-languages-jailbreak-gpt/) 4. ［52］ Yuan Youliang, Jiao Wenxiang, Wang Wenxuan, et al. GPT-4 is too smart to be safe: Stealthy chat with LLMs via cipher[J]. arXiv preprint, arXiv: 2308.06463, 2023 [find](https://scholar.google.com/scholar?q=%EF%BC%BB52%EF%BC%BD+Yuan+Youliang%2C+Jiao+Wenxiang%2C+Wang+Wenxuan%2C+et+al%2E+GPT%2D4+is+too+smart+to+be+safe%3A+Stealthy+chat+with+LLMs+via+cipher%5BJ%5D%2E+arXiv+preprint%2C+arXiv%3A+2308%2E06463%2C+2023) [our card](https://md.research.elamur.ai/free/papers/yuan-2023-gpt-4-too-smart-safe/)