---
title: "JailBreakV — paper by Luo et al. (2024) — summary and ideas"
description: "This paper tests whether jailbreak methods designed for text-only LLMs also work against MLLMs. 57 references, cited by 59 papers here."
authors: ["Weidi Luo", "Siyuan Ma", "Xiaogeng Liu", "Xiaoyu Guo", "Chaowei Xiao"]
year: 2024
venue: "arXiv:2404.03027"
source: "https://arxiv.org/abs/2404.03027"
doi: "10.48550/arxiv.2404.03027"
arxiv: "2404.03027"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/luo-2024-jailbreakv-benchmark-assessing-robustness-multimodal/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/luo-2024-jailbreakv-benchmark-assessing-robustness-multimodal/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/7/) / JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks

# JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks

**Authors:** [Weidi Luo](https://md.research.elamur.ai/free/en/authors/weidi-luo/), [Siyuan Ma](https://md.research.elamur.ai/free/en/authors/siyuan-ma/), [Xiaogeng Liu](https://md.research.elamur.ai/free/en/authors/xiaogeng-liu/), [Xiaoyu Guo](https://md.research.elamur.ai/free/en/authors/xiaoyu-guo/) and [Chaowei Xiao](https://md.research.elamur.ai/free/en/authors/chaowei-xiao/) · **Year:** 2024 · **Venue:** arXiv:2404.03027 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2404.03027) · [PDF](https://arxiv.org/pdf/2404.03027) · [DOI 10.48550/arxiv.2404.03027](https://doi.org/10.48550/arxiv.2404.03027) · [arXiv 2404.03027](https://arxiv.org/abs/2404.03027) · [OpenAlex](https://openalex.org/W4394007075)

**Citations:** 7 citations (OpenAlex, 2026-10-14) · [59](https://md.research.elamur.ai/free/en/papers/luo-2024-jailbreakv-benchmark-assessing-robustness-multimodal/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Multimodal LLMs are built on a text LLM backbone, yet prior safety work mostly targeted image-based jailbreaks. This paper tests whether jailbreak methods designed for text-only LLMs also work against MLLMs. The authors first assemble RedTeam-2K, a balanced set of 2,000 harmful queries spanning 16 safety policies. From these queries they produce 5,000 text jailbreak prompts using Template (GCG plus handcrafted templates), Persuade and Logic attacks. Each prompt is combined with four kinds of images, giving 20,000 cases. They add 8,000 more cases built with the image-based FigStep and Query-relevant attacks. The full benchmark, JailBreakV-28K, has 28,000 cases. Testing 10 open-source MLLMs, with Llama-Guard judging harmfulness, shows that the transferred text attacks succeed more often than the image-based ones. The type of image paired with the text barely changes the outcome. The study concludes that MLLMs inherit the weaknesses of their language backbones, so safety alignment must address text inputs as well as visual ones.

## Key points
- **RedTeam-2K:** 2,000 harmful queries covering 16 safety policies, drawn from 8 sources. These include AdvBench, BeaverTails, hh-rlhf and GPT-generated or handwritten items. About 48.5% of the queries were collected by the authors.
- **JailBreakV-28K composition:** 20,000 LLM-transfer cases, pairing text prompts with Nature, Random Noise, Blank or Stable Diffusion images, plus 8,000 cases from FigStep and Query-relevant attacks. The benchmark is more than five times larger than MM-SafetyBench, which has 5,040 pairs.
- **Text vs. image attacks:** Under Llama-Guard, the transferred text attacks reach an average ASR of 50.5% across MLLMs. The stronger of the two image-based attacks reaches at most 30%.
- **Image type matters little:** The average coefficient of variation in ASR across the four image types is only 9.0.
- **Other findings:** Most models are most vulnerable in the "Economic Health" and "Malware" categories. The dataset is available on Hugging Face, but access requires prior permission.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Peking University](https://md.research.elamur.ai/free/en/universities/peking-university/) — University, Beijing, China · 315 works · 43,267 citations
- [The Ohio State University](https://md.research.elamur.ai/free/en/universities/the-ohio-state-university/) — University, Columbus, United States · 90 works · 19,112 citations
- [University of Wisconsin–Madison](https://md.research.elamur.ai/free/en/universities/university-of-wisconsin-madison/) — University, Madison, United States · 54 works · 49,089 citations

📚 Shelves (3): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

🧭 Topics (4): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,341 works · [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 811 works · [🛡️ Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/) · 587 works

## Cite

Cite as: Weidi Luo, Siyuan Ma, Xiaogeng Liu, Xiaoyu Guo and Chaowei Xiao. 2024. JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks. arXiv:2404.03027.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{luo2024jailbreakv,
  title = {{JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attacks}},
  author = {Weidi Luo, Siyuan Ma, Xiaogeng Liu, Xiaoyu Guo and Chaowei Xiao},
  year = {2024},
  doi = {10.48550/arxiv.2404.03027},
  eprint = {2404.03027},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2404.03027}
}
```

- [← previous Imitate, Explore, and Self-Improve: A Reproduction Report on Slow-thinking Reasoning Systems](https://md.research.elamur.ai/free/en/papers/min-2024-imitate-explore-self-improve-reproduction/)
- [next → Knowledge of Knowledge: Exploring Known-Unknowns Uncertainty with Large Language Models](https://md.research.elamur.ai/free/en/papers/amayuelas-2024-knowledge-knowledge-exploring-known-unknowns/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/7/)

## References

57 references, 30 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Usage policies - openai. [https://openai.com/policies/usage-policies](https://openai.com/policies/usage-policies), 2024. URL https://openai.com/policies/usage-policies. Accessed: 2024-01-12. [↗](https://openai.com/policies/usage-policies)
2. Stability AI. Stable Diffusion XL Base 1.0: A Diffusion-based Text-to-Image Generative Model. [https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0), 2023. Accessed: 2024-03-24. [↗](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0)
3. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, P… — [Qwen Technical Report](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-technical-report/)
4. Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.[12966, 2023b](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-vl-versatile-vision-language/).
5. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Sh… [find](https://scholar.google.com/scholar?q=Yuntao+Bai%2C+Andy+Jones%2C+Kamal+Ndousse%2C+Amanda+Askell%2C+Anna+Chen%2C+Nova+DasSarma%2C+Dawn+Drain%2C+Stanislav+Fort%2C+Deep+Ganguli%2C+Tom+Henighan%2C+Nicholas+Joseph%2C+Saurav+Kadavath%2C+Jackson+Kernion%2C+Tom+Conerly%2C+)
6. Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, and Eric Wong. [Jailbreaking black box large language models in twenty queries](https://md.research.elamur.ai/free/en/papers/chao-2023-jailbreaking-black-box-large-language/), 2023.
7. Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. [Instructblip: Towards general-purpose vision-language models with instruction tuning](https://md.research.elamur.ai/free/en/papers/dai-2023-instructblip-towards-general-purpose-vision/), 2023.
8. Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255, 2009. doi: 10.1109/CVPR.2009.5206848. [find](https://scholar.google.com/scholar?q=Jia+Deng%2C+Wei+Dong%2C+Richard+Socher%2C+Li%2DJia+Li%2C+Kai+Li%2C+and+Li+Fei%2DFei%2E+Imagenet%3A+A+large%2Dscale+hierarchical+image+database%2E+In+2009+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition%2C+pp%2E+248%E2%80%93)
9. Yue Deng, Wenxuan Zhang, Sinno Jialin Pan, and Lidong Bing. [Multilingual jailbreak challenges in large language models](https://md.research.elamur.ai/free/en/papers/deng-2023-multilingual-jailbreak-challenges-large-language/), 2023.
10. Yue Deng, Wenxuan Zhang, Sinno Jialin Pan, and Lidong Bing. [Multilingual jailbreak challenges in large language models](https://md.research.elamur.ai/free/en/papers/deng-2023-multilingual-jailbreak-challenges-large-language/). In The Twelfth International Conference on Learning Representations, 2024. URL [https://openreview.net/forum?id=vESNKdEMGp](https://openreview.net/forum?id=vESNKdEMGp). [↗](https://openreview.net/forum?id=vESNKdEMGp)
11. Xiaoyi Dong, Pan Zhang, Yuhang Zang, Yuhang Cao, Bin Wang, Linke Ouyang, Xilin Wei, Songyang Zhang, Haodong Duan, Maosong Cao, Wenwei Zhang, Yining Li, Hang Yan, Yang Gao, Xinyue Zhang, Wei Li, Jingwen Li, Kai Chen, Conghui He, Xingcheng Zhang, Yu Qiao, Dahua Lin, and Jiaqi Wang. Internlm-xcomposer2… — [InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model](https://md.research.elamur.ai/free/en/papers/dong-2024-internlm-xcomposer2-mastering-free/)
12. Yinpeng Dong, Huanran Chen, Jiawei Chen, Zhengwei Fang, Xiao Yang, Yichi Zhang, Yu Tian, Hang Su, and Jun Zhu. How Robust is Google’s Bard to Adversarial Image Attacks? arXiv preprint arXiv:2309.11751, 2023. — [arXiv:2309.11751](https://arxiv.org/abs/2309.11751) [find](https://scholar.google.com/scholar?q=Yinpeng+Dong%2C+Huanran+Chen%2C+Jiawei+Chen%2C+Zhengwei+Fang%2C+Xiao+Yang%2C+Yichi+Zhang%2C+Yu+Tian%2C+Hang+Su%2C+and+Jun+Zhu%2E+How+Robust+is+Google%E2%80%99s+Bard+to+Adversarial+Image+Attacks%3F+arXiv+preprint+arXiv%3A2309%2E11751)
13. Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. [Glm: General language model pretraining with autoregressive blank infilling](https://md.research.elamur.ai/free/en/papers/du-2022-glm-general-language-model-pretraining/), 2022.
14. Peng Gao, Jiaming Han, Renrui Zhang, Ziyi Lin, Shijie Geng, Aojun Zhou, Wei Zhang, Pan Lu, Conghui He, Xiangyu Yue, Hongsheng Li, and Yu Qiao. [Llama-adapter v2: Parameter-efficient visual instruction model](https://md.research.elamur.ai/free/en/papers/gao-2023-llama-adapter-v2-parameter-efficient/), 2023.
15. Yichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang, Tianshuo Cong, Anyu Wang, Sisi Duan, and Xiaoyun Wang. [Figstep: Jailbreaking large vision-language models via typographic visual prompts](https://md.research.elamur.ai/free/en/papers/gong-2023-figstep-jailbreaking-large-vision-language/), 2023.
16. Dongchen Han, Xiaojun Jia, Yang Bai, Jindong Gu, Yang Liu, and Xiaochun Cao. Ot-attack: Enhancing adversarial transferability of vision-language models via optimal transport optimization, 2023. [find](https://scholar.google.com/scholar?q=Dongchen+Han%2C+Xiaojun+Jia%2C+Yang+Bai%2C+Jindong+Gu%2C+Yang+Liu%2C+and+Xiaochun+Cao%2E+Ot%2Dattack%3A+Enhancing+adversarial+transferability+of+vision%2Dlanguage+models+via+optimal+transport+optimization%2C+2023%2E)
17. Adib Hasan, Ileana Rugina, and Alex Wang. Pruning for protection: Increasing jailbreak resistance in aligned llms without fine-tuning, 2024. [find](https://scholar.google.com/scholar?q=Adib+Hasan%2C+Ileana+Rugina%2C+and+Alex+Wang%2E+Pruning+for+protection%3A+Increasing+jailbreak+resistance+in+aligned+llms+without+fine%2Dtuning%2C+2024%2E)
18. Muyang He, Yexin Liu, Boya Wu, Jianhao Yuan, Yueze Wang, Tiejun Huang, and Bo Zhao. Efficient multimodal learning from data-centric perspective, 2024. [find](https://scholar.google.com/scholar?q=Muyang+He%2C+Yexin+Liu%2C+Boya+Wu%2C+Jianhao+Yuan%2C+Yueze+Wang%2C+Tiejun+Huang%2C+and+Bo+Zhao%2E+Efficient+multimodal+learning+from+data%2Dcentric+perspective%2C+2024%2E)
19. Jinyi Hu, Yuan Yao, Chongyi Wang, Shan Wang, Yinxu Pan, Qianyu Chen, Tianyu Yu, Hanghao Wu, Yue Zhao, Haoye Zhang, Xu Han, Yankai Lin, Jiao Xue, Dahai Li, Zhiyuan Liu, and Maosong Sun. [Large multilingual models pivot zero-shot multimodal learning across languages](https://md.research.elamur.ai/free/en/papers/hu-2023-large-multilingual-models-pivot-zero/), 2024.
20. Yangsibo Huang, Samyak Gupta, Mengzhou Xia, Kai Li, and Danqi Chen. [Catastrophic jailbreak of open-source llms via exploiting generation](https://md.research.elamur.ai/free/en/papers/huang-2023-catastrophic-jailbreak-open-source-llms/), 2023.
21. Yangsibo Huang, Samyak Gupta, Mengzhou Xia, Kai Li, and Danqi Chen. [Catastrophic jailbreak of open-source LLMs via exploiting generation](https://md.research.elamur.ai/free/en/papers/huang-2023-catastrophic-jailbreak-open-source-llms/). In The Twelfth International Conference on Learning Representations, 2024. URL [https://openreview.net/forum?id=r42tSSCHPh](https://openreview.net/forum?id=r42tSSCHPh). [↗](https://openreview.net/forum?id=r42tSSCHPh)
22. Hugging Face. all-mpnet-base-v2. [https://huggingface.co/sentence-transformers/all-mpnet-base-v2](https://huggingface.co/sentence-transformers/all-mpnet-base-v2), 2024. Accessed: 2024-01-19. [↗](https://huggingface.co/sentence-transformers/all-mpnet-base-v2)
23. Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, and Madian Khabsa. [Llama guard: Llm-based input-output safeguard for human-ai conversations](https://md.research.elamur.ai/free/en/papers/inan-2023-llama-guard-llm-based-input/), 2023.
24. Jiaming Ji, Mickel Liu, Juntao Dai, Xuehai Pan, Chi Zhang, Ce Bian, Chi Zhang, Ruiyang Sun, Yizhou Wang, and Yaodong Yang. [Beavertails: Towards improved safety alignment of llm via a human-preference dataset](https://md.research.elamur.ai/free/en/papers/ji-2023-beavertails-towards-improved-safety-alignment/), 2023.
25. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroi… [find](https://scholar.google.com/scholar?q=Albert+Q%2E+Jiang%2C+Alexandre+Sablayrolles%2C+Arthur+Mensch%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guillaume+Lample%2C+Lucile+Saulnier%2C+Le%CC%81lio+Renard+La)
26. Mahnaz Koupaee and William Yang Wang. Wikihow: A large scale text summarization dataset, 2018. [find](https://scholar.google.com/scholar?q=Mahnaz+Koupaee+and+William+Yang+Wang%2E+Wikihow%3A+A+large+scale+text+summarization+dataset%2C+2018%2E)
27. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. [Efficient memory management for large language model serving with pagedattention](https://md.research.elamur.ai/free/en/papers/kwon-2023-efficient-memory-management-large-language/). In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023.
28. Yifan Li, Hangyu Guo, Kun Zhou, Wayne Xin Zhao, and Ji-Rong Wen. Images are achilles’ heel of alignment: Exploiting visual vulnerabilities for jailbreaking multimodal large language models, 2024a. [find](https://scholar.google.com/scholar?q=Yifan+Li%2C+Hangyu+Guo%2C+Kun+Zhou%2C+Wayne+Xin+Zhao%2C+and+Ji%2DRong+Wen%2E+Images+are+achilles%E2%80%99+heel+of+alignment%3A+Exploiting+visual+vulnerabilities+for+jailbreaking+multimodal+large+language+models%2C+2024a%2E)
29. Yuhui Li, Fangyun Wei, Jinjing Zhao, Chao Zhang, and Hongyang Zhang. RAIN: Your language models can align themselves without finetuning. In The Twelfth International Conference on Learning Representations, 2024b. URL [https://openreview.net/forum?id=pETSfWMUzy](https://openreview.net/forum?id=pETSfWMUzy). [↗](https://openreview.net/forum?id=pETSfWMUzy)
30. Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. [Improved baselines with visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-improved-baselines-visual-instruction-tuning/), 2023a.
31. Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. [Autodan: Generating stealthy jailbreak prompts on aligned large language models](https://md.research.elamur.ai/free/en/papers/liu-2023-autodan-generating-stealthy-jailbreak-prompts/), 2024a.
32. Xin Liu, Yichen Zhu, Jindong Gu, Yunshi Lan, Chao Yang, and Yu Qiao. [Mm-safetybench: A benchmark for safety evaluation of multimodal large language models](https://md.research.elamur.ai/free/en/papers/liu-2023-mm-safetybench-benchmark-safety-evaluation/), 2024b.
33. Yi Liu, Gelei Deng, Zhengzi Xu, Yuekang Li, Yaowen Zheng, Ying Zhang, Lida Zhao, Tianwei Zhang, and Yang Liu. [Jailbreaking chatgpt via prompt engineering: An empirical study](https://md.research.elamur.ai/free/en/papers/liu-2023-jailbreaking-chatgpt-prompt-engineering-empirical/), 2023b.
34. Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, David Forsyth, and Dan Hendrycks. [Harmbench: A standardized evaluation framework for automated red teaming and robust refusal](https://md.research.elamur.ai/free/en/papers/mazeika-2024-harmbench-standardized-evaluation-framework-automated/), 2024.
35. Meta AI. Llama 2 - acceptable use policy. [https://ai.meta.com/llama/use-policy/](https://ai.meta.com/llama/use-policy/), 2024. Accessed: 2024-01-19. [↗](https://ai.meta.com/llama/use-policy/)
36. Microsoft. Phi-2: A 2.7 billion parameter transformer model. [https://huggingface.co/microsoft/phi-2](https://huggingface.co/microsoft/phi-2), 2023. Accessed: 2024-03-04. [↗](https://huggingface.co/microsoft/phi-2)
37. Zhenxing Niu, Haodong Ren, Xinbo Gao, Gang Hua, and Rong Jin. Jailbreaking attack against multimodal large language model, 2024. [find](https://scholar.google.com/scholar?q=Zhenxing+Niu%2C+Haodong+Ren%2C+Xinbo+Gao%2C+Gang+Hua%2C+and+Rong+Jin%2E+Jailbreaking+attack+against+multimodal+large+language+model%2C+2024%2E)
38. Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Peter Henderson, Mengdi Wang, and Prateek Mittal. [Visual adversarial examples jailbreak aligned large language models](https://md.research.elamur.ai/free/en/papers/qi-2023-visual-adversarial-examples-jailbreak-aligned/). In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 21527–21536, 2024.
39. Christian Schlarmann and Matthias Hein. On the adversarial robustness of multi-modal foundation models, 2023. [find](https://scholar.google.com/scholar?q=Christian+Schlarmann+and+Matthias+Hein%2E+On+the+adversarial+robustness+of+multi%2Dmodal+foundation+models%2C+2023%2E)
40. Erfan Shayegani, Yue Dong, and Nael Abu-Ghazaleh. [Jailbreak in pieces: Compositional adversarial attacks on multi-modal language models](https://md.research.elamur.ai/free/en/papers/shayegani-2023-jailbreak-pieces-compositional-adversarial-attacks/), 2023.
41. Erfan Shayegani, Yue Dong, and Nael Abu-Ghazaleh. [Jailbreak in pieces: Compositional adversarial attacks on multi-modal language models](https://md.research.elamur.ai/free/en/papers/shayegani-2023-jailbreak-pieces-compositional-adversarial-attacks/). In The Twelfth International Conference on Learning Representations, 2024. URL [https://openreview.net/forum?id=plmBsXHxgR](https://openreview.net/forum?id=plmBsXHxgR). [↗](https://openreview.net/forum?id=plmBsXHxgR)
42. Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. ”do anything now”: Characterizing and evaluating in-the-wild jailbreak prompts on large language models, 2023. — ["Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models](https://md.research.elamur.ai/free/en/papers/shen-2023-anything-now-characterizing-evaluating-wild/)
43. InfiMM Team. Infimm: Advancing multimodal understanding from flamingo’s legacy through diverse llm integration, 2024. URL [https://huggingface.co/Infi-MM/](https://huggingface.co/Infi-MM/). [↗](https://huggingface.co/Infi-MM/)
44. InternLM Team. Internlm: A multilingual language model with progressively enhanced capabilities. [https://github.com/InternLM/InternLM](https://github.com/InternLM/InternLM), 2023a. [↗](https://github.com/InternLM/InternLM)
45. The Vicuna Team. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023b. URL [https://lmsys.org/blog/2023-03-30-vicuna/](https://lmsys.org/blog/2023-03-30-vicuna/). Accessed: 2024-03-04. [↗](https://lmsys.org/blog/2023-03-30-vicuna/)
46. Baichuan Intelligent Technology. Baichuan-7b: An open-source large-scale pre-trained model. [https://huggingface.co/baichuan-inc/Baichuan-7B](https://huggingface.co/baichuan-inc/Baichuan-7B), 2023. Accessed: 2024-03-04. [↗](https://huggingface.co/baichuan-inc/Baichuan-7B)
47. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,… — [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/)
48. Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, Nathan Sarrazin, Omar Sanseviero, Alexander M. Rush, and Thomas Wolf. Zephyr: Direct distillation of lm alignment, 2023. [find](https://scholar.google.com/scholar?q=Lewis+Tunstall%2C+Edward+Beeching%2C+Nathan+Lambert%2C+Nazneen+Rajani%2C+Kashif+Rasul%2C+Younes+Belkada%2C+Shengyi+Huang%2C+Leandro+von+Werra%2C+Cle%CC%81mentine+Fourrier%2C+Nathan+Habib%2C+Nathan+Sarrazin%2C+Omar+Sanseviero%2C+A)
49. Yu Wang, Xiaogeng Liu, Yu Li, Muhao Chen, and Chaowei Xiao. Adashield: Safeguarding multimodal large language models from structure-based attack via adaptive shield prompting, 2024. [find](https://scholar.google.com/scholar?q=Yu+Wang%2C+Xiaogeng+Liu%2C+Yu+Li%2C+Muhao+Chen%2C+and+Chaowei+Xiao%2E+Adashield%3A+Safeguarding+multimodal+large+language+models+from+structure%2Dbased+attack+via+adaptive+shield+prompting%2C+2024%2E)
50. Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. [Jailbroken: How does llm safety training fail?](https://md.research.elamur.ai/free/en/papers/wei-2023-jailbroken-llm-safety-training-fail/) Advances in Neural Information Processing Systems, 36, 2024.
51. Jin Xu, Xiaojiang Liu, Jianhao Yan, Deng Cai, Huayang Li, and Jian Li. Learning to break the loop: Analyzing and mitigating repetitions for neural text generation, 2022. [find](https://scholar.google.com/scholar?q=Jin+Xu%2C+Xiaojiang+Liu%2C+Jianhao+Yan%2C+Deng+Cai%2C+Huayang+Li%2C+and+Jian+Li%2E+Learning+to+break+the+loop%3A+Analyzing+and+mitigating+repetitions+for+neural+text+generation%2C+2022%2E)
52. Nan Xu, Fei Wang, Ben Zhou, Bang Zheng Li, Chaowei Xiao, and Muhao Chen. Cognitive overload: Jailbreaking large language models with overloaded logical thinking, 2024. [find](https://scholar.google.com/scholar?q=Nan+Xu%2C+Fei+Wang%2C+Ben+Zhou%2C+Bang+Zheng+Li%2C+Chaowei+Xiao%2C+and+Muhao+Chen%2E+Cognitive+overload%3A+Jailbreaking+large+language+models+with+overloaded+logical+thinking%2C+2024%2E)
53. Zhiyuan Yu, Xiaogeng Liu, Shunning Liang, Zach Cameron, Chaowei Xiao, and Ning Zhang. Don’t listen to me: Understanding and exploring jailbreak prompts of large language models, 2024. [find](https://scholar.google.com/scholar?q=Zhiyuan+Yu%2C+Xiaogeng+Liu%2C+Shunning+Liang%2C+Zach+Cameron%2C+Chaowei+Xiao%2C+and+Ning+Zhang%2E+Don%E2%80%99t+listen+to+me%3A+Understanding+and+exploring+jailbreak+prompts+of+large+language+models%2C+2024%2E)
54. Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen tse Huang, Pinjia He, Shuming Shi, and Zhaopeng Tu. [GPT-4 is too smart to be safe: Stealthy chat with LLMs via cipher](https://md.research.elamur.ai/free/en/papers/yuan-2023-gpt-4-too-smart-safe/). In The Twelfth International Conference on Learning Representations, 2024. URL [https://openreview.net/forum?id=MbfAK4s61A](https://openreview.net/forum?id=MbfAK4s61A). [↗](https://openreview.net/forum?id=MbfAK4s61A)
55. Yi Zeng, Hongpeng Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia, and Weiyan Shi. [How johnny can persuade llms to jailbreak them: Rethinking persuasion to challenge ai safety by humanizing llms](https://md.research.elamur.ai/free/en/papers/zeng-2024-how-johnny-can-persuade-llms/), 2024.
56. Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongxuan Li, Ngai-Man Cheung, and Min Lin. On evaluating adversarial robustness of large vision-language models, 2023. [find](https://scholar.google.com/scholar?q=Yunqing+Zhao%2C+Tianyu+Pang%2C+Chao+Du%2C+Xiao+Yang%2C+Chongxuan+Li%2C+Ngai%2DMan+Cheung%2C+and+Min+Lin%2E+On+evaluating+adversarial+robustness+of+large+vision%2Dlanguage+models%2C+2023%2E)
57. Andy Zou, Zifan Wang, J. Zico Kolter, and Matt Fredrikson. [Universal and transferable adversarial attacks on aligned language models](https://md.research.elamur.ai/free/en/papers/zou-2023-universal-transferable-adversarial-attacks-aligned/), 2023.

*Updated 2 November 2026*