---
title: "HiddenDetect — paper by Jiang et al. (2025)"
description: "The authors build a Refusal Token Set from model replies to harmful image-text inputs (e.g., \"sorry\"… 57 references, cited by 4 papers here."
authors: ["Yilei Jiang", "Xinyan Gao", "Tianshuo Peng", "Yingshui Tan", "Xiaoyong Zhu", "Bo Zheng", "Xiangyu Yue"]
year: 2025
venue: "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2025.acl-long.724/"
doi: "10.18653/v1/2025.acl-long.724"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/jiang-2025-hiddendetect-detecting-jailbreak-attacks-against/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/jiang-2025-hiddendetect-detecting-jailbreak-attacks-against/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/6/) / HiddenDetect: Detecting Jailbreak Attacks against Multimodal Large Language Models via Monitoring Hidden States

# HiddenDetect: Detecting Jailbreak Attacks against Multimodal Large Language Models via Monitoring Hidden States

**Authors:** Yilei Jiang, Xinyan Gao, Tianshuo Peng, [Yingshui Tan](https://md.research.elamur.ai/free/en/authors/yingshui-tan/), Xiaoyong Zhu, [Bo Zheng](https://md.research.elamur.ai/free/en/authors/bo-zheng/) and [Xiangyu Yue](https://md.research.elamur.ai/free/en/authors/xiangyu-yue/) · **Year:** 2025 · **Venue:** [Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2025](https://md.research.elamur.ai/free/en/venues/acl/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.acl-long.724/) · [PDF](https://aclanthology.org/2025.acl-long.724.pdf) · [DOI 10.18653/v1/2025.acl-long.724](https://doi.org/10.18653/v1/2025.acl-long.724) · [OpenAlex](https://openalex.org/W4412945189)

**Citations:** 11 citations (OpenAlex, 2026-10-14) · [4](https://md.research.elamur.ai/free/en/papers/jiang-2025-hiddendetect-detecting-jailbreak-attacks-against/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Vision-language models (LVLMs) are easier to jailbreak than text-only LLMs. Common defenses rely on safety fine-tuning or protective prompts, which are costly or tend to over-refuse. This paper asks whether a model's own hidden states already show that a prompt is harmful before any output is generated.

The authors build a Refusal Token Set from model replies to harmful image-text inputs (e.g., "sorry", "cannot"). They expand it by projecting each layer's last-token hidden state into vocabulary space and collecting refusal-like top tokens. The result is a sparse binary Refusal Vector. Cosine similarity between this vector and each layer's projected state measures refusal strength. A few-shot comparison of safe and unsafe prompts identifies the layers that separate them best, and the scores from these layers are aggregated into a detection score.

The resulting detector, HiddenDetect, needs no training and beats perplexity-, self-detection- and gradient-based baselines on several LVLMs. It also reveals that image inputs delay the internal safety signal.

## Key points
- **Training-free detection:** HiddenDetect uses no fine-tuning, external classifier or defensive prompt. It only reads intermediate activations via the unembedding layer.
- **Benchmarks and models:** evaluated on LLaVA, CogVLM and Qwen-VL with XSTest, FigTxt, FigImg and MM-SafetyBench. Benign samples come from MM-Vet, and AUROC is the metric.
- **Main results:** average AUROC is 0.922 / 0.903 / 0.826 on LLaVA / CogVLM / Qwen-VL. Gradient Cuff reaches 0.791 / 0.769 / 0.716, while Perplexity and Self-detection score roughly 0.638–0.734.
- **Safety-aware layers:** in LLaVA-v1.6-Vicuna-7B, layers 16–29 are the most safety-aware. Restricting the score to these layers beats both using all layers and excluding them. Up-weighting these layers raises refusals on borderline benign queries.
- **Delayed refusal under images:** refusal semantics appear later in the network for multimodal jailbreaks, especially those using SD-generated images, and this lag correlates with attack success. The method only flags unsafe prompts and does not correct responses. Code: github.com/leigest519/HiddenDetect.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (2): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works

🧭 Topics (3): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 657 works · [🛡️ Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/) · 517 works

## Cite

Cite as: Yilei Jiang, Xinyan Gao, Tianshuo Peng, Yingshui Tan, Xiaoyong Zhu, Bo Zheng and Xiangyu Yue. 2025. HiddenDetect: Detecting Jailbreak Attacks against Multimodal Large Language Models via Monitoring Hidden States. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{jiang2025hiddendetect,
  title = {{HiddenDetect: Detecting Jailbreak Attacks against Multimodal Large Language Models via Monitoring Hidden States}},
  author = {Yilei Jiang, Xinyan Gao, Tianshuo Peng, Yingshui Tan, Xiaoyong Zhu, Bo Zheng and Xiangyu Yue},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.724},
  url = {https://doi.org/10.18653/v1/2025.acl-long.724}
}
```

- [← previous FR-Spec: Accelerating Large-Vocabulary Language Models via Frequency-Ranked Speculative Sampling](https://md.research.elamur.ai/free/en/papers/zhao-2025-fr-spec-accelerating-large-vocabulary/)
- [next → Improve LLM-as-a-Judge Ability as a General Ability](https://md.research.elamur.ai/free/en/papers/yu-2025-improve-llm-as-judge-ability/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/6/)

## References

57 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774. large vision-language models to align and interact with humans via natur…
2. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. [find](https://scholar.google.com/scholar?q=Wei%2DLin+Chiang%2C+Zhuohan+Li%2C+Zi+Lin%2C+Ying+Sheng%2C+Zhanghao+Wu%2C+Hao+Zhang%2C+Lianmin+Zheng%2C+Siyuan+Zhuang%2C+Yonghao+Zhuang%2C+Joseph+E%2E+Gonzalez%2C+Ion+Stoica%2C+and+Eric+P%2E+Xing%2E+2023%2E+Vicuna%3A+An+open%2Dsource+cha)
3. Yinpeng Dong, Huanran Chen, Jiawei Chen, Zhengwei Fang, Xiao Yang, Yichi Zhang, Yu Tian, Hang Su, and Jun Zhu. 2023. How robust is google’s bard to adversarial image attacks? arXiv preprint arXiv:2309.11751. — [arXiv:2309.11751](https://arxiv.org/abs/2309.11751) [find](https://scholar.google.com/scholar?q=Yinpeng+Dong%2C+Huanran+Chen%2C+Jiawei+Chen%2C+Zhengwei+Fang%2C+Xiao+Yang%2C+Yichi+Zhang%2C+Yu+Tian%2C+Hang+Su%2C+and+Jun+Zhu%2E+2023%2E+How+robust+is+google%E2%80%99s+bard+to+adversarial+image+attacks%3F+arXiv+preprint+arXiv%3A2309)
4. Xuefeng Du, Reshmi Ghosh, Robert Sim, Ahmed Salem, Vitor Carvalho, Emily Lawton, Yixuan Li, and Jack W. Stokes. 2024. Vlmguard: Defending vlms against malicious prompts via unlabeled data. Preprint, arXiv:2410.00296. — [arXiv:2410.00296](https://arxiv.org/abs/2410.00296) [find](https://scholar.google.com/scholar?q=Xuefeng+Du%2C+Reshmi+Ghosh%2C+Robert+Sim%2C+Ahmed+Salem%2C+Vitor+Carvalho%2C+Emily+Lawton%2C+Yixuan+Li%2C+and+Jack+W%2E+Stokes%2E+2024%2E+Vlmguard%3A+Defending+vlms+against+malicious+prompts+via+unlabeled+data%2E+Preprint%2C+a)
5. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783.
6. Samar Fares, Klea Ziu, Toluwani Aremu, Nikita Durasov, Martin Takáč, Pascal Fua, Karthik Nandakumar, and Ivan Laptev. 2024. Mirrorcheck: Efficient adversarial defense for vision-language models. arXiv preprint arXiv:2406.09250. — [arXiv:2406.09250](https://arxiv.org/abs/2406.09250) [find](https://scholar.google.com/scholar?q=Samar+Fares%2C+Klea+Ziu%2C+Toluwani+Aremu%2C+Nikita+Durasov%2C+Martin+Tak%C3%A1c%CC%8C%2C+Pascal+Fua%2C+Karthik+Nandakumar%2C+and+Ivan+Laptev%2E+2024%2E+Mirrorcheck%3A+Efficient+adversarial+defense+for+vision%2Dlanguage+models%2E+arXi)
7. Xiaohan Fu, Zihan Wang, Shuheng Li, Rajesh K Gupta, Niloofar Mireshghallah, Taylor Berg-Kirkpatrick, and Earlence Fernandes. 2023. Misusing tools in large language models with visual adversarial examples. arXiv preprint arXiv:2310.03185. — [arXiv:2310.03185](https://arxiv.org/abs/2310.03185) [find](https://scholar.google.com/scholar?q=Xiaohan+Fu%2C+Zihan+Wang%2C+Shuheng+Li%2C+Rajesh+K+Gupta%2C+Niloofar+Mireshghallah%2C+Taylor+Berg%2DKirkpatrick%2C+and+Earlence+Fernandes%2E+2023%2E+Misusing+tools+in+large+language+models+with+visual+adversarial+examp)
8. Gabriel Alon and Michael Kamfonas. 2023. [Detecting language model attacks with perplexity](https://md.research.elamur.ai/free/en/papers/alon-2023-detecting-language-model-attacks-perplexity/). Preprint, arXiv:2308.14132.
9. Yichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang, Tianshuo Cong, Anyu Wang, Sisi Duan, and Xiaoyun Wang. 2023. Figstep: Jailbreaking large vision-language models via typographic visual prompts. arXiv preprint arXiv:2311.05608. [find](https://scholar.google.com/scholar?q=Yichen+Gong%2C+Delong+Ran%2C+Jinyuan+Liu%2C+Conglei+Wang%2C+Tianshuo+Cong%2C+Anyu+Wang%2C+Sisi+Duan%2C+and+Xiaoyun+Wang%2E+2023%2E+Figstep%3A+Jailbreaking+large+vision%2Dlanguage+models+via+typographic+visual+prompts%2E+arXi)
10. Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. [Qwen-vl: A frontier large vision-language model with versatile abilities](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-vl-versatile-vision-language/). arXiv preprint arXiv:2308.12966.
11. Yunhao Gou, Kai Chen, Zhili Liu, Lanqing Hong, Hang Xu, Zhenguo Li, Dit-Yan Yeung, James T Kwok, and Yu Zhang. 2024. Eyes closed, safety on: Protecting multimodal llms via image-to-text transformation. arXiv preprint arXiv:2403.09572. — [arXiv:2403.09572](https://arxiv.org/abs/2403.09572) [find](https://scholar.google.com/scholar?q=Yunhao+Gou%2C+Kai+Chen%2C+Zhili+Liu%2C+Lanqing+Hong%2C+Hang+Xu%2C+Zhenguo+Li%2C+Dit%2DYan+Yeung%2C+James+T+Kwok%2C+and+Yu+Zhang%2E+2024%2E+Eyes+closed%2C+safety+on%3A+Protecting+multimodal+llms+via+image%2Dto%2Dtext+transformation)
12. Luke Bailey, Euan Ong, Stuart Russell, and Scott Emmons. 2023. Image hijacks: Adversarial images can control generative models at runtime. arXiv preprint arXiv:2309.00236. — [arXiv:2309.00236](https://arxiv.org/abs/2309.00236) [find](https://scholar.google.com/scholar?q=Luke+Bailey%2C+Euan+Ong%2C+Stuart+Russell%2C+and+Scott+Emmons%2E+2023%2E+Image+hijacks%3A+Adversarial+images+can+control+generative+models+at+runtime%2E+arXiv+preprint+arXiv%3A2309%2E00236%2E+%E2%80%94+%5BarXiv%3A2309%2E00236%5D%28https%3A%2F)
13. Jihao Gu, Yingyao Wang, Pi Bu, Chen Wang, Ziming Wang, Tengtao Song, Donglai Wei, Jiale Yuan, Yingxiu Zhao, Yancheng He, Shilong Li, Jiaheng Liu, Meng Cao, Jun Song, Yingshui Tan, Xiang Li, Wenbo Su, Zhicheng Zheng, Xiaoyong Zhu, and Bo Zheng. 2025. Chinesesimplevqa – "see the world, discover knowle… [find](https://scholar.google.com/scholar?q=Jihao+Gu%2C+Yingyao+Wang%2C+Pi+Bu%2C+Chen+Wang%2C+Ziming+Wang%2C+Tengtao+Song%2C+Donglai+Wei%2C+Jiale+Yuan%2C+Yingxiu+Zhao%2C+Yancheng+He%2C+Shilong+Li%2C+Jiaheng+Liu%2C+Meng+Cao%2C+Jun+Song%2C+Yingshui+Tan%2C+Xiang+Li%2C+Wenbo+Su%2C+)
14. Chaochao Chen, Yizhao Zhang, Yuyuan Li, Jun Wang, Lianyong Qi, Xiaolong Xu, Xiaolin Zheng, and Jianwei Yin. 2024. Post-training attribute unlearning in recommender systems. ACM Transactions on Information Systems, 43(1):1–28. [find](https://scholar.google.com/scholar?q=Chaochao+Chen%2C+Yizhao+Zhang%2C+Yuyuan+Li%2C+Jun+Wang%2C+Lianyong+Qi%2C+Xiaolong+Xu%2C+Xiaolin+Zheng%2C+and+Jianwei+Yin%2E+2024%2E+Post%2Dtraining+attribute+unlearning+in+recommender+systems%2E+ACM+Transactions+on+Informa)
15. Yangyi Chen, Karan Sikka, Michael Cogswell, Heng Ji, and Ajay Divakaran. 2023. [find](https://scholar.google.com/scholar?q=Yangyi+Chen%2C+Karan+Sikka%2C+Michael+Cogswell%2C+Heng+Ji%2C+and+Ajay+Divakaran%2E+2023%2E)
16. Dress: Instructing Yancheng He, Shilong Li, Jiaheng Liu, Yingshui Tan, Weixun Wang, Hui Huang, Xingyuan Bu, Hangyu Guo, Chengwei Hu, Boren Zheng, Zhuoran Lin, Xuepeng Liu, Dekai Sun, Shirong Lin, Zhicheng Zheng, Xiaoyong Zhu, Wenbo Su, and Bo Zheng. 2024. [Chinese simpleqa: A chinese factuality evalu…](https://md.research.elamur.ai/free/en/papers/he-2024-chinese-simpleqa-chinese-factuality/)
17. Xiaomeng Hu, Pin-Yu Chen, and Tsung-Yi Ho. 2024. Gradient cuff: Detecting jailbreak attacks on large language models by exploring refusal loss landscapes. arXiv preprint arXiv:2403.00867. — [arXiv:2403.00867](https://arxiv.org/abs/2403.00867) [find](https://scholar.google.com/scholar?q=Xiaomeng+Hu%2C+Pin%2DYu+Chen%2C+and+Tsung%2DYi+Ho%2E+2024%2E+Gradient+cuff%3A+Detecting+jailbreak+attacks+on+large+language+models+by+exploring+refusal+loss+landscapes%2E+arXiv+preprint+arXiv%3A2403%2E00867%2E+%E2%80%94+%5BarXiv%3A240)
18. Jen-Tse Huang, Dasen Dai, Jen-Yuan Huang, Youliang Yuan, Xiaoyuan Liu, Wenxuan Wang, Wenxiang Jiao, Pinjia He, and Zhaopeng Tu. 2025. Visfactor: Benchmarking fundamental visual cognition in multimodal large language models. Preprint, arXiv:2502.16435. — [arXiv:2502.16435](https://arxiv.org/abs/2502.16435) [find](https://scholar.google.com/scholar?q=Jen%2DTse+Huang%2C+Dasen+Dai%2C+Jen%2DYuan+Huang%2C+Youliang+Yuan%2C+Xiaoyuan+Liu%2C+Wenxuan+Wang%2C+Wenxiang+Jiao%2C+Pinjia+He%2C+and+Zhaopeng+Tu%2E+2025%2E+Visfactor%3A+Benchmarking+fundamental+visual+cognition+in+multimodal)
19. Yilei Jiang, Yingshui Tan, and Xiangyu Yue. 2024. Rapguard: Safeguarding multimodal large language models via rationale-aware defensive prompting. Preprint, arXiv:2412.18826. — [arXiv:2412.18826](https://arxiv.org/abs/2412.18826) [find](https://scholar.google.com/scholar?q=Yilei+Jiang%2C+Yingshui+Tan%2C+and+Xiangyu+Yue%2E+2024%2E+Rapguard%3A+Safeguarding+multimodal+large+language+models+via+rationale%2Daware+defensive+prompting%2E+Preprint%2C+arXiv%3A2412%2E18826%2E+%E2%80%94+%5BarXiv%3A2412%2E18826%5D%28http)
20. Shen Li, Liuyi Yao, Lan Zhang, and Yaliang Li. 2024a. Safety layers in aligned large language models: The key to llm security. Preprint, arXiv:2408.17003. — [arXiv:2408.17003](https://arxiv.org/abs/2408.17003) [find](https://scholar.google.com/scholar?q=Shen+Li%2C+Liuyi+Yao%2C+Lan+Zhang%2C+and+Yaliang+Li%2E+2024a%2E+Safety+layers+in+aligned+large+language+models%3A+The+key+to+llm+security%2E+Preprint%2C+arXiv%3A2408%2E17003%2E+%E2%80%94+%5BarXiv%3A2408%2E17003%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F24)
21. Siyuan Li, Juanxi Tian, Zedong Wang, Luyuan Zhang, Zicheng Liu, Weiyang Jin, Yang Liu, Baigui Sun, and Stan Z. Li. 2024b. Unveiling the backbone-optimizer coupling bias in visual representation learning. Preprint, arXiv:2410.06373. — [arXiv:2410.06373](https://arxiv.org/abs/2410.06373) [find](https://scholar.google.com/scholar?q=Siyuan+Li%2C+Juanxi+Tian%2C+Zedong+Wang%2C+Luyuan+Zhang%2C+Zicheng+Liu%2C+Weiyang+Jin%2C+Yang+Liu%2C+Baigui+Sun%2C+and+Stan+Z%2E+Li%2E+2024b%2E+Unveiling+the+backbone%2Doptimizer+coupling+bias+in+visual+representation+learni)
22. Yuyuan Li, Chaochao Chen, Yizhao Zhang, Weiming Liu, Lingjuan Lyu, Xiaolin Zheng, Dan Meng, and Jun Wang. 2023. Ultrare: Enhancing receraser for recommendation unlearning via error decomposition. Advances in Neural Information Processing Systems, 36:12611–12625. [find](https://scholar.google.com/scholar?q=Yuyuan+Li%2C+Chaochao+Chen%2C+Yizhao+Zhang%2C+Weiming+Liu%2C+Lingjuan+Lyu%2C+Xiaolin+Zheng%2C+Dan+Meng%2C+and+Jun+Wang%2E+2023%2E+Ultrare%3A+Enhancing+receraser+for+recommendation+unlearning+via+error+decomposition%2E+Adva)
23. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023a. [Visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-visual-instruction-tuning/). arxiv preprint arxiv:2304.08485.
24. Wei Liu, Zhongyu Niu, Lang Gao, Zhiying Deng, Jun Wang, Haozhao Wang, and Ruixuan Li. 2025. Adversarial cooperative rationalization: The risk of spurious correlations in even clean datasets. Preprint, arXiv:2505.02118. — [arXiv:2505.02118](https://arxiv.org/abs/2505.02118) [find](https://scholar.google.com/scholar?q=Wei+Liu%2C+Zhongyu+Niu%2C+Lang+Gao%2C+Zhiying+Deng%2C+Jun+Wang%2C+Haozhao+Wang%2C+and+Ruixuan+Li%2E+2025%2E+Adversarial+cooperative+rationalization%3A+The+risk+of+spurious+correlations+in+even+clean+datasets%2E+Preprint%2C)
25. Xin Liu, Yichen Zhu, Yunshi Lan, Chao Yang, and Yu Qiao. 2023b. Query-relevant images jailbreak large multi-modal models. arXiv preprint arXiv:2311.17600. [find](https://scholar.google.com/scholar?q=Xin+Liu%2C+Yichen+Zhu%2C+Yunshi+Lan%2C+Chao+Yang%2C+and+Yu+Qiao%2E+2023b%2E+Query%2Drelevant+images+jailbreak+large+multi%2Dmodal+models%2E+arXiv+preprint+arXiv%3A2311%2E17600%2E)
26. Xin Liu, Yichen Zhu, Yunshi Lan, Chao Yang, and Yu Qiao. 2023c. Query-relevant images jailbreak large multi-modal models. arXiv preprint arXiv:2311.17600. [find](https://scholar.google.com/scholar?q=Xin+Liu%2C+Yichen+Zhu%2C+Yunshi+Lan%2C+Chao+Yang%2C+and+Yu+Qiao%2E+2023c%2E+Query%2Drelevant+images+jailbreak+large+multi%2Dmodal+models%2E+arXiv+preprint+arXiv%3A2311%2E17600%2E)
27. Xin Liu, Yichen Zhu, Yunshi Lan, Chao Yang, and Yu Qiao. 2024. Safety of multimodal large language models on images and text. arXiv preprint arXiv:2402.00357. — [arXiv:2402.00357](https://arxiv.org/abs/2402.00357) [find](https://scholar.google.com/scholar?q=Xin+Liu%2C+Yichen+Zhu%2C+Yunshi+Lan%2C+Chao+Yang%2C+and+Yu+Qiao%2E+2024%2E+Safety+of+multimodal+large+language+models+on+images+and+text%2E+arXiv+preprint+arXiv%3A2402%2E00357%2E+%E2%80%94+%5BarXiv%3A2402%2E00357%5D%28https%3A%2F%2Farxiv%2Eorg%2Fab)
28. Haochen Luo, Jindong Gu, Fengyuan Liu, and Philip Torr. 2024a. An image is worth 1000 lies: Transferability of adversarial images across prompts on vision-language models. In ICLR. [find](https://scholar.google.com/scholar?q=Haochen+Luo%2C+Jindong+Gu%2C+Fengyuan+Liu%2C+and+Philip+Torr%2E+2024a%2E+An+image+is+worth+1000+lies%3A+Transferability+of+adversarial+images+across+prompts+on+vision%2Dlanguage+models%2E+In+ICLR%2E)
29. Weidi Luo, Siyuan Ma, Xiaogeng Liu, Xiaoyu Guo, and Chaowei Xiao. 2024b. Jailbreakv-28k: A benchmark for assessing the robustness of multimodal large language models against jailbreak attacks. arXiv preprint arXiv:2404.03027. [find](https://scholar.google.com/scholar?q=Weidi+Luo%2C+Siyuan+Ma%2C+Xiaogeng+Liu%2C+Xiaoyu+Guo%2C+and+Chaowei+Xiao%2E+2024b%2E+Jailbreakv%2D28k%3A+A+benchmark+for+assessing+the+robustness+of+multimodal+large+language+models+against+jailbreak+attacks%2E+arXiv+p)
30. Neel Nanda, Andrew Lee, and Martin Wattenberg. 2023. Emergent linear representations in world models of self-supervised sequence models. arXiv preprint arXiv:2309.00941. [find](https://scholar.google.com/scholar?q=Neel+Nanda%2C+Andrew+Lee%2C+and+Martin+Wattenberg%2E+2023%2E+Emergent+linear+representations+in+world+models+of+self%2Dsupervised+sequence+models%2E+arXiv+preprint+arXiv%3A2309%2E00941%2E)
31. OpenAI. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv:2303.08774.
32. Preprint, Kiho Park, Yo Joong Choe, and Victor Veitch. 2023. [The linear representation hypothesis and the geometry of large language models](https://md.research.elamur.ai/free/en/papers/park-2023-linear-representation-hypothesis-geometry-large/). arXiv preprint arXiv:2311.03658.
33. Renjie Pi, Tianyang Han, Yueqi Xie, Rui Pan, Qing Lian, Hanze Dong, Jipeng Zhang, and Tong Zhang. 2024. [MLLM-Protector: Ensuring MLLM’s safety without hurting performance](https://md.research.elamur.ai/free/en/papers/pi-2024-mllm-protector-ensuring-mllm-s/). arXiv preprint arXiv:2401.02906.
34. Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Mengdi Wang, and Prateek Mittal. 2023. Visual adversarial examples jailbreak large language models. arXiv preprint arXiv:2306.13213. [find](https://scholar.google.com/scholar?q=Xiangyu+Qi%2C+Kaixuan+Huang%2C+Ashwinee+Panda%2C+Mengdi+Wang%2C+and+Prateek+Mittal%2E+2023%2E+Visual+adversarial+examples+jailbreak+large+language+models%2E+arXiv+preprint+arXiv%3A2306%2E13213%2E)
35. Paul Röttger, Hannah Rose Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy. 2024. [Xstest: A test suite for identifying exaggerated safety behaviours in large language models](https://md.research.elamur.ai/free/en/papers/rottger-2023-xstest-test-suite-identifying-exaggerated/). Preprint, arXiv:2308.01263.
36. Christian Schlarmann and Matthias Hein. 2023. On the adversarial robustness of multi-modal foundation models. In ICCV. [find](https://scholar.google.com/scholar?q=Christian+Schlarmann+and+Matthias+Hein%2E+2023%2E+On+the+adversarial+robustness+of+multi%2Dmodal+foundation+models%2E+In+ICCV%2E)
37. Erfan Shayegani, Yue Dong, and Nael Abu-Ghazaleh. 2023. Plug and pray: Exploiting off-the-shelf components of multi-modal models. arXiv preprint arXiv:2307.14539. [find](https://scholar.google.com/scholar?q=Erfan+Shayegani%2C+Yue+Dong%2C+and+Nael+Abu%2DGhazaleh%2E+2023%2E+Plug+and+pray%3A+Exploiting+off%2Dthe%2Dshelf+components+of+multi%2Dmodal+models%2E+arXiv+preprint+arXiv%3A2307%2E14539%2E)
38. Yingshui Tan, Boren Zheng, Baihui Zheng, Kerui Cao, Huiyun Jing, Jincheng Wei, Jiaheng Liu, Yancheng He, Wenbo Su, Xiangyong Zhu, Bo Zheng, and Kaifu Zhang. 2024. [Chinese safetyqa: A safety short-form factuality benchmark for large language models](https://md.research.elamur.ai/free/en/papers/tan-2024-chinese-safetyqa-safety-short/). Preprint, arXiv:2412.15265.
39. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023a. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). arXiv preprint arXiv:2302.13971.
40. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023b. [Llama 2: Open foundation and fine-tuned chat models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/). arXiv preprint arXiv:2307.09288.
41. Haoqin Tu, Chenhang Cui, Zijun Wang, Yiyang Zhou, Bingchen Zhao, Junlin Han, Wangchunshu Zhou, Huaxiu Yao, and Cihang Xie. 2023. How many unicorns are in this image? a safety evaluation benchmark for vision llms. arXiv preprint arXiv:2311.16101. — [arXiv:2311.16101](https://arxiv.org/abs/2311.16101) [find](https://scholar.google.com/scholar?q=Haoqin+Tu%2C+Chenhang+Cui%2C+Zijun+Wang%2C+Yiyang+Zhou%2C+Bingchen+Zhao%2C+Junlin+Han%2C+Wangchunshu+Zhou%2C+Huaxiu+Yao%2C+and+Cihang+Xie%2E+2023%2E+How+many+unicorns+are+in+this+image%3F+a+safety+evaluation+benchmark+for+)
42. Shangqing Tu, Zhuoran Pan, Wenxuan Wang, Zhexin Zhang, Yuliang Sun, Jifan Yu, Hongning Wang, Lei Hou, and Juanzi Li. 2024. Knowledge-to-jailbreak: One knowledge point worth one attack. Preprint, arXiv:2406.11682. — [arXiv:2406.11682](https://arxiv.org/abs/2406.11682) [find](https://scholar.google.com/scholar?q=Shangqing+Tu%2C+Zhuoran+Pan%2C+Wenxuan+Wang%2C+Zhexin+Zhang%2C+Yuliang+Sun%2C+Jifan+Yu%2C+Hongning+Wang%2C+Lei+Hou%2C+and+Juanzi+Li%2E+2024%2E+Knowledge%2Dto%2Djailbreak%3A+One+knowledge+point+worth+one+attack%2E+Preprint%2C+arXiv)
43. Yuxuan Wan, Wenxuan Wang, Yiliu Yang, Youliang Yuan, Jen-tse Huang, Pinjia He, Wenxiang Jiao, and Michael Lyu. 2024. LogicAsker: Evaluating and improving the logical reasoning ability of large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,… [find](https://scholar.google.com/scholar?q=Yuxuan+Wan%2C+Wenxuan+Wang%2C+Yiliu+Yang%2C+Youliang+Yuan%2C+Jen%2Dtse+Huang%2C+Pinjia+He%2C+Wenxiang+Jiao%2C+and+Michael+Lyu%2E+2024%2E+LogicAsker%3A+Evaluating+and+improving+the+logical+reasoning+ability+of+large+languag)
44. Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong, Ji Qi, Yan Wang, Junhui Ji, Zhuoyi Yang, Lei Zhao, Xixuan Song, Jiazheng Xu, Bin Xu, Juanzi Li, Yuxiao Dong, Ming Ding, and Jie Tang. 2023. [CogVLM: Visual expert for pretrained language models](https://md.research.elamur.ai/free/en/papers/wang-2023-cogvlm-visual-expert-pretrained/). arXiv preprint arXiv:2311.03079.
45. Wenxuan Wang, Kuiyi Gao, Zihan Jia, Youliang Yuan, Jen tse Huang, Qiuzhi Liu, Shuai Wang, Wenxiang Jiao, and Zhaopeng Tu. 2024a. Chain-of-jailbreak attack for image generation models via editing step by step. Preprint, arXiv:2410.03869. — [arXiv:2410.03869](https://arxiv.org/abs/2410.03869) [find](https://scholar.google.com/scholar?q=Wenxuan+Wang%2C+Kuiyi+Gao%2C+Zihan+Jia%2C+Youliang+Yuan%2C+Jen+tse+Huang%2C+Qiuzhi+Liu%2C+Shuai+Wang%2C+Wenxiang+Jiao%2C+and+Zhaopeng+Tu%2E+2024a%2E+Chain%2Dof%2Djailbreak+attack+for+image+generation+models+via+editing+step+)
46. Yue Xu, Xiuyuan Qi, Zhan Qin, and Wenjie Wang. 2024. Defending jailbreak attack in vlms via cross-modality information detector. arXiv preprint arXiv:2407.21659. — [arXiv:2407.21659](https://arxiv.org/abs/2407.21659) [find](https://scholar.google.com/scholar?q=Yue+Xu%2C+Xiuyuan+Qi%2C+Zhan+Qin%2C+and+Wenjie+Wang%2E+2024%2E+Defending+jailbreak+attack+in+vlms+via+cross%2Dmodality+information+detector%2E+arXiv+preprint+arXiv%3A2407%2E21659%2E+%E2%80%94+%5BarXiv%3A2407%2E21659%5D%28https%3A%2F%2Farxiv%2Eorg)
47. Qinghao Ye, Haiyang Xu, Jiabo Ye, Ming Yan, Haowei Liu, Qi Qian, Ji Zhang, Fei Huang, and Jingren Zhou. 2023. [mplug-owl2: Revolutionizing multi-modal large language model with modality collaboration](https://md.research.elamur.ai/free/en/papers/ye-2023-mplug-owl2-revolutionizing-multi-modal/). arXiv preprint arXiv:2311.04257.
48. Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen tse Huang, Pinjia He, Shuming Shi, and Zhaopeng Tu. 2024. [Gpt-4 is too smart to be safe: Stealthy chat with llms via cipher](https://md.research.elamur.ai/free/en/papers/yuan-2023-gpt-4-too-smart-safe/). Preprint, arXiv:2308.06463.
49. Xiaoyu Zhang, Cen Zhang, Tianlin Li, Yihao Huang, Xiaojun Jia, Xiaofei Xie, Yang Liu, and Chao Shen. 2023. A mutation-based method for multimodal jailbreaking attack detection. arXiv preprint arXiv:2312.10766. — [arXiv:2312.10766](https://arxiv.org/abs/2312.10766) [find](https://scholar.google.com/scholar?q=Xiaoyu+Zhang%2C+Cen+Zhang%2C+Tianlin+Li%2C+Yihao+Huang%2C+Xiaojun+Jia%2C+Xiaofei+Xie%2C+Yang+Liu%2C+and+Chao+Shen%2E+2023%2E+A+mutation%2Dbased+method+for+multimodal+jailbreaking+attack+detection%2E+arXiv+preprint+arXiv%3A23)
50. Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongxuan Li, Ngai-Man Man Cheung, and Min Lin. 2024. On evaluating adversarial robustness of large vision-language models. Advances in Neural Information Processing Systems, 36. [find](https://scholar.google.com/scholar?q=Yunqing+Zhao%2C+Tianyu+Pang%2C+Chao+Du%2C+Xiao+Yang%2C+Chongxuan+Li%2C+Ngai%2DMan+Man+Cheung%2C+and+Min+Lin%2E+2024%2E+On+evaluating+adversarial+robustness+of+large+vision%2Dlanguage+models%2E+Advances+in+Neural+Informatio)
51. Changshi Zhou, Haichuan Xu, Jiarui Hu, Feng Luan, Zhipeng Wang, Yanchao Dong, Yanmin Zhou, and Bin He. 2024. Ssfold: Learning to fold arbitrary crumpled cloth using graph dynamics from human demonstration. Preprint, arXiv:2411.02608. — [arXiv:2411.02608](https://arxiv.org/abs/2411.02608) [find](https://scholar.google.com/scholar?q=Changshi+Zhou%2C+Haichuan+Xu%2C+Jiarui+Hu%2C+Feng+Luan%2C+Zhipeng+Wang%2C+Yanchao+Dong%2C+Yanmin+Zhou%2C+and+Bin+He%2E+2024%2E+Ssfold%3A+Learning+to+fold+arbitrary+crumpled+cloth+using+graph+dynamics+from+human+demonstra)
52. Yongshuo Zong, Ondrej Bohdal, Tingyang Yu, Yongxin Yang, and Timothy Hospedales. 2024. Safety fine-tuning at (almost) no cost: A baseline for vision large language models. [find](https://scholar.google.com/scholar?q=Yongshuo+Zong%2C+Ondrej+Bohdal%2C+Tingyang+Yu%2C+Yongxin+Yang%2C+and+Timothy+Hospedales%2E+2024%2E+Safety+fine%2Dtuning+at+%28almost%29+no+cost%3A+A+baseline+for+vision+large+language+models%2E)
53. Wenxuan Wang, Xiaoyuan Liu, Kuiyi Gao, Jen tse Huang, Youliang Yuan, Pinjia He, Shuai Wang, and Zhaopeng Tu. 2025. Can’t see the forest for the trees: Benchmarking multimodal safety awareness for multimodal llms. Preprint, arXiv:2502.11184. [find](https://scholar.google.com/scholar?q=Wenxuan+Wang%2C+Xiaoyuan+Liu%2C+Kuiyi+Gao%2C+Jen+tse+Huang%2C+Youliang+Yuan%2C+Pinjia+He%2C+Shuai+Wang%2C+and+Zhaopeng+Tu%2E+2025%2E+Can%E2%80%99t+see+the+forest+for+the+trees%3A+Benchmarking+multimodal+safety+awareness+for+mult)
54. Yu Wang, Xiaogeng Liu, Yu Li, Muhao Chen, and Chaowei Xiao. 2024b. Adashield: Safeguarding multimodal large language models from structure-based attack via adaptive shield prompting. ECCV. [find](https://scholar.google.com/scholar?q=Yu+Wang%2C+Xiaogeng+Liu%2C+Yu+Li%2C+Muhao+Chen%2C+and+Chaowei+Xiao%2E+2024b%2E+Adashield%3A+Safeguarding+multimodal+large+language+models+from+structure%2Dbased+attack+via+adaptive+shield+prompting%2E+ECCV%2E)
55. Zihao Wang, Lin Gui, Jeffrey Negrea, and Victor Veitch. 2024c. Concept algebra for (score-based) text-controlled generative models. Advances in Neural Information Processing Systems, 36. [find](https://scholar.google.com/scholar?q=Zihao+Wang%2C+Lin+Gui%2C+Jeffrey+Negrea%2C+and+Victor+Veitch%2E+2024c%2E+Concept+algebra+for+%28score%2Dbased%29+text%2Dcontrolled+generative+models%2E+Advances+in+Neural+Information+Processing+Systems%2C+36%2E)
56. Yuanwei Wu, Xiang Li, Yixin Liu, Pan Zhou, and Lichao Sun. 2023. Jailbreaking gpt-4v via self-adversarial attacks with system prompts. arXiv preprint arXiv:2311.09127. — [arXiv:2311.09127](https://arxiv.org/abs/2311.09127) [find](https://scholar.google.com/scholar?q=Yuanwei+Wu%2C+Xiang+Li%2C+Yixin+Liu%2C+Pan+Zhou%2C+and+Lichao+Sun%2E+2023%2E+Jailbreaking+gpt%2D4v+via+self%2Dadversarial+attacks+with+system+prompts%2E+arXiv+preprint+arXiv%3A2311%2E09127%2E+%E2%80%94+%5BarXiv%3A2311%2E09127%5D%28https%3A%2F%2Farx)
57. Yueqi Xie, Minghong Fang, Renjie Pi, and Neil Gong. 2024. Gradsafe: Detecting unsafe prompts for llms via safety-critical gradient analysis. arXiv preprint arXiv:2402.13494. — [arXiv:2402.13494](https://arxiv.org/abs/2402.13494) [find](https://scholar.google.com/scholar?q=Yueqi+Xie%2C+Minghong+Fang%2C+Renjie+Pi%2C+and+Neil+Gong%2E+2024%2E+Gradsafe%3A+Detecting+unsafe+prompts+for+llms+via+safety%2Dcritical+gradient+analysis%2E+arXiv+preprint+arXiv%3A2402%2E13494%2E+%E2%80%94+%5BarXiv%3A2402%2E13494%5D%28https)

*Updated 31 October 2026*