---
title: "Defeating Cerberus — paper by Zhang et al. (2026)"
description: "Existing PII benchmarks are text-only, so the authors first build multimodal test sets by turning text PII corpora into images. 42 references."
authors: ["Boyang Zhang", "Istemi Ekin Akkus", "Ruichuan Chen", "Alice Dethise", "Klaus Satzke", "Ivica Rimac", "Yang Zhang"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: EACL 2026"
source: "https://aclanthology.org/2026.findings-eacl.154/"
doi: "10.18653/v1/2026.findings-eacl.154"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2026-defeating-cerberus-privacy-leakage-mitigation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2026-defeating-cerberus-privacy-leakage-mitigation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/29/) / Defeating Cerberus: Privacy-Leakage Mitigation in Vision Language Models

# Defeating Cerberus: Privacy-Leakage Mitigation in Vision Language Models

**Authors:** [Boyang Zhang](https://md.research.elamur.ai/free/en/authors/boyang-zhang/), Istemi Ekin Akkus, Ruichuan Chen, Alice Dethise, Klaus Satzke, Ivica Rimac and [Yang Zhang](https://md.research.elamur.ai/free/en/authors/yang-zhang/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: EACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-eacl.154/) · [PDF](https://aclanthology.org/2026.findings-eacl.154.pdf) · [DOI 10.18653/v1/2026.findings-eacl.154](https://doi.org/10.18653/v1/2026.findings-eacl.154) · [OpenAlex](https://openalex.org/W7140090786)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Vision Language Models can read personal data from images such as scans or ID cards and perform PII extraction at scale. Their safety guardrails can be bypassed through jailbreaks or backdoors, and the visual channel adds another attack surface.

Existing PII benchmarks are text-only, so the authors first build multimodal test sets by turning text PII corpora into images. They use two strategies: direct conversion (rendering Enron emails or synthetic records as documents and tables, with simulated scanning noise) and context injection.

They then propose a concept-guided mitigation inspired by interpretability research. It locates internal states of the LLM backbone linked to PII-related content and amplifies them with a steering coefficient, so the model declines such requests. No retraining or fine-tuning is needed.

The steered models refuse PII tasks 93.3% of the time on average, with little harm to unrelated tasks. One fixed configuration works across all datasets and on PII types absent from the demonstration data. This offers a cheap, deployable privacy safeguard that may transfer to other multimodal LLMs built on similar backbones.

## Key points
- New multimodal PII datasets include PII-Table (tables from ai4privacy synthetic data, with optional "scanned" artifacts), image versions of the Enron email corpus, and CelebA-Info. The authors note that text-to-image models cannot yet render legible PII realistically, which is why they converted text corpora instead.
- Average refusal rate on PII-related tasks is 93.3%, while refusals on non-PII tasks stay negligible. Steering settings were not re-tuned between datasets, and the method also caught unseen types such as social security and credit card numbers.
- Results vary by PII type on CelebA-Info. Address extraction is refused in over 97% of cases for all three backbones, but email is weaker (79.1% for Vicuna-7B), and other email and phone results sit above 80%.
- The steering coefficient study on the Mistral-7B backbone found a clear gap, around 0.4–0.6, between where mitigation weakens and where benign tasks start to suffer. This makes it possible to pick a safe minimal value.
- The method only works if the model already has refusal behavior to amplify, and smaller models may lack the needed concepts. Code and dataset tools are released at github.com/Nokia-Bell-Labs/cerberus.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Helmholtz Center for Information Security](https://md.research.elamur.ai/free/en/labs/helmholtz-center-for-information-security/) — Research institute, Saarbrücken, Germany · 20 works · 1,172 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,954 works · [🤝 Sociology and Political Science](https://md.research.elamur.ai/free/en/shelves/social-sciences/sociology-and-political-science/) · 626 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 811 works

## Cite

Cite as: Boyang Zhang, Istemi Ekin Akkus, Ruichuan Chen, Alice Dethise, Klaus Satzke, Ivica Rimac and Yang Zhang. 2026. Defeating Cerberus: Privacy-Leakage Mitigation in Vision Language Models. Findings of the Association for Computational Linguistics: EACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhang2026defeating,
  title = {{Defeating Cerberus: Privacy-Leakage Mitigation in Vision Language Models}},
  author = {Boyang Zhang, Istemi Ekin Akkus, Ruichuan Chen, Alice Dethise, Klaus Satzke, Ivica Rimac and Yang Zhang},
  booktitle = {Findings of the Association for Computational Linguistics: EACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-eacl.154},
  url = {https://doi.org/10.18653/v1/2026.findings-eacl.154}
}
```

- [← previous DeepSynth-Eval: Objectively Evaluating Information Consolidation in Deep Survey Writing](https://md.research.elamur.ai/free/en/papers/zhang-2026-deepsynth-eval-objectively-evaluating-information/)
- [next → Defending LLMs against Jailbreak Attacks via Template-Based ICL with a Defensive Suffix](https://md.research.elamur.ai/free/en/papers/ni-2026-defending-llms-against-jailbreak-attacks/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/29/)

## References

42 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. [https://github.com/meta-llama/llama3/](https://github.com/meta-llama/llama3/). [↗](https://github.com/meta-llama/llama3/)
2. Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. 2024. [Refusal in language models is mediated by a single direction](https://md.research.elamur.ai/free/en/papers/arditi-2024-refusal-language-models-mediated-single/). Preprint, arXiv:2406.11717.
3. Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. [Qwen-VL: A Frontier Large Vision-Language Model with Versatile Abilities](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-vl-versatile-vision-language/). CoRR abs/2308.12966.
4. Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, Quyet V. Do, Yan Xu, and Pascale Fung. 2023. [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://md.research.elamur.ai/free/en/papers/bang-2023-multitask-multilingual-multimodal-evaluation-chatgpt/). CoRR abs/2302.0402…
5. James Betker, Gabriel Goh, Li Jing, † TimBrooks, Jianfeng Wang, Linjie Li, † LongOuyang, † JuntangZhuang, † JoyceLee, † YufeiGuo, † Wesam-Manassra, † PrafullaDhariwal, † CaseyChu, † YunxinJiao, and Aditya Ramesh. 2023. Improving image generation with better captions. [find](https://scholar.google.com/scholar?q=James+Betker%2C+Gabriel+Goh%2C+Li+Jing%2C+%E2%80%A0+TimBrooks%2C+Jianfeng+Wang%2C+Linjie+Li%2C+%E2%80%A0+LongOuyang%2C+%E2%80%A0+JuntangZhuang%2C+%E2%80%A0+JoyceLee%2C+%E2%80%A0+YufeiGuo%2C+%E2%80%A0+Wesam%2DManassra%2C+%E2%80%A0+PrafullaDhariwal%2C+%E2%80%A0+CaseyChu%2C+%E2%80%A0+YunxinJiao%2C+and+Ad)
6. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. [find](https://scholar.google.com/scholar?q=Wei%2DLin+Chiang%2C+Zhuohan+Li%2C+Zi+Lin%2C+Ying+Sheng%2C+Zhanghao+Wu%2C+Hao+Zhang%2C+Lianmin+Zheng%2C+Siyuan+Zhuang%2C+Yonghao+Zhuang%2C+Joseph+E%2E+Gonzalez%2C+Ion+Stoica%2C+and+Eric+P%2E+Xing%2E+2023%2E+Vicuna%3A+An+open%2Dsource+cha)
7. Gelei Deng, Yi Liu, Yuekang Li, Kailong Wang, Ying Zhang, Zefeng Li, Haoyu Wang, Tianwei Zhang, and Yang Liu. 2023. Jailbreaker: Automated Jailbreak Across Multiple Large Language Model Chatbots. CoRR abs/2307.08715. — [arXiv:2307.08715](https://arxiv.org/abs/2307.08715) [find](https://scholar.google.com/scholar?q=Gelei+Deng%2C+Yi+Liu%2C+Yuekang+Li%2C+Kailong+Wang%2C+Ying+Zhang%2C+Zefeng+Li%2C+Haoyu+Wang%2C+Tianwei+Zhang%2C+and+Yang+Liu%2E+2023%2E+Jailbreaker%3A+Automated+Jailbreak+Across+Multiple+Large+Language+Model+Chatbots%2E+CoRR)
8. GDPR. 2016. Regulation (eu) 2016/679 of the european parliament and of the council of 27 april 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing directive 95/46/ec (general data protection regulation) (text w… [find](https://scholar.google.com/scholar?q=GDPR%2E+2016%2E+Regulation+%28eu%29+2016%2F679+of+the+european+parliament+and+of+the+council+of+27+april+2016+on+the+protection+of+natural+persons+with+regard+to+the+processing+of+personal+data+and+on+the+free+)
9. Yichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang, Tianshuo Cong, Anyu Wang, Sisi Duan, and Xiaoyun Wang. 2023. [FigStep: Jailbreaking Large Vision-language Models via Typographic Visual Prompts](https://md.research.elamur.ai/free/en/papers/gong-2023-figstep-jailbreaking-large-vision-language/). CoRR abs/2311.05608.
10. Xiangming Gu, Xiaosen Zheng, Tianyu Pang, Chao Du, Qian Liu, Ye Wang, Jing Jiang, and Min Lin. 2024. Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast. In International Conference on Machine Learning (ICML). PMLR. [find](https://scholar.google.com/scholar?q=Xiangming+Gu%2C+Xiaosen+Zheng%2C+Tianyu+Pang%2C+Chao+Du%2C+Qian+Liu%2C+Ye+Wang%2C+Jing+Jiang%2C+and+Min+Lin%2E+2024%2E+Agent+Smith%3A+A+Single+Image+Can+Jailbreak+One+Million+Multimodal+LLM+Agents+Exponentially+Fast%2E+In+)
11. Bryan Klimt and Yiming Yang. 2004. The Enron Corpus: A New Dataset for Email Classification Research. In European Conference on Machine Learning (ECML), pages 217–226. Springer. [find](https://scholar.google.com/scholar?q=Bryan+Klimt+and+Yiming+Yang%2E+2004%2E+The+Enron+Corpus%3A+A+New+Dataset+for+Email+Classification+Research%2E+In+European+Conference+on+Machine+Learning+%28ECML%29%2C+pages+217%E2%80%93226%2E+Springer%2E)
12. Iro Laina, Christian Rupprecht, and Nassir Navab. 2019. Towards Unsupervised Image Captioning With Shared Multimodal Embeddings. In IEEE International Conference on Computer Vision (ICCV), pages 7413–7423. IEEE. [find](https://scholar.google.com/scholar?q=Iro+Laina%2C+Christian+Rupprecht%2C+and+Nassir+Navab%2E+2019%2E+Towards+Unsupervised+Image+Captioning+With+Shared+Multimodal+Embeddings%2E+In+IEEE+International+Conference+on+Computer+Vision+%28ICCV%29%2C+pages+7413%E2%80%93)
13. Andrew Lee, Xiaoyan Bai, Itamar Pres, Martin Wattenberg, Jonathan K. Kummerfeld, and Rada Mihalcea. 2024. [A mechanistic understanding of alignment algorithms: A case study on DPO and toxicity](https://md.research.elamur.ai/free/en/papers/lee-2024-mechanistic-understanding-alignment-algorithms-case/). In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024.…
14. Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. 2024. [Llava-onevision: Easy visual task transfer](https://md.research.elamur.ai/free/en/papers/li-2024-llava-onevision-easy-visual-task/). Preprint, arXiv:2408.03326.
15. Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2023a. [Improved Baselines with Visual Instruction Tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-improved-baselines-visual-instruction-tuning/). CoRR abs/2310.03744.
16. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023b. [Visual Instruction Tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-visual-instruction-tuning/). In Annual Conference on Neural Information Processing Systems (NeurIPS). NeurIPS.
17. Yi Liu, Gelei Deng, Zhengzi Xu, Yuekang Li, Yaowen Langdon Holmes, Scott Crossley, Perpetual Baffour, Zheng, Ying Zhang, Lida Zhao, Tianwei Zhang, Jules King, Lauryn Burleigh, Maggie Demkin, and Yang Liu. 2023c. Jailbreaking ChatGPT via Ryan Holbrook, Walter Reade, and Addison Prompt Engineering: An… [find](https://scholar.google.com/scholar?q=Yi+Liu%2C+Gelei+Deng%2C+Zhengzi+Xu%2C+Yuekang+Li%2C+Yaowen+Langdon+Holmes%2C+Scott+Crossley%2C+Perpetual+Baffour%2C+Zheng%2C+Ying+Zhang%2C+Lida+Zhao%2C+Tianwei+Zhang%2C+Jules+King%2C+Lauryn+Burleigh%2C+Maggie+Demkin%2C+and+Yang+)
18. CoRR Howard. 2024. [The learning agency lab - pii data abs/2305.13860](https://md.research.elamur.ai/free/en/papers/liu-2023-jailbreaking-chatgpt-prompt-engineering-empirical/). detection. [https://kaggle.com/competitions/](https://kaggle.com/competitions/) pii-detection-removal-from-educational-data. [↗](https://kaggle.com/competitions/)
19. Zhendong Liu, Yuanbi Nie, Yingshui Tan, Xiangyu Yue, Kaggle. Qiushi Cui, Chongjun Wang, Xiaoyong Zhu, and Bo Zheng. 2024. Safety Alignment for Vision Lan-Hai Huang, Zhengyu Zhao, Michael Backes, Yun guage Models. CoRR abs/2405.13581. — [arXiv:2405.13581](https://arxiv.org/abs/2405.13581) [find](https://scholar.google.com/scholar?q=Zhendong+Liu%2C+Yuanbi+Nie%2C+Yingshui+Tan%2C+Xiangyu+Yue%2C+Kaggle%2E+Qiushi+Cui%2C+Chongjun+Wang%2C+Xiaoyong+Zhu%2C+and+Bo+Zheng%2E+2024%2E+Safety+Alignment+for+Vision+Lan%2DHai+Huang%2C+Zhengyu+Zhao%2C+Michael+Backes%2C+Yun+g)
20. Shen, and Yang Zhang. 2023. Composite Backdoor Attacks Against Large Language Models. CoRR Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. abs/2310.07676. 2015. Deep Learning Face Attributes in the Wild. In IEEE International Conference on Computer Vision Jie Huang, Hanyin Shao, and Kevin Chen-… [find](https://scholar.google.com/scholar?q=Shen%2C+and+Yang+Zhang%2E+2023%2E+Composite+Backdoor+Attacks+Against+Large+Language+Models%2E+CoRR+Ziwei+Liu%2C+Ping+Luo%2C+Xiaogang+Wang%2C+and+Xiaoou+Tang%2E+abs%2F2310%2E07676%2E+2015%2E+Deep+Learning+Face+Attributes+in+t)
21. IEEE. 2022. Are Large Pre-Trained Language Models Leaking Your Personal Information? In Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2038–2047. ACL. [find](https://scholar.google.com/scholar?q=IEEE%2E+2022%2E+Are+Large+Pre%2DTrained+Language+Models+Leaking+Your+Personal+Information%3F+In+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP%29%2C+pages+2038%E2%80%932047%2E+ACL%2E)
22. Wen Huang, Hongbin Liu, Minxin Guo, and Neil Zhenqiang Gong. 2024. Visual hallucinations of multimodal large language models. arXiv preprint arXiv:2402.14683. — [arXiv:2402.14683](https://arxiv.org/abs/2402.14683) [find](https://scholar.google.com/scholar?q=Wen+Huang%2C+Hongbin+Liu%2C+Minxin+Guo%2C+and+Neil+Zhenqiang+Gong%2E+2024%2E+Visual+hallucinations+of+multimodal+large+language+models%2E+arXiv+preprint+arXiv%3A2402%2E14683%2E+%E2%80%94+%5BarXiv%3A2402%2E14683%5D%28https%3A%2F%2Farxiv%2Eorg%2Fab)
23. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… [find](https://scholar.google.com/scholar?q=Albert+Q%2E+Jiang%2C+Alexandre+Sablayrolles%2C+Arthur+Mensch%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guillaume+Lample%2C+Lucile+Saulnier%2C+L%C3%A9lio+Renard+Lav)
24. Nils Lukas, Ahmed Salem, Robert Sim, Shruti Tople, Lukas Wutschitz, and Santiago Zanella Béguelin. 2023. [Analyzing Leakage of Personally Identifiable Information in Language Models](https://md.research.elamur.ai/free/en/papers/lukas-2023-analyzing-leakage-personally-identifiable-information/). In IEEE Symposium on Security and Privacy (S&P), pages 346–363. IEEE.
25. Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. 2021. [Docvqa: A dataset for vqa on document images](https://md.research.elamur.ai/free/en/papers/mathew-2020-docvqa-dataset-vqa-document-images/). In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 2200–2209.
26. Lingbo Mo, Zeyi Liao, Boyuan Zheng, Yu Su, Chaowei Xiao, and Huan Sun. 2024. A Trembling House of Cards? Mapping Adversarial Attacks against Language Agents. CoRR abs/2402.10196. — [arXiv:2402.10196](https://arxiv.org/abs/2402.10196) [find](https://scholar.google.com/scholar?q=Lingbo+Mo%2C+Zeyi+Liao%2C+Boyuan+Zheng%2C+Yu+Su%2C+Chaowei+Xiao%2C+and+Huan+Sun%2E+2024%2E+A+Trembling+House+of+Cards%3F+Mapping+Adversarial+Attacks+against+Language+Agents%2E+CoRR+abs%2F2402%2E10196%2E+%E2%80%94+%5BarXiv%3A2402%2E10196%5D%28)
27. Jiquan Ngiam, Aditya Khosla, Mingyu Kim, Juhan Nam, Honglak Lee, and Andrew Y. Ng. 2011. Multimodal Deep Learning. In International Conference on Machine Learning (ICML), pages 689–696. Omnipress. [find](https://scholar.google.com/scholar?q=Jiquan+Ngiam%2C+Aditya+Khosla%2C+Mingyu+Kim%2C+Juhan+Nam%2C+Honglak+Lee%2C+and+Andrew+Y%2E+Ng%2E+2011%2E+Multimodal+Deep+Learning%2E+In+International+Conference+on+Machine+Learning+%28ICML%29%2C+pages+689%E2%80%93696%2E+Omnipress%2E)
28. Xinyu Zhang, Xipin Wei, Xuancheng Ren, Xuejing Liu, Yang Fan, Yang Yao, Yichang Zhang, Yu Wan, Yunfei Chu, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, Zhifang Guo, and Zhihao Fan. 2024. Qwen2 technical report. Preprint, arXiv:2407.10671. [find](https://scholar.google.com/scholar?q=Xinyu+Zhang%2C+Xipin+Wei%2C+Xuancheng+Ren%2C+Xuejing+Liu%2C+Yang+Fan%2C+Yang+Yao%2C+Yichang+Zhang%2C+Yu+Wan%2C+Yunfei+Chu%2C+Yuqiong+Liu%2C+Zeyu+Cui%2C+Zhenru+Zhang%2C+Zhifang+Guo%2C+and+Zhihao+Fan%2E+2024%2E+Qwen2+technical+repor)
29. Mansi Phute, Alec Helbling, Matthew Hull, ShengYun Peng, Sebastian Szyller, Cory Cornelius, and Duen Horng Chau. 2024. [Llm self defense: By self examination, llms know they are being tricked](https://md.research.elamur.ai/free/en/papers/phute-2023-llm-self-defense-self-examination/). Preprint, arXiv:2308.07308.
30. Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. 2023. [A Survey on Multimodal Large Language Models](https://md.research.elamur.ai/free/en/papers/yin-2023-survey-multimodal-large-language-models/). CoRR abs/2306.13549.
31. Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. 2024. [Do Anything Now: Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models](https://md.research.elamur.ai/free/en/papers/shen-2023-anything-now-characterizing-evaluating-wild/). In ACM SIGSAC Conference on Computer and Communications Security (CCS). ACM.
32. Bowei Tian, Xuntao Lyu, Meng Liu, Hongyi Wang, and Ang Li. 2025. Why representation engineering works: A theoretical and empirical study in vision-language models. Preprint, arXiv:2503.22720. — [arXiv:2503.22720](https://arxiv.org/abs/2503.22720) [find](https://scholar.google.com/scholar?q=Bowei+Tian%2C+Xuntao+Lyu%2C+Meng+Liu%2C+Hongyi+Wang%2C+and+Ang+Li%2E+2025%2E+Why+representation+engineering+works%3A+A+theoretical+and+empirical+study+in+vision%2Dlanguage+models%2E+Preprint%2C+arXiv%3A2503%2E22720%2E+%E2%80%94+%5BarXiv)
33. ZheZhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, Rui Zheng, Xiaoran Fan, Xiao Wang, Limao Xiong, Yuhao Zhou, Weiran Wang, Changhao Jiang, Yicheng Zou, Xiangyang Liu, Zhangyue Yin, Shihan Dou, Rongxiang Weng, Wensen Cheng, Qi Zhan… — [The Rise and Potential of Large Language Model Based Agents: A Survey](https://md.research.elamur.ai/free/en/papers/xi-2023-rise-potential-large-language/)
34. Yueqi Xie, Jingwei Yi, Jiawei Shao, Justin Curl, Lingjuan Lyu, Qifeng Chen, Xing Xie, and Fangzhao Wu. 2023. Defending ChatGPT against jailbreak attack via self-reminders. Nature Machine Intelligence. [find](https://scholar.google.com/scholar?q=Yueqi+Xie%2C+Jingwei+Yi%2C+Jiawei+Shao%2C+Justin+Curl%2C+Lingjuan+Lyu%2C+Qifeng+Chen%2C+Xing+Xie%2C+and+Fangzhao+Wu%2E+2023%2E+Defending+ChatGPT+against+jailbreak+attack+via+self%2Dreminders%2E+Nature+Machine+Intelligence%2E)
35. Jiashu Xu, Mingyu Derek Ma, Fei Wang, Chaowei Xiao, and Muhao Chen. 2023. Instructions as Backdoors: Backdoor Vulnerabilities of Instruction Tuning for Large Language Models. CoRR abs/2305.14710. [find](https://scholar.google.com/scholar?q=Jiashu+Xu%2C+Mingyu+Derek+Ma%2C+Fei+Wang%2C+Chaowei+Xiao%2C+and+Muhao+Chen%2E+2023%2E+Instructions+as+Backdoors%3A+Backdoor+Vulnerabilities+of+Instruction+Tuning+for+Large+Language+Models%2E+CoRR+abs%2F2305%2E14710%2E)
36. Jun Yan, Vansh Gupta, and Xiang Ren. 2023. BITE: Textual Backdoor Attacks with Iterative Trigger Injection. In Annual Meeting of the Association for Computational Linguistics (ACL), pages 12951– 12968. ACL. [find](https://scholar.google.com/scholar?q=Jun+Yan%2C+Vansh+Gupta%2C+and+Xiang+Ren%2E+2023%2E+BITE%3A+Textual+Backdoor+Attacks+with+Iterative+Trigger+Injection%2E+In+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28ACL%29%2C+pages+12951%E2%80%93+12968)
37. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Juny… — [Breaking Agents: Compromising Autonomous LLM Agents Through Malfunction Amplification](https://md.research.elamur.ai/free/en/papers/zhang-2025-breaking-agents-compromising-autonomous-llm/)
38. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). In Annual Conference on Neural Information Processing Syst…
39. Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2023. [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://md.research.elamur.ai/free/en/papers/zhu-2023-minigpt-4-enhancing-vision-language/). CoRR abs/2304.10592.
40. Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J. Zico Kolter, and Dan Hendry… — [Representation Engineering: A Top-Down Approach to AI Transparency](https://md.research.elamur.ai/free/en/papers/zou-2023-representation-engineering-top-down-approach/)
41. Andy Zou, Long Phan, Justin Wang, Derek Duenas, Maxwell Lin, Maksym Andriushchenko, Rowan Wang, Zico Kolter, Matt Fredrikson, and Dan Hendrycks. 2024. [Improving alignment and robustness with circuit breakers](https://md.research.elamur.ai/free/en/papers/zou-2024-improving-alignment-robustness-circuit-breakers/). Preprint, arXiv:2406.04313.
42. Andy Zou, Zifan Wang, J. Zico Kolter, and Matt Fredrikson. 2023b. [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://md.research.elamur.ai/free/en/papers/zou-2023-universal-transferable-adversarial-attacks-aligned/). CoRR abs/2307.15043.

*Updated 2 November 2026*