---
title: "SafeMT: Multi-turn Safety for Multimodal Language Models (2026)"
description: "Existing benchmarks mostly test single image-plus-prompt inputs, but real users talk to these models… 67 references, cited by 3 papers here."
authors: ["Han Zhu", "Juntao Dai", "Jiaming Ji", "Haoran Li", "Chengkun Cai", "Pengcheng Wen", "Chi-Min Chan", "Boyuan Chen", "Yaodong Yang", "Sirui Han", "Yike Guo"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.1920/"
doi: "10.18653/v1/2026.acl-long.1920"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhu-2026-safemt-multi-turn-safety-multimodal/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhu-2026-safemt-multi-turn-safety-multimodal/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/6/) / SafeMT: Multi-turn Safety for Multimodal Language Models

# SafeMT: Multi-turn Safety for Multimodal Language Models

**Authors:** [Han Zhu](https://md.research.elamur.ai/free/en/authors/han-zhu/), [Juntao Dai](https://md.research.elamur.ai/free/en/authors/juntao-dai/), [Jiaming Ji](https://md.research.elamur.ai/free/en/authors/jiaming-ji/), [Haoran Li](https://md.research.elamur.ai/free/en/authors/haoran-li/), [Chengkun Cai](https://md.research.elamur.ai/free/en/authors/chengkun-cai/), Pengcheng Wen, [Chi-Min Chan](https://md.research.elamur.ai/free/en/authors/chi-min-chan/), [Boyuan Chen](https://md.research.elamur.ai/free/en/authors/boyuan-chen/), [Yaodong Yang](https://md.research.elamur.ai/free/en/authors/yaodong-yang/), [Sirui Han](https://md.research.elamur.ai/free/en/authors/sirui-han/) and [Yike Guo](https://md.research.elamur.ai/free/en/authors/yike-guo/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.1920/) · [PDF](https://aclanthology.org/2026.acl-long.1920.pdf) · [DOI 10.18653/v1/2026.acl-long.1920](https://doi.org/10.18653/v1/2026.acl-long.1920) · [OpenAlex](https://openalex.org/W4415263141)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [3](https://md.research.elamur.ai/free/en/papers/zhu-2026-safemt-multi-turn-safety-multimodal/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
SafeMT targets a gap in the safety evaluation of multimodal large language models (MLLMs). Existing benchmarks mostly test single image-plus-prompt inputs, but real users talk to these models over many turns, and harmful intent can be spread across a conversation. The authors combined red-team queries from earlier datasets with their own manually written ones, paired them with images, and had GPT-4o, Claude-3.7-Sonnet and Gemini-2.0-flash expand them into dialogues of 2, 4, 6 and 8 turns. They then manually removed refusal phrases and safety reminders that would otherwise warn the model under test. They also propose a Safety Index (SI), which extends ASR to account for how many turns a model withstands and how stable its defense stays. Tests on 20 models show that attacks succeed more often as dialogues get longer. A plug-and-play dialogue safety moderator, which detects concealed intent and supplies relevant safety rules, lowers multi-turn ASR more than LlavaGuard. This makes SafeMT a practical resource for testing and protecting conversational MLLMs.

## Key points
- SafeMT contains 10,000 samples: 2,000 image-paired harmful queries and 8,000 dialogues built from them. It covers 17 scenarios grouped into five types (Ethics, Illegality, Certification, Sensitivity, Harmful Impact) and uses four jailbreak methods.
- For quality control, 200 randomly sampled dialogues of each length were reviewed by hand. The dataset is published on Hugging Face (HappyCorpse/SafeMT).
- Evaluation of 20 MLLMs reports ASR, Refusal Rate and the new SI. Attack success rises with the number of turns, which suggests safety filters are strongest early in a conversation and weaken later.
- The moderator improved safety for all tested models except LLaVA-NeXT-7B and LLaVA-NeXT-13B. It caused over-refusal in Llama-3.2-11B/90B-Vision-Instruct and helped most in scenarios that require professional licenses.
- Limitations: dialogues use a single image each, with no multi-image or adversarial-image attacks, and refining the generated dialogues required substantial manual effort.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (3): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (3): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 657 works · [🛡️ Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/) · 517 works

## Cite

Cite as: Han Zhu, Juntao Dai, Jiaming Ji, Haoran Li, Chengkun Cai, Pengcheng Wen, Chi-Min Chan, Boyuan Chen, Yaodong Yang, Sirui Han and Yike Guo. 2026. SafeMT: Multi-turn Safety for Multimodal Language Models. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhu2026safemt,
  title = {{SafeMT: Multi-turn Safety for Multimodal Language Models}},
  author = {Han Zhu, Juntao Dai, Jiaming Ji, Haoran Li, Chengkun Cai, Pengcheng Wen, Chi-Min Chan, Boyuan Chen, Yaodong Yang, Sirui Han and Yike Guo},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1920},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1920}
}
```

- [← previous SAFE-QAQ: End-to-End Slow-Thinking Audio-Text Fraud Detection via Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/wang-2026-safe-qaq-end-end-slow/)
- [next → SAT: Balancing Reasoning Accuracy and Efficiency with Stepwise Adaptive Thinking](https://md.research.elamur.ai/free/en/papers/huang-2026-sat-balancing-reasoning-accuracy-efficiency/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/6/)

## References

67 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Rawan AlSaad, Alaa Abd-Alrazaq, Sabri Boughorbel, Arfan Ahmed, Max-Antoine Renault, Rafat Damseh, and Javaid Sheikh. 2024. Multimodal large language models in health care: applications, challenges, and future outlook. Journal of medical Internet research, 26:e59505. 2024. Many-shot jailbreaking. Adv… [find](https://scholar.google.com/scholar?q=Rawan+AlSaad%2C+Alaa+Abd%2DAlrazaq%2C+Sabri+Boughorbel%2C+Arfan+Ahmed%2C+Max%2DAntoine+Renault%2C+Rafat+Damseh%2C+and+Javaid+Sheikh%2E+2024%2E+Multimodal+large+language+models+in+health+care%3A+applications%2C+challenges%2C+an)
2. Anthropic. 2025. Claude 3.7 sonnet system card. [find](https://scholar.google.com/scholar?q=Anthropic%2E+2025%2E+Claude+3%2E7+sonnet+system+card%2E)
3. Hongbo Bai, Yujin Zhou, Yile Wu, Chi-Min Chan, Pengcheng Wen, Kunhao Pan, Sirui Han, and Yike Guo. 2026. Glance-or-gaze: Incentivizing lmms to adaptively focus search via reinforcement learning. arXiv preprint arXiv:2601.13942. [find](https://scholar.google.com/scholar?q=Hongbo+Bai%2C+Yujin+Zhou%2C+Yile+Wu%2C+Chi%2DMin+Chan%2C+Pengcheng+Wen%2C+Kunhao+Pan%2C+Sirui+Han%2C+and+Yike+Guo%2E+2026%2E+Glance%2Dor%2Dgaze%3A+Incentivizing+lmms+to+adaptively+focus+search+via+reinforcement+learning%2E+arXiv)
4. Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. [Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-vl-versatile-vision-language/). arXiv preprint arXiv:2308.12966.
5. Chengkun Cai, Xu Zhao, Haoliang Liu, Zhongyu Jiang, Tianfang Zhang, Zongkai Wu, Jenq-Neng Hwang, and Lei Li. 2024. [The role of deductive and inductive reasoning in large language models](https://md.research.elamur.ai/free/en/papers/cai-2025-role-deductive-inductive-reasoning-large/). arXiv preprint arXiv:2410.02892.
6. Chuxue Cao, Han Zhu, Jiaming Ji, Qichao Sun, Zhenghao Zhu, Wu Yinyu, Josef Dai, Yaodong Yang, Sirui Han, and Yike Guo. 2025a. SafeLawBench: Towards safe alignment of large language models. In Findings of the Association for Computational Linguistics: ACL 2025, pages 14015–14048, Vienna, Austria. Ass… [find](https://scholar.google.com/scholar?q=Chuxue+Cao%2C+Han+Zhu%2C+Jiaming+Ji%2C+Qichao+Sun%2C+Zhenghao+Zhu%2C+Wu+Yinyu%2C+Josef+Dai%2C+Yaodong+Yang%2C+Sirui+Han%2C+and+Yike+Guo%2E+2025a%2E+SafeLawBench%3A+Towards+safe+alignment+of+large+language+models%2E+In+Findings)
7. Hongye Cao, Yanming Wang, Sijia Jing, Ziyue Peng, Zhixin Bai, Zhe Cao, Meng Fang, Fan Feng, Boyan Wang, Jiaheng Liu, Tianpei Yang, Jing Huo, Yang Gao, Fanyu Meng, Xi Yang, Chao Deng, and Junlan Feng. 2025b. Safedialbench: A fine-grained safety benchmark for large language models in multi-turn dialog… — [arXiv:2502.11090](https://arxiv.org/abs/2502.11090) [find](https://scholar.google.com/scholar?q=Hongye+Cao%2C+Yanming+Wang%2C+Sijia+Jing%2C+Ziyue+Peng%2C+Zhixin+Bai%2C+Zhe+Cao%2C+Meng+Fang%2C+Fan+Feng%2C+Boyan+Wang%2C+Jiaheng+Liu%2C+Tianpei+Yang%2C+Jing+Huo%2C+Yang+Gao%2C+Fanyu+Meng%2C+Xi+Yang%2C+Chao+Deng%2C+and+Junlan+Feng%2E+)
8. Jiayi Chen, Urs Hengartner, and Hassan Khan. 2024. Shrimps: A framework for evaluating multi-user, multi-modal implicit authentication systems. Computers & Security, 137:103594. [find](https://scholar.google.com/scholar?q=Jiayi+Chen%2C+Urs+Hengartner%2C+and+Hassan+Khan%2E+2024%2E+Shrimps%3A+A+framework+for+evaluating+multi%2Duser%2C+multi%2Dmodal+implicit+authentication+systems%2E+Computers+%26+Security%2C+137%3A103594%2E)
9. Yang Chen, Ethan Mendes, Sauvik Das, Wei Xu, and Alan Ritter. 2023. Can language models be instructed to protect personal information? Preprint, arXiv:2310.02224. — [arXiv:2310.02224](https://arxiv.org/abs/2310.02224) [find](https://scholar.google.com/scholar?q=Yang+Chen%2C+Ethan+Mendes%2C+Sauvik+Das%2C+Wei+Xu%2C+and+Alan+Ritter%2E+2023%2E+Can+language+models+be+instructed+to+protect+personal+information%3F+Preprint%2C+arXiv%3A2310%2E02224%2E+%E2%80%94+%5BarXiv%3A2310%2E02224%5D%28https%3A%2F%2Farxiv%2Eor)
10. Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Mané. 2016. [Concrete problems in ai safety](https://md.research.elamur.ai/free/en/papers/amodei-2016-concrete-problems-ai-safety/). Preprint, arXiv:1606.06565.
11. Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, Lixin Gu, Xuehui Wang, Qingyun Li, Yimin Ren, Zixuan Chen, Jiapeng Luo, Jiahao Wang, Tan Jiang, Bo Wang, and 23 others. 2025. [Expanding performance boundaries of open-source multi…](https://md.research.elamur.ai/free/en/papers/chen-2024-expanding-performance-boundaries-open/)
12. Cem Anil, Esin Durmus, Nina Panickssery, Mrinank Sharma, Joe Benton, Sandipan Kundu, Joshua Batson, Meg Tong, Jesse Mu, Daniel Ford, and 1 others. [find](https://scholar.google.com/scholar?q=Cem+Anil%2C+Esin+Durmus%2C+Nina+Panickssery%2C+Mrinank+Sharma%2C+Joe+Benton%2C+Sandipan+Kundu%2C+Joshua+Batson%2C+Meg+Tong%2C+Jesse+Mu%2C+Daniel+Ford%2C+and+1+others%2E)
13. Alison Fang and J Perkins. 2024. Large language models (llms): Risks and policy implications. MIT Science Policy Review, 5:134–45. [find](https://scholar.google.com/scholar?q=Alison+Fang+and+J+Perkins%2E+2024%2E+Large+language+models+%28llms%29%3A+Risks+and+policy+implications%2E+MIT+Science+Policy+Review%2C+5%3A134%E2%80%9345%2E)
14. Junfeng Fang, Yukai Wang, Ruipeng Wang, Zijun Yao, Kun Wang, An Zhang, Xiang Wang, and Tat-Seng Chua. 2025. Safemlrm: Demystifying safety in multi-modal large reasoning models. arXiv preprint arXiv:2504.08813. — [arXiv:2504.08813](https://arxiv.org/abs/2504.08813) [find](https://scholar.google.com/scholar?q=Junfeng+Fang%2C+Yukai+Wang%2C+Ruipeng+Wang%2C+Zijun+Yao%2C+Kun+Wang%2C+An+Zhang%2C+Xiang+Wang%2C+and+Tat%2DSeng+Chua%2E+2025%2E+Safemlrm%3A+Demystifying+safety+in+multi%2Dmodal+large+reasoning+models%2E+arXiv+preprint+arXiv%3A25)
15. Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, Andy Jones, Sam Bowman, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Nelson Elhage, Sheer El-Showk, Stanislav Fort, and 17 others. 2022. [Red teaming…](https://md.research.elamur.ai/free/en/papers/ganguli-2022-red-teaming-language-models-reduce/)
16. Yichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang, Tianshuo Cong, Anyu Wang, Sisi Duan, and Xiaoyun Wang. 2023. Figstep: Jailbreaking large vision-language models via typographic visual prompts. arXiv preprint arXiv:2311.05608. [find](https://scholar.google.com/scholar?q=Yichen+Gong%2C+Delong+Ran%2C+Jinyuan+Liu%2C+Conglei+Wang%2C+Tianshuo+Cong%2C+Anyu+Wang%2C+Sisi+Duan%2C+and+Xiaoyun+Wang%2E+2023%2E+Figstep%3A+Jailbreaking+large+vision%2Dlanguage+models+via+typographic+visual+prompts%2E+arXi)
17. Google. 2024a. Introducing gemini 2.0: our new ai model for the agentic era. [find](https://scholar.google.com/scholar?q=Google%2E+2024a%2E+Introducing+gemini+2%2E0%3A+our+new+ai+model+for+the+agentic+era%2E)
18. Google. 2024b. Policy guidelines for the gemini app. [find](https://scholar.google.com/scholar?q=Google%2E+2024b%2E+Policy+guidelines+for+the+gemini+app%2E)
19. Yunhao Gou, Kai Chen, Zhili Liu, Lanqing Hong, Hang Xu, Zhenguo Li, Dit-Yan Yeung, James T Kwok, and Yu Zhang. 2024. Eyes closed, safety on: Protecting multimodal llms via image-to-text transformation. In European Conference on Computer Vision, pages 388–404. Springer. [find](https://scholar.google.com/scholar?q=Yunhao+Gou%2C+Kai+Chen%2C+Zhili+Liu%2C+Lanqing+Hong%2C+Hang+Xu%2C+Zhenguo+Li%2C+Dit%2DYan+Yeung%2C+James+T+Kwok%2C+and+Yu+Zhang%2E+2024%2E+Eyes+closed%2C+safety+on%3A+Protecting+multimodal+llms+via+image%2Dto%2Dtext+transformation)
20. Tianle Gu, Zeyang Zhou, Kexin Huang, Dandan Liang, Yixu Wang, Haiquan Zhao, Yuanqi Yao, Xingge Qiao, Keqing Wang, Yujiu Yang, Yan Teng, Yu Qiao, and Yingchun Wang. 2024. Mllmguard: A multidimensional safety evaluation suite for multimodal large language models. Preprint, arXiv:2406.07594. — [arXiv:2406.07594](https://arxiv.org/abs/2406.07594) [find](https://scholar.google.com/scholar?q=Tianle+Gu%2C+Zeyang+Zhou%2C+Kexin+Huang%2C+Dandan+Liang%2C+Yixu+Wang%2C+Haiquan+Zhao%2C+Yuanqi+Yao%2C+Xingge+Qiao%2C+Keqing+Wang%2C+Yujiu+Yang%2C+Yan+Teng%2C+Yu+Qiao%2C+and+Yingchun+Wang%2E+2024%2E+Mllmguard%3A+A+multidimensional+)
21. Lukas Helff, Felix Friedrich, Manuel Brack, Kristian Kersting, and Patrick Schramowski. 2025. Llava-guard: An open vlm-based framework for safeguarding vision datasets and models. Preprint, arXiv:2406.05113. — [arXiv:2406.05113](https://arxiv.org/abs/2406.05113) [find](https://scholar.google.com/scholar?q=Lukas+Helff%2C+Felix+Friedrich%2C+Manuel+Brack%2C+Kristian+Kersting%2C+and+Patrick+Schramowski%2E+2025%2E+Llava%2Dguard%3A+An+open+vlm%2Dbased+framework+for+safeguarding+vision+datasets+and+models%2E+Preprint%2C+arXiv%3A2406)
22. Hong Kong Generative AI Research Hong Kong Digital Policy Office and Development Center. 2025. Hong kong generative artificial intelligence technical and application guideline. [find](https://scholar.google.com/scholar?q=Hong+Kong+Generative+AI+Research+Hong+Kong+Digital+Policy+Office+and+Development+Center%2E+2025%2E+Hong+kong+generative+artificial+intelligence+technical+and+application+guideline%2E)
23. Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, and 1 others. 2023. [Llama guard: Llm-based input-output safeguard for human-ai conversations](https://md.research.elamur.ai/free/en/papers/inan-2023-llama-guard-llm-based-input/). arXiv preprint arXiv:2312.06674.
24. Jiaming Ji, Wenqi Chen, Kaile Wang, Donghai Hong, Sitong Fang, Boyuan Chen, Jiayi Zhou, Juntao Dai, Sirui Han, Yike Guo, and Yaodong Yang. 2025a. Mitigating deceptive alignment via self-monitoring. Preprint, arXiv:2505.18807. — [arXiv:2505.18807](https://arxiv.org/abs/2505.18807) [find](https://scholar.google.com/scholar?q=Jiaming+Ji%2C+Wenqi+Chen%2C+Kaile+Wang%2C+Donghai+Hong%2C+Sitong+Fang%2C+Boyuan+Chen%2C+Jiayi+Zhou%2C+Juntao+Dai%2C+Sirui+Han%2C+Yike+Guo%2C+and+Yaodong+Yang%2E+2025a%2E+Mitigating+deceptive+alignment+via+self%2Dmonitoring%2E+Pr)
25. Jiaming Ji, Xinyu Chen, Rui Pan, Han Zhu, Conghui Zhang, Jiahao Li, Donghai Hong, Boyuan Chen, Jiayi Zhou, Kaile Wang, and 1 others. 2025b. Safe rlhf-v: Safe reinforcement learning from human feedback in multimodal large language models. arXiv preprint arXiv:2503.17682. — [arXiv:2503.17682](https://arxiv.org/abs/2503.17682) [find](https://scholar.google.com/scholar?q=Jiaming+Ji%2C+Xinyu+Chen%2C+Rui+Pan%2C+Han+Zhu%2C+Conghui+Zhang%2C+Jiahao+Li%2C+Donghai+Hong%2C+Boyuan+Chen%2C+Jiayi+Zhou%2C+Kaile+Wang%2C+and+1+others%2E+2025b%2E+Safe+rlhf%2Dv%3A+Safe+reinforcement+learning+from+human+feedback)
26. Jiaming Ji, Mickel Liu, Josef Dai, Xuehai Pan, Chi Zhang, Ce Bian, Boyuan Chen, Ruiyang Sun, Yizhou Wang, and Yaodong Yang. 2023. [Beavertails: Towards improved safety alignment of llm via a human-preference dataset](https://md.research.elamur.ai/free/en/papers/ji-2023-beavertails-towards-improved-safety-alignment/). Advances in Neural Information Processing Systems, 36:24678–24704.
27. Haoran Li, Wenbin Hu, Huihao Jing, Yulin Chen, Qi Hu, Sirui Han, Tianshu Chu, Peizhao Hu, and Yangqiu Song. 2025a. [PrivaCI-bench: Evaluating privacy with contextual integrity and legal compliance](https://md.research.elamur.ai/free/en/papers/li-2025-privaci-bench-evaluating-privacy-contextual/). In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: L…
28. Lei Li, Sen Jia, Jianhao Wang, Zhaochong An, Jiaang Li, Jenq-Neng Hwang, and Serge Belongie. 2025b. Chatmotion: A multimodal multi-agent for human motion analysis. arXiv preprint arXiv:2502.18180. — [arXiv:2502.18180](https://arxiv.org/abs/2502.18180) [find](https://scholar.google.com/scholar?q=Lei+Li%2C+Sen+Jia%2C+Jianhao+Wang%2C+Zhaochong+An%2C+Jiaang+Li%2C+Jenq%2DNeng+Hwang%2C+and+Serge+Belongie%2E+2025b%2E+Chatmotion%3A+A+multimodal+multi%2Dagent+for+human+motion+analysis%2E+arXiv+preprint+arXiv%3A2502%2E18180%2E+%E2%80%94+%5B)
29. Lijun Li, Bowen Dong, Ruohui Wang, Xuhao Hu, Wangmeng Zuo, Dahua Lin, Yu Qiao, and Jing Shao. 2024. [Salad-bench: A hierarchical and comprehensive safety benchmark for large language models](https://md.research.elamur.ai/free/en/papers/li-2024-salad-bench-hierarchical-comprehensive-safety/). Preprint, arXiv:2402.05044.
30. Shuaimin Li, Xuanang Chen, Yuanfeng Song, Yunze Song, Chen Jason Zhang, Fei Hao, and Lei Chen. 2025c. prompt4vis: prompting large language models with example mining for tabular data visualization. The VLDB Journal, 34. [find](https://scholar.google.com/scholar?q=Shuaimin+Li%2C+Xuanang+Chen%2C+Yuanfeng+Song%2C+Yunze+Song%2C+Chen+Jason+Zhang%2C+Fei+Hao%2C+and+Lei+Chen%2E+2025c%2E+prompt4vis%3A+prompting+large+language+models+with+example+mining+for+tabular+data+visualization%2E+Th)
31. Fenglin Liu, Tingting Zhu, Xian Wu, Bang Yang, Chenyu You, Chenyang Wang, Lei Lu, Zhangdaihong Liu, Yefeng Zheng, Xu Sun, and 1 others. 2023a. A medical multimodal large language model for future pandemics. NPJ Digital Medicine, 6(1):226. [find](https://scholar.google.com/scholar?q=Fenglin+Liu%2C+Tingting+Zhu%2C+Xian+Wu%2C+Bang+Yang%2C+Chenyu+You%2C+Chenyang+Wang%2C+Lei+Lu%2C+Zhangdaihong+Liu%2C+Yefeng+Zheng%2C+Xu+Sun%2C+and+1+others%2E+2023a%2E+A+medical+multimodal+large+language+model+for+future+pand)
32. Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2023b. [Improved baselines with visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-improved-baselines-visual-instruction-tuning/).
33. Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. 2024a. Llava-next: Improved reasoning, ocr, and world knowledge. [find](https://scholar.google.com/scholar?q=Haotian+Liu%2C+Chunyuan+Li%2C+Yuheng+Li%2C+Bo+Li%2C+Yuanhan+Zhang%2C+Sheng+Shen%2C+and+Yong+Jae+Lee%2E+2024a%2E+Llava%2Dnext%3A+Improved+reasoning%2C+ocr%2C+and+world+knowledge%2E)
34. Libin Liu, Shen Chen, Sen Jia, Jingzhe Shi, Zhongyu Jiang, Can Jin, Wu Zongkai, Jenq-Neng Hwang, and Lei Li. 2024b. Graph canvas for controllable 3d scene generation. arXiv preprint arXiv:2412.00091. — [arXiv:2412.00091](https://arxiv.org/abs/2412.00091) [find](https://scholar.google.com/scholar?q=Libin+Liu%2C+Shen+Chen%2C+Sen+Jia%2C+Jingzhe+Shi%2C+Zhongyu+Jiang%2C+Can+Jin%2C+Wu+Zongkai%2C+Jenq%2DNeng+Hwang%2C+and+Lei+Li%2E+2024b%2E+Graph+canvas+for+controllable+3d+scene+generation%2E+arXiv+preprint+arXiv%3A2412%2E00091%2E+)
35. Parker Liu, Chenxin Li, Zhengxin Li, Yipeng Wu, Wuyang Li, Zhiqin Yang, Zhenyuan Zhang, Yunlong Lin, Sirui Han, and Brandon Y. Feng. 2025. Ir3d-bench: Evaluating vision-language model scene understanding as agentic inverse rendering. Preprint, arXiv:2506.23329. — [arXiv:2506.23329](https://arxiv.org/abs/2506.23329) [find](https://scholar.google.com/scholar?q=Parker+Liu%2C+Chenxin+Li%2C+Zhengxin+Li%2C+Yipeng+Wu%2C+Wuyang+Li%2C+Zhiqin+Yang%2C+Zhenyuan+Zhang%2C+Yunlong+Lin%2C+Sirui+Han%2C+and+Brandon+Y%2E+Feng%2E+2025%2E+Ir3d%2Dbench%3A+Evaluating+vision%2Dlanguage+model+scene+understand)
36. Xin Liu, Yichen Zhu, Jindong Gu, Yunshi Lan, Chao Yang, and Yu Qiao. 2024c. Mm-safetybench: A benchmark for safety evaluation of multimodal large language models. Preprint, arXiv:2311.17600. [find](https://scholar.google.com/scholar?q=Xin+Liu%2C+Yichen+Zhu%2C+Jindong+Gu%2C+Yunshi+Lan%2C+Chao+Yang%2C+and+Yu+Qiao%2E+2024c%2E+Mm%2Dsafetybench%3A+A+benchmark+for+safety+evaluation+of+multimodal+large+language+models%2E+Preprint%2C+arXiv%3A2311%2E17600%2E)
37. Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Hao Yang, Yaofeng Sun, Chengqi Deng, Hanwei Xu, Zhenda Xie, and Chong Ruan. 2024. [Deepseek-vl: Towards real-world vision-language understanding](https://md.research.elamur.ai/free/en/papers/lu-2024-deepseek-vl-real-world/). Preprint, arXiv:2403.05525.
38. Jinwei Lu, Yuanfeng Song, Chen Zhang, and Raymond Chi-Wing Wong. 2026. Multivis-agent: A multi-agent framework with logic rules for reliable and comprehensive cross-modal data visualization. Proc. ACM Manag. Data, 4(1). [find](https://scholar.google.com/scholar?q=Jinwei+Lu%2C+Yuanfeng+Song%2C+Chen+Zhang%2C+and+Raymond+Chi%2DWing+Wong%2E+2026%2E+Multivis%2Dagent%3A+A+multi%2Dagent+framework+with+logic+rules+for+reliable+and+comprehensive+cross%2Dmodal+data+visualization%2E+Proc%2E+ACM)
39. Weidi Luo, Siyuan Ma, Xiaogeng Liu, Xiaoyu Guo, and Chaowei Xiao. 2024. Jailbreakv: A benchmark for assessing the robustness of multimodal large language models against jailbreak attacks. arXiv preprint arXiv:2404.03027. [find](https://scholar.google.com/scholar?q=Weidi+Luo%2C+Siyuan+Ma%2C+Xiaogeng+Liu%2C+Xiaoyu+Guo%2C+and+Chaowei+Xiao%2E+2024%2E+Jailbreakv%3A+A+benchmark+for+assessing+the+robustness+of+multimodal+large+language+models+against+jailbreak+attacks%2E+arXiv+prepri)
40. Meta. 2025. Llama 3.2: Revolutionizing edge ai and vision with open, customizable models. [find](https://scholar.google.com/scholar?q=Meta%2E+2025%2E+Llama+3%2E2%3A+Revolutionizing+edge+ai+and+vision+with+open%2C+customizable+models%2E)
41. Yutao Mou, Shikun Zhang, and Wei Ye. 2024. Sg-bench: Evaluating llm safety generalization across diverse tasks and prompt types. Preprint, arXiv:2410.21965. — [arXiv:2410.21965](https://arxiv.org/abs/2410.21965) [find](https://scholar.google.com/scholar?q=Yutao+Mou%2C+Shikun+Zhang%2C+and+Wei+Ye%2E+2024%2E+Sg%2Dbench%3A+Evaluating+llm+safety+generalization+across+diverse+tasks+and+prompt+types%2E+Preprint%2C+arXiv%3A2410%2E21965%2E+%E2%80%94+%5BarXiv%3A2410%2E21965%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F)
42. OpenAI. Usage policies. OpenAI, :, Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander M adry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, and 10 others. 202… — [GPT-4o System Card](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/)
43. Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Peter Henderson, Mengdi Wang, and Prateek Mittal. 2024. Visual adversarial examples jailbreak aligned large language models. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages 21527–21536. [find](https://scholar.google.com/scholar?q=Xiangyu+Qi%2C+Kaixuan+Huang%2C+Ashwinee+Panda%2C+Peter+Henderson%2C+Mengdi+Wang%2C+and+Prateek+Mittal%2E+2024%2E+Visual+adversarial+examples+jailbreak+aligned+large+language+models%2E+In+Proceedings+of+the+AAAI+confe)
44. Katrin Renz, Long Chen, Ana-Maria Marcu, Jan Hüner-mann, Benoit Hanotte, Alice Karnsund, Jamie Shotton, Elahe Arani, and Oleg Sinavski. 2024. Carllava: Vision language models for camera-only closed-loop driving. arXiv preprint arXiv:2406.10165. — [arXiv:2406.10165](https://arxiv.org/abs/2406.10165) [find](https://scholar.google.com/scholar?q=Katrin+Renz%2C+Long+Chen%2C+Ana%2DMaria+Marcu%2C+Jan+H%C3%BCner%2Dmann%2C+Benoit+Hanotte%2C+Alice+Karnsund%2C+Jamie+Shotton%2C+Elahe+Arani%2C+and+Oleg+Sinavski%2E+2024%2E+Carllava%3A+Vision+language+models+for+camera%2Donly+closed%2Dlo)
45. Erfan Shayegani, Yue Dong, and Nael Abu-Ghazaleh. 2023. Jailbreak in pieces: Compositional adversarial attacks on multi-modal language models. arXiv preprint arXiv:2307.14539. [find](https://scholar.google.com/scholar?q=Erfan+Shayegani%2C+Yue+Dong%2C+and+Nael+Abu%2DGhazaleh%2E+2023%2E+Jailbreak+in+pieces%3A+Compositional+adversarial+attacks+on+multi%2Dmodal+language+models%2E+arXiv+preprint+arXiv%3A2307%2E14539%2E)
46. Yuanfeng Song, Jinwei Lu, and Raymond Chi-Wing Wong. 2026. Covis: Neural and llm-driven multi-turn interactions for conversational text-to-visualization generation. The VLDB Journal, 35. [find](https://scholar.google.com/scholar?q=Yuanfeng+Song%2C+Jinwei+Lu%2C+and+Raymond+Chi%2DWing+Wong%2E+2026%2E+Covis%3A+Neural+and+llm%2Ddriven+multi%2Dturn+interactions+for+conversational+text%2Dto%2Dvisualization+generation%2E+The+VLDB+Journal%2C+35%2E)
47. Spotdraft. 2024. Ai & llm use policy: Development and implementation. [find](https://scholar.google.com/scholar?q=Spotdraft%2E+2024%2E+Ai+%26+llm+use+policy%3A+Development+and+implementation%2E)
48. Hao Sun, Zhexin Zhang, Jiawen Deng, Jiale Cheng, and Minlie Huang. 2023. Safety assessment of chinese large language models. Preprint, arXiv:2304.10436. [find](https://scholar.google.com/scholar?q=Hao+Sun%2C+Zhexin+Zhang%2C+Jiawen+Deng%2C+Jiale+Cheng%2C+and+Minlie+Huang%2E+2023%2E+Safety+assessment+of+chinese+large+language+models%2E+Preprint%2C+arXiv%3A2304%2E10436%2E)
49. Core Team, Zihao Yue, Zhenru Lin, Yifan Song, Weikun Wang, Shuhuai Ren, Shuhao Gu, Shicheng Li, Peidian Li, Liang Zhao, Lei Li, Kainan Bao, Hao Tian, Hailin Zhang, Gang Wang, Dawei Zhu, Cici, Chenhong He, Bowen Ye, and 55 others. 2025a. [Mimo-vl technical report](https://md.research.elamur.ai/free/en/papers/team-2025-llm-mimo-vl-technical-report/). Preprint, arXiv:2506.03569.
50. Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, and Radu Soricut et.al. 2024. [Gemini: A family of highly capable multimodal models](https://md.research.elamur.ai/free/en/papers/gemini-team-2023-gemini-family-highly-capable-multimodal/). Preprint, arXiv:2312.11805.
51. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, and 1 others. 2025b. Gemma 3 technical report. arXiv preprint arXiv:2503.19786. [find](https://scholar.google.com/scholar?q=Gemma+Team%2C+Aishwarya+Kamath%2C+Johan+Ferret%2C+Shreya+Pathak%2C+Nino+Vieillard%2C+Ramona+Merhej%2C+Sarah+Perrin%2C+Tatiana+Matejovicova%2C+Alexandre+Ram%C3%A9%2C+Morgane+Rivi%C3%A8re%2C+and+1+others%2E+2025b%2E+Gemma+3+technical+re)
52. Kimi Team, Angang Du, Bohong Yin, Bowei Xing, Bowen Qu, Bowen Wang, Cheng Chen, Chenlin Zhang, Chenzhuang Du, Chu Wei, Congcong Wang, Dehao Zhang, Dikang Du, Dongliang Wang, Enming Yuan, Enzhe Lu, Fang Li, Flood Sung, Guangda Wei, and 76 others. 2025c. [Kimi-vl technical report](https://md.research.elamur.ai/free/en/papers/kimi-team-2025-kimi-vl-technical-report/). Preprint, arXiv:2504.…
53. V Team, Wenyi Hong, Wenmeng Yu, Xiaotao Gu, Guo Wang, Guobing Gan, Haomiao Tang, Jiale Cheng, Ji Qi, Junhui Ji, Lihang Pan, Shuaiqi Duan, Weihan Wang, Yan Wang, Yean Cheng, Zehai He, Zhe Su, Zhen Yang, Ziyang Pan, and 74 others. 2026. [Glm-4.5v and glm-4.1v-thinking: Towards versatile multimodal reas…](https://md.research.elamur.ai/free/en/papers/team-2025-v-glm-4-5v-glm/)
54. Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu, Chang Zhou, Jingren Zhou, and Junyang Lin. 2024. [Qwen2-vl: Enhancing vision-language model’s perception of the w…](https://md.research.elamur.ai/free/en/papers/wang-2024-qwen2-vl-enhancing-vision/)
55. Pengcheng Wen, Jiaming Ji, Chi-Min Chan, Juntao Dai, Donghai Hong, Yaodong Yang, Sirui Han, and Yike Guo. 2026. Thinkpatterns-21k: A systematic study on the impact of thinking patterns in llms. Preprint, arXiv:2503.12918. — [arXiv:2503.12918](https://arxiv.org/abs/2503.12918) [find](https://scholar.google.com/scholar?q=Pengcheng+Wen%2C+Jiaming+Ji%2C+Chi%2DMin+Chan%2C+Juntao+Dai%2C+Donghai+Hong%2C+Yaodong+Yang%2C+Sirui+Han%2C+and+Yike+Guo%2E+2026%2E+Thinkpatterns%2D21k%3A+A+systematic+study+on+the+impact+of+thinking+patterns+in+llms%2E+Prepri)
56. Cui Yakun, Peng Qi, Fushuo Huo, Hang Du, Weijie Shi, Juntao Dai, Zhenghao Zhu, Sirui Han, and Yike Guo. 2025. [Perception, understanding and reasoning, a multimodal benchmark for video fake news detection](https://md.research.elamur.ai/free/en/papers/yakun-2026-perception-understanding-reasoning-multimodal-benchmark/). arXiv preprint arXiv:2510.24816.
57. Hao Yang, Qianghua Zhao, and Lei Li. 2024. Chain-of-thought in large language models: Decoding, projection, and activation. arXiv preprint arXiv:2412.03944. — [arXiv:2412.03944](https://arxiv.org/abs/2412.03944) [find](https://scholar.google.com/scholar?q=Hao+Yang%2C+Qianghua+Zhao%2C+and+Lei+Li%2E+2024%2E+Chain%2Dof%2Dthought+in+large+language+models%3A+Decoding%2C+projection%2C+and+activation%2E+arXiv+preprint+arXiv%3A2412%2E03944%2E+%E2%80%94+%5BarXiv%3A2412%2E03944%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F)
58. Zonghao Ying, Aishan Liu, Siyuan Liang, Lei Huang, Jinyang Guo, Wenbo Zhou, Xianglong Liu, and Dacheng Tao. 2024. Safebench: A safety evaluation framework for multimodal large language models. arXiv preprint arXiv:2410.18927. — [arXiv:2410.18927](https://arxiv.org/abs/2410.18927) [find](https://scholar.google.com/scholar?q=Zonghao+Ying%2C+Aishan+Liu%2C+Siyuan+Liang%2C+Lei+Huang%2C+Jinyang+Guo%2C+Wenbo+Zhou%2C+Xianglong+Liu%2C+and+Dacheng+Tao%2E+2024%2E+Safebench%3A+A+safety+evaluation+framework+for+multimodal+large+language+models%2E+arXiv+p)
59. Wenhao You, Bryan Hooi, Yiwei Wang, Youke Wang, Zong Ke, Ming-Hsuan Yang, Zi Huang, and Yujun Cai. 2025. Mirage: Multimodal immersive reasoning and guided exploration for red-team jailbreak attacks. Preprint, arXiv:2503.19134. — [arXiv:2503.19134](https://arxiv.org/abs/2503.19134) [find](https://scholar.google.com/scholar?q=Wenhao+You%2C+Bryan+Hooi%2C+Yiwei+Wang%2C+Youke+Wang%2C+Zong+Ke%2C+Ming%2DHsuan+Yang%2C+Zi+Huang%2C+and+Yujun+Cai%2E+2025%2E+Mirage%3A+Multimodal+immersive+reasoning+and+guided+exploration+for+red%2Dteam+jailbreak+attacks%2E+P)
60. Erxin Yu, Jing Li, Ming Liao, Siqi Wang, Gao Zuchen, Fei Mi, and Lanqing Hong. 2024. CoSafe: Evaluating large language model safety in multi-turn dialogue coreference. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17494–17508, Miami, Florida, USA. A… [find](https://scholar.google.com/scholar?q=Erxin+Yu%2C+Jing+Li%2C+Ming+Liao%2C+Siqi+Wang%2C+Gao+Zuchen%2C+Fei+Mi%2C+and+Lanqing+Hong%2E+2024%2E+CoSafe%3A+Evaluating+large+language+model+safety+in+multi%2Dturn+dialogue+coreference%2E+In+Proceedings+of+the+2024+Confe)
61. Tongxin Yuan, Zhiwei He, Lingzhong Dong, Yiming Wang, Ruijie Zhao, Tian Xia, Lizhen Xu, Binglin Zhou, Fangqi Li, Zhuosheng Zhang, Rui Wang, and Gongshen Liu. 2024. [R-judge: Benchmarking safety risk awareness for llm agents](https://md.research.elamur.ai/free/en/papers/yuan-2024-r-judge-benchmarking-safety-risk/). Preprint, arXiv:2401.10019.
62. Zhexin Zhang, Leqi Lei, Lindong Wu, Rui Sun, Yongkang Huang, Chong Long, Xiao Liu, Xuanyu Lei, Jie Tang, and Minlie Huang. 2024. [Safetybench: Evaluating the safety of large language models](https://md.research.elamur.ai/free/en/papers/zhang-2023-safetybench-evaluating-safety-large/). Preprint, arXiv:2309.07045.
63. Kaiyuan Zheng, Qinghua Zhao, and Lei Li. 2025. Reassessing the role of chain-of-thought in sentiment analysis: Insights and limitations. In International Conference on Intelligent Computing, pages 89–100. Springer. [find](https://scholar.google.com/scholar?q=Kaiyuan+Zheng%2C+Qinghua+Zhao%2C+and+Lei+Li%2E+2025%2E+Reassessing+the+role+of+chain%2Dof%2Dthought+in+sentiment+analysis%3A+Insights+and+limitations%2E+In+International+Conference+on+Intelligent+Computing%2C+pages+89%E2%80%93)
64. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, and 1 others. 2023. [Judging llm-as-a-judge with mt-bench and chatbot arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). Advances in Neural Information Processing Systems, 36:46595–46623.
65. Xiaoji Zheng, Lixiu Wu, Zhijie Yan, Yuanrong Tang, Hao Zhao, Chen Zhong, Bokui Chen, and Jiangtao Gong. 2024. Large language models powered context-aware motion prediction in autonomous driving. In 2024 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 980– 985. IEEE. [find](https://scholar.google.com/scholar?q=Xiaoji+Zheng%2C+Lixiu+Wu%2C+Zhijie+Yan%2C+Yuanrong+Tang%2C+Hao+Zhao%2C+Chen+Zhong%2C+Bokui+Chen%2C+and+Jiangtao+Gong%2E+2024%2E+Large+language+models+powered+context%2Daware+motion+prediction+in+autonomous+driving%2E+In+20)
66. Yujin Zhou, Pengcheng Wen, Jiale Chen, Boqin Yin, Han Zhu, Jiaming Ji, Juntao Dai, Chi-Min Chan, and Sirui Han. 2026. What, whether and how? unveiling process reward models for thinking with images reasoning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 29071– 2… [find](https://scholar.google.com/scholar?q=Yujin+Zhou%2C+Pengcheng+Wen%2C+Jiale+Chen%2C+Boqin+Yin%2C+Han+Zhu%2C+Jiaming+Ji%2C+Juntao+Dai%2C+Chi%2DMin+Chan%2C+and+Sirui+Han%2E+2026%2E+What%2C+whether+and+how%3F+unveiling+process+reward+models+for+thinking+with+images+re)
67. Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, Zhangwei Gao, Erfei Cui, Xuehui Wang, Yue Cao, Yangzhou Liu, Xingguang Wei, Hongjie Zhang, Haomin Wang, Weiye Xu, and 32 others. 2025. [Internvl3: Exploring advanced training and test-…](https://md.research.elamur.ai/free/en/papers/zhu-2025-internvl3-exploring-advanced-training/)

*Updated 31 October 2026*