---
title: "TROJail — paper by Xiong et al. (2026) — summary and ideas"
description: "The paper addresses training automated attackers that break the safety guardrails of black-box LLMs… 67 references, cited by 3 papers here."
authors: ["Xiqiao Xiong", "Ouxiang Li", "Zhuo Liu", "Moxin Li", "Wentao Shi", "Fengbin Zhu", "Qifan Wang", "Fuli Feng"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.2220/"
doi: "10.18653/v1/2026.acl-long.2220"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/xiong-2026-trojail-trajectory-level-optimization-multi/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/xiong-2026-trojail-trajectory-level-optimization-multi/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/6/) / TROJail: Trajectory-Level Optimization for Multi-Turn Large Language Model Jailbreaks with Process Rewards

# TROJail: Trajectory-Level Optimization for Multi-Turn Large Language Model Jailbreaks with Process Rewards

**Authors:** Xiqiao Xiong, Ouxiang Li, [Zhuo Liu](https://md.research.elamur.ai/free/en/authors/zhuo-liu/), [Moxin Li](https://md.research.elamur.ai/free/en/authors/moxin-li/), [Wentao Shi](https://md.research.elamur.ai/free/en/authors/wentao-shi/), [Fengbin Zhu](https://md.research.elamur.ai/free/en/authors/fengbin-zhu/), [Qifan Wang](https://md.research.elamur.ai/free/en/authors/qifan-wang/) and [Fuli Feng](https://md.research.elamur.ai/free/en/authors/fuli-feng/) · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.2220/) · [PDF](https://aclanthology.org/2026.acl-long.2220.pdf) · [DOI 10.18653/v1/2026.acl-long.2220](https://doi.org/10.18653/v1/2026.acl-long.2220) · [OpenAlex](https://openalex.org/W7166827396)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [3](https://md.research.elamur.ai/free/en/papers/xiong-2026-trojail-trajectory-level-optimization-multi/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper addresses training automated attackers that break the safety guardrails of black-box LLMs over several dialogue turns. Existing trained attackers, such as Siren, MTSA and HARM, use DPO or rejection sampling to optimize each turn on its own. Because they chase immediate harmfulness, they undervalue early prompts that look harmless but set up the eventual breach. The authors instead treat attacking as multi-turn reinforcement learning. The reward is the harmfulness of the last response in the conversation.

This end-of-dialogue signal is sparse, so TROJail, built on multi-turn GRPO, adds two heuristic process rewards to the per-turn advantage estimates:
- one punishes intermediate prompts so aggressive that they trigger refusals;
- the other rewards responses that move steadily closer in meaning to the original harmful request.

Tests on HarmBench, StrongREJECT and JailbreakBench show higher attack success rates than eight baselines across victim models. The gains hold most clearly on the hardest prompts. The work matters for red-teaming and for informing multi-turn safety alignment.

## Key points
- Two process rewards: over-harm penalization (r_h1) and semantic relevance progression (r_h2). Both supplement the outcome reward r_o.
- In an ablation with Gemma-2-9B-IT as the victim, average ASR rises with each reward component:

  | Rewards used | Average ASR (%) |
  |---|---|
  | r_o only | 65.39 |
  | r_o + r_h1 | 75.50 |
  | r_o + r_h2 | 70.43 |
  | All three | 77.75 |

  For the full model, HarmBench ASR is 83.83%.
- TROJail is compared against eight baselines in total: ArtPrompt and ReNeLLM, AutoDan-Turbo and Jailbreak-R1, plus ActorAttack, X-Teaming, Siren and MTSA.
- On harder prompts, TROJail loses performance much more slowly than the baselines and adaptively uses more dialogue turns.
- Code is published at github.com/xxiqiao/TROJail. Stated limitations are that the method has no explicit diversity objective and no adversarially trained defenses.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Meta](https://md.research.elamur.ai/free/en/companies/meta/) — Company, Menlo Park, United States · 183 works · 228,558 citations
- [National University of Singapore](https://md.research.elamur.ai/free/en/universities/national-university-of-singapore/) — University, Singapore, Singapore · 180 works · 34,304 citations
- [University of Science and Technology of China](https://md.research.elamur.ai/free/en/universities/university-of-science-and-technology-of-china/) — University, Hefei, China · 108 works · 112,698 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🛡️ Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/) · 517 works

## Cite

Cite as: Xiqiao Xiong, Ouxiang Li, Zhuo Liu, Moxin Li, Wentao Shi, Fengbin Zhu, Qifan Wang and Fuli Feng. 2026. TROJail: Trajectory-Level Optimization for Multi-Turn Large Language Model Jailbreaks with Process Rewards. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{xiong2026trojail,
  title = {{TROJail: Trajectory-Level Optimization for Multi-Turn Large Language Model Jailbreaks with Process Rewards}},
  author = {Xiqiao Xiong, Ouxiang Li, Zhuo Liu, Moxin Li, Wentao Shi, Fengbin Zhu, Qifan Wang and Fuli Feng},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.2220},
  url = {https://doi.org/10.18653/v1/2026.acl-long.2220}
}
```

- [← previous TreeDiff: AST-Guided Code Generation with Diffusion LLMs](https://md.research.elamur.ai/free/en/papers/zeng-2026-treediff-ast-guided-code-generation/)
- [next → Uni-MMMU: A Massive Multi-discipline Multimodal Unified Benchmark](https://md.research.elamur.ai/free/en/papers/zou-2026-uni-mmmu-massive-multi-discipline/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/6/)

## References

67 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774.
2. Alex J Chan, Hao Sun, Samuel Holt, and Mihaela Van Der Schaar. 2024. Dense reward for free in reinforcement learning from human feedback. In Proceedings of the 41st International Conference on Machine Learning, pages 6136–6154. [find](https://scholar.google.com/scholar?q=Alex+J+Chan%2C+Hao+Sun%2C+Samuel+Holt%2C+and+Mihaela+Van+Der+Schaar%2E+2024%2E+Dense+reward+for+free+in+reinforcement+learning+from+human+feedback%2E+In+Proceedings+of+the+41st+International+Conference+on+Machine)
3. Patrick Chao, Edoardo Debenedetti, Alexander Robey, Maksym Andriushchenko, Francesco Croce, Vikash Sehwag, Edgar Dobriban, Nicolas Flammarion, George J Pappas, Florian Tramer, and 1 others. 2024. [Jailbreakbench: An open robustness benchmark for jailbreaking large language models](https://md.research.elamur.ai/free/en/papers/chao-2024-jailbreakbench-open-robustness-benchmark-jailbreaking/). Advances in Neural…
4. Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, and Eric Wong. 2025. [Jailbreaking black box large language models in twenty queries](https://md.research.elamur.ai/free/en/papers/chao-2023-jailbreaking-black-box-large-language/). In 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pages 23–42.
5. Ganqu Cui, Lifan Yuan, Zefan Wang, Hanbin Wang, Wendi Li, Bingxiang He, Yuchen Fan, Tianyu Yu, Qixin Xu, Weize Chen, and 1 others. 2025. [Process reinforcement through implicit rewards](https://md.research.elamur.ai/free/en/papers/cui-2025-process-reinforcement-through-implicit-rewards/). arXiv preprint arXiv:2502.01456.
6. Boyi Deng, Wenjie Wang, Fuli Feng, Yang Deng, Qifan Wang, and Xiangnan He. 2023. Attack prompt generation for red teaming and defending large language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2176–2189, Singapore. Association for Computational Linguisti… [find](https://scholar.google.com/scholar?q=Boyi+Deng%2C+Wenjie+Wang%2C+Fuli+Feng%2C+Yang+Deng%2C+Qifan+Wang%2C+and+Xiangnan+He%2E+2023%2E+Attack+prompt+generation+for+red+teaming+and+defending+large+language+models%2E+In+Findings+of+the+Association+for+Comput)
7. Peng Ding, Jun Kuang, Dan Ma, Xuezhi Cao, Yunsen Xian, Jiajun Chen, and Shujian Huang. 2024. A wolf in sheep’s clothing: Generalized nested jailbreak prompts can fool large language models easily. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computationa… [find](https://scholar.google.com/scholar?q=Peng+Ding%2C+Jun+Kuang%2C+Dan+Ma%2C+Xuezhi+Cao%2C+Yunsen+Xian%2C+Jiajun+Chen%2C+and+Shujian+Huang%2E+2024%2E+A+wolf+in+sheep%E2%80%99s+clothing%3A+Generalized+nested+jailbreak+prompts+can+fool+large+language+models+easily%2E+In+)
8. Team Gemini, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, and 1 others. 2023. [Gemini: a family of highly capable multimodal models](https://md.research.elamur.ai/free/en/papers/gemini-team-2023-gemini-family-highly-capable-multimodal/). arXiv preprint arXiv:2312.11805.
9. Google. 2026. Gemini api documentation. [find](https://scholar.google.com/scholar?q=Google%2E+2026%2E+Gemini+api+documentation%2E)
10. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 o… — [The Llama 3 Herd of Models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/)
11. Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, and 1 others. 2024. Deepseek-coder: When the large language model meets programming–the rise of code intelligence. arXiv preprint arXiv:2401.14196. [find](https://scholar.google.com/scholar?q=Daya+Guo%2C+Qihao+Zhu%2C+Dejian+Yang%2C+Zhenda+Xie%2C+Kai+Dong%2C+Wentao+Zhang%2C+Guanting+Chen%2C+Xiao+Bi%2C+Yu+Wu%2C+YK+Li%2C+and+1+others%2E+2024%2E+Deepseek%2Dcoder%3A+When+the+large+language+model+meets+programming%E2%80%93the+rise)
12. Weiyang Guo, Jing Li, Wenya Wang, Yu Li, Daojing He, Jun Yu, and Min Zhang. 2025a. [MTSA: Multi-turn safety alignment for LLMs through multi-round red-teaming](https://md.research.elamur.ai/free/en/papers/guo-2025-mtsa-multi-turn-safety-alignment/). In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 26424–26442, Vienna…
13. Weiyang Guo, Zesheng Shi, Zhuo Li, Yequan Wang, Xuebo Liu, Wenya Wang, Fangming Liu, Min Zhang, and Jing Li. 2025b. Jailbreak-r1: Exploring the jailbreak capabilities of llms via reinforcement learning. Preprint, arXiv:2506.00782. — [arXiv:2506.00782](https://arxiv.org/abs/2506.00782) [find](https://scholar.google.com/scholar?q=Weiyang+Guo%2C+Zesheng+Shi%2C+Zhuo+Li%2C+Yequan+Wang%2C+Xuebo+Liu%2C+Wenya+Wang%2C+Fangming+Liu%2C+Min+Zhang%2C+and+Jing+Li%2E+2025b%2E+Jailbreak%2Dr1%3A+Exploring+the+jailbreak+capabilities+of+llms+via+reinforcement+learnin)
14. Zhang-Wei Hong, Idan Shenfeld, Tsun-Hsuan Wang, Yung-Sung Chuang, Aldo Pareja, James Glass, Akash Srivastava, and Pulkit Agrawal. 2024. Curiosity-driven red-teaming for large language models. arXiv preprint arXiv:2402.19464. — [arXiv:2402.19464](https://arxiv.org/abs/2402.19464) [find](https://scholar.google.com/scholar?q=Zhang%2DWei+Hong%2C+Idan+Shenfeld%2C+Tsun%2DHsuan+Wang%2C+Yung%2DSung+Chuang%2C+Aldo+Pareja%2C+James+Glass%2C+Akash+Srivastava%2C+and+Pulkit+Agrawal%2E+2024%2E+Curiosity%2Ddriven+red%2Dteaming+for+large+language+models%2E+arXiv+pr)
15. Peichun Hua, Hao Li, Shanghao Shi, Zhiyuan Yu, and Ning Zhang. 2026. [Rethinking jailbreak detection of large vision language models with representational contrastive scoring](https://md.research.elamur.ai/free/en/papers/hua-2026-rethinking-jailbreak-detection-large-vision/). Preprint, arXiv:2512.12069.
16. OpenAI: Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, Aleksander M adry, Alex Baker-Whitcomb, Alex Beutel, Alex Borzunov, Alex Carney, Alex Chow, Alex Kirillov, Alex Nichol, Alex Paino, and 399 others. 2024… — [GPT-4o System Card](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/)
17. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… [find](https://scholar.google.com/scholar?q=Albert+Q%2E+Jiang%2C+Alexandre+Sablayrolles%2C+Arthur+Mensch%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guillaume+Lample%2C+Lucile+Saulnier%2C+L%C3%A9lio+Renard+Lav)
18. Fengqing Jiang, Zhangchen Xu, Luyao Niu, Zhen Xiang, Bhaskar Ramasubramanian, Bo Li, and Radha Poovendran. 2024. [ArtPrompt: ASCII art-based jailbreak attacks against aligned LLMs](https://md.research.elamur.ai/free/en/papers/jiang-2024-artprompt-ascii-art-based-jailbreak/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), page…
19. Seanie Lee, Minsu Kim, Lynn Cherif, David Dobre, Juho Lee, Sung Ju Hwang, Kenji Kawaguchi, Gauthier Gidel, Yoshua Bengio, Nikolay Malkin, and 1 others. 2024. Learning diverse attacks on large language models for robust red-teaming and safety tuning. arXiv preprint arXiv:2405.18540. — [arXiv:2405.18540](https://arxiv.org/abs/2405.18540) [find](https://scholar.google.com/scholar?q=Seanie+Lee%2C+Minsu+Kim%2C+Lynn+Cherif%2C+David+Dobre%2C+Juho+Lee%2C+Sung+Ju+Hwang%2C+Kenji+Kawaguchi%2C+Gauthier+Gidel%2C+Yoshua+Bengio%2C+Nikolay+Malkin%2C+and+1+others%2E+2024%2E+Learning+diverse+attacks+on+large+language)
20. Junxian Li, Beining Xu, Simin Chen, Jiatong Li, Jingdi Lei, Haodong Zhao, and Di Zhang. 2026. Iag: Input-aware backdoor attack on vlm-based visual grounding. Preprint, arXiv:2508.09456. — [arXiv:2508.09456](https://arxiv.org/abs/2508.09456) [find](https://scholar.google.com/scholar?q=Junxian+Li%2C+Beining+Xu%2C+Simin+Chen%2C+Jiatong+Li%2C+Jingdi+Lei%2C+Haodong+Zhao%2C+and+Di+Zhang%2E+2026%2E+Iag%3A+Input%2Daware+backdoor+attack+on+vlm%2Dbased+visual+grounding%2E+Preprint%2C+arXiv%3A2508%2E09456%2E+%E2%80%94+%5BarXiv%3A2508%2E)
21. Linbao Li, Yannan Liu, Daojing He, and Yu Li. 2025a. One model transfer to all: On robust jailbreak prompts generation against llms. arXiv preprint arXiv:2505.17598. — [arXiv:2505.17598](https://arxiv.org/abs/2505.17598) [find](https://scholar.google.com/scholar?q=Linbao+Li%2C+Yannan+Liu%2C+Daojing+He%2C+and+Yu+Li%2E+2025a%2E+One+model+transfer+to+all%3A+On+robust+jailbreak+prompts+generation+against+llms%2E+arXiv+preprint+arXiv%3A2505%2E17598%2E+%E2%80%94+%5BarXiv%3A2505%2E17598%5D%28https%3A%2F%2Farxiv)
22. Nathaniel Li, Ziwen Han, Ian Steneker, Willow Primack, Riley Goodside, Hugh Zhang, Zifan Wang, Cristina Menghini, and Summer Yue. 2024. Llm defenses are not robust to multi-turn human jailbreaks yet. Preprint, arXiv:2408.15221. — [arXiv:2408.15221](https://arxiv.org/abs/2408.15221) [find](https://scholar.google.com/scholar?q=Nathaniel+Li%2C+Ziwen+Han%2C+Ian+Steneker%2C+Willow+Primack%2C+Riley+Goodside%2C+Hugh+Zhang%2C+Zifan+Wang%2C+Cristina+Menghini%2C+and+Summer+Yue%2E+2024%2E+Llm+defenses+are+not+robust+to+multi%2Dturn+human+jailbreaks+yet%2E+)
23. Ouxiang Li, Yuan Wang, Xinting Hu, Houcheng Jiang, Tao Liang, Yanbin Hao, Guojun Ma, and Fuli Feng. 2025b. Speed: Scalable, precise, and efficient concept erasure for diffusion models. arXiv preprint arXiv:2503.07392. — [arXiv:2503.07392](https://arxiv.org/abs/2503.07392) [find](https://scholar.google.com/scholar?q=Ouxiang+Li%2C+Yuan+Wang%2C+Xinting+Hu%2C+Houcheng+Jiang%2C+Tao+Liang%2C+Yanbin+Hao%2C+Guojun+Ma%2C+and+Fuli+Feng%2E+2025b%2E+Speed%3A+Scalable%2C+precise%2C+and+efficient+concept+erasure+for+diffusion+models%2E+arXiv+preprint+)
24. Xiaogeng Liu, Peiran Li, Edward Suh, Yevgeniy Vorobeychik, Zhuoqing Mao, Somesh Jha, Patrick McDaniel, Huan Sun, Bo Li, and Chaowei Xiao. 2024. Autodan-turbo: A lifelong agent for strategy self-exploration to jailbreak llms. arXiv preprint arXiv:2410.05295. — [arXiv:2410.05295](https://arxiv.org/abs/2410.05295) [find](https://scholar.google.com/scholar?q=Xiaogeng+Liu%2C+Peiran+Li%2C+Edward+Suh%2C+Yevgeniy+Vorobeychik%2C+Zhuoqing+Mao%2C+Somesh+Jha%2C+Patrick+McDaniel%2C+Huan+Sun%2C+Bo+Li%2C+and+Chaowei+Xiao%2E+2024%2E+Autodan%2Dturbo%3A+A+lifelong+agent+for+strategy+self%2Dexplor)
25. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. [Roberta: A robustly optimized bert pretraining approach](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/). Preprint, arXiv:1907.11692.
26. Yue Liu, Shengfang Zhai, Mingzhe Du, Yulin Chen, Tri Cao, Hongcheng Gao, Cheng Wang, Xinfeng Li, Kun Wang, Junfeng Fang, Jiaheng Zhang, and Bryan Hooi. 2025. Guardreasoner-vl: Safeguarding vlms via reinforced reasoning. Preprint, arXiv:2505.11049. — [arXiv:2505.11049](https://arxiv.org/abs/2505.11049) [find](https://scholar.google.com/scholar?q=Yue+Liu%2C+Shengfang+Zhai%2C+Mingzhe+Du%2C+Yulin+Chen%2C+Tri+Cao%2C+Hongcheng+Gao%2C+Cheng+Wang%2C+Xinfeng+Li%2C+Kun+Wang%2C+Junfeng+Fang%2C+Jiaheng+Zhang%2C+and+Bryan+Hooi%2E+2025%2E+Guardreasoner%2Dvl%3A+Safeguarding+vlms+via+re)
27. Micha Livne, Zulfat Miftahutdinov, Elena Tutubalina, Maksim Kuznetsov, Daniil Polykovskiy, Annika Brundyn, Aastha Jhunjhunwala, Anthony Costa, Alex Aliper, Alán Aspuru-Guzik, and 1 others. 2024. nach0: multimodal natural and chemical languages foundation model. Chemical Science, 15(22):8380– 8389. [find](https://scholar.google.com/scholar?q=Micha+Livne%2C+Zulfat+Miftahutdinov%2C+Elena+Tutubalina%2C+Maksim+Kuznetsov%2C+Daniil+Polykovskiy%2C+Annika+Brundyn%2C+Aastha+Jhunjhunwala%2C+Anthony+Costa%2C+Alex+Aliper%2C+Al%C3%A1n+Aspuru%2DGuzik%2C+and+1+others%2E+2024%2E+nach0)
28. Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, David Forsyth, and Dan Hendrycks. 2024. [Harmbench: a standardized evaluation framework for automated red teaming and robust refusal](https://md.research.elamur.ai/free/en/papers/mazeika-2024-harmbench-standardized-evaluation-framework-automated/). In Proceedings of the 41st International Con…
29. Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. 2022. [Red teaming language models with language models](https://md.research.elamur.ai/free/en/papers/perez-2022-red-teaming-language-models-language/). In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3419–3448, Abu…
30. Alberto Purpura, Sahil Wadhwa, Jesse Zymet, Akshay Gupta, Andy Luo, Melissa Kazemi Rad, Swapnil Shinde, and Mohammad Shahed Sorower. 2025. Building safe genai applications: An end-to-end overview of red teaming for large language models. In Proceedings of the 5th Workshop on Trustworthy NLP (TrustNL… [find](https://scholar.google.com/scholar?q=Alberto+Purpura%2C+Sahil+Wadhwa%2C+Jesse+Zymet%2C+Akshay+Gupta%2C+Andy+Luo%2C+Melissa+Kazemi+Rad%2C+Swapnil+Shinde%2C+and+Mohammad+Shahed+Sorower%2E+2025%2E+Building+safe+genai+applications%3A+An+end%2Dto%2Dend+overview+of+r)
31. Salman Rahman, Liwei Jiang, James Shiffer, Genglin Liu, Sheriff Issaka, Md Rizwan Parvez, Hamid Palangi, Kai-Wei Chang, Yejin Choi, and Saadia Gabriel. 2025. X-teaming: Multi-turn jailbreaks and defenses with adaptive multi-agents. In Second Conference on Language Modeling. [find](https://scholar.google.com/scholar?q=Salman+Rahman%2C+Liwei+Jiang%2C+James+Shiffer%2C+Genglin+Liu%2C+Sheriff+Issaka%2C+Md+Rizwan+Parvez%2C+Hamid+Palangi%2C+Kai%2DWei+Chang%2C+Yejin+Choi%2C+and+Saadia+Gabriel%2E+2025%2E+X%2Dteaming%3A+Multi%2Dturn+jailbreaks+and+defen)
32. Qibing Ren, Hao Li, Dongrui Liu, Zhanxu Xie, Xiaoya Lu, Yu Qiao, Lei Sha, Junchi Yan, Lizhuang Ma, and Jing Shao. 2024. Derail yourself: Multi-turn llm jailbreak attack through self-discovered clues. [find](https://scholar.google.com/scholar?q=Qibing+Ren%2C+Hao+Li%2C+Dongrui+Liu%2C+Zhanxu+Xie%2C+Xiaoya+Lu%2C+Yu+Qiao%2C+Lei+Sha%2C+Junchi+Yan%2C+Lizhuang+Ma%2C+and+Jing+Shao%2E+2024%2E+Derail+yourself%3A+Multi%2Dturn+llm+jailbreak+attack+through+self%2Ddiscovered+clues%2E)
33. Mark Russinovich, Ahmed Salem, and Ronen Eldan. 2025. Great, now write an article about that: the crescendo multi-turn llm jailbreak attack. In Proceedings of the 34th USENIX Conference on Security Symposium, SEC ’25, USA. USENIX Association. [find](https://scholar.google.com/scholar?q=Mark+Russinovich%2C+Ahmed+Salem%2C+and+Ronen+Eldan%2E+2025%2E+Great%2C+now+write+an+article+about+that%3A+the+crescendo+multi%2Dturn+llm+jailbreak+attack%2E+In+Proceedings+of+the+34th+USENIX+Conference+on+Security+Sy)
34. Mikayel Samvelyan, Sharath Chandra Raparthy, Andrei Lupu, Eric Hambro, Aram H. Markosyan, Manish Bhatt, Yuning Mao, Minqi Jiang, Jack Parker-Holder, Jakob Foerster, Tim Rocktäschel, and Roberta Raileanu. 2025. Rainbow teaming: open-ended generation of diverse adversarial prompts. In Proceedings of t… [find](https://scholar.google.com/scholar?q=Mikayel+Samvelyan%2C+Sharath+Chandra+Raparthy%2C+Andrei+Lupu%2C+Eric+Hambro%2C+Aram+H%2E+Markosyan%2C+Manish+Bhatt%2C+Yuning+Mao%2C+Minqi+Jiang%2C+Jack+Parker%2DHolder%2C+Jakob+Foerster%2C+Tim+Rockt%C3%A4schel%2C+and+Roberta+Railea)
35. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). Preprint, arXiv:2402.03300.
36. Wentao Shi, Mengqi Yuan, Junkang Wu, Qifan Wang, and Fuli Feng. 2024. Direct multi-turn preference optimization for language agents. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 2312–2324, Miami, Florida, USA. Association for Computational Linguist… [find](https://scholar.google.com/scholar?q=Wentao+Shi%2C+Mengqi+Yuan%2C+Junkang+Wu%2C+Qifan+Wang%2C+and+Fuli+Feng%2E+2024%2E+Direct+multi%2Dturn+preference+optimization+for+language+agents%2E+In+Proceedings+of+the+2024+Conference+on+Empirical+Methods+in+Natur)
37. Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El-Kishky, Aidan McLaughlin, Aiden Low, AJ Ostrow, Akhila Ananthram, Akshay Nathan, Alan Luo, Alec Helyar, Aleksander Madry, Aleksandr Efremov, Aleksandra Spyra, Alex Baker-Whitcomb, Alex Beutel, Alex Karpenko, and 465 others. 2025… — [OpenAI GPT-5 System Card](https://md.research.elamur.ai/free/en/papers/singh-2025-openai-gpt-5-system-card/)
38. Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2020. [Mpnet: Masked and permuted pre-training for language understanding](https://md.research.elamur.ai/free/en/papers/song-2020-mpnet-masked-permuted-pre-training/). Preprint, arXiv:2004.09297.
39. Yifan Song, Da Yin, Xiang Yue, Jie Huang, Sujian Li, and Bill Yuchen Lin. 2024. [Trial and error: Exploration-based trajectory optimization of LLM agents](https://md.research.elamur.ai/free/en/papers/song-2024-trial-error-exploration-based-trajectory/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7584–7600, Bangkok, Thai…
40. Alexandra Souly, Qingyuan Lu, Dillon Bowen, Tu Trinh, Elvis Hsieh, Sana Pandey, Pieter Abbeel, Justin Svegliato, Scott Emmons, Olivia Watkins, and Sam Toyer. 2025. A strongreject for empty jailbreaks. In Proceedings of the 38th International Conference on Neural Information Processing Systems, NIPS… [find](https://scholar.google.com/scholar?q=Alexandra+Souly%2C+Qingyuan+Lu%2C+Dillon+Bowen%2C+Tu+Trinh%2C+Elvis+Hsieh%2C+Sana+Pandey%2C+Pieter+Abbeel%2C+Justin+Svegliato%2C+Scott+Emmons%2C+Olivia+Watkins%2C+and+Sam+Toyer%2E+2025%2E+A+strongreject+for+empty+jailbreaks%2E)
41. Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, Johan Ferret, Peter Liu, Pouya Tafti, Abe Friesen, Michelle Casbon, Sabela Ramos, Ravin Kumar, Charline Le Lan, Sammy Jerome, and 179… [find](https://scholar.google.com/scholar?q=Gemma+Team%2C+Morgane+Riviere%2C+Shreya+Pathak%2C+Pier+Giuseppe+Sessa%2C+Cassidy+Hardin%2C+Surya+Bhupatiraju%2C+L%C3%A9onard+Hussenot%2C+Thomas+Mesnard%2C+Bobak+Shahriari%2C+Alexandre+Ram%C3%A9%2C+Johan+Ferret%2C+Peter+Liu%2C+Pouya+Ta)
42. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, and 49 others… — [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/)
43. Cheng Wang, Yue Liu, Baolong Bi, Duzhen Zhang, Zhong-Zhi Li, Yingwei Ma, Yufei He, Shengju Yu, Xinfeng Li, Junfeng Fang, Jiaheng Zhang, and Bryan Hooi. 2025a. Safety in large reasoning models: A survey. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 3468–3482, Suzhou… [find](https://scholar.google.com/scholar?q=Cheng+Wang%2C+Yue+Liu%2C+Baolong+Bi%2C+Duzhen+Zhang%2C+Zhong%2DZhi+Li%2C+Yingwei+Ma%2C+Yufei+He%2C+Shengju+Yu%2C+Xinfeng+Li%2C+Junfeng+Fang%2C+Jiaheng+Zhang%2C+and+Bryan+Hooi%2E+2025a%2E+Safety+in+large+reasoning+models%3A+A+surve)
44. Fengxiang Wang, Ranjie Duan, Peng Xiao, Xiaojun Jia, Shiji Zhao, Cheng Wei, YueFeng Chen, Chongwen Wang, Jialing Tao, Hang Su, Jun Zhu, and Hui Xue. 2025b. Mrj-agent: An effective jailbreak agent for multi-round dialogue. Preprint, arXiv:2411.03814. — [arXiv:2411.03814](https://arxiv.org/abs/2411.03814) [find](https://scholar.google.com/scholar?q=Fengxiang+Wang%2C+Ranjie+Duan%2C+Peng+Xiao%2C+Xiaojun+Jia%2C+Shiji+Zhao%2C+Cheng+Wei%2C+YueFeng+Chen%2C+Chongwen+Wang%2C+Jialing+Tao%2C+Hang+Su%2C+Jun+Zhu%2C+and+Hui+Xue%2E+2025b%2E+Mrj%2Dagent%3A+An+effective+jailbreak+agent+for+)
45. Wenhui Wang, Hangbo Bao, Shaohan Huang, Li Dong, and Furu Wei. 2021. MiniLMv2: Multi-head self-attention relation distillation for compressing pre-trained transformers. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 2140–2151, Online. Association for Computation… [find](https://scholar.google.com/scholar?q=Wenhui+Wang%2C+Hangbo+Bao%2C+Shaohan+Huang%2C+Li+Dong%2C+and+Furu+Wei%2E+2021%2E+MiniLMv2%3A+Multi%2Dhead+self%2Dattention+relation+distillation+for+compressing+pre%2Dtrained+transformers%2E+In+Findings+of+the+Association+)
46. Xi Wang, Songlei Jian, Shasha Li, Xiaopeng Li, Bin Ji, Ma Jun, Xiaodong Liu, Jing Wang, Jianfeng Zhang, Jie Yu, Feilong Bao, and Wangbaosheng. 2025c. Stand on the shoulders of giants: Building JailExpert from previous attack experience. In Proceedings of the 2025 Conference on Empirical Methods in N… [find](https://scholar.google.com/scholar?q=Xi+Wang%2C+Songlei+Jian%2C+Shasha+Li%2C+Xiaopeng+Li%2C+Bin+Ji%2C+Ma+Jun%2C+Xiaodong+Liu%2C+Jing+Wang%2C+Jianfeng+Zhang%2C+Jie+Yu%2C+Feilong+Bao%2C+and+Wangbaosheng%2E+2025c%2E+Stand+on+the+shoulders+of+giants%3A+Building+JailExp)
47. Xi Wang, Songlei Jian, Shasha Li, Xiaopeng Li, Zhaoye Li, Bin Ji, Baosheng Wang, and Jie Yu. 2026. Jpu: Bridging jailbreak defense and unlearning via on-policy path rectification. Preprint, arXiv:2601.03005. [find](https://scholar.google.com/scholar?q=Xi+Wang%2C+Songlei+Jian%2C+Shasha+Li%2C+Xiaopeng+Li%2C+Zhaoye+Li%2C+Bin+Ji%2C+Baosheng+Wang%2C+and+Jie+Yu%2E+2026%2E+Jpu%3A+Bridging+jailbreak+defense+and+unlearning+via+on%2Dpolicy+path+rectification%2E+Preprint%2C+arXiv%3A2601)
48. Zihan Wang, Kangrui Wang, Qineng Wang, Pingyue Zhang, Linjie Li, Zhengyuan Yang, Xing Jin, Kefan Yu, Minh Nhat Nguyen, Licheng Liu, Eli Gottlieb, Yiping Lu, Kyunghyun Cho, Jiajun Wu, Li Fei-Fei, Lijuan Wang, Yejin Choi, and Manling Li. 2025d. Ragen: Understanding self-evolution in llm agents via mul… [find](https://scholar.google.com/scholar?q=Zihan+Wang%2C+Kangrui+Wang%2C+Qineng+Wang%2C+Pingyue+Zhang%2C+Linjie+Li%2C+Zhengyuan+Yang%2C+Xing+Jin%2C+Kefan+Yu%2C+Minh+Nhat+Nguyen%2C+Licheng+Liu%2C+Eli+Gottlieb%2C+Yiping+Lu%2C+Kyunghyun+Cho%2C+Jiajun+Wu%2C+Li+Fei%2DFei%2C+Lijua)
49. Zixuan Weng, Xiaolong Jin, Jinyuan Jia, and Xiangyu Zhang. 2025. Foot-in-the-door: A multi-turn jailbreak for llms. Preprint, arXiv:2502.19820. [find](https://scholar.google.com/scholar?q=Zixuan+Weng%2C+Xiaolong+Jin%2C+Jinyuan+Jia%2C+and+Xiangyu+Zhang%2E+2025%2E+Foot%2Din%2Dthe%2Ddoor%3A+A+multi%2Dturn+jailbreak+for+llms%2E+Preprint%2C+arXiv%3A2502%2E19820%2E)
50. Lixiang Yan, Lele Sha, Linxuan Zhao, Yuheng Li, Roberto Martinez-Maldonado, Guanliang Chen, Xinyu Li, Yueqiao Jin, and Dragan Gašević. 2024. Practical and ethical challenges of large language models in education: A systematic scoping review. British Journal of Educational Technology, 55(1):90– 112. [find](https://scholar.google.com/scholar?q=Lixiang+Yan%2C+Lele+Sha%2C+Linxuan+Zhao%2C+Yuheng+Li%2C+Roberto+Martinez%2DMaldonado%2C+Guanliang+Chen%2C+Xinyu+Li%2C+Yueqiao+Jin%2C+and+Dragan+Ga%C5%A1evic%CC%81%2E+2024%2E+Practical+and+ethical+challenges+of+large+language+models+)
51. Siyu Yan, Long Zeng, Xuecheng Wu, Chengcheng Han, Kongcheng Zhang, Chong Peng, Xuezhi Cao, Xunliang Cai, and Chenjuan Guo. 2025. MUSE: MCTS-driven red teaming framework for enhanced multi-turn dialogue safety in large language models. In Proceedings of the 2025 Conference on Empirical Methods in Nat… [find](https://scholar.google.com/scholar?q=Siyu+Yan%2C+Long+Zeng%2C+Xuecheng+Wu%2C+Chengcheng+Han%2C+Kongcheng+Zhang%2C+Chong+Peng%2C+Xuezhi+Cao%2C+Xunliang+Cai%2C+and+Chenjuan+Guo%2E+2025%2E+MUSE%3A+MCTS%2Ddriven+red+teaming+framework+for+enhanced+multi%2Dturn+dialogu)
52. Qwen: An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, and 23 others. 2025a. Qwen2.5 technical report. Preprint, ar… [find](https://scholar.google.com/scholar?q=Qwen%3A+An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Haoran+Wei%2C+Huan+Lin%2C+Jian+Yang%2C+Jianhong+Tu%2C+Jianwei+Zhang%2C+Jianxin+Yang%2C+Jiaxi+Yan)
53. Xikang Yang, Biyu Zhou, Xuehai Tang, Jizhong Han, and Songlin Hu. 2025b. Chain of attack: Hide your intention through multi-turn interrogation. In Findings of the Association for Computational Linguistics: ACL 2025, pages 9881–9901, Vienna, Austria. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Xikang+Yang%2C+Biyu+Zhou%2C+Xuehai+Tang%2C+Jizhong+Han%2C+and+Songlin+Hu%2E+2025b%2E+Chain+of+attack%3A+Hide+your+intention+through+multi%2Dturn+interrogation%2E+In+Findings+of+the+Association+for+Computational+Linguis)
54. Zonghao Ying, Deyue Zhang, Zonglei Jing, Yisong Xiao, Quanchen Zou, Aishan Liu, Siyuan Liang, Xiangzheng Zhang, Xianglong Liu, and Dacheng Tao. 2025. Reasoning-augmented conversation for multi-turn jailbreak attacks on large language models. Preprint, arXiv:2502.11054. [find](https://scholar.google.com/scholar?q=Zonghao+Ying%2C+Deyue+Zhang%2C+Zonglei+Jing%2C+Yisong+Xiao%2C+Quanchen+Zou%2C+Aishan+Liu%2C+Siyuan+Liang%2C+Xiangzheng+Zhang%2C+Xianglong+Liu%2C+and+Dacheng+Tao%2E+2025%2E+Reasoning%2Daugmented+conversation+for+multi%2Dturn+ja)
55. Lifan Yuan, Wendi Li, Huayu Chen, Ganqu Cui, Ning Ding, Kaiyan Zhang, Bowen Zhou, Zhiyuan Liu, and Hao Peng. 2024a. Free process rewards without process labels. arXiv preprint arXiv:2412.01981. [find](https://scholar.google.com/scholar?q=Lifan+Yuan%2C+Wendi+Li%2C+Huayu+Chen%2C+Ganqu+Cui%2C+Ning+Ding%2C+Kaiyan+Zhang%2C+Bowen+Zhou%2C+Zhiyuan+Liu%2C+and+Hao+Peng%2E+2024a%2E+Free+process+rewards+without+process+labels%2E+arXiv+preprint+arXiv%3A2412%2E01981%2E)
56. Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen tse Huang, Pinjia He, Shuming Shi, and Zhaopeng Tu. 2024b. [Gpt-4 is too smart to be safe: Stealthy chat with llms via cipher](https://md.research.elamur.ai/free/en/papers/yuan-2023-gpt-4-too-smart-safe/). Preprint, arXiv:2308.06463.
57. Siliang Zeng, Quan Wei, William Brown, Oana Frunza, Yuriy Nevmyvaka, and Mingyi Hong. 2025. Reinforcing multi-turn reasoning in llm agents via turn-level credit assignment. Preprint, arXiv:2505.11821. — [arXiv:2505.11821](https://arxiv.org/abs/2505.11821) [find](https://scholar.google.com/scholar?q=Siliang+Zeng%2C+Quan+Wei%2C+William+Brown%2C+Oana+Frunza%2C+Yuriy+Nevmyvaka%2C+and+Mingyi+Hong%2E+2025%2E+Reinforcing+multi%2Dturn+reasoning+in+llm+agents+via+turn%2Dlevel+credit+assignment%2E+Preprint%2C+arXiv%3A2505%2E11821%2E)
58. Wenjun Zeng, Yuchi Liu, Ryan Mullins, Ludovic Peran, Joe Fernandez, Hamza Harkous, Karthik Narasimhan, Drew Proud, Piyush Kumar, Bhaktipriya Radharapu, Olivia Sturman, and Oscar Wahltinez. 2024a. Shieldgemma: Generative ai content moderation based on gemma. Preprint, arXiv:2407.21772. — [arXiv:2407.21772](https://arxiv.org/abs/2407.21772) [find](https://scholar.google.com/scholar?q=Wenjun+Zeng%2C+Yuchi+Liu%2C+Ryan+Mullins%2C+Ludovic+Peran%2C+Joe+Fernandez%2C+Hamza+Harkous%2C+Karthik+Narasimhan%2C+Drew+Proud%2C+Piyush+Kumar%2C+Bhaktipriya+Radharapu%2C+Olivia+Sturman%2C+and+Oscar+Wahltinez%2E+2024a%2E+Shie)
59. Yi Zeng, Hongpeng Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia, and Weiyan Shi. 2024b. [How johnny can persuade LLMs to jailbreak them: Rethinking persuasion to challenge AI safety by humanizing LLMs](https://md.research.elamur.ai/free/en/papers/zeng-2024-how-johnny-can-persuade-llms/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Lo…
60. Jinchuan Zhang, Yan Zhou, Yaxin Liu, Ziming Li, and Songlin Hu. 2024a. Holistic automated red teaming for large language models through top-down test case generation and multi-turn interaction. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 13711–137… [find](https://scholar.google.com/scholar?q=Jinchuan+Zhang%2C+Yan+Zhou%2C+Yaxin+Liu%2C+Ziming+Li%2C+and+Songlin+Hu%2E+2024a%2E+Holistic+automated+red+teaming+for+large+language+models+through+top%2Ddown+test+case+generation+and+multi%2Dturn+interaction%2E+In+Pro)
61. Yi Zhang, An Zhang, XiuYu Zhang, Leheng Sheng, Yuxin Chen, Zhenkai Liang, and Xiang Wang. 2025a. Alphaalign: Incentivizing safety alignment with extremely simplified reinforcement learning. CoRR, abs/2507.14987. — [arXiv:2507.14987](https://arxiv.org/abs/2507.14987) [find](https://scholar.google.com/scholar?q=Yi+Zhang%2C+An+Zhang%2C+XiuYu+Zhang%2C+Leheng+Sheng%2C+Yuxin+Chen%2C+Zhenkai+Liang%2C+and+Xiang+Wang%2E+2025a%2E+Alphaalign%3A+Incentivizing+safety+alignment+with+extremely+simplified+reinforcement+learning%2E+CoRR%2C+abs%2F)
62. Yichi Zhang, Siyuan Zhang, Yao Huang, Zeyu Xia, Zhengwei Fang, Xiao Yang, Ranjie Duan, Dong Yan, Yinpeng Dong, and Jun Zhu. 2025b. Stair: Improving safety alignment with introspective reasoning. Preprint, arXiv:2502.02384. — [arXiv:2502.02384](https://arxiv.org/abs/2502.02384) [find](https://scholar.google.com/scholar?q=Yichi+Zhang%2C+Siyuan+Zhang%2C+Yao+Huang%2C+Zeyu+Xia%2C+Zhengwei+Fang%2C+Xiao+Yang%2C+Ranjie+Duan%2C+Dong+Yan%2C+Yinpeng+Dong%2C+and+Jun+Zhu%2E+2025b%2E+Stair%3A+Improving+safety+alignment+with+introspective+reasoning%2E+Prepr)
63. Yiming Zhang, Jianfeng Chi, Hailey Nguyen, Kartikeya Upasani, Daniel M. Bikel, Jason Weston, and Eric Michael Smith. 2024b. Backtracking improves generation safety. Preprint, arXiv:2409.14586. — [arXiv:2409.14586](https://arxiv.org/abs/2409.14586) [find](https://scholar.google.com/scholar?q=Yiming+Zhang%2C+Jianfeng+Chi%2C+Hailey+Nguyen%2C+Kartikeya+Upasani%2C+Daniel+M%2E+Bikel%2C+Jason+Weston%2C+and+Eric+Michael+Smith%2E+2024b%2E+Backtracking+improves+generation+safety%2E+Preprint%2C+arXiv%3A2409%2E14586%2E+%E2%80%94+%5BarXi)
64. Haiquan Zhao, Chenhan Yuan, Fei Huang, Xiaomeng Hu, Yichang Zhang, An Yang, Bowen Yu, Dayiheng Liu, Jingren Zhou, Junyang Lin, Baosong Yang, Chen Cheng, Jialong Tang, Jiandong Jiang, Jianwei Zhang, Jijie Xu, Ming Yan, Minmin Sun, Pei Zhang, and 24 others. 2025. [Qwen3guard technical report](https://md.research.elamur.ai/free/en/papers/zhao-2025-qwen3guard-technical-report/). Preprint,…
65. Yi Zhao and Youzhi Zhang. 2025. Siren: A learning-based multi-turn attack framework for simulating real-world human jailbreak behaviors. Preprint, arXiv:2501.14250. — [arXiv:2501.14250](https://arxiv.org/abs/2501.14250) [find](https://scholar.google.com/scholar?q=Yi+Zhao+and+Youzhi+Zhang%2E+2025%2E+Siren%3A+A+learning%2Dbased+multi%2Dturn+attack+framework+for+simulating+real%2Dworld+human+jailbreak+behaviors%2E+Preprint%2C+arXiv%3A2501%2E14250%2E+%E2%80%94+%5BarXiv%3A2501%2E14250%5D%28https%3A%2F%2Farxiv%2E)
66. Yifei Zhou, Andrea Zanette, Jiayi Pan, Sergey Levine, and Aviral Kumar. 2024. Archer: Training language model agents via hierarchical multi-turn rl. Preprint, arXiv:2402.19446. [find](https://scholar.google.com/scholar?q=Yifei+Zhou%2C+Andrea+Zanette%2C+Jiayi+Pan%2C+Sergey+Levine%2C+and+Aviral+Kumar%2E+2024%2E+Archer%3A+Training+language+model+agents+via+hierarchical+multi%2Dturn+rl%2E+Preprint%2C+arXiv%3A2402%2E19446%2E)
67. Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. 2023. [Universal and transferable adversarial attacks on aligned language models](https://md.research.elamur.ai/free/en/papers/zou-2023-universal-transferable-adversarial-attacks-aligned/). Preprint, arXiv:2307.15043.

*Updated 31 October 2026*