---
title: "Automated Progressive Red Teaming — Jiang 2025"
description: "This COLING 2025 paper presents APRT (Automated Progressive Red Teaming), which treats red… 33 references, cited by 9 papers here."
authors: ["Bojian Jiang", "Yi Jing", "Tong Wu", "Tianhao Shen", "Deyi Xiong", "Qing Yang"]
year: 2025
venue: "Proceedings of the 31st International Conference on Computational Linguistics"
source: "https://aclanthology.org/2025.coling-main.260/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/jiang-2025-automated-progressive-red-teaming/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/jiang-2025-automated-progressive-red-teaming/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/8/) / Automated Progressive Red Teaming

# Automated Progressive Red Teaming

**Authors:** Bojian Jiang, Yi Jing, [Tong Wu](https://md.research.elamur.ai/free/en/authors/tong-wu/), [Tianhao Shen](https://md.research.elamur.ai/free/en/authors/tianhao-shen/), [Deyi Xiong](https://md.research.elamur.ai/free/en/authors/deyi-xiong/) and [Qing Yang](https://md.research.elamur.ai/free/en/authors/qing-yang/) · **Year:** 2025 · **Venue:** [Proceedings of the 31st International Conference on Computational Linguistics](https://md.research.elamur.ai/free/en/venues/coling/) ([COLING 2025](https://md.research.elamur.ai/free/en/venues/coling/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.coling-main.260/) · [PDF](https://aclanthology.org/2025.coling-main.260.pdf) · [OpenAlex](https://openalex.org/W4400434186)

**Citations:** 1 citations (OpenAlex, 2026-10-14) · 9 in our library, of the 149,439 papers whose bibliographies we read

## Summary
Human red teaming of LLMs works but is slow and costly, and most automated attackers either stay frozen or never learn from the target model's feedback. This COLING 2025 paper presents APRT (Automated Progressive Red Teaming), which treats red teaming as a skill a model can learn over repeated rounds. An Intention Expanding LLM generates varied harmful seed requests. An Evil Maker discards duplicates and requests that already get safe replies. An Intention Hiding LLM then rewrites the remaining requests into disguised prompts that keep the original goal. Two Reward LLMs judge the Target LLM's answers, one for safety and one for helpfulness. An active-learning step picks the hard, successful cases to retrain the hiding model in the next round. The authors also introduce Attack Effectiveness Rate (AER), which counts answers that are both harmful and actually useful, and report that it agrees closely with human judgment. On AdvBench Harmful Behaviors, APRT beats strong baselines, and its attacks carry over from open models to commercial APIs. This suggests that trainable, feedback-driven attackers are a scalable way to audit model safety.

## Key points
- Human evaluation shows APRT obtains unsafe-but-helpful answers from 54% of attacks on Llama-3-8B-Instruct. Prompts that broke Llama-3 transfer to GPT-4o at 50% and to Claude-3.5 at 39%, with both closed models accessed via API.
- AER is proposed to address weaknesses of ASR (Attack Success Rate) and of judging with the GPT API.
- The main point of contrast is MART (Ge et al., 2023). MART adds every successful attack to its training data, whereas APRT's active-learning selection yields higher AER over training epochs.
- Ablations show that removing the Intention Expanding LLM lowers AER. Visualizations show that after 4 training rounds, successful attacks become both more numerous and more diverse.
- Code and seed data are released at github.com/tjunlp-lab/APRT. A stated limitation is that Reward LLMs judge less accurately as attack prompts become more sophisticated, and strong Reward LLMs are expensive to build.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Tianjin University](https://md.research.elamur.ai/free/en/universities/tianjin-university/) — University, Tianjin, China · 41 works · 1,128 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🛡️ Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/) · 517 works

## Cite

Cite as: Bojian Jiang, Yi Jing, Tong Wu, Tianhao Shen, Deyi Xiong and Qing Yang. 2025. Automated Progressive Red Teaming. Proceedings of the 31st International Conference on Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{jiang2025automated,
  title = {{Automated Progressive Red Teaming}},
  author = {Bojian Jiang, Yi Jing, Tong Wu, Tianhao Shen, Deyi Xiong and Qing Yang},
  booktitle = {Proceedings of the 31st International Conference on Computational Linguistics},
  year = {2025},
  url = {https://aclanthology.org/2025.coling-main.260/}
}
```

- [← previous Audio-centric Video Understanding Benchmark without Text Shortcut](https://md.research.elamur.ai/free/en/papers/yang-2025-audio-centric-video-understanding-benchmark/)
- [next → Beyond Dialogue: A Profile-Dialogue Alignment Framework Towards General Role-Playing Language Model](https://md.research.elamur.ai/free/en/papers/yu-2025-beyond-dialogue-profile-dialogue-alignment/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/8/)

## References

33 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Gabriel Alon and Michael Kamfonas. 2023. [Detecting language model attacks with perplexity](https://md.research.elamur.ai/free/en/papers/alon-2023-detecting-language-model-attacks-perplexity/). Preprint, arXiv:2308.14132.
2. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Sh… [find](https://scholar.google.com/scholar?q=Yuntao+Bai%2C+Andy+Jones%2C+Kamal+Ndousse%2C+Amanda+Askell%2C+Anna+Chen%2C+Nova+DasSarma%2C+Dawn+Drain%2C+Stanislav+Fort%2C+Deep+Ganguli%2C+Tom+Henighan%2C+Nicholas+Joseph%2C+Saurav+Kadavath%2C+Jackson+Kernion%2C+Tom+Conerly%2C+)
3. Rishabh Bhardwaj and Soujanya Poria. 2023. Red-teaming large language models using chain of utterances for safety-alignment. Preprint, arXiv:2308.09662. [find](https://scholar.google.com/scholar?q=Rishabh+Bhardwaj+and+Soujanya+Poria%2E+2023%2E+Red%2Dteaming+large+language+models+using+chain+of+utterances+for+safety%2Dalignment%2E+Preprint%2C+arXiv%3A2308%2E09662%2E)
4. Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, and Eric Wong. 2024. [Jailbreaking black box large language models in twenty queries](https://md.research.elamur.ai/free/en/papers/chao-2023-jailbreaking-black-box-large-language/). Preprint, arXiv:2310.08419.
5. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. 2023. Vicuna: An open-source chatbot impressing GPT-4 with 90%* ChatGPT quality. See [https://vicuna.lmsys](https://vicuna.lmsys). org (accessed 14 April 2023), 2(3):6. [↗](https://vicuna.lmsys)
6. Ganqu Cui, Lifan Yuan, Ning Ding, Guanming Yao, Wei Zhu, Yuan Ni, Guotong Xie, Zhiyuan Liu, and Maosong Sun. 2023. UltraFeedback: Boosting language models with high-quality feedback. Preprint, arXiv:2310.01377. [find](https://scholar.google.com/scholar?q=Ganqu+Cui%2C+Lifan+Yuan%2C+Ning+Ding%2C+Guanming+Yao%2C+Wei+Zhu%2C+Yuan+Ni%2C+Guotong+Xie%2C+Zhiyuan+Liu%2C+and+Maosong+Sun%2E+2023%2E+UltraFeedback%3A+Boosting+language+models+with+high%2Dquality+feedback%2E+Preprint%2C+arXiv%3A2)
7. Peng Ding, Jun Kuang, Dan Ma, Xuezhi Cao, Yunsen Xian, Jiajun Chen, and Shujian Huang. 2024. A Wolf in Sheep’s Clothing: Generalized nested jailbreak prompts can fool large language models easily. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computationa… [find](https://scholar.google.com/scholar?q=Peng+Ding%2C+Jun+Kuang%2C+Dan+Ma%2C+Xuezhi+Cao%2C+Yunsen+Xian%2C+Jiajun+Chen%2C+and+Shujian+Huang%2E+2024%2E+A+Wolf+in+Sheep%E2%80%99s+Clothing%3A+Generalized+nested+jailbreak+prompts+can+fool+large+language+models+easily%2E+In+)
8. Suyu Ge, Chunting Zhou, Rui Hou, Madian Khabsa, Yi-Chia Wang, Qifan Wang, Jiawei Han, and Yuning Mao. 2023. [MART: Improving LLM safety with multi-round automatic red-teaming](https://md.research.elamur.ai/free/en/papers/ge-2024-mart-improving-llm-safety-multi/). Preprint, arXiv:2311.07689.
9. Zhang-Wei Hong, Idan Shenfeld, Tsun-Hsuan Wang, Yung-Sung Chuang, Aldo Pareja, James Glass, Akash Srivastava, and Pulkit Agrawal. 2024. Curiosity-driven red-teaming for large language models. Preprint, arXiv:2402.19464. — [arXiv:2402.19464](https://arxiv.org/abs/2402.19464) [find](https://scholar.google.com/scholar?q=Zhang%2DWei+Hong%2C+Idan+Shenfeld%2C+Tsun%2DHsuan+Wang%2C+Yung%2DSung+Chuang%2C+Aldo+Pareja%2C+James+Glass%2C+Akash+Srivastava%2C+and+Pulkit+Agrawal%2E+2024%2E+Curiosity%2Ddriven+red%2Dteaming+for+large+language+models%2E+Preprint)
10. Jiaming Ji, Mickel Liu, Juntao Dai, Xuehai Pan, Chi Zhang, Ce Bian, Boyuan Chen, Ruiyang Sun, Yizhou Wang, and Yaodong Yang. 2024. [Beavertails: towards improved safety alignment of LLM via a human-preference dataset](https://md.research.elamur.ai/free/en/papers/ji-2023-beavertails-towards-improved-safety-alignment/). In Proceedings of the 37th International Conference on Neural Information Processin…
11. Erik Jones, Anca Dragan, Aditi Raghunathan, and Jacob Steinhardt. 2023. Automatically auditing large language models via discrete optimization. In International Conference on Machine Learning, pages 15307–15329. PMLR. [find](https://scholar.google.com/scholar?q=Erik+Jones%2C+Anca+Dragan%2C+Aditi+Raghunathan%2C+and+Jacob+Steinhardt%2E+2023%2E+Automatically+auditing+large+language+models+via+discrete+optimization%2E+In+International+Conference+on+Machine+Learning%2C+pages+1)
12. Daniel Kang, Xuechen Li, Ion Stoica, Carlos Guestrin, Matei Zaharia, and Tatsunori Hashimoto. 2024. Exploiting programmatic behavior of LLMs: Dual-use through standard security attacks. In 2024 IEEE Security and Privacy Workshops (SPW), pages 132– 143. [find](https://scholar.google.com/scholar?q=Daniel+Kang%2C+Xuechen+Li%2C+Ion+Stoica%2C+Carlos+Guestrin%2C+Matei+Zaharia%2C+and+Tatsunori+Hashimoto%2E+2024%2E+Exploiting+programmatic+behavior+of+LLMs%3A+Dual%2Duse+through+standard+security+attacks%2E+In+2024+IEEE+S)
13. Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. 2024. [AutoDAN: Generating stealthy jailbreak prompts on aligned large language models](https://md.research.elamur.ai/free/en/papers/liu-2023-autodan-generating-stealthy-jailbreak-prompts/). Preprint, arXiv:2310.04451.
14. Rodrigo Revilla Llaca, Victoria Leskoschek, Vitor Costa Paiva, Cătălin Lupău, Philip Lippmann, and Jie Yang. 2023. Student-teacher prompting for red teaming to improve guardrails. In Proceedings of the ART of Safety: Workshop on Adversarial testing and Red-Teaming for generative AI, pages 11–23. [find](https://scholar.google.com/scholar?q=Rodrigo+Revilla+Llaca%2C+Victoria+Leskoschek%2C+Vitor+Costa+Paiva%2C+Ca%CC%86ta%CC%86lin+Lupa%CC%86u%2C+Philip+Lippmann%2C+and+Jie+Yang%2E+2023%2E+Student%2Dteacher+prompting+for+red+teaming+to+improve+guardrails%2E+In+Proceedings+of)
15. AI @ Meta Llama Team. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). Preprint, arXiv:2407.21783.
16. Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, David Forsyth, and Dan Hendrycks. 2024. [HarmBench: A standardized evaluation framework for automated red teaming and robust refusal](https://md.research.elamur.ai/free/en/papers/mazeika-2024-harmbench-standardized-evaluation-framework-automated/). Preprint, arXiv:2402.04249.
17. Ninareh Mehrabi, Palash Goyal, Christophe Dupuy, Qian Hu, Shalini Ghosh, Richard Zemel, Kai-Wei Chang, Aram Galstyan, and Rahul Gupta. 2023. [FLIRT: Feedback loop in-context red teaming](https://md.research.elamur.ai/free/en/papers/mehrabi-2024-flirt-feedback-loop-context-red/). Preprint, arXiv:2308.04265.
18. Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum Anderson, Yaron Singer, and Amin Karbasi. 2023. [Tree of attacks: Jailbreaking black-box LLMs automatically](https://md.research.elamur.ai/free/en/papers/mehrotra-2023-tree-attacks-jailbreaking-black-box/). arXiv preprint arXiv:2312.02119.
19. Bhaktipriya Radharapu, Kevin Robinson, Lora Aroyo, and Preethi Lahoti. 2023. AART: AI-Assisted Red-Teaming with Diverse Data Generation for New LLM-powered Applications. Preprint, arXiv:2311.08592. [find](https://scholar.google.com/scholar?q=Bhaktipriya+Radharapu%2C+Kevin+Robinson%2C+Lora+Aroyo%2C+and+Preethi+Lahoti%2E+2023%2E+AART%3A+AI%2DAssisted+Red%2DTeaming+with+Diverse+Data+Generation+for+New+LLM%2Dpowered+Applications%2E+Preprint%2C+arXiv%3A2311%2E08592%2E)
20. Burr Settles. 2009. [Active learning literature survey](https://md.research.elamur.ai/free/en/papers/settles-2009-active-learning-literature-survey/). Technical report, University of Wisconsin-Madison Department of Computer Sciences.
21. Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace, and Sameer Singh. 2020. [AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts](https://md.research.elamur.ai/free/en/papers/shin-2020-autoprompt-eliciting-knowledge-language-models/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4222– 4235.
22. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,… — [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/)
23. Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. 2021. Universal adversarial triggers for attacking and analyzing nlp. Preprint, arXiv:1908.07125. [find](https://scholar.google.com/scholar?q=Eric+Wallace%2C+Shi+Feng%2C+Nikhil+Kandpal%2C+Matt+Gardner%2C+and+Sameer+Singh%2E+2021%2E+Universal+adversarial+triggers+for+attacking+and+analyzing+nlp%2E+Preprint%2C+arXiv%3A1908%2E07125%2E)
24. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318.
25. Yuxia Wang, Haonan Li, Xudong Han, Preslav Nakov, and Timothy Baldwin. 2024. [Do-Not-Answer: Evaluating safeguards in LLMs](https://md.research.elamur.ai/free/en/papers/wang-2024-do-not-answer-evaluating-safeguards/). In Findings of the Association for Computational Linguistics: EACL 2024, pages 896–911, St. Julian’s, Malta. Association for Computational Linguistics.
26. Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. 2022. [Red teaming language models with language models](https://md.research.elamur.ai/free/en/papers/perez-2022-red-teaming-language-models-language/). Preprint, arXiv:2202.03286.
27. Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2023. [Jailbroken: How does LLM safety training fail?](https://md.research.elamur.ai/free/en/papers/wei-2023-jailbroken-llm-safety-training-fail/) In Advances in Neural Information Processing Systems, volume 36, pages 80079–80110. Curran Associates, Inc.
28. Fangzhou Wu, Xiaogeng Liu, and Chaowei Xiao. 2023. DeceptPrompt: Exploiting LLM-driven code generation via adversarial natural language instructions. Preprint, arXiv:2312.04730. — [arXiv:2312.04730](https://arxiv.org/abs/2312.04730) [find](https://scholar.google.com/scholar?q=Fangzhou+Wu%2C+Xiaogeng+Liu%2C+and+Chaowei+Xiao%2E+2023%2E+DeceptPrompt%3A+Exploiting+LLM%2Ddriven+code+generation+via+adversarial+natural+language+instructions%2E+Preprint%2C+arXiv%3A2312%2E04730%2E+%E2%80%94+%5BarXiv%3A2312%2E04730%5D%28h)
29. Xinwei Wu, Weilong Dong, Shaoyang Xu, and Deyi Xiong. 2024. Mitigating privacy seesaw in large language models: Augmented privacy neuron editing via activation patching. In Findings of the Association for Computational Linguistics: ACL 2024, pages 5319–5332, Bangkok, Thailand. Association for Comput… [find](https://scholar.google.com/scholar?q=Xinwei+Wu%2C+Weilong+Dong%2C+Shaoyang+Xu%2C+and+Deyi+Xiong%2E+2024%2E+Mitigating+privacy+seesaw+in+large+language+models%3A+Augmented+privacy+neuron+editing+via+activation+patching%2E+In+Findings+of+the+Association)
30. Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen tse Huang, Pinjia He, Shuming Shi, and Zhaopeng Tu. 2024. [GPT-4 is too smart to be safe: Stealthy chat with LLMs via cipher](https://md.research.elamur.ai/free/en/papers/yuan-2023-gpt-4-too-smart-safe/). Preprint, arXiv:2308.06463.
31. Yi Zeng, Hongpeng Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia, and Weiyan Shi. 2024. [How johnny can persuade LLMs to jailbreak them: Rethinking persuasion to challenge ai safety by humanizing LLMs](https://md.research.elamur.ai/free/en/papers/zeng-2024-how-johnny-can-persuade-llms/). Preprint, arXiv:2401.06373.
32. Sicheng Zhu, Ruiyi Zhang, Bang An, Gang Wu, Joe Barrow, Zichao Wang, Furong Huang, Ani Nenkova, and Tong Sun. 2023. Autodan: Interpretable gradient-based adversarial attacks on large language models. In First Conference on Language Modeling. [find](https://scholar.google.com/scholar?q=Sicheng+Zhu%2C+Ruiyi+Zhang%2C+Bang+An%2C+Gang+Wu%2C+Joe+Barrow%2C+Zichao+Wang%2C+Furong+Huang%2C+Ani+Nenkova%2C+and+Tong+Sun%2E+2023%2E+Autodan%3A+Interpretable+gradient%2Dbased+adversarial+attacks+on+large+language+models%2E+)
33. Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. 2023. [Universal and transferable adversarial attacks on aligned language models](https://md.research.elamur.ai/free/en/papers/zou-2023-universal-transferable-adversarial-attacks-aligned/). Preprint, arXiv:2307.15043.

*Updated 30 October 2026*