---
title: "Defending Against Alignment-Breaking Attacks via Robustly"
description: "The authors propose RA-LLM (Robustly Aligned LLM), a wrapper around an already aligned model… 49 references, cited by 61 papers here."
authors: ["Bochuan Cao", "Yuanpu Cao", "Lu Lin", "Jinghui Chen"]
year: 2024
venue: "Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2024.acl-long.568/"
doi: "10.18653/v1/2024.acl-long.568"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/cao-2024-defending-against-alignment-breaking-attacks/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/cao-2024-defending-against-alignment-breaking-attacks/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/3/) / Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM

# Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM

**Authors:** Bochuan Cao, Yuanpu Cao, Lu Lin and Jinghui Chen · **Year:** 2024 · **Venue:** [Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2024](https://md.research.elamur.ai/free/en/venues/acl/2024/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2024.acl-long.568/) · [PDF](https://aclanthology.org/2024.acl-long.568.pdf) · [DOI 10.18653/v1/2024.acl-long.568](https://doi.org/10.18653/v1/2024.acl-long.568) · [OpenAlex](https://openalex.org/W4402671676)

**Citations:** 46 citations (OpenAlex, 2026-10-14) · [61](https://md.research.elamur.ai/free/en/papers/cao-2024-defending-against-alignment-breaking-attacks/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Safety-aligned large language models can still be tricked into producing harmful output. Attackers attach jailbreak prompts to a malicious request, either written by hand (for example, role-play setups) or found automatically through optimization. The authors propose RA-LLM (Robustly Aligned LLM), a wrapper around an already aligned model that needs no retraining or fine-tuning. Its alignment check works like this:

- The defense deletes a random share of the input tokens many times, using Monte Carlo sampling.
- It asks the underlying model whether each shortened version is acceptable.
- A request is accepted only if most of these copies still pass.

This works because adversarial prompts tend to break under small perturbations, while benign requests usually survive token removal. The defense relies only on the model's own alignment, not a third-party harm classifier, so it covers more than harmfulness, such as privacy or ethics. The authors also give a theoretical argument for why the scheme works. On open-source LLMs, attack success rates fell from close to 100% to roughly 10% or below, for both optimized and handcrafted jailbreaks. This makes RA-LLM a cheap, practical safeguard to deploy.

## Key points
- **Low cost to adopt:** RA-LLM sits on top of an existing aligned model with no further training. It uses no external harmful-content detector, which avoids the false positives that keyword-sensitive LLM judges tend to produce.
- **Adaptive attack by repetition:** Repeating the adversarial prompt several times, so that random dropping cannot remove all of it, did not defeat the defense. Attack success stayed near 5% on Harmful Behaviors.
- **Adaptive attack against RA-LLM itself:** When TAP and AutoDAN targeted RA-LLM directly, they produced no working jailbreaks (0% success). GCG cannot be used because the defense is non-differentiable.
- **Adaptive attack via EoT:** An Expectation over Transformation variant of GCG was tested on Vicuna-7B, dropping 30% of tokens at each step. Its loss did not converge. Directly optimizing the check's pass probability would cost about 20× more per iteration.
- **Limitations:** Benign requests are slightly affected. On Guanaco-7B-chat-HF, the benign answering rate fell from 95.3% to 92.0%. Extreme cases, such as very long or very short adversarial prompts, were not evaluated.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Pennsylvania State University](https://md.research.elamur.ai/free/en/universities/pennsylvania-state-university/) — University, State College, United States · 22 works · 4,878 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 1,343 works · [🛡️ Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/) · 209 works

## Cite

Cite as: Bochuan Cao, Yuanpu Cao, Lu Lin and Jinghui Chen. 2024. Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{cao2024defending,
  title = {{Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM}},
  author = {Bochuan Cao, Yuanpu Cao, Lu Lin and Jinghui Chen},
  booktitle = {Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2024},
  doi = {10.18653/v1/2024.acl-long.568},
  url = {https://doi.org/10.18653/v1/2024.acl-long.568}
}
```

- [← previous Data Engineering for Scaling Language Models to 128K Context](https://md.research.elamur.ai/free/en/papers/fu-2024-data-engineering-scaling-language-models/)
- [next → GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solvers](https://md.research.elamur.ai/free/en/papers/li-2024-gsm-plus-comprehensive-benchmark-evaluating/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/3/)

## References

49 references, 28 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani Srivastava, and Kai-Wei Chang. 2018. [Generating natural language adversarial examples](https://md.research.elamur.ai/free/en/papers/alzantot-2018-generating-natural-language-adversarial-examples/). arXiv preprint arXiv:1804.07998.
2. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. 2022. [Constitutional ai: Harmlessness from ai feedback](https://md.research.elamur.ai/free/en/papers/bai-2022-constitutional-ai-harmlessness-ai-feedback/). arXiv preprint arXiv:2212.08073.
3. Yuanpu Cao, Bochuan Cao, and Jinghui Chen. 2023. Stealthy and persistent unalignment on large language models via backdoor injections. arXiv preprint arXiv:2312.00027. [find](https://scholar.google.com/scholar?q=Yuanpu+Cao%2C+Bochuan+Cao%2C+and+Jinghui+Chen%2E+2023%2E+Stealthy+and+persistent+unalignment+on+large+language+models+via+backdoor+injections%2E+arXiv+preprint+arXiv%3A2312%2E00027%2E)
4. Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. 2017. Deep reinforcement learning from human preferences. Advances in neural information processing systems, 30. [find](https://scholar.google.com/scholar?q=Paul+F+Christiano%2C+Jan+Leike%2C+Tom+Brown%2C+Miljan+Martic%2C+Shane+Legg%2C+and+Dario+Amodei%2E+2017%2E+Deep+reinforcement+learning+from+human+preferences%2E+Advances+in+neural+information+processing+systems%2C+30%2E)
5. Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. 2019. Certified adversarial robustness via randomized smoothing. In international conference on machine learning, pages 1310–1320. PMLR. [find](https://scholar.google.com/scholar?q=Jeremy+Cohen%2C+Elan+Rosenfeld%2C+and+Zico+Kolter%2E+2019%2E+Certified+adversarial+robustness+via+randomized+smoothing%2E+In+international+conference+on+machine+learning%2C+pages+1310%E2%80%931320%2E+PMLR%2E)
6. Xinshuai Dong, Anh Tuan Luu, Rongrong Ji, and Hong Liu. 2021. Towards robustness against natural language word substitutions. arXiv preprint arXiv:2107.13541. — [arXiv:2107.13541](https://arxiv.org/abs/2107.13541) [find](https://scholar.google.com/scholar?q=Xinshuai+Dong%2C+Anh+Tuan+Luu%2C+Rongrong+Ji%2C+and+Hong+Liu%2E+2021%2E+Towards+robustness+against+natural+language+word+substitutions%2E+arXiv+preprint+arXiv%3A2107%2E13541%2E+%E2%80%94+%5BarXiv%3A2107%2E13541%5D%28https%3A%2F%2Farxiv%2Eorg%2Fab)
7. Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. 2017. [Hotflip: White-box adversarial examples for text classification](https://md.research.elamur.ai/free/en/papers/ebrahimi-2018-hotflip-white-box-adversarial-examples/). arXiv preprint arXiv:1712.06751.
8. Ji Gao, Jack Lanchantin, Mary Lou Soffa, and Yanjun Qi. 2018. [Black-box generation of adversarial text sequences to evade deep learning classifiers](https://md.research.elamur.ai/free/en/papers/gao-2018-black-box-generation-adversarial-text/). In 2018 IEEE Security and Privacy Workshops (SPW), pages 50–56. IEEE.
9. Dongyoung Go, Tomasz Korbak, Germán Kruszewski, Jos Rozen, Nahyeon Ryu, and Marc Dymetman. 2023. Aligning language models with preferences through f-divergence minimization. arXiv preprint arXiv:2302.08215. [find](https://scholar.google.com/scholar?q=Dongyoung+Go%2C+Tomasz+Korbak%2C+Germ%C3%A1n+Kruszewski%2C+Jos+Rozen%2C+Nahyeon+Ryu%2C+and+Marc+Dymetman%2E+2023%2E+Aligning+language+models+with+preferences+through+f%2Ddivergence+minimization%2E+arXiv+preprint+arXiv%3A2302%2E)
10. Julian Hazell. 2023. Large language models can be used to effectively scale spear phishing campaigns. arXiv preprint arXiv:2305.06972. — [arXiv:2305.06972](https://arxiv.org/abs/2305.06972) [find](https://scholar.google.com/scholar?q=Julian+Hazell%2E+2023%2E+Large+language+models+can+be+used+to+effectively+scale+spear+phishing+campaigns%2E+arXiv+preprint+arXiv%3A2305%2E06972%2E+%E2%80%94+%5BarXiv%3A2305%2E06972%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2305%2E06972%29)
11. Alec Helbling, Mansi Phute, Matthew Hull, and Duen Horng Chau. 2023. Llm self defense: By self examination, llms know they are being tricked. arXiv preprint arXiv:2308.07308. [find](https://scholar.google.com/scholar?q=Alec+Helbling%2C+Mansi+Phute%2C+Matthew+Hull%2C+and+Duen+Horng+Chau%2E+2023%2E+Llm+self+defense%3A+By+self+examination%2C+llms+know+they+are+being+tricked%2E+arXiv+preprint+arXiv%3A2308%2E07308%2E)
12. Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, and Tom Goldstein. 2023. [Baseline defenses for adversarial attacks against aligned language models](https://md.research.elamur.ai/free/en/papers/jain-2023-baseline-defenses-adversarial-attacks-against/).
13. Robin Jia, Aditi Raghunathan, Kerem Göksel, and Percy Liang. 2019. [Certified robustness to adversarial word substitutions](https://md.research.elamur.ai/free/en/papers/jia-2019-certified-robustness-adversarial-word-substitutions/). arXiv preprint arXiv:1909.00986.
14. Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. 2020. [Is bert really robust? a strong baseline for natural language attack on text classification and entailment](https://md.research.elamur.ai/free/en/papers/jin-2019-bert-really-robust-strong-baseline/). In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 8018–8025.
15. Daniel Kang, Xuechen Li, Ion Stoica, Carlos Guestrin, Matei Zaharia, and Tatsunori Hashimoto. 2023. Exploiting programmatic behavior of llms: Dual-use through standard security attacks. arXiv preprint arXiv:2302.05733. — [arXiv:2302.05733](https://arxiv.org/abs/2302.05733) [find](https://scholar.google.com/scholar?q=Daniel+Kang%2C+Xuechen+Li%2C+Ion+Stoica%2C+Carlos+Guestrin%2C+Matei+Zaharia%2C+and+Tatsunori+Hashimoto%2E+2023%2E+Exploiting+programmatic+behavior+of+llms%3A+Dual%2Duse+through+standard+security+attacks%2E+arXiv+preprint)
16. Nitish Shirish Keskar, Bryan McCann, Lav R Varshney, Caiming Xiong, and Richard Socher. 2019. [Ctrl: A conditional transformer language model for controllable generation](https://md.research.elamur.ai/free/en/papers/keskar-2019-ctrl-conditional-transformer-language-model/). arXiv preprint arXiv:1909.05858.
17. Tomasz Korbak, Kejian Shi, Angelica Chen, Rasika Vinayak Bhalerao, Christopher Buckley, Jason Phang, Samuel R Bowman, and Ethan Perez. 2023. [Pretraining language models with human preferences](https://md.research.elamur.ai/free/en/papers/korbak-2023-pretraining-language-models-human-preferences/). In International Conference on Machine Learning, pages 17506–17533. PMLR.
18. Aounon Kumar, Chirag Agarwal, Suraj Srinivas, Soheil Feizi, and Hima Lakkaraju. 2023. [Certifying llm safety against adversarial prompting](https://md.research.elamur.ai/free/en/papers/kumar-2023-certifying-llm-safety-against-adversarial/). arXiv preprint arXiv:2309.02705.
19. Haoran Li, Dadi Guo, Wei Fan, Mingshi Xu, and Yangqiu Song. 2023. [Multi-step jailbreaking privacy attacks on chatgpt](https://md.research.elamur.ai/free/en/papers/li-2023-multi-step-jailbreaking-privacy-attacks/). arXiv preprint arXiv:2304.05197.
20. Jinfeng Li, Shouling Ji, Tianyu Du, Bo Li, and Ting Wang. 2018. [Textbugger: Generating adversarial text against real-world applications](https://md.research.elamur.ai/free/en/papers/li-2018-textbugger-generating-adversarial-text-against/). arXiv preprint arXiv:1812.05271.
21. Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. 2023. [Autodan: Generating stealthy jailbreak prompts on aligned large language models](https://md.research.elamur.ai/free/en/papers/liu-2023-autodan-generating-stealthy-jailbreak-prompts/). arXiv preprint arXiv:2310.04451.
22. Rishabh Maheshwary, Saket Maheshwary, and Vikram Pudi. 2021. Generating natural language attacks in a hard label black box setting. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 13525–13533. [find](https://scholar.google.com/scholar?q=Rishabh+Maheshwary%2C+Saket+Maheshwary%2C+and+Vikram+Pudi%2E+2021%2E+Generating+natural+language+attacks+in+a+hard+label+black+box+setting%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+vo)
23. Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum Anderson, Yaron Singer, and Amin Karbasi. 2023. [Tree of attacks: Jailbreaking black-box llms automatically](https://md.research.elamur.ai/free/en/papers/mehrotra-2023-tree-attacks-jailbreaking-black-box/). arXiv preprint arXiv:2312.02119.
24. Takeru Miyato, Andrew M Dai, and Ian Goodfellow. 2016. [Adversarial training methods for semi-supervised text classification](https://md.research.elamur.ai/free/en/papers/miyato-2016-adversarial-training-methods-semi-supervised/). arXiv preprint arXiv:1605.07725.
25. Ha-Thanh Nguyen. 2023. A brief report on lawgpt 1.0: A virtual legal assistant based on gpt-3. arXiv preprint arXiv:2302.05729. [find](https://scholar.google.com/scholar?q=Ha%2DThanh+Nguyen%2E+2023%2E+A+brief+report+on+lawgpt+1%2E0%3A+A+virtual+legal+assistant+based+on+gpt%2D3%2E+arXiv+preprint+arXiv%3A2302%2E05729%2E)
26. Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. [Ms marco: A human-generated machine reading comprehension dataset](https://md.research.elamur.ai/free/en/papers/bajaj-2016-ms-marco-human-generated/).
27. OpenAI. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/).
28. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in Neural Information Processing Systems, 35:27730–27744.
29. Shuhuai Ren, Yihe Deng, Kun He, and Wanxiang Che. 2019. [Generating natural language adversarial examples through probability weighted word saliency](https://md.research.elamur.ai/free/en/papers/ren-2019-generating-natural-language-adversarial-examples/). In Proceedings of the 57th annual meeting of the association for computational linguistics, pages 1085– 1097.
30. Paul Röttger, Hannah Rose Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy. 2023. [Xstest: A test suite for identifying exaggerated safety behaviours in large language models](https://md.research.elamur.ai/free/en/papers/rottger-2023-xstest-test-suite-identifying-exaggerated/). arXiv preprint arXiv:2308.01263.
31. Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. 2023. [" do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models](https://md.research.elamur.ai/free/en/papers/shen-2023-anything-now-characterizing-evaluating-wild/). arXiv preprint arXiv:2308.03825.
32. Zhouxing Shi, Huan Zhang, Kai-Wei Chang, Minlie Huang, and Cho-Jui Hsieh. 2020. Robustness verification for transformers. arXiv preprint arXiv:2002.06622. — [arXiv:2002.06622](https://arxiv.org/abs/2002.06622) [find](https://scholar.google.com/scholar?q=Zhouxing+Shi%2C+Huan+Zhang%2C+Kai%2DWei+Chang%2C+Minlie+Huang%2C+and+Cho%2DJui+Hsieh%2E+2020%2E+Robustness+verification+for+transformers%2E+arXiv+preprint+arXiv%3A2002%2E06622%2E+%E2%80%94+%5BarXiv%3A2002%2E06622%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F20)
33. Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. 2020. Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 33:3008– 3021. [find](https://scholar.google.com/scholar?q=Nisan+Stiennon%2C+Long+Ouyang%2C+Jeffrey+Wu%2C+Daniel+Ziegler%2C+Ryan+Lowe%2C+Chelsea+Voss%2C+Alec+Radford%2C+Dario+Amodei%2C+and+Paul+F+Christiano%2E+2020%2E+Learning+to+summarize+with+human+feedback%2E+Advances+in+Neural)
34. Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting. 2023. Large language models in medicine. Nature medicine, pages 1–11. [find](https://scholar.google.com/scholar?q=Arun+James+Thirunavukarasu%2C+Darren+Shu+Jeng+Ting%2C+Kabilan+Elangovan%2C+Laura+Gutierrez%2C+Ting+Fang+Tan%2C+and+Daniel+Shu+Wei+Ting%2E+2023%2E+Large+language+models+in+medicine%2E+Nature+medicine%2C+pages+1%E2%80%9311%2E)
35. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288. [find](https://scholar.google.com/scholar?q=Hugo+Touvron%2C+Louis+Martin%2C+Kevin+Stone%2C+Peter+Albert%2C+Amjad+Almahairi%2C+Yasmine+Babaei%2C+Nikolay+Bashlykov%2C+Soumya+Batra%2C+Prajjwal+Bhargava%2C+Shruti+Bhosale%2C+et+al%2E+2023%2E+Llama+2%3A+Open+foundation+and+fi)
36. Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2023a. [Jailbroken: How does llm safety training fail?](https://md.research.elamur.ai/free/en/papers/wei-2023-jailbroken-llm-safety-training-fail/) arXiv preprint arXiv:2307.02483.
37. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). Advances in Neural Information Processing Systems, 35:24824–24837.
38. Zeming Wei, Yifei Wang, and Yisen Wang. 2023b. [Jailbreak and guard aligned language models with only few in-context demonstrations](https://md.research.elamur.ai/free/en/papers/wei-2023-jailbreak-guard-aligned-language-models/). arXiv preprint arXiv:2310.06387.
39. Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann. 2023. [Bloomberggpt: A large language model for finance](https://md.research.elamur.ai/free/en/papers/wu-2023-bloomberggpt-large-language-model-finance/).
40. Han Xu, Yao Ma, Hao-Chen Liu, Debayan Deb, Hui Liu, Ji-Liang Tang, and Anil K Jain. 2020a. Adversarial attacks and defenses in images, graphs and text: A review. International Journal of Automation and Computing, 17:151–178. [find](https://scholar.google.com/scholar?q=Han+Xu%2C+Yao+Ma%2C+Hao%2DChen+Liu%2C+Debayan+Deb%2C+Hui+Liu%2C+Ji%2DLiang+Tang%2C+and+Anil+K+Jain%2E+2020a%2E+Adversarial+attacks+and+defenses+in+images%2C+graphs+and+text%3A+A+review%2E+International+Journal+of+Automation+an)
41. Jing Xu, Da Ju, Margaret Li, Y-Lan Boureau, Jason Weston, and Emily Dinan. 2020b. Recipes for safety in open-domain chatbots. arXiv preprint arXiv:2010.07079. [find](https://scholar.google.com/scholar?q=Jing+Xu%2C+Da+Ju%2C+Margaret+Li%2C+Y%2DLan+Boureau%2C+Jason+Weston%2C+and+Emily+Dinan%2E+2020b%2E+Recipes+for+safety+in+open%2Ddomain+chatbots%2E+arXiv+preprint+arXiv%3A2010%2E07079%2E)
42. Kaidi Xu, Zhouxing Shi, Huan Zhang, Yihan Wang, Kai-Wei Chang, Minlie Huang, Bhavya Kailkhura, Xue Lin, and Cho-Jui Hsieh. 2020c. Automatic perturbation analysis for scalable certified robustness and beyond. Advances in Neural Information Processing Systems, 33:1129–1141. [find](https://scholar.google.com/scholar?q=Kaidi+Xu%2C+Zhouxing+Shi%2C+Huan+Zhang%2C+Yihan+Wang%2C+Kai%2DWei+Chang%2C+Minlie+Huang%2C+Bhavya+Kailkhura%2C+Xue+Lin%2C+and+Cho%2DJui+Hsieh%2E+2020c%2E+Automatic+perturbation+analysis+for+scalable+certified+robustness+and+)
43. Muchao Ye, Jinghui Chen, Chenglin Miao, Han Liu, Ting Wang, and Fenglong Ma. 2023. Pat: Geometry-aware hard-label black-box adversarial attacks on text. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 3093–3104. [find](https://scholar.google.com/scholar?q=Muchao+Ye%2C+Jinghui+Chen%2C+Chenglin+Miao%2C+Han+Liu%2C+Ting+Wang%2C+and+Fenglong+Ma%2E+2023%2E+Pat%3A+Geometry%2Daware+hard%2Dlabel+black%2Dbox+adversarial+attacks+on+text%2E+In+Proceedings+of+the+29th+ACM+SIGKDD+Conferenc)
44. Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Pinjia He, Shuming Shi, and Zhaopeng Tu. 2023. [Gpt-4 is too smart to be safe: Stealthy chat with llms via cipher](https://md.research.elamur.ai/free/en/papers/yuan-2023-gpt-4-too-smart-safe/). arXiv preprint arXiv:2308.06463.
45. Jiehang Zeng, Jianhan Xu, Xiaoqing Zheng, and Xuanjing Huang. 2023. Certified robustness to text adversarial attacks by randomized [mask]. Computational Linguistics, 49(2):395–427. [find](https://scholar.google.com/scholar?q=Jiehang+Zeng%2C+Jianhan+Xu%2C+Xiaoqing+Zheng%2C+and+Xuanjing+Huang%2E+2023%2E+Certified+robustness+to+text+adversarial+attacks+by+randomized+%5Bmask%5D%2E+Computational+Linguistics%2C+49%282%29%3A395%E2%80%93427%2E)
46. Yi Zhou, Xiaoqing Zheng, Cho-Jui Hsieh, Kai-Wei Chang, and Xuanjing Huan. 2021. Defense against synonym substitution-based adversarial attacks via dirichlet neighborhood ensemble. In Association for Computational Linguistics (ACL). [find](https://scholar.google.com/scholar?q=Yi+Zhou%2C+Xiaoqing+Zheng%2C+Cho%2DJui+Hsieh%2C+Kai%2DWei+Chang%2C+and+Xuanjing+Huan%2E+2021%2E+Defense+against+synonym+substitution%2Dbased+adversarial+attacks+via+dirichlet+neighborhood+ensemble%2E+In+Association+for+C)
47. Chen Zhu, Yu Cheng, Zhe Gan, Siqi Sun, Tom Goldstein, and Jingjing Liu. 2019. [Freelb: Enhanced adversarial training for natural language understanding](https://md.research.elamur.ai/free/en/papers/zhu-2019-freelb-enhanced-adversarial-training-natural/). arXiv preprint arXiv:1909.11764.
48. Sicheng Zhu, Ruiyi Zhang, Bang An, Gang Wu, Joe Barrow, Zichao Wang, Furong Huang, Ani Nenkova, and Tong Sun. 2023. Autodan: Automatic and interpretable adversarial attacks on large language models. arXiv preprint arXiv:2310.15140. [find](https://scholar.google.com/scholar?q=Sicheng+Zhu%2C+Ruiyi+Zhang%2C+Bang+An%2C+Gang+Wu%2C+Joe+Barrow%2C+Zichao+Wang%2C+Furong+Huang%2C+Ani+Nenkova%2C+and+Tong+Sun%2E+2023%2E+Autodan%3A+Automatic+and+interpretable+adversarial+attacks+on+large+language+models%2E+a)
49. Andy Zou, Zifan Wang, J Zico Kolter, and Matt Fredrikson. 2023. [Universal and transferable adversarial attacks on aligned language models](https://md.research.elamur.ai/free/en/papers/zou-2023-universal-transferable-adversarial-attacks-aligned/). arXiv preprint arXiv:2307.15043.

*Updated 24 October 2026*