---
title: "OR-Bench — paper by Cui et al. (2024) — summary and ideas"
description: "The authors build an automated pipeline to fill this gap. 83 references, cited by 64 papers here."
authors: ["Justin Cui", "Wei-Lin Chiang", "Ion Stoica", "Cho-Jui Hsieh"]
year: 2024
venue: "arXiv:2405.20947"
source: "https://arxiv.org/abs/2405.20947"
doi: "10.48550/arxiv.2405.20947"
arxiv: "2405.20947"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/cui-2024-bench-over-refusal-benchmark-large/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/cui-2024-bench-over-refusal-benchmark-large/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/4/) / OR-Bench: An Over-Refusal Benchmark for Large Language Models

# OR-Bench: An Over-Refusal Benchmark for Large Language Models

**Authors:** Justin Cui, [Wei-Lin Chiang](https://md.research.elamur.ai/free/en/authors/wei-lin-chiang/), [Ion Stoica](https://md.research.elamur.ai/free/en/authors/ion-stoica/) and [Cho-Jui Hsieh](https://md.research.elamur.ai/free/en/authors/cho-jui-hsieh/) · **Year:** 2024 · **Venue:** arXiv:2405.20947 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2405.20947) · [PDF](https://arxiv.org/pdf/2405.20947) · [DOI 10.48550/arxiv.2405.20947](https://doi.org/10.48550/arxiv.2405.20947) · [arXiv 2405.20947](https://arxiv.org/abs/2405.20947) · [OpenAlex](https://openalex.org/W4399317891)

**Citations:** 9 citations (OpenAlex, 2026-10-14) · [64](https://md.research.elamur.ai/free/en/papers/cui-2024-bench-over-refusal-benchmark-large/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Safety-tuned LLMs often refuse harmless requests that merely look risky. Before this work, the problem had no large-scale way to be measured. The only dedicated test set, XSTest, has 250 hand-written prompts, and recent strong models now answer almost all of them. The authors build an automated pipeline to fill this gap. First, a less restricted model (Mixtral 8*7B) produces harmful seed prompts for each refusal category. Next, an LLM rewrites each seed into a benign but borderline version. Finally, an ensemble of LLM moderators removes any rewrites that are still toxic. The result is OR-Bench, which has three parts: a broad over-refusal set, a hard subset that even frontier models struggle with, and a toxic control set that penalizes models that answer everything. The authors test 32 LLMs and find that safety and over-refusal rise together: models that block more toxic requests also reject more benign ones. The benchmark gives the field a renewable, model-agnostic testbed for the trade-off between safety and helpfulness. The paper was published at ICML 2025.

## Key points
- **Resources:** OR-Bench-80K has 80,000 seemingly risky but safe prompts across 10 categories, such as violence, privacy, hate and sexual content. OR-Bench-Hard-1K has about 1,000 difficult prompts, and OR-Bench-Toxic has 600 harmful prompts. The data is on Hugging Face (bench-llms) and the code is on GitHub (justincui03/or-bench).
- **Evaluation scope:** The authors test 32 open-source and black-box LLMs from 8 families. On the hard and toxic sets, the Spearman rank correlation between toxic-prompt rejection and over-refusal is 0.89.
- **Model findings:**
  - Claude models are the safest but refuse the most benign prompts, while Mistral models accept most requests.
  - Newer GPT-3.5-turbo versions refuse less but are also less safe.
  - Larger models do not necessarily strike a better balance.
- **Defenses and system prompts:**
  - Jailbreak defenses such as ICL and SmoothLLM raise over-refusal; ICL rejects the most of both toxic and benign prompts.
  - A system prompt asking for safe, helpful answers made GPT-3.5-turbo-0125 reject about 35% more toxic prompts and about 55% more benign ones. For Mistral-large-latest, the figures were about 20% and 10%.
- **Robustness:** Sampling temperature barely changes the results. For example, Claude-3-Haiku's rejection rate on the hard subset moves only from 96.2 to 95.5 across temperatures 0.0–1.0.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 744 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 221 works

🧭 Topics (3): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 1,344 works · [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 685 works · [🛡️ Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/) · 209 works

## Cite

Cite as: Justin Cui, Wei-Lin Chiang, Ion Stoica and Cho-Jui Hsieh. 2024. OR-Bench: An Over-Refusal Benchmark for Large Language Models. arXiv:2405.20947.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{cui2024or,
  title = {{OR-Bench: An Over-Refusal Benchmark for Large Language Models}},
  author = {Justin Cui, Wei-Lin Chiang, Ion Stoica and Cho-Jui Hsieh},
  year = {2024},
  doi = {10.48550/arxiv.2405.20947},
  eprint = {2405.20947},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2405.20947}
}
```

- [← previous Iterative Translation Refinement with Large Language Models](https://md.research.elamur.ai/free/en/papers/chen-2024-iterative-translation-refinement-large-language/)
- [next → Agent Workflow Memory](https://md.research.elamur.ai/free/en/papers/wang-2024-agent-workflow-memory/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/4/)

## References

83 references, 41 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. GPT-4 AI is Great, But at a Hefty Price Tag ;) — community.openai.com. — [Jailbreaking Black Box Large Language Models in Twenty Queries](https://md.research.elamur.ai/free/en/papers/chao-2023-jailbreaking-black-box-large-language/) [https://community.openai.com](https://community.openai.com), a. Gpt-4-0125-preview INCREDIBLY slower than 3.5 turbo. https://community.openai.com/, b. Chao, P., Robey, A., Dobriban, E., Hassani, H., Pappas, G. J., and Wong, E. Jailbreaking black box large lan… [↗](https://community.openai.com)
2. Chiang, W.-L., Li, Z., Lin, Z., Sheng, Y., Wu, Z., Zhang, H., Zheng, L., Zhuang, S., Zhuang, Y., Gonzalez, J. E., Stoica, I., and Xing, E. P. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023. URL [https://lmsys.org/blog/](https://lmsys.org/blog/) 2023-03-30-vicuna/. [↗](https://lmsys.org/blog/)
3. Abdin, M., Jacobs, S. A., Awan, A. A., Aneja, J., Awadallah, A., Awadalla, H., Bach, N., Bahree, A., Bakhtiari, A., Behl, H., et al. [Phi-3 technical report: A highly capable language model locally on your phone](https://md.research.elamur.ai/free/en/papers/abdin-2024-phi-3-technical-report-highly/). arXiv preprint arXiv:2404.14219, 2024.
4. Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25(70):1–53, 2024. [find](https://scholar.google.com/scholar?q=Chung%2C+H%2E+W%2E%2C+Hou%2C+L%2E%2C+Longpre%2C+S%2E%2C+Zoph%2C+B%2E%2C+Tay%2C+Y%2E%2C+Fedus%2C+W%2E%2C+Li%2C+Y%2E%2C+Wang%2C+X%2E%2C+Dehghani%2C+M%2E%2C+Brahma%2C+S%2E%2C+et+al%2E+Scaling+instruction%2Dfinetuned+language+models%2E+Journal+of+Machine+Learning+Research)
5. An, B., Zhu, S., Zhang, R., Panaitescu-Liess, M.-A., Xu, Y., and Huang, F. Automatic pseudo-harmful prompt generation for evaluating false refusals in large language models. arXiv preprint arXiv:2409.00598, 2024. — [arXiv:2409.00598](https://arxiv.org/abs/2409.00598) [find](https://scholar.google.com/scholar?q=An%2C+B%2E%2C+Zhu%2C+S%2E%2C+Zhang%2C+R%2E%2C+Panaitescu%2DLiess%2C+M%2E%2DA%2E%2C+Xu%2C+Y%2E%2C+and+Huang%2C+F%2E+Automatic+pseudo%2Dharmful+prompt+generation+for+evaluating+false+refusals+in+large+language+models%2E+arXiv+preprint+arXiv%3A2409%2E)
6. Claude. Claude 2.1 Refuses to kill a Python process — Hacker News — news.ycombinator.com. [https://news.ycombinator.com/item?id=38371115](https://news.ycombinator.com/item?id=38371115), 2023. [Accessed 08-05-2024]. [↗](https://news.ycombinator.com/item?id=38371115)
7. Anthropic. Introducing the next generation of Claude — anthropic.com. [https://www.anthropic.com/](https://www.anthropic.com/) news/claude-3-family, 2024a. [Accessed 07-05-2024]. [↗](https://www.anthropic.com/)
8. Claude3. Introducing the next generation of claude \\ anthropic. [https://www.anthropic.com/news/](https://www.anthropic.com/news/) claude-3-family, 2024. [↗](https://www.anthropic.com/news/)
9. Anthropic. Claude 3 addendum.pdf. [https://www-cdn.anthropic.com/Model_Card_](https://www-cdn.anthropic.com/Model_Card_) Claude_3_Addendum.pdf, 2024b. [↗](https://www-cdn.anthropic.com/Model_Card_)
10. Dai, J., Pan, X., Sun, R., Ji, J., Xu, X., Liu, M., Wang, Y., and Yang, Y. [Safe rlhf: Safe reinforcement learning from human feedback](https://md.research.elamur.ai/free/en/papers/dai-2023-safe-rlhf-safe-reinforcement-learning/). arXiv preprint arXiv:2310.12773, 2023.
11. Anwar, U., Saparov, A., Rando, J., Paleka, D., Turpin, M., Hase, P., Lubana, E. S., Jenner, E., Casper, S., Sourbut, O., et al. Foundational challenges in assuring alignment and safety of large language models. arXiv preprint arXiv:2404.09932, 2024. [find](https://scholar.google.com/scholar?q=Anwar%2C+U%2E%2C+Saparov%2C+A%2E%2C+Rando%2C+J%2E%2C+Paleka%2C+D%2E%2C+Turpin%2C+M%2E%2C+Hase%2C+P%2E%2C+Lubana%2C+E%2E+S%2E%2C+Jenner%2C+E%2E%2C+Casper%2C+S%2E%2C+Sourbut%2C+O%2E%2C+et+al%2E+Foundational+challenges+in+assuring+alignment+and+safety+of+large+langua)
12. Davidson, T., Bhattacharya, D., and Weber, I. [Racial bias in hate speech and abusive language detection datasets](https://md.research.elamur.ai/free/en/papers/davidson-2019-racial-bias-hate-speech-abusive/). arXiv preprint arXiv:1905.12516, 2019.
13. Dixon, L., Li, J., Sorensen, J., Thain, N., and Vasserman, L. Measuring and mitigating unintended bias in text classification. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, pp. 67–73, 2018. [find](https://scholar.google.com/scholar?q=Dixon%2C+L%2E%2C+Li%2C+J%2E%2C+Sorensen%2C+J%2E%2C+Thain%2C+N%2E%2C+and+Vasserman%2C+L%2E+Measuring+and+mitigating+unintended+bias+in+text+classification%2E+In+Proceedings+of+the+2018+AAAI%2FACM+Conference+on+AI%2C+Ethics%2C+and+Society)
14. Jiang, L., Rao, K., Han, S., Ettinger, A., Brahman, F., Kumar, S., Mireshghallah, N., Lu, X., Sap, M., Choi, Y., et al. [Wildteaming at scale: From in-the-wild jailbreaks to (adversarially) safer language models](https://md.research.elamur.ai/free/en/papers/jiang-2024-wildteaming-scale-wild-jailbreaks-adversarially/). Advances in Neural Information Processing Systems, 37:47094–47165, 2024b.
15. Dong, Q., Li, L., Dai, D., Zheng, C., Wu, Z., Chang, B., Sun, X., Xu, J., and Sui, Z. [A survey on in-context learning](https://md.research.elamur.ai/free/en/papers/dong-2023-survey-context-learning/). arXiv preprint arXiv:2301.00234, 2022.
16. Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D. [Scaling laws for neural language models](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/). arXiv preprint arXiv:2001.08361, 2020.
17. Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., Fan, A., et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. [find](https://scholar.google.com/scholar?q=Dubey%2C+A%2E%2C+Jauhri%2C+A%2E%2C+Pandey%2C+A%2E%2C+Kadian%2C+A%2E%2C+Al%2DDahle%2C+A%2E%2C+Letman%2C+A%2E%2C+Mathur%2C+A%2E%2C+Schelten%2C+A%2E%2C+Yang%2C+A%2E%2C+Fan%2C+A%2E%2C+et+al%2E+The+llama+3+herd+of+models%2E+arXiv+preprint+arXiv%3A2407%2E21783%2C+2024%2E)
18. Kim, H., Yu, Y., Jiang, L., Lu, X., Khashabi, D., Kim, G., Choi, Y., and Sap, M. [Prosocialdialog: A prosocial backbone for conversational agents](https://md.research.elamur.ai/free/en/papers/kim-2022-prosocialdialog-prosocial-backbone-conversational-agents/). arXiv preprint arXiv:2205.12688, 2022.
19. Feuer, B., Goldblum, M., Datta, T., Nambiar, S., Besaleli, R., Dooley, S., Cembalest, M., and Dickerson, J. P. Style over substance: Failure modes of llm judges in alignment benchmarking. arXiv preprint arXiv:2409.15268, 2024. — [arXiv:2409.15268](https://arxiv.org/abs/2409.15268) [find](https://scholar.google.com/scholar?q=Feuer%2C+B%2E%2C+Goldblum%2C+M%2E%2C+Datta%2C+T%2E%2C+Nambiar%2C+S%2E%2C+Besaleli%2C+R%2E%2C+Dooley%2C+S%2E%2C+Cembalest%2C+M%2E%2C+and+Dickerson%2C+J%2E+P%2E+Style+over+substance%3A+Failure+modes+of+llm+judges+in+alignment+benchmarking%2E+arXiv+prepri)
20. Koike, R., Kaneko, M., and Okazaki, N. Outfox: Llm-generated essay detection through in-context learning with adversarially generated examples. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 21258–21266, 2024. [find](https://scholar.google.com/scholar?q=Koike%2C+R%2E%2C+Kaneko%2C+M%2E%2C+and+Okazaki%2C+N%2E+Outfox%3A+Llm%2Dgenerated+essay+detection+through+in%2Dcontext+learning+with+adversarially+generated+examples%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Inte)
21. Ganguli, D., Lovitt, L., Kernion, J., Askell, A., Bai, Y., Kadavath, S., Mann, B., Perez, E., Schiefer, N., Ndousse, K., et al. [Red teaming language models to reduce harms: methods, scaling behaviors, and lessons learned](https://md.research.elamur.ai/free/en/papers/ganguli-2022-red-teaming-language-models-reduce/). arxiv, 2022. less. Refusal in LLMs is mediated by a single direction — LessWro…
22. Ge, S., Zhou, C., Hou, R., Khabsa, M., Wang, Y.-C., Wang, Q., Han, J., and Mao, Y. [Mart: Improving llm safety with multi-round automatic red-teaming](https://md.research.elamur.ai/free/en/papers/ge-2024-mart-improving-llm-safety-multi/). arXiv preprint arXiv:2311.07689, 2023.
23. Levy, S., Allaway, E., Subbiah, M., Chilton, L., Patton, D., McKeown, K., and Wang, W. Y. Safetext: A benchmark for exploring physical safety in language models. arXiv preprint arXiv:2210.10045, 2022. [find](https://scholar.google.com/scholar?q=Levy%2C+S%2E%2C+Allaway%2C+E%2E%2C+Subbiah%2C+M%2E%2C+Chilton%2C+L%2E%2C+Patton%2C+D%2E%2C+McKeown%2C+K%2E%2C+and+Wang%2C+W%2E+Y%2E+Safetext%3A+A+benchmark+for+exploring+physical+safety+in+language+models%2E+arXiv+preprint+arXiv%3A2210%2E10045%2C+2022%2E)
24. Han, S., Rao, K., Ettinger, A., Jiang, L., Lin, B. Y., Lambert, N., Choi, Y., and Dziri, N. [Wildguard: Open one-stop moderation tools for safety risks, jailbreaks, and refusals of llms](https://md.research.elamur.ai/free/en/papers/han-2024-wildguard-open-one-stop-moderation/). arXiv preprint arXiv:2406.18495, 2024.
25. Li, L., Dong, B., Wang, R., Hu, X., Zuo, W., Lin, D., Qiao, Y., and Shao, J. [Salad-bench: A hierarchical and comprehensive safety benchmark for large language models](https://md.research.elamur.ai/free/en/papers/li-2024-salad-bench-hierarchical-comprehensive-safety/). arXiv preprint arXiv:2402.05044, 2024a.
26. Hartvigsen, T., Gabriel, S., Palangi, H., Sap, M., Ray, D., and Kamar, E. [Toxigen: A large-scale machine-generated dataset for adversarial and implicit hate speech detection](https://md.research.elamur.ai/free/en/papers/hartvigsen-2022-toxigen-large-scale-machine-generated/). arXiv preprint arXiv:2203.09509, 2022.
27. Li, X., Wang, R., Cheng, M., Zhou, T., and Hsieh, C.-J. Drattack: Prompt decomposition and reconstruction makes powerful llm jailbreakers. arXiv preprint arXiv:2402.16914, 2024b. — [arXiv:2402.16914](https://arxiv.org/abs/2402.16914) [find](https://scholar.google.com/scholar?q=Li%2C+X%2E%2C+Wang%2C+R%2E%2C+Cheng%2C+M%2E%2C+Zhou%2C+T%2E%2C+and+Hsieh%2C+C%2E%2DJ%2E+Drattack%3A+Prompt+decomposition+and+reconstruction+makes+powerful+llm+jailbreakers%2E+arXiv+preprint+arXiv%3A2402%2E16914%2C+2024b%2E+%E2%80%94+%5BarXiv%3A2402%2E16914%5D%28)
28. Inan, H., Upasani, K., Chi, J., Rungta, R., Iyer, K., Mao, Y., Tontchev, M., Hu, Q., Fuller, B., Testuggine, D., et al. [Llama guard: Llm-based input-output safeguard for human-ai conversations](https://md.research.elamur.ai/free/en/papers/inan-2023-llama-guard-llm-based-input/). arXiv preprint arXiv:2312.06674, 2023.
29. Lin, Z., Wang, Z., Tong, Y., Wang, Y., Guo, Y., Wang, Y., and Shang, J. [Toxicchat: Unveiling hidden challenges of toxicity detection in real-world user-ai conversation](https://md.research.elamur.ai/free/en/papers/lin-2023-toxicchat-unveiling-hidden-challenges-toxicity/). arXiv preprint arXiv:2310.17389, 2023.
30. Liu, H., Li, C., Wu, Q., and Lee, Y. J. [Visual instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-visual-instruction-tuning/). Advances in neural information processing systems, 36, 2024.
31. Ji, J., Liu, M., Dai, J., Pan, X., Zhang, C., Bian, C., Chen, B., Sun, R., Wang, Y., and Yang, Y. [Beavertails: Towards improved safety alignment of llm via a human-preference dataset](https://md.research.elamur.ai/free/en/papers/ji-2023-beavertails-towards-improved-safety-alignment/). Advances in Neural Information Processing Systems, 36, 2024.
32. Llama3. Introducing meta llama 3: The most capable openly available llm to date. [https://ai.meta.com/blog/meta-llama-3/](https://ai.meta.com/blog/meta-llama-3/), 2024. [↗](https://ai.meta.com/blog/meta-llama-3/)
33. Jiang, A. Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D. S., Casas, D. d. l., Hanna, E. B., Bressand, F., et al. Mixtral of experts. arXiv preprint arXiv:2401.04088, 2024a. [find](https://scholar.google.com/scholar?q=Jiang%2C+A%2E+Q%2E%2C+Sablayrolles%2C+A%2E%2C+Roux%2C+A%2E%2C+Mensch%2C+A%2E%2C+Savary%2C+B%2E%2C+Bamford%2C+C%2E%2C+Chaplot%2C+D%2E+S%2E%2C+Casas%2C+D%2E+d%2E+l%2E%2C+Hanna%2C+E%2E+B%2E%2C+Bressand%2C+F%2E%2C+et+al%2E+Mixtral+of+experts%2E+arXiv+preprint+arXiv%3A2401%2E04088%2C+)
34. Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., et al. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). arXiv preprint arXiv:2005.14165, 2020.
35. McIntosh, T. R., Susnjak, T., Liu, T., Watters, P., and Halgamuge, M. N. Inadequacies of large language model benchmarks in the era of generative artificial intelligence. arXiv preprint arXiv:2402.09880, 2024. — [arXiv:2402.09880](https://arxiv.org/abs/2402.09880) [find](https://scholar.google.com/scholar?q=McIntosh%2C+T%2E+R%2E%2C+Susnjak%2C+T%2E%2C+Liu%2C+T%2E%2C+Watters%2C+P%2E%2C+and+Halgamuge%2C+M%2E+N%2E+Inadequacies+of+large+language+model+benchmarks+in+the+era+of+generative+artificial+intelligence%2E+arXiv+preprint+arXiv%3A2402%2E098)
36. Röttger, P., Kirk, H. R., Vidgen, B., Attanasio, G., Bianchi, F., and Hovy, D. [Xstest: A test suite for identifying exaggerated safety behaviours in large language models](https://md.research.elamur.ai/free/en/papers/rottger-2023-xstest-test-suite-identifying-exaggerated/). arXiv preprint arXiv:2308.01263, 2023. mistral. mistralai/Mistral-7B-Instruct-v0.3 · Hugging Face — huggingface.co. https://hug…
37. Samvelyan, M., Raparthy, S. C., Lupu, A., Hambro, E., Markosyan, A. H., Bhatt, M., Mao, Y., Jiang, M., Parker-Holder, J., Foerster, J., et al. Rainbow teaming: Open-ended generation of diverse adversarial prompts. arXiv preprint arXiv:2402.16822, 2024. — [arXiv:2402.16822](https://arxiv.org/abs/2402.16822) [find](https://scholar.google.com/scholar?q=Samvelyan%2C+M%2E%2C+Raparthy%2C+S%2E+C%2E%2C+Lupu%2C+A%2E%2C+Hambro%2C+E%2E%2C+Markosyan%2C+A%2E+H%2E%2C+Bhatt%2C+M%2E%2C+Mao%2C+Y%2E%2C+Jiang%2C+M%2E%2C+Parker%2DHolder%2C+J%2E%2C+Foerster%2C+J%2E%2C+et+al%2E+Rainbow+teaming%3A+Open%2Dended+generation+of+diverse+adversa)
38. Mistral. Mistral AI — Frontier AI in your hands — mistral.ai. [https://mistral.ai/](https://mistral.ai/), 2024. [Accessed 07-05-2024]. [↗](https://mistral.ai/)
39. Sap, M., Card, D., Gabriel, S., Choi, Y., and Smith, N. A. [The risk of racial bias in hate speech detection](https://md.research.elamur.ai/free/en/papers/sap-2019-risk-racial-bias-hate-speech/). In Proceedings of the 57th annual meeting of the association for computational linguistics, pp. 1668–1678, 2019.
40. OpenAI. Chatgpt. [https://www.openai.com](https://www.openai.com), 2023. Accessed: ¡date-of-access¿. [↗](https://www.openai.com)
41. Stiennon, N., Ouyang, L., Wu, J., Ziegler, D., Lowe, R., Voss, C., Radford, A., Amodei, D., and Christiano, P. F. Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 33: 3008–3021, 2020. [find](https://scholar.google.com/scholar?q=Stiennon%2C+N%2E%2C+Ouyang%2C+L%2E%2C+Wu%2C+J%2E%2C+Ziegler%2C+D%2E%2C+Lowe%2C+R%2E%2C+Voss%2C+C%2E%2C+Radford%2C+A%2E%2C+Amodei%2C+D%2E%2C+and+Christiano%2C+P%2E+F%2E+Learning+to+summarize+with+human+feedback%2E+Advances+in+Neural+Information+Processing+S)
42. OpenAI. Introducing the model spec — openai. 2024. [find](https://scholar.google.com/scholar?q=OpenAI%2E+Introducing+the+model+spec+%E2%80%94+openai%2E+2024%2E)
43. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in neural information processing systems, 35:27730–27744, 2022.
44. Sun, L., Huang, Y., Wang, H., Wu, S., Zhang, Q., Gao, C., Huang, Y., Lyu, W., Zhang, Y., Li, X., et al. [Trustllm: Trustworthiness in large language models](https://md.research.elamur.ai/free/en/papers/huang-2024-trustllm-trustworthiness-large-language-models/). arXiv preprint arXiv:2401.05561, 2024.
45. Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pp. 311–318, 2002.
46. Taori, R., Gulrajani, I., Zhang, T., Dubois, Y., Li, X., Guestrin, C., Liang, P., and Hashimoto, T. B. Stanford alpaca: An instruction-following llama model. [https://github.com/tatsu-lab/](https://github.com/tatsu-lab/) stanford_alpaca, 2023a. [↗](https://github.com/tatsu-lab/)
47. Qi, X., Zeng, Y., Xie, T., Chen, P.-Y., Jia, R., Mittal, P., and Henderson, P. [Fine-tuning aligned language models compromises safety, even when users do not intend to!](https://md.research.elamur.ai/free/en/papers/qi-2023-fine-tuning-aligned-language-models/) arXiv preprint arXiv:2310.03693, 2023.
48. Taori, R., Gulrajani, I., Zhang, T., Dubois, Y., Li, X., Guestrin, C., Liang, P., and Hashimoto, T. B. Alpaca: A strong, replicable instruction-following model. Stanford Center for Research on Foundation Models. [https://crfm.stanford](https://crfm.stanford). edu/2023/03/13/alpaca. html, 3(6):7, 2023b. [↗](https://crfm.stanford)
49. Qiu, H., Zhang, S., Li, A., He, H., and Lan, Z. Latent jailbreak: A benchmark for evaluating text safety and output robustness of large language models. arXiv preprint arXiv:2307.08487, 2023. — [arXiv:2307.08487](https://arxiv.org/abs/2307.08487) [find](https://scholar.google.com/scholar?q=Qiu%2C+H%2E%2C+Zhang%2C+S%2E%2C+Li%2C+A%2E%2C+He%2C+H%2E%2C+and+Lan%2C+Z%2E+Latent+jailbreak%3A+A+benchmark+for+evaluating+text+safety+and+output+robustness+of+large+language+models%2E+arXiv+preprint+arXiv%3A2307%2E08487%2C+2023%2E+%E2%80%94+%5BarXiv)
50. Team, G., Mesnard, T., Hardin, C., Dadashi, R., Bhupatiraju, S., Pathak, S., Sifre, L., Rivière, M., Kale, M. S., Love, J., et al. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295, 2024. [find](https://scholar.google.com/scholar?q=Team%2C+G%2E%2C+Mesnard%2C+T%2E%2C+Hardin%2C+C%2E%2C+Dadashi%2C+R%2E%2C+Bhupatiraju%2C+S%2E%2C+Pathak%2C+S%2E%2C+Sifre%2C+L%2E%2C+Rivie%CC%80re%2C+M%2E%2C+Kale%2C+M%2E+S%2E%2C+Love%2C+J%2E%2C+et+al%2E+Gemma%3A+Open+models+based+on+gemini+research+and+technology%2E+arXiv+pr)
51. Rafailov, R., Sharma, A., Mitchell, E., Manning, C. D., Ermon, S., and Finn, C. [Direct preference optimization: Your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/). Advances in Neural Information Processing Systems, 36, 2024.
52. Tedeschi, S., Friedrich, F., Schramowski, P., Kersting, K., Navigli, R., Nguyen, H., and Li, B. Alert: A comprehensive benchmark for assessing large language models’ safety through red teaming. arXiv preprint arXiv:2404.08676, 2024. — [arXiv:2404.08676](https://arxiv.org/abs/2404.08676) [find](https://scholar.google.com/scholar?q=Tedeschi%2C+S%2E%2C+Friedrich%2C+F%2E%2C+Schramowski%2C+P%2E%2C+Kersting%2C+K%2E%2C+Navigli%2C+R%2E%2C+Nguyen%2C+H%2E%2C+and+Li%2C+B%2E+Alert%3A+A+comprehensive+benchmark+for+assessing+large+language+models%E2%80%99+safety+through+red+teaming%2E+arXiv+)
53. Reid, M., Savinov, N., Teplyashin, D., Lepikhin, D., Lillicrap, T., Alayrac, J.-b., Soricut, R., Lazaridou, A., Firat, O., Schrittwieser, J., et al. [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://md.research.elamur.ai/free/en/papers/gemini-team-2024-gemini-1-5-unlocking-multimodal/). arXiv preprint arXiv:2403.05530, 2024.
54. Thakur, A. S., Choudhary, K., Ramayapally, V. S., Vaidyanathan, S., and Hupkes, D. Judging the judges: Evaluating alignment and vulnerabilities in llms-as-judges. arXiv preprint arXiv:2406.12624, 2024. [find](https://scholar.google.com/scholar?q=Thakur%2C+A%2E+S%2E%2C+Choudhary%2C+K%2E%2C+Ramayapally%2C+V%2E+S%2E%2C+Vaidyanathan%2C+S%2E%2C+and+Hupkes%2C+D%2E+Judging+the+judges%3A+Evaluating+alignment+and+vulnerabilities+in+llms%2Das%2Djudges%2E+arXiv+preprint+arXiv%3A2406%2E12624%2C+2024)
55. Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). arXiv preprint arXiv:2302.13971, 2023a.
56. Robey, A., Wong, E., Hassani, H., and Pappas, G. J. [Smoothllm: Defending large language models against jailbreaking attacks](https://md.research.elamur.ai/free/en/papers/robey-2023-smoothllm-defending-large-language-models/). arXiv preprint arXiv:2310.03684, 2023.
57. Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023b. [find](https://scholar.google.com/scholar?q=Touvron%2C+H%2E%2C+Martin%2C+L%2E%2C+Stone%2C+K%2E%2C+Albert%2C+P%2E%2C+Almahairi%2C+A%2E%2C+Babaei%2C+Y%2E%2C+Bashlykov%2C+N%2E%2C+Batra%2C+S%2E%2C+Bhargava%2C+P%2E%2C+Bhosale%2C+S%2E%2C+et+al%2E+Llama+2%3A+Open+foundation+and+fine%2Dtuned+chat+models%2E+arXiv+prepri)
58. Xu, J., Ju, D., Li, M., Boureau, Y.-L., Weston, J., and Dinan, E. Recipes for safety in open-domain chatbots. arXiv preprint arXiv:2010.07079, 2020. [find](https://scholar.google.com/scholar?q=Xu%2C+J%2E%2C+Ju%2C+D%2E%2C+Li%2C+M%2E%2C+Boureau%2C+Y%2E%2DL%2E%2C+Weston%2C+J%2E%2C+and+Dinan%2C+E%2E+Recipes+for+safety+in+open%2Ddomain+chatbots%2E+arXiv+preprint+arXiv%3A2010%2E07079%2C+2020%2E)
59. Xu, L., Zhao, K., Zhu, L., and Xue, H. Sc-safety: A multi-round open-ended question adversarial safety benchmark for large language models in chinese. arXiv preprint arXiv:2310.05818, 2023. — [arXiv:2310.05818](https://arxiv.org/abs/2310.05818) [find](https://scholar.google.com/scholar?q=Xu%2C+L%2E%2C+Zhao%2C+K%2E%2C+Zhu%2C+L%2E%2C+and+Xue%2C+H%2E+Sc%2Dsafety%3A+A+multi%2Dround+open%2Dended+question+adversarial+safety+benchmark+for+large+language+models+in+chinese%2E+arXiv+preprint+arXiv%3A2310%2E05818%2C+2023%2E+%E2%80%94+%5BarXiv%3A2)
60. Tuan, Y.-L., Chen, X., Smith, E. M., Martin, L., Batra, S., Celikyilmaz, A., Wang, W. Y., and Bikel, D. M. Towards safety and helpfulness balanced responses via controllable large language models. arXiv preprint arXiv:2404.01295, 2024. — [arXiv:2404.01295](https://arxiv.org/abs/2404.01295) [find](https://scholar.google.com/scholar?q=Tuan%2C+Y%2E%2DL%2E%2C+Chen%2C+X%2E%2C+Smith%2C+E%2E+M%2E%2C+Martin%2C+L%2E%2C+Batra%2C+S%2E%2C+Celikyilmaz%2C+A%2E%2C+Wang%2C+W%2E+Y%2E%2C+and+Bikel%2C+D%2E+M%2E+Towards+safety+and+helpfulness+balanced+responses+via+controllable+large+language+models%2E+arX)
61. Xu, Z., Liu, Y., Deng, G., Li, Y., and Picek, S. Llm jailbreak attack versus defense techniques–a comprehensive study. arXiv preprint arXiv:2402.13457, 2024. — [arXiv:2402.13457](https://arxiv.org/abs/2402.13457) [find](https://scholar.google.com/scholar?q=Xu%2C+Z%2E%2C+Liu%2C+Y%2E%2C+Deng%2C+G%2E%2C+Li%2C+Y%2E%2C+and+Picek%2C+S%2E+Llm+jailbreak+attack+versus+defense+techniques%E2%80%93a+comprehensive+study%2E+arXiv+preprint+arXiv%3A2402%2E13457%2C+2024%2E+%E2%80%94+%5BarXiv%3A2402%2E13457%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs)
62. Wang, B., Xu, C., Wang, S., Gan, Z., Cheng, Y., Gao, J., Awadallah, A. H., and Li, B. Adversarial glue: A multi-task benchmark for robustness evaluation of language models. arXiv preprint arXiv:2111.02840, 2021. [find](https://scholar.google.com/scholar?q=Wang%2C+B%2E%2C+Xu%2C+C%2E%2C+Wang%2C+S%2E%2C+Gan%2C+Z%2E%2C+Cheng%2C+Y%2E%2C+Gao%2C+J%2E%2C+Awadallah%2C+A%2E+H%2E%2C+and+Li%2C+B%2E+Adversarial+glue%3A+A+multi%2Dtask+benchmark+for+robustness+evaluation+of+language+models%2E+arXiv+preprint+arXiv%3A2111%2E0)
63. Yu, Y., Zhuang, Y., Zhang, J., Meng, Y., Ratner, A. J., Krishna, R., Shen, J., and Zhang, C. [Large language model as attributed training data generator: A tale of diversity and bias](https://md.research.elamur.ai/free/en/papers/yu-2023-large-language-model-attributed-training/). Advances in Neural Information Processing Systems, 36, 2024.
64. Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., and Zhou, D. [Self-consistency improves chain of thought reasoning in language models](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/). arXiv preprint arXiv:2203.11171, 2022.
65. Yuan, T., He, Z., Dong, L., Wang, Y., Zhao, R., Xia, T., Xu, L., Zhou, B., Li, F., Zhang, Z., et al. [R-judge: Benchmarking safety risk awareness for llm agents](https://md.research.elamur.ai/free/en/papers/yuan-2024-r-judge-benchmarking-safety-risk/). arXiv preprint arXiv:2401.10019, 2024.
66. Wang, Y., Shi, Z., Bai, A., and Hsieh, C.-J. Defending llms against jailbreaking attacks via backtranslation. arXiv preprint arXiv:2402.16459, 2024. [find](https://scholar.google.com/scholar?q=Wang%2C+Y%2E%2C+Shi%2C+Z%2E%2C+Bai%2C+A%2E%2C+and+Hsieh%2C+C%2E%2DJ%2E+Defending+llms+against+jailbreaking+attacks+via+backtranslation%2E+arXiv+preprint+arXiv%3A2402%2E16459%2C+2024%2E)
67. Wei, A., Haghtalab, N., and Steinhardt, J. [Jailbroken: How does llm safety training fail?](https://md.research.elamur.ai/free/en/papers/wei-2023-jailbroken-llm-safety-training-fail/) Advances in Neural Information Processing Systems, 36, 2024.
68. Zeng, W., Liu, Y., Mullins, R., Peran, L., Fernandez, J., Harkous, H., Narasimhan, K., Proud, D., Kumar, P., Radharapu, B., et al. Shieldgemma: Generative ai content moderation based on gemma. arXiv preprint arXiv:2407.21772, 2024a. — [arXiv:2407.21772](https://arxiv.org/abs/2407.21772) [find](https://scholar.google.com/scholar?q=Zeng%2C+W%2E%2C+Liu%2C+Y%2E%2C+Mullins%2C+R%2E%2C+Peran%2C+L%2E%2C+Fernandez%2C+J%2E%2C+Harkous%2C+H%2E%2C+Narasimhan%2C+K%2E%2C+Proud%2C+D%2E%2C+Kumar%2C+P%2E%2C+Radharapu%2C+B%2E%2C+et+al%2E+Shieldgemma%3A+Generative+ai+content+moderation+based+on+gemma%2E+arXiv+p)
69. Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., et al. [Emergent abilities of large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-emergent-abilities-large-language-models/). arXiv preprint arXiv:2206.07682, 2022a.
70. Zeng, Y., Lin, H., Zhang, J., Yang, D., Jia, R., and Shi, W. [How johnny can persuade llms to jailbreak them: Rethinking persuasion to challenge ai safety by humanizing llms](https://md.research.elamur.ai/free/en/papers/zeng-2024-how-johnny-can-persuade-llms/). arXiv preprint arXiv:2401.06373, 2024b.
71. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). Advances in neural information processing systems, 35:24824–24837, 2022b.
72. Zhang, S., Dong, L., Li, X., Zhang, S., Sun, X., Wang, S., Li, J., Hu, R., Zhang, T., Wu, F., et al. [Instruction tuning for large language models: A survey](https://md.research.elamur.ai/free/en/papers/zhang-2023-instruction-tuning-large-language-models/). arXiv preprint arXiv:2308.10792, 2023.
73. Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., and Artzi, Y. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675, 2019. [find](https://scholar.google.com/scholar?q=Zhang%2C+T%2E%2C+Kishore%2C+V%2E%2C+Wu%2C+F%2E%2C+Weinberger%2C+K%2E+Q%2E%2C+and+Artzi%2C+Y%2E+Bertscore%3A+Evaluating+text+generation+with+bert%2E+arXiv+preprint+arXiv%3A1904%2E09675%2C+2019%2E)
74. Wei, Z., Wang, Y., and Wang, Y. [Jailbreak and guard aligned language models with only few in-context demonstrations](https://md.research.elamur.ai/free/en/papers/wei-2023-jailbreak-guard-aligned-language-models/). arXiv preprint arXiv:2310.06387, 2023.
75. Zheng, C., Yin, F., Zhou, H., Meng, F., Zhou, J., Chang, K.-W., Huang, M., and Peng, N. On prompt-driven safeguarding for large language models. In ICLR 2024 Workshop on Secure and Trustworthy Large Language Models. [find](https://scholar.google.com/scholar?q=Zheng%2C+C%2E%2C+Yin%2C+F%2E%2C+Zhou%2C+H%2E%2C+Meng%2C+F%2E%2C+Zhou%2C+J%2E%2C+Chang%2C+K%2E%2DW%2E%2C+Huang%2C+M%2E%2C+and+Peng%2C+N%2E+On+prompt%2Ddriven+safeguarding+for+large+language+models%2E+In+ICLR+2024+Workshop+on+Secure+and+Trustworthy+Large+L)
76. Xie, T., Qi, X., Zeng, Y., Huang, Y., Sehwag, U. M., Huang, K., He, L., Wei, B., Li, D., Sheng, Y., et al. Sorry-bench: Systematically evaluating large language model safety refusal behaviors. arXiv preprint arXiv:2406.14598, 2024a. [find](https://scholar.google.com/scholar?q=Xie%2C+T%2E%2C+Qi%2C+X%2E%2C+Zeng%2C+Y%2E%2C+Huang%2C+Y%2E%2C+Sehwag%2C+U%2E+M%2E%2C+Huang%2C+K%2E%2C+He%2C+L%2E%2C+Wei%2C+B%2E%2C+Li%2C+D%2E%2C+Sheng%2C+Y%2E%2C+et+al%2E+Sorry%2Dbench%3A+Systematically+evaluating+large+language+model+safety+refusal+behaviors%2E+arXiv+p)
77. Zheng, L., Chiang, W.-L., Sheng, Y., Li, T., Zhuang, S., Wu, Z., Zhuang, Y., Li, Z., Lin, Z., Xing, E., et al. [Lmsys-chat-1m: A large-scale real-world llm conversation dataset](https://md.research.elamur.ai/free/en/papers/zheng-2023-lmsys-chat-1m-large-scale/). arXiv preprint arXiv:2309.11998, 2023.
78. Xie, X., Song, J., Zhou, Z., Huang, Y., Song, D., and Ma, L. Online safety analysis for llms: a benchmark, an assessment, and a path forward. arXiv preprint arXiv:2404.08517, 2024b. — [arXiv:2404.08517](https://arxiv.org/abs/2404.08517) [find](https://scholar.google.com/scholar?q=Xie%2C+X%2E%2C+Song%2C+J%2E%2C+Zhou%2C+Z%2E%2C+Huang%2C+Y%2E%2C+Song%2C+D%2E%2C+and+Ma%2C+L%2E+Online+safety+analysis+for+llms%3A+a+benchmark%2C+an+assessment%2C+and+a+path+forward%2E+arXiv+preprint+arXiv%3A2404%2E08517%2C+2024b%2E+%E2%80%94+%5BarXiv%3A2404%2E0851)
79. Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E., et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36, 2024. [find](https://scholar.google.com/scholar?q=Zheng%2C+L%2E%2C+Chiang%2C+W%2E%2DL%2E%2C+Sheng%2C+Y%2E%2C+Zhuang%2C+S%2E%2C+Wu%2C+Z%2E%2C+Zhuang%2C+Y%2E%2C+Lin%2C+Z%2E%2C+Li%2C+Z%2E%2C+Li%2C+D%2E%2C+Xing%2C+E%2E%2C+et+al%2E+Judging+llm%2Das%2Da%2Djudge+with+mt%2Dbench+and+chatbot+arena%2E+Advances+in+Neural+Information+Pr)
80. Xie, Y., Yi, J., Shao, J., Curl, J., Lyu, L., Chen, Q., Xie, X., and Wu, F. Defending chatgpt against jailbreak attack via self-reminders. Nature Machine Intelligence, 5(12): 1486–1496, 2023. [find](https://scholar.google.com/scholar?q=Xie%2C+Y%2E%2C+Yi%2C+J%2E%2C+Shao%2C+J%2E%2C+Curl%2C+J%2E%2C+Lyu%2C+L%2E%2C+Chen%2C+Q%2E%2C+Xie%2C+X%2E%2C+and+Wu%2C+F%2E+Defending+chatgpt+against+jailbreak+attack+via+self%2Dreminders%2E+Nature+Machine+Intelligence%2C+5%2812%29%3A+1486%E2%80%931496%2C+2023%2E)
81. Zhou, X. [Challenges in automated debiasing for toxic language detection](https://md.research.elamur.ai/free/en/papers/zhou-2021-challenges-automated-debiasing-toxic-language/). University of Washington, 2020.
82. Zhu, K., Wang, J., Zhou, J., Wang, Z., Chen, H., Wang, Y., Yang, L., Ye, W., Gong, N. Z., Zhang, Y., et al. Promptbench: Towards evaluating the robustness of large language models on adversarial prompts. arXiv preprint arXiv:2306.[04528, 2023](https://md.research.elamur.ai/free/en/papers/zhu-2023-promptrobust-towards-evaluating-robustness-large/).
83. Zou, A., Wang, Z., Kolter, J. Z., and Fredrikson, M. [Universal and transferable adversarial attacks on aligned language models](https://md.research.elamur.ai/free/en/papers/zou-2023-universal-transferable-adversarial-attacks-aligned/). arXiv preprint arXiv:2307.15043, 2023.

*Updated 24 October 2026*