---
title: "SimpleSafetyTests — paper by Vidgen et al. (2023)"
description: "SST therefore offers a lightweight baseline check for critical risks before deployment. 72% accuracy. 39 references, cited by 37 papers here."
authors: ["Bertie Vidgen", "Nino Scherrer", "Hannah Rose Kirk", "Rebecca Qian", "Anand Kannappan", "Scott A. Hale", "Paul Rottger"]
year: 2023
venue: "arXiv:2311.08370"
source: "https://arxiv.org/abs/2311.08370"
doi: "10.48550/arxiv.2311.08370"
arxiv: "2311.08370"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/vidgen-2023-simplesafetytests-test-suite-identifying-critical/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/vidgen-2023-simplesafetytests-test-suite-identifying-critical/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/11/) / SimpleSafetyTests: a Test Suite for Identifying Critical Safety Risks in Large Language Models

# SimpleSafetyTests: a Test Suite for Identifying Critical Safety Risks in Large Language Models

**Authors:** [Bertie Vidgen](https://md.research.elamur.ai/free/en/authors/bertie-vidgen/), [Nino Scherrer](https://md.research.elamur.ai/free/en/authors/nino-scherrer/), [Hannah Rose Kirk](https://md.research.elamur.ai/free/en/authors/hannah-rose-kirk/), Rebecca Qian, Anand Kannappan, [Scott A. Hale](https://md.research.elamur.ai/free/en/authors/scott-a-hale/) and [Paul Rottger](https://md.research.elamur.ai/free/en/authors/paul-rottger/) · **Year:** 2023 · **Venue:** arXiv:2311.08370 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2311.08370) · [PDF](https://arxiv.org/pdf/2311.08370) · [DOI 10.48550/arxiv.2311.08370](https://doi.org/10.48550/arxiv.2311.08370) · [arXiv 2311.08370](https://arxiv.org/abs/2311.08370) · [OpenAlex](https://openalex.org/W4388717463)

**Citations:** 5 citations (OpenAlex, 2026-10-14) · [37](https://md.research.elamur.ai/free/en/papers/vidgen-2023-simplesafetytests-test-suite-identifying-critical/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Large language models that lack proper safeguards can comply with clearly harmful requests, such as giving bomb-making guidance or encouraging self-harm. Developers need a fast, cheap way to detect these failures. The authors built SimpleSafetyTests (SST), a compact set of 100 synthetic English prompts written by hand. Each prompt is one that nearly any deployment should refuse. The prompts cover five severe harm areas and are phrased either as questions seeking information or as direct commands. They ran 15 instruction-tuned LLMs (11 open, 4 closed-source) on SST, both with and without a safety-oriented system prompt. Trained annotators then labelled all 3,000 outputs. Several models proved seriously vulnerable, open models far more often than closed ones. A protective system prompt reduced unsafe outputs but did not eliminate them. The team also used the human labels to test whether existing safety classifiers could automate the evaluation. A zero-shot GPT-4 moderation prompt clearly outperformed widely used tools. SST therefore offers a lightweight baseline check for critical risks before deployment.

## Key points
- **Dataset:** SST has 100 prompts, evenly split across five harm areas and two categories (Information and Advice Seeking vs. Instructions and Actions). The harm areas are Suicide/Self-Harm/Eating Disorders, Physical Harm, Illegal Items, Scams & Fraud, and Child Abuse. SST is released on GitHub and HuggingFace, while model responses are available only on request.
- **Annotation:** Three annotators labelled 3,000 responses (15 models × 2 setups × 100 prompts). They fully agreed in 92% of cases, with a Fleiss' Kappa of 0.83.
- **Overall results:** 20% of all responses were unsafe, with 27% for open models and 2% for closed models. The worst model reached 73% unsafe. Llama2 (13B) and Claude 2.1 produced no unsafe outputs, and Falcon (40B) and GPT-4 each produced only one.
- **System prompt:** Adding Mistral's "guardrail" system prompt lowered the unsafe rate from 25% to 16%.
- **Automated filters:** Perspective API reached 72% accuracy and the OpenAI moderation API about 70%. LlamaGuard was also tested. A zero-shot GPT-4 prompt scored best at 89%, with 88% on safe and 95% on unsafe responses.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Oxford](https://md.research.elamur.ai/free/en/universities/university-of-oxford/) — University, Oxford, United Kingdom · 122 works · 160,277 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,954 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 626 works

🧭 Topics (3): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,341 works · [🛡️ Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/) · 587 works

## Cite

Cite as: Bertie Vidgen, Nino Scherrer, Hannah Rose Kirk, Rebecca Qian, Anand Kannappan, Scott A. Hale and Paul Rottger. 2023. SimpleSafetyTests: a Test Suite for Identifying Critical Safety Risks in Large Language Models. arXiv:2311.08370.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{vidgen2023simplesafetytests,
  title = {{SimpleSafetyTests: a Test Suite for Identifying Critical Safety Risks in Large Language Models}},
  author = {Bertie Vidgen, Nino Scherrer, Hannah Rose Kirk, Rebecca Qian, Anand Kannappan, Scott A. Hale and Paul Rottger},
  year = {2023},
  doi = {10.48550/arxiv.2311.08370},
  eprint = {2311.08370},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2311.08370}
}
```

- [← previous Paloma: A Benchmark for Evaluating Language Model Fit](https://md.research.elamur.ai/free/en/papers/magnusson-2023-paloma-benchmark-evaluating-language-model/)
- [next → The ROOTS Search Tool: Data Transparency for LLMs](https://md.research.elamur.ai/free/en/papers/piktus-2023-roots-search-tool-data-transparency/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/11/)

## References

39 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ioannis Agrafiotis, Jason R C Nurse, Michael Goldsmith, Sadie Creese, and David Upton. A taxonomy of cyber-harms: Defining the impacts of cyber-attacks and understanding how they propagate. Journal of Cybersecurity, 4(1):tyy006, 10 2018. ISSN 2057-2085. doi: 10.1093/cybsec/tyy006. URL [https://doi.or…](https://doi.or…) — [doi:10.1093/cybsec/tyy006](https://doi.org/10.1093/cybsec/tyy006) [↗](https://doi.or…)
2. Anthropic. Core Views on AI Safety. core-views-on-ai-safety, 2023. [https://www.anthropic.com/index/](https://www.anthropic.com/index/) [↗](https://www.anthropic.com/index/)
3. Aiman El Asam and Adrienne Katz. Vulnerable young people and their experience of online risks. Human–Computer Interaction, 33(4):281–304, 2018. doi: 10.1080/07370024.2018.1437544. — [doi:10.1080/07370024.2018.1437544](https://doi.org/10.1080/07370024.2018.1437544) [find](https://scholar.google.com/scholar?q=Aiman+El+Asam+and+Adrienne+Katz%2E+Vulnerable+young+people+and+their+experience+of+online+risks%2E+Human%E2%80%93Computer+Interaction%2C+33%284%29%3A281%E2%80%93304%2C+2018%2E+doi%3A+10%2E1080%2F07370024%2E2018%2E1437544%2E+%E2%80%94+%5Bdoi%3A10%2E1080%2F07370)
4. Michele Banko, Brendon MacKeen, and Laurie Ray. A unified taxonomy of harmful content. In Proceedings of the Fourth Workshop on Online Abuse and Harms, pp. 125–137, Online, November 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.alw-1.16. URL https://aclanthology.org/2020.alw… [↗](https://aclanthology.org/2020.alw…)
5. Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21, pp. 610–623, New York, NY, USA, 2021. Associati… — [doi:10.1145/3442188.3445922](https://doi.org/10.1145/3442188.3445922) [find](https://scholar.google.com/scholar?q=Emily+M%2E+Bender%2C+Timnit+Gebru%2C+Angelina+McMillan%2DMajor%2C+and+Shmargaret+Shmitchell%2E+On+the+dangers+of+stochastic+parrots%3A+Can+language+models+be+too+big%3F+In+Proceedings+of+the+2021+ACM+Conference+on+Fa)
6. Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, Shyamal Buch, Dallas Card, Rodrigo Castellon, Niladri Chatterji, Annie Chen, Kathleen Creel, Jared Quincy Davis, Dora De… — [On the Opportunities and Risks of Foundation Models](https://md.research.elamur.ai/free/en/papers/bommasani-2021-opportunities-risks-foundation-models/)
7. Alan Chan, Rebecca Salganik, Alva Markelius, Chris Pang, Nitarshan Rajkumar, Dmitrii Krasheninnikov, Lauro Langosco, Zhonghao He, Yawen Duan, Micah Carroll, Michelle Lin, Alex Mayhew, Katherine Collins, Maryam Molamohammadi, John Burden, Wanru Zhao, Shalaleh Rismani, Konstantinos Voudouris, Umang Bh… — [doi:10.1145/3593013.3594033](https://doi.org/10.1145/3593013.3594033) [find](https://scholar.google.com/scholar?q=Alan+Chan%2C+Rebecca+Salganik%2C+Alva+Markelius%2C+Chris+Pang%2C+Nitarshan+Rajkumar%2C+Dmitrii+Krasheninnikov%2C+Lauro+Langosco%2C+Zhonghao+He%2C+Yawen+Duan%2C+Micah+Carroll%2C+Michelle+Lin%2C+Alex+Mayhew%2C+Katherine+Collin)
8. Cohere. Responsibility - Developing safer language models. responsibility, 2023. [https://cohere.com/](https://cohere.com/) [↗](https://cohere.com/)
9. The Atlantic Council. Scaling Trust on the Web. [https://www.atlanticcouncil.org/](https://www.atlanticcouncil.org/) in-depth-research-reports/report/scaling-trust/, 2023. [↗](https://www.atlanticcouncil.org/)
10. Josef Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji, Xinbo Xu, Mickel Liu, Yizhou Wang, and Yaodong Yang. [Safe rlhf: Safe reinforcement learning from human feedback](https://md.research.elamur.ai/free/en/papers/dai-2023-safe-rlhf-safe-reinforcement-learning/), 2023.
11. Emily Dinan, Gavin Abercrombie, A. Bergman, Shannon Spruit, Dirk Hovy, Y-Lan Boureau, and Verena Rieser. SafetyKit: First aid for measuring safety in open-domain conversational systems. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)… [find](https://scholar.google.com/scholar?q=Emily+Dinan%2C+Gavin+Abercrombie%2C+A%2E+Bergman%2C+Shannon+Spruit%2C+Dirk+Hovy%2C+Y%2DLan+Boureau%2C+and+Verena+Rieser%2E+SafetyKit%3A+First+aid+for+measuring+safety+in+open%2Ddomain+conversational+systems%2E+In+Proceedings)
12. World Economic Forum. Toolkit for Digital Safety Design Interventions and Innovations: Typology of Online Harms. [https://www3.weforum.org/docs/WEF_Typology_of_Online_](https://www3.weforum.org/docs/WEF_Typology_of_Online_) Harms_2023.pdf, 2023. [↗](https://www3.weforum.org/docs/WEF_Typology_of_Online_)
13. Matt Gardner, Yoav Artzi, Victoria Basmov, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai Elazar, Ananth Gottumukkala, Nitish Gupta, Hannaneh Hajishirzi, Gabriel Ilharco, Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F. Liu, Phoebe Mulcaire, Qiang Ning, Sameer Singh, N… — [Evaluating Models' Local Decision Boundaries via Contrast Sets](https://md.research.elamur.ai/free/en/papers/gardner-2020-evaluating-models-local-decision-boundaries/)
14. Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith. [RealToxicityPrompts: Evaluating neural toxic degeneration in language models](https://md.research.elamur.ai/free/en/papers/gehman-2020-realtoxicityprompts-evaluating-neural-toxic-degeneration/). In Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 3356–3369, Online, November 2020. Association for Computational Li…
15. Amelia Glaese, Nat McAleese, Maja Tr˛ebacz, John Aslanides, Vlad Firoiu, Timo Ewalds, Maribeth Rauh, Laura Weidinger, Martin Chadwick, Phoebe Thacker, Lucy Campbell-Gillingham, Jonathan Uesato, Po-Sen Huang, Ramona Comanescu, Fan Yang, Abigail See, Sumanth Dathathri, Rory Greig, Charlie Chen, Doug F… — [Improving alignment of dialogue agents via targeted human judgements](https://md.research.elamur.ai/free/en/papers/glaese-2022-improving-alignment-dialogue-agents-targeted/)
16. Susan Hao, Piyush Kumar, Sarah Laszlo, Shivani Poddar, Bhaktipriya Radharapu, and Renee Shelby. Safety and fairness for content moderation in generative models, 2023. [find](https://scholar.google.com/scholar?q=Susan+Hao%2C+Piyush+Kumar%2C+Sarah+Laszlo%2C+Shivani+Poddar%2C+Bhaktipriya+Radharapu%2C+and+Renee+Shelby%2E+Safety+and+fairness+for+content+moderation+in+generative+models%2C+2023%2E)
17. Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar. [ToxiGen: A large-scale machine-generated dataset for adversarial and implicit hate speech detection](https://md.research.elamur.ai/free/en/papers/hartvigsen-2022-toxigen-large-scale-machine-generated/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long…
18. Md Saroar Jahan and Mourad Oussalah. A systematic review of hate speech automatic detection using natural language processing. CoRR, abs/2106.00742, 2021. URL [https://arxiv.org/](https://arxiv.org/) abs/2106.00742. — [arXiv:2106.00742](https://arxiv.org/abs/2106.00742) [↗](https://arxiv.org/)
19. Jiaming Ji, Borong Zhang, Jiayi Zhou, Xuehai Pan, Weidong Huang, Ruiyang Sun, Yiran Geng, Yifan Zhong, Juntao Dai, and Yaodong Yang. Safety-gymnasium: A unified safe reinforcement learning benchmark, 2023. [find](https://scholar.google.com/scholar?q=Jiaming+Ji%2C+Borong+Zhang%2C+Jiayi+Zhou%2C+Xuehai+Pan%2C+Weidong+Huang%2C+Ruiyang+Sun%2C+Yiran+Geng%2C+Yifan+Zhong%2C+Juntao+Dai%2C+and+Yaodong+Yang%2E+Safety%2Dgymnasium%3A+A+unified+safe+reinforcement+learning+benchmark%2C+)
20. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… [find](https://scholar.google.com/scholar?q=Albert+Q%2E+Jiang%2C+Alexandre+Sablayrolles%2C+Arthur+Mensch%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guillaume+Lample%2C+Lucile+Saulnier%2C+L%C3%A9lio+Renard+Lav)
21. Haoran Li, Dadi Guo, Wei Fan, Mingshi Xu, Jie Huang, Fanpu Meng, and Yangqiu Song. [Multi-step jailbreaking privacy attacks on chatgpt](https://md.research.elamur.ai/free/en/papers/li-2023-multi-step-jailbreaking-privacy-attacks/), 2023.
22. Zi Lin, Zihan Wang, Yongqi Tong, Yangkun Wang, Yuxin Guo, Yujia Wang, and Jingbo Shang. [Toxicchat: Unveiling hidden challenges of toxicity detection in real-world user-ai conversation](https://md.research.elamur.ai/free/en/papers/lin-2023-toxicchat-unveiling-hidden-challenges-toxicity/), 2023.
23. LMSYS. LMSYS Chatbot Arena Leaderboard. [https://chat.lmsys.org/?leaderboard](https://chat.lmsys.org/?leaderboard), 2023. [↗](https://chat.lmsys.org/?leaderboard)
24. Mary McHugh. Interrater reliability: The kappa statistic. Biochemia medica : časopis Hrvatskoga društva medicinskih biokemičara / HDMB, 22:276–82, 10 2012. doi: 10.11613/BM.2012.031. — [doi:10.11613/bm.2012.031](https://doi.org/10.11613/bm.2012.031) [find](https://scholar.google.com/scholar?q=Mary+McHugh%2E+Interrater+reliability%3A+The+kappa+statistic%2E+Biochemia+medica+%3A+c%CC%8Casopis+Hrvatskoga+dru%C5%A1tva+medicinskih+biokemic%CC%8Cara+%2F+HDMB%2C+22%3A276%E2%80%9382%2C+10+2012%2E+doi%3A+10%2E11613%2FBM%2E2012%2E031%2E+%E2%80%94+%5Bdoi%3A10%2E11613)
25. Mistral. Guardrailing. [https://docs.mistral.ai/usage/guardrailing/](https://docs.mistral.ai/usage/guardrailing/), 2023. [↗](https://docs.mistral.ai/usage/guardrailing/)
26. OpenAI. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/), 2023.
27. OpenAI. Product Safety Standards. [https://openai.com/safety-standards](https://openai.com/safety-standards), 2024. [↗](https://openai.com/safety-standards)
28. Digital Trust & Safety Partnership. Trust & Safety Glossary of Terms. dtspartnership.org/glossary/, 2023. https:// [find](https://scholar.google.com/scholar?q=Digital+Trust+%26+Safety+Partnership%2E+Trust+%26+Safety+Glossary+of+Terms%2E+dtspartnership%2Eorg%2Fglossary%2F%2C+2023%2E+https%3A%2F%2F)
29. Guilherme Penedo, Quentin Malartic, Daniel Hesslow, Ruxandra Cojocaru, Alessandro Cappelli, Hamza Alobeidli, Baptiste Pannier, Ebtesam Almazrouei, and Julien Launay. The refinedweb dataset for falcon llm: Outperforming curated corpora with web data, and web data only, 2023. [find](https://scholar.google.com/scholar?q=Guilherme+Penedo%2C+Quentin+Malartic%2C+Daniel+Hesslow%2C+Ruxandra+Cojocaru%2C+Alessandro+Cappelli%2C+Hamza+Alobeidli%2C+Baptiste+Pannier%2C+Ebtesam+Almazrouei%2C+and+Julien+Launay%2E+The+refinedweb+dataset+for+falcon+)
30. Vinodkumar Prabhakaran, Zeerak Waseem, Seyi Akiwowo, and Bertie Vidgen. Online abuse and human rights: WOAH satellite session at RightsCon 2020. In Proceedings of the Fourth Workshop on Online Abuse and Harms, pp. 1–6, Online, November 2020. Association for Computational Linguistics. doi: 10.18653/v… [find](https://scholar.google.com/scholar?q=Vinodkumar+Prabhakaran%2C+Zeerak+Waseem%2C+Seyi+Akiwowo%2C+and+Bertie+Vidgen%2E+Online+abuse+and+human+rights%3A+WOAH+satellite+session+at+RightsCon+2020%2E+In+Proceedings+of+the+Fourth+Workshop+on+Online+Abuse+a)
31. Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. [Fine-tuning aligned language models compromises safety, even when users do not intend to!](https://md.research.elamur.ai/free/en/papers/qi-2023-fine-tuning-aligned-language-models/), 2023.
32. Paul Rottger, Bertie Vidgen, Dirk Hovy, and Janet Pierrehumbert. [Two contrasting data annotation paradigms for subjective NLP tasks](https://md.research.elamur.ai/free/en/papers/rottger-2022-two-contrasting-data-annotation-paradigms/). In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 175– 190, Seattl…
33. Paul Röttger, Hannah Rose Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy. [Xstest: A test suite for identifying exaggerated safety behaviours in large language models](https://md.research.elamur.ai/free/en/papers/rottger-2023-xstest-test-suite-identifying-exaggerated/), 2023.
34. Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. ["do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models](https://md.research.elamur.ai/free/en/papers/shen-2023-anything-now-characterizing-evaluating-wild/), 2023.
35. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/), 2023a.
36. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,… — [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/)
37. Bertie Vidgen, Alex Harris, Dong Nguyen, Rebekah Tromble, Scott Hale, and Helen Margetts. [Challenges and frontiers in abusive content detection](https://md.research.elamur.ai/free/en/papers/vidgen-2019-challenges-frontiers-abusive-content-detection/). In Proceedings of the Third Workshop on Abusive Language Online, pp. 80–93, Florence, Italy, August 2019. Association for Computational Linguistics. doi: 1…
38. Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. [Jailbroken: How does llm safety training fail?](https://md.research.elamur.ai/free/en/papers/wei-2023-jailbroken-llm-safety-training-fail/), 2023.
39. Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, Zac Kenton, Sasha Brown, Will Hawkins, Tom Stepleton, Courtney Biles, Abeba Birhane, Julia Haas, Laura Rimell, Lisa Anne Hendricks, William Isaac, Sean L… — [Ethical and social risks of harm from Language Models](https://md.research.elamur.ai/free/en/papers/weidinger-2021-ethical-social-risks-harm-language/)

*Updated 2 November 2026*