---
title: "SafetyPrompts — paper by Rottger et al. (2024)"
description: "Looking at model release papers and popular benchmarks, the authors show that safety evaluation… 156 references, cited by 29 papers here."
authors: ["Paul Rottger", "Fabio Pernisi", "Bertie Vidgen", "Dirk Hovy"]
year: 2024
venue: "arXiv:2404.05399"
source: "https://arxiv.org/abs/2404.05399"
doi: "10.48550/arxiv.2404.05399"
arxiv: "2404.05399"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/rottger-2024-safetyprompts-systematic-review-open-datasets/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/rottger-2024-safetyprompts-systematic-review-open-datasets/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/9/) / SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety

# SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety

**Authors:** [Paul Rottger](https://md.research.elamur.ai/free/en/authors/paul-rottger/), Fabio Pernisi, [Bertie Vidgen](https://md.research.elamur.ai/free/en/authors/bertie-vidgen/) and [Dirk Hovy](https://md.research.elamur.ai/free/en/authors/dirk-hovy/) · **Year:** 2024 · **Venue:** arXiv:2404.05399 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2404.05399) · [PDF](https://arxiv.org/pdf/2404.05399) · [DOI 10.48550/arxiv.2404.05399](https://doi.org/10.48550/arxiv.2404.05399) · [arXiv 2404.05399](https://arxiv.org/abs/2404.05399) · [OpenAlex](https://openalex.org/W4394642192)

**Citations:** 4 citations (OpenAlex, 2026-10-14) · [29](https://md.research.elamur.ai/free/en/papers/rottger-2024-safetyprompts-systematic-review-open-datasets/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Work on making large language models safe has produced many datasets very quickly. They come from separate groups with different aims, from toxicity and social bias to long-term risks such as power-seeking. Because of this, it is hard to find the right resource or see what is missing. Röttger et al. respond with the first systematic review of openly available text datasets for evaluating or improving LLM safety. They found candidates through a community-driven, iterative process built around the SafetyPrompts.com catalogue, plus snowball search through cited references. Each of the 144 included datasets (published June 2018 – December 2024) was annotated with 23 structured attributes, covering creation, format, purpose, language, licensing and publication venue. The review finds:

- **Growth:** output has risen steeply since ChatGPT, mostly from academia.
- **Trend:** datasets are increasingly fully synthetic.
- **Gaps:** non-English and naturalistic data are clearly scarce.

Looking at model release papers and popular benchmarks, the authors show that safety evaluation is inconsistent and uses only a small share of existing datasets, often older ones. The work matters as a living map of the field and as an argument for standardising safety evaluation.

## Key points
- **Candidate search:** started from 44 seed datasets (January 2024), then added 51 community suggestions via Twitter/Reddit and 49 more via snowball search. Two authors reviewed each candidate; the cutoff date was 17 December 2024.
- **Inclusion criteria:** text-only datasets that are publicly downloadable from GitHub or Hugging Face. Image, audio, multimodal and code-generation datasets were excluded.
- **Growth by year:** only 9 datasets (6.3%) appeared up to 2020, then 16 in 2021, 17 in 2022, 43 in 2023 and 59 in 2024.
- **Purpose categories:** broad safety (53, e.g. SimpleSafetyTests, Anthropic-RedTeam), bias (33, e.g. BBQ, BOLD), narrow safety (25, e.g. SycophancyEval), and value alignment (23, e.g. ETHICS, GlobalOpinionQA).
- **Use in model releases:** the most-used datasets date from 2020/2022, such as the autocomplete-style RealToxicityPrompts and ToxiGen. The authors recommend:
  - more recent, chat-style datasets;
  - shared quality standards (e.g. BetterBench);
  - harness-style evaluation protocols for reproducible results.

  Data and code are available at github.com/paul-rottger/safetyprompts-paper.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Bocconi University](https://md.research.elamur.ai/free/en/universities/bocconi-university/) — University, Milan, Italy · 28 works · 3,077 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,954 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 626 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [🛡️ Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/) · 587 works

## Cite

Cite as: Paul Rottger, Fabio Pernisi, Bertie Vidgen and Dirk Hovy. 2024. SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety. arXiv:2404.05399.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{rottger2024safetyprompts,
  title = {{SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety}},
  author = {Paul Rottger, Fabio Pernisi, Bertie Vidgen and Dirk Hovy},
  year = {2024},
  doi = {10.48550/arxiv.2404.05399},
  eprint = {2404.05399},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2404.05399}
}
```

- [← previous Over-Reasoning and Redundant Calculation of Large Language Models](https://md.research.elamur.ai/free/en/papers/chiang-2024-over-reasoning-redundant-calculation-large/)
- [next → So Hateful! Building a Multi-Label Hate Speech Annotated Arabic Dataset](https://md.research.elamur.ai/free/en/papers/zaghouani-2024-so-hateful-building-multi-label/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/9/)

## References

156 references, 67 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Aakanksha; Ahmadian, A.; Ermis, B.; Goldfarb-Tarrant, S.; Kreutzer, J.; Fadaee, M.; and Hooker, S. 2024. [The Multilingual Alignment Prism: Aligning Global and Local Preferences to Reduce Harm](https://md.research.elamur.ai/free/en/papers/aakanksha-2024-multilingual-alignment-prism-aligning-global/). In Al-Onaizan, Y.; Bansal, M.; and Chen, Y.-N., eds., Proceedings of the 2024 Conference on Empirical Metho…
2. Abdin, M.; Jacobs, S. A.; Awan, A. A.; Aneja, J.; Awadallah, A.; Awadalla, H.; Bach, N.; Bahree, A.; Bakhtiari, A.; Bao, J.; Behl, H.; Benhaim, A.; Bilenko, M.; Bjorck, J.; Bubeck, S.; Cai, Q.; Cai, M.; Mendes, C. C. T.; Chen, W.; Chaudhary, V.; Chen, D.; Chen, D.; Chen, Y.-C.; Chen, Y.-L.; Chopra,… — [Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone](https://md.research.elamur.ai/free/en/papers/abdin-2024-phi-3-technical-report-highly/)
3. Ahn, J.; and Oh, A. 2021. Mitigating Language-Dependent Ethnic Bias in BERT. In Moens, M.-F.; Huang, X.; Specia, L.; and Yih, S. W.-t., eds., Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 533–549. Online and Punta Cana, Dominican Republic: Association for Co… [find](https://scholar.google.com/scholar?q=Ahn%2C+J%2E%3B+and+Oh%2C+A%2E+2021%2E+Mitigating+Language%2DDependent+Ethnic+Bias+in+BERT%2E+In+Moens%2C+M%2E%2DF%2E%3B+Huang%2C+X%2E%3B+Specia%2C+L%2E%3B+and+Yih%2C+S%2E+W%2E%2Dt%2E%2C+eds%2E%2C+Proceedings+of+the+2021+Conference+on+Empirical+Methods+in)
4. Dices dataset: Diversity in conversational ai evaluation for safety. Advances in Neural Information Processing Systems, 36. Bai, Y.; Jones, A.; Ndousse, K.; Askell, A.; Chen, A.; Das-Sarma, N.; Drain, D.; Fort, S.; Ganguli, D.; Henighan, T.; et al. 2022. Training a helpful and harmless assistant wit… [find](https://scholar.google.com/scholar?q=Dices+dataset%3A+Diversity+in+conversational+ai+evaluation+for+safety%2E+Advances+in+Neural+Information+Processing+Systems%2C+36%2E+Bai%2C+Y%2E%3B+Jones%2C+A%2E%3B+Ndousse%2C+K%2E%3B+Askell%2C+A%2E%3B+Chen%2C+A%2E%3B+Das%2DSarma%2C+N%2E%3B+Drain%2C)
5. Bajaj, D.; Lei, Y.; Tong, J.; and Huang, R. 2024. Evaluating Gender Bias of LLMs in Making Morality Judgements. In Al-Onaizan, Y.; Bansal, M.; and Chen, Y.-N., eds., Findings of the Association for Computational Linguistics: EMNLP 2024, 15804–15818. Miami, Florida, USA: Association for Computational… [find](https://scholar.google.com/scholar?q=Bajaj%2C+D%2E%3B+Lei%2C+Y%2E%3B+Tong%2C+J%2E%3B+and+Huang%2C+R%2E+2024%2E+Evaluating+Gender+Bias+of+LLMs+in+Making+Morality+Judgements%2E+In+Al%2DOnaizan%2C+Y%2E%3B+Bansal%2C+M%2E%3B+and+Chen%2C+Y%2E%2DN%2E%2C+eds%2E%2C+Findings+of+the+Association+for+Co)
6. Beeching, E.; Fourrier, C.; Habib, N.; Han, S.; Lambert, N.; Rajani, N.; Sanseviero, O.; Tunstall, L.; and Wolf, T. 2023. Open LLM Leaderboard. [https://huggingface.co/](https://huggingface.co/) spaces/HuggingFaceH4/open llm leaderboard. [↗](https://huggingface.co/)
7. Bender, E. M. 2011. On achieving and evaluating language-independence in NLP. Linguistic Issues in Language Technology, 6. [find](https://scholar.google.com/scholar?q=Bender%2C+E%2E+M%2E+2011%2E+On+achieving+and+evaluating+language%2Dindependence+in+NLP%2E+Linguistic+Issues+in+Language+Technology%2C+6%2E)
8. Bhardwaj, R.; Do, D. A.; and Poria, S. 2024. [Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmetic](https://md.research.elamur.ai/free/en/papers/bhardwaj-2024-language-models-are-homer-simpson/). In Ku, L.-W.; Martins, A.; and Srikumar, V., eds., Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volum…
9. Bhardwaj, R.; and Poria, S. 2023. [Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignment](https://md.research.elamur.ai/free/en/papers/bhardwaj-2023-red-teaming-large-language-models/). arXiv:2308.09662.
10. Bhatt, M.; Chennabasappa, S.; Nikolaidis, C.; Wan, S.; Evtimov, I.; Gabi, D.; Song, D.; Ahmad, F.; Aschermann, C.; Fontana, L.; et al. 2023. Purple llama cyberseceval: A secure coding benchmark for language models. arXiv preprint arXiv:2312.04724. — [arXiv:2312.04724](https://arxiv.org/abs/2312.04724) [find](https://scholar.google.com/scholar?q=Bhatt%2C+M%2E%3B+Chennabasappa%2C+S%2E%3B+Nikolaidis%2C+C%2E%3B+Wan%2C+S%2E%3B+Evtimov%2C+I%2E%3B+Gabi%2C+D%2E%3B+Song%2C+D%2E%3B+Ahmad%2C+F%2E%3B+Aschermann%2C+C%2E%3B+Fontana%2C+L%2E%3B+et+al%2E+2023%2E+Purple+llama+cyberseceval%3A+A+secure+coding+benchmark+for+la)
11. Bhatt, S.; Dev, S.; Talukdar, P.; Dave, S.; and Prabhakaran, V. 2022. Re-contextualizing Fairness in NLP: The Case of India. In He, Y.; Ji, H.; Li, S.; Liu, Y.; and Chang, C.-H., eds., Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and… [find](https://scholar.google.com/scholar?q=Bhatt%2C+S%2E%3B+Dev%2C+S%2E%3B+Talukdar%2C+P%2E%3B+Dave%2C+S%2E%3B+and+Prabhakaran%2C+V%2E+2022%2E+Re%2Dcontextualizing+Fairness+in+NLP%3A+The+Case+of+India%2E+In+He%2C+Y%2E%3B+Ji%2C+H%2E%3B+Li%2C+S%2E%3B+Liu%2C+Y%2E%3B+and+Chang%2C+C%2E%2DH%2E%2C+eds%2E%2C+Proceedings+of+)
12. Bianchi, F.; Kalluri, P.; Durmus, E.; Ladhak, F.; Cheng, M.; Nozza, D.; Hashimoto, T.; Jurafsky, D.; Zou, J.; and Caliskan, A. 2023. Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and… [find](https://scholar.google.com/scholar?q=Bianchi%2C+F%2E%3B+Kalluri%2C+P%2E%3B+Durmus%2C+E%2E%3B+Ladhak%2C+F%2E%3B+Cheng%2C+M%2E%3B+Nozza%2C+D%2E%3B+Hashimoto%2C+T%2E%3B+Jurafsky%2C+D%2E%3B+Zou%2C+J%2E%3B+and+Caliskan%2C+A%2E+2023%2E+Easily+Accessible+Text%2Dto%2DImage+Generation+Amplifies+Demographic+St)
13. Bianchi, F.; Suzgun, M.; Attanasio, G.; Rottger, P.; Jurafsky, D.; Hashimoto, T.; and Zou, J. 2024. [Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions](https://md.research.elamur.ai/free/en/papers/bianchi-2023-safety-tuned-llamas-lessons-improving/). In The Twelfth International Conference on Learning Representations.
14. Bolukbasi, T.; Chang, K.-W.; Zou, J. Y.; Saligrama, V.; and Kalai, A. T. 2016. [Man is to computer programmer as woman is to homemaker? debiasing word embeddings](https://md.research.elamur.ai/free/en/papers/bolukbasi-2016-man-computer-programmer-woman-homemaker/). Advances in neural information processing systems, 29.
15. Brahman, F.; Kumar, S.; Balachandran, V.; Dasigi, P.; Pyatkin, V.; Ravichander, A.; Wiegreffe, S.; Dziri, N.; Chandu, K.; Hessel, J.; Tsvetkov, Y.; Smith, N. A.; Choi, Y.; and Hajishirzi, H. 2024. [The Art of Saying No: Contextual Noncompliance in Language Models](https://md.research.elamur.ai/free/en/papers/brahman-2024-art-saying-no-contextual-noncompliance/). In The Thirty-eight Conference on Ne…
16. Brown, T.; Mann, B.; Ryder, N.; Subbiah, M.; Kaplan, J. D.; Dhariwal, P.; Neelakantan, A.; Shyam, P.; Sastry, G.; Askell, A.; et al. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems, 33: 1877– 1901.
17. Caliskan, A.; Bryson, J. J.; and Narayanan, A. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334): 183–186. Carlini, N.; Nasr, M.; Choquette-Choo, C. A.; Jagielski, M.; Gao, I.; Koh, P. W.; Ippolito, D.; Tramèr, F.; and Schmidt, L. 2023. Are al… [find](https://scholar.google.com/scholar?q=Caliskan%2C+A%2E%3B+Bryson%2C+J%2E+J%2E%3B+and+Narayanan%2C+A%2E+2017%2E+Semantics+derived+automatically+from+language+corpora+contain+human%2Dlike+biases%2E+Science%2C+356%286334%29%3A+183%E2%80%93186%2E+Carlini%2C+N%2E%3B+Nasr%2C+M%2E%3B+Choquette%2DChoo)
18. Cercas Curry, A.; Abercrombie, G.; and Rieser, V. 2021. [ConvAbuse: Data, Analysis, and Benchmarks for Nuanced Detection in Conversational AI](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 7388–7403. Online and Punta Cana, Dominican Republic: Association f…
19. Chen, Y.; Arunasalam, A.; and Celik, Z. B. 2023. Can Large Language Models Provide Security & Privacy Advice? Measuring the Ability of LLMs to Refute Misconceptions. In Proceedings of the 39th Annual Computer Security Applications Conference, ACSAC ’23, 366–378. New York, NY, USA: Association for Co… [find](https://scholar.google.com/scholar?q=Chen%2C+Y%2E%3B+Arunasalam%2C+A%2E%3B+and+Celik%2C+Z%2E+B%2E+2023%2E+Can+Large+Language+Models+Provide+Security+%26+Privacy+Advice%3F+Measuring+the+Ability+of+LLMs+to+Refute+Misconceptions%2E+In+Proceedings+of+the+39th+Annual+)
20. Chiang, W.-L.; Zheng, L.; Sheng, Y.; Angelopoulos, A. N.; Li, T.; Li, D.; Zhang, H.; Zhu, B.; Jordan, M.; Gonzalez, J. E.; and Stoica, I. 2024. [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://md.research.elamur.ai/free/en/papers/chiang-2024-chatbot-arena-open-platform-evaluating/). arXiv:2403.04132.
21. Cui, J.; Chiang, W.-L.; Stoica, I.; and Hsieh, C.-J. 2024. [OR-Bench: An Over-Refusal Benchmark for Large Language Models](https://md.research.elamur.ai/free/en/papers/cui-2024-bench-over-refusal-benchmark-large/). arXiv preprint arXiv:2405.20947.
22. Cui, S.; Zhang, Z.; Chen, Y.; Zhang, W.; Liu, T.; Wang, S.; and Liu, T. 2023. Fft: Towards harmlessness evaluation and analysis for llms with factuality, fairness, toxicity. arXiv preprint arXiv:2311.18580. — [arXiv:2311.18580](https://arxiv.org/abs/2311.18580) [find](https://scholar.google.com/scholar?q=Cui%2C+S%2E%3B+Zhang%2C+Z%2E%3B+Chen%2C+Y%2E%3B+Zhang%2C+W%2E%3B+Liu%2C+T%2E%3B+Wang%2C+S%2E%3B+and+Liu%2C+T%2E+2023%2E+Fft%3A+Towards+harmlessness+evaluation+and+analysis+for+llms+with+factuality%2C+fairness%2C+toxicity%2E+arXiv+preprint+arXiv%3A2311%2E)
23. DeepMind, G. 2024. Gemma 2: Improving Open Language Models at a Practical Size. [find](https://scholar.google.com/scholar?q=DeepMind%2C+G%2E+2024%2E+Gemma+2%3A+Improving+Open+Language+Models+at+a+Practical+Size%2E)
24. Dhamala, J.; Sun, T.; Kumar, V.; Krishna, S.; Pruksachatkun, Y.; Chang, K.-W.; and Gupta, R. 2021. BOLD: Dataset and Metrics for Measuring Biases in Open-Ended Language Generation. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21, 862–872. New York,… [find](https://scholar.google.com/scholar?q=Dhamala%2C+J%2E%3B+Sun%2C+T%2E%3B+Kumar%2C+V%2E%3B+Krishna%2C+S%2E%3B+Pruksachatkun%2C+Y%2E%3B+Chang%2C+K%2E%2DW%2E%3B+and+Gupta%2C+R%2E+2021%2E+BOLD%3A+Dataset+and+Metrics+for+Measuring+Biases+in+Open%2DEnded+Language+Generation%2E+In+Proceedings+of+t)
25. Dinan, E.; Abercrombie, G.; Bergman, A.; Spruit, S.; Hovy, D.; Boureau, Y.-L.; and Rieser, V. 2022. SafetyKit: First Aid for Measuring Safety in Open-domain Conversational Systems. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 411… [find](https://scholar.google.com/scholar?q=Dinan%2C+E%2E%3B+Abercrombie%2C+G%2E%3B+Bergman%2C+A%2E%3B+Spruit%2C+S%2E%3B+Hovy%2C+D%2E%3B+Boureau%2C+Y%2E%2DL%2E%3B+and+Rieser%2C+V%2E+2022%2E+SafetyKit%3A+First+Aid+for+Measuring+Safety+in+Open%2Ddomain+Conversational+Systems%2E+In+Proceedings+of+t)
26. Dinan, E.; Humeau, S.; Chintagunta, B.; and Weston, J. 2019. [Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attack](https://md.research.elamur.ai/free/en/papers/dinan-2019-build-it-break-it-fix/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Languag…
27. Dua, D.; Wang, Y.; Dasigi, P.; Stanovsky, G.; Singh, S.; and Gardner, M. 2019. [DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs](https://md.research.elamur.ai/free/en/papers/dua-2019-drop-reading-comprehension-benchmark-requiring/). In Burstein, J.; Doran, C.; and Solorio, T., eds., Proceedings of the 2019 Conference of the North American Chapter of the Association…
28. Durmus, E.; Nguyen, K.; Liao, T.; Schiefer, N.; Askell, A.; Bakhtin, A.; Chen, C.; Hatfield-Dodds, Z.; Hernandez, D.; Joseph, N.; Lovitt, L.; McCandlish, S.; Sikder, O.; Tamkin, A.; Thamkul, J.; Kaplan, J.; Clark, J.; and Ganguli, D. 2024. [Towards Measuring the Representation of Subjective Global Op…](https://md.research.elamur.ai/free/en/papers/durmus-2023-towards-measuring-representation-subjective-global/)
29. Emelin, D.; Le Bras, R.; Hwang, J. D.; Forbes, M.; and Choi, Y. 2021. Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequences. In Moens, M.-F.; Huang, X.; Specia, L.; and Yih, S. W.-t., eds., Proceedings of the 2021 Conference on Empirical Methods in Natural Language… [find](https://scholar.google.com/scholar?q=Emelin%2C+D%2E%3B+Le+Bras%2C+R%2E%3B+Hwang%2C+J%2E+D%2E%3B+Forbes%2C+M%2E%3B+and+Choi%2C+Y%2E+2021%2E+Moral+Stories%3A+Situated+Reasoning+about+Norms%2C+Intents%2C+Actions%2C+and+their+Consequences%2E+In+Moens%2C+M%2E%2DF%2E%3B+Huang%2C+X%2E%3B+Specia%2C+L%2E%3B+a)
30. Esiobu, D.; Tan, X.; Hosseini, S.; Ung, M.; Zhang, Y.; Fernandes, J.; Dwivedi-Yu, J.; Presani, E.; Williams, A.; and Smith, E. 2023. ROBBIE: Robust Bias Evaluation of Large Generative Language Models. In Bouamor, H.; Pino, J.; and Bali, K., eds., Proceedings of the 2023 Conference on Empirical Metho… [find](https://scholar.google.com/scholar?q=Esiobu%2C+D%2E%3B+Tan%2C+X%2E%3B+Hosseini%2C+S%2E%3B+Ung%2C+M%2E%3B+Zhang%2C+Y%2E%3B+Fernandes%2C+J%2E%3B+Dwivedi%2DYu%2C+J%2E%3B+Presani%2C+E%2E%3B+Williams%2C+A%2E%3B+and+Smith%2C+E%2E+2023%2E+ROBBIE%3A+Robust+Bias+Evaluation+of+Large+Generative+Language+Models%2E)
31. Felkner, V.; Chang, H.-C. H.; Jang, E.; and May, J. 2023. WinoQueer: A Community-in-the-Loop Benchmark for Anti-LGBTQ+ Bias in Large Language Models. In Rogers, A.; Boyd-Graber, J.; and Okazaki, N., eds., Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume… [find](https://scholar.google.com/scholar?q=Felkner%2C+V%2E%3B+Chang%2C+H%2E%2DC%2E+H%2E%3B+Jang%2C+E%2E%3B+and+May%2C+J%2E+2023%2E+WinoQueer%3A+A+Community%2Din%2Dthe%2DLoop+Benchmark+for+Anti%2DLGBTQ%2B+Bias+in+Large+Language+Models%2E+In+Rogers%2C+A%2E%3B+Boyd%2DGraber%2C+J%2E%3B+and+Okazaki%2C+N%2E%2C+e)
32. Forbes, M.; Hwang, J. D.; Shwartz, V.; Sap, M.; and Choi, Y. 2020. Social Chemistry 101: Learning to Reason about Social and Moral Norms. In Webber, B.; Cohn, T.; He, Y.; and Liu, Y., eds., Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 653– 670. Onli… [find](https://scholar.google.com/scholar?q=Forbes%2C+M%2E%3B+Hwang%2C+J%2E+D%2E%3B+Shwartz%2C+V%2E%3B+Sap%2C+M%2E%3B+and+Choi%2C+Y%2E+2020%2E+Social+Chemistry+101%3A+Learning+to+Reason+about+Social+and+Moral+Norms%2E+In+Webber%2C+B%2E%3B+Cohn%2C+T%2E%3B+He%2C+Y%2E%3B+and+Liu%2C+Y%2E%2C+eds%2E%2C+Proceeding)
33. Ganguli, D.; Lovitt, L.; Kernion, J.; Askell, A.; Bai, Y.; Kadavath, S.; Mann, B.; Perez, E.; Schiefer, N.; Ndousse, K.; et al. 2022. [Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned](https://md.research.elamur.ai/free/en/papers/ganguli-2022-red-teaming-language-models-reduce/). arXiv preprint arXiv:2209.07858.
34. Gao, L.; Tow, J.; Biderman, S.; Black, S.; DiPofi, A.; Foster, C.; Golding, L.; Hsu, J.; McDonell, K.; Muennighoff, N.; Phang, J.; Reynolds, L.; Tang, E.; Thite, A.; Wang, B.; Wang, K.; and Zou, A. 2021. A framework for few-shot language model evaluation. In Zenodo. [find](https://scholar.google.com/scholar?q=Gao%2C+L%2E%3B+Tow%2C+J%2E%3B+Biderman%2C+S%2E%3B+Black%2C+S%2E%3B+DiPofi%2C+A%2E%3B+Foster%2C+C%2E%3B+Golding%2C+L%2E%3B+Hsu%2C+J%2E%3B+McDonell%2C+K%2E%3B+Muennighoff%2C+N%2E%3B+Phang%2C+J%2E%3B+Reynolds%2C+L%2E%3B+Tang%2C+E%2E%3B+Thite%2C+A%2E%3B+Wang%2C+B%2E%3B+Wang%2C+K%2E%3B+and+Zou%2C+A%2E+20)
35. Zenodo. Garg, N.; Schiebinger, L.; Jurafsky, D.; and Zou, J. 2018. Word embeddings quantify 100 years of gender and ethnic stereotypes. Proceedings of the National Academy of Sciences, 115(16): E3635–E3644. [find](https://scholar.google.com/scholar?q=Zenodo%2E+Garg%2C+N%2E%3B+Schiebinger%2C+L%2E%3B+Jurafsky%2C+D%2E%3B+and+Zou%2C+J%2E+2018%2E+Word+embeddings+quantify+100+years+of+gender+and+ethnic+stereotypes%2E+Proceedings+of+the+National+Academy+of+Sciences%2C+115%2816%29%3A+E3635%E2%80%93)
36. Gehman, S.; Gururangan, S.; Sap, M.; Choi, Y.; and Smith, N. A. 2020. [RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models](https://md.research.elamur.ai/free/en/papers/gehman-2020-realtoxicityprompts-evaluating-neural-toxic-degeneration/). In Findings of the Association for Computational Linguistics: EMNLP 2020, 3356– 3369. Online: Association for Computational Linguistics.
37. Ghanim, M. A.; Almohaimeed, S.; Zheng, M.; Solihin, Y.; and Lou, Q. 2024. [Jailbreaking LLMs with Arabic Transliteration and Arabizi](https://md.research.elamur.ai/free/en/papers/ghanim-2024-jailbreaking-llms-arabic-transliteration-arabizi/). In Al-Onaizan, Y.; Bansal, M.; and Chen, Y.-N., eds., Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 18584– 18600. Miami, Flor…
38. Ghosh, S.; Varshney, P.; Galinkin, E.; and Parisien, C. 2024. AEGIS: Online Adaptive AI Content Safety Moderation with Ensemble of LLM Experts. arXiv preprint arXiv:2404.05993. — [arXiv:2404.05993](https://arxiv.org/abs/2404.05993) [find](https://scholar.google.com/scholar?q=Ghosh%2C+S%2E%3B+Varshney%2C+P%2E%3B+Galinkin%2C+E%2E%3B+and+Parisien%2C+C%2E+2024%2E+AEGIS%3A+Online+Adaptive+AI+Content+Safety+Moderation+with+Ensemble+of+LLM+Experts%2E+arXiv+preprint+arXiv%3A2404%2E05993%2E+%E2%80%94+%5BarXiv%3A2404%2E05993%5D%28ht)
39. Guo, Y.; Cui, G.; Yuan, L.; Ding, N.; Sun, Z.; Sun, B.; Chen, H.; Xie, R.; Zhou, J.; Lin, Y.; Liu, Z.; and Sun, M. 2024. Controllable Preference Optimization: Toward Controllable Multi-Objective Alignment. In Al-Onaizan, Y.; Bansal, M.; and Chen, Y.-N., eds., Proceedings of the 2024 Conference on Em… [find](https://scholar.google.com/scholar?q=Guo%2C+Y%2E%3B+Cui%2C+G%2E%3B+Yuan%2C+L%2E%3B+Ding%2C+N%2E%3B+Sun%2C+Z%2E%3B+Sun%2C+B%2E%3B+Chen%2C+H%2E%3B+Xie%2C+R%2E%3B+Zhou%2C+J%2E%3B+Lin%2C+Y%2E%3B+Liu%2C+Z%2E%3B+and+Sun%2C+M%2E+2024%2E+Controllable+Preference+Optimization%3A+Toward+Controllable+Multi%2DObjective+Align)
40. Gupta, P.; Yau, L. Q.; Low, H. H.; Lee, I.-S.; Lim, H. M.; Teoh, Y. X.; Hng, K. J.; Liew, D. W.; Bhardwaj, R.; Bhardwaj, R.; and Poria, S. 2024a. WalledEval: A Comprehensive Safety Evaluation Toolkit for Large Language Models. In Hernandez Farias, D. I.; Hope, T.; and Li, M., eds., Proceedings of th… [find](https://scholar.google.com/scholar?q=Gupta%2C+P%2E%3B+Yau%2C+L%2E+Q%2E%3B+Low%2C+H%2E+H%2E%3B+Lee%2C+I%2E%2DS%2E%3B+Lim%2C+H%2E+M%2E%3B+Teoh%2C+Y%2E+X%2E%3B+Hng%2C+K%2E+J%2E%3B+Liew%2C+D%2E+W%2E%3B+Bhardwaj%2C+R%2E%3B+Bhardwaj%2C+R%2E%3B+and+Poria%2C+S%2E+2024a%2E+WalledEval%3A+A+Comprehensive+Safety+Evaluation+Toolkit+)
41. Hall, S. M.; Abrantes, F. G.; Zhu, H.; Sodunke, G.; Shtedritski, A.; and Kirk, H. R. 2023. VisoGender: A dataset for benchmarking gender bias in image-text pronoun resolution. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track. [find](https://scholar.google.com/scholar?q=Hall%2C+S%2E+M%2E%3B+Abrantes%2C+F%2E+G%2E%3B+Zhu%2C+H%2E%3B+Sodunke%2C+G%2E%3B+Shtedritski%2C+A%2E%3B+and+Kirk%2C+H%2E+R%2E+2023%2E+VisoGender%3A+A+dataset+for+benchmarking+gender+bias+in+image%2Dtext+pronoun+resolution%2E+In+Thirty%2Dseventh+Confer)
42. Han, S.; Rao, K.; Ettinger, A.; Jiang, L.; Lin, B. Y.; Lambert, N.; Choi, Y.; and Dziri, N. 2024a. [WildGuard: Open One-stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs](https://md.research.elamur.ai/free/en/papers/han-2024-wildguard-open-one-stop-moderation/). In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track.
43. Han, T.; Kumar, A.; Agarwal, C.; and Lakkaraju, H. 2024b. MedSafetyBench: Evaluating and Improving the Medical Safety of Large Language Models. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track. Hansson, S.; Mavromatakis, K.; Adesam, Y.; Bouma, G.;… [find](https://scholar.google.com/scholar?q=Han%2C+T%2E%3B+Kumar%2C+A%2E%3B+Agarwal%2C+C%2E%3B+and+Lakkaraju%2C+H%2E+2024b%2E+MedSafetyBench%3A+Evaluating+and+Improving+the+Medical+Safety+of+Large+Language+Models%2E+In+The+Thirty%2Deight+Conference+on+Neural+Information+Pro)
44. Hartvigsen, T.; Gabriel, S.; Palangi, H.; Sap, M.; Ray, D.; and Kamar, E. 2022. [ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection](https://md.research.elamur.ai/free/en/papers/hartvigsen-2022-toxigen-large-scale-machine-generated/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 330…
45. Hendrycks, D.; Burns, C.; Basart, S.; Critch, A.; Li, J.; Song, D.; and Steinhardt, J. 2021. [Aligning AI With Shared Human Values](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-aligning-ai-shared-human-values/). In International Conference on Learning Representations.
46. Hendrycks, D.; Burns, C.; Basart, S.; Zou, A.; Mazeika, M.; Song, D.; and Steinhardt, J. 2020. Measuring Massive Multi-task Language Understanding. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Hendrycks%2C+D%2E%3B+Burns%2C+C%2E%3B+Basart%2C+S%2E%3B+Zou%2C+A%2E%3B+Mazeika%2C+M%2E%3B+Song%2C+D%2E%3B+and+Steinhardt%2C+J%2E+2020%2E+Measuring+Massive+Multi%2Dtask+Language+Understanding%2E+In+International+Conference+on+Learning+Representati)
47. Hendrycks, D.; Mazeika, M.; Zou, A.; Patel, S.; Zhu, C.; Navarro, J.; Song, D.; Li, B.; and Steinhardt, J. 2022. What Would Jiminy Cricket Do? Towards Agents That Behave Morally. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2). Holtermann,… [find](https://scholar.google.com/scholar?q=Hendrycks%2C+D%2E%3B+Mazeika%2C+M%2E%3B+Zou%2C+A%2E%3B+Patel%2C+S%2E%3B+Zhu%2C+C%2E%3B+Navarro%2C+J%2E%3B+Song%2C+D%2E%3B+Li%2C+B%2E%3B+and+Steinhardt%2C+J%2E+2022%2E+What+Would+Jiminy+Cricket+Do%3F+Towards+Agents+That+Behave+Morally%2E+In+Thirty%2Dfifth+Confe)
48. Huang, J.; Shao, H.; and Chang, K. C.-C. 2022. Are Large Pre-Trained Language Models Leaking Your Personal Information? In Goldberg, Y.; Kozareva, Z.; and Zhang, Y., eds., Findings of the Association for Computational Linguistics: EMNLP 2022, 2038–2047. Abu Dhabi, United Arab Emirates: Association f… [find](https://scholar.google.com/scholar?q=Huang%2C+J%2E%3B+Shao%2C+H%2E%3B+and+Chang%2C+K%2E+C%2E%2DC%2E+2022%2E+Are+Large+Pre%2DTrained+Language+Models+Leaking+Your+Personal+Information%3F+In+Goldberg%2C+Y%2E%3B+Kozareva%2C+Z%2E%3B+and+Zhang%2C+Y%2E%2C+eds%2E%2C+Findings+of+the+Association+)
49. Huang, K.; Liu, X.; Guo, Q.; Sun, T.; Sun, J.; Wang, Y.; Zhou, Z.; Wang, Y.; Teng, Y.; Qiu, X.; Wang, Y.; and Lin, D. 2024a. Flames: Benchmarking Value Alignment of LLMs in Chinese. In Duh, K.; Gomez, H.; and Bethard, S., eds., Proceedings of the 2024 Conference of the North American Chapter of the… [find](https://scholar.google.com/scholar?q=Huang%2C+K%2E%3B+Liu%2C+X%2E%3B+Guo%2C+Q%2E%3B+Sun%2C+T%2E%3B+Sun%2C+J%2E%3B+Wang%2C+Y%2E%3B+Zhou%2C+Z%2E%3B+Wang%2C+Y%2E%3B+Teng%2C+Y%2E%3B+Qiu%2C+X%2E%3B+Wang%2C+Y%2E%3B+and+Lin%2C+D%2E+2024a%2E+Flames%3A+Benchmarking+Value+Alignment+of+LLMs+in+Chinese%2E+In+Duh%2C+K%2E%3B+Gomez%2C)
50. Huang, Y.; Gupta, S.; Xia, M.; Li, K.; and Chen, D. 2024b. [Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation](https://md.research.elamur.ai/free/en/papers/huang-2023-catastrophic-jailbreak-open-source-llms/). In The Twelfth International Conference on Learning Representations.
51. Huang, Y.; and Xiong, D. 2024. CBBQ: A Chinese Bias Benchmark Dataset Curated with Human-AI Collaboration for Large Language Models. In Calzolari, N.; Kan, M.-Y.; Hoste, V.; Lenci, A.; Sakti, S.; and Xue, N., eds., Proceedings of the 2024 Joint International Conference on Computational Linguistics,… [find](https://scholar.google.com/scholar?q=Huang%2C+Y%2E%3B+and+Xiong%2C+D%2E+2024%2E+CBBQ%3A+A+Chinese+Bias+Benchmark+Dataset+Curated+with+Human%2DAI+Collaboration+for+Large+Language+Models%2E+In+Calzolari%2C+N%2E%3B+Kan%2C+M%2E%2DY%2E%3B+Hoste%2C+V%2E%3B+Lenci%2C+A%2E%3B+Sakti%2C+S%2E%3B+and+)
52. Torino, Italia: ELRA and ICCL. Jha, A.; Mostafazadeh Davani, A.; Reddy, C. K.; Dave, S.; Prabhakaran, V.; and Dev, S. 2023. SeeGULL: A Stereotype Benchmark with Broad Geo-Cultural Coverage Leveraging Generative Models. In Rogers, A.; Boyd-Graber, J.; and Okazaki, N., eds., Proceedings of the 61st An… [find](https://scholar.google.com/scholar?q=Torino%2C+Italia%3A+ELRA+and+ICCL%2E+Jha%2C+A%2E%3B+Mostafazadeh+Davani%2C+A%2E%3B+Reddy%2C+C%2E+K%2E%3B+Dave%2C+S%2E%3B+Prabhakaran%2C+V%2E%3B+and+Dev%2C+S%2E+2023%2E+SeeGULL%3A+A+Stereotype+Benchmark+with+Broad+Geo%2DCultural+Coverage+Leveraging+)
53. Ji, J.; Liu, M.; Dai, J.; Pan, X.; Zhang, C.; Bian, C.; Chen, B.; Sun, R.; Wang, Y.; and Yang, Y. 2023. [BeaverTails: Towards Improved Safety Alignment of LLM via a Human-Preference Dataset](https://md.research.elamur.ai/free/en/papers/ji-2023-beavertails-towards-improved-safety-alignment/). In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track.
54. Jiang, L.; Rao, K.; Han, S.; Ettinger, A.; Brahman, F.; Kumar, S.; Mireshghallah, N.; Lu, X.; Sap, M.; Choi, Y.; and Dziri, N. 2024. [WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models](https://md.research.elamur.ai/free/en/papers/jiang-2024-wildteaming-scale-wild-jailbreaks-adversarially/). In The Thirty-eighth Annual Conference on Neural Information Processing Sys…
55. Jin, J.; Kim, J.; Lee, N.; Yoo, H.; Oh, A.; and Lee, H. 2024a. [KoBBQ: Korean Bias Benchmark for Question Answering](https://md.research.elamur.ai/free/en/papers/jin-2024-kobbq-korean-bias-benchmark-question/). Transactions of the Association for Computational Linguistics, 12: 507–524. Jin, Z.; Kleiman-Weiner, M.; Piatti, G.; Levine, S.; Liu, J.; Adauto, F. G.; Ortu, F.; Strausz, A.; Sachan,…
56. Joshi, P.; Santy, S.; Budhiraja, A.; Bali, K.; and Choudhury, M. 2020. [The State and Fate of Linguistic Diversity and Inclusion in the NLP World](https://md.research.elamur.ai/free/en/papers/joshi-2020-state-fate-linguistic-diversity-inclusion/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 6282–6293. Online: Association for Computational Linguistics.
57. Kaiyom, F.; Ahmed, A.; Mai, Y.; Klyman, K.; Bommasani, R.; and Liang, P. 2024. HELM Safety: Towards Standardized Safety Evaluations of Language Models. Stanford Center for Research on Foundation Models. [find](https://scholar.google.com/scholar?q=Kaiyom%2C+F%2E%3B+Ahmed%2C+A%2E%3B+Mai%2C+Y%2E%3B+Klyman%2C+K%2E%3B+Bommasani%2C+R%2E%3B+and+Liang%2C+P%2E+2024%2E+HELM+Safety%3A+Towards+Standardized+Safety+Evaluations+of+Language+Models%2E+Stanford+Center+for+Research+on+Foundation+Model)
58. Kim, H.; Yu, Y.; Jiang, L.; Lu, X.; Khashabi, D.; Kim, G.; Choi, Y.; and Sap, M. 2022. [ProsocialDialog: A Prosocial Backbone for Conversational Agents](https://md.research.elamur.ai/free/en/papers/kim-2022-prosocialdialog-prosocial-backbone-conversational-agents/). In Goldberg, Y.; Kozareva, Z.; and Zhang, Y., eds., Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 4005–402…
59. Kirk, H. R.; Jun, Y.; Volpin, F.; Iqbal, H.; Benussi, E.; Dreyer, F.; Shtedritski, A.; and Asano, Y. 2021. Bias out-of-the-box: An empirical analysis of intersectional occupational biases in popular generative language models. Advances in neural information processing systems, 34: 2611–2624. Kirk, H… [find](https://scholar.google.com/scholar?q=Kirk%2C+H%2E+R%2E%3B+Jun%2C+Y%2E%3B+Volpin%2C+F%2E%3B+Iqbal%2C+H%2E%3B+Benussi%2C+E%2E%3B+Dreyer%2C+F%2E%3B+Shtedritski%2C+A%2E%3B+and+Asano%2C+Y%2E+2021%2E+Bias+out%2Dof%2Dthe%2Dbox%3A+An+empirical+analysis+of+intersectional+occupational+biases+in+popular+g)
60. Koenecke, A.; Nam, A.; Lake, E.; Nudell, J.; Quartey, M.; Mengesha, Z.; Toups, C.; Rickford, J. R.; Jurafsky, D.; and Goel, S. 2020. Racial disparities in automated speech recognition. Proceedings of the National Academy of Sciences, 117(14): 7684–7689. [find](https://scholar.google.com/scholar?q=Koenecke%2C+A%2E%3B+Nam%2C+A%2E%3B+Lake%2C+E%2E%3B+Nudell%2C+J%2E%3B+Quartey%2C+M%2E%3B+Mengesha%2C+Z%2E%3B+Toups%2C+C%2E%3B+Rickford%2C+J%2E+R%2E%3B+Jurafsky%2C+D%2E%3B+and+Goel%2C+S%2E+2020%2E+Racial+disparities+in+automated+speech+recognition%2E+Proceedings+of+)
61. Kour, G.; Zalmanovici, M.; Zwerdling, N.; Goldbraich, E.; Fandina, O.; Anaby Tavor, A.; Raz, O.; and Farchi, E. 2023. Unveiling Safety Vulnerabilities of Large Language Models. In Gehrmann, S.; Wang, A.; Sedoc, J.; Clark, E.; Dhole, K.; Chandu, K. R.; Santus, E.; and Sedghamiz, H., eds., Proceedings… [find](https://scholar.google.com/scholar?q=Kour%2C+G%2E%3B+Zalmanovici%2C+M%2E%3B+Zwerdling%2C+N%2E%3B+Goldbraich%2C+E%2E%3B+Fandina%2C+O%2E%3B+Anaby+Tavor%2C+A%2E%3B+Raz%2C+O%2E%3B+and+Farchi%2C+E%2E+2023%2E+Unveiling+Safety+Vulnerabilities+of+Large+Language+Models%2E+In+Gehrmann%2C+S%2E%3B+Wang%2C+)
62. LakeraAI. 2023a. Gandalf Prompt Injection. In Hugging Face Dataset. [find](https://scholar.google.com/scholar?q=LakeraAI%2E+2023a%2E+Gandalf+Prompt+Injection%2E+In+Hugging+Face+Dataset%2E)
63. LakeraAI. 2023b. Mosscap Prompt Injection. In Hugging Face Dataset. [find](https://scholar.google.com/scholar?q=LakeraAI%2E+2023b%2E+Mosscap+Prompt+Injection%2E+In+Hugging+Face+Dataset%2E)
64. Lambert, N.; Pyatkin, V.; Morrison, J.; Miranda, L.; Lin, B. Y.; Chandu, K.; Dziri, N.; Kumar, S.; Zick, T.; Choi, Y.; Smith, N. A.; and Hajishirzi, H. 2024. Reward-Bench: Evaluating Reward Models for Language Modeling. arXiv:2403.13787. [find](https://scholar.google.com/scholar?q=Lambert%2C+N%2E%3B+Pyatkin%2C+V%2E%3B+Morrison%2C+J%2E%3B+Miranda%2C+L%2E%3B+Lin%2C+B%2E+Y%2E%3B+Chandu%2C+K%2E%3B+Dziri%2C+N%2E%3B+Kumar%2C+S%2E%3B+Zick%2C+T%2E%3B+Choi%2C+Y%2E%3B+Smith%2C+N%2E+A%2E%3B+and+Hajishirzi%2C+H%2E+2024%2E+Reward%2DBench%3A+Evaluating+Reward+Models+for)
65. Lee, J.; Kim, M.; Kim, S.; Kim, J.; Won, S.; Lee, H.; and Choi, E. 2024. KorNAT: LLM Alignment Benchmark for Korean Social Values and Common Knowledge. In Ku, L.-W.; Martins, A.; and Srikumar, V., eds., Findings of the Association for Computational Linguistics: ACL 2024, 11177–11213. Bangkok, Thaila… [find](https://scholar.google.com/scholar?q=Lee%2C+J%2E%3B+Kim%2C+M%2E%3B+Kim%2C+S%2E%3B+Kim%2C+J%2E%3B+Won%2C+S%2E%3B+Lee%2C+H%2E%3B+and+Choi%2C+E%2E+2024%2E+KorNAT%3A+LLM+Alignment+Benchmark+for+Korean+Social+Values+and+Common+Knowledge%2E+In+Ku%2C+L%2E%2DW%2E%3B+Martins%2C+A%2E%3B+and+Srikumar%2C+V%2E%2C+eds)
66. Levy, S.; Adler, W.; Karver, T. S.; Dredze, M.; and Kaufman, M. R. 2024. Gender Bias in Decision-Making with Large Language Models: A Study of Relationship Conflicts. In Al-Onaizan, Y.; Bansal, M.; and Chen, Y.-N., eds., Findings of the Association for Computational Linguistics: EMNLP 2024, 5777–580… [find](https://scholar.google.com/scholar?q=Levy%2C+S%2E%3B+Adler%2C+W%2E%3B+Karver%2C+T%2E+S%2E%3B+Dredze%2C+M%2E%3B+and+Kaufman%2C+M%2E+R%2E+2024%2E+Gender+Bias+in+Decision%2DMaking+with+Large+Language+Models%3A+A+Study+of+Relationship+Conflicts%2E+In+Al%2DOnaizan%2C+Y%2E%3B+Bansal%2C+M%2E%3B+an)
67. Levy, S.; Allaway, E.; Subbiah, M.; Chilton, L.; Patton, D.; McKeown, K.; and Wang, W. Y. 2022. SafeText: A Benchmark for Exploring Physical Safety in Language Models. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2407–2421. Abu Dhabi, United Arab Emirate… [find](https://scholar.google.com/scholar?q=Levy%2C+S%2E%3B+Allaway%2C+E%2E%3B+Subbiah%2C+M%2E%3B+Chilton%2C+L%2E%3B+Patton%2C+D%2E%3B+McKeown%2C+K%2E%3B+and+Wang%2C+W%2E+Y%2E+2022%2E+SafeText%3A+A+Benchmark+for+Exploring+Physical+Safety+in+Language+Models%2E+In+Proceedings+of+the+2022+Confe)
68. Li, L.; Dong, B.; Wang, R.; Hu, X.; Zuo, W.; Lin, D.; Qiao, Y.; and Shao, J. 2024a. [SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models](https://md.research.elamur.ai/free/en/papers/li-2024-salad-bench-hierarchical-comprehensive-safety/). In Ku, L.-W.; Martins, A.; and Srikumar, V., eds., Findings of the Association for Computational Linguistics: ACL 2024, 3923–3…
69. Li, N.; Pan, A.; Gopal, A.; Yue, S.; Berrios, D.; Gatti, A.; Li, J. D.; Dombrowski, A.-K.; Goel, S.; Mukobi, G.; Helm-Burger, N.; Lababidi, R.; Justen, L.; Liu, A. B.; Chen, M.; Barrass, I.; Zhang, O.; Zhu, X.; Tamirisa, R.; Bharathi, B.; Herbert-Voss, A.; Breuer, C. B.; Zou, A.; Mazeika, M.; Wang,… — [The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning](https://md.research.elamur.ai/free/en/papers/li-2024-wmdp-benchmark-measuring-reducing-malicious/)
70. Li, T.; Khashabi, D.; Khot, T.; Sabharwal, A.; and Srikumar, V. 2020. [UNQOVERing Stereotyping Biases via Underspecified Questions](https://md.research.elamur.ai/free/en/papers/li-2020-unqovering-stereotyping-biases-underspecified-questions/). In Cohn, T.; He, Y.; and Liu, Y., eds., Findings of the Association for Computational Linguistics: EMNLP 2020, 3475–3489. Online: Association for Computational Linguisti…
71. Liang, P.; Bommasani, R.; Lee, T.; Tsipras, D.; Soylu, D.; Yasunaga, M.; Zhang, Y.; Narayanan, D.; Wu, Y.; Kumar, A.; et al. 2023. Holistic Evaluation of Language Models. Transactions on Machine Learning Research. [find](https://scholar.google.com/scholar?q=Liang%2C+P%2E%3B+Bommasani%2C+R%2E%3B+Lee%2C+T%2E%3B+Tsipras%2C+D%2E%3B+Soylu%2C+D%2E%3B+Yasunaga%2C+M%2E%3B+Zhang%2C+Y%2E%3B+Narayanan%2C+D%2E%3B+Wu%2C+Y%2E%3B+Kumar%2C+A%2E%3B+et+al%2E+2023%2E+Holistic+Evaluation+of+Language+Models%2E+Transactions+on+Machine+Learn)
72. Liang, P. P.; Wu, C.; Morency, L.-P.; and Salakhutdinov, R. 2021. [Towards understanding and mitigating social biases in language models](https://md.research.elamur.ai/free/en/papers/liang-2021-towards-understanding-mitigating-social-biases/). In International Conference on Machine Learning, 6565–6576.
73. PMLR. Lin, S.; Hilton, J.; and Evans, O. 2022. [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/). In Muresan, S.; Nakov, P.; and Villavicencio, A., eds., Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 3214–3252. Dublin, Ireland:…
74. Lin, Z.; Wang, Z.; Tong, Y.; Wang, Y.; Guo, Y.; Wang, Y.; and Shang, J. 2023. [ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversation](https://md.research.elamur.ai/free/en/papers/lin-2023-toxicchat-unveiling-hidden-challenges-toxicity/). In Bouamor, H.; Pino, J.; and Bali, K., eds., Findings of the Association for Computational Linguistics: EMNLP 2023, 4694–470…
75. Liu, C.; Zhao, F.; Qing, L.; Kang, Y.; Sun, C.; Kuang, K.; and Wu, F. 2023. A chinese prompt attack dataset for llms with evil content. arXiv preprint arXiv:2309.11830. — [arXiv:2309.11830](https://arxiv.org/abs/2309.11830) [find](https://scholar.google.com/scholar?q=Liu%2C+C%2E%3B+Zhao%2C+F%2E%3B+Qing%2C+L%2E%3B+Kang%2C+Y%2E%3B+Sun%2C+C%2E%3B+Kuang%2C+K%2E%3B+and+Wu%2C+F%2E+2023%2E+A+chinese+prompt+attack+dataset+for+llms+with+evil+content%2E+arXiv+preprint+arXiv%3A2309%2E11830%2E+%E2%80%94+%5BarXiv%3A2309%2E11830%5D%28https%3A%2F%2Far)
76. Lourie, N.; Le Bras, R.; and Choi, Y. 2021. Scruples: A corpus of community ethical judgments on 32,000 real-life anecdotes. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, 13470–13479. [find](https://scholar.google.com/scholar?q=Lourie%2C+N%2E%3B+Le+Bras%2C+R%2E%3B+and+Choi%2C+Y%2E+2021%2E+Scruples%3A+A+corpus+of+community+ethical+judgments+on+32%2C000+real%2Dlife+anecdotes%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+volume+35)
77. Luccioni, S.; Akiki, C.; Mitchell, M.; and Jernite, Y. 2024. Stable bias: Evaluating societal representations in diffusion models. Advances in Neural Information Processing Systems, 36. [find](https://scholar.google.com/scholar?q=Luccioni%2C+S%2E%3B+Akiki%2C+C%2E%3B+Mitchell%2C+M%2E%3B+and+Jernite%2C+Y%2E+2024%2E+Stable+bias%3A+Evaluating+societal+representations+in+diffusion+models%2E+Advances+in+Neural+Information+Processing+Systems%2C+36%2E)
78. Manerba, M.; Stanczak, K.; Guidotti, R.; and Augenstein, I. 2024. Social Bias Probing: Fairness Benchmarking for Language Models. In Al-Onaizan, Y.; Bansal, M.; and Chen, Y.-N., eds., Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 14653–14671. Miami, Florida,… [find](https://scholar.google.com/scholar?q=Manerba%2C+M%2E%3B+Stanczak%2C+K%2E%3B+Guidotti%2C+R%2E%3B+and+Augenstein%2C+I%2E+2024%2E+Social+Bias+Probing%3A+Fairness+Benchmarking+for+Language+Models%2E+In+Al%2DOnaizan%2C+Y%2E%3B+Bansal%2C+M%2E%3B+and+Chen%2C+Y%2E%2DN%2E%2C+eds%2E%2C+Proceedings+of+t)
79. Mazeika, M.; Phan, L.; Yin, X.; Zou, A.; Wang, Z.; Mu, N.; Sakhaee, E.; Li, N.; Basart, S.; Li, B.; Forsyth, D.; and Hendrycks, D. 2024. [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://md.research.elamur.ai/free/en/papers/mazeika-2024-harmbench-standardized-evaluation-framework-automated/). In Forty-first International Conference on Machine Learning.
80. Mazeika, M.; Zou, A.; Mu, N.; Phan, L.; Wang, Z.; Yu, C.; Khoja, A.; Jiang, F.; O’Gara, A.; Sakhaee, E.; Xiang, Z.; Rajabi, A.; Hendrycks, D.; Poovendran, R.; Li, B.; and Forsyth, D. 2023. [find](https://scholar.google.com/scholar?q=Mazeika%2C+M%2E%3B+Zou%2C+A%2E%3B+Mu%2C+N%2E%3B+Phan%2C+L%2E%3B+Wang%2C+Z%2E%3B+Yu%2C+C%2E%3B+Khoja%2C+A%2E%3B+Jiang%2C+F%2E%3B+O%E2%80%99Gara%2C+A%2E%3B+Sakhaee%2C+E%2E%3B+Xiang%2C+Z%2E%3B+Rajabi%2C+A%2E%3B+Hendrycks%2C+D%2E%3B+Poovendran%2C+R%2E%3B+Li%2C+B%2E%3B+and+Forsyth%2C+D%2E+2023%2E)
81. TDC 2023 (LLM Edition): The Trojan Detection Challenge. In NeurIPS Competition Track. [find](https://scholar.google.com/scholar?q=TDC+2023+%28LLM+Edition%29%3A+The+Trojan+Detection+Challenge%2E+In+NeurIPS+Competition+Track%2E)
82. Meta. 2024. [The Llama 3 Herd of Models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/).
83. Meyer, J.; Rauchenstein, L.; Eisenberg, J. D.; and Howell, N. 2020. [Artie Bias Corpus: An Open Dataset for Detecting Demographic Bias in Speech Applications](https://md.research.elamur.ai/free/en/papers/meyer-2020-artie-bias-corpus-open-dataset/). In Calzolari, N.; Béchet, F.; Blache, P.; Choukri, K.; Cieri, C.; Declerck, T.; Goggi, S.; Isahara, H.; Maegaard, B.; Mariani, J.; Mazo, H.;…
84. Mireshghallah, N.; Kim, H.; Zhou, X.; Tsvetkov, Y.; Sap, M.; Shokri, R.; and Choi, Y. 2024. Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Mireshghallah%2C+N%2E%3B+Kim%2C+H%2E%3B+Zhou%2C+X%2E%3B+Tsvetkov%2C+Y%2E%3B+Sap%2C+M%2E%3B+Shokri%2C+R%2E%3B+and+Choi%2C+Y%2E+2024%2E+Can+LLMs+Keep+a+Secret%3F+Testing+Privacy+Implications+of+Language+Models+via+Contextual+Integrity+Theory%2E+In+)
85. Mou, Y.; Zhang, S.; and Ye, W. 2024. SG-Bench: Evaluating LLM Safety Generalization Across Diverse Tasks and Prompt Types. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track. [find](https://scholar.google.com/scholar?q=Mou%2C+Y%2E%3B+Zhang%2C+S%2E%3B+and+Ye%2C+W%2E+2024%2E+SG%2DBench%3A+Evaluating+LLM+Safety+Generalization+Across+Diverse+Tasks+and+Prompt+Types%2E+In+The+Thirty%2Deight+Conference+on+Neural+Information+Processing+Systems+Datas)
86. Mu, N.; Chen, S.; Wang, Z.; Chen, S.; Karamardian, D.; Aljeraisy, L.; Alomair, B.; Hendrycks, D.; and Wagner, D. 2024. Can LLMs Follow Simple Rules? arXiv:2311.04235. — [arXiv:2311.04235](https://arxiv.org/abs/2311.04235) [find](https://scholar.google.com/scholar?q=Mu%2C+N%2E%3B+Chen%2C+S%2E%3B+Wang%2C+Z%2E%3B+Chen%2C+S%2E%3B+Karamardian%2C+D%2E%3B+Aljeraisy%2C+L%2E%3B+Alomair%2C+B%2E%3B+Hendrycks%2C+D%2E%3B+and+Wagner%2C+D%2E+2024%2E+Can+LLMs+Follow+Simple+Rules%3F+arXiv%3A2311%2E04235%2E+%E2%80%94+%5BarXiv%3A2311%2E04235%5D%28https%3A%2F%2Farxi)
87. Nadeem, M.; Bethke, A.; and Reddy, S. 2021. [StereoSet: Measuring stereotypical bias in pretrained language models](https://md.research.elamur.ai/free/en/papers/nadeem-2021-stereoset-measuring-stereotypical-bias-pretrained/). In Zong, C.; Xia, F.; Li, W.; and Navigli, R., eds., Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference…
88. Nangia, N.; Vania, C.; Bhalerao, R.; and Bowman, S. R. 2020. [CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models](https://md.research.elamur.ai/free/en/papers/nangia-2020-crows-pairs-challenge-dataset-measuring/). In Webber, B.; Cohn, T.; He, Y.; and Liu, Y., eds., Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 195…
89. Nozza, D.; Bianchi, F.; and Hovy, D. 2021. [HONEST: Measuring Hurtful Sentence Completion in Language Models](https://md.research.elamur.ai/free/en/papers/nozza-2021-honest-measuring-hurtful-sentence-completion/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2398–2406. Online: Association for Computati…
90. Ontocord.AI. 2023. OIG-Moderation. In Hugging Face Dataset. [find](https://scholar.google.com/scholar?q=Ontocord%2EAI%2E+2023%2E+OIG%2DModeration%2E+In+Hugging+Face+Dataset%2E)
91. Ouyang, S.; Wang, S.; Liu, Y.; Zhong, M.; Jiao, Y.; Iter, D.; Pryzant, R.; Zhu, C.; Ji, H.; and Han, J. 2023. The Shifted and The Overlooked: A Task-oriented Investigation of User-GPT Interactions. In Bouamor, H.; Pino, J.; and Bali, K., eds., Proceedings of the 2023 Conference on Empirical Methods… [find](https://scholar.google.com/scholar?q=Ouyang%2C+S%2E%3B+Wang%2C+S%2E%3B+Liu%2C+Y%2E%3B+Zhong%2C+M%2E%3B+Jiao%2C+Y%2E%3B+Iter%2C+D%2E%3B+Pryzant%2C+R%2E%3B+Zhu%2C+C%2E%3B+Ji%2C+H%2E%3B+and+Han%2C+J%2E+2023%2E+The+Shifted+and+The+Overlooked%3A+A+Task%2Doriented+Investigation+of+User%2DGPT+Interactions%2E+In)
92. Pan, A.; Chan, J. S.; Zou, A.; Li, N.; Basart, S.; Woodside, T.; Zhang, H.; Emmons, S.; and Hendrycks, D. 2023. Do the rewards justify the means? measuring trade-offs between rewards and ethical behavior in the machiavelli benchmark. In International Conference on Machine Learning, 26837– 26867. [find](https://scholar.google.com/scholar?q=Pan%2C+A%2E%3B+Chan%2C+J%2E+S%2E%3B+Zou%2C+A%2E%3B+Li%2C+N%2E%3B+Basart%2C+S%2E%3B+Woodside%2C+T%2E%3B+Zhang%2C+H%2E%3B+Emmons%2C+S%2E%3B+and+Hendrycks%2C+D%2E+2023%2E+Do+the+rewards+justify+the+means%3F+measuring+trade%2Doffs+between+rewards+and+ethical+behav)
93. PMLR. Parrish, A.; Chen, A.; Nangia, N.; Padmakumar, V.; Phang, J.; Thompson, J.; Htut, P. M.; and Bowman, S. 2022. [BBQ: A hand-built bias benchmark for question answering](https://md.research.elamur.ai/free/en/papers/parrish-2022-bbq-hand-built-bias-benchmark/). In Muresan, S.; Nakov, P.; and Villavicencio, A., eds., Findings of the Association for Computational Linguistics: ACL 2022, 20…
94. Parrish, A.; Kirk, H. R.; Quaye, J.; Rastogi, C.; Bartolo, M.; Inel, O.; Ciro, J.; Mosquera, R.; Howard, A.; Cukierski, W.; et al. 2023. Adversarial Nibbler: A Data-Centric Challenge for Improving the Safety of Text-to-Image Models. arXiv preprint arXiv:2305.14384. — [arXiv:2305.14384](https://arxiv.org/abs/2305.14384) [find](https://scholar.google.com/scholar?q=Parrish%2C+A%2E%3B+Kirk%2C+H%2E+R%2E%3B+Quaye%2C+J%2E%3B+Rastogi%2C+C%2E%3B+Bartolo%2C+M%2E%3B+Inel%2C+O%2E%3B+Ciro%2C+J%2E%3B+Mosquera%2C+R%2E%3B+Howard%2C+A%2E%3B+Cukierski%2C+W%2E%3B+et+al%2E+2023%2E+Adversarial+Nibbler%3A+A+Data%2DCentric+Challenge+for+Improving+the)
95. Perez, E.; Ringer, S.; Lukosiute, K.; Nguyen, K.; Chen, E.; Heiner, S.; Pettit, C.; Olsson, C.; Kundu, S.; Kadavath, S.; Jones, A.; Chen, A.; Mann, B.; Israel, B.; Seethor, B.; McKinnon, C.; Olah, C.; Yan, D.; Amodei, D.; Amodei, D.; Drain, D.; Li, D.; Tran-Johnson, E.; Khundadze, G.; Kernion, J.; L… — [Discovering Language Model Behaviors with Model-Written Evaluations](https://md.research.elamur.ai/free/en/papers/perez-2022-discovering-language-model-behaviors-model/)
96. Pikuliak, M.; Oresko, S.; Hrckova, A.; and Simko, M. 2024. Women Are Beautiful, Men Are Leaders: Gender Stereotypes in Machine Translation and Language Modeling. In Al-Onaizan, Y.; Bansal, M.; and Chen, Y.-N., eds., Findings of the Association for Computational Linguistics: EMNLP 2024, 3060–3083. Mi… [find](https://scholar.google.com/scholar?q=Pikuliak%2C+M%2E%3B+Oresko%2C+S%2E%3B+Hrckova%2C+A%2E%3B+and+Simko%2C+M%2E+2024%2E+Women+Are+Beautiful%2C+Men+Are+Leaders%3A+Gender+Stereotypes+in+Machine+Translation+and+Language+Modeling%2E+In+Al%2DOnaizan%2C+Y%2E%3B+Bansal%2C+M%2E%3B+and+Che)
97. Pistilli, G.; Leidinger, A.; Jernite, Y.; Kasirzadeh, A.; Luccioni, A. S.; and Mitchell, M. 2024. CIVICS: Building a Dataset for Examining Culturally-Informed Values in Large Language Models. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, 7(1): 1132–1144. [find](https://scholar.google.com/scholar?q=Pistilli%2C+G%2E%3B+Leidinger%2C+A%2E%3B+Jernite%2C+Y%2E%3B+Kasirzadeh%2C+A%2E%3B+Luccioni%2C+A%2E+S%2E%3B+and+Mitchell%2C+M%2E+2024%2E+CIVICS%3A+Building+a+Dataset+for+Examining+Culturally%2DInformed+Values+in+Large+Language+Models%2E+Proceedi)
98. Qi, X.; Zeng, Y.; Xie, T.; Chen, P.-Y.; Jia, R.; Mittal, P.; and Henderson, P. 2024. [Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!](https://md.research.elamur.ai/free/en/papers/qi-2023-fine-tuning-aligned-language-models/) In The Twelfth International Conference on Learning Representations.
99. Qiu, H.; Zhang, S.; Li, A.; He, H.; and Lan, Z. 2023. Latent jailbreak: A benchmark for evaluating text safety and output robustness of large language models. arXiv preprint arXiv:2307.08487. — [arXiv:2307.08487](https://arxiv.org/abs/2307.08487) [find](https://scholar.google.com/scholar?q=Qiu%2C+H%2E%3B+Zhang%2C+S%2E%3B+Li%2C+A%2E%3B+He%2C+H%2E%3B+and+Lan%2C+Z%2E+2023%2E+Latent+jailbreak%3A+A+benchmark+for+evaluating+text+safety+and+output+robustness+of+large+language+models%2E+arXiv+preprint+arXiv%3A2307%2E08487%2E+%E2%80%94+%5BarXiv)
100. Radharapu, B.; Robinson, K.; Aroyo, L.; and Lahoti, P. 2023. AART: AI-Assisted Red-Teaming with Diverse Data Generation for New LLM-powered Applications. In Wang, M.; and Zitouni, I., eds., Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track, 380–39… [find](https://scholar.google.com/scholar?q=Radharapu%2C+B%2E%3B+Robinson%2C+K%2E%3B+Aroyo%2C+L%2E%3B+and+Lahoti%2C+P%2E+2023%2E+AART%3A+AI%2DAssisted+Red%2DTeaming+with+Diverse+Data+Generation+for+New+LLM%2Dpowered+Applications%2E+In+Wang%2C+M%2E%3B+and+Zitouni%2C+I%2E%2C+eds%2E%2C+Proceeding)
101. Rashkin, H.; Smith, E. M.; Li, M.; and Boureau, Y.-L. 2019. Towards Empathetic Open-domain Conversation Models: A New Benchmark and Dataset. In Korhonen, A.; Traum, D.; and Màrquez, L., eds., Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 5370– 5381. Floren… [find](https://scholar.google.com/scholar?q=Rashkin%2C+H%2E%3B+Smith%2C+E%2E+M%2E%3B+Li%2C+M%2E%3B+and+Boureau%2C+Y%2E%2DL%2E+2019%2E+Towards+Empathetic+Open%2Ddomain+Conversation+Models%3A+A+New+Benchmark+and+Dataset%2E+In+Korhonen%2C+A%2E%3B+Traum%2C+D%2E%3B+and+Ma%CC%80rquez%2C+L%2E%2C+eds%2E%2C+Proceed)
102. Reimao, R.; and Tzerpos, V. 2019. For: A dataset for synthetic speech detection. In 2019 International Conference on Speech Technology and Human-Computer Dialogue (SpeD), 1–10. [find](https://scholar.google.com/scholar?q=Reimao%2C+R%2E%3B+and+Tzerpos%2C+V%2E+2019%2E+For%3A+A+dataset+for+synthetic+speech+detection%2E+In+2019+International+Conference+on+Speech+Technology+and+Human%2DComputer+Dialogue+%28SpeD%29%2C+1%E2%80%9310%2E)
103. IEEE. Reuel, A.; Hardy, A.; Smith, C.; Lamparth, M.; Hardy, M.; and Kochenderfer, M. 2024. BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practices. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track. [find](https://scholar.google.com/scholar?q=IEEE%2E+Reuel%2C+A%2E%3B+Hardy%2C+A%2E%3B+Smith%2C+C%2E%3B+Lamparth%2C+M%2E%3B+Hardy%2C+M%2E%3B+and+Kochenderfer%2C+M%2E+2024%2E+BetterBench%3A+Assessing+AI+Benchmarks%2C+Uncovering+Issues%2C+and+Establishing+Best+Practices%2E+In+The+Thirty%2Deight)
104. Ricker, J.; Damm, S.; Holz, T.; and Fischer, A. 2022. [Towards the detection of diffusion model deepfakes](https://md.research.elamur.ai/free/en/papers/rottger-2023-xstest-test-suite-identifying-exaggerated/). arXiv preprint arXiv:2210.14571. Röttger, P.; Kirk, H.; Vidgen, B.; Attanasio, G.; Bianchi, F.; and Hovy, D. 2024. XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large La…
105. Rudinger, R.; Naradowsky, J.; Leonard, B.; and Van Durme, B. 2018. [Gender Bias in Coreference Resolution](https://md.research.elamur.ai/free/en/papers/rudinger-2018-gender-bias-coreference-resolution/). In Walker, M.; Ji, H.; and Stent, A., eds., Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume…
106. Santurkar, S.; Durmus, E.; Ladhak, F.; Lee, C.; Liang, P.; and Hashimoto, T. 2023. [Whose opinions do language models reflect?](https://md.research.elamur.ai/free/en/papers/santurkar-2023-whose-opinions-language-models-reflect/) In Proceedings of the 40th International Conference on Machine Learning, ICML’23.
107. JMLR.org. Scherrer, N.; Shi, C.; Feder, A.; and Blei, D. 2023. [Evaluating the Moral Beliefs Encoded in LLMs](https://md.research.elamur.ai/free/en/papers/scherrer-2023-evaluating-moral-beliefs-encoded-llms/). In Thirty-seventh Conference on Neural Information Processing Systems.
108. Schulhoff, S.; Pinto, J.; Khan, A.; Bouchard, L.-F.; Si, C.; Anati, S.; Tagliabue, V.; Kost, A.; Carnahan, C.; and Boyd-Graber, J. 2023. Ignore This Title and HackAPrompt: Exposing Systemic Vulnerabilities of LLMs Through a Global Prompt Hacking Competition. In Bouamor, H.; Pino, J.; and Bali, K., e… [find](https://scholar.google.com/scholar?q=Schulhoff%2C+S%2E%3B+Pinto%2C+J%2E%3B+Khan%2C+A%2E%3B+Bouchard%2C+L%2E%2DF%2E%3B+Si%2C+C%2E%3B+Anati%2C+S%2E%3B+Tagliabue%2C+V%2E%3B+Kost%2C+A%2E%3B+Carnahan%2C+C%2E%3B+and+Boyd%2DGraber%2C+J%2E+2023%2E+Ignore+This+Title+and+HackAPrompt%3A+Exposing+Systemic+Vulnerabil)
109. Schwemmer, C.; Knight, C.; Bello-Pardo, E. D.; Oklobdzija, S.; Schoonvelde, M.; and Lockhart, J. W. 2020. Diagnosing gender bias in image recognition systems. Socius, 6: 2378023120967171. [find](https://scholar.google.com/scholar?q=Schwemmer%2C+C%2E%3B+Knight%2C+C%2E%3B+Bello%2DPardo%2C+E%2E+D%2E%3B+Oklobdzija%2C+S%2E%3B+Schoonvelde%2C+M%2E%3B+and+Lockhart%2C+J%2E+W%2E+2020%2E+Diagnosing+gender+bias+in+image+recognition+systems%2E+Socius%2C+6%3A+2378023120967171%2E)
110. Shaikh, O.; Zhang, H.; Held, W.; Bernstein, M.; and Yang, D. 2023. [On Second Thought, Let’s Not Think Step by Step!](https://md.research.elamur.ai/free/en/papers/shaikh-2023-second-thought-let-s-not/) Bias and Toxicity in Zero-Shot Reasoning. In Rogers, A.; Boyd-Graber, J.; and Okazaki, N., eds., Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics…
111. Sharma, M.; Tong, M.; Korbak, T.; Duvenaud, D.; Askell, A.; Bowman, S. R.; DURMUS, E.; Hatfield-Dodds, Z.; Johnston, S. R.; Kravec, S. M.; Maxwell, T.; McCandlish, S.; Ndousse, K.; Rausch, O.; Schiefer, N.; Yan, D.; Zhang, M.; and Perez, E. 2024. [Towards Understanding Sycophancy in Language Models](https://md.research.elamur.ai/free/en/papers/sharma-2023-towards-understanding-sycophancy-language-models/).…
112. Shen, X.; Chen, Z.; Backes, M.; Shen, Y.; and Zhang, Y. 2024. [” do anything now”: Characterizing and evaluating in-the-wild jailbreak prompts on large language models](https://md.research.elamur.ai/free/en/papers/shen-2023-anything-now-characterizing-evaluating-wild/). In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, 1671–1685.
113. Sheng, E.; Chang, K.-W.; Natarajan, P.; and Peng, N. 2019. [The Woman Worked as a Babysitter: On Biases in Language Generation](https://md.research.elamur.ai/free/en/papers/sheng-2019-woman-worked-as-babysitter-biases/). In Inui, K.; Jiang, J.; Ng, V.; and Wan, X., eds., Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Con…
114. Shi, C.; Wang, X.; Ge, Q.; Gao, S.; Yang, X.; Gui, T.; Zhang, Q.; Huang, X.; Zhao, X.; and Lin, D. 2024. Navigating the overkill in large language models. arXiv preprint arXiv:2401.17633. [find](https://scholar.google.com/scholar?q=Shi%2C+C%2E%3B+Wang%2C+X%2E%3B+Ge%2C+Q%2E%3B+Gao%2C+S%2E%3B+Yang%2C+X%2E%3B+Gui%2C+T%2E%3B+Zhang%2C+Q%2E%3B+Huang%2C+X%2E%3B+Zhao%2C+X%2E%3B+and+Lin%2C+D%2E+2024%2E+Navigating+the+overkill+in+large+language+models%2E+arXiv+preprint+arXiv%3A2401%2E17633%2E)
115. Siddiq, M. L.; and Santos, J. C. S. 2022. SecurityEval dataset: mining vulnerability examples to evaluate machine learning-based code generation techniques. In Proceedings of the 1st International Workshop on Mining Software Repositories Applications for Privacy and Security, MSR4P&S 2022, 29–33. Ne… [find](https://scholar.google.com/scholar?q=Siddiq%2C+M%2E+L%2E%3B+and+Santos%2C+J%2E+C%2E+S%2E+2022%2E+SecurityEval+dataset%3A+mining+vulnerability+examples+to+evaluate+machine+learning%2Dbased+code+generation+techniques%2E+In+Proceedings+of+the+1st+International+Wor)
116. Smith, E. M.; Hall, M.; Kambadur, M.; Presani, E.; and Williams, A. 2022. [“I’m sorry to hear that”: Finding New Biases in Language Models with a Holistic Descriptor Dataset](https://md.research.elamur.ai/free/en/papers/smith-2022-i-m-sorry-hear-that/). In Goldberg, Y.; Kozareva, Z.; and Zhang, Y., eds., Proceedings of the 2022 Conference on Empirical Methods in Natural Languag…
117. Sotnikova, A.; Cao, Y. T.; Daumé III, H.; and Rudinger, R. 2021. Analyzing Stereotypes in Generative Text Inference Tasks. In Zong, C.; Xia, F.; Li, W.; and Navigli, R., eds., Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, 4052–4065. Online: Association for Computationa… [find](https://scholar.google.com/scholar?q=Sotnikova%2C+A%2E%3B+Cao%2C+Y%2E+T%2E%3B+Daume%CC%81+III%2C+H%2E%3B+and+Rudinger%2C+R%2E+2021%2E+Analyzing+Stereotypes+in+Generative+Text+Inference+Tasks%2E+In+Zong%2C+C%2E%3B+Xia%2C+F%2E%3B+Li%2C+W%2E%3B+and+Navigli%2C+R%2E%2C+eds%2E%2C+Findings+of+the+Associa)
118. Souly, A.; Lu, Q.; Bowen, D.; Trinh, T.; Hsieh, E.; Pandey, S.; Abbeel, P.; Svegliato, J.; Emmons, S.; Watkins, O.; and Toyer, S. 2024. A StrongREJECT for Empty Jailbreaks. In ICLR 2024 Workshop on Reliable and Responsible Foundation Models. [find](https://scholar.google.com/scholar?q=Souly%2C+A%2E%3B+Lu%2C+Q%2E%3B+Bowen%2C+D%2E%3B+Trinh%2C+T%2E%3B+Hsieh%2C+E%2E%3B+Pandey%2C+S%2E%3B+Abbeel%2C+P%2E%3B+Svegliato%2C+J%2E%3B+Emmons%2C+S%2E%3B+Watkins%2C+O%2E%3B+and+Toyer%2C+S%2E+2024%2E+A+StrongREJECT+for+Empty+Jailbreaks%2E+In+ICLR+2024+Workshop+on+Re)
119. Srivastava, A.; Rastogi, A.; Rao, A.; Shoeb, A. A. M.; Abid, A.; Fisch, A.; Brown, A. R.; Santoro, A.; Gupta, A.; Garriga-Alonso, A.; et al. 2023. [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://md.research.elamur.ai/free/en/papers/srivastava-2022-imitation-game-quantifying-extrapolating-capabilities/). Transactions on Machine Learning Research.
120. Sun, D.; Abzaliev, A.; Kotek, H.; Klein, C.; Xiu, Z.; and Williams, J. 2023a. DELPHI: Data for Evaluating LLMs’ Performance in Handling Controversial Issues. In Wang, M.; and Zitouni, I., eds., Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing: Industry Track, 82… [find](https://scholar.google.com/scholar?q=Sun%2C+D%2E%3B+Abzaliev%2C+A%2E%3B+Kotek%2C+H%2E%3B+Klein%2C+C%2E%3B+Xiu%2C+Z%2E%3B+and+Williams%2C+J%2E+2023a%2E+DELPHI%3A+Data+for+Evaluating+LLMs%E2%80%99+Performance+in+Handling+Controversial+Issues%2E+In+Wang%2C+M%2E%3B+and+Zitouni%2C+I%2E%2C+eds%2E%2C+Procee)
121. Sun, H.; Xu, G.; Deng, J.; Cheng, J.; Zheng, C.; Zhou, H.; Peng, N.; Zhu, X.; and Huang, M. 2022. [On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmark](https://md.research.elamur.ai/free/en/papers/sun-2022-safety-conversational-models-taxonomy-dataset/). In Muresan, S.; Nakov, P.; and Villavicencio, A., eds., Findings of the Association for Computational Linguistics: ACL 2022, 390…
122. Sun, H.; Zhang, Z.; Deng, J.; Cheng, J.; and Huang, M. 2023b. [Safety assessment of chinese large language models](https://md.research.elamur.ai/free/en/papers/sun-2023-safety-assessment-chinese-large-language/). arXiv preprint arXiv:2304.10436.
123. Sun, L.; Huang, Y.; Wang, H.; Wu, S.; Zhang, Q.; Gao, C.; Huang, Y.; Lyu, W.; Zhang, Y.; Li, X.; et al. 2024. [Trustllm: Trustworthiness in large language models](https://md.research.elamur.ai/free/en/papers/huang-2024-trustllm-trustworthiness-large-language-models/). arXiv preprint arXiv:2401.05561.
124. Tamkin, A.; Askell, A.; Lovitt, L.; Durmus, E.; Joseph, N.; Kravec, S.; Nguyen, K.; Kaplan, J.; and Ganguli, D. 2023a. Evaluating and mitigating discrimination in language model decisions. arXiv preprint arXiv:2312.03689. [find](https://scholar.google.com/scholar?q=Tamkin%2C+A%2E%3B+Askell%2C+A%2E%3B+Lovitt%2C+L%2E%3B+Durmus%2C+E%2E%3B+Joseph%2C+N%2E%3B+Kravec%2C+S%2E%3B+Nguyen%2C+K%2E%3B+Kaplan%2C+J%2E%3B+and+Ganguli%2C+D%2E+2023a%2E+Evaluating+and+mitigating+discrimination+in+language+model+decisions%2E+arXiv+prepr)
125. Tamkin, A.; Askell, A.; Lovitt, L.; Durmus, E.; Joseph, N.; Kravec, S.; Nguyen, K.; Kaplan, J.; and Ganguli, D. 2023b. Evaluating and mitigating discrimination in language model decisions. arXiv preprint arXiv:2312.03689. Tan, X. E.; Hansanti, P.; Wood, C.; Yu, B.; Ropers, C.; and Costa-jussà, M. R… [find](https://scholar.google.com/scholar?q=Tamkin%2C+A%2E%3B+Askell%2C+A%2E%3B+Lovitt%2C+L%2E%3B+Durmus%2C+E%2E%3B+Joseph%2C+N%2E%3B+Kravec%2C+S%2E%3B+Nguyen%2C+K%2E%3B+Kaplan%2C+J%2E%3B+and+Ganguli%2C+D%2E+2023b%2E+Evaluating+and+mitigating+discrimination+in+language+model+decisions%2E+arXiv+prepr)
126. Tedeschi, S.; Friedrich, F.; Schramowski, P.; Kersting, K.; Navigli, R.; Nguyen, H.; and Li, B. 2024. [ALERT: A Comprehensive Benchmark for Assessing Large Language Models’ Safety through Red Teaming](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). arXiv preprint arXiv:2404.08676. Touvron, H.; Lavril, T.; Izacard, G.; Martinet, X.; Lachaux, M.-A.;…
127. Toyer, S.; Watkins, O.; Mendes, E.; Svegliato, J.; Bailey, L.; Wang, T.; Ong, I.; Elmaaroufi, K.; Abbeel, P.; Darrell, T.; Ritter, A.; and Russell, S. 2023. Tensor Trust: Interpretable Prompt Injection Attacks from an Online Game. In NeurIPS 2023 Workshop on Instruction Tuning and Instruction Follow… [find](https://scholar.google.com/scholar?q=Toyer%2C+S%2E%3B+Watkins%2C+O%2E%3B+Mendes%2C+E%2E%3B+Svegliato%2C+J%2E%3B+Bailey%2C+L%2E%3B+Wang%2C+T%2E%3B+Ong%2C+I%2E%3B+Elmaaroufi%2C+K%2E%3B+Abbeel%2C+P%2E%3B+Darrell%2C+T%2E%3B+Ritter%2C+A%2E%3B+and+Russell%2C+S%2E+2023%2E+Tensor+Trust%3A+Interpretable+Prompt+Injectio)
128. Ung, M.; Xu, J.; and Boureau, Y.-L. 2022. [SaFeRDialogues: Taking Feedback Gracefully after Conversational Safety Failures](https://md.research.elamur.ai/free/en/papers/vidgen-2024-introducing-v0-5-ai-safety/). In Muresan, S.; Nakov, P.; and Villavicencio, A., eds., Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 6462–648…
129. Wang, B.; Chen, W.; Pei, H.; Xie, C.; Kang, M.; Zhang, C.; Xu, C.; Xiong, Z.; Dutta, R.; Schaeffer, R.; et al. 2024a. [DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models](https://md.research.elamur.ai/free/en/papers/wang-2023-decodingtrust-comprehensive-assessment-trustworthiness-gpt/). Advances in Neural Information Processing Systems, 36.
130. Wang, W.; Tu, Z.; Chen, C.; Yuan, Y.; Huang, J.-t.; Jiao, W.; and Lyu, M. 2024b. All Languages Matter: On the Multilingual Safety of LLMs. In Ku, L.-W.; Martins, A.; and Srikumar, V., eds., Findings of the Association for Computational Linguistics: ACL 2024, 5865–5877. Bangkok, Thailand: Association… [find](https://scholar.google.com/scholar?q=Wang%2C+W%2E%3B+Tu%2C+Z%2E%3B+Chen%2C+C%2E%3B+Yuan%2C+Y%2E%3B+Huang%2C+J%2E%2Dt%2E%3B+Jiao%2C+W%2E%3B+and+Lyu%2C+M%2E+2024b%2E+All+Languages+Matter%3A+On+the+Multilingual+Safety+of+LLMs%2E+In+Ku%2C+L%2E%2DW%2E%3B+Martins%2C+A%2E%3B+and+Srikumar%2C+V%2E%2C+eds%2E%2C+Findings+o)
131. Wang, Y.; Li, H.; Han, X.; Nakov, P.; and Baldwin, T. 2024c. [Do-Not-Answer: Evaluating Safeguards in LLMs](https://md.research.elamur.ai/free/en/papers/wang-2024-do-not-answer-evaluating-safeguards/). In Graham, Y.; and Purver, M., eds., Findings of the Association for Computational Linguistics: EACL 2024, 896–911. St. Julian’s, Malta: Association for Computational Linguistics.
132. Wei, A.; Haghtalab, N.; and Steinhardt, J. 2023. [Jailbroken: How Does LLM Safety Training Fail?](https://md.research.elamur.ai/free/en/papers/wei-2023-jailbroken-llm-safety-training-fail/) arXiv preprint arXiv:2307.02483.
133. Wolfe, R.; Yang, Y.; Howe, B.; and Caliskan, A. 2023. Contrastive Language-Vision AI Models Pretrained on Web-Scraped Multimodal Data Exhibit Sexual Objectification Bias. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’23, 1174–1185. New York, NY, USA:… [find](https://scholar.google.com/scholar?q=Wolfe%2C+R%2E%3B+Yang%2C+Y%2E%3B+Howe%2C+B%2E%3B+and+Caliskan%2C+A%2E+2023%2E+Contrastive+Language%2DVision+AI+Models+Pretrained+on+Web%2DScraped+Multimodal+Data+Exhibit+Sexual+Objectification+Bias%2E+In+Proceedings+of+the+2023+AC)
134. Xie, T.; Qi, X.; Zeng, Y.; Huang, Y.; Sehwag, U. M.; Huang, K.; He, L.; Wei, B.; Li, D.; Sheng, Y.; et al. 2024. [Sorry-bench: Systematically evaluating large language model safety refusal](https://md.research.elamur.ai/free/en/papers/xie-2024-sorry-bench-systematically-evaluating-large/) behaviors. arXiv preprint arXiv:2406.14598.
135. Xu, G.; Liu, J.; Yan, M.; Xu, H.; Si, J.; Zhou, Z.; Yi, P.; Gao, X.; Sang, J.; Zhang, R.; et al. 2023. [Cvalues: Measuring the values of chinese large language models from safety to responsibility](https://md.research.elamur.ai/free/en/papers/xu-2023-cvalues-measuring-values-chinese/). arXiv preprint arXiv:2307.09705.
136. Xu, J.; Ju, D.; Li, M.; Boureau, Y.-L.; Weston, J.; and Dinan, E. 2021. [Bot-Adversarial Dialogue for Safe Conversational Agents](https://md.research.elamur.ai/free/en/papers/xu-2021-bot-adversarial-dialogue-safe-conversational/). In Toutanova, K.; Rumshisky, A.; Zettlemoyer, L.; Hakkani-Tur, D.; Beltagy, I.; Bethard, S.; Cotterell, R.; Chakraborty, T.; and Zhou, Y., eds., Proceedings of the 2021 Co…
137. Yang, A.; Yang, B.; Hui, B.; Zheng, B.; Yu, B.; Zhou, C.; Li, C.; Li, C.; Liu, D.; Huang, F.; Dong, G.; Wei, H.; Lin, H.; Tang, J.; Wang, J.; Yang, J.; Tu, J.; Zhang, J.; Ma, J.; Yang, J.; Xu, J.; Zhou, J.; Bai, J.; He, J.; Lin, J.; Dang, K.; Lu, K.; Chen, K.; Yang, K.; Li, M.; Xue, M.; Ni, N.; Zhan… [find](https://scholar.google.com/scholar?q=Yang%2C+A%2E%3B+Yang%2C+B%2E%3B+Hui%2C+B%2E%3B+Zheng%2C+B%2E%3B+Yu%2C+B%2E%3B+Zhou%2C+C%2E%3B+Li%2C+C%2E%3B+Li%2C+C%2E%3B+Liu%2C+D%2E%3B+Huang%2C+F%2E%3B+Dong%2C+G%2E%3B+Wei%2C+H%2E%3B+Lin%2C+H%2E%3B+Tang%2C+J%2E%3B+Wang%2C+J%2E%3B+Yang%2C+J%2E%3B+Tu%2C+J%2E%3B+Zhang%2C+J%2E%3B+Ma%2C+J%2E%3B+Yang%2C+J%2E%3B+Xu%2C+J%2E%3B+Zho)
138. Yu, E.; Li, J.; Liao, M.; Wang, S.; Zuchen, G.; Mi, F.; and Hong, L. 2024a. CoSafe: Evaluating Large Language Model Safety in Multi-Turn Dialogue Coreference. In Al-Onaizan, Y.; Bansal, M.; and Chen, Y.-N., eds., Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,… [find](https://scholar.google.com/scholar?q=Yu%2C+E%2E%3B+Li%2C+J%2E%3B+Liao%2C+M%2E%3B+Wang%2C+S%2E%3B+Zuchen%2C+G%2E%3B+Mi%2C+F%2E%3B+and+Hong%2C+L%2E+2024a%2E+CoSafe%3A+Evaluating+Large+Language+Model+Safety+in+Multi%2DTurn+Dialogue+Coreference%2E+In+Al%2DOnaizan%2C+Y%2E%3B+Bansal%2C+M%2E%3B+and+Chen%2C+)
139. Yu, J.; Li, L.; and Lan, Z. 2024a. Beyond Binary Classification: A Fine-grained Safety Dataset for Large Language Models. [find](https://scholar.google.com/scholar?q=Yu%2C+J%2E%3B+Li%2C+L%2E%3B+and+Lan%2C+Z%2E+2024a%2E+Beyond+Binary+Classification%3A+A+Fine%2Dgrained+Safety+Dataset+for+Large+Language+Models%2E)
140. IEEE Access. Yu, J.; Li, L.; and Lan, Z. 2024b. Beyond Binary Classification: A Fine-Grained Safety Dataset for Large Language Models. IEEE Access, 12: 64717–64726. [find](https://scholar.google.com/scholar?q=IEEE+Access%2E+Yu%2C+J%2E%3B+Li%2C+L%2E%3B+and+Lan%2C+Z%2E+2024b%2E+Beyond+Binary+Classification%3A+A+Fine%2DGrained+Safety+Dataset+for+Large+Language+Models%2E+IEEE+Access%2C+12%3A+64717%E2%80%9364726%2E)
141. Yu, J.; Lin, X.; Yu, Z.; and Xing, X. 2024b. [Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts](https://md.research.elamur.ai/free/en/papers/yu-2023-gptfuzzer-red-teaming-large-language/). arXiv preprint arXiv:2309.10253.
142. Yu, L.; Leng, Y.; Huang, Y.; Wu, S.; Liu, H.; Ji, X.; Zhao, J.; Song, J.; Cui, T.; Cheng, X.; Liutao, L.; and Xiong, D. 2024c. CMoralEval: A Moral Evaluation Benchmark for Chinese Large Language Models. In Ku, L.-W.; Martins, A.; and Srikumar, V., eds., Findings of the Association for Computational… [find](https://scholar.google.com/scholar?q=Yu%2C+L%2E%3B+Leng%2C+Y%2E%3B+Huang%2C+Y%2E%3B+Wu%2C+S%2E%3B+Liu%2C+H%2E%3B+Ji%2C+X%2E%3B+Zhao%2C+J%2E%3B+Song%2C+J%2E%3B+Cui%2C+T%2E%3B+Cheng%2C+X%2E%3B+Liutao%2C+L%2E%3B+and+Xiong%2C+D%2E+2024c%2E+CMoralEval%3A+A+Moral+Evaluation+Benchmark+for+Chinese+Large+Language+Model)
143. Yuan, X.; Li, J.; Wang, D.; Chen, Y.; Mao, X.; Huang, L.; Xue, H.; Wang, W.; Ren, K.; and Wang, J. 2024. S-Eval: Automatic and Adaptive Test Generation for Benchmarking Safety Evaluation of Large Language Models. arXiv preprint arXiv:2405.14191. — [arXiv:2405.14191](https://arxiv.org/abs/2405.14191) [find](https://scholar.google.com/scholar?q=Yuan%2C+X%2E%3B+Li%2C+J%2E%3B+Wang%2C+D%2E%3B+Chen%2C+Y%2E%3B+Mao%2C+X%2E%3B+Huang%2C+L%2E%3B+Xue%2C+H%2E%3B+Wang%2C+W%2E%3B+Ren%2C+K%2E%3B+and+Wang%2C+J%2E+2024%2E+S%2DEval%3A+Automatic+and+Adaptive+Test+Generation+for+Benchmarking+Safety+Evaluation+of+Large+Lang)
144. Zhang, M.; Pan, X.; and Yang, M. 2023. Jade: A linguistics-based safety evaluation platform for llm. arXiv preprint arXiv:2311.00286. — [arXiv:2311.00286](https://arxiv.org/abs/2311.00286) [find](https://scholar.google.com/scholar?q=Zhang%2C+M%2E%3B+Pan%2C+X%2E%3B+and+Yang%2C+M%2E+2023%2E+Jade%3A+A+linguistics%2Dbased+safety+evaluation+platform+for+llm%2E+arXiv+preprint+arXiv%3A2311%2E00286%2E+%E2%80%94+%5BarXiv%3A2311%2E00286%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2311%2E00286%29)
145. Zhang, W.; Lei, X.; Liu, Z.; An, M.; Yang, B.; Zhao, K.; Wang, K.; and Lian, S. 2024a. CHiSafetyBench: A Chinese Hierarchical Safety Benchmark for Large Language Models. arXiv preprint arXiv:2406.10311. — [arXiv:2406.10311](https://arxiv.org/abs/2406.10311) [find](https://scholar.google.com/scholar?q=Zhang%2C+W%2E%3B+Lei%2C+X%2E%3B+Liu%2C+Z%2E%3B+An%2C+M%2E%3B+Yang%2C+B%2E%3B+Zhao%2C+K%2E%3B+Wang%2C+K%2E%3B+and+Lian%2C+S%2E+2024a%2E+CHiSafetyBench%3A+A+Chinese+Hierarchical+Safety+Benchmark+for+Large+Language+Models%2E+arXiv+preprint+arXiv%3A2406%2E1031)
146. Zhang, Y.; Mai, Y.; Roberts, J. S.; Bommasani, R.; Dubois, Y.; and Liang, P. 2024b. HELM Instruct: A Multidimensional Instruction Following Evaluation Framework with Absolute Ratings. [find](https://scholar.google.com/scholar?q=Zhang%2C+Y%2E%3B+Mai%2C+Y%2E%3B+Roberts%2C+J%2E+S%2E%3B+Bommasani%2C+R%2E%3B+Dubois%2C+Y%2E%3B+and+Liang%2C+P%2E+2024b%2E+HELM+Instruct%3A+A+Multidimensional+Instruction+Following+Evaluation+Framework+with+Absolute+Ratings%2E)
147. Zhang, Z.; Lei, L.; Wu, L.; Sun, R.; Huang, Y.; Long, C.; Liu, X.; Lei, X.; Tang, J.; and Huang, M. 2024c. [SafetyBench: Evaluating the Safety of Large Language Models](https://md.research.elamur.ai/free/en/papers/zhang-2023-safetybench-evaluating-safety-large/). In Ku, L.-W.; Martins, A.; and Srikumar, V., eds., Proceedings of the 62nd Annual Meeting of the Association for Computational Lingu…
148. Zhao, D.; Wang, A.; and Russakovsky, O. 2021. Understanding and evaluating racial biases in image captioning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 14830–14840. [find](https://scholar.google.com/scholar?q=Zhao%2C+D%2E%3B+Wang%2C+A%2E%3B+and+Russakovsky%2C+O%2E+2021%2E+Understanding+and+evaluating+racial+biases+in+image+captioning%2E+In+Proceedings+of+the+IEEE%2FCVF+International+Conference+on+Computer+Vision%2C+14830%E2%80%9314840%2E)
149. Zhao, J.; Fang, M.; Shi, Z.; Li, Y.; Chen, L.; and Pechenizkiy, M. 2023. CHBias: Bias Evaluation and Mitigation of Chinese Conversational Language Models. In Rogers, A.; Boyd-Graber, J.; and Okazaki, N., eds., Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (V… [find](https://scholar.google.com/scholar?q=Zhao%2C+J%2E%3B+Fang%2C+M%2E%3B+Shi%2C+Z%2E%3B+Li%2C+Y%2E%3B+Chen%2C+L%2E%3B+and+Pechenizkiy%2C+M%2E+2023%2E+CHBias%3A+Bias+Evaluation+and+Mitigation+of+Chinese+Conversational+Language+Models%2E+In+Rogers%2C+A%2E%3B+Boyd%2DGraber%2C+J%2E%3B+and+Okazaki%2C+)
150. Zhao, J.; Wang, T.; Yatskar, M.; Ordonez, V.; and Chang, K.-W. 2018. [Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods](https://md.research.elamur.ai/free/en/papers/zhao-2018-gender-bias-coreference-resolution-evaluation/). In Walker, M.; Ji, H.; and Stent, A., eds., Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:…
151. Zhao, W.; Mondal, D.; Tandon, N.; Dillion, D.; Gray, K.; and Gu, Y. 2024a. WorldValuesBench: A Large-Scale Benchmark Dataset for Multi-Cultural Value Awareness of Language Models. In Calzolari, N.; Kan, M.-Y.; Hoste, V.; Lenci, A.; Sakti, S.; and Xue, N., eds., Proceedings of the 2024 Joint Internat… [find](https://scholar.google.com/scholar?q=Zhao%2C+W%2E%3B+Mondal%2C+D%2E%3B+Tandon%2C+N%2E%3B+Dillion%2C+D%2E%3B+Gray%2C+K%2E%3B+and+Gu%2C+Y%2E+2024a%2E+WorldValuesBench%3A+A+Large%2DScale+Benchmark+Dataset+for+Multi%2DCultural+Value+Awareness+of+Language+Models%2E+In+Calzolari%2C+N%2E%3B+Ka)
152. Torino, Italia: ELRA and ICCL. Zhao, W.; Ren, X.; Hessel, J.; Cardie, C.; Choi, Y.; and Deng, Y. 2024b. (InThe)WildChat: 570K ChatGPT Interaction Logs In The Wild. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Torino%2C+Italia%3A+ELRA+and+ICCL%2E+Zhao%2C+W%2E%3B+Ren%2C+X%2E%3B+Hessel%2C+J%2E%3B+Cardie%2C+C%2E%3B+Choi%2C+Y%2E%3B+and+Deng%2C+Y%2E+2024b%2E+%28InThe%29WildChat%3A+570K+ChatGPT+Interaction+Logs+In+The+Wild%2E+In+The+Twelfth+International+Confere)
153. Zheng, L.; Chiang, W.-L.; Sheng, Y.; Li, T.; Zhuang, S.; Wu, Z.; Zhuang, Y.; Li, Z.; Lin, Z.; Xing, E.; Gonzalez, J. E.; Stoica, I.; and Zhang, H. 2024. LMSYS-1M: A Large-Scale Real-World LLM Conversation Dataset. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Zheng%2C+L%2E%3B+Chiang%2C+W%2E%2DL%2E%3B+Sheng%2C+Y%2E%3B+Li%2C+T%2E%3B+Zhuang%2C+S%2E%3B+Wu%2C+Z%2E%3B+Zhuang%2C+Y%2E%3B+Li%2C+Z%2E%3B+Lin%2C+Z%2E%3B+Xing%2C+E%2E%3B+Gonzalez%2C+J%2E+E%2E%3B+Stoica%2C+I%2E%3B+and+Zhang%2C+H%2E+2024%2E+LMSYS%2D1M%3A+A+Large%2DScale+Real%2DWorld+LLM+Conversa)
154. Zhou, J.; Deng, J.; Mi, F.; Li, Y.; Wang, Y.; Huang, M.; Jiang, X.; Liu, Q.; and Meng, H. 2022. Towards Identifying Social Bias in Dialog Systems: Framework, Dataset, and Benchmark. In Goldberg, Y.; Kozareva, Z.; and Zhang, Y., eds., Findings of the Association for Computational Linguistics: EMNLP 2… [find](https://scholar.google.com/scholar?q=Zhou%2C+J%2E%3B+Deng%2C+J%2E%3B+Mi%2C+F%2E%3B+Li%2C+Y%2E%3B+Wang%2C+Y%2E%3B+Huang%2C+M%2E%3B+Jiang%2C+X%2E%3B+Liu%2C+Q%2E%3B+and+Meng%2C+H%2E+2022%2E+Towards+Identifying+Social+Bias+in+Dialog+Systems%3A+Framework%2C+Dataset%2C+and+Benchmark%2E+In+Goldberg%2C+Y%2E%3B+K)
155. Ziems, C.; Yu, J.; Wang, Y.-C.; Halevy, A.; and Yang, D. 2022. [The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systems](https://md.research.elamur.ai/free/en/papers/ziems-2022-moral-integrity-corpus-benchmark-ethical/). In Muresan, S.; Nakov, P.; and Villavicencio, A., eds., Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)…
156. Zou, A.; Wang, Z.; Carlini, N.; Nasr, M.; Kolter, J. Z.; and Fredrikson, M. 2023. [Universal and transferable adversarial attacks on aligned language models](https://md.research.elamur.ai/free/en/papers/zou-2023-universal-transferable-adversarial-attacks-aligned/). arXiv preprint arXiv:2307.15043.

*Updated 2 November 2026*