---
title: "The Art of Saying No — paper by Brahman et al. (2024)"
description: "The authors propose a taxonomy of contextual noncompliance with five broad categories… 114 references, cited by 48 papers here."
authors: ["Faeze Brahman", "Sachin Kumar", "Vidhisha Balachandran", "Pradeep Dasigi", "Valentina Pyatkin", "Abhilasha Ravichander", "Sarah Wiegreffe", "Nouha Dziri", "Khyathi Chandu", "Jack Hessel", "Yulia Tsvetkov", "Noah A. Smith", "Yejin Choi", "Hannaneh Hajishirzi"]
year: 2024
venue: "arXiv:2407.12043"
source: "https://arxiv.org/abs/2407.12043"
doi: "10.48550/arxiv.2407.12043"
arxiv: "2407.12043"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/brahman-2024-art-saying-no-contextual-noncompliance/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/brahman-2024-art-saying-no-contextual-noncompliance/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/5/) / The Art of Saying No: Contextual Noncompliance in Language Models

# The Art of Saying No: Contextual Noncompliance in Language Models

**Authors:** [Faeze Brahman](https://md.research.elamur.ai/free/en/authors/faeze-brahman/), [Sachin Kumar](https://md.research.elamur.ai/free/en/authors/sachin-kumar/), [Vidhisha Balachandran](https://md.research.elamur.ai/free/en/authors/vidhisha-balachandran/), [Pradeep Dasigi](https://md.research.elamur.ai/free/en/authors/pradeep-dasigi/), [Valentina Pyatkin](https://md.research.elamur.ai/free/en/authors/valentina-pyatkin/), [Abhilasha Ravichander](https://md.research.elamur.ai/free/en/authors/abhilasha-ravichander/), [Sarah Wiegreffe](https://md.research.elamur.ai/free/en/authors/sarah-wiegreffe/), [Nouha Dziri](https://md.research.elamur.ai/free/en/authors/nouha-dziri/), [Khyathi Chandu](https://md.research.elamur.ai/free/en/authors/khyathi-chandu/), [Jack Hessel](https://md.research.elamur.ai/free/en/authors/jack-hessel/), [Yulia Tsvetkov](https://md.research.elamur.ai/free/en/authors/yulia-tsvetkov/), [Noah A. Smith](https://md.research.elamur.ai/free/en/authors/noah-a-smith/), [Yejin Choi](https://md.research.elamur.ai/free/en/authors/yejin-choi/) and [Hannaneh Hajishirzi](https://md.research.elamur.ai/free/en/authors/hannaneh-hajishirzi/) · **Year:** 2024 · **Venue:** [arXiv:2407.12043](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/) ([NeurIPS 2024](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/2024/)) · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2407.12043) · [PDF](https://arxiv.org/pdf/2407.12043) · [DOI 10.48550/arxiv.2407.12043](https://doi.org/10.48550/arxiv.2407.12043) · [arXiv 2407.12043](https://arxiv.org/abs/2407.12043) · [OpenAlex](https://openalex.org/W7103750124)

**Citations:** 8 citations (OpenAlex, 2026-10-14) · [48](https://md.research.elamur.ai/free/en/papers/brahman-2024-art-saying-no-contextual-noncompliance/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Most research on language model refusal treats it as a safety issue. This paper argues that chat models should also decline or qualify many requests that are not harmful. The authors propose a taxonomy of contextual noncompliance with five broad categories: incomplete, indeterminate, unsupported, humanizing and safety-related requests. Each category has finer subcategories and its own recommended response style, which ranges from outright refusal to stating a limitation or adding a disclaimer.

From this taxonomy they build CoCoNot, which has two parts:
- a human-verified evaluation set, plus a contrastive set of prompts that models should answer, used to detect over-refusal;
- a synthetic training corpus of requests paired with appropriate noncompliant replies.

Strong models such as GPT-4 and Llama-3 still wrongly comply with up to 30% of requests in some categories. Adding the corpus to instruction tuning from a base model works, but it is costly and causes over-refusal. Continued LoRA finetuning of already instruction-tuned models balances noncompliance with general capability more effectively. The work broadens how model trustworthiness is evaluated.

## Key points
- The taxonomy covers five categories, and a single request can belong to several. Examples include false presuppositions, underspecified queries, universal and model unknowns, and modality, length and temporal limits. It also covers anthropomorphizing requests and safety risks such as privacy and copyright violations.
- The CoCoNot evaluation suite contains 1000 noncompliance prompts, released at huggingface.co/datasets/allenai/coconot. Code and models are at github.com/allenai/noncompliance.
- The largest gaps appear in the "incomplete" and "unsupported" categories, where GPT-4 and Llama-3 comply incorrectly up to 30% of the time.
- Full finetuning of instruction-tuned models degrades general skills and increases over-refusal. LoRA continued finetuning avoids most of this, and preference tuning further reduces over-refusals.
- Limitations:
  - Nearly all of the data is synthetic and was generated with GPT models.
  - The safety norms rely on a US-centric notion of legality.
  - Access to the dataset is gated behind a content warning and terms of use.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 543 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🛡️ Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/) · 517 works

## Cite

Cite as: Faeze Brahman, Sachin Kumar, Vidhisha Balachandran, Pradeep Dasigi, Valentina Pyatkin, Abhilasha Ravichander, Sarah Wiegreffe, Nouha Dziri, Khyathi Chandu, Jack Hessel, Yulia Tsvetkov, Noah A. Smith, Yejin Choi and Hannaneh Hajishirzi. 2024. The Art of Saying No: Contextual Noncompliance in Language Models. arXiv:2407.12043.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{brahman2024art,
  title = {{The Art of Saying No: Contextual Noncompliance in Language Models}},
  author = {Faeze Brahman, Sachin Kumar, Vidhisha Balachandran, Pradeep Dasigi, Valentina Pyatkin, Abhilasha Ravichander, Sarah Wiegreffe, Nouha Dziri, Khyathi Chandu, Jack Hessel, Yulia Tsvetkov, Noah A. Smith, Yejin Choi and Hannaneh Hajishirzi},
  year = {2024},
  doi = {10.48550/arxiv.2407.12043},
  eprint = {2407.12043},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2407.12043}
}
```

- [← previous Massively Multi-Cultural Knowledge Acquisition & LM Benchmarking](https://md.research.elamur.ai/free/en/papers/fung-2024-massively-multi-cultural-knowledge-acquisition/)
- [next → Democratizing LLMs for Low-Resource Languages by Leveraging their English Dominant Abilities with Linguistically-Diverse Prompts](https://md.research.elamur.ai/free/en/papers/nguyen-2024-democratizing-llms-low-resource-languages/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/5/)

## References

114 references, 55 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Gavin Abercrombie, Amanda Cercas Curry, Tanvi Dinkar, Verena Rieser, and Zeerak Talat. Mirages. on anthropomorphism in dialogue systems. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 4776–4790, Singapo… [find](https://scholar.google.com/scholar?q=Gavin+Abercrombie%2C+Amanda+Cercas+Curry%2C+Tanvi+Dinkar%2C+Verena+Rieser%2C+and+Zeerak+Talat%2E+Mirages%2E+on+anthropomorphism+in+dialogue+systems%2E+In+Houda+Bouamor%2C+Juan+Pino%2C+and+Kalika+Bali+%28eds%2E%29%2C+Proceeding)
2. Gustaf Ahdritz, Tian Qin, Nikhil Vyas, Boaz Barak, and Benjamin L. Edelman. Distinguishing the knowable from the unknowable with language models, 2024. URL [https://arxiv.org/](https://arxiv.org/) abs/2402.03563. — [arXiv:2402.03563](https://arxiv.org/abs/2402.03563) [↗](https://arxiv.org/)
3. Alfonso Amayuelas, Liangming Pan, Wenhu Chen, and William Wang. [Knowledge of knowledge: Exploring known-unknowns uncertainty with large language models](https://md.research.elamur.ai/free/en/papers/amayuelas-2024-knowledge-knowledge-exploring-known-unknowns/). arXiv preprint arXiv:2305.13712, 2023. URL https://arxiv.org/abs/2305.13712. [↗](https://arxiv.org/abs/2305.13712)
4. Anthropic. Introducing the next generation of claude. 2024. URL [https://www.anthropic.com/news/claude-3-family](https://www.anthropic.com/news/claude-3-family). [↗](https://www.anthropic.com/news/claude-3-family)
5. Usman Anwar, Abulhair Saparov, Javier Rando, Daniel Paleka, Miles Turpin, Peter Hase, Ekdeep Singh Lubana, Erik Jenner, Stephen Casper, Oliver Sourbut, Benjamin L. Edelman, Zhaowei Zhang, Mario Günther, Anton Korinek, Jose Hernandez-Orallo, Lewis Hammond, Eric Bigelow, Alexander Pan, Lauro Langosco,… [find](https://scholar.google.com/scholar?q=Usman+Anwar%2C+Abulhair+Saparov%2C+Javier+Rando%2C+Daniel+Paleka%2C+Miles+Turpin%2C+Peter+Hase%2C+Ekdeep+Singh+Lubana%2C+Erik+Jenner%2C+Stephen+Casper%2C+Oliver+Sourbut%2C+Benjamin+L%2E+Edelman%2C+Zhaowei+Zhang%2C+Mario+G%C3%BCnthe)
6. Akari Asai and Eunsol Choi. Challenges in information-seeking QA: Unanswerable questions and paragraph retrieval. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Join… [find](https://scholar.google.com/scholar?q=Akari+Asai+and+Eunsol+Choi%2E+Challenges+in+information%2Dseeking+QA%3A+Unanswerable+questions+and+paragraph+retrieval%2E+In+Chengqing+Zong%2C+Fei+Xia%2C+Wenjie+Li%2C+and+Roberto+Navigli+%28eds%2E%29%2C+Proceedings+of+the+)
7. Joris Baan, Nico Daheim, Evgenia Ilia, Dennis Ulmer, Haau-Sing Li, Raquel Fernández, Barbara Plank, Rico Sennrich, Chrysoula Zerva, and Wilker Aziz. Uncertainty in natural language generation: From theory to applications. 2023. URL https://arxiv.org/abs/ 2307.15703. [↗](https://arxiv.org/abs/)
8. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova Dassarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, John Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Shaun… [find](https://scholar.google.com/scholar?q=Yuntao+Bai%2C+Andy+Jones%2C+Kamal+Ndousse%2C+Amanda+Askell%2C+Anna+Chen%2C+Nova+Dassarma%2C+Dawn+Drain%2C+Stanislav+Fort%2C+Deep+Ganguli%2C+Tom+Henighan%2C+Nicholas+Joseph%2C+Saurav+Kadavath%2C+John+Kernion%2C+Tom+Conerly%2C+She)
9. Michele Banko, Brendon MacKeen, and Laurie Ray. A unified taxonomy of harmful content. In Seyi Akiwowo, Bertie Vidgen, Vinodkumar Prabhakaran, and Zeerak Waseem (eds.), Proceedings of the Fourth Workshop on Online Abuse and Harms, pp. 125–137, Online, November 2020. Association for Computational Lin… [find](https://scholar.google.com/scholar?q=Michele+Banko%2C+Brendon+MacKeen%2C+and+Laurie+Ray%2E+A+unified+taxonomy+of+harmful+content%2E+In+Seyi+Akiwowo%2C+Bertie+Vidgen%2C+Vinodkumar+Prabhakaran%2C+and+Zeerak+Waseem+%28eds%2E%29%2C+Proceedings+of+the+Fourth+Works)
10. Federico Bianchi, Mirac Suzgun, Giuseppe Attanasio, Paul Rottger, Dan Jurafsky, Tatsunori Hashimoto, and James Zou. [Safety-tuned LLaMAs: Lessons from improving the safety of large language models that follow instructions](https://md.research.elamur.ai/free/en/papers/bianchi-2023-safety-tuned-llamas-lessons-improving/). In The Twelfth International Conference on Learning Representations, 2024. URL…
11. Dan Biderman, Jacob Portes, Jose Javier Gonzalez Ortiz, Mansheej Paul, Philip Greengard, Connor Jennings, Daniel King, Sam Havens, Vitaliy Chiley, Jonathan Frankle, Cody Blakeney, and John Patrick Cunningham. LoRA learns less and forgets less. Transactions on Machine Learning Research, 2024. ISSN 28… [find](https://scholar.google.com/scholar?q=Dan+Biderman%2C+Jacob+Portes%2C+Jose+Javier+Gonzalez+Ortiz%2C+Mansheej+Paul%2C+Philip+Greengard%2C+Connor+Jennings%2C+Daniel+King%2C+Sam+Havens%2C+Vitaliy+Chiley%2C+Jonathan+Frankle%2C+Cody+Blakeney%2C+and+John+Patrick+Cun)
12. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeff Wu, Clemens Winter, Chr… — [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
13. Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom B Brown, Dawn Song, Ulfar Erlingsson, et al. [Extracting training data from large language models](https://md.research.elamur.ai/free/en/papers/carlini-2020-extracting-training-data-large-language/). In USENIX Security Symposium, volume 6, 2021.
14. Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. Quantifying memorization across neural language models. In The Eleventh International Conference on Learning Representations, 2023. URL [https://openreview.net/forum?id=TatRHT_1cK](https://openreview.net/forum?id=TatRHT_1cK). [↗](https://openreview.net/forum?id=TatRHT_1cK)
15. Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, and Eric Wong. [Jailbreaking black box large language models in twenty queries](https://md.research.elamur.ai/free/en/papers/chao-2023-jailbreaking-black-box-large-language/), 2023.
16. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. [Evaluating large language models trained on code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/). arXiv e-prints, pp. arXiv–2107, 2021.
17. Jeffrey Cheng, Marc Marone, Orion Weller, Dawn Lawrie, Daniel Khashabi, and Benjamin Van Durme. Dated data: Tracing knowledge cutoffs in large language models, 2024. [find](https://scholar.google.com/scholar?q=Jeffrey+Cheng%2C+Marc+Marone%2C+Orion+Weller%2C+Dawn+Lawrie%2C+Daniel+Khashabi%2C+and+Benjamin+Van+Durme%2E+Dated+data%3A+Tracing+knowledge+cutoffs+in+large+language+models%2C+2024%2E)
18. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023. URL [https://lmsys.org/blog/2023-03-30-vicuna/…](https://lmsys.org/blog/2023-03-30-vicuna/…) [↗](https://lmsys.org/blog/2023-03-30-vicuna/…)
19. Christopher Clark and Matt Gardner. [Simple and effective multi-paragraph reading comprehension](https://md.research.elamur.ai/free/en/papers/clark-2017-simple-effective-multi-paragraph-reading/). In Iryna Gurevych and Yusuke Miyao (eds.), Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 845–855, Melbourne, Australia, July 2018. As…
20. Jonathan H. Clark, Eunsol Choi, Michael Collins, Dan Garrette, Tom Kwiatkowski, Vitaly Nikolaev, and Jennimaria Palomaki. [TyDi QA: A benchmark for information-seeking question answering in typologically diverse languages](https://md.research.elamur.ai/free/en/papers/clark-2020-tydi-qa-benchmark-information-seeking/). Transactions of the Association for Computational Linguistics, 8:454–470, 2020…
21. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/), 2021. URL https://arxiv.org/abs/2110.14168. [↗](https://arxiv.org/abs/2110.14168)
22. Lorraine Code. Epistemic Responsibility. Published for Brown University Press by University Press of New England, Hanover, N.H., 1987. [find](https://scholar.google.com/scholar?q=Lorraine+Code%2E+Epistemic+Responsibility%2E+Published+for+Brown+University+Press+by+University+Press+of+New+England%2C+Hanover%2C+N%2EH%2E%2C+1987%2E)
23. Josef Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji, Xinbo Xu, Mickel Liu, Yizhou Wang, and Yaodong Yang. [Safe RLHF: Safe reinforcement learning from human feedback](https://md.research.elamur.ai/free/en/papers/dai-2023-safe-rlhf-safe-reinforcement-learning/). In The Twelfth International Conference on Learning Representations, 2024. URL [https://openreview.net/forum?id=TyFrPOKYXw](https://openreview.net/forum?id=TyFrPOKYXw). [↗](https://openreview.net/forum?id=TyFrPOKYXw)
24. Leon Derczynski, Hannah Rose Kirk, Vidhisha Balachandran, Sachin Kumar, Yulia Tsvetkov, M. R. Leiser, and Saif Mohammad. Assessing language model deployment with risk cards, 2023. URL https://arxiv.org/abs/2303.18190. — [arXiv:2303.18190](https://arxiv.org/abs/2303.18190) [↗](https://arxiv.org/abs/2303.18190)
25. Erik Derner and Kristina Batistič. Beyond the safeguards: Exploring the security risks of chatgpt, 2023. URL https://arxiv.org/abs/2305.08005. — [arXiv:2305.08005](https://arxiv.org/abs/2305.08005) [↗](https://arxiv.org/abs/2305.08005)
26. Shrey Desai and Greg Durrett. Calibration of pre-trained transformers. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 295–302, Online, November 2020. Association for Computational Lingu… [find](https://scholar.google.com/scholar?q=Shrey+Desai+and+Greg+Durrett%2E+Calibration+of+pre%2Dtrained+transformers%2E+In+Bonnie+Webber%2C+Trevor+Cohn%2C+Yulan+He%2C+and+Yang+Liu+%28eds%2E%29%2C+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natural+)
27. Shangbin Feng, Weijia Shi, Yike Wang, Wenxuan Ding, Orevaoghene Ahia, Shuyue Stella Li, Vidhisha Balachandran, Sunayana Sitaram, and Yulia Tsvetkov. Teaching LLMs to abstain across languages via multilingual feedback. In Empirical Methods in Natural Language Processing (EMNLP), 2024. [find](https://scholar.google.com/scholar?q=Shangbin+Feng%2C+Weijia+Shi%2C+Yike+Wang%2C+Wenxuan+Ding%2C+Orevaoghene+Ahia%2C+Shuyue+Stella+Li%2C+Vidhisha+Balachandran%2C+Sunayana+Sitaram%2C+and+Yulia+Tsvetkov%2E+Teaching+LLMs+to+abstain+across+languages+via+multi)
28. Shangbin Feng, Weijia Shi, Yike Wang, Wenxuan Ding, Vidhisha Balachandran, and Yulia Tsvetkov. Don’t hallucinate, abstain: Identifying LLM knowledge gaps via multi-LLM collaboration. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association fo… [find](https://scholar.google.com/scholar?q=Shangbin+Feng%2C+Weijia+Shi%2C+Yike+Wang%2C+Wenxuan+Ding%2C+Vidhisha+Balachandran%2C+and+Yulia+Tsvetkov%2E+Don%E2%80%99t+hallucinate%2C+abstain%3A+Identifying+LLM+knowledge+gaps+via+multi%2DLLM+collaboration%2E+In+Lun%2DWei+Ku%2C+An)
29. Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, Andy Jones, Sam Bowman, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Nelson Elhage, Sheer El-Showk, Stanislav Fort, Zac Hatfield-Dodds, Tom Henighan… — [Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned](https://md.research.elamur.ai/free/en/papers/ganguli-2022-red-teaming-language-models-reduce/)
30. Matt Gardner, Yoav Artzi, Victoria Basmov, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai Elazar, Ananth Gottumukkala, Nitish Gupta, Hannaneh Hajishirzi, Gabriel Ilharco, Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F. Liu, Phoebe Mulcaire, Qiang Ning, Sameer Singh, N… — [Evaluating Models' Local Decision Boundaries via Contrast Sets](https://md.research.elamur.ai/free/en/papers/gardner-2020-evaluating-models-local-decision-boundaries/)
31. Ian J. Goodfellow, Mehdi Mirza, Da Xiao, Aaron Courville, and Yoshua Bengio. An empirical investigation of catastrophic forgetting in gradient-based neural networks, 2015. [find](https://scholar.google.com/scholar?q=Ian+J%2E+Goodfellow%2C+Mehdi+Mirza%2C+Da+Xiao%2C+Aaron+Courville%2C+and+Yoshua+Bengio%2E+An+empirical+investigation+of+catastrophic+forgetting+in+gradient%2Dbased+neural+networks%2C+2015%2E)
32. Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar. Toxigen: Controlling language models to generate implied and adversarial toxicity. In Annual Meeting of the Association for Computational Linguistics, volume 1, 2022. [find](https://scholar.google.com/scholar?q=Thomas+Hartvigsen%2C+Saadia+Gabriel%2C+Hamid+Palangi%2C+Maarten+Sap%2C+Dipankar+Ray%2C+and+Ece+Kamar%2E+Toxigen%3A+Controlling+language+models+to+generate+implied+and+adversarial+toxicity%2E+In+Annual+Meeting+of+the+)
33. Richard Heersmink, Barend de Rooij, María Jimena Clavel Vázquez, and Matteo Colombo. A phenomenology and epistemology of large language models: Transparency, trust, and trustworthiness. [find](https://scholar.google.com/scholar?q=Richard+Heersmink%2C+Barend+de+Rooij%2C+Mar%C3%ADa+Jimena+Clavel+V%C3%A1zquez%2C+and+Matteo+Colombo%2E+A+phenomenology+and+epistemology+of+large+language+models%3A+Transparency%2C+trust%2C+and+trustworthiness%2E)
34. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). In International Conference on Learning Representations, 2020.
35. Joel Hestness, Sharan Narang, Newsha Ardalani, Gregory Diamos, Heewoo Jun, Hassan Kianinejad, Md Mostofa Ali Patwary, Yang Yang, and Yanqi Zhou. [Deep learning scaling is predictable, empirically](https://md.research.elamur.ai/free/en/papers/hestness-2017-deep-learning-scaling-predictable-empirically/). arXiv preprint arXiv:1712.00409, 2017.
36. Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. [Training compute-optimal large language models](https://md.research.elamur.ai/free/en/papers/hoffmann-2022-training-compute-optimal-large-language/). arXiv preprint arXiv:2203.15556, 2022.
37. Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. [LoRA: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). In International Conference on Learning Representations, 2022. URL [https://openreview.net/forum?id=nZeVKeeFYf9](https://openreview.net/forum?id=nZeVKeeFYf9). [↗](https://openreview.net/forum?id=nZeVKeeFYf9)
38. Chengsong Huang, Qian Liu, Bill Yuchen Lin, Tianyu Pang, Chao Du, and Min Lin. [Lorahub: Efficient cross-task generalization via dynamic lora composition](https://md.research.elamur.ai/free/en/papers/huang-2023-lorahub-efficient-cross-task-generalization/). arXiv preprint arXiv:2307.13269, 2023.
39. Jie Huang, Hanyin Shao, and Kevin Chen-Chuan Chang. Are large pre-trained language models leaking your personal information? EMNLP Findings, 2022. [find](https://scholar.google.com/scholar?q=Jie+Huang%2C+Hanyin+Shao%2C+and+Kevin+Chen%2DChuan+Chang%2E+Are+large+pre%2Dtrained+language+models+leaking+your+personal+information%3F+EMNLP+Findings%2C+2022%2E)
40. Hamish Ivison, Yizhong Wang, Valentina Pyatkin, Nathan Lambert, Matthew Peters, Pradeep Dasigi, Joel Jang, David Wadden, Noah A. Smith, Iz Beltagy, and Hannaneh Hajishirzi. [Camels in a changing climate: Enhancing lm adaptation with tulu 2](https://md.research.elamur.ai/free/en/papers/ivison-2023-camels-changing-climate-enhancing-lm/), 2023.
41. Alon Jacovi, Ana Marasović, Tim Miller, and Yoav Goldberg. Formalizing trust in artificial intelligence: Prerequisites, causes and goals of human trust in ai. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21, pp. 624–635, New York, NY, USA, 2021. As… — [doi:10.1145/3442188.3445923](https://doi.org/10.1145/3442188.3445923) [find](https://scholar.google.com/scholar?q=Alon+Jacovi%2C+Ana+Marasovic%CC%81%2C+Tim+Miller%2C+and+Yoav+Goldberg%2E+Formalizing+trust+in+artificial+intelligence%3A+Prerequisites%2C+causes+and+goals+of+human+trust+in+ai%2E+In+Proceedings+of+the+2021+ACM+Conferenc)
42. Robin Jia and Percy Liang. [Adversarial examples for evaluating reading comprehension systems](https://md.research.elamur.ai/free/en/papers/jia-2017-adversarial-examples-evaluating-reading-comprehension/). In Martha Palmer, Rebecca Hwa, and Sebastian Riedel (eds.), Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp. 2021–2031, Copenhagen, Denmark, September 2017. Associ…
43. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix,… [find](https://scholar.google.com/scholar?q=Albert+Q%2E+Jiang%2C+Alexandre+Sablayrolles%2C+Arthur+Mensch%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guillaume+Lample%2C+Lucile+Saulnier%2C+L%C3%A9lio+Renard+Lav)
44. Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gianna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock,… [find](https://scholar.google.com/scholar?q=Albert+Q%2E+Jiang%2C+Alexandre+Sablayrolles%2C+Antoine+Roux%2C+Arthur+Mensch%2C+Blanche+Savary%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Emma+Bou+Hanna%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guill)
45. Liwei Jiang, Kavel Rao, Seungju Han, Allyson Ettinger, Faeze Brahman, Sachin Kumar, Niloofar Mireshghallah, Ximing Lu, Maarten Sap, Yejin Choi, and Nouha Dziri. [Wildteaming at scale: From in-the-wild jailbreaks to (adversarially) safer language models](https://md.research.elamur.ai/free/en/papers/jiang-2024-wildteaming-scale-wild-jailbreaks-adversarially/). In Proceedings of the Neural Information Proces…
46. Zhengbao Jiang, Jun Araki, Haibo Ding, and Graham Neubig. How can we know when language models know? on the calibration of language models for question answering. Transactions of the Association for Computational Linguistics, 9:962–977, 2021. [find](https://scholar.google.com/scholar?q=Zhengbao+Jiang%2C+Jun+Araki%2C+Haibo+Ding%2C+and+Graham+Neubig%2E+How+can+we+know+when+language+models+know%3F+on+the+calibration+of+language+models+for+question+answering%2E+Transactions+of+the+Association+for+C)
47. Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac Hatfield-Dodds, Nova DasSarma, Eli Tran-Johnson, et al. Language models (mostly) know what they know. arXiv preprint arXiv:2207.05221, 2022. [find](https://scholar.google.com/scholar?q=Saurav+Kadavath%2C+Tom+Conerly%2C+Amanda+Askell%2C+Tom+Henighan%2C+Dawn+Drain%2C+Ethan+Perez%2C+Nicholas+Schiefer%2C+Zac+Hatfield%2DDodds%2C+Nova+DasSarma%2C+Eli+Tran%2DJohnson%2C+et+al%2E+Language+models+%28mostly%29+know+what+th)
48. Amita Kamath, Robin Jia, and Percy Liang. Selective question answering under domain shift. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 5684–5696, Online, July 2020. Association… [find](https://scholar.google.com/scholar?q=Amita+Kamath%2C+Robin+Jia%2C+and+Percy+Liang%2E+Selective+question+answering+under+domain+shift%2E+In+Dan+Jurafsky%2C+Joyce+Chai%2C+Natalie+Schluter%2C+and+Joel+Tetreault+%28eds%2E%29%2C+Proceedings+of+the+58th+Annual+Meet)
49. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. [Scaling laws for neural language models](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/). arXiv preprint arXiv:2001.08361, 2020.
50. Antonia Karamolegkou, Jiaang Li, Li Zhou, and Anders Søgaard. [Copyright violations and large language models](https://md.research.elamur.ai/free/en/papers/karamolegkou-2023-copyright-violations-large-language-models/). In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 7403–7412, Singapore, December 2023. Associat…
51. Kimiya Keyvan and Jimmy Xiangji Huang. How to approach ambiguous queries in conversational search: A survey of techniques, approaches, tools, and challenges. ACM Comput. Surv., 55(6), dec 2022. ISSN 0360-0300. doi: 10.1145/3534965. URL https://doi.org/10.1145/3534965. [↗](https://doi.org/10.1145/3534965)
52. Minsu Kim and James Thorne. Epistemology of language models: Do language models have holistic knowledge? arXiv preprint arXiv:2403.12862, 2024. [find](https://scholar.google.com/scholar?q=Minsu+Kim+and+James+Thorne%2E+Epistemology+of+language+models%3A+Do+language+models+have+holistic+knowledge%3F+arXiv+preprint+arXiv%3A2403%2E12862%2C+2024%2E)
53. Najoung Kim, Phu Mon Htut, Samuel R Bowman, and Jackson Petty. Qa 2 : Question answering with questionable assumptions. arXiv preprint arXiv:2212.10003, 2022. [find](https://scholar.google.com/scholar?q=Najoung+Kim%2C+Phu+Mon+Htut%2C+Samuel+R+Bowman%2C+and+Jackson+Petty%2E+Qa+2+%3A+Question+answering+with+questionable+assumptions%2E+arXiv+preprint+arXiv%3A2212%2E10003%2C+2022%2E)
54. Hannah Rose Kirk, Bertie Vidgen, Paul Röttger, and Scott A Hale. [Personalisation within bounds: A risk taxonomy and policy framework for the alignment of large language models with personalised feedback](https://md.research.elamur.ai/free/en/papers/kirk-2023-personalisation-within-bounds-risk-taxonomy/). arXiv preprint arXiv:2303.05453, 2023.
55. James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming catastrophic forgetting in neural networks. Proc… [find](https://scholar.google.com/scholar?q=James+Kirkpatrick%2C+Razvan+Pascanu%2C+Neil+Rabinowitz%2C+Joel+Veness%2C+Guillaume+Desjardins%2C+Andrei+A%2E+Rusu%2C+Kieran+Milan%2C+John+Quan%2C+Tiago+Ramalho%2C+Agnieszka+Grabska%2DBarwinska%2C+Demis+Hassabis%2C+Claudia+Clop)
56. Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. Clam: Selective clarification for ambiguous questions with generative language models. In ICML Workshop on Deployable Generative AI, 2023. URL [https://openreview.net/forum?id=VQWuqgSoVN\#all](https://openreview.net/forum?id=VQWuqgSoVN\#all). [↗](https://openreview.net/forum?id=VQWuqgSoVN\#all)
57. Sachin Kumar, Vidhisha Balachandran, Lucille Njoo, Antonios Anastasopoulos, and Yulia Tsvetkov. [Language generation models can cause harm: So what can we do about it? an actionable survey](https://md.research.elamur.ai/free/en/papers/kumar-2023-language-generation-models-can-cause/). In Andreas Vlachos and Isabelle Augenstein (eds.), Proceedings of the 17th Conference of the European Chapter o…
58. Seanie Lee, Minsu Kim, Lynn Cherif, David Dobre, Juho Lee, Sung Ju Hwang, Kenji Kawaguchi, Gauthier Gidel, Yoshua Bengio, Nikolay Malkin, et al. Learning diverse attacks on large language models for robust red-teaming and safety tuning. arXiv preprint arXiv:2405.18540, 2024. — [arXiv:2405.18540](https://arxiv.org/abs/2405.18540) [find](https://scholar.google.com/scholar?q=Seanie+Lee%2C+Minsu+Kim%2C+Lynn+Cherif%2C+David+Dobre%2C+Juho+Lee%2C+Sung+Ju+Hwang%2C+Kenji+Kawaguchi%2C+Gauthier+Gidel%2C+Yoshua+Bengio%2C+Nikolay+Malkin%2C+et+al%2E+Learning+diverse+attacks+on+large+language+models+for+r)
59. California Legislature. Senate bill no. 1001. [https://digitaldemocracy.calmatters.org/bills/ca_201720180sb1001](https://digitaldemocracy.calmatters.org/bills/ca_201720180sb1001). Accessed: 2024-06-05. [↗](https://digitaldemocracy.calmatters.org/bills/ca_201720180sb1001)
60. Omer Levy, Minjoon Seo, Eunsol Choi, and Luke Zettlemoyer. [Zero-shot relation extraction via reading comprehension](https://md.research.elamur.ai/free/en/papers/levy-2017-zero-shot-relation-extraction-reading/). In Roger Levy and Lucia Specia (eds.), Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017), pp. 333– 342, Vancouver, Canada, August 2017. Associa…
61. Haoran Li, Dadi Guo, Wei Fan, Mingshi Xu, and Yangqiu Song. [Multi-step jailbreaking privacy attacks on chatgpt](https://md.research.elamur.ai/free/en/papers/li-2023-multi-step-jailbreaking-privacy-attacks/). arXiv preprint arXiv:2304.05197, 2023.
62. Shuyue Stella Li, Vidhisha Balachandran, Shangbin Feng, Jonathan Ilgen, Emma Pierson, Pang Wei Koh, and Yulia Tsvetkov. MediQ: Question-asking LLMs for adaptive and reliable clinical reasoning. In Advances in Neural Information Processing Systems, 2024. [find](https://scholar.google.com/scholar?q=Shuyue+Stella+Li%2C+Vidhisha+Balachandran%2C+Shangbin+Feng%2C+Jonathan+Ilgen%2C+Emma+Pierson%2C+Pang+Wei+Koh%2C+and+Yulia+Tsvetkov%2E+MediQ%3A+Question%2Dasking+LLMs+for+adaptive+and+reliable+clinical+reasoning%2E+In+Adv)
63. Tao Li, Daniel Khashabi, Tushar Khot, Ashish Sabharwal, and Vivek Srikumar. [UNQOVERing stereotyping biases via underspecified questions](https://md.research.elamur.ai/free/en/papers/li-2020-unqovering-stereotyping-biases-underspecified-questions/). In Findings of the Association for Computational Linguistics: EMNLP 2020, pp. 3475–3489, Online, November 2020. Association for Computational Linguistics. doi: 10.…
64. Xuan Li, Zhanke Zhou, Jianing Zhu, Jiangchao Yao, Tongliang Liu, and Bo Han. [Deepinception: Hypnotize large language model to be jailbreaker](https://md.research.elamur.ai/free/en/papers/li-2023-deepinception-hypnotize-large-language-model/), 2024.
65. Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Alpacaeval: An automatic evaluator of instruction-following models. [https://github.com/tatsu-lab/alpaca_eval](https://github.com/tatsu-lab/alpaca_eval), 2023. [↗](https://github.com/tatsu-lab/alpaca_eval)
66. Stephanie Lin, Jacob Hilton, and Owain Evans. [Truthfulqa: Measuring how models mimic human falsehoods](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3214–3252, 2022.
67. Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. [Autodan: Generating stealthy jailbreak prompts on aligned large language models](https://md.research.elamur.ai/free/en/papers/liu-2023-autodan-generating-stealthy-jailbreak-prompts/), 2023.
68. Yang Liu, Yuanshun Yao, Jean-Francois Ton, Xiaoying Zhang, Ruocheng Guo Hao Cheng, Yegor Klochkov, Muhammad Faaiz Taufiq, and Hang Li. Trustworthy llms: a survey and guideline for evaluating large language models’ alignment. arXiv preprint arXiv:2308.[05374, 2023](https://md.research.elamur.ai/free/en/papers/liu-2023-trustworthy-llms-survey-guideline-evaluating/).
69. Nils Lukas, Ahmed Salem, Robert Sim, Shruti Tople, Lukas Wutschitz, and Santiago Zanella-Béguelin. Analyzing leakage of personally identifiable information in language models. arXiv preprint arXiv:2302.00539, 2023. [find](https://scholar.google.com/scholar?q=Nils+Lukas%2C+Ahmed+Salem%2C+Robert+Sim%2C+Shruti+Tople%2C+Lukas+Wutschitz%2C+and+Santiago+Zanella%2DB%C3%A9guelin%2E+Analyzing+leakage+of+personally+identifiable+information+in+language+models%2E+arXiv+preprint+arXiv%3A230)
70. Bodhisattwa Prasad Majumder, Sudha Rao, Michel Galley, and Julian McAuley. Ask what’s missing and what’s useful: Improving clarification question generation using global knowledge. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: H… [find](https://scholar.google.com/scholar?q=Bodhisattwa+Prasad+Majumder%2C+Sudha+Rao%2C+Michel+Galley%2C+and+Julian+McAuley%2E+Ask+what%E2%80%99s+missing+and+what%E2%80%99s+useful%3A+Improving+clarification+question+generation+using+global+knowledge%2E+In+Proceedings+of+t)
71. Todor Markov, Chong Zhang, Sandhini Agarwal, Florentine Eloundou Nekoul, Theodore Lee, Steven Adler, Angela Jiang, and Lilian Weng. [A holistic approach to undesired content detection in the real world](https://md.research.elamur.ai/free/en/papers/markov-2022-holistic-approach-undesired-content-detection/). Proceedings of the AAAI Conference on Artificial Intelligence, 37 (12):15009–15018, Jun. 2023. doi…
72. Todor Markov, Chong Zhang, Sandhini Agarwal, Florentine Eloundou Nekoul, Theodore Lee, Steven Adler, Angela Jiang, and Lilian Weng. [A holistic approach to undesired content detection in the real world](https://md.research.elamur.ai/free/en/papers/markov-2022-holistic-approach-undesired-content-detection/). Proceedings of the AAAI Conference on Artificial Intelligence, 37 (12):15009–15018, Jun. 2023. doi…
73. Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, David Forsyth, and Dan Hendrycks. [Harmbench: A standardized evaluation framework for automated red teaming and robust refusal](https://md.research.elamur.ai/free/en/papers/mazeika-2024-harmbench-standardized-evaluation-framework-automated/), 2024.
74. Sabrina J Mielke, Arthur Szlam, Emily Dinan, and Y-Lan Boureau. Reducing conversational agents’ overconfidence through linguistic calibration. Transactions of the Association for Computational Linguistics, 10:857–872, 2022. [find](https://scholar.google.com/scholar?q=Sabrina+J+Mielke%2C+Arthur+Szlam%2C+Emily+Dinan%2C+and+Y%2DLan+Boureau%2E+Reducing+conversational+agents%E2%80%99+overconfidence+through+linguistic+calibration%2E+Transactions+of+the+Association+for+Computational+Linguis)
75. Sewon Min, Julian Michael, Hannaneh Hajishirzi, and Luke Zettlemoyer. Ambigqa: Answering ambiguous open-domain questions. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 5783–5797, 2020. [find](https://scholar.google.com/scholar?q=Sewon+Min%2C+Julian+Michael%2C+Hannaneh+Hajishirzi%2C+and+Luke+Zettlemoyer%2E+Ambigqa%3A+Answering+ambiguous+open%2Ddomain+questions%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natural+Language)
76. Yasumasa Onoe, Michael J.Q. Zhang, Eunsol Choi, and Greg Durrett. Creak: A dataset for commonsense reasoning over entity knowledge. OpenReview, 2021. [find](https://scholar.google.com/scholar?q=Yasumasa+Onoe%2C+Michael+J%2EQ%2E+Zhang%2C+Eunsol+Choi%2C+and+Greg+Durrett%2E+Creak%3A+A+dataset+for+commonsense+reasoning+over+entity+knowledge%2E+OpenReview%2C+2021%2E)
77. OpenAI. Introducing chatgpt. 2022. URL [https://openai.com/blog/chatgpt](https://openai.com/blog/chatgpt). [↗](https://openai.com/blog/chatgpt)
78. OpenAI. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/), 2023.
79. Valentina Pyatkin, Jena D Hwang, Vivek Srikumar, Ximing Lu, Liwei Jiang, Yejin Choi, and Chandra Bhagavatula. Clarifydelphi: Reinforced clarification questions with defeasibility rewards for social and moral situations. In Proceedings of the 61st Annual Meeting of the Association for Computational L… [find](https://scholar.google.com/scholar?q=Valentina+Pyatkin%2C+Jena+D+Hwang%2C+Vivek+Srikumar%2C+Ximing+Lu%2C+Liwei+Jiang%2C+Yejin+Choi%2C+and+Chandra+Bhagavatula%2E+Clarifydelphi%3A+Reinforced+clarification+questions+with+defeasibility+rewards+for+social+an)
80. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. [Direct preference optimization: Your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/). In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Process…
81. Pranav Rajpurkar, Robin Jia, and Percy Liang. Know what you don’t know: Unanswerable questions for SQuAD. In Iryna Gurevych and Yusuke Miyao (eds.), Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 784–789, Melbourne, Australia, Ju… — [Know What You Don't Know: Unanswerable Questions for SQuAD](https://md.research.elamur.ai/free/en/papers/rajpurkar-2018-know-what-you-don-t/)
82. Sudha Rao and Hal Daumé III. Learning to ask good questions: Ranking clarification questions using neural expected value of perfect information. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 2737–2746, 2018. [find](https://scholar.google.com/scholar?q=Sudha+Rao+and+Hal+Daum%C3%A9+III%2E+Learning+to+ask+good+questions%3A+Ranking+clarification+questions+using+neural+expected+value+of+perfect+information%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Associ)
83. Abhilasha Ravichander. Question answering for privacy policies: Combining computational and legal. In Proceedings of the 2019 Conference on Empirical Methods in Natural, pp. 4947–4958. Association for Computational Linguistics, 2019. [find](https://scholar.google.com/scholar?q=Abhilasha+Ravichander%2E+Question+answering+for+privacy+policies%3A+Combining+computational+and+legal%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural%2C+pp%2E+4947%E2%80%934958%2E+Association+for)
84. Nils Reimers and Iryna Gurevych. [Making monolingual sentence embeddings multilingual using knowledge distillation](https://md.research.elamur.ai/free/en/papers/reimers-2020-making-monolingual-sentence-embeddings-multilingual/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 11 2020. URL https://arxiv.org/abs/2004.09813. [↗](https://arxiv.org/abs/2004.09813)
85. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Semantically equivalent adversarial rules for debugging NLP models. In Iryna Gurevych and Yusuke Miyao (eds.), Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 856–865, Melbour… [find](https://scholar.google.com/scholar?q=Marco+Tulio+Ribeiro%2C+Sameer+Singh%2C+and+Carlos+Guestrin%2E+Semantically+equivalent+adversarial+rules+for+debugging+NLP+models%2E+In+Iryna+Gurevych+and+Yusuke+Miyao+%28eds%2E%29%2C+Proceedings+of+the+56th+Annual+Me)
86. Paul Röttger, Hannah Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy. [XSTest: A test suite for identifying exaggerated safety behaviours in large language models](https://md.research.elamur.ai/free/en/papers/rottger-2023-xstest-test-suite-identifying-exaggerated/). In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Proceedings of the 2024 Conference of the North American…
87. Paul Röttger, Fabio Pernisi, Bertie Vidgen, and Dirk Hovy. [Safetyprompts: a systematic review of open datasets for evaluating and improving large language model safety](https://md.research.elamur.ai/free/en/papers/rottger-2024-safetyprompts-systematic-review-open-datasets/). arXiv preprint arXiv:2404.05399, 2024.
88. Paul Röttger, Fabio Pernisi, Bertie Vidgen, and Dirk Hovy. [Safetyprompts: a systematic review of open datasets for evaluating and improving large language model safety](https://md.research.elamur.ai/free/en/papers/rottger-2024-safetyprompts-systematic-review-open-datasets/). 2024. URL [https://safetyprompts.com/](https://safetyprompts.com/). [↗](https://safetyprompts.com/)
89. Joni Salminen, Hind Almerekhi, Milica Milenković, Soon-gyo Jung, Jisun An, Haewoon Kwak, and Bernard Jansen. Anatomy of online hate: Developing a taxonomy and machine learning models for identifying and classifying hate in online news media. Proceedings of the International AAAI Conference on Web a… [find](https://scholar.google.com/scholar?q=Joni+Salminen%2C+Hind+Almerekhi%2C+Milica+Milenkovic%CC%81%2C+Soon%2Dgyo+Jung%2C+Jisun+An%2C+Haewoon+Kwak%2C+and+Bernard+Jansen%2E+Anatomy+of+online+hate%3A+Developing+a+taxonomy+and+machine+learning+models+for+identifying+)
90. Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al. [Beyond the imitation game: Quantifying and extrapolating the capabilities of language models](https://md.research.elamur.ai/free/en/papers/srivastava-2022-imitation-game-quantifying-extrapolating-capabilities/). arXiv preprint arXiv:2206.04615, 2022.…
91. Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al. [Beyond the imitation game: Quantifying and extrapolating the capabilities of language models](https://md.research.elamur.ai/free/en/papers/srivastava-2022-imitation-game-quantifying-extrapolating-capabilities/). Transactions on Machine Learning Resear…
92. Lichao Sun, Yue Huang, Haoran Wang, Siyuan Wu, Qihui Zhang, Yuan Li, Chujie Gao, Yixin Huang, Wenhan Lyu, Yixuan Zhang, Xiner Li, Zhengliang Liu, Yixin Liu, Yijue Wang, Zhikun Zhang, Bertie Vidgen, Bhavya Kailkhura, Caiming Xiong, Chaowei Xiao, Chunyuan Li, Eric Xing, Furong Huang, Hao Liu, Heng Ji,… — [TrustLLM: Trustworthiness in Large Language Models](https://md.research.elamur.ai/free/en/papers/huang-2024-trustllm-trustworthiness-large-language-models/)
93. Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Le, Ed Chi, Denny Zhou, et al. [Challenging big-bench tasks and whether chain-of-thought can solve them](https://md.research.elamur.ai/free/en/papers/suzgun-2023-challenging-big-bench-tasks-whether/). In Findings of the Association for Computational Linguistics: ACL 2023, pp. 1…
94. Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, Pouya Tafti, Léonard Hussenot, Pier Giuseppe Sessa, Aakanksha Chowdhery, Adam Roberts, Aditya Barua, Alex Botev, Alex Castro-Ros, Ambrose Sl… [find](https://scholar.google.com/scholar?q=Gemma+Team%2C+Thomas+Mesnard%2C+Cassidy+Hardin%2C+Robert+Dadashi%2C+Surya+Bhupatiraju%2C+Shreya+Pathak%2C+Laurent+Sifre%2C+Morgane+Rivi%C3%A8re%2C+Mihir+Sanjay+Kale%2C+Juliette+Love%2C+Pouya+Tafti%2C+L%C3%A9onard+Hussenot%2C+Pier+Gius)
95. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). ArXiv, abs/230…
96. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). arXiv preprint arXiv:2302.13971, 2023.
97. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,… — [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/)
98. Neeraj Varshney, Swaroop Mishra, and Chitta Baral. Investigating selective prediction approaches across several tasks in IID, OOD, and adversarial settings. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Findings of the Association for Computational Linguistics: ACL 2022, pp. 19… [find](https://scholar.google.com/scholar?q=Neeraj+Varshney%2C+Swaroop+Mishra%2C+and+Chitta+Baral%2E+Investigating+selective+prediction+approaches+across+several+tasks+in+IID%2C+OOD%2C+and+adversarial+settings%2E+In+Smaranda+Muresan%2C+Preslav+Nakov%2C+and+Ali)
99. Boxin Wang, Weixin Chen, Hengzhi Pei, Chulin Xie, Mintong Kang, Chenhui Zhang, Chejian Xu, Zidi Xiong, Ritik Dutta, Rylan Schaeffer, Sang Truong, Simran Arora, Mantas Mazeika, Dan Hendrycks, Zinan Lin, Yu Cheng, Sanmi Koyejo, Dawn Song, and Bo Li. Decodingtrust: A comprehensive assessment of trustwo… — [DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models](https://md.research.elamur.ai/free/en/papers/wang-2023-decodingtrust-comprehensive-assessment-trustworthiness-gpt/)
100. Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Raghavi Chandu, David Wadden, Kelsey MacMillan, Noah A. Smith, Iz Beltagy, and Hannaneh Hajishirzi. [How far can camels go? exploring the state of instruction tuning on open resources](https://md.research.elamur.ai/free/en/papers/wang-2023-far-camels-go-exploring-state/), 2023. URL https://arxiv.org/abs/2306.… [↗](https://arxiv.org/abs/2306.…)
101. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language models with self-generated instructions. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Asso… [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Yeganeh+Kordi%2C+Swaroop+Mishra%2C+Alisa+Liu%2C+Noah+A%2E+Smith%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+Self%2Dinstruct%3A+Aligning+language+models+with+self%2Dgenerated+instructions%2E+In+Anna+Roger)
102. Yuxia Wang, Haonan Li, Xudong Han, Preslav Nakov, and Timothy Baldwin. [Do-not-answer: Evaluating safeguards in LLMs](https://md.research.elamur.ai/free/en/papers/wang-2024-do-not-answer-evaluating-safeguards/). In Yvette Graham and Matthew Purver (eds.), Findings of the Association for Computational Linguistics: EACL 2024, pp. 896–911, St. Julian’s, Malta, March 2024. Association for Computa…
103. Laura Weidinger, Jonathan Uesato, Maribeth Rauh, Conor Griffin, Po-Sen Huang, John Mellor, Amelia Glaese, Myra Cheng, Borja Balle, Atoosa Kasirzadeh, Courtney Biles, Sasha Brown, Zac Kenton, Will Hawkins, Tom Stepleton, Abeba Birhane, Lisa Anne Hendricks, Laura Rimell, William Isaac, Julia Haas, Sea… — [doi:10.1145/3531146](https://doi.org/10.1145/3531146) [find](https://scholar.google.com/scholar?q=Laura+Weidinger%2C+Jonathan+Uesato%2C+Maribeth+Rauh%2C+Conor+Griffin%2C+Po%2DSen+Huang%2C+John+Mellor%2C+Amelia+Glaese%2C+Myra+Cheng%2C+Borja+Balle%2C+Atoosa+Kasirzadeh%2C+Courtney+Biles%2C+Sasha+Brown%2C+Zac+Kenton%2C+Will+Hawk)
104. Spencer Whitehead, Suzanne Petryk, Vedaad Shakib, Joseph E. Gonzalez, Trevor Darrell, Anna Rohrbach, and Marcus Rohrbach. Reliable visual question answering: Abstain rather than answer incorrectly. ArXiv, abs/2204.13631, 2022. URL [https://api.semanticscholar.org/CorpusID:248426953](https://api.semanticscholar.org/CorpusID:248426953). [↗](https://api.semanticscholar.org/CorpusID:248426953)
105. Jingjing Xu, Yuechen Wang, Duyu Tang, Nan Duan, Pengcheng Yang, Qi Zeng, Ming Zhou, and Xu Sun. Asking clarification questions in knowledge-based question answering. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (eds.), Proceedings of the 2019 Conference on Empirical Methods in Natural La… [find](https://scholar.google.com/scholar?q=Jingjing+Xu%2C+Yuechen+Wang%2C+Duyu+Tang%2C+Nan+Duan%2C+Pengcheng+Yang%2C+Qi+Zeng%2C+Ming+Zhou%2C+and+Xu+Sun%2E+Asking+clarification+questions+in+knowledge%2Dbased+question+answering%2E+In+Kentaro+Inui%2C+Jing+Jiang%2C+Vince)
106. Yuqing Yang, Ethan Chern, Xipeng Qiu, Graham Neubig, and Pengfei Liu. Alignment for honesty, 2024. URL https://arxiv.org/abs/2312.07000. [↗](https://arxiv.org/abs/2312.07000)
107. Xinyan Yu, Sewon Min, Luke Zettlemoyer, and Hannaneh Hajishirzi. [CREPE: Open-domain question answering with false presuppositions](https://md.research.elamur.ai/free/en/papers/yu-2023-crepe-open-domain-question-answering/). In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pa…
108. Michael Zhang and Eunsol Choi. SituatedQA: Incorporating extra-linguistic contexts into QA. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 7371–7387, Online and Punta Can… [find](https://scholar.google.com/scholar?q=Michael+Zhang+and+Eunsol+Choi%2E+SituatedQA%3A+Incorporating+extra%2Dlinguistic+contexts+into+QA%2E+In+Marie%2DFrancine+Moens%2C+Xuanjing+Huang%2C+Lucia+Specia%2C+and+Scott+Wen%2Dtau+Yih+%28eds%2E%29%2C+Proceedings+of+the+2021)
109. Michael JQ Zhang and Eunsol Choi. Clarify when necessary: Resolving ambiguity through interaction with lms. arXiv preprint arXiv:2311.09469, 2023. URL [https://arxiv.org/](https://arxiv.org/) abs/2311.09469. [↗](https://arxiv.org/)
110. Zhexin Zhang, Leqi Lei, Lindong Wu, Rui Sun, Yongkang Huang, Chong Long, Xiao Liu, Xuanyu Lei, Jie Tang, and Minlie Huang. [SafetyBench: Evaluating the safety of large language models](https://md.research.elamur.ai/free/en/papers/zhang-2023-safetybench-evaluating-safety-large/). In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Proceedings of the 62nd Annual Meeting of the Association f…
111. Wenting Zhao, Xiang Ren, Jack Hessel, Claire Cardie, Yejin Choi, and Yuntian Deng. [Wildchat: 1m chatGPT interaction logs in the wild](https://md.research.elamur.ai/free/en/papers/zhao-2024-wildchat-1m-chatgpt-interaction-logs/). In The Twelfth International Conference on Learning Representations, 2024. URL [https://openreview.net/forum?id=Bl8u7ZRlbM](https://openreview.net/forum?id=Bl8u7ZRlbM). [↗](https://openreview.net/forum?id=Bl8u7ZRlbM)
112. Qihuang Zhong, Liang Ding, Juhua Liu, Bo Du, and Dacheng Tao. ROSE doesn’t do that: Boosting the safety of instruction-tuned large language models with reverse prompt contrastive decoding. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computational Linguist… [find](https://scholar.google.com/scholar?q=Qihuang+Zhong%2C+Liang+Ding%2C+Juhua+Liu%2C+Bo+Du%2C+and+Dacheng+Tao%2E+ROSE+doesn%E2%80%99t+do+that%3A+Boosting+the+safety+of+instruction%2Dtuned+large+language+models+with+reverse+prompt+contrastive+decoding%2E+In+Lun%2DWei+)
113. Xuhui Zhou, Hyunwoo Kim, Faeze Brahman, Liwei Jiang, Hao Zhu, Ximing Lu, Frank Xu, Bill Yuchen Lin, Yejin Choi, Niloofar Mireshghallah, Ronan Le Bras, and Maarten Sap. Haicosystem: An ecosystem for sandboxing safety risks in human-ai interactions. arXiv, 2024. URL http://arxiv.org/abs/2409.16427. — [arXiv:2409.16427](https://arxiv.org/abs/2409.16427) [↗](http://arxiv.org/abs/2409.16427)
114. Terry Yue Zhuo, Yujin Huang, Chunyang Chen, and Zhenchang Xing. Exploring ai ethics of chatgpt: A diagnostic analysis. ArXiv, abs/2301.[12867, 2023](https://md.research.elamur.ai/free/en/papers/zhuo-2023-red-teaming-chatgpt-jailbreaking-bias/). URL [https://api.semanticscholar.org/CorpusID:256390238](https://api.semanticscholar.org/CorpusID:256390238). [↗](https://api.semanticscholar.org/CorpusID:256390238)

*Updated 30 October 2026*