---
title: "Automatically Auditing Large Language Models via Discrete"
description: "Because tokens are discrete and such pairs are scarce, the authors introduce ARCA, a… 60 references, cited by 73 papers here."
authors: ["Erik Jones", "Anca Dragan", "Aditi Raghunathan", "Jacob Steinhardt"]
year: 2023
venue: "arXiv:2303.04381"
source: "https://arxiv.org/abs/2303.04381"
doi: "10.48550/arxiv.2303.04381"
arxiv: "2303.04381"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/jones-2023-automatically-auditing-large-language-models/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/jones-2023-automatically-auditing-large-language-models/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/8/) / Automatically Auditing Large Language Models via Discrete Optimization

# Automatically Auditing Large Language Models via Discrete Optimization

**Authors:** [Erik Jones](https://md.research.elamur.ai/free/en/authors/erik-jones/), [Anca Dragan](https://md.research.elamur.ai/free/en/authors/anca-dragan/), [Aditi Raghunathan](https://md.research.elamur.ai/free/en/authors/aditi-raghunathan/) and [Jacob Steinhardt](https://md.research.elamur.ai/free/en/authors/jacob-steinhardt/) · **Year:** 2023 · **Venue:** arXiv:2303.04381 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2303.04381) · [PDF](https://arxiv.org/pdf/2303.04381) · [DOI 10.48550/arxiv.2303.04381](https://doi.org/10.48550/arxiv.2303.04381) · [arXiv 2303.04381](https://arxiv.org/abs/2303.04381) · [OpenAlex](https://openalex.org/W4323709479)

**Citations:** 16 citations (OpenAlex, 2026-10-14) · [73](https://md.research.elamur.ai/free/en/papers/jones-2023-automatically-auditing-large-language-models/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Testing language models for harmful or bizarre behavior before release is hard. Such failures can be rare, and people struggle to write the prompts that trigger them. Jones et al. treat auditing as a search problem. The auditor writes a scoring function over a prompt and its greedy completion, then looks for high-scoring pairs. One example is a harmless-looking prompt about a celebrity that yields a toxic continuation. Because tokens are discrete and such pairs are scarce, the authors introduce ARCA, a coordinate-ascent method. ARCA changes one prompt or output token at a time. It ranks candidate tokens using exact log-probabilities combined with averaged gradient-based estimates, then rescores the top few exactly. Applied to GPT-2 (762M) and GPT-J (6B), ARCA uncovered many toxic statements about celebrities, prompts that switch from French or German into English, and prompts that elicit specific names. It found more such cases than AutoPrompt and the adversarial attack of Guo et al. (2021). The approach matters because adapting it to a new behavior only requires a new objective.

## Key points
- Audited models are GPT-2 (762M parameters) and GPT-J (6B). Prompt naturalness and transfer were checked with GPT-3 davinci-002.
- Objectives are built from parts such as a toxicity classifier, a language detector, unigram/perplexity terms, fixed prefixes, or an exact target output. An example of an exact target is "Florida governor → Rick Scott".
- In the authors' comparisons, ARCA outperformed AutoPrompt and Guo et al. (2021). On the French/German→English task, AutoPrompt found essentially nothing.
- Prompts found on GPT-J were more natural than those found on GPT-2. Their log-perplexity under GPT-3 was about 0.6–5 points lower, with no explicit naturalness term.
- In a transfer test, 100 three-token GPT-2 prompts with toxic two-token completions were fed to GPT-3. GPT-3 matched the first token for about half of them and produced toxic text for about 20%. Code is at github.com/ejones313/auditing-llms.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Carnegie Mellon University](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/) — University, Pittsburgh, United States · 598 works · 191,225 citations
- [University of California, Berkeley](https://md.research.elamur.ai/free/en/universities/university-of-california-berkeley/) — University, Berkeley, United States · 209 works · 244,782 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,954 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 626 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works

## Cite

Cite as: Erik Jones, Anca Dragan, Aditi Raghunathan and Jacob Steinhardt. 2023. Automatically Auditing Large Language Models via Discrete Optimization. arXiv:2303.04381.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{jones2023automatically,
  title = {{Automatically Auditing Large Language Models via Discrete Optimization}},
  author = {Erik Jones, Anca Dragan, Aditi Raghunathan and Jacob Steinhardt},
  year = {2023},
  doi = {10.48550/arxiv.2303.04381},
  eprint = {2303.04381},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2303.04381}
}
```

- [← previous Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewards](https://md.research.elamur.ai/free/en/papers/rame-2023-rewarded-soups-towards-pareto-optimal/)
- [next → Dimensional Modeling of Emotions in Text with Appraisal Theories: Corpus Creation, Annotation Reliability, and Prediction](https://md.research.elamur.ai/free/en/papers/troiano-2023-dimensional-modeling-emotions-text-appraisal/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/8/)

## References

60 references, 34 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Abubakar Abid, Maheen Farooqi, and James Zou. [Persistent anti-muslim bias in large language models](https://md.research.elamur.ai/free/en/papers/abid-2021-persistent-anti-muslim-bias-large/). arXiv preprint arXiv:2101.05783, 2021.
2. Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani Srivastava, and Kai-Wei Chang. [Generating natural language adversarial examples](https://md.research.elamur.ai/free/en/papers/alzantot-2018-generating-natural-language-adversarial-examples/). In Empirical Methods in Natural Language Processing (EMNLP), 2018.
3. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, T. Henighan, Nicholas Joseph, Saurav Kadavath, John Kernion, Tom Conerly, S. El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, S. Kravec… [find](https://scholar.google.com/scholar?q=Yuntao+Bai%2C+Andy+Jones%2C+Kamal+Ndousse%2C+Amanda+Askell%2C+Anna+Chen%2C+Nova+DasSarma%2C+Dawn+Drain%2C+Stanislav+Fort%2C+Deep+Ganguli%2C+T%2E+Henighan%2C+Nicholas+Joseph%2C+Saurav+Kadavath%2C+John+Kernion%2C+Tom+Conerly%2C+S%2E+E)
4. Emily Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchel. On the dangers of stochastic parrots: Can language models be too big? In ACM Conference on Fairness, Accountability, and Transparency (FAccT), 2021. [find](https://scholar.google.com/scholar?q=Emily+Bender%2C+Timnit+Gebru%2C+Angelina+McMillan%2DMajor%2C+and+Shmargaret+Shmitchel%2E+On+the+dangers+of+stochastic+parrots%3A+Can+language+models+be+too+big%3F+In+ACM+Conference+on+Fairness%2C+Accountability%2C+and+)
5. Su Lin Blodgett, Gilsinia Lopez, Alexandra Olteanu, Robert Sim, and Hanna Wallach. Stereotyping norwegian salmon: An inventory of pitfalls in fairness benchmark datasets. In Association for Computational Linguistics (ACL), 2021. [find](https://scholar.google.com/scholar?q=Su+Lin+Blodgett%2C+Gilsinia+Lopez%2C+Alexandra+Olteanu%2C+Robert+Sim%2C+and+Hanna+Wallach%2E+Stereotyping+norwegian+salmon%3A+An+inventory+of+pitfalls+in+fairness+benchmark+datasets%2E+In+Association+for+Computatio)
6. Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, Erik Brynjolfsson, Shyamal Buch, Dallas Card, Rodrigo Castellon, Niladri Chatterji, Annie Chen, Kathleen Creel, Jared Quincy Davis, Dorotty… — [On the Opportunities and Risks of Foundation Models](https://md.research.elamur.ai/free/en/papers/bommasani-2021-opportunities-risks-foundation-models/)
7. Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. Nuanced metrics for measuring unintended bias with real data for text classification. In World Wide Web (WWW), pages 491–500, 2019. [find](https://scholar.google.com/scholar?q=Daniel+Borkan%2C+Lucas+Dixon%2C+Jeffrey+Sorensen%2C+Nithum+Thain%2C+and+Lucy+Vasserman%2E+Nuanced+metrics+for+measuring+unintended+bias+with+real+data+for+text+classification%2E+In+World+Wide+Web+%28WWW%29%2C+pages+491)
8. Tom B. Brown, Nicholas Carlini, Chiyuan Zhang, Catherine Olsson, Paul Christiano, and Ian Goodfellow. Unrestricted adversarial examples. arXiv preprint arXiv:1809.08352, 2018. — [arXiv:1809.08352](https://arxiv.org/abs/1809.08352) [find](https://scholar.google.com/scholar?q=Tom+B%2E+Brown%2C+Nicholas+Carlini%2C+Chiyuan+Zhang%2C+Catherine+Olsson%2C+Paul+Christiano%2C+and+Ian+Goodfellow%2E+Unrestricted+adversarial+examples%2E+arXiv+preprint+arXiv%3A1809%2E08352%2C+2018%2E+%E2%80%94+%5BarXiv%3A1809%2E08352%5D%28htt)
9. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… — [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
10. Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, Alina Oprea, and Colin Raffel. [Extracting training data from large language models](https://md.research.elamur.ai/free/en/papers/carlini-2020-extracting-training-data-large-language/). arXiv preprint arXiv:2012.07805, 2020.
11. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhai… — [Evaluating Large Language Models Trained on Code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
12. Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. [Plug and play language models: A simple approach to controlled text generation](https://md.research.elamur.ai/free/en/papers/dathathri-2019-plug-play-language-models-simple/). In International Conference on Learning Representations (ICLR), 2020.
13. Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. [Hotflip: White-box adversarial examples for text classification](https://md.research.elamur.ai/free/en/papers/ebrahimi-2018-hotflip-white-box-adversarial-examples/). In Association for Computational Linguistics (ACL), 2018.
14. Sahaj Garg, Vincent Perot, Nicole Limtiaco, Ankur Taly, Ed H Chi, and Alex Beutel. Counterfactual fairness in text classification through robustness. In Association for the Advancement of Artificial Intelligence (AAAI), pages 219–226, 2019. [find](https://scholar.google.com/scholar?q=Sahaj+Garg%2C+Vincent+Perot%2C+Nicole+Limtiaco%2C+Ankur+Taly%2C+Ed+H+Chi%2C+and+Alex+Beutel%2E+Counterfactual+fairness+in+text+classification+through+robustness%2E+In+Association+for+the+Advancement+of+Artificial+I)
15. Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A Smith. [Realtoxicityprompts: Evaluating neural toxic degeneration in language models](https://md.research.elamur.ai/free/en/papers/gehman-2020-realtoxicityprompts-evaluating-neural-toxic-degeneration/). arXiv preprint arXiv:2009.11462, 2020.
16. Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. [Explaining and harnessing adversarial examples](https://md.research.elamur.ai/free/en/papers/goodfellow-2014-explaining-harnessing-adversarial-examples/). In International Conference on Learning Representations (ICLR), 2015.
17. Will Grathwohl, Kevin Swersky, Milad Hashemi, David Duvenaud, and Chris J. Maddison. Oops I took a gradient: Scalable sampling for discrete distributions. In International Conference on Machine Learning (ICML), 2021. [find](https://scholar.google.com/scholar?q=Will+Grathwohl%2C+Kevin+Swersky%2C+Milad+Hashemi%2C+David+Duvenaud%2C+and+Chris+J%2E+Maddison%2E+Oops+I+took+a+gradient%3A+Scalable+sampling+for+discrete+distributions%2E+In+International+Conference+on+Machine+Learni)
18. Sophie Groenwold, Lily Ou, Aesha Parekh, Samhita Honnavalli, Sharon Levy, Diba Mirza, and William Yang Wang. [Investigating african-american vernacular english in transformer-based text generation](https://md.research.elamur.ai/free/en/papers/groenwold-2020-investigating-african-american-vernacular-english/). In Empirical Methods in Natural Language Processing (EMNLP), 2020.
19. Chuan Guo, Alexandre Sablayrolles, Hervé Jégou, and Douwe Kiela. [Gradient-based adversarial attacks against text transformers](https://md.research.elamur.ai/free/en/papers/guo-2021-gradient-based-adversarial-attacks-against/). In Empirical Methods in Natural Language Processing (EMNLP), 2021.
20. Laura Hanu and Unitary team. Detoxify. Github. [https://github.com/unitaryai/detoxify](https://github.com/unitaryai/detoxify), 2020. [↗](https://github.com/unitaryai/detoxify)
21. Babak Hemmatian and Lav R. Varshney. Debiased large language models still associate muslims with uniquely violent acts. arXiv preprint arXiv:2208.04417, 2022. — [arXiv:2208.04417](https://arxiv.org/abs/2208.04417) [find](https://scholar.google.com/scholar?q=Babak+Hemmatian+and+Lav+R%2E+Varshney%2E+Debiased+large+language+models+still+associate+muslims+with+uniquely+violent+acts%2E+arXiv+preprint+arXiv%3A2208%2E04417%2C+2022%2E+%E2%80%94+%5BarXiv%3A2208%2E04417%5D%28https%3A%2F%2Farxiv%2Eorg%2Fab)
22. Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, Tom Hennigan, Eric Noland, Katie Millican, George van den Driessche, Bogdan Damoc, Aurelia Guy, Simon Osindero, Karen Simonyan, E… [find](https://scholar.google.com/scholar?q=Jordan+Hoffmann%2C+Sebastian+Borgeaud%2C+Arthur+Mensch%2C+Elena+Buchatskaya%2C+Trevor+Cai%2C+Eliza+Rutherford%2C+Diego+de+Las+Casas%2C+Lisa+Anne+Hendricks%2C+Johannes+Welbl%2C+Aidan+Clark%2C+Tom+Hennigan%2C+Eric+Noland%2C+Ka)
23. Eric Jang, Shixiang Gu, and Ben Poole. [Categorical reparameterization with Gumbel-softmax](https://md.research.elamur.ai/free/en/papers/jang-2016-categorical-reparameterization-gumbel-softmax/). arXiv preprint arXiv:1611.01144, 2017.
24. Robin Jia and Percy Liang. [Adversarial examples for evaluating reading comprehension systems](https://md.research.elamur.ai/free/en/papers/jia-2017-adversarial-examples-evaluating-reading-comprehension/). In Empirical Methods in Natural Language Processing (EMNLP), 2017.
25. Erik Jones and Jacob Steinhardt. Capturing failures of large language models via human cognitive biases. In Advances in Neural Information Processing Systems (NeurIPS), 2022. [find](https://scholar.google.com/scholar?q=Erik+Jones+and+Jacob+Steinhardt%2E+Capturing+failures+of+large+language+models+via+human+cognitive+biases%2E+In+Advances+in+Neural+Information+Processing+Systems+%28NeurIPS%29%2C+2022%2E)
26. Armand Joulin, Edouard Grave, Piotr Bojanowski, Matthijs Douze, Hérve Jégou, and Tomas Mikolov. [Fasttext.zip: Compressing text classification models](https://md.research.elamur.ai/free/en/papers/joulin-2016-fasttext-zip-compressing-text-classification/). arXiv preprint arXiv:1612.03651, 2016a.
27. Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. Bag of tricks for efficient text classification. arXiv preprint arXiv:1607.01759, 2016b. [find](https://scholar.google.com/scholar?q=Armand+Joulin%2C+Edouard+Grave%2C+Piotr+Bojanowski%2C+and+Tomas+Mikolov%2E+Bag+of+tricks+for+efficient+text+classification%2E+arXiv+preprint+arXiv%3A1607%2E01759%2C+2016b%2E)
28. Diederik Kingma and Jimmy Ba. [Adam: A method for stochastic optimization](https://md.research.elamur.ai/free/en/papers/kingma-2014-adam-method-stochastic-optimization/). In International Conference on Learning Representations (ICLR), 2015.
29. Ben Krause, Akhilesh Deepak Gotmare, Bryan McCann, Nitish Shirish Keskar, Shafiq Joty, Richard Socher, and Nazneen Fatema Rajani. [Gedi: Generative discriminator guided sequence generation](https://md.research.elamur.ai/free/en/papers/krause-2021-gedi-generative-discriminator-guided-sequence/). In Findings of Empirical Methods in Natural Language Processing (Findings of EMNLP), 2021.
30. Sachin Kumar, Eric Malmi, Aliaksei Severyn, and Yulia Tsvetkov. Controlled text generation as continuous optimization with multiple constraints. In Advances in Neural Information Processing Systems (NeurIPS), 2021. [find](https://scholar.google.com/scholar?q=Sachin+Kumar%2C+Eric+Malmi%2C+Aliaksei+Severyn%2C+and+Yulia+Tsvetkov%2E+Controlled+text+generation+as+continuous+optimization+with+multiple+constraints%2E+In+Advances+in+Neural+Information+Processing+Systems+%28N)
31. Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, Yuhuai Wu, Behnam Neyshabur, Guy Gur-Ari, and Vedant Misra. [Solving quantitative reasoning problems with language models](https://md.research.elamur.ai/free/en/papers/lewkowycz-2022-solving-quantitative-reasoning-problems-language/). arXiv preprint arXiv:22…
32. Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, and Xipeng Qiu. [BERT-ATTACK: Adversarial attack against BERT using BERT](https://md.research.elamur.ai/free/en/papers/li-2020-bert-attack-adversarial-attack-against/). In Empirical Methods in Natural Language Processing (EMNLP), 2020.
33. Xiang Lisa Li, John Thickstun, Ishaan Gulrajani, Percy Liang, and Tatsunori Hashimoto. [Diffusion-LM improves controllable text generation](https://md.research.elamur.ai/free/en/papers/li-2022-diffusion-lm-improves-controllable-text/). In Advances in Neural Information Processing Systems (NeurIPS), 2022a.
34. Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, Thomas Hubert, Peter Choy, Cyprien de Masson d’Autume, Igor Babuschkin, Xinyun Chen, Po-Sen Huang, Johannes Welbl, Sven Gowal, Alexey Cherepanov, James M… — [Competition-Level Code Generation with AlphaCode](https://md.research.elamur.ai/free/en/papers/li-2022-competition-level-code-generation-alphacode/)
35. Alisa Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah A. Smith, and Yejin Choi. [Dexperts: Decoding-time controlled text generation with experts and anti-experts](https://md.research.elamur.ai/free/en/papers/liu-2021-dexperts-decoding-time-controlled-text/). In Association for Computational Linguistics (ACL), 2021.
36. Moin Nadeem, Anna Bethke, and Siva Reddy. [Stereoset: Measuring stereotypical bias in pretrained language models](https://md.research.elamur.ai/free/en/papers/nadeem-2021-stereoset-measuring-stereotypical-bias-pretrained/). arXiv preprint arXiv:2004.09456, 2020.
37. Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huam Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. A conversational paradigm for program synthesis. arXiv preprint arXiv:2203.13474, 2022. [find](https://scholar.google.com/scholar?q=Erik+Nijkamp%2C+Bo+Pang%2C+Hiroaki+Hayashi%2C+Lifu+Tu%2C+Huam+Wang%2C+Yingbo+Zhou%2C+Silvio+Savarese%2C+and+Caiming+Xiong%2E+A+conversational+paradigm+for+program+synthesis%2E+arXiv+preprint+arXiv%3A2203%2E13474%2C+2022%2E)
38. Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. [Red teaming language models with language models](https://md.research.elamur.ai/free/en/papers/perez-2022-red-teaming-language-models-language/). arXiv preprint arXiv:2202.03286, 2022.
39. Lianhui Qin, Sean Welleck, Daniel Khashabi, and Yejin Choi. COLD decoding: Energy-based constrained text generation with langevin dynamics. In Advances in Neural Information Processing Systems (NeurIPS), 2022. [find](https://scholar.google.com/scholar?q=Lianhui+Qin%2C+Sean+Welleck%2C+Daniel+Khashabi%2C+and+Yejin+Choi%2E+COLD+decoding%3A+Energy%2Dbased+constrained+text+generation+with+langevin+dynamics%2E+In+Advances+in+Neural+Information+Processing+Systems+%28NeurIP)
40. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. OpenAI Blog, 1(8), 2019. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+and+Ilya+Sutskever%2E+Language+models+are+unsupervised+multitask+learners%2E+OpenAI+Blog%2C+1%288%29%2C+2019%2E)
41. Jack W. Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, J. Aslanides, Sarah Henderson, Roman Ring, Susannah Young, Eliza Rutherford, Tom Hennigan, Jacob Menick, Albin Cassirer, Richard Powell, G. V. D. Driessche, Lisa Anne Hendricks, Maribeth Rauh, Po-Sen Huang, A… [find](https://scholar.google.com/scholar?q=Jack+W%2E+Rae%2C+Sebastian+Borgeaud%2C+Trevor+Cai%2C+Katie+Millican%2C+Jordan+Hoffmann%2C+Francis+Song%2C+J%2E+Aslanides%2C+Sarah+Henderson%2C+Roman+Ring%2C+Susannah+Young%2C+Eliza+Rutherford%2C+Tom+Hennigan%2C+Jacob+Menick%2C+Alb)
42. Ashwin Ram, Rohit Prasad, Chandra Khatri, Anu Venkatesh, Raefer Gabriel, Qing Liu, Jeff Nunn, Behnam Hedayatnia, Ming Cheng, Ashish Nagar, Eric King, Kate Bland, Amanda Wartick, Yi Pan, Han Song, Sk Jayadevan, Gene Hwang, and Art Pettigrue. Conversational ai: The science behind the alexa prize. arXi… [find](https://scholar.google.com/scholar?q=Ashwin+Ram%2C+Rohit+Prasad%2C+Chandra+Khatri%2C+Anu+Venkatesh%2C+Raefer+Gabriel%2C+Qing+Liu%2C+Jeff+Nunn%2C+Behnam+Hedayatnia%2C+Ming+Cheng%2C+Ashish+Nagar%2C+Eric+King%2C+Kate+Bland%2C+Amanda+Wartick%2C+Yi+Pan%2C+Han+Song%2C+Sk+J)
43. Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh. Beyond accuracy: Behavioral testing of NLP models with CheckList. In Association for Computational Linguistics (ACL), pages 4902–4912, 2020. [find](https://scholar.google.com/scholar?q=Marco+Tulio+Ribeiro%2C+Tongshuang+Wu%2C+Carlos+Guestrin%2C+and+Sameer+Singh%2E+Beyond+accuracy%3A+Behavioral+testing+of+NLP+models+with+CheckList%2E+In+Association+for+Computational+Linguistics+%28ACL%29%2C+pages+4902%E2%80%93)
44. Sascha Rothe, Shashi Narayan, and Aliaksei Severyn. [Leveraging pre-trained checkpoints for sequence generation tasks](https://md.research.elamur.ai/free/en/papers/rothe-2019-leveraging-pre-trained-checkpoints-sequence/). Transactions of the Association for Computational Linguistics (TACL), 8:264–280, 2020.
45. Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, and Nanyun Peng. [The woman worked as a babysitter: On biases in language generation](https://md.research.elamur.ai/free/en/papers/sheng-2019-woman-worked-as-babysitter-biases/). In Empirical Methods in Natural Language Processing (EMNLP), 2019.
46. Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh. [Autoprompt: Eliciting knowledge from language models with automatically generated prompts](https://md.research.elamur.ai/free/en/papers/shin-2020-autoprompt-eliciting-knowledge-language-models/). In Empirical Methods in Natural Language Processing (EMNLP), 2020.
47. Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul Christiano. Learning to summarize from human feedback. In Advances in Neural Information Processing Systems (NeurIPS), 2020. [find](https://scholar.google.com/scholar?q=Nisan+Stiennon%2C+Long+Ouyang%2C+Jeff+Wu%2C+Daniel+M%2E+Ziegler%2C+Ryan+Lowe%2C+Chelsea+Voss%2C+Alec+Radford%2C+Dario+Amodei%2C+and+Paul+Christiano%2E+Learning+to+summarize+from+human+feedback%2E+In+Advances+in+Neural+Info)
48. Haoran Sun, Hanjun Dai, Wei Xia, and Arun Ramamurthy. Path auxiliary proposal for MCMC in discrete space. In International Conference on Learning Representations (ICLR), 2022. [find](https://scholar.google.com/scholar?q=Haoran+Sun%2C+Hanjun+Dai%2C+Wei+Xia%2C+and+Arun+Ramamurthy%2E+Path+auxiliary+proposal+for+MCMC+in+discrete+space%2E+In+International+Conference+on+Learning+Representations+%28ICLR%29%2C+2022%2E)
49. Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. [Intriguing properties of neural networks](https://md.research.elamur.ai/free/en/papers/szegedy-2013-intriguing-properties-neural-networks/). In International Conference on Learning Representations (ICLR), 2014.
50. Leonard Tang, Elizabeth Ke, Nikhil Singh, Nakul Verma, and Iddo Drori. Solving probability and statistics problems by program synthesis. arXiv preprint arXiv:2111.08276, 2021. — [arXiv:2111.08276](https://arxiv.org/abs/2111.08276) [find](https://scholar.google.com/scholar?q=Leonard+Tang%2C+Elizabeth+Ke%2C+Nikhil+Singh%2C+Nakul+Verma%2C+and+Iddo+Drori%2E+Solving+probability+and+statistics+problems+by+program+synthesis%2E+arXiv+preprint+arXiv%3A2111%2E08276%2C+2021%2E+%E2%80%94+%5BarXiv%3A2111%2E08276%5D%28htt)
51. Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, YaGuang Li, Hongrae Lee, Huaixiu Steven Zheng, Amin Ghafouri, Marcelo Menegali, Yanping Huang, Maxim Krikun, Dmitry Lepikhin, James Qin, Dehao Chen, Yuanzho… — [LaMDA: Language Models for Dialog Applications](https://md.research.elamur.ai/free/en/papers/thoppilan-2022-lamda-language-models-dialog-applications/)
52. Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. Universal adversarial triggers for attacking and analyzing NLP. In Empirical Methods in Natural Language Processing (EMNLP), 2019. [find](https://scholar.google.com/scholar?q=Eric+Wallace%2C+Shi+Feng%2C+Nikhil+Kandpal%2C+Matt+Gardner%2C+and+Sameer+Singh%2E+Universal+adversarial+triggers+for+attacking+and+analyzing+NLP%2E+In+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP%29%2C+201)
53. Ben Wang and Aran Komatsuzaki. GPT-J-6B: A 6 billion parameter autoregressive language model, 2021. [find](https://scholar.google.com/scholar?q=Ben+Wang+and+Aran+Komatsuzaki%2E+GPT%2DJ%2D6B%3A+A+6+billion+parameter+autoregressive+language+model%2C+2021%2E)
54. Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang, Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, Zac Kenton, Sasha Brown, Will Hawkins, Tom Stepleton, Courtney Biles, Abeba Birhane, Julia Haas, Laura Rimell, Lisa Anne Hendricks, William Isaac, Sean L… — [Ethical and social risks of harm from Language Models](https://md.research.elamur.ai/free/en/papers/weidinger-2021-ethical-social-risks-harm-language/)
55. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, R’emi Louf, Morgan Funtowicz, and Jamie Brew. HuggingFace’s [transformers: State-of-the-art natural language processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/). arXiv preprint arXiv:1910.03771, 2019.
56. Albert Xu, Eshaan Pathak, Eric Wallace, Suchin Gururangan, Maarten Sap, and Dan Klein. [Detoxifying language models risks marginalizing minority voices](https://md.research.elamur.ai/free/en/papers/xu-2021-detoxifying-language-models-risks-marginalizing/). In North American Association for Computational Linguistics (NAACL), 2021a.
57. Jing Xu, Da Ju, Margaret Li, Y-Lan Boureau, Jason Weston, and Emily Dinan. [Bot-adversarial dialogue for safe conversational agents](https://md.research.elamur.ai/free/en/papers/xu-2021-bot-adversarial-dialogue-safe-conversational/). In North American Association for Computational Linguistics (NAACL), 2021b.
58. Kevin Yang and Dan Klein. [Fudge: Controlled text generation with future discriminators](https://md.research.elamur.ai/free/en/papers/yang-2021-fudge-controlled-text-generation-future/). In North American Association for Computational Linguistics (NAACL), 2021.
59. Ruqi Zhang, Xingchao Liu, and Qiang Liu. A langevin-like sampler for discrete distributions. In International Conference on Machine Learning (ICML), 2022. [find](https://scholar.google.com/scholar?q=Ruqi+Zhang%2C+Xingchao+Liu%2C+and+Qiang+Liu%2E+A+langevin%2Dlike+sampler+for+discrete+distributions%2E+In+International+Conference+on+Machine+Learning+%28ICML%29%2C+2022%2E)
60. Daniel M. Ziegler, Seraphina Nix, Lawrence Chan, Tim Bauman, Peter Schmidt-Nielsen, Tao Lin, Adam Scherlis, Noa Nabeshima, Ben Weinstein-Raun, Daniel de Haas, Buck Shlegeris, and Nate Thomas. Adversarial training for high-stakes reliability. In Advances in Neural Information Processing Systems (Neur… [find](https://scholar.google.com/scholar?q=Daniel+M%2E+Ziegler%2C+Seraphina+Nix%2C+Lawrence+Chan%2C+Tim+Bauman%2C+Peter+Schmidt%2DNielsen%2C+Tao+Lin%2C+Adam+Scherlis%2C+Noa+Nabeshima%2C+Ben+Weinstein%2DRaun%2C+Daniel+de+Haas%2C+Buck+Shlegeris%2C+and+Nate+Thomas%2E+Adversar)

*Updated 2 November 2026*