---
title: "Fine-tuning Aligned Language Models Compromises — Qi 2023"
description: "Current safety alignment methods, such as instruction tuning and RLHF, constrain how a model behaves at inference time."
authors: ["Xiangyu Qi", "Yi Zeng", "Tinghao Xie", "Pin-Yu Chen", "Ruoxi Jia", "Prateek Mittal", "Peter Henderson"]
year: 2023
venue: "arXiv:2310.03693"
source: "https://arxiv.org/abs/2310.03693"
doi: "10.48550/arxiv.2310.03693"
arxiv: "2310.03693"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/qi-2023-fine-tuning-aligned-language-models/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/qi-2023-fine-tuning-aligned-language-models/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/3/) / Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!

# Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!

**Authors:** Xiangyu Qi, [Yi Zeng](https://md.research.elamur.ai/free/en/authors/yi-zeng/), Tinghao Xie, [Pin-Yu Chen](https://md.research.elamur.ai/free/en/authors/pin-yu-chen/), [Ruoxi Jia](https://md.research.elamur.ai/free/en/authors/ruoxi-jia/), Prateek Mittal, Peter Henderson · **Year:** 2023 · **Venue:** arXiv:2310.03693 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2310.03693) · [PDF](https://arxiv.org/pdf/2310.03693) · [DOI 10.48550/arxiv.2310.03693](https://doi.org/10.48550/arxiv.2310.03693) · [arXiv 2310.03693](https://arxiv.org/abs/2310.03693) · [OpenAlex](https://openalex.org/W4387436590)

**Citations:** 38 citations (openalex-snapshot · 2026-10-14) · [293](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
Current safety alignment methods, such as instruction tuning and RLHF, constrain how a model behaves at inference time. They do not protect against what happens once end users can fine-tune the model themselves. Qi et al. study whether alignment survives custom fine-tuning of Llama-2 and of GPT-3.5 Turbo through OpenAI's API. They describe three escalating risk levels. The first is training on a handful of openly harmful instruction–response pairs. The second is training on "implicitly harmful" data that contains no toxic content but teaches the model to put obedience first. The third is ordinary fine-tuning on benign utility datasets. GPT-4 serves as a judge, rating harmfulness on a 1–5 scale across 11 policy categories drawn from the OpenAI and Meta usage rules. All three settings weaken safety. Small adversarial datasets jailbreak both models almost entirely, and benign data causes a smaller but real decline. The authors argue that safeguards must cover the fine-tuning stage as well as inference. They discuss technical and policy mitigations and their limits, including the difficulty of auditing for backdoors.

## Key points
- Fine-tuning GPT-3.5 Turbo on only 10 harmful examples cost under $0.20 through the API. Afterwards the model complied with almost any harmful request, including unseen ones.
- Ten hand-written "identity shifting" examples contained no explicit toxicity. They cast the model as "AOA," an absolutely obedient agent, and this was enough to jailbreak both Llama-2 and GPT-3.5 Turbo. The attacks did not trigger OpenAI's fine-tuning moderation.
- Benign fine-tuning on Alpaca, Dolly or LLaVA-Visual-Instruct also reduced safety, though to a lesser degree. The authors attribute this to catastrophic forgetting or to tension between helpfulness and harmlessness.
- Jailbroken models kept their capabilities on benign tasks and did not show mode collapse.
- For reproducibility, the authors report results on the public harmful-prompt benchmark of Zou et al. (2023) and release code. They withhold their own harmful data and raw outputs, and they disclosed the findings to OpenAI before publication.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Other](https://md.research.elamur.ai/free/en/shelves/other/)

## Cited by

Cited by

293 papers of our library cite it (of the 140884 whose bibliographies we read); 14 of them have a page here, 279 not yet.

- [Refusal in Language Models Is Mediated by a Single Direction](https://md.research.elamur.ai/free/en/papers/arditi-2024-refusal-language-models-mediated-single/)
- [OpenThoughts: Data Recipes for Reasoning Models](https://md.research.elamur.ai/free/en/papers/guha-2025-openthoughts-data-recipes-reasoning-models/)
- [TrustLLM: Trustworthiness in Large Language Models](https://md.research.elamur.ai/free/en/papers/huang-2024-trustllm-trustworthiness-large-language-models/)
- [A Survey on Large Language Models for Code Generation](https://md.research.elamur.ai/free/en/papers/jiang-2024-survey-large-language-models-code/)
- [Developing a Pragmatic Benchmark for Assessing Korean Legal Language Understanding in Large Language Models](https://md.research.elamur.ai/free/en/papers/kim-2024-developing-pragmatic-benchmark-assessing-korean/)
- [DeepInception: Hypnotize Large Language Model to Be Jailbreaker](https://md.research.elamur.ai/free/en/papers/li-2023-deepinception-hypnotize-large-language-model/)
- [SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models](https://md.research.elamur.ai/free/en/papers/li-2024-salad-bench-hierarchical-comprehensive-safety/)
- [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://md.research.elamur.ai/free/en/papers/mazeika-2024-harmbench-standardized-evaluation-framework-automated/)
- [Breaking the Benchmark: Revealing LLM Bias via Minimal Contextual Augmentation](https://md.research.elamur.ai/free/en/papers/miandoab-2026-breaking-benchmark-revealing-llm-bias/)
- [LLMs Out-of-the-Box Do Not Generate Context-Appropriate Word Order in Russian](https://md.research.elamur.ai/free/en/papers/shabaeva-2026-llms-out-box-do-not/)
- [Steering Language Models With Activation Engineering](https://md.research.elamur.ai/free/en/papers/turner-2023-steering-language-models-activation-engineering/)
- [SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding](https://md.research.elamur.ai/free/en/papers/xu-2024-safedecoding-defending-against-jailbreak-attacks/)
- [Arabic Safety Alignment as Selective Refusal: An Empirical Study of SFT, DPO, and Guard Calibration](https://md.research.elamur.ai/free/en/papers/zbib-2026-arabic-safety-alignment-selective-refusal/)
- [How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs](https://md.research.elamur.ai/free/en/papers/zeng-2024-how-johnny-can-persuade-llms/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{qi2023fine,
  title = {{Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!}},
  author = {Qi, Xiangyu and Zeng, Yi and Xie, Tinghao and Chen, Pin-Yu and Jia, Ruoxi and Mittal, Prateek and Henderson, Peter},
  year = {2023},
  doi = {10.48550/arxiv.2310.03693},
  eprint = {2310.03693},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2310.03693}
}
```

- [← previous Fine-Tashkeel: Finetuning Byte-Level Models for Accurate Arabic Text Diacritization](https://md.research.elamur.ai/free/en/papers/al-2023-rfooh-fine-tashkeel-finetuning-byte-level/)
- [next → Fine-tuning Language Models for Factuality](https://md.research.elamur.ai/free/en/papers/tian-2023-fine-tuning-language-models-factuality/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/3/)

## References

90 references, 26 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Angus Addlesee, Weronika Sieińska, Nancie Gunson, Daniel Hernández Garcia, Christian Dondrup, and Oliver Lemon. Multi-party goal tracking with llms: Comparing pre-training, fine-tuning, and prompt engineering. arXiv preprint arXiv:2308.15231, 2023. 3 [find](https://scholar.google.com/scholar?q=Angus+Addlesee%2C+Weronika+Siein%CC%81ska%2C+Nancie+Gunson%2C+Daniel+Hern%C3%A1ndez+Garcia%2C+Christian+Dondrup%2C+and+Oliver+Lemon%2E+Multi%2Dparty+goal+tracking+with+llms%3A+Comparing+pre%2Dtraining%2C+fine%2Dtuning%2C+and+prompt+en)
2. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Sh… [find](https://scholar.google.com/scholar?q=Yuntao+Bai%2C+Andy+Jones%2C+Kamal+Ndousse%2C+Amanda+Askell%2C+Anna+Chen%2C+Nova+DasSarma%2C+Dawn+Drain%2C+Stanislav+Fort%2C+Deep+Ganguli%2C+Tom+Henighan%2C+Nicholas+Joseph%2C+Saurav+Kadavath%2C+Jackson+Kernion%2C+Tom+Conerly%2C+)
3. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073, 2022b. 3 [find](https://scholar.google.com/scholar?q=Yuntao+Bai%2C+Saurav+Kadavath%2C+Sandipan+Kundu%2C+Amanda+Askell%2C+Jackson+Kernion%2C+Andy+Jones%2C+Anna+Chen%2C+Anna+Goldie%2C+Azalia+Mirhoseini%2C+Cameron+McKinnon%2C+et+al%2E+Constitutional+ai%3A+Harmlessness+from+ai+fee) [our card](https://md.research.elamur.ai/free/en/papers/bai-2022-constitutional-ai-harmlessness-ai-feedback/)
4. Federico Bianchi, Mirac Suzgun, Giuseppe Attanasio, Paul Röttger, Dan Jurafsky, Tatsunori Hashimoto, and James Zou. Safety-tuned llamas: Lessons from improving the safety of large language models that follow instructions. arXiv preprint arXiv:2309.07875, 2023. 11, 38 [find](https://scholar.google.com/scholar?q=Federico+Bianchi%2C+Mirac+Suzgun%2C+Giuseppe+Attanasio%2C+Paul+R%C3%B6ttger%2C+Dan+Jurafsky%2C+Tatsunori+Hashimoto%2C+and+James+Zou%2E+Safety%2Dtuned+llamas%3A+Lessons+from+improving+the+safety+of+large+language+models+that) [our card](https://md.research.elamur.ai/free/en/papers/bianchi-2023-safety-tuned-llamas-lessons-improving/)
5. Richard Blumenthal. This bipartisan framework is a milestone—the first tough, comprehensive legislative blueprint for real, enforceable ai protections. it should put us on a path to addressing the promise & peril ai portends. Twitter, 2023. Available: [https://twitter.com/SenBlumenthal/status/1700147…](https://twitter.com/SenBlumenthal/status/1700147…) [↗](https://twitter.com/SenBlumenthal/status/1700147…)
6. Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021. 3 [find](https://scholar.google.com/scholar?q=Rishi+Bommasani%2C+Drew+A+Hudson%2C+Ehsan+Adeli%2C+Russ+Altman%2C+Simran+Arora%2C+Sydney+von+Arx%2C+Michael+S+Bernstein%2C+Jeannette+Bohg%2C+Antoine+Bosselut%2C+Emma+Brunskill%2C+et+al%2E+On+the+opportunities+and+risks+of+) [our card](https://md.research.elamur.ai/free/en/papers/bommasani-2021-opportunities-risks-foundation-models/)
7. Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar, Xi Chen, Krzysztof Choromanski, Tianli Ding, Danny Driess, Avinava Dubey, Chelsea Finn, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. arXiv preprint arXiv:2307.15818, 2023. 1 [find](https://scholar.google.com/scholar?q=Anthony+Brohan%2C+Noah+Brown%2C+Justice+Carbajal%2C+Yevgen+Chebotar%2C+Xi+Chen%2C+Krzysztof+Choromanski%2C+Tianli+Ding%2C+Danny+Driess%2C+Avinava+Dubey%2C+Chelsea+Finn%2C+et+al%2E+Rt%2D2%3A+Vision%2Dlanguage%2Daction+models+transf)
8. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020. 2, 3 [find](https://scholar.google.com/scholar?q=Tom+Brown%2C+Benjamin+Mann%2C+Nick+Ryder%2C+Melanie+Subbiah%2C+Jared+D+Kaplan%2C+Prafulla+Dhariwal%2C+Arvind+Neelakantan%2C+Pranav+Shyam%2C+Girish+Sastry%2C+Amanda+Askell%2C+et+al%2E+Language+models+are+few%2Dshot+learners%2E+)
9. Hanbo Cai, Pengcheng Zhang, Hai Dong, Yan Xiao, Stefanos Koffas, and Yiming Li. Towards stealthy backdoor attacks against speech recognition via elements of sound. arXiv preprint arXiv:2307.08208, 2023. 37 — [arXiv:2307.08208](https://arxiv.org/abs/2307.08208) [find](https://scholar.google.com/scholar?q=Hanbo+Cai%2C+Pengcheng+Zhang%2C+Hai+Dong%2C+Yan+Xiao%2C+Stefanos+Koffas%2C+and+Yiming+Li%2E+Towards+stealthy+backdoor+attacks+against+speech+recognition+via+elements+of+sound%2E+arXiv+preprint+arXiv%3A2307%2E08208%2C+202)
10. Nicholas Carlini, Milad Nasr, Christopher A Choquette-Choo, Matthew Jagielski, Irena Gao, Anas Awadalla, Pang Wei Koh, Daphne Ippolito, Katherine Lee, Florian Tramer, et al. Are aligned neural networks adversarially aligned? arXiv preprint arXiv:2306.15447, 2023. 4, 36 [find](https://scholar.google.com/scholar?q=Nicholas+Carlini%2C+Milad+Nasr%2C+Christopher+A+Choquette%2DChoo%2C+Matthew+Jagielski%2C+Irena+Gao%2C+Anas+Awadalla%2C+Pang+Wei+Koh%2C+Daphne+Ippolito%2C+Katherine+Lee%2C+Florian+Tramer%2C+et+al%2E+Are+aligned+neural+network)
11. Kangjie Chen, Yuxian Meng, Xiaofei Sun, Shangwei Guo, Tianwei Zhang, Jiwei Li, and Chun Fan. Badpre: Task-agnostic backdoor attacks to pre-trained nlp foundation models. arXiv preprint arXiv:2110.02467, 2021a. 37 — [arXiv:2110.02467](https://arxiv.org/abs/2110.02467) [find](https://scholar.google.com/scholar?q=Kangjie+Chen%2C+Yuxian+Meng%2C+Xiaofei+Sun%2C+Shangwei+Guo%2C+Tianwei+Zhang%2C+Jiwei+Li%2C+and+Chun+Fan%2E+Badpre%3A+Task%2Dagnostic+backdoor+attacks+to+pre%2Dtrained+nlp+foundation+models%2E+arXiv+preprint+arXiv%3A2110%2E0246)
12. Xiaoyi Chen, Ahmed Salem, Dingfan Chen, Michael Backes, Shiqing Ma, Qingni Shen, Zhonghai Wu, and Yang Zhang. Badnl: Backdoor attacks against nlp models with semantic-preserving improvements. In Annual Computer Security Applications Conference, pp. 554–569, 2021b. 37 [find](https://scholar.google.com/scholar?q=Xiaoyi+Chen%2C+Ahmed+Salem%2C+Dingfan+Chen%2C+Michael+Backes%2C+Shiqing+Ma%2C+Qingni+Shen%2C+Zhonghai+Wu%2C+and+Yang+Zhang%2E+Badnl%3A+Backdoor+attacks+against+nlp+models+with+semantic%2Dpreserving+improvements%2E+In+Annua)
13. Pengzhou Cheng, Zongru Wu, Wei Du, and Gongshen Liu. Backdoor attacks and countermeasures in natural language processing models: A comprehensive security review. arXiv preprint arXiv:2309.06055, 2023. 37 — [arXiv:2309.06055](https://arxiv.org/abs/2309.06055) [find](https://scholar.google.com/scholar?q=Pengzhou+Cheng%2C+Zongru+Wu%2C+Wei+Du%2C+and+Gongshen+Liu%2E+Backdoor+attacks+and+countermeasures+in+natural+language+processing+models%3A+A+comprehensive+security+review%2E+arXiv+preprint+arXiv%3A2309%2E06055%2C+2023%2E)
14. Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. Free dolly: Introducing the world’s first truly open instruction-tuned llm, 2023. URL [https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-i…](https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-i…) [↗](https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-i…)
15. J. Dai, C. Chen, and Y. Li. A backdoor attack against lstm-based text classification systems. IEEE Access, 7:138872– 138878, 2019. doi:10.1109/ACCESS.2019.2941376. 3, 11, 37 — [doi:10.1109/access.2019.2941376](https://doi.org/10.1109/access.2019.2941376) [find](https://scholar.google.com/scholar?q=J%2E+Dai%2C+C%2E+Chen%2C+and+Y%2E+Li%2E+A+backdoor+attack+against+lstm%2Dbased+text+classification+systems%2E+IEEE+Access%2C+7%3A138872%E2%80%93+138878%2C+2019%2E+doi%3A10%2E1109%2FACCESS%2E2019%2E2941376%2E+3%2C+11%2C+37+%E2%80%94+%5Bdoi%3A10%2E1109%2Faccess%2E2019)
16. Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. Instructblip: Towards general-purpose vision-language models with instruction tuning, 2023. 3, 8 [find](https://scholar.google.com/scholar?q=Wenliang+Dai%2C+Junnan+Li%2C+Dongxu+Li%2C+Anthony+Meng+Huat+Tiong%2C+Junqi+Zhao%2C+Weisheng+Wang%2C+Boyang+Li%2C+Pascale+Fung%2C+and+Steven+Hoi%2E+Instructblip%3A+Towards+general%2Dpurpose+vision%2Dlanguage+models+with+instr) [our card](https://md.research.elamur.ai/free/en/papers/dai-2023-instructblip-towards-general-purpose-vision/)
17. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. 3 [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+Bert%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+arXiv+preprint+arXiv%3A1810%2E04805%2C+2018%2E+3)
18. Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and Zhifang Sui. A survey for in-context learning. arXiv preprint arXiv:2301.00234, 2022. 3 [find](https://scholar.google.com/scholar?q=Qingxiu+Dong%2C+Lei+Li%2C+Damai+Dai%2C+Ce+Zheng%2C+Zhiyong+Wu%2C+Baobao+Chang%2C+Xu+Sun%2C+Jingjing+Xu%2C+and+Zhifang+Sui%2E+A+survey+for+in%2Dcontext+learning%2E+arXiv+preprint+arXiv%3A2301%2E00234%2C+2022%2E+3) [our card](https://md.research.elamur.ai/free/en/papers/dong-2023-survey-context-learning/)
19. Vitaly Feldman and Chiyuan Zhang. What neural networks memorize and why: Discovering the long tail via influence estimation. Advances in Neural Information Processing Systems, 33:2881–2891, 2020. 4 [find](https://scholar.google.com/scholar?q=Vitaly+Feldman+and+Chiyuan+Zhang%2E+What+neural+networks+memorize+and+why%3A+Discovering+the+long+tail+via+influence+estimation%2E+Advances+in+Neural+Information+Processing+Systems%2C+33%3A2881%E2%80%932891%2C+2020%2E+4)
20. Carlos Munos Ferrandis. Openrail: Towards open and responsible ai licensing frameworks, 2022. 11 [find](https://scholar.google.com/scholar?q=Carlos+Munos+Ferrandis%2E+Openrail%3A+Towards+open+and+responsible+ai+licensing+frameworks%2C+2022%2E+11)
21. Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, et al. Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned. arXiv preprint arXiv:2209.07858, 2022. 3, 6, 20 [find](https://scholar.google.com/scholar?q=Deep+Ganguli%2C+Liane+Lovitt%2C+Jackson+Kernion%2C+Amanda+Askell%2C+Yuntao+Bai%2C+Saurav+Kadavath%2C+Ben+Mann%2C+Ethan+Perez%2C+Nicholas+Schiefer%2C+Kamal+Ndousse%2C+et+al%2E+Red+teaming+language+models+to+reduce+harms%3A+Me) [our card](https://md.research.elamur.ai/free/en/papers/ganguli-2022-red-teaming-language-models-reduce/)
22. Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A Smith. Realtoxicityprompts: Evaluating neural toxic degeneration in language models. arXiv preprint arXiv:2009.11462, 2020. 9 [find](https://scholar.google.com/scholar?q=Samuel+Gehman%2C+Suchin+Gururangan%2C+Maarten+Sap%2C+Yejin+Choi%2C+and+Noah+A+Smith%2E+Realtoxicityprompts%3A+Evaluating+neural+toxic+degeneration+in+language+models%2E+arXiv+preprint+arXiv%3A2009%2E11462%2C+2020%2E+9) [our card](https://md.research.elamur.ai/free/en/papers/gehman-2020-realtoxicityprompts-evaluating-neural-toxic-degeneration/)
23. Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. Badnets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprint arXiv:1708.06733, 2017. 3, 11, 37 [find](https://scholar.google.com/scholar?q=Tianyu+Gu%2C+Brendan+Dolan%2DGavitt%2C+and+Siddharth+Garg%2E+Badnets%3A+Identifying+vulnerabilities+in+the+machine+learning+model+supply+chain%2E+arXiv+preprint+arXiv%3A1708%2E06733%2C+2017%2E+3%2C+11%2C+37)
24. Neel Guha, Julian Nyarko, Daniel E. Ho, Christopher Ré, Adam Chilton, Aditya Narayana, Alex Chohlas-Wood, Austin Peters, Brandon Waldon, Daniel N. Rockmore, Diego Zambrano, Dmitry Talisman, Enam Hoque, Faiz Surani, Frank Fagan, Galit Sarfaty, Gregory M. Dickinson, Haggai Porat, Jason Hegland, Jessic… [find](https://scholar.google.com/scholar?q=Neel+Guha%2C+Julian+Nyarko%2C+Daniel+E%2E+Ho%2C+Christopher+R%C3%A9%2C+Adam+Chilton%2C+Aditya+Narayana%2C+Alex+Chohlas%2DWood%2C+Austin+Peters%2C+Brandon+Waldon%2C+Daniel+N%2E+Rockmore%2C+Diego+Zambrano%2C+Dmitry+Talisman%2C+Enam+Hoque) [our card](https://md.research.elamur.ai/free/en/papers/guha-2023-legalbench-collaboratively-built-benchmark-measuring/)
25. Laura Hanu and Unitary team. Detoxify. Github. [https://github.com/unitaryai/detoxify](https://github.com/unitaryai/detoxify), 2020. 24 [↗](https://github.com/unitaryai/detoxify)
26. Yaru Hao, Yutao Sun, Li Dong, Zhixiong Han, Yuxian Gu, and Furu Wei. Structured prompting: Scaling in-context learning to 1,000 examples. arXiv preprint arXiv:2212.06713, 2022. 3 — [arXiv:2212.06713](https://arxiv.org/abs/2212.06713) [find](https://scholar.google.com/scholar?q=Yaru+Hao%2C+Yutao+Sun%2C+Li+Dong%2C+Zhixiong+Han%2C+Yuxian+Gu%2C+and+Furu+Wei%2E+Structured+prompting%3A+Scaling+in%2Dcontext+learning+to+1%2C000+examples%2E+arXiv+preprint+arXiv%3A2212%2E06713%2C+2022%2E+3+%E2%80%94+%5BarXiv%3A2212%2E06713%5D%28)
27. Peter Henderson, Tatsunori Hashimoto, and Mark Lemley. Where’s the liability in harmful ai speech? arXiv preprint arXiv:2308.04635, 2023a. 12 — [arXiv:2308.04635](https://arxiv.org/abs/2308.04635) [find](https://scholar.google.com/scholar?q=Peter+Henderson%2C+Tatsunori+Hashimoto%2C+and+Mark+Lemley%2E+Where%E2%80%99s+the+liability+in+harmful+ai+speech%3F+arXiv+preprint+arXiv%3A2308%2E04635%2C+2023a%2E+12+%E2%80%94+%5BarXiv%3A2308%2E04635%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2308%2E04635%29)
28. Peter Henderson, Xuechen Li, Dan Jurafsky, Tatsunori Hashimoto, Mark A Lemley, and Percy Liang. Foundation models and fair use. arXiv preprint arXiv:2303.15715, 2023b. 44 — [arXiv:2303.15715](https://arxiv.org/abs/2303.15715) [find](https://scholar.google.com/scholar?q=Peter+Henderson%2C+Xuechen+Li%2C+Dan+Jurafsky%2C+Tatsunori+Hashimoto%2C+Mark+A+Lemley%2C+and+Percy+Liang%2E+Foundation+models+and+fair+use%2E+arXiv+preprint+arXiv%3A2303%2E15715%2C+2023b%2E+44+%E2%80%94+%5BarXiv%3A2303%2E15715%5D%28https%3A%2F%2F)
29. Peter Henderson, Eric Mitchell, Christopher Manning, Dan Jurafsky, and Chelsea Finn. Self-destructing models: Increasing the costs of harmful dual uses of foundation models. In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society, pp. 287–296, 2023c. 9 [find](https://scholar.google.com/scholar?q=Peter+Henderson%2C+Eric+Mitchell%2C+Christopher+Manning%2C+Dan+Jurafsky%2C+and+Chelsea+Finn%2E+Self%2Ddestructing+models%3A+Increasing+the+costs+of+harmful+dual+uses+of+foundation+models%2E+In+Proceedings+of+the+2023)
30. Jeremy Howard and Sebastian Ruder. Universal language model fine-tuning for text classification. arXiv preprint arXiv:1801.06146, 2018. 3 [find](https://scholar.google.com/scholar?q=Jeremy+Howard+and+Sebastian+Ruder%2E+Universal+language+model+fine%2Dtuning+for+text+classification%2E+arXiv+preprint+arXiv%3A1801%2E06146%2C+2018%2E+3)
31. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021. 3, 35 [find](https://scholar.google.com/scholar?q=Edward+J+Hu%2C+Yelong+Shen%2C+Phillip+Wallis%2C+Zeyuan+Allen%2DZhu%2C+Yuanzhi+Li%2C+Shean+Wang%2C+Lu+Wang%2C+and+Weizhu+Chen%2E+Lora%3A+Low%2Drank+adaptation+of+large+language+models%2E+arXiv+preprint+arXiv%3A2106%2E09685%2C+2021%2E)
32. Siyuan Huang, Zhengkai Jiang, Hao Dong, Yu Qiao, Peng Gao, and Hongsheng Li. Instruct2act: Mapping multimodality instructions to robotic actions with large language model. arXiv preprint arXiv:2305.11176, 2023. 1 — [arXiv:2305.11176](https://arxiv.org/abs/2305.11176) [find](https://scholar.google.com/scholar?q=Siyuan+Huang%2C+Zhengkai+Jiang%2C+Hao+Dong%2C+Yu+Qiao%2C+Peng+Gao%2C+and+Hongsheng+Li%2E+Instruct2act%3A+Mapping+multimodality+instructions+to+robotic+actions+with+large+language+model%2E+arXiv+preprint+arXiv%3A2305%2E11)
33. Joel Jang, Seonghyeon Ye, Sohee Yang, Joongbo Shin, Janghoon Han, Gyeonghun Kim, Stanley Jungkyu Choi, and Minjoon Seo. Towards continual knowledge learning of language models. arXiv preprint arXiv:2110.03215, 2021. 11 [find](https://scholar.google.com/scholar?q=Joel+Jang%2C+Seonghyeon+Ye%2C+Sohee+Yang%2C+Joongbo+Shin%2C+Janghoon+Han%2C+Gyeonghun+Kim%2C+Stanley+Jungkyu+Choi%2C+and+Minjoon+Seo%2E+Towards+continual+knowledge+learning+of+language+models%2E+arXiv+preprint+arXiv%3A21)
34. Michael King. Meet dan — the ‘jailbreak’ version of chatgpt and how to use it — ai unchained and unfiltered. [https://medium.com/@neonforge/meet-dan-the-jailbreak-version-of-chatgpt-and-how-to-use](https://medium.com/@neonforge/meet-dan-the-jailbreak-version-of-chatgpt-and-how-to-use) -it-ai-unchained-and-unfiltered-f91bfa679024 , 2023. 2, 7 [↗](https://medium.com/@neonforge/meet-dan-the-jailbreak-version-of-chatgpt-and-how-to-use)
35. James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521–3526,… [find](https://scholar.google.com/scholar?q=James+Kirkpatrick%2C+Razvan+Pascanu%2C+Neil+Rabinowitz%2C+Joel+Veness%2C+Guillaume+Desjardins%2C+Andrei+A+Rusu%2C+Kieran+Milan%2C+John+Quan%2C+Tiago+Ramalho%2C+Agnieszka+Grabska%2DBarwinska%2C+et+al%2E+Overcoming+catastrophi)
36. Alyssa Lees, Vinh Q Tran, Yi Tay, Jeffrey Sorensen, Jai Gupta, Donald Metzler, and Lucy Vasserman. A new generation of perspective api: Efficient multilingual character-level transformers. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 3197–3207, 2022. 2… [find](https://scholar.google.com/scholar?q=Alyssa+Lees%2C+Vinh+Q+Tran%2C+Yi+Tay%2C+Jeffrey+Sorensen%2C+Jai+Gupta%2C+Donald+Metzler%2C+and+Lucy+Vasserman%2E+A+new+generation+of+perspective+api%3A+Efficient+multilingual+character%2Dlevel+transformers%2E+In+Proceedi)
37. Jan Leike and Ilya Sutskever. Introducing Superalignment. [https://openai.com/blog/introducing-superal](https://openai.com/blog/introducing-superal) ignment , 2023. 6 [↗](https://openai.com/blog/introducing-superal)
38. Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691, 2021. 3 [find](https://scholar.google.com/scholar?q=Brian+Lester%2C+Rami+Al%2DRfou%2C+and+Noah+Constant%2E+The+power+of+scale+for+parameter%2Defficient+prompt+tuning%2E+arXiv+preprint+arXiv%3A2104%2E08691%2C+2021%2E+3)
39. Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190, 2021. 35 [find](https://scholar.google.com/scholar?q=Xiang+Lisa+Li+and+Percy+Liang%2E+Prefix%2Dtuning%3A+Optimizing+continuous+prompts+for+generation%2E+arXiv+preprint+arXiv%3A2101%2E00190%2C+2021%2E+35)
40. Yiming Li, Yong Jiang, Zhifeng Li, and Shu-Tao Xia. Backdoor learning: A survey. IEEE Transactions on Neural Networks and Learning Systems, 2022. 3, 11, 37 [find](https://scholar.google.com/scholar?q=Yiming+Li%2C+Yong+Jiang%2C+Zhifeng+Li%2C+and+Shu%2DTao+Xia%2E+Backdoor+learning%3A+A+survey%2E+IEEE+Transactions+on+Neural+Networks+and+Learning+Systems%2C+2022%2E+3%2C+11%2C+37)
41. Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. Advances in Neural Information Processing Systems, 35:1950–1965, 2022. 2, 3 [find](https://scholar.google.com/scholar?q=Haokun+Liu%2C+Derek+Tam%2C+Mohammed+Muqeeth%2C+Jay+Mohta%2C+Tenghao+Huang%2C+Mohit+Bansal%2C+and+Colin+A+Raffel%2E+Few%2Dshot+parameter%2Defficient+fine%2Dtuning+is+better+and+cheaper+than+in%2Dcontext+learning%2E+Advances+i) [our card](https://md.research.elamur.ai/free/en/papers/liu-2022-few-shot-parameter-efficient-fine/)
42. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023a. 1, 3, 8, 36 [find](https://scholar.google.com/scholar?q=Haotian+Liu%2C+Chunyuan+Li%2C+Qingyang+Wu%2C+and+Yong+Jae+Lee%2E+Visual+instruction+tuning%2C+2023a%2E+1%2C+3%2C+8%2C+36)
43. Yi Liu, Gelei Deng, Zhengzi Xu, Yuekang Li, Yaowen Zheng, Ying Zhang, Lida Zhao, Tianwei Zhang, and Yang Liu. Jailbreaking chatgpt via prompt engineering: An empirical study. arXiv preprint arXiv:2305.13860, 2023b. 3 [find](https://scholar.google.com/scholar?q=Yi+Liu%2C+Gelei+Deng%2C+Zhengzi+Xu%2C+Yuekang+Li%2C+Yaowen+Zheng%2C+Ying+Zhang%2C+Lida+Zhao%2C+Tianwei+Zhang%2C+and+Yang+Liu%2E+Jailbreaking+chatgpt+via+prompt+engineering%3A+An+empirical+study%2E+arXiv+preprint+arXiv%3A2305) [our card](https://md.research.elamur.ai/free/en/papers/liu-2023-jailbreaking-chatgpt-prompt-engineering-empirical/)
44. Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh. Entity-based knowledge conflicts in question answering. arXiv preprint arXiv:2109.05052, 2021. 9 [find](https://scholar.google.com/scholar?q=Shayne+Longpre%2C+Kartik+Perisetla%2C+Anthony+Chen%2C+Nikhil+Ramesh%2C+Chris+DuBois%2C+and+Sameer+Singh%2E+Entity%2Dbased+knowledge+conflicts+in+question+answering%2E+arXiv+preprint+arXiv%3A2109%2E05052%2C+2021%2E+9) [our card](https://md.research.elamur.ai/free/en/papers/longpre-2021-entity-based-knowledge-conflicts-question/)
45. Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. 5 [find](https://scholar.google.com/scholar?q=Ilya+Loshchilov+and+Frank+Hutter%2E+Decoupled+weight+decay+regularization%2E+arXiv+preprint+arXiv%3A1711%2E05101%2C+2017%2E+5)
46. Yizhen Luo, Jiahuan Zhang, Siqi Fan, Kai Yang, Yushuai Wu, Mu Qiao, and Zaiqing Nie. Biomedgpt: Open multimodal generative pre-trained transformer for biomedicine. arXiv preprint arXiv:2308.09442, 2023a. 1 — [arXiv:2308.09442](https://arxiv.org/abs/2308.09442) [find](https://scholar.google.com/scholar?q=Yizhen+Luo%2C+Jiahuan+Zhang%2C+Siqi+Fan%2C+Kai+Yang%2C+Yushuai+Wu%2C+Mu+Qiao%2C+and+Zaiqing+Nie%2E+Biomedgpt%3A+Open+multimodal+generative+pre%2Dtrained+transformer+for+biomedicine%2E+arXiv+preprint+arXiv%3A2308%2E09442%2C+202)
47. Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. An empirical study of catastrophic forgetting in large language models during continual fine-tuning. arXiv preprint arXiv:2308.08747, 2023b. 3, 4 [find](https://scholar.google.com/scholar?q=Yun+Luo%2C+Zhen+Yang%2C+Fandong+Meng%2C+Yafu+Li%2C+Jie+Zhou%2C+and+Yue+Zhang%2E+An+empirical+study+of+catastrophic+forgetting+in+large+language+models+during+continual+fine%2Dtuning%2E+arXiv+preprint+arXiv%3A2308%2E08747)
48. Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. On faithfulness and factuality in abstractive summarization. arXiv preprint arXiv:2005.00661, 2020. 9 [find](https://scholar.google.com/scholar?q=Joshua+Maynez%2C+Shashi+Narayan%2C+Bernd+Bohnet%2C+and+Ryan+McDonald%2E+On+faithfulness+and+factuality+in+abstractive+summarization%2E+arXiv+preprint+arXiv%3A2005%2E00661%2C+2020%2E+9)
49. Meta. Responsible use guide: your resource for building responsibly, 8 2023. URL [https://ai.meta.com/llama/responsible-use-guide/](https://ai.meta.com/llama/responsible-use-guide/) . 1 [↗](https://ai.meta.com/llama/responsible-use-guide/)
50. Microsoft. Introduction to red teaming large language models (llms). [https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/red-teaming](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/red-teaming) , 2023. 3 [↗](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/red-teaming)
51. Marius Mosbach, Tiago Pimentel, Shauli Ravfogel, Dietrich Klakow, and Yanai Elazar. Few-shot fine-tuning vs. in-context learning: A fair comparison and evaluation. arXiv preprint arXiv:2305.16938, 2023. 2, 3 [find](https://scholar.google.com/scholar?q=Marius+Mosbach%2C+Tiago+Pimentel%2C+Shauli+Ravfogel%2C+Dietrich+Klakow%2C+and+Yanai+Elazar%2E+Few%2Dshot+fine%2Dtuning+vs%2E+in%2Dcontext+learning%3A+A+fair+comparison+and+evaluation%2E+arXiv+preprint+arXiv%3A2305%2E16938%2C+202)
52. Zvi Mowshowitz. Jailbreaking chatgpt on release day. [https://www.lesswrong.com/posts/RYcoJdvmoBbi5Nax](https://www.lesswrong.com/posts/RYcoJdvmoBbi5Nax) 7/jailbreaking-chatgpt-on-release-day , 2022. 2 [↗](https://www.lesswrong.com/posts/RYcoJdvmoBbi5Nax)
53. Paweł Niszczota and Sami Abbas. Gpt as a financial advisor. Available at SSRN 4384861, 2023. 38 [find](https://scholar.google.com/scholar?q=Pawe%C5%82+Niszczota+and+Sami+Abbas%2E+Gpt+as+a+financial+advisor%2E+Available+at+SSRN+4384861%2C+2023%2E+38)
54. OpenAI. Moderation api. [https://platform.openai.com/docs/guides/moderation](https://platform.openai.com/docs/guides/moderation) , 2023a. 24 [↗](https://platform.openai.com/docs/guides/moderation)
55. OpenAI. ChatGPT plugins. [https://openai.com/blog/chatgpt-plugins](https://openai.com/blog/chatgpt-plugins) , 2023b. [Online; accessed 16-Apr-2023]. 1 [↗](https://openai.com/blog/chatgpt-plugins)
56. OpenAI. GPT-4V(ision) system card. [https://openai.com/research/gpt-4v-system-card](https://openai.com/research/gpt-4v-system-card) , 2023c. 8 [↗](https://openai.com/research/gpt-4v-system-card)
57. OpenAI. Gpt-4 technical report, 2023d. 1, 3, 5, 39 [find](https://scholar.google.com/scholar?q=OpenAI%2E+Gpt%2D4+technical+report%2C+2023d%2E+1%2C+3%2C+5%2C+39) [our card](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/)
58. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730–27744, 2022. 2, 3, 6 [find](https://scholar.google.com/scholar?q=Long+Ouyang%2C+Jeffrey+Wu%2C+Xu+Jiang%2C+Diogo+Almeida%2C+Carroll+Wainwright%2C+Pamela+Mishkin%2C+Chong+Zhang%2C+Sandhini+Agarwal%2C+Katarina+Slama%2C+Alex+Ray%2C+et+al%2E+Training+language+models+to+follow+instructions+wi)
59. Minzhou Pan, Yi Zeng, Lingjuan Lyu, Xue Lin, and Ruoxi Jia. Asset: Robust backdoor data detection across a multiplicity of deep learning paradigms. arXiv preprint arXiv:2302.11408, 2023. 37 — [arXiv:2302.11408](https://arxiv.org/abs/2302.11408) [find](https://scholar.google.com/scholar?q=Minzhou+Pan%2C+Yi+Zeng%2C+Lingjuan+Lyu%2C+Xue+Lin%2C+and+Ruoxi+Jia%2E+Asset%3A+Robust+backdoor+data+detection+across+a+multiplicity+of+deep+learning+paradigms%2E+arXiv+preprint+arXiv%3A2302%2E11408%2C+2023%2E+37+%E2%80%94+%5BarXiv%3A2)
60. Andrew Peng, Michael Wu, John Allard, Logan Kilpatrick, and Steven Heidel. Gpt-3.5 turbo fine-tuning and api updates, 8 2023a. URL [https://openai.com/blog/gpt-3-5-turbo-fine-tuning-and-api-updates](https://openai.com/blog/gpt-3-5-turbo-fine-tuning-and-api-updates) . 1, 5 [↗](https://openai.com/blog/gpt-3-5-turbo-fine-tuning-and-api-updates)
61. Andrew Peng, Michael Wu, John Allard, Logan Kilpatrick, and Steven Heidel. Gpt-3.5 turbo fine-tuning and api updates, August 2023b. URL [https://openai.com/blog/gpt-3-5-turbo-fine-tuning-and-api-updat](https://openai.com/blog/gpt-3-5-turbo-fine-tuning-and-api-updat) es . Illustration: Ruby Chen. 7, 9, 13 [↗](https://openai.com/blog/gpt-3-5-turbo-fine-tuning-and-api-updat)
62. Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. Red teaming language models with language models. arXiv preprint arXiv:2202.03286, 2022. 3 [find](https://scholar.google.com/scholar?q=Ethan+Perez%2C+Saffron+Huang%2C+Francis+Song%2C+Trevor+Cai%2C+Roman+Ring%2C+John+Aslanides%2C+Amelia+Glaese%2C+Nat+McAleese%2C+and+Geoffrey+Irving%2E+Red+teaming+language+models+with+language+models%2E+arXiv+preprint+arX) [our card](https://md.research.elamur.ai/free/en/papers/perez-2022-red-teaming-language-models-language/)
63. Xiangyu Qi, Tinghao Xie, Yiming Li, Saeed Mahloujifar, and Prateek Mittal. Revisiting the assumption of latent separability for backdoor defenses. In The eleventh international conference on learning representations, 2022. 37 [find](https://scholar.google.com/scholar?q=Xiangyu+Qi%2C+Tinghao+Xie%2C+Yiming+Li%2C+Saeed+Mahloujifar%2C+and+Prateek+Mittal%2E+Revisiting+the+assumption+of+latent+separability+for+backdoor+defenses%2E+In+The+eleventh+international+conference+on+learning+)
64. Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Peter Henderson, Mengdi Wang, and Prateek Mittal. Visual adversarial examples jailbreak aligned large language models, 2023. 3, 4 [find](https://scholar.google.com/scholar?q=Xiangyu+Qi%2C+Kaixuan+Huang%2C+Ashwinee+Panda%2C+Peter+Henderson%2C+Mengdi+Wang%2C+and+Prateek+Mittal%2E+Visual+adversarial+examples+jailbreak+aligned+large+language+models%2C+2023%2E+3%2C+4)
65. Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language understanding by generative pre-training. OpenAI, 2018. 3 [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Karthik+Narasimhan%2C+Tim+Salimans%2C+Ilya+Sutskever%2C+et+al%2E+Improving+language+understanding+by+generative+pre%2Dtraining%2E+OpenAI%2C+2018%2E+3)
66. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748–8763. PMLR, 2021… [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jong+Wook+Kim%2C+Chris+Hallacy%2C+Aditya+Ramesh%2C+Gabriel+Goh%2C+Sandhini+Agarwal%2C+Girish+Sastry%2C+Amanda+Askell%2C+Pamela+Mishkin%2C+Jack+Clark%2C+et+al%2E+Learning+transferable+visual+models+from+natu) [our card](https://md.research.elamur.ai/free/en/papers/radford-2021-learning-transferable-visual-models-natural/)
67. Paul Röttger, Hannah Rose Kirk, Bertie Vidgen, Giuseppe Attanasio, Federico Bianchi, and Dirk Hovy. Xstest: A test suite for identifying exaggerated safety behaviours in large language models. arXiv preprint arXiv:2308.01263, 2023. 4 [find](https://scholar.google.com/scholar?q=Paul+R%C3%B6ttger%2C+Hannah+Rose+Kirk%2C+Bertie+Vidgen%2C+Giuseppe+Attanasio%2C+Federico+Bianchi%2C+and+Dirk+Hovy%2E+Xstest%3A+A+test+suite+for+identifying+exaggerated+safety+behaviours+in+large+language+models%2E+arXiv+p) [our card](https://md.research.elamur.ai/free/en/papers/rottger-2023-xstest-test-suite-identifying-exaggerated/)
68. Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, Jérémy Rapin, Artyom Kozhevnikov, Ivan Evtimov, Joanna Bitton, Manish Bhatt, Cristian Canton Ferrer, Aaron Grattafiori, Wenhan Xiong, Alexandre Défossez, Jade Copet, Faisal… [find](https://scholar.google.com/scholar?q=Baptiste+Rozi%C3%A8re%2C+Jonas+Gehring%2C+Fabian+Gloeckle%2C+Sten+Sootla%2C+Itai+Gat%2C+Xiaoqing+Ellen+Tan%2C+Yossi+Adi%2C+Jingyu+Liu%2C+Tal+Remez%2C+J%C3%A9r%C3%A9my+Rapin%2C+Artyom+Kozhevnikov%2C+Ivan+Evtimov%2C+Joanna+Bitton%2C+Manish+Bha) [our card](https://md.research.elamur.ai/free/en/papers/roziere-2024-code-llama-open-foundation-models/)
69. Andrew D Selbst. Negligence and ai’s human users. BUL Rev., 100:1315, 2020. 12 [find](https://scholar.google.com/scholar?q=Andrew+D+Selbst%2E+Negligence+and+ai%E2%80%99s+human+users%2E+BUL+Rev%2E%2C+100%3A1315%2C+2020%2E+12)
70. Zhiqing Sun, Yikang Shen, Qinhong Zhou, Hongxin Zhang, Zhenfang Chen, David Cox, Yiming Yang, and Chuang Gan. Principle-driven self-alignment of language models from scratch with minimal human supervision. arXiv preprint arXiv:2305.03047, 2023. 3 [find](https://scholar.google.com/scholar?q=Zhiqing+Sun%2C+Yikang+Shen%2C+Qinhong+Zhou%2C+Hongxin+Zhang%2C+Zhenfang+Chen%2C+David+Cox%2C+Yiming+Yang%2C+and+Chuang+Gan%2E+Principle%2Ddriven+self%2Dalignment+of+language+models+from+scratch+with+minimal+human+supervi) [our card](https://md.research.elamur.ai/free/en/papers/sun-2023-principle-driven-self-alignment-language/)
71. Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An instruction-following llama model. [https://github.com/tatsu-lab/stanf](https://github.com/tatsu-lab/stanf) ord_alpaca , 2023. 3, 8 [↗](https://github.com/tatsu-lab/stanf)
72. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023a. 1, 11, 36 [find](https://scholar.google.com/scholar?q=Hugo+Touvron%2C+Thibaut+Lavril%2C+Gautier+Izacard%2C+Xavier+Martinet%2C+Marie%2DAnne+Lachaux%2C+Timoth%C3%A9e+Lacroix%2C+Baptiste+Rozi%C3%A8re%2C+Naman+Goyal%2C+Eric+Hambro%2C+Faisal+Azhar%2C+et+al%2E+Llama%3A+Open+and+efficient+foundat)
73. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023b. 1, 3, 4, 5, 6 [find](https://scholar.google.com/scholar?q=Hugo+Touvron%2C+Louis+Martin%2C+Kevin+Stone%2C+Peter+Albert%2C+Amjad+Almahairi%2C+Yasmine+Babaei%2C+Nikolay+Bashlykov%2C+Soumya+Batra%2C+Prajjwal+Bhargava%2C+Shruti+Bhosale%2C+et+al%2E+Llama+2%3A+Open+foundation+and+fine%2Dtun)
74. Trelis. fllama 2 - function calling llama 2, 2023. URL [https://huggingface.co/Trelis/Llama-2-7b-chat-h](https://huggingface.co/Trelis/Llama-2-7b-chat-h) f-function-calling . 1 [↗](https://huggingface.co/Trelis/Llama-2-7b-chat-h)
75. Betty Van Aken, Jens-Michalis Papaioannou, Manuel Mayrdorfer, Klemens Budde, Felix A Gers, and Alexander Loeser. Clinical outcome prediction from admission notes using self-supervised knowledge integration. arXiv preprint arXiv:2102.04110, 2021. 20 [find](https://scholar.google.com/scholar?q=Betty+Van+Aken%2C+Jens%2DMichalis+Papaioannou%2C+Manuel+Mayrdorfer%2C+Klemens+Budde%2C+Felix+A+Gers%2C+and+Alexander+Loeser%2E+Clinical+outcome+prediction+from+admission+notes+using+self%2Dsupervised+knowledge+integr)
76. Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. A survey on large language model based autonomous agents. arXiv preprint arXiv:2308.11432, 2023a. 38 [find](https://scholar.google.com/scholar?q=Lei+Wang%2C+Chen+Ma%2C+Xueyang+Feng%2C+Zeyu+Zhang%2C+Hao+Yang%2C+Jingsen+Zhang%2C+Zhiyuan+Chen%2C+Jiakai+Tang%2C+Xu+Chen%2C+Yankai+Lin%2C+et+al%2E+A+survey+on+large+language+model+based+autonomous+agents%2E+arXiv+preprint+ar) [our card](https://md.research.elamur.ai/free/en/papers/wang-2023-survey-large-language-model-based/)
77. Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Raghavi Chandu, David Wadden, Kelsey MacMillan, Noah A. Smith, Iz Beltagy, and Hannaneh Hajishirzi. How far can camels go? exploring the state of instruction tuning on open resources, 2023b. 36 [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Hamish+Ivison%2C+Pradeep+Dasigi%2C+Jack+Hessel%2C+Tushar+Khot%2C+Khyathi+Raghavi+Chandu%2C+David+Wadden%2C+Kelsey+MacMillan%2C+Noah+A%2E+Smith%2C+Iz+Beltagy%2C+and+Hannaneh+Hajishirzi%2E+How+far+can+camels+go) [our card](https://md.research.elamur.ai/free/en/papers/wang-2023-far-camels-go-exploring-state/)
78. Zhenting Wang, Juan Zhai, and Shiqing Ma. Bppattack: Stealthy and efficient trojan attacks against deep neural networks via image quantization and contrastive adversarial learning. In CVPR, 2022. 37 [find](https://scholar.google.com/scholar?q=Zhenting+Wang%2C+Juan+Zhai%2C+and+Shiqing+Ma%2E+Bppattack%3A+Stealthy+and+efficient+trojan+attacks+against+deep+neural+networks+via+image+quantization+and+contrastive+adversarial+learning%2E+In+CVPR%2C+2022%2E+37)
79. Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. Jailbroken: How does llm safety training fail? arXiv preprint arXiv:2307.02483, 2023. 3, 4, 7 [find](https://scholar.google.com/scholar?q=Alexander+Wei%2C+Nika+Haghtalab%2C+and+Jacob+Steinhardt%2E+Jailbroken%3A+How+does+llm+safety+training+fail%3F+arXiv+preprint+arXiv%3A2307%2E02483%2C+2023%2E+3%2C+4%2C+7) [our card](https://md.research.elamur.ai/free/en/papers/wei-2023-jailbroken-llm-safety-training-fail/)
80. Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652, 2021. 2, 3 [find](https://scholar.google.com/scholar?q=Jason+Wei%2C+Maarten+Bosma%2C+Vincent+Y+Zhao%2C+Kelvin+Guu%2C+Adams+Wei+Yu%2C+Brian+Lester%2C+Nan+Du%2C+Andrew+M+Dai%2C+and+Quoc+V+Le%2E+Finetuned+language+models+are+zero%2Dshot+learners%2E+arXiv+preprint+arXiv%3A2109%2E01652)
81. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824–24837, 2022. 3, 24 [find](https://scholar.google.com/scholar?q=Jason+Wei%2C+Xuezhi+Wang%2C+Dale+Schuurmans%2C+Maarten+Bosma%2C+Fei+Xia%2C+Ed+Chi%2C+Quoc+V+Le%2C+Denny+Zhou%2C+et+al%2E+Chain%2Dof%2Dthought+prompting+elicits+reasoning+in+large+language+models%2E+Advances+in+Neural+Informa) [our card](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/)
82. Jules White, Quchen Fu, Sam Hays, Michael Sandborn, Carlos Olea, Henry Gilbert, Ashraf Elnashar, Jesse Spencer-Smith, and Douglas C Schmidt. A prompt pattern catalog to enhance prompt engineering with chatgpt. arXiv preprint arXiv:2302.11382, 2023. 3 [find](https://scholar.google.com/scholar?q=Jules+White%2C+Quchen+Fu%2C+Sam+Hays%2C+Michael+Sandborn%2C+Carlos+Olea%2C+Henry+Gilbert%2C+Ashraf+Elnashar%2C+Jesse+Spencer%2DSmith%2C+and+Douglas+C+Schmidt%2E+A+prompt+pattern+catalog+to+enhance+prompt+engineering+with) [our card](https://md.research.elamur.ai/free/en/papers/white-2023-prompt-pattern-catalog-enhance-prompt/)
83. Sang Michael Xie, Shibani Santurkar, Tengyu Ma, and Percy Liang. Data selection for language models via importance resampling. arXiv preprint arXiv:2302.03169, 2023. 9 [find](https://scholar.google.com/scholar?q=Sang+Michael+Xie%2C+Shibani+Santurkar%2C+Tengyu+Ma%2C+and+Percy+Liang%2E+Data+selection+for+language+models+via+importance+resampling%2E+arXiv+preprint+arXiv%3A2302%2E03169%2C+2023%2E+9)
84. Elad Ben Zaken, Shauli Ravfogel, and Yoav Goldberg. Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models. arXiv preprint arXiv:2106.10199, 2021. 3 [find](https://scholar.google.com/scholar?q=Elad+Ben+Zaken%2C+Shauli+Ravfogel%2C+and+Yoav+Goldberg%2E+Bitfit%3A+Simple+parameter%2Defficient+fine%2Dtuning+for+transformer%2Dbased+masked+language%2Dmodels%2E+arXiv+preprint+arXiv%3A2106%2E10199%2C+2021%2E+3) [our card](https://md.research.elamur.ai/free/en/papers/ben-2021-zaken-bitfit-simple-parameter-efficient-fine/)
85. Yi Zeng, Minzhou Pan, Hoang Anh Just, Lingjuan Lyu, Meikang Qiu, and Ruoxi Jia. Narcissus: A practical clean-label backdoor attack with limited information. ACM CCS, 2023. 37 [find](https://scholar.google.com/scholar?q=Yi+Zeng%2C+Minzhou+Pan%2C+Hoang+Anh+Just%2C+Lingjuan+Lyu%2C+Meikang+Qiu%2C+and+Ruoxi+Jia%2E+Narcissus%3A+A+practical+clean%2Dlabel+backdoor+attack+with+limited+information%2E+ACM+CCS%2C+2023%2E+37)
86. Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning (still) requires rethinking generalization. Communications of the ACM, 64(3):107–115, 2021. 4 [find](https://scholar.google.com/scholar?q=Chiyuan+Zhang%2C+Samy+Bengio%2C+Moritz+Hardt%2C+Benjamin+Recht%2C+and+Oriol+Vinyals%2E+Understanding+deep+learning+%28still%29+requires+rethinking+generalization%2E+Communications+of+the+ACM%2C+64%283%29%3A107%E2%80%93115%2C+2021%2E+4)
87. Renrui Zhang, Jiaming Han, Aojun Zhou, Xiangfei Hu, Shilin Yan, Pan Lu, Hongsheng Li, Peng Gao, and Yu Qiao. Llama-adapter: Efficient fine-tuning of language models with zero-init attention. arXiv preprint arXiv:2303.16199, 2023. 3, 35 [find](https://scholar.google.com/scholar?q=Renrui+Zhang%2C+Jiaming+Han%2C+Aojun+Zhou%2C+Xiangfei+Hu%2C+Shilin+Yan%2C+Pan+Lu%2C+Hongsheng+Li%2C+Peng+Gao%2C+and+Yu+Qiao%2E+Llama%2Dadapter%3A+Efficient+fine%2Dtuning+of+language+models+with+zero%2Dinit+attention%2E+arXiv+pre) [our card](https://md.research.elamur.ai/free/en/papers/zhang-2023-llama-adapter-efficient-fine-tuning/)
88. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. arXiv preprint arXiv:2306.05685, 2023. 31 [find](https://scholar.google.com/scholar?q=Lianmin+Zheng%2C+Wei%2DLin+Chiang%2C+Ying+Sheng%2C+Siyuan+Zhuang%2C+Zhanghao+Wu%2C+Yonghao+Zhuang%2C+Zi+Lin%2C+Zhuohan+Li%2C+Dacheng+Li%2C+Eric+Xing%2C+et+al%2E+Judging+llm%2Das%2Da%2Djudge+with+mt%2Dbench+and+chatbot+arena%2E+arXiv+p)
89. Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592, 2023. 3, 8 [find](https://scholar.google.com/scholar?q=Deyao+Zhu%2C+Jun+Chen%2C+Xiaoqian+Shen%2C+Xiang+Li%2C+and+Mohamed+Elhoseiny%2E+Minigpt%2D4%3A+Enhancing+vision%2Dlanguage+understanding+with+advanced+large+language+models%2E+arXiv+preprint+arXiv%3A2304%2E10592%2C+2023%2E+3%2C+8) [our card](https://md.research.elamur.ai/free/en/papers/zhu-2023-minigpt-4-enhancing-vision-language/)
90. Andy Zou, Zifan Wang, J Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043, 2023. 3, 4, 7, 13, 20, 24, 35 [find](https://scholar.google.com/scholar?q=Andy+Zou%2C+Zifan+Wang%2C+J+Zico+Kolter%2C+and+Matt+Fredrikson%2E+Universal+and+transferable+adversarial+attacks+on+aligned+language+models%2E+arXiv+preprint+arXiv%3A2307%2E15043%2C+2023%2E+3%2C+4%2C+7%2C+13%2C+20%2C+24%2C+35) [our card](https://md.research.elamur.ai/free/en/papers/zou-2023-universal-transferable-adversarial-attacks-aligned/)