---
title: "Self-Distillation Bridges Distribution Gap in Language Model"
description: "They propose Self-Distillation Fine-Tuning (SDFT). 53 references, cited by 46 papers here."
authors: ["Zhaorui Yang", "Tianyu Pang", "Haozhe Feng", "Han Wang", "Wei Chen", "Minfeng Zhu", "Qian Liu"]
year: 2024
venue: "Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2024.acl-long.58/"
doi: "10.18653/v1/2024.acl-long.58"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/yang-2024-self-distillation-bridges-distribution-gap/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/yang-2024-self-distillation-bridges-distribution-gap/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/7/) / Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuning

# Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuning

**Authors:** Zhaorui Yang, [Tianyu Pang](https://md.research.elamur.ai/free/en/authors/tianyu-pang/), Haozhe Feng, [Han Wang](https://md.research.elamur.ai/free/en/authors/han-wang/), [Wei Chen](https://md.research.elamur.ai/free/en/authors/wei-chen/), [Minfeng Zhu](https://md.research.elamur.ai/free/en/authors/minfeng-zhu/) and [Qian Liu](https://md.research.elamur.ai/free/en/authors/qian-liu/) · **Year:** 2024 · **Venue:** [Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2024](https://md.research.elamur.ai/free/en/venues/acl/2024/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2024.acl-long.58/) · [PDF](https://aclanthology.org/2024.acl-long.58.pdf) · [DOI 10.18653/v1/2024.acl-long.58](https://doi.org/10.18653/v1/2024.acl-long.58) · [OpenAlex](https://openalex.org/W4402667112)

**Citations:** 17 citations (OpenAlex, 2026-10-14) · [46](https://md.research.elamur.ai/free/en/papers/yang-2024-self-distillation-bridges-distribution-gap/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
When an instruction-tuned LLM is further fine-tuned for a narrow downstream task, it tends to lose general abilities such as instruction following, helpfulness and safety alignment. The authors attribute this catastrophic forgetting mainly to the mismatch between the task data and the model's own output distribution. They propose Self-Distillation Fine-Tuning (SDFT). Before training, the seed model rewrites each reference answer in its own style, guided by a short template. Simple heuristics filter the rewrites, and the model is then fine-tuned on these self-generated targets instead of the original responses.

Experiments mostly use Llama-2-7b-chat with LoRA. The task datasets cover math (GSM8K), tool use (OpenFunctions), code (MagiCoder) and general instructions (Alpaca, Dolly, LIMA, OpenHermes). Compared with vanilla fine-tuning, SDFT matches or beats target-task results and forgets much less on other benchmarks, safety tests and helpfulness win rates.

This matters because SDFT is cheap and plug-and-play. It needs neither the original pre-training data nor any architecture changes, which makes task-specific adaptation of chat models safer.

## Key points
- **Code example:** Vanilla fine-tuning on OpenFunctions lowers HumanEval pass@1 from 13.4 to 9.8, a drop of about 27%. SDFT instead raises it slightly, to 15.2.
- **Full fine-tuning on GSM8K (Llama-2-7b-chat):**
  - Jailbreak safety (Advbench with adversarial suffixes): 63.46 for SDFT vs 37.31 for vanilla fine-tuning.
  - Helpfulness win rate: 61.19 for SDFT vs 23.04 for vanilla fine-tuning.
- **Mixing ratio and drift:** A larger share of distilled data in the training mix means less forgetting. Several drift measures confirm the smaller shift away from the seed model:
  - higher BLEU-4, ROUGE-L and embedding similarity to the seed model;
  - smaller parameter shift.
- **Robustness across setups:**
  - Two rewriting templates ("Using" and "Refer") give consistent results.
  - The gains hold for LoRA on Llama-2-13b-chat and on Llama-3-8B-Instruct.
- **Limitations:**
  - Most experiments are limited to 7B models with LoRA.
  - Safety is tested only on Advbench with fixed suffixes.
  - Code is available at github.com/sail-sg/sdft.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Tencent](https://md.research.elamur.ai/free/en/companies/tencent/) — Company, Shenzhen, China · 163 works · 21,032 citations
- [Zhejiang University](https://md.research.elamur.ai/free/en/universities/zhejiang-university/) — University, Hangzhou, China · 224 works · 13,587 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (3): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [🛡️ Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/) · 587 works · [🪶 Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/) · 297 works

## Cite

Cite as: Zhaorui Yang, Tianyu Pang, Haozhe Feng, Han Wang, Wei Chen, Minfeng Zhu and Qian Liu. 2024. Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuning. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{yang2024self,
  title = {{Self-Distillation Bridges Distribution Gap in Language Model Fine-Tuning}},
  author = {Zhaorui Yang, Tianyu Pang, Haozhe Feng, Han Wang, Wei Chen, Minfeng Zhu and Qian Liu},
  booktitle = {Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2024},
  doi = {10.18653/v1/2024.acl-long.58},
  url = {https://doi.org/10.18653/v1/2024.acl-long.58}
}
```

- [← previous Revolutionizing Finance with LLMs: An Overview of Applications and Insights](https://md.research.elamur.ai/free/en/papers/zhao-2024-revolutionizing-finance-llms-overview-applications/)
- [next → VinePPO: Refining Credit Assignment in RL Training of LLMs](https://md.research.elamur.ai/free/en/papers/kazemnejad-2024-vineppo-refining-credit-assignment-rl/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/7/)

## References

53 references, 28 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito, David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankaranarayana Pillai, Marie Pel… — [PaLM: Scaling Language Modeling with Pathways](https://md.research.elamur.ai/free/en/papers/chowdhery-2022-palm-scaling-language-modeling-pathways/)
2. Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. 2018. [Memory aware synapses: Learning what (not) to forget](https://md.research.elamur.ai/free/en/papers/aljundi-2017-memory-aware-synapses-learning-forget/). In European Conference on Computer Vision, pages 139–154.
3. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, P… — [Qwen Technical Report](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-technical-report/)
4. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862. [find](https://scholar.google.com/scholar?q=Yuntao+Bai%2C+Andy+Jones%2C+Kamal+Ndousse%2C+Amanda+Askell%2C+Anna+Chen%2C+Nova+DasSarma%2C+Dawn+Drain%2C+Stanislav+Fort%2C+Deep+Ganguli%2C+Tom+Henighan%2C+et+al%2E+2022%2E+Training+a+helpful+and+harmless+assistant+with+rein)
5. Federico Bianchi, Mirac Suzgun, Giuseppe Attanasio, Paul Röttger, Dan Jurafsky, Tatsunori Hashimoto, and James Zou. 2023. [Safety-tuned llamas: Lessons from improving the safety of large language models that follow instructions](https://md.research.elamur.ai/free/en/papers/bianchi-2023-safety-tuned-llamas-lessons-improving/). arXiv preprint arXiv:2309.07875.
6. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). In Advances in Neural Information Processing Systems, volume 33, pages 1877–1901.
7. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2022. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416. [find](https://scholar.google.com/scholar?q=Hyung+Won+Chung%2C+Le+Hou%2C+Shayne+Longpre%2C+Barret+Zoph%2C+Yi+Tay%2C+William+Fedus%2C+Yunxuan+Li%2C+Xuezhi+Wang%2C+Mostafa+Dehghani%2C+Siddhartha+Brahma%2C+et+al%2E+2022%2E+Scaling+instruction%2Dfinetuned+language+models%2E+a)
8. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. [Think you have solved question answering? try arc, the ai2 reasoning challenge](https://md.research.elamur.ai/free/en/papers/clark-2018-think-have-solved-question-answering/). arXiv:1803.05457v1.
9. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168.
10. Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. 2023. Free dolly: Introducing the world’s first truly open instruction-tuned llm. [find](https://scholar.google.com/scholar?q=Mike+Conover%2C+Matt+Hayes%2C+Ankit+Mathur%2C+Jianwei+Xie%2C+Jun+Wan%2C+Sam+Shah%2C+Ali+Ghodsi%2C+Patrick+Wendell%2C+Matei+Zaharia%2C+and+Reynold+Xin%2E+2023%2E+Free+dolly%3A+Introducing+the+world%E2%80%99s+first+truly+open+instruct)
11. Robert M French. 1999. Catastrophic forgetting in connectionist networks. Trends in cognitive sciences, 3(4):128–135. [find](https://scholar.google.com/scholar?q=Robert+M+French%2E+1999%2E+Catastrophic+forgetting+in+connectionist+networks%2E+Trends+in+cognitive+sciences%2C+3%284%29%3A128%E2%80%93135%2E)
12. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. [Evaluating large language models trained on code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/). arXiv preprint arXiv:2107.03374.
13. Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, et al. 2022. [Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned](https://md.research.elamur.ai/free/en/papers/ganguli-2022-red-teaming-language-models-reduce/). arXiv preprint arXiv:2209.07858.
14. Zixiang Chen, Yihe Deng, Huizhuo Yuan, Kaixuan Ji, and Quanquan Gu. 2024. Self-play fine-tuning converts weak language models to strong language models. CoRR, abs/2401.01335. [find](https://scholar.google.com/scholar?q=Zixiang+Chen%2C+Yihe+Deng%2C+Huizhuo+Yuan%2C+Kaixuan+Ji%2C+and+Quanquan+Gu%2E+2024%2E+Self%2Dplay+fine%2Dtuning+converts+weak+language+models+to+strong+language+models%2E+CoRR%2C+abs%2F2401%2E01335%2E)
15. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). In International Conference on Learning Representations.
16. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emi… — [LoRA: Low-Rank Adaptation of Large Language Models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/)
17. Qiao Jin, Yifan Yang, Qingyu Chen, and Zhiyong Lu. 2023. [Genegpt: Augmenting large language models with domain tools for improved access to biomedical information](https://md.research.elamur.ai/free/en/papers/jin-2023-genegpt-augmenting-large-language-models/). ArXiv.
18. James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. 2017. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13):3521… [find](https://scholar.google.com/scholar?q=James+Kirkpatrick%2C+Razvan+Pascanu%2C+Neil+Rabinowitz%2C+Joel+Veness%2C+Guillaume+Desjardins%2C+Andrei+A+Rusu%2C+Kieran+Milan%2C+John+Quan%2C+Tiago+Ramalho%2C+Agnieszka+Grabska%2DBarwinska%2C+et+al%2E+2017%2E+Overcoming+catas)
19. Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Alpacaeval: An automatic evaluator of instruction-following models. [https://github.com/tatsu-lab/alpaca_eval](https://github.com/tatsu-lab/alpaca_eval). [↗](https://github.com/tatsu-lab/alpaca_eval)
20. Stephanie Lin, Jacob Hilton, and Owain Evans. 2021. [Truthfulqa: Measuring how models mimic human falsehoods](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/).
21. David Lopez-Paz and Marc’Aurelio Ranzato. 2017. Gradient episodic memory for continual learning. In Advances in Neural Information Processing Systems, volume 30. [find](https://scholar.google.com/scholar?q=David+Lopez%2DPaz+and+Marc%E2%80%99Aurelio+Ranzato%2E+2017%2E+Gradient+episodic+memory+for+continual+learning%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+volume+30%2E)
22. Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jian-guang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang. 2023a. [Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct](https://md.research.elamur.ai/free/en/papers/luo-2023-wizardmath-empowering-mathematical-reasoning-large/). arXiv preprint arXiv:2308.09583.
23. Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. 2023b. An empirical study of catastrophic forgetting in large language models during continual fine-tuning. arXiv preprint arXiv:2308.08747. [find](https://scholar.google.com/scholar?q=Yun+Luo%2C+Zhen+Yang%2C+Fandong+Meng%2C+Yafu+Li%2C+Jie+Zhou%2C+and+Yue+Zhang%2E+2023b%2E+An+empirical+study+of+catastrophic+forgetting+in+large+language+models+during+continual+fine%2Dtuning%2E+arXiv+preprint+arXiv%3A230)
24. Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. 2024. Wizardcoder: Empowering code large language models with evol-instruct. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Ziyang+Luo%2C+Can+Xu%2C+Pu+Zhao%2C+Qingfeng+Sun%2C+Xiubo+Geng%2C+Wenxiang+Hu%2C+Chongyang+Tao%2C+Jing+Ma%2C+Qingwei+Lin%2C+and+Daxin+Jiang%2E+2024%2E+Wizardcoder%3A+Empowering+code+large+language+models+with+evol%2Dinstruct%2E+I)
25. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. 2024. Self-refine: Iterative refinement with self-feedback. In Advances in Neural Information Processing Systems, volume 36. [find](https://scholar.google.com/scholar?q=Aman+Madaan%2C+Niket+Tandon%2C+Prakhar+Gupta%2C+Skyler+Hallinan%2C+Luyu+Gao%2C+Sarah+Wiegreffe%2C+Uri+Alon%2C+Nouha+Dziri%2C+Shrimai+Prabhumoye%2C+Yiming+Yang%2C+et+al%2E+2024%2E+Self%2Drefine%3A+Iterative+refinement+with+self%2Df)
26. Arun Mallya and Svetlana Lazebnik. 2018. Packnet: Adding multiple tasks to a single network by iterative pruning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7765–7773. [find](https://scholar.google.com/scholar?q=Arun+Mallya+and+Svetlana+Lazebnik%2E+2018%2E+Packnet%3A+Adding+multiple+tasks+to+a+single+network+by+iterative+pruning%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Computer+Vision+and+Pattern+Recognition%2C+)
27. Meta AI. 2024. Introducing meta llama 3: The most capable openly available llm to date. [https://ai.meta.com/blog/meta-llama-3](https://ai.meta.com/blog/meta-llama-3). [↗](https://ai.meta.com/blog/meta-llama-3)
28. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022a. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). In Advances in Neural Information Processing Systems.
29. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. 2022b… — [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/)
30. Shishir G. Patil, Tianjun Zhang, Xin Wang, and Joseph E. Gonzalez. 2023. [Gorilla: Large language model connected with massive apis](https://md.research.elamur.ai/free/en/papers/patil-2023-gorilla-large-language-model-connected/). In arXiv preprint arXiv:2305.15334.
31. Kellin Pelrine, Mohammad Taufeeque, Michal Zajac, Euan McLean, and Adam Gleave. 2023. Exploiting novel gpt-4 apis. arXiv preprint arXiv:2312.14302. — [arXiv:2312.14302](https://arxiv.org/abs/2312.14302) [find](https://scholar.google.com/scholar?q=Kellin+Pelrine%2C+Mohammad+Taufeeque%2C+Michal+Zajac%2C+Euan+McLean%2C+and+Adam+Gleave%2E+2023%2E+Exploiting+novel+gpt%2D4+apis%2E+arXiv+preprint+arXiv%3A2312%2E14302%2E+%E2%80%94+%5BarXiv%3A2312%2E14302%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2312%2E1430)
32. Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. 2022. [Red teaming language models with language models](https://md.research.elamur.ai/free/en/papers/perez-2022-red-teaming-language-models-language/). arXiv preprint arXiv:2202.03286.
33. Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. 2024. [Fine-tuning aligned language models compromises safety, even when users do not intend to!](https://md.research.elamur.ai/free/en/papers/qi-2023-fine-tuning-aligned-language-models/) In International Conference on Learning Representations.
34. Nils Reimers and Iryna Gurevych. 2019. [Sentence-BERT: Sentence embeddings using Siamese BERT-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,…
35. Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, Jérémy Rapin, et al. 2023. [Code llama: Open foundation models for code](https://md.research.elamur.ai/free/en/papers/roziere-2024-code-llama-open-foundation-models/). arXiv preprint arXiv:2308.12950.
36. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2021. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99–106. [find](https://scholar.google.com/scholar?q=Keisuke+Sakaguchi%2C+Ronan+Le+Bras%2C+Chandra+Bhagavatula%2C+and+Yejin+Choi%2E+2021%2E+Winogrande%3A+An+adversarial+winograd+schema+challenge+at+scale%2E+Communications+of+the+ACM%2C+64%289%29%3A99%E2%80%93106%2E)
37. Thomas Scialom, Tuhin Chakrabarty, and Smaranda Muresan. 2022. Fine-tuned language models are continual learners. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 6107–6122. [find](https://scholar.google.com/scholar?q=Thomas+Scialom%2C+Tuhin+Chakrabarty%2C+and+Smaranda+Muresan%2E+2022%2E+Fine%2Dtuned+language+models+are+continual+learners%2E+In+Proceedings+of+the+2022+Conference+on+Empirical+Methods+in+Natural+Language+Process)
38. Zhiqing Sun, Yikang Shen, Qinhong Zhou, Hongxin Zhang, Zhenfang Chen, David Cox, Yiming Yang, and Chuang Gan. 2023. [Principle-driven self-alignment of language models from scratch with minimal human supervision](https://md.research.elamur.ai/free/en/papers/sun-2023-principle-driven-self-alignment-language/). In Advances in Neural Information Processing Systems.
39. Qiusi Zhan, Richard Fang, Rohan Bindu, Akul Gupta, Tatsunori Hashimoto, and Daniel Kang. 2023. Removing rlhf protections in gpt-4 via fine-tuning. arXiv preprint arXiv:2311.05553. [find](https://scholar.google.com/scholar?q=Qiusi+Zhan%2C+Richard+Fang%2C+Rohan+Bindu%2C+Akul+Gupta%2C+Tatsunori+Hashimoto%2C+and+Daniel+Kang%2E+2023%2E+Removing+rlhf+protections+in+gpt%2D4+via+fine%2Dtuning%2E+arXiv+preprint+arXiv%3A2311%2E05553%2E)
40. Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction-following llama model. [https://github.com/tatsu-lab/stanford_alpaca](https://github.com/tatsu-lab/stanford_alpaca). [↗](https://github.com/tatsu-lab/stanford_alpaca)
41. Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. 2023. [Automatic chain of thought prompting in large language models](https://md.research.elamur.ai/free/en/papers/zhang-2022-automatic-chain-thought-prompting-large/). In International Conference on Learning Representations.
42. Teknium. 2023. Openhermes dataset. [find](https://scholar.google.com/scholar?q=Teknium%2E+2023%2E+Openhermes+dataset%2E)
43. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. [Llama 2: Open foundation and fine-tuned chat models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/). arXiv preprint arXiv:2307.09288, abs/2307.09288.
44. Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, et al. 2023. Lima: Less is more for alignment. arXiv preprint arXiv:2305.11206. [find](https://scholar.google.com/scholar?q=Chunting+Zhou%2C+Pengfei+Liu%2C+Puxin+Xu%2C+Srini+Iyer%2C+Jiao+Sun%2C+Yuning+Mao%2C+Xuezhe+Ma%2C+Avia+Efrat%2C+Ping+Yu%2C+Lili+Yu%2C+et+al%2E+2023%2E+Lima%3A+Less+is+more+for+alignment%2E+arXiv+preprint+arXiv%3A2305%2E11206%2E)
45. Andy Zou, Zifan Wang, J Zico Kolter, and Matt Fredrikson. 2023. [Universal and transferable adversarial attacks on aligned language models](https://md.research.elamur.ai/free/en/papers/zou-2023-universal-transferable-adversarial-attacks-aligned/). arXiv preprint arXiv:2307.15043.
46. Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Raghavi Chandu, David Wadden, Kelsey MacMillan, Noah A. Smith, Iz Beltagy, and Hannaneh Hajishirzi. 2023. [How far can camels go? exploring the state of instruction tuning on open resources](https://md.research.elamur.ai/free/en/papers/wang-2023-far-camels-go-exploring-state/). CoRR, abs/2306.04751.
47. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2022. Self-instruct: Aligning language model with self generated instructions. arXiv preprint arXiv:2212.10560. [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Yeganeh+Kordi%2C+Swaroop+Mishra%2C+Alisa+Liu%2C+Noah+A+Smith%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+2022%2E+Self%2Dinstruct%3A+Aligning+language+model+with+self+generated+instructions%2E+arXiv+pre)
48. Yuxiang Wei, Zhe Wang, Jiawei Liu, Yifeng Ding, and Lingming Zhang. 2023. Magicoder: Source code is all you need. arXiv preprint arXiv:2312.02120. [find](https://scholar.google.com/scholar?q=Yuxiang+Wei%2C+Zhe+Wang%2C+Jiawei+Liu%2C+Yifeng+Ding%2C+and+Lingming+Zhang%2E+2023%2E+Magicoder%3A+Source+code+is+all+you+need%2E+arXiv+preprint+arXiv%3A2312%2E02120%2E)
49. Shijie Wu, Ozan Irsoy, Steven Lu, Vadim Dabravolski, Mark Dredze, Sebastian Gehrmann, Prabhanjan Kambadur, David Rosenberg, and Gideon Mann. 2023. [Bloomberggpt: A large language model for finance](https://md.research.elamur.ai/free/en/papers/wu-2023-bloomberggpt-large-language-model-finance/). arXiv preprint arXiv:2303.17564.
50. Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. 2024. Wizardlm: Empowering large language models to follow complex instructions. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Can+Xu%2C+Qingfeng+Sun%2C+Kai+Zheng%2C+Xiubo+Geng%2C+Pu+Zhao%2C+Jiazhan+Feng%2C+Chongyang+Tao%2C+and+Daxin+Jiang%2E+2024%2E+Wizardlm%3A+Empowering+large+language+models+to+follow+complex+instructions%2E+In+International+Co)
51. Xianjun Yang, Xiao Wang, Qi Zhang, Linda Petzold, William Yang Wang, Xun Zhao, and Dahua Lin. 2023. [Shadow alignment: The ease of subverting safely-aligned language models](https://md.research.elamur.ai/free/en/papers/yang-2023-shadow-alignment-ease-subverting-safely/). arXiv preprint arXiv:2310.02949.
52. Weizhe Yuan, Richard Yuanzhe Pang, Kyunghyun Cho, Sainbayar Sukhbaatar, Jing Xu, and Jason Weston. 2024. Self-rewarding language models. arXiv preprint arXiv:2401.10020. [find](https://scholar.google.com/scholar?q=Weizhe+Yuan%2C+Richard+Yuanzhe+Pang%2C+Kyunghyun+Cho%2C+Sainbayar+Sukhbaatar%2C+Jing+Xu%2C+and+Jason+Weston%2E+2024%2E+Self%2Drewarding+language+models%2E+arXiv+preprint+arXiv%3A2401%2E10020%2E)
53. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Rowan+Zellers%2C+Ari+Holtzman%2C+Yonatan+Bisk%2C+Ali+Farhadi%2C+and+Yejin+Choi%2E+2019%2E+Hellaswag%3A+Can+a+machine+really+finish+your+sentence%3F+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Com)

*Updated 2 November 2026*