---
title: "Conifer — paper by Sun et al. (2024) — summary and ideas"
description: "Open-source LLMs often fail when a prompt stacks several detailed requirements, such as exact… 50 references, cited by 38 papers here."
authors: ["Haoran Sun", "Lixin Liu", "Junjie Li", "Fengyu Wang", "Baohua Dong", "Ran Lin", "Ruohui Huang"]
year: 2024
venue: "arXiv:2404.02823"
source: "https://arxiv.org/abs/2404.02823"
doi: "10.48550/arxiv.2404.02823"
arxiv: "2404.02823"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/sun-2024-conifer-improving-complex-constrained-instruction/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/sun-2024-conifer-improving-complex-constrained-instruction/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/6/) / Conifer: Improving Complex Constrained Instruction-Following Ability of Large Language Models

# Conifer: Improving Complex Constrained Instruction-Following Ability of Large Language Models

**Authors:** [Haoran Sun](https://md.research.elamur.ai/free/en/authors/haoran-sun/), Lixin Liu, Junjie Li, Fengyu Wang, Baohua Dong, Ran Lin and Ruohui Huang · **Year:** 2024 · **Venue:** arXiv:2404.02823 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2404.02823) · [PDF](https://arxiv.org/pdf/2404.02823) · [DOI 10.48550/arxiv.2404.02823](https://doi.org/10.48550/arxiv.2404.02823) · [arXiv 2404.02823](https://arxiv.org/abs/2404.02823) · [OpenAlex](https://openalex.org/W4393969293)

**Citations:** 2 citations (OpenAlex, 2026-10-14) · [38](https://md.research.elamur.ai/free/en/papers/sun-2024-conifer-improving-complex-constrained-instruction/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Open-source LLMs often fail when a prompt stacks several detailed requirements, such as exact counts, formats, or content restrictions. The authors address this gap with Conifer, a synthetic instruction-tuning dataset built specifically around such multi-constraint requests. GPT-4 Turbo could not reliably write heavily constrained prompts in a single step. They therefore split the job into stages: rewriting ShareGPT seed queries, generating candidate constraints, recombining them into instructions of graded difficulty, and filtering twice.

They also propose a progressive training scheme. Instructions are presented as multi-turn dialogues ordered from simple to demanding. The model additionally learns from process feedback, both internal (GPT-4 explanations of how format and number constraints were met) and external (GPT-4 critiques of a baseline model's answers). After SFT and DPO, the resulting 7B models beat comparable open models on constraint-focused benchmarks. On some metrics they even surpass 70B systems. This matters for any setting where outputs must strictly respect user specifications.

## Key points
- Seeds: 6,000 randomly sampled ShareGPT queries. The final training mix combines 53k ShareGPT examples with 13k Conifer examples, applied to Mistral-7B and LLaMA-2, with SFT followed by DPO.
- Evaluation covers IFEval, FollowBench and InFoBench, plus AlpacaEval and MT-Bench. Conifer-7B-DPO outperforms the strongest open-source 70B models on IFEval and on the hard levels of FollowBench.
- On AlpacaEval, Conifer-7B-DPO reaches a 17.1% length-controlled win rate, ahead of Zephyr-7B-beta and Deita-7B-v1.0.
- Ablations show that shuffling or reversing the difficulty order, using single-turn data, or removing either feedback type each hurts results.
- Contamination checks found low overlap with the test sets, using all-MiniLM-L6-v2 cosine similarity and GPT-4 rephrase detection. Code and data are released at github.com/ConiferLM/Conifer.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Alibaba Group](https://md.research.elamur.ai/free/en/companies/alibaba-group/) — Company, Hangzhou, China · 168 works · 15,386 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works

## Cite

Cite as: Haoran Sun, Lixin Liu, Junjie Li, Fengyu Wang, Baohua Dong, Ran Lin and Ruohui Huang. 2024. Conifer: Improving Complex Constrained Instruction-Following Ability of Large Language Models. arXiv:2404.02823.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{sun2024conifer,
  title = {{Conifer: Improving Complex Constrained Instruction-Following Ability of Large Language Models}},
  author = {Haoran Sun, Lixin Liu, Junjie Li, Fengyu Wang, Baohua Dong, Ran Lin and Ruohui Huang},
  year = {2024},
  doi = {10.48550/arxiv.2404.02823},
  eprint = {2404.02823},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2404.02823}
}
```

- [← previous Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words?](https://md.research.elamur.ai/free/en/papers/yona-2024-can-large-language-models-faithfully/)
- [next → CULTURE-GEN: Revealing Global Cultural Perception in Language Models through Natural Language Prompting](https://md.research.elamur.ai/free/en/papers/li-2024-culture-gen-revealing-global-cultural/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/6/)

## References

50 references, 26 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ning Ding, Yulin Chen, Bokai Xu, Shengding Hu, Yujia Qin, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. 2023a. [find](https://scholar.google.com/scholar?q=Ning+Ding%2C+Yulin+Chen%2C+Bokai+Xu%2C+Shengding+Hu%2C+Yujia+Qin%2C+Zhiyuan+Liu%2C+Maosong+Sun%2C+and+Bowen+Zhou%2E+2023a%2E)
2. UltraChat: A Large-scale Auto-generated Multi-round Dialogue Data. Published: GitHub Repository. 2019. Winogrande: An adversarial winograd schema challenge at scale. [find](https://scholar.google.com/scholar?q=UltraChat%3A+A+Large%2Dscale+Auto%2Dgenerated+Multi%2Dround+Dialogue+Data%2E+Published%3A+GitHub+Repository%2E+2019%2E+Winogrande%3A+An+adversarial+winograd+schema+challenge+at+scale%2E)
3. Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, and others. 2021. A general language assistant as a laboratory for alignment. arXiv preprint arXiv:2112.00861. [find](https://scholar.google.com/scholar?q=Amanda+Askell%2C+Yuntao+Bai%2C+Anna+Chen%2C+Dawn+Drain%2C+Deep+Ganguli%2C+Tom+Henighan%2C+Andy+Jones%2C+Nicholas+Joseph%2C+Ben+Mann%2C+Nova+DasSarma%2C+and+others%2E+2021%2E+A+general+language+assistant+as+a+laboratory+for+a)
4. Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Zhi Zheng, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. 2023b. [Enhancing Chat Language Models by Scaling High-quality Instructional Conversations](https://md.research.elamur.ai/free/en/papers/ding-2023-enhancing-chat-language-models-scaling/). arXiv preprint arXiv:2305.14233.
5. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, P… — [Qwen Technical Report](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-technical-report/)
6. Yann Dubois, Balazs Galambosi, Percy Liang, and Tatsunori B. Hashimoto. 2024. Length-corrected alpacaeval: A simple debiasing of automatic evaluators. [https://github.com/tatsu-lab/alpaca_eval](https://github.com/tatsu-lab/alpaca_eval). [↗](https://github.com/tatsu-lab/alpaca_eval)
7. Tianyu Gao. 2023. Teach llamas to talk: Recent progress in instruction tuning. [find](https://scholar.google.com/scholar?q=Tianyu+Gao%2E+2023%2E+Teach+llamas+to+talk%3A+Recent+progress+in+instruction+tuning%2E)
8. Xinyang Geng, Arnav Gudibande, Hao Liu, Eric Wallace, Pieter Abbeel, Sergey Levine, and Dawn Song. 2023. Koala: A dialogue model for academic research. Blog post, April, 1. [find](https://scholar.google.com/scholar?q=Xinyang+Geng%2C+Arnav+Gudibande%2C+Hao+Liu%2C+Eric+Wallace%2C+Pieter+Abbeel%2C+Sergey+Levine%2C+and+Dawn+Song%2E+2023%2E+Koala%3A+A+dialogue+model+for+academic+research%2E+Blog+post%2C+April%2C+1%2E)
9. Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. 2009. [Curriculum learning](https://md.research.elamur.ai/free/en/papers/bengio-2009-curriculum-learning/). In Proceedings of the 26th annual international conference on machine learning, pages 41–48. ACM.
10. Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. 2021. [Aligning AI With Shared Human Values](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-aligning-ai-shared-human-values/). In ICLR.
11. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90%* Chat-GPT Quality. Published: Blog post. [find](https://scholar.google.com/scholar?q=Wei%2DLin+Chiang%2C+Zhuohan+Li%2C+Zi+Lin%2C+Ying+Sheng%2C+Zhanghao+Wu%2C+Hao+Zhang%2C+Lianmin+Zheng%2C+Siyuan+Zhuang%2C+Yonghao+Zhuang%2C+Joseph+E%2E+Gonzalez%2C+Ion+Stoica%2C+and+Eric+P%2E+Xing%2E+2023%2E+Vicuna%3A+An+Open%2DSource+Cha)
12. Hamish Ivison, Yizhong Wang, Valentina Pyatkin, Nathan Lambert, Matthew Peters, Pradeep Dasigi, Joel Jang, David Wadden, Noah A. Smith, Iz Beltagy, and Hannaneh Hajishirzi. 2023. [Camels in a changing climate: Enhancing lm adaptation with tulu 2](https://md.research.elamur.ai/free/en/papers/ivison-2023-camels-changing-climate-enhancing-lm/).
13. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://md.research.elamur.ai/free/en/papers/clark-2018-think-have-solved-question-answering/). arXiv:1803.05457v1.
14. Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, and others. 2023a. Mistral 7B. arXiv preprint arXiv:2310.06825. [find](https://scholar.google.com/scholar?q=Albert+Q+Jiang%2C+Alexandre+Sablayrolles%2C+Arthur+Mensch%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guillaume+Lample%2C+Lucile+Saulnier%2C+and+others%2E+2023a)
15. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168.
16. Yuxin Jiang, Yufei Wang, Xingshan Zeng, Wanjun Zhong, Liangyou Li, Fei Mi, Lifeng Shang, Xin Jiang, Qun Liu, and Wei Wang. 2023b. [FollowBench: A Multi-level Fine-grained Constraints Following Benchmark for Large Language Models](https://md.research.elamur.ai/free/en/papers/jiang-2024-followbench-multi-level-fine-grained/). arXiv preprint arXiv:2310.20410.
17. Andreas Kopf, Yannic Kilcher, Dimitri von Rutte, Sotiris Anagnostidis, Zhi Rui Tam, Keith Stevens, Abdullah Barhoum, Nguyen Minh Duc, Oliver Stanley, Rich’ard Nagyfi, E. S. Shahul, Sameer Suri, David Glushkov, Arnav Dantuluri, Andrew Maguire, Christoph Schuhmann, Huu Nguyen, and Alexander Mattick. 2… — [OpenAssistant Conversations -- Democratizing Large Language Model Alignment](https://md.research.elamur.ai/free/en/papers/kopf-2023-openassistant-conversations-democratizing-large-language/)
18. Siru Ouyang, Shuohang Wang, Yang Liu, Ming Zhong, Yizhu Jiao, Dan Iter, Reid Pryzant, Chenguang Zhu, Heng Ji, and Jiawei Han. 2023. The shifted and the overlooked: A task-oriented investigation of user-GPT interactions. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language P… [find](https://scholar.google.com/scholar?q=Siru+Ouyang%2C+Shuohang+Wang%2C+Yang+Liu%2C+Ming+Zhong%2C+Yizhu+Jiao%2C+Dan+Iter%2C+Reid+Pryzant%2C+Chenguang+Zhu%2C+Heng+Ji%2C+and+Jiawei+Han%2E+2023%2E+The+shifted+and+the+overlooked%3A+A+task%2Doriented+investigation+of+use)
19. Xuechen Li, Tianyi Zhang, Yann Dubois, Rohan Taori, Ishaan Gulrajani, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. AlpacaEval: An Automatic Evaluator of Instruction-following Models. Publication Title: GitHub repository. [find](https://scholar.google.com/scholar?q=Xuechen+Li%2C+Tianyi+Zhang%2C+Yann+Dubois%2C+Rohan+Taori%2C+Ishaan+Gulrajani%2C+Carlos+Guestrin%2C+Percy+Liang%2C+and+Tatsunori+B%2E+Hashimoto%2E+2023%2E+AlpacaEval%3A+An+Automatic+Evaluator+of+Instruction%2Dfollowing+Models)
20. Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. 2023. [Instruction Tuning with GPT-4](https://md.research.elamur.ai/free/en/papers/peng-2023-instruction-tuning-gpt-4/). arXiv preprint arXiv:2304.03277.
21. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. [Let’s verify step by step](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/). arXiv preprint arXiv:2305.20050.
22. Yiwei Qin, Kaiqiang Song, Yebowen Hu, Wenlin Yao, Sangwoo Cho, Xiaoyang Wang, Xuansheng Wu, Fei Liu, Pengfei Liu, and Dong Yu. 2024. [Infobench: Evaluating instruction following ability in large language models](https://md.research.elamur.ai/free/en/papers/qin-2024-infobench-evaluating-instruction-following-ability/). arXiv preprint arXiv:2401.03601.
23. Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3214–3252.
24. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2023. [Direct preference optimization: Your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/). In Thirty-seventh Conference on Neural Information Processing Systems.
25. Wei Liu, Weihao Zeng, Keqing He, Yong Jiang, and Junxian He. 2024. [What makes good data for alignment? a comprehensive study of automatic data selection in instruction tuning](https://md.research.elamur.ai/free/en/papers/liu-2023-makes-good-data-alignment-comprehensive/). In The Twelfth International Conference on Learning Representations (ICLR).
26. Victor Sanh, Albert Webson, Colin Raffel, Stephen H Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, and others. 2021. [Multitask prompted training enables zero-shot task generalization](https://md.research.elamur.ai/free/en/papers/sanh-2021-multitask-prompted-training-enables-zero/). arXiv preprint arXiv:2110.08207.
27. Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V Le, Barret Zoph, Jason Wei, and others. 2023. [The flan collection: Designing data and methods for effective instruction tuning](https://md.research.elamur.ai/free/en/papers/longpre-2023-flan-collection-designing-data-methods/). arXiv preprint arXiv:2301.13688.
28. Jiao Sun, Yufei Tian, Wangchunshu Zhou, Nan Xu, Qian Hu, Rahul Gupta, John Wieting, Nanyun Peng, and Xuezhe Ma. 2023. Evaluating Large Language Models on Controlled Generation Tasks. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 3155–3168, Singapore… [find](https://scholar.google.com/scholar?q=Jiao+Sun%2C+Yufei+Tian%2C+Wangchunshu+Zhou%2C+Nan+Xu%2C+Qian+Hu%2C+Rahul+Gupta%2C+John+Wieting%2C+Nanyun+Peng%2C+and+Xuezhe+Ma%2E+2023%2E+Evaluating+Large+Language+Models+on+Controlled+Generation+Tasks%2E+In+Proceedings+of)
29. Renze Lou, Kai Zhang, Jian Xie, Yuxuan Sun, Janice Ahn, Hanzi Xu, Yu su, and Wenpeng Yin. 2024. MUFFIN: Curating multi-faceted instructions for improving instruction following. In The Twelfth International Conference on Learning Representations (ICLR). [find](https://scholar.google.com/scholar?q=Renze+Lou%2C+Kai+Zhang%2C+Jian+Xie%2C+Yuxuan+Sun%2C+Janice+Ahn%2C+Hanzi+Xu%2C+Yu+su%2C+and+Wenpeng+Yin%2E+2024%2E+MUFFIN%3A+Curating+multi%2Dfaceted+instructions+for+improving+instruction+following%2E+In+The+Twelfth+Internat)
30. Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford Alpaca: An Instruction-following LLaMA model. Published: GitHub repository. [find](https://scholar.google.com/scholar?q=Rohan+Taori%2C+Ishaan+Gulrajani%2C+Tianyi+Zhang%2C+Yann+Dubois%2C+Xuechen+Li%2C+Carlos+Guestrin%2C+Percy+Liang%2C+and+Tatsunori+B%2E+Hashimoto%2E+2023%2E+Stanford+Alpaca%3A+An+Instruction%2Dfollowing+LLaMA+model%2E+Published%3A+)
31. Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. 2022. Cross-Task Generalization via Natural Language Crowdsourcing Instructions. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3470–3487, Dublin, Irelan… [find](https://scholar.google.com/scholar?q=Swaroop+Mishra%2C+Daniel+Khashabi%2C+Chitta+Baral%2C+and+Hannaneh+Hajishirzi%2E+2022%2E+Cross%2DTask+Generalization+via+Natural+Language+Crowdsourcing+Instructions%2E+In+Proceedings+of+the+60th+Annual+Meeting+of+th)
32. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and others. 2023. [Llama 2: Open foundation and fine-tuned chat models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/). arXiv preprint arXiv:2307.09288.
33. OpenAI. 2022. Chatgpt: Optimizing language models for dialogue. OpenAI. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2022%2E+Chatgpt%3A+Optimizing+language+models+for+dialogue%2E+OpenAI%2E)
34. OpenAI. 2023. [GPT-4 Technical Report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). abs/2303.08774. ArXiv: 2303.08774.
35. CoRR, Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Shengyi Huang, Kashif Rasul, Alexander M. Rush, and Thomas Wolf. 2023a. The alignment handbook. [https://github.com/](https://github.com/) huggingface/alignment-handbook. [↗](https://github.com/)
36. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nat… [find](https://scholar.google.com/scholar?q=Long+Ouyang%2C+Jeffrey+Wu%2C+Xu+Jiang%2C+Diogo+Almeida%2C+Carroll+Wainwright%2C+Pamela+Mishkin%2C+Chong+Zhang%2C+Sandhini+Agarwal%2C+Katarina+Slama%2C+Alex+Ray%2C+and+Lewis+Tunstall%2C+Edward+Beeching%2C+Nathan+Lambert%2C+Nazn)
37. Hao Zhao, Maksym Andriushchenko, Francesco Croce, and Nicolas Flammarion. 2024. Long is more for alignment: A simple but tough-to-beat baseline for instruction fine-tuning. [find](https://scholar.google.com/scholar?q=Hao+Zhao%2C+Maksym+Andriushchenko%2C+Francesco+Croce%2C+and+Nicolas+Flammarion%2E+2024%2E+Long+is+more+for+alignment%3A+A+simple+but+tough%2Dto%2Dbeat+baseline+for+instruction+fine%2Dtuning%2E)
38. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. [Judging LLM-as-a-judge with MT-Bench and Chatbot Arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). In NeurIPS Datasets and Benchmarks Track. _eprint: 2306.05…
39. Guan Wang, Sijie Cheng, Xianyuan Zhan, Xiangang Li, Sen Song, and Yang Liu. 2023. [OpenChat: Advancing Open-source Language Models with Mixed-Quality Data](https://md.research.elamur.ai/free/en/papers/wang-2023-openchat-advancing-open-source-language/). arXiv preprint arXiv:2309.11235.
40. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2022. Self-Instruct: Aligning Language Model with Self Generated Instructions. arXiv preprint arXiv:2212.10560. [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Yeganeh+Kordi%2C+Swaroop+Mishra%2C+Alisa+Liu%2C+Noah+A+Smith%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+2022%2E+Self%2DInstruct%3A+Aligning+Language+Model+with+Self+Generated+Instructions%2E+arXiv+pre)
41. Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, and others. 2023a. Lima: Less is more for alignment. arXiv preprint arXiv:2305.11206. [find](https://scholar.google.com/scholar?q=Chunting+Zhou%2C+Pengfei+Liu%2C+Puxin+Xu%2C+Srini+Iyer%2C+Jiao+Sun%2C+Yuning+Mao%2C+Xuezhe+Ma%2C+Avia+Efrat%2C+Ping+Yu%2C+Lili+Yu%2C+and+others%2E+2023a%2E+Lima%3A+Less+is+more+for+alignment%2E+arXiv+preprint+arXiv%3A2305%2E11206%2E)
42. Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2021. [Finetuned language models are zero-shot learners](https://md.research.elamur.ai/free/en/papers/wei-2021-finetuned-language-models-zero-shot/). arXiv preprint arXiv:2109.01652.
43. Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. 2023b. [Instruction-Following Evaluation for Large Language Models](https://md.research.elamur.ai/free/en/papers/zhou-2023-instruction-following-evaluation-large-language/). arXiv preprint arXiv:2311.07911.
44. Tianwen Wei, Liang Zhao, Lichang Zhang, Bo Zhu, Lijie Wang, Haihua Yang, Biye Li, Cheng Cheng, Weiwei Lü, Rui Hu, et al. 2023. [Skywork: A more open bilingual foundation model](https://md.research.elamur.ai/free/en/papers/wei-2023-skywork-more-open-bilingual/). arXiv preprint arXiv:2310.19341.
45. Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. 2024. Wizardlm: Empowering large language models to follow complex instructions. In The Twelfth International Conference on Learning Representations (ICLR). [find](https://scholar.google.com/scholar?q=Can+Xu%2C+Qingfeng+Sun%2C+Kai+Zheng%2C+Xiubo+Geng%2C+Pu+Zhao%2C+Jiazhan+Feng%2C+Chongyang+Tao%2C+and+Daxin+Jiang%2E+2024%2E+Wizardlm%3A+Empowering+large+language+models+to+follow+complex+instructions%2E+In+The+Twelfth+Inte)
46. Shuo Yang, Wei-Lin Chiang, Lianmin Zheng, Joseph E. Gonzalez, and Ion Stoica. 2023. [Rethinking benchmark and contamination for language models with rephrased samples](https://md.research.elamur.ai/free/en/papers/yang-2023-rethinking-benchmark-contamination-language-models/). arXiv preprint arXiv:2311.04850.
47. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. HellaSwag: Can a Machine Really Finish Your Sentence? In The Annual Meeting of the Association for Computational Linguistics (ACL). [find](https://scholar.google.com/scholar?q=Rowan+Zellers%2C+Ari+Holtzman%2C+Yonatan+Bisk%2C+Ali+Farhadi%2C+and+Yejin+Choi%2E+2019%2E+HellaSwag%3A+Can+a+Machine+Really+Finish+Your+Sentence%3F+In+The+Annual+Meeting+of+the+Association+for+Computational+Linguisti)
48. Hanqing Zhang, Haolin Song, Shaoyu Li, Ming Zhou, and Dawei Song. 2023a. A survey of controllable text generation using transformer-based pre-trained language models. ACM Computing Surveys. [find](https://scholar.google.com/scholar?q=Hanqing+Zhang%2C+Haolin+Song%2C+Shaoyu+Li%2C+Ming+Zhou%2C+and+Dawei+Song%2E+2023a%2E+A+survey+of+controllable+text+generation+using+transformer%2Dbased+pre%2Dtrained+language+models%2E+ACM+Computing+Surveys%2E)
49. Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Fei Wu, and Guoyin Wang. 2023b. [Instruction tuning for large language models: A survey](https://md.research.elamur.ai/free/en/papers/zhang-2023-instruction-tuning-large-language-models/). arXiv preprint arXiv:2308.10792.
50. Yizhe Zhang, Guoyin Wang, Chunyuan Li, Zhe Gan, Chris Brockett, and William B Dolan. 2020. Pointer: Constrained Text Generation via Insertion-based Generative Pre-training. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8649–8670. [find](https://scholar.google.com/scholar?q=Yizhe+Zhang%2C+Guoyin+Wang%2C+Chunyuan+Li%2C+Zhe+Gan%2C+Chris+Brockett%2C+and+William+B+Dolan%2E+2020%2E+Pointer%3A+Constrained+Text+Generation+via+Insertion%2Dbased+Generative+Pre%2Dtraining%2E+In+Proceedings+of+the+2020+)

*Updated 30 October 2026*