---
title: "Camels in a Changing Climate — Ivison 2023"
description: "After the first TÜLU release, open instruction tuning moved quickly."
authors: ["Hamish Ivison", "Yizhong Wang", "Valentina Pyatkin", "Nathan Lambert", "Matthew Peters", "Pradeep Dasigi", "Joel Jang", "David Wadden", "Noah A. Smith", "Iz Beltagy", "Hannaneh Hajishirzi"]
year: 2023
venue: "arXiv:2311.10702"
source: "https://arxiv.org/abs/2311.10702"
doi: "10.48550/arxiv.2311.10702"
arxiv: "2311.10702"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ivison-2023-camels-changing-climate-enhancing-lm/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ivison-2023-camels-changing-climate-enhancing-lm/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/2/) / Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2

# Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2

**Authors:** [Hamish Ivison](https://md.research.elamur.ai/free/en/authors/hamish-ivison/), [Yizhong Wang](https://md.research.elamur.ai/free/en/authors/yizhong-wang/), [Valentina Pyatkin](https://md.research.elamur.ai/free/en/authors/valentina-pyatkin/), [Nathan Lambert](https://md.research.elamur.ai/free/en/authors/nathan-lambert/), [Matthew Peters](https://md.research.elamur.ai/free/en/authors/matthew-peters/), [Pradeep Dasigi](https://md.research.elamur.ai/free/en/authors/pradeep-dasigi/), [Joel Jang](https://md.research.elamur.ai/free/en/authors/joel-jang/), [David Wadden](https://md.research.elamur.ai/free/en/authors/david-wadden/), [Noah A. Smith](https://md.research.elamur.ai/free/en/authors/noah-a-smith/), [Iz Beltagy](https://md.research.elamur.ai/free/en/authors/iz-beltagy/), [Hannaneh Hajishirzi](https://md.research.elamur.ai/free/en/authors/hannaneh-hajishirzi/) · **Year:** 2023 · **Venue:** arXiv:2311.10702 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2311.10702) · [PDF](https://arxiv.org/pdf/2311.10702) · [DOI 10.48550/arxiv.2311.10702](https://doi.org/10.48550/arxiv.2311.10702) · [arXiv 2311.10702](https://arxiv.org/abs/2311.10702) · [OpenAlex](https://openalex.org/W4388890574)

**Citations:** 8 citations (openalex-snapshot · 2026-10-14) · [191](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
After the first TÜLU release, open instruction tuning moved quickly. Stronger base models, better distilled datasets and simpler adaptation methods all appeared. This work from AI2 and the University of Washington tests these advances one by one and keeps the useful ones in TÜLU 2, a suite of LLAMA-2 models at 7B, 13B and 70B.

The authors built TÜLU-V2-mix, a smaller but higher-quality instruction mixture. It adds sources such as LIMA, WizardLM Evol-Instruct, Open-Orca and science-literature tasks, and drops Dolly. They trained on it with an 8,192-token context. They then applied DPO on UltraFeedback, following the Zephyr-Beta recipe. They also compared QLoRA with full finetuning and trained CODE TÜLU 2 on top of CODE LLAMA.

Among the open models they evaluated, theirs scored highest, and on several benchmarks they matched or beat GPT-3.5-turbo-0301. TÜLU 2+DPO 70B was the first stable DPO run at the 70B scale. All checkpoints, data and code are public, so the work serves as a reproducible baseline for post-training research.

## Key points
- **TÜLU-V2-mix** has 326,154 examples, down from 490,445 in V1, yet scores about 8% higher on average than the old mix. It is released as `allenai/tulu-v2-sft-mixture` on Hugging Face, and the training and evaluation code is in the `allenai/open-instruct` GitHub repository.
- **DPO setup:** UltraFeedback, 3 epochs, a very low learning rate of 5×10⁻⁷. It raises AlpacaEval scores by about 13% on average across model sizes, with almost no change on MMLU or coding benchmarks. Among open-weight models, the 70B DPO model leads on MT-Bench.
- **QLoRA** does worse than full finetuning, especially on long-form generation (about a 20% gap on AlpacaEval). The overall gap shrinks with model size, from about 10% to about 3%.
- **CODE TÜLU 2** (7B/13B/34B) improves Codex-Eval by about 70% over TÜLU 2 and beats both CODE LLAMA and CODE LLAMA-Instruct on average. The 7B version matches TÜLU 2+DPO 70B on Codex-Eval, but AlpacaEval drops by about 20%.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Other](https://md.research.elamur.ai/free/en/shelves/other/)

## Cited by

Cited by

191 papers of our library cite it (of the 140673 whose bibliographies we read); 11 of them have a page here, 180 not yet.

- [Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models](https://md.research.elamur.ai/free/en/papers/chen-2025-towards-reasoning-era-survey-long/)
- [WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs](https://md.research.elamur.ai/free/en/papers/han-2024-wildguard-open-one-stop-moderation/)
- [Tulu 3: Pushing Frontiers in Open Language Model Post-Training](https://md.research.elamur.ai/free/en/papers/lambert-2024-tulu-3-pushing-frontiers-open/)
- [From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipeline](https://md.research.elamur.ai/free/en/papers/li-2024-crowdsourced-data-high-quality-benchmarks/)
- [DataComp-LM: In search of the next generation of training sets for language models](https://md.research.elamur.ai/free/en/papers/li-2024-datacomp-lm-search-next-generation/)
- [SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models](https://md.research.elamur.ai/free/en/papers/li-2024-salad-bench-hierarchical-comprehensive-safety/)
- [Skywork-Reward: Bag of Tricks for Reward Modeling in LLMs](https://md.research.elamur.ai/free/en/papers/liu-2024-skywork-reward-bag-tricks/)
- [Generative Representational Instruction Tuning](https://md.research.elamur.ai/free/en/papers/muennighoff-2024-generative-representational-instruction-tuning/)
- [Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing](https://md.research.elamur.ai/free/en/papers/xu-2024-magpie-alignment-data-synthesis-scratch/)
- [MAmmoTH2: Scaling Instructions from the Web](https://md.research.elamur.ai/free/en/papers/yue-2024-mammoth2-scaling-instructions-web/)
- [JudgeLM: Fine-tuned Large Language Models are Scalable Judges](https://md.research.elamur.ai/free/en/papers/zhu-2023-judgelm-fine-tuned-large-language/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{ivison2023camels,
  title = {{Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2}},
  author = {Ivison, Hamish and Wang, Yizhong and Pyatkin, Valentina and Lambert, Nathan and Peters, Matthew and Dasigi, Pradeep and Jang, Joel and Wadden, David and Smith, Noah A. and Beltagy, Iz and Hajishirzi, Hannaneh},
  year = {2023},
  doi = {10.48550/arxiv.2311.10702},
  eprint = {2311.10702},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2311.10702}
}
```

- [← previous CamelParser2.0: A State-of-the-Art Dependency Parser for Arabic](https://md.research.elamur.ai/free/en/papers/ud-2023-elshabrawy-camelparser2-0-state-art-dependency/)
- [next → Can AI-Generated Text be Reliably Detected?](https://md.research.elamur.ai/free/en/papers/sadasivan-2023-ai-generated-text-reliably-detected/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/2/)

## References

46 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. R. Anil, A. M. Dai, O. Firat, M. Johnson, D. Lepikhin, A. Passos, S. Shakeri, E. Taropa, P. Bailey, Z. Chen, et al. Palm 2 technical report. arXiv preprint arXiv:2305.10403, 2023. [find](https://scholar.google.com/scholar?q=R%2E+Anil%2C+A%2E+M%2E+Dai%2C+O%2E+Firat%2C+M%2E+Johnson%2C+D%2E+Lepikhin%2C+A%2E+Passos%2C+S%2E+Shakeri%2C+E%2E+Taropa%2C+P%2E+Bailey%2C+Z%2E+Chen%2C+et+al%2E+Palm+2+technical+report%2E+arXiv+preprint+arXiv%3A2305%2E10403%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/anil-2023-palm-2-technical-report/)
2. Y. Bai, A. Jones, K. Ndousse, A. Askell, A. Chen, N. DasSarma, D. Drain, S. Fort, D. Ganguli, T. Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022. [find](https://scholar.google.com/scholar?q=Y%2E+Bai%2C+A%2E+Jones%2C+K%2E+Ndousse%2C+A%2E+Askell%2C+A%2E+Chen%2C+N%2E+DasSarma%2C+D%2E+Drain%2C+S%2E+Fort%2C+D%2E+Ganguli%2C+T%2E+Henighan%2C+et+al%2E+Training+a+helpful+and+harmless+assistant+with+reinforcement+learning+from+human+feedb)
3. I. Cachola, K. Lo, A. Cohan, and D. Weld. TLDR: Extreme summarization of scientific documents. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 4766–4777, Online, Nov. 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.findings-emnlp. 428. URL https… [find](https://scholar.google.com/scholar?q=I%2E+Cachola%2C+K%2E+Lo%2C+A%2E+Cohan%2C+and+D%2E+Weld%2E+TLDR%3A+Extreme+summarization+of+scientific+documents%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+2020%2C+pages+4766%E2%80%934777%2C+Online%2C+Nov%2E+2) [our card](https://md.research.elamur.ai/free/en/papers/cachola-2020-tldr-extreme-summarization-scientific-documents/)
4. S. Chaudhary. Code alpaca: An instruction-following llama model for code generation. GitHub repository, 2023. URL [https://github.com/sahil280114/codealpaca](https://github.com/sahil280114/codealpaca). [↗](https://github.com/sahil280114/codealpaca)
5. M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021. [find](https://scholar.google.com/scholar?q=M%2E+Chen%2C+J%2E+Tworek%2C+H%2E+Jun%2C+Q%2E+Yuan%2C+H%2E+P%2E+d%2E+O%2E+Pinto%2C+J%2E+Kaplan%2C+H%2E+Edwards%2C+Y%2E+Burda%2C+N%2E+Joseph%2C+G%2E+Brockman%2C+et+al%2E+Evaluating+large+language+models+trained+on+code%2E+arXiv+preprint+arXiv%3A2107%2E0337) [our card](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
6. W.-L. Chiang, Z. Li, Z. Lin, Y. Sheng, Z. Wu, H. Zhang, L. Zheng, S. Zhuang, Y. Zhuang, J. E. Gonzalez, I. Stoica, and E. P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. Blog post, March 2023. URL [https://lmsys.org/blog/2023-03-30-vicuna/](https://lmsys.org/blog/2023-03-30-vicuna/). [↗](https://lmsys.org/blog/2023-03-30-vicuna/)
7. H. W. Chung, L. Hou, S. Longpre, B. Zoph, Y. Tay, W. Fedus, E. Li, X. Wang, M. Dehghani, S. Brahma, et al. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416, 2022. [find](https://scholar.google.com/scholar?q=H%2E+W%2E+Chung%2C+L%2E+Hou%2C+S%2E+Longpre%2C+B%2E+Zoph%2C+Y%2E+Tay%2C+W%2E+Fedus%2C+E%2E+Li%2C+X%2E+Wang%2C+M%2E+Dehghani%2C+S%2E+Brahma%2C+et+al%2E+Scaling+instruction%2Dfinetuned+language+models%2E+arXiv+preprint+arXiv%3A2210%2E11416%2C+2022%2E)
8. G. Cui, L. Yuan, N. Ding, G. Yao, W. Zhu, Y. Ni, G. Xie, Z. Liu, and M. Sun. Ultrafeedback: Boosting language models with high-quality feedback. arXiv preprint arXiv:2310.01377, 2023. [find](https://scholar.google.com/scholar?q=G%2E+Cui%2C+L%2E+Yuan%2C+N%2E+Ding%2C+G%2E+Yao%2C+W%2E+Zhu%2C+Y%2E+Ni%2C+G%2E+Xie%2C+Z%2E+Liu%2C+and+M%2E+Sun%2E+Ultrafeedback%3A+Boosting+language+models+with+high%2Dquality+feedback%2E+arXiv+preprint+arXiv%3A2310%2E01377%2C+2023%2E)
9. P. Dasigi, K. Lo, I. Beltagy, A. Cohan, N. A. Smith, and M. Gardner. A dataset of information-seeking questions and answers anchored in research papers. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… [find](https://scholar.google.com/scholar?q=P%2E+Dasigi%2C+K%2E+Lo%2C+I%2E+Beltagy%2C+A%2E+Cohan%2C+N%2E+A%2E+Smith%2C+and+M%2E+Gardner%2E+A+dataset+of+information%2Dseeking+questions+and+answers+anchored+in+research+papers%2E+In+Proceedings+of+the+2021+Conference+of+the+No) [our card](https://md.research.elamur.ai/free/en/papers/dasigi-2021-dataset-information-seeking-questions-answers/)
10. Databricks. Free dolly: Introducing the world’s first truly open instruction-tuned llm. Blog post, 2023. URL [https://www.databricks.com/blog/2023/04/12/](https://www.databricks.com/blog/2023/04/12/) dolly-first-open-commercially-viable-instruction-tuned-llm. [↗](https://www.databricks.com/blog/2023/04/12/)
11. T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer. Qlora: Efficient finetuning of quantized llms. arXiv preprint arXiv:2305.14314, 2023. [find](https://scholar.google.com/scholar?q=T%2E+Dettmers%2C+A%2E+Pagnoni%2C+A%2E+Holtzman%2C+and+L%2E+Zettlemoyer%2E+Qlora%3A+Efficient+finetuning+of+quantized+llms%2E+arXiv+preprint+arXiv%3A2305%2E14314%2C+2023%2E)
12. N. Ding, Y. Chen, B. Xu, S. Hu, Y. Qin, Z. Liu, M. Sun, and B. Zhou. Ultrachat: A large-scale auto-generated multi-round dialogue data. GitHub Repository, 2023. URL [https://github.com/thunlp/ultrachat](https://github.com/thunlp/ultrachat). [↗](https://github.com/thunlp/ultrachat)
13. Y. Dubois, X. Li, R. Taori, T. Zhang, I. Gulrajani, J. Ba, C. Guestrin, P. Liang, and T. B. Hashimoto. Alpacafarm: A simulation framework for methods that learn from human feedback. arXiv preprint arXiv:2305.14387, 2023. [find](https://scholar.google.com/scholar?q=Y%2E+Dubois%2C+X%2E+Li%2C+R%2E+Taori%2C+T%2E+Zhang%2C+I%2E+Gulrajani%2C+J%2E+Ba%2C+C%2E+Guestrin%2C+P%2E+Liang%2C+and+T%2E+B%2E+Hashimoto%2E+Alpacafarm%3A+A+simulation+framework+for+methods+that+learn+from+human+feedback%2E+arXiv+preprint+arX) [our card](https://md.research.elamur.ai/free/en/papers/dubois-2023-alpacafarm-simulation-framework-methods-that/)
14. X. Geng. Easylm: A simple and scalable training framework for large language models, 2023. URL [https://github.com/young-geng/EasyLM](https://github.com/young-geng/EasyLM). [↗](https://github.com/young-geng/EasyLM)
15. C. Gulcehre, T. L. Paine, S. Srinivasan, K. Konyushkova, L. Weerts, A. Sharma, A. Siddhant, A. Ahern, M. Wang, C. Gu, et al. Reinforced self-training (rest) for language modeling. arXiv preprint arXiv:2308.08998, 2023. [find](https://scholar.google.com/scholar?q=C%2E+Gulcehre%2C+T%2E+L%2E+Paine%2C+S%2E+Srinivasan%2C+K%2E+Konyushkova%2C+L%2E+Weerts%2C+A%2E+Sharma%2C+A%2E+Siddhant%2C+A%2E+Ahern%2C+M%2E+Wang%2C+C%2E+Gu%2C+et+al%2E+Reinforced+self%2Dtraining+%28rest%29+for+language+modeling%2E+arXiv+preprint+arXiv) [our card](https://md.research.elamur.ai/free/en/papers/gulcehre-2023-reinforced-self-training-rest-language/)
16. T. Hartvigsen, S. Gabriel, H. Palangi, M. Sap, D. Ray, and E. Kamar. TOXIGEN: Controlling Language Models to Generate Implied and Adversarial Toxicity. In ACL, 2022. URL https://arxiv.org/abs/2203.09509. [↗](https://arxiv.org/abs/2203.09509) [our card](https://md.research.elamur.ai/free/en/papers/hartvigsen-2022-toxigen-large-scale-machine-generated/)
17. A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. d. l. Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023. [find](https://scholar.google.com/scholar?q=A%2E+Q%2E+Jiang%2C+A%2E+Sablayrolles%2C+A%2E+Mensch%2C+C%2E+Bamford%2C+D%2E+S%2E+Chaplot%2C+D%2E+d%2E+l%2E+Casas%2C+F%2E+Bressand%2C+G%2E+Lengyel%2C+G%2E+Lample%2C+L%2E+Saulnier%2C+et+al%2E+Mistral+7b%2E+arXiv+preprint+arXiv%3A2310%2E06825%2C+2023%2E)
18. A. Köpf, Y. Kilcher, D. von Rütte, S. Anagnostidis, Z.-R. Tam, K. Stevens, A. Barhoum, N. M. Duc, O. Stanley, R. Nagyfi, et al. Openassistant conversations–democratizing large language model alignment. arXiv preprint arXiv:2304.07327, 2023. [find](https://scholar.google.com/scholar?q=A%2E+K%C3%B6pf%2C+Y%2E+Kilcher%2C+D%2E+von+R%C3%BCtte%2C+S%2E+Anagnostidis%2C+Z%2E%2DR%2E+Tam%2C+K%2E+Stevens%2C+A%2E+Barhoum%2C+N%2E+M%2E+Duc%2C+O%2E+Stanley%2C+R%2E+Nagyfi%2C+et+al%2E+Openassistant+conversations%E2%80%93democratizing+large+language+model+alignment) [our card](https://md.research.elamur.ai/free/en/papers/kopf-2023-openassistant-conversations-democratizing-large-language/)
19. E. Lehman, J. DeYoung, R. Barzilay, and B. C. Wallace. Inferring which medical treatments work from reports of clinical trials. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short… [find](https://scholar.google.com/scholar?q=E%2E+Lehman%2C+J%2E+DeYoung%2C+R%2E+Barzilay%2C+and+B%2E+C%2E+Wallace%2E+Inferring+which+medical+treatments+work+from+reports+of+clinical+trials%2E+In+Proceedings+of+the+2019+Conference+of+the+North+American+Chapter+of+t)
20. B. Lester, R. Al-Rfou, and N. Constant. The power of scale for parameter-efficient prompt tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3045–3059, Online and Punta Cana, Dominican Republic, Nov. 2021. Association for Computational Linguistic… [find](https://scholar.google.com/scholar?q=B%2E+Lester%2C+R%2E+Al%2DRfou%2C+and+N%2E+Constant%2E+The+power+of+scale+for+parameter%2Defficient+prompt+tuning%2E+In+Proceedings+of+the+2021+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+3045%E2%80%93)
21. X. Li, T. Zhang, Y. Dubois, R. Taori, I. Gulrajani, C. Guestrin, P. Liang, and T. B. Hashimoto. Alpacaeval: An automatic evaluator of instruction-following models. Github repository, 2023. URL [https://github.com/tatsu-lab/alpaca_eval](https://github.com/tatsu-lab/alpaca_eval). [↗](https://github.com/tatsu-lab/alpaca_eval)
22. W. Lian, B. Goodson, E. Pentland, A. Cook, C. Vong, and "Teknium". Openorca: An open dataset of gpt augmented flan reasoning traces. https://https://huggingface.co/Open-Orca/ OpenOrca, 2023. [find](https://scholar.google.com/scholar?q=W%2E+Lian%2C+B%2E+Goodson%2C+E%2E+Pentland%2C+A%2E+Cook%2C+C%2E+Vong%2C+and+%22Teknium%22%2E+Openorca%3A+An+open+dataset+of+gpt+augmented+flan+reasoning+traces%2E+https%3A%2F%2Fhttps%3A%2F%2Fhuggingface%2Eco%2FOpen%2DOrca%2F+OpenOrca%2C+2023%2E)
23. S. Lin, J. Hilton, and O. Evans. Truthfulqa: Measuring how models mimic human falsehoods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3214–3252, 2022. [find](https://scholar.google.com/scholar?q=S%2E+Lin%2C+J%2E+Hilton%2C+and+O%2E+Evans%2E+Truthfulqa%3A+Measuring+how+models+mimic+human+falsehoods%2E+In+Proceedings+of+the+60th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volume+1%3A+Long+Pap) [our card](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/)
24. Y. Luan, L. He, M. Ostendorf, and H. Hajishirzi. Multi-task identification of entities, relations, and coreference for scientific knowledge graph construction. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3219–3232, Brussels, Belgium, Oct.-Nov. 201… [find](https://scholar.google.com/scholar?q=Y%2E+Luan%2C+L%2E+He%2C+M%2E+Ostendorf%2C+and+H%2E+Hajishirzi%2E+Multi%2Dtask+identification+of+entities%2C+relations%2C+and+coreference+for+scientific+knowledge+graph+construction%2E+In+Proceedings+of+the+2018+Conference+on) [our card](https://md.research.elamur.ai/free/en/papers/luan-2018-multi-task-identification-entities-relations/)
25. MosaicML. Introducing mpt-7b: A new standard for open-source, commercially usable llms. Blog post, 2023. URL [https://www.mosaicml.com/blog/mpt-7b](https://www.mosaicml.com/blog/mpt-7b). [↗](https://www.mosaicml.com/blog/mpt-7b)
26. S. Mukherjee, A. Mitra, G. Jawahar, S. Agarwal, H. Palangi, and A. Awadallah. Orca: Progressive learning from complex explanation traces of gpt-4, 2023. [find](https://scholar.google.com/scholar?q=S%2E+Mukherjee%2C+A%2E+Mitra%2C+G%2E+Jawahar%2C+S%2E+Agarwal%2C+H%2E+Palangi%2C+and+A%2E+Awadallah%2E+Orca%3A+Progressive+learning+from+complex+explanation+traces+of+gpt%2D4%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/mukherjee-2023-orca-progressive-learning-complex-explanation/)
27. L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al. Training Language Models to Follow Instructions with Human Feedback. In Advances in Neural Information Processing Systems (NeurIPS), 2022. [find](https://scholar.google.com/scholar?q=L%2E+Ouyang%2C+J%2E+Wu%2C+X%2E+Jiang%2C+D%2E+Almeida%2C+C%2E+L%2E+Wainwright%2C+P%2E+Mishkin%2C+C%2E+Zhang%2C+S%2E+Agarwal%2C+K%2E+Slama%2C+A%2E+Ray%2C+et+al%2E+Training+Language+Models+to+Follow+Instructions+with+Human+Feedback%2E+In+Advances+in)
28. B. Peng, C. Li, P. He, M. Galley, and J. Gao. Instruction tuning with gpt-4. arXiv preprint arXiv:2304.03277, 2023. [find](https://scholar.google.com/scholar?q=B%2E+Peng%2C+C%2E+Li%2C+P%2E+He%2C+M%2E+Galley%2C+and+J%2E+Gao%2E+Instruction+tuning+with+gpt%2D4%2E+arXiv+preprint+arXiv%3A2304%2E03277%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/peng-2023-instruction-tuning-gpt-4/)
29. R. Rafailov, A. Sharma, E. Mitchell, S. Ermon, C. D. Manning, and C. Finn. Direct preference optimization: Your language model is secretly a reward model. arXiv preprint arXiv:2305.18290, 2023. [find](https://scholar.google.com/scholar?q=R%2E+Rafailov%2C+A%2E+Sharma%2C+E%2E+Mitchell%2C+S%2E+Ermon%2C+C%2E+D%2E+Manning%2C+and+C%2E+Finn%2E+Direct+preference+optimization%3A+Your+language+model+is+secretly+a+reward+model%2E+arXiv+preprint+arXiv%3A2305%2E18290%2C+2023%2E)
30. B. Roziere, J. Gehring, F. Gloeckle, S. Sootla, I. Gat, X. E. Tan, Y. Adi, J. Liu, T. Remez, J. Rapin, et al. Code llama: Open foundation models for code. arXiv preprint arXiv:2308.12950, 2023. [find](https://scholar.google.com/scholar?q=B%2E+Roziere%2C+J%2E+Gehring%2C+F%2E+Gloeckle%2C+S%2E+Sootla%2C+I%2E+Gat%2C+X%2E+E%2E+Tan%2C+Y%2E+Adi%2C+J%2E+Liu%2C+T%2E+Remez%2C+J%2E+Rapin%2C+et+al%2E+Code+llama%3A+Open+foundation+models+for+code%2E+arXiv+preprint+arXiv%3A2308%2E12950%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/roziere-2024-code-llama-open-foundation-models/)
31. M. Santacroce, Y. Lu, H. Yu, Y. Li, and Y. Shen. Efficient rlhf: Reducing the memory usage of ppo, 2023. [find](https://scholar.google.com/scholar?q=M%2E+Santacroce%2C+Y%2E+Lu%2C+H%2E+Yu%2C+Y%2E+Li%2C+and+Y%2E+Shen%2E+Efficient+rlhf%3A+Reducing+the+memory+usage+of+ppo%2C+2023%2E)
32. P. Singhal, T. Goyal, J. Xu, and G. Durrett. A long way to go: Investigating length correlations in rlhf. arXiv preprint arXiv:2310.03716, 2023. [find](https://scholar.google.com/scholar?q=P%2E+Singhal%2C+T%2E+Goyal%2C+J%2E+Xu%2C+and+G%2E+Durrett%2E+A+long+way+to+go%3A+Investigating+length+correlations+in+rlhf%2E+arXiv+preprint+arXiv%3A2310%2E03716%2C+2023%2E)
33. C. Snell, I. Kostrikov, Y. Su, M. Yang, and S. Levine. Offline rl for natural language generation with implicit language q learning. arXiv preprint arXiv:2206.11871, 2022. — [arXiv:2206.11871](https://arxiv.org/abs/2206.11871) [find](https://scholar.google.com/scholar?q=C%2E+Snell%2C+I%2E+Kostrikov%2C+Y%2E+Su%2C+M%2E+Yang%2C+and+S%2E+Levine%2E+Offline+rl+for+natural+language+generation+with+implicit+language+q+learning%2E+arXiv+preprint+arXiv%3A2206%2E11871%2C+2022%2E+%E2%80%94+%5BarXiv%3A2206%2E11871%5D%28https%3A%2F)
34. S. Sun, D. Gupta, and M. Iyyer. Exploring the impact of low-rank adaptation on the performance, efficiency, and regularization of rlhf, 2023. [find](https://scholar.google.com/scholar?q=S%2E+Sun%2C+D%2E+Gupta%2C+and+M%2E+Iyyer%2E+Exploring+the+impact+of+low%2Drank+adaptation+on+the+performance%2C+efficiency%2C+and+regularization+of+rlhf%2C+2023%2E)
35. M. Suzgun, N. Scales, N. Schärli, S. Gehrmann, Y. Tay, H. W. Chung, A. Chowdhery, Q. V. Le, E. H. Chi, D. Zhou, et al. Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261, 2022. [find](https://scholar.google.com/scholar?q=M%2E+Suzgun%2C+N%2E+Scales%2C+N%2E+Sch%C3%A4rli%2C+S%2E+Gehrmann%2C+Y%2E+Tay%2C+H%2E+W%2E+Chung%2C+A%2E+Chowdhery%2C+Q%2E+V%2E+Le%2C+E%2E+H%2E+Chi%2C+D%2E+Zhou%2C+et+al%2E+Challenging+big%2Dbench+tasks+and+whether+chain%2Dof%2Dthought+can+solve+them%2E+arXiv+pr) [our card](https://md.research.elamur.ai/free/en/papers/suzgun-2023-challenging-big-bench-tasks-whether/)
36. H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023a. [find](https://scholar.google.com/scholar?q=H%2E+Touvron%2C+T%2E+Lavril%2C+G%2E+Izacard%2C+X%2E+Martinet%2C+M%2E%2DA%2E+Lachaux%2C+T%2E+Lacroix%2C+B%2E+Rozi%C3%A8re%2C+N%2E+Goyal%2C+E%2E+Hambro%2C+F%2E+Azhar%2C+et+al%2E+Llama%3A+Open+and+efficient+foundation+language+models%2E+arXiv+preprint+arXiv%3A)
37. H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023b. [find](https://scholar.google.com/scholar?q=H%2E+Touvron%2C+L%2E+Martin%2C+K%2E+Stone%2C+P%2E+Albert%2C+A%2E+Almahairi%2C+Y%2E+Babaei%2C+N%2E+Bashlykov%2C+S%2E+Batra%2C+P%2E+Bhargava%2C+S%2E+Bhosale%2C+et+al%2E+Llama+2%3A+Open+foundation+and+fine%2Dtuned+chat+models%2E+arXiv+preprint+arXiv%3A2)
38. L. Tunstall, E. Beeching, N. Lambert, N. Rajani, K. Rasul, Y. Belkada, S. Huang, L. von Werra, C. Fourrier, N. Habib, et al. Zephyr: Direct distillation of lm alignment. arXiv preprint arXiv:2310.16944, 2023. [find](https://scholar.google.com/scholar?q=L%2E+Tunstall%2C+E%2E+Beeching%2C+N%2E+Lambert%2C+N%2E+Rajani%2C+K%2E+Rasul%2C+Y%2E+Belkada%2C+S%2E+Huang%2C+L%2E+von+Werra%2C+C%2E+Fourrier%2C+N%2E+Habib%2C+et+al%2E+Zephyr%3A+Direct+distillation+of+lm+alignment%2E+arXiv+preprint+arXiv%3A2310%2E1694)
39. D. Wadden, S. Lin, K. Lo, L. L. Wang, M. van Zuylen, A. Cohan, and H. Hajishirzi. Fact or fiction: Verifying scientific claims. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7534–7550, Online, Nov. 2020. Association for Computational Linguis… [find](https://scholar.google.com/scholar?q=D%2E+Wadden%2C+S%2E+Lin%2C+K%2E+Lo%2C+L%2E+L%2E+Wang%2C+M%2E+van+Zuylen%2C+A%2E+Cohan%2C+and+H%2E+Hajishirzi%2E+Fact+or+fiction%3A+Verifying+scientific+claims%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natural+La) [our card](https://md.research.elamur.ai/free/en/papers/wadden-2020-fact-or-fiction-verifying-scientific/)
40. G. Wang, S. Cheng, X. Zhan, X. Li, S. Song, and Y. Liu. Openchat: Advancing open-source language models with mixed-quality data. arXiv preprint arXiv:2309.11235, 2023a. [find](https://scholar.google.com/scholar?q=G%2E+Wang%2C+S%2E+Cheng%2C+X%2E+Zhan%2C+X%2E+Li%2C+S%2E+Song%2C+and+Y%2E+Liu%2E+Openchat%3A+Advancing+open%2Dsource+language+models+with+mixed%2Dquality+data%2E+arXiv+preprint+arXiv%3A2309%2E11235%2C+2023a%2E) [our card](https://md.research.elamur.ai/free/en/papers/wang-2023-openchat-advancing-open-source-language/)
41. Y. Wang, H. Ivison, P. Dasigi, J. Hessel, T. Khot, K. R. Chandu, D. Wadden, K. MacMillan, N. A. Smith, I. Beltagy, et al. How far can camels go? exploring the state of instruction tuning on open resources. arXiv preprint arXiv:2306.04751, 2023b. [find](https://scholar.google.com/scholar?q=Y%2E+Wang%2C+H%2E+Ivison%2C+P%2E+Dasigi%2C+J%2E+Hessel%2C+T%2E+Khot%2C+K%2E+R%2E+Chandu%2C+D%2E+Wadden%2C+K%2E+MacMillan%2C+N%2E+A%2E+Smith%2C+I%2E+Beltagy%2C+et+al%2E+How+far+can+camels+go%3F+exploring+the+state+of+instruction+tuning+on+open+resou) [our card](https://md.research.elamur.ai/free/en/papers/wang-2023-far-camels-go-exploring-state/)
42. J. Wei, X. Wang, D. Schuurmans, M. Bosma, E. Chi, Q. Le, and D. Zhou. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903, 2022. [find](https://scholar.google.com/scholar?q=J%2E+Wei%2C+X%2E+Wang%2C+D%2E+Schuurmans%2C+M%2E+Bosma%2C+E%2E+Chi%2C+Q%2E+Le%2C+and+D%2E+Zhou%2E+Chain+of+thought+prompting+elicits+reasoning+in+large+language+models%2E+arXiv+preprint+arXiv%3A2201%2E11903%2C+2022%2E) [our card](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/)
43. C. Xu, Q. Sun, K. Zheng, X. Geng, P. Zhao, J. Feng, C. Tao, and D. Jiang. Wizardlm: Empowering large language models to follow complex instructions. arXiv preprint arXiv:2304.12244, 2023. [find](https://scholar.google.com/scholar?q=C%2E+Xu%2C+Q%2E+Sun%2C+K%2E+Zheng%2C+X%2E+Geng%2C+P%2E+Zhao%2C+J%2E+Feng%2C+C%2E+Tao%2C+and+D%2E+Jiang%2E+Wizardlm%3A+Empowering+large+language+models+to+follow+complex+instructions%2E+arXiv+preprint+arXiv%3A2304%2E12244%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/xu-2023-wizardlm-empowering-large-pre-trained/)
44. Xwin-LM Team. Xwin-lm, 2023. URL [https://github.com/Xwin-LM/Xwin-LM](https://github.com/Xwin-LM/Xwin-LM). [↗](https://github.com/Xwin-LM/Xwin-LM)
45. L. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. P. Xing, H. Zhang, J. E. Gonzalez, and I. Stoica. Judging llm-as-a-judge with mt-bench and chatbot arena. In NeurIPS Datasets and Benchmarks Track, 2023. [find](https://scholar.google.com/scholar?q=L%2E+Zheng%2C+W%2E%2DL%2E+Chiang%2C+Y%2E+Sheng%2C+S%2E+Zhuang%2C+Z%2E+Wu%2C+Y%2E+Zhuang%2C+Z%2E+Lin%2C+Z%2E+Li%2C+D%2E+Li%2C+E%2E+P%2E+Xing%2C+H%2E+Zhang%2C+J%2E+E%2E+Gonzalez%2C+and+I%2E+Stoica%2E+Judging+llm%2Das%2Da%2Djudge+with+mt%2Dbench+and+chatbot+arena%2E+In+Neu)
46. C. Zhou, P. Liu, P. Xu, S. Iyer, J. Sun, Y. Mao, X. Ma, A. Efrat, P. Yu, L. Yu, et al. Lima: Less is more for alignment. arXiv preprint arXiv:2305.11206, 2023. [find](https://scholar.google.com/scholar?q=C%2E+Zhou%2C+P%2E+Liu%2C+P%2E+Xu%2C+S%2E+Iyer%2C+J%2E+Sun%2C+Y%2E+Mao%2C+X%2E+Ma%2C+A%2E+Efrat%2C+P%2E+Yu%2C+L%2E+Yu%2C+et+al%2E+Lima%3A+Less+is+more+for+alignment%2E+arXiv+preprint+arXiv%3A2305%2E11206%2C+2023%2E)