---
title: "Language Models can Self-Lengthen to Generate Long Texts"
description: "The Qwen Team proposes Self-Lengthen, an iterative scheme that needs only seed instructions and… 41 references, cited by 13 papers here."
authors: ["Shanghaoran Quan", "Tianyi Tang", "Bowen Yu", "An Yang", "Dayiheng Liu", "Bofei Gao", "Jianhong Tu", "Yichang Zhang", "Jingren Zhou", "Junyang Lin"]
year: 2024
venue: "arXiv:2410.23933"
source: "https://arxiv.org/abs/2410.23933"
doi: "10.48550/arxiv.2410.23933"
arxiv: "2410.23933"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/quan-2024-language-models-self-lengthen-generate/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/quan-2024-language-models-self-lengthen-generate/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/9/) / Language Models can Self-Lengthen to Generate Long Texts

# Language Models can Self-Lengthen to Generate Long Texts

**Authors:** [Shanghaoran Quan](https://md.research.elamur.ai/free/en/authors/shanghaoran-quan/), [Tianyi Tang](https://md.research.elamur.ai/free/en/authors/tianyi-tang/), [Bowen Yu](https://md.research.elamur.ai/free/en/authors/bowen-yu/), [An Yang](https://md.research.elamur.ai/free/en/authors/an-yang/), [Dayiheng Liu](https://md.research.elamur.ai/free/en/authors/dayiheng-liu/), [Bofei Gao](https://md.research.elamur.ai/free/en/authors/bofei-gao/), [Jianhong Tu](https://md.research.elamur.ai/free/en/authors/jianhong-tu/), [Yichang Zhang](https://md.research.elamur.ai/free/en/authors/yichang-zhang/), [Jingren Zhou](https://md.research.elamur.ai/free/en/authors/jingren-zhou/) and [Junyang Lin](https://md.research.elamur.ai/free/en/authors/junyang-lin/) · **Year:** 2024 · **Venue:** arXiv:2410.23933 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2410.23933) · [PDF](https://arxiv.org/pdf/2410.23933) · [DOI 10.48550/arxiv.2410.23933](https://doi.org/10.48550/arxiv.2410.23933) · [arXiv 2410.23933](https://arxiv.org/abs/2410.23933) · [OpenAlex](https://openalex.org/W4404347798)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [13](https://md.research.elamur.ai/free/en/papers/quan-2024-language-models-self-lengthen-generate/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Open models such as Qwen2 and LLaMA3 read long inputs well but rarely produce coherent answers longer than about 2,000 words. The authors trace this to post-training data that consists mostly of short responses. Changing the decoding settings makes outputs longer but noticeably lowers their quality. Existing fixes have drawbacks. One turns long human-written texts into instructions through backtranslation, which raises coverage and copyright problems. Another imitates stronger proprietary systems like GPT-4o, which is limited by terms of use.

The Qwen Team proposes Self-Lengthen, an iterative scheme that needs only seed instructions and an open-source instruct model. A Generator writes a draft. An Extender splits the draft at punctuation and expands it in two stages, roughly doubling its length. The lengthened texts are then used to fine-tune both roles, and the cycle repeats. In benchmarks and human evaluation, the method outperforms backtranslation and imitation baselines on long-form writing without hurting general abilities. This matters because long-output training data can be bootstrapped from the model itself, without external corpora or closed models.

## Key points
- Self-Lengthen has two roles, Generator and Extender, both initialized from the same instruct model. Each round (macro-iteration) retrains both roles on the expanded outputs, and output length roughly doubles per iteration with similar quality.
- The resource comparison is explicit. Suri (backtranslation) needs high-quality human-written text, LongWriter (behavior imitation) relies on GPT-4/GPT-4o, and Self-Lengthen uses only seed instructions and an open instruct model.
- In the authors' tests, four strong instruct models (Mistral-Large-Instruct, Llama3.1-70B-Instruct, Qwen2-72B-Instruct, Qwen2-7B-Instruct) could not go much beyond 2,000 words when asked for a specific length.
- The method was tested on Qwen2 and LLaMA3 backbones, and general performance on MMLU and AlignBench stayed stable.
- The technique was integrated into the Qwen2.5 series, raising maximum output length from about 1,000 to 8,000 words. Code is available at github.com/QwenLM/Self-Lengthen.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Alibaba Group](https://md.research.elamur.ai/free/en/companies/alibaba-group/) — Company, Hangzhou, China · 168 works · 15,386 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works

## Cite

Cite as: Shanghaoran Quan, Tianyi Tang, Bowen Yu, An Yang, Dayiheng Liu, Bofei Gao, Jianhong Tu, Yichang Zhang, Jingren Zhou and Junyang Lin. 2024. Language Models can Self-Lengthen to Generate Long Texts. arXiv:2410.23933.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{quan2024language,
  title = {{Language Models can Self-Lengthen to Generate Long Texts}},
  author = {Shanghaoran Quan, Tianyi Tang, Bowen Yu, An Yang, Dayiheng Liu, Bofei Gao, Jianhong Tu, Yichang Zhang, Jingren Zhou and Junyang Lin},
  year = {2024},
  doi = {10.48550/arxiv.2410.23933},
  eprint = {2410.23933},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2410.23933}
}
```

- [← previous Infinity-MM: Scaling Multimodal Performance with Large-Scale and High-Quality Instruction Data](https://md.research.elamur.ai/free/en/papers/gu-2024-infinity-mm-scaling-multimodal-performance/)
- [next → Major Entity Identification: A Generalizable Alternative to Coreference Resolution](https://md.research.elamur.ai/free/en/papers/manikantan-2024-major-entity-identification-generalizable-alternative/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/9/)

## References

41 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774, 2023.
2. Anthropic. Anthropic: Introducing claude 3.5 sonnet, 2024. URL [https://www.anthropic.com/news/claude-3-5-sonnet](https://www.anthropic.com/news/claude-3-5-sonnet). [↗](https://www.anthropic.com/news/claude-3-5-sonnet)
3. Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, et al. [Longbench: A bilingual, multitask benchmark for long context understanding](https://md.research.elamur.ai/free/en/papers/bai-2023-longbench-bilingual-multitask-benchmark/). arXiv preprint arXiv:2308.14508, 2023.
4. Yushi Bai, Jiajie Zhang, Xin Lv, Linzhi Zheng, Siqi Zhu, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. [Longwriter: Unleashing 10,000+ word generation from long context llms](https://md.research.elamur.ai/free/en/papers/bai-2024-longwriter-unleashing-10-000/). arXiv preprint arXiv:2408.07055, 2024.
5. Boxi Cao, Keming Lu, Xinyu Lu, Jiawei Chen, Mengjie Ren, Hao Xiang, Peilin Liu, Yaojie Lu, Ben He, Xianpei Han, et al. Towards scalable automated alignment of llms: A survey. arXiv preprint arXiv:2406.01252, 2024. — [arXiv:2406.01252](https://arxiv.org/abs/2406.01252) [find](https://scholar.google.com/scholar?q=Boxi+Cao%2C+Keming+Lu%2C+Xinyu+Lu%2C+Jiawei+Chen%2C+Mengjie+Ren%2C+Hao+Xiang%2C+Peilin+Liu%2C+Yaojie+Lu%2C+Ben+He%2C+Xianpei+Han%2C+et+al%2E+Towards+scalable+automated+alignment+of+llms%3A+A+survey%2E+arXiv+preprint+arXiv%3A2406)
6. John Joon Young Chung, Wooseok Kim, Kang Min Yoo, Hwaran Lee, Eytan Adar, and Minsuk Chang. Talebrush: Sketching stories with generative pretrained language models. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, pp. 1–19, 2022. [find](https://scholar.google.com/scholar?q=John+Joon+Young+Chung%2C+Wooseok+Kim%2C+Kang+Min+Yoo%2C+Hwaran+Lee%2C+Eytan+Adar%2C+and+Minsuk+Chang%2E+Talebrush%3A+Sketching+stories+with+generative+pretrained+language+models%2E+In+Proceedings+of+the+2022+CHI+Conf)
7. Andy Coenen, Luke Davis, Daphne Ippolito, Emily Reif, and Ann Yuan. Wordcraft: A human-ai collaborative editor for story writing. arXiv preprint arXiv:2107.07430, 2021. — [arXiv:2107.07430](https://arxiv.org/abs/2107.07430) [find](https://scholar.google.com/scholar?q=Andy+Coenen%2C+Luke+Davis%2C+Daphne+Ippolito%2C+Emily+Reif%2C+and+Ann+Yuan%2E+Wordcraft%3A+A+human%2Dai+collaborative+editor+for+story+writing%2E+arXiv+preprint+arXiv%3A2107%2E07430%2C+2021%2E+%E2%80%94+%5BarXiv%3A2107%2E07430%5D%28https%3A%2F%2Far)
8. Guanting Dong, Keming Lu, Chengpeng Li, Tingyu Xia, Bowen Yu, Chang Zhou, and Jingren Zhou. Self-play with execution feedback: Improving instruction-following capabilities of large language models. arXiv preprint arXiv:2406.13542, 2024. [find](https://scholar.google.com/scholar?q=Guanting+Dong%2C+Keming+Lu%2C+Chengpeng+Li%2C+Tingyu+Xia%2C+Bowen+Yu%2C+Chang+Zhou%2C+and+Jingren+Zhou%2E+Self%2Dplay+with+execution+feedback%3A+Improving+instruction%2Dfollowing+capabilities+of+large+language+models%2E+ar)
9. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/), 2024.
10. Angela Fan, Mike Lewis, and Yann Dauphin. [Hierarchical neural story generation](https://md.research.elamur.ai/free/en/papers/fan-2018-hierarchical-neural-story-generation/). arXiv preprint arXiv:1805.04833, 2018.
11. Angela Fan, Mike Lewis, and Yann Dauphin. Strategies for structuring story generation. arXiv preprint arXiv:1902.01109, 2019. [find](https://scholar.google.com/scholar?q=Angela+Fan%2C+Mike+Lewis%2C+and+Yann+Dauphin%2E+Strategies+for+structuring+story+generation%2E+arXiv+preprint+arXiv%3A1902%2E01109%2C+2019%2E)
12. Bofei Gao, Feifan Song, Yibo Miao, Zefan Cai, Zhe Yang, Liang Chen, Helan Hu, Runxin Xu, Qingxiu Dong, Ce Zheng, et al. Towards a unified view of preference learning for [large language models: A survey](https://md.research.elamur.ai/free/en/papers/minaee-2024-large-language-models-survey/). arXiv preprint arXiv:2409.02795, 2024. — [arXiv:2409.02795](https://arxiv.org/abs/2409.02795)
13. Seraphina Goldfarb-Tarrant, Haining Feng, and Nanyun Peng. Plan, write, and revise: an interactive system for open-domain story generation. arXiv preprint arXiv:1904.02357, 2019. [find](https://scholar.google.com/scholar?q=Seraphina+Goldfarb%2DTarrant%2C+Haining+Feng%2C+and+Nanyun+Peng%2E+Plan%2C+write%2C+and+revise%3A+an+interactive+system+for+open%2Ddomain+story+generation%2E+arXiv+preprint+arXiv%3A1904%2E02357%2C+2019%2E)
14. Chi Han, Qifan Wang, Wenhan Xiong, Yu Chen, Heng Ji, and Sinong Wang. Lm-infinite: Simple on-the-fly length generalization for large language models. arXiv preprint arXiv:2308.16137, 2023. — [arXiv:2308.16137](https://arxiv.org/abs/2308.16137) [find](https://scholar.google.com/scholar?q=Chi+Han%2C+Qifan+Wang%2C+Wenhan+Xiong%2C+Yu+Chen%2C+Heng+Ji%2C+and+Sinong+Wang%2E+Lm%2Dinfinite%3A+Simple+on%2Dthe%2Dfly+length+generalization+for+large+language+models%2E+arXiv+preprint+arXiv%3A2308%2E16137%2C+2023%2E+%E2%80%94+%5BarXiv%3A23)
15. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). arXiv preprint arXiv:2009.03300, 2020.
16. Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023. [find](https://scholar.google.com/scholar?q=Albert+Q+Jiang%2C+Alexandre+Sablayrolles%2C+Arthur+Mensch%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guillaume+Lample%2C+Lucile+Saulnier%2C+et+al%2E+Mistral+7b)
17. Gregory Kamradt. Needle in a haystack - pressure testing llms, 2023. URL [https://github.com/gkamradt/LLMTest_NeedleInAHaystack](https://github.com/gkamradt/LLMTest_NeedleInAHaystack). [↗](https://github.com/gkamradt/LLMTest_NeedleInAHaystack)
18. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. [Efficient memory management for large language model serving with pagedattention](https://md.research.elamur.ai/free/en/papers/kwon-2023-efficient-memory-management-large-language/). In Proceedings of the 29th Symposium on Operating Systems Principles, pp. 611–626, 2023.
19. Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. [A diversity-promoting objective function for neural conversation models](https://md.research.elamur.ai/free/en/papers/li-2016-diversity-promoting-objective-function-neural/). arXiv preprint arXiv:1510.03055, 2015.
20. Xian Li, Ping Yu, Chunting Zhou, Timo Schick, Omer Levy, Luke Zettlemoyer, Jason Weston, and Mike Lewis. [Self-alignment with instruction backtranslation](https://md.research.elamur.ai/free/en/papers/li-2023-self-alignment-instruction-backtranslation/). arXiv preprint arXiv:2308.06259, 2023.
21. Xiao Liu, Xuanyu Lei, Shengyuan Wang, Yue Huang, Zhuoer Feng, Bosi Wen, Jiale Cheng, Pei Ke, Yifan Xu, Weng Lam Tam, et al. [Alignbench: Benchmarking chinese alignment of large language models](https://md.research.elamur.ai/free/en/papers/liu-2023-alignbench-benchmarking-chinese-alignment/). arXiv preprint arXiv:2311.18743, 2023.
22. Pratyush Maini, Skyler Seto, He Bai, David Grangier, Yizhe Zhang, and Navdeep Jaitly. Rephrasing the web: A recipe for compute and data-efficient language modeling. arXiv preprint arXiv:2401.16380, 2024. [find](https://scholar.google.com/scholar?q=Pratyush+Maini%2C+Skyler+Seto%2C+He+Bai%2C+David+Grangier%2C+Yizhe+Zhang%2C+and+Navdeep+Jaitly%2E+Rephrasing+the+web%3A+A+recipe+for+compute+and+data%2Defficient+language+modeling%2E+arXiv+preprint+arXiv%3A2401%2E16380%2C+20)
23. Saurav Pawar, SM Tonmoy, SM Zaman, Vinija Jain, Aman Chadha, and Amitava Das. The what, why, and how of context length extension techniques in large language models–a detailed survey. arXiv preprint arXiv:2401.07872, 2024. — [arXiv:2401.07872](https://arxiv.org/abs/2401.07872) [find](https://scholar.google.com/scholar?q=Saurav+Pawar%2C+SM+Tonmoy%2C+SM+Zaman%2C+Vinija+Jain%2C+Aman+Chadha%2C+and+Amitava+Das%2E+The+what%2C+why%2C+and+how+of+context+length+extension+techniques+in+large+language+models%E2%80%93a+detailed+survey%2E+arXiv+preprint+a)
24. Chau Minh Pham, Simeng Sun, and Mohit Iyyer. Suri: Multi-constraint instruction following for long-form text generation. arXiv preprint arXiv:2406.19371, 2024. — [arXiv:2406.19371](https://arxiv.org/abs/2406.19371) [find](https://scholar.google.com/scholar?q=Chau+Minh+Pham%2C+Simeng+Sun%2C+and+Mohit+Iyyer%2E+Suri%3A+Multi%2Dconstraint+instruction+following+for+long%2Dform+text+generation%2E+arXiv+preprint+arXiv%3A2406%2E19371%2C+2024%2E+%E2%80%94+%5BarXiv%3A2406%2E19371%5D%28https%3A%2F%2Farxiv%2Eorg%2Fa)
25. Shanghaoran Quan. Automatically generating numerous context-driven sft data for llms across diverse granularity. arXiv preprint arXiv:2405.16579, 2024a. — [arXiv:2405.16579](https://arxiv.org/abs/2405.16579) [find](https://scholar.google.com/scholar?q=Shanghaoran+Quan%2E+Automatically+generating+numerous+context%2Ddriven+sft+data+for+llms+across+diverse+granularity%2E+arXiv+preprint+arXiv%3A2405%2E16579%2C+2024a%2E+%E2%80%94+%5BarXiv%3A2405%2E16579%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2405)
26. Shanghaoran Quan. Dmoerm: Recipes of mixture-of-experts for effective reward modeling. arXiv preprint arXiv:2403.01197, 2024b. [find](https://scholar.google.com/scholar?q=Shanghaoran+Quan%2E+Dmoerm%3A+Recipes+of+mixture%2Dof%2Dexperts+for+effective+reward+modeling%2E+arXiv+preprint+arXiv%3A2403%2E01197%2C+2024b%2E)
27. Haoran Que, Feiyu Duan, Liqun He, Yutao Mou, Wangchunshu Zhou, Jiaheng Liu, Wenge Rong, Zekun Moore Wang, Jian Yang, Ge Zhang, et al. Hellobench: Evaluating long text generation capabilities of large language models. arXiv preprint arXiv:2409.16191, 2024. — [arXiv:2409.16191](https://arxiv.org/abs/2409.16191) [find](https://scholar.google.com/scholar?q=Haoran+Que%2C+Feiyu+Duan%2C+Liqun+He%2C+Yutao+Mou%2C+Wangchunshu+Zhou%2C+Jiaheng+Liu%2C+Wenge+Rong%2C+Zekun+Moore+Wang%2C+Jian+Yang%2C+Ge+Zhang%2C+et+al%2E+Hellobench%3A+Evaluating+long+text+generation+capabilities+of+large+)
28. Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. Zero: Memory optimizations toward training trillion parameter models. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pp. 1–16. IEEE, 2020. [find](https://scholar.google.com/scholar?q=Samyam+Rajbhandari%2C+Jeff+Rasley%2C+Olatunji+Ruwase%2C+and+Yuxiong+He%2E+Zero%3A+Memory+optimizations+toward+training+trillion+parameter+models%2E+In+SC20%3A+International+Conference+for+High+Performance+Computing)
29. Yijia Shao, Yucheng Jiang, Theodore A Kanell, Peter Xu, Omar Khattab, and Monica S Lam. [Assisting in writing wikipedia-like articles from scratch with large language models](https://md.research.elamur.ai/free/en/papers/shao-2024-assisting-writing-wikipedia-like-articles/). arXiv preprint arXiv:2402.14207, 2024.
30. Xiaofei Sun, Zijun Sun, Yuxian Meng, Jiwei Li, and Chun Fan. Summarize, outline, and elaborate: Long-text generation via hierarchical supervision from extractive summaries. arXiv preprint arXiv:2010.07074, 2020. [find](https://scholar.google.com/scholar?q=Xiaofei+Sun%2C+Zijun+Sun%2C+Yuxian+Meng%2C+Jiwei+Li%2C+and+Chun+Fan%2E+Summarize%2C+outline%2C+and+elaborate%3A+Long%2Dtext+generation+via+hierarchical+supervision+from+extractive+summaries%2E+arXiv+preprint+arXiv%3A2010%2E0)
31. Bowen Tan, Zichao Yang, Maruan AI-Shedivat, Eric P Xing, and Zhiting Hu. Progressive generation of long text with pretrained language models. arXiv preprint arXiv:2006.15720, 2020. [find](https://scholar.google.com/scholar?q=Bowen+Tan%2C+Zichao+Yang%2C+Maruan+AI%2DShedivat%2C+Eric+P+Xing%2C+and+Zhiting+Hu%2E+Progressive+generation+of+long+text+with+pretrained+language+models%2E+arXiv+preprint+arXiv%3A2006%2E15720%2C+2020%2E)
32. Tiannan Wang, Jiamin Chen, Qingrui Jia, Shuai Wang, Ruoyu Fang, Huilin Wang, Zhaowei Gao, Chunzhao Xie, Chuou Xu, Jihong Dai, et al. [Weaver: Foundation models for creative writing](https://md.research.elamur.ai/free/en/papers/wang-2024-weaver-foundation-models-creative-writing/). arXiv preprint arXiv:2401.17268, 2024.
33. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language models with self-generated instructions. arXiv preprint arXiv:2212.10560, 2022. [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Yeganeh+Kordi%2C+Swaroop+Mishra%2C+Alisa+Liu%2C+Noah+A+Smith%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+Self%2Dinstruct%3A+Aligning+language+models+with+self%2Dgenerated+instructions%2E+arXiv+preprint)
34. Xiaohan Xu, Ming Li, Chongyang Tao, Tao Shen, Reynold Cheng, Jinyang Li, Can Xu, Dacheng Tao, and Tianyi Zhou. A survey on knowledge distillation of large language models. arXiv preprint arXiv:2402.13116, 2024. — [arXiv:2402.13116](https://arxiv.org/abs/2402.13116) [find](https://scholar.google.com/scholar?q=Xiaohan+Xu%2C+Ming+Li%2C+Chongyang+Tao%2C+Tao+Shen%2C+Reynold+Cheng%2C+Jinyang+Li%2C+Can+Xu%2C+Dacheng+Tao%2C+and+Tianyi+Zhou%2E+A+survey+on+knowledge+distillation+of+large+language+models%2E+arXiv+preprint+arXiv%3A2402%2E13)
35. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. Qwen2 technical report. arXiv preprint arXiv:2407.10671, 2024. [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chang+Zhou%2C+Chengpeng+Li%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+et+al%2E+Qwen2+technical+report%2E+arXiv+preprint+arXiv%3A2407%2E10671%2C+2024%2E)
36. Kevin Yang, Dan Klein, Nanyun Peng, and Yuandong Tian. [Doc: Improving long story coherence with detailed outline control](https://md.research.elamur.ai/free/en/papers/yang-2023-doc-improving-long-story-coherence/). arXiv preprint arXiv:2212.10077, 2022a.
37. Kevin Yang, Yuandong Tian, Nanyun Peng, and Dan Klein. Re3: Generating longer stories with recursive reprompting and revision. arXiv preprint arXiv:2210.06774, 2022b. [find](https://scholar.google.com/scholar?q=Kevin+Yang%2C+Yuandong+Tian%2C+Nanyun+Peng%2C+and+Dan+Klein%2E+Re3%3A+Generating+longer+stories+with+recursive+reprompting+and+revision%2E+arXiv+preprint+arXiv%3A2210%2E06774%2C+2022b%2E)
38. Lili Yao, Nanyun Peng, Ralph Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan. Plan-and-write: Towards better automatic storytelling. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 7378–7385, 2019. [find](https://scholar.google.com/scholar?q=Lili+Yao%2C+Nanyun+Peng%2C+Ralph+Weischedel%2C+Kevin+Knight%2C+Dongyan+Zhao%2C+and+Rui+Yan%2E+Plan%2Dand%2Dwrite%3A+Towards+better+automatic+storytelling%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligenc)
39. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. [Judging llm-as-a-judge with mt-bench and chatbot arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). Advances in Neural Information Processing Systems, 36:46595–46623, 2023.
40. Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, and Zheyan Luo. [Llamafactory: Unified efficient fine-tuning of 100+ language models](https://md.research.elamur.ai/free/en/papers/zheng-2024-llamafactory-unified-efficient-fine-tuning/). arXiv preprint arXiv:2403.13372, 2024.
41. Wangchunshu Zhou, Yuchen Eleanor Jiang, Peng Cui, Tiannan Wang, Zhenxin Xiao, Yifan Hou, Ryan Cotterell, and Mrinmaya Sachan. [Recurrentgpt: Interactive generation of (arbitrarily) long text](https://md.research.elamur.ai/free/en/papers/zhou-2023-recurrentgpt-interactive-generation-arbitrarily-long/). arXiv preprint arXiv:2305.13304, 2023.

*Updated 30 October 2026*