---
title: "Self-Instructed Derived Prompt Generation Meets In-Context"
description: "Rewriting prompts is a common way to get better answers from LLMs that cannot be fine-tuned, such… 53 references, cited by 5 papers here."
authors: ["Zhuo Li", "Yuhao Du", "Jinpeng Hu", "Xiang Wan", "Anningzhe Gao"]
year: 2025
venue: "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2025.acl-long.92/"
doi: "10.18653/v1/2025.acl-long.92"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/li-2025-self-instructed-derived-prompt-generation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/li-2025-self-instructed-derived-prompt-generation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/9/) / Self-Instructed Derived Prompt Generation Meets In-Context Learning: Unlocking New Potential of Black-Box LLMs

# Self-Instructed Derived Prompt Generation Meets In-Context Learning: Unlocking New Potential of Black-Box LLMs

**Authors:** [Zhuo Li](https://md.research.elamur.ai/free/en/authors/zhuo-li/), [Yuhao Du](https://md.research.elamur.ai/free/en/authors/yuhao-du/), [Jinpeng Hu](https://md.research.elamur.ai/free/en/authors/jinpeng-hu/), [Xiang Wan](https://md.research.elamur.ai/free/en/authors/xiang-wan/) and [Anningzhe Gao](https://md.research.elamur.ai/free/en/authors/anningzhe-gao/) · **Year:** 2025 · **Venue:** [Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2025](https://md.research.elamur.ai/free/en/venues/acl/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.acl-long.92/) · [PDF](https://aclanthology.org/2025.acl-long.92.pdf) · [DOI 10.18653/v1/2025.acl-long.92](https://doi.org/10.18653/v1/2025.acl-long.92) · [OpenAlex](https://openalex.org/W4412886763)

**Citations:** 4 citations (OpenAlex, 2026-10-14) · [5](https://md.research.elamur.ai/free/en/papers/li-2025-self-instructed-derived-prompt-generation/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Rewriting prompts is a common way to get better answers from LLMs that cannot be fine-tuned, such as GPT4. However, rewritten prompts often drift from the user's request by narrowing the topic or shifting its focus. The authors leave the original prompt untouched. Instead, they train a separate generator to produce a "derived prompt", a closely related and better-phrased variant.

The generator is trained with a self-instructed reinforcement learning loop:
- it proposes a derived prompt;
- a frozen response model answers it;
- a reward model scores that answer;
- the generator is updated with a KL penalty against a reference copy.

Training needs only original prompts, so no curated original/refined pairs are collected. At inference, the derived prompt and its answer serve as a one-shot in-context demonstration placed before the original query.

On several downstream datasets the method reportedly beats prompt refiners such as BPO and PAS, including on GPT4. This shows that black-box models can be improved without weight access while still respecting what the user asked.

## Key points
- **Motivating failures:** Refiners can distort intent. Examples come from Llama3-8B-Instruct and BPO, e.g. reducing a question about green tea's health benefits to cancer prevention alone.
- **Training setup:** The training objective is PPO-style RLHF. Its reward comes from a frozen response model's answer to the derived prompt, so no prompt-pair data is collected. Among the compared methods (OP, BPO, PAS, ours), only this one needs no data collection.
- **Inference cost:**

  | Method | Generation length | Inference time (s) |
  |---|---|---|
  | OP | 383.8 | 12.06 |
  | BPO | 439.1 | 14.14 |
  | PAS | 465.3 | 15.13 |
  | Ours | 521.6 | 16.18 |

  The authors present this as an acceptable trade-off for the quality gains.
- **Compatibility:** The framework works as a plug-and-play add-on to existing refinement methods.
- **Limitations:**
  - Only the one-shot setting is explored.
  - RL training adds compute.
  - Extra LLM queries are needed to build each demonstration.
  - Math and coding tasks are not yet handled well.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Shenzhen Research Institute of Big Data](https://md.research.elamur.ai/free/en/universities/shenzhen-research-institute-of-big-data/) — University, Shenzhen, China · 17 works · 1,721 citations
- [Hefei University of Technology](https://md.research.elamur.ai/free/en/universities/hefei-university-of-technology/) — University, Hefei, China · 10 works · 1,615 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works

## Cite

Cite as: Zhuo Li, Yuhao Du, Jinpeng Hu, Xiang Wan and Anningzhe Gao. 2025. Self-Instructed Derived Prompt Generation Meets In-Context Learning: Unlocking New Potential of Black-Box LLMs. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{li2025self,
  title = {{Self-Instructed Derived Prompt Generation Meets In-Context Learning: Unlocking New Potential of Black-Box LLMs}},
  author = {Zhuo Li, Yuhao Du, Jinpeng Hu, Xiang Wan and Anningzhe Gao},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.92},
  url = {https://doi.org/10.18653/v1/2025.acl-long.92}
}
```

- [← previous SafeKey: Amplifying Aha-Moment Insights for Safety Reasoning](https://md.research.elamur.ai/free/en/papers/zhou-2025-safekey-amplifying-aha-moment-insights/)
- [next → Sibyl: Empowering Empathetic Dialogue Generation in Large Language Models via Sensible and Visionary Commonsense Inference](https://md.research.elamur.ai/free/en/papers/wang-2025-sibyl-empowering-empathetic-dialogue-generation/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/9/)

## References

53 references, 27 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. \#\# Acknowledgments This work is supported by the Shenzhen Science and Technology Program JCYJ20220818103001002), the Guangdong Provincial Key Laboratory of Big Data Computing, The Chinese University of Hong Kong, Shenzhen, the Longgang District Special Funds for Science and Technology Innovation (LG… [find](https://scholar.google.com/scholar?q=%5C%23%5C%23+Acknowledgments+This+work+is+supported+by+the+Shenzhen+Science+and+Technology+Program+JCYJ20220818103001002%29%2C+the+Guangdong+Provincial+Key+Laboratory+of+Big+Data+Computing%2C+The+Chinese+University)
2. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022a. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862. [find](https://scholar.google.com/scholar?q=Yuntao+Bai%2C+Andy+Jones%2C+Kamal+Ndousse%2C+Amanda+Askell%2C+Anna+Chen%2C+Nova+DasSarma%2C+Dawn+Drain%2C+Stanislav+Fort%2C+Deep+Ganguli%2C+Tom+Henighan%2C+et+al%2E+2022a%2E+Training+a+helpful+and+harmless+assistant+with+rei)
3. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. 2022b. [Constitutional ai: Harmlessness from ai feedback](https://md.research.elamur.ai/free/en/papers/bai-2022-constitutional-ai-harmlessness-ai-feedback/). arXiv preprint arXiv:2212.08073.
4. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems, 33:1877–1901.
5. Jiale Cheng, Xiao Liu, Kehan Zheng, Pei Ke, Hongning Wang, Yuxiao Dong, Jie Tang, and Minlie Huang. 2023. [Black-box prompt optimization: Aligning large language models without model training](https://md.research.elamur.ai/free/en/papers/cheng-2024-black-box-prompt-optimization-aligning/). arXiv preprint arXiv:2311.04155.
6. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See [https://vicuna.lmsys](https://vicuna.lmsys). org (accessed 14 April 2023), 2(3):6. [↗](https://vicuna.lmsys)
7. Jinpeng Hu, DanDan Guo, Yang Liu, Zhuo Li, Zhihong Chen, Xiang Wan, and Tsung-Hui Chang. 2023. A simple yet effective subsequence-enhanced approach for cross-domain ner. Proceedings of the AAAI Conference on Artificial Intelligence, 37(11):12890– 12898. [find](https://scholar.google.com/scholar?q=Jinpeng+Hu%2C+DanDan+Guo%2C+Yang+Liu%2C+Zhuo+Li%2C+Zhihong+Chen%2C+Xiang+Wan%2C+and+Tsung%2DHui+Chang%2E+2023%2E+A+simple+yet+effective+subsequence%2Denhanced+approach+for+cross%2Ddomain+ner%2E+Proceedings+of+the+AAAI+Confer)
8. Jinpeng Hu, Zhuo Li, Zhihong Chen, Zhen Li, Xiang Wan, and Tsung-Hui Chang. 2022. Graph enhanced contrastive learning for radiology findings summarization. Preprint, arXiv:2204.00203. [find](https://scholar.google.com/scholar?q=Jinpeng+Hu%2C+Zhuo+Li%2C+Zhihong+Chen%2C+Zhen+Li%2C+Xiang+Wan%2C+and+Tsung%2DHui+Chang%2E+2022%2E+Graph+enhanced+contrastive+learning+for+radiology+findings+summarization%2E+Preprint%2C+arXiv%3A2204%2E00203%2E)
9. Zisu Huang, Xiaohua Wang, Feiran Zhang, Zhibo Xu, Cenyuan Zhang, Xiaoqing Zheng, and Xuanjing Huang. 2024. Enhancing the capability and robustness of large language models through reinforcement learning-driven query refinement. arXiv preprint arXiv:2407.01461. — [arXiv:2407.01461](https://arxiv.org/abs/2407.01461) [find](https://scholar.google.com/scholar?q=Zisu+Huang%2C+Xiaohua+Wang%2C+Feiran+Zhang%2C+Zhibo+Xu%2C+Cenyuan+Zhang%2C+Xiaoqing+Zheng%2C+and+Xuanjing+Huang%2E+2024%2E+Enhancing+the+capability+and+robustness+of+large+language+models+through+reinforcement+learni)
10. Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. 2023. Free dolly: Introducing the world’s first truly open instruction-tuned llm. Company Blog of Databricks. [find](https://scholar.google.com/scholar?q=Mike+Conover%2C+Matt+Hayes%2C+Ankit+Mathur%2C+Jianwei+Xie%2C+Jun+Wan%2C+Sam+Shah%2C+Ali+Ghodsi%2C+Patrick+Wendell%2C+Matei+Zaharia%2C+and+Reynold+Xin%2E+2023%2E+Free+dolly%3A+Introducing+the+world%E2%80%99s+first+truly+open+instruct)
11. Zhengbao Jiang, Frank F Xu, Jun Araki, and Graham Neubig. 2020. [How can we know what language models know?](https://md.research.elamur.ai/free/en/papers/jiang-2020-how-can-we-know-what/) Transactions of the Association for Computational Linguistics, 8:423–438.
12. DeepSpeed. 2024. Deepspeed. [https://www.deepspeed.ai/](https://www.deepspeed.ai/). Accessed: 2024-08-08. [↗](https://www.deepspeed.ai/)
13. Weize Kong, Spurthi Amba Hombaiah, Mingyang Zhang, Qiaozhu Mei, and Michael Bendersky. 2024. Prewrite: Prompt rewriting with reinforcement learning. arXiv preprint arXiv:2401.08189. [find](https://scholar.google.com/scholar?q=Weize+Kong%2C+Spurthi+Amba+Hombaiah%2C+Mingyang+Zhang%2C+Qiaozhu+Mei%2C+and+Michael+Bendersky%2E+2024%2E+Prewrite%3A+Prompt+rewriting+with+reinforcement+learning%2E+arXiv+preprint+arXiv%3A2401%2E08189%2E)
14. Mingkai Deng, Jianyu Wang, Cheng-Ping Hsieh, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric P Xing, and Zhiting Hu. 2022. [Rlprompt: Optimizing discrete text prompts with reinforcement learning](https://md.research.elamur.ai/free/en/papers/deng-2022-rlprompt-optimizing-discrete-text-prompts/). arXiv preprint arXiv:2205.12548.
15. Jacob Devlin. 2018. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). arXiv preprint arXiv:1810.04805.
16. Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, and Zhifang Sui. 2022. [A survey on in-context learning](https://md.research.elamur.ai/free/en/papers/dong-2023-survey-context-learning/). arXiv preprint arXiv:2301.00234.
17. Yuhao Du, Zhuo Li, Pengyu Cheng, Zhihong Chen, Yuejiao Xie, Xiang Wan, and Anningzhe Gao. 2025a. Simplify rlhf as reward-weighted sft: A variational method. Preprint, arXiv:2502.11026. — [arXiv:2502.11026](https://arxiv.org/abs/2502.11026) [find](https://scholar.google.com/scholar?q=Yuhao+Du%2C+Zhuo+Li%2C+Pengyu+Cheng%2C+Zhihong+Chen%2C+Yuejiao+Xie%2C+Xiang+Wan%2C+and+Anningzhe+Gao%2E+2025a%2E+Simplify+rlhf+as+reward%2Dweighted+sft%3A+A+variational+method%2E+Preprint%2C+arXiv%3A2502%2E11026%2E+%E2%80%94+%5BarXiv%3A2502%2E1)
18. Yuhao Du, Zhuo Li, Pengyu Cheng, Xiang Wan, and Anningzhe Gao. 2025b. Atoxia: Red-teaming large language models with target toxic answers. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 3251–3266, Albuquerque, New Mexico. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Yuhao+Du%2C+Zhuo+Li%2C+Pengyu+Cheng%2C+Xiang+Wan%2C+and+Anningzhe+Gao%2E+2025b%2E+Atoxia%3A+Red%2Dteaming+large+language+models+with+target+toxic+answers%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A)
19. Chrisantha Fernando, Dylan Banarse, Henryk Michalewski, Simon Osindero, and Tim Rocktäschel. 2023. [Promptbreeder: Self-referential self-improvement via prompt evolution](https://md.research.elamur.ai/free/en/papers/fernando-2023-promptbreeder-self-referential-self-improvement/). arXiv preprint arXiv:2309.16797.
20. Adi Haviv, Jonathan Berant, and Amir Globerson. 2021. Bertese: Learning to speak to bert. arXiv preprint arXiv:2103.05327. [find](https://scholar.google.com/scholar?q=Adi+Haviv%2C+Jonathan+Berant%2C+and+Amir+Globerson%2E+2021%2E+Bertese%3A+Learning+to+speak+to+bert%2E+arXiv+preprint+arXiv%3A2103%2E05327%2E)
21. Harrison Lee, Samrat Phatale, Hassan Mansoor, Kellie Lu, Thomas Mesnard, Colton Bishop, Victor Carbune, and Abhinav Rastogi. 2023. Rlaif: Scaling reinforcement learning from human feedback with ai feedback. arXiv preprint arXiv:2309.00267. [find](https://scholar.google.com/scholar?q=Harrison+Lee%2C+Samrat+Phatale%2C+Hassan+Mansoor%2C+Kellie+Lu%2C+Thomas+Mesnard%2C+Colton+Bishop%2C+Victor+Carbune%2C+and+Abhinav+Rastogi%2E+2023%2E+Rlaif%3A+Scaling+reinforcement+learning+from+human+feedback+with+ai+fee)
22. Zhuo Li, Yuhao Du, Xiaoqi Jiao, Yiwen Guo, Yuege Feng, Xiang Wan, Anningzhe Gao, and Jinpeng Hu. 2025. [Add-one-in: Incremental sample selection for large language models via a choice-based greedy paradigm](https://md.research.elamur.ai/free/en/papers/li-2025-add-one-incremental-sample-selection/). Preprint, arXiv:2503.02359.
23. Ziniu Li, Tian Xu, Yushun Zhang, Yang Yu, Ruoyu Sun, and Zhi-Quan Luo. 2023. Remax: A simple, effective, and efficient method for aligning large language models. arXiv preprint arXiv:2310.10505. [find](https://scholar.google.com/scholar?q=Ziniu+Li%2C+Tian+Xu%2C+Yushun+Zhang%2C+Yang+Yu%2C+Ruoyu+Sun%2C+and+Zhi%2DQuan+Luo%2E+2023%2E+Remax%3A+A+simple%2C+effective%2C+and+efficient+method+for+aligning+large+language+models%2E+arXiv+preprint+arXiv%3A2310%2E10505%2E)
24. Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. 2021. [What makes good in-context examples for gpt-3?](https://md.research.elamur.ai/free/en/papers/liu-2022-what-makes-good-context-examples/) arXiv preprint arXiv:2101.06804.
25. Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023. [Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing](https://md.research.elamur.ai/free/en/papers/liu-2021-pre-train-prompt-predict-systematic/). ACM Computing Surveys, 55(9):1–35.
26. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2023. Self-refine: Iterative refinement with… [find](https://scholar.google.com/scholar?q=Aman+Madaan%2C+Niket+Tandon%2C+Prakhar+Gupta%2C+Skyler+Hallinan%2C+Luyu+Gao%2C+Sarah+Wiegreffe%2C+Uri+Alon%2C+Nouha+Dziri%2C+Shrimai+Prabhumoye%2C+Yiming+Yang%2C+Shashank+Gupta%2C+Bodhisattwa+Prasad+Majumder%2C+Katherine+Her)
27. Sewon Min, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2021. [Noisy channel language model prompting for few-shot text classification](https://md.research.elamur.ai/free/en/papers/min-2021-noisy-channel-language-model-prompting/). arXiv preprint arXiv:2108.04106.
28. R OpenAI et al. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). ArXiv, 2303:08774.
29. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in neural information processing systems, 35:27730–27744.
30. Reid Pryzant, Dan Iter, Jerry Li, Yin Tat Lee, Chenguang Zhu, and Michael Zeng. 2023. [Automatic prompt optimization with" gradient descent" and beam search](https://md.research.elamur.ai/free/en/papers/pryzant-2023-automatic-prompt-optimization-gradient-descent/). arXiv preprint arXiv:2305.03495.
31. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. [Llama 2: Open foundation and fine-tuned chat models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/). arXiv preprint arXiv:2307.09288.
32. Xingchen Wan, Ruoxi Sun, Hanjun Dai, Sercan O Arik, and Tomas Pfister. 2023. Better zero-shot reasoning with self-adaptive prompting. arXiv preprint arXiv:2305.14106. [find](https://scholar.google.com/scholar?q=Xingchen+Wan%2C+Ruoxi+Sun%2C+Hanjun+Dai%2C+Sercan+O+Arik%2C+and+Tomas+Pfister%2E+2023%2E+Better+zero%2Dshot+reasoning+with+self%2Dadaptive+prompting%2E+arXiv+preprint+arXiv%3A2305%2E14106%2E)
33. Yidong Wang, Zhuohao Yu, Zhengran Zeng, Linyi Yang, Cunxiang Wang, Hao Chen, Chaoya Jiang, Rui Xie, Jindong Wang, Xing Xie, et al. 2023. [Pandalm: An automatic evaluation benchmark for llm instruction tuning optimization](https://md.research.elamur.ai/free/en/papers/wang-2023-pandalm-automatic-evaluation-benchmark-llm/). arXiv preprint arXiv:2306.05087.
34. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. 2024. [Direct preference optimization: Your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/). Advances in Neural Information Processing Systems, 36.
35. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2022. Self-instruct: Aligning language models with self-generated instructions. arXiv preprint arXiv:2212.10560. [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Yeganeh+Kordi%2C+Swaroop+Mishra%2C+Alisa+Liu%2C+Noah+A+Smith%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+2022%2E+Self%2Dinstruct%3A+Aligning+language+models+with+self%2Dgenerated+instructions%2E+arXiv+pr)
36. Laria Reynolds and Kyle McDonell. 2021. [Prompt programming for large language models: Beyond the few-shot paradigm](https://md.research.elamur.ai/free/en/papers/reynolds-2021-prompt-programming-large-language-models/). In Extended abstracts of the 2021 CHI conference on human factors in computing systems, pages 1–7.
37. Albert Webson and Ellie Pavlick. 2021. [Do prompt-based models really understand the meaning of their prompts?](https://md.research.elamur.ai/free/en/papers/webson-2022-do-prompt-based-models-really/) arXiv preprint arXiv:2109.01247.
38. Ohad Rubin, Jonathan Herzig, and Jonathan Berant. 2021. [Learning to retrieve prompts for in-context learning](https://md.research.elamur.ai/free/en/papers/rubin-2021-learning-retrieve-prompts-context-learning/). arXiv preprint arXiv:2112.08633.
39. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Filip+Wolski%2C+Prafulla+Dhariwal%2C+Alec+Radford%2C+and+Oleg+Klimov%2E+2017%2E+Proximal+policy+optimization+algorithms%2E+arXiv+preprint+arXiv%3A1707%2E06347%2E)
40. Taylor Shin, Yasaman Razeghi, Robert L Logan IV, Eric Wallace, and Sameer Singh. 2020. [Autoprompt: Eliciting knowledge from language models with automatically generated prompts](https://md.research.elamur.ai/free/en/papers/shin-2020-autoprompt-eliciting-knowledge-language-models/). arXiv preprint arXiv:2010.15980.
41. Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al. 2022. [Beyond the imitation game: Quantifying and extrapolating the capabilities of language models](https://md.research.elamur.ai/free/en/papers/srivastava-2022-imitation-game-quantifying-extrapolating-capabilities/). arXiv preprint arXiv:2206.04615.
42. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. 2024a. Qwen2 technical report. arXiv preprint arXiv:2407.10671. [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chang+Zhou%2C+Chengpeng+Li%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+et+al%2E+2024a%2E+Qwen2+technical+report%2E+arXiv+preprint+arXiv%3A2407%2E10671%2E)
43. Songhua Yang, Hanjie Zhao, Senbin Zhu, Guangyu Zhou, Hongfei Xu, Yuxiang Jia, and Hongying Zan. 2024b. Zhongjing: Enhancing the chinese medical capabilities of large language model through expert feedback and real-world multi-turn dialogue. In Proceedings of the AAAI Conference on Artificial Intelli… [find](https://scholar.google.com/scholar?q=Songhua+Yang%2C+Hanjie+Zhao%2C+Senbin+Zhu%2C+Guangyu+Zhou%2C+Hongfei+Xu%2C+Yuxiang+Jia%2C+and+Hongying+Zan%2E+2024b%2E+Zhongjing%3A+Enhancing+the+chinese+medical+capabilities+of+large+language+model+through+expert+feed)
44. Seonghyeon Ye, Doyoung Kim, Joel Jang, Joongbo Shin, and Minjoon Seo. 2022. Guess the instruction! flipped learning makes language models stronger zero-shot learners. arXiv preprint arXiv:2210.02969. — [arXiv:2210.02969](https://arxiv.org/abs/2210.02969) [find](https://scholar.google.com/scholar?q=Seonghyeon+Ye%2C+Doyoung+Kim%2C+Joel+Jang%2C+Joongbo+Shin%2C+and+Minjoon+Seo%2E+2022%2E+Guess+the+instruction%21+flipped+learning+makes+language+models+stronger+zero%2Dshot+learners%2E+arXiv+preprint+arXiv%3A2210%2E02969%2E+)
45. Fan Yin, Jesse Vig, Philippe Laban, Shafiq Joty, Caiming Xiong, and Chien-Sheng Jason Wu. 2023. Did you read the instructions? rethinking the effectiveness of task definitions in instruction learning. arXiv preprint arXiv:2306.01150. [find](https://scholar.google.com/scholar?q=Fan+Yin%2C+Jesse+Vig%2C+Philippe+Laban%2C+Shafiq+Joty%2C+Caiming+Xiong%2C+and+Chien%2DSheng+Jason+Wu%2E+2023%2E+Did+you+read+the+instructions%3F+rethinking+the+effectiveness+of+task+definitions+in+instruction+learning%2E)
46. Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. 2020. Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 33:3008– 3021. [find](https://scholar.google.com/scholar?q=Nisan+Stiennon%2C+Long+Ouyang%2C+Jeffrey+Wu%2C+Daniel+Ziegler%2C+Ryan+Lowe%2C+Chelsea+Voss%2C+Alec+Radford%2C+Dario+Amodei%2C+and+Paul+F+Christiano%2E+2020%2E+Learning+to+summarize+with+human+feedback%2E+Advances+in+Neural)
47. JD Zamfirescu-Pereira, Richmond Y Wong, Bjoern Hartmann, and Qian Yang. 2023. Why johnny can’t prompt: how non-ai experts try (and fail) to design llm prompts. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pages 1–21. [find](https://scholar.google.com/scholar?q=JD+Zamfirescu%2DPereira%2C+Richmond+Y+Wong%2C+Bjoern+Hartmann%2C+and+Qian+Yang%2E+2023%2E+Why+johnny+can%E2%80%99t+prompt%3A+how+non%2Dai+experts+try+%28and+fail%29+to+design+llm+prompts%2E+In+Proceedings+of+the+2023+CHI+Conferenc)
48. Derek Tam, Rakesh R Menon, Mohit Bansal, Shashank Srivastava, and Colin Raffel. 2021. [Improving and simplifying pattern exploiting training](https://md.research.elamur.ai/free/en/papers/tam-2021-improving-simplifying-pattern-exploiting-training/). arXiv preprint arXiv:2103.11955.
49. Tianjun Zhang, Xuezhi Wang, Denny Zhou, Dale Schuurmans, and Joseph E Gonzalez. 2022. Tempera: Test-time prompting via reinforcement learning. arXiv preprint arXiv:2211.11890. — [arXiv:2211.11890](https://arxiv.org/abs/2211.11890) [find](https://scholar.google.com/scholar?q=Tianjun+Zhang%2C+Xuezhi+Wang%2C+Denny+Zhou%2C+Dale+Schuurmans%2C+and+Joseph+E+Gonzalez%2E+2022%2E+Tempera%3A+Test%2Dtime+prompting+via+reinforcement+learning%2E+arXiv+preprint+arXiv%3A2211%2E11890%2E+%E2%80%94+%5BarXiv%3A2211%2E11890%5D%28htt)
50. Yao Zhao, Rishabh Joshi, Tianqi Liu, Misha Khalman, Mohammad Saleh, and Peter J Liu. 2023. [Slic-hf: Sequence likelihood calibration with human feedback](https://md.research.elamur.ai/free/en/papers/zhao-2023-slic-hf-sequence-likelihood-calibration/). arXiv preprint arXiv:2305.10425.
51. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2024a. [Judging llm-as-a-judge with mt-bench and chatbot arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). Advances in Neural Information Processing Systems, 36.
52. Miao Zheng, Hao Liang, Fan Yang, Haoze Sun, Tianpeng Li, Lingchu Xiong, Yan Zhang, Youzhen Wu, Kun Li, Yanjun Shen, Mingan Lin, Tao Zhang, Guosheng Dong, Yujing Qiao, Kun Fang, Weipeng Chen, Bin Cui, Wentao Zhang, and Zenan Zhou. 2024b. Pas: Data-efficient plug-and-play prompt augmentation system. P… — [arXiv:2407.06027](https://arxiv.org/abs/2407.06027) [find](https://scholar.google.com/scholar?q=Miao+Zheng%2C+Hao+Liang%2C+Fan+Yang%2C+Haoze+Sun%2C+Tianpeng+Li%2C+Lingchu+Xiong%2C+Yan+Zhang%2C+Youzhen+Wu%2C+Kun+Li%2C+Yanjun+Shen%2C+Mingan+Lin%2C+Tao+Zhang%2C+Guosheng+Dong%2C+Yujing+Qiao%2C+Kun+Fang%2C+Weipeng+Chen%2C+Bin+Cui%2C+)
53. Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. 2022. [Large language models are human-level prompt engineers](https://md.research.elamur.ai/free/en/papers/zhou-2022-large-language-models-human-level/). arXiv preprint arXiv:2211.01910.

*Updated 30 October 2026*