---
title: "Joint Prompt Optimization of Stacked LLMs using Variational"
description: "An LLM proposes candidate prompts based on the current prompt, a batch of inputs, gold targets and… 57 references, cited by 18 papers here."
authors: ["Alessandro Sordoni", "Xingdi Yuan", "Marc-Alexandre Cote", "Matheus Pereira", "Adam Trischler", "Ziang Xiao", "Arian Hosseini", "Friederike Niedtner", "Nicolas Le Roux"]
year: 2023
venue: "arXiv:2306.12509"
source: "https://arxiv.org/abs/2306.12509"
doi: "10.48550/arxiv.2306.12509"
arxiv: "2306.12509"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/sordoni-2023-joint-prompt-optimization-stacked-llms/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/sordoni-2023-joint-prompt-optimization-stacked-llms/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/10/) / Joint Prompt Optimization of Stacked LLMs using Variational Inference

# Joint Prompt Optimization of Stacked LLMs using Variational Inference

**Authors:** [Alessandro Sordoni](https://md.research.elamur.ai/free/en/authors/alessandro-sordoni/), [Xingdi Yuan](https://md.research.elamur.ai/free/en/authors/xingdi-yuan/), [Marc-Alexandre Cote](https://md.research.elamur.ai/free/en/authors/marc-alexandre-cote/), Matheus Pereira, [Adam Trischler](https://md.research.elamur.ai/free/en/authors/adam-trischler/), [Ziang Xiao](https://md.research.elamur.ai/free/en/authors/ziang-xiao/), [Arian Hosseini](https://md.research.elamur.ai/free/en/authors/arian-hosseini/), Friederike Niedtner and Nicolas Le Roux · **Year:** 2023 · **Venue:** [arXiv:2306.12509](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/) ([NeurIPS 2023](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/2023/)) · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2306.12509) · [PDF](https://arxiv.org/pdf/2306.12509) · [DOI 10.48550/arxiv.2306.12509](https://doi.org/10.48550/arxiv.2306.12509) · [arXiv 2306.12509](https://arxiv.org/abs/2306.12509) · [OpenAlex](https://openalex.org/W7133202500)

**Citations:** 5 citations (OpenAlex, 2026-10-14) · [18](https://md.research.elamur.ai/free/en/papers/sordoni-2023-joint-prompt-optimization-stacked-llms/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper treats an instruction-following LLM as a stochastic "language layer." The layer maps a text input to a distribution over text outputs, and its trainable parameters are natural-language prompts rather than weights. Stacking such layers gives a Deep Language Network (DLN). The authors first improve prompt optimization for a single layer (DLN-1) by extending Automatic Prompt Engineer (APE). An LLM proposes candidate prompts based on the current prompt, a batch of inputs, gold targets and the model's own predictions. The candidates are then ranked by length-normalized log-likelihood. For two stacked layers (DLN-2), the text produced by the first layer is treated as a latent variable. Both prompts are learned jointly with variational inference over an approximate posterior on that hidden output. Experiments on reasoning and natural language understanding tasks show that learned DLN-1 prompts beat APE on several tasks, and DLN-2 improves over a single layer. This suggests that stacks of smaller, weaker LLMs could approach GPT-4-level results. The approach matters because it makes LLM pipelines modular, trainable and reusable.

## Key points
- The authors are from Microsoft Research Montréal and MILA. The paper was presented at NeurIPS 2023, and the code is released at github.com/microsoft/deep-language-networks.
- DLN-1 training samples N candidate prompts (proposal temperature 0.7) using a "backward" template. It keeps the candidate with the best length-normalized log-probability of the targets and adds backtracking and a memory of strong prompts.
- The learned prompts combine instructions with verbalized hard examples from the task. They therefore mix zero-shot directives with in-context-learning-style content.
- Chain-Of-Thought (CoT) is reframed as a DLN-2 with a residual connection, and SC-CoT as marginalization over CoTs drawn from a task-agnostic prior. DLN-2 instead learns a task-specific prior over successful reasoning traces.
- Stated limitations and future work: the forward and backward templates are hand-engineered, and only 1- and 2-layer networks are tested, although the framework supports arbitrary directed acyclic graphs. GPT-3 also tends to always produce an answer, which raises the question of fine-tuning "stackable" LLMs.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works

## Cite

Cite as: Alessandro Sordoni, Xingdi Yuan, Marc-Alexandre Cote, Matheus Pereira, Adam Trischler, Ziang Xiao, Arian Hosseini, Friederike Niedtner and Nicolas Le Roux. 2023. Joint Prompt Optimization of Stacked LLMs using Variational Inference. arXiv:2306.12509.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{sordoni2023joint,
  title = {{Joint Prompt Optimization of Stacked LLMs using Variational Inference}},
  author = {Alessandro Sordoni, Xingdi Yuan, Marc-Alexandre Cote, Matheus Pereira, Adam Trischler, Ziang Xiao, Arian Hosseini, Friederike Niedtner and Nicolas {Le Roux}},
  year = {2023},
  doi = {10.48550/arxiv.2306.12509},
  eprint = {2306.12509},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2306.12509}
}
```

- [← previous InSCIt: Information-Seeking Conversations with Mixed-Initiative Interactions](https://md.research.elamur.ai/free/en/papers/wu-2023-inscit-information-seeking-conversations-mixed/)
- [next → Modeling Cross-Cultural Pragmatic Inference with Codenames Duet](https://md.research.elamur.ai/free/en/papers/shaikh-2023-modeling-cross-cultural-pragmatic-inference/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/10/)

## References

57 references, 31 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Bansal, T., Jha, R., and McCallum, A. (2020). Learning to few-shot learn across diverse natural language classification tasks. In Proceedings of the 28th International Conference on Computational Linguistics, pages 5108–5123, Barcelona, Spain (Online). International Committee on Computational Lingui… [find](https://scholar.google.com/scholar?q=Bansal%2C+T%2E%2C+Jha%2C+R%2E%2C+and+McCallum%2C+A%2E+%282020%29%2E+Learning+to+few%2Dshot+learn+across+diverse+natural+language+classification+tasks%2E+In+Proceedings+of+the+28th+International+Conference+on+Computational+Ling)
2. Bender, E. M., Gebru, T., McMillan-Major, A., and Mitchell, M. (2021). On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 610–623. [find](https://scholar.google.com/scholar?q=Bender%2C+E%2E+M%2E%2C+Gebru%2C+T%2E%2C+McMillan%2DMajor%2C+A%2E%2C+and+Mitchell%2C+M%2E+%282021%29%2E+On+the+dangers+of+stochastic+parrots%3A+Can+language+models+be+too+big%3F+In+Proceedings+of+the+2021+ACM+conference+on+fairness%2C+acco)
3. Blei, D. M., Kucukelbir, A., and McAuliffe, J. D. (2017). Variational inference: A review for statisticians. Journal of the American statistical Association, 112(518):859–877. [find](https://scholar.google.com/scholar?q=Blei%2C+D%2E+M%2E%2C+Kucukelbir%2C+A%2E%2C+and+McAuliffe%2C+J%2E+D%2E+%282017%29%2E+Variational+inference%3A+A+review+for+statisticians%2E+Journal+of+the+American+statistical+Association%2C+112%28518%29%3A859%E2%80%93877%2E)
4. Blodgett, S. L., Liao, Q. V., Olteanu, A., Mihalcea, R., Muller, M., Scheuerman, M. K., Tan, C., and Yang, Q. (2022). Responsible language technologies: Foreseeing and mitigating harms. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems, CHI EA ’22, New York, NY,… [find](https://scholar.google.com/scholar?q=Blodgett%2C+S%2E+L%2E%2C+Liao%2C+Q%2E+V%2E%2C+Olteanu%2C+A%2E%2C+Mihalcea%2C+R%2E%2C+Muller%2C+M%2E%2C+Scheuerman%2C+M%2E+K%2E%2C+Tan%2C+C%2E%2C+and+Yang%2C+Q%2E+%282022%29%2E+Responsible+language+technologies%3A+Foreseeing+and+mitigating+harms%2E+In+Extended+Ab)
5. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020). [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems, 33:1877–1901.
6. Creswell, A., Shanahan, M., and Higgins, I. (2022). Selection-inference: Exploiting large language models for interpretable logical reasoning. arXiv preprint arXiv:2205.09712. [find](https://scholar.google.com/scholar?q=Creswell%2C+A%2E%2C+Shanahan%2C+M%2E%2C+and+Higgins%2C+I%2E+%282022%29%2E+Selection%2Dinference%3A+Exploiting+large+language+models+for+interpretable+logical+reasoning%2E+arXiv+preprint+arXiv%3A2205%2E09712%2E)
7. Deng, M., Wang, J., Hsieh, C., Wang, Y., Guo, H., Shu, T., Song, M., Xing, E. P., and Hu, Z. (2022). [Rlprompt: Optimizing discrete text prompts with reinforcement learning](https://md.research.elamur.ai/free/en/papers/deng-2022-rlprompt-optimizing-discrete-text-prompts/). In Goldberg, Y., Kozareva, Z., and Zhang, Y., editors, Proceedings of the 2022 Conference on Empirical Methods in Natural Langu…
8. Dohan, D., Xu, W., Lewkowycz, A., Austin, J., Bieber, D., Lopes, R. G., Wu, Y., Michalewski, H., Saurous, R. A., Sohl-Dickstein, J., et al. (2022). Language model cascades. arXiv preprint arXiv:2207.10342. [find](https://scholar.google.com/scholar?q=Dohan%2C+D%2E%2C+Xu%2C+W%2E%2C+Lewkowycz%2C+A%2E%2C+Austin%2C+J%2E%2C+Bieber%2C+D%2E%2C+Lopes%2C+R%2E+G%2E%2C+Wu%2C+Y%2E%2C+Michalewski%2C+H%2E%2C+Saurous%2C+R%2E+A%2E%2C+Sohl%2DDickstein%2C+J%2E%2C+et+al%2E+%282022%29%2E+Language+model+cascades%2E+arXiv+preprint+arXiv%3A2207%2E1)
9. Du, Y., Li, S., Torralba, A., Tenenbaum, J. B., and Mordatch, I. (2023). [Improving factuality and reasoning in language models through multiagent debate](https://md.research.elamur.ai/free/en/papers/du-2023-improving-factuality-reasoning-language-models/).
10. Gao, T., Fisch, A., and Chen, D. (2021). [Making pre-trained language models better few-shot learners](https://md.research.elamur.ai/free/en/papers/gao-2021-making-pre-trained-language-models/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 3…
11. Hao, S., Gu, Y., Ma, H., Hong, J. J., Wang, Z., Wang, D. Z., and Hu, Z. (2023). Reasoning with language model is planning with world model. arXiv preprint arXiv:2305.14992. [find](https://scholar.google.com/scholar?q=Hao%2C+S%2E%2C+Gu%2C+Y%2E%2C+Ma%2C+H%2E%2C+Hong%2C+J%2E+J%2E%2C+Wang%2C+Z%2E%2C+Wang%2C+D%2E+Z%2E%2C+and+Hu%2C+Z%2E+%282023%29%2E+Reasoning+with+language+model+is+planning+with+world+model%2E+arXiv+preprint+arXiv%3A2305%2E14992%2E)
12. Honovich, O., Shaham, U., Bowman, S. R., and Levy, O. (2022). Instruction induction: From few examples to natural language task descriptions. [find](https://scholar.google.com/scholar?q=Honovich%2C+O%2E%2C+Shaham%2C+U%2E%2C+Bowman%2C+S%2E+R%2E%2C+and+Levy%2C+O%2E+%282022%29%2E+Instruction+induction%3A+From+few+examples+to+natural+language+task+descriptions%2E)
13. Hosseini, A., Reddy, S., Bahdanau, D., Hjelm, R. D., Sordoni, A., and Courville, A. C. (2021). [Understanding by understanding not: Modeling negation in language models](https://md.research.elamur.ai/free/en/papers/hosseini-2021-understanding-understanding-not-modeling-negation/). In Toutanova, K., Rumshisky, A., Zettlemoyer, L., Hakkani-Tür, D., Beltagy, I., Bethard, S., Cotterell, R., Chakraborty, T., and Zh…
14. Huang, W., Xia, F., Xiao, T., Chan, H., Liang, J., Florence, P., Zeng, A., Tompson, J., Mordatch, I., Chebotar, Y., et al. (2022). Inner monologue: Embodied reasoning through planning with language models. arXiv preprint arXiv:2207.05608. [find](https://scholar.google.com/scholar?q=Huang%2C+W%2E%2C+Xia%2C+F%2E%2C+Xiao%2C+T%2E%2C+Chan%2C+H%2E%2C+Liang%2C+J%2E%2C+Florence%2C+P%2E%2C+Zeng%2C+A%2E%2C+Tompson%2C+J%2E%2C+Mordatch%2C+I%2E%2C+Chebotar%2C+Y%2E%2C+et+al%2E+%282022%29%2E+Inner+monologue%3A+Embodied+reasoning+through+planning+with+language+mo)
15. Khot, T., Trivedi, H., Finlayson, M., Fu, Y., Richardson, K., Clark, P., and Sabharwal, A. (2023). Decomposed prompting: A modular approach for solving complex tasks. International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Khot%2C+T%2E%2C+Trivedi%2C+H%2E%2C+Finlayson%2C+M%2E%2C+Fu%2C+Y%2E%2C+Richardson%2C+K%2E%2C+Clark%2C+P%2E%2C+and+Sabharwal%2C+A%2E+%282023%29%2E+Decomposed+prompting%3A+A+modular+approach+for+solving+complex+tasks%2E+International+Conference+on+Learn)
16. Kingma, D. P. and Welling, M. (2022). Auto-encoding variational bayes. [find](https://scholar.google.com/scholar?q=Kingma%2C+D%2E+P%2E+and+Welling%2C+M%2E+%282022%29%2E+Auto%2Dencoding+variational+bayes%2E)
17. Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., and Iwasawa, Y. (2022). [Large language models are zero-shot reasoners](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/). In Koyejo, S., Mohamed, S., Agarwal, A., Belgrave, D., Cho, K., and Oh, A., editors, Advances in Neural Information Processing Systems, volume 35, pages 22199–22213. Curran Associates,…
18. Lazaridou, A., Gribovskaya, E., Stokowiec, W., and Grigorev, N. (2022). [Internet-augmented language models through few-shot prompting for open-domain question answering](https://md.research.elamur.ai/free/en/papers/lazaridou-2022-internet-augmented-language-models-through/). arXiv preprint arXiv:2203.05115.
19. Liu, J., Liu, A., Lu, X., Welleck, S., West, P., Le Bras, R., Choi, Y., and Hajishirzi, H. (2022a). Generated knowledge prompting for commonsense reasoning. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3154– 3169. [find](https://scholar.google.com/scholar?q=Liu%2C+J%2E%2C+Liu%2C+A%2E%2C+Lu%2C+X%2E%2C+Welleck%2C+S%2E%2C+West%2C+P%2E%2C+Le+Bras%2C+R%2E%2C+Choi%2C+Y%2E%2C+and+Hajishirzi%2C+H%2E+%282022a%29%2E+Generated+knowledge+prompting+for+commonsense+reasoning%2E+In+Proceedings+of+the+60th+Annual+Meeting+o)
20. Liu, J., Shen, D., Zhang, Y., Dolan, B., Carin, L., and Chen, W. (2021). [What makes good in-context examples for gpt-3?](https://md.research.elamur.ai/free/en/papers/liu-2022-what-makes-good-context-examples/)
21. Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., and Neubig, G. (2023). [Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing](https://md.research.elamur.ai/free/en/papers/liu-2021-pre-train-prompt-predict-systematic/). ACM Computing Surveys, 55(9):1–35.
22. Liu, R., Wei, J., Gu, S. S., Wu, T.-Y., Vosoughi, S., Cui, C., Zhou, D., and Dai, A. M. (2022b). Mind’s eye: Grounded language model reasoning through simulation. arXiv preprint arXiv:2210.05359. [find](https://scholar.google.com/scholar?q=Liu%2C+R%2E%2C+Wei%2C+J%2E%2C+Gu%2C+S%2E+S%2E%2C+Wu%2C+T%2E%2DY%2E%2C+Vosoughi%2C+S%2E%2C+Cui%2C+C%2E%2C+Zhou%2C+D%2E%2C+and+Dai%2C+A%2E+M%2E+%282022b%29%2E+Mind%E2%80%99s+eye%3A+Grounded+language+model+reasoning+through+simulation%2E+arXiv+preprint+arXiv%3A2210%2E05359%2E)
23. Lu, Y., Bartolo, M., Moore, A., Riedel, S., and Stenetorp, P. (2022). [Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity](https://md.research.elamur.ai/free/en/papers/lu-2022-fantastically-ordered-prompts-where-find/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8086–809…
24. Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., Welleck, S., Majumder, B. P., Gupta, S., Yazdanbakhsh, A., and Clark, P. (2023). Self-refine: Iterative refinement with self-feedback. [find](https://scholar.google.com/scholar?q=Madaan%2C+A%2E%2C+Tandon%2C+N%2E%2C+Gupta%2C+P%2E%2C+Hallinan%2C+S%2E%2C+Gao%2C+L%2E%2C+Wiegreffe%2C+S%2E%2C+Alon%2C+U%2E%2C+Dziri%2C+N%2E%2C+Prabhumoye%2C+S%2E%2C+Yang%2C+Y%2E%2C+Welleck%2C+S%2E%2C+Majumder%2C+B%2E+P%2E%2C+Gupta%2C+S%2E%2C+Yazdanbakhsh%2C+A%2E%2C+and+Clark%2C+P%2E+%282023%29%2E)
25. Mialon, G., Dessì, R., Lomeli, M., Nalmpantis, C., Pasunuru, R., Raileanu, R., Rozière, B., Schick, T., Dwivedi-Yu, J., Celikyilmaz, A., Grave, E., LeCun, Y., and Scialom, T. (2023). [Augmented language models: a survey](https://md.research.elamur.ai/free/en/papers/mialon-2023-augmented-language-models-survey/).
26. Miao, Y. and Blunsom, P. (2016a). [Language as a latent variable: Discrete generative models for sentence compression](https://md.research.elamur.ai/free/en/papers/miao-2016-language-as-latent-variable-discrete/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 319–328, Austin, Texas. Association for Computational Linguistics.
27. Miao, Y. and Blunsom, P. (2016b). [Language as a latent variable: Discrete generative models for sentence compression](https://md.research.elamur.ai/free/en/papers/miao-2016-language-as-latent-variable-discrete/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 319–328.
28. Min, S., Lyu, X., Holtzman, A., Artetxe, M., Lewis, M., Hajishirzi, H., and Zettlemoyer, L. (2022). [Rethinking the role of demonstrations: What makes in-context learning work?](https://md.research.elamur.ai/free/en/papers/min-2022-rethinking-role-demonstrations-what-makes/) In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11048–11064, Abu Dhabi, Uni…
29. Mukherjee, S. and Awadallah, A. H. (2020). Xtremedistil: Multi-stage distillation for massive multilingual models. In Jurafsky, D., Chai, J., Schluter, N., and Tetreault, J. R., editors, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5… [find](https://scholar.google.com/scholar?q=Mukherjee%2C+S%2E+and+Awadallah%2C+A%2E+H%2E+%282020%29%2E+Xtremedistil%3A+Multi%2Dstage+distillation+for+massive+multilingual+models%2E+In+Jurafsky%2C+D%2E%2C+Chai%2C+J%2E%2C+Schluter%2C+N%2E%2C+and+Tetreault%2C+J%2E+R%2E%2C+editors%2C+Proceedings+o)
30. Nye, M., Andreassen, A. J., Gur-Ari, G., Michalewski, H., Austin, J., Bieber, D., Dohan, D., Lewkowycz, A., Bosma, M., Luan, D., et al. (2021). [Show your work: Scratchpads for intermediate computation with language models](https://md.research.elamur.ai/free/en/papers/al-2021-show-your-work-scratchpads-intermediate/). arXiv preprint arXiv:2112.00114.
31. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022). [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in Neural Information Processing Systems, 35:27730–27744.
32. Prasad, A., Hase, P., Zhou, X., and Bansal, M. (2022). [Grips: Gradient-free, edit-based instruction search for prompting large language models](https://md.research.elamur.ai/free/en/papers/prasad-2023-grips-gradient-free-edit-based/). arXiv preprint arXiv:2203.07281.
33. Press, O., Zhang, M., Min, S., Schmidt, L., Smith, N. A., and Lewis, M. (2022). Measuring and narrowing the compositionality gap in language models. arXiv preprint arXiv:2210.03350. [find](https://scholar.google.com/scholar?q=Press%2C+O%2E%2C+Zhang%2C+M%2E%2C+Min%2C+S%2E%2C+Schmidt%2C+L%2E%2C+Smith%2C+N%2E+A%2E%2C+and+Lewis%2C+M%2E+%282022%29%2E+Measuring+and+narrowing+the+compositionality+gap+in+language+models%2E+arXiv+preprint+arXiv%3A2210%2E03350%2E)
34. Pryzant, R., Iter, D., Li, J., Lee, Y. T., Zhu, C., and Zeng, M. (2023). [Automatic prompt optimization with" gradient descent" and beam search](https://md.research.elamur.ai/free/en/papers/pryzant-2023-automatic-prompt-optimization-gradient-descent/). arXiv preprint arXiv:2305.03495.
35. Rubin, O., Herzig, J., and Berant, J. (2022). [Learning to retrieve prompts for in-context learning](https://md.research.elamur.ai/free/en/papers/rubin-2021-learning-retrieve-prompts-context-learning/). In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2655–2671.
36. Sanh, V., Debut, L., Chaumond, J., and Wolf, T. (2019). Distilbert, a distilled version of BERT: smaller, faster, cheaper and lighter. CoRR, abs/1910.01108. [find](https://scholar.google.com/scholar?q=Sanh%2C+V%2E%2C+Debut%2C+L%2E%2C+Chaumond%2C+J%2E%2C+and+Wolf%2C+T%2E+%282019%29%2E+Distilbert%2C+a+distilled+version+of+BERT%3A+smaller%2C+faster%2C+cheaper+and+lighter%2E+CoRR%2C+abs%2F1910%2E01108%2E)
37. Shin, T., Razeghi, Y., Logan IV, R. L., Wallace, E., and Singh, S. (2020). [Autoprompt: Eliciting knowledge from language models with automatically generated prompts](https://md.research.elamur.ai/free/en/papers/shin-2020-autoprompt-eliciting-knowledge-language-models/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4222–4235.
38. Shinn, N., Labash, B., and Gopinath, A. (2023). [Reflexion: an autonomous agent with dynamic memory and self-reflection](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/). arXiv preprint arXiv:2303.11366.
39. Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Garriga-Alonso, A., et al. (2022). [Beyond the imitation game: Quantifying and extrapolating the capabilities of language models](https://md.research.elamur.ai/free/en/papers/srivastava-2022-imitation-game-quantifying-extrapolating-capabilities/). arXiv preprint arXiv:2206.04615.
40. Su, H., Kasai, J., Wu, C. H., Shi, W., Wang, T., Xin, J., Zhang, R., Ostendorf, M., Zettlemoyer, L., Smith, N. A., et al. (2023). Selective annotation makes language models better few-shot learners. International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Su%2C+H%2E%2C+Kasai%2C+J%2E%2C+Wu%2C+C%2E+H%2E%2C+Shi%2C+W%2E%2C+Wang%2C+T%2E%2C+Xin%2C+J%2E%2C+Zhang%2C+R%2E%2C+Ostendorf%2C+M%2E%2C+Zettlemoyer%2C+L%2E%2C+Smith%2C+N%2E+A%2E%2C+et+al%2E+%282023%29%2E+Selective+annotation+makes+language+models+better+few%2Dshot+learners%2E+I)
41. Sun, Z., Wang, X., Tay, Y., Yang, Y., and Zhou, D. (2022). Recitation-augmented language models. arXiv preprint arXiv:2210.01296. [find](https://scholar.google.com/scholar?q=Sun%2C+Z%2E%2C+Wang%2C+X%2E%2C+Tay%2C+Y%2E%2C+Yang%2C+Y%2E%2C+and+Zhou%2C+D%2E+%282022%29%2E+Recitation%2Daugmented+language+models%2E+arXiv+preprint+arXiv%3A2210%2E01296%2E)
42. Suzgun, M., Scales, N., Schärli, N., Gehrmann, S., Tay, Y., Chung, H. W., Chowdhery, A., Le, Q. V., Chi, E. H., Zhou, D., , and Wei, J. (2022). [Challenging big-bench tasks and whether chain-of-thought can solve them](https://md.research.elamur.ai/free/en/papers/suzgun-2023-challenging-big-bench-tasks-whether/). arXiv preprint arXiv:2210.09261.
43. Tang, R., Lu, Y., Liu, L., Mou, L., Vechtomova, O., and Lin, J. (2019). Distilling task-specific knowledge from BERT into simple neural networks. CoRR, abs/1903.12136. [find](https://scholar.google.com/scholar?q=Tang%2C+R%2E%2C+Lu%2C+Y%2E%2C+Liu%2C+L%2E%2C+Mou%2C+L%2E%2C+Vechtomova%2C+O%2E%2C+and+Lin%2C+J%2E+%282019%29%2E+Distilling+task%2Dspecific+knowledge+from+BERT+into+simple+neural+networks%2E+CoRR%2C+abs%2F1903%2E12136%2E)
44. Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., Rodriguez, A., Joulin, A., Grave, E., and Lample, G. (2023a). [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). arXiv preprint arXiv:2302.13971.
45. Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., Bikel, D., Blecher, L., Ferrer, C. C., Chen, M., Cucurull, G., Esiobu, D., Fernandes, J., Fu, J., Fu, W., Fuller, B., Gao, C., Goswami, V., Goyal, N., Hartshorn, A., Hossei… — [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/)
46. Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., and Zhou, D. (2023a). [Self-consistency improves chain of thought reasoning in language models](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/). International Conference on Learning Representations.
47. Wang, X., Zhu, W., Saxon, M., Steyvers, M., and Wang, W. Y. (2023b). Large language models are implicitly topic models: Explaining and finding good demonstrations for in-context learning. [find](https://scholar.google.com/scholar?q=Wang%2C+X%2E%2C+Zhu%2C+W%2E%2C+Saxon%2C+M%2E%2C+Steyvers%2C+M%2E%2C+and+Wang%2C+W%2E+Y%2E+%282023b%29%2E+Large+language+models+are+implicitly+topic+models%3A+Explaining+and+finding+good+demonstrations+for+in%2Dcontext+learning%2E)
48. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E. H., Le, Q. V., and Zhou, D. (2022). [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). In NeurIPS.
49. Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., et al. (2022). Taxonomy of risks posed by language models. In 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 214–229. [find](https://scholar.google.com/scholar?q=Weidinger%2C+L%2E%2C+Uesato%2C+J%2E%2C+Rauh%2C+M%2E%2C+Griffin%2C+C%2E%2C+Huang%2C+P%2E%2DS%2E%2C+Mellor%2C+J%2E%2C+Glaese%2C+A%2E%2C+Cheng%2C+M%2E%2C+Balle%2C+B%2E%2C+Kasirzadeh%2C+A%2E%2C+et+al%2E+%282022%29%2E+Taxonomy+of+risks+posed+by+language+models%2E+In+2022+ACM+Con)
50. Wu, T., Terry, M., and Cai, C. J. (2022). Ai chains: Transparent and controllable human-ai interaction by chaining large language model prompts. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, pages 1–22. [find](https://scholar.google.com/scholar?q=Wu%2C+T%2E%2C+Terry%2C+M%2E%2C+and+Cai%2C+C%2E+J%2E+%282022%29%2E+Ai+chains%3A+Transparent+and+controllable+human%2Dai+interaction+by+chaining+large+language+model+prompts%2E+In+Proceedings+of+the+2022+CHI+Conference+on+Human+Fact)
51. Xu, C., Sun, Q., Zheng, K., Geng, X., Zhao, P., Feng, J., Tao, C., and Jiang, D. (2023a). Wizardlm: Empowering large language models to follow complex instructions. [find](https://scholar.google.com/scholar?q=Xu%2C+C%2E%2C+Sun%2C+Q%2E%2C+Zheng%2C+K%2E%2C+Geng%2C+X%2E%2C+Zhao%2C+P%2E%2C+Feng%2C+J%2E%2C+Tao%2C+C%2E%2C+and+Jiang%2C+D%2E+%282023a%29%2E+Wizardlm%3A+Empowering+large+language+models+to+follow+complex+instructions%2E)
52. Xu, W., Banburski-Fahey, A., and Jojic, N. (2023b). Reprompting: Automated chain-of-thought prompt inference through gibbs sampling. arXiv preprint arXiv:2305.09993. — [arXiv:2305.09993](https://arxiv.org/abs/2305.09993) [find](https://scholar.google.com/scholar?q=Xu%2C+W%2E%2C+Banburski%2DFahey%2C+A%2E%2C+and+Jojic%2C+N%2E+%282023b%29%2E+Reprompting%3A+Automated+chain%2Dof%2Dthought+prompt+inference+through+gibbs+sampling%2E+arXiv+preprint+arXiv%3A2305%2E09993%2E+%E2%80%94+%5BarXiv%3A2305%2E09993%5D%28https%3A%2F%2Farxiv)
53. Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T. L., Cao, Y., and Narasimhan, K. (2023a). Tree of thoughts: Deliberate problem solving with large language models. [find](https://scholar.google.com/scholar?q=Yao%2C+S%2E%2C+Yu%2C+D%2E%2C+Zhao%2C+J%2E%2C+Shafran%2C+I%2E%2C+Griffiths%2C+T%2E+L%2E%2C+Cao%2C+Y%2E%2C+and+Narasimhan%2C+K%2E+%282023a%29%2E+Tree+of+thoughts%3A+Deliberate+problem+solving+with+large+language+models%2E)
54. Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., and Cao, Y. (2023b). [React: Synergizing reasoning and acting in language models](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/). International Conference on Learning Representations.
55. Zhang, Z., Zhang, A., Li, M., and Smola, A. (2023). [Automatic chain of thought prompting in large language models](https://md.research.elamur.ai/free/en/papers/zhang-2022-automatic-chain-thought-prompting-large/). International Conference on Learning Representations.
56. Zhou, D., Schärli, N., Hou, L., Wei, J., Scales, N., Wang, X., Schuurmans, D., Bousquet, O., Le, Q., and Chi, E. (2023a). [Least-to-most prompting enables complex reasoning in large language models](https://md.research.elamur.ai/free/en/papers/al-2022-least-most-prompting-enables-complex/). International Conference on Learning Representations.
57. Zhou, Y., Muresanu, A. I., Han, Z., Paster, K., Pitis, S., Chan, H., and Ba, J. (2023b). [Large language models are human-level prompt engineers](https://md.research.elamur.ai/free/en/papers/zhou-2022-large-language-models-human-level/). International Conference on Learning Representations. Contents in Appendices:

*Updated 30 October 2026*