---
title: "DSPy: Compiling Declarative Language Model — Khattab 2023"
description: "Pipelines that chain language model calls usually depend on long prompt templates tuned by hand."
authors: ["Omar Khattab", "Arnav Singhvi", "Paridhi Maheshwari", "Zhiyuan Zhang", "Keshav Santhanam", "Sri Vardhamanan", "Saiful Haq", "Ashutosh Sharma", "Thomas T. Joshi", "Hanna Moazam", "Heather Miller", "Matei Zaharia", "Christopher Potts"]
year: 2023
venue: "arXiv:2310.03714"
source: "https://arxiv.org/abs/2310.03714"
doi: "10.48550/arxiv.2310.03714"
arxiv: "2310.03714"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/khattab-2023-dspy-compiling-declarative-language-model/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/khattab-2023-dspy-compiling-declarative-language-model/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/3/) / DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines

# DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines

**Authors:** [Omar Khattab](https://md.research.elamur.ai/free/en/authors/omar-khattab/), Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, [Keshav Santhanam](https://md.research.elamur.ai/free/en/authors/keshav-santhanam/), Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, [Matei Zaharia](https://md.research.elamur.ai/free/en/authors/matei-zaharia/) et al. (13 authors) · **Year:** 2023 · **Venue:** arXiv:2310.03714 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2310.03714) · [PDF](https://arxiv.org/pdf/2310.03714) · [DOI 10.48550/arxiv.2310.03714](https://doi.org/10.48550/arxiv.2310.03714) · [arXiv 2310.03714](https://arxiv.org/abs/2310.03714) · [OpenAlex](https://openalex.org/W4387430850)

**Citations:** 59 citations (openalex-snapshot · 2026-10-14) · [202](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Pipelines that chain language model calls usually depend on long prompt templates tuned by hand. These prompts are fragile and often fail to transfer across models, domains or inputs. The authors present DSPy, a Python programming model that borrows ideas from neural network frameworks such as PyTorch. Developers write pipelines as graphs of modules, and each module carries a natural-language signature that states what the step should produce rather than how to prompt for it. Techniques like Chain of Thought and ReAct are turned into reusable, learnable modules. A compiler then tunes the whole program against a user-supplied metric. It does this with "teleprompters," optimizers that run the pipeline on a small training set, keep traces that succeed as demonstrations, and use them to build few-shot prompts or to finetune smaller models. The authors test this on GSM8K math problems and HotPotQA multi-hop question answering. Compiled DSPy programs beat both standard few-shot prompting and expert-written demonstrations. The work matters because it replaces manual prompt crafting with systematic optimization, which also lets small open models perform well.

## Key points
- DSPy is built on three abstractions: signatures (typed input/output declarations), modules (composable replacements for hand-written prompting techniques) and teleprompters (optimizers such as BootstrapFewShot that maximize a metric).
- With GPT-3.5, compiled programs improved from 33% to 82% on GSM8K and from 32% to 46% on HotPotQA. With llama2-13b-chat, they improved from 9% to 47% and from 22% to 41%.
- Compared with standard few-shot prompting, the self-bootstrapped pipelines generally gained more than 25% for GPT-3.5 and more than 65% for llama2-13b-chat. Compared with expert-created demonstrations, the gains were up to 5–46% and 16–40%.
- Programs compiled for small open models, including T5-Large (770M parameters) and llama2-13b-chat, rival expert-written prompt chains running on proprietary GPT-3.5. Compilation takes minutes to tens of minutes.
- DSPy is the successor to the Demonstrate–Search–Predict (DSP) framework and is released at github.com/stanfordnlp/dspy.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Other](https://md.research.elamur.ai/free/en/shelves/other/)

## Cited by

Cited by

202 papers of our library cite it (of the 140673 whose bibliographies we read); 8 of them have a page here, 194 not yet.

- [Quiet-STaR: Language models can teach themselves to think before speaking](https://md.research.elamur.ai/free/en/papers/al-2024-quiet-star-language-models-teach/)
- [The Prompt Report: A Systematic Survey of Prompt Engineering Techniques](https://md.research.elamur.ai/free/en/papers/schulhoff-2024-prompt-report-systematic-survey-prompt/)
- [Assisting in Writing Wikipedia-like Articles From Scratch with Large Language Models](https://md.research.elamur.ai/free/en/papers/shao-2024-assisting-writing-wikipedia-like-articles/)
- [Integrating TEI, NER/NEL, Textometry, and Linked Data for a Semantically Enriched Interview Corpus](https://md.research.elamur.ai/free/en/papers/stankovic-2026-integrating-tei-ner-nel-textometry/)
- [To Write or to Automate Linguistic Prompts, That Is the Question](https://md.research.elamur.ai/free/en/papers/torron-2026-write-or-automate-linguistic-prompts/)
- [OpenHands: An Open Platform for AI Software Developers as Generalist Agents](https://md.research.elamur.ai/free/en/papers/wang-2024-openhands-open-platform-ai/)
- [LLM-as-a-Jury for Machine Translation Publishability Assessment](https://md.research.elamur.ai/free/en/papers/yanishevsky-2026-llm-as-jury-machine-translation/)
- [SGLang: Efficient Execution of Structured Language Model Programs](https://md.research.elamur.ai/free/en/papers/zheng-2023-sglang-efficient-execution-structured-language/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{khattab2023dspy,
  title = {{DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines}},
  author = {Khattab, Omar and Singhvi, Arnav and Maheshwari, Paridhi and Zhang, Zhiyuan and Santhanam, Keshav and Vardhamanan, Sri and Haq, Saiful and Sharma, Ashutosh and Joshi, Thomas T. and Moazam, Hanna and Miller, Heather and Zaharia, Matei and Potts, Christopher},
  year = {2023},
  doi = {10.48550/arxiv.2310.03714},
  eprint = {2310.03714},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2310.03714}
}
```

- [← previous Dotless Representation of Arabic Text: Analysis and Modeling](https://md.research.elamur.ai/free/en/papers/al-2023-shaibani-dotless-representation-arabic-text-analysis/)
- [next → Dual Cache for Long Document Neural Coreference Resolution](https://md.research.elamur.ai/free/en/papers/guo-2023-dual-cache-long-document-neural/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/3/)

## References

67 references, 30 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 2623–2631, 2019. [find](https://scholar.google.com/scholar?q=Takuya+Akiba%2C+Shotaro+Sano%2C+Toshihiko+Yanase%2C+Takeru+Ohta%2C+and+Masanori+Koyama%2E+Optuna%3A+A+next%2Dgeneration+hyperparameter+optimization+framework%2E+In+Proceedings+of+the+25th+ACM+SIGKDD+international+con) [our card](https://md.research.elamur.ai/free/en/papers/akiba-2019-optuna-next-generation-hyperparameter-optimization/)
2. Rami Al-Rfou, Guillaume Alain, Amjad Almahairi, Christof Angermueller, Dzmitry Bahdanau, Nicolas Ballas, Frédéric Bastien, Justin Bayer, Anatoly Belikov, Alexander Belopolsky, et al. Theano: A Python framework for fast computation of mathematical expressions. arXiv e-prints, pp. arXiv–1605, 2016. [find](https://scholar.google.com/scholar?q=Rami+Al%2DRfou%2C+Guillaume+Alain%2C+Amjad+Almahairi%2C+Christof+Angermueller%2C+Dzmitry+Bahdanau%2C+Nicolas+Ballas%2C+Fre%CC%81de%CC%81ric+Bastien%2C+Justin+Bayer%2C+Anatoly+Belikov%2C+Alexander+Belopolsky%2C+et+al%2E+Theano%3A+A+Pytho) [our card](https://md.research.elamur.ai/free/en/papers/the-2016-theano-development-team-theano-python-framework-fast-computation/)
3. James Bergstra, Olivier Breuleux, Frédéric Bastien, Pascal Lamblin, Razvan Pascanu, Guillaume Desjardins, Joseph Turian, David Warde-Farley, and Yoshua Bengio. Theano: A CPU and GPU math compiler in Python. In Proc. 9th python in science conf, volume 1, pp. 3–10, 2010. [find](https://scholar.google.com/scholar?q=James+Bergstra%2C+Olivier+Breuleux%2C+Fre%CC%81de%CC%81ric+Bastien%2C+Pascal+Lamblin%2C+Razvan+Pascanu%2C+Guillaume+Desjardins%2C+Joseph+Turian%2C+David+Warde%2DFarley%2C+and+Yoshua+Bengio%2E+Theano%3A+A+CPU+and+GPU+math+compiler+in)
4. James Bergstra, Frédéric Bastien, Olivier Breuleux, Pascal Lamblin, Razvan Pascanu, Olivier Delalleau, Guillaume Desjardins, David Warde-Farley, Ian Goodfellow, Arnaud Bergeron, et al. Theano: Deep learning on gpus with Python. In NIPS 2011, BigLearning Workshop, Granada, Spain, volume 3. [find](https://scholar.google.com/scholar?q=James+Bergstra%2C+Fre%CC%81de%CC%81ric+Bastien%2C+Olivier+Breuleux%2C+Pascal+Lamblin%2C+Razvan+Pascanu%2C+Olivier+Delalleau%2C+Guillaume+Desjardins%2C+David+Warde%2DFarley%2C+Ian+Goodfellow%2C+Arnaud+Bergeron%2C+et+al%2E+Theano%3A+Deep+)
5. James Bergstra, Daniel Yamins, and David Cox. Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision architectures. In International conference on machine learning, pp. 115–123. PMLR, 2013. [find](https://scholar.google.com/scholar?q=James+Bergstra%2C+Daniel+Yamins%2C+and+David+Cox%2E+Making+a+science+of+model+search%3A+Hyperparameter+optimization+in+hundreds+of+dimensions+for+vision+architectures%2E+In+International+conference+on+machine+l)
6. Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021. [find](https://scholar.google.com/scholar?q=Rishi+Bommasani%2C+Drew+A+Hudson%2C+Ehsan+Adeli%2C+Russ+Altman%2C+Simran+Arora%2C+Sydney+von+Arx%2C+Michael+S+Bernstein%2C+Jeannette+Bohg%2C+Antoine+Bosselut%2C+Emma+Brunskill%2C+et+al%2E+On+the+opportunities+and+risks+of+) [our card](https://md.research.elamur.ai/free/en/papers/bommasani-2021-opportunities-risks-foundation-models/)
7. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020. [find](https://scholar.google.com/scholar?q=Tom+Brown%2C+Benjamin+Mann%2C+Nick+Ryder%2C+Melanie+Subbiah%2C+Jared+D+Kaplan%2C+Prafulla+Dhariwal%2C+Arvind+Neelakantan%2C+Pranav+Shyam%2C+Girish+Sastry%2C+Amanda+Askell%2C+et+al%2E+Language+models+are+few%2Dshot+learners%2E+)
8. Harrison Chase. langchain. Hwchase17/langchain. Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. Reading Wikipedia to answer open-domain questions. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1870–1879, Vancouver, C… [find](https://scholar.google.com/scholar?q=Harrison+Chase%2E+langchain%2E+Hwchase17%2Flangchain%2E+Danqi+Chen%2C+Adam+Fisch%2C+Jason+Weston%2C+and+Antoine+Bordes%2E+Reading+Wikipedia+to+answer+open%2Ddomain+questions%2E+In+Proceedings+of+the+55th+Annual+Meeting+o) [our card](https://md.research.elamur.ai/free/en/papers/chen-2017-reading-wikipedia-answer-open-domain/)
9. Lingjiao Chen, Matei Zaharia, and James Zou. Frugalgpt: How to use large language models while reducing cost and improving performance. arXiv preprint arXiv:2305.05176, 2023. [https://gist.github.com/g8a9/07c2be12ae02cfad4aa430d77dc940cb](https://gist.github.com/g8a9/07c2be12ae02cfad4aa430d77dc940cb) [↗](https://gist.github.com/g8a9/07c2be12ae02cfad4aa430d77dc940cb) [our card](https://md.research.elamur.ai/free/en/papers/chen-2023-frugalgpt-use-large-language-models/)
10. URL [https://github.com/hwchase17/2022](https://github.com/hwchase17/2022). Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W Cohen. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. arXiv preprint arXiv:2211.12588, 2022. [↗](https://github.com/hwchase17/2022)
11. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311, 2022. [find](https://scholar.google.com/scholar?q=Aakanksha+Chowdhery%2C+Sharan+Narang%2C+Jacob+Devlin%2C+Maarten+Bosma%2C+Gaurav+Mishra%2C+Adam+Roberts%2C+Paul+Barham%2C+Hyung+Won+Chung%2C+Charles+Sutton%2C+Sebastian+Gehrmann%2C+et+al%2E+Palm%3A+Scaling+language+modeling+w)
12. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021. [find](https://scholar.google.com/scholar?q=Karl+Cobbe%2C+Vineet+Kosaraju%2C+Mohammad+Bavarian%2C+Mark+Chen%2C+Heewoo+Jun%2C+Lukasz+Kaiser%2C+Matthias+Plappert%2C+Jerry+Tworek%2C+Jacob+Hilton%2C+Reiichiro+Nakano%2C+et+al%2E+Training+verifiers+to+solve+math+word+prob) [our card](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/)
13. Ronan Collobert, Samy Bengio, and Johnny Mariéthoz. Torch: a modular machine learning software library. Technical report, Idiap, 2002. [find](https://scholar.google.com/scholar?q=Ronan+Collobert%2C+Samy+Bengio%2C+and+Johnny+Marie%CC%81thoz%2E+Torch%3A+a+modular+machine+learning+software+library%2E+Technical+report%2C+Idiap%2C+2002%2E)
14. David Dohan, Winnie Xu, Aitor Lewkowycz, Jacob Austin, David Bieber, Raphael Gontijo Lopes, Yuhuai Wu, Henryk Michalewski, Rif A Saurous, Jascha Sohl-Dickstein, et al. Language model cascades. arXiv preprint arXiv:2207.10342, 2022. [find](https://scholar.google.com/scholar?q=David+Dohan%2C+Winnie+Xu%2C+Aitor+Lewkowycz%2C+Jacob+Austin%2C+David+Bieber%2C+Raphael+Gontijo+Lopes%2C+Yuhuai+Wu%2C+Henryk+Michalewski%2C+Rif+A+Saurous%2C+Jascha+Sohl%2DDickstein%2C+et+al%2E+Language+model+cascades%2E+arXiv+p)
15. Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, et al. Rarr: Researching and revising what language models say, using language models. In Proceedings of the 61st Annual Meeting of the Association for Computat… [find](https://scholar.google.com/scholar?q=Luyu+Gao%2C+Zhuyun+Dai%2C+Panupong+Pasupat%2C+Anthony+Chen%2C+Arun+Tejasvi+Chaganty%2C+Yicheng+Fan%2C+Vincent+Zhao%2C+Ni+Lao%2C+Hongrae+Lee%2C+Da%2DCheng+Juan%2C+et+al%2E+Rarr%3A+Researching+and+revising+what+language+models+s) [our card](https://md.research.elamur.ai/free/en/papers/gao-2023-rarr-researching-revising-what-language/)
16. Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. Pal: Program-aided language models. In International Conference on Machine Learning, pp. 10764–10799. PMLR, 2023b. [find](https://scholar.google.com/scholar?q=Luyu+Gao%2C+Aman+Madaan%2C+Shuyan+Zhou%2C+Uri+Alon%2C+Pengfei+Liu%2C+Yiming+Yang%2C+Jamie+Callan%2C+and+Graham+Neubig%2E+Pal%3A+Program%2Daided+language+models%2E+In+International+Conference+on+Machine+Learning%2C+pp%2E+10764%E2%80%93)
17. Qingyan Guo, Rui Wang, Junliang Guo, Bei Li, Kaitao Song, Xu Tan, Guoqing Liu, Jiang Bian, and Yujiu Yang. Connecting large language models with evolutionary algorithms yields powerful prompt optimizers. arXiv preprint arXiv:2309.08532, 2023. [find](https://scholar.google.com/scholar?q=Qingyan+Guo%2C+Rui+Wang%2C+Junliang+Guo%2C+Bei+Li%2C+Kaitao+Song%2C+Xu+Tan%2C+Guoqing+Liu%2C+Jiang+Bian%2C+and+Yujiu+Yang%2E+Connecting+large+language+models+with+evolutionary+algorithms+yields+powerful+prompt+optimize)
18. Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. Realm: Retrieval-augmented language model pre-training. arXiv preprint arXiv:2002.08909, 2020. URL https://arxiv.org/abs/2002.08909. [↗](https://arxiv.org/abs/2002.08909)
19. Braden Hancock, Paroma Varma, Stephanie Wang, Martin Bringmann, Percy Liang, and Christopher Ré. Training classifiers with natural language explanations. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1884– 1895. Association f… [find](https://scholar.google.com/scholar?q=Braden+Hancock%2C+Paroma+Varma%2C+Stephanie+Wang%2C+Martin+Bringmann%2C+Percy+Liang%2C+and+Christopher+Re%CC%81%2E+Training+classifiers+with+natural+language+explanations%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+)
20. Bin Hu, Chenyang Zhao, Pu Zhang, Zihao Zhou, Yuanhang Yang, Zenglin Xu, and Bin Liu. Enabling intelligent interactions between an agent and an LLM: A reinforcement learning approach. arXiv preprint arXiv:2306.03604, 2023. URL https://arxiv.org/abs/2306.03604. — [arXiv:2306.03604](https://arxiv.org/abs/2306.03604) [↗](https://arxiv.org/abs/2306.03604)
21. Jiaxin Huang, Shixiang Shane Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. Large language models can self-improve. arXiv preprint arXiv:2210.11610, 2022. [find](https://scholar.google.com/scholar?q=Jiaxin+Huang%2C+Shixiang+Shane+Gu%2C+Le+Hou%2C+Yuexin+Wu%2C+Xuezhi+Wang%2C+Hongkun+Yu%2C+and+Jiawei+Han%2E+Large+language+models+can+self%2Dimprove%2E+arXiv+preprint+arXiv%3A2210%2E11610%2C+2022%2E) [our card](https://md.research.elamur.ai/free/en/papers/huang-2023-large-language-models-can-self/)
22. Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. Few-shot learning with retrieval augmented language models. arXiv preprint arXiv:2208.03299, 2022. [find](https://scholar.google.com/scholar?q=Gautier+Izacard%2C+Patrick+Lewis%2C+Maria+Lomeli%2C+Lucas+Hosseini%2C+Fabio+Petroni%2C+Timo+Schick%2C+Jane+Dwivedi%2DYu%2C+Armand+Joulin%2C+Sebastian+Riedel%2C+and+Edouard+Grave%2E+Few%2Dshot+learning+with+retrieval+augmente)
23. Ehud Karpas, Omri Abend, Yonatan Belinkov, Barak Lenz, Opher Lieber, Nir Ratner, Yoav Shoham, Hofit Bata, Yoav Levine, Kevin Leyton-Brown, et al. Mrkl systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning. arXiv preprin… — [arXiv:2205.00445](https://arxiv.org/abs/2205.00445) [find](https://scholar.google.com/scholar?q=Ehud+Karpas%2C+Omri+Abend%2C+Yonatan+Belinkov%2C+Barak+Lenz%2C+Opher+Lieber%2C+Nir+Ratner%2C+Yoav+Shoham%2C+Hofit+Bata%2C+Yoav+Levine%2C+Kevin+Leyton%2DBrown%2C+et+al%2E+Mrkl+systems%3A+A+modular%2C+neuro%2Dsymbolic+architecture+t)
24. Omar Khattab, Christopher Potts, and Matei Zaharia. Baleen: Robust Multi-Hop Reasoning at Scale via Condensed Retrieval. In Thirty-Fifth Conference on Neural Information Processing Systems, 2021a. [find](https://scholar.google.com/scholar?q=Omar+Khattab%2C+Christopher+Potts%2C+and+Matei+Zaharia%2E+Baleen%3A+Robust+Multi%2DHop+Reasoning+at+Scale+via+Condensed+Retrieval%2E+In+Thirty%2DFifth+Conference+on+Neural+Information+Processing+Systems%2C+2021a%2E)
25. Omar Khattab, Christopher Potts, and Matei Zaharia. Relevance-guided supervision for openqa with ColBERT. Transactions of the Association for Computational Linguistics, 9:929–944, 2021b. [find](https://scholar.google.com/scholar?q=Omar+Khattab%2C+Christopher+Potts%2C+and+Matei+Zaharia%2E+Relevance%2Dguided+supervision+for+openqa+with+ColBERT%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+9%3A929%E2%80%93944%2C+2021b%2E)
26. Omar Khattab, Keshav Santhanam, Xiang Lisa Li, David Hall, Percy Liang, Christopher Potts, and Matei Zaharia. Demonstrate-search-predict: Composing retrieval and language models for knowledge-intensive nlp. arXiv preprint arXiv:2212.14024, 2022. [find](https://scholar.google.com/scholar?q=Omar+Khattab%2C+Keshav+Santhanam%2C+Xiang+Lisa+Li%2C+David+Hall%2C+Percy+Liang%2C+Christopher+Potts%2C+and+Matei+Zaharia%2E+Demonstrate%2Dsearch%2Dpredict%3A+Composing+retrieval+and+language+models+for+knowledge%2Dintensiv) [our card](https://md.research.elamur.ai/free/en/papers/khattab-2022-demonstrate-search-predict-composing-retrieval/)
27. Tushar Khot, Harsh Trivedi, Matthew Finlayson, Yao Fu, Kyle Richardson, Peter Clark, and Ashish Sabharwal. Decomposed prompting: A modular approach for solving complex tasks. arXiv preprint arXiv:2210.02406, 2022. [find](https://scholar.google.com/scholar?q=Tushar+Khot%2C+Harsh+Trivedi%2C+Matthew+Finlayson%2C+Yao+Fu%2C+Kyle+Richardson%2C+Peter+Clark%2C+and+Ashish+Sabharwal%2E+Decomposed+prompting%3A+A+modular+approach+for+solving+complex+tasks%2E+arXiv+preprint+arXiv%3A2210)
28. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. arXiv preprint arXiv:2205.11916, 2022. [find](https://scholar.google.com/scholar?q=Takeshi+Kojima%2C+Shixiang+Shane+Gu%2C+Machel+Reid%2C+Yutaka+Matsuo%2C+and+Yusuke+Iwasawa%2E+Large+language+models+are+zero%2Dshot+reasoners%2E+arXiv+preprint+arXiv%3A2205%2E11916%2C+2022%2E) [our card](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/)
29. Angeliki Lazaridou, Elena Gribovskaya, Wojciech Stokowiec, and Nikolai Grigorev. Internet-augmented language models through few-shot prompting for open-domain question answering. arXiv preprint arXiv:2203.05115, 2022. [find](https://scholar.google.com/scholar?q=Angeliki+Lazaridou%2C+Elena+Gribovskaya%2C+Wojciech+Stokowiec%2C+and+Nikolai+Grigorev%2E+Internet%2Daugmented+language+models+through+few%2Dshot+prompting+for+open%2Ddomain+question+answering%2E+arXiv+preprint+arXiv%3A) [our card](https://md.research.elamur.ai/free/en/papers/lazaridou-2022-internet-augmented-language-models-through/)
30. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive nlp tasks. In H. Larochelle, M. Ranzato, R. Hadsell,… [find](https://scholar.google.com/scholar?q=Patrick+Lewis%2C+Ethan+Perez%2C+Aleksandra+Piktus%2C+Fabio+Petroni%2C+Vladimir+Karpukhin%2C+Naman+Goyal%2C+Heinrich+Ku%CC%88ttler%2C+Mike+Lewis%2C+Wen%2Dtau+Yih%2C+Tim+Rockta%CC%88schel%2C+Sebastian+Riedel%2C+and+Douwe+Kiela%2E+Retrieva)
31. Jerry Liu. LlamaIndex, 11 2022. URL [https://github.com/jerryjliu/llama](https://github.com/jerryjliu/llama) index. [↗](https://github.com/jerryjliu/llama)
32. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback. arXiv preprint arXiv:2303.17651, 2023. [find](https://scholar.google.com/scholar?q=Aman+Madaan%2C+Niket+Tandon%2C+Prakhar+Gupta%2C+Skyler+Hallinan%2C+Luyu+Gao%2C+Sarah+Wiegreffe%2C+Uri+Alon%2C+Nouha+Dziri%2C+Shrimai+Prabhumoye%2C+Yiming+Yang%2C+et+al%2E+Self%2Drefine%3A+Iterative+refinement+with+self%2Dfeedbac)
33. Bryan McCann, Nitish Shirish Keskar, Caiming Xiong, and Richard Socher. The natural language decathlon: Multitask learning as question answering. arXiv:1806.08730, 2018. URL https://arxiv.org/abs/1806.08730. [↗](https://arxiv.org/abs/1806.08730) [our card](https://md.research.elamur.ai/free/en/papers/mccann-2018-natural-language-decathlon-multitask-learning/)
34. Microsoft. Semantic kernel. 2023. URL [https://learn.microsoft.com/semantic-kernel/](https://learn.microsoft.com/semantic-kernel/). [↗](https://learn.microsoft.com/semantic-kernel/)
35. Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, and John Schulman. WebGPT: Browser-assisted qu… [find](https://scholar.google.com/scholar?q=Reiichiro+Nakano%2C+Jacob+Hilton%2C+Suchir+Balaji%2C+Jeff+Wu%2C+Long+Ouyang%2C+Christina+Kim%2C+Christopher+Hesse%2C+Shantanu+Jain%2C+Vineet+Kosaraju%2C+William+Saunders%2C+Xu+Jiang%2C+Karl+Cobbe%2C+Tyna+Eloundou%2C+Gretchen+K) [our card](https://md.research.elamur.ai/free/en/papers/nakano-2021-webgpt-browser-assisted-question-answering/)
36. OpenAI. Gpt-4 technical report, 2023. [find](https://scholar.google.com/scholar?q=OpenAI%2E+Gpt%2D4+technical+report%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/)
37. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02155, 2022. [find](https://scholar.google.com/scholar?q=Long+Ouyang%2C+Jeff+Wu%2C+Xu+Jiang%2C+Diogo+Almeida%2C+Carroll+L+Wainwright%2C+Pamela+Mishkin%2C+Chong+Zhang%2C+Sandhini+Agarwal%2C+Katarina+Slama%2C+Alex+Ray%2C+et+al%2E+Training+language+models+to+follow+instructions+wit)
38. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and S… [find](https://scholar.google.com/scholar?q=Adam+Paszke%2C+Sam+Gross%2C+Francisco+Massa%2C+Adam+Lerer%2C+James+Bradbury%2C+Gregory+Chanan%2C+Trevor+Killeen%2C+Zeming+Lin%2C+Natalia+Gimelshein%2C+Luca+Antiga%2C+Alban+Desmaison%2C+Andreas+Kopf%2C+Edward+Yang%2C+Zachary+De) [our card](https://md.research.elamur.ai/free/en/papers/paszke-2019-pytorch-imperative-style-high-performance/)
39. Mohammadreza Pourreza and Davood Rafiei. Din-sql: Decomposed in-context learning of text-to-sql with self-correction. arXiv preprint arXiv:2304.11015, 2023. [find](https://scholar.google.com/scholar?q=Mohammadreza+Pourreza+and+Davood+Rafiei%2E+Din%2Dsql%3A+Decomposed+in%2Dcontext+learning+of+text%2Dto%2Dsql+with+self%2Dcorrection%2E+arXiv+preprint+arXiv%3A2304%2E11015%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/pourreza-2023-din-sql-decomposed-context-learning/)
40. Ofir Press, Muru Zhang, Sewon Min, Ludwig Schmidt, Noah A Smith, and Mike Lewis. Measuring and narrowing the compositionality gap in language models. arXiv preprint arXiv:2210.03350, 2022. [find](https://scholar.google.com/scholar?q=Ofir+Press%2C+Muru+Zhang%2C+Sewon+Min%2C+Ludwig+Schmidt%2C+Noah+A+Smith%2C+and+Mike+Lewis%2E+Measuring+and+narrowing+the+compositionality+gap+in+language+models%2E+arXiv+preprint+arXiv%3A2210%2E03350%2C+2022%2E)
41. Reid Pryzant, Dan Iter, Jerry Li, Yin Tat Lee, Chenguang Zhu, and Michael Zeng. Automatic prompt optimization with” gradient descent” and beam search. arXiv preprint arXiv:2305.03495, 2023. [find](https://scholar.google.com/scholar?q=Reid+Pryzant%2C+Dan+Iter%2C+Jerry+Li%2C+Yin+Tat+Lee%2C+Chenguang+Zhu%2C+and+Michael+Zeng%2E+Automatic+prompt+optimization+with%E2%80%9D+gradient+descent%E2%80%9D+and+beam+search%2E+arXiv+preprint+arXiv%3A2305%2E03495%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/pryzant-2023-automatic-prompt-optimization-gradient-descent/)
42. Peng Qi, Xiaowen Lin, Leo Mehr, Zijian Wang, and Christopher D. Manning. Answering complex open-domain questions through iterative query generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Languag… [find](https://scholar.google.com/scholar?q=Peng+Qi%2C+Xiaowen+Lin%2C+Leo+Mehr%2C+Zijian+Wang%2C+and+Christopher+D%2E+Manning%2E+Answering+complex+open%2Ddomain+questions+through+iterative+query+generation%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+)
43. Peng Qi, Haejun Lee, Oghenetegiri Sido, Christopher D Manning, et al. Retrieve, rerank, read, then iterate: Answering open-domain questions of arbitrary complexity from text. arXiv preprint arXiv:2010.12527, 2020. URL https://arxiv.org/abs/2010.12527. — [arXiv:2010.12527](https://arxiv.org/abs/2010.12527) [↗](https://arxiv.org/abs/2010.12527)
44. Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. Improving language understanding by generative pre-training. Ms, OpenAI, 2018. URL [https://openai.com/blog/language-unsupervised/](https://openai.com/blog/language-unsupervised/). [↗](https://openai.com/blog/language-unsupervised/)
45. Alexander J Ratner, Christopher M De Sa, Sen Wu, Daniel Selsam, and Christopher Ré. Data programming: Creating large training sets, quickly. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett (eds.), Advances in Neural Information Processing Systems 29, pp. 3567–3575. Curran Associa… [find](https://scholar.google.com/scholar?q=Alexander+J+Ratner%2C+Christopher+M+De+Sa%2C+Sen+Wu%2C+Daniel+Selsam%2C+and+Christopher+Re%CC%81%2E+Data+programming%3A+Creating+large+training+sets%2C+quickly%2E+In+D%2E+D%2E+Lee%2C+M%2E+Sugiyama%2C+U%2E+V%2E+Luxburg%2C+I%2E+Guyon%2C+and+R%2E)
46. Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, and Matei Zaharia. Col-BERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction. arXiv preprint arXiv:2112.01488, 2021. [find](https://scholar.google.com/scholar?q=Keshav+Santhanam%2C+Omar+Khattab%2C+Jon+Saad%2DFalcon%2C+Christopher+Potts%2C+and+Matei+Zaharia%2E+Col%2DBERTv2%3A+Effective+and+Efficient+Retrieval+via+Lightweight+Late+Interaction%2E+arXiv+preprint+arXiv%3A2112%2E01488%2C+) [our card](https://md.research.elamur.ai/free/en/papers/santhanam-2022-colbertv2-effective-efficient-retrieval-lightweight/)
47. Timo Schick, Jane Dwivedi-Yu, Roberto Dessı̀, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. arXiv preprint arXiv:2302.04761, 2023. [find](https://scholar.google.com/scholar?q=Timo+Schick%2C+Jane+Dwivedi%2DYu%2C+Roberto+Dess%C4%B1%CC%80%2C+Roberta+Raileanu%2C+Maria+Lomeli%2C+Luke+Zettlemoyer%2C+Nicola+Cancedda%2C+and+Thomas+Scialom%2E+Toolformer%3A+Language+models+can+teach+themselves+to+use+tools%2E+arXi) [our card](https://md.research.elamur.ai/free/en/papers/schick-2023-toolformer-language-models-can-teach/)
48. Zhihong Shao, Yeyun Gong, Yelong Shen, Minlie Huang, Nan Duan, and Weizhu Chen. Synthetic prompting: Generating chain-of-thought demonstrations for large language models. arXiv preprint arXiv:2302.00618, 2023. — [arXiv:2302.00618](https://arxiv.org/abs/2302.00618) [find](https://scholar.google.com/scholar?q=Zhihong+Shao%2C+Yeyun+Gong%2C+Yelong+Shen%2C+Minlie+Huang%2C+Nan+Duan%2C+and+Weizhu+Chen%2E+Synthetic+prompting%3A+Generating+chain%2Dof%2Dthought+demonstrations+for+large+language+models%2E+arXiv+preprint+arXiv%3A2302%2E006)
49. Noah Shinn, Beck Labash, and Ashwin Gopinath. Reflexion: an autonomous agent with dynamic memory and self-reflection. arXiv preprint arXiv:2303.11366, 2023. [find](https://scholar.google.com/scholar?q=Noah+Shinn%2C+Beck+Labash%2C+and+Ashwin+Gopinath%2E+Reflexion%3A+an+autonomous+agent+with+dynamic+memory+and+self%2Dreflection%2E+arXiv+preprint+arXiv%3A2303%2E11366%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/)
50. Chenglei Si, Zhe Gan, Zhengyuan Yang, Shuohang Wang, Jianfeng Wang, Jordan Boyd-Graber, and Lijuan Wang. Prompting gpt-3 to be reliable. arXiv preprint arXiv:2210.09150, 2022. [find](https://scholar.google.com/scholar?q=Chenglei+Si%2C+Zhe+Gan%2C+Zhengyuan+Yang%2C+Shuohang+Wang%2C+Jianfeng+Wang%2C+Jordan+Boyd%2DGraber%2C+and+Lijuan+Wang%2E+Prompting+gpt%2D3+to+be+reliable%2E+arXiv+preprint+arXiv%3A2210%2E09150%2C+2022%2E) [our card](https://md.research.elamur.ai/free/en/papers/si-2022-prompting-gpt-3-reliable/)
51. Zhiqing Sun, Xuezhi Wang, Yi Tay, Yiming Yang, and Denny Zhou. Recitation-augmented language models. arXiv preprint arXiv:2210.01296, 2022. — [arXiv:2210.01296](https://arxiv.org/abs/2210.01296) [find](https://scholar.google.com/scholar?q=Zhiqing+Sun%2C+Xuezhi+Wang%2C+Yi+Tay%2C+Yiming+Yang%2C+and+Denny+Zhou%2E+Recitation%2Daugmented+language+models%2E+arXiv+preprint+arXiv%3A2210%2E01296%2C+2022%2E+%E2%80%94+%5BarXiv%3A2210%2E01296%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2210%2E01296%29)
52. Seiya Tokui, Kenta Oono, Shohei Hido, and Justin Clayton. Chainer: a next-generation open source framework for deep learning. In Proceedings of workshop on machine learning systems (LearningSys) in the twenty-ninth annual conference on neural information processing systems (NIPS), volume 5, pp. 1–6,… [find](https://scholar.google.com/scholar?q=Seiya+Tokui%2C+Kenta+Oono%2C+Shohei+Hido%2C+and+Justin+Clayton%2E+Chainer%3A+a+next%2Dgeneration+open+source+framework+for+deep+learning%2E+In+Proceedings+of+workshop+on+machine+learning+systems+%28LearningSys%29+in+th)
53. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288, 2023. [find](https://scholar.google.com/scholar?q=Hugo+Touvron%2C+Louis+Martin%2C+Kevin+Stone%2C+Peter+Albert%2C+Amjad+Almahairi%2C+Yasmine+Babaei%2C+Nikolay+Bashlykov%2C+Soumya+Batra%2C+Prajjwal+Bhargava%2C+Shruti+Bhosale%2C+et+al%2E+Llama+2%3A+Open+foundation+and+fine%2Dtun)
54. Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. arXiv preprint arXiv:2212.10509, 2022. [find](https://scholar.google.com/scholar?q=Harsh+Trivedi%2C+Niranjan+Balasubramanian%2C+Tushar+Khot%2C+and+Ashish+Sabharwal%2E+Interleaving+retrieval+with+chain%2Dof%2Dthought+reasoning+for+knowledge%2Dintensive+multi%2Dstep+questions%2E+arXiv+preprint+arXiv%3A22) [our card](https://md.research.elamur.ai/free/en/papers/trivedi-2022-interleaving-retrieval-chain-thought-reasoning/)
55. Fei Wang, James Decker, Xilun Wu, Gregory Essertel, and Tiark Rompf. Backpropagation with callbacks: Foundations for efficient and expressive differentiable programming. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett (eds.), Advances in Neural Information Proces… [find](https://scholar.google.com/scholar?q=Fei+Wang%2C+James+Decker%2C+Xilun+Wu%2C+Gregory+Essertel%2C+and+Tiark+Rompf%2E+Backpropagation+with+callbacks%3A+Foundations+for+efficient+and+expressive+differentiable+programming%2E+In+S%2E+Bengio%2C+H%2E+Wallach%2C+H%2E+L)
56. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, and Denny Zhou. Rationale-augmented ensembles in language models. arXiv preprint arXiv:2207.00747, 2022a. [find](https://scholar.google.com/scholar?q=Xuezhi+Wang%2C+Jason+Wei%2C+Dale+Schuurmans%2C+Quoc+Le%2C+Ed+Chi%2C+and+Denny+Zhou%2E+Rationale%2Daugmented+ensembles+in+language+models%2E+arXiv+preprint+arXiv%3A2207%2E00747%2C+2022a%2E)
57. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171, 2022b. [find](https://scholar.google.com/scholar?q=Xuezhi+Wang%2C+Jason+Wei%2C+Dale+Schuurmans%2C+Quoc+Le%2C+Ed+Chi%2C+and+Denny+Zhou%2E+Self%2Dconsistency+improves+chain+of+thought+reasoning+in+language+models%2E+arXiv+preprint+arXiv%3A2203%2E11171%2C+2022b%2E) [our card](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/)
58. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903, 2022. [find](https://scholar.google.com/scholar?q=Jason+Wei%2C+Xuezhi+Wang%2C+Dale+Schuurmans%2C+Maarten+Bosma%2C+Ed+Chi%2C+Quoc+Le%2C+and+Denny+Zhou%2E+Chain+of+thought+prompting+elicits+reasoning+in+large+language+models%2E+arXiv+preprint+arXiv%3A2201%2E11903%2C+2022%2E) [our card](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/)
59. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoe… [find](https://scholar.google.com/scholar?q=Thomas+Wolf%2C+Lysandre+Debut%2C+Victor+Sanh%2C+Julien+Chaumond%2C+Clement+Delangue%2C+Anthony+Moi%2C+Pierric+Cistac%2C+Tim+Rault%2C+Remi+Louf%2C+Morgan+Funtowicz%2C+Joe+Davison%2C+Sam+Shleifer%2C+Patrick+von+Platen%2C+Clara+M) [our card](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/)
60. Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V Le, Denny Zhou, and Xinyun Chen. Large language models as optimizers. arXiv preprint arXiv:2309.03409, 2023. [find](https://scholar.google.com/scholar?q=Chengrun+Yang%2C+Xuezhi+Wang%2C+Yifeng+Lu%2C+Hanxiao+Liu%2C+Quoc+V+Le%2C+Denny+Zhou%2C+and+Xinyun+Chen%2E+Large+language+models+as+optimizers%2E+arXiv+preprint+arXiv%3A2309%2E03409%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/yang-2023-large-language-models-optimizers/)
61. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W Cohen, Ruslan Salakhutdinov, and Christopher D Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. arXiv preprint arXiv:1809.09600, 2018. [find](https://scholar.google.com/scholar?q=Zhilin+Yang%2C+Peng+Qi%2C+Saizheng+Zhang%2C+Yoshua+Bengio%2C+William+W+Cohen%2C+Ruslan+Salakhutdinov%2C+and+Christopher+D+Manning%2E+Hotpotqa%3A+A+dataset+for+diverse%2C+explainable+multi%2Dhop+question+answering%2E+arXiv+) [our card](https://md.research.elamur.ai/free/en/papers/yang-2018-hotpotqa-dataset-diverse-explainable-multi/)
62. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629, 2022. [find](https://scholar.google.com/scholar?q=Shunyu+Yao%2C+Jeffrey+Zhao%2C+Dian+Yu%2C+Nan+Du%2C+Izhak+Shafran%2C+Karthik+Narasimhan%2C+and+Yuan+Cao%2E+React%3A+Synergizing+reasoning+and+acting+in+language+models%2E+arXiv+preprint+arXiv%3A2210%2E03629%2C+2022%2E) [our card](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/)
63. Ori Yoran, Tomer Wolfson, Ben Bogin, Uri Katz, Daniel Deutch, and Jonathan Berant. Answering questions by meta-reasoning over multiple chains of thought. arXiv preprint arXiv:2304.13007, 2023. [find](https://scholar.google.com/scholar?q=Ori+Yoran%2C+Tomer+Wolfson%2C+Ben+Bogin%2C+Uri+Katz%2C+Daniel+Deutch%2C+and+Jonathan+Berant%2E+Answering+questions+by+meta%2Dreasoning+over+multiple+chains+of+thought%2E+arXiv+preprint+arXiv%3A2304%2E13007%2C+2023%2E)
64. Eric Zelikman, Yuhuai Wu, and Noah D Goodman. Star: Bootstrapping reasoning with reasoning. arXiv preprint arXiv:2203.14465, 2022. [find](https://scholar.google.com/scholar?q=Eric+Zelikman%2C+Yuhuai+Wu%2C+and+Noah+D+Goodman%2E+Star%3A+Bootstrapping+reasoning+with+reasoning%2E+arXiv+preprint+arXiv%3A2203%2E14465%2C+2022%2E) [our card](https://md.research.elamur.ai/free/en/papers/zelikman-2022-star-bootstrapping-reasoning-reasoning/)
65. Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. Automatic chain of thought prompting in large language models. arXiv preprint arXiv:2210.03493, 2022. [find](https://scholar.google.com/scholar?q=Zhuosheng+Zhang%2C+Aston+Zhang%2C+Mu+Li%2C+and+Alex+Smola%2E+Automatic+chain+of+thought+prompting+in+large+language+models%2E+arXiv+preprint+arXiv%3A2210%2E03493%2C+2022%2E) [our card](https://md.research.elamur.ai/free/en/papers/zhang-2022-automatic-chain-thought-prompting-large/)
66. Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. ExpeL: LLM agents are experiential learners. arXiv preprint arXiv:2308.10144, 2023a. URL [https://arxiv.org/pdf/2308.10144](https://arxiv.org/pdf/2308.10144). [↗](https://arxiv.org/pdf/2308.10144)
67. Xu Zhao, Yuxi Xie, Kenji Kawaguchi, Junxian He, and Qizhe Xie. Automatic model selection with large language models for reasoning. arXiv preprint arXiv:2305.14333, 2023b. [find](https://scholar.google.com/scholar?q=Xu+Zhao%2C+Yuxi+Xie%2C+Kenji+Kawaguchi%2C+Junxian+He%2C+and+Qizhe+Xie%2E+Automatic+model+selection+with+large+language+models+for+reasoning%2E+arXiv+preprint+arXiv%3A2305%2E14333%2C+2023b%2E)