---
title: "Reflexion: Language Agents with Verbal — Shinn 2023"
description: "Agents built on large language models can act in games, compilers or APIs, but they struggle to improve by trial and error."
authors: ["Noah Shinn", "Federico Cassano", "Edward Berman", "Ashwin Gopinath", "Karthik Narasimhan", "Shunyu Yao"]
year: 2023
venue: "arXiv:2303.11366"
source: "https://arxiv.org/abs/2303.11366"
doi: "10.48550/arxiv.2303.11366"
arxiv: "2303.11366"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/) / Reflexion: Language Agents with Verbal Reinforcement Learning

# Reflexion: Language Agents with Verbal Reinforcement Learning

**Authors:** [Noah Shinn](https://md.research.elamur.ai/free/en/authors/noah-shinn/), Federico Cassano, Edward Berman, Ashwin Gopinath, [Karthik Narasimhan](https://md.research.elamur.ai/free/en/authors/karthik-narasimhan/), [Shunyu Yao](https://md.research.elamur.ai/free/en/authors/shunyu-yao/) · **Year:** 2023 · **Venue:** arXiv:2303.11366 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2303.11366) · [PDF](https://arxiv.org/pdf/2303.11366) · [DOI 10.48550/arxiv.2303.11366](https://doi.org/10.48550/arxiv.2303.11366) · [arXiv 2303.11366](https://arxiv.org/abs/2303.11366) · [OpenAlex](https://openalex.org/W7133222210)

**Citations:** 930 citations (OpenAlex, 2026-10-14) · [1272](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/cited-by/) in our library, of the 140884 papers whose bibliographies we read

## Summary
Agents built on large language models can act in games, compilers or APIs, but they struggle to improve by trial and error. Conventional reinforcement learning would need many samples and costly fine-tuning of very large models. Reflexion avoids updating weights entirely. After each attempt, a feedback signal is turned into a short written critique of what went wrong. That signal can be a binary reward, a hand-written heuristic, exact-match grading, or the agent's own unit tests. The critique is stored in an episodic memory and given back to the model as context on the next try. Three LLM roles share the work: an Actor (built on ReAct or Chain-of-Thought prompting), an Evaluator and a Self-Reflection model. The authors test the method on sequential decision-making (AlfWorld), knowledge-heavy question answering (HotPotQA) and code generation (HumanEval, MBPP and their new LeetcodeHardGym). It beats baselines without reflection on all three task types and reaches state-of-the-art pass@1 on HumanEval. This matters because it is a cheap and interpretable alternative to gradient-based RL: agents can learn from their mistakes in a few trials, and people can read the reflections they write.

## Key points
- On HumanEval (Python), Reflexion scores 91% pass@1, compared with 80% for plain GPT-4. Reported absolute gains are +22% on AlfWorld within 12 trials, +20% on HotPotQA and up to +11% on HumanEval.
- The paper introduces LeetcodeHardGym, a code-generation RL environment with 40 hard-level Leetcode problems in 19 programming languages. Code, demos and datasets are released at github.com/noahshinn024/reflexion.
- An ablation on the 50 hardest HumanEval Rust problems (GPT-4) gives 0.60 for the baseline, 0.52 without test generation, 0.60 without self-reflection and 0.68 for full Reflexion. Both components are needed.
- The quality of self-written tests limits the method. On MBPP Python, 16.3% of solutions that pass the internal tests are actually wrong, compared with 1.4% on HumanEval. This explains why Reflexion falls short of the GPT-4 baseline on MBPP.
- Limitations include possible convergence to local optima and memory that is only a sliding window of fixed size. Test-driven feedback is also hard to use for non-deterministic, impure or concurrent code.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Computing & GPUs](https://md.research.elamur.ai/free/en/shelves/computing/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{shinn2023reflexion,
  title = {{Reflexion: Language Agents with Verbal Reinforcement Learning}},
  author = {Shinn, Noah and Cassano, Federico and Berman, Edward and Gopinath, Ashwin and Narasimhan, Karthik and Yao, Shunyu},
  year = {2023},
  doi = {10.48550/arxiv.2303.11366},
  eprint = {2303.11366},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2303.11366}
}
```

- [← previous A Survey on Large Language Model based Autonomous Agents](https://md.research.elamur.ai/free/en/papers/wang-2023-survey-large-language-model-based/)
- [next → BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://md.research.elamur.ai/free/en/papers/li-2023-blip-2-bootstrapping-language-image/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/)

## References

31 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ahn, M., Brohan, A., Brown, N., Chebotar, Y., Cortes, O., David, B., Finn, C., Gopalakrishnan, K., Hausman, K., Herzog, A., et al. (2022). Do as i can, not as i say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.01691. [find](https://scholar.google.com/scholar?q=Ahn%2C+M%2E%2C+Brohan%2C+A%2E%2C+Brown%2C+N%2E%2C+Chebotar%2C+Y%2E%2C+Cortes%2C+O%2E%2C+David%2C+B%2E%2C+Finn%2C+C%2E%2C+Gopalakrishnan%2C+K%2E%2C+Hausman%2C+K%2E%2C+Herzog%2C+A%2E%2C+et+al%2E+%282022%29%2E+Do+as+i+can%2C+not+as+i+say%3A+Grounding+language+in+robotic+affo) [our card](https://md.research.elamur.ai/free/en/papers/ahn-2022-i-i-say-grounding-language/)
2. Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., et al. (2021). Program synthesis with large language models. arXiv preprint arXiv:2108.07732. [find](https://scholar.google.com/scholar?q=Austin%2C+J%2E%2C+Odena%2C+A%2E%2C+Nye%2C+M%2E%2C+Bosma%2C+M%2E%2C+Michalewski%2C+H%2E%2C+Dohan%2C+D%2E%2C+Jiang%2C+E%2E%2C+Cai%2C+C%2E%2C+Terry%2C+M%2E%2C+Le%2C+Q%2E%2C+et+al%2E+%282021%29%2E+Program+synthesis+with+large+language+models%2E+arXiv+preprint+arXiv%3A2108%2E077) [our card](https://md.research.elamur.ai/free/en/papers/austin-2021-program-synthesis-large-language-models/)
3. Brooks, E., Walls, L., Lewis, R. L., and Singh, S. (2022). In-context policy iteration. arXiv preprint arXiv:2210.03821. — [arXiv:2210.03821](https://arxiv.org/abs/2210.03821) [find](https://scholar.google.com/scholar?q=Brooks%2C+E%2E%2C+Walls%2C+L%2E%2C+Lewis%2C+R%2E+L%2E%2C+and+Singh%2C+S%2E+%282022%29%2E+In%2Dcontext+policy+iteration%2E+arXiv+preprint+arXiv%3A2210%2E03821%2E+%E2%80%94+%5BarXiv%3A2210%2E03821%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2210%2E03821%29)
4. Cassano, F., Gouwar, J., Nguyen, D., Nguyen, S., Phipps-Costin, L., Pinckney, D., Yee, M.-H., Zi, Y., Anderson, C. J., Feldman, M. Q., Guha, A., Greenberg, M., and Jangda, A. (2022). Multipl-e: A scalable and extensible approach to benchmarking neural code generation. [find](https://scholar.google.com/scholar?q=Cassano%2C+F%2E%2C+Gouwar%2C+J%2E%2C+Nguyen%2C+D%2E%2C+Nguyen%2C+S%2E%2C+Phipps%2DCostin%2C+L%2E%2C+Pinckney%2C+D%2E%2C+Yee%2C+M%2E%2DH%2E%2C+Zi%2C+Y%2E%2C+Anderson%2C+C%2E+J%2E%2C+Feldman%2C+M%2E+Q%2E%2C+Guha%2C+A%2E%2C+Greenberg%2C+M%2E%2C+and+Jangda%2C+A%2E+%282022%29%2E+Multipl%2De%3A+A+scal)
5. Chen, B., Zhang, F., Nguyen, A., Zan, D., Lin, Z., Lou, J.-G., and Chen, W. (2022). Codet: Code generation with generated tests. arXiv preprint arXiv:2207.10397. [find](https://scholar.google.com/scholar?q=Chen%2C+B%2E%2C+Zhang%2C+F%2E%2C+Nguyen%2C+A%2E%2C+Zan%2C+D%2E%2C+Lin%2C+Z%2E%2C+Lou%2C+J%2E%2DG%2E%2C+and+Chen%2C+W%2E+%282022%29%2E+Codet%3A+Code+generation+with+generated+tests%2E+arXiv+preprint+arXiv%3A2207%2E10397%2E)
6. Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. d. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. (2021). Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374. [find](https://scholar.google.com/scholar?q=Chen%2C+M%2E%2C+Tworek%2C+J%2E%2C+Jun%2C+H%2E%2C+Yuan%2C+Q%2E%2C+Pinto%2C+H%2E+P%2E+d%2E+O%2E%2C+Kaplan%2C+J%2E%2C+Edwards%2C+H%2E%2C+Burda%2C+Y%2E%2C+Joseph%2C+N%2E%2C+Brockman%2C+G%2E%2C+et+al%2E+%282021%29%2E+Evaluating+large+language+models+trained+on+code%2E+arXiv+prepri) [our card](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
7. Chen, X., Lin, M., Schärli, N., and Zhou, D. (2023). Teaching large language models to self-debug. arXiv preprint arXiv:2304.05128. [find](https://scholar.google.com/scholar?q=Chen%2C+X%2E%2C+Lin%2C+M%2E%2C+Sch%C3%A4rli%2C+N%2E%2C+and+Zhou%2C+D%2E+%282023%29%2E+Teaching+large+language+models+to+self%2Ddebug%2E+arXiv+preprint+arXiv%3A2304%2E05128%2E)
8. Côté, M.-A., Kádár, A., Yuan, X., Kybartas, B., Barnes, T., Fine, E., Moore, J., Hausknecht, M., El Asri, L., Adada, M., et al. (2019). Textworld: A learning environment for text-based games. In Computer Games: 7th Workshop, CGW 2018, Held in Conjunction with the 27th International Conference on Art… [find](https://scholar.google.com/scholar?q=C%C3%B4t%C3%A9%2C+M%2E%2DA%2E%2C+K%C3%A1d%C3%A1r%2C+A%2E%2C+Yuan%2C+X%2E%2C+Kybartas%2C+B%2E%2C+Barnes%2C+T%2E%2C+Fine%2C+E%2E%2C+Moore%2C+J%2E%2C+Hausknecht%2C+M%2E%2C+El+Asri%2C+L%2E%2C+Adada%2C+M%2E%2C+et+al%2E+%282019%29%2E+Textworld%3A+A+learning+environment+for+text%2Dbased+games%2E+In+Compu) [our card](https://md.research.elamur.ai/free/en/papers/cote-2018-textworld-learning-environment-text-based/)
9. Goodman, N. (2023). Meta-prompt: A simple self-improving language agent. noahgood-man.substack.com. [find](https://scholar.google.com/scholar?q=Goodman%2C+N%2E+%282023%29%2E+Meta%2Dprompt%3A+A+simple+self%2Dimproving+language+agent%2E+noahgood%2Dman%2Esubstack%2Ecom%2E)
10. Kim, G., Baldi, P., and McAleer, S. (2023). Language models can solve computer tasks. arXiv preprint arXiv:2303.17491. [find](https://scholar.google.com/scholar?q=Kim%2C+G%2E%2C+Baldi%2C+P%2E%2C+and+McAleer%2C+S%2E+%282023%29%2E+Language+models+can+solve+computer+tasks%2E+arXiv+preprint+arXiv%3A2303%2E17491%2E) [our card](https://md.research.elamur.ai/free/en/papers/kim-2023-language-models-solve-computer-tasks/)
11. Lam, W., Winter, S., Wei, A., Xie, T., Marinov, D., and Bell, J. (2020). A large-scale longitudinal study of flaky tests. Proc. ACM Program. Lang., 4(OOPSLA). [find](https://scholar.google.com/scholar?q=Lam%2C+W%2E%2C+Winter%2C+S%2E%2C+Wei%2C+A%2E%2C+Xie%2C+T%2E%2C+Marinov%2C+D%2E%2C+and+Bell%2C+J%2E+%282020%29%2E+A+large%2Dscale+longitudinal+study+of+flaky+tests%2E+Proc%2E+ACM+Program%2E+Lang%2E%2C+4%28OOPSLA%29%2E)
12. Le, H., Wang, Y., Gotmare, A. D., Savarese, S., and Hoi, S. C. H. (2022). Coderl: Mastering code generation through pretrained models and deep reinforcement learning. Advances in Neural Information Processing Systems, 35:21314–21328. [find](https://scholar.google.com/scholar?q=Le%2C+H%2E%2C+Wang%2C+Y%2E%2C+Gotmare%2C+A%2E+D%2E%2C+Savarese%2C+S%2E%2C+and+Hoi%2C+S%2E+C%2E+H%2E+%282022%29%2E+Coderl%3A+Mastering+code+generation+through+pretrained+models+and+deep+reinforcement+learning%2E+Advances+in+Neural+Information+Pr) [our card](https://md.research.elamur.ai/free/en/papers/le-2022-coderl-mastering-code-generation-through/)
13. Li, R., Allal, L. B., Zi, Y., Muennighoff, N., Kocetkov, D., Mou, C., Marone, M., Akiki, C., Li, J., Chim, J., et al. (2023). Starcoder: may the source be with you! arXiv preprint arXiv:2305.06161. [find](https://scholar.google.com/scholar?q=Li%2C+R%2E%2C+Allal%2C+L%2E+B%2E%2C+Zi%2C+Y%2E%2C+Muennighoff%2C+N%2E%2C+Kocetkov%2C+D%2E%2C+Mou%2C+C%2E%2C+Marone%2C+M%2E%2C+Akiki%2C+C%2E%2C+Li%2C+J%2E%2C+Chim%2C+J%2E%2C+et+al%2E+%282023%29%2E+Starcoder%3A+may+the+source+be+with+you%21+arXiv+preprint+arXiv%3A2305%2E06161%2E)
14. Li, Y., Choi, D., Chung, J., Kushman, N., Schrittwieser, J., Leblond, R., Eccles, T., Keeling, J., Gimeno, F., Dal Lago, A., et al. (2022). Competition-level code generation with alphacode. Science, 378(6624):1092–1097. [find](https://scholar.google.com/scholar?q=Li%2C+Y%2E%2C+Choi%2C+D%2E%2C+Chung%2C+J%2E%2C+Kushman%2C+N%2E%2C+Schrittwieser%2C+J%2E%2C+Leblond%2C+R%2E%2C+Eccles%2C+T%2E%2C+Keeling%2C+J%2E%2C+Gimeno%2C+F%2E%2C+Dal+Lago%2C+A%2E%2C+et+al%2E+%282022%29%2E+Competition%2Dlevel+code+generation+with+alphacode%2E+Science%2C+3) [our card](https://md.research.elamur.ai/free/en/papers/li-2022-competition-level-code-generation-alphacode/)
15. Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., et al. (2023). Self-refine: Iterative refinement with self-feedback. arXiv preprint arXiv:2303.17651. [find](https://scholar.google.com/scholar?q=Madaan%2C+A%2E%2C+Tandon%2C+N%2E%2C+Gupta%2C+P%2E%2C+Hallinan%2C+S%2E%2C+Gao%2C+L%2E%2C+Wiegreffe%2C+S%2E%2C+Alon%2C+U%2E%2C+Dziri%2C+N%2E%2C+Prabhumoye%2C+S%2E%2C+Yang%2C+Y%2E%2C+et+al%2E+%282023%29%2E+Self%2Drefine%3A+Iterative+refinement+with+self%2Dfeedback%2E+arXiv+prepr)
16. Nair, V., Schumacher, E., Tso, G., and Kannan, A. (2023). Dera: Enhancing large language model completions with dialog-enabled resolving agents. arXiv preprint arXiv:2303.17071. [find](https://scholar.google.com/scholar?q=Nair%2C+V%2E%2C+Schumacher%2C+E%2E%2C+Tso%2C+G%2E%2C+and+Kannan%2C+A%2E+%282023%29%2E+Dera%3A+Enhancing+large+language+model+completions+with+dialog%2Denabled+resolving+agents%2E+arXiv+preprint+arXiv%3A2303%2E17071%2E)
17. Nakano, R., Hilton, J., Balaji, S., Wu, J., Ouyang, L., Kim, C., Hesse, C., Jain, S., Kosaraju, V., Saunders, W., et al. (2021). Webgpt: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332. [find](https://scholar.google.com/scholar?q=Nakano%2C+R%2E%2C+Hilton%2C+J%2E%2C+Balaji%2C+S%2E%2C+Wu%2C+J%2E%2C+Ouyang%2C+L%2E%2C+Kim%2C+C%2E%2C+Hesse%2C+C%2E%2C+Jain%2C+S%2E%2C+Kosaraju%2C+V%2E%2C+Saunders%2C+W%2E%2C+et+al%2E+%282021%29%2E+Webgpt%3A+Browser%2Dassisted+question%2Danswering+with+human+feedback%2E+arXiv+) [our card](https://md.research.elamur.ai/free/en/papers/nakano-2021-webgpt-browser-assisted-question-answering/)
18. OpenAI (2023). Gpt-4 technical report. ArXiv. [find](https://scholar.google.com/scholar?q=OpenAI+%282023%29%2E+Gpt%2D4+technical+report%2E+ArXiv%2E) [our card](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/)
19. Park, J. S., O’Brien, J. C., Cai, C. J., Morris, M. R., Liang, P., and Bernstein, M. S. (2023). Generative agents: Interactive simulacra of human behavior. arXiv preprint arXiv:2304.03442. [find](https://scholar.google.com/scholar?q=Park%2C+J%2E+S%2E%2C+O%E2%80%99Brien%2C+J%2E+C%2E%2C+Cai%2C+C%2E+J%2E%2C+Morris%2C+M%2E+R%2E%2C+Liang%2C+P%2E%2C+and+Bernstein%2C+M%2E+S%2E+%282023%29%2E+Generative+agents%3A+Interactive+simulacra+of+human+behavior%2E+arXiv+preprint+arXiv%3A2304%2E03442%2E)
20. Paul, D., Ismayilzada, M., Peyrard, M., Borges, B., Bosselut, A., West, R., and Faltings, B. (2023). Refiner: Reasoning feedback on intermediate representations. arXiv preprint arXiv:2304.01904. [find](https://scholar.google.com/scholar?q=Paul%2C+D%2E%2C+Ismayilzada%2C+M%2E%2C+Peyrard%2C+M%2E%2C+Borges%2C+B%2E%2C+Bosselut%2C+A%2E%2C+West%2C+R%2E%2C+and+Faltings%2C+B%2E+%282023%29%2E+Refiner%3A+Reasoning+feedback+on+intermediate+representations%2E+arXiv+preprint+arXiv%3A2304%2E01904%2E) [our card](https://md.research.elamur.ai/free/en/papers/paul-2023-refiner-reasoning-feedback-intermediate-representations/)
21. Pryzant, R., Iter, D., Li, J., Lee, Y. T., Zhu, C., and Zeng, M. (2023). Automatic prompt optimization with" gradient descent" and beam search. arXiv preprint arXiv:2305.03495. [find](https://scholar.google.com/scholar?q=Pryzant%2C+R%2E%2C+Iter%2C+D%2E%2C+Li%2C+J%2E%2C+Lee%2C+Y%2E+T%2E%2C+Zhu%2C+C%2E%2C+and+Zeng%2C+M%2E+%282023%29%2E+Automatic+prompt+optimization+with%22+gradient+descent%22+and+beam+search%2E+arXiv+preprint+arXiv%3A2305%2E03495%2E) [our card](https://md.research.elamur.ai/free/en/papers/pryzant-2023-automatic-prompt-optimization-gradient-descent/)
22. Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Zettlemoyer, L., Cancedda, N., and Scialom, T. (2023). Toolformer: Language models can teach themselves to use tools. arXiv preprint arXiv:2302.04761. [find](https://scholar.google.com/scholar?q=Schick%2C+T%2E%2C+Dwivedi%2DYu%2C+J%2E%2C+Dess%C3%AC%2C+R%2E%2C+Raileanu%2C+R%2E%2C+Lomeli%2C+M%2E%2C+Zettlemoyer%2C+L%2E%2C+Cancedda%2C+N%2E%2C+and+Scialom%2C+T%2E+%282023%29%2E+Toolformer%3A+Language+models+can+teach+themselves+to+use+tools%2E+arXiv+preprint+ar) [our card](https://md.research.elamur.ai/free/en/papers/schick-2023-toolformer-language-models-can-teach/)
23. Shen, Y., Song, K., Tan, X., Li, D., Lu, W., and Zhuang, Y. (2023). Hugginggpt: Solving ai tasks with chatgpt and its friends in huggingface. arXiv preprint arXiv:2303.17580. [find](https://scholar.google.com/scholar?q=Shen%2C+Y%2E%2C+Song%2C+K%2E%2C+Tan%2C+X%2E%2C+Li%2C+D%2E%2C+Lu%2C+W%2E%2C+and+Zhuang%2C+Y%2E+%282023%29%2E+Hugginggpt%3A+Solving+ai+tasks+with+chatgpt+and+its+friends+in+huggingface%2E+arXiv+preprint+arXiv%3A2303%2E17580%2E) [our card](https://md.research.elamur.ai/free/en/papers/shen-2023-hugginggpt-solving-ai-tasks-chatgpt/)
24. Shridhar, M., Yuan, X., Côté, M.-A., Bisk, Y., Trischler, A., and Hausknecht, M. (2021). ALFWorld: Aligning Text and Embodied Environments for Interactive Learning. In Proceedings of the International Conference on Learning Representations (ICLR). [find](https://scholar.google.com/scholar?q=Shridhar%2C+M%2E%2C+Yuan%2C+X%2E%2C+C%C3%B4t%C3%A9%2C+M%2E%2DA%2E%2C+Bisk%2C+Y%2E%2C+Trischler%2C+A%2E%2C+and+Hausknecht%2C+M%2E+%282021%29%2E+ALFWorld%3A+Aligning+Text+and+Embodied+Environments+for+Interactive+Learning%2E+In+Proceedings+of+the+International) [our card](https://md.research.elamur.ai/free/en/papers/shridhar-2020-alfworld-aligning-text-embodied-environments/)
25. Sutton, R. S. and Barto, A. G. (2018). Reinforcement Learning: An Introduction. The MIT Press, second edition. [find](https://scholar.google.com/scholar?q=Sutton%2C+R%2E+S%2E+and+Barto%2C+A%2E+G%2E+%282018%29%2E+Reinforcement+Learning%3A+An+Introduction%2E+The+MIT+Press%2C+second+edition%2E)
26. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Chi, E., Le, Q., and Zhou, D. (2022). Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903. [find](https://scholar.google.com/scholar?q=Wei%2C+J%2E%2C+Wang%2C+X%2E%2C+Schuurmans%2C+D%2E%2C+Bosma%2C+M%2E%2C+Chi%2C+E%2E%2C+Le%2C+Q%2E%2C+and+Zhou%2C+D%2E+%282022%29%2E+Chain+of+thought+prompting+elicits+reasoning+in+large+language+models%2E+arXiv+preprint+arXiv%3A2201%2E11903%2E) [our card](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/)
27. Xie, Y., Kawaguchi, K., Zhao, Y., Zhao, X., Kan, M.-Y., He, J., and Xie, Q. (2023). Decomposition enhances reasoning via self-evaluation guided decoding. arXiv preprint arXiv:2305.00633. [find](https://scholar.google.com/scholar?q=Xie%2C+Y%2E%2C+Kawaguchi%2C+K%2E%2C+Zhao%2C+Y%2E%2C+Zhao%2C+X%2E%2C+Kan%2C+M%2E%2DY%2E%2C+He%2C+J%2E%2C+and+Xie%2C+Q%2E+%282023%29%2E+Decomposition+enhances+reasoning+via+self%2Devaluation+guided+decoding%2E+arXiv+preprint+arXiv%3A2305%2E00633%2E) [our card](https://md.research.elamur.ai/free/en/papers/xie-2023-self-evaluation-guided-beam-search/)
28. Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W. W., Salakhutdinov, R., and Manning, C. D. (2018). HotpotQA: A dataset for diverse, explainable multi-hop question answering. In Conference on Empirical Methods in Natural Language Processing (EMNLP). [find](https://scholar.google.com/scholar?q=Yang%2C+Z%2E%2C+Qi%2C+P%2E%2C+Zhang%2C+S%2E%2C+Bengio%2C+Y%2E%2C+Cohen%2C+W%2E+W%2E%2C+Salakhutdinov%2C+R%2E%2C+and+Manning%2C+C%2E+D%2E+%282018%29%2E+HotpotQA%3A+A+dataset+for+diverse%2C+explainable+multi%2Dhop+question+answering%2E+In+Conference+on+Empiric) [our card](https://md.research.elamur.ai/free/en/papers/yang-2018-hotpotqa-dataset-diverse-explainable-multi/)
29. Yao, S., Chen, H., Yang, J., and Narasimhan, K. (preprint). Webshop: Towards scalable real-world web interaction with grounded language agents. In ArXiv. [find](https://scholar.google.com/scholar?q=Yao%2C+S%2E%2C+Chen%2C+H%2E%2C+Yang%2C+J%2E%2C+and+Narasimhan%2C+K%2E+%28preprint%29%2E+Webshop%3A+Towards+scalable+real%2Dworld+web+interaction+with+grounded+language+agents%2E+In+ArXiv%2E) [our card](https://md.research.elamur.ai/free/en/papers/yao-2022-webshop-towards-scalable-real-world/)
30. Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., and Cao, Y. (2023). ReAct: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR). [find](https://scholar.google.com/scholar?q=Yao%2C+S%2E%2C+Zhao%2C+J%2E%2C+Yu%2C+D%2E%2C+Du%2C+N%2E%2C+Shafran%2C+I%2E%2C+Narasimhan%2C+K%2E%2C+and+Cao%2C+Y%2E+%282023%29%2E+ReAct%3A+Synergizing+reasoning+and+acting+in+language+models%2E+In+International+Conference+on+Learning+Representations+) [our card](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/)
31. Yoran, O., Wolfson, T., Bogin, B., Katz, U., Deutch, D., and Berant, J. (2023). Answering questions by meta-reasoning over multiple chains of thought. arXiv preprint arXiv:2304.13007. [find](https://scholar.google.com/scholar?q=Yoran%2C+O%2E%2C+Wolfson%2C+T%2E%2C+Bogin%2C+B%2E%2C+Katz%2C+U%2E%2C+Deutch%2C+D%2E%2C+and+Berant%2C+J%2E+%282023%29%2E+Answering+questions+by+meta%2Dreasoning+over+multiple+chains+of+thought%2E+arXiv+preprint+arXiv%3A2304%2E13007%2E)