---
title: "EVOLvE — paper by Nie et al. (2024) — summary and ideas"
description: "They release BanditBench, a set of natural-language multi-armed bandit (MAB) and contextual bandit… 49 references, cited by 8 papers here."
authors: ["Allen Nie", "Yi Su", "Bo Chang", "Jonathan N. Lee", "Ed H. Chi", "Quoc V. Le", "Minmin Chen"]
year: 2024
venue: "arXiv:2410.06238"
source: "https://arxiv.org/abs/2410.06238"
doi: "10.48550/arxiv.2410.06238"
arxiv: "2410.06238"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/nie-2024-evolve-evaluating-optimizing-llms-context/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/nie-2024-evolve-evaluating-optimizing-llms-context/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/14/) / EVOLvE: Evaluating and Optimizing LLMs For In-Context Exploration

# EVOLvE: Evaluating and Optimizing LLMs For In-Context Exploration

**Authors:** [Allen Nie](https://md.research.elamur.ai/free/en/authors/allen-nie/), [Yi Su](https://md.research.elamur.ai/free/en/authors/yi-su/), Bo Chang, Jonathan N. Lee, [Ed H. Chi](https://md.research.elamur.ai/free/en/authors/ed-h-chi/), [Quoc V. Le](https://md.research.elamur.ai/free/en/authors/quoc-viet-le/) and Minmin Chen · **Year:** 2024 · **Venue:** arXiv:2410.06238 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2410.06238) · [PDF](https://arxiv.org/pdf/2410.06238) · [DOI 10.48550/arxiv.2410.06238](https://doi.org/10.48550/arxiv.2410.06238) · [arXiv 2410.06238](https://arxiv.org/abs/2410.06238) · [OpenAlex](https://openalex.org/W4403344710)

**Citations:** 1 citations (OpenAlex, 2026-10-14) · [8](https://md.research.elamur.ai/free/en/papers/nie-2024-evolve-evaluating-optimizing-llms-context/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
LLMs are increasingly used as agents that must choose actions and learn from rewards. However, it is unclear whether they can trade off exploration against exploitation when they only observe feedback for their own choices. The authors study this in-context, with no weight updates, using bandit problems as a stateless form of reinforcement learning.

They release BanditBench, a set of natural-language multi-armed bandit (MAB) and contextual bandit (CB) environments. The environments vary the number of arms, reward distributions, difficulty and textual framing. They then test two ways of transferring knowledge from provably good algorithms such as UCB into LLMs:
- **Inference-time support:** the prompt includes summary statistics or algorithm-guided hints.
- **Algorithm distillation:** synthetic trajectories from optimal algorithms are used as few-shot demonstrations or for fine-tuning.

When given only raw interaction histories, LLMs explore poorly. Both enhancement strategies help, and fine-tuning lets smaller models outperform larger ones. A gap to classical algorithms still remains. The work offers a reproducible way to measure and improve LLM decision-making under uncertainty, which matters for applications such as recommendation and personalization.

## Key points
- **Benchmark:** BanditBench covers both MAB and CB tasks. It extends earlier work by Krishnamurthy et al. (2024) and can be installed via `pip install banditbench`. The code is at github.com/allenanie/EVOLvE.
- **Models evaluated:** Gemma-2B, Gemma-9B, Gemini-1.5 Flash and Gemini-1.5 Pro, compared against UCB as the classical reference.
- **History formats:** Raw History (RH) and Summarized History (SH) are compared. SH reports per-arm empirical means and pull counts. An algorithm-guided (AG) variant adds further support. The paper also formalizes when a textualization counts as "sufficient."
- **Fine-tuning result:** Oracle Behavior Fine-Tuning (OFT) brought Gemini-1.5 Flash close to UCB on contextual bandits, and the fine-tuned model generalized across domains.
- **Regret analysis:** The authors fit a parametric model to regret curves. Pro models with AG or SH support reach nearly sub-linear regret, and so do Flash models after OFT. Gemma models stay in the linear-regret regime under nearly all methods.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Stanford University](https://md.research.elamur.ai/free/en/universities/stanford-university/) — University, Stanford, United States · 567 works · 335,189 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🎯 Decision Sciences](https://md.research.elamur.ai/free/en/shelves/decision-sciences/) · 306 works · [🎯 Management Science and Operations Research](https://md.research.elamur.ai/free/en/shelves/decision-sciences/management-science-and-operations-research/) · 145 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 4,145 works

## Cite

Cite as: Allen Nie, Yi Su, Bo Chang, Jonathan N. Lee, Ed H. Chi, Quoc V. Le and Minmin Chen. 2024. EVOLvE: Evaluating and Optimizing LLMs For In-Context Exploration. arXiv:2410.06238.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{nie2024evolve,
  title = {{EVOLvE: Evaluating and Optimizing LLMs For In-Context Exploration}},
  author = {Allen Nie, Yi Su, Bo Chang, Jonathan N. Lee, Ed H. Chi, Quoc V. Le and Minmin Chen},
  year = {2024},
  doi = {10.48550/arxiv.2410.06238},
  eprint = {2410.06238},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2410.06238}
}
```

- [← previous EDDA: An Encoder-Decoder Data Augmentation Framework for Zero-Shot Stance Detection](https://md.research.elamur.ai/free/en/papers/ding-2024-edda-encoder-decoder-data-augmentation/)
- [next → From Human Judgements to Predictive Models: Unravelling Acceptability in Code-Mixed Sentences](https://md.research.elamur.ai/free/en/papers/kodali-2024-human-judgements-predictive-models-unravelling/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/14/)

## References

49 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. P. Auer. Finite-time analysis of the multiarmed bandit problem, 2002. [find](https://scholar.google.com/scholar?q=P%2E+Auer%2E+Finite%2Dtime+analysis+of+the+multiarmed+bandit+problem%2C+2002%2E)
2. C. Bai, Y. Zhang, S. Qiu, Q. Zhang, K. Xu, and X. Li. Online preference alignment for language models via count-based exploration. arXiv preprint arXiv:2501.12735, 2025. — [arXiv:2501.12735](https://arxiv.org/abs/2501.12735) [find](https://scholar.google.com/scholar?q=C%2E+Bai%2C+Y%2E+Zhang%2C+S%2E+Qiu%2C+Q%2E+Zhang%2C+K%2E+Xu%2C+and+X%2E+Li%2E+Online+preference+alignment+for+language+models+via+count%2Dbased+exploration%2E+arXiv+preprint+arXiv%3A2501%2E12735%2C+2025%2E+%E2%80%94+%5BarXiv%3A2501%2E12735%5D%28https%3A%2F%2Fa)
3. Y. Bai, F. Chen, H. Wang, C. Xiong, and S. Mei. Transformers as statisticians: Provable in-context learning with in-context algorithm selection. Advances in neural information processing systems, 36: 57125–57211, 2023. [find](https://scholar.google.com/scholar?q=Y%2E+Bai%2C+F%2E+Chen%2C+H%2E+Wang%2C+C%2E+Xiong%2C+and+S%2E+Mei%2E+Transformers+as+statisticians%3A+Provable+in%2Dcontext+learning+with+in%2Dcontext+algorithm+selection%2E+Advances+in+neural+information+processing+systems%2C+36%3A+)
4. E. Brooks, L. Walls, R. L. Lewis, and S. Singh. Large language models can implement policy iteration. Advances in Neural Information Processing Systems, 36, 2024. [find](https://scholar.google.com/scholar?q=E%2E+Brooks%2C+L%2E+Walls%2C+R%2E+L%2E+Lewis%2C+and+S%2E+Singh%2E+Large+language+models+can+implement+policy+iteration%2E+Advances+in+Neural+Information+Processing+Systems%2C+36%2C+2024%2E)
5. W. Chu, L. Li, L. Reyzin, and R. Schapire. Contextual bandits with linear payoff functions. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pages 208–214. JMLR Workshop and Conference Proceedings, 2011. [find](https://scholar.google.com/scholar?q=W%2E+Chu%2C+L%2E+Li%2C+L%2E+Reyzin%2C+and+R%2E+Schapire%2E+Contextual+bandits+with+linear+payoff+functions%2E+In+Proceedings+of+the+Fourteenth+International+Conference+on+Artificial+Intelligence+and+Statistics%2C+pages+2)
6. N. D. Daw, J. P. O’doherty, P. Dayan, B. Seymour, and R. J. Dolan. Cortical substrates for exploratory decisions in humans. Nature, 441(7095):876–879, 2006. [find](https://scholar.google.com/scholar?q=N%2E+D%2E+Daw%2C+J%2E+P%2E+O%E2%80%99doherty%2C+P%2E+Dayan%2C+B%2E+Seymour%2C+and+R%2E+J%2E+Dolan%2E+Cortical+substrates+for+exploratory+decisions+in+humans%2E+Nature%2C+441%287095%29%3A876%E2%80%93879%2C+2006%2E)
7. N. Felicioni, L. Maystre, S. Ghiassian, and K. Ciosek. On the importance of uncertainty in decision-making with large language models. arXiv preprint arXiv:2404.02649, 2024. — [arXiv:2404.02649](https://arxiv.org/abs/2404.02649) [find](https://scholar.google.com/scholar?q=N%2E+Felicioni%2C+L%2E+Maystre%2C+S%2E+Ghiassian%2C+and+K%2E+Ciosek%2E+On+the+importance+of+uncertainty+in+decision%2Dmaking+with+large+language+models%2E+arXiv+preprint+arXiv%3A2404%2E02649%2C+2024%2E+%E2%80%94+%5BarXiv%3A2404%2E02649%5D%28https)
8. K. Gandhi, D. Lee, G. Grand, M. Liu, W. Cheng, A. Sharma, and N. D. Goodman. [Stream of search (sos): Learning to search in language](https://md.research.elamur.ai/free/en/papers/gandhi-2024-stream-search-sos-learning-search/). arXiv preprint arXiv:2404.03683, 2024.
9. S. J. Gershman. Deconstructing the human algorithms for exploration. Cognition, 173:34–42, 2018. [find](https://scholar.google.com/scholar?q=S%2E+J%2E+Gershman%2E+Deconstructing+the+human+algorithms+for+exploration%2E+Cognition%2C+173%3A34%E2%80%9342%2C+2018%2E)
10. D. Guo, D. Yang, H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang, X. Bi, et al. [Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). arXiv preprint arXiv:2501.12948, 2025.
11. F. M. Harper and J. A. Konstan. The movielens datasets: History and context. Acm transactions on interactive intelligent systems (tiis), 5(4):1–19, 2015. [find](https://scholar.google.com/scholar?q=F%2E+M%2E+Harper+and+J%2E+A%2E+Konstan%2E+The+movielens+datasets%3A+History+and+context%2E+Acm+transactions+on+interactive+intelligent+systems+%28tiis%29%2C+5%284%29%3A1%E2%80%9319%2C+2015%2E)
12. J. He-Yueya, N. D. Goodman, and E. Brunskill. [Evaluating and optimizing educational content with large language model judgments](https://md.research.elamur.ai/free/en/papers/he-yueya-2024-evaluating-optimizing-educational-content-large/). arXiv preprint arXiv:2403.02795, 2024.
13. Y. Koren, R. Bell, and C. Volinsky. Matrix factorization techniques for recommender systems. Computer, 42(8):30–37, 2009. [find](https://scholar.google.com/scholar?q=Y%2E+Koren%2C+R%2E+Bell%2C+and+C%2E+Volinsky%2E+Matrix+factorization+techniques+for+recommender+systems%2E+Computer%2C+42%288%29%3A30%E2%80%9337%2C+2009%2E)
14. A. Krishnamurthy, K. Harris, D. J. Foster, C. Zhang, and A. Slivkins. Can large language models explore in-context? arXiv preprint arXiv:2403.15371, 2024. — [arXiv:2403.15371](https://arxiv.org/abs/2403.15371) [find](https://scholar.google.com/scholar?q=A%2E+Krishnamurthy%2C+K%2E+Harris%2C+D%2E+J%2E+Foster%2C+C%2E+Zhang%2C+and+A%2E+Slivkins%2E+Can+large+language+models+explore+in%2Dcontext%3F+arXiv+preprint+arXiv%3A2403%2E15371%2C+2024%2E+%E2%80%94+%5BarXiv%3A2403%2E15371%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F24)
15. J. Langford and T. Zhang. The epoch-greedy algorithm for contextual multi-armed bandits. Advances in neural information processing systems, 20(1):96–1, 2007. [find](https://scholar.google.com/scholar?q=J%2E+Langford+and+T%2E+Zhang%2E+The+epoch%2Dgreedy+algorithm+for+contextual+multi%2Darmed+bandits%2E+Advances+in+neural+information+processing+systems%2C+20%281%29%3A96%E2%80%931%2C+2007%2E)
16. M. Laskin, L. Wang, J. Oh, E. Parisotto, S. Spencer, R. Steigerwald, D. Strouse, S. Hansen, A. Filos, E. Brooks, et al. In-context reinforcement learning with algorithm distillation. arXiv preprint arXiv:2210.14215, 2022. — [arXiv:2210.14215](https://arxiv.org/abs/2210.14215) [find](https://scholar.google.com/scholar?q=M%2E+Laskin%2C+L%2E+Wang%2C+J%2E+Oh%2C+E%2E+Parisotto%2C+S%2E+Spencer%2C+R%2E+Steigerwald%2C+D%2E+Strouse%2C+S%2E+Hansen%2C+A%2E+Filos%2C+E%2E+Brooks%2C+et+al%2E+In%2Dcontext+reinforcement+learning+with+algorithm+distillation%2E+arXiv+preprint+ar)
17. J. Lee, A. Xie, A. Pacchiano, Y. Chandak, C. Finn, O. Nachum, and E. Brunskill. Supervised pretraining can learn in-context reinforcement learning. Advances in Neural Information Processing Systems, 36, 2024. [find](https://scholar.google.com/scholar?q=J%2E+Lee%2C+A%2E+Xie%2C+A%2E+Pacchiano%2C+Y%2E+Chandak%2C+C%2E+Finn%2C+O%2E+Nachum%2C+and+E%2E+Brunskill%2E+Supervised+pretraining+can+learn+in%2Dcontext+reinforcement+learning%2E+Advances+in+Neural+Information+Processing+Systems%2C+3)
18. L. Li, W. Chu, J. Langford, and R. E. Schapire. A contextual-bandit approach to personalized news article recommendation. In Proceedings of the 19th international conference on World wide web, pages 661–670, 2010. [find](https://scholar.google.com/scholar?q=L%2E+Li%2C+W%2E+Chu%2C+J%2E+Langford%2C+and+R%2E+E%2E+Schapire%2E+A+contextual%2Dbandit+approach+to+personalized+news+article+recommendation%2E+In+Proceedings+of+the+19th+international+conference+on+World+wide+web%2C+pages+6)
19. L. Li, Y. Zhang, D. Liu, and L. Chen. Large language models for generative recommendation: A survey and visionary discussions. arXiv preprint arXiv:2309.01157, 2023a. [find](https://scholar.google.com/scholar?q=L%2E+Li%2C+Y%2E+Zhang%2C+D%2E+Liu%2C+and+L%2E+Chen%2E+Large+language+models+for+generative+recommendation%3A+A+survey+and+visionary+discussions%2E+arXiv+preprint+arXiv%3A2309%2E01157%2C+2023a%2E)
20. L. H. Li, J. Hessel, Y. Yu, X. Ren, K.-W. Chang, and Y. Choi. Symbolic chain-of-thought distillation: Small models can also" think" step-by-step. arXiv preprint arXiv:2306.[14050, 2023b](https://md.research.elamur.ai/free/en/papers/li-2023-symbolic-chain-thought-distillation-small/).
21. L. Lin, Y. Bai, and S. Mei. Transformers as decision makers: Provable in-context reinforcement learning via supervised pretraining. In The Twelfth International Conference on Learning Representations, 2023. [find](https://scholar.google.com/scholar?q=L%2E+Lin%2C+Y%2E+Bai%2C+and+S%2E+Mei%2E+Transformers+as+decision+makers%3A+Provable+in%2Dcontext+reinforcement+learning+via+supervised+pretraining%2E+In+The+Twelfth+International+Conference+on+Learning+Representations%2C)
22. O. Liu, D. Fu, D. Yogatama, and W. Neiswanger. Dellma: A framework for decision making under uncertainty with large language models. arXiv preprint arXiv:2402.02392, 2024a. — [arXiv:2402.02392](https://arxiv.org/abs/2402.02392) [find](https://scholar.google.com/scholar?q=O%2E+Liu%2C+D%2E+Fu%2C+D%2E+Yogatama%2C+and+W%2E+Neiswanger%2E+Dellma%3A+A+framework+for+decision+making+under+uncertainty+with+large+language+models%2E+arXiv+preprint+arXiv%3A2402%2E02392%2C+2024a%2E+%E2%80%94+%5BarXiv%3A2402%2E02392%5D%28https%3A)
23. Z. Liu, Z. Zhou, Y. Wang, C. Yang, and Y. Qiao. Inference-time language model alignment via integrated value guidance. arXiv preprint arXiv:2409.17819, 2024b. [find](https://scholar.google.com/scholar?q=Z%2E+Liu%2C+Z%2E+Zhou%2C+Y%2E+Wang%2C+C%2E+Yang%2C+and+Y%2E+Qiao%2E+Inference%2Dtime+language+model+alignment+via+integrated+value+guidance%2E+arXiv+preprint+arXiv%3A2409%2E17819%2C+2024b%2E)
24. S. Mirchandani, F. Xia, P. Florence, B. Ichter, D. Driess, M. G. Arenas, K. Rao, D. Sadigh, and A. Zeng. [Large language models as general pattern machines](https://md.research.elamur.ai/free/en/papers/mirchandani-2023-large-language-models-general-pattern/). arXiv preprint arXiv:2307.04721, 2023.
25. V. Mnih. Playing atari with deep reinforcement learning. arXiv preprint arXiv:1312.5602, 2013. [find](https://scholar.google.com/scholar?q=V%2E+Mnih%2E+Playing+atari+with+deep+reinforcement+learning%2E+arXiv+preprint+arXiv%3A1312%2E5602%2C+2013%2E)
26. S. Mudgal, J. Lee, H. Ganapathy, Y. Li, T. Wang, Y. Huang, Z. Chen, H.-T. Cheng, M. Collins, T. Strohman, et al. Controlled decoding from language models. arXiv preprint arXiv:2310.17022, 2023. [find](https://scholar.google.com/scholar?q=S%2E+Mudgal%2C+J%2E+Lee%2C+H%2E+Ganapathy%2C+Y%2E+Li%2C+T%2E+Wang%2C+Y%2E+Huang%2C+Z%2E+Chen%2C+H%2E%2DT%2E+Cheng%2C+M%2E+Collins%2C+T%2E+Strohman%2C+et+al%2E+Controlled+decoding+from+language+models%2E+arXiv+preprint+arXiv%3A2310%2E17022%2C+2023%2E)
27. A. Nie, Y. Chandak, M. Suzara, A. Malik, J. Woodrow, M. Peng, M. Sahami, E. Brunskill, and C. Piech. The gpt surprise: Offering large language model chat in a massive coding class reduced engagement but increased adopters’ exam performances. Technical report, Center for Open Science, 2024. [find](https://scholar.google.com/scholar?q=A%2E+Nie%2C+Y%2E+Chandak%2C+M%2E+Suzara%2C+A%2E+Malik%2C+J%2E+Woodrow%2C+M%2E+Peng%2C+M%2E+Sahami%2C+E%2E+Brunskill%2C+and+C%2E+Piech%2E+The+gpt+surprise%3A+Offering+large+language+model+chat+in+a+massive+coding+class+reduced+engagement+b)
28. I. Osband, D. Russo, and B. Van Roy. (more) efficient reinforcement learning via posterior sampling. Advances in Neural Information Processing Systems, 26, 2013. [find](https://scholar.google.com/scholar?q=I%2E+Osband%2C+D%2E+Russo%2C+and+B%2E+Van+Roy%2E+%28more%29+efficient+reinforcement+learning+via+posterior+sampling%2E+Advances+in+Neural+Information+Processing+Systems%2C+26%2C+2013%2E)
29. L. Pan, H. Xie, and R. C. Wilson. Large language models think too fast to explore effectively. arXiv preprint arXiv:2501.18009, 2025. — [arXiv:2501.18009](https://arxiv.org/abs/2501.18009) [find](https://scholar.google.com/scholar?q=L%2E+Pan%2C+H%2E+Xie%2C+and+R%2E+C%2E+Wilson%2E+Large+language+models+think+too+fast+to+explore+effectively%2E+arXiv+preprint+arXiv%3A2501%2E18009%2C+2025%2E+%E2%80%94+%5BarXiv%3A2501%2E18009%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2501%2E18009%29)
30. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, et al. Scikit-learn: Machine learning in python. the Journal of machine Learning research, 12:2825–2830, 2011. [find](https://scholar.google.com/scholar?q=F%2E+Pedregosa%2C+G%2E+Varoquaux%2C+A%2E+Gramfort%2C+V%2E+Michel%2C+B%2E+Thirion%2C+O%2E+Grisel%2C+M%2E+Blondel%2C+P%2E+Prettenhofer%2C+R%2E+Weiss%2C+V%2E+Dubourg%2C+et+al%2E+Scikit%2Dlearn%3A+Machine+learning+in+python%2E+the+Journal+of+machine+Le)
31. D. A. Pomerleau. Efficient training of artificial neural networks for autonomous navigation. Neural computation, 3(1):88–97, 1991. [find](https://scholar.google.com/scholar?q=D%2E+A%2E+Pomerleau%2E+Efficient+training+of+artificial+neural+networks+for+autonomous+navigation%2E+Neural+computation%2C+3%281%29%3A88%E2%80%9397%2C+1991%2E)
32. Y. Qu, B. Wang, Y. Jiang, J. Shao, Y. Mao, C. Wang, C. Liu, and X. Ji. Choices are more important than efforts: Llm enables efficient multi-agent exploration. arXiv preprint arXiv:2410.02511, 2024. — [arXiv:2410.02511](https://arxiv.org/abs/2410.02511) [find](https://scholar.google.com/scholar?q=Y%2E+Qu%2C+B%2E+Wang%2C+Y%2E+Jiang%2C+J%2E+Shao%2C+Y%2E+Mao%2C+C%2E+Wang%2C+C%2E+Liu%2C+and+X%2E+Ji%2E+Choices+are+more+important+than+efforts%3A+Llm+enables+efficient+multi%2Dagent+exploration%2E+arXiv+preprint+arXiv%3A2410%2E02511%2C+2024%2E+%E2%80%94+)
33. N. Rahn, P. D’Oro, and M. G. Bellemare. Controlling large language model agents with entropic activation steering. arXiv preprint arXiv:2406.00244, 2024. — [arXiv:2406.00244](https://arxiv.org/abs/2406.00244) [find](https://scholar.google.com/scholar?q=N%2E+Rahn%2C+P%2E+D%E2%80%99Oro%2C+and+M%2E+G%2E+Bellemare%2E+Controlling+large+language+model+agents+with+entropic+activation+steering%2E+arXiv+preprint+arXiv%3A2406%2E00244%2C+2024%2E+%E2%80%94+%5BarXiv%3A2406%2E00244%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F240)
34. S. C. Raparthy, E. Hambro, R. Kirk, M. Henaff, and R. Raileanu. Generalization to new sequential decision making tasks with in-context learning. arXiv preprint arXiv:2312.03801, 2023. — [arXiv:2312.03801](https://arxiv.org/abs/2312.03801) [find](https://scholar.google.com/scholar?q=S%2E+C%2E+Raparthy%2C+E%2E+Hambro%2C+R%2E+Kirk%2C+M%2E+Henaff%2C+and+R%2E+Raileanu%2E+Generalization+to+new+sequential+decision+making+tasks+with+in%2Dcontext+learning%2E+arXiv+preprint+arXiv%3A2312%2E03801%2C+2023%2E+%E2%80%94+%5BarXiv%3A2312%2E03)
35. M. Reid, N. Savinov, D. Teplyashin, D. Lepikhin, T. Lillicrap, J.-b. Alayrac, R. Soricut, A. Lazaridou, O. Firat, J. Schrittwieser, et al. [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://md.research.elamur.ai/free/en/papers/gemini-team-2024-gemini-1-5-unlocking-multimodal/). arXiv preprint arXiv:2403.05530, 2024.
36. N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao. [Reflexion: Language agents with verbal reinforcement learning](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/). Advances in Neural Information Processing Systems, 36, 2024.
37. K. Singhal, S. Azizi, T. Tu, S. S. Mahdavi, J. Wei, H. W. Chung, N. Scales, A. Tanwani, H. Cole-Lewis, S. Pfohl, et al. [Large language models encode clinical knowledge](https://md.research.elamur.ai/free/en/papers/singhal-2022-large-language-models-encode-clinical/). Nature, 620(7972):172–180, 2023.
38. A. Slivkins et al. Introduction to multi-armed bandits. Foundations and Trends® in Machine Learning, 12(1-2):1–286, 2019. [find](https://scholar.google.com/scholar?q=A%2E+Slivkins+et+al%2E+Introduction+to+multi%2Darmed+bandits%2E+Foundations+and+Trends%C2%AE+in+Machine+Learning%2C+12%281%2D2%29%3A1%E2%80%93286%2C+2019%2E)
39. C. Snell, J. Lee, K. Xu, and A. Kumar. [Scaling llm test-time compute optimally can be more effective than scaling model parameters](https://md.research.elamur.ai/free/en/papers/snell-2024-scaling-llm-test-time-compute/). arXiv preprint arXiv:2408.03314, 2024.
40. R. S. Sutton. Reinforcement learning: An introduction. A Bradford Book, 2018. [find](https://scholar.google.com/scholar?q=R%2E+S%2E+Sutton%2E+Reinforcement+learning%3A+An+introduction%2E+A+Bradford+Book%2C+2018%2E)
41. G. Team, M. Riviere, S. Pathak, P. G. Sessa, C. Hardin, S. Bhupatiraju, L. Hussenot, T. Mesnard, B. Shahriari, A. Ramé, et al. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118, 2024. [find](https://scholar.google.com/scholar?q=G%2E+Team%2C+M%2E+Riviere%2C+S%2E+Pathak%2C+P%2E+G%2E+Sessa%2C+C%2E+Hardin%2C+S%2E+Bhupatiraju%2C+L%2E+Hussenot%2C+T%2E+Mesnard%2C+B%2E+Shahriari%2C+A%2E+Ram%C3%A9%2C+et+al%2E+Gemma+2%3A+Improving+open+language+models+at+a+practical+size%2E+arXiv+prepri)
42. G. Wang, Y. Xie, Y. Jiang, A. Mandlekar, C. Xiao, Y. Zhu, L. Fan, and A. Anandkumar. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291, 2023a. [find](https://scholar.google.com/scholar?q=G%2E+Wang%2C+Y%2E+Xie%2C+Y%2E+Jiang%2C+A%2E+Mandlekar%2C+C%2E+Xiao%2C+Y%2E+Zhu%2C+L%2E+Fan%2C+and+A%2E+Anandkumar%2E+Voyager%3A+An+open%2Dended+embodied+agent+with+large+language+models%2E+arXiv+preprint+arXiv%3A2305%2E16291%2C+2023a%2E)
43. P. Wang, Z. Wang, Z. Li, Y. Gao, B. Yin, and X. Ren. [Scott: Self-consistent chain-of-thought distillation](https://md.research.elamur.ai/free/en/papers/wang-2023-scott-self-consistent-chain-thought/). arXiv preprint arXiv:2305.01879, 2023b.
44. Z. Wang, S. Cai, G. Chen, A. Liu, X. Ma, and Y. Liang. [Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents](https://md.research.elamur.ai/free/en/papers/wang-2023-describe-explain-plan-select-interactive/). arXiv preprint arXiv:2302.01560, 2023c.
45. R. C. Wilson, A. Geana, J. M. White, E. A. Ludvig, and J. D. Cohen. Humans use directed and random exploration to solve the explore–exploit dilemma. Journal of experimental psychology: General, 143 (6):2074, 2014. [find](https://scholar.google.com/scholar?q=R%2E+C%2E+Wilson%2C+A%2E+Geana%2C+J%2E+M%2E+White%2C+E%2E+A%2E+Ludvig%2C+and+J%2E+D%2E+Cohen%2E+Humans+use+directed+and+random+exploration+to+solve+the+explore%E2%80%93exploit+dilemma%2E+Journal+of+experimental+psychology%3A+General%2C+143+%286)
46. Z. Xi, W. Chen, X. Guo, W. He, Y. Ding, B. Hong, M. Zhang, J. Wang, S. Jin, E. Zhou, et al. [The rise and potential of large language model based agents: A survey](https://md.research.elamur.ai/free/en/papers/xi-2023-rise-potential-large-language/). arXiv preprint arXiv:2309.07864, 2023.
47. S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao. [React: Synergizing reasoning and acting in language models](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/). In International Conference on Learning Representations (ICLR), 2023.
48. P. Yu, J. Xu, J. Weston, and I. Kulikov. Distilling system 2 into system 1. arXiv preprint arXiv:2407.06023, 2024. [find](https://scholar.google.com/scholar?q=P%2E+Yu%2C+J%2E+Xu%2C+J%2E+Weston%2C+and+I%2E+Kulikov%2E+Distilling+system+2+into+system+1%2E+arXiv+preprint+arXiv%3A2407%2E06023%2C+2024%2E)
49. H. S. Zheng, S. Mishra, H. Zhang, X. Chen, M. Chen, A. Nova, L. Hou, H.-T. Cheng, Q. V. Le, E. H. Chi, et al. Natural plan: Benchmarking llms on natural language planning. arXiv preprint arXiv:2406.04520, 2024. [find](https://scholar.google.com/scholar?q=H%2E+S%2E+Zheng%2C+S%2E+Mishra%2C+H%2E+Zhang%2C+X%2E+Chen%2C+M%2E+Chen%2C+A%2E+Nova%2C+L%2E+Hou%2C+H%2E%2DT%2E+Cheng%2C+Q%2E+V%2E+Le%2C+E%2E+H%2E+Chi%2C+et+al%2E+Natural+plan%3A+Benchmarking+llms+on+natural+language+planning%2E+arXiv+preprint+arXiv%3A2406%2E04)

*Updated 3 November 2026*