---
title: "DS-Agent — paper by Guo et al. (2024) — summary and ideas"
description: "Agents built on large language models often fail at automated machine learning because they… 53 references, cited by 43 papers here."
authors: ["Siyuan Guo", "Cheng Deng", "Ying Wen", "Hechang Chen", "Yi Chang", "Jun Wang"]
year: 2024
venue: "arXiv:2402.17453"
source: "https://arxiv.org/abs/2402.17453"
doi: "10.48550/arxiv.2402.17453"
arxiv: "2402.17453"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/guo-2024-ds-agent-automated-data-science/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/guo-2024-ds-agent-automated-data-science/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/8/) / DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoning

# DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoning

**Authors:** [Siyuan Guo](https://md.research.elamur.ai/free/en/authors/siyuan-guo/), [Cheng Deng](https://md.research.elamur.ai/free/en/authors/cheng-deng/), [Ying Wen](https://md.research.elamur.ai/free/en/authors/ying-wen/), Hechang Chen, [Yi Chang](https://md.research.elamur.ai/free/en/authors/yi-chang/) and [Jun Wang](https://md.research.elamur.ai/free/en/authors/jun-wang/) · **Year:** 2024 · **Venue:** arXiv:2402.17453 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2402.17453) · [PDF](https://arxiv.org/pdf/2402.17453) · [DOI 10.48550/arxiv.2402.17453](https://doi.org/10.48550/arxiv.2402.17453) · [arXiv 2402.17453](https://arxiv.org/abs/2402.17453) · [OpenAlex](https://openalex.org/W4392271261)

**Citations:** 7 citations (OpenAlex, 2026-10-14) · [43](https://md.research.elamur.ai/free/en/papers/guo-2024-ds-agent-automated-data-science/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Agents built on large language models often fail at automated machine learning because they propose poorly grounded experiment plans and hallucinate, even with GPT-4. DS-Agent addresses this with case-based reasoning (CBR) rather than costly fine-tuning. The authors built a case bank from recent Kaggle competitions covering text, time-series and tabular data. It holds cleaned winning-team write-ups and top leaderboard code, which GPT-3.5 condenses into textual insights.

In a development stage, the agent:
- pulls up relevant cases,
- re-ranks them based on execution feedback,
- drafts and runs a plan,
- revises it over several iterations,
- stores successful solutions.

A cheaper deployment stage skips the iteration loop. It adapts a stored solution from a similar task and generates code in a single pass, which places less demand on the underlying model.

Across 30 tasks, DS-Agent ranks best with both GPT-4 and GPT-3.5 and costs little per run. The work shows that reusing expert and past solutions can substitute for parameter updates when building practical data-science agents.

## Key points
- **Benchmark:** 30 tasks in total, 12 for development and 18 for deployment. Baselines include ResearchAgent, AutoGPT and LangChain.
- **Development stage:** DS-Agent with GPT-4 completes all 12 tasks (100% success rate).
- **Deployment stage, one-pass rate:**
  - GPT-3.5 reaches 85% and GPT-4 reaches 99%, against 56% and 60% for the strongest baseline.
  - Mixtral-8x7b-Instruct improves from 6% to 31%.
- **Cost per run:**
  - Development: about $1.60 with GPT-4 and $0.06 with GPT-3.5.
  - Deployment: roughly $0.135 with GPT-4 and $0.0045 with GPT-3.5.
- **Retrieval and ablation findings:**
  - Retrieval ranks cases by cosine similarity of embeddings.
  - Reusing past full solutions works better than reusing textual Kaggle insights.
  - Zero-shot generation performs worst, and adding more in-context cases lowers performance.
  - Code and data are released on GitHub (guosyjlu/DS-Agent).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Shanghai Jiao Tong University](https://md.research.elamur.ai/free/en/universities/shanghai-jiao-tong-university/) — University, Shanghai, China · 232 works · 20,305 citations
- [University College London](https://md.research.elamur.ai/free/en/universities/university-college-london/) — University, London, United Kingdom · 119 works · 23,545 citations
- [Jilin University](https://md.research.elamur.ai/free/en/universities/jilin-university/) — University, Changchun, China · 31 works · 5,302 citations

📚 Shelves (3): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,888 works

## Cite

Cite as: Siyuan Guo, Cheng Deng, Ying Wen, Hechang Chen, Yi Chang and Jun Wang. 2024. DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoning. arXiv:2402.17453.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{guo2024ds,
  title = {{DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoning}},
  author = {Siyuan Guo, Cheng Deng, Ying Wen, Hechang Chen, Yi Chang and Jun Wang},
  year = {2024},
  doi = {10.48550/arxiv.2402.17453},
  eprint = {2402.17453},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2402.17453}
}
```

- [← previous Cultural Adaptation of Recipes](https://md.research.elamur.ai/free/en/papers/cao-2024-cultural-adaptation-recipes/)
- [next → Fast Best-of-N Decoding via Speculative Rejection](https://md.research.elamur.ai/free/en/papers/sun-2024-fast-best-n-decoding-speculative/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/8/)

## References

53 references, 23 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Aamodt, A. and Plaza, E. Case-based reasoning: Foundational issues, methodological variations, and system approaches. AI communications, 7(1):39–59, 1994. [find](https://scholar.google.com/scholar?q=Aamodt%2C+A%2E+and+Plaza%2C+E%2E+Case%2Dbased+reasoning%3A+Foundational+issues%2C+methodological+variations%2C+and+system+approaches%2E+AI+communications%2C+7%281%29%3A39%E2%80%9359%2C+1994%2E)
2. Boiko, D. A., MacKnight, R., Kline, B., and Gomes, G. [Autonomous chemical research with large language models](https://md.research.elamur.ai/free/en/papers/boiko-2023-autonomous-chemical-research-large-language/). Nature, 624(7992):570–578, 2023.
3. Brohan, A., Chebotar, Y., Finn, C., Hausman, K., Herzog, A., Ho, D., Ibarz, J., Irpan, A., Jang, E., Julian, R., et al. [Do as i can, not as i say: Grounding language in robotic affordances](https://md.research.elamur.ai/free/en/papers/ahn-2022-i-i-say-grounding-language/). In Conference on Robot Learning, pp. 287– 318. PMLR, 2023.
4. Carta, T., Romac, C., Wolf, T., Lamprier, S., Sigaud, O., and Oudeyer, P.-Y. Grounding large language models in interactive environments with online reinforcement learning. In Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research… [find](https://scholar.google.com/scholar?q=Carta%2C+T%2E%2C+Romac%2C+C%2E%2C+Wolf%2C+T%2E%2C+Lamprier%2C+S%2E%2C+Sigaud%2C+O%2E%2C+and+Oudeyer%2C+P%2E%2DY%2E+Grounding+large+language+models+in+interactive+environments+with+online+reinforcement+learning%2E+In+Proceedings+of+the+40th+)
5. Chase, H. LangChain, October 2022. URL [https://github.com/langchain-ai/langchain](https://github.com/langchain-ai/langchain). [↗](https://github.com/langchain-ai/langchain)
6. Chen, B., Shu, C., Shareghi, E., Collier, N., Narasimhan, K., and Yao, S. [Fireact: Toward language agent fine-tuning](https://md.research.elamur.ai/free/en/papers/chen-2023-fireact-toward-language-agent-fine/). arXiv preprint arXiv:2310.05915, 2023.
7. Christianos, F., Papoudakis, G., Zimmer, M., Coste, T., Wu, Z., Chen, J., Khandelwal, K., Doran, J., Feng, X., Liu, J., et al. Pangu-agent: A fine-tunable generalist agent with structured reasoning. arXiv preprint arXiv:2312.14878, 2023. — [arXiv:2312.14878](https://arxiv.org/abs/2312.14878) [find](https://scholar.google.com/scholar?q=Christianos%2C+F%2E%2C+Papoudakis%2C+G%2E%2C+Zimmer%2C+M%2E%2C+Coste%2C+T%2E%2C+Wu%2C+Z%2E%2C+Chen%2C+J%2E%2C+Khandelwal%2C+K%2E%2C+Doran%2C+J%2E%2C+Feng%2C+X%2E%2C+Liu%2C+J%2E%2C+et+al%2E+Pangu%2Dagent%3A+A+fine%2Dtunable+generalist+agent+with+structured+reasoning%2E+a)
8. De Bie, T., De Raedt, L., Hernández-Orallo, J., Hoos, H. H., Smyth, P., and Williams, C. K. Automating data science. Communications of the ACM, 65(3):76–87, 2022. [find](https://scholar.google.com/scholar?q=De+Bie%2C+T%2E%2C+De+Raedt%2C+L%2E%2C+Herna%CC%81ndez%2DOrallo%2C+J%2E%2C+Hoos%2C+H%2E+H%2E%2C+Smyth%2C+P%2E%2C+and+Williams%2C+C%2E+K%2E+Automating+data+science%2E+Communications+of+the+ACM%2C+65%283%29%3A76%E2%80%9387%2C+2022%2E)
9. Deng, C., Zhang, T., He, Z., Chen, Q., Shi, Y., Zhou, L., Fu, L., Zhang, W., Wang, X., Zhou, C., Lin, Z., and He, J. K2: A foundation language model for geoscience knowledge understanding and utilization. 2023. URL [https://api.semanticscholar.org/CorpusID:259108887](https://api.semanticscholar.org/CorpusID:259108887). [↗](https://api.semanticscholar.org/CorpusID:259108887)
10. Erickson, N., Mueller, J., Shirkov, A., Zhang, H., Larroy, P., Li, M., and Smola, A. Autogluon-tabular: Robust and accurate automl for structured data. arXiv preprint arXiv:2003.06505, 2020. — [arXiv:2003.06505](https://arxiv.org/abs/2003.06505) [find](https://scholar.google.com/scholar?q=Erickson%2C+N%2E%2C+Mueller%2C+J%2E%2C+Shirkov%2C+A%2E%2C+Zhang%2C+H%2E%2C+Larroy%2C+P%2E%2C+Li%2C+M%2E%2C+and+Smola%2C+A%2E+Autogluon%2Dtabular%3A+Robust+and+accurate+automl+for+structured+data%2E+arXiv+preprint+arXiv%3A2003%2E06505%2C+2020%2E+%E2%80%94+%5BarXiv%3A)
11. Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., and Wang, H. [Retrieval-augmented generation for large language models: A survey](https://md.research.elamur.ai/free/en/papers/gao-2023-retrieval-augmented-generation-large-language/). arXiv preprint arXiv:2312.10997, 2023.
12. Guo, C., Tian, Z., Tang, J., Wang, P., Wen, Z., Yang, K., and Wang, T. A case-based reasoning framework for adaptive prompting in cross-domain text-to-sql. arXiv preprint arXiv:2304.13301, 2023. — [arXiv:2304.13301](https://arxiv.org/abs/2304.13301) [find](https://scholar.google.com/scholar?q=Guo%2C+C%2E%2C+Tian%2C+Z%2E%2C+Tang%2C+J%2E%2C+Wang%2C+P%2E%2C+Wen%2C+Z%2E%2C+Yang%2C+K%2E%2C+and+Wang%2C+T%2E+A+case%2Dbased+reasoning+framework+for+adaptive+prompting+in+cross%2Ddomain+text%2Dto%2Dsql%2E+arXiv+preprint+arXiv%3A2304%2E13301%2C+2023%2E+%E2%80%94+%5Bar)
13. Hollmann, N., Müller, S., and Hutter, F. Large language models for automated data science: Introducing caafe for context-aware automated feature engineering. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. [find](https://scholar.google.com/scholar?q=Hollmann%2C+N%2E%2C+Mu%CC%88ller%2C+S%2E%2C+and+Hutter%2C+F%2E+Large+language+models+for+automated+data+science%3A+Introducing+caafe+for+context%2Daware+automated+feature+engineering%2E+In+Thirty%2Dseventh+Conference+on+Neural+In)
14. Hong, S., Lin, Y., Liu, B., Wu, B., Li, D., Chen, J., Zhang, J., Wang, J., Zhang, L., Zhuge, M., et al. [Data interpreter: An llm agent for data science](https://md.research.elamur.ai/free/en/papers/hong-2025-data-interpreter-llm-agent-data/). arXiv preprint arXiv:2402.18679, 2024a.
15. Hong, S., Zheng, X., Chen, J., Cheng, Y., Wang, J., Zhang, C., Wang, Z., Yau, S. K. S., Lin, Z., Zhou, L., et al. MetaGPT: Meta programming for multi-agent collaborative framework. In The Twelfth International Conference on Learning Representations, 2024b. URL [https://openreview.net/forum?id=VtmBAG…](https://openreview.net/forum?id=VtmBAG…) [↗](https://openreview.net/forum?id=VtmBAG…)
16. Huang, Q., Vora, J., Liang, P., and Leskovec, J. Benchmarking large language models as ai research agents. arXiv preprint arXiv:2310.[03302, 2023](https://md.research.elamur.ai/free/en/papers/huang-2023-mlagentbench-evaluating-language-agents/).
17. Hutter, F., Kotthoff, L., and Vanschoren, J. Automated machine learning: methods, systems, challenges. Springer Nature, 2019. [find](https://scholar.google.com/scholar?q=Hutter%2C+F%2E%2C+Kotthoff%2C+L%2E%2C+and+Vanschoren%2C+J%2E+Automated+machine+learning%3A+methods%2C+systems%2C+challenges%2E+Springer+Nature%2C+2019%2E)
18. Jiang, A. Q., Sablayrolles, A., Roux, A., Mensch, A., Savary, B., Bamford, C., Chaplot, D. S., Casas, D. d. l., Hanna, E. B., Bressand, F., et al. Mixtral of experts. arXiv preprint arXiv:2401.04088, 2024. [find](https://scholar.google.com/scholar?q=Jiang%2C+A%2E+Q%2E%2C+Sablayrolles%2C+A%2E%2C+Roux%2C+A%2E%2C+Mensch%2C+A%2E%2C+Savary%2C+B%2E%2C+Bamford%2C+C%2E%2C+Chaplot%2C+D%2E+S%2E%2C+Casas%2C+D%2E+d%2E+l%2E%2C+Hanna%2C+E%2E+B%2E%2C+Bressand%2C+F%2E%2C+et+al%2E+Mixtral+of+experts%2E+arXiv+preprint+arXiv%3A2401%2E04088%2C+)
19. Karmaker, S. K., Hassan, M. M., Smith, M. J., Xu, L., Zhai, C., and Veeramachaneni, K. Automl to date and beyond: Challenges and opportunities. ACM Computing Surveys (CSUR), 54(8):1–36, 2021. [find](https://scholar.google.com/scholar?q=Karmaker%2C+S%2E+K%2E%2C+Hassan%2C+M%2E+M%2E%2C+Smith%2C+M%2E+J%2E%2C+Xu%2C+L%2E%2C+Zhai%2C+C%2E%2C+and+Veeramachaneni%2C+K%2E+Automl+to+date+and+beyond%3A+Challenges+and+opportunities%2E+ACM+Computing+Surveys+%28CSUR%29%2C+54%288%29%3A1%E2%80%9336%2C+2021%2E)
20. Kim, G., Baldi, P., and McAleer, S. M. [Language models can solve computer tasks](https://md.research.elamur.ai/free/en/papers/kim-2023-language-models-solve-computer-tasks/). In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
21. Kolodner, J. L. An introduction to case-based reasoning. Artificial intelligence review, 6(1):3–34, 1992. [find](https://scholar.google.com/scholar?q=Kolodner%2C+J%2E+L%2E+An+introduction+to+case%2Dbased+reasoning%2E+Artificial+intelligence+review%2C+6%281%29%3A3%E2%80%9334%2C+1992%2E)
22. Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J., Zhang, H., and Stoica, I. [Efficient memory management for large language model serving with pagedattention](https://md.research.elamur.ai/free/en/papers/kwon-2023-efficient-memory-management-large-language/). In Proceedings of the 29th Symposium on Operating Systems Principles, pp. 611–626, 2023.
23. Lai, Y., Li, C., Wang, Y., Zhang, T., Zhong, R., Zettlemoyer, L., Yih, W.-t., Fried, D., Wang, S., and Yu, T. Ds-1000: A natural and reliable benchmark for data science code generation. In International Conference on Machine Learning, pp. 18319–18345. PMLR, 2023. [find](https://scholar.google.com/scholar?q=Lai%2C+Y%2E%2C+Li%2C+C%2E%2C+Wang%2C+Y%2E%2C+Zhang%2C+T%2E%2C+Zhong%2C+R%2E%2C+Zettlemoyer%2C+L%2E%2C+Yih%2C+W%2E%2Dt%2E%2C+Fried%2C+D%2E%2C+Wang%2C+S%2E%2C+and+Yu%2C+T%2E+Ds%2D1000%3A+A+natural+and+reliable+benchmark+for+data+science+code+generation%2E+In+Internation)
24. LeDell, E. and Poirier, S. H2o automl: Scalable automatic machine learning. In Proceedings of the AutoML Workshop at ICML, volume 2020. [find](https://scholar.google.com/scholar?q=LeDell%2C+E%2E+and+Poirier%2C+S%2E+H2o+automl%3A+Scalable+automatic+machine+learning%2E+In+Proceedings+of+the+AutoML+Workshop+at+ICML%2C+volume+2020%2E)
25. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-t., Rocktäschel, T., et al. [Retrieval-augmented generation for knowledge-intensive nlp tasks](https://md.research.elamur.ai/free/en/papers/lewis-2020-retrieval-augmented-generation-knowledge-intensive/). Advances in Neural Information Processing Systems, 33:9459–9474, 2020.
26. Lin, Z., Deng, C., Zhou, L., Zhang, T., Xu, Y., Xu, Y., He, Z., Shi, Y., Dai, B., Song, Y., Zeng, B., Chen, Q., Shi, T., Huang, T., Xu, Y., Wang, S., Fu, L., Zhang, W., He, J., Ma, C., Zhu, Y., Wang, X., and Zhou, C. Geogalactica: A scientific large language model in geoscience. ArXiv, abs/2401.0043… — [arXiv:2401.00434](https://arxiv.org/abs/2401.00434) [find](https://scholar.google.com/scholar?q=Lin%2C+Z%2E%2C+Deng%2C+C%2E%2C+Zhou%2C+L%2E%2C+Zhang%2C+T%2E%2C+Xu%2C+Y%2E%2C+Xu%2C+Y%2E%2C+He%2C+Z%2E%2C+Shi%2C+Y%2E%2C+Dai%2C+B%2E%2C+Song%2C+Y%2E%2C+Zeng%2C+B%2E%2C+Chen%2C+Q%2E%2C+Shi%2C+T%2E%2C+Huang%2C+T%2E%2C+Xu%2C+Y%2E%2C+Wang%2C+S%2E%2C+Fu%2C+L%2E%2C+Zhang%2C+W%2E%2C+He%2C+J%2E%2C+Ma%2C+C%2E%2C+Zhu%2C+Y%2E%2C+Wang%2C+)
27. Liu, Y., Tang, X., Cai, Z., Lu, J., Zhang, Y., Shao, Y., Deng, Z., Hu, H., Yang, Z., An, K., et al. Ml-bench: Large language models leverage open-source libraries for machine learning tasks. arXiv preprint arXiv:2311.[09835, 2023](https://md.research.elamur.ai/free/en/papers/tang-2023-ml-bench-evaluating-large-language/).
28. Mahadi Hassan, M., Knipper, A., and Kanti Karmaker Santu, S. Chatgpt as your personal data scientist. arXiv e-prints, pp. arXiv–2305, 2023. [find](https://scholar.google.com/scholar?q=Mahadi+Hassan%2C+M%2E%2C+Knipper%2C+A%2E%2C+and+Kanti+Karmaker+Santu%2C+S%2E+Chatgpt+as+your+personal+data+scientist%2E+arXiv+e%2Dprints%2C+pp%2E+arXiv%E2%80%932305%2C+2023%2E)
29. OpenAI. Introducing chatgpt. 2022. URL [https://openai.com/blog/chatgpt](https://openai.com/blog/chatgpt). [↗](https://openai.com/blog/chatgpt)
30. OpenAI. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv:2303.08774, 2023. arXiv preprint Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., et al. Scikit-learn: Machine learning in python. the Journal of machine Learning research, 12…
31. Pei, H., Yang, B., Liu, J., and Chang, K. C.-C. Active surveillance via group sparse bayesian learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(3):1133–1148, 2020. [find](https://scholar.google.com/scholar?q=Pei%2C+H%2E%2C+Yang%2C+B%2E%2C+Liu%2C+J%2E%2C+and+Chang%2C+K%2E+C%2E%2DC%2E+Active+surveillance+via+group+sparse+bayesian+learning%2E+IEEE+Transactions+on+Pattern+Analysis+and+Machine+Intelligence%2C+44%283%29%3A1133%E2%80%931148%2C+2020%2E)
32. Pei, H., Chen, T., Chen, A., Deng, H., Tao, J., Wang, P., and Guan, X. Hago-net: Hierarchical geometric massage passing for molecular representation learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pp. 14572–14580, 2024a. [find](https://scholar.google.com/scholar?q=Pei%2C+H%2E%2C+Chen%2C+T%2E%2C+Chen%2C+A%2E%2C+Deng%2C+H%2E%2C+Tao%2C+J%2E%2C+Wang%2C+P%2E%2C+and+Guan%2C+X%2E+Hago%2Dnet%3A+Hierarchical+geometric+massage+passing+for+molecular+representation+learning%2E+In+Proceedings+of+the+AAAI+Conference+on+)
33. Pei, H., Xiong, Y., Wang, P., Tao, J., Liu, J., Deng, H., Ma, J., and Guan, X. Memory disagreement: A pseudo-labeling measure from training dynamics for semi-supervised graph learning. In Proceedings of the ACM on Web Conference 2024, pp. 434–445, 2024b. [find](https://scholar.google.com/scholar?q=Pei%2C+H%2E%2C+Xiong%2C+Y%2E%2C+Wang%2C+P%2E%2C+Tao%2C+J%2E%2C+Liu%2C+J%2E%2C+Deng%2C+H%2E%2C+Ma%2C+J%2E%2C+and+Guan%2C+X%2E+Memory+disagreement%3A+A+pseudo%2Dlabeling+measure+from+training+dynamics+for+semi%2Dsupervised+graph+learning%2E+In+Proceedings+)
34. Romera-Paredes, B., Barekatain, M., Novikov, A., Balog, M., Kumar, M. P., Dupont, E., Ruiz, F. J., Ellenberg, J. S., Wang, P., Fawzi, O., et al. [Mathematical discoveries from program search with large language models](https://md.research.elamur.ai/free/en/papers/paredes-2023-mathematical-discoveries-program-search-large/). Nature, pp. 1–3, 2023.
35. Rubin, O., Herzig, J., and Berant, J. [Learning to retrieve prompts for in-context learning](https://md.research.elamur.ai/free/en/papers/rubin-2021-learning-retrieve-prompts-context-learning/). In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2655–2671, 2022.
36. Shen, Y., Song, K., Tan, X., Li, D., Lu, W., and Zhuang, Y. Hugginggpt: Solving ai tasks with chatgpt and its friends in huggingface. In Advances in Neural Information Processing Systems, 2023. [find](https://scholar.google.com/scholar?q=Shen%2C+Y%2E%2C+Song%2C+K%2E%2C+Tan%2C+X%2E%2C+Li%2C+D%2E%2C+Lu%2C+W%2E%2C+and+Zhuang%2C+Y%2E+Hugginggpt%3A+Solving+ai+tasks+with+chatgpt+and+its+friends+in+huggingface%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+2023%2E)
37. Shi, W., Min, S., Yasunaga, M., Seo, M., James, R., Lewis, M., Zettlemoyer, L., and Yih, W.-t. [Replug: Retrieval-augmented black-box language models](https://md.research.elamur.ai/free/en/papers/shi-2023-replug-retrieval-augmented-black-box/). arXiv preprint arXiv:2301.12652, 2023.
38. Shinn, N., Cassano, F., Gopinath, A., Narasimhan, K. R., and Yao, S. [Reflexion: Language agents with verbal reinforcement learning](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/). In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
39. Significant Gravitas. AutoGPT, 2023. URL [https://github.com/Significant-Gravitas/](https://github.com/Significant-Gravitas/) AutoGPT. [↗](https://github.com/Significant-Gravitas/)
40. Sourati, Z., Ilievski, F., Sandlin, H.-Â., and Mermoud, A. Case-based reasoning with language models for classification of logical fallacies. arXiv preprint arXiv:2301.11879, 2023. — [arXiv:2301.11879](https://arxiv.org/abs/2301.11879) [find](https://scholar.google.com/scholar?q=Sourati%2C+Z%2E%2C+Ilievski%2C+F%2E%2C+Sandlin%2C+H%2E%2DA%CC%82%2E%2C+and+Mermoud%2C+A%2E+Case%2Dbased+reasoning+with+language+models+for+classification+of+logical+fallacies%2E+arXiv+preprint+arXiv%3A2301%2E11879%2C+2023%2E+%E2%80%94+%5BarXiv%3A2301%2E1187)
41. Sun, W., Yan, L., Ma, X., Wang, S., Ren, P., Chen, Z., Yin, D., and Ren, Z. [Is ChatGPT good at search? investigating large language models as re-ranking agents](https://md.research.elamur.ai/free/en/papers/sun-2023-chatgpt-good-search-investigating-large/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 14918–14937, 2023.
42. Wang, L., Yang, N., and Wei, F. [Learning to retrieve in-context examples for large language models](https://md.research.elamur.ai/free/en/papers/wang-2024-learning-retrieve-context-examples-large/). arXiv preprint arXiv:2307.07164, 2023a.
43. Wang, L., Yang, N., and Wei, F. [Learning to retrieve in-context examples for large language models](https://md.research.elamur.ai/free/en/papers/wang-2024-learning-retrieve-context-examples-large/). arXiv preprint arXiv:2307.07164, 2023b.
44. Wang, Z., Cai, S., Chen, G., Liu, A., Ma, X., and Liang, Y. Describe, explain, plan and select: Interactive planning with LLMs enables open-world multi-task agents. In Thirty-seventh Conference on Neural Information Processing Systems, 2023c. URL [https://openreview.net/forum?id=KtvPdGb31Z](https://openreview.net/forum?id=KtvPdGb31Z). [↗](https://openreview.net/forum?id=KtvPdGb31Z)
45. Watson, I. and Marir, F. Case-based reasoning: A review. The knowledge engineering review, 9(4):327–354, 1994. [find](https://scholar.google.com/scholar?q=Watson%2C+I%2E+and+Marir%2C+F%2E+Case%2Dbased+reasoning%3A+A+review%2E+The+knowledge+engineering+review%2C+9%284%29%3A327%E2%80%93354%2C+1994%2E)
46. Wu, Q., Bansal, G., Zhang, J., Wu, Y., Li, B., Zhu, E., Jiang, L., Zhang, X., Zhang, S., Liu, J., Awadallah, A. H., White, R. W., Burger, D., and Wang, C. [Autogen: Enabling next-gen llm applications via multi-agent conversation](https://md.research.elamur.ai/free/en/papers/wu-2023-autogen-enabling-next-gen-llm/) framework. 2023.
47. Yang, Z., Du, X., Cambria, E., and Cardie, C. End-to-end case-based reasoning for commonsense knowledge base completion. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pp. 3491–3504, 2023. [find](https://scholar.google.com/scholar?q=Yang%2C+Z%2E%2C+Du%2C+X%2E%2C+Cambria%2C+E%2E%2C+and+Cardie%2C+C%2E+End%2Dto%2Dend+case%2Dbased+reasoning+for+commonsense+knowledge+base+completion%2E+In+Proceedings+of+the+17th+Conference+of+the+European+Chapter+of+the+Associatio)
48. Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K. R., and Cao, Y. [React: Synergizing reasoning and acting in language models](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/). In The Eleventh International Conference on Learning Representations, 2022.
49. Zeng, A., Liu, M., Lu, R., Wang, B., Liu, X., Dong, Y., and Tang, J. [Agenttuning: Enabling generalized agent abilities for llms](https://md.research.elamur.ai/free/en/papers/zeng-2023-agenttuning-enabling-generalized-agent/). arXiv preprint arXiv:2310.12823, 2023.
50. Zhang, L., Zhang, Y., Ren, K., Li, D., and Yang, Y. [Mlcopilot: Unleashing the power of large language models in solving machine learning tasks](https://md.research.elamur.ai/free/en/papers/zhang-2024-mlcopilot-unleashing-power-large-language/). arXiv preprint arXiv:2304.14979, 2023a.
51. Zhang, P., Xiao, S., Liu, Z., Dou, Z., and Nie, J.-Y. Retrieve anything to augment large language models. arXiv preprint arXiv:2310.07554, 2023b. — [arXiv:2310.07554](https://arxiv.org/abs/2310.07554) [find](https://scholar.google.com/scholar?q=Zhang%2C+P%2E%2C+Xiao%2C+S%2E%2C+Liu%2C+Z%2E%2C+Dou%2C+Z%2E%2C+and+Nie%2C+J%2E%2DY%2E+Retrieve+anything+to+augment+large+language+models%2E+arXiv+preprint+arXiv%3A2310%2E07554%2C+2023b%2E+%E2%80%94+%5BarXiv%3A2310%2E07554%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2310%2E07554%29)
52. Zhang, S., Gong, C., Wu, L., Liu, X., and Zhou, M. [Automl-gpt: Automatic machine learning with gpt](https://md.research.elamur.ai/free/en/papers/zhang-2023-automl-gpt-automatic-machine-learning/). arXiv preprint arXiv:2305.02499, 2023c.
53. Zhao, A., Huang, D., Xu, Q., Lin, M., Liu, Y.-J., and Huang, G. Expel: Llm agents are experiential learners. Thirty-Eighth AAAI Conference on Artificial Intelligence, 2024. [find](https://scholar.google.com/scholar?q=Zhao%2C+A%2E%2C+Huang%2C+D%2E%2C+Xu%2C+Q%2E%2C+Lin%2C+M%2E%2C+Liu%2C+Y%2E%2DJ%2E%2C+and+Huang%2C+G%2E+Expel%3A+Llm+agents+are+experiential+learners%2E+Thirty%2DEighth+AAAI+Conference+on+Artificial+Intelligence%2C+2024%2E)

*Updated 3 November 2026*