---
title: "promptolution — paper by Zehle et al. (2026)"
description: "The authors present promptolution, an open-source Python framework that combines LLM interfaces… 54 references, cited by 2 papers here."
authors: ["Tom Zehle", "Timo Heiß", "Moritz Schlager", "Matthias Aßenmacher", "Matthias Feurer"]
year: 2026
venue: "Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 3: System Demonstrations)"
source: "https://aclanthology.org/2026.eacl-demo.21/"
doi: "10.18653/v1/2026.eacl-demo.21"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zehle-2026-promptolution-unified-modular-framework-prompt/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zehle-2026-promptolution-unified-modular-framework-prompt/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/8/) / promptolution: A Unified, Modular Framework for Prompt Optimization

# promptolution: A Unified, Modular Framework for Prompt Optimization

**Authors:** Tom Zehle, Timo Heiß, Moritz Schlager, [Matthias Aßenmacher](https://md.research.elamur.ai/free/en/authors/matthias-assenmacher/) and Matthias Feurer · **Year:** 2026 · **Venue:** [Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 3: System Demonstrations)](https://md.research.elamur.ai/free/en/venues/eacl/) ([EACL 2026](https://md.research.elamur.ai/free/en/venues/eacl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.eacl-demo.21/) · [PDF](https://aclanthology.org/2026.eacl-demo.21.pdf) · [DOI 10.18653/v1/2026.eacl-demo.21](https://doi.org/10.18653/v1/2026.eacl-demo.21) · [OpenAlex](https://openalex.org/W7140120597)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · 2 in our library, of the 149,439 papers whose bibliographies we read

## Summary
Automatic prompt optimization can substantially boost LLM task performance, but it is hard to adopt. Most optimizers live in isolated, poorly maintained research repositories. Existing libraries are closed-source, implement only one algorithm, or force applications into their own abstractions (as DSPy does). The authors present promptolution, an open-source Python framework that combines LLM interfaces, NLP tasks, evaluation metrics and several discrete prompt optimizers in one modular design where components are interchangeable.

The user supplies a development set, few-shot examples, a task description, token budget limits and optional starting prompts. A chosen optimizer then rewrites candidate prompts iteratively, typically with a meta-LLM, and logs the whole trajectory through callbacks. The result is a plain prompt string that can replace the old prompt in any pipeline, regardless of model provider.

The authors report that comparisons against unoptimized prompts and other libraries confirm the tool's utility and competitiveness. Its low abstraction level and support for reproducible, multi-seed experiments make it especially suited to benchmarking prompt optimizers.

## Key points
- Implemented optimizers are OPRO, EvoPrompt (genetic-algorithm and differential-evolution variants) and CAPO, which tunes instructions and few-shot examples together. GEPA, TextGrad, MIPRO and PromptWizard are named as not yet included.
- Prompt sensitivity example: two nearly synonymous GSM8K prompts run with Llama-3.3-70B scored 37.6% and 53.8% test accuracy.
- The authors compare promptolution with DSPy, CoolPrompt, promptomatix, prompt-ops, TextGrad, AdalFlow, PromptWizard and PromptIM on four criteria: multiple optimizers, extensibility, low abstraction and non-invasive integration. Only promptolution meets all four.
- Recommended uses are improving prompts in existing pipelines, benchmarking optimizers and adding prompt tuning to LLM benchmarks. For end-to-end application development, the authors consider DSPy the better fit.
- Planned work includes DSPy and hydra interfaces, a graphical experiment-tracking interface, multi-agent prompt optimization and optimizer ensembling in the style of ELPO. The code is hosted at AutoML/Promptolution.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Ludwig-Maximilians-Universität München](https://md.research.elamur.ai/free/en/universities/ludwig-maximilians-universitat-munchen/) — University, Munich, Germany · 107 works · 56,478 citations
- [TU Dortmund University](https://md.research.elamur.ai/free/en/universities/tu-dortmund-university/) — University, Dortmund, Germany · 7 works · 9,785 citations
- [Munich Center for Machine Learning](https://md.research.elamur.ai/free/en/universities/munich-center-for-machine-learning/) — University, Munich, Germany · 57 works · 1,094 citations
- [Lamarr Institute for Machine Learning and Artificial Intelligence](https://md.research.elamur.ai/free/en/labs/lamarr-institute-for-machine-learning-and-artificial-intelligence/) — Research institute, Sankt Augustin, Germany · 6 works · 121 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

## Cite

Cite as: Tom Zehle, Timo Heiß, Moritz Schlager, Matthias Aßenmacher and Matthias Feurer. 2026. promptolution: A Unified, Modular Framework for Prompt Optimization. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 3: System Demonstrations).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zehle2026promptolution,
  title = {{promptolution: A Unified, Modular Framework for Prompt Optimization}},
  author = {Tom Zehle, Timo Heiß, Moritz Schlager, Matthias Aßenmacher and Matthias Feurer},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 3: System Demonstrations)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-demo.21},
  url = {https://doi.org/10.18653/v1/2026.eacl-demo.21}
}
```

- [← previous Prompting Instruction-tuned LLMs for Semantic Similarity Values](https://md.research.elamur.ai/free/en/papers/snelder-2026-prompting-instruction-tuned-llms-semantic/)
- [next → Prune as You Generate: Online Rollout Pruning for Faster and Better RLVR](https://md.research.elamur.ai/free/en/papers/xu-2026-prune-as-you-generate-online/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/8/)

## References

54 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Adalflow. 2025. Build and Optimize LM Workflows. Last accessed: 11/30/2025. [find](https://scholar.google.com/scholar?q=Adalflow%2E+2025%2E+Build+and+Optimize+LM+Workflows%2E+Last+accessed%3A+11%2F30%2F2025%2E)
2. Meng Jiang, Christopher Potts, Koushik Sen, Alexandros G. Dimakis, Ion Stoica, Dan Klein, Matei Zaharia, and Omar Khattab. 2025. GEPA: Reflective prompt evolution can outperform reinforcement learning. arXiv:2507.19457 [cs.CL]. [find](https://scholar.google.com/scholar?q=Meng+Jiang%2C+Christopher+Potts%2C+Koushik+Sen%2C+Alexandros+G%2E+Dimakis%2C+Ion+Stoica%2C+Dan+Klein%2C+Matei+Zaharia%2C+and+Omar+Khattab%2E+2025%2E+GEPA%3A+Reflective+prompt+evolution+can+outperform+reinforcement+learning)
3. Amazon Bedrock. 2025. User Guide: Optimize a prompt. Last accessed: 11/25/2025. [find](https://scholar.google.com/scholar?q=Amazon+Bedrock%2E+2025%2E+User+Guide%3A+Optimize+a+prompt%2E+Last+accessed%3A+11%2F25%2F2025%2E)
4. Anthropic. 2025. Prompt engineering: Use our prompt improver to optimize your prompts. Last accessed: 11/25/2025. [find](https://scholar.google.com/scholar?q=Anthropic%2E+2025%2E+Prompt+engineering%3A+Use+our+prompt+improver+to+optimize+your+prompts%2E+Last+accessed%3A+11%2F25%2F2025%2E)
5. Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, and 29 others. 2023. [Qwen technical report](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-technical-report/). arXiv:2309.16609 [cs.CL].
6. T. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, and 12 others. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). In Proceedings of the…
7. Jiale Cheng, Xiao Liu, Kehan Zheng, Pei Ke, Hongning Wang, Yuxiao Dong, Jie Tang, and Minlie Huang. 2024. [Black-box prompt optimization: Aligning large language models without model training](https://md.research.elamur.ai/free/en/papers/cheng-2024-black-box-prompt-optimization-aligning/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long P…
8. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv:2110.14168 [cs.LG].
9. Wendi Cui, Jiaxin Zhang, Zhuohang Li, Hao Sun, Damien Lopez, Kamalika Das, Bradley A. Malin, and Sricharan Kumar. 2025. Heuristic-based search algorithm in automatic instruction-focused prompt optimization: A survey. In Findings of the Association for Computational Linguistics: ACL 2025, pages 22093… [find](https://scholar.google.com/scholar?q=Wendi+Cui%2C+Jiaxin+Zhang%2C+Zhuohang+Li%2C+Hao+Sun%2C+Damien+Lopez%2C+Kamalika+Das%2C+Bradley+A%2E+Malin%2C+and+Sricharan+Kumar%2E+2025%2E+Heuristic%2Dbased+search+algorithm+in+automatic+instruction%2Dfocused+prompt+optimiz)
10. Eshaan Agarwal, Joykirat Singh, Vivek Dani, Raghav Magazine, Tanuja Ganu, and Akshay Nambi. 2024. PromptWizard: Task-aware prompt optimization framework. arXiv:2405.18369 [cs.CL]. — [arXiv:2405.18369](https://arxiv.org/abs/2405.18369) [find](https://scholar.google.com/scholar?q=Eshaan+Agarwal%2C+Joykirat+Singh%2C+Vivek+Dani%2C+Raghav+Magazine%2C+Tanuja+Ganu%2C+and+Akshay+Nambi%2E+2024%2E+PromptWizard%3A+Task%2Daware+prompt+optimization+framework%2E+arXiv%3A2405%2E18369+%5Bcs%2ECL%5D%2E+%E2%80%94+%5BarXiv%3A2405%2E18369%5D)
11. Ning Ding, Shengding Hu, Weilin Zhao, Yulin Chen, Zhiyuan Liu, Haitao Zheng, and Maosong Sun. 2022. [OpenPrompt: An open-source framework for prompt-learning](https://md.research.elamur.ai/free/en/papers/ding-2022-openprompt-open-source-framework-prompt/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 105–113, Dublin, Irel…
12. Lakshya A. Agrawal, Shangyin Tan, Dilara Soylu, Noah Ziems, Rishi Khare, Krista Opsahl-Ong, Arnav Singhvi, Herumb Shandilya, Michael J. Ryan, Matthias Feurer and Frank Hutter. 2019. Hyperparameter optimization. In Frank Hutter, Lars Kotthoff, and Joaquin Vanschoren, editors, Automated Machine Learni… [find](https://scholar.google.com/scholar?q=Lakshya+A%2E+Agrawal%2C+Shangyin+Tan%2C+Dilara+Soylu%2C+Noah+Ziems%2C+Rishi+Khare%2C+Krista+Opsahl%2DOng%2C+Arnav+Singhvi%2C+Herumb+Shandilya%2C+Michael+J%2E+Ryan%2C+Matthias+Feurer+and+Frank+Hutter%2E+2019%2E+Hyperparameter+opt)
13. Qingyan Guo, Rui Wang, Junliang Guo, Bei Li, Kaitao Song, Xu Tan, Guoqing Liu, Jiang Bian, and Yujiu Yang. 2024. Connecting large language models with evolutionary algorithms yields powerful prompt optimizers. In The Twelfth International Conference on Learning Representations (ICLR’24). ICLR. Publi… [find](https://scholar.google.com/scholar?q=Qingyan+Guo%2C+Rui+Wang%2C+Junliang+Guo%2C+Bei+Li%2C+Kaitao+Song%2C+Xu+Tan%2C+Guoqing+Liu%2C+Jiang+Bian%2C+and+Yujiu+Yang%2E+2024%2E+Connecting+large+language+models+with+evolutionary+algorithms+yields+powerful+prompt+op)
14. Junda He, Christoph Treude, and David Lo. 2025. LLM-based multi-agent systems for software engineering: Literature review, vision, and the road ahead. ACM Transactions on Software Engineering and Methodology, 34(5):1–30. [find](https://scholar.google.com/scholar?q=Junda+He%2C+Christoph+Treude%2C+and+David+Lo%2E+2025%2E+LLM%2Dbased+multi%2Dagent+systems+for+software+engineering%3A+Literature+review%2C+vision%2C+and+the+road+ahead%2E+ACM+Transactions+on+Software+Engineering+and+Meth)
15. William F. Hinthorn and Masahiro Nishimi. 2025. Promptim. Last accessed: 11/25/2025. [find](https://scholar.google.com/scholar?q=William+F%2E+Hinthorn+and+Masahiro+Nishimi%2E+2025%2E+Promptim%2E+Last+accessed%3A+11%2F25%2F2025%2E)
16. Zhengbao Jiang, Frank Xu, Jun Araki, and Graham Neubig. 2020. [How can we know what language models know?](https://md.research.elamur.ai/free/en/papers/jiang-2020-how-can-we-know-what/) Transactions of the Association for Computational Linguistics, 8:423–438.
17. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. [Efficient memory management for large language model serving with pagedattention](https://md.research.elamur.ai/free/en/papers/kwon-2023-efficient-memory-management-large-language/). In Proceedings of the 29th Symposium on Operating Systems Principles (SOSP ’23), pages 611–6…
18. George Lee and Ivan Nardini. 2024. Announcing Public Preview of Vertex AI Prompt Optimizer. Last accessed: 11/25/2025. [find](https://scholar.google.com/scholar?q=George+Lee+and+Ivan+Nardini%2E+2024%2E+Announcing+Public+Preview+of+Vertex+AI+Prompt+Optimizer%2E+Last+accessed%3A+11%2F25%2F2025%2E)
19. Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. [The power of scale for parameter-efficient prompt tuning](https://md.research.elamur.ai/free/en/papers/lester-2021-power-scale-parameter-efficient-prompt/). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3045–3059. Association for Computational Linguistics.
20. Wenwu Li, Xiangfeng Wang, Wenhao Li, and Bo Jin. 2025. A survey of automatic prompt engineering: An optimization perspective. arXiv:2502.11560 [cs.AI]. — [arXiv:2502.11560](https://arxiv.org/abs/2502.11560) [find](https://scholar.google.com/scholar?q=Wenwu+Li%2C+Xiangfeng+Wang%2C+Wenhao+Li%2C+and+Bo+Jin%2E+2025%2E+A+survey+of+automatic+prompt+engineering%3A+An+optimization+perspective%2E+arXiv%3A2502%2E11560+%5Bcs%2EAI%5D%2E+%E2%80%94+%5BarXiv%3A2502%2E11560%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2502%2E)
21. Jina AI. 2025. PromptPerfect: AI Prompt Optimizer. Last accessed: 11/25/2025. [find](https://scholar.google.com/scholar?q=Jina+AI%2E+2025%2E+PromptPerfect%3A+AI+Prompt+Optimizer%2E+Last+accessed%3A+11%2F25%2F2025%2E)
22. Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean-bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penchev, Gaël Liu,… [find](https://scholar.google.com/scholar?q=Aishwarya+Kamath%2C+Johan+Ferret%2C+Shreya+Pathak%2C+Nino+Vieillard%2C+Ramona+Merhej%2C+Sarah+Perrin%2C+Tatiana+Matejovicova%2C+Alexandre+Ram%C3%A9%2C+Morgane+Rivi%C3%A8re%2C+Louis+Rouillard%2C+Thomas+Mesnard%2C+Geoffrey+Cideron%2C+Je)
23. Shubhra Karmaker Santu and Dongji Feng. 2023. TELeR: A general taxonomy of LLM prompts for benchmarking complex tasks. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 14197–14203. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Shubhra+Karmaker+Santu+and+Dongji+Feng%2E+2023%2E+TELeR%3A+A+general+taxonomy+of+LLM+prompts+for+benchmarking+complex+tasks%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+2023%2C+pages+1)
24. Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. 2024. DSPy: Compiling declarative language model calls into state-of-the-art pipelines. In… [find](https://scholar.google.com/scholar?q=Omar+Khattab%2C+Arnav+Singhvi%2C+Paridhi+Maheshwari%2C+Zhiyuan+Zhang%2C+Keshav+Santhanam%2C+Sri+Vardhamanan%2C+Saiful+Haq%2C+Ashutosh+Sharma%2C+Thomas+Joshi%2C+Hanna+Moazam%2C+Heather+Miller%2C+Matei+Zaharia%2C+and+Christoph)
25. Ron Kohavi and George H. John. 1995. Automatic parameter selection by minimizing estimated error. In Proceedings of the Twelfth International Conference on Machine Learning (ICML’95). Morgan Kaufmann Publishers. [find](https://scholar.google.com/scholar?q=Ron+Kohavi+and+George+H%2E+John%2E+1995%2E+Automatic+parameter+selection+by+minimizing+estimated+error%2E+In+Proceedings+of+the+Twelfth+International+Conference+on+Machine+Learning+%28ICML%E2%80%9995%29%2E+Morgan+Kaufmann+)
26. Nikita Kulin, Viktor Zhuravlev, Artur Khairullin, Alena Sitkina, and Sergey Muravyov. 2025. CoolPrompt: Automatic prompt optimization framework for Large Language Models. In Proceedings of the 38th Conference of Open Innovations Association FRUCT, Issue 1 (Full papers), pages 158–166, Helsinki, Finl… [find](https://scholar.google.com/scholar?q=Nikita+Kulin%2C+Viktor+Zhuravlev%2C+Artur+Khairullin%2C+Alena+Sitkina%2C+and+Sergey+Muravyov%2E+2025%2E+CoolPrompt%3A+Automatic+prompt+optimization+framework+for+Large+Language+Models%2E+In+Proceedings+of+the+38th+Co)
27. Xiang Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4582– 4597… [find](https://scholar.google.com/scholar?q=Xiang+Li+and+Percy+Liang%2E+2021%2E+Prefix%2Dtuning%3A+Optimizing+continuous+prompts+for+generation%2E+In+Proceedings+of+the+59th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+and+the+11th+Int)
28. Xinyi Li, Sai Wang, Siqi Zeng, Yu Wu, and Yi Yang. 2024. A survey on LLM-based multi-agent systems: workflow, infrastructure, and challenges. Vicinagearth, 1(1):9. [find](https://scholar.google.com/scholar?q=Xinyi+Li%2C+Sai+Wang%2C+Siqi+Zeng%2C+Yu+Wu%2C+and+Yi+Yang%2E+2024%2E+A+survey+on+LLM%2Dbased+multi%2Dagent+systems%3A+workflow%2C+infrastructure%2C+and+challenges%2E+Vicinagearth%2C+1%281%29%3A9%2E)
29. Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023. [Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing](https://md.research.elamur.ai/free/en/papers/liu-2021-pre-train-prompt-predict-systematic/). ACM Computing Surveys, 55(9):195:1–195:35.
30. Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. 2022. [Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity](https://md.research.elamur.ai/free/en/papers/lu-2022-fantastically-ordered-prompts-where-find/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p…
31. Meta-Llama. 2025. prompt-ops: An open-source tool for LLM prompt optimization. Last accessed: 11/30/2025. [find](https://scholar.google.com/scholar?q=Meta%2DLlama%2E+2025%2E+prompt%2Dops%3A+An+open%2Dsource+tool+for+LLM+prompt+optimization%2E+Last+accessed%3A+11%2F30%2F2025%2E)
32. Rithesh Murthy, Ming Zhu, Liangwei Yang, Jielin Qiu, Juntao Tan, Shelby Heinecke, Caiming Xiong, Silvio Savarese, and Huan Wang. 2025. Promptomatix: An automatic prompt optimization framework for Large Language Models. arXiv:2507.14241 [cs.CL]. — [arXiv:2507.14241](https://arxiv.org/abs/2507.14241) [find](https://scholar.google.com/scholar?q=Rithesh+Murthy%2C+Ming+Zhu%2C+Liangwei+Yang%2C+Jielin+Qiu%2C+Juntao+Tan%2C+Shelby+Heinecke%2C+Caiming+Xiong%2C+Silvio+Savarese%2C+and+Huan+Wang%2E+2025%2E+Promptomatix%3A+An+automatic+prompt+optimization+framework+for+Larg)
33. Krista Opsahl-Ong, Michael J Ryan, Josh Purtell, David Broman, Christopher Potts, Matei Zaharia, and Omar Khattab. 2024. [Optimizing instructions and demonstrations for multi-stage language model programs](https://md.research.elamur.ai/free/en/papers/ong-2024-optimizing-instructions-demonstrations-multi-stage/).
34. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9340–9366. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=In+Proceedings+of+the+2024+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP%29%2C+pages+9340%E2%80%939366%2E+Association+for+Computational+Linguistics%2E)
35. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. [Train…](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/)
36. Guanghui Qin and Jason Eisner. 2021. [Learning how to ask: Querying LMs with mixtures of soft prompts](https://md.research.elamur.ai/free/en/papers/qin-2021-learning-ask-querying-lms-mixtures/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5203–5212. Association for Computational Ling…
37. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+and+Ilya+Sutskever%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E+OpenAI+blog%2C+1%288%29%3A9%2E)
38. Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher Manning, Andrew Ng, and Christopher Potts. 2013. [Recursive deep models for semantic compositionality over a sentiment treebank](https://md.research.elamur.ai/free/en/papers/socher-2013-recursive-deep-models-semantic-compositionality/). In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1631–164…
39. Dilara Soylu, Christopher Potts, and Omar Khattab. 2024. Fine-tuning and prompt optimization: Two great steps that work better together. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 10696–10710. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Dilara+Soylu%2C+Christopher+Potts%2C+and+Omar+Khattab%2E+2024%2E+Fine%2Dtuning+and+prompt+optimization%3A+Two+great+steps+that+work+better+together%2E+In+Proceedings+of+the+2024+Conference+on+Empirical+Methods+in+N)
40. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. [LLaMA: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). arXiv:23…
41. Xingchen Wan, Ruoxi Sun, Hootan Nakhost, and Sercan Arik. 2024. Teach better or show smarter? on instructions and exemplars in automatic prompt optimization. In Proceedings of the 37th International Conference on Advances in Neural Information Processing Systems (NeurIPS’24), pages 58174–58244. Curr… [find](https://scholar.google.com/scholar?q=Xingchen+Wan%2C+Ruoxi+Sun%2C+Hootan+Nakhost%2C+and+Sercan+Arik%2E+2024%2E+Teach+better+or+show+smarter%3F+on+instructions+and+exemplars+in+automatic+prompt+optimization%2E+In+Proceedings+of+the+37th+International+C)
42. Jules White, Quchen Fu, Sam Hays, Michael Sandborn, Carlos Olea, Henry Gilbert, Ashraf Elnashar, Jesse Spencer-Smith, and Douglas Schmidt. 2023. [A prompt pattern catalog to enhance prompt engineering with ChatGPT](https://md.research.elamur.ai/free/en/papers/white-2023-prompt-pattern-catalog-enhance-prompt/). In Proceedings of the 30th Conference on Pattern Languages of Programs, PLoP ’23, page…
43. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, and 3 others. 2020. [Transf…](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/)
44. Omry Yadan. 2019. Hydra - a framework for elegantly configuring complex applications. Last accessed: 11/30/2025. [find](https://scholar.google.com/scholar?q=Omry+Yadan%2E+2019%2E+Hydra+%2D+a+framework+for+elegantly+configuring+complex+applications%2E+Last+accessed%3A+11%2F30%2F2025%2E)
45. Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc Le, Denny Zhou, and Xinyun Chen. 2024. [Large language models as optimizers](https://md.research.elamur.ai/free/en/papers/yang-2023-large-language-models-optimizers/). In The Twelfth International Conference on Learning Representations (ICLR’24). ICLR. Published online: iclr.cc.
46. Li Yin and Zhangyang Wang. 2025. LLM-AutoDiff: Auto-differentiate any LLM workflow. arXiv:2501.16673 [cs.CL]. — [arXiv:2501.16673](https://arxiv.org/abs/2501.16673) [find](https://scholar.google.com/scholar?q=Li+Yin+and+Zhangyang+Wang%2E+2025%2E+LLM%2DAutoDiff%3A+Auto%2Ddifferentiate+any+LLM+workflow%2E+arXiv%3A2501%2E16673+%5Bcs%2ECL%5D%2E+%E2%80%94+%5BarXiv%3A2501%2E16673%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2501%2E16673%29)
47. Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Zhi Huang, Carlos Guestrin, and James Zou. 2024. TextGrad: Automatic “Differentiation” with Text. Last accessed: 11/30/2025. [find](https://scholar.google.com/scholar?q=Mert+Yuksekgonul%2C+Federico+Bianchi%2C+Joseph+Boen%2C+Sheng+Liu%2C+Zhi+Huang%2C+Carlos+Guestrin%2C+and+James+Zou%2E+2024%2E+TextGrad%3A+Automatic+%E2%80%9CDifferentiation%E2%80%9D+with+Text%2E+Last+accessed%3A+11%2F30%2F2025%2E)
48. Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Pan Lu, Zhi Huang, Carlos Guestrin, and James Zou. 2025. Optimizing generative AI by backpropagating language model feedback. Nature, 639(8055):609––616. [find](https://scholar.google.com/scholar?q=Mert+Yuksekgonul%2C+Federico+Bianchi%2C+Joseph+Boen%2C+Sheng+Liu%2C+Pan+Lu%2C+Zhi+Huang%2C+Carlos+Guestrin%2C+and+James+Zou%2E+2025%2E+Optimizing+generative+AI+by+backpropagating+language+model+feedback%2E+Nature%2C+639%2880)
49. Tom Zehle, Moritz Schlager, Timo Heiß, and Matthias Feurer. 2025. CAPO: Cost-aware prompt optimization. In Proceedings of the Fourth International Conference on Automated Machine Learning, volume 293 of Proceedings of Machine Learning Research, pages 18/1–45. PMLR. [find](https://scholar.google.com/scholar?q=Tom+Zehle%2C+Moritz+Schlager%2C+Timo+Hei%C3%9F%2C+and+Matthias+Feurer%2E+2025%2E+CAPO%3A+Cost%2Daware+prompt+optimization%2E+In+Proceedings+of+the+Fourth+International+Conference+on+Automated+Machine+Learning%2C+volume+293+)
50. Qing Zhang, Bing Xu, Xudong Zhang, Yifan Shi, Yang Li, Chen Zhang, Yik Chung Wu, Ngai Wong, Yijie Chen, Hong Dai, Xiansen Chen, and Mian Zhang. 2025. ELPO: Ensemble learning based prompt optimization for Large Language Models. arXiv:2511.16122 [cs.CL]. — [arXiv:2511.16122](https://arxiv.org/abs/2511.16122) [find](https://scholar.google.com/scholar?q=Qing+Zhang%2C+Bing+Xu%2C+Xudong+Zhang%2C+Yifan+Shi%2C+Yang+Li%2C+Chen+Zhang%2C+Yik+Chung+Wu%2C+Ngai+Wong%2C+Yijie+Chen%2C+Hong+Dai%2C+Xiansen+Chen%2C+and+Mian+Zhang%2E+2025%2E+ELPO%3A+Ensemble+learning+based+prompt+optimization+)
51. Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. [Calibrate before use: Improving few-shot performance of language models](https://md.research.elamur.ai/free/en/papers/zhao-2021-calibrate-before-use-improving-few/). In Proceedings of the 38th International Conference on Machine Learning (ICML’21), volume 139 of Proceedings of Machine Learning Research, pages 12697–12706…
52. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. [Judging LLM-as-a-judge with MT-bench and Chatbot Arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). In Proceedings of the 36th International Conference on Adv…
53. Y. Zhou, A. Ioan Muresanu, Z. Han, K. Paster, S. Pitis, H. Chan, and J. Ba. 2023. [Large language models are human-level prompt engineers](https://md.research.elamur.ai/free/en/papers/zhou-2022-large-language-models-human-level/). In The Eleventh International Conference on Learning Representations (ICLR’23). ICLR. Published online: iclr.cc.
54. Kaijie Zhu, Qinlin Zhao, Hao Chen, Jindong Wang, and Xing Xie. 2024. PromptBench: A unified library for evaluation of Large Language Models. Journal of Machine Learning Research, 25(254):1–22. [find](https://scholar.google.com/scholar?q=Kaijie+Zhu%2C+Qinlin+Zhao%2C+Hao+Chen%2C+Jindong+Wang%2C+and+Xing+Xie%2E+2024%2E+PromptBench%3A+A+unified+library+for+evaluation+of+Large+Language+Models%2E+Journal+of+Machine+Learning+Research%2C+25%28254%29%3A1%E2%80%9322%2E)

*Updated 31 October 2026*