---
title: "ExpertPrompting — paper by Xu et al. (2023)"
description: "The authors propose ExpertPrompting, a fully automatic technique. 15 references, cited by 73 papers here."
authors: ["Benfeng Xu", "An Yang", "Junyang Lin", "Quan Wang", "Chang Zhou", "Yongdong Zhang", "Zhendong Mao"]
year: 2023
venue: "arXiv:2305.14688"
source: "https://arxiv.org/abs/2305.14688"
doi: "10.48550/arxiv.2305.14688"
arxiv: "2305.14688"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/xu-2023-expertprompting-instructing-large-language-models/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/xu-2023-expertprompting-instructing-large-language-models/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/6/) / ExpertPrompting: Instructing Large Language Models to be Distinguished Experts

# ExpertPrompting: Instructing Large Language Models to be Distinguished Experts

**Authors:** [Benfeng Xu](https://md.research.elamur.ai/free/en/authors/benfeng-xu/), [An Yang](https://md.research.elamur.ai/free/en/authors/an-yang/), [Junyang Lin](https://md.research.elamur.ai/free/en/authors/junyang-lin/), [Quan Wang](https://md.research.elamur.ai/free/en/authors/quan-wang/), [Chang Zhou](https://md.research.elamur.ai/free/en/authors/chang-zhou/), [Yongdong Zhang](https://md.research.elamur.ai/free/en/authors/yongdong-zhang/) and [Zhendong Mao](https://md.research.elamur.ai/free/en/authors/zhendong-mao/) · **Year:** 2023 · **Venue:** arXiv:2305.14688 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2305.14688) · [PDF](https://arxiv.org/pdf/2305.14688) · [DOI 10.48550/arxiv.2305.14688](https://doi.org/10.48550/arxiv.2305.14688) · [arXiv 2305.14688](https://arxiv.org/abs/2305.14688) · [OpenAlex](https://openalex.org/W4378473736)

**Citations:** 55 citations (OpenAlex, 2026-10-14) · [73](https://md.research.elamur.ai/free/en/papers/xu-2023-expertprompting-instructing-large-language-models/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
How well an aligned LLM answers depends heavily on how the prompt is phrased, but existing prompting advice is either task-specific or needs manual adaptation. The authors propose ExpertPrompting, a fully automatic technique. For each instruction, the model first writes a detailed profile of a suitable specialist, such as a physicist for a question about atoms. It generates this profile through In-Context Learning from three hand-revised instruction–identity examples. It then answers while conditioned on that profile, and any self-referential openings are removed afterwards. The authors applied this to the 52k Alpaca instructions with gpt-3.5-turbo, producing a new instruction-following dataset. They used it to fine-tune LLaMA 7B with the Alpaca recipe, yielding ExpertLLaMA. Under GPT4-based judging, the expert-conditioned answers clearly beat plain ones. ExpertLLaMA outperforms open-source assistants such as Vicuna, Alpaca and LLaMA-GPT4 and reaches roughly 96% of ChatGPT's score on Vicuna80, though the authors say this figure needs firmer validation. The method matters because it improves both data and model quality cheaply across domains, without elaborate templates or iterative reasoning loops.

## Key points
- Expert identities are generated with k = 3 in-context exemplars. Expert answers average 138.30 words, compared with 108.44 for vanilla prompting and 108.67 for a fixed "act as an expert" baseline (Static DESC).
- On 500 sampled instructions, GPT4 preferred ExpertPrompting answers in 48.5% of cases and vanilla answers in 23% (243 wins, 143 ties, 115 losses). Answer order was randomized to reduce bias.
- ExpertLLaMA (LLaMA 7B) was compared on the unseen Vicuna80 set against Alpaca, LLaMA-GPT4, LLaMA-GPT-3.5, LLaMA-GPT-3.5+, Vicuna (13B), ChatGPT and Bard. It ranked behind only ChatGPT, at about 96% of its capability.
- Data and model are released at github.com/OFA-Sys/ExpertLLaMA. The authors plan to scale beyond the 52k Alpaca instructions.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Science and Technology of China](https://md.research.elamur.ai/free/en/universities/university-of-science-and-technology-of-china/) — University, Hefei, China · 145 works · 114,409 citations
- [Beijing University of Posts and Telecommunications](https://md.research.elamur.ai/free/en/universities/beijing-university-of-posts-and-telecommunications/) — University, Beijing, China · 78 works · 33,365 citations
- [Alibaba DAMO Academy](https://md.research.elamur.ai/free/en/companies/alibaba-damo-academy/) — Company, Hangzhou, China · 45 works · 6,874 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works

## Cite

Cite as: Benfeng Xu, An Yang, Junyang Lin, Quan Wang, Chang Zhou, Yongdong Zhang and Zhendong Mao. 2023. ExpertPrompting: Instructing Large Language Models to be Distinguished Experts. arXiv:2305.14688.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{xu2023expertprompting,
  title = {{ExpertPrompting: Instructing Large Language Models to be Distinguished Experts}},
  author = {Benfeng Xu, An Yang, Junyang Lin, Quan Wang, Chang Zhou, Yongdong Zhang and Zhendong Mao},
  year = {2023},
  doi = {10.48550/arxiv.2305.14688},
  eprint = {2305.14688},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2305.14688}
}
```

- [← previous Emergent autonomous scientific research capabilities of large language models](https://md.research.elamur.ai/free/en/papers/boiko-2023-emergent-autonomous-scientific-research/)
- [next → Human-like Summarization Evaluation with ChatGPT](https://md.research.elamur.ai/free/en/papers/gao-2023-human-like-summarization-evaluation-chatgpt/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/6/)

## References

15 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Fatih Kadir Akın and Contributors. 2023. Awesome chatgpt prompts. [https://github.com/f/awesome-chatgpt-prompts](https://github.com/f/awesome-chatgpt-prompts). [↗](https://github.com/f/awesome-chatgpt-prompts)
2. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. 2022. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862. [find](https://scholar.google.com/scholar?q=Yuntao+Bai%2C+Andy+Jones%2C+Kamal+Ndousse%2C+Amanda+Askell%2C+Anna+Chen%2C+Nova+DasSarma%2C+Dawn+Drain%2C+Stanislav+Fort%2C+Deep+Ganguli%2C+Tom+Henighan%2C+et+al%2E+2022%2E+Training+a+helpful+and+harmless+assistant+with+rein)
3. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chri… — [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
4. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. [find](https://scholar.google.com/scholar?q=Wei%2DLin+Chiang%2C+Zhuohan+Li%2C+Zi+Lin%2C+Ying+Sheng%2C+Zhanghao+Wu%2C+Hao+Zhang%2C+Lianmin+Zheng%2C+Siyuan+Zhuang%2C+Yonghao+Zhuang%2C+Joseph+E%2E+Gonzalez%2C+Ion+Stoica%2C+and+Eric+P%2E+Xing%2E+2023%2E+Vicuna%3A+An+open%2Dsource+cha)
5. Isa Fulford and Andrew Ng. 2023. Chatgpt prompt engineering for developers. Accessed on 17 May 2023. [find](https://scholar.google.com/scholar?q=Isa+Fulford+and+Andrew+Ng%2E+2023%2E+Chatgpt+prompt+engineering+for+developers%2E+Accessed+on+17+May+2023%2E)
6. Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2023. [Camel: Communicative agents for" mind" exploration of large scale language model society](https://md.research.elamur.ai/free/en/papers/li-2023-camel-communicative-agents-mind/). arXiv preprint arXiv:2303.17760.
7. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Gray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. Trai… — [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/)
8. Joon Sung Park, Joseph C O’Brien, Carrie J Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. arXiv preprint arXiv:2304.03442. [find](https://scholar.google.com/scholar?q=Joon+Sung+Park%2C+Joseph+C+O%E2%80%99Brien%2C+Carrie+J+Cai%2C+Meredith+Ringel+Morris%2C+Percy+Liang%2C+and+Michael+S+Bernstein%2E+2023%2E+Generative+agents%3A+Interactive+simulacra+of+human+behavior%2E+arXiv+preprint+arXiv%3A230)
9. Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. 2023. [Instruction tuning with gpt-4](https://md.research.elamur.ai/free/en/papers/peng-2023-instruction-tuning-gpt-4/). arXiv preprint arXiv:2304.03277.
10. Noah Shinn, Beck Labash, and Ashwin Gopinath. 2023. [Reflexion: an autonomous agent with dynamic memory and self-reflection](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/). arXiv preprint arXiv:2303.11366.
11. Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction-following llama model. [https://github.com/tatsu-lab/stanford_alpaca](https://github.com/tatsu-lab/stanford_alpaca). [↗](https://github.com/tatsu-lab/stanford_alpaca)
12. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). arXiv pr…
13. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. 2022. Self-instruct: Aligning language model with self generated instructions. [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Yeganeh+Kordi%2C+Swaroop+Mishra%2C+Alisa+Liu%2C+Noah+A%2E+Smith%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+2022%2E+Self%2Dinstruct%3A+Aligning+language+model+with+self+generated+instructions%2E)
14. Benfeng Xu, Quan Wang, Zhendong Mao, Yajuan Lyu, Qiaoqiao She, and Yongdong Zhang. 2023. $k$NN prompting: Beyond-context learning with calibration-free nearest neighbor inference. In The Eleventh International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Benfeng+Xu%2C+Quan+Wang%2C+Zhendong+Mao%2C+Yajuan+Lyu%2C+Qiaoqiao+She%2C+and+Yongdong+Zhang%2E+2023%2E+%24k%24NN+prompting%3A+Beyond%2Dcontext+learning+with+calibration%2Dfree+nearest+neighbor+inference%2E+In+The+Eleventh+Inte)
15. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. [ReAct: Synergizing reasoning and acting in language models](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/). In International Conference on Learning Representations (ICLR).

*Updated 2 November 2026*