---
title: "Unleashing the Emergent Cognitive Synergy in — Wang 2023"
description: "Large language models still make factual mistakes on knowledge-heavy tasks and stumble on problems that need slow, multi-step reasoning."
authors: ["Zhenhailong Wang", "Shaoguang Mao", "Wenshan Wu", "Tao Ge", "Furu Wei", "Heng Ji"]
year: 2023
venue: "arXiv:2307.05300"
source: "https://arxiv.org/abs/2307.05300"
doi: "10.48550/arxiv.2307.05300"
arxiv: "2307.05300"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wang-2023-unleashing-emergent-cognitive-synergy-large/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2023-unleashing-emergent-cognitive-synergy-large/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/5/) / Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration

# Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration

**Authors:** Zhenhailong Wang, [Shaoguang Mao](https://md.research.elamur.ai/free/en/authors/shaoguang-mao/), Wenshan Wu, [Tao Ge](https://md.research.elamur.ai/free/en/authors/tao-ge/), [Furu Wei](https://md.research.elamur.ai/free/en/authors/furu-wei/), [Heng Ji](https://md.research.elamur.ai/free/en/authors/heng-ji/) · **Institutions:** [University of Illinois Urbana-Champaign](https://md.research.elamur.ai/free/en/universities/university-of-illinois-urbana-champaign/) · [Microsoft Research Asia](https://md.research.elamur.ai/free/en/companies/microsoft-research-asia/) · **Year:** 2023 · **Venue:** arXiv:2307.05300 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2307.05300) · [PDF](https://arxiv.org/pdf/2307.05300) · [DOI 10.48550/arxiv.2307.05300](https://doi.org/10.48550/arxiv.2307.05300) · [arXiv 2307.05300](https://arxiv.org/abs/2307.05300) · [OpenAlex](https://openalex.org/W4384112348)

**Citations:** 11 citations (OpenAlex, 2026-10-14) · [128](https://md.research.elamur.ai/free/en/papers/wang-2023-unleashing-emergent-cognitive-synergy-large/cited-by/) in our library, of the 140884 papers whose bibliographies we read

## Summary
Large language models still make factual mistakes on knowledge-heavy tasks and stumble on problems that need slow, multi-step reasoning. Existing methods such as Chain-of-Thought mostly help with reasoning and do little against hallucination. The authors introduce Solo Performance Prompting (SPP), a zero-shot technique in which a single model plays several roles. For a given task, the model first decides which personas are needed, such as domain experts or target audiences. These personas then brainstorm relevant knowledge. Finally, a lead "AI Assistant" persona writes a draft, collects critiques from the others and revises it repeatedly. The prompt combines a general instruction with two hand-written demonstrations. To test SPP, the authors use a trio of benchmarks (Logic Grid Puzzle, Codenames Collaborative and Trivia Creative Writing) that span both knowledge-intensive and reasoning-intensive settings. With GPT-4, SPP outperforms Standard prompting, CoT and Self-Refine, reducing factual errors while keeping reasoning strong. The gains do not appear in weaker models. The approach offers a cheap single-model alternative to multi-agent LLM systems.

## Key points
- **Three-stage procedure:** SPP runs persona identification, then brainstorming, then iterative multi-persona collaboration, all inside one model with no fine-tuning or retrieval system.
- **Evaluation tasks:** Trivia Creative Writing and Codenames Collaborative are newly proposed. Together with Logic Grid Puzzle, they test knowledge integration and pure reasoning.
- **Emergent ability:** The "cognitive synergy" effect appears only in GPT-4 and is absent in GPT-3.5-turbo and Llama2-13b-chat. The authors compare this to children beginning role-play only at around age 2–3.
- **Persona ablation:** Dynamically chosen, fine-grained personas (e.g., "Film Expert") clearly beat SPP-Fixed-Persona, which uses two generic roles. Adding persona descriptions (SPP-Profile) brings no further gain, suggesting persona names alone are enough.
- **Prompt ablation and resources:** Removing the second demonstration (multi-persona) lowers performance, though SPP remains fairly robust. Code, data and prompts are released at github.com/MikeWangWZHL/Solo-Performance-Prompting.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Other](https://md.research.elamur.ai/free/en/shelves/other/)

Topics: [Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{wang2023unleashing,
  title = {{Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration}},
  author = {Wang, Zhenhailong and Mao, Shaoguang and Wu, Wenshan and Ge, Tao and Wei, Furu and Ji, Heng},
  year = {2023},
  doi = {10.48550/arxiv.2307.05300},
  eprint = {2307.05300},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2307.05300}
}
```

- [← previous The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning](https://md.research.elamur.ai/free/en/papers/lin-2023-unlocking-spell-base-llms-rethinking/)
- [next → DeepInception: Hypnotize Large Language Model to Be Jailbreaker](https://md.research.elamur.ai/free/en/papers/li-2023-deepinception-hypnotize-large-language-model/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/5/)

## References

43 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. 2023. Auto-gpt. [https://github.com/Significant-](https://github.com/Significant-) [↗](https://github.com/Significant-)
2. Gravitas/Auto-GPT. Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, et al. 2023. A multi-task, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. arXiv preprint arXiv:2302.0402… [find](https://scholar.google.com/scholar?q=Gravitas%2FAuto%2DGPT%2E+Yejin+Bang%2C+Samuel+Cahyawijaya%2C+Nayeon+Lee%2C+Wenliang+Dai%2C+Dan+Su%2C+Bryan+Wilie%2C+Holy+Lovenia%2C+Ziwei+Ji%2C+Tiezheng+Yu%2C+Willy+Chung%2C+et+al%2E+2023%2E+A+multi%2Dtask%2C+multilingual%2C+multimodal+) [our card](https://md.research.elamur.ai/free/en/papers/bang-2023-multitask-multilingual-multimodal-evaluation-chatgpt/)
3. Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George Bm Van Den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, et al. 2022. Improving language models by retrieving from trillions of tokens. In International conference on machine learn… [find](https://scholar.google.com/scholar?q=Sebastian+Borgeaud%2C+Arthur+Mensch%2C+Jordan+Hoffmann%2C+Trevor+Cai%2C+Eliza+Rutherford%2C+Katie+Millican%2C+George+Bm+Van+Den+Driessche%2C+Jean%2DBaptiste+Lespiau%2C+Bogdan+Damoc%2C+Aidan+Clark%2C+et+al%2E+2022%2E+Improving+)
4. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901. [find](https://scholar.google.com/scholar?q=Tom+Brown%2C+Benjamin+Mann%2C+Nick+Ryder%2C+Melanie+Subbiah%2C+Jared+D+Kaplan%2C+Prafulla+Dhariwal%2C+Arvind+Neelakantan%2C+Pranav+Shyam%2C+Girish+Sastry%2C+Amanda+Askell%2C+et+al%2E+2020%2E+Language+models+are+few%2Dshot+lear)
5. Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, et al. 2023. Sparks of artificial general intelligence: Early experiments with gpt-4. arXiv preprint arXiv:2303.12712. [find](https://scholar.google.com/scholar?q=S%C3%A9bastien+Bubeck%2C+Varun+Chandrasekaran%2C+Ronen+Eldan%2C+Johannes+Gehrke%2C+Eric+Horvitz%2C+Ece+Kamar%2C+Peter+Lee%2C+Yin+Tat+Lee%2C+Yuanzhi+Li%2C+Scott+Lundberg%2C+et+al%2E+2023%2E+Sparks+of+artificial+general+intelligenc) [our card](https://md.research.elamur.ai/free/en/papers/bubeck-2023-sparks-artificial-general-intelligence-early/)
6. Tianle Cai, Xuezhi Wang, Tengyu Ma, Xinyun Chen, and Denny Zhou. 2023. Large language models as tool makers. arXiv preprint arXiv:2305.17126. [find](https://scholar.google.com/scholar?q=Tianle+Cai%2C+Xuezhi+Wang%2C+Tengyu+Ma%2C+Xinyun+Chen%2C+and+Denny+Zhou%2E+2023%2E+Large+language+models+as+tool+makers%2E+arXiv+preprint+arXiv%3A2305%2E17126%2E)
7. Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. 2023. Teaching large language models to self-debug. arXiv preprint arXiv:2304.05128. [find](https://scholar.google.com/scholar?q=Xinyun+Chen%2C+Maxwell+Lin%2C+Nathanael+Sch%C3%A4rli%2C+and+Denny+Zhou%2E+2023%2E+Teaching+large+language+models+to+self%2Ddebug%2E+arXiv+preprint+arXiv%3A2304%2E05128%2E)
8. Petru L Curşeu, Nicoleta Meslec, Helen Pluut, and Gerardus JM Lucas. 2015. Cognitive synergy in groups and group-to-individual transfer of decision-making competencies. Frontiers in psychology, 6:1375. [find](https://scholar.google.com/scholar?q=Petru+L+Curs%CC%A7eu%2C+Nicoleta+Meslec%2C+Helen+Pluut%2C+and+Gerardus+JM+Lucas%2E+2015%2E+Cognitive+synergy+in+groups+and+group%2Dto%2Dindividual+transfer+of+decision%2Dmaking+competencies%2E+Frontiers+in+psychology%2C+6%3A137)
9. Ameet Deshpande, Vishvak Murahari, Tanmay Rajpurohit, Ashwin Kalyan, and Karthik Narasimhan. 2023. Toxicity in chatgpt: Analyzing persona-assigned language models. arXiv preprint arXiv:2304.05335. [find](https://scholar.google.com/scholar?q=Ameet+Deshpande%2C+Vishvak+Murahari%2C+Tanmay+Rajpurohit%2C+Ashwin+Kalyan%2C+and+Karthik+Narasimhan%2E+2023%2E+Toxicity+in+chatgpt%3A+Analyzing+persona%2Dassigned+language+models%2E+arXiv+preprint+arXiv%3A2304%2E05335%2E) [our card](https://md.research.elamur.ai/free/en/papers/deshpande-2023-toxicity-chatgpt-analyzing-persona-assigned/)
10. Yao Fu, Hao Peng, Tushar Khot, and Mirella Lapata. 2023. Improving language model negotiation with self-play and in-context learning from ai feedback. arXiv preprint arXiv:2305.10142. [find](https://scholar.google.com/scholar?q=Yao+Fu%2C+Hao+Peng%2C+Tushar+Khot%2C+and+Mirella+Lapata%2E+2023%2E+Improving+language+model+negotiation+with+self%2Dplay+and+in%2Dcontext+learning+from+ai+feedback%2E+arXiv+preprint+arXiv%3A2305%2E10142%2E) [our card](https://md.research.elamur.ai/free/en/papers/fu-2023-improving-language-model-negotiation-self/)
11. Yao Fu, Hao Peng, Ashish Sabharwal, Peter Clark, and Tushar Khot. 2022. Complexity-based prompting for multi-step reasoning. arXiv preprint arXiv:2210.00720. [find](https://scholar.google.com/scholar?q=Yao+Fu%2C+Hao+Peng%2C+Ashish+Sabharwal%2C+Peter+Clark%2C+and+Tushar+Khot%2E+2022%2E+Complexity%2Dbased+prompting+for+multi%2Dstep+reasoning%2E+arXiv+preprint+arXiv%3A2210%2E00720%2E) [our card](https://md.research.elamur.ai/free/en/papers/fu-2022-complexity-based-prompting-multi-step/)
12. Ben Goertzel. 2009. Cognitive synergy: A universal principle for feasible general intelligence. In 2009 8th IEEE International Conference on Cognitive Informatics, pages 464–468. IEEE. [find](https://scholar.google.com/scholar?q=Ben+Goertzel%2E+2009%2E+Cognitive+synergy%3A+A+universal+principle+for+feasible+general+intelligence%2E+In+2009+8th+IEEE+International+Conference+on+Cognitive+Informatics%2C+pages+464%E2%80%93468%2E+IEEE%2E)
13. Ben Goertzel. 2017. A formal model of cognitive synergy. In Artificial General Intelligence: 10th International Conference, AGI 2017, Melbourne, VIC, Australia, August 15-18, 2017, Proceedings 10, pages 13–22. Springer. [find](https://scholar.google.com/scholar?q=Ben+Goertzel%2E+2017%2E+A+formal+model+of+cognitive+synergy%2E+In+Artificial+General+Intelligence%3A+10th+International+Conference%2C+AGI+2017%2C+Melbourne%2C+VIC%2C+Australia%2C+August+15%2D18%2C+2017%2C+Proceedings+10%2C+pag)
14. Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, and Weizhu Chen. 2023. Critic: Large language models can self-correct with tool-interactive critiquing. [find](https://scholar.google.com/scholar?q=Zhibin+Gou%2C+Zhihong+Shao%2C+Yeyun+Gong%2C+Yelong+Shen%2C+Yujiu+Yang%2C+Nan+Duan%2C+and+Weizhu+Chen%2E+2023%2E+Critic%3A+Large+language+models+can+self%2Dcorrect+with+tool%2Dinteractive+critiquing%2E) [our card](https://md.research.elamur.ai/free/en/papers/gou-2023-critic-large-language-models-self/)
15. Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan Tompson, Igor Mordatch, Yevgen Chebotar, et al. 2022. Inner monologue: Embodied reasoning through planning with language models. arXiv preprint arXiv:2207.05608. [find](https://scholar.google.com/scholar?q=Wenlong+Huang%2C+Fei+Xia%2C+Ted+Xiao%2C+Harris+Chan%2C+Jacky+Liang%2C+Pete+Florence%2C+Andy+Zeng%2C+Jonathan+Tompson%2C+Igor+Mordatch%2C+Yevgen+Chebotar%2C+et+al%2E+2022%2E+Inner+monologue%3A+Embodied+reasoning+through+plannin)
16. Gautier Izacard, Patrick Lewis, Maria Lomeli, Lucas Hosseini, Fabio Petroni, Timo Schick, Jane Dwivedi-Yu, Armand Joulin, Sebastian Riedel, and Edouard Grave. 2022. Few-shot learning with retrieval augmented language models. arXiv preprint arXiv:2208.03299. [find](https://scholar.google.com/scholar?q=Gautier+Izacard%2C+Patrick+Lewis%2C+Maria+Lomeli%2C+Lucas+Hosseini%2C+Fabio+Petroni%2C+Timo+Schick%2C+Jane+Dwivedi%2DYu%2C+Armand+Joulin%2C+Sebastian+Riedel%2C+and+Edouard+Grave%2E+2022%2E+Few%2Dshot+learning+with+retrieval+au)
17. Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1601–1611, Vancouver… [find](https://scholar.google.com/scholar?q=Mandar+Joshi%2C+Eunsol+Choi%2C+Daniel+Weld%2C+and+Luke+Zettlemoyer%2E+2017%2E+TriviaQA%3A+A+large+scale+distantly+supervised+challenge+dataset+for+reading+comprehension%2E+In+Proceedings+of+the+55th+Annual+Meeting+) [our card](https://md.research.elamur.ai/free/en/papers/joshi-2017-triviaqa-large-scale-distantly-supervised/)
18. Daniel Kahneman. 2011. macmillan. [find](https://scholar.google.com/scholar?q=Daniel+Kahneman%2E+2011%2E+macmillan%2E)
19. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. arXiv preprint arXiv:2205.11916. [find](https://scholar.google.com/scholar?q=Takeshi+Kojima%2C+Shixiang+Shane+Gu%2C+Machel+Reid%2C+Yutaka+Matsuo%2C+and+Yusuke+Iwasawa%2E+2022%2E+Large+language+models+are+zero%2Dshot+reasoners%2E+arXiv+preprint+arXiv%3A2205%2E11916%2E) [our card](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/)
20. Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2023. Camel: Communicative agents for" mind" exploration of large scale language model society. arXiv preprint arXiv:2303.17760. [find](https://scholar.google.com/scholar?q=Guohao+Li%2C+Hasan+Abed+Al+Kader+Hammoud%2C+Hani+Itani%2C+Dmitrii+Khizbullin%2C+and+Bernard+Ghanem%2E+2023%2E+Camel%3A+Communicative+agents+for%22+mind%22+exploration+of+large+scale+language+model+society%2E+arXiv+prepri) [our card](https://md.research.elamur.ai/free/en/papers/li-2023-camel-communicative-agents-mind/)
21. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. 2023. Self-refine: Iterative refinement with self-feedback. arXiv preprint arXiv:2303.17651. [find](https://scholar.google.com/scholar?q=Aman+Madaan%2C+Niket+Tandon%2C+Prakhar+Gupta%2C+Skyler+Hallinan%2C+Luyu+Gao%2C+Sarah+Wiegreffe%2C+Uri+Alon%2C+Nouha+Dziri%2C+Shrimai+Prabhumoye%2C+Yiming+Yang%2C+et+al%2E+2023%2E+Self%2Drefine%3A+Iterative+refinement+with+self%2Df)
22. Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. On faithfulness and factuality in abstractive summarization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 1906–1919, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Joshua+Maynez%2C+Shashi+Narayan%2C+Bernd+Bohnet%2C+and+Ryan+McDonald%2E+2020%2E+On+faithfulness+and+factuality+in+abstractive+summarization%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Association+for+Comp)
23. Theo X Olausson, Jeevana Priya Inala, Chenglong Wang, Jianfeng Gao, and Armando Solar-Lezama. 2023. Demystifying gpt self-repair for code generation. arXiv preprint arXiv:2306.09896. [find](https://scholar.google.com/scholar?q=Theo+X+Olausson%2C+Jeevana+Priya+Inala%2C+Chenglong+Wang%2C+Jianfeng+Gao%2C+and+Armando+Solar%2DLezama%2E+2023%2E+Demystifying+gpt+self%2Drepair+for+code+generation%2E+arXiv+preprint+arXiv%3A2306%2E09896%2E)
24. OpenAI. 2023a. Gpt-35. [https://platform.openai.com/docs/models/gpt-3-5](https://platform.openai.com/docs/models/gpt-3-5). [↗](https://platform.openai.com/docs/models/gpt-3-5)
25. OpenAI. 2023b. Gpt-4 technical report. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2023b%2E+Gpt%2D4+technical+report%2E) [our card](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/)
26. Joon Sung Park, Joseph C O’Brien, Carrie J Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. arXiv preprint arXiv:2304.03442. [find](https://scholar.google.com/scholar?q=Joon+Sung+Park%2C+Joseph+C+O%E2%80%99Brien%2C+Carrie+J+Cai%2C+Meredith+Ringel+Morris%2C+Percy+Liang%2C+and+Michael+S+Bernstein%2E+2023%2E+Generative+agents%3A+Interactive+simulacra+of+human+behavior%2E+arXiv+preprint+arXiv%3A230)
27. Anthony D Pellegrini. 2009. The role of play in human development. Oxford University Press, USA. [find](https://scholar.google.com/scholar?q=Anthony+D+Pellegrini%2E+2009%2E+The+role+of+play+in+human+development%2E+Oxford+University+Press%2C+USA%2E)
28. Jean Piaget. 1954. The construction of reality in the child. [find](https://scholar.google.com/scholar?q=Jean+Piaget%2E+1954%2E+The+construction+of+reality+in+the+child%2E)
29. Chengwei Qin, Aston Zhang, Zhuosheng Zhang, Jiaao Chen, Michihiro Yasunaga, and Diyi Yang. 2023. Is chatgpt a general-purpose natural language processing task solver? arXiv preprint arXiv:2302.06476. [find](https://scholar.google.com/scholar?q=Chengwei+Qin%2C+Aston+Zhang%2C+Zhuosheng+Zhang%2C+Jiaao+Chen%2C+Michihiro+Yasunaga%2C+and+Diyi+Yang%2E+2023%2E+Is+chatgpt+a+general%2Dpurpose+natural+language+processing+task+solver%3F+arXiv+preprint+arXiv%3A2302%2E06476%2E) [our card](https://md.research.elamur.ai/free/en/papers/qin-2023-chatgpt-general-purpose-natural-language/)
30. Timo Schick, Jane Dwivedi-Yu, Zhengbao Jiang, Fabio Petroni, Patrick Lewis, Gautier Izacard, Qingfei You, Christoforos Nalmpantis, Edouard Grave, and Sebastian Riedel. 2022. Peer: A collaborative language model. arXiv preprint arXiv:2208.11663. [find](https://scholar.google.com/scholar?q=Timo+Schick%2C+Jane+Dwivedi%2DYu%2C+Zhengbao+Jiang%2C+Fabio+Petroni%2C+Patrick+Lewis%2C+Gautier+Izacard%2C+Qingfei+You%2C+Christoforos+Nalmpantis%2C+Edouard+Grave%2C+and+Sebastian+Riedel%2E+2022%2E+Peer%3A+A+collaborative+lang)
31. Noah Shinn, Beck Labash, and Ashwin Gopinath. 2023. Reflexion: an autonomous agent with dynamic memory and self-reflection. arXiv preprint arXiv:2303.11366. [find](https://scholar.google.com/scholar?q=Noah+Shinn%2C+Beck+Labash%2C+and+Ashwin+Gopinath%2E+2023%2E+Reflexion%3A+an+autonomous+agent+with+dynamic+memory+and+self%2Dreflection%2E+arXiv+preprint+arXiv%3A2303%2E11366%2E) [our card](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/)
32. Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. 2021. Retrieval augmentation reduces hallucination in conversation. arXiv preprint arXiv:2104.07567. [find](https://scholar.google.com/scholar?q=Kurt+Shuster%2C+Spencer+Poff%2C+Moya+Chen%2C+Douwe+Kiela%2C+and+Jason+Weston%2E+2021%2E+Retrieval+augmentation+reduces+hallucination+in+conversation%2E+arXiv+preprint+arXiv%3A2104%2E07567%2E) [our card](https://md.research.elamur.ai/free/en/papers/shuster-2021-retrieval-augmentation-reduces-hallucination-conversation/)
33. Steven A Sloman. 1996. The empirical case for two systems of reasoning. Psychological bulletin, 119(1):3. [find](https://scholar.google.com/scholar?q=Steven+A+Sloman%2E+1996%2E+The+empirical+case+for+two+systems+of+reasoning%2E+Psychological+bulletin%2C+119%281%29%3A3%2E)
34. Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al. 2022. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv preprint arXiv:2206.04615. [find](https://scholar.google.com/scholar?q=Aarohi+Srivastava%2C+Abhinav+Rastogi%2C+Abhishek+Rao%2C+Abu+Awal+Md+Shoeb%2C+Abubakar+Abid%2C+Adam+Fisch%2C+Adam+R+Brown%2C+Adam+Santoro%2C+Aditya+Gupta%2C+Adri%C3%A0+Garriga%2DAlonso%2C+et+al%2E+2022%2E+Beyond+the+imitation+game%3A+) [our card](https://md.research.elamur.ai/free/en/papers/srivastava-2022-imitation-game-quantifying-extrapolating-capabilities/)
35. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288. [find](https://scholar.google.com/scholar?q=Hugo+Touvron%2C+Louis+Martin%2C+Kevin+Stone%2C+Peter+Albert%2C+Amjad+Almahairi%2C+Yasmine+Babaei%2C+Nikolay+Bashlykov%2C+Soumya+Batra%2C+Prajjwal+Bhargava%2C+Shruti+Bhosale%2C+et+al%2E+2023%2E+Llama+2%3A+Open+foundation+and+fi)
36. Zhenhailong Wang, Xiaoman Pan, Dian Yu, Dong Yu, Jianshu Chen, and Heng Ji. 2022. Zemi: Learning zero-shot semi-parametric language models from multiple tasks. arXiv preprint arXiv:2210.00185. [find](https://scholar.google.com/scholar?q=Zhenhailong+Wang%2C+Xiaoman+Pan%2C+Dian+Yu%2C+Dong+Yu%2C+Jianshu+Chen%2C+and+Heng+Ji%2E+2022%2E+Zemi%3A+Learning+zero%2Dshot+semi%2Dparametric+language+models+from+multiple+tasks%2E+arXiv+preprint+arXiv%3A2210%2E00185%2E)
37. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023. Chain-of-thought prompting elicits reasoning in large language models. [find](https://scholar.google.com/scholar?q=Jason+Wei%2C+Xuezhi+Wang%2C+Dale+Schuurmans%2C+Maarten+Bosma%2C+Brian+Ichter%2C+Fei+Xia%2C+Ed+Chi%2C+Quoc+Le%2C+and+Denny+Zhou%2E+2023%2E+Chain%2Dof%2Dthought+prompting+elicits+reasoning+in+large+language+models%2E) [our card](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/)
38. Benfeng Xu, An Yang, Junyang Lin, Quan Wang, Chang Zhou, Yongdong Zhang, and Zhendong Mao. 2023. Expertprompting: Instructing large language models to be distinguished experts. arXiv preprint arXiv:2305.14688. [find](https://scholar.google.com/scholar?q=Benfeng+Xu%2C+An+Yang%2C+Junyang+Lin%2C+Quan+Wang%2C+Chang+Zhou%2C+Yongdong+Zhang%2C+and+Zhendong+Mao%2E+2023%2E+Expertprompting%3A+Instructing+large+language+models+to+be+distinguished+experts%2E+arXiv+preprint+arXiv%3A23)
39. Tianci Xue, Ziqi Wang, Zhenhailong Wang, Chi Han, Pengfei Yu, and Heng Ji. 2023. Rcot: Detecting and rectifying factual inconsistency in reasoning by reversing chain-of-thought. arXiv preprint arXiv:2305.11499. — [arXiv:2305.11499](https://arxiv.org/abs/2305.11499) [find](https://scholar.google.com/scholar?q=Tianci+Xue%2C+Ziqi+Wang%2C+Zhenhailong+Wang%2C+Chi+Han%2C+Pengfei+Yu%2C+and+Heng+Ji%2E+2023%2E+Rcot%3A+Detecting+and+rectifying+factual+inconsistency+in+reasoning+by+reversing+chain%2Dof%2Dthought%2E+arXiv+preprint+arXiv%3A2)
40. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. arXiv preprint arXiv:2305.10601. [find](https://scholar.google.com/scholar?q=Shunyu+Yao%2C+Dian+Yu%2C+Jeffrey+Zhao%2C+Izhak+Shafran%2C+Thomas+L+Griffiths%2C+Yuan+Cao%2C+and+Karthik+Narasimhan%2E+2023%2E+Tree+of+thoughts%3A+Deliberate+problem+solving+with+large+language+models%2E+arXiv+preprint+ar)
41. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. ArXiv, abs/2210.03629. [find](https://scholar.google.com/scholar?q=Shunyu+Yao%2C+Jeffrey+Zhao%2C+Dian+Yu%2C+Nan+Du%2C+Izhak+Shafran%2C+Karthik+Narasimhan%2C+and+Yuan+Cao%2E+2022%2E+React%3A+Synergizing+reasoning+and+acting+in+language+models%2E+ArXiv%2C+abs%2F2210%2E03629%2E) [our card](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/)
42. Zhuosheng Zhang, Aston Zhang, Mu Li, and Alex Smola. 2022. Automatic chain of thought prompting in large language models. [find](https://scholar.google.com/scholar?q=Zhuosheng+Zhang%2C+Aston+Zhang%2C+Mu+Li%2C+and+Alex+Smola%2E+2022%2E+Automatic+chain+of+thought+prompting+in+large+language+models%2E) [our card](https://md.research.elamur.ai/free/en/papers/zhang-2022-automatic-chain-thought-prompting-large/)
43. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric. P Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. [find](https://scholar.google.com/scholar?q=Lianmin+Zheng%2C+Wei%2DLin+Chiang%2C+Ying+Sheng%2C+Siyuan+Zhuang%2C+Zhanghao+Wu%2C+Yonghao+Zhuang%2C+Zi+Lin%2C+Zhuohan+Li%2C+Dacheng+Li%2C+Eric%2E+P+Xing%2C+Hao+Zhang%2C+Joseph+E%2E+Gonzalez%2C+and+Ion+Stoica%2E+2023%2E+Judging+llm%2Das)