---
title: "Evaluating and Inducing Personality in Pre-trained Language"
description: "The authors build the Machine Personality Inventory (MPI), a zero-shot multiple-choice benchmark… 61 references, cited by 103 papers here."
authors: ["Guangyuan Jiang", "Manjie Xu", "Song-Chun Zhu", "Wenjuan Han", "Chi Zhang", "Yixin Zhu"]
year: 2022
venue: "arXiv:2206.07550"
source: "https://arxiv.org/abs/2206.07550"
doi: "10.48550/arxiv.2206.07550"
arxiv: "2206.07550"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/jiang-2022-evaluating-inducing-personality-pre-trained/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/jiang-2022-evaluating-inducing-personality-pre-trained/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/3/) / Evaluating and Inducing Personality in Pre-trained Language Models

# Evaluating and Inducing Personality in Pre-trained Language Models

**Authors:** Guangyuan Jiang, [Manjie Xu](https://md.research.elamur.ai/free/en/authors/manjie-xu/), [Song-Chun Zhu](https://md.research.elamur.ai/free/en/authors/song-chun-zhu/), [Wenjuan Han](https://md.research.elamur.ai/free/en/authors/wenjuan-han/), [Chi Zhang](https://md.research.elamur.ai/free/en/authors/chi-zhang/) and [Yixin Zhu](https://md.research.elamur.ai/free/en/authors/yixin-zhu/) · **Year:** 2022 · **Venue:** [arXiv:2206.07550](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/) ([NeurIPS 2022](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/2022/)) · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2206.07550) · [PDF](https://arxiv.org/pdf/2206.07550) · [DOI 10.48550/arxiv.2206.07550](https://doi.org/10.48550/arxiv.2206.07550) · [arXiv 2206.07550](https://arxiv.org/abs/2206.07550) · [OpenAlex](https://openalex.org/W7133232449)

**Citations:** 65 citations (OpenAlex, 2026-10-14) · [103](https://md.research.elamur.ai/free/en/papers/jiang-2022-evaluating-inducing-personality-pre-trained/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
This NeurIPS 2023 paper asks whether personality psychology can provide a principled, quantitative way to describe how large language models behave, and whether a chosen personality can be deliberately instilled in them. The authors build the Machine Personality Inventory (MPI), a zero-shot multiple-choice benchmark based on the Big Five (OCEAN) trait theory. Its items come from public-domain IPIP and IPIP-NEO material and the BFI-S. In each item, a model rates how accurately a self-descriptive statement fits it.

Testing several LLMs on MPI, the authors find that consistent, human-like trait profiles appear only in more capable models, which they frame as an emergent ability. Alpaca and GPT-3.5 produce scores comparable to human population statistics.

They also propose PERSONALITY PROMPTING (P²), a prompt chain that combines psychological findings with descriptions generated by the model itself, to push each trait up or down. P² requires no fine-tuning. MPI scores and human-rated vignette tests both confirm its effect. The work offers a reproducible tool for analysing and controlling LLM behaviour.

## Key points
- MPI is released at two sizes, 120 and 1k items, and covers Openness, Conscientiousness, Extraversion, Agreeableness and Neuroticism.
- Alpaca and GPT-3.5 show stable trait tendencies that align with human norms, while weaker models lack this stability.
- P² was compared against a WORDS AUTO PROMPTING baseline in vignette tests with 100 Prolific participants, split 50 per method.
- In those tests, P² success rates ranged from about 0.45 to 0.92 across the ten trait directions (each factor induced positively and negatively). It beat the baseline in nine of ten, with the exception of positive Agreeableness (0.88 vs 0.92).
- Stated limitations include possible WEIRD bias from English-dominated training data and no treatment of personality disorders such as those measured by MMPI.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,341 works

## Cite

Cite as: Guangyuan Jiang, Manjie Xu, Song-Chun Zhu, Wenjuan Han, Chi Zhang and Yixin Zhu. 2022. Evaluating and Inducing Personality in Pre-trained Language Models. arXiv:2206.07550.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{jiang2022evaluating,
  title = {{Evaluating and Inducing Personality in Pre-trained Language Models}},
  author = {Guangyuan Jiang, Manjie Xu, Song-Chun Zhu, Wenjuan Han, Chi Zhang and Yixin Zhu},
  year = {2022},
  doi = {10.48550/arxiv.2206.07550},
  eprint = {2206.07550},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2206.07550}
}
```

- [← previous Document-Level Relation Extraction with Adaptive Focal Loss and Knowledge Distillation](https://md.research.elamur.ai/free/en/papers/tan-2022-document-level-relation-extraction-adaptive/)
- [next → Factuality Enhanced Language Models for Open-Ended Text Generation](https://md.research.elamur.ai/free/en/papers/lee-2022-factuality-enhanced-language-models-open/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/3/)

## References

61 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Aher, G. V., Arriaga, R. I., and Kalai, A. T. (2023). Using large language models to simulate multiple humans and replicate human subject studies. In International Conference on Machine Learning (ICML), pages 337–371. PMLR. 2, 3, 10 [find](https://scholar.google.com/scholar?q=Aher%2C+G%2E+V%2E%2C+Arriaga%2C+R%2E+I%2E%2C+and+Kalai%2C+A%2E+T%2E+%282023%29%2E+Using+large+language+models+to+simulate+multiple+humans+and+replicate+human+subject+studies%2E+In+International+Conference+on+Machine+Learning+%28ICML)
2. Atari, M., Xue, M. J., Park, P. S., Blasi, D., and Henrich, J. (2023). Which humans? PsyArXiv preprint. 10 [find](https://scholar.google.com/scholar?q=Atari%2C+M%2E%2C+Xue%2C+M%2E+J%2E%2C+Park%2C+P%2E+S%2E%2C+Blasi%2C+D%2E%2C+and+Henrich%2C+J%2E+%282023%29%2E+Which+humans%3F+PsyArXiv+preprint%2E+10)
3. Barrett, D., Hill, F., Santoro, A., Morcos, A., and Lillicrap, T. (2018). [Measuring abstract reasoning in neural networks](https://md.research.elamur.ai/free/en/papers/barrett-2018-measuring-abstract-reasoning-neural-networks/). In International Conference on Machine Learning (ICML). 2
4. Binz, M. and Schulz, E. (2023). Using cognitive psychology to understand GPT-3. Proceedings of the National Academy of Sciences (PNAS), 120(6):e2218523120. 2, 3 [find](https://scholar.google.com/scholar?q=Binz%2C+M%2E+and+Schulz%2C+E%2E+%282023%29%2E+Using+cognitive+psychology+to+understand+GPT%2D3%2E+Proceedings+of+the+National+Academy+of+Sciences+%28PNAS%29%2C+120%286%29%3Ae2218523120%2E+2%2C+3)
5. Black, S., Biderman, S., Hallahan, E., Anthony, Q., Gao, L., Golding, L., He, H., Leahy, C., McDonell, K., Phang, J., et al. (2022). [Gpt-neox-20b: An open-source autoregressive language model](https://md.research.elamur.ai/free/en/papers/black-2022-gpt-neox-20b-open-source/). Challenges & Perspectives in Creating Large Language Models, page 95. 6, 3
6. Black, S., Leo, G., Wang, P., Leahy, C., and Biderman, S. (2021). GPT-Neo: Large scale autoregressive language modeling with mesh-tensorflow, march 2021. URL https://doi.org/10.5281/zenodo, 5297715. 5, 3 — [doi:10.5281/zenodo](https://doi.org/10.5281/zenodo) [↗](https://doi.org/10.5281/zenodo)
7. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020). [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in Neural Information Processing Systems (NeurIPS). 6, 8, 3
8. Cattell, H. E. and Mead, A. D. (2008). The sixteen personality factor questionnaire (16PF). The SAGE Handbook of Personality Theory and Assessment: Personality Measurement and Testing (Volume 2), 2:135. 3 [find](https://scholar.google.com/scholar?q=Cattell%2C+H%2E+E%2E+and+Mead%2C+A%2E+D%2E+%282008%29%2E+The+sixteen+personality+factor+questionnaire+%2816PF%29%2E+The+SAGE+Handbook+of+Personality+Theory+and+Assessment%3A+Personality+Measurement+and+Testing+%28Volume+2%29%2C+2%3A13)
9. Chollet, F. (2019). On the measure of intelligence. arXiv preprint arXiv:1911.01547. 2 [find](https://scholar.google.com/scholar?q=Chollet%2C+F%2E+%282019%29%2E+On+the+measure+of+intelligence%2E+arXiv+preprint+arXiv%3A1911%2E01547%2E+2)
10. Costa, P. T. and McCrae, R. R. (1999). A five-factor theory of personality. In The Five-Factor Model of Personality: Theoretical Perspectives, pages 51–87. The Guilford Press New York, NY, USA. 4 [find](https://scholar.google.com/scholar?q=Costa%2C+P%2E+T%2E+and+McCrae%2C+R%2E+R%2E+%281999%29%2E+A+five%2Dfactor+theory+of+personality%2E+In+The+Five%2DFactor+Model+of+Personality%3A+Theoretical+Perspectives%2C+pages+51%E2%80%9387%2E+The+Guilford+Press+New+York%2C+NY%2C+USA%2E+4)
11. Costa Jr, P. T. and McCrae, R. R. (2008). The Revised Neo Personality Inventory (neo-pi-r). Sage Publications, Inc. 3 [find](https://scholar.google.com/scholar?q=Costa+Jr%2C+P%2E+T%2E+and+McCrae%2C+R%2E+R%2E+%282008%29%2E+The+Revised+Neo+Personality+Inventory+%28neo%2Dpi%2Dr%29%2E+Sage+Publications%2C+Inc%2E+3)
12. Dasgupta, I., Lampinen, A. K., Chan, S. C., Creswell, A., Kumaran, D., McClelland, J. L., and Hill, F. (2022). [Language models show human-like content effects on reasoning](https://md.research.elamur.ai/free/en/papers/dasgupta-2022-language-models-show-human-like/). arXiv preprint arXiv:2207.07051. 2, 3
13. De Raad, B. (2000). The big five personality factors: the psycholexical approach to personality. Hogrefe & Huber Publishers. 3 [find](https://scholar.google.com/scholar?q=De+Raad%2C+B%2E+%282000%29%2E+The+big+five+personality+factors%3A+the+psycholexical+approach+to+personality%2E+Hogrefe+%26+Huber+Publishers%2E+3)
14. Fan, L., Xu, M., Cao, Z., Zhu, Y., and Zhu, S.-C. (2022). Artificial social intelligence: A comparative and holistic view. CAAI Artificial Intelligence Research, 1(2):144–160. 2 [find](https://scholar.google.com/scholar?q=Fan%2C+L%2E%2C+Xu%2C+M%2E%2C+Cao%2C+Z%2E%2C+Zhu%2C+Y%2E%2C+and+Zhu%2C+S%2E%2DC%2E+%282022%29%2E+Artificial+social+intelligence%3A+A+comparative+and+holistic+view%2E+CAAI+Artificial+Intelligence+Research%2C+1%282%29%3A144%E2%80%93160%2E+2)
15. Farnadi, G., Zoghbi, S., Moens, M.-F., and De Cock, M. (2013). Recognising personality traits using facebook status updates. In Proceedings of the workshop on computational personality recognition at the AAAI conference on weblogs and social media. 3 [find](https://scholar.google.com/scholar?q=Farnadi%2C+G%2E%2C+Zoghbi%2C+S%2E%2C+Moens%2C+M%2E%2DF%2E%2C+and+De+Cock%2C+M%2E+%282013%29%2E+Recognising+personality+traits+using+facebook+status+updates%2E+In+Proceedings+of+the+workshop+on+computational+personality+recognition+at+)
16. Frank, M. C. (2023). Large language models as models of human cognition. PsyArXiv. 2 [find](https://scholar.google.com/scholar?q=Frank%2C+M%2E+C%2E+%282023%29%2E+Large+language+models+as+models+of+human+cognition%2E+PsyArXiv%2E+2)
17. Goldberg, L. R. et al. (1999). A broad-bandwidth, public domain, personality inventory measuring the lower-level facets of several five-factor models. Personality Psychology in Europe, 7(1):7–28. 4 [find](https://scholar.google.com/scholar?q=Goldberg%2C+L%2E+R%2E+et+al%2E+%281999%29%2E+A+broad%2Dbandwidth%2C+public+domain%2C+personality+inventory+measuring+the+lower%2Dlevel+facets+of+several+five%2Dfactor+models%2E+Personality+Psychology+in+Europe%2C+7%281%29%3A7%E2%80%9328%2E+4)
18. Goldberg, L. R., Johnson, J. A., Eber, H. W., Hogan, R., Ashton, M. C., Cloninger, C. R., and Gough, H. G. (2006). The international personality item pool and the future of public-domain personality measures. Journal of Research in Personality, 40(1):84–96. 4 [find](https://scholar.google.com/scholar?q=Goldberg%2C+L%2E+R%2E%2C+Johnson%2C+J%2E+A%2E%2C+Eber%2C+H%2E+W%2E%2C+Hogan%2C+R%2E%2C+Ashton%2C+M%2E+C%2E%2C+Cloninger%2C+C%2E+R%2E%2C+and+Gough%2C+H%2E+G%2E+%282006%29%2E+The+international+personality+item+pool+and+the+future+of+public%2Ddomain+personality+m)
19. Hathaway, S. R. and McKinley, J. C. (1951). Minnesota multiphasic personality inventory; manual, revised. Psychological Corporation. 10 [find](https://scholar.google.com/scholar?q=Hathaway%2C+S%2E+R%2E+and+McKinley%2C+J%2E+C%2E+%281951%29%2E+Minnesota+multiphasic+personality+inventory%3B+manual%2C+revised%2E+Psychological+Corporation%2E+10)
20. Jiang, G., Xu, M., Xin, S., Liang, W., Peng, Y., Zhang, C., and Zhu, Y. (2023). MEWL: Few-shot multimodal word learning with referential uncertainty. In International Conference on Machine Learning (ICML), pages 15144–15169. PMLR. 2 [find](https://scholar.google.com/scholar?q=Jiang%2C+G%2E%2C+Xu%2C+M%2E%2C+Xin%2C+S%2E%2C+Liang%2C+W%2E%2C+Peng%2C+Y%2E%2C+Zhang%2C+C%2E%2C+and+Zhu%2C+Y%2E+%282023%29%2E+MEWL%3A+Few%2Dshot+multimodal+word+learning+with+referential+uncertainty%2E+In+International+Conference+on+Machine+Learning+%28I)
21. Johnson, J. A. (2005). Ascertaining the validity of individual protocols from web-based personality inventories. Journal of Research in Personality, 39(1):103–129. 4 [find](https://scholar.google.com/scholar?q=Johnson%2C+J%2E+A%2E+%282005%29%2E+Ascertaining+the+validity+of+individual+protocols+from+web%2Dbased+personality+inventories%2E+Journal+of+Research+in+Personality%2C+39%281%29%3A103%E2%80%93129%2E+4)
22. Johnson, J. A. (2014). Measuring thirty facets of the five factor model with a 120-item public domain inventory: Development of the IPIP-NEO-120. Journal of Research in Personality, 51:78–89. 4, 5 [find](https://scholar.google.com/scholar?q=Johnson%2C+J%2E+A%2E+%282014%29%2E+Measuring+thirty+facets+of+the+five+factor+model+with+a+120%2Ditem+public+domain+inventory%3A+Development+of+the+IPIP%2DNEO%2D120%2E+Journal+of+Research+in+Personality%2C+51%3A78%E2%80%9389%2E+4%2C+5)
23. Kazdin, A. E., Association, A. P., et al. (2000). Encyclopedia of psychology, volume 2. American Psychological Association Washington, DC. 2, 1 [find](https://scholar.google.com/scholar?q=Kazdin%2C+A%2E+E%2E%2C+Association%2C+A%2E+P%2E%2C+et+al%2E+%282000%29%2E+Encyclopedia+of+psychology%2C+volume+2%2E+American+Psychological+Association+Washington%2C+DC%2E+2%2C+1)
24. Kwantes, P. J., Derbentseva, N., Lam, Q., Vartanian, O., and Marmurek, H. H. (2016). Assessing the big five personality traits with latent semantic analysis. Personality and Individual Differences, 102:229–233. 8, 9, 5 [find](https://scholar.google.com/scholar?q=Kwantes%2C+P%2E+J%2E%2C+Derbentseva%2C+N%2E%2C+Lam%2C+Q%2E%2C+Vartanian%2C+O%2E%2C+and+Marmurek%2C+H%2E+H%2E+%282016%29%2E+Assessing+the+big+five+personality+traits+with+latent+semantic+analysis%2E+Personality+and+Individual+Differences%2C+10)
25. Lake, B. M., Ullman, T. D., Tenenbaum, J. B., and Gershman, S. J. (2017). Building machines that learn and think like people. Behavioral and Brain Sciences, 40. 2 [find](https://scholar.google.com/scholar?q=Lake%2C+B%2E+M%2E%2C+Ullman%2C+T%2E+D%2E%2C+Tenenbaum%2C+J%2E+B%2E%2C+and+Gershman%2C+S%2E+J%2E+%282017%29%2E+Building+machines+that+learn+and+think+like+people%2E+Behavioral+and+Brain+Sciences%2C+40%2E+2)
26. Lang, F. R., John, D., Lüdtke, O., Schupp, J., and Wagner, G. G. (2011). Short assessment of the big five: Robust across survey methods except telephone interviewing. Behavior Research Methods, 43(2):548–567. 4, 8 [find](https://scholar.google.com/scholar?q=Lang%2C+F%2E+R%2E%2C+John%2C+D%2E%2C+L%C3%BCdtke%2C+O%2E%2C+Schupp%2C+J%2E%2C+and+Wagner%2C+G%2E+G%2E+%282011%29%2E+Short+assessment+of+the+big+five%3A+Robust+across+survey+methods+except+telephone+interviewing%2E+Behavior+Research+Methods%2C+43%282%29%3A)
27. Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., Stoyanov, V., and Zettlemoyer, L. (2020). [BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension](https://md.research.elamur.ai/free/en/papers/lewis-2020-bart-denoising-sequence-sequence-pre/). In Annual Meeting of the Association for Computational Linguistics (AC…
28. Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., and Neubig, G. (2023). [Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing](https://md.research.elamur.ai/free/en/papers/liu-2021-pre-train-prompt-predict-systematic/). ACM Computing Surveys, 55(9):1– 35. 7
29. Mairesse, F. and Walker, M. (2007). PERSONAGE: Personality generation for dialogue. In Annual Meeting of the Association for Computational Linguistics (ACL). 3 [find](https://scholar.google.com/scholar?q=Mairesse%2C+F%2E+and+Walker%2C+M%2E+%282007%29%2E+PERSONAGE%3A+Personality+generation+for+dialogue%2E+In+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28ACL%29%2E+3)
30. Mairesse, F., Walker, M. A., Mehl, M. R., and Moore, R. K. (2007). Using linguistic cues for the [automatic recognition of personality in conversation](https://md.research.elamur.ai/free/en/papers/mairesse-2006-automatic-recognition-personality-conversation/) and text. Journal of Artificial Intelligence Research, 30:457–500. 3
31. McCrae, R. R. and Costa Jr, P. T. (1997). Personality trait structure as a human universal. American Psychologist, 52(5):509. 4 [find](https://scholar.google.com/scholar?q=McCrae%2C+R%2E+R%2E+and+Costa+Jr%2C+P%2E+T%2E+%281997%29%2E+Personality+trait+structure+as+a+human+universal%2E+American+Psychologist%2C+52%285%29%3A509%2E+4)
32. McCrae, R. R. and John, O. P. (1992). An introduction to the five-factor model and its applications. Journal of Personality, 60(2):175–215. 4 [find](https://scholar.google.com/scholar?q=McCrae%2C+R%2E+R%2E+and+John%2C+O%2E+P%2E+%281992%29%2E+An+introduction+to+the+five%2Dfactor+model+and+its+applications%2E+Journal+of+Personality%2C+60%282%29%3A175%E2%80%93215%2E+4)
33. Mehl, M. R., Gosling, S. D., and Pennebaker, J. W. (2006). Personality in its natural habitat: manifestations and implicit folk theories of personality in daily life. Journal of Personality and Social Psychology, 90(5):862. 3, 7 [find](https://scholar.google.com/scholar?q=Mehl%2C+M%2E+R%2E%2C+Gosling%2C+S%2E+D%2E%2C+and+Pennebaker%2C+J%2E+W%2E+%282006%29%2E+Personality+in+its+natural+habitat%3A+manifestations+and+implicit+folk+theories+of+personality+in+daily+life%2E+Journal+of+Personality+and+Social)
34. Norman, W. T. (1963). Toward an adequate taxonomy of personality attributes: Replicated factor structure in peer nomination personality ratings. Journal of Abnormal and Social Psychology, 66(6):574. 3, 7 [find](https://scholar.google.com/scholar?q=Norman%2C+W%2E+T%2E+%281963%29%2E+Toward+an+adequate+taxonomy+of+personality+attributes%3A+Replicated+factor+structure+in+peer+nomination+personality+ratings%2E+Journal+of+Abnormal+and+Social+Psychology%2C+66%286%29%3A574%2E+3)
35. Oberlander, J. and Nowson, S. (2006). Whose thumb is it anyway? classifying author personality from weblog text. In International Conference on Computational Linguistics (COLING). 3 [find](https://scholar.google.com/scholar?q=Oberlander%2C+J%2E+and+Nowson%2C+S%2E+%282006%29%2E+Whose+thumb+is+it+anyway%3F+classifying+author+personality+from+weblog+text%2E+In+International+Conference+on+Computational+Linguistics+%28COLING%29%2E+3)
36. OpenAI (2023). [GPT-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2304.03442. 5
37. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022). [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). In Advances in Neural Information Processing Systems (NeurIPS). 6, 3
38. Park, J. S., O’Brien, J. C., Cai, C. J., Morris, M. R., Liang, P., and Bernstein, M. S. (2023). Generative agents: Interactive simulacra of human behavior. arXiv preprint arXiv:2304.03442. 3 [find](https://scholar.google.com/scholar?q=Park%2C+J%2E+S%2E%2C+O%E2%80%99Brien%2C+J%2E+C%2E%2C+Cai%2C+C%2E+J%2E%2C+Morris%2C+M%2E+R%2E%2C+Liang%2C+P%2E%2C+and+Bernstein%2C+M%2E+S%2E+%282023%29%2E+Generative+agents%3A+Interactive+simulacra+of+human+behavior%2E+arXiv+preprint+arXiv%3A2304%2E03442%2E+3)
39. Prasad, A., Hase, P., Zhou, X., and Bansal, M. (2023). [Grips: Gradient-free, edit-based instruction search for prompting large language models](https://md.research.elamur.ai/free/en/papers/prasad-2023-grips-gradient-free-edit-based/). In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics. 8
40. Raad, B. d. E. and Perugini, M. E. (2002). Big five factor assessment: Introduction. Hogrefe & Huber Publishers. 3 [find](https://scholar.google.com/scholar?q=Raad%2C+B%2E+d%2E+E%2E+and+Perugini%2C+M%2E+E%2E+%282002%29%2E+Big+five+factor+assessment%3A+Introduction%2E+Hogrefe+%26+Huber+Publishers%2E+3)
41. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J. (2020). [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). Journal of Machine Learning Research (JMLR), 21:1–67. 6, 2
42. Rahwan, I., Cebrian, M., Obradovich, N., Bongard, J., Bonnefon, J.-F., Breazeal, C., Crandall, J. W., Christakis, N. A., Couzin, I. D., Jackson, M. O., Jennings, N. R., Kamar, E., Kloumann, I. M., Larochelle, H., Lazer, D., McElreath, R., Mislove, A., Parkes, D. C., Pentland, A. S., Roberts, M. E.,… [find](https://scholar.google.com/scholar?q=Rahwan%2C+I%2E%2C+Cebrian%2C+M%2E%2C+Obradovich%2C+N%2E%2C+Bongard%2C+J%2E%2C+Bonnefon%2C+J%2E%2DF%2E%2C+Breazeal%2C+C%2E%2C+Crandall%2C+J%2E+W%2E%2C+Christakis%2C+N%2E+A%2E%2C+Couzin%2C+I%2E+D%2E%2C+Jackson%2C+M%2E+O%2E%2C+Jennings%2C+N%2E+R%2E%2C+Kamar%2C+E%2E%2C+Kloumann%2C+I%2E+M%2E%2C+Lar)
43. Sanh, V., Webson, A., Raffel, C., Bach, S., Sutawika, L., Alyafeai, Z., Chaffin, A., Stiegler, A., Le Scao, T., Raja, A., et al. (2022). [Multitask prompted training enables zero-shot task generalization](https://md.research.elamur.ai/free/en/papers/sanh-2021-multitask-prompted-training-enables-zero/). In International Conference on Learning Representations (ICLR). 6, 2
44. Shiffrin, R. and Mitchell, M. (2023). Probing the psychology of AI models. Proceedings of the National Academy of Sciences (PNAS), 120(10):e2300963120. 2 [find](https://scholar.google.com/scholar?q=Shiffrin%2C+R%2E+and+Mitchell%2C+M%2E+%282023%29%2E+Probing+the+psychology+of+AI+models%2E+Proceedings+of+the+National+Academy+of+Sciences+%28PNAS%29%2C+120%2810%29%3Ae2300963120%2E+2)
45. Shin, T., Razeghi, Y., Logan IV, R. L., Wallace, E., and Singh, S. (2020). [Autoprompt: Eliciting knowledge from language models with automatically generated prompts](https://md.research.elamur.ai/free/en/papers/shin-2020-autoprompt-eliciting-knowledge-language-models/). In Annual Conference on Empirical Methods in Natural Language Processing (EMNLP). 8
46. Taori, R., Gulrajani, I., Zhang, T., Dubois, Y., Li, X., Guestrin, C., Liang, P., and Hashimoto, T. B. (2023). Stanford alpaca: An instruction-following llama model. [https://github.com/tatsu-lab/stanford_](https://github.com/tatsu-lab/stanford_) alpaca. 6, 3 [↗](https://github.com/tatsu-lab/stanford_)
47. Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023). [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). arXiv preprint arXiv:2302.13971. 6, 3
48. Wang, Y., Kordi, Y., Mishra, S., Liu, A., Smith, N. A., Khashabi, D., and Hajishirzi, H. (2022). Self-instruct: Aligning language model with self generated instructions. arXiv preprint arXiv:2212.10560. 6 [find](https://scholar.google.com/scholar?q=Wang%2C+Y%2E%2C+Kordi%2C+Y%2E%2C+Mishra%2C+S%2E%2C+Liu%2C+A%2E%2C+Smith%2C+N%2E+A%2E%2C+Khashabi%2C+D%2E%2C+and+Hajishirzi%2C+H%2E+%282022%29%2E+Self%2Dinstruct%3A+Aligning+language+model+with+self+generated+instructions%2E+arXiv+preprint+arXiv%3A2212%2E1056)
49. Webb, T., Holyoak, K. J., and Lu, H. (2023). Emergent analogical reasoning in large language models. Nature Human Behaviour, pages 1–16. 3 [find](https://scholar.google.com/scholar?q=Webb%2C+T%2E%2C+Holyoak%2C+K%2E+J%2E%2C+and+Lu%2C+H%2E+%282023%29%2E+Emergent+analogical+reasoning+in+large+language+models%2E+Nature+Human+Behaviour%2C+pages+1%E2%80%9316%2E+3)
50. Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., et al. (2022a). [Emergent abilities of large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-emergent-abilities-large-language-models/). Transactions on Machine Learning Research. 3
51. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q. V., Zhou, D., et al. (2022b). [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). In Advances in Neural Information Processing Systems (NeurIPS), volume 35, pages 24824–24837. 7
52. Weinberg, R. S. and Gould, D. (2019). Foundations of sport and exercise psychology, 7E. Human kinetics. 2 [find](https://scholar.google.com/scholar?q=Weinberg%2C+R%2E+S%2E+and+Gould%2C+D%2E+%282019%29%2E+Foundations+of+sport+and+exercise+psychology%2C+7E%2E+Human+kinetics%2E+2)
53. Weiner, I. B. and Greene, R. L. (2017). Handbook of personality assessment. John Wiley & Sons. 4 [find](https://scholar.google.com/scholar?q=Weiner%2C+I%2E+B%2E+and+Greene%2C+R%2E+L%2E+%282017%29%2E+Handbook+of+personality+assessment%2E+John+Wiley+%26+Sons%2E+4)
54. Williams, A., Nangia, N., and Bowman, S. (2018). [A broad-coverage challenge corpus for sentence understanding through inference](https://md.research.elamur.ai/free/en/papers/williams-2018-broad-coverage-challenge-corpus-sentence/). In North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT). 2
55. Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020). [Transformers: State-of-the-art natural language processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/). In Annual Conference on Empirical Methods in Natural Language Processing (EMNLP). 6
56. Wong, L., Grand, G., Lew, A. K., Goodman, N. D., Mansinghka, V. K., Andreas, J., and Tenenbaum, J. B. (2023). From word models to world models: Translating from natural language to the probabilistic language of thought. arXiv preprint arXiv:2306.12672. 3 [find](https://scholar.google.com/scholar?q=Wong%2C+L%2E%2C+Grand%2C+G%2E%2C+Lew%2C+A%2E+K%2E%2C+Goodman%2C+N%2E+D%2E%2C+Mansinghka%2C+V%2E+K%2E%2C+Andreas%2C+J%2E%2C+and+Tenenbaum%2C+J%2E+B%2E+%282023%29%2E+From+word+models+to+world+models%3A+Translating+from+natural+language+to+the+probabilistic+l)
57. Yin, W., Hay, J., and Roth, D. (2019). [Benchmarking zero-shot text classification: Datasets, evaluation and entailment approach](https://md.research.elamur.ai/free/en/papers/yin-2019-benchmarking-zero-shot-text-classification/). In Annual Conference on Empirical Methods in Natural Language Processing (EMNLP). 2
58. Zhang, C., Gao, F., Jia, B., Zhu, Y., and Zhu, S.-C. (2019). Raven: A dataset for relational and analogical visual reasoning. In Conference on Computer Vision and Pattern Recognition (CVPR). 2 [find](https://scholar.google.com/scholar?q=Zhang%2C+C%2E%2C+Gao%2C+F%2E%2C+Jia%2C+B%2E%2C+Zhu%2C+Y%2E%2C+and+Zhu%2C+S%2E%2DC%2E+%282019%29%2E+Raven%3A+A+dataset+for+relational+and+analogical+visual+reasoning%2E+In+Conference+on+Computer+Vision+and+Pattern+Recognition+%28CVPR%29%2E+2)
59. Zhang, S., Dinan, E., Urbanek, J., Szlam, A., Kiela, D., and Weston, J. (2018). [Personalizing dialogue agents: I have a dog, do you have pets too?](https://md.research.elamur.ai/free/en/papers/zhang-2018-personalizing-dialogue-agents-i-have/) In Annual Meeting of the Association for Computational Linguistics (ACL). 3
60. Zhu, Y., Gao, T., Fan, L., Huang, S., Edmonds, M., Liu, H., Gao, F., Zhang, C., Qi, S., Wu, Y. N., et al. (2020). Dark, beyond deep: A paradigm shift to cognitive ai with humanlike common sense. Engineering, 6(3):310–345. 2 [find](https://scholar.google.com/scholar?q=Zhu%2C+Y%2E%2C+Gao%2C+T%2E%2C+Fan%2C+L%2E%2C+Huang%2C+S%2E%2C+Edmonds%2C+M%2E%2C+Liu%2C+H%2E%2C+Gao%2C+F%2E%2C+Zhang%2C+C%2E%2C+Qi%2C+S%2E%2C+Wu%2C+Y%2E+N%2E%2C+et+al%2E+%282020%29%2E+Dark%2C+beyond+deep%3A+A+paradigm+shift+to+cognitive+ai+with+humanlike+common+sense%2E+Engin)
61. Ziems, C., Held, W., Shaikh, O., Chen, J., Zhang, Z., and Yang, D. (2023). [Can large language models transform computational social science?](https://md.research.elamur.ai/free/en/papers/ziems-2023-large-language-models-transform-computational/) arXiv preprint arXiv:2305.03514. 3

*Updated 2 November 2026*