---
title: "Sentient Agent as a Judge — paper by Zhang et al. (2026)"
description: "The authors propose SAGE (Sentient Agent as a Judge), an automated framework in which an… 59 references, cited by 6 papers here."
authors: ["Bang Zhang", "Ruotian Ma", "Qingxuan Jiang", "Peisong Wang", "Jiaqi Chen", "Zheng Xie", "Xingyu Chen", "Yue Wang", "Fanghua Ye", "Jian Li", "Yifan Yang", "Zhaopeng Tu", "Xiaolong Li"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.1905/"
doi: "10.18653/v1/2026.findings-acl.1905"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2026-sentient-agent-as-judge-evaluating/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2026-sentient-agent-as-judge-evaluating/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/3/) / Sentient Agent as a Judge: Evaluating Higher-Order Social Cognition in Large Language Models

# Sentient Agent as a Judge: Evaluating Higher-Order Social Cognition in Large Language Models

**Authors:** [Bang Zhang](https://md.research.elamur.ai/free/en/authors/bang-zhang/), [Ruotian Ma](https://md.research.elamur.ai/free/en/authors/ruotian-ma/), [Qingxuan Jiang](https://md.research.elamur.ai/free/en/authors/qingxuan-jiang/), [Peisong Wang](https://md.research.elamur.ai/free/en/authors/peisong-wang/), [Jiaqi Chen](https://md.research.elamur.ai/free/en/authors/jiaqi-chen/), Zheng Xie, [Xingyu Chen](https://md.research.elamur.ai/free/en/authors/xingyu-chen/), [Yue Wang](https://md.research.elamur.ai/free/en/authors/yue-wang/), [Fanghua Ye](https://md.research.elamur.ai/free/en/authors/fanghua-ye/), [Jian Li](https://md.research.elamur.ai/free/en/authors/jian-li/), [Yifan Yang](https://md.research.elamur.ai/free/en/authors/yifan-yang/), [Zhaopeng Tu](https://md.research.elamur.ai/free/en/authors/zhaopeng-tu/) and [Xiaolong Li](https://md.research.elamur.ai/free/en/authors/xiaolong-li/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.1905/) · [PDF](https://aclanthology.org/2026.findings-acl.1905.pdf) · [DOI 10.18653/v1/2026.findings-acl.1905](https://doi.org/10.18653/v1/2026.findings-acl.1905) · [OpenAlex](https://openalex.org/W4417084321)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [6](https://md.research.elamur.ai/free/en/papers/zhang-2026-sentient-agent-as-judge-evaluating/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Standard LLM leaderboards reward textual skill and helpfulness, but they rarely check whether a model actually reads people well. That means noticing subtle feelings, inferring unstated goals, and responding with fitting empathy. The authors propose SAGE (Sentient Agent as a Judge), an automated framework in which an LLM-driven simulated user talks with the tested model over several turns. Each simulated user is built from a persona, a dialogue background, an explicit goal, and a concealed intention. After every reply, the agent reasons step by step about how its mood shifts, records a numeric emotion score, and writes its next message. The resulting emotion trajectory acts as the metric, and the logged inner thoughts explain each score. On a benchmark of 100 emotional-support scenarios, the final scores agree well with established psychological measures and with human annotators. A leaderboard of 18 models shows large differences that Arena-style rankings miss. This matters because LLMs increasingly serve as companions and advisors, where the user's emotional outcome is central.

## Key points
- Validation: the final emotion scores show a Pearson correlation of r = 0.82 with BLRI ratings and r = 0.79 with empathy measures computed for each utterance. Human judges agreed with the agent's emotional reasoning 85.3% of the time, and different judge-model families ranked models consistently (Spearman ρ ≥ 0.84).
- The Sentient Leaderboard covers 18 commercial and open-source models. Frontier systems such as GPT-4o-Latest and Gemini2.5-Pro score up to about 4× higher than older baselines, and the ordering differs noticeably from Arena.
- GPT-4o-Latest reaches the top score while also using tokens efficiently, which suggests strong social reasoning does not require long answers.
- The framework transferred zero-shot to a chitchat setting by changing only the goal and hidden-intention fields. Scores ranged from 37.8 (Qwen2.5-7B-Instruct) to 80.9 (GPT-4o-0326).
- Stated limitations:
  - The study covers English emotional-support dialogue only.
  - Emotion is reduced to a single scalar score.
  - The judge model may carry its own biases.
- Code is published at github.com/tencent/digitalhuman/SAGE.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,954 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 626 works

## Cite

Cite as: Bang Zhang, Ruotian Ma, Qingxuan Jiang, Peisong Wang, Jiaqi Chen, Zheng Xie, Xingyu Chen, Yue Wang, Fanghua Ye, Jian Li, Yifan Yang, Zhaopeng Tu and Xiaolong Li. 2026. Sentient Agent as a Judge: Evaluating Higher-Order Social Cognition in Large Language Models. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhang2026sentient,
  title = {{Sentient Agent as a Judge: Evaluating Higher-Order Social Cognition in Large Language Models}},
  author = {Bang Zhang, Ruotian Ma, Qingxuan Jiang, Peisong Wang, Jiaqi Chen, Zheng Xie, Xingyu Chen, Yue Wang, Fanghua Ye, Jian Li, Yifan Yang, Zhaopeng Tu and Xiaolong Li},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1905},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1905}
}
```

- [← previous Scaling Behaviors of LLM Reinforcement Learning Post-Training: An Empirical Study in Mathematical Reasoning](https://md.research.elamur.ai/free/en/papers/tan-2026-scaling-behaviors-llm-reinforcement-learning/)
- [next → Shifting Perspectives: Steering Vectors for Robust Bias Mitigation in LLMs](https://md.research.elamur.ai/free/en/papers/siddique-2026-shifting-perspectives-steering-vectors-robust/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/3/)

## References

59 references, 25 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Godfrey T Barrett-Lennard. 2015. The relationship inventory: A complete resource and guide. John Wiley & Sons. [find](https://scholar.google.com/scholar?q=Godfrey+T+Barrett%2DLennard%2E+2015%2E+The+relationship+inventory%3A+A+complete+resource+and+guide%2E+John+Wiley+%26+Sons%2E)
2. Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. 2023. [Chateval: Towards better llm-based evaluators through multi-agent debate](https://md.research.elamur.ai/free/en/papers/chan-2023-chateval-better-llm-based/). arXiv preprint arXiv:2308.07201.
3. Hongzhan Chen, Hehong Chen, Ming Yan, Wenshen Xu, Xing Gao, Weizhou Shen, Xiaojun Quan, Chenliang Li, Ji Zhang, Fei Huang, and 1 others. 2024a. Socialbench: Sociality evaluation of role-playing conversational agents. arXiv preprint arXiv:2403.13679. [find](https://scholar.google.com/scholar?q=Hongzhan+Chen%2C+Hehong+Chen%2C+Ming+Yan%2C+Wenshen+Xu%2C+Xing+Gao%2C+Weizhou+Shen%2C+Xiaojun+Quan%2C+Chenliang+Li%2C+Ji+Zhang%2C+Fei+Huang%2C+and+1+others%2E+2024a%2E+Socialbench%3A+Sociality+evaluation+of+role%2Dplaying+conver)
4. Shun Chen, Faith Liao, David Murphy, and Stephen Joseph. 2023. Development and validation of a 12-item version of the barrett-lennard relationship inventory (bl ri: mini) using item response theory. Current Psychology, 42(13):10566–10580. [find](https://scholar.google.com/scholar?q=Shun+Chen%2C+Faith+Liao%2C+David+Murphy%2C+and+Stephen+Joseph%2E+2023%2E+Development+and+validation+of+a+12%2Ditem+version+of+the+barrett%2Dlennard+relationship+inventory+%28bl+ri%3A+mini%29+using+item+response+theory%2E+C)
5. Zixiang Chen, Yihe Deng, Huizhuo Yuan, Kaixuan Ji, and Quanquan Gu. 2024b. Self-play fine-tuning converts weak language models to strong language models. arXiv preprint arXiv:2401.01335. [find](https://scholar.google.com/scholar?q=Zixiang+Chen%2C+Yihe+Deng%2C+Huizhuo+Yuan%2C+Kaixuan+Ji%2C+and+Quanquan+Gu%2E+2024b%2E+Self%2Dplay+fine%2Dtuning+converts+weak+language+models+to+strong+language+models%2E+arXiv+preprint+arXiv%3A2401%2E01335%2E)
6. Antoine Chevrot, Alexandre Vernotte, Jean-Rémy Falleri, Xavier Blanc, and Bruno Legeard. 2025. Are autonomous web agents good testers? arXiv preprint arXiv:2504.01495. — [arXiv:2504.01495](https://arxiv.org/abs/2504.01495) [find](https://scholar.google.com/scholar?q=Antoine+Chevrot%2C+Alexandre+Vernotte%2C+Jean%2DRe%CC%81my+Falleri%2C+Xavier+Blanc%2C+and+Bruno+Legeard%2E+2025%2E+Are+autonomous+web+agents+good+testers%3F+arXiv+preprint+arXiv%3A2504%2E01495%2E+%E2%80%94+%5BarXiv%3A2504%2E01495%5D%28https%3A%2F%2Far)
7. Wei-Lin Chiang, Lianmin Zheng, Ying Sheng, Anastasios Nikolas Angelopoulos, Tianle Li, Dacheng Li, Banghua Zhu, Hao Zhang, Michael Jordan, Joseph E Gonzalez, and 1 others. 2024. [Chatbot arena: An open platform for evaluating llms by human preference](https://md.research.elamur.ai/free/en/papers/chiang-2024-chatbot-arena-open-platform-evaluating/). In Forty-first International Conference on Machin…
8. Yann Dubois, Chen Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy S Liang, and Tatsunori B Hashimoto. 2023. [Alpacafarm: A simulation framework for methods that learn from human feedback](https://md.research.elamur.ai/free/en/papers/dubois-2023-alpacafarm-simulation-framework-methods-that/). Advances in Neural Information Processing Systems, 36:30039–30069.
9. Yinghui He, Yufan Wu, Yilin Jia, Rada Mihalcea, Yulong Chen, and Naihao Deng. 2023. Hi-tom: A benchmark for evaluating higher-order theory of mind reasoning in large language models. arXiv preprint arXiv:2310.16755. [find](https://scholar.google.com/scholar?q=Yinghui+He%2C+Yufan+Wu%2C+Yilin+Jia%2C+Rada+Mihalcea%2C+Yulong+Chen%2C+and+Naihao+Deng%2E+2023%2E+Hi%2Dtom%3A+A+benchmark+for+evaluating+higher%2Dorder+theory+of+mind+reasoning+in+large+language+models%2E+arXiv+preprint+ar)
10. Wenyue Hua, Xianjun Yang, Mingyu Jin, Zelong Li, Wei Cheng, Ruixiang Tang, and Yongfeng Zhang. 2024. Trustagent: Towards safe and trustworthy llm-based agents through agent constitution. In Trustworthy Multi-modal Foundation Models and AI Agents (TiFA). [find](https://scholar.google.com/scholar?q=Wenyue+Hua%2C+Xianjun+Yang%2C+Mingyu+Jin%2C+Zelong+Li%2C+Wei+Cheng%2C+Ruixiang+Tang%2C+and+Yongfeng+Zhang%2E+2024%2E+Trustagent%3A+Towards+safe+and+trustworthy+llm%2Dbased+agents+through+agent+constitution%2E+In+Trustworth)
11. Jen-tse Huang, Man Ho LAM, Eric John Li, Shujie Ren, Wenxuan Wang, Wenxiang Jiao, Zhaopeng Tu, and Michael Lyu. 2024a. Apathetic or empathetic? evaluating LLMs’ emotional alignments with humans. In The Thirty-eighth Annual Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Jen%2Dtse+Huang%2C+Man+Ho+LAM%2C+Eric+John+Li%2C+Shujie+Ren%2C+Wenxuan+Wang%2C+Wenxiang+Jiao%2C+Zhaopeng+Tu%2C+and+Michael+Lyu%2E+2024a%2E+Apathetic+or+empathetic%3F+evaluating+LLMs%E2%80%99+emotional+alignments+with+humans%2E+In+Th)
12. Jen-tse Huang, Eric John Li, Man Ho LAM, Tian Liang, Wenxuan Wang, Youliang Yuan, Wenxiang Jiao, Xing Wang, Zhaopeng Tu, and Michael Lyu. 2025. Competing large language models in multi-agent gaming environments. In The Thirteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Jen%2Dtse+Huang%2C+Eric+John+Li%2C+Man+Ho+LAM%2C+Tian+Liang%2C+Wenxuan+Wang%2C+Youliang+Yuan%2C+Wenxiang+Jiao%2C+Xing+Wang%2C+Zhaopeng+Tu%2C+and+Michael+Lyu%2E+2025%2E+Competing+large+language+models+in+multi%2Dagent+gaming+en)
13. Jen-tse Huang, Wenxuan Wang, Eric John Li, Man Ho LAM, Shujie Ren, Youliang Yuan, Wenxiang Jiao, Zhaopeng Tu, and Michael Lyu. 2024b. On the humanity of conversational AI: Evaluating the psychological portrayal of LLMs. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Jen%2Dtse+Huang%2C+Wenxuan+Wang%2C+Eric+John+Li%2C+Man+Ho+LAM%2C+Shujie+Ren%2C+Youliang+Yuan%2C+Wenxiang+Jiao%2C+Zhaopeng+Tu%2C+and+Michael+Lyu%2E+2024b%2E+On+the+humanity+of+conversational+AI%3A+Evaluating+the+psychological)
14. Yeonseok Jeong, Minsoo Kim, Seung-won Hwang, and Byung-Hak Kim. 2025. [Agent-as-judge for factual summarization of long narratives](https://md.research.elamur.ai/free/en/papers/jeong-2025-agent-as-judge-factual-summarization/). arXiv preprint arXiv:2501.09993.
15. Zachary Kenton, Noah Siegel, János Kramár, Jonah Brown-Cohen, Samuel Albanie, Jannis Bulian, Rishabh Agarwal, David Lindner, Yunhao Tang, Noah Goodman, and 1 others. 2024. On scalable oversight with weak llms judging strong llms. Advances in Neural Information Processing Systems, 37:75229–75276. [find](https://scholar.google.com/scholar?q=Zachary+Kenton%2C+Noah+Siegel%2C+Ja%CC%81nos+Krama%CC%81r%2C+Jonah+Brown%2DCohen%2C+Samuel+Albanie%2C+Jannis+Bulian%2C+Rishabh+Agarwal%2C+David+Lindner%2C+Yunhao+Tang%2C+Noah+Goodman%2C+and+1+others%2E+2024%2E+On+scalable+oversight+with)
16. Hyunwoo Kim, Melanie Sclar, Xuhui Zhou, Ronan Le Bras, Gunhee Kim, Yejin Choi, and Maarten Sap. 2023. Fantom: A benchmark for stress-testing machine theory of mind in interactions. arXiv preprint arXiv:2310.15421. [find](https://scholar.google.com/scholar?q=Hyunwoo+Kim%2C+Melanie+Sclar%2C+Xuhui+Zhou%2C+Ronan+Le+Bras%2C+Gunhee+Kim%2C+Yejin+Choi%2C+and+Maarten+Sap%2E+2023%2E+Fantom%3A+A+benchmark+for+stress%2Dtesting+machine+theory+of+mind+in+interactions%2E+arXiv+preprint+arXi)
17. Gregory G Kolden, Marjorie H Klein, Chia-Chiang Wang, and Sara B Austin. 2011. Congruence/genuineness. Psychotherapy, 48(1):65. [find](https://scholar.google.com/scholar?q=Gregory+G+Kolden%2C+Marjorie+H+Klein%2C+Chia%2DChiang+Wang%2C+and+Sara+B+Austin%2E+2011%2E+Congruence%2Fgenuineness%2E+Psychotherapy%2C+48%281%29%3A65%2E)
18. Harrison Lee, Samrat Phatale, Hassan Mansoor, Thomas Mesnard, Johan Ferret, Kellie Lu, Colton Bishop, Ethan Hall, Victor Carbune, Abhinav Rastogi, and 1 others. 2023. Rlaif vs. rlhf: Scaling reinforcement learning from human feedback with ai feedback. arXiv preprint arXiv:2309.00267. [find](https://scholar.google.com/scholar?q=Harrison+Lee%2C+Samrat+Phatale%2C+Hassan+Mansoor%2C+Thomas+Mesnard%2C+Johan+Ferret%2C+Kellie+Lu%2C+Colton+Bishop%2C+Ethan+Hall%2C+Victor+Carbune%2C+Abhinav+Rastogi%2C+and+1+others%2E+2023%2E+Rlaif+vs%2E+rlhf%3A+Scaling+reinforce)
19. Qintong Li, Piji Li, Zhaochun Ren, Pengjie Ren, and Zhumin Chen. 2022. Knowledge bridging for empathetic dialogue generation. Proceedings of the AAAI Conference on Artificial Intelligence, 36(10):10993– 11001. [find](https://scholar.google.com/scholar?q=Qintong+Li%2C+Piji+Li%2C+Zhaochun+Ren%2C+Pengjie+Ren%2C+and+Zhumin+Chen%2E+2022%2E+Knowledge+bridging+for+empathetic+dialogue+generation%2E+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+36%2810%29%3A1099)
20. Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Shuming Shi, and Zhaopeng Tu. 2024. [Encouraging divergent thinking in large language models through multi-agent debate](https://md.research.elamur.ai/free/en/papers/liang-2023-encouraging-divergent-thinking-large-language/). In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing.
21. Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. [Let’s verify step by step](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/). In The Twelfth International Conference on Learning Representations.
22. Siyang Liu, Chujie Zheng, Orianna Demasi, Sahand Sabour, Yu Li, Zhou Yu, Yong Jiang, and Minlie Huang. 2021. [Towards emotional support dialog systems](https://md.research.elamur.ai/free/en/papers/liu-2021-emotional-support-dialog-systems/). arXiv preprint arXiv:2106.01144.
23. Yinhong Liu, Han Zhou, Zhijiang Guo, Ehsan Shareghi, Ivan Vulić, Anna Korhonen, and Nigel Collier. 2024. [Aligning with human judgement: The role of pairwise preference in large language model evaluators](https://md.research.elamur.ai/free/en/papers/liu-2024-aligning-human-judgement-role-pairwise/). arXiv preprint arXiv:2403.16950.
24. Ryan Louie, Ananjan Nandi, William Fang, Cheng Chang, Emma Brunskill, and Diyi Yang. 2024. [Roleplay-doh: Enabling domain-experts to create llm-simulated patients via eliciting and adhering to principles](https://md.research.elamur.ai/free/en/papers/louie-2024-roleplay-doh-enabling-domain-experts/). arXiv preprint arXiv:2407.00870.
25. Mounica Maddela, Megan Ung, Jing Xu, Andrea Madotto, Heather Foran, and Y-Lan Boureau. 2023. Training models to generate, recognize, and reframe unhelpful thoughts. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 13641– 13660,… [find](https://scholar.google.com/scholar?q=Mounica+Maddela%2C+Megan+Ung%2C+Jing+Xu%2C+Andrea+Madotto%2C+Heather+Foran%2C+and+Y%2DLan+Boureau%2E+2023%2E+Training+models+to+generate%2C+recognize%2C+and+reframe+unhelpful+thoughts%2E+In+Proceedings+of+the+61st+Annual+M)
26. Justin M Mittelstädt, Julia Maier, Panja Goerke, Frank Zinn, and Michael Hermes. 2024. Large language models can outperform humans in social situational judgments. Scientific Reports, 14(1):27449. [find](https://scholar.google.com/scholar?q=Justin+M+Mittelsta%CC%88dt%2C+Julia+Maier%2C+Panja+Goerke%2C+Frank+Zinn%2C+and+Michael+Hermes%2E+2024%2E+Large+language+models+can+outperform+humans+in+social+situational+judgments%2E+Scientific+Reports%2C+14%281%29%3A27449%2E)
27. Xinyi Mou, Jingcong Liang, Jiayu Lin, Xinnong Zhang, Xiawei Liu, Shiyue Yang, Rong Ye, Lei Chen, Haoyu Kuang, Xuanjing Huang, and 1 others. 2024. Agentsense: Benchmarking social intelligence of language agents through interactive scenarios. arXiv preprint arXiv:2410.19346. [find](https://scholar.google.com/scholar?q=Xinyi+Mou%2C+Jingcong+Liang%2C+Jiayu+Lin%2C+Xinnong+Zhang%2C+Xiawei+Liu%2C+Shiyue+Yang%2C+Rong+Ye%2C+Lei+Chen%2C+Haoyu+Kuang%2C+Xuanjing+Huang%2C+and+1+others%2E+2024%2E+Agentsense%3A+Benchmarking+social+intelligence+of+langua)
28. Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah. 2023. [Orca: Progressive learning from complex explanation traces of gpt-4](https://md.research.elamur.ai/free/en/papers/mukherjee-2023-orca-progressive-learning-complex-explanation/). arXiv preprint arXiv:2306.02707.
29. Junsoo Park, Seungyeon Jwa, Meiying Ren, Daeyoung Kim, and Sanghyuk Choi. 2024. [Offsetbias: Leveraging debiased data for tuning evaluators](https://md.research.elamur.ai/free/en/papers/park-2024-offsetbias-leveraging-debiased-data-tuning/). arXiv preprint arXiv:2407.06551.
30. Zhen Qin, Rolf Jagerman, Kai Hui, Honglei Zhuang, Junru Wu, Le Yan, Jiaming Shen, Tianqi Liu, Jialu Liu, Donald Metzler, and 1 others. 2023. [Large language models are effective text rankers with pairwise ranking prompting](https://md.research.elamur.ai/free/en/papers/qin-2024-large-language-models-are-effective/). arXiv preprint arXiv:2306.17563.
31. Carl R Rogers. 2001. Client-centered/person-centered approach to therapy. Voprosy Psikhologii, (2):48–58. [find](https://scholar.google.com/scholar?q=Carl+R+Rogers%2E+2001%2E+Client%2Dcentered%2Fperson%2Dcentered+approach+to+therapy%2E+Voprosy+Psikhologii%2C+%282%29%3A48%E2%80%9358%2E)
32. Sahand Sabour, Siyang Liu, Zheyuan Zhang, June M Liu, Jinfeng Zhou, Alvionna S Sunaryo, Juanzi Li, Tatia Lee, Rada Mihalcea, and Minlie Huang. 2024. [Emobench: Evaluating the emotional intelligence of large language models](https://md.research.elamur.ai/free/en/papers/sabour-2024-emobench-evaluating-emotional-intelligence-large/). arXiv preprint arXiv:2402.12071.
33. Swarnadeep Saha, Omer Levy, Asli Celikyilmaz, Mohit Bansal, Jason Weston, and Xian Li. 2023. [Branch-solve-merge improves large language model evaluation and generation](https://md.research.elamur.ai/free/en/papers/saha-2023-branch-solve-merge-improves-large/). arXiv preprint arXiv:2310.15123.
34. Vinay Samuel, Henry Peng Zou, Yue Zhou, Shreyas Chaudhari, Ashwin Kalyan, Tanmay Rajpurohit, Ameet Deshpande, Karthik Narasimhan, and Vishvak Murahari. 2024. Personagym: Evaluating persona agents and llms. arXiv preprint arXiv:2407.18416. [find](https://scholar.google.com/scholar?q=Vinay+Samuel%2C+Henry+Peng+Zou%2C+Yue+Zhou%2C+Shreyas+Chaudhari%2C+Ashwin+Kalyan%2C+Tanmay+Rajpurohit%2C+Ameet+Deshpande%2C+Karthik+Narasimhan%2C+and+Vishvak+Murahari%2E+2024%2E+Personagym%3A+Evaluating+persona+agents+and+)
35. Maarten Sap, Ronan Le Bras, Daniel Fried, and Yejin Choi. 2022. Neural theory-of-mind? on the limits of social intelligence in large LMs. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3762–3780, Abu Dhabi, United Arab Emirates. Association for Compu… [find](https://scholar.google.com/scholar?q=Maarten+Sap%2C+Ronan+Le+Bras%2C+Daniel+Fried%2C+and+Yejin+Choi%2E+2022%2E+Neural+theory%2Dof%2Dmind%3F+on+the+limits+of+social+intelligence+in+large+LMs%2E+In+Proceedings+of+the+2022+Conference+on+Empirical+Methods+in+)
36. Omar Shaikh, Valentino Emil Chai, Michele Gelfand, Diyi Yang, and Michael S Bernstein. 2024. Rehearsal: Simulating conflict to teach conflict resolution. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, pages 1– 20. [find](https://scholar.google.com/scholar?q=Omar+Shaikh%2C+Valentino+Emil+Chai%2C+Michele+Gelfand%2C+Diyi+Yang%2C+and+Michael+S+Bernstein%2E+2024%2E+Rehearsal%3A+Simulating+conflict+to+teach+conflict+resolution%2E+In+Proceedings+of+the+2024+CHI+Conference+on+H)
37. Saul B Needleman and Christian D Wunsch. 1970. A general method applicable to the search for similarities in the amino acid sequence of two proteins. Journal of molecular biology, 48(3):443–453. [find](https://scholar.google.com/scholar?q=Saul+B+Needleman+and+Christian+D+Wunsch%2E+1970%2E+A+general+method+applicable+to+the+search+for+similarities+in+the+amino+acid+sequence+of+two+proteins%2E+Journal+of+molecular+biology%2C+48%283%29%3A443%E2%80%93453%2E)
38. Natalie Shapira, Mosh Levy, Seyed Hossein Alavi, Xuhui Zhou, Yejin Choi, Yoav Goldberg, Maarten Sap, and Vered Shwartz. 2023. Clever hans or neural theory of mind? stress testing social reasoning in large language models. arXiv preprint arXiv:2305.14763. [find](https://scholar.google.com/scholar?q=Natalie+Shapira%2C+Mosh+Levy%2C+Seyed+Hossein+Alavi%2C+Xuhui+Zhou%2C+Yejin+Choi%2C+Yoav+Goldberg%2C+Maarten+Sap%2C+and+Vered+Shwartz%2E+2023%2E+Clever+hans+or+neural+theory+of+mind%3F+stress+testing+social+reasoning+in+l)
39. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in neural information processing systems, 35:27730–27744.
40. Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. [Reflexion: Language agents with verbal reinforcement learning](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/). Advances in Neural Information Processing Systems, 36:8634–8652.
41. Samuel J Paech. 2023. [Eq-bench: An emotional intelligence benchmark for large language models](https://md.research.elamur.ai/free/en/papers/paech-2023-eq-bench-emotional-intelligence-benchmark/). arXiv preprint arXiv:2312.06281.
42. James WA Strachan, Dalila Albergo, Giulia Borghini, Oriana Pansardi, Eugenio Scaliti, Saurabh Gupta, Krati Saxena, Alessandro Rufo, Stefano Panzeri, Guido Manzi, and 1 others. 2024. Testing theory of mind in large language models and humans. Nature Human Behaviour, 8(7):1285–1295. [find](https://scholar.google.com/scholar?q=James+WA+Strachan%2C+Dalila+Albergo%2C+Giulia+Borghini%2C+Oriana+Pansardi%2C+Eugenio+Scaliti%2C+Saurabh+Gupta%2C+Krati+Saxena%2C+Alessandro+Rufo%2C+Stefano+Panzeri%2C+Guido+Manzi%2C+and+1+others%2E+2024%2E+Testing+theory+of+)
43. Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An instruction-following llama model. [https://github.com/tatsu-lab/](https://github.com/tatsu-lab/) stanford_alpaca. [↗](https://github.com/tatsu-lab/)
44. Quan Tu, Shilong Fan, Zihang Tian, and Rui Yan. 2024. [Charactereval: A chinese benchmark for role-playing conversational agent evaluation](https://md.research.elamur.ai/free/en/papers/tu-2024-charactereval-chinese-benchmark-role-playing/). arXiv preprint arXiv:2401.01275.
45. Ruiyi Wang, Haofei Yu, Wenxin Zhang, Zhengyang Qi, Maarten Sap, Graham Neubig, Yonatan Bisk, and Hao Zhu. 2024. Sotopia-π: Interactive learning of socially intelligent language agents. arXiv preprint arXiv:2403.08715. — [arXiv:2403.08715](https://arxiv.org/abs/2403.08715) [find](https://scholar.google.com/scholar?q=Ruiyi+Wang%2C+Haofei+Yu%2C+Wenxin+Zhang%2C+Zhengyang+Qi%2C+Maarten+Sap%2C+Graham+Neubig%2C+Yonatan+Bisk%2C+and+Hao+Zhu%2E+2024%2E+Sotopia%2D%CF%80%3A+Interactive+learning+of+socially+intelligent+language+agents%2E+arXiv+preprint+)
46. Xintao Wang, Yunze Xiao, Jen-tse Huang, Siyu Yuan, Rui Xu, Haoran Guo, Quan Tu, Yaying Fei, Ziang Leng, Wei Wang, and 1 others. 2023a. [Incharacter: Evaluating personality fidelity in role-playing agents through psychological interviews](https://md.research.elamur.ai/free/en/papers/wang-2024-incharacter-evaluating-personality-fidelity-role/). arXiv preprint arXiv:2310.17976.
47. Xuena Wang, Xueting Li, Zi Yin, Yue Wu, and Jia Liu. 2023b. Emotional intelligence of large language models. Journal of Pacific Rim Psychology, 17:18344909231213958. [find](https://scholar.google.com/scholar?q=Xuena+Wang%2C+Xueting+Li%2C+Zi+Yin%2C+Yue+Wu%2C+and+Jia+Liu%2E+2023b%2E+Emotional+intelligence+of+large+language+models%2E+Journal+of+Pacific+Rim+Psychology%2C+17%3A18344909231213958%2E)
48. Jinfeng Zhou, Zhuang Chen, Dazhen Wan, Bosi Wen, Yi Song, Jifan Yu, Yongkang Huang, Libiao Peng, Jiaming Yang, Xiyao Xiao, and 1 others. 2023a. [Characterglm: Customizing chinese conversational ai characters with large language models](https://md.research.elamur.ai/free/en/papers/zhou-2023-characterglm-customizing-chinese-conversational-ai/). arXiv preprint arXiv:2311.16832.
49. Jinfeng Zhou, Zhuang Chen, Bo Wang, and Minlie Huang. 2023b. Facilitating multi-turn emotional support conversation with positive emotion elicitation: A reinforcement learning approach. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)… [find](https://scholar.google.com/scholar?q=Jinfeng+Zhou%2C+Zhuang+Chen%2C+Bo+Wang%2C+and+Minlie+Huang%2E+2023b%2E+Facilitating+multi%2Dturn+emotional+support+conversation+with+positive+emotion+elicitation%3A+A+reinforcement+learning+approach%2E+In+Proceedings)
50. Jinfeng Zhou, Chujie Zheng, Bo Wang, Zheng Zhang, and Minlie Huang. 2023c. CASE: Aligning coarse-to-fine cognition and affection for empathetic response generation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 8223–8237, To… [find](https://scholar.google.com/scholar?q=Jinfeng+Zhou%2C+Chujie+Zheng%2C+Bo+Wang%2C+Zheng+Zhang%2C+and+Minlie+Huang%2E+2023c%2E+CASE%3A+Aligning+coarse%2Dto%2Dfine+cognition+and+affection+for+empathetic+response+generation%2E+In+Proceedings+of+the+61st+Annual+M)
51. Xuhui Zhou, Hao Zhu, Leena Mathur, Ruohong Zhang, Haofei Yu, Zhengyang Qi, Louis-Philippe Morency, Yonatan Bisk, Daniel Fried, Graham Neubig, and 1 others. 2023d. [Sotopia: Interactive evaluation for social intelligence in language agents](https://md.research.elamur.ai/free/en/papers/zhou-2023-sotopia-interactive-evaluation-social-intelligence/). arXiv preprint arXiv:2310.11667.
52. Shenghan Wu, Yang Deng, Yimo Zhu, Wynne Hsu, and Mong Li Lee. 2025. From personas to talks: Revisiting the impact of personas on llm-synthesized emotional support conversations. arXiv preprint arXiv:2502.11451. [find](https://scholar.google.com/scholar?q=Shenghan+Wu%2C+Yang+Deng%2C+Yimo+Zhu%2C+Wynne+Hsu%2C+and+Mong+Li+Lee%2E+2025%2E+From+personas+to+talks%3A+Revisiting+the+impact+of+personas+on+llm%2Dsynthesized+emotional+support+conversations%2E+arXiv+preprint+arXiv%3A2)
53. Lianghui Zhu, Xinggang Wang, and Xinlong Wang. 2023. [Judgelm: Fine-tuned large language models are scalable judges](https://md.research.elamur.ai/free/en/papers/zhu-2023-judgelm-fine-tuned-large-language/). arXiv preprint arXiv:2310.17631.
54. Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. 2023. [Wizardlm: Empowering large language models to follow complex instructions](https://md.research.elamur.ai/free/en/papers/xu-2023-wizardlm-empowering-large-pre-trained/). arXiv preprint arXiv:2304.12244.
55. Mingchen Zhuge, Changsheng Zhao, Dylan Ashley, Wenyi Wang, Dmitrii Khizbullin, Yunyang Xiong, Zechun Liu, Ernie Chang, Raghuraman Krishnamoorthi, Yuandong Tian, and 1 others. 2024. Agent-as-a-judge: Evaluate agents with agents. arXiv preprint arXiv:2410.10934. [find](https://scholar.google.com/scholar?q=Mingchen+Zhuge%2C+Changsheng+Zhao%2C+Dylan+Ashley%2C+Wenyi+Wang%2C+Dmitrii+Khizbullin%2C+Yunyang+Xiong%2C+Zechun+Liu%2C+Ernie+Chang%2C+Raghuraman+Krishnamoorthi%2C+Yuandong+Tian%2C+and+1+others%2E+2024%2E+Agent%2Das%2Da%2Djudge%3A+E)
56. Ruoxi Xu, Hongyu Lin, Xianpei Han, Le Sun, and Yingfei Sun. 2024. Academically intelligent llms are not necessarily socially intelligent. arXiv preprint arXiv:2403.06591. — [arXiv:2403.06591](https://arxiv.org/abs/2403.06591) [find](https://scholar.google.com/scholar?q=Ruoxi+Xu%2C+Hongyu+Lin%2C+Xianpei+Han%2C+Le+Sun%2C+and+Yingfei+Sun%2E+2024%2E+Academically+intelligent+llms+are+not+necessarily+socially+intelligent%2E+arXiv+preprint+arXiv%3A2403%2E06591%2E+%E2%80%94+%5BarXiv%3A2403%2E06591%5D%28https%3A%2F%2F)
57. Diyi Yang, Caleb Ziems, William Held, Omar Shaikh, Michael S Bernstein, and John Mitchell. 2024. Social skill training with large language models. arXiv preprint arXiv:2404.04204. [find](https://scholar.google.com/scholar?q=Diyi+Yang%2C+Caleb+Ziems%2C+William+Held%2C+Omar+Shaikh%2C+Michael+S+Bernstein%2C+and+John+Mitchell%2E+2024%2E+Social+skill+training+with+large+language+models%2E+arXiv+preprint+arXiv%3A2404%2E04204%2E)
58. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, and 1 others. 2023. [Judging llm-as-a-judge with mt-bench and chatbot arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). Advances in Neural Information Processing Systems, 36:46595–46623.
59. Jinfeng Zhou, Yuxuan Chen, Jianing Yin, Yongkang Huang, Yihan Shi, Xikun Zhang, Libiao Peng, Rongsheng Zhang, Tangjie Lv, Zhipeng Hu, and 1 others. 2025. [Crisp: Cognitive restructuring of negative thoughts through multi-turn supportive dialogues](https://md.research.elamur.ai/free/en/papers/zhou-2025-crisp-cognitive-restructuring-negative-thoughts/). arXiv preprint arXiv:2504.17238.

*Updated 2 November 2026*