---
title: "EMPATH — paper by Rao et al. (2026) — summary and ideas"
description: "The paper tackles automatic scoring of empathy in individual dialogue responses on a 5-point scale, as in the WASSA 2024… 62 references."
authors: ["Dongning Rao", "Zhihua Liang", "Zhihua Jiang"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.1790/"
doi: "10.18653/v1/2026.findings-acl.1790"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/rao-2026-empath-ensemble-method-automatic-fine/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/rao-2026-empath-ensemble-method-automatic-fine/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/28/) / EMPATH: An Ensemble Method for Automatic Fine-Grained Turn-Level Dialogue Empathy Evaluation with a Novel Emotional Distance Metric

# EMPATH: An Ensemble Method for Automatic Fine-Grained Turn-Level Dialogue Empathy Evaluation with a Novel Emotional Distance Metric

**Authors:** Dongning Rao, Zhihua Liang and Zhihua Jiang · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.1790/) · [PDF](https://aclanthology.org/2026.findings-acl.1790.pdf) · [DOI 10.18653/v1/2026.findings-acl.1790](https://doi.org/10.18653/v1/2026.findings-acl.1790) · [OpenAlex](https://openalex.org/W7166883852)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper tackles automatic scoring of empathy in individual dialogue responses on a 5-point scale, as in the WASSA 2024 shared task (Track 1.2). The authors note three gaps in earlier fine-tuned systems: no interpretable empathy-specific metric, little testing of classical dialogue metrics, and limited use of ensembling. Their framework, EMPATH, combines four kinds of sub-metrics. These are WASSA-oriented predictors such as emotion intensity, GPT-4o judgments grounded in psychology (IRI, RoPE), classical dialogue metrics, and a new Emotional Distance (ED) measure. ED compares embedded emotion and sentiment labels of the user turn and the response, and uses commonsense intent inferred via COMET or CICERO. The sub-metrics are merged with Huffman-Order (HO), which needs no labels. HO ranks sub-metrics by correlation and averages them starting from the weakest. EMPATH outperforms the WASSA 2024 winner, ConText, by about 3% on WASSA and by a larger margin on the EmpatheticExchanges (EX) dataset. This suggests that interpretable, ensemble-based empathy evaluation generalizes beyond the competition data.

## Key points
- Data: the WASSA 2024 dataset (14,472 items, reactions to news about harmed entities) and EX (4,949 samples, derived from EmpatheticDialogues).
- On EX, Pearson r rises from 0.4066 (ConText) to 0.4860 (EMPATH), a statistically significant gain.
- ED works better with a seven-class emotion classifier (MELD-tuned) than with a 27-class GoEmotions model, and "xNeed" proves more useful than "xWant".
- The classical metrics are NLL, DialogRPT, C-PMI and Holistic. They help with empathy scoring, with diversity, interest and reply length emerging as useful signals.
- In ablations, HO beats plain averaging and MLP ensembles, and removing ED causes a notable performance drop. Code is available at github.com/fip-lab/EMPATH.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Guangdong University of Technology](https://md.research.elamur.ai/free/en/universities/guangdong-university-of-technology/) — University, Guangzhou, China · 3 works · 112 citations
- [Jinan University](https://md.research.elamur.ai/free/en/universities/jinan-university/) — University, Guangzhou, China · 5 works · 11 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,552 works · [💭 Experimental and Cognitive Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/experimental-and-cognitive-psychology/) · 870 works

🧭 Topics (2): [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 972 works · [🗨️ Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) · 624 works

## Cite

Cite as: Dongning Rao, Zhihua Liang and Zhihua Jiang. 2026. EMPATH: An Ensemble Method for Automatic Fine-Grained Turn-Level Dialogue Empathy Evaluation with a Novel Emotional Distance Metric. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{rao2026empath,
  title = {{EMPATH: An Ensemble Method for Automatic Fine-Grained Turn-Level Dialogue Empathy Evaluation with a Novel Emotional Distance Metric}},
  author = {Dongning Rao, Zhihua Liang and Zhihua Jiang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1790},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1790}
}
```

- [← previous Emotion–Cause Pair Extraction in Conversations via Semantic Decoupling and Alignment](https://md.research.elamur.ai/free/en/papers/ma-2026-emotion-cause-pair-extraction-conversations/)
- [next → Empathy Applicability Modeling for General Health Queries](https://md.research.elamur.ai/free/en/papers/randhawa-2026-empathy-applicability-modeling-general-health/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/28/)

## References

62 references, 19 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774.
2. Valentin Barriere, João Sedoc, Shabnam Tafreshi, and Salvatore Giorgi. 2023. Findings of wassa 2023 shared task on empathy, emotion and personality detection in conversation and reactions to news articles. In Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, &… [find](https://scholar.google.com/scholar?q=Valentin+Barriere%2C+Jo%C3%A3o+Sedoc%2C+Shabnam+Tafreshi%2C+and+Salvatore+Giorgi%2E+2023%2E+Findings+of+wassa+2023+shared+task+on+empathy%2C+emotion+and+personality+detection+in+conversation+and+reactions+to+news+arti)
3. Xiao Bi, Deli Chen, Guanting Chen, Shanhuang Chen, Damai Dai, Chengqi Deng, Honghui Ding, Kai Dong, Qiushi Du, Zhe Fu, et al. 2024. Deepseek llm: Scaling open-source language models with longtermism. arXiv preprint arXiv:2401.02954. [find](https://scholar.google.com/scholar?q=Xiao+Bi%2C+Deli+Chen%2C+Guanting+Chen%2C+Shanhuang+Chen%2C+Damai+Dai%2C+Chengqi+Deng%2C+Honghui+Ding%2C+Kai+Dong%2C+Qiushi+Du%2C+Zhe+Fu%2C+et+al%2E+2024%2E+Deepseek+llm%3A+Scaling+open%2Dsource+language+models+with+longtermism%2E+)
4. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems, 33:1877–1901.
5. Laurianne Charrier, Alisa Rieger, Alexandre Galdeano, Amélie Cordier, Mathieu Lefort, and Salima Hassas. 2019. The rope scale: a measure of how empathic a robot is perceived. In 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pages 656–657. IEEE. [find](https://scholar.google.com/scholar?q=Laurianne+Charrier%2C+Alisa+Rieger%2C+Alexandre+Galdeano%2C+Am%C3%A9lie+Cordier%2C+Mathieu+Lefort%2C+and+Salima+Hassas%2E+2019%2E+The+rope+scale%3A+a+measure+of+how+empathic+a+robot+is+perceived%2E+In+2019+14th+ACM%2FIEEE+Int)
6. Chih-Yao Chen, Tun Min Hung, Yi-Li Hsu, and Lun-Wei Ku. 2023. Label-aware hyperbolic embeddings for fine-grained emotion classification. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10947–10958. [find](https://scholar.google.com/scholar?q=Chih%2DYao+Chen%2C+Tun+Min+Hung%2C+Yi%2DLi+Hsu%2C+and+Lun%2DWei+Ku%2E+2023%2E+Label%2Daware+hyperbolic+embeddings+for+fine%2Dgrained+emotion+classification%2E+In+Proceedings+of+the+61st+Annual+Meeting+of+the+Association+fo)
7. Deepanway Ghosal, Siqi Shen, Navonil Majumder, Rada Mihalcea, and Soujanya Poria. 2022. Cicero: A dataset for contextualized commonsense inference in dialogues. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5010–5028. [find](https://scholar.google.com/scholar?q=Deepanway+Ghosal%2C+Siqi+Shen%2C+Navonil+Majumder%2C+Rada+Mihalcea%2C+and+Soujanya+Poria%2E+2022%2E+Cicero%3A+A+dataset+for+contextualized+commonsense+inference+in+dialogues%2E+In+Proceedings+of+the+60th+Annual+Meeti)
8. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2024. Scaling instruction-finetuned language models. Journal of Machine Learning Research, 25(70):1–53. [find](https://scholar.google.com/scholar?q=Hyung+Won+Chung%2C+Le+Hou%2C+Shayne+Longpre%2C+Barret+Zoph%2C+Yi+Tay%2C+William+Fedus%2C+Yunxuan+Li%2C+Xuezhi+Wang%2C+Mostafa+Dehghani%2C+Siddhartha+Brahma%2C+et+al%2E+2024%2E+Scaling+instruction%2Dfinetuned+language+models%2E+J)
9. Eric Gilbert. 2014. Vader: A parsimonious rule-based model for sentiment analysis of social media text. In Proceedings of the international AAAI conference on web and social media, volume 8, pages 216–225. [find](https://scholar.google.com/scholar?q=Eric+Gilbert%2E+2014%2E+Vader%3A+A+parsimonious+rule%2Dbased+model+for+sentiment+analysis+of+social+media+text%2E+In+Proceedings+of+the+international+AAAI+conference+on+web+and+social+media%2C+volume+8%2C+pages+216)
10. Svetlana Churina, Preetika Verma, et al. 2024. [find](https://scholar.google.com/scholar?q=Svetlana+Churina%2C+Preetika+Verma%2C+et+al%2E+2024%2E)
11. Wassa 2024 shared task: Enhancing emotional intelligence with prompts. In Proceedings of the 14th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, pages 425–429. [find](https://scholar.google.com/scholar?q=Wassa+2024+shared+task%3A+Enhancing+emotional+intelligence+with+prompts%2E+In+Proceedings+of+the+14th+Workshop+on+Computational+Approaches+to+Subjectivity%2C+Sentiment%2C+%26+Social+Media+Analysis%2C+pages+425%E2%80%9342)
12. Mihaela Curmei, Andreas A Haupt, Benjamin Recht, and Dylan Hadfield-Menell. 2022. Towards psychologically-grounded dynamic preference models. In Proceedings of the 16th ACM Conference on Recommender Systems, pages 35–48. [find](https://scholar.google.com/scholar?q=Mihaela+Curmei%2C+Andreas+A+Haupt%2C+Benjamin+Recht%2C+and+Dylan+Hadfield%2DMenell%2E+2022%2E+Towards+psychologically%2Dgrounded+dynamic+preference+models%2E+In+Proceedings+of+the+16th+ACM+Conference+on+Recommender+S)
13. Mark HA Davis. 1995. A multidimensional approach to individual differences in empathy. Select Press. [find](https://scholar.google.com/scholar?q=Mark+HA+Davis%2E+1995%2E+A+multidimensional+approach+to+individual+differences+in+empathy%2E+Select+Press%2E)
14. Dorottya Demszky, Dana Movshovitz-Attias, Jeongwoo Ko, Alan Cowen, Gaurav Nemade, and Sujith Ravi. 2020. Goemotions: A dataset of fine-grained emotions. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4040–4054. [find](https://scholar.google.com/scholar?q=Dorottya+Demszky%2C+Dana+Movshovitz%2DAttias%2C+Jeongwoo+Ko%2C+Alan+Cowen%2C+Gaurav+Nemade%2C+and+Sujith+Ravi%2E+2020%2E+Goemotions%3A+A+dataset+of+fine%2Dgrained+emotions%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+th)
15. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
16. Salvatore Giorgi, Shreya Havaldar, Farhan Ahmed, Zuhaib Akhtar, Shalaka Vaidya, Gary Pan, Lyle H Ungar, H Andrew Schwartz, and Joao Sedoc. 2023. Psychological metrics for dialog system evaluation. arXiv preprint arXiv:2305.14757. — [arXiv:2305.14757](https://arxiv.org/abs/2305.14757) [find](https://scholar.google.com/scholar?q=Salvatore+Giorgi%2C+Shreya+Havaldar%2C+Farhan+Ahmed%2C+Zuhaib+Akhtar%2C+Shalaka+Vaidya%2C+Gary+Pan%2C+Lyle+H+Ungar%2C+H+Andrew+Schwartz%2C+and+Joao+Sedoc%2E+2023%2E+Psychological+metrics+for+dialog+system+evaluation%2E+arX)
17. Salvatore Giorgi, João Sedoc, Valentin Barriere, and Shabnam Tafreshi. 2024. Findings of wassa 2024 shared task on empathy and personality detection in interactions. In Proceedings of the 14th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, pages 369–379. [find](https://scholar.google.com/scholar?q=Salvatore+Giorgi%2C+Jo%C3%A3o+Sedoc%2C+Valentin+Barriere%2C+and+Shabnam+Tafreshi%2E+2024%2E+Findings+of+wassa+2024+shared+task+on+empathy+and+personality+detection+in+interactions%2E+In+Proceedings+of+the+14th+Worksho)
18. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2020. [Deberta: Decoding-enhanced bert with disentangled attention](https://md.research.elamur.ai/free/en/papers/he-2020-deberta-decoding-enhanced-bert-disentangled/). In International Conference on Learning Representations.
19. Lishan Huang, Zheng Ye, Jinghui Qin, Liang Lin, and Xiaodan Liang. 2020. [Grade: Automatic graph-enhanced coherence metric for evaluating open-domain dialogue systems](https://md.research.elamur.ai/free/en/papers/huang-2020-grade-automatic-graph-enhanced-coherence/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9230–9240.
20. Jena D Hwang, Chandra Bhagavatula, Ronan Le Bras, Jeff Da, Keisuke Sakaguchi, Antoine Bosselut, and Yejin Choi. 2021. [(comet-) atomic 2020: On symbolic and neural commonsense knowledge graphs](https://md.research.elamur.ai/free/en/papers/hwang-2020-comet-atomic-2020-symbolic-neural/). In Proceedings of the AAAI conference on artificial intelligence, volume 35, pages 6384–6392.
21. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783.
22. Allison Lahnala, Charles Welch, David Jurgens, and Lucie Flek. 2022. A critical reflection and forward perspective on empathy and natural language processing. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 2139–2158. [find](https://scholar.google.com/scholar?q=Allison+Lahnala%2C+Charles+Welch%2C+David+Jurgens%2C+and+Lucie+Flek%2E+2022%2E+A+critical+reflection+and+forward+perspective+on+empathy+and+natural+language+processing%2E+In+Findings+of+the+Association+for+Comput)
23. Robert Elliott, Arthur C Bohart, Jeanne C Watson, and Leslie S Greenberg. 2011. Empathy. Psychotherapy, 48(1):43. [find](https://scholar.google.com/scholar?q=Robert+Elliott%2C+Arthur+C+Bohart%2C+Jeanne+C+Watson%2C+and+Leslie+S+Greenberg%2E+2011%2E+Empathy%2E+Psychotherapy%2C+48%281%29%3A43%2E)
24. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. [Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension](https://md.research.elamur.ai/free/en/papers/lewis-2020-bart-denoising-sequence-sequence-pre/). In Proceedings of the 58th Annual Meeting of the…
25. Shaz Furniturewala and Kokil Jaidka. 2024. Empaths at wassa 2024 empathy and personality shared task: Turn-level empathy prediction using psychological indicators. In Proceedings of the 14th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, pages 404–411. [find](https://scholar.google.com/scholar?q=Shaz+Furniturewala+and+Kokil+Jaidka%2E+2024%2E+Empaths+at+wassa+2024+empathy+and+personality+shared+task%3A+Turn%2Dlevel+empathy+prediction+using+psychological+indicators%2E+In+Proceedings+of+the+14th+Workshop+)
26. Xiang Gao, Yizhe Zhang, Michel Galley, Chris Brockett, and William B Dolan. 2020. Dialogue response ranking training with large-scale human feedback data. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 386–395. [find](https://scholar.google.com/scholar?q=Xiang+Gao%2C+Yizhe+Zhang%2C+Michel+Galley%2C+Chris+Brockett%2C+and+William+B+Dolan%2E+2020%2E+Dialogue+response+ranking+training+with+large%2Dscale+human+feedback+data%2E+In+Proceedings+of+the+2020+Conference+on+Empi)
27. Anqi Li, Lizhi Ma, Yaling Mei, Hongliang He, Shuai Zhang, Huachuan Qiu, and Zhenzhong Lan. 2023. Understanding client reactions in online mental health counseling. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 10358– 10376,… [find](https://scholar.google.com/scholar?q=Anqi+Li%2C+Lizhi+Ma%2C+Yaling+Mei%2C+Hongliang+He%2C+Shuai+Zhang%2C+Huachuan+Qiu%2C+and+Zhenzhong+Lan%2E+2023%2E+Understanding+client+reactions+in+online+mental+health+counseling%2E+In+Proceedings+of+the+61st+Annual+Me)
28. Chia-Wei Liu, Ryan Lowe, Iulian Vlad Serban, Mike Noseworthy, Laurent Charlin, and Joelle Pineau. 2016. [How not to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation](https://md.research.elamur.ai/free/en/papers/liu-2016-how-not-evaluate-your-dialogue/). In Proceedings of the 2016 Conference on Empirical Methods in Natu…
29. Siyang Liu, Chujie Zheng, Orianna Demasi, Sahand Sabour, Yu Li, Zhou Yu, Yong Jiang, and Minlie Huang. 2021. [Towards emotional support dialog systems](https://md.research.elamur.ai/free/en/papers/liu-2021-emotional-support-dialog-systems/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Lan…
30. Yinhan Liu. 2019. [Roberta: A robustly optimized bert pretraining approach](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/). arXiv preprint arXiv:1907.11692, 364.
31. Xin Lu, Zhuojun Li, Yanpeng Tong, Yanyan Zhao, and Bing Qin. 2023. HIT-SCIR at WASSA 2023: Empathy and emotion analysis at the utterance-level and the essay-level. In Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis, pages 574–580, Toro… [find](https://scholar.google.com/scholar?q=Xin+Lu%2C+Zhuojun+Li%2C+Yanpeng+Tong%2C+Yanyan+Zhao%2C+and+Bing+Qin%2E+2023%2E+HIT%2DSCIR+at+WASSA+2023%3A+Empathy+and+emotion+analysis+at+the+utterance%2Dlevel+and+the+essay%2Dlevel%2E+In+Proceedings+of+the+13th+Workshop+)
32. Man Luo, Christopher J Warren, Lu Cheng, Haidar M Abdul-Muhsin, and Imon Banerjee. 2024. Assessing empathy in large language models with real-world physician-patient interactions. arXiv preprint arXiv:2405.16402. — [arXiv:2405.16402](https://arxiv.org/abs/2405.16402) [find](https://scholar.google.com/scholar?q=Man+Luo%2C+Christopher+J+Warren%2C+Lu+Cheng%2C+Haidar+M+Abdul%2DMuhsin%2C+and+Imon+Banerjee%2E+2024%2E+Assessing+empathy+in+large+language+models+with+real%2Dworld+physician%2Dpatient+interactions%2E+arXiv+preprint+arXiv)
33. Alvin R Mahrer. 1997. Empathy as therapist-client alignment. [find](https://scholar.google.com/scholar?q=Alvin+R+Mahrer%2E+1997%2E+Empathy+as+therapist%2Dclient+alignment%2E)
34. Shikib Mehri and Maxine Eskenazi. 2020. [Unsupervised evaluation of interactive dialog with dialogpt](https://md.research.elamur.ai/free/en/papers/mehri-2020-unsupervised-evaluation-interactive-dialog-dialogpt/). In 21th Annual Meeting of the Special Interest Group on Discourse and Dialogue, page 225.
35. Edwin C Montiel-Vázquez, Christian Arzate Cruz, Jorge Adolfo Ramírez Uresti, and Randy Gomez. 2024. Empatheticexchanges: Towards understanding the cues for empathy in dyadic conversations. IEEE Access. [find](https://scholar.google.com/scholar?q=Edwin+C+Montiel%2DV%C3%A1zquez%2C+Christian+Arzate+Cruz%2C+Jorge+Adolfo+Ram%C3%ADrez+Uresti%2C+and+Randy+Gomez%2E+2024%2E+Empatheticexchanges%3A+Towards+understanding+the+cues+for+empathy+in+dyadic+conversations%2E+IEEE+Access)
36. Damilola Omitaomu, Shabnam Tafreshi, Tingting Liu, Sven Buechel, Chris Callison-Burch, Johannes Eichstaedt, Lyle Ungar, and João Sedoc. 2022. Empathic conversations: A multi-level dataset of contextualized conversations. arXiv preprint arXiv:2205.12698. — [arXiv:2205.12698](https://arxiv.org/abs/2205.12698) [find](https://scholar.google.com/scholar?q=Damilola+Omitaomu%2C+Shabnam+Tafreshi%2C+Tingting+Liu%2C+Sven+Buechel%2C+Chris+Callison%2DBurch%2C+Johannes+Eichstaedt%2C+Lyle+Ungar%2C+and+Jo%C3%A3o+Sedoc%2E+2022%2E+Empathic+conversations%3A+A+multi%2Dlevel+dataset+of+contextua)
37. OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, Irwan Bello, Ja… — [GPT-4 Technical Report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/)
38. Bo Pang, Erik Nijkamp, Wenjuan Han, Linqi Zhou, Yixian Liu, and Kewei Tu. 2020. Towards holistic and automatic evaluation of open-domain dialogue generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 3619–3629… [find](https://scholar.google.com/scholar?q=Bo+Pang%2C+Erik+Nijkamp%2C+Wenjuan+Han%2C+Linqi+Zhou%2C+Yixian+Liu%2C+and+Kewei+Tu%2E+2020%2E+Towards+holistic+and+automatic+evaluation+of+open%2Ddomain+dialogue+generation%2E+In+Proceedings+of+the+58th+Annual+Meeting+)
39. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, July 6-12, 2002, Philadelphia, PA, USA, pages 311–318. ACL.
40. Chaehun Park, Seungil Lee, Daniel Rim, and Jaegul Choo. 2023. Density: Open-domain dialogue evaluation metric using density estimation. In Findings of the Association for Computational Linguistics: ACL 2023, pages 14222–14236. [find](https://scholar.google.com/scholar?q=Chaehun+Park%2C+Seungil+Lee%2C+Daniel+Rim%2C+and+Jaegul+Choo%2E+2023%2E+Density%3A+Open%2Ddomain+dialogue+evaluation+metric+using+density+estimation%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+AC)
41. Patrícia Pereira, Helena Moniz, and Joao Paulo Carvalho. 2024. Context at wassa 2024 empathy and personality shared task: History-dependent embedding utterance representations for empathy and emotion prediction in conversations. In Proceedings of the 14th Workshop on Computational Approaches to Subj… [find](https://scholar.google.com/scholar?q=Patr%C3%ADcia+Pereira%2C+Helena+Moniz%2C+and+Joao+Paulo+Carvalho%2E+2024%2E+Context+at+wassa+2024+empathy+and+personality+shared+task%3A+History%2Ddependent+embedding+utterance+representations+for+empathy+and+emotion+)
42. Juan Manuel Pérez, Mariela Rajngewerc, Juan Carlos Giudici, Damián A Furman, Franco Luque, Laura Alonso Alemany, and María Vanina Martínez. 2021. pysentimiento: A python toolkit for opinion mining and social nlp tasks. arXiv preprint arXiv:2106.09462. — [arXiv:2106.09462](https://arxiv.org/abs/2106.09462) [find](https://scholar.google.com/scholar?q=Juan+Manuel+P%C3%A9rez%2C+Mariela+Rajngewerc%2C+Juan+Carlos+Giudici%2C+Dami%C3%A1n+A+Furman%2C+Franco+Luque%2C+Laura+Alonso+Alemany%2C+and+Mar%C3%ADa+Vanina+Mart%C3%ADnez%2E+2021%2E+pysentimiento%3A+A+python+toolkit+for+opinion+mining+and)
43. Vitou Phy, Yang Zhao, and Akiko Aizawa. 2020. Deconstruct to reconstruct a configurable evaluation metric for open-domain dialogue systems. In Proceedings of the 28th International Conference on Computational Linguistics, pages 4164–4178. [find](https://scholar.google.com/scholar?q=Vitou+Phy%2C+Yang+Zhao%2C+and+Akiko+Aizawa%2E+2020%2E+Deconstruct+to+reconstruct+a+configurable+evaluation+metric+for+open%2Ddomain+dialogue+systems%2E+In+Proceedings+of+the+28th+International+Conference+on+Compu)
44. Vincent Pisztora and Jia Li. 2024. Learning performance maximizing ensembles with explainability guarantees. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 14617– 14624. [find](https://scholar.google.com/scholar?q=Vincent+Pisztora+and+Jia+Li%2E+2024%2E+Learning+performance+maximizing+ensembles+with+explainability+guarantees%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+volume+38%2C+pages+14617%E2%80%93+1)
45. Soujanya Poria, Devamanyu Hazarika, Navonil Majumder, Gautam Naik, Erik Cambria, and Rada Mihalcea. 2019. [Meld: A multimodal multi-party dataset for emotion recognition in conversations](https://md.research.elamur.ai/free/en/papers/poria-2018-meld-multimodal-multi-party-dataset/). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 527–536.
46. Steven Pulos, Jeff Elison, and Randy Lennon. 2004. The hierarchical structure of the interpersonal reactivity index. Social Behavior & Personality: an international journal, 32(4). [find](https://scholar.google.com/scholar?q=Steven+Pulos%2C+Jeff+Elison%2C+and+Randy+Lennon%2E+2004%2E+The+hierarchical+structure+of+the+interpersonal+reactivity+index%2E+Social+Behavior+%26+Personality%3A+an+international+journal%2C+32%284%29%2E)
47. Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeff+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+and+Ilya+Sutskever%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E)
48. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). Journal of machine learning research, 21(140):1–67.
49. Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2019. Towards empathetic open-domain conversation models: A new benchmark and dataset. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5370–5381. [find](https://scholar.google.com/scholar?q=Hannah+Rashkin%2C+Eric+Michael+Smith%2C+Margaret+Li%2C+and+Y%2DLan+Boureau%2E+2019%2E+Towards+empathetic+open%2Ddomain+conversation+models%3A+A+new+benchmark+and+dataset%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+)
50. Liliang Ren, Mankeerat Sidhu, Qi Zeng, Revanth Gangi Reddy, Heng Ji, and Cheng Xiang Zhai. 2023. Cpmi: Conditional pointwise mutual information for turn-level dialogue evaluation. In 3rd Workshop on Document-grounded Dialogue and Conversational Question Answering, DialDoc 2023, co-located with ACL 2… [find](https://scholar.google.com/scholar?q=Liliang+Ren%2C+Mankeerat+Sidhu%2C+Qi+Zeng%2C+Revanth+Gangi+Reddy%2C+Heng+Ji%2C+and+Cheng+Xiang+Zhai%2E+2023%2E+Cpmi%3A+Conditional+pointwise+mutual+information+for+turn%2Dlevel+dialogue+evaluation%2E+In+3rd+Workshop+on+D)
51. Nino Scherrer, Claudia Shi, Amir Feder, and David Blei. 2024. Evaluating the moral beliefs encoded in llms. Advances in Neural Information Processing Systems, 36. [find](https://scholar.google.com/scholar?q=Nino+Scherrer%2C+Claudia+Shi%2C+Amir+Feder%2C+and+David+Blei%2E+2024%2E+Evaluating+the+moral+beliefs+encoded+in+llms%2E+Advances+in+Neural+Information+Processing+Systems%2C+36%2E)
52. Ashish Sharma, Adam Miner, David Atkins, and Tim Althoff. 2020. [A computational approach to understanding empathy expressed in text-based mental health support](https://md.research.elamur.ai/free/en/papers/sharma-2020-computational-approach-understanding-empathy-expressed/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5263–5276.
53. Vishal Anand Shetty, Shauna Durbin, Meghan S Weyrich, Airín Denise Martínez, Jing Qian, and David L Chin. 2024. A scoping review of empathy recognition in text using natural language processing. Journal of the American Medical Informatics Association, 31(3):762–775. [find](https://scholar.google.com/scholar?q=Vishal+Anand+Shetty%2C+Shauna+Durbin%2C+Meghan+S+Weyrich%2C+Air%C3%ADn+Denise+Mart%C3%ADnez%2C+Jing+Qian%2C+and+David+L+Chin%2E+2024%2E+A+scoping+review+of+empathy+recognition+in+text+using+natural+language+processing%2E+Journ)
54. Xiaohui Song, Longtao Huang, Hui Xue, and Songlin Hu. 2022. Supervised prototypical contrastive learning for emotion recognition in conversation. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 5197– 5206. [find](https://scholar.google.com/scholar?q=Xiaohui+Song%2C+Longtao+Huang%2C+Hui+Xue%2C+and+Songlin+Hu%2E+2022%2E+Supervised+prototypical+contrastive+learning+for+emotion+recognition+in+conversation%2E+In+Proceedings+of+the+2022+Conference+on+Empirical+Met)
55. Yukun Wang, Jin Wang, and Xuejie Zhang. 2023. Ynu-hpcc at wassa-2023 shared task 1: Large-scale language model with lora fine-tuning for empathy detection and emotion classification. In Proceedings of the 13th Workshop on Computational Approaches to Subjectivity, Sentiment, & Social Media Analysis,… [find](https://scholar.google.com/scholar?q=Yukun+Wang%2C+Jin+Wang%2C+and+Xuejie+Zhang%2E+2023%2E+Ynu%2Dhpcc+at+wassa%2D2023+shared+task+1%3A+Large%2Dscale+language+model+with+lora+fine%2Dtuning+for+empathy+detection+and+emotion+classification%2E+In+Proceedings+of)
56. Anuradha Welivita, Yubo Xie, and Pearl Pu. 2021. A large-scale dataset for empathetic response generation. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 1251–1264. [find](https://scholar.google.com/scholar?q=Anuradha+Welivita%2C+Yubo+Xie%2C+and+Pearl+Pu%2E+2021%2E+A+large%2Dscale+dataset+for+empathetic+response+generation%2E+In+Proceedings+of+the+2021+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pa)
57. An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, et al. 2024. Qwen2 technical report. arXiv preprint arXiv:2407.10671. [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chang+Zhou%2C+Chengpeng+Li%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+et+al%2E+2024%2E+Qwen2+technical+report%2E+arXiv+preprint+arXiv%3A2407%2E10671%2E)
58. Jiamin Yang and David Jurgens. 2024. Modeling empathetic alignment in conversation. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 3127–3148. [find](https://scholar.google.com/scholar?q=Jiamin+Yang+and+David+Jurgens%2E+2024%2E+Modeling+empathetic+alignment+in+conversation%2E+In+Proceedings+of+the+2024+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+Linguistics)
59. Ryoma Yoshimura, Masahiro Kaneko, Tomoyuki Kajiwara, and Mamoru Komachi. 2020. Some: Reference-less sub-metrics optimized for manual evaluations of grammatical error correction. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6516–6522. [find](https://scholar.google.com/scholar?q=Ryoma+Yoshimura%2C+Masahiro+Kaneko%2C+Tomoyuki+Kajiwara%2C+and+Mamoru+Komachi%2E+2020%2E+Some%3A+Reference%2Dless+sub%2Dmetrics+optimized+for+manual+evaluations+of+grammatical+error+correction%2E+In+Proceedings+of+the+)
60. Chen Zhang, Luis Fernando D’Haro, Thomas Friedrichs, and Haizhou Li. 2022. Mdd-eval: Self-training on augmented data for multi-domain dialogue evaluation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 11657–11666. [find](https://scholar.google.com/scholar?q=Chen+Zhang%2C+Luis+Fernando+D%E2%80%99Haro%2C+Thomas+Friedrichs%2C+and+Haizhou+Li%2E+2022%2E+Mdd%2Deval%3A+Self%2Dtraining+on+augmented+data+for+multi%2Ddomain+dialogue+evaluation%2E+In+Proceedings+of+the+AAAI+Conference+on+Arti)
61. Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2020. [DIALOGPT : Large-scale generative pre-training for conversational response generation](https://md.research.elamur.ai/free/en/papers/zhang-2019-dialogpt-large-scale-generative-pre/). In Proceedings of the 58th Annual Meeting of the Association for Computational Ling…
62. Liu Zhuang, Lin Wayne, Shi Ya, and Zhao Jun. 2021. [A robustly optimized BERT pre-training approach with post-training](https://md.research.elamur.ai/free/en/papers/zhuang-2021-robustly-optimized-bert-pre-training/). In Proceedings of the 20th Chinese National Conference on Computational Linguistics, pages 1218–1227, Huhhot, China. Chinese Information Processing Society of China.

*Updated 1 November 2026*