---
title: "CAIR — paper by Zhang et al. (2026) — summary and ideas"
description: "The authors conclude that reward design matters more than model scale for genuine… 73.80% accuracy on MTMEUR. 39 references."
authors: ["Fengyu Zhang", "Bin Liu", "Jianhua Tao", "Zhuofan Wen", "Shun Chen", "Hailiang Yao", "Zhengqi Wen"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.1263/"
doi: "10.18653/v1/2026.findings-acl.1263"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2026-cair-causal-adaptive-information-based/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2026-cair-causal-adaptive-information-based/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/25/) / CAIR: Causal Adaptive Information-based Reinforcement Learning for Multimodal Emotion Reasoning

# CAIR: Causal Adaptive Information-based Reinforcement Learning for Multimodal Emotion Reasoning

**Authors:** Fengyu Zhang, [Bin Liu](https://md.research.elamur.ai/free/en/authors/bin-liu/), [Jianhua Tao](https://md.research.elamur.ai/free/en/authors/jianhua-tao/), [Zhuofan Wen](https://md.research.elamur.ai/free/en/authors/zhuofan-wen/), Shun Chen, Hailiang Yao and [Zhengqi Wen](https://md.research.elamur.ai/free/en/authors/zhengqi-wen/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.1263/) · [PDF](https://aclanthology.org/2026.findings-acl.1263.pdf) · [DOI 10.18653/v1/2026.findings-acl.1263](https://doi.org/10.18653/v1/2026.findings-acl.1263) · [OpenAlex](https://openalex.org/W7166819336)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper targets a weakness of multimodal emotion reasoning models that the authors call causal degeneracy. Models trained with SFT, or with RL rewards based only on accuracy or likelihood, often give fluent explanations that do not actually support the predicted emotion. CAIR is an RL framework built on GRPO. It starts from an SFT stage and treats the generated rationale as a causal mediator between the video and text input and the emotion label. A dual-path intrinsic verifier compares two predictions: one from the input alone and one from the input plus the rationale. The resulting Causal Mediation Reward (CMR) scores a rationale by how much it reduces predictive uncertainty. A Cognitive-load Adaptive Optimization (CLO) mechanism, motivated by the information bottleneck, gives more weight to ambiguous, high-entropy samples. CAIR reaches state-of-the-art accuracy on MTMEUR and competitive results on EmoBench-M, and its rationales are more faithful. The authors conclude that reward design matters more than model scale for genuine emotional reasoning.

## Key points
- CAIR reaches 73.80% accuracy on MTMEUR and 68.5% on the SCEA subset of EmoBench-M. It beats specialized SFT baselines by up to 14.4%.
- The rationale's value is measured as the shift between the model's prediction without it and with it, where adding the rationale is treated as an intervention. Shuffled or irrelevant rationales sharply reduce this information gain (ΔH).
- On CA-MER, CLO raises accuracy on samples with contradictory multimodal cues to 68.7%, compared with 58.4% for Uniform RL and 61.2% for Curriculum RL. Stability also improves: IQR falls to 3.8, from 8.2 and 6.5 respectively.
- Compared with SFT, CAIR's prediction confidence clusters above 0.85, and the variance of its predicted probabilities is 42% narrower.
- Limitations include:
  - reliance on ground-truth labels when computing rewards;
  - evaluation mostly on English benchmarks;
  - sensitivity to the base model's calibration.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Tsinghua University](https://md.research.elamur.ai/free/en/universities/tsinghua-university/) — University, Beijing, China · 415 works · 94,028 citations
- [Chinese Academy of Sciences](https://md.research.elamur.ai/free/en/government/chinese-academy-of-sciences/) — Government body, Beijing, China · 246 works · 24,951 citations
- [Institute of Automation](https://md.research.elamur.ai/free/en/labs/institute-of-automation/) — Research institute, Beijing, China · 87 works · 14,883 citations
- [University of Chinese Academy of Sciences](https://md.research.elamur.ai/free/en/universities/university-of-chinese-academy-of-sciences/) — University, Beijing, China · 188 works · 8,743 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,649 works · [💭 Experimental and Cognitive Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/experimental-and-cognitive-psychology/) · 930 works

🧭 Topics (3): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,685 works · [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 1,035 works · [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 811 works

## Cite

Cite as: Fengyu Zhang, Bin Liu, Jianhua Tao, Zhuofan Wen, Shun Chen, Hailiang Yao and Zhengqi Wen. 2026. CAIR: Causal Adaptive Information-based Reinforcement Learning for Multimodal Emotion Reasoning. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhang2026cair,
  title = {{CAIR: Causal Adaptive Information-based Reinforcement Learning for Multimodal Emotion Reasoning}},
  author = {Fengyu Zhang, Bin Liu, Jianhua Tao, Zhuofan Wen, Shun Chen, Hailiang Yao and Zhengqi Wen},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1263},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1263}
}
```

- [← previous CAI@LTEDI 2026: Multilingual Gender Inclusive Language Generation using Instruction-Guided mT5 Transformer Model](https://md.research.elamur.ai/free/en/papers/nair-2026-cai-ltedi-2026-multilingual-gender/)
- [next → CAIT: A Syntactic Parsing Toolkit for Child–Adult InTeractions](https://md.research.elamur.ai/free/en/papers/padovani-2026-cait-syntactic-parsing-toolkit-child/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/25/)

## References

39 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. [Qwen-vl: A frontier large vision-language model with versatile abilities](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-vl-versatile-vision-language/). arXiv preprint arXiv:2308.12966, 1(2):3.
2. Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Samuel Kim, Jeannette N Chang, Sungbok Lee, and Shrikanth S Narayanan. 2008. Iemocap: Interactive emotional dyadic motion capture database. Language resources and evaluation, 42(4):335–359. [find](https://scholar.google.com/scholar?q=Carlos+Busso%2C+Murtaza+Bulut%2C+Chi%2DChun+Lee%2C+Abe+Kazemzadeh%2C+Emily+Mower%2C+Samuel+Kim%2C+Jeannette+N+Chang%2C+Sungbok+Lee%2C+and+Shrikanth+S+Narayanan%2E+2008%2E+Iemocap%3A+Interactive+emotional+dyadic+motion+captur)
3. Linda Camras. 1980. Emotion: a psychoevolutionary synthesis. [find](https://scholar.google.com/scholar?q=Linda+Camras%2E+1980%2E+Emotion%3A+a+psychoevolutionary+synthesis%2E)
4. Lin Chen, Xilin Wei, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Zhenyu Tang, Li Yuan, and 1 others. 2024. Sharegpt4video: Improving video understanding and generation with better captions. Advances in Neural Information Processing Systems, 37:19472–19495. [find](https://scholar.google.com/scholar?q=Lin+Chen%2C+Xilin+Wei%2C+Jinsong+Li%2C+Xiaoyi+Dong%2C+Pan+Zhang%2C+Yuhang+Zang%2C+Zehui+Chen%2C+Haodong+Duan%2C+Zhenyu+Tang%2C+Li+Yuan%2C+and+1+others%2E+2024%2E+Sharegpt4video%3A+Improving+video+understanding+and+generation+w)
5. Zebang Cheng, Zhi-Qi Cheng, Jun-Yan He, Kai Wang, Yuxiang Lin, Zheng Lian, Xiaojiang Peng, and Alexander Hauptmann. 2024a. Emotion-llama: Multimodal emotion recognition and reasoning with instruction tuning. Advances in Neural Information Processing Systems, 37:110805–110853. [find](https://scholar.google.com/scholar?q=Zebang+Cheng%2C+Zhi%2DQi+Cheng%2C+Jun%2DYan+He%2C+Kai+Wang%2C+Yuxiang+Lin%2C+Zheng+Lian%2C+Xiaojiang+Peng%2C+and+Alexander+Hauptmann%2E+2024a%2E+Emotion%2Dllama%3A+Multimodal+emotion+recognition+and+reasoning+with+instruction+)
6. Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, and 1 others. 2024b. [Videollama 2: Advancing spatial-temporal modeling and audio understanding in video-llms](https://md.research.elamur.ai/free/en/papers/cheng-2024-videollama-2-advancing-spatial-temporal/). arXiv preprint arXiv:2406.07476.
7. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, and 1 others. 2021. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168.
8. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, and 1 others. 2025. [Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities](https://md.research.elamur.ai/free/en/papers/comanici-2025-gemini-2-5-pushing-frontier/). arX…
9. Yumeng Fu, Junjie Wu, Zhongjie Wang, Meishan Zhang, Yulin Wu, and Bingquan Liu. 2025. Bemerc: Behavior-aware mllm-based framework for multimodal emotion recognition in conversation. arXiv preprint arXiv:2503.23990. — [arXiv:2503.23990](https://arxiv.org/abs/2503.23990) [find](https://scholar.google.com/scholar?q=Yumeng+Fu%2C+Junjie+Wu%2C+Zhongjie+Wang%2C+Meishan+Zhang%2C+Yulin+Wu%2C+and+Bingquan+Liu%2E+2025%2E+Bemerc%3A+Behavior%2Daware+mllm%2Dbased+framework+for+multimodal+emotion+recognition+in+conversation%2E+arXiv+preprint+arX)
10. Zhiyuan Han, Beier Zhu, Yanlong Xu, Peipei Song, and Xun Yang. 2025. Benchmarking and bridging emotion conflicts for multimodal emotion reasoning. In Proceedings of the 33rd ACM International Conference on Multimedia, pages 5528–5537. [find](https://scholar.google.com/scholar?q=Zhiyuan+Han%2C+Beier+Zhu%2C+Yanlong+Xu%2C+Peipei+Song%2C+and+Xun+Yang%2E+2025%2E+Benchmarking+and+bridging+emotion+conflicts+for+multimodal+emotion+reasoning%2E+In+Proceedings+of+the+33rd+ACM+International+Conferen)
11. He Hu, Yucheng Zhou, Lianzhong You, Hongbo Xu, Qianning Wang, Zheng Lian, Fei Richard Yu, Fei Ma, and Laizhong Cui. 2025a. Emobench-m: Benchmarking emotional intelligence for multimodal large language models. arXiv preprint arXiv:2502.04424. — [arXiv:2502.04424](https://arxiv.org/abs/2502.04424) [find](https://scholar.google.com/scholar?q=He+Hu%2C+Yucheng+Zhou%2C+Lianzhong+You%2C+Hongbo+Xu%2C+Qianning+Wang%2C+Zheng+Lian%2C+Fei+Richard+Yu%2C+Fei+Ma%2C+and+Laizhong+Cui%2E+2025a%2E+Emobench%2Dm%3A+Benchmarking+emotional+intelligence+for+multimodal+large+language)
12. Jinpeng Hu, Hongchang Shi, Chongyuan Dai, Zhuo Li, Peipei Song, and Meng Wang. 2025b. Beyond emotion recognition: A multi-turn multimodal emotion understanding and reasoning benchmark. In Proceedings of the 33rd ACM International Conference on Multimedia, pages 5814–5823. [find](https://scholar.google.com/scholar?q=Jinpeng+Hu%2C+Hongchang+Shi%2C+Chongyuan+Dai%2C+Zhuo+Li%2C+Peipei+Song%2C+and+Meng+Wang%2E+2025b%2E+Beyond+emotion+recognition%3A+A+multi%2Dturn+multimodal+emotion+understanding+and+reasoning+benchmark%2E+In+Proceedings+)
13. Shengding Hu, Yuge Tu, Xu Han, Chaoqun He, Ganqu Cui, Xiang Long, Zhi Zheng, Yewei Fang, Yuxiang Huang, Weilin Zhao, and 1 others. 2024. [Minicpm: Unveiling the potential of small language models with scalable training strategies](https://md.research.elamur.ai/free/en/papers/hu-2024-minicpm-unveiling-potential-small-language/). arXiv preprint arXiv:2404.06395.
14. Bin Huang, Xin Wang, Hong Chen, Zihan Song, and Wenwu Zhu. 2024a. Vtimellm: Empower llm to grasp video moments. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14271–14280. [find](https://scholar.google.com/scholar?q=Bin+Huang%2C+Xin+Wang%2C+Hong+Chen%2C+Zihan+Song%2C+and+Wenwu+Zhu%2E+2024a%2E+Vtimellm%3A+Empower+llm+to+grasp+video+moments%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Computer+Vision+and+Pattern+Recognition%2C+pa)
15. Zhaopei Huang, Jinming Zhao, and Qin Jin. 2024b. Ecr-chain: Advancing generative language models to better emotion-cause reasoners through reasoning chains. arXiv preprint arXiv:2405.10860. — [arXiv:2405.10860](https://arxiv.org/abs/2405.10860) [find](https://scholar.google.com/scholar?q=Zhaopei+Huang%2C+Jinming+Zhao%2C+and+Qin+Jin%2E+2024b%2E+Ecr%2Dchain%3A+Advancing+generative+language+models+to+better+emotion%2Dcause+reasoners+through+reasoning+chains%2E+arXiv+preprint+arXiv%3A2405%2E10860%2E+%E2%80%94+%5BarXiv%3A2)
16. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. [Gpt-4o system card](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/). arXiv preprint arXiv:2410.21276.
17. Peng Jin, Ryuichi Takanobu, Wancai Zhang, Xiaochun Cao, and Li Yuan. 2024. Chat-univi: Unified visual representation empowers large language models with image and video understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13700–13710. [find](https://scholar.google.com/scholar?q=Peng+Jin%2C+Ryuichi+Takanobu%2C+Wancai+Zhang%2C+Xiaochun+Cao%2C+and+Li+Yuan%2E+2024%2E+Chat%2Dunivi%3A+Unified+visual+representation+empowers+large+language+models+with+image+and+video+understanding%2E+In+Proceedings+o)
18. Garapati Keerthana and Manik Gupta. 2025. Towards emotionally intelligent and responsible reinforcement learning. arXiv preprint arXiv:2511.10573. — [arXiv:2511.10573](https://arxiv.org/abs/2511.10573) [find](https://scholar.google.com/scholar?q=Garapati+Keerthana+and+Manik+Gupta%2E+2025%2E+Towards+emotionally+intelligent+and+responsible+reinforcement+learning%2E+arXiv+preprint+arXiv%3A2511%2E10573%2E+%E2%80%94+%5BarXiv%3A2511%2E10573%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2511%2E10573)
19. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). arXiv preprint arXiv:2402.03300.
20. Yuanyuan Sun and Ting Zhou. 2025. Dialoguemllm: transforming multimodal emotion recognition in conversation through instruction-tuned mllm. IEEE Access. [find](https://scholar.google.com/scholar?q=Yuanyuan+Sun+and+Ting+Zhou%2E+2025%2E+Dialoguemllm%3A+transforming+multimodal+emotion+recognition+in+conversation+through+instruction%2Dtuned+mllm%2E+IEEE+Access%2E)
21. Yao-Hung Hubert Tsai, Shaojie Bai, Paul Pu Liang, J Zico Kolter, Louis-Philippe Morency, and Ruslan Salakhutdinov. 2019. [Multimodal transformer for unaligned multimodal language sequences](https://md.research.elamur.ai/free/en/papers/tsai-2019-multimodal-transformer-unaligned-multimodal-language/). In Proceedings of the conference. Association for computational linguistics. Meeting, volume 2019, page 6558.
22. Cong Wang, Changfeng Gao, Yang Xiang, Zhihao Du, Keyu An, Han Zhao, Qian Chen, Xiangang Li, Yingming Gao, and Ya Li. 2025a. Rrpo: Robust reward policy optimization for llm-based emotional tts. arXiv preprint arXiv:2512.04552. — [arXiv:2512.04552](https://arxiv.org/abs/2512.04552) [find](https://scholar.google.com/scholar?q=Cong+Wang%2C+Changfeng+Gao%2C+Yang+Xiang%2C+Zhihao+Du%2C+Keyu+An%2C+Han+Zhao%2C+Qian+Chen%2C+Xiangang+Li%2C+Yingming+Gao%2C+and+Ya+Li%2E+2025a%2E+Rrpo%3A+Robust+reward+policy+optimization+for+llm%2Dbased+emotional+tts%2E+arXiv+p)
23. Peisong Wang, Ruotian Ma, Bang Zhang, Xingyu Chen, Zhiwei He, Kang Luo, Qingsong Lv, Qingxuan Jiang, Zheng Xie, Shanyi Wang, and 1 others. 2025b. Rlver: Reinforcement learning with verifiable emotion rewards for empathetic agents. arXiv preprint arXiv:2507.03112. — [arXiv:2507.03112](https://arxiv.org/abs/2507.03112) [find](https://scholar.google.com/scholar?q=Peisong+Wang%2C+Ruotian+Ma%2C+Bang+Zhang%2C+Xingyu+Chen%2C+Zhiwei+He%2C+Kang+Luo%2C+Qingsong+Lv%2C+Qingxuan+Jiang%2C+Zheng+Xie%2C+Shanyi+Wang%2C+and+1+others%2E+2025b%2E+Rlver%3A+Reinforcement+learning+with+verifiable+emotion+)
24. Pengcheng Li, Botao Zhao, Zuheng Kang, Junqing Peng, Xiaoyang Qu, Yayun He, and Jianzong Wang. 2025. Emo-rl: Emotion-rule-based reinforcement learning enhanced audio-language model for generalized speech emotion recognition. In Findings of the Association for Computational Linguistics: EMNLP 2025, p… [find](https://scholar.google.com/scholar?q=Pengcheng+Li%2C+Botao+Zhao%2C+Zuheng+Kang%2C+Junqing+Peng%2C+Xiaoyang+Qu%2C+Yayun+He%2C+and+Jianzong+Wang%2E+2025%2E+Emo%2Drl%3A+Emotion%2Drule%2Dbased+reinforcement+learning+enhanced+audio%2Dlanguage+model+for+generalized+spe)
25. Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, and 1 others. 2024. [Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution](https://md.research.elamur.ai/free/en/papers/wang-2024-qwen2-vl-enhancing-vision/). arXiv preprint arXiv:2409.12191.
26. Zheng Lian, Haiyang Sun, Licai Sun, Jiangyan Yi, Bin Liu, and Jianhua Tao. 2024. Affectgpt: Dataset and framework for explainable multimodal emotion recognition. arXiv preprint arXiv:2407.07653. — [arXiv:2407.07653](https://arxiv.org/abs/2407.07653) [find](https://scholar.google.com/scholar?q=Zheng+Lian%2C+Haiyang+Sun%2C+Licai+Sun%2C+Jiangyan+Yi%2C+Bin+Liu%2C+and+Jianhua+Tao%2E+2024%2E+Affectgpt%3A+Dataset+and+framework+for+explainable+multimodal+emotion+recognition%2E+arXiv+preprint+arXiv%3A2407%2E07653%2E+%E2%80%94+%5Bar)
27. Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang, and 1 others. 2025. Qwen2. 5-omni technical report. arXiv preprint arXiv:2503.20215. — [Qwen2.5-Omni Technical Report](https://md.research.elamur.ai/free/en/papers/xu-2025-qwen2-5-omni-technical/)
28. Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. 2024. [Video-llava: Learning united visual representation by alignment before projection](https://md.research.elamur.ai/free/en/papers/lin-2023-video-llava-learning-united-visual/). In Proceedings of the 2024 conference on empirical methods in natural language processing, pages 5971–5984.
29. Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Khan. 2024. [Video-chatgpt: Towards detailed video understanding via large vision and language models](https://md.research.elamur.ai/free/en/papers/maaz-2023-video-chatgpt-towards-detailed-video/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12585– 12602.
30. Soujanya Poria, Devamanyu Hazarika, Navonil Majumder, Gautam Naik, Erik Cambria, and Rada Mihalcea. 2019. [Meld: A multimodal multi-party dataset for emotion recognition in conversations](https://md.research.elamur.ai/free/en/papers/poria-2018-meld-multimodal-multi-party-dataset/). In Proceedings of the 57th annual meeting of the association for computational linguistics, pages 527–536.
31. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, and 1 others. 2024. [find](https://scholar.google.com/scholar?q=Zhihong+Shao%2C+Peiyi+Wang%2C+Qihao+Zhu%2C+Runxin+Xu%2C+Junxiao+Song%2C+Xiao+Bi%2C+Haowei+Zhang%2C+Mingchuan+Zhang%2C+YK+Li%2C+Yang+Wu%2C+and+1+others%2E+2024%2E)
32. Qize Yang, Detao Bai, Yi-Xing Peng, and Xihan Wei. 2025a. Omni-emotion: Extending video mllm with detailed face and audio modeling for multimodal emotion analysis. arXiv preprint arXiv:2501.09502. — [arXiv:2501.09502](https://arxiv.org/abs/2501.09502) [find](https://scholar.google.com/scholar?q=Qize+Yang%2C+Detao+Bai%2C+Yi%2DXing+Peng%2C+and+Xihan+Wei%2E+2025a%2E+Omni%2Demotion%3A+Extending+video+mllm+with+detailed+face+and+audio+modeling+for+multimodal+emotion+analysis%2E+arXiv+preprint+arXiv%3A2501%2E09502%2E+%E2%80%94+%5B)
33. Qu Yang, Mang Ye, and Bo Du. 2024. Emollm: Multimodal emotional understanding meets large language models. arXiv preprint arXiv:2406.16442. — [arXiv:2406.16442](https://arxiv.org/abs/2406.16442) [find](https://scholar.google.com/scholar?q=Qu+Yang%2C+Mang+Ye%2C+and+Bo+Du%2E+2024%2E+Emollm%3A+Multimodal+emotional+understanding+meets+large+language+models%2E+arXiv+preprint+arXiv%3A2406%2E16442%2E+%E2%80%94+%5BarXiv%3A2406%2E16442%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2406%2E16442%29)
34. Ting Yang, Li Chen, and Huimin Wang. 2025b. Towards open-ended emotional support conversations in llms via reinforcement learning with future-oriented rewards. arXiv preprint arXiv:2508.12935. — [arXiv:2508.12935](https://arxiv.org/abs/2508.12935) [find](https://scholar.google.com/scholar?q=Ting+Yang%2C+Li+Chen%2C+and+Huimin+Wang%2E+2025b%2E+Towards+open%2Dended+emotional+support+conversations+in+llms+via+reinforcement+learning+with+future%2Doriented+rewards%2E+arXiv+preprint+arXiv%3A2508%2E12935%2E+%E2%80%94+%5BarXi)
35. Amir Zadeh, Michael Chan, Paul Pu Liang, Edmund Tong, and Louis-Philippe Morency. 2019. Social-iq: A question answering benchmark for artificial social intelligence. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8807–8817. [find](https://scholar.google.com/scholar?q=Amir+Zadeh%2C+Michael+Chan%2C+Paul+Pu+Liang%2C+Edmund+Tong%2C+and+Louis%2DPhilippe+Morency%2E+2019%2E+Social%2Diq%3A+A+question+answering+benchmark+for+artificial+social+intelligence%2E+In+Proceedings+of+the+IEEE%2FCVF+Con)
36. Amir Zadeh, Paul Pu Liang, Soujanya Poria, Prateek Vij, Erik Cambria, and Louis-Philippe Morency. 2018. [Multimodal language analysis in the wild: Cmu-mosei dataset and interpretable dynamic fusion graph](https://md.research.elamur.ai/free/en/papers/zadeh-2018-multimodal-language-analysis-wild-cmu/). In ACL, pages 2236–2246.
37. Hang Zhang, Xin Li, and Lidong Bing. 2023. [Videollama: An instruction-tuned audio-visual language model for video understanding](https://md.research.elamur.ai/free/en/papers/zhang-2023-video-llama-instruction-tuned-audio/). arXiv preprint arXiv:2306.02858.
38. Ren Zhang, Guoliang Xu, Tianyu Zhou, Junjun Zhao, Bo Yu, Zhicheng Zhang, and Jianqin Yin. 2025. Emo-synergy: Synergizing task-specific and general experts for multimodal emotion recognition. In Proceedings of the 3rd International Workshop on Multimodal and Responsible Affective Computing, pages 30–… [find](https://scholar.google.com/scholar?q=Ren+Zhang%2C+Guoliang+Xu%2C+Tianyu+Zhou%2C+Junjun+Zhao%2C+Bo+Yu%2C+Zhicheng+Zhang%2C+and+Jianqin+Yin%2E+2025%2E+Emo%2Dsynergy%3A+Synergizing+task%2Dspecific+and+general+experts+for+multimodal+emotion+recognition%2E+In+Procee)
39. Jiaxing Zhao, Xihan Wei, and Liefeng Bo. 2025. R1-omni: Explainable omni-multimodal emotion recognition with reinforcement learning. arXiv preprint arXiv:2503.05379. — [arXiv:2503.05379](https://arxiv.org/abs/2503.05379) [find](https://scholar.google.com/scholar?q=Jiaxing+Zhao%2C+Xihan+Wei%2C+and+Liefeng+Bo%2E+2025%2E+R1%2Domni%3A+Explainable+omni%2Dmultimodal+emotion+recognition+with+reinforcement+learning%2E+arXiv+preprint+arXiv%3A2503%2E05379%2E+%E2%80%94+%5BarXiv%3A2503%2E05379%5D%28https%3A%2F%2Farxiv)

*Updated 2 November 2026*