---
title: "Emotion-Wheel-Guided Audio-Referred Text Representation for"
description: "The proposed framework, EMART, uses only text and audio. 70.9 F1 on IEMOCAP. 24 references."
authors: ["Eunseon Seong", "Harim Lee", "Dahye Kim", "Changhyun Kim", "Dong-Kyu Chae"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.1875/"
doi: "10.18653/v1/2026.acl-long.1875"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/seong-2026-emotion-wheel-guided-audio-referred/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/seong-2026-emotion-wheel-guided-audio-referred/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/28/) / Emotion-Wheel-Guided Audio-Referred Text Representation for Multimodal Emotion Recognition in Conversation

# Emotion-Wheel-Guided Audio-Referred Text Representation for Multimodal Emotion Recognition in Conversation

**Authors:** Eunseon Seong, Harim Lee, Dahye Kim, Changhyun Kim and Dong-Kyu Chae · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.1875/) · [PDF](https://aclanthology.org/2026.acl-long.1875.pdf) · [DOI 10.18653/v1/2026.acl-long.1875](https://doi.org/10.18653/v1/2026.acl-long.1875) · [OpenAlex](https://openalex.org/W7166846269)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper addresses multimodal Emotion Recognition in Conversation (ERC) and targets two weaknesses of earlier systems. First, they fuse modalities as if each carried the same kind and amount of information. Second, they treat emotion labels as unrelated categories, so confusing *happy* with *excited* costs as much as confusing *happy* with *sad*. The proposed framework, EMART, uses only text and audio. It splits RoBERTa-base in half. The lower six layers encode the dialogue text, with speaker and target-utterance markers. The upper six layers add cross-modal attention, so the text stream queries audio features from WavLM or Wav2vec. An alignment loss reduces the gap between the two modalities before fusion. In addition, emotions are placed at angles on Russell's circumplex model, and a supervised contrastive loss pushes classes apart in proportion to their angular distance. On IEMOCAP and MELD, this text-centric fusion beats concatenation, dot-product and plain attention fusion, and both losses add gains. The results suggest that modality-specific roles and psychologically grounded label geometry improve ERC.

## Key points
- With RoBERTa, EMART reaches about 70.8 Acc / 70.9 F1 on IEMOCAP and 66.7 Acc / 64.7 F1 on MELD. The best alternative fusion scored about 66.4 on IEMOCAP and 63.9 on MELD.
- Unimodal baselines show why text is treated as primary. Audio alone gets 53.85 Acc on IEMOCAP and 48.97 on MELD, while RoBERTa text alone gets about 64.7 on both.
- In the loss ablation, IEMOCAP Acc rises from 64.70 with no extra losses to 68.02 with the alignment loss only and 68.21 with the emotion-wheel loss only. Using both gives 70.79.
- Starting fusion at layer 6 works best. Fusing earlier (layers 0 or 3) disrupts text representations, and fusing at layer 9 leaves too few layers for interaction.
- The emotion-wheel loss outperforms SupCon and SCCL. Limitations noted by the authors are that visual input is not used and that results depend on the choice of audio encoder. Code: github.com/DILAB-HYU/EMART.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Hanyang University](https://md.research.elamur.ai/free/en/universities/hanyang-university/) — University, Seoul, South Korea · 6 works · 150 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,552 works · [💭 Experimental and Cognitive Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/experimental-and-cognitive-psychology/) · 870 works

🧭 Topics (3): [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 972 works · [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 657 works · [🎙️ Speech & spoken language](https://md.research.elamur.ai/free/en/topics/speech/) · 480 works

## Cite

Cite as: Eunseon Seong, Harim Lee, Dahye Kim, Changhyun Kim and Dong-Kyu Chae. 2026. Emotion-Wheel-Guided Audio-Referred Text Representation for Multimodal Emotion Recognition in Conversation. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{seong2026emotion,
  title = {{Emotion-Wheel-Guided Audio-Referred Text Representation for Multimodal Emotion Recognition in Conversation}},
  author = {Eunseon Seong, Harim Lee, Dahye Kim, Changhyun Kim and Dong-Kyu Chae},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1875},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1875}
}
```

- [← previous Emotion Transcription in Conversation: A Benchmark for Capturing Subtle and Complex Emotional States through Natural Language](https://md.research.elamur.ai/free/en/papers/tanaka-2026-emotion-transcription-conversation-benchmark-capturing/)
- [next → Emotions In Oral History Interviews: A Multimodal Approach to Holocaust Testimonies](https://md.research.elamur.ai/free/en/papers/mantaj-2026-emotions-oral-history-interviews-multimodal/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/28/)

## References

24 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Samuel Kim, Jeannette N Chang, Sungbok Lee, and Shrikanth S Narayanan. 2008. Iemocap: Interactive emotional dyadic motion capture database. Language resources and evaluation, 42(4):335–359. [find](https://scholar.google.com/scholar?q=Carlos+Busso%2C+Murtaza+Bulut%2C+Chi%2DChun+Lee%2C+Abe+Kazemzadeh%2C+Emily+Mower%2C+Samuel+Kim%2C+Jeannette+N+Chang%2C+Sungbok+Lee%2C+and+Shrikanth+S+Narayanan%2E+2008%2E+Iemocap%3A+Interactive+emotional+dyadic+motion+captur)
2. Feiyu Chen, Jie Shao, Shuyuan Zhu, and Heng Tao Shen. 2023. Multivariate, multi-frequency and multimodal: Rethinking graph neural networks for emotion recognition in conversation. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10761–10770. [find](https://scholar.google.com/scholar?q=Feiyu+Chen%2C+Jie+Shao%2C+Shuyuan+Zhu%2C+and+Heng+Tao+Shen%2E+2023%2E+Multivariate%2C+multi%2Dfrequency+and+multimodal%3A+Rethinking+graph+neural+networks+for+emotion+recognition+in+conversation%2E+In+2023+IEEE%2FCVF+Con)
3. Michal Ephratt. 2011. Linguistic, paralinguistic and extralinguistic speech and silence. Journal of pragmatics, 43(9):2286–2307. [find](https://scholar.google.com/scholar?q=Michal+Ephratt%2E+2011%2E+Linguistic%2C+paralinguistic+and+extralinguistic+speech+and+silence%2E+Journal+of+pragmatics%2C+43%289%29%3A2286%E2%80%932307%2E)
4. Lisa Feldman Barrett and James A Russell. 1998. Independence and bipolarity in the structure of current affect. Journal of personality and social psychology, 74(4):967. [find](https://scholar.google.com/scholar?q=Lisa+Feldman+Barrett+and+James+A+Russell%2E+1998%2E+Independence+and+bipolarity+in+the+structure+of+current+affect%2E+Journal+of+personality+and+social+psychology%2C+74%284%29%3A967%2E)
5. Deepanway Ghosal, Navonil Majumder, Soujanya Poria, Niyati Chhaya, and Alexander Gelbukh. 2019. [DialogueGCN: A graph convolutional neural network for emotion recognition in conversation](https://md.research.elamur.ai/free/en/papers/ghosal-2019-dialoguegcn-graph-convolutional-neural-network/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th Internation…
6. Dou Hu, Xiaolong Hou, Lingwei Wei, Lianxin Jiang, and Yang Mo. 2022. Mm-dfn: Multimodal dynamic fusion network for emotion recognition in conversations. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 7037–7041. [find](https://scholar.google.com/scholar?q=Dou+Hu%2C+Xiaolong+Hou%2C+Lingwei+Wei%2C+Lianxin+Jiang%2C+and+Yang+Mo%2E+2022%2E+Mm%2Ddfn%3A+Multimodal+dynamic+fusion+network+for+emotion+recognition+in+conversations%2E+In+ICASSP+2022%2D2022+IEEE+International+Conferen)
7. Jingwen Hu, Yuchen Liu, Jinming Zhao, and Qin Jin. 2021. MMGCN: Multimodal fusion via deep graph convolution network for emotion recognition in conversation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natu… [find](https://scholar.google.com/scholar?q=Jingwen+Hu%2C+Yuchen+Liu%2C+Jinming+Zhao%2C+and+Qin+Jin%2E+2021%2E+MMGCN%3A+Multimodal+fusion+via+deep+graph+convolution+network+for+emotion+recognition+in+conversation%2E+In+Proceedings+of+the+59th+Annual+Meeting+)
8. Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2020. Supervised contrastive learning. In Advances in neural information processing systems, pages 18661–18673. [find](https://scholar.google.com/scholar?q=Prannay+Khosla%2C+Piotr+Teterwak%2C+Chen+Wang%2C+Aaron+Sarna%2C+Yonglong+Tian%2C+Phillip+Isola%2C+Aaron+Maschinot%2C+Ce+Liu%2C+and+Dilip+Krishnan%2E+2020%2E+Supervised+contrastive+learning%2E+In+Advances+in+neural+informat)
9. Bryan Li, Dimitrios Dimitriadis, and Andreas Stolcke. 2019. Acoustic and lexical sentiment analysis for customer service calls. In ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 5876–5880. [find](https://scholar.google.com/scholar?q=Bryan+Li%2C+Dimitrios+Dimitriadis%2C+and+Andreas+Stolcke%2E+2019%2E+Acoustic+and+lexical+sentiment+analysis+for+customer+service+calls%2E+In+ICASSP+2019+%2D+2019+IEEE+International+Conference+on+Acoustics%2C+Speech)
10. Hang Li, Wenbiao Ding, Yu Kang, Tianqiao Liu, Zhongqin Wu, and Zitao Liu. 2021a. Ctal: Pre-training cross-modal transformer for audio-and-language representations. In Proceedings of the 2021 conference on empirical methods in natural language processing, pages 3966–3977. [find](https://scholar.google.com/scholar?q=Hang+Li%2C+Wenbiao+Ding%2C+Yu+Kang%2C+Tianqiao+Liu%2C+Zhongqin+Wu%2C+and+Zitao+Liu%2E+2021a%2E+Ctal%3A+Pre%2Dtraining+cross%2Dmodal+transformer+for+audio%2Dand%2Dlanguage+representations%2E+In+Proceedings+of+the+2021+conferenc)
11. Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi. 2021b. Align before fuse: Vision and language representation learning with momentum distillation. In Advances in neural information processing systems, pages 9694–9705. [find](https://scholar.google.com/scholar?q=Junnan+Li%2C+Ramprasaath+Selvaraju%2C+Akhilesh+Gotmare%2C+Shafiq+Joty%2C+Caiming+Xiong%2C+and+Steven+Chu+Hong+Hoi%2E+2021b%2E+Align+before+fuse%3A+Vision+and+language+representation+learning+with+momentum+distillatio)
12. Siyang Liu, Chujie Zheng, Orianna Demasi, Sahand Sabour, Yu Li, Zhou Yu, Yong Jiang, and Minlie Huang. 2021. [Towards emotional support dialog systems](https://md.research.elamur.ai/free/en/papers/liu-2021-emotional-support-dialog-systems/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Lan…
13. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. [Roberta: A robustly optimized bert pretraining approach](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/). arXiv preprint arXiv:1907.11692.
14. Navonil Majumder, Soujanya Poria, Devamanyu Hazarika, Rada Mihalcea, Alexander Gelbukh, and Erik Cambria. 2019. Dialoguernn: an attentive rnn for emotion detection in conversations. In Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Navonil+Majumder%2C+Soujanya+Poria%2C+Devamanyu+Hazarika%2C+Rada+Mihalcea%2C+Alexander+Gelbukh%2C+and+Erik+Cambria%2E+2019%2E+Dialoguernn%3A+an+attentive+rnn+for+emotion+detection+in+conversations%2E+In+Proceedings+of+)
15. Soujanya Poria, Devamanyu Hazarika, Navonil Majumder, Gautam Naik, Erik Cambria, and Rada Mihalcea. 2019a. [Meld: A multimodal multi-party dataset for emotion recognition in conversations](https://md.research.elamur.ai/free/en/papers/poria-2018-meld-multimodal-multi-party-dataset/). In Proceedings of the 57th annual meeting of the association for computational linguistics, pages 527–536.
16. Soujanya Poria, Navonil Majumder, Rada Mihalcea, and Eduard Hovy. 2019b. Emotion recognition in conversation: Research challenges, datasets, and recent advances. IEEE access, 7:100943–100953. [find](https://scholar.google.com/scholar?q=Soujanya+Poria%2C+Navonil+Majumder%2C+Rada+Mihalcea%2C+and+Eduard+Hovy%2E+2019b%2E+Emotion+recognition+in+conversation%3A+Research+challenges%2C+datasets%2C+and+recent+advances%2E+IEEE+access%2C+7%3A100943%E2%80%93100953%2E)
17. Jonathan Posner, James A Russell, and Bradley S Peterson. 2005. The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology. Development and psychopathology, 17(3):715–734. [find](https://scholar.google.com/scholar?q=Jonathan+Posner%2C+James+A+Russell%2C+and+Bradley+S+Peterson%2E+2005%2E+The+circumplex+model+of+affect%3A+An+integrative+approach+to+affective+neuroscience%2C+cognitive+development%2C+and+psychopathology%2E+Developme)
18. James A Russell. 1980. A circumplex model of affect. Journal of personality and social psychology, 39(6):1161. [find](https://scholar.google.com/scholar?q=James+A+Russell%2E+1980%2E+A+circumplex+model+of+affect%2E+Journal+of+personality+and+social+psychology%2C+39%286%29%3A1161%2E)
19. Björn Schuller, Stefan Steidl, Anton Batliner, Felix Burkhardt, Laurence Devillers, Christian MüLler, and Shrikanth Narayanan. 2013. Paralinguistics in speech and language—state-of-the-art and the challenge. Computer Speech & Language, 27(1):4–39. [find](https://scholar.google.com/scholar?q=Bj%C3%B6rn+Schuller%2C+Stefan+Steidl%2C+Anton+Batliner%2C+Felix+Burkhardt%2C+Laurence+Devillers%2C+Christian+M%C3%BCLler%2C+and+Shrikanth+Narayanan%2E+2013%2E+Paralinguistics+in+speech+and+language%E2%80%94state%2Dof%2Dthe%2Dart+and+the+cha)
20. Quan Tu, Yanran Li, Jianwei Cui, Bin Wang, Ji-Rong Wen, and Rui Yan. 2022. [MISC: A mixed strategy-aware model integrating COMET for emotional support conversation](https://md.research.elamur.ai/free/en/papers/tu-2022-misc-mixed-strategy-aware-model/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 308–319.
21. Kailai Yang, Tianlin Zhang, Hassan Alhuzali, and Sophia Ananiadou. 2023. Cluster-level contrastive learning for emotion recognition in conversations. IEEE Transactions on Affective Computing, 14(4):3269–3280. [find](https://scholar.google.com/scholar?q=Kailai+Yang%2C+Tianlin+Zhang%2C+Hassan+Alhuzali%2C+and+Sophia+Ananiadou%2E+2023%2E+Cluster%2Dlevel+contrastive+learning+for+emotion+recognition+in+conversations%2E+IEEE+Transactions+on+Affective+Computing%2C+14%284%29%3A32)
22. Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny. 2021. Barlow twins: Self-supervised learning via redundancy reduction. In International conference on machine learning, pages 12310– 12320. [find](https://scholar.google.com/scholar?q=Jure+Zbontar%2C+Li+Jing%2C+Ishan+Misra%2C+Yann+LeCun%2C+and+St%C3%A9phane+Deny%2E+2021%2E+Barlow+twins%3A+Self%2Dsupervised+learning+via+redundancy+reduction%2E+In+International+conference+on+machine+learning%2C+pages+12310%E2%80%93+)
23. Tao Zhang and Zhenhua Tan. 2025. ECERC: Evidence-cause attention network for multi-modal emotion recognition in conversation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2064–2077. [find](https://scholar.google.com/scholar?q=Tao+Zhang+and+Zhenhua+Tan%2E+2025%2E+ECERC%3A+Evidence%2Dcause+attention+network+for+multi%2Dmodal+emotion+recognition+in+conversation%2E+In+Proceedings+of+the+63rd+Annual+Meeting+of+the+Association+for+Computati)
24. Botao Zhao, Xiaoyang Qu, Zuheng Kang, Junqing Peng, Jing Xiao, and Jianzong Wang. 2025. Accon: Angle-compensated contrastive regularizer for deep regression. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 22750–22758. [find](https://scholar.google.com/scholar?q=Botao+Zhao%2C+Xiaoyang+Qu%2C+Zuheng+Kang%2C+Junqing+Peng%2C+Jing+Xiao%2C+and+Jianzong+Wang%2E+2025%2E+Accon%3A+Angle%2Dcompensated+contrastive+regularizer+for+deep+regression%2E+In+Proceedings+of+the+AAAI+Conference+on+A)

*Updated 1 November 2026*