---
title: "Evaluating the Expressive Appropriateness of Speech in Rich"
description: "The authors define a task they call context-rich expressive appropriateness evaluation for Mandarin and build… 70.8% accuracy. 43 references."
authors: ["Tianrui Wang", "Ziyang Ma", "Yizhou Peng", "Haoyu Wang", "Zhikang Niu", "Zikang Huang", "Yihao Wu", "Yi-Wen Chao", "Yu Jiang", "Yuheng Lu", "Guanrou Yang", "Xuanchen Li", "Hexin Liu", "Chunyu Qiang", "Cheng Gong", "Yifan Yang", "Tianchi Liu", "Junyu Wang", "Nana Hou", "Meng Ge", "Fuming You", "Yang Wei", "Zhongqian Sun", "Hu Haifeng", "Xiaobao Wang", "Eng Siong Chng", "Xie Chen", "Longbiao Wang", "Jianwu Dang"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.411/"
doi: "10.18653/v1/2026.acl-long.411"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wang-2026-evaluating-expressive-appropriateness-speech-rich/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2026-evaluating-expressive-appropriateness-speech-rich/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/29/) / Evaluating the Expressive Appropriateness of Speech in Rich Contexts

# Evaluating the Expressive Appropriateness of Speech in Rich Contexts

**Authors:** [Tianrui Wang](https://md.research.elamur.ai/free/en/authors/tianrui-wang/), [Ziyang Ma](https://md.research.elamur.ai/free/en/authors/ziyang-ma/), Yizhou Peng, [Haoyu Wang](https://md.research.elamur.ai/free/en/authors/haoyu-wang/), [Zhikang Niu](https://md.research.elamur.ai/free/en/authors/zhikang-niu/), Zikang Huang, Yihao Wu, Yi-Wen Chao, Yu Jiang, [Yuheng Lu](https://md.research.elamur.ai/free/en/authors/yuheng-lu/), Guanrou Yang, Xuanchen Li, [Hexin Liu](https://md.research.elamur.ai/free/en/authors/hexin-liu/), [Chunyu Qiang](https://md.research.elamur.ai/free/en/authors/chunyu-qiang/), Cheng Gong, [Yifan Yang](https://md.research.elamur.ai/free/en/authors/yifan-yang/), Tianchi Liu, Junyu Wang, Nana Hou, Meng Ge, Fuming You, Yang Wei, Zhongqian Sun, Hu Haifeng, Xiaobao Wang, Eng Siong Chng, [Xie Chen](https://md.research.elamur.ai/free/en/authors/xie-chen/), [Longbiao Wang](https://md.research.elamur.ai/free/en/authors/longbiao-wang/) and Jianwu Dang · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.411/) · [PDF](https://aclanthology.org/2026.acl-long.411.pdf) · [DOI 10.18653/v1/2026.acl-long.411](https://doi.org/10.18653/v1/2026.acl-long.411) · [OpenAlex](https://openalex.org/W7166845635)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Current speech evaluation tools mostly score naturalness, signal quality or emotional intensity for single utterances. They cannot tell whether a delivery suits its story or dialogue situation: a line that sounds angry in isolation may need to convey restrained amusement given the preceding conversation. The authors define a task they call context-rich expressive appropriateness evaluation for Mandarin and build the CEAEval framework around it. Its dataset, CEAEval-D, is drawn from performed audiobooks. A manually labelled subset pairs real human speech with long narrative context and fifteen annotation dimensions. The evaluator, CEAEval-M, separates long-context textual reasoning (a planner) from perceptual scoring of the audio (a judge). It also uses distillation from automatically captioned data, an adaptive bias that keeps attention on audio, and reinforcement learning. The output is a 0–5 score with an explanatory rationale. On a human-annotated test set, the system agrees with human raters better than existing speech evaluators, and this agreement holds as context grows longer. This matters for assessing audiobook TTS and conversational agents.

## Key points
- **CEAEval-D corpus:** 84 Mandarin audiobook works (2 of them high-quality TTS), totalling 3,505 hours.
  - Weak captions for the full corpus were produced with Qwen3-Omni-Captioner.
  - 16.1 hours were human-annotated: 14.65 h for training and 1.45 h for testing, with no overlapping samples.
- **Annotation scheme:** 15 dimensions covering expressive attributes and overall appropriateness, with reported high inter-annotator agreement.
  - The attributes include emotion, intonation/rhythm, recording conditions and paralinguistic vocalizations.
- **Results:** CEAEval-M reaches a linear correlation of 0.72 with human scores and 70.8% accuracy, outperforming prior speech evaluation and analysis systems.
- **Releases:**
  - The annotated subset will be published under CC-BY-NC, with each segment under 10 minutes.
  - Model checkpoints will also be released.
  - The full 3,505-hour corpus will not be distributed.
- **Limitations:** The work covers Mandarin only, and context is modelled from text rather than from neighbouring audio.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [National University of Singapore](https://md.research.elamur.ai/free/en/universities/national-university-of-singapore/) — University, Singapore, Singapore · 180 works · 34,304 citations
- [Tencent](https://md.research.elamur.ai/free/en/companies/tencent/) — Company, Shenzhen, China · 134 works · 19,743 citations
- [Shanghai Jiao Tong University](https://md.research.elamur.ai/free/en/universities/shanghai-jiao-tong-university/) — University, Shanghai, China · 149 works · 16,161 citations
- [Nanyang Technological University](https://md.research.elamur.ai/free/en/universities/nanyang-technological-university/) — University, Singapore, Singapore · 156 works · 25,644 citations
- [Tianjin University](https://md.research.elamur.ai/free/en/universities/tianjin-university/) — University, Tianjin, China · 41 works · 1,128 citations
- [Kuaishou](https://md.research.elamur.ai/free/en/companies/kuaishou/) — Company, Beijing, China · 15 works · 95 citations
- [China Telecom](https://md.research.elamur.ai/free/en/companies/china-telecom-cn/) — Company, Beijing, China · 7 works · 16 citations
- [China Telecom (China)](https://md.research.elamur.ai/free/en/companies/china-telecom/) — Company, Changsha, China · 6 works · 15 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,552 works · [💭 Experimental and Cognitive Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/experimental-and-cognitive-psychology/) · 870 works

🧭 Topics (1): [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,220 works

## Cite

Cite as: Tianrui Wang, Ziyang Ma, Yizhou Peng, Haoyu Wang, Zhikang Niu, Zikang Huang, Yihao Wu, Yi-Wen Chao, Yu Jiang, Yuheng Lu, Guanrou Yang, Xuanchen Li, Hexin Liu, Chunyu Qiang, Cheng Gong, Yifan Yang, Tianchi Liu, Junyu Wang, Nana Hou, Meng Ge, Fuming You, Yang Wei, Zhongqian Sun, Hu Haifeng, Xiaobao Wang, Eng Siong Chng, Xie Chen, Longbiao Wang and Jianwu Dang. 2026. Evaluating the Expressive Appropriateness of Speech in Rich Contexts. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wang2026evaluating,
  title = {{Evaluating the Expressive Appropriateness of Speech in Rich Contexts}},
  author = {Tianrui Wang, Ziyang Ma, Yizhou Peng, Haoyu Wang, Zhikang Niu, Zikang Huang, Yihao Wu, Yi-Wen Chao, Yu Jiang, Yuheng Lu, Guanrou Yang, Xuanchen Li, Hexin Liu, Chunyu Qiang, Cheng Gong, Yifan Yang, Tianchi Liu, Junyu Wang, Nana Hou, Meng Ge, Fuming You, Yang Wei, Zhongqian Sun, Hu Haifeng, Xiaobao Wang, Eng Siong Chng, Xie Chen, Longbiao Wang and Jianwu Dang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.411},
  url = {https://doi.org/10.18653/v1/2026.acl-long.411}
}
```

- [← previous Evaluating the Effect of Question Wording Variations on Answer Consistency in Large Language Models](https://md.research.elamur.ai/free/en/papers/takayama-2026-evaluating-effect-question-wording-variations/)
- [next → Evaluating the Impact of Source Diversity for RAG in Historical Research](https://md.research.elamur.ai/free/en/papers/mahadeshwar-2026-evaluating-impact-source-diversity-rag/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/29/)

## References

43 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. 1996. Methods for subjective determination of transmission quality. Technical Report ITU-T Recommendation P.800, International Telecommunication Union. [find](https://scholar.google.com/scholar?q=1996%2E+Methods+for+subjective+determination+of+transmission+quality%2E+Technical+Report+ITU%2DT+Recommendation+P%2E800%2C+International+Telecommunication+Union%2E)
2. Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J Hewett, Mojan Javaheripi, Piero Kauffmann, and 1 others. 2024. [Phi-4 technical report](https://md.research.elamur.ai/free/en/papers/abdin-2024-phi-4-technical-report/). arXiv preprint arXiv:2412.08905.
3. Philip Anastassiou, Jiawei Chen, Jitong Chen, Yuanzhe Chen, Zhuo Chen, Ziyi Chen, Jian Cong, Lelai Deng, Chuang Ding, Lu Gao, and 1 others. 2024. [Seed-tts: A family of high-quality versatile speech generation models](https://md.research.elamur.ai/free/en/papers/anastassiou-2024-seed-tts-family-high/). arXiv preprint arXiv:2406.02430.
4. Jacob Benesty, Jingdong Chen, Yiteng Huang, and Israel Cohen. 2009. Pearson Correlation Coefficient, pages 1–4. Springer Berlin Heidelberg, Berlin, Heidelberg. [find](https://scholar.google.com/scholar?q=Jacob+Benesty%2C+Jingdong+Chen%2C+Yiteng+Huang%2C+and+Israel+Cohen%2E+2009%2E+Pearson+Correlation+Coefficient%2C+pages+1%E2%80%934%2E+Springer+Berlin+Heidelberg%2C+Berlin%2C+Heidelberg%2E)
5. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, Luke Marris, Sam Petulla, Colin Gaffney, Asaf Aharoni, Nathan Lintz, Tiago Cardal Pais, Henrik Jacobsson, Idan Szpektor, Nan-Jiang Jiang, and 3416 [other…](https://md.research.elamur.ai/free/en/papers/comanici-2025-gemini-2-5-pushing-frontier/)
6. Jian Cong, Shan Yang, Na Hu, Guangzhi Li, Lei Xie, and Dan Su. 2021. Controllable context-aware conversational speech synthesis. arXiv preprint arXiv:2106.10828. — [arXiv:2106.10828](https://arxiv.org/abs/2106.10828) [find](https://scholar.google.com/scholar?q=Jian+Cong%2C+Shan+Yang%2C+Na+Hu%2C+Guangzhi+Li%2C+Lei+Xie%2C+and+Dan+Su%2E+2021%2E+Controllable+context%2Daware+conversational+speech+synthesis%2E+arXiv+preprint+arXiv%3A2106%2E10828%2E+%E2%80%94+%5BarXiv%3A2106%2E10828%5D%28https%3A%2F%2Farxiv%2Eorg)
7. Heinrich Dinkel, Gang Li, Jizhong Liu, Jian Luan, Yadong Niu, Xingwei Sun, Tianzi Wang, Qiyang Xiao, Junbo Zhang, and Jiahao Zhou. 2025. Midashenglm: Efficient audio understanding with general audio captions. arXiv preprint arXiv:2508.03983. — [arXiv:2508.03983](https://arxiv.org/abs/2508.03983) [find](https://scholar.google.com/scholar?q=Heinrich+Dinkel%2C+Gang+Li%2C+Jizhong+Liu%2C+Jian+Luan%2C+Yadong+Niu%2C+Xingwei+Sun%2C+Tianzi+Wang%2C+Qiyang+Xiao%2C+Junbo+Zhang%2C+and+Jiahao+Zhou%2E+2025%2E+Midashenglm%3A+Efficient+audio+understanding+with+general+audio+c)
8. Szu-Wei Fu, Yu Tsao, Hsin-Te Hwang, and Hsin-Min Wang. 2018. Quality-net: An end-to-end non-intrusive speech quality assessment model based on blstm. arXiv preprint arXiv:1808.05344. — [arXiv:1808.05344](https://arxiv.org/abs/1808.05344) [find](https://scholar.google.com/scholar?q=Szu%2DWei+Fu%2C+Yu+Tsao%2C+Hsin%2DTe+Hwang%2C+and+Hsin%2DMin+Wang%2E+2018%2E+Quality%2Dnet%3A+An+end%2Dto%2Dend+non%2Dintrusive+speech+quality+assessment+model+based+on+blstm%2E+arXiv+preprint+arXiv%3A1808%2E05344%2E+%E2%80%94+%5BarXiv%3A1808%2E053)
9. Zhifu Gao, Zerui Li, Jiaming Wang, Haoneng Luo, Xian Shi, Mengzhe Chen, Yabin Li, Lingyun Zuo, Zhihao Du, and Shiliang Zhang. 2023. Funasr: A fundamental end-to-end speech recognition toolkit. In Interspeech 2023, pages 1593–1597. [find](https://scholar.google.com/scholar?q=Zhifu+Gao%2C+Zerui+Li%2C+Jiaming+Wang%2C+Haoneng+Luo%2C+Xian+Shi%2C+Mengzhe+Chen%2C+Yabin+Li%2C+Lingyun+Zuo%2C+Zhihao+Du%2C+and+Shiliang+Zhang%2E+2023%2E+Funasr%3A+A+fundamental+end%2Dto%2Dend+speech+recognition+toolkit%2E+In+Inte)
10. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, and 1 others. 2025. [Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-r1-incentivizing-reasoning-capability/). arXiv preprint arXiv:2501.12948.
11. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, and 1 others. 2022. [Lora: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). ICLR, 1(2):3.
12. Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, and 1 others. 2024. [Gpt-4o system card](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/). arXiv preprint arXiv:2410.21276.
13. Chae-Bin Im, Sang-Hoon Lee, Seung-Bin Kim, and Seong-Whan Lee. 2022. Emoq-tts: Emotion intensity quantization for fine-grained controllable emotional text-to-speech. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6317–6321. IEEE. [find](https://scholar.google.com/scholar?q=Chae%2DBin+Im%2C+Sang%2DHoon+Lee%2C+Seung%2DBin+Kim%2C+and+Seong%2DWhan+Lee%2E+2022%2E+Emoq%2Dtts%3A+Emotion+intensity+quantization+for+fine%2Dgrained+controllable+emotional+text%2Dto%2Dspeech%2E+In+ICASSP+2022%2D2022+IEEE+Internati)
14. Shengpeng Ji, Yifu Chen, Minghui Fang, Jialong Zuo, Jingyu Lu, Hanting Wang, Ziyue Jiang, Long Zhou, Shujie Liu, Xize Cheng, and 1 others. 2024. Wavchat: A survey of spoken dialogue models. arXiv preprint arXiv:2411.13577. [find](https://scholar.google.com/scholar?q=Shengpeng+Ji%2C+Yifu+Chen%2C+Minghui+Fang%2C+Jialong+Zuo%2C+Jingyu+Lu%2C+Hanting+Wang%2C+Ziyue+Jiang%2C+Long+Zhou%2C+Shujie+Liu%2C+Xize+Cheng%2C+and+1+others%2E+2024%2E+Wavchat%3A+A+survey+of+spoken+dialogue+models%2E+arXiv+prep)
15. Shengpeng Ji, Tianle Liang, Yangzhuo Li, Jialong Zuo, Minghui Fang, Jinzheng He, Yifu Chen, Zhengqing Liu, Ziyue Jiang, Xize Cheng, and 1 others. 2025. Wavreward: Spoken dialogue models with generalist reward evaluators. arXiv preprint arXiv:2505.09558. — [arXiv:2505.09558](https://arxiv.org/abs/2505.09558) [find](https://scholar.google.com/scholar?q=Shengpeng+Ji%2C+Tianle+Liang%2C+Yangzhuo+Li%2C+Jialong+Zuo%2C+Minghui+Fang%2C+Jinzheng+He%2C+Yifu+Chen%2C+Zhengqing+Liu%2C+Ziyue+Jiang%2C+Xize+Cheng%2C+and+1+others%2E+2025%2E+Wavreward%3A+Spoken+dialogue+models+with+generalis)
16. Changhao Jiang, Jiajun Sun, Yifei Cao, Jiabao Zhuang, Hui Li, Baoyu Fan, Tao Ji, Tao Gui, and Qi Zhang. 2025. Speechrole: A large-scale dataset and benchmark for evaluating speech role-playing agents. arXiv preprint arXiv:2508.02013. — [arXiv:2508.02013](https://arxiv.org/abs/2508.02013) [find](https://scholar.google.com/scholar?q=Changhao+Jiang%2C+Jiajun+Sun%2C+Yifei+Cao%2C+Jiabao+Zhuang%2C+Hui+Li%2C+Baoyu+Fan%2C+Tao+Ji%2C+Tao+Gui%2C+and+Qi+Zhang%2E+2025%2E+Speechrole%3A+A+large%2Dscale+dataset+and+benchmark+for+evaluating+speech+role%2Dplaying+agents%2E)
17. Gabriel Mittag, Babak Naderi, Assmaa Chehadi, and Sebastian Möller. 2021. Nisqa: A deep cnn-self-attention model for multidimensional speech quality prediction with crowdsourced datasets. arXiv preprint arXiv:2104.09494. [find](https://scholar.google.com/scholar?q=Gabriel+Mittag%2C+Babak+Naderi%2C+Assmaa+Chehadi%2C+and+Sebastian+M%C3%B6ller%2E+2021%2E+Nisqa%3A+A+deep+cnn%2Dself%2Dattention+model+for+multidimensional+speech+quality+prediction+with+crowdsourced+datasets%2E+arXiv+prepri)
18. KimiTeam, Ding Ding, Zeqian Ju, Yichong Leng, Songxiang Liu, Tong Liu, Zeyu Shang, Kai Shen, Wei Song, Xu Tan, Heyi Tang, Zhengtao Wang, Chu Wei, Yifei Xin, Xinran Xu, Jianwei Yu, Yutao Zhang, Xinyu Zhou, Y. Charles, and 21 others. 2025. [Kimi-audio technical report](https://md.research.elamur.ai/free/en/papers/kimiteam-2025-kimi-audio-technical-report/). Preprint, arXiv:2504.18425.
19. Saif M Mohammad. 2025. [Nrc vad lexicon v2: Norms for valence, arousal, and dominance for over 55k english terms](https://md.research.elamur.ai/free/en/papers/mohammad-2025-nrc-vad-lexicon-v2-norms/). arXiv preprint arXiv:2503.23547.
20. Alexander H Liu, Andy Ehrenberg, Andy Lo, Clément Denoix, Corentin Barreau, Guillaume Lample, Jean-Malo Delignon, Khyathi Raghavi Chandu, Patrick von Platen, Pavankumar Reddy Muddireddy, and 1 others. 2025. Voxtral. arXiv preprint arXiv:2507.13264. [find](https://scholar.google.com/scholar?q=Alexander+H+Liu%2C+Andy+Ehrenberg%2C+Andy+Lo%2C+Cl%C3%A9ment+Denoix%2C+Corentin+Barreau%2C+Guillaume+Lample%2C+Jean%2DMalo+Delignon%2C+Khyathi+Raghavi+Chandu%2C+Patrick+von+Platen%2C+Pavankumar+Reddy+Muddireddy%2C+and+1+others%2E)
21. Shi Liu, Kecheng Zheng, and Wei Chen. 2024. Paying more attention to image: A training-free method for alleviating hallucination in lvlms. In European Conference on Computer Vision, pages 125–140. Springer. [find](https://scholar.google.com/scholar?q=Shi+Liu%2C+Kecheng+Zheng%2C+and+Wei+Chen%2E+2024%2E+Paying+more+attention+to+image%3A+A+training%2Dfree+method+for+alleviating+hallucination+in+lvlms%2E+In+European+Conference+on+Computer+Vision%2C+pages+125%E2%80%93140%2E+Spr)
22. Chen-Chou Lo, Szu-Wei Fu, Wen-Chin Huang, Xin Wang, Junichi Yamagishi, Yu Tsao, and Hsin-Min Wang. 2019. Mosnet: Deep learning-based objective assessment for voice conversion. [find](https://scholar.google.com/scholar?q=Chen%2DChou+Lo%2C+Szu%2DWei+Fu%2C+Wen%2DChin+Huang%2C+Xin+Wang%2C+Junichi+Yamagishi%2C+Yu+Tsao%2C+and+Hsin%2DMin+Wang%2E+2019%2E+Mosnet%3A+Deep+learning%2Dbased+objective+assessment+for+voice+conversion%2E)
23. Ziyang Ma, Ruiyang Xu, Zhenghao Xing, Yunfei Chu, Yuxuan Wang, Jinzheng He, Jin Xu, Pheng-Ann Heng, Kai Yu, Junyang Lin, Eng Siong Chng, and Xie Chen. 2025. Omni-captioner: Data pipeline, models, and benchmark for omni detailed perception. Preprint, arXiv:2510.12720. — [arXiv:2510.12720](https://arxiv.org/abs/2510.12720) [find](https://scholar.google.com/scholar?q=Ziyang+Ma%2C+Ruiyang+Xu%2C+Zhenghao+Xing%2C+Yunfei+Chu%2C+Yuxuan+Wang%2C+Jinzheng+He%2C+Jin+Xu%2C+Pheng%2DAnn+Heng%2C+Kai+Yu%2C+Junyang+Lin%2C+Eng+Siong+Chng%2C+and+Xie+Chen%2E+2025%2E+Omni%2Dcaptioner%3A+Data+pipeline%2C+models%2C+and+)
24. Ziyang Ma, Zhisheng Zheng, Jiaxin Ye, Jinchao Li, Zhifu Gao, Shiliang Zhang, and Xie Chen. 2024. emotion2vec: Self-supervised pre-training for speech emotion representation. In Findings of the Association for Computational Linguistics: ACL 2024, pages 15747–15760. [find](https://scholar.google.com/scholar?q=Ziyang+Ma%2C+Zhisheng+Zheng%2C+Jiaxin+Ye%2C+Jinchao+Li%2C+Zhifu+Gao%2C+Shiliang+Zhang%2C+and+Xie+Chen%2E+2024%2E+emotion2vec%3A+Self%2Dsupervised+pre%2Dtraining+for+speech+emotion+representation%2E+In+Findings+of+the+Associa)
25. Georgia Maniati, Alexandra Vioni, Nikolaos Ellinas, Karolos Nikitaras, Konstantinos Klapsas, June Sig Sung, Gunu Jho, Aimilios Chalamandaris, and Pirros Tsiakoulis. 2022. Somos: The samsung open mos dataset for the evaluation of neural text-to-speech synthesis. arXiv preprint arXiv:2204.03040. — [arXiv:2204.03040](https://arxiv.org/abs/2204.03040) [find](https://scholar.google.com/scholar?q=Georgia+Maniati%2C+Alexandra+Vioni%2C+Nikolaos+Ellinas%2C+Karolos+Nikitaras%2C+Konstantinos+Klapsas%2C+June+Sig+Sung%2C+Gunu+Jho%2C+Aimilios+Chalamandaris%2C+and+Pirros+Tsiakoulis%2E+2022%2E+Somos%3A+The+samsung+open+mos+d)
26. Ruskin Raj Manku, Yuzhi Tang, Xingjian Shi, Mu Li, and Alex Smola. 2025. Emergenttts-eval: Evaluating tts models on complex prosodic, expressiveness, and linguistic challenges using model-as-a-judge. arXiv preprint arXiv:2505.23009. — [arXiv:2505.23009](https://arxiv.org/abs/2505.23009) [find](https://scholar.google.com/scholar?q=Ruskin+Raj+Manku%2C+Yuzhi+Tang%2C+Xingjian+Shi%2C+Mu+Li%2C+and+Alex+Smola%2E+2025%2E+Emergenttts%2Deval%3A+Evaluating+tts+models+on+complex+prosodic%2C+expressiveness%2C+and+linguistic+challenges+using+model%2Das%2Da%2Djudge%2E+)
27. Kenneth O McGraw and Seok P Wong. 1996. Forming inferences about some intraclass correlation coefficients. Psychological methods, 1(1):30. [find](https://scholar.google.com/scholar?q=Kenneth+O+McGraw+and+Seok+P+Wong%2E+1996%2E+Forming+inferences+about+some+intraclass+correlation+coefficients%2E+Psychological+methods%2C+1%281%29%3A30%2E)
28. Mary L McHugh. 2012. Interrater reliability: the kappa statistic. Biochemia medica, 22(3):276–282. [find](https://scholar.google.com/scholar?q=Mary+L+McHugh%2E+2012%2E+Interrater+reliability%3A+the+kappa+statistic%2E+Biochemia+medica%2C+22%283%29%3A276%E2%80%93282%2E)
29. Kyeongman Park, Seongho Joo, and Kyomin Jung. 2025. Multiactor-audiobook: Zero-shot audiobook generation with faces and voices of multiple speakers. arXiv preprint arXiv:2505.13082. — [arXiv:2505.13082](https://arxiv.org/abs/2505.13082) [find](https://scholar.google.com/scholar?q=Kyeongman+Park%2C+Seongho+Joo%2C+and+Kyomin+Jung%2E+2025%2E+Multiactor%2Daudiobook%3A+Zero%2Dshot+audiobook+generation+with+faces+and+voices+of+multiple+speakers%2E+arXiv+preprint+arXiv%3A2505%2E13082%2E+%E2%80%94+%5BarXiv%3A2505%2E1308)
30. Chandan KA Reddy, Vishak Gopal, and Ross Cutler. 2021. Dnsmos: A non-intrusive perceptual objective speech quality metric to evaluate noise suppressors. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6493–6497. IEEE. [find](https://scholar.google.com/scholar?q=Chandan+KA+Reddy%2C+Vishak+Gopal%2C+and+Ross+Cutler%2E+2021%2E+Dnsmos%3A+A+non%2Dintrusive+perceptual+objective+speech+quality+metric+to+evaluate+noise+suppressors%2E+In+ICASSP+2021%2D2021+IEEE+International+Conferen)
31. Nils Reimers and Iryna Gurevych. 2019. [Sentence-bert: Sentence embeddings using siamese bert-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics.
32. Flávio Ribeiro, Dinei Florêncio, Cha Zhang, and Michael Seltzer. 2011. Crowdmos: An approach for crowdsourcing mean opinion score studies. In 2011 IEEE international conference on acoustics, speech and signal processing (ICASSP), pages 2416–2419. IEEE. [find](https://scholar.google.com/scholar?q=Fl%C3%A1vio+Ribeiro%2C+Dinei+Flor%C3%AAncio%2C+Cha+Zhang%2C+and+Michael+Seltzer%2E+2011%2E+Crowdmos%3A+An+approach+for+crowdsourcing+mean+opinion+score+studies%2E+In+2011+IEEE+international+conference+on+acoustics%2C+speech+an)
33. Jiatong Shi, Jionghao Han, Yichen Lu, Santiago Pascual, Pengfei Wu, Chenye Cui, Shinji Watanabe, Chao Weng, and Cong Zhou. 2025. Speech-drame: A framework for human-aligned benchmarks in speech role-play. arXiv preprint arXiv:2511.01261. — [arXiv:2511.01261](https://arxiv.org/abs/2511.01261) [find](https://scholar.google.com/scholar?q=Jiatong+Shi%2C+Jionghao+Han%2C+Yichen+Lu%2C+Santiago+Pascual%2C+Pengfei+Wu%2C+Chenye+Cui%2C+Shinji+Watanabe%2C+Chao+Weng%2C+and+Cong+Zhou%2E+2025%2E+Speech%2Ddrame%3A+A+framework+for+human%2Daligned+benchmarks+in+speech+role%2Dp)
34. Mong Yuan Sim, Wei Emma Zhang, Xiang Dai, and Biaoyan Fang. 2025. Can vlms actually see and read? a survey on modality collapse in vision-language models. In Findings of the Association for Computational Linguistics: ACL 2025, pages 24452–24470. [find](https://scholar.google.com/scholar?q=Mong+Yuan+Sim%2C+Wei+Emma+Zhang%2C+Xiang+Dai%2C+and+Biaoyan+Fang%2E+2025%2E+Can+vlms+actually+see+and+read%3F+a+survey+on+modality+collapse+in+vision%2Dlanguage+models%2E+In+Findings+of+the+Association+for+Computatio)
35. Changli Tang, Wenyi Yu, Guangzhi Sun, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun MA, and Chao Zhang. 2024. [SALMONN: Towards generic hearing abilities for large language models](https://md.research.elamur.ai/free/en/papers/tang-2023-salmonn-towards-generic-hearing-abilities/). In The Twelfth International Conference on Learning Representations.
36. Ashish Tawari and Mohan Manubhai Trivedi. 2010. Speech emotion analysis: Exploring the role of context. IEEE Transactions on multimedia, 12(6):502– 509. [find](https://scholar.google.com/scholar?q=Ashish+Tawari+and+Mohan+Manubhai+Trivedi%2E+2010%2E+Speech+emotion+analysis%3A+Exploring+the+role+of+context%2E+IEEE+Transactions+on+multimedia%2C+12%286%29%3A502%E2%80%93+509%2E)
37. Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, and 1 others. 2024. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295. [find](https://scholar.google.com/scholar?q=Gemma+Team%2C+Thomas+Mesnard%2C+Cassidy+Hardin%2C+Robert+Dadashi%2C+Surya+Bhupatiraju%2C+Shreya+Pathak%2C+Laurent+Sifre%2C+Morgane+Rivi%C3%A8re%2C+Mihir+Sanjay+Kale%2C+Juliette+Love%2C+and+1+others%2E+2024%2E+Gemma%3A+Open+models+b)
38. Fei Tian, Xiangyu Tony Zhang, Yuxin Zhang, Haoyang Zhang, Yuxin Li, Daijiao Liu, Yayue Deng, Dong-hang Wu, Jun Chen, Liang Zhao, Chengyuan Yao, Hexin Liu, Eng Siong Chng, Xuerui Yang, Xiangyu Zhang, Daxin Jiang, and Gang Yu. 2025. [Step-audio-r1 technical report](https://md.research.elamur.ai/free/en/papers/tian-2025-step-audio-r1-technical/). Preprint, arXiv:2511.15848.
39. Junyu Wang, Ziyang Ma, Zhengding Luo, Tianrui Wang, Meng Ge, Xiaobao Wang, and Longbiao Wang. 2025a. Pay more attention to audio: Mitigating imbalance of cross-modal attention in large audio language models. arXiv preprint arXiv:2509.18816. — [arXiv:2509.18816](https://arxiv.org/abs/2509.18816) [find](https://scholar.google.com/scholar?q=Junyu+Wang%2C+Ziyang+Ma%2C+Zhengding+Luo%2C+Tianrui+Wang%2C+Meng+Ge%2C+Xiaobao+Wang%2C+and+Longbiao+Wang%2E+2025a%2E+Pay+more+attention+to+audio%3A+Mitigating+imbalance+of+cross%2Dmodal+attention+in+large+audio+language+)
40. Siyin Wang, Wenyi Yu, Xianzhao Chen, Xiaohai Tian, Jun Zhang, Lu Lu, Yu Tsao, Junichi Yamagishi, Yuxuan Wang, and Chao Zhang. 2025b. [Qualispeech: A speech quality assessment dataset with natural language reasoning and descriptions](https://md.research.elamur.ai/free/en/papers/wang-2025-qualispeech-speech-quality-assessment-dataset/). arXiv preprint arXiv:2503.20290.
41. Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang, and 1 others. 2025. Qwen2. 5-omni technical report. arXiv preprint arXiv:2503.20215. — [Qwen2.5-Omni Technical Report](https://md.research.elamur.ai/free/en/papers/xu-2025-qwen2-5-omni-technical/)
42. Ruiqi Yan, Xiquan Li, Wenxi Chen, Zhikang Niu, Chen Yang, Ziyang Ma, Kai Yu, and Xie Chen. 2025. Uro-bench: A comprehensive benchmark for end-to-end spoken dialogue models. arXiv preprint arXiv:2502.17810. — [arXiv:2502.17810](https://arxiv.org/abs/2502.17810) [find](https://scholar.google.com/scholar?q=Ruiqi+Yan%2C+Xiquan+Li%2C+Wenxi+Chen%2C+Zhikang+Niu%2C+Chen+Yang%2C+Ziyang+Ma%2C+Kai+Yu%2C+and+Xie+Chen%2E+2025%2E+Uro%2Dbench%3A+A+comprehensive+benchmark+for+end%2Dto%2Dend+spoken+dialogue+models%2E+arXiv+preprint+arXiv%3A2502%2E1)
43. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). arXiv preprint arXiv:2505.09388.

*Updated 1 November 2026*