---
title: "AffectCodec — paper by Shi et al. (2026) — summary and ideas"
description: "Earlier studies show that emotional nuance is often lost in this step, even when overall reconstruction sounds good. 77 references."
authors: ["Jiacheng Shi", "Hongfei Du", "Xinyuan Song", "Y. Alicia Hong", "Yanfu Zhang", "Ashley Gao"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.442/"
doi: "10.18653/v1/2026.findings-acl.442"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/shi-2026-affectcodec-emotion-preserving-neural-speech/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/shi-2026-affectcodec-emotion-preserving-neural-speech/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/21/) / AffectCodec: Emotion-Preserving Neural Speech Codec for Expressive Speech Modeling

# AffectCodec: Emotion-Preserving Neural Speech Codec for Expressive Speech Modeling

**Authors:** Jiacheng Shi, Hongfei Du, [Xinyuan Song](https://md.research.elamur.ai/free/en/authors/xinyuan-song/), Y. Alicia Hong, [Yanfu Zhang](https://md.research.elamur.ai/free/en/authors/yanfu-zhang/) and [Ashley Gao](https://md.research.elamur.ai/free/en/authors/ashley-gao/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.442/) · [PDF](https://aclanthology.org/2026.findings-acl.442.pdf) · [DOI 10.18653/v1/2026.findings-acl.442](https://doi.org/10.18653/v1/2026.findings-acl.442) · [OpenAlex](https://openalex.org/W7166838889)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Neural speech codecs turn audio into discrete tokens for speech language models. Earlier studies show that emotional nuance is often lost in this step, even when overall reconstruction sounds good. AffectCodec makes emotion preservation an explicit training goal instead of an incidental side effect.

The codec uses an acoustic encoder with residual vector quantization (RVQ). It draws guidance from three frozen teachers: an emotion recognition model, a self-supervised speech model, and an ASR model paired with a text encoder. On top of this backbone it adds three mechanisms:
- **Latent modulation:** before quantization, cross-attention injects emotion and semantic cues into the acoustic latents.
- **Relation-preserving distillation:** the quantized representations are pushed to keep the relational structure of the emotion and semantic spaces.
- **Emotion-weighted soft alignment:** tokens are aligned with text embeddings, with extra weight on frames where emotion changes.

The authors evaluate reconstruction, emotion recognition (EMO-SUPERB, Codec-SUPERB) and zero-shot TTS. In TTS, AffectCodec gives the best emotion similarity and recall on emotional data while keeping content accuracy competitive. This matters for building expressive speech generation on top of tokenized speech.

## Key points
- **Zero-shot TTS on LibriSpeech:** WER 2.51, SIM-O 0.80, UTMOS 4.29. CosyVoice 2 reaches a slightly lower WER (2.45).
- **Emotional datasets:**
  - EmoVoice-DB: Emo_SIM 0.91 and recall 0.41, versus the best baseline's 0.87 and 0.37.
  - SECAP: Emo_SIM 0.84 and recall 0.49, plus the top UTMOS of 3.20.
- **Ablation:**
  - The guided-latent module (EG-Latent) raises reconstruction Emo SIM from 0.87 to 0.90 and MUSHRA from 86.27 to 88.31.
  - The full model reaches Emo SIM 0.94 and MUSHRA 90.71.
- **Distillation (RP-Distill):** mainly improves intelligibility, with WER falling from 5.75 to 5.18.
- **Baselines and limitations:** compared systems include MaskGCT, F5-TTS, FireRedTTS, CosyVoice 2, Llasa and SparkTTS. The authors acknowledge that the model is computationally heavy. A public demo is available at jiachengqaq.github.io/affectcodec_demo.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Emory University](https://md.research.elamur.ai/free/en/universities/emory-university/) — University, Atlanta, United States · 36 works · 39,779 citations
- [George Mason University](https://md.research.elamur.ai/free/en/universities/george-mason-university/) — University, Fairfax, United States · 24 works · 1,288 citations
- [William & Mary](https://md.research.elamur.ai/free/en/universities/william-mary/) — University, Williamsburg, United States · 5 works · 357 citations

📚 Shelves (7): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,552 works · [💭 Experimental and Cognitive Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/experimental-and-cognitive-psychology/) · 870 works · [🔊 Signal Processing](https://md.research.elamur.ai/free/en/shelves/computer-science/signal-processing/) · 452 works

🧭 Topics (2): [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 972 works · [🎙️ Speech & spoken language](https://md.research.elamur.ai/free/en/topics/speech/) · 480 works

## Cite

Cite as: Jiacheng Shi, Hongfei Du, Xinyuan Song, Y. Alicia Hong, Yanfu Zhang and Ashley Gao. 2026. AffectCodec: Emotion-Preserving Neural Speech Codec for Expressive Speech Modeling. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{shi2026affectcodec,
  title = {{AffectCodec: Emotion-Preserving Neural Speech Codec for Expressive Speech Modeling}},
  author = {Jiacheng Shi, Hongfei Du, Xinyuan Song, Y. Alicia Hong, Yanfu Zhang and Ashley Gao},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.442},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.442}
}
```

- [← previous AEQ-Bench: Measuring Empathy of Omni-Modal Large Models](https://md.research.elamur.ai/free/en/papers/luo-2026-aeq-bench-measuring-empathy-omni/)
- [next → Afri-MCQA: Multimodal Cultural Question Answering for African Languages](https://md.research.elamur.ai/free/en/papers/tonja-2026-afri-mcqa-multimodal-cultural-question/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/21/)

## References

77 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. 2020. [wav2vec 2.0: A framework for self-supervised learning of speech representations](https://md.research.elamur.ai/free/en/papers/baevski-2020-wav2vec-2-0-framework-self/). Advances in neural information processing systems, 33:12449–12460.
2. AmirAli Bagher Zadeh, Paul Pu Liang, Soujanya Poria, Erik Cambria, and Louis-Philippe Morency. 2018. [Multimodal language analysis in the wild: CMU-MOSEI dataset and interpretable dynamic fusion graph](https://md.research.elamur.ai/free/en/papers/zadeh-2018-multimodal-language-analysis-wild-cmu/). In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume…
3. Loïc Barrault, Yu-An Chung, Mariano Coria Meglioli, David Dale, Ning Dong, Mark Duppenthaler, Paul-Ambroise Duquenne, Brian Ellis, Hady Elsahar, Justin Haaheim, and 1 others. 2023. [Seamless: Multilingual expressive and streaming speech translation](https://md.research.elamur.ai/free/en/papers/communication-2023-seamless-multilingual-expressive-streaming-speech/). arXiv preprint arXiv:2312.05187.
4. Yoshua Bengio, Nicholas Léonard, and Aaron Courville. 2013. Estimating or propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv:1308.3432. [find](https://scholar.google.com/scholar?q=Yoshua+Bengio%2C+Nicholas+L%C3%A9onard%2C+and+Aaron+Courville%2E+2013%2E+Estimating+or+propagating+gradients+through+stochastic+neurons+for+conditional+computation%2E+arXiv+preprint+arXiv%3A1308%2E3432%2E)
5. Marina Bosi, Karlheinz Brandenburg, Schuyler Quackenbush, Louis Fielder, Kenzo Akagiri, Hendrik Fuchs, and Martin Dietz. 1997. Iso/iec mpeg-2 advanced audio coding. Journal of the Audio engineering society, 45(10):789–814. [find](https://scholar.google.com/scholar?q=Marina+Bosi%2C+Karlheinz+Brandenburg%2C+Schuyler+Quackenbush%2C+Louis+Fielder%2C+Kenzo+Akagiri%2C+Hendrik+Fuchs%2C+and+Martin+Dietz%2E+1997%2E+Iso%2Fiec+mpeg%2D2+advanced+audio+coding%2E+Journal+of+the+Audio+engineering+so)
6. Karlheinz Brandenburg. 1994. Iso-mpeg-1 audio: A generic standard for coding of high-quality digital audio. J. Audio Eng. Soc., 42(10):780–792. [find](https://scholar.google.com/scholar?q=Karlheinz+Brandenburg%2E+1994%2E+Iso%2Dmpeg%2D1+audio%3A+A+generic+standard+for+coding+of+high%2Dquality+digital+audio%2E+J%2E+Audio+Eng%2E+Soc%2E%2C+42%2810%29%3A780%E2%80%93792%2E)
7. Carlos Busso, Murtaza Bulut, Chi-Chun Lee, Abe Kazemzadeh, Emily Mower, Samuel Kim, Jeannette N Chang, Sungbok Lee, and Shrikanth S Narayanan. 2008. Iemocap: Interactive emotional dyadic motion capture database. Language resources and evaluation, 42(4):335–359. [find](https://scholar.google.com/scholar?q=Carlos+Busso%2C+Murtaza+Bulut%2C+Chi%2DChun+Lee%2C+Abe+Kazemzadeh%2C+Emily+Mower%2C+Samuel+Kim%2C+Jeannette+N+Chang%2C+Sungbok+Lee%2C+and+Shrikanth+S+Narayanan%2E+2008%2E+Iemocap%3A+Interactive+emotional+dyadic+motion+captur)
8. Carlos Busso, Reza Lotfian, Kusha Sridhar, Ali N Salman, Wei-Cheng Lin, Lucas Goncalves, Srinivas Parthasarathy, Abinay Reddy Naini, Seong-Gyun Leem, Luz Martinez-Lucas, and 1 others. 2025. The msp-podcast corpus. arXiv preprint arXiv:2509.09791. — [arXiv:2509.09791](https://arxiv.org/abs/2509.09791) [find](https://scholar.google.com/scholar?q=Carlos+Busso%2C+Reza+Lotfian%2C+Kusha+Sridhar%2C+Ali+N+Salman%2C+Wei%2DCheng+Lin%2C+Lucas+Goncalves%2C+Srinivas+Parthasarathy%2C+Abinay+Reddy+Naini%2C+Seong%2DGyun+Leem%2C+Luz+Martinez%2DLucas%2C+and+1+others%2E+2025%2E+The+msp%2Dpo)
9. Carlos Busso, Srinivas Parthasarathy, Alec Burmania, Mohammed AbdelWahab, Najmeh Sadoughi, and Emily Mower Provost. 2016. Msp-improv: An acted corpus of dyadic interactions to study emotion perception. IEEE Transactions on Affective Computing, 8(1):67–80. [find](https://scholar.google.com/scholar?q=Carlos+Busso%2C+Srinivas+Parthasarathy%2C+Alec+Burmania%2C+Mohammed+AbdelWahab%2C+Najmeh+Sadoughi%2C+and+Emily+Mower+Provost%2E+2016%2E+Msp%2Dimprov%3A+An+acted+corpus+of+dyadic+interactions+to+study+emotion+perception)
10. Houwei Cao, David G Cooper, Michael K Keutmann, Ruben C Gur, Ani Nenkova, and Ragini Verma. 2014. Crema-d: Crowd-sourced emotional multimodal actors dataset. IEEE transactions on affective computing, 5(4):377–390. [find](https://scholar.google.com/scholar?q=Houwei+Cao%2C+David+G+Cooper%2C+Michael+K+Keutmann%2C+Ruben+C+Gur%2C+Ani+Nenkova%2C+and+Ragini+Verma%2E+2014%2E+Crema%2Dd%3A+Crowd%2Dsourced+emotional+multimodal+actors+dataset%2E+IEEE+transactions+on+affective+computing%2C+)
11. Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, and 1 others. 2022. Wavlm: Large-scale self-supervised pre-training for full stack speech processing. IEEE Journal of Selected Topics in Signal Processing, 16(6):1505–1518. [find](https://scholar.google.com/scholar?q=Sanyuan+Chen%2C+Chengyi+Wang%2C+Zhengyang+Chen%2C+Yu+Wu%2C+Shujie+Liu%2C+Zhuo+Chen%2C+Jinyu+Li%2C+Naoyuki+Kanda%2C+Takuya+Yoshioka%2C+Xiong+Xiao%2C+and+1+others%2E+2022%2E+Wavlm%3A+Large%2Dscale+self%2Dsupervised+pre%2Dtraining+for+)
12. Yushen Chen, Zhikang Niu, Ziyang Ma, Keqi Deng, Chunhui Wang, JianZhao JianZhao, Kai Yu, and Xie Chen. 2025. [F5-tts: A fairytaler that fakes fluent and faithful speech with flow matching](https://md.research.elamur.ai/free/en/papers/chen-2025-f5-tts-fairytaler-that-fakes/). In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Paper…
13. Huang-Cheng Chou, Wei-Cheng Lin, Lien-Chiang Chang, Chyi-Chang Li, Hsi-Pin Ma, and Chi-Chun Lee. 2017. Nnime: The nthu-ntua chinese interactive multimodal emotion corpus. In 2017 Seventh international conference on affective computing and intelligent interaction (ACII), pages 292–298. IEEE. [find](https://scholar.google.com/scholar?q=Huang%2DCheng+Chou%2C+Wei%2DCheng+Lin%2C+Lien%2DChiang+Chang%2C+Chyi%2DChang+Li%2C+Hsi%2DPin+Ma%2C+and+Chi%2DChun+Lee%2E+2017%2E+Nnime%3A+The+nthu%2Dntua+chinese+interactive+multimodal+emotion+corpus%2E+In+2017+Seventh+international)
14. Djork-Arné Clevert, Thomas Unterthiner, and Sepp Hochreiter. 2015. [Fast and accurate deep network learning by exponential linear units (elus)](https://md.research.elamur.ai/free/en/papers/clevert-2015-fast-accurate-deep-network-learning/). arXiv preprint arXiv:1511.07289, 4(5):11.
15. Alexandre Défossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi. 2022. [High fidelity neural audio compression](https://md.research.elamur.ai/free/en/papers/defossez-2022-high-fidelity-neural-audio-compression/). arXiv preprint arXiv:2210.13438.
16. Alexandre Défossez, Laurent Mazaré, Manu Orsini, Amélie Royer, Patrick Pérez, Hervé Jégou, Edouard Grave, and Neil Zeghidour. 2024. [Moshi: a speech-text foundation model for real-time dialogue](https://md.research.elamur.ai/free/en/papers/defossez-2024-moshi-speech-text-foundation-model/). arXiv preprint arXiv:2410.00037.
17. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies,…
18. Zhihao Du, Yuxuan Wang, Qian Chen, Xian Shi, Xiang Lv, Tianyu Zhao, Zhifu Gao, Yexin Yang, Changfeng Gao, Hui Wang, and 1 others. 2024a. Cosyvoice 2: Scalable streaming speech synthesis with large language models. arXiv preprint arXiv:2412.10117. [find](https://scholar.google.com/scholar?q=Zhihao+Du%2C+Yuxuan+Wang%2C+Qian+Chen%2C+Xian+Shi%2C+Xiang+Lv%2C+Tianyu+Zhao%2C+Zhifu+Gao%2C+Yexin+Yang%2C+Changfeng+Gao%2C+Hui+Wang%2C+and+1+others%2E+2024a%2E+Cosyvoice+2%3A+Scalable+streaming+speech+synthesis+with+large+lan)
19. Zhihao Du, Shiliang Zhang, Kai Hu, and Siqi Zheng. 2024b. Funcodec: A fundamental, reproducible and integrable open-source toolkit for neural speech codec. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 591–595. IEEE. [find](https://scholar.google.com/scholar?q=Zhihao+Du%2C+Shiliang+Zhang%2C+Kai+Hu%2C+and+Siqi+Zheng%2E+2024b%2E+Funcodec%3A+A+fundamental%2C+reproducible+and+integrable+open%2Dsource+toolkit+for+neural+speech+codec%2E+In+ICASSP+2024%2D2024+IEEE+International+Confe)
20. Benjamin Elizalde, Soham Deshmukh, Mahmoud Al Ismail, and Huaming Wang. 2023. Clap learning audio concepts from natural language supervision. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE. [find](https://scholar.google.com/scholar?q=Benjamin+Elizalde%2C+Soham+Deshmukh%2C+Mahmoud+Al+Ismail%2C+and+Huaming+Wang%2E+2023%2E+Clap+learning+audio+concepts+from+natural+language+supervision%2E+In+ICASSP+2023%2D2023+IEEE+International+Conference+on+Acous)
21. Xiaoxue Gao, Chen Zhang, Yiming Chen, Huayun Zhang, and Nancy F Chen. 2025. Emo-dpo: Controllable emotional speech synthesis through direct preference optimization. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE. [find](https://scholar.google.com/scholar?q=Xiaoxue+Gao%2C+Chen+Zhang%2C+Yiming+Chen%2C+Huayun+Zhang%2C+and+Nancy+F+Chen%2E+2025%2E+Emo%2Ddpo%3A+Controllable+emotional+speech+synthesis+through+direct+preference+optimization%2E+In+ICASSP+2025%2D2025+IEEE+Internatio)
22. Jort F Gemmeke, Daniel PW Ellis, Dylan Freedman, Aren Jansen, Wade Lawrence, R Channing Moore, Manoj Plakal, and Marvin Ritter. 2017. Audio set: An ontology and human-labeled dataset for audio events. In 2017 IEEE international conference on acoustics, speech and signal processing (ICASSP), pages 77… [find](https://scholar.google.com/scholar?q=Jort+F+Gemmeke%2C+Daniel+PW+Ellis%2C+Dylan+Freedman%2C+Aren+Jansen%2C+Wade+Lawrence%2C+R+Channing+Moore%2C+Manoj+Plakal%2C+and+Marvin+Ritter%2E+2017%2E+Audio+set%3A+An+ontology+and+human%2Dlabeled+dataset+for+audio+events%2E)
23. Hao-Han Guo, Yao Hu, Kun Liu, Fei-Yu Shen, Xu Tang, Yi-Chen Wu, Feng-Long Xie, Kun Xie, and Kai-Tuo Xu. 2024. Fireredtts: A foundation text-to-speech framework for industry-level generative speech applications. arXiv preprint arXiv:2409.03283. — [arXiv:2409.03283](https://arxiv.org/abs/2409.03283) [find](https://scholar.google.com/scholar?q=Hao%2DHan+Guo%2C+Yao+Hu%2C+Kun+Liu%2C+Fei%2DYu+Shen%2C+Xu+Tang%2C+Yi%2DChen+Wu%2C+Feng%2DLong+Xie%2C+Kun+Xie%2C+and+Kai%2DTuo+Xu%2E+2024%2E+Fireredtts%3A+A+foundation+text%2Dto%2Dspeech+framework+for+industry%2Dlevel+generative+speech+app)
24. Yiwei Guo, Chenpeng Du, Xie Chen, and Kai Yu. 2023. Emodiff: Intensity controllable emotional text-to-speech with soft-label guidance. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE. [find](https://scholar.google.com/scholar?q=Yiwei+Guo%2C+Chenpeng+Du%2C+Xie+Chen%2C+and+Kai+Yu%2E+2023%2E+Emodiff%3A+Intensity+controllable+emotional+text%2Dto%2Dspeech+with+soft%2Dlabel+guidance%2E+In+ICASSP+2023%2D2023+IEEE+International+Conference+on+Acoustics%2C+S)
25. Andrey Guzhov, Federico Raue, Jörn Hees, and Andreas Dengel. 2022. Audioclip: Extending clip to image, text and audio. In ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 976–980. IEEE. [find](https://scholar.google.com/scholar?q=Andrey+Guzhov%2C+Federico+Raue%2C+J%C3%B6rn+Hees%2C+and+Andreas+Dengel%2E+2022%2E+Audioclip%3A+Extending+clip+to+image%2C+text+and+audio%2E+In+ICASSP+2022%2D2022+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal)
26. Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. 2021. [Hubert: Self-supervised speech representation learning by masked prediction of hidden units](https://md.research.elamur.ai/free/en/papers/hsu-2021-hubert-self-supervised-speech-representation/). IEEE/ACM transactions on audio, speech, and language processing, 29:3451–3460.
27. Shengpeng Ji, Ziyue Jiang, Hanting Wang, Jialong Zuo, and Zhou Zhao. 2024. Mobilespeech: A fast and high-fidelity framework for mobile zero-shot text-to-speech. arXiv preprint arXiv:2402.09378. [find](https://scholar.google.com/scholar?q=Shengpeng+Ji%2C+Ziyue+Jiang%2C+Hanting+Wang%2C+Jialong+Zuo%2C+and+Zhou+Zhao%2E+2024%2E+Mobilespeech%3A+A+fast+and+high%2Dfidelity+framework+for+mobile+zero%2Dshot+text%2Dto%2Dspeech%2E+arXiv+preprint+arXiv%3A2402%2E09378%2E)
28. Shengpeng Ji, Ziyue Jiang, Wen Wang, Yifu Chen, Minghui Fang, Jialong Zuo, Qian Yang, Xize Cheng, Zehan Wang, Ruiqi Li, and 1 others. 2025. Wavtokenizer: an efficient acoustic discrete codec tokenizer for audio language modeling. arXiv preprint arXiv:2408.16532. — [arXiv:2408.16532](https://arxiv.org/abs/2408.16532) [find](https://scholar.google.com/scholar?q=Shengpeng+Ji%2C+Ziyue+Jiang%2C+Wen+Wang%2C+Yifu+Chen%2C+Minghui+Fang%2C+Jialong+Zuo%2C+Qian+Yang%2C+Xize+Cheng%2C+Zehan+Wang%2C+Ruiqi+Li%2C+and+1+others%2E+2025%2E+Wavtokenizer%3A+an+efficient+acoustic+discrete+codec+tokenizer)
29. Zeqian Ju, Yuancheng Wang, Kai Shen, Xu Tan, Detai Xin, Dongchao Yang, Yanqing Liu, Yichong Leng, Kaitao Song, Siliang Tang, and 1 others. 2024. Naturalspeech 3: Zero-shot speech synthesis with factorized codec and diffusion models. arXiv preprint arXiv:2403.03100. — [arXiv:2403.03100](https://arxiv.org/abs/2403.03100) [find](https://scholar.google.com/scholar?q=Zeqian+Ju%2C+Yuancheng+Wang%2C+Kai+Shen%2C+Xu+Tan%2C+Detai+Xin%2C+Dongchao+Yang%2C+Yanqing+Liu%2C+Yichong+Leng%2C+Kaitao+Song%2C+Siliang+Tang%2C+and+1+others%2E+2024%2E+Naturalspeech+3%3A+Zero%2Dshot+speech+synthesis+with+factor)
30. Jungil Kong, Jaehyeon Kim, and Jaekyoung Bae. 2020. Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis. Advances in neural information processing systems, 33:17022– 17033. [find](https://scholar.google.com/scholar?q=Jungil+Kong%2C+Jaehyeon+Kim%2C+and+Jaekyoung+Bae%2E+2020%2E+Hifi%2Dgan%3A+Generative+adversarial+networks+for+efficient+and+high+fidelity+speech+synthesis%2E+Advances+in+neural+information+processing+systems%2C+33%3A17)
31. Rithesh Kumar, Prem Seetharaman, Alejandro Luebs, Ishaan Kumar, and Kundan Kumar. 2023. High-fidelity audio compression with improved rvqgan. Advances in Neural Information Processing Systems, 36:27980–27993. [find](https://scholar.google.com/scholar?q=Rithesh+Kumar%2C+Prem+Seetharaman%2C+Alejandro+Luebs%2C+Ishaan+Kumar%2C+and+Kundan+Kumar%2E+2023%2E+High%2Dfidelity+audio+compression+with+improved+rvqgan%2E+Advances+in+Neural+Information+Processing+Systems%2C+36%3A2798)
32. Reza Lotfian and Carlos Busso. 2017. Building naturalistic emotionally balanced speech corpus by retrieving emotional speech from existing podcast recordings. IEEE Transactions on Affective Computing, 10(4):471–483. [find](https://scholar.google.com/scholar?q=Reza+Lotfian+and+Carlos+Busso%2E+2017%2E+Building+naturalistic+emotionally+balanced+speech+corpus+by+retrieving+emotional+speech+from+existing+podcast+recordings%2E+IEEE+Transactions+on+Affective+Computing%2C)
33. Ziyang Ma, Zhisheng Zheng, Jiaxin Ye, Jinchao Li, Zhifu Gao, Shiliang Zhang, and Xie Chen. 2024. emotion2vec: Self-supervised pre-training for speech emotion representation. In Findings of the Association for Computational Linguistics: ACL 2024, pages 15747–15760. [find](https://scholar.google.com/scholar?q=Ziyang+Ma%2C+Zhisheng+Zheng%2C+Jiaxin+Ye%2C+Jinchao+Li%2C+Zhifu+Gao%2C+Shiliang+Zhang%2C+and+Xie+Chen%2E+2024%2E+emotion2vec%3A+Self%2Dsupervised+pre%2Dtraining+for+speech+emotion+representation%2E+In+Findings+of+the+Associa)
34. Andrew Cameron Morris, Viktoria Maier, and Phil D Green. 2004. From wer and ril to mer and wil: improved evaluation measures for connected speech recognition. In Interspeech, pages 2765–2768. [find](https://scholar.google.com/scholar?q=Andrew+Cameron+Morris%2C+Viktoria+Maier%2C+and+Phil+D+Green%2E+2004%2E+From+wer+and+ril+to+mer+and+wil%3A+improved+evaluation+measures+for+connected+speech+recognition%2E+In+Interspeech%2C+pages+2765%E2%80%932768%2E)
35. Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. 2015. Librispeech: an asr corpus based on public domain audio books. In 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE. [find](https://scholar.google.com/scholar?q=Vassil+Panayotov%2C+Guoguo+Chen%2C+Daniel+Povey%2C+and+Sanjeev+Khudanpur%2E+2015%2E+Librispeech%3A+an+asr+corpus+based+on+public+domain+audio+books%2E+In+2015+IEEE+international+conference+on+acoustics%2C+speech+and+)
36. Julian D Parker, Anton Smirnov, Jordi Pons, CJ Carr, Zack Zukowski, Zach Evans, and Xubo Liu. 2024. Scaling transformers for low-bitrate high-quality speech coding. arXiv preprint arXiv:2411.19842. — [arXiv:2411.19842](https://arxiv.org/abs/2411.19842) [find](https://scholar.google.com/scholar?q=Julian+D+Parker%2C+Anton+Smirnov%2C+Jordi+Pons%2C+CJ+Carr%2C+Zack+Zukowski%2C+Zach+Evans%2C+and+Xubo+Liu%2E+2024%2E+Scaling+transformers+for+low%2Dbitrate+high%2Dquality+speech+coding%2E+arXiv+preprint+arXiv%3A2411%2E19842%2E+%E2%80%94+)
37. Tianyi Peng and Yang Xiao. 2025. Dark experience for incremental keyword spotting. In Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). [find](https://scholar.google.com/scholar?q=Tianyi+Peng+and+Yang+Xiao%2E+2025%2E+Dark+experience+for+incremental+keyword+spotting%2E+In+Proc%2E+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Processing+%28ICASSP%29%2E)
38. Jean-Marc Valin, Gregory Maxwell, Timothy B Terriberry, and Koen Vos. 2016. High-quality, low-delay music coding in the opus codec. arXiv preprint arXiv:1602.04845. — [arXiv:1602.04845](https://arxiv.org/abs/1602.04845) [find](https://scholar.google.com/scholar?q=Jean%2DMarc+Valin%2C+Gregory+Maxwell%2C+Timothy+B+Terriberry%2C+and+Koen+Vos%2E+2016%2E+High%2Dquality%2C+low%2Ddelay+music+coding+in+the+opus+codec%2E+arXiv+preprint+arXiv%3A1602%2E04845%2E+%E2%80%94+%5BarXiv%3A1602%2E04845%5D%28https%3A%2F%2Farxiv%2E)
39. Wenze Ren, Yi-Cheng Lin, Huang-Cheng Chou, Haibin Wu, Yi-Chiao Wu, Chi-Chun Lee, Hung-yi Lee, Hsin-Min Wang, and Yu Tsao. 2024. Emo-codec: An in-depth look at emotion preservation capacity of legacy and neural codec models with subjective and objective evaluations. In 2024 Asia Pacific Signal and In… [find](https://scholar.google.com/scholar?q=Wenze+Ren%2C+Yi%2DCheng+Lin%2C+Huang%2DCheng+Chou%2C+Haibin+Wu%2C+Yi%2DChiao+Wu%2C+Chi%2DChun+Lee%2C+Hung%2Dyi+Lee%2C+Hsin%2DMin+Wang%2C+and+Yu+Tsao%2E+2024%2E+Emo%2Dcodec%3A+An+in%2Ddepth+look+at+emotion+preservation+capacity+of+legacy+a)
40. Aaron Van Den Oord, Oriol Vinyals, and 1 others. 2017a. Neural discrete representation learning. Advances in neural information processing systems, 30. [find](https://scholar.google.com/scholar?q=Aaron+Van+Den+Oord%2C+Oriol+Vinyals%2C+and+1+others%2E+2017a%2E+Neural+discrete+representation+learning%2E+Advances+in+neural+information+processing+systems%2C+30%2E)
41. Antony W Rix, Michael P Hollier, Andries P Hekstra, and John G Beerends. 2002. Perceptual evaluation of speech quality (pesq) the new itu standard for end-to-end speech quality assessment part i–time-delay compensation. Journal of the Audio Engineering Society, 50(10):755–764. [find](https://scholar.google.com/scholar?q=Antony+W+Rix%2C+Michael+P+Hollier%2C+Andries+P+Hekstra%2C+and+John+G+Beerends%2E+2002%2E+Perceptual+evaluation+of+speech+quality+%28pesq%29+the+new+itu+standard+for+end%2Dto%2Dend+speech+quality+assessment+part+i%E2%80%93time%2D)
42. Björn Schuller, Stefan Steidl, and Anton Batliner. 2009. The interspeech 2009 emotion challenge. [find](https://scholar.google.com/scholar?q=Bj%C3%B6rn+Schuller%2C+Stefan+Steidl%2C+and+Anton+Batliner%2E+2009%2E+The+interspeech+2009+emotion+challenge%2E)
43. Roneel V Sharan, Cecilia Mascolo, and Björn W Schuller. 2024. Emotion recognition from speech signals by mel-spectrogram and a cnn-rnn. In 2024 46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pages 1–4. IEEE. [find](https://scholar.google.com/scholar?q=Roneel+V+Sharan%2C+Cecilia+Mascolo%2C+and+Bj%C3%B6rn+W+Schuller%2E+2024%2E+Emotion+recognition+from+speech+signals+by+mel%2Dspectrogram+and+a+cnn%2Drnn%2E+In+2024+46th+Annual+International+Conference+of+the+IEEE+Enginee)
44. Jiacheng Shi, Hongfei Du, Yangfan He, Y Alicia Hong, and Ye Gao. 2025a. Emotion-aligned generation in diffusion text to speech models via preference-guided optimization. arXiv preprint arXiv:2509.25416. — [arXiv:2509.25416](https://arxiv.org/abs/2509.25416) [find](https://scholar.google.com/scholar?q=Jiacheng+Shi%2C+Hongfei+Du%2C+Yangfan+He%2C+Y+Alicia+Hong%2C+and+Ye+Gao%2E+2025a%2E+Emotion%2Daligned+generation+in+diffusion+text+to+speech+models+via+preference%2Dguided+optimization%2E+arXiv+preprint+arXiv%3A2509%2E2541)
45. Jiacheng Shi, Hongfei Du, Y Alicia Hong, and Ye Gao. 2025b. Plug-and-play emotion graphs for compositional prompting in zero-shot speech emotion recognition. arXiv preprint arXiv:2509.25458. — [arXiv:2509.25458](https://arxiv.org/abs/2509.25458) [find](https://scholar.google.com/scholar?q=Jiacheng+Shi%2C+Hongfei+Du%2C+Y+Alicia+Hong%2C+and+Ye+Gao%2E+2025b%2E+Plug%2Dand%2Dplay+emotion+graphs+for+compositional+prompting+in+zero%2Dshot+speech+emotion+recognition%2E+arXiv+preprint+arXiv%3A2509%2E25458%2E+%E2%80%94+%5BarXiv%3A)
46. Jiacheng Shi, Yanfu Zhang, and Ye Gao. 2025c. Clep-dg: Contrastive learning for speech emotion domain generalization via soft prompt tuning. arXiv preprint arXiv:2507.04048. — [arXiv:2507.04048](https://arxiv.org/abs/2507.04048) [find](https://scholar.google.com/scholar?q=Jiacheng+Shi%2C+Yanfu+Zhang%2C+and+Ye+Gao%2E+2025c%2E+Clep%2Ddg%3A+Contrastive+learning+for+speech+emotion+domain+generalization+via+soft+prompt+tuning%2E+arXiv+preprint+arXiv%3A2507%2E04048%2E+%E2%80%94+%5BarXiv%3A2507%2E04048%5D%28https)
47. Yao Shi, Hui Bu, Xin Xu, Shaoji Zhang, and Ming Li. 2020. Aishell-3: A multi-speaker mandarin tts corpus and the baselines. arXiv preprint arXiv:2010.11567. [find](https://scholar.google.com/scholar?q=Yao+Shi%2C+Hui+Bu%2C+Xin+Xu%2C+Shaoji+Zhang%2C+and+Ming+Li%2E+2020%2E+Aishell%2D3%3A+A+multi%2Dspeaker+mandarin+tts+corpus+and+the+baselines%2E+arXiv+preprint+arXiv%3A2010%2E11567%2E)
48. Premjeet Singh, Shefali Waldekar, Md Sahidullah, and Goutam Saha. 2022. Analysis of constant-q filterbank based representations for speech emotion recognition. Digital Signal Processing, 130:103712. [find](https://scholar.google.com/scholar?q=Premjeet+Singh%2C+Shefali+Waldekar%2C+Md+Sahidullah%2C+and+Goutam+Saha%2E+2022%2E+Analysis+of+constant%2Dq+filterbank+based+representations+for+speech+emotion+recognition%2E+Digital+Signal+Processing%2C+130%3A103712%2E)
49. Shreya G Upadhyay, Woan-Shiuan Chien, Bo-Hao Su, Lucas Goncalves, Ya-Tse Wu, Ali N Salman, Carlos Busso, and Chi-Chun Lee. 2023. An intelligent infrastructure toward large scale naturalistic affective speech corpora collection. In 2023 11th International Conference on Affective Computing and Intelli… [find](https://scholar.google.com/scholar?q=Shreya+G+Upadhyay%2C+Woan%2DShiuan+Chien%2C+Bo%2DHao+Su%2C+Lucas+Goncalves%2C+Ya%2DTse+Wu%2C+Ali+N+Salman%2C+Carlos+Busso%2C+and+Chi%2DChun+Lee%2E+2023%2E+An+intelligent+infrastructure+toward+large+scale+naturalistic+affective)
50. Aaron Van Den Oord, Oriol Vinyals, and 1 others. 2017b. Neural discrete representation learning. Advances in neural information processing systems, 30. [find](https://scholar.google.com/scholar?q=Aaron+Van+Den+Oord%2C+Oriol+Vinyals%2C+and+1+others%2E+2017b%2E+Neural+discrete+representation+learning%2E+Advances+in+neural+information+processing+systems%2C+30%2E)
51. Dimitrios Ververidis and Constantine Kotropoulos. 2006. Emotional speech recognition: Resources, features, and methods. Speech communication, 48(9):1162–1181. [find](https://scholar.google.com/scholar?q=Dimitrios+Ververidis+and+Constantine+Kotropoulos%2E+2006%2E+Emotional+speech+recognition%3A+Resources%2C+features%2C+and+methods%2E+Speech+communication%2C+48%289%29%3A1162%E2%80%931181%2E)
52. Chen Wang, Minpeng Liao, Zhongqiang Huang, Junhong Wu, Chengqing Zong, and Jiajun Zhang. 2024a. Blsp-emo: Towards empathetic large speech-language models. arXiv preprint arXiv:2406.03872. [find](https://scholar.google.com/scholar?q=Chen+Wang%2C+Minpeng+Liao%2C+Zhongqiang+Huang%2C+Junhong+Wu%2C+Chengqing+Zong%2C+and+Jiajun+Zhang%2E+2024a%2E+Blsp%2Demo%3A+Towards+empathetic+large+speech%2Dlanguage+models%2E+arXiv+preprint+arXiv%3A2406%2E03872%2E)
53. Chengyi Wang, Sanyuan Chen, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, and 1 others. 2023. [Neural codec language models are zero-shot text to speech synthesizers](https://md.research.elamur.ai/free/en/papers/wang-2023-neural-codec-language-models-zero/). arXiv preprint arXiv:2301.02111.
54. Sijie Wang, Rui She, Qiyu Kang, Xingchao Jian, Kai Zhao, Yang Song, and Wee Peng Tay. 2024b. Distil-vpr: Cross-modal knowledge distillation for visual place recognition. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages 10377–10385. [find](https://scholar.google.com/scholar?q=Sijie+Wang%2C+Rui+She%2C+Qiyu+Kang%2C+Xingchao+Jian%2C+Kai+Zhao%2C+Yang+Song%2C+and+Wee+Peng+Tay%2E+2024b%2E+Distil%2Dvpr%3A+Cross%2Dmodal+knowledge+distillation+for+visual+place+recognition%2E+In+Proceedings+of+the+AAAI+con)
55. Xinsheng Wang, Mingqi Jiang, Ziyang Ma, Ziyu Zhang, Songxiang Liu, Linqin Li, Zheng Liang, Qixi Zheng, Rui Wang, Xiaoqin Feng, and 1 others. 2025. Spark-tts: An efficient llm-based text-to-speech model with single-stream decoupled speech tokens. arXiv preprint arXiv:2503.01710. — [arXiv:2503.01710](https://arxiv.org/abs/2503.01710) [find](https://scholar.google.com/scholar?q=Xinsheng+Wang%2C+Mingqi+Jiang%2C+Ziyang+Ma%2C+Ziyu+Zhang%2C+Songxiang+Liu%2C+Linqin+Li%2C+Zheng+Liang%2C+Qixi+Zheng%2C+Rui+Wang%2C+Xiaoqin+Feng%2C+and+1+others%2E+2025%2E+Spark%2Dtts%3A+An+efficient+llm%2Dbased+text%2Dto%2Dspeech+mode)
56. Yuancheng Wang, Haoyue Zhan, Liwei Liu, Ruihong Zeng, Haotian Guo, Jiachen Zheng, Qiang Zhang, Xueyao Zhang, Shunsi Zhang, and Zhizheng Wu. 2024c. Maskgct: Zero-shot text-to-speech with masked generative codec transformer. arXiv preprint arXiv:2409.00750. — [arXiv:2409.00750](https://arxiv.org/abs/2409.00750) [find](https://scholar.google.com/scholar?q=Yuancheng+Wang%2C+Haoyue+Zhan%2C+Liwei+Liu%2C+Ruihong+Zeng%2C+Haotian+Guo%2C+Jiachen+Zheng%2C+Qiang+Zhang%2C+Xueyao+Zhang%2C+Shunsi+Zhang%2C+and+Zhizheng+Wu%2E+2024c%2E+Maskgct%3A+Zero%2Dshot+text%2Dto%2Dspeech+with+masked+generat)
57. Haibin Wu, Huang-Cheng Chou, Kai-Wei Chang, Lucas Goncalves, Jiawei Du, Jyh-Shing Roger Jang, Chi-Chun Lee, and Hung-Yi Lee. 2024a. Emo-superb: An in-depth look at speech emotion recognition. arXiv preprint arXiv:2402.13018. — [arXiv:2402.13018](https://arxiv.org/abs/2402.13018) [find](https://scholar.google.com/scholar?q=Haibin+Wu%2C+Huang%2DCheng+Chou%2C+Kai%2DWei+Chang%2C+Lucas+Goncalves%2C+Jiawei+Du%2C+Jyh%2DShing+Roger+Jang%2C+Chi%2DChun+Lee%2C+and+Hung%2DYi+Lee%2E+2024a%2E+Emo%2Dsuperb%3A+An+in%2Ddepth+look+at+speech+emotion+recognition%2E+arXiv+pr)
58. Haibin Wu, Ho-Lam Chung, Yi-Cheng Lin, Yuan-Kuei Wu, Xuanjun Chen, Yu-Chi Pai, Hsiu-Hsuan Wang, Kai-Wei Chang, Alex Liu, and Hung-yi Lee. 2024b. Codec-superb: An in-depth analysis of sound codec models. In Findings of the Association for Computational Linguistics: ACL 2024, pages 10330–10348. [find](https://scholar.google.com/scholar?q=Haibin+Wu%2C+Ho%2DLam+Chung%2C+Yi%2DCheng+Lin%2C+Yuan%2DKuei+Wu%2C+Xuanjun+Chen%2C+Yu%2DChi+Pai%2C+Hsiu%2DHsuan+Wang%2C+Kai%2DWei+Chang%2C+Alex+Liu%2C+and+Hung%2Dyi+Lee%2E+2024b%2E+Codec%2Dsuperb%3A+An+in%2Ddepth+analysis+of+sound+codec+model)
59. Haibin Wu, Xiaofei Wang, Sefik Emre Eskimez, Manthan Thakker, Daniel Tompkins, Chung-Hsien Tsai, Canrun Li, Zhen Xiao, Sheng Zhao, Jinyu Li, and 1 others. 2024c. Laugh now cry later: Controlling time-varying emotional states of flow-matching-based zero-shot text-to-speech. In 2024 IEEE Spoken Langua… [find](https://scholar.google.com/scholar?q=Haibin+Wu%2C+Xiaofei+Wang%2C+Sefik+Emre+Eskimez%2C+Manthan+Thakker%2C+Daniel+Tompkins%2C+Chung%2DHsien+Tsai%2C+Canrun+Li%2C+Zhen+Xiao%2C+Sheng+Zhao%2C+Jinyu+Li%2C+and+1+others%2E+2024c%2E+Laugh+now+cry+later%3A+Controlling+time%2D)
60. Pengfei Wu, Zhenhua Ling, Lijuan Liu, Yuan Jiang, Hongchuan Wu, and Lirong Dai. 2019. End-to-end emotional speech synthesis using style tokens and semi-supervised training. In 2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). IEEE. [find](https://scholar.google.com/scholar?q=Pengfei+Wu%2C+Zhenhua+Ling%2C+Lijuan+Liu%2C+Yuan+Jiang%2C+Hongchuan+Wu%2C+and+Lirong+Dai%2E+2019%2E+End%2Dto%2Dend+emotional+speech+synthesis+using+style+tokens+and+semi%2Dsupervised+training%2E+In+2019+Asia%2DPacific+Signal)
61. Yi-Chiao Wu, Israel D Gebru, Dejan Marković, and Alexander Richard. 2023a. Audiodec: An open-source streaming high-fidelity neural audio codec. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE. [find](https://scholar.google.com/scholar?q=Yi%2DChiao+Wu%2C+Israel+D+Gebru%2C+Dejan+Markovic%CC%81%2C+and+Alexander+Richard%2E+2023a%2E+Audiodec%3A+An+open%2Dsource+streaming+high%2Dfidelity+neural+audio+codec%2E+In+ICASSP+2023%2D2023+IEEE+International+Conference+on+Ac)
62. Yusong Wu, Ke Chen, Tianyu Zhang, Yuchen Hui, Taylor Berg-Kirkpatrick, and Shlomo Dubnov. 2023b. Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (I… [find](https://scholar.google.com/scholar?q=Yusong+Wu%2C+Ke+Chen%2C+Tianyu+Zhang%2C+Yuchen+Hui%2C+Taylor+Berg%2DKirkpatrick%2C+and+Shlomo+Dubnov%2E+2023b%2E+Large%2Dscale+contrastive+language%2Daudio+pretraining+with+feature+fusion+and+keyword%2Dto%2Dcaption+augmentat)
63. Yang Xiao and Rohan Kumar Das. 2024. Where’s that voice coming? Continual learning for sound source localization. In Proc. IEEE International Conference on Multimedia and Expo (ICME). [find](https://scholar.google.com/scholar?q=Yang+Xiao+and+Rohan+Kumar+Das%2E+2024%2E+Where%E2%80%99s+that+voice+coming%3F+Continual+learning+for+sound+source+localization%2E+In+Proc%2E+IEEE+International+Conference+on+Multimedia+and+Expo+%28ICME%29%2E)
64. Yang Xiao and Rohan Kumar Das. 2025. UCIL: An unsupervised class incremental learning approach for sound event detection. In Proc. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). [find](https://scholar.google.com/scholar?q=Yang+Xiao+and+Rohan+Kumar+Das%2E+2025%2E+UCIL%3A+An+unsupervised+class+incremental+learning+approach+for+sound+event+detection%2E+In+Proc%2E+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Process)
65. Yang Xiao, Han Yin, Jisheng Bai, and Rohan Kumar Das. 2025. Dg-sed: Domain generalization for sound event detection with heterogeneous training data. In 2025 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). [find](https://scholar.google.com/scholar?q=Yang+Xiao%2C+Han+Yin%2C+Jisheng+Bai%2C+and+Rohan+Kumar+Das%2E+2025%2E+Dg%2Dsed%3A+Domain+generalization+for+sound+event+detection+with+heterogeneous+training+data%2E+In+2025+Asia%2DPacific+Signal+and+Information+Proces)
66. Detai Xin, Xu Tan, Shinnosuke Takamichi, and Hiroshi Saruwatari. 2024a. Bigcodec: Pushing the limits of low-bitrate neural speech codec. arXiv preprint arXiv:2409.05377. — [arXiv:2409.05377](https://arxiv.org/abs/2409.05377) [find](https://scholar.google.com/scholar?q=Detai+Xin%2C+Xu+Tan%2C+Shinnosuke+Takamichi%2C+and+Hiroshi+Saruwatari%2E+2024a%2E+Bigcodec%3A+Pushing+the+limits+of+low%2Dbitrate+neural+speech+codec%2E+arXiv+preprint+arXiv%3A2409%2E05377%2E+%E2%80%94+%5BarXiv%3A2409%2E05377%5D%28https%3A%2F%2Fa)
67. Z Xin, Z Dong, L Shimin, Z Yaqian, and Q Xipeng. 2024b. Speechtokenizer: Unified speech tokenizer for speech language models. In Proc. Int. Conf. Learn. Representations, pages 1–21. [find](https://scholar.google.com/scholar?q=Z+Xin%2C+Z+Dong%2C+L+Shimin%2C+Z+Yaqian%2C+and+Q+Xipeng%2E+2024b%2E+Speechtokenizer%3A+Unified+speech+tokenizer+for+speech+language+models%2E+In+Proc%2E+Int%2E+Conf%2E+Learn%2E+Representations%2C+pages+1%E2%80%9321%2E)
68. Junichi Yamagishi, Christophe Veaux, and Kirsten Mac-Donald. 2019. Cstr vctk corpus: English multi-speaker corpus for cstr voice cloning toolkit (version 0.92). The Rainbow Passage which the speakers read out can be found in the International Dialects of English Archive:(http://web. ku. edu/˜ idea/r… [find](https://scholar.google.com/scholar?q=Junichi+Yamagishi%2C+Christophe+Veaux%2C+and+Kirsten+Mac%2DDonald%2E+2019%2E+Cstr+vctk+corpus%3A+English+multi%2Dspeaker+corpus+for+cstr+voice+cloning+toolkit+%28version+0%2E92%29%2E+The+Rainbow+Passage+which+the+speakers+)
69. Dongchao Yang, Songxiang Liu, Rongjie Huang, Jinchuan Tian, Chao Weng, and Yuexian Zou. 2023. Hifi-codec: Group-residual vector quantization for high fidelity audio codec. arXiv preprint arXiv:2305.02765. [find](https://scholar.google.com/scholar?q=Dongchao+Yang%2C+Songxiang+Liu%2C+Rongjie+Huang%2C+Jinchuan+Tian%2C+Chao+Weng%2C+and+Yuexian+Zou%2E+2023%2E+Hifi%2Dcodec%3A+Group%2Dresidual+vector+quantization+for+high+fidelity+audio+codec%2E+arXiv+preprint+arXiv%3A2305%2E02)
70. Guanrou Yang, Chen Yang, Qian Chen, Ziyang Ma, Wenxi Chen, Wen Wang, Tianrui Wang, Yifan Yang, Zhikang Niu, Wenrui Liu, and 1 others. 2025. Emovoice: Llm-based emotional text-to-speech model with freestyle text prompting. In Proceedings of the 33rd ACM International Conference on Multimedia, pages 1… [find](https://scholar.google.com/scholar?q=Guanrou+Yang%2C+Chen+Yang%2C+Qian+Chen%2C+Ziyang+Ma%2C+Wenxi+Chen%2C+Wen+Wang%2C+Tianrui+Wang%2C+Yifan+Yang%2C+Zhikang+Niu%2C+Wenrui+Liu%2C+and+1+others%2E+2025%2E+Emovoice%3A+Llm%2Dbased+emotional+text%2Dto%2Dspeech+model+with+free)
71. Zhen Ye, Peiwen Sun, Jiahe Lei, Hongzhan Lin, Xu Tan, Zheqi Dai, Qiuqiang Kong, Jianyi Chen, Jiahao Pan, Qifeng Liu, and 1 others. 2025a. Codec does matter: Exploring the semantic shortcoming of codec for audio language model. In Proceedings of the AAAI Conference on Artificial Intelligence, volume… [find](https://scholar.google.com/scholar?q=Zhen+Ye%2C+Peiwen+Sun%2C+Jiahe+Lei%2C+Hongzhan+Lin%2C+Xu+Tan%2C+Zheqi+Dai%2C+Qiuqiang+Kong%2C+Jianyi+Chen%2C+Jiahao+Pan%2C+Qifeng+Liu%2C+and+1+others%2E+2025a%2E+Codec+does+matter%3A+Exploring+the+semantic+shortcoming+of+codec)
72. Zhen Ye, Xinfa Zhu, Chi-Min Chan, Xinsheng Wang, Xu Tan, Jiahe Lei, Yi Peng, Haohe Liu, Yizhu Jin, Zheqi Dai, and 1 others. 2025b. Llasa: Scaling train-time and inference-time compute for llama-based speech synthesis. arXiv preprint arXiv:2502.04128. — [arXiv:2502.04128](https://arxiv.org/abs/2502.04128) [find](https://scholar.google.com/scholar?q=Zhen+Ye%2C+Xinfa+Zhu%2C+Chi%2DMin+Chan%2C+Xinsheng+Wang%2C+Xu+Tan%2C+Jiahe+Lei%2C+Yi+Peng%2C+Haohe+Liu%2C+Yizhu+Jin%2C+Zheqi+Dai%2C+and+1+others%2E+2025b%2E+Llasa%3A+Scaling+train%2Dtime+and+inference%2Dtime+compute+for+llama%2Dbased+)
73. Han Yin, Yang Xiao, Rohan Kumar Das, Jisheng Bai, Haohe Liu, Wenwu Wang, and Mark D Plumbley. 2025. EnvSDD: Benchmarking environmental sound deepfake detection. In Proc. Interspeech, pages 201– 205. [find](https://scholar.google.com/scholar?q=Han+Yin%2C+Yang+Xiao%2C+Rohan+Kumar+Das%2C+Jisheng+Bai%2C+Haohe+Liu%2C+Wenwu+Wang%2C+and+Mark+D+Plumbley%2E+2025%2E+EnvSDD%3A+Benchmarking+environmental+sound+deepfake+detection%2E+In+Proc%2E+Interspeech%2C+pages+201%E2%80%93+205%2E)
74. Neil Zeghidour, Alejandro Luebs, Ahmed Omran, Jan Skoglund, and Marco Tagliasacchi. 2021. [Soundstream: An end-to-end neural audio codec](https://md.research.elamur.ai/free/en/papers/zeghidour-2021-soundstream-end-end-neural-audio/). IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30:495–507.
75. Heiga Zen, Viet Dang, Rob Clark, Yu Zhang, Ron J Weiss, Ye Jia, Zhifeng Chen, and Yonghui Wu. 2019. [Libritts: A corpus derived from librispeech for text-to-speech](https://md.research.elamur.ai/free/en/papers/zen-2019-libritts-corpus-derived-librispeech-text/). arXiv preprint arXiv:1904.02882.
76. Aohan Zeng, Zhengxiao Du, Mingdao Liu, Kedong Wang, Shengmin Jiang, Lei Zhao, Yuxiao Dong, and Jie Tang. 2024. [Glm-4-voice: Towards intelligent and human-like end-to-end spoken chatbot](https://md.research.elamur.ai/free/en/papers/zeng-2024-glm-4-voice-intelligent/). arXiv preprint arXiv:2412.02612.
77. Yaoxun Xu, Hangting Chen, Jianwei Yu, Qiaochu Huang, Zhiyong Wu, Shi-Xiong Zhang, Guangzhi Li, Yi Luo, and Rongzhi Gu. 2024. Secap: Speech emotion captioning with large language model. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 19323–19331. [find](https://scholar.google.com/scholar?q=Yaoxun+Xu%2C+Hangting+Chen%2C+Jianwei+Yu%2C+Qiaochu+Huang%2C+Zhiyong+Wu%2C+Shi%2DXiong+Zhang%2C+Guangzhi+Li%2C+Yi+Luo%2C+and+Rongzhi+Gu%2E+2024%2E+Secap%3A+Speech+emotion+captioning+with+large+language+model%2E+In+Proceedings+)

*Updated 1 November 2026*