---
title: "CosyVoice — paper by Du et al. (2024) — summary and ideas"
description: "LLM-based zero-shot text-to-speech systems usually rely on speech tokens learned… 2.89% WER. 32 references, cited by 89 papers here."
authors: ["Zhihao Du", "Qian Chen", "Shiliang Zhang", "Kai Hu", "Heng Lu", "Yexin Yang", "Hangrui Hu", "Siqi Zheng", "Yue Gu", "Ziyang Ma", "Zhifu Gao", "Zhijie Yan"]
year: 2024
venue: "arXiv:2407.05407"
source: "https://arxiv.org/abs/2407.05407"
doi: "10.48550/arxiv.2407.05407"
arxiv: "2407.05407"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/du-2024-cosyvoice-scalable-multilingual-zero-shot/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/du-2024-cosyvoice-scalable-multilingual-zero-shot/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/4/) / CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokens

# CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokens

**Authors:** [Zhihao Du](https://md.research.elamur.ai/free/en/authors/zhihao-du/), [Qian Chen](https://md.research.elamur.ai/free/en/authors/qian-chen/), [Shiliang Zhang](https://md.research.elamur.ai/free/en/authors/shiliang-zhang/), Kai Hu, [Heng Lu](https://md.research.elamur.ai/free/en/authors/heng-lu/), [Yexin Yang](https://md.research.elamur.ai/free/en/authors/yexin-yang/), [Hangrui Hu](https://md.research.elamur.ai/free/en/authors/hangrui-hu/), [Siqi Zheng](https://md.research.elamur.ai/free/en/authors/siqi-zheng/), [Yue Gu](https://md.research.elamur.ai/free/en/authors/yue-gu/), [Ziyang Ma](https://md.research.elamur.ai/free/en/authors/ziyang-ma/), [Zhifu Gao](https://md.research.elamur.ai/free/en/authors/zhifu-gao/) and [Zhijie Yan](https://md.research.elamur.ai/free/en/authors/zhijie-yan/) · **Year:** 2024 · **Venue:** arXiv:2407.05407 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2407.05407) · [PDF](https://arxiv.org/pdf/2407.05407) · [DOI 10.48550/arxiv.2407.05407](https://doi.org/10.48550/arxiv.2407.05407) · [arXiv 2407.05407](https://arxiv.org/abs/2407.05407) · [OpenAlex](https://openalex.org/W4400518537)

**Citations:** 4 citations (OpenAlex, 2026-10-14) · [89](https://md.research.elamur.ai/free/en/papers/du-2024-cosyvoice-scalable-multilingual-zero-shot/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary

LLM-based zero-shot text-to-speech systems usually rely on speech tokens learned without supervision. Such tokens carry little explicit semantic content and align poorly with text. The authors, from Alibaba's Speech Lab, address this with a supervised semantic speech (S3) tokenizer. They take a multilingual ASR encoder (a finetuned version of their in-house SenseVoice model), split it in two and place a vector quantization layer between the halves, so the discrete tokens are learned under an ASR objective.

On top of these tokens they build CosyVoice. An autoregressive language model turns BPE text encodings and a speaker x-vector into speech tokens. An optimal-transport conditional flow matching model then maps the tokens to Mel spectrograms, which HifiGAN converts to waveforms. The pipeline needs no phonemizer or forced aligner.

The supervised tokens outperform unsupervised ones in content consistency and speaker similarity, and adding training data keeps improving quality. Generated English and Chinese speech comes close to the real recordings. Models and code are publicly released, which makes CosyVoice a practical multilingual TTS baseline.

## Key points

- **Architecture:** the S3 tokenizer quantizes the encoder output with a codebook updated by EMA. The flow matching model uses classifier-free guidance (strength β = 0.7), a cosine scheduler and masked conditions.
- **English results (LibriTTS test-clean, Whisper-Large V3):**
  - CosyVoice reaches 2.89% WER against 2.66% for the original recordings and 8.32% for ChatTTS.
  - Speaker similarity is 74.30, compared with 69.67 for the originals.
  - ASR re-ranking over 5 samples lowers WER to 1.51%.
- **Chinese results (AISHELL-3, Paraformer-zh):**
  - CER is 3.82%, falling to 1.84% with re-ranking.
  - Speaker similarity is 81.58, compared with 74.15 for the originals.
  - CosyVoice makes far fewer insertion and deletion errors than ChatTTS.
- **Synthetic ASR training data (Librispeech):** synthetic speech alone gives results comparable to the 960-hour original. Adding speech synthesized from MLS text helps further, which suggests that text diversity matters a lot for ASR training.
- **Emotion control:** CosyVoice-instruct with style instructions is clearly more accurate than CosyVoice-base on six target emotions. This was measured with emotion2vec on 100 English utterances per emotion.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Alibaba Group](https://md.research.elamur.ai/free/en/companies/alibaba-group/) — Company, Hangzhou, China · 201 works · 17,173 citations

📚 Shelves (7): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🔊 Signal Processing](https://md.research.elamur.ai/free/en/shelves/computer-science/signal-processing/) · 493 works · [🩺 Medicine](https://md.research.elamur.ai/free/en/shelves/medicine/) · 275 works · [🩺 Physiology](https://md.research.elamur.ai/free/en/shelves/medicine/physiology/) · 35 works

🧭 Topics (2): [🌍 Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) · 882 works · [🎙️ Speech & spoken language](https://md.research.elamur.ai/free/en/topics/speech/) · 519 works

## Cite

Cite as: Zhihao Du, Qian Chen, Shiliang Zhang, Kai Hu, Heng Lu, Yexin Yang, Hangrui Hu, Siqi Zheng, Yue Gu, Ziyang Ma, Zhifu Gao and Zhijie Yan. 2024. CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokens. arXiv:2407.05407.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{du2024cosyvoice,
  title = {{CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokens}},
  author = {Zhihao Du, Qian Chen, Shiliang Zhang, Kai Hu, Heng Lu, Yexin Yang, Hangrui Hu, Siqi Zheng, Yue Gu, Ziyang Ma, Zhifu Gao and Zhijie Yan},
  year = {2024},
  doi = {10.48550/arxiv.2407.05407},
  eprint = {2407.05407},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2407.05407}
}
```

- [← previous Retrieval Head Mechanistically Explains Long-Context Factuality](https://md.research.elamur.ai/free/en/papers/wu-2024-retrieval-head-mechanistically-explains-long/)
- [next → CultureBank: An Online Community-Driven Knowledge Base Towards Culturally Aware Language Technologies](https://md.research.elamur.ai/free/en/papers/shi-2024-culturebank-online-community-driven-knowledge/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/4/)

## References

32 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. James Betker. 2023. Better speech synthesis through scaling. CoRR, abs/2305.07243. — [arXiv:2305.07243](https://arxiv.org/abs/2305.07243) [find](https://scholar.google.com/scholar?q=James+Betker%2E+2023%2E+Better+speech+synthesis+through+scaling%2E+CoRR%2C+abs%2F2305%2E07243%2E+%E2%80%94+%5BarXiv%3A2305%2E07243%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2305%2E07243%29)
2. Yafeng Chen, Siqi Zheng, Hui Wang, Luyao Cheng, Qian Chen, and Jiajun Qi. 2023. An enhanced res2net with local and global feature fusion for speaker verification. In Interspeech. ISCA. [find](https://scholar.google.com/scholar?q=Yafeng+Chen%2C+Siqi+Zheng%2C+Hui+Wang%2C+Luyao+Cheng%2C+Qian+Chen%2C+and+Jiajun+Qi%2E+2023%2E+An+enhanced+res2net+with+local+and+global+feature+fusion+for+speaker+verification%2E+In+Interspeech%2E+ISCA%2E)
3. Happy Sad CosyVoice-base CosyVoice-instruct w/o instruction 1.00±0.00 1.00±0.00 0.98±0.01 0.45±0.05 0.98±0.02 0.77±0.04 Table 10: Comparison of emotion control accuracy between CosyVoice-base-300M and CosyVoice-instruct-300M. “±” joins the mean and standard deviation for each evaluation metric. Trai… [find](https://scholar.google.com/scholar?q=Happy+Sad+CosyVoice%2Dbase+CosyVoice%2Dinstruct+w%2Fo+instruction+1%2E00%C2%B10%2E00+1%2E00%C2%B10%2E00+0%2E98%C2%B10%2E01+0%2E45%C2%B10%2E05+0%2E98%C2%B10%2E02+0%2E77%C2%B10%2E04+Table+10%3A+Comparison+of+emotion+control+accuracy+between+CosyVoice%2Dbase%2D300M+and)
4. Alexandre Défossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi. 2022. [High fidelity neural audio compression](https://md.research.elamur.ai/free/en/papers/defossez-2022-high-fidelity-neural-audio-compression/). CoRR, abs/2210.13438.
5. Chenpeng Du, Yiwei Guo, Feiyu Shen, Zhijun Liu, Zheng Liang, Xie Chen, Shuai Wang, Hui Zhang, and Kai Yu. 2024. Unicats: A unified context-aware text-to-speech framework with contextual vq-diffusion and vocoding. In AAAI, pages 17924–17932. AAAI Press. [find](https://scholar.google.com/scholar?q=Chenpeng+Du%2C+Yiwei+Guo%2C+Feiyu+Shen%2C+Zhijun+Liu%2C+Zheng+Liang%2C+Xie+Chen%2C+Shuai+Wang%2C+Hui+Zhang%2C+and+Kai+Yu%2E+2024%2E+Unicats%3A+A+unified+context%2Daware+text%2Dto%2Dspeech+framework+with+contextual+vq%2Ddiffusion+a)
6. Zhifu Gao, Shiliang Zhang, Ian McLoughlin, and Zhijie Yan. 2022. Paraformer: Fast and accurate parallel transformer for non-autoregressive end-to-end speech recognition. In Interspeech, pages 2063– 2067. ISCA. [find](https://scholar.google.com/scholar?q=Zhifu+Gao%2C+Shiliang+Zhang%2C+Ian+McLoughlin%2C+and+Zhijie+Yan%2E+2022%2E+Paraformer%3A+Fast+and+accurate+parallel+transformer+for+non%2Dautoregressive+end%2Dto%2Dend+speech+recognition%2E+In+Interspeech%2C+pages+2063%E2%80%93+20)
7. Wenhao Guan, Qi Su, Haodong Zhou, Shiyu Miao, Xingjia Xie, Lin Li, and Qingyang Hong. 2023. Reflow-tts: A rectified flow model for high-fidelity text-to-speech. CoRR, abs/2309.17056. — [arXiv:2309.17056](https://arxiv.org/abs/2309.17056) [find](https://scholar.google.com/scholar?q=Wenhao+Guan%2C+Qi+Su%2C+Haodong+Zhou%2C+Shiyu+Miao%2C+Xingjia+Xie%2C+Lin+Li%2C+and+Qingyang+Hong%2E+2023%2E+Reflow%2Dtts%3A+A+rectified+flow+model+for+high%2Dfidelity+text%2Dto%2Dspeech%2E+CoRR%2C+abs%2F2309%2E17056%2E+%E2%80%94+%5BarXiv%3A2309%2E170)
8. Yiwei Guo, Chenpeng Du, Ziyang Ma, Xie Chen, and Kai Yu. 2023. Voiceflow: Efficient text-to-speech with rectified flow matching. CoRR, abs/2309.05027. — [arXiv:2309.05027](https://arxiv.org/abs/2309.05027) [find](https://scholar.google.com/scholar?q=Yiwei+Guo%2C+Chenpeng+Du%2C+Ziyang+Ma%2C+Xie+Chen%2C+and+Kai+Yu%2E+2023%2E+Voiceflow%3A+Efficient+text%2Dto%2Dspeech+with+rectified+flow+matching%2E+CoRR%2C+abs%2F2309%2E05027%2E+%E2%80%94+%5BarXiv%3A2309%2E05027%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2309%2E0)
9. Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual. [find](https://scholar.google.com/scholar?q=Jonathan+Ho%2C+Ajay+Jain%2C+and+Pieter+Abbeel%2E+2020%2E+Denoising+diffusion+probabilistic+models%2E+In+Advances+in+Neural+Information+Processing+Systems+33%3A+Annual+Conference+on+Neural+Information+Processing+S)
10. Jonathan Ho and Tim Salimans. 2022a. [Classifier-free diffusion guidance](https://md.research.elamur.ai/free/en/papers/ho-2022-classifier-free-diffusion-guidance/). CoRR, abs/2207.12598.
11. Jonathan Ho and Tim Salimans. 2022b. [Classifier-free diffusion guidance](https://md.research.elamur.ai/free/en/papers/ho-2022-classifier-free-diffusion-guidance/). arXiv preprint arXiv:2207.12598.
12. Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. 2021. Hubert: Self-supervised Angry Surprised Fearful Disgusted 0.59±0.03 0.83±0.04 0.49±0.12 0.26±0.02 0.64±0.03 0.28±0.06 0.88±0.01 0.87±0.03 0.83±0.04 0.46±0.06 0.93±0.02 0.45±0.16… [find](https://scholar.google.com/scholar?q=Wei%2DNing+Hsu%2C+Benjamin+Bolte%2C+Yao%2DHung+Hubert+Tsai%2C+Kushal+Lakhotia%2C+Ruslan+Salakhutdinov%2C+and+Abdelrahman+Mohamed%2E+2021%2E+Hubert%3A+Self%2Dsupervised+Angry+Surprised+Fearful+Disgusted+0%2E59%C2%B10%2E03+0%2E83%C2%B10%2E04+)
13. Shengpeng Ji, Jialong Zuo, Minghui Fang, Ziyue Jiang, Feiyang Chen, Xinyu Duan, Baoxing Huai, and Zhou Zhao. 2023. Textrolspeech: A text style control speech corpus with codec language text-to-speech models. CoRR, abs/2308.14430. — [arXiv:2308.14430](https://arxiv.org/abs/2308.14430) [find](https://scholar.google.com/scholar?q=Shengpeng+Ji%2C+Jialong+Zuo%2C+Minghui+Fang%2C+Ziyue+Jiang%2C+Feiyang+Chen%2C+Xinyu+Duan%2C+Baoxing+Huai%2C+and+Zhou+Zhao%2E+2023%2E+Textrolspeech%3A+A+text+style+control+speech+corpus+with+codec+language+text%2Dto%2Dspeech+)
14. Eugene Kharitonov, Damien Vincent, Zalán Borsos, Raphaël Marinier, Sertan Girgin, Olivier Pietquin, Matt Sharifi, Marco Tagliasacchi, and Neil Zeghidour. 2023. [Speak, read and prompt: High-fidelity text-to-speech with minimal supervision](https://md.research.elamur.ai/free/en/papers/kharitonov-2023-speak-read-prompt-high-fidelity/). Trans. Assoc. Comput. Linguistics, 11:1703–1718.
15. Jungil Kong, Jaehyeon Kim, and Jaekyoung Bae. 2020. [Hifi-gan: Generative adversarial networks for efficient and high fidelity speech synthesis](https://md.research.elamur.ai/free/en/papers/kong-2020-hifi-gan-generative-adversarial-networks/). In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020,…
16. Mateusz Lajszczak, Guillermo Cámbara, Yang Li, Fatih Beyhan, Arent van Korlaar, Fan Yang, Arnaud Joly, Álvaro Martín-Cortinas, Ammar Abbas, Adam Michalski, Alexis Moinet, Sri Karlapati, Ewa Muszynska, Haohan Guo, Bartosz Putrycz, Soledad López Gambino, Kayeon Yoo, Elena Sokolova, and Thomas Drugman.… — [arXiv:2402.08093](https://arxiv.org/abs/2402.08093) [find](https://scholar.google.com/scholar?q=Mateusz+Lajszczak%2C+Guillermo+C%C3%A1mbara%2C+Yang+Li%2C+Fatih+Beyhan%2C+Arent+van+Korlaar%2C+Fan+Yang%2C+Arnaud+Joly%2C+%C3%81lvaro+Mart%C3%ADn%2DCortinas%2C+Ammar+Abbas%2C+Adam+Michalski%2C+Alexis+Moinet%2C+Sri+Karlapati%2C+Ewa+Muszynska%2C)
17. Matthew Le, Apoorv Vyas, Bowen Shi, Brian Karrer, Leda Sari, Rashel Moritz, Mary Williamson, Vimal Manohar, Yossi Adi, Jay Mahadeokar, et al. 2024. Voicebox: Text-guided multilingual universal speech generation at scale. Advances in neural information processing systems, 36. [find](https://scholar.google.com/scholar?q=Matthew+Le%2C+Apoorv+Vyas%2C+Bowen+Shi%2C+Brian+Karrer%2C+Leda+Sari%2C+Rashel+Moritz%2C+Mary+Williamson%2C+Vimal+Manohar%2C+Yossi+Adi%2C+Jay+Mahadeokar%2C+et+al%2E+2024%2E+Voicebox%3A+Text%2Dguided+multilingual+universal+speech+)
18. Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. 2023. [Flow matching for generative modeling](https://md.research.elamur.ai/free/en/papers/lipman-2022-flow-matching-generative-modeling/). In ICLR. Open-Review.net.
19. Ziyang Ma, Zhisheng Zheng, Jiaxin Ye, Jinchao Li, Zhifu Gao, Shiliang Zhang, and Xie Chen. 2023. emotion2vec: Self-supervised pre-training for speech emotion representation. CoRR, abs/2312.15185. [find](https://scholar.google.com/scholar?q=Ziyang+Ma%2C+Zhisheng+Zheng%2C+Jiaxin+Ye%2C+Jinchao+Li%2C+Zhifu+Gao%2C+Shiliang+Zhang%2C+and+Xie+Chen%2E+2023%2E+emotion2vec%3A+Self%2Dsupervised+pre%2Dtraining+for+speech+emotion+representation%2E+CoRR%2C+abs%2F2312%2E15185%2E)
20. Shivam Mehta, Ruibo Tu, Jonas Beskow, Éva Székely, and Gustav Eje Henter. 2023. Matcha-tts: A fast TTS architecture with conditional flow matching. CoRR, abs/2309.03199. — [arXiv:2309.03199](https://arxiv.org/abs/2309.03199) [find](https://scholar.google.com/scholar?q=Shivam+Mehta%2C+Ruibo+Tu%2C+Jonas+Beskow%2C+%C3%89va+Sz%C3%A9kely%2C+and+Gustav+Eje+Henter%2E+2023%2E+Matcha%2Dtts%3A+A+fast+TTS+architecture+with+conditional+flow+matching%2E+CoRR%2C+abs%2F2309%2E03199%2E+%E2%80%94+%5BarXiv%3A2309%2E03199%5D%28https%3A%2F%2Fa)
21. Alexander Quinn Nichol and Prafulla Dhariwal. 2021. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning, pages 8162–8171. PMLR. [find](https://scholar.google.com/scholar?q=Alexander+Quinn+Nichol+and+Prafulla+Dhariwal%2E+2021%2E+Improved+denoising+diffusion+probabilistic+models%2E+In+International+Conference+on+Machine+Learning%2C+pages+8162%E2%80%938171%2E+PMLR%2E)
22. Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. 2015. Librispeech: an asr corpus based on public domain audio books. In 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP), pages 5206–5210. IEEE. [find](https://scholar.google.com/scholar?q=Vassil+Panayotov%2C+Guoguo+Chen%2C+Daniel+Povey%2C+and+Sanjeev+Khudanpur%2E+2015%2E+Librispeech%3A+an+asr+corpus+based+on+public+domain+audio+books%2E+In+2015+IEEE+international+conference+on+acoustics%2C+speech+and+)
23. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. [Robust speech recognition via large-scale weak supervision](https://md.research.elamur.ai/free/en/papers/radford-2022-robust-speech-recognition-large-scale/). In International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learn…
24. Paul K. Rubenstein, Chulayuth Asawaroengchai, Duc Dung Nguyen, Ankur Bapna, Zalán Borsos, Félix de Chaumont Quitry, Peter Chen, Dalia El Badawy, Wei Han, Eugene Kharitonov, Hannah Muckenhirn, Dirk Padfield, James Qin, Danny Rozenberg, Tara N. Sainath, Johan Schalkwyk, Matthew Sharifi, Michelle Tadmo… — [AudioPaLM: A Large Language Model That Can Speak and Listen](https://md.research.elamur.ai/free/en/papers/rubenstein-2023-audiopalm-large-language-model-that/)
25. Yao Shi, Hui Bu, Xin Xu, Shaoji Zhang, and Ming Li. 2021. AISHELL-3: A multi-speaker mandarin TTS corpus. In Interspeech, pages 2756–2760. ISCA. [find](https://scholar.google.com/scholar?q=Yao+Shi%2C+Hui+Bu%2C+Xin+Xu%2C+Shaoji+Zhang%2C+and+Ming+Li%2E+2021%2E+AISHELL%2D3%3A+A+multi%2Dspeaker+mandarin+TTS+corpus%2E+In+Interspeech%2C+pages+2756%E2%80%932760%2E+ISCA%2E)
26. David Snyder, Daniel Garcia-Romero, Gregory Sell, Daniel Povey, and Sanjeev Khudanpur. 2018. X-vectors: Robust DNN embeddings for speaker recognition. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2018, Calgary, AB, Canada, April 15-20, 2018, pages 5329–533… [find](https://scholar.google.com/scholar?q=David+Snyder%2C+Daniel+Garcia%2DRomero%2C+Gregory+Sell%2C+Daniel+Povey%2C+and+Sanjeev+Khudanpur%2E+2018%2E+X%2Dvectors%3A+Robust+DNN+embeddings+for+speaker+recognition%2E+In+2018+IEEE+International+Conference+on+Acoustic)
27. Alexander Tong, Nikolay Malkin, Guillaume Huguet, Yanlei Zhang, Jarrid Rector-Brooks, Kilian Fatras, Guy Wolf, and Yoshua Bengio. 2023. Improving and generalizing flow-based generative models with minibatch optimal transport. In ICML Workshop on New Frontiers in Learning, Control, and Dynamical Syst… [find](https://scholar.google.com/scholar?q=Alexander+Tong%2C+Nikolay+Malkin%2C+Guillaume+Huguet%2C+Yanlei+Zhang%2C+Jarrid+Rector%2DBrooks%2C+Kilian+Fatras%2C+Guy+Wolf%2C+and+Yoshua+Bengio%2E+2023%2E+Improving+and+generalizing+flow%2Dbased+generative+models+with+min)
28. Team TongyiSpeech. 2024. Funaudiollm: Voice understanding and generation foundation models for natural interaction between humans and llms. [find](https://scholar.google.com/scholar?q=Team+TongyiSpeech%2E+2024%2E+Funaudiollm%3A+Voice+understanding+and+generation+foundation+models+for+natural+interaction+between+humans+and+llms%2E)
29. Chengyi Wang, Sanyuan Chen, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, Lei He, Sheng Zhao, and Furu Wei. 2023. [Neural codec language models are zero-shot text to speech synthesizers](https://md.research.elamur.ai/free/en/papers/wang-2023-neural-codec-language-models-zero/). CoRR, abs/2301.02111.
30. Dongchao Yang, Jinchuan Tian, Xu Tan, Rongjie Huang, Songxiang Liu, Xuankai Chang, Jiatong Shi, Sheng Zhao, Jiang Bian, Xixin Wu, Zhou Zhao, Shinji Watanabe, and Helen Meng. 2023. Uniaudio: An audio foundation model toward universal audio generation. CoRR, abs/2310.00704. — [arXiv:2310.00704](https://arxiv.org/abs/2310.00704) [find](https://scholar.google.com/scholar?q=Dongchao+Yang%2C+Jinchuan+Tian%2C+Xu+Tan%2C+Rongjie+Huang%2C+Songxiang+Liu%2C+Xuankai+Chang%2C+Jiatong+Shi%2C+Sheng+Zhao%2C+Jiang+Bian%2C+Xixin+Wu%2C+Zhou+Zhao%2C+Shinji+Watanabe%2C+and+Helen+Meng%2E+2023%2E+Uniaudio%3A+An+audio+f)
31. Lingxuan Ye, Changfeng Gao, Gaofeng Cheng, Liuping Luo, and Qingwei Zhao. 2024. ASQ: an ultra-low bit rate asr-oriented speech quantization method. IEEE Signal Process. Lett., 31:221–225. [find](https://scholar.google.com/scholar?q=Lingxuan+Ye%2C+Changfeng+Gao%2C+Gaofeng+Cheng%2C+Liuping+Luo%2C+and+Qingwei+Zhao%2E+2024%2E+ASQ%3A+an+ultra%2Dlow+bit+rate+asr%2Doriented+speech+quantization+method%2E+IEEE+Signal+Process%2E+Lett%2E%2C+31%3A221%E2%80%93225%2E)
32. Heiga Zen, Viet Dang, Rob Clark, and et al. 2019. [Libritts: A corpus derived from librispeech for text-to-speech](https://md.research.elamur.ai/free/en/papers/zen-2019-libritts-corpus-derived-librispeech-text/). arXiv:1904.02882.

*Updated 2 November 2026*