---
title: "ZipVoice-Dialog — paper by Zhu et al. (2026)"
description: "The authors present ZipVoice-Dialog, a non-autoregressive zero-shot model built on flow matching and… 54 references, cited by 2 papers here."
authors: ["Han Zhu", "Wei Kang", "Liyong Guo", "Zengwei Yao", "Fangjun Kuang", "Weiji Zhuang", "Zhaoqing Li", "Zhifeng Han", "Dong Zhang", "Xin Zhang", "Xingchen Song", "Lingxuan Ye", "Long Lin", "Daniel Povey"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.1928/"
doi: "10.18653/v1/2026.findings-acl.1928"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhu-2026-zipvoice-dialog-non-autoregressive-spoken/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhu-2026-zipvoice-dialog-non-autoregressive-spoken/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/6/) / ZipVoice-Dialog: Non-Autoregressive Spoken Dialogue Generation with Flow Matching

# ZipVoice-Dialog: Non-Autoregressive Spoken Dialogue Generation with Flow Matching

**Authors:** [Han Zhu](https://md.research.elamur.ai/free/en/authors/han-zhu/), Wei Kang, Liyong Guo, Zengwei Yao, Fangjun Kuang, [Weiji Zhuang](https://md.research.elamur.ai/free/en/authors/weiji-zhuang/), Zhaoqing Li, Zhifeng Han, [Dong Zhang](https://md.research.elamur.ai/free/en/authors/dong-zhang/), [Xin Zhang](https://md.research.elamur.ai/free/en/authors/xin-zhang/), Xingchen Song, Lingxuan Ye, Long Lin and [Daniel Povey](https://md.research.elamur.ai/free/en/authors/daniel-povey/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.1928/) · [PDF](https://aclanthology.org/2026.findings-acl.1928.pdf) · [DOI 10.18653/v1/2026.findings-acl.1928](https://doi.org/10.18653/v1/2026.findings-acl.1928) · [OpenAlex](https://openalex.org/W4414691940)

**Citations:** 1 citations (OpenAlex, 2026-10-14) · [2](https://md.research.elamur.ai/free/en/papers/zhu-2026-zipvoice-dialog-non-autoregressive-spoken/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Synthesizing two-speaker spoken conversations is harder than single-speaker TTS. A model must switch voices at the right moments and learn text-to-speech alignment across different timbres. Existing dialogue systems are mostly autoregressive, which makes them slow and prone to skipped or repeated words. The authors present ZipVoice-Dialog, a non-autoregressive zero-shot model built on flow matching and initialized from the monologue model ZipVoice (Zipformer backbone, Vocos vocoder).

Plain flow matching trained on dialogues fails to align speech with text. Two fixes address this:
- **Curriculum training:** the model first learns from monologue data and is then fine-tuned on dialogues.
- **Speaker-turn embeddings:** two learnable embeddings added to the text condition tell the model which speaker is talking.

Extra techniques extend the model to stereo output. The team also releases OpenDialog, a 6.8k-hour open dialogue corpus mined from in-the-wild audio, and a reproducible evaluation benchmark. The compact model beats MoonCast and Dia in speed, intelligibility, turn accuracy and speaker similarity. This makes fast, open dialogue synthesis practical.

## Key points
- **Model size and speed:** ZipVoice-Dialog has 123M parameters, compared with 2.67B for MoonCast and 1.61B for Dia. Its real-time factor is 0.063, versus 0.953 for MoonCast and 1.663 for Dia, making it more than 15× faster.
- **English test set:** WER is 3.25, against 23.62 for MoonCast and 11.80 for Dia. On the short English subset, cpWER (which also checks speaker attribution) is 3.27, against 16.53 and 12.59.
- **Chinese test set:** WER is 3.17 versus 15.85 for MoonCast. In a 10-listener subjective test, SMOS was 3.86 versus 2.35, and MoonCast's CMOS was −1.17.
- **Training data:** OpenDialog (6.8k h) alone is enough to train a competitive model. Adding a 0.8k-hour manually annotated in-house set improves the balance between speaker similarity and quality.
- **Availability and scope:** code, checkpoints and OpenDialog are released at github.com/k2-fsa/ZipVoice. Limitations include restricted expressiveness at this scale, subjective tests run only in Chinese, and experiments covering two speakers only.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Xiaomi](https://md.research.elamur.ai/free/en/companies/xiaomi/) — Company, Beijing, China · 9 works · 334 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🗨️ Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) · 624 works

## Cite

Cite as: Han Zhu, Wei Kang, Liyong Guo, Zengwei Yao, Fangjun Kuang, Weiji Zhuang, Zhaoqing Li, Zhifeng Han, Dong Zhang, Xin Zhang, Xingchen Song, Lingxuan Ye, Long Lin and Daniel Povey. 2026. ZipVoice-Dialog: Non-Autoregressive Spoken Dialogue Generation with Flow Matching. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhu2026zipvoice,
  title = {{ZipVoice-Dialog: Non-Autoregressive Spoken Dialogue Generation with Flow Matching}},
  author = {Han Zhu, Wei Kang, Liyong Guo, Zengwei Yao, Fangjun Kuang, Weiji Zhuang, Zhaoqing Li, Zhifeng Han, Dong Zhang, Xin Zhang, Xingchen Song, Lingxuan Ye, Long Lin and Daniel Povey},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.1928},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.1928}
}
```

- [← previous What Breaks Knowledge Graph based RAG? Benchmarking and Empirical Insights into Reasoning under Incomplete Knowledge](https://md.research.elamur.ai/free/en/papers/zhou-2026-what-breaks-knowledge-graph-based/)
- [next → ZoFia: Zero-Shot Fake News Detection with Entity-Guided Retrieval and Multi-LLM Interaction](https://md.research.elamur.ai/free/en/papers/wu-2026-zofia-zero-shot-fake-news/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/6/)

## References

54 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Philip Anastassiou, Jiawei Chen, Jitong Chen, Yuanzhe Chen, Zhuo Chen, Ziyi Chen, Jian Cong, Lelai Deng, Chuang Ding, Lu Gao, Mingqing Gong, Peisong Huang, Qingqing Huang, Zhiying Huang, Yuanyuan Huo, Dongya Jia, Chumin Li, Feiya Li, Hui Li, and 27 others. 2024. [Seed-tts: A family of high-quality ve…](https://md.research.elamur.ai/free/en/papers/anastassiou-2024-seed-tts-family-high/)
2. Hervé Bredin, Ruiqing Yin, Juan Manuel Coria, Gregory Gelly, Pavel Korshunov, Marvin Lavechin, Diego Fustes, Hadrien Titeux, Wassim Bouaziz, and Marie-Philippe Gill. 2020. Pyannote. audio: neural building blocks for speaker diarization. In ICASSP 2020-2020 IEEE International conference on acoustics,… [find](https://scholar.google.com/scholar?q=Herv%C3%A9+Bredin%2C+Ruiqing+Yin%2C+Juan+Manuel+Coria%2C+Gregory+Gelly%2C+Pavel+Korshunov%2C+Marvin+Lavechin%2C+Diego+Fustes%2C+Hadrien+Titeux%2C+Wassim+Bouaziz%2C+and+Marie%2DPhilippe+Gill%2E+2020%2E+Pyannote%2E+audio%3A+neural+buil)
3. Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, Jian Wu, Long Zhou, Shuo Ren, Yanmin Qian, Yao Qian, Jian Wu, Michael Zeng, Xiangzhan Yu, and Furu Wei. 2022. Wavlm: Large-scale self-supervised pre-training for full stack… [find](https://scholar.google.com/scholar?q=Sanyuan+Chen%2C+Chengyi+Wang%2C+Zhengyang+Chen%2C+Yu+Wu%2C+Shujie+Liu%2C+Zhuo+Chen%2C+Jinyu+Li%2C+Naoyuki+Kanda%2C+Takuya+Yoshioka%2C+Xiong+Xiao%2C+Jian+Wu%2C+Long+Zhou%2C+Shuo+Ren%2C+Yanmin+Qian%2C+Yao+Qian%2C+Jian+Wu%2C+Michael+Ze)
4. Sanyuan Chen, Chengyi Wang, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, Lei He, Sheng Zhao, and Furu Wei. 2025. [Neural codec language models are zero-shot text to speech synthesizers](https://md.research.elamur.ai/free/en/papers/wang-2023-neural-codec-language-models-zero/). IEEE Transactions on Audio, Speech and Language Processing, 33:705–…
5. Yushen Chen, Zhikang Niu, Ziyang Ma, Keqi Deng, Chunhui Wang, Jian Zhao, Kai Yu, and Xie Chen. 2024. [F5-tts: A fairytaler that fakes fluent and faithful speech with flow matching](https://md.research.elamur.ai/free/en/papers/chen-2025-f5-tts-fairytaler-that-fakes/). arXiv preprint arXiv:2410.06885.
6. Jian Cong, Shan Yang, Na Hu, Guangzhi Li, Lei Xie, and Dan Su. 2021. Controllable context-aware conversational speech synthesis. In Proc. [find](https://scholar.google.com/scholar?q=Jian+Cong%2C+Shan+Yang%2C+Na+Hu%2C+Guangzhi+Li%2C+Lei+Xie%2C+and+Dan+Su%2E+2021%2E+Controllable+context%2Daware+conversational+speech+synthesis%2E+In+Proc%2E)
7. Interspeech 2021, pages 4658–4662. [find](https://scholar.google.com/scholar?q=Interspeech+2021%2C+pages+4658%E2%80%934662%2E)
8. Jordan Darefsky, Ge Zhu, and Zhiyao Duan. 2024. Parakeet: A natural sounding, conversational text-to-speech model. [https://jordandarefsky.com/](https://jordandarefsky.com/) blog/2024/parakeet/. Blog post. [↗](https://jordandarefsky.com/)
9. Alexandre Défossez, Laurent Mazaré, Manu Orsini, Amélie Royer, Patrick Pérez, Hervé Jégou, Edouard Grave, and Neil Zeghidour. 2024. [Moshi: a speech-text foundation model for real-time dialogue](https://md.research.elamur.ai/free/en/papers/defossez-2024-moshi-speech-text-foundation-model/). arXiv preprint arXiv:2410.00037.
10. Najim Dehak, Patrick J Kenny, Réda Dehak, Pierre Dumouchel, and Pierre Ouellet. 2010. Front-end factor analysis for speaker verification. IEEE Transactions on Audio, Speech, and Language Processing, 19(4):788–798. [find](https://scholar.google.com/scholar?q=Najim+Dehak%2C+Patrick+J+Kenny%2C+R%C3%A9da+Dehak%2C+Pierre+Dumouchel%2C+and+Pierre+Ouellet%2E+2010%2E+Front%2Dend+factor+analysis+for+speaker+verification%2E+IEEE+Transactions+on+Audio%2C+Speech%2C+and+Language+Processing%2C+1)
11. Wei Deng, Siyi Zhou, Jingchen Shu, Jinchao Wang, and Lu Wang. 2025. Indextts: An industrial-level controllable and efficient zero-shot text-to-speech system. arXiv preprint arXiv:2502.05512. — [arXiv:2502.05512](https://arxiv.org/abs/2502.05512) [find](https://scholar.google.com/scholar?q=Wei+Deng%2C+Siyi+Zhou%2C+Jingchen+Shu%2C+Jinchao+Wang%2C+and+Lu+Wang%2E+2025%2E+Indextts%3A+An+industrial%2Dlevel+controllable+and+efficient+zero%2Dshot+text%2Dto%2Dspeech+system%2E+arXiv+preprint+arXiv%3A2502%2E05512%2E+%E2%80%94+%5BarXiv%3A)
12. Brecht Desplanques, Jenthe Thienpondt, and Kris Demuynck. 2020. Ecapa-tdnn: Emphasized channel attention, propagation and aggregation in tdnn based speaker verification. In Proc. [find](https://scholar.google.com/scholar?q=Brecht+Desplanques%2C+Jenthe+Thienpondt%2C+and+Kris+Demuynck%2E+2020%2E+Ecapa%2Dtdnn%3A+Emphasized+channel+attention%2C+propagation+and+aggregation+in+tdnn+based+speaker+verification%2E+In+Proc%2E)
13. Interspeech 2020, pages 3830–3834. [find](https://scholar.google.com/scholar?q=Interspeech+2020%2C+pages+3830%E2%80%933834%2E)
14. Zhihao Du, Yuxuan Wang, Qian Chen, Xian Shi, Xiang Lv, Tianyu Zhao, Zhifu Gao, Yexin Yang, Changfeng Gao, Hui Wang, Fan Yu, Huadai Liu, Zhengyan Sheng, Yue Gu, Chong Deng, Wen Wang, Shiliang Zhang, Zhijie Yan, and Jingren Zhou. 2024. Cosyvoice 2: Scalable streaming speech synthesis with large langua… [find](https://scholar.google.com/scholar?q=Zhihao+Du%2C+Yuxuan+Wang%2C+Qian+Chen%2C+Xian+Shi%2C+Xiang+Lv%2C+Tianyu+Zhao%2C+Zhifu+Gao%2C+Yexin+Yang%2C+Changfeng+Gao%2C+Hui+Wang%2C+Fan+Yu%2C+Huadai+Liu%2C+Zhengyan+Sheng%2C+Yue+Gu%2C+Chong+Deng%2C+Wen+Wang%2C+Shiliang+Zhang%2C+Zh)
15. Sefik Emre Eskimez, Xiaofei Wang, Manthan Thakker, Canrun Li, Chung-Hsien Tsai, Zhen Xiao, Hemin Yang, Zirun Zhu, Min Tang, Xu Tan, Yanqing Liu, Sheng Zhao, and Naoyuki Kanda. 2024. E2 tts: Embarrassingly easy fully non-autoregressive zero-shot tts. In 2024 IEEE Spoken Language Technology Workshop (… [find](https://scholar.google.com/scholar?q=Sefik+Emre+Eskimez%2C+Xiaofei+Wang%2C+Manthan+Thakker%2C+Canrun+Li%2C+Chung%2DHsien+Tsai%2C+Zhen+Xiao%2C+Hemin+Yang%2C+Zirun+Zhu%2C+Min+Tang%2C+Xu+Tan%2C+Yanqing+Liu%2C+Sheng+Zhao%2C+and+Naoyuki+Kanda%2E+2024%2E+E2+tts%3A+Embarrassi)
16. Yu-Kuan Fu, Cheng-Kuang Lee, Hsiu-Hsuan Wang, and Hung-yi Lee. 2024. Investigating the effects of large-scale pseudo-stereo data and different speech foundation model on dialogue generative spoken language model. arXiv preprint arXiv:2407.01911. — [arXiv:2407.01911](https://arxiv.org/abs/2407.01911) [find](https://scholar.google.com/scholar?q=Yu%2DKuan+Fu%2C+Cheng%2DKuang+Lee%2C+Hsiu%2DHsuan+Wang%2C+and+Hung%2Dyi+Lee%2E+2024%2E+Investigating+the+effects+of+large%2Dscale+pseudo%2Dstereo+data+and+different+speech+foundation+model+on+dialogue+generative+spoken+lan)
17. Zhifu Gao, ShiLiang Zhang, Ian McLoughlin, and Zhijie Yan. 2022. Paraformer: Fast and accurate parallel transformer for non-autoregressive end-to-end speech recognition. In Proc. [find](https://scholar.google.com/scholar?q=Zhifu+Gao%2C+ShiLiang+Zhang%2C+Ian+McLoughlin%2C+and+Zhijie+Yan%2E+2022%2E+Paraformer%3A+Fast+and+accurate+parallel+transformer+for+non%2Dautoregressive+end%2Dto%2Dend+speech+recognition%2E+In+Proc%2E)
18. Interspeech 2022, pages 2063–2067. [find](https://scholar.google.com/scholar?q=Interspeech+2022%2C+pages+2063%E2%80%932067%2E)
19. Hao-Han Guo, Yao Hu, Kun Liu, Fei-Yu Shen, Xu Tang, Yi-Chen Wu, Feng-Long Xie, Kun Xie, and Kai-Tuo Xu. 2024. Fireredtts: A foundation text-to-speech framework for industry-level generative speech applications. arXiv preprint arXiv:2409.03283. — [arXiv:2409.03283](https://arxiv.org/abs/2409.03283) [find](https://scholar.google.com/scholar?q=Hao%2DHan+Guo%2C+Yao+Hu%2C+Kun+Liu%2C+Fei%2DYu+Shen%2C+Xu+Tang%2C+Yi%2DChen+Wu%2C+Feng%2DLong+Xie%2C+Kun+Xie%2C+and+Kai%2DTuo+Xu%2E+2024%2E+Fireredtts%3A+A+foundation+text%2Dto%2Dspeech+framework+for+industry%2Dlevel+generative+speech+app)
20. Haohan Guo, Shaofei Zhang, Frank K Soong, Lei He, and Lei Xie. 2021. Conversational end-to-end tts for voice agents. In 2021 IEEE Spoken Language Technology Workshop (SLT), pages 403–409. IEEE. [find](https://scholar.google.com/scholar?q=Haohan+Guo%2C+Shaofei+Zhang%2C+Frank+K+Soong%2C+Lei+He%2C+and+Lei+Xie%2E+2021%2E+Conversational+end%2Dto%2Dend+tts+for+voice+agents%2E+In+2021+IEEE+Spoken+Language+Technology+Workshop+%28SLT%29%2C+pages+403%E2%80%93409%2E+IEEE%2E)
21. Haorui He, Zengqiang Shang, Chaoren Wang, Xuyuan Li, Yicheng Gu, Hua Hua, Liwei Liu, Chen Yang, Jiaqi Li, Peiyang Shi, Yuancheng Wang, Kai Chen, 38725 9 [find](https://scholar.google.com/scholar?q=Haorui+He%2C+Zengqiang+Shang%2C+Chaoren+Wang%2C+Xuyuan+Li%2C+Yicheng+Gu%2C+Hua+Hua%2C+Liwei+Liu%2C+Chen+Yang%2C+Jiaqi+Li%2C+Peiyang+Shi%2C+Yuancheng+Wang%2C+Kai+Chen%2C+38725+9)
22. Pengyuan Zhang, and Zhizheng Wu. 2024. Emilia: An extensive, multilingual, and diverse speech dataset for large-scale speech generation. In 2024 IEEE Spoken Language Technology Workshop (SLT), pages 885–890. IEEE. [find](https://scholar.google.com/scholar?q=Pengyuan+Zhang%2C+and+Zhizheng+Wu%2E+2024%2E+Emilia%3A+An+extensive%2C+multilingual%2C+and+diverse+speech+dataset+for+large%2Dscale+speech+generation%2E+In+2024+IEEE+Spoken+Language+Technology+Workshop+%28SLT%29%2C+pages+8)
23. Jonathan Ho and Tim Salimans. 2021. [Classifier-free diffusion guidance](https://md.research.elamur.ai/free/en/papers/ho-2022-classifier-free-diffusion-guidance/). In NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications.
24. Zeqian Ju, Dongchao Yang, Jianwei Yu, Kai Shen, Yichong Leng, Zhengtao Wang, Xu Tan, Xinyu Zhou, Tao Qin, and Xiangyang Li. 2025. Mooncast: High-quality zero-shot podcast generation. arXiv preprint arXiv:2503.14345. — [arXiv:2503.14345](https://arxiv.org/abs/2503.14345) [find](https://scholar.google.com/scholar?q=Zeqian+Ju%2C+Dongchao+Yang%2C+Jianwei+Yu%2C+Kai+Shen%2C+Yichong+Leng%2C+Zhengtao+Wang%2C+Xu+Tan%2C+Xinyu+Zhou%2C+Tao+Qin%2C+and+Xiangyang+Li%2E+2025%2E+Mooncast%3A+High%2Dquality+zero%2Dshot+podcast+generation%2E+arXiv+preprint+ar)
25. Rithesh Kumar, Prem Seetharaman, Alejandro Luebs, Ishaan Kumar, and Kundan Kumar. 2023. High-fidelity audio compression with improved rvqgan. Advances in Neural Information Processing Systems, 36:27980–27993. [find](https://scholar.google.com/scholar?q=Rithesh+Kumar%2C+Prem+Seetharaman%2C+Alejandro+Luebs%2C+Ishaan+Kumar%2C+and+Kundan+Kumar%2E+2023%2E+High%2Dfidelity+audio+compression+with+improved+rvqgan%2E+Advances+in+Neural+Information+Processing+Systems%2C+36%3A2798)
26. Nari Labs. 2025. Dia: A tts model capable of generating ultra-realistic dialogue in one pass. [https://github.com/nari-labs/dia](https://github.com/nari-labs/dia). [↗](https://github.com/nari-labs/dia)
27. Matthew Le, Apoorv Vyas, Bowen Shi, Brian Karrer, Leda Sari, Rashel Moritz, Mary Williamson, Vimal Manohar, Yossi Adi, Jay Mahadeokar, and Wei-Ning Hsu. 2023. Voicebox: Text-guided multilingual universal speech generation at scale. Advances in neural information processing systems, 36:14005–14034. [find](https://scholar.google.com/scholar?q=Matthew+Le%2C+Apoorv+Vyas%2C+Bowen+Shi%2C+Brian+Karrer%2C+Leda+Sari%2C+Rashel+Moritz%2C+Mary+Williamson%2C+Vimal+Manohar%2C+Yossi+Adi%2C+Jay+Mahadeokar%2C+and+Wei%2DNing+Hsu%2E+2023%2E+Voicebox%3A+Text%2Dguided+multilingual+univer)
28. Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Matthew Le. 2023. [Flow matching for generative modeling](https://md.research.elamur.ai/free/en/papers/lipman-2022-flow-matching-generative-modeling/). In The Eleventh International Conference on Learning Representations.
29. Rui Liu, Yifan Hu, Yi Ren, Xiang Yin, and Haizhou Li. 2024. Generative expressive conversational speech synthesis. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 4187–4196. [find](https://scholar.google.com/scholar?q=Rui+Liu%2C+Yifan+Hu%2C+Yi+Ren%2C+Xiang+Yin%2C+and+Haizhou+Li%2E+2024%2E+Generative+expressive+conversational+speech+synthesis%2E+In+Proceedings+of+the+32nd+ACM+International+Conference+on+Multimedia%2C+pages+4187%E2%80%93419)
30. Haitian Lu, Gaofeng Cheng, Liuping Luo, Leying Zhang, Yanmin Qian, and Pengyuan Zhang. 2025. Slide: Integrating speech language model with llm for spontaneous spoken dialogue generation. arXiv preprint arXiv:2501.00805. — [arXiv:2501.00805](https://arxiv.org/abs/2501.00805) [find](https://scholar.google.com/scholar?q=Haitian+Lu%2C+Gaofeng+Cheng%2C+Liuping+Luo%2C+Leying+Zhang%2C+Yanmin+Qian%2C+and+Pengyuan+Zhang%2E+2025%2E+Slide%3A+Integrating+speech+language+model+with+llm+for+spontaneous+spoken+dialogue+generation%2E+arXiv+preprin)
31. Ziqiao Meng, Qichao Wang, Wenqian Cui, Yifei Zhang, Bingzhe Wu, Irwin King, Liang Chen, and Peilin Zhao. 2024. Parrot: Autoregressive spoken dialogue language modeling with decoder-only transformers. In Audio Imagination: NeurIPS 2024 Workshop AI-Driven Speech, Music, and Sound Generation. [find](https://scholar.google.com/scholar?q=Ziqiao+Meng%2C+Qichao+Wang%2C+Wenqian+Cui%2C+Yifei+Zhang%2C+Bingzhe+Wu%2C+Irwin+King%2C+Liang+Chen%2C+and+Peilin+Zhao%2E+2024%2E+Parrot%3A+Autoregressive+spoken+dialogue+language+modeling+with+decoder%2Donly+transformers%2E+)
32. Kentaro Mitsui, Yukiya Hono, and Kei Sawada. 2023. Towards human-like spoken dialogue generation between ai agents from written dialogue. arXiv preprint arXiv:2310.01088. — [arXiv:2310.01088](https://arxiv.org/abs/2310.01088) [find](https://scholar.google.com/scholar?q=Kentaro+Mitsui%2C+Yukiya+Hono%2C+and+Kei+Sawada%2E+2023%2E+Towards+human%2Dlike+spoken+dialogue+generation+between+ai+agents+from+written+dialogue%2E+arXiv+preprint+arXiv%3A2310%2E01088%2E+%E2%80%94+%5BarXiv%3A2310%2E01088%5D%28https%3A%2F%2F)
33. Tu Anh Nguyen, Eugene Kharitonov, Jade Copet, Yossi Adi, Wei-Ning Hsu, Ali Elkahky, Paden Tomasello, Robin Algayres, Benoit Sagot, Abdelrahman Mohamed, and Emmanuel Dupoux. 2023. [Generative spoken dialogue language modeling](https://md.research.elamur.ai/free/en/papers/nguyen-2023-generative-spoken-dialogue-language-modeling/). Transactions of the Association for Computational Linguistics, 11:250–266.
34. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2023. [Robust speech recognition via large-scale weak supervision](https://md.research.elamur.ai/free/en/papers/radford-2022-robust-speech-recognition-large-scale/). In International conference on machine learning, pages 28492–28518. PMLR.
35. Chandan KA Reddy, Vishak Gopal, and Ross Cutler. 2022. Dnsmos p. 835: A non-intrusive perceptual objective speech quality metric to evaluate noise suppressors. In ICASSP 2022-2022 IEEE international conference on acoustics, speech and signal processing (ICASSP), pages 886–890. IEEE. [find](https://scholar.google.com/scholar?q=Chandan+KA+Reddy%2C+Vishak+Gopal%2C+and+Ross+Cutler%2E+2022%2E+Dnsmos+p%2E+835%3A+A+non%2Dintrusive+perceptual+objective+speech+quality+metric+to+evaluate+noise+suppressors%2E+In+ICASSP+2022%2D2022+IEEE+international+c)
36. Yi Ren, Yangjun Ruan, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, and Tie-Yan Liu. 2019. Fastspeech: Fast, robust and controllable text to speech. Advances in neural information processing systems, 32. [find](https://scholar.google.com/scholar?q=Yi+Ren%2C+Yangjun+Ruan%2C+Xu+Tan%2C+Tao+Qin%2C+Sheng+Zhao%2C+Zhou+Zhao%2C+and+Tie%2DYan+Liu%2E+2019%2E+Fastspeech%3A+Fast%2C+robust+and+controllable+text+to+speech%2E+Advances+in+neural+information+processing+systems%2C+32%2E)
37. Takaaki Saeki, Detai Xin, Wataru Nakata, Tomoki Koriyama, Shinnosuke Takamichi, and Hiroshi Saruwatari. 2022. Utmos: Utokyo-sarulab system for voicemos challenge 2022. [find](https://scholar.google.com/scholar?q=Takaaki+Saeki%2C+Detai+Xin%2C+Wataru+Nakata%2C+Tomoki+Koriyama%2C+Shinnosuke+Takamichi%2C+and+Hiroshi+Saruwatari%2E+2022%2E+Utmos%3A+Utokyo%2Dsarulab+system+for+voicemos+challenge+2022%2E)
38. Kai Shen, Zeqian Ju, Xu Tan, Yanqing Liu, Yichong Leng, Lei He, Tao Qin, Sheng Zhao, and Jiang Bian. 2024. Naturalspeech 2: Latent diffusion models are natural and zero-shot speech and singing synthesizers. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Kai+Shen%2C+Zeqian+Ju%2C+Xu+Tan%2C+Yanqing+Liu%2C+Yichong+Leng%2C+Lei+He%2C+Tao+Qin%2C+Sheng+Zhao%2C+and+Jiang+Bian%2E+2024%2E+Naturalspeech+2%3A+Latent+diffusion+models+are+natural+and+zero%2Dshot+speech+and+singing+synthes)
39. Hubert Siuzdak. 2024. Vocos: Closing the gap between time-domain and fourier-based neural vocoders for high-quality audio synthesis. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Hubert+Siuzdak%2E+2024%2E+Vocos%3A+Closing+the+gap+between+time%2Ddomain+and+fourier%2Dbased+neural+vocoders+for+high%2Dquality+audio+synthesis%2E+In+The+Twelfth+International+Conference+on+Learning+Representations)
40. David Snyder, Daniel Garcia-Romero, Gregory Sell, Daniel Povey, and Sanjeev Khudanpur. 2018. X-vectors: Robust dnn embeddings for speaker recognition. In 2018 IEEE international conference on acoustics, speech and signal processing (ICASSP), pages 5329–5333. IEEE. [find](https://scholar.google.com/scholar?q=David+Snyder%2C+Daniel+Garcia%2DRomero%2C+Gregory+Sell%2C+Daniel+Povey%2C+and+Sanjeev+Khudanpur%2E+2018%2E+X%2Dvectors%3A+Robust+dnn+embeddings+for+speaker+recognition%2E+In+2018+IEEE+international+conference+on+acoustic)
41. Yakun Song, Zhuo Chen, Xiaofei Wang, Ziyang Ma, and Xie Chen. 2025. Ella-v: Stable neural codec language modeling with alignment-guided sequence reordering. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 25174– 25182. [find](https://scholar.google.com/scholar?q=Yakun+Song%2C+Zhuo+Chen%2C+Xiaofei+Wang%2C+Ziyang+Ma%2C+and+Xie+Chen%2E+2025%2E+Ella%2Dv%3A+Stable+neural+codec+language+modeling+with+alignment%2Dguided+sequence+reordering%2E+In+Proceedings+of+the+AAAI+Conference+on+Ar)
42. Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. 2020. [Score-based generative modeling through stochastic differential equations](https://md.research.elamur.ai/free/en/papers/song-2020-score-based-generative-modeling-through/). arXiv preprint arXiv:2011.13456.
43. Xinsheng Wang, Mingqi Jiang, Ziyang Ma, Ziyu Zhang, Songxiang Liu, Linqin Li, Zheng Liang, Qixi Zheng, Rui Wang, Xiaoqin Feng, Weizhen Bian, Zhen Ye, Sitong Cheng, Ruibin Yuan, Zhixian Zhao, Xinfa Zhu, Jiahao Pan, Liumeng Xue, Pengcheng Zhu, and 6 others. 2025. Spark-tts: An efficient llm-based text… — [arXiv:2503.01710](https://arxiv.org/abs/2503.01710) [find](https://scholar.google.com/scholar?q=Xinsheng+Wang%2C+Mingqi+Jiang%2C+Ziyang+Ma%2C+Ziyu+Zhang%2C+Songxiang+Liu%2C+Linqin+Li%2C+Zheng+Liang%2C+Qixi+Zheng%2C+Rui+Wang%2C+Xiaoqin+Feng%2C+Weizhen+Bian%2C+Zhen+Ye%2C+Sitong+Cheng%2C+Ruibin+Yuan%2C+Zhixian+Zhao%2C+Xinfa+Zhu)
44. Yuancheng Wang, Haoyue Zhan, Liwei Liu, Ruihong Zeng, Haotian Guo, Jiachen Zheng, Qiang Zhang, Xueyao Zhang, Shunsi Zhang, and Zhizheng Wu. 2024. Maskgct: Zero-shot text-to-speech with masked generative codec transformer. arXiv preprint arXiv:2409.00750. — [arXiv:2409.00750](https://arxiv.org/abs/2409.00750) [find](https://scholar.google.com/scholar?q=Yuancheng+Wang%2C+Haoyue+Zhan%2C+Liwei+Liu%2C+Ruihong+Zeng%2C+Haotian+Guo%2C+Jiachen+Zheng%2C+Qiang+Zhang%2C+Xueyao+Zhang%2C+Shunsi+Zhang%2C+and+Zhizheng+Wu%2E+2024%2E+Maskgct%3A+Zero%2Dshot+text%2Dto%2Dspeech+with+masked+generati)
45. Shinji Watanabe, Michael Mandel, Jon Barker, Emmanuel Vincent, Ashish Arora, Xuankai Chang, Sanjeev Khudanpur, Vimal Manohar, Daniel Povey, Desh Raj, David Snyder, Aswin Shanmugam Subramanian, Jan Trmal, Bar Ben Yair, Christoph Boeddeker, Zhaoheng Ni, Yusuke Fujita, Shota Horiguchi, Naoyuki Kanda, a… — [arXiv:2004.09249](https://arxiv.org/abs/2004.09249) [find](https://scholar.google.com/scholar?q=Shinji+Watanabe%2C+Michael+Mandel%2C+Jon+Barker%2C+Emmanuel+Vincent%2C+Ashish+Arora%2C+Xuankai+Chang%2C+Sanjeev+Khudanpur%2C+Vimal+Manohar%2C+Daniel+Povey%2C+Desh+Raj%2C+David+Snyder%2C+Aswin+Shanmugam+Subramanian%2C+Jan+Trm)
46. Jinlong Xue, Yayue Deng, Fengping Wang, Ya Li, Yingming Gao, Jianhua Tao, Jianqing Sun, and Jiaen Liang. 2023. M 2-ctts: End-to-end multi-scale multi-modal conversational text-to-speech synthesis. In ICASSP 2023-2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP),… [find](https://scholar.google.com/scholar?q=Jinlong+Xue%2C+Yayue+Deng%2C+Fengping+Wang%2C+Ya+Li%2C+Yingming+Gao%2C+Jianhua+Tao%2C+Jianqing+Sun%2C+and+Jiaen+Liang%2E+2023%2E+M+2%2Dctts%3A+End%2Dto%2Dend+multi%2Dscale+multi%2Dmodal+conversational+text%2Dto%2Dspeech+synthesis%2E+In+)
47. Yifan Yang, Shujie Liu, Jinyu Li, Yuxuan Hu, Haibin Wu, Hui Wang, Jianwei Yu, Lingwei Meng, Haiyang Sun, Yanqing Liu, Yan Lu, Kai Yu, and Xie Chen. 2025. Pseudo-autoregressive neural codec language models for efficient zero-shot text-to-speech synthesis. In Proceedings of the 33rd ACM International… [find](https://scholar.google.com/scholar?q=Yifan+Yang%2C+Shujie+Liu%2C+Jinyu+Li%2C+Yuxuan+Hu%2C+Haibin+Wu%2C+Hui+Wang%2C+Jianwei+Yu%2C+Lingwei+Meng%2C+Haiyang+Sun%2C+Yanqing+Liu%2C+Yan+Lu%2C+Kai+Yu%2C+and+Xie+Chen%2E+2025%2E+Pseudo%2Dautoregressive+neural+codec+language+mo)
48. Zehui Yang, Yifan Chen, Lei Luo, Runyan Yang, Lingxuan Ye, Gaofeng Cheng, Ji Xu, Yaohui Jin, Qingqing Zhang, Pengyuan Zhang, Lei Xie, and Yonghong Yan. 2022. Open source magicdata-ramc: A rich annotated mandarin conversational (ramc) speech dataset. In Proc. [find](https://scholar.google.com/scholar?q=Zehui+Yang%2C+Yifan+Chen%2C+Lei+Luo%2C+Runyan+Yang%2C+Lingxuan+Ye%2C+Gaofeng+Cheng%2C+Ji+Xu%2C+Yaohui+Jin%2C+Qingqing+Zhang%2C+Pengyuan+Zhang%2C+Lei+Xie%2C+and+Yonghong+Yan%2E+2022%2E+Open+source+magicdata%2Dramc%3A+A+rich+annotat)
49. Interspeech 2022, pages 1736–1740. [find](https://scholar.google.com/scholar?q=Interspeech+2022%2C+pages+1736%E2%80%931740%2E)
50. Zengwei Yao, Liyong Guo, Xiaoyu Yang, Wei Kang, Fangjun Kuang, Yifan Yang, Zengrui Jin, Long Lin, and Daniel Povey. 2024. Zipformer: A faster and better encoder for automatic speech recognition. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Zengwei+Yao%2C+Liyong+Guo%2C+Xiaoyu+Yang%2C+Wei+Kang%2C+Fangjun+Kuang%2C+Yifan+Yang%2C+Zengrui+Jin%2C+Long+Lin%2C+and+Daniel+Povey%2E+2024%2E+Zipformer%3A+A+faster+and+better+encoder+for+automatic+speech+recognition%2E+In+Th)
51. Leying Zhang, Yao Qian, Xiaofei Wang, Manthan Thakker, Dongmei Wang, Jianwei Yu, Haibin Wu, Yuxuan Hu, Jinyu Li, Yanmin Qian, and Sheng Zhao. 2025. Covomix2: Advancing zero-shot dialogue generation with fully non-autoregressive flow matching. arXiv preprint arXiv:2506.00885. — [arXiv:2506.00885](https://arxiv.org/abs/2506.00885) [find](https://scholar.google.com/scholar?q=Leying+Zhang%2C+Yao+Qian%2C+Xiaofei+Wang%2C+Manthan+Thakker%2C+Dongmei+Wang%2C+Jianwei+Yu%2C+Haibin+Wu%2C+Yuxuan+Hu%2C+Jinyu+Li%2C+Yanmin+Qian%2C+and+Sheng+Zhao%2E+2025%2E+Covomix2%3A+Advancing+zero%2Dshot+dialogue+generation+wi)
52. Leying Zhang, Yao Qian, Long Zhou, Shujie Liu, Dongmei Wang, Xiaofei Wang, Midia Yousefi, Yanmin Qian, Jinyu Li, Lei He, Sheng Zhao, and Michael Zeng. 2024. Covomix: Advancing zero-shot speech generation for human-like multi-talker conversations. Advances in Neural Information Processing Systems, 37… [find](https://scholar.google.com/scholar?q=Leying+Zhang%2C+Yao+Qian%2C+Long+Zhou%2C+Shujie+Liu%2C+Dongmei+Wang%2C+Xiaofei+Wang%2C+Midia+Yousefi%2C+Yanmin+Qian%2C+Jinyu+Li%2C+Lei+He%2C+Sheng+Zhao%2C+and+Michael+Zeng%2E+2024%2E+Covomix%3A+Advancing+zero%2Dshot+speech+generat)
53. Zhitong Zhou, Qingqing Zhang, Lei Luo, Jiechen Liu, and Ruohua Zhou. 2025. Open-source full-duplex conversational datasets for natural and interactive speech synthesis. arXiv preprint arXiv:2509.04093. — [arXiv:2509.04093](https://arxiv.org/abs/2509.04093) [find](https://scholar.google.com/scholar?q=Zhitong+Zhou%2C+Qingqing+Zhang%2C+Lei+Luo%2C+Jiechen+Liu%2C+and+Ruohua+Zhou%2E+2025%2E+Open%2Dsource+full%2Dduplex+conversational+datasets+for+natural+and+interactive+speech+synthesis%2E+arXiv+preprint+arXiv%3A2509%2E04093)
54. Han Zhu, Wei Kang, Zengwei Yao, Liyong Guo, Fangjun Kuang, Zhaoqing Li, Weiji Zhuang, Long Lin, and Daniel Povey. 2025. Zipvoice: Fast and high-quality zero-shot text-to-speech with flow matching. arXiv preprint arXiv:2506.13053. — [arXiv:2506.13053](https://arxiv.org/abs/2506.13053) [find](https://scholar.google.com/scholar?q=Han+Zhu%2C+Wei+Kang%2C+Zengwei+Yao%2C+Liyong+Guo%2C+Fangjun+Kuang%2C+Zhaoqing+Li%2C+Weiji+Zhuang%2C+Long+Lin%2C+and+Daniel+Povey%2E+2025%2E+Zipvoice%3A+Fast+and+high%2Dquality+zero%2Dshot+text%2Dto%2Dspeech+with+flow+matching%2E+arX)

*Updated 31 October 2026*