---
title: "VoiceStar — paper by Peng et al. (2026) — summary and ideas"
description: "Autoregressive neural codec language model (NCLM) TTS systems usually concatenate text and… 82 references, cited by 4 papers here."
authors: ["Puyuan Peng", "Zhisheng Zheng", "Shang-Wen Li", "Abdelrahman Mohamed", "David Harwath"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.570/"
doi: "10.18653/v1/2026.findings-acl.570"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/peng-2026-voicestar-robust-zero-shot-autoregressive/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/peng-2026-voicestar-robust-zero-shot-autoregressive/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/4/) / VoiceStar: Robust Zero-Shot Autoregressive TTS with Duration Control and Extrapolation

# VoiceStar: Robust Zero-Shot Autoregressive TTS with Duration Control and Extrapolation

**Authors:** Puyuan Peng, Zhisheng Zheng, [Shang-Wen Li](https://md.research.elamur.ai/free/en/authors/shang-wen-li/), [Abdelrahman Mohamed](https://md.research.elamur.ai/free/en/authors/abdelrahman-mohamed/) and [David Harwath](https://md.research.elamur.ai/free/en/authors/david-harwath/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.570/) · [PDF](https://aclanthology.org/2026.findings-acl.570.pdf) · [DOI 10.18653/v1/2026.findings-acl.570](https://doi.org/10.18653/v1/2026.findings-acl.570) · [OpenAlex](https://openalex.org/W7166842291)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [4](https://md.research.elamur.ai/free/en/papers/peng-2026-voicestar-robust-zero-shot-autoregressive/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Autoregressive neural codec language model (NCLM) TTS systems usually concatenate text and speech tokens and leave alignment to ordinary positional encodings. As a result, they cannot easily be told how long the output should be, and they degrade when asked to produce speech longer than their training clips. VoiceStar addresses both problems. It is an encoder-decoder NCLM: IPA phonemes go into the encoder, and the decoder predicts Encodec speech tokens. Its central idea, Progress-Monitoring RoPE (PM-RoPE), places text and speech positions on a shared scale tied to the target length. This gives the model an early rough alignment, a way to specify duration, and the ability to run past the training length. Continuation-Prompt Mixed (CPM) training sometimes uses a different utterance from the same speaker as the prompt, which reduces the gap between training and inference. It also makes prompt speed perturbation possible. On short-form benchmarks VoiceStar is competitive with the state of the art. On 30–50 s generation it clearly beats F5-TTS and MaskGCT, which suggests length extrapolation for zero-shot voice cloning is achievable without long-form training data.

## Key points
- The authors describe VoiceStar as the first zero-shot TTS model offering both duration control and extrapolation beyond training length. All compared models were trained on clips of at most 30 s.
- On the 40–50 s extrapolation range, VoiceStar's WER is 11.91, compared with 52.44 for F5-TTS and 82.29 for MaskGCT. On the 30–40 s range, it is 7.27 versus 34.15 and 13.81.
- On short-form benchmarks, VoiceStar reaches a WER of 2.64 on LibriSpeech-PC and 2.15 on Seed-TTS (en), the latter matching ground truth. It also has the best speaker-similarity CMOS on LibriSpeech-PC (0.60).
- Additional techniques include speed perturbation of the prompt during training and prompt repetition at inference. Ablations show that mixing continuation and same-speaker prompts works better than either prompting style alone.
- Limitations: speaker similarity on long-form output lags slightly behind the best models, and the 840M model generates slower than real time (RTF > 1). Code and model are available at github.com/jasonppy/VoiceStar.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [The University of Texas at Austin](https://md.research.elamur.ai/free/en/universities/the-university-of-texas-at-austin/) — University, Austin, United States · 109 works · 56,942 citations

📚 Shelves (7): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🔊 Signal Processing](https://md.research.elamur.ai/free/en/shelves/computer-science/signal-processing/) · 452 works · [🩺 Medicine](https://md.research.elamur.ai/free/en/shelves/medicine/) · 241 works · [🩺 Physiology](https://md.research.elamur.ai/free/en/shelves/medicine/physiology/) · 29 works

## Cite

Cite as: Puyuan Peng, Zhisheng Zheng, Shang-Wen Li, Abdelrahman Mohamed and David Harwath. 2026. VoiceStar: Robust Zero-Shot Autoregressive TTS with Duration Control and Extrapolation. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{peng2026voicestar,
  title = {{VoiceStar: Robust Zero-Shot Autoregressive TTS with Duration Control and Extrapolation}},
  author = {Puyuan Peng, Zhisheng Zheng, Shang-Wen Li, Abdelrahman Mohamed and David Harwath},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.570},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.570}
}
```

- [← previous VN-MTEB: Vietnamese Massive Text Embedding Benchmark](https://md.research.elamur.ai/free/en/papers/pham-2026-vn-mteb-vietnamese-massive-text/)
- [next → VulAgent: Hypothesis-Validation Driven Multi-Agent Architecture for Vulnerability Detection](https://md.research.elamur.ai/free/en/papers/wang-2026-vulagent-hypothesis-validation-driven-multi/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/4/)

## References

82 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Philip Anastassiou, Jiawei Chen, Jitong Chen, Yuanzhe Chen, Zhuo Chen, Ziyi Chen, Jian Cong, Lelai Deng, Chuang Ding, Lu Gao, Mingqing Gong, Peisong Huang, Qingqing Huang, Zhiying Huang, Yuanyuan Huo, Dongya Jia, Chumin Li, Feiya Li, Hui Li, and 27 others. 2024. [Seed-tts: A family of high-quality ve…](https://md.research.elamur.ai/free/en/papers/anastassiou-2024-seed-tts-family-high/)
2. Eric Battenberg, R. J. Skerry-Ryan, Daisy Stanton, Soroosh Mariooryad, Matt Shannon, Julian Salazar, and David Kao. 2024. Very attentive tacotron: Robust and unbounded length generalization in autoregressive transformer-based text-to-speech. ArXiv, abs/2410.22179. [find](https://scholar.google.com/scholar?q=Eric+Battenberg%2C+R%2E+J%2E+Skerry%2DRyan%2C+Daisy+Stanton%2C+Soroosh+Mariooryad%2C+Matt+Shannon%2C+Julian+Salazar%2C+and+David+Kao%2E+2024%2E+Very+attentive+tacotron%3A+Robust+and+unbounded+length+generalization+in+autoreg)
3. Zalán Borsos, Raphaël Marinier, Damien Vincent, Eugene Kharitonov, Olivier Pietquin, Matthew Sharifi, Dominik Roblek, Olivier Teboul, David Grangier, Marco Tagliasacchi, and Neil Zeghidour. 2022. [Audiolm: A language modeling approach to audio generation](https://md.research.elamur.ai/free/en/papers/borsos-2022-audiolm-language-modeling-approach-audio/). IEEE/ACM Transactions on Audio, Speech, and L…
4. Zalán Borsos, Matthew Sharifi, Damien Vincent, Eugene Kharitonov, Neil Zeghidour, and Marco Tagliasacchi. 2023. Soundstorm: Efficient parallel audio generation. ArXiv, abs/2305.09636. — [arXiv:2305.09636](https://arxiv.org/abs/2305.09636) [find](https://scholar.google.com/scholar?q=Zal%C3%A1n+Borsos%2C+Matthew+Sharifi%2C+Damien+Vincent%2C+Eugene+Kharitonov%2C+Neil+Zeghidour%2C+and+Marco+Tagliasacchi%2E+2023%2E+Soundstorm%3A+Efficient+parallel+audio+generation%2E+ArXiv%2C+abs%2F2305%2E09636%2E+%E2%80%94+%5BarXiv%3A2305%2E09)
5. Chen Chen, Yuchen Hu, Wen Wu, Helin Wang, Chng Eng Siong, and Chao Zhang. 2024a. Enhancing zero-shot text-to-speech synthesis with human feedback. ArXiv, abs/2406.00654. — [arXiv:2406.00654](https://arxiv.org/abs/2406.00654) [find](https://scholar.google.com/scholar?q=Chen+Chen%2C+Yuchen+Hu%2C+Wen+Wu%2C+Helin+Wang%2C+Chng+Eng+Siong%2C+and+Chao+Zhang%2E+2024a%2E+Enhancing+zero%2Dshot+text%2Dto%2Dspeech+synthesis+with+human+feedback%2E+ArXiv%2C+abs%2F2406%2E00654%2E+%E2%80%94+%5BarXiv%3A2406%2E00654%5D%28https%3A%2F%2Fa)
6. Sanyuan Chen, Shujie Liu, Long Zhou, Yanqing Liu, Xu Tan, Jinyu Li, Sheng Zhao, Yao Qian, and Furu Wei. 2024b. Vall-e 2: Neural codec language models are human parity zero-shot text to speech synthesizers. Preprint, arXiv:2406.05370. — [arXiv:2406.05370](https://arxiv.org/abs/2406.05370) [find](https://scholar.google.com/scholar?q=Sanyuan+Chen%2C+Shujie+Liu%2C+Long+Zhou%2C+Yanqing+Liu%2C+Xu+Tan%2C+Jinyu+Li%2C+Sheng+Zhao%2C+Yao+Qian%2C+and+Furu+Wei%2E+2024b%2E+Vall%2De+2%3A+Neural+codec+language+models+are+human+parity+zero%2Dshot+text+to+speech+synthesi)
7. Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, Jian Wu, Long Zhou, Shuo Ren, Yanmin Qian, Yao Qian, Micheal Zeng, and Furu Wei. 2021. Wavlm: Large-scale self-supervised pre-training for full stack speech processing. IEE… [find](https://scholar.google.com/scholar?q=Sanyuan+Chen%2C+Chengyi+Wang%2C+Zhengyang+Chen%2C+Yu+Wu%2C+Shujie+Liu%2C+Zhuo+Chen%2C+Jinyu+Li%2C+Naoyuki+Kanda%2C+Takuya+Yoshioka%2C+Xiong+Xiao%2C+Jian+Wu%2C+Long+Zhou%2C+Shuo+Ren%2C+Yanmin+Qian%2C+Yao+Qian%2C+Micheal+Zeng%2C+and+F)
8. Yushen Chen, Zhikang Niu, Ziyang Ma, Keqi Deng, Chunhui Wang, Jian Zhao, Kai Yu, and Xie Chen. 2024c. [F5-tts: A fairytaler that fakes fluent and faithful speech with flow matching](https://md.research.elamur.ai/free/en/papers/chen-2025-f5-tts-fairytaler-that-fakes/). ArXiv, abs/2410.06885.
9. Jade Copet, Felix Kreuk, Itai Gat, Tal Remez, David Kant, Gabriel Synnaeve, Yossi Adi, and Alexandre Defossez. 2023. Simple and controllable music generation. ArXiv, abs/2306.05284. — [arXiv:2306.05284](https://arxiv.org/abs/2306.05284) [find](https://scholar.google.com/scholar?q=Jade+Copet%2C+Felix+Kreuk%2C+Itai+Gat%2C+Tal+Remez%2C+David+Kant%2C+Gabriel+Synnaeve%2C+Yossi+Adi%2C+and+Alexandre+Defossez%2E+2023%2E+Simple+and+controllable+music+generation%2E+ArXiv%2C+abs%2F2306%2E05284%2E+%E2%80%94+%5BarXiv%3A2306%2E0528)
10. Alexandre Defossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi. 2022. [High fidelity neural audio compression](https://md.research.elamur.ai/free/en/papers/defossez-2022-high-fidelity-neural-audio-compression/). ArXiv, abs/2210.13438.
11. Alexandre D’efossez, Laurent Mazar’e, Manu Orsini, Am’elie Royer, Patrick P’erez, Herv’e J’egou, Edouard Grave, and Neil Zeghidour. 2024. [Moshi: a speech-text foundation model for real-time dialogue](https://md.research.elamur.ai/free/en/papers/defossez-2024-moshi-speech-text-foundation-model/). ArXiv, abs/2410.00037.
12. Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022. Llm.int8(): 8-bit matrix multiplication for transformers at scale. Preprint, arXiv:2208.07339. [find](https://scholar.google.com/scholar?q=Tim+Dettmers%2C+Mike+Lewis%2C+Younes+Belkada%2C+and+Luke+Zettlemoyer%2E+2022%2E+Llm%2Eint8%28%29%3A+8%2Dbit+matrix+multiplication+for+transformers+at+scale%2E+Preprint%2C+arXiv%3A2208%2E07339%2E)
13. Anuj Diwan, Zhisheng Zheng, David Harwath, and Eunsol Choi. 2025. Scaling rich style-prompted text-to-speech datasets. [find](https://scholar.google.com/scholar?q=Anuj+Diwan%2C+Zhisheng+Zheng%2C+David+Harwath%2C+and+Eunsol+Choi%2E+2025%2E+Scaling+rich+style%2Dprompted+text%2Dto%2Dspeech+datasets%2E)
14. Chenpeng Du, Yiwei Guo, Hankun Wang, Yifan Yang, Zhikang Niu, Shuai Wang, Hui Zhang, Xie Chen, and Kai Yu. 2024a. Vall-t: Decoder-only generative transducer for robust and decoding-controllable text-to-speech. [find](https://scholar.google.com/scholar?q=Chenpeng+Du%2C+Yiwei+Guo%2C+Hankun+Wang%2C+Yifan+Yang%2C+Zhikang+Niu%2C+Shuai+Wang%2C+Hui+Zhang%2C+Xie+Chen%2C+and+Kai+Yu%2E+2024a%2E+Vall%2Dt%3A+Decoder%2Donly+generative+transducer+for+robust+and+decoding%2Dcontrollable+text%2Dt)
15. Zhihao Du, Qian Chen, Shiliang Zhang, Kai Hu, Heng Lu, Yexin Yang, Hangrui Hu, Siqi Zheng, Yue Gu, Ziyang Ma, Zhifu Gao, and Zhijie Yan. 2024b. Cosyvoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens. ArXiv, abs/2407.05407. [find](https://scholar.google.com/scholar?q=Zhihao+Du%2C+Qian+Chen%2C+Shiliang+Zhang%2C+Kai+Hu%2C+Heng+Lu%2C+Yexin+Yang%2C+Hangrui+Hu%2C+Siqi+Zheng%2C+Yue+Gu%2C+Ziyang+Ma%2C+Zhifu+Gao%2C+and+Zhijie+Yan%2E+2024b%2E+Cosyvoice%3A+A+scalable+multilingual+zero%2Dshot+text%2Dto%2Dspe)
16. Zhihao Du, Yuxuan Wang, Qian Chen, Xian Shi, Xiang Lv, Tianyu Zhao, Zhifu Gao, Yexin Yang, Changfeng Gao, Hui Wang, Fan Yu, Huadai Liu, Zhengyan Sheng, Yue Gu, Chong Deng, Wen Wang, Shiliang Zhang, Zhijie Yan, and Jing-Ru Zhou. 2024c. Cosyvoice 2: Scalable streaming speech synthesis with large langu… [find](https://scholar.google.com/scholar?q=Zhihao+Du%2C+Yuxuan+Wang%2C+Qian+Chen%2C+Xian+Shi%2C+Xiang+Lv%2C+Tianyu+Zhao%2C+Zhifu+Gao%2C+Yexin+Yang%2C+Changfeng+Gao%2C+Hui+Wang%2C+Fan+Yu%2C+Huadai+Liu%2C+Zhengyan+Sheng%2C+Yue+Gu%2C+Chong+Deng%2C+Wen+Wang%2C+Shiliang+Zhang%2C+Zh)
17. Jonathan Duddington, Reece Dunn, and the eSpeak NG Community. eSpeak NG: Speech synthesiser. [https://github.com/espeak-ng/](https://github.com/espeak-ng/) espeak-ng. Accessed: March 15, 2025. [↗](https://github.com/espeak-ng/)
18. Sefik Emre Eskimez, Xiaofei Wang, Manthan Thakker, Canrun Li, Chung-Hsien Tsai, Zhen Xiao, Hemin Yang, Zirun Zhu, Min Tang, Xu Tan, Yanqing Liu, Sheng Zhao, and Naoyuki Kanda. 2024. E2 tts: Embarrassingly easy fully non-autoregressive zero-shot tts. 2024 IEEE Spoken Language Technology Workshop (SLT… [find](https://scholar.google.com/scholar?q=Sefik+Emre+Eskimez%2C+Xiaofei+Wang%2C+Manthan+Thakker%2C+Canrun+Li%2C+Chung%2DHsien+Tsai%2C+Zhen+Xiao%2C+Hemin+Yang%2C+Zirun+Zhu%2C+Min+Tang%2C+Xu+Tan%2C+Yanqing+Liu%2C+Sheng+Zhao%2C+and+Naoyuki+Kanda%2E+2024%2E+E2+tts%3A+Embarrassi)
19. Qingkai Fang, Shoutao Guo, Yan Zhou, Zhengrui Ma, Shaolei Zhang, and Yang Feng. 2024. Llama-omni: Seamless speech interaction with large language models. ArXiv, abs/2409.06666. [find](https://scholar.google.com/scholar?q=Qingkai+Fang%2C+Shoutao+Guo%2C+Yan+Zhou%2C+Zhengrui+Ma%2C+Shaolei+Zhang%2C+and+Yang+Feng%2E+2024%2E+Llama%2Domni%3A+Seamless+speech+interaction+with+large+language+models%2E+ArXiv%2C+abs%2F2409%2E06666%2E)
20. Hao-Han Guo, Kun Liu, Fei-Yu Shen, Yi-Chen Wu, Fenglong Xie, Kun Xie, and Kai-Tuo Xu. 2024a. Fireredtts: A foundation text-to-speech framework for industry-level generative speech applications. ArXiv, abs/2409.03283. — [arXiv:2409.03283](https://arxiv.org/abs/2409.03283) [find](https://scholar.google.com/scholar?q=Hao%2DHan+Guo%2C+Kun+Liu%2C+Fei%2DYu+Shen%2C+Yi%2DChen+Wu%2C+Fenglong+Xie%2C+Kun+Xie%2C+and+Kai%2DTuo+Xu%2E+2024a%2E+Fireredtts%3A+A+foundation+text%2Dto%2Dspeech+framework+for+industry%2Dlevel+generative+speech+applications%2E+ArXiv%2C)
21. Haohan Guo, Fenglong Xie, Dongchao Yang, Xixin Wu, and Helen M. Meng. 2024b. Speaking from coarse to fine: Improving neural codec language model via multi-scale speech coding and generation. ArXiv, abs/2409.11630. — [arXiv:2409.11630](https://arxiv.org/abs/2409.11630) [find](https://scholar.google.com/scholar?q=Haohan+Guo%2C+Fenglong+Xie%2C+Dongchao+Yang%2C+Xixin+Wu%2C+and+Helen+M%2E+Meng%2E+2024b%2E+Speaking+from+coarse+to+fine%3A+Improving+neural+codec+language+model+via+multi%2Dscale+speech+coding+and+generation%2E+ArXiv%2C+ab)
22. Zhifang Guo, Yichong Leng, Yihan Wu, Sheng Zhao, and Xuejiao Tan. 2022. Prompttts: Controllable text-to-speech with text descriptions. [find](https://scholar.google.com/scholar?q=Zhifang+Guo%2C+Yichong+Leng%2C+Yihan+Wu%2C+Sheng+Zhao%2C+and+Xuejiao+Tan%2E+2022%2E+Prompttts%3A+Controllable+text%2Dto%2Dspeech+with+text+descriptions%2E)
23. ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. [find](https://scholar.google.com/scholar?q=ICASSP+2023+%2D+2023+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Processing+%28ICASSP%29%2C+pages+1%E2%80%935%2E)
24. Bing Han, Long Zhou, Shujie Liu, Sanyuan Chen, Lingwei Meng, Yanmin Qian, Yanqing Liu, Sheng Zhao, Jinyu Li, and Furu Wei. 2024. Vall-e r: Robust and efficient zero-shot text-to-speech synthesis via monotonic alignment. ArXiv, abs/2406.07855. — [arXiv:2406.07855](https://arxiv.org/abs/2406.07855) [find](https://scholar.google.com/scholar?q=Bing+Han%2C+Long+Zhou%2C+Shujie+Liu%2C+Sanyuan+Chen%2C+Lingwei+Meng%2C+Yanmin+Qian%2C+Yanqing+Liu%2C+Sheng+Zhao%2C+Jinyu+Li%2C+and+Furu+Wei%2E+2024%2E+Vall%2De+r%3A+Robust+and+efficient+zero%2Dshot+text%2Dto%2Dspeech+synthesis+via+m)
25. Haorui He, Zengqiang Shang, Chaoren Wang, Xuyuan Li, Yicheng Gu, Hua Hua, Liwei Liu, Chen Yang, Jiaqi Li, Peiyang Shi, Yuancheng Wang, Kai Chen, Pengyuan Zhang, and Zhizheng Wu. 2024. Emilia: An extensive, multilingual, and diverse speech dataset for large-scale speech generation. 2024 IEEE Spoken L… [find](https://scholar.google.com/scholar?q=Haorui+He%2C+Zengqiang+Shang%2C+Chaoren+Wang%2C+Xuyuan+Li%2C+Yicheng+Gu%2C+Hua+Hua%2C+Liwei+Liu%2C+Chen+Yang%2C+Jiaqi+Li%2C+Peiyang+Shi%2C+Yuancheng+Wang%2C+Kai+Chen%2C+Pengyuan+Zhang%2C+and+Zhizheng+Wu%2E+2024%2E+Emilia%3A+An+exten)
26. Yuchen Hu, Chen Chen, Siyin Wang, Chng Eng Siong, and Chao Zhang. 2024. Robust zero-shot text-to-speech synthesis with reverse inference optimization. ArXiv, abs/2407.02243. — [arXiv:2407.02243](https://arxiv.org/abs/2407.02243) [find](https://scholar.google.com/scholar?q=Yuchen+Hu%2C+Chen+Chen%2C+Siyin+Wang%2C+Chng+Eng+Siong%2C+and+Chao+Zhang%2E+2024%2E+Robust+zero%2Dshot+text%2Dto%2Dspeech+synthesis+with+reverse+inference+optimization%2E+ArXiv%2C+abs%2F2407%2E02243%2E+%E2%80%94+%5BarXiv%3A2407%2E02243%5D%28https)
27. Shehzeen Samarah Hussain, Paarth Neekhara, Xuesong Yang, Edresson Casanova, Subhankar Ghosh, Mikyas T. Desta, Roy Fejgin, Rafael Valle, and Jason Li. 2025. Koel-tts: Enhancing llm based speech generation with preference alignment and classifier free guidance. [find](https://scholar.google.com/scholar?q=Shehzeen+Samarah+Hussain%2C+Paarth+Neekhara%2C+Xuesong+Yang%2C+Edresson+Casanova%2C+Subhankar+Ghosh%2C+Mikyas+T%2E+Desta%2C+Roy+Fejgin%2C+Rafael+Valle%2C+and+Jason+Li%2E+2025%2E+Koel%2Dtts%3A+Enhancing+llm+based+speech+generat)
28. Shengpeng Ji, Jia li Zuo, Minghui Fang, Ziyue Jiang, Feiyang Chen, Xinyu Duan, Baoxing Huai, and Zhou Zhao. 2023. Textrolspeech: A text style control speech corpus with codec language text-to-speech models. ArXiv, abs/2308.14430. — [arXiv:2308.14430](https://arxiv.org/abs/2308.14430) [find](https://scholar.google.com/scholar?q=Shengpeng+Ji%2C+Jia+li+Zuo%2C+Minghui+Fang%2C+Ziyue+Jiang%2C+Feiyang+Chen%2C+Xinyu+Duan%2C+Baoxing+Huai%2C+and+Zhou+Zhao%2E+2023%2E+Textrolspeech%3A+A+text+style+control+speech+corpus+with+codec+language+text%2Dto%2Dspeech+m)
29. Ziyue Jiang, Jinglin Liu, Yi Ren, Jinzheng He, Zhenhui Ye, Shengpeng Ji, Qian Yang, Chen Zhang, Pengfei Wei, Chunfeng Wang, Xiang Yin, Zejun MA, and Zhou Zhao. 2024. Mega-TTS 2: Boosting prompting mechanisms for zero-shot speech synthesis. In The Twelfth International Conference on Learning Represen… [find](https://scholar.google.com/scholar?q=Ziyue+Jiang%2C+Jinglin+Liu%2C+Yi+Ren%2C+Jinzheng+He%2C+Zhenhui+Ye%2C+Shengpeng+Ji%2C+Qian+Yang%2C+Chen+Zhang%2C+Pengfei+Wei%2C+Chunfeng+Wang%2C+Xiang+Yin%2C+Zejun+MA%2C+and+Zhou+Zhao%2E+2024%2E+Mega%2DTTS+2%3A+Boosting+prompting+mec)
30. Wei Kang, Xiaoyu Yang, Zengwei Yao, Fangjun Kuang, Yifan Yang, Liyong Guo, Long Lin, and Daniel Povey. 2023. Libriheavy: A 50,000 hours asr corpus with punctuation casing and context. [find](https://scholar.google.com/scholar?q=Wei+Kang%2C+Xiaoyu+Yang%2C+Zengwei+Yao%2C+Fangjun+Kuang%2C+Yifan+Yang%2C+Liyong+Guo%2C+Long+Lin%2C+and+Daniel+Povey%2E+2023%2E+Libriheavy%3A+A+50%2C000+hours+asr+corpus+with+punctuation+casing+and+context%2E)
31. ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 10991–10995. [find](https://scholar.google.com/scholar?q=ICASSP+2024+%2D+2024+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Processing+%28ICASSP%29%2C+pages+10991%E2%80%9310995%2E)
32. Eugene Kharitonov, Damien Vincent, Zalán Borsos, Raphaël Marinier, Sertan Girgin, Olivier Pietquin, Matthew Sharifi, Marco Tagliasacchi, and Neil Zeghidour. 2023. [Speak, read and prompt: High-fidelity text-to-speech with minimal supervision](https://md.research.elamur.ai/free/en/papers/kharitonov-2023-speak-read-prompt-high-fidelity/). Transactions of the Association for Computational Linguist…
33. Sungwon Kim, Kevin J. Shih, Rohan Badlani, João Felipe Santos, Evelina Bakhturina, Mikyas T. Desta, Rafael Valle, Sungroh Yoon, and Bryan Catanzaro. 2023. P-flow: A fast and data-efficient zero-shot tts through speech prompting. In Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Sungwon+Kim%2C+Kevin+J%2E+Shih%2C+Rohan+Badlani%2C+Jo%C3%A3o+Felipe+Santos%2C+Evelina+Bakhturina%2C+Mikyas+T%2E+Desta%2C+Rafael+Valle%2C+Sungroh+Yoon%2C+and+Bryan+Catanzaro%2E+2023%2E+P%2Dflow%3A+A+fast+and+data%2Defficient+zero%2Dshot+t)
34. Felix Kreuk, Gabriel Synnaeve, Adam Polyak, Uriel Singer, Alexandre Defossez, Jade Copet, Devi Parikh, Yaniv Taigman, and Yossi Adi. 2022. Audiogen: Textually guided audio generation. ArXiv, abs/2209.15352. [find](https://scholar.google.com/scholar?q=Felix+Kreuk%2C+Gabriel+Synnaeve%2C+Adam+Polyak%2C+Uriel+Singer%2C+Alexandre+Defossez%2C+Jade+Copet%2C+Devi+Parikh%2C+Yaniv+Taigman%2C+and+Yossi+Adi%2E+2022%2E+Audiogen%3A+Textually+guided+audio+generation%2E+ArXiv%2C+abs%2F2209%2E)
35. Mateusz Lajszczak, Guillermo Cámbara, Yang Li, Fatih Beyhan, Arent van Korlaar, Fan Yang, Arnaud Joly, Álvaro Martín-Cortinas, Ammar Abbas, Adam Michalski, Alexis Moinet, Sri Karlapati, Ewa Muszy’nska, Haohan Guo, Bartosz Putrycz, Soledad López Gambino, Kayeon Yoo, Elena Sokolova, and Thomas Drugman… — [arXiv:2402.08093](https://arxiv.org/abs/2402.08093) [find](https://scholar.google.com/scholar?q=Mateusz+Lajszczak%2C+Guillermo+C%C3%A1mbara%2C+Yang+Li%2C+Fatih+Beyhan%2C+Arent+van+Korlaar%2C+Fan+Yang%2C+Arnaud+Joly%2C+%C3%81lvaro+Mart%C3%ADn%2DCortinas%2C+Ammar+Abbas%2C+Adam+Michalski%2C+Alexis+Moinet%2C+Sri+Karlapati%2C+Ewa+Muszy%E2%80%99nska)
36. Kushal Lakhotia, Evgeny Kharitonov, Wei-Ning Hsu, Yossi Adi, Adam Polyak, Benjamin Bolte, Tu Nguyen, Jade Copet, Alexei Baevski, Adel Ben Mohamed, and Emmanuel Dupoux. 2021. [On generative spoken language modeling from raw audio](https://md.research.elamur.ai/free/en/papers/lakhotia-2021-generative-spoken-language-modeling-raw/). Transactions of the Association for Computational Linguistics, 9:1336–1…
37. Matt Le, Apoorv Vyas, Bowen Shi, Brian Karrer, Leda Sari, Rashel Moritz, Mary Williamson, Vimal Manohar, Yossi Adi, Jay Mahadeokar, and Wei-Ning Hsu. 2023. Voicebox: Text-guided multilingual universal speech generation at scale. ArXiv, abs/2306.15687. — [arXiv:2306.15687](https://arxiv.org/abs/2306.15687) [find](https://scholar.google.com/scholar?q=Matt+Le%2C+Apoorv+Vyas%2C+Bowen+Shi%2C+Brian+Karrer%2C+Leda+Sari%2C+Rashel+Moritz%2C+Mary+Williamson%2C+Vimal+Manohar%2C+Yossi+Adi%2C+Jay+Mahadeokar%2C+and+Wei%2DNing+Hsu%2E+2023%2E+Voicebox%3A+Text%2Dguided+multilingual+universal)
38. Keon Lee, Dong Won Kim, Jaehyeon Kim, Seungjun Chung, and Jaewoong Cho. 2024. Ditto-tts: Diffusion transformers for scalable text-to-speech without domain-specific factors. [find](https://scholar.google.com/scholar?q=Keon+Lee%2C+Dong+Won+Kim%2C+Jaehyeon+Kim%2C+Seungjun+Chung%2C+and+Jaewoong+Cho%2E+2024%2E+Ditto%2Dtts%3A+Diffusion+transformers+for+scalable+text%2Dto%2Dspeech+without+domain%2Dspecific+factors%2E)
39. Yichong Leng, Zhifang Guo, Kai Shen, Xu Tan, Zeqian Ju, Yanqing Liu, Yufei Liu, Dongchao Yang, Leying Zhang, Kaitao Song, Lei He, Xiang-Yang Li, Sheng Zhao, Tao Qin, and Jiang Bian. 2023. Prompttts 2: Describing and generating voices with text prompt. ArXiv, abs/2309.02285. — [arXiv:2309.02285](https://arxiv.org/abs/2309.02285) [find](https://scholar.google.com/scholar?q=Yichong+Leng%2C+Zhifang+Guo%2C+Kai+Shen%2C+Xu+Tan%2C+Zeqian+Ju%2C+Yanqing+Liu%2C+Yufei+Liu%2C+Dongchao+Yang%2C+Leying+Zhang%2C+Kaitao+Song%2C+Lei+He%2C+Xiang%2DYang+Li%2C+Sheng+Zhao%2C+Tao+Qin%2C+and+Jiang+Bian%2E+2023%2E+Prompttts+2%3A)
40. Bohan Li, Hankun Wang, Situo Zhang, Yiwei Guo, and Kai Yu. 2025. Fast and high-quality autoregressive speech synthesis via speculative decoding. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE. [find](https://scholar.google.com/scholar?q=Bohan+Li%2C+Hankun+Wang%2C+Situo+Zhang%2C+Yiwei+Guo%2C+and+Kai+Yu%2E+2025%2E+Fast+and+high%2Dquality+autoregressive+speech+synthesis+via+speculative+decoding%2E+In+ICASSP+2025%2D2025+IEEE+International+Conference+on+Ac)
41. Yingahao Aaron Li, Rithesh Kumar, and Zeyu Jin. 2024. Dmospeech: Direct metric optimization via distilled diffusion model in zero-shot speech synthesis. [find](https://scholar.google.com/scholar?q=Yingahao+Aaron+Li%2C+Rithesh+Kumar%2C+and+Zeyu+Jin%2E+2024%2E+Dmospeech%3A+Direct+metric+optimization+via+distilled+diffusion+model+in+zero%2Dshot+speech+synthesis%2E)
42. Guanghou Liu, Yongmao Zhang, Yinjiao Lei, Yunlin Chen, Rui Wang, Zhifei Li, and Linfu Xie. 2023. Promptstyle: Controllable style transfer for text-to-speech with natural language descriptions. ArXiv, abs/2305.19522. — [arXiv:2305.19522](https://arxiv.org/abs/2305.19522) [find](https://scholar.google.com/scholar?q=Guanghou+Liu%2C+Yongmao+Zhang%2C+Yinjiao+Lei%2C+Yunlin+Chen%2C+Rui+Wang%2C+Zhifei+Li%2C+and+Linfu+Xie%2E+2023%2E+Promptstyle%3A+Controllable+style+transfer+for+text%2Dto%2Dspeech+with+natural+language+descriptions%2E+ArXiv%2C+)
43. Philipos C Loizou. 2011. Speech quality assessment. In Multimedia analysis, processing and communications, pages 623–654. Springer. [find](https://scholar.google.com/scholar?q=Philipos+C+Loizou%2E+2011%2E+Speech+quality+assessment%2E+In+Multimedia+analysis%2C+processing+and+communications%2C+pages+623%E2%80%93654%2E+Springer%2E)
44. Daniel Lyth and Simon King. 2024. Natural language guidance of high-fidelity text-to-speech with synthetic annotations. ArXiv, abs/2402.01912. — [arXiv:2402.01912](https://arxiv.org/abs/2402.01912) [find](https://scholar.google.com/scholar?q=Daniel+Lyth+and+Simon+King%2E+2024%2E+Natural+language+guidance+of+high%2Dfidelity+text%2Dto%2Dspeech+with+synthetic+annotations%2E+ArXiv%2C+abs%2F2402%2E01912%2E+%E2%80%94+%5BarXiv%3A2402%2E01912%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2402%2E01912%29)
45. Ziyang Ma, Ya-Zhen Song, Chenpeng Du, Jian Cong, Zhuo Chen, Yuping Wang, Yuxuan Wang, and Xie Chen. 2024. Language model can listen while speaking. ArXiv, abs/2408.02622. — [arXiv:2408.02622](https://arxiv.org/abs/2408.02622) [find](https://scholar.google.com/scholar?q=Ziyang+Ma%2C+Ya%2DZhen+Song%2C+Chenpeng+Du%2C+Jian+Cong%2C+Zhuo+Chen%2C+Yuping+Wang%2C+Yuxuan+Wang%2C+and+Xie+Chen%2E+2024%2E+Language+model+can+listen+while+speaking%2E+ArXiv%2C+abs%2F2408%2E02622%2E+%E2%80%94+%5BarXiv%3A2408%2E02622%5D%28https%3A%2F%2F)
46. Soumi Maiti, Yifan Peng, Shukjae Choi, Jee weon Jung, Xuankai Chang, and Shinji Watanabe. 2023. Voxtlm: Unified decoder-only models for consolidating speech recognition, synthesis and speech, text continuation tasks. [find](https://scholar.google.com/scholar?q=Soumi+Maiti%2C+Yifan+Peng%2C+Shukjae+Choi%2C+Jee+weon+Jung%2C+Xuankai+Chang%2C+and+Shinji+Watanabe%2E+2023%2E+Voxtlm%3A+Unified+decoder%2Donly+models+for+consolidating+speech+recognition%2C+synthesis+and+speech%2C+text+con)
47. ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 13326–13330. [find](https://scholar.google.com/scholar?q=ICASSP+2024+%2D+2024+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Processing+%28ICASSP%29%2C+pages+13326%E2%80%9313330%2E)
48. Paarth Neekhara, Shehzeen Samarah Hussain, Subhankar Ghosh, Jason Li, Rafael Valle, Rohan Badlani, and Boris Ginsburg. 2024. Improving robustness of llm-based speech synthesis by learning monotonic alignment. ArXiv, abs/2406.17957. — [arXiv:2406.17957](https://arxiv.org/abs/2406.17957) [find](https://scholar.google.com/scholar?q=Paarth+Neekhara%2C+Shehzeen+Samarah+Hussain%2C+Subhankar+Ghosh%2C+Jason+Li%2C+Rafael+Valle%2C+Rohan+Badlani%2C+and+Boris+Ginsburg%2E+2024%2E+Improving+robustness+of+llm%2Dbased+speech+synthesis+by+learning+monotonic+al)
49. Tan Dat Nguyen, Ji-Hoon Kim, Jeongsoo Choi, Shukjae Choi, Jinseok Park, Younglo Lee, and Joon Son Chung. 2025. Accelerating codec-based speech synthesis with multi-token prediction and speculative decoding. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing… [find](https://scholar.google.com/scholar?q=Tan+Dat+Nguyen%2C+Ji%2DHoon+Kim%2C+Jeongsoo+Choi%2C+Shukjae+Choi%2C+Jinseok+Park%2C+Younglo+Lee%2C+and+Joon+Son+Chung%2E+2025%2E+Accelerating+codec%2Dbased+speech+synthesis+with+multi%2Dtoken+prediction+and+speculative+dec)
50. Yuto Nishimura, Takumi Hirose, Masanari Ohi, Hideki Nakayama, and Nakamasa Inoue. 2024. Hall-e: Hierarchical neural codec language model for minute-long zero-shot text-to-speech synthesis. ArXiv, abs/2410.04380. — [arXiv:2410.04380](https://arxiv.org/abs/2410.04380) [find](https://scholar.google.com/scholar?q=Yuto+Nishimura%2C+Takumi+Hirose%2C+Masanari+Ohi%2C+Hideki+Nakayama%2C+and+Nakamasa+Inoue%2E+2024%2E+Hall%2De%3A+Hierarchical+neural+codec+language+model+for+minute%2Dlong+zero%2Dshot+text%2Dto%2Dspeech+synthesis%2E+ArXiv%2C+abs%2F)
51. Puyuan Peng, Po-Yao Huang, Shang-Wen Li, Abdelrahman Mohamed, and David Harwath. 2024. Voicecraft: Zero-shot speech editing and text-to-speech in the wild. In Annual Meeting of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Puyuan+Peng%2C+Po%2DYao+Huang%2C+Shang%2DWen+Li%2C+Abdelrahman+Mohamed%2C+and+David+Harwath%2E+2024%2E+Voicecraft%3A+Zero%2Dshot+speech+editing+and+text%2Dto%2Dspeech+in+the+wild%2E+In+Annual+Meeting+of+the+Association+for+Com)
52. Ofir Press, Noah A. Smith, and Mike Lewis. 2022. Train short, test long: Attention with linear biases enables input length extrapolation. Preprint, arXiv:2108.12409. [find](https://scholar.google.com/scholar?q=Ofir+Press%2C+Noah+A%2E+Smith%2C+and+Mike+Lewis%2E+2022%2E+Train+short%2C+test+long%3A+Attention+with+linear+biases+enables+input+length+extrapolation%2E+Preprint%2C+arXiv%3A2108%2E12409%2E)
53. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2022. [Robust speech recognition via large-scale weak supervision](https://md.research.elamur.ai/free/en/papers/radford-2022-robust-speech-recognition-large-scale/). ArXiv, abs/2212.04356.
54. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2023. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). Preprint, arXiv:1910.10683.
55. Takaaki Saeki, Detai Xin, Wataru Nakata, Tomoki Koriyama, Shinnosuke Takamichi, and Hiroshi Saruwatari. 2022. Utmos: Utokyo-sarulab system for voicemos challenge 2022. Preprint, arXiv:2204.02152. [find](https://scholar.google.com/scholar?q=Takaaki+Saeki%2C+Detai+Xin%2C+Wataru+Nakata%2C+Tomoki+Koriyama%2C+Shinnosuke+Takamichi%2C+and+Hiroshi+Saruwatari%2E+2022%2E+Utmos%3A+Utokyo%2Dsarulab+system+for+voicemos+challenge+2022%2E+Preprint%2C+arXiv%3A2204%2E02152%2E)
56. Kai Shen, Zeqian Ju, Xu Tan, Yanqing Liu, Yichong Leng, Lei He, Tao Qin, Sheng Zhao, and Jiang Bian. 2023. Naturalspeech 2: Latent diffusion models are natural and zero-shot speech and singing synthesizers. ArXiv, abs/2304.09116. — [arXiv:2304.09116](https://arxiv.org/abs/2304.09116) [find](https://scholar.google.com/scholar?q=Kai+Shen%2C+Zeqian+Ju%2C+Xu+Tan%2C+Yanqing+Liu%2C+Yichong+Leng%2C+Lei+He%2C+Tao+Qin%2C+Sheng+Zhao%2C+and+Jiang+Bian%2E+2023%2E+Naturalspeech+2%3A+Latent+diffusion+models+are+natural+and+zero%2Dshot+speech+and+singing+synthes)
57. Yakun Song, Zhuo Chen, Xiaofei Wang, Ziyang Ma, and Xie Chen. 2024. Ella-v: Stable neural codec language modeling with alignment-guided sequence reordering. [find](https://scholar.google.com/scholar?q=Yakun+Song%2C+Zhuo+Chen%2C+Xiaofei+Wang%2C+Ziyang+Ma%2C+and+Xie+Chen%2E+2024%2E+Ella%2Dv%3A+Stable+neural+codec+language+modeling+with+alignment%2Dguided+sequence+reordering%2E)
58. Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Yunfeng Liu. 2023. [Roformer: Enhanced transformer with rotary position embedding](https://md.research.elamur.ai/free/en/papers/su-2021-roformer-enhanced-transformer-rotary-position/). Preprint, arXiv:2104.09864.
59. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2023. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). Preprint, arXiv:1706.03762.
60. Ashish Vaswani, Noam M. Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Neural Information Processing Systems.
61. Apoorv Vyas, Bowen Shi, Matt Le, Andros Tjandra, Yi-Chiao Wu, Baishan Guo, Jiemin Zhang, Xinyue Zhang, Robert Adkins, W.K.F. Ngan, Jeff Wang, Ivan Cruz, Bapi Akula, Akinniyi Tunde Akinyemi, Brian Ellis, Rashel Moritz, Yael Yungster, Alice Rakotoarison, Liang Tan, and 5 others. 2023. Audiobox: Unifie… — [arXiv:2312.15821](https://arxiv.org/abs/2312.15821) [find](https://scholar.google.com/scholar?q=Apoorv+Vyas%2C+Bowen+Shi%2C+Matt+Le%2C+Andros+Tjandra%2C+Yi%2DChiao+Wu%2C+Baishan+Guo%2C+Jiemin+Zhang%2C+Xinyue+Zhang%2C+Robert+Adkins%2C+W%2EK%2EF%2E+Ngan%2C+Jeff+Wang%2C+Ivan+Cruz%2C+Bapi+Akula%2C+Akinniyi+Tunde+Akinyemi%2C+Brian+Elli)
62. Chengyi Wang, Sanyuan Chen, Yu Wu, Zi-Hua Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, Lei He, Sheng Zhao, and Furu Wei. 2023a. [Neural codec language models are zero-shot text to speech synthesizers](https://md.research.elamur.ai/free/en/papers/wang-2023-neural-codec-language-models-zero/). ArXiv, abs/2301.02111.
63. Hankun Wang, Chenpeng Du, Yiwei Guo, Shuai Wang, Xie Chen, and Kai Yu. 2024a. Attention-constrained inference for robust decoder-only text-to-speech. 2024 IEEE Spoken Language Technology Workshop (SLT), pages 630–637. [find](https://scholar.google.com/scholar?q=Hankun+Wang%2C+Chenpeng+Du%2C+Yiwei+Guo%2C+Shuai+Wang%2C+Xie+Chen%2C+and+Kai+Yu%2E+2024a%2E+Attention%2Dconstrained+inference+for+robust+decoder%2Donly+text%2Dto%2Dspeech%2E+2024+IEEE+Spoken+Language+Technology+Workshop+%28SLT)
64. Helin Wang, Meng Yu, Jiarui Hai, Chen Chen, Yuchen Hu, Rilin Chen, Najim Dehak, and Dong Yu. 2024b. Ssr-speech: Towards stable, safe and robust zero-shot text-based speech editing and synthesis. ArXiv, abs/2409.07556. — [arXiv:2409.07556](https://arxiv.org/abs/2409.07556) [find](https://scholar.google.com/scholar?q=Helin+Wang%2C+Meng+Yu%2C+Jiarui+Hai%2C+Chen+Chen%2C+Yuchen+Hu%2C+Rilin+Chen%2C+Najim+Dehak%2C+and+Dong+Yu%2E+2024b%2E+Ssr%2Dspeech%3A+Towards+stable%2C+safe+and+robust+zero%2Dshot+text%2Dbased+speech+editing+and+synthesis%2E+ArXiv)
65. Jiaming Wang, Zhihao Du, Qian Chen, Yunfei Chu, Zhifu Gao, Zerui Li, Kai Hu, Xiaohuan Zhou, Jin Xu, Ziyang Ma, Wen Wang, Siqi Zheng, Chang Zhou, Zhijie Yan, and Shiliang Zhang. 2023b. Lauragpt: Listen, attend, understand, and regenerate audio with gpt. ArXiv, abs/2310.04673. — [arXiv:2310.04673](https://arxiv.org/abs/2310.04673) [find](https://scholar.google.com/scholar?q=Jiaming+Wang%2C+Zhihao+Du%2C+Qian+Chen%2C+Yunfei+Chu%2C+Zhifu+Gao%2C+Zerui+Li%2C+Kai+Hu%2C+Xiaohuan+Zhou%2C+Jin+Xu%2C+Ziyang+Ma%2C+Wen+Wang%2C+Siqi+Zheng%2C+Chang+Zhou%2C+Zhijie+Yan%2C+and+Shiliang+Zhang%2E+2023b%2E+Lauragpt%3A+Listen)
66. Tianrui Wang, Long Zhou, Zi-Hua Zhang, Yu Wu, Shujie Liu, Yashesh Gaur, Zhuo Chen, Jinyu Li, and Furu Wei. 2023c. Viola: Unified codec language models for speech recognition, synthesis, and translation. ArXiv, abs/2305.16107. — [arXiv:2305.16107](https://arxiv.org/abs/2305.16107) [find](https://scholar.google.com/scholar?q=Tianrui+Wang%2C+Long+Zhou%2C+Zi%2DHua+Zhang%2C+Yu+Wu%2C+Shujie+Liu%2C+Yashesh+Gaur%2C+Zhuo+Chen%2C+Jinyu+Li%2C+and+Furu+Wei%2E+2023c%2E+Viola%3A+Unified+codec+language+models+for+speech+recognition%2C+synthesis%2C+and+translatio)
67. Xinsheng Wang, Mingqi Jiang, Ziyang Ma, Ziyu Zhang, Songxiang Liu, Linqin Li, Zheng Liang, Qixi Zheng, Rui Wang, Xiaoqin Feng, Weizhen Bian, Zhen Ye, Sitong Cheng, Ruibin Yuan, Zhixian Zhao, Xinfa Zhu, Jiahao Pan, Liumeng Xue, Pengcheng Zhu, and 6 others. 2025a. Spark-tts: An efficient llm-based tex… [find](https://scholar.google.com/scholar?q=Xinsheng+Wang%2C+Mingqi+Jiang%2C+Ziyang+Ma%2C+Ziyu+Zhang%2C+Songxiang+Liu%2C+Linqin+Li%2C+Zheng+Liang%2C+Qixi+Zheng%2C+Rui+Wang%2C+Xiaoqin+Feng%2C+Weizhen+Bian%2C+Zhen+Ye%2C+Sitong+Cheng%2C+Ruibin+Yuan%2C+Zhixian+Zhao%2C+Xinfa+Zhu)
68. Yuancheng Wang, Haoyue Zhan, Liwei Liu, Ruihong Zeng, Haotian Guo, Jiachen Zheng, Qiang Zhang, Xueyao Zhang, Shunsi Zhang, and Zhizheng Wu. 2024c. Maskgct: Zero-shot text-to-speech with masked generative codec transformer. ArXiv, abs/2409.00750. — [arXiv:2409.00750](https://arxiv.org/abs/2409.00750) [find](https://scholar.google.com/scholar?q=Yuancheng+Wang%2C+Haoyue+Zhan%2C+Liwei+Liu%2C+Ruihong+Zeng%2C+Haotian+Guo%2C+Jiachen+Zheng%2C+Qiang+Zhang%2C+Xueyao+Zhang%2C+Shunsi+Zhang%2C+and+Zhizheng+Wu%2E+2024c%2E+Maskgct%3A+Zero%2Dshot+text%2Dto%2Dspeech+with+masked+generat)
69. Yuancheng Wang, Jiachen Zheng, Junan Zhang, Xueyao Zhang, Huan Liao, and Zhizheng Wu. 2025b. Metis: A foundation speech generation model with masked generative pre-training. [find](https://scholar.google.com/scholar?q=Yuancheng+Wang%2C+Jiachen+Zheng%2C+Junan+Zhang%2C+Xueyao+Zhang%2C+Huan+Liao%2C+and+Zhizheng+Wu%2E+2025b%2E+Metis%3A+A+foundation+speech+generation+model+with+masked+generative+pre%2Dtraining%2E)
70. Yihan Wu, Soumi Maiti, Yifan Peng, Wangyou Zhang, Chenda Li, Yuyue Wang, Xihua Wang, Shinji Watanabe, and Ruihua Song. 2024. Speechcomposer: Unifying multiple speech tasks with prompt composition. ArXiv, abs/2401.18045. — [arXiv:2401.18045](https://arxiv.org/abs/2401.18045) [find](https://scholar.google.com/scholar?q=Yihan+Wu%2C+Soumi+Maiti%2C+Yifan+Peng%2C+Wangyou+Zhang%2C+Chenda+Li%2C+Yuyue+Wang%2C+Xihua+Wang%2C+Shinji+Watanabe%2C+and+Ruihua+Song%2E+2024%2E+Speechcomposer%3A+Unifying+multiple+speech+tasks+with+prompt+composition%2E+ArX)
71. Detai Xin, Xu Tan, Kai Shen, Zeqian Ju, Dongchao Yang, Yuancheng Wang, Shinnosuke Takamichi, Hiroshi Saruwatari, Shujie Liu, Jinyu Li, and Sheng Zhao. 2024. Rall-e: Robust codec language modeling with chain-of-thought prompting for text-to-speech synthesis. ArXiv, abs/2404.03204. — [arXiv:2404.03204](https://arxiv.org/abs/2404.03204) [find](https://scholar.google.com/scholar?q=Detai+Xin%2C+Xu+Tan%2C+Kai+Shen%2C+Zeqian+Ju%2C+Dongchao+Yang%2C+Yuancheng+Wang%2C+Shinnosuke+Takamichi%2C+Hiroshi+Saruwatari%2C+Shujie+Liu%2C+Jinyu+Li%2C+and+Sheng+Zhao%2E+2024%2E+Rall%2De%3A+Robust+codec+language+modeling+with)
72. Wang Xu, Shuo Wang, Weilin Zhao, Xu Han, Yukun Yan, Yudi Zhang, Zhe Tao, Zhiyuan Liu, and Wanxiang Che. 2024. Enabling real-time conversations with minimal training costs. ArXiv, abs/2409.11727. [find](https://scholar.google.com/scholar?q=Wang+Xu%2C+Shuo+Wang%2C+Weilin+Zhao%2C+Xu+Han%2C+Yukun+Yan%2C+Yudi+Zhang%2C+Zhe+Tao%2C+Zhiyuan+Liu%2C+and+Wanxiang+Che%2E+2024%2E+Enabling+real%2Dtime+conversations+with+minimal+training+costs%2E+ArXiv%2C+abs%2F2409%2E11727%2E)
73. Dongchao Yang, Songxiang Liu, Rongjie Huang, Guangzhi Lei, Chao Weng, Helen M. Meng, and Dong Yu. 2023. Instructtts: Modelling expressive tts in discrete latent space with natural language style prompt. ArXiv, abs/2301.13662. — [arXiv:2301.13662](https://arxiv.org/abs/2301.13662) [find](https://scholar.google.com/scholar?q=Dongchao+Yang%2C+Songxiang+Liu%2C+Rongjie+Huang%2C+Guangzhi+Lei%2C+Chao+Weng%2C+Helen+M%2E+Meng%2C+and+Dong+Yu%2E+2023%2E+Instructtts%3A+Modelling+expressive+tts+in+discrete+latent+space+with+natural+language+style+promp)
74. Zengwei Yao, Liyong Guo, Xiaoyu Yang, Wei Kang, Fangjun Kuang, Yifan Yang, Zengrui Jin, Long Lin, and Daniel Povey. 2024. Zipformer: A faster and better encoder for automatic speech recognition. In ICLR. [find](https://scholar.google.com/scholar?q=Zengwei+Yao%2C+Liyong+Guo%2C+Xiaoyu+Yang%2C+Wei+Kang%2C+Fangjun+Kuang%2C+Yifan+Yang%2C+Zengrui+Jin%2C+Long+Lin%2C+and+Daniel+Povey%2E+2024%2E+Zipformer%3A+A+faster+and+better+encoder+for+automatic+speech+recognition%2E+In+IC)
75. Zhen Ye, Xinfa Zhu, Chi min Chan, Xinsheng Wang, Xu Tan, Jiahe Lei, Yi Peng, Haohe Liu, Yizhu Jin, Zheqi Dai, Hongzhan Lin, Jianyi Chen, Xingjian Du, Liumeng Xue, Yunlin Chen, Zhifei Li, Lei Xie, Qiuqiang Kong, Yi-Ting Guo, and Wei Xue. 2025. Llasa: Scaling train-time and inference-time compute for… [find](https://scholar.google.com/scholar?q=Zhen+Ye%2C+Xinfa+Zhu%2C+Chi+min+Chan%2C+Xinsheng+Wang%2C+Xu+Tan%2C+Jiahe+Lei%2C+Yi+Peng%2C+Haohe+Liu%2C+Yizhu+Jin%2C+Zheqi+Dai%2C+Hongzhan+Lin%2C+Jianyi+Chen%2C+Xingjian+Du%2C+Liumeng+Xue%2C+Yunlin+Chen%2C+Zhifei+Li%2C+Lei+Xie%2C+Qiuq)
76. Wenyi Yu, Siyin Wang, Xiaoyu Yang, Xianzhao Chen, Xiaohai Tian, Jun Zhang, Guangzhi Sun, Lu Lu, Yuxuan Wang, and Chao Zhang. 2024. Salmonn-omni: A codec-free llm for full-duplex speech understanding and generation. ArXiv, abs/2411.18138. — [arXiv:2411.18138](https://arxiv.org/abs/2411.18138) [find](https://scholar.google.com/scholar?q=Wenyi+Yu%2C+Siyin+Wang%2C+Xiaoyu+Yang%2C+Xianzhao+Chen%2C+Xiaohai+Tian%2C+Jun+Zhang%2C+Guangzhi+Sun%2C+Lu+Lu%2C+Yuxuan+Wang%2C+and+Chao+Zhang%2E+2024%2E+Salmonn%2Domni%3A+A+codec%2Dfree+llm+for+full%2Dduplex+speech+understanding+a)
77. Neil Zeghidour, Alejandro Luebs, Ahmed Omran, Jan Skoglund, and Marco Tagliasacchi. 2021. [Soundstream: An end-to-end neural audio codec](https://md.research.elamur.ai/free/en/papers/zeghidour-2021-soundstream-end-end-neural-audio/). IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30:495–507.
78. Hao Zhang, Weiwei Li, Rilin Chen, Vinay Kothapally, Meng Yu, and Dong Yu. 2025. Llm-enhanced dialogue management for full-duplex spoken dialogue systems. [find](https://scholar.google.com/scholar?q=Hao+Zhang%2C+Weiwei+Li%2C+Rilin+Chen%2C+Vinay+Kothapally%2C+Meng+Yu%2C+and+Dong+Yu%2E+2025%2E+Llm%2Denhanced+dialogue+management+for+full%2Dduplex+spoken+dialogue+systems%2E)
79. Xin Zhang, Xiang Lyu, Zhihao Du, Qian Chen, Dong Zhang, Hangrui Hu, Chaohong Tan, Tianyu Zhao, Yuxuan Wang, Bin Zhang, Heng Lu, Yaqian Zhou, and Xipeng Qiu. 2024. Intrinsicvoice: Empowering llms with intrinsic real-time voice interaction abilities. ArXiv, abs/2410.08035. — [arXiv:2410.08035](https://arxiv.org/abs/2410.08035) [find](https://scholar.google.com/scholar?q=Xin+Zhang%2C+Xiang+Lyu%2C+Zhihao+Du%2C+Qian+Chen%2C+Dong+Zhang%2C+Hangrui+Hu%2C+Chaohong+Tan%2C+Tianyu+Zhao%2C+Yuxuan+Wang%2C+Bin+Zhang%2C+Heng+Lu%2C+Yaqian+Zhou%2C+and+Xipeng+Qiu%2E+2024%2E+Intrinsicvoice%3A+Empowering+llms+with+)
80. Zi-Hua Zhang, Long Zhou, Chengyi Wang, Sanyuan Chen, Yu Wu, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, Lei He, Sheng Zhao, and Furu Wei. 2023. Speak foreign languages with your own voice: Cross-lingual neural codec language modeling. ArXiv, abs/2303.03926. — [arXiv:2303.03926](https://arxiv.org/abs/2303.03926) [find](https://scholar.google.com/scholar?q=Zi%2DHua+Zhang%2C+Long+Zhou%2C+Chengyi+Wang%2C+Sanyuan+Chen%2C+Yu+Wu%2C+Shujie+Liu%2C+Zhuo+Chen%2C+Yanqing+Liu%2C+Huaming+Wang%2C+Jinyu+Li%2C+Lei+He%2C+Sheng+Zhao%2C+and+Furu+Wei%2E+2023%2E+Speak+foreign+languages+with+your+own+vo)
81. Zhisheng Zheng, Puyuan Peng, Anuj Diwan, Cong Phuoc Huynh, Xiaohang Sun, Zhu Liu, Vimal Bhat, and David Harwath. 2025. Voicecraft-x: Unifying multilingual, voice-cloning speech synthesis and speech editing. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pa… [find](https://scholar.google.com/scholar?q=Zhisheng+Zheng%2C+Puyuan+Peng%2C+Anuj+Diwan%2C+Cong+Phuoc+Huynh%2C+Xiaohang+Sun%2C+Zhu+Liu%2C+Vimal+Bhat%2C+and+David+Harwath%2E+2025%2E+Voicecraft%2Dx%3A+Unifying+multilingual%2C+voice%2Dcloning+speech+synthesis+and+speech+ed)
82. Yongxin Zhu, Dan Su, Liqiang He, Linli Xu, and Dong Yu. 2024. Generative pre-trained speech language model with efficient hierarchical transformer. ArXiv, abs/2406.00976. [find](https://scholar.google.com/scholar?q=Yongxin+Zhu%2C+Dan+Su%2C+Liqiang+He%2C+Linli+Xu%2C+and+Dong+Yu%2E+2024%2E+Generative+pre%2Dtrained+speech+language+model+with+efficient+hierarchical+transformer%2E+ArXiv%2C+abs%2F2406%2E00976%2E)

*Updated 1 November 2026*