---
title: "Autoregressive Speech Synthesis without Vector Quantization"
description: "The method replaces cross-entropy with a regression objective, adding a spectrogram… 1.47% WER. 44 references, cited by 15 papers here."
authors: ["Lingwei Meng", "Long Zhou", "Shujie Liu", "Sanyuan Chen", "Bing Han", "Shujie Hu", "Yanqing Liu", "Jinyu Li", "Sheng Zhao", "Xixin Wu", "Helen M. Meng", "Furu Wei"]
year: 2025
venue: "Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2025.acl-long.65/"
doi: "10.18653/v1/2025.acl-long.65"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/meng-2025-autoregressive-speech-synthesis-without-vector/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/meng-2025-autoregressive-speech-synthesis-without-vector/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/4/) / Autoregressive Speech Synthesis without Vector Quantization

# Autoregressive Speech Synthesis without Vector Quantization

**Authors:** Lingwei Meng, [Long Zhou](https://md.research.elamur.ai/free/en/authors/long-zhou/), [Shujie Liu](https://md.research.elamur.ai/free/en/authors/shujie-liu/), [Sanyuan Chen](https://md.research.elamur.ai/free/en/authors/sanyuan-chen/), [Bing Han](https://md.research.elamur.ai/free/en/authors/bing-han/), Shujie Hu, [Yanqing Liu](https://md.research.elamur.ai/free/en/authors/yanqing-liu/), [Jinyu Li](https://md.research.elamur.ai/free/en/authors/jinyu-li/), [Sheng Zhao](https://md.research.elamur.ai/free/en/authors/sheng-zhao/), [Xixin Wu](https://md.research.elamur.ai/free/en/authors/xixin-wu/), [Helen M. Meng](https://md.research.elamur.ai/free/en/authors/helen-meng/) and [Furu Wei](https://md.research.elamur.ai/free/en/authors/furu-wei/) · **Year:** 2025 · **Venue:** [Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2025](https://md.research.elamur.ai/free/en/venues/acl/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.acl-long.65/) · [PDF](https://aclanthology.org/2025.acl-long.65.pdf) · [DOI 10.18653/v1/2025.acl-long.65](https://doi.org/10.18653/v1/2025.acl-long.65) · [OpenAlex](https://openalex.org/W4412886819)

**Citations:** 10 citations (OpenAlex, 2026-10-14) · [15](https://md.research.elamur.ai/free/en/papers/meng-2025-autoregressive-speech-synthesis-without-vector/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Codec-based zero-shot TTS systems such as VALL-E represent speech as vector-quantized tokens. These tokens lose fidelity compared with continuous features. Sampling from them can also produce silences or noise loops, and the systems need a two-pass AR+NAR decoder. MELLE removes quantization entirely: a single decoder-only language model predicts continuous mel-spectrogram frames one step at a time, conditioned on BPE text and a speech prompt. A vocoder then turns the frames into audio.

The method replaces cross-entropy with a regression objective, adding a spectrogram flux loss that discourages repetitive frames. A latent sampling module based on variational inference restores randomness in continuous space. An optional reduction factor lets the model emit several frames per step.

The model was trained on Libriheavy (50K hours) and LibriSpeech (960 hours) and tested zero-shot on LibriSpeech test-clean. It matches VALL-E 2 on objective metrics and beats it in listening tests, while being faster. This suggests continuous tokens are a simpler, viable alternative to codec codes for autoregressive TTS.

## Key points
- On the continuation task, MELLE reaches 1.47% WER, lower than the ground-truth recordings (1.61%). That is 47.9% lower in relative terms than VALL-E and 8.1% lower than VALL-E 2.
- In subjective tests, MELLE scores MOS 4.20 (ground truth 4.29) and CMOS −0.032 against ground truth. Its SMOS of 4.40 exceeds the 3.94 given to real speech.
- Generating 10 s of audio takes 7.32 s for VALL-E/VALL-E 2 and 5.49 s for MELLE. With reduction factors 2 and 4, MELLE-R2 needs 2.76 s and MELLE-R4 needs 1.40 s.
- In ablations, both latent sampling and spectrogram flux loss improve robustness and speaker similarity, with the largest gains on the cross-sentence task. Latent sampling mainly helps preserve the prompt speaker's voice.
- The authors list three limitations: output quality is bounded by the vocoder, evaluation is English-only, and mel-spectrograms are the only continuous target tested. A Mandarin MELLE was also trained and is shown on the demo page.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/) — Company, Redmond, United States · 378 works · 161,630 citations
- [Chinese University of Hong Kong](https://md.research.elamur.ai/free/en/universities/chinese-university-of-hong-kong/) — University, Hong Kong, China · 159 works · 16,154 citations

📚 Shelves (7): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,649 works · [💭 Experimental and Cognitive Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/experimental-and-cognitive-psychology/) · 930 works · [🔊 Signal Processing](https://md.research.elamur.ai/free/en/shelves/computer-science/signal-processing/) · 493 works

🧭 Topics (2): [🎙️ Speech & spoken language](https://md.research.elamur.ai/free/en/topics/speech/) · 519 works · [🪶 Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/) · 297 works

## Cite

Cite as: Lingwei Meng, Long Zhou, Shujie Liu, Sanyuan Chen, Bing Han, Shujie Hu, Yanqing Liu, Jinyu Li, Sheng Zhao, Xixin Wu, Helen M. Meng and Furu Wei. 2025. Autoregressive Speech Synthesis without Vector Quantization. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{meng2025autoregressive,
  title = {{Autoregressive Speech Synthesis without Vector Quantization}},
  author = {Lingwei Meng, Long Zhou, Shujie Liu, Sanyuan Chen, Bing Han, Shujie Hu, Yanqing Liu, Jinyu Li, Sheng Zhao, Xixin Wu, Helen M. Meng and Furu Wei},
  booktitle = {Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.acl-long.65},
  url = {https://doi.org/10.18653/v1/2025.acl-long.65}
}
```

- [← previous Automatic Speech Recognition Biases in Newcastle English: an Error Analysis](https://md.research.elamur.ai/free/en/papers/serditova-2025-automatic-speech-recognition-biases-newcastle/)
- [next → Do Large Language Models have an English Accent? Evaluating and Improving the Naturalness of Multilingual LLMs](https://md.research.elamur.ai/free/en/papers/guo-2025-do-large-language-models-have/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/4/)

## References

44 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Philip Anastassiou, Jiawei Chen, Jitong Chen, Yuanzhe Chen, Zhuo Chen, Ziyi Chen, Jian Cong, Lelai Deng, Chuang Ding, Lu Gao, et al. 2024. [Seed-TTS: A family of high-quality versatile speech generation models](https://md.research.elamur.ai/free/en/papers/anastassiou-2024-seed-tts-family-high/). arXiv preprint arXiv:2406.02430.
2. Junyi Ao, Rui Wang, Long Zhou, Chengyi Wang, Shuo Ren, Yu Wu, Shujie Liu, Tom Ko, Qing Li, Yu Zhang, Zhihua Wei, Yao Qian, Jinyu Li, and Furu Wei. 2022. [SpeechT5: Unified-modal encoder-decoder pre-training for spoken language processing](https://md.research.elamur.ai/free/en/papers/ao-2022-speecht5-unified-modal-encoder-decoder/). In Proceedings of the 60th Annual Meeting of the Association f…
3. He Bai, Tatiana Likhomanenko, Ruixiang Zhang, Zijin Gu, Zakaria Aldeneh, and Navdeep Jaitly. 2024. dMel: Speech tokenization made simple. arXiv preprint arXiv:2407.15835. — [arXiv:2407.15835](https://arxiv.org/abs/2407.15835) [find](https://scholar.google.com/scholar?q=He+Bai%2C+Tatiana+Likhomanenko%2C+Ruixiang+Zhang%2C+Zijin+Gu%2C+Zakaria+Aldeneh%2C+and+Navdeep+Jaitly%2E+2024%2E+dMel%3A+Speech+tokenization+made+simple%2E+arXiv+preprint+arXiv%3A2407%2E15835%2E+%E2%80%94+%5BarXiv%3A2407%2E15835%5D%28https%3A%2F%2F)
4. James Betker. 2023. Better speech synthesis through scaling. arXiv preprint arXiv:2305.07243. — [arXiv:2305.07243](https://arxiv.org/abs/2305.07243) [find](https://scholar.google.com/scholar?q=James+Betker%2E+2023%2E+Better+speech+synthesis+through+scaling%2E+arXiv+preprint+arXiv%3A2305%2E07243%2E+%E2%80%94+%5BarXiv%3A2305%2E07243%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2305%2E07243%29)
5. Zalán Borsos, Matt Sharifi, Damien Vincent, Eugene Kharitonov, Neil Zeghidour, and Marco Tagliasacchi. 2023. SoundStorm: Efficient parallel audio generation. arXiv preprint arXiv:2305.09636. — [arXiv:2305.09636](https://arxiv.org/abs/2305.09636) [find](https://scholar.google.com/scholar?q=Zal%C3%A1n+Borsos%2C+Matt+Sharifi%2C+Damien+Vincent%2C+Eugene+Kharitonov%2C+Neil+Zeghidour%2C+and+Marco+Tagliasacchi%2E+2023%2E+SoundStorm%3A+Efficient+parallel+audio+generation%2E+arXiv+preprint+arXiv%3A2305%2E09636%2E+%E2%80%94+%5BarXiv%3A)
6. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems, 33:1877–1901.
7. Edresson Casanova, Julian Weber, Christopher D Shulby, Arnaldo Candido Junior, Eren Gölge, and Moacir A Ponti. 2022. YourTTS: Towards zero-shot multi-speaker TTS and zero-shot voice conversion for everyone. In International Conference on Machine Learning, pages 2709–2720. [find](https://scholar.google.com/scholar?q=Edresson+Casanova%2C+Julian+Weber%2C+Christopher+D+Shulby%2C+Arnaldo+Candido+Junior%2C+Eren+G%C3%B6lge%2C+and+Moacir+A+Ponti%2E+2022%2E+YourTTS%3A+Towards+zero%2Dshot+multi%2Dspeaker+TTS+and+zero%2Dshot+voice+conversion+for+eve)
8. Liang Chen, Zekun Wang, Shuhuai Ren, Lei Li, Haozhe Zhao, Yunshui Li, Zefan Cai, Hongcheng Guo, Lei Zhang, Yizhe Xiong, et al. 2024a. Next token prediction towards multimodal intelligence: A comprehensive survey. arXiv preprint arXiv:2412.18619. — [arXiv:2412.18619](https://arxiv.org/abs/2412.18619) [find](https://scholar.google.com/scholar?q=Liang+Chen%2C+Zekun+Wang%2C+Shuhuai+Ren%2C+Lei+Li%2C+Haozhe+Zhao%2C+Yunshui+Li%2C+Zefan+Cai%2C+Hongcheng+Guo%2C+Lei+Zhang%2C+Yizhe+Xiong%2C+et+al%2E+2024a%2E+Next+token+prediction+towards+multimodal+intelligence%3A+A+comprehen)
9. Sanyuan Chen, Shujie Liu, Long Zhou, Yanqing Liu, Xu Tan, Jinyu Li, Sheng Zhao, Yao Qian, and Furu Wei. 2024b. VALL-E 2: Neural codec language models are human parity zero-shot text to speech synthesizers. arXiv preprint arXiv:2406.05370. — [arXiv:2406.05370](https://arxiv.org/abs/2406.05370) [find](https://scholar.google.com/scholar?q=Sanyuan+Chen%2C+Shujie+Liu%2C+Long+Zhou%2C+Yanqing+Liu%2C+Xu+Tan%2C+Jinyu+Li%2C+Sheng+Zhao%2C+Yao+Qian%2C+and+Furu+Wei%2E+2024b%2E+VALL%2DE+2%3A+Neural+codec+language+models+are+human+parity+zero%2Dshot+text+to+speech+synthesi)
10. Sanyuan Chen, Chengyi Wang, Zhengyang Chen, Yu Wu, Shujie Liu, Zhuo Chen, Jinyu Li, Naoyuki Kanda, Takuya Yoshioka, Xiong Xiao, et al. 2022. WavLM: Large-scale self-supervised pre-training for full stack speech processing. IEEE Journal of Selected Topics in Signal Processing, 16(6):1505–1518. [find](https://scholar.google.com/scholar?q=Sanyuan+Chen%2C+Chengyi+Wang%2C+Zhengyang+Chen%2C+Yu+Wu%2C+Shujie+Liu%2C+Zhuo+Chen%2C+Jinyu+Li%2C+Naoyuki+Kanda%2C+Takuya+Yoshioka%2C+Xiong+Xiao%2C+et+al%2E+2022%2E+WavLM%3A+Large%2Dscale+self%2Dsupervised+pre%2Dtraining+for+full+st)
11. Alexandre Défossez, Jade Copet, Gabriel Synnaeve, and Yossi Adi. 2023. [High fidelity neural audio compression](https://md.research.elamur.ai/free/en/papers/defossez-2022-high-fidelity-neural-audio-compression/). Transactions on Machine Learning Research. Featured Certification, Reproducibility Certification.
12. Zhihao Du, Qian Chen, Shiliang Zhang, Kai Hu, Heng Lu, Yexin Yang, Hangrui Hu, Siqi Zheng, Yue Gu, Ziyang Ma, et al. 2024. [CosyVoice: A scalable multilingual zero-shot text-to-speech synthesizer based on supervised semantic tokens](https://md.research.elamur.ai/free/en/papers/du-2024-cosyvoice-scalable-multilingual-zero-shot/). arXiv preprint arXiv:2407.05407.
13. Sefik Emre Eskimez, Xiaofei Wang, Manthan Thakker, Canrun Li, Chung-Hsien Tsai, Zhen Xiao, Hemin Yang, Zirun Zhu, Min Tang, Xu Tan, et al. 2024. E2 TTS: Embarrassingly easy fully non-autoregressive zero-shot TTS. arXiv preprint arXiv:2406.18009. — [arXiv:2406.18009](https://arxiv.org/abs/2406.18009) [find](https://scholar.google.com/scholar?q=Sefik+Emre+Eskimez%2C+Xiaofei+Wang%2C+Manthan+Thakker%2C+Canrun+Li%2C+Chung%2DHsien+Tsai%2C+Zhen+Xiao%2C+Hemin+Yang%2C+Zirun+Zhu%2C+Min+Tang%2C+Xu+Tan%2C+et+al%2E+2024%2E+E2+TTS%3A+Embarrassingly+easy+fully+non%2Dautoregressive+ze)
14. Anmol Gulati, James Qin, Chung-Cheng Chiu, Niki Parmar, Yu Zhang, Jiahui Yu, Wei Han, Shibo Wang, Zhengdong Zhang, Yonghui Wu, and Ruoming Pang. 2020. [Conformer: Convolution-augmented transformer for speech recognition](https://md.research.elamur.ai/free/en/papers/gulati-2020-conformer-convolution-augmented-transformer-speech/). In Proc.
15. Interspeech 2020, pages 5036–5040. [find](https://scholar.google.com/scholar?q=Interspeech+2020%2C+pages+5036%E2%80%935040%2E)
16. Bing Han, Long Zhou, Shujie Liu, Sanyuan Chen, Lingwei Meng, Yanming Qian, Yanqing Liu, Sheng Zhao, Jinyu Li, and Furu Wei. 2024. VALL-E R: Robust and efficient zero-shot text-to-speech synthesis via monotonic alignment. arXiv preprint arXiv:2406.07855. — [arXiv:2406.07855](https://arxiv.org/abs/2406.07855) [find](https://scholar.google.com/scholar?q=Bing+Han%2C+Long+Zhou%2C+Shujie+Liu%2C+Sanyuan+Chen%2C+Lingwei+Meng%2C+Yanming+Qian%2C+Yanqing+Liu%2C+Sheng+Zhao%2C+Jinyu+Li%2C+and+Furu+Wei%2E+2024%2E+VALL%2DE+R%3A+Robust+and+efficient+zero%2Dshot+text%2Dto%2Dspeech+synthesis+via+)
17. Wei-Ning Hsu, Benjamin Bolte, Yao-Hung Hubert Tsai, Kushal Lakhotia, Ruslan Salakhutdinov, and Abdelrahman Mohamed. 2021. [HuBERT: Self-supervised speech representation learning by masked prediction of hidden units](https://md.research.elamur.ai/free/en/papers/hsu-2021-hubert-self-supervised-speech-representation/). IEEE/ACM Trans. Audio, Speech and Lang. Proc., 29:3451–3460.
18. Ziyue Jiang, Yi Ren, Zhenhui Ye, Jinglin Liu, Chen Zhang, Qian Yang, Shengpeng Ji, Rongjie Huang, Chunfeng Wang, Xiang Yin, et al. 2023. Mega-TTS: Zero-shot text-to-speech at scale with intrinsic inductive bias. arXiv preprint arXiv:2306.03509. — [arXiv:2306.03509](https://arxiv.org/abs/2306.03509) [find](https://scholar.google.com/scholar?q=Ziyue+Jiang%2C+Yi+Ren%2C+Zhenhui+Ye%2C+Jinglin+Liu%2C+Chen+Zhang%2C+Qian+Yang%2C+Shengpeng+Ji%2C+Rongjie+Huang%2C+Chunfeng+Wang%2C+Xiang+Yin%2C+et+al%2E+2023%2E+Mega%2DTTS%3A+Zero%2Dshot+text%2Dto%2Dspeech+at+scale+with+intrinsic+indu)
19. Zeqian Ju, Yuancheng Wang, Kai Shen, Xu Tan, Detai Xin, Dongchao Yang, Yanqing Liu, Yichong Leng, Kaitao Song, Siliang Tang, et al. 2024. Naturalspeech 3: Zero-shot speech synthesis with factorized codec and diffusion models. arXiv preprint arXiv:2403.03100. — [arXiv:2403.03100](https://arxiv.org/abs/2403.03100) [find](https://scholar.google.com/scholar?q=Zeqian+Ju%2C+Yuancheng+Wang%2C+Kai+Shen%2C+Xu+Tan%2C+Detai+Xin%2C+Dongchao+Yang%2C+Yanqing+Liu%2C+Yichong+Leng%2C+Kaitao+Song%2C+Siliang+Tang%2C+et+al%2E+2024%2E+Naturalspeech+3%3A+Zero%2Dshot+speech+synthesis+with+factorized+co)
20. Wei Kang, Xiaoyu Yang, Zengwei Yao, Fangjun Kuang, Yifan Yang, Liyong Guo, Long Lin, and Daniel Povey. 2024. Libriheavy: a 50,000 hours ASR corpus with punctuation casing and context. In ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 10991–1… [find](https://scholar.google.com/scholar?q=Wei+Kang%2C+Xiaoyu+Yang%2C+Zengwei+Yao%2C+Fangjun+Kuang%2C+Yifan+Yang%2C+Liyong+Guo%2C+Long+Lin%2C+and+Daniel+Povey%2E+2024%2E+Libriheavy%3A+a+50%2C000+hours+ASR+corpus+with+punctuation+casing+and+context%2E+In+ICASSP+2024%2D2)
21. Jaehyeon Kim, Keon Lee, Seungjun Chung, and Jaewoong Cho. 2024. CLaM-TTS: Improving neural codec language model for zero-shot text-to-speech. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Jaehyeon+Kim%2C+Keon+Lee%2C+Seungjun+Chung%2C+and+Jaewoong+Cho%2E+2024%2E+CLaM%2DTTS%3A+Improving+neural+codec+language+model+for+zero%2Dshot+text%2Dto%2Dspeech%2E+In+The+Twelfth+International+Conference+on+Learning+Repres)
22. Diederik P Kingma and Max Welling. 2014. Autoencoding variational bayes. In The International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Diederik+P+Kingma+and+Max+Welling%2E+2014%2E+Autoencoding+variational+bayes%2E+In+The+International+Conference+on+Learning+Representations%2E)
23. Jungil Kong, Jaehyeon Kim, and Jaekyoung Bae. 2020. [HiFi-GAN: Generative adversarial networks for efficient and high fidelity speech synthesis](https://md.research.elamur.ai/free/en/papers/kong-2020-hifi-gan-generative-adversarial-networks/). In Advances in Neural Information Processing Systems, volume 33, pages 17022–17033.
24. Rithesh Kumar, Prem Seetharaman, Alejandro Luebs, Ishaan Kumar, and Kundan Kumar. 2023. [High-fidelity audio compression with improved RVQGAN](https://md.research.elamur.ai/free/en/papers/kumar-2023-high-fidelity-audio-compression-improved/). Advances in Neural Information Processing Systems, 36:27980–27993.
25. Mateusz Łajszczak, Guillermo Cámbara, Yang Li, Fatih Beyhan, Arent van Korlaar, Fan Yang, Arnaud Joly, Álvaro Martín-Cortinas, Ammar Abbas, Adam Michalski, et al. 2024. BASE TTS: Lessons from building a billion-parameter text-to-speech model on 100K hours of data. arXiv preprint arXiv:2402.08093. — [arXiv:2402.08093](https://arxiv.org/abs/2402.08093) [find](https://scholar.google.com/scholar?q=Mateusz+%C5%81ajszczak%2C+Guillermo+C%C3%A1mbara%2C+Yang+Li%2C+Fatih+Beyhan%2C+Arent+van+Korlaar%2C+Fan+Yang%2C+Arnaud+Joly%2C+%C3%81lvaro+Mart%C3%ADn%2DCortinas%2C+Ammar+Abbas%2C+Adam+Michalski%2C+et+al%2E+2024%2E+BASE+TTS%3A+Lessons+from+building)
26. Matthew Le, Apoorv Vyas, Bowen Shi, Brian Karrer, Leda Sari, Rashel Moritz, Mary Williamson, Vimal Manohar, Yossi Adi, Jay Mahadeokar, and Wei-Ning Hsu. 2023. Voicebox: Text-guided multilingual universal speech generation at scale. In Thirty-seventh Conference on Neural Information Processing System… [find](https://scholar.google.com/scholar?q=Matthew+Le%2C+Apoorv+Vyas%2C+Bowen+Shi%2C+Brian+Karrer%2C+Leda+Sari%2C+Rashel+Moritz%2C+Mary+Williamson%2C+Vimal+Manohar%2C+Yossi+Adi%2C+Jay+Mahadeokar%2C+and+Wei%2DNing+Hsu%2E+2023%2E+Voicebox%3A+Text%2Dguided+multilingual+univer)
27. Naihan Li, Shujie Liu, Yanqing Liu, Sheng Zhao, and Ming Liu. 2019. Neural speech synthesis with transformer network. Proceedings of the AAAI Conference on Artificial Intelligence, page 6706–6713. [find](https://scholar.google.com/scholar?q=Naihan+Li%2C+Shujie+Liu%2C+Yanqing+Liu%2C+Sheng+Zhao%2C+and+Ming+Liu%2E+2019%2E+Neural+speech+synthesis+with+transformer+network%2E+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+page+6706%E2%80%936713%2E)
28. Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He. 2024a. Autoregressive image generation without vector quantization. arXiv preprint arXiv:2406.11838. [find](https://scholar.google.com/scholar?q=Tianhong+Li%2C+Yonglong+Tian%2C+He+Li%2C+Mingyang+Deng%2C+and+Kaiming+He%2E+2024a%2E+Autoregressive+image+generation+without+vector+quantization%2E+arXiv+preprint+arXiv%3A2406%2E11838%2E)
29. Yinghao Aaron Li, Cong Han, Vinay Raghavan, Gavin Mischler, and Nima Mesgarani. 2024b. StyleTTS 2: Towards human-level text-to-speech through style diffusion and adversarial training with large speech language models. Advances in Neural Information Processing Systems, 36. [find](https://scholar.google.com/scholar?q=Yinghao+Aaron+Li%2C+Cong+Han%2C+Vinay+Raghavan%2C+Gavin+Mischler%2C+and+Nima+Mesgarani%2E+2024b%2E+StyleTTS+2%3A+Towards+human%2Dlevel+text%2Dto%2Dspeech+through+style+diffusion+and+adversarial+training+with+large+speech)
30. Zhijun Liu, Shuai Wang, Sho Inoue, Qibing Bai, and Haizhou Li. 2024. Autoregressive diffusion transformer for text-to-speech synthesis. arXiv preprint arXiv:2406.05551. — [arXiv:2406.05551](https://arxiv.org/abs/2406.05551) [find](https://scholar.google.com/scholar?q=Zhijun+Liu%2C+Shuai+Wang%2C+Sho+Inoue%2C+Qibing+Bai%2C+and+Haizhou+Li%2E+2024%2E+Autoregressive+diffusion+transformer+for+text%2Dto%2Dspeech+synthesis%2E+arXiv+preprint+arXiv%3A2406%2E05551%2E+%E2%80%94+%5BarXiv%3A2406%2E05551%5D%28https%3A%2F%2Far)
31. OpenAI. 2023. [GPT-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774.
32. Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. 2015. LibriSpeech: An ASR corpus based on public domain audio books. In ICASSP, pages 5206–5210. [find](https://scholar.google.com/scholar?q=Vassil+Panayotov%2C+Guoguo+Chen%2C+Daniel+Povey%2C+and+Sanjeev+Khudanpur%2E+2015%2E+LibriSpeech%3A+An+ASR+corpus+based+on+public+domain+audio+books%2E+In+ICASSP%2C+pages+5206%E2%80%935210%2E)
33. Krishna C. Puvvada, Nithin Rao Koluguri, Kunal Dhawan, Jagadeesh Balam, and Boris Ginsburg. 2024. Discrete audio representation as an alternative to mel-spectrograms for speaker and speech recognition. In ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (IC… [find](https://scholar.google.com/scholar?q=Krishna+C%2E+Puvvada%2C+Nithin+Rao+Koluguri%2C+Kunal+Dhawan%2C+Jagadeesh+Balam%2C+and+Boris+Ginsburg%2E+2024%2E+Discrete+audio+representation+as+an+alternative+to+mel%2Dspectrograms+for+speaker+and+speech+recognition)
34. Yi Ren, Yangjun Ruan, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, and Tie-Yan Liu. 2019. FastSpeech: Fast, robust and controllable text to speech. In NeurIPS, pages 3165–3174. [find](https://scholar.google.com/scholar?q=Yi+Ren%2C+Yangjun+Ruan%2C+Xu+Tan%2C+Tao+Qin%2C+Sheng+Zhao%2C+Zhou+Zhao%2C+and+Tie%2DYan+Liu%2E+2019%2E+FastSpeech%3A+Fast%2C+robust+and+controllable+text+to+speech%2E+In+NeurIPS%2C+pages+3165%E2%80%933174%2E)
35. Hubert Siuzdak. 2024. Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Hubert+Siuzdak%2E+2024%2E+Vocos%3A+Closing+the+gap+between+time%2Ddomain+and+Fourier%2Dbased+neural+vocoders+for+high%2Dquality+audio+synthesis%2E+In+The+Twelfth+International+Conference+on+Learning+Representations)
36. Yakun Song, Zhuo Chen, Xiaofei Wang, Ziyang Ma, and Xie Chen. 2024. ELLA-V: Stable neural codec language modeling with alignment-guided sequence reordering. arXiv preprint arXiv:2401.07333. — [arXiv:2401.07333](https://arxiv.org/abs/2401.07333) [find](https://scholar.google.com/scholar?q=Yakun+Song%2C+Zhuo+Chen%2C+Xiaofei+Wang%2C+Ziyang+Ma%2C+and+Xie+Chen%2E+2024%2E+ELLA%2DV%3A+Stable+neural+codec+language+modeling+with+alignment%2Dguided+sequence+reordering%2E+arXiv+preprint+arXiv%3A2401%2E07333%2E+%E2%80%94+%5BarXiv%3A2)
37. Michael Tschannen, Cian Eastwood, and Fabian Mentzer. 2023. GIVT: Generative infinite-vocabulary transformers. arXiv preprint arXiv:2312.02116. — [arXiv:2312.02116](https://arxiv.org/abs/2312.02116) [find](https://scholar.google.com/scholar?q=Michael+Tschannen%2C+Cian+Eastwood%2C+and+Fabian+Mentzer%2E+2023%2E+GIVT%3A+Generative+infinite%2Dvocabulary+transformers%2E+arXiv+preprint+arXiv%3A2312%2E02116%2E+%E2%80%94+%5BarXiv%3A2312%2E02116%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2312%2E02116%29)
38. Apoorv Vyas, Bowen Shi, Matthew Le, Andros Tjandra, Yi-Chiao Wu, Baishan Guo, Jiemin Zhang, Xinyue Zhang, Robert Adkins, William Ngan, et al. 2023. Audiobox: Unified audio generation with natural language prompts. arXiv preprint arXiv:2312.15821. — [arXiv:2312.15821](https://arxiv.org/abs/2312.15821) [find](https://scholar.google.com/scholar?q=Apoorv+Vyas%2C+Bowen+Shi%2C+Matthew+Le%2C+Andros+Tjandra%2C+Yi%2DChiao+Wu%2C+Baishan+Guo%2C+Jiemin+Zhang%2C+Xinyue+Zhang%2C+Robert+Adkins%2C+William+Ngan%2C+et+al%2E+2023%2E+Audiobox%3A+Unified+audio+generation+with+natural+lang)
39. Chengyi Wang, Sanyuan Chen, Yu Wu, Ziqiang Zhang, Long Zhou, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, et al. 2023. [Neural codec language models are zero-shot text to speech synthesizers](https://md.research.elamur.ai/free/en/papers/wang-2023-neural-codec-language-models-zero/). arXiv preprint arXiv:2301.02111.
40. Yuxuan Wang, R.J. Skerry-Ryan, Daisy Stanton, Yonghui Wu, Ron J. Weiss, Navdeep Jaitly, Zongheng Yang, Ying Xiao, Zhifeng Chen, Samy Bengio, Quoc Le, Yannis Agiomyrgiannakis, Rob Clark, and Rif A. Saurous. 2017. Tacotron: Towards end-to-end speech synthesis. In Proc. [find](https://scholar.google.com/scholar?q=Yuxuan+Wang%2C+R%2EJ%2E+Skerry%2DRyan%2C+Daisy+Stanton%2C+Yonghui+Wu%2C+Ron+J%2E+Weiss%2C+Navdeep+Jaitly%2C+Zongheng+Yang%2C+Ying+Xiao%2C+Zhifeng+Chen%2C+Samy+Bengio%2C+Quoc+Le%2C+Yannis+Agiomyrgiannakis%2C+Rob+Clark%2C+and+Rif+A%2E+Sau)
41. Interspeech 2017, pages 4006– 4010. [find](https://scholar.google.com/scholar?q=Interspeech+2017%2C+pages+4006%E2%80%93+4010%2E)
42. Detai Xin, Xu Tan, Kai Shen, Zeqian Ju, Dongchao Yang, Yuancheng Wang, Shinnosuke Takamichi, Hiroshi Saruwatari, Shujie Liu, Jinyu Li, et al. 2024. RALL-E: Robust codec language modeling with chain-of-thought prompting for text-to-speech synthesis. arXiv preprint arXiv:2404.03204. — [arXiv:2404.03204](https://arxiv.org/abs/2404.03204) [find](https://scholar.google.com/scholar?q=Detai+Xin%2C+Xu+Tan%2C+Kai+Shen%2C+Zeqian+Ju%2C+Dongchao+Yang%2C+Yuancheng+Wang%2C+Shinnosuke+Takamichi%2C+Hiroshi+Saruwatari%2C+Shujie+Liu%2C+Jinyu+Li%2C+et+al%2E+2024%2E+RALL%2DE%3A+Robust+codec+language+modeling+with+chain%2Dof)
43. Neil Zeghidour, Alejandro Luebs, Ahmed Omran, Jan Skoglund, and Marco Tagliasacchi. 2021. [SoundStream: An end-to-end neural audio codec](https://md.research.elamur.ai/free/en/papers/zeghidour-2021-soundstream-end-end-neural-audio/). IEEE/ACM Transactions on Audio, Speech, and Language Processing, 30:495–507.
44. Ziqiang Zhang, Long Zhou, Chengyi Wang, Sanyuan Chen, Yu Wu, Shujie Liu, Zhuo Chen, Yanqing Liu, Huaming Wang, Jinyu Li, et al. 2023. Speak foreign languages with your own voice: Cross-lingual neural codec language modeling. arXiv preprint arXiv:2303.03926. — [arXiv:2303.03926](https://arxiv.org/abs/2303.03926) [find](https://scholar.google.com/scholar?q=Ziqiang+Zhang%2C+Long+Zhou%2C+Chengyi+Wang%2C+Sanyuan+Chen%2C+Yu+Wu%2C+Shujie+Liu%2C+Zhuo+Chen%2C+Yanqing+Liu%2C+Huaming+Wang%2C+Jinyu+Li%2C+et+al%2E+2023%2E+Speak+foreign+languages+with+your+own+voice%3A+Cross%2Dlingual+neural+)

*Updated 2 November 2026*