---
title: "WaveNet: A Generative Model for Raw Audio — Oord 2016"
description: "WaveNet investigates whether autoregressive neural networks, previously successful for images and text, can model raw audio directly, a signal with at…"
authors: ["Aaron van den Oord", "Sander Dieleman", "Heiga Zen", "Karen Simonyan", "Oriol Vinyals", "Alex Graves", "Nal Kalchbrenner", "Andrew Senior", "Koray Kavukcuoglu"]
year: 2016
venue: "arXiv:1609.03499"
source: "https://arxiv.org/abs/1609.03499"
doi: "10.48550/arxiv.1609.03499"
arxiv: "1609.03499"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/oord-2016-wavenet-generative-model-raw-audio/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/oord-2016-wavenet-generative-model-raw-audio/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2016](https://md.research.elamur.ai/free/en/papers/2016/) / WaveNet: A Generative Model for Raw Audio

# WaveNet: A Generative Model for Raw Audio

**Authors:** [Aaron van den Oord](https://md.research.elamur.ai/free/en/authors/aaron-van-den-oord/), Sander Dieleman, [Heiga Zen](https://md.research.elamur.ai/free/en/authors/heiga-zen/), [Karen Simonyan](https://md.research.elamur.ai/free/en/authors/karen-simonyan/), [Oriol Vinyals](https://md.research.elamur.ai/free/en/authors/oriol-vinyals/), [Alex Graves](https://md.research.elamur.ai/free/en/authors/alex-graves/), [Nal Kalchbrenner](https://md.research.elamur.ai/free/en/authors/nal-kalchbrenner/), [Andrew Senior](https://md.research.elamur.ai/free/en/authors/andrew-senior/), [Koray Kavukcuoglu](https://md.research.elamur.ai/free/en/authors/koray-kavukcuoglu/) · **Year:** 2016 · **Venue:** arXiv:1609.03499 · **Licence:** [arXiv — non-exclusive licence to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1609.03499) · [PDF](https://arxiv.org/pdf/1609.03499) · [DOI 10.48550/arxiv.1609.03499](https://doi.org/10.48550/arxiv.1609.03499) · [arXiv 1609.03499](https://arxiv.org/abs/1609.03499) · [OpenAlex](https://openalex.org/W2519091744)

**Citations:** 4912 citations (openalex-snapshot · 2026-10-14) · [255](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
WaveNet investigates whether autoregressive neural networks, previously successful for images and text, can model raw audio directly, a signal with at least 16,000 values per second. Each sample is predicted from all earlier ones. Instead of recurrence, the network stacks causal convolutions whose dilation doubles at every layer and then restarts. This makes the receptive field grow exponentially with depth while computation stays cheap and training remains parallel. Audio is µ-law companded and quantized to 256 classes, so the output can be a softmax. Gated activations, residual connections and skip connections support deep stacks. Extra inputs steer generation in two ways: global conditioning (e.g., speaker identity) and local conditioning (e.g., upsampled linguistic features for TTS). In blind crowdsourced listening tests for North American English and Mandarin Chinese, WaveNet TTS was judged more natural than strong LSTM-based parametric and HMM-driven concatenative systems. A single network could also produce many voices, generate realistic music fragments, and serve as a phoneme recognizer. The work showed that learning waveforms end-to-end is practical, which reshaped speech synthesis.

## Key points
- **Architecture:** Dilation follows the pattern 1, 2, 4, …, 512, repeated several times. Each such block covers 1024 samples. Pooling is not used, so the output keeps the input's time resolution.
- **Output representation:** Audio is µ-law encoded with µ = 255 and quantized to 256 values (8-bit) rather than predicting 65,536 16-bit levels. The TTS models used 16 kHz, 8-bit µ-law audio without any post-processing.
- **Multi-speaker experiment:** A single model was trained on the VCTK corpus (44 hours, 109 speakers), conditioned only on a one-hot speaker ID. Without text input, it produces speech-like babble in distinct voices.
- **TTS evaluation:**
  - Two conditioning variants were compared: linguistic features only, and linguistic features plus F0.
  - The test set was 100 held-out sentences, with eight native-speaker raters per pair or stimulus.
  - Roughly 40% of ratings were discarded because the raters were not using headphones.
- **Other uses:** The same architecture produced novel musical fragments and gave promising phoneme recognition results.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Other](https://md.research.elamur.ai/free/en/shelves/other/)

Topics: [Speech & spoken language](https://md.research.elamur.ai/free/en/topics/speech/)

## Cited by

Cited by

255 papers of our library cite it (of the 140884 whose bibliographies we read); 22 of them have a page here, 233 not yet.

- [Representation Learning with Contrastive Predictive Coding](https://md.research.elamur.ai/free/en/papers/oord-2018-representation-learning-contrastive-predictive-coding/)
- [On the Opportunities and Risks of Foundation Models](https://md.research.elamur.ai/free/en/papers/bommasani-2021-opportunities-risks-foundation-models/)
- [Evaluating Large Language Models Trained on Code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://md.research.elamur.ai/free/en/papers/song-2019-generative-modeling-estimating-gradients-data/)
- [SoundStream: An End-to-End Neural Audio Codec](https://md.research.elamur.ai/free/en/papers/zeghidour-2021-soundstream-end-end-neural-audio/)
- [Dissecting Contextual Word Embeddings: Architecture and Representation](https://md.research.elamur.ai/free/en/papers/peters-2018-dissecting-contextual-word-embeddings-architecture/)
- [Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement](https://md.research.elamur.ai/free/en/papers/lee-2018-deterministic-non-autoregressive-neural-sequence/)
- [Fast and Accurate Entity Recognition with Iterated Dilated Convolutions](https://md.research.elamur.ai/free/en/papers/strubell-2017-fast-accurate-entity-recognition-iterated/)
- [Denoising Diffusion Implicit Models](https://md.research.elamur.ai/free/en/papers/song-2020-denoising-diffusion-implicit-models/)
- [High Fidelity Neural Audio Compression](https://md.research.elamur.ai/free/en/papers/defossez-2022-high-fidelity-neural-audio-compression/)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://md.research.elamur.ai/free/en/papers/perez-2017-film-visual-reasoning-general-conditioning/)
- [Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers](https://md.research.elamur.ai/free/en/papers/wang-2023-neural-codec-language-models-zero/)
- [Compressive Transformers for Long-Range Sequence Modelling](https://md.research.elamur.ai/free/en/papers/rae-2019-compressive-transformers-long-range-sequence/)
- [Text and Code Embeddings by Contrastive Pre-Training](https://md.research.elamur.ai/free/en/papers/neelakantan-2022-text-code-embeddings-contrastive-pre/)
- [Structured Denoising Diffusion Models in Discrete State-Spaces](https://md.research.elamur.ai/free/en/papers/austin-2021-structured-denoising-diffusion-models-discrete/)
- [Taskmaster-1: Toward a Realistic and Diverse Dialog Dataset](https://md.research.elamur.ai/free/en/papers/byrne-2019-taskmaster-1-realistic-diverse-dialog/)
- [Tensor2Tensor for Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/vaswani-2018-tensor2tensor-neural-machine-translation/)
- [On Generative Spoken Language Modeling from Raw Audio](https://md.research.elamur.ai/free/en/papers/lakhotia-2021-generative-spoken-language-modeling-raw/)
- [Moshi: a speech-text foundation model for real-time dialogue](https://md.research.elamur.ai/free/en/papers/defossez-2024-moshi-speech-text-foundation-model/)
- [Arabic Text-To-Speech (TTS) Data Preparation](https://md.research.elamur.ai/free/en/papers/masri-2022-arabic-text-speech-tts-data/)
- [MazzikaAI: A knowledge-based performance-to-prompt compiler for real-time Arabic maqam accompaniment with a streaming text-to-music model](https://md.research.elamur.ai/free/en/papers/du-2026-mazzikaai-knowledge-based-performance-prompt/)
- [ArFake: A Robust Framework for Multi-Dialect Arabic Speech Spoofing Detection Benchmark](https://md.research.elamur.ai/free/en/papers/elsetohy-2025-arfake-robust-framework-multi-dialect/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{oord2016wavenet,
  title = {{WaveNet: A Generative Model for Raw Audio}},
  author = {Oord, Aaron van den and Dieleman, Sander and Zen, Heiga and Simonyan, Karen and Vinyals, Oriol and Graves, Alex and Kalchbrenner, Nal and Senior, Andrew and Kavukcuoglu, Koray},
  year = {2016},
  doi = {10.48550/arxiv.1609.03499},
  eprint = {1609.03499},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1609.03499}
}
```

- [← previous Hierarchical Attention Networks for Document Classification](https://md.research.elamur.ai/free/en/papers/yang-2016-hierarchical-attention-networks-document-classification/)
- [next → Neural Architecture Search with Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/zoph-2016-neural-architecture-search-reinforcement-learning/)
- [↑ back to papers of 2016](https://md.research.elamur.ai/free/en/papers/2016/)

## References

58 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Agiomyrgiannakis, Yannis. Vocaine the vocoder and applications is speech synthesis. In ICASSP, pp. 4230–4234, 2015. [find](https://scholar.google.com/scholar?q=Agiomyrgiannakis%2C+Yannis%2E+Vocaine+the+vocoder+and+applications+is+speech+synthesis%2E+In+ICASSP%2C+pp%2E+4230%E2%80%934234%2C+2015%2E)
2. Bishop, Christopher M. Mixture density networks. Technical Report NCRG/94/004, Neural Computing Research Group, Aston University, 1994. [find](https://scholar.google.com/scholar?q=Bishop%2C+Christopher+M%2E+Mixture+density+networks%2E+Technical+Report+NCRG%2F94%2F004%2C+Neural+Computing+Research+Group%2C+Aston+University%2C+1994%2E)
3. Chen, Liang-Chieh, Papandreou, George, Kokkinos, Iasonas, Murphy, Kevin, and Yuille, Alan L. Semantic image segmentation with deep convolutional nets and fully connected CRFs. In ICLR, 2015. URL http://arxiv.org/abs/1412.7062. — [arXiv:1412.7062](https://arxiv.org/abs/1412.7062) [↗](http://arxiv.org/abs/1412.7062)
4. Chiba, Tsutomu and Kajiyama, Masato. The Vowel: Its Nature and Structure. Tokyo-Kaiseikan, 1942. [find](https://scholar.google.com/scholar?q=Chiba%2C+Tsutomu+and+Kajiyama%2C+Masato%2E+The+Vowel%3A+Its+Nature+and+Structure%2E+Tokyo%2DKaiseikan%2C+1942%2E)
5. Dudley, Homer. Remaking speech. The Journal of the Acoustical Society of America, 11(2):169– 177, 1939. [find](https://scholar.google.com/scholar?q=Dudley%2C+Homer%2E+Remaking+speech%2E+The+Journal+of+the+Acoustical+Society+of+America%2C+11%282%29%3A169%E2%80%93+177%2C+1939%2E)
6. Dutilleux, Pierre. An implementation of the “algorithme à trous” to compute the wavelet transform. In Combes, Jean-Michel, Grossmann, Alexander, and Tchamitchian, Philippe (eds.), Wavelets: Time-Frequency Methods and Phase Space, pp. 298–304. Springer Berlin Heidelberg, 1989. [find](https://scholar.google.com/scholar?q=Dutilleux%2C+Pierre%2E+An+implementation+of+the+%E2%80%9Calgorithme+a%CC%80+trous%E2%80%9D+to+compute+the+wavelet+transform%2E+In+Combes%2C+Jean%2DMichel%2C+Grossmann%2C+Alexander%2C+and+Tchamitchian%2C+Philippe+%28eds%2E%29%2C+Wavelets%3A+Time%2DFreq)
7. Fan, Yuchen, Qian, Yao, and Xie, Feng-Long, Soong Frank K. TTS synthesis with bidirectional LSTM based recurrent neural networks. In Interspeech, pp. 1964–1968, 2014. [find](https://scholar.google.com/scholar?q=Fan%2C+Yuchen%2C+Qian%2C+Yao%2C+and+Xie%2C+Feng%2DLong%2C+Soong+Frank+K%2E+TTS+synthesis+with+bidirectional+LSTM+based+recurrent+neural+networks%2E+In+Interspeech%2C+pp%2E+1964%E2%80%931968%2C+2014%2E)
8. Fant, Gunnar. Acoustic Theory of Speech Production. Mouton De Gruyter, 1970. [find](https://scholar.google.com/scholar?q=Fant%2C+Gunnar%2E+Acoustic+Theory+of+Speech+Production%2E+Mouton+De+Gruyter%2C+1970%2E)
9. Garofolo, John S., Lamel, Lori F., Fisher, William M., Fiscus, Jonathon G., and Pallett, David S. DARPA TIMIT acoustic-phonetic continuous speech corpus CD-ROM. NIST speech disc 1-1.1. NASA STI/Recon technical report, 93, 1993. [find](https://scholar.google.com/scholar?q=Garofolo%2C+John+S%2E%2C+Lamel%2C+Lori+F%2E%2C+Fisher%2C+William+M%2E%2C+Fiscus%2C+Jonathon+G%2E%2C+and+Pallett%2C+David+S%2E+DARPA+TIMIT+acoustic%2Dphonetic+continuous+speech+corpus+CD%2DROM%2E+NIST+speech+disc+1%2D1%2E1%2E+NASA+STI%2FRecon+)
10. Gonzalvo, Xavi, Tazari, Siamak, Chan, Chun-an, Becker, Markus, Gutkin, Alexander, and Silen, Hanna. Recent advances in Google real-time HMM-driven unit selection synthesizer. In Interspeech, 2016. URL [http://research.google.com/pubs/pub45564.html](http://research.google.com/pubs/pub45564.html). [↗](http://research.google.com/pubs/pub45564.html)
11. He, Kaiming, Zhang, Xiangyu, Ren, Shaoqing, and Sun, Jian. Deep residual learning for image recognition. CoRR, abs/1512.03385, 2015. [find](https://scholar.google.com/scholar?q=He%2C+Kaiming%2C+Zhang%2C+Xiangyu%2C+Ren%2C+Shaoqing%2C+and+Sun%2C+Jian%2E+Deep+residual+learning+for+image+recognition%2E+CoRR%2C+abs%2F1512%2E03385%2C+2015%2E)
12. Hochreiter, S. and Schmidhuber, J. Long short-term memory. Neural Comput., 9(8):1735–1780, 1997. [find](https://scholar.google.com/scholar?q=Hochreiter%2C+S%2E+and+Schmidhuber%2C+J%2E+Long+short%2Dterm+memory%2E+Neural+Comput%2E%2C+9%288%29%3A1735%E2%80%931780%2C+1997%2E)
13. Holschneider, Matthias, Kronland-Martinet, Richard, Morlet, Jean, and Tchamitchian, Philippe. A real-time algorithm for signal analysis with the help of the wavelet transform. In Combes, Jean-Michel, Grossmann, Alexander, and Tchamitchian, Philippe (eds.), Wavelets: Time-Frequency Methods and Phase… [find](https://scholar.google.com/scholar?q=Holschneider%2C+Matthias%2C+Kronland%2DMartinet%2C+Richard%2C+Morlet%2C+Jean%2C+and+Tchamitchian%2C+Philippe%2E+A+real%2Dtime+algorithm+for+signal+analysis+with+the+help+of+the+wavelet+transform%2E+In+Combes%2C+Jean%2DMichel%2C+)
14. Hoshen, Yedid, Weiss, Ron J., and Wilson, Kevin W. Speech acoustic modeling from raw multi-channel waveforms. In ICASSP, pp. 4624–4628. IEEE, 2015. [find](https://scholar.google.com/scholar?q=Hoshen%2C+Yedid%2C+Weiss%2C+Ron+J%2E%2C+and+Wilson%2C+Kevin+W%2E+Speech+acoustic+modeling+from+raw+multi%2Dchannel+waveforms%2E+In+ICASSP%2C+pp%2E+4624%E2%80%934628%2E+IEEE%2C+2015%2E)
15. Hunt, Andrew J. and Black, Alan W. Unit selection in a concatenative speech synthesis system using a large speech database. In ICASSP, pp. 373–376, 1996. [find](https://scholar.google.com/scholar?q=Hunt%2C+Andrew+J%2E+and+Black%2C+Alan+W%2E+Unit+selection+in+a+concatenative+speech+synthesis+system+using+a+large+speech+database%2E+In+ICASSP%2C+pp%2E+373%E2%80%93376%2C+1996%2E)
16. Imai, Satoshi and Furuichi, Chieko. Unbiased estimation of log spectrum. In EURASIP, pp. 203– 206, 1988. [find](https://scholar.google.com/scholar?q=Imai%2C+Satoshi+and+Furuichi%2C+Chieko%2E+Unbiased+estimation+of+log+spectrum%2E+In+EURASIP%2C+pp%2E+203%E2%80%93+206%2C+1988%2E)
17. Itakura, Fumitada. Line spectrum representation of linear predictor coefficients of speech signals. The Journal of the Acoust. Society of America, 57(S1):S35–S35, 1975. [find](https://scholar.google.com/scholar?q=Itakura%2C+Fumitada%2E+Line+spectrum+representation+of+linear+predictor+coefficients+of+speech+signals%2E+The+Journal+of+the+Acoust%2E+Society+of+America%2C+57%28S1%29%3AS35%E2%80%93S35%2C+1975%2E)
18. Itakura, Fumitada and Saito, Shuzo. A statistical method for estimation of speech spectral density and formant frequencies. Trans. IEICE, J53A:35–42, 1970. [find](https://scholar.google.com/scholar?q=Itakura%2C+Fumitada+and+Saito%2C+Shuzo%2E+A+statistical+method+for+estimation+of+speech+spectral+density+and+formant+frequencies%2E+Trans%2E+IEICE%2C+J53A%3A35%E2%80%9342%2C+1970%2E)
19. ITU-T. Recommendation G. 711. Pulse Code Modulation (PCM) of voice frequencies, 1988. [find](https://scholar.google.com/scholar?q=ITU%2DT%2E+Recommendation+G%2E+711%2E+Pulse+Code+Modulation+%28PCM%29+of+voice+frequencies%2C+1988%2E)
20. Józefowicz, Rafal, Vinyals, Oriol, Schuster, Mike, Shazeer, Noam, and Wu, Yonghui. Exploring the limits of language modeling. CoRR, abs/1602.02410, 2016. URL http://arxiv.org/abs/1602.02410. [↗](http://arxiv.org/abs/1602.02410)
21. Juang, Biing-Hwang and Rabiner, Lawrence. Mixture autoregressive hidden Markov models for speech signals. IEEE Trans. Acoust. Speech Signal Process., pp. 1404–1413, 1985. [find](https://scholar.google.com/scholar?q=Juang%2C+Biing%2DHwang+and+Rabiner%2C+Lawrence%2E+Mixture+autoregressive+hidden+Markov+models+for+speech+signals%2E+IEEE+Trans%2E+Acoust%2E+Speech+Signal+Process%2E%2C+pp%2E+1404%E2%80%931413%2C+1985%2E)
22. Kameoka, Hirokazu, Ohishi, Yasunori, Mochihashi, Daichi, and Le Roux, Jonathan. Speech analysis with multi-kernel linear prediction. In Spring Conference of ASJ, pp. 499–502, 2010. (in Japanese). [find](https://scholar.google.com/scholar?q=Kameoka%2C+Hirokazu%2C+Ohishi%2C+Yasunori%2C+Mochihashi%2C+Daichi%2C+and+Le+Roux%2C+Jonathan%2E+Speech+analysis+with+multi%2Dkernel+linear+prediction%2E+In+Spring+Conference+of+ASJ%2C+pp%2E+499%E2%80%93502%2C+2010%2E+%28in+Japanese%29%2E)
23. Karaali, Orhan, Corrigan, Gerald, Gerson, Ira, and Massey, Noel. Text-to-speech conversion with neural networks: A recurrent TDNN approach. In Eurospeech, pp. 561–564, 1997. [find](https://scholar.google.com/scholar?q=Karaali%2C+Orhan%2C+Corrigan%2C+Gerald%2C+Gerson%2C+Ira%2C+and+Massey%2C+Noel%2E+Text%2Dto%2Dspeech+conversion+with+neural+networks%3A+A+recurrent+TDNN+approach%2E+In+Eurospeech%2C+pp%2E+561%E2%80%93564%2C+1997%2E)
24. Kawahara, Hideki, Masuda-Katsuse, Ikuyo, and de Cheveigné, Alain. Restructuring speech representations using a pitch-adaptive time-frequency smoothing and an instantaneous-frequency-based f 0 extraction: possible role of a repetitive structure in sounds. Speech Commn., 27:187– 207, 1999. [find](https://scholar.google.com/scholar?q=Kawahara%2C+Hideki%2C+Masuda%2DKatsuse%2C+Ikuyo%2C+and+de+Cheveigne%CC%81%2C+Alain%2E+Restructuring+speech+representations+using+a+pitch%2Dadaptive+time%2Dfrequency+smoothing+and+an+instantaneous%2Dfrequency%2Dbased+f+0+extract)
25. Kawahara, Hideki, Estill, Jo, and Fujimura, Osamu. Aperiodicity extraction and control using mixed mode excitation and group delay manipulation for a high quality speech analysis, modification and synthesis system STRAIGHT. In MAVEBA, pp. 13–15, 2001. [find](https://scholar.google.com/scholar?q=Kawahara%2C+Hideki%2C+Estill%2C+Jo%2C+and+Fujimura%2C+Osamu%2E+Aperiodicity+extraction+and+control+using+mixed+mode+excitation+and+group+delay+manipulation+for+a+high+quality+speech+analysis%2C+modification+and+syn)
26. Law, Edith and Von Ahn, Luis. Input-agreement: a new mechanism for collecting data using human computation games. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1197–1206. ACM, 2009. [find](https://scholar.google.com/scholar?q=Law%2C+Edith+and+Von+Ahn%2C+Luis%2E+Input%2Dagreement%3A+a+new+mechanism+for+collecting+data+using+human+computation+games%2E+In+Proceedings+of+the+SIGCHI+Conference+on+Human+Factors+in+Computing+Systems%2C+pp%2E+119)
27. Maia, Ranniery, Zen, Heiga, and Gales, Mark J. F. Statistical parametric speech synthesis with joint estimation of acoustic and excitation model parameters. In ISCA SSW7, pp. 88–93, 2010. [find](https://scholar.google.com/scholar?q=Maia%2C+Ranniery%2C+Zen%2C+Heiga%2C+and+Gales%2C+Mark+J%2E+F%2E+Statistical+parametric+speech+synthesis+with+joint+estimation+of+acoustic+and+excitation+model+parameters%2E+In+ISCA+SSW7%2C+pp%2E+88%E2%80%9393%2C+2010%2E)
28. Morise, Masanori, Yokomori, Fumiya, and Ozawa, Kenji. WORLD: A vocoder-based high-quality speech synthesis system for real-time applications. IEICE Trans. Inf. Syst., E99-D(7):1877–1884, 2016. [find](https://scholar.google.com/scholar?q=Morise%2C+Masanori%2C+Yokomori%2C+Fumiya%2C+and+Ozawa%2C+Kenji%2E+WORLD%3A+A+vocoder%2Dbased+high%2Dquality+speech+synthesis+system+for+real%2Dtime+applications%2E+IEICE+Trans%2E+Inf%2E+Syst%2E%2C+E99%2DD%287%29%3A1877%E2%80%931884%2C+2016%2E)
29. Moulines, Eric and Charpentier, Francis. Pitch synchronous waveform processing techniques for text-to-speech synthesis using diphones. Speech Commn., 9:453–467, 1990. [find](https://scholar.google.com/scholar?q=Moulines%2C+Eric+and+Charpentier%2C+Francis%2E+Pitch+synchronous+waveform+processing+techniques+for+text%2Dto%2Dspeech+synthesis+using+diphones%2E+Speech+Commn%2E%2C+9%3A453%E2%80%93467%2C+1990%2E)
30. Muthukumar, P. and Black, Alan W. A deep learning approach to data-driven parameterizations for statistical parametric speech synthesis. arXiv:1409.8558, 2014. — [arXiv:1409.8558](https://arxiv.org/abs/1409.8558) [find](https://scholar.google.com/scholar?q=Muthukumar%2C+P%2E+and+Black%2C+Alan+W%2E+A+deep+learning+approach+to+data%2Ddriven+parameterizations+for+statistical+parametric+speech+synthesis%2E+arXiv%3A1409%2E8558%2C+2014%2E+%E2%80%94+%5BarXiv%3A1409%2E8558%5D%28https%3A%2F%2Farxiv%2Eorg%2Fab)
31. Nair, Vinod and Hinton, Geoffrey E. Rectified linear units improve restricted Boltzmann machines. In ICML, pp. 807–814, 2010. [find](https://scholar.google.com/scholar?q=Nair%2C+Vinod+and+Hinton%2C+Geoffrey+E%2E+Rectified+linear+units+improve+restricted+Boltzmann+machines%2E+In+ICML%2C+pp%2E+807%E2%80%93814%2C+2010%2E)
32. Nakamura, Kazuhiro, Hashimoto, Kei, Nankaku, Yoshihiko, and Tokuda, Keiichi. Integration of spectral feature extraction and modeling for HMM-based speech synthesis. IEICE Trans. Inf. Syst., E97-D(6):1438–1448, 2014. [find](https://scholar.google.com/scholar?q=Nakamura%2C+Kazuhiro%2C+Hashimoto%2C+Kei%2C+Nankaku%2C+Yoshihiko%2C+and+Tokuda%2C+Keiichi%2E+Integration+of+spectral+feature+extraction+and+modeling+for+HMM%2Dbased+speech+synthesis%2E+IEICE+Trans%2E+Inf%2E+Syst%2E%2C+E97%2DD%286%29%3A1)
33. Palaz, Dimitri, Collobert, Ronan, and Magimai-Doss, Mathew. Estimating phoneme class conditional probabilities from raw speech signal using convolutional neural networks. In Interspeech, pp. 1766–1770, 2013. [find](https://scholar.google.com/scholar?q=Palaz%2C+Dimitri%2C+Collobert%2C+Ronan%2C+and+Magimai%2DDoss%2C+Mathew%2E+Estimating+phoneme+class+conditional+probabilities+from+raw+speech+signal+using+convolutional+neural+networks%2E+In+Interspeech%2C+pp%2E+1766%E2%80%931770)
34. Peltonen, Sari, Gabbouj, Moncef, and Astola, Jaakko. Nonlinear filter design: methodologies and challenges. In IEEE ISPA, pp. 102–107, 2001. [find](https://scholar.google.com/scholar?q=Peltonen%2C+Sari%2C+Gabbouj%2C+Moncef%2C+and+Astola%2C+Jaakko%2E+Nonlinear+filter+design%3A+methodologies+and+challenges%2E+In+IEEE+ISPA%2C+pp%2E+102%E2%80%93107%2C+2001%2E)
35. Poritz, Alan B. Linear predictive hidden Markov models and the speech signal. In ICASSP, pp. 1291–1294, 1982. [find](https://scholar.google.com/scholar?q=Poritz%2C+Alan+B%2E+Linear+predictive+hidden+Markov+models+and+the+speech+signal%2E+In+ICASSP%2C+pp%2E+1291%E2%80%931294%2C+1982%2E)
36. Rabiner, Lawrence and Juang, Biing-Hwang. Fundamentals of Speech Recognition. PrenticeHall, 1993. [find](https://scholar.google.com/scholar?q=Rabiner%2C+Lawrence+and+Juang%2C+Biing%2DHwang%2E+Fundamentals+of+Speech+Recognition%2E+PrenticeHall%2C+1993%2E)
37. Sagisaka, Yoshinori, Kaiki, Nobuyoshi, Iwahashi, Naoto, and Mimura, Katsuhiko. ATR ν-talk speech synthesis system. In ICSLP, pp. 483–486, 1992. [find](https://scholar.google.com/scholar?q=Sagisaka%2C+Yoshinori%2C+Kaiki%2C+Nobuyoshi%2C+Iwahashi%2C+Naoto%2C+and+Mimura%2C+Katsuhiko%2E+ATR+%CE%BD%2Dtalk+speech+synthesis+system%2E+In+ICSLP%2C+pp%2E+483%E2%80%93486%2C+1992%2E)
38. Sainath, Tara N., Weiss, Ron J., Senior, Andrew, Wilson, Kevin W., and Vinyals, Oriol. Learning the speech front-end with raw waveform CLDNNs. In Interspeech, pp. 1–5, 2015. [find](https://scholar.google.com/scholar?q=Sainath%2C+Tara+N%2E%2C+Weiss%2C+Ron+J%2E%2C+Senior%2C+Andrew%2C+Wilson%2C+Kevin+W%2E%2C+and+Vinyals%2C+Oriol%2E+Learning+the+speech+front%2Dend+with+raw+waveform+CLDNNs%2E+In+Interspeech%2C+pp%2E+1%E2%80%935%2C+2015%2E)
39. Takaki, Shinji and Yamagishi, Junichi. A deep auto-encoder based low-dimensional feature extraction from FFT spectral envelopes for statistical parametric speech synthesis. In ICASSP, pp. 5535–5539, 2016. [find](https://scholar.google.com/scholar?q=Takaki%2C+Shinji+and+Yamagishi%2C+Junichi%2E+A+deep+auto%2Dencoder+based+low%2Ddimensional+feature+extraction+from+FFT+spectral+envelopes+for+statistical+parametric+speech+synthesis%2E+In+ICASSP%2C+pp%2E+5535%E2%80%935539%2C+2)
40. Takamichi, Shinnosuke, Toda, Tomoki, Black, Alan W., Neubig, Graham, Sakriani, Sakti, and Nakamura, Satoshi. Postfilters to modify the modulation spectrum for statistical parametric speech synthesis. IEEE/ACM Trans. Audio Speech Lang. Process., 24(4):755–767, 2016. [find](https://scholar.google.com/scholar?q=Takamichi%2C+Shinnosuke%2C+Toda%2C+Tomoki%2C+Black%2C+Alan+W%2E%2C+Neubig%2C+Graham%2C+Sakriani%2C+Sakti%2C+and+Nakamura%2C+Satoshi%2E+Postfilters+to+modify+the+modulation+spectrum+for+statistical+parametric+speech+synthesis%2E+)
41. Theis, Lucas and Bethge, Matthias. Generative image modeling using spatial LSTMs. In NIPS, pp. 1927–1935, 2015. [find](https://scholar.google.com/scholar?q=Theis%2C+Lucas+and+Bethge%2C+Matthias%2E+Generative+image+modeling+using+spatial+LSTMs%2E+In+NIPS%2C+pp%2E+1927%E2%80%931935%2C+2015%2E)
42. Toda, Tomoki and Tokuda, Keiichi. A speech parameter generation algorithm considering global variance for HMM-based speech synthesis. IEICE Trans. Inf. Syst., E90-D(5):816–824, 2007. [find](https://scholar.google.com/scholar?q=Toda%2C+Tomoki+and+Tokuda%2C+Keiichi%2E+A+speech+parameter+generation+algorithm+considering+global+variance+for+HMM%2Dbased+speech+synthesis%2E+IEICE+Trans%2E+Inf%2E+Syst%2E%2C+E90%2DD%285%29%3A816%E2%80%93824%2C+2007%2E)
43. Toda, Tomoki and Tokuda, Keiichi. Statistical approach to vocal tract transfer function estimation based on factor analyzed trajectory hmm. In ICASSP, pp. 3925–3928, 2008. [find](https://scholar.google.com/scholar?q=Toda%2C+Tomoki+and+Tokuda%2C+Keiichi%2E+Statistical+approach+to+vocal+tract+transfer+function+estimation+based+on+factor+analyzed+trajectory+hmm%2E+In+ICASSP%2C+pp%2E+3925%E2%80%933928%2C+2008%2E)
44. Tokuda, Keiichi. Speech synthesis as a statistical machine learning problem. [http://www.sp](http://www.sp). nitech.ac.jp/˜tokuda/tokuda_asru2011_for_pdf.pdf, 2011. Invited talk given at ASRU. [↗](http://www.sp)
45. Tokuda, Keiichi and Zen, Heiga. Directly modeling speech waveforms by neural networks for statistical parametric speech synthesis. In ICASSP, pp. 4215–4219, 2015. [find](https://scholar.google.com/scholar?q=Tokuda%2C+Keiichi+and+Zen%2C+Heiga%2E+Directly+modeling+speech+waveforms+by+neural+networks+for+statistical+parametric+speech+synthesis%2E+In+ICASSP%2C+pp%2E+4215%E2%80%934219%2C+2015%2E)
46. Tokuda, Keiichi and Zen, Heiga. Directly modeling voiced and unvoiced components in speech waveforms by neural networks. In ICASSP, pp. 5640–5644, 2016. [find](https://scholar.google.com/scholar?q=Tokuda%2C+Keiichi+and+Zen%2C+Heiga%2E+Directly+modeling+voiced+and+unvoiced+components+in+speech+waveforms+by+neural+networks%2E+In+ICASSP%2C+pp%2E+5640%E2%80%935644%2C+2016%2E)
47. Tuerk, Christine and Robinson, Tony. Speech synthesis using artificial neural networks trained on cepstral coefficients. In Proc. Eurospeech, pp. 1713–1716, 1993. [find](https://scholar.google.com/scholar?q=Tuerk%2C+Christine+and+Robinson%2C+Tony%2E+Speech+synthesis+using+artificial+neural+networks+trained+on+cepstral+coefficients%2E+In+Proc%2E+Eurospeech%2C+pp%2E+1713%E2%80%931716%2C+1993%2E)
48. Tüske, Zoltán, Golik, Pavel, Schlüter, Ralf, and Ney, Hermann. Acoustic modeling with deep neural networks using raw time signal for LVCSR. In Interspeech, pp. 890–894, 2014. [find](https://scholar.google.com/scholar?q=Tu%CC%88ske%2C+Zolta%CC%81n%2C+Golik%2C+Pavel%2C+Schlu%CC%88ter%2C+Ralf%2C+and+Ney%2C+Hermann%2E+Acoustic+modeling+with+deep+neural+networks+using+raw+time+signal+for+LVCSR%2E+In+Interspeech%2C+pp%2E+890%E2%80%93894%2C+2014%2E)
49. Uria, Benigno, Murray, Iain, Renals, Steve, Valentini-Botinhao, Cassia, and Bridle, John. Modelling acoustic feature dependencies with artificial neural networks: Trajectory-RNADE. In ICASSP, pp. 4465–4469, 2015. van den Oord, Aäron, Kalchbrenner, Nal, and Kavukcuoglu, Koray. Pixel recurrent neural… — [arXiv:1601.06759](https://arxiv.org/abs/1601.06759) [find](https://scholar.google.com/scholar?q=Uria%2C+Benigno%2C+Murray%2C+Iain%2C+Renals%2C+Steve%2C+Valentini%2DBotinhao%2C+Cassia%2C+and+Bridle%2C+John%2E+Modelling+acoustic+feature+dependencies+with+artificial+neural+networks%3A+Trajectory%2DRNADE%2E+In+ICASSP%2C+pp%2E+4465)
50. Wu, Yi-Jian and Tokuda, Keiichi. Minimum generation error training with direct log spectral distortion on LSPs for HMM-based speech synthesis. In Interspeech, pp. 577–580, 2008. [find](https://scholar.google.com/scholar?q=Wu%2C+Yi%2DJian+and+Tokuda%2C+Keiichi%2E+Minimum+generation+error+training+with+direct+log+spectral+distortion+on+LSPs+for+HMM%2Dbased+speech+synthesis%2E+In+Interspeech%2C+pp%2E+577%E2%80%93580%2C+2008%2E)
51. Yamagishi, Junichi. English multi-speaker corpus for CSTR voice cloning toolkit, 2012. URL [http://homepages.inf.ed.ac.uk/jyamagis/page3/page58/page58.html](http://homepages.inf.ed.ac.uk/jyamagis/page3/page58/page58.html). [↗](http://homepages.inf.ed.ac.uk/jyamagis/page3/page58/page58.html)
52. Yoshimura, Takayoshi. Simultaneous modeling of phonetic and prosodic parameters, and characteristic conversion for HMM-based text-to-speech systems. PhD thesis, Nagoya Institute of Technology, 2002. [find](https://scholar.google.com/scholar?q=Yoshimura%2C+Takayoshi%2E+Simultaneous+modeling+of+phonetic+and+prosodic+parameters%2C+and+characteristic+conversion+for+HMM%2Dbased+text%2Dto%2Dspeech+systems%2E+PhD+thesis%2C+Nagoya+Institute+of+Technology%2C+2002%2E)
53. Yu, Fisher and Koltun, Vladlen. Multi-scale context aggregation by dilated convolutions. In ICLR, 2016. URL http://arxiv.org/abs/1511.07122. — [arXiv:1511.07122](https://arxiv.org/abs/1511.07122) [↗](http://arxiv.org/abs/1511.07122)
54. Zen, Heiga. An example of context-dependent label format for HMM-based speech synthesis in English, 2006. URL [http://hts.sp.nitech.ac.jp/?Download](http://hts.sp.nitech.ac.jp/?Download). o Vocoder analysis Model training Speech Text analysis Text l Training Figure 6: Outline of statistical parametric speech synthesis. [↗](http://hts.sp.nitech.ac.jp/?Download)
55. Zen, Heiga, Tokuda, Keiichi, and Kitamura, Tadashi. Reformulating the HMM as a trajectory model by imposing explicit relationships between static and dynamic features. Comput. Speech Lang., 21(1):153–173, 2007. [find](https://scholar.google.com/scholar?q=Zen%2C+Heiga%2C+Tokuda%2C+Keiichi%2C+and+Kitamura%2C+Tadashi%2E+Reformulating+the+HMM+as+a+trajectory+model+by+imposing+explicit+relationships+between+static+and+dynamic+features%2E+Comput%2E+Speech+Lang%2E%2C+21%281%29%3A153%E2%80%93)
56. Zen, Heiga, Tokuda, Keiichi, and Black, Alan W. Statistical parametric speech synthesis. Speech Commn., 51(11):1039–1064, 2009. [find](https://scholar.google.com/scholar?q=Zen%2C+Heiga%2C+Tokuda%2C+Keiichi%2C+and+Black%2C+Alan+W%2E+Statistical+parametric+speech+synthesis%2E+Speech+Commn%2E%2C+51%2811%29%3A1039%E2%80%931064%2C+2009%2E)
57. Zen, Heiga, Senior, Andrew, and Schuster, Mike. Statistical parametric speech synthesis using deep neural networks. In Proc. ICASSP, pp. 7962–7966, 2013. [find](https://scholar.google.com/scholar?q=Zen%2C+Heiga%2C+Senior%2C+Andrew%2C+and+Schuster%2C+Mike%2E+Statistical+parametric+speech+synthesis+using+deep+neural+networks%2E+In+Proc%2E+ICASSP%2C+pp%2E+7962%E2%80%937966%2C+2013%2E)
58. Zen, Heiga, Agiomyrgiannakis, Yannis, Egberts, Niels, Henderson, Fergus, and Szczepaniak, Przemysław. Fast, compact, and high quality LSTM-RNN based statistical parametric speech synthesizers for mobile devices. In Interspeech, 2016. URL https://arxiv.org/abs/1606.06061. — [arXiv:1606.06061](https://arxiv.org/abs/1606.06061) [↗](https://arxiv.org/abs/1606.06061)