WaveNet: A Generative Model for Raw Audio
Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals + 4 more
TLDR
WaveNet is a deep autoregressive model that generates highly natural raw audio waveforms, achieving state-of-the-art results in text-to-speech and music generation.
Key contributions
- Introduces a fully probabilistic, autoregressive neural network for raw audio generation conditioned on all previous samples.
- Achieves superior naturalness in text-to-speech for multiple languages and can model multiple speakers within a single model.
- Demonstrates versatility by generating realistic music and performing well on phoneme recognition tasks.
Why it matters
This paper matters because it presents a novel approach to audio generation that directly models raw waveforms, overcoming limitations of traditional parametric and concatenative methods. By producing highly natural-sounding speech and realistic music, WaveNet advances the state of the art in audio synthesis and has broad implications for speech technology, music generation, and audio-based machine learning tasks.
Original Abstract
This paper introduces WaveNet, a deep neural network for generating raw audio waveforms. The model is fully probabilistic and autoregressive, with the predictive distribution for each audio sample conditioned on all previous ones; nonetheless we show that it can be efficiently trained on data with tens of thousands of samples per second of audio. When applied to text-to-speech, it yields state-of-the-art performance, with human listeners rating it as significantly more natural sounding than the best parametric and concatenative systems for both English and Mandarin. A single WaveNet can capture the characteristics of many different speakers with equal fidelity, and can switch between them by conditioning on the speaker identity. When trained to model music, we find that it generates novel and often highly realistic musical fragments. We also show that it can be employed as a discriminative model, returning promising results for phoneme recognition.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.