Universality of Gaussian-Mixture Reverse Kernels in Conditional Diffusion
Nafiz Ishtiaque, Syed Arefinul Haque, Kazi Ashraful Alam, Fatima Jahara
TLDR
This paper proves the universality of conditional diffusion models using Gaussian-mixture reverse kernels with ReLU networks for approximating target distributions.
Key contributions
- Proves conditional diffusion models with Gaussian-mixture reverse kernels universally approximate target distributions.
- Decomposes output error into a vanishing terminal mismatch and per-step reverse-kernel errors.
- Solves per-step approximation using Norets' Gaussian-mixture theory and ReLU bounds.
- Shows the neural reverse-kernel class is dense in conditional KL under exact terminal matching.
Why it matters
This paper provides a theoretical foundation for conditional diffusion models, proving their universal approximation capabilities. This work is crucial for advancing generative AI, offering insights into designing more robust and effective models.
Original Abstract
We prove that conditional diffusion models whose reverse kernels are finite Gaussian mixtures with ReLU-network logits can approximate suitably regular target distributions arbitrarily well in context-averaged conditional KL divergence, up to an irreducible terminal mismatch that typically vanishes with increasing diffusion horizon. A path-space decomposition reduces the output error to this mismatch plus per-step reverse-kernel errors; assuming each reverse kernel factors through a finite-dimensional feature map, each step becomes a static conditional density approximation problem, solved by composing Norets' Gaussian-mixture theory with quantitative ReLU bounds. Under exact terminal matching the resulting neural reverse-kernel class is dense in conditional KL.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.