Even More Guarantees for Variational Inference in the Presence of Symmetries
Lena Zellinger, Antonio Vergari
TLDR
This paper extends robust variational inference, providing conditions for exact mean recovery under target symmetries and offering guidelines for VI setup.
Key contributions
- Extends robust VI theory for location-scale families under target symmetries.
- Derives sufficient conditions for exact mean recovery using forward KL and α-divergences.
- Demonstrates how optimization can fail without these sufficient conditions.
- Offers initial guidelines for selecting variational families and α-values in VI.
Why it matters
Variational inference often struggles with misspecification. This work provides crucial theoretical guarantees for recovering target means, even with misspecified models. It helps practitioners make better choices for variational families and divergence types, improving VI reliability.
Original Abstract
When approximating an intractable density via variational inference (VI) the variational family is typically chosen as a simple parametric family that very likely does not contain the target. This raises the question: Under which conditions can we recover characteristics of the target despite misspecification? In this work, we extend previous results on robust VI with location-scale families under target symmetries. We derive sufficient conditions guaranteeing exact recovery of the mean when using the forward Kullback-Leibler divergence and $α$-divergences. We further show how and why optimization can fail to recover the target mean in the absence of our sufficient conditions, providing initial guidelines on the choice of the variational family and $α$-value.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.