Identifiability of Potentially Degenerate Gaussian Mixture Models With Piecewise Affine Mixing
Danru Xu, Sébastien Lachapelle, Sara Magliacane
TLDR
This paper establishes identifiability for latent variables in degenerate Gaussian mixture models observed through piecewise affine mixing functions.
Key contributions
- Addresses identifiability of latent variables in degenerate Gaussian mixture models.
- Considers observations transformed via piecewise affine mixing functions.
- Leverages sparsity regularization for identifiability up to permutation and scaling.
- Proposes a two-stage method to estimate latent variables using sparsity and Gaussianity.
Why it matters
This work tackles the challenging problem of causal representation learning where latent variables are degenerate, making PDFs ill-defined. It provides theoretical identifiability results and a practical estimation method, advancing our ability to recover true latent variables from complex observations.
Original Abstract
Causal representation learning (CRL) aims to identify the underlying latent variables from high-dimensional observations, even when variables are dependent with each other. We study this problem for latent variables that follow a potentially degenerate Gaussian mixture distribution and that are only observed through the transformation via a piecewise affine mixing function. We provide a series of progressively stronger identifiability results for this challenging setting in which the probability density functions are ill-defined because of the potential degeneracy. For identifiability up to permutation and scaling, we leverage a sparsity regularization on the learned representation. Based on our theoretical results, we propose a two-stage method to estimate the latent variables by enforcing sparsity and Gaussianity in the learned representations. Experiments on synthetic and image data highlight our method's effectiveness in recovering the ground-truth latent variables.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.