Slithering Through Gaps: Capturing Discrete Isolated Modes via Logistic Bridging
TLDR
HiSS is a novel Gibbs sampler that uses logistic bridging to efficiently capture discrete isolated modes in complex, high-dimensional distributions.
Key contributions
- Introduces Hyperbolic Secant-squared Gibbs-Sampling (HiSS) for multimodal discrete distributions.
- Employs a Metropolis-within-Gibbs framework with a logistic convolution kernel.
- Couples discrete and auxiliary continuous variables to bridge disconnected modes.
- Achieves superior mixing and convergence, outperforming alternatives on various tasks.
Why it matters
Sampling complex, high-dimensional discrete distributions with multiple disconnected modes is a major challenge for existing gradient-based methods. This paper introduces a novel approach that effectively navigates these rugged landscapes, ensuring better mixing and convergence. It offers a significant advancement for fields relying on such sampling, from physics to machine learning.
Original Abstract
High-dimensional and complex discrete distributions often exhibit multimodal behavior due to inherent discontinuities, posing significant challenges for sampling. Gradient-based discrete samplers, while effective, frequently become trapped in local modes when confronted with rugged or disconnected energy landscapes. This limits their ability to achieve adequate mixing and convergence in high-dimensional multimodal discrete spaces. To address these challenges, we propose \emph{Hyperbolic Secant-squared Gibbs-Sampling (HiSS)}, a novel family of sampling algorithms that integrates a \emph{Metropolis-within-Gibbs} framework to enhance mixing efficiency. HiSS leverages a logistic convolution kernel to couple the discrete sampling variable with the continuous auxiliary variable in a joint distribution. This design allows the auxiliary variable to encapsulate the true target distribution while facilitating easy transitions between distant and disconnected modes. We provide theoretical guarantees of convergence and demonstrate empirically that HiSS outperforms many popular alternatives on a wide variety of tasks, including Ising models, binary neural networks, and combinatorial optimization.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.