MIRA: A Score for Conditional Distribution Accuracy and Model Comparison
Sammy Sharief, Justine Zeghal, Gabriel Missael Barco, Pablo Lemos, Yashar Hezaveh + 1 more
TLDR
MIRA is a new sample-based score for evaluating conditional distribution accuracy and comparing models, simplifying Bayesian model comparison.
Key contributions
- Introduces Mira, a sample-based score for assessing conditional distribution accuracy.
- Derives an analytic expression for the Mira statistic using joint samples from the true data-generating process.
- Enables model comparison by quantifying alignment between candidate model's and true conditional distributions.
- Facilitates Bayesian model comparison via direct posterior validation, bypassing challenging evidence computation.
Why it matters
This paper introduces a novel, sample-based method for evaluating conditional distribution accuracy and comparing models. It simplifies Bayesian model comparison by enabling direct posterior validation, avoiding computationally intensive evidence calculations. This makes complex model assessment more accessible and efficient.
Original Abstract
We introduce Mira, a sample-based score for assessing the accuracy of a candidate conditional distribution using only joint samples from the true data-generating process. Relying on the principle that distributions coincide if they assign equal probability mass to all regions, we derive an analytic expression for the Mira statistic, whose average defines the Mira score. This formulation further allows us to compute theoretical reference values and uncertainty estimates when the candidate distribution matches the true one. This framework enables model comparison by quantifying the alignment between the conditional distribution of a candidate model and the true data generating process. Consequently, Mira enables Bayesian model comparison through direct posterior validation, bypassing the challenging evidence computation. We demonstrate its effectiveness across several toy problems and Bayesian inference tasks.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.