Bayesian Sensitivity of Causal Inference Estimators under Evidence-Based Priors
Nikita Dhawan, Daniel Shen, Leonardo Cotta, Chris J. Maddison
TLDR
This paper introduces Bayesian Sensitivity Value (BSV) to assess causal inference robustness by using evidence-based priors, avoiding pessimistic worst-case assumptions.
Key contributions
- Generalizes s-value framework to assess sensitivity of three common causal inference assumptions.
- Argues that existing worst-case sensitivity analyses can be uninformative or unrealistic.
- Introduces Bayesian Sensitivity Value (BSV) for expected sensitivity using evidence-based priors.
- Estimates BSV via Monte Carlo and applies it to an observational study on diabetes treatments.
Why it matters
Causal inference is vital but often relies on untestable assumptions. Existing sensitivity analyses can be overly pessimistic, leading to uninformative results. This work provides a more realistic and evidence-based approach to assess the robustness of causal conclusions, improving their practical utility.
Original Abstract
Causal inference, especially in observational studies, relies on untestable assumptions about the true data-generating process. Sensitivity analysis helps us determine how robust our conclusions are when we alter these underlying assumptions. Existing frameworks for sensitivity analysis are concerned with worst-case changes in assumptions. In this work, we argue that using such pessimistic criteria can often become uninformative or lead to conclusions contradicting our prior knowledge about the world. To demonstrate this claim, we generalize the recent s-value framework (Gupta & Rothenhäusler, 2023) to estimate the sensitivity of three different common assumptions in causal inference. Empirically, we find that, indeed, worst-case conclusions about sensitivity can rely on unrealistic changes in the data-generating process. To overcome this, we extend the s-value framework with a new sensitivity analysis criterion: Bayesian Sensitivity Value (BSV), which computes the expected sensitivity of an estimate to assumption violations under priors constructed from real-world evidence. We use Monte Carlo approximations to estimate this quantity and illustrate its applicability in an observational study on the effect of diabetes treatments on weight loss.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.