Embodied Interpretability: Linking Causal Understanding to Generalization in Vision-Language-Action Models
Hanxin Zhang, Mingshuo Xu, Abdulqader Dhafer, Shigang Yue, Hongbiao Dong + 1 more
TLDR
This paper introduces ISS and NMR, interventional methods to assess causal influence in VLA models, linking it to generalization and identifying spurious correlations.
Key contributions
- Formulates visual-action attribution as an interventional estimation problem.
- Introduces Interventional Significance Score (ISS) to estimate causal influence of visual regions.
- Presents Nuisance Mass Ratio (NMR) to quantify attribution to task-irrelevant features.
- Shows NMR predicts generalization and ISS gives more faithful explanations than other methods.
Why it matters
VLA policies often fail under distribution shift due to reliance on spurious correlations. This paper offers a novel diagnostic approach using interventional attribution (ISS, NMR) to identify causal misalignment. This can lead to more robust and generalizable embodied AI systems.
Original Abstract
Vision-Language-Action (VLA) policies often fail under distribution shift, suggesting that decisions may depend on spurious visual correlations rather than task-relevant causes. We formulate visual-action attribution as an interventional estimation problem. Accordingly, we introduce the Interventional Significance Score (ISS), an interventional masking procedure for estimating the causal influence of visual regions on action predictions, and the Nuisance Mass Ratio (NMR), a scalar measure of attribution to task-irrelevant features. We analyze the statistical properties of ISS and show that it admits unbiased estimation, and we characterize conditions under which action prediction error provides a valid proxy for causal influence. Experiments across diverse manipulation tasks indicate that NMR predicts generalization behavior and that ISS yields more faithful explanations than existing interpretability methods. These results suggest that interventional attribution provides a simple diagnostic approach for identifying causal misalignment in embodied policies.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.