Structural interpretability in SVMs with truncated orthogonal polynomial kernels
Víctor Soto-Larrosa, Nuria Torrado, Edmundo J. Huertas
TLDR
Introduces ORCA, a post-training framework for interpreting SVMs with orthogonal polynomial kernels by analyzing structural complexity.
Key contributions
- Proposes Orthogonal Representation Contribution Analysis (ORCA) for post-training SVM interpretability.
- Leverages finite-dimensional RKHS and its orthonormal basis to expand the decision function.
- Quantifies model complexity using Orthogonal Kernel Contribution (OKC) indices across interaction orders and degrees.
- Requires no retraining or surrogate models, offering a direct and efficient diagnostic tool.
Why it matters
This paper introduces a novel, direct method for understanding the internal structure of SVMs, moving beyond simple predictive accuracy. It provides crucial insights into how different features and their interactions contribute to the model's decision-making process, without needing retraining.
Original Abstract
We study post-training interpretability for Support Vector Machines (SVMs) built from truncated orthogonal polynomial kernels. Since the associated reproducing kernel Hilbert space is finite-dimensional and admits an explicit tensor-product orthonormal basis, the fitted decision function can be expanded exactly in intrinsic RKHS coordinates. This leads to Orthogonal Representation Contribution Analysis (ORCA), a diagnostic framework based on normalized Orthogonal Kernel Contribution (OKC) indices. These indices quantify how the squared RKHS norm of the classifier is distributed across interaction orders, total polynomial degrees, marginal coordinate effects, and pairwise contributions. The methodology is fully post-training and requires neither surrogate models nor retraining. We illustrate its diagnostic value on a synthetic double-spiral problem and on a real five-dimensional echocardiogram dataset. The results show that the proposed indices reveal structural aspects of model complexity that are not captured by predictive accuracy alone.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.