PRIM-cipal components analysis
Tianhao Liu, Daniel Andrés Díaz-Pachón, J. Sunil Rao
TLDR
This paper explores unsupervised No Free Lunch Theorems, showing two opposite optimal bump-hunting strategies for elliptical distributions.
Key contributions
- Proves two equally optimal, opposite bump-hunting strategies for elliptical distributions.
- Shows peeling smallest PCs maximizes variance/Frobenius norm, while peeling largest PCs minimizes them.
- Inspires PRIM-based bump-hunting algorithms by minimizing variance or volume.
- Demonstrates on Fashion-MNIST that largest PCs capture multiplicity, smallest PCs isolate popular styles.
Why it matters
This paper addresses the underexplored area of unsupervised No Free Lunch Theorems. It provides a theoretical foundation for bump-hunting strategies in elliptical distributions, revealing two opposite optimal approaches. This work inspires new PRIM-based algorithms and offers novel insights into data structure.
Original Abstract
Supervised No Free Lunch Theorems (NFLTs) are well studied, yet unsupervised NFLTs remain underexplored. For elliptical distributions, we prove that there exist two equally optimal, scientifically meaningful bump-hunting strategies that are exact opposites, with no universal winner. Specifically, peeling $k$ orthogonal dimensions from $\mathbb{R}^d$ ($d \ge k$), retaining an inter-quantile region of probability $1-α$ per peeled dimension, maximizes total variance and Frobenius norm when the $k$ smallest principal components (called pettiest components) are selected, and minimizes them when the selected dimensions are the $k$ leading principal components. These optima inspire PRIM-based bump-hunting algorithms either by minimizing variance or by minimizing volume, thereby motivating an NFLT. We test our results on the Fashion-MNIST database, showing that peeling the largest principal components captures multiplicity, while peeling the smallest principal components isolates popular styles.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.