The Virtue of Sparsity in Complexity
Nima Afsharhajari, Jonathan Yu-Meng Li
TLDR
This paper shows that in asset pricing, complexity in feature spaces complements factor sparsity, enabling discovery of parsimonious risk structures.
Key contributions
- Distinguishes capacity sparsity (feature space dimensionality) from factor sparsity (parsimonious risk structure).
- Argues that expanding capacity sparsity enables the discovery of factor sparsity in asset pricing.
- Shows nonlinear feature expansions with basis pursuit outperform ridgeless benchmarks in high complexity.
- Demonstrates complexity's gains arise from identifying sparse risk structures, not from retaining more factors.
Why it matters
This paper resolves the tension between sparsity and complexity in high-dimensional asset pricing, showing they are complementary. It provides a novel framework where expanding feature spaces helps identify parsimonious risk structures. This approach leads to improved out-of-sample portfolio performance.
Original Abstract
Sparsity or complexity? In modern high-dimensional asset pricing, these are often viewed as competing principles: richer feature spaces appear to favor complexity, while economic intuition has long favored parsimony. We show that this tension is misplaced. We distinguish capacity sparsity-the dimensionality of the candidate feature space-from factor sparsity-the parsimonious structure of priced risks-and argue that the two are complements: expanding capacity enables the discovery of factor sparsity. Revisiting the benchmark empirical design of Didisheim et al. (2025) and pushing it to higher complexity regimes, we show that nonlinear feature expansions combined with basis pursuit yield portfolios whose out-of-sample performance dominates ridgeless benchmarks beyond a critical complexity threshold. The evidence shows that the gains from complexity arise not from retaining more factors, but from enlarging the space from which a sparse structure of priced risks can be identified. The virtue of complexity in asset pricing operates through factor sparsity.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.