Spurious Predictability in Financial Machine Learning
TLDR
This paper introduces a falsification audit to detect spurious predictability in financial machine learning, addressing common methodological artifacts.
Key contributions
- Introduces a falsification audit to detect spurious predictability in financial machine learning workflows.
- Tests predictive models against synthetic zero-predictability and microstructure placebo environments.
- Quantifies selection-induced performance inflation using an absolute magnitude gap for passing workflows.
- Empirically confirms many financial ML findings are methodological artifacts, not genuine predictability.
Why it matters
Many financial ML models show spurious predictability due to adaptive search. This paper introduces a crucial audit methodology to falsify misleading findings, helping practitioners distinguish genuine signals from artifacts. This improves the reliability of financial ML research and prevents costly errors.
Original Abstract
Adaptive specification search generates statistically significant backtests even under martingale-difference nulls. We introduce a falsification audit testing complete predictive workflows against synthetic reference classes, including zero-predictability environments and microstructure placebos. Workflows generating significant walk-forward evidence in these environments are falsified. For passing workflows, we quantify selection-induced performance inflation using an absolute magnitude gap linking optimized in-sample evidence to disjoint walk-forward realizations, adjusted for effective multiplicity. Simulations validate extreme-value scaling under correlated searches and demonstrate detection power under genuine structure. Empirical case studies confirm that many apparent findings represent methodological artifacts rather than genuine predictability.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.