Spectral Thompson sampling
Tomas Kocak, Michal Valko, Remi Munos, Shipra Agrawal
TLDR
SpectralTS efficiently solves graph bandit problems by leveraging an effective dimension, achieving comparable regret with improved computational performance.
Key contributions
- Introduces SpectralTS, an algorithm for bandit problems with smooth payoffs on an underlying graph.
- Leverages an "effective dimension d" to address scalability issues in large-scale graph bandit problems.
- Proves a regret bound of d*sqrt(T ln N), comparable to state-of-the-art but more efficient.
- Shows competitive empirical performance on both synthetic and real-world datasets.
Why it matters
Graph bandit problems are crucial in recommender systems and advertising, but traditional algorithms struggle with scalability. SpectralTS provides a computationally efficient solution with strong theoretical regret guarantees. This makes it a practical and effective alternative for large-scale applications.
Original Abstract
Thompson Sampling (TS) has attracted a lot of interest due to its good empirical performance, in particular in the computational advertising. Though successful, the tools for its performance analysis appeared only recently. In this paper, we describe and analyze SpectralTS algorithm for a bandit problem, where the payoffs of the choices are smooth given an underlying graph. In this setting, each choice is a node of a graph and the expected payoffs of the neighboring nodes are assumed to be similar. Although the setting has application both in recommender systems and advertising, the traditional algorithms would scale poorly with the number of choices. For that purpose we consider an effective dimension d, which is small in real-world graphs. We deliver the analysis showing that the regret of SpectralTS scales as d*sqrt(T ln N) with high probability, where T is the time horizon and N is the number of choices. Since a d*sqrt(T ln N) regret is comparable to the known results, SpectralTS offers a computationally more efficient alternative. We also show that our algorithm is competitive on both synthetic and real-world data.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.