Scattered Hypothesis Generation for Open-Ended Event Forecasting
He Chang, Zhulin Tao, Lifang Yang, Xianglin Huang, Yunshan Ma
TLDR
SCATTER is an RL framework that generates diverse, plausible hypotheses for open-ended event forecasting, moving beyond single-outcome predictions.
Key contributions
- Introduces "scatter forecasting" and hypothesis generation for diverse, open-ended event predictions.
- Proposes SCATTER, a reinforcement learning framework optimizing inclusiveness and diversity.
- Uses a novel hybrid reward with validity, intra-group, and inter-group diversity components.
- Validity-gated score prevents mode collapse by ensuring contextually plausible future explorations.
Why it matters
This paper addresses a critical limitation in LLM-based event forecasting by embracing the inherent uncertainty of real-world events. By generating a diverse set of plausible future hypotheses, SCATTER provides a more comprehensive and robust approach to risk management. This shift from pinpoint to scatter forecasting offers significant advancements for real-world applications.
Original Abstract
Despite the importance of open-ended event forecasting for risk management, current LLM-based methods predominantly target only the most probable outcomes, neglecting the intrinsic uncertainty of real-world events. To bridge this gap, we advance open-ended event forecasting from pinpoint forecasting to scatter forecasting by introducing the proxy task of hypothesis generation. This paradigm aims to generate an inclusive and diverse set of hypotheses that broadly cover the space of plausible future events. To this end, we propose SCATTER, a reinforcement learning framework that jointly optimizes inclusiveness and diversity of the hypothesis. Specifically, we design a novel hybrid reward that consists of three components: 1) a validity reward that measures semantic alignment with observed events, 2) an intra-group diversity reward to encourage variation within sampled responses, and 3) an inter-group diversity reward to promote exploration across distinct modes. By integrating the validity-gated score into the overall objective, we confine the exploration of wildly diversified outcomes to contextually plausible futures, preventing the mode collapse issue. Experiments on two real-world benchmark datasets, i.e., OpenForecast and OpenEP, demonstrate that SCATTER significantly outperforms strong baselines. Our code is available at https://github.com/Sambac1/SCATTER.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.