RecNextEval: A Reference Implementation for Temporal Next-Batch Recommendation Evaluation
Tze-Kean Ng, Joshua Teng-Khing Khoo, Aixin Sun
TLDR
RecNextEval provides a temporal next-batch recommendation evaluation framework, using time-window splits to prevent data leakage and simulate production.
Key contributions
- Reference implementation for temporal next-batch recommendation evaluation.
- Employs time-window data splits to prevent data leakage in evaluation.
- Promotes model development that accurately simulates production environments.
- Open-source library and GUI for public access and use.
Why it matters
Current RecSys evaluation methods often suffer from data leakage and don't reflect real-world production. RecNextEval addresses this by providing a robust, temporal evaluation framework. This improves reproducibility and encourages more realistic recommender system development.
Original Abstract
A good number of toolkits have been developed in Recommender Systems (RecSys) research to promote fair evaluation and reproducibility. However, recent critical examinations of RecSys evaluation protocols have raised concerns regarding the validity of existing evaluation pipelines. In this demonstration, we present RecNextEval, a reference implementation of an evaluation framework specifically designed for next-batch recommendation. RecNextEval utilizes a time-window data split to ensure models are evaluated along a global timeline, effectively minimizing data leakage. Our implementation highlights the inherent complexities of RecSys evaluation and encourages a shift toward model development that more accurately simulates production environments. The RecNextEval library and its accompanying GUI interface are open-source and publicly accessible.
📬 Weekly AI Paper Digest
Get the top 10 AI/ML arXiv papers from the week — summarized, scored, and delivered to your inbox every Monday.