Paper Info
Title
Do Loyal Users Enjoy Better Recommendations?: Understanding Recommender Accuracy from a Time Perspective.
Abstract
In academic research, recommender systems are often evaluated on benchmark datasets, without much consideration about the global timeline. Hence, we are unable to answer questions like: Do loyal users enjoy better recommendations than non-loyal users? Loyalty can be defined by the time period a user has been active in a recommender system, or by the number of historical interactions a user has. In this paper, we offer a comprehensive analysis of recommendation results along global timeline. We conduct experiments with five widely used models, i.e., BPR, NeuMF, LightGCN, SASRec and TiSASRec, on four benchmark datasets, i.e., MovieLens-25M, Yelp, Amazon-music, and Amazon-electronic. Our experiment results give an answer "No" to the above question. Users with many historical interactions suffer from relatively poorer recommendations. Users who stay with the system for a shorter time period enjoy better recommendations. Both findings are counter-intuitive. Interestingly, users who have recently interacted with the system, with respect to the time point of the test instance, enjoy better recommendations. The finding on recency applies to all users, regardless of users' loyalty. Our study offers a different perspective to understand recommender accuracy, and our findings could trigger a revisit of recommender model design. The code is available in \url{https://github.com/putatu/recommenderLoyalty.
Year
DOI
Venue
2022
10.1145/3539813.3545124
International Conference on the Theory of Information Retrieval (ICTIR)
DocType
Citations 
PageRank 
Conference
0
0.34
References 
Authors
0
4
Name
Order
Citations
PageRank
Yitong Ji100.34
Aixin Sun23071156.89
Jie Zhang3841185.41
Chenliang Li433.50