Title
User effort vs. accuracy in rating-based elicitation
Abstract
One of the unresolved issues when designing a recommender system is the number of ratings -- i.e., the profile length -- that should be collected from a new user before providing recommendations. A design tension exists, induced by two conflicting requirements. On the one hand, the system must collect "enough"ratings from the user in order to learn her/his preferences and improve the accuracy of recommendations. On the other hand, gathering more ratings adds a burden on the user, which may negatively affect the user experience. Our research investigates the effects of profile length from both a subjective (user-centric) point of view and an objective (accuracy-based) perspective. We carried on an offline simulation with three algorithms, and a set of online experiments involving overall 960 users and four recommender algorithms, to measure which of the two contrasting forces influenced by the number of collected ratings -- recommendations relevance and burden of the rating process -- has stronger effects on the perceived quality of the user experience. Moreover, our study identifies the potentially optimal profile length for an explicit, rating based, and human controlled elicitation strategy.
Year
DOI
Venue
2012
10.1145/2365952.2365963
RecSys
Keywords
Field
DocType
recommendations relevance,recommender algorithm,rating-based elicitation,conflicting requirement,rating process,recommender system,user experience,optimal profile length,new user,design tension,user effort,profile length,elicitation,accuracy,negative affect
Recommender system,Data mining,User experience design,Computer science
Conference
Citations 
PageRank 
References 
21
0.97
24
Authors
3
Name
Order
Citations
PageRank
Paolo Cremonesi1130687.23
Franca Garzottto2210.97
Roberto Turrin385934.94