Title
Impact of item consumption on assessment of recommendations in user studies.
Abstract
In user studies of recommender systems, participants typically cannot consume the recommended items. Still, they are asked to assess recommendation quality and other aspects related to user experience by means of questionnaires. Without having listened to recommended songs or watched suggested movies, however, this might be an error-prone task, possibly limiting validity of results obtained in these studies. In this paper, we investigate the effect of actually consuming the recommended items. We present two user studies conducted in different domains showing that in some cases, differences in the assessment of recommendations and in questionnaire results occur. Apparently, it is not always possible to adequately measure user experience without allowing users to consume items. On the other hand, depending on domain and provided information, participants sometimes seem to approximate the actual value of recommendations reasonably well.
Year
DOI
Venue
2018
10.1145/3240323.3240375
RecSys '18: Twelfth ACM Conference on Recommender Systems Vancouver British Columbia Canada October, 2018
Keywords
Field
DocType
Recommender Systems, Experimentation, User Studies
Recommender system,User experience design,Computer science,Human–computer interaction,Artificial intelligence,User studies,Limiting,Machine learning
Conference
ISBN
Citations 
PageRank 
978-1-4503-5901-6
0
0.34
References 
Authors
13
4
Name
Order
Citations
PageRank
Benedikt Loepp18810.71
Tim Donkers2416.15
Timm Kleemann313.05
Jürgen Ziegler41028300.31