Title
Examining the generalizability of the User Engagement Scale (UES) in exploratory search
Abstract
The user experience is an integral component of interactive information retrieval (IIR). However, there is a twofold problem in its measurement. Firstly, while many IIR studies have relied on a single dimension of user feedback, that of satisfaction, experience is a much more complex concept. IIR in general, and exploratory search more specifically, are dynamic, multifaceted experiences that evoke pragmatic and hedonic needs, expectations, and outcomes that are not adequately captured by user satisfaction. Secondly, questionnaires, which are typically the means in which user's attitudes and perceptions are measured, are not typically subjected to rigorous reliability and validity testing. To address these issues, we administered the multidimensional User Engagement Scale (UES) in an exploratory search environment to assess users' perceptions of the Perceived Usability (PUs), Aesthetics (AE), Novelty (NO), Felt Involvement (FI), Focused Attention (FA), and Endurability (EN) aspects of the experience. In a typical laboratory-style study, 381 participants performed three relatively complex search tasks using a novel search interface, and responded to the UES immediately upon completion. We used Principal Axis Factor Analysis and Multiple Regression to examine the factor structure of UES items and the relationships amongst factors. Results showed that three of the six sub-scales (PUs, AE, FA) were stable, while NO, FI and EN merged to form a single factor. We discuss recommendations for revising and validating the UES in light of these findings.
Year
DOI
Venue
2013
10.1016/j.ipm.2012.08.005
Inf. Process. Manage.
Keywords
Field
DocType
exploratory search environment,complex search task,ues item,novel search interface,user engagement scale,user satisfaction,user experience,iir study,exploratory search,multifaceted experience,user feedback,evaluation
Generalizability theory,User experience design,Information retrieval,Computer science,User engagement,Usability,Novelty,Endurability,Multimedia,Perception,Applied psychology,Exploratory search
Journal
Volume
Issue
ISSN
49
5
0306-4573
Citations 
PageRank 
References 
44
1.56
27
Authors
2
Name
Order
Citations
PageRank
Heather L. O'Brien162433.21
Elaine G. Toms2127796.35