Title
Investigating self-reporting behavior in long-term studies
Abstract
Self-reporting techniques, such as data logging or a diary, are frequently used in long-term studies, but prone to subjects' forgetfulness and other sources of inaccuracy. We conducted a six-week self-reporting study on smartphone usage in order to investigate the accuracy of self-reported information, and used logged data as ground truth to compare the subjects' reports against. Subjects never recorded more than 70% and, depending on the requested reporting interval, down to less than 40% of actual app usages. They significantly overestimated how long they used apps. While subjects forgot self-reports when no automatic reminders were sent, a high reporting frequency was perceived as uncomfortable and burdensome. Most significantly, self-reporting even changed the actual app usage of users and hence can lead to deceptive measures if a study relies on no other data sources. With this contribution, we provide empirical quantitative long-term data on the reliability of self-reported data collected with mobile devices. We aim to make researchers aware of the caveats of self-reporting and give recommendations for maximizing the reliability of results when conducting large-scale, long-term app usage studies.
Year
DOI
Venue
2013
10.1145/2470654.2481406
CHI
Keywords
Field
DocType
self-reporting behavior,actual app usage,high reporting frequency,self-reported data,six-week self-reporting study,long-term study,smartphone usage,requested reporting interval,long-term app usage study,empirical quantitative long-term data,data source,ground truth,mobile device,col
Data logger,Internet privacy,Long term learning,Computer science,Mobile device,Ground truth
Conference
Citations 
PageRank 
References 
19
0.79
15
Authors
5
Name
Order
Citations
PageRank
Andreas Möller122218.35
Matthias Kranz244237.93
Barbara Schmid3190.79
Luis Roalter420817.19
Stefan Diewald517915.37