Be the first to like this
Self-reporting techniques, such as data logging or a diary, are frequently used in long-term studies, but prone to subjects’ forgetfulness and other sources of inaccuracy. We conducted a six-week self-reporting study on smartphone usage in or- der to investigate the accuracy of self-reported information, and used logged data as ground truth to compare the sub- jects’ reports against. Subjects never recorded more than 70% and, depending on the requested reporting interval, down to less than 40% of actual app usages. They significantly over- estimated how long they used apps. While subjects forgot self-reports when no automatic reminders were sent, a high reporting frequency was perceived as uncomfortable and bur- densome. Most significantly, self-reporting even changed the actual app usage of users and hence can lead to deceptive measures if a study relies on no other data sources.
With this contribution, we provide empirical quantitative long-term data on the reliability of self-reported data col- lected with mobile devices. We aim to make researchers aware of the caveats of self-reporting and give recommenda- tions for maximizing the reliability of results when conduct- ing large-scale, long-term app usage studies.