I gave a talk on Diaries for Data Collection to the staff and PhD students of Department of Computer Science at Bina Nusantra University, Indonesia, on 17 December 2020.
In this presentation, I shared my experience as a researcher who often uses diaries for data collection in research projects. I have been using diaries since my PhD years, and some of my PhD & MSc IT students also used diaries in their research studies. Diary is not as popular as other data collection methods, however, is able to provide insights where privacy is most concerned. As a UX researcher, I can see the benefits of a diary to my research findings.
2. What is a diary?
• Diaries capture life as it is lived by an individual over time.
• “Provide a record of an ever-changing present” (Allport, 1942)
• author’s thoughts, internal and external events &
observations
• a chronological composite of snapshots and reflections a
few minutes, a day, or a week at a time
• non-textual “accessories to a life story” (Plummer, 2001),
such as photographs and other material/digital objects
• Construed broadly, a diary might be a collage of text and non-
text, or might consist solely of non-textual materials. (Allport,
1942; Plummer, 2001)
3.
4. What is a research diary?
• A research method used to collect qualitative data about user behaviors, activities, and experiences
over time.
• In a diary study, data is self-reported by participants longitudinally — that is, over an extended period
of time that can range from a few days to even a month or longer.
• During the defined reporting period, study participants are asked to keep a diary and log specific
information about activities being studied.
• To help participants remember to fill in their diary, sometimes they are periodically prompted (for
example, through a notification received daily or at select times during the day).
5. When to conduct a
diary study
• Habits — What time of day do users engage with a product? If and how they
choose to share content with others?
• Usage scenarios — In what capacity do users engage with a product? What
are their primary tasks? What are their workflows for completing longer-term
tasks? (These scenarios can be used for user testing later in the process.)
• Attitudes and motivations — What motivates people to perform specific
tasks? How are users feeling and thinking?
• Changes in behaviors and perceptions — How learnable is a system? How
loyal are customers over time? How do they perceive a brand after engaging
with the corresponding organization?
• Customer journeys — What is the typical customer journey and cross-
channel user experience as customers interact with your organization using
different devices and channels such as, email, phone, websites, mobile
applications, kiosks, social media, and online chat? What is the cumulative
effect of multiple service touchpoints?
6. Scope of diary study
Product or Website —
Understanding all
interactions with a site
(e.g., an intranet) over
the course of a month.
Behavior — Gathering
general information
about user behavior (e.g.,
smartphone usage,
college-student web-
visitation patterns)
General activity —
Understanding how
people complete general
activities (e.g., sharing
information via social
tools or shopping online)
A specific activity —
Understanding how
people complete specific
activities (e.g., buying a
new car or planning a
vacation)
7. Diary study methodology
• Define the focus of the study and the long-term behaviors that you need to
understand.
• Define a timeline, select tools for participants to report data, recruit participants, and
prepare instructions or support materials.
Planning and Preparation.
• Schedule a face-to-face meeting or phone call with each participant to discuss the
details of the study.
Pre-study brief.
• Create clear and detailed instructions for logging. Give users example log entries to
help them understand the level of detail you need from them.
Logging period.
8. Type of logging: In-Situ
• When participants engage in a relevant activity, they must report all important details about that
activity right away. Participants are asked to log information about relevant activities in the situation
they occur (or in situ).
• Since this technique requires participants to take the time to report this information at the time of
the event, this technique is best reserved for situations when you don’t foresee a large volume of
diary entries occurring or if the context is such that participant’s daily activities will not be adversely
affected by logging in situ.
• In-situ logging is best supported by channels and devices that can handle structured long-form text
entry such as, email, web-form questionnaires, traditional paper diaries, or digital customer-insight
tools. Audio or video diaries are also great tools for participants, but the output may need to be
transcribed for analysis.
9. Type of logging: Snippet
• Participants only record short snippets of information about activities as they occur. Then, at the end of
each day, or when participants have time, they elaborate on each snippet by providing additional details
about the activity.
• This 2-step technique ensures that relevant information is captured in situ, before being forgotten but
without requiring participants to provide extensive detail at the time of capture, which can be intrusive
and unnatural in certain situations.
• Common channels for study participants to report snippets to researchers include email, text message,
Twitter, or Facebook. These channels are widely familiar for short-form communication.
• Participants are encouraged to use their mobile phones to report events as they happen, since these
devices are accessible. Expanding upon reported snippets can be accomplished with the channels and
tools mentioned above for more in-depth reporting.
• Consider asking participants to expand on their snippets by filling out a questionnaire about each of
them. This will enable you to get specific and consistent insights about each snippet.
10. Diary study methodology (cont’d)
• After the study, evaluate all the information provided by each
participant.
• Plan a follow-up interview to discuss logs in detail.
Post-study interview.
• Because diary studies are longitudinal, they generate a large
amount of qualitative data.
• Evaluate the behaviors you’ve targeted throughout the study.
Data Analysis.
11. Our Experience
1: Children &
Social Media
(Azmi, N. H.,
2018)
1) With whom are you access SNS and where do
you access SNS
2) Something that attracts you in SNS and
something that bothers you in SNS
4) Image you want to put as profile picture in SNS
To pursue
hobby
To follow
celebrity
To keep in
touch
12. Our Experience 2: Elderly & Mobile Quran
(Ahmad, N. A., 2018)
Peaceful Calm Connectedness Fear God Motivated Energised
14. Our Experience 4: Student Learning
Experience & Instagram (2019-2020)
Going to
class
Meeting
people
Doing
coursework
Expressing
emotions
Rewarding
themselves
15. Motivating participants
• Diary studies require time and dedication from participants. To ensure you get
the level of involvement you need from participants, provide an incentive that
will keep them engaged.
• For participants that are engaged, recognize their efforts and ask them to
keep up the good work.
• For participants that are less engaged, give encouragement or offer to answer
any questions they may have to get them on track.
16. Advice for conducting a diary study
Plan for an
appropriate
reporting
period.
01
Recruit
dedicated
users.
02
Be on top of
the data as it
comes in.
03
Conduct a
pilot study.
04
17. References
• Ahmad, N. A. 2018. Islamic Spiritual User Experience (iSUX) Model: A Case Study
of Muslim Older Adults Using Al-Quran Mobile Application, PhD Thesis, UiTM
• Ali, S. F. 2014. Use of Mobile Phone by Single Mothers, MscIT Dissertation, UiTM
• Azmi, N. H. 2017. Modelling Child Persona for Social Behaviour in Social Network
Sites, PhD Thesis, UiTM
• Salazar, K. 2016. Diary Studies: Understanding Long-Term User Behavior and
Experiences. Retrieved at https://www.nngroup.com/articles/diary-studies/
• Shelbe, L. & Wildemuth, B. 2009. Research Diaries in Applications of social
research methods to questions in information and library science, Libraries
Unlimited.