1. Tips for Collecting
High-Quality
Qualitative Data
With
Dr. Rachel Carmen Ceasar, Assistant Professor of Clinical
Population & Public Health Sciences, KSOM Population & Public
Health Sciences, University of Southern California
Dr. Katherine Guevara, Associate Director of Clinical & Translational
Research Education Programs, SC CTSI Workforce Development
at the University of Southern California
2. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Welcometo“TipsforCollectingHigh-
QualityQualitativeData”w/RachelCeasar
1
As people are joining…say hello by writing in the chat:
Name
What department
you’re from
3. Rachel Carmen Ceasar • Katherine Guevara
12.02.21 workshop
Tips for Collecting
High-Quality Qualitative Data
4. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
00. Introductions
01. Qualitative research is…
02. Activity 1-2: Preparing your
interview guide
03. Activity 3: Conducting your
interview [time permitting]
04. Next steps, Debrief, Q&A
Agenda
3
5. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Women on
fl
oor reviewing notes.
00.
Introductions
4
6. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Women smiling with crying baby strapped to her on Zoom call.
Hi! My name is Rachel (she/her).
• Current life: Assistant Professor,
Population and Public Health
Sciences, USC Keck
• Past life: anthropologist + research
consultant
• Current joys: Mochi & Wa
ffl
es
(Net
fl
ix), Sally Rooney?, pregnant
women and cannabis use research
5
10. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Shareinthechat.
9
What do you struggle with the most when it comes to collecting qualitative data?
OR
What do you see as the biggest hurdles to doing qualitative research?
11. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
• Basic theory and applications of
qualitative research.
• Interview guide development.
• Examples of good and bad
interviewing based on real studies.
• Delegation tips on where you can
assign data collection tasks to junior
team members and new researchers.
What we will cover
10
12. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Various knickknacks laid out.
01. Qualitative
research is…
11
13. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
I’m a qualitative
researcher.
Shareinthechat.
12
More of a numbers
person—quant.
How familiar are you with qualitative research?
I’ve used some
qualitative
approaches in my
work.
14. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
•1. Method: rigorous method that listens,
watches, and interacts with people.
•2. Mindset: approaching problems as
they exist within a person’s life and their
various relationships and environments
•3. Social justice: Acknowledging people
as complex, dynamic, and social beings.
Qualitative research is…de
fi
ned
13
15. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Let it
fl
ow
Ethnography
14
Explore and play, but I need some questions
answered
Ethnographic
approachto
qualitative
research
Just answer all my
questions, please
Survey
16. Mock up drawing of passengers boarding an air taxi at a vertiport.
15
IMAGE: TECH CRUNCH
18. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Close up sliced bread.
Slow research movement
Qualitative research
• Local is best—acknowledge di
ff
erences,
context.
• Value and preserve what’s working.
• Take pause for the now and resist racing
ahead.
• Strive towards knowledge, not info
acquisition.
• Build in iterative processes to inform work.
17
IMAGE: WITH SOURDOUGH
19. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Not so great for…
• How user-friendly something is
(usability testing)
• Getting precise figures to a specific
question (survey)
• Testing out a physical service / product
(demo)
18
Qualitative research is great for…
• Tailoring interventions for particular
populations and settings
• Show similarities and differences among
populations
• Understand local variations across and
within settings
• Developing person-informed systems and
policies.
20. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
•1. Method: 2. Mindset: 3. Social justice
Qualitative research is…RECAP
19
21. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Food on plate and white plates
Stretch!
Bianca Bakery
20
22. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Drawing of computer on table with tablet and notepads.
02. Activity 1:
Preparing your
interview guide
21
23. 22
RESEARCH
QUESTION
SECONDARY
LITERATURE REVIEW
ANALYSIS
SYNTHESIS
DISSEMINATION
DESIGN & METHODS
DATA COLLECTION &
MANAGEMENT
1 2 3
4
5
6
7
Define research scope and
objectives: QUESTION
Review existing
literature, trends,
and research
reports. STAT / INFO
THAT’S KNOWN
Recruit X. Develop
research materials and
data management plan
in prep for field. WHO
AND WHAT METHOD
Conduct audio and video
recorded interviews with X
about their knowledge,
attitudes, and practice of Y.
WHO TO TALK TO AND
ABOUT WHAT
Codify data and search for
patterns using qualitative
computer-assisted data analysis
software to identify potential
models. STATEMENTS /
THEORIES ABOUT DATA
Evolve statements /
theories into concrete
next steps according to
needs and gaps of
community / literature.
OPPORTUNITY AREAS /
DISCUSSION
Share out next steps via
final deliverable.
OUTCOMES / FINAL
DELIVERABLE / PEER-
REVIEWED ARTICLE /
NEXT GRANT
Research map (example)
27. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
•Kickoff meeting: align on research
objectives + design/methods + recruitment
•Secondary data: what’s been done (stats,
other departments, reports)
•Stakeholder interviews: buy-in + stakes
—>interview guide
Prep work
Developing interview guide
26
28. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
•Rundown of roles + responsibilities
•Lead researcher: asking questions +
listening
•Secondary researcher: listening +
filling in + tech
•Expert: content or community
member
Prep work
Day of interview
27
31. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Activity 1
Using ethnographic
approach to strengthen
interview
Project goal: Design new Cancer Portal based
participants’ experiences.
Our task as ethnographers: What feedback
would you give the PI to this study so that this
interview guide better gets at the interviewee’s
experience with the Portal?
• Make a copy of Bad Interview Guide for your
team [LINK].
• Add comments for the PI to tweak approach—
not necessarily re-writing individual questions
30
32. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Smallgroups.
ProvidethePIfeedbackoninterviewguide.
31
How can we make this interview guide better so that it gets
at the interviewee’s experience?
Task: Make a copy of bad interview guide + add comments.
33. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Brick wall painted in bright colors in the shape of a face.
10 tips to better interviews [LINK].
Cheat sheet
1. No hypothetical questions.
2. No close-ended questions.
3. No leading questions.
4. Be polite.
5. Be quiet.
6. It’s not about you.
7. Avoid validation.
8. Keep it conversational.
9. Using silence.
10. Avoid intervention or dictating what’s normal.
Finding meaning in patterns
32
34. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Two women sitting at a desk talking and taking notes.
#1: No hypothetical questions
Better interviews, 1
• Avoid hypothetical or scenario questions
• Plant questions in actual situations, local
• Situate questions grounded in little details
they’ve already thrown out
• Example:
• XX Would you use this service in the
future? XX
• What’s missing from this service?
33
35. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Two women sitting at desk and talking.
#2: No close-ended questions
Better interviews, 2
• Avoid yes/no questions—it’s not a survey
• Example:
• XX Do you like this EMR feature? XX
• How has this EMR feature helped you in your
work?
• XX Do you use cannabis a lot? XX
• When did you last use cannabis?
34
36. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Man standing and addressing group sitting at a desk.
#3: No leading questions
Better interviews, 3
• Let participants paint the picture for you
in their own words—don’t
fi
ll it in for
them
• Example:
• XX Do you get testing every month?
If not, why not? XX
• How often do you go to the doctor /
clinic?
35
37. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Man and woman sitting at a desk talking.
#4: Be polite
Better interviews, 4
• Avoid using “you guys” to refer to women and other
colloquiums (“Oh that sucks”)
• Start with titles (Dr., Mrs. X) and let them direct you on how
they want to be called
• Don’t test them or prove them wrong—you’re not trying to
uncover THE TRUTH
• Examples:
• XX That’s funny because in an earlier interview, someone
said that that hospital is closed now? XX
• XX So they just took your opioids for no reason at all? XX
• XX Interviewer: The stat I heard was that more women
are misdiagnosed for bladder cancer, is that true?
User: I don’t know that stat. XX
36
38. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Woman and man sitting at desk.
#5: Be quiet.
Better interviews, 5
• Avoid “hmmms” and responding every
time participant responds
• Show you’re listening with non-verbal
clues
37
41. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Two women sitting down and talking in front of a computer.
#6: It’s not about you
Better interviews, 6
• “Listening session”
• Example:
XX User: Back in September of last year, I had a
problem with my heart where it was beating too
fast. I don’t know if you know what an ablation is.
Interviewer: I do b/c my dad, yeah.
User: So they had to —
Interviewer: —My dad had to have open heart
surgery, and I remember—- XX
40
42. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Two men sitting down at table and talking.
#7: Avoid validation.
Better interviews, 7
• Don’t validate or give +/- of you feel
about their answer
• Example:
XX Interviewer: Why do you prefer
fentanyl over heroin?
User: Convenience.
Interviewer: That’s cool. XX
41
43. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
A woman drawing on a white pad in front of a group.
#7: Avoid validation.
Better interviews, 7
• Don’t push participant in a direction of how you want
them to respond (eg, what’s good / cool)
• ‘We’ve never heard that before’
• ‘That’s really great”
• Example:
XX Interviewer: Where do you frequently purchase
vaping products?
User: There’s a corner shop where I live—they let me
hang out and bring my dog so.
Interviewer: Awesome, I love that. XX
42
44. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
A group of men and women working at a table together.
#7: Avoid validation.
Better interviews, 7
• Don’t need to give feedback
• Example:
XX User: I kinda just wanted to smoke
my weed at that party, and others were
doing it too so I did.
Interviewer: It’s normal. So have you
ever introduced anyone to smoking? XX
43
45. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
A group of men and women looking at papers together.
#8: Keep it conversational.
Better interviews, 8
• Not trying to get to all your questions or get
to next question—it’s not a survey.
• Example:
XX User: I mean I’m aware of the health risk
involved with vaping because last week—
Interviewer: —Ok. So some people vape b/c
it’s really hard to stop smoking cigarettes.
Would you say that sentence resonates with
you? XX
44
46. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Two men sitting at a table talking.
#8: Keep it conversational.
Better interviews, 8
• It’s not a survey.
• Example:
XX Interviewer: Do you remember what you used the
fi
rst
time you used cannabis?
User: No I don’t.
Interviewer: Okay. Do you remember liking it?
User: Yeah.
Interviewer: Did it have mostly THC in it? B/c sometimes
people mix CBD…
User: No, I’m not sure. XX
45
47. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Two women sitting at a table talking.
#8: Keep it conversational.
Better interviews, 8
• The order of the interview guide is just a suggestion
• Memorize the interview guide and have a second
interviewer ask follow up questions from the guide
and their listening
• Example:
XX User: Before it was like okay, there’s just a joint
and a bong but now there’s pens, and devices, and
all kinds of names for them it’s so funny.
Interviewer: Ok, ok we are actually about to get into
those questions right now. So—what’s in your joint?
XX
46
48. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
A woman and man sitting in chairs and talking.
#9: Using silence
Better interviews 9
• When in doubt what to do with yourself,
take a pause
• Look at their body language—are they
tired and uncomfortable?
• Breaks and breaths—maybe they
need a bathroom break?
47
49. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Two women sitting at a table and talking to each other.
#10: Avoid intervention or dictating what’s normal
Better interviews,
10
• Not an opportunity to give medical advice
(even if you’re a medical professional!) or life
advice
• Example:
XX User: I used to run cross country but
since the accident, I’m so reliant on my pain
meds to just do basic walking. I know these
pills are killing me I know it.
Interviewer: Well we don’t know that yet. XX
48
50. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Men and women looking at papers together on desk.
#10: Avoid intervention or dictating what’s normal
Better interviews,
10
• Listening vs. reacting
• Don’t wait to get to sensitive question to say it’s
sensitive
• Example:
XX User: We’ll sometimes get together my dad and I
and smoke and drink together, like on the weekends.
We’ll like smoke some cannabis, nothing crazy.
Interviewer: Right so you guys will have a few beers
and eat dinner type thing?
User: Sort of…He doesn’t eat dinner. XX
49
51. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Woman with pompoms and eye stickers on her face.
Helen Kim, BFA
• Ethnographic projects:
• Ktown is my Town
• Home is where the heart is
VoyageLA
50
52.
53. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Activity 2
Using ethnographic
approach to create focus
group guide
Project goal: Create a better COVID
testing experience.
Our task as ethnographers: Capture the
technical + conceptual in one guide.
• Make a copy of template [LINK].
• Write 2-5 questions for 5 min interview.
• Write probes to each question.
52
54. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Smallgroups.
CreateinterviewguideinGoogleDocs.
53
How can we create a better COVID testing experience?
Task: Make a copy of the template.
Write 2-5 questions + probes.
55. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Drawing of
fi
gures standing 2 meters from one another.
03. Activity 3:
Conducting your
interview
54
IMAGE: LSE
56. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Woman and man sitting on bench talking.
Activity 3
Conducting the one-
on-one interview
Project goal: Create a better COVID test
experience.
Our task as ethnographers: Maintain
conversation while weaving in your questions.
• Assign roles to interviewing team:
• 1: Research lead: conduct focus group
• 2: Secondary researcher: follow-up + key
takeaways / quotes
55
57. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Smallgroups
Conductinterview.
56
How can we create a better COVID testing experience?
(5 min each group)
Task for interviewers: Maintain conversation.
Task for interviewees: Answer truthfully based on actual
experiences.
58. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Woman standing and speaking with microphone to group of people
sitting.
04. Next
steps
57
65. A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
rceasar@usc.edu
Debrief+Q&A
64
66. Rachel Carmen Ceasar • Katherine Guevara
12.02.21 workshop
Tips for Collecting
High-Quality Qualitative Data
67. In-depth Interview Focus group
Focuses on complexity and depth of one
participant
Promotes discussion between
participants on a single topic
Best for exploring sensitive, personal topics
(death, sex, substance use)
Best for exploring community norms
with a group that interacts with each
other
Used when participants are unique and
have conflicting views
Used when participants are similar to
one another (socioeconomic status, age)
Ideal for when participants are asked about
information they would be unlikely to share
in a group of strangers (marijuana use)
Ideal for when participants would be
reluctant to share information one-on-
one (idea generation of clinic design,
problem identification of ER service)
vs.
68. Request a Consult - BERD
https://sc-ctsi.org/resources/qualitative-
research-consultation
69. Request a Consult - ERC
https://sc-ctsi.org/resources/education-resource-center
70. Thank You!
SC CTSI | www.sc-ctsi.org Phone: (323) 442-4032 Email: info@sc-ctsi.org Twitter: @SoCalCTSI
Cite us: This work was supported by grants UL1TR001855 and UL1TR000130 from the National Center for Advancing
Translational Science (NCATS) of the U.S. National Institutes of Health. The content is solely the responsibility of the authors
and does not necessarily represent the official views of the National Institutes of Health.
3