Tips for collecting_high_quality_qualitative_data

Tips for Collecting
High-Quality
Qualitative Data
With
Dr. Rachel Carmen Ceasar, Assistant Professor of Clinical
Population & Public Health Sciences, KSOM Population & Public
Health Sciences, University of Southern California
Dr. Katherine Guevara, Associate Director of Clinical & Translational
Research Education Programs, SC CTSI Workforce Development
at the University of Southern California

A recording and modi
fi
ed version of this presentation will be available to you for personal reference afterwards.
Welcometo“TipsforCollectingHigh-
QualityQualitativeData”w/RachelCeasar
1
As people are joining…say hello by writing in the chat:
Name
What department
you’re from

Rachel Carmen Ceasar • Katherine Guevara

12.02.21 workshop
Tips for Collecting

High-Quality Qualitative Data

fi
00. Introductions

01. Qualitative research is…

02. Activity 1-2: Preparing your
interview guide

03. Activity 3: Conducting your
interview [time permitting]

04. Next steps, Debrief, Q&A
Agenda
3

fi
Women on
fl
oor reviewing notes.
00.
Introductions
4

fi
Women smiling with crying baby strapped to her on Zoom call.
Hi! My name is Rachel (she/her).

• Current life: Assistant Professor,
Population and Public Health
Sciences, USC Keck

• Past life: anthropologist + research
consultant

• Current joys: Mochi & Wa
ffl
es
(Net
fl
ix), Sally Rooney?, pregnant
women and cannabis use research
5

Woman outside with backpack on.
7

8
IMAGE: TECH CRUNCH
Women in clinical sta
ff
room writing down notes.

fi
Shareinthechat.
9
What do you struggle with the most when it comes to collecting qualitative data?

OR

What do you see as the biggest hurdles to doing qualitative research?

fi
• Basic theory and applications of
qualitative research.

• Interview guide development.

• Examples of good and bad
interviewing based on real studies.

• Delegation tips on where you can
assign data collection tasks to junior
team members and new researchers.
What we will cover
10

fi
Various knickknacks laid out.
01. Qualitative
research is…
11

fi
I’m a qualitative
researcher.
Shareinthechat.
12
More of a numbers
person—quant.
How familiar are you with qualitative research?
I’ve used some
qualitative
approaches in my
work.

fi
•1. Method: rigorous method that listens,
watches, and interacts with people.

•2. Mindset: approaching problems as
they exist within a person’s life and their
various relationships and environments

•3. Social justice: Acknowledging people
as complex, dynamic, and social beings.
Qualitative research is…de
fi
ned
13

fi
Let it
fl
ow
Ethnography
14
Explore and play, but I need some questions
answered
Ethnographic
approachto
qualitative
research
Just answer all my
questions, please
Survey

Mock up drawing of passengers boarding an air taxi at a vertiport.
15
IMAGE: TECH CRUNCH

16
IMAGE: TECH CRUNCH
Women in clinical sta
ff
room writing down notes.

fi
Close up sliced bread.
Slow research movement
Qualitative research
• Local is best—acknowledge di
ff
erences,
context.

• Value and preserve what’s working.

• Take pause for the now and resist racing
ahead.

• Strive towards knowledge, not info
acquisition.

• Build in iterative processes to inform work.
17
IMAGE: WITH SOURDOUGH

fi
Not so great for…

• How user-friendly something is
(usability testing)

• Getting precise figures to a specific
question (survey)

• Testing out a physical service / product
(demo)
18
Qualitative research is great for…

• Tailoring interventions for particular
populations and settings

• Show similarities and differences among
populations

• Understand local variations across and
within settings

• Developing person-informed systems and
policies.

fi
•1. Method: 2. Mindset: 3. Social justice
Qualitative research is…RECAP
19

fi
Food on plate and white plates
Stretch!
Bianca Bakery
20

fi
Drawing of computer on table with tablet and notepads.
02. Activity 1:
Preparing your
interview guide
21

22
RESEARCH
QUESTION
SECONDARY
LITERATURE REVIEW
ANALYSIS
SYNTHESIS
DISSEMINATION
DESIGN & METHODS
DATA COLLECTION &
MANAGEMENT
1 2 3
4
5
6
7
Define research scope and
objectives: QUESTION

Review existing
literature, trends,
and research
reports. STAT / INFO
THAT’S KNOWN
Recruit X. Develop
research materials and
data management plan
in prep for field. WHO
AND WHAT METHOD
Conduct audio and video
recorded interviews with X
about their knowledge,
attitudes, and practice of Y.
WHO TO TALK TO AND
ABOUT WHAT
Codify data and search for
patterns using qualitative
computer-assisted data analysis
software to identify potential
models. STATEMENTS /
THEORIES ABOUT DATA
Evolve statements /
theories into concrete
next steps according to
needs and gaps of
community / literature.
OPPORTUNITY AREAS /
DISCUSSION
Share out next steps via
final deliverable.
OUTCOMES / FINAL
DELIVERABLE / PEER-
REVIEWED ARTICLE /
NEXT GRANT
Research map (example)

Woman wearing white labcoat sitting in lab.
24

fi
•Kickoff meeting: align on research
objectives + design/methods + recruitment

•Secondary data: what’s been done (stats,
other departments, reports)

•Stakeholder interviews: buy-in + stakes

—>interview guide
Prep work

Developing interview guide
26

fi
•Rundown of roles + responsibilities

•Lead researcher: asking questions +
listening

•Secondary researcher: listening +
filling in + tech

•Expert: content or community
member
Prep work

Day of interview
27

Audio recording equipment on desk.
28

Laptop connected to monitor on desk.
29

fi
Activity 1
Using ethnographic
approach to strengthen
interview
Project goal: Design new Cancer Portal based
participants’ experiences.

Our task as ethnographers: What feedback
would you give the PI to this study so that this
interview guide better gets at the interviewee’s
experience with the Portal?

• Make a copy of Bad Interview Guide for your
team [LINK].

• Add comments for the PI to tweak approach—
not necessarily re-writing individual questions
30

fi
Smallgroups.

ProvidethePIfeedbackoninterviewguide.
31
How can we make this interview guide better so that it gets
at the interviewee’s experience?

Task: Make a copy of bad interview guide + add comments.

fi
Brick wall painted in bright colors in the shape of a face.
10 tips to better interviews [LINK].
Cheat sheet
1. No hypothetical questions.

2. No close-ended questions.

3. No leading questions.

4. Be polite.

5. Be quiet.

6. It’s not about you.

7. Avoid validation.

8. Keep it conversational.

9. Using silence.

10. Avoid intervention or dictating what’s normal.
Finding meaning in patterns
32

fi
Two women sitting at a desk talking and taking notes.
#1: No hypothetical questions
Better interviews, 1
• Avoid hypothetical or scenario questions

• Plant questions in actual situations, local

• Situate questions grounded in little details
they’ve already thrown out

• Example:

• XX Would you use this service in the
future? XX

• What’s missing from this service?
33

fi
Two women sitting at desk and talking.
#2: No close-ended questions
• Avoid yes/no questions—it’s not a survey

• Example:

• XX Do you like this EMR feature? XX

• How has this EMR feature helped you in your
work?

• XX Do you use cannabis a lot? XX

• When did you last use cannabis?
34

fi
Man standing and addressing group sitting at a desk.
#3: No leading questions
• Let participants paint the picture for you
in their own words—don’t
fi
ll it in for
them

• Example:

• XX Do you get testing every month?
If not, why not? XX

• How often do you go to the doctor /
clinic?
35

fi
Man and woman sitting at a desk talking.
#4: Be polite
• Avoid using “you guys” to refer to women and other
colloquiums (“Oh that sucks”)

• Start with titles (Dr., Mrs. X) and let them direct you on how
they want to be called

• Don’t test them or prove them wrong—you’re not trying to
uncover THE TRUTH

• Examples:

• XX That’s funny because in an earlier interview, someone
said that that hospital is closed now? XX

• XX So they just took your opioids for no reason at all? XX

• XX Interviewer: The stat I heard was that more women
are misdiagnosed for bladder cancer, is that true?
User: I don’t know that stat. XX
36

fi
Woman and man sitting at desk.
#5: Be quiet.
• Avoid “hmmms” and responding every
time participant responds

• Show you’re listening with non-verbal
clues
37

fi
Two women sitting down and talking in front of a computer.
#6: It’s not about you
• “Listening session”

• Example:

XX User: Back in September of last year, I had a
problem with my heart where it was beating too
fast. I don’t know if you know what an ablation is.

Interviewer: I do b/c my dad, yeah.

User: So they had to —

Interviewer: —My dad had to have open heart
surgery, and I remember—- XX
40

fi
Two men sitting down at table and talking.
#7: Avoid validation.
• Don’t validate or give +/- of you feel
about their answer

• Example:

XX Interviewer: Why do you prefer
fentanyl over heroin?

User: Convenience.

Interviewer: That’s cool. XX
41

fi
A woman drawing on a white pad in front of a group.
• Don’t push participant in a direction of how you want
them to respond (eg, what’s good / cool)

• ‘We’ve never heard that before’

• ‘That’s really great”

• Example:

XX Interviewer: Where do you frequently purchase
vaping products?

User: There’s a corner shop where I live—they let me
hang out and bring my dog so.

Interviewer: Awesome, I love that. XX
42

fi
A group of men and women working at a table together.
• Don’t need to give feedback

• Example:

XX User: I kinda just wanted to smoke
my weed at that party, and others were
doing it too so I did.

Interviewer: It’s normal. So have you
ever introduced anyone to smoking? XX
43

fi
A group of men and women looking at papers together.
#8: Keep it conversational.
• Not trying to get to all your questions or get
to next question—it’s not a survey.

• Example:

XX User: I mean I’m aware of the health risk
involved with vaping because last week—

Interviewer: —Ok. So some people vape b/c
it’s really hard to stop smoking cigarettes.
Would you say that sentence resonates with
you? XX
44

fi
Two men sitting at a table talking.
• It’s not a survey.

• Example:

XX Interviewer: Do you remember what you used the
fi
rst
time you used cannabis?

User: No I don’t.

Interviewer: Okay. Do you remember liking it?

User: Yeah.

Interviewer: Did it have mostly THC in it? B/c sometimes
people mix CBD…

User: No, I’m not sure. XX
45

fi
Two women sitting at a table talking.
• The order of the interview guide is just a suggestion

• Memorize the interview guide and have a second
interviewer ask follow up questions from the guide
and their listening

• Example:

XX User: Before it was like okay, there’s just a joint
and a bong but now there’s pens, and devices, and
all kinds of names for them it’s so funny.

Interviewer: Ok, ok we are actually about to get into
those questions right now. So—what’s in your joint?
XX
46

fi
A woman and man sitting in chairs and talking.
#9: Using silence
Better interviews 9
• When in doubt what to do with yourself,
take a pause

• Look at their body language—are they
tired and uncomfortable?

• Breaks and breaths—maybe they
need a bathroom break?
47

fi
Two women sitting at a table and talking to each other.
#10: Avoid intervention or dictating what’s normal
Better interviews,
10
• Not an opportunity to give medical advice
(even if you’re a medical professional!) or life
advice

• Example:

XX User: I used to run cross country but
since the accident, I’m so reliant on my pain
meds to just do basic walking. I know these
pills are killing me I know it.

Interviewer: Well we don’t know that yet. XX
48

fi
Men and women looking at papers together on desk.
#10: Avoid intervention or dictating what’s normal
Better interviews,
10
• Listening vs. reacting

• Don’t wait to get to sensitive question to say it’s
sensitive

• Example:

XX User: We’ll sometimes get together my dad and I
and smoke and drink together, like on the weekends.
We’ll like smoke some cannabis, nothing crazy.

Interviewer: Right so you guys will have a few beers
and eat dinner type thing?

User: Sort of…He doesn’t eat dinner. XX
49

fi
Woman with pompoms and eye stickers on her face.
Helen Kim, BFA
• Ethnographic projects:

• Ktown is my Town

• Home is where the heart is
VoyageLA
50

fi
Activity 2
Using ethnographic
approach to create focus
group guide
Project goal: Create a better COVID
testing experience.

Our task as ethnographers: Capture the
technical + conceptual in one guide.

• Make a copy of template [LINK].

• Write 2-5 questions for 5 min interview.

• Write probes to each question.
52

fi
Smallgroups.

CreateinterviewguideinGoogleDocs.
53
How can we create a better COVID testing experience?

Task: Make a copy of the template.

Write 2-5 questions + probes.

fi
Drawing of
fi
gures standing 2 meters from one another.
03. Activity 3:
Conducting your
interview
54
IMAGE: LSE

fi
Woman and man sitting on bench talking.
Activity 3
Conducting the one-
on-one interview
Project goal: Create a better COVID test
experience.

Our task as ethnographers: Maintain
conversation while weaving in your questions.

• Assign roles to interviewing team:

• 1: Research lead: conduct focus group

• 2: Secondary researcher: follow-up + key
takeaways / quotes
55

fi
Smallgroups

Conductinterview.
56
How can we create a better COVID testing experience?
(5 min each group)

Task for interviewers: Maintain conversation.

Task for interviewees: Answer truthfully based on actual
experiences.

fi
Woman standing and speaking with microphone to group of people
sitting.
04. Next
steps
57

fi
rceasar@usc.edu
Debrief+Q&A
64

In-depth Interview Focus group
Focuses on complexity and depth of one
participant
Promotes discussion between
participants on a single topic
Best for exploring sensitive, personal topics

(death, sex, substance use)

Best for exploring community norms
with a group that interacts with each
other

Used when participants are unique and
have conflicting views

Used when participants are similar to
one another (socioeconomic status, age)

Ideal for when participants are asked about
information they would be unlikely to share
in a group of strangers (marijuana use)
Ideal for when participants would be
reluctant to share information one-on-
one (idea generation of clinic design,
problem identification of ER service)

vs.

Request a Consult - BERD
https://sc-ctsi.org/resources/qualitative-
research-consultation

Request a Consult - ERC
https://sc-ctsi.org/resources/education-resource-center

Thank You!
SC CTSI | www.sc-ctsi.org Phone: (323) 442-4032 Email: info@sc-ctsi.org Twitter: @SoCalCTSI
Cite us: This work was supported by grants UL1TR001855 and UL1TR000130 from the National Center for Advancing
Translational Science (NCATS) of the U.S. National Institutes of Health. The content is solely the responsibility of the authors
and does not necessarily represent the official views of the National Institutes of Health.
3

Tips for collecting_high_quality_qualitative_data

More Related Content

What's hot

Similar to Tips for collecting_high_quality_qualitative_data

More from SC CTSI at USC and CHLA

Recently uploaded

Tips for collecting_high_quality_qualitative_data