Presentation to the Qualitative Methods in Psychology section of the British Psychological Society in Cambridge UK on September 3, 2015

1. A Best Practices Approach
to Guide (Not Stifle)
Qualitative Inquiry
Margaret R. Roller
Paul J. Lavrakas
British Psychological Society QMiP Conference ● 3 September 2015 ● Cambridge

2.  Steady rise in scholarly publications of qualitative.
 Increasing demand for mixed-method designs.
 Doctoral students are increasingly utilizing
qualitative research designs.
“Qualitative Movement” in Psychology
Roller & Lavrakas 1
* Gergen, Josselson, & Freeman, 2015.
*

3. Roller & Lavrakas 2

4. Roller & Lavrakas 3
Psychologists embrace the…
associated with the lived experience, as
well as the unique attributes of qualitative.

5. Roller & Lavrakas 4

6.  How to weave together the
essence of qualitative research
& its rich contextual data with
quality steps to maximize the
usefulness of our research.
Unique Attributes Bring Unique
Challenges
Roller & Lavrakas 5

7. “If it is agreed that qualitative research can, in fact,
serve worthwhile (‘good’) purposes, then logically it
would serve those purposes only to the degree that
it is done (‘executed’) well, regardless of the
specific objectives that qualitative researchers are
striving to address.”(Roller & Lavrakas, 2015)
Roller & Lavrakas 6

8.  Strategies & criteria for thinking about quality in qualitative
research are not new.
 Various validation & verification strategies exist (e.g.,
Brinkmann & Kvale, 2015; Creswell, 2013; Maxwell, 2013).
 Guidelines for conducting & assessing rigorous qualitative
research across epistemological and ontological
perspectives (Morrow, 2005; Meyrick, 2006).
 Lincoln & Guba (1985) criteria for judging the
“trustworthiness” of qualitative outcomes.
 A post-hoc assessment – Are the outcomes worth paying
attention to?
Quality in Qualitative Research
Roller & Lavrakas 7

9.  Goes beyond validation, verification, & post-hoc
assessments.
 Develops critical thinking skills among researchers (&
users of the research) by showing how to give explicit
attention to quality issues related to conceptualization,
implementation, analysis, & reporting.
Total Quality Framework
“Brings greater rigor to qualitative research without
stifling or squelching the creative approaches and
interpretations that skilled qualitative researchers properly
embrace, practice, and celebrate.” (Roller & Lavrakas, 2015)
Roller & Lavrakas 8

10. Total Quality Framework
 Guides the researcher with tools to think about best
practices at each stage of the research process
regardless of qualitative method.
 Enables users of the research to ask how well the
research was conceptualized, implemented,
interpreted, & reported.
 Empowers anyone to formulate their own conclusions
about the usefulness of the research.
 Not a or but rather a degree of confidence.
Roller & Lavrakas 9

11. Total Quality Framework
Reflexivity & self-reflection
Saturation
Transparency
Verification
Transferability
Roller & Lavrakas 10
 Relies heavily on important qualitative constructs:

12. Total Quality Framework
Credible
Analyzable
Transparent
Useful
Roller & Lavrakas 11
 Based on the core belief that all qualitative inquiry –
regardless of epistemological or theoretical
orientation – should be

13. Roller & Lavrakas 12

14. Roller & Lavrakas 13

15.  Credibility has to do with data collection & the best
practices that impact complete and accurate data
collection.
 Two main considerations are Scope – the representativeness
of participants to the target population – and Data Gathering
– the reliability, validity, & potential bias of our data.
 For instance, non-response bias can be a problem if people
recruited don’t show up or fail to cooperate. To mitigate
this bias, the researcher needs to use techniques that help
gain cooperation from participants, such as…
Credibility is the first component of the TQF
Roller & Lavrakas 14

16.  Practical matters related to recruitment – Do you
identify the sponsor? The level of personalization.
Degree of flexibility in the location or mode. Special
considerations for sensitive or stigmatizing topics.
 An important consideration in Scope is determining
the number of research events – during initial design
AND in the field.
 The grounded theory approach of saturation – collecting
data until no new ideas or themes emerge – doesn’t go
far enough. For example, there are at least 9 other
questions the researcher should ask in an IDI study…
Roller & Lavrakas 15

17. Roller & Lavrakas 16
10 Questions to Evaluate Interview Completions
2. Did all interviewees provide clear, unambiguous answers to key questions or issues,
or does the researcher need to go back to some interviewees for clarification?
5. Can the researcher identify the sources for variation and contradictions in the data,
and the extent of unexplained differences?
7. Do the data confirm or deny what is already known about the subject matter?

18. Roller & Lavrakas 17
 Also important to Credibility is how information is obtained.
A best practice to developing guides is a funnel approach.

19. Roller & Lavrakas 18

20.  Analyzability has to do with achieving complete
and accurate analysis and interpretation of the
data.
 This involves the 2 broad categories of Processing
and Verification.
Analyzability is the second component
of the TQF
Roller & Lavrakas 19

21.  Processing
 Changing format, usually to transcriptions
• Qualities of the transcriptionist, e.g., skills, knowledge.
 Making sense of the data
• The TQF offers best practices for: selecting a unit of
analysis, code development and coding, identifying
categories and themes, drawing interpretations and
implications.
• (continued)
Roller & Lavrakas 20

22.  Processing
 Making sense of the data
• Computer-assisted qualitative data analysis software
(CAQDAS).
– Can be useful for organizing & visualizing codes, as well as
inter-coder reliability, but it is not sufficient.
– “We are concerned about analysis that treats words as brute
data waiting to be coded, labeled with other brute words
(and even counted)…” (St. Pierre & Jackson, 2014)
– It is the human brain, not CAQDAS, that is needed to identify
latent meaning & contextual nuances in qualitative data.
Roller & Lavrakas 21

23.  Verification
 Add meaning & depth to the data to better inform
researcher’s interpretations & implications.
 Evidence to support &/or refute findings.
 Types:
• Peer debriefings
• Triangulation
• Deviant cases
• Reflexive journal
– The reflexive journal chronicles the researcher’s
assumptions, values, & beliefs, and forces the
researcher to reflect on his/her knowledge…
Roller & Lavrakas 22

24. Roller & Lavrakas 23
What do I think I “know” from this/these participants?
How do I think I “know” it?
What assumptions did I make (what did I assume to be true) about the participant(s)?
What assumptions did I make about comments/responses to my questions?
How did my personal values, beliefs, life story, and/or social/economic status affect or
shape the questions I asked, the interjections I made, my listening skills, and/or
behavior?
Reflexive Journal Format Example

25. Roller & Lavrakas 24

26.  Transparency is concerned with the final document
– its completeness & how well it discloses details
of the study.
 “In order for the outcomes of qualitative research
to be valued, the strategies used must be
transparent and open to scrutiny.” (Martin, Sadlo, & Stew,
2006)
Transparency is the third component
of the TQF
Roller & Lavrakas 25

27. The goal is to provide the user or reader of the
document with sufficient information to: judge the
strengths & limitations of the research, evaluate
his/her confidence in the outcomes, & determine
the applicability of the research to other contexts,
i.e., the transferability.
The focus in Transparency is on the rich details and
content provided by way of Thick Description.
For example, a thick description of an IDI study
should include the…
Roller & Lavrakas 26

28. Roller & Lavrakas 27
Thick Description Details for IDIs
 Adequacy (i.e., comprehensiveness) of the lists (sampling frame) that
were used to represent the target population.
 Failure to interview all interviewees sampled, efforts that were made
to avoid this, and possible biases or weakness this may have caused.
 Field notes (e.g., note-taking procedures, examples from the field notebook).

29. Roller & Lavrakas 28

30.  Usefulness is the ultimate goal.
 What can & should be done with the study now
that it has been completed? What Is It Good For?
 Has the study confirmed or refuted certain
hypotheses or previously-held theories?
 Has it identified knowledge gaps?
 Has it offered recommendations for next steps,
action?
 Has the documentation given sufficient information to
determine its transferability to other contexts?
Usefulness is the fourth component of the TQF
Roller & Lavrakas 29

31. Roller & Lavrakas 30
 Qualitative psychology requires an approach that brings
greater rigor to research design without stifling the essence
of qualitative inquiry.
 The Total Quality Framework (TQF) uniquely guides the
researcher by way of best practices at each research phase.
 Central to the TQF is that all qualitative research must be
credible, analyzable, transparent, & ultimately useful.
 TQF best practices lead to more rigorous & better quality
research, enables users of the research to evaluate their
confidence in the outcomes, & aids in writing and reviewing
dissertations, proposals, & manuscripts.
Summary

32. Roller & Lavrakas
Brinkmann, S., & Kvale, S. (2015). Interviews: Learning the craft of qualitative research
interviewing (3rd ed.). Thousand Oaks, CA: Sage.
Creswell, J. W. (2013). Qualitative inquiry & research design (3rd ed.). Thousand Oaks, CA:
Sage Publications.
Gergen, K. J., Josselson, R., & Freeman, M. (2015). The promises of qualitative inquiry.
American Psychologist, 70(1), 1-9.
Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Beverly Hills, CA: Sage Publications.
Martin, M., Sadlo, G., & Stew, G. (2006). The phenomenon of boredom. Qualitative Research in
Psychology, 3(3), 193-211.
References

33. Roller & Lavrakas
Maxwell, J. A. (2013). Qualitative research design: An interactive approach (3rd ed.).
Thousand Oaks, CA: Sage Publications.
Roller, M. R. & Lavrakas, P. J. (2015). Applied qualitative research design: A Total Quality
Framework approach. New York: Guilford Press.
St. Pierre, E. A., & Jackson, A. Y. (2014) Qualitative data analysis after coding. Qualitative Inquiry,
20(6), 715-719.
References (cont.)

34. Thank You!
Margaret R. Roller
rmr@rollerresarch.com
www.rollerresearch.com
www.researchdesignreview.com
Paul J. Lavrakas
pjlavrakas@centurylink.net