Qualitative workshop

1. Managing
Qualitative Research
Khalid Mahmood, PhD
Professor of Library & Information Science
University of the Punjab
1

2. Acknowledgement
 This presentation is based on many
books, notes, websites and presentations
on the topic.
 The presenter pays his sincere gratitude to
all authors, professors and experts for
their efforts and contributions.
2

3. Agenda
 What is qualitative research?
 Qualitative traditions of inquiry
 Steps in qualitative study
 Ethical considerations
 Sampling
 Types of data
 Data collection
 Data analysis
 Validity, reliability and generalizability
3

4. What is
qualitative research?
4

5. Qualitative research…
 Allows the researcher to understand a
problem or phenomenon from the
perspectives of the people it involves.
 Reveals a complete picture of a certain
research issue.
 Seeks to provide a rich understanding of a
certain research issue.
5

6. In qualitative methods…
 Researcher collects data in a real environment.
 Researcher himself/herself is the key research
tool.
 Focus of research is a process or activity itself,
not just results of that process or activity.
 Data collected is most often verbal (non-
numerical).
 Verbal data analysis (rarely numerical).
6

7. Comparison of quantitative and
qualitative methods
QUALITATIVE QUANTITATIVE
Multiple realities Single reality
Reality is socially constructed Reality is objective
Reality is context interrelated Reality is context free
Holistic Reductionistic
Strong philosophical perspective Strong theoretical base
Reasoning is inductive Reasoning is deductive and
inductive
Discovery of meaning is the basis of
knowledge
Cause-and-effect relationships are
the bases of knowledge
Develops theory Tests theory
7

8. Comparison of quantitative and
qualitative methods(continued)
QUALITATIVE QUANTITATIVE
Meaning of concepts Measurement of variables
Process oriented Outcome oriented
Control unimportant Control important
Rich descriptions Precise measurement of variables
Basic element of analysis is words Basic element of analysis is
numbers
Uniqueness Generalization
Trustworthiness of findings Control of error
8

9. Qualitative traditions
of inquiry
9

10.  Biography
 Historical research
 Phenomenology
 Grounded theory
 Ethnography
 Ethnology
 Case study
 Symbolic interaction
10

11. Biography
 The study of an individual and her or his
experiences as told to the researcher or
found in documents and archival material.
 Life history—The study of an individual’s life
and how it reflects cultural themes of the
society.
11

12. Biography (continued)
 Oral history—The researcher gathers personal
recollections of events, their causes, and their
effects from an individual or several individuals.
 The researcher needs to collect extensive
information about the subject of the biography.
 The writer, using an interpretive approach,
needs to be able to bring himself or herself into
the narrative and acknowledge his or her
standpoint.
12

13. Historical research
 Studies available data to describe,
understand, and interpret past events.
 Uses primary sources of information.
 Does external and internal criticism of
documents or artifacts.
13

14. Phenomenology
 Describes the meaning of the lived experience
about a concept or a phenomenon for several
individuals.
 Determines what an experience means for the
persons who have had the experience and are
able to provide a comprehensive description of
it. From the individual descriptions, general or
universal meanings are derived, in other words,
the essences of structures of the experience.
14

15. Grounded theory
 Intends to generate or discover a theory that
relates to a particular situation.
 If little is known about a topic, grounded theory is
especially useful.
 Because the theory emerges from the data, it is
said to be grounded in the data.
 Data collection and analysis occur simultaneously,
until “saturation” is reached.
 Data reviewed and coded for categories and
themes.
15

16. Ethnography
 A description and interpretation of a cultural
or social group or system.
 The researcher examines the group’s
observable patterns of behavior, customs,
and ways of life.
 Involves prolonged observation of the group,
typically through participant observation.
16

17. Ethnography (continued)
 Field work
 Key informants
 Thick description
 Emic (insider group perspective) and Etic
(researcher’s interpretation of social life).
 Context important, needs holistic view.
 Needs grounding in anthropology.
17

18. Ethnography (continued)
 Many ethnographies may be written in a
narrative or story telling approach which may
be difficult for the audience accustomed to
usual social science writing.
 May incorporate quantitative data and
archival documents.
18

19. Ethnology
 Compares and analyzes the origins,
distribution, technology, religion, language,
and social structure of the ethnic, racial,
and/or national divisions of humanity.
19

20. Case study
 An exploration of a “bounded system” or a
case (or multiple cases) over time through
detailed, in-depth data collection involving
multiple sources of information rich in context.
 The context of the case involves situating the
case within its setting which may be physical,
social, historical and/or economic.
20

21. Symbolic interaction
 Investigates how people construct
meaning and shared perspectives by
interacting with others.
21

22. Steps in
qualitative study
22

23. 1. General research question
2. Select relevant site(s) and subjects
3. Collection of relevant data
4. Interpretation of data
5. Conceptual and theoretical work
6. Tighter specification of the research question
7. Collection of further data
8. Conceptual and theoretical work
9. Write up findings
23

24. 24

25. Ethical
considerations
25

26.  Mutual respect and trust
(prolonged interaction)
 Respect for social and cultural contexts
 Voluntary participation
 Informed consent
 Beneficence – doing good for others and
preventing harm
 Confidentiality
26

27. Sampling
27

28. Determining a sample
 Even if it were possible, it is not necessary to
collect data from everyone in a community.
 In qualitative research, the researcher needs to
define and select a sample.
 The study’s research objectives and the
characteristics of the study population determine
which and how many people to select.
28

29. Sample size
 Usually smaller than quantitative study.
 Two general guidelines: the number of
participants is sufficient when…
 the extent to which the selected participants represent
the range of potential participants in the setting
 the point at which the data gathered begin to be
redundant (data saturation)
29

30. Sampling methods
 No probability sampling
 Three of the most common sampling
methods are:
Purposive sampling
Quota sampling
Snowball sampling
30

31. Purposive sampling
 Purposive sampling groups participants according to pre-
selected criteria relevant to a particular research
question.
 ex. Vietnamese businessmen in the USA
 Sample sizes depend on:
 Resources and time available
 The study’s objectives
 If the researcher needs a specific number of participants,
quota sampling is better.
31

32. Quota sampling
 Quota sampling begins with two decisions:
 What characteristics?
 How many people?
 Characteristics are selected in order to find
participants who have experience with or
knowledge of the research topic.
 The researcher goes into the community and
selects the predetermined number of people
demonstrating the pre-selected characteristics.
32

33. Snowball sampling
 Snowball sampling is a form of purposive
sampling.
 Participants refer the researcher to other
potential participants.
 Snowball sampling is often used to find
and recruit “hidden populations” – groups
not easily accessible to researchers.
33

34. Types of data
34

35.  Written field notes
 Audio recordings of conversations
 Video recordings of activities
 Diary recordings of activities / thoughts
 Documents
 Depth information on:
 thoughts, views, interpretations
 priorities, importance
 processes, practices
 intended effects of actions
 feelings and experiences
35

36. Data collection
36

37.  Three data collection strategies:
1. Participant observation
2. In-depth interviews
3. Focus group interviews
 Qualitative researchers may combine
more than one method
37

38. Participant observation
 Intensive, usually long term, examination of a
social group, an organization, etc.
 Researcher becomes a participant in the lives of
group members
 Observes their behavior and learns meaning systems
(which are tied to language)
 Most closely associated with Ethnography, as
developed in Classical Anthropology
 Now done in a variety of disciplines
38

39. Participant observation (continued)
 Today most ethnographers take an overt
role
i.e., their identity as a researcher is known
to the people being studied
 Covert participation (i.e., identity
concealed from participants) is fraught
with ethical issues
39

40. Steps involved in participant
observation research
A. Gaining entry into the group
B. Developing and maintaining rapport
C. Developing a method for taking field
notes
D. Integrating data collection and data
analysis
40

41. Steps in participant observation:
Gaining entry into the group
 Take into consideration the type of group
 formal organizations require formal entry; involves
letter writing, permission requests, etc.
 Informal groups – different strategy needed
 Access may be gained through a gatekeeper
(an individual with special status)
 Want to involve key informants (those who are
most knowledgeable about the group)
41

42. Steps in participant observation:
Developing/maintaining rapport
 Researcher must work hard to develop
and maintain good relationships in the
field
e.g., be sure not to become associated
with one faction in a group or organization
 Researcher could be blamed for
problems that arise in the setting
42

43. Steps in participant observation:
Strategies for taking field notes
 Include descriptions and interpretations of
individuals, interactions, and events
 Distinguish descriptions from interpretations
 Record time and location of observations, as
well as key information (weather, events
happening and their significance)
 Keep theoretical memos – which are the
tentative interpretations emerging and being
assessed through further data collection
43

44. Field notes (continued)
 May not be possible or advisable to take
notes while in the field
 Important that they be done as soon after field
observation as possible
 Note-taking is time-consuming because it is
integral to guiding the data collection and
continuing the analysis
 e.g., field notes for When Prophecy Failed were
well over 1,000 typed pages
44

45. Steps in participant observation:
Integrating data collection and analysis
 Organizing field notes into different types of files
facilitates data analysis
 Master field file – complete journal of field notes;
number pages and include entry dates
 Background, history file – subfile organizing
background material
 Key character files – subfiles on key players in
the group or organization
 Analytic files – subfiles for different types of
observations or relationships
45

46. In-depth interviews
 Some studies cannot employ the participant
observation method
 In-depth interviews allow participants to describe
their experiences and the meaning of events
taking place in their lives
 Verbatim quotes capture the language and meaning
expressed by participants
 Interviews are flexible and allow for probing
 Interview method is quite diverse, adaptive
46

47. In-depth interviews(continued)
 Three key elements for the interview method to be successful:
1. Explicit purpose – researcher and informant are aware that the
discussion has a purpose
2. Ethnographic explanations – researcher tries out explanations on
the participants to see if they make sense
 Encourage the informants to use colloquial language, and teach the
researcher its meaning
3. Ethnographic questions include:
i. Descriptive questions – ask participants to describe their experiences
(e.g., their ideas, circumstances, viewpoints, dilemmas, etc)
ii. Structural questions – ask participants how they organize their world
(e.g., activities)
iii. Contrast questions – ask participants what is meant by specific
terminology
47

48. Interview do’s and don’ts
 Do listen more and talk less
 Do follow up on what is not clear and probe more deeply into
what is revealed
 Don’t use leading questions; do use open-ended questions
(“probes”)
 Don’t interrupt; do wait
 Do keep interviewee(s) focused
 Don’t be judgmental about or react to an interviewee’s
opinions, views, or beliefs
 Don’t engage in debate with an interviewee
 Do record everything the interviewee says and note
impressions of interviewee’s nonverbal behavior
48

49. Focus group interviews
 Interview format, but in a group setting
 6-12 participants with common experience
 Dates back to the 1940s – used to assess
effectiveness of morale-boosting radio shows
 1970s onward – used by market researchers
 1980s onward – used by academics
 Transcript of discussion is the data
 Plus accompanying notes
 Use content analysis or grounded theory approach to
analyze the data
49

50. Focus group interviews (continued)
 Strengths:
 Open-ended question
 Spontaneously deal with issues as they arise
 Cost-effective method of collecting data
 Less time-consuming
 Weaknesses:
 One or two participants may dominate
 Not done in a natural setting, so little “observation” to
help understand the experience of the participants
50

51. Data analysis
51

52.  Open coding
 Systematic coding
 Affinity diagramming
52

53. Open coding
 Treat data as answers to open-ended
questions
ask data specific questions
assign codes for answers
record theoretical notes
53

54. Example: Calendar routines
 Families were interviewed about their calendar
routines
 What calendars they had
 Where they kept their calendars
 What types of events they recorded
 …
 Written notes
 Audio recordings
54

55. Example: Calendar routines
 Step 1: translate field notes (optional)
paper digital 55

56. Example: Calendar routines
 Step 2: list questions / focal points
Where do families keep their calendars?
What uses do they have for their calendars?
Who adds to the calendars?
When do people check the calendars?
…
56

57. Example: Calendar routines
 Step 3: go through data and ask questions
Where do families keep their calendars?
57

58. Example: Calendar routines
 Step 3: go through data and ask questions
Where do families keep their calendars?
[KI]
Calendar Locations:
[KI] – the kitchen
[KI]
[KI]
58

59. Example: Calendar routines
 Step 3: go through data and ask questions
Where do families keep their calendars?
[KI]
Calendar Locations:
[KI] – the kitchen
[CR] – child’s room
[CR]
59

60. Example: Calendar routines
 Step 3: go through data and ask questions
Continue for the remaining questions….
[KI]
Calendar Locations:
[KI] – the kitchen
[CR] – child’s room
[CR]
60

61. Example: Calendar routines
 The result:
list of codes
frequency of each code
a sense of the importance of each code
frequency != importance
61

62. Example 2: Calendar contents
 Pictures were taken of family calendars
62

63. Example: Calendar contents
 Step 1: list questions / focal points
What type of events are on the calendar?
Who are the events for?
What other markings are made on the calendar?
…
63

64. Example: Calendar contents
 Step 2: go through data and ask questions
What types of events are on the calendar?
64

65. Example: Calendar contents
 Step 2: go through data and ask questions
What types of events are on the calendar?
Types of Events:
[FO] – family outing
[FO]
65

66. Example: Calendar contents
 Step 2: go through data and ask questions
What types of events are on the calendar?
Types of Events:
[FO] – family outing
[AN] - anniversary
[FO]
[AN]
66

67. Example: Calendar contents
 Step 2: go through data and ask questions
Continue for the remaining questions….
Types of Events:
[FO] – family outing
[AN] - anniversary
[FO]
[AN]
67

68. Reporting results
 Find the main themes
 Use quotes / scenarios to represent them
 Include counts for codes (optional)
68

69. Software: Microsoft Word
69

70. Software: Microsoft Excel
70

71. Software: ATLAS.ti
71

72. Software: NVivo
72

73. Systematic coding
 Categories are created ahead of time
from existing literature
from previous open coding
 Code the data just like open coding
73

74. Affinity diagramming
 Goal: what are the main
themes?
 Write ideas on sticky notes
 Place notes on a large wall /
surface
 Group notes hierarchically to
see main themes
74

75. Example: Calendar field study
 Families were given a
digital calendar to use in
their homes
 Thoughts / reactions
recorded:
 Weekly interview notes
 Audio recordings from interviews
75

76. Example: Calendar field study
 Step 1: Affinity notes
go through data and write observations down
on post-it notes
each note contains one idea
76

77. Example: Calendar field study
 Step 2: Diagram building
place all notes on a wall / surface
77

78. Example: Calendar field study
 Step 3: Diagram building
move notes into related columns / piles
78

79. Example: Calendar field study
 Step 3: Diagram building
move notes into related columns / piles
79

80. Example: Calendar field study
 Step 3: Diagram building
move notes into related columns / piles
80

81. Example: Calendar field study
 Step 3: Diagram building
move notes into related columns / piles
81

82. Example: Calendar field study
 Step 3: Diagram building
move notes into related columns / piles
82

83. Example: Calendar field study
 Step 3: Diagram building
move notes into related columns / piles
83

84. Example: Calendar field study
 Step 3: Diagram building
move notes into related columns / piles
84

85. Example: Calendar field study
 Step 4: Affinity labels
write labels describing each group
85

86. Example: Calendar field study
 Step 4: Affinity labels
write labels describing each group
Calendar placement
is a challenge
86

87. Example: Calendar field study
 Step 4: Affinity labels
write labels describing each group
Calendar placement
is a challenge
Interface visuals
affect usage
87

88. Example: Calendar field study
 Step 4: Affinity labels
write labels describing each group
Calendar placement
is a challenge
Interface visuals
affect usage
People check the
calendar when not at
home
88

89. Example: Calendar field study
 Step 5: Further refine groupings
Calendar placement
is a challenge
Interface visuals
affect usage
People check the
calendar when not at
home
89

90. Validity, reliability and
generalizability
90

91. Threats to validity
 Observer bias
 Invalid information resulting from the perspective the
researcher brings to the study and imposes upon it
 e.g., studying one’s own culture
 Observer effects
 The impact of the observer’s participation on the
setting or the participants being studied
 e.g., people may do things differently
91

92. Strategies to enhance validity
 Intensive, long term involvement
 more data, repeated observation and interviews
 Rich data
 full and detailed descriptions
 Respondent validation
 ask them if the reporting is correct
 Intervention
 interact with them and see how behavior changes
 Searching for negative cases and alternative explanations
 Triangulation
 collect data from a variety of settings and methods
 Quasi-statistics
 e.g., frequency counts of the argument
 Comparison
 multicase, multisite studies
92

93. Reliability
 It is a quantitative measure.
 This concept is irrelevant in qualitative
research.
 However, to test a qualitative study for
reliability, you need to convert data into
relevant numbers and determine efficacy
based on the results.
93

94. Generalization
 A generalization is usually thought of as a statement or
claim that applies to more than one individual, group, or
situation.
 The value of a generalization is that it allows us to have
expectations about the future.
 A limitation of qualitative research is that there is seldom
justification for generalizing the findings of a particular
study.
 Due to this problem, replication of qualitative studies
becomes more important than for quantitative studies.
94

95. Thanks to all
participants
95