1. The Connection of Interviewer Behavior and Characteristics with Responses to Open-Ended Questions
Erica Dalzell
Faculty Advisor, Merry Morash, Ph.D.
Doctoral Student Advisor, Elizabeth Adams, M.S.
Department of Criminal Justice, Michigan State University, East Lansing, MI
Abstract
Purpose
Limitations & Future Directions
Conclusion
•Determine if there is a connection between the ways a question
is framed with the response detail given.
•Potential for changes in the way interviewers are trained, hired,
and partnered with participants.
•Potential for changes in interview training to lead to richer, more
detailed data collected which would enhance research findings.
Recommendations Based on Findings
•Interviewers hired to conduct qualitative interviews should be trained
on how to ask questions to obtain fewer short responses and more
rich relevant detailed responses.
• Interviewers should be provided with examples of desired responses
to questions and also provided information on what type of question or
probe could be used to obtain the desired response.
•Interviewers should be trained on identifying questions that apply to
their participant, and whether they need to probe for more information
on responses.
•Interviewer’s question and probe framing should be assessed
periodically, and they should be provided with feedback on their
performance and tips on how to gather more detailed responses.
Recommendations for Future Research
•Due to a small sample size, similar research should be replicated with
larger samples.
•External validity may be limited due to the unique sample of this
research being limited to content from interviews with women on
supervision in Michigan, so similar research should be conducted on
different populations.
•Future research should examine the variation in responses based on
individual interview questions.
•Future studies should examine framing and responses in relation to
question placement as the first question of this interview was asked
word for word about 79% of the time with about 46% of these answers
being less than adequate, while the other 11 questions were asked
word for word about 23% of the time with less than adequate
responses about 48.6% of the time.
•Future studies should examine variation in the question and probe
framing in relation to responses based on race matching and non-
matching between interviewers and participants.
Methods
Contact Information
To best train interviewers to gather detailed qualitative data, it is
important to determine the response outcomes related to the
structure of the interview, framing of interview questions, and the
type of follow-up questions asked. The intent of this research is to
examine the connection between the way an interviewer asks a
series of questions and the detail and length of the participant’s
responses. The research was conducted by coding for variation in
the presentation of questions in association with the detail and
length of responses in a sample of 52 interviews with 11
interviewers who questioned 52 women about their prior offenses
The sample was selected to include an equal number of
interviewer-participant race matches and mismatches, so
matching could be studied in future research. More specifically
the sample was stratified based on interviewer and participants
with common (N=26) and differing (N=26) race. The outcomes of
this research could have an impact on the design of semi-
structured interview questions and the way interviewers are
trained to ask questions and probe for detailed responses.
Descriptive Data
Erica Dalzell
dalzelle@msu.edu
•Responses to confirming questions are likely to be short (85% of the
time), but the potential for no response or a rich relevant detailed
response are the same at about 7%.
•The likelihood that a participant will not respond (1%-2%) or lack
comprehension (6%-7%) of the question asked will remain nearly
constant (differing by 1% or less) regardless if whether the interviewer
asks the questions word for word, with slight variation, or an expanded
framing.
•The percentage of short responses is likely to decrease by 10% when
the interviewer asks the question word for word (40%), with slight
variation (30%), or using an expanded frame (20%).
•The likelihood that a participant will give a response with rich relevant
detail increases by about 13% when the interviewer asks a question using
an expanded frame versus asking the question word for word.
•Interviewers are about 12.6% more likely to get an adequate response
after asking an extra related probe versus a confirming probe, but only
are about half as likely (6%) to get a response with rich relevant detail
after asking an extra related probe versus a confirming probe.
•Interviewers were likely to skip a question when it was needed about
15% of the time and not use probes when they were necessary about
32% of the time.
Sample Selection
The sample of interviews was stratified by whether interviewer and
participant race matched, and then interviews within each group
were randomly selected. The sample then included 52 interviews
with participant and interviewer race matching as white (n=13) and
non-matching (n=13), and participant and interviewer race matching
as black (n=13) and non-matching (n=13).
Interview Questions
Q1: Can you tell me about the offense(s) that led to your current
conviction?
Q2: What actually happened? What events led up to your arrest?
Q3: Did you commit this offense with another person?
Q4: What was his or her role in the offense?
Q5: How did you get hooked up with this person?
Q6: Was there a victim?
Q7: Do you know what happened to that person?
Q8: Was anyone affected or hurt by your offense?
Q9: Were you treated fairly by the criminal justice officials?
Q10: How did the police treat you? How did the judge treat you?
Q11: Do you think your sentence was fair?
Q12: How do you feel about this now, looking back on it?
Obtaining Reliability
The reliability was obtained by having two individuals code one case
together using a codebook mutually designed. The individuals then
separately coded five cases and had an agreement rate of 74%. The
codebook was redesigned and the individuals separately coded
another 15 cases. These 15 cases provided a 96.6% agreement
rate between the two individuals.