1




Survey Research and
Other Ways of Asking
Questions
OUTLINE
 Introduction
 Topics Appropriate to Survey Research
 Guidelines for Asking Questions
 Questionnaire Construction
 Self-Administered Questionnaires
 In-Person Interview Surveys
 Telephone Surveys
 Other Ways of Asking Questions
3




•Survey research is perhaps the most
frequently used mode of observation in
sociology and political science, and surveys
are often used in criminal justice research as
well
•You have no doubt been a respondent in
some sort of survey, and you may have
conducted a survey yourself
4




•Counting Crime: asking people about
victimization counters problems of data
collected by police
•Self-Reports: dominant method for studying
the etiology of crime
  •Frequency/type of crimes committed
  •Prevalence (how many people commit crimes)
  committed by a broader population
5




•Perceptions and Attitudes: To learn how
people feel about crime and CJ policy
•Targeted Victim Surveys: Used to evaluate
policy innovations & program success
•Other Evaluation Uses: e.g., Measuring
community attitudes, citizen responses, etc.
  •Chicago Community Policing Evaluation
  Consortium
•General Purpose Crime Surveys
6




•How questions are asked is the single most
important feature of survey research
•Open-Ended: Respondent is asked to provide
his or her own answer
•Closed-Ended: Respondent selects an answer
from a list
  •Choices should be exhaustive and mutually
  exclusive
•Questions and Statements – (Likert scale)
7



•Make Items Clear: Avoid ambiguous
questions; do not ask “double-barreled”
questions
•Short Items are Best: Respondents like to
read and answer a question quickly
•Avoid Negative Items: Leads to
misinterpretation
•Avoid Biased Items and Terms: Do not ask
questions that encourage a certain answer
•Designing Self-Report Items: Use of
computer assisted interviewing techniques
8




•General questionnaire format – critical, must
be laid out properly - uncluttered
•Contingency Questions: Relevant only to
some respondents – answered only based on
their previous response
•Matrix Questions: Same set of answer
categories used by multiple questions
9




•Ordering may affect the answers given
•Estimate the effect of question order
•Perhaps devise more than one version
•Begin with most interesting questions
•End with duller, demographic data
  •This is opposite for in-person interview
  surveys
10




•Can be home-delivered
  •Researcher delivers questionnaire to home of
  sample respondent, explains the study, and
  then comes back later
•Mailed (sent and returned) survey is most
common
  •Researchers must reduce the trouble it takes
  to return a questionnaire
11




•Used to increase response rates
•Warning Mailings: “Address correction
requested” card sent out to determine
incorrect addresses and to “warn” residents to
expect questionnaire in mail
•Cover Letters: Detail why survey is being
conducted, why respondent was selected, why
is it important to complete questionnaire
  •Include institutional affiliation or sponsorship
12




•Monitoring returns: Pay close attention to
the response rate, assign #’s serially
•Follow-up mailings: Nonrespondents can be
sent a letter, or a letter and another
questionnaire; timing
•Acceptable response rates: 50%? 60%?
70%?
  •We would rather have a lack of response bias
  than a high response rate?
13




•Via Fax, Email, Web Site/Page
•Issues
  •Representativeness
  •Mixed in with, or mistaken for, spam
  •Requires access to Web
  •Sampling frame?
14




•Typically achieve higher response rates than
mail surveys (80-85% is considered good)
•Demeanor and appearance of interviewer
should be appropriate; interviewer should be
familiar with questionnaire and ask questions
precisely
•When more than one interviewer
administers, efforts must be coordinated and
controlled
•Practice interviewing
15




•Reported success in enhancing confidentiality
•Reported higher rates of self-reporting
  •Computer-assisted personal interview (CAPI)
  – Interviewers read questions from screens
  and then type in answers from respondents’
  •Computer-assisted self-interviewing (CASI) –
  Respondent keys in answers, which are
  scrambled so that interviewer cannot access
  them
16




•95.5% of all households have telephones
(2005, US Census Bureau)
•Random-Digit Dialing
  •Obviates unlisted number problem
  •Often results in business, pay phones, fax lines
•Saves money and time, provides safety to
interviewers, more convenient
•May be interpreted as bogus sales calls; ease
of hang-ups
17




•A set of computerized tools that aid
telephone interviewers and supervisors by
automating various data collection tasks
•Easier, faster, more accurate but more
expensive
•Formats responses into a data file as they
are keyed in
•Can automate contingency questions and
skip sequences
18




•Self-administered questionnaires are generally
cheaper, better for sensitive issues than interview
surveys
•Using mail: Local and national surveys are same
cost
•Interviews: More appropriate when respondent
literacy may be a problem, produce fewer
incompletes, achieve higher completion rates
•Validity low, reliability high in survey research
•Surveys are also inflexible, superficial in coverage
19




•Two variations:
  •General interview guide: Less structured, lists
  issues, topics, questions you wish to cover; no
  standardized order
  •Standardized open-ended interview: More
  structured, specific questions in specific order;
  useful in case studies, retrieves rich detail in
  responses
20




•12-15 people brought together to engage in
guided group discussion of some topic
•Members are selected to represent a target
population, but cannot make statistical
estimates about population
•Most useful when precise generalization to
larger group is not necessary
•May be used to guide interpretation of
questionnaires following survey administration
21




•Consider start-up costs
•Finding, training, paying interviewers is time
consuming and not cheap, and requires some
expertise
•Mail surveys are less expensive, and can be
conducted by 1-2 persons well
•The method you use depends on your
research question

Ch09 Survey Research

  • 1.
    1 Survey Research and OtherWays of Asking Questions
  • 2.
    OUTLINE  Introduction  TopicsAppropriate to Survey Research  Guidelines for Asking Questions  Questionnaire Construction  Self-Administered Questionnaires  In-Person Interview Surveys  Telephone Surveys  Other Ways of Asking Questions
  • 3.
    3 •Survey research isperhaps the most frequently used mode of observation in sociology and political science, and surveys are often used in criminal justice research as well •You have no doubt been a respondent in some sort of survey, and you may have conducted a survey yourself
  • 4.
    4 •Counting Crime: askingpeople about victimization counters problems of data collected by police •Self-Reports: dominant method for studying the etiology of crime •Frequency/type of crimes committed •Prevalence (how many people commit crimes) committed by a broader population
  • 5.
    5 •Perceptions and Attitudes:To learn how people feel about crime and CJ policy •Targeted Victim Surveys: Used to evaluate policy innovations & program success •Other Evaluation Uses: e.g., Measuring community attitudes, citizen responses, etc. •Chicago Community Policing Evaluation Consortium •General Purpose Crime Surveys
  • 6.
    6 •How questions areasked is the single most important feature of survey research •Open-Ended: Respondent is asked to provide his or her own answer •Closed-Ended: Respondent selects an answer from a list •Choices should be exhaustive and mutually exclusive •Questions and Statements – (Likert scale)
  • 7.
    7 •Make Items Clear:Avoid ambiguous questions; do not ask “double-barreled” questions •Short Items are Best: Respondents like to read and answer a question quickly •Avoid Negative Items: Leads to misinterpretation •Avoid Biased Items and Terms: Do not ask questions that encourage a certain answer •Designing Self-Report Items: Use of computer assisted interviewing techniques
  • 8.
    8 •General questionnaire format– critical, must be laid out properly - uncluttered •Contingency Questions: Relevant only to some respondents – answered only based on their previous response •Matrix Questions: Same set of answer categories used by multiple questions
  • 9.
    9 •Ordering may affectthe answers given •Estimate the effect of question order •Perhaps devise more than one version •Begin with most interesting questions •End with duller, demographic data •This is opposite for in-person interview surveys
  • 10.
    10 •Can be home-delivered •Researcher delivers questionnaire to home of sample respondent, explains the study, and then comes back later •Mailed (sent and returned) survey is most common •Researchers must reduce the trouble it takes to return a questionnaire
  • 11.
    11 •Used to increaseresponse rates •Warning Mailings: “Address correction requested” card sent out to determine incorrect addresses and to “warn” residents to expect questionnaire in mail •Cover Letters: Detail why survey is being conducted, why respondent was selected, why is it important to complete questionnaire •Include institutional affiliation or sponsorship
  • 12.
    12 •Monitoring returns: Payclose attention to the response rate, assign #’s serially •Follow-up mailings: Nonrespondents can be sent a letter, or a letter and another questionnaire; timing •Acceptable response rates: 50%? 60%? 70%? •We would rather have a lack of response bias than a high response rate?
  • 13.
    13 •Via Fax, Email,Web Site/Page •Issues •Representativeness •Mixed in with, or mistaken for, spam •Requires access to Web •Sampling frame?
  • 14.
    14 •Typically achieve higherresponse rates than mail surveys (80-85% is considered good) •Demeanor and appearance of interviewer should be appropriate; interviewer should be familiar with questionnaire and ask questions precisely •When more than one interviewer administers, efforts must be coordinated and controlled •Practice interviewing
  • 15.
    15 •Reported success inenhancing confidentiality •Reported higher rates of self-reporting •Computer-assisted personal interview (CAPI) – Interviewers read questions from screens and then type in answers from respondents’ •Computer-assisted self-interviewing (CASI) – Respondent keys in answers, which are scrambled so that interviewer cannot access them
  • 16.
    16 •95.5% of allhouseholds have telephones (2005, US Census Bureau) •Random-Digit Dialing •Obviates unlisted number problem •Often results in business, pay phones, fax lines •Saves money and time, provides safety to interviewers, more convenient •May be interpreted as bogus sales calls; ease of hang-ups
  • 17.
    17 •A set ofcomputerized tools that aid telephone interviewers and supervisors by automating various data collection tasks •Easier, faster, more accurate but more expensive •Formats responses into a data file as they are keyed in •Can automate contingency questions and skip sequences
  • 18.
    18 •Self-administered questionnaires aregenerally cheaper, better for sensitive issues than interview surveys •Using mail: Local and national surveys are same cost •Interviews: More appropriate when respondent literacy may be a problem, produce fewer incompletes, achieve higher completion rates •Validity low, reliability high in survey research •Surveys are also inflexible, superficial in coverage
  • 19.
    19 •Two variations: •General interview guide: Less structured, lists issues, topics, questions you wish to cover; no standardized order •Standardized open-ended interview: More structured, specific questions in specific order; useful in case studies, retrieves rich detail in responses
  • 20.
    20 •12-15 people broughttogether to engage in guided group discussion of some topic •Members are selected to represent a target population, but cannot make statistical estimates about population •Most useful when precise generalization to larger group is not necessary •May be used to guide interpretation of questionnaires following survey administration
  • 21.
    21 •Consider start-up costs •Finding,training, paying interviewers is time consuming and not cheap, and requires some expertise •Mail surveys are less expensive, and can be conducted by 1-2 persons well •The method you use depends on your research question