This document provides an overview of survey research methods. It discusses the history and uses of survey research, as well as strengths and weaknesses. Different survey methods like mail, telephone, and online surveys are examined. Key aspects of survey design like sampling, questionnaire construction, and question types are also reviewed. The document provides guidance on best practices for writing clear, unbiased questions and addressing issues that could influence responses.

1. Survey Research
Neuman and Robson Ch. 8

2. Introduction to Survey Research
 Survey research is a very old, and a very
popular, research technique
 1880 survey by Karl Marx in France
 25000 questionnaires sent to workers
 Late 19th
century “Chocolate Sociologists”
 Rowntree and Cadbury
 Community surveys to study poverty

3. Topics appropriate to survey
research:
 descriptive, explanatory, and exploratory
research purposes
 best method available to collect original
information about a population
 To measure attitudes and orientations

4. General features of survey research
 respondents are chosen through probability
sampling procedures
 Systematic questionnaire or interview
procedures
 Sophisticated statistical techniques

5. Strengths and Weaknesses of Survey Research:
 Advantages
 Reliability
 Can describe large population
 Flexibility
 Standardized
 Disadvantages
 Validity
 Superficiality
 Cannot modify questionnaire in field
 Artificiality

6. Public Opinion Polling Canada
 No official or mandatory regulation
 i.e. election polls
 Could influence voters
 “bandwagon” phenomenon
 “free will” effect
 Problems and discrepancies:
 20-40% answer “don’t know” or “no answer”
 Results often published as if out of 100%
 misrepresentation

7. Mail Surveys and Self-Administered
Questionnaires
 Hand-administered to a group or mailed to sample
 Mail distribution and return practices
 Monitoring returns
 Follow-up mailings
 Response rates: 50% adequate, 70% very good
 Additional factors:
 Sponsorship
 Inducement to respond

8. Disadvantages of mail surveys:
 Requires simple questions
 Cannot observe respondents
 No opportunity for probe
 No control over conditions or who responds
 Not suited for people who are illiterate or for
whom English not first language
 Low response rate can be a problem

9. Face-to-Face Interview
Surveys
 Very expensive
 Interviewers require training
 Use interview schedule
 Appearance and demeanour crucial
 Need familiarity with questionnaire
 Must follow question wording exactly
 Must record responses exactly
 Also record other events during interview

10. Advantages and Disadvantages of
Face-to-Face Interview Surveys
 Advantages:
 Can probe for responses
 Can keep respondent “on task”
 Disadvantages:
 Cost and training
 Interviewer bias

11. Telephone Surveys
 In past, could reach 95% of households
 Now, between 20 and 26% of Canadians
have no landline (see next slide)
 Use of telephone interview schedules
 Selecting a sample for telephone surveys
 Telephone directory
 Random Digit Dialing (RDD)
 Selecting a respondent from a household

12. Percentage of households in Canada that
use a cell phone only
(Source: Stats Can “The Daily”, 2014)
2008 2010 2013
All households 8 12.8 21
Households under 35
years of age 26.1 39.3 60.6
Households aged 55 and
older 1.9 3.8 6.4

13. Telephone Surveys
 Can achieve higher response rates
 Can decrease "don't knows" and "no answers”
 Interviewers can help clarify
 Telephone surveys have many advantages of face-
to-face but without high cost

14. Advantages and Disadvantages of
telephone surveys
 Advantages:
 cheaper to carry out than face-to-face interviews
 less time and effort
 more impersonal
 Disadvantages:
 large number of tries to achieve a small number of
successes.
 less motivation generated among respondents
 No visual cues/aids possible
 Limited interview length

15. Computer-assisted telephone
interviewing (CATI)
 interviewers sit at a computer terminal with a
screen
 answers given are directly entered into the
computer
 Very fast
 Often used in polling (i.e. Video: Ask a Silly
Question)

16. Internet (Web) Surveys
 In Canada, 95 per cent of Canadians in the highest
income quartile are connected to the Internet, yet
only 62 per cent in the lowest income quartile have
Internet access (2014)
 Use same basic design as mail surveys
 Advantages:
 Fast, cheap
 Can use graphic and visual effects
 Disadvantages:
 Coverage – not all groups use computers
 Privacy – need for encryption

17. Comparison of Survey Methods
Criteria Interview Mail Telephone Web
_________________________________________________________________
1. Cost high low moderate very low
2. Response rate high low high moderate
3. Level of control of high low moderate low
situation
4. Applicability to moderate high moderate high
geographically
dispersed populations
5. Applicability to high low high moderate
heterogeneous
populations
6. Obtaining detailed high moderate moderate low
information
7. Speed slow slow fast fastest

18. Secondary Analysis of Survey Data
 New analysis of data collected for another purpose
 Use of shared data and data archives (i.e. Census
and GSS)
 Advantages
 cheaper and faster
 Access to larger datasets
 Disadvantages
 problems of validity
 Questions may not meet your needs
 Sample may not be adequate

19. Questionnaire Construction
 usually includes a variety of demographic
questions, and one or more scales and
indexes aimed at collecting data on attitudes,
beliefs, or behaviours
 purpose of a survey is to try to collect
information about a sample in order to make
generalizations about the larger population

20. The general structure of a
good questionnaire:
 1. Title
 2. A brief introduction and explanation of research.
 3. A section of general demographic questions re:
gender, age, marital status, education. Note: more
sensitive demographic questions (i.e. income, home
value, etc.) might be better placed at the end of the
questionnaire.
 4. Section of non-threatening or less sensitive
behavioural and attitudinal questions.
 5. A section near the end with any sensitive or
threatening questions.
 6. Any additional demographic information.
 7. Conclusion with brief statement again thanking
respondents for their time and effort

21. Additional Considerations:
 1. Keep questionnaire as short as is reasonably
possible.
 2. Appearance of the questionnaire is important.
Don't crowd your questions, and use an easy-to-
read font. Leave some space between questions.
 3. Use bold font and underlining for titles/headings.
 4. Use examples and sample questions for clarity,
but be careful not to introduce bias.
 5. Include an open-ended question at the end of the
questionnaire asking for any additional information
and/or feedback. This question sometimes provides
some interesting information!

22. Constructing the questions:
 Create questions that use indicators which
will specifically address your hypotheses.
 Use existing scales or measures whenever
possible.
 You can also modify a scale or index to meet
the needs of your particular study if an
existing scale is not quite appropriate.
 The advantage in using an existing scale is
that its reliability and validity are already
known.

23. Types of questions to use:
 a. Simple questions that use a concrete
indicator (i.e. What is your age in years?___).
 b. Likert style questions are used for measuring
simple attitudes, beliefs, emotions, or behaviours
(i.e. When you think back to your high school
years, you feel: 1. Very unhappy 2. Somewhat
unhappy 3. Don't feel anything 4. Happy 5. Very
happy)
 c. Matrix questions are useful when you have a
number of related items. Matrix questions are
composed of a series of questions that have
identical response categories which are presented
in table format with the response category labels at
the top of the chart.

24. Types (cont.)
 c. Guttman scales: These are useful for measuring
the strength or intensity of an attitude.
 d. Rank-ordering: Rank ordering is a method that
works well with a small number of objects, such as
10. For example, instead of rating how interesting
each of a set of subject areas are, you could ask
the respondent to simply sort them in order of most
to least interesting.
 e. Paired comparisons: In this method, you
present items two at a time, and ask respondent to
pick which one has more of some attribute. For
example, you can present behavioural problems
and ask which one is more serious.

25. Closed or open-ended question
responses
 A closed response set refers to those
questions which have fixed categories for
answers (uses quantitative analysis)
 Open-ended questions allows for free
responses (qualitative analysis)

26. Filter questions and
contingency questions
 Filter questions are useful for sorting
respondents.
 Contingency questions allow respondents to
skip portions of the survey that may not be
relevant to them.

27. Principles of Question-Writing
 1. Keep questions as short and concise as possible.
 2. Choose your wording carefully.
 3. Try not to ask questions beyond a respondent's
capabilities.
 4. Avoid emotional language and the use of "loaded"
words.
 5. Watch out for prestige bias in your questions.
 6. Don't use leading questions like "You don't
smoke, do you?"
 7. Avoid ambiguity and vagueness.

28. Principles (cont.)
 8. Don't use double-barrelled questions.
 9. Avoid false premises or assumptions.
 10. If possible, don't ask respondent about future
intentions. The link between intentions and future
behaviour is tenuous.
 11. Try not to use negatives and double negatives in
your questions.
 12. Watch for explanatory statements that may bias
the answer to the question. Watch out also for
questions that might influence the answers to
subsequent questions.

29. Other Issues
 Aiding recall
 Provide special instructions or extra time
 Ask month by month, week by week, etc.
 Threatening questions
 People under-report illness, disability, deviance,
illegal activity, income/wealth
 Create “enhanced questions” or embed activity in
list of more serious activities

30. Issues (cont.)
 Social desirability bias
 People tend to over-report socially desirable
behaviours (being cultured, voting, giving to
charity, being good spouses or parents, etc.)
 Try to minimize importance of these activities or
present alternatives in questions
 Knowledge questions
 Make sure questions are appropriate level
 Phrase questions so respondents feel
comfortable not knowing the answer