The document discusses various types of survey research, including case studies, experiments, and sampling methods. It outlines the goals of surveys, advantages of different data collection methods, and challenges related to representativeness and randomness in sampling. Additionally, it provides guidelines on questionnaire design, avoiding common pitfalls, and ensuring clarity and validity in survey questions.

1. Types of research studies
1
CASE STUDIES
EXPERIMENTS
DESIGN STUDIES
SURVEYS
Survey Research (van Aalst)

2. Goals of survey research
 Measure or estimate a population on one or
more variables
 How many households have a computer < 3
years old, for different strata of income and
education levels?
 How many classrooms have more than one
computer, for schools in areas with different
socio-economic status?
 What is the distribution of income from work,
for Hong Kong, Taipei, and Singapore?
Survey Research (van Aalst) 2
4.Surveys

3. Example
2006 New Zealand Census
Survey Research (van Aalst) 3
4.Surveys

4. TIMSS 2011
Survey Research (van Aalst) 4
4.Surveys

5. Survey methods
 Generally uses descriptive statistics for
testing hypotheses or answering research
questions
 Can be done via questionnaires,
interviewing, and observation
 Questionnaires most common
 Interview and observation data need to be
quantified
 What are the advantages of each method?
Survey Research (van Aalst) 5
4.Surveys

6. Sampling
 Sometimes whole population is
measured
 All taxpayers in Hong Kong
 All persons entering Hong Kong by air
 Most often a representative and
randomly selected sample is used and
one makes inferences about the
population from that sample
Survey Research (van Aalst) 6
4.Surveys

7. Representative sample
 Are the following representative samples
of their populations?
 The men and women in this course as a
sample of all men and women taking M.
Sc. courses in Hong Kong?
 People in Hong Kong who have a
broadband connection to the Internet as a
sample of all people in Hong Kong?
Survey Research (van Aalst) 7
4.Surveys

8. Random sample
 Each member of the population has an
equal chance being selected for the
survey
 Maximizes the chance that patterns in the
sample match those in the population
 Statistically useful
 Mean value and variation in the population
can be estimated from knowing the mean
value and variation of the random sample
 When the sample is not random this kind of
estimation is less accurate
Survey Research (van Aalst) 8
4.Surveys

9. Problems with representativeness
and randomness
 Low response rates, say 20%
 sub-populations may be systematically
excluded
 Systematic bias
 A random sample is no longer random or
representative when some students do not
turn up for a test
 Sampling % of total population may over-
represent major urban centers and under-
represent rural areas
Survey Research (van Aalst) 9
4.Surveys

10. Planning a questionnaire
 Can use a flow chart technique to plan
the sequencing of questions
 Be able to anticipate the type & range of
responses that their questions are likely
to elicit
Survey Research (van Aalst) 10
4.Surveys

11. A flow chart technique for
question planning
Survey Research (van Aalst) 11
4.Surveys

12. Operationalizing the
questionnaire
 Clear purpose
 Clear on what needs to be included or
covered
 Exhaustive coverage of elements
 Asks most appropriate kinds of question
 Elicits most appropriate kinds of data to
answer the research purposes
 Asks for empirical data
Survey Research (van Aalst) 12
4.Surveys

13. Avoiding pitfalls in question writing
 Avoid leading questions which suggests
that there is only one acceptable answer
 “Do you prefer abstract, academic-type
courses, or practical courses that have
some pay-off in your day-to-day teaching?”
Survey Research (van Aalst) 13
4.Surveys

14. Avoiding pitfalls in question writing
 Avoid complex questions
 “Would you prefer a short, non-award
bearing course (3, 4 or 5 sessions) with
part-day release (e.g. Wednesday
afternoons) and one evening per week
attendance with financial reimbursement
for travel, or a longer, non-award bearing
course (6, 7 or 8 sessions) with full-day
release, or the whole course designed on
part-day release without evening
attendance?”
Survey Research (van Aalst) 14
4.Surveys

15. Avoiding pitfalls in question writing
 Avoid irritating questions or instructions
 “Have you ever attended an in-service
course of any kind during your entire
career?”
 Avoid questions that use negatives and
double negatives
 “How strongly do you feel that no teacher
should enroll on the in-service, award-
bearing course who has not completed at
least two years full-time teaching?”
Survey Research (van Aalst) 15
4.Surveys

16. Avoiding pitfalls in question writing
 Avoid ambiguity in words
 “Does your child regularly do homework?”
(Once a day, once a year, once a term, once a
week?)
 How many computers do you have in
school?
(Present but broken; out of school being
repaired, staffs’ or students’?)
Survey Research (van Aalst) 16
4.Surveys

17. Avoiding pitfalls in question writing
 Avoid any overlap of categories.
 “How old are you?
 15-20
 20-30
 30-40
 40-50”
Survey Research (van Aalst) 17
4.Surveys

18. Cover letters/sheets and follow-up
letters
 Provide title for project
 State purpose and expected benefits
 Mention sponsorship or permission for
the research
 Mention incentives for completion
 Provide assurances of confidentiality,
anonymity and non-traceability
 Thank respondents in advance for their
co-operation
Survey Research (van Aalst) 18
4.Surveys

19. Piloting the questionnaire
 Check clarity of questionnaire items,
instructions, and layout
 Gain feedback on validity of items
 Eliminate ambiguities
 Check time to complete
 Identify misunderstood or non-
completed items
 Check if it is too long or too short, too
easy or too difficult
Survey Research (van Aalst) 19
4.Surveys

20. Further reading
 Fowler, Jr., F. J. (2009). Survey research
methods, 4th edition. Thousand Oaks,
CA: SAGE.
Survey Research (van Aalst) 20