Narrated presentation for self-study before looking at specific examples; from an intro course for Masters-lavel students.

1. Types of research studies
1
CASE STUDIES
EXPERIMENTS
DESIGN STUDIES
SURVEYS
Survey Research (van Aalst)

2. Goals of survey research
 Measure or estimate a population on one or
more variables
 How many households have a computer < 3
years old, for different strata of income and
education levels?
 How many classrooms have more than one
computer, for schools in areas with different
socio-economic status?
 What is the distribution of income from work,
for Hong Kong, Taipei, and Singapore?
Survey Research (van Aalst) 2
4.Surveys

3. Example
2006 New Zealand Census
Survey Research (van Aalst) 3
4.Surveys

4. TIMSS 2011
Survey Research (van Aalst) 4
4.Surveys

5. Survey methods
 Generally uses descriptive statistics for
testing hypotheses or answering research
questions
 Can be done via questionnaires,
interviewing, and observation
 Questionnaires most common
 Interview and observation data need to be
quantified
 What are the advantages of each method?
Survey Research (van Aalst) 5
4.Surveys

6. Sampling
 Sometimes whole population is
measured
 All taxpayers in Hong Kong
 All persons entering Hong Kong by air
 Most often a representative and
randomly selected sample is used and
one makes inferences about the
population from that sample
Survey Research (van Aalst) 6
4.Surveys

7. Representative sample
 Are the following representative samples
of their populations?
 The men and women in this course as a
sample of all men and women taking M.
Sc. courses in Hong Kong?
 People in Hong Kong who have a
broadband connection to the Internet as a
sample of all people in Hong Kong?
Survey Research (van Aalst) 7
4.Surveys

8. Random sample
 Each member of the population has an
equal chance being selected for the
survey
 Maximizes the chance that patterns in the
sample match those in the population
 Statistically useful
 Mean value and variation in the population
can be estimated from knowing the mean
value and variation of the random sample
 When the sample is not random this kind of
estimation is less accurate
Survey Research (van Aalst) 8
4.Surveys

9. Problems with representativeness
and randomness
 Low response rates, say 20%
 sub-populations may be systematically
excluded
 Systematic bias
 A random sample is no longer random or
representative when some students do not
turn up for a test
 Sampling % of total population may over-
represent major urban centers and under-
represent rural areas
Survey Research (van Aalst) 9
4.Surveys

10. Planning a questionnaire
 Can use a flow chart technique to plan
the sequencing of questions
 Be able to anticipate the type & range of
responses that their questions are likely
to elicit
Survey Research (van Aalst) 10
4.Surveys

11. A flow chart technique for
question planning
Survey Research (van Aalst) 11
4.Surveys

12. Operationalizing the
questionnaire
 Clear purpose
 Clear on what needs to be included or
covered
 Exhaustive coverage of elements
 Asks most appropriate kinds of question
 Elicits most appropriate kinds of data to
answer the research purposes
 Asks for empirical data
Survey Research (van Aalst) 12
4.Surveys

13. Avoiding pitfalls in question writing
 Avoid leading questions which suggests
that there is only one acceptable answer
 “Do you prefer abstract, academic-type
courses, or practical courses that have
some pay-off in your day-to-day teaching?”
Survey Research (van Aalst) 13
4.Surveys

14. Avoiding pitfalls in question writing
 Avoid complex questions
 “Would you prefer a short, non-award
bearing course (3, 4 or 5 sessions) with
part-day release (e.g. Wednesday
afternoons) and one evening per week
attendance with financial reimbursement
for travel, or a longer, non-award bearing
course (6, 7 or 8 sessions) with full-day
release, or the whole course designed on
part-day release without evening
attendance?”
Survey Research (van Aalst) 14
4.Surveys

15. Avoiding pitfalls in question writing
 Avoid irritating questions or instructions
 “Have you ever attended an in-service
course of any kind during your entire
career?”
 Avoid questions that use negatives and
double negatives
 “How strongly do you feel that no teacher
should enroll on the in-service, award-
bearing course who has not completed at
least two years full-time teaching?”
Survey Research (van Aalst) 15
4.Surveys

16. Avoiding pitfalls in question writing
 Avoid ambiguity in words
 “Does your child regularly do homework?”
(Once a day, once a year, once a term, once a
week?)
 How many computers do you have in
school?
(Present but broken; out of school being
repaired, staffs’ or students’?)
Survey Research (van Aalst) 16
4.Surveys

17. Avoiding pitfalls in question writing
 Avoid any overlap of categories.
 “How old are you?
 15-20
 20-30
 30-40
 40-50”
Survey Research (van Aalst) 17
4.Surveys

18. Cover letters/sheets and follow-up
letters
 Provide title for project
 State purpose and expected benefits
 Mention sponsorship or permission for
the research
 Mention incentives for completion
 Provide assurances of confidentiality,
anonymity and non-traceability
 Thank respondents in advance for their
co-operation
Survey Research (van Aalst) 18
4.Surveys

19. Piloting the questionnaire
 Check clarity of questionnaire items,
instructions, and layout
 Gain feedback on validity of items
 Eliminate ambiguities
 Check time to complete
 Identify misunderstood or non-
completed items
 Check if it is too long or too short, too
easy or too difficult
Survey Research (van Aalst) 19
4.Surveys

20. Further reading
 Fowler, Jr., F. J. (2009). Survey research
methods, 4th edition. Thousand Oaks,
CA: SAGE.
Survey Research (van Aalst) 20