Research method

1. DESCRIPTIVE METHODS II
Topic 4: Survey Research

2. 2
Survey Research
• Survey research
• Describe thoughts, opinions, feelings
• Measure naturally occurring variables
• Allow predictions based on correlations
• Questionnaires
• Predetermined set of questions
• Sample represents a population
• Examine survey procedures and analyses for sources of bias

3. 3
Sampling in Survey Research
• Use sample to represent the larger population
• “Representative”: similar to
• Requires careful selection of a sample
• Goal: Generalize survey findings from representative
sample to the population

4. 4
Basic Terms of Sampling
• Population
• Set of all cases of interest (Business students, UTAR)
• Sampling Frame
• List of the members of a population (get the list of enrolled
students)
• Sample
• Subset of population drawn from sampling frame (300 students)
• Element
• Each member of the population

5. 5
Biased Samples
• A biased sample
• Characteristics of the sample differ systematically from those of
the population.
• Sample over-represents or under-represents segment(s) of a
population
• Eg. (300 students) 70% female and 30% male

6. 6
Biased Samples
•Two sources
• Selection bias
• Researcher’s procedures for selecting sample cause bias
• Response-rate bias
• Individuals selected for the sample do not complete the survey

7. 7
Approaches to Sampling
•“Sampling”
• Procedures used to obtain a sample
•Two basic approaches
• Probability sampling
• Nonprobability sampling Population
Sample

8. 8
Approaches to Sampling, continued

9. 9
Approaches to Sampling, continued

10. 10
Approaches to Sampling, continued
• Systematic (Random Sampling)

11. 11
Approaches to Sampling, continued
•Probability sampling
• All members of population have a specified chance of
being selected for the survey
• Simple random sample
• Random selection, random-digit dialing
• Stratified random sample
• Divide population into strata and sample proportionally
• Improves representativeness of sample

12. 12
Approaches to Sampling, continued
•Nonprobability sampling
• No guarantee each member of population has an equal
chance of being in the sample
• “Convenience sampling”
• Individuals are available and willing to respond to the survey
• Example: magazine surveys, call-in radio surveys
• Sample likely not representative of population

13. 13
Survey Methods
•Four methods for obtaining survey data
• Mail surveys
• Personal interviews
• Telephone interviews
• Internet surveys
•Each method has advantages and disadvantages.
•Choose method based on research question.

14. 14
Survey Methods, continued
•Ways to increase response rate
• Responding requires minimal effort
• Topic of survey is interesting to respondents
• Respondents identify with organization or sponsor of
survey

15. 15
Survey Research Designs
•“Research design”
• A plan for conducting a research project
• Choose method best suited for answering a particular
question
•Three types of survey research designs
• Cross-sectional design
• Successive independent samples design
• Longitudinal design

16. 16
Survey Research Designs, continued
•Cross-sectional survey design
• Select sample from one or more populations at one time
• Survey responses are used to
• Describe population (descriptive statistics)
• Make predictions for the population (correlations)
at that one point in time
• Compare populations
• Cannot assess change over time

17. 17
Survey Research Designs, continued
•Successive independent samples design
• A series of cross-sectional surveys over time
• A different sample from the population completes the
survey each time.
• Each sample is selected from the same population.
• Responses from each sample are used to describe
changes in the population over time.
• Problem: noncomparable successive samples

18. 18
Survey Research Designs, continued
•Longitudinal survey design
• Same sample of individuals completes the survey at
different points in time
• Assess how individuals change over time
• Responses from the sample are generalized to describe
changes over time in the population.
• Problems: attrition (loss of participants) and reactivity

19. 19
Measures in Survey Research
• Questionnaires
• Most frequently used to collect survey data
• Measure different types of variables
• Demographic variables using checklists
• Preferences, opinions, and attitudes
• Self-report scales
• Rating scales (assume interval level of measurement)
• All measures must be reliable and valid.

20. 20
Reliability and Validity
• Reliability refers to consistency of measurement.
• Test-retest reliability
• Administer measure two times to same sample
• High correlation between the two sets of scores indicates good reliability (r
> .80)

21. 21
Reliability and Validity, continued
• How to improve reliability?
• More items
• Greater variability among individuals on the factor being
measured
• Testing situation free of distractions
• Clear instructions
• A measure can be reliable but not valid.

22. 22
Reliability and Validity, continued
•Validity refers to the truthfulness of a measure.
• Assesses what it is intended to measure
• Construct validity
• Instrument measures the theoretical construct it was designed
to measure.

23. 23

24. 24
Reliability and Validity, continued
• Establishing construct validity:
• Convergent validity
• Extent to which two measures of the same construct are correlated (go
together)
• Discriminant validity
• Extent to which two measures of different constructs are not correlated (do
not go together)

25. 25
Thinking CriticallyAbout Survey Research
• Correspondence Between Reported and Actual Behavior
• Survey responses may not be truthful.
• Reactivity
• Social desirability
• Accept people’s responses as truthful unless there’s reason to
suspect otherwise.
• Use a multimethod approach to answering research questions.

26. 26
Thinking CriticallyAbout Survey Research, continued
• Correlation and causality
• “Correlation does not imply causality”
• Three possible causal inferences for any correlation
• A causes B
• B causes A
• Variable C causes both A and B

27. 27
Thinking CriticallyAbout Survey Research, continued
• Path analysis
• Statistical procedure to tease apart complex correlational
relationships among variables
• Mediators
• Variables used to explain a correlation between two variables
• Moderators
• Variables that affect direction or strength of correlation between two
variables

28. 28
Thinking CriticallyAbout Survey Research, continued
Diagram of direct and indirect relationships
Creative Process
Engagement
path a path b
Shyness Creativity
path c’ (direct)
Creative process engagement mediates the relationship between shyness
and creativity among undergraduate students. OR
Shyness has an indirect effect on creativity via creative process engagement.
path c(total effect: without mediator)

29. 28
Thinking CriticallyAbout Survey Research, continued
path b

30. 29
Thinking CriticallyAbout Survey Research, continued
• Path analysis example
• A moderator variable may affect the direction and strength of
these relationships.
• Possible moderators:
• Sex of the child
• Population density
(e.g., rural, urban)
• Personality features of children
(e.g., resilience)

31. 30
Thinking CriticallyAbout Survey Research, continued
•Path analysis
• Helps us to understand relationships among variables
• But these relationships are still correlational
• Cannot make definitive causal statements
• Other untested variables may be important