Slides from a guest lecture for Psychological Science: Applied

1. EPSY 1281| Lecture 5
Department of Educational Psychology

2. SURVEY RESEARCH

3. SURVEYS
• Surveys are used to describe opinions, attitudes, and
preferences
– specific or global
• Surveys are used to make predictions about behavior
– correlations – more in a moment
• A sample of people completes a questionnaire(s);
responses from the sample are used to describe the
population (e.g. lab section is sample from class population)
• Surveys involve using a predetermined set of questions

4. GETTING SURVEY DATA RIGHT
1936
PRESIDENTIAL
ELECTION

5. The mother of all botched political polls was a 1936 Literary
Digest straw poll survey said GOP challenger Alf
Landon would win in a landslide over Franklin Delano
Roosevelt, with 57 percent of the vote.
POLL FORECAST

6. SPOILER ALERT !
• The Literary Digest used
national straw polls in 1920,
1924, 1928 and 1932, and it
guessed the winner of each
presidential election.
• President Roosevelt won the
1936 election easily, with 63
percent of the vote.
• Literary Digest was out of
business the following year

7. WHAT WENT WRONG ?

8. 1936 POLITICAL SURVEY (POLL)
• The Digest polled about 2 million people, most of who were
magazine readers, car owners or telephone customers—
and had money during the Depression.

10. DATA COLLECTION MODES
• Mail Surveys
– Convenient BUT response bias (typically 30% only)
– How can we increase response rate?
• Personal Interviews
– Costly BUT more control
– Interviewer bias in selective recording or rewording
• Telephone Interviews
– Fast response BUT time, interviewer bias (TRUMP), memory, environment
• Internet Surveys
– Efficient and low-cost BUT response and selection bias, environment

11. QUESTIONNAIRES
• Most survey research relies on questionnaires
• Questionnaires are used to measure different types of variables:
– demographic variables
– preferences and attitudes
– knowledge
• most often these are measured with self-report scales
• participants respond on rating scales (e.g., Likert scales)

12. QUESTION TYPES

13. LIKERT SCALE
• In the 1930s, researcher Rensis Likert created a new approach for
measuring people’s attitudes.
• It involves presenting people with several statements—including both
favorable and unfavorable statements—about some person, group, or
idea.
• Respondents then express their agreement or disagreement on a 5-point
scale: Strongly Agree, Agree, Neither Agree nor
Disagree, Disagree, Strongly Disagree.
• Numbers are assigned to each response and summed across items.

14. QUESTION WRITING
• Number of questions on a topic?
• Types of questions?
– Open-ended vs. closed-ended
– No opinion option?
• Question sequence
• Language level - wording

15. QUESTION WRITING - BRUSO

16. MAKING QUESTIONNAIRES
• Decide what information should be sought and the type of questionnaire, write a
draft, pre-test, specify the procedures for its use
• Clear wording, simple and familiar vocabulary
• Clear and specific questions
• Do not involve leading, loaded, or double-barreled questions
• Short sentences (20 or fewer words)
• Demographics at the end
• Check for readability

17. SCALES
• Cumulative scales (a single number that is very
informative, Y/N response)
• Summated scales (Likert scales)
• Semantic differential scales (bipolar scales)

18. • Nominal – Provides labels, allows classification (equal-not
equal)
• Ordinal – Additionally provides the ability to order
observations (greater than/less than)
• Interval – Additionally provides distance information
(making addition & subtraction meaningful as well as
multiplication and division)
• Ratio – Provides a true zero (enabling meaningful ratios of
values)
MEASUREMENT LEVELS

19. EXPERIMENTS

20. WHY DO EXPERIMENTS ?
• Test hypotheses derived from theories
• Are demos experiments?
• Test the effectiveness of a treatment or program
• Experiments differ from other research designs
(e.g., observational, survey research) because they
allow researchers to examine the causes of
behavior.
– Researchers seek to meet the third goal of
psychological research: explanation.

21. What is an Experiment?
• An experiment must include:
– an independent variable (IV) and
– dependent variable(s) (DVs).
• Typically, experiments are done under
carefully controlled conditions (in a lab)

22. INDEPENDENT VARIABLES
• An independent variable
– is manipulated (controlled) by the
experimenter
– has at least two different conditions
(e.g., “treatment” and “control”
conditions).

23. DEPENDENT VARIABLES
• Dependent variables are:
– measured by the experimenter
– are used to determine the effect of the independent
variable.
– In most experiments, researchers measure several
dependent variables to learn the effect of the independent
variable.
• Dependent measures are used to measure the dependent
variable

24. CAUSAL INFERENCES
Three conditions must be met before we can make a
causal inference:
 Covariation: We must observe a relationship between
the independent and dependent variables
 Time-order relationship: The presumed cause
precedes the effect
 Elimination of plausible alternative causes: Using
control techniques, we rule out other possible causes
for the outcome

25. INTERNAL VALIDITY
• Internal Validity
– An experiment has internal validity when
we are able to state that the independent
variable caused differences between
conditions on the dependent variable
(i.e., a causal inference)
– For an experiment to be internally valid,
we must be able to rule out alternative
explanations for the study’s findings

26. CONFOUNDING
• Confoundings: When the independent variable of
interest and a different, extraneous variable are
allowed to covary (go together), a confounding is
present.
– Confoundings represent alternative
explanations for a study’s findings
– An experiment that has a confounding is not
internally valid
– An experiment that is free of confoundings has
internal validity

27. EXTERNAL VALIDITY
• External validity refers to the extent to which findings
from an experiment can be generalized to individuals,
settings, and conditions beyond the scope of a specific
experiment
• Questions of External Validity
– Would the same findings occur in different settings?
– Would the same findings occur with different
conditions?
– Would the same findings hold true for different
participants?

28. SAMPLING

29. • The goal is to use the sample to represent
some larger population
SAMPLING

30. • When we use sample data to represent the
population, we generalize the findings from the
sample to the population
• External validity refers to the extent to which the
results of a research study can be generalized to
different populations, settings, and condition
GENERALIZABILITY

31. • We can only generalize findings when the sample
is representative of the population
• Lots of psychology studies use college student
samples
• How do we obtain representative samples?
GENERALIZABILITY ISSUES

32. SAMPLING DETAILS
• Define the population of interest
• Obtain a sampling frame, a list of all members of the
population
• Select a sample, the subjects you will actually survey
• Get data from each element, which is an individual from
your population

33. SAMPLING PROCEDURES
• probability sample - all members do have an
equal chance of being sampled (simple random
samples)
– stratified random samples – populations are
divided into strata and simple random samples
are drawn from each strata

34. EXAMPLES
• Stratified Sampling: Some examples of strata commonly
used are States, Age and Sex. Other strata may be religion,
academic ability or marital status.
• Note: Stratification will always achieve greater precision
provided that the strata have been chosen so that members
of the same stratum are as similar as possible in respect of
the characteristic of interest. The bigger the differences
between the strata, the greater the gain in precision.

35. Α
Γ
Β
Ε
Δ
Θ
Η
Ι
Κ
Λ
Μ
Ν
Ξ
Ο
Π Ρ
Σ
Τ
Υ Χ Ψ
Ω
Ι Δ
ΡΧ
Ζ
SIMPLE RANDOM SAMPLE
Ξ
Ζ

36. Α ΓΒ ΕΔ
ΘΗ ΙΚΛ
Μ Ν Ξ ΟΠ
Τ Υ Χ ΨΩ
STRATIFIED SAMPLE
50%
25%
25%
Β Ε 25%
Θ Κ Μ
Π
50%
Χ Ω 25%

37. RESOURCES
• Price
– Chapter 6 (6.1)
– Chapter 9