2. Roadmap
• Research study announcement
• Exam 1 update
• Reflection Assignment #1
• Quick review & Ch. 5 wrap-up
• Begin Chapter 6: Research Validity
3. Reflection Assignment #1
• Due Tuesday (next class)
• Hard copy due at beginning of class
– STAPLE YOUR BUSINESS. I’m so serious.
• Cover page: Assignment document
(rubric for grading)
4. Quick Review
Random Nonrandom
Sampling Sampling
Simple Random Convenience
Stratified Random Quota
Cluster Random Purposive
Systematic Sampling Snowball
6. How do we determine sample size?
• If population <100, measure them all
– Special term for this?
• In general, get as big a sample as
possible
• Sample size calculator: G*Power
• Depends on lots of factors
8. MORE Validity—yay!
• Remember: validity has to do with
drawing accurate inferences
• So far: validity in the context of
measuring variables
• Now: validity in the context of setting
up studies
9. Research Validity
• Refers to the truthfulness of inferences
made from a research study
• Think of validity on a continuum rather
than all-or-none
• 4 major types of research validity
• Must prioritize
10. Types of Research Validity
• Statistical Conclusion Validity
• Construct Validity
• Internal Validity
• External Validity
11. Statistical Conclusion Validity
• Validity with which we can infer that
the IV and DV covary
– Covary = vary together
• The validity of the inferences we make
from our analyses
12. Stats Refresher: ―Statistically
Significant‖
• p <.05
• What does it mean?
– The observed relationship is probably NOT
due to chance alone
• Sometimes our stats are just wrong
• Chance
• Too little power (sample size)
• Type 1 / Type 2 error
13. Construct Validity
• Refresh: construct = ?
• Validity of the inferences we make
about constructs based on how we
measure them
• What does this sound like?
– The chapter 5 validity topics!
17. Group Activity: 5-7 minutes
• You’re applying for a grant to fund a
research project
• Identify research idea
– IV - operationalize
– DV - operationalize
– Hypothesis
• Explain how you will gather evidence of
construct validity in your measurements
18. Threats to Construct Validity
• Factors that impact how well our
operationalizations actually represent
constructs
• Pg 171, Table 6.2 – long list of threats
• We will focus on two major ones:
– Participant reactivity to the experimental situation
– Experimenter effects
19. Reactivity to the Experimental
Situation
(From the participant angle)
• Participants’ motives and perceptions
• Demand characteristics
• Positive self-presentation
20. Instruction set #1
We want to see how well you are able
to learn the following sets of letters.
Letters will appear in groups of 3 to
7, and each letter will appear on the
screen for 1 second. Following the
presentation of the letters, …
21. Instruction Set #2
In the following task, you will be
presented with groups of letters ranging
from 3 to 7 letters. Each letter will
appear on the screen for one second.
Your task is to…
22. Experimenter effects
• Researcher actions and characteristics
that influence the responses made by the
research participant
• Expectancies
– Clever Hans
• Attributes
– Biosocial
– Psychosocial
– Situational factors
24. Internal Validity
• The extent to which we can
accurately infer that the
independent and dependent
variables are causally related
Independent Observed Effect
Variable (DV)
25. Causally Related
Independent Observed Effect
Variable (DV)
Cause and effect are Cause must No other
related (covary) precede effect explanation is
plausible
27. Threats to Internal Validity
• History
• Maturation
• Instrumentation
• Testing
• Regression Artifact
• Attrition
• Selection
• Additive and Interactive Effects
28. History
• Any event occurring after the study
begins that could produce the observed
outcome
• Differential history: only one group
experiences history event
29. Maturation
• Changes in biological and psychological
conditions that occur with the passage of
time
– Factors within the individual
• Example: Head Start program and
achievement over a school year
30. Instrumentation
• Changes in the
assessment/measurement of the
dependent variable
• Example: multiple observers and
interviewers
31. Testing
• Changes in a person’s score on the
second administration of a test a result
of previously having taken the test
• Example: pre-test and post-test on
memory task
32. Regression Artifact
• A.k.a. regression toward the mean
• The tendency for extreme scores to
become less extreme on a second
assessment
33. Attrition
• Participant drop-out
– Don’t show up for appointment
– Decide to discontinue study
• Differential attrition is especially
problematic
34. Selection
• The choice of participants for the various
treatment groups based on different
criteria
– NOT random assignment
35. Additive & Interactive Effects
• The combined effect of several
threats to internal validity
