lecture 6 from a college level research methods in psychology course taught in the spring 2012 semester by Brian J. Piper, Ph.D. (psy391@gmail.com) at Linfield College, includes categorical, ordinal, interval, and ratio levels

1. Statistics II: Measurement & Data
Analysis
Brian J. Piper

2. Goals
• Operationalization
• Psychometrics (Reliability & Validity)
• Scales of Measurement
• Hypothesis Testing
• Advanced Topics
– Power
– Effect Size

3. Operationalization
• What areas of the brain are important for
music appreciation?
• Important = Active
– EEG (electrical activity)
– PET (sugar use)
– fMRI (oxygen use)

4. Operationalization 1: Spatial Function
• Do men and women differ in their spatial
abilities?
• Spatial Function = mental rotation

5. Operationalization 2: Motor Function
• Do men and women differ in their fine-motor
abilities?
• Motor Function = rotary pursuit

6. Reliability
• Consistency of measurement
• Types
– Test-re-test reliability
Time 2
Time 1

7. Split-Half Reliability
• Consistency of measurement on two-halves of
test
• Foundations for repeated measurements
Odd
Even

8. Extension: Short-Form of Wisconsin
(Berg) Card Sorting Test (BCST)

9. Extension: Short-Form of Wisconsin
(Berg) Card Sorting Test (BCST)
r(205) = +0.77
Fox et al. (in review). J Biol Biomed Reports.

10. Validity
• Does a test measure what it claims to?
• face “faith” validity: does it seem valid based on
intuition (non-numerical)

11. Criterion Validity
• Does performance on new measure match
with older “gold standard” measure?
• Continuous Performance Tests Example
Reaction Time (PEBL)
Reaction Time (Conner’s)

12. Construct Validity
• Does a test measure the construct it claims
to?
• Convergent Validity: Does test A correlate
(converge) with test B?
• Discriminant Validity: Does test A measure
something different (discriminate) than test
C?

13. Measurement Scales (Self-Test)
Level Definition
Nominal
Ordinal
Interval
Ratio

14. Measurement Scales (Self-Test)
Level Definition
Nominal categorical, e.g. sex
Ordinal ranking, e.g. Olympic medal
Interval equal spacing, e.g. IQ, ACT, SAT
Ratio true zero, e.g. Reaction Time

15. Hypothesis Testing
• Null hypothesis (H0): A = B
• Alternative hypothesis (HA): A ≠ B

16. Alpha
• The cut-off used to decide between H0 and HA
• Probability that finding is not due to chance (p
value)
• .05: conventional
• .10: liberal (some medical environments)
• .01: conservative, large N

17. Alpha
P value Decision
obtained
.50 H0
.11 H0
.06 H0
.0500000001 H0
.0499999999 HA

18. Decision Making
Reality
HO is True HO is False
Fail to reject H0 Correct decision
Decision
Reject H0 Correct decision

19. Decision Making
Reality
HO is True HO is False
Fail to reject H0 Correct decision
Decision
Reject H0 Type I error Correct decision
Type I Error: rejecting H0 when it is true

20. Decision Making
Reality
HO is True HO is False
Fail to reject H0 Correct decision Type II error
Decision
Reject H0 Type I error Correct decision
Type I Error: rejecting H0 when it is true
Type II Error: fail to reject H0 incorrectly

21. Publication Bias
• H0 results often don’t get shared
• Reasons:
– Journal prestige
– Research ego
– Higher standard
• Solution: registry?
– Replication?

22. Solution 1: Effect Size Distribution
• A quantitative index of the magnitude of
group difference’s
• Calculated as (Mean1 – Mean2)/SD
# Studies

23. Solution 1: Effect Size Distribution
• A quantitative index of the magnitude of
group difference’s
• Calculated as (Mean1 – Mean2)/SD
# Studies
# Studies

24. Solution 2: Power Analysis
• Power: the probability that a real effect will be
detected
• Probability of Type II error: Beta
• Power = 1 - Beta
N Power
50 0.40
100 0.70
500 0.80
1000 0.85

25. Other Terminology
• Population: all members of identifiable group
• Sample: a subset of the population
• Confidence Interval: inferential statistic,
contains range of where population mean sits

26. Margin of Error
• Is accurate if sample is representative of
population.

27. Summary
• Operationalization
• Reliability & Validity (face, criterion, construct)
• Scales of Measurement (nominal, ordinal,
interval, ratio)
• Hypothesis Testing: Type I versus Type II error
• Advanced Topics
– Power
– Effect Size