This document discusses the concept of reliability in scale development and measurement. It defines reliability as how similar scores would be if the same person completed the scale twice, representing the precision of measurement. Greater reliability occurs when there is less random error and more consistency among item responses. Common methods for estimating reliability include internal consistency (such as Cronbach's alpha), parallel measurements using correlations between two administrations, and split-half reliability which divides a scale in half. Higher inter-item correlations and more items result in higher reliability estimates.

1. Reliability of Scale

2. Conceptual Definition of
Reliability
• Tells you how similar the scores would be if
the same person completed the scale twice
– Replicability
• Represents the precision of your scale
• Does not tell you if you are measuring the
right thing, but does tell you how well you
are measuring it
• Has a mathematical definition

3. Reliability in Terms of True and
Observed Scores
• You have greater reliability when you have
less random error in your measurements
error
Random
Scores
True
in
y
Variabilit
Scores
True
in
y
Variabilit
Scores
Observed
in
y
Variabilit
Scores
True
in
y
Variabilit
y
Reliabilit




4. Reliability in Terms of Variances
and Covariances
• You have greater reliability when the
communal variability (covariances) is large
relative to the noncommunal variability
(variances)




s
Covariance
Variances
1
y
Reliabilit

5. Reliability as Internal
Consistency
• Internal consistency refers to the relations
among the items in your scale
• The reliability of a scale is usually
estimated by its internal consistency
• Chronbach’s alpha is the most common
method of determining internal consistency

6. Chronbach’s alpha
• Primarily determined by the mean inter-
item correlation
– Higher correlations = higher alpha
• Also influenced by the number of items
– More items = higher alpha
 r
k
r
k
1
1 




7. Reliability and Power
• Analyses will be more powerful when they
use variables with greater reliability
– Lower reliability = more random error
– Random error can’t relate to anything
• Largest correlation you can find with a scale
= square root of its reliability
    y
x
xy
xy r
r 

true
observed 

8. Estimating Reliability Using
Parallel Measurements
• You can also estimate reliability as the
correlation of two parallel measurements
– Random variability of the two measurements
shouldn’t relate
– Correlation between two measures of the same
person should just represent variability related
to the latent factor

9. Estimating Reliability Using
Parallel Measurements
• Even though both internal consistency and
parallel measurements can be used as
estimates of reliability, they are slightly
different
– “Random error” in internal consistency
reliability represents measurement error
– “Random error” in parallel measurements
reliability represents measurement error and
change in the construct over time

10. Split-Half Reliability
• One special case of parallel measurements
is when you divide your scale into two sets
of items and treat the scores on each half as
a parallel measurement
• People commonly use odd/even split
– First half/second half not appropriate because
scores in the second half might be affected by
fatigue where scores in the first half will not

11. Split-Half Reliability
• Must apply a correction because each of
your parallel measurements has half the
number of items found in the full scale
• To do this:
1. Estimate the mean inter-item correlation using
the formula
 split
split
split
split
1 r
k
k
r
r




12. Split-Half Reliability
2. Calculate reliability using the formula
• Chronbach’s alpha is actually the average
of all possible split-halves
 r
k
r
k
1
1
y
reliabilit
full
full


