# Scale development -- Reliability

Professor at University of the Punjab
Feb. 6, 2021
1 of 12

### Scale development -- Reliability

• 2. Conceptual Definition of Reliability • Tells you how similar the scores would be if the same person completed the scale twice – Replicability • Represents the precision of your scale • Does not tell you if you are measuring the right thing, but does tell you how well you are measuring it • Has a mathematical definition
• 3. Reliability in Terms of True and Observed Scores • You have greater reliability when you have less random error in your measurements error Random Scores True in y Variabilit Scores True in y Variabilit Scores Observed in y Variabilit Scores True in y Variabilit y Reliabilit   
• 4. Reliability in Terms of Variances and Covariances • You have greater reliability when the communal variability (covariances) is large relative to the noncommunal variability (variances)     s Covariance Variances 1 y Reliabilit
• 5. Reliability as Internal Consistency • Internal consistency refers to the relations among the items in your scale • The reliability of a scale is usually estimated by its internal consistency • Chronbach’s alpha is the most common method of determining internal consistency
• 6. Chronbach’s alpha • Primarily determined by the mean inter- item correlation – Higher correlations = higher alpha • Also influenced by the number of items – More items = higher alpha  r k r k 1 1    
• 7. Reliability and Power • Analyses will be more powerful when they use variables with greater reliability – Lower reliability = more random error – Random error can’t relate to anything • Largest correlation you can find with a scale = square root of its reliability     y x xy xy r r   true observed 
• 8. Estimating Reliability Using Parallel Measurements • You can also estimate reliability as the correlation of two parallel measurements – Random variability of the two measurements shouldn’t relate – Correlation between two measures of the same person should just represent variability related to the latent factor
• 9. Estimating Reliability Using Parallel Measurements • Even though both internal consistency and parallel measurements can be used as estimates of reliability, they are slightly different – “Random error” in internal consistency reliability represents measurement error – “Random error” in parallel measurements reliability represents measurement error and change in the construct over time
• 10. Split-Half Reliability • One special case of parallel measurements is when you divide your scale into two sets of items and treat the scores on each half as a parallel measurement • People commonly use odd/even split – First half/second half not appropriate because scores in the second half might be affected by fatigue where scores in the first half will not
• 11. Split-Half Reliability • Must apply a correction because each of your parallel measurements has half the number of items found in the full scale • To do this: 1. Estimate the mean inter-item correlation using the formula  split split split split 1 r k k r r   
• 12. Split-Half Reliability 2. Calculate reliability using the formula • Chronbach’s alpha is actually the average of all possible split-halves  r k r k 1 1 y reliabilit full full   