Feb. 6, 2021•0 likes•26 views

Download to read offline

Report

Education

Concept of reliability in scale development

Khalid MahmoodFollow

- 2. Conceptual Definition of Reliability • Tells you how similar the scores would be if the same person completed the scale twice – Replicability • Represents the precision of your scale • Does not tell you if you are measuring the right thing, but does tell you how well you are measuring it • Has a mathematical definition
- 3. Reliability in Terms of True and Observed Scores • You have greater reliability when you have less random error in your measurements error Random Scores True in y Variabilit Scores True in y Variabilit Scores Observed in y Variabilit Scores True in y Variabilit y Reliabilit
- 4. Reliability in Terms of Variances and Covariances • You have greater reliability when the communal variability (covariances) is large relative to the noncommunal variability (variances) s Covariance Variances 1 y Reliabilit
- 5. Reliability as Internal Consistency • Internal consistency refers to the relations among the items in your scale • The reliability of a scale is usually estimated by its internal consistency • Chronbach’s alpha is the most common method of determining internal consistency
- 6. Chronbach’s alpha • Primarily determined by the mean inter- item correlation – Higher correlations = higher alpha • Also influenced by the number of items – More items = higher alpha r k r k 1 1
- 7. Reliability and Power • Analyses will be more powerful when they use variables with greater reliability – Lower reliability = more random error – Random error can’t relate to anything • Largest correlation you can find with a scale = square root of its reliability y x xy xy r r true observed
- 8. Estimating Reliability Using Parallel Measurements • You can also estimate reliability as the correlation of two parallel measurements – Random variability of the two measurements shouldn’t relate – Correlation between two measures of the same person should just represent variability related to the latent factor
- 9. Estimating Reliability Using Parallel Measurements • Even though both internal consistency and parallel measurements can be used as estimates of reliability, they are slightly different – “Random error” in internal consistency reliability represents measurement error – “Random error” in parallel measurements reliability represents measurement error and change in the construct over time
- 10. Split-Half Reliability • One special case of parallel measurements is when you divide your scale into two sets of items and treat the scores on each half as a parallel measurement • People commonly use odd/even split – First half/second half not appropriate because scores in the second half might be affected by fatigue where scores in the first half will not
- 11. Split-Half Reliability • Must apply a correction because each of your parallel measurements has half the number of items found in the full scale • To do this: 1. Estimate the mean inter-item correlation using the formula split split split split 1 r k k r r
- 12. Split-Half Reliability 2. Calculate reliability using the formula • Chronbach’s alpha is actually the average of all possible split-halves r k r k 1 1 y reliabilit full full