Upcoming SlideShare
×

# Kappa

799 views
624 views

Published on

Brief account of the kappa statistic for measuring agreement

0 Likes
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

• Be the first to like this

Views
Total views
799
On SlideShare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
14
0
Likes
0
Embeds 0
No embeds

No notes for slide

### Kappa

1. 1. KappaIs a form of correlation for measuring agreement on twoor more diagnostic categories by two or more clinicians or methods.Why not use % agreement?Because just by chance there could be lots of agreement.Kappa can be defined as the proportion of agreements after chanceagreement is removed. Kappa of 0 occurs when agreement is no better than chance.
2. 2. Kappa of 1 indicates perfect agreement.Negative Kappa means that there is lessagreement than you’d expect by chance (veryrare) Categories may be ordinal or nominal
3. 3. How is it calculated?Patient ID Psychiatrist Psychologist1 1 22 2 23 2 24 3 35 3 36 3 37 3 38 3 49 3 410 4 411 4 412 4 3
4. 4. Category 1 2 3 4 1 2 1 11 3 1111 1 4 11 11
5. 5. Steps1. Add agreements = 2 + 4 + 2 = 82. Multiply number of times each judge used a category: (1x0) + (2x3) + (6x5) + (3x4)3. Add them up = 484. Apply formula
6. 6. Kappa = (N x agreements) – N as in 3 N2 – N as in 3Which = (12 x 8) – 48 = 96 – 48 = 48 = 0.50 144 – 48 96 96
7. 7. How large should Kappa be?Landis & Koch (1977) suggested0.0 – 0.20 = no or slight agreement0.21 – 0.40 = fair0.41 – 0.60 = moderate0.61 – 0.80 = good> 0.80 = very good
8. 8. Weighted KappaIn ordinary Kappa, all disagreements aretreated equally. Weighted Kappa takesmagnitude of discrepancy into account (oftenmost useful); is often higher than unweightedKappa.
9. 9. N.B. Be careful with Kappa if the prevalence ofone of the categories in very low (< 10%); thiswill underestimate level of agreement.Example:If 2 judges are very accurate (95%) a Kappa of0.61 with a prevalence of 10% will drop to •0.45 if prevalence is 5% •0.14 if prevalence is 1%.