Your SlideShare is downloading. ×

Kappa

361

Published on

Brief account of the kappa statistic for measuring agreement

Brief account of the kappa statistic for measuring agreement

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
361
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. KappaIs a form of correlation for measuring agreement on twoor more diagnostic categories by two or more clinicians or methods.Why not use % agreement?Because just by chance there could be lots of agreement.Kappa can be defined as the proportion of agreements after chanceagreement is removed. Kappa of 0 occurs when agreement is no better than chance.
  • 2. Kappa of 1 indicates perfect agreement.Negative Kappa means that there is lessagreement than you’d expect by chance (veryrare) Categories may be ordinal or nominal
  • 3. How is it calculated?Patient ID Psychiatrist Psychologist1 1 22 2 23 2 24 3 35 3 36 3 37 3 38 3 49 3 410 4 411 4 412 4 3
  • 4. Category 1 2 3 4 1 2 1 11 3 1111 1 4 11 11
  • 5. Steps1. Add agreements = 2 + 4 + 2 = 82. Multiply number of times each judge used a category: (1x0) + (2x3) + (6x5) + (3x4)3. Add them up = 484. Apply formula
  • 6. Kappa = (N x agreements) – N as in 3 N2 – N as in 3Which = (12 x 8) – 48 = 96 – 48 = 48 = 0.50 144 – 48 96 96
  • 7. How large should Kappa be?Landis & Koch (1977) suggested0.0 – 0.20 = no or slight agreement0.21 – 0.40 = fair0.41 – 0.60 = moderate0.61 – 0.80 = good> 0.80 = very good
  • 8. Weighted KappaIn ordinary Kappa, all disagreements aretreated equally. Weighted Kappa takesmagnitude of discrepancy into account (oftenmost useful); is often higher than unweightedKappa.
  • 9. N.B. Be careful with Kappa if the prevalence ofone of the categories in very low (< 10%); thiswill underestimate level of agreement.Example:If 2 judges are very accurate (95%) a Kappa of0.61 with a prevalence of 10% will drop to •0.45 if prevalence is 5% •0.14 if prevalence is 1%.

×