Effect Size
Upcoming SlideShare
Loading in...5

Effect Size



Presentation introducing a better hypothesis testing methodology: Effect Size.

Presentation introducing a better hypothesis testing methodology: Effect Size.



Total Views
Views on SlideShare
Embed Views



5 Embeds 162

http://www.scoop.it 119
http://www.slideshare.net 39
http://webcache.googleusercontent.com 2
https://blackboard.bsu.edu 1
http://www.slashdocs.com 1


Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Effect Size Effect Size Presentation Transcript

  • Beyond p value: Effect Size April 4, 2008 Guy Lion
  • P value vs Effect Size
    • By increasing sample size, you can show there is a statistically significant* difference between two Means. The Effect Size evaluates how material is that difference.
    P value = probability sample Means are the same . (1 – P) or C.L. = probability sample Means are different . Effect Size = how different sample Means are. *Statistically significant does not imply “significant.” [Webster: … of consequence.]
  • Effect Size in Plain English Large Effect Size is visible without looking at a large sample.
    • Other Effect Size examples:
    • The Kaplan course raises SAT math scores by 60 points;
    • This WOW initiative increases # Solutions by 30 units.
    • When analyzing Effect Size we deal with standardized units…
    With sea lions gender has a large Effect Size. With pugs gender has a small Effect Size
  • Effect Size Measures Cohen’s d = (Mean Pilot – Mean Control )/Pooled Stand. Deviation Cohen’s d is similar to the unpaired t test t value. It relies on Standard Deviations instead of Standard Errors. Hedges’ g is a more accurate version of Cohen’s d Hedges’ Cohen’s d Adjustment for small sample
  • Pilots vs Controls In this example, there is a 1.9 Standard deviation difference between Pilots and Controls.
  • Effect Size Info
  • An Example
  • ES Confidence Interval The Effect Size standard deviation formula allows to build Confidence Intervals around Effect Size values.
  • 1 st Nonparametric test: Gamma Index Recalculated Gamma Index to make it the same sign as Cohen's d and Hedges' g.
  • 2 nd Nonparametric test: Cliff Delta Cliff Delta ranges between +1 when all values of one group are higher than the values of the other group and – 1 when reverse is true. Two overlapping distributions would have a Cliff Delta of 0.
  • Cliff Delta efficiently… [(1 x 0) – (1 x 8)]/10 = -0.8
  • Cliff Delta vs Cohen’s d If distributions are Normal, Cliff Delta = Percent of Nonoverlap within a Cohen's d framework.
  • Effect Size and Sample Size requirement The above formula results from the algebraic transformation of: t stat or Z value = Difference in Means/Group Standard Error.
  • Testing Sample Size
  • Conclusion
    • P value does not tell us how different two samples are.
    • Effect Size and its Confidence Intervals give much information on how different two samples are.
    • Effect Size in units allows us to derive a required sample size to meet a p value  threshold.