Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

CORE: May the “Power” (Statistical) - Be with You!


Published on

About CORE:
The Culture of Research and Education (C.O.R.E.) webinar series is spearheaded by Dr. Bernice B. Rumala, CORE Chair & Program Director of the Ph.D. in Health Sciences program in collaboration with leaders and faculty across all academic programs.

This innovative and wide-ranging series is designed to provide continuing education, skills-building techniques, and tools for academic and professional development. These sessions will provide a unique chance to build your professional development toolkit through presentations, discussions, and workshops with Trident’s world-class faculty.

For further information about CORE or to present, you may contact Dr. Bernice B. Rumala at

Published in: Education
  • Be the first to comment

CORE: May the “Power” (Statistical) - Be with You!

  1. 1. May  the  “Power”  (Statistical) Be  with  You!   Dr.  Mickey  Shachar   C.O.R.E.  Webinar  Series 13  October,  2015
  2. 2. Power  Analysis Topics Error  Types Effect  Size Power  Analysis Hypothesis  Testing 2
  3. 3. ¡ RQ: Is  there  a  statistical  significant  difference  in students’  academic  performance  in  Math  between                           the  classes  of  Dr.  Adam  and  Dr.  Eve? ¡ Hnull: There  is  no  statistical  significant  difference  in students’  academic  performance  in  Math  between  the   classes  of  Dr.  Adam.  and  Dr.  Eve. You  are  the  Dean and  receive  the  following  report: 3
  4. 4. ¡ Report:An  Independent  Samples  T  Test  was  run   to  compare  the  means  of  a  Math  test  between  Dr.  Eve:                                                   M  =  90.96  (12.60)  and  Dr.  Adam:  M  =  89.32  (15.38),  yielding  a   statistical  significant  difference  with    t(1358)  =  2.164    p  =  .031  .   Hence  we  reject  Hnull and  conclude  that  the  Dr.  Eve’s  students   outperformed  Dr.  Adam’s  students. ¡ What  should  the  Dean do  based  on  these  accuratetrue  results?   § A: CritiqueDr.  Adam  on  his  students’  low  performance  and  set  a   deadline  and  minimal  score  for  him  to  meet. § B:  Promote  Dr.  Eve  and  let  Dr.  Adam  eat  his  heart  out. § C: Results  are  subject  to  chance due  to  small  sample  size,  and  we   need  to  rerun  study  with  a  larger  sample.   § D: Attend  Dr.  Shachar’s C.O.R.E.  Power Webinar   4
  5. 5. 5 q Problems with  Hypothesis  Significant Testing  -­‐ based  on  p  values  are: q The  p-­‐value depends  essentially  on  two   things:  the  size  of  the  effect  and the  size  of   the  sample.    One  would  get  a  ‘significant’   result  either  if  the  effect  were  very  big   (despite  having  only  a  small  sample)  or  if  the   sample  were  very  big  (even  if  the  actual  effect   size  were  tiny). q We  are  looking  at  “StatisticalSignificance”   and  not  at  “Practical Significance”.  
  6. 6. ¡ If  only  the  null  hypothesis  is   available  and  is  rejected,  at   most the  conclusion  is  that   “the  difference  is  not  zero” ¡ When  the  President  asks  the   Five-­‐Star  General  to  estimate the  war  casualty,  can  he  give   “not  zero”  as  a  satisfactory   answer?! 6
  7. 7. ¡ We  should  be  concerned   with  not  only  whether  a  null   hypothesis  is  false  or  not,  but   also  how  false  it  is. ¡ In  other  words,  if  the   difference  is  not  zero,  how   large the  difference  one   should  expect?   ¡ The  larger  the  effect  size  (the   difference  between  the  Hnull and  Halt Means)  is,  the   greater  the power of a test is. 7
  8. 8. A-­‐Priori-­‐ It  allows  you  to  decide,  in  the  process  of   designingan  experiment/study: ¡ How  large  a  sample  is  needed  to  enable  statistical judgments  that  are  accurate  and  reliable,  and ¡ How  likely  your  statistical  test  will  be  able  to  detect effects  of  a   given  size  in  a  particular  situation.   ¡ Without  these  calculations,  sample  size  may  be  too  high  or  too  low.   § If  sample  size  is  too  low,  the  experiment  will  lack  the  precision. § If  sample  size  is  too  large,  time  and  resources  will  be  wasted. Post-­‐Hoc  -­‐ It  allows  you  to  decide,  after study  was  executed: ¡ Whether  the  study  attained  an  acceptable  power,  and ¡ Whether  the  results  have  a  practical  significance. APA  -­‐ Publication  Requirements: ¡ All  study  publications  should  report  in  addition  to  p  values,  the   effect  sizes  (ES) and  their  Confidence  Interval  (CI). 8
  9. 9. Power Analysis Topics Error  Types Type  I  =  alpha Type  II  =  beta Power  =  1-­‐ beta Effect  Size Power  Analysis 9
  10. 10. ¡ The  null  hypothesis  is  either  true  or  false ¡ The  null  hypothesis  is  either  rejected or  not  rejected.   ¡ Only  4  possible  things  can  happen: State  of  the  World H0 State  of  the  World H1 Our  Decision   H0 Correct  Acceptance Type  II  Error   (beta) Our  Decision   H1 Type  I  Error  (alpha) Correct  Rejection 10
  11. 11. Common  acceptance  in  the  social  sciences: ¡ Type  I  error  -­‐ alpha, must  be  kept  at  or  below   .05 ¡ Type  II  error  -­‐ beta, must  be  kept  low as  well. ¡ "Statistical  power," which  is  equal  to  1  -­‐ beta,   must  be  kept  correspondingly  high. ¡ Ideally,  power  should  be  at  least  .80 to  detect   a  reasonable  departure  from  the  null   hypothesis. 11
  12. 12. Power  Analysis Topics Error  Types Effect Size ES  Indices Cohen’s  Conventions Power  Analysis 12
  13. 13. ¡ Effect  size  (ES)  is  a  name  given  to  a  family  of   indices that  measure  the  magnitude  of  a  treatment effect  (Becker,  2000). § Unlike  significance  tests,  these  indices  are  independent of  sample  size.   ¡ There  is  a  wide  array  of  formulas  used  to  measure  ES: § as  the  standardized  difference  between  two  means ‘d’   or  ‘g’ § as  the  correlation between  the  independent  variable   (IV)  classification  and  the  individual  scores  on  the   dependent  variable  (DV)    ‘r’. § Others:  OR,  HR,  RR,  etc. 13
  14. 14. The  simplest  form,  effect  size,  as  denoted                                 by  the  symbol  ‘d’  is  the  mean  difference  between   groups  in  standard  score  form  i.e.,  the  ratio  of  the   difference  between  the  means  to  the  standard   deviation. 14
  15. 15. Conventions Standardized   Difference  of   Means  ‘d’ Correlation  ‘r’ ‘Small’ 0.2 0.1 ‘Medium’ 0.5 0.3 ‘Large’ 0.8 0.5 15
  16. 16. PowerAnalysis Topics Error Types Effect Size PowerAnalysis a-priori post-hoc 16
  17. 17. The  factors  influencing  power  in  a  statistical  test: ¡ What  kind of  statistical  test  is  being  performed.   § You  will  need  to  calculate  a  different  effect  size  per  test  type!!! ¡ Sample  size.  In  general,  the  larger  the  sample  size,  the   larger  the  power.   ¡ The  size  of  experimental  effects.  If  the  null  hypothesis  is   wrong  by  a  substantial  amount,  power  will  be  higher  than   if  it  is  wrong  by  a  small  amount. ¡ The  level  of  error  in  experimental  measurements.  anything   that  enhances  the  accuracy  and  consistency  of   measurement  can  increase  statistical  power. 17
  18. 18. ¡ To  ensure  a  statistical  test  will  have   adequate  power,  one  usually  must  perform   special  analyses  prior  to  running  the   experiment,  to  calculate  how  large  an  N is   required. ¡ The  question  is,  "How  large  an  N is  necessary   to  produce  a  power that  is  reasonably  high"   in  this  situation,  while  maintaining  alpha at  a   reasonably  low  value  . 18
  19. 19. To  determine  the  sample  size  needed,                   we  play  with  four factors  (in  red  below): 1. Obtain  “ES”  -­‐ where  do  we  find  it? 1. Lit  review 2. Pilot 3. An  “educated  conjecture”   2. Define  alpha <=.05 3. Define  power (1-­‐beta)  .80 4. Calculate  sample  size  (by  stat  calculator)   see example 19
  20. 20. To  determine  the  sample  size  needed,                   we  play  with  four factors  (in  red  below): 1. Obtain  “ES”  -­‐ where  do  we  find  it? 1. Lit  review 2. Pilot 3. An  “educated  conjecture”   2. Define  alpha <=.05 3. Define  power (1-­‐beta)  .80 4. Calculate  sample  size  (can  use  Gpower) see example 20
  21. 21. 21 For  a  t  test  with:  ES=  .02,  Alpha=.05,  Power =  .8,   We  will  need  N=788 subjects  for  our  sample  
  22. 22. Now  that  we  are  done  with  our study,                       we  need  to  check  how  well  did  the  actual results  we  found  do  in  terms  of  power: Again,  we  play  with  four factors: 1. Input  “ES”  – from  our study 2. Define  alpha <=.05 3. Input  sample  size  -­‐ from  our study 4. Calculate  power  – can  use  G-­‐Power 22
  23. 23. 23 For  our    t  test  with:  ES=  .091,  Alpha=.05,  Sample  size   N=1360,  We  have  obtained  a  dismal.388  power  !!!
  24. 24. 24 ¡ Hypothesis  Testing  based  on  p  value  – provides  only  statisticalsignificance. ¡ Power  analysis  is  crucial for  your  study: ¡ A-­‐priori:  to  determine  required  sample  size ¡ Post-­‐hoc:   § To  calculate  and  examine  power from  actual research  study § To  examine  the  practical significance  of  the   research  findings.   ¡ If  you  fired Dr.  Adam  – Reinstatehim!!!
  25. 25. 25 ¡ “G  Power”  v.  (2015).  Buchner,   Erdfelder,  Faul,  &  Lang.   § To  download  software  for  free:   http://www.psycho.uni-­‐ ¡ Using  “G  Power”  for  Statistical  Power  and  Sample   Size  Analysis  (2008).  Eveland,  J.D.   § Download  instructions  to  follow  for  PPT. ¡ Becker,  L.  A.  (2000).  Effect  Sizes.  Retrieved:­‐size.html
  26. 26. Visit 26
  27. 27. 27
  28. 28. 28
  29. 29. Attention  Faculty,  Students,  Alumni  and  Guest  Speakers  in   Business,  Health  Sciences,  and  Education: ¡ Have  you  wanted  to  present  your  ongoing  scholarly  and   professional  work  to  a  general  audience? ¡ CORE Grand Rounds provides  a  platform  for   professional  development  and  increased  engagement   to  receive  constructive  feedback  from  peers  and   scholars-­‐in-­‐training. ¡ Email  Dr.  Bernice  B.  Rumala at  to  sign  up
  30. 30. 30 Thank You May the “power” be with you Dr. Mickey Shachar
  31. 31. ¡ To  receive  more  information  about  C.O.R.E.  please   visit  the  C.O.R.E.  webpage  at: ¡ For  further  information  about  Trident’s  doctoral   programs  in  educational  leadership,  business  and   health  sciences  please  visit  : ¡ Do  you  have  any  comments  for  C.O.R.E.,  you  may   email  Dr.  Bernice  B.  Rumala,  C.O.R.E.  Chair,  at: 31