Aron chpt 7 ed effect size f2011

1,640 views

Published on

Published in: Health & Medicine, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,640
On SlideShare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
61
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Aron chpt 7 ed effect size f2011

  1. 1. Making Sense of Statistical Significance <ul><li>Chapter 7 </li></ul><ul><li>Effect Size and Power </li></ul>
  2. 2. Effect Size <ul><li>An effect can be statistically significant without having much practical significance. </li></ul><ul><li>Effect Size </li></ul><ul><ul><li>It is a measure of the difference between populations. </li></ul></ul><ul><ul><li>It tells us how much something changes after a specific intervention. </li></ul></ul><ul><ul><li>It indicates the extent to which two populations do not overlap. </li></ul></ul><ul><ul><ul><li>how much populations are separated due to the experimental procedure </li></ul></ul></ul><ul><ul><li>With a smaller effect size, the populations will overlap more. </li></ul></ul>
  3. 3. Effect Size and Distribution Overlap
  4. 4. Figuring The Effect Size <ul><li>Raw Score Effect Size </li></ul><ul><ul><li>calculated by taking the difference between the Population 1 mean and the Population 2 mean </li></ul></ul><ul><li>Standardized Effect Size </li></ul><ul><ul><li>calculated by dividing the raw score effect size for each study by each study’s population standard deviation </li></ul></ul><ul><ul><li>This standardizes the difference between means in the same way a Z-score gives us a way to compare two scores on different measures. </li></ul></ul>
  5. 5. Effect Size Example <ul><li>If Population 1 had a mean of 90, Population 2 had a mean of 50, and the population standard deviation was 20, the effect size would be: </li></ul><ul><ul><li>(90 – 50) / 20 = 2 </li></ul></ul><ul><ul><ul><li>This indicates that the effect of the experimental manipulation (e.g., reading program) is to increase the scores (e.g., reading level) by 2 standard deviations. </li></ul></ul></ul>Copyright © 2011 by Pearson Education, Inc. All rights reserved
  6. 6. Formula for Calculating the Effect Size <ul><li>Effect Size = Population 1 M – Population 2 M </li></ul><ul><ul><ul><ul><li> Population SD </li></ul></ul></ul></ul><ul><ul><li>Population 1 M = the mean for the population that receives the experimental manipulation </li></ul></ul><ul><ul><li>Population 2 M = the mean of the known population (the basis for the comparison distribution) </li></ul></ul><ul><ul><li>Population SD = the standard deviation of the population of individuals </li></ul></ul><ul><ul><li>A negative effect size would mean that the mean of Population 1 is lower than the mean of Population 2. </li></ul></ul>
  7. 7. Example of Calculating the Effect Size <ul><li>For the sample of 64 fifth graders, the best estimate of the Population 1 mean is the sample mean of 220. </li></ul><ul><li>The mean of Population 2 = 200 and the standard deviation is 48. </li></ul><ul><li>Effect Size = Population 1 M – Population 2 M </li></ul><ul><ul><ul><ul><li> Population SD </li></ul></ul></ul></ul><ul><li>Effect Size = 220 – 200 </li></ul><ul><ul><ul><ul><li> 48 </li></ul></ul></ul></ul><ul><li>Effect Size = .42 </li></ul>Copyright © 2011 by Pearson Education, Inc. All rights reserved
  8. 8. Effect Size Conventions <ul><li>Standard rules about what to consider a small, medium, and large effect size </li></ul><ul><ul><li>based on what is typical in behavioral and social science research </li></ul></ul><ul><ul><ul><li>Cohen’s effect size conventions for mean differences: </li></ul></ul></ul>How Big? Effect Size (Cohen’s d) No practical effect Less than .20 Small effect size .20-.49 Medium effect size .50-.79 Large effect size .80 or greater
  9. 9. A More General Importance of Effect Size <ul><li>Knowing the effect size of a study lets you compare results with effect sizes found in other studies, even when the other studies have different population standard deviations. </li></ul><ul><li>Knowing what is a small or a large effect size helps you evaluate the overall importance of a result--- </li></ul><ul><li>PRACTICAL SIGNIFICANCE! </li></ul><ul><ul><li>A result may be statistically significant without having a very large effect. </li></ul></ul><ul><li>Meta-Analysis </li></ul><ul><ul><li>a procedure that combines results from different studies, even results using different methods or measurements </li></ul></ul><ul><ul><li>This is a quantitative rather than a qualitative review of the literature. </li></ul></ul><ul><ul><li>Effect sizes are a crucial part of this procedure. </li></ul></ul>
  10. 10. Statistical Power- The Ability to Achieve Your Goals! <ul><li>Probability that the study will produce a statistically significant result when the research hypothesis is really true </li></ul><ul><ul><li>When a study has only a small chance of being significant even if the research hypothesis is true, the study has low power. </li></ul></ul><ul><ul><li>When a study has a high chance of being significant when the study hypothesis is actually true, the study has high power. </li></ul></ul>
  11. 11. Remember…. <ul><li>If the research hypothesis is false, we do not want to get significant results. </li></ul><ul><li>If we reject the null when the research hypothesis is false, we commit a TYPE I ERROR. </li></ul><ul><li>But, even if the research hypothesis is true, we do not always get significant results. When we FAIL to reject the null hypothesis when the </li></ul>Copyright © 2011 by Pearson Education, Inc. All rights reserved
  12. 13. What determines the Power of a Study? Effect Size and Power <ul><li>If there is a is a mean difference in the population, you have more chance of getting a significant result in the study. </li></ul><ul><ul><li>Since the difference between population means is the main component of effect size, the bigger the effect size, the greater the power. </li></ul></ul><ul><ul><li>Effect size is also determined by the standard deviation of a population. </li></ul></ul><ul><ul><ul><li>The smaller the standard deviation, the bigger the effect size. </li></ul></ul></ul><ul><ul><ul><ul><li>The smaller the standard deviation, the greater the power. </li></ul></ul></ul></ul>
  13. 14. Sample Size <ul><li>The more people there are in the study, the greater the power is. </li></ul><ul><li>The larger the sample size, the smaller the standard deviation of the distribution of means becomes. </li></ul><ul><ul><li>The smaller the standard deviation of the distribution of means, the narrower the distribution of means—and the less overlap there is between distributions leading to higher power. </li></ul></ul><ul><ul><ul><li>Remember that though sample size and effect size both influence power, they have nothing to do with each other. </li></ul></ul></ul>
  14. 15. Figuring Needed Sample Size for a Given Level of Power <ul><li>The main reason researchers consider power is to help them decide how many people to include in their studies. </li></ul><ul><ul><li>Sample size has an important influence on power. </li></ul></ul><ul><ul><li>Researchers need to ensure that they have enough people in the study that they will be able see an effect if there is one. </li></ul></ul>Copyright © 2011 by Pearson Education, Inc. All rights reserved
  15. 16. Other Influences on Power <ul><li>Significance Level </li></ul><ul><ul><li>Less extreme significance levels (e.g., p < .10) mean more power because the shaded rejection area of the lower curve is bigger and more of the area in the upper curve is shaded. </li></ul></ul><ul><ul><li>More extreme significance levels (e.g., p < .001) mean less power because the shaded region in the lower curve is smaller. </li></ul></ul><ul><li>One- vs. Two-Tailed Tests </li></ul><ul><ul><li>Using a two-tailed test makes it harder to get significance on any one tail. </li></ul></ul><ul><ul><ul><li>Power is less with a two-tailed test than a one-tailed test. </li></ul></ul></ul>Copyright © 2011 by Pearson Education, Inc. All rights reserved
  16. 17. Statistical Significance vs. Practical Significance <ul><li>Statistical Significance vs. Practical Significance </li></ul><ul><ul><li>It is possible for a study with a small effect size to be significant. </li></ul></ul><ul><ul><ul><li>Though the results are statistically significant , they may not have any practical significance. </li></ul></ul></ul><ul><ul><ul><ul><li>e.g., if you tested a psychological treatment and your result is not big enough to make a difference that matters when treating patients </li></ul></ul></ul></ul><ul><li>Evaluating the practical significance of study results is important when studying hypotheses that have practical implications. </li></ul><ul><ul><li>e.g., whether a therapy treatment works, whether a particular math tutoring program actually helps to improve math skills, or whether sending mailing reminders increases the number of people who respond to the Census </li></ul></ul>
  17. 18. More things to think about…. <ul><li>With a small sample size, if a result is statistically significant, it is likely to be practically significant. </li></ul><ul><li>In a study with a large sample size, the effect size should also be considered. </li></ul>
  18. 19. Role of Power When a Result is Not Statistically Significant <ul><li>A nonsignificant result from a study with low power is truly inconclusive. </li></ul><ul><li>A nonsignificant result from a study with high power suggests that: </li></ul><ul><ul><li>the research hypothesis is false or </li></ul></ul><ul><ul><li>there is less of an effect than was predicted when calculating power </li></ul></ul>

×