Chapter 7<br />Effect Size and Power<br />Making Sense of Statistical Significance<br />
An effect can be statistically significant without having much practical significance.<br />Effect Size<br />It is a measu...
Effect Size and Distribution Overlap<br />
Raw Score Effect Size<br />calculated by taking the difference between the Population 1 mean and the Population 2 mean<br ...
If Population 1 had a mean of 90, Population 2 had a mean of 50, and the population standard deviation was 20, the effect ...
Effect Size = Population 1 M – Population 2 M<br />          Population SD<br />Population 1 M = the mean for the populati...
Example of Calculating the Effect Size<br />For the sample of 64 fifth graders, the best estimate of the Population 1 mean...
Standard rules about what to consider a small, medium, and large effect size <br />based on what is typical in behavioral ...
A More General Importance of Effect Size<br />Knowing the effect size of a study lets you compare results with effect size...
Statistical Power-The Ability to Achieve Your Goals!<br />Probability that the study will produce a statistically signific...
If the research hypothesis is false, we do not want to get significant results.<br />If we reject the null when the resear...
If there is a is a mean difference in the population, you have more chance of getting a significant result in the study.<b...
The more people there are in the study, the greater the power is.<br />The larger the sample size, the smaller the standar...
The main reason researchers consider power is to help them decide how many people to include in their studies.<br />Sample...
Significance Level<br />Less extreme significance levels (e.g., p < .10) mean more power because the shaded rejection area...
Statistical Significance vs. Practical Significance<br />It is possible for a study with a small effect size to be signifi...
With a small sample size, if a result is statistically significant, it is likely to be practically significant.<br />In a ...
A nonsignificant result from a study with low power is truly inconclusive.<br />A nonsignificant result from a study with ...
Upcoming SlideShare
Loading in …5
×

Aron chpt 7 ed effect size

2,267 views

Published on

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,267
On SlideShare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
28
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Aron chpt 7 ed effect size

  1. 1. Chapter 7<br />Effect Size and Power<br />Making Sense of Statistical Significance<br />
  2. 2. An effect can be statistically significant without having much practical significance.<br />Effect Size<br />It is a measure of the difference between populations.<br />It tells us how much something changes after a specific intervention.<br />It indicates the extent to which two populations do not overlap.<br />how much populations are separated due to the experimental procedure<br />With a smaller effect size, the populations will overlap more.<br />Effect Size<br />
  3. 3. Effect Size and Distribution Overlap<br />
  4. 4. Raw Score Effect Size<br />calculated by taking the difference between the Population 1 mean and the Population 2 mean<br />Standardized Effect Size<br />calculated by dividing the raw score effect size for each study by each study’s population standard deviation<br />This standardizes the difference between means in the same way a Z-score gives us a way to compare two scores on different measures.<br />Figuring The Effect Size<br />
  5. 5. If Population 1 had a mean of 90, Population 2 had a mean of 50, and the population standard deviation was 20, the effect size would be:<br />(90 – 50) / 20 = 2<br />This indicates that the effect of the experimental manipulation (e.g., reading program) is to increase the scores (e.g., reading level) by 2 standard deviations.<br />Copyright © 2011 by Pearson Education, Inc. All rights reserved<br />Effect Size Example<br />
  6. 6. Effect Size = Population 1 M – Population 2 M<br /> Population SD<br />Population 1 M = the mean for the population that receives the experimental manipulation<br />Population 2 M = the mean of the known population (the basis for the comparison distribution)<br />Population SD = the standard deviation of the population of individuals<br />A negative effect size would mean that the mean of Population 1 is lower than the mean of Population 2.<br />Formula for Calculating the Effect Size<br />
  7. 7. Example of Calculating the Effect Size<br />For the sample of 64 fifth graders, the best estimate of the Population 1 mean is the sample mean of 220.<br />The mean of Population 2 = 200 and the standard deviation is 48.<br />Effect Size = Population 1 M – Population 2 M<br /> Population SD<br />Effect Size = 220 – 200<br /> 48<br />Effect Size = .42<br />Copyright © 2011 by Pearson Education, Inc. All rights reserved<br />
  8. 8. Standard rules about what to consider a small, medium, and large effect size <br />based on what is typical in behavioral and social science research <br />Cohen’s effect size conventions for mean differences:<br />Effect Size Conventions<br />
  9. 9. A More General Importance of Effect Size<br />Knowing the effect size of a study lets you compare results with effect sizes found in other studies, even when the other studies have different population standard deviations.<br />Knowing what is a small or a large effect size helps you evaluate the overall importance of a result---<br />PRACTICAL SIGNIFICANCE!<br />A result may be statistically significant without having a very large effect.<br />Meta-Analysis<br />a procedure that combines results from different studies, even results using different methods or measurements<br />This is a quantitative rather than a qualitative review of the literature.<br />Effect sizes are a crucial part of this procedure.<br />
  10. 10. Statistical Power-The Ability to Achieve Your Goals!<br />Probability that the study will produce a statistically significant result when the research hypothesis is really true<br />When a study has only a small chance of being significant even if the research hypothesis is true, the study has low power.<br />When a study has a high chance of being significant when the study hypothesis is actually true, the study has high power.<br />
  11. 11. If the research hypothesis is false, we do not want to get significant results.<br />If we reject the null when the research hypothesis is false, we commit a TYPE I ERROR.<br />But, even if the research hypothesis is true, we do not always get significant results. When we FAIL to reject the null hypothesis when the <br />Copyright © 2011 by Pearson Education, Inc. All rights reserved<br />Remember…. <br />
  12. 12.
  13. 13. If there is a is a mean difference in the population, you have more chance of getting a significant result in the study.<br />Since the difference between population means is the main component of effect size, the bigger the effect size, the greater the power.<br />Effect size is also determined by the standard deviation of a population.<br />The smaller the standard deviation, the bigger the effect size.<br />The smaller the standard deviation, the greater the power.<br />What determines the Power of a Study?Effect Size and Power<br />
  14. 14. The more people there are in the study, the greater the power is.<br />The larger the sample size, the smaller the standard deviation of the distribution of means becomes.<br />The smaller the standard deviation of the distribution of means, the narrower the distribution of means—and the less overlap there is between distributions leading to higher power.<br />Remember that though sample size and effect size both influence power, they have nothing to do with each other.<br />Sample Size<br />
  15. 15. The main reason researchers consider power is to help them decide how many people to include in their studies.<br />Sample size has an important influence on power.<br />Researchers need to ensure that they have enough people in the study that they will be able see an effect if there is one.<br />Copyright © 2011 by Pearson Education, Inc. All rights reserved<br />Figuring Needed Sample Size for a Given Level of Power<br />
  16. 16. Significance Level<br />Less extreme significance levels (e.g., p < .10) mean more power because the shaded rejection area of the lower curve is bigger and more of the area in the upper curve is shaded.<br />More extreme significance levels (e.g., p < .001) mean less power because the shaded region in the lower curve is smaller.<br />One- vs. Two-Tailed Tests<br />Using a two-tailed test makes it harder to get significance on any one tail.<br />Power is less with a two-tailed test than a one-tailed test.<br />Copyright © 2011 by Pearson Education, Inc. All rights reserved<br />Other Influences on Power<br />
  17. 17. Statistical Significance vs. Practical Significance<br />It is possible for a study with a small effect size to be significant. <br />Though the results are statistically significant , they may not have any practical significance.<br />e.g., if you tested a psychological treatment and your result is not big enough to make a difference that matters when treating patients <br />Evaluating the practical significance of study results is important when studying hypotheses that have practical implications.<br />e.g., whether a therapy treatment works, whether a particular math tutoring program actually helps to improve math skills, or whether sending mailing reminders increases the number of people who respond to the Census<br />Statistical Significance vs. Practical Significance<br />
  18. 18. With a small sample size, if a result is statistically significant, it is likely to be practically significant.<br />In a study with a large sample size, the effect size should also be considered.<br />More things to think about….<br />
  19. 19. A nonsignificant result from a study with low power is truly inconclusive.<br />A nonsignificant result from a study with high power suggests that:<br />the research hypothesis is false or<br />there is less of an effect than was predicted when calculating power<br />Role of Power When a Result is Not Statistically Significant<br />

×