Hypothesis testing

1,719 views

Published on

Hypothesis testing by Dr. Badr Aljaser as part of the 5th Research Summer School - Jeddah at KAIMRC - WR

Published in: Education, Technology, Business
  • Be the first to comment

Hypothesis testing

  1. 1. Hypothesis Testing with One Sample
  2. 2. James Lind’s experiment
  3. 3. Hypothesis testingHypothesis testingDraw inferences about a populationbased on a sampleTesting a claim about a property ofa population
  4. 4. Statistical Inference∗ Inferences about a population are made onthe basis of results obtained from a sampledrawn from that population∗ Want to talk about the larger population fromwhich the subjects are drawn, not theparticular subjects!
  5. 5. What Do We Test ?∗ Effect or Difference we are interested in∗ Difference in Means or Proportions∗ Odds Ratio (OR)∗ Relative Risk (RR)∗ Correlation Coefficient∗ Clinically important difference∗ Smallest difference considered biologicallyor clinically relevant
  6. 6. Example: Gender Selection
  7. 7. Hypothesis TestingGoal: Make statement(s) regarding unknown populationparameter values based on sample data∗Elements of a hypothesis test:∗ Null hypothesis - Statement regarding the value(s) ofunknown parameter(s). Typically will imply no associationbetween explanatory and response variables in ourapplications (will always contain an equality)∗ Alternative hypothesis - Statement contradictory to the nullhypothesis (will always contain an inequality)
  8. 8. Null Hypothesis∗ Usually that there is no effect∗ Mean = 0∗ OR = 1∗ RR = 1∗ Correlation Coefficient = 0
  9. 9. Alternative Hypothesis∗ Contradicts the null∗ There is an effect∗ What you want to prove ?
  10. 10. Null Hypothesis expresses no differenceExample:H0: µ = 0Often said“H naught” Or any numberLater…….H0: µ1 = µ2
  11. 11. Alternative HypothesisH0: µ = 0; Null HypothesisHA: µ = 0; Alternative HypothesisResearcher’s predictions should bea priori, i.e. before looking at the data
  12. 12. Estimation: From the Sample∗ Point estimation∗ Mean∗ Median∗ Change in mean/median∗ Interval estimation∗ 95% Confidence interval∗ Variation
  13. 13. Parameters andReference DistributionsContinuous outcome data∗ Normal distribution: N( μ, σ2)∗ t distribution: tω (ω = degrees of freedom)∗Mean = (sample mean)∗Variance = s2(sample variance)Binary outcome data∗ Binomial distribution: B (n, p)X
  14. 14. Normal Distribution
  15. 15. t – Distribution
  16. 16. Binomial Distribution
  17. 17. Hypothesis TestingGoal: Make statement(s) regarding unknown populationparameter values based on sample dataElements of a hypothesis test:∗ Test statistic - Quantity based on sample data and nullhypothesis used to test between null and alternativehypotheses.∗ The test statistic is found by converting the sample statistic(proportion, mean or standard deviation) to a score (z, tz, t or xx22))
  18. 18. ∗ Critical region (Rejection region): Values of the teststatistic for which we reject the null in favor of thealternative hypothesisCritical Region, Significant level,Critical value and p-value
  19. 19. ∗ Significant level (α ): the probability that the teststatistic will fall in the critical region when the nullhypothesis is actually true.Critical Region, Significant level,Critical value and p-value
  20. 20. ∗ Critical value: is any value that separates the criticalregion from the values of the test statistic that do notlead to rejection of the null hypothesis .Critical Region, Significant level,Critical value and p-value
  21. 21. ∗ Two tailed: the critical region is in the two extremeregions (tails) under the curveTwo-Tailed, Left Tailed, Right Tailed
  22. 22. ∗ Left tailed: the critical region is in the extreme leftregion (tails) under the curveTwo-Tailed, Left Tailed, Right Tailed
  23. 23. ∗ Right tailed: the critical region is in the extreme rightregion (tails) under the curveTwo-Tailed, Left Tailed, Right Tailed
  24. 24. ∗ P-value (p-value or probability value: is theprobability of getting a value of the test statistic thatis at least as extreme as the one representing thesample data assuming the null hypothesis is true.∗ The null hypothesis is rejected if the p-value is verysmall such as 0.05 or less.Critical Region, Significant level,Critical value and p-value
  25. 25. Reject the null hypothesis (or other)Fail to reject the null hypothesisProve the null hypothesis to be trueAccept the null hypothesisSupport the null hypothesisStatisticallycorrectOk but misleading
  26. 26. ∗ Traditional Method: Rejection of the null hypothesisif the statistic falls within the critical region Fail to reject the null hypothesis if the test statistic doesnot fall within the critical region∗ P – value methodP – value method: rejection H0 if p-value < α (where αis the significant level such as 0.05)Decision Criterion
  27. 27. ∗ Another option:Another option: Instead of using a significant levelsuch as α = 0.05, simply identify the P value and leavethe decision to the reader∗ Confidence intervals:Confidence intervals: Because a Confidence intervalestimate of the population parameter contains thelikely values of that parameter, reject a claim that thepopulation parameter has a value that is not includedin the confidence intervalDecision Criterion
  28. 28. Statistical ErrorSometimes H0 will be rejected (based on large teststatistic & small p-value) even though H0 is really truei.e., if you had been able to measure the entirepopulation, not a sample, you would have foundno difference between and µ some value butbased on X you see a difference.The mistake of rejecting a true H0 will happen with frequency αSo, if H0 is true, it will be rejected ~5% of the time as α frequently = 0.05
  29. 29. 00 20Population mean = 0Sample mean = 20Conclude based on sample mean that population mean ≠ 0, but it reallydoes (H0 true), therefore you have falsely rejected H0Type I Errorpopulation=“True”Sample=What you seeH0 : mean = 0
  30. 30. Statistical ErrorSometimes H0 will be accepted (based on small test statistic & largep-value) even though H0 is really falsei.e., if you had been able to measure the entirepopulation, not a sample, you would have founda difference between and µ some value- butbased on X you do not see a difference.The mistake of accepting a false H0 will happenwith frequency β
  31. 31. 0Sample mean = 00 20Sample mean = 20Conclude based on sample mean that population mean = 0, but it reallydoes not (H0 really false), therefore you have falsely failed to reject H0Type II ErrorPopulation= “True”Sample= what you seeH0 : mean = 020
  32. 32. 1. The treatments do not differ, and we correctly concludethat they do not differ.2. The treatments do not differ, but we conclude that theydo differ.3. The treatments differ, but we conclude that they do notdiffer.4. The treatments do differ, and we correctly conclude thatthey do differ.Four Possibilities in Testing Whetherthe Treatments Differ
  33. 33. Type I error∗ Concluded thatthere is differencewhile in realitythere is nodifference∗ α probabilityType II error∗Concluded thatthere is nodifference while inreality there is adifference∗β probability
  34. 34. Controlling Type I & Type II ErrorsαβPower (1 – β)Sample Size
  35. 35. ∗ The power of the hypothesis test is the probability (1-(1- ββ))rejecting a false null hypothesis, which is computed byusing:∗ A particular significant level α∗ Sample size nn∗ A particular assumed value of the population parameterin the null hypothesis∗ A particular assumed value of the population parameterthat is alternative to the value in the null hypothesisPower of the test
  36. 36. Term Definitionsα = Probability of making a type I error= Probability of concluding the treatments differ when in realitythey do not differβ = Probability of making a type II error= Probability of concluding that the treatments do not differ whenin reality they do differPower = 1 - Probability of making a type II error= 1 - β= Probability of correctly concluding that the treatments differ= Probability of detecting a difference between the treatments ifthe treatments do in fact differ

×