Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

24 Est Testing

429 views

Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

24 Est Testing

  1. 1. Stat310 Estimation + Testing Hadley Wickham Saturday, 11 April 2009
  2. 2. 1. What makes a good estimator? 2. Recap & general strategy 3. Non-symmetric distributions 4. Testing Saturday, 11 April 2009
  3. 3. Low bias, low variance Low bias, high variance High bias, low variance High bias, high variance Saturday, 11 April 2009
  4. 4. Can combine both together to get mean squared error ˆ = E[(θ − θ)2 ] ˆ MSE(θ) ˆ = Var(θ) + Bias(θ, θ)2 ˆ ˆ MSE(θ) Saturday, 11 April 2009
  5. 5. Recap (Z + 1)/5 ~ SomeDistribution(θ, β) What, mathematically, is a 95% confidence interval around Z? Write down the steps you’d take to generate such an interval if you knew θ and β Saturday, 11 April 2009
  6. 6. Problem Y = g(X) Y ~ F(θ) (g has an inverse) Find a 1 - α confidence interval for X. i.e. Find a and b so that P(a < X < b) = 1 - α Saturday, 11 April 2009
  7. 7. Solution 1. Find a 1 - α confidence interval for Y. P(c < Y < d) = 1 - α a. If F is symmetric, then the bounds will -1(α/2) and d = F-1(1 - α/2) be c = F b. If F isn’t symmetric then it’s harder 2. a = g -1(c), b= g -1(d) Saturday, 11 April 2009
  8. 8. Example 340 333 334 332 333 336 350 348 331 344 (mean: 338, sd: 7.01) Find a 95% confidence interval for μ ¯n − µ X √ ∼ tn−1 s/ n Saturday, 11 April 2009
  9. 9. Saturday, 11 April 2009
  10. 10. More complicated case (n − 1)S 2 X ∼ χ (n − 1) 2 X= 2 σ Find 95% confidence interval for standard deviation in previous case (sd = 7.01, n = 10) Saturday, 11 April 2009
  11. 11. Standard deviation Find confidence interval for X ~ χ2(9). Generally want the shortest confidence interval, but hard to find when not symmetric. Any of the following are correct. The best has the smallest interval. Saturday, 11 April 2009
  12. 12. 0.10 0.08 0.06 0.04 0.02 0.00 0 5 10 15 20 25 30 Saturday, 11 April 2009
  13. 13. 0.10 (0.05, 1) (3.33,Inf) Length: Inf 0.08 0.06 0.04 0.02 0.00 0 5 10 15 20 25 30 Saturday, 11 April 2009
  14. 14. 0.10 (0.03, 0.99) (2.85,21.67) Length: 18.8 0.08 0.06 0.04 0.02 0.00 0 5 10 15 20 25 30 Saturday, 11 April 2009
  15. 15. 0.10 (0.025, 0.975) (2.7,19.0) Length: 16.3 0.08 0.06 0.04 0.02 0.00 0 5 10 15 20 25 30 Saturday, 11 April 2009
  16. 16. 0.10 (0.01, 0.96) (2.09,17.61) Length: 15.5 0.08 0.06 0.04 0.02 0.00 0 5 10 15 20 25 30 Saturday, 11 April 2009
  17. 17. 0.10 (0, 0.95) (0.0,16.9) Length: 16.9 0.08 0.06 0.04 0.02 0.00 0 5 10 15 20 25 30 Saturday, 11 April 2009
  18. 18. Your turn Find 95% confidence interval for the standard deviation (sd = 7.01, n = 10) P(2.09 < X < 17.61) = 0.95 (n − 1)S 2 X= 2 σ Saturday, 11 April 2009
  19. 19. Testing Saturday, 11 April 2009
  20. 20. Testing Very closely related to estimation (particularly confidence intervals) But point is to answer a yes/no question: Is the mean of the distribution equal to 0? Do X and Y have the same mean? Saturday, 11 April 2009
  21. 21. Your turn The following values have been drawn from a normal distribution with standard deviation 1. 2.9 2.1 3.0 3.2 1.2 3.0 3.3 1.2 2.3 1.5 (mean: 2.13) Is it possible they came from a normal distribution with mean 1.5? Saturday, 11 April 2009
  22. 22. Example Create 95% confidence interval. Is it inside? Create 90% confidence interval. Is it inside? … Or we can look up the value directly, using the cdf Saturday, 11 April 2009
  23. 23. Testing jargon No: Null hypothesis. Nothing is happening. (Thing we want to disprove) Yes: Alternative hypothesis. Something interesting is happening. Major complication: Saturday, 11 April 2009
  24. 24. Absence of evidence is not evidence of absence Saturday, 11 April 2009
  25. 25. Implication Means we never “accept” the null hypothesis, just “fail to reject” it. Null distribution is usually simple case for which we know the distribution Saturday, 11 April 2009
  26. 26. Your turn Null hypothesis: μ = 1.5 Alternative hypothesis: μ > 1.5 OR μ < 1.5 Under the null hypothesis what is the distribution of the mean? How does what we saw compare to the null distribution? Is it likely or not? Saturday, 11 April 2009
  27. 27. P-value P value gives us the probability, under the null hypothesis, that we would have seen a value equal to or more extreme than the value we observed. Strength of evidence for rejecting the null hypothesis. But we need a cut off to make a yes-no decision. How do we choose that cut off? Saturday, 11 April 2009
  28. 28. Errors What are the possible errors we can make? False positive. Choose alternative when null is correct. (aka Type 1) False negative. Choose null when alternative is true. (aka Type 2) Saturday, 11 April 2009
  29. 29. Terminology Probability of a false positive called α Probability of false negative called 1 - β How are the two related? Usually care more about false positives Saturday, 11 April 2009
  30. 30. Testing overview Write down null and alternative hypotheses. Compute test statistic. Convert to p-value. Compare p-value to alpha cut off. Saturday, 11 April 2009
  31. 31. Next time Some specific tests. i.e. for common situations what is the distribution under the null-hypothesis Saturday, 11 April 2009

×