Stat310          Estimation + Testing


                             Hadley Wickham
Saturday, 11 April 2009
1. What makes a good estimator?
                 2. Recap & general strategy
                 3. Non-symmetric distributio...
Low bias, low variance    Low bias, high variance




                High bias, low variance   High bias, high variance

...
Can combine both
   together to get mean
   squared error


             ˆ = E[(θ − θ)2 ]
                    ˆ
         M...
Recap
                     (Z + 1)/5 ~ SomeDistribution(θ, β)
                     What, mathematically, is a 95%
        ...
Problem

                     Y = g(X) Y ~ F(θ) (g has an inverse)
                     Find a 1 - α confidence interval fo...
Solution
                     1. Find a 1 - α confidence interval for Y.
                        P(c < Y < d) = 1 - α
     ...
Example
                     340 333 334 332 333 336 350 348 331
                     344 (mean: 338, sd: 7.01)
          ...
Saturday, 11 April 2009
More complicated case

          (n − 1)S                    2
                                           X ∼ χ (n − 1)
  ...
Standard deviation

                     Find confidence interval for X ~ χ2(9).
                     Generally want the sh...
0.10




   0.08




   0.06




   0.04




   0.02




   0.00

                  0       5   10   15   20   25   30

Sa...
0.10
                                                    (0.05, 1)
                                                   (3.3...
0.10
                                               (0.03, 0.99)
                                              (2.85,21.67...
0.10
                                             (0.025, 0.975)
                                                 (2.7,19....
0.10
                                               (0.01, 0.96)
                                              (2.09,17.61...
0.10
                                                 (0, 0.95)
                                                (0.0,16.9)...
Your turn
                     Find 95% confidence interval for the
                     standard deviation (sd = 7.01, n =...
Testing



Saturday, 11 April 2009
Testing

                     Very closely related to estimation
                     (particularly confidence intervals)
 ...
Your turn
                     The following values have been drawn
                     from a normal distribution with s...
Example
                     Create 95% confidence interval.
                     Is it inside?
                     Create...
Testing jargon
                     No: Null hypothesis. Nothing is
                     happening. (Thing we want to disp...
Absence of
     evidence is not
  evidence of absence

Saturday, 11 April 2009
Implication

                     Means we never “accept” the null
                     hypothesis, just “fail to reject” ...
Your turn
                     Null hypothesis: μ = 1.5
                     Alternative hypothesis: μ > 1.5 OR μ < 1.5
  ...
P-value
                     P value gives us the probability, under the
                     null hypothesis, that we wou...
Errors
                     What are the possible errors we can
                     make?
                     False posi...
Terminology
                     Probability of a false positive called α
                     Probability of false negati...
Testing overview

                     Write down null and alternative
                     hypotheses.
                  ...
Next time

                     Some specific tests.
                     i.e. for common situations what is the
          ...
Upcoming SlideShare
Loading in...5
×

24 Est Testing

309
-1

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
309
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

24 Est Testing

  1. 1. Stat310 Estimation + Testing Hadley Wickham Saturday, 11 April 2009
  2. 2. 1. What makes a good estimator? 2. Recap & general strategy 3. Non-symmetric distributions 4. Testing Saturday, 11 April 2009
  3. 3. Low bias, low variance Low bias, high variance High bias, low variance High bias, high variance Saturday, 11 April 2009
  4. 4. Can combine both together to get mean squared error ˆ = E[(θ − θ)2 ] ˆ MSE(θ) ˆ = Var(θ) + Bias(θ, θ)2 ˆ ˆ MSE(θ) Saturday, 11 April 2009
  5. 5. Recap (Z + 1)/5 ~ SomeDistribution(θ, β) What, mathematically, is a 95% confidence interval around Z? Write down the steps you’d take to generate such an interval if you knew θ and β Saturday, 11 April 2009
  6. 6. Problem Y = g(X) Y ~ F(θ) (g has an inverse) Find a 1 - α confidence interval for X. i.e. Find a and b so that P(a < X < b) = 1 - α Saturday, 11 April 2009
  7. 7. Solution 1. Find a 1 - α confidence interval for Y. P(c < Y < d) = 1 - α a. If F is symmetric, then the bounds will -1(α/2) and d = F-1(1 - α/2) be c = F b. If F isn’t symmetric then it’s harder 2. a = g -1(c), b= g -1(d) Saturday, 11 April 2009
  8. 8. Example 340 333 334 332 333 336 350 348 331 344 (mean: 338, sd: 7.01) Find a 95% confidence interval for μ ¯n − µ X √ ∼ tn−1 s/ n Saturday, 11 April 2009
  9. 9. Saturday, 11 April 2009
  10. 10. More complicated case (n − 1)S 2 X ∼ χ (n − 1) 2 X= 2 σ Find 95% confidence interval for standard deviation in previous case (sd = 7.01, n = 10) Saturday, 11 April 2009
  11. 11. Standard deviation Find confidence interval for X ~ χ2(9). Generally want the shortest confidence interval, but hard to find when not symmetric. Any of the following are correct. The best has the smallest interval. Saturday, 11 April 2009
  12. 12. 0.10 0.08 0.06 0.04 0.02 0.00 0 5 10 15 20 25 30 Saturday, 11 April 2009
  13. 13. 0.10 (0.05, 1) (3.33,Inf) Length: Inf 0.08 0.06 0.04 0.02 0.00 0 5 10 15 20 25 30 Saturday, 11 April 2009
  14. 14. 0.10 (0.03, 0.99) (2.85,21.67) Length: 18.8 0.08 0.06 0.04 0.02 0.00 0 5 10 15 20 25 30 Saturday, 11 April 2009
  15. 15. 0.10 (0.025, 0.975) (2.7,19.0) Length: 16.3 0.08 0.06 0.04 0.02 0.00 0 5 10 15 20 25 30 Saturday, 11 April 2009
  16. 16. 0.10 (0.01, 0.96) (2.09,17.61) Length: 15.5 0.08 0.06 0.04 0.02 0.00 0 5 10 15 20 25 30 Saturday, 11 April 2009
  17. 17. 0.10 (0, 0.95) (0.0,16.9) Length: 16.9 0.08 0.06 0.04 0.02 0.00 0 5 10 15 20 25 30 Saturday, 11 April 2009
  18. 18. Your turn Find 95% confidence interval for the standard deviation (sd = 7.01, n = 10) P(2.09 < X < 17.61) = 0.95 (n − 1)S 2 X= 2 σ Saturday, 11 April 2009
  19. 19. Testing Saturday, 11 April 2009
  20. 20. Testing Very closely related to estimation (particularly confidence intervals) But point is to answer a yes/no question: Is the mean of the distribution equal to 0? Do X and Y have the same mean? Saturday, 11 April 2009
  21. 21. Your turn The following values have been drawn from a normal distribution with standard deviation 1. 2.9 2.1 3.0 3.2 1.2 3.0 3.3 1.2 2.3 1.5 (mean: 2.13) Is it possible they came from a normal distribution with mean 1.5? Saturday, 11 April 2009
  22. 22. Example Create 95% confidence interval. Is it inside? Create 90% confidence interval. Is it inside? … Or we can look up the value directly, using the cdf Saturday, 11 April 2009
  23. 23. Testing jargon No: Null hypothesis. Nothing is happening. (Thing we want to disprove) Yes: Alternative hypothesis. Something interesting is happening. Major complication: Saturday, 11 April 2009
  24. 24. Absence of evidence is not evidence of absence Saturday, 11 April 2009
  25. 25. Implication Means we never “accept” the null hypothesis, just “fail to reject” it. Null distribution is usually simple case for which we know the distribution Saturday, 11 April 2009
  26. 26. Your turn Null hypothesis: μ = 1.5 Alternative hypothesis: μ > 1.5 OR μ < 1.5 Under the null hypothesis what is the distribution of the mean? How does what we saw compare to the null distribution? Is it likely or not? Saturday, 11 April 2009
  27. 27. P-value P value gives us the probability, under the null hypothesis, that we would have seen a value equal to or more extreme than the value we observed. Strength of evidence for rejecting the null hypothesis. But we need a cut off to make a yes-no decision. How do we choose that cut off? Saturday, 11 April 2009
  28. 28. Errors What are the possible errors we can make? False positive. Choose alternative when null is correct. (aka Type 1) False negative. Choose null when alternative is true. (aka Type 2) Saturday, 11 April 2009
  29. 29. Terminology Probability of a false positive called α Probability of false negative called 1 - β How are the two related? Usually care more about false positives Saturday, 11 April 2009
  30. 30. Testing overview Write down null and alternative hypotheses. Compute test statistic. Convert to p-value. Compare p-value to alpha cut off. Saturday, 11 April 2009
  31. 31. Next time Some specific tests. i.e. for common situations what is the distribution under the null-hypothesis Saturday, 11 April 2009
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×