Upcoming SlideShare
×

# 23 testing

766 views

Published on

0 Likes
Statistics
Notes
• Full Name
Comment goes here.

Are you sure you want to Yes No
• Be the first to comment

• Be the first to like this

Views
Total views
766
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
10
0
Likes
0
Embeds 0
No embeds

No notes for slide

### 23 testing

1. 1. Stat310 Hypothesis testing Hadley Wickham Wednesday, 21 April 2010
2. 2. 1. Quick review 2. More about the interval on the sampling distribution 3. More complicated example 4. Introduction to hypothesis testing (aka the statistical justice system) Wednesday, 21 April 2010
3. 3. Review Identify distribution that connects estimator and true value. Form conﬁdence interval for known (sampling) distribution. Write as probability statement. Back transform. Write as interval. Wednesday, 21 April 2010
4. 4. Steps Identify distribution that connects estimator and true value. Form conﬁdence interval for known (sampling) distribution. Write as probability statement. Back transform. Write as interval. Wednesday, 21 April 2010
5. 5. Steps Want P(a < Q < b) = 1 - α, and b - a to be as small as possible. If Q is symmetric, P(-a < Q < a) = 1 - α. So a = F -1(α/2), and there is no interval smaller. If Q isn’t symmetric, pick a = F -1(α/2), b=F -1(1 - α/2), but there might be a shorter interval. Wednesday, 21 April 2010
6. 6. Example We want a 90% conﬁdence interval, then two possible ends for the interval are F -1(0.05) and F -1(0.95) Wednesday, 21 April 2010
7. 7. Variance Xi iid Normal(5, σ2), i = 1, ..., 10 I ran the experiment and recorded the following results: 0.15 1.48 1.25 2.47 1.09 1.95 1.46 1.49 2.81 1.96 (var = 0.55) Find a 95% conﬁdence interval for the variance. (Hint: If X~χ2(9), FX(19) = 0.975 and FX(2.7) = 0.025). EC: Can you also make a conﬁdence interval for the standard deviation? Wednesday, 21 April 2010
8. 8. Rest of chapter The rest of chapter 6 just gives examples of the general method we learned this week. Any of it is fair game in the ﬁnal. But don’t memorise speciﬁcs: learn the general steps! Wednesday, 21 April 2010
9. 9. So far Given Xi iid SomeDist(a, b), and data, we can give estimates (including uncertainty) for a and b. Next: We’ll learn how to answer questions like is a > 0, or is b in [5, 10]. Wednesday, 21 April 2010
10. 10. Your turn The following values have generated from iid Normal(μ, 1): 2.9 2.1 3.0 3.2 1.2 3.0 3.3 1.2 2.3 1.5 (mean: 2.13) Is it possible they came from a normal distribution with mean 1.5? Wednesday, 21 April 2010
11. 11. Hypothesis testing The statistical justice system http://www.ﬂickr.com/photos/joegratz/117048243 Wednesday, 21 April 2010
12. 12. A suspect is accused of a crime. The suspect is declared guilty or innocent based on a trial. Each trial has a defence and a prosecution. On the basis of how evidence compares to a standard, the judge makes a decision to convict or acquit. Wednesday, 21 April 2010
13. 13. A dataset is accused of having a particular parameter value. The data is declared guilty or innocent based on the results of a statistical test. Each test has a null hypothesis and an alternative hypothesis. On the basis of how a test statistic compares to a standard distribution, we make the decision to reject the null or fail to reject the null hypothesis. Wednesday, 21 April 2010
14. 14. Your turn With a partner, write down the criminal equivalents to the following terms: null hypothesis, alternative hypothesis, test statistic, test distribution, reject the null hypothesis, fail to reject the null hypothesis Wednesday, 21 April 2010
15. 15. Hypthoses Every case has two sides: The null hypothesis (the defence). This is the default, or the status quo, what you need to argue against. The alternative hypothesis (the prosecution). This is the interesting case, but you need to prove it’s true. Wednesday, 21 April 2010
16. 16. In the statistical justice system evidence is based on the similarity between the accused and known innocents. The population of innocents, called the null distribution, is generated by the combination of null hypothesis and test statistic. To determine the guilt of the accused we compute the proportion of innocents who look more guilty than the accused. This is the p- value, the probability that the accused would look this guilty if they actually were innocent. Wednesday, 21 April 2010
17. 17. The lady tasting tea A thought experiment by R. A. Fisher (famous early statistician, 1890-1962) A lady at a tea party claims that she can tell the difference between putting the milk in ﬁrst and second. How can we be sure? Wednesday, 21 April 2010
18. 18. Experiment 8 cups. 4 milk ﬁrst, 4 milk second. Presented in random order. What is the null hypothesis? (What is the position of the defence?) What test-statistic might we use? (What can we measure to assess evidence?) What is the null-distribution? (What do innocents look like?) Wednesday, 21 April 2010
19. 19. Right Wrong # % 4 0 1 1% 3 1 16 23% 2 2 36 51% 1 3 16 23% 0 4 1 1% 70 100% http://www.uwsp.edu/education/wkirby/t&m/5LadyTastingTea.htm Wednesday, 21 April 2010
20. 20. 1. Write down null and alternative hypotheses (positions of defence and prosecution) 2. Figure out good test statistic (what numeric summary captures useful evidence) 3. Work out null distribution (distribution of innocents) 4. Calculate p-value by comparing actual value to null distribution (what proportion of true innocents look more guilty than the suspect) Wednesday, 21 April 2010
21. 21. Your turn The following values have generated from iid Normal(μ, 1): 2.9 2.1 3.0 3.2 1.2 3.0 3.3 1.2 2.3 1.5 (mean: 2.13) Does μ = 1.5 ? Wednesday, 21 April 2010
22. 22. Wednesday, 21 April 2010
23. 23. Reading Make sure you are familiar with the similarities and differences between the criminal and statistical justice systems. Read Sections 7.1 and 7.41 for a classical take on the issue. Can you give the errors names to match the statistical justice system? Wednesday, 21 April 2010