Successfully reported this slideshow.
Upcoming SlideShare
×

# Two sample t-test

19,198 views

Published on

From http://isites.harvard.edu/fs/docs/icb.topic154887.files/Two_sample_t-test.ppt

Published in: Education
• Full Name
Comment goes here.

Are you sure you want to Yes No
• The Complete Idiot's Guide to Statistics, 2nd Edition (Idiot's Guides) --- http://amzn.to/1T2ZU7E

Are you sure you want to  Yes  No
• Statistics Laminate Reference Chart: Parameters, Variables, Intervals, Proportions (Quickstudy: Academic ) --- http://amzn.to/1pUyTru

Are you sure you want to  Yes  No
• Statistics For Dummies --- http://amzn.to/1MisC2C

Are you sure you want to  Yes  No

### Two sample t-test

1. 1. Comparison of two samples Summer program Brian Healy
2. 2. Previous classes <ul><li>Hypothesis testing </li></ul><ul><ul><li>Null and Alternative hypotheses </li></ul></ul><ul><ul><li>Test statistic </li></ul></ul><ul><ul><li>p-value </li></ul></ul><ul><ul><li>Conclusion </li></ul></ul><ul><li>Confidence intervals </li></ul><ul><li>Comparison of CI to hypothesis test </li></ul><ul><li>Power and sample size </li></ul>
3. 3. What are we doing today? <ul><li>Two-sample t-test </li></ul><ul><ul><li>Paired t-test </li></ul></ul><ul><ul><li>Independent samples </li></ul></ul><ul><ul><ul><li>Equal variance </li></ul></ul></ul><ul><ul><ul><li>Unequal variance </li></ul></ul></ul><ul><li>Sample size for two samples </li></ul>
4. 4. Big picture <ul><li>Up to this point, we have only concerned ourselves with one sample. Often we want to compare one group to another. What happens when we are comparing two samples? </li></ul><ul><li>Variability in both samples, and potentially two samples are related </li></ul><ul><li>Much of the theory is the same </li></ul>
5. 5. Example <ul><li>One of the first studies I analyzed was a tumor size study. Having an accurate measure of tumor size is extremely important because it allows a physician to accurately determine if a tumor is growing, shrinking or remaining constant. </li></ul><ul><li>The problem is that often the measurements of the tumor size vary from physician to physician. </li></ul><ul><li>In the past, tumor size was measured using the linear distance across the tumor, but this was found to be very variable because of the irregular shape of some tumors. A new method called the RECIST criteria, which traces the outside of the tumor, measures the volume of the tumor. The volumetric method was believed to give more consistent measures of the volume of the tumor. </li></ul>
6. 6. Available data <ul><li>For a portion of the study, a pair of doctors were shown the same set of tumor pictures. The volume of the tumor was measured by two separate physicians under similar conditions. </li></ul><ul><li>Question of interest: Did the measurements from the two physicians significantly differ? </li></ul><ul><li>If not, then there would be no evidence that the volume measurements change based on physician. </li></ul>
7. 7. <ul><li>20 scans were measured by each physician (10 are shown here) </li></ul><ul><li>Measurements in cm 3 </li></ul><ul><li>What can you say about these samples? </li></ul><ul><ul><li>Two measurement on the same person </li></ul></ul><ul><ul><li>They are related so we must account for this </li></ul></ul><ul><ul><li>Much research in statistics deals with how to handle correlated data, but in this case it is pretty easy </li></ul></ul>19.7 20.5 10 27.5 29.3 9 25.4 23.0 8 20.3 21.8 7 24.8 24.0 6 28.0 26.8 5 18.5 15.7 4 14.2 14.5 3 20.3 22.3 2 17.2 15.8 1 Dr. 2 Dr. 1 Tumor
8. 8. Dependent sample <ul><li>We can measure the effect of the treatment in each person by taking the difference </li></ul><ul><li>Instead of having two samples, we can consider our dataset to be one sample of differences </li></ul><ul><ul><li>Just like the one sample problem </li></ul></ul>19.7 27.5 25.4 20.3 24.8 28.0 18.5 14.2 20.3 17.2 Dr. 2 0.8 1.8 -2.4 1.5 -0.8 -1.2 -2.8 0.3 2.0 -1.4 Difference 20.5 10 29.3 9 23.0 8 21.8 7 24.0 6 26.8 5 15.7 4 14.5 3 22.3 2 15.8 1 Dr. 1 Tumor
9. 9. Differences <ul><li>Volume from Dr. 1 </li></ul><ul><ul><li>Population mean: </li></ul></ul><ul><ul><li>Sample mean: </li></ul></ul><ul><li>Volume from Dr. 2 </li></ul><ul><ul><li>Population mean: </li></ul></ul><ul><ul><li>Sample mean: </li></ul></ul><ul><li>Difference </li></ul><ul><ul><li>Population mean: </li></ul></ul><ul><ul><li>Sample mean: </li></ul></ul>
10. 10. Distribution of differences <ul><li>Assuming d i ’s are normally distributed, can use t-distribution with n-1 dof where n is the number of differences </li></ul><ul><li>Standard deviation of differences </li></ul><ul><li>Test statistic acts just like one sample </li></ul>
11. 11. Picture <ul><li>We can see that the assumption of normality of the differences is reasonable in this case </li></ul>
12. 12. Paired t-test <ul><li>Two dependent samples; alpha=0.05 </li></ul><ul><li>Null hypothesis: No difference between physicians effect </li></ul><ul><li>Test statistic: t-statistic with dof </li></ul><ul><li>p-value=0.53 </li></ul><ul><li>Fail to reject null hypothesis </li></ul><ul><li>Conclusion: there is no evidence of a difference in tumor volume measurement based on physician </li></ul>
13. 13. Confidence interval <ul><li>Confidence interval for paired t-test constructed in the same way as one-sample t-test </li></ul><ul><li>For our example, the confidence interval is </li></ul><ul><li>(-1.01 0.54) </li></ul><ul><li>Note that the conclusion from the hypothesis test and the confidence interval are the same </li></ul>
14. 14. Paired t-test in R <ul><li>data<-read.table(G:IO232ummerairedscans.dat”, header=F) </li></ul><ul><li>dr1<-data[,1]; dr2<-data[,2] </li></ul><ul><li>t.test(dr1, dr2, paired=T) </li></ul><ul><ul><li>The output provides the p-value and the confidence interval </li></ul></ul><ul><ul><li>Paired t-test </li></ul></ul><ul><ul><li>data: data[, 1] and data[, 2] </li></ul></ul><ul><ul><li>t = -0.6456, df = 19, p-value = 0.5262 </li></ul></ul><ul><ul><li>alternative hypothesis: true difference in means is not equal to 0 </li></ul></ul><ul><ul><li>95 percent confidence interval: </li></ul></ul><ul><ul><li>-1.0180279 0.5380279 </li></ul></ul><ul><ul><li>sample estimates: </li></ul></ul><ul><ul><li>mean of the differences </li></ul></ul><ul><ul><li>-0.24 </li></ul></ul>
15. 15. Practice
16. 16. Extensions <ul><li>Some additional examples of paired samples are: </li></ul><ul><ul><li>Differences between left and right eye </li></ul></ul><ul><ul><li>Differences between dominant and recessive hand </li></ul></ul><ul><ul><li>Matched samples </li></ul></ul><ul><li>When you have more than two samples, techniques account for the correlation between the samples </li></ul><ul><ul><li>Multivariate / longitudinal data </li></ul></ul>
17. 17. Unpaired samples <ul><li>Often it is impractical to design study to use the same patients for both group </li></ul><ul><ul><li>Ex. Comparison of cholesterol in males and females </li></ul></ul><ul><ul><li>Ex. Time constraints </li></ul></ul><ul><li>Since the samples are not paired, we cannot use the difference between the individual samples </li></ul><ul><ul><li>Must adjust previous analysis </li></ul></ul>
18. 18. Example <ul><li>Another aspect of the tumor volume study was trying to compare the tumor volume among patients with different forms of cancer. The average tumor size is important to know the effect of treatment can be determined. </li></ul><ul><li>In this study, patients with brain, breast and liver tumors, but initially we will only compare the brain and breast cancers. </li></ul><ul><li>All of the tumors were measured using the RECIST method </li></ul>
19. 19. Null hypothesis <ul><li>The null hypothesis is that there is no difference between the volume of the tumor in the two forms of cancer </li></ul><ul><li>H 0 :  brain  =  breast  , or  brain –  breast =0 </li></ul><ul><li>More generally, we can test if the difference between two groups is a specific value,  1 -  2 =  </li></ul><ul><ul><li>This occurs when comparing two treatment groups and we are interested if the two groups are different by a specific amount </li></ul></ul>
20. 20. <ul><li>Each patient contributes one observation </li></ul><ul><li>Can estimate from the sample </li></ul><ul><ul><li>Mean and standard deviation in brain cancer group </li></ul></ul><ul><ul><li>with </li></ul></ul><ul><ul><li>Mean and standard deviation in breast cancer group </li></ul></ul><ul><ul><li>with </li></ul></ul><ul><li>Are the two groups the same? </li></ul><ul><ul><li>H 0 :  1 =  2 , or  1 -  2 =0 </li></ul></ul><ul><ul><li>To determine this, we are going to look at </li></ul></ul><ul><ul><li>We also need to know </li></ul></ul>
21. 21. Difference in the sample means <ul><li>We are going to use the difference of the means as our test statistic, but we need to estimate the variance of this difference to determine if the difference is significant </li></ul><ul><li>Basic form of test statistic: </li></ul><ul><ul><li>Standard deviations known unknown </li></ul></ul><ul><li>The estimate of the standard deviation changes when </li></ul><ul><ul><li>The samples have equal variance OR </li></ul></ul><ul><ul><li>The samples have unequal variance </li></ul></ul>
22. 22. Equal variance <ul><li>Sometimes we will be willing to assume that the variance in the two groups is equal: </li></ul><ul><li>If we know this variance, we can use the z-statistic </li></ul><ul><li>Often we have to estimate   with the sample variance from each of the samples, </li></ul><ul><li>Since we have two estimates of one quantity we pool the two estimates </li></ul>
23. 23. Equal variance continued <ul><li>The estimate of  is given by: </li></ul><ul><li>The t-statistic based on the pooled variance is very similar to the z-statistic as always: </li></ul><ul><li>The t-statistic has a t-distribution with </li></ul><ul><li>degrees of freedom </li></ul>
24. 24. <ul><li>For the tumor volume study, there were 20 brain cancer subjects and 28 breast cancer subjects </li></ul><ul><li>The summary statistics and histogram for the data are given here </li></ul><ul><li>What can you say about the distributions? </li></ul><ul><li>Does the equal variance assumption seem valid in this case? </li></ul>6.0 3.49 s 2 17.5 cm 3 16.2 cm 3 xbar 28 20 n Breast Brain
25. 25. Hypothesis test <ul><li>Two independent samples with equal variance; alpha = 0.05 </li></ul><ul><li>H 0 : mean brain tumor size = mean breast tumor size </li></ul><ul><li>p-value: 0.046 </li></ul><ul><li>Reject null hypothesis </li></ul><ul><li>Conclusion: There is a significant difference in the size of brain and breast cancer tumors </li></ul>
26. 26. R code <ul><li>If we only had the test statistics above, we can calculate the test statistic and then compare it to the t-distribution using </li></ul><ul><li>pt(-2.054 ,df=46) </li></ul><ul><li>to determine the area in the lower tail </li></ul><ul><li>How do we convert this into the appropriate p-value? </li></ul><ul><li>With the full data, we can use </li></ul><ul><li>data<-read.table(“cancer.dat”,header=T) </li></ul><ul><li>gr<-data[,1]; size<-data[,2] </li></ul><ul><li>t.test(size[(gr==0)], size[(gr==1)], var.equal=T) </li></ul>
27. 27. R output <ul><li>Two Sample t-test </li></ul><ul><li>data: size[(gr == 0)] and size[(gr == 1)] </li></ul><ul><li>t = -2.054, df = 46, p-value = 0.04568 </li></ul><ul><li>alternative hypothesis: true difference in means is not equal to 0 </li></ul><ul><li>95 percent confidence interval: </li></ul><ul><li>-2.65174438 -0.02682705 </li></ul><ul><li>sample estimates: </li></ul><ul><li>mean of x mean of y </li></ul><ul><li>16.15000 17.48929 </li></ul>
28. 28. Unequal variance <ul><li>Often, we are unwilling to assume that the variances are equal </li></ul><ul><li>We now write the test statistic as: </li></ul><ul><li>The distribution of this statistic is difficult to derive and we approximate the distribution using a t-distribution with  degrees of freedom </li></ul>
29. 29. <ul><li>This is called the Satterthwaite or Welch approximation </li></ul><ul><ul><li>When you complete a two-sample t-test in R and the variances are not assumed equal, this approximation is used </li></ul></ul>
30. 30. Example <ul><li>For the comparison of the brain cancers to the liver cancers, the variances are much more different. </li></ul><ul><li>Let’s use the unequal variance two sample t-test in this case </li></ul>14.4 3.49 s 2 19.35 cm 3 16.2 cm 3 xbar 20 n Liver Brain
31. 31. Example <ul><li>Two independent samples with equal variance; alpha = 0.05 </li></ul><ul><li>H 0 : mean brain tumor size = mean liver tumor size </li></ul><ul><li>p-value: 0.0044 </li></ul><ul><li>Reject null hypothesis </li></ul><ul><li>Conclusion: There is a significant difference in the size of the brain and liver tumor size </li></ul>
32. 32. R output <ul><li>> t.test(size[(gr==0)],size[(gr==2)]) </li></ul><ul><li>Welch Two Sample t-test </li></ul><ul><li>data: size[(gr == 0)] and size[(gr == 2)] </li></ul><ul><li>t = -3.1666, df = 22.48, p-value = 0.00439 </li></ul><ul><li>alternative hypothesis: true difference in means is not equal to 0 </li></ul><ul><li>95 percent confidence interval: </li></ul><ul><li>-5.288291 -1.105827 </li></ul><ul><li>sample estimates: </li></ul><ul><li>mean of x mean of y </li></ul><ul><li>16.15000 19.34706 </li></ul>
33. 33. Practice <ul><li>Get the dataset from the course folder </li></ul><ul><li>We want to compare the </li></ul>
34. 34. Can we test if the variances are equal? <ul><li>Since we can never be sure if the variances are equal, could we test if they are equal? </li></ul><ul><li>Of course we can!!! </li></ul><ul><ul><li>But, remember there is error in every statistical test </li></ul></ul><ul><ul><li>Sometimes it is just preferred to use the unequal variance unless there is a good reason </li></ul></ul>
35. 35. Equality of variance <ul><li>H 0 :          </li></ul><ul><li>To test this hypothesis, we use the sample variances: </li></ul><ul><li>If one of the variances is much larger than the other, this is evidence against the null </li></ul><ul><li>As we discussed a couple classes ago: </li></ul>
36. 36. Test of equality <ul><li>One way to test if the two variances are equal is to check if the ratio is equal to 1 </li></ul><ul><li>Under the null, the ratio simplifies to </li></ul><ul><li>The ratio of 2 chi-square random variables has an F-distribution </li></ul><ul><li>The F-distribution is defined by the numerator and denominator degrees of freedom </li></ul><ul><li>Here we have an F-distribution with n 1 -1 and n 2 -1 degrees of freedom </li></ul><ul><li>This works better with </li></ul>
37. 37. F-distribution <ul><li>Here is the F-distribution with 5 and 500 degrees of freedom </li></ul><ul><li>Note the skew of the distribution </li></ul>
38. 38. Example <ul><li>> var.test(size[(gr==1)],size[(gr==0)]) </li></ul><ul><li>F test to compare two variances </li></ul><ul><li>data: size[(gr == 1)] and size[(gr == 0)] </li></ul><ul><li>F = 1.719, num df = 27, denom df = 19, p-value = 0.2247 </li></ul><ul><li>alternative hypothesis: true ratio of variances is not equal to 1 </li></ul><ul><li>95 percent confidence interval: </li></ul><ul><li>0.710335 3.904512 </li></ul><ul><li>sample estimates: </li></ul><ul><li>ratio of variances </li></ul><ul><li>1.719033 </li></ul><ul><li>> var.test(size[(gr==2)],size[(gr==0)]) </li></ul><ul><li>F test to compare two variances </li></ul><ul><li>data: size[(gr == 2)] and size[(gr == 0)] </li></ul><ul><li>F = 4.1182, num df = 16, denom df = 19, p-value = 0.004156 </li></ul><ul><li>alternative hypothesis: true ratio of variances is not equal to 1 </li></ul><ul><li>95 percent confidence interval: </li></ul><ul><li>1.589643 11.111060 </li></ul><ul><li>sample estimates: </li></ul><ul><li>ratio of variances </li></ul><ul><li>4.118214 </li></ul>
39. 39. Power and sample size <ul><li>As with the one sample case, we can find power and sample size for a two sample problem </li></ul><ul><li>For two dependent samples, the power and sample size can be calculated exactly as in the one sample case because the paired t-test is a one sample problem </li></ul><ul><li>For two independent samples, the power and sample size is slightly different </li></ul>
40. 40. One sample case (review) <ul><li>To find the sample size in the one sample case we needed </li></ul><ul><ul><li>The hypothesized difference in the means </li></ul></ul><ul><ul><li>The alpha level </li></ul></ul><ul><ul><li>The power </li></ul></ul><ul><ul><li>The variance in the sample </li></ul></ul><ul><ul><li>One-sided or two sided test </li></ul></ul>
41. 41. Two sample case <ul><li>We still need to have the following pieces of information </li></ul>