Comparing the unpaired t test, small sample formula with the one with large sample formula and the Welch's test. The latter is an unpaired t test for samples of unequal sizes with unequal variances.

1. The Unpaired t test family Guy Lion April 2010

2. The Unpaired t test family

3. Small Sample vs Large Sample Formula Even with a really small sample size of 10, using either formula gave us exactly the same p values.

4. Unpaired t test vs Welch’s test That’s our starting point, we will then stress the difference in sample size and standard deviation (variance) between Pilots and Controls.

5. Unpaired t test vs Welch’s test stressing differences in variance and sample size The multiples were both applied on the Controls . Notice how both effects are in the same direction. They both show that the t test slightly understates the P value relative to the Welch’s test. But, the effects counter each other. This is because the key metric is the Standard Error = Standard Deviation/SQRT(n).

6. Unpaired t test vs Welch’s test stressing differences in variance and sample size. 2 nd try In an attempt to stress the difference between the two test P values, we applied the sample multiple to the Controls and the Standard deviation multiple to the Pilots . It really did not do that much.