Bajaj Allianz Life Insurance Company - Insurer Innovation Award 2024
Testing differences between means_The Basics
1. K E V I N B E R N H A R D T
T R O Y B U C K N E R
B R I A N G A L V I N
Testing Differences Between
Means: The Basics
2. Tests to use for comparing means
When comparing…
Note: ANOVA/Multiple Regression can be used with only 2 means –
personal preference.
Number of means Type of test
2 means t-test
3 or more means ANOVA/Multiple Regression
3. H0 and Ha
There is a better way of comparing means other than
saying, “They’re 1.5 SD apart. That’s quite a
difference.”
H0: Null hypothesis. What we are planning to reject.
Ha: Alternative/Research hypothesis. Defendable
statement based off data presented.
Mutually exclusive – both cannot be true. Reject one,
regard other as tenable.
4. Comparing means....
Alternative use of z-score equation: z = (x-x_bar)/σ
Goal: Compare between 2 means…..
“What’s the likelihood that you would obtain a sample
average of X if the population average is x_bar?”
NOTE: “Population” does not always mean statistical population.
Variable Previous use New use
x individual observation Sample mean
x_bar sample mean Population mean (...of
sample means)
σ population SD (or sample) Standard error of means
5. Standard Error
Standard error of mean can be estimated with the
following equation:
Sample SD (σ preferred)
Sample size
Courtesy of www.discover6sigma.org
Standard Error = manipulated observations
6. Interpreting Standard Error of the Mean
In terms of σ, we can accept H0.
Attribute 0.5 SD to sampling error.
Except....
7. Interpreting Standard Error of the Mean (cont.)
Standard deviation is of individual observations.
Comparison is between means.
Therefore SE is used.
Standard deviation > Standard Error
Affects distribution, not means.
Net Effect: Sample mean is further from population mean (2
SE), therefore we cannot accept H0 immediately.
8. Setting the alpha level....
p: Probability of mistakenly rejecting H0.
With p, you are saying that you are willing to make this
mistake 5% of the time (p = 0.05)
Calculating the probability of obtaining a given
sample mean:
NORM.DIST(value, mean, standard deviation, cumulative)
TRUE: total area to the left of “value” (aka the sample mean)
FALSE: probability that “value will occur
10. Using the t-Test vs. z-Test
Defining the decision rule
Null Hypothesis vs. Alternative Hypothesis
Both cannot be true
Alpha – error rate you have adopted
Normally 5%
Critical value is the criterion associated with the
error rate
11. Finding Critical Value for a z-Test
NORM.INV(area we’re interested in under the curve
that represents the distibution, mean of the
distribution, standard error of the mean)
12. Finding Critical Value for a t-Test
Used when you don’t know the population standard
deviation.
T.INV (probability you’re interested in, degrees of
freedom)
13. Comparing Critical Values
t-Test has slightly less statistical power than the z-
Test, because critical value is farther from the mean
due to thicker tails.
14. Statistical Power
When the mean is below (or above) the critical value,
then the null hypothesis is false and the alternative
hypothesis is therefore true.
Statistical power depends on the position of the
alternative hypothesis curve.
15. Beta
Beta = 1 – power.
If we would accept a true hypothesis 60% of the time
(power), then beta is 1 - .60 = 40%.
16. t-Test vs z-Test?
Use t-Test when the sample size is under 30, and z-
Test when it is over 30+
