Independent samples t-test

THE T-TEST FOR
TWO INDEPENDENT
SAMPLES

Katona’s (1940)
Comparison of
Two Methods of
Instruction
• Problem:
• Change the pattern into exactly four
squares by moving only three matchsticks
• Method:
• One group was taught how to do it
• One group was told to figure it out themselves
• After 3 weeks, both were given a new, novel problem
to solve
• Outcome:
• The “discovery method” group performed better on the
new problem
How do we explain the difference between
the two groups?
Which method
teaches how to
solve novel
problems more
effectively:
 The expository
method (learning by
memorization)
 The discovery
method (learning by
understanding)

Two Possible Explanations
1. The method of understanding (the Discovery Method)
produces better learning than the method of
memorization (the Expository Method)
 The difference between groups can be attributed to the
treatment
OR
2. Sampling error produced the difference between groups
 The treatments had no effect on learning
So how do we determine which explanation is correct?
A HYPOTHESIS TEST!

But wait… now we have TWO samples

THE INDEPENDENT-
MEASURES DESIGN

Independent-Measures Designs
A research design that uses a separate group of participants
for each treatment condition (or for each population)
• a.k.a. Between-Subjects Design
When we are comparing two (or more)
sets of data
• Comparing political parties on their levels of “caring”
• Comparing two different teaching methods
for teaching one subject
“Do these two samples come from one same population, or
two different populations?”

How do we obtain two sets of data?
Independent-Measures
(Between Subjects) Design
From two completely separate
groups of participants:
A separate sample of
individuals is used for
each treatment condition
(or population)
e.g., Men vs. Women,
Teaching Strategy A vs. B
From the same group of
participants:
A single sample of
individuals is measured
more than once on the DV
Obtain pre- and post-
intervention scores
e.g., Depression level before and
after therapy
Repeated-Measures
(Within Subjects) Design

Independent-Measures Designs
• Used in situations where there is no prior knowledge about
either of the two populations (or treatments) being
compared
• Population parameters (μ, σ)
are all unknown
• Variance must be estimated
from the sample data
• Are Democrats and
Republicans from the same
population of people?

CHAPTER 10.2
The t Statistic for an Independent-Measures
Research Design

Independent-Measures t-test
A comparison between two groups of separate individuals
• The question you are asking is:
Do these two samples come from one same population,
or two different populations?
OR
Are the two population means the same or different?
mpopulation1 = mpopulation2 ?
*Subscripts tell us which group the information belongs to:
   212121 SSSSMMnn 

The Hypotheses (two-tailed)
If μ1 is the mean of population 1
And μ2 is the mean of population 2
And we are interested in determining whether or not
there is a difference between the two
Then our hypotheses would be:
Null: There is no difference between the population means
Alternative: There is a difference between the population means
H0 :m1 -m2 = 0
H1 :m1 -m2 ¹ 0 H1 :m1 ¹ m2
or, simply

The t Statistics
Single-sample t statistic
Formula
Numerator Actual difference between
the sample mean and the
hypothesized population
mean
Denominator Amount of expected error
(estimated amount of error expected
when we use a sample mean to
represent a population mean)
t =
M -m
sM
Independent-measures t statistic
Actual difference between the two
sample means and the
hypothesized difference between
the population means
Amount of expected error
(estimated amount of error expected when you use a
sample mean difference to represent a population
mean difference)
 21
)()( 2121
MMs
MM
t





The Independent-Measures t Formula
 21
)()( 2121
MMs
MM
t




Difference between
the two sample
means
(obtained from
sample data)
Difference between
the two population
means
(obtained from H0)
The estimated
standard error of the
difference in means,
NOT the standard
error of M1 minus M2
t =
actual difference between M1 and M2
standard difference (If H0 is true) between M1 and M2
Measures the
amount of error that
is expected when
you use a sample
mean difference
(M1 – M2) to predict
a population mean
difference (μ1 - μ2)

Calculating the Estimated Standard Error
• Each of the two sample means represent their respective
population means
• M1 approximates μ1 with some error
• M2 approximates μ2 with some error
• Therefore we have two sources of error
• The estimated standard error for each mean can be
individually calculated
• Calculate sM for M1
• Calculate sM for M2
• Combine the individual errors to find the total amount of
error for the two samples
 21 MMs 
sM1
=
s1
n1
=
s1
2
n1
1M - 2M( )s = 1
2
s
1n
+ 2
2
s
2n
= 1
2
s
1n
+ 2
2
s
2n
sM2
=
s2
n2
=
s2
2
n2

The Estimated Standard Error
This formula can only be used
when sample sizes are equal
(when n is the same for both samples)
So what do we do when the sample sizes are NOT equal?
• This formula treats the two sample variances equally
• Remember the law of large numbers:
• statistics obtained from large samples tend to be more accurate than
statistics obtained from smaller samples
1M - 2M( )s = 1
2
s
1n
+ 2
2
s
2n

Used when samples are of unequal size
• Pooled variance is actually an average of the two sample
variances computed so that the larger sample (with the
higher df) carries more weight in determining the final
value.
Pooled Variance
21
212
dfdf
SSSS
sp



1M - 2M( )s = p
2
s
1n
+ p
2
s
2n

A Demonstration
Equal Sample Sizes
Sample 1
n = 6, SS = 50
Unequal Sample Sizes
=
50+30
5+5
=
80
10
= 8sp
2
=
SS1 + SS2
df1 + df2
s2
=
SS
df
=
30
5
= 6s2
=
SS
df
=
50
5
=10
Sample 2
n = 6, SS = 30
Individual variances:
for sample 1: for sample 2:
Pooled variance:
Sample 1
n = 3, SS = 20
=
20+ 48
2+8
=
68
10
= 6.8sp
2
=
SS1 + SS2
df1 + df2
s2
=
SS
df
=
48
8
= 6s2
=
SS
df
=
20
2
=10
Sample 2
n = 9, SS = 48
Individual variances:
for sample 1: for sample 2:
Pooled variance:

• Provides an unbiased measure of the standard error for a
sample mean difference
• Measures how accurately the difference between two
sample means represents the difference between the two
population means
The Estimated Standard Error Using
Pooled Variance
1M - 2M( )s = p
2
s
1n
+ p
2
s
2n

df for the Independent-Sample t Statistic
• Determined by the df values for the two separate
samples:
df for the first sample + df for the second sample
df1 +df2
)1()1( 21  nn

The Independent-Measures t Statistic Formula
  )(
212121
2121
)()(
MMMM s
MM
s
MM
t






t =
data-hypothesis
error

CHAPTER 10.3
Hypothesis Tests and Effect Size
with the Independent-Measures t Statistic

Test Procedures
1. State the hypotheses and select the α level
(no difference) α = .01
(there is a difference)
2. Locate the critical region by calculating df and then using
the t Distribution Table in your textbook (p.703)
2. Compute the test statistic
2. Make a decision
H0 :m1 -m2 = 0
H1 :m1 -m2 ¹ 0
df = df1 +df2
 21
)()( 2121
MMs
MM
t





tcrittcrit
Two-Tailed Tests: “Difference” Hypotheses
• If the group 1 should have a different mean than group 2
• Example: If you are testing whether room temperature (IV – the
treatment) changes test performance (DV), then you expect the first
group to have a different mean score than the treated group, but
the direction is uncertain
• You have two critical values (a negative
and positive value), and your calculated
t statistic must fall in one of the tails
(the critical region) of the distribution
H0 :m1 -m2 = 0
H1 :m1 -m2 ¹ 0
or
H0 :m1 = m2
H1 :m1 ¹ m2

tcrit
One-Tailed Tests: Directional Hypotheses
• If the group 1 should have a higher mean than group 2
• Example: If you are testing whether a therapy (IV – the treatment)
lowers depression (DV), then you should expect the first group to
have a higher depression score than the treated group
• You have one critical value (a positive
value), and your calculated t statistic
must fall in the tail (the critical region)
of the distribution
211
210
:
:




H
H

tcrit
One-Tailed Tests: Directional Hypotheses
• If the group 1 should have a lower mean than group 2
• Example: If you are testing whether special instruction (IV – the
treatment) increases SAT scores (DV), then you should expect the
first group to have a lower mean SAT score than the “treated”
group
• You have one critical value (a negative
value), and your calculated t statistic
must fall in the tail (the critical region)
of the distribution
H0 :m1 ³ m2
H1 :m1 < m2

Measuring Effect Size
Cohen’s d
• Produces a standardized measure of mean difference
• numerator: observed difference between the means
• denominator: difference expected to occur by chance
r2
• measures how much of the variability in scores can be
explained by the treatment effects
Confidence Intervals
• estimates the size of the population mean difference
between the two populations or treatment conditions
d =
M1 - M2
sp
2
dft
t
r

 2
2
2
m1 -m2 = M1-M2 ±t(sM1-M2
)

Example 10.1
Average High School Grade
Watched
Sesame Street
Did Not Watch
Sesame Street
86 99 90 79
87 97 89 83
91 94 82 86
97 89 83 81
98 92 85 92
n = 10 n = 10
M = 93 M = 85
SS = 200 SS = 160
A researcher is interested in
determining whether or not
TV viewing habits at age 5
are related to academic
performance in high school
• Obtained HS grades for:
• n = 10 students with a history
of watching Sesame Street
• n = 10 students who did not
watch Sesame Street
Research question:
Is there a significant difference between the two types of high school
students?

Independent Samples t-Test: Step 1
State the hypotheses and select the α level
There is no difference in mean grades between the two
types of high school students
There is a difference in mean grades between the two
types of high school students
We set α = .01
H0 :m1 -m2 = 0
H1 :m1 -m2 ¹ 0

Locate the critical region
df
Watched
Sesame Street
Did Not Watch
Sesame Street
86 99 90 79
87 97 89 83
91 94 82 86
97 89 83 81
98 92 85 92
n = 10 n = 10
M = 93 M = 85
SS = 200 SS = 160
= df1 +df2
= (n1-1)+(n2 -1)
= (10-1)+(10-1)
= (9)+(9) =18

Compute the test statistic
Think of this computation as having
3 steps:
1. Find the pooled variance
t
Watched
Sesame Street
Did Not Watch
Sesame Street
86 99 90 79
87 97 89 83
91 94 82 86
97 89 83 81
98 92 85 92
n = 10 n = 10
M = 93 M = 85
SS = 200 SS = 160
=
(M1 - M2 )-(m1 -m2 )
s M1-M2( )
21
212
dfdf
SSSS
sp



=
200+160
9+9
=
360
18
= 20

3 steps:
2. Use the pooled variance to compute
the estimated standard error
t
Watched
Sesame Street
Did Not Watch
Sesame Street
86 99 90 79
87 97 89 83
91 94 82 86
97 89 83 81
98 92 85 92
n = 10 n = 10
M = 93 M = 85
SS = 200 SS = 160
=
(M1 - M2 )-(m1 -m2 )
s M1-M2( )
=
20
10
+
20
10
1M - 2M( )s = p
2
s
1n
+ p
2
s
2n
= 2+2 = 4 = 2
sp
2
= 20

3 steps:
2. Use the pooled variance to compute
the estimated standard error
3. Compute the t statistic
t
Watched
Sesame Street
Did Not Watch
Sesame Street
86 99 90 79
87 97 89 83
91 94 82 86
97 89 83 81
98 92 85 92
n = 10 n = 10
M = 93 M = 85
SS = 200 SS = 160
=
(M1 - M2 )-(m1 -m2 )
s M1-M2( )
sM1-M2
= 2
sp
2
= 20
t =
(93-85)-0
2
=
8-0
2
=
8
2
= 4

Make a decision
• Our obtained t = 4.00 is in the critical region
The obtained sample mean difference is 4 times greater than would
be expected if there were no difference between the two populations

Independent Sample t-Test: Effect Size
• Cohen’s d
• r2
47% of the variability in scores can be explained by the
treatment effects
d =
M1 - M2
sp
2
=
93-85
20
=
8
4.7
=1.79
r2
=
t2
t2
+ df
=
42
42
+18
=
16
16+18
=
16
34
= 0.47

Independent Sample t-Test: Effect Size
• Confidence Intervals
• Our confidence interval is:
Students who watched Sesame Street have higher grades than those
who did not, and the mean difference between the two populations is
somewhere between 3.798 points and 12.202 points.
m1 -m2 = M1-M2 ±t(sM1-M2
) = 93-85±2.101(2) =8±4.202
8-4.202 =3.798 to 8+4.202 =12.202

Reporting the Results
The students who watched Sesame Street as children had higher
high school grades (M = 93, SD = 4.71) than the students who did
not watch the program (M = 85, SD = 4.22). The mean difference
was significant, t(18) = 4.00, p < .01, d = 1.79.
• When using SPSS, you will have the exact probability (in this case
p = .001), which you would report instead of “p < .01”
t(18) = 4.00, p = .001, d = 1.79.
• If you are using confidence intervals to describe the effect size, then
you would state:
…The mean difference was significant, t(18) = 4.00, p = .001,
95% CI [3.798, 12.202]

CHAPTER 10.4
Assumptions Underlying the
Independent-Measures t Formula

Assumptions that should be satisfied
1. The observations within each sample must be
independent
2. The two populations from which the samples are
selected must be normal
• Or, you must have large samples
3. The two populations from which the samples are
selected must have equal variances
• Homogeneity of variance assumption
• Single-sample t and z-score: σ remains unchanged

Homogeneity of Variance
• Requires that the two populations from which the samples are
obtained have equal variances
• Pooled variance only makes sense if we are averaging two values
measuring similarly shaped populations
• If this assumption is violated, then the t statistic contains two
questionable values:
1.the value for the population mean difference (which comes from H0)
2.the value for the pooled variance
• We cannot determine which of these two values is responsible for a t
that falls into the critical region
• You cannot be certain that rejecting H0 is correct when you obtain an
extreme value for t

Hartley’s F-max Test
• Compute the sample variances for every sample
individually
• Compare with critical values in F-max table (p. 704)
• k = number of separate samples (for the independent t-test, k = 2)
• df = n -1 for each sample variance (Hartley assumes all samples
are the same size)
• α = .05 (we generally use this larger alpha level)
• You want a SMALLER value
• rejecting means violating your assumption
F-max =
slargest
2
ssmallest
2

If the Homogeneity of Variance
Assumption Is Violated
• The alternative procedure consists of two steps
• The standard error is computed using the two separate
sample variances
• The value of degrees of freedom for the t statistic is
adjusted using the following equation:
df =
(V1 +V2 )2
V1
2
n1 -1
+
V2
2
n2 -1
where V1 =
s1
2
n1
and V2 =
s2
2
n2
1M - 2M( )s = 1
2
s
1n
+ 2
2
s
2n

Independent samples t-test

More Related Content

What's hot

Similar to Independent samples t-test

More from Kaori Kubo Germano, PhD

Recently uploaded

In this document

Independent samples t-test