A sample size that is too small increases the risks of overlooking important effects and detecting effects that are not truly present. With a larger sample size, the risks decrease but costs and time increase. The key factors in determining sample size are the desired power, significance level, expected effect size, and standard deviation. Sample size calculators can then determine the necessary sample for a given hypothesis test based on specifying values for these factors.

1. Sample SizeSample Size
Al h Ri kAlpha Risk
Test powerp
Delta
Cost
TimeTime
Resources
Week 3
Knorr-Bremse Group
Content
• What sample size do we need?What sample size do we need?
• How can the sample size influence our decision?
• How certain (with which power) do we interpret factor( p ) p
effects?
• How do we avoid misinterpretation of small effects
( l h i k)?(alpha risk)?
Knorr-Bremse Group 08 BB W3 sample size 08, D. Szemkus/H. Winkler Page 2/16

2. Sample Size and Risk
With ll l iWith a small sample size:
– the costs are lower
– the study is done faster
– the risk increases:
• to oversee an important effect
• to address a meaning to an effect which is in reality not importantto address a meaning to an effect which is in reality not important
– the confidence intervals increase and the determination of true
effects become more uncertain
With a large sample size:With a large sample size:
– cost and time will increase
h i k ll (b ill i )– the risks get smaller (but still exist)
• smaller effects can be detected easier (increased significance)
Knorr-Bremse Group 08 BB W3 sample size 08, D. Szemkus/H. Winkler Page 3/16
– the confidence intervals become smaller
Risks at the Hypothesis Testing
E i t l Th T th
To oversee an
important effect
C ?Experimental
Decision
The Truth
Ho is true Ha is true
Consequences?
Ho is true Ha is true
Accept H
Type 2 Error
Assumption
Accept Ho
β-risk
Negative error
Assumption
Reject H and
Type 1 Error
α riskReject Ho and
accept Ha
α-risk
Positive error
Detect something which
is actual not present
Consequences?
Knorr-Bremse Group 08 BB W3 sample size 08, D. Szemkus/H. Winkler Page 4/16
Consequences?

3. Sample Size Parameter
• n (sample size): number of samples that we need for each group. The
application is different depending on the situation, for example:
– For a 2k-p fractional DOE n is the number of runs
– For a two-sample t - test, n is the number of observations within the group
• α (alpha): the chance of an error type 1. The level of significance is
described by the p-value. (Probability of error usually 0,05)
• Power: the chance to detect a real effect. We want a high probability,
corresponding to 1- β (usually 0,9),
• β (beta): the chance for an error type 2 (usually 0,1)
• δ (delta): the size of the effect we want to detect, often expressed in units
of σ.
• σ (sigma): the evaluated standard deviation at constant factor setting
• Are 3 of these 4 items known (n, α, Power, and δ/σ), the missing one can
Knorr-Bremse Group 08 BB W3 sample size 08, D. Szemkus/H. Winkler Page 5/16
be calculated.
Some Examples
C f fComparison of 2 defect proportions:
You want to compare the defect rate (blow-out) of your truck tires with
th tit Thi i th l b h k tl Y hthe competitor. This is the only benchmark currently. You measure how
many tires have a blow-out on the road.
The blow-out rate of the competitor is about 1 5% (0 015)The blow-out rate of the competitor is about 1,5% (0,015).
You want to have 90% certainty (power) at minimum to detect a
significant difference at α = 0,05. Your product has a defect rate of 1,0significant difference at α 0,05. Your product has a defect rate of 1,0
% (0,010). The delta δ = 0,005.
For this example you need a sample size of n = 10374 per type of tire. A
study makes no sense if the required sample can not be investigated.
DOE:
You plan a 23 factorial DOE with replicates.
With an α = 0 05 you want detect an effect for the yield with a powerWith an α 0,05 you want detect an effect for the yield with a power
(certainty) of 80%. The difference you want to achieve is min 2% (delta).
The variation from the measurement phase is known, σ = 1.5%.
Knorr-Bremse Group 08 BB W3 sample size 08, D. Szemkus/H. Winkler Page 6/16
2 replicates (n = 16 runs) results in 64.8% power, 3 (n = 24) in 86.5%.

4. Comparison of 2 Groups (t-test)
The formula for calculating the sample size n for each group is:
F )(2 2
2/ βZZ +For a
2-sided test: ( )/
)(2
2
2/
σδ
βα ZZ
N
+
= .025
( )
-3 -2 -1 0 1 2 3
025.z
For a )(2 2
βα ZZ
N
+
=
025.
Z.025 = 1.960
Z 1 645
1-sided test:
( )/ 2
σδ
N = Z.05 = 1.645
Z.10 = 1.282
Z = 0 842Z.20 = 0.842
Reminder: β = 1- power
Knorr-Bremse Group 08 BB W3 sample size 08, D. Szemkus/H. Winkler Page 7/16
Reminder: β 1 power
Possibilities in Minitab
Minitab is able to calculate the sample size and power for
the following cases:the following cases:
– 1-sample Z (at known standard deviation)
1 l t– 1-sample t
– 2-sample t
– 1 Proportion
– 2 Proportions (like the tire example)2 Proportions (like the tire example)
– One-way ANOVA
2 l l f t i l D i (i l i f ti l f t i l)– 2-level factorial Design (inclusive fractional factorial)
– Plackett-Burman Design (special 2-level designs)
Stat
>Power and Sample Size
Knorr-Bremse Group 08 BB W3 sample size 08, D. Szemkus/H. Winkler Page 8/16
>Power and Sample Size

5. Example for Sample Size Calculation
Lets assume:
– We want to check if the handling time of customer complaints
at 2 similar companies are identical We plan an ANOVA or aat 2 similar companies are identical. We plan an ANOVA or a
2 sample t-test.
– Our best estimation for an expected mean is 15 days with a
standard deviation of σ = 2 days.
– The sample size should be large enough to discover a
difference of 2 days with a certainty of 95%.
– We are willing to take an alpha risk of 0,05.
– What sample size do you recommend?
Knorr-Bremse Group 08 BB W3 sample size 08, D. Szemkus/H. Winkler Page 9/16
Calculation of the Sample Size
To calculate the sample size we need the following
information:
α = 0.05
β = 0.05 (certainty of 95%)
δ/σ = 2 days / 2 days = 1y y
Test: ANOVA or 2 sample t-test
)(2 2
2/ βα ZZ
N
+
=
( ) 26
1.6451.962
2
≈
+×
=
( )/ 2
σδ
N =
( )
26
1
2
≈=
Lets use Minitab for this example to get an exact result:
Knorr-Bremse Group 08 BB W3 sample size 08, D. Szemkus/H. Winkler Page 10/16

6. Calculation of the Sample Size in Minitab
Stat
>Power and Sample Size
P d S l Si
p
>2-Sample t…
Power and Sample Size
2 S l t T t2-Sample t Test
Testing mean 1 = mean 2 (versus not =)
Calculating power for mean 1 = mean 2 + difference
Alpha = 0,05 Assumed standard deviation = 2
Sample Target
Difference Size Power Actual PowerDifference Size Power Actual Power
2 27 0,95 0,950077
The sample size is for each group
Knorr-Bremse Group 08 BB W3 sample size 08, D. Szemkus/H. Winkler Page 11/16
The sample size is for each group.
Calculation of the Sample Size in Minitab
Stat
>Power and Sample Size
The power curve shows the power
d di d lt (th diff f t t)
p
>2-Sample t…
depending on delta (the difference for test)
1,0
Sample
Power Curve for 2-Sample t Test
0,8
A lpha 0,05
StDev 2
A lternativ e Not =
A ssumptions
27
Size
p
0,6
0,4
Power
A lternativ e Not =
0,2
210-1-2
0,0
Difference
Knorr-Bremse Group 08 BB W3 sample size 08, D. Szemkus/H. Winkler Page 12/16

7. Sample Size Table
δ/σ 20% 10% 5% 1% β 20% 10% 5% 1% β 20% 10% 5% 1% β 20% 10% 5% 1% β
0,2 225 328 428 651 309 428 541 789 392 525 650 919 584 744 891 1202
α = 20% α = 10% α = 5% α = 1%
0,3 100 146 190 289 137 190 241 350 174 234 289 408 260 331 396 534
0,4 56 82 107 163 77 107 135 197 98 131 162 230 146 186 223 300
0,5 36 53 69 104 49 69 87 126 63 84 104 147 93 119 143 192
0,6 25 36 48 72 34 48 60 88 44 58 72 102 65 83 99 134
0,7 18 27 35 53 25 35 44 64 32 43 53 75 48 61 73 980,7 18 27 35 53 25 35 44 64 32 43 53 75 48 61 73 98
0,8 14 21 27 41 19 27 34 49 25 33 41 57 36 46 56 75
0,9 11 16 21 32 15 21 27 39 19 26 32 45 29 37 44 59
1,0 9 13 17 26 12 17 22 32 16 21 26 37 23 30 36 48
1,1 7 11 14 22 10 14 18 26 13 17 21 30 19 25 29 40
1 2 6 9 12 18 9 12 15 22 11 15 18 26 16 21 25 331,2 6 9 12 18 9 12 15 22 11 15 18 26 16 21 25 33
1,3 5 8 10 15 7 10 13 19 9 12 15 22 14 18 21 28
1,4 5 7 9 13 6 9 11 16 8 11 13 19 12 15 18 25
1,5 4 6 8 12 5 8 10 14 7 9 12 16 10 13 16 21
1,6 4 5 7 10 5 7 8 12 6 8 10 14 9 12 14 19
1,7 3 5 6 9 4 6 7 11 5 7 9 13 8 10 12 17
1,8 3 4 5 8 4 5 7 10 5 6 8 11 7 9 11 15
1,9 2 4 5 7 3 5 6 9 4 6 7 10 6 8 10 13
2,0 2 3 4 7 3 4 5 8 4 5 6 9 6 7 9 12
2 1 2 3 4 6 3 4 5 7 4 5 6 8 5 7 8 112,1 2 3 4 6 3 4 5 7 4 5 6 8 5 7 8 11
2,2 2 3 4 5 3 4 4 7 3 4 5 8 5 6 7 10
2,3 2 2 3 5 2 3 4 6 3 4 5 7 4 6 7 9
2,4 2 2 3 5 2 3 4 5 3 4 5 6 4 5 6 8
2,5 1 2 3 4 2 3 3 5 3 3 4 6 4 5 6 8
2,6 1 2 3 4 2 3 3 5 2 3 4 5 3 4 5 7
2,7 1 2 2 4 2 2 3 4 2 3 4 5 3 4 5 7
2,8 1 2 2 3 2 2 3 4 2 3 3 5 3 4 5 6
2,9 1 2 2 3 1 2 3 4 2 2 3 4 3 4 4 6
3 0 1 1 2 3 1 2 2 4 2 2 3 4 3 3 4 5
Knorr-Bremse Group 08 BB W3 sample size 08, D. Szemkus/H. Winkler Page 13/16
3,0 1 1 2 3 1 2 2 4 2 2 3 4 3 3 4 5
Sample Size for a paired t-test
NOTE: The Paired T-Test is equivalent to performing a One-Sample T-Test
on the paired differences!
1 Calculate standard deviation of the differences:
MAT A MAT B Boys Difference
13,2 14 1 -0,8
8 2 8 8 2 -0 6
Example with data from shoes.mtw:
1. Calculate standard deviation of the differences:
s = 0,38715 (If you know σ, run the 1-sample Z-Test)
2. Define parameters for calculation of sample size:
α power δ (Effect you want to detect e g 0 2)
8,2 8,8 2 -0,6
10,9 11,2 3 -0,3
14,3 14,2 4 0,1
10,7 11,8 5 -1,1
6,6 6,4 6 0,2
9 5 9 8 7 0 3α, power, δ (Effect you want to detect, e.g. 0.2)
3. Run the Minitab function Stat
>Power and Sample Size
9,5 9,8 7 -0,3
10,8 11,3 8 -0,5
8,8 9,3 9 -0,5
13,3 13,6 10 -0,3
Power and Sample Size
4. Fill in the data
p
>1-Sample t… 5. Evaluate session window
1-Sample t Test
Testing mean = null (versus not = null)
C l l ti f ll + diffCalculating power for mean = null + difference
Alpha = 0,05 Assumed standard deviation = 0,38715
Sample Target
Difference Size Power Actual Power
What is the
Knorr-Bremse Group 08 BB W3 sample size 08, D. Szemkus/H. Winkler Page 14/16
Difference Size Power Actual Power
0,2 42 0,9 0,904659 conclusion?

8. Exercise for Sample Sizes
Case α Power δ n
1 constant constant
2 constant constant
?
?2 constant constant
3 constant constant
?
?
4 constant constant ?
5 constant constant
6 constant
?
?6 constant
Whi h h ( d ) i t d b d th d ?
?
Which change (up or down) is expected based on the red arrow?
Case 1 for example: if alpha and power are constant and delta gets smaller
what is effect on the sample size n?
Knorr-Bremse Group 08 BB W3 sample size 08, D. Szemkus/H. Winkler Page 15/16
what is effect on the sample size n?
Summary
• The sample size depends on your needs:
• What is your experimental goal?
• What risk to you want to take?• What risk to you want to take?
• What difference you want to detect?
• What power should your result have?
• Before collecting data think about the sample size. Your results
should be powerful and your conclusions should be correct.
• With the right sample size we can better control our activities and
we can better avoid wrong decisions.
• The sample size is also determined by practical and economical
issues.
Knorr-Bremse Group 08 BB W3 sample size 08, D. Szemkus/H. Winkler Page 16/16