This document discusses determining appropriate sample sizes for quality control studies. It explains that sample sizes are determined based on acceptable levels of risk and variability in test results. Methods are presented for calculating sample sizes to estimate population means and standard deviations within defined confidence levels. Equations show how to determine the number of samples needed based on factors like the desired confidence interval, known standard deviation, and confidence level. Iterative calculations may be needed to precisely determine the sample size.

1. Basic Statistics for Quality Control and
Validation Studies: Session 2
• Steven S. Kuwahara, Ph.D.
• GXP BioTechnology, LLC
• PMB 506, 1669-2 Hollenbeck Ave.
• Sunnyvale, CA 94087-5402
• Tel. & FAX (408) 530-9338
• E-Mail: s.s.kuwahara@gmail.com
• Website: www.gxpbiotech.org
ValWkPHL1012S2 1

2. Sample Number Determination 1.
• One of the major difficulties with setting the
number of samples to take lies in determining the
levels of risk that are acceptable. It is in this area
that managerial inaction is often found, leaving a
QC supervisor or senior analyst to make the
decision on the level of risk the company will
accept. If this happens, management has failed its
responsibility.
ValWkPHL1012S2 2

3. Sample Number Determination 2.
• The problem is that all sampling plans, being statistical in
nature, will possess some risk. For instance, if we randomly
draw a new sample from a population we could assume or
predict that a test result from that sample will fall within
±3σ of the true average 99.7% of the time, but there is still
0.3% (3 parts-per-thousand) of the time when the result
will be outside the range for no reason other than random
error. Thus a good lot could be rejected. This is known as a
false positive or a Type I error.
• This is the type of error that is most commonly considered,
but there is type II error also.
ValWkPHL1012S2 3

4. Sample Number Determination 3.
• False positives occur when you declare that there is a
difference when one does not really exist (example given in
the previous slide). Sometimes called producer’s risk,
because the producer will dump a lot that was okay.
• False negatives occur when you declare that a difference
does not exist when, in fact, the difference does exist.
Sometimes called customer’s risk, because the customer
ends up with a defective product. It is also known as a
Type II error.
ValWkPHL1012S2 4

5. SIMPLIFIED FORM OF n CALCULATION
n for an ! to compare with a µ
xi − µ ⎛ s ⎞ = x − µ
t= t ⎜ ⎟ i
s ⎝ n ⎠
n
2 2 2 2
ts 2 ts
n
( )
= x−µ =Δ 2
n= 2
Δ
ValWkPHL1012S2 5

6. EXAMPLE OF SIMPLIFIED METHOD
WITH ITERATION
• Δ = 51- 50 = 1 s = ± 2 Z0.025=1.96
• n = (1.96)2 (2)2 / 1 = 3.8416 X 4 = 15.4 ~ 16
• t0.025,15= 2.131 (2.131)2 = 4.541161
• n = 4.54116 X 4 = 18.16 ~ 19
• t0.025,18= 2.101 (2.101)2 = 4.414201
• n = 4.414201 X 4 = 17.66 ~ 18
• t0.025,17= 2.110 (2.110)2 = 4.4521
• n = 4.4521 X 4 = 17.81 ~ 18
ValWkPHL1012S2 6

7. Sample Number Determination 6.
• Because of the need to define risk and consider the
level of variation that is present, sampling plans
that do not allow for these factors are not valid.
• Examples of these are: Take 10% of the lot below
N=200 and then 5% thereafter. The more famous
one is to take :
• in samples. N +1
ValWkPHL1012S2 7

8. DEVELOPMENT OF A SAMPLING PLAN
• Consider a situation where a product must contain
at least 42 mg/mL of a drug. At 41 mg/mL the
product fails. Because we want to allow for the test
and product variability, we decide that we want a
95% probability of accepting a lot that is at 42 mg/
mL, but we want only a 1% chance of accepting a
lot that is at 41 mg/mL.
• For the sampling plan we need to know the
number (n) of test results to take and average.
• We will accept the lot if the average () exceeds k
mg/mL.
ValWkPHL1012S2 8

9. SAMPLING PLAN CALCULATIONS A.
You will need the table of the normal distribution for this.
• Suppose we have a lot that is at 42.0 mg/mL.
•  would be normally distributed with µ=42.0
– And the SEM = s/!n. We want !>k
x − 42.0 k − 42.0
x= >
s s
n n
x = standard normal deviate
From a “normal” table (or “x” with ν = ∞) we want a
probability of 0.95 that “x” will be greater than the
“k” expression.
ValWkPHL1012S2 9

10. SAMPLING PLAN CALCULATIONS A1.
You will need a normal distribution table for this
• x0.95,∞ = 1.645 (cumulative probability of 0.95)
• We know that this must be greater than the “k”
expression.
• We also know that k must be less than 42.0 since
the smallest acceptable  will be 42.0.
• Therefore:
k − 42.0
= 1.645 since k < x
s
n
ValWkPHL1012S2 10

11. SAMPLING PLAN CALCULATIONS B.
• Now suppose that the correct value for the lot is 41.0 mg/
mL. So now µ = 41.0 and we want a probability of 0.01
that !>k. Now:
x − 41.0 k − 41.0
x= > = −2.326
s s
n n
k − 42.0 1.645
= = −0.707
k − 41.0 − 2.326
k = 41.59
ValWkPHL1012S2 11

12. SAMPLING PLAN CALCULATIONS C.
• Going back to the original equation for a passing result
and knowing that s = ± 0.45 (From our assay validation
studies?)
k − 42.0 41.59 − 42.0 − 0.41
= = = −1.64
s s s
n n n
2
[−1.64]s
= (− 0.41) or n =
([− 1.64][0.45])
2
n (− 0.41)
0.544644
n= = 3.24
0.1681
ValWkPHL1012S2 12

13. SAMPLING PLAN
• The sampling plan now says: To have a 95%
probability of accepting a lot at 42.0 mg/mL or
better and a 1% probability of accepting a lot at
41.0 mg/mL or worse, given a standard deviation
of ± 0.45 mg/mL for the test method; run four
samples and average them. Accept the lot if the
mean is 41.59 mg/mL or better.
• Note that the calculated value of n is close enough
to 3 that some would argue for 3 samples.
ValWkPHL1012S2 13

14. SAMPLE SIZES FOR MEANS
• Suppose we want to determine µ using a test where we
know the standard deviation (s) of the population.
• How many replicates will we need in the sample?
• The length of a confidence interval = L
2 2 2 2
2ts 2 4t s 4t s
L= L = n = 2 L = 2Δ
n n L
ValWkPHL1012S2 14

15. Recalculation of Earlier Problem.
2 2
4t s
n= 2
L
L = 2, s = ±2, t0.95,∞=1.960 (two sided)
2 2
4(2) (1.96 ) 61.4656
n= 2
=
(2) 4
n = 15.4 or 16
Iterate : (t )0.95,15 = 2.131 n = 18.16, (t )0.95,18 = 2.101 n = 17.66
(t )0.95,17 = 2.110 n = 17.81 so n = 18
ValWkPHL1012S2 15

16. Sample size for estimating µ
• Note the statement: We are determining the % of drug
present and we wish to bracket the true amount (µ%)
by ± 0.5% and do this with 95% confidence, so L = 2 x
0.5 = 1.0
• We have 22 previous estimates for which s = 0.45
• Now at the 95% level of significance (1–0.95), t0.975,21 =
2.080.
2 2
4(2.080) (0.45 )
n= 2
= 3.5
(1.0)
ValWkPHL1012S2 16

17. POOLED VARIANCE
2 2
n1 −1 s1 + n2 −1 s2
( ) ( )
sp =
n1 +n2 −2
ValWkPHL1012S2 17

18. Calculating the Confidence Interval, Sp
• The results of the four determinations are: 42.37%,
42.18%, 42.71%, 42.41%.
• ! = 42.42% and s = 0.22% (n2 – 1) = 3
• Using the extra 3 df and s = 0.22% we have:
2 2
21(0.45 ) + 3(0.22 )
Sp = = 0.43
21 + 3
ValWkPHL1012S2 18

19. Calculating the Confidence Interval, L
• Sp = s, the new estimate of the standard deviation, so a
new confidence interval can be calculated with 24 df.
t(0.975, 24)= 2.064.
L = 2(2.064)(0.43)
4
L = 0.88752, rather than 1.0.
( )
C.I. = ± L = 0.44376 or ± 0.45
2
C.I. 95% = 42.42 ± 0.45 or 41.97 - 42.87
Note that n = 4 not 25 for calculating L.
ValWkPHL1012S2 19

20. Sample Sizes for Estimating Standard Deviations. I.
• The problem is to choose n so that s at n – 1 will be
within a given ratio of s/σ.
• Examples are found in reproducibility,
repeatability, and intermediate precision
measurements.
• s = standard deviation experimentally determined.
σ = population or true standard deviation. s2 and
σ2 are corresponding variances.
• You will use n to derive s.
ValWkPHL1012S2 20

21. Sample Sizes for Estimating Standard Deviations. χ2
• This is the asymmetric
2
distribution for σ2.
• Now as an example, χ 2
=
(n − 1)s
assume n-1 = 12. At 12 df, n −1 2
χ2 will exceed 21.0261 5% σ
of the time and it will 2 2
exceed 5.2260 95% of the χ n −1 s
time. Therefore 90% of the = 2
time, χ2 will lie between
5.2260 and 21.0261 for 12
(n − 1) σ
df. 2 2
• Check your tables to ⎛ s ⎞ ⎛ χ ⎞ n −1
confirm this. ⎜ ⎟ = ⎜ ⎟
⎜ (n − 1) ⎟
⎝ σ ⎠ ⎝ ⎠
ValWkPHL1012S2 21

22. Confidence interval for the standard deviation.
• Given the data in the previous slide, we know that
(s2/σ2) will lie between (5.2260/12) and
(21.0261/12), or between 0.4355 and 1.7552.
• Thus the ratio of s/σ will lie between the square
roots of these numbers or between 0.66 and 1.32
or 0.66 < s/σ < 1.32. This gives:
• s/1.32 < σ < s/0.66. If you know s this gives you a
90% confidence interval for the standard
deviation.
• Now let’s reverse our thinking.
ValWkPHL1012S2 22

23. Sample Sizes for Estimating Standard Deviations.
Continued. I.
• Instead of the confidence interval, suppose we say
that we want to determine s to be within ± 20% of
σ with 90% confidence. So:
• 1 – 0.2 < s/σ < 1+ 0.2 or 0.8 < s/σ < 1.2
• This is the same as: 0.64 < (s/σ)2 < 1.44
• Since we want 90% confidence we use levels of
significance at 0.05 and 0.95.
• Now go to the χ2 table under the 0.95 column and
look for a combination where χ2/df is not < 0.64,
but df is as large as possible.
ValWkPHL1012S2 23

24. Sample Sizes for Estimating Standard Deviations.
Continued. II.
• Trial and error shows this number to be about 50.
• Next we go to the column under 0.05 and look for
a ratio that does not exceed 1.44, but df is as small
as possible.
• Trial and error will show this number to be
between 30 and 40.
• You must take the larger of the two numbers and
since df = n – 1, n = 51 replicates.
ValWkPHL1012S2 24

25. Do Not Panic. Consider This!
• Instead of the confidence interval, suppose we say
that we want to determine s to be within ± 50% of
σ with 95% confidence. So:
• 1 – 0.5 < s/σ < 1+ 0.5 or 0.5 < s/σ < 1.5
• This is the same as: 0.25 < (s/σ)2 < 2.25
• Since we want 95% confidence we use levels of
significance at 0.025 and 0.975.
• Now go to the χ2 table under the 0.975 column and
look for a combination where χ2/df is not < 0.25,
but df is as large as possible.
ValWkPHL1012S2 25

26. Greater Confidence, But Lesser Certainty
• Trial and error shows this number to be 8.
• Next we go to the column under 0.025 and look for
a ratio that does not exceed 2.25, but df is as small
as possible.
• Trial and error will show this number to be 8. The
same as the other df.
• You must take the larger of the two numbers and
but in this case df = 8 and n = 9.
• You have a greater confidence interval for a
smaller n.
ValWkPHL1012S2 26

27. n for Comparing Two Averages
x1 − x2
tα , df = Δ = x1 − x2 n1 = n2
σ 12 2
σ2
+
n1 n2
2
Δ 2
t α ,df
2
tα .df =
σ 2
σ 2
n
(σ 2
1 )
+ σ 2 = Δ2
2
1 2
+
n n
n=
2
tα , df (2 2
σ1 + σ 2 )
Δ2
ValWkPHL1012S2 27

28. Introduction to the Analysis of Variance
(ANOVA) I.
This method was aimed at deciding whether or not
differences among averages were due to
experimental or natural variations or true
differences among averages.
R.A. Fisher developed a method based on
comparing the variances of the treatment means
and the variances of the individual measurements
that generated the means.
The technique has been extended into the field
known as DOE or factorial experiments
ValWkPHL1012S2 28

29. Introduction to the Analysis of Variance
(ANOVA) II.
• The method is based on the use of the F-test and
the F-distribution (Named after him.)
– The F-distribution, and all distributions related to
errors, is a skewed, unsymmetrical distribution.
2
ns y
F= 2
s pooled
– S2y represents the variance among the treatments and
s2pooled is the variance of the individual results (system
noise).
ValWkPHL1012S2 29

30. Introduction to the Analysis of Variance
(ANOVA) III.
• F increases as the number of replicates increases.
– In simple ANOVA systems n is the same for all
treatments.
– By increasing n you amplify small differences between
the variances of the treatment means and the system
noise.
– An F value of 1.0 or less says that the system noise is
greater than the variance of the means. This suggests
that the differences among the means are due to
experimental or environmental variations.
ValWkPHL1012S2 30

31. Introduction to the Analysis of Variance
(ANOVA) IV.
• Because of the importance of system noise, before
doing an ANOVA or factorial experiment, you
should reduce variation in the system to a
minimum.
– You should remove all special cause variation and
minimize common cause variation.
– Methods such as Statistical Process Control (SPC)
should be used to reduce variations.
• Note: A system where special cause variation has been
eliminated and only common cause variation is left is known as
a system under statistical control.
ValWkPHL1012S2 31

32. Introduction to the Analysis of Variance (ANOVA)
V.
• The F-distribution depends on the number of degrees
of freedom of the numerator and denominator and the
level of type 1 error that you will accept.
– For each level of type 1 error there are different
distribution tables. The exact value of F then depends
on the number of degrees of freedom of the numerator
and denominator.
• If the calculated F exceeds the tabular F, it is then significant at
the1-α level. Where α is the level of type 1 error that you are
willing to accept.
• α is the p value. Most statistical software programs will
calculate the p value. Normally, you want 0.05 or 0.01.
• Type-1 error is where you falsely conclude that there is a
difference. AKA: False positive, producer’s risk.
ValWkPHL1012S2 32

33. Fairness of 4 sets of dice. (Taken from Anderson, MJ and
Whitcomb, PJ, DOE Simplified, CRC Press, Boca Raton, FL, 2007.)
• Frequency distribution for 56 rolls of dice.
Dots White Blue Green Purple
6 6+6 6+6 6+6 6
5 5 5 5 5
4 4 4+4 4+4 4
3 3+3+3+3+3 3+3+3+3 3+3+3+3 3+3+3+3+3
2 2+2+2 2+2+2+2 2+2+2+2 2+2+2+2+2
1 1+1 1 1 1
Mean (y) 3.14 3.29 3.29 2.93
Var. (s2) 2.59 2.37 2.37 1.76
n = 14 Grand Ave. = 3.1625
• Grand average = Total of all dots/56 dice (4X14)
ValWkPHL1012S2 33

34. Fairness of 4 sets of dice. Calculation of F.
Note differences in denominator.
Since F is much less than 1.0 we can assume that there is no
significant difference among the colors even without looking
at an F table.
s2 =
(3.14 − 3.1625)2 + (3.29 − 3.1625)2 + (3.29 − 3.1625)2 + (2.93 − 3.1625)2
y
4 −1
2
s y = 0.029
s 2 = 2.59 + 2.37 + +2.37 + 1.76 = 2.28
pooled
4
2
n * s y 14 * 0.029
F= 2 = = 0.18
s pooled 2.28
ValWkPHL1012S2 34

35. Fairness of 4 sets of dice. How about a loaded
set?
Dots White Blue Green Purple
6 1 3 6 1
5 1 2 5 2
4 1 3 1 3
3 2 4 1 1
2 5 1 0 2
1 4 1 1 5
Mean (y) 2.50 3.93 4.93 2.86
Var. (s2) 2.42 2.38 2.07 3.21
n = 14 Grand Ave. = 3.555
δ= -1.055 0.375 1.375 -0.695
δ2 = 1.1130 0.1406 1.8906 0.4830
Σδ2 = 3.6245 Σδ2/3 = s2y = 1.2082
ValWkPHL1012S2 35

36. Fairness of 4 sets of dice. How about a loaded
set? ANOVA
2 2.42 + 2.38 + 2.07 + 3.21
spooled = = 2.52 df = 4(14 - 1) = 52
4
2
s y = 1.21 df = 3 (4 - 1)
14 *1.21
F= = 6.71 F3,52 = 6.71
2.52
Tabular F3,52 = 2.839 − 2.758 at 5%, p = 0.05
and 4.313 - 4.126 at 1%, and 6.595 - 6.171 at 0.1%.
Range is for F3,40 to F3,60 . Significant at p = 0.001
ValWkPHL1012S2 36

37. Least Significant Difference
Lucy in the Sky with Diamonds (LSD)
• DO NOT EVER USE THIS METHOD WITHOUT THE
PROTECTION OF A SIGNIFICANT ANOVA
RESULT ! ! !
• There are 45 combinations of 10 results taken in pairs. If you focus
mainly on the high and low results, you are almost guaranteed to
encounter a type-1 error.
– This is why you need to use the ANOVA coupled with an LSD
determination.
• The LSD is based on the equations for confidence intervals.
n 2
LSD = ± t(1−α ,df ) × s pooled 2 / n s pooled =
∑s 1 i
n
ValWkPHL1012S2 37

38. LSD for the Current Problem
2.42 + 2.38 + 2.07 + 3.21
s pooled = = 2.52 = 1.59
4
2
LSD = 2.01 ×1.59 = ±1.21
14
at 99% LSD = ±1.333 for t (0.99,df =52 ) ≅ 2.68
• The (1-α) level of the t determines the level of
significance for the LSD.
• n = 14 for replicates, but s2pooled had 4X(14-1)
= 52 df.
ValWkPHL1012S2 38

39. So where are the bad dice?
• Given the LSD = ±1.333, the result can be displayed in
different ways.
• Plot the result as the mean of the average count of the
treatments (colors) ± ½ LSD.
– Then look for overlaps. A significant difference will not have
an overlap.
• Or take the difference between means and compare
them to the LSD.
– In the present case, the white and purple dice are similar, but
the green dice are definitely higher, with the blue dice
different from the white, but not from the green and only
marginally different from the purple.
ValWkPHL1012S2 39

40. White = 2.50 Blue = 3.93 Green=4.93 Purple=2.86
White = 2.50 1.43 2.43 0.36
Blue = 3.93 1.43 1.00 1.07
Green=4.93 2.43 1.00 2.07
Purple=2.86 0.36 1.07 2.07
For 95% confidence, the LSD is ± 1.21 and for 99%, the LDS
is ± 1.33.
So blue and green are different from white, and green is
different from purple and white at the 99% level.
White and purple are the same as are blue and green.
Purple is also similar to blue, but not to green.
All of this holds at the 99% level, thus at p = 0.01 we conclude
that blue and green dice run to higher numbers than white
and purple.
ValWkPHL1012S2 40