A document on design of experiments (DOE) is summarized as follows: DOE is a systematic method to determine the relationship between factors affecting a process and its output. Experiments involve the systematic collection of data with a focus on the design itself rather than results. Factors that can be modified are called controllable inputs or x-factors, while uncontrollable inputs cannot be changed. Hypothesis testing uses statistical methods to determine significant factors. Blocking and replication are used to reduce unwanted variations and estimate random error. Interactions occur when the simultaneous influence of two variables on a third is not additive. Well-designed experiments can optimize processes and solve problems.

1. DOE

2. Definition:
Design of experiments (DOE) is a systematic method to determine the relationship between factors affecting a
process and the output of that process. In other words, it is used to find cause-and-effect relationships. This
information is needed to manage process inputs in order to optimize the output.
Experiments involves:
• The systematic collection of data
• A focus on the design itself, rather than the results
• Planning changes to independent (input) variables and the effect on dependent variables or response variables
• Ensuring results are valid, easily interpreted, and definitive.
Common DOE Terms and Concepts:
•Controllable input factors, or x factors, are those input parameters that can be modified in an experiment or
process. For example, in cooking rice, these factors include the quantity and quality of the rice and the quantity of
water used for boiling.
•Uncontrollable input factors are those parameters that cannot be changed. In the rice-cooking example, this may
be the temperature in the kitchen. These factors need to be recognized to understand how they may affect the
response.
• Hypothesis testing helps determine the significant factors using statistical methods. There are two possibilities in
a hypothesis statement: the null and the alternative. The null hypothesis is valid if the status quo is true. The
alternative hypothesis is true if the status quo is not valid. Testing is done at a level of significance, which is based
on a probability.

3. Choosing the Levels of a Factor
Choosing the appropriate factor levels and the number of levels of a factor requires an expertise. For example, to study the
effect of the temperature on human comfort, most of us have some idea about the comfortable temperature. We know that
the temperature factor level below 50-degree Fahrenheit (10 degree Celsius) or above 100-degree Fahrenheit (37.8-degree
Celsius) will not produce any results that we don’t know. Therefore, running the temperature factor levels of such will be
wasted. Only subject matter experts know the ranges of the factor that could potentially provide some valid and useful
responses.
To determine the effect of a factor, generally two levels of a factor are enough.
Treatment/ Treatment Combinations
The word treatment in the design of experiment can be considered as the medical treatment. For example, if a patient is given a
treatment of a medicine, he/she is on a particular treatment. For a single factor, assigning a level of a factor is the same as
assigning a treatment such as providing a particular medication. When a patient is given multiple medications, we say that a
treatment combination or combination of factor levels is applied. Assume that a human comfort study uses two levels of the
temperature factor and two levels of the humidity factor, which results in four treatment combinations (low-low, low-high,
high-low, and high-high levels of the temperature and the humidity).
Response/Dependent Variable
Responses, or output measures, are the elements of the process outcome that gage the desired effect. In the cooking
example, the taste and texture of the rice are the responses. The controllable input factors can be modified to optimize the
output. The response makes more sense when some treatments (or combination of treatments are applied to experimental
units, and the response is observed and measured.
The relationship between the factors and responses is shown in Figure 1.

4. Figure 1: Process Factors and Responses
∙ Blocking and replication: Blocking is an experimental technique to avoid any unwanted variations in the input
or experimental process. For example, an experiment may be conducted with the same equipment to avoid any
equipment variations. Practitioners also replicate experiments, performing the same combination run more than
once, in order to get an estimate for the amount of random error that could be part of the process.
∙ Interaction: When an experiment has three or more variables, an interaction is a situation in which the
simultaneous influence of two variables on a third is not additive.

5. Importance:
• Designed experiments are an advanced and powerful analysis tool during projects.
• can filter out noise and discover significant process factors.
• The factors can be used to control response properties in a process.
• engineer a process to the exact specification their product or service requires.
• A well built experiment can save not only project time but also solve critical problems which have remained unseen in processes.
Purpose of Experimentation:
• Comparing Alternatives
• Identifying the Significant Inputs
• Reducing Variability
• Minimizing, Maximizing, or Targeting an Output
• Improving process or product "Robustness.
• Balancing Tradeoffs when there are multiple Critical to Quality Characteristics
Advantages:
• reduce time to design/develop new products & processes
• improve performance of existing processes
• improve reliability and performance of products
• achieve product & process robustness
• evaluation of materials, design alternatives, setting component & system tolerances, etc.

6. Three Principles of Design of Experiments
The three principles of experimental designs include (1) replication, (2) randomization, (3) and blocking determines the
appropriate statistical method (Hinkelmann & Kempthorne, 2008; Kempthorne, 1952). In absence of these basic principles, the
validity, precision, accuracy and sensitivity of the experiment will be questionable.
Replication Principle
The replication means that several experimental units will receive exact same treatment. Replications allow for the estimation
of experimental random error if there is no systematic variation between the experimental units treated alike.
Replication is often confused with repeated measure on the same experimental unit. Replications is the application of the
same treatment combination in more than one experimental unit, while repeated measurement is conducted on the same
experimental unit. If the same diet factor is applied/fed to 25 rabbits (25 experimental units), the experiments are replicated.
However, if a rabbit is measured every month for the next 25 months resulting in 25 measurements on a single cow (single
experimental unit), the experiment is NOT replicated, rather it is called a repeated measure design. Replication provides
precision to any experiments, including the repeated measure design.
Randomization Principle
The randomization principle ensures the validity of the experiment. The validity of the experiment will be questionable if
randomization is not performed appropriately. Not all the variations between the experimental units can be controlled.
Randomization accounts for these uncontrolled extraneous variations between experimental units.
However, there are natural restrictions in randomization such as a gender cannot be assigned to an individual; a place cannot be
chosen randomly; time cannot be randomized etc.

7. Blocking Principle
Block is defined by a set of homogenous experimental conditions. Blocking or local control is utilized to reduce experimental
error, resulting in better precision. Assume that we are testing a certain combination of diet on cattle growth. Same diet is fed in
the southern and northern part of a country. Experimental units (cattle/cows) from these two locations are exposed to two
different climates (e.g., hot humid vs cold dry). Therefore, there is a systematic variation, resulting in nonhomogeneous
experimental units from these two locations. To control these issues, the large experimental unit (the entire country) is divided
into two portions with homogenous conditions, southern and northern part in this case. The location factor is then called
blocked. In this diet experiment, the location is not a factor of interests, rather it is somewhat disturbing the experiment. This
type of factor is known as a nuisance factor. Generally, nuisance factors shall be blocked to improve the precision of the
experiment or to reduce the experimental error.
A smaller set of experimental units will always produce a reduced experimental error than a larger set of experimental units.
Blocking also performed whenever there is a restriction in randomization . Another common use of blocking is if there are not
enough experimental units available from one homogenous unit (e.g., samples from different batches of raw materials). If so, the
batches shall be blocked.
Outline of a Method
An experimental unit is assumed to respond from a treatment as well as from experimental design. Therefore, an observation
from an experiment could be generally modeled as in Equation 1 (Hinkelmann & Kempthorne, 2008).

8. A Simple One-factor Experiment
The comparison of two or more levels in a factor can be done using an F-test. This compares the variance of the
means of different factor levels with the individual variances, using this equation:
where:
n = the sample size
s2
Y-bar = the variance of the means, which is calculated by dividing the sum of variances of the individual means by
the degrees of freedom
s2pooled = pooled variance, or the average of the individual variances.
If the value of F (the test statistic) is greater than the F-critical value, it means there is a significant difference
between the levels, or one level is giving a response that is different from the others. s2
pooled is kept to a minimum, as
it is the noise or error term. If the F value is high, the probability (p-value) will fall below 0.05, indicating that there is a
significant difference between levels.
If F = 1, it means the factor has no effect.
As an example of a one-factor experiment, data from an incoming shipment of a product is given in Table 1.

9. Lot Data
A 61, 61, 57, 56, 60, 52, 62, 59, 62, 67, 55, 56, 52, 60, 59, 59, 60, 59, 49, 42, 55, 67, 53, 66, 60
B 56, 56, 61, 67, 58, 63, 56, 60, 55, 46, 62, 65, 63, 59, 60, 60, 59, 60, 65, 65, 62, 51, 62, 52, 58
C 62, 62, 72, 63, 51, 65, 62, 59, 62, 63, 68, 64, 67, 60, 59, 59, 61, 58, 65, 64, 70, 63, 68, 62, 61
D 70, 70, 50, 68, 71, 65, 70, 73, 70, 69, 64, 68, 65, 72, 73, 75, 72, 75, 64, 69, 60, 68, 66, 69, 72
Groups Count Sum Average Variance
A 25 1,449 57.96 31.54
B 25 1,483 59.32 23.14333
C 25 1,570 62.80 18.5
D 25 1,708 68.32 27.64333
ANOVA
Source of
Variation SS df MS F p-value F-crit
Between groups 1,601.16 3 533.72 21.17376 1.31 x 10-10 2.699394
Within groups 2,419.84 96 25.20667

10. If the value is below the critical F value, a value based on the accepted risk, then the null hypothesis is not rejected.
Otherwise, the null hypothesis is rejected to confirm that there is a relationship between the factor and the response.
Table 2 shows that the F is high, so there is a significant variation in the data. The practitioner can conclude that
there is a difference in the lot means.
Two-level Factorial Design
In this design, the factors are varied at two levels – low and high. Two-level designs have many advantages. Like:
• The size of the experiment is much smaller than other designs.
• The interactions of the factors can be detected.
For an example of a two-level factorial design, consider the cake-baking process. Three factors are studied: the
brand of flour, the temperature of baking and the baking time. The associated lows and highs of these factors are
listed in Table 3.
Factor Name Units
Low
Level (-)
High
Level (+)
A
Flour
brand Cost Cheap Costly
B Time Minutes 10 15
C
Tempera
ture
Degrees
(C) 70 80
Table 3: Cake-baking Factors and Their Associated Levels

11. The output responses considered are “taste” and “crust formation.” Taste was determined by a panel of experts,
who rated the cake on a scale of 1 (worst) to 10 (best). The ratings were averaged and multiplied by 10. Crust
formation is measured by the weight of the crust, the lower the better.
The experiment design, with the responses, is shown in Table 4.
Table 4: Settings of Input Factors and the Resulting Responses
Run Order A: Brand B: Time (min) C: Temp. (C)
Y1: Taste
(rating)
Y2: Crust
(grams)
1 Costly(+) 10(-) 70(-) 75 0.3
2 Cheap(-) 15(+) 70(-) 71 0.7
3 Cheap(-) 10(-) 80(+) 81 1.2
4 Costly(+) 15(+) 70(-) 80 0.7
5 Costly(+) 10(-) 80(+) 77 0.9
6 Costly(+) 15(+) 80(+) 32 0.3
7 Cheap(-) 15(+) 80(+) 42 0.5
8 Cheap(-) 10(-) 70(-) 74 3.1
Analysis of the results is shown in Table 5. Figures 2 to 4 show the average taste scores for each factor as it
changes from low to high levels. Figures 5 to 7 are interaction plots; they show the effect of the combined
manipulation of the factors

12. Table 5: ANOVA Table for the Taste
Response
Factor df SS MS F Effect Contrast p
F-crit at
1%
Brand 1 2.0 2.0 0.0816 -1 -4.00 0.82 16.47
Time 1 840.5 840.5 34.306 -20.5 -82.00 0.11
Brand x
time 1 0.5 0.5 0.0204 0.5 2.00 0.91
Temp 1 578.0 578.0 23.592 -17 -68.00 0.13
Brand x
temp 1 72.0 72.0 2.9388 -6 -24.00 0.34
Time x
temp 1 924.5 924.5 37.735 -21.5 -86.00 0.10
Brand x
time x
temp 1 24.5 24.5 1 -3.5 -14.00 0.50
Error 1 24.5 24.5
Total 7 2442.0

13. Figure 2: Average Taste Scores for Low and High Flour
Brand Levels Figure 3: Average Taste Scores for Low and High Bake
Time (Minutes) Levels
Figure 4: Average Taste Scores for Low and High
Baking Temperature (C) Levels
Figure 5: Average Taste Scores for Flour Brand by
Time (Minutes)

14. Figure 7: Average Taste Scores for
Time (Minutes) by Temperature (C)
From reading an F table, the critical F value at 1 percent is
16.47. As the actual value of F for time and temperature
exceed this value (time is at 34.306 and temperature is
23.592), it’s possible to conclude that both of them have a
significant effect on the taste of the product. This is also
evident from Figures 3 and 4, where the line is steep for
the variation of these two factors. Figure 7 also shows that
when the temperature is high, the taste sharply decreases
with time (as charring takes place).
For the crust formation, the data analysis is shown in
Table 6.

15. Factor df SS MS F Effect Contrast
F-crit at
1%
Brand 1 1.4 1.4 1.4938 -0.825 -3.30 16.47
Time 1 1.4 1.4 1.4938 -0.825 -3.30
Brand x
time 1 1.1 1.1 1.1536 0.725 2.90
Temp 1 0.5 0.5 0.4952 -0.475 -1.90
Brand x
temp 1 0.7 0.7 0.7257 0.575 2.30
Time x
temp 1 0.1 0.1 0.0672 0.175 0.70
Brand x
time x
temp 1 0.9 0.9 1 -0.675 -2.70
Error 1 0.9 0.9
Total 7 5.9
In this case the actual F value for the
three factors (brand, time and
temperature) are below the critical F
value for 1 percent (16.47). This shows
that these are not significant factors for
the crust formation in the cake. If further
optimization of the crust formation is
needed, then other factors, such as the
quantity of ingredients in the cake (eggs,
sugar and so on), should be checked.

16. F Test
An example with four independent groups and a continuous outcome measure. Suppose that the outcome is
systolic blood pressure, and we wish to test whether there is a statistically significant difference in mean systolic blood
pressures among the four groups. The sample data are organized as follows:
The hypotheses of interest in an ANOVA are as follows:
∙ H0: μ1 = μ2 = μ3 ... = μk
∙ H1: Means are not all equal.
where k = the number of independent comparison groups.
In this example, the hypotheses are:
∙ H0: μ1 = μ2 = μ3 = μ4
∙ H1: The means are not all equal.
The null hypothesis in ANOVA is always that there is no difference in means. The research or alternative hypothesis is always
that the means are not all equal and is usually written in words rather than in mathematical symbols.

17. Example:
A clinical trial is run to compare weight loss programs and participants are randomly assigned to one of the
comparison programs. Participants follow the assigned program for 8 weeks. The outcome of interest is weight loss,
defined as the difference in weight measured at the start of the study (baseline) and weight measured at the end of
the study (8 weeks), measured in pounds.
Three popular weight loss programs are considered. The calorie diet, low fat diet and the third is a low carbohydrate
diet. For comparison purposes, a fourth group is considered as a control group.

18. After 8 weeks, each patient's weight is again measured and the difference in weights is computed by subtracting the
8 week weight from the baseline weight. Positive differences indicate weight losses and negative differences indicate
weight gains.