Javier Garcia - Verdugo Sanchez - Six Sigma Training - W2 Design Of Experiments (DOE) Intro

Introduction to DOE
(D i f E i t)(Design of Experiment)
C t Pl t f A b t
55
58
61
1
Contour Plot of Ausbeute
64
67
70
73
76
0
Katal
76
79
82
1
160 170 180
-1
Temp
Holdvalues: Konz: 20,0
2k factorial designs
Week 2
Knorr-Bremse Group
2 factorial designs
About this Module
The Design of Experiment is one of the most effectiveThe Design of Experiment is one of the most effective
tools in the DMAIC cycle.
You receive with low effort many information about
processes, products and services. E.g. only 8 trials arep , p g y
sufficient to determine the effects of three variables on
one or more results (responses). The evaluation in( p )
Minitab is easy and user friendly.
The 2k factorial experiments give you an uncomplicated
introduction to this technique. All further experimental
designs are similar but in a modified form.
Knorr-Bremse Group 09 BB W2 DOE Intro 08, D. Szemkus/H. Winkler Page 2/78

Content of this Module
• Ways to learn
• Components of an experiment
• Experimental validation
St f l i i t• Steps for planning an experiment
• 2k factorial design2 factorial design
• Practical exercises
• Final report
DOE within the DMAIC Cycle
Control
Maintain Improvements
SPC
Define
Project charterSPC
Control Plans
Documentation
Project charter
(SMART)
Business Score Card
QFD + VOC
D QFD VOC
Strategic Goals
Project strategy
C M
Measure
Baseline AnalysisImprove
AI
Baseline Analysis
Process Map
C + E Matrix
Measurement System
Analyze
Definition of
p
Adjustment to the
Optimum
FMEA y
Process Capability
Definition of
critical Inputs
FMEA
S
FMEA
Statistical Tests
Simulation
Statistical Tests
Multi-Vari Studies
Regression
Tolerancing

Ways to Learn…
•• EmpiricEmpiric – Observation of natural, significant events
(M lti i St di )(Multivari-Studies)
• If you are lucky, a significant informative event occurs while you are
presentpresent
•• ExperimentalExperimental – Induce a informative event
• Manipulate Input-Variables in a way, that the effect on the Output-Manipulate Input Variables in a way, that the effect on the Output
Variables can be investigated
• Provoke the occurrence of an informative event
• Correctly performed experiments are:
• beneficial
• meaningful
What is an Experimental Design?
• A systematic series of runs where we manipulate directly various
Input Variables (X’s) while we observe the effects on the Output
Variables (Y’s).
• To determine which X’s have the strongest impact on the Yg p
• To determine how to set the influential X’s to center the Y on the
targettarget
• To determine how to set influential X’s to minimize the variability of Y
• To determine how to set influential X’s to minimize the influence of
Noise Variables
• A well planed experiment eliminates all possible causes which will
have an effect on the Y, except those we want to test! If an effect
th Y th it di tl b i d t th i toccurs on the Y, then it can directly be assigned to the input
variables (X’s) which are under investigation. This is possible
because these input variables are independent of each other.
because these input variables are independent of each other.

The Process
Analyze the process
Controllable Inputs
X1 X2 X3
Controllable Inputs
Quality
LSL USL
Quality
characteristics:
Outputs
The Process
Inputs:
Raw material,
parts etc
Y1, Y2, …
parts etc.
Z1 Z2 Z3
Not controllable Inputs
The Process
Improve the process
Controllable Inputs
X1 X2 X3
X
Controllable Inputs
Quality
X LSL USL
Quality
characteristics:
Outputs
The ProcessX
Inputs:
Raw material,
parts etc
Y1, Y2, …
parts etc.
X LSL USL
Z1 Z2 Z3
X

The Process
Control the process
Controllable Inputs
X1 X2 X3
Controllable Inputs
Quality
LSL USL
Quality
characteristics:
Outputs
The Process
Inputs:
Raw material,
parts etc
Y1, Y2, …
parts etc.
Z1 Z2 Z3
The Benefits of Experimentation
• Process characterization
• Determination which X’s most strongly influence the YDetermination which X s most strongly influence the Y
• Includes controllable factors as well as noise factors (X’s)
• Identifies critical process variables (mean and variation)• Identifies critical process variables (mean and variation)
• Identifies variables which have to be controlled
• Gives procedures for controlling inputs instead of control charts for
outputs
P ti i ti• Process optimization
• Determination of the setting of critical inputs
• Definition of appropriate specification limits
• Product designg
• Helps to understand the X’s early in the design phase
• Gives a guide line for robust designs
• Gives a guide line for robust designs

The Strategy of Experimentation
Collect information Fractional FactorialCollect information
Validate factors
A l b h i f
Mirror Plackett-
Burnam
2k F t i lAnalyze behavior of
important factors
E t bli h d l
2k Factorial
Center Points
BlockingEstablish a model
Determine optimized
dj t t
Blocking
Full Factorial
Box-Behnken
adjustments
RSM
Taguchi
EVOP
Taguchi
Knowledge and complexity define the type of experiment
The Strategy of Experimentation
Fractional factorial designs
Sort out uncritical factors Fold overSort out uncritical factors, Fold over
Plackett Burman Designs
2k factorial designs2 factorial designs
Center points
BlocksBlocks
Evaluate co variables
Full factorial designsg
RSM
Box Behnken
Evop
Taguchi
Mixed design
Knowledge and complexity define the type of experiment

Try and Error
• The problem: The actual gas consumption of a car is 12l/100km.
We like to achieve 8l/100km.
• Some possible actions:
• Change the brand of the gasChange the brand of the gas
• Change the type of gas
• Drive slower
• Tune the car
• Wash and polish the car
• Assemble new tires
• Change the tire pressure
• What happens if it helps?
• What happens if it doesn’t help?
Sequential Experimentation “OFAT”
Problem: Fuel consumption of a car is about 12l/100km
Speed km/h Octane Tire pressure l /100 km
105 91 2,1 11,5
105 91 2 4 10 4105 91 2,4 10,4
105 95 2,1 10,8
90 91 2,1 9,5
How may runs do we need to find out the settings of variables?
90 9 , 9,5
How may runs do we need to find out the settings of variables?
How do we explain the above results?
If there were more variables, how long would it take to get a
optimized solution?
What if there is a combination of two or more variables that lead to
the best fuel consumption?

Full Factorial Experiment
Problem: Fuel consumption of a car is about 12l/100km
Speed km/h Octane Tire pressure l / 100 km
90 91 2,1 9,5
105 91 2,1 11,5
90 95 2,1 9,2
105 95 2,1 10,8
90 91 2,4 8,2
105 91 2,4 10,4
90 95 2,4 7,5
OFAT Runs
105 95 2,4 10,1
OFAT Runs
What conclusion do you draw now?
Components of Experiments
• Output variables (responses)
• Current performance (Baseline)
• Measurement system capability
• Controllable input variables (factors)
• Variables
• Levels of setting
• Noise variables
• Narrow the validity
• Random order of the treatments
• Blocking
• Experimental Design
• Sample size
• Minimal size of the effect (delta)
• Alpha risk
• Beta risk (Power of the test)

Noise Factors
• Should be controlled what sometimes is difficult and not always
possible for all factors
• Possibilities for control
• Use randomization (random experimental design)Use randomization (random experimental design)
• Try to hold the variables constant
• Blocking Make this factor part of o r e periment• Blocking: Make this factor part of your experiment
• Try to replicate your experiment
• Try to hold noise variables constant
• An experiment with one machine / line only
• Conduct the experiment on one day or one shift
• Use the same operators for entire experimentUse the same operators for entire experiment
H d h th d i fl l i ?
How does each method influences your conclusion?
Randomization
• Randomize the experimental runs
• Randomize the allocation of treatments to samples and the runRandomize the allocation of treatments to samples and the run
order in which the individual runs are performed
• Randomize the investigation of samples in the lab
Scenario:Scenario:
• You have just bought a new hunting gun
• You have four different types of ammunition
• Your are planning an experiment in order to find out which
ammunition hits the target most accurate at a distance of 500 m
• Design an experiment
H d i th t i t?
How can you randomize that experiment?

Blocking
Include the noise factor into your experiment!
• A block is similar to rational subgroups as already mentioned in theg p y
sections Capability Analysis and Statistical Process Control (SPC)
• The variability between the blocks should be larger than the• The variability between the blocks should be larger than the
variability within a block
• Add day, shift or batch as a factor to your experiment
Minitab has the possibility to evaluate the effect of block factors. You
receive additional and valuable information.
H thi th d i fl l i ?
How can this method influence your conclusion?
Repeats vs. Replicates
• Repeats:
• Several samples with the same set up of experiment in a rapidSeveral samples with the same set up of experiment in a rapid
succession
• Replicates:Replicates:
• Samples with the same set up of the experiment (same factor level
settings or treatment) at different timessettings or treatment) at different times
• You may apply both methods to the same experiment.
• Both methods are directly linked to the sample size of the
experiment.
• Conduct the hunting gun experiment firstly based on repeats than
another experiment based on replicatesp p
What are the differences between your conclusion when you
compare repetition with replication?
compare repetition with replication?

Validity Space
• The scopes within we draw conclusion based on the results
• Two scopes: tight or broadTwo scopes: tight or broad
• Tight scope:
• Experiment focused on a certain part of an overall process
• Example: Only one shift one operator one machine one batch etc• Example: Only one shift, one operator, one machine, one batch etc…
• Less influence of noise variables
• Broad scope:
• Investigates the overall process (all machines all operators several• Investigates the overall process (all machines, all operators, several
batches, etc.). More data have to be collected over a longer time
frame. Is more affected by noise variables
Usually tight experimental designs will be used to evaluate the noise
variables Broad experimental designs will be used to forecast productivity
variables. Broad experimental designs will be used to forecast productivity
Validity Space
• Internal validity:
The limits within we draw conclusion based on the results
Internal validity:
• Do the input variables of the experiment really influence the output
variables (response) or is a noise variable the cause?variables (response) or is a noise variable the cause?
• Aims at short time studies.
• External validity:
• May we transfer our experience to similar processes, production
lines, other time periods etc.?
• Aims at long time studies.
• Validity of the statistical conclusion:
• Is there enough information for a valid statistical decision?

Risks for the Statistical Validity
• Little statistical power: Sample size to small• Little statistical power: Sample size to small
• Inaccurate measurement systems broaden the error• Inaccurate measurement systems broaden the error
• Happenstance data increase the variation of the• Happenstance data increase the variation of the
measurements
• Randomization and appropriate sample size selection
help to overcome these shortcomingshelp to overcome these shortcomings.
Standard Order of a 2k Experiment
The design matrix for 2k factorial experiments is usually displayed in
standard order A “ ” or “ 1” is the notation for the low level of astandard order. A - or -1 is the notation for the low level of a
factor, a “+” or “+1” is the notation for the high level.
2The example below shows the design matrix for a 22 factorial:
A 23 factorial looks like:
Speed Octane
-1 -1
1 -11 -1
-1 1
1 1
Speed Octane Tire pressure
-1 -1 -1
1 -1 -1
Note: the 2² Factorial
-1 1 -1
1 1 -1
-1 -1 1
1 1 1
is embedded in the 23
Factorial
1 -1 1
-1 1 1
1 1 1

Calculation of the Effects
Now we calculate the effects of the experiment.
We start with the factor speed. We sum the results (responses) of
the notations “1” and “-1” and calculate the difference of the average
values of “1” and “-1”.
Speed Octane Tire pressure l / 100 km
-1 -1 -1 9,5
1 -1 -1 11,5
1 1 1 9 2-1 1 -1 9,2
1 1 -1 10,8
-1 -1 1 8,2
1 -1 1 10,4
-1 1 1 7,5
1 1 1 10,1
Effect 2,1 -0,5 -1,2
( ) ( ) 1,26,87,10
4
5,72,82,95,9
4
1,104,108,105,11
=−=
+++
−
+++
=Speed
We state that there is an increase of average consumption of 2.1 units by
44
an increase of speed from 90 km/h to 105 km/h.
DOE Design Possibilities in Minitab
Stat
>DOE
>Factorial
>Create Fact. Design
>Display Avail. Design…
We want to use the
designs in the
GREENGREEN - fields
The number of runs is noted on the left side, the number of factors on the
top of the matrix
Exercise: Generate a matrix for 4 factors with Minitab (Full factorial)

Define a DOE Design in Minitab
Stat
>DOE
>Factorial
If the design has been not created by
Minitab, the factors and their settings low and
high has to be defined first!>Factorial
>Define Custom Fact. Design
high has to be defined first!
Interactions
• First we have calculated the main effects of thisFirst we have calculated the main effects of this
experiment. That means we have determined the main
effects of speed, octane number and tire pressure.effects of speed, octane number and tire pressure.
• We are also interested in the interactions of theseWe are also interested in the interactions of these
three factors. Is there a meaningful combination of
input settings which impact the fuel consumption?p g p p
• Lets have a look on the 22 factorial experiment againp g
and how we can determine interactions in a statistical
way. Then we will come back to our example.

Effects of Interactions
• The effects of the interactions are calculated in the
same way like the main effects. First we have to
d t i th “l l ” (1 d 1) f th i t tidetermine the “levels” (1 and -1) of the interaction
column.
• The “level” values of the interaction are the product of
the involved factors.
• Using the 2x2 example we can determine the
interactions of speed x octane by multiplication the
S d O t O t
interactions of speed x octane by multiplication the
values of speed and octane.
Speed Octane v x Oct
-1 -1 1
1 1 11 -1 -1
-1 1 -1
1 1 1
1 1 1
• Each interaction has the same number of runs for
each level like the single factors.
• The values for each factor and for each interaction are
independent We call this “Orthogonal”independent. We call this Orthogonal .
• Exercise: If you enter this matrix into Minitab andExercise: If you enter this matrix into Minitab and
correlate all the columns which each other what
correlation coefficient results?
Speed (v) Octane (Oct) Tire pressure (p) v x Oct v x p Oct x p v x Oct x p l/100 km
1 1 1 1 1 1 1 9 5-1 -1 -1 1 1 1 -1 9,5
1 -1 -1 -1 -1 1 1 11,5
-1 1 -1 -1 1 -1 1 9,2
1 1 -1 1 -1 -1 -1 10,8
1 1 1 1 1 1 1 8 2-1 -1 1 1 -1 -1 1 8,2
1 -1 1 -1 1 -1 -1 10,4
-1 1 1 -1 -1 1 -1 7,5
1 1 1 1 1 1 1 10,1

Speed (v) Octane (Oct) Tire pressure (p) v x Oct v x p Oct x p v x Oct x p l/100 km
-1 -1 -1 1 1 1 -1 9,5
1 -1 -1 -1 -1 1 1 11,5
-1 1 -1 -1 1 -1 1 9,2
1 1 -1 1 -1 -1 -1 10,8
-1 -1 1 1 -1 -1 1 8,2
1 -1 1 -1 1 -1 -1 10,4
-1 1 1 -1 -1 1 -1 7,5
1 1 1 1 1 1 1 10,1
Effect 2,1 -0,5 -1,2 0 0,3 0 0,2
The challenge is to figure out which of the effects aree c a e ge s to gu e out c o t e e ects a e
meaningful (significant). Minitab helps us to select the
factors and interactions which are significant.g
Precisely:
Which effect is significant and how large is the
acceptable (α-risk) risk to make an error?
acceptab e (α s ) s to a e a e o
Evaluation in Minitab
Stat
>DOE
>Factorial
1
>Factorial
>Analyze Fact. Design…
3
2
3

1st Evaluation with all effects
Factorial Fit: Fuel Consumption versus Speed; Octane; Tire Pressure
Estimated Effects and Coefficients for Fuel Consumption
(coded units)
Term Effect Coef
Constant 9,6500
S d 2 1000 1 0500 A
1,129
A Speed
Factor Name
Pareto Chart of the Effects
(response is Fuel Consumption, Alpha = 0,05)
Speed 2,1000 1,0500
Octane -0,5000 -0,2500
Tire Pressure -1,2000 -0,6000
Speed*Octane 0,0000 0,0000
Speed*Tire Pressure 0 3000 0 1500
AC
B
C
A
Term
A Speed
B O ctane
C Tire Pressure
Speed*Tire Pressure 0,3000 0,1500
Octane*Tire Pressure -0,0000 -0,0000
Speed*Octane*Tire Pressure 0,2000 0,1000 BC
AB
ABC
2,01,51,00,50,0
Effect
S = * PRESS = *
Analysis of Variance for Fuel Consumption (coded units)
Effect
Lenth's PSE = 0,3
The pareto shows that speedy p ( )
Source DF Seq SS Adj SS Adj MS F P
Main Effects 3 12,2000 12,2000 4,06667 * *
2-Way Interactions 3 0,1800 0,1800 0,06000 * *
The pareto shows that speed
and tire pressure are
significant. In the next step
d th d l ( ll t3-Way Interactions 1 0,0800 0,0800 0,08000 * *
Residual Error 0 * * *
Total 7 12,4600
we reduce the model (all not
significant effects will be
removed, step by step)!
2nd Evaluation with all effects
Factorial Fit: Fuel Consumption
versus Speed; Octane; Tire Pressure
S d
2,78
Pareto Chart of the Standardized Effects
(response is Fuel Consumption, Alpha = 0,05)
Estimated Effects and Coefficients for Fuel Consumption
(coded units) Tire Pressure
Speed
Term
Term Effect Coef SE Coef T P
Constant 9,6500 0,09014 107,06 0,000
Speed 2,1000 1,0500 0,09014 11,65 0,000
Octane -0,5000 -0,2500 0,09014 -2,77 0,050
Octane
121086420
Standardized Effect
Tire Pressure -1,2000 -0,6000 0,09014 -6,66 0,003
S = 0,254951 PRESS = 1,04
9 9 9 6 j 96 3
The session window shows
now clearly that the 3 main
R-Sq = 97,91% R-Sq(pred) = 91,65% R-Sq(adj) = 96,35%
Analysis of Variance for Fuel Consumption (coded units)
y
factors are significant only
with P values ≤ 0,05. The R-
Sq indicates that the variation
Main Effects 3 12,2000 12,2000 4,06667 62,56 0,001
Residual Error 4 0,2600 0,2600 0,06500
Total 7 12 4600
Sq indicates that the variation
in the fuel consumption is
caused by the change of the
main factor setting withTotal 7 12,4600 main factor setting with
97.91%. The remaining 2% of
variation are just by chance
( )
(random variation)

The graphical evaluation
Stat
>Quality Tools
>Multi Vari Chart>Multi-Vari Chart…
Multi-Vari Chart for Fuel Consumption by Octane - Speed
Here we can see in which
range the fuel consumption
can be adjusted by the
12
2,42,1
90 105
91
95
Octane
j y
factor setting.
Caution: This is a simplified
11
10
onsumption
Caution: This is a simplified
experiment which gives us
a linear relation. In reality
we may investigate also
9
8
FuelCo
we may investigate also
quadratic effects, e.g. for
the factor speed.
2,42,1
7
Tire Pressure
P l i bl S d
Panel variable: Speed
Example
The yield of a reactor depends on the settings of
temperature, concentration and presence/absence
of a catalyst
Stat
>DOE
of a catalyst.
We create a factorial experiment to figure out which
effect each of the single factors has
>Factorial
>Create fact. Design
>number of factors = 3effect each of the single factors has.
A graphical evaluation will help to rate the effects.
Th th ti l d l ill b t bli h d d th
>number of factors = 3
>Design = full fact
The mathematical model will be established and the
optimal results adjusted accordingly.
File: 2k factorial Design.mtw
As “factor” you enter
names and values
RunOrder CenterPt Blocks Temp Con. Catal. Yield
1 1 1 160 20 -1 60
File: 2k factorial Design.mtw
names and values.
Temp 160 180
Con. 20 40
2 1 1 180 20 -1 72
3 1 1 160 40 -1 54
4 1 1 180 40 -1 68
Catal. -1 1
Enter the yield in the
work sheet.
5 1 1 160 20 1 52
6 1 1 180 20 1 83
7 1 1 160 40 1 45
8 1 1 180 40 1 80
o s eet 8 1 1 180 40 1 80

The Steps in Minitab
1
Stat
>DOE
>Factorial>Factorial
3
2
The Mathematical Model of the Results
Results for: 2K FACTORIAL DESIGN.MTW
Factorial Fit: Yield versus Temp; Conc.; Catal.
Estimated Effects and Coefficients for Yield (coded units)
Term Effect Coef
Constant 64,250
T 23 000 11 500
Using the coefficients we can calculate the Fits:
Yield = 64 25 + 11 5xTemp -2 5xCon + 0 75xCatal +Temp 23,000 11,500
Conc. -5,000 -2,500
Catal. 1,500 0,750
Temp*Conc. 1,500 0,750
Temp*Catal 10 000 5 000
Yield = 64,25 + 11,5xTemp -2,5xCon. + 0,75xCatal. +
0,75xTemp*Con. + 5xTemp*Catal. + 0xConc*Catal. +
0.25xTemp*Con.*Catal.
A measure for the quality of our model is derivedTemp*Catal. 10,000 5,000
Conc.*Catal. -0,000 -0,000
Temp*Conc.*Catal. 0,500 0,250
A measure for the quality of our model is derived
from Measurement values - Fits = Residuals. The
Residuals should be as small as possible and their
mean should be 0mean should be 0
First statement about the variances
Analysis of Variance for Yield (coded units)
Some values can’t be calculated
because the model includes all the
interactions On the other side weMain Effects 3 1112,50 1112,50 370,833 * *
Residual Error 0 * * *
interactions. On the other side we
have just single values.
Total 7 1317,50

The Graphical Evaluation of the Effects
Pareto Chart Normal distribution
We see the effects declining in
accordance to their importance.
The red line is our chosen
Minitab shows the effects in a
normal distribution diagram. The
significant deviation of terms is
significance limit. highlighted.
A
5,97
A Temp
B C onc
Factor Name
Pareto Chart of the Effects
(response is Yield, Alpha = 0,10)
99
95
Not Significant
Significant
Effect Type
Normal Plot of the Effects
(response is Yield, Alpha = 0,10)
AB
B
AC
Term
B C onc.
C C atal. 90
80
70
60
50
40
30
Percent
A Temp
B C onc.
C C atal.
Factor Name
g
AC
A
BC
ABC
C
22000 22000
30
20
10
5
1
2520151050
Effect
Lenth's PSE = 2,25
2520151050-5
Effect
Lenth's PSE = 2,25
Simplification of the Model
As introduced before we are allowed to simplify our model to the significant
factors and interactions. The result will be more accurate because the
degrees of freedom are assigned more realistically Reduce step by step!degrees of freedom are assigned more realistically. Reduce step by step!
Estimated Effects and Coefficients for Yield (coded units)
T Eff t C f SE C f T P
Significant factors usually show
a P – value < 0,05. (Probability
f i fl 95%)
Constant 64,250 0,4564 140,76 0,000
Temp 23,000 11,500 0,4564 25,20 0,000
Conc. -5,000 -2,500 0,4564 -5,48 0,012
Catal 1 500 0 750 0 4564 1 64 0 199
of influence > 95%)Catal. 1,500 0,750 0,4564 1,64 0,199
Temp*Catal. 10,000 5,000 0,4564 10,95 0,002
S = 1 29099 PRESS = 35 5556S 1,29099 PRESS 35,5556
Analysis of Variance for Yield (coded units)y ( )
Main Effects 3 1112,50 1112,50 370,833 222,50 0,001
2-Way Interactions 1 200,00 200,00 200,000 120,00 0,002
A small portion of variance is not
explained in the reduced model, in
Residual Error 3 5,00 5,00 1,667
Total 7 1317,50
Minitab noted as error.

R2 and R2 adj.: Practical Significance
S = 1,29099 PRESS = 35,5556
• R² is a method within the
statistics, to show the practical 9962,0
5,1312Re2
===
gressionSS
Rstatistics, to show the practical
significance of an effect.
996,0
5,1317TotalSS
• R² adj. is a similar method to explain the practical significance of anR adj. is a similar method to explain the practical significance of an
effect. It is helpful, if we use several factors in a model. E.g. R2 adj. gets
smaller, if an additional factor is added in the model, because every
reduction of SS can be balanced by the loss of degrees of freedomreduction of SS error can be balanced by the loss of degrees of freedom.
The values for R² adj. are always a little bit smaller than for R².
6671MS
9911,0
7
5,1317
667,1
112
=−=−=
Total
Total
Error
DF
SS
MS
adjR
Total
• S is the pooled standard deviation (averaged within group variation) The
square root of S is the MS Error
square root of S is the MS Error.
PRESS and R2 (pred): Significance for Prediction
Prediction Sum of Squares (PRESS)
The predictive ability of the model will be assessed with the statistic PRESS. In
l th ll th l PRESS th b tt th di ti bilit f th d lgeneral, the smaller the value PRESS, the better the predictive ability of the model.
PRESS is used for the calculation of predicted R2. The interpretation of R2 (pred) is in
general more intuitive. The combination of these statistics can help to avoid an over
adjustment of model because it uses observations for the calculation which are notadjustment of model because it uses observations for the calculation which are not
considered in the model estimation. An over adjustment of models exists, which
explains apparently the relation between predictor and response variable based on
the data set used for the model calculation but which don’t deliver valid prediction forthe data set used for the model calculation, but which don t deliver valid prediction for
new observations.
PRESS is similar to the residual (Error) Sum of Squares (SSE) and presents the sum
of squares prediction error. PRESS is different to SSE, because each adjusted value,
ith, will be calculated for PRESS in the following procedure: Initially every ith
observation will be excluded from the data set. Subsequently the regression equation
ill b ti t d b d th i i 1 b ti d th di t d lwill be estimated based on the remaining n -1 observations and the predicted value
will be calculated with the help of the adjusted regression equation .
The predicted R2 indicates how well the model predicts responses for new observationsThe predicted R2 indicates, how well the model predicts responses for new observations.
973,0
51317
56,35
11)(2
=−=−=
SS
PRESS
predR
5,1317
)(
TotalSS
p

Graphical Support at the Evaluation
Al t t ith i t ti di d l t th i ifi tStat
>DOE
>Factorial
Always start with an interaction diagram and select the significant
effects. In the second step review the significant main effects which
have no interactions.
85
80
160
180
Temp
Interaction Plot for Yield
Data Means
>Factorial
>Factorial Plots…
75
70
65
Mean
Interaction plot
The slope indicates the significance of the
60
55
50
interactions.
M i Eff t Pl t f Yi ld
1-1
Catal.
67
66
Main Effects Plot for Yield
Data Means
Mean value of all trials
65
64
Mean
Mean value of all trials
with high level setting
4020
63
62
Mean value of all trials with
low level setting
4020
Conc.
low level setting
Graphical Support at the Evaluation
Stat
>DOE
>Factorial>Factorial
>Factorial Plots…
The highest value for yield will be
received with the following setting:
Temp: 180 degrees CTemp: 180 degrees C
Catalyst: with Catalyst 1
Concentration: 20%Concentration: 20%

The Multivari Chart
Stat
>Quality Tools
>Multivari Chart
Multi-Vari Chart for Yield by Catal. - Temp
>Multivari Chart…
80
4020
160 180 Catal.
-1
1
Yield
70
6060
50
Conc.
4020
40
Panel variable: Temp
The catalysts affect yield differently as a function of temperature . Changes
Panel variable: Temp
in the concentration have no big effect on the yield, the results are similar.
High temperature (if controllable) in combination with catalyst 1 shows the
b t lt i d d t f t ti tti
best results independent of concentration setting.
The Significance of the Model
Possibilities of model diagnostics in Minitab
• Normal distribution of the residuals: The observations should follow
a straight line in this diagram. Small deflections at both ends are
acceptable. Points fairly outside indicate an effect not considered
in this model.
• Histogram of residuals: Usually one expects a normal distribution
with a mean of 0. Strong deviations are indications of effects from
other factors (not included in this experiment)other factors (not included in this experiment)
• Run chart, (I-Chart) of residuals: Shows trends of the experiment.
Special causes will be highlighted by Minitab.
• Residuals against fits ( Calculated results): This plot should show ag ( ) p
random pattern of the residuals on both sides of the baseline. Pay
attention to patterns indicating trends.

Residual Diagrams
Stat
>DOE
>Factorial
Normal Probability Plot of the Residuals Residuals Versus the Fitted Values
Residual Plots for Yield
>Factorial
>Graphs…
rcent
99
90
50
sidual
1,0
0,5
0,0
Normal Probability Plot of the Residuals Residuals Versus the Fitted Values
Residual
Per
210-1-2
10
1
Fitted Value
Res
80706050
-0,5
-1,0
ncy
2,0
1,5
ual
1,0
0,5
Histogram of the Residuals Residuals Versus the Order of the Data
Frequen
1,00,50,0-0,5-1,0
1,0
0,5
0,0
Residu
87654321
0,0
-0,5
-1,0
Residual
,,,,,
Observation Order
Problem, Electrical Test of PCB
The portion of the retest is 28,8%... As a critical factor the quality of
the electrical contact has been determined.
The properties of the factor metal pin were investigated with a DOE.
DOE Planning :
• 3 factors, with 2 levels each:
Supplier : A - B
Diameter : thick - thin (diameter of the contact pin)
Material type : Cu - Ag
• Full factorial design with 2 repeats(2 tests per combination)
• Scope: 2 sets with 25 pieces each
• Result = Amount of retest for 2 x 25 pieces for each factor
combination
combination

Application of the Planned Design
th mat sup retest
thin c A 6
Stat
>DOE
>Factorial
1
thin c A 6
thick a A 7
thick c A 3
>Factorial
thin c A 7
thick c A 4
thick a B 4thick a B 4
thin c B 5
thin a A 5
2
3
thick c B 2
thick c B 1
thick a B 4thick a B 4
thin c B 4
thin a A 6
File: DOE Retest.mtw
First Evaluation
Aim: what´s significant
Factorial Fit: retest versus th; mat; sup
Estimated Effects and Coefficients for retest (coded units) Pareto Chart of the Standardized Effects
(response is retest, Alpha = 0,05)
Constant 4,6250 0,1976 23,40 0,000
th -1,7500 -0,8750 0,1976 -4,43 0,002
mat 1,2500 0,6250 0,1976 3,16 0,013
sup -1,5000 -0,7500 0,1976 -3,79 0,005
C
A
2,306
A th
B mat
C sup
Factor Name
p , , , , ,
th*mat 1,2500 0,6250 0,1976 3,16 0,013
th*sup -0,5000 -0,2500 0,1976 -1,26 0,242
mat*sup 0,5000 0,2500 0,1976 1,26 0,242
th*mat*sup -0,5000 -0,2500 0,1976 -1,26 0,242
BC
ABC
B
AB
Term
S = 0,790569 PRESS = 20
AC
BC
543210
Standardized Effect
Analysis of Variance for retest (coded units)
Main Effects 3 27,5000 27,5000 9,1667 14,67 0,001
2-Way Interactions 3 8,2500 8,2500 2,7500 4,40 0,042y , , , , ,
Residual Error 8 5,0000 5,0000 0,6250
Pure Error 8 5,0000 5,0000 0,6250
Total 15 41,7500

The Reduced Model
Here we get the best model from our experimental design
Pareto Chart of the Standardized Effects
(response is retest, Alpha = 0,05)
Factorial Fit: retest versus th; mat; sup
Estimated Effects and Coefficients for retest (coded units)
A
2,201
A th
B mat
C sup
Factor Name
Constant 4,6250 0,2132 21,69 0,000
th -1,7500 -0,8750 0,2132 -4,10 0,002
mat 1,2500 0,6250 0,2132 2,93 0,014
sup -1,5000 -0,7500 0,2132 -3,52 0,005
AB
C
Term
p , , , , ,
th*mat 1,2500 0,6250 0,2132 2,93 0,014
S = 0,852803 PRESS = 16,9256
B
43210
Standardized Effect
q , q(p ) , q( j) ,
Analysis of Variance for retest (coded units)y ( )
Main Effects 3 27,500 27,500 9,1667 12,60 0,001
Residual Error 11 8,000 8,000 0,7273, , ,
Lack of Fit 3 3,000 3,000 1,0000 1,60 0,264
Pure Error 8 5,000 5,000 0,6250
Total 15 41,750
The Necessary Model Diagnostic
Stat
>DOE
>Factorial
Normal Probability Plot Versus Fits
Residual Plots for retest
>Factorial
>Graphs…
99
90
50
cent
1,0
0,5
0 0
dual
y
21012
50
10
1
Perc
65432
0,0
-0,5
-1,0
Resid
210-1-2
Residual
65432
Fitted Value
Histogram Versus Order
4,8
3,6
2,4
equency
1,0
0,5
0,0
esidual
1,51,00,50,0-0,5-1,0
1,2
0,0
R id l
Fre
16151413121110987654321
-0,5
-1,0
Observation Order
Re
Residual Observation Order

Graphical Presentation of the Results
Stat
>DOE
>Factorial
Interaction Plot (data means) for retest
First we interpret the interaction diagram…
>Factorial
>Factorial Plots…
5,5
5,0
4,5
thin
thick
th
Mean
4,0
3,5
mat
ac
3,0
2,5… subsequent the main effects
Main Effects Plot (data means) for retest
5,5
5,0
4,5
th mat
a ec s o (da a ea s) o e es
Meanofretest
thickthin
4,0
3,5
ac
5,5
sup
5,0
4,5
4,0
3,5
BA
Translation to Reality
Multi-Vari Chart for retest by th - sup
Stat
>Quality Tools
>Multivari Chart
7
ac
A B
thin
th
y p>Multivari Chart…
7
6
thin
thick
etest
5
4
re
3
2
1
mat
ac
Panel variable: sup

The Practical Meaning of the Results
C o s t 23 key A-products identified, test time average 60,24 sec
new test time average at 45 90 sec : improvement 25 6%new test time average at 45,90 sec : improvement 25,6%
Retest rate measured for
Q u a l i t y
Retest rate measured for
11 key A-products : 28,8%
new retest rate at 6,9% :new retest rate at 6,9% :
improvement 21,9%
Stability Maintenance time for 23 key A-products at 84 min/day
new maint. time at 49 min/day : improvement 41,6%
The Helicopter Exercise
•• Purpose:Purpose: Investigation of a helicopter design with a 2k factorial
experimentexperiment
•• Goal:Goal: OptimizeOptimize airborne time
•• Output:Output: Airborne: time between launch until first ground contact
•• Procedure:Procedure:•• Procedure:Procedure:
• Select 3 factors for the investigation.
• Perform a 2x2x2 factorial experimental design with two
repeats per treatment.
H ld th fli ht h i ht t t!• Hold the flight height constant!
• Define the measurement system first!!!
• Follow the analysis “roadmap” and present your results on a
flipchart.

Wing length
2 2
Weight
• short
• long
1 1
• 1st clip
• 2nd clip• long • 2nd clip
DOE 1DOE 1
Shaft length
• short
Shaft width
• small 2 21 1
• long
small
• wide
11
2 2
Steps of the experiment
• Define the problem
• Establish the goal
• Select the output variablesp
• Select the input variables
• Define the levels of the variablesDefine the levels of the variables
• Select the experimental design
• Determine the tasks in the team• Determine the tasks in the team
• Collect the data
A l th d t• Analyze the data
• Draw statistical conclusions
• Replicate the results
• Develop practical solutions
• Implement the solutions

Example: Customer Survey
C t h th i j d t i d h th t b t• Customers have their own judgment in order whether to buy or not
to buy this product.
• Lets take 3 important criteria. We want to figure out how they affect
the decision of potential buyers.
• Here are criteria for buying a laptop
Weight (g) Battery Capacity (h) Display Size (cm)
1500 2 20
2200 3 302200 3 30
Example: Customer Survey
Please rate the interest for a purchase for the following combination of
factors (Types of Laptops) on a scale of 1 to 10 (10 = strongest interest)
First as an individual, than in the team
RunOrder Weight Capacity Display Rating
1 1500 2 20
2 2200 2 20
3 1500 3 20
4 2200 3 204 2200 3 20
5 1500 2 30
6 2200 2 306 2200 2 30
7 1500 3 30
8 2200 3 30
Now enter your rating in the Minitab file Laptop1.mpj
y g p p pj

Final Report
The report should include the following items:
• SummarySummary
• Problem description and background
• GoalsGoals
• Output variables
• Input variablesInput variables
• Experimental design
• Process / ProcedureProcess / Procedure
• Results and data analysis
• ConclusionsConclusions
Attachments
• Detailed data analysisDetailed data analysis
• Original data, if available
• Details of the instruments and procedure
Details of the instruments and procedure
Be Pro-active
DOE i ti t l• DOE is a pro-active tool.
• There are no bad experiments – only poorly planned andThere are no bad experiments only poorly planned and
performed ones.
N t i t ill h ith b kth h di i• Not every experiment will show up with breakthrough discoveries
for the world.
• Every experiment teaches you something.
N d t b i ti d lt i f ll• New data bring new questions and results in follow up
studies.

Summary
• Ways to learn
• Components of an experiment
• Experimental validation
• Steps for planning an experiment• Steps for planning an experiment
• 2k factorial designg
• Practical exercises
• Final report
Appendix DOE
Terminology
Planning
Examples

Terminology
Design (Layout): Complete specification of experimental runs,
which include block building, randomization, replicates, repeats
and attribution of factor/level combinations to experimental unitsand attribution of factor/level combinations to experimental units.
2k x 3k x 3k… factorial: Description of a basic design. A 2 x 3 x 3
design having three input variables, one with two levels and two
with three levels. The number of experimental runs (treatments)
is the product of the levels In this case we have 18 treatmentsis the product of the levels. In this case we have 18 treatments.
Response unit: A unit under observation and measured during the
experiment Also called analysis unitexperiment. Also called analysis unit.
Treatment combination: An experimental run with defined levels
f ffor all input variables, referred to as cell.
Balanced design: A design with equal numbers of experimentalg g q p
runs in every treatment combination or run.
Unbalanced design: A design with a uneven number of
Unbalanced design: A design with a uneven number of
experimental runs per treatment combination.
Terminology, Continued
Repetition: Running several samples on one treatment
combination.
Replication: Replication (repeating) of the entire experiment.
Effect (main effect): The average change in the response variable
due to the change of a factor from one level to another.
Interaction: Exists when an effects of one factor of the response
depends on the setting of other factors.
Experimental area: All possible factor / level combinations where
an experiment could be carried out.
Test run: One or more observations of the output variable for a
single combination of the experiment.g p
Confounding: One or more effects that cannot be separated
properly and assigned to a factor or a interaction
properly and assigned to a factor or a interaction.

Exercise
A friend of yours brags that he is able to differentiate several
types of beer from each other, especial “Jever” by tasting these
beers.
You ask him to prove it.
You plan an experiment:
Define purpose & goal− Define purpose & goal
− Outputs
− Inputs
> Controllable inputs
> Not controllable inputs
(noise) Review and rate your draft for(noise)
− Scheme of randomization
Review and rate your draft for
internal and external validity.
Present your results on a flip chart.
General Items for a DOE Planning
• Involvement of a team (cross-functional)
• Maximize prior knowledge
Pursue measurable objectives• Pursue measurable objectives
• Plan the execution of all phases (including confirmation)
• Rigorous sample size determination
• Allocate sufficient resources for data collection and analysis
• Write and review proposalp p
See also week 1
Module 05 “Thought Map”
last 3 pages
last 3 pages

Steps of Planning
• Define the problem with business scores (RTY, COQ, Capacity,
Productivity)Productivity)
• Name the goals of the experiment
• Define the output variables
• Define the input variables (factor selection)
• Define the levels for the input variablesp
• Choice an design for the experiment
S l k 1
• Plan and provide equipment, material, operator
• Review you proposal See also week 1
Review you proposal
last 3 pages
Questions with Respect to Planning
• What is the measurable goal?
• What will it cost?
• H d t bli h th l i ?• How do we establish the sample size?
• How does the plan for randomization looks like?
• Are our internal customers informed?
• What time will it take?• What time will it take?
• How do we analyze the data?
• Did we set up a control plan?
See also week 1
last 3 pages
last 3 pages

Experimental Goals
Centering or reduction of variation?
How small are the changes you want to detect (experimentalHow small are the changes you want to detect (experimental
“Delta”)?
Examples:Examples:
• Determination of the effect of material change on product reliability
• Defining of causes for the variation for a critical process
• Evaluation of the effect of cheaper material on the product performanceEvaluation of the effect of cheaper material on the product performance
• Determination of the effect of variation of the operator on the final
product/serviceproduct/service
• Evaluation of cause – effect relations on process inputs and product
characteristicscharacteristics
Usually investigation and determination of the effect of several factors on
the response (output)
Definition of the Output Variables
• Is the output qualitative or quantitative?
• Objective: centering variation reduction or both?Objective: centering, variation reduction, or both?
• What is the baseline? (mean and standard deviation evaluation)
• Is the output under statistical control?
• Does the output vary over time?• Does the output vary over time?
• How large a change in the output do you want to detect?
• Is the output normal distributed?
• How do we measure the output?
• Is the measurement system adequate?y q
• Are there multiple outputs? What are the priorities for these?

Selection of Factors
The initial identification of factors should include at least the following
sources:
• Process map
• Cause and effect matrix
• FMEA
• Multivari
• Literature review
• Brainstormingg
• Scientific theory
• Operator experience
• Customer/supplier inputs
• Ranking methods (or nominal group technique)g ( g )
Final selection of factors based on prioritization techniques,
technical knowledge, FMEA RPN’s, etc.
Factor Selection
List all KPIV’s and KPOV’s of the complete process
Identif al e adding and non al e adding steps in the processIdentify value adding and non value adding steps in the process
List for all sub process inputs and outputs
Split the inputs (factors) in controllable, non controllable (noise)
and SOP partsp
Define the process critical inputs of the current process
Use prioritizing tools to select the factors

Factor Level Selection
After factor definition the level settings have to be chosen. The first
Goal: Distinguish the vital inputs from a big number of inputs
(Screening)(Screening)
• If we vary the factors to extremes we will see an effect on select
broad or “bold” levels to include all possible components ofbroad or bold levels to include all possible components of
variation of the current process
• the response if there is one• the response if there is one
• May exaggerate the variation (unrealistic variation)
• “unrealistic” variation may be created
Examples of broad settings:p g
• Qualitative:
Method A vs B or Reactor 1 vs Reactor 2Method A vs. B or Reactor 1 vs. Reactor 2
• Quantitative:
5 minutes vs. 15 minutes or 2 bar vs. 4 bar
Factor Level Selection
If the vital factors have been determined the goal is now to
understand the interactions between the factors
(Ch t i ti )(Characterization)
− Information form earlier experiments will used to adjust the
f t tti di l t i th i t ti f thfactor settings accordingly to recognize the interactions of the
inputs
U ll th l l ill b t l F ll hi h l ti− Usually the levels will be set closer. Full or high resolution
fractional factorials are recommended
N t l i t id tif th ti i d f f i tNext goal is to identify the operating window of a group of input
variables and understand the experimental space near the
optimum (Optimization)p ( p )
− Levels of factor will be set closer together, usually
Th i t h ll b f f t ith− The experiments have a smaller number of factors with an
increased number of levels
S i l DOE d i ill b ft d ( W k 3 & 4)
− Special DOE designs will be often used (see Week 3 & 4)

Steps of the Experiment
Define the problem
Establish the goalEstablish the goal
Select the output variables
S l t th i t i blSelect the input variables
Define the levels of the variables
Select the experimental design
Determine the tasks in the team
Collect the data
Analyze the dataAnalyze the data
Draw statistical conclusions
Replicate the resultsReplicate the results
Develop practical solutions
Implement the solutions
General Recommendations
• Make sure that a business benefit is connected with your project
( i t)(experiment).
• Try not to answer all questions with one study. Rely on several
studies in a sequence.. (Rule of the thump: Spend less than 25%
of your budget for the first experiment)
• Use design with 2 levels in early project stages
• Proof your results always with a confirmation run• Proof your results always with a confirmation run
• Be prepared for changes!
• A final report is required!

Javier Garcia - Verdugo Sanchez - Six Sigma Training - W2 Design Of Experiments (DOE) Intro

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Javier Garcia - Verdugo Sanchez - Six Sigma Training - W2 Design Of Experiments (DOE) Intro

Similar to Javier Garcia - Verdugo Sanchez - Six Sigma Training - W2 Design Of Experiments (DOE) Intro (20)

More from J. García - Verdugo

More from J. García - Verdugo (14)

Recently uploaded

Recently uploaded (20)

Javier Garcia - Verdugo Sanchez - Six Sigma Training - W2 Design Of Experiments (DOE) Intro