DOE - KVL
(7.1)
Design Of Experiments
(031023)
Example: case study I
Day 1 1. 13:00-13:45h - Introduction; what was experimental design again?
2. 14:00-14:45h - Design in one formula: y = X.b
3. 15:00-15:45h - Statistical inference and ANOVA
Day 2 4. 09:00-12:00h - Hands-on DOE: computer exercise I *)
5. 13:00-13:45h - Data inspection and plotting (+ some PCA)
6. 14:00-14:45h - Miscellaneous subjects
7. 15:00-15:45h - Example: case study I
Day 3 8. 09:00-12:00h - Hands-on DOE: computer exercise II *)
9. 13:00-13:45h - Blocking and split-plot
10. 14:00-14:45h - Example: case study I
11. 15:00-15:45h - Introduction to mixed linear models
Participants can choose to take part in the theory (3 afternoons) sessions or theory + computer exercise
*)
(2 mornings) sessions.

DOE - KVL
(7.2)
Case study I
Design Of Experiments

DOE - KVL
(7.3)
Case study I
Color change during meat storage

DOE - KVL
(7.4)
Design Of Experiments
Different research areas
• Replicates are ‘perfect clones’
• Experiments are cheap
• Simulations give ‘perfect statistics’
• Motivation for a lot of developments
• Reasonably reproducible
• Sparse designs
• THE example in literature
• Natural variation
• Experiments are expensive
• More replicates
• More input of the experimenter

DOE - KVL
(7.5)
Case study I
Color change during meat storage
Loss of redness of pork-meat during storage time under different packaging conditions
© Lisbeth Dahlgård Nannerup, MLI/LMC Kvali-MAP project,
Department of Food Chemistry, KVL, Denmark
• Response variable is the color (redness) of the meat
• Measured as a so-called a-value
• Higher redness is more attractive towards consumers (optimization)

DOE - KVL
(7.6)
Case study I
Color change during meat storage
8 replicates ( )
The effects investigated are packaging conditions
• Storage days (12 levels)
• Percentage oxygen (4)
• Product/Headspace ratio (3)
• Temperature (3)
• Light exposure (3)

DOE - KVL
(7.7)
Case study I
Color change during meat storage
The experiment is a five (or six if you count replicates) dimensional factorial design
Days x Oxygen x Product/Headspace x Temperature x Light (x Animals)
?
1D 2D 3D … 6D

DOE - KVL
(7.8)
Case study I
Color change during meat storage
• 4 x 3 x 3 x 3 x 8 = 864 samples
• 864 x 12 = 10368 measurements
t (days)
0 21 34
L (Lux)
0 600 1200
10
T(°C) 8
5
1:1
P/H 1:1.5
1:3
0.0 0.5 1.0 1.5 O2 (%)

DOE - KVL
(7.9)
Color loss in pork
Quality of the response
• Each measurement outcome is based on 5 readings
• Minolta color measurement (a, b and L-value)
• 8mm diameter serves area
• a-value is used to express redness of meat
10
a-value
Standard deviation
over readings (σ) 0
0 5 10 15 day 20 25 30 35

DOE - KVL
(7.10)
Color loss in pork
Quality of the response
10 (randomly selected) samples are used to compute
the pooled standard deviation
3
σSamples
0
2
Measurement error :
1.58/√5 = 0.7 a-values
σpooled
σpooled all samples = 1.59
1
0 5 10 15 day 20 25 30 35

DOE - KVL
(7.11)
Color loss in pork
Replicates
10
5
a-value
0
Two design points O2 = 0.0% P/H = 1:1 T = 5°C L = 0Lux
-5
x8
10 O2 = 1.5% P/H = 1:3 T = 10°C L = 1200Lux
5
a-value
0
-5
0 5 10 15 day 20 25 30 35

DOE - KVL
(7.12)
Color loss in pork
Replicates?
Mean a-value over
x2 per production run
all design points
Storage (days)
8
a-value
First experimental run
2
8
a-value
Second experimental run
2
0 35 0 35
day day

DOE - KVL
(7.13)
Color loss in pork
Expected trends
Mean a-value over
Oxygen (%) Product/Headspace
8
all design/time points
a-value
2
0.0 0.5 1.0 1.5 1:1 1:1.5 1:3
Temperature (°C) Light (Lux)
8
a-value
2
5 10 0 600 1200
8

DOE - KVL
(7.14)
Color loss in pork
Response surface model for day 21
Storage time in the setup of this experiment requires special methods.
We will treat the data from day 21 as single design (108 points x 8 replicates)
L (Lux)
0 600 1200 Day 21
10
T(°C) 8
5
1:1
P/H 1:1.5
1:3
0.0 0.5 1.0 1.5 O2 (%)

DOE - KVL
(7.15)
Color loss in pork
Response surface model for day 21
An ANalysis Of Variance shows that the temperature as main effect and
interactions are not important.
√
√
X
√
√
X
√
X
√
X
X
√
X
X
X

DOE - KVL
(7.16)
Color loss in pork
Response surface model for day 2, 21 and 34
An ANalysis Of Variance shows that the temperature as main effect and
interactions are not important.
Day 2 Day 21 Day 34

DOE - KVL
(7.17)
Color loss in pork
Response surface model for day 21
An ANalysis Of Variance shows that the temperature as main effect and
interactions are not important.

DOE - KVL
(7.18)
Color loss in pork
Response surface model for day 21
a-value = b0 + b1 xO2 + b2 xP/H + b3 xL + b4 xO2 xP/H + b5 xO2 xL + b6 xP/H xL + b7 xO2.P/H xL
= y = X.b b = (X’.X)-1.X.y =
Confidence interval
b0 11.37 10.85 11.89
b1 (O2) -0.08 -0.63 0.48
b2 (P/H) -0.30 -0.56 -0.05
-0.2x10-3 -0.8x10-3 0.5x10-3
b3 (L)
b4 (O2.P/H) -0.07 -0.34 0.20
1.5x10-3 0.8x10-3 2.2x10-3
b5 (O2.L)
-0.4x10-3 -0.7x10-3 -0.0x10-3
b6 (P/H.L)
-1.2x10-3 -1.6x10-3 -0.8x10-3
b7 (O2.P/H.L)

DOE - KVL
(7.19)
Color loss in pork
Model residuals for day 21
0.999
0.99
0.95
0.90
0.75
0.50
0.25
0.10
0.05
0.01
0.001
-6 -4 -2 0 2 4 -6 -4 -2 0 2 4
Residuals (mildly) skewed towards low values, but for now we
assume ANOVA is ‘robust’ against this non-normality

DOE - KVL
(7.20)
Color loss in pork
Model residuals for day 21
10
Product/Headspace
1:1
1:1.5
0
1:3
-10
0 10
10
Light
0
600
0
1200
-10
0 10

DOE - KVL
(7.21)
Color loss in pork
Response surface for day 21
Light = 0Lux
1.5
6.6
6.8
6.4
6 .2
1
O2 (%)
6.6
6.4
6.8
0.5
6.6
7
6.
4
6.8
0
1:1.5
1:1 1:3
P/H

DOE - KVL
(7.22)
Color loss in pork
Response surface for day 21
Light = 600Lux
1.5
6.6
5.8
5.2
6.2
5 .6
3.
6
6
5
3. 4
4.4 .6
8
4
6.4
5.4
4.
4.
2
1
8
4.
5.2
4
5
5.6
5.8
6.6
O2 (%)
4.
6.2
6
6
4.
8
5.
6.4
4
5.2 5
0.5
5.
5. 6
8
6 5.4
6.2
6.6
5.6
5.8
6.
0
4
1:1.5
1:1 1:3
P/H

DOE - KVL
(7.23)
Color loss in pork
Response surface for day 21
Light = 1200Lux
1.5
6.6
12
1..4
5.6
3.8 4
5.2
1
4.2 4.4
2.4 2.6.8
3.6
6
4.8
6.2
5.8
5
3 3 .2
6.4
2
1. .8
3.4
12
2.
6
2
5.4
4.6
1 2.
4
3 2.
3. 4
2. 6
3.
8
4.2
3.
5.2 8
O2 (%)
6
5.6
3. 2
4.8
4
6
5
5.8
6.2
6.4
4.
4
3 .8 3 .6
5.4
0.5
4.
4.2
6
4
4.
5. 8 4.4
5
2
5.
4.6
6
5.
6
6.2
5.4
8
6 .4
4.8
5
0
1:1.5
1:1 1:3
P/H

DOE - KVL
(7.24)
Color loss in pork
Response surface for day 21
8
Light = 0Lux
a-value
Light = 600Lux
Light = 1200Lux
1
1.5
1.0
1:1
1:1.5 0.5 Oxygen (%)
1:3 0.0
Product/Headspace

DOE - KVL
(7.25)
Color loss in pork
Response surface for day 2
8
Light = 0Lux
a-value
Light = 600Lux
Light = 1200Lux
1
1.5
1.0
1:1
1:1.5 0.5 Oxygen (%)
1:3 0.0
Product/Headspace

DOE - KVL
(7.26)
Color loss in pork
Response surface for day 34
8
Light = 0Lux
a-value
Light = 600Lux
Light = 1200Lux
1
1.5
1.0
1:1
1:1.5 0.5 Oxygen (%)
1:3 0.0
Product/Headspace

DOE - KVL
(7.27)
Color loss in pork
Response surface for days 2, 21 and 34
Day 2 Day 21 Day 34
A workable model of for all the design factors,
supported by inspecting the collecting (raw) data, and…
‘Models are to be used, not believed’ (Henri Theil)

DOE - KVL
(7.28)
Data transformations
Non-normal residuals?
0.999
0.99
0.95
0.90
0.75
0.50
if y = a + bx 0.25
0.10
0.05
then y = a + bx 0.01
and s y = b s x 0.001
-6 -4 -2 0 2 4
(linear transformation leave the
residual distributions unattached)
y = log( x + constant ) Log-normal distributions are often fond in
nature (weight, size, etc.), time-series and
1
y= y= x y = x2 growth models
x
y = b0exp(b1x) ln(y) = ln(b0) + b1x
y’ = b0’ + b1x

DOE - KVL
(7.29)
Data transformation
Variance stabilizing
= (x + constant)λ
Sometimes a so-called Box-Cox transformation of the form y
can help to ‘stabilize’ (make more similar) the model errors
The ‘optimal’ transformation is
the one that minimizes the
sum-of-squares of residuals as
a function of λ
(Maximum Likelihood):
SSe
( x λ − 1)
y (λ ) =
λx λ −1
y (0) = x ln( x)
( ln x / n )
x =e∑ 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5
λ

DOE - KVL
(7.30)
Color loss in pork
Day 21 after transformation
0.999
0.99
0.95
0.90
0.75
0.50
0.25
0.10
0.05
0.01
0.001
-80 -60 -40 -20 0 20 40 60 80 -80 -60 -40 -20 0 20 40 60

DOE - KVL
(7.31)
Color loss in pork
Day 21 after transformation
Before
transformation

DOE - KVL
(7.32)
Color loss in pork
Day 21 after transformation
Product/Headspace
200
1:1
100
1:1.5
1:3
0
-100
0 100 200
200 Light
0
100
600
0
1200
-100
0 100 200

DOE - KVL
(7.33)
Color loss in pork
Day 21 after transformation
8
Light = 0Lux
Light = 600Lux
a-value
Light = 1200Lux
1
1.5
1.0
1:1
1:1.5 0.5 Oxygen (%)
1:3 0.0
Product/Headspace

DOE - KVL
(7.34)
GEMANOVA
Multiplicative-model analysis of the DOE-data
GEneralized Multiplicative ANalysis Of VAriannce (GEMANOVA)
Classical ANOVA - Additive model:
color = b0 + b1 xO2 + b2 xP/H + b3 xL + b4 xO2.P/H + b5 xO2.L + b6 xP/H.L + b7 xO2.P/H.L
GEMANOVA - Multiplicative model:
color = c0 cA ct cO2 cP/H cT cL

DOE - KVL
(7.35)
GEMANOVA = PARAFAC
Parallel factor analysis
Principal Component Analysis (PCA)
X E
= + +
PARAFAC
= + +
X E
A factor model in three (or more!) dimensional space
with scores/loadings in three (or more) directions

DOE - KVL
(7.36)
GEMANOVA
One factor model, all effects free
color = c0 cA ct cO2 cP/H cT cL c0 = 1092
Animal
0.37
0.29
0.28
0.33
Storage (days)
2 4 6 8 0 10 20 30
Prod./Headsp.
0.52
Oxygen (%) 0.60
0.49
0.52
1:1
0 0.5 1 1.5 1:1.5 1:3
0.62
Light (Lux)
Temperature (°C)
0.579
0.577 0.56
0 600 1200
5 8 10

DOE - KVL
(7.37)
GEMANOVA
One factor model, all effects free
color = c0 cA ct cO2 cP/H cT cL + e
Error
-6 -4 -2 0 2 4
Data range = [0.98 – 14.82]; RMSPfit = 1.47 color-values; R2 = 0.41

DOE - KVL
(7.38)
Jackknife re-sampling
Uncertainty estimation
θ = t (F ) F : Cumulative Distribution Function
()
θ = t F = u (x )
ˆ ˆ
x : data; plug-in principle
ˆ
P( sα / 2 ≤ θ ≤ s1−α / 2 ) = 1 − α
α-coverage probability
^ ^
θ − t df ;α / 2 (se ) ≤ θ ≤ θˆ − t df ;1−α / 2 (se )
ˆ
Uncertainty estimate
Estimates from new distribution
()
θ * = t F * = u (x* )
ˆ ˆ found from some re-sampling
strategy (*)
θ − θ ⇔ θˆ* − θˆ
ˆ The re-sampling assumption!

DOE - KVL
(7.39)
Jackknife re-sampling
Uncertainty estimation
n = #samples (= 8 animals)
(.) : re-sampling expectation
(bias J ) = (n − 1)(θˆ(.) − θˆ )
^
*
Jackknife bias estimate
( )
(seJ ) = n − 1 ∑ θˆ(*i ) − θˆ(.)
^ 2
*
Jackknife standard error estimate
n
^ ^
θ − zα / 2 (se ) ≤ θ ≤ θˆ − z1−α / 2 (se )
ˆ Assume normal distribution

DOE - KVL
(7.40)
GEMANOVA
One factor model, Jackknife 5% coverage probability
color = c0 cA ct cO2 cP/H cT cL c0 = 1052 - 1092 - 1133
RMSPfit = 1.47
Animal
0.38
0.29
R2 = 0.41
Storage (days)
0.32 0.27
2 4 6 8 0 10 20 30
Prod./Headsp.
Oxygen (%)
0.52 0.60
0.52
0.48
0 0.5 1 1.5 1:1 1:1.5 1:3
Light (Lux)
T (°C) 0.62
0.58
No effect!
0.57
0.54
5 8 10 0 600 1200

DOE - KVL
(7.41)
GEMANOVA
One factor model, Temperature effect eliminated
color = c0 cA ct cO2 cP/H 1T cL c0 = 607 - 631 - 654
RMSPfit = 1.47
Animal
0.38
0.29
R2 = 0.41
(same performance)
Storage (days)
0.32
0.27
2 4 6 8 0 10 20 30
Prod./Headsp.
Oxygen (%)
0.52 0.60
0.52
0.48
0 0.5 1 1.5 1:1 1:1.5 1:3
Light (Lux)
Temperature (°C) 0.62
1
0.54
5 8 10 0 600 1200