Design of experiments

1. Department of Biochemical and Chemical Engineering
Process Dynamics and Operations Group (DYN)
D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
13.06.2016
Optimal dynamic experiment design for
guaranteed parameter estimation
A.R. Gottu Mukkula and R. Paulen
Technische Universität Dortmund

2. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
2
Mathematical modelling
 Mathematical models play an important role in:
 Process design
 Control system design
 Process optimization

3. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
2
Mathematical modelling
 Mathematical models play an important role in:
 Process design
 Control system design
 Process optimization
 Steps for model development:
 Specification of model structure
 Design and realization of experiments
 Estimation of uncertain model parameters

4. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
2
Mathematical modelling
 Mathematical models play an important role in:
 Process design
 Control system design
 Process optimization
 Steps for model development:
 Specification of model structure
 Design and realization of experiments
 Estimation of uncertain model parameters

5. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
2
Mathematical modelling
 Mathematical models play an important role in:
 Process design
 Control system design
 Process optimization
 Steps for model development:
 Specification of model structure
 Design and realization of experiments
 Estimation of uncertain model parameters

6. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
2
Mathematical modelling
 Mathematical models play an important role in:
 Process design
 Control system design
 Process optimization
 Steps for model development:
 Specification of model structure
 Design and realization of experiments
 Estimation of uncertain model parameters

7. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
3
 What experiment(s) give the best data
 Control policy, initial conditions, sampling time
Optimal experiment design

8. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
3
 What experiment(s) give the best data
 Control policy, initial conditions, sampling time
 Criteria for ‘the best experiment’
 Most informative data
 Precise parameter estimation
 Efficient model discrimination
Optimal experiment design

9. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
3
 What experiment(s) give the best data
 Control policy, initial conditions, sampling time
 Criteria for ‘the best experiment’
 Most informative data
 Precise parameter estimation
 Efficient model discrimination
 Assumption:
 Usually: normal distribution of measurement error
 In this work: unknown-but-bounded measurement error
Optimal experiment design

10. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
3
 What experiment(s) give the best data
 Control policy, initial conditions, sampling time
 Criteria for ‘the best experiment’
 Most informative data
 Precise parameter estimation
 Efficient model discrimination
 Assumption:
 Usually: normal distribution of measurement error
 In this work: unknown-but-bounded measurement error
Optimal experiment design

11. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
3
 What experiment(s) give the best data
 Control policy, initial conditions, sampling time
 Criteria for ‘the best experiment’
 Most informative data
 Precise parameter estimation
 Efficient model discrimination
 Assumption:
 Usually: normal distribution of measurement error
 In this work: unknown-but-bounded measurement error
Optimal experiment design

12. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
3
 What experiment(s) give the best data
 Control policy, initial conditions, sampling time
 Criteria for ‘the best experiment’
 Most informative data
 Precise parameter estimation
 Efficient model discrimination
 Assumption:
 Usually: normal distribution of measurement error
 In this work: unknown-but-bounded measurement error
Optimal experiment design

13. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
4
 Approximate the joint-confidence region of parameters in the
parametric space using confidence ellipsoid
 Assumption: normally distributed measurement error
Classical parameter estimation

14. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
4
 Approximate the joint-confidence region of parameters in the
parametric space using confidence ellipsoid
 Assumption: normally distributed measurement error
Classical parameter estimation

15. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
4
 Approximate the joint-confidence region of parameters in the
parametric space using confidence ellipsoid
 Assumption: normally distributed measurement error
Classical parameter estimation

16. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
4
 Approximate the joint-confidence region of parameters in the
parametric space using confidence ellipsoid
 Assumption: normally distributed measurement error
Classical parameter estimation

17. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
4
 Approximate the joint-confidence region of parameters in the
parametric space using confidence ellipsoid
 Assumption: normally distributed measurement error
Classical parameter estimation

18. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
4
 Approximate the joint-confidence region of parameters in the
parametric space using confidence ellipsoid
 Assumption: normally distributed measurement error
Classical parameter estimation

19. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
4
 Approximate the joint-confidence region of parameters in the
parametric space using confidence ellipsoid
 Assumption: normally distributed measurement error
Classical parameter estimation

20. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
5
Classical Optimal Experiment Design (OED)

21. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
5
Classical Optimal Experiment Design (OED)

22. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
6
Classical OED : Design criteria
 Minimize or maximize a certain measure of covariance matrix
 Experimental design criteria
 A design:
 Minimize the trace of the covariance matrix

Franceschini, G., Macchietto, S., 2008.Model-based design of experiments for
parameter precision: State of the art. Chem. Eng. Sci. 63 (19), 4846–4872.

23. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
6
Classical OED : Design criteria
 Minimize or maximize a certain measure of covariance matrix
 Experimental design criteria
 A design:
 Minimize the trace of the covariance matrix

 D design:
 Minimize the determinant of the covariance
matrix

Franceschini, G., Macchietto, S., 2008.Model-based design of experiments for
parameter precision: State of the art. Chem. Eng. Sci. 63 (19), 4846–4872.

24. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
6
Classical OED : Design criteria
 Minimize or maximize a certain measure of covariance matrix
 Experimental design criteria
 A design:
 Minimize the trace of the covariance matrix

 D design:
 Minimize the determinant of the covariance
matrix

 Optimization problem:
Franceschini, G., Macchietto, S., 2008.Model-based design of experiments for
parameter precision: State of the art. Chem. Eng. Sci. 63 (19), 4846–4872.

25. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
7
 Parameter estimation with unknown-but-bounded measurement
error
Guaranteed Parameter Estimation (GPE)

26. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
7
 Parameter estimation with unknown-but-bounded measurement
error
Guaranteed Parameter Estimation (GPE)

27. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
7
 Parameter estimation with unknown-but-bounded measurement
error
 Measurements:
Guaranteed Parameter Estimation (GPE)

28. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
7
 Parameter estimation with unknown-but-bounded measurement
error
 Measurements:
Guaranteed Parameter Estimation (GPE)

29. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
7
 Parameter estimation with unknown-but-bounded measurement
error
 Measurements:
Guaranteed Parameter Estimation (GPE)

30. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
7
 Parameter estimation with unknown-but-bounded measurement
error
 Measurements:
Guaranteed Parameter Estimation (GPE)

31. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
Optimal Experiment Design for GPE
8

32. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
Optimal Experiment Design for GPE
8

33. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
9
 Step 1: over-approximate the GPE solution using an orthotope (box)
Methodology : Optimal Experiment Design for GPE
 Enclose the GPE solution set by finding the upper and lower limits of
each uncertain parameter
 Note: Similar to classical optimal experiment design, is assumed to be
given

34. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
10
Optimistic over-approximation of GPE set
 Objective: identify points
 Assumption:

35. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
10
Optimistic over-approximation of GPE set
 Objective: identify points
 Assumption:

36. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
10
Optimistic over-approximation of GPE set
 Objective: identify points
 Assumption:

37. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
10
Optimistic over-approximation of GPE set
 Objective: identify points
 Assumption:

38. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
Optimistic over-approximation of GPE set
 Objective: identify points
 Assumption:
10

39. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
Optimistic over-approximation of GPE set
 Objective: identify points
 Assumption:
10

40. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
11
Optimal experiment design for GPE: Design criteria
 Minimize or maximize a function of the over-approximation box
 Experimental design criteria
 A design:
 Minimize the perimeter of the box

 D design:
 Minimize the volume of the box


41. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
12
 Objective: identify the control input which yields the minimum
value for the design criterion over the over-approximation box
 OED for GPE problem:
Step 2 : Optimize over the over-approximation box

42. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
12
 Objective: identify the control input which yields the minimum
value for the design criterion over the over-approximation box
 OED for GPE problem:
Step 2 : Optimize over the over-approximation box

43. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
12
 Objective: identify the control input which yields the minimum
value for the design criterion over the over-approximation box
 OED for GPE problem:
Step 2 : Optimize over the over-approximation box
Hatz, K., Leyffer, S., Schlöder, J. P., Bock, H. G., 2013. Regularizing bilevel nonlinear programs by lifting. Preprint ANL/MCS-P4076-0613.

44. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
12
 Objective: identify the control input which yields the minimum
value for the design criterion over the over-approximation box
 OED for GPE problem:
Step 2 : Optimize over the over-approximation box
Hatz, K., Leyffer, S., Schlöder, J. P., Bock, H. G., 2013. Regularizing bilevel nonlinear programs by lifting. Preprint ANL/MCS-P4076-0613.

45. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
13
Case Study: Lotka-Volterra system
 Lotka-Volterra (predator-prey) model modified to control the prey
population
 Model :
 Measurements:
 Measurement error bounds: [-0.1,0.1]
 Fixed sampling time: 0.1
 Control input:
 Uncertain parameters: (True parameters = (0.9,1.5))

46. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
14
Case Study: Lotka-Volterra system
 Optimal experiment design for GPE – A criterion
 Solved using CasADi + Ipopt using sequential approach with CVODE
integrator
 GPE solution sets obtained from MC++ (Chachuat et al.)

47. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
14
Case Study: Lotka-Volterra system
 Optimal experiment design for GPE – A criterion
 Solved using CasADi + Ipopt using sequential approach with CVODE
integrator
 GPE solution sets obtained from MC++ (Chachuat et al.)
0.8 0.84 0.88 0.92 0.96 1
1.42
1.46
1.5
1.54
1.58
k
d
99% confidence ellipsoid for GPE A-design
GPE solution set for GPE A-design
0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
t
u
scaled
control input for GPE A-design

48. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
14
Case Study: Lotka-Volterra system
 Optimal A design for GPE (magenta) compared to classical A design
(green)
 Solved using CasADi + Ipopt using sequential approach with CVODE
integrator
 GPE solution sets obtained from MC++ (Chachuat et al.)
0.8 0.84 0.88 0.92 0.96 1
1.42
1.46
1.5
1.54
1.58
k
d
99% confidence ellipsoid for classical A-design
99% confidence ellipsoid for GPE A-design
GPE solution set for classical A-design
GPE solution set for GPE A-design
0 0.2 0.4 0.6 0.8 1
0
0.2
0.4
0.6
0.8
1
t
u
scaled
control input for classical A-design
control input for GPE A-design

49. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
15
 Worst-case over-approximation
 Assumption:
Global, Worst-case over-approximation
 Global over-approximation
 Assumption:

50. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
16
Conclusions
 Problem of optimal experiment design for guaranteed parameter
estimation formulated as a bi-level program

51. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
16
Conclusions
 Problem of optimal experiment design for guaranteed parameter
estimation formulated as a bi-level program
 The approximation of the GPE set is realized as a box (this might
include parts of parametric space not in the GPE set)

52. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
16
Conclusions
 Problem of optimal experiment design for guaranteed parameter
estimation formulated as a bi-level program
 The approximation of the GPE set is realized as a box (this might
include parts of parametric space not in the GPE set)
 This methodology for optimal experiment design for GPE always yields
a non-convex optimization problem (computationally intensive
problem; challenging to solve for global optimality)

53. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
16
Conclusions
 Problem of optimal experiment design for guaranteed parameter
estimation formulated as a bi-level program
 The approximation of the GPE set is realized as a box (this might
include parts of parametric space not in the GPE set)
 This methodology for optimal experiment design for GPE always yields
a non-convex optimization problem (computationally intensive
problem; challenging to solve for global optimality)
 Results of optimal experiment design for guaranteed parameter
estimation differ from the classical experiment design

54. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
17
Future work
 Look for alternative definite shapes (ellipsoid) to over-approximate the
GPE solution set
 Exploring the possibility of using other solvers to achieve the global
optimum
 Apply this method on robust control and design procedures

55. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
Acknowledgement
The research leading to these results has received funding from the
European Commission under grant agreement number 291458
(MOBOCON).

56. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
Backup
57

57. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
58
Lotka-Volterra system
 Model:
 Optimal D design for GPE compared to classical D design
Sager, S., 2013. Sampling decisions in optimum experimental design in the light of pontryagin’s maximum principle. SIAM Journal on Control and Optimization 51 (4), 3181–3207.
Telen, D., Logist, F., Derlinden, E. V., Tack, I., Impe, J. V., 2012. Optimal experiment design for dynamic bioprocesses: A multi-objective approach. Chemical Engineering Science 78, 82 – 97.

58. D
N
Y
D
D
N
N
Y
Y
Process Dynamics
and Operations
59
Semi-batch reactor system
 Model:
 Optimal A design for GPE
B. Srinivasan, S. Palanki, and D. Bonvin. Dynamic optimization of batch processes: I. Characterization of the nominal solution. Comput- ers & Chemical Engineering,
27(1):1–26, 2003.
0 0.2 0.4 0.6 0.8 1
0
0.02
0.04
0.06
0.08
0.1
t
u
GPE A design
0.052 0.054 0.056 0.058 0.06 0.062 0.064
-8
-7.8
-7.6
-7.4
-7.2
-7
-6.8
-6.6
-6.4
x 10
4
k
H
Guaranteed paramater estimation solution set
Optimistic over-approximation of GPE solution set