Introduction in local and regional (global) sensitiviy analysis. Part of the Eawag Summer school on System Analysis

1. Eawag: Swiss Federal Institute of Aquatic Science and Technology
Sensitivity Analysis
9th EAWAG Summer School in Environmental System Analysis 2017
Andreas Scheidegger
12.06.2017

2. What are all the knobs for?

3. Sensitivity Analysis
Investigate how a model output responds to
changes in model parameters and/or model inputs.
Factors
(SA does not make a difference of inputs and parameters)

4. Goals of Sensitivity Analysis
• Better understanding of the model and its mechanisms
• Sanity check: does the model behave as expected?
• Identification of influential and non-influential model
parameters
The model is investigated, not the underlying system!

5. Change Input
Sensitivity Analysis
?

6. Sensitivity Analysis
Change Parameter
?

7. Sensitivity Analysis
Change Parameter more
?

8. Sensitivity Analysis
Change input, but use different parameter
?

9. Sensitivity Analysis
Change input, but use different parameter

10. Local vs. Regional Sensitivity Analysis
Two approaches to sensitivity analysis:
Local Sensitivity Analysis
Regional (global) Sensitivity Analysis
Investigation of the sensitivity of model results on parameters at a given
reference point in parameter space
The results depend on the choice of the reference point and the
parameter ranges
The results do depend on the choice of the parameter distribution
Investigation of the sensitivity of model results to the parameters based
on a probability distribution of the model parameters

11. Local Sensitivity Analysis
Two approaches to sensitivity analysis:
Regional Sensitivity Analysis
The results do no longer depend on the choice of a single reference
point but on the choice of the parameter distribution
Investigation of the sensitivity of model results to the parameters based
on a probability distribution of the model parameters
Local Sensitivity Analysis
Investigation of the sensitivity of model results on parameters at a given
reference point in parameter space
The results depend on the choice of the reference point and the
parameter ranges

12. Local Sensitivity Analysis
Deterministic model function:
The sensitivity of a model result to a parameter depends on:
functional relationship provided by the model equations
selected ranges/distributions of parameters

13. Local Sensitivity Analysis
A “natural” measure: slope (= derivative) of the model function with
respect to the component of the parameter vector

14. Local Sensitivity Analysis
Sensitivity rankings: sensitivities for each model output can be difficult to evaluate. It can
be useful to average the squares of the sensitivity functions at
different values of the index i
average sensitivity of the
model for a given parameter j

15. Local Sensitivity Analysis
Local sensitivity analysis can provide useful insights into the
model mechanisms
It is computationally relatively inexpensive
Nonlinearities of the model are not taken into account
Parameter interactions are not observed

16. Regional Sensitivity Analysis
Two approaches to sensitivity analysis:
Local Sensitivity Analysis
Investigation of the sensitivity of model results on parameters at a given
reference point in parameter space
The results depend on the choice of the reference point and the
parameter ranges
Regional (global) Sensitivity Analysis
The results do no longer depend on the choice of a single reference
point but on the choice of the parameter distribution
Investigation of the sensitivity of model results to the parameters based
on a probability distribution of the model parameters

17. Regional Sensitivity Analysis – Variance decomposition
Variance-based techniques are based on a decomposition of the variance of the
model output into contributions due to different parameters.
A fraction of the variance of a model result i is due to the distribution of a parameter j

18. Regional Sensitivity Analysis– Variance decomposition
Variance-based sensitivity analysis is based on the comparison of the total variance
of the model output with a “reduced” variance when keeping one parameter fixed
The degree of variance reduction is a
measure of the contribution of the fixed
parameter to the total output variance
To remove the dependence of the conditional variance on where we fixed the
parameter, we take the expected value of these conditional variances with
respect to the marginal distribution of the selected parameter

19. Regional Sensitivity Analysis
Fourier amplitude sensitivity testing (FAST)
1. Change all inputs with in
different frequencies
2. Analyze frequency spectra of
outputs
Image source: https://homepages.thm.de/~hg54/mmk_2006/script/multimedia/multimedia.htm

20. Sensitivitypackage
Typical Workflow
Generate
parameter/input
combinations
Compute indices
loop over all parameter/input combinations

21. Regional sensitivity analysis
• Change all parameter together
• Computationally more
expensive
• Considers interactions
• Describes model behavior
across a parameter region
Sensitivity analysis
• Learn about a model, not a system
• Gives hints which parameter can be identified well from data
Summary
Local sensitivity analysis
• Change one parameter at the time
• Computationally cheap
• Simple interpretation
• No interactions
• Results only valid for one point in
parameter space