Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Uncertainty in risk engineering: concepts
Eric Marsden
<eric.marsden@risk-engineering.org>
‘‘When using a mathematical mod...
On the use of models in risk assessment
‘‘In that Empire, the Art of Cartography attained such Perfection that
the map of ...
Types of uncertainty
uncertainty
stochastic
variability
temporal
variability
spatial
variability
epistemic
uncertainty
mod...
Types of uncertainty
uncertainty
stochastic
variability
temporal
variability
spatial
variability
epistemic
uncertainty
mod...
Types of uncertainty
uncertainty
stochastic
variability
temporal
variability
spatial
variability
epistemic
uncertainty
mod...
Epistemic uncertainty and linguistic imprecision
C o m m u n i c a t i o n r e l i e s o n s h a r e d c o n t e x t , b u...
Illustration of linguistic imprecision
Forecast from US National Intelligence Estimate 29-51 Probability
of an Invasion of...
Uncertainty does not only concern the future
Bank of England projection of various
macroeconomic indicators use “fan chart...
Treatment of uncertainty
He who knows and knows he knows,
He is wise — follow him;
He who knows not and knows he knows not...
Types of uncertainty
‘‘As we know, there are known knowns. There are things
we know we know. We also know there are known
...
Goal of uncertainty modelling
Aims of quantitative uncertainty assessments:
▷ understand the influence of uncertainties
• ...
Five levels of integration of uncertainty in risk assessment
Hazard identification
Worst case approach
Quasi worst case
Be...
Integration level 0
▷ Undertake hazard identification
▷ Example: product is carcinogenic (yes/no)
▷ Suitable approach wher...
Integration level 1
▷ Worst-case approach
▷ Example: “What is the maximum number of potential
victims in a specified event...
Integration level 2
▷ Quasi worst-case and plausible upper bounds
• insurance industry is concerned with maximum forseeabl...
Integration level 3
▷ Best estimates, using point values at the median of the
parameters’ probability distributions
▷ Exam...
Integration level 4
▷ Probabilistic risk analysis based on mean
probabilities or future frequencies of events
• estimate p...
Risk measures
▷ A quantity used for the inference of the outputs of interest under
uncertainty will be called a quantity o...
Framework for uncertainty modelling
Step C: Propagation
Step A: SpecificationStep B: uncertainty
modelling (or
calibration...
Uncertainty in risk analysis
▷ It can be tempting for risk analysts to under-emphasize the degree of
uncertainty present i...
Uncertainty and decision-making
conventional risk
assessment
rely on past experience
of generic hazard
consider putative
c...
Statue “Politicians discussing global warming” by I. Cordal, Berlin
Further reading
▷ Slides on Sensitivity analysis from risk-engineering.org
▷ Book Uncertainty in Industrial Practice — A g...
Feedback welcome!
Was some of the content unclear? Which parts of the lecture
were most useful to you? Your comments to
fe...
Upcoming SlideShare
Loading in …5
×

Uncertainty in risk management: concepts

1,512 views

Published on

Learn about different categories of uncertainty (stochastic, epistemic, decision uncertainty), how uncertainty modelling is used in risk assessment, and different levels of sophistication in uncertainty representation in risk analysis.

Part of the free risk-engineering.org course materials.

Published in: Engineering
  • Be the first to comment

Uncertainty in risk management: concepts

  1. 1. Uncertainty in risk engineering: concepts Eric Marsden <eric.marsden@risk-engineering.org> ‘‘When using a mathematical model, careful attention must be given to uncertainties in the model. – Richard Feynman
  2. 2. On the use of models in risk assessment ‘‘In that Empire, the Art of Cartography attained such Perfection that the map of a single Province occupied the entirety of a City, and the map of the Empire, the entirety of a Province. In time, those Unconscionable Maps no longer satisfied, and the Cartographers Guilds struck a Map of the Empire whose size was that of the Empire, and which coincided point for point with it. The following Generations, who were not so fond of the Study of Cartography as their Forebears had been, saw that that vast map was Useless, and not without some Pitilessness was it, that they delivered it up to the Inclemencies of Sun and Winters. In the Deserts of the West, still today, there are Tattered Ruins of that Map, inhabited by Animals and Beggars; in all the Land there is no other Relic of the Disciplines of Geography. – Jorge Luis Borges, On Rigor in Science
  3. 3. Types of uncertainty uncertainty stochastic variability temporal variability spatial variability epistemic uncertainty model uncertainty parameter uncertainty decision uncertainty goals & objectives values & preferences ▷ Stochastic (or aleatory) uncertainty • related to the real variability of a population or a physical property • cannot be reduced • example: wind speed at Toulouse airport 100 days from now
  4. 4. Types of uncertainty uncertainty stochastic variability temporal variability spatial variability epistemic uncertainty model uncertainty parameter uncertainty decision uncertainty goals & objectives values & preferences ▷ Epistemic uncertainty • related to lack of knowledge or precision of a model parameter • model uncertainty: lack of confidence that the mathematical model is a “correct” formulation of the problem • parameter uncertainty: scientific knowledge insufficient to determine parameter exactly • in general, reducible with sufficient investment
  5. 5. Types of uncertainty uncertainty stochastic variability temporal variability spatial variability epistemic uncertainty model uncertainty parameter uncertainty decision uncertainty goals & objectives values & preferences ▷ Decision uncertainty • presence of ambiguity or controversy about how to quantify or compare social objectives • which risk metrics, which acceptance criteria? • how to aggregate the utilities of individuals? • how to discount delayed benefits against short-term benefits?
  6. 6. Epistemic uncertainty and linguistic imprecision C o m m u n i c a t i o n r e l i e s o n s h a r e d c o n t e x t , b u t t e r m s u s e d f o r d i s c u s s i n g l i k e l i h o o d a r e v e r y s u b j e c t i v e a n d “ f u z z y ”
  7. 7. Illustration of linguistic imprecision Forecast from US National Intelligence Estimate 29-51 Probability of an Invasion of Yugoslavia (1951): ‘‘Although it is impossible to determine which course the Kremlin is likely to adopt, we believe that the extent of Satellite military and propaganda preparations indicates that an attack on Yugoslavia in 1951 should be considered a serious possibility. Authors of the report were asked “what odds they had had in mind when they agreed to that wording”. Their answers ranged from 1:4 to 4:1. Image: Podgarić monument, former Yugoslavia
  8. 8. Uncertainty does not only concern the future Bank of England projection of various macroeconomic indicators use “fan charts” to illustrate the level of uncertainty in their predictions (probability mass in each colored band is 30%, 10% probability that outcomes lie outside of the colored area). Note that there is also uncertainty about data concerning the past. Figure source: bankofengland.co.uk/publications/Documents/inflationreport
  9. 9. Treatment of uncertainty He who knows and knows he knows, He is wise — follow him; He who knows not and knows he knows not, He is a child — teach him; He who knows and knows not he knows, He is asleep — wake him; He who knows not and knows not he knows not, He is a fool — shun him. Ancient arabic proverb
  10. 10. Types of uncertainty ‘‘As we know, there are known knowns. There are things we know we know. We also know there are known unknowns. That is to say we know there are some things we do not know. But there are also unknown unknowns, the ones we don’t know, we don’t know. – Donald Rumsfeld, February 2002, US DoD news briefing Image source: US DoD, public domain
  11. 11. Goal of uncertainty modelling Aims of quantitative uncertainty assessments: ▷ understand the influence of uncertainties • help prioritize any additional measurement, modeling or R&D efforts ▷ to qualify or accredit a model or a method of measurement • “this is of sufficient quality for this purpose” ▷ to influence design: compare relative performance and optimize the choice of a maintenance policy, an operation or the design of the system ▷ compliance: to demonstrate the system’s compliance with explicit criteria or regulatory thresholds • examples: nuclear or environmental licensing, aeronautical certification…
  12. 12. Five levels of integration of uncertainty in risk assessment Hazard identification Worst case approach Quasi worst case Best estimates Probabilistic risk analysis Adapted from Uncertainties in global climate change estimates, E. Paté-Cornell, Climatic Change, 1996:33:145-149
  13. 13. Integration level 0 ▷ Undertake hazard identification ▷ Example: product is carcinogenic (yes/no) ▷ Suitable approach where no numerical tradeoff required: • hazard is clearly defined and solution is simple and inexpensive • hazard is poorly known and would have catastrophic impact, so benefits of available solutions would dwarf the costs in any case
  14. 14. Integration level 1 ▷ Worst-case approach ▷ Example: “What is the maximum number of potential victims in a specified event?” ▷ Suitable approach when the worst case is clear and there is a reasonable solution to address the worst case ▷ Typical approach used for emergency planning ▷ Problem: no matter how conservative you are concerning parameters, someone can still highlight an “even worse” case which would require even more safety investment Image: The Great Wave off Kanagawa, K. Hokusai, 1825, public domain
  15. 15. Integration level 2 ▷ Quasi worst-case and plausible upper bounds • insurance industry is concerned with maximum forseeable loss ▷ Example: “What is the “maximal probable flood” or the “maximum credible earthquake” in this area?” ▷ Fundamentally, truncation of the probability distribution of the potential loss distribution ▷ Problems: • how to be coherent between “maximum probable flood” & “maximum credible earthquake”? • difficult to assess resulting level of safety • can’t guarantee that people in different locations are treated fairly Loss ? ⁇ ⁇?
  16. 16. Integration level 3 ▷ Best estimates, using point values at the median of the parameters’ probability distributions ▷ Example: “What is the ‘most credible’ estimate of the probability of an accident or of losses in an accident in a chemical plant?” ▷ Problem: a low probability outcome (even with hugely undesirable consequences) will be ignored in this approach
  17. 17. Integration level 4 ▷ Probabilistic risk analysis based on mean probabilities or future frequencies of events • estimate probability distribution of each input parameter • propagate uncertainty through model to obtain distribution of outputs of interest • stochastic “Monte Carlo” methods ▷ Example: “What is the probability of exceeding specified levels of losses in different degrees of failure of a particular dam?” parameter A parameter B parameter C y’ = f(y,t) output of model output of interest s l i d e s o n M o n t e C a r l o m e t h o d s a t r i s k - e n g i n e e r i n g . o r g
  18. 18. Risk measures ▷ A quantity used for the inference of the outputs of interest under uncertainty will be called a quantity of interest, or performance measure or risk measure in finance and economics ▷ Some examples: • percentages of error/uncertainty on the variables of interest (i.e. coefficient of variation) • confidence intervals on the variables of interest • quantile of the variable of interest (such as the value at risk in finance), possibly conditional on penalized inputs • probabilities of exceedance of a safety threshold or of an event of interest • expected value (cost, utility, fatalities…) of the consequences s l i d e s o n r i s k m e t r i c s a t r i s k - e n g i n e e r i n g . o r g
  19. 19. Framework for uncertainty modelling Step C: Propagation Step A: SpecificationStep B: uncertainty modelling (or calibration/assimilation) Inputs Uncertain: x Fixed: d Modelling through distributions Step C’: Sensitivityanalysis / ranking Decision criterion Ex: Proba ‹ 10-bFeedback process Variables of interest z=G(x,d) Quantity of interest ex: variance, exceedance System model G(x,d) Generic conceptual framework for uncertainty modelling, from Quantifying uncertainty in an industrial approach: an emerging consensus in an old epistemological debate, E. de Rocquigny, 2009, sapiens.revues.org/782
  20. 20. Uncertainty in risk analysis ▷ It can be tempting for risk analysts to under-emphasize the degree of uncertainty present in a risk analysis of a complex system • engineers are trained to deal with “hard facts” and not with judgments (“mechanical objectivity”, writes J. Downer) • laypeople often overreact to information on uncertainty in risk estimations • the authority of engineers and regulators is (seen to be) undermined by “admission” of uncertainty • there is often political pressure to de-emphasize the presence of uncertainty, to avoid challenges to policy decisions ▷ Professional ethics and the long-term credibility of technical risk assessment require uncertainties to be assessed, presented to stakeholders, and integrated in decision-making
  21. 21. Uncertainty and decision-making conventional risk assessment rely on past experience of generic hazard consider putative consequences and scenarios emphasison consequences eg. if serious/irreversibleor need toaddresssocietal concerns likelihoodincreasinglyuncertain consequences increasingly uncertain towardsignorance Source: Reducing risk, protecting people: HSE’s decision-making process, UK Health and Safety Executive, 2001, hse.gov.uk/risk/theory/r2p2.pdf
  22. 22. Statue “Politicians discussing global warming” by I. Cordal, Berlin
  23. 23. Further reading ▷ Slides on Sensitivity analysis from risk-engineering.org ▷ Book Uncertainty in Industrial Practice — A guide to quantitative uncertainty management, Wiley, 2008, isbn: 978-0-470-99447-4 ▷ Literature review of methods for representing uncertainty, Industrial Safety Cahiers number 2013-03, available from foncsi.org/en/ For more free course materials on risk engineering, visit risk-engineering.org
  24. 24. Feedback welcome! Was some of the content unclear? Which parts of the lecture were most useful to you? Your comments to feedback@risk-engineering.org (email) or @LearnRiskEng (Twitter) will help us to improve these course materials. Thanks! @LearnRiskEng fb.me/RiskEngineering google.com/+RiskengineeringOrgCourseware This presentation is distributed under the terms of the Creative Commons Attribution – Share Alike licence For more free course materials on risk engineering, visit risk-engineering.org

×