The document discusses the flaws of using heat maps and qualitative risk assessment methods. It argues that heat maps oversimplify risks, introduce biases, and prevent meaningful comparison of risks. The document recommends using a data-driven, quantitative approach to risk assessment that involves gathering risk data, modeling scenarios, and understanding probability distributions rather than relying on subjective ratings. It provides examples of how to model loss exposures and demonstrate a risk quantification model in practice using bow-tie analysis, decision trees, and Monte Carlo simulations.

2. Ditch your heat
maps
how best to quantify operational
and safety risks
March 22 2023
Hernan Huwyler and Christian Harris

3. Assessing risks with colors and adjectives is common
CERTAIN
LIKELY
POSSIBLE
UNLIKELY
RARE
INSIGNIFICANT MINOR SIGNIFICANT MAJOR CATASTROPHIC
LOW MODERATE
MODERATE
MODERATE
HIGH
HIGH
MODERATE
MODERATE
EXTREME
HIGH
HIGH
MODERATE
EXTREME
EXTREME
HIGH
MODERATE
IMPACT
LIKELIHOOD
LOW
LOW
LOW
LOW LOW
LOW
LOW LOW LOW

4. Heat maps are malpractice
Opinion-versus data-driven
The best available data are not used
Biases affect the communication
Aggregation is impossible
Adjectives and colors cannot be added
Values are compressed
Wrong allocation of resources
Investments in securities cannot be justified
Single scenarios are identified
Lack of corporate defense
Accepted risks create legal liabilities

5. You can improve with a data-driven assessment
Data
Decision
Model
Objective
Use risk data
Gather data on incidents and near-misses
Control data on accidents and losses
Model risks
Calculate the financial exposure to risks
Simulate scenarios to set reserves
Understand distributions
Identify patterns of losses and near misses
Foresee the effect of uncertainty
Facilitate better discussions
Impact decision-making with facts

6. Scientific research has proven the flaws of heatmaps
What is wrong about risk matrices, Tony Cox, 2008 > worse than useless
Further thoughts on the utility of risk matrices, David Ball, 2013 > untrustworthy
picture
Some extensions on risk matrix approach, Huihui Ni, 2010 > defects still left
unresolved
On the origin of probability consequence diagrams, Ben Ale, 2015 > single factor
impacts
Problems with scoring methods and ordinal scales, Doug Hubbard, 2010 >
arbitrary features of the scoring
Recommendations on the use and design of risk matrices, Niels Duijm, 2015
> aggregation is problematical
Back to Basics: Risk Matrices and ALARP, Glen Wilkinson, 2010 > unable to
compare risks

7. Measure physical security losses
Medical costs
Property losses
Production stops
DAMAGE
Response
Cleanup costs
Remediation
RESPONSE
Closures
Productivity losses
Emotional costs
CONTINUITY
Fines
Claims
Compensations
Revenue
COMPLIANCE
FIRST TIER SECOND TIER THIRD TIER FOURTH TIER

8. The bow-tie tool can help you define scenarios
CAUSE 1 CONSEQUENCE 1
CAUSE 2 CONSEQUENCE 2
Preventive controls
How threats by agents can
cause security events
Corrective controls
How the security
objectives are impacted
EVENT

9. Bow-tie analysis breaks down scenarios into components
CAUSE 1
CAUSE 2 CONSEQUENCE 2
Decision tree
EVENT
FIRST TIER
SECOND TIER
SECOND TIER
THIRD TIER
THIRD TIER

11. Decision trees are a simple tool for pricing risks
50%
Medical costs 15/30K USD
Property losses 50/100k USD
Earthwork and drainage
costs 100/200k USD
Earthwork costs
30/50k USD
Work closure
100/200 USD
No closure
Landslide may
lead to injuries
and damage
2 to 10 acres
Workhours
Normal work activity
Budgeted costs of 200K USD
No insurance
Overloading
Heavy rains
Undetected
weaknesses
50%
75%
25%
100%
Path 1 – (15+50)*100%+100*50%+100*75% = 190
Path 2 – (30+100)*100%+200*50%+200*75% = 380
Path 3 –(15+50)*100%+100*50%+0*25% =115
Path 4 –(30+100)*100%+200*50%+0*25% =230
Path 5 –(15+50)*100%+30*50%+0*100% =115
Path 6 – (30+100)*100%+50*50%+0*100% = 155
190K/380k
115K/230k
115K/155k
No closure
100%
420K/765k
Risk value
220K/565k
Path value

12. Consistent approach for a process to manage risks
SCENARIO
Quantified target at
risk
Threats
Vulnerabilities
Multiple experts
Locations
Single horizon
Plausibility test
IMPACT ASSESSMENT
Quantify losses
HSE data
Insurance claims
Industry estadistics
Tolerance test (deaths,
polution)
DECISION
Cancel activity
Reasses target
Cover by insurance
Outsource
Change parties
Increase preventive
or contigency
controls
Accept
Quantify annual occurences
Exposure rate to threats
Distributions
By tiers
Calibrated estimates
Sensitibity analysis
PROBABILITY

14. Model loss exposures in the best-and worst-case scenarios
MEASURE
Base case
Worst-case
Best-case
Confidence level
MODEL
Monte Carlo
simulations
Annual exposure at %
change
Histogram and loss
exceedance curve

15. Demonstration of a risk quantification model in practice
TEMPLATE
Integrated risk register with the model
in a MS Excel spreadsheet without
add-ons
RISK AND CONTROL
Cross-referencece the security
controls to the risk register to balance
the costs of measures

16. Demonstration of a risk quantification model in practice
Example for a road construction project
Cost per event from 2,500 to 45,000
… for 90% of the events
Upper limit at 10,000 due to insurance
Expected to occur each 10 to 33 years
… for 80% of the events
Average 6,250 * 6.5% = 406

17. Demonstration of a risk quantification model in practice
Scenario 1, 2, 3,…
Accumulated
losses
Scenario 1, 2, 3,…
- Top 75% worst-cases
- Base cases
- Top 25% worst-cases

18. Assessing risks based on
colors, adjectives, 5*5 scores,
or a wet finger in the wind is
simply paper compliance
Protecting people requires
having data-driven
conversations