The document discusses issues with using risk matrices to quantify risk. It notes that risk matrices treat risk categories as precise when they are actually blurred, as the boundaries between risk levels are inaccurate with no space between them. This means a risk could fall into multiple categories and its actual level cannot be determined. The document also notes other problems with treating risk levels as numbers that can be multiplied, when they represent probability distributions rather than precise points. Overall, it argues that risk matrices provide a false sense of accuracy and are not an effective way to reliably allocate resources to risks.

1. Fade in as talk starts.
1

2. I am sure you are familiar with the durable and ubiquitous risk matrix.
2

3. You know, the only =me that you see a quilt in the office.
3

4. The typical matrix is a grid as you see. The cells denote the crossing of a risk impact
level and a risk probability level. We label the crossings thus.
But, I see a problem.
4

5. Look at this junc=on. Forget the cells, look at the cell boundaries.
Here we have at a point three risk ‘ra=ngs’ touching. There’s nothing between them
There is nothing between ‘low’ and ‘high’. For example. They meet at a point, and, as
you know, a point is dimensionless.
Does this strike you as interes=ng?
Does it strike you as worrying?
5

6. Its everywhere!
Each of the four junc=ons have the problem, as is obvious.
And, what is the problem?
Well get to that.
6

7. It starts here.
It’s too easy to think that the category boundaries are as precise as they look. They
are not.
In reality they are very blurred and the high accuracy of appearance is lost in the
reality of low accuracy.
A scale with three divisions is not very accurate, as the accuracy is +/- the half-division
interval.
7

8. It is similar for all boundaries: between Medium and High Risk Impact is +/- half the
interval between them.
The inaccuracy of the matrix should start to be evident.
8

9. I think you can see where we are heading.
That is, we don’t know where we are heading!
The junc=on of the four risk classes at the crossing of the zones is a box and in that
box, somewhere, is the actual junc=on point.
We just don’t know for certain where it is.
9

10. In op=cs there’s the concept of the circle of confusion; this is like it. At the crossing of
the two rank boundaries, we cannot tell where we are.
10

11. With the accuracy limits of the matrix, we are instantly in the posi=on of being unable
to determine if a risk that has been carefully rated is Medium, High, or Extreme.
They are indis=nguishable.
How, then, can we hope to reliably and efficiently allocate resources to avoid both
under and over preparedness for a risk?
We cannot.
So, this is not risk management, this is a ceremony.
It is the equivalent of Caesar looking a goat entrails before a bable!
11

12. If we consider the axes as scales…and they certainly seem to be used that way,
the mid line of each band can be considered as the mean of a distribu=on.
That is, the place where it is most likely to be low, medium or high, whatever these
denote.
Similar problem to the arithme=c accuracy problem, but it makes the exercise a
different one.
12

13. As a probability distribu=on what we thought was a line (and the lines that we though
of as the band boundaries are not lines either), or rather a point, is not.
They are shapes.
High School maths will tell you that’s a whole different slice of pizza.
If we pretend the labels are numbers, as some users of the risk quilt do, we mul=ply
them, even no=onally (despite that there are nominals) and think that we are doing
something mathema=cally meaningful.
We are not. We are not doing anything meaningful in any way at all.
If you think you need to find the product of the points on the two scales, it is not
mul=plica=on that you need, but a whole other thing.
This is because you cannot ‘mulitply’ distribu=ons.
13

14. If you want to find the product of two distribu=ons, you cannot mul=ply them, as
they are not points, as I said; they are shapes.
You have to use the Mellin transform.
Good luck!
14

15. Is a calibrated matrix the beber way?
Glen Alleman has a discussion of the matrix on his blog at:
hbp://herdingcats.typepad.com/my_weblog/2010/07/risk-matrix.html
But we can’t escape the accuracy problem or the confidence margin of any es=mate
of probability.
Are you sure its 69%, or is it 71%? What is the fractal we are using?
Similarly for Budget Overrun.
How then can we determine between Likely-Very Likely and Significant-Severe?
We can’t with any reliability!
15

16. Whatever we do with risk, even using the gold-plated approach of Reference Class
Forecas=ng, we es=mate a probability; we have a band of confidence, or an interval,
for any risk.
One would hope that mostly the probability distribu=on would be Gaussian so we can
do handy things like produce a mean and standard devia=on;
16

17. but confron=ngly, some=mes it won’t be.
Some=mes it might be a Cauchy distribu=on (no=ce the fat tails, they are the
important bits), by which we are up the creek paddle-less for a mean.
It doesn’t exist.
17

18. But whatever the distribu=on, what are we lem with?
Using the probability of an event, based on experience, or expert elicita=on, we are
lem with ranges:
a cost range, or expecta=on of cost, as well as an expecta=on of the event occurring,
another range.
From this we can develop a range of values for the risk event occurring, to guide, note
the word “guide,” our planning.
For example we might develop an es=mate so that we are, say, 80% confident that
the risk cost will not exceed a calculated quantum.
18

19. The risk is quan=fied by developing a probability range and es=ma=ng the response
cost (or range) for the event.
We’ve now got some numbers to work with that capture both probability and cost/
cost proxies.
On a risk porqolio basis we can develop a response plan against our budget, or
increase the budget on a risk basis.
Here this tells us that we’ll take precau=ons; we might inspect annually. But apart
from that, we’ll accept the risk; we can afford to against a porqolio budget of
$billions.
And I should tell you, its worked. The wall of Parramaba Gaol has not collapsed in
over 100 years. Because of the input risk mi=ga=on. It was designed and built for the
natural condi=ons (founda=ons, wind speed, earthquake risk) and condi=ons of
service (layabouts leaning against it having a smoke).
19

20. The quilt can stay on the wall,
or we get sucked into Dilbert’s world.
20

21. Not where we want to end, but precisely where the ceremonial matrix leads us.
21

22. hbps://eight2late.wordpress.com/2009/07/01/cox’s-risk-matrix-theorem-and-its-
implica=ons-for-project-risk-management/
hbp://herdingcats.typepad.com/my_weblog/2010/07/risk-matrix.html
Schleier and Paterson, 2010, “Limita=ons of the Entomological Opera=onal Risk
Assessment Using Probabilis=c and Determinis=c Analyses“
Military Medicine, 175, 8:594.
22