Probabilistic Risk Management (With Notes)

Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
1/186
This briefing is an overview of the
probabilistic risk analysis
processes that can be applied to
our program. Although it may not
appear to be a “simple” overview,
this material is the tip of the
iceberg of this complex topic.
Just schedule analysis has been
addressed in detail here. The
cost aspects of forecasting and
simulation must be addressed as
well to complete the connections
between schedule and cost.
Probabilistic cost will be surveyed
here, but an in depth review is for
a later time.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005

2/186
An important aspect of education
and research in our business
domain, is “Fair Use” copyright
law.
All the material in this briefing is
accessible through the internet.
Conference proceedings journal
articles, company white papers
and other public sources form the
basis of much of this material
and are referenced in the
bibliography.
Some materials in this briefing
make references to other
copyrighted materials in the
course of research, investigation,
and analysis. These references
are solely intended for non–
commercial use and as such
have no intent to infringe on the
copyright holder. All attempts
have been made to acknowledge
the original copyright holder in
pursuit of fair use laws as
currently defined in the United
States.
December 2005

3/186
The concept that risk and the
management of risk is a
desirable part of our program is
not always appreciated or well
understood
Without risk there can be no
opportunities. The plans for the
program become static and
deterministic.
While risk and opportunity are
related, the management of risk
is not the complement of
opportunity. - even if this is a
popular notions these days.
See the Conrow, AT&L article for
detailed discussion of this
somewhat controversial topic.
The primary opportunity in
Programmatic Risk Management
is the avoidance of being late and
over budget on the planned
launch date.
December 2005

4/186
When we use the term “risk
tolerant IMS” it means a plan and
supporting that can tolerate risks.
Technical risks and programmatic
risks. These risks are built into
the program by its very nature.
These risks must be addressed
both technically and
programmatically.
The real challenge though is not
how to address them, but how to
recognize that they are being
addressed in a manner that
actually reduces the level of risk
as the program proceeds along
its path to final maturity.
A measure of “increasing
maturity” is the reduction of risk
made visible to the evaluator of
the IMS.
The materials here guide us
through the process of building a
risk tolerant IMS. But putting it to
work still requires practice.
December 2005

5/186
The credibility of the Integrated
Master Schedule (IMS) is the
critical success factor for both
our proposal and our execution
phase after the win.
Without a credible schedule and
the related cost credibility, there
is a low probability of a win.
The effort put into constructing a
credible schedule during the
proposal phase will pay off
(assuming the program structure
remains intact) during the
execution phase.
December 2005

6/186
The skills of creating and
managing a schedule and the
associated cost require special
understanding.
However, the planners are
usually the last in a long line of
“culprits” for finding the cause of
any failure.
This is a “no win” situation.
People skills, project
management skills, and some
level of technical skill is needed.
But most important is the people
skill, since the knowledge of how
to assemble a successful IMS
resides in the minds of others.
Getting this knowledge out and
on paper requires interpersonal
communication as a primary
process, not technical tools and
formal processes.
December 2005

7/186
Understanding the difference
between qualitative and
quantitative risk assessment is
important.
Our first approach is usually
qualitative.
But what is needed is
quantitative.
A specific measure of
programmatic risk, is the impact
of the mitigations or risk
retirement activities and measure
of the increasing maturity of the
program deliverables in the
presence of risk.
December 2005

8/186
Programmatic risk management
makes visible the technical risk
mitigation steps as well as the
alternative programmatic
processes in the presence of
these risks.
Alternative branching in the IMS
must be defined to a level of
detail that instills confidence that
the IMS properly represents a
“risk tolerant” plan.
December 2005

9/186
Since there is quite a bit of
material here, a quick overview
will get us started.
The executive overview should
leave the reader with a sense of
the important topics
• There are no point estimates
allowed in planning. All
estimates must be
probabilistic
• There are core issues with
simple (deterministic) PERT
and it is not to be trusted
• The use of a probabilistic tool
is useful, but understanding
how the underlying statistic
works is critical to its use in
planning and program
execution
December 2005

10/186
When asked “why are we doing
this?” many would answer –
because our customer wants us
to.
This would be too simple an
answer.
The main reason is, most
programs are simply too complex
not to have a better
understanding of how the
programmatic and technical risks
interact.
Not understanding the interaction
between these two types of risk
that creates the biggest risk.
Individually these risk “could” be
managed. But when combined
they behave in unpredictable and
maybe unknowable ways.
This is a core feature of any
system. See Systems Bible
December 2005

11/186
If we get only two concepts out of
this briefing they should be:
• There are multiple critical
paths in any executing
program. Asking “what is the
critical” indicates that the
questioner does not
understand the probabilistic
nature of the program
• PERT is a poor estimating
metric. It has built in biases
which under estimate the total
duration of the program.
Monte Carlo is a better
estimating tool, but it too
needs careful adjustment
before realistic numbers can
be derived.
December 2005

12/186
The DID–MGMT–81650
describes the Integrated Master
Schedule.
Integrating Programmatic and
Technical risk identification and
mitigation adds credibility to the
IMS and therefore to the overall
program.
Applying probabilistic risk
analysis to the IMS is mandated,
but care is needed to interpret
the results.
These tools aid in the evaluation,
but they are not replacements for
good program management
processes.
December 2005

13/186
The idea that uncertainty and the
risk that it produces can be
“programmed out” of the
schedule is a false hope.
Without understanding the
principles of Deming, the
management and the planning
staff will be “chasing their tail,”
trying to control the naturally
occurring variances in the plan.
The first approach is to set the
error bands wide enough to not
trigger an exception report for
these variances.
This approach is “good enough”
but what is missing is the
knowledge of “how wide is wide
enough?” for a specific set of
tasks or during a specific phase
of a program?
December 2005

14/186
The first step in the process of
adding credibility to the IMS is to
recognize that all task completion
times are random variables.
They are not “point” numbers
(scalars) but are “estimates” of
the completion time drawn from a
probability distribution of the
underlying population of all
completion times possible for the
specific task.
Modeling schedule durations are
random variables does not imply
these durations are “random.” It
reflects how a duration’s
uncertainty is influenced by the
underlying probabilistic nature of
the activity network.
December 2005

15/186
Building a credible IMS starts
with identifying the architecture of
the IMP and the supporting tasks
in the IMS.
Although this is restating the
obvious the process to do this is
actually quite hard.
Adding schedule and cost risk
identification and mitigation to the
process is the minimal result for
a winning proposal.
It cannot be emphasized enough
– the architecture of the IMS is
critical to identifying a risk
tolerant schedule. The “rats nest”
approach is simply unacceptable
to the success of any program.
December 2005

16/186
The goal of introducing
probabilistic schedule and cost
analysis is to improve the
probability of a “win” on the
proposal.
While winning is important,
executing the program is even
more important.
What ever “credibility” elements
were in the proposed IMS need
to be carried into the execution
schedule.
December 2005

17/186
The use of Monte Carlo for
assessing the IMS must be
turned into forecasting
performance.
This is done by identifying the
“hot spots” in the IMS through
sensitivity analysis, interventions
for these “hot spots” and the
measure of change resulting
from the intervention.
The important concept is to
connect metrics to measurable
benefits to the program. Without
this the creation of metrics is just
wasted effort.
December 2005

December 2005 18/186

19/186
Using risk and uncertainty as an
integral part of the planning
process is a sign of maturity.
Making decisions on the this risk
information improves maturity.
December 2005

20/186
When we speak of risk
management, either technical or
programmatic, the term usually
has a very localized context.
For the planning context risk
management must include both
technical and programmatic risk.
The technical risk aspects come
from external sources but are
directly represented in the IMS.
The programmatic impacts of this
technical risk must be explicitly
addressed.
This is the easy part.
The hard part is determining the
implicit programmatic risk that is
derived from the technical risk
and the risks that are derived
from the “architecture” of the
program itself.
This is where the true “risk
tolerant” IMS adds value.
December 2005

21/186
There are many approaches to
building a risk tolerant IMS. Our
current approach is to add risk
factors and margin to specific
areas of the IMS
The current approach to use a
Monte Carlo tool to assess where
this margin should be placed.
There are several other steps
along the way. Which steps to
take, how much effort to invest
and how to recognize the value
of this investment are some of
the management challenges as
well as the technical challenges.
December 2005

22/186
The difference between risk and
uncertainty needs to be
understood at some level.
For the most part the differences
are not important in the
beginning.
But once decisions start to be
made about mitigation steps,
branching probabilities for failure
modes, these differences
become more important.
December 2005

23/186
When we use the term
uncertainty or risk it means at
least 4 things.
First let’s sort out “uncertainty”
There are two classes of
uncertainty in large complex
programs.
• Static uncertainty emerges
from the natural variations in
the completion times of tasks.
This is a Deming uncertainty.
http://webserver.lemoyne.edu/
~wright/deming.htm is an
example of this type of
uncertainty
• The dynamic uncertainty is
about the unknowns and the
unknowable
December 2005

24/186
The static uncertainty in a
program can be addressed
directly in the plan with mitigation
tasks.
The dynamic uncertainty arises
from the dynamic interactions
between the tasks of the plan.
This interaction and the
outcomes to the end date cannot
be modeled with static
paradigms.
Monte Carlo simulation is an
approach to modeling these
interactions and their impact on
other elements of the plan
December 2005

25/186
Managing risk in the schedule
requires anticipation to identify
the risks, but also requires
understanding of the source of
risk, the impacts of these risks,
and the interaction between the
risks and the plan.
A process is needed to guide the
risk management activities. This
process must address both the
programmatic as well as
technical risk. The interaction
between programmatic and
technical risks must also be
managed.
These interactions must be
considered a “first order”
interaction.
The common approach is to
consider the technical risk as first
order and the programmatic risks
secondary.
The combination becomes a first
order interaction.
December 2005

26/186
As planners our goal must be to
produce a plan that has credibility
and integrity.
Credible plans are believable
plans
Integrity plans are trustworthy
plans.
Both attributes are needed for a
winning proposal and the follow
on execution.
The successful assessment of
the IMS during a proposal or
during execution by the customer
or DCMA depends on how
believable the plan is and how
well it can be assessed to
confirm this believability
December 2005

27/186
The assessment of the credibility
and integrity of the IMS can take
place by asking some questions.
These and similar questions
shine light on the underlying
attributes of the IMS in ways that
simple assessments do not.
These are not technical
assessment, like counting data in
the predecessors field, but are
architectural questions about the
“quality” of the IMS independent
of the technical details.
December 2005

28/186
NASA does risk management in
a specific way. We need to
understand their way as a
starting point.
Reading the NASA materials is a
start, but there is other research
available from conferences and
vendor web sites that needs to
be gathered and read as well.
Other government agencies as
well as civilian firms have similar
risk management approaches.
NASA’s approach is a good
starting because of manned
space flight’s inherent risk. And
NASA’s emphasis on Safety and
Mission Assurance.
December 2005

29/186
The IRMA tool developed at
NASA Johnson Space Center is
the basis of risk management for
a NASA side.
Although this approach is
focused on the technical risks the
programmatic risks appear in the
database.
As well there are other risk
management systems and
paradigms.
Active Risk Manager (ARM) is a
popular one as well,
http://www.strategicthought.com/
December 2005

30/186
The NASA Risk Management
Summary Card calls out
“schedule” impacts in three
places.
Connecting programmatic and
technical risk is a critical success
factor for a proposal as well as
an execution assessment.
December 2005

31/186
Adding probabilistic schedule and
risk analysis to the IMS can be
done through a structured
process.
1. The initiating event of the
risk is identified.
2. The result from this event is
described
3. The consequence that flow
from the scenario are
developed
4. The connections, flows,
interactions and correlations
between the scenarios are
modeled
5. The probability of
occurrence for each of these
scenarios is developed
6. The model of the probability
of occurrence and
consequences from the
occurrence are combined
December 2005

32/186
The Continuous Risk
Management paradigm found in
the technical risk world can be
applied to the programmatic risk
as well.
NASA has adopted Continuous
Risk Management (CRM)
through several guidelines listed
here.
The table summarizes how CRM
is managed in a structured
manner throughout the program
life,
December 2005

33/186
There is a difference between the
design evaluation of the IMS and
the risk evaluation.
The design evaluation describes
how the technical activities
needed to develop and deploy
the product – in this case a
manned spacecraft – must come
together in the right sequence to
make the planned completion
date.
The risk evaluation defines the
probabilistic completion model for
each task, the correlations
between the tasks and the
resulting probabilistic model.
This model is a Bayesian
Network of all the tasks.
December 2005

34/186
To construct an IMS with integrity
and credibility both technical and
programmatic risk must be
connected.
This process starts with the
identification of the technical
risks in ARM and their mitigations
in the IMS. This is the explicit risk
approach.
Next comes the explicit
programmatic risk activities. This
can be the well known margin
needed in front of major
milestones, program events or
deliverables.
Finally comes the implicit risk
mitigation activities that will be
needed to differentiate this IMS
from any other IMS to start to
build confidence that we have a
“risk tolerant” IMS.
December 2005

35/186
A pedagogical literature survey
from the RAND Corporation
supports the notion that
probabilistic risk assessment is
not seen in a favorable light by
management.
• It is too complex.
• The underlying statistic are not
will understood.
• “It’s the customers that are
asking for this.”
• There is little historical data to
calibrate the underlying
probability distribution functions
for task completion times.
All of these gaps must be closed
in some way in order to call our
IMS Risk Tolerant
December 2005

37/186
Managing in the presence of
uncertainty is the core behavior
for any modern program.
Trying to control this uncertainty
requires two basic
understandings:
1. The natural variations in the
schedule cannot be
sufficiently controlled to
remove risk. These are the
Deming variations and the
foreseen uncertainties
2. The unforeseen uncertainties
and the inherent chaos of the
program must be dealt with
through contingencies
Attempting to manage
uncertainty is limited to foreseen
risk. Managing in the presence of
uncertainty deals with unforeseen
and chaotic sources of risk
December 2005

38/186
When estimating the completion
times for tasks, there are three
primary problems.
1. A number produced by a
CAM or an IPT must be a
statistical estimate, not a
specific duration.
2. The meaning of “best” must
be established prior to
accepting the statistical
estimate
3. The collecting of the “most
likely” estimates cannot be
added in the sense of
adding scalar numbers,
since they are probability
distributions.
4. The “most likely” is NOT the
average completion time, it
is the completion time that
occurs most often from a
large sample of possible
completion times.
December 2005

39/186
The first approach to “planning”
the program is to ask the CAMs
or IPT Leads for each task in
their WBS or IMP/IMS area: “how
long with this take to do?”
The numbers that come back are
then entered in the duration field
on the schedule.
These numbers are not only
wrong they are dangerously
wrong.
They are “point” estimates that
live inside a probability
distribution.
The built in bias from the
approach has clinically be shown
to be optimistic or pessimistic,
but rarely “most likely.”
December 2005

40/186
The traditional approach is to roll
up the single point estimates into
a sum of the durations and
search for the longest path.
This is the Critical Path Method
(CPM) for assessing the finish
date of the plan.
The problem of course is these
“numbers” are not actual scalar
values. They are samples drawn
from probability distributions.
Addition is not mathematically
possible in the sense of addition,
defined over the set of natural
numbers (0, 1, 2, … ∞]
These probability distributions
can be “convolved” into a new
probability distribution, but a
better approach is Monte Carlo
December 2005

41/186
When asked “what is the most
likely” or the “best guess”
duration, the variety of answers
removes any chance of getting a
reasonable answer.
The meaning of “best” is
undefined in almost any situation
that has not taken explicit steps
to bound the answers.
Without calibrating the meaning
of “best” the planner cannot
bound the underlying probability
distribution of all the value that
are not “best” but could possibly
occur in the project
December 2005

42/186
When we use a term “best” or
“most likely” there is an implicit
assumption – often not
acknowledged – that other values
than “best” and “most likely” can
occur.
This is the probabilistic nature of
the duration estimate. A single
value cannot exist.
The actual shape of the
probability distributions is what is
needed for generating the “best”
estimate.
Without this knowledge, the
planner is guessing in the dark.
December 2005

43/186
Here are some steps to
producing “educated guesses.”
This is a model based approach
which depends on the maturity of
the data that is the basis of the
model.
While this is a high level
description, it needs raw data
underneath to make it valid.
Without this data the “guess” is of
little value.
What is missing in most cases is
any historical trends for the IMS
elements.
December 2005

44/186
Playing the 20 questions game is
on approach to calibrating the
“guess” for the duration.
This approach will get an answer
to without 10% to 20% in a few
questions.
This is a way to start the
“conversation” about duration
when the participants have
convinced themselves that they
can’t come up with the answer
because there is not enough
information.
December 2005

45/186
Another approach is to classify
the fidelity of the information.
This can be done with a 1, 2, 5
approach.
Gathering estimates by asking
for durations is the preferred
approach.
Instead, making a risk adjusted
estimate – duration and
confidence interval provides a
better approach.
This approach neutralizes the
guessing game by asking a risk
question first, then the duration.
The classification of risk provides
the lower and upper bounds of
the task duration. Along with the
underlying probability distribution,
this forms the basis of
probabilistic schedule analysis
December 2005

46/186
In all cases, uncertainty is the
normal mode for information
gathering.
When we ask a CAM or IPT for
an estimate and do not ask for
the risk associated with that
estimate and the confidence
intervals for that number we are
simply increasing the risk to the
program by absorbing unreliable
numbers.
This unacknowledged risk is
always present . By not making it
visible, the program is
mortgaging the future without
budgeting for the cost of paying
off the mortgage.
December 2005

47/186
Starting with a good topology for
the IMS is important. Not only
because the programmatic
activities need to be well defined,
but the sensitivity of the risk
analysis depends on a “properly
formed” IMS.
If the logic of the IMS is ill–
formed than the results of the risk
analysis will also be ill–formed.
December 2005

48/186
There are several elements of
the probability model for duration.
Not only are the activities from
the IPTs and CAMs important,
but the subcontractors play on
important role.
The data from the subcontractors
includes:
• Durations and the
probabilities
• The internal connectivity of
the activities that produce the
external; “milestones”
conveyed to the prime
contractor.
• The other programmatic risk
factors for the performance of
subcontractor work
December 2005

49/186
Although the formalities of the
probabilistic risk analysis are not
needed for this briefing. Here is
some background on
terminology.
If we are to learn to “speak” in
probabilistic programmatic risk,
these terms should become
familiar.
This is an almost endless topic,
but some understanding of
probability and statistics is
needed.
This of course requires some
effort and patience .
December 2005

50/186
We should not be drawn into the
illusion that the Central Limit
Theorem is operable for the
program.
This is the core assumption of
PERT and CPM based planning.
This requires normally distributed
completion time and
independence between tasks.
Neither can be verified in
practice.
As such the impact of making
these assumptions is “whistling in
the dark.”
The result is that the program is
late before it starts.
December 2005

51/186
The Central Limit Theorem can
be useful in many cases. But it
needs to be understood where it
is not useful.
The assumptions of the CLT
applied to the PERT problem
mask even more problems when
naively applied to estimating the
duration of a program.
December 2005

52/186
The core of the Central Limit
Theorem of the production of a
Gaussian probability distribution
by assembling a collection of
arbitrary probability distributions.
The primary assumptions that
these distributions are
independent provides the basis
of the CLT.
If the activities represented by
the arbitrary distributions are not
statistically independent – which
is hardly ever the case on a real
project – then the assumptions of
the Central Limit Theorem are
false and the probability
distribution of the program
completion time is no long
Gaussian distributed
December 2005

53/186
What happens is the rollup of the
most likely times of the critical
path activities is biased to an
optimistic location in the
project completion distribution.
This is the fundamental reason
PERT is not very useful.
This criticism is only partly true. If
a probabilistic PERT approach is
used or a Bayesian network
approach is used, then the
deterministic issues are
removed.
But it is easier to use a Monte
Carlo simulator since this avoids
gathering all the underlying
probabilistic distribution
information for an initial estimate
of the completion time of the
program
December 2005

54/186
The probability distribution
function describes the frequency
of occurrence of the events in the
underlying statistical process –
say the duration of a task
completion, the roll of a die, or
the time it takes a light bulb to
burn out.
The ordinate of the graph (the y
axis) is normalize to a scale of [0,
1] which represents the
probability percentage 0.10 =
10%
The abscissa represents the
range of values that can be found
in the underlying sample
population. In this case [0.0, …,
5.0]
The mode is the “most likely”
value to occur when samples are
drawn from the distribution.
December 2005

55/186
The standard deviation is a
description of the “spread” of the
probability distribution function
around the mean.
Without understanding the
standard deviation ,a point
estimate or even a sampled
estimate is of little value.
The shape of the probability
distribution is also important in
understanding the confidence in
a single number. These “higher
order moments” will be discussed
later, but for now no estimated
number should be used without
the standard deviation value
being attached.
December 2005

56/186
Looking at the population
statistics of a random process is
not very useful. Humans have a
hard time making any sense from
the graphs.
The Histogram view can show
the frequency of occurrence of
the various values – how often a
specific value occurs in the total
population of value or the
sampled population of values, but
more insight is needed.
The Cumulative Probability
Density is a way to show this.
The CDF shows the probability
that a sampled number drawn
from the population of all possible
numbers
December 2005

57/186
Various Probability Distribution
Functions (pdf) have similar
Cumulative Density Functions
(CDF).
This is important for several
reasons:
• The underlying probability
distribution function has great
influence on how the end
point values are weighted.
This has impact on the PERT
formula
• The cumulative distribution is
the source of random
numbers in Monte Carlo. For
a variety of pdf’s, similar
CDF’s are generated,
neutralizing the differences in
the pdf’s. Monte Carlo
isolates these underlying
differences. This may be good
or bad depending in the need.
December 2005

59/186
Any estimating process must
address the probabilistic
boundaries of the estimate.
Without this, planners and cost
estimators are hopelessly under
or over estimating duration and
associated cost.
The real issue is not over or
under estimate, but not knowing
which one it is or why.
This lack of knowledge about the
underlying statistical process
creates a greater risk.
December 2005

60/186
Making decisions in a risk neutral
manner is not advised.
We should always talk about risk
adjusted decisions, risk adjusted
values, and risk adjusted
outcomes.
The difference between
alternatives, uncertainties and
outcomes also needs to be
understood. They are not
interchangeable concepts
December 2005

61/186
Decision making must address
the different types of uncertainty.
Understanding how these
uncertainties impact the decision
is critical to selecting alternatives
December 2005

62/186
The idea that we can produce
“estimates” about the future in
the absence of models, historical
data, or a methodology for
discovering these models or
historical data is common in the
IMS planning realm.
Forecasting the future is sporty
business.
December 2005

63/186
The IMS contains “branches”
where the path of work makes a
change in direction.
These braches can be modeled
with a decision tree paradigm.
The risk management discipline
uses this approach. And it is
applicable to the construction of
the probabilistic branching found
in the network of tasks in the
IMS.
December 2005

64/186
In the “olde days” the line of
balance chart was used to
forecast the cost at completion.
December 2005

65/186
PERT and the Critical Path
Method are called out as explicit
methods to be used in the
planning process.
The formulas for PERT are
simplified models of the
underlying complexity of
probabilistic networks (Bayesian
Networks)
As such they have little or no
connection to the reality of the
IMS
December 2005

66/186
In deterministic PERT the
durations are defined as a three
point estimate and the PERT
formula is used to compute the
mean and standard deviation for
the program duration as well as
the critical path.
This is the algorithm used in
Microsoft Project when the PERT
tool bar is turned on and the
three point estimates entered into
the appropriate columns.
It is billed as probabilistic but in
fact the 3–point estimates work
against a fixed probability
distribution function with no way
to adjust its shape, bounds or
moments.
As well, there is no way to insert
the correlations that naturally
occur in the IMS.
December 2005

67/186
The estimates produced by the
deterministic three point data can
be used to construct a
probabilistic PERT if the
underlying probability
distributions are defined for each
task completion time.
The development of the
probability distributions requires
historical data as well as an
understanding the behavior of
each node in the network
(coupling).
This is a difficult task without the
proper tools and data sets.
With the Risk+ tool, individual
distribution functions can be
assigned to each task. But the
“tuning” of each function is
difficult.
December 2005

68/186
When we speak of probabilistic
risk analysis, we also need to
speak of the statistical nature of
the activity network.
When we speak of a probabilistic
activity network (a Bayesian
network) we also need to speak
in terms of probability.
A question that can be asked of
the network is – “ what is the
probability of completing this task
by a certain date?”
A second question that can be
asked is – “what are the
underlying statistics of the
activities of the network?”
A final question that needs to be
asked is “what is the inherent
uncertainty in these estimates?”
In other words – how good is our
ability to guess in the presence of
a statistical process?
December 2005

69/186
Once the activity durations are
treated as probability distributions
it can be seen that they can not
be “added” in the normal sense
to produce a program duration.
They must be “summed” in the
probabilistic sense. This can be
done with Monte Carlo or with
convolution of the Cumulative
Distribution Functions.
Again, add to this the correlation
issues (one task influencing the
outcome of another task), and
the simple approach of adding
the durations to come up with a
total duration falls apart.
December 2005

70/186
Here’s another look at distribution
functions.
This approach should be the
standard vocabulary for
discussing the IMS duration
estimates.
The topological integrity of the
IMS is important, but just as
important is our understanding of
how the activity durations have
been developed, their confidence
interval and the underlying
distribution of the values the
durations can take.
December 2005

71/186
A missing element is the
statistics of the “events” that
occur during the execution of the
program.
For example if a fixed date is
defined in the IMS (this is very
usual for things like IBR, PDR,
CDR), what is the underlying
confidence of that date.
The same is true for
subcontractor provided dates,
where the details of the
deliverables is not visible.
December 2005

72/186
With the input probability
distributions, the program
schedule can be treated as a
“system” with a response
function.
The “system” is a Bayesian
network where the elements of
the network are probabilistic and
the driving function is
probabilistic.
The “output” of the system is
therefore probabilistic as well.
December 2005

73/186
One probability distribution
commonly found in scheduling is
the Beta Functions.
This is a “tunable” probability
distribution function that has
been shown to closely match the
behavior of task completion
durations.
The term “closely” needs to be
used with care. The deviations
between actual completion times
and the “model” of completion
times needs to be assessed
before confidence in the
probabilistic results can be
useful.
December 2005

74/186
The Beta distribution is used for
PERT estimates. This use is
many times done with no
understanding of the shape or
the dynamics of the probability
distribution function. Beta is a
selection for Risk+ as well, with
no obvious way to change the
shape of the curve.
Some understanding of the
impact of the Beta function on
the outcome of the PERT formula
or the Monte Carlo simulation is
needed.
December 2005

75/186
There are many alternatives to
Beta. The Triangle distribution is
one. The triangle distribution has
an intuitive appeal due to its
simplicity useful for estimating
task durations.
But the triangle distribution still
has the problem that the most
likely value and the expected
(mean) value are not the same.
So when planning asks for the
“most likely” value many people
respond with the Mean, which
biases to result in the optimistic
direction.
December 2005

76/186
The triangle distribution can
better describe some statistical
processes, but it too needs
“tuning” for specific task duration
processes.
December 2005

77/186
BetaPERT is currently the vogue
in the probabilistic analysis world.
The BetaPERT distribution
provides a “tunable” curve where
the most likely “Mode” is near or
identical to the “mean” of the
distribution.
December 2005

79/186
The challenge to building a risk
tolerant IMS is the initial capture
of the task durations and the
sensitivity of the IMS to
correlations between tasks.
There is a optimism bias created
when a CAM is asked “what is
the duration?”
The answer is usually a “mean”
(average) duration rather than
the “Mode” (most likely).
If the Mean is used in place of
the Mode, then the three point
estimates are biased to start with
without the explicit knowledge of
the planning staff.
December 2005

80/186
The general flow of creating a
risk tolerant IMS looks like this.
The critical aspect is to get the
CAMs to identify the embedded
risks and the mitigation tasks for
those risks.
Once this is done, “planning” can
then assess if the mitigation
processes make sense in terms
of supporting the AC’s and SA’s
of the IMP/IMS.
Constant and continuous
feedback is needed for this to
work properly.
Without this feedback, the IMS is
assembled in the absence of the
knowledge base and the risk
tolerant aspects are lost or
become confused with the
mainline activities.
December 2005

81/186
The capturing of the risk
information is an interactive
process. A Kaizen is one way to
do this and probably the best.
Having the CAM fill out the “most
likely” durations and identify the
risk mitigations cannot be done
without direct contact.
Without this direct contact,
planners have not chance of
intervening in the process and
the IMS becomes a collection of
tasks rather than an “architected”
plan.
December 2005

82/186
The 3 point estimates required by
DiD–81650 have a variety of
uses.
They can be simple values used
for PERT calculations. These
calculations can be “made up” by
the IPT lead and entered into the
schedule.
A risk adjusted value can be
used from the Macro in Risk+.
The CAM or IPT Lead states the
relative risk in a number between
1 and 5. The macro defines the
percentage boundaries for the
classified risk.
Individual risk ranges can be
developed from historical
information. This is the best
approach, since it represents the
past.
December 2005

83/186
There are several classes of
programmatic risk. Although the
Pareto chart shows that scope
change is the most common,
delays are also common. These
come from the customer side
most often as well.
December 2005

84/186
The classification of risk results
in a percentile or quartile
classification scheme. This is a
better approach than asking
someone for the minimum and
maximum durations.
The challenge is to calibrate
these ranges in a meaningful
manner for the specific program.
There can be general
classification ranges, but having
them set for the specific program
is much better.
This of course requires that data
is kept from past programs,
normalized and then made
available in a form useful for
probabilistic risk analysis.
December 2005

85/186
During the data capturing
process where estimates are
extracted from the technical
experts, there is a natural
tendency to accept the numbers
at face value.
Without qualifying the numbers in
some statistical form, this
information is absorbed into the
IMS or Cost and becomes “fact.”
These “facts” then progress
through the program and are
never challenged for their lack of
statistical basis.
December 2005

86/186
The core problem with capturing
estimate from human beings is
they are biased.
Either negatively biased or
positively biased.
There is plenty of literature on
this effect and ways to overcome
it. For now we’ll just live with the
outcome of the bias
December 2005

87/186
Let’s take another tour of the
problems with PERT. These
issues are well documented in
the literature, but poorly
understood in practice.
The poor understanding comes
from the difficulty of the
explanation – statistical
conversations are usually not
very interesting; and the natural
tendency to look for easy
answers to complex problems.
The core issue is that without a
deep understanding of the errors
produced by the PERT equation,
the confidence in completion
dates and the risk tolerance of
the IMS is difficult to build.
When the actual numbers come
in (ACWP and BCWP) and they
don’t match the expectations – is
it the original plan or the
underlying performance?
December 2005

88/186
There are several myths about
PERT. The first is that is was
scientifically thought out in detail.
This is not the case. The book
The Management of Projects,
Peter W. G. Morris provides the
background on this development
as well as other project
management histories.
The second historical myth is that
PERT is a general purpose
approach. In fact it is very
specialized and is applicable to a
narrow range of activity
networks. Those with normally
distributed completion times,
statistically independent
relationships, ones where the
critical path does not change and
with the “most likely” estimate
actually representing the “mode”
of the underlying probability
distribution function.
December 2005

89/186
When a manger asks “what is the
critical path for this program?”
there are several thoughts and
actions:
• In a probabilistic activity
network there are many
critical paths, which change
as a function of time,
adjustments to the risk
profiles, and the completion of
work.
• Correlated activities are
influenced by off–critical–path
activities to place them on the
critical path.
So the answer to the
management question is “it
depends on what you mean by
critical and path.”
The real answer only comes by
moving away from the static
representations of the IMS to a
probabilistic representation – and
that requires much more effort.
December 2005

90/186
Once we recognize that the
activity network is probabilistic in
nature, the first choice (the naïve
choice) is to apply the PERT
method.
While this may be a useful “first”
choice it produces results that
are overly optimistic and
sometimes overly pessimistic.
Either way they are wrong from a
statistical point of view. They are
wrong because the assumptions
of PERT are wrong. These
assumptions are almost never
found to be true in practice.
Even if they were true, the
probability distribution function
used by PERT does not
represent any useful activity
completion time distribution.
December 2005

91/186
One of the “killer” assumptions in
PERT is the lack of
understanding of “merge bias.”
Merge Bias occurs when two or
more activities are joined at a
merge point. Usually a milestone
or a simple Finish to Start of
several tasks.
The result is the statistical
behavior of the activities prior to
this merge point influence the
statistics of the following
activities in undesirable ways.
December 2005

92/186
Since statistical distributions can
not be simply “added” the
duration of the downstream
activity is not the sum of the
duration of all the upstream
activities (or the longest activity).
Instead it is the statistical sum
(convolution) of the probability
distribution function (pdf)
Without understanding this, the
PERT estimate generates an
optimistic estimate of the
duration, since the PERT formula
simply adds the durations to
arrive at the total duration.
The PERT formula also adds the
individual activity variances to
arrive at a total project variance.
While this provided a simple
method to “guess” the total
duration it produces a poor model
for real analysis of risk.
December 2005

93/186
The PERT approach fails to
consider the “random variable”
nature of the dates in activity
network.
As well the correlation between
each of these random variables
is not considered.
The result is the potential for
large variances in the completion
time estimates – 15% is not
uncommon.
December 2005

94/186
The visual impacts of Merge
Point bias is show here. This is a
small and sample activity
network. A “real” network would
have different outcomes.
It is not important exactly how the
merge point bias impacts the
final completion date, but that the
merge point bias DOES impact
the final completion date.
December 2005

95/186
How the activity network is
arranged has significant impact
on the calculations for PERT.
Here are some examples.
Notice that the PERT mean (the
average) stays the same, while
the “real” mean and the variance
on that mean change
dramatically depending on the
arrangement.
December 2005

96/186
The reason for these changes
involves how the statistics are
“added” in the various
configurations.
The critical concept is that the
PERT calculations are unreliable
as a predictor of the completion
time in a probabilistic model of
the activities.
December 2005

97/186
The effect of the merge bias is
shown in the graph. It is unlikely
in any real plan that only three
parallel paths exists. This number
is usually much larger,
sometimes in the dozens.
All of this discussion is leading to
the suggestion that PERT is not
viable on any complex program.
December 2005

98/186
Use Monte Carlo, don’t use
PERT.
The problem of course is that
DID 81650 and even the
corporate guidelines either
require or strongly suggest the
use of PERT and CPM.
This can be done of course, but
don’t use the numbers for any
real planning processes.
December 2005

99/186
The use of Monte Carlo
simulation is a logical outcome of
the problems with PERT.
What is missing is the
understanding of how Monte
Carlo works, what it’s limitations
are, where it should not be used
and of course how to interpret
the outcomes when they don’t
meet our expectations.
Even though Monte Carlo is a
powerful tool it can produce
unexpected results. This section
is an attempt to give some
background on the mathematics
and stimulate further interest in
applying this tool to the problem
of schedule forecasting
December 2005

100/186
Monte Carlo simulations provide
a useful approach to modeling
schedule risk. But their value is
more than that.
Unlike PERT or other
deterministic approaches – even
though the three point estimates
are billed as probabilistic – Monte
Carlo examines the schedule
network independent of a critical
path, topological constraints or
other “human induced” problems.
It looks at the network as a
collection of nodes and arcs,
independent of the “meaning” of
this information and produces a
model of the behavior of these
nodes and arcs
December 2005

101/186
The concept behind Monte Carlo
is to sample the possible
durations for a task from the
population of all durations and
apply them to the schedule.
The population of possible
samples is defined by the
Cumulative Probability Density
(CDF) function for each task.
This in turn is defined by the 3–
point estimate for the task, which
selects the bounds in the CDF for
sampling.
Since there is no direct concept
of a Critical Path in Monte Carlo,
the near critical path tasks are
considered in the analysis of the
completion time.
As well the PERT biases
produced by the simple minded
PERT formula are avoided as
well.
December 2005

102/186
There are several “components”
to the Monte Carlo process. So
when we speak of Monte Carlo it
is both a process and a product –
in our current case Risk+
December 2005

103/186
Samples drawn from the
underlying distribution function
can produce an “error estimate”
on a completion date.
These error estimates are
different than the fixed
boundaries for PERT, since they
represent the actual probabilities
distribution error bounds
December 2005

104/186
The number of sample runs
needs to be sufficient to cover all
the possibilities in the pdf.
This is usually 500.
A production run for a Monte
Carlo simulation is around 2,000
to 3,000 iterations.
As the iteration count increases
the fidelity of the simulation
increases.
But there is a point where more
samples don’t add value. This
point can be determined by the
statistical performance of the
variance of these sample space.
December 2005

105/186
Since Monte Carlo does not need
to know about the Critical Path, it
is conceptually simpler to use.
A well formed network is needed
and the 3–point estimates need
to represent the proper risk
assessment.
December 2005

106/186
This is a view of how Risk+ sets
up the project file.
December 2005

107/186
The result is a cumulative
distribution and a probability
distribution function.
Interpreting this result is straight
forward.
The confidence of each date is
shown in the table on the right.
This is the probability of
completing the task by the date.
December 2005

108/186
A good IMS is needed.
The risk assessments should be
done with a ranking process
rather than specific 3–point
estimates. This disconnects the
personal opinions from the
assignment of risk.
A 5 level process is one
approach, but any odd numbered
level ranking is best.
The differences between the
levels should be geometric not
linear.
December 2005

109/186
Risk+ generates lots of
information useful for the
analysis of the program.
December 2005

December 2005 110/186

111/186
Constructing a robust IMS means
building a “risk tolerant” plan.
The robustness of the plan
means that it (the plan) can deal
with disruptions that occur
naturally through the course of
execution or un–naturally through
external events.
In either case the “robustness” of
the plan must be visible to the
evaluator without any detailed
explanation, beyond the IMS
narrative. No hand waving
explanations of how the plan
works. The risk tolerant aspects
most be obvious.
December 2005

112/186
Thinking about schedule
contingency is different in a PRA
context. For a simple project,
15% contingency is assumed.
But placing the contingency is
the first problem. The process is:
• Run Risk+ and watch the final
date.
• Compare the 80% confidence
date against the deterministic
date. This difference is the
first cut at the needed margin.
• Assign this duration across
the project in front of the
critical (high risk) milestones.
• Rerun Risk+ and add or
subtract this margin until the
desired confidence date is
achieved.
December 2005

113/186
More detailed statistics and
interpretations of the results an
be produced with Risk+.
This information can them be
used to perform further analysis
of the IMS. The analysis is what
we’re after, not just the date
produced by Risk+.
Like the PERT numbers, the
Risk+ numbers must be
interpreted with the
understanding of how they were
arrived at.
This is one of the purposed of
this briefing – to provide
knowledge of how to use this
approach and what its strengths
and weaknesses are.
December 2005

114/186
Incorporating Technical
Performance Measures (TPM)
with Monte Carlo is a powerful
way of showing how risk is
reduced and maturity increased
in a program.
At each step in the program –
each Program Event – a target
confidence interval for a
completion date can be forecast.
Along with the technical
performance measure, this
programmatic performance
measure approach results in a
“risk tolerant” IMS.
December 2005

115/186
Risk tolerance in the IMS
requires more than just the
planning processes. It requires
the connections to technical and
cost.
This has been stated before, but
it needs to be made not only
visible but actionable.
December 2005

116/186
Using a simple process steps,
risk tolerance can be developed
from the same processes used
by the technical risk engineers.
December 2005

117/186
The goal here is to move the
integrated risk tolerance –
technical, schedule, cost –
forward from a dis–integrated
plan to an integrated plan
December 2005

118/186
Read the chart as follows: The upper
horizontal band on the plot is “Ready
Early”. “Ready On–time” is the middle
band that also spans the launch
window. “Ready late” is the lower band,
which means a 6–month slip to the next
launch window and all associated costs
that go with that slip. The upper line
plotted is the deterministic
completion date (i.e. no risk) and the
lower line plotted with the 20th and 80th
percentile confidence bands on the risk–
adjusted completion date. The project’s
objective is to continue to invest in risk
mitigation actions until the band and the
area of highest likelihood is no longer in
the “Missed Launch Period” area of the
chart. Note the improving trend over
time indicating the success of the risk
mitigation actions as well some
“Accepted” risks passing their exposure
window without becoming problems.
Taken from [Risk Based Decision
Support techniques for Programs and
Projects]
http://www.futron.com/pdf/RBDSsupport
tech.pdf
December 2005

119/186
As Program Events progress the
risk mitigation processes need to
progress as well.
December 2005

120/186
Here’s a 4 step progress for
installing risk in the IMS and
producing a “risk tolerant” plan
December 2005

121/186
The use of branching
probabilities is important for the
assessment of the “risk
tolerance.”
December 2005

122/186
December 2005

123/186
The use of Risk+ and Monte
Carlo replaces the PERT
approach to schedule duration
probability analysis.
December 2005

124/186
The “goodness” of the IMS is
important to the quality of the
results
December 2005

125/186
The distribution to use for a task
depends on the underlying risk
profile.
Triangle is common, but it over
biased the risk on the high end.
Beta can be used, but the simple
Beta distributions in Risk+ may
not represent the real risk profile.
BetaPERT is the better one, but
Risk+ does not support it.
December 2005

126/186
Which tasks drive the sensitivity
of a completion date needs to be
understood. Not all tasks have
the same impact on the outcome.
The “tornado” chart is one way of
showing this.
The Power Law’s behind Pareto’s
rule is worth understanding for
many reasons, not just schedule
and cost modeling. Power Laws
occur across a wide variety of
domains, from moon crater sizes
to the frequency of words in
English.
http://www.nslij–
genetics.org/wli/zipf/ is a good
place to look for the impacts of
Power Laws on everyday life.
December 2005

127/186
In order to build a model of the
schedule we have to start with
the schedule. But first we have to
start with the model of the
schedule.
This is the role of the IMP, but the
connections between the ACs
are needed, not just the list of the
IMP elements.
From this model the schedule
elements can be arranged to
follow the strategy of the IMP
rather than represent the
passage of time and the
consumption of resources.
From there a model of the risk
areas, mitigations, parallel
development paths, reevaluation
points, and “hot spots” (sensitivity
analysis) can be extracted. This
information can them be used to
assess the robustness of the IMS
December 2005

128/186
The primary graphic for an IMS
evaluation is the cumulative
probability of a completion time.
This is technically referred to as
the Cumulative Density Function
(CDF)
This is the format most useful for
answering the question – how
long will this take?
December 2005

129/186
The confidence intervals
produced by the CDF can be
assessed over time against
targets.
These targets can be Technical
Performance Measures or any
other style of metric that is
connected with cost, schedule
and technical performance
December 2005

130/186
Another view is the confidence in
the schedule dates as a function
of time.
December 2005

131/186
It is important to understand the
sensitivity of a completion time to
the various “drivers” of this
sensitivity.
This makes visible the “hot spots”
in the IMS that require attention,
mitigation, or even re–planning to
reduce sensitivity
December 2005

132/186
In Monte Carlo each task can
take on a wide variety of roles. It
can be the driver for the total
schedule duration at one time,
and at another time (in the
simulation) have little effect on
the outcome.
The Criticality of the task is how
“important” it is as a function of
the number of simulation runs.
The higher the criticality of the
task, the more important it is to
look at the details and determine
what mitigations should take
place to keep this task lower in
the criticality index.
December 2005

133/186
When sensitivity and criticality
are combined a sense of the
cruciality. Cruciality is defined as
“a state of critical urgency.”
Although this sounds like a
redundancy term, it can be used
to focus our attention on those
tasks that are both critical and
sensitive.
It is important to understand the
sensitivity aspects, since these
can change and drive the
schedule in non–obvious ways.
December 2005

December 2005 134/186

135/186
Let’s look at some examples of
Monte Carlo
December 2005

136/186
The Monte Carlo simulation
makes use of the three point
estimates generated during the
PERT analysis. This numbers
represent the upper, lower and
most likely durations.
This values are then used to
draw random numbers from the
probability distribution for
evaluating the activity network.
The branching probabilities can
then be added for the
alternatives paths and risk
mitigation activities.
December 2005

137/186
The use of “expert judgment”
itself needs to be calibrated.
The unanswered question on this
program and many others is
“what does a good risk tolerant
IMS actually look like?”
The “units of measure” for risk
tolerance and the confidence in
the probabilistic estimates needs
to be established before the
estimating and modeling process
can be “calibrated”
December 2005

138/186
The ranking of risk or the ranking
of anything needs to be done in a
structured manner.
A geometric progression is a very
useful approach, since it forces
the focus on ranking.
December 2005

139/186
The “sense” of risk and real risk
need to be connected.
December 2005

140/186
Some type of risk ranking needs
to be developed for the IMS
tasks.
One approach is the TRL scale.
December 2005

141/186
When tasks are arranged in
series the cumulative probability
of completion is show in the table
on the right
December 2005

142/186
When the tasks are arranged in
parallel a different completion
profile results.
December 2005

143/186
All of this is very interesting in a
Power Point presentation –
marketecture it's called.
Let’s look at a real schedule and
start to apply some of the things
we’ve learned.
December 2005

144/186
This is a very simple construction
plan. The tasks are networked in
a way to show how the Risk+ tool
works.
December 2005

145/186
The first picture of a completion
time is the PERT assessment.
The task Construction Schedule
Margin (the end of this task is the
end of the planned margin) has a
target date of 2/8/06 and a
forecast PERT date of 3/6/06.
This shows there is not enough
margin by one month for this task
December 2005

146/186
The same task, evaluated with
Risk+ shows a different
completion date.
December 2005

147/186
Project Risk Analysis is part of
any good risk management
activity. This has been said
numerous times and needs to he
repeated daily.
Both the technical and the
programmatic risk aspects of the
program need to be shown in the
IMS.
Any questions, changes,
updates, suggestions – anything
that touches the IMS or the cost
model – needs to be assessed
from the point of view of
programmatic risk.
December 2005

148/186
The accuracy of the dates and
costs in the IMS is a “relative”
term.
±20% to start with is pretty good.
As the program proceeds
accuracy improves but it is
always a statistical estimate until
after the fact.
December 2005

149/186
If we take a deterministic
approach the planning then there
will be built in issues. The first is
that all estimates must include a
confidence interval or they are
wrong.
The “natural” approach to
estimating almost always results
in a bound that is too wide as
well as being optimistic or
pessimistic but hardly never
accurate.
December 2005

December 2005 150/186

151/186
Now that we’ve reached a fairly
detailed level of discussion
regarding programmatic risk
assessment, it’s time to talk
about cost risk assessment.
The first concept to understand is
that cost and schedule are
connected. This is obvious. But
they are not connected in any
linear manner.
December 2005

152/186
The basic principles of cost
estimating start with the
understanding of the uncertainty
in the estimates of cost.
These uncertainties must be
connected to the technical
uncertainties as well as the
programmatic and simple cost
variances.
The arithmetic addition of costs
creates a false number of the not
only the cost but any variance in
this cost.
Monte Carlo simulation is one
starting point, but like the
programmatic simulations, the
underlying probability
distributions must be understood
before the numbers have any
real meaning
December 2005

153/186
The connection between a
technical parameter is its cost is
not only potentially non–linear it
is probabilistically non-linear.
Knowing these Cost Element
Relationships (CER) is a critical
success factor for managing in
the presence of uncertainty for
both reducible and irreducible
risks that are created
December 2005

154/186
A simple 9 step (not so simple
actually) process can be used to
build a cost estimate.
Starting with the “likely” program
in the form of an IMS, the tasks
for delivering that program are
defined.
The underlying probability
distributions for the cost of each
delivering activity are developed.
This is much like the
development of the baseline IMS,
but the next step is much
different.
The correlation between each
WBS element is developed.
These correlations are used to
build a model of sensitivity of the
cost to changes in the tasks.
December 2005

155/186
At this point the Risk+ tool fails to
deliver what is needed. Wither
Crystal Ball or @RISK is needed
to connect these correlations
together.
The technical uncertainty of the
program is used to drive the cost
uncertainty. This is where the
technical and programmatic risk
assessments joins.
December 2005

156/186
The production of the familiar
probability curves for the
likelihood of cost is the result.
This curve tells is the likelihood
that some event will occur.
For example the probability of a
cost in this example.
December 2005

157/186
The risk margin in dollars is the
result needed to make this
connection.
Without this margin, the budget
for the program ‒ as a single
point estimate ‒ is not credible.
All programs operate in the
presence of uncertainty, so no
single point estimate can be
correct.
For reducible uncertainties,
specific actions can be take to
protect the outcome.
For irreducible uncertainties, only
margin can protect the outcomes
December 2005

158/186
Without the cost margin, the
result is the risk that the budget
is not sufficient to complete the
program without a cost overrun.
December 2005

159/186
Management reserve,
contingency and other budget
protection is then needed.
Much much reserve and
contingency comes from
modeling the program and
determining the level of
confidence needed to protect the
budget from overrun.
December 2005

160/186
With models of both the cost
uncertainties (reducible and
irreducible) and schedule
uncertainties (reducible and
irreducible) a Risk Adjusted
Integrated Master Schedule can
be created.
This IMS has Risk Handling
activities for the Reducible Risks
and Margin for the Irreducible
Risks
December 2005

161/186
With the Risk Handling content in
place, a new assessment of the
probabilistic outcomes can be
shown.
Before risk management and
after risk management.
This is no actionable information
to the decision makers.
Was the risk reduced enough to
start the program, continue the
program? Or are further risk
reduction activities needed?
December 2005

162/186
The cumulative cost curve – in
this example, can then be used
to make decisions using the
value at risk
December 2005

163/186
All programs operate in the
presence of uncertainty.
Making decisions in the presence
of this uncertainty can take place
with the proper information.
December 2005

December 2005 164/186

165/186
We’re near the end now, so your
brain is certainly getting full.
This is quite a bit of information
to absorb, but it needs to be
done before we can say we are
building “risk tolerant” plans.
December 2005

166/186
When probabilistic schedule
analysis is used it does not
replace the need for a well
formed project network. It only
replaces the use of PERT for
estimating the completion dates.
December 2005

167/186
The quality of the probabilistic
estimates is the foundation of
confidence.
The next step is to clearly identify
where in the IMS risks are being
mitigated, the impacts of this
mitigation and the overall
confidence in the master plan
resulting from this mitigation
December 2005

168/186
The correlations between cost,
schedule, and technical risk must
be made explicit.
A model of how these elements
interact is the basis for answering
the “what if” questions that occur
when the risk item becomes
active.
December 2005

169/186
Risk based schedule and cost
management is core to
programmatic integrity.
December 2005

December 2005 170/186

171/186
Like any good idea it can be
improved on forever.
These opportunities are much
harder to address than the
process so far. They require care
and effort to build a correlation
matrix for the tasks. They require
detailed understanding of the
underlying statistical processes
and the historical data that was
used to develop these
distributions.
For most projects this is beyond
the scope of the effort and may
be beyond the business interests
as well – since the pay back is
not clearly defined.
December 2005

172/186
The dependencies between tasks
is the basis of the correlation
function. This is very important if
a true model of the network is to
be developed. In the absence of
the correlations it is assumed
tasks are independent, which of
course can not be the case.
Building a Program risk
assessment requires that cost
and schedule be connected as
well – correlated. Cost and
schedule are not linear, so any
simple model of changes in one
linearly effecting the other cannot
work.
Finally the idea of a causal model
– a cause and a set of effects
provides deeper insight into the
risk behaviors of the network.
December 2005

173/186
There is too much information
here for a single digestive
process. The only way to absorb
all this is to start practicing
probabilistic schedule and cost
analysis and make the
knowledge appear in the output
information.
December 2005

174/186
December 2005

175/186
December 2005

176/186
December 2005

177/186
December 2005

178/186
December 2005

179/186
December 2005

180/186
December 2005

181/186
December 2005

182/186
December 2005

183/186
December 2005

184/186
December 2005

185/186
There are nearly unlimited
resources on the web. The
challenge of course is finding
them.
Here’s some know starting
points.
December 2005

186/186
This has been a long journey
over hopefully many weeks of
discussion and hands on
experience with Risk+ and real
project schedules.
Building a risk tolerant IMS is a
“practice” and practices require
proficiency. Proficiency comes
from “doing the work,” looking at
the results and making changes
for improvement.
This is just the beginning.
December 2005

Probabilistic Risk Management (With Notes)

More Related Content

What's hot

Similar to Probabilistic Risk Management (With Notes)

More from Glen Alleman

Recently uploaded

Probabilistic Risk Management (With Notes)