SlideShare a Scribd company logo
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
1/186
This briefing is an overview of the
probabilistic risk analysis
processes that can be applied to
our program. Although it may not
appear to be a “simple” overview,
this material is the tip of the
iceberg of this complex topic.
Just schedule analysis has been
addressed in detail here. The
cost aspects of forecasting and
simulation must be addressed as
well to complete the connections
between schedule and cost.
Probabilistic cost will be surveyed
here, but an in depth review is for
a later time.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
2/186
An important aspect of education
and research in our business
domain, is “Fair Use” copyright
law.
All the material in this briefing is
accessible through the internet.
Conference proceedings journal
articles, company white papers
and other public sources form the
basis of much of this material
and are referenced in the
bibliography.
Some materials in this briefing
make references to other
copyrighted materials in the
course of research, investigation,
and analysis. These references
are solely intended for non–
commercial use and as such
have no intent to infringe on the
copyright holder. All attempts
have been made to acknowledge
the original copyright holder in
pursuit of fair use laws as
currently defined in the United
States.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
3/186
The concept that risk and the
management of risk is a
desirable part of our program is
not always appreciated or well
understood
Without risk there can be no
opportunities. The plans for the
program become static and
deterministic.
While risk and opportunity are
related, the management of risk
is not the complement of
opportunity. - even if this is a
popular notions these days.
See the Conrow, AT&L article for
detailed discussion of this
somewhat controversial topic.
The primary opportunity in
Programmatic Risk Management
is the avoidance of being late and
over budget on the planned
launch date.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
4/186
When we use the term “risk
tolerant IMS” it means a plan and
supporting that can tolerate risks.
Technical risks and programmatic
risks. These risks are built into
the program by its very nature.
These risks must be addressed
both technically and
programmatically.
The real challenge though is not
how to address them, but how to
recognize that they are being
addressed in a manner that
actually reduces the level of risk
as the program proceeds along
its path to final maturity.
A measure of “increasing
maturity” is the reduction of risk
made visible to the evaluator of
the IMS.
The materials here guide us
through the process of building a
risk tolerant IMS. But putting it to
work still requires practice.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
5/186
The credibility of the Integrated
Master Schedule (IMS) is the
critical success factor for both
our proposal and our execution
phase after the win.
Without a credible schedule and
the related cost credibility, there
is a low probability of a win.
The effort put into constructing a
credible schedule during the
proposal phase will pay off
(assuming the program structure
remains intact) during the
execution phase.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
6/186
The skills of creating and
managing a schedule and the
associated cost require special
understanding.
However, the planners are
usually the last in a long line of
“culprits” for finding the cause of
any failure.
This is a “no win” situation.
People skills, project
management skills, and some
level of technical skill is needed.
But most important is the people
skill, since the knowledge of how
to assemble a successful IMS
resides in the minds of others.
Getting this knowledge out and
on paper requires interpersonal
communication as a primary
process, not technical tools and
formal processes.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
7/186
Understanding the difference
between qualitative and
quantitative risk assessment is
important.
Our first approach is usually
qualitative.
But what is needed is
quantitative.
A specific measure of
programmatic risk, is the impact
of the mitigations or risk
retirement activities and measure
of the increasing maturity of the
program deliverables in the
presence of risk.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
8/186
Programmatic risk management
makes visible the technical risk
mitigation steps as well as the
alternative programmatic
processes in the presence of
these risks.
Alternative branching in the IMS
must be defined to a level of
detail that instills confidence that
the IMS properly represents a
“risk tolerant” plan.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
9/186
Since there is quite a bit of
material here, a quick overview
will get us started.
The executive overview should
leave the reader with a sense of
the important topics
• There are no point estimates
allowed in planning. All
estimates must be
probabilistic
• There are core issues with
simple (deterministic) PERT
and it is not to be trusted
• The use of a probabilistic tool
is useful, but understanding
how the underlying statistic
works is critical to its use in
planning and program
execution
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
10/186
When asked “why are we doing
this?” many would answer –
because our customer wants us
to.
This would be too simple an
answer.
The main reason is, most
programs are simply too complex
not to have a better
understanding of how the
programmatic and technical risks
interact.
Not understanding the interaction
between these two types of risk
that creates the biggest risk.
Individually these risk “could” be
managed. But when combined
they behave in unpredictable and
maybe unknowable ways.
This is a core feature of any
system. See Systems Bible
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
11/186
If we get only two concepts out of
this briefing they should be:
• There are multiple critical
paths in any executing
program. Asking “what is the
critical” indicates that the
questioner does not
understand the probabilistic
nature of the program
• PERT is a poor estimating
metric. It has built in biases
which under estimate the total
duration of the program.
Monte Carlo is a better
estimating tool, but it too
needs careful adjustment
before realistic numbers can
be derived.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
12/186
The DID–MGMT–81650
describes the Integrated Master
Schedule.
Integrating Programmatic and
Technical risk identification and
mitigation adds credibility to the
IMS and therefore to the overall
program.
Applying probabilistic risk
analysis to the IMS is mandated,
but care is needed to interpret
the results.
These tools aid in the evaluation,
but they are not replacements for
good program management
processes.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
13/186
The idea that uncertainty and the
risk that it produces can be
“programmed out” of the
schedule is a false hope.
Without understanding the
principles of Deming, the
management and the planning
staff will be “chasing their tail,”
trying to control the naturally
occurring variances in the plan.
The first approach is to set the
error bands wide enough to not
trigger an exception report for
these variances.
This approach is “good enough”
but what is missing is the
knowledge of “how wide is wide
enough?” for a specific set of
tasks or during a specific phase
of a program?
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
14/186
The first step in the process of
adding credibility to the IMS is to
recognize that all task completion
times are random variables.
They are not “point” numbers
(scalars) but are “estimates” of
the completion time drawn from a
probability distribution of the
underlying population of all
completion times possible for the
specific task.
Modeling schedule durations are
random variables does not imply
these durations are “random.” It
reflects how a duration’s
uncertainty is influenced by the
underlying probabilistic nature of
the activity network.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
15/186
Building a credible IMS starts
with identifying the architecture of
the IMP and the supporting tasks
in the IMS.
Although this is restating the
obvious the process to do this is
actually quite hard.
Adding schedule and cost risk
identification and mitigation to the
process is the minimal result for
a winning proposal.
It cannot be emphasized enough
– the architecture of the IMS is
critical to identifying a risk
tolerant schedule. The “rats nest”
approach is simply unacceptable
to the success of any program.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
16/186
The goal of introducing
probabilistic schedule and cost
analysis is to improve the
probability of a “win” on the
proposal.
While winning is important,
executing the program is even
more important.
What ever “credibility” elements
were in the proposed IMS need
to be carried into the execution
schedule.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
17/186
The use of Monte Carlo for
assessing the IMS must be
turned into forecasting
performance.
This is done by identifying the
“hot spots” in the IMS through
sensitivity analysis, interventions
for these “hot spots” and the
measure of change resulting
from the intervention.
The important concept is to
connect metrics to measurable
benefits to the program. Without
this the creation of metrics is just
wasted effort.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005 18/186
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
19/186
Using risk and uncertainty as an
integral part of the planning
process is a sign of maturity.
Making decisions on the this risk
information improves maturity.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
20/186
When we speak of risk
management, either technical or
programmatic, the term usually
has a very localized context.
For the planning context risk
management must include both
technical and programmatic risk.
The technical risk aspects come
from external sources but are
directly represented in the IMS.
The programmatic impacts of this
technical risk must be explicitly
addressed.
This is the easy part.
The hard part is determining the
implicit programmatic risk that is
derived from the technical risk
and the risks that are derived
from the “architecture” of the
program itself.
This is where the true “risk
tolerant” IMS adds value.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
21/186
There are many approaches to
building a risk tolerant IMS. Our
current approach is to add risk
factors and margin to specific
areas of the IMS
The current approach to use a
Monte Carlo tool to assess where
this margin should be placed.
There are several other steps
along the way. Which steps to
take, how much effort to invest
and how to recognize the value
of this investment are some of
the management challenges as
well as the technical challenges.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
22/186
The difference between risk and
uncertainty needs to be
understood at some level.
For the most part the differences
are not important in the
beginning.
But once decisions start to be
made about mitigation steps,
branching probabilities for failure
modes, these differences
become more important.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
23/186
When we use the term
uncertainty or risk it means at
least 4 things.
First let’s sort out “uncertainty”
There are two classes of
uncertainty in large complex
programs.
• Static uncertainty emerges
from the natural variations in
the completion times of tasks.
This is a Deming uncertainty.
http://webserver.lemoyne.edu/
~wright/deming.htm is an
example of this type of
uncertainty
• The dynamic uncertainty is
about the unknowns and the
unknowable
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
24/186
The static uncertainty in a
program can be addressed
directly in the plan with mitigation
tasks.
The dynamic uncertainty arises
from the dynamic interactions
between the tasks of the plan.
This interaction and the
outcomes to the end date cannot
be modeled with static
paradigms.
Monte Carlo simulation is an
approach to modeling these
interactions and their impact on
other elements of the plan
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
25/186
Managing risk in the schedule
requires anticipation to identify
the risks, but also requires
understanding of the source of
risk, the impacts of these risks,
and the interaction between the
risks and the plan.
A process is needed to guide the
risk management activities. This
process must address both the
programmatic as well as
technical risk. The interaction
between programmatic and
technical risks must also be
managed.
These interactions must be
considered a “first order”
interaction.
The common approach is to
consider the technical risk as first
order and the programmatic risks
secondary.
The combination becomes a first
order interaction.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
26/186
As planners our goal must be to
produce a plan that has credibility
and integrity.
Credible plans are believable
plans
Integrity plans are trustworthy
plans.
Both attributes are needed for a
winning proposal and the follow
on execution.
The successful assessment of
the IMS during a proposal or
during execution by the customer
or DCMA depends on how
believable the plan is and how
well it can be assessed to
confirm this believability
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
27/186
The assessment of the credibility
and integrity of the IMS can take
place by asking some questions.
These and similar questions
shine light on the underlying
attributes of the IMS in ways that
simple assessments do not.
These are not technical
assessment, like counting data in
the predecessors field, but are
architectural questions about the
“quality” of the IMS independent
of the technical details.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
28/186
NASA does risk management in
a specific way. We need to
understand their way as a
starting point.
Reading the NASA materials is a
start, but there is other research
available from conferences and
vendor web sites that needs to
be gathered and read as well.
Other government agencies as
well as civilian firms have similar
risk management approaches.
NASA’s approach is a good
starting because of manned
space flight’s inherent risk. And
NASA’s emphasis on Safety and
Mission Assurance.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
29/186
The IRMA tool developed at
NASA Johnson Space Center is
the basis of risk management for
a NASA side.
Although this approach is
focused on the technical risks the
programmatic risks appear in the
database.
As well there are other risk
management systems and
paradigms.
Active Risk Manager (ARM) is a
popular one as well,
http://www.strategicthought.com/
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
30/186
The NASA Risk Management
Summary Card calls out
“schedule” impacts in three
places.
Connecting programmatic and
technical risk is a critical success
factor for a proposal as well as
an execution assessment.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
31/186
Adding probabilistic schedule and
risk analysis to the IMS can be
done through a structured
process.
1. The initiating event of the
risk is identified.
2. The result from this event is
described
3. The consequence that flow
from the scenario are
developed
4. The connections, flows,
interactions and correlations
between the scenarios are
modeled
5. The probability of
occurrence for each of these
scenarios is developed
6. The model of the probability
of occurrence and
consequences from the
occurrence are combined
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
32/186
The Continuous Risk
Management paradigm found in
the technical risk world can be
applied to the programmatic risk
as well.
NASA has adopted Continuous
Risk Management (CRM)
through several guidelines listed
here.
The table summarizes how CRM
is managed in a structured
manner throughout the program
life,
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
33/186
There is a difference between the
design evaluation of the IMS and
the risk evaluation.
The design evaluation describes
how the technical activities
needed to develop and deploy
the product – in this case a
manned spacecraft – must come
together in the right sequence to
make the planned completion
date.
The risk evaluation defines the
probabilistic completion model for
each task, the correlations
between the tasks and the
resulting probabilistic model.
This model is a Bayesian
Network of all the tasks.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
34/186
To construct an IMS with integrity
and credibility both technical and
programmatic risk must be
connected.
This process starts with the
identification of the technical
risks in ARM and their mitigations
in the IMS. This is the explicit risk
approach.
Next comes the explicit
programmatic risk activities. This
can be the well known margin
needed in front of major
milestones, program events or
deliverables.
Finally comes the implicit risk
mitigation activities that will be
needed to differentiate this IMS
from any other IMS to start to
build confidence that we have a
“risk tolerant” IMS.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
35/186
A pedagogical literature survey
from the RAND Corporation
supports the notion that
probabilistic risk assessment is
not seen in a favorable light by
management.
• It is too complex.
• The underlying statistic are not
will understood.
• “It’s the customers that are
asking for this.”
• There is little historical data to
calibrate the underlying
probability distribution functions
for task completion times.
All of these gaps must be closed
in some way in order to call our
IMS Risk Tolerant
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005 36/186
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
37/186
Managing in the presence of
uncertainty is the core behavior
for any modern program.
Trying to control this uncertainty
requires two basic
understandings:
1. The natural variations in the
schedule cannot be
sufficiently controlled to
remove risk. These are the
Deming variations and the
foreseen uncertainties
2. The unforeseen uncertainties
and the inherent chaos of the
program must be dealt with
through contingencies
Attempting to manage
uncertainty is limited to foreseen
risk. Managing in the presence of
uncertainty deals with unforeseen
and chaotic sources of risk
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
38/186
When estimating the completion
times for tasks, there are three
primary problems.
1. A number produced by a
CAM or an IPT must be a
statistical estimate, not a
specific duration.
2. The meaning of “best” must
be established prior to
accepting the statistical
estimate
3. The collecting of the “most
likely” estimates cannot be
added in the sense of
adding scalar numbers,
since they are probability
distributions.
4. The “most likely” is NOT the
average completion time, it
is the completion time that
occurs most often from a
large sample of possible
completion times.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
39/186
The first approach to “planning”
the program is to ask the CAMs
or IPT Leads for each task in
their WBS or IMP/IMS area: “how
long with this take to do?”
The numbers that come back are
then entered in the duration field
on the schedule.
These numbers are not only
wrong they are dangerously
wrong.
They are “point” estimates that
live inside a probability
distribution.
The built in bias from the
approach has clinically be shown
to be optimistic or pessimistic,
but rarely “most likely.”
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
40/186
The traditional approach is to roll
up the single point estimates into
a sum of the durations and
search for the longest path.
This is the Critical Path Method
(CPM) for assessing the finish
date of the plan.
The problem of course is these
“numbers” are not actual scalar
values. They are samples drawn
from probability distributions.
Addition is not mathematically
possible in the sense of addition,
defined over the set of natural
numbers (0, 1, 2, … ∞]
These probability distributions
can be “convolved” into a new
probability distribution, but a
better approach is Monte Carlo
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
41/186
When asked “what is the most
likely” or the “best guess”
duration, the variety of answers
removes any chance of getting a
reasonable answer.
The meaning of “best” is
undefined in almost any situation
that has not taken explicit steps
to bound the answers.
Without calibrating the meaning
of “best” the planner cannot
bound the underlying probability
distribution of all the value that
are not “best” but could possibly
occur in the project
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
42/186
When we use a term “best” or
“most likely” there is an implicit
assumption – often not
acknowledged – that other values
than “best” and “most likely” can
occur.
This is the probabilistic nature of
the duration estimate. A single
value cannot exist.
The actual shape of the
probability distributions is what is
needed for generating the “best”
estimate.
Without this knowledge, the
planner is guessing in the dark.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
43/186
Here are some steps to
producing “educated guesses.”
This is a model based approach
which depends on the maturity of
the data that is the basis of the
model.
While this is a high level
description, it needs raw data
underneath to make it valid.
Without this data the “guess” is of
little value.
What is missing in most cases is
any historical trends for the IMS
elements.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
44/186
Playing the 20 questions game is
on approach to calibrating the
“guess” for the duration.
This approach will get an answer
to without 10% to 20% in a few
questions.
This is a way to start the
“conversation” about duration
when the participants have
convinced themselves that they
can’t come up with the answer
because there is not enough
information.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
45/186
Another approach is to classify
the fidelity of the information.
This can be done with a 1, 2, 5
approach.
Gathering estimates by asking
for durations is the preferred
approach.
Instead, making a risk adjusted
estimate – duration and
confidence interval provides a
better approach.
This approach neutralizes the
guessing game by asking a risk
question first, then the duration.
The classification of risk provides
the lower and upper bounds of
the task duration. Along with the
underlying probability distribution,
this forms the basis of
probabilistic schedule analysis
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
46/186
In all cases, uncertainty is the
normal mode for information
gathering.
When we ask a CAM or IPT for
an estimate and do not ask for
the risk associated with that
estimate and the confidence
intervals for that number we are
simply increasing the risk to the
program by absorbing unreliable
numbers.
This unacknowledged risk is
always present . By not making it
visible, the program is
mortgaging the future without
budgeting for the cost of paying
off the mortgage.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
47/186
Starting with a good topology for
the IMS is important. Not only
because the programmatic
activities need to be well defined,
but the sensitivity of the risk
analysis depends on a “properly
formed” IMS.
If the logic of the IMS is ill–
formed than the results of the risk
analysis will also be ill–formed.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
48/186
There are several elements of
the probability model for duration.
Not only are the activities from
the IPTs and CAMs important,
but the subcontractors play on
important role.
The data from the subcontractors
includes:
• Durations and the
probabilities
• The internal connectivity of
the activities that produce the
external; “milestones”
conveyed to the prime
contractor.
• The other programmatic risk
factors for the performance of
subcontractor work
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
49/186
Although the formalities of the
probabilistic risk analysis are not
needed for this briefing. Here is
some background on
terminology.
If we are to learn to “speak” in
probabilistic programmatic risk,
these terms should become
familiar.
This is an almost endless topic,
but some understanding of
probability and statistics is
needed.
This of course requires some
effort and patience .
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
50/186
We should not be drawn into the
illusion that the Central Limit
Theorem is operable for the
program.
This is the core assumption of
PERT and CPM based planning.
This requires normally distributed
completion time and
independence between tasks.
Neither can be verified in
practice.
As such the impact of making
these assumptions is “whistling in
the dark.”
The result is that the program is
late before it starts.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
51/186
The Central Limit Theorem can
be useful in many cases. But it
needs to be understood where it
is not useful.
The assumptions of the CLT
applied to the PERT problem
mask even more problems when
naively applied to estimating the
duration of a program.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
52/186
The core of the Central Limit
Theorem of the production of a
Gaussian probability distribution
by assembling a collection of
arbitrary probability distributions.
The primary assumptions that
these distributions are
independent provides the basis
of the CLT.
If the activities represented by
the arbitrary distributions are not
statistically independent – which
is hardly ever the case on a real
project – then the assumptions of
the Central Limit Theorem are
false and the probability
distribution of the program
completion time is no long
Gaussian distributed
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
53/186
What happens is the rollup of the
most likely times of the critical
path activities is biased to an
optimistic location in the
probability distribution of the
project completion distribution.
This is the fundamental reason
PERT is not very useful.
This criticism is only partly true. If
a probabilistic PERT approach is
used or a Bayesian network
approach is used, then the
deterministic issues are
removed.
But it is easier to use a Monte
Carlo simulator since this avoids
gathering all the underlying
probabilistic distribution
information for an initial estimate
of the completion time of the
program
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
54/186
The probability distribution
function describes the frequency
of occurrence of the events in the
underlying statistical process –
say the duration of a task
completion, the roll of a die, or
the time it takes a light bulb to
burn out.
The ordinate of the graph (the y
axis) is normalize to a scale of [0,
1] which represents the
probability percentage 0.10 =
10%
The abscissa represents the
range of values that can be found
in the underlying sample
population. In this case [0.0, …,
5.0]
The mode is the “most likely”
value to occur when samples are
drawn from the distribution.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
55/186
The standard deviation is a
description of the “spread” of the
probability distribution function
around the mean.
Without understanding the
standard deviation ,a point
estimate or even a sampled
estimate is of little value.
The shape of the probability
distribution is also important in
understanding the confidence in
a single number. These “higher
order moments” will be discussed
later, but for now no estimated
number should be used without
the standard deviation value
being attached.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
56/186
Looking at the population
statistics of a random process is
not very useful. Humans have a
hard time making any sense from
the graphs.
The Histogram view can show
the frequency of occurrence of
the various values – how often a
specific value occurs in the total
population of value or the
sampled population of values, but
more insight is needed.
The Cumulative Probability
Density is a way to show this.
The CDF shows the probability
that a sampled number drawn
from the population of all possible
numbers
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
57/186
Various Probability Distribution
Functions (pdf) have similar
Cumulative Density Functions
(CDF).
This is important for several
reasons:
• The underlying probability
distribution function has great
influence on how the end
point values are weighted.
This has impact on the PERT
formula
• The cumulative distribution is
the source of random
numbers in Monte Carlo. For
a variety of pdf’s, similar
CDF’s are generated,
neutralizing the differences in
the pdf’s. Monte Carlo
isolates these underlying
differences. This may be good
or bad depending in the need.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005 58/186
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
59/186
Any estimating process must
address the probabilistic
boundaries of the estimate.
Without this, planners and cost
estimators are hopelessly under
or over estimating duration and
associated cost.
The real issue is not over or
under estimate, but not knowing
which one it is or why.
This lack of knowledge about the
underlying statistical process
creates a greater risk.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
60/186
Making decisions in a risk neutral
manner is not advised.
We should always talk about risk
adjusted decisions, risk adjusted
values, and risk adjusted
outcomes.
The difference between
alternatives, uncertainties and
outcomes also needs to be
understood. They are not
interchangeable concepts
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
61/186
Decision making must address
the different types of uncertainty.
Understanding how these
uncertainties impact the decision
is critical to selecting alternatives
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
62/186
The idea that we can produce
“estimates” about the future in
the absence of models, historical
data, or a methodology for
discovering these models or
historical data is common in the
IMS planning realm.
Forecasting the future is sporty
business.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
63/186
The IMS contains “branches”
where the path of work makes a
change in direction.
These braches can be modeled
with a decision tree paradigm.
The risk management discipline
uses this approach. And it is
applicable to the construction of
the probabilistic branching found
in the network of tasks in the
IMS.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
64/186
In the “olde days” the line of
balance chart was used to
forecast the cost at completion.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
65/186
PERT and the Critical Path
Method are called out as explicit
methods to be used in the
planning process.
The formulas for PERT are
simplified models of the
underlying complexity of
probabilistic networks (Bayesian
Networks)
As such they have little or no
connection to the reality of the
IMS
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
66/186
In deterministic PERT the
durations are defined as a three
point estimate and the PERT
formula is used to compute the
mean and standard deviation for
the program duration as well as
the critical path.
This is the algorithm used in
Microsoft Project when the PERT
tool bar is turned on and the
three point estimates entered into
the appropriate columns.
It is billed as probabilistic but in
fact the 3–point estimates work
against a fixed probability
distribution function with no way
to adjust its shape, bounds or
moments.
As well, there is no way to insert
the correlations that naturally
occur in the IMS.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
67/186
The estimates produced by the
deterministic three point data can
be used to construct a
probabilistic PERT if the
underlying probability
distributions are defined for each
task completion time.
The development of the
probability distributions requires
historical data as well as an
understanding the behavior of
each node in the network
(coupling).
This is a difficult task without the
proper tools and data sets.
With the Risk+ tool, individual
distribution functions can be
assigned to each task. But the
“tuning” of each function is
difficult.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
68/186
When we speak of probabilistic
risk analysis, we also need to
speak of the statistical nature of
the activity network.
When we speak of a probabilistic
activity network (a Bayesian
network) we also need to speak
in terms of probability.
A question that can be asked of
the network is – “ what is the
probability of completing this task
by a certain date?”
A second question that can be
asked is – “what are the
underlying statistics of the
activities of the network?”
A final question that needs to be
asked is “what is the inherent
uncertainty in these estimates?”
In other words – how good is our
ability to guess in the presence of
a statistical process?
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
69/186
Once the activity durations are
treated as probability distributions
it can be seen that they can not
be “added” in the normal sense
to produce a program duration.
They must be “summed” in the
probabilistic sense. This can be
done with Monte Carlo or with
convolution of the Cumulative
Distribution Functions.
Again, add to this the correlation
issues (one task influencing the
outcome of another task), and
the simple approach of adding
the durations to come up with a
total duration falls apart.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
70/186
Here’s another look at distribution
functions.
This approach should be the
standard vocabulary for
discussing the IMS duration
estimates.
The topological integrity of the
IMS is important, but just as
important is our understanding of
how the activity durations have
been developed, their confidence
interval and the underlying
distribution of the values the
durations can take.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
71/186
A missing element is the
statistics of the “events” that
occur during the execution of the
program.
For example if a fixed date is
defined in the IMS (this is very
usual for things like IBR, PDR,
CDR), what is the underlying
probability distribution of the
confidence of that date.
The same is true for
subcontractor provided dates,
where the details of the
deliverables is not visible.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
72/186
With the input probability
distributions, the program
schedule can be treated as a
“system” with a response
function.
The “system” is a Bayesian
network where the elements of
the network are probabilistic and
the driving function is
probabilistic.
The “output” of the system is
therefore probabilistic as well.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
73/186
One probability distribution
commonly found in scheduling is
the Beta Functions.
This is a “tunable” probability
distribution function that has
been shown to closely match the
behavior of task completion
durations.
The term “closely” needs to be
used with care. The deviations
between actual completion times
and the “model” of completion
times needs to be assessed
before confidence in the
probabilistic results can be
useful.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
74/186
The Beta distribution is used for
PERT estimates. This use is
many times done with no
understanding of the shape or
the dynamics of the probability
distribution function. Beta is a
selection for Risk+ as well, with
no obvious way to change the
shape of the curve.
Some understanding of the
impact of the Beta function on
the outcome of the PERT formula
or the Monte Carlo simulation is
needed.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
75/186
There are many alternatives to
Beta. The Triangle distribution is
one. The triangle distribution has
an intuitive appeal due to its
simplicity useful for estimating
task durations.
But the triangle distribution still
has the problem that the most
likely value and the expected
(mean) value are not the same.
So when planning asks for the
“most likely” value many people
respond with the Mean, which
biases to result in the optimistic
direction.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
76/186
The triangle distribution can
better describe some statistical
processes, but it too needs
“tuning” for specific task duration
processes.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
77/186
BetaPERT is currently the vogue
in the probabilistic analysis world.
The BetaPERT distribution
provides a “tunable” curve where
the most likely “Mode” is near or
identical to the “mean” of the
distribution.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005 78/186
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
79/186
The challenge to building a risk
tolerant IMS is the initial capture
of the task durations and the
sensitivity of the IMS to
correlations between tasks.
There is a optimism bias created
when a CAM is asked “what is
the duration?”
The answer is usually a “mean”
(average) duration rather than
the “Mode” (most likely).
If the Mean is used in place of
the Mode, then the three point
estimates are biased to start with
without the explicit knowledge of
the planning staff.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
80/186
The general flow of creating a
risk tolerant IMS looks like this.
The critical aspect is to get the
CAMs to identify the embedded
risks and the mitigation tasks for
those risks.
Once this is done, “planning” can
then assess if the mitigation
processes make sense in terms
of supporting the AC’s and SA’s
of the IMP/IMS.
Constant and continuous
feedback is needed for this to
work properly.
Without this feedback, the IMS is
assembled in the absence of the
knowledge base and the risk
tolerant aspects are lost or
become confused with the
mainline activities.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
81/186
The capturing of the risk
information is an interactive
process. A Kaizen is one way to
do this and probably the best.
Having the CAM fill out the “most
likely” durations and identify the
risk mitigations cannot be done
without direct contact.
Without this direct contact,
planners have not chance of
intervening in the process and
the IMS becomes a collection of
tasks rather than an “architected”
plan.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
82/186
The 3 point estimates required by
DiD–81650 have a variety of
uses.
They can be simple values used
for PERT calculations. These
calculations can be “made up” by
the IPT lead and entered into the
schedule.
A risk adjusted value can be
used from the Macro in Risk+.
The CAM or IPT Lead states the
relative risk in a number between
1 and 5. The macro defines the
percentage boundaries for the
classified risk.
Individual risk ranges can be
developed from historical
information. This is the best
approach, since it represents the
past.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
83/186
There are several classes of
programmatic risk. Although the
Pareto chart shows that scope
change is the most common,
delays are also common. These
come from the customer side
most often as well.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
84/186
The classification of risk results
in a percentile or quartile
classification scheme. This is a
better approach than asking
someone for the minimum and
maximum durations.
The challenge is to calibrate
these ranges in a meaningful
manner for the specific program.
There can be general
classification ranges, but having
them set for the specific program
is much better.
This of course requires that data
is kept from past programs,
normalized and then made
available in a form useful for
probabilistic risk analysis.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
85/186
During the data capturing
process where estimates are
extracted from the technical
experts, there is a natural
tendency to accept the numbers
at face value.
Without qualifying the numbers in
some statistical form, this
information is absorbed into the
IMS or Cost and becomes “fact.”
These “facts” then progress
through the program and are
never challenged for their lack of
statistical basis.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
86/186
The core problem with capturing
estimate from human beings is
they are biased.
Either negatively biased or
positively biased.
There is plenty of literature on
this effect and ways to overcome
it. For now we’ll just live with the
outcome of the bias
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
87/186
Let’s take another tour of the
problems with PERT. These
issues are well documented in
the literature, but poorly
understood in practice.
The poor understanding comes
from the difficulty of the
explanation – statistical
conversations are usually not
very interesting; and the natural
tendency to look for easy
answers to complex problems.
The core issue is that without a
deep understanding of the errors
produced by the PERT equation,
the confidence in completion
dates and the risk tolerance of
the IMS is difficult to build.
When the actual numbers come
in (ACWP and BCWP) and they
don’t match the expectations – is
it the original plan or the
underlying performance?
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
88/186
There are several myths about
PERT. The first is that is was
scientifically thought out in detail.
This is not the case. The book
The Management of Projects,
Peter W. G. Morris provides the
background on this development
as well as other project
management histories.
The second historical myth is that
PERT is a general purpose
approach. In fact it is very
specialized and is applicable to a
narrow range of activity
networks. Those with normally
distributed completion times,
statistically independent
relationships, ones where the
critical path does not change and
with the “most likely” estimate
actually representing the “mode”
of the underlying probability
distribution function.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
89/186
When a manger asks “what is the
critical path for this program?”
there are several thoughts and
actions:
• In a probabilistic activity
network there are many
critical paths, which change
as a function of time,
adjustments to the risk
profiles, and the completion of
work.
• Correlated activities are
influenced by off–critical–path
activities to place them on the
critical path.
So the answer to the
management question is “it
depends on what you mean by
critical and path.”
The real answer only comes by
moving away from the static
representations of the IMS to a
probabilistic representation – and
that requires much more effort.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
90/186
Once we recognize that the
activity network is probabilistic in
nature, the first choice (the naïve
choice) is to apply the PERT
method.
While this may be a useful “first”
choice it produces results that
are overly optimistic and
sometimes overly pessimistic.
Either way they are wrong from a
statistical point of view. They are
wrong because the assumptions
of PERT are wrong. These
assumptions are almost never
found to be true in practice.
Even if they were true, the
probability distribution function
used by PERT does not
represent any useful activity
completion time distribution.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
91/186
One of the “killer” assumptions in
PERT is the lack of
understanding of “merge bias.”
Merge Bias occurs when two or
more activities are joined at a
merge point. Usually a milestone
or a simple Finish to Start of
several tasks.
The result is the statistical
behavior of the activities prior to
this merge point influence the
statistics of the following
activities in undesirable ways.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
92/186
Since statistical distributions can
not be simply “added” the
duration of the downstream
activity is not the sum of the
duration of all the upstream
activities (or the longest activity).
Instead it is the statistical sum
(convolution) of the probability
distribution function (pdf)
Without understanding this, the
PERT estimate generates an
optimistic estimate of the
duration, since the PERT formula
simply adds the durations to
arrive at the total duration.
The PERT formula also adds the
individual activity variances to
arrive at a total project variance.
While this provided a simple
method to “guess” the total
duration it produces a poor model
for real analysis of risk.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
93/186
The PERT approach fails to
consider the “random variable”
nature of the dates in activity
network.
As well the correlation between
each of these random variables
is not considered.
The result is the potential for
large variances in the completion
time estimates – 15% is not
uncommon.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
94/186
The visual impacts of Merge
Point bias is show here. This is a
small and sample activity
network. A “real” network would
have different outcomes.
It is not important exactly how the
merge point bias impacts the
final completion date, but that the
merge point bias DOES impact
the final completion date.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
95/186
How the activity network is
arranged has significant impact
on the calculations for PERT.
Here are some examples.
Notice that the PERT mean (the
average) stays the same, while
the “real” mean and the variance
on that mean change
dramatically depending on the
arrangement.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
96/186
The reason for these changes
involves how the statistics are
“added” in the various
configurations.
The critical concept is that the
PERT calculations are unreliable
as a predictor of the completion
time in a probabilistic model of
the activities.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
97/186
The effect of the merge bias is
shown in the graph. It is unlikely
in any real plan that only three
parallel paths exists. This number
is usually much larger,
sometimes in the dozens.
All of this discussion is leading to
the suggestion that PERT is not
viable on any complex program.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
98/186
Use Monte Carlo, don’t use
PERT.
The problem of course is that
DID 81650 and even the
corporate guidelines either
require or strongly suggest the
use of PERT and CPM.
This can be done of course, but
don’t use the numbers for any
real planning processes.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
99/186
The use of Monte Carlo
simulation is a logical outcome of
the problems with PERT.
What is missing is the
understanding of how Monte
Carlo works, what it’s limitations
are, where it should not be used
and of course how to interpret
the outcomes when they don’t
meet our expectations.
Even though Monte Carlo is a
powerful tool it can produce
unexpected results. This section
is an attempt to give some
background on the mathematics
and stimulate further interest in
applying this tool to the problem
of schedule forecasting
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
100/186
Monte Carlo simulations provide
a useful approach to modeling
schedule risk. But their value is
more than that.
Unlike PERT or other
deterministic approaches – even
though the three point estimates
are billed as probabilistic – Monte
Carlo examines the schedule
network independent of a critical
path, topological constraints or
other “human induced” problems.
It looks at the network as a
collection of nodes and arcs,
independent of the “meaning” of
this information and produces a
model of the behavior of these
nodes and arcs
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
101/186
The concept behind Monte Carlo
is to sample the possible
durations for a task from the
population of all durations and
apply them to the schedule.
The population of possible
samples is defined by the
Cumulative Probability Density
(CDF) function for each task.
This in turn is defined by the 3–
point estimate for the task, which
selects the bounds in the CDF for
sampling.
Since there is no direct concept
of a Critical Path in Monte Carlo,
the near critical path tasks are
considered in the analysis of the
completion time.
As well the PERT biases
produced by the simple minded
PERT formula are avoided as
well.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
102/186
There are several “components”
to the Monte Carlo process. So
when we speak of Monte Carlo it
is both a process and a product –
in our current case Risk+
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
103/186
Samples drawn from the
underlying distribution function
can produce an “error estimate”
on a completion date.
These error estimates are
different than the fixed
boundaries for PERT, since they
represent the actual probabilities
distribution error bounds
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
104/186
The number of sample runs
needs to be sufficient to cover all
the possibilities in the pdf.
This is usually 500.
A production run for a Monte
Carlo simulation is around 2,000
to 3,000 iterations.
As the iteration count increases
the fidelity of the simulation
increases.
But there is a point where more
samples don’t add value. This
point can be determined by the
statistical performance of the
variance of these sample space.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
105/186
Since Monte Carlo does not need
to know about the Critical Path, it
is conceptually simpler to use.
A well formed network is needed
and the 3–point estimates need
to represent the proper risk
assessment.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
106/186
This is a view of how Risk+ sets
up the project file.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
107/186
The result is a cumulative
distribution and a probability
distribution function.
Interpreting this result is straight
forward.
The confidence of each date is
shown in the table on the right.
This is the probability of
completing the task by the date.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
108/186
A good IMS is needed.
The risk assessments should be
done with a ranking process
rather than specific 3–point
estimates. This disconnects the
personal opinions from the
assignment of risk.
A 5 level process is one
approach, but any odd numbered
level ranking is best.
The differences between the
levels should be geometric not
linear.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
109/186
Risk+ generates lots of
information useful for the
analysis of the program.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005 110/186
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
111/186
Constructing a robust IMS means
building a “risk tolerant” plan.
The robustness of the plan
means that it (the plan) can deal
with disruptions that occur
naturally through the course of
execution or un–naturally through
external events.
In either case the “robustness” of
the plan must be visible to the
evaluator without any detailed
explanation, beyond the IMS
narrative. No hand waving
explanations of how the plan
works. The risk tolerant aspects
most be obvious.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
112/186
Thinking about schedule
contingency is different in a PRA
context. For a simple project,
15% contingency is assumed.
But placing the contingency is
the first problem. The process is:
• Run Risk+ and watch the final
date.
• Compare the 80% confidence
date against the deterministic
date. This difference is the
first cut at the needed margin.
• Assign this duration across
the project in front of the
critical (high risk) milestones.
• Rerun Risk+ and add or
subtract this margin until the
desired confidence date is
achieved.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
113/186
More detailed statistics and
interpretations of the results an
be produced with Risk+.
This information can them be
used to perform further analysis
of the IMS. The analysis is what
we’re after, not just the date
produced by Risk+.
Like the PERT numbers, the
Risk+ numbers must be
interpreted with the
understanding of how they were
arrived at.
This is one of the purposed of
this briefing – to provide
knowledge of how to use this
approach and what its strengths
and weaknesses are.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
114/186
Incorporating Technical
Performance Measures (TPM)
with Monte Carlo is a powerful
way of showing how risk is
reduced and maturity increased
in a program.
At each step in the program –
each Program Event – a target
confidence interval for a
completion date can be forecast.
Along with the technical
performance measure, this
programmatic performance
measure approach results in a
“risk tolerant” IMS.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
115/186
Risk tolerance in the IMS
requires more than just the
planning processes. It requires
the connections to technical and
cost.
This has been stated before, but
it needs to be made not only
visible but actionable.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
116/186
Using a simple process steps,
risk tolerance can be developed
from the same processes used
by the technical risk engineers.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
117/186
The goal here is to move the
integrated risk tolerance –
technical, schedule, cost –
forward from a dis–integrated
plan to an integrated plan
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
118/186
Read the chart as follows: The upper
horizontal band on the plot is “Ready
Early”. “Ready On–time” is the middle
band that also spans the launch
window. “Ready late” is the lower band,
which means a 6–month slip to the next
launch window and all associated costs
that go with that slip. The upper line
plotted is the deterministic
completion date (i.e. no risk) and the
lower line plotted with the 20th and 80th
percentile confidence bands on the risk–
adjusted completion date. The project’s
objective is to continue to invest in risk
mitigation actions until the band and the
area of highest likelihood is no longer in
the “Missed Launch Period” area of the
chart. Note the improving trend over
time indicating the success of the risk
mitigation actions as well some
“Accepted” risks passing their exposure
window without becoming problems.
Taken from [Risk Based Decision
Support techniques for Programs and
Projects]
http://www.futron.com/pdf/RBDSsupport
tech.pdf
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
119/186
As Program Events progress the
risk mitigation processes need to
progress as well.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
120/186
Here’s a 4 step progress for
installing risk in the IMS and
producing a “risk tolerant” plan
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
121/186
The use of branching
probabilities is important for the
assessment of the “risk
tolerance.”
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
122/186
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
123/186
The use of Risk+ and Monte
Carlo replaces the PERT
approach to schedule duration
probability analysis.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
124/186
The “goodness” of the IMS is
important to the quality of the
results
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
125/186
The distribution to use for a task
depends on the underlying risk
profile.
Triangle is common, but it over
biased the risk on the high end.
Beta can be used, but the simple
Beta distributions in Risk+ may
not represent the real risk profile.
BetaPERT is the better one, but
Risk+ does not support it.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
126/186
Which tasks drive the sensitivity
of a completion date needs to be
understood. Not all tasks have
the same impact on the outcome.
The “tornado” chart is one way of
showing this.
The Power Law’s behind Pareto’s
rule is worth understanding for
many reasons, not just schedule
and cost modeling. Power Laws
occur across a wide variety of
domains, from moon crater sizes
to the frequency of words in
English.
http://www.nslij–
genetics.org/wli/zipf/ is a good
place to look for the impacts of
Power Laws on everyday life.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
127/186
In order to build a model of the
schedule we have to start with
the schedule. But first we have to
start with the model of the
schedule.
This is the role of the IMP, but the
connections between the ACs
are needed, not just the list of the
IMP elements.
From this model the schedule
elements can be arranged to
follow the strategy of the IMP
rather than represent the
passage of time and the
consumption of resources.
From there a model of the risk
areas, mitigations, parallel
development paths, reevaluation
points, and “hot spots” (sensitivity
analysis) can be extracted. This
information can them be used to
assess the robustness of the IMS
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
128/186
The primary graphic for an IMS
evaluation is the cumulative
probability of a completion time.
This is technically referred to as
the Cumulative Density Function
(CDF)
This is the format most useful for
answering the question – how
long will this take?
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
129/186
The confidence intervals
produced by the CDF can be
assessed over time against
targets.
These targets can be Technical
Performance Measures or any
other style of metric that is
connected with cost, schedule
and technical performance
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
130/186
Another view is the confidence in
the schedule dates as a function
of time.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
131/186
It is important to understand the
sensitivity of a completion time to
the various “drivers” of this
sensitivity.
This makes visible the “hot spots”
in the IMS that require attention,
mitigation, or even re–planning to
reduce sensitivity
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
132/186
In Monte Carlo each task can
take on a wide variety of roles. It
can be the driver for the total
schedule duration at one time,
and at another time (in the
simulation) have little effect on
the outcome.
The Criticality of the task is how
“important” it is as a function of
the number of simulation runs.
The higher the criticality of the
task, the more important it is to
look at the details and determine
what mitigations should take
place to keep this task lower in
the criticality index.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
133/186
When sensitivity and criticality
are combined a sense of the
cruciality. Cruciality is defined as
“a state of critical urgency.”
Although this sounds like a
redundancy term, it can be used
to focus our attention on those
tasks that are both critical and
sensitive.
It is important to understand the
sensitivity aspects, since these
can change and drive the
schedule in non–obvious ways.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005 134/186
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
135/186
Let’s look at some examples of
Monte Carlo
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
136/186
The Monte Carlo simulation
makes use of the three point
estimates generated during the
PERT analysis. This numbers
represent the upper, lower and
most likely durations.
This values are then used to
draw random numbers from the
probability distribution for
evaluating the activity network.
The branching probabilities can
then be added for the
alternatives paths and risk
mitigation activities.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
137/186
The use of “expert judgment”
itself needs to be calibrated.
The unanswered question on this
program and many others is
“what does a good risk tolerant
IMS actually look like?”
The “units of measure” for risk
tolerance and the confidence in
the probabilistic estimates needs
to be established before the
estimating and modeling process
can be “calibrated”
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
138/186
The ranking of risk or the ranking
of anything needs to be done in a
structured manner.
A geometric progression is a very
useful approach, since it forces
the focus on ranking.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
139/186
The “sense” of risk and real risk
need to be connected.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
140/186
Some type of risk ranking needs
to be developed for the IMS
tasks.
One approach is the TRL scale.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
141/186
When tasks are arranged in
series the cumulative probability
of completion is show in the table
on the right
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
142/186
When the tasks are arranged in
parallel a different completion
profile results.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
143/186
All of this is very interesting in a
Power Point presentation –
marketecture it's called.
Let’s look at a real schedule and
start to apply some of the things
we’ve learned.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
144/186
This is a very simple construction
plan. The tasks are networked in
a way to show how the Risk+ tool
works.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
145/186
The first picture of a completion
time is the PERT assessment.
The task Construction Schedule
Margin (the end of this task is the
end of the planned margin) has a
target date of 2/8/06 and a
forecast PERT date of 3/6/06.
This shows there is not enough
margin by one month for this task
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
146/186
The same task, evaluated with
Risk+ shows a different
completion date.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
147/186
Project Risk Analysis is part of
any good risk management
activity. This has been said
numerous times and needs to he
repeated daily.
Both the technical and the
programmatic risk aspects of the
program need to be shown in the
IMS.
Any questions, changes,
updates, suggestions – anything
that touches the IMS or the cost
model – needs to be assessed
from the point of view of
programmatic risk.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
148/186
The accuracy of the dates and
costs in the IMS is a “relative”
term.
±20% to start with is pretty good.
As the program proceeds
accuracy improves but it is
always a statistical estimate until
after the fact.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
149/186
If we take a deterministic
approach the planning then there
will be built in issues. The first is
that all estimates must include a
confidence interval or they are
wrong.
The “natural” approach to
estimating almost always results
in a bound that is too wide as
well as being optimistic or
pessimistic but hardly never
accurate.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005 150/186
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
151/186
Now that we’ve reached a fairly
detailed level of discussion
regarding programmatic risk
assessment, it’s time to talk
about cost risk assessment.
The first concept to understand is
that cost and schedule are
connected. This is obvious. But
they are not connected in any
linear manner.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
152/186
The basic principles of cost
estimating start with the
understanding of the uncertainty
in the estimates of cost.
These uncertainties must be
connected to the technical
uncertainties as well as the
programmatic and simple cost
variances.
The arithmetic addition of costs
creates a false number of the not
only the cost but any variance in
this cost.
Monte Carlo simulation is one
starting point, but like the
programmatic simulations, the
underlying probability
distributions must be understood
before the numbers have any
real meaning
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
153/186
The connection between a
technical parameter is its cost is
not only potentially non–linear it
is probabilistically non-linear.
Knowing these Cost Element
Relationships (CER) is a critical
success factor for managing in
the presence of uncertainty for
both reducible and irreducible
risks that are created
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
154/186
A simple 9 step (not so simple
actually) process can be used to
build a cost estimate.
Starting with the “likely” program
in the form of an IMS, the tasks
for delivering that program are
defined.
The underlying probability
distributions for the cost of each
delivering activity are developed.
This is much like the
development of the baseline IMS,
but the next step is much
different.
The correlation between each
WBS element is developed.
These correlations are used to
build a model of sensitivity of the
cost to changes in the tasks.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
155/186
At this point the Risk+ tool fails to
deliver what is needed. Wither
Crystal Ball or @RISK is needed
to connect these correlations
together.
The technical uncertainty of the
program is used to drive the cost
uncertainty. This is where the
technical and programmatic risk
assessments joins.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
156/186
The production of the familiar
probability curves for the
likelihood of cost is the result.
This curve tells is the likelihood
that some event will occur.
For example the probability of a
cost in this example.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
157/186
The risk margin in dollars is the
result needed to make this
connection.
Without this margin, the budget
for the program ‒ as a single
point estimate ‒ is not credible.
All programs operate in the
presence of uncertainty, so no
single point estimate can be
correct.
For reducible uncertainties,
specific actions can be take to
protect the outcome.
For irreducible uncertainties, only
margin can protect the outcomes
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
158/186
Without the cost margin, the
result is the risk that the budget
is not sufficient to complete the
program without a cost overrun.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
159/186
Management reserve,
contingency and other budget
protection is then needed.
Much much reserve and
contingency comes from
modeling the program and
determining the level of
confidence needed to protect the
budget from overrun.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
160/186
With models of both the cost
uncertainties (reducible and
irreducible) and schedule
uncertainties (reducible and
irreducible) a Risk Adjusted
Integrated Master Schedule can
be created.
This IMS has Risk Handling
activities for the Reducible Risks
and Margin for the Irreducible
Risks
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
161/186
With the Risk Handling content in
place, a new assessment of the
probabilistic outcomes can be
shown.
Before risk management and
after risk management.
This is no actionable information
to the decision makers.
Was the risk reduced enough to
start the program, continue the
program? Or are further risk
reduction activities needed?
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
162/186
The cumulative cost curve – in
this example, can then be used
to make decisions using the
value at risk
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
163/186
All programs operate in the
presence of uncertainty.
Making decisions in the presence
of this uncertainty can take place
with the proper information.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005 164/186
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
165/186
We’re near the end now, so your
brain is certainly getting full.
This is quite a bit of information
to absorb, but it needs to be
done before we can say we are
building “risk tolerant” plans.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
166/186
When probabilistic schedule
analysis is used it does not
replace the need for a well
formed project network. It only
replaces the use of PERT for
estimating the completion dates.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
167/186
The quality of the probabilistic
estimates is the foundation of
confidence.
The next step is to clearly identify
where in the IMS risks are being
mitigated, the impacts of this
mitigation and the overall
confidence in the master plan
resulting from this mitigation
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
168/186
The correlations between cost,
schedule, and technical risk must
be made explicit.
A model of how these elements
interact is the basis for answering
the “what if” questions that occur
when the risk item becomes
active.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
169/186
Risk based schedule and cost
management is core to
programmatic integrity.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005 170/186
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
171/186
Like any good idea it can be
improved on forever.
These opportunities are much
harder to address than the
process so far. They require care
and effort to build a correlation
matrix for the tasks. They require
detailed understanding of the
underlying statistical processes
and the historical data that was
used to develop these
distributions.
For most projects this is beyond
the scope of the effort and may
be beyond the business interests
as well – since the pay back is
not clearly defined.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
172/186
The dependencies between tasks
is the basis of the correlation
function. This is very important if
a true model of the network is to
be developed. In the absence of
the correlations it is assumed
tasks are independent, which of
course can not be the case.
Building a Program risk
assessment requires that cost
and schedule be connected as
well – correlated. Cost and
schedule are not linear, so any
simple model of changes in one
linearly effecting the other cannot
work.
Finally the idea of a causal model
– a cause and a set of effects
provides deeper insight into the
risk behaviors of the network.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
173/186
There is too much information
here for a single digestive
process. The only way to absorb
all this is to start practicing
probabilistic schedule and cost
analysis and make the
knowledge appear in the output
information.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
174/186
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
175/186
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
176/186
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
177/186
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
178/186
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
179/186
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
180/186
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
181/186
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
182/186
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
183/186
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
184/186
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
185/186
There are nearly unlimited
resources on the web. The
challenge of course is finding
them.
Here’s some know starting
points.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005
Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19
186/186
This has been a long journey
over hopefully many weeks of
discussion and hands on
experience with Risk+ and real
project schedules.
Building a risk tolerant IMS is a
“practice” and practices require
proficiency. Proficiency comes
from “doing the work,” looking at
the results and making changes
for improvement.
This is just the beginning.
Prepared for NNJ05111915R, by Glen B. Alleman
December 2005

More Related Content

What's hot

Narrated Version Dallas MPUG
Narrated Version Dallas MPUGNarrated Version Dallas MPUG
Narrated Version Dallas MPUG
Glen Alleman
 
Risk adjusted engineering management
Risk adjusted engineering managementRisk adjusted engineering management
Risk adjusted engineering management
Glen Alleman
 
EVM+Agile the darkside
EVM+Agile the darksideEVM+Agile the darkside
EVM+Agile the darkside
Glen Alleman
 
Process Flow and Narrative for Agile
Process Flow and Narrative for AgileProcess Flow and Narrative for Agile
Process Flow and Narrative for Agile
Glen Alleman
 
Making Agile Development work in Government Contracting
Making Agile Development work in Government ContractingMaking Agile Development work in Government Contracting
Making Agile Development work in Government Contracting
Glen Alleman
 
Probabilistic Schedule and Cost Analysis
Probabilistic Schedule and Cost AnalysisProbabilistic Schedule and Cost Analysis
Probabilistic Schedule and Cost Analysis
Glen Alleman
 
Capabilities based planning
Capabilities based planningCapabilities based planning
Capabilities based planning
Glen Alleman
 
WBS is Paramount
WBS is ParamountWBS is Paramount
WBS is Paramount
Glen Alleman
 
Earned Value Management and Agile
Earned Value Management and AgileEarned Value Management and Agile
Earned Value Management and Agile
Glen Alleman
 
Brownfields agile draft v11
Brownfields agile draft v11Brownfields agile draft v11
Brownfields agile draft v11
tony1234
 
Risk Management Guidance
Risk Management GuidanceRisk Management Guidance
Risk Management Guidance
Glen Alleman
 
Focus on the nine I's (v9)
Focus on the nine I's (v9)Focus on the nine I's (v9)
Focus on the nine I's (v9)
Glen Alleman
 
Immutable principles of project management (utah pmi)(v1)(no exercise)
Immutable principles of project management (utah pmi)(v1)(no exercise)Immutable principles of project management (utah pmi)(v1)(no exercise)
Immutable principles of project management (utah pmi)(v1)(no exercise)
Glen Alleman
 
Strategic portfolio management
Strategic portfolio managementStrategic portfolio management
Strategic portfolio management
Glen Alleman
 
Increasing the Probability of Project Success with Five Principles and Practices
Increasing the Probability of Project Success with Five Principles and PracticesIncreasing the Probability of Project Success with Five Principles and Practices
Increasing the Probability of Project Success with Five Principles and Practices
Glen Alleman
 
Performance based management in a nut shell (v5)
Performance based management in a nut shell (v5)Performance based management in a nut shell (v5)
Performance based management in a nut shell (v5)
Glen Alleman
 
The 5 Immutable principles of project management
The 5 Immutable principles of project managementThe 5 Immutable principles of project management
The 5 Immutable principles of project management
Glen Alleman
 
Cure for cost and schedule growth
Cure for cost and schedule growthCure for cost and schedule growth
Cure for cost and schedule growth
Glen Alleman
 
Establishing schedule margin using monte carlo simulation
Establishing schedule margin using monte carlo simulation Establishing schedule margin using monte carlo simulation
Establishing schedule margin using monte carlo simulation
Glen Alleman
 
Increasing the probability of program success
Increasing the probability of program successIncreasing the probability of program success
Increasing the probability of program success
Glen Alleman
 

What's hot (20)

Narrated Version Dallas MPUG
Narrated Version Dallas MPUGNarrated Version Dallas MPUG
Narrated Version Dallas MPUG
 
Risk adjusted engineering management
Risk adjusted engineering managementRisk adjusted engineering management
Risk adjusted engineering management
 
EVM+Agile the darkside
EVM+Agile the darksideEVM+Agile the darkside
EVM+Agile the darkside
 
Process Flow and Narrative for Agile
Process Flow and Narrative for AgileProcess Flow and Narrative for Agile
Process Flow and Narrative for Agile
 
Making Agile Development work in Government Contracting
Making Agile Development work in Government ContractingMaking Agile Development work in Government Contracting
Making Agile Development work in Government Contracting
 
Probabilistic Schedule and Cost Analysis
Probabilistic Schedule and Cost AnalysisProbabilistic Schedule and Cost Analysis
Probabilistic Schedule and Cost Analysis
 
Capabilities based planning
Capabilities based planningCapabilities based planning
Capabilities based planning
 
WBS is Paramount
WBS is ParamountWBS is Paramount
WBS is Paramount
 
Earned Value Management and Agile
Earned Value Management and AgileEarned Value Management and Agile
Earned Value Management and Agile
 
Brownfields agile draft v11
Brownfields agile draft v11Brownfields agile draft v11
Brownfields agile draft v11
 
Risk Management Guidance
Risk Management GuidanceRisk Management Guidance
Risk Management Guidance
 
Focus on the nine I's (v9)
Focus on the nine I's (v9)Focus on the nine I's (v9)
Focus on the nine I's (v9)
 
Immutable principles of project management (utah pmi)(v1)(no exercise)
Immutable principles of project management (utah pmi)(v1)(no exercise)Immutable principles of project management (utah pmi)(v1)(no exercise)
Immutable principles of project management (utah pmi)(v1)(no exercise)
 
Strategic portfolio management
Strategic portfolio managementStrategic portfolio management
Strategic portfolio management
 
Increasing the Probability of Project Success with Five Principles and Practices
Increasing the Probability of Project Success with Five Principles and PracticesIncreasing the Probability of Project Success with Five Principles and Practices
Increasing the Probability of Project Success with Five Principles and Practices
 
Performance based management in a nut shell (v5)
Performance based management in a nut shell (v5)Performance based management in a nut shell (v5)
Performance based management in a nut shell (v5)
 
The 5 Immutable principles of project management
The 5 Immutable principles of project managementThe 5 Immutable principles of project management
The 5 Immutable principles of project management
 
Cure for cost and schedule growth
Cure for cost and schedule growthCure for cost and schedule growth
Cure for cost and schedule growth
 
Establishing schedule margin using monte carlo simulation
Establishing schedule margin using monte carlo simulation Establishing schedule margin using monte carlo simulation
Establishing schedule margin using monte carlo simulation
 
Increasing the probability of program success
Increasing the probability of program successIncreasing the probability of program success
Increasing the probability of program success
 

Similar to Probabilistic Schedule and Cost Analysis

Probabilistic Cost, Schedule, and Risk management
Probabilistic Cost, Schedule, and Risk managementProbabilistic Cost, Schedule, and Risk management
Probabilistic Cost, Schedule, and Risk management
Glen Alleman
 
Continuous Risk Management
Continuous Risk ManagementContinuous Risk Management
Continuous Risk Management
Glen Alleman
 
How Traditional Risk Reporting Has Let Us Down
How Traditional Risk Reporting Has Let Us DownHow Traditional Risk Reporting Has Let Us Down
How Traditional Risk Reporting Has Let Us Down
Acumen
 
Building Risk Tolerance into the Program Plan and Schedule
Building Risk Tolerance into the Program Plan and ScheduleBuilding Risk Tolerance into the Program Plan and Schedule
Building Risk Tolerance into the Program Plan and Schedule
Glen Alleman
 
Information Technology Risk Management
Information Technology Risk ManagementInformation Technology Risk Management
Information Technology Risk Management
Glen Alleman
 
IRJET- Analysis of Risk Management in Construction Sector using Fault Tree...
IRJET- 	  Analysis of Risk Management in Construction Sector using Fault Tree...IRJET- 	  Analysis of Risk Management in Construction Sector using Fault Tree...
IRJET- Analysis of Risk Management in Construction Sector using Fault Tree...
IRJET Journal
 
Programmatic risk management workshop (slides)
Programmatic risk management workshop (slides)Programmatic risk management workshop (slides)
Programmatic risk management workshop (slides)
Glen Alleman
 
IT Risk managment combined
IT Risk managment combinedIT Risk managment combined
IT Risk managment combined
Glen Alleman
 
Earning Value from Earned Value
Earning Value from Earned ValueEarning Value from Earned Value
Earning Value from Earned Value
Glen Alleman
 
Measurement News Webinar
Measurement News WebinarMeasurement News Webinar
Measurement News Webinar
Glen Alleman
 
Cost Risk Analysis (CRA) by Pedram Daneshmand 19-Jan-2011
Cost Risk Analysis (CRA) by Pedram Daneshmand 19-Jan-2011Cost Risk Analysis (CRA) by Pedram Daneshmand 19-Jan-2011
Cost Risk Analysis (CRA) by Pedram Daneshmand 19-Jan-2011
Pedram Danesh-Mand
 
Risk management of the performance measurement baseline
Risk management of the performance measurement baselineRisk management of the performance measurement baseline
Risk management of the performance measurement baseline
Glen Alleman
 
Risk management (final review)
Risk management (final review)Risk management (final review)
Risk management (final review)
Glen Alleman
 
Managing cost and schedule risk
Managing cost and schedule riskManaging cost and schedule risk
Managing cost and schedule risk
Glen Alleman
 
Building a Credible Performance Measurement Baseline (PM Journal)
Building a Credible Performance Measurement Baseline (PM Journal)Building a Credible Performance Measurement Baseline (PM Journal)
Building a Credible Performance Measurement Baseline (PM Journal)
Glen Alleman
 
PERIL LIBRARY
PERIL LIBRARYPERIL LIBRARY
PERIL LIBRARY
Soban Ahmad
 
Risk Management in Five Easy Pieces
Risk Management in Five Easy PiecesRisk Management in Five Easy Pieces
Risk Management in Five Easy Pieces
Glen Alleman
 
Primavera Monte Carlo[1]
Primavera Monte Carlo[1]Primavera Monte Carlo[1]
Primavera Monte Carlo[1]
Mihai Buta
 
IRJET- Projects in Constructions due to Inadequate Risk Management
IRJET-  	  Projects in Constructions due to Inadequate Risk ManagementIRJET-  	  Projects in Constructions due to Inadequate Risk Management
IRJET- Projects in Constructions due to Inadequate Risk Management
IRJET Journal
 
Risk Organization for ERP Projects
Risk Organization for ERP ProjectsRisk Organization for ERP Projects

Similar to Probabilistic Schedule and Cost Analysis (20)

Probabilistic Cost, Schedule, and Risk management
Probabilistic Cost, Schedule, and Risk managementProbabilistic Cost, Schedule, and Risk management
Probabilistic Cost, Schedule, and Risk management
 
Continuous Risk Management
Continuous Risk ManagementContinuous Risk Management
Continuous Risk Management
 
How Traditional Risk Reporting Has Let Us Down
How Traditional Risk Reporting Has Let Us DownHow Traditional Risk Reporting Has Let Us Down
How Traditional Risk Reporting Has Let Us Down
 
Building Risk Tolerance into the Program Plan and Schedule
Building Risk Tolerance into the Program Plan and ScheduleBuilding Risk Tolerance into the Program Plan and Schedule
Building Risk Tolerance into the Program Plan and Schedule
 
Information Technology Risk Management
Information Technology Risk ManagementInformation Technology Risk Management
Information Technology Risk Management
 
IRJET- Analysis of Risk Management in Construction Sector using Fault Tree...
IRJET- 	  Analysis of Risk Management in Construction Sector using Fault Tree...IRJET- 	  Analysis of Risk Management in Construction Sector using Fault Tree...
IRJET- Analysis of Risk Management in Construction Sector using Fault Tree...
 
Programmatic risk management workshop (slides)
Programmatic risk management workshop (slides)Programmatic risk management workshop (slides)
Programmatic risk management workshop (slides)
 
IT Risk managment combined
IT Risk managment combinedIT Risk managment combined
IT Risk managment combined
 
Earning Value from Earned Value
Earning Value from Earned ValueEarning Value from Earned Value
Earning Value from Earned Value
 
Measurement News Webinar
Measurement News WebinarMeasurement News Webinar
Measurement News Webinar
 
Cost Risk Analysis (CRA) by Pedram Daneshmand 19-Jan-2011
Cost Risk Analysis (CRA) by Pedram Daneshmand 19-Jan-2011Cost Risk Analysis (CRA) by Pedram Daneshmand 19-Jan-2011
Cost Risk Analysis (CRA) by Pedram Daneshmand 19-Jan-2011
 
Risk management of the performance measurement baseline
Risk management of the performance measurement baselineRisk management of the performance measurement baseline
Risk management of the performance measurement baseline
 
Risk management (final review)
Risk management (final review)Risk management (final review)
Risk management (final review)
 
Managing cost and schedule risk
Managing cost and schedule riskManaging cost and schedule risk
Managing cost and schedule risk
 
Building a Credible Performance Measurement Baseline (PM Journal)
Building a Credible Performance Measurement Baseline (PM Journal)Building a Credible Performance Measurement Baseline (PM Journal)
Building a Credible Performance Measurement Baseline (PM Journal)
 
PERIL LIBRARY
PERIL LIBRARYPERIL LIBRARY
PERIL LIBRARY
 
Risk Management in Five Easy Pieces
Risk Management in Five Easy PiecesRisk Management in Five Easy Pieces
Risk Management in Five Easy Pieces
 
Primavera Monte Carlo[1]
Primavera Monte Carlo[1]Primavera Monte Carlo[1]
Primavera Monte Carlo[1]
 
IRJET- Projects in Constructions due to Inadequate Risk Management
IRJET-  	  Projects in Constructions due to Inadequate Risk ManagementIRJET-  	  Projects in Constructions due to Inadequate Risk Management
IRJET- Projects in Constructions due to Inadequate Risk Management
 
Risk Organization for ERP Projects
Risk Organization for ERP ProjectsRisk Organization for ERP Projects
Risk Organization for ERP Projects
 

More from Glen Alleman

Managing risk with deliverables planning
Managing risk with deliverables planningManaging risk with deliverables planning
Managing risk with deliverables planning
Glen Alleman
 
Increasing the Probability of Project Success
Increasing the Probability of Project SuccessIncreasing the Probability of Project Success
Increasing the Probability of Project Success
Glen Alleman
 
Process Flow and Narrative for Agile+PPM
Process Flow and Narrative for Agile+PPMProcess Flow and Narrative for Agile+PPM
Process Flow and Narrative for Agile+PPM
Glen Alleman
 
Practices of risk management
Practices of risk managementPractices of risk management
Practices of risk management
Glen Alleman
 
Principles of Risk Management
Principles of Risk ManagementPrinciples of Risk Management
Principles of Risk Management
Glen Alleman
 
Deliverables Based Planning, PMBOK® and 5 Immutable Principles of Project Suc...
Deliverables Based Planning, PMBOK® and 5 Immutable Principles of Project Suc...Deliverables Based Planning, PMBOK® and 5 Immutable Principles of Project Suc...
Deliverables Based Planning, PMBOK® and 5 Immutable Principles of Project Suc...
Glen Alleman
 
From Principles to Strategies for Systems Engineering
From Principles to Strategies for Systems EngineeringFrom Principles to Strategies for Systems Engineering
From Principles to Strategies for Systems Engineering
Glen Alleman
 
NAVAIR Integrated Master Schedule Guide guide
NAVAIR Integrated Master Schedule Guide guideNAVAIR Integrated Master Schedule Guide guide
NAVAIR Integrated Master Schedule Guide guide
Glen Alleman
 
Building a Credible Performance Measurement Baseline
Building a Credible Performance Measurement BaselineBuilding a Credible Performance Measurement Baseline
Building a Credible Performance Measurement Baseline
Glen Alleman
 
IMP / IMS Step by Step
IMP / IMS Step by StepIMP / IMS Step by Step
IMP / IMS Step by Step
Glen Alleman
 
DHS - Using functions points to estimate agile development programs (v2)
DHS - Using functions points to estimate agile development programs (v2)DHS - Using functions points to estimate agile development programs (v2)
DHS - Using functions points to estimate agile development programs (v2)
Glen Alleman
 
Making the impossible possible
Making the impossible possibleMaking the impossible possible
Making the impossible possible
Glen Alleman
 
Building the Performance Measurement Baseline
Building the Performance Measurement BaselineBuilding the Performance Measurement Baseline
Building the Performance Measurement Baseline
Glen Alleman
 
Program Management Office Lean Software Development and Six Sigma
Program Management Office Lean Software Development and Six SigmaProgram Management Office Lean Software Development and Six Sigma
Program Management Office Lean Software Development and Six Sigma
Glen Alleman
 
Policy and Procedure Rollout
Policy and Procedure RolloutPolicy and Procedure Rollout
Policy and Procedure Rollout
Glen Alleman
 
Integrated Master Plan Development
Integrated Master Plan DevelopmentIntegrated Master Plan Development
Integrated Master Plan Development
Glen Alleman
 
Project Management Theory
Project Management TheoryProject Management Theory
Project Management Theory
Glen Alleman
 
Seven Habits of a Highly Effective agile project manager
Seven Habits of a Highly Effective agile project managerSeven Habits of a Highly Effective agile project manager
Seven Habits of a Highly Effective agile project manager
Glen Alleman
 
Paradigm of agile project management (update)
Paradigm of agile project management (update)Paradigm of agile project management (update)
Paradigm of agile project management (update)
Glen Alleman
 
Deliverables based planning handbook
Deliverables based planning handbookDeliverables based planning handbook
Deliverables based planning handbook
Glen Alleman
 

More from Glen Alleman (20)

Managing risk with deliverables planning
Managing risk with deliverables planningManaging risk with deliverables planning
Managing risk with deliverables planning
 
Increasing the Probability of Project Success
Increasing the Probability of Project SuccessIncreasing the Probability of Project Success
Increasing the Probability of Project Success
 
Process Flow and Narrative for Agile+PPM
Process Flow and Narrative for Agile+PPMProcess Flow and Narrative for Agile+PPM
Process Flow and Narrative for Agile+PPM
 
Practices of risk management
Practices of risk managementPractices of risk management
Practices of risk management
 
Principles of Risk Management
Principles of Risk ManagementPrinciples of Risk Management
Principles of Risk Management
 
Deliverables Based Planning, PMBOK® and 5 Immutable Principles of Project Suc...
Deliverables Based Planning, PMBOK® and 5 Immutable Principles of Project Suc...Deliverables Based Planning, PMBOK® and 5 Immutable Principles of Project Suc...
Deliverables Based Planning, PMBOK® and 5 Immutable Principles of Project Suc...
 
From Principles to Strategies for Systems Engineering
From Principles to Strategies for Systems EngineeringFrom Principles to Strategies for Systems Engineering
From Principles to Strategies for Systems Engineering
 
NAVAIR Integrated Master Schedule Guide guide
NAVAIR Integrated Master Schedule Guide guideNAVAIR Integrated Master Schedule Guide guide
NAVAIR Integrated Master Schedule Guide guide
 
Building a Credible Performance Measurement Baseline
Building a Credible Performance Measurement BaselineBuilding a Credible Performance Measurement Baseline
Building a Credible Performance Measurement Baseline
 
IMP / IMS Step by Step
IMP / IMS Step by StepIMP / IMS Step by Step
IMP / IMS Step by Step
 
DHS - Using functions points to estimate agile development programs (v2)
DHS - Using functions points to estimate agile development programs (v2)DHS - Using functions points to estimate agile development programs (v2)
DHS - Using functions points to estimate agile development programs (v2)
 
Making the impossible possible
Making the impossible possibleMaking the impossible possible
Making the impossible possible
 
Building the Performance Measurement Baseline
Building the Performance Measurement BaselineBuilding the Performance Measurement Baseline
Building the Performance Measurement Baseline
 
Program Management Office Lean Software Development and Six Sigma
Program Management Office Lean Software Development and Six SigmaProgram Management Office Lean Software Development and Six Sigma
Program Management Office Lean Software Development and Six Sigma
 
Policy and Procedure Rollout
Policy and Procedure RolloutPolicy and Procedure Rollout
Policy and Procedure Rollout
 
Integrated Master Plan Development
Integrated Master Plan DevelopmentIntegrated Master Plan Development
Integrated Master Plan Development
 
Project Management Theory
Project Management TheoryProject Management Theory
Project Management Theory
 
Seven Habits of a Highly Effective agile project manager
Seven Habits of a Highly Effective agile project managerSeven Habits of a Highly Effective agile project manager
Seven Habits of a Highly Effective agile project manager
 
Paradigm of agile project management (update)
Paradigm of agile project management (update)Paradigm of agile project management (update)
Paradigm of agile project management (update)
 
Deliverables based planning handbook
Deliverables based planning handbookDeliverables based planning handbook
Deliverables based planning handbook
 

Recently uploaded

Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
Hiroshi SHIBATA
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
Jakub Marek
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
Jason Yip
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
ssuserfac0301
 
High performance Serverless Java on AWS- GoTo Amsterdam 2024
High performance Serverless Java on AWS- GoTo Amsterdam 2024High performance Serverless Java on AWS- GoTo Amsterdam 2024
High performance Serverless Java on AWS- GoTo Amsterdam 2024
Vadym Kazulkin
 
Christine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptxChristine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptx
christinelarrosa
 
Must Know Postgres Extension for DBA and Developer during Migration
Must Know Postgres Extension for DBA and Developer during MigrationMust Know Postgres Extension for DBA and Developer during Migration
Must Know Postgres Extension for DBA and Developer during Migration
Mydbops
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 
Christine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptxChristine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptx
christinelarrosa
 
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and BioinformaticiansBiomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Neo4j
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
Alex Pruden
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
Jason Packer
 
Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |
AstuteBusiness
 
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillinQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
LizaNolte
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Tosin Akinosho
 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
Pablo Gómez Abajo
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
Zilliz
 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
Ivo Velitchkov
 
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptxPRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
christinelarrosa
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
akankshawande
 

Recently uploaded (20)

Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
 
Taking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdfTaking AI to the Next Level in Manufacturing.pdf
Taking AI to the Next Level in Manufacturing.pdf
 
High performance Serverless Java on AWS- GoTo Amsterdam 2024
High performance Serverless Java on AWS- GoTo Amsterdam 2024High performance Serverless Java on AWS- GoTo Amsterdam 2024
High performance Serverless Java on AWS- GoTo Amsterdam 2024
 
Christine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptxChristine's Product Research Presentation.pptx
Christine's Product Research Presentation.pptx
 
Must Know Postgres Extension for DBA and Developer during Migration
Must Know Postgres Extension for DBA and Developer during MigrationMust Know Postgres Extension for DBA and Developer during Migration
Must Know Postgres Extension for DBA and Developer during Migration
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 
Christine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptxChristine's Supplier Sourcing Presentaion.pptx
Christine's Supplier Sourcing Presentaion.pptx
 
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and BioinformaticiansBiomedical Knowledge Graphs for Data Scientists and Bioinformaticians
Biomedical Knowledge Graphs for Data Scientists and Bioinformaticians
 
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...
 
Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
 
Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |
 
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham HillinQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
inQuba Webinar Mastering Customer Journey Management with Dr Graham Hill
 
Monitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdfMonitoring and Managing Anomaly Detection on OpenShift.pdf
Monitoring and Managing Anomaly Detection on OpenShift.pdf
 
Mutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented ChatbotsMutation Testing for Task-Oriented Chatbots
Mutation Testing for Task-Oriented Chatbots
 
Fueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte WebinarFueling AI with Great Data with Airbyte Webinar
Fueling AI with Great Data with Airbyte Webinar
 
Apps Break Data
Apps Break DataApps Break Data
Apps Break Data
 
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptxPRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
PRODUCT LISTING OPTIMIZATION PRESENTATION.pptx
 
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development ProvidersYour One-Stop Shop for Python Success: Top 10 US Python Development Providers
Your One-Stop Shop for Python Success: Top 10 US Python Development Providers
 

Probabilistic Schedule and Cost Analysis

  • 1. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 1/186 This briefing is an overview of the probabilistic risk analysis processes that can be applied to our program. Although it may not appear to be a “simple” overview, this material is the tip of the iceberg of this complex topic. Just schedule analysis has been addressed in detail here. The cost aspects of forecasting and simulation must be addressed as well to complete the connections between schedule and cost. Probabilistic cost will be surveyed here, but an in depth review is for a later time. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 2. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 2/186 An important aspect of education and research in our business domain, is “Fair Use” copyright law. All the material in this briefing is accessible through the internet. Conference proceedings journal articles, company white papers and other public sources form the basis of much of this material and are referenced in the bibliography. Some materials in this briefing make references to other copyrighted materials in the course of research, investigation, and analysis. These references are solely intended for non– commercial use and as such have no intent to infringe on the copyright holder. All attempts have been made to acknowledge the original copyright holder in pursuit of fair use laws as currently defined in the United States. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 3. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 3/186 The concept that risk and the management of risk is a desirable part of our program is not always appreciated or well understood Without risk there can be no opportunities. The plans for the program become static and deterministic. While risk and opportunity are related, the management of risk is not the complement of opportunity. - even if this is a popular notions these days. See the Conrow, AT&L article for detailed discussion of this somewhat controversial topic. The primary opportunity in Programmatic Risk Management is the avoidance of being late and over budget on the planned launch date. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 4. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 4/186 When we use the term “risk tolerant IMS” it means a plan and supporting that can tolerate risks. Technical risks and programmatic risks. These risks are built into the program by its very nature. These risks must be addressed both technically and programmatically. The real challenge though is not how to address them, but how to recognize that they are being addressed in a manner that actually reduces the level of risk as the program proceeds along its path to final maturity. A measure of “increasing maturity” is the reduction of risk made visible to the evaluator of the IMS. The materials here guide us through the process of building a risk tolerant IMS. But putting it to work still requires practice. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 5. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 5/186 The credibility of the Integrated Master Schedule (IMS) is the critical success factor for both our proposal and our execution phase after the win. Without a credible schedule and the related cost credibility, there is a low probability of a win. The effort put into constructing a credible schedule during the proposal phase will pay off (assuming the program structure remains intact) during the execution phase. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 6. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 6/186 The skills of creating and managing a schedule and the associated cost require special understanding. However, the planners are usually the last in a long line of “culprits” for finding the cause of any failure. This is a “no win” situation. People skills, project management skills, and some level of technical skill is needed. But most important is the people skill, since the knowledge of how to assemble a successful IMS resides in the minds of others. Getting this knowledge out and on paper requires interpersonal communication as a primary process, not technical tools and formal processes. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 7. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 7/186 Understanding the difference between qualitative and quantitative risk assessment is important. Our first approach is usually qualitative. But what is needed is quantitative. A specific measure of programmatic risk, is the impact of the mitigations or risk retirement activities and measure of the increasing maturity of the program deliverables in the presence of risk. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 8. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 8/186 Programmatic risk management makes visible the technical risk mitigation steps as well as the alternative programmatic processes in the presence of these risks. Alternative branching in the IMS must be defined to a level of detail that instills confidence that the IMS properly represents a “risk tolerant” plan. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 9. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 9/186 Since there is quite a bit of material here, a quick overview will get us started. The executive overview should leave the reader with a sense of the important topics • There are no point estimates allowed in planning. All estimates must be probabilistic • There are core issues with simple (deterministic) PERT and it is not to be trusted • The use of a probabilistic tool is useful, but understanding how the underlying statistic works is critical to its use in planning and program execution Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 10. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 10/186 When asked “why are we doing this?” many would answer – because our customer wants us to. This would be too simple an answer. The main reason is, most programs are simply too complex not to have a better understanding of how the programmatic and technical risks interact. Not understanding the interaction between these two types of risk that creates the biggest risk. Individually these risk “could” be managed. But when combined they behave in unpredictable and maybe unknowable ways. This is a core feature of any system. See Systems Bible Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 11. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 11/186 If we get only two concepts out of this briefing they should be: • There are multiple critical paths in any executing program. Asking “what is the critical” indicates that the questioner does not understand the probabilistic nature of the program • PERT is a poor estimating metric. It has built in biases which under estimate the total duration of the program. Monte Carlo is a better estimating tool, but it too needs careful adjustment before realistic numbers can be derived. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 12. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 12/186 The DID–MGMT–81650 describes the Integrated Master Schedule. Integrating Programmatic and Technical risk identification and mitigation adds credibility to the IMS and therefore to the overall program. Applying probabilistic risk analysis to the IMS is mandated, but care is needed to interpret the results. These tools aid in the evaluation, but they are not replacements for good program management processes. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 13. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 13/186 The idea that uncertainty and the risk that it produces can be “programmed out” of the schedule is a false hope. Without understanding the principles of Deming, the management and the planning staff will be “chasing their tail,” trying to control the naturally occurring variances in the plan. The first approach is to set the error bands wide enough to not trigger an exception report for these variances. This approach is “good enough” but what is missing is the knowledge of “how wide is wide enough?” for a specific set of tasks or during a specific phase of a program? Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 14. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 14/186 The first step in the process of adding credibility to the IMS is to recognize that all task completion times are random variables. They are not “point” numbers (scalars) but are “estimates” of the completion time drawn from a probability distribution of the underlying population of all completion times possible for the specific task. Modeling schedule durations are random variables does not imply these durations are “random.” It reflects how a duration’s uncertainty is influenced by the underlying probabilistic nature of the activity network. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 15. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 15/186 Building a credible IMS starts with identifying the architecture of the IMP and the supporting tasks in the IMS. Although this is restating the obvious the process to do this is actually quite hard. Adding schedule and cost risk identification and mitigation to the process is the minimal result for a winning proposal. It cannot be emphasized enough – the architecture of the IMS is critical to identifying a risk tolerant schedule. The “rats nest” approach is simply unacceptable to the success of any program. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 16. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 16/186 The goal of introducing probabilistic schedule and cost analysis is to improve the probability of a “win” on the proposal. While winning is important, executing the program is even more important. What ever “credibility” elements were in the proposed IMS need to be carried into the execution schedule. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 17. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 17/186 The use of Monte Carlo for assessing the IMS must be turned into forecasting performance. This is done by identifying the “hot spots” in the IMS through sensitivity analysis, interventions for these “hot spots” and the measure of change resulting from the intervention. The important concept is to connect metrics to measurable benefits to the program. Without this the creation of metrics is just wasted effort. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 18. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 Prepared for NNJ05111915R, by Glen B. Alleman December 2005 18/186
  • 19. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 19/186 Using risk and uncertainty as an integral part of the planning process is a sign of maturity. Making decisions on the this risk information improves maturity. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 20. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 20/186 When we speak of risk management, either technical or programmatic, the term usually has a very localized context. For the planning context risk management must include both technical and programmatic risk. The technical risk aspects come from external sources but are directly represented in the IMS. The programmatic impacts of this technical risk must be explicitly addressed. This is the easy part. The hard part is determining the implicit programmatic risk that is derived from the technical risk and the risks that are derived from the “architecture” of the program itself. This is where the true “risk tolerant” IMS adds value. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 21. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 21/186 There are many approaches to building a risk tolerant IMS. Our current approach is to add risk factors and margin to specific areas of the IMS The current approach to use a Monte Carlo tool to assess where this margin should be placed. There are several other steps along the way. Which steps to take, how much effort to invest and how to recognize the value of this investment are some of the management challenges as well as the technical challenges. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 22. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 22/186 The difference between risk and uncertainty needs to be understood at some level. For the most part the differences are not important in the beginning. But once decisions start to be made about mitigation steps, branching probabilities for failure modes, these differences become more important. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 23. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 23/186 When we use the term uncertainty or risk it means at least 4 things. First let’s sort out “uncertainty” There are two classes of uncertainty in large complex programs. • Static uncertainty emerges from the natural variations in the completion times of tasks. This is a Deming uncertainty. http://webserver.lemoyne.edu/ ~wright/deming.htm is an example of this type of uncertainty • The dynamic uncertainty is about the unknowns and the unknowable Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 24. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 24/186 The static uncertainty in a program can be addressed directly in the plan with mitigation tasks. The dynamic uncertainty arises from the dynamic interactions between the tasks of the plan. This interaction and the outcomes to the end date cannot be modeled with static paradigms. Monte Carlo simulation is an approach to modeling these interactions and their impact on other elements of the plan Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 25. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 25/186 Managing risk in the schedule requires anticipation to identify the risks, but also requires understanding of the source of risk, the impacts of these risks, and the interaction between the risks and the plan. A process is needed to guide the risk management activities. This process must address both the programmatic as well as technical risk. The interaction between programmatic and technical risks must also be managed. These interactions must be considered a “first order” interaction. The common approach is to consider the technical risk as first order and the programmatic risks secondary. The combination becomes a first order interaction. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 26. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 26/186 As planners our goal must be to produce a plan that has credibility and integrity. Credible plans are believable plans Integrity plans are trustworthy plans. Both attributes are needed for a winning proposal and the follow on execution. The successful assessment of the IMS during a proposal or during execution by the customer or DCMA depends on how believable the plan is and how well it can be assessed to confirm this believability Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 27. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 27/186 The assessment of the credibility and integrity of the IMS can take place by asking some questions. These and similar questions shine light on the underlying attributes of the IMS in ways that simple assessments do not. These are not technical assessment, like counting data in the predecessors field, but are architectural questions about the “quality” of the IMS independent of the technical details. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 28. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 28/186 NASA does risk management in a specific way. We need to understand their way as a starting point. Reading the NASA materials is a start, but there is other research available from conferences and vendor web sites that needs to be gathered and read as well. Other government agencies as well as civilian firms have similar risk management approaches. NASA’s approach is a good starting because of manned space flight’s inherent risk. And NASA’s emphasis on Safety and Mission Assurance. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 29. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 29/186 The IRMA tool developed at NASA Johnson Space Center is the basis of risk management for a NASA side. Although this approach is focused on the technical risks the programmatic risks appear in the database. As well there are other risk management systems and paradigms. Active Risk Manager (ARM) is a popular one as well, http://www.strategicthought.com/ Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 30. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 30/186 The NASA Risk Management Summary Card calls out “schedule” impacts in three places. Connecting programmatic and technical risk is a critical success factor for a proposal as well as an execution assessment. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 31. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 31/186 Adding probabilistic schedule and risk analysis to the IMS can be done through a structured process. 1. The initiating event of the risk is identified. 2. The result from this event is described 3. The consequence that flow from the scenario are developed 4. The connections, flows, interactions and correlations between the scenarios are modeled 5. The probability of occurrence for each of these scenarios is developed 6. The model of the probability of occurrence and consequences from the occurrence are combined Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 32. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 32/186 The Continuous Risk Management paradigm found in the technical risk world can be applied to the programmatic risk as well. NASA has adopted Continuous Risk Management (CRM) through several guidelines listed here. The table summarizes how CRM is managed in a structured manner throughout the program life, Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 33. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 33/186 There is a difference between the design evaluation of the IMS and the risk evaluation. The design evaluation describes how the technical activities needed to develop and deploy the product – in this case a manned spacecraft – must come together in the right sequence to make the planned completion date. The risk evaluation defines the probabilistic completion model for each task, the correlations between the tasks and the resulting probabilistic model. This model is a Bayesian Network of all the tasks. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 34. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 34/186 To construct an IMS with integrity and credibility both technical and programmatic risk must be connected. This process starts with the identification of the technical risks in ARM and their mitigations in the IMS. This is the explicit risk approach. Next comes the explicit programmatic risk activities. This can be the well known margin needed in front of major milestones, program events or deliverables. Finally comes the implicit risk mitigation activities that will be needed to differentiate this IMS from any other IMS to start to build confidence that we have a “risk tolerant” IMS. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 35. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 35/186 A pedagogical literature survey from the RAND Corporation supports the notion that probabilistic risk assessment is not seen in a favorable light by management. • It is too complex. • The underlying statistic are not will understood. • “It’s the customers that are asking for this.” • There is little historical data to calibrate the underlying probability distribution functions for task completion times. All of these gaps must be closed in some way in order to call our IMS Risk Tolerant Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 36. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 Prepared for NNJ05111915R, by Glen B. Alleman December 2005 36/186
  • 37. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 37/186 Managing in the presence of uncertainty is the core behavior for any modern program. Trying to control this uncertainty requires two basic understandings: 1. The natural variations in the schedule cannot be sufficiently controlled to remove risk. These are the Deming variations and the foreseen uncertainties 2. The unforeseen uncertainties and the inherent chaos of the program must be dealt with through contingencies Attempting to manage uncertainty is limited to foreseen risk. Managing in the presence of uncertainty deals with unforeseen and chaotic sources of risk Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 38. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 38/186 When estimating the completion times for tasks, there are three primary problems. 1. A number produced by a CAM or an IPT must be a statistical estimate, not a specific duration. 2. The meaning of “best” must be established prior to accepting the statistical estimate 3. The collecting of the “most likely” estimates cannot be added in the sense of adding scalar numbers, since they are probability distributions. 4. The “most likely” is NOT the average completion time, it is the completion time that occurs most often from a large sample of possible completion times. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 39. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 39/186 The first approach to “planning” the program is to ask the CAMs or IPT Leads for each task in their WBS or IMP/IMS area: “how long with this take to do?” The numbers that come back are then entered in the duration field on the schedule. These numbers are not only wrong they are dangerously wrong. They are “point” estimates that live inside a probability distribution. The built in bias from the approach has clinically be shown to be optimistic or pessimistic, but rarely “most likely.” Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 40. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 40/186 The traditional approach is to roll up the single point estimates into a sum of the durations and search for the longest path. This is the Critical Path Method (CPM) for assessing the finish date of the plan. The problem of course is these “numbers” are not actual scalar values. They are samples drawn from probability distributions. Addition is not mathematically possible in the sense of addition, defined over the set of natural numbers (0, 1, 2, … ∞] These probability distributions can be “convolved” into a new probability distribution, but a better approach is Monte Carlo Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 41. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 41/186 When asked “what is the most likely” or the “best guess” duration, the variety of answers removes any chance of getting a reasonable answer. The meaning of “best” is undefined in almost any situation that has not taken explicit steps to bound the answers. Without calibrating the meaning of “best” the planner cannot bound the underlying probability distribution of all the value that are not “best” but could possibly occur in the project Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 42. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 42/186 When we use a term “best” or “most likely” there is an implicit assumption – often not acknowledged – that other values than “best” and “most likely” can occur. This is the probabilistic nature of the duration estimate. A single value cannot exist. The actual shape of the probability distributions is what is needed for generating the “best” estimate. Without this knowledge, the planner is guessing in the dark. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 43. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 43/186 Here are some steps to producing “educated guesses.” This is a model based approach which depends on the maturity of the data that is the basis of the model. While this is a high level description, it needs raw data underneath to make it valid. Without this data the “guess” is of little value. What is missing in most cases is any historical trends for the IMS elements. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 44. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 44/186 Playing the 20 questions game is on approach to calibrating the “guess” for the duration. This approach will get an answer to without 10% to 20% in a few questions. This is a way to start the “conversation” about duration when the participants have convinced themselves that they can’t come up with the answer because there is not enough information. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 45. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 45/186 Another approach is to classify the fidelity of the information. This can be done with a 1, 2, 5 approach. Gathering estimates by asking for durations is the preferred approach. Instead, making a risk adjusted estimate – duration and confidence interval provides a better approach. This approach neutralizes the guessing game by asking a risk question first, then the duration. The classification of risk provides the lower and upper bounds of the task duration. Along with the underlying probability distribution, this forms the basis of probabilistic schedule analysis Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 46. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 46/186 In all cases, uncertainty is the normal mode for information gathering. When we ask a CAM or IPT for an estimate and do not ask for the risk associated with that estimate and the confidence intervals for that number we are simply increasing the risk to the program by absorbing unreliable numbers. This unacknowledged risk is always present . By not making it visible, the program is mortgaging the future without budgeting for the cost of paying off the mortgage. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 47. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 47/186 Starting with a good topology for the IMS is important. Not only because the programmatic activities need to be well defined, but the sensitivity of the risk analysis depends on a “properly formed” IMS. If the logic of the IMS is ill– formed than the results of the risk analysis will also be ill–formed. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 48. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 48/186 There are several elements of the probability model for duration. Not only are the activities from the IPTs and CAMs important, but the subcontractors play on important role. The data from the subcontractors includes: • Durations and the probabilities • The internal connectivity of the activities that produce the external; “milestones” conveyed to the prime contractor. • The other programmatic risk factors for the performance of subcontractor work Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 49. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 49/186 Although the formalities of the probabilistic risk analysis are not needed for this briefing. Here is some background on terminology. If we are to learn to “speak” in probabilistic programmatic risk, these terms should become familiar. This is an almost endless topic, but some understanding of probability and statistics is needed. This of course requires some effort and patience . Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 50. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 50/186 We should not be drawn into the illusion that the Central Limit Theorem is operable for the program. This is the core assumption of PERT and CPM based planning. This requires normally distributed completion time and independence between tasks. Neither can be verified in practice. As such the impact of making these assumptions is “whistling in the dark.” The result is that the program is late before it starts. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 51. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 51/186 The Central Limit Theorem can be useful in many cases. But it needs to be understood where it is not useful. The assumptions of the CLT applied to the PERT problem mask even more problems when naively applied to estimating the duration of a program. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 52. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 52/186 The core of the Central Limit Theorem of the production of a Gaussian probability distribution by assembling a collection of arbitrary probability distributions. The primary assumptions that these distributions are independent provides the basis of the CLT. If the activities represented by the arbitrary distributions are not statistically independent – which is hardly ever the case on a real project – then the assumptions of the Central Limit Theorem are false and the probability distribution of the program completion time is no long Gaussian distributed Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 53. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 53/186 What happens is the rollup of the most likely times of the critical path activities is biased to an optimistic location in the probability distribution of the project completion distribution. This is the fundamental reason PERT is not very useful. This criticism is only partly true. If a probabilistic PERT approach is used or a Bayesian network approach is used, then the deterministic issues are removed. But it is easier to use a Monte Carlo simulator since this avoids gathering all the underlying probabilistic distribution information for an initial estimate of the completion time of the program Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 54. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 54/186 The probability distribution function describes the frequency of occurrence of the events in the underlying statistical process – say the duration of a task completion, the roll of a die, or the time it takes a light bulb to burn out. The ordinate of the graph (the y axis) is normalize to a scale of [0, 1] which represents the probability percentage 0.10 = 10% The abscissa represents the range of values that can be found in the underlying sample population. In this case [0.0, …, 5.0] The mode is the “most likely” value to occur when samples are drawn from the distribution. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 55. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 55/186 The standard deviation is a description of the “spread” of the probability distribution function around the mean. Without understanding the standard deviation ,a point estimate or even a sampled estimate is of little value. The shape of the probability distribution is also important in understanding the confidence in a single number. These “higher order moments” will be discussed later, but for now no estimated number should be used without the standard deviation value being attached. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 56. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 56/186 Looking at the population statistics of a random process is not very useful. Humans have a hard time making any sense from the graphs. The Histogram view can show the frequency of occurrence of the various values – how often a specific value occurs in the total population of value or the sampled population of values, but more insight is needed. The Cumulative Probability Density is a way to show this. The CDF shows the probability that a sampled number drawn from the population of all possible numbers Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 57. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 57/186 Various Probability Distribution Functions (pdf) have similar Cumulative Density Functions (CDF). This is important for several reasons: • The underlying probability distribution function has great influence on how the end point values are weighted. This has impact on the PERT formula • The cumulative distribution is the source of random numbers in Monte Carlo. For a variety of pdf’s, similar CDF’s are generated, neutralizing the differences in the pdf’s. Monte Carlo isolates these underlying differences. This may be good or bad depending in the need. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 58. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 Prepared for NNJ05111915R, by Glen B. Alleman December 2005 58/186
  • 59. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 59/186 Any estimating process must address the probabilistic boundaries of the estimate. Without this, planners and cost estimators are hopelessly under or over estimating duration and associated cost. The real issue is not over or under estimate, but not knowing which one it is or why. This lack of knowledge about the underlying statistical process creates a greater risk. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 60. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 60/186 Making decisions in a risk neutral manner is not advised. We should always talk about risk adjusted decisions, risk adjusted values, and risk adjusted outcomes. The difference between alternatives, uncertainties and outcomes also needs to be understood. They are not interchangeable concepts Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 61. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 61/186 Decision making must address the different types of uncertainty. Understanding how these uncertainties impact the decision is critical to selecting alternatives Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 62. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 62/186 The idea that we can produce “estimates” about the future in the absence of models, historical data, or a methodology for discovering these models or historical data is common in the IMS planning realm. Forecasting the future is sporty business. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 63. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 63/186 The IMS contains “branches” where the path of work makes a change in direction. These braches can be modeled with a decision tree paradigm. The risk management discipline uses this approach. And it is applicable to the construction of the probabilistic branching found in the network of tasks in the IMS. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 64. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 64/186 In the “olde days” the line of balance chart was used to forecast the cost at completion. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 65. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 65/186 PERT and the Critical Path Method are called out as explicit methods to be used in the planning process. The formulas for PERT are simplified models of the underlying complexity of probabilistic networks (Bayesian Networks) As such they have little or no connection to the reality of the IMS Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 66. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 66/186 In deterministic PERT the durations are defined as a three point estimate and the PERT formula is used to compute the mean and standard deviation for the program duration as well as the critical path. This is the algorithm used in Microsoft Project when the PERT tool bar is turned on and the three point estimates entered into the appropriate columns. It is billed as probabilistic but in fact the 3–point estimates work against a fixed probability distribution function with no way to adjust its shape, bounds or moments. As well, there is no way to insert the correlations that naturally occur in the IMS. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 67. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 67/186 The estimates produced by the deterministic three point data can be used to construct a probabilistic PERT if the underlying probability distributions are defined for each task completion time. The development of the probability distributions requires historical data as well as an understanding the behavior of each node in the network (coupling). This is a difficult task without the proper tools and data sets. With the Risk+ tool, individual distribution functions can be assigned to each task. But the “tuning” of each function is difficult. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 68. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 68/186 When we speak of probabilistic risk analysis, we also need to speak of the statistical nature of the activity network. When we speak of a probabilistic activity network (a Bayesian network) we also need to speak in terms of probability. A question that can be asked of the network is – “ what is the probability of completing this task by a certain date?” A second question that can be asked is – “what are the underlying statistics of the activities of the network?” A final question that needs to be asked is “what is the inherent uncertainty in these estimates?” In other words – how good is our ability to guess in the presence of a statistical process? Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 69. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 69/186 Once the activity durations are treated as probability distributions it can be seen that they can not be “added” in the normal sense to produce a program duration. They must be “summed” in the probabilistic sense. This can be done with Monte Carlo or with convolution of the Cumulative Distribution Functions. Again, add to this the correlation issues (one task influencing the outcome of another task), and the simple approach of adding the durations to come up with a total duration falls apart. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 70. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 70/186 Here’s another look at distribution functions. This approach should be the standard vocabulary for discussing the IMS duration estimates. The topological integrity of the IMS is important, but just as important is our understanding of how the activity durations have been developed, their confidence interval and the underlying distribution of the values the durations can take. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 71. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 71/186 A missing element is the statistics of the “events” that occur during the execution of the program. For example if a fixed date is defined in the IMS (this is very usual for things like IBR, PDR, CDR), what is the underlying probability distribution of the confidence of that date. The same is true for subcontractor provided dates, where the details of the deliverables is not visible. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 72. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 72/186 With the input probability distributions, the program schedule can be treated as a “system” with a response function. The “system” is a Bayesian network where the elements of the network are probabilistic and the driving function is probabilistic. The “output” of the system is therefore probabilistic as well. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 73. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 73/186 One probability distribution commonly found in scheduling is the Beta Functions. This is a “tunable” probability distribution function that has been shown to closely match the behavior of task completion durations. The term “closely” needs to be used with care. The deviations between actual completion times and the “model” of completion times needs to be assessed before confidence in the probabilistic results can be useful. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 74. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 74/186 The Beta distribution is used for PERT estimates. This use is many times done with no understanding of the shape or the dynamics of the probability distribution function. Beta is a selection for Risk+ as well, with no obvious way to change the shape of the curve. Some understanding of the impact of the Beta function on the outcome of the PERT formula or the Monte Carlo simulation is needed. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 75. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 75/186 There are many alternatives to Beta. The Triangle distribution is one. The triangle distribution has an intuitive appeal due to its simplicity useful for estimating task durations. But the triangle distribution still has the problem that the most likely value and the expected (mean) value are not the same. So when planning asks for the “most likely” value many people respond with the Mean, which biases to result in the optimistic direction. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 76. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 76/186 The triangle distribution can better describe some statistical processes, but it too needs “tuning” for specific task duration processes. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 77. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 77/186 BetaPERT is currently the vogue in the probabilistic analysis world. The BetaPERT distribution provides a “tunable” curve where the most likely “Mode” is near or identical to the “mean” of the distribution. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 78. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 Prepared for NNJ05111915R, by Glen B. Alleman December 2005 78/186
  • 79. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 79/186 The challenge to building a risk tolerant IMS is the initial capture of the task durations and the sensitivity of the IMS to correlations between tasks. There is a optimism bias created when a CAM is asked “what is the duration?” The answer is usually a “mean” (average) duration rather than the “Mode” (most likely). If the Mean is used in place of the Mode, then the three point estimates are biased to start with without the explicit knowledge of the planning staff. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 80. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 80/186 The general flow of creating a risk tolerant IMS looks like this. The critical aspect is to get the CAMs to identify the embedded risks and the mitigation tasks for those risks. Once this is done, “planning” can then assess if the mitigation processes make sense in terms of supporting the AC’s and SA’s of the IMP/IMS. Constant and continuous feedback is needed for this to work properly. Without this feedback, the IMS is assembled in the absence of the knowledge base and the risk tolerant aspects are lost or become confused with the mainline activities. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 81. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 81/186 The capturing of the risk information is an interactive process. A Kaizen is one way to do this and probably the best. Having the CAM fill out the “most likely” durations and identify the risk mitigations cannot be done without direct contact. Without this direct contact, planners have not chance of intervening in the process and the IMS becomes a collection of tasks rather than an “architected” plan. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 82. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 82/186 The 3 point estimates required by DiD–81650 have a variety of uses. They can be simple values used for PERT calculations. These calculations can be “made up” by the IPT lead and entered into the schedule. A risk adjusted value can be used from the Macro in Risk+. The CAM or IPT Lead states the relative risk in a number between 1 and 5. The macro defines the percentage boundaries for the classified risk. Individual risk ranges can be developed from historical information. This is the best approach, since it represents the past. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 83. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 83/186 There are several classes of programmatic risk. Although the Pareto chart shows that scope change is the most common, delays are also common. These come from the customer side most often as well. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 84. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 84/186 The classification of risk results in a percentile or quartile classification scheme. This is a better approach than asking someone for the minimum and maximum durations. The challenge is to calibrate these ranges in a meaningful manner for the specific program. There can be general classification ranges, but having them set for the specific program is much better. This of course requires that data is kept from past programs, normalized and then made available in a form useful for probabilistic risk analysis. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 85. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 85/186 During the data capturing process where estimates are extracted from the technical experts, there is a natural tendency to accept the numbers at face value. Without qualifying the numbers in some statistical form, this information is absorbed into the IMS or Cost and becomes “fact.” These “facts” then progress through the program and are never challenged for their lack of statistical basis. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 86. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 86/186 The core problem with capturing estimate from human beings is they are biased. Either negatively biased or positively biased. There is plenty of literature on this effect and ways to overcome it. For now we’ll just live with the outcome of the bias Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 87. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 87/186 Let’s take another tour of the problems with PERT. These issues are well documented in the literature, but poorly understood in practice. The poor understanding comes from the difficulty of the explanation – statistical conversations are usually not very interesting; and the natural tendency to look for easy answers to complex problems. The core issue is that without a deep understanding of the errors produced by the PERT equation, the confidence in completion dates and the risk tolerance of the IMS is difficult to build. When the actual numbers come in (ACWP and BCWP) and they don’t match the expectations – is it the original plan or the underlying performance? Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 88. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 88/186 There are several myths about PERT. The first is that is was scientifically thought out in detail. This is not the case. The book The Management of Projects, Peter W. G. Morris provides the background on this development as well as other project management histories. The second historical myth is that PERT is a general purpose approach. In fact it is very specialized and is applicable to a narrow range of activity networks. Those with normally distributed completion times, statistically independent relationships, ones where the critical path does not change and with the “most likely” estimate actually representing the “mode” of the underlying probability distribution function. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 89. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 89/186 When a manger asks “what is the critical path for this program?” there are several thoughts and actions: • In a probabilistic activity network there are many critical paths, which change as a function of time, adjustments to the risk profiles, and the completion of work. • Correlated activities are influenced by off–critical–path activities to place them on the critical path. So the answer to the management question is “it depends on what you mean by critical and path.” The real answer only comes by moving away from the static representations of the IMS to a probabilistic representation – and that requires much more effort. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 90. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 90/186 Once we recognize that the activity network is probabilistic in nature, the first choice (the naïve choice) is to apply the PERT method. While this may be a useful “first” choice it produces results that are overly optimistic and sometimes overly pessimistic. Either way they are wrong from a statistical point of view. They are wrong because the assumptions of PERT are wrong. These assumptions are almost never found to be true in practice. Even if they were true, the probability distribution function used by PERT does not represent any useful activity completion time distribution. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 91. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 91/186 One of the “killer” assumptions in PERT is the lack of understanding of “merge bias.” Merge Bias occurs when two or more activities are joined at a merge point. Usually a milestone or a simple Finish to Start of several tasks. The result is the statistical behavior of the activities prior to this merge point influence the statistics of the following activities in undesirable ways. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 92. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 92/186 Since statistical distributions can not be simply “added” the duration of the downstream activity is not the sum of the duration of all the upstream activities (or the longest activity). Instead it is the statistical sum (convolution) of the probability distribution function (pdf) Without understanding this, the PERT estimate generates an optimistic estimate of the duration, since the PERT formula simply adds the durations to arrive at the total duration. The PERT formula also adds the individual activity variances to arrive at a total project variance. While this provided a simple method to “guess” the total duration it produces a poor model for real analysis of risk. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 93. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 93/186 The PERT approach fails to consider the “random variable” nature of the dates in activity network. As well the correlation between each of these random variables is not considered. The result is the potential for large variances in the completion time estimates – 15% is not uncommon. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 94. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 94/186 The visual impacts of Merge Point bias is show here. This is a small and sample activity network. A “real” network would have different outcomes. It is not important exactly how the merge point bias impacts the final completion date, but that the merge point bias DOES impact the final completion date. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 95. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 95/186 How the activity network is arranged has significant impact on the calculations for PERT. Here are some examples. Notice that the PERT mean (the average) stays the same, while the “real” mean and the variance on that mean change dramatically depending on the arrangement. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 96. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 96/186 The reason for these changes involves how the statistics are “added” in the various configurations. The critical concept is that the PERT calculations are unreliable as a predictor of the completion time in a probabilistic model of the activities. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 97. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 97/186 The effect of the merge bias is shown in the graph. It is unlikely in any real plan that only three parallel paths exists. This number is usually much larger, sometimes in the dozens. All of this discussion is leading to the suggestion that PERT is not viable on any complex program. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 98. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 98/186 Use Monte Carlo, don’t use PERT. The problem of course is that DID 81650 and even the corporate guidelines either require or strongly suggest the use of PERT and CPM. This can be done of course, but don’t use the numbers for any real planning processes. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 99. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 99/186 The use of Monte Carlo simulation is a logical outcome of the problems with PERT. What is missing is the understanding of how Monte Carlo works, what it’s limitations are, where it should not be used and of course how to interpret the outcomes when they don’t meet our expectations. Even though Monte Carlo is a powerful tool it can produce unexpected results. This section is an attempt to give some background on the mathematics and stimulate further interest in applying this tool to the problem of schedule forecasting Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 100. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 100/186 Monte Carlo simulations provide a useful approach to modeling schedule risk. But their value is more than that. Unlike PERT or other deterministic approaches – even though the three point estimates are billed as probabilistic – Monte Carlo examines the schedule network independent of a critical path, topological constraints or other “human induced” problems. It looks at the network as a collection of nodes and arcs, independent of the “meaning” of this information and produces a model of the behavior of these nodes and arcs Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 101. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 101/186 The concept behind Monte Carlo is to sample the possible durations for a task from the population of all durations and apply them to the schedule. The population of possible samples is defined by the Cumulative Probability Density (CDF) function for each task. This in turn is defined by the 3– point estimate for the task, which selects the bounds in the CDF for sampling. Since there is no direct concept of a Critical Path in Monte Carlo, the near critical path tasks are considered in the analysis of the completion time. As well the PERT biases produced by the simple minded PERT formula are avoided as well. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 102. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 102/186 There are several “components” to the Monte Carlo process. So when we speak of Monte Carlo it is both a process and a product – in our current case Risk+ Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 103. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 103/186 Samples drawn from the underlying distribution function can produce an “error estimate” on a completion date. These error estimates are different than the fixed boundaries for PERT, since they represent the actual probabilities distribution error bounds Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 104. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 104/186 The number of sample runs needs to be sufficient to cover all the possibilities in the pdf. This is usually 500. A production run for a Monte Carlo simulation is around 2,000 to 3,000 iterations. As the iteration count increases the fidelity of the simulation increases. But there is a point where more samples don’t add value. This point can be determined by the statistical performance of the variance of these sample space. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 105. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 105/186 Since Monte Carlo does not need to know about the Critical Path, it is conceptually simpler to use. A well formed network is needed and the 3–point estimates need to represent the proper risk assessment. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 106. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 106/186 This is a view of how Risk+ sets up the project file. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 107. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 107/186 The result is a cumulative distribution and a probability distribution function. Interpreting this result is straight forward. The confidence of each date is shown in the table on the right. This is the probability of completing the task by the date. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 108. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 108/186 A good IMS is needed. The risk assessments should be done with a ranking process rather than specific 3–point estimates. This disconnects the personal opinions from the assignment of risk. A 5 level process is one approach, but any odd numbered level ranking is best. The differences between the levels should be geometric not linear. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 109. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 109/186 Risk+ generates lots of information useful for the analysis of the program. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 110. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 Prepared for NNJ05111915R, by Glen B. Alleman December 2005 110/186
  • 111. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 111/186 Constructing a robust IMS means building a “risk tolerant” plan. The robustness of the plan means that it (the plan) can deal with disruptions that occur naturally through the course of execution or un–naturally through external events. In either case the “robustness” of the plan must be visible to the evaluator without any detailed explanation, beyond the IMS narrative. No hand waving explanations of how the plan works. The risk tolerant aspects most be obvious. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 112. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 112/186 Thinking about schedule contingency is different in a PRA context. For a simple project, 15% contingency is assumed. But placing the contingency is the first problem. The process is: • Run Risk+ and watch the final date. • Compare the 80% confidence date against the deterministic date. This difference is the first cut at the needed margin. • Assign this duration across the project in front of the critical (high risk) milestones. • Rerun Risk+ and add or subtract this margin until the desired confidence date is achieved. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 113. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 113/186 More detailed statistics and interpretations of the results an be produced with Risk+. This information can them be used to perform further analysis of the IMS. The analysis is what we’re after, not just the date produced by Risk+. Like the PERT numbers, the Risk+ numbers must be interpreted with the understanding of how they were arrived at. This is one of the purposed of this briefing – to provide knowledge of how to use this approach and what its strengths and weaknesses are. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 114. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 114/186 Incorporating Technical Performance Measures (TPM) with Monte Carlo is a powerful way of showing how risk is reduced and maturity increased in a program. At each step in the program – each Program Event – a target confidence interval for a completion date can be forecast. Along with the technical performance measure, this programmatic performance measure approach results in a “risk tolerant” IMS. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 115. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 115/186 Risk tolerance in the IMS requires more than just the planning processes. It requires the connections to technical and cost. This has been stated before, but it needs to be made not only visible but actionable. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 116. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 116/186 Using a simple process steps, risk tolerance can be developed from the same processes used by the technical risk engineers. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 117. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 117/186 The goal here is to move the integrated risk tolerance – technical, schedule, cost – forward from a dis–integrated plan to an integrated plan Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 118. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 118/186 Read the chart as follows: The upper horizontal band on the plot is “Ready Early”. “Ready On–time” is the middle band that also spans the launch window. “Ready late” is the lower band, which means a 6–month slip to the next launch window and all associated costs that go with that slip. The upper line plotted is the deterministic completion date (i.e. no risk) and the lower line plotted with the 20th and 80th percentile confidence bands on the risk– adjusted completion date. The project’s objective is to continue to invest in risk mitigation actions until the band and the area of highest likelihood is no longer in the “Missed Launch Period” area of the chart. Note the improving trend over time indicating the success of the risk mitigation actions as well some “Accepted” risks passing their exposure window without becoming problems. Taken from [Risk Based Decision Support techniques for Programs and Projects] http://www.futron.com/pdf/RBDSsupport tech.pdf Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 119. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 119/186 As Program Events progress the risk mitigation processes need to progress as well. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 120. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 120/186 Here’s a 4 step progress for installing risk in the IMS and producing a “risk tolerant” plan Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 121. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 121/186 The use of branching probabilities is important for the assessment of the “risk tolerance.” Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 122. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 122/186 Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 123. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 123/186 The use of Risk+ and Monte Carlo replaces the PERT approach to schedule duration probability analysis. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 124. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 124/186 The “goodness” of the IMS is important to the quality of the results Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 125. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 125/186 The distribution to use for a task depends on the underlying risk profile. Triangle is common, but it over biased the risk on the high end. Beta can be used, but the simple Beta distributions in Risk+ may not represent the real risk profile. BetaPERT is the better one, but Risk+ does not support it. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 126. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 126/186 Which tasks drive the sensitivity of a completion date needs to be understood. Not all tasks have the same impact on the outcome. The “tornado” chart is one way of showing this. The Power Law’s behind Pareto’s rule is worth understanding for many reasons, not just schedule and cost modeling. Power Laws occur across a wide variety of domains, from moon crater sizes to the frequency of words in English. http://www.nslij– genetics.org/wli/zipf/ is a good place to look for the impacts of Power Laws on everyday life. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 127. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 127/186 In order to build a model of the schedule we have to start with the schedule. But first we have to start with the model of the schedule. This is the role of the IMP, but the connections between the ACs are needed, not just the list of the IMP elements. From this model the schedule elements can be arranged to follow the strategy of the IMP rather than represent the passage of time and the consumption of resources. From there a model of the risk areas, mitigations, parallel development paths, reevaluation points, and “hot spots” (sensitivity analysis) can be extracted. This information can them be used to assess the robustness of the IMS Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 128. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 128/186 The primary graphic for an IMS evaluation is the cumulative probability of a completion time. This is technically referred to as the Cumulative Density Function (CDF) This is the format most useful for answering the question – how long will this take? Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 129. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 129/186 The confidence intervals produced by the CDF can be assessed over time against targets. These targets can be Technical Performance Measures or any other style of metric that is connected with cost, schedule and technical performance Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 130. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 130/186 Another view is the confidence in the schedule dates as a function of time. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 131. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 131/186 It is important to understand the sensitivity of a completion time to the various “drivers” of this sensitivity. This makes visible the “hot spots” in the IMS that require attention, mitigation, or even re–planning to reduce sensitivity Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 132. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 132/186 In Monte Carlo each task can take on a wide variety of roles. It can be the driver for the total schedule duration at one time, and at another time (in the simulation) have little effect on the outcome. The Criticality of the task is how “important” it is as a function of the number of simulation runs. The higher the criticality of the task, the more important it is to look at the details and determine what mitigations should take place to keep this task lower in the criticality index. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 133. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 133/186 When sensitivity and criticality are combined a sense of the cruciality. Cruciality is defined as “a state of critical urgency.” Although this sounds like a redundancy term, it can be used to focus our attention on those tasks that are both critical and sensitive. It is important to understand the sensitivity aspects, since these can change and drive the schedule in non–obvious ways. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 134. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 Prepared for NNJ05111915R, by Glen B. Alleman December 2005 134/186
  • 135. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 135/186 Let’s look at some examples of Monte Carlo Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 136. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 136/186 The Monte Carlo simulation makes use of the three point estimates generated during the PERT analysis. This numbers represent the upper, lower and most likely durations. This values are then used to draw random numbers from the probability distribution for evaluating the activity network. The branching probabilities can then be added for the alternatives paths and risk mitigation activities. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 137. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 137/186 The use of “expert judgment” itself needs to be calibrated. The unanswered question on this program and many others is “what does a good risk tolerant IMS actually look like?” The “units of measure” for risk tolerance and the confidence in the probabilistic estimates needs to be established before the estimating and modeling process can be “calibrated” Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 138. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 138/186 The ranking of risk or the ranking of anything needs to be done in a structured manner. A geometric progression is a very useful approach, since it forces the focus on ranking. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 139. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 139/186 The “sense” of risk and real risk need to be connected. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 140. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 140/186 Some type of risk ranking needs to be developed for the IMS tasks. One approach is the TRL scale. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 141. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 141/186 When tasks are arranged in series the cumulative probability of completion is show in the table on the right Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 142. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 142/186 When the tasks are arranged in parallel a different completion profile results. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 143. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 143/186 All of this is very interesting in a Power Point presentation – marketecture it's called. Let’s look at a real schedule and start to apply some of the things we’ve learned. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 144. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 144/186 This is a very simple construction plan. The tasks are networked in a way to show how the Risk+ tool works. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 145. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 145/186 The first picture of a completion time is the PERT assessment. The task Construction Schedule Margin (the end of this task is the end of the planned margin) has a target date of 2/8/06 and a forecast PERT date of 3/6/06. This shows there is not enough margin by one month for this task Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 146. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 146/186 The same task, evaluated with Risk+ shows a different completion date. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 147. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 147/186 Project Risk Analysis is part of any good risk management activity. This has been said numerous times and needs to he repeated daily. Both the technical and the programmatic risk aspects of the program need to be shown in the IMS. Any questions, changes, updates, suggestions – anything that touches the IMS or the cost model – needs to be assessed from the point of view of programmatic risk. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 148. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 148/186 The accuracy of the dates and costs in the IMS is a “relative” term. ±20% to start with is pretty good. As the program proceeds accuracy improves but it is always a statistical estimate until after the fact. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 149. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 149/186 If we take a deterministic approach the planning then there will be built in issues. The first is that all estimates must include a confidence interval or they are wrong. The “natural” approach to estimating almost always results in a bound that is too wide as well as being optimistic or pessimistic but hardly never accurate. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 150. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 Prepared for NNJ05111915R, by Glen B. Alleman December 2005 150/186
  • 151. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 151/186 Now that we’ve reached a fairly detailed level of discussion regarding programmatic risk assessment, it’s time to talk about cost risk assessment. The first concept to understand is that cost and schedule are connected. This is obvious. But they are not connected in any linear manner. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 152. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 152/186 The basic principles of cost estimating start with the understanding of the uncertainty in the estimates of cost. These uncertainties must be connected to the technical uncertainties as well as the programmatic and simple cost variances. The arithmetic addition of costs creates a false number of the not only the cost but any variance in this cost. Monte Carlo simulation is one starting point, but like the programmatic simulations, the underlying probability distributions must be understood before the numbers have any real meaning Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 153. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 153/186 The connection between a technical parameter is its cost is not only potentially non–linear it is probabilistically non-linear. Knowing these Cost Element Relationships (CER) is a critical success factor for managing in the presence of uncertainty for both reducible and irreducible risks that are created Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 154. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 154/186 A simple 9 step (not so simple actually) process can be used to build a cost estimate. Starting with the “likely” program in the form of an IMS, the tasks for delivering that program are defined. The underlying probability distributions for the cost of each delivering activity are developed. This is much like the development of the baseline IMS, but the next step is much different. The correlation between each WBS element is developed. These correlations are used to build a model of sensitivity of the cost to changes in the tasks. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 155. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 155/186 At this point the Risk+ tool fails to deliver what is needed. Wither Crystal Ball or @RISK is needed to connect these correlations together. The technical uncertainty of the program is used to drive the cost uncertainty. This is where the technical and programmatic risk assessments joins. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 156. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 156/186 The production of the familiar probability curves for the likelihood of cost is the result. This curve tells is the likelihood that some event will occur. For example the probability of a cost in this example. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 157. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 157/186 The risk margin in dollars is the result needed to make this connection. Without this margin, the budget for the program ‒ as a single point estimate ‒ is not credible. All programs operate in the presence of uncertainty, so no single point estimate can be correct. For reducible uncertainties, specific actions can be take to protect the outcome. For irreducible uncertainties, only margin can protect the outcomes Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 158. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 158/186 Without the cost margin, the result is the risk that the budget is not sufficient to complete the program without a cost overrun. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 159. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 159/186 Management reserve, contingency and other budget protection is then needed. Much much reserve and contingency comes from modeling the program and determining the level of confidence needed to protect the budget from overrun. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 160. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 160/186 With models of both the cost uncertainties (reducible and irreducible) and schedule uncertainties (reducible and irreducible) a Risk Adjusted Integrated Master Schedule can be created. This IMS has Risk Handling activities for the Reducible Risks and Margin for the Irreducible Risks Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 161. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 161/186 With the Risk Handling content in place, a new assessment of the probabilistic outcomes can be shown. Before risk management and after risk management. This is no actionable information to the decision makers. Was the risk reduced enough to start the program, continue the program? Or are further risk reduction activities needed? Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 162. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 162/186 The cumulative cost curve – in this example, can then be used to make decisions using the value at risk Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 163. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 163/186 All programs operate in the presence of uncertainty. Making decisions in the presence of this uncertainty can take place with the proper information. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 164. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 Prepared for NNJ05111915R, by Glen B. Alleman December 2005 164/186
  • 165. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 165/186 We’re near the end now, so your brain is certainly getting full. This is quite a bit of information to absorb, but it needs to be done before we can say we are building “risk tolerant” plans. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 166. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 166/186 When probabilistic schedule analysis is used it does not replace the need for a well formed project network. It only replaces the use of PERT for estimating the completion dates. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 167. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 167/186 The quality of the probabilistic estimates is the foundation of confidence. The next step is to clearly identify where in the IMS risks are being mitigated, the impacts of this mitigation and the overall confidence in the master plan resulting from this mitigation Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 168. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 168/186 The correlations between cost, schedule, and technical risk must be made explicit. A model of how these elements interact is the basis for answering the “what if” questions that occur when the risk item becomes active. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 169. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 169/186 Risk based schedule and cost management is core to programmatic integrity. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 170. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 Prepared for NNJ05111915R, by Glen B. Alleman December 2005 170/186
  • 171. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 171/186 Like any good idea it can be improved on forever. These opportunities are much harder to address than the process so far. They require care and effort to build a correlation matrix for the tasks. They require detailed understanding of the underlying statistical processes and the historical data that was used to develop these distributions. For most projects this is beyond the scope of the effort and may be beyond the business interests as well – since the pay back is not clearly defined. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 172. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 172/186 The dependencies between tasks is the basis of the correlation function. This is very important if a true model of the network is to be developed. In the absence of the correlations it is assumed tasks are independent, which of course can not be the case. Building a Program risk assessment requires that cost and schedule be connected as well – correlated. Cost and schedule are not linear, so any simple model of changes in one linearly effecting the other cannot work. Finally the idea of a causal model – a cause and a set of effects provides deeper insight into the risk behaviors of the network. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 173. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 173/186 There is too much information here for a single digestive process. The only way to absorb all this is to start practicing probabilistic schedule and cost analysis and make the knowledge appear in the output information. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 174. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 174/186 Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 175. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 175/186 Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 176. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 176/186 Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 177. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 177/186 Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 178. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 178/186 Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 179. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 179/186 Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 180. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 180/186 Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 181. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 181/186 Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 182. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 182/186 Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 183. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 183/186 Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 184. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 184/186 Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 185. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 185/186 There are nearly unlimited resources on the web. The challenge of course is finding them. Here’s some know starting points. Prepared for NNJ05111915R, by Glen B. Alleman December 2005
  • 186. Analysis of Probabilistic Schedule and Cost Last Updated: 7/8/19 186/186 This has been a long journey over hopefully many weeks of discussion and hands on experience with Risk+ and real project schedules. Building a risk tolerant IMS is a “practice” and practices require proficiency. Proficiency comes from “doing the work,” looking at the results and making changes for improvement. This is just the beginning. Prepared for NNJ05111915R, by Glen B. Alleman December 2005