Patterson.dan

Using the Risk Register in Integrated
Cost/Schedule Risk Analysis
with Monte Carlo Simulation

David T. Hulett, Ph.D.
Hulett & Associates, LLC
Dr. Dan Patterson, PMP
Acumen

NASA PM Challenge
Daytona Beach, FL
February 24-25, 2009

© 2009 Hulett & Associates,
AcumenPM

Agenda
• Explain “Risk Factors” approach
• Apply Risk Factors to schedule and cost risk
• Apply Risk Factors to simple space vehicle
development schedule as an example

AcumenPM

Limitations with the Traditional
3-point Estimate of Activity Duration
• Typical schedule risk analysis starts with the
activity that is impacted by risks
– Estimates the 3-point estimate for optimistic, most
likely and pessimistic duration
– Implies the risk is 100% likely with uncertain impact
• Which risks cause the most overall schedule
risk? Cannot say directly, but indirectly:
– Sensitivity to activity durations
– Criticality of activity durations

AcumenPM

Some Problems with Traditional
Approach
• Can tell which activities are crucial, but not
directly which risks are driving
• Makes poor use of the Risk Register that
is usually available
• Cannot decompose the overall schedule
risk into its components BY RISK
– Ability to assign the risk to its specific risk
drivers helps with communication of risk
causes and risk mitigation

AcumenPM

We Propose a Different Approach:
Start with the Risks Themselves
• Drive the schedule risk by the risks already
prioritized in the Risk Register
• For each risk, specify:
– Probability it will occur (NEW)
– Impact on time if it does (multiplicative factors, allows it to
affect different duration activities)
– Assign to the activities it will affect
• Starting with the risks themselves gives us benefits
– Links qualitative analysis to the quantitative analysis
– Estimates the impact of specific risks for prioritized
mitigation purposes
– Correlations between activities modeled automatically
AcumenPM

Simple Example of Risk Register Risks

• Use the Risk Factors module in Pertmaster 8
• Collect probability and impact data on risks
• Map the risks to activities

AcumenPM

Risk Factors Mechanics (1)
• The risk factor is assigned to one or several
activities, affecting their durations by a
multiplicative factor
– E.g., the factor may be .90 for optimistic, 1.0 for most
likely and 1.25 for pessimistic
– These factors multiply the schedule durations of the
activities to which they are assigned
• Risks can be assigned to one or more activities
• Activity durations can be influenced by one or
more risks
AcumenPM

Risk Factors Mechanics (2)
• Risk Factors are assigned a probability of
occurring on any iteration
– When the risk occurs, the factor used is chosen at
random from the 3-point estimate and operates on all
activities to which it is assigned
– When not occurring on an iteration the risk factor
takes the value 1.0, a neutral value
• When an activity is influenced by more than one
risk, their factors are multiplied together, if they
happen, on any iteration

AcumenPM

Risk Factor
Probability is 100%, Factor can be + or -

0 0 1 0 - C o n s tr u c tio n : D u r a tio n
1 0 0% 11 5
2 50
9 5 % 1 11

Here the 9 0 % 1 09

8 5 % 1 08
For the
Ranges are 2 00
8 0 % 1 06

7 5 % 1 05
examples we
based on 7 0 % 1 04 use an activity
6 5 % 1 04
deviations + with 100 days

Cumulative Frequency
6 0 % 1 03
1 50

and – from 5 5 % 1 02
in the
Hits

5 0 % 1 01

the Plan. 4 5 % 1 01

4 0 % 1 00
schedule
1 00
Probability 35% 99

30% 99
is 100% 25% 98

50 20% 97

15% 96

10% 95

5 % 94

0 0 % 90
90 95 10 0 1 05 1 10 11 5
D istrib u tio n (sta rt o f in te rva l)

AcumenPM

Assigning a Probability Less than 100%

0010 - Construction : Duration 0040 - Technology Design : Duration
100% 115 100% 130

Spike 95% 107 95% 123

contains
90% 103 1200 90% 120
Spike
85% 101 85% 118

70% of
2000
80% 100 80% 116 contains
the
75% 100

70% 100
1000 75% 114

70% 113
40% of
probability
65% 100 65% 111
the

1500
60% 100 800 60% 110

55% 100 55% 109 probability

Hits
Hits

50% 100 50% 107

45% 100 600 45% 105

1000 40% 100
40% 100

35% 100 35% 100

30% 100 400 30% 100

25% 100 25% 100

500 20% 100 20% 100

15% 100 200 15% 100

10% 99 10% 100

5% 97 5% 100

0 0% 100
0 0% 91
100 110 120 130
95 100 105 110 115
Distribution (start of interval) Distribution (start of interval)

AcumenPM

Assigning
More than One Risk to an Activity
• If more than one risk is acting on an
activity, the resulting ranges are the
multiplication of the factors
• The activity duration range is derived from
the risk factors that affect the risk
– Model how the activity duration range is
generated
– Focus on the causes of activity duration
ranges
AcumenPM

Two Risks affect One Activity
using Factors that Occur 100%

0040 - T echnology Design : Duration
100% 144

95% 130

90% 127
140
85% 124

80% 123

120 75% 121

70% 119

Range 100
65% 118

60% 117
from 90 to 55% 116
Hits

150 days, 80 50% 115

45% 114

Peak about 60
40% 113

35% 111
113 days 30% 110

40 25% 109

20% 108

15% 106
20
10% 104

5% 102

0 0% 93
100 110 120 130 140
Distribution (sta rt of inte rva l)

AcumenPM

Two Risks with Less than 100%
Probability Affecting one Activity

0040 - T echnology Design : Duration
1100 100% 144

95% 123

The spike at 1000 90% 119

85% 116

100 days 900
80% 113

75% 111
represents (1) 800
70% 110

the likelihood 700
65% 108

60% 106

that neither risk 600 55% 104
Hits

50% 102
occurs and (2) 500 45% 101

the chance that 400
40% 100

35% 100

100 days is 300
30% 100

25% 100
picked when 200
20% 100

one or both 100
15% 100

10% 99

occur 5% 97

0 0% 91
100 110 120 130 140
Distribution (sta rt of inte rva l)

AcumenPM

Risk Factors Model How Correlation Occurs
Coefficients are Calculated (1)
Risk #1
P = 50%, Factors
.95, 1.05, 1.15

Activity A Activity B

Activities A and B
Correlation is Calculated
to be 100%

AcumenPM

Risk Factors Model How Correlation Occurs
Coefficients are Calculated (2)
Risk #2 Risk #1 Risk #3
P = 25%, Factors P = 50%, Factors P = 45%, Factors
.8, .95, 1.05 .95, 1.05, 1.15 1.0, 1.1, 1.2

Activity A Activity B

Activities A and B
Correlation is Calculated
to be 48%

Correlation is modeled as it is caused in the project
Correlation coefficients are generated, not guessed

AcumenPM

Sensitivity to the Risk Factors

The tornado
diagram
focuses on
risks, not
activities

AcumenPM

Simple 2-Stage Space Vehicle Schedule

Software used: Pertmaster v. 8
AcumenPM

Case Study: Simple Space Vehicle
Development Schedule
• 87 month schedule
• 11 work activities linked, 3 major milestones
• Beginning 3 March 2008
• PDR on 11 September 2009
• CDR on 3 June 2011
• Delivery to launch site 12 June 2015

AcumenPM

Two Types of Risk
• Background risk based on typical general
risk, estimating error
– Used Quick Risk of -5% and +10%
• Discrete risks derived from Risk Register
– Summarized from detailed Risk Register
– These have a probability of occurring and an
impact on specific activities if they do
– Parallel to their Risk Register information
– Risk Register is used in data collection

AcumenPM

Standard 3-point Range Representing
Schedule Estimating Error

Background
risk:
Optimistic -5%
Pessimistic
+10%

AcumenPM

Results with
Schedule Estimating Error Only
Spacecraft Program
Entire Plan : Finish Date
100% 30/Dec/15
95% 27/Oct/15
400
90% 15/Oct/15
85% 07/Oct/15
80% 30/Sep/15
Deterministic:
75% 24/Sep/15 12JUN15 is <1%
300 70% 18/Sep/15

65% 14/Sep/15
60% 09/Sep/15
P-80 is 30SEP15,
55% 04/Sep/15 about 3.5 months
Hits

200
50% 01/Sep/15
45% 26/Aug/15
later than planned
40% 21/Aug/15
35% 18/Aug/15 Spread from P-5 to
30% 12/Aug/15
P-95 is 5JUL15 to
25% 07/Aug/15
100
20% 31/Jul/15 27OCT15 for 3.7
15% 24/Jul/15
10% 17/Jul/15
months
5% 06/Jul/15
0 0% 14/May/15
31/May/15 08/Sep/15 17/Dec/15
Distribution (start of interval)

AcumenPM

Risk Analysis on Space Vehicle Project
Risk Factors are from Risk Register
Time Impact Range Cost Impact Range
Probability Minimum Most Likely Maximum Minimum Most Likely Maximum
Requirements have not been
30% 95% 105% 120%
decided
Several alternative designs
60% 95% 100% 115%
considered
New designs not yet proven 40% 96% 103% 112%
Fabricaton requires new
50% 96% 105% 115%
materials
Lost know‐how since last full
30% 95% 100% 105%
spacecraft
Funding from Congress is
70% 90% 105% 115%
problematic
Schedule for testing is
100% 100% 120% 130%
aggressive
Cost Risk is based on immature
100% 95% 105% 110%
data

AcumenPM

Mapping Risks to Activities (1)
FS
Requirements FS Test FS
Risk Preliminary FS Final Design
Definition Fabrication Engine
Design
Requirements Not
X
Complete
Alternative Designs
X
Possible
Designs Not Proven X
New Materials in
X
Fabrication
Lost Know-How X
Funding Problematic X X X X
Testing Schedule
X
Aggressive
Cost Estimate is
based on Immature X X X X X
Data

AcumenPM

Mapping Risks to Activities (2)
US
US Final Integratio
Risk Preliminary US Fabrication US Test Integration
Design n Testing
Design
Requirements Not
Complete
Alternative Designs
X
Possible
Designs Not
X
Proven
New Materials in
X
Fabrication
Lost Know-How X X
Funding
X X X X X X
Problematic
Testing Schedule
X X
Aggressive
Cost Estimate is
based on Immature X X X X X X
Data

AcumenPM

Results Adding
Risk Factors to the Background Risk
Spacecraft for PMChallenge 2009
Entire Plan : Finish Date
100% 03/May/17
350

95% 05/Oct/16

90% 11/Aug/16

Baseline 12JUN 15 300 85% 01/Jul/16

80% 26/May/16
is only 3% likely 75% 02/May/16

250

The 80th percentile
70% 06/Apr/16

65% 15/Mar/16

(P-80) is 26MAY16, 60% 24/Feb/16

200
11.5 months later 55% 04/Feb/16

Hits
50% 15/Jan/16

Spread P-5 to P-95 150
45% 30/Dec/15

40% 16/Dec/15

is 13JUL15 to 35% 01/Dec/15

5OCT16, for 15.5 100
30% 16/Nov/15

months 25% 02/Nov/15

20% 15/Oct/15

15% 25/Sep/15
50

10% 27/Aug/15

5% 13/Jul/15

0 0% 21/Jan/15
20/Feb/15 08/Sep/15 26/Mar/16 12/Oct/16 30/Apr/17

AcumenPM

Activity Tornado Chart
from All-In Simulation
Duration Sensitivity

00025 - US Fabrication 81% Risky Activities:
80%
Fabrication,
00011 - FS Fabrication

Integration, Final
00028 - Integration 78%
Design, Preliminary
00009 - FS Final Design 76% Design, Testing
76%
This is the typical
00023 - US Final Design

sensitivity analysis of
00021 - US Preliminary Design 69%
3-point estimating
00007 - FS Preliminary Design 69%

00012 - Test FS Engine 63%

00029 - Integration Testing 62%

00026 - US Test 61%

AcumenPM

Risk Factor Tornado
from All-In Simulation
Driving Schedule Risk Factors

6 - Funding from Congress is problematic

4 - Fabricaton requires new materials

3 - New designs not yet proven
The main RISK,
7 - Schedule for testing is aggressive
however, is funding
from Congress, which
5 - Lost know-how since last full spacecraft affects all activities. This
is the main risk to
2 - Several alternative designs considered
mitigate, if possible
1 - Requirements have not been decided

0.00 0.20 0.40 0.60 0.80
Correlation

AcumenPM

Contribution of
Each Risk to the Time Contingency (1)
Explain the Contingency to the P-80
P-80 Date Contribution of Risk
All Risks In 26-May-16 Days Saved % of Contingency
Specific Risks Taken Out in Order
Funding Risk 22-Jan-16 125 36%

Testing Schedule is Aggressive 1-Dec-15 52 15%

Design Requires New Materials 28-Oct-15 34 10%

New Design Risk 15-Oct-15 13 4%
Alternative Design Risk 6-Oct-15 9 3%
Requirements Risk 1-Oct-15 5 1%
Lost Know How Risk 30-Sep-15 1 0%
Background Schedule Estimating Risks
Background Risk 12-Jun-15 110 32%
Total Contingency 349 100%

AcumenPM

Contribution of
Each Risk to the Time Contingency (2)
g g
100%

90%

26/May /16
01/Dec /15
12/J un/15

01/O c t/15
30/Sep/15
06/O c t/15
16/O c t/15
28/O c t/15

20/J an/16
80%

70%

Cum ulative P robability
60%

50%

40%

30%

20%

10%

0%
06/ Apr/ 15 15/ Jul/ 15 23/ Oct/ 15 31/ Jan/ 16 10/ May/ 16 18/ Aug/ 16 26/ Nov/ 16 06/ Mar/ 17

AcumenPM

Resources and Cost

Estimate is
$6.86 billion

AcumenPM

Cost Risk Results
Entire Plan : Cost
100% $9,283,008,684

95% $8,429,356,166
650

90% $8,234,937,806

Baseline $6.86 billion 600
85% $8,088,750,149

is only 5% likely 550
80% $7,976,966,619

75% $7,890,603,866

500

The 80th percentile 70% $7,813,239,105

450 65% $7,749,200,097

(P-80) is $7.98 B for 60% $7,687,432,085

a $1.1 B contingency Hits
400
55% $7,630,117,222

350 50% $7,568,332,545

Spread P-5 to P-95 300
45% $7,514,273,968

is $6.86 to $8.43, for 250
40% $7,452,396,460

35% $7,391,997,593

$1.57 B 200
30% $7,331,963,666

25% $7,258,107,546

150
20% $7,188,049,098

100 15% $7,101,752,271

10% $7,007,130,725
50
5% $6,861,086,752

0 0% $6,189,493,454
$7,000,000,000 $8,000,000,000 $9,000,000,000

AcumenPM

Contribution of
Each Risk to the Cost Contingency
Contributions of Individual Risks to Cost Risk
All Risks $ millions
Cost Risk is based on immature data 287
Funding from Congress is problematic 277
Fabricaton requires new materials 168
Schedule Estimate inaccurate 110
Schedule for testing is aggressive 63
New designs not yet proven 31
Lost know‐how since last full spacecraft 2
Several alternative designs considered 1
Requirements have not been decided 0

AcumenPM

Scatter Diagram of Time and Cost
Deterministic Point Inside both limits Outside both limits

3%

There is a 94% $9,200,000,000
1% 94%

chance of $9,000,000,000

overrunning both $8,800,000,000

cost and $8,600,000,000

schedule $8,400,000,000

Notice the slope $8,200,000,000

of the scatter – $8,000,000,000
Entire Plan: Cost

time drives cost $7,800,000,000

in this model $7,600,000,000

$7,400,000,000

$7,200,000,000

$7,000,000,000

5%
$6,800,000,000 $6,861,500,000

$6,600,000,000

$6,400,000,000

2% 12/ Jun/ 15 3%
$6,200,000,000
20/Feb/15 31/May/15 08/Sep/15 17/Dec/15 26/Mar/16 04/Jul/16 12/Oct/16 20/Jan/17 30/Apr/17
Entire Plan: Finish

AcumenPM

Probabilistic Cash Flow
Resource Flow for Cost
Filter: Entire Plan

Mean P20 P80

This monthly 9,000,000,000

probabilistic cash
8,000,000,000

flow can be
compared to the Deterministic Cost: $6,861,500,000 7,000,000,000

budget to adjust the 6,000,000,000

spending patterns

Cu m u lative
5,000,000,000

when considering

D eterm inis tic F inis h: 12/J un/15
risk 4,000,000,000

3,000,000,000

2,000,000,000

1,000,000,000

0
26/ Jul/ 08 11/ Feb/ 09 30/ Aug/ 09 18/ Mar/ 10 04/ Oct/ 10 22/ Apr/ 11 08/ Nov/ 11 26/ May/ 12 12/ Dec/ 12 30/ Jun/ 13 16/ Jan/ 14 04/ Aug/ 14 20/ Feb/ 15 08/ Sep/ 15 26/ Mar/ 16 12/ Oct/ 16 30/ Apr/ 17
Time

AcumenPM

Probabilistic Cash Flow
Compared to Planned

AcumenPM

Summary (1)
• The focus is on the risks, not their impact
• Risks “explain” the need for a contingency
• Management appreciates this focus on
risks
• Risk interviews are conducted at 10,000
foot level, where people typically think of
risk
• Interviews go faster, stick to the substance
AcumenPM

Summary (2)
• Use Risk Register for quantitative analysis
• Specific risks can be quantified and assigned to
schedule activities
– Quantification is probability and impact
– A risk can affect several activities
– An activity can be affected by several risks
• Risk Factors can be combined with other more
traditional approaches such as 3-point estimates
for background risk or probabilistic branching

AcumenPM

Summary (3)
• Schedule uncertainty creates cost
uncertainty
• Analysis of cost simultaneously with time
requires inserting the budget/resources
into the schedule and simulating both
together
• More accurate cost risk analysis and full
appreciation of the role of schedule risk in
creating cost risk
AcumenPM

Using the Risk Register in Integrated
Cost/Schedule Risk Analysis
with Monte Carlo Simulation

David T. Hulett, Ph.D.
Hulett & Associates, LLC
Dr. Dan Patterson, PMP
Acumenpm

NASA PM Challenge
Daytona Beach, FL
February 24-25, 2009

AcumenPM

Patterson.dan

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (20)

More from NASAPMC

More from NASAPMC (20)

Recently uploaded

Recently uploaded (20)

Patterson.dan