Risk management using risk+ (v5)

1 INCREASING THE PROBABILITY OF PROGRAM SUCCESS USING RISK+Glen B. Alleman A workshop on the principles and practices of Risk+ andNiwot Ridge LLC increasing the Probability of Program Success

A Warning We’re going to cover a lot of material in 3 hours2

Risk involves uncertainty. Uncertainty involves probability.3

4 Douglas Adams, Hitchhikers Guide to the Galaxy

MOTIVATION? Your motivation? Your motivation is your pay packet on Friday. Now get on with it. – Noel Coward, English actor, dramatist, & songwriter (1899 – 1973)5

 We have to know the underlying statistical behavior of the processes driving the project  This means cost, schedule, and technical performance measures with probabilistic models  We need to know how these three statistical drivers are coupled  What drives what?  What are the multipliers between each random6 variable?

Let’s start with the basics7

Remember High School Statistics8

The IMS is a collection of probabilistic processes all coupled together9

What does this really mean?10  In building a risk tolerant IMS, we’re interested in the probability of a successful outcome…  “What is the probability of making a desired completion date?”  But the underlying statistics of the tasks influence this probability  The statistics of the tasks, their arrangement in a network of tasks and correlation define how this probability based estimated developed.

There are real problems with those pesky Unknowns that get in the way of progress Imprint of a bird on our west facing family room second story window on a bright afternoon11 The Bird survived

The “units of measure” of Risk12  These classifications can be used to avoid asking the “3 point” question for each task.  Anchoring and Adjustment† of all estimating processes produces a bias.  Knowing this is necessary for credible estimates. Classification Uncertainty Overrun 1 Routine, been done before Low 0% to 2% 2 Routine, but possible difficulties Medium to Low 2% to 5% 3 Development, with little technical difficulty Medium 5% to 10% 4 Development, but some technical difficulty Medium High 10% to 15% 5 Significant effort, technical challenge High 15% to 25% 6 No experience in this area Very High 25% to 50% † Tversky and Khanemann Anchoring and Adjustment

We’re looking for knowledge of what is going to happen in he future, with a known level of confidence13 Harvard main library

15 What is Monte Carlo Simulation? With some principals behind us, let’s see how to use Risk+ to address the problem of forecasting the future of schedule and cost performance.

A Quick Look At Monte Carlo16 George Louis Leclerc, Comte de Buffon, asked what was the probability that the needle would fall across one of the lines, marked here in green. That outcome will occur only if A  l sin

17 Los Alamos Science, Special Issue 1987

Monte Carlo Simulation18  Monte Carlo Simulation is named after the city, in Monaco, of casinos on the French Rivera.  Monte Carlo …  Examines all paths not just the critical path.  Provides an accurate (true) estimate of completion:  Overall duration distribution  Confidence interval (accuracy range)  Sensitivity analysis of interacting tasks  Varied activity distribution types – not restricted to a single distribution  Schedule logic can include branching – both probabilistic and conditional  When resource loaded schedules are used – provides integrated cost and schedule probabilistic model.

20 What Are We Really After? We need to answer the question … What is the confidence we will complete “on or before” at date and “at or below” at cost? This is the question that should be asked and answered on a periodic basis. We need to have Schedule and Cost margin to protect the deliverables and our Budget At Completion.

Here is some advice on how to depict this margin and where to place this margin. No matter how we show manage these two elements in the IMS, if we don’t have margin we are late and over budget before we start. http://www.ndia.org/Divisions/Divisions/Procurement/Documents/PMSCommittee/CommitteeDocuments/21 WhitePapers/NDIAScheduleMarginWhitePaperFinal-2010(2).pdf

Confidence levels for margin change as the program proceeds22 As the program proceeds we want to have  Increased accuracy  Reduced schedule risk  Increasing visual confirmation Current Estimate Confidence that success can be reached

Our REAL goal here is to Manage Margin using probabilistic models23  Programmatic Margin is  Margin that is not used in the added between Development, IMS for risk mitigation will be Production and Integration & moved to the next sequence Test phases of risk alternatives  Risk Margin is added to the  This enables us to buy back IMS where risk alternatives schedule margin for activities are identified further downstream  This enables us to control the Downstream ripple effect of schedule shifts Plan B Duration of Plan B < Plan A + Margin Activities shifted to left 2 days on Margin activities Plan B 3 Days Margin Used Plan A 5 Days Margin First Identified Risk Alternative in IMS Plan A 5 Days Margin Second Identified Risk 2 days will be added to this margin task Alternative in IMS to bring schedule back on track

Sensitivity Analysis24  The schedule sensitivity of a task measures the closeness with which change in the task duration matches change in the project duration over the simulation.  This closeness is the correlation between changes in individual activities and their impacts on other activities.  A task with high schedule sensitivity is more likely to be a major driver of the project duration than a lower ranked task. : Models of the Schedule

Task Criticality Analysis25  A measure of the frequency that an activity in the project schedule is critical (Total Float = 0) in a simulation  If a task is critical in 500 of the 1,000 iterations of the simulation, it has a Criticality Index of 0.5  The higher the criticality index, the more certain it is that the task will always be critical in the project : Models of the Schedule

Cruciality shows each task’s tolerance to risk26  Cruciality = Schedule Sensitivity x Criticality  Schedule Sensitivity can be statistically misleading: A task with high sensitivity may not be on or near the critical path.  Thus a reduction in that task’s duration may have little effect on the project duration.  Cruciality sharpens the analytical focus:  It highlights critical or near–critical activities with high.  Schedule Sensitivity  These tasks are most likely to drive project duration. : Models of the Schedule

Guiding the Risk Factor Process means weighting each level of risk27  For tasks marked “Low” a reasonable approach is to score the maximum Min Most Max 10% greater than the minimum. Likely  The “Most Likely” is then scored as a geometric progression for the Low 1.0 1.04 1.10 remaining categories with a common Low+ 1.0 1.06 1.15 ratio of 1.5 Moderate 1.0 1.09 1.24  Tasks marked “Very High” are bound Moderate+ 1.0 1.14 1.36 at 200% of minimum. High 1.0 1.20 1.55  No viable project manager would like a task grow to three times the planned High+ 1.0 1.30 1.85 duration without intervention Very High 1.0 1.46 2.30  The geometric progress is somewhat Very High+ 1.0 1.68 3.00 arbitrary but it should be used instead of a linear progression : Examples of Monte Carlo

Progressive Risk Factors28  A geometric progression (1.534) of risk can be used.  The phrases associated with increasing risk have been shown at the Naval Research Laboratory to correlate with an engineers “sense” of increasing risk. : Examples of Monte Carlo

Risk Factor Attributes29  The “narrative” for each risk factor needs to be developed.  Each description is dependent on…  Discipline  Program stage  Complexity  Historical data  Current “risk state” of the program  This is currently missing from our efforts to quantify schedule and cost risk. : Examples of Monte Carlo

Accuracy30  Given a specified final cost or project duration, what is the probability of achieving this cost or duration?  Frequentist approach  Over many different projects, four out of five will cost less or be completed in less time than the specified cost or duration.  Bayesian approach  We would be willing to bet at 4 to 1 odds that the project will be under the 80% point in cost or duration.  Accuracy is needed to plan reserves.  Accuracy is needed when comparing competing proposals. : What is the Purpose of Project Risk Analysis?

Structured Thinking31  All estimates will be in error to some degree of variance.  Trying to quantify these errors will result in bounds too wide to be useful for decision making.  Risk analysis should be used to  Think about different aspects of the project  Try to put numbers against probabilities and impacts  Discuss with colleagues the different ideas and perceptions  Thinking things through carefully results in  Which elements of the programmatic and technical risk are represented in the IMS.  The process becomes more valuable than the numbers. : What is the Purpose of Project Risk Analysis?

To properly use Schedule Margin†32  Work must be represented in single units – either task or work packages.  The overall schedule margin must be related to the variation of individual units of work.  The importance of the units of work must be shared among all participants (ordinal ranking of work and its risk).  The schedule must be reasonable in some units of measure shared by all the participants. † “Protecting Earned Value Schedules with Schedule Margin,” Newbold, Budd, and Budd, http://www.prochain.com/pm/articles/ProtectingEVSchedules.pdf

Let’s Apply a Monte Carlo Simulation Tool The Monte Carlo trolley, or FERMIAC, was invented by Enrico Fermi and constructed by Percy King. The drums on the trolley were set according to the material being traversed and a random choice between fast and slow neutrons. Another random digit was used to determine the direction of motion, and a third was selected to give the distance to the next collision. The trolley was then operated by moving it across a two dimensional scale drawing of the nuclear device or reactor assembly being studied. The trolley drew a path as it rolled, stopping for changes in drum settings whenever a material boundary was crossed. This infant computer was used for about two years to determine, among other things, the change in neutron population with33 time in numerous types of nuclear systems.

35 A Small Diversion Most Likely Isn’t Likely to be the Most Likely When we say “most likely” what do we think this actually means? If you pick the wrong meaning, your Monte Carlo model will be seriously flawed.

The problem with “Most Likelies”36  For each activity the “best” estimate is …  The “most likely” duration – the mode of the distribution of durations? (Mode is the number that appears most often)  It’s 50th percentile duration – the median of the distribution? (Median is the number in the middle of all the numbers)  It’s expected duration – the mean of the distribution? (Mean is the average of all the numbers)  These definitions lead to values that are almost always different from each other.  Rolling up the “best” estimate of completion is almost never one of these.

Durations are Probability Estimates not Single Point Values37  We know this because…  “Best” estimate is not the only possible estimate, so other estimates must be considered “worse.”  Common use of the phrase “most likely duration” assumes that other possible durations are “less likely.”  “Mean,” “median,” and “mode” are statistical terms characteristic of probability distributions.  This implies activity distributions have probability distributions  They are random variables drawn from the probability distribution function (pdf).  “Actual” project duration is an uncertain quality that can be modeled as a sum of random variables  The pdf may be known or unknown.

3 Task Most Likely ≠ Project Most Likely38  PERT assumes probability distribution of the project times is the same as the tasks on the critical path.  Because other paths can become critical paths, PERT consistently underestimates the project completion time. 1+1=3 : Managing Uncertainty in the IMS

Probability Distribution Function is the Lifeblood of good planning39  Probability of occurrence as a function of the number of samples.  “The number of times a task duration appears in a Monte Carlo simulation.” : Managing Uncertainty in the IMS

Remember the quote about statistics40 Lies, Damn Lies, and Statistics – Benjamin Disraeli But we know better, we know that any estimate without a variance is not trustworthy. We know that the variances have to be calibrated from past performance to be credible

42 A “Real World” Schedule Analysis One should expect that the expected can be prevented, but the unexpected should have been expected. — Augustine Law XLVThis is a must own book for everyone in our business. It definesfundamental Laws of program and business management, whichare many times ignored – like the one above

Our Starting Point43  Risk+ Installed  Let’s define the needed fields  These are used by Risk+ to hold information and run the application.  If there are conflicts, you can make changes in Risk+ to work around your fields.

A Simple IMS44 By simple it means serial cascaded work efforts.

Initial Field Usage45  Minimum Remaining Duration  The duration that is least you’d expect this task to complete in  Most Likely Remaining Duration  The ML (Mode) of the duration  Maximum Remaining Duration  The duration that is the most you’d expect this task to complete in  Task Reporting ID  The tasks we want to watch

Define a View and Table for Risk+46 Start with the Gantt View and Entry Table Set up both to match the Risk+ field usage Use the default if there are no field conflicts

Fields used for Risk+ example47

Let’s actually “doing something”48  Initialize the Most Likely.  This sets the Most Likely duration to the same value that is in the “Duration” field of your IMS.  The “planned duration” now becomes the ML duration.  If this “planned duration” is bogus then your model will be as well.  Choose wisely.

Now the ML = DURATION step49  All the DURATION values have been moved to the ML field.  But remember our discussion of the ML’s  Choose them carefully  The next we’ll set the upper and lower limits of that ML value  Using risk factors.  OK, 3 point estimates if you have to.

Let’s do this the simple way50  Let’s pick MEDIUM confidence.  MEDIUM means  –25%  +25%  And a NORMAL (Gaussian) curve

Let’s have Risk+ do something for us51  Enter a “1” in the RPT field (Number 1)  This marks that ROW in the schedule as a work activity we want to see the Monte Carlo output for

Now we’re ready to run52  The RISK ANALYSIS command starts the process going.  Let’s make 200 iteration and look at the DURARTION ANALYSIS for the activities we are watching.

This is nice but what actually is Risk + doing?53  Risk+ is picking a random number from under the normal distribution within the range of the  Least remaining and most remaining  This is not some ordinary random number it is chosen through an algorithm called the Latin Hypercube - more on that later.  Risk+ then plugs that number into the “real” DURATION field and does that for all the DURATIONS in the schedule  Then the F9 key is pressed and the date is recorded for the finish of UID 41.  This is done 200 times and a histogram of all the dates that appeared for those 200 time is recorded.

And We Get54 Date: 11/29/2011 4:32:17 PM Completion Std Deviation: 2.06 days Samples: 500 95% Confidence Interval: 0.18 days Unique ID: 19 Each bar represents 1 day Name: End Work Package 3 0.20 1.0 Completion Probability Table 0.18 0.9 Cumulative Probability Prob Date Prob Date 0.16 0.8 0.05 Wed 3/7/12 0.55 Tue 3/13/12 0.14 0.7 0.10 Thu 3/8/12 0.60 Tue 3/13/12 Frequency 0.12 0.6 0.15 Thu 3/8/12 0.65 Tue 3/13/12 0.10 0.5 0.20 Fri 3/9/12 0.70 Wed 3/14/12 0.08 0.4 0.25 Fri 3/9/12 0.75 Wed 3/14/12 0.06 0.3 0.30 Fri 3/9/12 0.80 Wed 3/14/12 0.35 Mon 3/12/12 0.85 Thu 3/15/12 0.04 0.2 0.40 Mon 3/12/12 0.90 Thu 3/15/12 0.02 0.1 0.45 Mon 3/12/12 0.95 Fri 3/16/12 Fri 3/2/12 Mon 3/12/12 Tue 3/20/12 0.50 Mon 3/12/12 1.00 Tue 3/20/12 Completion Date

Learning to Speak in Risk+55  Risk +shows use the probability of finish “on or before” a date  It does NOT show the probability of success.  But even the “on or before” term is loaded with special meaning.  It means for the 500 iterations of Risk+ using the upper and lower bounds of the duration, drawn from the probability density function (pdf) with the Normal (Gaussian) shape, 60% of the finish dates were recorded to be on or before 3/12/12.

Medium confidence for a large project56 Date: 11/30/2011 6:05:35 PM Completion Std Deviation: 4.49 days Samples: 200 95% Confidence Interval: 0.62 days Unique ID: 17 Each bar represents 2 days Name: (SA) Systems Requirements Completed 0.22 1.0 Completion Probability Table 0.20 0.9 Cumulative Probability Prob Date Prob Date 0.17 0.8 0.05 Fri 5/4/12 0.55 Thu 5/17/12 0.7 0.10 Wed 5/9/12 0.60 Thu 5/17/12 Frequency 0.15 0.6 0.15 Thu 5/10/12 0.65 Fri 5/18/12 0.13 0.5 0.20 Fri 5/11/12 0.70 Mon 5/21/12 0.10 0.25 Mon 5/14/12 0.75 Mon 5/21/12 0.4 0.08 0.3 0.30 Mon 5/14/12 0.80 Tue 5/22/12 0.05 0.35 Tue 5/15/12 0.85 Wed 5/23/12 0.2 0.40 Tue 5/15/12 0.90 Thu 5/24/12 0.03 0.1 0.45 Wed 5/16/12 0.95 Mon 5/28/12 Wed 5/2/12 Wed 5/16/12 Mon 6/4/12 0.50 Wed 5/16/12 1.00 Mon 6/4/12 Completion Date

Low confidence for a large project57 Date: 11/30/2011 10:30:05 PM Completion Std Deviation: 9.14 days Samples: 200 95% Confidence Interval: 1.26 days Unique ID: 17 Each bar represents 3 days Name: (SA) Systems Requirements Completed 0.16 1.0 Completion Probability Table 0.9 0.14 Cumulative Probability Prob Date Prob Date 0.8 0.12 0.05 Thu 5/3/12 0.55 Fri 5/25/12 0.7 0.10 Tue 5/8/12 0.60 Mon 5/28/12 Frequency 0.10 0.6 0.15 Wed 5/9/12 0.65 Wed 5/30/12 0.08 0.5 0.20 Mon 5/14/12 0.70 Wed 5/30/12 0.4 0.25 Tue 5/15/12 0.75 Fri 6/1/12 0.06 0.3 0.30 Thu 5/17/12 0.80 Mon 6/4/12 0.04 0.35 Fri 5/18/12 0.85 Wed 6/6/12 0.2 0.02 0.40 Mon 5/21/12 0.90 Fri 6/8/12 0.1 0.45 Wed 5/23/12 0.95 Thu 6/14/12 Tue 4/24/12 Thu 5/24/12 Wed 6/27/12 0.50 Wed 5/23/12 1.00 Wed 6/27/12 Completion Date

Let’s run some simulations58

60 Basic Principles of Probabilistic Cost Now that the schedule can be produced using probabilistic methods, it’s time to talk about the cost. Cost does not have a linear relationship with schedule unfortunately. : Basic Principles of Probabilistic Cost

Basic Principles with Probabilistic Cost Estimating are coupled with scheduling61  Cost estimates usually involve many CERs  Each of these CERs has uncertainty (standard error)  CER input variables have uncertainty (technical uncertainty)  Must combine CER uncertainty with technical uncertainty for many CERs in an estimate  Usually cannot be done arithmetically; must use simulation to roll up costs derived from Monte Carlo samples  Add and multiply probability distributions rather than numbers  Statistically combining many uncertain, or randomly varying, numbers  Monte Carlo simulation  Take random sample from each CER and input parameter, add and multiply as necessary, then record total system cost as a single sample  Repeat the procedure thousands of times to develop a frequency histogram of the total system cost samples  This becomes the probability distribution of total system cost : Basic Principles of Probabilistic Cost

The Cost Probability Distributions as a function of the weighted cost drivers62 Combined Cost Modeling and Technical Uncertainty Cost = a + bXc Cost Modeling Uncertainty Cost Estimate Historical data point $ Cost estimating relationship Technical Uncertainty Standard percent error bounds Cost Driver (Weight) : Basic Principles of Probabilistic Cost

The Risk Adjusted Cost Estimate Connected To The IMS63  In the risk–adjusted cost estimate, we now combine discrete risk events and the uncertainty of the input distributions with the uncertainty of the CERs  Since the input distributions tend to be right–skewed, the expected cost tends to be larger than the baseline estimate  In addition, the risk–adjusted cost distribution tends to be wider than the baseline estimate  The difference between the expected cost of the risk– adjusted estimate and the expected cost of the baseline estimate is, by definition, the amount of RISK dollars included in the risk–adjusted estimate : Basic Principles of Probabilistic Cost

Baseline versus Risk Adjusted Cost Estimates Usually Show a Cost Increase64 Baseline vs. Risk-Adjusted Estimates Baseline: Mean = $102.6M Std Dev = $29.8M Risk–Adjusted: Mean = $122.6MLikelihood Std Dev = $42.8M 0 50 100 150 200 250 300 350 FY$M : Basic Principles of Probabilistic Cost

The S–Curve for Cost Modeling65 Cumulative Distribution Function 100% 90% 80% 80th percentileCumulative Probability 70% 50th percentile $153.5M 60% $114.7M 50% Baseline Estimate Mean $102.6M Risk–adjusted 40% Estimate Mean $122.6M 30% 20% 10% 0% $60 $80 $100 $120 $140 $160 $180 $200 FY00$M : Basic Principles of Probabilistic Cost

The Real Question Always Returns to… “But How Much Does It Cost? Really?”66  This is impossible to answer precisely  Decision–makers and cost analysts should always think of a cost estimate as a probability distribution, NOT as a deterministic number  The best we can provide is the probability distribution – If we think we can be any more precise, we’re fooling ourselves  It is up to the decision–maker to decide where he/she wants to set the budget  The probability distribution provides a quantitative basis for making this determination  Low budget = high probability of overrun  High budget = low probability of overrun : Basic Principles of Probabilistic Cost

68 Some More Parts to using Risk+ Just having the pictures is necessary, but knowing what they mean is required. Making changes to the IMS to increase the Probability of Program Success is the primary outcome from Monte Carlo Simulation.

Without Integrating $, Time, and TPM you’re driving in the rearview mirror69 Technical Performance (TPM)

Risk Management Demands a Well Defined Process

Statistics of a Triangle Distribution71 50% of all possible values are under this area of the curve. This is the definition of the median Minimum Maximum 1000 hrs 6830 hrs Mode = 2000 hrs Mean = 3879 hrs Median = 3415 hrs Basic Statistics

TPM Trends & Responses directly impact risk and credibility of the IMS72 Design ModelROM in Proposal Detailed Design Model Bench Scale Model Measurement Technical Performance Measure 28kg Prototype Measurement Vehicle Weight Flight 1st Article 26kg 25kg 23kg CA SFR SRR PDR CDR TRR Dr. Falk Chart – modified

Not A Mitigation Plan Mitigation is too late, the risk has turned into an issue. The money has been spent, and the time has passed.73

Ordinal versus Cardinal74 Ordinal Cardinal A variable is ordinally measurable A variable is cardinally measurable if ranking is possible for values of if a given interval between the variable. For example, a gold measures has a consistent meaning, medal reflects superior performance i.e., if the measure corresponds to to a silver or bronze medal in the points along a straight line. For Olympics, or you may prefer French example, height, output, and income toast to waffles, and waffles to oat are cardinally measurable. bran muffins. All variables that are cardinally measurable are also ordinally measurable, although the reverse may not be true.

Correcting Ordinal Risk Scales75  Classify and calibrate risk ranking in units meaningful to the decision makers  Risk rank 1, 2, 3, 4, is NOT sufficient  The Risk Rank must have a measurable value connected to the actual behavior of the system being assessed  Calibration coefficients between ordinal probability and consequences should also be used.  Ordinal analysis assumes ordering of the risks.  Cardinal analysis provides objective measures of probability and consequential impact.

Level Likelihood ValueNever multiply Likelihood E E Near Certainty E ≥ 90%by outcome. They are not“numbers,” they a D Highly Likely 74% ≤ D ≤ 90% Dprobability distributions. C Likely 40% ≤ C ≤ 60% COnly convolution is B Low Likelihood 20% ≤ B ≤ 40% Bpossible A Not Likely A ≤ 20% AThese are Cardinal measures of probability of occurrence and A B C D Econsequential impactLevel Technical Performance Schedule Cost Minimal or no consequence to A Minimal or no impact Minimal or no impact technical performance. Budget increase or unit Minor reduction in technical B Able to meet key dates production cost increases. performance or supportability. < (1% of Budget) Moderate reduction in Minor schedule slip. Budget increase or unit technical performance or Able to meet key C production cost increase supportability with limited milestones with no < (5% of Budget) impact on program objectives. schedule float. Significant degradation in Budget increase or unit technical performance or Program critical path D production cost increase major shortfall in affected < (10% of Budget) supportability. Cannot meet key Exceeds budget increase or Severe degradation in technical E program milestones. unit production cost performance. 76 Slip > X months threshold

Example of Ordinal Probability Complexity Scale†77 Definition of the Ordinal Scale Ranking Scale Level Greater than 20% of the interface design has been E altered because of modifications to the ICD’s. Greater than 15% but less than 20% of the interface design has been altered because of modifications of the D ICD’s. Greater than 10% but less than 15% of the interface design has been altered because of modifications of the C ICD’s. At least 5% but less than 10% of the interface design has B been altered because of modifications of the ICD’s. At least 5% of the interface design has been altered A because of modifications of the ICD’s. † Effective Risk Management: Some Keys to Success, Edmund Conrow, AIAA Press, 2003

A “real” risk Ordinal Ranking Table78Risk Percent Variance Interpretation of Risk RankingRank Normal business, technical & manufacturing A – 5% ≤ A ≤ 10% processes are applied Normal business & technical processes are B – 5% ≤ B≤ 15% applied; new or innovative manufacturing processes Flight software development & certification C – 5% ≤ C ≤ 35% processes Build & qualification of flight components, D – 10% ≤ D ≤ 25% subsystems & systems E – 10% ≤ E ≤ 35% Flight software qualification F – 5% ≤ F ≤ 175% ISS thermal vacuum acceptance testing

Project Train Wrecks Occur When There is… Inattention to budgetary responsibilities Work authorizations that are not always followed Issues with Budget and data reconciliation Lack of an integrated management system Baseline fluctuations and frequent replanning  Untimely and unrealistic Latest Revised Current period and retroactive Estimates (LRE) changes  Progress not monitored in a regular and Improper use of management consistent manner reserve  Lack of vertical and horizontal traceability EV techniques that do not cost and schedule data for corrective action reflect actual performance  Lack of internal surveillance and controls Lack of predictive variance  Managerial actions not demonstrated using analysis Earned Value 79

Our Final Check List80  Set up the Risk+ fields, flags, views, and tables for the program standard IMS.  Build an IMS that passes the DCMA 14 Point Assessment with all GREEN.  Build the Ordinal Risk Ranking table for the various risk categories on the program.  Assign risk ranking to each activities in the IMS, with the variances defined in the Ordinal Table.  Run Risk+ to see the confidence in the deliverables.  Develop the needed schedule margin to protect the delivery to at least the 80% confidence level.

Advice from the school of hard knocks81  Put margin in front of critical deliverables.  Build a margin burn down chart and allocate schedule margin just like you do MR for the PMB.  This real world advice is counter to the current DCMA guidance.

83 Putting This New Knowledge To Work

Managing margin is what Risk+ is all about84 CP Total Float Float Erosion: Critical Path Time Usage Acceptable Rate of Float Erosion Linear (CP Total Float ) 100 80 Time Now October 31, 2005 Critical Path - Time Reserve 60 40 Spacecraft Contract Delivery December 10, 2007 20 0 -20 -40

How much margin do we need?85 The Missing Link: Schedule Margin Management, Rick Price, PS–10, PMI–CPM EVM World 2008

Deterministic versus Probabilistic86 Baseline Plan Sep 2011 Oct 2011 Current Plan with risks is the Ready deterministic schedule Nov 2001 Early Plan Margin Dec 2011 Launch 20% Risk Period Margin Jan 2012 Mean Current Plan with risks is the Feb 2012 stochastic schedule The probability distribution can 80% Mar 2012 vary as a Missed function of time Launch ATLO Period CDR PDR FRR SRR Apr 2012

88 References

References89  “Protecting Earned Value with Schedule Margin,” http://www.prochain.com/pm/articles/ProtectingEVSchedules.pdf  Depicting Schedule Margin in the Integrated Master Schedule, http://www.ndia.org/Divisions/Divisions/Procurement/Documents/PMSCommittee/C ommitteeDocuments/WhitePapers/NDIAScheduleMarginWhitePaperFinal- 2010(2).pdf  Effective Risk Management: Some Keys to Success, Second Edition, Edmund Conrow, AIAA Press.  How to Lie with Statistics, Darrell Huff, Norton, 1954 (Available in paper back at any good book store)  DID DI–MGMT–81650 “A management method for accommodating schedule contingencies. It is a designated buffer and shall be identified separately and considered part of the baseline.

References90  Interfacing Risk and Earned Value Management, Association for Project Management, 150 West Wycombe Road, High Wycombe, Buckinghamshire, HP12 3AE, United Kingdom.  Practice Standard for Earned Value Management, Second Edition, Project Management Institute, 2011.  Effective Opportunity Management for Projects, David Hillson, Taylor and Francis, 2004.  Measuring Time: Improving Project Performance Using Earned Value, Mario Vanhoucke, Springer, 2009.  Performance Based Earned Value, Paul Solomon and Ralph Young, Wiley, 2007.  Effective Risk Management: Some Keys to Success, Edmund Conrow, AIAA Press, 2003.

Niwot Ridge LLC (: 303.241.9633 4347 Pebble Beach Drive -: glen.alleman@niwotridge.com Niwot, Colorado 8050391

Risk management using risk+ (v5)

by Glen Alleman on Jul 12, 2012

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 1

Statistics

Risk management using risk+ (v5) Presentation Transcript