0
Programmatic Risk Management Work (Handbook)    Programmatic Risk Management:    A “not so simple” introduction to the    ...
AgendaDuration      Topic 20 Minutes Risk Management in Five Easy Pieces 15 Minutes Basic Statistics for programmatic risk...
When we say “Risk Management”   What do we really mean?
Five Easy Pieces†:The Essentials ofManagingProgrammatic RiskManaging the risk to cost, schedule, and technical performance...
Hope is Not a Strategy  When General Custer was completely surrounded,  his chief scout asked, “General whats our strategy...
No Single Point Estimate can be correct without            knowing the varianceWhen estimating                        Sin...
Without Integrating $, Time, and TPMyou’re driving in the rearview mirror                      Technical                 P...
Without a model for risk management, you’re driving in the dark with                       the headlights turn offThe Risk...
Risk Communication is …An interactive process of exchange ofinformation and opinion amongindividuals, groups, and institut...
Basic Statistics for ProgrammaticRisk ManagementSince all point estimates are wrong, statistical estimates will be neededt...
Uncertainty and Risk are not the samething – don’t confuse them Uncertainty stems from                    Risk stems fro...
There are 2 types of Uncertaintyencountered in cost and schedule Static uncertainty is natural variation and  foreseen ri...
The Multiple Sources of Schedule Uncertainty   and Sorting Them Out is the Role of Planning Unknown interactions drive  u...
Statistics at a Glance Probability distribution – A          Bias –The expected deviation of the  function that describe...
Statistics Versus Probability              In building a risk tolerant               schedule, we’re interested in the   ...
Each path and each task along that path has a           probability distribution Any path could be critical depending on ...
Probability Distribution Functions are the Life           Blood of good planning Probability of  occurrence as a  functio...
Statistics of a Triangle DistributionTriangle                                  50% of all possible values are underdistrib...
Basics of Monte Carlo SimulationFar better an approximate answer to the right question, which is oftenvague, than an exact...
Monte Carlo Simulation Yes Monte Carlo is named after the  country full of casinos located on  the French Rivera Advanta...
First let’s be convinced that PERT haslimited usefulness The original paper (Malcolm 1959) states   – The method is “the ...
Critical Path and Mostly Likelies Critical Path’s are Deterministic – At least one path exists through   the network – Th...
Deterministic PERT Uses Three Point           Estimates In A Static Manner Durations are defined as three point estimates...
Foundation of Monte Carlo Theory George Louis Leclerc, Comte de Buffon, asked what was the probability that the needle wou...
Mechanics of Risk+ integrated withMicrosoft ProjectAny credible schedule is a credible model of its dynamic behavior. This...
The Simplest Risk+ elements     Task to “watch”         Most Likely     Distribution       (Number3)             (Duration...
The output of Risk+                               Date: 9/26/2005 2:14:02 PM                                          Comp...
A Well Formed Risk+ ScheduleFor Risk+ to provide useful information, the underlying schedule mustbe well formed on some si...
A Well formed Risk+ Schedule A good critical path network – No constraint dates – Lowest level tasks have predecessors an...
Analyzing the Risk+ Simulation Risk+ generates one or more of  the following outputs: – Earliest, expected, and latest   ...
Programmatic Risk RankingThe variance in task duration must be defined in some systematic way.Capturing three point values...
Thinking about risk ranking These classifications can be used to avoid asking the “3  point” question for each task This...
Steps in characterizing uncertainty Use an “envelope” method to characterize the minimum,  maximum and “most likely” Fit...
Sobering observations about 3 pointestimates when asking engineers In 1979, Tversky and Kahneman proposed an  alternative...
Building a Credible ScheduleA credible schedule contains a well formed network, explicit riskmitigations, proper margin fo...
Good schedules have a contingency plans The schedule contingency  needed to make the plan credible  can be derived from t...
Schedule quality and accuracy Accuracy range    – Similar for each estimate class   Consistent with estimate    – Level ...
Technical Performance Measures Technical Performance Measures are one method of showing risk by  done – Specific actions ...
The Monte Carlo Process starts with the 3 point                 estimates                 Estimates of the task duration ...
Expert Judgment is required to build a Risk             Management approach                  Expert judgment is typically...
Guiding the Risk Factor Process requires     careful weighting of each level of risk For tasks marked “Low” a reasonable ...
Assume now we have a well formed schedule – now                     what?                    With all the “bone head” ele...
We’re really after the management of schedule          margin as part of planning  Plan the risk alternatives that       ...
Successful margin management requires the        reuse of unused durations Programmatic Margin is added between          ...
Simulation Considerations Schedule logic and constraints  – Simplify logic – model only paths which, by    inspection, ma...
The contents of the schedule   Constraints   Lead/Lag   Task relationships   Durations   Network topology            ...
Simulation Considerations Selection of Probability Distributions   – Develop schedule simulation inputs concurrently     ...
Sensitivity Analysis describes whichtasks drive the completion times Concentrates on inputs most likely to  improve quali...
What we get in the end is a Credible   Model of the scheduleAll models are wrong. Somemodels are useful.– George Box (1919...
ConclusionAt this point there is too much information. Processing this informationwill take time, patience, and most of al...
Conclusions Project schedule status must be  assessed in terms of a critical path  through the schedule network Because ...
Conclusions Quality (accuracy) is measured at the  end points of achieved confidence  interval (suggest 80% level) Simul...
Conclusions Cost and schedule estimates are made up of many  independent elements.   – When each element is planned as be...
What are we really after in the end? As the program  proceeds so  does:   – Increasing     accuracy   – Reduced     sched...
Points to remember Good project management is good risk  management Risk management is how adults manage projects The o...
Usage is needed before understanding is    acquiredHere and elsewhere, we shall notobtain the best insights into thingsunt...
The End                                A planning algorithm from                                Aristotle’s De Motu Animal...
Resources1. “The Parameters of the Classical PERT: An Assessment of its Success,”   Rafael Herrerias Pleguezuelo,   http:/...
Resources8. “Teaching SuPERT,” Kenneth R. MacLeod and Paul F. Petersen,    Proceedings of the Decision Sciences 2003 Annua...
Resources14. “Schedule Risk Analysis: Why It Is Important and How to Do It, “Stephen A.    Book, Proceedings of the Ground...
Resources20. Project Risk Management Resource List, NASA Headquarters Library,    http://www.hq.nasa.gov/office/hqlibrary/...
Resources25. “Improved Three Point Approximation To Distribution Functions For    Application In Financial Decision Analys...
Resources30. “Triangular Approximations for Continuous Random Variables in Risk    Analysis,” David G. Johnson, The Busine...
Resources35. Probabilistic Risk Assessment Procedures Guide for NASA Managers and    Practitioners, Office of Safety and M...
Resources40. “Quantify Risk to Manage Cost and Schedule,” Fred Raymond, Acquisition    Review Quarterly, Spring 1999, pp. ...
Resources46. “Three Point Approximations for Continuous Random Variables,” Donald    Keefer and Samuel Bodily, Management ...
Resources51. “Experiences in Improving Risk Management Processes Using the Concepts    of the Riskit Method,” Jyrki Konito...
Resources
Niwot Ridge LLC    4347 Pebble Beach Drive     Niwot, Colorado 80503Performance Based Managementsm        Integrated Maste...
Upcoming SlideShare
Loading in...5
×

Programmatic risk management workshop (handbook)

1,761

Published on

Published in: Economy & Finance, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,761
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
41
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Programmatic risk management workshop (handbook)"

  1. 1. Programmatic Risk Management Work (Handbook) Programmatic Risk Management: A “not so simple” introduction to the complex but critical process of building a “credible” schedule Program Planning and Controls Workshop, Denver, Colorado October 6th and October 14th 2008
  2. 2. AgendaDuration Topic 20 Minutes Risk Management in Five Easy Pieces 15 Minutes Basic Statistics for programmatic risk management 15 Minutes Monte Carlo Simulation (MCS) theory 20 Minutes Mechanics of MSFT Project and Risk+ 15 Minutes Programmatic Risk Ranking 15 Minutes Building a Credible schedule 20 Minutes Conclusion120 Minutes
  3. 3. When we say “Risk Management” What do we really mean?
  4. 4. Five Easy Pieces†:The Essentials ofManagingProgrammatic RiskManaging the risk to cost, schedule, and technical performance is thebasis of a successful project management method.† With apologies to Carole Eastman and Bob Rafelson for their 1970 film staring Jack Nicholson Risk in Five Easy Pieces
  5. 5. Hope is Not a Strategy When General Custer was completely surrounded, his chief scout asked, “General whats our strategy?” Custer replied, “The first thing we need to do is make a note to ourselves – never get in this situation again.” Hope is not a strategy! A Strategy is the plan to successfully complete the project If the project’s success factors, the processes that deliver them, the alternatives when they fail, and the measurement of this success are not defined in meaningful ways for both the customer and managers of the project – Hope is the only strategy left. Risk in Five Easy Pieces
  6. 6. No Single Point Estimate can be correct without knowing the varianceWhen estimating  Single Point Estimates use sample data tocost and duration calculate a single value (a statistic) that serves asfor planning a "best guess" for an unknown (fixed or random)purposes using population parameterPoint Estimates  Bayesian Inference is a statistical inferenceresults in the where evidence or observations are used to inferleast likely result. the probability that a hypothesis may be trueA result with a  Identifying underlying statistical behavior of the50/50 chance ofbeing true. cost and schedule parameters of the project is the first step in forecasting future behavior  Without this information and the model in which it is used any statements about cost, schedule and completion dates are a 50/50 guesses Risk in Five Easy Pieces
  7. 7. Without Integrating $, Time, and TPMyou’re driving in the rearview mirror Technical Performance (TPM)Addressing customer satisfaction means incorporatingproduct requirements and planned quality into thePerformance Measurement Baseline to assure the trueperformance of the project is made visible. Risk in Five Easy Pieces
  8. 8. Without a model for risk management, you’re driving in the dark with the headlights turn offThe RiskManagementprocess to theright is used bythe US DOD anddiffers from thePMI approach inhow theprocesses areasare arranged.The key is tounderstand therelationshipsbetween theseareas. Risk Management means using a proven risk management process, adapting this to the project environment, and using this process for everyday decision making. Risk in Five Easy Pieces
  9. 9. Risk Communication is …An interactive process of exchange ofinformation and opinion amongindividuals, groups, and institutions;often involving multiple messages aboutthe nature of risk or expressingconcerns, opinions, or reactions to riskmessages or to legal or institutionalarrangements for risk management. Bad news is not wine. It does not improve with age — Colin Powell Risk in Five Easy Pieces
  10. 10. Basic Statistics for ProgrammaticRisk ManagementSince all point estimates are wrong, statistical estimates will be neededto construct a credible cost and schedule model Basic Statistics
  11. 11. Uncertainty and Risk are not the samething – don’t confuse them Uncertainty stems from  Risk stems from known unknown probability probability distributions distributions – Cost estimating methodology risk – Requirements change impacts resulting from improper models of – Budget Perturbations cost – Re–work, and re–test phenomena – Cost factors such as inflation, labor rates, labor rate burdens, – Contractual arrangements etc (contract type, prime/sub relationships, etc) – Configuration risk (variation in the technical inputs) – Potential for disaster (labor troubles, shuttle loss, satellite – Schedule and technical risk “falls over”, war, hurricanes, etc.) coupling – Probability that if a discrete event – Correlation between risk occurs it will invoke a project distributions delay Basic Statistics
  12. 12. There are 2 types of Uncertaintyencountered in cost and schedule Static uncertainty is natural variation and foreseen risks – Uncertainty about the value of a parameter Dynamic uncertainty is unforeseen uncertainty and “chaos” – Stochastic changes in the underlying environment – System time delays, interactions between the network elements, positive and negative feedback loops – Internal dependencies Basic Statistics
  13. 13. The Multiple Sources of Schedule Uncertainty and Sorting Them Out is the Role of Planning Unknown interactions drive uncertainty Dynamic uncertainty can be addressed by flexibility in the schedule – On ramps – Off ramps – Alternative paths – Schedule “crashing” opportunities Modeling of this dynamic uncertainty requires simulation rather than static PERT based path assessment – Changes in critical path are dependent on time and state of the network – The result is a stochastic network Basic Statistics
  14. 14. Statistics at a Glance Probability distribution – A  Bias –The expected deviation of the function that describes the expected value of a statistical probabilities of possible outcomes estimate from the quantity it in a "sample space.” estimates. Random variable – variable a  Correlation – A measure of the joint function of the result of a impact of two variables upon each statistical experiment in which other that reflects the simultaneous each outcome has a definite variation of quantities. probability of occurrence.  Percentile – A value on a scale of Determinism – a theory that 100 indicating the percent of a phenomena are causally distribution that is equal to or determined by preceding events or below it. natural laws.  Monte Carlo sampling – A modeling Standard deviation (sigma value) – technique that employs random An index that characterizes the sampling to simulate a population dispersion among the values in a being studied. population. Basic Statistics
  15. 15. Statistics Versus Probability  In building a risk tolerant schedule, we’re interested in the probability of a successful outcome – “What is the probability of making a desired completion date?”  But the underlying statistics of the tasks influence this probability  The statistics of the tasks, their arrangement in a network of tasks and correlation define how this probability based estimated developed. Basic Statistics
  16. 16. Each path and each task along that path has a probability distribution Any path could be critical depending on the convolution of the underlying task completion time probability distribution functions The independence or dependency of each task with others in the network, greatly influences the outcome of the total project duration Understanding this dependence is critical to assessing the credibility of the plan as well as the total completion time of that plan Basic Statistics
  17. 17. Probability Distribution Functions are the Life Blood of good planning Probability of occurrence as a function of the number of samples “The number of times a task duration appears in a Monte Carlo simulation” Basic Statistics
  18. 18. Statistics of a Triangle DistributionTriangle 50% of all possible values are underdistributions are this area of the curve. This is theuseful when there definition of the medianis limitedinformation aboutthe characteristicsof the randomvariables are allthat is available.This is common inproject cost and Minimum Maximumschedule estimates. 1000 hrs 6830 hrs Mode = 2000 hrs Mean = 3879 hrs Median = 3415 hrs Basic Statistics
  19. 19. Basics of Monte Carlo SimulationFar better an approximate answer to the right question, which is oftenvague, than an exact answer to the wrong question, which can alwaysbe made precise. — John W. Tukey, 1962 Basics of Monte Carlo
  20. 20. Monte Carlo Simulation Yes Monte Carlo is named after the country full of casinos located on the French Rivera Advantages of Monte Carlo over PERT is that Monte Carlo… – Examines all paths, not just the critical path – Provides an accurate (true) estimate of completion • Overall duration distribution • Confidence interval (accuracy range) – Sensitivity analysis of interacting tasks – Varied activity distribution types – not restricted to Beta – Schedule logic can include branching – both probabilistic and conditional – When resource loaded schedules are used – provides integrated cost and schedule probabilistic model Basics of Monte Carlo
  21. 21. First let’s be convinced that PERT haslimited usefulness The original paper (Malcolm 1959) states – The method is “the best that could be done in a real situation within tight time constraints.” – The time constraint was One Month The PERT time made the assumption that the standard deviation was about 1/6 of the range (b– a), resulting in the PERT formula. It has been shown that the PERT mean and standard deviation formulas are poor approximations for most Beta distributions (Keefer 1983 and Keefer 1993). – Errors up to 40% are possible for the PERT mean – Errors up to 550% are possible for the PERT standard deviation Basics of Monte Carlo
  22. 22. Critical Path and Mostly Likelies Critical Path’s are Deterministic – At least one path exists through the network – The critical path is identified by adding the “single point” estimates – The critical predicts the completion date only if everything goes according to plan (we all know this of course) Schedule execution is Probabilistic – There is a likelihood that some durations will comprise a path that is off the critical path – The single number for the estimate – the “single point estimate” is in fact a most likely estimate – The completion date is not the most likely date, but is a confidence interval in the probability distribution function resulting from the convolution of all the distributions along all the paths to the completion of the project Basics of Monte Carlo
  23. 23. Deterministic PERT Uses Three Point Estimates In A Static Manner Durations are defined as three point estimates – These estimates are very subjective if captured individually by asking… – “What is the Minimum, Maximum, and Most Likely” Critical path is defined from these estimates is the algebraic addition of three point estimates Project duration is based on the algebraic addition of the times along the critical path This approach has some serious problems from the outset – Durations must be independent – Most likely is not the same as the average Basics of Monte Carlo
  24. 24. Foundation of Monte Carlo Theory George Louis Leclerc, Comte de Buffon, asked what was the probability that the needle would fall across one of the lines, marked in green. That outcome occurs only if: A  l sin Basics of Monte Carlo
  25. 25. Mechanics of Risk+ integrated withMicrosoft ProjectAny credible schedule is a credible model of its dynamic behavior. Thisstarts with a Monte Carlo model of the schedule’s network of tasks Mechanics of Risk+
  26. 26. The Simplest Risk+ elements Task to “watch” Most Likely Distribution (Number3) (Duration3) (Number1) Optimistic Pessimistic (Duration1) (Duration2) Mechanics of Risk+
  27. 27. The output of Risk+ Date: 9/26/2005 2:14:02 PM Completion Std Deviation: 4.83 days Samples: 500 95% Confidence Interval: 0.42 days Task to “watch” Unique ID: 10 Each bar represents 2 days Name: Task 10 0.16 1.0 Completion Probability Table Cumulative Probability 0.9 0.14 Prob Date Prob Date 0.8 0.12 0.05 2/17/06 0.55 3/1/06 0.7 Frequency 0.10 2/21/06 0.60 3/2/06 0.10 0.6 0.15 2/22/06 0.65 3/3/06 0.08 0.5 0.20 2/22/06 0.70 3/3/06 0.4 0.25 2/23/06 0.75 3/6/06 0.06 0.3 0.30 2/24/06 0.80 3/7/06 80% confidence 0.04 0.35 2/27/06 0.85 3/8/06 0.2 0.40 2/27/06 0.90 3/9/06 that task will 0.02 0.1 0.45 2/28/06 0.95 3/13/06 complete by 2/10/06 3/1/06 3/17/06 0.50 3/1/06 1.00 3/17/06 Completion Date 3/7/06 The height of each box indicates  The standard deviation of the how often the project complete in a completion date and the 95% given interval during the run confidence interval of the expected The S–Curve shows the cumulative completion date are in the same probability of completing on or units as the “most likely remaining before a given date. duration” field in the schedule Mechanics of Risk+
  28. 28. A Well Formed Risk+ ScheduleFor Risk+ to provide useful information, the underlying schedule mustbe well formed on some simple way. Mechanics of Risk+
  29. 29. A Well formed Risk+ Schedule A good critical path network – No constraint dates – Lowest level tasks have predecessors and successors – 80% of relationships are finish to start Identify risk tasks – These are “reporting tasks” – Identify the preview task to watch during simulation runs Defining the probability distribution profile for each task – Bulk assignment is an easy way to start – A – F ranking is another approach – Individual risk profile assignments is best but tedious Mechanics of Risk+
  30. 30. Analyzing the Risk+ Simulation Risk+ generates one or more of the following outputs: – Earliest, expected, and latest completion date for each reporting task – Graphical and tabular displays of the completion date distribution for each reporting task – The standard deviation and confidence interval for the completion date distribution for each reporting task – The criticality index (percentage of time on the critical path) for each task – The duration mean and standard deviation for each task – Minimum, expected, and maximum cost for the total project – Graphical and tabular displays of cost distribution for the total project – The standard deviation and confidence interval for cost at the total project level Mechanics of Risk+
  31. 31. Programmatic Risk RankingThe variance in task duration must be defined in some systematic way.Capturing three point values is the least desirable. Programmatic Risk Ranking
  32. 32. Thinking about risk ranking These classifications can be used to avoid asking the “3 point” question for each task This information will be maintained in the IMS When updates are made the percentage change can be applied across all tasks Classification Uncertainty OverrunA Routine, been done before Low 0% to 2%B Routine, but possible difficulties Medium to Low 2% to 5%C Development, with little technical difficulty Medium 5% to 10%D Development, but some technical difficulty Medium High 10% to 15%E Significant effort, technical challenge High 15% to 25%F No experience in this area Very High 25% to 50% Programmatic Risk Ranking
  33. 33. Steps in characterizing uncertainty Use an “envelope” method to characterize the minimum, maximum and “most likely” Fit this data to a statistical distribution Use conservative assumptions Apply greater uncertainty to less mature technologies Confirm analysis matches intuition Remember Sir Francis Bacon’s quote about beginning with uncertainty and ending with certainty. If we start with a what we think is a valid number we will tend to continue with that valid number. When in fact we should speak only in terms of confidence intervals and probabilities of success. Programmatic Risk Ranking
  34. 34. Sobering observations about 3 pointestimates when asking engineers In 1979, Tversky and Kahneman proposed an alternative to Utility theory. Prospect theory asserts that people make predictably irrational decisions. The way that a choice of decisions is presented can sway a person to choose the less rational decision from a set of options. Once a problem is clearly and reasonably presented, rarely does a person think outside the bounds of the frame. Source: – “The Causes of Risk Taking By Project Managers,” Proceedings of the Project Management Institute Annual Seminars & Symposium November 1–10, 2001 • Nashville, Tennessee – Tversky, Amos, and Daniel Kahneman. 1981. The Framing of Decisions and the Psychology of Choice. Science 211 (January 30): 453–458 Programmatic Risk Ranking
  35. 35. Building a Credible ScheduleA credible schedule contains a well formed network, explicit riskmitigations, proper margin for these risks, and a clear and concisecritical path(s). All of this is prologue to analyzing the schedule. Building a Credible Schedule
  36. 36. Good schedules have a contingency plans The schedule contingency needed to make the plan credible can be derived from the Risk+ analysis The schedule contingency is the Is This Our amount of time added (or Contingency subtracted) from the baseline Plan ? schedule necessary to achieve the desired probability of an under run or over run. The schedule contingency can be determined through – Monte Carlo simulations (Risk+) – Best judgment from previous experience – Percentage factors based on historical experience – Correlation analysis for dependency impacts Building a Credible Schedule
  37. 37. Schedule quality and accuracy Accuracy range – Similar for each estimate class Consistent with estimate – Level of project definition – Purpose – Preparation effort Monte Carlo simulation – Analysis of results shows quality attained versus the quality sought (expected accuracy ranges) Achieving specified accuracy requirements – Select value at end points of confidence interval – Calculate percentages from base schedule completion date, including the contingency Building a Credible Schedule
  38. 38. Technical Performance Measures Technical Performance Measures are one method of showing risk by done – Specific actions taken in the IMS to move the compliance forward toward the goal Activities that assessing the increasing compliance to the technical performance measure can be show in the IMS – These can be Accomplishment Criteria Building a Credible Schedule
  39. 39. The Monte Carlo Process starts with the 3 point estimates  Estimates of the task duration are still needed, just like they are in PERTThese threepoint estimates – Three point estimates could be usedare not the PERT – But risk ranking and algorithmic generation of theones. “spreads” is a better approachThey are derivedfrom the ordinal  Duration estimates must be parametric rather thanrisk ranking numeric valuesprocess.This allows them – A geometric scale of parametric risk is one approach to be “calibrated” Branching probabilities need to be definedfor the domain,correlated with – Conditional paths through the schedule can be evaluatedthe technical risk using Monte Carlo toolsmodel. – This also demonstrate explicit risk mitigation planning to answer the question “what if this happens?” Building a Credible Schedule
  40. 40. Expert Judgment is required to build a Risk Management approach  Expert judgment is typically the basis of cost and scheduleBuilding the estimatesvariance – Expert judgment is usually the weakest area of process andvalues for the quantificationordinal risk – Translating from English (SOW) to mathematics (probabilisticrank is a risk model) is usually inconsistent at best and erroneous attechnical worstprocess,  One approachrequiringengineering – Plan for the “best case” and preclude a self–fulfilling prophesyjudgment. – Budget for the “most likely” and recognize risks and uncertainties – Protect for the “worst case” and acknowledge the conceivable in the risk mitigation plan  The credibility of the “best case” estimates if crucial to the success of this approach Building a Credible Schedule
  41. 41. Guiding the Risk Factor Process requires careful weighting of each level of risk For tasks marked “Low” a reasonable Min Most Max approach is to score the maximum 10% Likely greater than the minimum. Low 1.0 1.04 1.10 The “Most Likely” is then scored as a Low+ 1.0 1.06 1.15 geometric progression for the remaining categories with a common ratio of 1.5 Moderate 1.0 1.09 1.24 Tasks marked “Very High” are bound at Moderate+ 1.0 1.14 1.36 200% of minimum. High 1.0 1.20 1.55 – No viable project manager would like a task High+ 1.0 1.30 1.85 grow to three times the planned duration without intervention Very High 1.0 1.46 2.30 The geometric progress is somewhat Very High+ 1.0 1.68 3.00 arbitrary but it should be used instead of a linear progression Building a Credible Schedule
  42. 42. Assume now we have a well formed schedule – now what?  With all the “bone head” elementsFor the role of removed, we can say we have aPP&C is tomove “reporting well formed schedulepastperformance” to  But the real role of Planning is to“forecastingfuture forecast the future, provideperformance” itmust break the alternative Plan’s for this forecastmold of usingstatic models of and actively engage all thecost and participants in the projects in theschedule Planning Process Building a Credible Schedule
  43. 43. We’re really after the management of schedule margin as part of planning  Plan the risk alternatives that  Assign duration and resource “might” be needed estimates to both branches – Each mitigation has a Plan B  Turn off for alternative for a branch “success” path assessment – Keep alternatives as simple as  Turn off primary for a “failure” path possible (maybe one task) assessment  Assess probability of the alternative occurring Plan B30% Probability of failure 80% Confidence for completion with current margin70% Probability of success Plan A Current Margin Future Margin Duration of Plan B  Plan A + Margin Building a Credible Schedule
  44. 44. Successful margin management requires the reuse of unused durations Programmatic Margin is added between  Margin that is not used in the IMS for risk Development, Production and Integration mitigation will be moved to the next & Test phases sequence of risk alternatives Risk Margin is added to the IMS where – This enables us to buy back schedule margin risk alternatives are identified for activities further downstream – This enables us to control the ripple effect of schedule shifts on Margin activities Downstream Duration of Plan B < Plan A + Margin Activities shifted to Plan B left 2 days Plan B 3 Days Margin Used Plan A 5 Days Margin First Identified Risk Alternative in IMS Plan A 5 Days Margin Second Identified Risk 2 days will be added to this margin task Alternative in IMS to bring schedule back on track Building a Credible Schedule
  45. 45. Simulation Considerations Schedule logic and constraints – Simplify logic – model only paths which, by inspection, may have a significant bearing on the final result – Correlate similar activities – No open ends – Use only finish–to–start relationships with no lags – Model relationships other than finish–to–start as activities with base durations equal to the lag value – Eliminate all date constraints – Consider using branching for known alternatives Building a Credible Schedule
  46. 46. The contents of the schedule Constraints Lead/Lag Task relationships Durations Network topology Building a Credible Schedule
  47. 47. Simulation Considerations Selection of Probability Distributions – Develop schedule simulation inputs concurrently with the cost estimate • Early in process – use same subject matter experts • Convert confidence intervals into probability duration distributions – Number of distributions vary depending on software – Difficult to develop inputs required for distributions – Beta and Lognormal better than triangular; avoid exclusive use of Normal distribution Building a Credible Schedule
  48. 48. Sensitivity Analysis describes whichtasks drive the completion times Concentrates on inputs most likely to improve quality (accuracy) Identifies most promising opportunities where additional work will help to narrow input ranges Methods – Run multiple simulations – Use criticality index – “Tornado” or Pareto graph Building a Credible Schedule
  49. 49. What we get in the end is a Credible Model of the scheduleAll models are wrong. Somemodels are useful.– George Box (1919 – ) Concept generator from Ramon Lull’s Ars Magna (C. 1300) Building a Credible Schedule
  50. 50. ConclusionAt this point there is too much information. Processing this informationwill take time, patience, and most of all practice with the tools and theresults they produce. Conclusion
  51. 51. Conclusions Project schedule status must be assessed in terms of a critical path through the schedule network Because the actual durations of each task in the network are uncertain (they are random variables following a probability distribution function), the project schedule duration must be modeled statistically Conclusion
  52. 52. Conclusions Quality (accuracy) is measured at the end points of achieved confidence interval (suggest 80% level) Simulation results depend on: – Accuracy and care taken with base schedule logic – Use of subject matter experts to establish inputs – Selection of appropriate distribution types – Through analysis of multiple critical paths – Understanding which activities and paths have the greatest potential impact Conclusion
  53. 53. Conclusions Cost and schedule estimates are made up of many independent elements. – When each element is planned as best case – e.g. a probability of achievement of 10% – The probability of achieving best case for a two–element estimate is 1% – For three elements, 0.01% – For many elements, infinitesimal – In effect, it is zero. In the beginning no attempt should be made to distinguish between risk and uncertainty – Risk involves uncertainty but it is indeed more – For initial purposes it is unimportant – The effect is combined into one statistical factor called “risk,” which can be described by a single probability distribution function Conclusion
  54. 54. What are we really after in the end? As the program proceeds so does: – Increasing accuracy – Reduced schedule risk – Increasing visual confirmation Current Estimate Accuracy that success can be reached Conclusion
  55. 55. Points to remember Good project management is good risk management Risk management is how adults manage projects The only thing we manage is project risk Risks impact objectives Risks come from the decisions we make while trying to achieve the objectives Risks require a factual condition and have potential negative consequences that must be mitigated in the schedule Conclusion
  56. 56. Usage is needed before understanding is acquiredHere and elsewhere, we shall notobtain the best insights into thingsuntil we actually see them growingfrom the beginning.— Aristotle Conclusion
  57. 57. The End A planning algorithm from Aristotle’s De Motu Animalium c. 400 BCThis is actually the beginning, since building a risk tolerant, credible,robust schedule requires constant “execution” of the plan. Conclusion
  58. 58. Resources1. “The Parameters of the Classical PERT: An Assessment of its Success,” Rafael Herrerias Pleguezuelo, http://www.cyta.com.ar/biblioteca/bddoc/bdlibros/pert_van/PARAMETROS.PD F2. “Advanced Quantitative Schedule Risk Analysis,” David T. Hulett, Hulett & Associates, http://www.projectrisk.com/index.html3. “Schedule Risk Analysis Simplified,” David T. Hulett, Hulett & Associates, http://www.projectrisk.com/index.html4. “Project Risk Management: A Combined Analytical Hierarchy Process and Decision Tree Approach,” Prasanta Kumar Dey, Cost Engineering, Vol. 44, No. 3, March 2002.5. “Adding Probability to Your ‘Swiss Army Knife’,” John C. Goodpasture, Proceedings of the 30th Annual Project Management Institute 1999 Seminars and Symposium, October, 1999.6. “Modeling Uncertainty in Project Scheduling,” Patrick Leach, Proceedings of the 2005 Crystal Ball User Conference7. “Near Critical Paths Create Violations in the PERT Assumptions of Normality,” Frank Pokladnik and Robert Hill, University of Houston, Clear Lake, http://www.sbaer.uca.edu/research/dsi/2003/procs/237–4203.pdf Resources
  59. 59. Resources8. “Teaching SuPERT,” Kenneth R. MacLeod and Paul F. Petersen, Proceedings of the Decision Sciences 2003 Annual Meeting, Washington DC, http://www.sbaer.uca.edu/research/dsi/2003/by_track_paper.html9. “The Beginning of the Monte Carlo Method,” N. Metropolis, Los Alamos Science, Special Issue, 1987. http://www.fas.org/sgp/othergov/doe/lanl/pubs/00326866.pdf10. “Defining a Beta Distribution Function for Construction Simulation,” Javier Fente, Kraig Knutson, Cliff Schexnayder, Proceedings of the 1999 Winter Simulation Conference.11. “The Basics of Monte Carlo Simulation: A Tutorial,” S. Kandaswamy, Proceedings of the Project Management Institute Annual Seminars & Symposium, November, 2001.12. “The Mother of All Guesses: A User Friendly Guide to Statistical Estimation,” Francois Melese and David Rose, Armed Forces Comptroller, 1998, http://www.nps.navy.mil/drmi/graphics/StatGuide–web.pdf13. “Inverse Statistical Estimation via Order Statistics: A Resolution of the Ill– Posed Inverse problem of PERT Scheduling,” William F. Pickard, Inverse Problems 20, pp. 1565–1581, 2004 Resources
  60. 60. Resources14. “Schedule Risk Analysis: Why It Is Important and How to Do It, “Stephen A. Book, Proceedings of the Ground Systems Architecture Workshop (GSAW 2002), Aerospace Corporation, March 2002, http://sunset.usc.edu/GSAW/gsaw2002/s11a/book.pdf15. “Evaluation of the Risk Analysis and Cost Management (RACM) Model,” Matthew S. Goldberg, Institute for Defense Analysis, 1998. http://www.thedacs.com/topics/earnedvalue/racm.pdf16. “PERT Completion Times Revisited,” Fred E. Williams, School of Management, University of Michigan–Flint, July 2005, http://som.umflint.edu/yener/PERT%20Completion%20Revisited.htm17. “Overcoming Project Risk: Lessons from the PERIL Database,” Tom Hendrick , Program Manager, Hewlett Packard, 2003, http://www.failureproofprojects.com/Risky.pdf18. “The Heart of Risk Management: Teaching Project Teams to Combat Risk,” Bruce Chadbourne, 30th Annual Project Management Institute 1999 Seminara and Symposium, October 1999, http://www.risksig.com/Articles/pmi1999/rkalt01.pdf Resources
  61. 61. Resources20. Project Risk Management Resource List, NASA Headquarters Library, http://www.hq.nasa.gov/office/hqlibrary/ppm/ppm22.htm#art21. “Quantify Risk to Manage Cost and Schedule,” Fred Raymond, Acquisition Quarterly, Spring 1999, http://www.dau.mil/pubs/arq/99arq/raymond.pdf22. “Continuous Risk Management,” Cost Analysis Symposium, April 2005, http://www1.jsc.nasa.gov/bu2/conferences/NCAS2005/papers/5C_– _Cockrell_CRM_v1_0.ppt23. “A Novel Extension of the Triangular Distribution and its Parameter Estimation,” J. Rene van Dorp and Samuel Kotz, The Statistician 51(1), pp. 63 – 79, 2002. http://www.seas.gwu.edu/~dorpjr/Publications/JournalPapers/TheStatistician2 002.pdf24. “Distribution of Modeling Dependence Cause by Common Risk Factors,” J. Rene van Dorp, European Safety and Reliability 2003 Conference Proceedings, March 2003, http://www.seas.gwu.edu/~dorpjr/Publications/ConferenceProceedings/Esrel2 003.pdf Resources
  62. 62. Resources25. “Improved Three Point Approximation To Distribution Functions For Application In Financial Decision Analysis,” Michele E. Pfund, Jennifer E. McNeill, John W. Fowler and Gerald T. Mackulak, Department of Industrial Engineering, Arizona State University, Tempe, Arizona, http://www.eas.asu.edu/ie/workingpaper/pdf/cdf_estimation_submission.pdf26. “Analysis Of Resource–constrained Stochastic Project Networks Using Discrete–event Simulation,” Sucharith Vanguri, Masters Thesis, Mississippi State University, May 2005, http://sun.library.msstate.edu/ETD– db/theses/available/etd–04072005– 123743/restricted/SucharithVanguriThesis.pdf27. “Integrated Cost / Schedule Risk Analysis,” David T. Hulett and Bill Campbell, Fifth European Project Management Conference, June 2002.28. “Risk Interrelation Management – Controlling the Snowball Effect,” Olli Kuismanen, Tuomo Saari and Jussi Vähäkylä, Fifth European Project Management Conference, June 2002.29. The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century, David Salsburg, W. H. Freeman, 2001 Resources
  63. 63. Resources30. “Triangular Approximations for Continuous Random Variables in Risk Analysis,” David G. Johnson, The Business School, Loughborough University, Liecestershire.31. “Statistical Dependence through Common Risk Factors: With Applications in Uncertainty Analysis,” J. Rene van Dorp, European Journal of Operations Research, Volume 161(1), pp. 240–255.32. “Statistical Dependence in the risk analysis for Project Networks Using Monte Carlo Methods,” J. Rene van Dorp and M. R. Dufy, International Journal of Production Economics, 58, pp. 17–29, 1999. http://www.seas.gwu.edu/~dorpjr/Publications/JournalPapers/Prodecon1999.p df33. “Risk Analysis for Large Engineering Projects: Modeling Cost Uncertainty for Ship Production Activities,” M. R. Dufy and J. Rene van Dorp, Journal of Engineering Valuation and Cost Analysis, Volume 2. pp. 285–301, http://www.seas.gwu.edu/~dorpjr/Publications/JournalPapers/EVCA1999.pdf34. “Risk Based Decision Support techniques for Programs and Projects,” Barney Roberts and David Frost, Futron Risk Management Center of Excellence, http://www.futron.com/pdf/RBDSsupporttech.pdf Resources
  64. 64. Resources35. Probabilistic Risk Assessment Procedures Guide for NASA Managers and Practitioners, Office of Safety and Mission Assurance, April 2002. http://www.hq.nasa.gov/office/codeq/doctree/praguide.pdf36. “Project Planning: Improved Approach Incorporating Uncertainty,” Vahid Khodakarami, Norman Fenton, and Martin Neil, Track 15 EURAM2005: “Reconciling Uncertainty and Responsibility” European Academy of Management. http://www.dcs.qmw.ac.uk/~norman/papers/project_planning_khodakerami.pd f37. “A Distribution for Modeling Dependence Caused by Common Risk Factors,” J. Rene van Dorp, European Safety and Reliability 2003 Conference Proceedings, March 2003.38. “Probabilistic PERT,” Arthur Nadas, IBM Journal of Research and Development, 23(3), May 1979, pp. 339–347.39. “Ranked Nodes: A Simple and effective way to model qualitative in large– scale Bayesian Networks,” Norman Fenton and Martin Neil, Risk Assessment and Decision Analysis Research Group, Department of Computer Science, Queen Mary, University of London, February 21, 2005. Resources
  65. 65. Resources40. “Quantify Risk to Manage Cost and Schedule,” Fred Raymond, Acquisition Review Quarterly, Spring 1999, pp. 147–15441. “The Causes of Risk Taking by Project Managers,” Michael Wakshull, Proceedings of the Project Management Institute Annual Seminars & Symposium, November 2001.42. “Stochastic Project Duration Analysis Using PERT–Beta Distributions,” Ron Davis.43. “Triangular Approximation for Continuous Random Variables in Risk Analysis,” David G. Johnson, Decision Sciences Institute Proceedings 1998. http://www.sbaer.uca.edu/research/dsi/1998/Pdffiles/Papers/1114.pdf44. “The Cause of Risk Taking by Managers,” Michael N.Wakshull, Proceedings of the Project Management Institute Annual Seminars & Symposium November 1–10, 2001, Nashville Tennessee , http://www.risksig.com/Articles/pmi2001/21261.pdf45. “The Framing of Decisions and the Psychology of Choice,” Tversky, Amos, and Daniel Kahneman. 1981, Science 211 (January 30): 453–458, http://www.cs.umu.se/kurser/TDBC12/HT99/Tversky.html Resources
  66. 66. Resources46. “Three Point Approximations for Continuous Random Variables,” Donald Keefer and Samuel Bodily, Management Science, 29(5), pp. 595 – 609.47. “Better Estimation of PERT Activity Time Parameters,” Donald Keefer and William Verdini, Management Science, 39(9), pp. 1086 – 1091.48. “The Benefits of Integrated, Quantitative Risk Management,” Barney B. Roberts, Futron Corporation, 12th Annual International Symposium of the International Council on Systems Engineering, July 1–5, 2001, http://www.futron.com/pdf/benefits_QuantIRM.pdf49. “Sources of Schedule Risk in Complex Systems Development,” Tyson R. Browning, INCOSE Systems Engineering Journal, Volume 2, Issue 3, pp. 129 – 142, 14 September 1999, http://sbufaculty.tcu.edu/tbrowning/Publications/Browning%20(1999)–– SE%20Sch%20Risk%20Drivers.pdf50. “Sources of Performance Risk in Complex System Development,” Tyson R. Browning, 9th Annual International Symposium of INCOSE, June 1999, http://sbufaculty.tcu.edu/tbrowning/Publications/Browning%20(1999)–– INCOSE%20Perf%20Risk%20Drivers.pdf Resources
  67. 67. Resources51. “Experiences in Improving Risk Management Processes Using the Concepts of the Riskit Method,” Jyrki Konito, Gerhard Getto, and Dieter Landes, ACM SIGSOFT Software Engineering Notes , Proceedings of the 6th ACM SIGSOFT international symposium on Foundations of software engineering SIGSOFT 98/FSE-6, Volume 23 Issue 6, November 1998.52. “Anchoring and Adjustment in Software Estimation,” Jorge Aranda and Steve Easterbrook, Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering ESEC/FSE-1353. “The Monte Carlo Method,” W. F. Bauer, Journal of the Society of Industrial Mathematics, Volume 6, Number 4, December 1958, http://www.cs.fsu.edu/~mascagni/Bauer_1959_Journal_SIAM.pdf.54. “A Retrospective and Prospective Survey of the Monte Carlo Method,” John H. Molton, SIAM Journal, Volume 12, Number 1, January 1970, http://www.cs.fsu.edu/~mascagni/Halton_SIAM_Review_1970.pdf. Resources
  68. 68. Resources
  69. 69. Niwot Ridge LLC 4347 Pebble Beach Drive Niwot, Colorado 80503Performance Based Managementsm Integrated Master Plan Integrated Master Schedule Earned Value Risk Management Proposal Support Service Glen B. Alleman glen.alleman@niwotridge.com 303.241.9633
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×