Statistics v. Probability4 In building our risk tolerant IMS, we’re interested in the probability of a successful outcome “What is the probability of on or before planned completion date?” The underlying statistics of the cost, schedule and technical performance influence this probability. The statistics of the task durations, their arrangement in a network of tasks and correlation define how this probability based estimated developed.
Our DoD Risk Management Process5 Correct any Anticipate deviations what can go wrong Control Identify Track all Communicate actions Track Analyze Decide what is important Plan Plan to take action
Control Identify Risks Identify Communicate Track Analyze Plan6 Risk identification answers the question “What can go wrong?” by: Looking at current and proposed staffing, process, technical, supplier, operational, resources, and any other dependencies, Monitoring technical results especially failures, Reviewing potential shortfalls against expectations, Analyzing negative trends.
Control Analyze Risks Identify Communicate Track Analyze Plan7 Risk analysis answers the question “How big is the risk?” by: Considering the likelihood of the root cause occurrence; Identifying the possible consequences in terms of performance, schedule, and cost; and Identifying the risk level using the “Risk Reporting Matrix”
Control Planning the Risk Response Identify Communicate Track Analyze Plan8 Risk mitigation planning answers the question “What is the program approach for addressing this potential unfavorable consequence?” One or more of these mitigation options may apply: Avoiding risk by eliminating the root cause and/or the consequence, Controlling the cause or consequence, Transferring the risk, and/or Assuming the level of risk and continuing on the current program plan.
Control Implementing the Risk Pln Identify Communicate Track Analyze Plan9 Risk mitigation (plan) execution ensures successful risk mitigation occurs. It answers the question “How can the planned risk mitigation be implemented?” By: Determining what planning, budget, and requirements and contractual changes are needed, Providing a coordination vehicle for management and other stakeholders, Directing the teams to execute the defined and approved risk mitigation plans, Outlining the risk reporting requirements for on-going monitoring, and Documenting the change history.
Control Tracking the Risk Identify Communicate Track Analyze Plan10 Risk tracking ensures successful risk mitigation. It answers the question “How are things going?” by: Communicating risks to all affected stakeholders, Monitoring risk mitigation plans, Reviewing regular status updates, Displaying risk management dynamics by tracking risk status within the Risk Reporting Matrix, and Alerting management as to when risk mitigation plans should be implemented or adjusted.
Control Risk Control Processes Identify Communicate Track Analyze Plan11 The control function takes the tracking status reports for the watched and mitigated program risk and decides what to do with them based on the reported data. The general process of controlling risks includes: Analyzing the status reports Deciding how to proceed Executing the decisions
Putting this framework together with EV12 Program Manager Top N risks decision Control s assign responsibility Functional • Integrate Managers • Reprioritize Top N • Authorize • Functional area risks resources Work Package Analyze Plan Managers Review • Approve plans 2 assign non-top N Prioritize • Recommend EV Data Evaluate actions Classify • Develop plans Individuals/ Team Members risks risk status 3 1 EV Data Identify Track EV Data trends
Connecting EV and Risk Management13 1 EV data is assigned to Tasks and Work Packages, managed by CAMs and WP managers. They can review, prioritize, evaluate, classify the risks they know well at the lowest level in the IMS. 2 In the planning stages of the IMS (or BoE), risks are revealed during the normal course of work Identifying and analyzing these risks, again falls on the CAM and WP Managers. 3 During the Tracking activities, risks are including the in standard EVM performance assessment ETC, EAC, BCWR, and other going forward estimiates
Identification Activities Overview15 PRINCIPLES Statement of risk 1. Non-judgmental 2. Individual voice Context Individual 3. Multiple perspectives uncertainties List of risks Group/team uncertainties Project EV Data data
Analysis Activities Overview16 PRINCIPLES Statement 1. Most important Statement of risk 2. Vital few of risk 3. Leverage relationships Probability Context Impact Timeframe Classification Rank Classification Class 1 Class 2 Risk Risk Risk Risk Risk Class 3 Risk Risk List of risks Master list of risks Top N
Planning Activities Overview17 Statement of risk Context Resources Project goals Statement of risk Impact and constraints Context Probability Impact Timeframe Probability Classification Timeframe Rank Classification Rank Plan Approach Master list Classification of risks Class 1 Class 2 Top Risk Risk N Risk Risk PRINCIPLES Action plans Risk Class 3 1. Action-oriented Risk Risk 2. Ownership 3. Accountability
Tracking Activities Overview18 PRINCIPLES 1. Risk-driven measures Statement of risk 2. Know your audience Context Impact 3. Know what success looks like Probability Timeframe Classification Status reports Rank Plan Approach Resources • risks • mitigation plans Action plans Statement of risk Context Impact Probability Timeframe Classification Risk & mitigation Rank plan measure Plan Approach Project Status data
Control Activities Overview19 PRINCIPLES 1. Follow the plan Decisions 2. Conditions change • replan • close Status reports • invoke contingency • risks • continue • mitigation tracking plans Statement of risk Statement of risk Context Context Impact Impact Probability Probability Timeframe Timeframe Classification Classification Rank Project Rank Plan Approach Plan Approach Status data Status Control Decision
Management Reserve20 Calculate total project management reserve required based on statistical modeling past experience Total Funds 4000 3500 Management Reserve BAC 3000 2500 Schedule Reserve BCWS 2000 1500 1000 500 0 Jan Mar May Jul Sep Nov Jan Mar May
Management Reserve Strategy21 1200 1000 800 600 Estimate based on risk evaluation Estimate based on baseline straight-line 400 200 0 Jan Mar May Jul Sep Nov Jan Mar May
Risk Metrics connected the IMS22 12 10 PROBLEM DOMAIN MITIGATION 8 DOMAIN RISK* Pessimistic 6 Transition Thresholds Expected 4 Optimistic WATCH DOMAIN Accept 2 Event #1 2 3 4 5 6 Feb 96 Mar 96 Apr 96 May 96 Time
Risk–Earned Value23 6000 5000 Risk Exposure $$$ 4000 REWS: Risk Exposure Work Scheduled 3000 2000 1000 0 Feb Mar Apr May Jun
Risk Retirement Plan in the IMS24 25000 Time Now 20000 Risk Exposure Variance Schedule Risk Variance 15000 Actual to-date Estimate To Completion 10000 Plan To Completion 5000 0 Jan Mar May Jul Sep Nov Jan Mar May
Schedule and Funding Reserve25 Total Funds Negotiated 4500 Management 4000 Reserve 3500 BAC 3000 2500 2000 BCWS 1500 1000 500 Schedule Reserve 0 Jan Mar May Jul Sep Nov Jan Mar May TIME
Schedule and Funding Reserve26 Now negotiated Total Funds 4500 4000 Management Reserve Cost 3500 overrun EAC BAC 3000 2500 2000 BCWS Schedule 1500 slip 1000 500 Schedule ACWP BCWP Reserve 0 Jan Mar May Jul Sep Nov Jan Mar May TIME
Management Reserve Ratio (MMR)27 1.4 MR TFA BAC 1.3 MRR TFA TFA CPI EAC BAC 1.2 Ratio of total funds available against MRR 1.1 total estimated funds expended at CPI completion 1 MRR: Management Reserve Ratio 0.9 TFA: Total Funds Available 0.8
Risk Data Analysis Relationships28 Term Symbol Formula Checklist Actions Ratio of work accomplished against Risk Exposure ARWP REPI money spent (Efficiency Rating: Performance Index REWP Work Done for Resources Expended) Ratio of work accomplished against what Risk Schedule RSPI REWS should have been done (Efficiency Rating: Performance Index REWP Work done as compared to what should have been done) TFA Ratio of total funds available against = EAC total estimated funds expended at Management MRR Reserve Ratio completion or total funds available TFA against the budgeted cost at completion = X CPI multiplied by the cost performance index BAC
Risk Exposure Tracking29 25000 Time Now 20000 Risk Exposure Variance Risk Schedule Variance 15000 Actual to-date Estimate To Completion 10000 Plan To Completion 5000 0 Jan Mar May Jul Sep Nov Jan Mar May 1.4 1.3 MRR 1.2 1.1 Good 1 CPI 0.9 Not so Good 0.8
Reference Class Calibration30 Estimating in the presence of uncertainty Past performance used to calibrate future estimates Time phased improvements in upper and lower bounds
Reference Class Step By Step From Nobel Prize to Project Management, Flyvbjerg, 200631
Flyvbjerg’s 4–step Approach32 1) Form the reference class, a collection of similar-to projects for which there is both history and reasonable insight to the history so that adjustments for present time can be made. 2) Develop a true distribution of the reference class, and from that distribution calculate the cumulative probability. This probability curve, developed from reference class, the outside view. 3) Develop the inside view. The inside view is a traditional estimate by the project team. 4) Adjust the inside view based on the probability of historical outcome from the outside view. Develop a forecast using the reference class probability confidence curve. Pick a confidence limit, and then adjust the inside view to have a corresponding confidence.
Integrating the Cost, Schedule and Technical Risk Model for DI-MGMT-81650 Compliance33 Cost, Schedule, Technical Model† Research the Project Find Analogies Ask Endless Questions WBS Analyze the Results Probability Density What can go wrong? Task 100 Function How likely is it to go wrong? What is the cause? Task 101 What is the consequence? Task 102 Task 103 Monte Carlo Simulation Tool is Mandatory Task 104 Cumulative Distribution Function 1.0 Task 105 .8 .6 Task 106 .4 .2 0 Days, Facilities, Parts, People
Summary34 Earned Value methods used to indicate risks plan allocation of management reserve track risk exposure “buydown” track “buydown” investment Use for “critical” risks Account for actions that reduce risk Fixing risks is cheaper than fixing problems
35 Performance Based Management(sm), Copyright ® Glen B. Alleman, 2012