INTEGRATING RISK WITH EARNED VALUE PIKES PEAK REGIONAL CHAPTER PROJECT MANAGEMENT PRODUCTIVITY TOOLS SYMPOSIUM SATURDAY, FEBRUARY 28TH, 2009 GLEN B. ALLEMANThe notion of integrating cost, schedule, technical performance, and risk ispossible in theory. In practice care is needed to assure credible information isprovided to the Program Manager.
Todays Learning Objectives Cost, Schedule, Technical Performance, and Risk Are Inseparable All numeric values of cost, schedule, technical performance, and risk assessment are random variables drawn from some underlying probability distribution Managing these variables must be done through the eyes of a Risk Manager 2
There are usually twophases to many projects …1. Too early to tell2. Too late to stop 3
The FundamentalProgram Management Question Can we plan with sufficient accuracy, to allocate resources to reduce risk while meeting requirements, and obtain timely feedback in order to adjust plans and performance goals? 4
Dependencies Between The Inseparable Variables Cost Funding margin for Schedule margin for over under performance Over cost or Over cost or target baseline (OTB) under over performance schedule Over schedule Technical Performance or under Schedule performing Schedule margin for underperformance or schedule extension 5
Some Simple Definitions Programmatic and Technical Earned Value Technical Risk Performance Management Management MeasuresEstablishes project Looks to the A time phasedperformance status unknown future to progress plan forand extrapolates identify risks and the achievement ofthat information to recommend early the Technicalunderstand future action to be taken Performancetrends and to limit the threat or Parameters (TPP)allocation of maximize the that indicate keyresources needed exploitation of areas for riskto successfully opportunities reduction andmeet project program successmilestones APM EV-Risk Working Group, 2007 INCOSE G2SEBOK 6
The Integration of Risk and Earned Value is a SystemsEngineering Paradigm Cost Systems Engineering states Integrated Master Schedule & deliverables performance and Earned Value Management associated risks tracks risk activities Technical Performance Schedule Structure WBS contains risk mitigations and retirement efforts 7
But Many Times, The Information from Cost, Schedule, Techncial Performance, and RiskManagement Systems Gets Mixed Up When We Try to Put Them Together 8
Every Program Manager Should Know…9 When will this project finish? What is the statistical confidence in this date? Is the project’s schedule realistic and achievable? What will it cost when we are done? How is the project performing against its plan? What deliverables are slipping? How are we going to get the deliverables back on schedule? What does past performance say about the future performance? What is the impact on the schedule of any change requests? Is there enough schedule slack and cost reserve to cover the risk? Where is the risk in the schedule? How can we get the work done sooner to reduce risk? How can we recover from any foreseen delays? How can we work around a problem?
The First Step is to Start with Key Elements of Earned Value10 BCWS SV BCWP CV ACWP The Work planned, performed, and the cost of performing that work, is the common variable across each Earned Value variable
Some Useful EV Information EAC Floor Performance Factor BAC BCWPcum BAC EAC ACWP CPIcum CPIcum EAC Ceiling Performance Factor BAC BCWPcum BAC EAC ACWP CPIcum SPIcum CPIcum EVM Analyst’s EAC Range • Addresses Risk • Based on CPI & SPI trend analysis not on plugging numbers into a formula • Incorporates technical performance, schedule progress, CAIV and other program information http://www.daytonaero.com/Files/resource/31.pdf 11
ANSI 748B Says on Page 112 Plan all work scope for the program to completion. Integrate program work scope, schedule, and cost objectives into a baseline plan against which accomplishments may be measured. Objectively assess accomplishments at the work performance level. Analyze significant variances from the plan and forecast impacts. Provide data to higher levels for management decision making and implementation of management actions.
Some Criteria for Successful EVMS Beyond Full Compliance of the 32 Criteria Identify the Define a Work Integrate WBS and Schedule all Indentify Products OrganizationsBreakdown Structure OBS into a RAM Planned Work and Milestones doing the work Time Phase the Record all Direct Determine all Sum These Manage Action Budget Costs Variances Variances Plans13 Incorporate Changes
At The Same Time, Risk Management is Commonly Misunderstood It’s not about random chance, it’s about definingmitigations and retirement plans in the presence of uncertainty 14
Five Fundamental Principles of Risk Management1. Hope is not a strategy2. No single point estimate of cost or schedule can be correct3. Cost, Schedule, and Technical Performance are inseparable4. Risk management requires adherence to a well defined process5. Communication is the Number One success factor 15
DoD’s Approach to Successful Risk Management16
But ANSI 748B Also Says …17 Identify physical products, milestones, technical performance goals, or other indicators that will be used to measure progress. So why the disconnect between Risk and Earned Value? Isn’t the measure of risk, the rate of risk reduction an “…other indicators…”
Revisions & Change Control Define and Establish MR Authorize WorkOrganize the Issue Budget Establish PMB Measure Analyze Results Work Authorize Plan Performance Integrating I think youEarned Value should be more and Risk explicit aboutManagement the steps here. Plan Risk Perform Risk Develop Risk Assign Monitor and Activities Assessment Handling Plans Responsibilities Communicate Update Risk Register 18
SOME ACTIONABLE DETAILS FORPUTTING THESE FOUR CONCEPTSTOGETHERIn Theory there is no difference between Theory and Practice.In Practice there is — Yogi Berra
The Difference BetweenTechnical and Programmatic Risk There are two types of “uncertainty” on any sufficiently complex program – Technical – uncertainty about the functional and performance aspects of the program’s technology that impacts the produceability of the product or creates delays in the schedule – Programmatic – uncertainty about the duration and cost of the activities that deliver the So much for our strategy of winning functional and performance elements of the through technical dominance program, independent of the technical risk We’re interested in connecting the two in the schedule and cost model(s) – When the technical uncertainty arises what is the impact on the schedule and cost? – When the schedule or cost uncertainty arises what is the impact on the functional and performance aspects? 20
Deterministic Versus Probabilistic Deterministic Probabilistic• Each activity has a planned value • The program elements are not• For the schedule each task has a random, but they are random predecessor and a successor. variables drawn from a probability• The longest path through the distribution. network is the critical path • Three point estimates "can" be• The total duration of the project is used to describe task duration a fixed value - it is deterministic random variables• The total cost is the sum of all the • The total duration of the project is activity costs a random number• Risks are defined and handled as • The total cost is a random number static entities • Risks are stochastic processes that have probabilistic outcomes for cost, schedule and technical performance 21
Without these Principles The Program Train Wreck Starts When… Inattention to Lack of predictive budgetary variance analysis responsibilities Untimely and unrealistic Work authorizations Latest Revised Estimates that are not always (LRE) followed Progress not monitored Issues with Budget and in a regular and data reconciliation consistent manner Lack of an integrated Lack of vertical and management system horizontal traceability Baseline fluctuations cost and schedule data and frequent replanning for corrective action Current period and Lack of internal retroactive changes surveillance and Improper use of controls management reserve Managerial actions not Mary K. Evans Picture Library EV techniques that do demonstrated using not reflect actual Earned Value 22 performance
AN INTRODUCTION TO SOFTWAREENGINEERING INSTITUTE’SCONTINUOUS RISK MANAGEMENT(CRM)CRM is the Software Engineering Institute’s framework for managing risk in thecontext of system integration, COTS based product development, and themanagement of these activities.
Continuous Risk Management has Six Components http://www.sei.cmu.edu/risk/index.html 24
Putting Continuous Risk Management Work25 CRM Activity Project Representation Risk items with IMP/IMS #’s, CA/WP & resource Identify assignments Analyze Risk management responsibilities assigned Mitigation plans with durations and resource Plan assignments Track Status reported from Risk Management to IMS Control Risk tasks reporting in weekly status process Integrated Master Schedule (IMS) status reporting Communicate as Physical Percent Complete
This All Comes Together in The Risk Registry26 http://www.mitre.org/work/sepo/toolkits/risk/ToolsTechniques/RiskMatrix.html
PHYSICALLY CONNECTINGEARNED VALUE WITH RISKMANAGEMENT
Some Historical Background†28 Cost and schedule growth have been persistent problems for decades The use of Earned Value is common in DoD environments As is the use of Risk Management It’s the integration that is missing † “Integrating Risk Management with Earned Value Management,” NDIA Report, www.ndia.org
The Core Problems Before We Start29 Earned Value is a measure of quantity not quality Risk measures the probability of an unfavorable occurrence but rarely monetizes this outcome Risk management focuses on dealing with future events Earned Value focuses on reporting past performance
Starting with the Individual Elements30 Identify Analyze Plan Track Control Risk board Mitigation and Identify risk Connect PMB Vet members ofOrganization assessment of retirement owners with Risk registry the leadership impact owners Identify Probability PMB contains all Transfer Planning & Baseline the risk uncertainty and impact in risk active mitigation to PMB Budgeting registry response schedule activities and close risk Assign risk to Risk budgets Risk Board Transfer through Probability Accounting CA/WP cost defined and management of EV change impact on cost baseline allocated cost impacts control processes Risk registry Asses impacts Risk Board and Manage budget contains Analysis and IMS Board assignment by probabilities of dependencies impact analysis risk level cost and schedule Updated PMB Keep historical Update ETC from Revise PMB Revisions from risk Risk review records risk management forecast activities
First, A (Notional) Big Picture31 Risk ID Traceable to work elements in the IMS within the WBS Risk reduction Standard program waterfall metrics performance of TCPI connected to Earned and IEAC Value program performance
One Approach to Integrating Risk Management with Earned Value32 Control Integrate across functions Program Manager Reprioritize Authorize project Top N risks resources Decisions Control Assign Functional Managers Integrate Reprioritize Responsibility Authorize Top N risks Functional area resources Work Package Managers Analyze Plan Review Approve plans Prioritize EV Data Recommend actions Evaluate Develop plans Classify Individuals/Team 1 Members risks Risk Status EV Data Identify Track EV Data Trends Required 3 Indicators 2
EXAMPLE SCHEDULE RISKANALYSISMonte Carlo tools model the task durations with random variables for thedurations and build “pictures” of the likelihood of a task completely on orbefore a specific date.
Schedule for a Monte Carlo Risk analysis starts with acredible schedule and defines the probabilistic behavior of each activities and how it drives the deliverables 35 The Risk+ tool sets the upper and lower bounds of the possible durations
The output of Risk+ is a Probability Distribution Function and aCumulative Distribution of all the possible dates that “watched” activity could take.The result is a picture of the Confidence that the target date of 2/10/3 – can be met. It shows 40% – which is not good Date: 2/25/2009 3:34:12 PM Completion Std Deviation: 5.51d Samples: 300 95% Confidence Interval: 0.62d Unique ID: 30 Each bar represents 2d Name: Final Testing 0.16 1.0 Completion Probability Table 0.9 0.14 Cumulative Probability Prob Date Prob Date 0.8 0.12 0.05 Wed 1/29/03 0.55 Wed 2/12/03 0.7 0.10 Fri 1/31/03 0.60 Thu 2/13/03 Frequency 0.10 0.6 0.15 Tue 2/4/03 0.65 Fri 2/14/03 0.08 0.5 0.20 Wed 2/5/03 0.70 Fri 2/14/03 0.4 0.25 Wed 2/5/03 0.75 Mon 2/17/03 0.06 0.3 0.30 Thu 2/6/03 0.80 Tue 2/18/03 0.04 0.35 Fri 2/7/03 0.85 Wed 2/19/03 0.2 0.02 0.40 Mon 2/10/03 0.90 Thu 2/20/03 0.1 0.45 Mon 2/10/03 0.95 Tue 2/25/03 Mon 1/20/03 Tue 2/11/03 Mon 3/3/03 0.50 Tue 2/11/03 1.00 Mon 3/3/03 Completion Date 36
So with this information we can ask …37 What cost impact will there be? What resource impacts? What technical dependencies? What mitigation or retirement plans must be in place to increase the probability of success to something greater than 40%? What other interdependencies are there in the program?
Program Management is Risky Business Traditional Approaches Are Seriously Flawed Risk Management Is Not About Preventing Risk Risk Management Is About Managing In The Presence Of Risk This means managing Cost, Schedule, and Technical Performance 38
Source Materials39 Interfacing Risk & Earned Value: Management Reserve, “Technical Performance Measurement, Earned Value, and Risk Management: An Integrated Diagnostic Tool for Program Management,” Commander N. D. Pisano, SC, USN, Program Executive Office for Air ASW, Assault, and Special Mission Programs (PEO(A)) Practical Risk Management: The ATOM Method, David Hillson and Peter Simon, Management Concepts. Project Risk Analysis and Management Guide, APM Publishing. “Formal Risk Management,” DACS Gold Practices “Quantify Risk to Manage Cost and Schedule,” Fred Raymond, Acquisition Review Quarterly, Spring 1999, pp. 147–154 Probabilistic Risk Assessment Procedures Guide for NASA Managers and Practitioners, Office of Safety and Mission Assurance, April 2002 Development Of Risk Management Defense Extensions To The PMI Project Management Body Of Knowledge – Tutorial,” Edmund Conrow, Acquisition Review Quarterly, Spring, 2003. Effective Risk management: Some Keys to Success, Edmund Conrow, American Institute of Aeronautics and Astronautics, 2000.