Multidimensional RISKRISK INTEGRATED WITH SCHEDULE, COST, PERFORMANCE, ANDANYTHING ELSE YOU CAN THINK OF                  ...
NASA Uses Two Complementary Processes For Risk Management Risk-Informed Decision Making (RIDM)  – Emphasizes the proper u...
RIDM Selects Alternatives & CRM Addresses The ImplementationOf Alternatives   Risk-Informed Decision Making (RIDM)        ...
CRM Uses The Identify Step To Document Risks In The Form ofRisk StatementsRisk Statements have 3 distinct elements1. Scena...
Risk Is Typically Measured As Likelihood Times Consequence                                                                ...
The Identify Step of CRM Documents Risk In Multiple DimensionsTo Get A Complete Risk Picture                              ...
Managers Use Customized Criteria To Bin Risks Into A Risk Matrix                                                          ...
The Risk Matrix Provides The Framework For CRM Risk Analysis                                        Effective analysis ma...
Current Risk Matrix Development Methods Often Fail To Give AComplete Risk Picture                                   Notion...
MRisk Makes Use Of Anchor Points And Multidimensional-Distance        Measure To Determine Total Risk                 Cost...
Anchor Points & Mahalanobis Distance Make Risk AnalysisObjective & Logically Consistent  The anchor points make it possib...
MRisk Provides A Complete Risk Picture  MRisk addresses several shortcomings in the current methods 1. MRisk deals with a...
Mahalanobis Distance Mapping Tells Us How Far Each Risk IsFrom All Of The Other Risks, Thus Highlighting Outliers   Notion...
Mahalanobis Distance Is Based On The Interdependencies OfDimensions Consider two, random variables X and Y that consist o...
Two Dimensional Mahalanobis Distance Example  Consider again our two, random variables X and Y that consist of risk obser...
Legacy Methods By Contrast Assume Independence Of TheDimensions Of Risk  Consider again our two, random variables X and Y...
The MRisk Metric Calculates Distance While Accounting For ThePoint To Point Relationship                               Ma...
Case In Point: Multiple Risks Consider a risk scoring session involving  5 risks SMEs vote on the probability and  conse...
Using MRisk All The Events Fit Onto One ScaleEvent Prob            Perf        Cost Sched               Event Prob dmin dm...
MRisk Provides A Clear Picture Of The Risk Profile Regardless Of  The Number Of Dimensions InvolvedEvent Prob Perf Cost Sc...
Traditional Multivariate Methods Like Euclidean Distance May Not  Be As Clear Because They Don’t Consider RelationshipsEve...
MRisk Deals With Several Shortcomings In Risk Analysis  Just because we cannot visualize risk in multiple dimensions does...
Upcoming SlideShare
Loading in …5
×

Doug brown

19,351 views

Published on

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
19,351
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
47
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • The graph below lists seven dimensions of riskA single risk can affect more than one dimension
  • Doug brown

    1. 1. Multidimensional RISKRISK INTEGRATED WITH SCHEDULE, COST, PERFORMANCE, ANDANYTHING ELSE YOU CAN THINK OF NASA PM Challenge Feb 2011 0
    2. 2. NASA Uses Two Complementary Processes For Risk Management Risk-Informed Decision Making (RIDM) – Emphasizes the proper use of risk analysis to make risk-informed decisions that impact all risk dimensions including safety, technical, cost, schedule, etc… – Acknowledges the role that subject matter experts (SMEs) play in decisions. Emphasizes that the cumulative wisdom provided of SMEs is essential for integrating technical and nontechnical factors to produce sound decisions due to the availability of technical data and the complexity of missions – Source: NASA/SP-2010-576 NASA Risk-Informed Decision Making Handbook Continuous Risk Management (CRM) – To manage those risks associated with the performance levels that drove selection of a particular alternative (from RIDM) – A systematic and iterative process that efficiently identifies, analyzes, plans, tracks, controls, and communicates and documents risks associated with implementation of designs, plans, and processes – Source: NPR 8000.4A Agency Risk Management Procedural Requirements 1
    3. 3. RIDM Selects Alternatives & CRM Addresses The ImplementationOf Alternatives Risk-Informed Decision Making (RIDM) Continuous Risk Management (CRM) Identification of Alternatives Identify Decision Alternatives (Recognizing Opportunities) in the Context of Objectives Risk Analysis of Alternatives Risk Analysis (Integrated Perspective) and Development of the Technical Basis for Deliberation Risk-Informed Alternative Selection Deliberate and Select an Alternative and Associated Performance Commitments Informed by (not solely based on) Risk Analysis * Source: NASA/SP-2010-576 NASA Risk-Informed Decision Making Handbook 2
    4. 4. CRM Uses The Identify Step To Document Risks In The Form ofRisk StatementsRisk Statements have 3 distinct elements1. Scenario – A sequence of credible events that specifies the evolution of a system or process from a given state to a future state. In the context of risk management, scenarios are used to identify the ways in which a system or process in its current state can evolve to an undesirable state2. Likelihood – Probability of occurrence3. Consequence – The possible negative outcomes of the current conditions that are creating uncertainty there is aGiven SCENARIO LIKELIHOOD CONSEQUENCE will occur that, 3
    5. 5. Risk Is Typically Measured As Likelihood Times Consequence Consequence Likelihood RISKS Estimation of Estimation of Definitions Definitions the impact to the the likelihood program if the that the risk risk eventevent will occur occurs Likelihood x Consequence Quantitative Risk Score 4.79 10.73 29.30 48.94 4
    6. 6. The Identify Step of CRM Documents Risk In Multiple DimensionsTo Get A Complete Risk Picture People Safety Environment Schedule On-Orbit operations risk Configuration Management Technical Cost 5
    7. 7. Managers Use Customized Criteria To Bin Risks Into A Risk Matrix RISK MATRIX Likelihood Rating 5 Level Probability LIKELIHOOD 4 5 Very Likely Expected to happen 3 Could happen. Controls have significant limitations or 4 Likely uncertainties. 2 Could happen. Controls exist, with some limitations or 3 Possible uncertainties. 1 Not expected to happen. Controls have minor limitations 1 2 3 4 5 2 Unlikely or uncertainties. CONSEQUENCES Highly Extremely remote possibility that it will happen. Strong 1 Unlikely controls in place. CONSEQUENCE 1 Very Low 2 Low 3 Moderate 4 High 5 Very HighTechnical Negligible or no impact to Minor impact to achievement Some impact to achievement of Moderate impact to Major impact to achievement of achievement of Subsonic of Subsonic Transport System Subsonic Transport System achievement of Subsonic Subsonic Transport System Transport System Level Level Metrics, Technical Level Metrics, Technical Transport System Level Metrics, Level Metrics, Technical Metrics, Technical Deliverables, Technology Deliverables, Technology Technical Deliverables, Deliverables, Technology Deliverables, Technology Maturation, or KPP Goals Maturation, or KPP Goals Technology Maturation, or KPP Maturation, or KPP Goals Maturation, or KPP Goals GoalsSchedule Level 1 Milestone(s): Level 1 Milestone(s): Level 2 Milestone(s): Level 2 Milestone(s): Level 1 Milestone(s): ≤1 month impact > 1 month impact < 1 month impact ≥ 1 month impact > 2 month impact Level 2 Milestone(s): ≤ 2 Level 2 Milestone(s): month impact > 2 month impact Level 3,4 Milestone(s): ≤ 1 Level 3,4 Milestone(s): ≤ 2 Level 2 Milestone(s): Level 3,4 Milestone(s): Level 3,4 Milestone(s): month impact month impact ≥ 3 month impact ≤ 3 month impact >3 month impactCost Between 0% and 5% Between 5% and 10% Between 10% and 15% increase Between 15% and 20% increase Greater than 20% increase over increase over allocated increase over allocated budget over allocated budget (Sub- over allocated budget (Sub- that allocated budget (Sub- budget (Sub-Project, (Sub-Project, Element or Task Project, Element or Task level) Project, Element or Task level) Project, Element or Task level) Element or Task level) level)Safety Negligible or no impact Could cause the need for only May cause minor injury or May cause severe injury or May cause death or permanently minor first aid treatment occupational illness or minor occupational illness or major disabling injury or destruction of property damage property damage property 6 6
    8. 8. The Risk Matrix Provides The Framework For CRM Risk Analysis Effective analysis makes it possible to RISK MATRIX move total project risk from red to 5 green But how do you know you are focusedLIKELIHOOD 4 on the right project risks? 3 Focusing on the wrong risks may keep total project risk in the red? 2 1 1 2 3 4 5 CONSEQUENCES 7
    9. 9. Current Risk Matrix Development Methods Often Fail To Give AComplete Risk Picture Notional Representation Of Risks In Three Dimensions Cost Risk Schedule Risk Performance Risk 5 5 5 LIKELIHOOD LIKELIHOOD LIKELIHOOD 4 4 4 3 3 3 2 2 2 1 1 1 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 CONSEQUENCES CONSEQUENCES CONSEQUENCES =pt1 =pt2  Why are we looking at only one dimension at a time? =pt3  Should we call pt3(3,3,3) a Cost Risk, a Schedule Risk, or a Performance Risk?  Is pt2(1,4,1) more risky than the other points just because it has a high schedule severity?  Is pt1(3,2,3) just as risky as pt3(3,3,3)?  What if we have risk across four dimensions? Or five? Or Six?  How do we know we are focusing on the right risks? 8
    10. 10. MRisk Makes Use Of Anchor Points And Multidimensional-Distance Measure To Determine Total Risk Cost Risk Schedule Risk Performance Risk 5 5 5LIKELIHOOD LIKELIHOOD LIKELIHOOD 4 4 4 3 3 3 2 2 2 1 1 1 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 min (1,1,1) max (5,5,5) CONSEQUENCES CONSEQUENCES CONSEQUENCES dmin d =  The anchor points (1,1,1) and (5,5,5) come from our definition dmin + dmax of the consequence scale  Distance for each point is defined by the distance of that point from the minimum over the sum of the distance from the minimum and the maximum Consequence Scale  The distance value explains the precise consequence for each 100 90 80 risk regardless of the number of dimensions 70 60  The greater the distance the greater the consequence and vice 50 40 versa 30 20  This procedure is scalable to infinite dimensions of 10 0 consequence, i.e. (1,1,…,1n) (5,5,…,5n) L M1 M2 M3 Low Medium H1 H2 High H3 C1 C2 C3 Critical 9
    11. 11. Anchor Points & Mahalanobis Distance Make Risk AnalysisObjective & Logically Consistent  The anchor points make it possible for us to know relative risk – Anchor points allow us to make the distinction between a (3,3,3) and a (4,2,2) – A cost consequence of 3, schedule consequence of 3, and safety consequence of 3 has a distinct distance away from no consequence (1,1,1) and disaster (5,5,5)  Mahalanobis Distance keeps decision makers consistent in their thinking. By calculating risk based on the relationship between costs, schedule, safety, etc… MRisk identifies when violations of known relationships occur in the risk ranking process – For example, cost and schedule have a known relationship in the PM world Schedule Cost Scope 10
    12. 12. MRisk Provides A Complete Risk Picture  MRisk addresses several shortcomings in the current methods 1. MRisk deals with all of the dimensions of Risk simultaneously to provide a complete risk picture 2. MRisk makes risk analysis objective and consistent with SME judgment 3. MRisk provides more advanced statistical algorithms to Risk Management without changing the current processes or products RISK MATRIX 5 LIKELIHOOD Schedule 4 Cost 3 2 1 Scope 1 2 3 4 5 CONSEQUENCES 11
    13. 13. Mahalanobis Distance Mapping Tells Us How Far Each Risk IsFrom All Of The Other Risks, Thus Highlighting Outliers Notional Data Set From Risk Scoring 5 Point Sched Cost Perf …Dimn LIKELIHOOD 4 1 4 3 3 0 3 2 2 2 1 5 3 1 … … … … … 1 2 3 4 5 Outlier m 2 4 1 4 CONSEQUENCES Typical Point Using traditional distance measures the outlier point in the above dmin = (x-xmin)S-1(x-xmin) scenario could be masked by its proximity to the other points -1 Mahalanobis distance highlights the point as an outlier because of its dmax = (x-xmax)S (x-xmax) relative distance away from the group where Mahalanobis distance accounts for the relationship of each risk to S-1 is the Inv(Covariance Matrix) another and highlights the risks that are uncorrelated, thus detecting xmin = [1,1,1] xmax = [5,5,5] extreme risks more efficiently 12
    14. 14. Mahalanobis Distance Is Based On The Interdependencies OfDimensions Consider two, random variables X and Y that consist of risk observations for some project or program Those observations will have a variance and covariance Any set of random variables will have a Variance-Covariance matrix Obs1 Obs1 Obs2 Obs2 Obs3 Obs3 Obsn Obsn 13
    15. 15. Two Dimensional Mahalanobis Distance Example  Consider again our two, random variables X and Y that consist of risk observations for some project or program with the derived variance-covariance matrix  The distance between two points in XY-plane depends on the inverse of the variance- covariance matrix  It’s simple to expand this case to Schedule Risks(X) vs Cost Risks(Y) vs Technical Risks(Z) or any other type of risk comparison 5 4 Y0=(2,4) 3 Y 2 X0=(2,1) 1 1 2 3 4 5 X 14
    16. 16. Legacy Methods By Contrast Assume Independence Of TheDimensions Of Risk  Consider again our two, random variables X and Y that consist of risk observations for some project or program with the derived variance-covariance matrix  In Euclidean Measure the distance between two points in XY-plane depends on the inverse of the Identity matrix 5 4 Y0=(2,4) 3 Y 2 X0=(2,1) 1 1 2 3 4 5 X 15
    17. 17. The MRisk Metric Calculates Distance While Accounting For ThePoint To Point Relationship  Mahalanobis D2 is a multidimensional version of a z-score. It measures the distance of a case from the centroid (multidimensional mean) of a distribution, given the covariance (multidimensional variance) of the distribution.  A case is a multivariate outlier if the probability associated with its D2 is 0.001 or less. D2 follows a chi-square distribution with degrees of freedom equal to the number of variables included in the calculation.  Mahalanobis distance identifies observations which lie far away from the center of the data cloud, giving less weight to variables with large variances or to groups of highly correlated variables (Joliffe, 1986).  This distance has advantages to other distance measures like the Euclidean distance which ignores the covariance structure and thus treats all variables equally 16
    18. 18. Case In Point: Multiple Risks Consider a risk scoring session involving 5 risks SMEs vote on the probability and consequence (1,5) for five events across three dimensions: Performance, Cost, & Schedule The score for each event is recorded in the table belowEvent Prob Perf Cost Sched 1 2 1 2 1 2 4 4 4 3 3 4 3 4 5 4 3 5 1 3 5 4 4 2 1 17
    19. 19. Using MRisk All The Events Fit Onto One ScaleEvent Prob Perf Cost Sched Event Prob dmin dmax d dscaled 1 2 1 2 1 1 2 0.75 14.00 0.05 1.20 2 4 4 4 3 2 4 3.69 1.59 0.70 3.79 3 4 3 4 5 3 4 2.85 4.37 0.39 2.58 4 3 5 1 3 4 3 2.02 9.99 0.17 1.67 5 4 4 2 1 5 4 2.46 7.00 0.26 2.04  MRisk answers the question regarding Event 1 Event 3 Event 2 highest project risk  A (4,4,3) is more consequential than a Event 4 Event 5 (3,4,5) or a (5,1,3)  The current methods would have us focus our attention on the (3,4,5) and max (5,5,5) the (5,1,3) despite the fact that they min (1,1,1) are not the most consequential 18
    20. 20. MRisk Provides A Clear Picture Of The Risk Profile Regardless Of The Number Of Dimensions InvolvedEvent Prob Perf Cost Sched dscaled 5 1 2 1 2 1 1.20 2 4 4 4 3 3.79 3 4 3 4 5 2.58 4 LIKELIHOOD 4 3 5 1 3 1.67 5 4 4 2 1 2.04 3 2 1 1 2 3 4 5 CONSEQUENCES 19
    21. 21. Traditional Multivariate Methods Like Euclidean Distance May Not Be As Clear Because They Don’t Consider RelationshipsEvent Prob Perf Cost Sched Escaled 5 1 2 1 2 1 1.10 2 4 4 4 3 4.14 3 4 3 4 5 4.41 4 LIKELIHOOD 4 3 5 1 3 3.00 5 4 4 2 1 2.11 3 2 Lumping on severity despite differences 1 1 2 3 4 5 Possible collusion of extreme risks CONSEQUENCES 20
    22. 22. MRisk Deals With Several Shortcomings In Risk Analysis  Just because we cannot visualize risk in multiple dimensions doesn’t mean it’s not there. We all realize that Risk Management is a multi-dimensional problem that requires a multi- dimensional solution.  MRisk does not seek to change Risk Management from its current practices and procedures. It just revolutionizes Risk Analysis.  MRisk does not require any change to current data collection techniques for implementation  MRisk takes the data from the current risk methods and allows for interpretation of risks through a multidimensional lens  The use of Mahalanobis Distance as a measure of consequence takes into account the relationships that risk events have across dimensions, i.e. cost, schedule, etc… – Since we know cost relates to schedule, schedule relates to performance, performance relates to safety, etc… MRisk is most appropriate for measuring risk as it emphasizes the relationships among risks to calculate distance 21

    ×