S thomas sfield
Upcoming SlideShare
Loading in...5

S thomas sfield






Total Views
Views on SlideShare
Embed Views



0 Embeds 0

No embeds


Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment
  • Note risks associated with: System interfaces Understanding and verifying interface requirements Silos (vertical and horizontal interactions) Lack of communication Interface requirements NASA example: Hubble Telescope

S thomas sfield S thomas sfield Presentation Transcript

  • Presentation Themes• Inherent riskiness of uncertain times• Evolving risk management to stay relevant• What exactly is Risk Informed Decision-Making (RIDM)?• Why RIDM now, since we’re already doing Continuous Risk Management (CRM)?• RIDM is scary!• When to use RIDM and why• Actual steps (really?!)• Go forth & do good RIDM 2
  • Uncertainty = Reality – Perception?• We live in uncertain times! Whether at NASA or in industry, it’s no longer business as usual – belts are tighter!• Budgets are fluctuating, missions requirements are changing.• Advocates and naysayers alike point to risk – known unknowns – and scarier still, unknown unknowns!• However, simply acknowledging risk ≠ active risk management.• Since uncertainty is rife with risk, NASA requires a more flexible or agile approach to risk management to help decision makers adapt. 3
  • Evolving Risk Management• Risk is a triplet = a scenario with a chance or probability (L) of experiencing a negative consequence (C). − Ref. NASA Procedural Requirements (NPR) 8000.4A• In recent years, NASA Programs, Projects, and Directorates have employed Continuous Risk Management (CRM) to mitigate risk across key performance domains: − Cost (to achieve or regain required performance, cost to mitigate risk, cost of inaction) − Schedule (Critical Path) − Technical − Safety (and Mission Assurance) − Core Capabilities (Critical Assets) 4
  • Evolving Risk Management (cont.)• Why the need for more flexibility? − CRM presupposes a linear mission or a mission that has clearly defined end-objectives. – Today, missions can be less linear (especially for human spaceflight) and end-objectives can be dynamic as policies change. – In this environment, decision-makers must be flexible and even innovative to readily adapt to changes in mission objectives, funding, and resources. – To keep pace, we need to be innovative, systemic thinkers in the way we view risk! 5
  • Evolving Risk Management (cont.)• To remain relevant, risk management practices must evolve as NASA adapts/evolves into a 21st century agency: – For spaceflight, adaptation means a suite of capabilities independent of discrete mission objectives; i.e., build a launch capability whether the ultimate destination is a near-Earth- object, the Moon, or Mars. Convergence NEO Lunar Policy Mission Direction Objectives Mars Destination – As NASA redefines itself, the nature of risk management must shift as well: • From: Identifying and mitigating risk(s) that prevent a performing organization from meeting a known requirement. • To: identifying and mitigating performance shortfalls while helping decision-makers find the best solution to meet tactical and strategic objectives.• RIDM can help to evolve Risk Management practices to meet this challenge! – NPR 8000.4A uses RIDM as a pre-quel to CRM. – This can help frame the risk discussion from a systemic perspective. 6
  • Evolving Risk Management (cont.)Risk Management adds RIDM to CRM in an integrated framework to: • Foster proactive management of threats to performance • Better inform decision-making using relevant (timely) risk information • More effectively manage risks by focusing CRM on existing baseline performance requirements and on those emerging via the RIDM process • Through a graded approach, adapt to varying levels of risk over time. Not my problem! Too funny! Hey, we Interface is 50% right! followed Huh? the spec, I better get When should paid decision makers assess for risk outcomes? Risk Management – Avoiding Undesirable Consequences! 7
  • RIDM is Scary• RIDM terminology can seem daunting – not to worry, we will translate the key terms!• A key component of RIDM per NPR 8000.4A is advocacy for quantitative risk analyses to add rigor to decision processes and trades for: – Complex, multi-scenario projects – Projects with competing stakeholders – (fill-in your case).• However desirable, quantitative analyses are not always feasible. Can be costly and may not be feasible for activities that are smaller, lower dollar, or in the operational phase. – Graded approach allows you to match the rigor of analysis commensurate with the criticality and type of decision to be made.• This presentation discusses RIDM in practical ways and outlines a RIDM case study to illustrate the how’s & why’s – let’s get started! 8
  • Generic Risk Model Controls Controls/Response Risk ConsequenceActivity Event Consequence Consequence Preventive Measures Reduce Severity Mitigate or Accept (Based on Risk Scenarios, Requirements, Plans, Historical & Current Performance, Resources…) RIDM can help to clarify To develop or objectives: ? ? revise: To meet: ● Activities and Products? Activity Requirements Performance (“Shall requirements”) Capabilities objectives, o ● Support functions? ? Operations ver time (Insight/Oversight) Processes 9
  • Risk Model with RIDM (NPR 8000.4A) 10
  • Got Risk?RIDM Process• ID Scenario• ID Stakeholders• Performance Objectives• Performance Measures• Confirm Constraints• Analyze• Down Select• Risk Inform Decision Do you know where your alligators are today? 11
  • Key RIDM Terms RIDM Key# Pt Term "RIDM-Speak" Handbook Translation reference Determine depth of analysis needed to Rigor: Match level of analysis to Section reduce uncertainty. Not all risk analysis significance of decision, project1 √ Graded Approach, can/should be quantitative. Use qualitative p. 51 scale, likelihood, and/or level of analysis, as appropriate. impact. Performance Derived via an objectives hierarchy; General Mandated or delegated goals that2 √ Objective objectives at each level of an organization. term organizations must meet at each level. Metric to capture the extent to which a Performance system, process, or activity meets a Generic3 √ Figures of merit (FOM) Measure performance objective. Same measure term can be used for all alternatives. Alternative-independent level of Match performance objective to Sec 1.5, Performance performance: Using accepted measures, risk tolerance of decision-maker (to4 √ Part 3, pp. Commitment compare across alternatives, and consider 19-22 compare across alternatives at the risk tolerance of the decision-maker. same level of confidence). Evolving document of ground rules and p.17; Technical Basis assumptions used to structure a risk- App.D: Basis of Estimate (BOE) that5 √ for Deliberation informed decision. Specifies information Content evolves with each iteration of (TBfD) needed to characterize uncertainty for each Guide, pp. risk analysis decision alternative. 108-109 Ref: NASA Risk Management Handbook NHBK_2011_3422 12
  • Iterative Risk Decision ProcessMission: Mission Support:● Technical Objectives ● Capability● Tactical Operations ● Capacity / Infrastructure● Schedule / Discrete Milestones ● Institutional Goals● Logistics ● Supply Chain● Strategic Objectives ● Strategic Objectives Systems Engineering Handbook 13
  • RECAP• Evolving Risk Management = RIDM + CRM − Uncertainty and evolving Agency direction elevate the need for responsive Risk Management. − CRM is still a successful tool in the Risk Management tool box. − RIDM complements CRM by enhancing the risk analyses in a graded approach commensurate with decisions to be made. − RIDM and CRM used together throughout your iterative decision process can ensure timely risk information.• Risk-informed decisions = an adaptive process: − Factoring timely risk information into key decisions adaptation to dynamic environments. − CRM alone enables response to a potential risk. − Adding RIDM iteratively can help with early insight into individual risks as well as overall performance risks over time = flexibility. 14
  • Perspective: RIDM-to-CRM CRM-Centric RIDM-Centric• Start with a • Start with a Performance Performance Shortfall (Risk) Objective• Use mini-RIDMs • Frame the to evaluate/make decision space handling (graded decisions approach) • Use CRM to• Assess analyze risk of qualitatively or each alternative quantitatively • Down-select• Develop/ alternatives implement • Track/monitor mitigation via CRM Systems Engineering Handbook “Two Sides of the Same Coin” 15
  • Deciphering RIDM 1a. Identify Performance Objectives 1b. Derive Performance Measures 2. Identify Feasible Alternatives 3. Set the Framework (TBfD)* 4. Analyze Alternatives & Document 5. Risk-Normalize Commitments 6a. Deliberate & Decide 6b. Select an Alternative & Document 16
  • Illustrated Example* • Decision: 1. Select best location for Short Stack Hypergolic Servicing. 2. Base decision on performance objectives, performance measures, stakeholder considerations, and domain risks of each feasible alternative. • Alternatives: 1. Existing facilities (e.g., Pad) 2. New facility 3. Other?CARGO HAZARDOUSSERVICING FACILITY Map of KSC Facilities and Launch Pads *Loosely based on an actual risk scenario 17
  • Step 1a. Refine the DecisionIdentification of Alternatives 1a. Identify Performance • Concise statement of a performance shortfall. Objectives 1b. Derive • Establish common understanding of performance Performance Measures constraints. 2. Identify • Document assumptions and scope of decision. Feasible Alternatives Risk Analysis of Alternatives 3. Set the Framework (TBfD)*Risk-Informed Alternative Selection 4. Analyze Alternatives & Document 5. Risk-Normalize Commitments 6a. Deliberate & Decide 6b. Select an Alternative & Document Scenario for Off-line Servicing & Assembly 18
  • 1b. Criteria for SuccessIdentification of Alternatives 1a. Identify Performance • Identify all Stakeholders and their expectations. Objectives • Define Performance Objectives. 1b. Derive Performance • Characterize Performance Measures. Measures 2. Identify “Show me the money!” is a cool phrase. Feasible Alternatives For RM, it’s not just about the $$, we must consider all domains! Risk Analysis of Alternatives 3. Set the Framework (TBfD)*Risk-Informed Alternative Selection 4. Analyze Alternatives & Document 5. Risk-Normalize Commitments Program Ground Ops 6a. Deliberate & Decide HQ ? KSC 6b. Select an Alternative & EPA Public Document etc. 19
  • 2. Feasible AlternativesIdentification of Alternatives 1a. Identify Performance • Initially consider a range of solutions – even wild- Objectives eyed ideas okay at this point! 1b. Derive Performance Measures • Always include the “Do-Nothing” option. 2. Identify • Use Graded Approach to determine appropriate Feasible Alternatives level of rigor needed to refine your list of feasible Risk Analysis of Alternatives 3. Set the Framework alternatives. (TBfD)*Risk-Informed Alternative Selection 4. Analyze Alternatives & VAB (existing facility) Stop-n-Go Model (new alternative) Document 5. Risk-Normalize Commitments MPPF Pad (existing process) (existing facility) 6a. Deliberate & Decide Vehicle Integration Building (new facility) 6b. Select an Alternative & Document 20
  • 3. Set the FrameworkIdentification of Alternatives 1a. Identify Performance • Select “Key” Decision Parameters. Objectives – Note: Parameters must apply to ALL alternatives. 1b. Derive Performance Measures • Determine reasonable Performance Measures and 2. Identify criteria needed for a decision-maker to commit to Feasible Alternatives an alternative (minimum threshold). Risk Analysis of Alternatives 3. Set the Framework (TBfD)* • Establish weighted priority (basis) for making a decision.Risk-Informed Alternative Selection 4. Analyze Alternatives & Document Launch Availability 5. Risk-Normalize Contingency & Recovery Commitments “Green Pad” Life Cycle Cost 6a. Deliberate & Decide Safety 6b. Select an Flexibility Veh. Orientation Alternative & Document 21
  • 4. Analyze AlternativesIdentification of Alternatives 1a. Identify Performance • Conduct analysis of each alternative’s key Objectives performance measures. 1b. Derive Performance Measures • Prune “unsuitable” alternatives expeditiously based on key parameters: 2. Identify Feasible Alternatives – Reduces analysis cost / effort. Risk Analysis of Alternatives – Clarifies options for decision makers. 3. Set the Framework (TBfD)*Risk-Informed Alternative Selection 4. Analyze Alternatives & VAB (existing facility) Stop-n-Go Model (new alternative) Document 5. Risk-Normalize MPPF Commitments Pad (existing process) (existing facility) 6a. Deliberate & Decide Vehicle Integration Building (new facility) 6b. Select an Alternative & Document 22
  • Additional RIDM Terminology for Step 5 RIDM Key# Pt Term "RIDM-Speak" Handbook Translation reference Create means to compare “like” perspectives across alternatives by first establishing a Normalize performance to assess “common break-point of uncertainty” for each across a range of options analysis. • For any measure, allowable uncertainty is based on a number of factors For ex. Worst-case threshold • Set standards used to compare alternatives at the same level of uncertainty; allocated Risk Normalized resources represent some point value in an Section Avoid co-mingling analysis uncertainty6 √ Performance uncertainty range (.01 – .99) for each 3.3.1, with the uncertainty of an alternative; Commitment alternative. pp.73-78 each option produces its own PDF and CDF. • A common “confidence” level would be used to arrive at a “normalized” value of Apply the same confidence level (i.e., uncertainty for each alternative risk tolerance) to each alternative for a • Additional uncertainty considerations depend criterion (criteria can have different risk on risk tolerance of decision-makers. If risk- tolerance levels) neutral, an option with uncertain performance may be preferred if it performs at the “mean value” of a performance Risk-neutral not likely today! measure pdf. Ref: NASA Risk Management Handbook NHBK_2011_3422 23
  • Step 5. Risk Normalize CommitmentsIdentification of Alternatives 1a. Identify • Consider risk tolerance of the decision-maker and all Performance Objectives key stakeholders for each performance measure. 1b. Derive Performance Measures 2. Identify Feasible Alternatives Risk Analysis of Alternatives 3. Set the Framework (TBfD)* • Determine degree of certainty required for each performance measure for comparison.Risk-Informed Alternative Selection 4. Analyze Alternatives & • Each performance measure should yield a probability Document distribution function (pdf) for each alternative. 5. Risk-Normalize Commitments Note: 6a. Deliberate & Decide Pdf – Cdf applies for quantitative analyses; similarly for qualitative assessments, one should apply the same level of confidence when 6b. Select an evaluating alternatives (constructed scales). Alternative & Document 24
  • Risk Normalized Commitments• Each alternative yields its own PDF and corresponding cumulative distribution function (CDF). NOTES: 1. A common “uncertainty” level would be used to arrive at a “normalized” value of uncertainty for each alternative. 2. Assumed 20% Risk Tolerance = 80% Certainty (use 80% value of CDF) 25
  • Example of Analysis Documented on TBfD Matrix Step 1 Step 1b Step 2 Step 3 High- Level Objectives Performance Objectives Measures Pad VAB MPPF VIB S&G Weight Meets CARD  Yes Yes Yes Yes Yes 0% Launch Availability Optimal: Exceeds L.A. reqmts No Yes Yes Yes No 7% Construction/ refurbishmt ($M)  2.5 5 4 20 6 8% Life Cycle Cost Operations & Maintenance ($M) 4 2.5 4 1 1 10% Discovery 1 - Low Risk 3 1 1 1 2 6% Verification 2 - Mild Risk 2 1 1 1 1 8%Contingency Capabilities Repair 3 - Moderate Risk 1 2 1 1 3 3%Flexibility and Recovery 4 - Potentially Unreliable Off-nominal (de-servicing) 5 - Unreliable 3 2 3 1 1 2% Minimize Pad Time***(Clean Pad) Quan (Hr) 106 1 1 1 1 11% Processing Efficiency De-couple critical path Quan (Hr) 96 12 24 12 12 2% Quantity Distance Impact Quan (hrs contact) 3 16 20 20 3 10% Hypergol Contamination Quan (H/L) 3.2 2.8 2.2 1.6 0.8 10% Safety Adequate “clear” Y/N Yes Yes Yes Yes Yes 0% Minimize Wet Time Quan (Hr) 8 106 106 106 16 6% Scalability (flight rate) Flights/Yr 52 52 70 100 100 10% Schedule - Flexibility Re-Cycle Time (Hr) 48 48 12 12 0 7% Tally 85 113 161 178 140 100% Note: Conceptual only – data & values fabricated for illustrative purposes only. 26
  • Step 6a. Decide & DeliberateIdentification of Alternatives 1a. Identify Performance • Apply analysis results to TBfD and make Objectives recommendation to Decision-Maker. 1b. Derive Performance • Reconsider TBfD if no clear leader and report Measures results to Decision-Maker. 2. Identify Feasible Alternatives • Reconsider requirements / constraints if no Risk Analysis of Alternatives acceptable alternatives. 3. Set the Framework (TBfD)*Risk-Informed Alternative Selection 4. Analyze Alternatives & Document 5. Risk-Normalize Commitments Analysis Findings (for example): − Recommend New Facility 6a. Deliberate & − Document “Green Pad” Requirement Decide − Work to Minimize Wet Time 6b. Select an Alternative & Document 27
  • Recap: Risk Management = RIDM + CRM Identification of Alternatives RIDM: Inform decision-making through use of quantitative and qualitative risk info 1a. Identify 1b. Derive 2. Identify to establish baseline performance Performance Performance Feasible requirements for mission support Objectives Measures Alternatives organizations, programs and projects. Ref. RIDM Handbook Fig. 11. RIDM Process Steps (p.30) CRM: Risk Analysis of Alternatives Manage risk associated with implementing baseline performance requirements using a 3. Set the 4. Analyze 5. Risk- systematic and iterative process to Framework Alternatives & Normalize identify, analyze, plan, track, control, commTBfD:* (TBfD)* Document Commitments unicate and document risks.Technical Basisfor Deliberation;also known as(BOE) Basis ofEstimate Risk-Informed Alternative Selection 6a. 6b. Select an Build Deliberate & Alternative & Performance Decide Document Requirements Iterative 28
  • Concluding Remarks• Uncertainty, evolving objectives, scarce resources – require an evolution in the way we approach risk management.• The key is coupling RIDM with CRM to evolve Risk Management practices to be more pro-active vs. simply reactive.• Aim: Employ an integrative, systemic approach to yield relevant and timely Always information for missions and mission consider support. mission support• Doing so keeps NASA Risk Management risks! practices fresh and relevant to enable risk- informed decisions.• Go forth & do good RIDM (and CRM)! 29
  • JSCAC3 Sharon L. ThomasSharon.L.Thomas@nasa.gov(281) 244-7668JSCNA Scott L. Field, PMPScott.Field@nasa.gov(281) 335-2440 30