Fussell.louis
Upcoming SlideShare
Loading in...5
×
 

Fussell.louis

on

  • 12,905 views

 

Statistics

Views

Total Views
12,905
Views on SlideShare
12,905
Embed Views
0

Actions

Likes
0
Downloads
46
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Course material today is drawn from new NPR 8004A – Aency Risk Mangement PRocedureal Requirements, NASA Systems Engineering Handbook, NASA General Safety Program Requirements, SEI’s CRM Guidebook, and Examples from Futron’s RM experiences This is where people will sprinkle their own history
  • There are a number of definitions and uses for the term risk, but there is no universally accepted definition . What all definitions have in common is agreement that risk has two characteristics: uncertainty : An event may or may not happen. loss : An event has unwanted consequences or losses. This course uses the NPR 8000.4 definition of risk. Risk is a triplett of some scenario that leads to degrated performance, the likelihood of the scenario happening, and the consequence if scenario wer e to occur
  • The Continuous Risk Management paradigm illustrates a set of functions that are identified as continuous activities through out the life cycle of a project. The paradigm is a conceptual or abstract view of risk management. Same chart
  • In the new NASA RM paradigm, RM is the integration of Risk Informed Decision Making and CRM whereas in the past it was just CRM In order to put CRM to work it has to be implemented into RIDM The purpose of RIDM is to better inform through…. The purpose of CRM is to manage risk associated…..
  • The decision making process is fairly straightforward. If you go to any type of text for decision making process, you’ll see these steps You’ll need to understand your own biases before getting into decision making There are other APPEL courses available to help understand decision analysis and decision making PMSE – the first week deals with systems engineering, goes through decision making process; same with next two bullets Our focus on this class is how do we risk inform the decision analysis process We’re going to put on risk colored glasses to inform risk processes
  • CRM is also a continuous process by the circular style of graphic Developed at DoD with Carnegie Mellon in the 70’s and provides a disciplined environemtn, what can go wrong, implement actions, and track actions until successful completion, while documenting what you come up with
  • The NPR states that performance requirements flow down from agency strategic goals down to the Performance Requirements / Performance Measures to … The performance requirements stated…. The performance measures are metrics…. Performance criteria are…. Risk is the potential for performance…. Performance requirement may not always be expressed quantitatively The risk is the criteria of performance shortfalls This graphic is from the NPR and focusing on the LH side
  • Break down Walk through steps in RM process and describe process steps in how manage risk process Discussed remainder of class
  • Input Individuals have uncertainties and issues about the project and project progress that may or may not be risks . In group activities, individuals may collaborate to identify uncertainties and issues. The project data is supporting information that consists of items such as the schedule, budget, plans, work breakdown structure, etc. that may provide information helpful in identifying risks (e.g., previously unknown dependencies between module development schedules). Output For each risk, the statement of risk is captured along with the associated context A list containing all statements of risk
  • Pulled from RIDM handbook – is fairly generic mission objective hierarchy Have some affordability objectives to keep affordable – meet performance objectives This is how to determine what performance criteria are – or looking at goals objectives for particular mission and making sure they are in alignment and coming up with those performance measures
  • Input Individuals have uncertainties and issues about the project and project progress that may or may not be risks . In group activities, individuals may collaborate to identify uncertainties and issues. The project data is supporting information that consists of items such as the schedule, budget, plans, work breakdown structure, etc. that may provide information helpful in identifying risks (e.g., previously unknown dependencies between module development schedules). Output For each risk, the statement of risk is captured along with the associated context A list containing all statements of risk
  • Input Individuals have uncertainties and issues about the project and project progress that may or may not be risks . In group activities, individuals may collaborate to identify uncertainties and issues. The project data is supporting information that consists of items such as the schedule, budget, plans, work breakdown structure, etc. that may provide information helpful in identifying risks (e.g., previously unknown dependencies between module development schedules). Output For each risk, the statement of risk is captured along with the associated context A list containing all statements of risk
  • Input Individuals have uncertainties and issues about the project and project progress that may or may not be risks . In group activities, individuals may collaborate to identify uncertainties and issues. The project data is supporting information that consists of items such as the schedule, budget, plans, work breakdown structure, etc. that may provide information helpful in identifying risks (e.g., previously unknown dependencies between module development schedules). Output For each risk, the statement of risk is captured along with the associated context A list containing all statements of risk
  • The condition-consequence format provides a more complete picture of the risk, which is critical during mitigation planning . It can be read as follows: given the there is a possibility that will occur The condition component focuses on what is currently causing concern. This is something that is true or widely perceived to be true . This component provides information that is useful when determining how to mitigate a risk. The consequence component focuses on the intermediate and long-term impact of the risk . Understanding the depth and breadth of the impact is useful in determining how much time, resources, and effort should be allocated to the mitigation effort. A well-formed risk statement usually has only one condition, but may have one or more consequences. Use this example if needed : We have multiple customers and an unclear procedure for identifying and resolving customer (science) requirement conflicts; we may get conflicts that go undetected until the design phase and that could delay implementation while we try resolving science requirements conflicts. Statement is written – given that, possibility, consequence – want to really stick to events that are possible
  • Ask the students if the statements on the slides are good or bad risk statements. Have them explain their answers. Use the following as a discussion guideline. 1. “Object oriented development”—BAD What is the problem with OOD? What is the impact? 2. “The staff ...”—BETTER RESTATEMENT OF #1 - But not in correct form! Now you know that the staff needs training and time to train. State the problem in the correct format - condition and consequence Read slides briefly
  • FROM IDENTIFY Input for each risk, the statement of risk and associated context a list containing all statements of risk Output for each risk, values for impact, probability, timeframe, class, and rank added to the risk information risks organized into groups based on some common basis a master list of risks containing all risks and the priority ranking of the Top N risks If you already have risks classified into groups, you may want to classify any new risks first to see if they fit into already-established groups, which may or may not have mitigation efforts underway before you evaluate. As you get more information you may need to re-evaluate, reclassify, or reprioritize. For the case study we will first look at how to evaluate a set of risks, then we’ll look at how we can group the risks into sets. We may decide that some need to be mitigated as a set. Based on the evaluation we’ll select the most important risks to the project and prioritize them
  • FROM ANALYZE Input Before planning, for each risk we have statement of risk and supporting context values for impact, probability, timeframe, class, and rank the master list of all risks including the priority ranking of the top N all risks organized by class Constraints available resources for mitigation targets and limits set by the project (e.g., do not slip the schedule, use no more than 5% of team’s budget for risk mitigation) Output for each risk, updated information to include the planning approach taken (research, accept, watch, mitigate) a description of what action is to be taken to deal with the risk
  • FROM THE ACTION PLANNING STEP Input Before tracking, for each risk we have statement of risk and supporting context; values for impact, probability, timeframe, class, and rank; planning approach (research, accept, watch or mitigate) Before tracking for each risk we have an action plan that consists of one of the following: research plan, acceptance rationale, tracking requirements, or mitigation plan (action list or task plan) the current values for the “watched” and “mitigate” risk plans and measures Constraints—limits resources available for mitigation Supporting information—project information such as schedule and budget variances, critical path changes, and project/performance indicators can be used as triggers, thresholds, and risk- or plan-specific measures where appropriate. Outputs a variety of status reports highlighting the current values of the risk indicators and the status of action plans updated information for each risk, including current status data for the risk (e.g., measure, indicator, and trigger values)
  • FROM TRACK (previous activity) Input In track, we have a variety of status reports highlighting the current values of the risk indicators and the status of action plans. Before control, the risk information for each risk comprises the statement of risk, supporting context, impact, probability, timeframe, class, rank, plan approach, and status data (e.g., measure, indicator, and trigger values). Supporting information Project information, such as schedule and budget variations, critical path changes, and project/performance indicators can be used to support decision making where appropriate. This data can be considered when project personnel make control decisions. Output a decision that determines the next action for the risk or set of risks (replan, close the risk, invoke a contingency plan, continue tracking and executing the current plan) updated information for each risk or set of risks, including the control decision
  • Monitoring the quality of plan execution Are the plans being executed correctly? Are the results what was expected? Significant changes in risks changes in impact, probability, or timeframe
  • takes place throughout crm cycle that we’re always coming up with informaiton we need to document
  • talk about going fwd – RM process

Fussell.louis Fussell.louis Presentation Transcript

  • Project Risk Management Louis Fussell Used with permission
  • Course Content
    • Risk Management Overview
    • Risk-Informed Decision Making
      • Identification of Alternatives
      • Risk Analysis of Alternatives
      • Risk-Informed Alternative Selection
    • Continuous Risk Management
      • Identify
      • Analyze
      • Plan
      • Track
      • Control
      • Communicate and Document
  • Source of Course Material
    • NPR 8000.4A - Agency Risk Management Procedural Requirements
    • NASA/SP-2010-576 – Risk-Informed Decision Making Handbook
    • NASA/SP-2007-6105 – NASA Systems Engineering Handbook
    • Probabilistic Risk Assessment Procedures Guide for NASA Managers and Practitioners
    • NPR 7123.1A, NASA Systems Engineering Processes and Requirements
    • NASA/SP-2007-6105, NASA Systems Engineering Handbook
    • 2008 NASA Cost Estimating Handbook
    • NASA-STD-7009, Standard for Models and Simulations
    • SEI’s Continuous Risk Management Guidebook
  • Risk Management Overview
  • Definition of Risk
    • Risk is the potential for performance shortfalls , which may be realized in the future, with respect to achieving explicitly established and stated performance requirements .
    • Performance shortfalls may be related to institutional support for mission execution or related to any one or more of the following mission execution domains:
      • Safety (e.g., avoidance of injury, fatality, or destruction of key assets)
      • Technical (e.g., thrust or output, amount of observational data acquired)
      • Cost (e.g., execution within allocated cost)
      • Schedule (e.g., meeting milestones)
  • Operational Definition: Risk is a Set of Triplets
    • Likelihood
    Consequence Qualitative or Quantitative Qualitative or Quantitative Risk = (Scenario, Likelihood, Consequence) Scenario Includes Evaluation of Uncertainties
  • Risk Management is the Integration of Two Complementary Processes
    • Risk-informed Decision Making (RIDM)
      • To inform decision making through better use of risk information in establishing baseline performance requirements for program/projects and mission support organizations
    • Continuous Risk Management (CRM)
      • To manage risk associated with the implementation of baseline performance requirement
  • Risk-Informed Decision Making
    • Utilizes a diverse set of performance measures along with other considerations within a deliberative process to inform decision making.
      • “ Risk-informed” and not “risk-based”
    • Performance measures are situation-specific, and address programmatic and technical objectives
    • An alternative is selected for implementation through the RIDM process based on the performance measures
  • Risk-Informed Decision Making Process
    • Identification of decision alternatives ( decision context ) and considering a sufficient number and diversity of Performance Measures to constitute a comprehensive set for decision-making purposes
    • Risk analysis of decision alternatives is defined broadly in NPR 8000.4A as uncertainty analysis of performance associated with the alternative
    • Selection of a decision alternative informed by (not solely based on) Risk Analysis results
    Identification of Alternatives Identify Decision Alternatives (Recognizing Opportunities) in the Context of Objectives Risk Analysis of Alternatives Risk Analysis (Integrated Perspective) and Development of the Technical Basis for Deliberation Risk-Informed Alternative Selection Deliberate and Select an Alternative and Associated Performance Commitments Informed by (not solely based on) Risk Analysis To Requirements Baselining Risk-Informed Decision Making
  • NASA’s Continuous Risk Management Process
    • Developed in collaboration with DoD and Software Engineering Institute
    • It provides a disciplined environment for proactive decision making to:
      • Assess continually what could go wrong (risks)
      • Determine which risks are important to deal with
      • Implement action plans to deal with those risks
      • Assure, measure effectiveness of the implemented action plans
  • Risk Management in the Context of Agency Decision Making Strategic Goals Decompose Objectives into Imposed Constraints and Performance Measures ARCHITECTURE ALTERNATIVES SYSTEM ALTERNATIVES Program Requirements Project Requirements Risks to Reqs Risks to Reqs Reassess Alternatives or Rebaseline Reqs Reassess Alternatives or Rebaseline Reqs SUBSYSTEM ALTERNATIVES Subsystem Requirements Risks to Reqs Reassess Alternatives or Rebaseline Reqs CRM CRM CRM
  • Performance Requirements & Performance Measures Priorities Set Top-Down Requirements Flow Top-Down Risk Reported Bottom-Up Decisions Elevated Bottom-Up
  • Risk-Informed Decision Making (RIDM)
  • Risk-Informed Decision Making
    • The RIDM Process is divided into three Parts:
      • Part 1 – Identification of Alternatives
      • Part 2 – Risk Analysis of Alternatives
      • Part 3 – Risk-Informed Alternative Selection
    • Parts are divided into several steps
      • Course addresses each step sequentially
    • In practice, the RIDM process can be highly iterative
    Identification of Alternatives Step 1 – Understand Stakeholder Expectations and Derive Performance Measures from Objectives Step 2 – Compile Feasible Alternatives Risk Analysis of Alternatives Step 3 – Set the Framework and Choose the Analysis Methodologies Step 4 – Conduct the Risk Assessment and Document the Results Risk-Informed Alternative Selection Step 5 – Develop Risk Normalized Performance Commitments Step 6 – Deliberate, Select and Alternative, and Document the Decision Rationale To Requirements Baselining Risk-Informed Decision Making
  • Identification of Alternatives Overview Identification of Stakeholders Upper Level Requirements & Specifications Identification of Alternatives Step 1 – Understand Stakeholder Expectations and Derive Performance Measures from Objectives Step 2 – Compile Feasible Alternatives Requirements & Expectations Concept of Operations Performance Measures List of Alternatives
  • Performance Measures Example
    • Three divisions
      • Orbit Insertion Package
        • Propulsive
        • Aerobraking
      • Science Package
        • Low Fidelity
        • High Fidelity
      • Launch Vehicle
        • Small
        • Medium
        • Large
    • Alternatives are pruned
    Feasible Alternatives Trade Tree Example
  • Identification of Alternatives Overview Performance Measures List of Alternatives Risk Analysis of Alternatives Step 3 – Set the Framework and Choose the Analysis Methodologies Step 4 – Conduct the Risk Assessment and Document the Results Analysis Methodology Analysis of Performance Measures Performance Commitments Analysis Documentation
  • Set the Framework
    • Key elements of the risk analysis framework
      • Quantifies the performance measures
        • Natural, Constructed, or Proxy
        • Point estimates or distribution functions
      • Integrates the performance assessments
        • Numerically
        • Graphically
    • Framework challenges
      • Applicable to each alternative
      • Consistently addresses uncertainties
      • Preserves correlation between performance measures
        • Higher Fidelity => Longer Schedules => Higher Costs
      • Is transparent, traceable, and configuration managed
  • Analysis of Performance Measures
    • Often performance measure values are uncertain
      • Modeled using probability distribution functions
    • Cannot simply add distributions together
      • Must consider probability distribution parameters
      • Must consider correlation between performance measures
    • Integrating accomplished through Monte Carlo analysis
    ? = PM 2 PM 2 PM 3 PM 4 + + +
  • Develop the Technical Basis for Deliberation
    • The Technical Basis for Deliberation (TBfD) contains the information needed to risk-inform the selection of a decision alternative.
      • A statement of the top-level objectives and imposed constraints
      • The objectives hierarchy and performance measures
      • A summary description of the compiled decision alternatives, indicating pruned alternatives
      • A summary of the risk analysis framework and models
      • Scenario descriptions
      • Marginal performance measure pdfs and a summary of significant correlations
      • A tabulation of risk with respect to imposed constraints
      • Identification of significant risk drivers with respect to imposed constraints
      • Candidate performance measure risk tolerances
  • Identification of Alternatives Overview List of Alternatives Technical Basis for Deliberation Risk-Informed Alternative Selection Step 5 – Develop Risk Normalized Performance Commitments Step 6 – Deliberate, Select and Alternative, and Document the Decision Rationale Decision Decision Rationale Risk-Informed Selection Report
  • Performance Commitments for Imposed Constraints
    • Some performance measures have imposed constraints
      • Performance measures must remain within desired limits
      • This imposed constraint is the performance commitment
      • For example, spacecraft mass must be within launch vehicle capability
    • Performance measures for alternatives can be compared based on probability that imposed constraint will not be met
    • Difficult to choose when probability distribution functions overlap
      • More analysis may be required to reduce uncertainty
  • Performance Commitments for Non-Imposed Constraints
    • Many performance measure do not have imposed constraints
      • Risk analysis can help determine performance commitments
    • Determining a risk-normalized performance commitment
      • Determine probability distribution function of performance measure for each alternative
      • Obtain risk tolerance as a probability of failure from decision maker
      • Value of performance measure at the given risk tolerance is obtained from the probability distribution
      • This value is the risk-normalized performance commitment for that performance measure
        • Value is different for each alternative, but risk tolerance is the same
    0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
  • Deliberation
    • Deliberate contending alternatives are those remaining after pruning
      • May be iterative process
    • Rationales for elimination of non-contending alternatives include:
      • Infeasibility – Imposed constraints cannot be met within the risk tolerance of the decision maker
      • Dominance – Other alternatives exist that have superior candidate performance commitments on every performance measure
      • Inferior Performance in Key Areas – Alternatives that are markedly inferior in terms of their candidate performance commitments in key areas can be eliminated on that basis
    • Also consider potential for exceptionally high or poor performance
      • Reflecting stakeholder and decision-maker risk attitudes
  • Continuous Risk Management (CRM)
  • Continuous Risk Management
    • Six process steps
      • Identify
      • Analyze
      • Plan
      • Track
      • Control
      • Communicate & Document
  • Risk Identification Activities Overview
    • Identify
    • Capture statement of risk
    • Capture context of risk
    Statement of Risk Context List of Risks __________ __________ __________ __________ Group/team Uncertainties Problems, Issues, and Uncertainties Project Data
  • Components of a Risk Statement
    • Condition: a single phrase briefly describing current key events, circumstances, situations, etc. that are causing concern, doubt, anxiety, or uncertainty
    • Deviation: a single phrase that describes a possible change from a baseline plan
    • Asset: an element of the organizational or system element
    • Consequence: a single phrase or sentence that describes the adverse event and negative outcomes
    Condition Deviation there is a possibility of Given the adversely impacting Asset thereby leading to Impact
  • Example Risk Statement
    • Given that development of the spacecraft requires some as yet unavailable new technology and long-lead time components
    • there is a possibility of components will not being ready in time ,
    • adversely impacting spacecraft development
    • thereby leading to a launch slip
  • Risk Analysis Activities Overview Master List of Risks ________________ ________________ ________________ ________________ Top N Risk Risk Risk Risk Risk Risk Class 3 Risk Class 1 Class 2
    • Analyze
    • Evaluate
    • Classify
    • Prioritize
    List of Risks __________ __________ __________ __________ Classification Statement of Risk Context Likelihood Consequence Timeframe Classification Rank Statement of Risk Context
  • Quantitative Risk Analysis Schedule Analysis Mission Risk Analysis Bad Thing X Fails Y Fails Z Fails A project … … produces a product … … and a process that operates the product. Model : Integrated Master Schedule Method: Monte Carlo Schedule Simulation Tool: @Risk for Project Model: Functional Flow Block Diagrams Method: Monte Carlo Process Simulation Tool: ARENA Model: Fault Trees Method: Probabilistic Risk Analysis Tool: SAPHIRE … that generates values. WBS Task 100 Task 101 Task 102 Task 103 Task 104 Task 105 Task 106
  • Risk Planning Activities Overview Action Plans Project Goals & Constraints Resources
    • Plan
    • Assign responsibility
    • Determine approach
    • Define scope and actions
    Master List of Risks ________________ ________________ ________________ ________________ Top N Risk Risk Risk Risk Risk Risk Class 3 Risk Class 1 Class 2 Classification Statement of Risk Context Likelihood Consequence Timeframe Classification Rank Statement of Risk Context Likelihood Consequence Timeframe Classification Rank Plan Approach
  • Risk Planning Decisions
    • Negotiate with the next higher organization level elevation triggers
    • Feedback to RIDM risk handling plans which may lead to reconsideration of alternatives
    Risk(s) Close Accept Mitigate Watch Research Elevate Assign Risk Owner Assign Risk Owner Assign Risk Owner Assign Risk Owner Mitigation Plan Tracking Plan Research Plan Document Decision in Risk Management Database N N N N N Y Y Y Y Y
  • Risk Tracking Activities Overview Status Report Risk Action plans
    • Track
    • Acquire
    • Compile
    • Report
    Action Plans Resources Project Data Statement of Risk Context Likelihood Consequence Timeframe Classification Rank Plan Approach Statement of Risk Context Likelihood Consequence Timeframe Classification Rank Plan Approach Status Metrics
  • Risk Waterfall Chart Example Risk: Given that the Star Tracker is a new technology and may not perform to expectations, there is a possibility that the schedule may be impacted to allow time to fix shortfalls in performance Risk Exposure Time High Moderate Low 8/98 10/98 2/98 4/98 6/98 8/99 New ACS Selected Integration Testing Planned Tasks Actual Tasks Exit Criteria Met
  • Risk Control Activities Overview
    • Control
    • Evaluate
    • Decide
    Status Report Risk Action plans Project Data Statement of Risk Context Likelihood Consequence Timeframe Classification Rank Plan Approach Status Metrics Statement of Risk Context Likelihood Consequence Timeframe Classification Rank Plan Approach Status Metrics Control decision Decision Re-plan Close Invoke contingency Continue tracking
  • What Is Effective Risk Control?
    • Monitoring the quality of plan execution
    • Assessing the effectiveness of mitigation plans
    • Assessing significant changes in risks and trends
    • Determining appropriate responses
    • Executing the plan of attack
    • Communicating the above information
  • Communication and Documentation Takes Place Throughout CRM Risk Statements Risk Attributes Action Plans Metric Reports Decisions
  • Communication Attention
    • Elevate decision-making on particular “risks” to the next higher organizational level when:
      • They cause performance measures to exceed pre-defined thresholds
      • They can no longer be managed by the current-level organizational unit because:
        • Resources are not available
        • Decision authority is lacking
    • Ensure that cross-cutting risks and interdependencies between risks are properly identified as cross-cutting and either managed within the unit or elevated.
  • Risk Management Process
  • Information Flow in RIDM
    • Technical Authorities
    • Engineering
    • Safety & Mission Assurance
    • Health & Medical
    Decision Maker
    • Stakeholders
    • Internal
    • External
    Risk Analysts
    • Subject Matter Experts
    • Safety
    • Technical
    • Cost
    • Schedule
    Deliberation Decision (incl. risk acceptance) Risk Concurrence Consultation Objectives Values Objectives Values Performance Models Contending Alternatives Pros/Cons Analysis Results
  • Information Flow in CRM
    • Risk-Informed Selection Reports
    • Selected Alternative
    • Risk List
    • Risk Analysis of Selected Alternatives
    • Mitigation
    • Risk-driving Uncertainties
    Performance Requirements
    • Analyze I
    • Baseline RIDM Risk Analysis, Performance Measure Distributions & Performance Commitments
    • Analyze and Incorporate New Risks & Data
    • Risk-driving uncertainties
    • Control
    • Periodically Decide on Control Action
    • Track
    • Continuously Acquire & Compile Data
    • Run Risk Analysis Models w/ Current Data
    • Periodically Issue Tracking Reports
    • Plan
    • Baseline Mitigations
    • Handle Risk (Accept, Mitigate, Watch, Research, or Close)
    • Negotiate RIsk Thresholds
    • Analyze II
    • Assess Timeframe
    • Prioritize Risks According to Performance Measure Distribution Drivers
    • Develop Prioritized Risk Lists According to Mission Execution Domains
    • Identify
    • Baseline the Risk Database
    • Identify Implementation/New Risks
    • Translate & Document Risks
    • Communicate & Document
    • Maintain Risk Database
    • Decide & Implement Communication Methods
    • Report Risks to Higher Org Level as Necessary
    • Provide Feedback to Affected Units
    Update RIDM Risk Analysis? Inability to Control Risks? Develop New or Revise Mitigation Plans? New Risks with No Mitigation Available? RIDM Systems Engineering CRM RIDM RIDM RIDM RIDM Yes Yes Yes Yes No No No Re-Plan if Needed
  • Please Take the 1-Day Course Check SATERN for Dates