Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. NASA Activities in Risk AssessmentNASA Project Management Conference March 30-31, 2004 Michael G. Stamatelatos, Ph.D.,Director Safety and Assurance Requirements Division Office of Safety and Mission Assurance NASA Headquarters 1
  2. 2. NASA is a Pioneer and a Leader in Space; Therefore Its Business Is Inherently Risky International Space Station Safe assembly and operationSpace Transportation Space Shuttle Orbital Space Plane 2
  3. 3. Our GoalImprove risk awareness in the Agency Conduct PRA training for line and project managers and for personnelDevelop a corps of in-house PRA expertsTransition PRA from a curiosity object to baselinemethod for integrated system safety, reliability andrisk assessmentAdopt organization-wide risk informed culture PRA to become a way of life for safety and technical performance improvement and for cost reduction Implement risk-informed management process 3
  4. 4. Probabilistic Risk Assessment (PRA) Answers Three Basic Questions Risk is a set of triplets that answer the questions: 1) What can go wrong? (accident scenarios) 2) How likely is it? (probabilities) 3) What are the consequences? (adverse effects) Kaplan & Garrick, Risk Analysis, 1981 Event Event Sequence Sequence Frequency Modeling Evaluation 2. How frequently does it happen? (Scenario frequency quantification) EventInitiating Sequence Risk Event PRA Insights Logic IntegrationSelection Development 1. What can go wrong? Risk statement Decision Support (Definition of scenarios) 3. What are the consequences? (Scenario consequence quantification) Consequence Modeling PRA is generally used for low-probability and high- consequence events 4
  5. 5. Relationship Between Risk Management and Probabilistic Risk Assessment (PRA) Continuous Risk Management Method Technique Application Qualitative Qualitative Risk FMEA. FMEA. Risk Assessment MLD, MLD, Assessment ESD, ESD, Technical ETA, Technical ETA, Risk Risk FTA, FTA, RBD RBD and/or Probabilistic Probabilistic and/or Risk Risk Assessment Actuarial/ Program Program Assessment Actuarial/ Statistical Risk Risk Statistical Analyses Analyses Legend: Decision Decision FMEA - Failure Modes & Effects Analysis Analysis Analysis Management MLD - Master Logic DiagramManagement System ESD - Event Sequence Diagram System ETA - Event Tree Analysis FTA - Fault Tree Analysis RBD - Reliability Block Diagram 5
  6. 6. NASA Risk Management and Assessment Requirements• NPG 7120.5A, NASA Program and Project Management Processes and Requirements The program or project manager shall apply risk management principles as a decision-making tool which enables programmatic and technical success. Program and project decisions shall be made on the basis of an orderly risk management effort. Risk management includes identification, assessment, mitigation, and disposition of risk throughout the PAPAC (Provide Aerospace Products And Capabilities) process.• NPG 8000.4, Risk Management Procedures and Guidelines Provides additional information for applying risk management as required by NPG 7120.5A.• NPG 8705.x (draft) PRA Application Procedures and Guidelines Provides guidelines on how to apply PRA to NASA’s diversified programs and projects 6
  7. 7. How Does PRA Help Safety?Provides a basis for risk reduction through:1. Accident/Mishap Prevention (best)2. Accident/Mishap Consequence Mitigation Adverse Event in a Sequence Prevention Mitigation 7
  8. 8. The Concept of an Accident Scenario PIVOTAL EVENTS Pivotal Pivotal IE End State Event 1 Event n DetrimentalThe perturbation Mitigative consequence of Aggrevative interest (Quantity of intereset to decision-maker)Risk Scenario is a string of events that (if they occur) will lead toan undesired end state. 8
  9. 9. Exact vs. Uncertain Probabilities Uncertainty distribution of probability values ExactlyProbability known Probability probability value A narrower range means a more precise knowledge of the distribution, or less uncertainty 9
  10. 10. Quantification of Uncertainty Probability density function, e.g., probability of LOCVUncertainty Distribution: P(x) is the probability (or xth percentile ρ(x) confidence) that the result value is x P(x) median is the 50th percentile is area under 5% x 50% 95% curve between Median 0 and xUncertainty Range: Uncertainty range 5th percentile 95th percentile (spread) from the 5th to the 95th percentile Uncertainty (confidence) range 10
  11. 11. Event- and Fault-Tree Scenario Modeling No damage to No damage to Hydrazine leaks Leak detected Leak isolated flight critical scientific End state Leak not detected avionics equipment IE LD LI A S OK Loss of science Loss of Spacecraft Presure PresureController fails transducer 1 transducer 2 OK fails fails Loss of science CN P1 P2 common cause Loss of failure of P. Spacecraft transducers OK PP distribution Loss of science Loss of Spacecraft Fault tree Event tree 11
  12. 12. PRA Methodology Synopsis Inputs to Decision Making Process Master Logic Diagram (Hierarchical Logic) Event Sequence Diagram (Logic) End State: ES1 IE A B End State: OK End State: ES2 End State: ES2 End State: ES2 C D E LOGIC MODELING End State: ES1 End State: ES2 Event Tree (Inductive Logic) Fault Tree (Logic) One to Many Mapping of an ET-defined Scenario End Not A NEW STRUCTURE IE A B C D E State 1: OK Logic Gate Basic Event Internal initiating events One of these events External initiating events 2: ES1 Hardware components AND 3: ES2 Human error 4: ES2 Software error one or more Common cause of these 5: ES2 elementary Environmental conditions events 6: ES2 Other Link to another fault tree Probabilistic Treatment of Basic Events Model Integration and Quantification of Risk Scenarios Risk Results and Insights 30 5060 25 40 End State: ES2 Integration and quantification of50 PROBABILISTIC 2040 100 logic structures (ETs and FTs)30 15 30 20 and propagation of epistemic Displaying the results in tabular and graphical forms 1020 5 10 80 End State: ES1 uncertainties to obtain Ranking of risk scenarios10 0.01 0.02 0.03 0.04 0.02 0.04 0.06 0.08 0.02 0.04 0.06 0.08 60 Ranking of individual events (e.g., hardware failure, minimal cutsets (risk 40 scenarios in terms of human errors, etc.) Examples (from left to right): 20 basic events) Insights into how various systems interact Probability that the hardware x fails when needed likelihood of risk Tabulation of all the assumptions Probability that the crew fail to perform a task scenarios Probability that there would be a windy condition at the time of landing 0.01 0.02 0.03 0.04 0.05 uncertainty in the Identification of key parameters that greatly likelihood estimates influence the results The uncertainty in occurrence of an event is Presenting results of sensitivity studies characterized by a probability distribution 12
  13. 13. What Decision Types Can PRA Support? Safety improvement in design, operation, maintenance and upgrade (throughout life cycle); Mission success enhancement; Performance improvement; and Cost reduction for design, operation and maintenanceFor all these areas of application, PRA can help: Identify leading risk contributors and their relative values Indicate priorities for resource allocation Optimize results for given resource availability 13
  14. 14. Areas of PRA Application at NASA In Design and Conceptual Design (e.g., Crew Exploration Vehicle, Mars missions, Project Prometheus) For Upgrades (Space Shuttle) For Development/construction/assembly (e.g., International Space Station) When there are requirements for Safety Compliance (e.g., nuclear missions like Mars ’03; Project Prometheus, Mars Sample Return) 14
  15. 15. NASA Procedural Requirement NPR 8705 (Draft) CONSEQUENCE NASA PROGRAM/PROJECT CRITERIA / SPECIFICS PRA SCOPE CATEGORY (Classes and/or Examples) Planetary Protection Program Mars Sample Return Missions F Requirement Public Safety White House Approval Nuclear Payloads F (PD/NSC-25) (e.g., Cassini, Ulysses, Mars 2003) Human Safety and Space Missions with Flight Health Launch Vehicles F Termination Systems International Space Station F Human Space Flight Space Shuttle F Orbital Space Plane/Space Launch Initiative F High Strategic Importance Mars Program F Launch Window High Schedule Criticality F (e.g., planetary missions) Mission Success Earth Science Missions (for non-human L/S (e.g., EOS, QUICKSCAT) rated missions) Space Science Missions All Other Missions L/S (e.g., SIM, HESSI) Technology Demonstration/Validation (e.g., L/S EO-1, Deep Space 1) F = Full scope; L/S = Limited or SimplifiedPRA Scope Legend: F = Full scope; L = Limited scope; S = Simplified PRA 15
  16. 16. NASA Special PRA Methodology Needs Broad range of programs: Conceptual non-human rated science projects; Multi-stage design and construction of the International Space Station; Upgrades of the Space Shuttle Risk initiators that vary drastically with type of program Unique design and operating environments (e.g., microgravity effects on equipment and humans) Multi-phase approach in some scenario developments Unique external events (e.g., micro-meteoroids and orbital debris) Unique types of adverse consequences (e.g., fatigue and illness in space) and associated databases Different quantitative methods for human reliability (e.g., astronauts vs. other operating personnel) Quantitative methods for software reliability 16
  17. 17. Space Shuttle Probabilistic Risk Assessment 17
  18. 18. STS Nominal Mission Profile ASCENT completes ORBIT completes APU Shutdown TAL TIG-5t ~ 150kft) SSME start ENTRY completesAPU start 18
  19. 19. Current Shuttle PRA Results for LOCV (provisional) 1/25 100 median = 1 in 123P ro b a b ility D e n s ity F u n c tio n mean = 1 in 76 5th percentile = 1 in 617 95th percentile = 1 in 23 0 1/25 1/500 0.00 0.01 1/100 0.02 1/50 0.03 1/33 0.04 1/25 0.05 1/20 truncated Number of Missions 19
  20. 20. Summary of Shuttle PRA Historical ResultsProbability of Loss of Crew and Vehicle 1993 PRA 1988 PRA update the Median values (log scale) 2003 PRA 1998 PRA First PRA Galileo study Integrated Unpublished 1995 PRA conducted on results to reflect PRA with all analysis using First major the Space the current 0.1 elements. QRAS. No study of (April 1993) test Shuttle for 18 Orbiter Integration of 1996 PRA ascent only, 1997 PRA Shuttle PRA. and operational systems. elements. Bayesian up for Galileo First use of Analysis by experience Incl. MMOD Limited to 3 date of the Mission QRAS tool. SAIC with base of the Orbiter 1995 model by Update of input from Shuttle. systems and adding 17 1995 PRA with prime the propulsion new look at successful contractors 1/78 elements flights APU 0.01 1/90 1/123 1/245 1/147 1/145 1/161 1/254 1/245 0.001 SPRAT PRA Shuttle PRA Shuttle PRA Shuttle PRA Shuttle PRA Phase 1 - Galileo 1988 2003 1998 1997 1996 1995 1993 (Ascent (Ascent Ascent, only) only) Ascent and entry only entry, and orbital debris 20
  21. 21. Annual Voluntary Risks in Some Sports -Comparable in Magnitude to Shuttle Risk Professional stunting 1/100 Dedicated mountain climbing 1/167 Air show/air racing and acrobatics 1/200 Amateur flying in home-built aircraft 1/333 Experienced whitewater boating 1/370 Sport parachuting 1/500 Source: R. Wilson and E. Crouch, Risk-Benefit Analysis, Harvard University Press, 2001 21
  22. 22. International Space Station (ISS) PRA • 1999 -- The NASA Advisory Council recommended, the NASA Administrator concurred, and the ISS Program began a PRA. − The modeling will be QRAS- compatible. − First portion of PRA (through Flight 7A) - delivered in Dec. 2000; Second portion (through Flight 12A) delivered in July 2001. 22
  23. 23. 23
  24. 24. Important ISS PRA Findings MMOD: lead contributor to loss of station (LOS) risk Illness in space: lead contributor to loss of crew (LOC) risk 24
  25. 25. Approach to PRA for NASA Top-Level Designs (e.g., Crew Exploration Vehicle) Strawman Mission Level 1 Architecture Design ActivitiesRequirements Cost and DesignRisk an Safety Schedule Concepts Goals and RiskRequirements Assessments Definition of Safety, Operational Reliability, Mission Phases PRA Schedule,Requirements and Cost Trade-offs First-Order Risk GoalPerformance Allocation forRequirements Each Phase Design Engineering System Reliability AllocationPast RelatedPRA Efforts; Maturee.g., Shuttle Design Top Level PRA 25
  26. 26. Advanced PRA Methods or Tools QRAS (Quantitative Risk Assessment System) – a state of the art integrated PRA computer program Galileo/ASSAP – Dynamic fault tree program Software reliability methodology for use in PRA External event methodology for micro- meteoroid and orbital debris (MMOD) risk into the overall risk assessment 26
  27. 27. QRAS 1.7 Is Being Commercialized Space Shuttle time in seconds SSME SRB ORBITER ET R(i,x, t) = R 0(i,x, t) t1 t2 t3 t4 -6 0 128 510 where: SSME R is reliability of manifold weld HPOTP _______________ i is physical state LPFTP _______________ LPFTP HPFTP HEX MCC HEX _______________ x is vector of physical variables MCC _______________ t is time SRB TVC _______________ Bolt Manif Seal Engineering Modes NOZZLE _______________ Weld Fail to determine initiating event probability Fail Fail Mission TimelineElement/Subsystem Hierarchy Event Sequence Diagram Initiating Event Event Tree (quantification) Manif (furure unhancement: dynamic) Weld Is crack Is repair Yes Successful ManifoldFailure detectable? 100% effect.? op. Weld Success Prob. S = 0.9 S = ... S = ... No Failure LOV Prob. F = ... Success Prob. Is crack small enoung to survive Yes Successful F = ... S = ... LOV Prob. 1 mission? op. F = ... Mission Fail. Prob. No F = 0.1 S = 0.96 LOV Prob. F = 0.04 Loss of flow HPFTP cavitates to LPFTP No LOX rich op. Successful op. Fault Tree Risk Aggregation Probability Distribution (use to quantify pivotal events) for initiating event What if? Tool Box Probability of Results 1. Remove old subsystem and Bayesian Manifold Weld replace with design. Updating Failure 2. Change failure probabilities for Mathematica initiating events. 3. Eliminate failure modes. Others 4. Vary parameters of engineering models. (future) Pr 5% tile 95% tile median 27
  28. 28. In Summary, We Plan toContinue to improve risk awareness Conduct PRA training for line and project managers and for personnelContinue to develop a corps of in-house PRAexpertsTransition PRA to baseline method for safetyassessmentIntegrate risk assessment with system safetyand reliability assessmentAdopt organization-wide risk informed culture PRA to become a way of life for safety and technical performance improvement and for cost reduction Implement risk-informed management process 28