1. NASA‟s Safety Goal Policy for Human Space
(Key Concepts Behind the Policy, Implementation Approach Through
a Risk-informed Safety Case)
Presented at the NASA Project Management 2011 Conference
Long Beach, California
February 9-10, 2011
Homayoon Dezfuli, Ph.D.
NASA Technical Fellow (System Safety)
Office of Safety and Mission assurance (OSMA)
NASA Headquarters

2. Acknowledgments
• This presentation has benefited
substantially from the System
Safety Handbook development
work supported by
– Robert Youngblood, Idaho
National Laboratory (INL)
– Curtis Smith, INL
• The presentation has also
benefited from discussions with
my OSMA colleagues
– Michael Stamatelatos
– Frank Groen
– Alfredo Colon
2

3. Outline
• Background
• The Concept of Safety Thresholds and Goals
• Overview of the Risk-Informed Safety Case (RISC)
• Summary
3

4. BACKGROUND
4

5. Aerospace Advisory Safety Panel
(ASAP) Recommendation 2009-01-02a
• The ASAP recommends that NASA stipulate directly the HRR
acceptable risk levels—including confidence intervals for the
various categories of activities (e.g., cargo flights, human
flights)—to guide managers and engineers in evaluating “how
safe is safe enough.” These risk values should then be shared
with other organizations that might be considering the creation
of human-rated transport systems so that they are aware of the
criteria to be applied when transporting NASA personnel in
space. Existing thresholds that the Constellation Program has
established for various types of missions might serve as a
starting point for such criteria.
• NASA Accepted the recommendation and committed to
developing a safety goal policy for the human space flight.
5

6. THE CONCEPT OF SAFETY
TRESHOLDS AND GOALS
6

7. Safety Thresholds and Safety Goals
• In our response to ASAP we said
– Safety Goals are desirable safety performance levels for driving
safety improvements
– Safety Thresholds are criteria for risk acceptability decisions; not
meeting these values is not tolerable
– Both goals and thresholds are defined by Agency in terms of
aggregate risks
– The safety goal and threshold collectively
• Help designers with safety performance allocation
• Help decision makers to deal with safety-related decisions
- Risk acceptance
- Risk mitigation
- Safety optimization
7

8. Safety Regimes and Safety Decisions to
be Made
Standard of “Optimally and Sufficiently Safe” Standard of “Minimally Safe Level”
More than this May have diminishing return Less than this would be “intolerable”
GOAL TRESHOLD
Frequency Threshold
(to be met with ≥ X% probability)
SAFE ENOUGH TOLERABLE INTOLERABLE
Optimization Mitigation
Aggregate Frequency of Scenarios Leading to Loss of Crew
Increase in Decision Flexibility
• Keep alert for • Actively pursue safety • Don‟t proceed with
enhancements, but improvements via risk the acquisition
focus more on tradeoff studies • Fix design or
maintaining the good • Actively identify operation to meet
safety level that has unaccounted-for hazards via the threshold
been been achieved precursor analysis
8

9. OVERVIEW OF THE RISK-INFORMED
SAFETY CASE (RISC)
9

10. Practical Implications of Safety
Requirements Based on Risk Metrics
• We cannot “prove” ahead of time that the fraction of launch
failures in the limit of a large number of launches will be < X
• The „case” that P(event) < X needs to be supported by a
coherently-stated rationale providing both narration and
evidence that justifies the level of safety claimed
– Evidence includes operating experience, tests, integrated safety
analysis, etc.
• Risk-informed Safety Case (RISC): A documented body of
evidence that provides a convincing and valid argument that
the system is adequately safe
10

11. Risk-informed Safety Case (RISC)
• “Adequately safe for a given application in a given environment:”
– Safety Goal
– Other Safety Requirements
• To develop a safety case we need to:
– Make an explicit set of claims about the system(s)
• E.g., probability of accident is low
– Produce supporting evidence of sufficient caliber
• E.g., operating history, redundancy in design, …
– Provide a set of safety arguments that link claims to evidence
– Make clear assumptions and judgments underlying the claims
– Allow different viewpoints and levels of detail
• Part of the evidence comes from Probabilistic Safety Analysis (PSA):
Scenarios, Frequencies, Consequences
– Reliability aspects
– Phenomenology aspects (e.g., analysis of abort effectiveness)
– Operational and human error aspects
11

12. Pointillism vs. Coherent Safety Picture
System safety perspective
needs to be integrated and
coherent, as opposed to a
pointillistic portrayal of hazards
and controls
Pointillism is a style of painting in which small distinct points of primary colors
create the impression of a wide selection of secondary colors. The technique
relies on the perceptive ability of the eye and mind of the viewer to mix the
color spots into a fuller range of tones. Source: Wikipedia
12

13. The Coherent Case that Needs to be
Made to the Decision-Maker
13

14. Safety Case is a Basis for Decision-making and
a Roadmap for Implementation
• To decision makers, the safety case shows how the designers
have met their challenge, and why the design should be
approved
– It relates the design characteristics to the safety performance, and
shows what processes were followed
• To implementers (construction, manufacturing, installation,
maintenance), it shows what they have to do and how well they
have to do it
– What functions have to be maintained, what performance
allocations need to be satisfied
• To operators (astronauts, launch decision-makers), it shows
how to remain safe in flight
– It defines the operational envelope inside which operational
freedom is permitted
– Penetration of the envelope calls for changes to design or
operation, and/or reanalysis
14

15. Evolution of the Risk-Informed Safety Case (RISC)
over the Life Cycle
Safety Requirements Input to
Design Safety Case
Performed by
• Safety Goals
Acceptance
NASA
• Safety Requirements
• Technical Requirements • Very high confidence
Performed by • Process Requirements that system meets the
Provider • Analysis Protocols and tools for Safety threshold
Requirement Demonstration and Optimization • High confidence that
Performed jointly system is optimally
safe Deployment
Development of RISC
Optimization/
Risk
Management Trending of Safety
Develop & Justify Performance
Performance
Commitments
Design Integrated
Safety Analysis of Operating
Operation Experience / Precursor
Analysis
Risk-Informed Tolerable Analysis
Identification
of Hazards & Safety Case Region
Associate
Risks
Demonstrate
Identification of Satisfaction of
Controls Safety Requirements Intolerable
Based on Integrated
Safety Analysis /
Region Operating
Optimization
Optimization Input Experience Input
Performance Feedback

16. Raising the Bar for Safety Performance
(notional)
First Second group of Third group of
Flight Flights (2-5) Flights (6-10)
Optimization
RISC for
RISC for
Region
Flights
Flights > 10
RISC for
6-10
Flights
RISC for 2-5
First Flight
Tolerable
Region
Intolerable
Region
16

17. Role of System Safety in Developing the
RISC Key Claims of Safety Case
• Design Specification
• ConOps, Design intent, & Design
Safety Requirements specified “completely” for current
design phase
• Systematic Process to ID Hazards
• Comprehensive Hazard
Identification Process has been
Design, ConOps, … implemented based on
ConOps, Design Intent, Design
• Identified Hazards Controlled
• For each hazard, either a design
change has been made,
or
{(Hazardi, Control(s)i)} appropriate controls have been
identified for each hazard, and
resources have been allocated to
implement those controls
• Aggregate Risk OK
Risk ~ • Aggregate risk considerations are
{Scenarioi, Likelihoodi, Co satisfied, and there are no known
nsequencesi} additional cost-beneficial controls
or design modifications
17

18. Role of Scenario-Based Probabilistic Safety
Analysis (PSA) in Formulation of the RISC
• Probabilistic Safety Analysis” refers to a structured, probabilistic
treatment of scenarios, likelihoods, consequences
• Probabilistic Safety Analysis quantifies risk metrics
The Risk-informed Safety Case is not
the PSA Rather LOC
Probability
Mitigation/abort
Severity
(failure environment)
• The PSA is a thought process used to
Warning time
Type of environment
(LOC env)
Detection
guide formulation of the RISC S
Magnitude • Sensor
D
• Trigger value
Failure propagation Failure
(amplification, cascade,
• A convincing hazard analysis can help to
propagation
C
evolution) time
E • PDA
• SARA Operational
state (MET)
make the case that the problem is well
Active thermal control
E
system fails to provide
cooling (phase 3)
ATCS_TOP_P_3
C The ATCS fails (phase 8) Failure to provide power
(phase 3)
Electric power system
control fails (phase 3)
understood ATCS_FAILS ATCS_EPWS_POWER_P_3 ATCS_EPWS_CONTROL_P_3
N
I Heat collectors fail
(phase 8)
Heat transporters fail
(phase 8)
CEV ATCS CM
ATCS-HSNK-FTF-EVAP-P-3 1.09E-05
No power from electric
power system fails ATCS
No signal from Avionics
System fails ATCS
• The scenario set developed in the PSA
(phase 3)
ATCS_EPWS_BE_P_3 5.00E-06
(phase 3) 1.00E-04
ATCS_AVCS_BE_P_3
ATCS_HEAT_COLL ATCS_HEAT_TRANS CEV ATCS CM Coolant
ATCS-TANK-FTF-CM-P-3 1.09E-05
CEV ATCS CM heat
A
ATCS-HCOL-FTF-CCF-P-3 8.37E-06
CEV ATCS CM Heat CEV ATCS CM Heat
CEV ATCS CM heat
S
ATCS-HCOL-FTF-1-P-3 8.37E-05 ATCS-HTRN-FTF-1-P-3 3.64E-08
transport devices CCF
CEV ATCS CM Heat CEV ATCS CM Heat (phase 3) 3.64E-09
Pr( )
ATCS-HTRN-FTF-CCF-P-3
Collection Devices (Loop Transport Devices (Loop
2) (phase 3)8.37E-05 2) (phase 3)3.64E-08
can be used by designers to establish
ATCS-HCOL-FTF-2-P-3 ATCS-HTRN-FTF-2-P-3
R
T = 0.0 EDS, LSAM, and CEV in Depart Earth to Low LSAM Performs Lunar CEV In Lunar Orbit # End State
I
LEO Lunar Orbit Orbit Injection Injection (Phase - PH1)
LAUNCH PHASE_1 PHASE_2 PHASE_3 PHASE_4
what allocation of functional 1 MOON_TO_EARTH
Bayesian I
2 LOM_OR_LOC
capability, physical O 3
4
LOM_OR_LOC
LOM_OR_LOC
Probabilities
margin, redundancy, and element N If explosion then
5 LOM_OR_LOC
O
reliability can best satisfy safety targets S if launch_abort > 0.1 then
… else if
S
….
within real-world constraints ….
• Success paths credited to meet safety End if
targets are the appropriate conceptual
framework for narration of the safety Probabilistic Risk Analysis
Probabilistic Safety Analysis
case
18

19. SUMMARY
19

20. Summary
• Safety Goals and Thresholds will change the way in which System
Safety work is carried out and used. They:
– Are tools for implementing agency safety policy
– Will be used to guide design and system safety analysis
– Will play a key role in acquisition
• Safety goals and thresholds require integrated systems view of
system safety
• The major product of System Safety is the RISC
• The RISC is meant to show why a DM can have confidence in a
decision to proceed, and what has to be made to come true in order
to maintain that confidence
• The RISC brings together a diversity of evidence and analysis to
support a hierarchy of technical findings
– Integrates the traditional piece parts of System Safety processes
• High-level PSA results are part of the RISC, but PSA is not the sole
reason for confidence in the conclusion
– It is a tool for achieving integrated perspective
20

21. Summary (Cont.)
• To decision makers, the RISC shows how the designers have met
their challenge, and why the design should be approved
– It relates the design characteristics to the safety performance, and shows
what processes were followed
• To implementers, RISC shows what they have to do and how well they
have to do it
– What functions have to be maintained, what performance allocations need
to be satisfied
• To operators (astronauts, launch decision-makers), RISC shows how
to remain safe in flight
– It defines the operational envelope inside which operational freedom is
permitted
• Penetration of the envelope calls for changes to design or operation,
and/or reanalysis
• Changes in design or operation that would alter the RISC would alter
the basis for decisions, and correspondingly need review and re-
acceptance
21