National Aeronautics and Space AdministrationRisk Management during Integrated Systems ResearchTechnology DevelopmentPrese...
AbstractCo-Authors: Justin Hornback (former ERA RM), Gaudy Bezos-O’Connor (ERA DPM),Steve Hirshorn (ISRP SE&I Mgr), and Do...
Presentation Focus:    The Risk Management Challenge•   What is the appropriate risk management construct for an Aeronauti...
Presentation Outline•   What is Aeronautics Integrated Systems Research     – How does it differ from Aeronautics Fundamen...
What isAeronautics Integrated Systems Research?N+1, N+2 and N+3 reflect time periods for technology insertion into the air...
NASA Aeronautics Portfolio                                                         Integrated                             ...
Integrated Systems Research Program OverviewProgram Goal:Conduct research at an integrated system level on promising conce...
Traceability from National R&D Plan toERA Project Technical Challenges                                      National R&D P...
ERA Project Goals, Approach and Deliverables•    Project System-Level Performance Metrics:      • Simultaneous achievement...
ISRP and ERA Risk Management PlanImplement both Continuous Risk Management(CRM) and Risk Informed Decision Making (RIDM) –...
ISRP and ERA Project Risk Management Strategy•   Risk management at the program level will tend to be strategic and focuse...
Assessment of Risk in a TechnologyDevelopment Project•   Technology risks follow a different risk pattern than other types...
Technological Considerations For Risk Assessment     Considerations                                                       ...
ERA Project’s Risk Assessment Approach•   ERA has applied a tailored continuous risk management process that enable    ris...
ERA Project Phase 1 PortfolioContinuous Risk Management Process•   Risk Factors:     – Technical Risk        • Industry/OG...
Cost and                                                                                                     Schedule easy...
ERA Project Phase 1 PortfolioContinuous Risk Management Process•   Consequence Elements:    – Technical Risk       • Achie...
ERA Project Phase 1 PortfolioContinuous Risk Management Process•   Likelihood Criteria may be defined in either Generalist...
ERA Parent and Child Risks  •       All identified project risks start as Child Risks. Risks that apply to project level  ...
ERA Risk Reporting                                                       Rank Trend                                       ...
ERA Project Phase 1 Portfolio Weighting of RiskTranslating Child to Parent RisksERA Risk Assessment Matrix   ERA WBS      ...
ERA Project Phase 2 Portfolio Selection Criteria – Risk Posture• Example of Risk scoring                                  ...
Concluding RemarksOpening Question:• What is the appropriate risk management construct for an Aeronautics  Technology Deve...
Questions?•   Douglas Brown, ERA Risk Manager     Douglas.brown@nasa.gov     757.864.3515     LaRC•   Gaudy Bezos-O’Connor...
25
Upcoming SlideShare
Loading in …5
×

D brown gbezos_shirshorn

19,455 views

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
19,455
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
32
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • OK, a few thoughts on the utilization of Risk management in a Technology Development research project.  You may have hit upon some of these themes already in the charts, so the below can be used to enhance those items:Research, at its fundamental core, is the search for knowledge, the discovery of the unknown, and the validation of theories.  Research of technology is successful if knowledge is obtained, even if the technology doesn’t pan out. NPR 7120.8 outlines two broad paths of NASA research – R&D (Research & Development) and TD (Technology Development). R&D operates on the precepts above. TD is a little different – while still research, TD focuses on taking technologies that have shown promise during fundamental research and maturing them to the point that they can be demonstrated in relevant environments and within integrated systems.   A few differences between TD and R&D projects:                                                      Integrated Systems Research                                                                                 Fundamental Research     TRL                                               TRL 3-7; Uses TRL to track progress                                                                         TRL 1-4; Does not use TRL to track progress     Project Life Cycle                        Finite life with defined project termination date                                                   Long life with no project termination date     NPR 7120.8 Applicability          NPR 7120.8 Technology Development (TD)                                                              NPR 7120.8 Research & Technology (R&T)     Aircraft Generation                   N+1/N+2                                                                                                                     N+2/N+3     V&V Relevancy                         Relevant                                                                                                                        Not Relevant     Pedigree of Technology            Promising technologies with demonstrated pedigree through fundamental research.   Emergent/New technologies with no/little pedigree (starts with basic physics, but doesn’t start from scratch)     Risk & Risk Reduction                Integrated Systems research is to reduce risk of application of technology.          Fundamental research is to understand risk of application of the technology,  As research, the primary risks to a TD project are those that prevent the data from being collected.  However, being finite life projects with a defined termination date, completion of tasks within schedule also is a significant factor in TD risk management. TD projects share many common risk categories with Development projects, such as Technical, Cost, Schedule, Programmatic and Safety.  These are largely tracked the same way, utilizing a 5x5 Likelihood (L) vs. Consequence (C ) matrix.  However, the definitions of L and C may differ, in particular the the Technical Consequence. Risk acceptance may also differ between Research and Development projects.  Development projects, bringing a capability from concept to operational capability, has a constrained acceptance of risks.  Research projects are willing at accept grater risks.  TD projects fall somewhere in between, as they are research projects but constrained by limited schedule and budget. ISRP and ERA utilizes NPR 8000.4A for guidance on risk management, however 8000.4A was written primarily for development and operational projects, so tailoring of the guidance to make it applicable to research and technology development has been required.
  • * KPP goals are consistent with the National Plan for Aeronautics R&D Plan (2010) and in the context of NASA’s defined Subsonic Transport System-Level Metrics which are defined across a multiple of timeframes denoted as N+1 (2015-2020), N+2 (2020-2025), and N+3 (2025+) respectively, where N signifies the latest generation of aircraft currently in operation during that timeframe, e.g., relative to Boeing 777 with GE-90 engines that entered service in 1997. Currently the next generation aircraft in the N+1 (2015-2020), are expected to be further evolved tube-and-wing type configurations with engine installation under the wing, and enter into service during the next decade. The potential exists that in the N+2 timeframe (2020-2025 and beyond) an entirely different configuration concept must emerge and enter into service to meet national goals, perhaps first in a military transport role, followed by acceptance in the commercial fleet. multiple generations of future aircraft denoted available in future as N+1, N+2, and N+3, where N signifies latest generation of aircraft currently in operation;
  • Comment on how “issues” are also use this format for reporting, in addition to risk
  • D brown gbezos_shirshorn

    1. 1. National Aeronautics and Space AdministrationRisk Management during Integrated Systems ResearchTechnology DevelopmentPresented by:Douglas Brown, Environmental Responsible Aviation (ERA) Risk ManagerGaudy Bezos-O’Connor, ERA Deputy Project ManagerSteven Hirshorn, Integrated Systems Research Program Systems Engineering & Integration ManagerNASA Project Management (PM) Challenge 2012February 22-23, 2012www.nasa.gov
    2. 2. AbstractCo-Authors: Justin Hornback (former ERA RM), Gaudy Bezos-O’Connor (ERA DPM),Steve Hirshorn (ISRP SE&I Mgr), and Douglas Brown (ERA RM)The Environmental Responsible Aviation (ERA) Project, through system-level analyses, will select promisingN+2 vehicle and propulsion concepts and technologies to develop based on their potential benefit towardsimultaneously reducing fuel burn, noise and emissions.These concepts and technologies will then be matured and their performance will be evaluated at the systemand sub-system level in relevant environments.Risk management is a set of activities aimed at achieving success by proactively risk-informing the selectionof decision alternatives and then managing the implementation risks associated with the selected alternative.In technology development efforts the measure of success to apply risk management assessment criteria isnot easily defined. The ERA Risk Management Process developed a process to define and assess risks toachieving project success across a portfolio of independent sub-project technology development, that, whencombined are required to maximize likelihood of achieving ERA project goals. ERA project goals are to selectpromising N+2 vehicle and propulsion concepts and technologies based on their potential benefit towardsimultaneously reducing fuel burn, noise and emissions. The ERA risk management process added anotherlayer of risk assessment; identifying a contribution factor based on subject matter expert input for eachtechnology weighing each technology potential contribution to ERA goals. This was then applied to risksidentified to the ERA project. The additional weighting factor improves identification of the true impact ofproject risks to achieve goals compared to traditional risk management processes. This presentationdescribes the process used to develop the weighting factor and share ERAs experience duringimplementation. N+1, N+2 and N+3 reflect time periods for technology insertion into the aircraft fleet: •N+1: 2015-2020, N+2: 2020- 2025, N+3: 2025 + 2
    3. 3. Presentation Focus: The Risk Management Challenge• What is the appropriate risk management construct for an Aeronautics Technology Development Project responsible for maturing airframe and propulsion technologies from TRL 3 to 5/6 through Integrated Systems Research?• Factors: – Cultural and Project Execution Paradigm Shift from Fundamental Research to Technology Development – Technology Development Projects are finite life, not enduring – Geographically dispersed government team – High Degree of Industry/OGA Partnerships/Collaborations with significant costshare across Project Portfolio (FY10-12) – Varied or limited experience with project risk management outside of airworthiness risks for aeronautics flight research projects. – The large number of independent tasks part of the portfolio of ERA • No project integrated critical path – Must balance portfolio content versus schedule margin and budget reserves • Budget reserves address prototype test article challenges and unique capability investments. • Schedule margin address technical risks and facility challenges 3
    4. 4. Presentation Outline• What is Aeronautics Integrated Systems Research – How does it differ from Aeronautics Fundamental Research?• Overview of ARMD, ISRP and ERA Project• Aeronautics National Goals and ERA Project Technical Challenges• ERA Project Goals, Approach and Deliverables• ERA’s Risk Informed Decision Making Strategy 4
    5. 5. What isAeronautics Integrated Systems Research?N+1, N+2 and N+3 reflect time periods for technology insertion into the aircraft fleet: •N+1: 2015-2020 •N+2: 2020- 2025 •N+3: 2025 + 5
    6. 6. NASA Aeronautics Portfolio Integrated Systems Research ProgramFundamental Aeronautics Program Airspace Systems ProgramConduct cutting-edge research that will Conduct research at an integrated Directly address the fundamental ATMproduce innovative concepts, tools, and system-level on promising concepts and research needs for NextGen by dev-technologies to enable revolutionary technologies and explore/assess/demonstrate eloping revolutionary concepts,changes for vehicles that fly in all the benefits in a relevant environment capabilities, and technologies thatspeed regimes. will enable significant increases in the capacity, efficiency and flexibility of the NAS. Aviation Safety Program Conduct cutting-edge research that will produce innovative concepts, tools, and technologies to improve the intrinsic safety attributes of current and future aircraft. Aeronautics Test Program Preserve and promote the testing capabilities of one of the United States’ largest, most versatile and comprehensive set of flight and ground-based research facilities. 6
    7. 7. Integrated Systems Research Program OverviewProgram Goal:Conduct research at an integrated system level on promising conceptsand technologies and demonstrate the benefits in a relevantenvironmentEnvironmentally Responsible Aviation (ERA) ProjectExplore and assess new vehicle concepts and enablingtechnologies through system-level experimentation tosimultaneously reduce fuel burn, noise, and emissionsUnmanned Aircraft Systems (UAS) Integration in the NationalAirspace System (NAS) ProjectContribute capabilities that reduce technical barriers related to thesafety and operational challenges associated with enabling routineUAS access to the NAS 7
    8. 8. Traceability from National R&D Plan toERA Project Technical Challenges National R&D Plan Energy and Enhance Mobility National Security Environment ERA Project Goals: Simultaneous Achievement of the NASA Subsonic Transport System-Level Metrics (N+2 Timeframe) -75% LTO & -70% Cruise -42dB below Stage 4 -50% Aircraft Fuel/ NOx Emissions Community Noise Energy Consumption below CAEP6 ERA Project Technical Challenges (FY10-15) Advanced Airframe & Engine Innovative Flow Advanced Advanced UHB Combustors for Integration for Control Concepts Composites for Engines for SFC & LTO Oxides of Community Noise for Drag Reduction Weight Reduction Noise Reduction Ni reductions Reduction 8
    9. 9. ERA Project Goals, Approach and Deliverables• Project System-Level Performance Metrics: • Simultaneous achievement of the community noise, emissions and fuel burn metrics defined in the NASA Subsonic Transport System Level Metrics in the N+2 timeframe• Project Approach: • Combine rigorous systems analysis with large-scale, integrated systems research demonstrations of promising airframe and propulsion technology solutions to TRL 5/6 by 2015 • Increase the viable trade space of vehicle configurations that can simultaneously meet the goals• Project Definition: • 6-year life; • 2 Phases: Phase 1 (FY10-12); Phase 2 (FY 13-15)• Project Deliverables: Key Performance Parameters (KPPs) • Technology Readiness Level (TRL) Maturation Maps • Product Transition Opportunities: Technology Transition Maps • Vehicle-level system metrics to measure progress towards the Project Goals • Technical data to validate/enhance system and physics-based assessment tools 9
    10. 10. ISRP and ERA Risk Management PlanImplement both Continuous Risk Management(CRM) and Risk Informed Decision Making (RIDM) – 8000.4A, Agency Risk Management Procedural Reqs. – 7120.08, NASA R&T Program & Project Mgmt. Reqs. • Research and Technology Risk Management Propulsion – Flight hardware focused Airframe, Aeroacoustic, and – Risk management must balance the need Propulstion Airframe to conduct challenging technology AeroAcoustic Wind Tunnel Testing development that will realize significant gains. Lightweight Integrated Structures Testing Flight Testing and Advanced Vehicle Concepts Advanced Combustor Testing Advanced Propulsor Testing 10
    11. 11. ISRP and ERA Project Risk Management Strategy• Risk management at the program level will tend to be strategic and focused on ensuring the success of the projects, while risks that focus on the tactical technical / cost / schedule execution risks will be largely managed at the project- level with program insight. – ISRP Projects will manage their technical performance, schedule, and cost risks according to their Risk Management Plans. – Significant project risks or risks requiring resources beyond those available to the Project will be “Tracked” by or “Elevated” to the Program.• A common frame of reference for Likelihood and Consequence (L&C) exists between ISRP and its Projects and across Projects: – Stems from the need to be able to reference both Program and Project risks in an apples-to-apples comparison when communicating risks to the ISRP Program Director and to the Mission Directorate. – Enable traceability of risks from Project task level to ISRP Programmatic Risks• In order to ensure both Program and Project processes remain in sync and remain aware of risks at both levels, frequent good communications must be maintained . 11
    12. 12. Assessment of Risk in a TechnologyDevelopment Project• Technology risks follow a different risk pattern than other types of risks• The level of maturity for a technology affects the risk profiles of that technology, i.e. lower maturity levels come with higher risks• The technology landscape is constantly changing with new technologies coming online promising increased performance. Risk assessment must weigh the promise of new performance against the confidence of what has been done before• These considerations do not require us to change likelihood and consequence criteria but consider the lens through which we view these scales 12
    13. 13. Technological Considerations For Risk Assessment Considerations Rating ERA Likelihood Ratings Technology Maturity Support Base Value Technical Some Research Completed/ Never Done No Other Program Developing Similar Before Technology 5 Very High PTCS > 75% New Design Based On Existing One Other Program Developing Similar Technology Technology 4 High 50% < PTCS <= 75% More Than One Program Developing Major Redesign Of Existing Technology Similar Technology 3 Moderate 20% < PTCS <= 50% Minor Redesign A Few Parallel Programs 2 Low 5% < PTCS <= 20% Existing Multiple Parallel Programs 1 Very Low PTCS <= 5% Rating 5 4 3 2 1 ERA Risk Major impact to Moderate impact to Some impact to Minor impact to Negligible or no Ratings achievement of achievement of achievement of achievement of impact to Subsonic Transport Subsonic Transport Subsonic Transport Subsonic Transport achievement of Consequences System Level Metrics, System Level System Level System Level Subsonic Transport Technical Metrics, Technical Metrics, Technical Metrics, Technical System Level Deliverables, and KPP Deliverables, and Deliverables, and Deliverables, and Metrics, Technical Goals KPP Goals KPP Goals KPP Goals Deliverables, and KPP Goals Contingency No Acceptable Some Possible Single Acceptable A Few Known Several Acceptable Solutions Alternatives Alternatives Alternative Alternatives AlternativesConsiderations Reliability Factor Reliability May Not Fairly Confident Highly Confident Fairly Confident Highly Confident Be Increased Reliability Will Reliability Will Reliability Will Reliability Will Increase Somewhat Increase Somewhat Increase Increase Significantly Significantly 13
    14. 14. ERA Project’s Risk Assessment Approach• ERA has applied a tailored continuous risk management process that enable risk- informed decision making – For Phase 1 Portfolio (FY10-12) – For Phase 2 Portfolio Development in FY12 for authorization to proceed through a Key Decision Point R(KDP) Review Process• Risk Management Process defined: – Risk Factors for technical, cost and schedule – Consequence and Likelihood Definitions and Scoring – Parent-Child Risk Construct – ERA Project Risk Reporting – ERA Risk Factor Weighting • Current Phase 1 Portfolio • Assessment of Phase 2 Portfolio Opportunities 14
    15. 15. ERA Project Phase 1 PortfolioContinuous Risk Management Process• Risk Factors: – Technical Risk • Industry/OGA contributions • Technical Complexity (Test Article, Experiment/Test) • Technical Benefit /System Impact – Cost Risk: • Workplan cost estimate maturity/fidelity • Technical Complexity • Workplan resource availability • Acquisition/Procurement cost • Industry/OGA collaboration dependency – Schedule Risk: • Workplan maturity/WBS fidelity • Technical Complexity • Workforce resource availability • Facility/Flight test asset availability • Acquisition/Procurement schedule • Industry/OGA collaboration dependency 15
    16. 16. Cost and Schedule easy ERA Risk Consequence Criteria to quantify and assess. ERA Risk Ratings ConsequencesRating Decision/Value Technical Cost Schedule Communication Notification Level 1 (APGs) any impact Greater than 20% Integrated Major impact to achievement of increase over that Level 2 Milestone(s): Systems Aeronautics Subsonic Transport System Level 5 Metrics, Technical Deliverables, allocated budget (Sub- < 1 month impact Research Research Mission Project, Element or Program (ISRP) Directorate and KPP Goals Level 3,4 Milestone(s): ≤ 1 Task level) & Centers month impact Moderate impact to achievement of Between 15% and 20% Level 2 Milestone(s): < 1 month impact ERA Project Subsonic Transport System Level increase over allocated 4 Metrics, Technical Deliverables, budget (Sub-Project, Management ISRP & Centers Level 3,4 Milestone(s): ≤ 1 (PM) and KPP Goals Element or Task level) month impact Some impact to achievement of Between 10% and 15% Level 2 Milestone(s): < 1 month impact Subsonic Transport System Level increase over allocated 3 Metrics, Technical Deliverables, budget (Sub-Project, ERA PM Centers Level 3,4 Milestone(s): ≤ 1 and KPP Goals Element or Task level) month impact Minor impact to achievement of Between 5% and 10% Level 2 Milestone(s): ERA Sub- < 1 month impact ERA Project Manger Subsonic Transport System Level increase over allocated Project 2 Metrics, Technical Deliverables, budget (Sub-Project, Managers (PM)/ Deputy PM Level 3,4 Milestone(s): ≤ 1 (DPM) and KPP Goals Element or Task level) month impact (SPM) Negligible or no impact to Between 0% and 5% Level 2 Milestone(s): < 1 month impact achievement of Subsonic Transport increase over allocated DPM, Element & 1 System Level Metrics, Technical budget (Sub-Project, SPM Task Leads Level 3,4 Milestone(s): ≤ 1 Deliverables, and KPP Goals Element or Task level) month impact Challenge to define Negligible, Minor, 16 Moderate, and Major impact
    17. 17. ERA Project Phase 1 PortfolioContinuous Risk Management Process• Consequence Elements: – Technical Risk • Achievement of Subsonic Transport System Level Metrics, • Technical Deliverables, • Key Performance Parameters (KPPs) – Technical Challenges – TRL Maturation – Technology/Product Transition Roadmap – Cost Risk : • % increase over that allocated budget (Sub-Project, Element or Task level) – Schedule Risk: • Level 1 Milestones: ISRP Program Level • Level 2 Milestones: Project Level • Level 3,4 Milestones: Sub-Project and Task Level 17
    18. 18. ERA Project Phase 1 PortfolioContinuous Risk Management Process• Likelihood Criteria may be defined in either Generalist terms or Probabilistic terms: • Generalist: Improbable  Unlikely  May  Likely  Very Likely • Probabilistic: % or occurrence, Probability distribution (i.e. 10-5), etc. 18
    19. 19. ERA Parent and Child Risks • All identified project risks start as Child Risks. Risks that apply to project level metrics are then elevated to the Parent level and are then managed by the Risk Management Board. – Child risks reported to Risk Management board, votes to make the risk a parent based on the risks assessment at the project level. – Allows risk owner at the sub-project, element or task level to assess the risk in terms of their approved plan. – Allows ERA project management to assess the risk at the project level where they have a better understanding of the entire project • Preventing filtering or tweaking of the risk at the sub-project/element/task level. ID WBS Parent Parent Risk Trend # Open Date Risk Title Risk Statement L C Affinity Group Owner Given the high cost and schedule uncertainty Schedule (3)1.00 PM Y ERA Schedule 68 14-Sep-11 ERA Schedule inherent to technology development projects, 4 3 Cost (3) Fay Collier 5 and Resources and Resources there is a possibility that planned funding and Tim Warner schedules will not support meeting ERA goals. Given that the large wind tunnel facilities at Schedule (3)3.0.0, PT N ERA Schedule 6 15-Jun-11 Shared GRC share personnel and services, there is a 5 3 Cost (3) Ken3.1.3, and Resources Personnel & possibility of schedule conflicts arising between Suder g3.3.4, Services at GRC ERA and other test programs (e.g., FAP: SUP, Facilities FAP: HYP), resulting in potential ERA propulsion test schedule slips. Given the expectation for Continuing Technical (3)1.00 PM N ERA Schedule 24 15-Jun-11 Continuing Resolutions during FY12, there is possibility 5 3 Cost (3) Fay Collier and Resources g Resolutions that ERA technology development activities will Schedule (3) be delayed, resulting in schedule slips and loss of productivity 19
    20. 20. ERA Risk Reporting Rank Trend Risk ID # Affinity Group Approach (M,W,A,R) Risk Title Denotes assessment and number of “Child” Discrete Roughness Elements Cost (5) risks. Risk Matrix 1 h 11 Schedule (5) Technical (5) M Laminar Flow Glove Experiment (DRE) Programmatic Planning and Control Pultruded Rod Stitched Efficient 5 h 11,13 Cost (3) Unitized Structure (PRSEUS) 2 13 W Schedule (5) Programmatic Estimates and L Control I 4 68, 69 K E 3 g 65 Technical (5) Schedule (5) M Combustor Development and Test L 3 .. .. ... 65,66,76,77 UHB Geared Turbo Fan Engine g I 4 66 Technical (5) M Development Noise H Characteristics O 67 O 2 .. ....... D 5 g 76 Schedule (5) R ERA Key Decision Point (KDP) Schedule 1 64 6 5 77 Schedule (5) R Flow Control Experiment for AFC Rudder 1 2 3 4 5 CONSEQUENCE 7 g 68 Schedule (3) Cost (3) A ERA Schedule and ResourcesNote: The numbers on the risk matrix refer to the Risk ID numbers.Criticality L x C Trend Approach 8 g 69 Technical (3) M ERA Technical Challenges High Decreasing (Improving) M - Mitigate Insufficient Resources to Mature Increasing (Worsening) W - Watch Med A - Accept 9 i 67 Technical (3) R Vehicles Concepts and Associated Technologies to Simultaneously Unchanged meet ERA Goals R - Research Low New Since Last Period 10 i 64 Schedule (5) W Hybrid Wing Body (HWB) Community Noise Assessment Project “Parent” Risks 20
    21. 21. ERA Project Phase 1 Portfolio Weighting of RiskTranslating Child to Parent RisksERA Risk Assessment Matrix ERA WBS Sub- Level Matrix Title project Resource Milestone Milestone Demonstrate low-weight, damage-tolerant stitched composite structural Complete Noise concept on curved panel subjected to combined tension and internal Transmission 02.01.4 AT 1 pressure loads.(COLTS Large Scale Pressurized Fuselage Test Assessment of Complete) (Proposed FY12 APG)Complete PRSEUS panels KPP Technical Technical Subsonic Transport Technical Deliverable and Milestone Deliverable KPP Goal APG Challenge Maturation System Level Metric Validation Method Cost Weight Report development of predictive noise transmission models N/A N/A Yes N/A .75 for like structural concepts Very Low Low Medium High Very High 5 10 16 20 23 25 4 7 13 18 22 24 3 4 9 15 19 21 2 2 6 11 14 17 Very Low Low Medium High Very High 1 1 3 5 8 12 5 8 12 15 17 19 1 2 3 4 5 4 5 10 14 17 18 3 3 7 11 14 16 2 2 5 8 11 13 Weighted table, based 1 1 2 4 6 9 on WBS task weight. 1 2 3 4 5 *Values roundedIn this example, if this risk was assessed at a Likelihood of 4 and Consequence of 5 by the task lead. LxC: 4 x 5. The riskwould be scored at 24, red, at the ERA project level. By creating the weighting factor to reflect the task’s contribution to ERAgoals, the risk would be scored at LxC: 4x3 as the likelihood would not change, only the potential consequence. This allows thetask lead to asses the risk based on their understanding of their task and creates a tool for the ERA risk manager a quantifiable 21and traceable method to accurate assess task risks and communicate at the project level.
    22. 22. ERA Project Phase 2 Portfolio Selection Criteria – Risk Posture• Example of Risk scoring Low = 5, Medium = 3, High = 1 • Supporting rationale will be developed for each Low, Medium or High score Technical Risk Score (1/3 Cost Risk Score (1/3 weighting) Schedule Risk Score (1/3 weighting) weighting)RISK Workf Industr Techn Facility ITD Industry Facility ITD ITD Industry orce y Procur- Procure- Workforce ical Cum Availabil Comple Compon Availabi Complex Compl Compon Avail Compo ement ment Availability Benefi Score ity xity ent lity ity exity ent ability nent t(Weightin 10% 25% 10% 20% 35% 25% 10% 15% 20% 30% 25% 35% 40% 100% g)ITD #1 Low Low Low Low Low High High High High High High High High 4.8ITD #2 Low Low Low Low Low Low Low Low Low Low Low Low Low 5.0ITD Medi #3 um High High High High Low Low Low Low Low Low Low Low 2.8ITD #4 Low Low Low Low Low Low Low Low Low Low Low Low Low 4.6ITD# Mediu Mediu Mediu 5 High High m m m Low Low Low Low Low Low Low Low 2.5ITD# Mediu 6 High m Low Low Low Low Low Low Low Low Low Low Low 2.7ITD# Mediu Mediu Mediu Medi Medi Mediu 7 Low Low Low Low Low m Medium Medium m m um um m 5.0ITD# 8 Low Low Low Low Low Low Low Low Low Low Low Low Low 4.4ITD# Medi 9 um Low Low Low Low Low Low Low Low Low Low Low Low 4.2ITD# Mediu Mediu 10 Low Low m Medium m Low Low Low Low Low Low Low Low 3.9 22
    23. 23. Concluding RemarksOpening Question:• What is the appropriate risk management construct for an Aeronautics Technology Development Project responsible for maturing airframe and propulsion technologies from TRL 3 to 5/6 through Integrated Systems Research?Lessons Learned to date:• Tailoring Risk Management Processes to Aeronautics Integrated Systems Research Technology Development poses different challenges than spaceflight development• The technology landscape is constantly changing with new technologies coming online promising increased performance. Risk assessment must weigh the promise of new performance against the confidence of what has been done before• The realization of a risk is not failure, the knowledge gained identifies new foci for integrated systems research to continue the technology’s maturation path to enable technology transition into an aircraft system.• These considerations do not require us to change the Project’s likelihood and consequence risk criteria but consider the lens through which we view these scales 23
    24. 24. Questions?• Douglas Brown, ERA Risk Manager Douglas.brown@nasa.gov 757.864.3515 LaRC• Gaudy Bezos-O’Connor, ERA Deputy Project Manger Gaudy.m.bezos-oconnor@nasa.gov 757-864-5083 LaRC• Steven Hirshorn, Integrated Systems Research Program Systems Engineering & Integration Manager Steven.r.hirshorn@nasa.gov 202.358.0775 HQ 24
    25. 25. 25

    ×