1. When Failure Means Success:
Accepting Risk in Aerospace Projects
NASA Project Management Challenge 2009
Daniel L. Dumbacher, Director
Christopher E. Singer, Deputy Director
Engineering Directorate
Marshall Space Flight Center

2. Agenda
• Evolving from Saturn, to Shuttle, to Ares
• Expanding Frontiers for 50 Years and Counting
• Harnessing Risk Management Techniques and Tools
• Applying Human Space Flight Testing Philosophy
• Learning Lessons from the 1990s: Delta-Clipper Experimental Advanced
Demonstrator
• Learning Lessons from the 1990s: X-33 Single-Stage-to-Orbit Flight Demonstrator
• Transitioning from Shuttle to Ares: Hard-Won Lessons
• Reducing Shuttle Risk: HD Cameras Visualize ET Foam Loss
• Reducing Shuttle Risk: Main Engine Cutoff (ECO) Sensor
• Reducing Technical Risk for Ares I Crew and Ares V Cargo Launch Vehicles
• Systems Engineering Throughout the Project Lifecycle
• Testing for Knowledge versus Testing for Success
• Ares I Project Milestones
• Ares I-X Development Flight Test: Breaking the Systems Engineering Model
• Generating and Analyzing Data to Reduce Risk: Main Propulsion Test and
Integrated Ground Vibration Test
• Adopting Other Risk Reduction Methods: Project Lifecycle Management
• Engineering Knowledge Management System
• Conclusion: Reducing the Risk Inherent in the Human Exploration of Space
3/2/2009 5-397081 2

3. Evolving from Saturn, to Shuttle, to Ares
08 09 10 11 12 13 14 15 16 17 18 19 20 21 22
Moon Landing
Space Shuttle
by 2020
Operations through 2010 Expedition to Mars
Complete International
Space Station by
Honoring International
Partner Commitments
Ares I-X Test
Flight 2009
Ares I Rocket Ares V Rocket
First flight in 2015 First Flight in 2020
Lunar Crater
Observation and
Sensing Satellite
Lunar Reconnaissance Orbiter
Orion Crew Vehicle Development
Altair Lunar Lander Development
Ares I Rocket & V Rocket Development
Partnerships with Emerging Commercial Space Sector
Carry out Science Missions to Learn about our Planet, Solar System and Universe
Advance U.S. Technological Leadership in Aeronautics through Research
Provide Critical Capabilities to Support NASA's Missions
3/2/2009 5-397081 3

4. Expanding Frontiers for 50 Years and Counting
122 m
(400 ft)
Command/Service Altair
Module Lunar
Lander
91 m Lunar
Overall Vehicle Height, m (ft)
(300 ft) Lander Orion Crew Earth Departure
Exploration Vehicle Stage (EDS)
S-IVB (One J-2X engine)
(One J-2 engine)
Upper Stage
61 m (One J-2X engine)
(200 ft)
S-II External Tank
(Five J-2 engines) Two 5.5-Segment
Two four-Segment Reusable Solid
Reusable Solid Rocket Booster
30 m One 5-Segment (RSRBs)
Rocket Booster Reusable Solid
(100 ft) S-IC
(RSRB) Rocket Booster
(Five F-1 engines)
(RSRB) Core Stage
Orbiter (Six RS-68 Engines)
Three Main
0 Engines
Saturn V Space Shuttle Ares I Ares V
1967–1972 1981–Present First Flight 2015 First Flight 2020
Height: Height: Height: Height:
110.9 m (364 ft) 56.1 m (184.2 ft) 99.1 m (325 ft) 116.2 m (381.1 ft)
Payload Capability: Payload Capability: Payload Capability: Payload Capability:
44.9 metric tons (99,000 lbs) to TLI 25.0 mT (55,000 lbs) to LEO 25.5 mT (56,200 lbs) to LEO 187.7 mT (413,800 lbs) to LEO
118.8 metric tons (262,000 lbs) to LEO 71.1 mT (156,700 lbs) to TLI with Ares I
3/2/2009
62.8 mT (138,500 lbs) direct to TLI
5-397081 4

5. Harnessing Risk Management
Techniques and Tools
L 5
I
K 4
E
L
3
I
H
O 2
O
D 1
1 2 3 4 5
CONSEQUENCE
Likelihood Plus Consequence Equals Risk Level
3/2/2009 5-397081 5

6. Applying Human Space Flight Testing Philosophy
Considering the Crew Drives Levels of Analyses
3/2/2009 5-397081 6

7. Learning Lessons from the 1990s:
Delta-Clipper Experimental Advanced Demonstrator
Defining Hardware Limits through
Technology Development Flight Testing
3/2/2009 5-397081 7

8. Learning Lessons from the 1990s:
X-33 Single-Stage-to-Orbit Flight Demonstrator
Pushing the Limits of Technology
3/2/2009 5-397081 8

9. Transitioning from Shuttle to Ares:
Hard-Won Lessons
“The great liability of the engineer compared to men of
other professions is that his works are out in the open
where all can see them. His acts, step by step, are in
hard substance. He cannot bury his mistakes in the
grave like the doctors. He cannot argue them into thin
air or blame the judge like the lawyers. He cannot, like
the architects, cover his failures with trees and vines.
He cannot, like the politicians, screen his short-comings
by blaming his opponents and hope the people will
forget. The engineer simply cannot deny he did it. If his
works do not work, he is damned.”
— Herbert Hoover
U.S. Mining Engineer & Politician (1874 – 1964)
Applying 30 Years of Lessons Lived
3/2/2009 5-397081 9

10. Reducing Shuttle Risk:
HD Cameras Visualize ET Foam Loss
Regularly Scrubbing Requirements to Reflect Reality
3/2/2009 5-397081 10

11. Reducing Shuttle Risk:
Low-level Main Engine Cutoff (ECO) Sensor
Solving Potentially Critical Anomalies
3/2/2009 5-397081 11

12. Reducing Technical Risk for
Ares I Crew and Ares V Cargo Launch Vehicles
In House Upper Stage Design
and Vehicle Stack Integration
Designing for Life-Cycle Considerations:
Safety, Reliability, Affordability
3/2/2009 5-397081 12

13. Systems Engineering Throughout
the Project Lifecycle
Testing Philosophy Drives Failures
3/2/2009 5-397081 13
13

14. Testing for Knowledge Versus
Testing for Success
Pre-Phase A Phase A Phase B Phase C Phase D Phase E
Development Program SRR SDR PDR CDR
9
9
Mission
8 Operatio
8 ns
Orion II
System Qual
7 Tests/Develo
7 p Flight
Prototype
Orion I
6 Flight Demo
6 (Oper
Environ)
Prototype
TRL 5 Test Ares I-Y
5 (Rel Environ)
Component/Syst
4 em Ares I-X
4 Test
Component (Rel Environ)
3 Test
3 (Lab
Proof of Environ)
2 Concept
2 (Analytical
Test)
Application
1 Formulated
1
Basic
Principles
Technology Readiness Levels Drive Testing Objectives
3/2/2009 5-397081 14

15. Ares I Project Milestones
Ares I Project Milestones
Name FY08 FY09 FY10 FY11 FY12 FY13 FY14 FY15
Ares 1-X Ares 1-Y Or-1 Or-2 Or-3 Or-4
Ares I Flight Dates
Sep Mar Sep Mar Sep
July
IOC FOC
Upper Stage SDR PDR CDR DCR
HW Delivery Oct Aug Dec Apr
ISTA Ares I-Y
1yr 9mo Build Time GVT CF HF I Or-1 Or-2 Or-3 Or-4 Or-5
Prior to Delivery
Jan Feb Jun Dec Jun Dec Jun Dec
Flight & Integration Test Ares I-X HW
CDR to KSC
Ares 1-X
Mar Oct FS FS U/S Orion IGVT Testing
IVGVT
Empty Inert HW HW Complete Analysis
Complete
May
Dec Apr Jan Mar Jun
System-Level Tests Inform Major Engineering
Milestones & Validate Readiness
3/2/2009 5-397081 15

16. Ares I-X Development Flight Test:
Breaking the Systems Engineering Model
Validates Modeling and Simulation, and Tests Operations Concepts
3/2/2009 5-397081 16

17. Generating & Analyzing Data to Reduce Risk: Main
Propulsion Test & Integrated Vehicle Ground Vibration Test
Testing the Edges and Margins on the Ground
Verifies Design Performance & Validates Computer Models
3/2/2009 5-397081 17

18. Project Lifecycle Management Model:
Reducing Undefined, but Known, Risks
Manufacturing
Engineering
Manufacturing
People
Product
Engineering
Information
Platform
Concept
Engineering Technology Processes
Operation
s
Requirements
Decommissioning
Designing with the End in Mind
3/2/2009 5-397081 18

19. Engineering Knowledge Management System
Broad Range of Data
Resources/Types
Across the Enterprise
3/2/2009 5-397081 19

20. Conclusion: Reducing the Risk Inherent
in the Human Exploration of Space
Engineering is a great profession. There is
the satisfaction of watching a figment of the
imagination emerge through the aid of science
to a plan on paper. Then it moves to realization
in stone or metal or energy. Then it brings homes
to men or women. Then it elevates the standard
of living and adds to the comforts of life. This is
the engineer's high privilege.
— Herbert Hoover
U.S. Mining Engineer & Politician (1874 – 1964)
One Good Failure Is Worth a Thousand Successes
3/2/2009 5-397081 20