The left pie chart says, at launch, on what we spent our money; the right pie chart shows the portion of total growth in cost between PDR and launch, and where the growth occurred.
In the prior studies, Development Cost Growth, excluding L/V and Phase E was 56% from ATP and 37% from PDR.
If MRO is excluded, Average at launch drops to 27%
Excluding EO-1 would lower the average at launch to 6%; EO-1 growth was from $18M to $32MNew Horizons growth was 41%, or growth from $173M to $243M. It should be noted, that at ATP, the expected cost of the L//V was unknown, and the $174M was a placeholder.
Excluding EO-1 reduces average at launch to 11%; excluding New Horizons, lowers it further to 8%
Excluding EO-1 reduces growth from 58% to 53%The point here is that at ATP (KDP-B), reserves of 58% on LCC were needed, not the 25% on development cost only that is the industry standard.
Excluding EO-1 reduces growth at launch from 45% to 41%Only 5 missions out of 20 kept their LCC growth below 25% between PDR and Launch: GRACE (12%), LRO (16%), GENESIS (17%), MRO (23%), NEW HORIZONS (25%)
41% of total cost means reserves at PDR should be around 41%/.85, or around 48%32% of total cost means reserves at CDR should be around 32%/.6, or around 53%
StDev at PDR = 8.2 monthsStDev at CDR = 6.5 months
Life Cycle Cost Growth Study20 Science Mission Directorate (SMD) Missions Presented at the 2011 NASA Program Management Challenge 9-10 February 2011, Long Beach, CaliforniaClaude Freaner, Science Mission Directorate, NASA HQ 1
Background • Previous studies1, 2 have primarily examined Development Cost Growth in an attempt to determine when in the development lifecycle the growth occurs • Growth was studied for 20 SMD Missions launched between 2000 and 2009. – Mass, Power, Cost, and Schedule were examined • Launch vehicle and Phase E operations costs (planned versus actual) have also grown but the amounts/phasing of this growth were not included in the prior studies – Current study includes LV and Phase E operations cost1. “Using Historical NASA Cost and Schedule Growth to Set Future Program and Project Reserve Guidelines”, Bitten R., Emmons D., Freaner C.,IEEE Aerospace Conference, Big Sky, Montana, March 3-10, 20072, “Optimism in Early Conceptual Designs and Its Effect on Cost and Schedule Growth: An Update”, Bitten R., Freaner C., Emmons D., 2010NASA Program Management Challenge, Galveston, Texas, February 9-10, 2010 2
Study Approach• For a set of 20 missions in the study, the cost data were obtained from all of the CADRes for missions at ATP (Phase B start), PDR, CDR, and Launch.• The cost data was “binned” into these Work Breakdown Structure (WBS) element categories: – PMSEMA: Project Management, Systems Engineering, Mission Assurance – SCI/EPO: Science, Education, Public Outreach – Payload – Bus/AIT: Spacecraft Bus, System Assembly Integration and Test – GDS/MOS: pre-launch Ground Data System, Mission Operations – L/V: Launch Vehicle – Phase E: Post-Launch Operations and Data Analysis• Cost data has not been adjusted for “Full Cost” effects – Primarily impacts STEREO, CALIPSO, FERMI The majority of these missions were in development prior to the CADRe requirement existing, so separation of the costs into the standard WBS Level 2 elements was difficult. Therefore, the elements were combined into the above bins. 3
Database Description: 20 Missions Represent a Wide Range of Recent NASA Missions Key Launch Acquisition Number of Planetary? Program Science Type Center(s) Year Type Instruments Comments Advanced land imaging technology EO-1 NMP Earth Science GSFC 2000 Competed 5 demonstrator Collect samples of solar wind particles at GENESIS X Discovery Planetary Science JPL 2001 Competed 4 L1 point and return them to Earth• 5 Directed vs. 15 GRACE ESSP Earth Science JPL 2002 Competed 6 Earth Gravity Measurement Competed missions Spitzer Physics of the Cosmos Astrophysics JPL 2003 Directed 4 IR space telescope, the last of the Great Observatories GALEX Explorers Astrophysics JPL/CalTech 2003 Competed 1 UV space telescope SWIFT Explorers Astrophysics GSFC 2004 Competed 4 Gamma Ray burst detector• 7 Planetary missions MESSENGER X Discovery Planetary Science APL 2004 Competed 7 Investigate Mercury vs. 13 Earth or near- MRO X MEP Planetary Science JPL 2005 Directed 7 Investigate history of water on Mars Earth Orbiters Deep Impact X Discovery Planetary Science JPL 2005 Competed 3 Comet impactor Cloudsat ESSP Earth Science JPL 2006 Competed 1 Radar observation of clouds 2 spacecraft looking at solar dynamics - STEREO STP Heliospheric Science GSFC/APL 2006 Directed 4• 7 Planetary Science Earth leading and trailing orbits CALIPSO ESSP Earth Science LARC 2006 Competed 3 Aerosols vs. 5 Astrophysics New X New Planetary Science APL 2006 Competed 7 Investigate Pluto Horizons Frontiers vs. 5 Earth Science DAWN X Discovery Planetary Science JPL 2007 Competed 2 Investigate Ceres and Vesta protoplanets vs. 3 Heliophysics AIM Explorers Heliospheric Science LASP 2007 Competed 3 Aeronomy of Ice in Mesosphere missions Fermi (GLAST) Physics of the Cosmos Astrophysics GSFC 2008 Directed 2 Gamma Ray Telescope Interaction between solar wind and IBEX Explorers Heliospheric Science GSFC 2008 Competed 2 interstellar medium Kepler Discovery Astrophysics JPL 2009 Competed 1 Search for Earth-sized exoplanets Robotic ESMD/Planetary LRO X GSFC 2009 Directed 7 Origin of the Moon Lunar Science Carbon Dioxide Investigation. Mission OCO ESSP Earth Science JPL 2009 Competed 1 failed due to launch vehicle failure 4
Composition of Average Life Cycle Cost & Cost Growth Category as Percent of Distribution of Growth of LCC Average LCC at Launch From PDR to Launch Phase E 6% SCI/EPO 3% SCI/EPO PMSEM PMSE 2% Phase E L/V 11% A MA 9% 7% 12% L/V GDS/ 17% MOS Payload 12% 23% Payload GDS Bus/AIT 32% 6% 29% Bus/AIT 31% 5
“Portfolio” % Average LCC Cost Growth Average Cost Growth by Major WBS, 200% Reserves Not Included (20 Missions) 180% 160% 140% Percent Growth 120% 100% 80% ATP to LRD 60% 56% PDR to LRD 40% 40% 20% 0% Largest Percent Growth for PMSEMA & GDS/OSGrowth = (Total LCC at Launch/Total LCC at KDP) -1Note: ATP is equivalent to KDP-B 6
Total Cost Growth ($) by Major WBS Element (20 Missions) 1,000,000 800,000 Total Growth = $2.1B 600,000 Total Growth = $1.6B 400,000Cost Growth in $K 200,000 ATP to LRD PDR to LRD - (200,000) (400,000) Total Cost at Launch = $7.6B (600,000) Reserves shown were planned to cover all growth (800,000) Largest Absolute Dollar Growth for Payload & Spacecraft 7
Planned Phase E Cost Growth from ATP* (Phase B start) CALIPSO Excluded from Average 400% Mission #1 Mission #2 350% Mission #3 Mission #4 300% Mission #5 Mission #6 250% Mission #7 Mission #8 200% Mission #9 Mission #10 150% Mission #11 Mission #12 Mission #13 100% Mission #14 Mission #15 50% 27% 34% Mission #16 11% Mission #17 0% 0% Mission #18 ATP PDR CDR Launch Mission #19 -50% Mission #20 Average -100% Majority of Phase E Growth Occurs Prior to CDR * Note: Reserves not included 20
Planned Phase E Cost Growth from PDR* CALIPSO Excluded from Average 400% Mission #1 350% Mission #2 Mission #3 300% Mission #4 Mission #5 Mission #6 250% Mission #7 Mission #8 200% Mission #9 Mission #10 150% Mission #11 Mission #12 Mission #13 100% Mission #14 Mission #15 50% Mission #16 13% 21% Mission #17 0% 0% Mission #18 PDR CDR Launch Mission #19 Mission #20 -50% Average-100% Majority of Phase E Growth Occurs Prior to CDR* Note: Reserves not included 21
LCC Growth from ATP* (Phase B start) (Includes Planned Phase E) 180% Mission #1 Mission #2 160% Mission #3 Mission #4 140% Mission #5 Mission #6 120% Mission #7 Mission #8 Mission #9 100% Mission #10 Mission #11 80% Mission #12 58% Mission #13 60% Mission #14 Mission #15 40% Mission #16 Mission #17 20% 20% Mission #18 9% Mission #19 0% Mission #20 0% Average ATP PDR CDR Launch -20% Majority of LCC Growth Occurs After CDR* Note: Reserves not included Growth = Average ((Mission LCC at Launch/Mission LCC at KDP)-1) 22
LCC Growth from PDR* (Includes Planned Phase E)140% Mission #1 Mission #2120% Mission #3 Mission #4 Mission #5100% Mission #6 Mission #7 Mission #880% Mission #9 Mission #1060% Mission #11 Mission #12 45% Mission #1340% Mission #14 Mission #15 Mission #1620% Mission #17 10% Mission #18 0% Mission #19 0% Mission #20 PDR CDR LRD Average-20% Majority of LCC Growth Occurs After CDR* Note: Reserves not included Growth = Average ((Mission LCC at Launch/Mission LCC at KDP)-1) 23
Reserves Standards• JPL Flight Project Practices, Rev. 7: – At PDR, budget reserves must be 25% of cost to go. – At CDR, budget reserves must be 20% of cost to go. – At start of ATLO, budget reserves must be 20% of cost to go.• GSFC “Gold Rules” – At PDR, budget reserves must be 25% of cost to go.•LCC growth from PDR to Launch averages 41% of Total Cost (reserves excluded), not cost to go•LCC growth from CDR to Launch averages 32% of Total Cost (reserves excluded), not cost to go 24
Probability that 25% Reserves at PDR Are Sufficient• Growth from PDR to Launch average is 41% (excluding EO-1) – Standard Deviation = 21% – Probability that a mission with 25% reserves at PDR will stay within those reserves by Launch = 22% – To achieve 70% CL, reserves of 47% would have been needed• Growth from PDR to Launch average is 45% (including EO-1) – Standard Deviation = 28% – Probability that a mission with 25% reserves at PDR will stay within those reserves by Launch = 23% – To achieve 70% CL, reserves of 52% would have been needed 25
So…Why Do We Have Cost Growth?• Over-optimism at the start – Propensity for proposers to be in marketing mode – Initial Mass estimates are low • Average Payload Mass Growth PDR-Launch = 77% – Initial Schedule estimates are short – Cost… Likely bid to the cost cap or available budget, not what it realistically takes• Cost estimators can use wider ranges on parameters for estimating the input values used for cost risk analysis – Current Cost risk process appears to be underestimating the resource growth• Schedule slips – Average Launch date slip from PDR plan: 13+ Months – Average Launch date slip from CDR plan: 10- Months – Majority of schedule slips occur during ATLO.• “Stuff” happens – Harder than we thought – Suppliers have problems – Things break – Congress/OMB/NASA HQ change funding profiles 26
What Can We Do to Decrease Cost Growth?• Proper scoping of projects early in conceptual design to provide executable program plans• Require better Basis of Estimate – Require proposers to show relevant actuals from prior missions at the subsystem level – “Relevant actuals” means Mass, Power, Cost, Schedule – Risk assessment and quantification of risk• Independent validation of instrument resources, and the resulting spacecraft resources needed to meet mission requirements, would allow more accurate estimates• Increase reserves is one possible solution – Reduces number of missions per year – Costs will likely rise to beyond the new reserves after a short time.• Incentivize contracts – Large rewards to Center/Team for performing to initial budget/schedule – Punishment • Easy to apply to Corporations: take away fee, cost share any overruns, etc. • Difficult to apply to NASA Centers 27
The author wishes to express sincere appreciation to Robert E. Bitten and Debra L. Emmons, The Aerospace Corporation, for their help in the preparation of this presentation.Electronic copies of this and the two previous studies referenced on slide 2 may be obtained by sending an email to email@example.com 28