1. Discovery
New
National Aeronautics and Space Administration Frontiers
Improving Life-Cycle Cost Management of
Spacecraft Missions
Project Management Challenge
2010
Dennon Clardy, Program Manager
Discovery & New Frontiers
February 2010 Used with Permission
0

2. Discovery
Overview of Programs New
Frontiers
• Uncoupled program addressing high-priority science objectives in solar system exploration
• Frequent opportunities for science community to propose full investigations
• Fixed-price cost cap (cradle to grave), excluding the launch vehicle
• Full and open competition
• Principal Investigator-led project
• Established in 1992 • Established in 2003
• $425M cap per mission (FY09) • $625M cap per mission (FY09)
• Mission opportunities every 1.5 to 2 yr • Mission opportunities every 3 to 4 yr
• Open science competition for all solar • Addresses high-priority investigations
system objects, except for the Earth and identified by the National Academy of
the Sun Sciences
February 2010 1

3. Discovery
D&NF Science Mission Targets New
Frontiers
Kepler
– The Moon (Lunar Prospector, M3, GRAIL)
– Mars (Mars Pathfinder, ASPERA-3)
– Inner Planets (MESSENGER, Strofio)
Dawn
– Outer Planets (New Horizons, Juno)
– Comets (CONTOUR, Stardust, Deep Impact, EPOXI, NExT)
Stardust
– Asteroids (NEAR, Dawn)
– Interplanetary Space (Genesis) Mars Pathfinder
Genesis
– Extra-Solar System (Kepler)
MESSENGER
Juno
New Horizons
NEAR
Lunar Prospector
M3 on Chadrayaan-1
CONTOUR
ASPERA-3
on Mars Express
GRAIL
February 2010 2

4. Discovery
Program Organization New
Frontiers
Associate Administrator
for the
Phase B, C, D, E
Science Mission Directorate
SMD confirmation review at end of Phase B.
Agency PMC SMD AA makes confirmation decision.
D&NF PM becomes heavily engaged following
NASA Headquarters Program Director selection.
D&NF Program Executives D&NF
Program Mission
Mission Scientist s Program
Program Executive Scientist
Program Office D&NF
Program Manager
MSFC CMC
Mission Manager
Projects Principal
Investigators
NASA Center PMC or Programmatic Authority,
Appointed Board Responsibility, Reporting, &
Accountability
Project Manager
(Implementing Organization) Scientific Reporting &
Accountability
February 2010 3

5. Discovery
Study Impetus: Cost and Schedule New
Frontiers
Mission Cost Growth (percent of proposed cost)
60.0
Percent Cost Growth
50.0
40.0
30.0
Legend:
Historic Mission Data
20.0
Included in Study
10.0
0.0
-10.0
-20.0
Mission Schedule Growth (launch date slip)
50
45
Percent Cost Growth
40
Legend: 35
Historic Mission Data 30
Included in Study 25
20
15
10
5
0 0
0
February 2010 4

6. Discovery
Study Impetus & Research Plan New
Frontiers
“Assess the cost escapes that have occurred on recent D&NF missions.
Determine how the cost escapes are making it through our processes and
determine what reasonable things we can do to either prevent them or
manage them better.”
 Assess LCC growth at decision gates throughout the mission development
process, identifying causes affecting cost increases over the missions’ life
cycle
 Identify factors that contribute to the occurrence of unplanned costs and
significant mission cost cap overruns
 Based on findings, provide specific recommendations and implementation
plans to improve current processes and provide a greater level of insight to
make better-informed decisions throughout the mission life-cycle
Look for and address the “boulders” that are under the control of the program and project
February 2010 5

7. Discovery
Study Implementation New
Frontiers
Mission 1 Mission 2 Mission 3 Mission 4 Mission 5
Gate Gate Gate Gate Gate
Analysis Analysis Analysis Analysis Analysis
Milestone Timelines LCC & Development Histories Findings Matrices
INADEQUATE PLANNING FOR OPERATIONS / PHASE E Project
Mission Mgrs
OPTIMISTIC HARDWARE / SOFTWARE INHERITANCE and TECHNOLOGY READINESS ASSUMPTIONS Project
INEXPERIENCED PROJECT TEAM FOR PLANETARY MISSIONS Project
PE’s
INSUFFICIENT MANAGEMENT OF AND INSIGHT INTO CONTRACTOR TASKS/PERFORMANCE Project
TMCO
LACK OF OR INADEQUATE INTEGRATED PROJECT SCHEDULES Project
UNIQUE/SPECIAL TASKS/WORK OUTSIDE OF AREAS OF DEMONSTRATED EXPERTISE Project
Institutions PI’s
ADDITION OF NEW NASA REQUIREMENTS AFTER SELECTION Project
INSTABILITY IN NASA PROGRAM BUDGETS Project
NO LESSONS LEARNED FEEDBACK/FEEDFOWARD PROCESS Program
PM’s
INADEQUATE RISK MANAGEMENT PROCESS AT TRANSITION TO IMPLEMENTATION Program
INADEQUATE CONSIDERATION OF INDEPENDENT REVIEW TEAM FINDINGS AND RECOMMENDATIONS Program
INABILITY TO PROVIDE CREDIBLE COST ESTIMATES EARLY IN DEVELOPMENT Program
Contributors
List of Major Findings
Associated
Recommendations
February 2010 6

8. Discovery
Major Findings New
Frontiers
We did not find a “magic bullet” in any of our findings...
There is no single action that ensures cost and schedule control...
CONSIDERATION OF REVIEW TEAM FINDINGS
INEFFECTIVE MANAGEMENT STRUCTURE
INTEGRATED PROJECT SCHEDULES
INSUFFICIENT PROJECT INSIGHT
PROJECT TEAM INEXPERIENCE
INADEQUATE MISSION REPLANS
INADEQUATE PLANNING FOR OPERATIONS / PHASE E
HERITAGE and TECHNOLOGY ASSUMPTIONS
FAULT PROTECTION AND AUTONOMY
February 2010 7

9. Discovery
Consideration of Review Team Findings New
Frontiers
NASA commissions senior-level expert review panels, yet does not always
address (carefully consider and mitigate or refute) the panel’s conclusions and
recommendations
– In most instances, cost drivers had been identified as a weakness, finding, or deficiency
during reviews (source selection, milestone, or internal reviews)
– There was little evidence indicating a consistent approach to considering, analyzing, or
responding to the findings and recommendations
Review teams are there to help the project: project management needs to make the
best use of their knowledge and expertise
– Schedule sufficient time for review and analysis of issues (findings, issues, concerns, or
actions) from reviews
– Foster a project atmosphere that moves beyond “not invented here” to ensure that issues
are addressed
– Ensure follow-through with risks, liens or threats, or budget adjustments
– Document the disposition of all findings in a manner that facilitates resuscitating issues as
the situation changes
February 2010 8

10. Discovery
Ineffective Management Structure New
Frontiers
Ineffective management structure and unclear roles and responsibilities
resulted in cost and schedule impacts to missions
– Primarily occurred within projects involving multiple organizations
• Inconsistent project reporting and decision-making
• Unclear lines of technical authority
• Unconnected systems engineering across multiple organizations
• Unclear responsibility for system integration
– The management structure issues directly compounded the effects of other embedded
project issues (e.g., heritage and technology problems, cost control, prime contractor
inexperience, etc)
An effective Project and technical management structure should be validated and
continually reassessed
– Project formulation needs to ensure a clearly documented project organizational structure,
with a clear delineation of responsibilities and decision making authority
– Site visits should verify that the key parties clearly understand the organizational structure
and the decision process
– Reviews should include a briefing and assessment of the effectiveness of the Project
organization structure
February 2010 9

11. Discovery
Integrated Project Schedules New
Frontiers
The lack of a comprehensive, integrated Project schedule results in
uncoordinated activities, inefficiencies in resource management, and increased
costs
– Missing critical milestones and major events resulting in underestimated resources, and
insufficient data for tracking performance
– Missing logical relationships (interdependencies), or unidentified or incomplete critical paths,
resulting in underestimated resources, schedule delays, and poor decision making
– Multiple separate, uncoordinated schedules resulting in incomplete data for tracking
performance, missing logical relationships, and unidentified critical paths
Project management should ensure valid, comprehensive, Integrated Master Schedule
is in place on each project that forms the basis of and facilitates EVM reporting
– Document the proposed approach to developing and maintaining an integrated schedule,
including a description of roles and processes for integrating multiple schedules, if any, into a
single master schedule
– Use the Integrated Baseline Review (IBR) to confirm the content of the schedule, the
processes for maintenance, and its ability to support weekly, monthly, and annual reviews
and planning
– Perform a detailed review of the actual integrated schedule and associated processes as part
of project milestone reviews (PDR, CDR, SIR) and periodically throughout the life cycle
February 2010 10

12. Discovery
Insufficient Project Insight New
Frontiers
Insufficient project management and technical insight into contractor
performance results in poor communications, schedule delays, and technical
problems that manifest as cost over-runs
– Specifications, processes, and procedures that did not meet institutional standards due to
organization “cultural” differences or insufficient flow down of requirements
– Rework, retest, and waivers to hardware and software
– Hardware mishaps
– Additional personnel to perform the appropriate level of insight is added to correct issues
Project management should ensure that the structure and rationale for the approach to
the insight and oversight is documented and understood at all levels
– Document the oversight approach (including rationale and staffing levels) for appropriate
oversight based upon implementing center, prime contractor, and subcontractor/vendor
experience
– Include a clear plan for appropriate mission assurance oversight at the prime and
subcontractors/vendors
– Review and independently assess the effectiveness of their planned oversight approach at
major milestones
February 2010 11

13. Discovery
Project Team Inexperience New
Frontiers
Teams with major players with limited experience in planetary mission
development are a major contributor to program management issues resulting
in cost over-runs
– Complex or poorly-defined management structure, roles and responsibilities, and
communications
– Inadequate development schedules and implementation of performance measuring
techniques
– Inadequate performance oversight and configuration management: institute to prime and
prime to sub
– Inaccurate cost estimates, inadequate cost control and management of reserves
Given there may be strategic benefits to selecting “new” team members, the Project
and Program should assume a greater cost risk and operate with increased
insight/oversight for less experienced team members
– Assess the relevant experience for major partners and identify associated risks
– Develop mitigation plans, including mentoring and active sharing of lessons learned from
more experienced partners
– Identify and carry a cost threat, adjusted to reflect continuing team performance
– Review and independently assess the risks and associated mitigations at major milestones
February 2010 12

14. Discovery
Inadequate Mission Replans New
Frontiers
The impact of significant changes to mission scope, schedule, or funding
profiles were not sufficiently understood, resulting in unexpected cost
increases and schedule delays
– Program- and Project-driven changes, regardless of lifecycle timeframe, often result in
underestimates of the effects (i.e., duration and complexity) on the operations phase
• Replans during development tend to concentrate on Phase C/D, so impacts to operations receive
limited analysis and review
• Operations is often not well-defined early enough in the project development cycle to understand the
full impact of any changes
Program and Project management need to work together to ensure sufficient time to perform a
detailed assessment, develop a detailed revised baseline, and complete an independent
evaluation
– Carefully consider ancillary effects such as heritage assumptions (see Heritage and
Technology Assumptions)
– Specifically address any operational considerations and impacts (see Inadequate Planning
for Operations/Phase E )
– Document the cost and schedule impact, along with a complete description of all associated
drivers
February 2010 13

15. Discovery
Inadequate Planning for Operations/Phase E New
Frontiers
Phase E cost increases result from poorly scoped mission operations. Even
moderate yearly underestimates can present significant LCC impacts for
missions with long Phase E durations
– Early focus on hardware development schedules and cost (big ticket items)
– Underestimating the complexity of the operation, ignoring special requirements for a
particular mission type, or inadequate planning for sustaining engineering
Project Management should emphasize from the beginning the need for an full and
accurate life cycle cost estimate
– Emphasize the importance of a detailed operations concept that identifies plans for
equipment maintenance/replacement and parts obsolescence, longevity plan for
retaining/replacing key personnel
– Develop a Phase E staffing estimate by primary operations functional area that can clearly be
traced to and supported by the operations concept
– Review and independently assess the operations concept and staffing estimates, including
revisions to the operations concept driven by project replans
– Consider the risk of significant Phase D activities deferred into Phase E and the potential
impact to operations staffing
– Manage deferred Phase D work as development effort, including appropriate metrics and
schedules
February 2010 14

16. Discovery
Heritage and Technology Assumptions New
Frontiers
Optimistic hardware/software inheritance and technology readiness
assumptions cause significant cost and schedule growth in Phases C/D
– Heritage is used to justify decreased cost and risk, but can increase cost and risk
– Heritage is more than just a specific piece of hardware or software, it must be considered to
include all associated aspects (available documentation, knowledgeable personnel,
operational paradigm, spacecraft design, etc.)
– New or reapplied technology may be required to enable a mission
Project Management should realistically assess both the benefits and risks associated with
heritage and new technology, and ensure appropriate mitigations are identified and implemented
– Clearly demonstrate the relationship between heritage/technology rankings and cost/reserve
estimates
– Create a living reference for the relationship between heritage/technology rankings and
cost/reserve estimates, updated throughout formulation and implementation
– Capture high risk heritage/technology assumptions project risks and tracked for mitigation
actions
– Cost factors/threats should be updated after each review, particularly heritage reviews
– Review and independently assess the risks and associated mitigations at major milestones
– Select membership of SRB or internal review teamsto ensure members possess needed
areas of expertise
February 2010 15

17. Discovery
Fault Protection and Autonomy New
Frontiers
Missions underestimate the time and effort required to design, implement, and
test fault protection and autonomy (FPA) capabilities
– Developing an integrated hardware, software, and operations approach to FPA starting early
in the development life cycle
– Defining appropriate autonomy requirements and the proper level of fault protection
– Estimating resource requirements, including hardware (test beds)
Institutions, programs, and projects need to develop a better understanding of the cost
drivers associated with FPA to improve our processes and estimating capabilities
Project management needs to ensure FPA is addressed as a separate and equal
engineering discipline
– Emphasize the importance of a comprehensive FPA approach starting with the initial system
concept
– Develop an FPA development resource estimate that can clearly be traced to and supported
by the operations concept
– Define FPA to the same level of maturity as the hardware and software for the mission
concept throughout the mission life cycle
– Derive FPA from the operations concept, and implement a systems approach encompassing
hardware, software, and operations
February 2010 16

18. Discovery
Embedding and Impact Realization Timing New
Frontiers
Insight
Inexperience
Fault Prot.
Heritage/Tech.
Mitigation Actions must Mgt. Structure
Focus on pre CR period Phase E
in Project life cycle
Ability to descope content
Integrated Schedule
(cost) falls sharply post CR
Most Findings Most Impacts
Cost impacts Replan
Embedded pre CR Occurdominate Phase D
post CDR
Phase E
Phase A Phase B Phase C Phase D
(Operations & Data
(Concept Studies) (Preliminary Design) (Detailed Design) (Development)
Analysis)
Formulation Implementation
MRR Launch
SRR PDR/CRR/CR CDR FRR
LRR
Embedding Period Impact Period
February 2010 17

19. Discovery
Conclusion New
Frontiers
We did not find a “magic bullet” in any of our findings...
There are no surprises here...
except that we continue to have problems in these areas
So what do we do with the results?
– First, avoid the temptation to acknowledge and then put on a shelf along with most of the
previous cost growth studies
– Second, while there were no surprises, that does not mean there are no actions the
entire community can take to significantly improve performance against approved
budgets
– D/NF Program Office is implementing a fairly disciplined approach to
• Continually identify actions we can take to avoid the traps identified in the study
• Avoid the “checklist” mentality and ensure that project management activities and tools directly
contribute to planning, tracking, and controlling cost and schedule
– We are also available to engage in more detailed discussions concerning the study
approach, findings, and potential actions with groups inside and outside of the D/NF
community
The Discovery and New Frontiers Program Office Life Cycle Cost Study was performed under the
direction of Paul Gilbert (MSFC), led by Bryan Barley (MSFC), and supported by Kenny Mitchell
(MSFC-retired) and Marilyn Newhouse (CSC)
February 2010 18

20. Improving the Life-Cycle Cost Management of Discovery
Planetary Missions New
Frontiers
Supplemental Data and Backup
February 2010 19

21. Discovery
Acronyms New
Frontiers
AO Announcement of Opportunity PDS Planetary Data System
AR Acceptance Review PI Principle Investigator
ARR ATLO Readiness Review PLRA Program Level Requirements Appendix
ATLO Assembly, Test, and Launch Operations (to the Program Plan)
CDR Critical design Review PM Project Manager
CR Confirmation Review PO Program Office
CRR Confirmation Readiness Review PSD Planetary Science Division
CSR Concept Study Report RTG Radioisotope Thermoelectric Generator
D&NF PO Discovery and New Frontiers Program Office RM Resource Manager
DOE Department of Energy RY Real Year
DPI Deputy Principle Investigator SMD Science Mission Directorate
EM Engineering Model SRB Standing Review Board
EVM Earned Value Management SRR Systems Requirements Review
FPP Flight Practices and Procedures TMCO Technical, Management, Cost, and Other
FRR Flight Readiness Review WBS Work Breakdown Structure
FTE Full-Time Equivalent
FY Fiscal Year
IAT Independent Assessment Team
ICE Independent Cost Estimate
IRT Independent Review Team
LCC Life-Cycle Costs
LRR Launch Readiness Review
LV Launch Vehicle
MM Mission Manager
MOA Memorandum of Agreement
MRR Mission Readiness Review
PBR Project Budget Report
PCA Program Commitment Agreement
PDR Preliminary Design Review
February 2010 20

22. Discovery
Background
Generic Mission Development Process New
Frontiers
AO Development AO Implementation
Decision Gates are mission milestones at which
NASA makes decisions that commit funding to a
AO Release
project’s Life Cycle Cost (LCC):
 Announcement of Opportunity (AO)
Step 1 Step 2  Selection of Proposals – start of Phase A
 Selection of a project for development – start of Phase B
 Preliminary Design Review (PDR)
Competitive
 Confirmation Review (CR)
 Program Level Requirements Appendix (PLRA)
 Phase C/D/E design and mission-readiness reviews
Phase E
Phase A Phase B Phase C Phase D
(Operations & Data
(Concept Studies) (Preliminary Design) (Detailed Design) (Development)
Analysis)
Formulation Implementation
MRR Launch
SRR PDR/CRR/CR CDR FRR
LRR
6 – 24 mo 4 mo 6- 8 mo 6 – 12 mo 6 – 12 mo 24 – 36 mo 12 – 60+ mo
Science/Mission Selection led by SMD Program Implementation led by DiscoveryDiscovery PO
Program Implementation led by PO
February 2010 21

23. Discovery
Study Implementation
Issues Affecting Data Collection New
Frontiers
 Collection, analysis, and synthesis of data much more intensive than
anticipated
– Assumptions were made when necessary to correlate across missions, but
data still valid for identifying driving issues
 Lack of official/formal program documentation
– Little traceability to program-level decisions or direction (e.g., official letters,
documents)
– Inconsistent records of year-to-year or life-cycle phased cost commitments
and obligations to projects
 Understanding of common program operating principles
– Institutions (i.e., projects) not clear on definition of cost cap
Difficult to obtain and capture important/key aspects in the life of a mission
due to programmatic practices
February 2010 22

24. Discovery
High Level Summary of the Findings
Timing of the Embedding and Impacts New
Frontiers
Ability to descope content (cost) Cost impacts dominate Phase D
falls sharply post CR
Finding 1: Heritage/Technology
Finding 2: Insight
Finding 3: Phase E
Finding 4: Replan
Finding 5: Integrated Schedule
Finding 6: Fault Protection
Finding 7: Management Structure
Finding 8: Inexperience
Most findings embedded pre-CR Most impacts occur post-CDR
Phase E
Phase A Phase B Phase C Phase D
(Operations & Data
(Concept Studies) (Preliminary Design) (Detailed Design) (Development)
Analysis)
Formulation Implementation
MRR Launch
SRR PDR/CRR/CR CDR FRR
LRR
Embedding Interval Impact Interval
August 2007 23