• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Dennon.clardy
 

Dennon.clardy

on

  • 9,583 views

 

Statistics

Views

Total Views
9,583
Views on SlideShare
9,583
Embed Views
0

Actions

Likes
0
Downloads
3
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Dennon.clardy Dennon.clardy Presentation Transcript

    • Discovery New National Aeronautics and Space Administration Frontiers Improving Life-Cycle Cost Management of Spacecraft Missions Project Management Challenge 2010 Dennon Clardy, Program Manager Discovery & New FrontiersFebruary 2010 Used with Permission 0
    • DiscoveryOverview of Programs New Frontiers • Uncoupled program addressing high-priority science objectives in solar system exploration • Frequent opportunities for science community to propose full investigations • Fixed-price cost cap (cradle to grave), excluding the launch vehicle • Full and open competition • Principal Investigator-led project • Established in 1992 • Established in 2003 • $425M cap per mission (FY09) • $625M cap per mission (FY09) • Mission opportunities every 1.5 to 2 yr • Mission opportunities every 3 to 4 yr • Open science competition for all solar • Addresses high-priority investigations system objects, except for the Earth and identified by the National Academy of the Sun SciencesFebruary 2010 1
    • DiscoveryD&NF Science Mission Targets New Frontiers Kepler – The Moon (Lunar Prospector, M3, GRAIL) – Mars (Mars Pathfinder, ASPERA-3) – Inner Planets (MESSENGER, Strofio) Dawn – Outer Planets (New Horizons, Juno) – Comets (CONTOUR, Stardust, Deep Impact, EPOXI, NExT) Stardust – Asteroids (NEAR, Dawn) – Interplanetary Space (Genesis) Mars Pathfinder Genesis – Extra-Solar System (Kepler) MESSENGER Juno New Horizons NEAR Lunar Prospector M3 on Chadrayaan-1 CONTOUR ASPERA-3 on Mars Express GRAILFebruary 2010 2
    • DiscoveryProgram Organization New Frontiers Associate Administrator for the Phase B, C, D, E Science Mission Directorate SMD confirmation review at end of Phase B. Agency PMC SMD AA makes confirmation decision. D&NF PM becomes heavily engaged following NASA Headquarters Program Director selection. D&NF Program Executives D&NF Program Mission Mission Scientist s Program Program Executive Scientist Program Office D&NF Program Manager MSFC CMC Mission Manager Projects Principal Investigators NASA Center PMC or Programmatic Authority, Appointed Board Responsibility, Reporting, & Accountability Project Manager (Implementing Organization) Scientific Reporting & AccountabilityFebruary 2010 3
    • DiscoveryStudy Impetus: Cost and Schedule New Frontiers Mission Cost Growth (percent of proposed cost) 60.0Percent Cost Growth 50.0 40.0 30.0 Legend: Historic Mission Data 20.0 Included in Study 10.0 0.0 -10.0 -20.0 Mission Schedule Growth (launch date slip) 50 45 Percent Cost Growth 40Legend: 35 Historic Mission Data 30 Included in Study 25 20 15 10 5 0 0 0February 2010 4
    • DiscoveryStudy Impetus & Research Plan New Frontiers “Assess the cost escapes that have occurred on recent D&NF missions. Determine how the cost escapes are making it through our processes and determine what reasonable things we can do to either prevent them or manage them better.”  Assess LCC growth at decision gates throughout the mission development process, identifying causes affecting cost increases over the missions’ life cycle  Identify factors that contribute to the occurrence of unplanned costs and significant mission cost cap overruns  Based on findings, provide specific recommendations and implementation plans to improve current processes and provide a greater level of insight to make better-informed decisions throughout the mission life-cycle Look for and address the “boulders” that are under the control of the program and projectFebruary 2010 5
    • DiscoveryStudy Implementation New Frontiers Mission 1 Mission 2 Mission 3 Mission 4 Mission 5 Gate Gate Gate Gate Gate Analysis Analysis Analysis Analysis Analysis Milestone Timelines LCC & Development Histories Findings Matrices INADEQUATE PLANNING FOR OPERATIONS / PHASE E Project Mission Mgrs OPTIMISTIC HARDWARE / SOFTWARE INHERITANCE and TECHNOLOGY READINESS ASSUMPTIONS Project INEXPERIENCED PROJECT TEAM FOR PLANETARY MISSIONS Project PE’s INSUFFICIENT MANAGEMENT OF AND INSIGHT INTO CONTRACTOR TASKS/PERFORMANCE Project TMCO LACK OF OR INADEQUATE INTEGRATED PROJECT SCHEDULES Project UNIQUE/SPECIAL TASKS/WORK OUTSIDE OF AREAS OF DEMONSTRATED EXPERTISE Project Institutions PI’s ADDITION OF NEW NASA REQUIREMENTS AFTER SELECTION Project INSTABILITY IN NASA PROGRAM BUDGETS Project NO LESSONS LEARNED FEEDBACK/FEEDFOWARD PROCESS Program PM’s INADEQUATE RISK MANAGEMENT PROCESS AT TRANSITION TO IMPLEMENTATION Program INADEQUATE CONSIDERATION OF INDEPENDENT REVIEW TEAM FINDINGS AND RECOMMENDATIONS Program INABILITY TO PROVIDE CREDIBLE COST ESTIMATES EARLY IN DEVELOPMENT Program Contributors List of Major Findings Associated RecommendationsFebruary 2010 6
    • DiscoveryMajor Findings New Frontiers We did not find a “magic bullet” in any of our findings... There is no single action that ensures cost and schedule control... CONSIDERATION OF REVIEW TEAM FINDINGS INEFFECTIVE MANAGEMENT STRUCTURE INTEGRATED PROJECT SCHEDULES INSUFFICIENT PROJECT INSIGHT PROJECT TEAM INEXPERIENCE INADEQUATE MISSION REPLANS INADEQUATE PLANNING FOR OPERATIONS / PHASE E HERITAGE and TECHNOLOGY ASSUMPTIONS FAULT PROTECTION AND AUTONOMYFebruary 2010 7
    • DiscoveryConsideration of Review Team Findings New Frontiers NASA commissions senior-level expert review panels, yet does not always address (carefully consider and mitigate or refute) the panel’s conclusions and recommendations – In most instances, cost drivers had been identified as a weakness, finding, or deficiency during reviews (source selection, milestone, or internal reviews) – There was little evidence indicating a consistent approach to considering, analyzing, or responding to the findings and recommendations Review teams are there to help the project: project management needs to make the best use of their knowledge and expertise – Schedule sufficient time for review and analysis of issues (findings, issues, concerns, or actions) from reviews – Foster a project atmosphere that moves beyond “not invented here” to ensure that issues are addressed – Ensure follow-through with risks, liens or threats, or budget adjustments – Document the disposition of all findings in a manner that facilitates resuscitating issues as the situation changesFebruary 2010 8
    • DiscoveryIneffective Management Structure New Frontiers Ineffective management structure and unclear roles and responsibilities resulted in cost and schedule impacts to missions – Primarily occurred within projects involving multiple organizations • Inconsistent project reporting and decision-making • Unclear lines of technical authority • Unconnected systems engineering across multiple organizations • Unclear responsibility for system integration – The management structure issues directly compounded the effects of other embedded project issues (e.g., heritage and technology problems, cost control, prime contractor inexperience, etc) An effective Project and technical management structure should be validated and continually reassessed – Project formulation needs to ensure a clearly documented project organizational structure, with a clear delineation of responsibilities and decision making authority – Site visits should verify that the key parties clearly understand the organizational structure and the decision process – Reviews should include a briefing and assessment of the effectiveness of the Project organization structureFebruary 2010 9
    • DiscoveryIntegrated Project Schedules New Frontiers The lack of a comprehensive, integrated Project schedule results in uncoordinated activities, inefficiencies in resource management, and increased costs – Missing critical milestones and major events resulting in underestimated resources, and insufficient data for tracking performance – Missing logical relationships (interdependencies), or unidentified or incomplete critical paths, resulting in underestimated resources, schedule delays, and poor decision making – Multiple separate, uncoordinated schedules resulting in incomplete data for tracking performance, missing logical relationships, and unidentified critical paths Project management should ensure valid, comprehensive, Integrated Master Schedule is in place on each project that forms the basis of and facilitates EVM reporting – Document the proposed approach to developing and maintaining an integrated schedule, including a description of roles and processes for integrating multiple schedules, if any, into a single master schedule – Use the Integrated Baseline Review (IBR) to confirm the content of the schedule, the processes for maintenance, and its ability to support weekly, monthly, and annual reviews and planning – Perform a detailed review of the actual integrated schedule and associated processes as part of project milestone reviews (PDR, CDR, SIR) and periodically throughout the life cycleFebruary 2010 10
    • DiscoveryInsufficient Project Insight New Frontiers Insufficient project management and technical insight into contractor performance results in poor communications, schedule delays, and technical problems that manifest as cost over-runs – Specifications, processes, and procedures that did not meet institutional standards due to organization “cultural” differences or insufficient flow down of requirements – Rework, retest, and waivers to hardware and software – Hardware mishaps – Additional personnel to perform the appropriate level of insight is added to correct issues Project management should ensure that the structure and rationale for the approach to the insight and oversight is documented and understood at all levels – Document the oversight approach (including rationale and staffing levels) for appropriate oversight based upon implementing center, prime contractor, and subcontractor/vendor experience – Include a clear plan for appropriate mission assurance oversight at the prime and subcontractors/vendors – Review and independently assess the effectiveness of their planned oversight approach at major milestonesFebruary 2010 11
    • DiscoveryProject Team Inexperience New Frontiers Teams with major players with limited experience in planetary mission development are a major contributor to program management issues resulting in cost over-runs – Complex or poorly-defined management structure, roles and responsibilities, and communications – Inadequate development schedules and implementation of performance measuring techniques – Inadequate performance oversight and configuration management: institute to prime and prime to sub – Inaccurate cost estimates, inadequate cost control and management of reserves Given there may be strategic benefits to selecting “new” team members, the Project and Program should assume a greater cost risk and operate with increased insight/oversight for less experienced team members – Assess the relevant experience for major partners and identify associated risks – Develop mitigation plans, including mentoring and active sharing of lessons learned from more experienced partners – Identify and carry a cost threat, adjusted to reflect continuing team performance – Review and independently assess the risks and associated mitigations at major milestonesFebruary 2010 12
    • DiscoveryInadequate Mission Replans New Frontiers The impact of significant changes to mission scope, schedule, or funding profiles were not sufficiently understood, resulting in unexpected cost increases and schedule delays – Program- and Project-driven changes, regardless of lifecycle timeframe, often result in underestimates of the effects (i.e., duration and complexity) on the operations phase • Replans during development tend to concentrate on Phase C/D, so impacts to operations receive limited analysis and review • Operations is often not well-defined early enough in the project development cycle to understand the full impact of any changes Program and Project management need to work together to ensure sufficient time to perform a detailed assessment, develop a detailed revised baseline, and complete an independent evaluation – Carefully consider ancillary effects such as heritage assumptions (see Heritage and Technology Assumptions) – Specifically address any operational considerations and impacts (see Inadequate Planning for Operations/Phase E ) – Document the cost and schedule impact, along with a complete description of all associated driversFebruary 2010 13
    • DiscoveryInadequate Planning for Operations/Phase E New Frontiers Phase E cost increases result from poorly scoped mission operations. Even moderate yearly underestimates can present significant LCC impacts for missions with long Phase E durations – Early focus on hardware development schedules and cost (big ticket items) – Underestimating the complexity of the operation, ignoring special requirements for a particular mission type, or inadequate planning for sustaining engineering Project Management should emphasize from the beginning the need for an full and accurate life cycle cost estimate – Emphasize the importance of a detailed operations concept that identifies plans for equipment maintenance/replacement and parts obsolescence, longevity plan for retaining/replacing key personnel – Develop a Phase E staffing estimate by primary operations functional area that can clearly be traced to and supported by the operations concept – Review and independently assess the operations concept and staffing estimates, including revisions to the operations concept driven by project replans – Consider the risk of significant Phase D activities deferred into Phase E and the potential impact to operations staffing – Manage deferred Phase D work as development effort, including appropriate metrics and schedulesFebruary 2010 14
    • DiscoveryHeritage and Technology Assumptions New Frontiers Optimistic hardware/software inheritance and technology readiness assumptions cause significant cost and schedule growth in Phases C/D – Heritage is used to justify decreased cost and risk, but can increase cost and risk – Heritage is more than just a specific piece of hardware or software, it must be considered to include all associated aspects (available documentation, knowledgeable personnel, operational paradigm, spacecraft design, etc.) – New or reapplied technology may be required to enable a mission Project Management should realistically assess both the benefits and risks associated with heritage and new technology, and ensure appropriate mitigations are identified and implemented – Clearly demonstrate the relationship between heritage/technology rankings and cost/reserve estimates – Create a living reference for the relationship between heritage/technology rankings and cost/reserve estimates, updated throughout formulation and implementation – Capture high risk heritage/technology assumptions project risks and tracked for mitigation actions – Cost factors/threats should be updated after each review, particularly heritage reviews – Review and independently assess the risks and associated mitigations at major milestones – Select membership of SRB or internal review teamsto ensure members possess needed areas of expertiseFebruary 2010 15
    • DiscoveryFault Protection and Autonomy New Frontiers Missions underestimate the time and effort required to design, implement, and test fault protection and autonomy (FPA) capabilities – Developing an integrated hardware, software, and operations approach to FPA starting early in the development life cycle – Defining appropriate autonomy requirements and the proper level of fault protection – Estimating resource requirements, including hardware (test beds) Institutions, programs, and projects need to develop a better understanding of the cost drivers associated with FPA to improve our processes and estimating capabilities Project management needs to ensure FPA is addressed as a separate and equal engineering discipline – Emphasize the importance of a comprehensive FPA approach starting with the initial system concept – Develop an FPA development resource estimate that can clearly be traced to and supported by the operations concept – Define FPA to the same level of maturity as the hardware and software for the mission concept throughout the mission life cycle – Derive FPA from the operations concept, and implement a systems approach encompassing hardware, software, and operationsFebruary 2010 16
    • DiscoveryEmbedding and Impact Realization Timing New Frontiers Insight Inexperience Fault Prot. Heritage/Tech. Mitigation Actions must Mgt. Structure Focus on pre CR period Phase E in Project life cycle Ability to descope content Integrated Schedule (cost) falls sharply post CR Most Findings Most Impacts Cost impacts Replan Embedded pre CR Occurdominate Phase D post CDR Phase E Phase A Phase B Phase C Phase D (Operations & Data (Concept Studies) (Preliminary Design) (Detailed Design) (Development) Analysis) Formulation Implementation MRR Launch SRR PDR/CRR/CR CDR FRR LRR Embedding Period Impact PeriodFebruary 2010 17
    • DiscoveryConclusion New Frontiers We did not find a “magic bullet” in any of our findings... There are no surprises here... except that we continue to have problems in these areas So what do we do with the results? – First, avoid the temptation to acknowledge and then put on a shelf along with most of the previous cost growth studies – Second, while there were no surprises, that does not mean there are no actions the entire community can take to significantly improve performance against approved budgets – D/NF Program Office is implementing a fairly disciplined approach to • Continually identify actions we can take to avoid the traps identified in the study • Avoid the “checklist” mentality and ensure that project management activities and tools directly contribute to planning, tracking, and controlling cost and schedule – We are also available to engage in more detailed discussions concerning the study approach, findings, and potential actions with groups inside and outside of the D/NF community The Discovery and New Frontiers Program Office Life Cycle Cost Study was performed under the direction of Paul Gilbert (MSFC), led by Bryan Barley (MSFC), and supported by Kenny Mitchell (MSFC-retired) and Marilyn Newhouse (CSC)February 2010 18
    • Improving the Life-Cycle Cost Management of DiscoveryPlanetary Missions New Frontiers Supplemental Data and BackupFebruary 2010 19
    • DiscoveryAcronyms New Frontiers AO Announcement of Opportunity PDS Planetary Data System AR Acceptance Review PI Principle Investigator ARR ATLO Readiness Review PLRA Program Level Requirements Appendix ATLO Assembly, Test, and Launch Operations (to the Program Plan) CDR Critical design Review PM Project Manager CR Confirmation Review PO Program Office CRR Confirmation Readiness Review PSD Planetary Science Division CSR Concept Study Report RTG Radioisotope Thermoelectric Generator D&NF PO Discovery and New Frontiers Program Office RM Resource Manager DOE Department of Energy RY Real Year DPI Deputy Principle Investigator SMD Science Mission Directorate EM Engineering Model SRB Standing Review Board EVM Earned Value Management SRR Systems Requirements Review FPP Flight Practices and Procedures TMCO Technical, Management, Cost, and Other FRR Flight Readiness Review WBS Work Breakdown Structure FTE Full-Time Equivalent FY Fiscal Year IAT Independent Assessment Team ICE Independent Cost Estimate IRT Independent Review Team LCC Life-Cycle Costs LRR Launch Readiness Review LV Launch Vehicle MM Mission Manager MOA Memorandum of Agreement MRR Mission Readiness Review PBR Project Budget Report PCA Program Commitment Agreement PDR Preliminary Design ReviewFebruary 2010 20
    • DiscoveryBackgroundGeneric Mission Development Process New Frontiers AO Development AO Implementation Decision Gates are mission milestones at which NASA makes decisions that commit funding to a AO Release project’s Life Cycle Cost (LCC):  Announcement of Opportunity (AO) Step 1 Step 2  Selection of Proposals – start of Phase A  Selection of a project for development – start of Phase B  Preliminary Design Review (PDR) Competitive  Confirmation Review (CR)  Program Level Requirements Appendix (PLRA)  Phase C/D/E design and mission-readiness reviews Phase E Phase A Phase B Phase C Phase D (Operations & Data (Concept Studies) (Preliminary Design) (Detailed Design) (Development) Analysis) Formulation Implementation MRR Launch SRR PDR/CRR/CR CDR FRR LRR 6 – 24 mo 4 mo 6- 8 mo 6 – 12 mo 6 – 12 mo 24 – 36 mo 12 – 60+ mo Science/Mission Selection led by SMD Program Implementation led by DiscoveryDiscovery PO Program Implementation led by POFebruary 2010 21
    • DiscoveryStudy ImplementationIssues Affecting Data Collection New Frontiers  Collection, analysis, and synthesis of data much more intensive than anticipated – Assumptions were made when necessary to correlate across missions, but data still valid for identifying driving issues  Lack of official/formal program documentation – Little traceability to program-level decisions or direction (e.g., official letters, documents) – Inconsistent records of year-to-year or life-cycle phased cost commitments and obligations to projects  Understanding of common program operating principles – Institutions (i.e., projects) not clear on definition of cost cap Difficult to obtain and capture important/key aspects in the life of a mission due to programmatic practicesFebruary 2010 22
    • DiscoveryHigh Level Summary of the FindingsTiming of the Embedding and Impacts New Frontiers Ability to descope content (cost) Cost impacts dominate Phase D falls sharply post CR Finding 1: Heritage/Technology Finding 2: Insight Finding 3: Phase E Finding 4: Replan Finding 5: Integrated Schedule Finding 6: Fault Protection Finding 7: Management Structure Finding 8: Inexperience Most findings embedded pre-CR Most impacts occur post-CDR Phase E Phase A Phase B Phase C Phase D (Operations & Data (Concept Studies) (Preliminary Design) (Detailed Design) (Development) Analysis) Formulation Implementation MRR Launch SRR PDR/CRR/CR CDR FRR LRR Embedding Interval Impact IntervalAugust 2007 23