Overview of GAO Quicklook          PM Challenge         February 2012Katie Gallagher and Tracy Osborne             OCFO/SID
Outline• Background      – Why is this important?• GAO’s Approach      – What projects are involved?      – Why does GAO r...
Background• The GAO’s Assessment of Large-Scale [NASA] Projects is  Congressionally-mandated• GAO sends its findings to Co...
Why the Stakeholder Interest?• NASA’s historical inability to control cost and schedule  growth      – JWST, CxP, MSL grow...
Typical Results from Quicklook Reports• Most of NASA’s cost growth for the current  portfolio is attributed to projects ba...
Why do they ask about TRL’s and drawings?   GAO’S APPROACH1/22/2012                                      6
GAO’s Report is in Two Parts• Agency Observations    – Examines adherence to NASA policies such      as 7120.5E and 1000.5...
How does a project get picked?• GAO evaluates all spaceflight projects in formulation if a contract is  released that has ...
Projects Included in Audit                                                      2008                              2009    ...
GAO “Business Case” Model                                                PDR        CDR      SIR                          ...
Data Requested Annually                                                            Reporting YearData Category            ...
So, my project is in Quicklook…   WHAT SHOULD I EXPECT?1/22/2012                            12
The Quicklook Annual Cycle (2011)               DCI-                  Action Item Response               Addt’l Actions   ...
Process for Working with GAO                                                                        Extra explanation may ...
GAO Data Access Rights• Legislatively, the GAO has right of access to the data, regardless  of procurement sensitivity or ...
Data Collection Instruments                                                            Example Technical DCIGAO Data Colle...
Site Visits• GAO will prepare for site visits with the projects by reviewing project monthly  status review (MSR or PSR) p...
2-Page Project Summary Anatomy       Project Challenges   Overview             Project Timeline       Project UpdateProjec...
Draft 2-Page Project Summaries              What to look for                                How to Respond• Overview and P...
So we gave GAO all this data…   WHAT DID THE DATA SAY?1/22/2012                          20
GAO Quicklook – Mar 2011•     Maintained recommendation for GAO’s “Business Case Approach,” but recognized      improvemen...
Mar 2011 QLB - Cost Growth                                                                      The majority of           ...
Challenges Summary                                                            Quicklooks published in 2009, 2010, 2011    ...
Data Provided During 2011                              Technology Maturity and Drawing Release                     150.0% ...
Data Provided During 2011                                    Drawing Release                                          % of...
Data Provided During 2011                                                                        Software• 13 projects pro...
What is Headquarters saying to GAO?   AGENCY MESSAGES1/22/2012                                27
Formulation                                      • Formulation is an iterative    Resources         Requirements      proc...
PDR                  Road to Confirmation                                                            (For SMD Projects to ...
JCLs and UFE• The use of JCL in decision making represents a shift in the paradigm  of how project cost estimates, budgets...
Management vs Agency Commitments                   Cost                                     ScheduleAgency Baseline       ...
What happens when funds are not             appropriated or performance is poor?o When requested funds                    ...
EVM                 Total Project (Combined In- and Out- of House)                                          Out-of-House  ...
Technical Leading Indicators• NASA is enhancing the ability to monitor and assess the  stability of programs and projects ...
Any Questions?1/22/2012                    35
Upcoming SlideShare
Loading in …5
×

K gallagher tosborne

15,253 views
15,223 views

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
15,253
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
22
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Note: Funding Issues, Launch Issues, and Parts Issues were not addressed in the 2008 Quicklook Report. The 2008 Quicklook Report included issues for several projects that are now in Operations or are completed, and are not listed here (Dawn, Fermi, Herschel, Kepler, LRO, OCO, SDO, WISE).
  • NASA requires EVM reports to be in compliance with ANSI/EIA-748. CPR is a DOD format.analytical products and risk watch list are provided in MSRs
  • K gallagher tosborne

    1. 1. Overview of GAO Quicklook PM Challenge February 2012Katie Gallagher and Tracy Osborne OCFO/SID
    2. 2. Outline• Background – Why is this important?• GAO’s Approach – What projects are involved? – Why does GAO request technical data?• What should I expect if my project is in Quicklook?• What does NASA’s data say?• Agency Messages1/22/2012 2
    3. 3. Background• The GAO’s Assessment of Large-Scale [NASA] Projects is Congressionally-mandated• GAO sends its findings to Congress in an annual “QuickLook Book” (QLB) Report focusing on changes in project cost and schedule and providing GAO’s explanations for these changes• GAO is frequently called upon to testify about the results.• Engagement objective: “Identify and gauge the progress and potential risks associated with selected NASA acquisitions.” (i.e., projects)• GAO uses data from QuickLook Book (QLB) in other audits, such as the ongoing High Risk audit, and vice versa. 1/22/2012 3
    4. 4. Why the Stakeholder Interest?• NASA’s historical inability to control cost and schedule growth – JWST, CxP, MSL growth cited in several provisions establishing new reports. – A 2004 GAO finding that “NASA lacks discipline in cost estimation.” – Multiple Operating Plans per year to address cost growth or phasing.• Based on past Performance NASA is required to report to OMB, Congress and GAO routinely on Project cost and schedule. – All appropriations since FY 2008 have included direction for GAO to “Identify and gauge the progress and potential risks associated with selected NASA acquisitions.”1/22/2012 4
    5. 5. Typical Results from Quicklook Reports• Most of NASA’s cost growth for the current portfolio is attributed to projects baselined prior to 2009• NASA projects do not meet GAO-identified best practice standards for technology maturity and design stability• Other challenges that affect NASA project outcomes include Funding, Launch Vehicle, Development Partners, Parts Quality, Contractor Management1/22/2012 5
    6. 6. Why do they ask about TRL’s and drawings? GAO’S APPROACH1/22/2012 6
    7. 7. GAO’s Report is in Two Parts• Agency Observations – Examines adherence to NASA policies such as 7120.5E and 1000.5A – Examines potential systemic reasons for NASA’s cost growth – Assesses NASA’s efforts to improve cost and schedule growth, including JCL policy• Project Assessments – Provides an update on status of each project included in the audit – Outlines challenges specific to each project that GAO believes may lead to cost growth GAO Quicklook assesses performance of NASA’s major projects and the Agency’s management of those projects. 1/22/2012 7
    8. 8. How does a project get picked?• GAO evaluates all spaceflight projects in formulation if a contract is released that has development content and is greater than $25 M – Can include projects in Phase A and Pre-Phase A if they have such a contract• GAO evaluates all spaceflight projects in development with a life cycle cost greater than $250 M• GAO continues to track projects in Operations until NASA negotiates removal from Quicklook – Minimal data collection on Operations projects; main focus is cost and scientific results• For projects in formulation, GAO expects NASA to provide cost and schedule ranges• For projects in development, GAO expects NASA to provide baseline cost, schedule, design and technical information1/22/2012 8
    9. 9. Projects Included in Audit 2008 2009 2010 2011 Requirement for Inclusion * Estimated Lifecycle Cost * Estimated Lifecycle Cost * Estimated Lifecycle Cost * Estimated Lifecycle Cost >$250M >$250M close to $250M (if it is called close to $250M (if it is called Lifecycle * In development * In development out in the budget as a out in the budget as a * In formulation if it also has * In formulation if it also has discreet project then it may discreet project then it may Phase as of released a development released a development be included) be included) Project Entrance contract valued at >$50M contract valued at >$25M * In development * In development Conference * If in operations, it is not * In formulation if it also has * In formulation if it also has (April 2011) (consistent with other included released a development released a development external reporting contract valued at >$50M contract valued at >$50M requirements) * In operations, in the prime * In operations, in the prime mission phase mission phase 1 Aquarius Development X X X X 2 Ares Formulation X X X 3 Dawn Operations X X X 4 Fermi/GLAST Operations X X X 5 GPM Development X X X X 6 Glory Development X X X X 7 GRAIL Development X X X 8 Herschel Operations X X X X 9 ICESat II Formulation X X10 JWST Development X X X X11 Juno Development X X X12 Kepler Operations X X X X13 LADEE Development X X14 LDCM Development X X X X15 LRO Operations X X X X16 MAVEN Development X X17 Mars 2016 (TGO) Formulation X18 MMS Development X X X19 MPCV Pre-Formulation X20 MSL Development X X X X21 NPP Development X X X X22 OCO/OCO-2 Development X X X23 Orion Formulation X X X24 Phoenix Closed Out X25 RBSP Development X X X26 SDO Operations X X X X27 SLS Pre-Formulation X28 SMAP Formulation X X29 SOFIA Development X X X X30 Solar Probe Plus Formulation X X31 TDRS Development X X32 Wise Operations X X X X1/22/2012 9
    10. 10. GAO “Business Case” Model PDR CDR SIR Production Pre- Phase Phase Rarely Applies NASA phase A A B Phase C Phase D Phase E to NASA KDP-B KDP-C KDP-D Projects KDP-A KDP-D DOD Technology Development Engineering Manufacturing and Development Production and Deployment Operations and Support PDR MS-B CDR MS-C Concept and Technology Engineering Manufacturing and GAO Development Development Production Knowledge Knowledge Knowledge Point 1 Point 2 Point 3• Knowledge Point 1: Resources and Requirements Match – Indicated by a high level of technology maturity • Heritage is technology is appropriate in form, fit and function • New Technologies at least at TRL-6 (for NASA)• Knowledge Point 2: Product Design is stable “about midway through development” – Best practice is to achieve design stability by midpoint of development – GAO cites 90% of engineering drawings releasable as figure of merit• Knowledge Point 3: Manufacturing Processes are mature – Product shown to be producible within cost, schedule, and quality targets and have demonstrated reliability and Design demonstrates needed performance through realistic system-level testing1/22/2012 10
    11. 11. Data Requested Annually Reporting YearData Category Element 2008 2009 2010 2011 Frequency Design Stability X X X X Annual + Updates Critcal Technologies X X X X Annual + Updates Technical Heritage Technologies X X X Annual + Updates Software Complexity X X Annual Quality Parts Issues X X Annual MPAR Baseline X X X X As Occurs ARRA Funding X X Semi-annual KDP-B Est. LCC Range X X X X Semi-annual KDP-C Baseline X X As Occurs Cost ICEs X X As Occurs JCLs X X As Occurs Project-held UFE X X Monthly (in MSRs) MD-held UFE by year X Annual Key Milestones X X X X Semi-annual Schedule Agy vs Mgmt Schedule Comparison X Semi-annual Basic Information X X X X Semi-annual Contracts Award Fee Strucuture X X X Semi-annual EVM (CPR Formats) X X Monthly FAD/PCA X X X X As Occurs Project Plan X X X X As Occurs Control Plans X X X X As Occurs PDR/CDR Packages X X X X As OccursDocumentation CADRes X X As Occurs SRB Reports X X As Occurs KDP-C Decision Memos X As Occurs KDP-C Datasheets X As Occurs MSR Presentations X X Monthly 1/22/2012 GAO reviewed this list for 2011, and agreed to reduce frequency of some requests 11
    12. 12. So, my project is in Quicklook… WHAT SHOULD I EXPECT?1/22/2012 12
    13. 13. The Quicklook Annual Cycle (2011) DCI- Action Item Response Addt’l Actions contract Site Visits Agency 2-Pgr Cmts DCI- Site Visits – Cmts – MSFC C&s JPL &GSFC &JSC DCI- DCI- DCI- Other Data Requests Addt’l Data proj C&s C&s Entrance Exit Final ReportConference Conference DCI = “Data Collection Instrument” Three types: 1.Project DCI – for technical data 2.Cost and Schedule (C&S) DCI – provided by OCFO 3.Contracts DCI – provided by OP1/22/2012 13
    14. 14. Process for Working with GAO Extra explanation may be provided Quicklook Technical POC (TPOC) to help contextualize the data or to discusses request with GAO to better assist the GAO in answering understand purpose for their questions requested itemsGAO transmits Project drafts Response reviewed QLB TPOCrequest response by HQ & Ctr Mgmt transmits response to GAO • The TPOC and • Provide • Possible • ALR, PM, PE, relevant Audit document reviewers and reviewers Liaisons (ALR) and/or include PE, MD, will be copied should be explanation PCFO, copied on all Procurement, • When requests OCE, OSMA, LSP document or • PM please data is not • QLB TPOC and forward available, MD ALR will requests to ALR explain why coordinate the and QLB TPOC appropriate • Draft response when necessary provided to PE, reviews. ALR, and QLB TPOC Standard 3 week turn-around when not otherwise stipulated by GAO 1/22/2012 14
    15. 15. GAO Data Access Rights• Legislatively, the GAO has right of access to the data, regardless of procurement sensitivity or classification• NASA can negotiate with GAO regarding the provision of data; if there is a disagreement, GAO’s recourse is legislative action• GAO is bound by the same rules as NASA for handling the data, and NASA can specify data restrictions• NASA actively works with GAO on data formats, types and context for interpretationNASA provides the data requested and works with the GAO to addcontext and help the GAO analysts to understand NASA’s data. 1/22/2012 15
    16. 16. Data Collection Instruments Example Technical DCIGAO Data Collection Instrument: Design Stability • GAO asks PMs to fill inDate Submitted to NASA Project Office:NASA Project Office Response Required by: 5/3/2011 5/6/2011 a Data CollectionGAO Contact:Project: Joe Analyst Awesome Instrument (DCI)NASA Project Office Instructions Please provide or update the total number of releasable engineering drawings and the total number expected at key dates listed below. The term "releasable" refers to those drawings (physical hard copy or computer-generated packages) that are sent to be manufactured. Please annually and following calculate the number of drawings by using the number sheets within a package, not just the number of packages. Sometimes these drawings are considered the "build to" drawings. These drawings include details on the parts and work instructions needed to make the PDR and CDR product and reflect the results of testing. Change any dates or data as needed.Key Event Dates Defined: The PDR column should reflect information as of the Preliminary Design Review prior to KDP C (NAR). The CDR column should reflect information as of the • Data collectionCritical Design Review. The "Latest" column should reflect the the number of drawings releasable and expected as of the date this spreadsheet is returned to GAO. includesOverall Design KnowledgeKP ESTIMATE PDR CDR September 2010 update Latest – Critical and heritageDateNo. of Releasable 7/12/2010 Scheduled for July 2011 9/16/2010 5/3/2011 technology TRLs 1537 sheets expected for 947 spacecraft sheets released to date. 299 – Drawing release – Parts Quality issues, spacecraft, 818 drawings 10 released for spacecraft, drawings released forDrawings 0 expected for instruments. 16 released for instruments. instruments.No. of Total 2437 sheets of spacecraft including impact ($) – Software TLOC and 2437 sheets expected for drawings planned, 1023 1879 sheets expected for spacecraft, 1023 drawings drawings expected forDrawings Expected spacecraft. expected for instruments. instruments.Source Project Office Project Office Project Office Project Office BlocksIf the total number of drawings is not known or if the total number of drawings has changed, please provide an explanation in the space below. If the project has – Partners •completed CDR, please describe any measures and metrics used in addition to engineering drawings to assess design stability. If the project has yet to complete itsCDR, please identify the measures and metrics the project intends to use to assess design stability.See below for Spacecraft and Instruments comments. The project assesses design stability primarily thru the Systems Engineering organization. A weekly Systems Design Team GAO requests data asregularly addresses design, design margins, analyses, interfaces, subsystem and instrument design topics, and proposed changes. Proposed design changes are then processed thrustandard Configuration Control Boards. In addition, weekly status reviews as well as a Monthly Management Review take place that addresses overall status and issues. of PDR, CDR, and currentComments & Issues: Enter comments and issues below or on a separate sheet. • Procurement and PMs should NOT leave any sections blank; if the data is OCFO fill in similar DCIs on contracts, not available or not applicable, say so and explain why. cost, and schedule 1/22/2012 16
    17. 17. Site Visits• GAO will prepare for site visits with the projects by reviewing project monthly status review (MSR or PSR) packages; discussion questions will be transmitted in advance of the visit.• GAO may not visit every project included in the Quicklook; projects not visited will receive written questions based on the content of their monthly review packages.• PMs are invited to bring any charts or personnel that they need to help answer the questions; however, GAO prefers to have an informal discussion – NOT a lecture.• When preparing responses, keep in mind that Quicklook is about cost growth against the Agency’s MPAR Baseline; responses should keep in mind the full mission cost – including launch vehicle and UFE• Refer questions about things you are not responsible for to correct organization – LV questions should generally be directed to LSP – MD-held UFE questions should be directed to the MD, although it is ok to state the value• When in doubt, it is ok to take an action to provide a response after the visit. This is the project manager’s opportunity to add context and explain all the good work their team is doing. 1/22/2012 17
    18. 18. 2-Page Project Summary Anatomy Project Challenges Overview Project Timeline Project UpdateProjectEssentialsContractInfo Project Performance Summary Project Office Comments 1/22/2012 18
    19. 19. Draft 2-Page Project Summaries What to look for How to Respond• Overview and Project Essentials 1. Project comments – Is the information correct? – Will be printed in the “Project Office• Project Update and Summary Comments” section of the final report – Limited to 500 characters (appr 100 words) – Does the GAO accurately portray the status? 2. General Comments – Are there any errors in the analysis? – Focus on errors in understanding or interpretation of information previously• Project Challenges provided to GAO as well as points of – GAO’s challenge categories are fixed and disagreement with the GAO’s analysis. will not change – These comments will not appear in the – Assertion of a challenge should be report but may help drive the discussion with supported by the analysis described in the the GAO about how they can revise Project Update 3. Technical Comments• Contract Info, Project Performance, – Comments provided in this section will be and Project Timeline used to correct any factual errors in the – no action from PM required; Procurement report. and OCFO will review to ensure consistency – Corrections must be accompanied by with other external reporting evidenceGAO will transmit draft summaries directly to project managers with response requested in5 business days; Response must go through full agency review process in this time. 1/22/2012 19
    20. 20. So we gave GAO all this data… WHAT DID THE DATA SAY?1/22/2012 20
    21. 21. GAO Quicklook – Mar 2011• Maintained recommendation for GAO’s “Business Case Approach,” but recognized improvements due to JCL policy• Found average cost growth of 14.6% and schedule growth of 8 mo; nearly all growth attributed to projects baselined before 2009• Main Observations: – Technology Challenges: • NASA begins development with insufficient technology maturity • Evidence: Majority of projects passed PDR with critical technologies at TRL < 6 – Design Challenges: • Projects pass CDR with unstable design • Evidence: Nearly all projects passed CDR with <90% releasable drawings• Additional Observations – Funding Challenges: JWST, Ares, Orion funding “did not align with plan” – Launch Vehicle Challenges: mainly medium class – addressed more fully in another GAO report – Development Partner Challenges – Parts Challenges – addressed more fully in GAO Quality Parts Audit – Contractor Management Challenges• Report stated that GAO will conduct a more thorough analysis of NASA’s EVM data in future audits and that [cost] transparency is insufficient during formulation 1/22/2012 21
    22. 22. Mar 2011 QLB - Cost Growth The majority of NASA’s cost and schedule growth is attributed to projects baselined prior to 2009 NASA agrees with GAO’s cost and schedule growth figures1/22/2012 22
    23. 23. Challenges Summary Quicklooks published in 2009, 2010, 2011 12 2009 2010 10 # Projects with GAO-identified Issue 2011 8 6 4 2 0 Funding Issues Launch Issues Parts Issues Technology Design Issues Contractor Development Issues Issues Partner Issues Subjects of other GAO audits1/22/2012 23
    24. 24. Data Provided During 2011 Technology Maturity and Drawing Release 150.0% 150.0%Percentage Development Cost 130.0% JWST 130.0% JWST Percentage cost growth 110.0% 110.0% Class A GAO hypothesis 90.0% 90.0% Growth 70.0% MSL Class B 70.0% 50.0% Class C 50.0% NPP 30.0% NPP 30.0% SOFIA Class D 10.0% 10.0% Juno -10.0% -10.0% 0.00% 20.00% 40.00% 60.00% 80.00% 100.00%120.00% 0% 50% 100% GPM -30.0% -30.0% % Immature Technologies at PDR % releaseable drawings at CDRMSL did not provide drawing data; MAVEN is excluded from CDR analysis because CDR was not complete at time of DCI submission• Self-reported data from DCIs submitted to GAO appear to support the hypothesis that immature technologies at PDR leads to cost growth for NASA missions – The distribution does not support a strong correlation – JWST and MSL drive the correlation, but have cost growth clearly attributable to other factors.• The data shows low correlation between releasable drawings at CDR and cost growth 1/22/2012 24
    25. 25. Data Provided During 2011 Drawing Release % of Drawings Releasable at CDR100.00% 90.00% 80.00% 70.00% 60.00% 50.00% 40.00% 30.00% 20.00% 10.00% 0.00% Aquarius GPM GRAIL Juno JWST LADEE LDCM MMS MSL NPP OCO-2 RBSP SOFIA TDRS GAO Calculation Assumed NASA calculation CDR Now • There appears to be some disagreement between how NASA# Releasable A C and GAO calculate the percent of releasable drawings • NASA calculates the percentage of releasable drawings at CDRTotal # anticipated B D based upon the total number of drawings anticipated as of CDRNASA: % releasable = A/B • Drawing growth after CDR typically occurs as a result of riskGAO: % releasable = A/D realization; it is impossible to predict which risks will come to pass and which of those will require drawing changesNASA’s 90% guidance is based • By NASA’s calculation, several projects did meet the beston what is known at the time practice guideline of 90% releasable drawings at CDR 1/22/2012 25
    26. 26. Data Provided During 2011 Software• 13 projects provided complete software data – 4 projects - Aquarius, MSL, NPP, and JWST - reported growth in TLOC greater than 25% – 5 projects reported a decrease in TLOC since PDR – Juno, LDCM, RBSP, SOFIA, TDRS• After removing projects whose cost growth is clearly attributed to other factors, the mathematical correlation between development cost growth in growth in TLOC is high (.91) but the lack of significant distribution does not support a relationship between cost growth and software growth All Quicklook Projects MSL and Aquarius Excluded 200.0% 200.0% % Development Cost Growth % Development Cost Growth 150.0% 150.0% JWST JWST 100.0% 100.0% MSL 50.0% SOFIA 50.0% NPP LDCM NPP SOFIA Aquarius 0.0% GPM 0.0% -200% 0% 200% 400% 600% 800% -50% 0% 50% 100% 150% GPM -50.0% -50.0% % Growth in TLOC, PDR to Current % Growth in TLOC, PDR to Current 1/22/2012 26
    27. 27. What is Headquarters saying to GAO? AGENCY MESSAGES1/22/2012 27
    28. 28. Formulation • Formulation is an iterative Resources Requirements process that aims to balance requirements and resources. • At KDP-C the Decision Authority agrees that the design is sufficiently mature and the requirements match the resources • The confirmation decision results in an agreed-to cost and schedule baseline; budget is committed to the project as part of the PPBE process • The balance of requirements and resources is continuously assessed throughout a project’s lifecycleChanges in schedule or projected LCC during formulation shouldnot be treated as slips or cost growth; rather, this is a normal part ofthe formulation process. 1/22/2012 28
    29. 29. PDR Road to Confirmation (For SMD Projects to date) Project Report Project Report Project Report JCL JCL JCL SRB C&S (JCL) SRB C&S (JCL) SRB C&S (JCL) Assessment Assessment Assessment SRB Technical SRB Technical SRB Technical Assessment Assessment Assessment Project Response Project Response Approval Project Response Approval to SRB to SRB to Proceed to SRB to Proceed ICEs or ICAs1 ICEs or ICAs1 (Cat 2&3)2 ICEs or ICAs1 (Cat 1) Center Center Recommendation Recommendation Program Office SMD Recommendation Recommendation CMC Division (Theme) DPMC OCFO/SID Analyst APMC Recommendation Assessment The Confirmation decision takes into account many inputs. There is no “right answer”1. when available 1/22/2012 292. Cat 2 and 3 missions do not require an APMC, but Cat 1 missions do. With 7120.5E, the AA will approve entry to Implementation for all projects
    30. 30. JCLs and UFE• The use of JCL in decision making represents a shift in the paradigm of how project cost estimates, budgets and schedules are established.• This method of estimation moves beyond prior practices of calculating a “point estimate” and adding an additional level of reserves.• Because the UFE is a product of the probabilistic basis of estimates, any reduction to the UFE will reduce the probability of achieving the project cost and schedule targets• Usage of UFE should not be interpreted as a “funding issue;” the UFE is part of the full amount that NASA budgets for a mission.• The term “UFE” should be used rather than “reserves” to denote the more rigorous cost estimating policy NASA now uses With the release of 7120.5E, NASA no longer uses the term “reserves.” PMs should update charts to use the term “Project-held UFE.”1/22/2012 30
    31. 31. Management vs Agency Commitments Cost ScheduleAgency Baseline Agency BaselineCommitment Agency Commitment MD-Held Schedule • Based upon JCL and• Based upon JCL and UFE Margin other inputs other inputs Project- Project • At 70% JCL or as• At 70% JCL or as Schedule determined by the DA determined by the DA held UFE Margin • Difference between ABC• Difference between ABC and MA is additional and MA is budgeted as schedule margin held by MD-held UFE the AgencyManagement ManagementAgreement Agreement• Managed by PM  Managed by PM• Based upon Project’s  Based upon Schedule own Cost Estimate developed by PM• Should be at 50% JCL or  Should be at 50% JCL or greater greater Not to scaleIn some cases, the Decision Authority deems it prudent for the Agency to holdadditional schedule margin above the project’s schedule. In these cases, theManagement Schedule should not be viewed as acceleration of the Agency’sschedule 1/22/2012 31
    32. 32. What happens when funds are not appropriated or performance is poor?o When requested funds Notional Example are not appropriated, or 700 performance is FY10 Budget request poor, work is pushed 650 FY10 Appropriation + FY11 request to future years, and FY11 Appropriation + FY12 request budget requirements FY12 Appropriation + FY13 request increase in the out 600 years. 550o This may lead to cost growth and operating plan changes. 500o If reductions happen in 450 subsequent years, the profile becomes unsustainable. 400 FY10 FY11 FY12 FY13 FY14 1/22/2012 32
    33. 33. EVM Total Project (Combined In- and Out- of House) Out-of-House NASA projects tend to be complex; ContractsIIn-House CP & FPI CP & FPI CP & FPI Partners EVM is not necessarily applied to FFP >$50M >$20M <$20M all suppliers but it is addressed at 7 Certified 32Principles System Guidelines NA NA NA the total project level. 7 Principles applied across all elements of Total Project • NASA’s projects are accomplished through combined efforts of suppliers and NASA in-house efforts – Total In-House with targeted procurements – Prime contract with government providing Level-Of-Effort (LOE) oversight – Many unique mixtures of the above two scenarios • NASA applies EVM to contractors at the project level and requirements are flowed down to subcontractorsEVM is one of many project management tools that NASA uses. NASArecognizes EVM implementation as an area of concern and is committed toimprovement in this area 1/22/2012 33
    34. 34. Technical Leading Indicators• NASA is enhancing the ability to monitor and assess the stability of programs and projects as well as the maturity of their products through the development process• Comprehensive 3-Step Approach 1. The Entrance and Success Criteria for Lifecycle reviews are being enhanced to further account for product maturity • These will be documented in the Rev B update to NPR 7123.1 2. A required set of 3 parameters that will address product stability and maturity will be added as a NID to NPR 7123.1A • Mass Margins • Power Margins • Review item closures 3. A set of recommended Common Leading Indicators is being developed • Added to NPR 7120.5E Formulation Agreement and Program/Project Plan templates1/22/2012 34
    35. 35. Any Questions?1/22/2012 35

    ×