Lessons Learned Implementing EVM in the EVA Office<br />Project Management Challenge 2011<br />Long Beach, CA<br />Februar...
Presentation Agenda<br />Background<br />Lessons Learned<br />Management Reserve<br />Budget versus Funding<br />Risk Mana...
Background<br />Constellation (CxP) EVA Systems Project Office (ESPO) was selected as the first of two pilot projects for ...
Lessons Learned<br />4<br />
Management Reserve<br />Driving Event<br />CxP provided direction for no Management Reserve (MR) for the projects<br />Mis...
Budget versus Funding<br />Driving Event<br />Misunderstanding the difference between budget and funding for EVM by all le...
Risk Management Impact to EAC & MR<br />Driving Event<br />Project risks captured in Integrated Risk Management Assessment...
Work Authorization Traceability & Scope<br />Driving Event<br />The authorization document (PAD) did not authorize all of ...
Identical Structures and Titles for WBS and IMS<br />Driving Event<br />The WBS element nomenclature and hierarchy is not ...
WBS/Charge Numbers and EVM Control Accounts/Work Packages<br />Driving Event<br />NASA accounting processes equate a charg...
WP should have durations of four months or less
During a replan, a WP should be closed with s=p to date, with the remaining work planned in a new WP </li></ul>CxP Reporti...
Institutional or Project OBS<br />Driving Event<br />The Organizational Breakdown Structure (OBS) for the project Responsi...
Data Integration with Multiple Tools<br />Driving Event<br />The prime contractor uses different software tools (MS Projec...
Unique Task IDs
IMP Codes
Upcoming SlideShare
Loading in …5
×

Sorge.les

14,584 views

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
14,584
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
26
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Sorge.les

  1. 1. Lessons Learned Implementing EVM in the EVA Office<br />Project Management Challenge 2011<br />Long Beach, CA<br />February 9-10, 2011<br />Les Sorge/SGT, Inc.<br />Kristen C. Kehrer/NASA KSC<br />Used with permission<br />
  2. 2. Presentation Agenda<br />Background<br />Lessons Learned<br />Management Reserve<br />Budget versus Funding<br />Risk Management Impact to EAC & MR<br />Work Authorization Traceability & Scope<br />Identical Structures and Titles for WBS and IMS<br />WBS/Charge Numbers and EVM Control Accounts/Work Packages<br />Institutional or Project OBS<br />Data Integration with Multiple Tools<br />Integration of Prime Contractor Data (1 control account or multiple)<br />Setting and Validating the PMB<br />Questions & Answers<br />Acronyms and Reference Documents<br />2<br />
  3. 3. Background<br />Constellation (CxP) EVA Systems Project Office (ESPO) was selected as the first of two pilot projects for the NASA Earned Value Management (EVM) Capability Project<br />The goal of the pilot project was to capitalize off of the EVM implementation efforts in EVA to develop an Agency EVM capability, test the Agency EVM capability processes and tools for integrated in-house and contractor EVM, and to incorporate identified changes and lessons learned into the documentation for Agency wide streamlined EVM processes and training<br />The EVMS Capability team identified 33 gaps and 6 other areas of concern during the initial gap analysis of the ESPO pilot project in March, 2010<br />Closure plans were developed and documented with a plan to track and coordinate them to ensure proper vetting across affected organizations<br />The closure plans were to be implemented and tested through monthly reporting cycles – did the fix work?<br />The pilot project was prematurely terminated in June, 2010 because of changes/uncertainty within the Constellation Program and the ESPO<br />Lessons learned from the pilot project were documented as part of the EVM Capability Project<br />Two issue papers were written and presented to the EVM Capability Project Steering Committee for consideration<br />Inconsistency Between Plan and Performance<br />Business Rules and Actual Costs<br />3<br />
  4. 4. Lessons Learned<br />4<br />
  5. 5. Management Reserve<br />Driving Event<br />CxP provided direction for no Management Reserve (MR) for the projects<br />Misunderstanding of financial reserves (Unallocated Future Expense – UFE) vs budget reserves (Management Reserve - MR)<br />Lesson Learned – Allow projects to budget for and report MR to manage project risks and in-scope changes and use the ANSI/EIA-748 standard term of MR<br />5<br />Budget Process<br />Funding Process<br />NASA EVM Project<br />CONGRESS & OMB<br />Program Office<br />Agency<br />Baseline<br />Commitment<br />Management Baseline - PROGRAM<br />AUW<br />Management<br />Reserve<br />UFE<br />Project<br />Budget<br />Base<br />UFE<br />Undistributed<br />Budget<br />UFE<br />AUTHORIZED<br />PMB<br />AUTHORIZED<br />Distributed<br />Budget<br />Key:<br /> Held Back Funds/Funding Commitment Agreement<br /> Held Back Funds/Funding Management Agreement<br /> Not Strictly “Budget”, but should be inviolate<br /> Budget<br />Graphic modified from Chambers/Gibby 07/2010<br />
  6. 6. Budget versus Funding<br />Driving Event<br />Misunderstanding the difference between budget and funding for EVM by all levels of management<br />No distinction between EVM budget and funding driving replans of the EVM Performance Measurement Baseline (PMB) when funding changes occur<br />EVM not fully integrated with current NASA financial processes and software tools <br />Lesson Learned – To support EVM, NASA culture must differentiate between budget and funding and not replan the PMB (eliminate variances and trends) for every funding change. The validity and accuracy of the performance measurement and forecast data rely on a stable PMB<br />6<br />NASA EVM Project<br />EAC<br />The funding limit (obligation) represents the maximum funds available to accomplish the work<br />VAC > 15% Notification to Program / Mission Directorate<br />Program Office<br />VAC<br />BAC<br />Management Baseline - PROGRAM<br />Funding Plan Actuals/ETC/EAC<br />Time Now<br />The BCWS & BAC represent the work<br />Annual PPBE (Funding Process)<br />UFE<br />The ACWP, ETC, & EAC represent the funds (i.e. money) required for that work<br />Funding Limits<br />Obligations<br />AUW<br />Management<br />Reserve<br />AUTHORIZED<br />Undistributed<br />Budget<br />Initial Project Budget Base<br />(BAC with MR = EAC)<br />The BCWP represents the work accomplished<br />Distributed<br />Budget<br />Define Project Budget (Budget Process)<br />6<br />
  7. 7. Risk Management Impact to EAC & MR<br />Driving Event<br />Project risks captured in Integrated Risk Management Assessment (IRMA) tool but no process defined to utilize risks to develop initial project MR<br />No process defined to include risks in an EAC update or how it relates to uncertainty<br />Lesson Learned – Projects should develop processes to tie risk assessments to the development and justification of MR and EAC updates<br />7<br />Uncertainty (Funding Process)<br />Risk (Budget Process)<br />Sample Process<br />Identify Initial Risks & Risk Mitigation Plans<br />Generate a risk factor multiplier (0 to 1)<br />Develop rule of thumb for unknown risks<br />Define MR $<br />Calculate Uncertainty<br />Update Risks & Risk Mitigation Plans<br />Update EAC with Risk Updates<br />Update JCL for KDP gate reviews<br />Known Risks (known unknowns)<br />Joint Confidence Level (JCL)<br />Risk Register<br />Cost S-curve<br />Schedule S-curve<br />Unknown Risks (unknown unknowns)<br />???<br />Rules of thumb based on historical projects<br />MR $ = [Risk Impact $ x Risk Factor] + rule of thumb for unknown risks<br />70% JCL = Prob[Cost] × Prob[Schedule | Cost]<br />
  8. 8. Work Authorization Traceability & Scope<br />Driving Event<br />The authorization document (PAD) did not authorize all of the ESPO work scope, i.e. lunar content, even though work was being performed/actuals collected on that work<br />ESPO had to maintain a separate Deltek Cobra project with this scope<br />Authorizing documents are not kept current with updates<br />Limited scope description to reference for in-scope vs. out-of-scope discussions <br />Lesson Learned <br />Create authorization document(s) that cover the entire scope of work and use incremental authorization so all actual costs trace to an authorizing document<br />Maintain all levels of authorizing documents to provide traceability<br />Develop more detailed Statements of Work (SOW) between NASA Programs and Projects<br />8<br />Funding Process<br />Budget/Funding Process<br />NASA AA<br />MDAA<br />Program Mgr<br />Project Mgr<br />CAM<br />Center Org<br />Program PCA<br />FAD - Formulation Authorization Document<br />PCA - Program Commitment Agreement<br />PAD - Project Authorization Document<br />WAD - Work Authorization Document<br />ITA - Internal Task Agreement<br />* - Not currently considered an authorization document<br />Program FAD<br />Project FAD<br />PAD<br />WAD<br />JSC<br />ITA*<br />
  9. 9. Identical Structures and Titles for WBS and IMS<br />Driving Event<br />The WBS element nomenclature and hierarchy is not consistently used within the project software toolset<br />Lesson Learned – Use only the project approved WBS numbering scheme and nomenclature for integration and easy traceability of all project cost, schedule, and technical content<br />If CAMs prefer the schedule outline to match the technical WBS (e.g. without the 6 digit project code and numbering hierarchy), then create a separate data field for the financial WBS<br />9<br />Deltek Cobra<br />NASA MdM / SAP<br />Primavera<br />
  10. 10. WBS/Charge Numbers and EVM Control Accounts/Work Packages<br />Driving Event<br />NASA accounting processes equate a charge number to a WBS number, therefore new charge numbers essentially “change” the WBS<br />NASA SAP accounting system rule that funding obligations must occur at the same level the costs are collected, i.e. the charge number<br />De-obligating funds from a charge number is a labor intensive process<br />Lesson Learned – Project EVM implementations plans need to define how best to work with the existing NASA processes and also meet the intent of general guidelines for WP<br />10<br />WP Guidelines<br /><ul><li>WP should be segregated by elements of cost
  11. 11. WP should have durations of four months or less
  12. 12. During a replan, a WP should be closed with s=p to date, with the remaining work planned in a new WP </li></ul>CxP Reporting Level<br />Control Account<br />Work Packages / Charge Codes<br />Planning Packages<br />Control Account<br />
  13. 13. Institutional or Project OBS<br />Driving Event<br />The Organizational Breakdown Structure (OBS) for the project Responsibility Assignment Matrix (RAM) was changed from the NASA center institutional organizations to the project organizational structure (i.e. integrated product teams)<br />NASA operates with two organizational breakdown structures as a system of checks and balances<br />Lesson Learned – The project needs to define the RAM OBS that best reflects the organizational authority and how work scope is subdivided between the orgs<br />Questions to ask:<br />Who is ultimately responsible for the work and has the budget authority?<br />Who does the planning of the schedule and budget for that scope of work?<br />To what level is control required or appropriate?<br />Do the institutional organizations just provide extra bodies or do they provide a “finished product”? <br />Should other NASA centers be listed as separate organizations and therefore, separate control accounts within the RAM?<br />11<br />Institutional (functional) Org Structure<br />Project Org Structure<br />Project Coordination<br />Project Coordination<br />Staff members are grouped into departments or product teams<br />Staff members are grouped by functions or specialties such as Engineering, Safety & Mission Assurance, Public Affairs, etc.<br />
  14. 14. Data Integration with Multiple Tools<br />Driving Event<br />The prime contractor uses different software tools (MS Project, DeltekMpM) then the ESPO (Primavera, Deltek Cobra)<br />Lesson Learned – Ideally, all project partners should use the same software toolset, however, this may not be practical, so request for proposals (RFP) should contain specific language in the Data Requirements Document(s) (DRD) for EVM software tool interfaces<br />The project should develop an integration methodology prior to the RFP to help define the interface requirements<br />12<br />EVA Systems Project Office<br />Prime Contractor<br />MS Project<br />Primavera<br />Deltek<br />Cobra<br />DeltekMpM<br /><ul><li>Standard Code Fields
  15. 15. Unique Task IDs
  16. 16. IMP Codes
  17. 17. WBS
  18. 18. EVM Percent Complete
  19. 19. File Formats
  20. 20. File Types (.xls, .xml, etc)
  21. 21. Etc.</li></ul>Interface Requirements<br />Deltek<br />wInsight<br />Deltek<br />wInsight<br />
  22. 22. Integration of Prime Contractor Data(1 control account or multiple)<br />Driving Event<br />ESPO defined multiple control accounts (and therefore multiple WBS/charge numbers) for the prime contractor<br />Each charge number required obligations which sometimes created “cost over obligations” on a charge number even though the total cost was within the total obligations<br />Lesson Learned – Projects should develop contractor data integration plans and weigh the pros/cons of the different options<br />13<br />Example of Single Control Account<br />Multiple Control Accounts<br />Pros / Cons<br />731384.06.04<br />EVA HW Development<br />731384.06.04<br />EVA HW Development<br />731384.06.04.01<br />Suit Element<br />731384.06.04.01<br />Suit Element<br />.31 Govt CA<br />Govt CA<br />WP / PP<br />WP / PP<br />.32 Prime CA<br />731384.06.04.03<br />Vehicle Interface Element<br />WP / PP<br />731384.06.04.03<br />Vehicle Interface Element<br />Govt CA<br />WP / PP<br />.31 Govt CA<br />731384.06.04.05<br />CSSS Contract<br />(Prime CA)<br />WP / PP<br />.32 Prime CA<br />WP / PP<br />
  23. 23. Setting and Validating the PMB<br />Driving Event<br />Prime contractor operated for a year under letter contract without a Performance Measurement Baseline (PMB)<br />Constant, significant changes to the Constellation Program/Projects funding did not allow for a stable baseline so as to set a PMB<br />A PMB was developed for an Integrated Baseline Review (IBR), but the announcement to cancel the Constellation Program and associated reduction in funding changed the focus of the IBR to an FY10 review<br />Prolonged proposal submittal and authorization process encumbered the change process<br />Lesson Learned<br />Clearly define expectations and requirements to initialize the PMB and begin EVM reporting to include letter contracts<br />Utilize existing NASA resources such as the IBR Handbook, the EVM Handbook, and the Schedule Handbook to define appropriate EVM requirements<br />Projects and contractors should strive for timely contract authorization and definitization of the base contract and subsequent contract modifications<br />Given the fluid nature of funding, develop “right-sized” products<br />Schedules and budgets that are not overwhelming to update<br />Use of planning packages and summary level planning packages <br />Utilization of rolling wave planning<br />14<br />
  24. 24. Questions & Answers<br />15<br />
  25. 25. Acronyms and Reference Documents<br />Acronyms<br />AUW – Authorized Unpriced Work<br />CA – Control Account<br />CAM – Control Account Manager<br />CPR – Cost Performance Report<br />CSSS – Constellation Space Suit System<br />CxP – Constellation Program<br />DRD – Data Requirements Document<br />EAC – Estimate at Completion<br />ESPO – EVA Systems Project Office<br />ETC – Estimate to Completion<br />EVA – Extravehicular Activity<br />EVM – Earned Value Management<br />EVMS – Earned Value Management System<br />IBR – Integrated Baseline Review<br />IMS – Integrated Master Schedule<br />IPT – Integrated Product Team<br />IRMA – Integrated Risk Management Application<br />ITA - Internal Task Agreement<br />JCL - Joint Confidence Level<br />JSC – Johnson Space Center<br />MR – Management Reserve<br />OBS – Organizational Breakdown Structure<br />PAD – Project Authorization Document<br />PMB – Performance Measurement Baseline<br />PP – Planning Package<br />PPBE - Planning, Programming, Budgeting, and Execution<br />RAM – Responsibility Assignment Matrix<br />SAP – Systems Application and Products<br />UFE - Unallocated Future Expense<br />WBS – Work Breakdown Structure<br />WP – Work Package<br />Reference Documents<br />NPR 7120.5D – NASA Program and Project Management Processes and Requirements <br />CxP 72179 Anx01 – CxP ESPO EVM Implementation Plan<br />ANSI/EIA-748 – Earned Value Management Systems<br />16<br />

×