Confidence Level Estimating and Budgeting             February 2011              Tom Coonce
Topics• A brief review of confidence level  estimating, budget-setting policy and  rationale• Status of program and projec...
Recap of JCL Policy and Rationale•   NPD 1000.5    – Programs are to be baselined at a 70% joint confidence level    – Pro...
JCL – Defined by a Product, Refers to a                       Process•   JCL = Joint Confidence Level     – Identifies pro...
Acceptable JCL Methods• Resource-Loaded Schedule with Monte-Carlo simulation, with  three different techniques   – Bottoms...
An Example of JCL OutputVertical bar indicatesx-axis (Schedule)Confidence levelPoints representcost/Schedule resultpair fr...
Management Councils Revisions the             Policies and Expectations•   Eliminated the requirement at KDP-B and D, but ...
Management Councils Revisions the  Policies and Expectations (Continued)• Include cost and schedule range  estimates and a...
Final Proposed 7120.5E Language           (After the 4 August 2010 PMC)Under 2.5.4.2 Confidence Level Estimating and Budge...
So, What is Required to Meet the    KDP-B Programmatic Requirements?• For non-AO missions, projects must provide:   – A “h...
Programmatic Expectations at KDP-B      Non-AO Missions (Concluded)   – Basis of Estimates. Examples include:      • How t...
Status of Program and Project JCLs• During past two years, the following JCLs  were completed and presented at PMCs   – LD...
JCL Implementation Issues•    Most projects had not assembled multiple schedules into a top-level     electronically integ...
JCL Lessons Learned• Improve project control functions   – Ensure projects develop credible cost and schedule basis of    ...
Action Completed and Planned• Completed   – Created near term optimization techniques for MS Project/@Risk       • Schedul...
Backup
Detailed Program and Project Status              on JCLs                                      17
Detailed Program and Project Status        on JCLs (Concluded)                                      18
Upcoming SlideShare
Loading in …5
×

Coonce.tom

14,277 views

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
14,277
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
16
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Coonce.tom

  1. 1. Confidence Level Estimating and Budgeting February 2011 Tom Coonce
  2. 2. Topics• A brief review of confidence level estimating, budget-setting policy and rationale• Status of program and project Joint Cost and Schedule Confidence Level (JCL) calculations• JCL issues and lessons learned• Actions completed or planned to address lessons learned 2
  3. 3. Recap of JCL Policy and Rationale• NPD 1000.5 – Programs are to be baselined at a 70% joint confidence level – Projects are to be baselined/budgeted at JCL that supports the program approved JCL – Projects are to be funded at no less than a 50% JCL – JCLs are to be developed and maintained through launch – Program and project proposed cost and schedule baselines will be assessed by an independent review team• Policy Rationale – Improve cost and schedule performance – Improve expectations with stakeholders – Improve project planning – Improve understanding of project plans 3
  4. 4. JCL – Defined by a Product, Refers to a Process• JCL = Joint Confidence Level – Identifies probability that a given project or program’s cost will be equal or less then the targeted cost AND the schedule will be equal or less then the targeted schedule date – Per NPD 1000.5 used at KDP-C to set Program/Project Budget Baseline• JCL is More than the Scatter Plot – It’s a systematic process of integrating cost, schedule, and risk – Provides a cohesive and holistic picture of the project or program’s ability to achieve cost and schedule goals and to help the determination of reserves (schedule and cost) – Provides key Decision Support information – Does the project have enough funds? – Can the project meet the schedule? – What are areas of risk toward successful execution of the project? – What risk mitigation strategies provide the best project benefit? – What are project phasing (fund) needs? 4
  5. 5. Acceptable JCL Methods• Resource-Loaded Schedule with Monte-Carlo simulation, with three different techniques – Bottoms-up Resource Loaded Schedule or BURLS technique is based on resource or cost-loaded schedule that focuses on the project risk lists (known-known risks) and uncertainties (known and unknown unknowns) to set the cost and schedule possibilities – Cost Estimate/Program Assessment (CEPA) technique is based on a top-level resource or cost loaded schedule that focuses on project performance as well as project risks to set cost and schedule possibilities – Project Risk Evaluation Process or PREP technique is based on the projects’ top-level resource or cost loaded schedule that focuses on the project’s risk list as well as the views of an Independent panel of experts to set cost and schedule possibilities• Parametric – Uses cost and schedule estimating relationships, data dispersion, and relationships between cost and schedule to determine the overall cost and schedule target and associated JCL 5
  6. 6. An Example of JCL OutputVertical bar indicatesx-axis (Schedule)Confidence levelPoints representcost/Schedule resultpair from each iterationof the simulation Lower quadrant Frontier line identifies identifies iterations cost/schedule surface satisfying cost and that meets Target JCL schedule targets ~ the JCL Horizontal bar indicates Y-axis (Cost) confidence level 6
  7. 7. Management Councils Revisions the Policies and Expectations• Eliminated the requirement at KDP-B and D, but stipulated that KDP-Cs JCL must be based on probabilistic resource loaded schedules (May 2009 SMC) – Directed pilot projects be completed for projects at KDP-B• Eliminated the requirement to maintain the JCL during implementation (November 2009 PMC) – Project are not required to develop a JCL at any other time during implementation if it is “on plan” – Projects are to be monitored as part of the Baseline Performance Review (BPR) process – If a project is re-baselined, scored red in the BPR, or if BPR leadership directs, it must re-compute a JCL – The agency is to use existing metrics and/or establish new factors for determining when a project turns Red• Eliminated JCL requirement for loosely coupled programs, retained the requirement for tightly coupled programs, and required MDs to submit program impact assessment whenever a new project enters the program (June 2010 PMC) – Eliminate JCL language from NPD 1000.5; reconstitute and update in NPD 7120.5E 7
  8. 8. Management Councils Revisions the Policies and Expectations (Continued)• Include cost and schedule range estimates and associated probabilities for all projects at KDP-B with LCC >$250M (June and Aug 2010 PMCs) – Coordinate policy language and place into 7120.5E• Projects must develop, explain and defend plans and associated probability calculations at KDP-B and C (June 2010 PMC) 8
  9. 9. Final Proposed 7120.5E Language (After the 4 August 2010 PMC)Under 2.5.4.2 Confidence Level Estimating and Budgeting2.5.4.2.4 Projects with an expected life cycle cost in excess of $250million are required to develop cost and schedule estimates andassociated confidence levels for meeting those estimates.(1) At KDP B, a range of cost and schedule estimates and associated confidence levels of meeting those estimates will be provided at KDP-B for projects and single-project programs (see the footnote for paragraph 2.5.4.2.2). This policy applies to both competed (AO) and directed (non-AO) missions. The decision authority will use this information to guide the formulation activities and to establish the target cost and schedule ranges and resources phasing until KDP-C estimates are developed..(2) At KDP C, projects are budgeted to a 70 percent joint cost and schedule confidence level or as approved by the decision 9 authority.
  10. 10. So, What is Required to Meet the KDP-B Programmatic Requirements?• For non-AO missions, projects must provide: – A “high” development (thru launch) cost estimate and associated probability to achieve that cost – A “low” development cost estimate and associate probability to meeting that cost – Cost estimates must include all cost elements, including those for which the project managers is not directly responsible, e.g. the launch vehicle and external partners contributions – High and low schedule durations (thru launch) and associated probabilities of meeting those durations – A high level schedule of activities associated with the low and high life cycle cost estimates – High and low life cycle cost estimates – Year-by-year required resources to meet both the low and high estimates 10
  11. 11. Programmatic Expectations at KDP-B Non-AO Missions (Concluded) – Basis of Estimates. Examples include: • How the cost and schedules were derived • How risks (known and unknown were addressed in the estimates) • How the year-by-year resources were determined• For AO missions, the mission directorate, or designated entity, will provide the same information and calculations specified for the Non-AO missions to the decision authority• Expect to set budget at higher cost range number, but set development cost and schedule estimates at lower numbers – UFE held by program or mission directorate – AO mission development and life cycle costs based on winning bid 11
  12. 12. Status of Program and Project JCLs• During past two years, the following JCLs were completed and presented at PMCs – LDCM, MMS, MSL, SOFIA, GPM*, MAVEN, LADEE, NuStar, and IRIS,• Following JCLs are still in process – JWST• Pilot projects at KDP-B – CxP (cost loaded schedule) – SMAP (parametric) – MAVEN (parametric)* GPM used the CEPA technique; remainder used BURLS 12
  13. 13. JCL Implementation Issues• Most projects had not assembled multiple schedules into a top-level electronically integrated schedule. Required extensive time and effort• Most projects had not quantified their discrete risks nor were they linked to their cost or schedule estimates. Required extensive time and effort• Few projects had a basis for their cost estimates. Most were based on assigned budgets. Costs were usually not separated into fixed and variable• Most projects had lower level schedule of activities detail than costs• Project personnel were not staffed or trained to perform the analysis or use the tools• Use of low cost COTS scheduling and simulation software were inadequate to perform the task – Microsoft Project with @Risk took too long to perform computations• Projects and Standing Review Boards had relatively narrow opinions about discrete risks (Known-knowns)• Consultants were less than successful in convincing projects or SRB members to add other uncertainty (due to known and unknown-unknown risks) In spite of these issues, most projects developed improved plans and explained them better than without the JCL requirement 13
  14. 14. JCL Lessons Learned• Improve project control functions – Ensure projects develop credible cost and schedule basis of estimates – Ensure projects maintain an integrated master schedule that reflects costs and schedule risks, i.e., maintain resource-loaded schedules – Ensure projects continuously quantify and monitor risks and their impact on their plans• Provide projects with historical cost and schedule uncertainly data – To assembly level if possible, but to the subsystem level at a minimum• Provide projects with launch vehicle price and schedule uncertainty data• Provide projects with tools that will permit required turnaround times 14
  15. 15. Action Completed and Planned• Completed – Created near term optimization techniques for MS Project/@Risk • Schedule Shorting Tool • Risk Scenario & Automation Tool (R-SAT)• In-Process – Developing rapid stand alone simulation tools to be used in conjunction with MS Project (replaces @Risk) • DICE • JACS – Collecting cost and schedule uncertainties data – Collecting LV price and schedule uncertainty data – Brainstorming program control revitalization plan• To Be Done: – Deploy new simulations tools – Disseminate LV uncertainty data – Disseminate cost and schedule uncertainty data and educate project staffs on their use 15 – Decide on program control improvements and implement
  16. 16. Backup
  17. 17. Detailed Program and Project Status on JCLs 17
  18. 18. Detailed Program and Project Status on JCLs (Concluded) 18

×