Patrick K. Barker       Glen B. Alleman
      MCR, LLC              Lewis & Fowler
   pbarker@mcri.com    galleman@lewisan...
 Knowing what DONE looks like begins
                                     with the Integrated Master Plan.
              ...
Evidence of DONE         Measures of DONE             Beneficial Outcomes
     IMP           The collection of TPM, EV, an...
No Matter How Great and Destructive Your
Problems May Seem Now, Remember, You’ve
    Probably only Seen the Tip of Them


...
5/30
A Technical Performance Measures …
 State how well the program is achieving the
  planned performance requirements at the...
Document, Baseline,
           IMS, EVM                         Parameter
 IMP, Functional Baseline        Measures Of Ef...
Physical Percent Complete Requires A Tangible
         Measure of “Progress to Plan”
                           Tangible ...
IMP
Describes how the                       Major program milestones or assessment events that substantiate system
capabil...
Measure the progress to plan using Physical % Complete at the
               Accomplishment Criteria (AC) and CWBS level

...
0.0                       TOTAL BRILLIANT EYES PROGRAM
                                                                   ...
 Neither cost performance nor schedule performance nor technical
  performance measurement can be fully evaluated by itse...
How to Define DONE




 Determine Physical Percent Complete based in performance measurement
  – Compare actual performan...
Risk         Cost

IMS          PMB          SOW

      WBS          TPM


                                14/30
Perform     Determine                    Develop           Develop
                                                       ...
Making Trades
     Between
  Cost, Schedule,
        and
   Performance
Baselines is a Ponzi
      Scheme
     When we’re ...
A Really Simple Example of Physical
           Percent Complete
                          Painting a room is simple examp...
18/30
 The notion of measuring physical percent
                            complete is at the core of every project
          ...
 The increasing maturing of a product or service is described through Events or Milestones,
  Accomplishments, Criteria, ...
21/30
IEEE 1220: 6.8.1.5,             EIA–632: Glossary            CMMI for Development
   Performance–based                    ...
23/30
EV TPM       What this tells the PM                    Derived Measurements
                                              ...
Control Account
                      WP1: Chassis                      WP2: Array




              Completion Criteria &...
Using Periodic Program Events to Calibrate “Doneness”
                                                                    ...
Connecting Risk Management with Measures of DONE

                                                                    Risk...
Risk         Cost

IMS          PMB      SOW

      WBS          TPM




                            28/30
A Credible Performance Measurement System
     Assures the information is available to the decision maker, needed to deliv...
Connecting Measures of Cost, Schedule, and
Technical Performance
means the
difference
between …

Measuring Actual Physical...
Upcoming SlideShare
Loading in …5
×

Physical Percent Complete - june 2010 (6)

4,626 views
4,506 views

Published on

Published in: Technology, Business
1 Comment
2 Likes
Statistics
Notes
  • Great Presentation on combining Earned Vale and Technical Performance
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
4,626
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
168
Comments
1
Likes
2
Embeds 0
No embeds

No notes for slide

Physical Percent Complete - june 2010 (6)

  1. 1. Patrick K. Barker Glen B. Alleman MCR, LLC Lewis & Fowler pbarker@mcri.com galleman@lewisandfowler.com (703) – 898–6354 (303) – 241 9633 1/30
  2. 2.  Knowing what DONE looks like begins with the Integrated Master Plan.  Recognizing what DONE looks like when it arrives means measuring the planned Technical Performance.  Measuring Physical Percent Complete tells us how far we have moved toward DONE by calculating the “Earned Value” we’ve achieved.  Connecting Earned Value, Technical Performance, and Physical Percent Connect Quality with Performance Complete in the IMP establishes a credible measure of Progress to Plan.  Without these four elements we don’t have a clue to what DONE looks like or if we’ll ever get there as planned. 2/30
  3. 3. Evidence of DONE Measures of DONE Beneficial Outcomes IMP The collection of TPM, EV, and The IMP then defines Accomplishment PPC measures defined as “exit the measurable path to Criteria criteria” for the Work Packages the planned maturity. (AC) performing the work. Technical The actual technical Evidence that the Performance performance measured at a product or service is Measure defined date compared to meeting the planned (TPM) planned technical performance. maturity or quality. Evidence of completion Physical Percent Unit of measure to assess the of the planned work in Complete planned technical performance, units meaningful to the (PPC) evidenced by tangible outcomes. customer. Cost and schedule needed to The performance Earned Value deliver the planned technical measured in units of (EV) performance on baseline. dollars. 3/30
  4. 4. No Matter How Great and Destructive Your Problems May Seem Now, Remember, You’ve Probably only Seen the Tip of Them 4/30
  5. 5. 5/30
  6. 6. A Technical Performance Measures …  State how well the program is achieving the planned performance requirements at the planned delivery time.  Use actual or predicted values from: – Engineering measurements – Tests – Experiments – Prototypes  For Example: – Response time – Flight range – Power consumption – Static takeoff weight – Product Quality 6/30
  7. 7. Document, Baseline, IMS, EVM Parameter  IMP, Functional Baseline Measures Of Effectiveness (MOE)   IMP, WBS, Functional Baseline Measures Of Performance (MOP)   IMP, Allocated Baseline Technical Performance Measure  TPM Milestones And Planned  IMS  Values  Work Packages TPM% Complete Criteria  7/30
  8. 8. Physical Percent Complete Requires A Tangible Measure of “Progress to Plan”  Tangible evidentiary materials measure progress to plan.  Pre–defined existence of this evidence in meaningful units of measure established before starting work. No Hand Waving Allowed “Show Me The Money”  Progress is defined in these same units of measure. 8/30
  9. 9. IMP Describes how the Major program milestones or assessment events that substantiate system capabilities will be maturity (initial, progress, or final). These milestones or assessment events delivered and Events deliver the specific capabilities for the system on planned dates. how these Milestones capabilities will be recognized Specified result, substantiating a Milestone or Event, that indicates Accomplishment maturity or progress for each product or process Definitive measures substantiating the Accomplishment maturity Criteria level. Completion of specific work that ensures closure of a specified Accomplishment IMS Work activities performed to produce the deliverables that Work Packages and Tasks fulfill the requirements that enable the capabilities Supplemental Schedules 9/30
  10. 10. Measure the progress to plan using Physical % Complete at the Accomplishment Criteria (AC) and CWBS level Program Statement of CWBS Events Work Completed SA’s are Work structure Aligned entry criteria for aligned to Program Events SOW Significant Accomplishments Completed Work Aligned Packages are exit Defines Aligned criteria for Tasks Accomplishment CDRLs and Tasks Contained Criteria Aligned Deliverables Aligned in Work Packages Describes increasing Documents the product Work necessary to product maturity as 0/100 or maturity that is aligned with mature products EVMS SD guidance SOW and CWBS grouped by CWBS 10/30
  11. 11. 0.0 TOTAL BRILLIANT EYES PROGRAM Planned Profile Tolerance Band 1.0 SPACE BRILLIANT EYES SYSTEM Technical Parameter Value 1.1 System-Level Costs (e.g., Vehicle Weight, lb) 1.2 Space Vehicle (SV) Segment 15 1.2.1 SV Program Level 1.2.2 Space Vehicle Prime Mission Equipment Achieved To Date 1.2.2.1 Space Software 1.2.2.2 Space Vehicle 1.2.2.2.1 1.2.2.2.2 Space Vehicle IA&T Sensor Payload Variation 1.2.2.2.3 1.2.2.2.4 Insertion Vehicle Survivability 10 1.2.3 Prototype Lot 1.2.4 1.2.5 Spare Parts Technology and Producibility Threshold 1.2.6 Aerospace Ground Equipment 5 Planned 1.3 1.2.7 Launch Support Engineering Change Orders (ECOs) Value Current 1.4 1.5 Other Government Costs Risk Estimate A WBS can be displayed as a Milestones “linear” List. Program Cost is often Calculated by summing the Costs of Technical performance measurement have often been All Items on That List … but that treated as an engineering function only, not typically ignores correlation and basic math: linked with EVM, although it shares a common, almost distributions don’t add that way. identical, language structure with EVM. 2 1 6 4 3 5 The Schedule Network is Nonlinear, and Program Duration Cannot be Calculated by Adding Together the Durations of 3 1 6 9 2 4 All Activities in the Network. And since time is money it is important for schedule slips to reflect cost 6 9 2 1 4 2 1 11/30
  12. 12.  Neither cost performance nor schedule performance nor technical performance measurement can be fully evaluated by itself. 12/30
  13. 13. How to Define DONE  Determine Physical Percent Complete based in performance measurement – Compare actual performance against an integrated baseline planned performance of the integrated cost, schedule and technical goals.  This measurement is possible within a performance measurement system – Use the organization’s internal management control system to provide decision makers with specific performance information about the progress and expenditures against the planed progress and expenditures.  In many projects where EVM systems are implemented, what a CAM determines (“eyeballing”) as physical percent complete often differs from what is reported via the EV techniques. – This disconnect in measurement of “done–ness” is a clear sign that a cost, schedule and/or technical performance problem is going to materialize before the PM can properly react. 13/30
  14. 14. Risk Cost IMS PMB SOW WBS TPM 14/30
  15. 15. Perform Determine Develop Develop Approve Functional Scope and Technical Technical PMB Analysis Approach Logic Baseline Technical Develop Performance WBS Baseline Estimate Finalize Define Sequence Finalize Time Apportioned Activities Activities Schedule Durations Milestones Schedule Identify Performance Apportioned Milestones Baseline Cost Determine Prepare Resource Determine Resource Cost Load Funding Performance Requirement Estimate Schedule Constraints Baseline 15/30
  16. 16. Making Trades Between Cost, Schedule, and Performance Baselines is a Ponzi Scheme When we’re on baseline, the algebraic relationship between C,S,P, means when there is a change everyone loses 16/30
  17. 17. A Really Simple Example of Physical Percent Complete  Painting a room is simple example to focused on a notion of percent complete. But how about a flight avionics system?  What are the measures of physical percent complete? ‒ Design basis complete and verified ‒ Hardware platform increasing capabilities ‒ Software system increasing capabilities ‒ Integration and test of 1st flight article ‒ Operational verification and validation complete Where can we look for these measures of physical percent complete? It’s Obvious – in the Integrated Master Plan’s Accomplishment Criteria 17
  18. 18. 18/30
  19. 19.  The notion of measuring physical percent complete is at the core of every project management method.  It answers the question of what DONE is in units of measure meaningful to the program.  This helps us determine answers to No stretching the truth is allowed once we questions that include, but are not limited start using Technical to: Performance – What does DONE look like for today, for this Measures with week, or for this month? tangible evidence. – What does DONE look like for entry/exit into the technical review? – What does DONE look like for quality control? – What does DONE look like for the customer? 19/30
  20. 20.  The increasing maturing of a product or service is described through Events or Milestones, Accomplishments, Criteria, and their Work Packages or Planning Packages. The Structure of a  Each Event or Milestone represents the availability of one or more capabilities. Credible IMP  The presence of these capabilities is measured by the Accomplishments and their Criteria.  Accomplishments are the pre–conditions for the maturity assessment of the product or service at Program Events each Event or Milestone. Define the availability  This hierarchy decomposes the System Capabilities into Requirements, Work Packages, and the of a Capability at a point in activities which produce deliverables. This hierarchy also describes increasing program maturity time. resulting from the activities contained in the Work Packages.  Performance of the work activities, Work Packages, Criteria, Accomplishments, and Events or Milestones is measured in units of “physical percent complete” by connecting Earned Value with Technical Performance Measures. Accomplishments Represent requirements that enable Capabilities. Criteria Represent Work Packages that fulfill Requirements. Work Work Work Each WP must Package Package Package have a measure of Physical Work Percent Work Work Complete Package Package Package Work Work Work package Package Package 20/30
  21. 21. 21/30
  22. 22. IEEE 1220: 6.8.1.5, EIA–632: Glossary CMMI for Development Performance–based Requirements Development progress measurement TPMs are key to Predict future value of key Specific Practice (SP) 3.3, progressively assess technical parameters of the Analyze Requirements technical progress end system based on Typical work product: current assessments Technical Performance Measures Establish dates for Planned value profile is time– Sub Practice (SP): – Checking progress phased achievement projected Identify TPMs that will be – Meeting full conformance – Achievement to date tracked during development to requirements – Technical milestone where TPM evaluation is reported 22/30
  23. 23. 23/30
  24. 24. EV TPM What this tells the PM Derived Measurements Number of requirements approved, percentage How well the system is maturing as requirements growth, number TBD/TBR closure per plan, Requirements compared to expectations estimated impact of changes, defect profile, defect density, cycle time for requirements changes System Definition Change If changes are being made in a timely Approval rates, closure rates, cycle times, priority density Backlog manner Number interfaces approved, percent interface growth, Risk associated with interface number TBD/TBR closure per plan, estimated impact of Interface development and maturity interface changes, defect profile, defect density, cycle time for interface changes If requirements are being validated with Requirements validation rate. Requirement Validation applicable stakeholders appropriate to Percent requirements validated the level of requirement. If requirements are being verified Requirements verification rate. Requirement Verification appropriate to the level of requirement. Percent requirements verified Approval rate, distribution of dispositions, approval rate Work Product Approval Work progress and approval efficiency performance Technical/Design Review Progress in closing significant action Closure rates, action item closure performance, variance Action Closure items from technical reviews from thresholds Maturity of key components, Component –Subsystem– Element –System TRL, Technology Technology Maturity subsystems, elements to expectations opportunity exposure, technology obsolescence exposure Effectiveness of the risk management Percentage of risk handling actions closed on time, percent Technical Risk Handling process overdue, percent of risks meeting handling expectations Technical Staffing And Adequacy of technical effort and Technical effort staffing, variance. Skills dynamics of actual staffing mix Efficiency by labor category Current performance status, projections Delta performance (planned versus actual). Technical Performance and associated risk Delta performance to thresholds, objectives 24/30
  25. 25. Control Account WP1: Chassis WP2: Array Completion Criteria & Taking “Value” for Work Packages Chassis June July August Sept Current 100 150 200 250 Completion Requirement Design 80% All TPM within Build complete; Criteria Quality (drawings) pre–build All TPM Completed; TPM tolerances via validated by 2, 5 and 6 within refined actual PDR tolerances performance measurement via basic model; TPM 1 performance and 6 verified model via bench testing Cummulative 100 250 450 700 25/30
  26. 26. Using Periodic Program Events to Calibrate “Doneness” INPUT PROCESS OUTPUT Program Requirements & Program–level Risk Assessment of the Scope technical baseline Lvl 1 WBS Program Cost & Schedule Estimate Cumulative Distribution Function 1 0.9 IPT–level risks 0.8 0.7 Probability 0.6 0.5 0.4 When it comes right down to it, the (10’s) 0.3 0.2 0.1 0 IBR is all about the technical baseline. IBRs should accompany each major 250000 300000 350000 400000 450000 500000 550000 600000 650000 Total Cost Rollup of quantified process & product risks Date: 3/17/2009 8:35:32 AM Samples: 2000 Unique ID: 114 Name: FIST GEO-1 Completion Std Deviation: 71.23 days 95% Confidence Interval: 3.12 days Each bar represents 25 days baseline evolution. 0.16 0.14 1.0 0.9 Completion Probability Table In the end, the question implicitly Cumulative Probability Prob Date Prob Date 0.8 0.12 0.05 10/21/2010 0.55 3/24/2011 asked of the PM each time is: 0.7 0.10 11/11/2010 0.60 4/7/2011 Frequency 0.10 0.6 0.15 12/2/2010 0.65 4/25/2011 0.08 0.5 0.20 12/20/2010 0.70 5/10/2011 0.4 0.25 1/3/2011 0.75 5/27/2011 0.06 “What is your confidence level in your 0.30 1/14/2011 0.80 6/16/2011 0.04 0.02 0.3 0.2 0.1 0.35 0.40 0.45 1/28/2011 2/9/2011 2/22/2011 0.85 0.90 0.95 7/5/2011 7/26/2011 9/9/2011 CA–level risks 7/6/2010 3/17/2011 Completion Date 3/16/2012 0.50 3/8/2011 1.00 3/16/2012 program’s ability to hit cost, (100’s) IBR Guidance schedule, and scope targets associated with your technical baseline” 26/30
  27. 27. Connecting Risk Management with Measures of DONE Risk: CEV-037 - Loss of Critical Functions During Descent 24 Correlate the analytical model 22 Conduct focus splinter review 20 Develop analytical model to de Conduct Force and Moment Wind 18 Conduct Block 1 w ind tunnel te 16 Conduct w ind tunnel testing of Conduct w ind tunnel testing of 14 Flight Application of Spacecra 12 10 CEV block 5 w ind tunnel testin 8 6 4 In-Flight development tests of 2 Damaged TPS flight test 0 3.Jul.06 1.Jul.11 31.Mar.05 5.Oct.05 1.Jun.07 1.Jan.10 16.Dec.10 15.Sep.06 3.Apr.06 1.Apr.08 1.Aug.08 1.Apr.09 Planned Risk Level Planned (Solid=Linked, Hollow =Unlinked, Filled=Complete) Risk Response Milestone Date and Risk ID in traceable between IMS RM Tool and IMS 27/30
  28. 28. Risk Cost IMS PMB SOW WBS TPM 28/30
  29. 29. A Credible Performance Measurement System Assures the information is available to the decision maker, needed to deliver on‒time, on‒budget, and on‒specification = Technical Performance Measures Cost + Schedule Conventional Earned Value Cost Baseline Technical Performance Schedule Baseline  The Master Schedule  Earned Value is diluted by  Requirements are used to derive Basis of missing technical decomposed into physical Estimate (BOE) not the performance. deliverables. other way around.  Earned Value is diluted by  Deliverables are produced  Probabilistic cost postponed features. through Work Packages. estimating uses past  Earned Value is diluted by  Work Packages are performance and cost risk non compliant quality. assigned to accountable modeling.  All these dilutions require manager.  Labor, Materiel, and other adjustments to the  Work Packages are direct costs accounted for Estimate at Complete sequenced to form the in Work Packages. (EAC) and the To highest value stream with  Risk adjustments for all Complete Performance the lowest technical and elements of cost. Index (TCPI). programmatic risk. 29/30
  30. 30. Connecting Measures of Cost, Schedule, and Technical Performance means the difference between … Measuring Actual Physical Percent Complete against the Planned Physical Percent Complete at the time that Physical Percent Complete was planned to be achieved is the foundation of determining a program’s performance. Doing anything else leads to disappointment. 30/30

×