Technical Performance Measures


Published on

Technical Performance Measure course from PMI-College of Performance Management Conference, June 2010

Published in: Technology, Business
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Technical Performance Measures

  1. 1. CPM–500 EVM/SE/TPM Integration CPM–500–B/C/F : Integrating Systems Engineering with Earned Value Management Lesson 3 CPM–500F: Technical Performance Measures Glen B. Alleman Lewis & Fowler (303) 241 9633 June 2010 Naples, Florida Professional Education Program (Training Track) presented by PMI–College of Performance Management faculty Rights Reserved 1/63
  2. 2. CPM–500 EVM/SE/TPM Integration The Purpose Of This Series  This is the last “connected” lesson in a 3–part series that explores the integration of System Engineering (SE) and Earned Value Management (EVM).  You don’t need to attend all three lessons – but it helps in Connecting The Dots  Why have this series? – Often, project control specialists see the planning of the cost and schedule baselines as an isolated activity disconnected from the technical or engineering aspects. – This type of thinking significantly impacts development of the baseline which results in poor maintenance and eventual irrelevance of the baseline – We claim “victory” if after any one of these lessons, you can recognize the importance of integrating these disciplines such that he/she will advocate greater cooperation between the engineering and project control elements of his/her respective organization Rights Reserved 2/63
  3. 3. CPM–500 EVM/SE/TPM Integration Lesson 3 Objectives In this lesson we will:  Understand how to establish credibility for Technical Performance Measures by using Earned Value and Systems Engineering to measure progress  Establish the processes of defining and measuring technical performance  Establish the role of the technical baseline in program management  Learn how TPM’s are integrated into planning and execution at the control account level through examples and a hands on exercise Rights Reserved 3/63
  4. 4. CPM–500 EVM/SE/TPM Integration Can Earned Value Alone Get Us To Our Destination?  How do we increase visibility into the program’s performance?  How do we reduce cycle time to deliver the product?  How do we foster accountability?  How do we reduce risk?  How do we start this journey to success? Increasing the Probability of Success means we have to Connect The Dots to Reach Our Destination Rights Reserved 4/63
  5. 5. CPM–500 EVM/SE/TPM Integration To Achieve Success … We Need to … ©gapingvoid ltd Rights Reserved 5/63
  6. 6. CPM–500 EVM/SE/TPM Integration Learnings from Lesson 1 and 2 How cost and schedule (EV) are connected to the Performance Measurement Baseline Rights Reserved 6/63
  7. 7. CPM–500 EVM/SE/TPM Integration Increasing the Probability of Program Success Means … Building A Credible Performance Measurement Baseline Risk Cost IMP/IMS PMB SOW WBS TPM This is actually harder than it looks! Rights Reserved 7/63
  8. 8. CPM–500 EVM/SE/TPM Integration Doing This Starts With Some Guidance Systems engineering uses technical performance measurements to balance cost, schedule, and performance throughout the life cycle. Technical performance measurements compare actual versus planned technical development and design. They also report the degree to which system requirements are met in terms of performance, cost, schedule, and progress in implementing risk handling. Performance metrics are traceable to user–defined capabilities. ― Defense Acquisition Guide ( In The End ― It’s All About Systems Engineering Rights Reserved 8/63
  9. 9. CPM–500 EVM/SE/TPM Integration Guidance for Measures of Effectiveness, Performance, and Technical Performance Our starting point is not EVM, it’s Systems Engineering  MOE’s are an essential part of Systems Engineering, guided by IEEE 1220 and EIA 632.  System’s Engineers drive the content of all measurement items, customer or supplier. Rights Reserved 9/63
  10. 10. CPM–500 EVM/SE/TPM Integration Just A Reminder Of The … Primary Elements of Earned Value Cost Funding margin Schedule margin for for under Over cost or Over cost or over target baseline performance under over (OTB) performance schedule Over Technical schedule or Performance under Schedule performing Schedule margin for underperformance or schedule extension Rights Reserved 10/63
  11. 11. CPM–500 EVM/SE/TPM Integration Previous Approaches Using EV Are Mostly Unsuccessful In Connecting These  Traditional approaches to program management are retrospective – Cost and schedule of Earned Value – Risk Management – Systems Engineering  Reporting past performance – Sometimes 30 to 60 days old – Variances are reporting beyond the widow of opportunity for correction Rights Reserved 11/63
  12. 12. CPM–500 EVM/SE/TPM Integration It’s All Been Said Before. We Just Weren’t Listening… … the basic tenets of the process are the need for seamless management tools, that support an integrated approach … and “proactive identification and management of risk” for critical cost, schedule, and technical performance parameters. ― Secretary of Defense, Perry memo, May 1995 Why Is This Hard To Understand?  We seem to be focused on EV reporting, not the use of EV to manage the program.  Getting the CPR out the door is the end of Program Planning and Control’s efforts, not the beginning. Rights Reserved 12/63
  13. 13. CPM–500 EVM/SE/TPM Integration The Gap Seems To Start With A Common Problem Many Times, The Information from Cost, Schedule, Techncial Performance, and Risk Management Gets Mixed Up When We Try to Put Them Together Rights Reserved 13/63
  14. 14. CPM–500 EVM/SE/TPM Integration When We Put The Cart Before The Horse, We Discover …  EVM really doesn’t do its job effectively  Most of the time EV has no measure of quality or compliance with technical requirements.  EV measures progress to plan in units of “money,” not tangible value to the customer  Most EV System Descriptions fail to connect the dots between cost, schedule, and technical performance – even though instructed to do so in the official guidance Rights Reserved 14/63
  15. 15. CPM–500 EVM/SE/TPM Integration The NDIA EVM Intent Guide Says Notice the inclusion of Technical along with Cost and Schedule That’s the next step is generating Value from Earned Value EV MUST include the Technical Performance Measures Rights Reserved
  16. 16. CPM–500 EVM/SE/TPM Integration Back To Our Technical Performance Measures Technical Performance Measures do what they say, Measure the Technical Performance of the product or service produced by the program Rights Reserved 16/63
  17. 17. CPM–500 EVM/SE/TPM Integration What’s Our Motivation for “Connecting the Dots?” TPMs are a set of measures that provide the supplier and acquirer with insight into progress to plan of the technical solution, the associated risks, and emerging issues. Technical Performance Measures …  Provide program management with information to make better decisions  Increase the probability of delivering a solution that meets both the requirements and mission need We’ve been talking about this since as early as 1984, in Technical Performance Measurement Handbook, Defense Systems Management College, Fort Belvoir, VA 22060 Rights Reserved 17/63
  18. 18. CPM–500 EVM/SE/TPM Integration Measure of Effectiveness (MoE) The operational measures of success that are closely related to the achievements of the mission or operational objectives evaluated in the operational environment, under a specific set of condition Measures of Effectiveness …  Are stated by the buyer in units meaningful to the buyer  Focus on capabilities independent of any technical implementation MoE Belong to the End User “Technical Measurement,” INCOSE–TP–2003–020–01 Rights Reserved 18/63
  19. 19. CPM–500 EVM/SE/TPM Integration Measure of Performance (MoP) Measures that characterize physical or functional attributes relating to the system operation, measured or estimated under specific conditions Measures of Performance are …  Attributes that assure the system has the capability to perform  Assessment of system to assure it meets design requirements necessary to satisfy the MOE MoP’s belong to the Program – Developed by the Systems Engineer, Measured By CAMs, and Analyzed by PP&C “Technical Measurement,” INCOSE–TP–2003–020–01 Rights Reserved 19/63
  20. 20. CPM–500 EVM/SE/TPM Integration Key Performance Parameters (KPP) Represent the capabilities and characteristics so significant that failure to meet them can be cause for reevaluation, reassessing, or termination of the program Key Performance Parameters …  have a threshold or objective value  Characterize the major drivers of performance  Are considered Critical to Customer (CTC) The acquired defines the KPPs during the operational concept development – KPPs say what DONE looks like “Technical Measurement,” INCOSE–TP–2003–020–01 Rights Reserved 20/63
  21. 21. CPM–500 EVM/SE/TPM Integration Technical Performance Measures (TPM) Attributes that determine how well a system or system element is satisfying or expected to satisfy a technical requirement or goal Technical Performance Measures …  Assess design progress  Define compliance to performance requirements  Indentify technical risk  Are limited to critical thresholds  Include projected performance “Technical Measurement,” INCOSE–TP–2003–020–01 Rights Reserved 21/63
  22. 22. CPM–500 EVM/SE/TPM Integration Dependencies Between Measures Stakeholders Define Needs and Capabilities Suppliers Define Physical Solutions that in terms of Operational Scenarios meet the needs of the Stakeholders KPP Mission MoE MoP TPM Need Operational Measures that Measures used to measures of characterize assess design success related to physical or progress, the achievement functional compliance to of the mission or attributes relating performance operational to the system requirements, and objective being operation technical risks evaluated. “Coming to Grips with Measures of Effectiveness,” N. Sproles, Systems Engineering, Volume 3, Number 1, pp. 50–58 Rights Reserved 22/63
  23. 23. CPM–500 EVM/SE/TPM Integration “Candidates” for Technical Measures Concept Description Useful Life Physical Size and Stability Weight Volumetric capacity Accuracy Functional Correctness Power performance Supportability Maintainability All the “ilities” Dependability Reliability Reliability = Mean Time Failure Utilization Efficiency Response time Throughput Suitability for Purpose Readiness Rights Reserved 23/63 INCOSE Systems Engineering Handbook
  24. 24. CPM–500 EVM/SE/TPM Integration “Measures” of Technical Measures Attribute Description Measured technical progress or estimate of Achieved to Date progress Value of a technical parameter that is predicted to Current Estimate be achieved Point in time when an evaluation of a measure is Milestone accomplished Planned Value Predicted value of the technical parameter Planned Performance Profile representing the project time phased Profile demonstration of a technical parameter Tolerance Band Management alert limits Threshold Limiting acceptable value of a technical parameter  Demonstrated technical variance Variances INCOSE Systems Engineering Handbook  Predicted technical variance Rights Reserved 24/63
  25. 25. CPM–500 EVM/SE/TPM Integration A Familiar Graphic of TPMs TPM Upper Limit Planned Profile Current Estimate Planned Value Mean To Between Failure Threshold Variance Lower Limit Achieved to Date Milestones Time = Program Maturity Rights Reserved 25/63
  26. 26. CPM–500 EVM/SE/TPM Integration A Simple Method of Assembling the TPMs MOE / MOP KPP / TPM Risks Define the Assess the Select Technical planned impact on Risk Performance progress for from this Parameters each TPM progress Parameters Progress Risk  Weight XXXX  Speed XXXX  MTBF XXXX  Loiter Time XXXX Rights Reserved 26/63
  27. 27. CPM–500 EVM/SE/TPM Integration TPMs from an Actual Program James Webb Space Telescope Rights Reserved 27/63
  28. 28. CPM–500 EVM/SE/TPM Integration TPMs from an Actual Program Chandra X–Ray Telescope Rights Reserved 28/63
  29. 29. CPM–500 EVM/SE/TPM Integration What Does A Real Technical Performance Measure Look Like? Not that bagels are not interesting in Lesson 1 and 2, but let’s get ready to look at a flying machine. Rights Reserved 29/63
  30. 30. CPM–500 EVM/SE/TPM Integration The WBS for a UAV TPMs Start With The WBS 1.1 Air Vehicle 1.1.1 Sensor Platform 1.1.2 1.1.2 Airframe Airframe 1.1.3 Propulsion 1.1.4 On Board Comm 1.1.5 Auxiliary Equipment 1.1.6 Survivability Modules 1.1.7 Electronic Warfare Module 1.1.8 On Board Application & System SW 1.3 Mission Control / Ground Station SW 1.3.1 Signal Processing SW 1.3.2 Station Display 1.3.3 Operating System 1.3.4 ROE Simulations 1.3.5 Mission Commands Rights Reserved 30/63
  31. 31. CPM–500 EVM/SE/TPM Integration What Do We Need To Know About This Program Through TPMs  What WBS elements represent the TPMs?  What Work Packages produce these WBS elements?  Where do these Work Packages live in the IMS?  What are the Earned Value baseline values for these Work Packages?  How are going to measure all these variables?  What does the curve look like for these measurements? Rights Reserved 31/63
  32. 32. CPM–500 EVM/SE/TPM Integration Let’s Connect The Dots Technical and Programmatic Risks Connected to the WBS and IMS BCWS at the Work Package, rolled to the Control Account IMS contains all the Work Packages, BCWS, Risk Cost Risk mitigation Named plans, and rolls to IMP/IMS PMB SOW Deliverables the Integrated defined in the WBS Master Plan to WBS TPM measure increasing maturity TPMs attached to each critical deliverables in the WBS and identified in The Products and each Work Package in the Processes that produce IMS, used to assess them in a “well structured” maturity in the IMP decomposition in the WBS Rights Reserved 32/63
  33. 33. CPM–500 EVM/SE/TPM Integration Verifying Each TPM Evidence that we’re in compliance With our submitted ROM what are the values we need to get Do we know what we promised to CA through Integrated Baseline Review «how do we measure deliver, now that we’ve won? weight for each program event» The contributors to the vehicle weight are confirmed and the Can we proceed into preliminary SFR upper limits defined in the product architecture and design? requirements flow down database (DOORS) into a model Can we proceed into the System Do we know all drivers of vehicle weight? Can we bound their SRR Development and Demonstration upper limits? Can the subsystem owners be successful within (SDD) phase these constraints uses a high fidelity model? Can we start detailed design, and Does each subsystem designer have the target component meet the stated performance weight target and have some confidence they can stay below PDR requirements the upper bound? Can this be verified in some tangible way? within cost, schedule, Either through prior examples or a lab model? risk, and other constraints? Can the system proceed to Do we know all we need to know to start the fabrication of fabrication, demonstration, and test, the first articles of the flight vehicle. Some type of example, CDR with the within cost, schedule, risk, maybe a prototype is used to verify we’re inside the lines and other system constraints. Can the system ready to Does the assembled vehicle fall within the weight range limits TRR Rights Reserved proceed into formal test? for 1st flight – will this thing get off the ground? 33/63
  34. 34. CPM–500 EVM/SE/TPM Integration TPM Trends & Responses Design Model ROM in Proposal Detailed Design Model Bench Scale Model Measurement Technical Performance Measure 28kg Prototype Measurement Vehicle Weight Flight 1st Article 26kg 25kg 23kg CA SFR SRR PDR CDR TRR EV Taken, planned values met, tolerances kept, etc. Rights Reserved Dr. Falk Chart – modified 34/63
  35. 35. CPM–500 EVM/SE/TPM Integration The Assessment Of Weight As A Function Of Time  At Contract Award there is a Proposal grade estimate of vehicle weight  At System Functional Review, the Concept of Operations is validated for the weight  At System Requirements Review the weight targets are flowed down to the subsystems components  At PDR the CAD model starts the verification process  At CDR actual measurements are needed to verify all models  At Test Readiness Review we need to know how much fuel to put on board for the 1st flight test Rights Reserved 35/63
  36. 36. CPM–500 EVM/SE/TPM Integration The WBS for a UAV Airframe Weight TPM 1.1 Air Vehicle The planned weight is 1.1.1 Sensor Platform 25kg. The actual weight is 1.1.2 Airframe 25.5kg. 1.1.2 Airframe Close to plan! So we are doing okay, right? CA SFR SRR PDR CDR TRR Planned Value 28.0kg 27.0kg 26.0kg 25.0kg 24.0kg 23.0kg Actual Value 30.4kg 29.0kg 27.5kg 25.5kg Moderate Low Low Very Low (less Assessed Risk >2.0kg off 1–2 kg off 1–2 kg off than 1.0 kg to TRR target target target off target) Program– Actual Actual Program– Planned “Similar to” unique design measurement measurement ROM unique design Method Estimate model with of bench–test of prototype model validated data components airframe Actual “Similar to” Method Estimate ROM ROM ROM Here’s the Problem Rights Reserved 36/63
  37. 37. CPM–500 EVM/SE/TPM Integration Is This A Problem? You Bet’ya It’s A Problem!  The measurement is close to the planned value,  But the planned method of measurement is a program unique design model with validated data,  But the actual method of measurement is a rough order of magnitude estimate,  No improvement in fidelity since the System Functionality Review (SFR), and  The TPM provides no new information – so we’re probably late and don’t know it yet. Rights Reserved 37/63
  38. 38. CPM–500 EVM/SE/TPM Integration Raison d'etre for Technical Performance Measures The real purpose of Risk Cost Technical Performance Measures is to reduce IMP/IMS PMB SOW Programmatic and WBS TPM Technical RISK Rights Reserved 38/63
  39. 39. CPM–500 EVM/SE/TPM Integration Buying Down Risk with TPMs Risk: CEV-037 - Loss of Critical Functions During Descent  “Buying down” risk is 24 22 20 Correlate the analytical model Conduct focus splinter review Develop analytical model to de planned in the IMS.  MoE, MoP, and KPP Conduct Force and Moment Wind 18 Conduct Block 1 w ind tunnel te 16 Conduct w ind tunnel testing of Conduct w ind tunnel testing of defined in the work 14 Flight Application of Spacecra 12 package for the critical 10 CEV block 5 w ind tunnel testin 8 6 4 2 In-Flight development tests of Damaged TPS flight test measure – weight.  If we can’t verify 0 3.Jul.06 1.Jul.11 31.Mar.05 5.Oct.05 1.Jun.07 1.Jan.10 16.Dec.10 15.Sep.06 3.Apr.06 1.Apr.08 1.Aug.08 1.Apr.09 Planned Risk Level Weight risk reduced from Planned (Solid=Linked, Hollow =Unlinked, Filled=Complete) Weight confirmed ready to fly – it’s we’ve succeeded, then RED to Yellow GREEN at this point the risk did not get reduced.  The risk may have gotten worse. Rights Reserved 39/63
  40. 40. CPM–500 EVM/SE/TPM Integration Increasing the Probability of Success with Risk Management  Going outside the TPM limits always means cost and schedule impacts  “Coloring Inside the Lines” means knowing the how to keep the program GREEN, or at least stay close to So much for our strategy of winning GREEN through technical dominance Rights Reserved 40/63
  41. 41. CPM–500 EVM/SE/TPM Integration Connecting the EV Variables Integrating Cost, Schedulele, and Technical Performance Assures Program Management has the needed performance information to deliver on‒time, on‒budget, and on‒specification = Technical Performance Measures Cost + Schedule Conventional Earned Value Cost Baseline Technical Performance Schedule Baseline  Master Schedule used to  Earned Value is diluted  Requirements are derive Basis of Estimate by missing technical decomposed into (BOE) not the other way performance. physical deliverables. around.  Earned Value is diluted  Deliverables are  Probabilistic cost by postponed features. produced through Work estimating uses past  Earned Value is diluted Packages. performance and cost by non compliant quality.  Work Packages are risk modeling.  All these dilutions assigned to accountable  Labor, Materiel, and require adjustments to manager. other direct costs the Estimate at Complete  Work Packages are accounted for in Work (EAC) and the To sequenced to form the Packages Complete Performance highest value stream  Risk adjustments for all Index (TCPI). with the lowest technical elements of cost. Rights Reserved and programmatic risk. 41/63
  42. 42. CPM–500 EVM/SE/TPM Integration TPM Checklist MoE MoP TPM Traceable to needs, Traceable to applicable Traceable to applicable MoPs, goals, objectives, and MOEs, KPPs, system level system element performance, risks performance requirements, requirements, objectives, and risks risks, and WBS elements Defined with associated Focused on technical risks Further decomposed, KPPs and supports trades budgeted, and allocated to between alternative lower level system elements in solutions the WBS and IMS Each MoE independent Provided insight into Assigned an owner, the CAM from others system performance and Work Package Manager Each MoE independent Decomposed, budgeted Sources of measure identified of technical any solution and allocated to system and processes for generating elements the measures defined. Address the required Assigned an “owner,” the Integrated into the program’s KPPs CAM and Technical IMS as part of the exit criteria Manager Rights Reserved for the Work Package 42/63
  43. 43. CPM–500 EVM/SE/TPM Integration Increasing the Probability of Program Success Means … Building A Credible Performance Measurement Baseline Risk Cost IMP/IMS PMB SOW WBS TPM Using the Check List – “Connect the Dots” Rights Reserved 43/63
  44. 44. CPM–500 EVM/SE/TPM Integration Rights Reserved 44/63
  45. 45. CPM–500 EVM/SE/TPM Integration Backup Materials Knowledge is of two kinds. We know a subject ourselves, or we know where we can find information on it — Samuel Johnson Rights Reserved 45/63
  46. 46. CPM–500 EVM/SE/TPM Integration Many of Sources for Connecting the Dots OMB Circular A–11, Section 300 Interim Defense Acquisition Guidebook (DAG) 6/15/09 GAO Report 06–250 Systems Engineering Plan (SEP) Preparation Guide 4/08 DoDI 5000.02, Operation of the Defense WBS Handbook, Mil–HDBK–881A (WBS) 7/30/05 Acquisition System (POL) 12/08 Integrated Master Plan (IMP) & Integrated Guide for Integrating SE into DOD Acquisition Master Schedule Preparation & Use Guide Contracts 12/06 (IMS) 10/21/05 Defense Acquisition Program Support Guide to the Project Management Institute Body of Methodology (DAPS) V2.0 3/20/09 Knowledge (PMBOK Guide®), 4th Edition Standard for Application and Capability Maturity Model Integration (CMMI®) Management of the SE Process (IEEE 1220) IEEE 1220: Processes for Engineering a System (ANSI/EIA–632) NASA EVM Guide NPG 9501.3 Rights Reserved 46/63
  47. 47. CPM–500 EVM/SE/TPM Integration Office of Management and Budget Circular No. A–11, Section 300  Planning, Budgeting, Acquisition and Management of Capital Assets  Section 300–5 – Performance–based acquisition management – Based on EVMS standard – Measure progress towards milestones • Cost • Capability to meet specified requirements • Timeliness • Quality Rights Reserved 47/63
  48. 48. CPM–500 EVM/SE/TPM Integration Need: Accurate Performance Measurement Findings and GAO Report 06–250 Recommendations Information Technology: 2. If EVM is not Improve the Accuracy and implemented effectively, Reliability of Investment decisions based on Information inaccurate and potentially misleading information 3. Agencies not measuring actual versus expected performance in meeting IT performance goals. Rights Reserved 48/63
  49. 49. CPM–500 EVM/SE/TPM Integration DOD Guides: Technical Performance Department of Defense Guidelines for Technical Performance Measures DoDI 5000.02, Operation of the Defense Acquisition System (POL) 12/08 Interim Defense Acquisition Guidebook (DAG) 6/15/09 Systems Engineering Plan (SEP) Preparation Guide 4/08 WBS Handbook, Mil–HDBK–881A (WBS) 7/30/05 Integrated Master Plan (IMP) & Integrated Master Schedule Preparation & Use Guide (IMS) 10/21/05 Guide for Integrating SE into DOD Acquisition Contracts (Integ SE) 12/06 Defense Acquisition Program Support Methodology (DAPS) V2.0 3/20/09 Rights Reserved 49/63
  50. 50. CPM–500 EVM/SE/TPM Integration DoD: TPMs in Technical Baselines and Reviews Engineering Integrated IMP/IMS Systems DAPS WBS DAG POL SEP DoD Policy or Guide Technical Baselines: IMP/IMS Functional (SFR)    Allocated (PDR) Product (CDR) Event driven timing        Success criteria of technical review        Entry and exit criteria for technical reviews     Assess technical maturity  Reserved Rights    50/63
  51. 51. CPM–500 EVM/SE/TPM Integration DoD: TPMs in Integrated Plans Engineering Integrated IMP/IMS Systems DAPS WBS DAG POL SEP DoD Policy or Guide Integrated SEP with: IMP/IMS TPMs      EVM Integrated WBS with Requirement Specification      Statement of Work IMP/IMS/EVMS Link risk management, technical reviews, TPMs,    EVM, WBS, IMS Rights Reserved 51/63
  52. 52. CPM–500 EVM/SE/TPM Integration Guidance in Standards, Models, and Defense Acquisition Guide  Processes for Engineering a System (ANSI/EIA–632)  Standard for Application and Management of the SE Process (IEEE 1220)  Capability Maturity Model Integration (CMMI®) – CMMI for Development, Version 1.2 – CMMI for Acquisition, Version 1.2 – Using CMMI to Improve Earned Value Management, 2002  Guide to the Project Management Institute Body of Knowledge (PMBOK Guide®), 4th Edition Rights Reserved 52/63
  53. 53. CPM–500 EVM/SE/TPM Integration Technical Performance Measures (TPM) More Sources IEEE 1220:, EIA–632: Glossary CMMI for Development Performance–based Requirements progress measurement Development TPMs are key to Predict future value of key Specific Practice (SP) 3.3, progressively assess technical parameters of Analyze Requirements technical progress the end system based on Typical work product: current assessments TPMs Establish dates for Planned value profile is Subpractice: – Checking progress time–phased achievement Identify TPMs that will be – Meeting full projected tracked during conformance to • Achievement to date development requirements • Technical milestone where TPM evaluation is reported Rights Reserved 53/63
  54. 54. CPM–500 EVM/SE/TPM Integration PMBOK® Guide  Project Management Plan  Performance Measurement Baseline: – Typically integrates scope, schedule, and cost parameters of a project – May also include technical and quality parameters Rights Reserved 54/63
  55. 55. CPM–500 EVM/SE/TPM Integration PMBOK® Guide  Work Performance Measurements  Used to produce project activity metrics  Evaluate actual progress as compared to planned progress  Include, but are not limited to: – Planned vs. actual technical performance – Planned vs. actual schedule performance, and – Planned vs. actual cost performance. Rights Reserved 55/63
  56. 56. CPM–500 EVM/SE/TPM Integration TPMs in DAG and DAPS Defense Acquisition Guide  Performance measurement of WBS elements, using objective measures: – Essential for EVM and Technical Assessment activities  Use TPMs and Critical Technical Parameters (CTP) to report progress in achieving milestones DAPS  Use TPMs to determine whether % completion metrics accurately reflect quantitative technical progress and quality toward meeting Key Performance Parameters (KPP) and Critical Technical Parameters Rights Reserved 56/63
  57. 57. CPM–500 EVM/SE/TPM Integration TPMs in DAG  Compare the actual versus planned technical development and design  Report progress in the degree to which system performance requirements are met.  Plan is defined in terms of: – Expected performance at specific points • Defined in the WBS and IMS – Methods of measurement at those points – Variation limits for corrective action. Rights Reserved 57/63
  58. 58. CPM–500 EVM/SE/TPM Integration PMBOK® Guide  Technical Performance Measurement  Compares technical accomplishments… to … project management plan’s schedule of technical achievement  Requires definition of objective quantifiable measures of technical performance which can be used to compare actual results against targets.  Might include weight, transaction times, number of delivered defects, storage capacity etc.  Deviation, such as demonstrating more or less functionality than planned at a milestone…forecast degree of success in achieving the project’s scope. Rights Reserved 58/63
  59. 59. CPM–500 EVM/SE/TPM Integration CMMI–ACQ  Acquisition Technical Management  SP 1.3 Conduct Technical Reviews  Typical supplier deliverables  Progress reports and process, product, and service level measurements  TPMs Rights Reserved 59/63
  60. 60. CPM–500 EVM/SE/TPM Integration SMS Shall: Monitor Progress Against the Plan  Monitoring  Contractor SHALL monitor progress against plan to validate, approve, and maintain each baseline and functional architecture  Required Product Attributes  Each documented assessment includes:  TPMs, metrics  Metrics and technical parameters for tracking that are critical indicators of technical progress and achievement Rights Reserved 60/63
  61. 61. CPM–500 EVM/SE/TPM Integration NASA EVM Guide: Technical Performance • NASA EVM Guide NPG 9501.3 – 4.5 Technical Performance Requirements (TPR): When TPRs are used, – appropriate and relevant metrics… – must be defined in the solicitation – Appendix A.7, 14.1 TPR • Compares: • Expected performance and • Physical characteristics • With contractually specified values. • Basis for reporting established milestones • Progress toward meeting technical requirements Rights Reserved 61/63
  62. 62. CPM–500 EVM/SE/TPM Integration Derivation and Flow Down of TPMs Document, Baseline, IMS, EVM Parameter  IMP, Functional Baseline Measures Of Effectiveness (MOE)   IMP, WBS, Functional Baseline Measures Of Performance (MOP)   IMP, Allocated Baseline Technical Performance Measure  TPM Milestones And Planned  IMS  Values  Work Packages TPM% Complete Criteria  See next chart for linkage of technical baselines to technical reviews Rights Reserved 62/63
  63. 63. CPM–500 EVM/SE/TPM Integration Interesting Attributes of TPMs  Achieved to Date (sounds like EV)  Current Estimate (sounds like EAC/ETC)  Milestone  Planned (target) value (sounds like PV)  Planned performance profile (sounds like a PMB)  Tolerance band (sounds like reporting thresholds)  Threshold (yep, just what we thought)  Variance (sounds like variance!) Rights Reserved 63/63