Cpm 200 c technical performance measures - alleman (ppm)


Published on

Techncial Performance Measures are the core of measuring what "done" looks like.

Published in: Technology, Business
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Cpm 200 c technical performance measures - alleman (ppm)

  1. 1. CPM-200C: Technical Performance MeasuresGlen B. AllemanPrimePMglen.alleman@primepm.net 303-241-9633Date: May 2013IPMC 20121
  2. 2. TLO #1: The student will recognize what a Technical Performance Measurement (TPM) is inthe context of a program’s Performance Measurement Baseline and DOD Guidance.ELO #1.1: The student will be able to summarize the policy requirements for TechnicalPerformance Measures in government documentsELO #1.2: The student will be able to identify the activities during the Integrated BaselineReview (IBR) to confirm the TPMs are defined as assigned to WBS elements andidentified and assigned to work in the IMS.TLO #2: The student will be able identify and select TPMs using the example WorkBreakdown Structure from the program.ELO #2.1: The student will construct a set of TPMs from the provided WBSELO #2.2: With these constructed TPMs, the student will apply them to the WBS elementsof the notional program.TLO #3: The student will learn how to prepare descriptions of TPM needed to assess thephysical “progress to plan” of the program.ELO #3.1: The student will be able to define the upper and lower bounds for a TPM usingthe WBS and Statement of Work for the notional programTLO #4: The student will able to judge the validity of a TPM from a list of TPM’s from anotional program.ELO #4.1: The student will set up a control limit chart to track Technical PerformanceMeasure metrics for the assigned deliverable.Learning Objectives2
  3. 3. Components we’ll meet along the way toour destination to the credible PMBObjective Status and Essential Views to support the proactive management processes neededto keep the program GREENRisk ManagementSOWSOOConOpsWBSTechncial and OperationalRequirementsCWBS &CWBS DictionaryIntegrated Master Plan(IMP)Integrated Master Schedule(IMS)Measures ofEffectivenessMeasures ofPerformanceMeasures ofProgressJROCKey Performance ParametersProgram SpecificKey Performance ParametersTechnical PerformanceMeasuresEarned Value ManagementSystemTPMs LiveHerePerformance Measurement BaselineTLO #13
  4. 4. Components of our Final DestinationSowSOOConOpsWBSTechncial and OperationalRequirementsPerformance Measurement BaselineCWBS &CWBS DictionaryIntegrated Master Plan(IMP)Integrated Master Schedule(IMS)Earned Value ManagementSystemTechnical Performance Measurement (TPM) involves the predicting the future values of akey technical performance parameter of the higher-level end product underdevelopment, based on current assessments of products lower in the system structure.Continuous verification of actual versus anticipated achievement for selected technicalparameters confirms progress and identifies variances that might jeopardize meeting ahigher-level end product requirement. Assessed values falling outside establishedtolerances indicate the need for management attention and corrective action.A well thought out TPM program provides early warning of technical problems, supportsassessments of the extent to which operational requirements will be met, and assessesthe impacts of proposed changes made to lower-level elements in the system hierarchyon system performance.Techncial PerformanceMeasurementTLO #14https://dap.dau.mil/acquipedia/Pages/Default.aspx
  5. 5. Can Earned Value Alone Get Us ToOur Destination?• How do we increase visibility into program performance?• How do we reduce cycle time to deliver the product?• How do we foster accountability?• How do we reduce risk?• How do we start our journey to success?Increasing the Probability of Success means we have toConnect The Dots Between EVM and TPM to Reach Our Destination 5TLO #1: EOL #1.1
  6. 6. Identifying the TPMs starts withgood Systems Engineering7TLO #1: EOL #1.1
  7. 7. Some More GuidanceSystems engineering uses technical performancemeasurements to balance cost, schedule, and performancethroughout the life cycle. Technical performancemeasurements compare actual versus planned technicaldevelopment and design. They also report the degree towhich system requirements are met in terms of performance,cost, schedule, and progress in implementing risk handling.Performance metrics are traceable to user–definedcapabilities.― Defense Acquisition Guide(https://dag.dau.mil/Pages/Default.aspx)In The End ― It’s All About Systems Engineering10TLO #1: EOL #1.1
  8. 8. A Final Reminder …13TPMs are one source of Risk Management processesRisk Management Guide to DOD Acquisition, Sixth Edition (Version 1,0), Aug 2006TLO #1: EOL #1.1
  9. 9. One Secret ofProgramSuccess• Cost and schedulecompliance isnecessary but far fromsufficient for success.• TPM’s are the start ofsufficiency.• MoE’s and MoP’s areneeded as well.14http://www.happiness-project.com/happiness_project/2013/04/secret-of-adulthood-you-manage-what-you-measure/
  10. 10. Where can we find IBRTPM Guidance?• In the obvious place – the NDIAIntegrated Baseline Review Guide16TLO #1: EOL #1.2
  11. 11. Back To Our TechnicalPerformance MeasuresTechnical Performance Measures do whatthey say,Measure the Technical Performanceof the product or service produced by theprogram.TLO #217
  12. 12. The Primary Role of TMPs is to describe what donelooks like in units meaningful to the decision maker19 October 1899 Robert Goddard decided that he wanted to "fly without wings" to Moon.TLO #218
  13. 13. From Mission to TPM“Coming to Grips with Measures ofEffectiveness,” N. Sproles, SystemsEngineering, Volume 3, Number 1, pp. 50–58MoEKPPMoP TPMMissionNeedAcquirer Defines the Needs and Capabilitiesin terms of Operational ScenariosSupplier Defines Physical Solutions thatmeet the needs of the StakeholdersOperationalmeasures of successrelated to theachievement of themission oroperationalobjective beingevaluated.Measures thatcharacterize physicalor functionalattributes relatingto the systemoperation.Measures used toassess designprogress,compliance toperformancerequirements, andtechnical risks.TLO #224
  14. 14. One more reminder25IPMC November 2012, Gordon Kranz, PARCATLO #2: ELO #2.1
  15. 15. • Each “end item” in theWBS needs a TPM toconfirm the workproduced the plannedvalue.• These measures canbe found in the WBSDictionary, the “exitcriteria” for the WorkPackages and otherengineeringdocuments27
  16. 16.  This is where TPMsare connected withthe MoE’s andMoP’s For eachdeliverable fromthe program, allthe “measures”must be defined inunits meaningfulto the decisionmakers. Here’s some “real”examples.Connecting the MoE, MoP, KPP, and TPMs1. Provide PrecisionApproach for a 200FT/0.5 NM DH2. Provide bearing andrange to AC platform3. Provide ACsurveillance to GRNDplatformMeasures ofEffectiveness (MoE)1. Net Ready2. Guidance Quality3. Land Interoperability4. Manpower5. AvailabilityJROC Key PerformanceParameters (KPP)1. Net Ready IPv4/6 compliance 1Gb Ethernet2. Guidance quality Accuracy threshold p70@ 6M Integrity threshold 4M@ 10-6 /approach3. Land interoperability Processing capabilitymeets LB growth matrix4. Manpower MTBC >1000 hrs MCM < 2 hrs5. Availability Clear threshold >99% Jam threshold >90%Measures of Performance(MoP)1. Net Ready Standard message packets2. Guidance Quality Multipath allocationbudget Multipath bias protection3. Land Interoperability MOSA compliant Civil compliant4. Manpower Operating elapsed timemeters Standby elapsed timeindicators5. Availability Phase center variationsTechnical PerformanceMeasures (TPM)Mission Capabilities and Operational NeedTechnical Insight – Risk adjusted performance to plan28TLO #2: ELO #2.1
  17. 17. IDENTIFYING TECHNCIALPERFORMANCE MEASURESTechnical performance measures must be techncial and must describethe performance in units of measure meaningful to the decision makers.TPM’s are not counts of things delivered. TPMs are not assessments ofcompleted work through effort or cost.TPMs are measures of Technical PerformanceTLO #329
  18. 18. Measuring Cost and Schedule butNOT measuring Compliance withTechnical Performance SpecificationsTLO #330
  19. 19. Measures of Technical ParametersINCOSE Systems Engineering HandbookAttribute DescriptionAchieved to Date Measured technical progress or estimate ofprogress in units of performanceCurrent Estimate Value of a technical parameter that is predictedto be achieved now or in the futureMilestone Point in time when an evaluation of thetechnical measure is accomplishedPlanned Value  Predicted value of the technical parameterPlanned PerformanceProfile Profile representing the project time phaseddemonstration of a technical parameterTolerance Band  Management alert limitsThreshold Limiting acceptable value of a technicalparameterVariances Demonstrated technical variance Predicted technical varianceTLO #3: ELO # 3.131
  20. 20. A Familiar Graphic for TPMsVariancePlanned ValuePlanned ProfileCurrent EstimateMilestonesThresholdUpper LimitLower LimitMeanToBetweenFailureTime = Program MaturityAchieved to DateTPM32TLO #3: ELO # 3.1
  21. 21. TPMs from an Actual ProgramJames Webb Space Telescope34TLO #3: ELO # 3.1
  22. 22. 1.1 Air Vehicle1.1.1 Sensor Platform1.1.2 Airframe1.1.3 Propulsion1.1.4 On Board Comm1.1.5 Auxiliary Equipment1.1.6 SurvivabilityModules1.1.7 Electronic WarfareModule1.1.8 On BoardApplication &System SW1.3 Mission Control /Ground Station SW1.3.1 Signal ProcessingSW1.3.2 Station Display1.3.3 Operating System1.3.4 ROE Simulations1.3.5 Mission CommandsTPMs Start With The WBSThe WBS for a UAV1.1.2 Airframe36TLO #3: ELO # 3.1
  23. 23. Let’s Connect The DotsRiskSOWCostWBSIMP/IMSTPMPMBNamedDeliverablesdefined in the WBSBCWS at the WorkPackage, rolled to theControl AccountTPMs attached to eachcritical deliverables in theWBS and identified ineach Work Package in theIMS, used to assessmaturity in the IMPThe Products andProcesses that producethem in a “well structured”decomposition in the WBSIMS contains allthe WorkPackages, BCWS,Risk mitigationplans, and rolls tothe IntegratedMaster Plan tomeasureincreasing maturityTechnical and ProgrammaticRisks Connected to the WBSand IMS40TLO #3: ELO # 3.1
  24. 24. TPM Trends & ResponsesDr. Falk Chart – modifiedEV Taken, planned values met, tolerances kept, etc.25kg23kg28kg26kgPDRSRRSFRCA TRRCDRROM in ProposalDesign ModelBench Scale Model MeasurementDetailed Design ModelPrototype MeasurementFlight 1st ArticleTechnicalPerformanceMeasureVehicleWeight41TLO #3: ELO # 3.1
  25. 25. 1.1 Air Vehicle1.1.1 Sensor Platform1.1.2 AirframeAirframe Weight TPMThe WBS for a UAV1.1.2 AirframeCA SFR SRR PDR CDR TRRPlanned Value 28.0kg 27.0kg 26.0kg 25.0kg 24.0kg 23.0kgActual Value 30.4kg 29.0kg 27.5kg 25.5kgAssessed Riskto TRRModerate>2.0kg offtargetLow1–2 kg offtargetLow1–2 kg offtargetVery Low (lessthan 1.0 kgoff target)PlannedMethod“Similar to”EstimateROMProgram–unique designmodelProgram–unique designmodel withvalidated dataActualmeasurementof bench–testcomponentsActualmeasurementof prototypeairframeActualMethod“Similar to”EstimateROM ROM ROMThe planned weight is25kg. The actual weight is25.5kg.Close to plan! So we aredoing okay, right?Here’s the Problem43TLO #3: ELO # 3.1
  26. 26. Raison detre for TechnicalPerformance MeasuresThe real purpose ofTechnical PerformanceMeasures is to reduceProgrammatic andTechnical RISKRiskSOWCostWBSIMP/IMSTPMPMBTLO #444
  27. 27. Can TPM’s be our Leading Indicator?TLO #445
  28. 28. Key Elements of Good TPMs• Traceability – Requirements to WBS to TPMs to EVcontrol accounts.• Impact – How much WBS work, & therefore EVmoney, is covered by the TPM(s)? What is effect?• TPM Banding/Sensitivity – What banding (R/Y/G)and sensitivity (EV impact) should be used for eachTPM?• Technical Readiness Level – What’s the state of thetechnology supporting the requirement(s) for whichTPM is a metric?Implementing Technical Performance Measurement , Mike Ferraro , General Engineer, PEO/SYSCOM Conference46TLO #4: ELO # 4.1
  29. 29. Forecasting the Future withstatistical analysis of the Past48TLO #4: ELO # 4.1
  30. 30. Connecting the EV VariablesIntegrating Cost, Schedulele, and Technical PerformanceAssures Program Management has the needed performance information to deliveron‒time, on‒budget, and on‒specificationTechnical Performance MeasuresCost ScheduleConventional Earned Value+= Master Schedule is used toderive Basis of Estimate(BOE) not the other wayaround. Probabilistic costestimating uses pastperformance and cost riskmodeling. Labor, Materiel, and otherdirect costs accounted forin Work Packages. Risk adjustments for allelements of cost.Cost Baseline Earned Value is diluted bymissing technicalperformance. Earned Value is diluted bypostponed features. Earned Value is diluted bynon compliant quality. All these dilutions requireadjustments to theEstimate at Complete(EAC) and the To CompletePerformance Index (TCPI).Technical Performance Requirements aredecomposed into physicaldeliverables. Deliverables are producedthrough Work Packages. Work Packages areassigned to accountablemanager. Work Packages aresequenced to form thehighest value stream withthe lowest technical andprogrammatic risk.Schedule Baseline49TLO #4: ELO # 4.1
  31. 31. 50
  32. 32. TPM ChecklistMoE MoP TPMTraceable to needs,goals, objectives, andrisks.Traceable to applicableMOEs, KPPs, system levelperformance requirements,and risks.Traceable to applicable MoPs,system element performance,requirements, objectives,risks, and WBS elements.Defined with associatedKPPs.Focused on technical risksand supports tradesbetween alternativesolutions.Further decomposed,budgeted, and allocated tolower level system elements inthe WBS and IMS.Each MoE independentfrom others.Provided insight intosystem performance.Assigned an owner, the CAMand Work Package Manager.Each MoE independentof technical anysolution.Decomposed, budgetedand allocated to systemelements.Sources of measure identifiedand processes for generatingthe measures defined.Address the requiredKPPs.Assigned an “owner,” theCAM and TechnicalManager.Integrated into the program’sIMS as part of the exit criteriafor the Work Package. 51TLO #4: ELO # 4.1
  33. 33. Summary of TPM Benefits• Measures of results based on techncial achievement• Integrated technical performance with budget andschedule – providing an integrated systemsapproach to performance measurement• Establishes the linkage between engineering andcost/schedule considerations in programmanagement decision making• Measures the programmatic impact of techncialvariance in terms of risk• Reduces subjectivity through the establishment oftechnical performance baseline53TLO #4: ELO # 4.1
  34. 34. 54
  35. 35. 55
  36. 36. 56Glen B. Allemanglen.alleman@primepm.net+1 303 241 9633
  37. 37. DOD Guides:Technical PerformanceDepartment of Defense Guidelines for Technical Performance MeasuresDoDI 5000.02, Operation of the Defense Acquisition System (POL) 12/08Interim Defense Acquisition Guidebook (DAG) 6/15/09Systems Engineering Plan (SEP) Preparation Guide 4/08WBS Handbook, Mil–HDBK–881A (WBS) 7/30/05Integrated Master Plan (IMP) & Integrated Master Schedule Preparation &Use Guide (IMS) 10/21/05Guide for Integrating SE into DOD Acquisition Contracts (Integ SE) 12/06Defense Acquisition Program Support Methodology (DAPS) V2.0 3/20/0957TLO #4: ELO # 4.1
  38. 38. TPMs in DAG and DAPSDefense Acquisition Guide• Performance measurement of WBS elements, using objectivemeasures:– Essential for EVM and Technical Assessment activities• Use TPMs and Critical Technical Parameters (CTP) to reportprogress in achieving milestonesDAPS• Use TPMs to determine whether % completion metricsaccurately reflect quantitative technical progress and qualitytoward meeting Key Performance Parameters (KPP) and CriticalTechnical Parameters58
  39. 39. TPMs in DAG• Compare the actual versus planned technicaldevelopment and design• Report progress in the degree to which systemperformance requirements are met.• Plan is defined in terms of:– Expected performance at specific points• Defined in the WBS and IMS– Methods of measurement at those points– Variation limits for corrective action.59
  40. 40. NASA EVM Guide:Technical Performance• NASA EVM Guide NPG 9501.3– 4.5 Technical Performance Requirements (TPR): When TPRsare used,– appropriate and relevant metrics…– must be defined in the solicitation– Appendix A.7, 14.1 TPR• Compares:• Expected performance and• Physical characteristics• With contractually specified values.• Basis for reporting established milestones• Progress toward meeting technical requirements60
  41. 41. Document, Baseline,IMS, EVMParameter IMP, Functional Baseline Measures Of Effectiveness (MOE)  IMP, WBS, Functional Baseline Measures Of Performance (MOP)  IMS, Allocated Baseline Technical Performance Measure  IMS TPM Milestones And Planned Values  Work Packages TPM% Complete Criteria Derivation and FlowDown of TPMs61
  42. 42. 62Glen B. Allemanglen.alleman@primepm.net+1 303 241 9633