Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Cpm 200 C technical performance measures ipm2016

5,140 views

Published on

Techncial Performance Measures inform Physical Percent Complete which in turn informs Earned Value Management

Published in: Technology
  • Be the first to comment

Cpm 200 C technical performance measures ipm2016

  1. 1. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page IPM 2016 CPM-200: Scope and Organization Management Lesson C: Using Technical Performance Measures to Inform Earned Value Performance Glen B. Alleman Thomas J. Coonce Niwot Ridge, LLC Institute for Defense Analyses glen.alleman@niwotridge.com tcoonce@ida.org +1 303 241 9633 +1 703 362 2568
  2. 2. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Learning Objectives • TLO #1: The student will recognize what a Technical Performance Measure (TPM) is in the context of the Professional Education Program Case Study as well as DOD and INCOSE Systems Engineering Guidance. • TLO #2: The student will be able identify and select TPMs using the example Work Breakdown Structure from the Case Study. • TLO #3: The student will learn how to prepare descriptions of TPMs needed to assess the physical “progress to plan” of the program. • TLO #4: The student will able to judge the validity of a TPM from a list of TPM’s from the notional program. 2
  3. 3. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Components we’ll meet along the way to a credible PMB Objective Status and Essential Views to support the proactive management processes needed to keep the program GREEN Risk Management SOW SOO ConOps WBS Technical and Operational Requirements CWBS & CWBS Dictionary Integrated Master Plan (IMP) Integrated Master Schedule (IMS) Measures of Effectiveness Measures of Performance Measures of Progress Key Performance Parameters Program Specific Key Performance Parameters Technical Performance Measures Earned Value Management System TPMs Live Here Performance Measurement Baseline TLO #1 3
  4. 4. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Can Earned Value Alone Get Us To Our Destination? • How do we increase visibility into program performance? • How do we reduce cycle time to deliver the product? • How do we foster accountability? • How do we reduce risk? • How do we start our journey to success? Increasing the Probability of Success means … Connecting The Dots Between EVM and TPM to Reach Our Destination TLO #1 4
  5. 5. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Increasing the Probability of Program Success Means … Risk SOW Cost WBS IMP/IMS TPM PMB Building A Credible Performance Measurement Baseline This is actually harder than it looks! TLO #1 5
  6. 6. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Identifying the TPMs starts with good Systems Engineering TLO #1 6
  7. 7. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Some More Guidance Systems engineering uses technical performance measurements to balance cost, schedule, and performance throughout the life cycle. Technical performance measurements compare actual versus planned technical development and design. They also report the degree to which system requirements are met in terms of performance, cost, schedule, and progress in implementing risk handling. Performance metrics are traceable to user–defined capabilities. ― Defense Acquisition Guide (https://dag.dau.mil/Pages/Default.aspx) In The End ― It’s Always About Systems Engineering TLO #1 7
  8. 8. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page This Has All Been Said Before … the basic tenets of the process are the need for seamless management tools, that support an integrated approach … and “proactive identification and management of risk” for critical cost, schedule, and technical performance parameters. ― Secretary of Defense, Perry memo, May 1995 Why Is This Hard To Understand? § We seem to be focused on EV reporting, not the use of EV to manage the program. § We think getting the IPMR out the door is the end of Program Planning and Control’s efforts, not the beginning. TPM Handbook 1984 TLO #1 8
  9. 9. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Back To Our Technical Performance Measures Technical Performance Measures do what they say, Measure the Technical Performance of the product or service produced by the program. TLO #2 9
  10. 10. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page To develop TPMs we first need to define some other measures • Measures of Effectiveness (MoE) • Measures of Performance (MoP) • Key Performance Parameters (KPP) • Then we can define the Technical Performance Measures (TPM) 10
  11. 11. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Measure of Effectiveness (MoE) Measures of Effectiveness … • Are stated in units meaningful to the buyer, • Focus on capabilities independent of any technical implementation, • Are connected to the mission success. Operational measures of success that are closely related to the achievements of the mission or operational objectives evaluated in the operational environment, under a specific set of conditions. MoE’s Belong to the End User TLO #2 “Technical Measurement,” INCOSE–TP–2003–020–01 11
  12. 12. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Measure of Performance (MoP) Measures of Performance are … • Attributes that assure the system has the capability and capacity to perform, • Assessment of the system to assure it meets design requirements to satisfy the MoE. Measures that characterize physical or functional attributes relating to the system operation, measured or estimated under specific conditions. “Technical Measurement,” INCOSE–TP–2003–020–01 MoP’s belong to the Program – Developed by the Systems Engineer, Measured By CAMs, and Analyzed by PP&C TLO #2 12
  13. 13. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Key Performance Parameters (KPP) Key Performance Parameters … • Have a threshold or objective value, • Characterize the major drivers of performance, • Are considered Critical to Customer (CTC). Measures that Represent the capabilities and characteristics so significant that failure to meet them can be cause for reevaluation, reassessing, or termination of the program The acquirer defines the KPPs during the operational concept development – KPPs say what DONE looks like TLO #2 “Technical Measurement,” INCOSE–TP–2003–020–01 13
  14. 14. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Technical Performance Measures (TPM) Technical Performance Measures … • Assess design progress, • Define compliance to performance requirements, • Identify technical risk, • Are limited to critical thresholds, • Include projected performance. Attributes that determine how well a system or system element is satisfying or expected to satisfy a technical requirement or goal TLO #2 “Technical Measurement,” INCOSE–TP–2003–020–01 14
  15. 15. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page TPMs start with the WBS TLO #2 1.0 Safer Counties / Safer Cities 1.1 SC2 Prime Mission Equipment 1.1.1. Camera Systems 1.1.1.1 Camera 1.1.1.2 Camera Interface Hardware 1.1.1.3 Camera Integration 1.1.2 Wireless Tower System 1.1.2.1 Wireless Tower 1.1.2.2 Tower Interface Tower 1.1.2.3 Tower Integration, Test, Assembly, and Checkout 1.1.3 Monitoring Stations 1.1.3.1 PC Stations 1.1.3.2 PC Interface Hardware 1.1.3.3 Monitoring Stations Integration, Test, Assembly and Checkout 1.2 System Engineering /Program Management 1.3 System Test and Evaluation 1.4 Data 1.5 Training Let’s focus on the wireless towers and the services they provide to public safety, as well as convenience for the citizens. We’ll need MOPs, MOEs, KPPs, before we define the TPMs Then we’ll connect these TPM’s to assessing the performance of the work in the IMS. 15
  16. 16. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page § This is where TPMs are connected with the MoE’s and MoP’s § For each deliverable from the program, all the “measures” must be defined in units meaningful to the decision makers. § Here’s some “real” examples. Connecting the MoE, MoP, KPP, and TPMs 1. Provide bandwidth for 5,000 simultaneous voice and data users 2. Provide location service for 911 3. Provide 99.5% availability to users Measures of Effectiveness (MoE) 1. Net Ready 2. E911 Location Quality 3. Phone Interoperability 4. Manpower 5. Availability Key Performance Parameters (KPP) 1. Net Ready § IPv4/6 compliance § 1Gb Ethernet 2. Location quality § Accuracy threshold @60M § Integrity threshold @40M 3. Phone interoperability § Processing capability meets growth matrix 4. Manpower § MTBC >1000 hrs § MCM < 2 hrs 5. Availability § Clear threshold >99% § Noise threshold >90% Measures of Performance (MoP) 1. Net Ready § Standard message packets 2. Location Quality § Multipath allocation budget § Multipath bias protection 3. Phone Interoperability § MOSA compliant § Civil compliant 4. Manpower § Operating elapsed time meters § Standby elapsed time indicators 5. Availability § Phase center variations Technical Performance Measures (TPM) Mission Capabilities and Operational Need Technical Insight – Risk adjusted performance to plan TLO #2 16
  17. 17. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Identifying Technical Performance Measures Technical Performance Measures must be technical and must describe the performance in units of measure meaningful to the decision makers. TPM’s are not counts of things delivered. TPMs are not assessments of completed work through effort or cost. TPMs are measures of Technical Performance TLO #3 17
  18. 18. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page How can we possibly know we’re on schedule or on budget without knowing if we’re technically compliant? TLO #3 18
  19. 19. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Measures of Technical Parameters† Attribute Description Achieved to Date § Measured technical progress or estimate of progress in units of performance Current Estimate § Value of a technical parameter that is predicted to be achieved now or in the future Milestone § Point in time when an evaluation of the technical measure is accomplished Planned Value § Predicted value of the technical parameter Planned Performance Profile § Profile representing the project time phased demonstration of a technical parameter Tolerance Band § Management alert limits Threshold § Limiting acceptable value of a technical parameter Variances § Demonstrated technical variance § Predicted technical variance TLO #3 † INCOSE Systems Engineering Handbook 19
  20. 20. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page A Familiar Graphic for TPMs Variance Planned Value Planned Profile Current Estimate Milestones Threshold Upper Control Limit Lower Control Limit Mean To Between Failure Time = Program Maturity Achieved to Date TPM TLO #3 20
  21. 21. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page What Do We Need To Know About This Program Through The TPMs? • What WBS elements represent the TPMs? • What Work Packages produce these WBS elements? • Where do these Work Packages live in the IMS? • What are the Earned Value baseline values for these Work Packages? • How are we going to measure all these variables? • What does the curve look like for these measurements? TLO #3 21
  22. 22. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Let’s Connect Some More Dots TLO #3 Risk SOW Cost WBS IMP/IMS TPM PMB ❺ Deliverables defined in the SOW, traced to the WBS, with narratives and Measures of Performance (MoP) ❹ Budget at the Work Package level, rolled to the Control Accounts showing cost spreads for all work in the IMS ❻ Measures of Performance (MOP) for each critical deliverable in the WBS and identified in each Work Package in the IMS, used to assess maturity in the IMP ❶ The Products and Processes in a “well structured” decomposition, traceable to the deliverables ❷ IMS contains all the Work Packages, BCWS, Risk mitigation plans, with traces to the IMP measuring increasing maturity through Measures of Effectiveness (MOE) and KPPs ❸ Technical and Programmatic Risks Connected through the WBS, Risk Register, IMP and IMS The PMB is the Document of Record for the Program Performance is Measured through the PMB 22
  23. 23. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Raison d'etre for Technical Performance Measures The real purpose of Technical Performance Measures is to reduce Programmatic and Technical RISK Risk SOW Cost WBS IMP/IMS TPM PMB TLO #4 23
  24. 24. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Key Elements of A Good TPM • Traceability – Requirements to WBS to TPMs to EV control accounts. • Impact – How much WBS work, & therefore EV money, is covered by the TPM(s)? What is effect? • TPM Banding/Sensitivity – What banding (R/Y/G) and sensitivity (EV impact) should be used for each TPM? • Technical Readiness Level – What’s the state of the technology supporting the requirement(s) for which TPM is a metric? Implementing Technical Performance Measurement , Mike Ferraro , General Engineer, PEO/SYSCOM Conference TLO #4 24
  25. 25. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Determination of Physical % Complete must be informed by both task completion and technical status 4Progress of a set of tasks – Drawings Completed – Lines of Code Written – Work Products Produced – Reviews Completed • Progress of the technical status – Critical TPM Achievement – System Capabilities Met – Quality of Work Products – System Under Review Acceptable Measuring progress of what people do Measuring progress of the Result of what people do Progress is measured by effectiveness of outcomes to the end user Physical % Complete P%C TLO #4 BCWPi i=1 N ∑ = BCWSi × P%Ci i=1 N ∑ 25
  26. 26. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Connecting the EV Variables Integrating Cost, Schedulele, and Technical Performance Assures Program Management has the needed performance information to deliver on‒time, on‒budget, and on‒specification Technical Performance Measures Cost Schedule Conventional Earned Value + = § Master Schedule is used to derive Basis of Estimate (BOE) not the other way around. § Probabilistic cost estimating uses past performance and cost risk modeling. § Labor, Materiel, and other direct costs accounted for in Work Packages. § Risk adjustments for all elements of cost. Cost Baseline § Earned Value is diluted by missing technical performance. § Earned Value is diluted by postponed features. § Earned Value is diluted by non compliant quality. § All these dilutions require adjustments to the Estimate at Complete (EAC) and the To Complete Performance Index (TCPI). Technical Performance § Requirements are decomposed into physical deliverables. § Deliverables are produced through Work Packages. § Work Packages are assigned to accountable manager. § Work Packages are sequenced to form the highest value stream with the lowest technical and programmatic risk. Schedule Baseline TLO #4 26
  27. 27. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page TPM Checklist MoE MoP TPM Traceable to needs, goals, objectives, and risks. Traceable to applicable MOEs, KPPs, system level performance requirements, and risks. Traceable to applicable MoPs, system element performance, requirements, objectives, risks, and WBS elements. Defined with associated KPPs. Focused on technical risks and supports trades between alternative solutions. Further decomposed, budgeted, and allocated to lower level system elements in the WBS and IMS. Each MoE independent from others. Provided insight into system performance. Assigned an owner, the CAM and Work Package Manager. Each MoE independent of technical any solution. Decomposed, budgeted and allocated to system elements. Sources of measure identified and processes for generating the measures defined. Address the required KPPs. Assigned an “owner,” the CAM and Technical Manager. Integrated into the program’s IMS as part of the exit criteria for the Work Package. TLO #4 27
  28. 28. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Summary of TPM Benefits • Measures of results based on technical achievement • Integrated technical performance with budget and schedule – providing an integrated systems approach to performance measurement • Establishes the linkage between engineering and cost/schedule considerations in program management decision making • Measures the programmatic impact of technical variance in terms of risk • Reduces subjectivity through the establishment of technical performance baseline TLO #4 28
  29. 29. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Past Issues with Earned Value Management and Scheduling • Ability to take credit for all work accomplished • How to defend what work was earned • Identifying EV milestones in the schedule focused attention on those tasks so that maximum earned value could be taken • IMS was too detailed to manage the program • CAM’s used other means to manage and assess progress related to their work; i.e. detailed schedules • Detailed schedules did not reflect what was reported in IMS 29 Solution(– Quantifiable(Backup(Data((QBD)
  30. 30. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Quantifiable Backup Data (QBD) • A QBD is a supplement to the IMS to help manage discrete work at a lower level than is maintained in the IMS • The QBD measures the work objectively – 748-C calls out on Page 1 – Objectively assess accomplishments at the work performance level • QBD’s are part of the PMB • QBD’s do not replace the EVMS Earned Value Techniques (EVT) 30
  31. 31. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Quantifiable Backup Data (QBD) • A detailed listing of tasks necessary to complete all scope in a work package during the defined period of performance. • It is an approach used to objectively measure performance – Each task on the list is weighted – total weighting equals 100% of the work package BAC (weighting is important, should not be equal weighting on every task). – Upon completion of QBD development the QBD is placed under configuration control. – The CAM assesses physical percent complete of each QBD task. – The percent complete is calculated from the cumulative assessments. • The purpose of the QBD is to: – Ensure and demonstrate that all contract work is accounted for; – Ensure the schedule and budget are realistic and achievable; – Mitigate schedule and budget risks; and, – Provide a basis for objectively assessing progress for discretely measured work packages. 31
  32. 32. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page 32
  33. 33. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page CPM IPM 2016 Rights Reserved by the Organization on the Cover Page What We’ll Do In Our Hands On Work Shop Based on a notional case study for a county surveillance system, we will strategize about the system MOEs and MOPs and decide upon appropriate TPMs to inform our program performance and BCWP 33
  34. 34. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Case Study Set Up • The county is concerned about controlling disturbances, mitigating crime, and having forensic video and sound evidence for legal purposes • The county desires to procure a surveillance system with cameras and sound devices at strategic locations, send the data feeds via wireless technology, monitor, store and retrieve selected incidents 34
  35. 35. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Case Study Set Up (Continued) • The county wants near “real time” response and desires video quality such that they can distinguish facial features and read a license plates within 20 feet from the camera • It also wants the system to alert operators when there is evidence of a disturbance • It wants equipment that will allow them to store five years worth of data feeds collected 24/7 35
  36. 36. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Case Study Set Up (Concluded) • The county has submitted a Request for Proposal (RFP) to procure, install, test and train personnel. The county has set aside of budget of $40M and wants the system operational within 18 months. Because of the nature of the procurement, the county has required monthly IPMRs (See Handout 1 for details) • You are the proposal manager for the contract. Your immediate job is to – Determine the system’s “Measure of Effectiveness” – Select appropriate Technical Performance Measures to use to inform or drive reported cost and schedule progress – Propose how you will ensure the county that cost and schedule progress is consistent with measured technical progress 36
  37. 37. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Handout # 1 Safer County/Safer City Surveillance Background and Technical Requirements 37
  38. 38. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Exercise # 1 • What are possible Measures of Effectiveness (MOEs) of the System? • What are possible Measure of Performance (MOPs) of the System? • What are possible Technical Performance Measures (TPM) that could be used to drive or inform the cost and schedule progress? 38
  39. 39. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page All these measures used to inform BCWP Reminder: MOEs, MOPs, and TPMs are Connected to the IMP/IMS SOW ConOps WBS CWBS Technical Performance Measures Messages of Performance Measures of EffectivenessKPPs Integrated Master Plan Integrated Master Schedule Performance Measurement Baseline Technical and Operational Requirements 39
  40. 40. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Handout # 2 Safer County/Safer City Surveillance System School Solution 40
  41. 41. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Exercise # 2 • Discuss school solution 41
  42. 42. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Exercise # 3 • Which WBS elements & IMS Work Packages need to be informed by TPM progress? 42
  43. 43. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Handout # 3 Safer County/Safer City Surveillance System Work Breakdown Structure and Integrated Master Schedule 43
  44. 44. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Presentation of Workshop Results With our MOE’s, MOP’s, KPP’s, and TPM’s let’s assess out needed capabilities against the technical solution 44
  45. 45. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page Questions??? 45
  46. 46. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page OFFICIAL VERSION COLLEGE OF PERFORMANCE MANAGEMENT PROGRAM OF INSTRUCTION 500C VERSION 1.2 25 Mar 2015 Approvals: William Mathis, V.P of Education and Certification 25 Mar 2015 [Add appropriate approval individual name and title] Date [Add appropriate approval individual name and title] Date [Add appropriate approval individual name and title] Date 46
  47. 47. CPM IPM 2016 Rights Reserved by the Organization on the Cover Page RECORD OF CHANGES VERSION NUMBER DATE NUMBER OF FIGURE, TABLE OR PARAGRAPH A* M D TITLE OR BRIEF DESCRIPTION CHANGE REQUEST NUMBER 1 22 Feb 2014 A,M Format change for IPM 2014 1.1 18 Sep 2014 A,M Format change for IPMW 2014 1.2 25 Mar 2015 A,M Format changes Record of Changes 47

×