This document discusses selecting performance measures for process improvement. It explains that measures should link to critical customer requirements and business objectives. Input, process, and output indicators are identified by analyzing supplier inputs, internal processes, and delivered outputs. Relationships between indicators are mapped to understand how changing one affects others. The ideal measures are easy to collect, predict outputs, and drive appropriate actions. Teams should select a few key indicators at meaningful process points to monitor performance and identify improvement opportunities.

1. Measure Performance – 2.1 Presented by: Stephen Griffin, CEO Determine What to Measure

2. 2.1 Determine What to Measure Objective: To identify different types of measures and an understanding of how the measures relate to critical customer requirements. Key Topics: Performance Measurement. Input, Process, and Output Indicators. Indicator Relationships. 2.1 Determine What to Measure 2.2 Manage Measurement 2.3 Understand Variation 2.5 Managing the Measurement System 2.4 Determine Sigma Performance

3. Performance Measures: Achieving Customer Value Important decisions based on linking customer expectations to process performance Suppliers Process Inputs Business Processes Process Outputs Critical Customer Requirements Market Input Measures Process Measures Output Performance Measures Customer Value Good process decisions require comprehensive process data.

4. Process Elements and Indicator Relationships Suppliers : Inputs : Start Boundary ____________ Outputs : Customers : End Boundary ____________ Process

5. Input  Process  Output Indicators Input indicators Process indicators Output (performance) indicators Efficiency Measures Cost per transaction Time per activity Amount of rework Turnaround time Variability of an activity Effectiveness Measures Percent defective Number of errors Total response time Invoice/billing accuracy Revenue

6. Input  Process  Output Measurements

7. Process Output Indicators: CCRs and CBRs Output Indicators VOB ________ ________ CBRs ________ ________ Business Issues ________ ________ CCRs ________ ________ VOB - Voice of the Business CBR - Critical Business Requirements VOC - Voice of the Customer CCR - Critical Customer Requirements CCR’s ________ ________ ________ _________ ________ ________ ________ _________ VOC ________ ________ ________ _________ Customer Issues

8. EXAMPLES: CCR and C BR CCRs Output Indicators CBRs ________ ________ ________ ________ CCRs Price/Unit Delivery Time Dimensions Purity Reliability Color Service Level CBRs Cost/Unit Productivity Compliance with Regulations Changeover Time Safety Training Hours Critical to: The Business The Regulator The Employees Critical to: The Customer The Market

9. Success Derived from Project Focus Example: Medical Diagnostic Tube Life y 1 Oil Dielectric Quality GTD-3 y 2 Focal Spot Control GTD-7 y 3 Generators Spits GTD-17 y 4 Rotor Failures GD-14 y n Top level Y is big enough to be seen at OBU level – an operational business objective. Frequently the parent project does at least a verification of this top level Y with the customer. x 4,1 x 4,2 x 4,3 x 4,4 x 4,N After eight (8) projects From January ’05 to May ’06 the average tube life doubled. x 3,1 x 3,2 x 3,3 x 3,4 x 3,N x 1,1 x 1,2 x 1,3 x 1,4 x 1,N x 2,1 x 2,2 x 2,3 x 2,4 x 2,N

10. Selecting the Right Output Indicator Good output indicators relate directly to CCR. The correlation of the output indicator to the CCR is the most important consideration…otherwise the indicator is meaningless. A balance must be established between the following: Ease of obtaining data. Time available to the project team. Cost associated with obtaining the data for each indicator.

11. Clarify and Refine CCRs Before Selecting Indicators

12. Associated CCR Output Indicators Output Indicator Product delivery cycle time from the completion of the customer order to the delivery of the product. Number and type of vehicle specifications delivered correctly. Actual delivery time vs. promised. Delivery time for each vehicle. Number of times vehicles were delivered to location other than what is specified on agreement. CCR Product is delivered within three hours of order taken. Right vehicle is delivered at the right time to the right location. Output Indicator Number and type of vehicle specifications delivered correctly. Actual delivery time vs. promised delivery time for each vehicle. Number of times vehicles were delivered to location other than what is specified on agreement. CCR The vehicle delivered meets the vehicle specifications as described in the contract. The vehicle is delivered within the time specified in the contract. The vehicle is delivered at the location specified in the contract. An example of one CCR with four output indicators. Three CCRs with one corresponding output indicator. OR

13. Output Indicators Guidelines Designate output indicators for each CCR. Decide on an appropriate measurement frequency. Ensure measurement instructions are clear and understandable. Measure defects of the process. Use actual results for analysis. Think ahead and anticipate measures to enhance the defect analysis. Limit the number of indicators to three (3).

14. Objective: Identify a list of potential output indicators to evaluate the extent which the process meets CCRs. Instructions: List a critical customer requirement in the right column of the table below. Determine if the CCR describes a single or multiple customer expectations associated with an output. Identify ways to measure whether or not the process is meeting the critical customer requirement. What are the units of measure? When and where would the data be collected? Select one or more measures that may serve as good potential output indicators and list in the table below. Repeat the process for the remaining CCRs. Be prepared to discuss your conclusions with the class. Course Exercise: Identify Potential Output Indicators CCRs Output Indicator Critical Customer Requirement

15. Course Exercise: Link to Customer Objective: Identify the customer. Instructions: For each output indicator on the previous page: Identify the customer. What should be measured (operational definition)? How would the data be collected?

16. Measurement Considerations… One Rule: use as few indicators as possible . There is a cost/benefit and relevance relationship to measuring. Too many measures are costly, unproductive, and may divert attention from critical areas. Not enough or incorrect process indicators will not allow you to identify defects before they occur. The appropriate level of measurement will enable the organization to focus on the correct areas and provide a means for early detection of defects and out-of-control activities necessary to prevent customer dissatisfaction.

17. Clarifying CCRs As a class, consider each of the CCRs translated from Voice of the Customer (VOC). Confirm each CCR meets the criteria as described in the table below. If it does not, write the draft CCR in the VOC column and identify the key customer issues associated with the statement. Identify and list the final CCR in the appropriate space. Does this CCR need to be validated? If so, what validation data may be available?

18. CCRs and Multiple Output Indicators In the previous section, teams translated a variety of VOC data into critical customer requirements. Teams were careful to recognize some customer feedback and statements need to be clarified, and a process for specifying CCRs involves considering key issues customers may have with a product or service. From these issues, the team was able to specify the critical customer requirements of the process output. Some CCRs may be measured in terms of one specific expectation a customer has. Others may require several output indicators. The table shows how one CCR can have one or several associated output indicators. This table lists the second CCR from the above table, but shows it as three (3) different critical customer requirements associated with the delivery of a vehicle. Regardless of how the CCR is stated, teams must be sure to identify all CCRs associated with a process production service and make sure CCRs are measured by the output indicator. Output Indicator Product delivery cycle time from the completion of the customer order to the delivery of the product. Number and type of vehicle specifications delivered correctly. Actual delivery time vs. promised. Delivery time for each vehicle. Number of times vehicles were delivered to location other than what is specified on agreement. Product is delivered within three (3) hours of order taken. Right vehicle is delivered at the right time to the right location. Output Indicator Number and type of vehicle specifications delivered correctly. Actual delivery time vs. promised delivery time for each vehicle. Number of times vehicles were delivered to location other than what is specified on agreement. The vehicle delivered meets the vehicle specifications as described in the contract. The vehicle is delivered within the time specified in the contract. The vehicle is delivered at the location specified in the contract. Critical Customer Requirement Critical Customer Requirement

19. Next Step: Develop Process Indicators Organizations must have information about process performance before products or services are delivered. It allows a business to identify abnormal variation or special cause before a defect occurs. Early defect detection prevents dissatisfied customers and retains customers. As CCRs change and opportunities for improvement are identified, the business will know where in the process improvement resources need to be assigned. If the output is defective (e.g., not meeting CCRs) we will have data to begin looking for undesired variation and identify root cause.

20. Effective Process Indicators Effective process indicators must: Be based on facts and data. Predict, or lead, at least one output indicator. Provide valid and quantifiable data. Enable the evaluation of cause-and-effect relationships occurring inside the process. Lead the team to root cause analysis, if CCRs are not being met. This step allows the team to begin capturing data about the process to ultimately have an impact on the customer.

21. Select Where and What to Measure While reviewing the process map, choose logical control points to take measurements: Activities or decisions that if performed incorrectly or inadequately will result in high probability of a poor output indicator result. Decision blocks – How many rejects or special cases are there which are non-value-adding? Activities with a high volume of work built up. Hand-offs to the next functional area. Time-consuming activities. When the control point is identified, determine the best measure for that process indicator.

22. Course Exercise: Identify Additional 0utput Indicators Objective: Identify indicators necessary to measure the teams process performance, in terms of CCRs and business expectations. Instructions: As a team, review the customer value equation: Quality + Service = Customer Value Cost + Time Discuss and respond to the questions on the following page.

23. Course Exercise: Identify Additional 0utput Indicators (cont.) Discuss and respond to these questions: Do the CCRs used in the previous exercise account for what we want this process to accomplish? If not, what is missing? Do the CCRs from the previous exercise address the time it takes to perform the process? If not, is it important to make improvements to the process cycle time? If so, are any additional indicators needed? Is it important to reduce the cost of the process? If so, what additional indicator(s) may be needed? Do the output indicators measure the quality and service expectation of the customers and the business? If not, what additional indicator(s) may be needed? If necessary, work with leadership to establish critical requirements that may not be considered critical to the customer, but are very important to the business. Identify additional output indicators needed to achieve the business objectives, strategies, and goals. Review the output indicators selected with the criteria and questions addressed in Selecting the Right Output Indicators .

24. Selecting the Right Process Indicators In addition to making sure the indicators provide the team with valid and quantifiable data, teams must be sure what they are measuring actually enables them to evaluate the cause and effect relationships occurring inside the process. Below is a list of questions each team should review after identifying output, process, and input indicators: Are each of the process indicators true predictors , or leading indicators, of at least one output indicator? Do the process indicators evaluate areas of the process known to adversely affect the quality of the process output? Has the team identified process indicators for the process variables that most influence the ability of the process in meeting critical customer requirements and, therefore, the output indicators? If a critical customer requirement is not met, does the team know why (root cause)? What additional process indicators may be needed to answer this question?

25. Course Exercise: Identify Potential Process Indicators Objective: Determine potential process indicators that serve as good predictors of the ability to meet the CCRs. Instructions: Carefully review functional deployment maps created for the process. List the output indicators in the appropriate spaces of the table below. Working from the beginning of the process, identify the activities or decisions likely to have a significant influence on the output indicators. Of the activities or decisions identified, which ones are known, or are assumed to, have been a problem historically? Select the activities that serve as the best predictors of process output success. Of the measures identified, evaluate the process indicators selected by applying the characteristics of good measures , and selecting the right process indicators . Identify any activities or decisions currently under review or already selected as improvement opportunities? Do any of the steps or activities have readily accessible data available?

26. Selecting the Right Input Indicators Input indicators allow measurement of the consistency of the inputs to the process. Do the input indicators measure the critical requirements we have of our suppliers’ products or services? Do the indicators measure elements of the input known to affect the ability of our process to meet critical customer requirements? Are the input indicators true predictors , or leading indicators, of at least one process indicator? Do the indicators measure aspects of the input that would, within a specified tolerance, eliminate significant inspection, scrap, rework, or excessive cycle time?

27. Course Exercise: Identify Input Indicators Objective: Determine the input indicators that serve as the best “predictors” of the ability to meet critical process requirements. Instructions: Carefully review the list of inputs provided to the process. Of the inputs listed, which ones have the potential for having a drastic impact on process indicators, outcome indicators, and/or CCRs? Select the process inputs that, if not perfect (accurate, complete, timely, defect-free, etc.) will prevent the process from meeting CCRs or add significant cost and additional effort to the process. List the process requirement for your supplier's input. Determine the output indicators needed to help ensure process activities and decisions meet desired performance. Of the measures identified, select one or two input indicators by applying the characteristics of good measures and selecting the right indicators , shown earlier. Repeat the previous steps for the remaining process inputs.

28. Indicator Relationships Link output performance to process and input indicators. First, look to establish output indicators because they indicate how effective your process is at meeting CCRs. Once you understand the key output performance measures, determine what key input and process indicators you need in order to meet the desired outcomes and, therefore, satisfy customer requirements. You can use a relationship matrix to help show the relationship between the output performance measures and key input and process measures. The relationship matrix will help prioritize the most important measures. Note: The strength of the relationship is based on how likely changes in the input or process indicator will cause changes in the output indicator.

29. Link Output Performance to Process & Input Indicators Relationship of process & input measures Strong Relationship Medium Relationship Weak Relationship No Relationship Blank Output Performance Indicators Process & Input Indicators Call Abandon Rate Customer Satisfaction Answer Speed Employee Experience First Time Resolution The strength of the relationship is based on how likely changes in the input/process measure will cause changes in the output performance measure.

30. Course Exercise: Indicator Relationships Objective: To determine the strengths of the relationships between the team's Input, Process, and Output indicators. Instructions: List the draft output indicators on the far left column of the Relationship Matrix below. List the draft input and process indicators along the top row of the Relationship Matrix. As a team, discuss the following questions for each of the pairs of input or process indicators and the associated output indicators: To what extent do the input or process indicators predict the output? If the input or process indicator were to suddenly change, would there be a corresponding change in the output indicator? When would it occur? What other activities taking place in the process may have a more dramatic effect on the output? How may we measure that? Using the scale at the bottom of the page, assign a value of strong, medium, weak, or none to describe the relationship between each possible pair of indicators. Add any additional indicators the team may have identified and assign a corresponding relationship strength value. Be prepared to present and discuss the team conclusions with other course participants.

31. Measurement Plan: Uphold Principles The measure must be important to the customer. The measure must be easy to understand. The measure is sensitive to the right things. The measure must promote appropriate analysis and action. The data needed must be easy to obtain. The measure needs to be driven by business objectives. The measure needs to lead to process improvement.

32. Choose Measurement Indicators Wisely Choose as few indicators as possible. Too many measures are costly to collect and analyze. Too few measures will limit your ability to identify defects. Characteristics of Good Measures: Linked to CCRs; Simple to read and understand; Help drive appropriate corrective actions; Provide insight to cause-and-effect relationships; Representative and relevant to the process; Reveal trends and cycles; Easy to collect; Clearly defined; Statistically valid; and Repeatable.

33. Review and Transition In section 2.1 Determine What to Measure , we learned: The role data plays in process improvement. The cause-and-effect relationships that occur inside the team's process. How to determine the indicators needed to evaluate current process performance. In section 2.2 Manage Measurement , we will learn: The different types of data and how each type can provide the team with different insights and knowledge of a process. How to develop operational definitions and data collection plans to build validity and consistency in the data which the team gathers. 2.1 Determine What to Measure 2.2 Manage Measurement 2.3 Understand Variation 2.5 Managing the Measurement System 2.4 Determine Sigma Performance