Your SlideShare is downloading. ×
Measurement systems analysis v1.1
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Measurement systems analysis v1.1

1,155
views

Published on

Published in: Education

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,155
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
158
Comments
0
Likes
1
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
    CircleStandard
    1s
    2l
    3m
    4m
    5m
    6s
    7l
    8l
    9s
    10m
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • 48106_MSA rd ai
    03/04/14 01:00 PM
  • Transcript

    • 1. Measurement Systems Analysis (MSA) © 2001 Six Sigma Academy 1
    • 2. Why Measure? • To understand a decision: • Meet standards & specifications • Detection/reaction oriented • Short-term results • Stimulate continuous improvement: • Where to improve? • How much to improve? • Is improvement cost effective? • Prevention oriented • Long-term strategy “If you cannot measure, you cannot improve!” – Taguchi © 2001 Six Sigma Academy 2
    • 3. Measurement System As A Process Material Method Machine Cleanliness Sequence Cleanliness Temperature Temperature Timing Dimension Design Positioning Weight Precision Corrosion Calibration Location Hardness Resolution Set-up Conductivity Stability Density Preparation Wear Compliance-procedure Fatigue Vibration Attention Calculation error Atmospheric pressure Interpretation Speed Lighting Coordination Knowledge-instrument Temperature Dexterity Vision Humidity Cleanliness Environment © 2001 Six Sigma Academy Measurement Error People 3
    • 4. What Is An MSA? Scientific and objective method of analyzing the validity of a measurement system • A “tool” which quantifies: 1. Equipment Variation 2. Appraiser (Operator) Variation 3. The Total Variation of a Measurement System • MSA is NOT just Calibration • MSA is NOT just Gage Repeatability & Reproducibility (R&R) Measurement System Analysis is often a “project within a project” © 2001 Six Sigma Academy 4
    • 5. MSA Relationship To DMAIC Define Measure Analyze Improve Control Measurement Systems Analysis • Quantitative evaluation of tools and processes used in making discrete or variable observations Define Measure Analyze Improve Control Measurement Systems Control • Established, documented, and continuously carried out • Ensures measurement system maintains an acceptable status • Often referred to as “Long Term Gage Plan” © 2001 Six Sigma Academy 5
    • 6. MSA - A Starting Point Before you… • Make adjustments • Implement solutions • Run an experiment • Perform a complex statistical analysis You should… • Validate your measurement systems • Validate data and data collection systems MSA quantifies a major source of process variation © 2001 Six Sigma Academy 6
    • 7. Measurement Systems • Examples • Precision gage • Data collection form • Survey • School entrance exam • Customer satisfaction • On-time delivery report What is your system ? © 2001 Six Sigma Academy 7
    • 8. Types of Measurement System Analysis • Operational Definitions • Walking the Process • Gage R&R • Variable Data • Attribute Data © 2001 Six Sigma Academy 8
    • 9. MSA – Operational Definitions The Measurement System can be validated using Operational Definitions constructed by the Project Team to ensure that all measurement takers completely understand what is expected during the data collection phase. © 2001 Six Sigma Academy 9
    • 10. Developing Operational Definition • Operational definitions are descriptions written in a way that ensures consistent interpretation by different people • The operational definition method of description will be used throughout the DMAIC process © 2001 Six Sigma Academy 10
    • 11. • Operational Definition • The technique of defining an item, process or characteristic using Operational Definitions is an effective way to communicate between Team Members and other people involved in the project. Because Operational Definitions are so effective, the technique is used in a number of locations within the DMAIC process. Remember, to be effective, an Operation Definition must be written in a way that ensures consistent interpretation by different people.CC © 2001 Six Sigma Academy 11
    • 12. General Example – Operational Definitions • Examples of Operational Definitions for data collection: • Record the date that the lease company written notification arrives in the dealership using an MM/DD/YY format. • List any cosmetic preparation in excess of the standard predelivery process required to render the vehicle acceptable for retail consumer sale. • Record the weight of each package of coffee in ounces by pouring the coffee into the filter and placing the filter and coffee on the scale tray. • Record the length of time that coffee remains in the urn by recording the actual time of day each time the Brew button is pressed to recharge the urn. Use 24-hour clock and round to the nearest minute. © 2001 Six Sigma Academy 12
    • 13. MSA – Walking the Process “Walking the Process” is a method of conducting MSA when it is not possible to perform a Gage R&R. © 2001 Six Sigma Academy 13
    • 14. How to “Walk the Process • Develop Operational Definitions for each of the measures to be collected • Train data collectors prior to beginning the data collection activity • Follow the process from beginning to end and monitor the data collection activities to determine if data is being collected properly • Continue walking the process until the data compiled accurately reflects the existing process © 2001 Six Sigma Academy 14
    • 15. Components Of Measurement Error © 2001 Six Sigma Academy 15
    • 16. Components Of Measurement Error • • • • • • Resolution/Discrimination Accuracy (bias effects) Linearity Stability (consistency) Repeatability-test-retest (Precision) Reproducibility (Precision) Each component of measurement error can contribute to variation, causing wrong decisions to be made © 2001 Six Sigma Academy 16
    • 17. Categories Of Measurement Error Which Affect Location Accuracy/ Bias Linearity © 2001 Six Sigma Academy Stability 17
    • 18. Categories Of Measurement Error Which Affect Spread Precision Repeatability © 2001 Six Sigma Academy Reproducibility 18
    • 19. Resolution/Discrimination Resolution? Can change be detected? OK Accuracy/Bias? OK Linearity? OK Stability? OK Precision (R&R)? © 2001 Six Sigma Academy 19
    • 20. Resolution • • • • • • Simplest measurement system problem Poor resolution is a common issue Impact is rarely recognized and/or addressed Easily detected No special studies are necessary No “known standards” are needed © 2001 Six Sigma Academy 20
    • 21. Definitions: • Resolution/Discrimination • Capability to detect the smallest tolerable changes • Inadequate Measurement Units • Measurement units too large to detect variation present • Guideline: “10 Bucket Rule” • Increments in the measurement system should be one-tenth the product specification or process variation © 2001 Six Sigma Academy 21
    • 22. Resolution/Discrimination Poor Discrimination Same process output being measured 1 2 3 4 5 1 Better Discrimination 1 2 3 4 5 1.3 © 2001 Six Sigma Academy 22
    • 23. Resolution Actions • • • • • Measure to as many decimal places as possible Use a device that can measure smaller units Live with it, but document that the problem exists Larger sample size may overcome problem Priorities may need to involve other considerations: • Engineering tolerance • Process Capability • Cost and difficulty in replacing device © 2001 Six Sigma Academy 23
    • 24. Accuracy/Bias Resolution? OK Accuracy/Bias? Measurements are “shifted” from “true” value OK Linearity? OK Stability? OK Precision (R&R)? © 2001 Six Sigma Academy 24
    • 25. Accuracy/Bias Difference between the observed average value of measurements and the master value Master Value (Reference Standard) Master value is an accepted, traceable reference standard © 2001 Six Sigma Academy Average Value 25
    • 26. Accuracy/Bias x x x x xx x x x More accurate © 2001 Six Sigma Academy x x x x xx x x x Less accurate 26
    • 27. Accuracy/Bias Actions • • • • • Calibrate when needed/scheduled Use operations instructions Review specifications Review software logic Create Operational Definitions © 2001 Six Sigma Academy 27
    • 28. Linearity Resolution? OK Accuracy/Bias? OK Linearity? OK Measurement is not “true” and/or consistent across the range of the “gage” Stability? OK Precision (R&R)? © 2001 Six Sigma Academy 28
    • 29. Linearit Observed Average Value Bias No Bias Reference Value Full Range of Gage © 2001 Six Sigma Academy 29
    • 30. Linearity Actions • • • • Use only in restricted range Rebuild Use with correction factor/table/curve Sophisticated study required and will not be discussed in this course © 2001 Six Sigma Academy 30
    • 31. Stability Resolution? OK Accuracy/Bias? OK Linearity? OK Stability? Measurement drifts OK Precision (R&R)? © 2001 Six Sigma Academy 31
    • 32. Stability • Measurements remain constant and predictable over time • For both mean and standard deviation Master Value (Reference Standard) • No drifts, sudden shifts, cycles, etc. • Evaluated using control charts Time 1 Time 2 © 2001 Six Sigma Academy 32
    • 33. Stability Actions • • • • Change/adjust components Establish “life” timeframe Use control charts Use/update current SOP © 2001 Six Sigma Academy 33
    • 34. Precision Resolution? OK Accuracy/Bias? OK Linearity? OK Stability? OK Precision (R&R)? © 2001 Six Sigma Academy Repeatability and Reproducibility 34
    • 35. Precision σ2total = σ2product/process + σ2repeatability + σ2reproducibility Master Value Good Precision Poor Precision A B Also known as Gage R&R © 2001 Six Sigma Academy 35
    • 36. Repeatability (A Component Of Precision) • Variation that occurs when repeated measurements are made of the same item under absolutely identical conditions • Same: • Operator • Set-up • Units • Environmental conditions • Short-term © 2001 Six Sigma Academy 36
    • 37. Reproducibility (A Component Of Precision) The variation that results when different conditions are used to make the measurements • Different: • Operators • Set-ups • Test units • Environmental conditions • Locations • Companies • Long-term © 2001 Six Sigma Academy 37
    • 38. R&R Actions Repeatability • Repair, replace, adjust equipment • SOP Reproducibility • Training • SOP © 2001 Six Sigma Academy 38
    • 39. Attribute Measurement Studies © 2001 Six Sigma Academy 39
    • 40. Purpose Of Attribute MSA • • • • • Assess standards against customers’ requirements Determine if all appraisers use the same criteria Quantify repeatability and reproducibility of operators Identify how well measurement system conforms to a “known master” Discover areas where: • Training is needed • Procedures are lacking • Standards are not defined © 2001 Six Sigma Academy 40
    • 41. Attribute MSA - Excel Method • Allows for R&R analysis within and between appraisers • Test for effectiveness against standard • Limited to nominal data at two levels © 2001 Six Sigma Academy 41
    • 42. Attribute MSA Example 5 Attribute Legend (used in computations) 1 Pass 2 Fail Open file MSA-Attribute.xlsOperator #1 Known Population Sample # 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 © 2001 Six Sigma Academy Attribute Pass Pass Pass Pass Fail Fail Pass Pass Fail Pass Pass Pass Pass Pass Fail Pass Pass Pass Fail Pass Pass Pass Pass Pass Fail Pass Pass Pass Fail Pass Try #1 Pass Pass Pass Pass Fail Pass Pass Pass Fail Pass Pass Pass Pass Pass Fail Pass Pass Pass Fail Pass Pass Fail Pass Pass Fail Pass Pass Pass Fail Pass Try #2 Pass Pass Pass Pass Fail Pass Pass Pass Fail Pass Pass Pass Pass Pass Fail Pass Pass Pass Fail Pass Pass Fail Pass Pass Fail Pass Pass Pass Fail Pass DATE: 1/4/2001 NAME: Acme Employee PRODUCT: Widgets BUSINESS: Earth Products Operator #2 Try #1 Try #2 Pass Pass Pass Pass Pass Pass Pass Pass Fail Fail Pass Pass Pass Pass Pass Pass Fail Fail Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Fail Fail Pass Pass Pass Pass Pass Pass Fail Fail Pass Pass Pass Pass Pass Pass Pass Pass Pass Pass Fail Fail Pass Pass Pass Pass Pass Pass Fail Fail Pass Pass Microsoft Excel Worksheet Operator #3 Try #1 Try #2 Pass Pass Pass Pass Pass Pass Fail Pass Pass Fail Pass Pass Pass Pass Pass Pass Fail Fail Pass Pass Pass Pass Pass Pass Pass Pass Fail Pass Pass Fail Pass Pass Pass Pass Pass Pass Fail Fail Pass Pass Pass Pass Pass Pass Pass Pass Fail Pass Fail Fail Pass Pass Pass Pass Pass Pass Fail Fail Pass Pass 42
    • 43. Scoring Example % APPRAISER SCORE - > 100.00% 78.57% 100.00% % SCORE VS. ATTRIBUTE - > 78.57% 64.29% 71.43% SCREEN % EFFECTIVE SCORE - > 57.14% SCREEN % EFFECTIVE SCORE vs. ATTRIBUTE - > 42.86% • 100% is target for all scores • <100% indicates training required • % Appraiser score = repeatability • Screen % Effectiveness Score = reproducibility • % Score vs. Attribute • individual error against a known population • Screen % Effective vs. Attribute • Total error against a known population © 2001 Six Sigma Academy 43
    • 44. Statistical Report © 2001 Six Sigma Academy 44
    • 45. Statistical Report © 2001 Six Sigma Academy 45
    • 46. Statistical Report Continued © 2001 Six Sigma Academy 46
    • 47. Attribute MSA – MINITAB™ Method • • • • Allows for R&R analysis within and between appraisers Test for effectiveness against standard Allow nominal data with two levels Allows for ordinal data with more than two levels © 2001 Six Sigma Academy 47
    • 48. MINITAB Method - Data Entry • Same data as Excel example • Arranged in multiple columns • Data can also be stacked in single column © 2001 Six Sigma Academy 48
    • 49. Attribute Study - MINITAB Analysis Attribute MSA.mpj Attribute MSA.MPJ Tool Bar Menu > Stat > Quality Tools > Attribute Gage R&R Study © 2001 Six Sigma Academy 49
    • 50. Attribute Study - MINITAB Analysis Continued 1. Select “Single Column” if data is stacked 1. Select “Multiple Columns” if data is un-stacked 2. Enter number of appraisers and trials 3. Enter name of column with “Known” © 2001 Six Sigma Academy 4. Select OK 50
    • 51. Attribute MSA - MINITAB Graphical Output Date of study: 1/03/2001 Reported by: Jose Name of product: XYZ Report Misc: Assessment Agreement Lower variation within appraiser Within Appraiser Appraiser vs Standard Lower variation appraiser vs. standard 100 100 [ , ] 95.0% CI Percent 90 Percent Percent 90 80 80 70 Higher variation within appraiser 70 60 Bob Sue Appraiser Tom Bob Sue Tom Appraiser Higher variation appraiser vs. standard Not included if no “Known” © 2001 Six Sigma Academy 51
    • 52. Attribute MSA – MINITAB Session Window Results Each Appraiser vs. Standard Individual vs. Standard Assessment Agreement Appraiser # Inspected # Matched Percent (%) 95.0% CI Bob 30 28 93.3 ( 77.9, 99.2) Sue 30 29 96.7 ( 82.8, 99.9) Tom 30 24 80.0 ( 61.4, 92.3) # Matched: Appraiser's assessment across trials agrees with standard. Assessment Disagreement Appraiser # Pass/Fail Percent (%) # Fail/Pass Percent (%) # Mixed Percent (%) Bob 1 3.3 1 3.3 0 0.0 Sue 1 3.3 0 0.0 0 0.0 Tom 1 3.3 0 0.0 5 16.7 # Pass/Fail: Assessments across trials = Pass/standard = Fail. Disagreement assessment (repeatability) # Fail/Pass: Assessments across trials = Fail/standard = Pass. # Mixed: Assessments across trials are not identical. Between Appraisers Assessment Agreement # Inspected # Matched Percent (%) 30 24 95.0% CI 80.0 ( 61.4, 92.3) # Matched: All appraisers' assessments agree with each other. All Appraisers vs. Standard Assessment Agreement # Inspected # Matched Percent (%) 30 23 Total agreement (against known) 95.0% CI 76.7 ( 57.7, 90.1) # Matched: All appraisers' assessments agree with standard. © 2001 Six Sigma Academy Between appraisers (reproducibility) 52
    • 53. MINITAB Method - Ordinal Data Entry Ordinal MSA.mtw • Survey data rated on a 1 to 5 scale • Arranged in multiple columns © 2001 Six Sigma Academy Minitab Worksheet 53
    • 54. Attribute Study - Ordinal Select “categories of the attribute data are ordered” Analysis is same as 2 level data © 2001 Six Sigma Academy 54
    • 55. Industrial Attribute MSA Exercise • • • • Evaluate samples supplied by instructor Determine the screen and appraiser scores Interpret the results Recommend actions iGrafx Professional Document attributecircles.MPJ © 2001 Six Sigma Academy 55
    • 56. Variables Measurement Studies © 2001 Six Sigma Academy 56
    • 57. Six Step Variables MSA 1. 2. 3. 4. 5. 6. Conduct initial gage calibration (or verification) Perform trials and data collection Obtain statistics via MINITAB Analyze, interpret results Check for inadequate measurement units On-going evaluation • What would be your long-term gage plan ? © 2001 Six Sigma Academy 57
    • 58. Trials And Data Collection • Generally two to three operators • Generally 5-10 process outputs to measure • Each process output is measured 2-3 times (replicated) by each operator O p e r1 P1 1 2 P2 3 1 2 O p e r2 P3 3 1 2 P4 3 1 2 P5 3 1 2 P1 3 1 2 O p e r3 ... 3 1 2 P5 3 1 2 P1 3 1 2 ... 3 1 2 P5 3 1 Randomization is Critical © 2001 Six Sigma Academy 58 2 3
    • 59. Randomization, Repeats, Replicates Randomization • Runs are made in an arbitrary vs. patterned order • Average out effects of noise or unknown factors • Tradeoff - Invalid results versus slight inconvenience (if any) Repeats • Running more than one sample of a single run • Results are averaged Replication • Running entire experiment in a time sequence • MSA allows for repeatability study © 2001 Six Sigma Academy 59
    • 60. Variables MSA - MINITAB Example Variable MSA.mtw USL=1. Replicate 1 5 LSL=0. 5 © 2001 Six Sigma Academy Replicate 2 Variable MSA.MTW (Randomized order) 60
    • 61. MSA Using MINITAB 10 Process Outputs 3 Operators 2 Replicates USL=1. 5 LSL=0. Replicate 1 Replicate 2 (Randomized order) 5 • Have Operator 1 measure all samples once (as shown in the outlined block) • Then, have Operator 2 measure all samples once • Continue until all operators have measured samples once (this is Replicate 1) • Repeat these steps for the required number of Replicates • Enter data into MINITAB in 3 columns as shown © 2001 Six Sigma Academy 61
    • 62. Manipulate The Data Your data in MINITAB should initially look like this. You will need to STACK your data so that all like data is in one column only Use the commands > Manip > Stack > Stack Blocks of Columns (Stack all Process Outputs, Operators, and Responses so that they are in one column only) Now you are ready to run the macro for data analysis © 2001 Six Sigma Academy 62
    • 63. Stacked And Ready For Analysis Note: c10, c11, c12 are the columns in which the respective data are found IN OUR EXAMPLE. You must have ALL data STACKED in these columns Enter titles © 2001 Six Sigma Academy 63
    • 64. Prepare The Analysis Use the commands > Stat > Quality Tools > Gage R&R Study (Crossed) Each process output measured by each operator OR > Gage R&R Study (Nested) For “destructive tests” where each process output is measured uniquely by each operator © 2001 Six Sigma Academy 64
    • 65. Choose Method Of Analysis Enter Gage Info and Options ANOVA method is preferred • Gives more information © 2001 Six Sigma Academy 65
    • 66. Adding Tolerance (Optional) Upper Specification Limit (USL) Minus Lower Specification Limit (LSL) For this example: USL=1.0 USL=1.0 LSL=0.5 LSL=0.6 USL - LSL=0.50 © 2001 Six Sigma Academy 66
    • 67. MSA Output: Session Window Graphs Two-Way ANOVA Table With Interaction DF SS MS F P 9 2.05871 0.228745 39.7178 0.00000 Operator 2 0.04800 0.024000 4.1672 0.03256 Operator*Part 18 0.10367 0.005759 4.4588 0.00016 Repeatability 30 0.03875 0.001292 Total 59 2.24912 Gage R&R Total Gage R&R 0.004437 100 Gage R&R Repeat Reprod Part Part-to-Part 1 2 3 R Chart by Operator 0.15 Sample Range VarComp By Part 1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 %Contribution %Study Var %Tolerance 0 %Contribution Source Components of Variation 200 Percent Part (of VarComp) 10.67 0.001292 3.10 Reproducibility 0.003146 7.56 Operator 0.000912 2.19 Operator*Part 0.002234 0.037164 89.33 Total Variation 0.041602 UCL=0.1252 0.05 R=0.03833 0.00 LCL=0 100.00 1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 Operator 1 StdDev Study Var %Study Var (5.15*SD) (%SV) 0.066615 0.34306 32.66 1 0.035940 0.18509 17.62 0.056088 0.28885 27.50 3 0.030200 0.15553 14.81 Part 1 2 3 1 2 3 4 5 6 7 8 31.11 Operator*Part 0.047263 0.24340 23.17 48.68 Part-To-Part 0.192781 0.99282 94.52 198.56 Total Variation 0.203965 1.05042 100.00 210.08 Number of Distinct Categories = 4 © 2001 Six Sigma Academy Operator 1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 57.77 Operator 10 3 37.02 Reproducibility 9 2 68.61 Repeatability 8 (SV/Toler) Total Gage R&R 7 %Tolerance (SD) 6 Operator*Part Interaction UCL=0.8796 Mean=0.8075 LCL=0.7354 0 Source 2 5 1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 3 Xbar Chart by Operator 5.37 Part-To-Part 2 0.10 0 Sample Mean Repeatability 1 4 By Operator Average Source Gage name: Date of study: Reported by: Tolerance: Misc: Gage R&R (ANOVA) for Response What does all this mean? 67 9 10
    • 68. Graphical Output - 6 Graphs In All MSA Gage R&R (ANOVA) for Response Health Side Components of Variation By Part Percent 200 1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 %Contribution %Study Var %Tolerance 100 0 Gage R&R Repeat Reprod Part Part-to-Part 1 2 3 R Chart by Operator Sample Range 0.15 1 2 3 0.05 R=0.03833 0.00 LCL=0 0 Operator 1 Average Sample Mean © 2001 Six Sigma Academy 7 8 9 10 If only 1 operator, you won’t get these graphs 3 Operator*Part Interaction 3 UCL=0.8796 Mean=0.8075 LCL=0.7354 0 6 2 Xbar Chart by Operator 2 5 By Operator UCL=0.1252 1 4 1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.10 1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 MSA Troubleshoot Side Gage name: Date of study: Reported by: Tolerance: Misc: Operator 1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 Part 1 2 3 1 2 3 4 5 6 7 8 9 10 If nested study, you won’t get this graph 68
    • 69. Destructive Test Gage name: Date of study: Reported by: Tolerance: Misc: Gage R&R (Nested) for Response Components of Variation By Part (Operator) Percent 100 %Contribution %Study Var 18 17 16 50 15 14 13 0 Gage R&R Repeat Reprod Part Operator Part-to-Part 6 7 8 9 10 11 12 13 14 15 1 2 3 4 5 Billie Nathan Steve R Chart by Operator Sample Range 5 Billie Nathan By Operator 18 Steve UCL=4.290 4 3 17 16 2 15 R=1.313 1 0 LCL=0 14 13 Operator Billie Nathan Steve Xbar Chart by Operator Sample Mean 18 17 Billie Nathan Steve UCL=17.62 16 15 Mean=15.15 14 13 12 © 2001 Six Sigma Academy LCL=12.68 Operator by process output interaction is not applicable 69
    • 70. Graphical Output Metrics Chart Output • Xbar Chart: Shows sampled process output variety • Reproducibility/bias/sensitivity • R Chart: Helps identify unusual measurements • Resolution/repeatability • Bar Chart: Distinguishes R&R from Process Output to Process Output • Components of variation These are your leading graphical indicators © 2001 Six Sigma Academy 70
    • 71. Gage name: Bar Charts For Components Date of study: Gage R&R (ANOVA) for Response Reported by: Tolerance: Misc: Needs help Components of Variation Percent 100 3 %Contribution %Study Var 2 50 1 0 Gage R&R Repeat Reprod Part Part-to-Part 1 R Chart by Operator 1 Sample Range 4 2 3 3 UCL=3.915 2 2 1 R=1.198 0 LCL=0 0 1 Operator 1 Xbar Chart by Operator 3 2 1 1 2 Operato 3 2.0 UCL=3.654 Answers: “Where is the variation?” Mean=1.401 Average 4 ple Mean 3 By 3 Much better © 2001 Six Sigma Academy 2 1.5 71
    • 72. Closer Look At The Xbar & R Charts R Chart: Exposes gage Repeatability, resolution & stability Xbar Chart: Test of sensitivity, bias, & population variety Xbar: at least 50% outside limits; R chart: in control © 2001 Six Sigma Academy 72
    • 73. More R Chart Indicators R Chart 1 Sample Range 0.005 Randy 2 Rbar too small? 3 0.004 0.003 0.002 UCL=0.001416 0.001 R=4.33E-04 LCL=0 0.000 0 R Chart by Operator Sample Range 0.15 1 2 Plateaus 3 UCL=0.1252 0.10 0.05 R=0.03833 0.00 LCL=0 0 Both may indicate poor gage resolution © 2001 Six Sigma Academy 73
    • 74. Tabular Output Metrics %Contribution %Study %Tolerance Number of Distinct Categories © 2001 Six Sigma Academy 74
    • 75. % Contribution σ σ 2 % Contribution = R&R 2 * 100 TOTAL • Measurement System Variation (R&R) as a percentage of Total Observed Process Variation % Contribution • Includes both repeatability and reproducibility 9% 1% © 2001 Six Sigma Academy 75
    • 76. % Study Variation σ % Study Variation = σ R &R * 100 TOTAL • Looks at standard deviations instead of variance • Measurement System Standard Deviation (R&R) as a percentage of Total Observed Process Standard Deviation % Study • Includes both repeatability and reproducibility Variation 30% 10% © 2001 Six Sigma Academy 76
    • 77. % Tolerance Precision to Tolerance P/T % Tolerance = 5.15 * σR&R * 100 Tolerance • Measurement error as a percent of tolerance • Includes both repeatability and reproducibility • 5.15 Study Variation = 99% Acceptance Criteria % Tolerance 30% 10% © 2001 Six Sigma Academy 77
    • 78. Distinct Categories  2 σ Process Output   Number of Distinct Categories = 2 *  2  σ R &R    • Number of divisions that the Measurement System can accurately measure across the process variation • How well a measurement process can detect process output variationprocess shifts and improvement Number of Distinct • Less than 5 indicates Attribute conditions Categories 5 10 © 2001 Six Sigma Academy 78
    • 79. Acceptability Summary Tabular Method % Contribution Process Control % Study Variation Product Control % Tolerance Number of Distinct Categories 9% 30% 30% 5 1% 10% 10% 10 Desirable to Have All 4 Indicators Say “Go” © 2001 Six Sigma Academy 79
    • 80. Keys To Successful MSA • Define and validate measurement process • Identify known elements of the measurement process (operators, gages, SOP, setup, etc.) • Clarify purpose and strategy for evaluation • Set acceptance criteria • Implement preventive/corrective action procedures • Establish on-going assessment criteria and schedules © 2001 Six Sigma Academy 80
    • 81. Gage R&R - Which % Gage R&R Do I Use? Depending on how variable your process is as compared to tolerance, your % Gage R&R values as a percent of Study variation, Tolerance and Process Variation will be quite different. For example: Consider a very stable process with low variability. Percent Tolerance will indicate that your gauge is very good (low % GRR) with high discrimination. On the other hand, when compared to process variation, the GRR will be poor (High % GRR). As your process improves, you will need to move to more precise gauges if you wish to “see” decreases in variation due to the measuring system. On the other hand, if you truly only want to be able to tell when production is becoming less capable, then you are only interested in the precision of the gauge as it relates to your customer’s specification. See the Appendix at the end of this module for further examples © 2001 Six Sigma Academy 81
    • 82. Gage R&R, Graphical Output: Gage name: Date of study: Reported by: Tolerance: Misc: Gage R&R (ANOVA) for Measure Gage #020371 01/01/1998 Six Sigma BB 1.5 mm Buffalo, NY Plant Operator Average Operator*Part Interaction Gage name: Date of study: Reported by: Tolerance: Misc: 1 Gage R&R (ANOVA) for Measure 2 3 1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 Gage #020371 01/01/1998 Six Sigma BB 1.5 mm Buffalo, NY Plant By Operator 1 2 3 4 5 6 Part ID 7 8 9 10 1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 Gage #020371 01/01/1998 Six Sigma BB 1.5 mm Buffalo, NY Plant By Part 1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 1 2 3 4 5 6 7 8 9 10 Part ID 1 • Operator * Part Interaction: Gage name: Date of study: Reported by: Tolerance: Misc: Gage R&R (ANOVA) for Measure 2 3 Oper ID • Shows if any given part(s) was hard to manage for any given operator(s) • Appears as though at least two of the operators had trouble measuring part #10 • What would the ideal graph look like? • By Operator: • Shows if any operator(s) had higher or lower readings (on average) than the others • What would the ideal graph look like? • By Part: • Shows the ability of all of our operators to obtain the same readings for each part • Also shows the ability of our measurement system to distinguish between parts (amount of overlap) • What would be the ideal graph look like? © 2001 Six Sigma Academy 82
    • 83. Gage R&R, Xbar & R: • How do we evaluate the X-bar & R-chart? • Why are the data points out of control on the X-bar and R chart? Sample Mean Gage R&R (ANOVA) for Measure 1.1 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 Xbar Chart by Operator 1 2 Gage #020371 01/01/1998 Six Sigma BB 1.5 mm Buffalo, NY Plant 3 3.0SL=0.8796 X=0.8075 -3.0SL=0.7354 0 0.15 Sample Range Gage name: Date of study: Reported by: Tolerance: Misc: R Chart by Operator 1 2 3 3.0SL=0.1252 0.10 0.05 R=0.03833 0.00 -3.0SL=0.000 0 © 2001 Six Sigma Academy 83
    • 84. Minitab, Gage Run Chart: • Generates a run chart of measurements by operator and part id • Allows us to visualize repeatability and reproducibility within and between operator and part • The center line is the overall average of the parts • STAT > Quality Tools > Gage Run Chart Runchart of Measure by Part, Operator 1.08 0.98 0.88 0.78 0.68 0.58 0.48 0.38 Part Num 1 2 3 4 5 1.08 0.98 0.88 0.78 0.68 0.58 0.48 0.38 Part Num 6 7 8 9 10 Measure Measure 1 2 3 © 2001 Six Sigma Academy 84
    • 85. P/T Ratio Effect on Capability 6.0 Actual Cp 5.0 P/T Ratio 4.0 0% 10% 20% 30% 40% 50% 60% 70% 3.0 2.0 1.0 0.0 0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0 Observed Cp © 2001 Six Sigma Academy 85
    • 86. % R&R Vs. Capability Which Might Need The Most Attention? Measurement System or Process Capability Process %R&R Obs. Cp Decision ? 1 10% 0.5 ? 2 60% 1.4 ? 3 60% 0.5 ? 4 70% 6.5 ? © 2001 Six Sigma Academy 86
    • 87. % R&R Vs. Capability Which Might Need The Most Attention? Measurement System or Process Capability Process %R&R Obs. Cp Decision ? 1 10% 0.5 Capability 2 60% 1.4 Measurement 3 60% 0.5 Maybe Both 4 70% 6.5 Measurement *Note: Process Step 4 Would improving %R&R really be worth the effort ? © 2001 Six Sigma Academy 87
    • 88. Handling Poor Gage Capability: • • • • • If a dominant source of variation is repeatability (equipment), you need to replace, repair, or otherwise adjust the equipment. If, in consultation with the equipment vendor or upon searches of industry literature, you find that the gage technology that you are using is “state-of-the-art” and it is performing to its specifications, you should still fix the gage. One temporary solution to this problem is to use signal averaging (see next page). If a dominant source of variation is operator (reproducibility), you must address this via training and definition of the standard operating procedure. You should look for differences between operators to give you some indication as to whether it is a training, skill, and/or procedure problem. Evaluate the specifications. Are they reasonable? If the gage capability is marginal (as high as 30% of study variation) and the process is operating at a high capability (Ppk greater than 2), then the gage is probably not hindering you and you can continue to use it. © 2001 Six Sigma Academy 88
    • 89. Controlling Repeatability: • Note: If you want to decrease your gage error take advantage of the standard error square root of the sample. • The signal averaging technique uses: 1 n • • • • n = the number of repeat measures taken on the same part the measurement = the average of “n” readings Example: a gage error of 50% can be cut in half if your point estimate is an average of 4 repeat measurements 1 = 1/ 2 4 This technique should be used as a short term approach to perform a study, but you must fix the gage. © 2001 Six Sigma Academy x x x xx x x xx x x Distribution of Individuals x xx x x xx Distribution of Means 89
    • 90. Other Statistical Indexes The Signal-to-Noise Ratio (S/N Ratio) relates the product variation to the measurement system variation. The S/N Ratio should be as large as possible. S / N Ratio = σ σ P MS The Discrimination Index provides the number of divisions that the Measurement System can accurately measure across the part (sample) variation. If this index is less than 4, then it is inadequate to provide data for a study. If the index is 4, then it is equivalent to a go/no-go gage. We would like to see the value of 5 or greater. σ p Discrim=   σ  * 1.41   ms  © 2001 Six Sigma Academy 90
    • 91. Effects of P/T and S/N Ratios • The effect of P/T on Cpk • Large P/T reduces the process Cpk from the true value to some smaller observed value. • The effect of P/T on part assessment • Large P/T increases the probability that we will misclassify product as defective when it’s really good and vice versa. • The effect of S/N ratio on control chart sensitivity • small S/N increases the time before an out-of-control process is detected by a control chart (refer to X-bar & range) • The Effect of the Discrimination Index • If the Index = 2, only attribute data is available and sample sizes must be larger. • If the Index is 5 to 10, then discrimination is finer and sample sizes can be smaller. © 2001 Six Sigma Academy 91
    • 92. Calibration Steps • Determine if the measurement system needs to be recalibrated • Determine the minimum number of measurements needed to make this decision • Take data and make decision • If yes, recalibrate system • Why don’t we just recalibrate? • Normal variation causes the measurement to be slightly different each time it is used • Recalibration should be done only when the measurements are off by more than the normal variation • Recalibrating a system when it is not needed can increase the variability in the measurements © 2001 Six Sigma Academy 92
    • 93. Appendix © 2001 Six Sigma Academy 93
    • 94. Interpreting Variables GR&R Results Presented on the following slides are four Variable Gage R&R results - % Study (P/TV - Precision to Total Variation) and % Tolerance (P/T - Precision to Tolerance) along with a representative graphical illustration to help visualize the results and any required action to improve the Measurement System. Also discussed is the effect of the GR&R on Cp. – There are an infinite number of GR&R results(combinations of % Study and % Tolerance) use these four relatively extreme scenarios to help you determine what actions that you need take given your own results. Remember we are looking for GR&R results of < 10%, although anything less than 30% is considered barely acceptable (proceed with caution). – These graphs are not drawn to scale, therefore, when reviewing this information do not compare the relative size of the histograms between the scenarios, rather, compare the histograms within the scenario to the Spec Limits. Actual data was not used to create these histograms. – These examples assume 10 parts were selected that represent the long-term capability of the process being investigated. Three operators, 2 trial. – No assumptions have been made as to the problem with the Measurement System. – Actual data was not used to calculate the Cp indices. They were visually estimated, but are assumed reasonable. © 2001 Six Sigma Academy 94
    • 95. Scenario #1 15% - % Study 15% - % Tolerance LSL USL Tolerance Observed (Total Variation) Part Contribution (Part Variation) 5 0 6 0 7 0 8 0 9 0 Gage Contribution (Precision) © 2001 Six Sigma Academy In this example we observe a GR&R result that is acceptable, where the % Study Variation is the same as the % Tolerance Variation. The results are the same because the relative size of the Total Variation -PV (5.15*sTotal) and the Tolerance- T (USL - LSL) are the same. Therefore, when we take the P/TV or P/T ratio, where P is the Precision of the Gage (5.15* sms) it is well below 30%. This gage is deemed acceptable, no action is required. The only action is to improve the Process Capability. Furthermore, the observed Cp of this process is probably close to 1, as it appears 6 standard deviations of the process can fit inside the tolerance once. Finally, as a result of the acceptable GR&R values the observed Cp (what we measure) is considered to be the actual Cp. 95
    • 96. Scenario # 2 70% - % Study 70% - % Tolerance USL LSL Tolerance Observed (Total Variation) In this example we observe a GR&R where the % Study Variation is the same as the % Tolerance Variation, however the results are extremely unacceptable. The results are the same because the relative size of the Total Variation -TV (5.15*sTotal) and the Tolerance- T (USL - LSL) are the same. Therefore, when we take the P/TV or P/T ratio, where P is the gage contribution (5.15* sms) it is very much above 30%. Thus, indicating the Measurement System can not effectively discern part to part differences. The impact of a poor GR&R results is to inflate the variability of the product standard deviation. Part Contribution In this example we absolutely need to fix the (Part Variation) Measurement System!!! 5 0 6 0 7 0 8 0 9 0 Gage Contribution (Precision) © 2001 Six Sigma Academy Finally the observed Cp of this process (using this poor gage) is probably close to 0.5, as it appears that only half of the 6 standard deviations of the process can fit inside the tolerance. The actual Cp is probably much higher maybe closer to 1 or 1.5. If the measurement system were improved and deemed acceptable the observed Cp would reflect actual Cp. 96
    • 97. Scenario #3 70% - % Study 5% - % Tolerance LSL USL Tolerance Here we observe a GR&R where the % Study Variation is extremely unacceptable and the % Tolerance Variation is very acceptable. How can this be? In this example the Gage Precision - P (5.15* sms) compared to the Total Variation - TV (5.15*sTotal) P/TV is quite large - 70%. However, when we compare the Gage Precision with to the Tolerance (USL - LSL) P/T we observe a very acceptable GR&R - 5%. Observed (Total Variation) Do we need to fix our Measurement System? Well that depends, if we are still looking for process improvement then we should fix the measurement system. If, however, we do not need to improve the Part Contribution process capability then our measurement system is (Part Variation) acceptable. 5 0 6 0 7 0 8 0 9 0 Gage Contribution (Precision) © 2001 Six Sigma Academy In this example our observed Cp is probably close to 2 (99.73% of our process variability close can fit into our customer tolerance), where as the actual Cp may be significantly higher. If for some reason the PV began to increase to the size of the Tolerance then we would observe our gage as acceptable. 97
    • 98. Scenario #4 5% - % Study 70% - % Tolerance LSL USL Tolerance Observed (Total Variation) Part Contribution (Part Variation) 5 0 6 0 7 0 8 0 9 0 Gage Contribution (Precision) © 2001 Six Sigma Academy Here we observe a GR&R where the % Study Variation is acceptable and the % Tolerance Variation is very unacceptable. How can this be? In this example the Gage Precision - P (5.15* sms) compared to the Total Variation - PV (5.15*sTotal) P/TV is very small - 5%. However, when we compare the Gage Precision with to the Tolerance (USL - LSL) P/T we observe a very large GR&R 70%%. Do we need to fix our Measurement System? Yes, we need to fix the measurement system. In this example, the observed Cp will be the actual Cp and it is probably about 0.2 to 0.4. However, as we work our Six Sigma project and reduce the variability of our KPOV to improve our Process Capability our % Study Variation will become worse (% Tolerance, will remain constant). When our Process Variation is the same size as the Tolerance, both GR&R’s will be 70% and our observed Cp will not reflect the actual. Therefore improvement of the measurement system is required. 98