These slides are part of the ESTIEM Lean Six Sigma Green Belt course which also includes 13hr of video by Gregory H. Watson. For more info about the ESTIEM Lean Six Sigma Green Belt course visit https://internal.estiem.org/leansixsigma

1. European Students of Industrial Engineering and Management
www.estiem.org
Measurement system analysis
for publication
1

2. European Students of Industrial Engineering and Management
www.estiem.org
It’s Feedback time!
2
Persons giving feedback
Link to the feedback form
(can be found on Schoology and
WhatsApp)
Instructor to give feedback toPersons giving feedback
INSTRUCTOR 1
INSTRUCTOR 2

3. European Students of Industrial Engineering and Management
www.estiem.org
What do you remember about DMAIC?
3
Project
Charter
SIPOC
EDA
VA vs.
NVA
DEFINE
Fishbone
Diagram
Deployment
Diagram
Identifying
the BN
MEASURE
PPA
MSA

4. European Students of Industrial Engineering and Management
www.estiem.org
Remember the bolts… Let’s measure them!
Bolt needs to be exact size in order to fit the nut
• Above 605 → doesn’t fit in
• Below 595 → doesn’t hold tight

5. European Students of Industrial Engineering and Management
www.estiem.org
What kind of problems might we see
in the measurement systems?
5

6. European Students of Industrial Engineering and Management
www.estiem.org
How can measurement problems develop?
- and at what points?
6
Measure this with this by this operator

7. European Students of Industrial Engineering and Management
www.estiem.org
How about now?
7
Measure this with these by these people

8. European Students of Industrial Engineering and Management
www.estiem.org
▪ Biased measurements
▪ Difference between the measurements of different persons
▪ Difference between the measurements of a same person
▪ Unclear criteria for labeling items as “good” or “bad”
▪ Too low resolution in the measurement system
▪ …
Examples of measurement system problems,
many more exist!
8

9. European Students of Industrial Engineering and Management
www.estiem.org
What could you do as a Green Belt
to check if the measurement system
is good enough?
9

10. European Students of Industrial Engineering and Management
www.estiem.org
Sequence of Measurement System Analysis
10
Operational definitions
Do Attribute
Agreement
Analysis
Determine
Measurement
Capability
Conduct
Gage Analysis
On Factors
A B C

11. European Students of Industrial Engineering and Management
www.estiem.org
Sequence of Measurement System Analysis
11
Operational definitions
Do Attribute
Agreement
Analysis
Determine
Measurement
Capability
Conduct
Gage Analysis
On Factors
A B C

12. European Students of Industrial Engineering and Management
www.estiem.org
Let’s start with the base,
the operational definitions
12

13. European Students of Industrial Engineering and Management
www.estiem.org
Operational definitions describe
what a certain measurement means, and
how it is performed.
13

14. European Students of Industrial Engineering and Management
www.estiem.org
What kind of problems there might be with
the following operational definition?
14

15. European Students of Industrial Engineering and Management
www.estiem.org
• MEASURE: Product lead time
• Measured as the difference in time between the order and the
shipping date
What might be wrong with the following
operational definition?
15

16. European Students of Industrial Engineering and Management
www.estiem.org
• MEASURE: Product lead time
• Measured as the difference in time between the order and the
shipping date
Possible problems:
• What is the customer order time – might be the actual order
placement date, or when the order arrived in the factory?
• When is the product shipped? When it exits the factory? When it exits
the distribution center to the customer?
What might be wrong with the following
operational definition?
16

17. European Students of Industrial Engineering and Management
www.estiem.org
• CHECK: The catapult cup
must be
• A circle
• Diameter of 7-9mm
• Measured with a ruler
How could you improve the following
operational definition?
17

18. European Students of Industrial Engineering and Management
www.estiem.org
• CHECK: The catapult cup
must be
• A circle
• Diameter of 7-9mm
• Measured with a ruler
How could you improve the following
operational definition?
18
• CHECK: The catapult cup
must be
• Should be a circle
• Difference between min and
max radius from the center of
the cup +/- 0.5mm
• Diameter:
• Min diameter > 7mm
• Max diameter < 9mm
• You could add a picture on
how to measure!

19. European Students of Industrial Engineering and Management
www.estiem.org
Any questions about the
operational definitions?
19

20. European Students of Industrial Engineering and Management
www.estiem.org
Time to work…
20

21. European Students of Industrial Engineering and Management
www.estiem.org
SPONSOR MESSAGE
21

22. European Students of Industrial Engineering and Management
www.estiem.org
Sequence of Measurement System Analysis
22
Operational definitions
Do Attribute
Agreement
Analysis
Determine
Measurement
Capability
Conduct
Gage Analysis
On Factors
A B C

23. European Students of Industrial Engineering and Management
www.estiem.org
Discuss with a pair,
what is the Attribute Agreement Analysis?
23

24. European Students of Industrial Engineering and Management
www.estiem.org
Attribute Agreement Analysis tests,
if EVERY PERSON draws the
SAME CONCLUSION of the product
EVERY TIME
E.g. every person always labels the product
as a defect or good product
24

25. European Students of Industrial Engineering and Management
www.estiem.org
What could be processes where
Attribute Agreement Analysis
could be useful?
25

26. European Students of Industrial Engineering and Management
www.estiem.org
Attribute Agreement Analysis can be used e.g. in:
•Do all measurers spot the defective products?
•Do all persons label the invoices as “ready to be sent”
or “needs to be reworked”?
•Should the scratch on a car be repaired or not?
26

27. European Students of Industrial Engineering and Management
www.estiem.org
For measurement studies you have to collect
the DATA IN THE RIGHT FORMAT!
Discuss what are the requirements for
collecting data for the
Attribute Agreement Analysis
27

28. European Students of Industrial Engineering and Management
www.estiem.org
EVERY PERSON should
measure EACH PART
the SAME NUMBER OF TIMES!
28

29. European Students of Industrial Engineering and Management
www.estiem.org
Every measurer should measure each part
the same number of times!
29
Person A Person B Person C
Part A
Part A
Part B
Part B
Part C
Part C

30. European Students of Industrial Engineering and Management
www.estiem.org
Person A Person B Person C
Part A GOOD BAD GOOD
Part A GOOD GOOD GOOD
Part B GOOD GOOD GOOD
Part B BAD GOOD BAD
Part C GOOD GOOD GOOD
Part C GOOD GOOD GOOD
What can you observe from the results?
30

31. European Students of Industrial Engineering and Management
www.estiem.org
Person A Person B Person C
Part A GOOD BAD GOOD
Part A GOOD GOOD GOOD
Part B GOOD GOOD GOOD
Part B BAD GOOD BAD
Part C GOOD GOOD GOOD
Part C GOOD GOOD GOOD
What can you observe from the results?
31
Reproducibility error

32. European Students of Industrial Engineering and Management
www.estiem.org
Person A Person B Person C
Part A GOOD BAD GOOD
Part A GOOD GOOD GOOD
Part B GOOD GOOD GOOD
Part B BAD GOOD BAD
Part C GOOD GOOD GOOD
Part C GOOD GOOD GOOD
What can you observe from the results?
32
Repeatability error

33. European Students of Industrial Engineering and Management
www.estiem.org
Flash quiz:
What is a common improvement to do if the
attribute agreement analysis shows
differences between the measurers?
33

34. European Students of Industrial Engineering and Management
www.estiem.org
Time to work…
34

35. European Students of Industrial Engineering and Management
www.estiem.org
SPONSOR MESSAGE
35

36. European Students of Industrial Engineering and Management
www.estiem.org
Sequence of Measurement System Analysis
36
Operational definitions
Do Attribute
Agreement
Analysis
Determine
Measurement
Capability
Conduct
Gage Analysis
On Factors
A B C

37. European Students of Industrial Engineering and Management
www.estiem.org
Possible types of measurement
capability erros
37
Linearity Stability Calibration
Bias Discrimination

38. European Students of Industrial Engineering and Management
www.estiem.org
Measurement capability can be checked in various
ways, but the basic concepts are important
38
Linearity Stability Calibration
Bias Discrimination

39. European Students of Industrial Engineering and Management
www.estiem.org
Measurement capability can be checked in various
ways, but the basic concepts are important
39
Linearity Stability Calibration
Bias Discrimination

40. European Students of Industrial Engineering and Management
www.estiem.org
Measurement capability can be checked in various
ways, but the basic concepts are important
40
Linearity Stability Calibration
Bias Discrimination

41. European Students of Industrial Engineering and Management
www.estiem.org
Measurement capability can be checked in various
ways, but the basic concepts are important
41
Linearity Stability Calibration
Bias Discrimination

42. European Students of Industrial Engineering and Management
www.estiem.org
Measurement capability can be checked in various
ways, but the basic concepts are important
42
Linearity Stability Calibration
Bias Discrimination

43. European Students of Industrial Engineering and Management
www.estiem.org
Any questions / are these clear for you?
43
Linearity Stability Calibration
Bias Discrimination

44. European Students of Industrial Engineering and Management
www.estiem.org
Sequence of Measurement System Analysis
44
Operational definitions
Do Attribute
Agreement
Analysis
Determine
Measurement
Capability
Conduct
Gage Analysis
On Factors
A B C

45. European Students of Industrial Engineering and Management
www.estiem.org
Types of Measurement System Analyses
45
Attribute
Agreement
Analysis
Attribute Gage
R&R Study
Variable Gage
R&R (Nested)
Variable Gage
R&R (Crossed)
Gage Bias and
Linearity Study
Gage Run Chart

46. European Students of Industrial Engineering and Management
www.estiem.org
Types of Measurement System Analyses
46
Attribute
Agreement
Analysis
Attribute Gage
R&R Study
Variable Gage
R&R (Nested)
Variable Gage
R&R (Crossed)
Gage Bias and
Linearity Study
Gage Run Chart

47. European Students of Industrial Engineering and Management
www.estiem.org
Gage R&R stands for a Gage study for
Repeatability and Reproducibility
47
Repeatability Reproducibility
SAME PART & SAME MEASURER:
Are the results consistent?
SAME PART & DIFFERENT MEASURER:
Are the results consistent?
Person A Person B Person C
Part A 0,85 1,05 1,02
Part A 0,92 1,02 1,04
Part B 1,06 1,32 1,22
Part B 1,03 1,13 1,19
Part C 0,53 0,63 0,98
Part C 0,65 0,62 0,97
Person A Person B Person C
Part A 0,85 1,05 1,02
Part A 0,92 1,02 1,04
Part B 1,06 1,32 1,22
Part B 1,03 1,13 1,19
Part C 0,53 0,63 0,98
Part C 0,65 0,62 0,97

48. European Students of Industrial Engineering and Management
www.estiem.org
Crossed designs is the simplest design for
Gage R&R Analysis
48
Operator Part Measurement
A 1
A 1
B 1
B 1
C 1
C 1
A 2
A 2
Etc. Etc.
Remember from the
Attribute Agreement
Analysis?
EVERY OPERATOR should
measure EACH PART the
SAME NUMBER OF TIMES!

49. European Students of Industrial Engineering and Management
www.estiem.org

50. European Students of Industrial Engineering and Management
www.estiem.org
Left side of Gage R&R describes
how the process is performing overall
50
Variation from
real differences between parts
or measurement errors?
Are there clear outliers in the
measurements?
Is the measurement system
capable to detect differences?

51. European Students of Industrial Engineering and Management
www.estiem.org
Variation from real differences between parts
or measurement errors?
51
We want to see all part-to-part

52. European Students of Industrial Engineering and Management
www.estiem.org
Are there clear outliers in the
measurements?
52
We want to see all points down
Blue point = Difference between the MAX and MIN measurement
Control limits = 3 standard deviations up from the center line

53. European Students of Industrial Engineering and Management
www.estiem.org
Measurement system capable to detect
differences? The number of distinct categories!
53
4 distinct categoriesWe want to see at least 5
Black point = Average of one operator measurements of a part
Control limits = Calculated using the repeatability error estimate

54. European Students of Industrial Engineering and Management
www.estiem.org
Left side of Gage R&R describes
how the process is performing overall
54
Variation from
real differences between parts
or measurement errors?
Are there clear outliers in the
measurements?
Is the measurement system
capable to detect differences?

55. European Students of Industrial Engineering and Management
www.estiem.org
The right side of the Gage R&R helps you to
guide your inquiry further
55
Are there differences between
measurements of the same part?
Are the operators similar in their
distributions?
Is there a consistent bias in the
operator measurements?

56. European Students of Industrial Engineering and Management
www.estiem.org
Response by part shows if some parts have
more variance in measurements than others
56
We want to see the dots close together

57. European Students of Industrial Engineering and Management
www.estiem.org
In Response by Operator, you want to see a
straight line in the operator means
57
Straight line, good reproducibility

58. European Students of Industrial Engineering and Management
www.estiem.org
Boxplots and points can be different
illustrations of data …
58
Points used when there are <10 data
points
Boxplots used when there are >9
data points

59. European Students of Industrial Engineering and Management
www.estiem.org
In Part * Operator you look for continuous biases
– is some operator constantly on top/bottom
59
No constantly lower/higher values

60. European Students of Industrial Engineering and Management
www.estiem.org
The right side of the Gage R&R helps you to
guide your inquiry further
60
Are there differences between
measurements of the same part?
Are the operators similar in their
distributions?
Is there a consistent bias in the
operator measurements?

61. European Students of Industrial Engineering and Management
www.estiem.org
Now it’s time to practice interpreting
the charts in groups
61

62. European Students of Industrial Engineering and Management
www.estiem.org

63. European Students of Industrial Engineering and Management
www.estiem.org

64. European Students of Industrial Engineering and Management
www.estiem.org
Extra round
64

65. European Students of Industrial Engineering and Management
www.estiem.org

66. European Students of Industrial Engineering and Management
www.estiem.org

67. European Students of Industrial Engineering and Management
www.estiem.org
Final round
67

68. European Students of Industrial Engineering and Management
www.estiem.org

69. European Students of Industrial Engineering and Management
www.estiem.org

70. European Students of Industrial Engineering and Management
www.estiem.org
Question 1/3:
What kind of measurement system problems
might not be seen with the Gage R&R study?

71. European Students of Industrial Engineering and Management
www.estiem.org
Question 2/3:
Describe a practical example where could
you use Gage R&R study

72. European Students of Industrial Engineering and Management
www.estiem.org
Question 3/3:
If Gage R&R study deems the measurement
system as incapable, what kind of options
you have as a Green Belt?

73. European Students of Industrial Engineering and Management
www.estiem.org
Time to work…
73

74. European Students of Industrial Engineering and Management
www.estiem.org
SPONSOR MESSAGE
74

75. European Students of Industrial Engineering and Management
www.estiem.org
It’s Feedback time!
75
Persons giving feedback
Link to the feedback form
(can be found on Schoology and
WhatsApp)
Instructor to give feedback toPersons giving feedback
INSTRUCTOR 1
INSTRUCTOR 2

76. European Students of Industrial Engineering and Management
www.estiem.org
It’s Feedback time!
Feedback box
76

77. European Students of Industrial Engineering and Management
www.estiem.org
Usage license of the
materials
https://creativecommons.org/licenses/by-nc/4.0/legalcode
Contact leansixsigma@estiem.org
for commercial usage of the materials
Materials of the trainings have been created based on the materials provided by
ESTIEM Summer Academy Professor Gregory H. Watson