Six sigma-in-measurement-systems-evaluating-the-hidden-factory (2)

Six Sigma in Measurement Systems:
Evaluating the Hidden Factory
Inputs Operation Inspect First Time
slide 1
Rework
Hidden Factory
Scrap
NOT
OK
Correct
OK
Time, cost, people
Bill Rodebaugh
Director, Six Sigma
GRACE

Objectives
 The Hidden Factory Concept
 What is a Hidden Factory?
 What is a Measurement System’s Role in the Hidden
slide 2
Factory?
 Review Key Measurement System metrics including
%GR&R and P/T ratio
 Case Study at W. R. GRACE
 Measurement Study Set-up and Minitab Analysis
 Linkage to Process
 Benefits of an Improved Measurement System
 How to Improve Measurement Systems in an
Organization

The Hidden Factory -- Process/Production
Inputs Operation Inspect First Time
slide 3
Rework
Hidden Factory
Scrap
NOT
OK
Correct
OK
Time, cost, people
•What Comprises the Hidden Factory in a Process/Production Area?
•Reprocessed and Scrap materials -- First time out of spec, not reworkable
•Over-processed materials -- Run higher than target with higher
than needed utilities or reagents
•Over-analyzed materials -- High Capability, but multiple in-process
samples are run, improper SPC leading to over-control

The Hidden Factory -- Measurement Systems
slide 4
Re-test
Hidden Factory
Waste
OK
NOT
OK
Sample Lab Work
Inputs
Inspect Production
Time, cost, people
•What Comprises the Hidden Factory in a Laboratory Setting?
•Incapable Measurement Systems -- purchased, but are unusable
due to high repeatability variation and poor discrimination
•Repetitive Analysis -- Test that runs with repeats to improve known
variation or to unsuccessfully deal with overwhelming sampling issues
•Laboratory “Noise” Issues -- Lab Tech to Lab Tech Variation, Shift to
Shift Variation, Machine to Machine Variation, Lab to Lab Variation

The Hidden Factory Linkage
 Production Environments generally rely upon in-process
sampling for adjustment
 As Processes attain Six Sigma performance they begin
to rely less on sampling and more upon leveraging the
few influential X variables
 The few influential X variables are determined largely
through multi-vari studies and Design of
Experimentation (DOE)
 Good multi-vari and DOE results are based upon
acceptable measurement analysis
slide 5

Objectives
slide 6
Factory?
%GR&R and P/T ratio
Organization

Possible Sources of Process Variation
Measurement Variation
Variation due
to gage
2  Pr  Pr 
2
  Re  Re
We will look at “repeatability” and “reproducibility” as primary
contributors to measurement error
slide 7
Stability Linearity
Long-term
Process Variation
Short-term
Process Variation
Variation
w/i sample
Actual Process Variation
Repeatability Calibration
Variation due
to operators
Observed Process Variation
Measuremen t System
2
Actua l ocess
2
Observed ocess
producibility
2
peatability
2
Measuremen t System

How Does Measurement Error Appear?
slide 8
30 40 50 60 70 80 90 100 110
15
10
5
0
Observ ed
Frequency
LSL USL
Actual process variation -
No measurement error
Observed process
variation -
With measurement error
30 40 50 60 70 80 90 100 110
15
10
5
0
Process
Frequency
LSL USL

Measurement System Terminology
 Discrimination - Smallest detectable increment between two measured values
slide 9
 Accuracy related terms
 True value - Theoretically correct value
 Bias - Difference between the average value of all measurements of a sample and the
true value for that sample
 Precision related terms
 Repeatability - Variability inherent in the measurement system under constant
conditions
 Reproducibility - Variability among measurements made under different conditions
(e.g. different operators, measuring devices, etc.)
 Stability - distribution of measurements that remains constant and predictable over time for
both the mean and standard deviation
 Linearity - A measure of any change in accuracy or precision over the range of instrument
capability

Measurement Capability Index - P/T
 Precision to Tolerance Ratio
. *
MS /
 Addresses what percent of the tolerance is taken up by
slide 10
measurement error
 Includes both repeatability and reproducibility
 Operator x Unit x Trial experiment
 Best case: 10% Acceptable: 30%
Usually expressed
as percent P T
Tolerance

515 
Note: 5.15 standard deviations accounts for 99% of Measurement System (MS) variation.
The use of 5.15 is an industry standard.

Measurement Capability Index - % GR&R
MS
 Addresses what percent of the Observed Process Variation is
taken up by measurement error
 %R&R is the best estimate of the effect of measurement
systems on the validity of process improvement studies (DOE)
 Includes both repeatability and reproducibility
 As a target, look for %R&R < 30%
slide 11
Usually expressed
as percent
R R x 100
Observed Pr
ocess Variation
% &




Objectives
slide 12
Factory?
%GR&R and P/T ratio
Organization

Case Study Background
 Internal Raw Material, A1, is necessary for Final Product production
 Expensive Raw Material to produce – produced at 4 locations Worldwide
 Cost savings can be derived directly from improved product quality, CpKs
 Internal specifications indirectly linked to financial targets for production costs are used to
slide 13
calculate CpKs
 If CTQ1 of A1 is too low, then more A1 material is added to achieve overall quality – higher
quality means less quantity is needed – this is the project objective
 High Impact Six Sigma project was chartered to improve an important quality variable,
CTQ1
 The measurement of CTQ1 was originally not questioned, but the team decided to study
the effectiveness of this measurement
 The %GR&R, P/T ratio, and Bias were studied
 Each of the Worldwide locations were involved in the study
 Initial project improvements have somewhat equalized performance across sites. Small
level improvements are masked by the measurement effectiveness of CTQ1

CTQ1 MSA Study Design (Crossed)
slide 14
Site 1 Lab
6 analyses/site/sample
2 samples taken from each site
2*4 Samples should be representative
Each site analyzes other site’s sample.
Each plant does 48 analyses
6*8*4=196 analyses
Site 1 Sample 1 Site 1 Sample 2
Op 1 Op 2 Op 3
T1 T2
Site 2 Lab Site 3 Lab Site 4 Lab
Site 2 Sample 1…..

CTQ1 MSA Study Results (Minitab Output)
UCL=52.45
UCL=851.5
slide 15
Gage name:
Date of study:
Reported by:
Tolerance:
Misc:
Z-14 MSA
JULY 2002
All Labs
110
Surface Area
120
100
80
60
40
20
100 CB1 CB2 CB3 LC1 LC2 LC3 V1 V2 V3 W1 W2 W3
50
0
850
800
750
Xbar Chart by Operator
Sample Mean
Mean=821.3
LCL=791.1
0
0
R Chart by Operator
Sample Range
R=16.05
LCL=0
1 2 3 4 5 6 7 8
1 2 3 4 5 6 7 8
890
840
790
890
840
790
900
850
800
Sample
Operator
Operator*Sample Interaction
Average
CB1
CB2
CB3
LC1
LC2
LC3
V1
V2
V3
W1
W2
CB1 CB2 CB3 LC1 LC2 LC3 V1 V2 V3 W1 W2 W3
740
Oper
Response By Operator
740
Sample
Response By Sample
%Contribution
%Study Var
%Tolerance
Gage R&R Repeat Reprod Part-to-Part
0
Components of Variation
Percent

CTQ1 MSA Study Results (Minitab Session)
Source DF SS MS F P
Sample 7 14221 2031.62 5.0079 0.00010
Operator 11 53474 4861.27 11.9829 0.00000
Operator*Sample 77 31238 405.68 1.4907 0.03177
Repeatability 96 26125 272.14
slide 16
Total 191 125058
%Contribution
Source VarComp (of VarComp)
Total Gage R&R 617.39 90.11
Repeatability 272.14 39.72
Reproducibility 345.25 50.39
Operator 278.47 40.65
Operator*Sample 66.77 9.75
Part-To-Part 67.75 9.89
Sample, Operator,
& Interaction are
Significant

CTQ1 MSA Study Results
slide 17
Site %GRR
P/T
Ratio
R-bar
Equal Variances
within Groups
Mean
Differences
(Tukey Comp.)
All
94.3
(78.6 – 100)*
116 16.05 No (0.004) Only 1,2 No Diff.
Site 1
38.9
(30.0 – 47.6)
29 7.22 Yes (0.739) All Pairs No Diff.
Site 2
91.0
(70.7 – 100)
96 17.92 Yes (0.735) Only 1,2 Diff.
Site 3
80.0
(60.8 – 94.8)
79 20.37 Yes (0.158) All Pairs No Diff.
Site 4
98.0
(64.8 – 100)
120 18.67 Yes (0.346) Only 2,3 No Diff.
*Conf Int not calculated with Minitab, Based upon R&R Std Dev

Dotplot of All Samples over All Sites
slide 18
WO SA
VF SA
LC SA
CB SA
890
840
790
740
C17
C16
Dotplots of C16 by C17
(group means are indicated by lines)
Site 1 Site 2 Site 3 Site 4

CTQ1 MSA Study Results (Minitab Session)
slide 19
Analysis of Variance for Site
Source DF SS MS F P
Site 3 37514 12505 26.86 0.000
Error 188 87518 466
Total 191 125032
Individual 95% CIs For Mean
Based on Pooled StDev
Level N Mean StDev -+---------+---------+---------+-----
Site 1 48 824.57 15.38 (---*---)
Site 2 48 819.42 22.11 (---*---)
Site 3 48 800.98 20.75 (---*---)
Site 4 48 840.13 26.58 (---*---)
-+---------+---------+---------+-----
Pooled StDev = 21.58 795 810 825 840
Site and Operator are closely related

790
60
40
20
X-bar R of All Samples for All Sites
50
slide 20
0
850
800
750
Xbar Chart by Operator
Sample Mean
UCL=52.45
UCL=851.5
Mean=821.3
LCL=791.1
0
0
R Chart by Operator
Sample Range
R=16.05
LCL=0
1 2 740
Sample
890
Discrimination
Index 840
is “0”,
however can
790
probably see
differences of 5
900
850
1 800
Sample
Operator*Average
CB1 CB2 CB3 740
Oper
0
Percent
Most of the
samples are
seen as “noise”

50
70 W1 W2 W3
60
50
40
30
20
10
900 W1 W2 W3
•Mean differences are seen in X-bar area
•Most of the samples are seen as “noise”
slide 21
0
850
800
Xbar Chart by WO OP
Sample Mean
UCL=60.99
UCL=875.2
Mean=840.1
LCL=805.0
0
0
R Chart by WO OP
Sample Range
R=18.67
LCL=0
0
Percent
X-bar R of All Samples for Site 4

810
CTQ1 MSA Study Results – Process Linkage
Site 2 Example
860 LC1 LC2 LC3
2
6662
2
22
2
slide 22
0
850
840
830
820
810
800
790
780
Xbar Chart by LC OP
Sample Mean
UCL=853.1
Mean=819.4
LCL=785.7
0
0
Sample R=17.92
LCL=0
LC OP*Sample 850
840
Average
830
820
810
800
MSA Study
Results with
Mean = 819.4
1 2 3 790
Sample
LC1 760
LC OP
1000
900
800
700
Individual Value
1
1
6
1
6
1
4
222 4
6
1
1
2
1
5
1 1
6
1
1
66
222
2
55
Subgroup 0 100 200 300 400
UCL=899.2
Mean=832.5
LCL=765.8
150
100
50
Moving Range
1
1
1
1
11
1
1
1
1
1
1
1
UCL=81.95
I and MR Chart for TSA (t)
2002 Historical
Process
Results with
Mean = 832.5
Selected Samples are Representative

CTQ1 MSA Study Results 810
– Process Linkage
Site 2 Example
2
6662
2
22
UCL=58.54
UCL=853.1
2
66
222
slide 23
50
1000
1
1
1
1
1
1
1 1
100 LC1 LC2 LC3
900
800
50
700
6
6
4
222 4
6
1
2
5
6
1
1
860 LC1 LC2 LC3
0
850
840
830
820
810
800
790
780
Xbar Chart by LC OP
Individual Value
Sample Mean
Mean=819.4
LCL=785.7
0
0
R Chart by LC OP
Sample Range
R=17.92
LCL=0
1 2 3 4 5 6 7 8
UCL=899.2
MSA Study Results
with Range = 17.92,
Calc for Subgroup
UCL=81.95
1 2 3 4 5 6 7 8
55
860
810
850
840
830
820
810
800
790
Sample
LC OP
LC OP*Sample Interaction
2
Average
LC1
LC2
LC3
LC1 LC2 LC3
760
LC OP
By LC OP
760
Sample
%Tolerance
0
Percent
1
Subgroup 0 100 200 300 400
Mean=832.5
LCL=765.8
150
100
50
0
Moving Range
1
22
1
2
222
2
1
1
11
1
1
1
1
1
2
2
1
2
2
R=25.08
LCL=0
I and MR Chart for TSA (t)
2002 Historical
Process
Results with
Range = 25.08
Calc for pt to pt
When comparing the MSA with process operation, a large
percentage of pt-to-pt variation is MS error (70%) --- a
back check of proper test sample selection

Site 2 Example
 Key issue for Process Improvement Efforts is “When will we see
change?”
 Initial Improvements to A1 process were made
 Control Plan Improvements to A1 process were initiated
 Site 2 Baseline Values were higher than other sites
 Small step changes in mean and reduction in variation will achieve goal
 How can Site 2 see small, real change with a Measurement System with
Use Power and Sample Size Calculator with and without impact
of MS variation. Lack of clarity in process improvement work,
results in missed opportunity for improvement and continued
use of non-optimal parameters
slide 24
70+% GR&R?

Site 2 Example
Simulated Reduction of Pt to Pt variation by 70% decreases
time to observe savings by over 9X.
slide 25
2-Sample t Test
Alpha = 0.05 Sigma = 22.23
Sample Target Actual
Difference Size Power Power
2 2117 0.9000 0.9000
4 530 0.9000 0.9002
6 236 0.9000 0.9002
8 133 0.9000 0.9001
10 86 0.9000 0.9020
12 60 0.9000 0.9023
14 44 0.9000 0.9007
16 34 0.9000 0.9018
18 27 0.9000 0.9017
20 22 0.9000 0.9016
2-Sample t Test
Alpha = 0.05 Sigma = 6.67
Sample Target Actual
Difference Size Power Power
2 192 0.9000 0.9011
4 49 0.9000 0.9036
6 22 0.9000 0.9015
8 13 0.9000 0.9074
10 9 0.9000 0.9188
12 7 0.9000 0.9361
14 5 0.9000 0.9156
16 4 0.9000 0.9091
18 4 0.9000 0.9555
20 3 0.9000 0.9095

Site 2 Example
Benefits of An Improved MS
 Realized Savings for a Process Improvement Effort
 For A1, an increase of 1 number of CTQ1 is approximately $1 per ton
 Change of 10 numbers, 1000 Tons produced in 1 month (832  842)
 $1 * 10 * 1000 = $10,000
 More trust in all laboratory numbers for CTQ1
 Ability to make process changes earlier with R-bar at 6.67
 Previously, it would be pointless to make any process changes within the 22 point
range. Would you really see the change?
 As the Six Sigma team pushes the CTQ1 value higher, DOEs and other
slide 26
tools will have greater benefit

Objectives
slide 27
Factory?
%GR&R and P/T ratio
Organization

Measurement Improvement in the Organization
 Initial efforts for MS improvement are driven on a BB/GB project basis
 Six Sigma Black Belts and Green Belts Perform MSAs during Project Work
 Lab Managers and Technicians are Part of Six Sigma Teams
 Measurement Systems are Improved as Six Sigma Projects are Completed
 Intermediate efforts have general Operations training for lab personnel,
mostly laboratory management
 Lab efficiency and machine set-up projects are started
 The %GR&R concept has not reached the technician level
 Current efforts enhance technician level knowledge and dramatically
increase the number of MS projects
 MS Task Force initiated (3 BBs lead effort)
 Develop Six Sigma Analytical GB training
 All MS projects are chartered and reviewed; All students have a project
 Division-wide database of all MS results is implemented
slide 28

Measurement Improvement in the Organization
 Develop common methodology for Analytical GB training
Six Sigma Step Action Typical Six Sigma Tools Used
Define  Target measurement
slide 29
system for study
 Identify KPOVs
Project Charter
Measure  Identify KPIVs
 Evaluate KPOV
performance
“Soft” tools: Process Map, Cause & Effect
Matrix, FMEA
“Stat” tools: Minitab Graphics, SPC,
Capability Analysis
Analyze  Measurement System
Analysis
Gage R&R, ANOVA, Variance Components,
Regression, Graphical Interpretation
Improve  Reduce Reproducibility
 Reduce Repeatability
 Reduce Operator or
Instrument Bias
“Soft” tools: Fishbone Diagram, Focused
FMEA
“Stat” tools: D-Study, t-Tests and
Regression, Design of Experiments
Control  Final Report
 Control Plan for KPIVs
SPC, Reaction Plans, Control Plans, ISO
synergy, Mistake Proofing

Final Thoughts
 The Hidden Factory is explored throughout all Six Sigma programs
 One area of the Hidden Factory in Production Environments is
slide 30
Measurement Systems
 Simply utilizing Operations Black Belts and Green Belts to improve
Measurement Systems on a project by project basis is not the long term
answer
 The GRACE Six Sigma organization is driving Measurement System
Improvement through:
 Tailored training to Analytical Resources
 Similar Six Sigma review and project protocol
 Communication to the entire organization regarding Measurement System
performance
 As in the case study, attaching business/cost implications to poorly performing
measurement systems

Six sigma-in-measurement-systems-evaluating-the-hidden-factory (2)

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (9)

Similar to Six sigma-in-measurement-systems-evaluating-the-hidden-factory (2)

Similar to Six sigma-in-measurement-systems-evaluating-the-hidden-factory (2) (20)

Recently uploaded

Recently uploaded (20)

Six sigma-in-measurement-systems-evaluating-the-hidden-factory (2)