Teaching Six Sigma Using Six Sigma a
DMAIC Approach
Brandon Theiss
Brandon.Theiss@gmail.com
2013 ASQ WCQI Session (M30)

About Me• Academics
– MS Industrial Engineering Rutgers University (hopefully)
– BS Electrical & Computer Engineering Rutgers University
– BA Physics Rutgers University
• Awards
– ASQ Top 40 Leader in Quality Under 40
– ASQ National Education Quality Excellence Award Finalist
– IIE Early Career Achievement Award Winner 2013
• Professional
– Principal Industrial Engineer -Medrtonic
– Master Black belt- American Standard Brands
– Systems Engineer- Johnson Scale Co
• Certifications
– ASQ Certified Manager of Quality/ Org Excellence Cert # 13788
– ASQ Certified Quality Auditor Cert # 41232
– ASQ Certified Quality Engineer Cert # 56176
– ASQ Certified Reliability Engineer Cert #7203
– ASQ Certified Six Sigma Green Belt Cert # 3962
– ASQ Certified Six Sigma Black Belt Cert # 9641
– ASQ Certified Software Quality Engineer Cert # 4941
– Licensed to practice before United States Patent and Trademark Office
• Publications
– Going with the Flow- The importance of collecting data without holding up your processes- Quality Progress March
2011
– "Numbers Are Not Enough: Improved Manufacturing Comes From Using Quality Data the Right Way" (cover story).
Industrial Engineering Magazine- Journal of the Institute of Industrial Engineers September (2011): 28-33. Print

Learning Objectives
• Apply Six Sigma to the Teaching of Six Sigma
• Create Practitioner Academic Partnerships
• Uniquely Apply SPC Charts
• Use Statistical Hypothesis testing to improve
learning outcomes

Motivation
• Teaching the tools, techniques and Methods of Lean Six
Sigma is inherently difficult in academic setting.
• When taught in a industrial setting students have a
common motivation (the improved welfare of the
company), similar levels of education and knowledge of
domain specific information. Students are encouraged to
learn by applying the material to their daily activities.
• This is not possible in an academic setting particularly in a
mixed environment that includes everything from
undergraduate juniors through senior PhD researchers.
• In addition undergraduate students tend either lack
professional or have experience in Fields that are not
traditionally thought of as benefiting or implementing Six
Sigma (waitressing, check out clerk etc.)

5
Putting some numbers to the motivation
• Lean Six Sigma is a commonly adopted business
improvement technique which integrates, the scientific method,
statistics and defect reduction to obtain tangible results.
•Within 50 miles of Rutgers there are 2,249 active job listings
for the phrase “six sigma green belt”
•Non University Affiliated Classes are available however are
prohibitively expensive for most students ~$2,000.
•ASQ de facto industry standard for Greenbelt Certification
•Current Industrial Engineering Undergraduate and Graduate
programs do not prepare students to effectively implement the
Six Sigma toolkit.
•Salary Report indicates Certified Green belts earn $12,000
more per year

Class Demographics
• 71 Students Registered
– 57 At Student Tuition Rate ($296)
– 14 At Professional Tuition Rate ($495)
0.0%
5.0%
10.0%
15.0%
20.0%
25.0%
30.0%
35.0%
40.0%
Junior Year Senior
Year
BA/BS Some
Grdudate
MA/MS/JD PhD/PE
Highest Accademic Grade
Completed
24222018161412108642
20
15
10
5
0
Years Of Work Exprience
Frequency
3
Histogram of Years Of Work Exprience

Solution
• The beauty of the Six Sigma Methodology is that it can be applied
to any process.
• The definition of a process is quite broad and can be reduced to any
verb- noun combination.
• Therefore the collective process which the class studied and
improved was to
Pass [the]
ASQ
Certified Six Sigma Green
Belt Exam
• Therefore the foundational Six Sigma Concept of DMAIC (Define
Measure Analyze Improve Control) represents both the material
covered in the course as well as the pedagogical method used for
instruction

About the Course & Partnership
• Offered as a Non-Credit extracurricular course
at Rutgers University in Piscataway NJ
• Co-Sponsored by the Rutgers Student Chapter
of the Institute for Industrial Engineers (IIE) and
the Princeton NJ section of American Society for
Quality (ASQ)
• Open and advertised to all members of the
Rutgers Community (students, staff and faculty)
as well as the surrounding public
• Objective of the course was to train students to
pass the June 2nd 2012 administration of the
ASQ Certified Six Sigma Green Belt Exam

Course Syllabus
1. Introduction, Sample Exam
2. Review Exam, Define 1
3. Define 2, Measure 1
4. Measure 2, Measure 3
5. Measure 4, Sample 50 Question
Exam
6. Review Exam, Analyze 1
7. Analyze 2, Analyze 3
8. Improve 1, Sample 50 Question Exam
9. Review Exam, Control 1
10. Sample 100 Question Exam
11. Review Exam, Additional Questions
Define Measure Analyze Improve Control
• Project Definition
• Team Dynamics
• Brainstorming
• Process Mapping
• Measurement
Systems
• Histograms
• Box Plots
• Dot Plots
• Probability
Plots
• Control Charts
• Inferential
Statistics
• Confidence
Intervals
• Hypothesis
Tests
• Regression
Analysis
• Pareto Charts
• Process
Capability
• Lean

Pre Test
• On the first night of classes students were
given an introductory survey of Six Sigma
by means of a worked example applying
DMAIC to the Starbucks Experience from a
Customers Prospective.
• Students were then given a copy of the
Certified Six Sigma Green Belt Handbook by
Roderick A. Munro
• Then given a 50 Question Multiple Choice
Test representative of the ASQ CSSGB Exam
• The Test was administered on two
successive nights (Monday and Tuesday)

Measurement System
• An Apperson GradeMaster™ 600 Test
Scanner was utilized which enabled test to
be scored and returned immediately upon
student submission at the exam site.
• In addition all of each answer to every
question was downloaded to connected
computer enabling further detailed analysis

MONDAY RESULTS

Test Scores
84.00%72.00%60.00%48.00%36.00%
9
8
7
6
5
4
3
2
1
0
Test Scores
Frequency
Mean 0.5589
StDev 0.1177
N 35
Histogram of Test Scores
Normal

Test for Normality
1.00.90.80.70.60.50.40.30.2
99
95
90
80
70
60
50
40
30
20
10
5
1
Test Score
Percent
Mean 0.5589
StDev 0.1177
N 35
AD 0.396
P-Value 0.352
Probability Plot of Test Score
Normal - 95% CI

Is process in Control?
343128252219161310741
1.0
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
Observation
IndividualValue
_
X=0.5589
UCL=0.9468
LCL=0.1709
I Chart of Test Score

Is the Process Capable?
0.840.720.600.480.36
LSL
LSL 0.78
Target *
USL *
Sample Mean 0.558857
Sample N 35
StDev (Within) 0.120985
StDev (O v erall) 0.117718
Process Data
C p *
C PL -0.61
C PU *
C pk -0.61
Pp *
PPL -0.63
PPU *
Ppk -0.63
C pm *
O v erall C apability
Potential (Within) C apability
PPM < LSL 971428.57
PPM > USL *
PPM Total 971428.57
O bserv ed Performance
PPM < LSL 966214.72
PPM > USL *
PPM Total 966214.72
Exp. Within Performance
PPM < LSL 969849.40
PPM > USL *
PPM Total 969849.40
Exp. O v erall Performance
Within
Overall
Process Capability of Test Scores
overall standard
deviation for the
entire study
overall standard deviation for
the entire study if special
cause eliminated
based on variation
within subgroups

Are there bad questions?
464136312621161161
1.0
0.8
0.6
0.4
0.2
0.0
Sample
Proportion
_
P=0.441
UCL=0.693
LCL=0.189
1
1
1
1
1
1
11
11
P Chart of Wrong

Does the order the exams are turned in effect the
score?
3330272421181512963
0.9
0.8
0.7
0.6
0.5
0.4
0.3
Index
TestScore
MAPE 15.9381
MAD 0.0840
MSD 0.0124
Accuracy Measures
Actual
Fits
Variable
Trend Analysis Plot for Test Score
Linear Trend Model
Yt = 0.5018 + 0.00317*t

TUESDAY RESULTS

Test Scores

Test for Normality

Is the process in Control?
28252219161310741
90.00%
80.00%
70.00%
60.00%
50.00%
40.00%
30.00%
20.00%
Observation
IndividualValue
_
X=55.93%
UCL=84.62%
LCL=27.25%
1
I Chart of Scores

Is the process capable?

Are there Bad Questions?
464136312621161161
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
Sample
Proportion
_
P=0.441
UCL=0.717
LCL=0.164
1
1
1
11
1
1
1
P Chart of Incorrect

272421181512963
0.9
0.8
0.7
0.6
0.5
0.4
Index
Scores
MAPE 13.9747
MAD 0.0779
MSD 0.0100
Accuracy Measures
Actual
Fits
Variable
Trend Analysis Plot for Scores
Linear Trend Model
Yt = 0.5614 - 0.000138*t
Does the order exams are turned in
effect test scores?

COMBINED RESULTS

Combined Test Scores
0.840.720.600.480.36
20
15
10
5
0
Combined
Frequency
Mean 0.5591
StDev 0.1099
N 64
Histogram of Combined
Normal

Test Scores
0.84
0.72
0.60
0.48
0.36
9
8
7
6
5
4
3
2
1
0
84.00%
72.00%
60.00%
48.00%
36.00%
9
8
7
6
5
4
3
2
1
0
Monday
Frequency
Tuesday
Mean 0.5589
StDev 0.1177
N 35
Monday
Mean 0.5593
StDev 0.1018
N 29
Tuesday
Histogram of Monday, Tuesday
Normal

Is there a difference Between Classes?
0.9
0.8
0.7
0.6
0.5
0.4
0.3
Monday Tuesday
Boxplot of Monday, Tuesday

Is there a statistical Difference?
Anova: Single Factor
SUMMARY
Groups Count Sum Average Variance
Monday 35 19.56 0.558857 0.013857
Tuesday 29 16.22 0.55931 0.010357
ANOVA
Source of Variation SS df MS F P-value F crit
Between Groups 3.26E-06 1 3.26E-06 0.000265 0.987056 3.995887
Within Groups 0.76114 62 0.012276
Total 0.761144 63

Is the variation different?

32
464136312621161161
1.0
0.8
0.6
0.4
0.2
0.0
Sample
Proportion
_
P=0.441
UCL=0.627
LCL=0.255
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
P Chart of Wrong
What Can we See from the Out of
Control Points?

Brainstorming Techniques
• At the beginning of class students were asked as a group to
brainstorm ideas for why they failed the pre-test
– Only 4 ideas were proposed
• Students were taught the different brainstorming techniques
contained in the CSSGB Body of Knowledge
– Nominal Group Technique
– Multi-Voting
– Affinity Diagrams
– Force Field Analysis
– Tree Diagrams
– Cause and Effect Diagrams
• Students were then broken up into 6 different groups, assigned one
of the brainstorming techniques and given the task to brainstorm
why they failed the pre-test

Brainstorming Techniques Continued
• Students then presented their results to the
Group

Brainstorming Results
Cause and Effect (Fishbone)
Affinity Diagram

Brainstorming Results
Tree Diagram
Force Field Analysis

Brainstorming Results
Multi-Voting
Nominal Group Technique

Brainstorming Continued
• Students then told to return to their groups
and apply their “favorite” of the brainstorming
techniques to the task how can you Pass the
midterm exam
• Students Found the positive formulation of
the task much more challenging and most
groups stayed with the same technique they
used for the Negative version.

Team Dynamics
• The 3rd weeks lesson began with an
introduction of the Tuckman cycle of team
dynamics
• Students were asked
to reflect upon their
experience in the
brainstorming activity
to see if their
experiences paralleled
those predicted by the
model

Process Mapping
• The second portion of the 3rd Class was spent
introducing the process mapping strategies in
the CSSGB BoK
– SIPOC (Suppliers Inputs Process Outputs Customers)
– Process Mapping
– Value Stream
Mapping

Process Mapping Continued
• Students were again divided into 6 groups. Each group was assigned a map type and
told to Map the Exam Taking Process at either a Micro or Macro Level
• Micro Level Groups Handled the Physical steps of taking the exam such as reading
the question, locating the answer and filling in the bubbles
• Macro Groups Handled the all of the preparation leading up to taking the exam
• The point was to emphasize that the same tools techniques and methods can be
used on the very micro level (an operator tightening a bolt) to the very macro level
(the operations of a fortune 500 company)

42
SIPOC at a even higher level
Input
• Students
• Body of
Knowledge
• Instructor
• Textbook
• Facilities
Supplier
•ASQ Princeton
•ASQ Corporate
•Rutgers University
Output
• Knowledge
• Certification
Customers
• Future Employers
• Current Employers
• Students
• Rutgers University
• ASQ Princeton
• Rutgers IIE
Educate
Students in Six
Sigma
Process
Identify Educational
Shortcoming
Create Course
Develop
Methodology
Locate Students
Teach Students
Administer Test

Control Charts
• Class 4 Introduced Students to the Control Charts Covered in the
CSSGB BoK
– I-MR
– X Bar-R
– X Bar- S
– P
– NP
– U
– C
• Students were emailed prior to class a Microsoft Excel Workbook
containing the test results and told to bring their laptops to class
• Students were asked to do the following by hand (with Excel
helping for the calculations):
– I-MR Chart for Test Scores
– P Chart testing for “Bad Questions”
– NP Chart testing for “Bad Questions”
– C Chart for the number of wrong responses per exam
– U Chart for the number of wrong responses per exam

Control Charts Results
NP Chart
C Chart

Midterm Analysis

Midterm Exam Results

Pre Class Exam Results

Comparison

Does a T-Test Indicate there was improvement?
t-Test: Two-Sample Assuming Unequal Variances
Mid Pre
Mean 0.607234 0.561702
Variance 0.014373 0.01111
Observations 47 47
Hypothesized Mean Difference 0
df 91
t Stat 1.955429
P(T<=t) one-tail 0.0268
t Critical one-tail 1.661771
P(T<=t) two-tail 0.0536
t Critical two-tail 1.986377

Does ANOVA Indicate there was
Improvement?
Anova: Single Factor
SUMMARY
Groups Count Sum Average Variance
Pre Total 64 35.78 0.559063 0.012082
Mid Total 53 31.72 0.598491 0.013705
ANOVA
Source of Variation SS df MS F P-value F crit
Between Groups 0.045069 1 0.045069 3.516685 0.06329 3.923599
Within Groups 1.473823 115 0.012816
Total 1.518892 116

Change in Scores

Is the Change in Control?
-15
-10
-5
0
5
10
15
C Chart of Change in # of Correct Responses
UCL = 8.29
LCL = -3.74
Mid= 2.28

Is the change in Scores Significant?
t-Test: Paired Two Sample for Means
Mid Pre
Mean 0.607234043 0.561702
Variance 0.014372618 0.01111
Observations 47 47
Pearson Correlation 0.689206844
Hypothesized Mean Difference 0
df 46
t Stat 3.475995635
P(T<=t) one-tail 0.000560995
t Critical one-tail 1.678660414
P(T<=t) two-tail 0.00112199
t Critical two-tail 2.012895599

Not all Material on the Exam has been
Covered in Class

Midterm Comparison

Pre Test Comparison

Comparison of Results for Material
that has been Covered
Mid CoveredPre Covered
1.0
0.9
0.8
0.7
0.6
0.5
0.4
0.3
Subscripts
CoveredScores
Boxplot of Covered Scores

Comparison of Covered Material
0.90.80.70.60.50.40.3
12
10
8
6
4
2
0
0.90.80.70.60.50.40.3
Pre Covered
Frequency
Mid Covered
Mean 0.5785
StDev 0.1252
N 64
Pre Covered
Mean 0.6516
StDev 0.1174
N 53
Mid Covered
Histogram of Pre Covered, Mid Covered
Normal

Does ANOVA Indicate there was
improvement?
Anova: Single Factor
SUMMARY
Groups Count Sum Average Variance
Pre Covered 64 37.02632 0.578536 0.015686
Mid Covered 53 34.53333 0.651572 0.013785
ANOVA
Source of Variation SS df MS F P-value F crit
Between Groups 0.154648 1 0.154648 10.43065 0.001616 3.923599
Within Groups 1.70503 115 0.014826
Total 1.859678 116

Comparison of Results for Material
that has not been Covered
Mid Not CoveredPre Not Covered
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0.0
Subscripts
Scores
Boxplot of Scores

Comparison of Material Not Covered

Does ANOVA indicate the Exam was
harder?
Anova: Single Factor
SUMMARY
Groups Count Sum Average Variance
Pre Not Covered 64 31.83333 0.497396 0.01785
Mid Not Covered 53 27.5 0.518868 0.024926
ANOVA
Source of Variation SS df MS F P-value F crit
Between Groups 0.013367 1 0.013367 0.635003 0.427168 3.923599
Within Groups 2.420698 115 0.02105
Total 2.434065 116

Is the Exam Taking Process Capable?

Control Charts with Minitab
• Students were emailed a Microsoft Excel Workbook with the Mid-
Term data set
• It was heavily suggested that students purchase the Minitab
academic license and bring their laptops to class.
• Students then divided themselves into groups around those who
purchased the software and created the analysis control charts on
the preceding slides.

Hypothesis Testing Exercises
• In week 8 students were introduced to the hypothesis tests covered
in CSSGB BoK
– Z Test
– Student T
– Two Sample T (known variance)
– Two Sample T (unknown variance)
– Paired T Test
– ANOVA
– Chi Squared T
– F Test
• Students were emailed a data set containing both the Pre-Test and
Mid-Term data and asked to perform each of the listed test using
either Minitab or Microsoft Excel. The emphasis was placed on the
conclusions from the data

Confidence Intervals
• Not all students took the Mid-Term that took the
pre-test.
• This enabled students to utilize inferential
statistics to draw conclusions about the
population parameters (mean and variance
particularly)
• By using the class data set provided students
were able to calculate their confidence in the
overall population parameters for the average
test score as well as the standard deviation of the
entire class

Improve-Control
• Improve and Control are not an emphasis in the CSSGB BoK. For the
coverage of the material and extended example of the Starbucks
Experience from a customers perspective is presented.
• When introducing Lean and the types of Waste the process of making
various beverages are presented. Students then proposed improvement
strategies to minimize the ‘Muda’
Triple Tall Half Hot Half Cold Americano
(Current State)
Triple Tall Half Hot Half Cold Americano
(Future State)

Final Exam Analysis

Exam Scores

Doesn’t Look Normal

It’s Bi-Modal!

Did the scores Improve?

Was The Difference Significant?
Anova: Single Factor
SUMMARY
Groups Count Sum Average Variance
Pre 64 35.78 0.559063 0.012082
Mid 47 28.54 0.607234 0.014373
Final 40 30.43 0.76075 0.020084
ANOVA
Source of Variation SS df MS F P-value F crit
Between Groups 1.029282 2 0.514641 34.534 4.91E-13 3.057197
Within Groups 2.205562 148 0.014902
Total 3.234844 150

Individual Improvement
Variable N N* Mean StDev Minimum Q1 Median Q3
Change 36 0 0.1939 0.1419 -0.0600 0.0675 0.2000 0.2875

Was the Individual Improvement
Significant?
t-Test: Paired Two Sample for Means
Final Pre
Mean 0.750556 0.556667
Variance 0.019743 0.010023
Observations 36 36
Pearson Correlation 0.342582
Hypothesized Mean Difference 0
df 35
t Stat 8.199954
P(T<=t) one-tail 5.8E-10
t Critical one-tail 1.689572
P(T<=t) two-tail 1.16E-09
t Critical two-tail 2.030108

Where there Hard Questions?

Pareto Chart on Topic
Count 3 3 2 2 2 1 1 1
Percent 20.0 20.0 13.3 13.3 13.3 6.7 6.7 6.7
Cum % 20.0 40.0 53.3 66.7 80.0 86.7 93.3 100.0
Question Topic
FM
EA
Control Charts
Confidence
Interval
Team
s
Process
Capablity
Error
Hypothesis
Basic
Stats
16
14
12
10
8
6
4
2
0
100
80
60
40
20
0
Count
Percent
Pareto Chart of Question Topic

Initial Process Capability

Final Process Capability

Results
• Ruba Amarin
• Margit Barot
• Miriam Bicej
• Matthew Brown
• Salem El-Nimri
• William Ewart
• Elizabeth Fuschetti
• Robert Gaglione
• Thomas Hansen
• Tarun Jada
• Javier Jaramillo
• Michael Kagan
• Anoop Krishnamurthy
• Timothy Lin
• Helen Liou
• Rebecca Marzec
• Charles Ott
• Sneha Patil
• Eugene Reshetov
• Matthew Rodis
• Thomas Schleicher
• Dante Triana
• Albert Tseng
• Bond Wann
• Paul White
• Sun Wong
• Shih Yen
• Jacob Ziegler
28 out of 37 Students that took the June 2nd Exam Passed the June 2nd Exam
Nationally 788 out of 1160 individuals passed the exam

Was the Result Significant?
Rutgers ASQ

Results
• Students test scores improved on average
19.4%
• 76% of Students Passed the exam compared
to 68% National Average
• Increased ASQ Princeton Membership by 62
members
• Largest Ever Fund Raiser for the Rutgers IIE

83
Added Benefit
• From the funds generated by the course Rutgers was
able to send 21 Students to the national IIE
Conference in Orlando (shown above)

84
It took a team
• Nate Manco
– ASQ Princeton Education Chair
• Richard Herczeg
– ASQ Princeton Section President
• Jeff Metzler
– Rutgers IIE President
• Dr. James Luxhoj
– Rutgers Industrial and Systems Eng
• Brandon Theiss
– Instructor
• Cindy Ielmini
– Rutgers Industrial and Systems Eng

Lessons Learned
• Using the passing the exam process as a class example
for the implementation of the tools and techniques of
Six Sigma is an effective methodology
• There is demand for teaching Six Sigma in an academic
setting
• The joint venture between Rutgers and ASQ is feasible
and mutually beneficial.
• Having a diverse student population increases the
overall performance of the group.
• Students need to be adequately qualified to sit for ASQ
exam prior to taking the course.

86
We are sharing the Results
• Presented results at Institute of Industrial Engineers Lean and Six Sigma
Conference
• Will be presented at the ASQ International Conference on Quality

87
Progress continues onward
• Course Scheduled to Run again in the Spring through Official Continuing
Education Office
• First of its kind joint meeting with ASQ Princeton and Rutgers IIE in
which the course results were presented.

Questions?
• Contact info
– Brandon Theiss
– Brandon.theiss@gmail.com
– Connect to me on LinkedIn