SSBB Project submitted for certification.

1. Six Sigma Black Belt
Project
Service Order rework
reduction in the Order
Support Center
1

2. Table of contents
Executive Summary (Pages 3-8)

Define phase (Slides 9-19)

• (Overview, Tools used, Tollgate)
Measure phase (Slides 20-40)

• (Overview, Tools used, Analysis & Conclusion, Tollgate)
Analyze phase (Slides 41-90)

• (Overview, Tools used, Analysis & Conclusion, Tollgate)
Improve phase (Slides 91-126)

• (Overview, Tools used, Analysis & Conclusion, Tollgate)
Implement phase (Slides 127-131)

• (Overview, Tools used, Analysis & Conclusion, Tollgate)
Control phase (Slides 132-137)

• (Overview, Tools used, Analysis & Conclusion, Tollgate)
Project Conclusion & Final Analysis (Slide 138)

Appendix (Slides139-144)

• Appendix A- Acronyms and terms
• Appendix B- Error acronyms and definitions
• Appendix C- Cost to rework calculations
• Appendix D- References
• Appendix E- Software
Project status (Page 145)

2

3. Executive Summary
Service Order Rework Reduction in the Order Support

Center (OSC)
Six σ Project

Submitted to:

Dr. Gloria Pursell

CQE/SPSU

Prepared by:

David Appleby, PM II

Process Management & Improvement

BIOS

10/10/2005

3

4. Executive Summary
(Introduction)
Executive Summary

The Order Support Center (OSC) of the Broadband Internet Operations and
Support (BIOS) group wants to reduce operating costs and monitor the quality
of work being done by its employees. This was initiated by Jeff White, GM of the
BIOS. The process owner and Champion is Morris Jackson, Senior Manager of
the OSC. Currently, there are about 36 different projects being worked in the
OSC. Morris and I determined that the amount of rework on service orders due
to errors being generated by Customer Service Associates (CSA) in the OSC
would have the most impact on reducing operating cost.
My initial observations were 3 fold. Number 1: the Transactional error rate on
service orders is 31.74%. This was based on 4 months (11-04 to 02-05) of data
from an End of Order Activity (EOA) report. Number 2: based on an Engineered
Service Measure (ESM) of 2‟12” and a loaded hourly rate for a CSA of $31.45, a
single transaction error cost the center ~$1.15. Number 3: The annual cost to
rework self generated transaction errors was calculated to be ~$288,000.00. (60
CSA‟s * 260 work days * 16 transaction errors per CSA per day * $1.15).
We decided to use the 6σ DMAIIC methodology to improve the process.
4

5. Executive Summary
(Define & Measure phases)
Executive Summary

Define phase: The Champion and I put together a process improvement team
consisting of 6 CSA‟s and 2 OSC staff support managers. We created a team
charter, a SIPOC diagram and process flow map for service order creation and
correction. I also created a Gantt Chart, a brief Business case and did a Cost of
Quality assessment.
Measure phase: A data collection plan was created. We decided to use the End
of Order Activity report. This was the 1st time this report would be used to do a
qualitative & quantitative analysis of any activity in the OSC. We also
determined the following criteria for an opportunity (a transaction), a defect (a
transaction error) and a defective (error classified by type). I also generated a
control chart (u chart), a histogram, and calculated the process σ level & DPMO
for the initial error rate determined in the define phase. A Pareto chart was
created for defectives. The control chart showed that the process was out of
control and the histogram showed bi-modality in the data set. The rework cost
PMO was ~$365,000.00 ($1.15 per transaction error * DPMO).
5

6. Executive Summary
(Analyze phase)
Executive Summary

Analyze phase: Here we want to take a deeper look at what was observed
during the 1st two phases of the project. My 2 main concerns were the out of
control process and the bi-modality in the data set. The inference drawn from
the bi-modality is that we have more than 1 process occurring within the data
set. The special causes were from high error rates by 4 CSA‟s who either did not
know how to correct errors on service orders or were not following
documented procedures. These outliers were addressed with the champion who
had the issue corrected. The bi-modality in the data set was a result of how the
OSC handled service orders. There are 4 groups in the OSC. 1 group handles
service orders and rework on a regular basis (~54% of this type of work) while
the other 3 groups work the other 35 or so remaining projects and handle
service orders and rework on a part time basis (~46%). The data set was
stratified and the groups were analyzed (using ANOVA) for differences between
the groups. I also did a correlation and regression analysis of each group to see
if there was a correlation between the number of errors and transactions and
how much. Once we determined everyone was not doing the same thing, the
team and I did a root cause analysis (RCA) and generated a cause and effect
diagram to see what it would take to fix the process. We also determined the
procedures needed to identify and correct errors by type.
6

7. Executive Summary
(Improve & Implement phases)
Executive Summary

Improve phase: The RCA and Ishikawa diagram revealed 5 major categories
which need improvement. They include: training, Quality control /assurance,
coaching & development, load balancing and documentation. Using these 5
categories, the team brainstormed about 2-6 items for each that we felt would
improve the process. The categories and items within each were the basis of the
Improvement plan. I ran a simulation using Crystal Ball to see what effect some
of these improvements might have if any. We also looked at the results of the
July and August 2005 EOA report and found that the process is coming into
control and improving. When compared to the error rate found in the define
and measure phase, the rate has decreased from 31.74% to 22.21% with a
concomitant reduction in operating cost for those months of $9,903.00. Quality
control did not exist in the OSC in any way before this project started.
Implement phase: From the ideas generated in the Improve phase we came up
with an AIL or Action Item List. There are14 items which detail what should be
done, who the item is assigned to, when it was assigned, when it should be
completed, a follow up date and the status of the item (open or closed). The AIL
was distributed to everyone who has an action item. It is the responsibility of
the Champion/ Process Owner to ensure the items are completed.
7

8. Executive Summary
Executive Summary

Control phase: The new Quality manager and the 6σ professional (me) created a quality
assurance and control plan to monitor, evaluate and provide feedback to CSA‟s on
transaction errors generated to service orders. The information will be used to identify
those that need additional training and coaching & development.
Conclusion: The most critical things I found on this project was the fact that: #1 No one
was evaluating or monitoring the quality of work being performed by CSA‟s in the OSC. #2
No feedback was being given to CSA‟s on the quality of their work. This situation is not
only being rectified on this project, but other OSC projects as well. #3 No one realized the
process was not in control. The initial findings revealed that the process was out of control
with multiple processes occurring within the data set. Root Cause Analysis found the lack
of Training, Documentation, Coaching & Developing, Quality Assurance & Control and
Load balancing to be the primary reasons for the out of control process. An action plan for
process improvement and control was created and implemented. Current measurements
show the process is coming into control. To date the operating cost overrun has been
reduced $9,903.00, (based on 7 & 8/2005 numbers), with a projected savings of $95,000.00
or more over the next six months.
The next 135 slides show, in detail, the steps taken on this project to identify, analyze and
reduce the amount of rework on service orders in the OSC and thereby reduce the overall
operating cost of this center.
8

9. Define phase (Overview)
Some of the questions answered by the Define phase of the

DMAIIC process include:
Why this project?

What is the business case for this project?

Who is the customer?

What is the current state of the process?

What is the scope?

What are the deliverables?

What is the project completion date?

Who are the champions? Team members? SME‟s (Subject

Matter Experts)?
What resources are needed?

9

10. Define phase (Tools used)
The following tools were used to answer the

questions raised during the Define phase:
Business case

Team charter

Baseline data

Cost of quality assessment

SIPOC diagram

Process flow map

Project work flow (Gantt chart)

10

11. Business case
(The need for this project)
This project was initiated to reduce the

amount of rework on service orders handled
by Customer Service Associates (CSA‟s) in the
Order Support Center (OSC).
Currently, CSA‟s are generating rework on

service orders at a rate of about 32%.
This rework equates to an operational cost

overrun of approximately $288,000.00 per
year.
11

12. C E NTE R F OR QUAL ITY E XC E L L E NC E
S outhern Polytec hnic S tate Univers ity
S ix S igma Projec t C harter
Project Name Service order rework reduction
Blackbelt David Appleby Telephone Number (404)499-3793
Champion Morris Jackson Master Black Belt
Start Date 2/17/05 Target Completion Date 2/17/06
E lement Description T eam C harter
T he process in which
1. Process: T he process being investigated is how failed service orders are
opportunity exists. corrected by C SA’s (C ustomer Service Associates) in the
Bellsouth DSL OSC (Order Support C enter).
Describe the Project’ s Purpose
2. Project Description: T he scope of this project is to investigate errors generated by
and scope. C SA’s (C ustomer Service Associates) handling service orders.
T he purpose will be to r educe the number of errors generated by
C SA’s (C ustomer Service Associates) handling service orders.
C urrently C SA’s who correct service orders are generating
rework on those orders at a rate of approximately 32% .
3. Objective: What improvement is targeted
BSL 1
and what will be the impact to GOAL units
the business?
% E rrors
32% error 25.6% error
1. Reduce the number of
rate on rate
manual errors generated by
service
a CSA correcting service
orders
order.
Dollars
$288,000 per $230,400.00
2. Reduce the cost of
year per year.
reworking service orders
reworking
service
orders.
3.
4.
1. B eginning 4Q 2005 we expect a 20% decrease in self generated
What is the improvement in
4. Business Results:
errors on service orders being corrected by CSA’ s. T his should
business performance
reduce operating cost in the OSC by $57,600.00 per annum.
anticipated and when?
T eam members: Marcia Holcomb, Alice L eiker, Andrea Anderson,
Who are the full -time members
5. T eam members:
Cassandra B lack, Hayden Satterfield, Greg Mickle.
and any expert consultants?
C onsultants: Andrea B raunstein, Robin Owen, Morris Jackson,
Andrew Hinton, Jeff Geyer.
T he way failed service orders are corrected by CSA’ s in the

Which part of the process will
6. Project Scope:
B ellSouth DSL OSC.
be investigated?
12

13. Project charter (continued)
S ix S igma Project C harter
T he final customers are subscribers either purchasing a new

Who is the final customer,
7. Benefit to E xternal
service or changing an existing service. T hey will see their new
what benefits will they see and
Customers:
service delivered on time and correct. T hese are their most
what are their most critical
critical requirements.
requirements?
Project Start 2/17/05
Give the key milestones/dates.
8. Schedule:
“D” Completion 3/31/2005

D- Define
“M” Completion 4/30/2005

M- Measurement
“A” Completion 5/31/2005

A- Analysis
“I” Completion 6/30/2005

I- Improvement
“I” Completion 7/15/2005

I- Implement
“C” Completion 7/31/2005

C- Control
Project Completion 10/01/2005
13

14. Baseline data
Between 11/01/2004 and 01/31/2005

185085 service order transactions were
generated by CSA‟s in the OSC.
59227 were errors that had to be worked

or reworked.
Total error rate is ~0.32 (32%)

Error rate was calculated by, Total Errors/

Total Transactions (Te/Tt)
14

15. Cost of poor Quality
The annual cost to rework service orders was calculated using the

following data:
Each CSA generates an average of 50 transactions on service orders

per day.
32% of this is rework or 16 transaction errors per day.

Using an established ESM (Engineered Service Measurement) of 2

minutes 12 seconds per error, CSA‟s are spending 35.2 minutes
(0.59hrs) per day reworking self generated errors.
A CSA‟s loaded hourly wage is $31.45.

The OSC spends $18.45 per CSA per day to rework service orders

($31.45*0.59 hours).
This is ~$1.15 per transaction error. i.e. ($18.45/16 errors)

Assuming 260 work days & 60 CSA‟s, the OSC spends approximately

$288,000.00 per year reworking service orders. ($18.45*60*260 =
$287,830.40 )
15

16. Service order handling SIPOC
S ix S igma Projec t
S IPOC
S ervic e order handling ac c urac y
Proc es s
S upplier Output
1. S ervice orders are
as s igned to C S A’s for
S ubs cribers
 A s ervice order with no errors .

handling
Internet S ervice P roviders (IS P )

C us tomer S ervice As s ociates (C S A)

2. Handling a s ervice
Internal B ellS outh C us tomers

order includes :
is s uing, updating and
correcting.
3. T he C S A proces s es
C us tomer
the s ervice order. It is
Input s ent to the next
proces s or.
S ubs cribers

S ervice orders

Internet S ervice P roviders (IS P )

S OE G (S ervice Order E ntry G ateway)

S OC S (S ervice Order C reation S ys tem)
 4. If the order fails it
flows back through the
proces s .
B AS S

Input requirements :

S IPOC = S upplier,
5. If the order is 100%
Data entered is c orrec t
 correct it flows
Input, Proc es s ,
downs tream to the
C orrec tly formatted

Output, C us tomer
next s ys tem.
E ntered in the c orrec t fields

16

17. Service Order handling
Process flow (Level 2)
C S A s ervic e order
handling proc es s flow
Order as s igned to
S tart C S A by P roces s order
as s ignment tool
C S A res earches
Is s ue order in
S OE G , MOB I, C S R ,
B OM?
B OC R IS , P S O,
OR ION & NMS , before
Y es
is s uing the order.
No
Is s ue order in
C ons ult Orbit, Y es
B AS S ?
Methods & Any ques tions
P rocedures and/or Y es about handling
P roces s es & the order?
P rocedures No
No
Is s ue order in C heck for errors
S OC S before is s uing
R eject,
R eject Handle or
Handle
C ancel
C ancel Did
C hoos e correct
Any errors ? No S OE G auto Y es Note in S OE G
reject reas on
populate?
C ancel the order
Y es
Note reas on in
C orrect errors
S OE G
us ing documented
No Manually populate
Did the methods and
cancel flow procedures
Y es
into B OC R IS ,
S OE G ?
Manually cancel
No
the order
17

18. Project time line (Gantt Chart)
Six σ Black Belt project
Mar 2005 Apr 2005 May 2005 Jun 2005 Jul 2005
ID Task Name Start Finish Duration
2/27 3/6 3/13 3/20 3/27 4/3 4/10 4/17 4/24 5/1 5/8 5/15 5/22 5/29 6/5 6/12 6/19 6/26 7/3 7/10 7/17
1 Define phase 2/17/2005 3/17/2005 4.2w
2 Measure phase 3/17/2005 4/14/2005 4.2w
3 Analyze phase 4/14/2005 6/16/2005 9.2w
4 Improve phase 6/9/2005 7/7/2005 4.2w
5 Implement phase 6/30/2005 7/21/2005 3.2w
6 Control phase 7/21/2005 8/18/2005 4.2w
18

19. Define phase (Tollgate)
Team Charter completed and submitted

(3/26/2005).
SIPOC created.

Baseline established (based on 3 months of

data).
Cost of Quality (rework) quantified.

Business case defined.

Process flows and project time line defined.

19

20. Measure phase (Overview)
Some of the questions answered by the

Measure phase of the DMAIIC process
include:
What is an opportunity? A defect?

What type of data exists in the data set?

What data collection plan will be used?

How will the data be validated?

Is there adequate data on the process?

20

21. Measure phase (Tools used)
Data collection plan

Data score cards

Control chart

σ Level

DPMO

Histograms

Pareto

21

22. Opportunities, Defects &
Defectives
A service order transaction is defined as creating,

changing, updating or correcting fields on service orders.
A single service order can have multiple transactions.

Each transaction entered incorrectly is considered an error

by the OSC (a defect or non conformance).
A specific error type is generated depending on the

transaction type.
Error types are defectives or non conforming.

Because all service orders must be 100% correct, only

transactions and transaction errors are reported.
22

23. Measure phase (Qualifications)
Data will be collected that is Specific, Measurable,

Actionable, Relevant & Timely (S.M.A.R.T.).
2 types of data will be collected and analyzed.

They are: attribute count & attribute classification.

The EOA or End of Order Activity report records data on

specific CSA‟s and groups. This report tabulates the
number of errors and transactions for each CSA as well as
by group.
An EOA report can also be obtained for data on specific

error types and classifications. This report classifies errors
by type.
23

24. Data qualifications (Count data)
Data was gathered from a service order EOA (End of

Activity) report.
This data represents 4 months of CSA work/rework

activity (11/04-02/05).
Census data was collected.

Data type is attribute, count, defects (errors) and variable.

A single order can have multiple transactions and each

transaction, if entered incorrectly, generates a transaction
error.
An order with errors will fail and is then sent back through

the system to be corrected.
Error rate was calculated by dividing total # of errors by

total # of transactions. (Te/Tt).
24

25. Raw count data score card
11/04 - 02/05 (OSC)
OS C # Transactions Errors Error rate
Total: 234086 74302 31.74%
Mean: 4036 1281 31.74%
Median: 2944 1061
Deviation: 3546 924
Minimum: 448 147
Maximum: 19884 5208
Range: 19436 5061 26.04%
25

26. Current σ Level, DPMO &
Rework cost PMO
σ Level = 0.97

DPMO = 317,413

Rework cost per million opportunities:

• $1.15 per error reworked (based on earlier
calculations) * DPMO = $365,024.95
26

27. ODDVZ
ODDWF
ODDBW
ODDBK
ODDVU
ODDAN
variable, the recommended control chart is a
ODDAL
27
ODDBD
ODDVE
ODDRF
ODDAT
Control chart (u chart)
ODDWQ
ODDRO
ODDAH
Because the data is attribute, count &
ODDVL
ODDBL
ODDRG
ODDMI
ODDWD
ODDRQ
for 11-04 to 02-05
ODDWL
ODDJG
ODDMP
ODDWE
ODDVI
Error rate
ODDMR
ODDRL
ODDMM
ODDRC
ODDBY
u C hart daily e rror rate
ODDJH
ODDJN
ODDVN
ODDBG
ODDJR
ODDMK
ODDRH
ODDBI
ODDBE
ODDWJ
ODDVG
ODDMJ
ODDRB
ODDAU
ODDJS
ODDMB
ODDVO
ODDMH
u chart.
ODDJT
ODDJR
ODDML
L C L =0.04246
C E N=0.31741
UC L =0.59237
ODDBH
ODDAC
ODDMO
ODDBX
ODDJF
ODDMG
ODDBN

1
0
0.8
0.6
0.4
0.2
-0.2

28. Control chart interpretation
Each data point represents the average error rate per CSA

per day.
The data is for the period from 11-04 to 02-05.

The X axis represents each CSA by ODD code.

6 data points are shown exceeding the UCL.

These 6 data points represent CSA‟s whose error rate is

nearly 100%. These individuals have been identified and
have been covered.
4 data points are exceeding the LCL.

Although these 4 data points are beyond the lower

control limits, this is not a bad thing since the ultimate
goal is zero errors.
The process is out of control.

28

29. M
# CSA's
0
1
2
3
4
5
6
7
8
9

ean = 0.3886
S td Dev = 0.187
Normal Dis tribution
K S T es t p-value = .1844
0.
10
9
0. to
15 <=
0. to 0.
19 <= 15
1 0
0. to .1 9
<1
23 =
20
0. to . 23
27 <= 2
30
0. to . 2
31 <= 73
30
0. to . 3
35 <= 13
40
0. to . 35
39 <= 4
50
0. to . 39
43 <= 5
60
0. to . 4
C S A error rate (11/
47 <= 36
70
E rror rate
0. to . 4
51 <= 77
70
0. to . 51
55 <= 7
80
2004-2/
to . 5
0. < 58
59 =
1st pass histogram for the OSC
9 0.
0. to 599
2005)
64 <=
to 0
11/04-
< . 64
=
0.
6
81
0.
80
3
to
<
=
OSC Error rate 11/04-02/05
0.
84
0.
4
88
4
29
to
<
=
0.
92
5

30. # CS A's
-2
. 87
Mean = 0. 0
0
1
2
3
4
5
6
7
8
9
to
S td Dev = 1. 0
<=
-2
.
-2
. 65
Normal Dis tribution
2
43
4
to
K S Tes t p-value = . 1844
<=
-2
. 21
6
-1
. 56
1
to
-1
. 34 <=
3 -1
to .
-1
. 12 <= 34 3
5 -1
-0 to .1
.9
0 7 <= 2 5
-0
-0 to .9
.6
8 9 <= 0 7
-0
-0 to .6
.4
7 1 <= 8 9
-0
-0 to .4
.2
5 2 <= 7 1
-0
.
-0 to
.0 <= 25 2
34 -0
0. to . 03
18 <= 4
4
t 0.
0. o < 184
40 =
Z score
2 0.4
0
0. to
62 <= 2
to 0.
0.
83 <= 6 2
9 0
1. to .83
05 <= 9
7 1
1. to .0 5
27 <= 7
5
to 1.2
<= 75
1.
49
3
OS C data Z trans form (4 months 11/04-02/05)
30
Z transformed data histogram

31. 1st pass observation
(OSC histogram)
Data is not normally distributed but indicates

bimodality.
Several things can account for this including:

More than 1 process occurring within the data set.

Differences in experience or training levels.

“Loose” adherence to or misinterpretation of documented

procedures.
Sub groups operating within the team.

2 outlier data points are also evident. This can be an

indication of special causes.
These outliers represent 4 individuals. (3 in the .803-

844 bin and 1 in the .884-.925 bin)
31

32. Composition of the OSC
The OSC (Order Support Center) is made up of

approximately 60 CSA‟s (Customer Service
Associates), Management and Staff support.
CSA‟s are divided into 4 groups.

All CSA‟s handle service orders.

Handling consists of issuing, updating or correcting

service orders.
Groups 1,2 & 4 handle errors, projects and take calls.

A single group (Group 3) handles errors only.

32

33. Histogram by group
The histogram on the next slide was

created to show how each group‟s error
rate contributes to the OSC histogram
found in slide 29.
Each group has been assigned its own

color.
33

34. Histogram by group (11/04-02/05)
(11/04-
Error rate by group 11-04 to 02-05
9
8
7
6
5
4
3
2
1
0
0 0.05 0.11 0.15 0.19 0.23 0.27 0.31 0.36 0.4 0.44 0.48 0.52 0.56 0.6 0.64 0.68 0.72 0.77 0.81 0.85 0.89 0.93 0.97 1.01
Error rate
Group 3 Group 1 Group 2 Group 4
34

35. Histograms side by side comparison
Error rate by group 11-04 to 02-05
Normal Distribution
Mean = 0.3886
CSA error rate (11/2004-2/2005) 9
S td Dev = 0.187
KS Test p-value = .1844
9
8
8
7
7
6
6
5
5
# CS A's
4
4
3
3
2
2
1
1
0 0
0 0.05 0.11 0.15 0.19 0.23 0.27 0.31 0.36 0.4 0.44 0.48 0.52 0.56 0.6 0.64 0.68 0.72 0.77 0.81 0.85 0.89 0.93 0.97 1.01
<= .1 5
1
<= .6 4
1
2
3
<= 13
4
5
6
<= 77
7
0. o <= 58
<= 9
44
25
to 0.2 3
to 0.2 7
to 0.3 5
to 0.3 9
to 0.4 3
to 0.5 1
to .5 9
to 0 .19
68
to 0.3
to 0.4
0. 5
0. 8
0. 9
0
0
Error rate
0.
0
<=
<=
<=
<=
<=
<=
<=
<=
<=
<=
to
to
to
to
to
t
9
9
10
15
59
64
1
2
3
3
4
5
6
7
7
8
3
4
19
23
27
31
35
39
43
47
51
55
80
88
0.
0.
0.
0.
0.
0.
0.
0.
0.
0.
0.
0.
0.
0.
0.
Group 3 Group 1 Group 2 Group 4
Error rate
35

36. Data qualifications (Classified data)
Data was gathered from a service order EOA (End of Order

Activity) report that classifies data by error type.
Census data was used.

Data type is attribute, classification, defectives and variable.

Three major and two minor error types are identified. They are:

 SOER (Service Order Error Request) errors
 FMT (ForMaT) errors
 OPEC (On-line Pending Edit CRIS) errors
 FACS & LIST
Since FACS and LIST account for less than 2% of the errors we

will focus on the 3 major error categories.
CRIS (Customer Records Information System)

SOCS (Service Order Creation System)

36

37. Classified data scorecard:
Top 3 error types in the 5 error
categories (02-2005)
(02-
Total E rrors by Type: % Total E rrors by Type:
F ACS 153 F ACS 1.48%
F MT 3272 F MT 31.55%
LIS T 11 LIS T 0.11%
OP E C 963 OP E C 9.28%
S OE R 5973 S OE R 57.59%
Total E rrors : 10372
Top 3 errors per type by F ID,S ection & Code:
E RROR TY P E F ID E RROR S E CTION E RRROR CODE # of errors
F ACS E S OI OTHE R 153
F MT F MT 1 1161
F MT F MT 36 287
F MT F MT 14 215
LIS T L111 11
OP E C O004 316
OP E C O852 157
OP E C O930 73
S OE R F MT S &E 434 673
S OE R GF S &E 10 480
S OE R F MT S &E 5 368
Total: 3894
37

38. 02-
02-05 Errors sorted by type
Top 20 errors by category (02-2005)
120.00%
# of errors
% of Total
10000
100.00%
99.89% 100.00%
98.41%
8000 89.11%
80.00%
6000
60.00%
57.50%
4000
40.00%
2000 20.00%
0 0.00%
S OER FMT OPEC FACS L IS T Total
5949 3270 963 153 11 10346
# of errors
57.50% 89.11% 98.41% 99.89% 100.00%
cumulative
Category
38

39. Measure phase
(Analysis & Conclusion)
The Control chart generated shows the process

is out of control
The σ Level and cost to rework show there are

opportunities for improvement.
New reports are being generated to look at

these issues. This is the first time the EOA
reports have been used in the OSC.
Further analysis will be needed to find out why

the data set is showing bi-modality.
39

40. Measure phase (Tollgate)
Control chart generated (u-chart).

Type of data identified (count & classified)

Method to collect and identify data was

implemented (End of Order Activity report)
Methods to observe & analyze data

implemented (histograms, pareto chart, control
chart & data score cards, sigma level, DPMO &
cost to rework per million opportunities)
40

41. Analyze phase (Overview)
Some of the questions answered by the

Analyze phase of the DMAIIC process include:
What is the current state of the process?

What factors might be causing the poor

quality?
What can we do to improve the process?

41

42. Analyze phase (Tools used)
Box plot

Histogram

Stratified data (Data door)

Pareto chart

ANOVA

Correlation

Regression analysis

Root cause analysis

Cause & Effect diagram

DOE

42

43. Analyze Strategy
Each of the 2 data types will be

analyzed separately.
Count data will be analyzed using

histograms, score cards, box plots,
correlation & regression analysis and
ANOVA.
The classified data will be analyzed

using score cards and Pareto charts.
43

44. Histograms side by side comparison
This slide is re-presented to show what was

observed during the Measure phase.
Normal Dis tribution
Error rate by group 11-04 to 02-05
Mean = 0.3886
CS A error rate (11/2004-2/2005)
S td Dev = 0.187
K S Tes t p-value = .1844
9
9
8
8
7
7
6
6
5
5
# CS A's
4 4
3 3
2 2
1 1
0 0
0 0.05 0.11 0.15 0.19 0.23 0.27 0.31 0.36 0.4 0.44 0.48 0.52 0.56 0.6 0.64 0.68 0.72 0.77 0.81 0.85 0.89 0.93 0.97 1.01
19 <= 1 5
=1
<= 6 4
1
=2
31 <= 73
=3
39 <= 54
=5
47 <= 36
=7
=7
0. o < 58
64 <= 9
4
5
0. to < .2 3
0. to < .3 1
0. to < .3 9
0. to < .4 7
0. to < .5 1
to .5 9
84
92
0. to < .19
68
to 0.2
to 0.3
4
5
0.
0.
0.
0.
0.
0.
0.
Error rate
0
0
0
0
0
0
0
15 <=
59 =
<=
<=
to
to
to
0. to
to
to
t
9
9
10
1
2
3
3
4
5
6
7
7
8
3
4
23
27
35
43
51
55
80
88
0.
0.
0.
Group 3 Group 1 Group 2 Group 4
0.
0.
0.
0.
0.
Error rate
44

45. 1st pass observation
(OSC histogram)
Data is not normally distributed but indicates

bimodality.
Several things can account for this including:

multiple processes occurring within the data set.

differences in experience or training levels.

“loose” adherence to or misinterpretation of documented

procedures.
2 outlier data points are also evident.

These are composed of 4 individuals. (3 in the .803-

844 bin and 1 in the .884-.925 bin)
45

46. Stratification of OSC data
A brief “analysis” of each group‟s central

tendencies is followed by its associated
histogram.
These analyses are based on 4 months

(11/04-02/05) of data from the EOA
report discussed in the measure phase.
46

47. Group 1 central tendencies
(error rate)
Mean: 0.4057

Median: 0.4370

Standard Deviation: 0.1083

Minimum: 0.1996

Maximum: 0.5281

Range: 0.3285

Median is to the right of the mean indicating

the group‟s performance is skewed left.
No outliers evident.

47

48. Group 1 error rate (11/04-02/05)
(11/04-
Normal Dis tribution
Mean = 0. 4057
E rror rate group 1
S td Dev = 0. 1083
K S Tes t p-value = . 3631
6
5
4
# CS A's
3
2
1
0
0. 1996 0. 2465 0. 2935 0. 3404 0. 3873 0. 4343 0. 4812
to <= to <= to <= to <= to <= to <= to <=
0. 2465 0. 2935 0. 3404 0. 3873 0. 4343 0. 4812 0. 5281
error rate
48

49. Group 2 central tendencies
(error rate)
Mean: 0.4945

Median: 0.4489

Standard Deviation: 0.2266

Minimum: 0.1100

Maximum: 0.8414

Range: 0.7314

Median is to the left of the mean indicating the

group‟s performance is skewed right.
One outlier representing 3 individuals is evident.

49

50. Group 2 error rate (11/04-02/05)
(11/04-
Normal Dis tribution
M ean = 0.4945
E rror rate group 2
S td Dev = 0.2266
K S T es t p-value = .4754
6
5
4
# CSA's
3
2
1
0
0.11 to 0.215 to 0.319 to 0.423 to 0.528 to 0.737 to
<= 0.215 <= 0.319 <= 0.423 <= 0.528 <= 0.632 <= 0.841
error rate
50

51. Group 3 central tendencies
(error rate)
Mean: 0.2287

Median: 0.2231

Standard Deviation: 0.0663

Minimum: 0.1094

Maximum: 0.3416

Range: 0.2322

Median is slightly left of the mean indicating group

performance is skewed slightly right. (For all intents
and purposes, there is no skewing).
No outliers evident.

51

52. Group 3 error rate (11/04-02/05)
(11/04-
Normal Dis tribution
E rror rate group 3
M ean = 0.2287
S td Dev = 0.0664
K S T es t p-value = .5291
6
5
4
# CSA's
3
2
1
0
0.1094 0.1352 0.161 0.1868 0.2126 0.2384 0.2642 0.29 to 0.3158
to <= to <= to <= to <= to <= to <= to <= <= to <=
0.1352 0.161 0.1868 0.2126 0.2384 0.2642 0.29 0.3158 0.3416
error rate
52

53. Group 4 central tendencies
(error rate)
Mean: 0.4595

Median: 0.4739

Standard Deviation: 0.1848

Minimum: 0.1675

Maximum: 0.9252

Range: 0.7577

Median is to the right of the mean indicating group

performance is skewed left.
One extreme outlier representing one individual is

evident.
53

54. Group 4 error rate (11/04-02/05)
(11/04-
Normal Dis tribution
M ean = 0.4595
E rror rate group 4
S td Dev = 0.1848
K S T es t p-value = .5482
6
5
4
# CSA's
3
2
1
0
0.167 to 0.262 to 0.357 to 0.452 to 0.546 to 0.641 to 0.831 to
<= 0.262 <= 0.357 <= 0.452 <= 0.546 <= 0.641 <= 0.736 <= 0.925
error rate
54

55. 4 groups Box plots
E rror rate Groups 1-4 (11/04-02-05)
E rror ra te
1
0. 9
0. 8
0. 7
0. 6
1s t quartile
Min
0. 5 Median
Max
3rd quartile
0. 4
0. 3
0. 2
0. 1
0
Group1 Group2 Group3 Group4
55

56. Histogram comparison of the entire
OSC to Groups (1,2 &4) & (group 3)
Normal Dis tribution
Group 1,2 & 4 (11-04 to 02-05)
Mean = 0.4549
S td Dev = 0.1808
KS Tes t p-value = .2666
9
8
Normal Dis tribution
Mean = 0.3886
CSA error rate (11/2004-2/2005) 7
S td Dev = 0.187
K S Tes t p-value = .1844 6
9
5
# CSA's
8 4
3
7
2
6 1
0
5 0.11 0.161 0.212 0.263 0.314 0.365 0.416 0.467 0.518 0.569 0.62 0.823 0.874
# CS A's
to <= to <= to <= to <= to <= to <= to <= to <= to <= to <= to <= to <= to <=
0.161 0.212 0.263 0.314 0.365 0.416 0.467 0.518 0.569 0.62 0.67 0.874 0.925
Error rate
4
3
Normal Dis tribution
E rror rate group 3
Mean = 0.2287
2 S td Dev = 0.0664
KS Tes t p-value = .5291
9
1
8
0 7
0.15 0.232 0.313 0.395 0.477 0.558 0.64 0.803 0.884
to <= to <= to <= to <= to <= to <= to <= to <= to <= 6
0.191 0.273 0.354 0.436 0.517 0.599 0.681 0.844 0.925
5
# CSA's
Error rate
4
3
2
1
0
0.1094 0.1352 0.161 to 0.1868 0.2126 0.2384 0.2642 0.29 to 0.3158
to <= to <= <= to <= to <= to <= to <= <= to <=
0.1352 0.161 0.1868 0.2126 0.2384 0.2642 0.29 0.3158 0.3416
e rror rate
56

57. Testing for differences
between groups
(error rate)
As stated earlier, the OSC is divided into four groups.

1 group (Group 3) corrects errors on service orders

only. (Approximately 54% of all transactions in the
OSC are corrected by group 3)
The other 3 groups handle all other functions

including correcting errors on service orders.
A single factor ANOVA was run on the error rate for

all 4 groups.
A second single factor ANOVA was run on groups

1,2 & 4 only.
57

58. All groups test (Error rate)
Hypothesis:

H0 = The null hypothesis is this: There is no

statistical difference in error rates between
the 4 groups in the OSC.
Ha = The alternate is this: At least 1 of the 4

groups error rate will be statistically different
from the other groups.
α =.05

58

59. ANOVA (All groups)
Anova: S ingle F actor
S UMMARY
Groups Count S um Average Varianc e
GRP1 12 4.868114 0.405676 0.011722
GRP2 13 6.428758 0.49452 0.051349
GRP3 17 3.887908 0.2287 0.004406
GRP4 16 7.352564 0.459535 0.034154
ANOVA
S ourc e of
Variation SS df MS F P-value F c rit
Between Groups 0.664509 3 0.221503 9.007344 6.21E -05 2.775764
Within Groups 1.327934 54 0.024591
Total 1.992443 57
59

60. Conclusion (All groups)
Running a 4 level single factor ANOVA found

the following:
• F table = 2.7758
• F test = 9.0073
• P-value = 6.21E-05
Since the F table value was less than the F test

value and the P-value was less than α, we can
reject the null hypothesis and conclude that at
least 1 group‟s error rate was significantly
different from the other 3 at the 95% level.
60

61. Groups 1,2 & 4 test
(Error rate)
Hypothesis:

H0 = The null hypothesis is this: There is no

statistical difference in error rates between
the 3 groups in the OSC that do not rework
errors on a regular basis.
Ha = The alternate is this: At least 1 of the 3

groups error rate will be statistically different
from the other groups.
α =.05

61

62. ANOVA (Groups 1,2 & 4)
Anova: S ingle Factor
S UMMARY
Groups Count S um Average Variance
GRP1 12 4.868114 0.405676 0.011722
GRP2 13 6.428758 0.49452 0.051349
GRP4 16 7.352564 0.459535 0.034154
ANOVA
S ourc e of
Variation SS df MS F P-value F crit
Between Groups 0.049826 2 0.024913 0.752879 0.477904 3.244821
Within Groups 1.257434 38 0.03309
Total 1.307261 40
62

63. Conclusion (groups 1,2 & 4)
Running a 3 level single factor ANOVA found

the following:
• F table = 3.2448
• F test = 0.7529
• P-value = 0.4779
Since the F table value was greater than the F

test value and the P-value was greater than α,
we can fail to reject the null hypothesis and
conclude that no group‟s error rate was
significantly different from any other group
tested at the 95% confidence level.
63

64. Analysis between
groups
The ANOVA results for the 4 groups showed a

statistical difference in error rates between the 4
groups within the OSC. (Confidence level = 95%)
It also showed there was no statistical difference in

error rates between the 3 groups (1,2 & 4) that
handle error corrections (rework) on a part time
basis. (Confidence level = 95%)
Conclusion: there is a statistical difference between

the way group 3 handles errors compared to groups
1,2 & 4.
64

65. Correlation & Regression
An analysis was done to see if there is a

correlation between the number of
transactions (x, independent variable) and the
number of errors generated (y, dependent
variable).
Results on the next slide.

65

66. Correlation results
Group 1 correlation Group 3 correlation
Tranz/day E rrors /day Tranz/day E rrors /day
Tranz/day 1 Tranz/day 1
E rrors /day 0.919722 1 E rrors /day 0.747944 1
Group 2 correlation Group 4 correlation
Tranz/day E rrors /day Tranz/day E rrors /day
Tranz/day 1 Tranz/day 1
E rrors /day 0.804942 1 E rrors /day 0.899662 1
66

67. Correlation conclusion
The results show a strong positive correlation

between the number of transactions and the number
of errors generated for groups 1 & 4. (.92 &.90)
There is also a positive correlation for groups 2 & 3.

But not as strong as the results for 1 & 4. (.80 &.75)
Conclusion: groups 1 & 4 generate errors at a

greater rate than 2 & 3.
A simple regression study should reveal how much

for each group.
67

68. Group 1 simple linear regression analysis
(y=errors x=transactions)
S UMMAR Y OUT P UT
R egres s ion S tatis tics
Multiple R 0.919721576
R S quare 0.845887778
Adjusted R S quare 0.830476556
S tandard E rror 4.128727063
Observations 12
ANOVA
df SS MS F S ignificance F
R egression 1 935.6383549 935.6384 54.88778 2.2921E -05
R esidual 10 170.4638716 17.04639
T otal 11 1106.102227
Upper L ower
Coefficients S tandard E rror t S tat P -value L ower 95% 95% 95.0% Upper 95.0%
Intercept 0.617855099 2.180649915 0.283335 0.782699 -4.240936541 5.4766467 -4.2409365 5.476646739
T ranz/day 0.369477102 0.049871186 7.408629 2.29E -05 0.258357157 0.480597 0.25835716 0.480597048
R E S IDUAL OUT P UT
Obs ervation P redicted E rrors /day R es iduals
1 8.386111178 -1.498611178
2 11.36502032 3.997479684
3 7.305390653 1.682109347
4 10.23811515 -2.038115154
5 15.07364673 4.401353268
6 7.480892277 1.069107723
7 19.40114729 -1.938647294
8 10.71381692 3.573683077
9 38.17982103 3.882678974
10 21.31780976 -5.342809763
11 3.555198063 0.244801937
12 16.74553062 -8.033030621
68