Defect Analysis & Prevention, Data Mining & Visualization of Defect Matrix

By
Aniruddha Sahasrabudhe
Consultant, Engineering Process Excellence & Integration
http://in.linkedin.com/pub/aniruddha-sahasrabudhe/8/a19/414
E mail: a_sahasrabudhe@hotmail.com
November - 2010
Defect Analysis & Prevention
Data Mining & Visualization of Defect Matrix using MS Excel

of 7
Abstract
For economic and operational reasons large engineering organizations outsource typical
activities like conversion of existing drawings to 3d CAD models (using Pro Engineer or Catia
or UG softwares), creation of manufacturing drawings, designing and detailing of BIW
Fixtures, 3d models to Meshed models using Hypermesh or ANSA for FE Analysis, Reverse
engineering of parts, CNC programming etc. to reputed Engineering Service Providers (ESPs).
Expectation from ESPs is that in a short period of time they should deliver Defect free
services, qualify as most favoured supplier of engineering services and become an integral
part of supply chain of outsourcing organization.
Generally the process begins with a pilot job i.e. a small part (5 to 10% of total work) is
outsourced to ESPs with the necessary technical details i.e. Statement of work (SOW),
Quality Standards/ Work instructions/Reference models, drawings, Acceptance Quality Plan
(AQP), etc. Quality of pilot job delivered is one of the important criterion for shortlisting or
approving ESP as a supplier of engineering services. It is assessed on the basis of Number of
Defects found, Severity of defects (Minor or Major), First time yield (FTY) i.e. Number of
Defect free jobs or files delivered / total number of jobs or files assigned, on time delivery
within agreed cost, turnaround time for rework if any, number of resources used , speed,
clarity and preciseness of communication etc.
Feedback of Quality of pilot job delivered (number of defects, first time yield etc.) is given to
the ESP for needful corrective action. In order to become an integral part of supply chain of
outsourcing organization it is essential for ESP’s team to have a shared vision of Customer’s
Critical Requirements, CCRs of Quality, Reliability, Manufacturing Process, Cost, Delivery
period, Service etc. and a very clear understanding of definition of “Defect”. A very
systematic and disciplined study of defect data and associated processes on part of ESP is
required to ensure that future jobs will be defect free.
Defect Analysis and Prevention is a very systematic process and its application varies from
situation to situation. Every defect results into an expensive rework for the organization,
adds to COPQ (Cost of Poor Quality) and directly impacts the bottom line – profits. It is
therefore very essential for every organization to know and set KPI’s, Key Performance
Indicators which reflect on effectiveness of organizational processes i.e. how they are
performing in terms of Output /Unit of Input. If issues related to process Defect are not
addressed at the right time they can escalate into a very difficult situation for the
organization.
Recent recalls of defective vehicles by Mercedes-Benz & Toyota are best examples of such
situations
-- Mercedes-Benz is recalling 85,000 of its 2010 C-Class and the 2010-11 E-Class models
because a steering problem could make the vehicles difficult to control and
--Toyota is recalling 3.8 million vehicles worldwide, including 740,000 vehicles in the United
States that could leak brake fluid and cause stopping power to “gradually decline.” Recalled
models are Avalon, Camry, ES 350, IS 250 and IS350, Tacoma, Tundra and Prius.
Complex problems of this magnitude require complex process modelling & FMEA (Failure
Mode and Effects Analysis) of Parts, Systems, Process, Service and Software and long-time to
fix till a reliable solution is found and approved by a competent Traffic safety authority.
This simple case study is about a situation (see “Overview of Situation“ on left) where an
engineering organization found 427 defects in the pilot job of 3d modelling, detailing,
meshing & CNC programming outsourced to an ESP. The analysis shows results of data
mining of “Defect Matrix” using “Data Analysis pack “ and data visualization using column
charts & pie graph functionalities in MS Excel. Defects D1, D2, D3 pertain to 3d modelling,
D4 & D5 to detailing D6, D7 & D8 to Meshing & D9 &D10 to CNC programming. The results
are useful for both the organizations. Outsourcing organization for comparing performance
of ESPs & for ESP for doing “Gap Analysis” i.e. to know its current process capability (“As is
“condition) and efforts required to comply with customer’s requirement -- Defect free
service, “As should be” condition.
A Picture is worth a thousand words (or Numbers) – For a better understanding and
visualization, Defect numbers have been transformed into graphics and displayed in various
ways with appropriate comments / conclusions.
Based on results of analysis, corrective and preventive actions have been suggested to
minimise chances of recurrence of same defects for identical jobs in future. A detailed
technical discussion of 427 Defects D1….D10 is beyond the scope of this article hence not
covered. It is presumed that reader of this article is conversant with application of misc.
formulas, “Data Analysis pack” and charting functionalities of MS Excel, hence only results
have been shown and discussed.
-
OVERVIEW OF SITUATION
PROCESS MODEL:
Raw Correlation Matrix with data of 10 types of
Defects (D1..D10) for team of 10 Resources
(R1..R10)
A) Inputs from outsourcing engineering
organization: --
Statement of Work (SOW) and CD containing 77
data files for pilot job of 3d Modelling, Detailing,
Meshing, “Acceptance Quality Plan (AQP) “ and
some reference examples for each of these tasks
was given to ESP.
B) Deliverables expected from ESP:
3DModels, Manufacturing drgs, Meshed models
& CNC programs as per “AQP“within agreed
delivery time.
C) ESP’s estimation shown that to complete the
pilot job as per delivery time requirement of
outsourcing organization, a team of 10 resources
(R1...R10) will be required. ESP checked
competency inventory and selected 10 resources
who worked on the data and delivered the pilot
job within agreed delivery time.
D) Quality of pilot job: Outsourcing organization
checked quality of pilot job and found a total of
427 Defects which were informed to ESP. The
data of 427 defects has been co related with the
resources and arranged into 10 types of unique
“Defect categories (D1...D10) as shown in the
matrix on page 2. ESP is required to correct the
defects and upload corrected data as soon as
possible.
Requirement of Engineering Service
Provider, ESP
1) A detailed analysis of Defect data in the
matrix on page 3 and get an idea of the
performance of each resource and the team
using “Data Analysis” pack in MS Excel since this
is readily available with him.
2) Graphic visualization of raw correlation
Matrix for better understanding of Defect data.
3) Process Capability & Gap Analysis:
An idea of current Process Capability & Sigma
Level of the team & compare it with desired 6
Sigma level (3.4 Defects per Million
Opportunities) and
4) CAPA -- Corrective and Preventive Action/s
Action that need to be taken to Correct detected
Non conformity (427 Defects) and Preventive
action to avoid recurrence of Defects for future
jobs.

of 7
Raw Correlation Matrix of Resources R1..R10 & Defects D1..D10
Data Mining from Raw Correlation Matrix showing meaningful information for each resource & the team
Above matrix displays Maximum and Minimum number of type of Defect, Total number of Defects for each resource & the team.
Performance of Team --Defect numbers in above table are transformed into graphics for better visualization & understanding
Conclusions from Data Mining table & Pie graphs
Resource with Maximum Defects Maximum of total Defects
= R1 (51 out of 427, 11.94%) = D10 (58 out of 427, 13.58 %)
Resource with Minimum Defects Minimum of total Defects
= R6 (31 out of 427, 7.26%) = D7 (34 out of 427, 7.96 %)
Team Resources → R1 R2 R3 R4 R5 R6 R7 R8 R9 R10
Defect types ↓
D1 3 5 7 3 8 3 1 5 5 5
D2 5 2 3 7 4 4 3 3 7 2
D3 8 3 9 3 5 2 7 1 8 3
D4 4 1 6 3 6 5 3 6 2 3
D5 7 3 2 5 4 1 2 5 7 4
D6 3 7 3 6 2 3 6 5 4 7
D7 5 1 4 4 4 3 3 5 1 4
D8 2 2 7 2 3 2 5 6 4 4
D9 4 5 3 1 3 5 7 3 5 3
D10 10 8 5 4 7 3 4 6 5 6
Additional available Information
Number of data files assigned 8 12 7 9 5 9 6 6 8 7
Opportunity for Defects / File 10 10 10 10 10 10 10 10 10 10
Maximum No of Defects possible 80 120 70 90 50 90 60 60 80 70
TeamResources → R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 Total of % of Total Number
Defect type ↓ Defects in Row of Defects
D1 3 5 7 3 8 3 1 5 5 5 45 10.54
D2 5 2 3 7 4 4 3 3 7 2 40 9.37
D3 8 3 9 3 5 2 7 1 8 3 49 11.48
D4 4 1 6 3 6 5 3 6 2 3 39 9.13
D5 7 3 2 5 4 1 2 5 7 4 40 9.37
D6 3 7 3 6 2 3 6 5 4 7 46 10.77
D7 5 1 4 4 4 3 3 5 1 4 34 7.96
D8 2 2 7 2 3 2 5 6 4 4 37 8.67
D9 4 5 3 1 3 5 7 3 5 3 39 9.13
D10 10 8 5 4 7 3 4 6 5 6 58 13.58
Total numberof Defects forResource =SUM(D1:D10) 51 37 49 38 46 31 41 45 48 41 427 100.00
% of Total Defect by Resource =((SUM(D1:D10)/427)*100 11.94 8.67 11.48 8.90 10.77 7.26 9.60 10.54 11.24 9.60 100.00
Colourindication
Cell in this colourshows Maximumvalue of Defect in the column forthat Resource
Cell in this colourshows Minimumvalue of Defect in the column forthat Resource
0
2
4
6
8
10
12
R1 R2 R3 R4 R5 R6 R7 R8 R9 R10
D1 D2 D3 D4 D5 D6 D7 D8 D9 D10
Pie graph 1 showing % contribution
of each resource to total of 427
Defects
Pie graph 2 showing all
Defects (D1..D10) & their
% contribution to total of
427 Defects.

of 7
Defect Matrix Transformed to Matrix of Statistically important numbers & information for all Resources
Note: % Yield = (1-(Total number of defects of resource / Maximum number of defects possible))*100.
Defect Data of numbers for Resources R1..R4 transformed to graphics
Team Resources → R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 Total of % of Total Number
Defect type ↓ Defects in Row of Defects
D1 3 5 7 3 8 3 1 5 5 5 45 10.54
D2 5 2 3 7 4 4 3 3 7 2 40 9.37
D3 8 3 9 3 5 2 7 1 8 3 49 11.48
D4 4 1 6 3 6 5 3 6 2 3 39 9.13
D5 7 3 2 5 4 1 2 5 7 4 40 9.37
D6 3 7 3 6 2 3 6 5 4 7 46 10.77
D7 5 1 4 4 4 3 3 5 1 4 34 7.96
D8 2 2 7 2 3 2 5 6 4 4 37 8.67
D9 4 5 3 1 3 5 7 3 5 3 39 9.13
D10 10 8 5 4 7 3 4 6 5 6 58 13.58
Total number of Defects for Resource = SUM(D1:D10) 51 37 49 38 46 31 41 45 48 41 427 100.00
% of Total Defect by Resource = ((SUM(D1:D10)/427)*100 11.94 8.67 11.48 8.90 10.77 7.26 9.60 10.54 11.24 9.60 100.00
Colour indication
Mean 5.100 3.700 4.900 3.800 4.600 3.100 4.100 4.500 4.800 4.300
Standard Error 0.795 0.775 0.722 0.573 0.600 0.407 0.657 0.522 0.696 0.517
Median 4.500 3.000 4.500 3.500 4.000 3.000 3.500 5.000 5.000 4.000
Mode 3.000 5.000 3.000 3.000 4.000 3.000 3.000 5.000 5.000 3.000
Standard Deviation 2.514 2.452 2.283 1.814 1.897 1.287 2.079 1.650 2.201 1.636
Sample Variance 6.322 6.011 5.211 3.289 3.600 1.656 4.322 2.722 4.844 2.678
Kurtosis -0.001 -0.778 -0.856 -0.232 -0.468 -0.430 -1.169 0.833 -0.410 -1.093
Skewness 0.856 0.659 0.503 0.369 0.600 0.164 0.206 -1.206 -0.309 0.350
Range 8.000 7.000 7.000 6.000 6.000 4.000 6.000 5.000 7.000 5.000
Minimum 2.000 1.000 2.000 1.000 2.000 1.000 1.000 1.000 1.000 2.000
Maximum 10.000 8.000 9.000 7.000 8.000 5.000 7.000 6.000 8.000 7.000
Sum 51.000 37.000 49.000 38.000 46.000 31.000 41.000 45.000 48.000 43.000
Count 10.000 10.000 10.000 10.000 10.000 10.000 10.000 10.000 10.000 10.000
Confidence Level(95.0%) 1.799 1.754 1.633 1.297 1.357 0.920 1.487 1.180 1.575 1.171
Number of data files assigned 8 12 7 9 5 9 6 6 8 7
Opportunity for Defects / File 10 10 10 10 10 10 10 10 10 10
Maximum No of Defects possible 80 120 70 90 50 90 60 60 80 70
% Yield 36.25 69.17 30.00 57.78 8.00 65.56 31.67 25.00 40.00 41.43
Defects per Million Opportunities, DPMO = 637500 308333.3 700000 422222.2 920000 344444.4 683333.3 750000 600000 585714.3
Short term Sigma Level of Resource 1.15 2 0.98 1.70 0.09 1.9 1.02 0.83 1.25 1.28
Long term Sigma Level of Resource -0.35 0.5 -0.52 0.2 -1.41 0.4 -0.48 -0.67 -0.25 -0.22
Average Short term Sigma Level of Team
Average Long term Sigma Level of Team
Calculation of Defects per File & Defects per Million Opportunities
Finding Short term and Long term Sigma Level (Calculated using a normal DPMO, Sigma Level calculator)
1.26
-0.24
RESULTS OF STATISTICAL ANALYSIS OF DATA OF DEFECTS
Cell in this colour shows Maximum value of Defect in the column for that Resource
Cell in this colour shows Minimum value of Defect in the column for that Resource
These results
are obtained
by applying
“Data Analysis”
function in
MS Excel to
Raw Data Matrix
These results
are obtained
by using std.
“Six Sigma
Calculator“
51 Defects for Resource R1 37 Defects for Resource R2 49 Defects for Resource R3 38 Defects for Resource R4

of 7
Defect Data of numbers for Resources R5..R10 transformed to graphics
Column chart showing all Defects D1..D10 for all Resources R1..R10
41 Defects for Resource R1041 Defects for Resource R1048 Defects for Resource R948 Defects for Resource R9
Defect D1 (45/427 =10..54%) Defect D2 (40/427 =9.37 %) Defect D3 (49/427 = 11.48%) Defect D4 (39/427=9.13 %)
Defect D5 (40/427 = 9.37%) Defect D6 (46/427 = 10.77 %) Defect D7 (34/427 = 7.96%) Defect D8 (37/427 = 8.67%)
Defect D10 (58/427 =13.58 %)Defect D9 (39/427= 9.13%)

of 7
Summary of Analysis
1. Number of resources used by ESP = 10 (R1...R10)
2. Number of Defects found by Customer = 427 (D1...D10)
3. Resource with Maximum number of Defects = R1 (51 Defects out of 427, i.e. 11.94%)
4. Resource with Minimum number of Defects = R6 (31 Defects out of 427, i.e. 7.26 %)
5. Resource with Maximum First time Yield, FTY = R5 (69.17%)
6. Resource with Minimum First time Yield, FTY = R2 (8.0 %)
7. Maximum Defect out of 10 types (D1...D10) = D10 (58 out of 427 i.e. 13.58%)
8. Minimum Defect out of 10 types (D1...D10) = D7 (34 out of 427 i.e. 7.96%)
9. Average short & long term Sigma Level of team = 1.26 & -0.24
10. Comparing short & long term Sigma Level of team with 6 Sigma level given in following table it can be seen that a very large effort will be
required to reach 6 Sigma level i.e. 3.4 DPMO.
Sigma level DPMO Percent defective Percentage yield Short-term C
pk
Long-term C
pk
1 691,462 69% 31% 0.33 –0.17
2 308,538 31% 69% 0.67 0.17
3 66,807 6.7% 93.3% 1.00 0.5
4 6,210 0.62% 99.38% 1.33 0.83
5 233 0.023% 99.977% 1.67 1. 17
6 3.4 0.00034% 99.99966% 2.00 1.5
7 0.019 0.0000019% 99.9999981% 2.33 1.83
CAPA -- Suggested Corrective and Preventive actions
Corrective actions: (What it was supposed to be? What actually was found? What needs to be corrected?)
1) Project Leader (PL)/Project Manager (PM) should present results of Defect Analysis done on pages 2 – 4 to the entire team.
2) PL/PM should conduct a Brainstorming session and RCA, Root cause Analysis (5Why’s &1H) for the defects in the meeting.
Root cause analysis should address questions related to Quality check system in the team i.e. If all the resources have knowledge of
Acceptance Quality Plan, AQP & customer’s definition of Defect? Why the defects were not noticed & corrected before data files
were uploaded to customer? Human errors -- Forgetfulness, Incorrect understanding of customer’s requirement, Defects due to
slowness and fatigue during the work.
3) After 2), concerned resources should check and correct all defects on their own in soft or hard copy and maintain a record.
4) PL/PM should ensure that all defects are put through a final toll gate quality check before uploading to customer. Inform details of
final upload to customer.
Preventive Actions: (Redefining execution processes to have in built detection and correction mechanism for Defects)
1) PL /PM should check and modify execution process by incorporating additional quality checks at appropriate stages.
2) Inform about modified execution processes to all members of the team.
3) Implement modified process for the next set of identical data from the customer.
4) Revisit competency inventory of resources. Explore possibility of various options of strengthening the team by imparting technical
training in the areas where maximum number of defects occurred. Training can be imparted by in-house or external faculty or
interaction with representative/s of customer through video conferencing or desktop sharing sessions.
Conclusion:
Process model of Raw Matrix containing data of Resources and defects has been analysed and information @ serial nos 1, 2, 3 & 4 on page 2
required by ESP has been provided using “Data Analysis” and column charting & pie graph functionalities in MS Excel.

of 7
Glossary of few Quality related terms and their definitions for better understanding of concepts
Defect:
Any characteristic of a Product or Service that
does not conform to its intended implicit or
explicit specification / requirement. Defect
could be biggest irritant for a Product / Service
for not achieving desired quality.
Defective:
Any Product or Service that contains at least
one defect and does not conform to or fails to
meet Customer’s implicit or explicit
specification / requirement.
Typically combination of Defects results into a
Defective Product. Functionality of a Product /
Service may be affected depending on the
impact of the defect. For example any fitment
defect in doors, windows, seats, headlamps
etc. may not affect movement of the car
during daytime but will cause discomfort to
passengers and pose a risk to the safety of car
and passengers during night and the car will
be termed as a defective car.
DPU: Defects Per Unit is the average number
of defects observed when sampling a
population.
Defects per million opportunities (DPMO):
Is the average number of defects per unit
observed during an average production run
divided by the number of opportunities to
make a defect on the product under study
during that run normalized to one million.
Cost Of Poor Quality (COPQ):
The costs incurred by organization for
correcting poor quality products or services.
These are incurred due to four factors:
Internal Failure Costs (Cost of rework incurred
for correcting non conformities / Defects
found before the customer receives the
Product or Service)
External failure costs (Costs incurred for
correcting non conformities / Defects found
after the customer receives the Product or
Service)
Appraisal costs (Costs incurred to determine
the degree of conformance to Quality
requirements) and
Prevention Costs (Costs incurred to keep
Failure and Appraisal costs to a minimum)
CTQ: Critical to Quality (Critical "Y") –
Attributes most important to the customer
which will directly impact Quality of Product /
Service.
Customer Needs, Expectations – Needs, as
defined by customers, which meet their basic
requirements and standards.
Voice of the Business (VOB): Represents the
needs of the business and the key
stakeholders of the business. It is usually items
such as profitability, revenue, growth, market
share, etc.
Voice of the Customer (VOC): Represents the
expressed and non-expressed needs, wants
and desires of the recipient of a process
output, a product or a service. It’s is usually
expressed as specifications, requirements or
expectations.
Voice of the Process (VOP): Represents the
performance and capability of a process to
achieve both business and customer needs. It
is usually expressed in some form of an
efficiency and/or effectiveness metric.
Six Sigma:
A data-driven method for achieving near
perfect Quality. Sigma is the Greek letter
used to denote standard deviation, or the
measure of variation from the mean, which
in production terms is used to imply a
Defect. Higher value of sigma indicates
fewer defects. In true Six Sigma
environments, companies operate at a
quality level of six standard deviations from
the mean or at 3.4 defect opportunities per
million operations (DPMO). Six Sigma
methodologies can be applied to Design,
Production or Service and Customer-
oriented activities. It is based on Statistical
Tools and Techniques of Quality
Management.
Poka-yoke:
A Japanese term that means mistake
proofing. A poka-yoke device is one that
prevents incorrect parts from being made or
assembled or easily identifies a flaw or error
Baka-yoke: A Japanese term for a
manufacturing technique for preventing
mistakes by designing the manufacturing
process, equipment and tools so an
operation literally cannot be performed
incorrectly. In addition for preventing
incorrect operation, the technique usually
provides a warning signal of some sort for
incorrect performance
FMEA:
Failure Mode and Effects Analysis: A
systematized group of activities to recognize
and evaluate the potential failure of a
product , service or process and its effects,
identify actions that could eliminate or
reduce the occurrence of the potential
failure and document the process.
DFMEA (Design Failure Mode and Effect
Analysis) is the application of the Failure
Mode and Effects Analysis (FMEA) method
specifically to product/service design
Error Modes and Effects Analysis (EMEA)
A procedure in which each potential error
made in every sub-process of a process is
analyzed to determine its effect on other
sub-processes and on the required accuracy
of the process.
An EMEA is also used to prioritize & rank
potential causes of process or human
failures as well as create, launch and
evaluate preventative actions.
Process Capability:
Product/Service that your process can
produce/deliver.
Control – The state of stability, normal
variation and predictability. Process of
regulating and guiding operations and
processes using quantitative data.
Process Audit:
A timely planned process or system of
inspection to ensure that Product / Service
specifications conform to documented
Quality standards. An Audit also brings out
discrepancies between the documented
standards and the standards followed and
also might show how well or how badly the
documented standards support the
processes currently followed.
Variance – A change in a process or business
practice that may alter its expected
outcome.
DFSS:
(Design for Six Sigma) is a systematic
methodology utilizing tools, training and
measurements to enable us to design
products and processes that meet
customer expectations of Quality , Cost,
and can be produced at Six Sigma quality
levels.
DMAIC:
(Define, Measure, Analyze, Improve and
Control) is a process for continued
improvement. It is systematic, scientific
and fact based. This closed-loop process
eliminates unproductive steps, often
focuses on new measurements, and
applies technology.
Process Mapping:
Illustrated description of how things get
done, which enables participants to
visualize an entire process and identify
areas of strength and weaknesses. It helps
reduce cycle time and defects while
recognizing the value of individual
contributions.
Root Cause Analysis:
Study of original reason for non-
conformance with a process. When the
root cause is removed or corrected, the
non-conformance will be eliminated.
Kurtosis:
A statistical measure used to describe the
distribution of observed data around the
mean.
Skewness:
Describes asymmetry from the normal
distribution in a set of statistical data.
Confidence Level: Extent to which an
assumption or number is likely to be true.
Standard deviation: It is a measure of
variability or diversity and shows how
much variation or 'dispersion' there is from
the 'average' (Mean or expected/budgeted
value)
Variation:
What the customer sees and feels.
Stable Operations: Ensuring consistent,
predictable processes to improve what the
customer sees and feels.
Mode: The value or item occurring most
frequently in a series of observations or
statistical data.
The most often occurring value in the data
set. A data set may contain more than one
mode, e.g., if there are exactly 2 values or
items that appear in the data the same
number of times, we say the data set is bi-
modal
Quality Function Deployment (QFD)
A systematic process used to integrate
customer requirements into every aspect
of the design and delivery of products and
services.
Range
A measure of the variability in a data set. It
is the difference between the largest and
smallest values in a data set.
Corrective & Preventive Actions:
Corrective actions are actions undertaken
to correct the observed non conformity /
Defect in a Product / Service.
Preventive Actions are the actions
undertaken to avoid the recurrence of non
– conformity / Defect in the Product /
Service in future.

Defect Analysis & Prevention, Data Mining & Visualization of Defect Matrix

More Related Content

What's hot

Similar to Defect Analysis & Prevention, Data Mining & Visualization of Defect Matrix

Recently uploaded

Defect Analysis & Prevention, Data Mining & Visualization of Defect Matrix