Defect Analysis & Prevention, Data Mining & Visualization of Defect Matrix

3,348 views

Published on

Published in: Business, Technology, Education
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,348
On SlideShare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
5
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Defect Analysis & Prevention, Data Mining & Visualization of Defect Matrix

  1. 1. By Aniruddha Sahasrabudhe Consultant, Engineering Process Excellence & Integration http://in.linkedin.com/pub/aniruddha-sahasrabudhe/8/a19/414 E mail: a_sahasrabudhe@hotmail.com November - 2010 Defect Analysis & Prevention Data Mining & Visualization of Defect Matrix using MS Excel
  2. 2. Page 2 of 7 Abstract For economic and operational reasons large engineering organizations outsource typical activities like conversion of existing drawings to 3d CAD models (using Pro Engineer or Catia or UG softwares), creation of manufacturing drawings, designing and detailing of BIW Fixtures, 3d models to Meshed models using Hypermesh or ANSA for FE Analysis, Reverse engineering of parts, CNC programming etc. to reputed Engineering Service Providers (ESPs). Expectation from ESPs is that in a short period of time they should deliver Defect free services, qualify as most favoured supplier of engineering services and become an integral part of supply chain of outsourcing organization. Generally the process begins with a pilot job i.e. a small part (5 to 10% of total work) is outsourced to ESPs with the necessary technical details i.e. Statement of work (SOW), Quality Standards/ Work instructions/Reference models, drawings, Acceptance Quality Plan (AQP), etc. Quality of pilot job delivered is one of the important criterion for shortlisting or approving ESP as a supplier of engineering services. It is assessed on the basis of Number of Defects found, Severity of defects (Minor or Major), First time yield (FTY) i.e. Number of Defect free jobs or files delivered / total number of jobs or files assigned, on time delivery within agreed cost, turnaround time for rework if any, number of resources used , speed, clarity and preciseness of communication etc. Feedback of Quality of pilot job delivered (number of defects, first time yield etc.) is given to the ESP for needful corrective action. In order to become an integral part of supply chain of outsourcing organization it is essential for ESP’s team to have a shared vision of Customer’s Critical Requirements, CCRs of Quality, Reliability, Manufacturing Process, Cost, Delivery period, Service etc. and a very clear understanding of definition of “Defect”. A very systematic and disciplined study of defect data and associated processes on part of ESP is required to ensure that future jobs will be defect free. Defect Analysis and Prevention is a very systematic process and its application varies from situation to situation. Every defect results into an expensive rework for the organization, adds to COPQ (Cost of Poor Quality) and directly impacts the bottom line – profits. It is therefore very essential for every organization to know and set KPI’s, Key Performance Indicators which reflect on effectiveness of organizational processes i.e. how they are performing in terms of Output /Unit of Input. If issues related to process Defect are not addressed at the right time they can escalate into a very difficult situation for the organization. Recent recalls of defective vehicles by Mercedes-Benz & Toyota are best examples of such situations -- Mercedes-Benz is recalling 85,000 of its 2010 C-Class and the 2010-11 E-Class models because a steering problem could make the vehicles difficult to control and --Toyota is recalling 3.8 million vehicles worldwide, including 740,000 vehicles in the United States that could leak brake fluid and cause stopping power to “gradually decline.” Recalled models are Avalon, Camry, ES 350, IS 250 and IS350, Tacoma, Tundra and Prius. Complex problems of this magnitude require complex process modelling & FMEA (Failure Mode and Effects Analysis) of Parts, Systems, Process, Service and Software and long-time to fix till a reliable solution is found and approved by a competent Traffic safety authority. This simple case study is about a situation (see “Overview of Situation“ on left) where an engineering organization found 427 defects in the pilot job of 3d modelling, detailing, meshing & CNC programming outsourced to an ESP. The analysis shows results of data mining of “Defect Matrix” using “Data Analysis pack “ and data visualization using column charts & pie graph functionalities in MS Excel. Defects D1, D2, D3 pertain to 3d modelling, D4 & D5 to detailing D6, D7 & D8 to Meshing & D9 &D10 to CNC programming. The results are useful for both the organizations. Outsourcing organization for comparing performance of ESPs & for ESP for doing “Gap Analysis” i.e. to know its current process capability (“As is “condition) and efforts required to comply with customer’s requirement -- Defect free service, “As should be” condition. A Picture is worth a thousand words (or Numbers) – For a better understanding and visualization, Defect numbers have been transformed into graphics and displayed in various ways with appropriate comments / conclusions. Based on results of analysis, corrective and preventive actions have been suggested to minimise chances of recurrence of same defects for identical jobs in future. A detailed technical discussion of 427 Defects D1….D10 is beyond the scope of this article hence not covered. It is presumed that reader of this article is conversant with application of misc. formulas, “Data Analysis pack” and charting functionalities of MS Excel, hence only results have been shown and discussed. - OVERVIEW OF SITUATION PROCESS MODEL: Raw Correlation Matrix with data of 10 types of Defects (D1..D10) for team of 10 Resources (R1..R10) A) Inputs from outsourcing engineering organization: -- Statement of Work (SOW) and CD containing 77 data files for pilot job of 3d Modelling, Detailing, Meshing, “Acceptance Quality Plan (AQP) “ and some reference examples for each of these tasks was given to ESP. B) Deliverables expected from ESP: 3DModels, Manufacturing drgs, Meshed models & CNC programs as per “AQP“within agreed delivery time. C) ESP’s estimation shown that to complete the pilot job as per delivery time requirement of outsourcing organization, a team of 10 resources (R1...R10) will be required. ESP checked competency inventory and selected 10 resources who worked on the data and delivered the pilot job within agreed delivery time. D) Quality of pilot job: Outsourcing organization checked quality of pilot job and found a total of 427 Defects which were informed to ESP. The data of 427 defects has been co related with the resources and arranged into 10 types of unique “Defect categories (D1...D10) as shown in the matrix on page 2. ESP is required to correct the defects and upload corrected data as soon as possible. Requirement of Engineering Service Provider, ESP 1) A detailed analysis of Defect data in the matrix on page 3 and get an idea of the performance of each resource and the team using “Data Analysis” pack in MS Excel since this is readily available with him. 2) Graphic visualization of raw correlation Matrix for better understanding of Defect data. 3) Process Capability & Gap Analysis: An idea of current Process Capability & Sigma Level of the team & compare it with desired 6 Sigma level (3.4 Defects per Million Opportunities) and 4) CAPA -- Corrective and Preventive Action/s Action that need to be taken to Correct detected Non conformity (427 Defects) and Preventive action to avoid recurrence of Defects for future jobs.
  3. 3. Page 3 of 7 Raw Correlation Matrix of Resources R1..R10 & Defects D1..D10 Data Mining from Raw Correlation Matrix showing meaningful information for each resource & the team Above matrix displays Maximum and Minimum number of type of Defect, Total number of Defects for each resource & the team. Performance of Team --Defect numbers in above table are transformed into graphics for better visualization & understanding Conclusions from Data Mining table & Pie graphs Resource with Maximum Defects Maximum of total Defects = R1 (51 out of 427, 11.94%) = D10 (58 out of 427, 13.58 %) Resource with Minimum Defects Minimum of total Defects = R6 (31 out of 427, 7.26%) = D7 (34 out of 427, 7.96 %) Team Resources → R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 Defect types ↓ D1 3 5 7 3 8 3 1 5 5 5 D2 5 2 3 7 4 4 3 3 7 2 D3 8 3 9 3 5 2 7 1 8 3 D4 4 1 6 3 6 5 3 6 2 3 D5 7 3 2 5 4 1 2 5 7 4 D6 3 7 3 6 2 3 6 5 4 7 D7 5 1 4 4 4 3 3 5 1 4 D8 2 2 7 2 3 2 5 6 4 4 D9 4 5 3 1 3 5 7 3 5 3 D10 10 8 5 4 7 3 4 6 5 6 Additional available Information Number of data files assigned 8 12 7 9 5 9 6 6 8 7 Opportunity for Defects / File 10 10 10 10 10 10 10 10 10 10 Maximum No of Defects possible 80 120 70 90 50 90 60 60 80 70 TeamResources → R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 Total of % of Total Number Defect type ↓ Defects in Row of Defects D1 3 5 7 3 8 3 1 5 5 5 45 10.54 D2 5 2 3 7 4 4 3 3 7 2 40 9.37 D3 8 3 9 3 5 2 7 1 8 3 49 11.48 D4 4 1 6 3 6 5 3 6 2 3 39 9.13 D5 7 3 2 5 4 1 2 5 7 4 40 9.37 D6 3 7 3 6 2 3 6 5 4 7 46 10.77 D7 5 1 4 4 4 3 3 5 1 4 34 7.96 D8 2 2 7 2 3 2 5 6 4 4 37 8.67 D9 4 5 3 1 3 5 7 3 5 3 39 9.13 D10 10 8 5 4 7 3 4 6 5 6 58 13.58 Total numberof Defects forResource =SUM(D1:D10) 51 37 49 38 46 31 41 45 48 41 427 100.00 % of Total Defect by Resource =((SUM(D1:D10)/427)*100 11.94 8.67 11.48 8.90 10.77 7.26 9.60 10.54 11.24 9.60 100.00 Colourindication Cell in this colourshows Maximumvalue of Defect in the column forthat Resource Cell in this colourshows Minimumvalue of Defect in the column forthat Resource 0 2 4 6 8 10 12 R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 D1 D2 D3 D4 D5 D6 D7 D8 D9 D10 Pie graph 1 showing % contribution of each resource to total of 427 Defects Pie graph 2 showing all Defects (D1..D10) & their % contribution to total of 427 Defects.
  4. 4. Page 4 of 7 Defect Matrix Transformed to Matrix of Statistically important numbers & information for all Resources Note: % Yield = (1-(Total number of defects of resource / Maximum number of defects possible))*100. Defect Data of numbers for Resources R1..R4 transformed to graphics Team Resources → R1 R2 R3 R4 R5 R6 R7 R8 R9 R10 Total of % of Total Number Defect type ↓ Defects in Row of Defects D1 3 5 7 3 8 3 1 5 5 5 45 10.54 D2 5 2 3 7 4 4 3 3 7 2 40 9.37 D3 8 3 9 3 5 2 7 1 8 3 49 11.48 D4 4 1 6 3 6 5 3 6 2 3 39 9.13 D5 7 3 2 5 4 1 2 5 7 4 40 9.37 D6 3 7 3 6 2 3 6 5 4 7 46 10.77 D7 5 1 4 4 4 3 3 5 1 4 34 7.96 D8 2 2 7 2 3 2 5 6 4 4 37 8.67 D9 4 5 3 1 3 5 7 3 5 3 39 9.13 D10 10 8 5 4 7 3 4 6 5 6 58 13.58 Total number of Defects for Resource = SUM(D1:D10) 51 37 49 38 46 31 41 45 48 41 427 100.00 % of Total Defect by Resource = ((SUM(D1:D10)/427)*100 11.94 8.67 11.48 8.90 10.77 7.26 9.60 10.54 11.24 9.60 100.00 Colour indication Mean 5.100 3.700 4.900 3.800 4.600 3.100 4.100 4.500 4.800 4.300 Standard Error 0.795 0.775 0.722 0.573 0.600 0.407 0.657 0.522 0.696 0.517 Median 4.500 3.000 4.500 3.500 4.000 3.000 3.500 5.000 5.000 4.000 Mode 3.000 5.000 3.000 3.000 4.000 3.000 3.000 5.000 5.000 3.000 Standard Deviation 2.514 2.452 2.283 1.814 1.897 1.287 2.079 1.650 2.201 1.636 Sample Variance 6.322 6.011 5.211 3.289 3.600 1.656 4.322 2.722 4.844 2.678 Kurtosis -0.001 -0.778 -0.856 -0.232 -0.468 -0.430 -1.169 0.833 -0.410 -1.093 Skewness 0.856 0.659 0.503 0.369 0.600 0.164 0.206 -1.206 -0.309 0.350 Range 8.000 7.000 7.000 6.000 6.000 4.000 6.000 5.000 7.000 5.000 Minimum 2.000 1.000 2.000 1.000 2.000 1.000 1.000 1.000 1.000 2.000 Maximum 10.000 8.000 9.000 7.000 8.000 5.000 7.000 6.000 8.000 7.000 Sum 51.000 37.000 49.000 38.000 46.000 31.000 41.000 45.000 48.000 43.000 Count 10.000 10.000 10.000 10.000 10.000 10.000 10.000 10.000 10.000 10.000 Confidence Level(95.0%) 1.799 1.754 1.633 1.297 1.357 0.920 1.487 1.180 1.575 1.171 Number of data files assigned 8 12 7 9 5 9 6 6 8 7 Opportunity for Defects / File 10 10 10 10 10 10 10 10 10 10 Maximum No of Defects possible 80 120 70 90 50 90 60 60 80 70 % Yield 36.25 69.17 30.00 57.78 8.00 65.56 31.67 25.00 40.00 41.43 Defects per Million Opportunities, DPMO = 637500 308333.3 700000 422222.2 920000 344444.4 683333.3 750000 600000 585714.3 Short term Sigma Level of Resource 1.15 2 0.98 1.70 0.09 1.9 1.02 0.83 1.25 1.28 Long term Sigma Level of Resource -0.35 0.5 -0.52 0.2 -1.41 0.4 -0.48 -0.67 -0.25 -0.22 Average Short term Sigma Level of Team Average Long term Sigma Level of Team Calculation of Defects per File & Defects per Million Opportunities Finding Short term and Long term Sigma Level (Calculated using a normal DPMO, Sigma Level calculator) 1.26 -0.24 RESULTS OF STATISTICAL ANALYSIS OF DATA OF DEFECTS Cell in this colour shows Maximum value of Defect in the column for that Resource Cell in this colour shows Minimum value of Defect in the column for that Resource These results are obtained by applying “Data Analysis” function in MS Excel to Raw Data Matrix These results are obtained by using std. “Six Sigma Calculator“ 51 Defects for Resource R1 37 Defects for Resource R2 49 Defects for Resource R3 38 Defects for Resource R4 51 Defects for Resource R1 37 Defects for Resource R2 49 Defects for Resource R3 38 Defects for Resource R4
  5. 5. Page 5 of 7 Defect Data of numbers for Resources R5..R10 transformed to graphics Column chart showing all Defects D1..D10 for all Resources R1..R10 41 Defects for Resource R1041 Defects for Resource R1048 Defects for Resource R948 Defects for Resource R9 46 Defects for Resource R5 31 Defects for Resource R6 41 Defects for Resource R7 45 Defects for Resource R8 46 Defects for Resource R5 31 Defects for Resource R6 41 Defects for Resource R7 45 Defects for Resource R8 Defect D1 (45/427 =10..54%) Defect D2 (40/427 =9.37 %) Defect D3 (49/427 = 11.48%) Defect D4 (39/427=9.13 %) Defect D5 (40/427 = 9.37%) Defect D6 (46/427 = 10.77 %) Defect D7 (34/427 = 7.96%) Defect D8 (37/427 = 8.67%) Defect D10 (58/427 =13.58 %)Defect D9 (39/427= 9.13%)
  6. 6. Page 6 of 7 Summary of Analysis 1. Number of resources used by ESP = 10 (R1...R10) 2. Number of Defects found by Customer = 427 (D1...D10) 3. Resource with Maximum number of Defects = R1 (51 Defects out of 427, i.e. 11.94%) 4. Resource with Minimum number of Defects = R6 (31 Defects out of 427, i.e. 7.26 %) 5. Resource with Maximum First time Yield, FTY = R5 (69.17%) 6. Resource with Minimum First time Yield, FTY = R2 (8.0 %) 7. Maximum Defect out of 10 types (D1...D10) = D10 (58 out of 427 i.e. 13.58%) 8. Minimum Defect out of 10 types (D1...D10) = D7 (34 out of 427 i.e. 7.96%) 9. Average short & long term Sigma Level of team = 1.26 & -0.24 10. Comparing short & long term Sigma Level of team with 6 Sigma level given in following table it can be seen that a very large effort will be required to reach 6 Sigma level i.e. 3.4 DPMO. Sigma level DPMO Percent defective Percentage yield Short-term C pk Long-term C pk 1 691,462 69% 31% 0.33 –0.17 2 308,538 31% 69% 0.67 0.17 3 66,807 6.7% 93.3% 1.00 0.5 4 6,210 0.62% 99.38% 1.33 0.83 5 233 0.023% 99.977% 1.67 1. 17 6 3.4 0.00034% 99.99966% 2.00 1.5 7 0.019 0.0000019% 99.9999981% 2.33 1.83 CAPA -- Suggested Corrective and Preventive actions Corrective actions: (What it was supposed to be? What actually was found? What needs to be corrected?) 1) Project Leader (PL)/Project Manager (PM) should present results of Defect Analysis done on pages 2 – 4 to the entire team. 2) PL/PM should conduct a Brainstorming session and RCA, Root cause Analysis (5Why’s &1H) for the defects in the meeting. Root cause analysis should address questions related to Quality check system in the team i.e. If all the resources have knowledge of Acceptance Quality Plan, AQP & customer’s definition of Defect? Why the defects were not noticed & corrected before data files were uploaded to customer? Human errors -- Forgetfulness, Incorrect understanding of customer’s requirement, Defects due to slowness and fatigue during the work. 3) After 2), concerned resources should check and correct all defects on their own in soft or hard copy and maintain a record. 4) PL/PM should ensure that all defects are put through a final toll gate quality check before uploading to customer. Inform details of final upload to customer. Preventive Actions: (Redefining execution processes to have in built detection and correction mechanism for Defects) 1) PL /PM should check and modify execution process by incorporating additional quality checks at appropriate stages. 2) Inform about modified execution processes to all members of the team. 3) Implement modified process for the next set of identical data from the customer. 4) Revisit competency inventory of resources. Explore possibility of various options of strengthening the team by imparting technical training in the areas where maximum number of defects occurred. Training can be imparted by in-house or external faculty or interaction with representative/s of customer through video conferencing or desktop sharing sessions. Conclusion: Process model of Raw Matrix containing data of Resources and defects has been analysed and information @ serial nos 1, 2, 3 & 4 on page 2 required by ESP has been provided using “Data Analysis” and column charting & pie graph functionalities in MS Excel.
  7. 7. Page 7 of 7 Glossary of few Quality related terms and their definitions for better understanding of concepts Defect: Any characteristic of a Product or Service that does not conform to its intended implicit or explicit specification / requirement. Defect could be biggest irritant for a Product / Service for not achieving desired quality. Defective: Any Product or Service that contains at least one defect and does not conform to or fails to meet Customer’s implicit or explicit specification / requirement. Typically combination of Defects results into a Defective Product. Functionality of a Product / Service may be affected depending on the impact of the defect. For example any fitment defect in doors, windows, seats, headlamps etc. may not affect movement of the car during daytime but will cause discomfort to passengers and pose a risk to the safety of car and passengers during night and the car will be termed as a defective car. DPU: Defects Per Unit is the average number of defects observed when sampling a population. Defects per million opportunities (DPMO): Is the average number of defects per unit observed during an average production run divided by the number of opportunities to make a defect on the product under study during that run normalized to one million. Cost Of Poor Quality (COPQ): The costs incurred by organization for correcting poor quality products or services. These are incurred due to four factors: Internal Failure Costs (Cost of rework incurred for correcting non conformities / Defects found before the customer receives the Product or Service) External failure costs (Costs incurred for correcting non conformities / Defects found after the customer receives the Product or Service) Appraisal costs (Costs incurred to determine the degree of conformance to Quality requirements) and Prevention Costs (Costs incurred to keep Failure and Appraisal costs to a minimum) CTQ: Critical to Quality (Critical "Y") – Attributes most important to the customer which will directly impact Quality of Product / Service. Customer Needs, Expectations – Needs, as defined by customers, which meet their basic requirements and standards. Voice of the Business (VOB): Represents the needs of the business and the key stakeholders of the business. It is usually items such as profitability, revenue, growth, market share, etc. Voice of the Customer (VOC): Represents the expressed and non-expressed needs, wants and desires of the recipient of a process output, a product or a service. It’s is usually expressed as specifications, requirements or expectations. Voice of the Process (VOP): Represents the performance and capability of a process to achieve both business and customer needs. It is usually expressed in some form of an efficiency and/or effectiveness metric. Six Sigma: A data-driven method for achieving near perfect Quality. Sigma is the Greek letter used to denote standard deviation, or the measure of variation from the mean, which in production terms is used to imply a Defect. Higher value of sigma indicates fewer defects. In true Six Sigma environments, companies operate at a quality level of six standard deviations from the mean or at 3.4 defect opportunities per million operations (DPMO). Six Sigma methodologies can be applied to Design, Production or Service and Customer- oriented activities. It is based on Statistical Tools and Techniques of Quality Management. Poka-yoke: A Japanese term that means mistake proofing. A poka-yoke device is one that prevents incorrect parts from being made or assembled or easily identifies a flaw or error Baka-yoke: A Japanese term for a manufacturing technique for preventing mistakes by designing the manufacturing process, equipment and tools so an operation literally cannot be performed incorrectly. In addition for preventing incorrect operation, the technique usually provides a warning signal of some sort for incorrect performance FMEA: Failure Mode and Effects Analysis: A systematized group of activities to recognize and evaluate the potential failure of a product , service or process and its effects, identify actions that could eliminate or reduce the occurrence of the potential failure and document the process. DFMEA (Design Failure Mode and Effect Analysis) is the application of the Failure Mode and Effects Analysis (FMEA) method specifically to product/service design Error Modes and Effects Analysis (EMEA) A procedure in which each potential error made in every sub-process of a process is analyzed to determine its effect on other sub-processes and on the required accuracy of the process. An EMEA is also used to prioritize & rank potential causes of process or human failures as well as create, launch and evaluate preventative actions. Process Capability: Product/Service that your process can produce/deliver. Control – The state of stability, normal variation and predictability. Process of regulating and guiding operations and processes using quantitative data. Process Audit: A timely planned process or system of inspection to ensure that Product / Service specifications conform to documented Quality standards. An Audit also brings out discrepancies between the documented standards and the standards followed and also might show how well or how badly the documented standards support the processes currently followed. Variance – A change in a process or business practice that may alter its expected outcome. DFSS: (Design for Six Sigma) is a systematic methodology utilizing tools, training and measurements to enable us to design products and processes that meet customer expectations of Quality , Cost, and can be produced at Six Sigma quality levels. DMAIC: (Define, Measure, Analyze, Improve and Control) is a process for continued improvement. It is systematic, scientific and fact based. This closed-loop process eliminates unproductive steps, often focuses on new measurements, and applies technology. Process Mapping: Illustrated description of how things get done, which enables participants to visualize an entire process and identify areas of strength and weaknesses. It helps reduce cycle time and defects while recognizing the value of individual contributions. Root Cause Analysis: Study of original reason for non- conformance with a process. When the root cause is removed or corrected, the non-conformance will be eliminated. Kurtosis: A statistical measure used to describe the distribution of observed data around the mean. Skewness: Describes asymmetry from the normal distribution in a set of statistical data. Confidence Level: Extent to which an assumption or number is likely to be true. Standard deviation: It is a measure of variability or diversity and shows how much variation or 'dispersion' there is from the 'average' (Mean or expected/budgeted value) Variation: What the customer sees and feels. Stable Operations: Ensuring consistent, predictable processes to improve what the customer sees and feels. Mode: The value or item occurring most frequently in a series of observations or statistical data. The most often occurring value in the data set. A data set may contain more than one mode, e.g., if there are exactly 2 values or items that appear in the data the same number of times, we say the data set is bi- modal Quality Function Deployment (QFD) A systematic process used to integrate customer requirements into every aspect of the design and delivery of products and services. Range A measure of the variability in a data set. It is the difference between the largest and smallest values in a data set. Corrective & Preventive Actions: Corrective actions are actions undertaken to correct the observed non conformity / Defect in a Product / Service. Preventive Actions are the actions undertaken to avoid the recurrence of non – conformity / Defect in the Product / Service in future.

×