This document presents a case study on optimizing manufacturing test times for printed circuit boards using the DMAIC (Define, Measure, Analyze, Improve, Control) process at a Cisco manufacturing facility. The optimization project reduced advance flying probe test times by over 50% without compromising product quality. Statistical analysis over 6 months confirmed the time savings. The test time optimization provided a significant return on investment. The case study demonstrates how DMAIC can be effectively used for manufacturing process improvement.

1. MANUFACTURING TEST TIME OPTIMIZATION CASE STUDY:
BY THE SIX SIGMA DMAIC PROCESS
1
Lee Kong Hui and 2
Jason Ho
Cisco Systems (Malaysia) Sdn. Bhd.,
Penang, MALAYSIA.
1
konlee@cisco.com; 2
jasho@cisco.com
ABSTRACT
Lean manufacturing has been the buzzword and gained a lot
of interest in the industry since the last decade. It has
become a phenomenon in the industry and a widely
accepted fact that operation excellence will be the key focus
area to provide the leading advantage against the various
competitors.
This paper presented a case study on manufacturing test
time optimization using DMAIC approach. Successful
restructuring of an advance flying probe (AFP) test program
had enabled the test time reduction by more than 50.0%.
Statistical analysis and six sigma (6σ) first pass yield (FPY)
monitoring for 6 months after new test program
implementation had confirmed that the manufacturing test
time optimization was achieved without compromising the
product quality. In correlation to the manufacturing test time
optimization achieved, it came along with a very significant
return of investment (ROI) for the effort put in. The case
study shared here is evidenced of DMAIC as a structured
tool that can be effectively used for manufacturing
processes improvement opportunities.
Key words: Lean Manufacturing, Test Time Optimization,
DMAIC, Six Sigma, Return of Investment (ROI)
INTRODUCTION
Nowadays, all enterprises have to speed up their paces to
meet customer demands in an ever changing and innovative
era. Manufacturing operation efficiency of the equipment or
testers used during a product assembly and functional test
always represent a major economic stake for their business
concern. The main challenges and the sources of
ineffectiveness live in the choice of the equipment or tester
operation, usage and running time especially when the
machine plays a vital role in the manufacturing processes or
functional test coverage to maximize output and maintain
product quality. For this purpose, to remain competitive in
the market and ensure survival of the businesses, be credible
and contributing to the success of the company, one must
continually adapt to the advancement of technical areas,
processes simplification and waste elimination, while at the
same time uphold the quality and reliability of its product.
Lean 6σ via the 5 steps DMAIC (Define-Measure-Analyze-
Improve-Control) is a method commonly used in the
industry to simplify the manufacturing processes and to
improve quality based on mastering statically controlled
manufacturing related processes. It is often described as an
approach to trouble shoot manufacturing related issues and a
management style that is based on a highly regulated
organization dedicated to managing improvement project [1,
2]. Accordingly, the much improved production efficiency
will increase output and ultimately yield significant positive
financial benefits.
The presented case study here was an actual improvement
project selected from a few initiatives that were successfully
completed at one of our manufacturing partner sites in 2015
with significant test time reduction and annual cost saving
to the company. This work aims to study the 6σ DMAIC
method from the perspective of scientific theories in the
field of problem solving as had been published in the
operations research, management science and Industrial
Engineering literatures [1, 3 to 7], while at the same time
proven with actual manufacturing achievement. This case
study had helped the authors to develop an in-depth
understanding of the DMAIC method from a goal-theoretic
perspective with its feasibility and practical application
evidenced from the test time reduction and cost saving
achieved. The discussion herein will be based on the
standard DMAIC structure practiced within Cisco Systems
Inc. highlighting a few key area of interest, while at the
same time only briefly touch on those intellectual property
right related matters to avoid legal conflict. Opportunity for
advance flying probe test time reduction had been
confirmed as the area of interest, in which the measurement
metrics will be based on advance flying probe test first pass
yield (FPY) and unit output per hour (UPH).
In addition to the significant test time reduction and annual
cost saving achieved, the case study presented was also
submitted and certified as a Green Belt project. DMAIC has
gained increasing favor to be implemented as an effective
approach that can provide convincing results.
METHODOLOGY
The methodology used in this study is DMAIC with the
consideration of 6 months base line data. In a brief and
concise structure, DMAIC methodology follows below
phases in a sequential manner. The determination of each
step’s output is supported by both statistical and non-
statistical tools. Methods were chosen according to the
effect of the tools that can give the most significant,
accurate and comprehensive representation of the data. The
5 phases of DMAIC methodology are:
Proceedings of the South East Asia Technical Training Conference on Electronics Assembly, 2016, SMTA.

2. Define - Establish the S.M.A.R.T. Goal
Measure - Document the baseline measurement
Analyze - Identify the root cause
Improve - Select and implement a solution
Control - Sustain the gains
RESULTS AND DISCUSSION
Define Phase
In the Define phase, a 6σ team identifies its problem, creates
the problem statement, confirm customer expectations,
perform process mapping to identify area that is critical to
quality (CTQ), document the SMART goals and construct
the Project Charter. A SMART goal is a goal that is
Specific, Measurable, Achievable, Relevant and Time
bound.
Aside from the Project Charter, Voice of the Customers
(VOC) or Stakeholder Analysis, SMART goals
identification, Project Timeline and Saving, there are 2 main
outcomes from the Define Phase that is critical in
contributing to the success of problem solving. They are the
SIPOC (Supplier, Input, Process, Output and Customers)
and CTQ analysis. Table 1 is showing the SIPOC analysis
of the selected project. SIPOC analysis can help to identify
the key elements suppliers need to provide to each process
for each operation to meet optimized manufacturing
requirement. Along the process activities or operations,
advance flying probe test determine the start and end points
associated with the problem and the major steps in solving
the issue. With the output target and receiving customers
visible, it helps to provide a clear sense of direction to the
team.
Table 2 is showing some of the main requirement that our
team need to fulfill with their respective measurement
metrics in order to minimize or eliminate possible risk that
can impact the achievement of our preset goals.
Table 1: SIPOC analysis of the selected project.
Process Start at: Test Script Development Process End at: Capacity utilization reduction or cost saving
achieved
SSuupppplliieerrss IInnppuuttss
PPrroocceessss AAccttiivviittiieess //
OOppeerraattiioonnss
OOuuttppuuttss CCuussttoommeerrss
Product Operation Test parameter Test script / test
coverage
Test yield and reliability Global Manufacturing
Operation (Test
Development Engineer)
Manufacturing Partner
Test Engineering
Test program setup Production test ETE yield and test time Global Manufacturing
Operation (Test
Development Engineer
and Manufacturing
Engineer)
Manufacturing Partner
Test Engineering
Test time Capacity analysis Capacity utilization Manufacturing Partner
Business Unit
Table 2: Critical to Quality.
Measure Phase
Executing the Measure Phase means establishing project
metrics and documents the base line. Establishing project
metrics can help to quantify improvement, measure
customer satisfaction and track financial performance. In
parallel, documenting the base line can help to provide a
visual representation of the process aids in seeing all steps,
map the process as it is and capture the accompanying
descriptive data in detail. Data collected in the measure
phase will be processed in the Analyze Phase.
Process mapping in the Measure Phase had confirmed that
advance flying probe test time was very long for each cycle
and our manufacturing partner only has one advance flying
probe tester qualified for a few business units or product
families. Line of investigation further identified that test
capacity constraint and machine down time were the 2 main
contributors that had presented a problem statement worth
to explore with an optimization initiative. Advance flying
probe test FPY data collected for 2 models over a period of
CUSTOMER Requirement Metric Goal / Risk
Supply Planner
High quality throughput % of ETE FPY ~98.0% FPY on advance flying
probe test. Reduce variation.
Fast turnaround time ≥50% advance flying probe test
time reduction
99% accuracy. Reduce
variation.
Customer (Direct Fulfillment) Deliver product on time On time shipment 99%. Reduce mean.
Proceedings of the South East Asia Technical Training Conference on Electronics Assembly, 2016, SMTA.

3. 4 months (June till September’14) had shown very good
yield achieving 100.0% most of the time (Figure 1 and 2).
Stable and good FPY status had provided good opportunity
for advance flying probe test time reduction. This was
captured as the base line that requires continuous review
upon completion of the test time reduction and not to
achieve it at the expense of product quality.
Subsequently, potential root cause had been confined to
advance flying probe tester preventative maintenance
effectiveness and advance flying probe test program logic
and program setup as had been shown in Table 3. This was
based on the analysis done on the process mapping,
preventative maintenance log book, test technology and test
program coverage.
Figure 1: Model 1 advance flying probe FPY from June 2014 until January 2015.
Figure 2: Model 2 advance flying probe FPY from June 2014 until January 2015.
Proceedings of the South East Asia Technical Training Conference on Electronics Assembly, 2016, SMTA.

4. Table 3: Potential root causing
Analyze Phase
Basically, the Analyze Phase can be divided into 2
categories, which are Process Analysis and Data Analysis.
The intended outcomes of the Analyze Phase are to identify
the potential root cause(s), group the root cause(s) and
isolate the probable root cause(s). There are a few methods
that can be used for root cause analysis, such as run chart,
scatter plot, cause and effect diagrams, 5-Whys, process
mapping, pareto chart and etc. The fish bone diagram and 5-
Whys approach will be dealt with detail in this case study.
In the Analyze Phase, there are 5 major steps we had
performed to identify the source of the problem. The most
basic cause(s) that can reasonably be identified, when fixed,
will prevent or significantly reduce the likelihood of the
problem recurrence.
1. Benchmarking
2. Root cause analysis- Qualitative
3. Root cause analysis- Quantitative
4. Process map analysis
5. Root cause summary
The benchmarking step was performed to gauge and
compare the current initiatives feasibility and to establish
significant confident when handling the initiatives. Indeed,
any prior experience or success examples can provide a
good sense of confident and direction. Due to the concern
on Intellectual Property right sensitivity, we will only focus
on Step 2 and 3 in the Analyze Phase. Figure 3 Fish bone
Diagram with 4M+2E methods and Table 4, 5-Whys
approach used in the qualitative root cause analysis had
identified advance flying probe tester program setup or test
sequence limitation, test coverage and detection limit as the
root cause of the problem. Both the selected models FPY
had achieved 6σ goal target and is stable to allow us pursue
test time reduction opportunities. Although, there is one
failure happened on Model 1 captured in August’14 Wk01,
this is not detrimental or causing any impact to product
quality be it in the manufacturing shop floor or in the field
as the failure was detected, reworked and retested prior to
the product shipment.
Figure 3: Fish bone diagram used in the qualitative root cause analysis.
Potential Root Cause Supporting Data Information source Validation step
Advance flying probe tester
PM effectiveness
Multiple party involved at the
same time
Process Map and PM log book Detail Process Map analysis
with Role
Advance flying probe test
program logic and program
setup
Data trend on each product
family varies
Test Technology or Test
program coverage
Comparison of Technology
Proceedings of the South East Asia Technical Training Conference on Electronics Assembly, 2016, SMTA.

5. Table 4: 5-Whys approach used in the qualitative root cause analysis.
Improve Phase
The Improve Phase was functioned through a strong brain
storming session with the respective area or functional team
members leading to a few possible solutions. All the team
member ideas were gathered for further discussion to
identify the best possible solutions. Cost and benefits of the
solution were considered in detail before their final selection
and implementation.
Each solution was discussed and assessed how they address
the root cause(s) discovered in the Analyze Phase. Solutions
were evaluated and ranked according to their effectiveness,
the ease of implementation and the magnitude of them
addressing the root cause(s). Solutions were selected based
on collective decision from the team. Changing the advance
flying probe test sequence had been identified and selected
as the best possible option. It is very promising that this
solution does not involve any resources investment or cause
any visible drop in FPY trend. However, technical detail of
the changes will not be discussed in-depth in this case study.
Table 5: Solution risk assessment for the implemented improvement solutions.
Note: N/A – Not applicable
Proceedings of the South East Asia Technical Training Conference on Electronics Assembly, 2016, SMTA.

6. Figure 4: t-test analysis for Model 1, Prob > /t/ <0.0001. Unit calculated in minutes.
Figure 4: t-test analysis for Model 1, Prob > /t/ <0.0001. Unit calculated in minutes.
Figure 5: t-test analysis for Model 2, Prob > /t/ <0.0001. Unit calculated in minutes.
For the solution risk assessment, it was completed
leveraging the use of a modified FMEA template (Table 5).
Three failure modes were identified with the highest risk
probability number ranging from 28 to 48. They are listed
below.
1. Wrong test program setup sequence
2. Wrong test program coverage or detection was
setup
3. Wrong detection limit was defined in the test
program
Based on the Control Plan implemented, the solution risk
identified were considered as manageable and did not post
any possible exposure as they will be detected during the
manufacturing stage and corrected in the production floor
prior to the finish good shipment.
Under the Improve Phase, a Student's t-test was also used to
analyze and confirm the significant impact of the
solution implemented. Student's t-test is a statistical
hypothesis test in which the test statistic follows a Student’s
t-distribution if the null hypothesis (Advance flying probe
test time change) is supported. Figure 4 and 5 t-test analysis
for both Model 1 and 2 have Prob > /t/ <0.0001, which had
confirmed there were significant impact from the solution
implemented. Advance flying probe test time was reduced
by more than half in both cases. Unit per hour (UPH) output
for Model 1 had increased from 0.4 to 1.5 units, while
Model 2 had increased from 2.9 to 6.0 units.
Control Phase
From the literatures review done, there have been a few
authors stressing the importance of Control Phase and
solution standardization to ensure the long term success of
Proceedings of the South East Asia Technical Training Conference on Electronics Assembly, 2016, SMTA.

7. their projects [8, 9]. Improving the metrics and solving the
issue for the time being will be meaningless if it is not
sustainable in the long run. Control Phase execution
involved 4 basic fundamental approaches, which are:
1. Control and standardize the solution.
2. Identify the long-term owner.
3. Develop and deploy the transition plan.
4. Close the improvement activity.
Under the control selection matrix, management score card
and review, automated system check and training and
operators certification were selected and implemented as the
control plan. Control plans selected are capable of
addressing solution risk as had been identified in the
Improve Phase. In addition to solution risk mitigation,
control plans selected also helped to ensure the
sustainability of the long term solution. Automated system
check is always a preferred choice of control plan as it is a
robust detection process that is not affected by human error
or work fatigue.
All the other 3 elements were not discussed in detail here as
they were just the routine project management attributes.
CONCLUSION
Successful implementation and growing organizational
interest in 6σ DMAIC method have been exploding in the
last few years. It is rapidly becoming a major driving force
for many technology-driven and project-driven
organizations [10]. Given the nature of this kind of
improvement project available data, some classical 6σ and
DMAIC tools have been tested and applied since the
beginning until initiative completion for the presented case
study. This case study shows that DMAIC is a structured
and an effective tool that can help continuous improvement
practitioners handle their project effectively until successful
closure.
Statistical data analysis had shown strong correlation
between Advance flying probe tester program setup or test
sequence optimization with significant Advance flying
probe test time reduction. Based on the financial benefit
gained from the completed case study (>$24k USD cost
saving) per annum in addition to the significant increase of
production output, it is very encouraging to the authors to
embark on another similar new improvement initiative. The
statistical aspect of 6σ is very much data driven and must
always complement business perspectives and challenges to
the best of a company interest to implement 6σ projects
successfully. At the same time, management involvement,
organizational commitment, project management skills,
continuous improvement mentality and continuous training
are also important influencing factors to the successful
completion of any valuable 6σ DMAIC projects.
ACKNOWLEDGEMENTS
The authors would like to express our sincere gratitude to
John Galang for his support and sponsorship to carry out
this 6σ Green Belt project successfully. Both Ted Roy and
Kah Keong Leong were also gratefully acknowledged for
their help in reviewing and proof reading this manuscript.
REFERENCES
[1] Mast, J. D. and Lokkerbol, J. (2012). An analysis of the
Six Sigma DMAIC method from the perspective of
problem solving, Int. J. Production Economics, 139, 604-
604-614.
[2] Youssouf, A., Rachid, C. and Ion, V. (2014).
Contribution to The Optimization of Strategy of
Maintenance by Lean Six Sigma, Physics Procedia, 55,
512 – 518.
[3] Gowen III, C. R. and Tallon, W. J. (2005). Effect of
technological intensity on the relationships among Six
Sigma design, electronic-business, and competitive
advantage: A dynamic capabilities model study, Journal
of High Technology Management Research, 16, 59-87.
[4] Linderman, K., Schroeder, R. G., Zaheer, S. and Choo,
A. S. (2003). Six Sigma: a goal-theoretic perspective,
Journal of Operations Management, 21, 193-203.
[5] Schroeder, R. G., Linderman, K., Liedtke, C. and Choo,
A. S. (2008). Six Sigma: Definition and underlying
theory, Journal of Operations Management, 26, 536–554.
[6] Singh, A. K. and Khanduja, D. (2014). Defining Quality
Management in Auto Sector: A Six-Sigma Perception,
Procedia Materials Science, 5, 2645-2653.
[7] Baril, C., Yacout, S. and Clément, B. (2011). Design for
Six Sigma through collaborative multiobjective
optimization, Computers & Industrial Engineering, 60,
43-55.
[8] Rohini, R. and Mallikarjun, J. (2011). Six Sigma:
Improving the Quality of Operation Theatre, Procedia -
Social and Behavioral Sciences, 25, 273-280.
[9] Vore, K. D. (2008). A six-sigma approach to stability
testing, Journal of Pharmaceutical and Biomedical
Analysis, 47, 413-421.
[10] Kwak, Y. H. and Anbari, F. T. (2006). Benefits,
obstacles, and future of six sigma approach,
Technovation, 26, 708-715.
Proceedings of the South East Asia Technical Training Conference on Electronics Assembly, 2016, SMTA.
View publication statsView publication stats