2. Journal of Industrial Technology • Volume 21, Number 3 • July 2005 through September 2005 • www.nait.org
Developing a Practical
Quality System to Improve
Visual Inspections
by Mr. Rathel R. Smith, Dr. Neal Callahan and Dr. Shawn Strong
Introduction inspector to be consistent in their own
While quality assurance programs evaluations while reproducibility is the
Mr. Rathel R. (Dick) Smith is an assistant professor capability of a group of inspectors to be
of Industrial Management at Southwest Missouri place considerable emphasis on using
State University. Mr. Smith is also an active consul- variable data to control process and consistent with each other.
tant to the manufacturing community, specializing
is quality systems and ISO 9001. product conformance, there is often
unavoidable reliance on visual inspec- Purpose and Scope
tions of attribute characteristics, which The purpose of this article is to dem-
is often subjective in nature. Attribute onstrate the practical application of
characteristics include aspects such a quality system process that can be
as physical appearance, cleanliness, used to improve consistency in visual
shades of color, etc. While in some inspections where the use of attribute
instances these characteristics can actu- measuring equipment is either too
ally be measured, the cost of suitable cost-prohibitive or too time-consuming.
equipment is often excessive or the Reducing variation in the shop floor en-
evaluation process too time consuming vironment leads to remarkable produc-
when compared to the results. Thus, tivity improvements (Emiliani 1998).
there needs to be an effective evaluation Although the material used in this
system to improve visual inspections study (wood) has several interrelated
to fill this gap. Reliance on subjective attributes, this article focuses on those
Dr. R. Neal Callahan is an Assistant Professor human judgment, and the variation in characteristics that directly influence
in the Department of Industrial Management at those judgments when compared from the effective utilization of wood for
Southwest Missouri State University. His research
and teaching interests include computer integrated person to person, creates quality evalu- manufactured components. The scope
manufacturing and quality systems. Dr. Callahan
received his Ph.D. in Engineering Management from
ation problems inhibiting the effective of this article includes an overview of
the University of Missouri-Rolla. use of visual inspection. the proposed quality system, includ-
The importance of gauge control ing development, methodology of the
in traditional variable data systems is study, analysis of collected data, and a
well established (Besterfield 2004). summary of findings.
Evaluating the effectiveness of the
measuring system is an important part Visual Inspection Improvement
of quality control and process improve- Quality System Overview
ment activities. For any measurement Any system developed should have
being reported: Observed value = True mechanisms built in to allow for con-
value + Measurement Error (Mont- tinuous monitoring and improvement
gomery 2005). When applying gauge (Eldar and Ronen1995; Donnabedi-
control methods, measurement error is an1986; Williamson1978). Developing
split into two categories: Repeatability, and implementing a quality system to
which is caused by equipment varia- improve visual inspections created sev-
tion, and Reproducibility, which is a eral challenging areas for this study. Of
Dr. Shawn Strong is an associate professor and de- result of appraiser variation (Besterfield particular concern was the development
partment head of Industrial Management at South-
west Missouri State University. Dr. Strong teaches
2004). Little research has been done of a model for the proposed system
courses in project management, fluid power, in the area of visual inspection gauge while considering documented and
electronics, and mechanical systems. Dr. Strong
received his PhD from Iowa State University. control, even though this method is accepted quality systems practices and
commonly relied upon in many areas of the application of those principles to a
industry including manufacturing and manufacturing environment. The qual-
construction. For the purposes of this ity system model developed through
article, repeatability in visual inspec- this study (see Figure 1) illustrates the
tion is the capability of an individual structure and systematic process used.
2
3. Journal of Industrial Technology • Volume 21, Number 3 • July 2005 through September 2005 • www.nait.org
Methodology wood utilization. These include sap Introducing inspectors to an
Background and heart side saw marks, knots, rotten expanded definition and numbering
Working on a quality improvement and want wood, and sap side white system required a well-planned train-
project with a wood products manufac- wood. ing program. The two primary goals for
turer presented an ideal opportunity to this training were to give inspectors the
conduct this exploratory study to test Define Characteristics and Assign necessary skills and tools to complete
the feasibility and effectiveness of the Numeric Values inspection evaluations consistently.
proposed quality system model in a Once critical characteristics were During the training process for this
real world setting. Two limitations to identified, definitions were developed study, sample boards, written defini-
this study were the fact that only three to enhance inspector understanding. tions, and pictures were used to stimu-
individuals were designated as qual- Word descriptions, actual samples, and late discussion and enhance learning on
ity inspectors and the time constraints pictures served as the descriptive means what specific value to assign for each
associated with removing those indi- for characteristic classification. For characteristic type.
viduals from the production floor for analysis purposes, it was also important
participation in the study. According to to assign numeric values to identified Set Up and Conduct Pre Review
Besterfield (2004), the number of parts, subjective characteristics. In some in- Study
appraisers, and trials often vary in a stances, the use of a zero or one repre- The process used for data collection
gauge control study. However, two to sents yes/no evaluations. For example, in this study was to have each inspec-
three trials and two to three appraisers the board either did or did not have saw tor do an evaluation of each of fifteen
are optimum. marks. The use of a sliding scale also selected and numbered sample boards.
Effective utilization of raw materi- provided an opportunity to weight the The numbering of the boards simplified
als (wood) is a major profitability issue range (such as one being minimal, five identification by the individual coordi-
for this wood product manufacturer. being average, and 10 being perfect). nating the study throughout the vari-
They use a general term “proof” as one ous trials of the Pre and Post studies.
measure of effective utilization. In its Training Inspectors on Characteristics Instructions were given to each par-
simplest definition, proof is leaving As with any new system, it is important ticipant (inspector) to visually inspect
the appropriate amount of white wood, to provide training. However, accord- the boards and record their evaluation
knots, saw marks and other imperfec- ing to Motwani, Frahm, and Kathawala on a tally sheet until assessment of all
tions while processing component (1994, p. 8), “Training needs to be characteristics on all fifteen boards
boards. During preliminary rough-cut preceded by a well articulated strategy was completed. The data collection
and plane operations, complete removal that employees can understand.” process included three separate trials
of these imperfections causes excessive using the same boards. Allowing one
wood waste while leaving the appro-
priate amount of imperfections has no Figure 1: Quality System Model
negative impact on the appearance and
functionality of the product. This evalu-
ation of proof (imperfections/character-
istics) is based on visual inspection and
subjective evaluation, thus enabling the
testing of this quality system.
Identify Critical Characteristics
In most instances, critical character-
istics (measurable) data are defined
during product and/or process develop-
ment. Control of these characteristics
is achieved through the assignment of
a target value, allowable tolerances,
and implementing a capable measure-
ment system. When critical character-
istics are more subjective-based, the
desired attributes are more difficult to
identify, define, standardize, measure,
and control. For this study, production
and quality managers provided input
to identify nine distinct critical charac-
teristics deemed appropriate to control
3
4. Journal of Industrial Technology • Volume 21, Number 3 • July 2005 through September 2005 • www.nait.org
to two weeks between trials minimized ation by Sample and Characteristic evaluations of board 8 were consistent
participants’ ability to recall previous Summary (Table 1) for “heart side saw across all participants while board 10
trial evaluations. marks” is just one example of the nine indicated confusion within and between
total characteristics studied and review the participants. This review certainly
Analysis of Data in this study. From the Participant resulted in a group discussion to arrive
Analysis included comparison of each Evaluation by Sample and Character- at consensus as to what the evaluations
participant’s results by sample and istic Summary report each participant should have been.
characteristic. During the three trials, was able to see differences in their While the review discussed above
each of the three inspectors evaluated actual evaluations from trial to trial allowed for individual-by-individual,
fifteen boards resulting in 135 data (repeatability). For example, Participant characteristic-by-characteristic, and
points for each characteristic. There 1 on sample board 3 did not see the board-by-board review, participants and
were nine total characteristics bringing heart side saw marks during Trial 1 but management were also interested in a
the total to 1215 data points for both did during Trials 2 and 3 (see Table 1). more detailed analysis. The data shown
the Pre and Post studies. This number However, Participant 3 made the same in Table 2 provided by-characteristic
of participants and trials is comparable evaluation during each trail for sample average evaluation scores and standard
to that suggested by Besterfield (2004) boards 1, 2, 4, 5, 6 and 8. Participant 2 deviations over three trials for all par-
and mentioned in the Methodology was inconsistent on all three trials for ticipants. This average also provided
section of this paper. Three types of sample boards 6 and 12. This review useful information relative to the qual-
summary reports were created from the process obviously caused the partici- ity of wood and capability of produc-
analysis of data as follows: pant to wonder why their evaluation tion operators to leave appropriate
1. The Participant Evaluation by was not consistent from trial to trial. proof. The standard deviation reflected
Sample and Characteristic Summary Also incorporated into this review the overall repeatability capability by
(example Table 1) was a summary was the opportunity for each partici- characteristic and thus provided insight
report of the actual evaluations pant, to look at his or her evaluations into which characteristic presented
posted on the tally sheet by the three in comparison to other participants. problems in consistent evaluation for
participants over the three trials. For example, on board 4 Participants 1 participants. These results presented
This simple report provided insight and 3 gave 0 points for all trials while opportunity for management to iden-
into both the repeatability and repro- Participant 2 gave scores of 8, 10, tify those specific characteristics that
ducibility capability of participants. and 10 respectively. In addition, the should be focused upon in future
2. The By-Characteristic Summary
(example Table 2 and Table 4) pro- Table 1: Participant Evaluation by Sample and Characteristic Summary
vided a brief but useful data analysis
for each of the nine characteristics.
Characteristic: Heart Side Saw Marks
Average and standard deviation of
participant evaluations provided
insight into which characteristics ap- Participant 1 Participant 2 Participant 3
peared more problematic. Sample T1 T2 T3 T1 T2 T3 T1 T2 T3
3. The Participant Comparison by
1 10 10 10 0 0 0 0 0 0
Characteristic (example Table 3 and
Table 5) provided overall and by 2 0 0 0 0 0 0 0 0 0
characteristic participant-to-partici- 3 0 10 10 8 10 10 0 10 10
pant comparison. The average of 4 0 0 0 10 8 8 0 0 0
trials, range between trails, trial stan-
dard deviation and difference from 5 10 10 10 0 0 0 8 8 8
group average provided insight into 6 0 0 0 8 5 10 0 0 0
individual participant repeatability 7 8 8 8 0 0 0 8 10 10
and group reproducibility. In addi-
tion, it provided useful information 8 10 10 10 8 10 10 10 10 10
for determining which characteristics 9 10 10 10 0 0 0 8 10 0
needed to be focused on in training. 10 0 8 8 10 5 10 0 0 10
11 8 8 8 0 0 0 10 8 0
Review with All Participants
Once the participants completed their 12 10 10 8 5 10 0 10 8 0
three trials with all fifteen boards, the 13 0 0 0 0 8 8 0 0 0
reports described in the previous sec- 14 10 10 8 10 10 10 8 8 10
tion were used for a group review of
study results. The Participant Evalu- 15 0 0 0 0 0 0 10 10 8
4
5. Journal of Industrial Technology • Volume 21, Number 3 • July 2005 through September 2005 • www.nait.org
definition improvement and participant Set Up and Conduct Post Review three trials between the Pre and Post
training. One side benefit with this Study Review studies presented additional
report is that it also gave management After the review with all participants, information useful in evaluating the ef-
insight into what characteristics should and revisions of the various definitions fectiveness of training and for discover-
be focused upon in operator training to of characteristics were completed, ing new improvement opportunities.
improve overall utilization. In addi- participants underwent retraining. The Table 4 presents summaries of
tion, this data established a reference purpose for additional training was individual characteristic data. There
point to compare effectiveness of this to review the revised definitions and was a reduction in the average par-
Review with All Participants phase of reinforce lessons previously learned. ticipant standard deviation between
the defined quality system after the Post Once retrained, participants repeated the Pre and Post Review study in six
Review Study. This report was also the characteristic evaluations exercise of the nine characteristics indicating
shared with participants (inspectors) in after a two-month period. This allowed improvement in repeatability. In ad-
an effort to keep them totally involved for conducting of a Post Review study dition, the average reduction in aver-
in the entire process. in the same manner and using the same age standard deviation for the six was
A third report, the Participant sample boards and data collection tech- .419 as compared to .230 for the other
Comparison by Characteristic (Table niques as the Pre Review study. Data three. Table 4 also aided in identifying
3) was prepared to evaluate the consis- analysis for the Post study utilized the problematic characteristics such as Sap
tency within each participant and allow same processes as the Pre Study. Side White Wood which experienced
for comparison. The average of trials After completion of an analysis of a .400 increase in average participant
and difference from group average the data captured in the Post set of tri- standard deviation between the Pre and
were used to evaluate the relative evalu- als, a determination relative to the level Post Study; thus indicating that either
ation range (high- low) of participants of improvement participants experi- an improved definition needed to be
compared to the group. Participant enced between the Pre Review and Post developed or participants required ad-
3, at 18.6222 evaluated lower than Review studies was made. The By- ditional training.
Participant 2, at 23.7111 base on the Characteristic Summary and Participant In addition to the Pre/ Post dif-
overall evaluation score for all boards. Comparison by Characteristic reports ferences noticed in both average and
This report was also prepared for each from the Pre Review study provided standard deviation, there was a reduc-
of the nine characteristics such that par- a base line to evaluate the level of tion for all characteristics as shown in
ticipants could identify which specific progress made by participants. On an Table 5. A reduction of 4.2222 in the
characteristics contributed to overall individual characteristic basis, looking range between participants relative to
participant differences. Addition- at the average standard deviation across the average of trails suggests that there
ally, the range between trials and trial
standard deviation demonstrated the Table 2
individual capability of each participant
to repeat their evaluations. Participant By Characteristic Summary (Pre Review)
3 with a trial standard deviation of Sap Side Averages Std Deviation
3.9202 as compared to Participant 2 White Wood 5.67 0.36
with a 5.5964 trial standard deviation. Missing Wood 1.30 0.98
However, Participant 3 with a differ- Knots 0.67 0.32
ence from group average of -0.4074 Saw Marks 1.63 0.38
and a trial standard deviation of 5.1448 Rotten 0.59 0.26
demonstrated better evaluation capabil-
Heart Side
ity. The range between participants
Missing Wood 1.10 0.83
was included in this report to remind
participants that the primary objective Knots 2.99 0.44
was to improve consistency and that the Saw Marks 4.52 2.05
lower this range, the better. Rotten 0.73 0.72
Table 3
Participant Comparison by Characteristic (Pre Review all characteristics)
Participant 1 Participant 2 Participant 3 Range
Average of Trials 20.5556 23.7111 18.6222 5.0889
Range Between Trials 9.4667 10.2000 7.0667 3.1333
Trial Std Dev 5.1448 5.5964 3.9202 1.6762
Difference From Group Average -0.4074 2.7481 -2.3407
5
6. Journal of Industrial Technology • Volume 21, Number 3 • July 2005 through September 2005 • www.nait.org
was improvement in participant ability Table 4
to reproduce their own evaluations. By Characteristics Summary (Pre Post Review Comparison)
It is equally important, to notice the
reduction, in the averages between Pre Averages Standard Deviations
and Post studies for participants relative Sap Side Pre Post Difference Pre Post Difference
to both their range between trails and White Wood 5.67 5.10 0.58 0.36 0.76 -0.40
their trial standard deviations. Specifi- Missing Wood 1.30 0.37 0.93 0.98 0.64 0.33
cally, there was a reduction of 4.2889 Knots 0.67 0.71 -0.04 0.32 0.00 0.32
in range between trials and 2.2957 for Saw Marks 1.63 1.78 -0.15 0.38 0.05 0.33
trial standard deviation further indicat- Rotten 0.59 0.00 0.59 0.26 0.00 0.26
ing that inspector capability to repeat Heart Side
their evaluations was improved. Missing Wood 1.10 2.81 -1.72 0.83 0.96 -0.14
Knots 2.99 0.72 2.27 0.44 0.59 -0.15
Establish Improvement Plan Saw Marks 4.52 3.56 0.96 2.05 1.50 0.55
As with all closed-looped systems, it Rotten 0.73 0.00 0.73 0.72 0.00 0.72
is important to encourage continuous
improvement. After reports from the Average Decrease When Decreased -0.419
Pre and Post studies were analyzed, the Average Increased When Increased
following became primary comparison 0.230
possibilities and potential opportunities
for improvement: by characteristic and all characteris- Summary of Results
1. Identification of participants whose tics evaluation scores. . Results from this exploratory study in-
evaluation averages did not improve 3. Identification of any characteristic dicate that the proposed quality system
in comparison to the group. This was that presented difficulty for most model is useful in improving repeat-
a comparison by characteristic and participants as indicated by: charac- ability and reproducibility in visual
all characteristics evaluation scores. teristic averages, standard deviations inspections. While this system presents
2. Identification of participants whose and a review of individual sample several steps, perhaps the most impor-
evaluation ranges and standard devi- scores. tant for improving visual inspection is
ations did not improve in comparison 4. Identification of changes in the group the ability to review Pre and Post study
to the group. This comparison was averages between studies. results with participants. The proposed
Table 5
Participant Comparison by Characteristic (Pre Post Comparison all characteristics)
Pre Review
Participant 1 Participant 2 Participant 3 Range Average
Average of Trials 20.5556 23.7111 18.6222 5.0889 20.9630
Range Between Trials 9.4667 10.2000 7.0667 3.1333 8.9111
Trial Std Dev 5.1448 5.5964 3.9202 1.6762 4.8871
Difference From Group Average -0.4074 2.7481 -2.3407
Post Review
Participant 1 Participant 2 Participant 3 Range Average
Average of Trials 19.1111 18.2444 18.8000 0.8667 18.7185
Range Between Trials 4.5333 5.3333 4.0000 1.3333 4.6222
Trial Std Dev 2.5175 2.9472 2.3094 0.6378 2.5914
Difference From Group Average 0.3926 -0.4741 0.0815
Pre-Post Pre-Post
Range Average
Difference Difference
Average of Trials 4.2222 2.2444
Range Between Trials 1.8000 4.2889
Trial Std Dev 1.0384 2.2957
6
7. Journal of Industrial Technology • Volume 21, Number 3 • July 2005 through September 2005 • www.nait.org
quality system provides vital informa- and their strong belief that the entire business sectors, and educational enti-
tion towards the development of an process enhanced their understanding ties use such data. Organizations today
affective and individualized inspector of what evaluations were appropriate are better understanding the value of
improvement plan during the Pre and for the various levels of characteris- converting data into useful informa-
Post Reviews. tics. In addition, a t-test to determine tion. At the same time, managers fear
Specific components of the review whether the differences found between making decisions from information that
process and potential for improving Pre and Post studies were statistically is not reliable due to error associated
visual inspection results include the fol- significant. The results are listed in with measurement repeatability and/or
lowing: first, the individual inspector’s Table 6. The All Characteristic Totals reproducibility. The studied quality
review of their evaluations by board presented in Table 5 are the summation, system has demonstrated potential to
and by trial provides a foundation by participant, for evaluations made for remove or substantially reduce fears
for enhancing repeatability. Second, all characteristics. The t-test resulted relative to visual inspection, thereby
the individual inspector’s review by in an overall p-value of 0.0045, which improving decision-making ability;
board and by trial of their evaluation as indicates statistical significance for the and, improved decision-making im-
compared to other inspectors provides a reduction of total scores from Pre to proves the potential for success.
foundation to enhance reproducibility. Post training.
Third, the group discussion involving References
all participant inspectors provides an Recommendations for Further Besterfield, D. H. (2004), Quality Con-
open forum to clarify understanding Study trol (7th ed.), Englewood Cliffs, New
of critical characteristic and identify Since all variation in this study came Jersey: Prentice Hall
those characteristics that appear to be from human input, training methodol- Eldar, R. and Ronen, R. (1995), Imple-
more problematic. Fourth, the group ogy should be evaluated and future data mentation and evaluation of a qual-
discussion provides opportunity for analyzed to determine: ity assurance program, International
the individual conducting the study to 1. If statistically, acceptable levels of Journal of Health Care Quality
target those characteristics that appear repeatability can, be maintained by Assurance, 8 (1), 28
to be more problematic for improved individual inspectors. Emiliani, M. L. (1998), Continuous
definition or additional inspector train- 2. If statistically, acceptable levels of personal improvement, Journal of
ing. Fifth, the data analysis and group reproducibility can, be maintained Workplace, Learning, 10 (1), 32
discussion are beneficial to establish an between groups of inspectors. Montgomery, D. C., (2005), Introduc-
improvement plan for each individual 3. If this methodology, can be repli- tion to Statistical Quality Control,
inspector involved in the study. cated in other settings where visual New York: John Wiley and Sons
There is evidence that the proposed inspection is relied upon. Motwani, J. G.; Frahm, M. L.; Katha-
system is an effective tool for improv- wala, Y., (1994), The key to quality
ing visual inspections. This support Conclusion improvement, Training for Quality,
comes first, and foremost, from the Many production enterprises utilize 2 (2), 8
positive feedback participants provided visual inspection subjective data. Most
during the various stages of the study manufacturers, service industries, other
Table 6
Pre Post Review All Characteristic Evaluation Totals and t-Test Results
Participant 1 Participant 2 Participant 3
PRE 352 310 263 359 341 367 280 270 288
POST 300 312 282 251 241 233 283 230 240
p-value
All Participants 0.0045
Participant 1 0.3614
Participant 2 0.0001
Participant 3 0.0862
7
