Image-Based Real-Time Detection and
Classification of Defects on Metal Surface
Pankaj Pansari, Kundan Singh
Department of Electrical Engineering
Indian Institute of Technology, Kharagpur
Abstract – This paper describes a real-time system which detects II. INITIAL SEGMENTATION OF DEFECTS
surface-detects on metals given the images of the surfaces, and
characterizes them through classification into suitable classes. The primary step is to extract maximum information about
The algorithm first segments out the defects through a
the defect from the gray-scale image and to produce a binary
combination of segmentation methods, and then processes them
image with the defect being represented by black regions on a
for more accurate and faithful representation of the defect-
regions. From this representation, parameters are extracted and
white background. The defect-regions segmented out must be
are compared to a known set of values, which helps in the accurate. The segmented regions must not under-represent the
characterization of the defects. defect in the image, that is, portions of the defect must not be
left out. Also, the defect-regions must not be exaggerated in
their representation. The segmentation step is crucial, since the
I. INTRODUCTION success of the system depends on how faithfully the defect can
be brought out, and presented as input to the classifying
The metal industry has assumed utmost importance in system.
today’s economy. The industry seeks to attain perfection in the The fact that the segmentation method must be able to
quality of the metal products. Often, due to poor quality of raw handle variation in illumination and presence of noise, makes
materials or mediocre performance of machinery, defects arise the step challenging.
in the metal product. The defects may be internal or external. Defects in metal surfaces take various forms, such as
In view of the scale of production, it becomes necessary to scratch, crack, bruise, blister, rupture and even salt-and-pepper
have a reliable and autonomous defect-detection system. defect. The characterizing nature of each defect is quite
While reliable techniques such as ultrasound method exist different from others. For example, scratch and cracks are
for internal defect detection, here we seek to develop an edge-like features, blisters are of a scattered nature with local-
image-processing application, which detects the defects on the uniformity and global-variances and salt-and-pepper defect is
surface, by seeking information from the images of the metal- remarkable because of the uneven texture. On the other hand,
surfaces and classifies any detected defect in the major each segmentation method extracts particular features. For
categories. The entire application has been developed using example, adaptive thresholding does not very accurately
the OpenCV library, thus providing real-time processing, segment out the edge-like defects, whereas edge-detectors
which yields fast results. cannot segment out defects with local uniformity. As a result,
The algorithm consists of 3 basic sections: to be successful for all of these, our algorithm integrates the
1. Initial Segmentation of the defects, if any. following three segmentation methods, depending on the
2. Processing of segmented image for a more accurate context:
representation of the defect regions.
3. Classification of the defects on the basis of shape, size A. Global Thresholding
and texture parameters. B. Adaptive Thresholding
The algorithm has been designed to be robust against C. Edge detection
illumination variances and the presence of noise, which is so
often the case in industrial applications. Also, since the
requirement is for a real-time system, the algorithm does not
involve very intensive computation, and does not require
dedicated, expensive hardware for processing.

Fig. 1 Clockwise from top: (a) Image of Damage-defect (b) After Optimum-Global Thresholding (c) After Adaptive
Thresholding (d) After Edge-detection (e) After Integration
A. Global Thresholding where c is the specified offset. Offset is necessary, so that
homogeneous regions, not having any defect, are set to
The simplest of the thresholding techniques, it involves background as white pixels. We found c=10 optimum for our
partitioning the image histogram by using a single global application.
threshold. Segmentation is then accomplished by scanning the
image pixel by pixel and labeling each pixel as object or
background depending on whether the gray-value of that pixel C. Edge Detection
is greater or less than the value of the threshold. The method
works only in highly controlled environment. For our The detection of edges gives an outline of the defect regions
application, due to the non-uniformity of illumination, this for spread-out homogeneous defects such as damage and
hardly provides accurate segmentation. massive-rupture and fairly accurate representation of line
defects such as scratches and cracks.
B. Adaptive Thresholding
D. Integrated Segmentation Algorithm
Imaging factors such as uneven illumination can transform a
perfectly segmentable histogram into a histogram that cannot
The key principle is that since the regions of defect are
be partitioned effectively by a single threshold. The approach
characterized by large gradient values in their neighborhood,
is to divide the original image into sub-images and then utilize
we adaptively threshold such regions and globally threshold
a different threshold to segment each sub-image. There are
the other regions with a small threshold-value such that only
two important parameters. The first is the size of the image.
very dark regions of the grayscale image are represented in the
Smaller sub-images result in a lot of noise while large ones
binary image (this includes large homogeneous defect
have the same disadvantages as global thresholding. The
regions).
second is the threshold-value for the sub-images.
For adaptive threshloding, our algorithm first calculates the
average intensity-value, from the sub-image. Then it calculates
the threshold for each sub-image, T as follows:
T = average intensity – c

Fig. 2 Clockwise from top: (a) Image of Head-mark defect (b) After Global thresholding (c) After Adaptive thresholding
(d) After edge-detection (e) After Integration (f) After majority black-pixel detection and edge-linking
The steps of the integrated algorithm are as follows: A. Majority Black-Pixel Count
1. Apply Sobel-edge detection to the grayscale image to This process helps to eliminate noise pixels and emphasize
yield a binary image containing an approximate edge the defect-regions. It is based on the fact that noise-pixels are
representation. generally diffused, which means that pixel density is less in
2. In the neighborhood of each pixel (60X60 was optimum such regions as compared to defect regions.
for us), count the number of edge-pixels, as given by A 3X3 neighborhood of each pixel is considered in the
Sobel-filtered image. segmented binary-image and the number of black pixels is
3. If the count exceeds a certain threshold, which was 25 counted. If the number of black-pixels exceeds a certain
for our case, then threshold it adaptively using the sub- threshold, then the pixel is labeled black, else it is labeled
image size as the neighborhood window chosen in the white. In this way, even if some features of the defect are not
previous step. Otherwise, threshold it globally using a present in the segmented image, they come to be represented
pre-determined optimum threshold. after this step.
4. Add this thresholded image to the binary image
obtained by Sobel-edge detection. B. Edge Linking
The image obtained in the last step is next processed as Majority black-pixel count is generally followed by edge-
described in Section III. linking. This is because the first method generally results in
attenuation of line defects, since the defect pixels do not have
a large neighborhood black-pixel count. Further this method
III. PROCESSING OF THE SEGMENTED IMAGE enhances the edges further as compared to original binary
images.
The segmented image is processed further for noise- This method is based on the fact that the edge-pixels in a
reduction through majority black-pixel count and edge- certain close neighborhood have similar Sobel gradient
linking. magnitude G,
G=|Gx| + |Gy|
and similar gradient-angles (tan-1 (Gy/Gx)).

Fig. 3 Decision-tree for Classification Algorithm
IV. CLASSIFICATION OF DEFECTS
Classification of defects is context-based. The method one 1. Level I
adopts is very essentially linked to the type of defects one is
dealing with, the resolution of the images and the level upto The parameters used for classification at Level I are as
which one wants to classify, that is, whether one wants the follows:
classes to be broad or narrow. In other words, the classes for Mx = maximum representation (number of black-
classifying have to be defined depending on the application. pixels) which can be found in a single
However, the classes must be remarkably different to avoid horizontal-band in the entire image. We used
ambiguities. 30Xwidth window.
For our application we formed seven different classes, out My = maximum representation (number of black-
of the sample defect samples. The major differences among pixels) which can be found in a single
the classes lie in size or shape or texture. Hence, we vertical-band in the entire image. We used
formulated parameters to quantify each of these. Our process heightX30 window.
is tree-based. At each parent node, the parameter values are x = Mx normalized by width
matched against a given data-set to decide the child-node. y = My normalized by height
The method presented here needs a form of training, in that, thickness = measure of the maximum lateral continuous
the data-set has to be formed before the system is used for spread.
automated inspection. This is done through extracting the %area = percentage of defect area out of the total
range of parameter-values for each class of defect, by giving image area.
sample defect images as input. Needless to say, the greater the For our case, the classification at Level I is demonstrated in
number of samples, the better trained the system will be. Fig. 4.

Fig. 5 The output of the algorithm, classifying the input
defect image as Damage or Rupture.
Fig. 4 Classification at Level I for our application
CONCLUSION
2. Level II
This paper presented an algorithm which could detect and
At this level, it is checked whether the defect is localized in
classify defects even in presence of illumination-variations,
nature. Note that since the defect has reached this level, it is
noise and poor-resolution. The algorithm did not require
not a line-defect. This type of defect is usually called a bruise.
intensive computation at any step, thereby eliminating the
First, the dense regions of the defect are emphasized by
need for expensive hardware for processing.
retaining only completely-filled small-sized windows,
An important aspect is training in the classification part.
extended once in X and once in Y direction. If a small region,
While excellent methods using weak-classifiers like Ada-
say a square of 50X50, exists such that a major portion, say
boost exist, they are very general. Making them perform
70%, of the entire defect region lies in it, then it is classified as
satisfactorily would require a huge amount of training data,
a bruise, else it proceeds to the next level.
which is not so easily available in case of metal-defects.
Moreover, since they extract lots of features from each defect,
3. Level III
the computational requirement increases significantly to make
them run in real-time. On the other hand, we have explicitly
At this level, it is checked whether a defect has dense, large
defined the parameters, which makes the classification
and uniform regions. Such defects are referred to as damages
algorithm specialized towards metal-defect classifications.
or ruptures. The condition is on the same lines as Level II.
Only the size of the window is different, and the percentage of
occupancy is different. For our case, we selected a window of
REFERENCES
30X30, which needed to be 100% filled for a defect to be a
damage or a rupture. [1] Rafael C. Gonzalez and Richard E. Woods, “Digital
Image-Processing”.
4. Level IV [2] Gayubo, Gonzalez, Feunte, Miguel and Peran, “On-line
machine-vision system to detect split defects in sheet-
The final level checks whether a defect is scale defect or metal forming processes”, ICPR, 2006
salt-and-pepper defect. To do this, it uses the texture property [3] Newman and Jain, “A survey of automated visual
of these. It takes up the adaptively thresholded image, and inspection”, Computer Vision and Image
checks whether sufficiently dense regions can be found to Understanding, Volume 61, Issue 2, Pages 231-262
classify it as scales, or whether it has the grainy texture to be
classified as salt-and-pepper defect.