Image-based Detection of Defects on Metal Surfaces
Image-Based Real-Time Detection andClassification of Defects on Metal SurfaceProf. Pranab Kumar Dutta, Pankaj Pansari, Kundan Singh M. GangadaranDepartment of Electrical Engineering AAPC Group, R&D Centre for Iron and Steel,Indian Institute of Technology, Kharagpur SAIL, Ranchi- firstname.lastname@example.orgAbstract – This paper describes a real-time system which detectssurface-detects on metals given the images of the surfaces, andcharacterizes them through classification into suitable classes.The algorithm first segments out the defects through acombination of segmentation methods, and then processes themfor more accurate and faithful representation of the defect-regions. From this representation, parameters are extracted andare compared to a known set of values, which helps in thecharacterization of the defects.I. INTRODUCTIONThe metal industry has assumed utmost importance intoday’s economy. The industry seeks to attain perfection in thequality of the metal products. Often, due to poor quality of rawmaterials or mediocre performance of machinery, defects arisein the metal product. The defects may be internal or external.In view of the scale of production, it becomes necessary tohave a reliable and autonomous defect-detection system.While reliable techniques such as ultrasound method existfor internal defect detection, here we seek to develop animage-processing application, which detects the defects on thesurface, by seeking information from the images of the metal-surfaces and classifies any detected defect in the majorcategories. The entire application has been developed usingthe OpenCV library, thus providing real-time processing,which yields fast results.The algorithm consists of 3 basic sections:1. Initial Segmentation of the defects, if any.2. Processing of segmented image for a more accuraterepresentation of the defect regions.3. Classification of the defects on the basis of shape, sizeand texture parameters.The algorithm has been designed to be robust againstillumination variances and the presence of noise, which is sooften the case in industrial applications. Also, since therequirement is for a real-time system, the algorithm does notinvolve very intensive computation, and does not requirededicated, expensive hardware for processing.II. INITIAL SEGMENTATION OF DEFECTSThe primary step is to extract maximum information aboutthe defect from the gray-scale image and to produce a binaryimage with the defect being represented by black regions on awhite background. The defect-regions segmented out must beaccurate. The segmented regions must not under-represent thedefect in the image, that is, portions of the defect must not beleft out. Also, the defect-regions must not be exaggerated intheir representation. The segmentation step is crucial, since thesuccess of the system depends on how faithfully the defect canbe brought out, and presented as input to the classifyingsystem.The fact that the segmentation method must be able tohandle variation in illumination and presence of noise, makesthe step challenging.Defects in metal surfaces take various forms, such asscratch, crack, bruise, blister, rupture and even salt-and-pepperdefect. The characterizing nature of each defect is quitedifferent from others. For example, scratch and cracks areedge-like features, blisters are of a scattered nature with local-uniformity and global-variances and salt-and-pepper defect isremarkable because of the uneven texture. On the other hand,each segmentation method extracts particular features. Forexample, adaptive thresholding does not very accuratelysegment out the edge-like defects, whereas edge-detectorscannot segment out defects with local uniformity. As a result,to be successful for all of these, our algorithm integrates thefollowing three segmentation methods, depending on thecontext:A. Global ThresholdingB. Adaptive ThresholdingC. Edge detection
Fig. 1 Clockwise from top: (a) Image of Damage-defect (b) After Optimum-Global Thresholding (c) After AdaptiveThresholding (d) After Edge-detection (e) After IntegrationA. Global ThresholdingThe simplest of the thresholding techniques, it involvespartitioning the image histogram by using a single globalthreshold. Segmentation is then accomplished by scanning theimage pixel by pixel and labeling each pixel as object orbackground depending on whether the gray-value of that pixelis greater or less than the value of the threshold. The methodworks only in highly controlled environment. For ourapplication, due to the non-uniformity of illumination, thishardly provides accurate segmentation.B. Adaptive ThresholdingImaging factors such as uneven illumination can transform aperfectly segmentable histogram into a histogram that cannotbe partitioned effectively by a single threshold. The approachis to divide the original image into sub-images and then utilizea different threshold to segment each sub-image. There aretwo important parameters. The first is the size of the image.Smaller sub-images result in a lot of noise while large oneshave the same disadvantages as global thresholding. Thesecond is the threshold-value for the sub-images.For adaptive threshloding, our algorithm first calculates theaverage intensity-value, from the sub-image. Then it calculatesthe threshold for each sub-image, T as follows:T = average intensity – cwhere c is the specified offset. Offset is necessary, so thathomogeneous regions, not having any defect, are set tobackground as white pixels. We found c=10 optimum for ourapplication.C. Edge DetectionThe detection of edges gives an outline of the defect regionsfor spread-out homogeneous defects such as damage andmassive-rupture and fairly accurate representation of linedefects such as scratches and cracks.D. Integrated Segmentation AlgorithmThe key principle is that since the regions of defect arecharacterized by large gradient values in their neighborhood,we adaptively threshold such regions and globally thresholdthe other regions with a small threshold-value such that onlyvery dark regions of the grayscale image are represented in thebinary image (this includes large homogeneous defectregions).
Fig. 2 Clockwise from top: (a) Image of Head-mark defect (b) After Global thresholding (c) After Adaptive thresholding(d) After edge-detection (e) After Integration (f) After majority black-pixel detection and edge-linkingThe steps of the integrated algorithm are as follows:1. Apply Sobel-edge detection to the grayscale image toyield a binary image containing an approximate edgerepresentation.2. In the neighborhood of each pixel (60X60 was optimumfor us), count the number of edge-pixels, as given bySobel-filtered image.3. If the count exceeds a certain threshold, which was 25for our case, then threshold it adaptively using the sub-image size as the neighborhood window chosen in theprevious step. Otherwise, threshold it globally using apre-determined optimum threshold.4. Add this thresholded image to the binary imageobtained by Sobel-edge detection.The image obtained in the last step is next processed asdescribed in Section III.III. PROCESSING OF THE SEGMENTED IMAGEThe segmented image is processed further for noise-reduction through majority black-pixel count and edge-linking.A. Majority Black-Pixel CountThis process helps to eliminate noise pixels and emphasizethe defect-regions. It is based on the fact that noise-pixels aregenerally diffused, which means that pixel density is less insuch regions as compared to defect regions.A 3X3 neighborhood of each pixel is considered in thesegmented binary-image and the number of black pixels iscounted. If the number of black-pixels exceeds a certainthreshold, then the pixel is labeled black, else it is labeledwhite. In this way, even if some features of the defect are notpresent in the segmented image, they come to be representedafter this step.B. Edge LinkingMajority black-pixel count is generally followed by edge-linking. This is because the first method generally results inattenuation of line defects, since the defect pixels do not havea large neighborhood black-pixel count. Further this methodenhances the edges further as compared to original binaryimages.This method is based on the fact that the edge-pixels in acertain close neighborhood have similar Sobel gradientmagnitude G,G=|Gx| + |Gy|and similar gradient-angles (tan-1(Gy/Gx)).
Fig. 3 Decision-tree for Classification AlgorithmIV. CLASSIFICATION OF DEFECTSClassification of defects is context-based. The method oneadopts is very essentially linked to the type of defects one isdealing with, the resolution of the images and the level uptowhich one wants to classify, that is, whether one wants theclasses to be broad or narrow. In other words, the classes forclassifying have to be defined depending on the application.However, the classes must be remarkably different to avoidambiguities.For our application we formed seven different classes, outof the sample defect samples. The major differences amongthe classes lie in size or shape or texture. Hence, weformulated parameters to quantify each of these. Our processis tree-based. At each parent node, the parameter values arematched against a given data-set to decide the child-node.The method presented here needs a form of training, in that,the data-set has to be formed before the system is used forautomated inspection. This is done through extracting therange of parameter-values for each class of defect, by givingsample defect images as input. Needless to say, the greater thenumber of samples, the better trained the system will be.1. Level IThe parameters used for classification at Level I are asfollows:Mx = maximum representation (number of black-pixels) which can be found in a singlehorizontal-band in the entire image. We used30Xwidth window.My = maximum representation (number of black-pixels) which can be found in a singlevertical-band in the entire image. We usedheightX30 window.x = Mx normalized by widthy = My normalized by heightthickness = measure of the maximum lateral continuousspread.%area = percentage of defect area out of the totalimage area.For our case, the classification at Level I is demonstrated inFig. 4.
Fig. 4 Classification at Level I for our application2. Level IIAt this level, it is checked whether the defect is localized innature. Note that since the defect has reached this level, it isnot a line-defect. This type of defect is usually called a bruise.First, the dense regions of the defect are emphasized byretaining only completely-filled small-sized windows,extended once in X and once in Y direction. If a small region,say a square of 50X50, exists such that a major portion, say70%, of the entire defect region lies in it, then it is classified asa bruise, else it proceeds to the next level.3. Level IIIAt this level, it is checked whether a defect has dense, largeand uniform regions. Such defects are referred to as damagesor ruptures. The condition is on the same lines as Level II.Only the size of the window is different, and the percentage ofoccupancy is different. For our case, we selected a window of30X30, which needed to be 100% filled for a defect to be adamage or a rupture.4. Level IVThe final level checks whether a defect is scale defect orsalt-and-pepper defect. To do this, it uses the texture propertyof these. It takes up the adaptively thresholded image, andchecks whether sufficiently dense regions can be found toclassify it as scales, or whether it has the grainy texture to beclassified as salt-and-pepper defect.Fig. 5 The output of the algorithm, classifying the inputdefect image as Damage or Rupture.CONCLUSIONThis paper presented an algorithm which could detect andclassify defects even in presence of illumination-variations,noise and poor-resolution. The algorithm did not requireintensive computation at any step, thereby eliminating theneed for expensive hardware for processing.An important aspect is training in the classification part.While excellent methods using weak-classifiers like Ada-boost exist, they are very general. Making them performsatisfactorily would require a huge amount of training data,which is not so easily available in case of metal-defects.Moreover, since they extract lots of features from each defect,the computational requirement increases significantly to makethem run in real-time. On the other hand, we have explicitlydefined the parameters, which makes the classificationalgorithm specialized towards metal-defect classifications.REFERENCES Rafael C. Gonzalez and Richard E. Woods, “DigitalImage-Processing”. Gayubo, Gonzalez, Feunte, Miguel and Peran, “On-linemachine-vision system to detect split defects in sheet-metal forming processes”, ICPR, 2006 Newman and Jain, “A survey of automated visualinspection”, Computer Vision and ImageUnderstanding, Volume 61, Issue 2, Pages 231-262