The Development of Mechatronic Machine Vision System for Inspection Of Ceramic Plates
•Download as PPTX, PDF•
2 likes•1,025 views
This is my conducted presentation for the research carried out to design a mechatronic vision system for inspection of garnished ceramic wall plates.
Recommended
More Related Content
Viewers also liked
Viewers also liked (20)
More from Waleed El-Badry
More from Waleed El-Badry (7)
Recently uploaded
Recently uploaded (20)
The Development of Mechatronic Machine Vision System for Inspection Of Ceramic Plates
- 1. Ain Shams University M.U.S.T DESIGN AND IMPLEMENTATION OF FLEXIBLE MANUFACTURING CELL FOR THE INSPECTION OF CERAMIC WALL PLATES Thesis submitted as partial fulfilment for MSc. degree in design and production engineering Waleed A. El-Badry T.A, Mechatronics Department, College of Engineering, M.U.S.T Supervisors Prof. Farid A. Tolbah Dr. Ahmed M. Aly
- 2. Current Inspection systems is restricted to detection of cracks and colours only and ignores quality of shapes. The available inspection method for quality of shapes of drawn objects in ceramic wall plates is carried out manually. Geometry and colours are inspected automatically in the image as a whole irrespective of its location. Manufacturing ceramic wall plates in industry is witnessing a large growth trend, therefore a need to perform fully automated inspection system became mandatory. 2
- 3. Developing an automated system for inspection of ceramic sorting system with respect to color matching and quality of shapes Applying an appropriate methodology for the quality inspection of drawn shapes in ceramic plates using modified fuzzy c-mean and fuzzy logic. Thereafter, implementing such a methodology in a machine vision system. 3
- 4. Since early 90s , automatic inspection played a key rule in manufacturing . As quality may degrade with visual inspection due to several aspect: Eye Fatigue accuracy of repeated measurements lack of mass production 4
- 5. 5
- 6. Image Manipulation Storing Images Computer Vision Image Presentation Image Compression 6
- 7. Source Of Illumination Image Camera Processing Machine Mechatronics Vision Mechanical Interfacing Handling Circuitry for System Manipulators Software Program 7
- 8. Literature Survey Many researches in this field were not published due to concession of production companies. Previous research contributions were focusing on dealing with ceramic plate “as a whole” irrespective of the painted geometry of each sculpted object. 8
- 9. Tile Vs. Garnished Plates Ceramic Tile Garnished Plate 9
- 10. 10
- 11. Image Gamma Image Soft Colour Acquisition Correction Calibration Partitioning Grouping Tuning Fuzzy Geometric General Block Diagram Logic Rules Features 11
- 12. Gamma Image Soft Image Calibratio Partitionin Colour Acquisition Correction n Grouping g Tuning Fuzzy Logic Geometric Features Rules Image Acquisition • For calibration and inspection as well. • Acquisition speed of 60 fps with 400 MB/S (Demanded in ceramic inspection). CCD Firewire Colour Camera IEEE 1394 Frame-grabber 640 X 480 resolution 12
- 13. Gamma Image Soft Image Calibratio Partitionin Colour Acquisition Correction n Grouping g Tuning Fuzzy Logic Geometric Features Rules Gamma Correction or often simply gamma, is the name of a nonlinear operation used to code and decode luminance in video or still images systems. This phenomena results from the output displays. Images which are not properly corrected can look either bleached out, or too dark Before After 13
- 14. Image Gamma Image Soft Acquisitio Calibratio Partitionin Colour n Correction n g Grouping Tuning Fuzzy Logic Geometric Features Rules NEED FOR CALIBRATION ? Camera holds a fixed number of pixels The more pixels you use to map a feature, the better accuracy you get Better Accuracy→ Closer Lens 14
- 15. Image Gamma Image Soft Acquisitio Calibratio Partitionin Colour n Correction n g Grouping Tuning Fuzzy Logic Geometric Features Rules DISCTORTION METRICS 15
- 16. Image Gamma Image Soft Acquisitio Calibratio Partitionin Colour n Correction n g Grouping Tuning Fuzzy Logic Geometric Features Rules NEED FOR CALIBRATION ? Distorted Calibrated 16
- 17. Image Gamma Image Soft Acquisitio Calibratio Partitionin Colour n Correction n g Grouping Tuning Fuzzy Logic Geometric Features Rules Proposed Novel Algorithm For Correcting and Calibrating Lens Distortion Template Colour Automatic Particle Acquisitio Plane Thresholdin Groupin n Extraction g g Displayin Mapping Distance Particle g particles to its Measuremen Measuremen corrected expected ts ts place image • An Image of calibration grid is acquired. • Red colour plane is extracted. 17
- 18. Image Gamma Image Soft Acquisitio Calibratio Partitionin Colour n Correction n g Grouping Tuning Fuzzy Logic Geometric Features Rules Proposed Novel Algorithm For Correcting and Calibrating Lens Distortion Template Colour Automatic Particle Acquisitio Plane Thresholdin Groupin n Extraction g g Displayin Mapping Distance Particle g particles to its Measuremen Measuremen corrected expected ts ts place image • Adaptive thresholding via clustering method. • Suitable in variable illumination over surface. 18
- 19. Image Gamma Image Soft Acquisitio Calibratio Partitionin Colour n Correction n g Grouping Tuning Fuzzy Logic Geometric Features Rules Proposed Novel Algorithm For Correcting and Calibrating Lens Distortion Template Colour Automatic Particle Acquisitio Plane Thresholdin Groupin n Extraction g g Displayin Mapping Distance Particle g particles to its Measuremen Measuremen corrected expected ts ts place image • Tracing boundaries (checking neighbored pixels) • Excluding open traces (incomplete circles) 19
- 20. Image Gamma Image Soft Acquisitio Calibratio Partitionin Colour n Correction n g Grouping Tuning Fuzzy Logic Geometric Features Rules Proposed Novel Algorithm For Correcting and Calibrating Lens Distortion Template Colour Automatic Particle Acquisitio Plane Thresholdin Groupin n Extraction g g Displayin Mapping Distance Particle g particles to its Measuremen Measuremen corrected expected ts ts place image 20
- 21. Image Gamma Image Soft Acquisitio Calibratio Partitionin Colour n Correction n g Grouping Tuning Fuzzy Logic Geometric Features Rules Proposed Novel Algorithm For Correcting and Calibrating Lens Distortion Template Colour Automatic Particle Acquisitio Plane Thresholdin Groupin n Extraction g g Displayin Mapping Distance Particle g particles to its Measuremen Measuremen corrected expected ts ts place image • Correcting skewness (circle center, distance from neighbors) • Creating lookup table for correcting incoming images. 21
- 22. Image Gamma Image Soft Acquisitio Calibratio Partitionin Colour n Correction n g Grouping Tuning Fuzzy Logic Geometric Features Rules Distorted Image Corrected Image 22
- 23. Image Gamma Image Soft Acquisitio Calibratio Partitionin Colour n Correction n g Grouping Tuning Fuzzy Logic Geometric Features Rules 23
- 24. Image Gamma Image Soft Acquisitio Calibratio Partitionin Colour n Correction n g Grouping Tuning Fuzzy Logic Geometric Features Rules Soft Partitioning and grouping 24
- 25. Image Gamma Image Soft Acquisitio Calibratio Partitionin Colour n Correction n g Grouping Tuning Fuzzy Logic Geometric Features Rules 25
- 26. Image Gamma Image Soft Acquisitio Calibratio Partitionin Colour n Correction n g Grouping Tuning Fuzzy Logic Geometric Features Rules Features Vector for Pattern Matching Geometric Feature Extraction Features to be extracted from Lines Arcs Angle Pixel RGB each pattern Geometry Orientation Colour Space RGB HI (Ri,Gi,Bi) Angle measured from reference axis wI ØI 26
- 27. Image Gamma Image Soft Acquisitio Calibratio Partitionin Colour n Correction n g Grouping Tuning Fuzzy Logic Geometric Features Rules Geometric Feature Extraction Captured Image Edge Detection after calibration Prewitt Filter Feature Extraction 1- Corner Detection 2- Rake for Edge Measurement 3- Geometry Extraction 27
- 28. Image Gamma Image Soft Acquisitio Calibratio Partitionin Colour n Correction n g Grouping Tuning Fuzzy Logic Geometric Features Rules Degree of membership 1 0.8 0.6 Bad Good Shiny Classification 0.4 (Crisp) 0.2 0 Acceptable 0 0.2 0.4 0.6 0.8 1 Colour Quality Colour Spot 1 Degree of membership 0.8 Colour Mismatch 0.6 Inaccurate Tolerated Accurate 0.4 0.2 Fuzzy 0 0 0.2 0.4 0.6 0.8 1 Membership Accuracy of Geometry function 28
- 29. Image Gamma Image Soft Acquisitio Calibratio Partitionin Colour n Correction n g Grouping Tuning Fuzzy Logic Geometric Features Rules Accuracy of Geometry Inaccurate Tolerated Accurate Quality Bad Colour Spot Colour Colour Mismatch mismatch of Colour Spot Colour Spot Acceptable Good Colour Shiny Colour Spot Acceptable Acceptable Fuzzy Rules 29
- 30. 30
- 31. Image Gamma Image Colour Acquisition Correction Calibration Matching Classification Evaluation of Feature Fuzzy Rules Extraction General Block Diagram 31
- 32. 32
- 33. Mechanical Design For the generic FMC 33
- 34. Inspection Zone Stations for Classification 34
- 35. Flexible arms 35
- 36. USB Enabled Chip Microcontroller Schematic Diagram EEPROM (USB Enumeration) 36
- 37. Layout 37
- 38. Software Architecture MATLAB for Visual .NET Builder Basic.NET Image Fuzzy Logic Fuzzy C-Mean Processing and User Interface Calibration 38
- 39. 39
- 40. 40
- 41. I- Colour Spot II- Colour Mismatch III- Rotated Colour Accuracy of Fig. No. Acceptable Colour Mismatch Colour Spot Quality Geometry I 0.61 0.68 0.29 0.0 0.5 II 0.12 0.8 0.15 0.46 0.1 III 0.92 0.85 0.59 0.0 0.05 41
- 42. Type Automatic Detection Acceptable 95.3 % Colour Spot 97.3% Defects Detection Colour Mismatch 100 % Rate Total Accuracy 97.5 % Corercted Detection 90 80 Effect of Object Orientation on 70 Defects detection 60 50 0 50 100 150 200 250 300 Drawn Objects Orientation (in degrees) 42
- 43. 120 100 80 60 1 40 2 20 0 Colour spots Colour Mismatch Geometry 1 Ovidiu Ghita, Tim Carew and Paul Whelan, A vision-based system for inspecting painted slates, Journal of Sensor Review, Vol. 26, No.2, 2006 2 Proposed Algorithm 43
- 44. Nearest Neighbour Correct Match(67%) Incorrect Match(33%) Fuzzy C-Means Correct Match(94%) Incorrect Match(6%) 44
- 45. Criteria Time (ms) Image acquisition 16.67 Gamma correction 23 Image calibration 86 Prewitt filter 15 Geometric feature extraction 120 Fuzzy clustering 15 Sec Color Matching 80 Evaluation of fuzzy rules 69 Criteria Specification USB actual transfer speed 1MB / S Inspection Area 20 X 10 cm2 45
- 46. 46
- 47. A proposed calibration algorithm was to correct the lens distortion by means of software. A new visualization approach for colour grouping was also proposed using fuzzy c-mean clustering technique. A fuzzy inference engine was built to classify the garnished ceramic plates into three common categories (acceptable, ceramic plate having colour spots, and ceramic plate having colour mismatched drawn objects) 47
- 48. A test rig was developed to emulate the production environment. The system shows promising results in terms of accuracy in correct classification and withstanding against variability in illumination distribution. The system is considered novel compared to other published work since it is the only work which considered geometric features of drawn objects up to the time of submitting this thesis. 48
- 49. [1] A. M. Aly and W. A. El-Badry, "Design and Implementation of Flexible Manufacturing Cell for Quality Inspection of Garnished Ceramic Wall Plates" in 19th Conference of French Congress of Mechanics Marseille, France, 2009. [2] F. A. Tolbah, A. M. Aly, and W. A. El-Badry, "Automated grading system for garnished wall plates: A mechatronic approach" presented at the 8th International Conference on Production Engineering and Design for Development, Cairo, Egypt, 2010. [3] F. A. Tolba, A. M. Aly, and W. A. El-Badry, "An Enhanced Vision System for Sorting Ceramic Plates Based on Hybrid Algorithm and USB Interfacing Circuitry" presented at the 27th National Radio Science Conference, Menoufia, Egypt, 2010. 49
- 50. Supervisors REFEREES COLLEAGUES FAMILY MY BEST FRIEND 50
- 51. 51
Editor's Notes
- Venerable Professors,My name is Waleed El-Badry. A T.A in college of engineering at Misr University for Science and Technology. Today I’m enthusiastic to defend my thesis which is meant to develop an FMC for inspection of Ceramic Wall Plates. This research was conducted under supervision of ……
- Ceramic industry is witnessing a golden era in Egypt. Many ceramic manufacturers succeeded to become serious competitors not only in local market, but also worldwide. Quality assurance development is an asset to sustain their product among rapid growing technology in this field.A deep survey conducted shows that almost all published papers targeting utilization of image processing techniques to inspect ceramic plates focuses only on how to manipulate the image to extract cracks or colour defects. Shapes and geometry has a paramount importance when it comes to garnished wall plates. Even though they yielded a reasonable solution, they abandoned a very important aspect from perspective of industry. Putting into considerations practicality in designing a vision system such as camera type, illumination, Lens Distortion is one of the giant problems confronting success of such a system.
- Ceramic industry is witnessing a golden era in Egypt. Many ceramic manufacturers succeeded to become serious competitors not only in local market, but also worldwide. Quality assurance development is an asset to sustain their product among rapid growing technology in this field.A deep survey conducted shows that almost all published papers targeting utilization of image processing techniques to inspect ceramic plates focuses only on how to manipulate the image to extract cracks or colour defects. Shapes and geometry has a paramount importance when it comes to garnished wall plates. Even though they yielded a reasonable solution, they abandoned a very important aspect from perspective of industry. Putting into considerations practicality in designing a vision system such as camera type, illumination, Lens Distortion is one of the giant problems confronting success of such a system.
- Automated inspection emerges in early 90s when commercial PCs became more popular and less expensive. There was a thirst in market to take advantage of PC to monitor products automatically. Even though microprocessor based systems were already on duty, PC monitoring systems offers more granularity in terms of Human Machine Interface (HMI) and extensibility via Supervisory Control and Data Acquisition (SCADA).By utilisation of PC based systems, several errors could be removed that may be caused by human-eyed systems such as:Eye Fatigue : results from continuous monitoring of movable parts on the production line.Accuracy of repeated measurements: In other words, repeatability error that depends on inspector’s skills in general. Lack of mass production: since inspection using human eye may lead to a bottleneck in production cycle.
- Even Though Computer Vision and Machine Vision seems interchangeable terms in several papers, we strongly disagree with it. Computer vision is identical to digital image processing where the primary objective is to perform actions on stored images. These actions could be:Storing Images: changing image formats to be stored efficiently.Image Compression: that is meant to minimize the image size with acceptable quality to speed up transmissibility over networks or internet.Image Manipulation: that might be needed to enhance the image by removing noise for instance, or showing a desired feature like edge detection.Image Display: which is aimed to represent the image for specific purpose like image representation in gray scale or in frequency domain. Also one of the recent examples is display of thermal imagesof passengers to detect potential infection of Swine Flu.
- Machine Vision:Mechanical System for Executing Decision of InspectionImage Processing For Image ManipulationCamera Choice is critical based on applicationInterfacing between mechanical and controllerSoftware which is used as GUIMachine Vision is then suitable for Mechatronics Research
- A survey has been conducted in this regard. One of the observations was that new research activities started to demolish as a consequence of company concession. Meanwhile, published papers were carrying the inspection process on the ceramic plate as “a hole image” irrespective of each shape.
- The above image is one of the examples of ceramic tiles. Researchers published several papers since 90s to extract cracks or colour spots from the whole ceramic.Nowadays, the evolution of ceramic industry opens the innovation of manufacturing ceramic plates with drawings that has different shapes and dimensions. Thus demanding new inspection strategy besides the previous ones to verify shapes and geometry as well.
- And now I’m going to show you the first step in our proposed algorithm which is the learning phase to shape and geometry of available paintings on the garnished plate. This activity is carried offline, in other words, it is carried out away from the production line.
- This flow chart is demonstrate the general steps for learning phase. The next slides will dig into each step.
- Image acquisition is to be carried out using a firewire colour camera. An IEEE 1394 interfacing card has an acquisition speed of up to 400Mb/S. This made inspection phase easy without need to stop the conveyor for inspecting each ceramic plate.Choosing the correct camera involves various considerations:Throughput – The rate at which image data can be transferred over the bus.Effective cost – The overall component price of a system, including the camera, cables, frame grabbers, and software.Cable length – The maximum possible distance between the camera and the PC without repeaters.Standardized interface – A measure of ease of use and future scalability. Plug-and-play interfaces make some camera buses easier to use and allow for future system upgrades without significant rework.Power over cable – The ability of the camera bus to provide power to the camera over the same cable.Camera availability – A measure of the number of different camera types available, how long the camera bus has been available, and the overall acceptance of the standard in the vision industry.CPU usage – The amount of CPU available to process images during image acquisition.I/O synchronization – The ease at which triggering and overall system communication is addressed and handled within the camera bus.
- Gamma correction enhances image colour that might be displayed improperly on screen.
- Lens distortion is a very common problem when acquiring a live image. This problem can be solved using Telecentric lens. However, some telecentric lenses costs more than the camera itself.So, our research team managed to correct almost all the image distortion that takes the barrel shape using software calibration and paper grid with equally distant dots.Paper grid is converted to binary image to obtain dots and by mapping each distorted dot to its appropriate place, a new corrected image can be constructed from the distorted one..
- Screenshot of the developed softwareThe bench marking is shown
- Colour grouping is meant to isolate objects with similar colours to be prepared for feature extraction. Since pixels of same object may have variability in colours, thus partitioning could be achieved by iterative Fuzzy C-Means. Where each centre depicts the object RGB colour. This information is taken based on operator observation.Finding all regions that share the same colour with degree of membership is carried out by finding the local minimum of the objective function. This is implemented by applying the formulas as shown.This iteration results in mapping each pixel to each colour group with a degree of membership. The pixel belongs to the colour that has the biggest degree of membership. Number of coloured objects is determined by operator.
- In the screenshot:Calibration then clustering3D display of clustersBenchmarking is shown ( 15 Sec) [Offline]
- Geometric features comprises any objects outer boundaries, orientation with respect to a predefined access and colour array.Reference axis can be placed at centre of any object or at a fixed place in the image.
- After colour grouping, the image is converted to gray scale, then Prewitt filter is applied for edge detection using the shown formula. The filter is 3X3 window. Corners can then be easily identified and edge measurements takes place using rakes. Each object’s dimensions and orientation is stored for online matching stage.
- Three fuzzy input variables: Quality of Colour, Accuracy of Geometry (dimensions and angles) and the classification decision are modelled with the shown membership functions.The output membership function comprises three ramp functions ranges from 0 to 1.
- The slide shows rules obtained from expert domain. Rules are evaluated based on The smallest value of maximum (SOM).
- After learning phase, automated inspection is ready to be executed online
- Inspection phase has similar stages to learning peer such as image acquisition, gamma correction and image calibration.In colour matching, clusters centres used in learning phase (fuzzy c-mean) is used for giving a matching score to each colour group found.Geometric matching is also carried out based on stored features in learning phase and a score is given to each found object.Then colour and geometry scores are fuzzified into the fuzzy logic membership functions and all fuzzy rules are evaluated. Then the output is defuzzified and determine the class of the ceramic plate.
- And now we are going to show you how we implemented the mechatronic system for
- The mechanical design has been carried out to design a multipurpose inspection cell. The target was to create a flexible cell that can be used to inspect different types of products.
- The inspection carriage transports each ceramic to its appropriate station based on inspection criteria by means of thread where it is mounted on via ball screw to minimize friction and maximize transportation speed..
- Inspection carriage has two flexible arms where halogen illumination can be adjusted to minimize reflectivity of object under inspection.
- Software was developed using two hybrid programming languages: MATLAB which proves to perform fuzzy c-mean and fuzzy logic in a competitive time. Visual Basic.NET where image processing and image calibration is much faster than MATLAB. All MATLAB used functions were compiled using MATLAB for .NET builder.
- The following results were obtained after exposing the system to three different ceramic classes. The system automatically classifies each ceramic plate according to the highest score from the fuzzy rules.
- The proposed algorithm demonstrated a promising accuracy shown on table. However, rotated plates affects the accuracy since resolution of height to width of camera is not the same. Moreover, severe reflectivity of ceramic plates could degrade the system accuracy as well besides lens distortion that can be solved in a better way using telecentric lens.
- The benchmarking comparison shown demonstrated results of proposed algorithm to one of the well known paper published by Ovidiu and his colleagues from Dublin City University in Ireland. The authors has a novel approach in combining color and geometry for quality inspection of garnished wall plates.
- Fuzzy c-mean yielded better results than nearest neighbour technique. It is expected since the centre of each cluster is tuned via each iteration pass. Nearest Neighbour is adequate where one of the features of each class has no chance to overlap with another class.
- Another criteria that were ignored by many publications is the time required for the algorithm to be executed. The key importance of such a test would exhibit the impact on productivity. From the shown benchmarks that were obtained on a Pentium 4 processor and a 1GHz RAM PC running Windows XP, an 800 cm2 of garnished plates per second can be classified. However, this number is subjected to variation with respect to mechanical handling system.