Download to read offline
Uploaded byCristiano Rafael Steffens
ICRA 2016 - Interactive section Presentation
1) The document describes a robotic vision system for mapping welding grooves on metallic surfaces using machine vision techniques without complex optical setups. 2) A prototype system was developed using an FPGA, camera, and welding robot to autonomously map groove dimensions in real-time to improve welding quality over manual processes. 3) Experimental results found the Gaussian filtering and line segment detection approach achieved sub-millimeter accuracy and repeatability in measuring groove widths, outperforming other filtering and edge detection algorithms tested. Further improvements to lighting and image processing are ongoing.
More Related Content
Similar to ICRA 2016 - Interactive section Presentation
More from Cristiano Rafael Steffens
![Polymer [ बहुलक ] Chemistry Notes PDF - Irfanullah Mehar - JJ Sir Chemistry.pdf](https://cdn.slidesharecdn.com/ss_thumbnails/polymerchemistrynotespdf-irfanullahmehar-jjsirchemistry-260210172118-3f9b37f7-thumbnail.jpg?width=640&height=640&fit=bounds)
ICRA 2016 - Interactive section Presentation
- 1. A Robotic Systemfor Welding Groove Mapping: Machine Vision on Metallic Surfaces BQ Leonardo, CR Steffens, SC Silva Fil., JL Mór, V Hüttner, EA Leivas, VS Rosa and SSC Botelho cristianosteffens@furg.br Center of Computer Science, Federal University of Rio Grande, Brazil 1
- 2. A Robotic Systemfor Welding Groove Mapping: Machine Vision on Metallic Surfaces Welding is easy! Isn’t it? • Manual process affects the quality of the weld • Rework • Material waste • Weak and breakable final product • Reproducibility and regularity • The human side • Welding is unhealthy – ergonomy, heat and fumes • Laborious and repetitive task BQ Leonardo, CR Steffens, SC Silva Fil., JL Mór, V Hüttner, EA Leivas, VS Rosa and SSC Botelho Federal University of Rio Grande – Brazil – http://c3.furg.br Typical Setup of a Linear Welding System Proposed Video-Based Measurement System Algorithm Structure Use Case • BUG-O MDS Welding Robot • Robust Modular Robot • Can be used on a large variety of surfaces • Able to make different welding seams • Lincoln Flextec 450 Power source • Lincoln wire feeder Contributions • Modular VBM for linear welding robots • End-to-end embedded welding system prototype • Computer Vision applied to reflective surfaces, without the need of structured light, polarized lenses or complex optical arrangements • State of the art algorithms offer better cost-benefit • Developed a complete solution, featuring illumination, image acquisition and processing, robot operation and welding equipment setup 2
- 3. Welding is easy!Isn’t it? • Manual process affects the quality of the weld • Rework • Material waste • Weak and breakable final product • Reproducibility and regularity • The human side • Welding is unhealthy – ergonomy, heat and fumes • Laborious and repetitive task 3
- 4. Prior approaches forwelding process automation • A combination of structured illumination laser and camera, as used in Kawahara (1983), Drews et al. (1986), De Xu (2004), Liu (2010), and Zhang et al. (2014) • A touch sensor based approach as in Kim and Na (2000); • Techniques where the arc current feedback is explored, as in Dilthey and Gollnick (1998) and Halmøy (1999); 4
- 5. Typical Setup ofa Linear Welding System Typical linear welding robot installation 5
- 6. Typical Setup ofa Linear Welding System Operating. Source: BUG-O Systems (2013) 6
- 7. The BUG-O MDSWelding Robot • Robust Modular Robot • Rails and Carriages • Linear Weaver • Pendulum Weaver • Can be used on a large variety of surfaces • Able to make different welding seams • Weldor adjusts the linear rail and the parameters in runtime 7
- 8. Proposed Vision-based MeasurementSystem High-level architecture of the vision-based measurement system 8
- 9. Prior Work VBM Zhang, W.Ke, Q. Ye, and J. Jiao, “A novel laser vision sensor for weld line detection on wall- climbing robot,” Optics & Laser Technology, vol. 60, pp. 69–79, 2014. 9
- 10. Prior Work VBM Drews, B.Frassek, and K. Willms, “Optical sensor systems for automated arc welding,” Robotics, vol. 2, no. 1, pp. 31–43, 1986 10
- 11. Prior Work VBM D. Xu,M. Tan, X. Zhao, and Z. Tu, “Seam tracking and visual control for robotic arc welding based on structured light stereo vision,” International Journal of Automation and Computing, vol. 1, no. 1, pp.63–75, 2004. 11
- 12. Proposed Vision-based MeasurementSystem Image acquisition setup Welding groove properties 12
- 13. Overview of theVBM System 13
- 14. Machine Vision System(HW) Altera DE0-Nano FPGA. Source: Altera Terasic D5M CMOS Camera Source: AlteraIllumination 14
- 15. Machine Vision System(HW) FPGA + FTDI breakout (PC communication) Overview of the prototype 15
- 16. Machine Vision System(HW) Overview of the prototype 16
- 17. Algorithm Structure • Normalizationand Histogram Equalization • Noise reduction (Gaussian, Mean, Median filters) • Edge and line detection (Canny + Hough, PPHT, LSWMS, EDLines) • Heuristics and Non-Maxima Suppression • Pixel to metric unit conversion 17
- 18. Algorithm Comparison Gap AGap B Algorithm Average Std. Dev. Average Std. Dev. Gauss + EDLines + NMS 19.992 0.370 5.487 0.407 Mean + EDLines + NMS 20.103 0.383 5.863 0.739 Median + EDLines + NMS 20.214 0.290 5.555 0.505 Gauss + LSWMS + NMS 18.991 2.197 5.846 0.797 Mean + LSWMS + NMS 19.643 0.389 6.256 1.134 Median + LSWMS + NMS 18.626 2.640 6.051 1.123 Gauss + Canny +PPHT + NMS 7.987 10.312 2.051 2.657 Mean + Canny + PPHT + NMS 20.151 0.392 5.333 0.417 Median + Canny + PPHT + NMS 23.184 6.380 5.521 1.304 Gauss + Canny + Hough + NMS 0.000 0.000 0.000 0.000 Mean + Canny + Hough + NMS 1.953 6.176 0.581 1.838 Median + Canny + Hough + NMS 4.017 8.470 1.128 2.391 Ground Truth 19.916 0.251 6.558 0.258 18
- 19. Integration Figure 7 –System Integration in Embedded Hardware 19
- 20. Ongoing: Results ofthe Measurement System (Best-Case) Gap B - Plate Bottom Gap A - Plate Top Mean Error Std. Dev. Mean Error Std. Dev. 0.143mm 0.084mm 0.780mm 0.157mm Measured/position x Ground Truth Repeatability Table 1 – Gaussian filtering + LSD by Von Gioi (2012) 20
- 21. Ongoing: Debevec’s HDRComposition HDR Input images HDR composed Line segment detection Final groove modeling 21
- 22. Ongoing: Method Comparison Figure17 – Error and Std. Deviation in millimeters for Gap A (smaller is better) 22
- 23.
- 24. Ongoing and FutureWork • Explore lighting options, noise suppression algorithms and image composition techniques to improve the system • Bilateral and L0 gradient minimization filtering (not trivial to implement) • Compare Debevec’s multi-exposure composition to other approaches that minimize the computational cost and are hardware-friendly • Online application - mapping while welding • Deep learning based image restoration • Produce a general purpose welding workcell 24
- 25. Conclusion • Modular Vision-BasedMeasurement for linear welding robots • End-to-end embedded welding system prototype • Computer Vision applied to reflective surfaces, without the need of structured light, polarized lenses or complex optical arrangements • State of the art algorithms offer better cost-benefit ratio • Developed a complete solution, featuring illumination, image acquisition and processing, robot operation and welding equipment setup 25
- 26.
Editor's Notes
- #2 Hello! My name is Cristiano Steffens and we are presenting a Robotic System for Welding Groove Mapping: Using Machine Vision on Metallic Surfaces. I am with the center of Computer Science at the Federal University of Rio Grande, in Brazil
- #4 Welding is a fundamental task in the heavy steel industry. Its automation is required in order to keep pace with the demanding and competitive market. Today, many industries still rely on manual welding process for industrial production. The manual process, however, has a direct impact on the quality of the final product. First and more visible at the business management level, as the manual process depends on the weldor’s ability and skill it is susceptible to human error. A moment of inattention is often enough to result in hours of rework and material waste. Sometimes, even if the visual inspection does not show that the process failed at some point, the final product may be rejected on the quality inspection stage or even reach the final consumer. You see, nobody wants to buy a buy a pig in a poke! (SIC) Finally, it is very difficult to ensure reproducibility and regularity on a manual process. Now, if we look at the welding activity from the operator side, we can note that the automation of the process would have an impact on its quality of life. Welding fumes have been associated to many lung diseases. The heat produced during the process, which the operator is subjected during many hours a day has also been associated with male infertility. The position and the repetitive movements are also associated with many occupational diseases.
- #5 When we talk about linear welding automation, 3 main approaches can be enumerated: The first approaches where based on structured illumination and cameras or specialized sensors. The main idea here is that you can project a pattern on the welding plate surface and it allows you to recognize the geometry. Latter, a touch and sensor has been proposed. It is a mechanical device that is able to follow or measure a groove. Some techniques also tried to find the proper settings for each welding plate by starting the process and then adjust the ecquipment parameters based on the electric arc measurement.
- #6 Here we can observe the typical setup for FCAW and GMAW welding used in shipyards. A 2 DoF robot is used to carry the welding gun and execute the weaving. A separate power source is used. The system runs over rails which can be attached to the ship hulk in many different positions.
- #7 Here we have a Picture of the robot operating.
- #9 The main contribution of our work is a VBM module to be used on top of the welding robot structure, assuming the control of the welding process. From a top view input image, we extract the groove edges and find its geometric properties. An Image Acquition module is implemented used off the shelf componentes and a Altera DE-0 Nano FPGA board. The designing and programming of this system is done using VHDL and Verilog hardware description languages. The Operations Unit is implemented in a standard PC software, which is developed in C++ taking advantage of many functionalities provided in the OpenCv Library.
- #10 present a cross-structure light (CSL) sensor, that consists a structured light projector and a camera, for weld line detection. The structured light projector projects cross laser beams on the weldment to form cross stripes, which are captured in images by a CCD camera for measurement. We use feature points, a planar target and a homograph matrix to calibrate the sensor. We also propose an effective approach to extract laser stripes in images for weld line detection. Experiments show that the CSL sensor can capture 3D information of the weldment with very low measurement error, and the weld line detection approach is effective in wall-climbing robotic platform navigation.
- #11 In many cases the automation of arc welding processes cannot be realized because the permissible workpiece tolerances are exceeded. Extensive workpiece preparations are often not practicable because of economic reasons. Therefore appropriate sensing systems for seam tracking and joint recognition have to be developed, which allow an adaptive control of the welding process and guarantee a satisfactory quality of the weld. Some special developments of optical sensing systems for automated arc welding are presented in this article.
- #12 presents a technology about real-time seam tracking, which is necessary to overcome the deficiencies of the teaching-playback welding robots in seam tracking control during gas tungsten arc welding (GTAW) process. A set of vision sensor system has been designed for the welding robot, which can acquire clear and steady welding images. By analyzing the features of welding images, a new improved Canny algorithm has been proposed to detect the edges of seam and pool, and extract the characteristic parameters of welding images. Based on the analysis of the characteristic of the real-time seam tracking, a segmented self-adaptive PID controller is introduced to the system, and some experiments have been done to testify whether the accuracy of the technology can meet the requirements of quality control of seam forming
- #13 On the right, we have a representation of a commonly used V-shapped welging groove. Here, we look for the properties that describe the groove geometry. Gap A is the distance within plates in the top part. Gap B is the distance within the places on the bottom part of the welding plates. Plate thinkness and computed bevel angle complete the description. On the left, we presente the trigonometrical basis that enable us to compute the geometry knowing only the working distance, lens and sensor properties.
- #18 The algorithm follows a straightforward approach. First we work on contrast enhancement and noise reduction. Then, we use line segment detectors to determine the edges of the welding groove. Once we have the candidate regions we apply Heuristics and NMS to remove any false positives. From the triangulation we can calculate the pixel to metric conversion. The measured groove dimensions are then used as input for the equipment configuration step.
- #19 Results show the EDLines from Akinlar et al. (2011) algorithm provides the best results. Using any of the noise suppression kernels its standard deviation is still lower than the other tested combinations on both Gap A and Gap B measurements. The next best solutions, which minimize error and standard deviation are obtained using the LSWMS algorithm by From Nieto et al (2011).
- #20 The proposed system is integrated in embedded hardware as shown in the figure.
- #26 In conclusion, we buit a modular VBM system for linear welding robots. We show that machine vision can be applied even in an hard contexto such as metallic reflective surfaces and that it can be done avoiding complicated hardware setups. We found that state of the art algorithms offer a better cost benefit ratio. And, finally, we presented a complete solution, featuring illumination, image aquisition and processing, robot operation and welding equipment setup.