[ 溶接学会論文集第 33 巻第 2 号 p. 156s-16s (215)] Image processing of the weld pool and tracking of the welding line in pulsed MAG welding * by Satoshi Yamane**, Katsuhito Shirota***, Sota Tsukano*** and Da Lu Wang*** In order to trace the welding line in the lap welding, weld pool in the pulsed MAG welding was taken by using the CCD camera. The weld pool was observed in the front and rear to the welding direction. The features of the weld pool were investigated to recognize the welding pool. The welding line was detected by using image processing and the digital controller was designed to trace the welding line. Key Words: Weld pool image, MAG welding, Image processing, Digital controller 1. Introduction Recently, it is required to reduce the production cost in the joining of materials in a thin board. The automation and the high efficiency arc welding have attracted the attention of many researchers 1), since the skill workers are reduced year by year. The mechanical strength of the welded joint in the lap welding depends on the welding quality and the tracking of the welding line. If the groove welding was carried out, it was easy to trace the welding line by an arc sensor due to the weaving 2). Since torch is not weaved in the lap welding, it is difficult to apply the arc sensor to the tracking of the welding line. Welders watched the molten region by eyes with the face shield and operated the welding torch so as to trace the welding line. Kuo and Wu took the welding line with a CCD camera and an assist lamp and proposed the image processing method to detect the welding line 3). The authors tried to use a CCD camera instead of human eyes during the GMAW welding. As the arc light is too strong, the taking of the weld pool is difficult. In pipe GTAW, Tsai et al 4) took the weld pool on back side of base metals without the arc light and adjusted the welding current to control a quality of the welding. It is required to take clear image of the weld pool. The welding current was synchronized with the timing of the shutter of the CCD camera. The authors proposed the visual welding system to take a clear weld pool image 5), in which the welding current reduced to 3A at the opening of the shutter. To apply this system conventional welding power sources was difficult. There is spatters in GMAW. It is one of the reasons in image processing error. In this research, the welding current is not reduced for the opening of the shutter. Since the pulsed welding current was used, the shutter opens at the base current duration. Since the image depends on the exposure time of the CCD camera, the relationship between the exposure time and the image was investigated. It is important to get the features of the taken image to detect the welding line. During the welding, the weld pool was observed from the front and the rear of the welding torch. From the experiments, it is shown that it is easy to detect the welding line in the rear image. The taken image of the weld pool was processed to detect the welding line. After that, the digital controller was designed by taking account into the dynamic behavior of welding robot axis. The fundamental experiment was carried out to verify the validity of the image processing method and the digital controller. 2. Experimental procedure 2.1 System configuration Robotic welding system is shown in Fig.1. There is three kinds * Received: 214.11.28 ** Member, Graduate, Graduate School of Science and Engineering Saitama University *** Student Member, Graduate School of Science and Engineering Saitama University Fig.1 Robotic welding system.
溶接学会論文集 第 33 巻 (215) 第 2 号 157s of personal computers, PCs. Two are for the real time robot control and the welding power source control. Welding conditions were stored in the PC controlling the robot. This PC sends the welding conditions to the PC controlling the power source with Ethernet. The trigger signal to open the shutter of the CCD camera, which was Hitachi KP-F12Cl camera, was synchronized with the current waveform. Its signal was generated by PC for the power source control and was sent to the CCD camera. The camera sent the taken image to third PC with gigabits Ethernet as digital number. Its PC processed its image, detected the welding line and sent to the PC controlling the robot with RS-232C. The CCD camera was fixed on the torch axis as shown in Fig.2 (a). The torch is inclined to the lap area as shown in Fig.2 (b). 2.2 Waveform of welding current and the timing of trigger signal The pulsed MAG welding was applied in the lap welding as shown in Fig.3. The peak current and the base current are 45 A and 6 A, respectively. The frequency of the pulsed current is 6 Hz. The weld pool is taken with the CCD camera, of which the shutter is opened by generating the trigger signal in the base current duration to reduce the affection of the arc light. Moreover, the interference filter of 95 nm was used to reduce the arc light 6,7), too. The typical weld pool images of the weld pool from the front and the rear were shown in Fig.4. When the weld pool was taken from the front, it is difficult to recognize the welding line, as shown in Fig.4 (a). But, the brightness of the welding line region in the weld pool image became darker than weld pool region, as shown in Fig.4(b). The welding line can be detected by using this image feature. The affection of the exposure time was investigated by taking the weld pool at 1.2 ms and 2.5 ms of the exposure time. The images are shown in Fig.5. The weld pool image at 1.2 ms is clear than 2.5 ms. But, the distribution of the brightness was not stable on the weld pool and the droplet or the spatter was observed in the image. Since its brightness was high, the dark region may be disappeared. Therefore, the image processing of the weld pool becomes difficult in the exposure time of 1.2 ms. On the hand, when the exposure time of 2.5 ms is about 2 times longer than exposure time of 1.2 ms, the brightness on the weld pool image becomes smooth. Since the spatter or the droplet is moved than the exposure time quickly, the taking of them becomes difficult. Although the image becomes like a blurred image, the brightness distribution becomes smooth. This image is suitable for the image processing than 1.2 ms exposure time. Therefore, the weld pool was taken by exposure time of 2.5 ms. (a) General view (b) Front view. Fig. 2 Relationship between the travelling direction and CCD camera. [V] (a) Taken image with CCD camera at the front position. Voltage 3 Triger signal Current [A] 2 4 2 [V] 5 Shutter open 5 1 Time [ms] Fig. 3 Relationship between the current and trigger of CCD camera. (b) Taken image with CCD camera at the rear position. Fig. 4 Typical weld pool image of pulsed MAG welding.
158s 研究論文 YAMANE et al.: Image processing of the weld pool and tracking of the welding line in pulsed MAG 2.3 Image processing of the weld pool It is important to recognize the welding line corresponding to the lap area. The authors tried to process the weld pool image by using HALCON 8), which is the image processing software for machine vision with an integrated development environment, due to ability of maintenance. The electrode wire position in the image was found during the set-up of the system. Since the arc discharges from tip of the electrode wire to the weld pool, the brightness distribution between the tip and the weld pool becomes high. As the welding line under the arc cannot Welding direction be found, first, the arc region was found in the weld pool image. The brightness distribution on the vertical line at the center of the electrode is investigated. Since the arc region is bright and the weld pool region is darker than the arc region, the threshold is found by using its histogram and OTSU method 9), which is one of the discriminant analysis method based on interclass variance. The binarization is applied to the image and the boundary between the arc and the weld pool is found. The ROI, Region Of Interest, was set to the region, including the lap region, under the boundary. The authors checked the brightness distribution on the horizontal line in ROI in Fig.4 (b). Their brightness distributions is shown in Fig.6. Although the brightness Welding near the direction welding line becomes dark than weld pool, it is higher than outside of the weld pool. The brightness on the horizontal line was divided into three group. It is difficult to apply OTSU method to recognize the boundary to the welding line. The authors apply binarization twice. First, the thereshold is found by applying OTSU method to the horizontal line. In first binarization, the mixture of the weld pool region and the welding line region is separated from the outside region of the weld pool in the image. Then, the author apply OTSU method to the horizontal line on the mixture region to find the threshold. In second binarization to the mixture region to divide into two group. Since the brightness of the weld pool is higher than the welding line region, in second binarization, the boundary of the welding line region is found. The horizontal line can be divided into five part according to the threshold in second binarization. Two is the weld pool region with the brightness over threshold. Another two is the weld pool region with the brightness below threshold. One is the welding line region. If the horizontal line is divided into three parts, the welding line region is disappeared. In this case, there is no welding line region. Therefore, several kinds of the horizontal lines are investigated. The each center of the welding line region is calculated and is averaged as the welding line position. If the spatter or the droplet is taken into the image, the welding line region is disappeared. Since the speed of the spatter or droplet is too high for an interval of taking the image, there is no spatter or no droplet in the next image. But the motion of the weld pool is slow for the interval of taking the image. The sequence images are shown in Fig.7. The minimum value at the same pixel position in Fig.7 (a) and Fig.7 (b) is selected as the new image. As the result, the spatter is neglected as shown in Fig. (c). 2.4 Design of the digital controller The characteristic of the digital controller in the control block diagram as shown in Fig.8 is D C z z 1 G z R z z 1 C (1) z where G[z], R[z] and C[z] is dynamic behavior of the robot, the reference and the desired response, respectively. The discrete time system to calculate the manipulator value u[n] was found from Eq. (1). u[ n] d e[ n] d1e[ n 1] d 2e[ n 2] d12e[ n 12 ] f1 u[ n 1] f 2 u[ n 2] f12 u[ n 12 ] (2) The manipulating value is calculated by the error and the (a) Exposure time is 1.2ms. (b) Exposure time is 2.5ms. Fig.5 Effect of exposure time to image. Fig.6 Distribution of the brightness in Fig.4(b).
溶接学会論文集 第 33 巻 (215) 第 2 号 159s manipulating variables, i.e., Eq. (2) corresponds to the weighted mean calculation. Even if the deviation e[n] has some noise, its affection is small in Eq. (2). Droplet 3. Result and discussion Fundamental experiments at the welding current of 18 A, the travelling speed of 1 mm/s and the wire feed rate of 98 mm/s were carried out under the shift from the welding line. The shift distance of the electrode wire from the welding line is -1., -.5,,.5, 1.mm. The processing result is shown in Fig.9. The horizontal axis corresponding to the number of the sampling, of which interval is 1ms. Since the weld pool is transient state in 1s after the start of the welding, the dark region in the weld pool corresponding to the lap area was not formed. Since the detected value depends on the position of the welding line, this value is useful as the deviation e[n] from the wire center to the lap area. The fundamental experiment was carried out to investigate the detection ability for the variation of the welding line. The start point is set on the welding line. The ending of the welding is sifted to 2mm away from the welding line. The bead appearance and the image processing result are shown in Fig.1. The deviation was detected as shown in Fig.1 (a). The deviation is small in the beginning of the welding and becomes big in the ending of the welding. The behavior of the deviation was agreed with the bead appearance in Fig.1 (b). The weld pool image was changed according to the progress of the welding. The validity of the detection was verified. In this setting of the CCD camera, 2mm corresponds to 8 pixels on the image. The suitable position of the torch is.5 mm away from the lap, i.e., the reference is.5mm. In order to examine the performance of the tracking system, the control experiment was carried out at the same condition as the above mentioned experiment, i.e., the ending of the welding was sifted to 2mm away from the welding line. The experiment result and the bead appearance is shown in Fig.11. In the case, the detection is almost kept constant without the teaching point to the robot, as shown in Fig.11 (a). The variation of the detection is below 15 pixels. Since the image of the molten area was processed, the image was influenced by reflection of the arc in the molten area. The dark part on the molten area may take place due to a diffused reflection of the unstable arc. The image processing error was caused at 4.5 s in Fig.11 (a) due to the dark part. As the digital controller has robustness, the influence was small to trace the welding line. Figure 11 (b) shows the image and the bead appearance. The weld pool image was almost kept (a) First weld pool image with droplet. (b) Second weld pool image without droplet. (c) Minimum between two images. Fig.7 Relationship of succeeds images. Fig.8 Block diagram of the penetration depth control. Fig.9 Image processing result.
16s 研究論文 YAMANE et al.: Image processing of the weld pool and tracking of the welding line in pulsed MAG [mm] Deviation 2 1 4 8 Time [s] (a) Behavior of detection point. (a)bead appearance and weld pool images. Fig.1 Fundamental experiment result for detection of the welding line without tracking. [mm] Deviation 2 1 Tracking start Reference 4 8 Time [s] (a) Behavior of detection point. (b) Bead appearance and weld pool images. Fig.11 Tracking experiment result. constant. A good tracking result was obtained and the validity of the tracking system was verified. Reference 4. Conclusions It is important to trace the welding line. But it is difficult to apply the arc sensor in lap welding. The weld pool image was observed with the CCD camera. The interference filter of 95nm was used and the shutter of the CCD camera was synchronized with the base current duration to avoid the affection of the arc light. The welding line was found by using image processing. When the exposure time is long, the image becomes stable in the lap welding and it is easy to process the image. The deviation from the welding line can be detected by using the image processing. The tracking of the welding line was carried out by using the digital controller so that the deviation becomes small. The validity of the image processing and the tracking system was verified by carrying out the welding experiment. 1) T.Ueyama, T.Ohanawa, M.Tanaka, K.Nakata, STWJ, 6(25) 75-759 2) S.Yamane, H.Yuzawa, Y.Kaneko, H.Yamamoto, M.Hirakawa, K.Oshima,: Image Processing and Control of Weld pool in Switch Back Welding without Backing Plate, Quar. J. JWS, 23(25),65-7 (in Japanese) 3) Hsing-Chia Kuo, Li-Jen Wu: An image tracking system for welded seams using fuzzy logic, J. of Mat. Proc. Tech., 12 (22), 169 185 4) Chiung-Hsin Tsai, Kuang-Hua Hou, Han-Tung Chuang: Fuzzy control of pulsed GTA welds by using real-time root bead image feedback, J. of Mat. Proc. Tech., 176 (26), 158-167 5) K.Oshima, et al : Digital Control of Torch Position and Weld Pool in MIG Welding Using Image Processing Device, Trans. of IEEE IAS, 28(1992), 67-612 6) W Lucas, L Mcllroy,J.S Smith : Review of Industrial Sensors for Control of Arc Welding, Proc. of IIW Commission XII, Doc.XII-1566-99(1999), 45-61 7) H.Wada, Y.Manabe, S.Inoue, H.Yoritaka : Group-control System of Narrow-Gap MIG Welding, Sensors and Control Systems in Arc Welding(199), II-61-II-64(in Japanese) 8) MVTec Software GmbH: HALCON 12 User's Manuals (214) 9) N. Otsu, A threshold selection method from gray level histograms", IEEE Trans. Syst. Man. Cybern., SMC-9 (1979), 62-66