The Development of Surface Inspection System Using the Real-time Image Processing JONGHAK LEE, CHANGHYUN PARK, JINGYANG JUNG Instrumentation and Control Research Group POSCO Technical Research Laboratories Pohang P.O.Box 36 Pohang-shi, Gyungbuk, 790-785 KOREA Abstract: We have developed an innovative surface inspection system for automated quality control for steel productions in POSCO. We had ever installed the various kinds of surface inspection systems, such as a linear CCD and a laser typed surface inspection system at cold rolled strip production lines. But these systems cannot fulfill the sufficient detection and classification rate, and real time processing performance. In order to increase detection and classification rate, we have used the Dark, Bright and Transition field illumination and area type CCD camera, and for the real time image processing, parallel computing has been used. In this paper, we introduce the automatic surface inspection system and real time image processing technique using the object detection, defect detection, classification algorithm and its performance obtained at the production line. Key-Words: - surface inspection system, CCD, object detection, defect detection, classification, cold rolled strip 1 Introduction In this paper we introduce the Surface Inspection System for cold rolled steel strip which was developed jointly with Posco (Pohang Iron and Steel Co. in Korea) and Parsytec (Parsytec Computer Co. in Germany) as a developer and a user of Inspection System. Posco have experienced lots of Surface Inspection Systems, which are equipped with laser techniques, on our several cold-rolling production lines past. But most of these systems have not satisfied operators for inspection of surface defects on the strip. Therefore most systems have been gradually away from operators and even abandoned from production lines finally. The main reasons are the number of the detected defects is quite limited, the resolution of the detectable defect size is low, and the whole inspection of the coil in high line speed is not achieved. In addition, it is difficult to adapt these systems due to the lack of flexibility and scalability to each production line which has own characteristic facility and produces different kind of materials. In order to overcome these problems, Posco decided to develop our own Surface Inspection System. For the development, we needed to prove the feasibility of an innovative technological approach about optical system for detection and system integration for real time processing. The central requirement for the surface inspection system, which can provide a certain set of features and functionality superior to other surface inspection systems available in industry and research, is as followings ; to prove the new concept of design for optical system with matrix camera and flash light at a speed up to 1000 mpm(meter per menute), to develop a real-time system for high speed data communication and processing and on-line visualization, to prove image capture and detection of smallest defects, to prove the concept of multiple matrix cameras to cover a large field of view up to 2m, to build a trainable system which can be adapted to individual defect characteristics, to classify detected defects on cold rolled material according to a user specified description, to design the system as flexible and scalable to adapt to various line conditions, to derive knowledge of the production process and production site to merge the gained experience and facts into design of the system, to develop the image processing software to detect and classify at high production speed.
2 Typical Surface Defects The number of defects that are controlled in production line of Posco reaches almost one hundred. But we can categorize them into major defect classes. In this section we describe the characteristics of defect classes that mainly controlled at line and targeted by the developed Surface Inspection System. Dent Dents occur when hard particles, grains of metal or sand, fall between a roll and a strip leaving imprints on the strip surface. The shape of dent depends on the side of the strip coil, if one side will show a deepening, the other shows rising. Roll Mark Roll marks are periodical surface defects. When roll takes slightly damage or a friction takes place between upper roll and lower one or strip, it continuously imprints roll marks into the strip surface with each rotation. Even if roll mark occurs on one side, there is no influence on other side unlike dent. Dull Mark Dull marks occur when steel particles are pressed into the surface during the dull treatment on surface of strip. Dull marks are shaped round or elliptical and are mostly distinguishable from the back ground by a lighter gray value. Slip Mark Slip marks occur when starting the line with a new strip coil. Two windings of the steel strip may slip against each other. The same defect may also be caused by abrupt stops of the line, however, when the strip slips over the rolls. Slip marks are small short scratches in strip moving direction and appear in a group. Scratch Scratches occur mostly due to an uneven tensioned steel strip during the production process or scratch by other material. The slips over a still standing roll thus causing the scratches parallel to strip moving direction. The features of scratch are line type. Scale Scales occur by scale being embedded into the surface of the material as a result of insufficient pressure water spraying (de-scaling) during hot rolling process. The scale may be removed again by pickling but the defect may not be completely eradicated during cold rolling process. The appearance of these flaws ranges from dot and linear to wide-area defects. Rust Rusts occur when leave strip coil long time under humidity environment. Because rust easily develops under high temperatures and high humidity of the environment, it usually occurs summer times. The shape of Rust is scattered small round dots and appears extensive area of strip coil. It has yellow to brown color. Dirty Dirty occurs when leaves cinders at electrical cleaning line or remain substances from environment. These substances are imprinted on the strip while is delivering on the run out table. Dirty has black or brown color. The shape is very various. Oil drop Oil drop occurs when the strip contaminated with oil is rolled at the temper pass mill. Oil drop has white color and is hard to erase. The shape usually is like spot or egg. 3 Development of Surface Inspection System 3.1 System Configuration In develop a Surface Inspection System we have focused on real time detection in maximum line speed of 1000 meter-per-minute (mpm) and strip width of about 2m, high resolution of defect size of 300µm by 300µm, high classification rates and own graphic user interface for automated quality control. The figure 1 shows overall concept of our Surface Inspection System. The system consists of three units. Dark Field Illuminatio n Data Base Inspection Console Bright Field Illumination Figure 1 System configuration Quality Control Unit One essential unit is an image capture system integrated in an inspection bridge consisting of a bright field and a dark field inspection unit. It surveys
about 20cm bandwidth along the moving direction of steel strip. Both illuminations together allow best detection of roughness changes (mainly dark field) and color changes (mainly bright field). This was unachievable target that the conventional system couldn t get. Thus a new generation of image acquisition and computing technology was required. The second unit is the image-processing computer, which collects the high volume of image data in real time and performs the image processing algorithms for detection and classification. It also calculates statistical data about the inspection process and records all information in database. And the third unit is the peripherals of the inspection system. This unit belongs to the graphic user interfaces (GUI) for research and production usage, the supervisory control computer (SCC), and other control terminals like the Sync-PC inside the Inspection Bridge. 3.2 Illumination Structure A illumination structure is essential part to determine the performance of Surface Inspection System since it makes defect image visualize on screen and analyze for detection of defect, feature extraction and classification. In our inspection system, we have used matrix camera for image capture and two different kinds of illumination method, bright field and dark field. field illumination has fine characteristic to visualize colored defects such as rust, dirt, temper color and etc. In contrast to bright field the dark field illumination has parallel lights and camera positioned at dark region where it has no reflection lights if the reflecting surface is totally even. In these situation dark field illumination visualize only roughnesschanged defects such as scratch, scale, slip mark and so on. The parallel emission lights for dark and bright field are infrared LED matrix illumination. 3.3 Software Description The many conventional inspection system have been equipped with various hardware like a signal processing board for detection and classification, which strongly depends on maker, are not easy to upgrade and does not have much flexibility for application. On the contrary, all functionality of our developed inspection system is implemented based on software algorithm. There are many kind of software modules having algorithms for object detection, defect detection, defect analysis and classification. 3.3.1 Object Detection The Object Detection (OD) algorithm is important factor to guarantee the real time performance of inspection system. The main rule of OD is to determine whether the captured image from camera has defects or not and where their position, if it has, is on the image. The OD needs reference image that has no defects on it. The reference image is automatically made by OD algorithm and updated periodically. Image Capture Fig. 2 Bright Field and Dark Field Illumination The figure 2 shows the principle of bright field and dark field illumination methods. In bright field illumination we have made its emission lights diffused, and its incident angle is same to reflection angle for camera. With this arrangement, the bright Buffer Object Image Obj CCD Camera Ref Reference Image F Reference Difference Histogram Image Threshold D Send to DD F Defect Fig. 3 OD(Object Detection) algorithm
The figure 3 shows the block diagram of OD algorithm. The object image that is captured from camera is compared with reference image, and then by some multi threshold method the existence of defects and the their position on it is decided. The object image and its defect information are sent to the next defect detection module if it has any defect. If the object image has no defects it is stored at reference image buffer. The reference image is updated by averaging the defect free images in the reference image buffer. 3.3.2 Defect Detection Defect detection (DD) algorithm segments defects image out of the object images sent by OD. The Figure 4 shows block diagram of DD algorithm. DD has sequencer and rule selector (SRS) for preprocessing object defect image. The sequencer algorithm assembles positions of defect to produce unit-processing region (i.e. sequencers), which are sent to filter bank in DD. The rule selector algorithm selects appropriate filters according to shape of sequencers. Thus this SRS saves many processing times. Defect Detection (DD) OD Preprocessing Sequencer Rule Selector Classification (CL) Filter Bank Linear Filter Area Filter Disk Filter Point Filter Region Analysis Classification Neural Network Fig. 4 Defect Detection and CLassification DD algorithm has a filter bank that consists of many kinds of filter to detect defects. At the moment the filter bank is composed of 4 kinds of filters such as disk type, point type, area type and line type. Each filter type has a characteristic to detect its typical defect kinds. For example disk type filter can segment defects such as roll mark, dull mark and oil drop. On the other hand, line type filter has a special ability to find out scratch and scale. And area type filter detects sort of irregular and volume detects. Finally point type filter can detect small but high contrast defects. The filter bank can be further extended by adding additional filters for detecting new type of defects that are not detected before. 3.3.3 Classification Software The outputs of filter bank are merged and then labeled by region analysis (RA) algorithm. RA joins all segmented pixels, which belong to a certain type of defect. And then RA give unique labels number to each defect types. The feature extraction (FE) algorithm extracts features on each labeled region. We have used three kinds of different feature categories. The geometry feature category extracts out all the features based on the shape, size and location of defect in an image. And the gray value feature category finds out features depending on the distribution of gray values of the segmented pixels. The third binary feature category evaluates the different segmentation algorithm, which is used in the DD filter bank. The 76 features can be extracted for classification in FE algorithm. Finally the CL (classification) algorithm normalizes the extracted feature values and classifies the defect name and grade. The CL in this system uses back propagation neural network algorithm whereas the conventional Surface Inspection Systems simply use table lookup table. The neural network has three layers including one hidden layer. The number of input neurons is 36, which means neural network use 36 kinds of feature set. 4 Performance Test In detection rate test, we compared the two defect maps of the coil, each of which was made separately by human inspector and Surface Inspection System at low and high line speed. To verify these test results, we examined the test coil at off-line. Following Table 1 shows the result. In classification rate test, we had the neural network trained eight defect names, which mainly occur in our production line. The main defects include dent, dirty, dull mark, roll-mark, rust, scale, scratch and slip mark. The test result shows about 90% of classification rate for name of defect, and about 98% for detection rate of defect.
dull mark and slip mark. speed Length Defect Detectio Classifi- mpm meter type (%) cation (%) 1 30 500 P/T 96.5 85.1 2 30 400 Dent 99.5 86.2 3 30 500 Scale 100 85.5 Hole 4 30 1000 Scale 100 95.1 S/M 5 30 500 6 30 500 7 30 600 S/M E/P D/M Rust Scale C/M 100 94.0 91.0 85.2 95.8 87.6 8 30 300 E/C 100 93.9 9 30-600 2200 10 1000 8000 Dirty E/P O/D 100 89.2 100 96.4 10 Coil Average 98.6 89.8 Table 1 Performance Test Results 5 Conclusions Posco have done a series of project for developing a surface inspection system to fulfill the customer s satisfaction for quality, to achieve the automated inspection and to effectively maintain the production process. As a result of these project, we already installed several surface inspection systems in the cold rolling mill, Hot rolling mill, stainless steel mill at the last year. From these projects, we can reach a conclusion for requisite condition as follows; parallel computing processors with matrix CCD camera allows high-speed surface inspection at 1000 meter /minute. software based system for all signal processing allows its application easy to tune, simple to maintain, and flexible to adapt in various environment. Reference [1]Choi Se-Ho, Development of SDD online signal processing system for cold rolled strip(1), Research Report, RIST, 1994 [2]A.K.Jain, Fundamental of digital image processing, Prentice Hall, 1989. [3]J.Serra, Image analysis and mathematical morphology, Academic Press, 1982. [4]I.Pitas, N.Sidiropoulos Pattern recognition of binary image objects by using morphological shape decomposition, Computer vision graphics and image processing, pp.279-305, Academic Press, 1992. [5]V.Cantoni, S.Levialdi, Pyramidal systems for computer vision, Spring Verlag, 1986. [6]R.Cypher, J.L.C.Sanz, SIMD architectures and algorithms for image processing and computer vision, IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. ASSP-39, No. 12, pp. 2158-2174, Dec. 1989. [7]SDD Development Team, SDD manual, Research Report, Parsytec, 1997 [8]Faustino Obeso, Intelligent on-line surface inspection on a skin pass mill, Iron and Steel Engineer, 1997. [9]John C. Badger, Automated surface inspection system, Iron and Steel Engineer, 1997. [10]Kim kyung Min, Development of surface inspection algorithm for cold rolling mill, CASE, vol. 3, No. 2, pp179-186, April, 1997..The surface Inspection system has to have : high resolution that can detect to at least 0.3mm 0.3mm of defect size. wide range detection capability that can detect to colored defect as well as rugged defect. high classification rates that can classify at least 10 kinds of defect in type of defect. From the results of performance test, our developed surface inspection system features : dark and bright field camera with IR LED illumination structure has shown high detection rates for rugged defect such as scratch, scale, roll mark,