Advanced Mechatronic System For In-Line Automated Optical Inspection Of Metal Parts Tomasz Giesko, Adam Mazurkiewicz, Andrzej Zbrowski Institute for Sustainable Technologies National Research Institute Pulaskiego 6/10 str., 26-600 Radom, Poland tomasz.giesko@itee.radom.pl, adam.mazurkiewicz@itee.radom.pl Abstract: The paper presents a solution of the mechatronic system for in-line optical inspection of the mass produced metal parts in the automotive industry. To replace human visual inspection, a method based on machine vision, computer image analysis and mechatronic modules were applied. To inspect a variety of defects (e.g. cracks, scratches, grinding defects, spots, material losses) a sequence of 5 pictures of the object is analysed by specialised algorithms during an inspection cycle lasting a maximum of 300 ms. In order to achieve micrometric measurement accuracy and high inspection process efficiency, crucial problems concerning the cleaning of the inspected surface, surface illumination and mechatronic hardware were studied and solved. Some aspects of knowledge transfer to the industry are discussed. The developed inspection system has been successfully implemented in the manufacturing line of bearing rollers. Tests results and the basic features of the system are presented. Keywords: optical inspection, metal parts, mechatronic system. I. INTRODUCTION In industry, particularly in the automotive branch, there are strong trends to apply highly efficient automated systems for quality inspection of mass produced parts [1-3]. Manufacturers needs are focused on the achievement of the highest product quality, according to a zero defect strategy. During the last decade, automated inspection has become a global trend in industry. Now, mechatronic systems for quality inspection help to extend an advantage in the competition of manufactures [4-5]. Bearing production is an example of intense competition between companies where the reduction of part defects is one of main goals. In the production of bearing rollers, surface defects on metal parts are a crucial problem that causes damage to bearings. There is a variety of defects found on bearing rollers that are difficult to detect in many cases [6]. By necessity, human visual inspection is still being applied widely in manufacturing lines. There are critical limitations to this method, such as nonrepetitive and limited accuracy with varying reliability and effectiveness [7]. The critical factors were identified during extended investigation in the production line of bearing rollers. The mechanical impurities, condensates, protective liquid, vibration, and lighting changes can disturb the process of image acquisition and negatively influence the quality of inspection. The conditions in the line were defined as very hard. In the first phase of work, 20 defects identified under human inspection were presented by the manufacturer [8]. The main defect classes of the face of bearing rollers were face geometry, presented by ellipticity and eccentricity, material defects, such as loss of material, surface flaws, such as cracks and scratches, grinding defects and spots. The requirements for measurement accuracy of about 50 microns were placed on the inspection system by the manufacturer. An industrial system should ensure in-line inspection and selection of the parts with a speed appropriate to the line rate. In this case, for the bearing production line, with an output of about 10,000 parts per hour, an inspection process rate of 2-3 parts per second was expected. This case study shows that research should be concentrated on the following: Proper visualisation of the inspected surface to detect all defects, A specialised algorithms for image analysis, The elimination or reduction of factors making the optical inspection difficult, and The development of the mechatronic hardware that will ensure a high rate of parts transportation through the inspection zone. 33
CCD camera Front lighting Dark-field lighting Surface to be inspected Back lighting Figure 1. Arrangement of the vision system. Trigger signals 1 2 3 4 5 Pictures Figure 2. Picture acquisition and lighting timing. Transfer to PC Surface inspection on metal parts involves specific methodology problems. The particularly significant effects are surface reflection and different contrasting surface texture in the micro-scale [9]. The shadow effect can cause measurement errors and faulty identification of defects [8]. The problem of lighting source angle to achieve an optimal illumination of the metal surface considering microstructure was discussed in [6]. Solutions of mechatronic hardware applied in industrial inspection systems of metal parts are not described. II. OPTICAL INSPECTION METHOD The vision system that enables one to visualise defects collected in the specially prepared catalogue consists of a monochromatic CCD camera, low-angle lens, and a multi-lighting system (Fig. 1). Using the 1.3 megapixel camera, an optical resolution of about 30 microns was achieved. Illumination problems in optical inspection have been presented in a number of papers [5, 10]. To ensure appropriate illumination of the inspected surface, the lighting system was developed during the research of the interaction of light with a highly reflective metal surface. There are three LED illuminators in the system: front and dark-field ring-lights, and a backlight. The front diffused red light was used to expose the part s edges and a special class of material defects. In the case of micro-defects, such as cracks, scratches and grinding defects, the directional dark-field red and blue lighting was applied. The best illumination angle has been determined as a compromise between the need to achieve high contrast of surface defects and the limited minimum distance between the vision head and the face of the roller. The backlight illuminator mounted under the roller was used for enhancing the edge contrasts when measuring the geometry of the face of the roller. Only when the geometry of the vision system is adjusted during experiments do surface defects on the face of the roller become appropriately visible. The problem particularly concerns lighting setup. In this system, the following 5 lighting techniques were applied to ensure the visualisation of identified defects: red colour front ring-lighting, red colour dark field ring-lighting, blue colour dark field ring-lighting, directional dark field lighting and white light back lighting. This means that a collection of 5 pictures of the same surface are analysed in one inspection cycle. Due to needs related to highspeed inspection, a total inspection time per part (including image acquisition and analysis, part moving) should be less than 300 ms. triggering, camera exposure, and image acquisition are synchronised using a PC platform (Fig. 2). In automatic inspection, if the part is moving, the problem of image blur requires analysing [2]. Theoretically, images can be captured while parts are moving continuously under the camera. In this situation, solving the image blur problem is possible by reducing the camera exposure time. Of course, it involves the necessity of increasing lighting intensity. However, when high precision metal surfaces are inspected and high measurement resolution is required, an accurate and repeatable positioning of the part under the camera ensures the minimization of errors. Experiments demonstrated that vertical deviation of a few degrees of the roller caused problems in surface visualisation because of changes in reflected light. In conclusion, the stable position of the bearing roller is needed to obtain high quality images of the inspected surface. III. EXPERIMENTAL STUDIES AND SOLUTION In general, the measurement methodology was based on a machine vision method and mechatronic hardware. The experimental setup was arranged to study the following problems: Surface defects detection under variable conditions including illumination, kinematics and impurities; Optimization of illumination; Cleaning the roller surface of grinding mud and condensed liquid; The feeding and selecting of parts while keeping the pace adjusted to inspection speed. 34
All the known defects of the metal surface were analysed under illumination applying lighting techniques mentioned previously and classified in groups according to origin of the defect (Fig. 3). Most of them include process defects, particularly grinding. During advanced experiments, the catalogue of defects was extended from 20 to 30 types, when the optical system extended the possibility of detecting subtle defects. In the image analysis and defect identification the morphology dilation and contour recognition algorithms were mainly applied [8]. For defects, special features were extracted and assigned to them. In consideration of the limited inspection cycle time the number of features were optimised and reduced to less than 30. According to specific characters of defects, microstructure and surface reflectance among them, a number of types of illumination were modelled. Images of selected defects that were obtained using implemented lighting system are presented in Fig. 4. Front lighting was achieved by the red colour ring illuminator lighting directly the surface and was employed to detect macroscale surface defects: geometrical errors, material losses, and outflows (see Fig. 1). In cases, such as scratches, micro-scale grinding defects, and corrosion, dark-field red and blue colour lighting were used. In dark-field lighting the low angle of illumination was used. A special solution was developed to detect material lamination after cold forging. The dark-field ring illuminator consists of three concentric segments of the angle 120 (Fig. 5). The surface is illuminated in Scene 1, Scene 2 and Scene 3 under the directional lighting. Three captured images present generated shadows of defects on the surface. The reference image is created as the superposition of three component images where the gray level in each point in the reference image is determined as the average of corresponding points of three images. Next, every component image (1, 2 and 3) is compared to reference image. Differences of light intensity in every point on the surface are determined in relation to the reference image. Using following image analysis algorithms: operation of segmentation, nonlinearity correction and contouring it is possible to determine the shape of defect. Material Process Forging Hardening Grinding Transportation Loss of material Crack Loss of material Hollow absence Lamination defect Outflow on the edge Absence of hardening Double grinding Absence of grinding Non-uniform grinding Scratches Chafe marks Scratches on the edge Corrosion Spots Figure 3. Tree of defects (selected from 30 classified). Loss of material Crack Outflow Non-concentricity Non-ellipticity Corrosion Double grinding Grinding defect Scratches Figure 4. Images of defects. 35
CCD camera PC Step 1 Sucking out Step 2 Blowing away Step 3 Vaporizing Compressed air Front stream Compressed air Front stream Hot air - Scene 3 - Scene 1 Image composition Vacuum Roller Angle stream Figure 7. Cleaning of rollers before inspection. - Scene 2 part [ n+2] Image acquisition Figure 5. Image superposition method. Step - t 2 t a Step - t 3 Analysis Decision part [ n+1] part [ n+3] Cleaning t c t s Selection Step - t 1 out Clear surface Liquid impurities t f Picking up part [ n] in Figure 6. Reflectance disturbances on the surface introduced by liquid. Presented defects inspection is possible when a surface is free of liquid or solid impurities. These impurities that are settled on the face of the roller are in the form of solid particles of grinding material and liquid marks (Fig. 6). When the size of impurities exceeds measurement resolution, they can be interpreted as defects by the system. Liquid marks and smudges cause following effects: reflections, glows, and lens phenomena. These effects are strongly intensified on a high-reflectance polished metal surface. Experiments have shown the absolute necessity of cleaning the face of the roller before inspection. Effective cleaning of the surface is a critical factor in the inspection. In the developed method of metal surface cleaning, a pneumatic technique was applied (Fig. 7). In the first step, rollers are washed in a special machine. After washing, rollers are fed into the cleaning setup, where, by using an appropriately shaped stream of compressed air, impurities are blown away together with liquid from the cleaned surface. Removed impurities are immediately sucked up by the absorber. In the next step, the remains are blown away by a set of air jets. The stream of warm air allows the evaporation of the remains of liquids from the surface. Figure 8. Task diagram. The developed cleaning method was tested on a series of rollers of different diameters. The obtained results have shown that this method is effective in a number of applications for the cleaning of metal surfaces from solid and liquid impurities. In order to obtain a high inspection rate, a part transporting system was developed. All parts are moved through the following zones: cleaning, inspection, and selection. The rotary feeder was used to feed rollers and keep them in a stable position while capturing the images. The process control of mechatronic modules was designed where tasks are executed simultaneously. Three tasks were executed at the same time: cleaning the (n+1) part before inspection, capturing the images of (n+2) part, and directing the (n+3) part into a quality group (Fig. 8). Image analysis and classification are processed when the part is moving to the selector. The cycle time was specified using a simulation model, considering the time needed for the inspection of the current part in the line. Minimum cycle time is determined by the following equation: 3 i t t t max ; t ; t ; t, min 1 f c a s 36
where: t i the time of feeder step (rotary cycle of feeder consists of 3 steps; t i < 300 ms), t f the time of part picking up from the supplier pipe (t f < 200 ms), t c the time of cleaning (t c < 200 ms), t a the time of inspection (t a < 250 ms), and t s the time of selector activation (t s < 100 ms). Presented values of time parameters were determined experimentally and are related to the developed system. IV. SYSTEM ARCHITECTURE The system architecture is based on the following main principles: modular structure, flexibility, the capability to expand its features, user-friendliness, and reliability. The developed Automatic Optical Inspection (AOI) system structure is shown in Fig. 9. There is a CCD camera and the illumination setup in the vision module. LED illuminators are controlled by a PCI input/output card embedded in the PC. Mechatronic modules are controlled by a PLC that enables the avoidance of excessive CPU issues. The sensors were applied to control the mechatronic modules and to detect the presence of the parts in the inspection area. The computer system interface provides communication between the operator and the AOI system, setting the inspection parameters and the inspection process monitoring. The software includes the catalogue of defects and statistical classification. The Intranet/Internet connection enables the operator to oversee the inspection process from any place. Access to the system is possible after authorization. The applied remote service operation, also called telemonitoring, is an advanced and innovative solution that enables optimisation of system setup parameters. In view of high efficiency requirements, the duo-core processor based PC was applied in the system. As shown in Fig. 9, input data including digitized images are transferred from the camera to the PC. After image analysis, the generated output data are used to operate the actuators in mechatronic modules. Implemented in the manufacturing line, the AOI system runs in open-loop process. The feeding of the next part is possible when inspection and selection of the current part are finished. Using continuous on-line information from the inspection system, line personnel are able to correct manufacturing process parameters to meet quality demands. Cleaning setup Vision module CCD camera Illumination setup Feeder Selector Image data transfer FireWire PLC RS232 Figure 9. System structure. Interface Analysis PC Software I/O PCI Intranet/Internet Execution Figure 10. Implemented inspection system of bearing rollers. V. TESTS RESULTS AND DISCUSSION The system was implemented in the manufacturing line of bearing rollers (as shown in Fig. 10). During several days of system testing, the obtained results proved its usefulness. To verify the sensitivity and reliability of the system, two experiments were performed. In the first one, a batch of about 30,000 rollers was inspected by controllers, and then it was inspected with the use of the designed AOI system. Among parts at first classified by human controllers as good, 1.5% were recognized by the system as defective. At the same time, a small number of parts classified by human controllers as defective were verified by the system and recognized as good because all measured defects did not exceed the limits of parameters. In another experiment, 100,000 parts were inspected by the AOI system. At the end of line, rollers were inspected again by human controllers. About 100 rollers were recognized as defective in this way; however, another analysis showed that all detected parts had surface defects that did not exceed the setup limits. Experimental results proved the high sensitivity and repeatability of automatic optical inspection. The rate of Decision 37
inspection and selection process depends directly on time parameters shown in Equation 1. When the continuous feeding of rollers was assured the process rate value was 3 parts/second. Performed tests proved that the zero defect level was fully achieved for defects that are included in the catalogue of defects. The development and application of the automated optical inspection system is a model example of knowledge transfer to the industry. A number of problems in the implementation of the presented system in the company arose from the staff s mistrust of the system s capabilities and their low experience of advanced inspection systems. Because of that, training courses on the new inspection methods, apart from in-service training, were carried out. After a short time, the trained workers were ready to operate the machine independently. VI. CONCLUSIONS The advanced optical inspection methods are very well suitable for full automation of parts inspection in mass production, fulfilling the high requirements of inspection accuracy and process effectiveness. A modern mechatronic approach enables the system implementation and integration with the existing manufacturing line and keeping the line rate without changing the positions of machines. The synchronisation and simultaneous execution of tasks were used to achieve a high inspection speed. The developed and implemented inspection system is a solution that meets the high maintenance requirements in difficult industrial conditions, particularly in the grinding lines of bearing rollers. The test results obtained during a one-year exploitation of the system have proved its features and usefulness. The increase of quality products and the reduction of compliance costs were noticeable at that time. The developed system of optical inspection is distinguished by the following: The possibility to inspect on-line many more defects during one inspection cycle in comparison with human inspection, High inspection speed that enables the application in high rate manufacturing lines, and The flexibility of hardware and software that enables the reconfiguration and application of the system due to user s needs. Thanks to the flexible and reconfigurable hardware structure and programming of the parameters, the developed systems can be applied in various industrial inspection systems where high cleanness of the inspected surface is required. ACKNOWLEDGMENT This scientific work was financed by the Ministry of Science and Higher Education of The Republic of Poland and carried out within the Multi-Year Programme Development of innovative systems of manufacturing and maintenance 2004-2008. Research was carried out by the team of scientists and engineers from Institute for Sustainable Technologies and Wroclaw University of Technology. REFERENCES [1] G. Pahl, W. Beitz, J. Feldhusen, K. H. Grote, Engineering Design, Springer-Verlag, London, 2007. [2] P. West, High Speed, Real Time Machine Vision, Imagination and Automated Vision Systems. [3] N. Zeuch, Understanding and Applying Machine Vision, Marcel Dekker Inc., New York, 2000. [4] A. Dashchenko, Manufacturing Technologies for Machines of the Future, Springer-Verlag, Berlin, 2003. [5] A. D. Marshall, R. R. Martin, Computer Vision, Models and Inspection, Word Scientific Publishing Co. Pte. Ltd., 1993. [6] F. Perkopf and P. O Leary, Visual Inspection of Machined Metallic High-Precision Surfaces, EURASIP Journal on Applied Signal Processing, vol. 7, 2002, pp. 667-678. [7] C. Demant, B. Estreicher-Abel, and P. Waszkewitz, Industrial Image Processing, Visual Quality Control in Manufacturing, Springer-Verlag, Berlin, 1999. [8] A. Mazurkiewicz, Development of Innovativeness Systems of Manufacturing and Maintenance 2004-2008, Final Report, Institute for Sustainable Technologies, National Research Institute, Radom, 2009. [9] T. Giesko, A. Zbrowski, and P. Czajka, Laser Profilometers for Surface Inspection and Profile Measurement, Problemy Eksploatacji 1/2007, Institute for Sustainable Technologies, National Research Institute, pp. 97-108. [10] B. G. Batchelor and P. F. Whelan, Intelligent Vision Systems for Industry, Springer-Verlag, 1997. 38