Industrial Computer Vision

Transcription

1 K. S. Niel, Industrial Computer Vision, Proceedings of Image and Vision Computing New Zealand 2007, pp , Hamilton, New Zealand, December Industrial Computer Vision K.S. Niel 1 1 Dept. Automation, University of Applied Sciences Upper Austria, 4600 Wels, Austria. k.niel@fh-wels.at Abstract Three different out of many aspects of the use of machine vision in the industry will be highlighted: There is the challenge to move scientific efforts to the practical given factors in manufacturing environments. How the community of machine vision is established in the economy. At least what is the moving spirit of a faculty to promote R&D. In order to successfully implement a computer vision system within a running industrial process there are many key points to be aware of. There are various barriers and pitfalls associated with some of the interfaces: Timing, material transport, mechanics, environmental pollution (steam, heat, water, and oil), maintenance, and budgetary considerations, to mention a few. The wide field of view necessary when implementing a machine vision system drastically increases the amount of measures required to keep the system in operation. The failure of many MV systems is caused by paying too little attention to minor interface effects. The situation of machine vision community in Austria, which in relation roughly is comparable to the EU, is described: Austria has 8 Mio inhabitants; its income depends on industry, agriculture and tourism. There are about 200 engineers working full time in the R&D field of machine vision. Industrial computer vision takes about 50 %. There are seven university institutes, seven R&D associations and some SME in this field. R&D at the University of Applied Sciences Upper Austria deals with a combination of different scientific/technological areas like sensor system, physics, and machine vision. This R&D supports mainly the local manufacturing industry. Three examples will show the result of the collaboration between different approaches to the given task. Keywords: Machine vision, industrial applications, technological approaches, machine vision community. 1 Introduction Speaking of machine vision within the scientific community, one may hear a variety of statements about its importance: There is no need for research all essential algorithms have been known for decades (Fourier-, Hilbert-, Radon-, Hough-Transformation, morphology, PCA, MSC, etc.). The algorithms are well designed and stable the actual problem still is the missing computational power; research should be done to speed up the evaluation process. There is a lot to do image processing nowadays is on the cusp of really supporting getting a sensor which is comparable to the human eye (face recognition, guiding system for autonomous vehicles, robots, etc.). In industry there are a vast amount of measurement techniques for processes which could be, but are not aided by machine vision [1]. The faith in cameras by engineers, who are responsible for the process, is weak. On the one hand investments in machine vision solutions are high, while the result is not guaranteed, and therefore the risk of false investments is significant. On the other hand there are many articles in technical journals presenting only success stories. The message is subtle: Just buy a camera, do some parameter adjustments and you get an ever lasting, self-repairing, intelligent machine vision sensor. As in other fields of science there is a gap between the academic and the industrial standpoints. This paper outlines three aspects to illuminating the status of the computer vision within the industry. 2 Success of a machine vision project In order to successfully implement a computer vision system within a running industrial process there are several key points to be aware of. The sensors selected have to be suited to the specified environmental conditions. The equipment on stock has reached a suitable status to be able to fulfil specific needs at acceptable costs. The main algorithms (libraries) are well tested and have reached a stable status. The evaluation units can be selected 198

2 for nearly every class of robustness needed. These are the core issues, in addition to the intrinsic machine vision solution, which comprises the content of this and similar conferences, but is not the main topic of this paper. Barriers and pitfalls to the implementation of industrial computer vision applications lie within the various interfaces to the process: Timing, material transport, mechanics, environmental pollution (steam, heat, water, and oil), maintenance, and budgetary savings to mention some. The large field of view necessarily implemented by machine vision applications increases drastically the amount of measures to keep the system in operation. A lot of applications fail caused by paying too less attention to minor interface effects. 2.1 Tasks To get from the idea to the product a lot of different tasks have to be fulfilled in order to successfully implement the application. The academic task, developing the centre algorithm takes just 5 % of the total implementation requirements [2]. Other tasks at the end deal with maintenance, or training of the client to keep the system running. At the beginning one has to prove the idea to get funds for the R&D process. He has to organise the mechanic construction to put the lightning and capturing devices on site. Compared to standard sensors like a light barrier or a thermo couple the measuring field of a machine vision system is large. To keep it clean under all circumstances is a challenge e.g. in hot rolling mills like given in figure 1. To make a rough estimate essential tasks are listed as follows (list not exhaustive): HW components: camera, lightning, frame grabber SW components: operating system, MMI, sequence control, communication, metrology, error handling, statistic, camera and lightning control Algorithms: edge detection, threshold, line detection, plausibility check, optical rectification, calibration, pattern matching, error rejection Theory: radiation, optic, noise, coordinate transformation, classification, stereo, fuzzy logic, Fourier transformation, principal component analysis, multi scale analysis Process: heat, speed, pressure, physic, deformation, assembling Environment: oil, steam, heat, ice, water, vibration, dew point Automation: reliability, quality, sensor, actor, visualisation, process leading system Mechanic: stability, potential free, vibration, accessibility Operation: clear, straight forward, alarm, error handling, hints for maintenance Maintenance: calibration, accessibility, signal check, change of components, training Client: acceptance, ROI, reference, training, language, sub supplier, set to work, support Sales: advertisement, brochure, contract, guarantee, calculation, discount, delivery R&D: idea, funding, paper, networking, simulation, literature, measurements, rapports, patents, conferences, product design, prototype. Figure 1: Industrial Machine Vision Application - shape detection at a rougher mill for steel plates. If just one of the many tasks fails the aim of the project will not be reached. There has been one for example where a side effect killed the whole application: Within the rolling process at a hot rolling mill of a steel plant one out of many parameters to operate the material is the centre line deviation during the transport on the roller table. The steel plate is transported at up to 10 m/s. If it crashes into the side construction either the quality of the strip is reduced or the continuous process has to be stopped to remove the stuck material. There have been arranged six stereoscopic camera systems in a row to detect the centre line deviation and by this information to control the lateral movement of the stripes. The 199

3 system worked well for weeks after erection. But after this period the optical system got errors due to the heavy vibrations of the mechanics where the cameras had been mounted on. The maintenance effort increased dramatically. Finally the system was closed and dismantled. The management realised, that the monetary turn over would never be reached. A similar application with a comparable HW but divergent metrological aspects succeeded at the first plant. And this system has been sold several times to other clients too since than. 2.2 Partners The main partners of a project are the developer, the man with the money, the maintenance personnel, and the process engineer [3]. Who has the most relevant impact to the correct operation of the system on site? Every one of the above mentioned partners can be blamed for: The developer knows the system best, the man with the money decides whether or no to start the project, the maintenance stuff knows the system and its behaviour within the automation level, and the process engineer has the best knowledge about the relevant measures and needs for the production line. There is no way to answer this question properly. For machine vision the uncertainty regarding the understanding of the system in detail is big for those, who are not direct involved in development. On the one hand it must be easy: what a human can see a machine has to see equal or even much better because of the reliability of the sensor. On the other hand: how can the camera decide between a grey object and the grey background? This leak of understanding leads to wrong emphases of the distracting effects to the system. It is well known, that a good illumination of the scene makes it easier for the software to get proper results. But it should additional be clear, that a good knowledge of the process by the system developer makes the system reliable during a long period of operation. Therefore all the partners have to have a close contact for setting up and keeping the system alive for long terms. 2.3 Barriers and pitfalls Technological barriers on realising camera driven sensor systems differ from standard sensors like light barriers. The most important fact is the measuring field, which is big. The camera has the advantage to collect a huge amount of information stream (about 10 MByte/s) with one single sensor. But just a small piece of this information stream is necessary to detect the requested result (from 1 bit/s OK / NOK up to about 400 Byte/s e.g. X/Y shape of a plate). The goal of industrial machine vision is to find the one bit out of 10 MByte and to forget the other MByte the classical needle in a haystack approach. The aim of the theories in machine vision is to help finding the needle. The reduction of information has to follow certain steps depending on the application. There is always: the image of the scene illuminated by any light source and captured by the camera some pre-processing (e.g. flat field correction, noise suppression, region of interest, threshold) object selection and classification evaluation of the result. The art of applicable solutions is to find the correct tools to get the needle and to reject distortions at the same time. Pitfalls lie in unexpected situations which do not fit to the original design of the application. 2.4 Economic planning When the system is in operation and the bill is paid there will be periodical costs for keeping the system alive. Generally it is a hard job to calculate these additional costs. It needs some experiences regarding the production and maintenance procedures. In table 1 there is one example how to get roughly the estimated expenses. Table 1: Calculation of maintenance expenses (e.g.). Item (e.g.) unit EUR p.u. units p.y. EUR p.y. Cleaning 2 h ,000 front glass Check device 3 h ,400 on site Maintain 10 h ,000 device in shop Recalibrate 2 x 3 h ,400 device on site Repair / reassemble / recalibrate 20 h ,200 Redundant system while the original one is out of order 24 h ,800 Maintenance - Total EUR per year 22,800 p.u. per unit; p.y. per year Depending on the system costs sometimes the maintenance during live time needs more money than to get the new system. Unfortunately there is sometimes very low awareness at the plant engineers to pay special attention to the machine vision application including fine optics, micromechanics, and wide field of measurement. This leads to 200

4 Table 2: Employees related to computer vision in Austria Organisation Type / origin Empl Focus Universities (7) R&D, education / automation, biology, computer science, electrical engineering, mathematics, physics Research institutes (7) Engineering companies (10) R&D / computer science, electrical engineering, mathematics, physics Engineering, some R&D / automation 38 Automation, medicine, metrology and control engineering, remote sensing, restoration, sensor technology 114 2D texture, 3D restoration, automation, high speed, high performance, medical engineering, metrology and control engineering, remote sensing, sensor technology 36 Automation, engineering, metrology, sensor technology decreasing reliability of the system within the automation process and after some months of operation the application is switched in stand by mode because it just makes troubles. The task of a process engineer is to know the certain behaviour of the sensor systems. The task of a developing engineer is to look for measures to keep the system easy to use and robust against the environment within manufacturing processes. These environmental conditions sometimes can be very harsh. Putting the tasks of development and maintenance together in a proper way will increase the number of successful operating computer vision systems. 3 Computer vision community in Austria (resp. EU) The situation of machine vision community in Austria, which in relation roughly is comparable to the EU, is as follows: Austria has 8 Mio inhabitants; its income depends on industry, agriculture and tourism. There are about 200 engineers working full time in the R&D field of machine vision. Industrial computer vision takes about 60 %, the rest is medical imaging, environmental and remote sensing, restoration. There are seven university institutes, seven R&D associations, and ten SME in this field (only institutes counted which are originated in Austria) (see table 2). One huge manufacturing process is the steel industry. There the automation process has reached a high level, the monetary productivity is high and therefore money is not essential for sensors if they fulfil the needs of the process. The second group deals with supplying the European car industry. There the reliability is important too and the costs of new systems including the necessary maintenance procedures have to have a pay back of 6 to 24 months. After all there are a lot of small slot applications where the owners more or less don t have an exact knowledge about the according costs of the system when they think about a machine vision solution. Overall Austria has a well organised industrial infrastructure and the automation level is pretty good. Looking to the amount of employees working exclusively in the R&D field of computer vision there is a promising background for industrial machine vision. 3.1 Different faculties There are some different approaches dealing with image processing. Depending on the education of the involved engineers solutions for equal tasks appear different: There is electrical, informatics, mathematics, communication engineers in the near field, mechanical engineers or material scientists in between, and scientists in medicine, biology, agriculture, geodesy in the far field to machine/computer vision respectively image processing science. The focus on an industrial machine vision application will differ between the computer science engineer, the mathematician, the physicist, the electrical and the automation engineer. Additionally every faculty approach has its advantages but also it s borderline. Every faculty group will concentrate on certain classes of applications. Applications without a pressure group get lost. Clever net workers identify these slots and encourage different faculty owners to merge their knowledge and find new areas for making business. 3.2 Different integration level The integration level differs from institution to institution. Generally the academic is interested in profound and systematically developed algorithms. In general they are paid per faculty owner by the public government. R&D institutes rely on the needs of the industry, take the state of the art, and realise prototypes for the certain requirement. They are paid per R&D project by funds which are raised by the public or by companies profiting from the increased knowledge. Engineering companies are close to the client. They implement the applications for the 201

5 manufacturing process and perform maintenance also. They are paid per successful application by the client. At least there are the distributors who deliver components. 3.3 Cooperation There is an increasing network between the different players of the community. Every student of our university has to do his practical term at a company. Most of these first relationships tend to result in hiring of the graduates by these companies. There are some firms who in the last years hired a lot of graduates (one or two per year just in the field of machine vision). This sums up in natural cooperation between the local industry and our university. Additionally it is our own interest to get and stay in contact with the Austrian, the European, and the world wide community in the field of computer vision. 4 R&D at the University of Applied Sciences of Upper Austria R&D at the University of Applied Sciences Upper Austria deals with a combination of different scientific/technological areas like sensor system, physics, and machine vision. This R&D supports the local manufacturing industry. All of the faculty members have been working for the industry in advance they got faculty members. This relationship is kept alive after the formal act of closing the employee status at the companies. Additionally there are no big groups at the university within one desired technological branch. The number of faculty members is about 40. We meet daily and so the mathematician talks to the physicist or the control engineer on a frequent and informal level. So tasks of e.g. implementing a certain machine vision algorithm are discussed by many kinds of technology aspects. This sometimes leads to solutions which are not common practice. In the following there will be shown three interdisciplinary examples of R&D-projects: 4.1 Example 1 Active thermography The first of two examples shows an industrial application of non destructive tests of compound materials for the Airbus plane: By means of checking the propagation of temperature waves defects below the surface are detected [4]. The advantage compared to the actual standard by ultrasonic sensors is a reduction of measuring time by the factor 100. This technology now is additionally used for crack detection of metallic sow plates. A thermal wave is stimulated at the surface of the test object. This is done depending on the material using a dirac impulse (flash light), step function response (shuttered heat emitting source), or a periodic signal (induction at metals). An infrared camera takes a sequence of images. By means of the evaluated physical model, image capturing and processing the defects below the surface can be detected (figure 2). Figure 2: Detection of porosity defects left to right 5 %, 4 %, 3 %, 2%, 1 %; upper line active thermography, lower line ultrasonic. In comparison to the ultrasonic approach porosity defects can be detected more accurate and beyond a certain percentage limit. And this method is 100 times faster. 4.2 Example 2 Computed tomography classification Since 2004 the Campus Wels possesses an industrial CT (see figure 3) and has a long term project running with several objectives: Geometrical measurement of outer and inner geometry, comparison actual/nominal dimension Beam simulation for model estimation Porosity classification down to 4 µm Surface extraction e.g. for reverse engineering Supporting R&D of fuel cell tubes. 202

6 Figure 3: Dual energy computed tomography DECT at University of Applied Sciences Upper Austria, Campus Wels. There are about five permanent scientific assistants and several graduates. Machine vision is an essential topic. E.g. for creating surface models of multimaterial components using dual energy computed tomography DECT. An application scenario described at one of the works is metrology and dimensional measurement of multi-material components in industrial high resolution 3D X-Ray computed tomography (3DCT). The basis of this method is the dual source / dual exposure technology using the different X-Ray imaging modalities of a high precision micro-focus and a high energy macro-focus X-Ray source [5]. Another work deals with characterising, segmentation, and classifying porosity within compound materials (figure 4). microstructure [6]. Unless the method seems to be simple, up to know it has not been used for quantification of the so called Tiger stripe effect as shown in figure 5. Now this equipment is on to get the status as a company standard. Up to now the appearance has been checked visual and therefore there has been no standard for quantifying this effect. For the automation process of collecting data concerning material, injection speed, texture appearance it was necessary to have a system objective quantifying the effect. Tiger Stripes occur on injection moulded flat plastic parts. They have a fine 3D-structure. These parts are used to make car dashboards or bumpers. Under a certain angle of view the above mentioned Tiger Stripes are visible. The quality of such plates decreases caused by this texture. This texture can be avoided by modifying the injection speed of the plates. Figure 4: Porosity classification on compound materials. This project in total has a volume of 3.3 Mio and contains a lot of manufacturing companies throughout Austria and Europe. 4.3 Example 3 Tiger stripe quantification The third example implements a bright field surface detection to define the quality factor of injection moulded plastic plates regardless of their Figure 5: Tiger Stripes Evaluation and Quantification. The first step is to smooth the image; the second step is to find the Tiger Stripe by an adaptive (polygon of 6 th degree) horizontal threshold; the third step is to detect the periodicity and the amplitude of the remaining grey values and thereof to calculate the Tiger Stripe Value. This procedure has been tested by a given number of plates with different colours, injection speeds, and surface structures. The result compared to visible outputs is a reliable and accepted by the consortium to become the standard tests method for Tiger Stripes. 5 Conclusion Industrial Computer Vision fills the gap between the computer vision science and the industrial application level. Science and industry have different aims to fulfil: 203

7 There is the theory dealing with algorithms, modelling, physic, mathematic constraints and elegant solutions. Algorithms work because desired preconditions and disregards are taken into consideration. The world of theory is well described, readable, and logic. There is a map one can follow when he wants to get from point A to point B. Time is a physical dimension only. Within the manufacturing industry the constraints is money, and money. There is no time to really go behind certain effects. Just one has to find a work around and the solution is ready. One of the most important things on getting a system running is to get the sign of the contractor that the system is running. Engineers who will find their jobs in computer vision will have to bridge the gap. They have to understand the scientific map of theories and algorithms and they have to understand the needs of the process automation. They additionally have to check the possible distortions to the measuring field. of Applied Sciences Upper Austria, Austria, Proceedings Review of Progress in Quantitative NDE 2006, Hilton Portland & Executive Tower, Portland, USA, August 2006 [5] C. Heinzl, J. Kastner, E. Gröller, Surface Extraction from Multi-Material Components for Metrology using Dual Energy CT, IEEE Transactions on Visualization and Computer Graphics, Nov./Dec. 2007, Vol. 13, No. 6, pp [6] R. Gahleitner, K.S. Niel, S. Frank, Characterizing of Surface Defects on Plastic Specimens with an Optical Measurement System, future presentation at Image Processing: Machine Vision Applications, Proceedings of SPIE Vol. #6813, San Jose, California, USA, January Acknowledgements I acknowledge the impact of my prior and actual colleagues: Namely at vatron GmbH (Linz, Austria) to H. Aberl and F.X. Rubenzucker who gave me the idea of implementing fine optics within a harsh steel plant and sending me to the clients worldwide to implement our products. There I learned to properly finalise the project and to hand it over to the client for his satisfaction. At the University of Applied Sciences Upper Austria I mention G. Hendorfer and C. Derndorfer who gave me the chance to continue and develop the lecturers about machine vision for our students. Doing this job I learnt a lot about the theory of the algorithms I used intuitively at my first job. Thanks to J.R. Price and F. Meriaudeau who showed me the way to IS&T/SPIE. References [1] N. Bauer, Guidline for Industrial Image Processing, Fraunhofer Allianz Vision, 2003, ISBN [2] K.S. Niel, SC767 Practical Implementation of Machine Vision Systems Within Technical Processes, Short Course at IS&T/SPIE 19 th Annual Symposium Electronic Imaging 2007, San Jose, California, USA, February 2007 [3] K.S. Niel, Panel Discussion Machine Vision in Industry: Barriers and Pitfalls, Machine Vision Applications in Industrial Inspection XIII, Electronic Imaging 2005, San Jose, California, USA, January 2005 [4] G. Hendorfer, G. Mayr, G. Zauner, M. Haslhofer, R.Pree, A new method to produce high resolution porosity images by means of active thermography, University 204