Machine Vision Techniques Firstsight Vision Ltd,The Old Barn, Grange Court,Tongham, Surrey, GU10 1DW
1.0 Introduction In machine vision applications the importance of the illumination is almost always underestimated. Specialist illumination is sophisticated and there is science to understanding how and why a certain type of illumination is suited to a specific application. s do not see objects, they only see light reflected from objects. Machine vision illumination controls how the object appears to the camera. For example, light is reflected differently from a ball bearing than from a flat white label or a printed circuit board and therefore different lighting techniques are required. It may be argued that the illumination is the most critical part of an imaging system. s are far less versatile than the human eye and lighting conditions often need to be optimised for a camera to detect things that the human eye can see in uncontrolled conditions. This is particularly so when dealing with complex shapes or reflective components. 1.1 Spectral Luminous Efficiency & CCD Sensitivity The human eye is most sensitive to a wavelength of 555 nm. As the wavelength grows longer or shorter, the human eye becomes less sensitive. (Although there is some variation depending on the individual, most people can view a wavelength range of about 380 nm to 760 nm.) The relative sensitivity of the human eye to light is called the spectral luminous efficiency. 1.0 The CCS LDR 120B Ringlight Inadequate illumination can often make the difference between a system that works reliably and one that does not. If an application requires several software filters to be used before an image can be analysed correctly, always reconsider the illumination. Optimising the illumination can often eliminate the need for image filtering and give a far better overall solution. Reletive sensitivity 0.8 0.6 CCD 0.4 0.2 Human Eye 0 400 500 600 700 800 900 1000 Wavelength nm Rather than trying to design and develop custom (and often complex) illumination for an application, there are many benefits in using an off the shelf specialist unit: Cost Effectiveness - Saves time & money on research and design Proven - Thousands already in service Reliability - Long life, minimal service Spectral Luminous Efficiency Repeatability - High quality units available in quantity without 'in-house' manufacture The diagram above shows spectral luminous efficiency for both the human eye and for a typical CCD sensor. Spectral luminous efficiency is a measure based on a value of 1.0 for the spectral luminous efficiency of 555 nm light. Large Variety - Many types and techniques available with advice Wavelengths longer than the visible range are called infrared light and wavelengths shorter than the visible range are called ultraviolet light. The table clearly shows that a CCD s sensitivity to infrared light is far better than the human eye, and it is important to consider this difference in sensitivity when designing illumination systems for machine vision applications.this will be discussed in detail later. The CCS LDQ 200A Bar System 1
1.2 Considerations There are a number of considerations when determining which type of illumination is best for a particular job: Is it a monochrome or colour application? Is it a high speed or low speed application? What physical size (area) needs to be illuminated? What is the nature of the object - geometry, reflectivity? What is the feature of particular interest? What does the service life of the light need to be? What are the mechanical constraints/environmental considerations (background, physical size)? How these questions are answered will determine which light is best suited to an application. The following sections give a brief explanation of the technology involved in illumination. 1.3 Types There are three basic types of illumination commonly used for machine vision, fibre optic (halogen), fluorescent and LED. There are benefits and shortcomings with each type, which often means that particular applications are suited to particular types of illumination. Structured lighting, using lasers to illuminate the object, will be discussed later as a separate, special case. 1.4 Fibre Optic The basic idea of fibre optic illumination is to harness the intensity of a lightsource, through an optical fibre lightguide and into a light 'adaptor positioned close to the object. Light adaptors most commonly take the form of linelights and ring-lights of various sizes and lengths. The most significant losses in these systems occur at the light guide junctions, but there are also losses in the optical fibre guides themselves. Because there is increasing transmission loss per meter, it is not practical to use fibre optic lightguides in excess of 5m as these losses become excessive and the lightguides become very expensive. Basic halogen lightsources generally include a stabilised power supply and a housing for a halogen (150W) bulb which is focused onto the end of a fibre bundle plugged into the front. Alternatively, an ultra high powerd LED unit, such as the HLV LED light source from CCS, can be used, which has a similar intensity to a 100w halogen based light source. Intensity Intensity LEDs Life Cost/ Performance Response Speed Halogen Lamps Life Cost/ Performance Response Speed Fluorescent Lamps Degree of Design Freedom Heat Radiation Degree of Design Freedom Heat Radiation The diagrams to the left show that different illuminators have very different performance characteristics. Important considerations include: Cost Performance Output Intensity Unit Life Heat Dissipation Halogen has the highest output but scores poorly on life and design flexibility. LEDs are clearly the best all round solution. The most powerful lightsources available incorporate Nickel Metal Halide and mercury vapour bulbs. This type of lightsource can produce approximately 5 times greater light intensity than a 150W halogen. These units are expensive but provide the ultimate intensity for very high speed applications. 1.5 Fluorescent Fluorescent tubes are a common form of illumination for domestic purposes but have a limited use in machine vision, due to the limited variety of size and shape and the fact that they are AC devices that flicker and cannot be strobed. Intensity Cost/ Performance Life Response Speed Degree of Design Freedom Heat Radiation Fluorescent lamps are of limited use in machine vision environments, as thay suffer from inherent flicker and cannot be strobed or intensity controlled. Specialist fluorescent devices include ultra-violets, which are particularly interesting when dealing with fluorescent targets, such as those used in security printing. 2
1.6 LED For practically all machine vision applications, LEDs (Light Emitting Diodes) now provide the most appropriate solution.this is generally because LEDs now produce high light intensity at a relatively low cost and have a long service life typically up to 100,000 hours. Another consideration is that they only require DC power and have a tough, flexible cable with no restriction on length. Some modern LED lights focus the LEDs with lenses The reason that red LEDs are so popular is that they are the most widely used and therefore the cheapest LEDs available. Other colours use coatings such as phosphor to achieve the colour, making them more expensive. Most specialist illumination in the Firstsight Vision range utilises LED technology. The CCS HLV 3M RGB Tri-Colour Lightsource The CCS LN 200 Convergent Beam Linelight 1.6.1 Recent Developments in LED As with any manufactured item, LEDs vary significantly when produced. Most manufacturers of machine vision illumination use LEDs as they come. However, it is considerably more beneficial to batch grade them to maintain consistency within a specific unit. This becomes particularly important when dealing with the new generation of high intensity output LEDs.The diagram below shows the scatter in relative intensity for a typical batch of LEDs. CCS grade the LEDs into 6 bands thus providing very low intensity variation between LEDs used in a single unit. FAIL 6 Graded Bands 1.7 and Colour The colour of the illumination used with monochrome cameras is often thought to be unimportant and this is true, when the object is also monochrome. Problems can arise when the object is actually coloured and a monochrome image is required. 1.7.1 Colour Theories R B G R M Y B C G Subtractive (pigment) Additive (light) 50 60 70 80 90 100 LED Intensity (%) The light intensity from a constantly illuminated LED was generally not considered to be as bright as that from a halogen bulb, although modern 'high intensity' LEDs can produce similar levels of illumination, if overdriven in short bursts via a strobe controller. Compact LED lightsources are now available with a power output equivalent to a 100W halogen source. High power LEDs are now also commonly used in focused spotlights and linelights as well as more traditional diffused backlights and ringlights. LED illuminators generally use banks of LEDs mounted on a PCB and a separate diffuser, which covers and diffuses the output of the individual LEDs into a uniform area of light. Although red is the most common colour for LEDs they are actually available in a variety of different colours. Standard colours include red, green, blue, white and infra-red. three colour LED mixing can be used to create intermediate colours. LEDs are therefore appropriate for colour applications. RGB/CMY Colour Wheel It is important to understand that there are two different colour systems: Additive and Subtractive. Coloured pigments, (as in printed inks) are termed Subtractive. As the diagram above shows, when all 3 colours (RGB) are combined together they make a dark brownish colour, close to black. Light on the other hand is an Additive system and when the same three colours of light are mixed, white is produced. An object is seen to have a particular colour because it reflects that colour and absorbs all the others. For example, a red object absorbs green and blue light and reflects the red. White objects reflect all colours. 3
1.7.2 Colour Considerations The pictures below show images from a monochrome camera when illuminating different coloured objects with Red, Green and Blue light. As viewed with the human eye under white light Monochrome image under red illumination Monochrome image under green illumination Monochrome image under white illumination Monochrome image under blue illumination Using ultraviolet illumination in conjunction with UV sensitive camera/sensor technology allows for ultra-fine resolution in applications where features that are sub-micron need to be resolved. UV light is invisible to the human eye (and most CCD cameras) and lies slightly below the violet end of the visible spectrum. UV rays have shorter wavelengths than visible light. 1.7.3 Infra-Red Infra-red (IR) illumination can be used to diminish colour effects that can pollute monochrome images. The example shown here with childrens coloured crayons shows the effect quite dramatically. The labels are all different colours and show up under normal lighting conditions as very different grey values. If the desired image output requires text to be read in good contrast, then using infra-red illumination can increase the contrast while reducing the colour 'pollution' in the image. When using colour cameras, similar rules apply. It is important that the illumination has all colours (RGB) and they are present in similar amounts otherwise the camera will see different colours at different intensities, upsetting the colour balance. The hue of the light is often referred to as the colour temperature. When referring to light, a wavelength is defined as the distance between the crests of two waves and is often measured in units called nanometers. A nanometer is a billionth of a meter, or about 1/25,000,000 inch. White light Infra-Red The graph shows different colour spectra for 3 different halogen lamps. EKE is a standard bulb, HC is a high colour temperature bulb and ER is an extended range bulb. 1.7.2 Ultra Violet Standard techniques using visible light make it possible to recognize and resolve features down to approx. 0.5µm (500nm). However, to visualise smaller features, the visible light spectrum is no longer sufficient. (The sole use of visible light is also insufficient to comply with the Nyquist sampling criterion, which states that an original signal can be recovered without any loss if the samples are no larger than half the size of the finest detail in the signal).to resolve smaller structures, the reduction of the wavelength is the most practical and effective solution. Infra-red illumination can also be used as a way of reducing the effects of ambient illumination, especially since there are several enhanced IR sensitive cameras available. By using IR illumination with (infra-red) band-pass filtering for the camera lens, variations in ambient lighting are significantly reduced. Common indoor ambient light sources tend to have a low IR content, therefore using IR illumination and filtering out other wavelengths can result in a very stable lighting system. 1.7.4 Flicker A fundamental of machine vision illumination, is that the light needs to appear flicker-free to the camera. As the camera section will discuss in more detail later, camera images are taken at discrete instances in time whether the camera is 'free running' or triggered. The illumination must either be synchronised to this (in the case of strobe illumination) or be constant (flicker-free). Standard AC illumination is not suitable, therfore all illuminators need to be either stabilised DC (as is the case with LEDs) or high frequency AC. 4
1.8 Techniques There are several standard illumination techniques used to emphasise particular features of any object. This is not to be confused with the type (fibre optic, LED, etc). technique considerations include; whether the object is dull or reflective, flat or complex in shape; if 'through holes' are to be detected or the object is opaque; if it is surface defects which are of interest, and so on. Basic techniques are briefly explained in the rest of this section and summarised into a selection guide at the beginning of the Products Section. 1.8.1 Direct and Diffuse Light Before discussing the details of the different illumination techniques, it is important to understand the terms Direct and Diffuse. These affect the quality of light that is produced. Direct light has an uninterupted path between it s source (such as LEDs) and the target. Diffuse light has passed through an opaque diffuser which softens and disperses the light making it less intense but more uniform. Direct LED Work Diffusing material 1.8.2 Bright-field This is the most common technique for illuminating diffuse nonreflective objects. The term bright-field refers to the mounting position of the illuminator. If a camera is positioned pointing towards a plain mirror, the bright-field is the area in which any reflected light is within the FOV (Field of view) of the camera. Please refer to the diagram (below, left). Most ring-lights, spotlights and linelights would be used in the bright field. The CCS LDR-120B 1.8.3 Dark-field This is a technique used to highlight surface defects, scratches or engraving and is particularly useful when inspecting reflective objects. Usually in the form of a large ringlight with a low angle, this technique is also totally dependant on the mounting position of the illuminator. The dark-field is the opposite to bright-field, that is the area outside the FOV of the camera when viewing a plain mirror. Please refer to the diagram (below, left). Large ringlights, spotlights and linelights may be used in the dark field. The CCS LDR-176LA-1 The image below shows different lighting techniques diagramatically. There are discrete areas or positions around the target that define the technique being used. Dark field Hot Spot Bright field On-Axis Uniform Intensity Dark field This sequence of images shows the principle of dark field illumination when detecting surface scratches or defects on a flat object such as a coin. As the first diagram shows, on a flat surface there is a low angle of incidence, and most of the light is reflected out of the field of view of the camera. The second diagram shows that only where there is a surface feature does the light get scattered, some of it then entering the camera, thus revealing the scratch or deformation. Light Path Light Path Target Target Back light The last diagram is a close-up ray diagram of the light hitting an edge. 5
Diffuser Light Source 1.8.4 Back light A technique of positioning diffuse illumination behind the object to give a silhouette. This technique is normally used if 'through holes' are of interest or the object is generally opaque with darker/brighter variations in the areas of interest. This is a very useful technique and requires no space in front of the object. View of reflective component using ring-light. (see dark reflection of lens) Same component using on-axis illumination. (note the even illumination.) Diagram of a typical set-up for back light illumination Backlight 1.8.6 Advanced On-Axis Advanced on-axis illumination offers an improved performance with the addition of a reflection chamber and no direct path between the light source and the object. These illuminators are suited to illuminating uneven surfaces that have more profile such as wrinkled paper, threads of screws, PCBs with solder components, moulded plastics etc. 1.8.5 On-Axis (or Co-Axial) This is a special technique used for illuminating reflective objects. A beam splitter is used to reduce the reflection of the camera lens or the illuminator itself in the resulting image. This is a fairly complex technique and there are several variations on the same theme depending on the shape of the object. The images (left) show the fill level of a bottle being checked. The first image taken under normal domestic lighting. The second image was obtained using a high intensity backlight to create a clear image of the liquid surface which shows up as a dark line that is then very easily detected by the camera. Half Mirror Ray diagram of on-axis illumination Half Mirror Diffuser Ray diagram of advanced on-axis illumination Light Source The NER SCDI 75 The example below shows printed foil packaging which is highly reflective and has a significant profile (ie. it is not flat).the advanced on axis technique provides very even illumination with minimal highlights. The technique of on-axis illumination is ideally suited to plain reflective, mirror-like objects with no (or very little) profile or any surfaces which have diffuse backgrounds. Examples include PCB fiducials, reflective labels, print inspection, surface flaws, polished silicon wafers etc. The NER DOAL 50 Print on foil packaging using standard ring-light illumination Print on foil packaging using the advanced on-axis technique 6
Light Source 1.8.7 CDI or Cloudy Day Illuminator Dome (Patented by Northeast Robotics) The Continuous Diffuse technique offers the highest level illumination performance available. It has been designed for the most difficult applications and combines off-axis spherical reflected light and on-axis light to provide light incident from all angles. This illuminator is sometimes refered to as 'Cloudy Day' illumination as it produces no shadows. It can be used to illuminate the most complex specular shapes including ball bearings, surgical instruments, knurled foil, compact discs etc. The images below show how an object with a very complex shape can be evenly illuminated with the CDI. View of aluminium drinks can using standard ring light View of aluminium drinks can using CDI 1.8.8 Collimated A ball bearing is one the most difficult objects to illuminate. This sequence of images shows the resulting image under various lighting conditions including the CDI. As can be seen from the examples shown here, the CDI is the only type off illuminator that can light the surface if the ball bearing without in some way exposing the illuminator itself. A ball bearing, in normal lighting view with fibre optic ring light view with fluorescent ring light view with diffused dome light Light emitted from any locally positioned source propogates in a radial fashion, and dispurses as it gets further from the source. Light from a distant source such as the sun (considered to be from an infinite distance) strikes any surface uniformly. The rays are parallel or collimated. Using light from a collimated light source is useful for detecting shallow flaws and dents in flat, reflective objects. Sunlight Light from a light bulb The images show the reflections of the light lights quite clearly. In normal lighting conditions all that can be seen is the reflection of the cmaera and the observer, and with the on-axis light the internal arrangment are quite obvious. The diffuse dome illuminator is better but shows the unilluminated hole left at the top of the dome. view as lit with an on-axis light view as lit with an advanced onaxis view as lit by CDI 45 o Beam Splitter Collimating lens Collimated light beam Ray diagram of a collimated on-axis illuminator The NER CDI Illuminator The CCS MSU Illuminator The example below shows a shalow dent in a battery being made clearly visible with the use of a collimated illuminator. Ray diagram of the CDI illuminator A battery with shallow dents under standard on-axis illumination The same battery as imaged under collimated illumination 7
To obtain the most accurate 3D information, very narrow and sharp illuminated lines are required with minimal background illumination. This is usually achieved by using laser technology to produce accurate lines together with a narrow pass filter on the camera, only transmitting the reflected laser light and no ambient light. Diagram showing deflection of light rays due to a dent LASER stands for: Light Amplification through the Stimulated Emission of Radiation Structure of a laser diode As the object passes under the laser, a 3D image can be generated from a series of slices as shown below. Light from a collimated source will be reflected directly back into the sensor, except where there is a depression. This will appear as a darker patch where the light is deflected away. 1.8.9 Structured Lighting The Automated Imaging Association (AIA) defines structured light as the process of illuminating an object (from a known angle) with a specific light pattern.observing the lateral position and distortion of the image can be useful in determining depth information. Structured lighting is used in many applications to obtain depth perception and for 3D inspection. In its simplest form, a line of light is generated and viewed obliquely. The distortions in the line can then be translated into height variations. Illuminating an object with structured light and looking at the way the light structure is changed by the object, gives us information about the three dimensional shape of the object. This image shows a laser being used to profile the surface of an object using a series of laser lines Structured lighting is also widely used in allignment applications where pointers are required for alignment for cutting material or for aiming purposes. Laser Fanned laser light This diagram shows the general arrangment of a laser used for a 3D profiling application Uses for this technique include: inspection of food items to check for correct volume or product deformations and foreign objects, amd also checking the surface profile of objects against known good examples etc. Output Lasers can be used to assist with allignment and cutting and also in several medical applications 8
1.9 Accessories In a machine vision environment, there will often be times when using standard illumination products cannot quite achieve the desired results. In these situations, there are a number of filters and other accessories that allow the user to fine-tune the lighting to meet certain requirements. The products featured below represent the most common illumination accessories, but the list is not exhaustive. Please call to discuss any specific requirements you may have. 1.9.4 Diffusers Diffusers allow direct LED lights to be converted to diffuse, indirect units. These are available in a range of different formats to suit most illuminators. 1.9.1 Polarising Filters These work with a polarising plate to reduce illumination glare. Example Product image as viewed without using a polarising filter. The same product as viewed when using a polarising filter. 1.9.5 Lens Attachment Rings These are used for attaching lights directly onto the camera lens, thus removing the need for any external brackets or clamps. 1.9.2 Sharp Cut Filters These cut out wavelengths of 60nm and less to prevent external stray light. Available for standard C-Mount threads: M16, M25. 1.9.6 Light Adapter Ring These are used to attach diffusion plates and polarising filters to different CCS lights. Example 1.9.7 Flexible Arm This is a very novel and flexible attachment system compatible with all the CCS range of lights. 1.9.3 Light Control Film These are a cost effective way of producing pseudo collimated illumination by only passing light with a defined angle. Example 9