Optical Systems. in Image Processing

Transcription

1 Optical Systems in Image Processing

2

3 Introduction When introducing an image processing system, the performance and functions of the system or the board are naturally important, but selection and setup of the optical system are also important considerations. Before considering the introduction of an advanced grayscale image processing system that can perform high-speed pre-processing to improve image quality, the optical system that produces the work images should be reconsidered. It may be possible to improve image quality by simply changing the lighting equipment, lenses or using optical filters so that even an inexpensive binary imaging system will produce satisfactory results. Also, pre-processing for image quality improvement involves alteration of raw data, or, in other words a loss of raw data. In that sense, there is some risk associated with pre-processing. The processing is simpler if the raw data from the acquired image can be used for inspection, recognition and measurement. Doing so makes the processing that much faster, requiring less programming work and allows the use of less expensive equipment. This document explains the very basic elements of the optical system (lighting, lenses, and filters), but it by no means provides a detailed description. It simply offers hint-like information for obtaining good images. For a detailed explanation and the latest information, we recommend that you refer to the documentation provided by the manufacturers of the equipment. 1

4 About Lighting An image that is beautiful to the human eye is not necessarily an image that is suitable for image processing. It is important to produce an image that suits the purposes of the image processing. Lighting plays an important role in doing that. Here we provide a simple explanation of illumination for image processing. Things to Consider When considering illumination for image processing, the following points are important: Appropriate Brightness Of course, it is necessary to provide sufficient light for the sensitivity of the camera s CCD, but simple brightness alone is not enough. Clear contrast between the object image to be processed and the background are also important. Screening Unwanted Light Sunlight or other light that varies in brightness over time is undesirable. You must filter out all unwanted light if possible, or at least adjust the amount of provided illumination to reduce the effects of unwanted light. Stable Light Source Light source placement and strength that varies with time and circumstances are also undesirable. Note that the luminance of fluorescent lighting in particular changes considerably with temperature. Lighting Suited to the Purpose What do you want to accomplish with image processing? The lighting you use should ensure that the information needed for your purposes remains in the image. For example, if your objective is to find scratches, you likely will not mind if the scratches appear clearly as white, while the rest of the image is black. Make the Lighting as Uniform as Possible As much as possible, eliminate differences in lightness and darkness (shading) and reflections that produce local white areas (halation). 2

5 Types of Light Sources Light sources differ mainly in light wavelength and brightness. Halogen Lamps The brightness and color temperature of these lamps change very little during use. That feature makes this type one of the most often used light sources for image processing. They are compact and nearly white in color, highly efficient, have a high color temperature, and are durable. They are often used with fiber guides, thus providing cool illumination. This feature is also suitable for image processing. LED A light source often used on semiconductor fabrication lines, etc. LEDs are said to have a longer lifetime than halogen lamps, and a cooling fan for the light source is not needed because the emitted heat is low. Generally, the luminance is low and mostly reddish in color, but high luminance LEDs that emit white or blue light have been developed in recent years. Fluorescent Lights Fluorescent lighting was introduced to the world with the catch phrase, shadow-free illumination. In other words, it is diffuse illumination. These inexpensive lamps come in ring shapes and in various wattages, but a disadvantage is that their luminance changes considerably with temperature. Also, the lamps are lit by the cycling of an alternating current power source, so lamps of high frequency (10 KHz or higher) are generally used in image processing. Power (%) Optical Output Power vs. Temperature (reference) Ambient temperature ( C) Tungsten Lamps These are commonly called light bulbs. They are inexpensive and come in numerous shapes and wattages. When operated on alternating currents, there is some fluctuation in illumination, so they are generally used with a stabilized DC power source for image processing. Metal Halide These have lower luminance than halogen lamps, but their color is almost the same as sunlight. Xenon Lamps These have higher luminance than halogen lamps and are often used as strobe light sources in image processing. 3

6 This is illumination with ultraviolet light. Ultraviolet light activates light-sensitive materials, but there are also metals that have high-reflectivity, and other metals that have low reflectivity for ultraviolet wavelengths. Those properties can be used to create image contrasts. Illumination Methods Illumination methods are broadly divided into two types: transmitted illumination and reflected illumination. A combination of these two methods is called compound illumination. 1. Transmitted Illumination This is also known as back lighting. Usually a diffusion plate (a milk-colored plastic plate, for example), is used to create surface illumination. The illumination comes from behind the subject and produces a silhouette for clean imaging of the outline of the piece. If the piece is translucent, the internal structure (the color of a liquid crystal color filter, etc.) can be seen. Transmitted illumination is suitable for binary image processing and facilitates outline shape inspection and dimension measurement, etc. Camera 2. Reflected Illumination In this method, light strikes the subject of the image and is reflected to the imaging device. There are various types that differ in direction of illumination and the shape of the light emitter. Incident light Reflected light (1) Coaxial Illumination This is also called bright-field illumination. The illumination comes from the direction of the imaging plane Half of the camera, which is accomplished by placing a mirror half-mirror between the camera and the subject. This technique is effective for observing protrusions and depressions on silicon wafers, LCD panels and other highly reflective subjects. Camera Subject Light source 4

7 (2) Oblique Illumination This is also called dark-field illumination. The light strikes the subject from an angle. Ideal pieces that have low reflectivity, and for detecting protrusions and depressions on the surface of work Light source Camera Light source (including highly reflective subjects). Various kinds of light emitting surface shapes are used, including ring lighting, line lighting, and fiber lighting. This is the most general of illumination methods. Subject 5

8 Ring Illumination The light-emitting surface is annular, so light is emitted from all angles. (The photograph to the right shows an example from NPI.) Line Illumination The light-emitting surface has a linear shape, which is suitable for use with line sensor cameras. (The photograph to the right shows an example from NPI.) Fiber Illumination This is also called straight illumination or spot illumination. Fiber illumination is also used as a coaxial light guide. (The photograph to the right shows an example from NPI.) (3) Specular Reflection Illumination If the work has a mirrored surface, the angle of incidence and the angle of reflection are the same in this type off illumination. This is called specular reflection. It is used for imaging surface texture. Light source Camera (4) Diffuse Illumination This is also called uniform illumination. A simple way to achieve diffuse illumination is to use fluorescent light at a distance, but that is not suitable for introduction to a fabrication line. A practical method is to attach a diffusion adapter to a ring lamp to diffuse reflected light. A well-known way to effect shadow-less illumination is to use a dome light. It is suitable for observing spherical or R shaped work, and soldered parts, etc. Camera Light source Diffusion reflector Camera Diffusion reflector Diffusion plate Diffusion plate Subject 6

9 Specific examples of diffuse lighting (the photographs and diagrams below are examples from Moritex) 7

10 (5) Slit Illumination The light is emitted as a straight line and used in the light plane intersection technique (3D measurement). It is implemented by passing light from a condensing lens through a slit or by using a laser. Camera Slit projector 3-D object 3. Compound Illumination Simple illumination is sometimes not sufficient for good imaging, so it may be necessary to combine illumination methods according to the purpose. For example, transmission and reflection can be used together to image both the outline and the surfaces of the work peace. Camera Light source (for reflection) Light source (for transmission) Accessories Condensing Lens With fiber illumination, the light emitting part is just a point from which the light itself spreads out. The spreading can be prevented with a condensing lens. The commonly known condensing lens produces a round or rectangular light spot. 8

11 Condensing lens luminance characteristics and illumination ranges (the diagrams below are examples from Moritex) 9

12 Subjects (work pieces) and Appropriate Illumination (From documents provided by Moritex) 10

13 Reference Terminology Term Notation Explanation Luminous flux m (lumens) The amount of light radiated by the lamp Luminous The intensity of the light (the amount of light radiated within a cd (candela) intensity unit solid angle in a given direction) Luminance The amount of light per unit area (luminous intensity); the (lux) brightness of the area illuminated; specified in the JIS as the standard unit of illumination Luminous flux[lm] The brightness of an object viewed from a given direction (light Brightness cd/m 2,nt (candela/m2) (Nit) intensity from a unit normal projection area in a given direction); as opposed to luminance, which represents how much light reaches a unit area, brightness represents how bright that illuminated area appears from a given direction. Color temperature Coloring Color rendering index Rated power consumption Rated lamp power (Lamp) Efficiency Lifetime Rated lifetime Luminance intensity distribution curve Beam angle (beam spread) k (Kelvin) Ra W (watt) m/w (lumen-watt) h (hour) h (hour) The characteristic that determines the color of light emitted by a lamp; a high color temperature is blue-white and a low color temperature is tinged with red. How the light from the lamp renders the actual colors of the objects illuminated An index of how faithfully colors are reproduced. Values closer to 100 indicate a more natural color rendering, and value above 80% is considered to be satisfactory in practice. (JIS) An indication of how much power the lamp consumes; displayed on the lamp and published in catalogs. The total luminous flux generated by the lamp divided by the lamp s power consumption The average time from when the lamp is first turned on until the lamp will no longer light or until the brightness decreases to 70% (60% for some types) of the original value (after 100 hours of operation) whichever comes first The average lifetime measured under prescribed test conditions and wattage; the lifetime published in catalogs etc. Indicates the intensity of the light output by light source in different directions A representation of the degree of light condensing as the angle between the two points to the left and right of the central light intensity (maximum intensity) where the intensity is ½ (or 1/10) the central value Beam luminous flux m (lumen) The luminous flux within the beam angle 50% luminance 100% luminance Luminous [cd] Beam 0 Brightness [cd/m 2 ] Luminance [x] 90 Direction in which luminance is 1/2 Distribution curve Central intensity (maximum intensity) Optical Spectrum When natural light is passed through a prism and projected onto a screen, it separates into colors of increasing wavelength from violet to blue, blue-green, yellow, orange, reddish orange, and red, etc. in a banded pattern like a rainbow. This banded pattern is called the visible light spectrum and it represents Electromagnetic wavelength and visible light spectrum nm 0.001nm 10nm 20nm 200nm 380nm 780nm 0.002nm 0.22nm 1nm 100km Sound Radio waves Far infrared Near infrared Visible Near infrared Far X-rays Gamma rays Cosmic rays Violet Near ultraviolet Blue Blue-green Green Yellow-green Yellow 380nm Orange-red Red Ne 9 * nm is the unit of wavelength, read as nanometer.1 nm = 10 m 11

14 the components of visible light. About Lens Lens Selection The visual field angle, depth of field, brightness and other factors vary with the lens you select. When selecting a lens system, consider the size of the work piece, resolution, installation conditions and other such factors. Visual Field Size This is not simply a problem of whether the work piece is imaged as large or small; consideration must be given to resolution, as accuracy depends on the physical dimensions to which one pixel corresponds (one millimeter or one micrometer, for example). Deciding on the visual field size is the same as deciding on the resolution. Physical Constraints The lens and the camera on which it is mounted are one component of the system configuration, so interference with peripheral equipment or the illumination system must be considered. Other constrains arise, such as the need to use microscope lens systems for extremely small work pieces. Depth of Field A short depth of field is good for preventing interference from a background that is at a different distance than the subject; when that is not the case, a longer depth of field is probably better. You must also consider the effect that depth of field has on the range of focus. Lens Performance In addition to the focal length, depth of field, f-number and other indexes of a basic lenses performance, aberration and ghosting must be considered. The types of aberration include chromatic aberration and the five types of non-chromatic aberration identified by Seidel (distortion aberration, field curvature, astigmatic aberration, coma aberration, and spherical aberration). However, these aberrations vary with the distance to the subject and the lens aperture, and they may not be specified in the catalog, so their use in lens selection is limited. Having said that, if you use a reasonably priced lens produced by a major lens manufacturer, you can expect these factors to be of little consequence. That is because the resolution of the camera CCD is lower than the resolution of the lens system. It is always safest to avoid cheap or dubious lens systems. If a lens system is explicitly labeled for use with monitoring or for image processing, select the product that is specifically for use in image processing. 12

15 Types of Lens CCTV lens This is the most generally used lens for TV. The Asahi Precision Cosmicar lens is among the most well-known systems of this type. They are relatively inexpensive and come in many varieties. Macro Lens This type of lens is designed for close up imaging. Ordinary lenses are designed with aberration compensation for infinite distances with the assumption that the subject will be relatively far from the lens. A macro lens system, however, is designed for imaging objects that are a short distance from the lens. Zoom Lens These lenses have a variable focal length. It is easy to change the magnification and visual field size, but the resolution and aberration are considered to be generally poor. Macro-scope Lens These are variously referred to as microscope lenses, macro-scope lenses, mirrored tube lenses (because they are tube shaped and contain a mirror). These lenses are used mostly for imaging very small work pieces. Tele-centric Lens In this lens, the chief rays are parallel to the optical axis, so the image size does not change even if the object moves in the direction of the optical axis. It is suitable for imaging multiple objects that are at different distances from the plane of the CCD, when imaging multiple work objects of the same shape, or multiple bar-shaped work pieces at the same time. (See the ray diagram on the next page or the effect diagram on the page after the next.) 13

16 Tele-centric Optical System (from Moritex documents) Ordinary lens Object tele-centric Dual tele-centric CCD CCD CCD Aperture Chief rays Chief rays Work Piece Work Piece Work Piece Merit Merit Merit Compact Fewer lenses, thus lower cost Work piece size or position does not change with distance from the lens. Compact when used with coaxial lighting. Same as at the left, but higher accuracy when the variance in camera flange-back dimensions are high. Demerit Work piece size or position changes with distance from the lens. Demerit Larger than ordinary lens when coaxial lighting is not used. Demerit Same as at left, but also more expensive. 14

17 Effect of atele-centric lens (from Melles Griot documents) Ordinary Lens Tele-centric Lens Ordinary Lens Tele-centric Lens 15

18 Basic Knowledge about Lenses Lens Mounts The lens mount is where the lens is attached to the camera body. Industrial TV cameras other than ultra-compact cameras and other special cameras use the C mount as the standard mount. CCTV lenses and the lens marketed for image processing also use the C mount, so there is generally no problem with lens mounting. If you want to use a lens designed for single lens reflex cameras, you could install a C mount adapter between the lens and the camera, but the large size of the lens itself can be a problem. Focal Length This is the distance from the lens to its point of focus. A lens system consists of multiple lenses and has two optical points, which are called the first and second optical points. The focal length is the distance from the second optical point to the point of focus. A CCTV lens is designed with the surface of the CCD as the focal point. Second optical point First optical point Focal length Point of focus (CCD plane) Angle of View When you replace a lens with one of a different focal length, the size of the image and the width of field that is imaged will also change. The width of field expressed as an angle is called the angle of view (diagonal angle of view). This is the same angle as seen from the second optical point to the plane of the CCD, and is denoted in the catalog as the 1/2 included angle, where the 1/2 refers to a 1/2-inch CCD. Image sensor 1 Diagonal viewing 2/3 angle Second optical point Image circle Focal plane (image sensor surface) Image sensor Focal length 16

19 Width of Field This expresses how much is contained within the field of view. It varies with the focal length of the lens. Generally, it is given as w W h f = =. H L W: Subject width w: CCD width H: Subject height h: CCD height f : Focal distance L: Distance to the subject W w L f For example, if a 50-cm high object that is two meters away from a 2/3-inch CCD camera is to fit into the screen of a monitor TV, we can calculate f =26.4 mm from the equation given above. There is no lens that has a focal length of exactly 26.4 mm, but we can select a 25 mm lens, which is the closest to that value. h f 6.6 f = = H L 500 2, 000 F Number The F-number indicates the brightness of the lens. It is determined by the effective aperture of the lens (the diameter of the luminous flux that enters the lens without being blocked by the lens frame) and the focal length of the lens. It is generally expressed as F=f/D (f is the focal length and D is the effective aperture). Larger F values mean a darker lens, and conversely, smaller values mean a brighter lens. A point of caution is the problem of the amount of ambient light. The brightness indicated by the F-number is not guaranteed over the entire screen. The wider the lens angle (shorter focal length) and the more open the lens is (iris fully open to give the lowest F number), the darker are the edges of the screen relative to the center. This may result in the margins of the screen being dark. 17

20 Subject Luminance and Image Luminance If you know the brightness of the subject, you can better select which lens to use. The luminance of the image surface, Ec, is obtained in the following way: T R Ec = E F ( m + 1) Magnification m is V/L Subject Image T: Lens transmittance Approx. 0.8 R: Subject reflectivity Approx. 0.2, depending on the subject F: F number M: Magnification Es : Subject luminance Usually, a b, so L m = f and f m =. a a V m = = L b a b v Depth of Field When the subject is in focus and the image is sharp, it is possible to adjust the focus back and forth to some degree. The range in which the focus can be changed while the image remains sharp is called the depth of field. The depth of field is obtained in the following way. f f H = C F B( H + f ) T1 = H + B B( H f ) T 2 = H B H: Past the focal length (the distance at which the far end of the depth of field is infinity) f : Focal length B: Subject distance (distance from the subject to the CCD) T1: First optical point T2: Second optical point C: Minimum distortion (aberration) Generally, you can obtain a larger depth of field by 1) using a lens that has a shorter focal length, 2) using a large F-number (smaller effective aperture, reduced), and 3) increasing the distance to the subject. 18

21 Accessories for close-up imaging Close-Up Ring This determines the range in which the lens focus can be adjusted. If you want to image at very close distances, one method is to insert an extension tube between the lens and the camera to change the focal length. Such a tube is called a close-up ring. Rings of various lengths are available on the market. Parallel light CCD Non-parallel light Width of close-up ring Close-Up Lens This is a type of lens that can be installed in advance to allow close-up imaging. It works by converting the light rays from the subject to parallel rays. Close-up lens focal length 19

22 About Optical Parts (accessories) Optical Filters When the reflectivity of the subject and the background are about the same, obtaining contrast (difference between light and dark) is a problem with a monochrome camera. In such cases, however, if the subject and background differ in color, an optical filter can be used to enhance the contrast. Care is required, however, because the illumination, the optical filter and the spectral sensitivity of the camera are all inter-related. ND Filter An ND (neutral density) filter adjusts the amount of light. It is used when the lens aperture cannot be reduced. They are generally sold on the market with transmittance values between 25% and 70%. The wavelength characteristics in the visible range are flat. PL Filter The PL filter is a polarizing filter. Polarizing filters transmit only light of a particular polarization plane. The lens is attached to a mechanism that allows the filter to be rotated to change the direction of the polarization plane. A polarizing filter is effective for reducing the glare from reflective surfaces such as metal or glass. These filters block some of the light, and in doing so, darken the image. It is effective to attach this type of filter on both the illumination side and on the lens side, as illustrated in the figure below. Polarizing filters Camera Light source Camera filter direction Subject Light-source filter direction Infrared filters CCDs are highly sensitive to infrared light, so an infrared blocking filter is usually placed in front of the CCD. Nevertheless, flames and other high temperature subjects emit enough light energy to create a smear effect (similar to the trail of light seen behind a fireball), even though it is not that bright to the human eye. In such cases, improvement can be achieved by using an infrared blocking filter. 20

23 Color Filters These are glass filters that are colored red, yellow, blue or other colors. Generally speaking, subject colors that are the same as the filter color are brightened, and opposite colors are darkened. This property can be used to enhance image contrast. The following points are also useful to know: A red filter tends to darken the blue and green areas, and is effective for black and white subjects as well. A blue filter intensifies red and yellow areas, and brightens blue and green areas. A green filter intensifies blue and red areas, and brightens yellow and green areas. Band-Pass Filters Multiple layers of thin film are deposited on an optical glass substrate to form a single-color filter that transmits only light of a particular wavelength band. For example, Kenko offers 41 standard types ranging from 400 nm to 800 nm in 10 nm increments. To be sure of what type will be effective, it will probably be necessary to do a spectral analysis of the subject and background before purchasing a band-pass filter. 21

24 Mirrors Mirrors are convenient for observing areas that are not otherwise visible, and to change the angle of the optical axis. For example, a half-mirror can be used to implement the coaxial illumination described in the section on About Illumination. Cone mirrors can be used to view the inside surface of tubes (figure below). Camera I L R (R) Cone mirror 90 L L 2 L I L 2R (L) R Prisms Like mirrors, prisms allow indirect viewing of the work and indirect illumination. They are also effective for making an optical system more compact. (See the examples on the next page.) 22

25 Prism Adapters (from Moritex documents) Side View Type (mirror image) These can be used to bend the optical axis by 90, to save space or to view objects from the side. The image is reversed as in a mirror image. Lens CCD camera Optical Axis Shifting Type This can allow two CCD cameras to be used side-by-side without spatial interference when doing a minimum two-point alignment. CCD cameras Lens Side View Type (without image reversal) This can bend the optical axis by 90, to save space or to view objects from the side. Lens CCD camera 23

26 About Camera Characteristics The photosensitive imaging element of the camera (CCD) is intimately related to the optical system. Here we describe the characteristics of cameras (industrial monochrome TV cameras) that relate to the optical elements. CCD Size The camera manufacturers catalogs use various terms regarding size, including screen and imaging sizes, optical sizes, and image format sizes. Here, we use CCD size. The size is usually expressed in inches, and the types illustrated below are available (dimensions are in millimeters). 1 2/3 1/2 1/ A lens can be used with an imaging element that is smaller than the size the lens was designed for, but it cannot be used with a larger element. For example, a lens for use with a 1/2 imaging element can be used with a camera that has a 1/2 or 1/3 element, but it cannot be used with a 2/3 or 1 element camera. The reason is not that the lens will not fit, but that the corners of the image will be lost when the lens is used with an imaging element that is too large. Also note that the image size will differ when the same lens is mounted on a 1/2 imaging element camera and a 1/3 imaging element camera. Using the lens with the camera that has the smaller imaging element produces a smaller visual field, so the subject will appear that much larger with it. 24

27 Spectral Sensitivity CCDs do not have uniform sensitivity to color (light wavelength). CCDs are particularly sensitive to infrared light, so most cameras use an infrared light (IR) blocking filter in front of the CCD. Spectral sensitivity Relative sensitivity Without IR blocking filter Gamma (γ) Characteristic Wavelength (nm) The output signal level that corresponds to the light that reaches the CCD is called the Gamma Characteristic. The output signal level V that corresponds to the brightness of the incident light (optical power) P is approximated by the following formula.? V = kp ( k = cons tan t) Signal level (V) γ=1 γ=0.7 Brightness(optical power) Thisγvalue is somewhat nonlinear, and forγ= 1, the incident light and output signal level are proportional. For television camera monitoring, a gamma value of aboutγ=0.7 is suitable. (See the figure on the right.) For image processing, aγ=1 is generally better because it allows the light intensity to be read out directly. Most cameras allow selection of the gamma value, so select the value as needed. AGC AGC (Automatic Gain Control) is an internal feedback circuit in the camera that adjusts gain automatically. When the subject luminance changes, this mechanism automatically adjusts the gain to within a certain range, thus maintaining a certain output video signal level. However, because this function does not reflect the true luminance in the video signal, it is probably not suitable for use in tasks such as image measurement or image inspection. Nearly all cameras allow the AGC to be turned on or off, so that you can choose to use or not use it depending to your purposes. 25

28 Smear When a high-luminance spot of light strikes the surface of a CCD, a bright band called smear may appear on the display screen. This phenomenon appears as a vertical band-like tail and occurs when there is an overflow of charge accumulation in the CCD. Nearly all cameras have measures for suppressing smear, but in image processing that must deal with bright spots, care must be taken in adjusting the lens aperture. Blooming Like smear, blooming may be caused by high luminance incident light. It occurs when a charge overflows to the surrounding imaging elements in the CCD. Cameras also have countermeasures for this phenomenon. Moiré When a camera images subjects that consist of very small points, interference fringes called moiré may appear on the display screen. White Clipping In this phenomenon, bright spots of light striking the imaging element degrade the contrast in surrounding dark areas, making them difficult to observe. White clipping is a circuit function for reducing that effect by cutting off the image signal above a certain level. This function facilitates viewing for simple monitoring purposes. However, linearity with respect to the incident light is lost, so, for image processing, it is better to operate the camera within the range where the clipping circuit does not operate. Charge Accumulation Mode There are two types of charge accumulation (the electrical charge that accumulates in correspondence to the light and dark areas of the subject detected by the CCD): frame accumulation and field accumulation. Frame accumulation occurs in 1/30 th of a second, and field accumulation occurs in 1/60 th of a second. From these facts we can see that frame accumulation takes longer, so when used with light sources that turn on and off at the frequency of alternating current, such as ordinary fluorescent light, frame accumulation will produce less screen flicker. Shutter The shutter is very important when imaging moving objects, but faster shutter speeds mean shorter exposure time and thus darker screen images. In other words, fast shutter speeds require brighter illumination and wider iris settings of the lens. The lens iris setting affects the depth of field, so care is needed in that respect. 26

29 90X Series About Optical Systems August 2008, First Edition, First Printing Published by Fast Corporation , Shimotsuruma, Yamato City, Kanagawa, Japan Technical Support: FAX TEL Software: TEL

30 B (B000796)