Automated Inspection With Machine Vision

Size: px

Start display at page:

Download "Automated Inspection With Machine Vision"

Jasper Chase
6 years ago
Views:

1 Automated Inspection With Machine Vision Part 2 Stanley N. Hack, D.Sc., PE ConsulTech Engineering, PLLC November 9, 2015

2 PRESENTATION GOALS Understanding Machine Vision and Its Uses Understanding Machine Vision Components Appreciating the Interacting Complexities of Machine Vision Components and Processing Disclaimer: Most of ConsulTech Engineering s clients regard their processes, which include machine vision applications, as highly proprietary. Many of the applications presented have been devised for this presentation. 2

3 PRESENTATION SCOPE Part I Review: CNY Engineering Expo 2014 Machine Vision Definition Machine Vision Uses Machine Vision Components Illumination Part II: CNY Engineering Expo 2015 Machine Vision Components Cameras and Sensors Optics Part III: CNY Engineering Expo 2016 Machine Vision Processors Machine Vision Software Machine Vision Systems Final Exam 3

4 MACHINE VISION DEFINTION Machine Vision is defined as the technology and methods used to automatically inspect materials, components, and manufactured systems using image-based sensors and systems. Key Words Automatic Inspection Materials, Components, Manufactured Systems Image-Based 4

The system has a priori knowledgeof the imaged object(s), including object shape, size, position, and attributes.

5 MACHINE VISION DEFINITION Machine Vision has the following attributes: The input is an image, and the output is a set of data, such as feature existence, object type, object or feature location, and measurements. The system analyzes inanimate objects. The system has a priori knowledgeof the imaged object(s), including object shape, size, position, and attributes. The system performs its processing repeatedly and often rapidly. The imaging environment, including illumination, geometry, and motion, is controlled by the Machine Vision system. 5

6 MACHINE VISION ENVIRONMENT Machine Vision is unique among all of the computer imaging modalities (computer vision, image processing, medical imaging, remote sensing) in that many aspects of the imaging environment can often be controlled Illumination Sensor positioning Sensor size and resolution Image magnification and orientation Optical filtering Reflections A large component of Machine Vision system design is the optimization of the captured images to make the software s job feasible, easier, and/or faster. 6

7 USES OF MACHINE VISION Quality Control Process Control Robotic Guidance Images courtesy of MicroscanSystems, Inc., NERLITE Lighting Solutions, Renton, WA NDA Expiration 01 December 2007 Image courtesy of Matrox Electronic Systems Ltd., Dorval, Quebec, Canada Image courtesy of Matrox Electronic Systems Ltd., Dorval, Quebec, Canada 7

verification Cracks, dents, and other defects * Defect Inspection * Little or no a priori knowledgeof

8 USES OF MACHINE VISION Manufacturing Quality Control Dimension verification Parts placement accuracy Debris detection * Label placement Label printing Coatings integrity Circuit continuity Color verification Cracks, dents, and other defects * Defect Inspection * Little or no a priori knowledgeof the imaged object(s). Images courtesy of Matrox Electronic Systems Ltd., Dorval, Quebec, Canada Placement Inspection 8

9 USES OF MACHINE VISION Manufacturing Process Control Temperature control Cutting / grinding adjustments Parts sorting Flow and speed control Timing control Robot control Baking Temperature Control Grinding Marks and Defects Robot Welding 9

10 MACHINE VISION COMPONENTS Work station image courtesy of Comark LLC, Milford, MA 10

Machine Vision Components The Three Most Important Components Illumination Illumination Illumination Detailed in Part 1 The Other Most Important Components Cameras and Sensors Optics Synchronization

11 Machine Vision Components The Three Most Important Components Illumination Illumination Illumination Detailed in Part 1 The Other Most Important Components Cameras and Sensors Optics Synchronization (Motion Control) Processing Image courtesy of Operations Technology, Inc., Blairstown, NJ A large component of Machine Vision system design is the optimization of the captured images to make the software s job feasible, easier, and/or faster. 11

Illumination Review Automated Inspection with

Theorem 1 If adding shadows to a captured image

the processing, configure the illumination to

12 Illumination Review Automated Inspection with Machine Vision Part 1 summarized in two theorems: Theorem 1 If adding shadows to a captured image helps the processing, configure the illumination to add the appropriate shadowing. Theorem 2 If shadows in a captured image hurts the processing, configure the illumination to remove shadowing. Figures courtesy of Illumination Technologies, Elbridge, NY 12

Illumination Review Imaging Geometry On-axis

Practical Guide to Machine Vision Lighting,

13 Illumination Review Imaging Geometry On-axis or coaxial Dark field Partial bright field or directional Back lighting Diffuse, dome, or cloudy day Structured Figures from - A Practical Guide to Machine Vision Lighting, Daryl Martin, Advanced Illumination, Inc.,

14 Illumination Review Ultraviolet Spectrum ~ 100 nm 400 nm Visible Spectrum ~ 390 nm 700 nm Infrared Spectrum ~ 750 nm 100 μm 14

15 Cameras and Sensors 15

o Transmission region (shift register) A control circuit causes each capacitor to transfer its contents to its neighbor

16 Sensors CCD(Charge-Coupled Device) o Bell Labs 1970 o Photoactive region (capacitor array) Each capacitor accumulates an electric charge proportional to the light intensity at that location. o Transmission region (shift register) A control circuit causes each capacitor to transfer its contents to its neighbor (operating as a shift register) The last capacitor in the array dumps its charge into a charge amplifier, which converts the charge into a voltage that is read out 16

17 Sensors CMOS(Complimentary Metal Oxide Semiconductor) o First suggested in o Commercialized in late 1980 s early 1990 s o Machine Vision-quality sensors now available from Sony, CMOSIS, and ON Semiconductor o Active Pixel Sensor(APS) each pixel includes its own amplifier o Until recently, lower image quality than CCD o Requires less power than CCD o Lower cost than CCD o Less blooming than CCD o Potential rolling shutter effect o Less quantum efficiency than CCD o Traditionally used for less demanding applications such as cell phones and photography cameras (measurement precision not required) 17

Sensors Bolometer o Measures the power of incident electromagnetic radiation via the heating

infrared spectrum Long-Wave Infrared (LWIR) o 8 to 14 µm o Thermal imaging Mid-Wave Infrared

o 0.9 to 1.7 μm o Low light level imaging (night vision, fog,.

18 Sensors Bolometer o Measures the power of incident electromagnetic radiation via the heating of a material with a temperaturedependent electrical resistance o Predominantly used in the infrared spectrum Long-Wave Infrared (LWIR) o 8 to 14 µm o Thermal imaging Mid-Wave Infrared (MWIR) o 3 to 5 μm o Long distance tracking through the atmosphere Short-Wave Infrared (SWIR) o 0.9 to 1.7 μm o Low light level imaging (night vision, fog,..) Long Distance Tracking (MWIR) Low Light Level Imaging (SWIR) LWIR Bolometer Core Courtesy of Xenics, NV, Leuven, Belgium 18

Camera Specifications Sensor o Sensor Size o Shutter Control o Sensor Resolution (Pixel Count) o Pixel Size o Frame Rate o Quantum Efficiency o Signal-to-Noise Ratio Trigger Capability o

19 Camera Specifications Sensor o Sensor Size o Shutter Control o Sensor Resolution (Pixel Count) o Pixel Size o Frame Rate o Quantum Efficiency o Signal-to-Noise Ratio Trigger Capability o Synchronous o Asynchronous o Exposure Control Camera Type o Area Scan o Line Scan o Color / Black & White Interface Area Scan and Line Scan Courtesy of Edmund Optics Asynchronous Exposure Control 19

Sensors Shutter Control Global Shutter o All pixels are exposed simultaneously o Film cameras use a mechanical shutter o Available with CCD o Newly available with CMOS Rolling Shutter o Sequential

20 Sensors Shutter Control Global Shutter o All pixels are exposed simultaneously o Film cameras use a mechanical shutter o Available with CCD o Newly available with CMOS Rolling Shutter o Sequential exposure start and stop for each row of pixels o Can create motion artifacts o Artifact of legacy CMOS sensors Global Shutter (Courtesy Basler AG) Flash Motion Artifact Due to Rolling Shutter (courtesy Wikipedia) Roling Shutter (Courtesy Basler AG) 20

21 Camera Specifications Interface o Camera Link Specialized Interface MB/S Up to 300 m Cable o USB Vision USB-3.0 Up to 350 MB/S Up to 100 m Cable o GigE Vision Gigabit Ethernet Up to 100 MB/S Up to 100 m Cable o CoaXPress Coax Cable Up to 6.25 Gb/S Up to 100 m Cable o Analog (Legacy) RS-170 (B&W) NTSC, CCIR, SECAM (Color) o Others FireWire or IEEE-1394 USB

Windowed frame rate Lens Mount o C-Mount (16 mm) o F-Mount (Nikon

22 Camera Specifications B&W o Gray Levels (bits) Color o 3 CCD o Bayer Filter o Color Levels (bits) Frame Rate o Total frame rate o Windowed frame rate Lens Mount o C-Mount (16 mm) o F-Mount (Nikon 35 mm) o K-Mount (Pentax 35 mm) o Large Format -M37 X 0.75, Bayer Filter 22

23 Camera Specifications CMOS Sensors UV Blue Green Red IR Quantum Efficiency (QE) Plot Sony CMOS Sensors Courtesy of Point Grey Research, Inc., Richmond, Canada 23

24 Camera Specifications Quantum Efficiency (QE) of Infrared Detection Sensor Materials Courtesy The Optical Society of America, Washington, DC 24

25 Camera Specifications Video Frame Sizes Frame Size Diagonal (mm) Width (mm) Height (mm) Aspect Ratio 4/ : : 3 1/ : 1 2/ : 3 1/ : 3 1/ : 3 1/ : 3 1/ : 1 1/ : 1 High Definition (HD) 16 : 9 Shaded areas indicate standard broadcast aspect ratios Film Frame Sizes Frame Size Diagonal (mm) Width (mm) Height (mm) Aspect Ratio 35 mm (slide) : 1 35 mm (movie) : 1 16 mm : 1 8 mm : 1 Super 8 mm : 1 IMAX : 1 Cinema aspect ratios: 1.85:1 - normal widescreen and 2.39:1 - anamorphic widescreen (rectangular pixels) Television aspect ratios: 4:3 - standard definition and 16:9 - high definition 25

26 Camera Specifications Relationship between bit depth and signal-to-noise ratio Bit-depth o Defines the number of discrete values of gray or of a color o Defines the contrast resolving power o Examples: Bit depth = 8-bits 256 shades of gray or color (RGB) Bit depth = 12-bits 4096 shades of gray or color (RGB) Signal-to-Noise Ratio (SNR) o Primarily due to electronic noise o Can include quantum noise in photon-limited systems Relationship o Pixel bit depth resolution must be greater than the SNR o Example 1: SNR = 40 db = 1 : 10,000 > 4,096 > 256 o Example 2: SNR = 30 db = 1 : 1,000 > 256 < 4,096 26

27 Smart Cameras Internal processing capability o CPU o FPGA Graphical development software Uses o Bar code reading o Optical character recognition o Simple detection and measurement applications Cognex EasyBuilder and InSight Camera Dalsa Sherlock and BOA Camera National Instruments LabView Matrox Design Assistant and IRIS Camera 27

Other Sensors Vacuum Tube - Vidicon, Plumbicon, Orthicon, Saticon, Newvicon

0 via Commons - https://commons.wikimedia.org/wiki/file:orthicon.

svg Solid State CID (Charge Injection Device) Every pixel in a CID array can be

28 Other Sensors Vacuum Tube - Vidicon, Plumbicon, Orthicon, Saticon, Newvicon "Orthicon" by Tecchese-Own work. Licensed under CC BY-SA 3.0 via Commons - Solid State CID (Charge Injection Device) Every pixel in a CID array can be individually addressed via electrical indexing of row and column electrodes Each pixel is read non-destructively Asynchronous trigger and clear Long integration times possible 28

29 Other Sensors X-Ray Ultra-Sound Electrical Impedance Seismic (sound waves) LIDAR (Light RADAR) o Time-of-Flight o Interferometry Courtesy of Google, Inc., Mountain View, CA 29

$Optics Snell's Law (aka Snell Descartes Law, Law of Refraction) Defines the relationship between the angles of incidence and refraction, when referring to light or other waves passing through a$

30 Optics Snell's Law (aka Snell Descartes Law, Law of Refraction) Defines the relationship between the angles of incidence and refraction, when referring to light or other waves passing through a boundary (interface) between two different isotropic media (uniform in all orientations), such as water, glass, or air. = = where: angle measured from the normal of the boundary velocity of light in the respective medium wavelength of light in the respective medium refractive index of the respective medium 30

Lens Specifications Lens Type o Finite / Finite Conjugates o Infinite / Infinite Conjugates o Infinite / Finite Conjugates Size (Diameter) Field-of-View (FOV) Depth-of-Field (DOF) Working

31 Lens Specifications Lens Type o Finite / Finite Conjugates o Infinite / Infinite Conjugates o Infinite / Finite Conjugates Size (Diameter) Field-of-View (FOV) Depth-of-Field (DOF) Working Distance (WD) Magnification Numerical Aperture (NA) or f/stop Chromatic Optical Aberrations Resolution -Modulation Transfer Function (MTF) Contrast Courtesy of Edmund Optics, Barrington, NJ 31

32 Finite / Finite Conjugate Lens Type Single Element + F = M = = = where: F = effective focal length of lens system O = object-to-lens distance I = image-to-lens distance (image on sensor) M = magnification H I = half height of image H O = half height of object 32

33 Finite / Finite Conjugate Lens Type Two Elements F = where: F = effective focal length of lens system F I = focal length of lens closest to image F O = focal length of lens closest to object O = object-to-lens distance I = image-to-lens distance (image on sensor) M = = = d = distance between elements M = magnification H I = half height of image H O = half height of object 33

34 Finite / Finite Conjugate Lens Type Single Element Revisited + F = M = = = where: F = effective Focal Length of lens system WD = Working Distance (WD) or object-to-lens distance BF = Back Focus or image-to-lens distance M = Magnification H S = half of Sensor Height or half height of image H O = half of Field-of-Viewor half height of object 34

$optical axis since the refractive index of a glass is wavelength dependent. Mitigated using lens coatings.$

35 Chromatic Optical Aberrations Transverse chromatic aberration (TCA)occurs white light is used, and the red, yellow, and blue wavelengths focus at separate points in a vertical plane. Longitudinal chromatic aberration (LCA) occurs when different wavelengths focus at different points along the horizontal optical axis since the refractive index of a glass is wavelength dependent. Mitigated using lens coatings. where: C = red (656.3 nm) d = yellow light (587.6 nm) F = blue light (486.1 nm) 35

36 Depth-of-Field Depth-of-Fieldis the increment surrounding the working distance in which the object is in focus. Dependent upon the lens aperture (f/stop) Dependent upon the sensor pixel size which defines the Circle of Confusion 36

37 Depth-of-Field Increased Aperture and Constant Circle of Confusion DOF Decreases 37

38 Depth-of-Field Numerical Aperture NA = sin Image-Space Numerical Aperture (lens is focused to infinity) NA = sin = sin f-number f N = = ~ where: D = lens aperture (entrance pupil) diameter = index of refraction (1.0 in air) = half-angle of the maximum cone of light that can enter or exit the lens NA = numerical aperture NA = image-space numerical aperture (lens focused to infinity) F = lens focal length f N = f-number of aperture 38

39 Infinite / Infinite Conjugate Lens Type Magnification notdependent upon back focus or working distance M d + where: M = magnification F O = focal length of object-side lens F I = focal length of image-side lens H O = half height of object H I = half height of image d = distance between lens elements D = lens aperture diameter 39

40 Infinite / Finite Conjugate Lens Type M where: M = magnification H O = half height of object H I = half height of image F = lens focal length I = lens-to-image distance (back focus) = image-side angle, dependent upon focal length D = lens aperture diameter Magnification notdependent upon working distance 40

Double-Sided Telecentric Lens o Infinite/Infinite conjugate lens o Magnification

41 Telecentric Lens Single-Sided Telecentric Lens o Infinite/Finite conjugate lens o Magnification dependent on focal length o Magnification dependent on back-focus Double-Sided Telecentric Lens o Infinite/Infinite conjugate lens o Magnification dependent on focal length only Uses o Camera lens o Back-light lens Courtesy of Edmund Optics, Barrington, NJ 41

42 Telecentric Lens Experiment Configuration Conventional Lens Image Telecentric Lens Image Conventional Lens / Diffuse Backlight Telecentric Lens / Telecentric Backlight Courtesy of Edmund Optics, Barrington, NJ 42

Modulation Transfer Function (MTF) Plot of perceived contrasts between black and white objects verses linepairs per unit

43 Resolving Power Resolution Number of line-pairs per unit distance that are resolved by an imaging system. Contrast Relative gray-levels of black and white objects produced by an imaging system. Modulation Transfer Function (MTF) Plot of perceived contrasts between black and white objects verses linepairs per unit distance. Limiting Resolution Function of field-of-view, magnification, and sensor size (pixel count). Line-Pair Images Resolution Depiction Courtesy of Edmund Optics, Barrington, NJ 43

44 Resolving Power Object Imaging Lens Image Object Imaging Lens Image White 100% Contrast 20% Contrast White Black Black Courtesy of Edmund Optics, Barrington, NJ 44

45 Modulation Transfer Function (MTF) Courtesy of Edmund Optics, Barrington, NJ 45

Circles Red Circles actual locations imaged points with distortion present predicted locations of

46 Distortion Spherical Aberation % where: AD = actual distance of imaged points from center of field PD = predicted distance of imaged points from center of field with no distortion Focus Range Black Circles Red Circles actual locations imaged points with distortion present predicted locations of imaged points without distortion Light incident near the edges of the lens come to focus too early. 46

47 Test Targets IEEE Resolution Chart Depth-of-Field Test Target EIA Gray Level Chart USAF 1951 Resolution Chart 47

48 Test Targets Projected Distortion Test Pattern Fixed Frequency Distortion Chart TV Test Pattern TV Color Bar Pattern 48

49 Infrared Optics (Lens) Materials % transmittance SWIR 0.9 to 1.7 µm MWIR 3 to 5 µm LWIR 8 to 14 µm wavelength (μm) From R.E. Fisher, et al., Optical System Design, 2 nd Ed. Courtesy McGraw Hill, New York, NY 49

50 PART III: CNY Engineering Expo 2016 Machine Vision Processors Machine Vision Software Machine Vision Systems 50

51 SUMMARY Machine Vision is defined as the technology and methods used to automatically inspect materials, components, and manufactured systems using image-based sensors and systems. Machine Vision is used for Quality Control, Process Control, and Robotic Guidance. Machine Vision Systems include the following components: Illuminators Cameras and Sensors Optics (Lenses) Synchronization (Motion Control) Processors Machine Vision Software is able to extract, highlight, and manipulate information from a captured image, but it cannot add information. In other words, if the software can t see it, it can t process it. Machine Vision Systems Engineering is required for success. 51

52 QUESTIONS?? 52

53 REVIEW QUESTIONS and ANSWERS 1. What is Machine Vision? Machine Vision is the technology and methods used to automatically inspect materials, components, and manufactured systems using image-based sensors and systems. 2. What are the uses of Machine Vision? Machine Vision is used for Quality Control, Process Control, and Robotic Guidance. 3. What are the components that compose a Machine Vision System? A Machine Vision System includes illuminators, sensors (cameras), optics (lenses), synchronizers (motion controllers and/or encoders), and processors. 4. What types of sensors are used in Machine Vision cameras? CCD, CMOS, and bolometers. 5. What are some of the parameters used to specify lenses? Size, focal length, resolving power (MTF and contrast). 6. How excited are you to attend Part III of this series? EXTREMELY! 7. How many PDHs did you earn by attending this session? 53

54 Thank You For Attending Automated Inspection With Machine Vision Part 2 Stanley N. Hack, D.Sc., PE November 9,

IMAGE SENSOR SOLUTIONS. KAC-96-1/5" Lens Kit. KODAK KAC-96-1/5" Lens Kit. for use with the KODAK CMOS Image Sensors. November 2004 Revision 2

IMAGE SENSOR SOLUTIONS. KAC-96-1/5 Lens Kit. KODAK KAC-96-1/5 Lens Kit. for use with the KODAK CMOS Image Sensors. November 2004 Revision 2 KODAK for use with the KODAK CMOS Image Sensors November 2004 Revision 2 1.1 Introduction Choosing the right lens is a critical aspect of designing an imaging system. Typically the trade off between image