Applied Machine Vision ME Machine Vision Class Doug Britton GTRI 12/1/2005 Not everybody trusts paintings but people believe photographs. Ansel Adams
Machine Vision Components Product Camera/Sensor Illumination Optics Trigger Acquisition card? Processor/PC Software Controller digital I/O A system capable of acquiring one or more images using an optical noncontact sensing device capable of processing, analyzing and measuring various characteristics so decisions can be made. - Machine Vision Online
Steps Toward Success Understand the problem you are solving Look at current processes or solutions Learn about the environment Gather production information Dimensions, orientation, presentation of product Types and range of expected defects Production rates and distribution of defects Know the processing constraints Design a MV solution that maximizes probability of success You don't take a photograph, you make it. Ansel Adams
Review: EM Spectrum & Light Is it a wave or particle? Who cares!
Characterize Product & Defects Spectral absorption & reflectance Surface texture specular or diffuse Fluorescence properties Can humans see defects? What about non-visible properties?
Spectral Characterization Transmission or Reflectance, % 100 90 80 70 60 50 40 30 20 10 0 300 500 700 900 1100 1300 1500 1700 1900 Wavelength, nanometers Package Inspection Example Yellow Foam Reflectance White Foam Reflectance White Foam Wrapper Transmission Cryovac Bag Transmission Clear Bag Transmission
Common Light Sources Sunlight Tungsten Mercury Vapor Halogen Fluorescent LED Laser
Choosing A Light Source Match the spectral response of product & defect with light source that give good contrast Align spectral peaks of light with spectral reflectance/absorption of product/defect
Types of Illumination Diffuse Front Directional Pros: minimizes shadows & specular reflections Cons: surface features less distinct Type: fluorescent linears & rings Pros: strong, relatively even illumination Cons: shadows, glare Type: single (shown) and dual fiber optic light guides From EdmundOptics.com
Illumination Cont. Glancing Structured Light Pros: shows surface defects/topology Cons: hot spots, severe shadowing Type: fiber optic light guides Pros: surface feature & contour extraction Cons: intense source; absorbed by some colors Type: line generating laser diodes, fiber optic line light guides From EdmundOptics.com
Illumination Cont. Polarized Light Diffuse Axial Pros: even illumination, removes specularities Cons: lower intensity through polarizer Type: filter attaches to many existing lenses and light sources Pros: shadow-free, even illumination; little glare Cons: lower intensity through the beamsplitter Type: LED axial source, fiber optic-driven axial adapters From EdmundOptics.com
Illumination Cont. Brightfield/Backlight Pros: High contrast for edge detection. Cons: Eliminates surface detail. Type: Fiber optic backlights, LED backlights. Darkfield Pros: High contrast of internal & surface details. Cons: Poor edge contrast. Not useful - opaque objects. Type: Fiber optic darkfield attachment, line light guides. From EdmundOptics.com
From EdmundOptics.com Illumination Summary Application Requirements Type of Object Under Inspection Illumination Type Suggested Reduction of Specularity Shiny Object Diffuse Front, Diffuse Axial, Polarizing Even Illumination of Object Any Type of Object Diffuse Front, Diffuse Axial, Ring Guide Highlight Surface Defects or Topology Highlight Texture of Object with Shadows Nearly Flat (2-D) Object Any Type of Object Single Directional, Structured Light Directional, Structured Light Reduce Shadows Object with Protrusions 3-D Object Diffuse Front, Diffuse Axial, Ring Guide Highlight Defects within Object Transparent Object Dark Field Silhouetting Object Any Type of Object Backlighting 3-D Shape Profiling of Object Object with Protrusions, 3-D Object Structured Light
Imaging Geometries Field of View (FOV) Viewable area in object space Working Distance (WD) Distance - front of lens to object Spatial Resolution Smallest feature size distinguished by MV system Depth of Field (DOF) Max object depth that can be kept in focus From EdmundOptics.com
Sensors Infra Red (IR) VOx microbolometer Near IR InGaAs Visible Silicon CCD CMOS arrays Ultra Violet CMOS arrays NMOS detectors CCD Sensor Photons Electronics Pixel Data Filter Detector
Pixels and CCD arrays Square vs Rectangular pixels Larger pixels/unit area (7.5 micron) More photons absorbed = less noise Lower resolution Smaller pixels/unit area (4.5 micron) Fewer photons = more noise Higher resolution CCD dimensions impact FOV & lens specification
Resolution & Dynamic Range Spatial Resolution Smallest feature size distinguishable Require minimum 2 pixels/line pair Dynamic range Goal is to maximize contrast Application dependent
Spatial Resolution Example Object is 4 mm Sesame Seed Want at least 2 pixels on each seed (4 mm)/(2 pixels) = 2 mm/pixel minimum Notice nothing mentioned about: Sensor size Working distance Lens focal length Resolution requirements often dictate the choice of sensor and optics.
Review: Simple Lens Equation Focal Point Lens Object Image Optical Axis v f Optical Center u 1 u 1 + = v 1 f f = lens focal length u = working distance v = distance to sensor
Focal Length Calculation Assume pin hole camera Given: Spatial resolution Sensor size Field of view Calculate Working distance Focal length HFOV := 1.75ft Calculate the required focal length for the problem!! CCD size 1/3in ccd 4.8x3.6mm VFOV := 1.3ft SENSOR_HGT := 8ft H_PIX := 1024 V_PIX := 768 OFFSET := 0ft wo := 4.8 mm wox := 3.6 mm WD = 8ft H_THETA = 12.484deg V_THETA = 9.29deg f = 21.943mm fx = 22.154mm H_RES = 48.762 1 in V_RES = 49.231 1 in
Lens Distortion No object information is lost Information is only misplaced in the image. Pincushion Barrel
Cameras Area scan Traditional sensor Interlaced video Legacy from TV Every other line Issues with freezing frames/motion Progressive scan Each row of pixels scanned out one at a time Line scan Single line/array of pixels Successive lines form image Timing/trigger and lighting crucial Color vs. Monochrome Can you get away without color?
Frame Rate vs. Shutter Speed Frame rate Rate that camera is produces frames TV 30 frames/sec interlaced Shutter speed Exposure time Integration time of sensor Short integration time -> freezes motion, but requires a lot of light 1/60 1/10,000 sec
Color Cameras 3-chip color Uses prisms to split color to 3 CCD s Best quality color True resolution Single chip color Bayer tile configuration Filter each pixel Twice the green pixels More human sensitivity in green
Color Camera Spectrum Sony ICX-424AQ sensor 1/3 single chip color
Grayscale Cameras Integrate across all three RGB color bands Most CCD s respond in NIR ~700-900nm
Filtering Application Pharmaceutical inspection Pills are different colors Want to make sure that each package contains the right pills Can you use a monochrome camera? How do we make it work? Edmund optics example
Bun Imaging System Cloudy Day Illuminator diffuse light Single chip color camera with integration of 2.5 msec 100% product inspection Inspection items: Bun color avg. & dist. Garnish coverage & dist. 2D circular size/shape Grease/contaminants Feedback control of Oven
Bun Imaging System - Database Provides immediate feedback to operators Stores & catalogs quality data Available to managers over network for real-time analysis Inspection Processor & System Local Log File Local User Remote User Remote User Remote User Bun Classifier Data Aggregation Database Queuing Database Specific Dequeuing Network Database Server
Color Data Without Controller Data collected on bun line at Bakery 70.00 65.00 60.00 L-Value 55.00 50.00 11:05 11:14 11:23 11:32 11:41 11:50 11:59 12:08 12:17 12:26 12:35 12:44 12:53 13:02 13:12 13:21 Time
Color Data with PI Controller Test of PI controller to regulate L-value after it was purposely deviated from the setpoint 70.00 L-Value 65.00 60.00 55.00 L-value Target L-value Min. Range Max Range 50.00 15:36 15:39 15:42 15:46 15:49 15:52 15:55 15:58 Time