imec image sensors and vision systems maarten.willems@imec.be
IMEC FOCUS: SPECIALTY IMAGING Design innovation Process Technologies Machine Vision Industrial Instrumentation Key features: High speed Low power Low noise High-end transports, security & surveillance microscopy Life-Science & Medical Imaging High QE Radiation Hard + Space, Physics & Scientific Non- Visible sensing Spectral sensing 2
IMEC - POSITIONING 200mm / 8 wafer fab 0.13um CMOS based process imaging R&D imager DESIGN CMOS imager MANUFACTURING imager PACKAGING/TEST Dedicated team with 3-shift operations Qualification (characterization, reliability, testing) Optical package, assembly & test Vision system design (HW/SW) Co-design (of specific pixels, analog, digital blocks) Custom circuit calibration, simulation, verification Radiation-hard designs INFRASTRUCTURE DESIGN ENVIRONMENT PROCESS TECHNOLOGIES Specific substrate Pixels (3T, 4T, trench isolation, embedded CCD pixels in CMOS) BSI Backside illumination Hyperspectral filters Special ARCs (CMOS compatible) Stitching / Butting capabilities for large area imagers Radiation-hard coatings Micro-bumping, 3D integration with TSV interconnects
IMEC IMAGE SENSORS & VISION SYSTEMS Line scan imager Cell sorter Microscopy imaging X-Ray Electron detectors UV / EUV imager & detectors HYPER SPECTRAL VISION SYSTEM Snapshot imager Evaluation System LENS FREE VISION SYSTEM Industrial NON VISIBLE IMAGE SENSORS NIR Backside illumination (BSI) STEREO VISION VISION SYSTEM 3D stereo -vision ORGANIC IMAGE SENSORS LOW NOISE HIGH SPEED HIGH QE IMAGE SENSORS embedded CCD (in CMOS) 4
IMEC BACKSIDE ILLUMINATION PLATFORM Quantum Efficiency QE Higher fill factor Better sensitivity in visible Ultraviolet NIR detection Based on IMEC s 0.13 micron Cu BEOL process: Proprietary FSI process with support for variable pixel size 1 Advanced bonding, thinning, passivation modules 0.9 Dedicated ARCs developments 0.8 1 0.8 Quantum Yield taken into account 0.7 0.6 0.5 0.6 0.4 0.3 0.4 0.2 0.1 0 300 400 500 600 700 800 900 1000 Wavelength (nm) 0.2 0 Wavelength (nm) 250 450 650 850 5
IMEC ECCD PLATFORM Extra module in imec 130nm CMOS/CIS technology Narrow gap, single poly electrodes Customizable, BSI compatible CCD device Fully CMOS compatible for ROIC design Excellent charge transfer efficiency (CTE): > 99.9987 % First TDI imager available CCD pixels PG-1 PG-2 PG-3 PG-4 TX CMOS read-out VDD RST VDD SF RS Out read-out sequence FD (TDI = time delay integration)
ORGANIC PHOTO-DETECTORS (OPD) Hyperspectral OPD imaging Many patterned OPDs Ultra-thin layers of organic materials (<100 nm) Deposited from solution or evaporated High absorption coefficient (compared to Si) Broadband or Selectively sensitive to UV-VIS-NIR light OPD technologies @ imec High-sensitivity imaging 1. OPD on CMOS backplane readout 2. OPD on flexible backplane readout Three patterned OPDs Near-Infrared enhancement Color filters on Si +NIR OPD pixels Enhanced performance (crosstalk, resolution, dynamic range, bandwidth) New functionalities (thin, light-weight, large-area)
IMEC IMAGE SENSORS & VISION SYSTEMS Line scan imager Cell sorter Microscopy imaging X-Ray Electron detectors UV / EUV imager & detectors HYPER SPECTRAL VISION SYSTEM Snapshot imager Evaluation System LENS FREE VISION SYSTEM Industrial NON VISIBLE IMAGE SENSORS NIR Backside illumination (BSI) STEREO VISION VISION SYSTEM 3D stereo -vision ORGANIC IMAGE SENSORS LOW NOISE HIGH SPEED HIGH QE IMAGE SENSORS embedded CCD (in CMOS)
HYPERSPECTRAL IMAGING Improves vision and discrimination power by using spectral signature information of surface material / object being captured n 1 2
SPECTRAL IMAGING OPENS A NEW DIMENSION Spectrometer Color camera Hyperspectral camera λ y x B G R λ1,2,3 λ1..n y x Accurate spectral analysis of one spatial pixel only Seeing RGB colors of one image only spectral signature of images reveals objects chemical surface composition
CONVERGENCE OF IMAGING & SPECTROSCOPY High discrimination power SPECTROSCOPY Single point with highest spectral resolution IMEC approach: spectral filters integrated on pixels Bayer pattern OR Tiled pattern OR Wedge pattern IMAGING SPECTROSCOPY RGB filters on multiple pixels High speed, high spatial resolution IMAGING
IMEC APPROACH: FABRY-PEROT SPECTRAL FILTERS ON TOP OF IMAGE SENSOR L θ θ Wavelength selection depends on cavity length L k 2nLcos Tx (%) Narrow-band & high transmission efficiencies spectral filters Different cavity heights = different spectral wavelengths captured! FWHM ~ 5-20nm
SPECTRAL FILTERS APPLIED TO COMMERCIAL SENSOR WAFERS USING SEMICONDUCTOR WAFER PROCESSING Example of IMEC hyperspectral filter structures processed at wafer-level on top of commercial CMOS image sensor wafer (here on CMOSIS s CMV2000 & CMV4000 sensors)
IMEC HYPERSPECTRAL TECHNOLOGY USP Filter response Filters fully integrated on pixels and CMOS compatible - Extra post-processing step in standard image sensor production - Small feature sizes OK as based on standard lithography - No assembly, no alignment nor stray light issues Optical filters & imager can be customized to match with final application requirements - Number of filters, central wavelength, FWHM - Possibility to fill sensor with ONLY selected bands of interest (not restricted to continuous wavelength, or range or specific line arrangement) FWHM 1 2 n Continuous Line-scan Custom Line-scan Snapshot Tiled Snapshot Bayer Mosaic All tunable: results in high speed, compact & cost-effective hyperspectral solution!
SUMMARY OF HSI SENSOR DESIGN APPROACHES Line-scan wedge design Snapshot tiled area design Snapshot mosaic per-pixel design Or Or SCANNING movement needed Highest spatial & spectral resolution achievable (e.g. 100 spectral band images of 4MPx resolution today) NO SCAN = real-time HSI data-cubes acquisition Spatial vs. Spectral resolution trade-off (e.g. 32 band images of 256x256 resolution today) Optical duplicator needed (e.g. 32 lens-array today) NO SCAN = real-time HSI data-cubes acquisition Spatial vs. Spectral resolution trade-off (e.g. 16 band images of 512x272 resolution today) Potential resolution reconstruction increase by de-mosaicing algorithms NO optical duplicator needed (standard lens camera integration)
1088 pixels LINE-SCAN SENSOR DESIGN (GEN 1) CMOSIS CMV2000... 2048 pixels 8 pixels 1000nm 8 pixels 996nm 8 pixels 992nm.................... 100 spectral bands of 2048x8 spatial pixels each 8 pixels 608nm 8 pixels 604nm 8 pixels 600nm Key specifications - Spectral resolution: 100 bands in 600-1000nm with 4nm incremental steps - FWHM: ~ 15nm - Spatial resolution: 2048 pixels x length of scan - Speed: up to 340 fps (full sensor frame)
1088 pixels LINE-SCAN SENSOR DESIGN (GEN 2) CMOSIS CMV2000 8 pixels 900nm 8 pixels 895nm 8 pixels 890nm.................. 100 spectral bands of 2048x8 spatial pixels each 2048 pixels 8 pixels 480nm 8 pixels 475nm 8 pixels 470nm Key specifications - Spectral resolution: 100 bands in 470-900nm with 5nm incremental steps - FWHM: ~ 15nm - Spatial resolution: 2048 pixels x length of scan - Speed: up to 340 fps (full sensor frame) Sensor available Q4-2014
1088 pixels SNAPSHOT TILED SENSOR DESIGN 1 tile = 256x256 pixels CMOSIS CMV2000 792nm 582nm 775nm 761nm 841nm 874nm 630nm 899nm 820nm 833nm 725nm 688nm 801nm 865nm 920nm 885nm 700nm 715nm 644nm 595nm 895nm 911nm 606nm 905nm 676nm 856nm 656nm 621nm 557nm 907nm 737nm 600nm 2048 pixels Key specifications - Spectral resolution: 32bands in 600-1000nm with 12nm incremental steps (optical duplicator needed) - FWHM: ~ 15nm - Spatial resolution: 256x256 for each spectral band - Speed: up to 340 data-cubes / s (max sensor limit)
1088 pixels SNAPSHOT MOSAIC SENSOR DESIGN (GEN1) 4x4 mosaic = 16 bands CMOSIS CMV2000 465 nm 546 nm 586 nm 630 nm 474 nm 534 nm 578 nm 624 nm 485 nm 522 nm 562 nm 608 nm 2x1mosaics 496 nm 510 nm 548 nm 600 nm 6x6 mosaic 4x4 mosaic Key specifications 2048 pixels - Spectral resolution: 4x4 mosaic (1filter / pixel) = 16bands in 465-630nm with ~11nm incremental steps - FWHM: ~ 15nm - Spatial resolution: from 512x272 (per band) up to 2Mpx (per band) depending on de-mosaicing algorithm - Speed: up to 340 data-cubes / s (max sensor limit) - No optical duplicator element needed = simplified camera design
IMEC HYPERSPECTRAL FILTER TECHNOLOGY ROADMAP Snapshot Mosaic per-pixel Integration of Panchromatic + spectral filters 4x4 Mosaic GEN1 470-630nm (16bands) Mixing Panchromatic + HSI filters 4x4 Mosaic 600-900nm GEN2 5x5 Mosaic 600-1000nm GEN3 Mixing RGB CF + HSI filters Snapshot Tile HSI Tiled filter pattern GEN1 600-1000nm (32bands) HSI Tiled filter pattern 470-1000nm Line-scan Wedge HSI Wedge filter pattern GEN1 600-1000nm (100bands) HSI Wedge filter pattern GEN2 465-900nm (100bands) HSI Wedge filter pattern 400-1000nm (100bands) 2012 2013 2014 2015 2016
WHERE IS THE KEY SPECTRAL INFORMATION? Color measurements 380-720m Chlorophyll absorption / vegetation reflectance 510-970nm Water absorption 970nm +1450nm +1850nm Fluorescence imaging peaks 450-720m Oxygenation in blood 500-900m CH4 2300nm CO2 2100nm +3500nm + 4800nm (nm) 350 400 450 500 550 600 650 700 750 800 850 900 950 1000 1500 3500 5000 450-700m Skin tones / tissue chromophores 750-950m Hydrocarbon organic compounds (Ethanol, Benzene, Aromatics, ETBE, MTBE, Toluene...) 1700nm -1900nm 2300nm -2400nm Minerals mapping PVC / plastic recycling
COMMERCIAL CAMERA PARTNERS FOR HSI SENSOR INTEGRATION High speed camera Cameralink or CoaXPress interface Compact optical design with ZIFF socket to swap different HSI sensors Robust camera implementation for use in industrial, medical and global security Intelligent D3 camera platform Giga/Ethernet interface Focus on remote sensing, UAV & machine vision Smart camera Giga/Ethernet interface Reprogrammable FPGA pre-processing hypercube data inside camera Focus on industrial & machine vision Intelligent vision module Giga/Ethernet interface Motorized optics Embedded HSI data processing include storage Multi-camera head option USB 3.0 vision camera High-speed USB3.0 camera interface Ultra compact camera: 26.4 x 26.4 x 21.2 mm 27grams weight only! Handheld mobile device First proof-of-concept of IMEC s HSI sensors integration to mobile Android OS open platform USB2.0 interface
NEW PARTNERSHIP WITH XIMEA WORLD S SMALLEST INDUSTRIAL USB3 VISION AND HYPERSPECTRAL IMAGING CAMERA 465nm 510nm 548nm 600nm 474nm 522nm 562nm 608nm 485nm 534nm 578nm 624nm 496nm 546nm 586nm 630nm http://www2.imec.be/be_en/press/imec-news/imec-ximea-hyperspectral-imaging-camera.html
Precision farming: Detection, diagnosis and control of plant diseases environmental stresses weeds and invasive species Growth monitoring Treatment optimisation Yield prediction RGB Hyperspectral
KEY DRIVING HYPERSPECTRAL INDUSTRIAL APPLICATIONS Remote sensing UAV/drones & nano-satelites for: Precision agriculture Environment monitoring Terrestrial / maritime earth observation Life-science / spectroscopy instrumentation Imaging spectroscopy analyzers DNA sequencers / flow cytometers Water monitoring analyzers Blood / urine analyzers Machine vision / Optical sorting Food sorting / quality grading Pharmaceutical defect inspection Industrial inspection (plastic, ceramic, glass, etc...) Robotic machine vision Mining / Mineralogy Print quality inspection Automotive & Transport Night vision systems Fuel monitoring systems Medical imaging Security / Surveillance Surgery-guided imaging Fluorescence microscopy Endoscopy Ophthalmology / retina imaging Wounds imaging Industrial gas leaks monitoring Intrusion detection / authentication Rescue Forensics
THERE ARE ALSO HANDHELD & CONSUMER APPLICATIONS! Food quality grading (e.g. sugar content in fruits, monitoring calorie intake, etc...) Cosmetic / Skin tone measurement (e.g. make-up advice, etc...) Source L OREAL Skin-care / personalized medicine (e.g. melanoma, vitiligo, diabetic ulcers, wound care...) True Colors (e.g. white balancing, calibration, ALS ) Anti counterfeiting spectral tag readers
IMEC HYPERSPECTRAL IMAGING ROADMAP Establishing partnerships with multiple camera partners Public demo Multispectral video 32 bands at 180 fps Public demo 600 1000nm 100 spectral bands HSI Evaluation Systems HSI Snapshot imager HSI cameras New filter architectures Per-pixel mosaic design 470 600 nm additional coverage 400-1000nm full coverage Explore new concepts IR filters bands coverage (1-2.5um) Microscopy set-up Medical applications HSI Line-scan imager Full system with imec sensor + camera + HSI software 2012 2013 2014 2015+
IMEC IMAGE SENSORS & VISION SYSTEMS Line scan imager Cell sorter Microscopy imaging X-Ray Electron detectors UV / EUV imager & detectors HYPER SPECTRAL VISION SYSTEM Snapshot imager Evaluation System LENS FREE VISION SYSTEM Industrial (MEMS based) NON VISIBLE IMAGE SENSORS NIR Backside illumination (BSI) STEREO VISION VISION SYSTEM 3D stereo -vision ORGANIC IMAGE SENSORS LOW NOISE HIGH SPEED HIGH QE IMAGE SENSORS embedded CCD (in CMOS)
LENS-FREE IMAGING VS. CONVENTIONAL MICROSCOPE: COMPACT, COST-EFFECTIVE, SCALABLE MATCHED TO APPLICATION! Traditional phase-contrast microscope - Bulky, optically and mechanically complex - Expensive high quality lenses required - Limited field of view at high resolution - Focusing by mechanical movement of optical elements Not suitable for large-area inspection, because of limited single microscope throughput and difficulty to put many in parallel Lens-Free Imaging (LFI) microscope - No lenses (minimal optical and mech. components) = miniaturized + low cost - Large field-of-view ( FOV > 100mm 2 depending on imager size) - High resolution (e.g. < 1.4µm capable) - Auto-focus is part of post-processing steps achieved at software level Removing lenses enables very compact and cost-effective system and integrated approach can be massively parallelized for large area inspection
EXAMPLE OF LENS-FREE IMAGING LARGE FIELD OF VIEW WITH HIGH RESOLUTION High level of detail (1.23um resolution) Large field of view = imager size ~ 6.4mm x 4.6mm
RESOLUTION EXCEEDS LARGE FOV MICROSCOPE IMAGING Large FOV LFI Microscopy 4x 100 µm
IMEC IMAGE SENSORS & VISION SYSTEMS Line scan imager Cell sorter Microscopy imaging X-Ray Electron detectors UV / EUV imager & detectors HYPER SPECTRAL VISION SYSTEM Snapshot imager Evaluation System LENS FREE VISION SYSTEM Industrial (MEMS based) NON VISIBLE IMAGE SENSORS NIR Backside illumination (BSI) STEREO VISION VISION SYSTEM 3D stereo -vision ORGANIC IMAGE SENSORS LOW NOISE HIGH SPEED HIGH QE IMAGE SENSORS embedded CCD (in CMOS)
STEREOMATCHING DEPTH MAP GENERATION Compact form factor: CMOS sensor-based Latency: Fairly low latency (<1 frame) Object shape: Object shape integrity Lighting condition: Good for indoor/outdoor Background: Can deal with complex background Distance range: Very long (vs. active TOF with IR) Reliability: with supplement Confidence index
KEY SPECIFICATION ITEMS Spec. Item Number Remark Disparity Levels 128 (1/8) 64 and 32 are also supported ; 1/8 fractional disparity Frame Rate & Resolution 60FPS 1280x720 1920x1080 is also supported Pixel Clock Rate (MHz) 100 MHz For TSMC 90nm library, using Cadence RC synthesis FPGA Clock Rate (MHz) 74 MHz For side-by-side 1920x1080 MDE/s 7962.6 MDE/s = (Frame Rate) * (Resolution) * (Dispartiy) / 10E6 Power Consumption 82.4 mw For using Cadence RC vectorless power analysis Extra DRAM size NO No frame buffer required Ambient Light > 100 Lux Depends also on camera SNR Rectification Real-time Through software register setting Gate Count 610 K MIN TYPICAL : 610k (for logic, VGA 60FPS, 128 level ) Latency <0.2 frame rate 2011 45
IMEC IMAGE SENSORS & VISION SYSTEMS Line scan imager Cell sorter Microscopy imaging X-Ray Electron detectors UV / EUV imager & detectors HYPER SPECTRAL VISION SYSTEM Snapshot imager Evaluation System LENS FREE VISION SYSTEM Industrial (MEMS based) NON VISIBLE IMAGE SENSORS NIR Backside illumination (BSI) STEREO VISION VISION SYSTEM 3D stereo -vision ORGANIC IMAGE SENSORS LOW NOISE HIGH SPEED HIGH QE IMAGE SENSORS embedded CCD (in CMOS)
Thank you!