Factors Affecting Pixel Scaling Limits for cellphone imaging systems

Size: px

Start display at page:

Download "Factors Affecting Pixel Scaling Limits for cellphone imaging systems"

Daniella Gallagher
6 years ago
Views:

1 Factors Affecting Pixel Scaling Limits for cellphone imaging systems October 28, 2010 Richard Crisp

2 Agenda Pixel Scaling Limits Optical Considerations Image Sensor Considerations Moore s Law Considerations Image Processing S/N improvement Resolution improvement

3 Key Question Q: How small can pixels be and what limits them? A: How good of an image do you want? Source: Janesick S/N = 28 S/N = 5.3 S/N = 3.6 S/N = 0.83 Photon Shot Noise impact on S/N

Key issues for cellphone application Number one problemis getting enough photons to the image sensor considering: Overall module size Limits image sensor photosensitive area Exposure parameters Max

4 Key issues for cellphone application Number one problemis getting enough photons to the image sensor considering: Overall module size Limits image sensor photosensitive area Exposure parameters Max integration time, minimum lux Imaging Optics F#, magnification You must trade-off: Signal to Noise ( SNR ) vs Resolution Resolution, Pixel size = Less signal (photons) per pixel Lower SNR Source: Sony

5 Fixed sensor die size: Pixels must shrink for higher resolution cameras Source: Chen 3T Pixel Layout Highest resolution Poorest low light response Highest LP/mm req t Fixed die sizes Sensor diagonals of: 1/2.3 1/3.2 1/4 1/10 etc Lowest resolution Best low light response Lowest LP/mm req t

6 Part 1: Delivering Light to the Sensor

7 Pixel Geometry: How many photons is your pixel receiving? Incident Photons/ Pixel Maximum Theoretical SNR Photons Source: Catrysse

8 Pixel Size, System Noise Impact Source: Janesick

9 Pixel Size Impact C D = contrast Well capacity limits Maximum S/N & Dynamic range Source: Janesick Limited by pixel size

(lux) L * N L (Photons/cm 2 -sec) Scene Luminance How much does the lens spread the light

10 Delivering photons to the sensor: the impact of Imaging Lens F# and magnification F#, m, T L m= magnification T L = Transmission of Optics N L = photons/cm 2 -sec/lux L=Luminance level (lux) L * N L (Photons/cm 2 -sec) Scene Luminance How much does the lens spread the light flux? (magnification) L * N L T L 4*F# 2 *(1+m) 2 Incident Luminance at Sensor (Photons/cm2-sec)

11 F#, system noise impact Source: Janesick Read Noise (e-) Air

$Optics and the Airy Disk: Focal ratio: Sets spot size for diffraction limited optics Source:$

12 Optics and the Airy Disk: Focal ratio: Sets spot size for diffraction limited optics Source: Catrysse Airy Disk Diameter (microns) Airy Disk Diameter ~3 microns, f/2.8 ~10 microns, f/8.0

13 NyquistSampling of Airy Disk Pixel Pitch: Sized to fit Airy Disk (spot): 550 nm test wavelength Pixel Spot (Airy disk) Pixel OVT BSI2 Sony Exmor-R OVT BSI 1 Pixel size (microns) Optimum F# Airy Diameter (microns) Optical resolution LP/mm NyquistSampling Criteria: =1.22 # Exact NyquistSampling: 2 pixels to cover Airy Diameter (spot)

14 What changes for color imaging? Color Pixel Monochrome Pixel = Color Masked Pixel Color Masked Pixel Color Masked Pixel Color Masked Pixel Marketing will call this four pixels Sampling theory treats the fourpixel unit as one pixel

15 Color Masked Pixels vsmonochrome Pixels: optimum spot size, F# changes Spot (Airy disk) Spot (Airy disk) Pixel Pixel Pixel Pixel Pixel Pixel Pixel Pixel Pixel Pixel 2x2 Sampling unit: treated as a single pixel Same sensor base pixel size: different optimum spot size 2:1 difference in optimum F# 2:1 difference in LP/mm required by optics

16 Why Proper Sampling is important: Aliasing Artifact Example: Color image sensor Source: Catrysse Undersampled f/2.8, 3 micron pixel Undersampled f/8, 3 micron pixel Proper sampling f/2.8, 1.5 micron pixel Must properly match F# to pixel size to avoid aliasing artifacts

17 Aliasing Artifact Examples

optics to offer benefit Fco= Fcutoff= 1/(λ*f#) Air Unit to unit manufacturing optical tolerances

18 Imaging optics: MTF impact on image quality Good MTF Poor MTF Good MTF Sharp Image Source: Lomheim Poor MTF Blurry Image Source: Smith [(LP/mm) / Fco] Smaller Pixels Require more LP/mm from the optics to offer benefit Fco= Fcutoff= 1/(λ*f#) Air Unit to unit manufacturing optical tolerances degrade MTF, faster F#: tighter tolerancing req d Need Good MTF for Sharp Image: more difficult at higher LP/mm

19 Summary: Pixel Size, F#, LP/mm for optics Larger Pixels Smaller Pixels Less Demanding Imaging Optics: Slower F# Lower LP/mm Req t Easier to get good MTF More Demanding Imaging Optics: Faster F# Higher LP/mm Req t Harder to get good MTF Can low cost optics support LP/mm with good MTF?

20 Key Points Pixel size is limited by several factors: physical size of camera module, desired low light response, minimum tolerable S/N etc High resolution sensors demand high quality optics: LP/mm and MTF Optics: for interesting sensor size formats what F#, LP/mm, MTF is feasible for low cost optics? Can wafer level deliver? What is the competition? What sort of manufacturabletolerances are achievable for such optics in production? Little benefit realized by shrinking pixels unless sensor and optics can rise to challenge

21 Part 2: Image Sensor Design: Pixel Size Considerations

22 Image Sensors A perfect image sensor No light lost reaching photosensor: color filter array has perfect transmission, no light scattering reaching photosensor 100% Quantum efficiency: No wasted photons, each one interacts with the silicon No crosstalk: all the light winds up in the correct pixels, all the charge collected in the proper pixels Perfect charge collection: no charge lost once generated No read noise, no RTN No dark signal noise No fixed pattern noise Unlimited well capacity Image sensor design / Wafer fab process: determines how close to perfection you get

23 Real world challenge: Getting light to the pixel Source: Catrysse

24 Microlensesto the rescue? Used to concentrate and redirect light onto photosensitive region of pixel Source: Pain

25 Microlens design: Energy flow analysis for small pixels Used to concentrate and redirect light onto photosensitive region of pixel Source: Huo Smaller pixels: harder to get light to photodiode

26 Microlensoperational regimes Source: Huo

27 Microlens: Refraction Regime Source: Huo

28 Microlens: Refraction Regime Optical Efficiency Source: Huo

29 Microlens: Refraction Regime Optical Crosstalk Source: Huo

30 Microlens: Diffraction Regime Source: Huo

$Diffraction$ Regime

31 Microlens: Diffraction Regime Source: Huo Source: Huo

32 Microlenschallenges: sub 2 micron pixels Poor Energy delivery to photodiode degrades QE, SNR Crosstalk Source: Huo 1.75 micron 1.4 micron 0.97micron

33 Optimum MicrolensRadius Unmanufacturable requirements in diffraction regime Source: Huo

34 MicrolensApplication Problems Source: Pain Obscuration, scattering

35 Pixel Vignetting: QE reduction off axis by frontside wiring optical tunnel B A Center of sensor (A) Edge of sensor Air (B) QE reduction degrades SNR for off-axis portions of image Source: Catrysse Air

36 Lightpipe Details Source: Gambino

37 Fixing the Optical Tunnel with BSI Frontside Illuminated Backside Illuminated Source: Pain ie: faster F/#

38 For Frontside Illumination: Red light penetrates deepest: has worst diffusion MTF/ Crosstalk Source: Roy

39 For Backside Illumination: Blue light penetrates least: has worst diffusion MTF/ Crosstalk Source: Roy

40 What about CCD vscmos? (cross sections) w/microlens Source: Meisenzahl wo/microlens FSI Full Frame CCD (requires mech shutter) Source: Janesick FSI Interline CCD (w/microlens) (global electronic snap-shutter) Source: Pain FSI CMOS BSI CMOS

41 CCD vscmos Sensor type Pixel Vignetting Shuttering System Integration CMOS Bigissue: optical tunnel from frontside wiring, small pixel apertures CCD Less of an issue: no wiringtunnel in standard CCDs: better angular response typically but microlensesused on interline architecture and some full-frame Rolling shutter (motion artifacts) or global snap-shutter (pixel fill factor issue) or mechanical shutter (cost, size, reliability) Mechanicalshutter (cost, size, reliability) or global snap shutter (for interline architecture at cost of fill factor). Shutterless frame transfer can be used for small arrays High: can build all support circuitry on-die if desired Low: fab process incompatible (classically) with integration of other analog and logic functions Speed High speedvia multiple parallel outputs if needed. Sensor has random access characteristics: easier for video Challenge: slower readout due to serial nature of CCD

42 Part 3: Moore s Law Scaling Considerations

43 Basic CMOS Pixel Cell Pixel cell M1 M2M3 Source: Chen

44 Shrinking Transistors: improve fill-factor (DSLR use of Scaling) Lower Well Capacity Lower Max SNR Lower QE Higher Well Capacity Higher Max SNR Higher QE Source: Chen Large transistors Same pixel pitch Low fill factor Small transistors Same pixel pitch High fill factor

45 Shrinking Transistors: increase resolution (Cellphone use of Scaling) High Well Capacity Higher SNR Low Well Capacity Lower SNR Small transistors Smaller Pixel Pitch Low to moderate fill factor Source: Chen Large transistors Larger Pixel Pitch Low fill factor

46 Exploiting Moore s Law Scaling Adding transistors to pixel for enhanced functionality Potentially useful functions enabled by scaled process technology Global snap-shutter Eliminates motion artifacts On-chip pixel summing (binning) Improves low light performance at expense of resolution Good for video, good for low light stills High Dynamic Range Pixel Switched MIM capacitors for well capacity expansion Sample/reset during integration Multi-sampled sense amplifier Reduces read noise by multiple samples of charge Digital Pixel Sensor: Adds sample/hold and A/D converter circuitry into each pixel Parallel, all digital architecture supports high speed video

47 Advanced Pixel Design: Implementation Challenges Classic drawbacks to increased pixel complexity Trades sensitivity (fill factor) for functionality Wiring/power busing through array Additional array noise sources to manage: increased array wiring needs BSI can reduce fill factor impact of additional pixel wiring Metal can cover pixel on frontside of wafer: light enters from back Aggressively scaled wafer fabrication process mitigates classic drawbacks Small transistors / aggressive contacted metal pitch Can reduce impact of pixel complexity on fill factor Maintains reasonable low light performance Don t excessively shrink the pixel, shrink the transistors/metalization instead

48 Low light stills, video Trades resolution for low light response 2x2 binning makes four pixels behave as one: 4X the light collection area Source: Janesick

49 Switched MIM Capacitors Used for dynamic range expansion Turn on MIM MOSFET when signals are strong to increase pixel well capacity: improves immunity to saturation Source: Janesick

Scaled transistor issues: Moore s Law Scaling & RTN (RTS) RTN= Random Telegraph Noise (Signal) Source: Janesick Random trapping/detrappingof charge carriers in

50 Scaled transistor issues: Moore s Law Scaling & RTN (RTS) RTN= Random Telegraph Noise (Signal) Source: Janesick Random trapping/detrappingof charge carriers in source-follower transistor s channel: modulates transistor conductivity Smaller geometry transistors more affected by RTN Ultimate limit to on-chip noise: establishes noise floor

51 Buried Channel: reduced RTS/RTN Fab process change Source: Janesick Ion implanted channel doping Source: Janesick Modification to Fab process: Buried Channel Transistor

52 What about Quantum Films / Electron Multiplying Amplifiers? Basic idea: increase # photoelectrons per incident photon Quantum films Electron multiplying amplifiers (CCD) Improve sensitivity in low light conditions, overcome read noise limitations Basic mechanism: more than one photoelectron generated per interacting photon (n i >1) Sounds interesting but: Increases photon shot noise

53 Signal to Noise (shot noise limited) = = = = Incident Photons are vital to S/N

54 Source: Janesick n i = quantum yield

55 Source: Janesick

56 Source: Janesick

57 Source: Janesick

58 Sensitivities What SNR of system Imaging Optics Dynamic range Low light response Pixel Size Microlens Front vs Backside illumination Fab process = no impact, 5 = major impact

59 Connections / Dependencies Pixel size: sets low light performance (still image and video), Maximum SNR, Dynamic Range Co-design of Sensor/Optics Design the Sensor based on cost-effective optical manufacturing Realizable LP/MM, F#, MTF for low-cost design suited for high volume manufacturing Design the Optics based on rationally chosen pixel design / size Low light response Shuttering (electronic or mechanical) Moore s law scaling can permit better performing on-chip snap shuttering due to reduced impact on Fill-factor/QE than unscaled transistors Required dynamic range (well capacity/system noise) Cost effective wafer fab process FSI or BSI BSI: bulk vssoi

60 Part 4: Image processing considerations

61 Source: Janesick

62 Source: Janesick

63 Source: Janesick

64 Processing: FLAT FIELDING FPN REMOVAL RAW C DET = 0.04 CORRECTED Source: Janesick

65 Processing: Time sequential combining of images How do you coregister all classes of images for averaging? Flat Fielded Raw Images (With sensor FPN) Source: Janesick

66 Superresolution Source: Lusser Group of low resolution undersampled and shifted images are co-registered and combined Increase in effective resolution attained

registered/combined Source: Lusser Telephoto reference

67 Super Resolution Examples Note glasses are apparent 8 sequential video frames 10 sequential video frames registered/combined Source: Lusser Telephoto reference Superresolution

68 Processing: What Can We Do? SNR improvement Flat Fielding: improvement in SNR, more important with poor illumination uniformity from optics, higher well capacities Time-sequential multi-image summing (but how to coregister the images in the general case?) Resolution improvement Superresolution: combining multiple shifted timesequential images for resolution improvement Good way to get 2x resolution improvement But need registration algorithm, motion complicates registration task

69 References Photon Transfer: DN -> Lambda, Janesick, J.R. SPIE Press Roadmap for CMOS Image sensors; Moore meets Planck and Sommerfeld ; P. Catrysse, B. Wandell, EI SPIE (2005) QE Reduction in CMOS Image Sensors due to Pixel Vignetting, Catrysse et al. Proceedings of the SPIE vol 3965 (2000) B. Pain: International Image Sensor Workshop 2009 Modern Optical Engineering : W.J. Smith, McGraw-Hill MicrolensPerformance Limits in sub 2 micro-meter CMOS Image Sensors : Huo, Fesenmaier, Catrysse, EI SPIE How Small Should Pixel Size Be?, Chen et al, Proceedings of SPIE vol3965 (2000) Fundamental Performance Differences of CMOS and CCD Sensors Part IV ; Janesick, et al. SPIE 2010 Photon Transfer short course: Janesick, SPIE Electronic Imaging Mpand 39 MpFull Frame CCD Image Sensors with Improved Charge Capacity and Angular Response, Meisenzahl et al, SPIE UCLA Extension CCD & CMOS Image Sensor Short Course notes (2007), Janesick, J., Pain, B., Lomheim, T. CMOS Image Sensor with High Refractive Index Lightpipe, Gambinoet al. IEEE Image Sensor Workshop (2009) BSI-Technical Challenges, Roy et al. IEEE Image Sensor workshop (2009) Image Fusion: A Powerful Tool for Object Identification, F. Lusser, NATO2006

70 Source:

71 Backup

72 Color vsmonochrome Optics Requirements: F#, LP/mm OVT BSI2, Sony Exmor-R Omni BSI 1 Monochrome pixel size (microns) 550 nm test wavelength 550 nm test wavelength Color Airy Optical masked Airy Optimum Diameter resolution pixel size Optimum Diameter F# (microns) : LP/mm (microns) F# (microns) Optical resolution : LP/mm Monochrome Color

73 Source: Pain

74 Source:

Camera Selection Criteria. Richard Crisp May 25, 2011

Camera Selection Criteria. Richard Crisp May 25, 2011 Camera Selection Criteria Richard Crisp rdcrisp@earthlink.net www.narrowbandimaging.com May 25, 2011 Size size considerations Key issues are matching the pixel size to the expected spot size from the optical