Lecture Notes 10 Image Sensor Optics Imaging optics Space-invariant model Space-varying model Pixel optics Transmission Vignetting Microlens EE 392B: Image Sensor Optics 10-1
Image Sensor Optics Microlens Imaging Optics Pixel Optics Image sensor optics consist of (a) imaging optics and (b) pixel optics EE 392B: Image Sensor Optics 10-2
Imaging Optics: Linear Space-Invariant Model Y Y Entrance Pupil X Exit Pupil X Z Object L (W/m 2 sr) Generalized Model (f/#, T, R) Image I (W/m 2 ) A generalized model for rotationally symmetric imaging optics J.W. Goodman, Introduction to Fourier Optics, 2nd Ed, Ch 6, pp. 126-171 (1996) EE 392B: Image Sensor Optics 10-3
Imaging Optics: Camera Equation I ideal ( ) ( ) ( ) ( ) T λ R x, y; λ ( x, y; λ ) π 1+ 4 1+ m f /# 2 2 L scene x m y, ; λ m I ideal : ideal, unaberrated, geometric image irradiance distribution W m L scene 2 ( ) : Lambertian object radiance distribution W sr m 2 ( ) m = z z T R : f : /# i o transmittance : magnification 4 relative illumination factor (e.g., cos fall-off) = f D: f-number Imaging optics map 2D object radiance distribution (W/sr m 2 ) into 2D image irradiance distribution (W/m 2 ) M.V. Klein and T.E. Furtak, Optics,, 2nd Ed, Ch 4, pp. 193-262 (1986) EE 392B: Image Sensor Optics 10-4
Resolution Limited by Diffraction Plane wave Converging wave Plane wave Converging wave Perfect geometric point Diffraction limited point Infinite Aperture Optics Finite Aperture Optics Diffraction, which is caused at aperture edges, is the fundamental reason why a plane wave can not be focussed into a perfect geometric point EE 392B: Image Sensor Optics 10-5
Diffraction: Space Domain Description Image formation process with incoherent illumination can be described in the space domain by a convolution operation I x, y; λ PSF x, y; λ I x, y; λ image ( ) = ( ) ( ) ideal I image : blurred, aberrated, distorted image irradiance distribution I ideal : ideal, unaberrated, geometric image irradiance distribution PSF : point spread function, i.e., response of optical system in image plane to a point excitation in object plane : convolution operator EE 392B: Image Sensor Optics 10-6
Diffraction: Frequency Domain Description Alternatively, image formation in shift-invariant systems can be viewed as a linear filtering process in the frequency domain { image ( )} = { ( )} ideal ( ) = OTF ( f x, f y; λ ) F { Iideal ( x, y; λ )} { } F I x, y; λ F PSF x, y; λ F I x, y; λ 1 (, ; λ ) = F { OTF (, ; λ ) { ( ) } I x y f f F I x, y; λ image x y ideal The optical transfer function (OTF) is normalized to preserve radiometry. Its magnitude is the modulation transfer function (MTF) ( x y ) = ( x y ) = MTF f, f ; λ OTF f, f ; λ F { PSF ( x, y; λ )} { ( )} = PSF x, y; λ f, F 0 f =0 y x EE 392B: Image Sensor Optics 10-7
Example: Diffraction-limited Lens 1 0.9 f/# = 2.0, λ = 486 nm 0.8 0.7 0.6 MTF 0.5 0.4 0.3 0.2 0.1 0 0 200 400 600 800 1000 1200 Spatial frequency (lp/mm) MTF(f r ) = 2 π (ϕ cos ϕ sin ϕ) with ϕ = cos 1 (f r λf/#) and f r,cutoff = 1 λf/# EE 392B: Image Sensor Optics 10-8
System MTF If the system is linear and space-invariant, the system MTF (optics + sensor) in the frequency domain can then be easily computed ( ) = ( ) ( ) MTF diffusion ( f x, f y; λ) MTF f, f ; λ MTF f, f ; λ MTF f, f system x y optics x y geometric x y To obtain the image spectrum we apply the MTF as a linear filter ( ) = ( ) ( ) I f, f ; λ MTF f, f ; λ I f, f ; λ image x y system x y object x y EE 392B: Image Sensor Optics 10-9
Modeling Real Imaging Lenses Vignetting Distortion Illumination Effects Wavefront Aberrations Point Spread Function (PSF) Modulation Transfer Function (MTF) P. Y. Maeda, P. B. Catrysse, and B. A. Wandell, Proc. SPIE Int. Soc. Opt. Eng. 5678, pp. 48-58 (2005) EE 392B: Image Sensor Optics 10-10
Imaging Optics: Linear Space-Varying Model Y Y Entrance Pupil X Exit Pupil Ω k X Z Object L (W/m 2 sr) Generalized Model (f/#, T, R) Image I (W/m 2 ) Image plane is segmented into isoplanatic regions: Ω 1, Ω 2,..., Ω n P. Y. Maeda, P. B. Catrysse, and B. A. Wandell, Proc. SPIE Int. Soc. Opt. Eng. 5678, pp. 48-58 (2005) EE 392B: Image Sensor Optics 10-11
Linear Space-Varying Image Formation Let Ω k, k = 1,..., n be aplanatic image segments, then I x, y; λ PSF x, y; λ I x, y; λ or I image image Ωk ideal, Ω k ( ) = ( ) ( ) 1 ( x, y; λ ) = F OTF Ω ( f, ; ) F, Ω (, ; ) k x f y λ Iideal x y λ k k k { } P. Y. Maeda, P. B. Catrysse, and B. A. Wandell, Proc. SPIE Int. Soc. Opt. Eng. 5678, pp. 48-58 (2005) EE 392B: Image Sensor Optics 10-12
Example: Double Gauss f/2.0 Lens P. Y. Maeda, P. B. Catrysse, and B. A. Wandell, Proc. SPIE Int. Soc. Opt. Eng. 5678, p. 48-58 (2005) EE 392B: Image Sensor Optics 10-13
Double Gauss f/2.0 Lens: MTF P. Y. Maeda, P. B. Catrysse, and B. A. Wandell, Proc. SPIE Int. Soc. Opt. Eng. 5678, p. 48-58 (2005) EE 392B: Image Sensor Optics 10-14
Pixel Optics Optical Efficiency Incident Photons Transmitted Photons Accepted Photons Reflected Photons Rejected or Scattered Photons Quantum Efficiency Collected Electrons Uncollected Electrons P. B. Catrysse and B. A. Wandell, J. Opt. Soc. Am. A 19, pp. 1610-1620 (2002) H. Rhodes et al., IEEE Workshop on Microelectronics and Electron Devices, pp. 7-18 (2004) EE 392B: Image Sensor Optics 10-15
Incident Photons 10 x 10-3 Photometric Exposure (lux-sec) 9 8 7 6 4000 5 4 3 2000 2 1000 1 2 3 4 5 6 7 8 Pixel size (µm) P. B. Catrysse and B. A. Wandell, Proc. SPIE Int. Soc. Opt. Eng. 5678, pp. 1-13 (2005) EE 392B: Image Sensor Optics 10-16
Pixel Transmittance F + (z ) F - (z ) Layer 0 Layer 1 I 01 L 1 Layer j-1 Layer j I j-1,j L j Transmission Layer m Layer m+1 L m I m,m+1 F + (z ) F - (z) F. Abeles, Ann. de Phys. 5, pp. 596-640 (1950) P. B. Catrysse and B. A. Wandell, J. Opt. Soc. Am. A 19, pp. 1610-1620 (2002) EE 392B: Image Sensor Optics 10-17
Example: Transmittance 0.18µm CMOS 1 0.8 Transmittance 0.6 0.4 0.7 µm air Si 3 N 4 0.2 8.3 µm SiO 2 0 400 500 600 700 800 Wavelength (nm) Si Pixel transmittance is λ-dependent (even for dispersion-free materials) P. B. Catrysse and B. A. Wandell, J. Opt. Soc. Am. A 19, pp. 1610-1620 (2002) EE 392B: Image Sensor Optics 10-18
Example: Transmittance 0.18µm CMOS 0.8 0.6 Transmission T 0.4 0.2 Si 3 N 4 SiO 2 Si 0 0 20 40 60 80 100 Angle (degrees) Pixel transmittance (λ-averaged) is approximately independent of angle P. B. Catrysse and B. A. Wandell, J. Opt. Soc. Am. A 19, pp. 1610-1620 (2002) EE 392B: Image Sensor Optics 10-19
Pixel Vignetting No Vignetting Pixel Vignetting Pixel Cross-talk P. B. Catrysse and B. A. Wandell, J. Opt. Soc. Am. A 19, pp. 1610-1620 (2002) EE 392B: Image Sensor Optics 10-20
Pixel Vignetting: Effect of Pixel Height 0.7 0.6 0 degrees 11 degrees 23 degrees 0.5 Pixel response 0.4 0.3 0.2 0.1 0 1 2 3 4 5 6 7 Number of metal layers Reduction in optical efficiency as a function of the number of metal layers in a 0.18µm standard CMOS process (f/1.8 imaging lens) P. B. Catrysse and B. A. Wandell, J. Opt. Soc. Am. A 19, pp. 1610-1620 (2002) EE 392B: Image Sensor Optics 10-21
Pixel Vignetting: Effect of Technology Scaling 0.8 0.7 0 degrees 11 degrees 23 degrees Pixel response 0.6 0.5 0.4 0.3 0.4 0.35 0.3 0.25 0.2 0.15 Feature size (µm) Reduction in optical efficiency for a standard APS pixel with a 30% fill-factor using 2 metal layers as a function of the feature size of CMOS technology P. B. Catrysse and B. A. Wandell, J. Opt. Soc. Am. A 19, pp. 1610-1620 (2002) EE 392B: Image Sensor Optics 10-22
Optical Efficiency Definition: Optical efficiency (OE) is the ratio of the photons incident of the substrate and the photons incident on the pixel surface Sources of photon loss: Back-reflections in dielectric stack (air-sio 2, SiO 2 -Si) Photons absorbed in dielectric stack (SiON) Pixel transmittance T(λ,θ) Photons scattered away from pixel (pixel cross-talk) Photons rejected by metal Pixel vignetting V(x,y,θ) Description: OE(x,y,λ,θ) = T(λ,θ) V(x,y,θ) EE 392B: Image Sensor Optics 10-23
Microlens Focus light onto photo-sensitive region increases effective fill factor from 25-40% to 60-80% (and sensitivity by 2X) Less effective if photosensitive area is irregularly shaped A. Theuwissen, Solid State Imaging with Charge-Coupled Devices, Kluwer (1995) S.G. Wuu et al., High performance 0.25 µm CMOS color imager technology with non-silicide source/drain pixel, IEDM Tech. Dig., pp. 705-708 (2000) EE 392B: Image Sensor Optics 10-24
Lens material requirements: Microlens Fabrication Highly transparent in the visible light region Index of refraction > 1.59 Can be applied below 500C No degradation or aging Semiconductor processing compatible Can be patterned with feature size commensurate with the pixel size Lens materials are typically i-line or DUV resists Base materials are acrylic-based resists, polyimide resists, epoxy resists, polyorganosiloxane, polyorganosilicate EE 392B: Image Sensor Optics 10-25
Microlens and the Main Lens Microlens is optimized for a specific main lens system Rays incident on the microlens form a cone with NA = sin ϕ NA varies as a function of the size and position of the exit pupil Principle ray at the periphery of the sensor has an angle δ, chief ray angle (CRA), between the ray and the optical axis (δ depends on the position of the pixel on the sensor) A. Theuwissen, Solid State Imaging with Charge-Coupled Devices, Kluwer (1995) EE 392B: Image Sensor Optics 10-26
Microlens and F-Number High F-number: rays are parallel (NA 0) Low F-number: rays arrive at an angle (NA large) microlens effectiveness low A. Theuwissen, Solid State Imaging with Charge-Coupled Devices, Kluwer (1995) EE 392B: Image Sensor Optics 10-27
Micro lens: How to concentrate photons Micro-lens Metal Wires Photodiode Shallow Pixel Deep Pixel EE 392B: Image Sensor Optics 10-28
Micro lens: How to concentrate photons D µ-lens D µ-lens f µ-lens f' µ-lens Shallow Pixel Deep Pixel EE 392B: Image Sensor Optics 10-29
Micro lens: How to concentrate photons 2NA 2NA' Shallow Pixel Deep Pixel NA = 1 1+4(f/#) 2 NA = 1 1+4(f/# ) 2 EE 392B: Image Sensor Optics 10-30
Micro lens: How to concentrate photons 2.44 f/# λ 2.44 f/#' λ Shallow Pixel Deep Pixel f/# = f µlens D µlens f/# = f µlens D µlens EE 392B: Image Sensor Optics 10-31
Micro lens: How to concentrate photons Conservation of etendue G = 2NA imaging w pixel = 2NA µlens w diode Etendue limits light collection efficiency (NA) Bigger NA allows bigger etendue Concentration depends on ratio of NA of microlens and imaging lens For a 2x space concentration (w diode = w pixel /2) 2NA µlens > NA imaging Diffraction limits spot size (f-number) Smaller f-number allows smaller spot size EE 392B: Image Sensor Optics 10-32
Micro lens: Concentration (a) (b) 10 f/4.0 f/2.8 64 f/4.0 f/2.0 Concentration 36 f/2.8 16 f/2.0 4 f/1.4 10 0 8 6 Pixel size (µm) 4 2 2 4 6 8 Pixel height (µm) 10 Pixel height (µm) 8 6 f/1.4 4 2 2 4 6 8 10 Pixel size (µm) P. B. Catrysse and B. A. Wandell, Proc. SPIE Int. Soc. Opt. Eng. 5678, p. 1-13 (2005) EE 392B: Image Sensor Optics 10-33
Pixel Position For large CRA (pixel away from the center of the lens, or exit pupil close to the sensor), ray may not focus on the photo-sensitive region Effect: non-uniform sensitivity profile across image sensor A. Theuwissen, Solid State Imaging with Charge-Coupled Devices, Kluwer (1995) EE 392B: Image Sensor Optics 10-34
Micro lens: Redirection (w/o offset) 1 0.8 Pixel width: 3µm Pixel Height: 8 µm f/1.4 f/2.0 f/2.8 f/4.0 0.6 OE 0.4 0.2 0 0 5 10 15 20 θ (deg) CR P. B. Catrysse and B. A. Wandell, Proc. SPIE Int. Soc. Opt. Eng. 5678, p. 1-13 (2005) EE 392B: Image Sensor Optics 10-35
Micro lens: Redirection (with offset) Pixel width: 3µm Pixel Height: 8 µm 1 f/4.0 f/2.8 0.8 Optical Efficiency 0.6 0.4 = f ml tan(θ) f/2.0 f/1.4 0.2 f ml θ 0 0 5 10 15 20 θ (deg) P. B. Catrysse and B. A. Wandell, Proc. SPIE Int. Soc. Opt. Eng. 5678, p. 1-13 (2005) EE 392B: Image Sensor Optics 10-36
Optical Efficiency: Summary Without microlens: OE(x,y,λ,θ) = T(λ,θ) V(x,y,θ) With microlens: OE(x,y,λ,θ) = T(λ,θ) V(x,y,θ) ML(x,y,θ) where ML(x,y,θ) represents a correction factor to account for the concentration and/or redirection performed by the microlens EE 392B: Image Sensor Optics 10-37
Improving OE Optimizing dielectric stack thickness Make sure dielectrics are not light absorbing Utilize different dielectric materials to achieve total internal reflection Add an airgap between pixels (total internal reflection) EE 392B: Image Sensor Optics 10-38