Photons and solid state detection Photons represent discrete packets ( quanta ) of optical energy Energy is hc/! (h: Planck s constant, c: speed of light,! : wavelength) For solid state detection, photons are converted to electrons Electrons can be measured as a current (flow through resistor) or as a voltage (accumulation in capacitor) Detection can be done at a single point (point = small area over which all detected photons are summed up) e.g. photodiode, photomultiplier tubes (PMT) Multiple point detectors can be arrayed to detect an image eg. complementary metal oxide semiconductor (CMOS) or charge coupled device (CCD) sensors. Recall shot noise: photon production is inherently noisy with a standard deviation proportional to the square root of intensity
Quantum efficiency and spectral sensitivity Photodiode responsivities. www.thorlabs.com Quantum efficiency = number of electrons generated / incident photon Measured as a unit less fraction or in ampere/watt (called responsivity) QE and responsivity vary with the wavelength of light
Photoelectric effect Valence band (VB) is the energy band with bound electrons Conduction band (CB) is the energy band with free electrons Difference is the energy band gap Photoelectric effect describes how electrons can be kicked out of the valence band on supplying energy E g will move electrons to the CB. Eventually they will fall back Ionization energy (IE) will eject an electron from the material. This electron is lost
Photomultiplier Tube Photoelectric effect dictates ejection of an electron from the photocathode Photoelectrons focused and made to strike an electrode (called dynode) Each dynode is at a +ve potential (10s-100s volts) above the previous one The electric field between dynodes accelerates electrons as they strike subsequent dynodes Secondary emission causes more electrons to be emitted than the number that struck a dynode causing a multiplicative gain (up to 10 8 ) The anode collects all electrons and leads to a current
Photodiode pn junction Pure semiconductor is an insulator but can be doped to p type (fixed negative charges, free holes) or n type (fixed positive charges, free electrons) A hole is not a real thing, it is the lack of an electron A depletion region forms with no free charges. There exists an electric field across this region. Region can be expanded by reverse biasing (n region voltage>p region voltage by a few volts)
Photodiode operation Photoelectric effect dictates generation of an electron-hole pair (EHP) In the p or the n region the EHP disappears due to recombination In or near the depletion region, the photogenerated free carriers come under the influence of the electric field and causes a current The fact that useful EHP are generated only in a specific region plays a role in determining QE and spectral sensitivity Photocurrent is linearly proportional to the optical power incident
Dark current and read noise Dark current is the output of a photodetector is the absence of any light. It is not a constant offset that can be rejected but is also noisy Caused by thermal vibrations leading to EHP creation (PD) or electron ejection (PMT) Limits the lowest light levels that can be detected Read noise is the excess electrical noise (apart from photon shot noise) that appears at the output of a photodetector Independent of optical power, so dominates at low light intensities Dark current (and dark current noise) and read noise can be reduced by cooling the detector
Charge coupled device imager Photoelectrons collected in potential wells A well is created if the gate voltage is high By using a cyclic sequence of voltages the collected photoelectrons are moved
Charge coupled device imager Photoelectrons of each pixel are moved horizontally (within the same row) into a series of output wells The charge is then transferred vertically A charge-to-voltage amplifier converts the photoelectron charge into a voltage Key ideas The readout is serial While the array is being read, it MUST be protected from light Charge Transfer Efficiency is a critical parameter and measures how much chargeislostpertransfer.itisdefinedas: (charge after transfer)/(charge before transfer)
Complementary metal oxide semiconductor imager Column selection Passive pixel array Any pixel can be connected to the charge-to-voltage amplifier using a row and column select mechanism Row selection Switches implemented using MOS transistors Because of the long metal lines between the pixels and the amplifier, this does not work very well Charge to voltage amplifier Output
Complementary metal oxide semiconductor imager Column selection Active pixel array Each pixel has its own chargeto-voltage amplifier Row selection Amps and switches implemented by MOS transistors Random access scheme is retained Key feature: processing circuits can be implemented at the pixel or at the column level Key feature: can read one pixel while others are collecting light Output
Array imager features Resolution: number of x and y pixels More lead to more detail, but for a given array size more means each pixel is smaller -> smaller photodiode -> poorer sensitivity Pixel pitch: size of one pixel. Down to 1-2 µm using latest fabrication technology Array size (format): size of the array in units of length. Must match image size Fixed pattern noise: spatial noise that is fixed in time. Due to fabrication related differences between individual pixels. Worse in CMOS because each pixel has its own circuitry Fill factor: ratio of area of photodiode to area of pixel Circuits (switches, amps) take up valuable photodiode space Corrective measures Microlenses: per pixel lens increasing light gathering Back-illuminated sensors Frame rate: images per second < 1/(readout time + exposure time) Intensity resolution (how fine a difference) & dynamic range (maximum/minimum)
Pixel pitch & sampling requirements Diffraction sets a resolution limit System magnification maps that resolution limit on to an image sensor Pixel pitch needs to be fine enough for adequate sampling Analogous to Nyquist sampling theorem if a signal has a maximum frequency content of B Hz, it must be sampled at > 2B Hz for reconstruction 200 nm apart (object) 20 µm apart (image) 100x magnification Pixel pitch >20 µm >10 µm <10 µm > : just greater than < : just lesser than Pixel pitch at least 2x smaller than optical resolution limit (Rayleigh limit)
Digital image representation An image is a two-dimensional array a[x,y] x,y indicate the position of the pixel within an image a[x,y] is the intensity of that pixel. 1 white, 0 black [ 0 1 ] 1 0 For an NxM image x: 1, 2,. N y: 1, 2,. M
11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 11111111111111111111100000000000000000000000000000000000000000000000000000000000000000000000000000000000000111111111111111111111 11111111111111111111100000000000000000000000000000000000000000000000000000000000000000000000000000000000000111111111111111111111 11111111111111111111100000000000000000000000000000000000000000000000000000000000000000000000000000000000000111111111111111111111 11111111111111111111100000000000000000000000000000000000000000000000000000000000000000000000000000000000000111111111111111111111 11111111111111111111100000000001011110001111101000000000000000000000000000000000001011110001111010000000000111111111111111111111 11111111111111111111100000000001111111011111111000000000000001111111000000000000001111111011111111000000000111111111111111111111 11111111111111111111100000010001111111011111111000000000000011000001100000000010001111111011111111000000000111111111111111111111 11111111111111111111100000010001111111001111111000000000000011111111100000000010001111111001111111000000000111111111111111111111 11111111111111111111100000010001111111011111111000000000011111111111111100000010001111111011111111000000000111111111111111111111 11111111111111111111100000010001111111011111111000000000101111111111111100000010001111111011111111000000000111111111111111111111 11111111111111111111100000010001111111001111111000000000111110111111111110000010001111111001111111000000000111111111111111111111 11111111111111111111100000010001111111011111111000000001111111011101111110000010001111111011111111000000000111111111111111111111 11111111111111111111100000010001111111011111111000000001111111001011111110000010001111111011111111000000000111111111111111111111 11111111111111111111100000010001111111001111111000000001011111011101111011000010001111111001111111000000000111111111111111111111 11111111111111111111100000010001111111011111111000000001111110011001111011000010001111111011111111000000000111111111111111111111 11111111111111111111100000010001111111011111111000000001111100110110111111000010001111111011111111000000000111111111111111111111 11111111111111111111100000010001111111001111111000000000111111100111111110000010001111111011111111000000000111111111111111111111 11111111111111111111100000010001111111011111111000000000011111110111111000000010001111111011111111000000000111111111111111111111 11111111111111111111100000010001111000000001110000000000111111110111111100000010001111000000001110000000000111111111111111111111 11111111111111111111100000010000000010001000000000000000011011111111101100000010000000010001000000000000000111111111111111111111 11111111111111111111100000010011111110001111111000000000001101111110111000000010011111110001111111000000000111111111111111111111 11111111111111111111100000010000000000000000000000000000000111111101110000000010000000000000000000000000000111111111111111111111 11111111111111111111100000010000000000000000000000000000000011100111100000000010000000000000000000001000000111111111111111111111 11111111111111111111100000000000000000000000000000000000000001000000000000000000000000000000000000000000000111111111111111111111 11111111111111111111100000000000000000000000000000000000000000000000000000000000000000000000000000000000000111111111111111111111 11111111111111111111100000000000000000000000000000000000000000000000000000000000000000000000000000000000000111111111111111111111 11111111111111111111100000000000000000000000000000000000000000000000000000000000000000000000000000000000000111111111111111111111 11111111111111111111100000000000000000000000000000000000000000000000000000000000000000000000000000000000000111111111111111111111 11111111111111111111100000000000000000000000000000001111111111111111111111111000000000000000000000000000000111111111111111111111 11111111111111111111100000000000000000000000001111111111111111111111111111111111111000000000000000000000000111111111111111111111 11111111111111111111100000000000000000000011111111111111111111111111111111111111111111100000000000000000000111111111111111111111 11111111111111111111100000000000000000111001111111111111111111111111111111111111111111100100000000000000000111111111111111111111 11111111111111111111100000000000000111111000111111111111111111111111111111111111111111000111100000000000000111111111111111111111 11111111111111111111100000000000111111111000011111111111111111111111111111111111111110000111111100000000000111111111111111111111 11111111111111111111100000000011111111111000001111111111111111111111111111111111111100001111111111000000000111111111111111111111 11111111111111111111100000001111111111111000000111111111111111111111111111111111111000001111111111110000000111111111111111111111 11111111111111111111100000111111111111111100000011111111111111111111111111111111110000001111111111111100000111111111111111111111 11111111111111111111100001111111111111111100000001111111111100000000011111111111100000011111111111111111000111111111111111111111 11111111111111111111100111111100001111111110000000011111110000000000001111111110000000011111111000111111100111111111111111111111 11111111111111111111100111111000001111111111000000000111100000000000000111111000000000111111111000011111100111111111111111111111 11111111111111111111100111111000011111111111000000000000000000100001000001000000000001111111111000011111100111111111111111111111 11111111111111111111100111111100001111111111100000000000000110100101110000000000000011111111110000011111100111111111111111111111 11111111111111111111100111111110000111111111110000000000000111101111100000000000000111111111110001111111100111111111111111111111 11111111111111111111100111111111000111111111111000000000000000011000000000000000001111111001100001111111100111111111111111111111 11111111111111111111100111111111000011001110000000000000000000000000000000000000000000000001100011111111100111111111111111111111 11111111111111111111100111111111000011000000000000000000100000000000010010000000000000000011000001111111100111111111111111111111 11111111111111111111100111111110000001100000000000000000011001000000001001000000000000000011000001111111100111111111111111111111 11111111111111111111100111111110000001110000000000000000011001000001001101000000000000000010000001111111100111111111111111111111 11111111111111111111100111111100001000110000000000000000100101101001101111000000000000000100010000111111100111111111111111111111 11111111111111111111100111111100000100011000000000010000000110101001111011001000000000011100010000011111100111111111111111111111 11111111111111111111100111111000000100011100000000010000000100110111010000001000000000111000100000011111100111111111111111111111 11111111111111111111100111110000000010001111000000001000000000110011000000010000000011111000100000001111100111111111111111111111 11111111111111111111100111110000000010001111100001100111000000100000000001100111000011110001000000000111100111111111111111111111 11111111111111111111100111100000000001000111110011100011111100000000001111000111101111110001000000000111100111111111111111111111 11111111111111111111110111000000000000000111111111000010000111111111110001000011111111100010000000000011100111111111111111111111 11111111111111111111110011000000000000100011111111000010001010000000100001000011111111000100000000000011101111111111111111111111 11111111111111001111110010000000000000000001111111000001111010000000111111000011111111000000000000000001101111100011111111111111 11111111111110000011110010000000000000000001111111000000000010000001000000000011111110000000000000000001001111000001111111111111 11111111111101100001110010000000000000000000111111100000000010000001000000000111111110000000000000000001001110000100111111111111 11111111111001100001110010000000000000000000111111100000000010000001000000000111111100000000000000000001001100000110011111111111 11111111110011100000110010000000000000000000011111110000000010000000100000000111111100000000000000000001001100000111001111111111 11111111100111100000111000000000000000000000011111110000000010000000100000001111111000000000000000000001011100000111100111111111 11111111001111100000011001000000000000000010001111111000000100000000100000001111110000000000000000000001011000000111110011111111 11111100011111100000011001000000000000000010000111111000000111000001110000011111110001000000000000000010011000000111111001111111 11110001111111100000011001100000000000000111000111111100000100111110010000111111100011100000000000000010011000000111111110001111 10000111111111110000011100100000000000000111000011111110001000000000010001111111100011100000000000000110010000001111111111100011 10111111111111110000001100110000000000001111100011111110000011000011010001111111000111110000000000001100110000001110111111111101 10111111111111110000001100111000000000001111110001111111000010100100010011111111000111110000000000011100110000001100111001111101 10111111111100110000001110011100000000001111110001111111001000100010010011111110001111110000000000111100110000011101110000111101 10111111111100111000001110011110000000000111111000111111001001000010001011111100001111100000000001111001110000011101100110111101 10011111100000111000001110011111000000000011111000011111101000000000001011111100011111100000000001111001110000011101001110111001 11011100011001111000001111001110000000000001111100011111101000111000001011111000111111000000000001111011110000111100001110111011 11011101110011111100001111001110000000000000111110001111100111110011110011111000111110000000000001110011110000111111111000111011 11001111100111111100001111100110000000000000011110001111100000000000000011110001111000000000000001110011110000111111111001110011 11001110001111111110001111100110000000000000000111000111100000000000000111100001100000000000000001100111110001111111111111110111 11101111111110001110001111100011000000000000000001000111110000000000001111100010000000000000000001100111110001110000111011110111 11100111111100000110011111110011000000000000000000100011111100111111111111000010000000000000000011001111110011100000000001100111 11100111000001110111011111111001100000000000000000100001111111111111111111000100000000000000000011001111110011100011011111101111 11110111001111111111001111111001100000000000000000110001111111111111111110001100000000000000000110011111110011111101011111101111 11110011101111111111101111111100110000000000000000111000111111111111111110001100000000000000001110011111110111111110111111001111 11111011111100000001100111111100111000000000000000111000111111111111111100011100000000000000011100111111100111111111000111011111 11111011111001111101110111111110011110000000000000111100011111111111111000011110000000000000111001111111101110011111100111011111 11111001111001111101110011111110001111000000000001111100011111111111111000111110000000000011111001111111011110000111101110111111 11111101111011111101111011111111001111110000000001111110001111111111110001111110000000000111110011111110011111110001111110111111 11111100111011111001111101111111100111111000000001111110000111111111110001111111000000001111100111111110111001111000011101111111 11111110111001111011111100111111110011111000000011111111000111111111100011111111000000011111000111111100111001111101111101111111 11111110011100000011111110111111110001111100000011111111100011111111100011111111100000011111001111111001111010111111111011111111 11111111011111111111111111011111111000111100000011111111100011111111000111111111100000111110011111111011111110011100111011111111 11111111001111111111111111001111111100111110000111111111110001111110000111111111100000111100111111110011101111001001110111111111 11111111101111111111111111100111111110011110000111111111110000111110001111111111100001111001111111100111001111100011110111111111 11111111100111111100111111110111111111001111000011111111111000111100011111111111100001110011111111001110011100111111100111111111 11111111100011111111111111111011111111100111100011111111111100011100011111111111000011100111111111011110111000011111000111111111 11111111100011111111111111111001111111110011110001111111111100011000111111111111001111001111111110111110110011101111000111111111 11111111100001111111110000111100111111111001111111111111111110001000111111111111111110011111111100111111000111101110000111111111 11111111100001111111110110011110011111111100011111111111111110000001111111111111111100111111111001111111111111011100000111111111 11111111100000111110110011011111001111111110001111111111111111000001111111111111111001111111110011111111111000011100000111111111 11111111100000011100110011011111100111111111000111111111111111000011111111111111100011111111100111111111111001111000000111111111 11111111100000001100110011111111110011111111110011111111111111100111111111111111000111111111001110011110111111110000000111111111 11111111110000001110111011110011111001111111111000111111111111100111111111111100011111111110011110011111111111110000001111111111 11111111110000000110010011110001111100111111111100011111111111111111111111111000111111111100111110001111111111100000001111111111 11111111110000000011100111100111111110011111111111000111111111111111111111100011111111111001111110010111111111000000011111111111 11111111111000001001111111001111111111101111111111110001111111111111111110000111111111100111111111011011111110010000011111111111 11111111111100000100111110010111001111110011111111111000011111111111111000011111111111001111000111001100111100100000111111111111 11111111111100000110111000111011000111111001111111111110000111111111100001111111111110011110000001101100111101100000111111111111 11111111111110000011011101111100011111111100011111111111100001111110000111111111111001111111110110000111111011000001111111111111 11111111111111000001101111111100111100101111001111111111111000000000011111111111110011111111111011110111110110000011111111111111 11111111111111110000100111111001111100010111100011111111111111000011111111111111001111100011111001111111101000001111111111111111 11111111111111111110011011100111111001100111111000111111111111111111111111111100011111100001111110011111011001111111111111111111 11111111111111111111111101110011100111110011111110011111111111111111111111110001111111110100011110111110111111111111111111111111 11111111111111111111111110111111001111100111111111100011111111111111111111000111111011111010000111111101111111111111111111111111 11111111111111111111111111011111011111001110011111111000111111111111111100011111100011111000100011111011111111111111111111111111 11111111111111111111111111100111001111011110001111111110001111111111100001111111000111111001111111100111111111111111111111111111 11111111111111111111111111110011000100011110111111111111110001111100001111111111011110111011111111001111111111111111111111111111 11111111111111111111111111111001110100111100111100111111111110000001111111111110111000011100111110011111111111111111111111111111 11111111111111111111111111111100111100111101111000011111111111111111111110001110011011011001111101111111111111111111111111111111 11111111111111111111111111111111001111111001111011111111111111111111111100000011011111000111110011111111111111111111111111111111 11111111111111111111111111111111100111100011110011111100011111111111000101111001001110011111100111111111111111111111111111111111 11111111111111111111111111111111111001110011110111111110000111111000001101111101100000111110011111111111111111111111111111111111 11111111111111111111111111111111111100111111100111011100111111111000011101111101110001111100111111111111111111111111111111111111 11111111111111111111111111111111111111001111001111001100111111111110011101111101111111110011111111111111111111111111111111111111 11111111111111111111111111111111111111100011100011011000001111111111011100111101111111000111111111111111111111111111111111111111 11111111111111111111111111111111111111111001111000111011111111111111001110000011111100011111111111111111111111111111111111111111 11111111111111111111111111111111111111111110011111100011111111101111101111000111110001111111111111111111111111111111111111111111 11111111111111111111111111111111111111111111100011110001111111101111100111111111000111111111111111111111111111111111111111111111 11111111111111111111111111111111111111111111111000111100011111111111100111111100011111111111111111111111111111111111111111111111 11111111111111111111111111111111111111111111111110000111111111111111111111100011111111111111111111111111111111111111111111111111 11111111111111111111111111111111111111111111111111110001111111111111111100001111111111111111111111111111111111111111111111111111 11111111111111111111111111111111111111111111111111111110000111111111100001111111111111111111111111111111111111111111111111111111 11111111111111111111111111111111111111111111111111111111110000011000001111111111111111111111111111111111111111111111111111111111 11111111111111111111111111111111111111111111111111111111111111000011111111111111111111111111111111111111111111111111111111111111 11111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111111 Digital image representation
Spatial resolution Larger pixels lead to lesser detail because a pixel is the smallest unit of an image. It can hold only one intensity.
Intensity resolution Light intensity is continuous (red), but a digital image can only store discrete values (blue) Due to the binary system used in computers, the number of distinct levels in an N-bit image is 2 N Almost all computer displays are 8-bit which means they can show 256 distinct shades between black (gray level 0) and white (gray level 255) A good imager will have >8 bits, however most common image formats JPG, BMP, PNG are only 8 bit leading to potential loss of information TIFF image format can store up to 16 bits so is preferred Why are more bits useful? 8 bit 4 bit 8 bit 4 bit
Saturation and dynamic range Light intensity Saturation level. Any light intensity higher that this can not be differentiated from this level. level 2 N -1 level 2 N -2 level 2 N -3 level 2 level 1 level 0 Digital pixel levels Dynamic range (DR) is the ratio of the maximum to minimum light that can be detected ie (2 N -1)/1 May be smaller due to noise 2 N -1/(noise floor) Usually expressed in decibel (db) 20 log 10 (2 N -1)
Color image representation Split into three color channels red, green and blue Each channel can be thought of as a grayscale image
Sensing color Sense 1 color channel at a time with a grayscale/monochrome imager and combine images in software. Limitations? Bayer array: color filter array on top of each individual pixel color filters Why twice as many green? Needs interpolation to form image pixels Foveon sensor: uses 3D photodiodes and exploits the dependence of light penetration on wavelength Sensitivity, particularly of deeper structures, suffers blue diode green diode red diode http://www.foveon.com/article.php?a=67
Intensity scaling b For an N-bit imager 255 a! 0 b " 2 N -1 b a >256 levels in image 0 256 levels for display a <256 levels in image Note <256 levels does not neccesarily indicate <8-bit imager. Consider a 12-bit imager (light dependent levels from 0 to 4095) imaging in very low light. It is possible that the brightest pixel only goes up to <256
Contrast Contrast is the percieved difference between various intensity levels in an image Consider the same scene imaged using two cameras. Both have 65 levels. Neither image is scaled Intensity scaling enhances contrast allowing visualization of fine details Given images c, d can you tell which came from which camera? a: Levels 0-64 b: Levels 150-214 IMPORTANT to be aware of scaling before comparing images c: a scaled to 0-255 d: b scaled to 0-255
Temporal noise and SNR Observe the intensity of 1 particular pixel over multiple images Low noise High noise Both have same mean Pixel level Frame number Multiple images of the same scene Signal to noise ratio = mean(pixel level) / standard deviation(pixel level) SNR (in db) = 20 log 10 (SNR) What happens to SNR with increasing light intensity?
Imaging parameters Light available: number of photons available per unit time Exposure time: duration for which photons are collected for 1 image Quantum efficiency: electrons generated per photon Amplifier gain: electron-to-voltage or electron-to-current conversion factor How do we optimise SNR? Maximize collected photons signal # number(photons) shot noise # [number(photons)] 0.5 Minimize read noise
Photon Transfer Curve Noise in imagers read noise, shot noise, fixed pattern noise FPN can be eliminated. How? Read noise independent of light intensity Shot noise dependent on light intensity Output signal of imager dependent on light intensity PTC is a log-log plot of signal vs. noise log(noise) Three regions read noise shot noise saturation log(signal)