Global Shutter CMOS Image Sensor With Wide Dynamic Range

Transcription

1 PAPER IDENIFICAION NUMBER Global Shutter CMOS Image Sensor i ide Dynamic Range Alexander Beleny, Alexander Fish, Member IEEE, Arur Spiva and Orly Yadid-Pecht, Fellow IEEE Abstract A novel concept for global shutter CMOS image sensors wi ide Dynamic Range (DR) implementation is presented. he proposed imager is based on e multisampling DR approach and it allows an efficient global shutter pixel implementation achieving small pixel size and high fill factor. he proposed imager provides wide DR by applying adaptive exposure time to each pixel, according to e local illumination intensity level. wo pixel configurations, employing different inds of a -bit in-pixel memory were implemented. An imager, including two different pixels was designed and simulated in 0.8µm CMOS technology. System architecture and operation are discussed and simulation results are presented. Index erms Active Pixel Sensor (APS), ide Dynamic Range (DR), Global Shutter Imager. I. RODUCION Driven by e demand for low-power dissipation in state-of-eart portable image systems, CMOS imagers became very attractive. CMOS imagers offer significant advantages in terms of low-power, low-voltage and monoliic integration, rivaling traditional CCDs []-[4]. Sensor Dynamic Range (DR) is one of e most important figures of merit in state of e art CMOS image sensors. he DR problem exists in cases where e sensor should capture scenes having a wide range of illumination. Bright scenes and wide variations in intrascene illumination can arise in many situations: driving at night, photographing people in front of a window, observing an aircraft landing at night, and imaging objects for studies in meteorology or astronomy. Generally, dynamic range can be increased in two ways: e first one is noise reduction and us expanding e dynamic range toward darer scenes. he second meod is incident light uration level expansion, us improving e dynamic range toward brighter scenes. In is paper, e last approach is discussed. A narrow DR of image sensors entails uration of a pixel wi high sensitivity, in a case of high illumination levels, and part of e information can be lost. DR insufficiency of conventional video cameras is a serious problem in realizing a robust vision system for taing images consisting of varying illumination conditions in e same scene. Different solutions for widening e DR in CMOS image sensors have been presented in recent years [5]-[6]. Manuscript received March 6, 007. A. Beleny is wi e LSI Systems Center, POB 653, Beer-Sheva 8405, Israel ( beleny@ee.bgu.ac.il). A. Fish is wi wi Dept of Electrical and Computer Engineering, University of Calgary, Alberta, Canada NN4.( fish@atips.ca). A. Spiva is wi e LSI Systems Center, POB 653, Beer-Sheva 8405, Israel ( spivaar@bgu.ac.il). O. Yadid-Pecht is wi LSI Systems Center, POB 653, Beer-Sheva 8405, Israel (phone: ; fax: ; oyp@ee.bgu.ac.il). his paper presents a novel implementation of a wide DR (DR) CMOS image sensor. he proposed imager operates bo in global and rolling shutter modes of operation and achieves very high dynamic range expansion of up to 0dB. his implementation is a continuation of our previous wor, where we have implemented an ultra low-power DR sensor, operating in e global shutter mode []. However, a number of significant improvements differ is newly proposed sensor architecture from our previous solution. he main advantage of e presented imager is a simpler design, increased fill factor, lower FPN and erefore better image quality and improved spatial resolution. All ese improvements were allowed by implementation of a -bit in-pixel memory and performing most of e required processing for DR expansion in e array periphery, while still allowing global shutter operation. wo pixel configurations, employing different inds of a -bit in pixel memory are shown here. Section II describes e system architecture, system building blocs and briefly presents e DR algorim. Section III depicts e pixel structure implementations and e general principle of operation. Section I discusses simulation results. Conclusions and future research are outlined in Section. II. SYSEM ARCHIECURE AND DR ALGORIHM DESCRIPION Figure shows e general architecture of e proposed imager using e designed prototype photograph as a template. he sensor consists of a pixel array, one row (vertical) decoder, two column (horizontal) decoders, column readout circuits, processing circuits and digital memory. In order to share e processing circuits among e pixels in a column, e design maes use of a column parallel architecture. In is architecture, e pixel array, e memory array, and e processing elements are separated. Each pixel contains a - bit memory cell at allows e possibility of independent reset of each pixel. i is, e adjustability of integration time can be performed for each pixel, and nondestructive readout of e pixel can be performed at any time during e integration period. he processing element contains e uration detection circuitry at is shared by all pixels in a column. Because of is column parallel architecture, e pixel array contains a minimum amount of additional circuitry and ere is a little sacrifice in fill factor in comparison wi previous shot designs. A. DR Algorim Description he design of e imager is based on our previously proposed DR algorim [0], which was adapted here to allow an efficient global shutter sensor implementation. According to our algorim, e required expansion of e DR is determined by a series of bits. he total integration time is subdivided into several integration times, which are progressively shorter, according to e down-going series:,,..., ()

2 PAPER IDENIFICAION NUMBER Figure. General architecture of e proposed wide DR global shutter APS. where > and represents e full integration time. At e beginning of e frame, all pixels in e imager are reset simultaneously to ensure -shot operation of e imager. hen e photodiode output of each pixel in row is compared wi an appropriate reshold, at certain time points given by:,..., Δ t () where is a short time associated wi row and given as: ( N + ) decision = (3) where decision is e decision time and will be described later on. he comparison is performed by enabling e column shared comparator wi constant reshold value to all pixels in e array, in a rowby row manner, i.e. each comparator wi a single pixel in a row, applied to all rows, one at a time. he comparison checs wheer each pixel in e specific row is going to be urated at e next integration slot. his binary information is saved locally in e -bit in-pixel memory, and is transmitted to e external digital storage in e upper part of e sensor, associated wi each pixel. hen, if any of e checs determines at e pixel will urate at e end of e current integration time, e pixel is reset again and is allowed to start integrate light again, but for a shorter period of time. Note at is operation is applied simultaneously to all pixels in e array to guarantee shot operation. his operation enables proper scaling of e value being read out and enables e pixel value to be described in a floating-point representation (4). his way, e actual pixel value would be: alue = Man Man EP = where alue is e actual pixel value, Man (Mantissa) is e analog or digitized output value at has been read out at e time point, EP is e exponent value, at is stored in e digital memory bloc and describes e scaling factor, i.e. which part of e integration time is actually effective. he exponent value is retrieved from a digital memory also at e end of overall integration time. EP (4) B. hreshold oltage Considerations he overall idea of e algorim is to avoid e effect of pixel uration. As previously mentioned, e algorim checs a reshold point at first sub-integration period and taes a decision based on e anticipation at during e whole integration period e pixel will not be urated. herefore, assuming at Δ t = 0,, meaning at e pixel value was first compared at e intrinsic (eoretical) reshold value should be chosen in such a way so at a straight line (dash-doted line number in Figure, at describes discharging of e pixel during, rough reset (photodiode reset voltage) at t=0 and reshold voltage at e first sub-integration period will not cross e pixel uration voltage before e whole integration time is due. he equation of is straight line is given by: () = (5) t reset t is a maximum pixel voltage swing at is e difference between and reset values. o find e value of e intrinsic reshold voltage, e ( ) is substituted into e line equation, resulting in e following formula: + (6) In real designs each comparator has its own offset voltage. herefore, for two different comparators having two different offset voltages e comparison will be performed at different points even if e same reshold voltage was set. Figure shows an example of two pixels, discharging by e same illumination level and being processed using two comparators having different offset values. he same was applied for bo cases. As can be seen, ere is an immunity to change in comparator offsets, since in bo cases e final results are e same. In e first case (shown by e solid line number ), e pixel value didn't pass e reshold voltage and erefore it was not reset at e first chec. In e second case (shown by e dashes line number 3), e pixel value did pass e reshold voltage + and erefore it was reset at e first chec. offset However, e final results (given by equation (4)) remain similar for bo cases. Note, in e second case e SNR of e pixel is reduced since e integration time was reduced in is case. + reset offset Figure. hreshold voltage derivations As mentioned, due to mismatches in e fabrication process each comparator has its own offset voltage. herefore, is offset should

3 PAPER IDENIFICAION NUMBER be taen into account when e reshold voltage is calculated. he reshold voltage is now given by: = + (7) offset In e present shot DR algorim implementation ere is a certain delay between comparison times of different rows ( 0 and erefore e difference in time between row and row i is given by ( i ) - see equation (3)) since e comparison is done in row by row manner, but e final result is not affected. he reason for is immunity to comparison time differences can be explained by e fact at e difference in comparison time is equivalent to e differences in comparator offsets. For example, if pixel values in row i are compared to at and pixel values in i row are compared to at using e same comparator, is is equivalent to e case when pixels in bo rows are compared at e same time, but using different comparators having different offsets voltages and, resulting in different offset offset _ ( and ) for each row. _ he following explanations are wi reference to Figure 3 and assuming Δ t. he reshold voltage for e pixel >> decision _ array, operated in shot mode, is chosen to be higher an in order not to allow uration of pixel A in e first row and pixel B in e last row during one uration detection chec Δ t N. ( ) is e total time required for e uration detection N process for all rows in an N by M array, where N is number of rows and M is number of columns (see Figure 3). By substituting value of from equation (7) into (8) = _ e get e _ value: _ ( ) where is given by (7) and is defined in (3). (8) (9) ( ) I. PIEL DESCRIPION AND OPERAION (0) his sub-section describes e proposed pixel and explains e general principle of its operation. Figure 4 shows e general architecture of a single pixel and its corresponding processing circuitry and a digital memory. Bo e processing circuitry and a digital memory are located in e array periphery. he proposed pixel consists of a: () Photodiode, () Photodiode Reset switch, controlled by AND gate output, (3) Shutter switch, (4) -bit in-pixel digital memory (bo static and dynamic memories can be implemented), (5) Analog buffer for analog signal readout, (6) Row Select switch, (7) AND gate to control locally reset operation of e pixel and (8) _Reset switch, used to reset e floating diffusion capacitance C. he processing circuits consist of AND, two OR and Latch logical elements. In addition, analog Comparator is employed to compare e pixel output to e predefined reshold voltage ( _ ). Note, bo e processing circuit and e pixel, presented in is paper can be designed in different ways, while still implementing e described algorim. Herein, only one processing circuit and two different pixel examples are presented in detail. he presented circuit operates as follows: at e beginning of e frame e photodiode capacitance C is reset by applying "Array Reset" = '' and by loading e high digital value '' to e in-pixel memory by applying "Global Reset"=''. he Shutter switch is "off" during e reset period. reset _ Figure 3: hreshold voltage considerations for sensor array operated in shot mode of operation On e oer hand, to reduce e influence of time variations in different rows comparisons, each row can be used wi its own appropriate reshold voltage, as shown in e following equation: Figure 4. Schematic of a single pixel and its corresponding processing circuitry and digital memory. he reset phase is stopped by applying "Array Reset " = '0' and e photodiode capacitance C starts discharging, according to e energy of incident light. During e photodiode integration period e "Global Reset"='0'. Before reaching e certain time, when e first row pixels start comparison ( ( ) Δ t ), e C capacitor is pre-charged to DD voltage using e _Reset switch. Once e C precharge is completed, e Shutter switch is switched "on" and allows charge transfer between e photodiode capacitance C and C floating diffusion capacitance. he voltage on e

4 PAPER IDENIFICAION NUMBER photodiode capacitor at e end of is charge transfer is similar to e voltage at could be achieved by discharging e photodiode C and C capacitances connected togeer from e beginning of e integration. his is shown in Figure 5. he Shutter switch remains switched "on" till e end of e integration time. reset C C + C C + C Figure 5: Charge sharing between photodiode capacitance and floating diffusion capacity For a pixel in row, at e first time point = Δ t (see equation ) e voltage on e photodiode capacitance is read out using e analog buffer and is compared wi an appropriate reshold. he comparator _ result is transmitted directly to e in-pixel memory and to e external digital memory, associated wi e pixel ("First bit"='' is applied). In case <, meaning at e pixel will urate _ at e end of e integration time, bo mentioned memories are loaded wi ''. Oerwise, ( >, meaning at e pixel _ will not urate at e end of e integration time), '0' is loaded to e memories. he described process is en sequentially repeated for each row. At e ( ( ) ) time point, e "Array Reset" signal is activated and e photodiodes in all pixels in e array are reset (or not), according to e data saved in e in-pixel memories. hese simultaneous resets ensure e global shutter operation of e imager. At e time point ( ( ) Δ t ), e voltage on e photodiode is read out and is compared again wi an appropriate reshold. he binary information concerning _ having e reset applied at ( ) time point, or not, is retrieved from e corresponding digital memory (input to Latch in Figure 4) and "AND"ed wi e result of e photodiode voltage comparison wi ("First bit"='0' is applied). In case at _ < and e retrieved digital data is '' (meaning at pixel _ was reset at e previous time point), bo in-pixel and external memories are loaded wi '' and e pixel will reset again. In case <, but e retrieved digital data is '0' (meaning at e _ pixel was not reset at e previous time point), bo in-pixel and external memories are loaded wi '0' and e pixel will continue integration wiout reset. Finally, if >, independent of _ e retrieved data, bo in-pixel and external memories are loaded wi '0' and e pixel will continue integration wiout reset. At e ( ( ) ) time point, e "Array Reset" signal is activated again and e photodiodes in all pixels in e array are reset (or not), according to e data saved in e in-pixel memories. he same process is performed for all pixels in e array for all remaining time 3 ( )...( Δ t ). points ( At e end of e full integration time, e capacitor C is disconnected from e photodiode by turning off e Shutter switch. Once is charge transfer has been completed, e photodiode is able to begin a new frame exposure, and e charge on C is held ere until it is read out at its assigned time in a row-by-row readout sequence rough e output chain. he value of e readout signal is associated wi e analog value Man (see equation 4). he exponent value is retrieved from a digital memory also at is time. In order to improve e performance of e imager, Δ t should be ept as small as possible and reduction in decision is required. As previously mentioned, decision is e time, required to decide wheer each pixel in e specific row is going to be urated at e next integration slot and an reset or no. herefore it can be described as: = + + () decision mem _ read comp mem _ write where is e time required to retrieve e digital mem _ read information from e memory during decision process (e algorim relies on previous stored information), comp is e time required to accomplish e digital processing and photodiode voltage comparison and is e time required to write e digital information mem _ write into e memory. Figure 6 show possible transistor implementations of e proposed imager pixel. Figure 6(a) shows e implementation where e static in-pixel memory is used, while Figure 6(b) presents e dynamic memory version. Figure 6: Pixel implementation wi a) static in-pixel memory, b) dynamic inpixel memory. III. SIMULAION RESULS A test chip having two different 64x64 sensor arrays has been implemented in a standard SMC 0.8µm CMOS technology available rough MOSIS. he implemented arrays differ by eir inpixel -bit memory design: while e pixels in e first array employ in-pixel static memory, e dynamic memory is used in e second array. Figure 7 shows an example of e pixel layout employing e static in-pixel memory. Each pixel has a size of 7μm x 7μm and a fill factor of 0% for e static memory case and of 5% for e dynamic memory case. Figure 7:Pixel layout, employing in-pixel -bit static memory.

5 PAPER IDENIFICAION NUMBER Figure 8 presents an example of a single pixel simulation. he simulation has been carried for = 30msec for two different illumination levels. I. CONCLUSIONS AND FURHER RESEARCH A novel concept for global shutter CMOS image sensors wi ide Dynamic Range (DR) implementation was presented. he proposed imager provides wide DR by applying adaptive exposure time to each pixel, according to e local illumination intensity level. Different pixel configurations were designed and simulated. A test chip, having 64x64 arrays has been implemented in a standard SMC 0.8µm CMOS technology. System architecture and operation were discussed and simulation results were presented. Furer research includes fabrication of e proposed sensor and testing. ACKNOLEDGEMENS he auors acnowledge e Israeli Ministry of Science for funding is project. Figure 8: Single pixel simulation example for two different illumination levels. he figure shows e voltages on e photodiode, on e C capacitance and e value stored in e in-pixel bit memory C ("in_pixel_mem") for two sequential frames. wo bit DR expansion was applied, i.e. comparisons were performed at t = Δ t wi 0 = ( ) Δ ) and = during e first frame and at ( ) 4 ( ) t = Δ t and 3 = Δ during e second frame. indicates e end of e first frame and e start of e second frame. In e presented simulation e illumination level has been decreased at e second frame. It can be seen, at for e first frame, passed e reshold, during e first comparison ( _ 0 in Figure 8), meaning at e pixel will urate at e end of e integration time and erefore e pixel was reset and '' was written into e in-pixel digital memory. For oer comparisons (at, 3 and 4 ), didn't pass e reshold, meaning at e pixel will not urate at e end of e integration time and erefore e pixel was not reset and '0' was written into e in-pixel digital memory. Figure 9 zooms in to e first comparison during e first frame ( 0 in Figure 8). 0- time point indicates e time when e C capacitor is reset using _Reset switch (see Figure 4), 0- time point indicates e operation of e Shutter switch when e charge transfer between e photodiode C and C capacitances occurs. pd Finally, at 0- time bo e photodiode and C capacitor are reset, according to e comparison result. A variety of additional simulations were carried out to ensure proper system operation. REFERENCES [] E. Fossum, CMOS image sensors: Electronic camera-on-a-chip, IEEE rans. on Electron Devices, vol. 44, p. 689, 997. [] H. S. ong, CMOS image sensors-recent advances and device scale considerations, in IEDM ech. Dig., 997, pp [3] K. Cho, A. I. Krymsi, and E.R. Fossum, "A.5-550μ 76 x 44 autonomous CMOS active pixel image sensor", IEEE rans. on Electron Devices, Special Issue on Image Sensors, vol. 50, pp.96-05, January, 003. [4] O. Yadid-Pecht and R. Etienne-Cummings, " CMOS imagers: from phototransduction to image processing", Kluwer Academic Publishers. [5] E. Culurciello, R. Etienne-Cummings and K. Boahen, Arbitrated address event representation digital image sensor, 00 IEEE International Solid-State Circuits Conference, Digest of echnical Papers, ISSCC (Cat. No.0CH3777), IEEE 00, pp. 9-3, Piscataway, NJ. [6] L. G. McIlra. A Low-Power Low-Noice Ultrawide-Dynamic-Range CMOS Imager wi Pixel-Parallel A/D Conversion, IEEE J. Solid State Circuits, ol. 36, No. 5, pp , May 00. [7] D..D. Yang, A. El Gamal, B. Fowler and H. ian, A 640x5 CMOS Image Sensor wi Ultra ide Dynamic Range Floating Point Pixel Level ADC, IEEE ISSCC, A 7.5, 999. [8] D. Stoppa, A. Simoni, L. Gonzo, M. Gottardi and G-F. Dalla Betta, Novel CMOS image sensor wi a 3-dB dynamic range, IEEE J. Solid State Circuits, ol.37, No., Dec 00. [9]. Lule, M. agner, M. erhoven, H. Keller and M. Bohm, pixel,0db Imager in FA echnology, IEEE J. Solid State Circuits, ol.35, No.5, May 000. [0] O. Yadid-Pecht, A. Beleny " In-Pixel Autoexposure CMOS APS. "IEEE Journal of Solid-State Circuits, ol. 38, No. 8, pp , August 003. []. Hamamoto, K. Aizawa. A computational image sensor wi adaptive pixel based integration time, IEEE J. Solid State Circuits, ol. 36, No. 4, pp , April 00. [] A. Fish, A. Beleny and O. Yadid-Pecht, "Low Power Global Shutter CMOS Active Pixel Image Sensor wi Ultra-High Dynamic Range", submitted to IEEE Journal on Circuits and Systems II, October 004. [3] O. Yadid-Pecht, "ide dynamic range sensors", Optical Engineering, ol. 38, No. 0, pp , October 999. [4] K. A. Boahen and A. G. Andreou, A contrast sensitive retina wi reciprocal synapses, Advances in Neural Information Processing, ol. 4, pp , 99. [5] P. M. Acosta-Serafini, I. Masai, C.G. Sodini, A /3" GA linear wide dynamic range CMOS image sensor implementing a predictive multiple sampling algorim wi overlapping integration intervals, IEEE Journal of Solid-State Circuits, ol. 39, Issue:9, Sept. 004, pp: [6] S. Kavusi and A. El Gamal, Folded Multiple-Capture: An Architecture for High Dynamic Range Disturbance-olerant Focal Plane Array, in Proceedings of e SPIE Infrared echnology and Applications, ol. 5406, Orlando, FL, April 004, pp Figure 9: Zoom-in on e first comparison during e first frame ( 0 ).