COMPACT digital cameras now require a high pixel count,

Transcription

1 2998 IEEE JOURNAL OF SOLID-STATE CIRCUITS, VOL. 41, NO. 12, DECEMBER 2006 A 1/1.8-inch 6.4 MPixel 60 frames/s CMOS Image Sensor With Seamless Mode Change Satoshi Yoshihara, Yoshikazu Nitta, Masaru Kikuchi, Ken Koseki, Yoshiharu Ito, Yoshiaki Inada, Souichiro Kuramochi, Hayato Wakabayashi, Masafumi Okano, Hiromi Kuriyama, Junichi Inutsuka, Akari Tajima, Tadashi Nakajima, Yoshiharu Kudoh, Fumihiko Koga, Yasuo Kasagi, Shinya Watanabe, and Tetsuo Nomoto, Member, IEEE Abstract A 1/1.8-inch 6.4 MPixel 60 frames/s CMOS image sensor fabricated in a m single-poly triple-metal (1P3M) process is described. A zigzag-shaped 1.75 T/pixel architecture and a 10-bit counter-type column parallel ADC enables m 2 pixels. The resulting pixel has 38% fill factor and 12ke-/lux.s sensitibity. In addition, full frame and 2 2 binning modes are interchangeable without an extra invalid frame. Index Terms A/D converter, CMOS image sensor, dual CDS. I. INTRODUCTION COMPACT digital cameras now require a high pixel count, high imaging performance, and low power consumption. Pixel size miniaturization is necessary to achieve a high pixel count in an adequate optical format. The trend graph for the frame rate is shown in Fig. 1. Although the number of pixels has been increased, image sensors have been able to read out all of the pixels at only a few frames per second. The advantages of a CMOS image sensor are low power and easy system integration with on-chip circuits. Among CMOS image sensors, transistor-sharing techniques are widely used to make small pixels have better imaging performance [1] [3]. High-speed CMOS image sensors with on-chip ADC have been developed [4] [6], and digital double-sampling architecture is proposed to remove device variation and circuit offset that causes vertical fixed pattern noise (FPN) [5], [6]. Using these advantages, the readout speed can be made dramatically faster than that of CCDs. High speed and high pixel count will greatly expand the possibilities of the coming new digital camera world. To realize high-speed imaging, we have developed a 6.4 MPixel, 60 frames/s CMOS image sensor. There are three points needed to enable a 6.4 MPixel 60 frames/s sensor: 10/12 bit column-parallel analog-to-digital (A/D) converters, a zigzag-shaped four pixel sharing technique, and a 12-bit parallel LVDS interface. Manuscript received June 7, 2006; revised August 3, S. Yoshihara, Y. Nitta, M. Kikucchi, Y. Ito, Y. Inada, S. Kuramochi, H. Wakabayashi, M. Okana, Y. Kudoh, F. Koga, and T. Nomoto are with Sony Corporation, Kanagawa , Japan. K. Koseki, H. Kuriyama, J. Inutsuka, and A. Tajima are with Sony LSI Design Inc., Kanagawa , Japan. Y. Kasagi and S. Watanabe are with Sony Semiconductor Kyushu Corporation, Nagasaki , Japan. Digital Object Identifier /JSSC Fig. 1. Trend graph for frame rate. Fig. 2. Block diagram of the sensor. II. BASIC DEVICE ARCHITECTURE AND COLUMN-PARALLEL A/D CONVERTER The block diagram of the sensor is shown in Fig. 2. The sensor consists of a pixel block, column parallel counter-type A/D converters, control logic and peripheral circuits. The column A/D converters are composed of comparators, counters, and latches. Peripheral circuits include a phase-locked loop, generating a 216-MHz counter clock from a 54-MHz input clock. A 10/12 bit parallel LVDS interface circuit was chosen, enabling data rates up to 432 MHz. Column parallel A/D converters allow for low bandwidth readouts which enables image sensor to realize low noise characteristics. This is a key advantage over wide bandwidth single output amplifiers in CCDs. In order to realize these high frame rates, CMOS image sensors need to achieve both a high pixel throughput and high image quality /$ IEEE

2 YOSHIHARA et al.: A 1/1.8-inch 6.4 MPIXEL 60 frames/s CMOS IMAGE SENSOR WITH SEAMLESS MODE CHANGE 2999 Fig. 3. Column-parallel A/D converter. (a) Conventional A/D. (b) Columninline dual CDS. A conventional column-parallel A/D converter is shown in Fig. 3(a). The single-slope A/D converter is composed mainly of comparators, data latches, a ramp generator and a synchronous counter. Single-slope A/D converters have the advantage of a low bandwidth readout, which allows for low noise characteristics, and of high accuracy in the A/D conversion. On the other hand, single-slope A/D converters have a disadvantage in high-speed imaging, because they require a long time for the A/D conversion. A synchronous counter is used for digital sampling. Column-to-column variations of clock skew, which cause conversion error, are generated when a high-speed clock is used. Additionally, analog correlated double sampling circuits (CDS) need a large capacitor to keep noise at a minimum. The key development aspects of this architecture are shown in Fig. 3(b). A high-speed clock is utilized to reduce the A/D conversion time. Digital CDS, which performs digital double sampling and subtraction with column-inline counters, has been introduced. The schematic diagram for a column A/D converter is shown in Fig. 4. Pixel operation requires three control signals,,, and controlled by row decoders. The column comparators are driven by a ramp generator (DAC) and the pixel output by connecting series capacitor. The column counters, composed of ripple counters, perform the A/D conversion by counting the number of clocks until the comparator output changes. Ripple counters have the advantage of not needing to be synchronized with the high-speed clock. Digital CDS is obtained by changing up/down counting of the ripple counters using the clock selectors. The timing chart for the pixels and the comparators is shown in Fig. 5. The analog CDS sequence is as follows. First, the reset signal resets the pixels causing the reset level of the sensor output to appear at the pixel output (controlled by ) [A1]. After that, the input and output of the comparators are connected through the transistor Tcr (controlled by ) [A2]. This eliminates the offset of the comparators and the pixel outputs which causes FPN when the control signal is turned on. There is still some deviation remaining in the comparator input voltage which corresponds to the [A3]. The signal level appears when the control signal opens the transfer gate (controlled by Fig. 4. Schematic diagram for a column A/D converter. ) [A4]. The comparator output is turned on again in the same manner as the reset level [A5]. The column ripple counters perform the A/D conversion by counting the number of digital clock cycles. The counters stop when the clock latch negates the PLL clock with the comparator output. The ripple counters are set to down count period during reset readout [D1]. By changing the select signal, the ripple counters are set to up counting during the signal readout [D2]. The counters digitally subtract the conversion of the reset signal from the sensor signal. By using this dual CDS, the analog sensor signal is converted to the corrected digital output signal in the individual columns in parallel. When the dual CDS is finished, the digital data is transferred to the data latches included in each counter block. This pipelines the A/D conversion of the nth row and horizontal data transfer of the th row. The advantage of this dual CDS architecture is a high noise suppression capability because FPN cancellation is performed twice, once in the analog domain and once in the digital domain. An analog CDS is used to eliminate the analog offset of the pixel and comparator [A2] and to reduce the A/D conversion period for the reset signal [A3]. Additionally, a high-speed, 216 MHz clock is utilized to reduce the digital double sampling period [A3, A5]. Ripple counters are advantageous in this application because it is unnecessary for them to be synchronized with the high-speed clock. Column-to-column variations of clock skew and counter delay which cause A/D conversion error are corrected by digital CDS [D1,D2]. III. TRANSISTOR-SHARING TECHNIQUE Transistor-sharing techniques are widely used to make smaller pixels and to get high-imaging performance. The pixel arrangement of a zigzag-shaped four-pixel sharing technique is shown in Fig. 6. Bayer-pattern primary color filters and microlenses are fabricated on-chip. The layout design and the circuit diagram are shown in Fig. 7. The resulting transistor

3 3000 IEEE JOURNAL OF SOLID-STATE CIRCUITS, VOL. 41, NO. 12, DECEMBER 2006 Fig. 5. Timing chart from the pixels to the comparators. Fig. 6. Pixel arrangement of a zigzag-shaped four-pixel sharing technique. count is 1.75 transistors/pixel. The upper two pixels share the first floating diffusion and the lower two pixels share the second floating diffusion. The first and second floating diffusion areas are connected together by wiring. The shared floating diffusion node is also connected to the reset transistor and the amplifier transistor. In this configuration, the same color Gr and Gb correspond to a single column circuit. So the deviation of vertical neighboring Gr and Gb is negligible. Fig. 7. Layout design and the circuit diagram. TABLE I DESIGN SPECIFICATIONS IV. TIMING SEQUENCE This sensor has multiple modes of operation (Table I). The timing sequences of this sensor are as follows. The horizontal readout sequence at 6.4 MPixel and 60 frames/s is shown in Fig. 8. To obtain the maximum resolution at 60 frames/s, the resolution of VRAMP is set to 10 bits. After the reset operation, the reset level of the pixel is A/D converted. At that time, the A/D converter ramp signal is suppressed to 256 counts because the reset voltage is much smaller than the signal voltage. During A/D conversion of the signal voltage, the full 1024 range is used. Further, both the readout of the previously selected row and the A/D conversions of the currently selected row are performed simultaneously. As a result, the horizontal scanning time is 7.2 s.

4 YOSHIHARA et al.: A 1/1.8-inch 6.4 MPIXEL 60 frames/s CMOS IMAGE SENSOR WITH SEAMLESS MODE CHANGE 3001 Fig. 10. Vertical summation technique. Fig. 8. Horizontal readout sequence at 6.4 MPixels and 60 frames/s. Fig. 11. Readout sequence of the 222 binning mode. Fig. 9. Vertical accessing at 6.4 MPixels and 60 frame/s. The vertical timing for 6.4 MPixel 60 frames/s operation is shown in Fig. 9. First, the transfer pulse and reset pulse are opened at the same time. Charge accumulation starts when they are closed. Then, after the V sync pulse is received, the transfer pulse is open again and the charge of the pixel is read out to the vertical source follower output. There are 2310 vertical lines, each read out at 7.2 s, therefore all 6.4 MPixels can be read out within 1/60 of a second. This sensor can also be operated in high resolution mode. By increasing the number of bits during A/D conversion, an output resolution of 12 bits is achieved. The horizontal scanning time is extended to 28.8 s and all pixels can be read out within 1/15 of a second. The vertical summation technique is shown in Fig. 10. Vertical rows are summed digitally using this column A/D conversion technique. Each of the pixels is converted independently. Digital CDS is also accomplished the same way (Section II). The first row of pixels is converted in a similar way as the 6.4 MPixel 60 frames/s mode, i.e., down count of the reset level and up count of the signal level. The counter code is maintained without resetting the first row s pixel data. After that, the second row s pixel data is converted and the counter code sums that data. To prevent the counter code from overflowing, the counter has a 13-bit depth. Consequently, the two signals can be summed accurately. The readout sequence of the 2 2 binning mode is shown in Fig. 11. For the summation of the same color pixels of vertical rows, these are selected one after the other. Vertical signals are summed at the column counters, and horizontal signals of the pixels are digitally summed in the output circuit. Furthermore, this sensor also has a draft mode, in which 1/5 of vertical rows are read out. This draft mode results in a frame rate of 300 frames/s. V. SEAMLESS MODE CHANGE In conventional CMOS image sensors, it is necessary to insert an invalid frame to obtain a certain integration time for various readout modes. An example of a conventional sensor s mode change is shown in Fig. 12. When the mode is switched from 2 2 binning to 6.4 MPixel mode, a mismatch of the integration time occurs. The readout sequence is switched to 6.4 MPixel, but shutter operation of the previous mode was 2 2 binning. This is because the integration time of adjacent color rows is different between two modes, and then, the current frame of 6.4 MPixel becomes an invalid frame. The column parallel A/D conversion architecture of this sensor is suitable for the seamless mode change. There is no difference in the A/D conversion without switching the order of vertical rows. So, whether the signals are summed or not, the data of individual pixels are the same. Moreover, there are three key points to realize seamless mode change. To change the order of vertical rows at the shutter timing just after readout timing, shutter pulses and readout pulses are controlled independently, and the two modes can be switched by setting

5 3002 IEEE JOURNAL OF SOLID-STATE CIRCUITS, VOL. 41, NO. 12, DECEMBER 2006 TABLE II MEASURED DATA OF THE PIXEL Fig. 12. Example of a conventional sensor s mode change. Fig. 14. Measured nonlinearity at 60 frames/s. Fig. 13. Example of seamless sequence from 222 binning to 6.4 M. the serial communication one frame prior to the actual mode change. An example of seamless sequence from 2 2 binning to 6.4 M is shown in Fig. 13. The serial communication of 6.4 M is set during a previous frame, but the readout sequence continues in the 2 2 binning mode. While the readout operation of 2 2 binning continues, the shutter operation for the next frame is switched to the sequence of 6.4 M. As a result, a seamless mode change can be done without generating a mismatch between each row s integration times. Therefore, there is no invalid frame between 2 2 binning and 6.4 M. The reverse mode transition, from 6.4 M to 2 2 binning, is done the same way. VI. MEASUREMENT RESULTS The design specifications are shown in Table I. The sensor is fabricated using a m, single-poly triple-metal process. Pixel size is m. Zigzag-shaped 1.75 transistors/pixel architecture enables m pixels with high saturation and sensitivity. The input clock rate is 54 MHz, and the maximum pixel rate is 432 MHz. By using this sharing technique, high performance imaging is achieved. The measured pixel data is shown in Table II. The conversion gain is measured at 40uV/e-. The measured quantum efficiency of the green pixel is 48% at a wavelength of 550 nm. The sensitivity of the green pixels is e-/lux.s. A saturation signal of e- is achieved without image lag. These data include the effect of a color filter and an on-chip microlens. The aperture ratio, i.e., fill factor, is 38% without an on-chip microlens. The measured nonlinearity value at 60 frames/s is shown in Fig. 14. It is defined as the ratio of the deviation from linear approximation line to saturation signal. The horizontal and vertical axes show the integration time and the digital output which is normalized with saturation signal, respectively. The linearity of the A/D converter depends on the characteristics of the source follower, comparator and slope-d/a converter. Measured nonlinearity at 60 frames/s is 0.3% at 90% of saturation. The measured fixed pattern noises are shown in Fig. 15. It was taken with an analog gain of 18 db and a digital gain of 30 db. There is significant column-to-column FPN after analog CDS is finished. And after the dual CDS is finished, column FPN

6 YOSHIHARA et al.: A 1/1.8-inch 6.4 MPIXEL 60 frames/s CMOS IMAGE SENSOR WITH SEAMLESS MODE CHANGE 3003 Fig. 17. Total power consumption of all modes. Fig. 15. Measured fixed pattern noises. TABLE III MEASUREMENT RESULTS Fig. 18. Example of a reproduced 6.4 MPixel image at 60 frames/s. Fig. 16. Measured random noise. is decreased by 15.7 times. Column FPN is negligibly small because digital CDS eliminates offset precisely. Measurement results are shown in Table III. Measured vertical FPN is 0.38e- rms. Vertical FPN is much smaller than random noise at dark. So an image can be obtained without any vertical noise, even if the gain is raised. Measured random noise is approximately 7e- rms, as shown in Fig. 16. The random noise components from pixel and circuit are comparable. The output signal in 2 2 binning mode is divided by four in the on-chip output circuit in order to equalize the signal level. So, measured random noise during 2 2 binning is exactly half of that at 6.4 MPixel at 10 bits. Random noise of the 1/5 intermittent mode is the same as 6.4 MPixel at 10 bits. Total power consumption of each mode is approximately 360 mw, as shown in Fig. 17, because the column operations of all modes are almost the same. An example of a reproduced 6.4 MPixel image at 60 frames/s is shown in Fig. 18. Although the image is captured at 60 frames/s, the image quality is still high. A comparison of image distortion at 60 frames/s and 15 frames/s is shown in Fig. 19. An image distortion at 60 frames/s is one-fourth of that at 15 frames/s. An example of seamless mode change is shown in Fig. 20. One 6.4 MPixel image is actually obtained within continuous 2 2 binning mode, and the upper right picture is that of 6.4 MPixel 60 frames/s mode. Note the difference in resolution between the two modes. The user can freely record moving pictures and still images without any limitations on the imaging performance. A micrograph of the chip is shown in Fig. 21. VII. CONCLUSION Continuous 60 frames/s high-performance imaging has been demonstrated. By dividing the readout sequence, seamless image capture between two modes is also available. In conclusion, by using the techniques discussed in this paper, both video and still images can be captured from the same sensor operated in full 6.4 MPixel resolution. New opportunities in the digital imaging world are being sought for high-speed CMOS

7 3004 IEEE JOURNAL OF SOLID-STATE CIRCUITS, VOL. 41, NO. 12, DECEMBER 2006 Fig. 19. Comparison of image distortion at 60 frames/s and 15 frames/s. (a) Reproduced image of stationary subject. (b) Reproduced image of moving subject at 15 frames/s. (c) Reproduced image of moving subject at 60 frames/s. image sensors, and this work clearly shows one new application for CMOS image ACKNOWLEDGMENT Fig. 20 Example of seamless mode change. The authors acknowledge the contributions of T. Shoji, S. Sakane, K. Adachi, K. Kitagata, S. Kamogawa, Y. Yamagata, A. Nishimura, S. Ohki, M. Sato, K. Imamura, K. Masuda, M. Itoh, and K. Kaneko. The authors wish to thank H. Nomura, Y. Muramatsu, K. Amano, T. Toyama, J. Yamamoto, K. Mishina, A. Suzuki, T. Taura, A. Kato, and Y. Yasui for encouragement in this study, and they also acknowledge N. Fukushima and Y. Nakamoto for technical support. REFERENCES [1] H. Takahashi et al., A 3.9 um pixel pitch VGA format 10b digital image sensor with 1.5-transistor/pixel, in IEEE ISSCC Dig. Tech. Papers, Feb. 2004, pp [2] M. Mori et al., A 1/4 in 2 MPixel CMOS image sensor with 1.75 transistor/pixel, in IEEE ISSCC Dig. Tech. Papers, Feb. 2004, pp [3] K. Mabuchi et al., CMOS image sensor using a floating diffusion driving buried photodiode, in IEEE ISSCC Dig. Tech. Papers, Feb. 2004, pp [4] A. Krymski et al., A high-speed, 240-frames/s, 4.1-MPixel CMOS sensor, IEEE Trans. Electron Devices, vol. 50, pp , Jan [5] W. Yang et al., Integrated CMOS imaging system, in IEEE ISSCC Dig. Tech. Papers, 1999, pp [6] Y. Nitta et al., High-speed digital double sampling with analog CDS on column parallel ADC architecture for low-noise active pixel sensor, in IEEE ISSCC Dig. Tech. Papers, Feb. 2006, pp [7] S. Yoshihara et al., A 1/1.8-inch 6.4MPixel 60 frames/s CMOS Image Sensor with Seamless Mode Change, in IEEE ISSCC Dig. Tech. Papers, Feb. 2006, pp Fig. 21. Micrograph of the chip.

8 YOSHIHARA et al.: A 1/1.8-inch 6.4 MPIXEL 60 frames/s CMOS IMAGE SENSOR WITH SEAMLESS MODE CHANGE 3005 Satoshi Yoshihara received the B.S. degree in electronic engineering from Kyoto University, Kyoto, Japan, in In 1991, he joined Sony Corporation, Japan. He has worked on the development of CCD image sensor and CMOS image sensor. Souichiro Kuramochi received the Master of Engineering degree in applied physics from the University of Tokyo, Japan, in In 2003, he joined Sony Corporation, Japan. He has worked on the development of CMOS image Yoshikazu Nitta was born in Yamanashi, Japan, on November 24, He received the B.S. and M.S. degrees in electronic engineering from Tohoku University, Sendai, Japan, in 1986 and 1988, respectively. In 1988, he joined Mitsubishi Electric Corporation, Hyogo, Japan. In 2003, he joined Sony Corporation, Atsugi, Japan, where he is currently working on highspeed CMOS image Hayato Wakabayashi received the M.S. degree in engineering science from Osaka University, Osaka, Japan, in In 2004, he joined Sony Corporation, Japan. He has worked on the development of CMOS image Masaru Kikuchi joined Sony Corporation, Japan, in He has worked on the development of CMOS image Masafumi Okano received the M.S. degree in bioengineering from Waseda University, Tokyo, Japan, in In 2003, he joined Sony Corporation, Japan. He has worked on the development of CMOS image Ken Koseki received the B.S. degree in physics from Niigata University, Niigata, Japan, in In 1998, he joined Sony LSI Design Inc., Japan. He has worked on the development of CMOS image Hiromi Kuriyama joined Sony LSI Design Inc., Japan, in He has worked on the development of digital circuit for TV, audio and CMOS image Yoshiharu Ito received the B.S. degree in electronic engineering from Aoyama Gakuin University, Tokyo, Japan, in In 1989, he joined Sony Corporation, Japan. He has worked on the development of CMOS analog and digital circuit design and CMOS image Junichi Inutsuka joined Sony Nagasaki Corporation (now Sony Semiconductor Kyushu Corporation) in He has worked on the development of CMOS A/D converter macro and analog macros. In 2006, he transferred to Sony LSI Design Inc., where he is now working on CMOS image sensor design. Yoshiaki Inada joined Sony Corporation, Japan, in He has worked on the development of CMOS image Akari Tajima received the B.S. degree from Kyusyu Institute of Technology, Fukuoka, Japan, in She joined Sony Nagasaki Corporation (now Sony Semiconductor Kyushu Corporation) in She has worked on the development of CMOS A/D converter macro and analog macros. In 2006, she transferred to Sony LSI Design Inc., Japan.

9 3006 IEEE JOURNAL OF SOLID-STATE CIRCUITS, VOL. 41, NO. 12, DECEMBER 2006 Tadashi Nakajima received the M.E. degree in industrial chemistry from Seikei University, Tokyo, Japan, in He joined Sony Corporation, Japan, in 2003, where he has been engaged in the development of semiconductor devices. Yasuo Kasagi received the B.S. degree in electronic engineering from Tokyo University of Agriculture and Technology, Tokyo, Japan, in In 2000, he joined Sony Semiconductor Kyushu Corporation. He is presently engaged in development of process integration for CMOS image Yoshiharu Kudoh received the B.E. and M.E. degrees of mechanical engineering from Tohoku University, Sendai, Japan, in 1995 and He joined Sony Corporation in 2003, where he has been working in research on image sensors devices such as CMOS image Shinya Watanabe joined Sony Corporation, Japan, in He has worked on the development of CCD image sensor and CMOS image Fumihiko Koga received the M.E. degree in physical electronics from Tokyo Institute of Technology, Tokyo, Japan, in He joined Sony Corporation, Japan, in 2003, where he has been engaged in the development of semiconductor devices. Tetsuo Nomoto (M 06) received the B.S. and M.S. degrees in applied physics from Tohoku University, Sendai, Japan, in 1988 and 1990, respectively. In 1990, he joined Olympus Optical Corporation, Nagano, Japan, where he was involved in the development of charge modulation device image He joined Sony Corporation, Kanagawa, Japan, in 2001, where he has been engaged in the development of CMOS active pixel Mr. Nomoto is a member of the Physical Society of Japan.