A vision sensor with on-pixel ADC and in-built light adaptation mechanism

Transcription

1 Microelectronics Journal 33 (2002) A vision sensor with on-pixel ADC and in-built light adaptation mechanism Amine Bermak*, Abdessellam Bouzerdoum, Kamran Eshraghian School of Engineering and Mathematics, Edith Cowan University, 100 Joondalup Drive, Perth, WA 6027, Australia Received 1 February 2002; revised 4 July 2002; accepted 23 July 2002 Abstract In this paper we propose an on-pixel Analogue-to-Digital Converter (ADC) based on pulse frequency modulation (PFM) scheme. This PFM based converter presents a viable solution for pixel-level based ADC. It uses a simple and robust circuit that can be implemented in a compact area resulting in a 23% fill-factor for a digital vision sensor in 0.25 mm CMOS technology. An in-built light adaptation mechanism has also been implemented which allows the sensor to better adapt to low-light intensity or to adjust the sensor saturation level. As a consequence, the proposed sensor features a programmable dynamic range. Image lag is eliminated since a reset of the photodetector is performed after the conversion period. A prototype comprising a pixel array has been implemented in CMOS 0.25 mm technology. Each pixel occupies an area of mm 2 with an average power consumption of 85 mw per pixel. q 2002 Elsevier Science Ltd. All rights reserved. Keywords: CMOS vision sensors; CMOS imagers; Light adaptation; VLSI; Pulse frequency modulation 1. Introduction The deep miniaturisation and the very high level of integration in microelectronics have changed the concept of today s electronic systems. After the single-chip radio receiver and the single-chip television, the single-chip camera is appearing on the scene. This is only possible in a standard CMOS technology that allows the realisation of photodetectors together with smart signal processing operations using both analog and digital circuits. A smart vision sensor is, therefore, a device that incorporates with a photodetector array some signal processing capabilities. The reapprochement between acquisition and processing is particularly interesting because it results in improved sensor performance since the acquired image can be locally processed on the CMOS chip. Moreover, CMOS technology offers significant advantages compared to the traditional CCD technology in terms of power consumption and system cost. For these reasons, great attention has been paid to the CMOS vision sensors over the last decade [1]. Although much work has been done in the field of CMOS vision systems, CMOS sensors are not as mature as their * Corresponding author. Tel.: þ ; fax: þ address: a.bermak@ecu.edu.au (A. Bermak). CCD counterparts. A significant effort is particularly required to improve the image quality in CMOS vision sensors. Recent research work based on the back-illuminated approach [2,3] and the twin-well technology [4] have been conducted in order to improve the fill-factor and hence the image quality of the CMOS imager. Significant improvements in terms of quantum efficiency and fill-factor have been obtained using these technologies; however, the use of such sensors is still very limited to laboratory prototypes since they rely on expensive non-standard processes. As a consequence, standard CMOS technology appears to be the most appropriate candidate for consumer electronic applications such as machine vision, robotics, automotive, security, and mobile video communication. In most of these modern applications it is required to integrate the Analogue-to-Digital Converter (ADC) on the chip; this lowers further the cost of the system, reduces the power consumption and allows on-chip digital processing. In order to obtain these benefits, one must study all the possibilities and take into account the trade-offs involved in such a VLSI implementation. ADCs can be integrated at the chip level [5 7], at the column level [8 10], or at the pixel level [11 14]. The chip level approach is at present the most widely used. In this approach, a single ADC is used for the whole imager, which reduces the total silicon area and allows the realisation of high fill-factor vision sensor since /02/$ - see front matter q 2002 Elsevier Science Ltd. All rights reserved. PII: S (02)

2 1092 A. Bermak et al. / Microelectronics Journal 33 (2002) most of the pixel area would be dedicated to the photodetectors. However, by using a chip level ADC, the signal-to-noise ratio is dramatically reduced since the weak analog output of the pixel will have to drive the whole column bus and this problem becomes even worse for very large sensor arrays. In addition very high speed ADC, operating at about 10 M samples/s for video, is required for the chip level ADC approach, which increases the power consumption. To lower the ADC operating speed, the column-level approach is used. In this approach, a battery of ADCs, each dedicated to one or more columns of the vision sensor, is used. The ADCs are operated in parallel, and therefore, low-to-medium-speed ADC architecture can be used [8 10]. To lower the ADC required speed even further the pixel-level approach is used. In this approach, an ADC is dedicated to each pixel and the array is operated in parallel. Consequently, very low speed converters can be used and a high level of parallelism is obtained using ADCs operating at tens of samples per second [13]. An efficient exploitation of the parallelism can reduce global power consumption and significantly speed-up data conversion, thus increasing the frame rate of the imager [13,14]. In addition, when the ADC is integrated within the pixel the load capacitance seen at the pixel output is negligible as compared to the one that is associated with the array and column based ADC. This reduces the KTC noise, which is a limiting factor of array based ADCs. Unfortunately, the main drawback of an onpixel ADC is a reduced fill-factor of the image sensor since a significant part of the pixel area will be dedicated to the ADC circuitry. As a consequence, implementing high fillfactor on-pixel ADC is a real challenge that faces researchers and CMOS imager designers. In this paper we propose an on-pixel ADC based on a pulse frequency modulation (PFM) scheme that can be integrated with very limited number of transistors, leading to a high fill-factor imager with on-pixel ADC. In Section 2, we introduce the principle of the PFM based ADC. In Section 3, we present the sensor in-built light adaptation mechanism. In Section 4, we describe the VLSI design, present the simulation results and the performance of the onpixel ADC. Section 5 presents a conclusion. 2. Pixel-level ADC The principle behind the proposed on-pixel PFM based converter is shown in the block diagram of Fig. 1(A). The circuit is composed of a photosensitive device (photodiode P d ) with its internal capacitance C d ; a PMOS reset transistor, a feedback circuit, a clocked comparator and a digital counter/register. The light falling onto the photodiode P d is responsible for discharging the internal capacitor of the photodiode C d : This results in a decreasing voltage V d across the node of the photodiode as shown in Fig. 1(B). This voltage V d is compared to a reference voltage V ref using a voltage comparator triggered by a clock signal Clk. When the voltage Vd reaches the reference voltage V ref ; the output of the comparator V out is switched high and a feedback circuit is used to reset the photodiode. A pulse is generated at the output of the comparator whenever the voltage V d reaches V ref : Fig. 1(B) shows the voltage across the photodiode V d (top figure) where T d represents the discharge time of the capacitor from V dd to V ref and can be expressed by T d ¼ ðv dd 2 V ref ÞC d ð1þ i d where i d and C d are the photocurrent and the photodiode capacitor, respectively. The frequency f of the voltage across the photodiode V d is inversely proportional to T d and therefore can be expressed as: f < 1 T d ¼ i d k with k ¼ðV dd 2 V ref ÞC d The switching frequency of the comparator output V out ; which is the same as that of V d (Fig. 1(B)), is directly proportional to ð2þ Fig. 1. (A) Block diagram of the proposed pixel based ADC. (B) Voltages of the different nodes of the circuit. V d is the voltage at the sensing photodiode node, V out is the output voltage of the comparator and En is the Enable signal of the counter/register.

3 A. Bermak et al. / Microelectronics Journal 33 (2002) the capacitor discharge current i d : Therefore, a conversion from photocurrent to pulse frequency is obtained. The output of the ADC is a signal which is very similar to a PFM scheme whereby the frequency of the signal is proportional to the photocurrent as described by Eq. (2). The voltage V ref is obtained internally at the pixel level using a voltage divider. A drop of the supply voltage V dd will induce a similar drop on V ref resulting in a constant ðv dd 2 V ref Þ within the array. The frequency f is therefore insensitive to variations in the supply voltage. This feature is very interesting for highresolution image sensor arrays where variations in the supply voltage of distant pixels are very likely to be significant. The output of the comparator is directly connected to the input of a digital counter/register with an Enable signal (En) provided externally as shown in Fig. 1(A). The counter is only operational during a counting period T cnt (Fig. 1(B), bottom), set by the external En signal. The number of impulses detected by the counter is therefore given by: N imp. T cnt T d By replacing T d by its value we obtain: T N imp. cnt i d ðv dd 2 V ref ÞC d 3. Sensor in-built light adaptation mechanism Adaptation to light intensity is a very important criteria for vision sensors because natural light levels can vary by over eight orders of magnitude [15]. The dynamic range of conventional CMOS imagers is mainly limited by the photodiode linear response, causing blooming to occur for ð3þ ð4þ typical natural scene illumination conditions. For this reason at least a global adaptation to the mean light intensity is required in CMOS vision sensors. This is traditionally achieved through a complex analog circuit implementing an automatic gain control within the sensor array. In our proposed digital sensor, this function can be simply implemented by modulating the T cnt signal according to the mean light intensity. If the sensor is being used in a given scene where illumination levels result in a photocurrent varying from 0 to i dm ; then T cnt will be modulated such that 2 n impulses resulting from the PFM scheme are counted for the maximum photocurrent i dm ; where n is the number of bits. By substituting N imp ¼ 2 n and i d ¼ i dm in Eq. (4) we deduce the optimum T cnt as follow: T cnt ¼ 2 n ðv dd 2 V ref Þ C i d ð5þ dm In our proposed digital sensor, the Enable signal of the counter, which is high during T cnt ; is therefore used to adapt the ADC to different illumination levels so that the brightest pixel would always generate 2 n pulses and hence would correspond to the full range of the ADC. According to this adaptation mechanism, for each environment where the photocurrent varies from 0 to i dm ; a corresponding optimum T cnt can be obtained using Eq. (5). This T cnt will correspond to the minimum counting time that allows the use of the ADCs full range. In other words, T cnt will optimise the dynamic range as well as the conversion speed of the ADC. Fig. 2(A) shows the variation of T cnt as a function of the photocurrent i dm for different precision (6, 8, and 10 bits). It can be seen from this figure that a longer conversion time is required for increased accuracy as well as higher photocurrent. Table 1 shows a typical example in which we consider the use of the ADC for three different environments: (i) environment 1 corresponds to a typical Fig. 2. (A) Optimum T cnt as function of the illumination levels for different precisions (6, 8 and 10 bits). (B) SQNR as function of T cnt for different environments.

4 1094 A. Bermak et al. / Microelectronics Journal 33 (2002) Table 1 Conversion results for an 8-bit accuracy using optimal T cnt values (Eq. (5)) for three different environments. For the three cases the full range of the ADC (00 FF) is being used Environment (1) Typical bright office (M1) (2) Typical outdoor light (M2) (3) Typical sunlight (M3) Photocurrent, i d (na) Optimal T cnt (ms) Digital output (Hex) FF FF FF bright office (Fig. 2(A), M1); (ii) environment 2 corresponds to a typical outdoor light (Fig. 2(A), M2), and (iii) environment 3 corresponds to a typical sunlight (Fig. 2(A), M3). In this example T cnt is obtained from Eq. (4), with V dd ¼ 2:5 V,V ref ¼ 1V,C d ¼ 1 ff and n ¼ 8 bits. For the three cases, the ADC operates within its full dynamic range (digital output: 00 FF). Fig. 2(B) shows the variation of the Signal to Quantisation Noise Ratio (SQNR) as a function of T cnt for different environments. For each environment, there exists one, and only one T cnt ; that optimises both the conversion speed and the SQNR. For each environment one must first estimate the maximum photocurrent and then calculate the optimum T cnt (Eq. (5)) that yields to the maximum SQNR without dramatically affecting the conversion speed. As an example, we can see from Fig. 2(B) that the optimum T cnt is equal to 48, 96 and 192 ms for photocurrents equal to 8, 4 and 2 na, respectively. It can be noticed from Fig. 2(B) that, on the one hand, by choosing T cnt lower than the optimum value, the SQNR is dramatically decreased. On the other hand, by choosing T cnt higher than the optimum value, the conversion speed is reduced without any gain in terms of SQNR performance (flat curve at about SQNR ¼ 48 db). Table 2 shows a typical example where T cnt is chosen constant (equal to 48 ms) for three different environments. In this situation, we can see that the ADC will operate within its full range (digital output: 00 FF) only for environment 3 while a 1-bit loss (digital output: 00 7F) is obtained for environment 2 and a 2-bit loss (digital output: 00 3F) is obtained for environment 1. These results are confirmed by the SQNR graph of Fig. 2(B) where we can note that keeping T cnt constant and reducing i dm by half results in a 6 db loss in SQNR, which corresponds to 1 bit at the digital output. The previous example shows that the Enable signal provides the possibility to adapt the sensor to any illumination level and allows the ADC to operate within its full range providing an increase in dynamic range of the photosensor. This last feature is usually obtained through a complex gain control amplifier integrated within the photodetectors array. In our case, this is simply realised by using a digital Enable signal of a conventional digital counter. Table 2 Conversion results for an 8-bit accuracy using fixed T cnt (Eq. (5)) for three different environments. As it can be seen from this table, 1 bit is lost for environment 2 corresponding to 6 db in terms of SQNR (Fig. 2(B), point N2) and 2 bits are lost for environment 3 corresponding to 12 db loss of SQNR (Fig. 2(B), point N1) Environment (1) Typical bright office (N1) (2) Typical outdoor light (N2) (3): Typical sunlight (N3) 4. VLSI design Photocurrent, i d (na) Fixed T cnt (ms) Digital output (Hex) F F FF The pixel-level ADC proposed in this paper operates in two different phases: (i) counting phase and a (ii) shift-out phase. During the counting phase, the counter/register is configured as a counter allowing to count the impulses generated by the comparator during a user defined T cnt period. During this period, all the pixels are operated in parallel resulting in full parallelism. Once the conversion is achieved, the shift-out mode is set and the digital values of the pixels stored in the internal registers are read-out. In this scheme, no read-out circuit is needed and hence the silicon area is fully dedicated to the photodetector array. Fig. 3, right and left shows the electrical simulation of the proposed approach for two different levels of illumination. Figures from top to bottom correspond to node photodiode voltage V d ; comparator clock, Counter/register Enable signal, and the digital outputs, respectively. It is clear from this figure that the frequency of the diode node voltage V d (top figure) depends on the illumination level. The final digital values are fed-out serially using the shift register during the shift-out phase. Additional gating circuitry has been included within each pixel in order to switch all the analog circuitry part to a stand-by mode during the shift-out phase. This has resulted in a significant reduction in power consumption during the shift-out phase. The average power consumption per pixel is estimated at 85 mw, where more than 95% of it is dissipated during the counting phase. A read-out speed of up to 1K frames/s could be obtained for an array of in typical office light condition corresponding to a counting phase of 576 ms and a shiftout phase of 234 ms. Although, it appears that the PFM counting phase is fairly slow; it does not, however, affect the overall performance of the vision sensor array since the conversion is done entirely in parallel. Fig. 4(A) shows the layout of the pixel based ADC. It has been implemented using CMOS 0.25 mm technology and occupies an area of mm 2 with a fill-factor of

5 A. Bermak et al. / Microelectronics Journal 33 (2002) Fig. 3. Simulation results of the pixel based ADC for different level of illumination (left and right figures). Wave forms from top to bottom correspond to node photodiode voltage V d, comparator clock, counter/register Enable signal, and the digital outputs, respectively. 23%. Careful attention has been paid to the layout of the pixel by separating the analog and digital parts and using guard rings and grounded shields. Different power supplies were also used for analog and digital circuits. Fig. 4(B) shows the layout of the array. It occupies a silicon area of 10 mm 2 with an average power consumption of 87 mw. Table 3 summarises the performance of the prototype. Fig. 4. (A) Layout of a single pixel occupying an area of mm 2. (B) Layout of the final VLSI prototype comprising pixel array integrated using 0.25 mm CMOS technology.

6 1096 A. Bermak et al. / Microelectronics Journal 33 (2002) Table 3 Performance of the prototype Feature Specification are also due to Dr Farid Boussaid and Prof. Paul Jespers for helpful discussions. Resolution Pixel size (x y ) mm 2 Fill-factor 23% Read-out speed 1K frames/s Power consumption 87 mw ADC On-pixel PFM-based ADC Chip size 10 mm 2 5. Conclusion In this paper we have presented a digital vision sensor based on the PFM scheme. The proposed approach presents a very viable solution for pixel-level based ADC. It uses a very simple and robust circuit that can be implemented in a compact area leading to a 23% fill-factor, which compares very favourably with the 3% fill-factor reported in Ref. [11]. An in-built light adaptation mechanism has also been implemented whereby it is possible to either improve the low light performance of the digital sensor or to modify its saturation level. As a consequence, the proposed sensor features a programmable dynamic range. This feature reduces considerably the blooming effect that can occur for typical natural scene illumination conditions. In addition, the proposed PFM-based ADC is insensitive to variations of the supply and reference voltages. The voltage V ref is obtained internally using a voltage divider. A drop of the supply voltage V dd will induce a similar drop on V ref resulting in a constant ðv dd 2 V ref Þ: Since the photocurrent i d is proportional to ðv dd 2 V ref Þ; our image vision sensor is insensitive to variations of the supply voltage of distant pixels which is very likely to be significant particularly in high-resolution image sensors. A prototype including an array of pixels has been designed and fabricated using CMOS 0.25 mm technology. The circuit occupies an area of 10 mm 2 and an estimated average power consumption of 87 mw. Currently, research work is being conducted on optimising further the performance of this vision sensor. Particularly, attention is being placed on optimising the power consumption by using other conversion schemes such as Pulse Width Modulation. Acknowledgements The authors would like to gratefully thank the financial support of the Australian Research Council (ARC). Thanks References [1] E. Fossum, CMOS image sensors: electronic camera-on-chip, IEEE Transactions on Electron Devices 44 (10) (1997) [2] B.I. Nemirovsky, Quantum efficiency and cross talk of an improved backside-illuminated indium antimonide focal-plane array, IEEE Transactions on Electron Devices 38 (8) (1991) [3] B.I. Nemirovsky, Surface passivation of backside-illuminated indium antimonide focal plane array, IEEE Transactions on Electron Devices 40 (2) (1993) [4] G. Meynants, B. Dierickx, D. Scheffer, CMOS active pixel image sensor with CCD performance, AFPAEC Europto/SPIE, Zurich; Proceedings of SPIE 3410 (1998) [5] M. loinaz, K. Singh, A. Blanksby, D. Inglis, K. Azadet, B. Ackland, A 200 mw 3.3 V CMOS camera IC producing b video at 30 frames/s, ISSCC Digital Technical Papers, San Francisco, CA, 1998, pp [6] S. Smith, J. Hurwitz, M. Torrie, D. Baxter, A. Holmes, M. Panaghiston, R. Henderson, A. Murray, S. Anderson, P. Denyer, A single-chip pixel CMOS NTSC video camera, ISSCC Digital Technical Papers, San Francisco, CA, 1998, pp [7] A. Bermak, A. Bouzerdoum, K. Eshraghian, A high fill-factor native logarithmic pixel: simulation, design and layout optimization, Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS 2000), Geneva, Switzerland V (May) (2000) [8] S. Mendis, S. Kemeny, E. Fossum, A CMOS active pixel for highly integrated imaging systems, IEEE IEDM Tech. Dig., San Jose, CA, USA (1993) [9] B. Pain, E. Fossum, Approaches and analysis for on focal-plane analog-to-digital analog-to-digital conversion, Proceedings of SPIE, Orlando, FL, USA, April 1994, pp [10] S. Decker, R. McGrath, K. Brehmer, C. Sodini, A CMOS imaging array with wide dynamic range pixels and column-parallel digital output, ISSCC Digital Technical Papers, San Francisco, CA, USA, Feb 1998, pp [11] B. Fowler, A. El Gamal, D.X.D. Yang, A CMOS area image sensor with pixel-level A/D conversion, ISSCC Digital Technical Papers, San Francisco, CA, USA, Feb 1994, pp [12] D. Yang, B. Fowler, A. El Gamal, A pixel CMOS area image sensor with multiplexed pixel level A/D conversion, Proceedings of the IEEE 1996 Custom Integrated Circuits Conference, San Diego, CA, USA, 1996, pp [13] D. Yang, B. Fowler, A. El Gamal, A Nyquist-rate pixel-level ADC for CMOS image sensors, IEEE Journal of Solid-State Circuits 34 (3) (March 1999) [14] S. Kleinfelder, S.H. Lim, X.Q. Liu, A. El Gamal, A 10,000 frames/s 0.18 mm CMOS digital pixel sensor with pixel-level memory, Proceedings of the 2001 IEEE International Solid-State Circuits Conference, San Francisco, CA, USA, Feb 2001, pp [15] G.C. Holst, CCD Arrays, Cameras and Displays, SPIE Optical Engineering Press, 1996.