Folded Multiple-Capture: An Architecture for High Dynamic Range Disturbance-Tolerant Focal Plane Array

Transcription

1 Folded Muliple-Capure: An Archiecure for High Dynamic Range Disurbance-Toleran Focal Plane Array Sam Kavusi and Abbas El Gamal Deparmen of Elecrical Engineering, Sanford Universiy, Sanford, CA ABSTRACT Earlier sudies have shown ha muliple capure can achieve high SNR, bu canno saisfy he high dynamic range (HDR) and high speed requiremens of he Verically-Inegraed-Sensor-Array (VISA) projec. Synchronous selfrese, on he oher hand, can achieve hese requiremens, bu suffers from poor SNR. Exended couning can achieve high dynamic range a high frame rae and wih good SNR, bu a he expense of high power consumpion. The paper proposes a new HDR focal plane array archiecure, denoed by folded-muliple capure (FMC), which by combining feaures of he synchronous self-rese and muliple capure schemes, can saisfy he VISA requiremens a a fracion of he power dissipaion and wih more robusness o device variaions han exended couning. The archiecure is also capable of deecing subframe disurbances, e.g., due o laser jamming, and correcing for i. Keywords: IR focal plane array, ROIC, high dynamic range, verical inegraion 1. INTRODUCTION Significan developmens in focal plane array echnologies and archiecures have been made in recen years. CMOS echnology scaling has enabled he inegraion of analog and digial processing in he focal plane down o he pixel level. New archiecures ha ake advanage of such inegraion o enhance image sensor performance have been developed (e.g., [1, 2]). An example is he Digial Pixel Sensor, where digiizaion is performed a he pixel-level providing ulra high speed and dynamic range imaging capabiliies. 3 Verical inegraion provides he promise for even higher levels of pixel-level inegraion. 4 7 This is of paricular imporance o acical IR imaging applicaions. An IR imaging sysem may be used in differen environmens wih widely varying emperaure ranges and objec speeds. I also needs o be able o olerae undesired disurbances, e.g., due o laser jamming or sun reflecion. Such disurbances cause spikes in he phoocurren resuling in parial or complee loss of informaion. The wide range of iner-scene emperaure and he presence of disurbances require very high dynamic range of 120dB or more a 1000 frames/sec or more. 8 These requiremens canno be achieved wih oday s IR imaging sysems. To address he high dynamic range problem of convenional focal plane arrays, several sensor readou archiecures have been proposed in recen years. In previous papers, 9 11 we compared several of hese schemes based on heir SNR, implemenaion complexiy, and power consumpion. In [10], we found ha muliple capure achieves high SNR over he exended range, bu canno achieve he required 120dB of dynamic range a 1000 frames/sec. On he oher hand, synchronous self-rese can achieve very high DR a high frame rae, bu suffers from poor SNR a boh he low and he exended ends. In a companion paper, 11 we analyzed he exended couning scheme 12 and showed ha i can achieve high dynamic range a high frame rae and wih good SNR a he exended end, bu a he expense of high power consumpion. In his paper we describe a new high dynamic range FPA archiecure, which we denoe as folded-muliplecapure (FMC). The archiecure combines synchronous self-rese for signal folding and disurbance deecion wih muliple capure for achieving high SNR a boh he high and low ends of he dynamic range. I comprises a per-pixel analog-fron-end (AFE), a fine ADC sage, and a digial-signal-processor/conroller (DSPC) sage. The AFE, which is he key par of he archiecure, performs programmable gain conrol, synchronous selfrese, sample-and-hold, as well as enables disurbance deecion. I can be implemened wih relaxed circui requiremens and is robus o device variaions. The ADC is shared by several neighboring pixels and is opimized Correspondence: skavusi@sanford.edu, abbas@isl.sanford.edu; Telephone: ; Fax: Infrared Technology and Applicaions XXX, edied by Bjørn F. Andresen, Gabor F. Fulop, Proc. of SPIE Vol (SPIE, Bellingham, WA, 2004) X/04/$15 doi: /

2 for low power. The DSPC is also shared among several pixels. I esimaes each pixel s signal from he ADC and AFE oupus and conrols he AFE and ADC in order o adap he operaion o he imaging condiions. The FMC archiecure can be readily implemened using verical inegraion and is compaible wih is area, inerconnecion and power consrains. In he following secion, we describe he FMC archiecure, analyze is dynamic range and SNR, and discuss how i can be used o comba disurbances. In Secion 3, we argue ha FMC can achieve comparable, if no beer SNR, a he same DR as exended couning 12 wih significanly lower power consumpion and more robus AFE implemenaion. Needed background, erminology, and noaions follow he companion paper 11 and will no, herefore, be repeaed here. 2. FOLDED MULTIPLE CAPTURE In he following subsecion we describe he FMC archiecure, and discuss is dynamic range. In Subsecion 2.2, we analyze SNR for differen capure schemes and show how capure imes may be seleced. In Subsecion 2.3, we show how disurbance deecion is performed and is limiaions FMC Archiecure and DR The block diagram of he FMC scheme is shown in Figure 1(a). The scheme employs a synchronous self-rese modulaor bu uses a differen capure and filering scheme han he synchronous self-rese wih residue readou scheme 13 analyzed in [10]. The inegraor oupu is periodically compared, a he rising edge of he clock signal CLK, o a hreshold volage V h. When v() >V h, he comparaor flips and he inegraor is rese. During his operaion, he inegraor value is capured and digiized by he fine ADC a imes 1, 2,..., n, where i {(k i +1/2) clk :0 k i < in / clk }, as shown in Figure 1(b). We define he effecive inegraion ime las i, 1 i n, o be he ime from he las rese before i o i as shown in Figure 1(b). Clearly, he effecive capure inegraion imes las 1, las 2,..., las n can be readily calculaed from he rese sequence. Using he capure values and heir effecive inegraion imes (v( i ), las i ), 1 i n, he phoocurren is esimaed. This esimaion can be performed using only he nonsauraed capure wih he longes inegraion ime, by an appropriae leas-squares fi of he capures. Since he FMC archiecure explois he accuracy of clock references, i can be implemened wih relaxed circui requiremens. The AFE par of he FMC archiecure corresponds o he modulaor block in Figure 1(a) and is implemened per-pixel. I comprises a CTIA, a comparaor wih relaxed specificaions, and a sample-andhold. The V h and clock signals as well as a conrol signal for he sample-and-hold are roued o all pixels. The accuracy of V h and he CTIA bias are also relaxed, while he accuracy of he clock jier is criical (beer han 1nsec), which is no difficul o achieve in sub-micron echnologies. The ADC, which is shared by a block of several neighboring pixels, is opimized for low power and only needs o run a modes speed. Candidae ADC archiecures include muli-channel pipeline and successive approximaion. The filer is implemened as par of a DSPC, which is shared by a block of pixels. I esimaes he rese periods and effecive inegraion imes for each pixel in is block. I hen sors he capure values and deecs anomalies due o sauraion, moion, or disurbance and esimaes he phoocurren. These funcions can be implemened using a few couners, memory for soring he imes and capure values, an ALU, and glue logic. The DSPC also provides he conrol signals for he AFE and ADC in order o adap he sysem operaion o he imaging condiions. In a verically inegraed implemenaion, he deecor would occupy he op layer, he AFE would occupy one or more layers opimized for analog performance, he ADC would occupy one or more layers opimized for mixed signal, and he DSPC would occupy one or more layers opimized for logic performance and memory densiy. Now, we analyze he DR and SNR for he FMC archiecure. To quanify DR, noe ha Also i max = qq max clk / Proc. of SPIE Vol. 5406

3 Rese v() Fine ADC i ph Filer i ph V h Binary Sequence Modulaor CLK v (a) V max V h las_1 1 las_2 2 clk n las_n Sequence v (b) v( 1 ) v( 2 ) v( n ) las_n las_2 las_1 (c) Figure 1. (a) Block Diagram of he folded-muliple-capure archiecure. (b) Modulaor oupu. (c) Use of n capures o esimae phoocurren. i min = qσ Readou eff in, where σ Readou eff is he sandard deviaion of he effecive readou noise. Thus, 2.2. Analysis of SNR DR = 2Q max in σ Readou eff clk. To quanify SNR, we need o consider he capure scheme, he number of capures, and he ype of filer used. Firs we quanify SNR for a single capure a ime 0 < in ha is independen of pixel signal. Noe ha, for a given i ph, he ime o sauraion sa, rese period rese, and ime afer he las rese las are given by sa = qq max qqh, rese = clk, and las = i ph i ph clk rese rese. Proc. of SPIE Vol

4 Now, define { las / φ(i ph,)= sa (i ph ), if las / sa (i ph ) 1 0, if las / sa (i ph ) > 1. Noe ha SNR monoonically increases wih φ and is equal o Q max when φ = 1. Assuming sho noise dominaes, i.e., φ>σreadou 2 /Q max, SNR for a single capure a ime is given by SNR(i ph,) φ(i ph,)q max. φ is ploed in Figure 2. Noe he large dips in φ as i ph varies. Noe also ha hese dips are larges when rese occurs righ before he capure ime or when he inegraor is sauraed a φ (db) i ph (A) Figure 2. φ(i ph, clk )versusi ph. Q h =0.9Q max, clk =1µsec. There are wo approaches o eliminaing hese large dips and providing a guaranee on he minimum SNR in he exended range. The firs approach is o adap o he phoocurren in each pixel. This approach guaranees close o peak SNR in he exended range, bu a he expense of addiional per-pixel circuis o selec he bes capure ime afer he firs rese occurs. The second approach is o capure several samples a globally se imes. This approach eliminaes he need for exra per-pixel circuis, bu a he expense of performing more A/D conversions. We now show ha by judiciously selecing he capure imes, one can guaranee good SNR across he exended range using no more han 4 capures. Assuming n capures a imes n, where i {(k i +1/2) clk,for0 k i < in / clk }, define φ( i ), for 1 i n, as before. We wish o find he smalles number of capures and each capure s ime o guaranees a prescribed minimum SNR in he exended range. Mahemaically, we wish o find he smalles n and capure imes 1, 2,..., n such ha: max(φ( i )) >α i ph >Q max / in and 0 <α<1. i This problem can be reformulaed as follows. For a given m = rese / clk > 2, he range of i ph is beween qq h /m clk and qq h /(m 1) clk (see Figure 3). Thus for a given α, assuming Q h =0.9Q max, he se of capure 354 Proc. of SPIE Vol. 5406

5 ime indices such ha φ α are given by K m = where n max = in / clk. { j : α j+0.5 m { j : α j+0.5 m j+0.5 m m 1 m }, for 2 m 5 j+0.5 m 1}, for 5 <m n max, v V max V h m clk Figure 3. Range of i ph values ha generae he same m. Now, consider he (n max 1) n max marix A (nmax 1) n max wih enries a mk = { 1 if k Km 0 oherwise Noe ha he ones in he kh column correspond o he ranges of phoocurrens for which φ α. Thus he problem of deermining he minimum number of capures and each capure s ime reduces o selecing he minimum number of columns of A such ha heir logical OR is he all ones column vecor. The indices of he capure imes correspond o he indices of he seleced columns. The problem as formulaed is NP-complee. Now we describe a heurisic algorihm for finding a subopimal soluion o he problem. The algorihm is greedy, i sars by choosing he column wih maximum number of ones. If here is more han one column wih he same maximum number of ones, we selec one of hem a random. Nex, we form a new marix by deleing he seleced column and he rows corresponding o he ones in he column. The procedure is repeaed unil all rows of he marix are deleed. The indices of he seleced columns in he original marix deermine he capure imes. We applied his procedure o differen values of α and n max = 1000 and found several 3 capure soluions wih α =0.33 and several 4 capure soluions wih α =0.5. Table 1 provides some of hese soluions. α =0.33 (101,157,370) (143,334,818) (250,333,383) α =0.5 (143,157,335,502) (148,335,502,969) (143,337,502,850) Table 1. Capure ime indices for α =0.33 and α =0.5 for n max = Figure 4 plos SNR versus i ph for 3 capures wih α =0.33 and compares i o he reference sensor wih he same well capaciy, 11 which is opimized for low read noise. In addiion o he 3 capures, a readou a =0,for eliminaing rese offse, is assumed for boh plos. A leas-squares fi is used o esimae he phoocurren. The resuls are obained using Mone Carlo simulaions and ake ino consideraion finie rese duraion, comparaor and rese noise and offse, sample-and-hold swiched capacior noise, and ADC quanizaion. Noe ha in spie of Proc. of SPIE Vol

6 he large comparaor noise σ Comparaor and offse σ Offse, close o opimal SNR is achieved in he exended range. Furher, SNR is higher han ha of he reference sensor due o he use of muliple capures and leas-squares fi. 60 Reference FMC 50 SNR(dB) i ph (A) Figure 4. SNR versus i ph for folded-muliple-capure wih opimal four global capure imes, assuming Q max = 625, 000e, Q h /Q max =0.9, in =1msec, clk =1µsec, rese duraion of 0.1µsec, σ Swich = σ Rese =120e, σ Readou = 40e, σ Comparaor = 1000e, σ Offse = 18000e and achieves DR= 174dB Disurbance Tolerance In a number of applicaions disurbances, for example, due o pulsed or fas moving laser jammer or sun reflecion, insananeously appear in he camera field of view. Such a disurbance causes large spikes in pixel phoocurren, resuling in complee or parial loss of informaion. Idealizing such a spike by a dela funcion curren a s,he oal pixel phoocurren during a frame can be represened by i ph () =i ph + dδ( s ), 0 s in, where i ph is he scene induced phoocurren. The inegraor value is given by v() = 1 C (i ph + d), V max if v() <V max oherwise, where C is he inegraor capaciance. In a convenional archiecure, spikes may be deeced by operaing he focal-plane array a a very high frame rae, deecing each spike and eliminaing i before performing frame addiion and accumulaion. This soluion comes a he expense of high power dissipaion and degradaion in SNR, since he CTIA and he high resoluion ADCs mus run a very high speeds, and indeed may no be feasible o implemen. Using he FMC archiecure, spikes can be deeced wihou he need o operae a high frame raes. To explain how his deecion can be performed, consider he example in Figure 5. The figure shows he oupu of he inegraor for a low phoocurren, which would incur he mos severe disorion due o a spike. Since he phoocurren is low, he rese sequence wihou a spike has no ones. In he presence of a spike, he inegraor 356 Proc. of SPIE Vol. 5406

7 sauraes causing he rese sequence o conain a 1. This anomaly in he rese sequence is easily deeced and used o deermine wheher or no a capure is used in he phoocurren esimaion. If a capure occurs during he ime beween he disurbance and he rese (as shown in Figure 5), i is discarded. Oherwise, all capures are used. In general, he rese sequence for a consan phoocurren is mosly periodic, wih small variaion in he period due o noise. To deec a spike, he capure values and heir effecive inegraion imes are inspeced afer discarding sauraed capures. If a capure value is ouside he expeced range as prediced by all capure values and heir effecive inegraion imes, i is also discarded. Noe ha he same procedure can also be used o deec and correc for image blur as discussed in. 14 Figure 6 demonsraes he disurbance olerance feaure of he FMC archiecure. The op image is of a HDR scene. The lef image below is a simulaed image of he same scene wih disurbance due o a laser shone across i, as would be capured by a convenional FPA. The hird image simulaes he oupu of he FMC scheme wih disurbance deecion and correcion. Noe ha wih he excepion of minor arifacs around he person and he monior, he presence of he disurbance is compleely eliminaed. v V max Low Sequence Spike v V max Low & Spike Sequence Figure 5. Modulaor oupu wih and wihou disurbance. This deecion scheme, of course, assumes ha here are enough capures o deec an anomaly in he modulaor oupu. Furher, if muliple spikes occur, successful deecion may be possible depending on he placemen of he spikes and he number and placemen of he capures. 3. FMC VERSUS EXTENDED COUNTING In his secion we compare he FMC archiecure o he exended couning archiecure 12 analyzed in he companion paper. 11 We firs compare he wo schemes SNR. Then we show ha FMC consumes significanly less power han exended couning for he same DR and frame rae. Finally we highligh he robusness advanages of FMC. Figure 7 plos he SNR for FMC and exended couning for wo values of he sandard deviaion of subracion offse. Noe ha in boh exended couning plos, FMC achieves beer SNR in he low end due o muliple Proc. of SPIE Vol

8 Figure 6. Top: Image of HDR scene. Boom lef: Simulaed image wih moving laser. Boom righ: Simulaed image using FMC wih disurbance olerance. capure wih esimaion. A he exended range, he SNR of exended couning depends heavily on he process parameer variaions and can be higher han FMC under opimisic assumpions. To esimae he power consumpion for he wo archiecures, noe ha in exended couning power consumpion is dominaed by he CTIA componen. 11 Assuming an ADC figure-of-meri(fom) of 0.5pJ/conversion, 15 which is quie aggressive, o achieve 20 bis of DR a 1000 frames/sec for a array, power consumpion would be 0.5pJ = 34Wa. Now by comparing he DR formulas for exended couning 11 and FMC (see Secion 2.1), i is easy o see ha FMC requires 1/4h he clock speed of exended couning o achieve he same dynamic range and frame rae. The CTIA gain-bandwidh requiremen of he AFE amplifier in he FMC archiecure is also more relaxed han ha for exended couning, since seling ime error is cancelled in he self-rese operaion of FMC bu no in charge subracion. Assuming ha a he same clock speed, he CTIA in he exended couning scheme requires approximaely wice he bandwidh of he CTIA in he self-rese scheme, he gain-bandwidh of he self-rese CTIA is 2 4 = 8 imes lower. As a resul, assuming MOS square-law, he power consumpion of he CTIA in FMC is lower han ha in he exended couning scheme by a facor of 64. Using he same FOM for he fine ADC sage he power consumpion of he ADC sage of FMC assuming four capures is 0.5pJ = 1Wa. In conclusion, for he same 120dB DR and 1000 frames/sec speed, he FMC archiecure dissipaes less han 2Wa compared o 34Wa using exended couning. Furher, as demonsraed in he previous secion, he FMC scheme is capable of deecing and correcing disurbances a he subframe scale. The exended couning scheme canno deec such disurbances wihou increasing frame rae, and consequenly power consumpion. We now compare he robusness of FMC and exended couning o device mismaches and noise. Firs noe ha in FMC, rese and comparaor offses are cancelled via background calibraion, 16 since only he slope of he inegraor ramp is esimaed. Anoher imporan robusness feaure of FMC is ha he seling error componen 358 Proc. of SPIE Vol. 5406

9 80 70 EC1 EC2 FMC 60 SNR (db) i (A) ph Figure 7. SNR versus i ph for folded-muliple-capure vs. exended-couning. Assuming Q max = 625, 000e, in = 1msec, σ Readou = 40e, σ Comparaor, σ Swich = 127e. FMC (four capures) assumes Q h /Q max = 0.9, σ Offse = 18000e, clk = 1µsec. EC 1 assumes clk = 0.1µsec, σ Offse = 76e σ Swich = 127e ; EC 2 assumes clk = 0.1µsec, σ Offse = 610e, σ Swich = 127e. of rese offse is equal for all reses. This makes i possible o reduce rese duraion wihou increasing he amplifier bias curren. Comparaor offse is also immaerial in exended couning. However, by conras, no background calibraion is possible for exended couning. The charge subracion offse, which is due swich pedesal error, capacior mismach, bounces in reference volage, and seling error, is accumulaed during he inegraion ime and herefore canno be cancelled. Alhough some of hese offse componens can be cancelled by foreground calibraion, 16 he componen due o seling error canno since i is ime varying and signal dependen, and herefore canno be cancelled. A low phoocurrens he dominan noise componens are rese, swiched capacior and read noise. In FMC rese noise is cancelled as par of he background calibraion. The oher componens as well as he 1/f noise are reduced by he leas-squares fi [17]. By comparison, a low phoocurrens he aforemenioned noise sources are no cancelled in exended couning. ACKNOWLEDGMENTS The work in his paper was parially suppored under DARPA Microsysems Technology Office Award No. N We wish o hank Professors B.A. Wooley, B. Gray and B. Wandell, Dr. D. Su, Dr. D. Yang, Dr. B. Fowler, A. Agah, H. Eloukhy, A. Ercan, H. Kakavand, S. Lee, D. O Brien, and K. Salama for helpful discussions. REFERENCES 1. L. J. Kozlowski, K. Vural, W. E. Tennan, W. E. Kleinhans, and I. S. Gergis, Progress oward highperformance infrared imaging sysems-on-a-chip, in Infrared Technology and Applicaions XXVI, B. F. Andresen, G. F. Fulop, and M. Srojnik, eds., Proc. SPIE 4130, pp , December L. P. Chen, M. J. Hewi, D. J. Gulbransen, K. L. Peijohn, B. Chen, and R. Wyles, Overview of advances in high-performance ROIC designs for use wih IRFPAs, in Infrared Deecors and Focal Plane Arrays VI, E. L. Dereniak and R. E. Sampson, eds., Proc. SPIE 4028, pp , July Proc. of SPIE Vol

10 3. S. Kleinfelder, S. Lim, X. Liu, and A. El Gamal, A 10,000 frames/s CMOS digial pixel sensor, IEEE Journal of Solid-Sae Circuis 36(12), pp , December L. J. Kozlowski, Y. Bai, M. Loose, A. B. Joshi, G. W. Hughes, and J. D. Garne, Large area visible arrays: performance of hybrid and monolihic alernaives, in Survey and Oher Telescope Technologies and Discoveries, J. A. Tyson and S. Wolff, eds., Proc. SPIE 4836, pp , December J. Burns, L. McIlrah, C. Keas, C. Lewis, A. Loomis, K. Warner, and P. Wya, Three-dimensional inegraed circuis for low-power high-bandwidh sysems on a chip, IEEE Inernaional Solid-Sae Circuis Conference, pp , February K. Banerjee, S. J. Souri, P. Kapur, and K. C. Saraswa, 3-D ICs: A novel chip design for improving deep submicron inerconnec performance and sysems-on-chip inegraion, Proceedings of he IEEE 89(5), pp , May S. Benhien, T. Lulé, B. Schneider, M. Wagner, M. Verhoeven, and M. Bohm, Verically inegraed sensors for advanced imaging applicaions, IEEE Journal of Solid-Sae Circuis 35(7), pp , July S. B. Horn, P. R. Noron, J. D. Murphy, and R. E. Clemen, Verically inegraed sensor arrays (VISA), SPIE Defense and Securiy Symposium (Invied Paper), April D. Yang and A. El Gamal, Comparaive analysis of SNR for image sensors wih enhanced dynamic range, in Sensors, Cameras, and Sysems for Scienific/Indusrial Applicaions, M.M.Blouke and G.M.W.Jr., eds., Proc. SPIE 3649, pp , April S. Kavusi and A. El Gamal, Quaniaive sudy of high dynamic range image sensor archiecures, in Sensors, Cameras, and Sysems for Scienific/Indusrial Applicaions, M.M.Blouke,G.M.W.Jr.,and R. J. Moa, eds., Proc. SPIE 5301, January S. Kavusi and A. El Gamal, Quaniaive sudy of high dynamic range Σ -based image sensor archiecures, SPIE Defense and Securiy Symposium, April C. Jansson, A high-resoluion, compac, and low-power ADC suiable for array implemenaion in sandard CMOS, IEEE Transacions on Circuis and Sysems I 42(11), pp , November J. Rhee and Y. Joo, Wide dynamic range CMOS image sensor wih pixel level ADC, Elecronics Leers 39(4), pp , February X. Liu and A. El Gamal, Synhesis of high dynamic range moion blur free image from muliple capures, IEEE Transacions on Circuis and Sysems I: Fundamenal Theory and Applicaions 50(4), pp , April R. H. Walden, Analog-o-digial converer survey and analysis, IEEE Journal on Seleced Areas in Communicaions 17(4), pp , April F. Daihong, K. Dyer, S. Lewis, and P. Hurs, A digial background calibraion echnique for ime-inerleaved analog-o-digial converers, IEEE Journal of Solid-Sae Circuis 33(12), pp , April A. Fowler and I. Galey, Noise reducion sraegy for hybrid IR focal plane arrays, in Infrared Sensors: Deecors, Elecronics and Signal Processing, Proc. SPIE 1541, pp , July Proc. of SPIE Vol. 5406