Camera Calibration and Performance Evaluation of Depth From Defocus (DFD)

Please verfy that () all pages are present, () all fgures are acceptable, (3) all fonts and specal characters are correct, and (4) all text and fgures ft wthn the margn lnes shown on ths revew document. Return to your MySPIE ToDo lst and approve or dsapprove ths submsson. Camera Calbraton and Performance Evaluaton of Depth From Defocus (DFD) Tao Xan, Mural Subbarao Dept. of Electrcal & Computer Engneerng* State Unv. of ew York at Stony Brook, Stony Brook, Y, USA 794-35 ABSTRACT Real-tme and accurate autofocusng of statonary and movng objects s an mportant problem n modern dgtal cameras. Depth From Defocus (DFD) s a technque for autofocusng that needs only two or three mages recorded wth dfferent camera parameters. In practce, there exst many factors that affect the performance of DFD algorthms, such as nonlnear sensor response, lens vgnettng, and magnfcaton varaton. In ths paper, we present calbraton methods and algorthms for these three factors. Ther correctness and effects on the performance of DFD have been nvestgated wth experments. Keywords: Autofocusng, Depth From Defocus (DFD), camera calbraton, nonlnear sensor response, lens vgnettng, magnfcaton varaton. ITRODUCTIO Depth From Defocus (DFD) s a technque for autofocusng that needs only two or three mages recorded wth dfferent camera parameters. It recovers depth nformaton by estmatng the degree of blur. Due to the nherent advantage of beng local n nature, the spatal doman approach for DFD s more sutable for real-tme autofocusng applcatons. In practce, there exst many factors that affect the performance of DFD algorthms. In partcular, nonlnear sensor response, lens vgnettng, and magnfcaton varaton affect the accuracy of DFD. In order to mplement DFD on offthe-shelf commercal dgtal cameras, these factors need to be calbrated and corrected. In ths paper, we present new calbraton methods for these three factors. Ther correctness and effects on performance of DFD have been evaluated wth experments. Most dgtal cameras utlze the nonlnear sensor response to extend the dynamc gray-level range through a log-lke or gamma transform. DFD theory requres nverse mappng of ths non-lnear response to lnear response through calbraton. The ntensty measured by the mage sensor depends on llumnaton, exposure perod, and reflectance. A method s proposed and tested for correctng ths non-lnear sensor response. Optcal vgnettng s the phenomenon where the effectve lght energy transmtted by the optcal system decreases wth ncreasng nclnaton of lght rays wth respect to the optcal axs. A vgnettng calbraton method s mplemented and tested for ts effects on DFD performance. In DFD based autofocusng where the lens poston s moved, the magnfcaton of an object wll change when two mages are recorded wth dfferent camera parameters. A magnfcaton calbraton method s mplemented and the estmaton error has been evaluated. The calbraton methods for nonlnear response and vgnettng correcton are drect methods based on llumnaton measurement usng a dgtal lux tester. They do not need expensve and strctly controlled laboratory envronment and * E-mal: {txan, mural}@ece.sunysb.edu; Tel: 63 63-949; WWW: www.ece.sunysb.edu/~cvl 6- V. (p. of ) / Color: o / Format: Letter / Date: 9/4/5 5:4:9 PM

Please verfy that () all pages are present, () all fgures are acceptable, (3) all fonts and specal characters are correct, and (4) all text and fgures ft wthn the margn lnes shown on ths revew document. Return to your MySPIE ToDo lst and approve or dsapprove ths submsson. 3. OLIEAR SESOR RESPOSE COMPESATIO The formaton of a dgtal mage on the mage sensor of a camera can be descrbed by: τ + g( = q ( y, λ, t) s( λ) dλdt s where g ( s the photo-quantty of the specfc sensor element ( x, ; q s ( y, λ, t) s the actual lght energy fallng on the mage sensor ( x, ; s (λ) s the spectral senstvty of an element of the sensor. τ denotes the ntegraton perod, whch s controlled by exposure tme of the camera. From ths equaton, photo-quantty g ( s nether radometrc nor photometrc unt, snce t also related to the sensor spectral senstvty s (λ). For a specfc camera system, the photo-qualty depends on the lght energy fallng on the sensor cell per unt tme, and camera exposure tme. Once the parameters of DFD ( s, s, f, D for STM) are fxed, the measurement from DFD algorthms should only be related to the object dstance, and should not be affected by other changes such as llumnaton and camera exposure. However, most dgtal cameras utlze the nonlnear sensor response to extend the dynamc gray-level range through transforms (e.g. log(z)). That means more graylevels are assgned to the photo-qualty range wth hgher probabltes whle less graylevels are assgned to the photo-qualty range wth lower probabltes. 3.. Error analyss of non-lnear sensor response The nonlnear sensor response s a pont-wse mappng, whch can be formulated by a functon K: g '( = K[ g( ] (7) where g '( s the dstorted ntensty after pont-wse sensor response mappng, and g ( s the orgnal photoquantty formed as n Eqn. (6). If dgtal mages are quantzed to n bts, the pont-wse sensor mappng can be expressed by the transform vector k wthout sacrfcng generalty: K( ) = k I + b (8) where b s the dark offset, I s the orgnal photo-quantty vector, and k s the th coeffcent to map from level n the orgnal photo-quantty g to dstorted ntensty g '. n [ ] t [ k k k ] I = Λ (9) k = Λ () n For a lnear mappng, the components n the coeffcent vector for each level should be the same,.e. k = k j = k ; whle for a nonlnear mappng, k k s vald for some level, j. j (6) Due to nonlnear sensor response, the sgma σ ' n Eqn. (4) can be calculated from: ( g' g') β 4( ksg ksg) β σ ' = = β g' ( ks+ ks) β g where g ' and g ' correspond to the dstorted blur mages acqured at dfferent lens step. () The sgma error due to sensor response ε s expressed by the dfference between sgma calculated from dstorted mage pars and the ones wthout dstorton. ks ks ( g+ g) ε = σ' σ = () ks + ks β g From Eqn. (), the sgma error ε s not only related to the system parameter β, but also depends on camera nonlnear mappng coeffcent and the mage tself. The sgma error ε s transferred to the step error through the sgma-step mappng SS ( σ, ε ). A dependence on llumnaton and/or exposure s hence ntroduced. Ths s not desrable n autofocusng applcatons. 6- V. (p.3 of ) / Color: o / Format: Letter / Date: 9/4/5 5:4:9 PM

Please verfy that () all pages are present, () all fgures are acceptable, (3) all fonts and specal characters are correct, and (4) all text and fgures ft wthn the margn lnes shown on ths revew document. Return to your MySPIE ToDo lst and approve or dsapprove ths submsson. Ths sgma error ε can be elmnated only f k s = k n Eqn. (). Snce the value of s g and g are arbtrary n ntenstes n the range of [, ], the followng equaton should be vald for any ntensty to compensate for ε : k s = k s = k (3) Eqn. (3) demonstrates ε can be elmnated when there exsts a lnear relaton between observed ntensty and photoquantty. The lnearzaton can be obtaned by nverse mappng of sensor response K. g = K ( g' ) = K [ K ( g)] = g (4) 3.. Drect calbraton Sensor response calbraton s needed to compensate for ts non-lnearty. There are several approaches to measure the sensor response, through statstcal averagng on the whole mage for arbtrary scenes, such as [3] and [4]. However averagng on the mage plane ncludes the effect of vgnettng, whch wll be dscussed n Secton 4. The resultng response s a weghted average of on-axs ponts and off-axs ponts, and the weght depends on scene content. A drect measurement method s presented here to calbrate the nonlnear response for off-the-shelf cameras. The setup of the nonlnear sensor response calbraton s brefly outlned n Fg.. A dffusve whte screen WS s llumnated by multple lght sources from L to L. The lght sources are controlled from the lnear lamp controller module 4 LC to create varable/adjustable llumnaton. The ntensty at the central area of the whte screen s measured by a Dgtal Lux Tester YF-65. The mage of the whte screen s acqured by a dgtal camera to be calbrated. A lookup table s establshed by changng the llumnaton ncrementally whle recordng the mage at each llumnaton step. The relatonshp between camera gray-levels and normalzed llumnaton gray-levels s shown n Fg.. Mean brghtness n a * mage regon s used for reducng nose n the central area of the whte screen. To evaluate the effect of nonlnear sensor response, a seres of DFD experments were conducted under dfferent photoquantty condtons. The ambent llumnaton s 53 lux measured at the center of an object plane, and the dstance from the front surface of lens to the object plane s 54 mm. Accordng to Eqn. (6), the photo-quantty can be changed by ether llumnaton level or camera shutter speed. We control the shutter speed to obtan a wder range on photoquantty. The shutter speed changes from 5.65 ms to 5 ms, whch correspond to a change factor of 3 n photoquantty, as dsplayed n Fg. 3. In Fg.3, Images range from under-exposed as n (a), (b), to over-exposed as n (e), (f). The photo-quantty s doubled at each stage from (a) to (f), however the gray level of observed mage does not ncrease correspondngly due to camera range compresson. About 384 DFD experments were conducted at 8 dfferent random postons and 6 dfferent exposure levels. The 8 randomly selected postons are 35., 383., 474., 538.6, 63., 784., 58.4 and 353.6 mm respectvely, whch are measured from the front surface of the lens. The correspondng lens steps are obtaned from Depth From Focus (DFF) experments, and they are 5, 38, 58, 74,,, 3 and 45 lens step respectvely. As shown n Fg. 4, STM wthout sensor response compensaton has a mean error of up to 4 lens steps (Dstance 63. mm, Shutter Speed 5.65 ms), whle the correspondng RMS lens step error s.354 step, the mean error reflects a systematc bas whch s predcted by Eqn. (). After nonlnear sensor response compensaton, at the same photoquantty, the mean lens step error s reduced to.667 step, and all 8 DFD measurements get exactly the same step number, snce RMS lens step error s. The detaled results of DFD wthout/wth nonlnear sensor response compensaton are summarzed n Table and. Table shows the mean and RMS lens step error before nonlnear sensor response compensaton, whle Table shows errors after sensor response calbraton. When the photo-quantty contnues to ncrease from (e) to (f), step shfts n far feld for both DFF and DFD can be observed. In the mage (f), the sensor s already saturated, and the observed mage s no longer a correct measure of photo-quantty. In ths extreme condton, the error of DFD wth sensor compensaton ( 4.43±. 36 step) s stll better than the one wthout compensaton ( 8.667 ±. 535 step). 6- V. (p.4 of ) / Color: o / Format: Letter / Date: 9/4/5 5:4:9 PM