3D Scenes Recovery through an Active Camera Based on Blur Assessment of the Resulting Image

Transcription

1 3 Scenes Recovery through an Actve Camera Based on Blur Assessment of the Resultng Image K. Alexev, I. Nkolova, G. Zapryanov Key Words: 3 scene recovery; depth from defocus. Abstract. The paper consders the task of recovery of 3 nformaton about the scene from sngle camera mages. The basc dea s to extract the useful depth nformaton from the mages automatcally and effcently. epth percepton wth sngle standard vdeo survellance camera s a challengng problem. The dffcultes n dervng the dstance to the observed objects n the scene can be partally overcome usng actve pan, tlt, zoom (PTZ) cameras and sutable control of camera parameters. There are several technques for depth recovery. Here, the task of depth estmaton n the context of the well known depth from defocus approach s consdered. In ths paper, t s proposed the problem to be solved usng defocus blur. The characterstcs of the approach are dscussed. Expermental studes, usng test patterns and real objects are presented. Introducton People perceve a huge amount of nformaton through ther eyes. espte the autors dfferng opnons regardng the concrete percentage of vsual nformaton n the general flow of nformaton, t s evdent that at least 90 percent of all nformaton enterng from the outsde world s perceved through vson. Vson s the act of seeng, the human ablty derved from the combnaton of the mage formng optcal system of the eye, the array of lght senstve receptors n the retna of the eye, and the nformaton processng capacty of the retna and human bran. Applcatons of nstruments wth a smlar ablty to sense a scene nclude broadcast televson, montorng ndustral processes, qualty control durng manufacture, vewng naccessble or hazardous places, adng medcal dagnoss, and remote sensng from satelltes and space probes, to name but a few. All these applcatons use vdeo cameras wth CC or CMOS sensors. The avalablty of powerful processors enables the use of complex algorthms for flterng and analyss of vdeo mages. espte the ntensve progress n the area, the exstng vdeo survellance systems serously lag behnd the qualty of nformaton processng characterstc for the human bengs. One of the major dfferences n ths process s the ablty to create a 3 model of the observed scene, whle n the techncal systems, wth rare exceptons, there s no such opportunty. The purpose of ths paper s to examne the possbltes for 3 scene recovery usng an actve camera. A relatvely smple and effcent approach s offered and analyzed. 1. Needs of Recoverng the epth of the Scene and Basc Approaches The wdespread CC and CMOS cameras accept lght radated or reflected by the observed scene so that a 2 mage of the object s formed drectly on the lght-senstve surface of the sensor. In that regstraton the locaton of objects such as angular coordnates n horzontal and vertcal drecton remans the same, but nformaton about the dstance to the objects s lost. A wdely used term, durng the last years, s ntellgent or smart cameras. One of the experts n the area, a technology specalst at Avnet Electroncs Marketng, John Weber [1] says that ntellgent cameras are an exctng new technology, able to perform: vdeo processng n real tme; moton detecton by constant scannng the feld of vew lookng for changes; vdeo trpware (detect moton when enterng a specfc area of crossng a preprogrammed lne trppng the wre); people countng (separate humans from non-humans, and count people enterng and extng a zone of nterest); object appearance/dsappearance (scan the feld for changes to detect mssng objects or scan the feld of vew for new objects that reman statonary); face recognton (analyze facal features), object localzaton and trackng and etc. The mplementaton of most of these features s mpossble n the absence of nformaton about the depth of the scene. Therefore, summarzng the above, we can come to the concluson that reconstructon of the thrd dmenson s crtcally mportant for determnng the actual spatal arrangement of objects, object trackng, understandng the spatal-temporal relatonshps between objects, evaluaton of ther behavor, and predctng future events. Scentsts have long attempted to develop hardware and software tools for 3 recoverng. Nowadays, there are professonal CMOS vdeo cameras, specally desgned to capture vdeo wth depth nformaton, the so called depth camera or 3 camera [2,3]. Such cameras have sensors that are able to measure the depth of each of the captured pxels. The objects n the scene are then arranged n layers n the Z axs, whch gves a depth map that any software applcaton can use. Unfortunately, they are a long way away from the qualty of the CC and CMOS used today. Òher resoluton s hghly lmted several tmes lower, and the prce s hgher. Therefore, more research efforts are put nto a software soluton to the problem wth standard vdeo sensors. The extracton of the dstance to an object at each pont n a scene s referred to as 3 magng or depth sensng. Many methods have been developed durng the years n order to obtan the 3 coordnates of the objects usng 2 camera mages. All of them explot the varatons of acquston parameters. There are a few approaches to solvng ths ssue: Mult sensors approach (stereo vson). Usng the perspectve and the context of mages. Usng a fxed camera and a controllable lght source (shape from shadng). Camera calbraton

2 gure 1. Mult-sensor approach for depth estmaton gure 4. Optcal axs bas under zoom ( X,Y,Z ) ocused mage plane x ( x 0, y 0 ) ( x, y) C e x e y e z X Object f Z f ob fm gure 2. Image formaton process: Pn-hole camera model gure 5. Image formaton process: Thn lens camera model Center of rotaton (CR) Angle of vew eld of vew ( OV) Optcal axs ocused mage plane Aperture Object Real mage plane f ob fm Optcal center (OC) rm gure 3. Optcal and rotatonal centers offset gure 6. epth estmaton usng blur from defocu

3 By means of manageable change of camera parameters (depth from focus /defocus). The mult sensor approach s based on the usage of frames from the observed scene, taken smultaneously (at the same tme) from at least two sensors (fgure 1). In the case of two vdeo sensors located close to each other, the method s called stereo vson. The frames must have an overlappng (general) part on the ground of whch regstraton s done usng some of the known methods (cross correlaton or other). The performed regstraton ams at fndng correspondng ponts n the frames receved from dfferent sensors. Wth a pror known nformaton about the locaton of cameras a successful trangulaton can be the source of depth nformaton about these ponts. ue to the fact that the camera determnes the angular coordnates wth hgh accuracy, the method s also dstngushed wth hgh accuracy. It s addtonally requred to use at least one more camera and software for detecton, regstraton and trangulaton of these ponts of the object. Some elementary facts are used such as: the more closely located objects occlude the ones standng behnd; when the camera moves wth a unform moton, the dstant statonary objects are vsualzed n the frames as movng wth much slower speed; parallel straght lnes ntersect n the dstance; usage of a pror knowledge regardng the envronment (sem-determned) n whch human socety lves rectangular rooms wth horzontal floors and celngs, doors and wndows and so on. Wth ths a pror knowledge, even people usng only one eye, are able to estmate farly well the dstance to an object. The method belongs to the methods of artfcal ntellgence and allows very good results to be acheved wth mnmal hardware requrements. Unfortunately, t s poorly developed and does not produce good results due to the lack of knowledge about the envronment. One well-developed approach for 3 reconstructon, known as shape from shadng [4,5] uses a statonary camera and a projector that llumnates the scene wth a pror known (as poston and modulaton) source of lght. Normally, a strpped lght pattern that produces two sets of lght planes (n the horzontal and vertcal drectons respectvely) s used. Each lght plane creates a deformed curve on the object surface and one or more lne segments on the plane. These lne segments and curves, together wth other lght planes, wll ntersect at many feature ponts. The camera captures the llumnated scene n ts mage whch s afterwards used for recalbraton and subsequent reconstructon. Unfortunately, the presence of such a lght projector sgnfcantly lmts the practcal applcatons of ths approach n real condtons. Moreover, t s dffcult to fulfl such task for non-convex objects. All methods descrbed above requre any addtonal equpment or are a subject of the mage context analyss. These two condtons are severely lmtng the applcaton of the approaches n real scenes. Unlke them, the ablty to restore the depth of the scene usng only one camera and analyzng a few frames has no such restrctons. Therefore, our attenton focuses on such technques n the followng sectons. 2. epth Estmaton through ynamc Calbraton of an Actve Camera Recovery of the depth of the scene s possble by usng only one, but actve camera. Such cameras gve the opportunty to adjust automatcally ther pan and/or tlt rotatons and optcal parameters: focus, rs and zoom. The objectve of camera calbraton s to determne all the requred parameters for obtanng the world coordnates from the pxel coordnates of a gven pont n an mage frame [6]-[8]. The man parameters of nterest regardng modelng the way that cameras project the three-dmensonal world nto a two-dmensonal mage are the mage center and the focal length. Most calbraton methods descrbed n the lterature rely on the usage of specal calbraton targets or patterns placed n front of the system n several precsely measurable postons. These calbraton targets or patterns are used n order to determne the camera parameters, ncludng the poston and orentaton (pan/tlt angles) of the camera n the space. Tradtonally, ths camera calbraton task s performed manually [9]-[12]. Ths s a laborous work and n the case of actve cameras neffectve because the camera system must be recalbrated any tme a component s moved or parameter settngs are changed. Thus t s necessary for the system to be able to auto-recalbrate tself (dynamc calbraton) wthout requrng operator s nterference [13]-[16]. The general prncples of the auto calbraton procedures, based on camera rotaton [17,18], are as follows: Capturng a set of overlappng mage frames of the observed scene all taken from a statonary camera, performng rotatons only. etectng mage feature ponts. Establshng correspondng ponts n the mages. Computng the 2 projectve transformaton (homography) for the mages wth correspondng ponts. Extractng the camera calbraton matrx from the matrx of homography. Computng the feature pont s depth, gven the parameters of the camera calbraton matrx. The camera calbraton process requres the constructon of a camera model. Ths model descrbes the process of transformng the 3 scenes nto 2 frames durng the mage formaton. The most commonly used camera model s the one based on the pn-hole camera (fgure 2). Ths s an dealzed camera model: () t s assumed that the camera performs a perfect lnear perspectve transformaton; () the projecton of the observed objects on the sensor s n the absence of lenses and nfntesmal aperture. Under these condtons, spatal coordnates of the objects only re-scale, losng the depth of the scene. Mathematcal descrpton of the model s as follows: f x s u0 j j x = KX, K = 0 f y v0,

4 where X s the -th object pont from the observed scene, when j j capturng the j-th frame, x s ts mage projecton on the mage plane. The matrx K s called an nternal calbraton matrx wth parameters: f x and f y focal length n pxels, s skew constant that descrbes any non-orthogonalty of the lnes and rows of pxels n the CMOS/CC, and the mage center coordnates ( u 0, v0) n pxels. Let the camera rotaton be performed and k-th frame be captured: k k x = KX. Rotatng the camera n a statonary scene could be seen as rotaton of the scene (the pont of the scene) opposte to the angle of the camera wth a matrx of rotaton R. Hence the correspondence of ponts of the object s: k j X = RX Substtutng the coordnates from the prevous equatons, we get: k 1 j x = KRK x, 1 where KRK determnes the matrx of homography H 1 ( H = KRK ). If we denote the unknown components of the matrx as: A H = G B E H C F. 1 then the relevant equatons for x and y coordnates of the correspondng feature ponts of the two mages wll get the followng form: j j k j k j T k [ x y x x x y ][ A B C E F G H ] = x j j k j k j T k [ x y 1 y x y y ][ A B C E F G H] = y At least 4 feature ponts are necessary to compose 8 equatons n order to fnd the components of the homography matrx. However, all authors of publcatons and studes on the subject recommend usng much more ponts. The resultng system equatons are overdetermned and are usually solved by the least square method or SV. In our experments, nether ths method (ncludng ts several modfcatons), nor any other method such as SV and others leads to acceptable results for the camera parameters. The analyss of the results shows that the condtons of the experment do not conform to the smple camera model chosen, namely: We work n real condtons, not n a specalzed laboratory. We do not use a known calbratng patterns, nstead, we work wth relatvely lmted number of feature ponts of the real scene. We work wth mass producton actve PTZ IP cameras, not specally desgned for the purposes of 3 scene percepton. or a rotatng and zoomng camera, the nternal camera calbraton parameters (focal length and mage center coordnates) do not reman fxed. The analyss also shows that the center of rotaton of our expermental camera (Axs 214 PTZ IP) and ts optcal center do not concde (the rotatonal center s behnd the optcal one). Practcally, ths means that the camera realzes larger rotatonal angles than those whch have been assgned to t (fgure 3). Moreover, t was expermentally found that the optcal center s shfted away from the optcal axs of the camera. Under these condtons, the conventonal pn-hole camera model proves napplcable. Therefore t s therefore necessary to use a realstc model of the optcal system of camera (thn-lens, even thck-lens optcal model). Thn lens model provdes an accurate descrpton only n the case of an deal system of lenses,.e. a lens wthout aberratons, as well as for the ponts, whch are near to the optcal axs. At real world cameras, the dstance between the lens and the mage detector s a functon to the dstance between the camera and the object, the focal length and the drecton (or angular poston) of a pont n the scene. etermnng ths functon permts the well-known formula of thn lens to be replaced and other equatons to be expressed. Even n the case of lack of accurate parametrc representaton of the functon, a look-up table could be used nstead. The dsadvantage s that the computatonal complexty ncreases. The optcal system of the modern cameras s very complex and dffcult to be modeled n detal. The man requrement to the model s to gve correct presentaton of functonal dependency of optcal parameters lke focus length, rs, zoom, and others. The model also has to be able to recognze possble errors and ther dstrbutons. or example, tunng up the varo (zoom parameter) moves the lens, that results n the change n the optcal axs poston. Often the errors from ths change are bg enough to dscredt the entre process of calbraton (fgure 4). 3. epth Recoverng by ocusng Today, all cameras use dfferent auto-focusng systems and algorthms. The auto-focus system establshed n most PTZ IP cameras s based on one of the approaches of the passve focus. Usually they use the fact that the accurately focused mage has the hghest contrast among all mages n the same scene. epth nformaton by focusng s a technque, whch estmates depth by searchng for the most n-focus (sharpest) poston over mult-focus mages (a sequence of mages taken by changng the camera parameter focus lttle by lttle). The depth recoverng by tunng the camera focus can be enforced upon the followng algorthm: The mage s dvded nto wndows of nterest. An algorthm for obtanng a maxmum contrast nformaton for each wndow of nterest s repeated untl the recorded contrast s the hghest, and the lens s focused. The receved focus dstance s reported and used after that to determne the real zones depth. The analyss of the method shows that t s a relatvely accurate one. The focusng algorthms are well developed and

5 studed, although not a sngle camera producng company gves detals about the auto-focusng algorthms t uses. The drawbacks of the contrast detecton method are: () t requres the scene or the area of focusng to have a hgh contrast, whch s not always the case; () an acceptable number of zones the smaller the zones number s, the less the tme requred for focusng s. There s a rsk of the occurrence of enormous naccuraces. Increasng the number of zones makes the algorthm slower. Thus CPU tme for processng the contrast nformaton dramatcally ncreases. Therefore, snce the algorthm for ths method s well establshed, t falls outsde our research nterest. 4. epth Estmaton by Analyzng Blur from efocus The defocus nformaton n the mage of an object formed by a camera system can be used to determne the dstance (depth) of the object from the camera system. The general prncple of the methods for depth estmaton by defocus explots the physcal effect produced by the modfcaton of the focus length or the aperture of a lens, and the dstance to an object, on a receved mage. When a camera s focused on an object at a certan dstance, other objects, both closer and farther than the focus dstance, form spots more or less blurred accordng to ther dstance to the mage plane (fgure 6). When the sensor s placed at a pont correspondng to the lens focus length, a clear (sharp) mage s produced. In case the sensor s nearer or farther away from the lens than the correspondng lens focus length, the mage becomes blurred due to the ntersecton of lght rays ether n front of or behnd the sensor (mage) plane. Another factor affectng the blur s the lens aperture. ecreasng a lens openng narrows the lght rays passng through the lens. Practcally, ths means that a small lens openng s used to record as clearly as possble several objects at varyng dstances. Even when the rays from some objects do not ntersect perfectly on the sensor plane, the blur ahead or behnd the sensor s neglgble and the mage stll appears sharp. When the aperture s relatvely larger (.e. the lens openng ncreases), the blur spot dameter becomes larger. Methods proposed n the lterature for depth estmaton from blur [19]-[23] use dfferent optcal propertes of the camera model. The most frequently used model wth an ntermedate level of complexty whch conssts of mage plane and replaces the mult-lenses camera optc wth a thn lens s used to derve some basc characterstcs of focusng based on geometrcal optcs (fgure 5). The man equaton descrbng dependences n ths model s based on the Gaussan lens law: (1) 1 f 1 1 = + ob fm where f s the lens focal length, ob s the dstance from the object pont to the lens center, and fm s the dstance from the lens center to the focused mage of the observed object. rom equaton (1) that follows that for a chosen focal length there s an ndefnte number of couples ( ob, fm ), satsfyng the equaton. Obvously, there s a lack of concdence between the model and the real camera. Ths shows that further restrcton needs to be ntroduced ncreasng the credblty and adequacy of the model to the real stuaton. Ths lmtaton stems from the realzaton of optcal sensors. Choosng a sutable zoom settng, the user ndrectly defnes the scale parameter M the rato between the sze L m of the mage of an object on the CC (CMOS) matrx and the actual sze L ob of the object Lm ( M = ). The scale unquely defnes the relaton L ob L d M = m fm ccd sensor = =. In the last expresson wth Lob ob d FOV d ccd sensor s denoted the effectve dagonal of the CC matrx and wth d FOV the dagonal of OV ( eld Of Vew) at dstance L ob. urthermore, an addtonal vald lmtaton s: ob + fm = = const. After these prelmnary remarks on the model of the optcs transformng the real 3 objects nto a 2 mage, we can proceed wth the explanaton of the mechansm used for determnng the depth of the object. The problem presentaton s depcted on fgure 6. Let σ denote blur spot (crcle of confuson) dameter, 2 rm be the dstance from the lens center to the plane of the taken mage, B be the aperture sze (dameter), 2 ob and fm be prevously defned dstances from the lens center to the object and to the plan of the focused mage. All these parameters are related by the followng equaton: B2 (2) σ 2 = abs( rm fm ) fm If the equaton s solved accordng to unknown followng results are receved: and (3) σ rm, the fm 2 rm = fm + for rm fm B2 σ = fm 2 rm fm B for rm < fm 2 In these expressons there s another parameter wth unknown value fm but t s easy to fnd t usng the Gaussan lens law (1):

6 1 2 rob gure 7. ependency between blur spot dameter and dstance to the object n case of a camera focused on dstance of 300 cm fm Usng fob = f rm ob dstance to the object rob : frm rob = f, obtaned by (3), we can fnally calculate the rm The ambguty n determnng the rob s due to the lack of functon monotony (fgure 7). or one and the same value of the blur dameter, the object can be located at two dfferent dstances. An example of the above statement s presented n fgure 7 where Pont 1 corresponds to the dstance of 220 cm and Pont 2 corresponds to the dstance of 470 cm n ths case, an enormous naccuracy n estmatng the dstance can be detected. Ths uncertanty could be resolved by the presence of more blur measurements, obtaned for dfferent focal length settngs of the camera (fgure 8). gure 8 depcts the measurements receved from two frames. The second frame s taken usng focus length correspondng to object dstance of 400 cm. The estmated dstances from the second frame are denoted by 3 and 4. It s shown that Pont 3 corresponds to Pont 1 and the estmated dstance to the object s 220 cm. or ponts 2 and 4 there s not any concdence. Ths example demonstrates that n the absence of nose even two frames are suffcent to fnd the dstance to every feature n the observed scene. In the case of nose a careful analyss, mage feature selecton and parameter choce have to be done for correct dstance estmaton [24]. The lnes (edges, contours) of the mage are the most commonly preferred features to be processed. The choce of lnes of valdated by the fact that lnes usually determne the plane borders and the scenes contan many lnes and partcularly straght lnes. There are many well-developed relatvely smple algorthms for lne determnaton. The gradent analyss of mage ntensty n the drecton, orthogonal to the lne, s used to estmate blur spot dameter. In many cases the gradent analyss s appled to a part of gure 8. Solvng the ambguty problem usng two frame a lne. The ntegraton reduces the nfluence of addtve Gaussan nose and mproves the accuracy of the result. Another way of mprovng the precson of the estmate s by usng a larger aperture. Equaton (2) shows that the sze of the blur spot ncreases n proporton to the sze of the aperture. Bgger aperture sze ncreases the photon flow and the correspondng sgnal to nose rato s also enhanced. The choce of sutable focus length s a more complcated problem. Shorter focal length mproves the sgnal to nose rato but the blur spot dameter ncreases, n case that the focused dstance s shorter than the desred one. In the case of remote objects, however, the short focus length loses ts attractveness. The functon s almost horzontal for remote objects (fgure 7) and the estmate may be receved wthout suffcent accuracy n the presence of nose. It s therefore necessary to recommend workng wth the focus lengths close to the searched focus length correspondng to the searched dstance or slghtly smaller than t. The result of the above analyss s that n the case of observed scenes wth sgnfcant depth dfferences the usage of several frames (more than two) wll enhance the accuracy of depth restoraton. The last parameter whose mpact on the qualty of restoraton wll be commented s the choce of an approprate scale. Increasng the camera zoom leads to the growth of the scale and to the proportonal ncrease of the sze of blur spot. Usng larger zoom when the objects are too close to the camera leads to such blurrng of the object contours that most of the methods for edge detecton cannot detect edges n the object mage. 5. Expermental Results of epth Restoraton Usng efocus Blur Our expermental work has two goals: () to verfy the applcablty of our mathematcal model to the practcal camera system we use and to explore the dependency between the camera parameters and the scene characterstcs and () to test the evaluaton accuracy of the recovered depths n a real scene. Two sets of experments are conducted. In the frst group of experments seven planar patterns, havng two types of vertcal edges wth hgh contrast are placed at dfferent, a prory known dstances from the camera. The frst type of black and whte

7 Insde edge Outsde edge (a) (b) gure 9. Expermental scenaro: (à) Test templates; (b) Insde and Outsde edges edges s called nsde (fgure 9 (b)). They belong to the same plane of the pattern. The plane s orthogonal to the camera optcal axs and, consequently, both sdes of the edge are at the same depth. The patterns are placed unformly n depth (fgure 9 (a)). The second type of edges s formed on the transton from one pattern to another pattern (from one plane n a gven depth to another plane n other depth). Such edges have maxmal contrast (black and whte) as well, but lttle dfferent contrast transton characterstcs due to transton from one plane to another at a pror known dstance. These edges are called outsde edges (fgure 9 (b)). Two dentcal experments are conducted: the frst one for shorter dstances (1-4 meters) and the second ones for longer dstances (4-7 meters). The patterns (templates) are postoned at ntervals of 50 centmeters. In each experment dfferent zoom The second group of experments concerns real partally structured scenes wth many vertcal lnes. At frst, the edges of the objects are determned and the dstances to them are measured n advance (fgure 11). Then, the camera s focused consequently on dstances, correspondng to the frst group of experments. All experments are conducted under the same zoom settngs (zoom x6). A Canny edge detecton algorthm s used to determne edges n the captured mage frame. On the bass of the obtaned results for the blurrng of these edges we can determne the dstances to them by usng the functonal dependences found n the frst experment. ve mage frames were analyzed by a dstance estmaton algorthm and the derved results are weghed. table 1 summarzes the results for the recovered depth on the ground of the data obtaned from the functonal dagrams of outsde and nsde edges separately. Edge # Truth depth [cm] Outsde Edge Graphcs Recovered depth [cm] Table 1: Accuracy evaluaton of multple depth recovery Error [%] Insde Edge Graphcs Recovered depth [cm] Error [%] * settngs are used n accordance wth the results from the prevous secton. The camera s focused consecutvely on each template under dfferent zooms n the range of x6-x9. The wdth of blur s calculated automatcally for the dfferent camera parameter settngs. The dfference n pxel ntensty s used n order to reduce the nfluence of the changes n llumnaton. The nfluence of the addtve Gaussan nose s lowered by ntegratng up to a hundred ponts per lne. The receved functonal relatons between blur spot dameter and dstance to the object are depcted correspondngly on fgure 10 (a), fgure 10 (c )and fgure 10 (e) for nsde edges and on fgure 10 (b), fgure 10 (d), fgure 10 (f) for outsde edges. * The obtaned functonal dependences from experments wth 6x zoom cannot be relably used to assess dstances exceedng 4 meters. or bgger dstances the fluctuaton from addtve Gaussan nose (approxmately 4 pxels) s close to functonal gradent and relable results cannot be receved (fgure 10 (d)). 6. Analyss of Results and Concludng Remarks The accuracy receved at ths early stage of expermental

8 (à) (b) (c) (d) (e) (f) gure 10. Expermental data about blur spot dameter accordng to the dstance to the objects for nsde (à, c) and outsde (b, d) edges for templates on 1-4 and 4-7 m and x6 zoom; expermental data about blur spot dameter accordng to the dstance to the objects for nsde (e) and outsde (f) edges for templates on 4-7 m and x9 zoom studes n estmatng the scene depth s promsng. The thorough analyss and careful tunng of parameters of the algorthms may lmt to a few percent the errors n the dstances evaluaton. Two parameters have a sgnfcant nfluence on depth estmaton results: the edge detecton approach and the blurrng model used to represent the optcal system of the camera. The standard approach to edge detecton s based on the ntensty analyss. The qualty of edge detecton and localzaton can vary over

9 gure 11. Real scene scenaro Threshold = 40 Threshold = 50 gure 12. Edge detecton wth Sobel gradent operator for two thresholds (à) Insde edge blur (b) Outsde edge blur gure 13. An expermentally derved functonal dependency for the wdth of blur wth camera focused at 1.50 m on patterns, postoned at dstances 1 to 4 m from t, under zoom x

10 a broad range, especally n real scenes, where edges may have dfferent local structure. The gradent operators, used by edge detecton algorthms, fal to relably detect and localze edges when the blur scale, contrast and mage nose exceed some admssble threshold. gure 12 presents some results obtaned va Sobel gradent operator. The analyzed scene s focused at dstance 1.0m. In ths case, the objects standng behnd are sgnfcantly blurred, whch affects the qualty of the edge detecton results and, thus, the opportunty to properly estmate the dstance s reduced. Therefore, the choce of a sutable threshold has a great sgnfcance to the fnal edge detecton results, because ths wll determne the admssble depth of the scene, whch can be determned. If the scene contans objects whch are outsde of some admssble zone, addtonal actons should be taken. Two approaches can be appled n that case n order to receve correct nformaton n such case. The frst one requres consequent focusng of the camera on each of the analyzed objects n the scene. A fnal decson can be made usng several well-focused patches. Obvously, such technque s tme consumng. The second approach s drectly related wth the adjustment of another camera parameter the rs, n order to ncrease the admssble depth of feld for the focused scene. he followng problems arse n the context of estmatng the dstance to the scene objects: An ambguty exsts when determnng the dstance n real scenes wth the help of the depth from blur approach. or example, when the camera s focused at dfferent dstances (fgure 13) and the blur wdth s 25 pxels, usng the expermental curves (fgure 10 (b)) we can found that three dstances are possble: 110 cm, 335 cm and 395 cm. The same results are observed for most of the nvestgated object edges as well. Sometmes the dstance estmaton error s sgnfcant. The problem should be solved usng addtonal measurements. Increasng the dstance to the test targets and usng a short-focal-length lens, the expermentally found functonal dependency (fgures 10 (c) (d)) could not be used for relable dstance estmaton, because the dfferences between the blur wdths are very small (approxmately 3-9 or 4-8 pxels). That leads to results wth sgnfcant errors. or ths reason, f we ncrease the dstance to an object, we should also ncrease the zoom settngs. Based on the performed expermental work wth test patterns and real scene targets, the followng conclusons can be done: Estmatng the scene depth by defocus requres at least two (better 3-5) mage frames captured at dfferent focused dstances. Increasng the dstance between the object and the camera, t s oblgatory to work wth a bgger zoom (.e. wth a longfocus-length lens). As t can be seen from fgure 10 (e) and fgure 10 (f), at zoom x9 and target dstance from 4 to 7 meters, the derved expermental dependences could be used for practcal purposes. In the case of ambguty when estmatng the dstance to an object through ts degree of blurrng, actve camera focus management s necessary, so that we can determne n whch part of the graphcs the object s located. It s preferable to stuate the camera focus around or behnd the object, rather than n front of t. In most cases, the evaluaton of the dstance on the bass of the expermental data for the nsde edges s more accurate than the one for the outsde edges (table 1). Acknowledgments Ths paper s partally supported by the Bulgaran Mnstry of Educaton and Scence under grants VU-MI-204/06. References 1. Weber, J. Smart Camera Bascs: Matchng Image Sensor, Analytcs and Processor Performance, Chu, C. W., S. Hwang and S. K. Jung. Calbraton- ree Approach to 3- Reconstructon usng Lght Strpe Projectons on a Cube rame. In Proceedngs of IEEE 3rd Internatonal Conference on 3- gtal Imagng and Modelng, Quebec Cty, QC, Canada, June 2001, L, Y.. and S. Y. Chen. Automatc Recalbraton of an Actve Structured Lght Vson System. IEEE Transactons on Robotcs and Automaton, 19, Aprl 2003, No. 2, pp Trucco, E., A. Verr. Introductory Technques for 3- Computer Vson. Englewood Clffs, NJ: Prentce Hall augeras, O. Stratfcaton of Three-mensonal vson: Projectve, Affne and Metrc Representatons. JOSA A, 12, Issue 3, Trucco, E., A. Verr. Introductory Technques for 3- Computer Vson. Englewood Clffs, NJ: Prentce Hall Huynh,. Q., R. A. Owens. and P. E. Hartmann. Calbratng a Structured Lght Strpe System: a Novel Approach. Internatonal Journal of Computer Vson, 33, Sep. 1999, No. 1, Hebert, P. A Self-Referenced Hand-Held Range Sensor. In Proceedngs of IEEE Thrd Internatonal Conference on 3 gtal Imagng and Modellng, Quebec Cty, Canada, May 2001, McIvor, A. M. Nonlnear Calbraton of a Laser Strpe Profler. Optcal Engneerng, 41, Jan. 2002, No. 1, Zhang, Z. A lexble New Technque for Camera Calbraton. IEEE Transactons on Pattern Analyss and Machne Intellgence, 22, 1998, No. 11, Zomet, A., L. Wolf and A. Shashua. Omn-rg: Lnear Self-recalbraton of a Rg wth Varyng Internal and External Parameters. In Proceedngs of the 8th IEEE Internatonal Conference on Computer Vson, 1, 2001, Huang, C. T., O. R.Mtchell. ynamc Camera Calbraton. Internatonal Symposum on Computer Vson, Nov 1995, Seo, Y., K. S. Hong. Theory and Practce on the Self-Calbraton of a Rotatng and Zoomng Camera from Two Vews. IEE Proceedngs on Vson, Image, Sgnal Processng, 148, June 2001, Sturm, P.. Self-Calbraton of a Movng Zoom-Lens Camera by Pre- Calbraton. Journal of IVC, 15, 1997, No. 8, Hartley, R., A. Zsserman. Multple Vew Geometry n Computer vson. 2nd Edton, Cambrdge Unversty Press, Cambrdge, e Agapto, L., E. Hayman, I.. Red, R. I. Hartley. Self-calbraton of Zoomng Cameras from a Sngle Vewpont, Panoramc vson: Sensors, Theory, and Applcatons, Sprnger-Verlag New York, Inc., Secaucus, NJ, Krotkov, E. ocusng. Internatonal Journal of Computer Vson, 1, 1987, No.3, Z Subbrarao, M. Parallel epth Recovery by Changng Camera Parameters. In Proceedngs of ICCV, 1988,

11 21. Pentland, A. P. A New Sense of epth of eld. IEEE Transactons on Pattern Analyss and Machne Intellgence, 9, July 1987, No. 4, Ens, J., P. Lawrence. An Investgaton of Methods for etermnng epth from ocus. IEEE Trans. PAMI, 15, 1993, No 2, MA, J., S. Olsen. epth rom Zoomng. J. Opt. Soc, 7, 1990, No 10, Nkolova, I., G. Zapryanov, K. Alexev. etectng of Unque Image eatures by Usng Camera wth Controllable Parameters. In Proceedngs of the ourth Internatonal Bulgaran-Greek Conference Computer Scence 2008, September 18-19,2008, Kavala, Greece, 3, Manuscrpt receved on Krl Alexev receved a MSc. degree n Cybernetcs from Kev Polytechnc Insttute n He s currently a research assocate at the Insttute for Parallel Processng, Bulgaran Academy of Scences. Hs scentfc research nterests nclude sensor networks, computer networks, track ntaton and target trackng, dentfcaton, dgtal mage processng, 3 scene restoraton. Contacts: Bulgaran Academy of Scences, Insttute for Parallel Processng, Sofa, Bulgara Georg Zapryanov s an assstant professor of computer systems engneerng at Techncal Unversty of Sofa. He receved the MSc. degree n Computer Systems and Technologes from the epartment of Computer Systems at the aculty of Computer Systems and Control. Hs professonal and scentfc research nterests nclude algorthm desgn and analyss, dgtal photography, mage sensors and dgtal mage processng. Contacts: Techncal Unversty of Sofa, Computer Systems epartment, Sofa, Bulgara Iva Nkolova s an assstant professor of computer systems engneerng, at Techncal Unversty of Sofa. She receved the MSc. degree n Computer Systems and Technologes from the epartment of Computer Systems at the aculty of Computer Systems and Control. The major felds of her professonal and scentfc research nterests nclude computer modelng, computer archtectures and parallel nformaton processng, dgtal mage processng. Contacts: Techncal Unversty of Sofa, Computer Systems epartment, Sofa, Bulgara