Linear Feature Based Aerial Triangulation. A. Akav, G. H. Zalmanson and Y. Doytsher

Transcription

1 near Feature Based Aeral Trangulaton A. Akav, G. H. Zalmanson and Y. oytsher epartment of Transportaton and Geo-Informaton Engneerng Faculty of Cvl and Envronmental Engneerng Technon Israel nsttute of technology Technon Cty, Hafa, Israel (akav, garry, KEY WORS: feature based photogrammetry, orentaton parameter estmaton, regstraton, homography, fundamental matrx ABSTRACT: For the past ffteen years lne photogrammetry has been an extremely actve area of research n the photogrammetry and computer vson communtes. It dffers from tradtonal analytcal photogrammetry n the nature of the prmtves employed n a varety of ts fundamental tasks. Whle n tradtonal photogrammetry zero-dmensonal enttes,.e., ponts are exclusvely used as a drvng power n ts varous orentaton and explotaton procedures n lne photogrammetry as the name suggests lnear, that s one-dmensonal features often correspondng to elongated man-made features n the object space are employed. Of course, that means that no pror correspondence between dstnct pont n object space and ts projecton n the mage s requred and the entre lnear feature (wth arbtrary geometry) s accommodated n the approprate mathematcal model. However, despte a great effort n that feld, only the resecton problem,.e. the soluton of the exteror orentaton from lnear features' correspondences has been thoroughly nvestgated so far. Two addtonal fundamental photogrammetrc problems - space ntersecton and relatve orentaton, completng a trple of the most basc photogrammetc procedures needed to support feature-based trangulaton have not been adequately addressed n the lterature. Ths paper provdes that mssng lnk by presentng a procedure for relatve orentaton parameters estmaton from lnear features. We restrct our attenton n ths paper to planer curves only. We start wth the smple dea of optmzaton procedure usng ICP algorthm and proceed to the recovery of the homography matrx nduced by the plane of the curve n space.. INTROUCTION ne Photogrammetry (P) has been a tremendously actve feld of research for almost two decades. Over these years many researchers have argued n favor of accommodatng lnear features nstead of ponts for dfferent photogrammetrc tasks. Some of ther central arguments are set forth as follows:. In many typcal scenaros, lnear features can be detected more relably than pont features (Mkhal, 99).. Images of urban and man made envronment are rch of lnear features (Habb, ).. Close range applcatons employed n ndustral metrology often lack an adequate amount of natural pont features, thus requrng a costly use of artfcal marks for automatng the nvolved mensuraton tasks. (Kubk, 989) 4. Matchng lnear features s easer and more relable than matchng pont features (Zalmanson, ). Ths paper presents possble solutons for the classc problem of determnng the relatve orentaton parameters. The procedures proposed here are based on usng free form - planar curves nstead of conjugate ponts. In the resent years we are wtnessng the entrance of more and more dgtal photogrammetry workstatons. evelopng automatc processes for photogrammetrc applcatons has attracted a large body of research n the photogrammetry and computer vson communtes. The natural step towards automatc aeral trangulaton would be to adopt hgher level enttes for determnng orentaton parameters. Autonomous solutons for relatve orentaton wth lnear features employng Hough search technques have been proposed by Habb (, ). Solutons for relatve orentaton usng a subclass of lnear features, namely, planar curves and conc sectons have been ntroduced by Shashua () who dealt wth rd degree algebrac planar curves, Ma (99) who used planar concs and Petsa () who worked wth straght coplanar lnes.. USING PANAR FREE FORM CURVES We represent free form curves n mage space by a sequence of - ponts. Tryng to represent such curves n polynomal or parametrc form would yeld a more smplfed mathematcal modelng but at the same tme would result n some loss of nformaton due to nherent generalzaton process beng nvolved. The procedures shown here are vald for planar curves. We start wth the smple dea of recoverng the relatve orentaton parameters from free form planar curves. Every planar curve adds parameters to the overall soluton. The redundancy and the mnmum number of planar curves needed for recoverng the R.O.P wll be dscussed later.. Smplest dea Frst we have to determne ntal parameters. Snce we try to recover the relatve orentaton, we can refer to the model space as the object space n exteror orentaton. Intal parameters for the relatve orentaton can be determned n the classcal way, assumng aeral photos, most lkely near-vertcal and hghly correlated. ependent relatve orentaton model had been chosen, whch defnes the model coordnate system parallel to the frst (left) mage's coordnate system. As for the plane n the model space, horzontal plane can be used to determne ntal parameters. After determnng all ntal parameters needed, one can project the curve from both mages to the plane n the object/model space and get the ntersectons of the surfaces created by the

2 correspondences curves from the mages wth the plane. Those ntersectons must be dentcal to get the full overall soluton. When applyng ths procedure wth ntal parameters we wll get two separated curves. Now n order to brng the two curves closer ICP algorthm s proposed. X Y Z [ a b c] where R rotaton matrx T dsplacement vector f focal length x R y + T x f R y f () Fgure. Projecton of correspondences curves. ICP teratve closest pont The ICP algorthm, frst ntroduced n (Besl and McKay, 99) can be used wth several representatons of geometrc datasets, such as pont sets, lne segments set, mplct curves, surfaces etc. The geometrc datasets used n ths paper are pont sets, representng the free form curves. The datasets we deal wth are the projected curve from the frst (left) mage and the second (rght) mage of the photogrammetrc model. As mentoned before the data sets are pont sequences, Xl and Xr for left and rght projected curves respectvely. ICP steps:. Compute the closest ponts: YkC(Pk,X) where C s an operator for fndng the closest pont between P and X.. Compute the regstraton to mnmze the sum of square dstance between the closest ponts found. (qk,dk)q(po,yk).. Apply the regstraton: P k+ qk (Po). 4. Stop the teraton when the change mean squares error small then threshold. wth X beng the model shape, P the data shape and Y representng the closest ponts found. In our case there s no model shape and data shape, both shapes are data changng wth the refnement of the parameters (R.O.P + plane parameters). Pont sequences are obtaned by computng the planar curves as ntersectons of cones havng the perspectve center of the mages as ther orgn wth mage space curves and the plane. The plane equaton s represented by parameters for nstance: ax+by+cz. Any pont from the pont sequence could be computed by multplyng the vector, startng wth the perspectve center through the pont from mage plane, by scale factor. The scale factor can easly found usng plane parameters. Full descrpton of ICP algorthm can be seen n Besl and McKay work(99). Projectng all the ponts n both mages through equaton () yelds the planar curves shown n fgure. The Eucldean dstance d(xp,l) between the pont xp and the lne segment l s computed usng equaton () where x and x are ponts determnng the lne segment l. d d ( x x ) ( x x ) ( x x )( y y ) ( x x )( y y ) x x ( x x ) + ( y y ) p p wth the frst equaton beng a - vector equaton wth - pont vectors x, x and xp, and wth the second equaton correspondng to mage coordnates. The closest pont xo s the pont satsfes the equalty d (xp, xo) mn d (xp, l) {...n}. Wth the resultant correspondng pont sets the regstraton s computed usng least squares optmzaton.. Experments wth synthetc curves Synthetc planar curves were projected from model/object space to the mages usng relatve orentaton parameters as exteror ones. ICP algorthm was performed for the recovery of the relatve parameters of the photogrammetrc model and the plane parameters. Hgh senstvty has been observed to ntal values of the plane parameters. The prmary reason for these unsatsfyng results and the numercal problems confronted s a possble hgh correlaton between the plane parameters and the bendng angles of the model mages. The plane representaton could lead to numercal problems because the multplers of X and Y are nearly neglgble compared to Z multpler. We therefore suggest a dfferent representaton for the plane. n snθ cosϕ n snθ snϕ n cosθ n ( X Xo) + n ( Y Yo) + n ( Z Zo) n X + n Y + n Z where : -angle from XY plane -turnng angle around Z axs n unt vector of plane normal the dstance of the plane from orgn p () ()

3 The last representaton changes the sought parameters to, and, but leads to another problem. If the plane s horzontal so, the dervatve accordng to s nfnte, because t does not change the normal vector when multplyng wth. Therefore, when dealng wth planes close to horzontal, numerc problems are expected.. FINING FUNAMENTA MATRIX USING HOMOGRAPHY MAPPING. Homography mappng Homography mappng transfer ponts from one mage to the second mage as f they were on the plane n the object space (Hartley ). As seen n fgure. ponts on a plane are related to correspondences pont on the mages of the photogrammetrc model. In fact ths s a projectve, havng 8 free parameters. The 8 parameters can be obtaned from the 5 relatve orentaton parameters and the planar parameters. The homography matrx can be computed drectly from the relatve orentaton parameters and the plane parameters. The common way to compute the homography matrx s by determnng the coordnate system of the model parallel to the coordnate system of the left mage. By choosng ths coordnate system the rotaton matrx of the left mage s the dentty matrx and the translaton vector s zero. The rotaton matrx and the translaton vector of the rght mage can be obtaned from the relatve orentaton parameters. The homography matrx can be computed wth rotaton matrx R and the dsplacement vector T accordng to equaton (6). H (6) [ R + T n' / ] where : R rotaton matrx T - dsplacement vector n unt vector of the plane normal dstance from the orgn The homogenous coordnate are obtaned by dvdng the mage coordnate by focal length as follows: x u y v U () Usng homogenous coordnate makes the homography mappng correct up to scale. Multplyng the homogenous coordnate wth the homography matrx H s smply lnear procedure, but gettng the rght scale requres a determnaton of a scale factor. vdng the outcome vector by the rd coordnate can answer ths queston so the transformaton from one mage to the other can be wrtten as follows: Fgure. homography mappng The homography nduced by the plane s unque (see Tsa (98)), meanng that every planar curve can contrbute one homography. The homography transfer operator s lnear for homogenous coordnates and the mappng from one mage to the other s unque up to a scale factor. The homography matrx: h h h H h4 h5 h (4) 6 h h8 and the mappng from rght to left mage vectors are readly gven by U H (5) l U r where Ur, and Ul are homogenous coordnates n rght and left mages respectvely. One should notce that the mappng can be from the rght mage to the left mage and vce versa. In ths paper we have chosen the one from rght mage to the left mage. where: h x" + h y" + h x' h x" + h y" + h4 x" + h5 y" + h6 y' h x" + h y" (8) x", y" rght mage coordnate x' y' left mage coordnate h.. h8 component of the homography matrx Equatons (8) remnd the collnear equatons, but one should remember that the collnear equatons transform from object space to mage space whle those equatons transform from mage space to another mage space.. Fundamental matrx Yet another well-known relaton between two mages s the eppolar geometry. One pont from frst mage determnes lne n the second mage. The fundamental matrx defnes ths relaton wth the constran x' t F x'', obeyng the coplanar condton. Usng the homography matrx we can wrte the constran (Hx") t F x" x t H t F x for any pont on the plane that nduced the specfc homography. Hence, the matrx (Ht F) must be skew-symmetrc, namely t t H F + F H (9)

4 Note that equaton (9) shows that n fact two homography matrces provde suffcent set of lnear equatons for the fundamental matrx.. Fndng the homography matrx usng ICP In ths secton we descrbe procedure to fnd the homography matrces of free form planar curves usng ICP algorthm. Ths tme the ICP algorthm s carred out n mage plane. The chosen mage s the frst (left) mage of the photogrammetrc model. Every curve n the rght mage that was dgtzed and matched to curve from the left mage s transformed to the left mage usng ntal homography matrx. The ntal component of the homography can be computed usng regular assumpton for ntal relatve orentaton parameter and applyng equaton (6). One possble value set for ntal components of H s smply: Ho {,,,,,,,,}. After we determne the ntal component of the homography matrx we transform the curve from rght mage plane to the left mage plane and compute the closest ponts between the two corresponds curves. The regstraton s computed usng least square adjustment and evenberg-marquardt (Ftzgbbon ) algorthm s tested as well. The proposed algorthm s sketched as follows: fnd the correspondng curves n both mages set the ntal homography matrx to H{,,,,,,,,}. Transform the curve from rght mage to left mage usng the ntal matrx. Fnd the closest ponts between the curves. Compute the desgn matrx A and the error vector e for the current components of the homography matrx. Compute the update vector x for the current components. Update the homography matrx H H + x Contnue to transfer the curve and update the homography matrx untl maxmum dstance between closest ponts smaller than tolerance. Compared to the algorthm presented n the prevous secton, the current procedure reduces the dmenson of the problem from to. In addton, here, only one transformaton from rght mage plane to left mage plane s requred..4 esgn matrx for regstraton The error s actually the dstance computed between the closest ponts obtaned. The optmal stuaton s that all the dstances are zero. The desgn matrx s computed by the takng dervatves of the dstance functon wth respect to any component of the homography matrx. The dstance s computed between every pont that was transformed from rght to left mage and the closest pont to t from the correspond curve. When the curve s a free form curve then the dstance s actually computed between the pont and the closest segment to the pont. The segment from the left mage curve s for now consdered as fxed. The dervatve of equaton () for xp and yp (the transformed pont from rght mage) s: ( y y) ( x x) () where: s the dstance between ponts, of the closest segment from left mage curve. Now for the dervatves of equaton (8) for h..h8 we rewrte equaton (8) as: and: so : y y y y Nx xp Ny yp () + () + y y x x x x x x y y y y 8 xr yr ( ) x yr ( ) f Nx x x xr + f Nx x x yr + f Ny xr f Ny yr where: xr, yr are the rght mage coordnates of the specfc pont..5 Experments () The proposed procedure has been tested on synthetc data showng good convergng between the curves as shown n fgure 6. The ntal homography matrx component for ths experment was H{,,9/f,,,,,,} where f s the focal length and the value 9 s gven to keep the model scale close to the mage scale. Fgure 4 descrbes the synthetc mages of the curves. For example, one of the curves s transformed from rght mage to the left one usng ntal values for the homography matrx. After two teraton of ICP algorthm we get close to the

5 orgnal curve as can be seen n fgure 6. Fgure 6 shows enlargement of a part from the specfc curve where the ponts ndcated by 'o' are the ponts transferred from rght mage usng the computed homography matrx. Intal curve transformed Curve after two teraton Orgnal left mage curve Fgure 6. Enlargement of the converged curve Fgure. model space Further to the advantages of the homography-based algorthm mentoned above,.e., fewer transformatons requred and a reducton of the problem dmenson from - to -, probably the most sgnfcant one pertans to ts nsenstvty to the plane parameters assocated wth our planar curves. Ths unlke the algorthm presented n secton beng subject to sngulartes assocated wth nearly horzontal planar curves..6. Recoverng rotaton matrx and translaton vector drectly from homography matrx. Fgure 4. left and rght mages In fgure 5 we can see the curve transformed from the rght mage usng ntal homography matrx. Fgure 5. Curve transformed wth ntal values From equaton (9), we need homography matrces to recover the fundamental matrx. As shown n Tsa(98) the homography matrx can be decomposed usng SV and the rotaton matrx, the translaton vector and the plane parameters can be recovered. Two possble solutons for the recoverng of the relatve orentaton parameters are obtaned usng Tsa recoverng. The possblty of two dfferent solutons usng drect recoverng from the homography matrx s well suted the need of two curves provdng two homography matrces for the recovery of the fundamental matrx (Shashua ). Computng the rotaton matrx and the translaton vector from the homography matrx s as follows: t [ UV ] H s det( U ) det( V ) λ λ δ ± λ λ λ + sλδ α λ ( + δ ) β ± α α R U sβ T [ βu β V sα λ + sα U ] λ t (4) where: U U are the st and rd vector of U.. are the sngular values of H

6 Two optons to obtan the eppolar geometry of the stereo model have been shown. Whle the frst one requres two homography matrces for the recovery of the fundamental matrx and the last requre only one, but provde two dfferent solutons. Both optons lead to the need of at least two planar curves to get unque soluton. Equaton (9) provdes 6 lnear equatons for each homography matrx so east squares adjustment has to be performed havng two or more homography matrces. Equatons (4) provde two solutons for R and T for each homography matrx. When havng more than one homography matrx we need some elmnaton procedures to get the rght soluton. 4. SUMMARY Two methods for recoverng the relatve orentaton parameters from free-form planar curves have been presented and tested. Whle both perform qute well n most cases, the homographybased algorthm exhbted more robust behavor n terms of not beng senstve to some sngular confguratons observed for the alternatve method. A forthcomng paper on the subject wll present a more thorough analyss of the aforementoned methods ncludng experments wth real data. Petsa, E., and Karras, G.E., Constraned ne-photogrammetrc -Reconstructon from Stereopars. IAPRS, vol. XXXIII, Amsterdam,. Shashua, A., and Kamnsk, J.Y., On calbraton and reconstructon from planar curves. Proc. of the European Conference on Computer Vson (ECCV), June, ubln, Ireland. Tsa, R.Y., and Thomas, S.H., Estmaton Three-mensonal Moton Parameters of a Rgd Planar Patch. 98. IEEE Trans. Acoust., Speech, Sgn. Process., 9:4-5. Tsa, R.Y., Thomas, S.H., and Zhu, W.., Estmaton Tree- mensonal Moton Parameters of a Rgd Planar Patch II: Sngular value decomposton. 98. IEEE Trans. Acoust., Speech, Sgn. Process., : Zalmanson, H.G., Herarchcal Recovery of Exteror Orentaton from Parametrc and Natural - Curves.. IAPRS Vol. XXXIII, Amsterdam. REFERENCES Besl, P. J., and McKay, A Method for regstraton of shapes. IEEE transacton on pattern analyss and machne ntellgence. Vol. 4, no., Feb. 99. Ftzgbbon, A.W., Robust regstraton of and pont sets. In proc. Brtsh machne vson conference. Manchester, sep.. pp Habb, A., and Kelley., Sngle-Photo Resecton Usng the Modfed Hough Transform. Photogrammetry Engneerng and Remote Sensng. Vol. 6, No. 8 August, pp Habb, A., and Kelley,., Automatc Relatve Orentaton of arge Scale Imagery Over Urban Areas Usng Modfed Iterated Hough Transform.. ISPRS Journal of Photogrammetry and Remote Sensng Vol. 56, pp Habb, A., and ee, Y., and Morgan, M., Automatc Matchng and Three-mensonal Reconstructon of Free-Form near Features from Stereo Images. Photogrammetry Engneerng and Remote Sensng. Vol. 69, No., February, pp Hartley, R., and Zsserman, A., Multple vew geometry n computer vson. Cambrdge unversty press.. Kraus, K., Photogrammetry vol., Ferd. ummler Verlag. Bonn 99. Kubk, K., Relatve and Absolute Orentaton Based on near Features ISPRS Journal of Photogrammetry and Remote Sensng Vol. 46, pp Ma, S.., Concs-Based Stereo, Moton Estmaton, and Pose etermnaton. 99. Internatonal Journal of computer vson : pp. -5. Mkhal, E., near features for photogrammetrc resttuton and object completon. Integratng Photogrammetrc Technques wth Scene Analyss and Machne Vson, SPIE Proc. No. 944, pages 6-, Orlando, F, 99.