Study on the Effect of Object to Camera Distance on Polynomial Expansion Coefficients in Barrel Distortion Correction

Transcription

1 Study on the Effet of Objet to Camera Distane on Polynomial Expansion Coeffiients in Barrel Distortion Corretion H. Tian, T. Srikanthan, K. Vijayan Asari, S. K. Lam Center for High Performane Embedded Systems, Nanyang Tehnologial University, Singapore Department of Eletrial and Computer Engineering, Old Dominion University, Norfolk, USA Abstrat Videoendosopy is beoming inreasingly popular in surgial proedures. Wide-angle lenses are ommonly employed in suh appliations for enhaned viewing apability. However, images aptured with these lenses suffer from barrel distortion. D distortion orretion for images aptured with wide-angle lenses has been widely investigated. In linial appliations, it is neessary to inorporate orretion tehniques that are independent of the distane from the objet to the amera lens. In this paper, we prove that for a wide-angle amera lens with fixed foal length, the distortion orretion oeffiients remain the same for distanes within the minimum and maximum range (depth of field). Experiments have also been performed to verify this. Keywords: barrel distortion, distortion orretion, expansion oeffiients. Introdution Minimally invasive therapy (MIT) is beoming inreasingly popular in surgial proedures as it minimizes the destrution of healthy organs and tissues through the use of natural or artifiial orifies of the body. Videoendosopy has beome one of the most ommonly aepted forms of diagnosti and therapeuti proedures with the advent of miniature CCD amera and assoiated miroeletronis [][]. The eletroni endosope has an exellent advantage over the onventional endosope using fiber image guide as it failitates observation, doumentation, and eletrial manipulation of the internal struture images of the gastrointestinal trat. In these endosopes, ameras with wide viewing angle lens (fisheye lens) are used to enhane the imaging apability by apturing a large field in a single image (i.e., large areas of the muosa an be visualized rapidly [3]). However, images formed with these lenses suffer from spatial distortion, referred to in optis as barrel distortion due to the wide-angle nature of the endosope s objetive lens. Barrel distortion introdues nonlinear hanges in the image, due to whih, image areas near the distortion enter are ompressed less, while areas farther from the enter are ompressed more. Beause of this, the outer areas of the images look signifiantly smaller that their atual size. This inhomogeneous image ompression introdues signifiant errors in the results obtained during feature extration. Continuous estimation of quantitative parameters, suh as area and perimeter, is of onsiderable importane while performing linial endosopy. Unless the distortion is orreted, estimation errors ould be very large [4]-[6]. Moreover, the distortion auses ompliations while using token mathing tehniques for pattern reognition. Distortion orretion is also a prerequisite for the amera alibration to obtain extrinsi and intrinsi amera parameters [7][8]. Several researhers have presented various mathematial models of the image distortion and tehniques to find the model parameters to omplete the distortion-orretion proedure. Simith et. al. [9] gave a formulation in whih distortion was assumed to be purely radial, and orthogonal Chebyshev polynomials were used to determine the model parameters. Hideaki et. al. [0] presented a different method for estimating the model parameters, in whih a moment matrix was obtained from a set of image points, and distorted grid lines in the image were straightened on the basis of the smallest harateristi root of the moment matrix. Vijayan et. al. [] proposed a new tehnique based on least square estimation to obtain the oeffiients of the orretion polynomial. Furthermore, a method for aurate determination of the ritial points on the alibration grid, based on a dual-step approah is also presented. The proposed tehnique is independent of the orientation of the alibration hart, hene preise plaement of the hart and the plaement errors in distortion-orretion formulation are avoided. The proposed distortion orretion tehnique is muh faster than the existing tehniques and gives suffiiently aurate results, whih makes the onept of on-line alibration of the endosopi amera feasible. This is desirable to enhane the auray of the quantitative results obtained from endosopi images by mahine analysis.

2 IEEE Pro. 5 th Southwest Sym. Image Analysis and Interpretation (SSIAI), April 00 However, all the methods mentioned above have been presented for D distortion orretion, whereby the distane from the objet to amera lens is always onstant. For pratial videoendosopy, it is imperative to explore the effet of varying this distane on the expansion oeffiients of the D distortion orretion algorithms. In this paper, we prove that for a fixed foal length wideangle amera, the set of expansion oeffiients obtained at a ertain distane for D distortion orretion has no relation with the distane from the objet to amera lens. Simulations have also been performed to verify this.. -D distortion orretion theoretial model Vijayan et al. [] proposed a new tehnique based on least square estimation to orret the non-linear distortion. The distortion orretion algorithm assumes that the distortion is radial about the distortion enter of the image aptured by the wide-angle amera lens. Although nonlinear magnifiation of the distorted endosopi image in two dimensions is needed to orret the barrel distortion, the assumption preludes the loss of generality, as a typial endosope lens is irularly symmetri within narrow preision limit [9]. This assumption simplifies the model by onverting a -D distortion problem into a onedimensional (-D) problem. Let the distorted and orreted ( or undistorted) image spaes be represented by (U, V ) and (U, V), respetively, and the distortion enter and the orreted enter by (u, v ) and (u, v ). The distortion enter (u, v ) is a point in the distorted image spae suh that the straight lines in the objet spae passing through it remains straight in the image spae. The orreted enter (u, v ) is a point in the orreted image spae about whih the expansion of distorted image gives a final orreted image. In the distorted image spae, magnitude ρ of a vetor P from the distortion enter to any pixel loation (u, v ) and the angle θ made by this vetor from the horizontal U -axis are given by magnitudes of the two vetors in distorted and orreted images as n ρ ρ' (3) N a n n where a n s are the expansion oeffiients. As the distortion has been assumed to be purely radial, there will be no hange in the arguments of the orresponding vetors P and P, i.e., θ θ. After obtaining the magnitude of the new vetor, the new pixel loation in the orreted image spae an be alulated as u u + ρ osθ ', v v + ρ sinθ ' (4) To map eah pixel from the distorted image spae onto the orreted image spae, there are N + 4 unknowns, viz., N expansion oeffiients (a n s), distortion enter (u, v ), and orreted enter (u, v ). The proposed distortion orretion method involves two steps listed as follows: ) Estimation of Distortion and Correted Center A reasonably orret estimation of the distortion enter is essential for effetive determination of the expansion oeffiients. The distortion enter is a fixed point for a partiular amera and, one alulated, an be used for all the images obtained from that amera. The distortion enter is omputed based on the adjaent rows and olumns of opposite urvatures. Please see Fig.. (r) (r) ( y ) k (r) () () ( y k () k () () () ) (x ), y (r) k ρ' ( u' u ') + ( v' v ') v' v ' θ ' artan u' u ' () (r) (r) ( y ) Let the same pixel be assigned to a new loation (u,v) in the orreted image spae and the magnitude ρ and argument θ of the orresponding vetor P drawn from the orreted enter to the new pixel loation are ρ ( u u ) + ( v v ) v v θ artan u u The objetive of the mathematial model is to obtain a relation between the vetors P and P. An expansion polynomial of degree N is defined to relate the () Fig.. Best-fit polynomial urves of adjaent rows and olumns ( ) Their urvatures are defined as k, k, k ( ) and k, respetively. ( y ), ( y ), ( ) ( ) ( ) ( ) ( y ), ( y ) is the stationary point on the urve at whih the above urvature is omputed, respetively. The distortion enter

3 IEEE Pro. 5 th Southwest Sym. Image Analysis and Interpretation (SSIAI), April 00 ' ' ( u, v ) is estimated by interpolating the four ( ) ( ) urvatures k, k, k, and k as ' u ( ) ( ) ( ) ( k x + k x ( ) ( ) k + k ) ( r ) ( r ) ( r ) ( r ) ' k y + k y v (5) ( r ) ( r ) k + k parallel with eah other, O represents the lens and d, d are two different distanes from the lens to the objet plane, whereby distanes d, d are not nearer or beyond the depth of field. OO and the size of image plane is onstant for a partiular wide-angle amera with a fixed foal length. Assuming a set of polynomial expansion oeffiients a n s have been obtained for distane d. We will now try to prove that the same set of expansion oeffiients a n s an be applied to any distane range. The orreted image enter is found based on the riterion that in the orreted image, pixel distanes between the dot enters should be the same for all the grid lines in the horizontal and the vertial diretions. The orreted image enter is estimated by applying the expansion polynomial to this pixel loation in the distorted image, whih is obtained by iteratively minimizing the variation in distanes between the test dot enters in the orreted image. ) Estimation of Expansion Coeffiients The expansion oeffiients are estimated on the basis of the degree of straightness of the points, whih lie on a straight line before imaging. These are estimated in the distorted image spae by straightening the grid lines of a distorted grid image. The least squares estimation and a line searh approah of global onvergene for the iterative proedure are used to obtain the optimum expansion oeffiients a n s as β an( Δ + ) an( + αn E(, E( an for n, N (6) where α is the onvergene rate parameter, β is the expansion index, E( is the total error for the whole grid image and E/ a n is the error gradient. Here, α is hosen to ensure that for every (Δ+)th iteration, E(Δ+)<E(. 3. Effet of objet to amera distane on polynomial expansion oeffiients In this setion, we will prove that for a wide angle amera with fixed foal length (i.e. keeping the aperture onstant), and image plane size, the expansion oeffiients a n s obtained at a ertain distane is appliable for orreting distorted images aptured at varying distane. Fig. desribes a simplified relation between the image plane, lens and objet plane. Assuming they are in l l a O' O O O Im age Plane d d Objet Plane Fig.. Basi relations of imaging optis Proof: Without loss of generality, the -D objet plane an be regarded as a omposition of infinite ontinuous parallel lines. Assuming that at distane d, the -D objet plane an be fully aptured onto the image plane. This means that eah parallel line in the objet plane orresponds to a urve in the image plane. Let s assume that the size of the -D objet plane is S and l is an arbitrary parallel line in the plane at distane d. Also, is the orresponding urve in the image plane of the line l and a, the distane from O to the line l. When the objet plane goes nearer to the lens at distane d, only one part of the -D objet plane d ( ) S an be d aptured onto the image plane. For line l, there must exist a orresponding line l in this nearer objet plane with a distane d a from O, whih is refleted onto d the same urve as the line l. From the known ondition that a n s an be used to straighten the urve, the same applies to orreting the distorted image aptured at distane d. The similar proof an be done for further distane ranging from d to the maximum distane. Hene, we an onlude that the distane of objet to amera has no effet on polynomial expansion oeffiients in barrel distortion orretion for a wideangle amera with a fixed foal length.

4 IEEE Pro. 5 th Southwest Sym. Image Analysis and Interpretation (SSIAI), April 00 This an also be understood easily by the following explanations: For a fixed foal length wide-angle amera, the size of the image plane is always kept the same. No matter how the distane ranges from the objet to the amera, the urvature and the loation of eah pixel in the distorted image array are kept the same throughout in the image plane. The only differene is within the range of the depth of field, when the objet is nearer, less information will be aptured in the image plane, but the image is bigger. If the objet is further, more information will be aptured in the image plane, but the image is smaller. Fig.3. (a) Distorted image at distane of 8.6m (b) Correted image orresponding to (a) 4. Experimental results To verify the above theory, an experimental grid ontaining a retangular array of dots was used. A amera sensor with a resolution of 480x640 pixels and a wideangle lens of 58 degree was used to apture the images. The amera was oriented perpendiular to the grid surfae at a distane of 8.6 m. Fig. 3(a), learly shows the barrel distortion effets of the wide-angle amera. We have known that the order of expansion oeffiients N taken as 4 an obtain good distortion orretion effet []. They are omputed as follows: a a a a The distorted image enter is [59, 334]. The orreted image size is and orreted image enter is [595, 768]. These parameters an be diretly used for the distortion orretion of the images. The orreted image is shown in Fig. 3(b). Sine magnifiation is ontrolled in order not to lose the information around the enter of distortion, the marginal area is enlarged and the whole image beomes larger than the original one. From the omparison with the original image, the effet of the orretion is obvious. In the linial ase, imaging distane is variable. Therefore, it is neessary to investigate if the parameters obtained above are appliable to images reorded at different imaging distanes. Fig. 4 and Fig. 5 show the images reorded at the distane 4 m and m from the wide-angle amera and the orreted effets respetively. We an see that the same set of expansion oeffiients lends well for these different distanes. Fig. 4. (a) Distorted image at distane of 4m (b) Correted image orresponding to (a) Fig. 5. (a) Distorted image at distane of m (b) Correted image orresponding to (a) 5. Conlusion In this paper, we study the relationship between the polynomial expansion oeffiients and the distane from the objet to the amera lens in barrel distortion orretion. We found that for wide angle amera lens with fixed foal lengths, whereby the size of the sensor plane is always onstant, the urvature of eah pixel in the image array will remain the same irregardless of the distane of the objetive image to the amera lens. Hene, the expansion oeffiients obtained at the ertain distane an be used to orret the images aptured at any distane within the depth of field range. This provides the impetus for employing existing distortion orretion methods to linial appliations.

5 IEEE Pro. 5 th Southwest Sym. Image Analysis and Interpretation (SSIAI), April 00 Referene [] H. E. Murphy, and D. W. Lake, ``Video Endosope/boresope Cameras'', Tehnial Paper, IQ87-375, 987. [] W. J. Smith, Modern 0ptial Engineering the Design of 0ptial Systems, MGraw-Hill, In. USA, 990. [3] H. Kato and J. P. Barron, Eletroni Videoendosopy, Switzerland: Harwood, 993. [4] A. Sonnenberg, M. Giger, L. Kern, C. Noll, K. Stuby, K. B. Weber, and A. L. Blum, How Reliable Is Determination of Uler Size by Endosopy, Brit. Med. J., vol. 4, 979, pp [5] Y. Hatada, S. Iwane, H. Tohno, T. Baba, A. Munakata, and Y. Yoshida, A New Method for the Measurement of the Gastri and Coloni Lesions with an Eletroni Endosope and Image Proessor, Gastrointestinal Endosope, vol. 37, 99, pp.75. [6] C. Margulies, B. Krevsky, and M. F. Catalano, How Aurate Are Endosopi Estimates of Size, Gastrointestinal Endosopy, vol. 40, 994, pp [7] Q. M. Lin and S. D. Ma, Parametri and Nonparametri Approahes for Camera Calibration: Analysis of Imaging Errors and Their Compensation, in Pro. Se. Asian Conf. Computer Vision, Singapore, 995, pp. II 05-II 09. [8] M. Li and L. Jean-Mar, Some Aspets of Zoom Lens Camera Calibration, IEEE Trans. Pattern Anal. Mahine Intell., vol. 8, Nov. 996, pp [9] W. E. Smith, N. Vakil, and S. A. Maislin, Corretion of Distortion in Endosopi Images, IEEE Trans. Med. Imag., vol., no., Mar. 99, pp.7-. [0] H. Haneishi, Y. Yagihashi, and Y. Miyake, A New Method for Distortion Corretion of Eletroni Endosope Images, IEEE Trans. Med. Imaging, vol. 4, no. 3, Sept. 995, pp [] K. V. Asari, S. Kumar, and D. Radhakrishnan, A New Approah for Nonlinear Distortion Corretion in Endosopi Images Based on Least Squares Estimation, IEEE Trans. Med. Imaging., vol. 8, no. 4, Apr. 999.