Texure and Disincness Analysis for Naural Feaure Exracion Kai-Ming Kiang, Richard Willgoss School of Mechanical and Manufacuring Engineering, Universiy of New Souh Wales, Sydne NSW 2052, Ausralia. kai-ming.kiang@suden.unsw.edu.au, r.willgoss@unsw.edu.au Alan Blair School of Compuer Science and Engineering, Universiy of New Souh Wales, Sydne NSW 2052, Ausralia. blair@cse.unsw.edu.au Absrac One of he basic requiremens for auonomous navigaion in an unexplored and ofen complex environmen is o be able o lock on o naural feaures. This paper presens a mehod for exracing feaures disincive enough o navigae wih. The mehod consiss of hree pars. Firsl i selecs a se of ineres poins from he images which are invarian o mos changes in condiions; secondl i analyses he exure disribuion of he local ineres regions around ineres poins seleced; hirdl i picks ou disincive feaures from he original se of ineres poins. The mehod has been implemened wihin a SLAM framework designed for use in a exure-rich environmen such as he Grea Barrier Reef. The resuls have shown ha his mehod has significan advanages over oher widely used mehods in his specific environmen. The speed of implemenaion is faser and he number of feaures needed o process is reduced. Inroducion Auonomous navigaion in an unexplored environmen is more challenging han in one ha is conrolled because of exra effor needed o make sense of sensor inpus. In paricular, underwaer environmens are mosly unexplored and do no have GPS access. Therefore navigaion in hese environmens requires he use of mehods such as Simulaneous Localizaion and Mapping (SLAM) [Csorba, 997; Williams e al., 2002]. However, mos exising SLAM algorihms have relied on poin-based arificial landmarks ha do no exis in an unexplored environmen. SLAM can be unreliable if naural landmarks are used when hey lack descripive represenaion. Developmens from compuer vision research exrac feaures wih represenaions ha are invarian o scaling, disorion and perspecive [Carneiro and Jepson, 2002; Mikolajczyk and Schmid, 2002; Tuyelaars and Van Gool, 2000]. These developmens could poenially be used for robo navigaion and are already capuring aenion from he roboic communiies [Kragic and Chrisensen, 2005]. In paricular, Scale Invarian Feaure Transformaion (SIFT) [Lowe, 2004] was repored as a mehod robus in represening feaures. Is descripors were claimed o be invarian under changes in scale, roaion, shif and illuminaion condiions. A performance es comparing differen feaure exracion mehods was repored by Mikolajczyk and Schmid [2005] ha indicaed SIFT generally performed he bes amongs hese mehods. Moreover, here was a modificaion o SIFT using principle componen analysis ha improved is performance furher [Ke and Sukhankar, 2004]. However, as hese mehods were mosly designed for non real-ime objec recogniion purposes, compuaional efficiency may no have been he major concern. The mehods ended o generae a large number of feaures ha maximized accuracy and sabiliy. For real-ime SLAM applicaions, i is compuaionally infeasible o compare such large ses of feaures from a series of images ha have been capured. We presened Texure Analysis (TA) [Kiang e al., 2004] for feaure exracion purposes ha was designed o improve performance speed. The descripors for TA represened he frequency disribuion of he local ineres region of an ineres poin. This mehod is poenially an appropriae choice of descripor for a exure-rich environmen such as found a he Grea Barrier Reef because here are many exures o work wih. However, TA is designed o be a generic soluion and can also be used for navigaing a mobile robo hrough a sree, guiding an aircraf and negoiaing wooded scenery. An improved version of TA is now presened in his paper. Besides requiring represenaive feaures, SLAM also requires a selecion mehod ha can minimise he se of feaures picked for similariy maching. For his reason Disincness Analysis (DA) was devised as a echnique o minimize he number of feaures seleced [Kiang e al., 2005]. DA can exrac he disincively rare feaures from hose iniially seleced ha minimises he number ha are needed for processing. In his paper, besides presening he improved version of TA, furher work on combining TA and DA is presened. Moreover, resuls are presened ha have been
obained from he implemenaion of he combined TA/DA mehod in a SLAM framework for use in an underwaer environmen. 2 Ineres Poin Selecion The mehod of feaure exracion presened here consiss of hree sages, namely Ineres Poin Selecion, Texure Analysis (TA) and Disincness Analysis (DA). This secion describes how ineres poins are seleced. Ineres poins should be invarian o roaion, shif, scale, illuminaion and affine ransformaion such ha, when he examining region is o be analysed again under differen condiions, he same poins would be eviden. Two ways of selecing ineres poins have been repored are Harris Corner [Harris and Sephen, 988] and exrema in Difference of Gaussian (DOG) [Lindeberg, 994], which are he mos widely used mehods. Bu Mikolajczyk and Schmid [2005] have poined ou ha he mehod of selecing ineres poins is independen of he choice of mehod for generaing feaure descripors. In his paper, exrema of DOG, he mehod used in SIFT [Lowe, 2004], has been chosen for ineres poin selecion and will now be briefly described. A DOG image is compued by convolving he original image I( wih a Gaussian funcion of sandard derivaion σ, o obain a blurred version of he original image B(σ). Tha is: B( σ ) = G( σ ) I ( () where * is he convoluion operaor in x and y plane. The Gaussian funcion G is applied o he resuling image B sequenially obaining B(kσ) where kσ represens he number of convoluions applied o he original image. The DOG image is hen defined o be he difference of hese wo images. Tha is: D( σ ) = ( G( kσ )) I ( = B( kσ ) B( σ ) Noe ha he variable σ depends on he complexiy of he image. Besides applying Gaussian convoluion, which resuls in a blurring effec, down-sampling is also applied. Images are downsized in raios of wo. A pyramid of DOG images is hen consruced. From he pyramid, exrema can be seleced direcly by comparing each pixel wih is neighbouring pixel in spaial and scale domain for a prese radius around he pixel. These DOG exrema become he ineres poins. 3 Texure Analysis Afer selecing ineres poins, he local region of each poin was reaed as a feaure candidae ha required furher analysis. The local regions were each limied o 32 x 32 pixel recangular segmens. Even hough he square segmens were chosen o simplify Fourier Transform calculaion, a Hanning window was applied on he ransformed regions making hem approximaely circular and cenred a an ineres poin such ha invarian properies were preserved. Differen exures show up as differen paerns (2) in he Fourier Transform. TA used he Fourier Transform of he local regions of he ineres poins as a basis of is choice for descripors. This choice is suiable for represening feaures from images of a exure-rich environmen. The Discree Fourier Transform adjused by a Hanning window was calculaed as follow: S pg m ( ω ) = q( k) x[ k] e W k W jω k where W is he original local region and q(k) is he Hanning window funcion defined as: πk + cos( ) q( k) = τ for k < τ, 2 (4) = 0 oherwise. The resuling ransform, which is also an image, represened he disribuion of frequency of he ineres region. The Hanning window, besides generaing a circular region, was also needed o smoohly correc he boundary effec of he Fourier Transform. This ransformed image was hen pariioned ino 26 useful regions discarding area conaining sparse informaion. Firsl i was divided ino concenric semi-annuli one pixel wide. Then for he second and hird smalles annuli, hey were furher subdivided ino 4 and 8 angular secors respecively. A diagram illusraing his pariioning scheme is shown in Figure. The pariions were more densely disribued in he cenre ha represened he lower frequencies. The reason for his is ha, in naural images, lower frequencies would nearly always conribue more han higher frequencies. A more deailed descripion of he lower frequencies is herefore imporan. Moreover, only half of he plane was needed o be analysed because Fourier Transforms are always symmerical. In order o have roaional invariance, he angular segmening process for he inner circles was referenced o he mean gradien direcion. The calculaion process for his mean gradien direcion was he same as in SIFT bu wihou he need o resample he gradien image. Figure : The 26 Pariions of he Frequency Disribuion. (3)
Each value in he Fourier Transform was a complex number. Adding magniudes of each of hese complex values wihin a pariion gave he srengh of ha pariion. By applying he calculaion o all 26 pariions, a frequency disribuion was obained. The resuling 26 values ha represened exure properies were hen normalised and could hen be used as he descripors for a paricular ineres poin. 4 Disincness Analysis 4. The Probabiliy of Occurrence Numerous ineres poins were normally generaed as feaures from he firs selecion process described in Secion 2. In he lieraure, none of he feaure exracion mehods looked o minimizing his number afer he ineres poins had been ransformed ino descripors. This issue is addressed by DA proposed here and is described in his secion. The erm disincness has been used in reporing research referring o a special propery of a paricular ype of ineres poin such as he exrema of DOG or Harris Corners. The special propery usually refers o invarians in condiions and sabiliy. I is however no relaed o he frequency of occurrence of such poins wihin an image or an environmen. In hese mehods, he number of ineres poins is no necessarily minimized. Ineres poins such as he exrema of DOG are ofen common wihin an image. The number can be as high as housands for a 640x480 pixel image. Usuall in objec recogniion, i is desirable o exrac more raher han less ineres poins o enable robusness in maching. However, in real-ime navigaion, he compuaion ime is a criical requiremen. If all ineres poins are o be used as landmarks, since he compuaion ime for mos cases is proporional o O(N 2 ), where N is he number of sae variables needed o represen he landmarks and he robo pose, he compuaion ime is grealy increased. Therefore he need o minimize he number of ineres poins while, a he same ime, no penalising he performance of recogniion is he main objecive. However, a he raw pixel level of an image, i is difficul o find a cerain ype of poin ha rarely occurs and is invarianly sable. On he oher hand, feaure ransformaion provided a more expressive represenaion for describing each ineres poin. Hence, i is preferable o selec he disincive se of ineres poins a he TA level of absracion. Figure 2: Simple diagram of a disincive objec among oher less disincive objecs. The quesion hen arises as o how a few relevan feaures ou of a poenially large se should be seleced. For example, in Figure 2, i would be bes o remember he cenre objec because i is he only one ha is unique. If one selecs any of he oher objecs ha are similar o each oher, i will be hard o disinguish beween hem laer on. Since he descripors represen he feaures, hey become elemens of feaure vecors in he descripor parameric space. Disincness can be judged from analysing and comparing hese vecors. If we consider all of he descripors in he feaure vecors as independen random variables, he probabiliy of occurrence for each feaure can hen be calculaed by finding a model for is probabiliy densiy funcion. In he simples case, he disribuion could be approximaed o a muli-dimensional Gaussian. This approximaion is o be jusified in Subsecion 7.2. The probabiliy of occurrence of a feaure can hen be calculaed as follows: f ( x) = x (2π ) m de where µ is he mean vecor: and C is he covariance marix: n R j exp ( x µ ) C ( x µ ) ( C) 2 R j (5) µ = x[ n] (6) R n C = ( x[ n] µ ) ( x[ n] µ ) (7) R j DA can hen be made on he basis ha he lower he probabili he more disinc a feaure is judged o be. 4.2 Global Disincness DA is a process of minimizing he number of feaures while reaining sabiliy of analysis. Sabiliy refers o he abiliy o pick ou he same feaure invarian o any changes in shif, roaion, scale and illuminaion. However, since feaures deeced in one image need no be he same as in subsequen images, DA mus herefore range over many images o embody global disincness. In doing so, i is hen possible o selec feaures ha are boh disincive and likely o be found in muliple images in he environmen capured a differen imes and locaions. Denoing he mean and covariance for he global disincness by µ and C respecively and by µ c and C c for he curren image, µ is obained and updaed using he following formula: µ = µ λ + µ ( ) (8) c λ where λ is he innovaion facor, which deermines how much he sysem relies on hisory versus new daa. C is obained and updaed using he following formula: C T ( E ( XY) µ ( x) µ ( j = (9) where E(XY) is he expecaion value of he produc of wo dimensions X and Y, which can be calculaed from:
E ( XY ) = E( XY ) λ E( XY ) c ( λ) (0) E(XY) - and E(XY) c can be obained by rearrangemen of he previous formulae using E(XY) as he subjec wih he appropriae µ and C. Equaions 8 and 9 are used for ieraively updaing. To iniialise µ and C, hey are assigned o be equal o µ c and C c for he firs inpu image. µ and C require he sysem o run over a series of images in order o converge o he rue global disincness. A pracical soluion is o ake a safe walk in he environmen of ineres before using ha daa for exploring more of he environmen. 5 Maching feaures across images Having obained he disincive se of feaures exraced from each image, i is hen possible o mach feaures across differen images. Maching feaures requires calculaing a noional disance beween he wo feaure vecors. In his analysis, disance is defined as he Euclidean disance of he feaure vecors. The maching sraegy is defined by using a hreshold funcion relaing o he closes and second closes mach of a paricular feaure as follows: D D / D D. () A B A C < 7 Resuls In he following experimens, TA presened in Secion 3 and DA presened in Secion 4 are considered as wo independen mehods. For reference, some of he resuls presened here were compared wih SIFT when applicable. TA is considered as an alernaive o SIFT while DA is considered as an add-on o boh TA and SIFT. Figure 3: The underwaer vehicle. (Couresy of ACFR, Universiy of Sydne Ausralia) where D A is a feaure vecor on image one; D B and D C are closes and second closes feaure vecors from anoher image respecively. If he above inequaliy is rue, D A is considered o be he same as D B. Since every feaure is originally an ineres poin ha is eiher a maximum or a minimum of DOG, his propery could also be uilized for maching purposes. I is herefore beneficial o separaely soring he maxima and minima and find maches only wihin he same ype. 6 Experimenal A submersible vehicle (see Figure 3) which was used for capuring underwaer images and acquiring sonar daa simulaneously (couresy of ACFR, Universiy of Sydne Ausralia) was chosen o be he source of images used in he presen analysis. The configuraion of he submersible was se such ha he camera was always looking downwards ono he sea floor. This configuraion minimised he geomerical disorion ha could had been caused by differen viewpoins. The vehicle acquired images and sonar daa as i ravelled underwaer. Some of he images are shown in Figure 4. More deails of he configuraion of his vehicle and implemenaion can be found in [Williams and Mahon, 2004] where, using he sonar daa and he images, he pah ha i ravelled could be rerieved as shown in Figure 5. The original capured underwaer images were ransferred o an exernal compuer for offline esing and o be available o research groups. In his paper, TA and DA were wrien in C++, running code ha was embedded in such a way ha i could run as he fron end o SLAM-conrolled guidance. A sequence of approximaely 3000 images was used for esing he efficacy of he presen echnique. Figure 4: Some Images capured. (Couresy of ACFR, Universiy of Sydne Ausralia)
Figure 5: A pah of he underwaer vehicle was ploed using he images and sonar daa. The image frame numbers along he pah is shown wih seleced images (Original daa derived from [Williams and Mahon, 2004] and processed ino graph form by he presen auhors). 7. Maching Capabiliy The Maching Capabiliy of TA was esed firs. Boh SIFT and TA were used o find maches across he sequence of images. In order o have relaively few feaures on an image such ha inspecion of maching capabiliy was kep efficien, DA was applied o boh SIFT and TA. For example, one of hese image pairs, which conained less han 0 feaure maches, is shown in Figure 6. These feaures were mached using TA. Figure 7 shows he resuls for a se of image pairs where he maching capabiliies TA are esed. The resuls for SIFT are no shown because he false posiive were zero. For TA, he percenage of correc maches was 98.5%. The disadvanage of SIFT was he ime required for processing whereas he processing ime for TA was approximaely /3 of ha for SIFT. This was due mainly o he lower dimensionaliy of TA. Provided he number of disincive feaures could be limied o around 0 per image, a 98.5% accuracy per mach in TA was an accepable level of maching capabiliy and virually equivalen o he capabiliy of SIFT. In Figure 8, he eigenvalues of he 26 dimensional descripors of TA and he 28 dimensional descripors of SIFT are ploed and arranged in descending order. Since heir dimensions were differen in lengh, hese dimensions were adjused o fi he horizonal axis of he graph. This graph showed ha he eigenvalues for TA were spread more evenly across descripor space han for SIFT. This resul suggesed ha he choice of he descripors for TA conveyed more informaion han SIFT per descripor. Number of Feaures Figure 6: Feaure Maching using TA. 35 30 Correc False Posiive 25 20 5 0 5 0 4 7 0 3 6 Image Pair Number Figure 7: Correc Mach and False Posiive for Texure Analysis 9 22 25 28 3
Weighed Value 0.8 0.6 0.4 0.2 0 SIFT Eigenvalue (scaled) TA Figure 0: Disribuion of SIFT feaures in he op 3 principle direcions. Figure 8: Relaive Eigenvalue magniudes of SIFT and TA feaures. 7.2 Disincness The key assumpion for esing DA was ha he feaure descripors used were based on Gaussian disribuion. If he disribuion was no Gaussian, DA may no obain a valid disincness of an exraced feaure. I is imporan o noe ha, provided he disribuion was unimodal, he disincness calculaed would no significanly deviae from ha obained if he disribuion was Gaussian. In Figure 9, he disribuion for a large se (~3000) of exure feaures generaed from differen underwaer images in he series is ploed. Only he op 3 principle direcions of he disribuion are shown limiing a principle componen analysis o he mos significan. I can be seen ha he disribuion was close o a Gaussian. Figure 9: The Disribuion of a large lis of feaures ploed in he op 3 principle direcions of Texure Analysis. For comparison, he same feaures for a SIFT analysis are ploed in Figure 0. As can be seen, SIFT is a bimodal disribuion. Such a disribuion was caused by he deviaion beween he wo ypes of feaures generaed by SIFT, namely maxima and minima of DOG. By ignoring he sign during SIFT gradien calculaions, he new disribuion can be reploed and is shown in Figure. In so doing, he disribuion of SIFT feaures, wih he sign of he gradien ignored, urned ou also o be approximaely Gaussian. Based on he resuls presened, he disribuion for boh TA and SIFT could be assumed o be Gaussian.. Therefore DA could poenially be applied o any local descripor-based feaure exracion echnique. Figure : Disribuion of SIFT (Gradien sign ignored) feaures in he op 3 principle direcions. 7.3 Sabiliy A final es was conduced o check on he sabiliy of chosen feaures. I is re-emphasized ha by sable, we mean ha he same feaure should be picked ou invarian o any changes in shif, roaion, scale and illuminaion. A new series of image pairs were used and o which TA was applied. These image pairs conained overlapping regions such ha DA had o range over images in which i was known feaures were coninuous. DA was applied o each image and inspecion made wihin overlapping regions o coun he number of disincive feaures ha appeared wihin a few pixels in corresponding locaions of he images. By comparing his number wih he number of feaures ha did no correspond in boh of he images, a measure of sabiliy was obained. Figure 2 shows he couns of feaures ha were regarded as sable and unsable in overlapping pars of images. Approximaely half of he feaures seleced as disincive in one image appeared in boh images. This raio is largely independen of he number of feaures deeced and is significanly influenced by he sabiliy of he iniial selecion process using exrema of DOG described in Secion 2. The sabiliy of he iniial DOG selecion process is of iself low and is a major facor in limiing he raio shown in Figure 2. Based on his fac, he raio was deemed a relaively high hi rae for racking disincive feaures hrough image sequences. In a SLAM cone i only requires o mach a few sable feaures correcly across wo images o rack from image o image and evenually enable loop closure. I was concluded ha he resuls showed significan promise for enabling map building in a SLAM conex.
Number of Feaures 0 9 8 7 6 5 4 3 2 0 Sable Unsable 3 5 7 9 3 5 7 9 Image Pair Number Figure 2: An analysis of finding sable landmarks over 20 pairs of images. 8 Conclusion The work presened in his repor showed ha Disincness Analysis and Texure Analysis are suiable choices for use as feaure exracion echniques especially in a exure-rich environmen such as he Grea Barrier Reef. Texure Analysis can be used for exracing feaures from naural images and represen hem wih invarian descripors. Disincness Analysis can hen be used for reducing he se of feaures generaed and mainaining he maching capabiliy wih an accepable level of performance. Based on he resuls repored here, he opporuniy of having a fas and robus feaure exracion echnique could be considered as feasible. Acknowledgemens This work is financially suppored by he Ausralian Cooperaive Research Cenre for Inelligen Manufacuring Sysems & Technologies (CRC IMST) and by he Ausralian Research Council Cenre of Excellence for Auonomous Sysems (ARC CAS). References [Carneiro and Jepson, 2002] G. Carneiro and A. D. Jepson, Phase-based local feaures, 7h European Conference on Compuer Vision, Copenhagen, vol., pp. 282-296, 2002. [Csorba, 997] M. Csorba, Simulaneously Localisaion and Mapping, PhD hesis of Roboics Research Group, Deparmen of Engineering Science, Universiy of Oxford, 997. [Harris and Sephen, 988] C. Harris and M. Sephen, A combined Corner and edge deecor, Alvey Vision Conference, pp 47-5, 988. [Ke and Sukhankar, 2004] Y. Ke and R. Sukhankar, PCA-SIFT: A more Disincive Represenaion for Local Image Descripors, Compuer Vision and Paern Recogniion, 2004. [Kiang e al., 2004] K. Kiang, R. A. Willgoss, A. Blair, Disincive Feaure Analysis of Naural Landmarks as a Fron end for SLAM applicaions, 2nd Inernaional Conference on Auonomous Robos and Agens, New Zealand, pp. 206-2, 2004. [Kiang e al., 2005] K. Kiang, R. A. Willgoss, A. Blair, Disincness Analysis on Naural Landmark Descripors, he 5h Inernaional Conference on Field and Service Roboics, 2005. [Kragic and Chrisensen, 2005] D. Kragic and H. I. Chrisensen, Advances in Robo Vision, Roboics and Auonomous Sysems, Vol. 52, Issue -3, 2005. [Lindeberg, 994] T. Lindeberg, Scale Space Theory: A basic ool for analysing srucures a differen scales. Journal of Applied Saisics, 2:2, pp 224-270, 994. [Lowe, 2004] D.G. Lowe, Disincive image feaures from scale-invarian keypoin, Inernaional Journal of Compuer Vision, 60, 2, pp 9-0, 2004. [Mikolajczyk and Schmid, 2002] K. Mikolajczyk and C. Schmid, An affine invarian ineres poin deecor, 8h European Conference on Compuer Vision, pp. 28-42, 2002. [Mikolajczyk and Schmid, 2005] K.Mikolajczyk, C. Schmid, A performance evaluaion of local descripors, Paern Analysis & Machine Inelligence, 2005. [Thrun e al., 2003] S. Thrun, D. Hähnel, D. Ferguson, M. Monemerlo, R. Triebel, W. Burgard, C. Baker, Z. Omohundro, S. Thayer, and W. Whiaker, A sysem for volumeric roboic mapping of underground mines, Inernaional Conference on Roboics and Auomaion, 2003. [Tuyelaars and Van Gool, 2000] T. Tuyelaars and L. Van Gool, Wide baseline sereo maching based on local, affinely invarian regions, h Briish Machine Vision Conference, pp. 42-425, 2000. [Williams e al., 2002] S.B. Williams, G. Dissanayake, H.F. Durran-Whye, Field Deploymen of he Simulaneously Localisaion and Mapping Algorihm, 5TH IFAC World Congress on Auomaic Conrol, 2002. [Williams and Mahon, 2004] S.B. Williams and I. Mahon, Simulaneous Localisaion and Mapping on he Grea Barrier Reef, Inernaional Conference on Roboics and Auomaion, 2004.