Image Fusion for Face Recognition

Size: px

Start display at page:

Download "Image Fusion for Face Recognition"

Gwendoline Heath
6 years ago
Views:

Image Fuso for Face Recogto Xuerog Che Zhoglag Jg Zhehua L Isttute of Aerospace Iformato ad Cotrol, School of Electroc, Iformato ad Electrcal Egeerg, Shagha Jaotog Uversty, Shagha, P.R.Cha, 23 xrche@sjtu.

Recetly, frared (IR) magery of huma face s cosdered as a promsg alteratve to vsble magery due to ts relatve sestve to llumato chages. However, IR has ts ow lmtatos.

Recogto rate s used to evaluate the proposed scheme. Expermetal results show the scheme mproves recogto performace substatally.

Itroducto I recet years face recogto has receved substatal atteto [], but stll remaed very challegg real applcatos. Extrsc magg parameters such as pose varato, facal expresso ad especally llumato (Fg.

1 Image Fuso for Face Recogto Xuerog Che Zhoglag Jg Zhehua L Isttute of Aerospace Iformato ad Cotrol, School of Electroc, Iformato ad Electrcal Egeerg, Shagha Jaotog Uversty, Shagha, P.R.Cha, 23 xrche@sjtu.edu.c Abstract - Despte the varety of approaches ad tools studed, face recogto s ot accurate or robust eough to be used ucotrolled evromets. Recetly, frared (IR) magery of huma face s cosdered as a promsg alteratve to vsble magery due to ts relatve sestve to llumato chages. However, IR has ts ow lmtatos. I order to fuse formato from the two modaltes to acheve better result, we propose a ew fuso recogto scheme by usg o-separable wavelet frame trasform (NWF) ad depedet compoet aalyss (ICA). Recogto rate s used to evaluate the proposed scheme. Expermetal results show the scheme mproves recogto performace substatally. Keywords: Image fuso, face recogto, o-separable wavelet frame trasform, depedet compoet aalyss. Itroducto I recet years face recogto has receved substatal atteto [], but stll remaed very challegg real applcatos. Extrsc magg parameters such as pose varato, facal expresso ad especally llumato (Fg. ), cause much dffculty correct recogto. Recetly, a umber of studes have show that frared (IR) magery of huma faces offers a promsg alteratve to vsble magery [2]. IR magery s early varat uder all lghtg codtos cludg total darkess. However, IR has ts ow lmtatos [2][3]. Oe lmtato s that IR s opaque to glass (Fg. 2), whch makes a large porto of the face wearg eyeglasses hd. Other lmtatos clude ts sestvty to ambet temperature, wd, ad metabolc process the subject. Several fuso recogto schemes have bee proposed. Sgh et al. [4] employed Geetc Algorthms (GAs) to fd a optmum strategy to perform the fuso wavelet doma ad egespace doma, respectvely. But the scheme may ot be easly realzed due to the huge computato of GAs ad exstece of may optmal solutos foud by the GAs. Heo et al. [5] dscussed data fuso wth weghted averagg (Df), decso fuso wth average matchg score (Fa) ad decso fuso wth hghest matchg score (Fh). But the three fuso-based recogto techques are too smple to fuse formato from sesors effectvely ad acheve satsfactory result. I ths paper, we propose a ew fuso recogto scheme. Frst, we obta a composte mage from regstered vsble ad IR face mage by performg NWF, the extract depedet features from the fused mage by usg ICA, ad last, classfy the ew face wth op-match method. We use recogto rate to evaluate the scheme, ad the expermetal results show recogto performace s mproved substatally. he rest of the paper s orgazed as follow: I Secto 2, we gve a bref revew of mathematcal tools used the paper. Our fuso recogto scheme s descrbed detal Secto 3. he face dataset ad expermetal procedure used to evaluate the proposed scheme are gve Secto 4. Expermetal results are preseted ad dscussed Secto 5. Fally, Secto 6 cotas our coclusos ad plas for future research. (a) (b) Fg.. he same vsble face. (a) Frotal llumato ad (b) Lateral llumato (a) (b) Fg. 2. he same IR face. (a) Wthout eyeglasses ad (b) Wth eyeglasses 2 Revew of mathematcal tools 2. No-separable wavelet frame trasform (NWF) Image fuso ca produce ew mages wth more complete formato, whch are more sutable for huma /5/$2. 25 IEEE

2 vso percepto, object detecto ad automatc target recogto. ay mage fuso approaches have bee proposed. Recetly, wth the developmet of wavelet trasform, the dscrete wavelet trasform (DW) [6] ad the dscrete wavelet frame trasform (DWF) [7] have become powerful tools for mage fuso. Whe the separable 2-D wavelet s used to process mages, t oly apples the -D wavelet to the rows ad colums respectvely. No-separable wavelet trasform [8] allows true processg of mages. Images are treated as areas stead of rows ad colums. he advatage wth oseparable wavelet s havg better frequecy characterstc, drectoal property ad more degree of freedom. By elmatg the resamplg process ad chagg the flter coeffcets of the o-separable wavelet trasform, the o-separable wavelet frame trasform (NWF) [9] s obtaed, whch has the property of shft-varace ad s more sutable for mage fuso. he aalyss ad sythess structure of the 2-D NWF s show Fg. 3. H ad G are the 2-D aalyss prototype flters, H ad G are the 2-D sythess prototype flters, D s the o-separable samplg matrx. I ths paper, D s gve by = D () [ H, [ G, [ H ad [ G are the dlated verso of H, G, H ad G. f [ H [ G f + g + [ H [ G Fg. 3. he aalyss ad sythess structure of the 2-D NWF. I each decomposto level, the NWF decomposto volves covolutos by serto of a approprate umber of zeros to low-pass ad hgh-pass flters ad skppg the dow-samplg. So each frequecy bad has the same sze as the source mage. he desg of 2-D o-separable flters s geerally more dffcult tha the desg of -D flters. cclella trasformato [] has bee show to be a useful techque for desgg 2-D o-separable flters. he method ca trasform -D prototype flters to 2-D zero phase FIR flters, ad the 2-D flters parameterzed by the cclella trasformato have the propertes of the -D prototype flters. 2.2 Idepedet compoet aalyss (ICA) he ICA techque [] ams to fd a lear trasform for the put data usg a bass as statstcally depedet as possble. hus, ICA ca be cosdered a geeralzato of f prcpal compoet aalyss (PCA) [2]. PCA tres to obta a represetato of the puts based o ucorrelated varables, whereas ICA provdes a represetato based o statstcally depedet varables. N Let a face mage be represeted as a vector Γ R. he average face of the trag set s defed by Ψ = = where s the umber of faces the trag set. Each face dffers from the average by the vector. he covarace matrx s estmated by Φ = Γ Ψ = Γ C = Φ Φ (2) (3) Usg PCA, a ew face mage Γ s trasformed to lower dmesoal vector y R by y = U Γ Ψ (4) ( ) where U [ u, u, 2, ] u = cossts of egevectors N N U R correspodg to the leadg egevalues of C, ad. y captures the most expressve features of Γ. he PCA output s the processed by the ICA method. PCA s optmal ad useful oly wth respect to data compresso ad decorrelato of low (secod) order statstcs. ICA, whch expads o PCA, cosders hgher 2 statstcs [3]. order ( ) Let y be a dmesoal radom vector correspodg to PCA output defed by Eq. 4. he covarace matrx of y s defed by C E y E y y E y y {[ ( )] [ ( )] } = (5) where E () s the expectato operator. be factorzed as where C y R ca C y = FΛF (6) m m Λ R s dagoal real postve ad m m F R trasforms y to z R, where y = Fz, m of z are such that the m compoets ( ) depedet. A fast fxed-pot algorthm for ICA s descrbed [4]. 3 Fuso recogto scheme Our fuso recogto scheme s descrbed below. We assume that each face s represeted by a par of mages, take smultaeously ad co-regstered, oe the IR spectrum ad oe the vsble. he goal of fuso s to take advatage of the merts of each modalty to mprove the overall recogto accuracy. Fg. 4 llustrates the ma steps of our scheme. 3. Image fuso wth NWF

3 Frst, we perform NWF decomposto for each put source ad obta two multresoluto (R) represetato, oe for vsble face mage ad oe for IR face mage. he, we combe the coeffcets of two R represetato to costruct a composte R represetato, followg some specfc fuso rules. Fally, the fused mage s obtaed by applyg the verse NWF o the composte R represetato. I ths paper, we perform a 3-level decomposto. We combe the coeffcets of the R decomposto of each put by selectg at each posto the coeffcet wth a maxmum absolute value, except for the approxmato coeffcets from the lowest resoluto where we take the average. 3.2 Feature extracto wth ICA After performg NWF, we get a ew fused face mage. PCA the reduces the dmesoalty of the mage vector Γ ad derves the lower dmesoal feature vector y Vsble fac e defed by Eq. 4. Fally ICA method derves a trasformato matrx F defed by Eq. 6. he ew feature vector z of the mage s thus defed by y = U ( Γ Ψ) = Fz (7) where U s the egespaces of fused tra set ad Ψ s the average fused face. 3.3 Classfcato wth op-match he vector z correspodg to ICA output, s used to fd whch oe the ew face belogs to a umber of predefed face classes. op-match s a smple ad useful classfcato method, whch s to fd the face class k that mmzes the Euclda dstace ε k = z Ω k (8) where Ω k s a vector descrbg the k th face class. NWF PCA ICA op-match Result IR face Image Fuso Feature Extracto Classfcato Fg. 4. Flowchart of the fuso recogto scheme. 4 Face database ad expermetal procedure I our expermets, we used the face database collected by Equox Corporato [5]. All data were acqured wth a ewly developed sesor capable of capturg smultaeous co-regstered vdeo sequeces wth a vsble CCD array ad a ucooled LWIR mcro-bolometer. he format cossts of pxel mage pars, coregstered to wth /3 pxel, where the vsble mage has 8 bts of grayscale resoluto ad the LWIR has 2 bts. Before expermets, each mage was preprocessed to extract the face, ad the sze of mage was reduced to 8 4 (Fg. ). I order to compare ad evaluate our fuso recogto scheme overall, we created a trag set ad 5 test sets for each modalty. here were 44 subjects wth a total of 88 mages the trag set, ad 396 mages each test set. Our crtera for trag ad test were as follow: rag set: frotal llumato, o eyeglasses, o expresso est_o set: frotal llumato, o eyeglasses, o expresso est_llumato set: lateral llumato, o eyeglasses, o expresso est_eyeglasses set: frotal llumato, wth eyeglasses, o expresso est_expresso set: frotal llumato, o eyeglasses, wth expresso est_all set: lateral llumato, wth eyeglasses, wth expresso I expermets, we frst compared recogto performace of proposed scheme wth geeral face recogto algorthms (PCA ad ICA) whe appled to ether vsble or frared modalty aloe. he we compared fuso recogto performace by usg dfferet mage fuso techques. Before mage fuso, orgal 2 bts grayscale LWIR mages were coverted to 8 bts ad hstogram equalzed. 5 Expermetal results I expermets, we selected the Daubeches s 9/7 borthogoal flters as the -dmesoal prototype flters for desgg the o-separable flters. I all cases, the umber of features of PCA or ICA was set 6, ad classfcato method was op-match. able compares recogto performace of proposed scheme wth PCA ad ICA whe appled to ether vsble or frared modalty aloe. Vsble results are reported above the correspodg LWIR results. Comparg row 2, 3, 4 wth row respectvely, we get the fluece of llumato, eyeglasses, ad expresso. As expected, varato llumato codtos betwee trag set ad test set resulted markedly reduced performace for vsble magery, whle t had o sgfcat effect o performace of LWIR magery. Although ICA acheved better result tha PCA llumato test of vsble magery, the recogto performace was oly 72.73%.

he eyeglasses had more fluece for LWIR tha vsble magery, sce glass was completely opaque the LWIR. Eve ICA dd t mprove the recogto performace of LWIR wth eyeglasses.

4 he eyeglasses had more fluece for LWIR tha vsble magery, sce glass was completely opaque the LWIR. Eve ICA dd t mprove the recogto performace of LWIR wth eyeglasses. Expresso had almost o effect o both vsble magery ad LWIR our expermets. Comparg row 5 wth row, we get the mxed fluece of the three extrsc magg parameters. I all the test expermets, recogto performace of the proposed scheme was above 9%, except 84.85% the test_all expermet, where recogto performace was mproved by 56%/22% compared to PCA/ICA usg vsble mages ad 2%/6% usg LWIR mages. Example fused face mages by performg NWF ad other mage fuso techques (Averagg, DW ad DWF) are show Fg. 5. Recogto performace of ICA usg dfferet fused face mages test_all expermet are compared able 2. he Daubeches s 9/7 borthogoal flters are also used for the DW ad the DWF wth the same decomposto level ad coeffcets fuso rules as NWF. Fg. 5 shows that the NWF fuso method outperforms the other three techques due to the fact that t s better able to preserve edges ad reduce the rgg artfacts aroud them. hese subjectve vsual comparsos are corroborated by the results able 2. I the expermet, our proposed scheme acheved better recogto performace tha usg other mage fuso techques. able : Recogto performace of PCA, ICA ad the proposed scheme (%). PCA ICA he proposed scheme est_o est_llumato est_eyeglasses est_expresso est_all (a) (b) (c) (d) (e) (f) Fg. 5 he orgal face mages ad fused face mages. (a) he orgal vsble face mage, (b) he orgal LWIR face mage, (c) Fused face mage by Averagg, (d) Fused face mage by DW, (e) Fused face mage by DWF ad (f) Fused face mage by NWF able 2: Recogto performace by usg dfferet mage fuso techques ad ICA (%). Averagg DW DWF NWF est_all

5 6 Coclusos We preseted a ew fuso recogto scheme to combe mportat formato from vsble ad frared mages for robust face recogto. Our scheme s magebased, usg o-separable wavelet frame trasform (NWF) ad depedet compoet aalyss (ICA). Expermetal results show mage fuso wth NWF s better tha fuso wth other geeral techques, ad our scheme mproves recogto performace substatally. Future work cludes desgg more effectve schemes to deal wth rotated face ad other challegg stuatos. However, the advace depeds o the collecto of challegg data. Further cosderato should also be gve to feature-based, decso-based fuso recogto, ad vdeo fuso recogto, whch combes formato ot oly from dfferet sesors, but also from dfferet frames. Ackowledgemets hs work was jotly supported by Natoal Natural Scece Foudato of Cha (63758), Cha Avato Scece Foudato (2D573) ad Cha PH.D Dscple Specal foudato ( ). Refereces [] W. Zhao, R. Chellappa, A. Rosefeld ad P. J. Phllps. Face Recogto: A Lterature Survey. AC Computg Survey, 35: , 23. [2] F. Prokosk. Hstory, Curret Status, ad Future of Ifrared Idetfcato. I IEEE Workshop o Computer Vso Beyod the Vsble Spectrum, pages 5-4, Hlto Head, 2. [3] J. Wlder, P. J. Phllps, C. H. Jag ad S. Weer. Comparso of Vsble ad Ifra-red Imagery for Face Recogto. I Proc. of 2 d It. Cof. o Automatc Face & Gesture Recogto, pages 82-87, Kllgto, 996. [4] S. Sgh, A. Gyaourova, G. Bebs ad I. Pavlds. Ifrared ad Vsble Image Fuso for Face Recogto. I SPIE Defese ad Securty Symposum (Bometrc echology for Huma Idetfcato), volume 544, pages , Orlado, 24. [5] J. Heo, S. G. Kog, B. R. Abd ad. A. Abd. Fuso of Vsual ad hermal Sgatures wth Eyeglass Removal for Robust Face Recogto. I IEEE Workshop o Object rackg ad Classfcato Beyod the Vsble Spectrum, pages 94-99, Washgto, 24. [6] H. L, B. S. ajuath ad S. K. tra. ultsesor Image Fuso Usg the Wavelet rasform. Graphcal odels ad Image Process, 57(3): , 995. [7] S.. L,. K. Iaes ad Y. N. Wag. Usg the Dscrete Wavelet Frame rasform to erge Ladsat ad SPO Pachromatc Images. Iformato Fuso, 3(): 7-23, 22. [8] J. Kovačevć ad. Vetterl. Noseparable woad hree-dmesoal Wavelets. IEEE ras. o Sgal Processg, 43(5): , 995. [9] J. S. Pa ad J. W. Wag. exture Segmetato Usg Separable ad No-separable Wavelet Frames. IEICE ras. o Fudametals of Electrocs, Commucato ad Computer Sceces, E82-A(8): , 999. [] J. Kovačevć ad. Vetterl. Noseparable multdmesoal perfect recostructo flter baks ad wavelet bases for R. IEEE ras. o Iformato heory, 38(2): , 992. []. S. Bartlett, J. R. ovella ad. J. Sejowsk. Face Recogto by Idepedet Compoet Aalyss. IEEE ras. o Neural Networks, 3(6): , 22. [2]. urk ad A. Petlad. Face Recogto Usg Egeface. I IEEE Cof. o Computer Vso ad Patter Recogto, pages , 99. [3] C. Lu ad H. Wechsler. Idepedet Compoet Aalyss of Gabor Features for Face Recogto. IEEE ras. Neural Networks, 4(4): , 23. [4] A. Hyväre ad E. Oja. A Fast Fxed-Pot Algorthm for Idepedet Compoet Aalyss. Neural Computato, 9: , 997. [5]

Block-based Feature-level Multi-focus Image Fusion

Block-based Feature-level Mult-focus Image Fuso Abdul Bast Sddqu, M. Arfa Jaffar Natoal Uversty of Computer ad Emergg Sceces Islamabad, Paksta {bast.sddqu,arfa.affar}@u.edu.pk Ayyaz Hussa, Awar M. Mrza