Effective Representation Using ICA for Face Recognition Robust to Local Distortion and Partial Occlusion

Transcription

1 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 27, NO. 12, DECEMBER Effectve Representaton Usng ICA for Face Recognton Robust to Local Dstorton and Partal Occluson Jongsun Km, Student Member, IEEE, Jongmoo Cho, Member, IEEE, Juneho Y, Member, IEEE, and Matthew Turk, Senor Member, IEEE Abstract The performance of face recognton methods usng subspace projecton s drectly related to the characterstcs of ther bass mages, especally n the cases of local dstorton or partal occluson. In order for a subspace projecton method to be robust to local dstorton and partal occluson, the bass mages generated by the method should exhbt a part-based local representaton. We propose an effectve part-based local representaton method named locally salent ICA (LS-ICA) method for face recognton that s robust to local dstorton and partal occluson. The LS-ICA method only employs locally salent nformaton from mportant facal parts n order to maxmze the beneft of applyng the dea of recognton by parts. It creates part-based local bass mages by mposng addtonal localzaton constrant n the process of computng ICA archtecture I bass mages. We have contrasted the LS-ICA method wth other part-based representatons such as LNMF (Localzed Nonnegatve Matrx Factorzaton) and LFA (Local Feature Analyss). Expermental results show that the LS-ICA method performsbetter than PCA,ICA archtecturei, ICA archtecture II, LFA,and LNMF methods, especally n the cases of partal occlusons and local dstortons. Index Terms Face recognton, part-based local representaton, ICA, LS-ICA. 1 INTRODUCTION æ OVER the past 10 years, canoncal subspace projecton technques such as PCA, ICA, and FLD have been wdely used n the face recognton research [1], [2], [3], [4], [5]. These technques represent a face as a lnear combnaton of low rank bass mages. They employ feature vectors consstng of coeffcents that are obtaned by smply projectng facal mages onto ther bass mages. In order for a subspace projecton-based method to be robust to partal occlusons and local dstortons, ts bass mages should effectvely realze a part-based local representaton. Local representaton provdes robustness to partal occlusons and local dstortons because successful face recognton can be acheved by representng some mportant facal parts that correspond to feature regons such as eyes, eyebrows, nose, and lps. Ths recognton by parts paradgm [11] has been popular n the object recognton research because the approach can be successfully appled to the problem of object recognton wth occluson. Facal mage representatons based on dfferent bass mages are llustrated n Fg. 1. ICA can be appled to face recognton n two dfferent representatons: ICA archtecture I and II [2]. Please refer to Secton 2.1 for more descrpton about these two representatons. PCA and ICA archtecture II bass mages, as shown n Fgs. 1a and 1b, respectvely, dsplay global propertes n the sense that they assgn sgnfcant weghts to potentally all the pxels. Ths accords wth the fact that PCA bass mages are just scaled versons of global Fourer flters [13]. In contrast, ICA archtecture I bass mages are. J. Km, J.M. Cho, and J. Y are wth the School of Informaton & Communcaton Engneerng, Bometrcs Engneerng Research Center, Sungkyunkwan Unversty, Korea. E-mal: {jskm, jmcho, jhy}@ece.skku.ac.kr.. M. Turk s wth the Computer Scence Department, Unversty of Calforna, Santa Barbara E-mal: mturk@cs.ucsb.edu. Manuscrpt receved 30 Mar. 2004; revsed 11 Apr. 2005; accepted 18 Apr. 2005; publshed onlne 13 Oct Recommended for acceptance by R. Basr. For nformaton on obtanng reprnts of ths artcle, please send e-mal to: tpam@computer.org, and reference IEEECS Log Number TPAMI spatally more localzed. Ths local property of ICA archtecture I bass mages makes the performance of ICA archtecture I based recognton methods robust to partal occlusons and local dstortons, such as local changes n facal expresson, because spatally local features only nfluence small parts of facal mages. However, ICA archtecture I bass mages do not dsplay perfectly local characterstcs, n the sense that pxels that do not belong to locally salent feature regons stll have some nonzero weght values. These pxel values n nonsalent regons would appear as nose and contrbute to the degradaton of the recognton. Among representatve part-based local representatons are Local Feature Analyss (LFA) [6] and Local Nonnegatve Matrx Factorzaton (LNMF) [14] methods. The LFA method extracts local features based on second-order statstcs. However, bass mages from the LFA representaton are not perfectly localzed as shown n Fg. 1f. Thus, pxels n nonsalent regons degrade the recognton performance n the case of partal occlusons and local dstortons. Recently, the LNMF method was reported n the lterature, whch led to an mproved verson of Nonnegatve Matrx Factorzaton (NMF) [12]. In the LNMF method, localty constrants are mposed on the factorzed matrces from NMF n terms of sparsty n matrx components. They successfully localzed the components n bass mages. However, the localty constrants do not guarantee that meanngful facal features should be localzed n ther bass mages. As an example of the LNMF representaton n Fg. 1e llustrates, some LNMF bass mages represent regons such as cheek, forehead, and jaw that are not dscrmnant features for face recognton. We propose new bass mages based on ICA archtecture I, called LS-ICA (locally salent ICA) bass mages, where only locally salent feature regons are retaned. The dea of recognton by parts can be effectvely realzed for face recognton usng LS-ICA bass mages snce each LS-ICA bass mage represents only locally salent regons. These regons correspond to mportant facal feature regons such as eyes, eyebrows, nose, and lps. Note that ICA archtecture I produces bass mages that are localzed edge flters [13], and they correspond to meanngful facal feature regons. Our method for face recognton s characterzed by two deas: The frst s the creaton of the LS-ICA bass mages usng a modfed verson of Kurtoss maxmzaton to remove resdual nonlocal modulaton n ICA archtecture I bass mages; these are used to represent faces. The second dea s to use LS-ICA bass mages n the decreasng order of class separablty so as to maxmze the recognton performance. Expermental results show that LS-ICA performs better than PCA, ICA archtecture I, ICA archtecture II, LFA, and LNMF, especally n the cases of partal occlusons and local dstortons such as local changes n facal expresson. The rest of ths paper s organzed as follows: Secton 2 brefly descrbes the ICA, LFA, and LNMF methods that are most relevant to our research. We present the proposed LS-ICA method n Secton 3. Secton 4 gves expermental results. 2 RELATED WORK 2.1 ICA (Independent Component Analyss) ICA s a wdely used subspace projecton technque that projects data from a hgh-dmensonal space to a lower-dmensonal space [2], [3], [4]. Ths technque s a generalzaton of PCA that decorrelates the hgh-order statstcs n addton to the secondorder moments. In ths research, we compute ICA bass mages usng the FastICA algorthm [3] whle other methods such as InfoMax [2] or Maxmum lkelhood [4] can also be employed. The FastICA method computes ndependent components by maxmzng non-gaussanty of whtened data dstrbuton usng a kurtoss maxmzaton process. The kurtoss measures the non- Gaussanty and the sparseness of the face representatons [13]. The FastICA algorthm s brefly descrbed as follows: Let S be the vectors of unknown source sgnals and x be vectors of observed mxtures. If A s an unknown mxng matrx, then the mxng model can be wrtten as X ¼ AS. The task s to estmate the ndependent /05/$20.00 ß 2005 IEEE Publshed by the IEEE Computer Socety

2 1978 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 27, NO. 12, DECEMBER 2005 Fg. 1. Facal mage representatons usng (a) PCA, (b) ICA archtecture I, (c) ICA archtecture II, (d) proposed LS-ICA, (e) LNMF, and (f) LFA bass mages: A face s represented as a lnear combnaton of bass mages. The bass mages were computed from a set of mages randomly selected from the AR database. Usng LS-ICA bass mages, the concept of recognton by parts can be effectvely mplemented for face recognton. source sgnals U by computng the separatng matrx W that corresponds to the mxng matrx A usng the followng relaton: U ¼ WX ¼ WAS: Frst, the observed samples are whtened. Let us denote the whtened samples by Z. Then, we search for the matrx such that the lnear projecton of the whtened samples by the matrx W has maxmum non-gaussanty of data dstrbuton. The kurtoss of U ¼ W T Z s computed as n (2) and the separatng vector W s obtaned by maxmzng the kurtoss [3]. n o n o 2 kurtðu Þ ¼ E ðu Þ 4 3 E ðu Þ 2 : ð2þ ICA can be appled to face recognton n two dfferent archtectures [2]. The ICA archtecture I consders the nput face mages, X, as a lnear combnaton of statstcally ndependent bass mages, S, combned by an unknown matrx, A. The coeffcents obtaned by projectng nput mages onto the statstcally ndependent bass mages are not statstcally ndependent. On the other hand, the ICA archtecture II fnds statstcally ndependent coeffcents that represent nput mages. The ICA archtecture II bass mages dsplay global propertes as shown n Fg. 1c. Snce the kurtoss maxmzaton yelds sparseness of bass mages, the ICA archtecture I bass mages are spatally localzed edge flters [13]. However, they do not dsplay perfectly local characterstcs n the sense that pxels that do not belong to locally salent feature regons ð1þ stll have some nonzero weght values. These pxel values would contrbute to the degradaton of the recognton performance n the case of local dstorton and partal occluson. 2.2 LFA (Local Feature Analyss) LFA defnes a set of local topographc kernels that are derved from the prncpal component egenvectors E and coeffcents D accordng to covarance matrx S usng the followng equaton: K ¼ ED 1 2 E T 1 ; where D 1 2 ¼ dag p ffffffff ¼ 1;...;p: ð3þ s are egenvalues of the covarance matrx, S. The rows of K contan the kernels. The kernels are topographc n that they are ndexed spatally. The number of kernels corresponds to the number of pxels n an mage. The LFA uses a sparsfcaton algorthm n order to reduce the dmensonalty of the representaton. The algorthm teratvely selects kernels that have the largest mean reconstructon error. We are concerned wth bass mages from the LFA method. They are not perfectly localzed and pxels n nonsalent regons contrbute to the degradaton of the recognton performance. 2.3 LNMF (Local Nonnegatve Matrx Factorzaton) The LNMF method [14] s a technque that mproves the standard NMF method [12]. It s amed at learnng spatally localzed, partsbased subspace representaton of bass mages by mposng addtonal constrants on the NMF bass. The followng objectve functon s used to compute LNMF bass mages:

3 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 27, NO. 12, DECEMBER Fg. 2. Example mages from AT&T (left), AR (mddle), and FERET (rght) facal databases. TABLE 1 FERET Database Used n the Experment DXjjBH ð Þ¼ Xm X n ¼1 j¼1! X j log X j X j þ½bhš ½BHŠ j þu j X V ; j ð4þ where ; > 0 are some constants, U ¼ B T B, V ¼ HH T, and B; H 0 means that all entres of bass mages B and coeffcents H are nonnegatve. The mnmzaton of U ¼ B T B mposes both maxmum sparsty n H and mnmum redundancy between dfferent bases. On the other hand, by maxmzng P V, bass components that carry much nformaton about the tranng mages are retaned. Refer to [14] for further justfcaton of the objectve functon. The LNMF update rule for H uses square root as n (5) to satsfy the addtonal constrants. The update for B s dentcal to that of NMF [15]. H ðtþ1þ aj sffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff X ¼ H ðtþ aj B T X ðtþ j a : ð5þ ðb ðtþ H ðtþ Þ j Fg. 1e shows an example of the LNMF representaton. As descrbed earler, the addtonal constrants only focus on localty and t s not necessarly guaranteed that meanngful facal features are localzed n ther bass mages. 3 THE LS-ICA (LOCALLY SALIENT ICA) METHOD The computaton of LS-ICA bass mages conssts of two steps: The frst step s concerned wth the creaton of part-based local bass mages based on ICA archtecture I. The second s to order the bass mages obtaned n the order of class separablty for good recognton performance. The LS-ICA method creates part-based local bass mages by mposng addtonal localzaton constrant n the process of the kurtoss maxmzaton. The soluton at each teraton step s weghted so that t becomes sparser by only emphaszng large pxel values. Localzaton emerges from ths sparsfcaton. Let u be a soluton vector at an teraton step, we can defne a weghted soluton as b, where b ¼ ju j u and b ¼ b=kbk. >1s a small constant. The kurtoss s maxmzed n terms of b nstead of u as n (6). n o n o 2 kurtðbþ ¼ E ðbþ 4 3 E ðbþ 2 : ð6þ A soluton to the above functon can be found by usng the followng update rules: 3 w ðtþ1þ ¼ E ju j Z ju j w T ðþ t Z ; ð7þ where w s a separatng vector and Z contans whtened mage samples. The resultng bass mage s b ¼ ju j w T Z. As an alternatve method, we would lke to pont out that other smple operatons such as morphologcal operatons can be employed to detect salent regons from ICA archtecture I bass mages. We then compute a measure of class separablty, r, for each LS-ICA bass vector and sort the LS-ICA bass vectors n the decreasng order of class separablty [2]. To compute r for each LS-ICA bass vector, the between-class varablty between and wthn-class varablty wthn of ts correspondng projecton coeffcents of tranng mages are obtaned as follows: between ¼ ðm MÞ 2 ; ð8þ 2: wthn ¼ j h j M ð9þ M and M are the total mean and the mean of each class, and h j s the coeffcent of the jth tranng mage n class. The class separablty, r, s then defned as the rato r ¼ between : ð10þ wthn We then create new LS-ICA bass mages from the LS-ICA bass mages selected n the decreasng order of the class separablty. Ths way, we can acheve both dmensonalty reducton and good recognton performance. The LS-ICA representaton s based on the dea of localzed edge flters that come from ICA bass mages. The resultng bass mages contan localzed facal features that are dscrmnant for face recognton. We have calculated parwse mutual nformaton for the bass mages from (2) versus (6) usng the followng jont entropy used n [16]: IX;Y ð Þ ¼ HX ð ÞþHY ð Þ HX;Y ð Þ; ð11þ where HX;Y ð Þ s the jont entropy of X and Y. The mean values of the parwse mutual nformaton for (2) versus (6) are and , respectvely. Ths expermentally shows that the sparsfcaton also enhances the ndependence soluton. 4 EXPERIMENTAL RESULTS We have used several facal mage databases such as the FERET [8], AR [9], and AT&T [10] databases n order to compare the recognton

4 1980 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 27, NO. 12, DECEMBER 2005 Fg. 3. Example AT&T mages havng random occludng patches of szes (from left to rght) 10 x 10, 20 x 20, and 30 x 30. performance of LS-ICA wth that of PCA, ICA archtecture I, ICA archtecture II, LFA, and LNMF methods. Fg. 2 shows example mages from these databases. For far comparsons wth the above methods, ther bass mages were also used n the decreasng order of class separablty, r. We have computed recognton performances for three dfferent dstance measures (L1, L2, cosne) to see f there s any performance varaton dependng on the dstance measure used [7]. We have used subsets of the FERET mages under sgnfcantly dfferent lghtng and facal expresson. The whole set of mages, U, used n the experment, conssts of four subsets called fa, ba, bj, and bk, as summarzed n Table 1. For the experment, we have dvded the whole set U nto tranng set (T), gallery set (G), and probe set (P). No one wthn the tranng set (T) s ncluded n the gallery or the probe sets. The AR database contans 800 frontal facal mages from 100 subjects. The number of mages used for tranng and testng are 200 and 600 mages, respectvely. Test mages contan local dstortons and occlusons such as changes n facal expresson and sunglasses worn. In order to show the performance comparsons under occluson, we have used the AT&T database. It conssts of 400 mages of 40 persons, 10 mages per person. The mages are taken aganst a dark homogeneous background and the subjects are n an uprght, frontal poston wth tolerance for some sde movement. A set of 10 mages for each person s randomly parttoned nto fve tranng mages and fve testng mages. The occluson s smulated n an mage by usng a whte patch of sze s s wth s 2 f10; 20; 30g at a random locaton as shown n Fg. 3. Fgs. 4 and 5 show the recognton performances of PCA, ICA archtecture I, ICA archtecture II, LNMF, LFA, and LS- ICA methods for the three facal databases. The recognton rate of the LS-ICA method was consstently better than that of PCA, ICA archtecture I, ICA archtecture II, LNMF, and LFA methods regardless of the dstance measures used. In Fg. 4, ICA archtecture I also consstently outperformed PCA except the case where the L1 measure was used for the FERET database. Ths also accords wth the expermental results of Draper et al. [7]. Fg. 5 compares the sx representatons under varyng degrees of occluson, n terms of the recognton accuraces versus the sze s s of occludng patch for s 2 f10; 20; 30g. The LS-ICA and LNMF methods performed better than the other methods under partal occluson, especally as the patch sze ncreases. The LFA and ICA archtecture I methods appear nfluenced by pxels not belongng to salent regons. The ICA archtecture I method showed better performance than the LFA method. Bartlett [16] have also reported that the ICA representaton performs better than the LFA representaton n cases of facal expresson analyss and face recognton. Ths expermentally shows Fg. 4. The recognton performance of PCA, ICA1, ICA2, LNMF, LFA, and LS-ICA methods for the AR and FERET ( ba - bj set and ba - bk set) facal databases.

5 IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 27, NO. 12, DECEMBER Fg. 5. Recognton performance versus the sze (n 10 x 10, 20 x 20, and 30 x 30) of occludng patches for PCA, ICA1, ICA2, LNMF, LFA, and the proposed LS-ICA method for the AT&T database. that the ICA archtecture I better represents some mportant facal parts than the LFA method. The LS-ICA method that only makes use of locally salent nformaton from mportant facal parts acheved hgher recognton rates than the LNMF method. We can see that the LS-ICA representaton s an effectve part-based local representaton for face recognton robust to local dstorton and partal occluson. 5 CONCLUSION We have proposed the LS-ICA method that only employs locally salent nformaton from mportant facal parts n order to maxmze the beneft of applyng the dea of recognton by parts to the problem of face recognton under partal occluson and local dstorton. The performance of the LS-ICA method was consstently better than other representatve local representaton based methods regardless of the dstance measures used. As expected, the effect was the greatest n the cases of facal mages that have partal occlusons and local dstortons such as changes n facal expresson. ACKNOWLEDGMENTS Ths work was supported by the Korea Scence Engneerng Foundaton (KOSEF) through the Bometrcs Engneerng Research Center (BERC) at Yonse Unversty and BK21. REFERENCES [1] M.A. Turk and A.P. Pentland, Egenfaces for Recognton, Cogntve Neuroscence, vol. 3, no. 1, pp , [2] M.S. Bartlett, J.R. Movellan, and T.J. Sejnowsk, Face Recognton by Independent Component Analyss, IEEE Trans. Neural Networks, vol. 13, no. 6, pp , [3] A. Hyvarnen and E. Oja, Independent Component Analyss: A Tutoral, [4] A. Hyvärnen, The Fxed-Pont Algorthm and Maxmum Lkelhood Estmaton for Independent Component Analyss, Neural Processng Letters, vol. 10, pp. 1-5, [5] P. Belhumeur, J. Hespanha, and D. Kregman, Egenfaces versus Fsherfaces: Recognton Usng Class Specfc Lnear Projecton, IEEE Pattern Analyss and Machne Intellgence, vol. 19, no. 7, pp , [6] P. Penev and J. Atck, Local Feature Analyss: A General Statstcal Theory for Object Representaton, Network: Computaton n Neural Systems, vol. 7, no. 3, pp , [7] B.A. Draper, K. Baek, M.S. Bartlett, and J.R. Beverdge, Recognzng Faces wth PCA and ICA, Computer Vson and Image Understandng, vol. 91, no. 1, pp , [8] P.J. Phllps, H. Moon, S.A. Rzv, and P.J. Rauss, The FERET Evaluaton Methodology for Face Recognton Algorthms, IEEE Pattern Analyss and Machne Intellgence, vol. 22, no. 10, pp , [9] A.M. Martnez and R. Benavente, The AR Face Database, CVC Tech, [10] [11] A.P. Pentland, Recognton by Parts, IEEE Proc. Frst Int l Conf. Computer Vson, pp , [12] D.D. Lee and H.S. Seung, Learnng the Parts of Objects by Nonnegatve Matrx Factorzaton, Nature, vol. 401, pp , [13] A.J. Bell and T.J. Sejnowsk, The Independent Components of Natural Scenes Are Edge Flters, Vson Research, vol. 37, no. 23, pp , [14] S.Z. L, X.W. Hou, and H.J. Zhang, Learnng Spatally Localzed, Parts- Based Representaton, Computer Vson and Pattern Recognton, vol. 1, pp , [15] S. Wld, J. Curry, and A. Dougherty, Motvatng Non-Negatve Matrx Factorzatons, Proc. Eghth SIAM Conf. Appled Lnear Algebra, July [16] M.S. Bartlett, Face Image Analyss by Unsupervsed Learnng. Kluwer Academc, For more nformaton on ths or any other computng topc, please vst our Dgtal Lbrary at