Recognton of Low-Resoluton Face Images usng Sparse Codng of Local Features M. Saad Shakeel and Kn-Man-Lam Centre for Sgnal Processng, Department of Electronc and Informaton Engneerng he Hong Kong Polytechnc Unversty, Hong Kong E-mal: saad.ee.shakeel@connect.polyu.hk, enkmlam@polyu.edu.hk Abstract In ths paper, we propose a new approach for recognton of low-resoluton face mages by usng sparse codng of local features. he proposed algorthm extracts Gabor features from a low-resoluton gallery mage and a query mage at dfferent scales and orentatons, then projects the features separately nto a new low-dmensonal feature space usng sparse codng that preserves the sparse structure of the local features. o determne the smlarty between the projected features, a coeffcent vector s estmated by usng lnear regresson that determnes the relatonshp between the projected gallery and query features. On the bass of ths coeffcent vector, resdual values wll be computed to classfy the mages. o valdate our proposed method, experments were performed usng three databases (ORL, Extended-Yale B, and CAS-PEAL-R), whch contan mages wth dfferent facal expressons and lghtng condtons. Expermental results show that our method outperforms varous classcal and state-of-the-art face recognton methods. I. INRODUCION Performance of real-world face bometrc systems degrades sgnfcantly when face mages taken from survellance cameras are of very low-resoluton (LR), wth szes less than 30 30 pxels. radtonal methods [-3] cannot perform well when mages are of very poor qualty and low resoluton. LR mages lose a lot of mportant dscrmnatve nformaton (facal features) makng the recognton task qute dffcult. Super-resoluton (SR) has been employed as a possble soluton by mprovng the mage resoluton and qualty before face recognton. However, experments have shown that recognton accuracy decreases, nstead of ncreases, because of the dstorton and appearance s artfacts n the superresolved mage. Apart from SR technques, Hennngs- Yeomans et al. [4] presented an alternatve soluton, whch smultaneously performs hallucnaton and recognton by usng a jont objectve functon. hs approach does not seem effcent n terms of computatonal tme, because an optmzaton process s needed for each probe or query (test) mage. Zou et al. [5] proposed a method for face superresoluton by learnng the relatonshp between the hghresoluton (HR) and LR mage spaces usng dscrmnant constrant wth class label nformaton. An mproved method based on smultaneous super-resoluton and recognton was proposed by Jan et al. n [3]. It performs face super-resoluton and face recognton by usng the propertes of sngular values of face mages at dfferent resolutons. For LR face recognton, the super-resolved HR features, rather than the super-resolved face mages, are needed. herefore, Pong et al. [4] proposed performng super-resoluton n the feature doman, and HR features are estmated drectly for face recognton. hs approach can avod estmatng HR facal features from dstorted super-resolved, HR face mages. L et al. [5] proposed a method whch projects HR and LR mages nto a unfed feature space by tranng coupled mappngs to learn the relatonshp between HR and LR samples. Smlarly, Ren et al. [6] proposed learnng a common subspace by usng coupled kernel embeddng that compared multmodal data by usng a new, effcent smlarty measure. Bswas et al. [7] utlzed mult-dmensonal scalng to project HR and LR samples nto a common feature space that preserves the geometrcal structure of tranng and testng samples, and also guarantees the dstance between the HR and LR samples to be approxmately the same as that of the two correspondng HR mages. he prevously mentoned methods [5, 7] do not extract dscrmnant nformaton (facal features) from the mages before projectng them nto a unfed feature space whch results n reducng ther recognton accuracy n unconstraned envronments. herefore, there s a need to extract local features robust to dfferent knds of varatons before dscrmnant subspace analyss s employed. In ths paper, we propose a new approach for recognton of LR face mages, based on sparse codng of local features, by seekng an optmum sparse matrx to project tranng and testng features onto a new low-dmensonal feature space, where the sparse structure of the local features s preserved. o overcome the problem of dstorton and artfacts caused by super-resoluton algorthms, a two-stage morphologcal preprocessng method s proposed, whch conssts of the top and the bottom-hat flterng, for mprovng the vsual qualty of an mage, wthout creatng any artfacts or dstorton n the processed mage. Inspred by the performance of local-feature descrptors n pattern recognton tasks, -D Gabor features are extracted from facal mages at fve scales and eght orentatons. hese features are robust aganst local dstortons caused by pose, expresson and llumnaton varatons. o project the extracted tranng and testng features nto a lowdmensonal feature space, a new objectve functon, based on sparse codng, s proposed to seek a feature space, where the sparse structure of the features s preserved. Fnally, lnear regresson s used to determne the relatonshp between the tranng and testng features n terms of a coeffcent vector,
whch s used for computng the resduals for classfcaton. Our proposed method also utlzes the class label nformaton to enhance the dscrmnablty of face mages. Furthermore, t s not necessary for our proposed objectve functon to deal wth model parameters especally neghborhood sze whch reduce the tme-complexty of our method as compared to varous lnear and non-lnear mappng methods such as LPP [6], NPE [7] etc. An extra advantage s ts capablty to employ the local structure of data by usng the sparsty pror. Extracton of local features, (.e. Gabor wavelets) makes our method robust to varatons n pose, expresson and lghtng condtons. II. PREPROCESSING AND FEAURE EXRACION In ths secton, we wll frst present the preprocessng operatons appled to face mages, before local features are extracted. hen, Gabor wavelets are ntroduced. In the subsequent sectons, we wll descrbe our sparse codng method and the use of lnear regresson for LR face recognton. A. Morphologcal pre-processng: op-hat flterng s a morphologcal mage processng operaton that extracts small detals from mages and also removes poor-contrast features. Illumnaton varaton s also one of the man challenges n face recognton that can be overcome by ths operaton, as t also allevates the effects of non-unform llumnaton. Let be the gven grayscale mage, and be a structurng element. hen, top-hat flterng s gven as t ( I) = I IΘswhere Θ denotes the openng operaton. op-hat flterng can be consdered an excellent tool for extractng brght features from an mage by correctng the effects of non-unform llumnaton, whle the bottom-hat flterng extracts dark features. In our method, the dfference between the top-hat and bottom-hat fltered outputs are added to the face mage, so as to acheve local-contrast enhancement. Mathematcally, t can be wrtten as follows: I = I+ I I CE H BH () where I and H I are the top and bottom-hat fltered mages BH respectvely, and ICE s the enhanced face mage. hs operaton s performed on both tranng and testng mages before feature extracton s performed. Fg. shows an orgnal face mage and the correspondng mage, after the top and bottom-hat flterng. expressons. It has been found that local features are more robust to these varatons. Spatal-frequency analyss decomposes an mage nto sub-mages, whch provde useful nformaton about the mage s structure at multple scales and orentatons, and exhbt good characterstcs of spacefrequency localzaton. Such decompostons yeld varous mage representatons, n terms of drectonal orentaton, multple scales, frequency selectvty, etc. Gabor wavelets (GWs) utlze both the spatal and frequency domans to acheve optmum representatons. It s found that the smple cells n the vsual cortex of mammalan brans can be modeled usng Gabor functons. It has an ablty to extract abundant nformaton from local regons at dfferent scales of frequences and orentatons. GW s a complex exponental modulated by a Gaussan functon, whch s represented mathematcally as follows: x y φ f, θ ( xy, ) = exp[( ){( ) + ( )}]exp( ( π fθ )) σ n () x σy where, denotes the poston of pxels n the spatal doman, and represents the selectve frequency and orentaton, respectvely, and s the standard devaton of the Gaussan functon. In our method, features are extracted at 5 scales and 8 orentatons as shown n Fg.. Fg. 3 llustrates the outputs of the varous Gabor flters. By extractng nformaton from LR face mages at dfferent scales and orentatons, recognton accuracy can be greatly mproved. Fg. Gabor Flters at dfferent scales and orentatons. Fg.3 Gabor features of a LR mage (6 6) at fve scales and eght orentatons (a) Fg. (a) Orgnal Image, and (b) after the top and bottom-hat flterng. B. Gabor Wavelets Face recognton n an unconstraned envronment s stll a challengng problem because of msrepresentatons caused by pose varatons, poor lghtng condtons and dfferent facal (b) III. SPARSE CODING OF LOCAL FEAURES Sparse representaton [8] s proven to be an effcent technque for face recognton, even n unconstraned envronments. In ths representaton, each face mage s represented as a sparse, lnear combnaton of other tranng samples. he relatonshp between the tranng and testng samples s represented by the sparse coeffcent vector y = kx. Here s a sparse coeffcent
vector whle x s a data matrx whose columns are the tranng samples from the th class. here should be strong correlaton between the samples of the same class, whle ths correlaton decreases between samples from dfferent classes. herefore, the coeffcents correspondng to samples from the same class should have certan values n the sparse coeffcent vector, whle other coeffcents should be zero. hs s known as the sparsty property of reconstructed samples. Consder M tranng samples denoted as X = [ x, x,..., x ] M, where x j s th the j sample. Suppose that there are c known classes, and n samples for each class. Assume that each sample can be lnearly reconstructed by usng M samples; ths s the sparse representaton s property of recovered samples. Our objectve s to fnd a low-dmensonal feature space, where the sparse features and the correspondng errors are mnmzed. Let us denote the sparse coeffcent vectors of samples are K k k k m = [,,..., ], where k s the sparse vector of th sample. Sparse representaton ams at representng test mages usng the smallest number of tranng samples. hs can be represented as follows: mn k 0 (3) s.. t y = kx hs equaton counts the number of non-zero components n k, and can be solved by usng norm optmzaton,.e. mn k (4) s.. t y = kx Equaton (4) can be wrtten as: mn k = k + k +,..., + k cn (5) s.. t y = Xk = Xk + Xk +,... + Xk j where y s the query (test) mage, k s a sparse coeffcent vector and X s the matrx contanng tranng (gallery) samples. In our algorthm, we propose a new objectve functon that seeks an optmal projecton matrx to map local features to a new low-dmensonal feature space, such that sparsty s acheved and the correspondng sparse reconstructon error s also mnmzed. he objectve functon s as follows: j= j p = arg mn p y p Xk (6) Equaton (6) can be wrtten as: M j= p M j mn p ( ( y Xk )( y Xk ) p (7) j j j j and can also be wrtten n matrx form as follows: cn he constrant n (8) s p XX p =, whch can provde a stable soluton. o solve (6), Lagrange multpler s appled and the followng result s obtaned: L( p, λ) = p X( I K K + K K) X p λ( p XX p ) (9) o determne the optmal projecton matrx p, the dervatve s L set at zero,.e. = 0 hs provdes the followng soluton: p px ( I K K + K K) X = λ XX p (0) Fnally, a new low-dmensonal feature space wll be constructed by usng a number of egenvectors wth the largest non-zero egenvalues. IV. CLASSIFICAION USING LINEAR REGRESSION After projectng the respectve local features of the tranng and testng samples onto a new low-dmensonal sparse feature space, the next step s to determne the relatonshp between the projected features for classfcaton. Wth the assumpton that the extracted features le n a lnear subspace, a lnear model can be developed that represents the probe mage as a sparse lnear combnaton of the tranng samples n a gallery. o verfy that a face mage belongs to the th class, t s represented as a lnear combnaton of the gallery mages of the same class. Mathematcally, t can be wrtten as follows: y = α x () where and are the probe and gallery mages from the th class, whle s the coeffcent vector determned by the least squares method. Once the coeffcent vectors have been found, the next step s to compute the resduals for each class, and the probe mage wll be assgned to the class wth mnmum reconstructon error. It can be wrtten as: j = mn y α x () Fg. 4 shows the overall structure of our proposed algorthm. mn p p X( I K K + K K) X p p XX p (8) Fg.4. Flowchart of our proposed method
V. EXPERIMENAL RESULS o evaluate the effectveness of our proposed method, experments were conducted on three publc datasets, ncludng the ORL, Extended-Yale B, and CAS-PEAL-R databases. A. Experments on the ORL Database he ORL database contans 400 mages wth 40 classes (0 samples/class), wth varyng poses, lghtng condtons, and facal expressons. For face recognton, the gallery samples are down-sampled to have the same resoluton as the probe mages. he mages of each subject are dvded nto dfferent numbers of tranng and testng samples. In our experments, resolutons of probe mages are set at 5 5, 0 0 and 6 6 respectvely. Fg.5 shows some mages from the ORL database, and the correspondng LR mages. Fg.5. Samples of orgnal and LR mages from the ORL database. B. Experments on the Extended Yale B Database he Extended Yale B database contans,43 mages wth 38 subjects under 64 llumnaton condtons. In order to address the small-sample-sze problem, we randomly selected 0 mages per subject (380 mages) for experments. Dfferent numbers of tranng and testng mages were selected from each subject. Fg.6 shows some mages from the Extended Yale B database, and the correspondng LR mages. Fg.7. Samples of orgnal and LR mages from the CAS-PEAL-R database. D. Comparatve Analyss o evaluate the performance of our proposed method, we compare t wth dfferent classcal and state-of-the-art face recognton algorthms, ncludng SRC [8], LDA [], MDS [7], and the two-step algorthm (frst super-resolvng LR mages usng the algorthm proposed n [0] based on three neghbors). Fgs 8 and 9 show the recognton rates wth dfferent numbers of tranng and testng samples for the ORL and Extended-Yale B databases, wth the LR mages of sze 6 6. Fg. 0 shows the recognton accuracy versus tranng samples per subject for the CAS-PEAL-R database, wth the LR mages of sze 0 0. Our method acheves the hghest recognton rate of 93%, wth 5 tranng, samples per subject for the ORL database. For the Yale-B database, the hghest recognton rate acheved by our method s 94.73% for LR mage of sze 5 5. he hghest recognton rate of our proposed method s 97.%, for the CAS-PEAL-R database wth 3 tranng samples per subject and mage resoluton of 5 5. Our method outperforms all other methods n comparson, n terms of recognton accuracy. Fg.6. Samples of orgnal and LR mages from the Extended Yale-B database. C. Experments on the CAS-PEAL-R Database CAS-PEAL-R s a large-scale Chnese database contans 30,900 mages of,040 subjects. It has several categores, ncludng dfferent facal expressons, lghtng condtons, and backgrounds. here are 379 subjects wth 5 dfferent facal expressons. We randomly selected frontal-vew mages of 90 dstnct subjects, wth 5 samples per subject. Resolutons of the probe mages selected for experments are 0 0 and 5 5. Images n the CAS-PEAL-R database were captured at a dstance, so the Vola-Jones face-detecton algorthm [] was frst used to detect the face regon n each mage. Fg. 7 shows some mages from the CAS-Peal-R database, and the correspondng LR mages. Fg.8. Recognton accuracy wth dfferent numbers of tranng samples per subject (LR mage:6 6) for the ORL database. Fg.9. Recognton accuracy wth dfferent numbers of tranng samples per subject (LR mage:6 6) for the Extended Yale-B database
Fg.0. Recognton accuracy wth dfferent numbers of tranng samples per subject (LR mage:0 0) for the CAS-PEAL-R database VI. CONCLUSION We have proposed a new approach for recognton of lowresoluton face mages wthout performng super-resoluton. Our method s based on sparse representaton of local features, where local Gabor features are projected onto a new lowdmensonal feature space, where the sparse structure of the data s preserved. After projecton, lnear regresson s appled to estmate the coeffcent vectors that defne the relatonshp between the tranng and testng features. Classfcaton s then performed based on computng the resdual errors of representng a testng feature n terms of tranng features. he test mage s assgned to the class wth mnmum reconstructon error. ACKNOWLEDGMEN [8] J. Wrght, A. Y. Yang, and A. Ganesh, Robust face recognton va sparse representaton, IEEE rans. Pattern Anal. Mach. Intell., vol. 3, no., pp. 0-7, 008. [9] L. Zhang, M. Yang, and X. Feng. "Sparse representaton or collaboratve representaton: Whch helps face recognton?" n Proc. Int. Conf. Comput. Vs. (ICCV), 0. [0] H. Chang, D.-Y. Yeung, and Y. Xong, Super-resoluton through neghbor embeddng, n Proc. IEEE Conf. Comput. Vs. Pattern Recognt., pp. 75-8, 004. [] P. Vola and M. J. Jones, Robust real-tme face detecton, Internatonal Journal of Computer Vson, vol. 57, no., pp. 37-54, 004. [] L. Shen and L. Ba, A revew on Gabor wavelets for face recognton, Pattern Analyss and Applcatons, vol. 9, no., pp. 73-9, 006. [3] M. Jan and K.-M. Lam, Smultaneous Hallucnaton and Recognton of low-resoluton faces based on sngular value decomposton, IEEE rans. on Crcuts and Systems for Vdeo echnology, vol. 5, no., pp. 76-77, 05. [4] K.-H. Pong and K.-M. Lam, Mult-resoluton feature fuson for face recognton, Pattern Recognt., vol. 47, no., pp. 556-567, 04. [5] W. W. W. Zou and P. C. Yuen, Very low resoluton face recognton problem, IEEE rans. Image Process., vol., no., pp. 37-340, 0. [6] X. He and P. Nyog, Localty preservng projectons, n Proc. Conf. Advances n Neural Informaton Processng Systems, vol. 6, p. 53, 004. [7] X. He, D. Ca, S. Yan and H. J. Zhang, Neghborhood preservng embeddng, n Proc. Int. Conf. Comput. Vs. (ICCV), vol., pp. 08-3, 005. he work descrbed n ths paper was supported by an nternal grant from he Hong Kong Polytechnc Unversty, Hong Kong. REFERENCES [] M. urk and A. Pentland, Egenfaces for recognton, J. Cognt. Neurosc, vol.3, no., pp. 7-86, 99. [] P.N. Belhumeur, J.P. Hespanha and D.J. Kregman, Egenfaces vs Fsherfaces: Recognton usng class specfc lnear projecton, IEEE rans. Pattern Anal. Mach. Intell., vol. 7, no. 7, pp. 7-70, 997. [3] X. He, S. Yan, Y. Hu, P. Nyog, and H. Zhang, Face recognton usng Laplacanfaces, IEEE rans. Pattern Anal. Mach. Intell., vol. 7, no. 3, pp. 38-340, 005. [4] P.H. Hennngs-Yeomans, S. Baker, and B. V. K. V. Kumar, Smultaneous super-resoluton and feature extracton for recognton of low-resoluton faces, n Proc. IEEE Conf. Comput. Vs. Pattern Recognt., pp. -8, 008. [5] B. L, H. Chang, S. Shan, and X. Chen, Low-resoluton Face Recognton va Coupled Localty Preservng Mappngs, IEEE Sgnal Process. Lett., vol. 7, no., pp. 0-3, 00. [6] C.-X. Ren, D.-Q. Da, and H. Yan, Coupled kernel embeddng for low resoluton face mage recognton, IEEE rans. Image Process., vol., no. 8, pp. 3770-3783, 0. [7] S. Bswas, K. W. Bowyer, and P. J. Flynn, Multdmensonal scalng for matchng low-resoluton Face mages, IEEE rans. Pattern Anal. Mach. Intell., vol. 34, no. 0, pp. 09-030, 0.