A Comparison of South East Asian Face Emotion Classification Based on Optimized Ellipse Data Using Clustering Technique

Transcription

1 A Comprson of South Est Asn Fce Emoton Clssfcton Bsed on Optmzed Ellpse Dt Usng Clusterng Technque K. Muthukruppn, S. Thrugnnm, R. Ngrjn, M. Rzon, S. Ycob, M. Muthukumrn3, nd T. Rmchndrn3 School of Scence nd Engneerng, Mnpl Interntonl Unversty (MIU), Nl, Neger Sembln, Mlys School of Mechtroncs Engneerng, Unverst Mlys Perls (UnMAP), 0600 Jejw, Perls, Mlys 3 Deprtment of Computer Scence nd Networked System, Sunwy Unversty, Bndr Sunwy, Selngor, Mlys Eml: krthgyn@eee.org, krthgyn.muthukruppn@mu.edu.my n combntons n the nput fce-profle vdeo. A recognton rte of 87% s reported []. The moton sgntures produced re then clssfed usng Support Vector Mchnes s ether non-expressve or s one of the sx bsc emotons. The completed system s demonstrted n two smple but effectve computng pplctons tht respond n rel-tme to the fcl expressons of the user, thereby provdng the potentl for mprovements n the ntercton between computer nd user [3]. The unverslly ccepted ctegores of emoton, s ppled n humn computer ntercton re: Sd, Anger, Joy, Fer, Dsgust (or Dslke) nd Surprse. In ths pper, mge preprocessng, flterng, edge detecton methods tht re sutble for feture extrcton re presented nd compred towrds pplyng t to get lp nd eye fetures. A set of ftness functons for the GA methods re lso proposed s sutble for fce emoton recognton. Such n pproch of determnng the emotons re hghly suted for personlzed fce. Fuzzy C-men clusterng s mplemented to clssfy the emoton bsed on the optmzed vlues. The subject South Est Asn; The two subject hve been used for emoton clssfcton. The developed process flow for the mge processng, feture extrcton nd clssfyng the emotons s shown n Fg.. Abstrct In ths pper, usng set of rregulr nd regulr ellpse fttng equtons usng Genetc lgorthm (GA) re ppled to the lp nd eye fetures to clssfy the humn emotons. Two South Est Asn (SEA) fces re consdered n ths work for the emoton clssfcton. There re sx emotons nd one neutrl re consdered s the output. Ech subject shows unque chrcterstc of the lp nd eye fetures for vrous emotons. GA s dopted to optmze rregulr ellpse chrcterstcs of the lp nd eye fetures n ech emoton. Tht s, the top porton of lp confgurton s prt of one ellpse nd the bottom of dfferent ellpse. Two ellpse bsed ftness equtons re proposed for the lp confgurton nd relevnt prmeters tht defne the emotons re lsted. The GA method hs cheved resonbly successful clssfcton of emoton. In some emotons clssfcton, optmzed dt vlues of one emoton re messed or overlpped to other emoton rnges. In order to overcome the overlppng problem between the emoton optmzed vlues nd t the sme tme to mprove the clssfcton, fuzzy clusterng method (FCM) of pproch hs been mplemented to offer better clssfcton. The GA-FCM pproch offers resonbly good clssfcton wthn the rnges of clusters nd t hd been proven by pplyng to two SEA subjects nd hs seen mprovement compred to the erler work. Index Terms ellpse ftness functon, genetc lgorthm, emoton recognton, fuzzy clusterng I. INTRODUCTION In recent yers, there hs been growng nterest n mprovng ll spects of ntercton between humns nd computers especlly n the re of humn emoton recognton by observng fcl expresson. Ekmn nd Fresen developed the most comprehensve system for syntheszng fcl expresson bsed on wht they cll s cton unts []. In order to determne the ctegory of emoton, 5 fcl ponts n fce-profle sequence hs been recommended. The lgorthm performs both utomtc segmentton of n nput vdeo mges of fcl expressons nd recognton of 7 AUs occurrng lone or Fgure. Process flow of mge processng Mnuscrpt receved Jnury, 05; revsed My 30, Journl of Imge nd Grphcs do: 0.878/jog.3..-5

2 II. FACE IMAGE PROCESSING As the frst step n mge processng, the regon of nterest (ROI) hs been selected n the cqured mge. The ROI mge s converted nto gryscle mge (0-55). Before obtnng the fltered gryscle mge, hstogrm equlzton method hs been ppled. In the mge sequence, the hstogrm equlzed mge s fltered usng verge nd medn flters [4] n order to mke the mge smoother. Fnlly, Sobel edge detecton method s ppled to the fltered mge. Due to lght ntensty vrton, the segmentton process hs not been crred out successfully. In the edge detected mge of the whole fce, the eyes re properly segmented wheres the lp segmentton s poor for South Est Asen fce; so the hstogrm equlzed mge s splt nto eyes ROI nd lp ROI regons. The ROI lp regon s cropped from the full mge. Vrous edge detecton methods such s Sobel, Prewtt, Cnny, Roberts nd Log hve been ppled to the mge. A comprson hs been mde mong the edge detecton methods nd t s found tht the Sobel edge detecton method [5] performed well compred to other four methods. III. FEATURE EXTRACTION A feture extrcton method s now to be ppled to the edge detected mge to extrct fetures. Three feture extrcton methods re consdered nd ther cpbltes re compred n order for doptng the one tht s sutble for the proposed fce emoton recognton problem. They re projecton profle, contour profle nd moments [6]. The performnce of ech of the bove descrbed feture extrctng methods s compred wth respect to processng tme usng the edge detected mge of the lps nd eyes. The projecton profle s found to perform well n feture extrcton wth regrds to the processng tme nd s dopted here. The projecton profle hs been lso found to hve performed well n erler works [5]-[7]. IV. FACE EMOTION RECOGNITION USING GENETIC ALGORITHM In the erly 970s, John Hollnd, one of the founders evolutonry computtons, ntroduced the concept of genetc lgorthm [8]. Genetc lgorthm (GA) s heurstc method used to fnd pproxmte solutons to solve problems through pplcton of the prncples of evolutonry bology. GA dopts bologclly-derved technques such s nhertnce, mutton, nturl selecton, nd recombnton (or crossover). Some spects of vson system nd mge processng methodologes hve been dscussed n erler lterture towrds pproxmtng the fce s best ellpse usng GA. In the feture extrcton stge, the GA s ppled to extrct the fcl fetures such s the eyes, nose nd mouth, n set of predefned sub regons. Some smulton hs been crred out [4]. A method tht extrcts regon of eyes out of fcl mge by GA hs been suggested recently [9]. In ths pper, the fce fetures such s eyes nd lps re consdered n dfferent perspectve. The humn lp shpe s more of towrds combnton of two ellpse nd we cll ths s s rregulr ellpse. The word rregulr mens tht the ellpse hs two dfferent mnor xes wheren mjor xe remns the sme. Lengths of mnor xes of the lp feture for ech emoton re computed. The mjor xs s more or less fxed for the eye of prtculr person. The mjor xs (consdered to be fxed) nd two mnor xes re b nd b (to be computed). It s shown n Fg.. The humn eye shpe s more of towrds ellpse. The preprocessed eye mge s consdered s n ellpse. Then, the mnor xs of the eye feture cn represent n emoton. The ellpse cn be prmeterzed by ts mnor nd mjor xes. The mjor nd the mnor xes re (more or less fxed) nd b (to be computed) respectvely [7]. Ths s shown n Fg. 3. The ellpse s defned by ts equton s n Equton (). A generl form of the regulr ellpse defned by x y () b Fgure. Irregulr ellpse wth mnor nd mjor xs Fgure 3. Ellpse wth mnor nd mjor xs In ths work, new set of ftness functons s suggested n order to fnd the three mnor xes so tht the emoton chnges cn be recognzed. The ftness functon, to be dscussed ltter, for pplyng GA, s derved to optmlly compute sem-mnor xes, b, b nd b, for the top lp re, bottom lp re nd eye re respectvely; emotons cn thus be relted to the vlues of b, b nd b. A. Ftness Functon A ftness functon s prtculr type of objectve functon tht quntfes the optmlty of soluton (tht s, chromosome) n GA problem so tht ths chromosome my be rnked gnst ll the other chromosomes. A ftness vlue reflectng the mount of overlppng between the regons covered by the overld boundres s computed for ech chromosome. A pr of ndvduls re selected wth probblty proportonl to ther ftness nd mted to reproduce ther next generton. The process s repetedly performed wth the sme number of ndvduls of the prevous epoch. The ftness functon, Equton (), (3) nd (4) wth b, b nd b re derved bsed on the generl ellpse equton, Equton () s m n row() f ( x) col( j) X ( j () fx 0 05 Journl of Imge nd Grphcs

3 ( ) ( j (3) m n f x col j X fx 0 row() m n row() f ( x) col( j) X ( (4) j In Eq. (), (3) nd (4), col(j) s sum of whte pxels occuped by the column nd row() s number of rows of whte pxels n the row. Equtons (), (3) nd (4) re ftness functons of top lp, bottom lp nd eye respectvely. The lp nd eye fetures hve been gven s nput to the genetc lgorthm to fnd the optmzed vlues of b& b nd b. The selected vlues of GA prmeters re ndcted n the Tble I. The process of optmzton hs been crred out for 5 tmes for ech emoton. Ths process of optmzton s found to be gvng fvorble three mnor xs vlues b, b nd b for top lp re, bottom lp re nd eye re respectvely. Tble II ndctes the mnully mesured vlues of b, b nd b nd the correspondng optmzed vlues of X, X nd X for SEA nd SEA s shown n Fg. 4 for SEA. The emoton bsed on mnor xes of the lp feture nd eye feture cn now be estmted. The experment result shows tht the two mnor xs (b nd b) of the lp feture nd one mnor xs (b) of eye feture re dfferent for ech emoton there by dstnctons re possble. However, the rnges of (b, b, b) of one emoton sometmes overlp wth those of other emotons. Fuzzy C-Men clusterng pproch s dopted to crcumvent ths problem. TABLE I. PARAMETER SETTINGS FOR GA PROCESSING Generton 50 Populton sze 0 Selecton Functon Roulette Mutton Gussn Crossover Scttered Stll generton 50 Stll tme 0 TABLE II. OPTIMIZED VALUES OF THE FEATURES FOR SEA AND SEA SEA SEA Emotons Mnully Computed Men Vlue (n pxels) Optmzed Men Vlue by GA (n pxels) Mnully Computed Men Vlue (n pxels) Optmzed Men Vlue by GA (n pxels) b b b X X X b b b X X X Neutrl Fer Hppy Sd Angry Dslke Surprse Fgure 4. Optmzed vlues of the fetures for se V. EMOTION CLASSIFICATION USING FUZZY C-MEAN CLUSTERING Fuzzy clusterng hs been wdely ppled n pttern recognton, mge processng nd dt nlyss. The fuzzy c-men (FCM) lgorthm hs been mplemented to prtton fnte collecton nto collecton of c fuzzy clusters [0]-[]. From these optmzed vlue, ech emoton hs ts own rnge of vlue for lp nd eye, n some cses the emoton rnges overlp wth other emoton rnge such tht need some ntellgence to cluster the ech emoton. The purpose of FCM here s to clusters the gven dt bsed on the group or emoton, tht mens sme rnge of dt re clustered together. Even though, the rnges re overlpped, the clusterng comes wth proper clssfcton of emotons. Here, FCM s employed to clssfy the emoton for the optmzed 3-D dt of top lp, bottom lp nd eye vlue. The proposed FCM model s s shown n Fg. 5. It shows 3 optmzed fetures vlue s nputs to the model for two SEA nd. It offers 7 center cluster vlues of the group dt such s C to C7. The prmeter settng for clusterng settng s gven n Tble III for SEA nd SEA. The clustered center pont vlue of the 3-D Dt set of emoton s gven n the Tble IV nd Tble V for SEA nd SEA respectvely. 05 Journl of Imge nd Grphcs 3

4 Fgure 5. FCM Model TABLE III. PARAMETER SETTINGS FOR CLUSTERING TECHNIQUE S. No Prmeters Implemented Method of Intlzton Rndom Exponent of prtton mtrx 3 Number of clusters 7 4 Dstnce Functon Euclden dstnce TABLE IV. CLUSTERED CENTER POINT SEA X Y Z Expected Emoton Hppy Neutrl Angry Fer Nerly dslke Surprse Sd TABLE V. CLUSTERED CENTER POINT SEA X Y Z Expected Emoton Fer Sd Dslke Hppy Surprse Angry Neutrl VI. DISCUSSION In ths study on South Est Asen subjects for SEA nd SEA, sx emotons nd one neutrl hd been consdered. The verge nd medn flters were ppled to smoothen the mge. The Sobel edge detecton ws found to perform well, snce t offered better segmentton thn other 4 methods. The GA ws then ppled to get the optmzed vlues of the mnor xes, b, b & b of the rregulr ellpse nd regulr ellpse by usng set of proposed ftness functons. These optmzed vlues of emotons rnge were overlpped. In order to overcome ths problem, FCM ws mplemented to offer better clssfcton. The prmeters were set to run the FCM method. All optmzed emoton dt were gven to FCM model. The FCM clustered the optmzed emoton dt were clustered (seven clusters). Bsed on the rnge of GA, the emoton of ech center ws determned. All center vlues cme under ther respectve emoton rnge except one. All other center s emoton ws determned by ther respectve rnge. The unknown rnge cnnot be determned nd ngry emoton center s lso mssng for SEA. In SEA, t s ble to dentfy the ngry emoton. Ths FCM were cpble of gvng of center vlue ll emoton. It offered of bout 85.7% successful clssfcton for SEA nd 88.57% for SEA. It shows tht the clssfcton rte hs been mproved nd ble to determne the center poston for Angry compred to SEA wth the sme FCM Model. The reson behnd the vrton n the rte nd mssng emoton for SEA s due to subject emoton expresson whch s unque cnnot be generlzed. VII. CONCLUSION In ths pper, set of sutble sequences n mge processng nd new ftness functons for the GA pplcton hve been proposed for determnng the toplp, bottom-lp nd eye fetures. The suggested sequence of mge processng offers cceptble mges. The proposed method hd shown successful clssfctons of emoton. Ths method of emoton clssfcton s generl pproch to lps nd eyes of ny fce. It offered of rnge between 85.7% to 88.57% clssfcton rte for SEA bsed on SEA nd SEA. The mplementton of FCM plyed the vtl role n clssfyng the emotons bsed on the optmzed vlues of top lp, bottom lp nd eye. Ths s more sutble to South Est Asn subject. In SEA, ble to provde the ngry emoton center pont compred to SEA. It hs shown the mprovement. REFERENCES [] H. L, Computer recognton of humn emoton, n Proc. Interntonl Symposum on Intellgent Multmed, Vdeo nd Speech Processng, Hong Kong, My 00, pp [] M. Pnt nd I. Ptrs, Dynmcs of fcl expresson: recognton of fcl ctons nd ther temporl segments from fce profle mge sequences, IEEE Trnsctons on Systems, Mn, nd Cybernetcs Prt B: Cybernetcs, vol. 36, no., pp , Apr. 006 [3] K. Anderson nd P. W. McOwn, A rel-tme utomted system for the recognton of humn fcl expressons, IEEE Trnsctons on Systems, Mn, nd Cybernetcs Prt B: Cybernetcs, vol. 36, no., pp , Feb [4] G. G. Yen nd N. Nthnndn, Fcl feture extrcton usng genetc lgorthm, n Proc. Congress on Evolutonry computton, My -7, 00, vol., pp [5] M. Krthgyn, M. Rzon, S. Ycob, nd R. Ngrjn, An edge detecton nd feture extrcton method sutble for fce emoton detecton under uneven lghtng, n Proc. The nd Regonl Conference on Artfcl Lfe nd Robotcs (AROB 06), Hty, Thlnd, Jul. 4-5, 006. [6] R. Ngrjn, S. Ycob, P. Pndyn, M. Krthgyn, S. H. Amn, nd M. Khld, A rel tme mrkng nspecton scheme for semconductor ndustres, Interntonl Journl of Advnce Mnufcturng Technology, vol. 34, pp , 006. [7] M. Krthgyn, M. Rzon, et l., Development of personfed fce emoton recognton technque usng ftness functon, Journl of Artfcl Lfe nd Robotcs, vol., pp , 007. [8] M. Negnevtsky, Artfcl Intellgence, Englnd: Addson Wesley, Person educton Lmted, 00. [9] H. Tn, K. Terd, S. Oe, nd J. Ymguch, Detectng of one s eye from fcl mge by usng genetc lgorthm, n Proc. The 7 th Annul Conference of the IEEE Industrl Electroncs Socety, 00, pp Journl of Imge nd Grphcs 4

5 [0] Z. Yn, Y. Tng, F. Sun, nd Z. Sun, Fuzzy clusterng wth novel seprble crteron, Tsnghu Scence & Technology, vol., pp , Feb [] F. Hoppner, F. Klwonn, R. Kruse, nd T. Runkler, Fuzzy Cluster Anlyss Methods for Clssfcton, Dt Anlyss nd Imge Recognton, Englnd: John Wley & Sons, 999. [] R. J. Almed nd J. M. C. Sous, Comprson of fuzzy clusterng lgorthms for clssfcton, n Proc. Interntonl Symposum on Evolvng Fuzzy Systems, Lke Dstrct, UK, Sep. 006, pp. -7. Krthgyn Muthukruppn hd receved Bchelor s Degree from Peryr Unversty n Electroncs nd Communcton Engneerng (ECE), Msters n Electrcl nd Electroncs Engneerng (EEE) from Unverst Mlys Sbh nd PhD Mechtroncs Engneerng from Unverst Mlys Perls. Krthgyn s n Assstnt Professor n Electrcl nd Electroncs Engneerng t Mnpl Interntonl Unversty (MIU), Mlys. Krth hd been ttched to few unverstes nd Compnes n hs creer nd hve more thn 0 yers of experences. He hd been workng n Intel R&D center nd MIMOS Berhd (Ntonl R&D Center) Mlys before jonng MIU. He s lso Chrtered Engneer (Engneerng Councl, UK). He s secretry for IEEE System, Mn nd Cybernetcs socety (Mlys chpter). He s member of IEEE Systems, Mn nd Cybernetcs Socety (USA), Insttute of Engneer nd Technology (IET) nd MIAENG. He s lso member s n Interntonl edtor n nformton technology journl, Kore. Hs nme ws ncluded n the Mrqus slver jublee edton of Who s Who n the World (USA) 00. He hs been prt of revewng commttees, orgnzng commttee for few Interntonl conferences nd Journls. He hs more thn 50 publctons n book chpter, nterntonl journls nd conferences. Hs reserch nterests focusng n the feld of mge processng, Artfcl Intellgence, Dt Mnng, Bomedcl, Vson, Semntc Technology, Pttern Recognton, Mchne Lernng nd Humn Computer Intercton. Professor Rmchndrn Ngrjn obtned hs BE(Hons), M.Tech nd PhD degrees from Mdrs Unversty, IIT Knpur nd Mdrs Unversty, Ind, respectvely. He worked n techng nd reserch postons n unverstes of Ind, Irq, Mlys, the UK nd Jpn. He s currently wth Unversty Mlys Perls (UnMAP), Mlys, s Professor n the School of Mechtroncs Engneerng. He hs been workng n the felds of Robotcs nd Intellgent Control for severl yers. Severl Mster, Doctorl nd Post-Doctorl students hve receved ther degrees nd credentls through hs supervson. He hs severl contrbutons s peer revewed Journl nd Conference ppers, Books, Book chpters, monogrphs nd documented reserch reports. He hs offered nvted lectures n unverstes of Ind, Mlys, Jpn nd Twn. Hs current felds of nterest re n Hosptl Robots, Emoton Controlled Mchnes nd Robot bsed SLAM. He hs edted wth Professor Junzo Wtd, Wsed Unversty, Ktkyusu, Jpn, specl ssue for BMSA on Bo-sgnl: Dt cquston, processng nd control. Professor Ngrjn s Lfe Fellow of IE (Ind), Senor Member of IEEE (USA), Member, Assocton of Bomedcl Soft computng (BMSA), Jpn, nd Member, IET (UK). Professor Mohmd Rzon receved hs B.Eng, M.Eng n Electrcl nd Electroncs from Unversty of Tokushm Jpn n 993 nd 9995 respectvely. From 995 to 997, he ws softwre engneer t System LSI Lbortory, Mtsubsh Electrc Corp, Itm, Jpn. He obtned Dr.Eng from Deprtment of Computer Scences nd Intellgent Systems, Ot Unversty, Jpn n 00. He ws the hed deprtment of Bomedcl Engneerng, n the Unversty of Mly nd currently s Professor nd Den for engneerng fculty n Unverst Mlys Terenggnu. Hs reserch nterests nclude pttern recognton, fce nlyss, bometrcs systems nd sgnl processng. Professor Szl Ycob receved hs BEng n Electrcl Engneerng from Unverst Mly nd lter pursued hs MSc n System Engneerng t Unversty of Surrey nd PhD n Control Engneerng from Unversty of Sheffeld, Unted Kngdom. He ws promoted to Assocte Professor by Unverst Mlys Sbh nd lter pponted s the frst den of the School of Engneerng nd Informton Technology from August 998 tll August 004. Currently, he s servng t Unversty Mlys Perls s Professor n School of Mechtronc Engneerng nd ws the Den of the School from Mc 005 tll Februry 007. He hs publshed more thn 00 ppers n Journls nd Conference Proceedngs. He hs supervsed more thn 5 postgrdute students n ether Mster or Doctor of Phlosophy levels. Hs reserch nterests re n Artfcl Intellgence pplctons n the felds of coustcs, vson nd robotcs. In recognton of hs expertse, severl reserch grnts hve been wrded to hm by Mnstry of Scence nd Technology to tke up projects n Ptent Lftng Robot, Actve Nose Cncellton, Acoustc Anlyss nd Clssfcton of Pthologcl Voce. In 005, hs journl pper n Intellgent Vson ws publshed nd wrded The Sr Thoms Wrd Memorl Prze by Insttuton of Engneer (Ind). Medls n the Ntonl nd Interntonl Exhbton were conferred to hs work on Robotc Force sensor nd Nvgton Ad for Vsully Impred respectvely. He receved Chrted Engneer sttus by the Engneerng Councl, Unted Kngdom n 005 nd lso member to the Insttute of Engneerng nd Technology, Unted Kngdom snce 003. Muthukumrn Mruthpp hs frst clss Mster s degree n Softwre Engneerng from Mdur Kmrj Unversty, nd he hs done hs bchelor degree n mthemtcs t Bhrthdsn Unversty. He hs been ttched, fmlr n vrous eductonl systems nd hve more thn 9 yers of experences n educton. He hs been prt of orgnzng commttee t the locl unverstes nd hs reserch nterests focusng n the feld of mge processng, ubqutous computng, bg dt, pttern recognton, nd Mchne Lernng. Thnhrn Rmchndrn hd receved Bchelor s Degree from Unversty of Hertfordshre UK on Electrcl nd Electroncs Engneerng nd mster of Informton Technology n Network Computng from Open Unversty Mlys. Thnhrn s Lecturer t Unversty nd Techncl Drector t Joslb Technologes. He hs lmost 7 yers of experence n cdemc nd Industrl R&D projects. He s member of Insttute of Engneer nd Technology (IET) nd Member of IEEE Consumer Electroncs. He s the executve member IEEE Mlys Chpter Consumer Electroncs crryng the portfolo of Professonl development nd cdemc reserch collborton. He s lso techncl consultnt for SIRIM Mlys of Automton nd Robotc cluster. Hs reserch nterest s focused to Robotcs, Intellgent Systems nd Wreless Sensor Network. Assocte Professor Srgunm Thrugnnm hd receved Bchelor s Degree from Bhrthr Unversty n Appled Scences nd Computer Technology, Ind, Msters n Computer Scence from Guru Ghsds Unversty, nd Mster of Busness Admnstrton from Mdur Kmrj Unversty, Ind. Srgunm s n Assocte Professor n Computer Engneerng t Mnpl Interntonl Unversty (MIU), Mlys. Srgunm hs been ttched to unverstes nd colleges n hs creer nd hve more thn 0 yers of experence n cdemcs. He hd been workng s Drector of Acdemcs nd Vce Presdent of Acdemcs before jonng MIU. He s member of IEEE Computer Socety (USA), Insttute of Engneer nd Technology (IET) nd MIAENG. Hs reserch nterests focusng n the feld of Computer Archtecture, Bo-sgnl processng, Artfcl Intellgence for Dt Mnng nd Humn Computer Intercton. 05 Journl of Imge nd Grphcs 5