{webb, cc.gatech.edu. {yorke3, ece.gatech.edu

Transcription

1 A Dscrmnatve Classfer Learnng Approach to Image Modelng and Identfcaton Byungk Byun, Chn-Hu Lee, 2 Steve Webb, and 2 Calton Pu School of Electrcal & Computer Engr. Georga Insttute of Technology Atlanta, GA , USA {yorke3, ece.gatech.edu 2 College of Computng Georga Insttute of Technology Atlanta, GA , USA {webb, cc.gatech.edu ABSTRACT We propose a dscrmnatve classfer learnng approach to mage modelng for spam mage dentfcaton. We analyze a large number of mages extracted from the SpamArchve spam corpora and dentfy four key spam mage propertes: color moment, color heterogenety, conspcuousness, and self-smlarty. These propertes emerge from a large varety of spam mages and are more robust than smply usng vsual content to model mages. We apply mult-class characterzaton to model mages sent wth emals. A maxmal fgure-of-mert (MFoM) learnng algorthm s then proposed to desgn classfers for spam mage dentfcaton. Expermental results on about 240 spam and legtmate mages show that mult-class characterzaton s more sutable than sngleclass characterzaton for spam mage dentfcaton. Our proposed framework classfes 8.5% of spam mages correctly and msclassfes only 5.6% of legtmate mages. We also demonstrate the generalzaton capabltes of our proposed framework on the TREC 2005 emal corpus. Mult-class characterzaton agan outperforms sngle-class characterzaton for the TREC 2005 emal corpus. Our results show that the technque operates robustly, even when the mages n the testng set are very dfferent from the tranng mages.. INTRODUCTION Text-based learnng flters have grown n sophstcaton and effectveness n flterng emal spam [3, 5, 7]. In response, spammers have adopted a number of countermeasures to crcumvent these text-based flters. Currently, one of the most popular spam constructon technques nvolves embeddng text messages nto mages and sendng ether pure mage-based spam or a combnaton of mages and text (typcally legtmate-lookng text wth legtmate content). Ths strategy, usually called magespam, has been successful n bypassng text-based spam flters, posng a new challenge for spam researchers [24]. Attempts to use optcal character recognton (OCR) technques to convert spam mages back to text for processng by text-based flters have been foled [5]. An effectve response by spammers s the applcaton of CAPTCHA (Completely Automated Publc Turng test to tell Computers and Humans Apart) technques, whch are desgned to preserve readablty by humans but capable of effectvely confusng the OCR algorthms [7, 20]. In ths paper, we explore a dfferent mage analyss approach that s capable of dscernng more physcal propertes. Our hypothess s that these dstnctve propertes can help mage flters separate spam mages from legtmate mages. Several approaches that extract spam mages propertes have been proposed. In [24], the exstence of text-regons, the number of banners and graphc mages, and the locaton of mages were extracted as spam mages dstnctve propertes. In [], the exstence of text-regons, saturaton of color, and the heterogenety of color were dentfed. In both papers, good results were reported for detectng spam mages. Despte the success of these prevous studes, the most ndcatve propertes of spam mages stll need to be found to enhance model dscrmnaton performance and robustness aganst mage varaton. In the two approaches dscussed above, the exstence of text-regons was assumed to be the most ndcatve property. In ths study, we analyze a number of spam mages and defne four specfc propertes: color moments, color heterogenety, conspcuousness, and self-smlarty. We use several mage and sgnal processng technques, such as extractng color nformaton, clusterng, and evaluatng the outputs of log-gabor flter banks, to perform feature extracton. We also explot mult-class characterzaton of spam and legtmate mages to account for the dversty n mages that covers a wde range of vsual content. In [, 24], spam mages are modeled wthn a sngle class. However, as mentoned above, mages that are transmtted n emals are dffcult to model. Due to these dffcultes, t s desrable to adopt mult-class characterzaton technques that characterze spam and legtmate mages wth multple models. These models are defned as sub-classes, and n our expermental framework, spam and legtmate mages are further dvded nto three sub-classes, respectvely. Snce we use more than one model to descrbe spam and legtmate mages, we need to defne a decson rule. We propose usng three decson rules: selectng the maxmum score, takng an arthmetc average, and takng a geometrc average. Then, an MFoM-based dscrmnatve learnng algorthm s proposed to dentfy spam mages. Wth the decson rule that selects maxmum scores, experments show that our framework works better than an approach explotng sngle-class characterzaton. Specfcally, our framework yelds a spam mage dentfcaton CEAS 2007 Fourth Conference on Emal and Ant-Spam, August 2-3, 2007, Mountan Vew, Calforna USA

2 rate of 8.5% and a false postve rate of only 5.6%. We also test our framework on a dfferent dataset to prove that our framework has strong generalzaton capabltes. Concretely, we used our framework to evaluate a test set of mages that were extracted from the TREC 2005 emal corpus. Although these mages are qute dstnct from the mages used n the tranng set, our framework was able to mantan a hgh level of performance. These postve results are consstent wth our hypothess, and they also encourage further nvestgaton nto the spam mages as an effectve component technology for separatng complex spam emals (e.g., those contanng text and multmeda nformaton) from legtmate emals. The remander of ths paper s organzed as follows. In Secton 2, we defne mult-class characterzaton n detal. In secton 3, we descrbe dstnctve propertes for feature extracton. Secton 4 proposes decson rules for mult-class characterzaton, and Secton 5 explans our MFoM-based learnng algorthm. Wth Secton 6, we conclude ths paper. 2. MULTI-CLASS CHARACTERIZATION In most mage modelng technques rangng from content-based to concept-based, a sngle class model s learned for a sngle mage class. For example, n mage categorzaton problems, a sngle classfer s constructed to model each ndvdual mage class [4, 2, 2]. Smlarly, n a concept model mplementaton problem, each concept wll be modeled by a sngle classfer that characterzes the concept [2, 8]. In many other cases, ths sngle class characterzaton scheme has been adopted as well. However, n spam mage dentfcaton, the above scheme may not be sutable because the varety of mages that are transmtted n emals s too large to capture n a sngle model. A spam mage mght contan text messages wth a unform background; t mght contan a complex background wthout any text messages, or t mght contan any number of other varatons. The dversty of legtmate mages s also ggantc. Snce mult-class characterzaton uses multple models to characterze a sngle class, we beleve t should be appled to help cover the wde varety of mages found n emals. Thus, n our proposed framework, a spam mage class and a legtmate mage class are descrbed wth several models, whch are called sub-classes. In [], they defned four categores ( photos, baby, graphcs, and screenshot ) to represent legtmate mages. However, mult-class characterzaton was not appled n ther study. Each category was regarded as a separate mage class not a sub-class. Thus, nstead of unfed spam mage dentfcaton results (.e., spam vs. legtmate), four separate classfcaton results (.e., spam vs. each legtmate category) were reported. Mult-class characterzaton s extremely advantageous n spam mage categorzaton because t mproves both effectveness and robustness. Wth a sngle-class characterzaton method, more sophstcated class models (e.g., support vector machnes) must be used to reflect the complexty of the problem. However, usng a mult-class characterzaton method, smple classfers can effectvely descrbe complcated mage classes. Addtonally, robustness can be acheved through mult-class characterzaton. Robustness s a very mportant ssue n spam mage dentfcaton because the mages found n emal can consst of vrtually anythng. Mult-class characterzaton s ntended to cover as many varatons as possble by ntroducng sub-classes, and as a result, t provdes more robustness than sngle-class characterzaton. To mplement mult-class characterzaton, two ssues must be addressed: () groupng mages nto several sub-classes and (2) desgnng decson rules and classfers. In the followng secton, we apply mult-class characterzaton to spam and legtmate mages. We explan how to desgn decson rules and classfers n Sectons 4 and Mult-Class Characterzaton of s For mult-class characterzaton, we prepared spam mages from SpamArchve and grouped them nto two groups: synthetc and non-synthetc mages. Synthetc mages are mages made wth any artfcal technques, whereas non-synthetc mages are mages wth no artfcal modfcatons. Typcal non-synthetc mages are sexual or female mages, and most synthetc mages are advertsng pctures. Synthetc mages can be parttoned nto two regons: message regons and background regons. However, some of the mages do not contan message regons. Addtonally, background regons are extremely dverse. For example, both unform and complex backgrounds are represented. Therefore, synthetc mages are characterzed further based on two crtera: complexty of background and exstence of text messages. Non-synthetc mages can be characterzed by ther content. In partcular, nonsynthetc mages can be grouped nto sexual and non-sexual mages. Based on these observatons, 5 sub-classes were generated. Table summarzes the mult-class characterzaton results for spam mages. Table. Mult-class Characterzaton of s Synthetc Text Content Text_Smple Yes Yes N/A Text_Complex Yes Yes N/A No_Text Yes No N/A Non_Synthetc_Se xual No No Sexual Non_Synthetc_Ot hers No No Others We can characterze spam mages n many dfferent ways. In partcular, n our experments, the bottom three sub-classes are combned and an Other sub-class s defned. Fgure shows examples for every sub-class except Non_Synthetc_Sexual because that sub-class contans pornographc content. 2.2 Mult-Class Characterzaton of Legtmate Images Unfortunately, a standardzed legtmate emal mage data set has not been made publcly avalable. Moreover, t s dffcult to obtan enough legtmate emal mage data due to several reasons ncludng copyrght ssues. The Enron Corpus (the largest publcly avalable legtmate emal corpus) does not contan any attachments, and we were only able to extract 288 legtmate mages from the TREC 2005 emal corpus. Thus, we utlzed mages across the 20 dfferent classes from Corel CDs, nstead. As mentoned earler, the goal of mult-class characterzaton s to

3 cover as many varatons as possble. Therefore, we also added more mages from Google Image Search 2 by searchng for four keywords: sports, baby, maps and drectons, and cartoons, to emulate more realstc stuatons. The keywords are chosen based on our observatons that mages belongng to such keywords are most frequently seen n legtmate emals. The characterzaton of legtmate mages s qute smple because metadata s avalable. In partcular, predefned class names are the metadata for CorelCDs, and keywords are used for Google Image Search. All mages from CorelCDs wth sports and baby mages from Google Image Search are ntegrated nto one subclass: photos. The rest of the mages are characterzed based on ther metadata: maps and drectons and cartoons. Fgure 2 shows examples of each legtmate mage s sub-class. (a) 3. DISTINCTIVE PROPERTIES OF SPAM IMAGES Extractng dstnctve propertes from spam mages s a crucal part of a spam mage dentfcaton task. Well-extracted features provde dscrmnaton and robustness. In ths secton, we dentfy four key propertes of spam mages: color moments, color heterogenety, conspcuousness, and self-smlarty. We descrbe the characterstcs of these propertes and explan feature extracton procedures. 3. Color Moments The frst dstnctve property of spam mages s color moments. Color s one of the most wdely used vsual features n mage retreval problems, and t s relatvely robust to background complcaton and nvarant to mage sze or orentaton [6]. In spam mages, several notable color characterstcs can be found such as dscontnuous dstrbutons, hgh ntensty, domnant peaks, etc. Fgure 3 llustrates such color characterstcs, comparng color dstrbutons of a spam mage (the upper mages), wth those of a legtmate mage (the lower mages). In the upperrght mage, t s easly notceable that the dstrbuton s not contnuous and that domnant peaks exst all over the channels, whle the lower-rght mage does not have these characterstcs. (b) (c) (d) Fgure. Spam mage examples; (a) Text_Smple, (b) Text_Complex, (c) No_Text, (d) Non_Synthetc (b) (a) (c) Fgure 2. Legtmate mage examples; (a) photos, (b) maps and drectons, (c) cartoons Fgure 3. Color dstrbutons of a spam mage and a legtmate mage n the HSV color space; focus on dscontnuty and domnant peaks n the upper-rght mage. The smplest way to extract these color characterstcs s to use color hstograms [9]. However, snce most of the nformaton s embedded n low-order moments, color moments can be used nstead [8]. In our framework, the frst and second central moments are computed. Frst, all mages are transformed to the HSV color space. HSV s a color representaton method that s smlar to the way humans perceve color [0, ]. Then, n the HSV color space, the frst and second central moments are computed for every channel. 2

4 3.2 Color Heterogenety Legtmate mages typcally convey a much larger number of colors than spam mages. Even n a person s face, large varatons can be observed, whch account for shades or llumnatons n the face. In contrast wth legtmate mages, color n spam mages usually stays constant. The background s generally flled wth the same colors, and a sngle sentence usually conssts of the same fonts and colors. In [], ths property was defned as a color heterogenety feature. They frst dentfed text regons and non-text regons, quantzng each regon wth at most N colors through a mnmum varance quantzaton algorthm. Then, RMS errors between the orgnal mages and the quantzed mages were calculated. We compute the RMS errors between the orgnal mages and the quantzed mages as well, but we do not make any separaton between the text-regon and non-text regons because we assume that color heterogenety s a global feature. Fgure 4 shows four legtmate mages and four spam mages: ()- (4) are legtmate, and (5)-(8) are spam. ()-(3) are n the photo sub-class, and (4) s n cartoon sub-class. (5) and (6) are n Text_Complex, and (7) and (8) are n Text_Smple. () (2) (3) (4) (5) (6) (7) (8) Fgure 4. Legtmate and spam mage examples; ()-(4) legtmate mages, (5)-(8) spam mages dfferent color heterogenety values so sngle-class characterzaton would blur the dscrmnatve characterstcs of spam mages. Fgure 6 plots the dstrbutons of RMS errors of spam mages and legtmate mages. Ths fgure clearly shows that most of the spam mages have fewer RMS errors than most of the legtmate mages. Fgure 6. Dstrbutons of color heterogenety feature; N s set to 8. X-axs corresponds to heterogenety values. Y-axs represents probabltes. 3.3 Conspcuousness Spammers want spam messages to be easly notceable to recevers so that desred actons can be generated (e.g., readng the message, clckng a lnk, etc.). Thus, t s natural for spam mages to use hghly contrasted colors. We defne ths property as conspcuousness, whch means obvous to the eye. In practce, we can dentfy many spam mages that use hghly saturated colors wth contrastng whte or black or whte contrastng black. In Fgure 4, mage (7) uses a hghly saturated yellow background color. In Fgure 4, mage (8) uses pure blue and red n text messages, whch s contrasted wth a whte-lke background. Ths property wll become manfest lookng from the SV plane, whch s a subspace of an HSV color space. If an mage s hghly conspcuous, hgh densty s expected to be seen at whte, black, and saturated colors. In the SV plane, such colors corresponded to three ponts: (0,), (0,0), and (,), respectvely. Based on ths dea, translatng the conspcuousness of mages nto a feature vector s done as follows. Frst, we represent an mage n the SV plane. Then, usng a k- means algorthm, we learn M centrods of the mage s SV dstrbutons. Wth the M centrods, we compute average dstances between the centrods and the above three ponts. In partcular, we search for the closest pont for each centrod and add the dstance up. Fnally, we average the total sum, and the average s the extracted conspcuousness. Fgure 5. Color heterogenetes of mages n Fgure 4; N s set to 8. X-axs corresponds to the mage numbers n Fgure 4. Y-axs represents the computed color heterogenetes. Fgure 5 shows the above mages resultng color heterogenety feature values. As the fgure shows, all of the four legtmate mages have larger values than the four spam mages. Addtonally, the mages n the Text_Complex sub-class ((5) and (6)) have larger values than mages n the Text_Smple subclass ((7) and (8)). Ths observaton llustrates the need for multclass characterzaton. Those two sub-classes exhbt consderably Fgure 7. Dstrbutons of conspcuousness feature; M s set to 8. X-axs represents the values of conspcuousness feature and y-axs represents probabltes.

5 Fgure 7 shows dstrbutons of the extracted conspcuousness feature for spam and legtmate mages. Ths fgure shows that for about half of the spam mages, the extracted feature values are less than 0.3, whereas the feature values are greater than 0.3 for over 97% of the legtmate mages. 3.4 Self-Smlarty In spam mages, some characterstc patterns can be found from text messages or the background. These patterns produce another dstnctve property of spam mages called self-smlarty. The reason why ths feature s called self-smlarty s because a smlarty between macroblocks of mages s measured. The term macroblock usually refers to a square block of MxM pxels, where M s small. The unform background of spam mages would be the smplest example that hgh self-smlarty s measured. In spam mages, the background s much more unform than legtmate mages. If the unform background s segmented nto macroblocks, only one type of macroblock wll be seen, and n turn, hgh selfsmlarty wll be measured. Text messages n spam mages also present hgh self-smlarty. In text-embedded spam mages, the fonts and szes that are used are typcally dfferent from sentence to sentence. However, n a sngle sentence, font and sze are constant. Therefore, smlar macroblocks wll be generated, and n turn, hgh self-smlarty would be expected. Based on our experments, text messages n dfferent fonts and szes create smlar macroblocks as well. To extract self-smlarty, we need to learn representatve patterns from data frst. We expect that nosy varatons n the data would be removed whle emphaszng self-smlarty through ths learnng process. Frst, we segment a whole mage nto several macro blocks. The sze of a macroblock s globally set to 32x32. When the sze of an mage s smaller than the sze of our macroblock, the sze s set to the mnmum of the two. Smlarty between macroblocks s computed usng a log-gabor flter bank [6, 3, 4]. Unlke a typcal Gabor flter bank, log-gabor flters can desgn arbtrarly large bandwdth whle keepng a DC component at zero [3]. We apply a log-gabor flter bank to each macroblock and compute a mean and a varance of each flter bank s output. We then explot an unsupervsed clusterng algorthm such as a k-means clusterng algorthm. Centrods of the resultng clusters become the representatve patterns. Now, mages are ndexed wth these patterns. Frst, the patterns are numbered wth to N, where N s the number of centrods, and each of the macroblocks s ndexed wth the closest pattern. We then count how often each pattern has appeared n an mage. Ths wll generate N dmensonal vectors. We fnally normalze these vectors as the sum of each element equals to. These become the extracted self-smlarty vectors. 4. MULTI-CLASS DECISION RULES In a typcal spam mage dentfcaton problem, for whch sngleclass characterzaton s used, the decson rule s farly smple: choose a class that maxmzes certan scores. Let X be a test vector, and let C and C represent a spam mage class and a legtmate mage class, respectvely. Moreover, let Λ and Λ be the correspondng parameter sets for C and C. Then, the decson rule can be descrbed n Eq () Choose C f g ( X, Λ ) > g( X, Λ ), Choose C otherwse, where g ( X, Λ ) and g ( X, Λ ) are class score functons for postve and negatve classes, respectvely. Mult-class characterzaton broadens the choces of decson rules n spam mage dentfcaton. Eq. () stll functons as an overall decson rule; however, snce multple sub-classes are used to descrbe an mage class, class score functons for a spam and legtmate class, g ( X, Λ ) and g ( X, Λ ) can be computed dfferently. The frst method s to assgn the maxmum scores between subclasses to g X, Λ ) and g X, Λ ). ( ( Here, we let X, Λ ) and g X, Λ ) be the sub-class s score g ( ( functons, where Λ andλ are the parameter sets for sub-class C and C. C s the th sub-class of a spam mage class C, and C s the th sub-class of a legtmate mage class C. Another way s to compute arthmetc averages of the sub-classes, whch s defned as: M g( X, Λ ) = g( X, Λ ), M = N (3) g( X, Λ ) = g( X, Λ ), N where M and N are the numbers of sub-classes for spam mages and legtmate mages, respectvely. The thrd approach s to compute geometrc averages of the subclasses. As an example, the equaton to compute a geometrc average for a spam mage class s shown n Eq. (4) where η s a postve constant. g ( X, Λ ) = max( g( X, Λ g( X, Λ Snce dfferent decson rules yeld dfferent decson boundares, the performances of above methods should vary. The frst method concerns the most relevant sub-classes n a spam and a legtmate mage class. The second and thrd methods measure average scores. We determne the best decson rule emprcally, and n secton 6, the performances of above three approaches are measured and compared aganst each other. 5. MFOM-BASED CLASSIFIER LEARNING In classfer learnng wth mult-class characterzaton, we estmate parameters Λ for classfers, gven tranng data { X, C}, wth Λ= { Λ, Λ M, N} and M ) = max( g( X, Λ N C = C C. Here, we adopt an MFoM-based learnng approach proposed n [9]. In an MFoM-based learnng approach, = )), )). () (2) M η g ( X, Λ ) log exp{ (, )} η = g X Λ, (4) M =

6 the classfer parameters Λ are estmated by drectly optmzng a certan performance metrc of classfers (e.g., detecton errors, F, or precson). A typcal performance metrc used s an average detecton error rate (DER). The DER s defned as an arthmetc average of false postve rate and false negatve rate: FP FN DER =, (5) 2N T C C where N s the number of classes, T s the total number of tranng data, ( ) s a cardnalty, and FP and FN are false postve error and false negatve error for the th class. However, we cannot optmze ths performance metrc analytcally snce FP and FN are dscrete enttes. Therefore, n MFoM-based learnng, a contnuous and dfferentable functon, called a class loss functon l, s ntroduced to approxmate FP and FN for class. Snce both FP and FN are computed from the error counts made from class, a class loss functon should smulate these error counts. In partcular a class loss functon should be close to zero when X s correctly classfed and should be close to one when X s msclassfed. Summng over the entre tranng data set, FP and FN can be approxmated wth as: where I( ) s an ndcator functon. For choosng a class loss functon, any functons satsfyng the above behavor would work. Typcally, a sgmod functon s adopted, defned as: where α and β are all postve constants, servng as parameters that determne the sze of the learnng wndow and the offset of decson boundary, and d s a functon of X and Λ, called a class msclassfcaton functon. Defnng a class msclassfcaton functon s the most mportant consderaton n an MFoM-based classfer learnng approach. d should be consstent wth the class loss functon s behavor. In partcular, d should be negatve for correct decsons and postve for ncorrect ones. Usng the same notaton used n Secton 4, we can defned as: where FP FN d d =g( X, Λ ) g( X, Λ ), (8) = = X X Λ s the parameter set of a competng class for class, whch s Λ for C or Λ for C, and g s constructed by one of the methods dscussed n Secton 4. It s easy to check that the decson rule n Secton 4 s equvalent to the followng: Choose C f d < 0, Choose ( l ) I( X C ), l I( X C ), l =, exp{ α ( d β )} C otherwse, l (6) (7) (9) where C C represents a competng class for class, whch s C for C. If a classfer s assumed to be perfect, for C or l becomes close to zero when becomes close to one. l X C ; otherwse, In sum, a DER s now approxmated wth a contnuous and dfferentable functon as a result. The approxmated functon now becomes our obectve functon. Summng over all classes, the overall obectve functon L( T, Λ) can be wrtten as follows: L( T, Λ) = 2N ( l (, )) ( ) ( (, )) ( ) X Λ I X C l X Λ I X C, T C C X The smlar approaches can be appled for any other performance metrcs such as precson, recall, or F to the correspondng construct obectve functons. In our framework, a DER s the preferred metrc. Fnally, the maxmzaton of the obectve functon L ( T, Λ) can be done by a generalzed probablstc decent algorthm. One of the mportant propertes of MFoM-based learnng approaches s that a relatve dstance between class and ts competng classes wll be maxmzed when the performance metrc s optmzed. To see how ths dstance s maxmzed, let us look at the class msclassfcaton functon d for class. It s easy to see that the absolute value of s the separaton between class and ts competng class. Snce a bgger separaton mples a smaller error, the value of d wll be ncreasng as the obectve functon s maxmzed. Ths mght mply the robustness of an MFoM-based classfer learnng approach. 6. EXPERIMENTAL RESULTS We extracted 90,70 mage fles from the SpamArchve spam corpora and used them as our spam mages. Unfortunately, most of the extracted fles were malware propagaton vehcles (.e., malware embedded n a fle wth an mage fle suffx). Many of the fles also contaned formattng errors. After elmnatng these malformed fles and repeated mages, the data set contaned 669 dstnct mages. We wll refer to these mages as our SpamArchve mage data set. For legtmate mages, we obtaned 267 mages from Corel CDs and 358 mages from Google Image Search. We wll refer to these mages as the CG mage data set. In Secton 2, spam mages are characterzed wth 5 sub-classes, and legtmate mages are characterzed wth 3 sub-classes. Through several prelmnary experments, we decded to ntegrate three sub-classes ( Non_Text, Non_Synthetc_Other, and Non_Synthetc_Sexual ) nto one sub-class ( Others ). Then, we were left wth a total of 6 sub-classes: 3 for spam mages and 3 for legtmate mages. The followng s a summary of the resultng mult-class characterzaton used and the numbers of mages that belong to each sub-class: d a) The SpamArchve mage data set Text_Smple : 360 mages, Text_Complex : 4 mages, Others : 68 mages (0)

7 b) The CG mage data set Photos : 404 mages, Cartoons : 5 mages, Maps : 05 mages Next, we determned the parameters for feature extracton. For the color heterogenety feature, we defne the number of colors used for quantzaton to be 8. The number of centrods s also set to 8 for the conspcuousness feature. When extractng the selfsmlarty feature, we set the number of patterns to be equal to 64. Feature fuson s done by concatenatng four features drectly, whch yelds a 72-dmensonal feature vector. To decde the best feature fuson method, we carred out classfcaton experments wth two other feature fuson methods, whch are an MBT approach [23] and a weghted sum approach [22], and compared the results. Among the three feature fuson methods, concatenatng four features outperforms the other two methods consstently. For an MFoM-based learnng approach, a lnear dscrmnant functon s adopted for a sub-class scorng functons X, Λ ) g ( and X, Λ ). A lnear dscrmnant functon s defned as: g ( where w and b are parameters. In our framework, therefore, w and b for the th sub-class of spam mages and w and b for the th T g( X, Λ ) = w X b, () sub-class of legtmate mages are gong to be estmated. α and β are determned emprcally n a way that for ntal teraton, the obectve functon fluctuates and then converges subtly. For dfferent decson rules, dfferent α and β should be used. 6. Comparson of Decson Rules To determne the best decson rule for spam mage dentfcaton wth mult-class characterzaton, we evaluated the three decson rules specfed n Secton 4 usng the SpamArchve and CG datasets. Frst, the class msclassfcaton functon s derved from each of the decson rules, and each MFoM-based classfer s traned accordngly. 0% of the mages are randomly chosen for testng, and the rest of the mages are used n the tranng stage. We repeat ths random selecton 0 tmes and average all of the results. Table 2 gves the results for the frst decson rule selectng the maxmum values (D), the second decson rule arthmetc average (D2), and the thrd decson rule geometrc average (D3). Table 2. Comparson of Decson Rules Identfcaton Rate(%) d Postve(%) D D D Ths table shows that the frst decson rule (D) acheves the best for dentfyng spam mages, and the thrd decson rule (D3) has the lowest false postve rate. The second decson rule (D2) performs worst, compared to the other two rules. For overall performance, the frst decson rule appears to be the best for spam mage dentfcaton wth mult-class characterzaton. 6.2 Mult-Class vs. Sngle-Class Characterzaton In ths secton, we compare mult-class characterzaton approaches and sngle-class characterzaton approaches. For a sngle-class characterzaton approach, the decson rule defned n Eq. () s adopted. For the class dscrmnant functons g ( X, Λ ) and g ( X, Λ ), a lnear dscrmnant functon, specfed as n Eq. (), s used, and an MFoM-based classfer s traned. For a mult-class characterzaton approach, the results are borrowed from Secton 6., whch were obtaned usng the frst decson rule (D). Table 3 shows the comparson between sngleclass characterzaton and mult-class characterzaton. Table 3. Mult-Class vs. Sngle-Class Identfcaton Rate(%) Postve(%) Sngle-Class Mult-Class Ths table shows that a mult-class characterzaton approach works better than a sngle-class characterzaton. The mprovement n false postve rate s qute sgnfcant, and the spam mage dentfcaton rate mprovement s also notceable. Therefore, as clamed, mult-class characterzaton s more effectve and sutable for spam mage dentfcaton. Addtonally, snce low false postve rates are typcally more mportant than hgh spam dentfcaton rates n spam mage dentfcaton, we can acheve another preferred characterstc through mult-class characterzaton. 6.3 Comparson wth Other Technques It s mpossble to compare other prevously proposed technques [, 24] wth our proposed framework drectly. The man reason concerns dfferences n the datasets that were used. In [], legtmate mages were retreved from Google Image Search, and n [24], legtmate mages only conssted of mages selected from CorelCDs. However, t s possble to poston our proposed framework by analyzng those other technques results. Table 4 shows the performance of []. In that work, sngle-class characterzaton was used. Snce only par-wse performances were reported, we average all of ther performances and compute an overall performance. They also used two dstnct datasets, SPAM- and SPAM-2, for spam mages. Table 4. Identfcaton Result (Aradhey, et al., 2005) Dataset Identfcaton Rate(%) Postve(%) SPAM SPAM Average Table 5 gves the results n [24]. In that work, three classfcaton results, dependng on SVM parameters, were reported. Among

8 those results, we take a classfcaton result most wdely used n [24] Table 5. Spam Detecton Results (Wu, et al., 2005) Identfcaton Rate(%) Postve(%) Based on the fgures n the above two tables, we observe that our proposed framework generally obtans better results and sometmes comparable results. Snce we use multple sources to construct our datasets, whereas other technques reled on a sngle source such as ether CorelCDs or an mage search engne, we beleve that our tranng and test datasets are more realstc and more dffcult to model. Therefore, we beleve that our proposed framework works as well as (f not better than) prevous technques. 6.4 Generalzaton Capabltes In spam mage dentfcaton, generalzaton s rather mportant because the varaton of mages s mmense. We test generalzaton capabltes of our approaches wth completely dfferent datasets. In the tranng stage, the SpamArchve and CG mage data sets, totalng 246 mages, are used. In the testng stage, mages extracted from the TREC 2005 emal corpus are used.,249 spam mage fles and 288 legtmate mage fles were extracted and processed usng our normalzaton processes. Unfortunately, these data sets contaned overlappng mages (.e., mages found n both legtmate and spam emals), whch we were forced to remove. From our observatons, most overlappng mages were ether cons or background templates. 7 spam mages and 52 legtmate mages were retaned, and the results of our analyss are shown n Table 6. Table 6. Identfcaton Results on TREC 2005 Corpus Identfcaton Rate(%) Postve(%) Sngle-Class Mult-Class In Table 6, we observe that our false postve rate ncreased by 3.5%. One explanaton for ths ncrease s the dfference between CG mages and TREC 2005 legtmate mages. Ths result clearly reflects the need for a standardzed data set for legtmate emal mages. On the other hand, we observed mprovements n the spam mage dentfcaton rate, compared wth the results n Table 4. Ths proves the generalzaton capabltes of our proposed framework for dentfyng spam mages. We also observe that mult-class characterzaton acheves better than sngle-class characterzaton, even for the TREC 2005 emal corpus. postve rate s mproved by 5.9% and spam mage dentfcaton rate s ncreased by.8% as well. The mprovement n false postve rate s more emnent as observed n Secton 6.2. If we recall that the legtmate mages used for tranng are not from real emals, t can be clamed that a mult-class characterzaton approach s much advantageous for spam mage dentfcaton tasks snce t s very dffcult to obtan approprate tranng data. 7. CONCLUSION AND FUTURE WORK In ths paper, we propose a framework for spam mage dentfcaton that apples mult-class characterzaton and MFoMbased learnng. We dentfy four key propertes of spam mages: color moments, color heterogenety, conspcuousness, and selfsmlarty. By combnng mult-class characterzaton and these four propertes for feature extracton, expermental results show that our proposed technque s effectve. For our SpamArchve spam mage data set, we obtan a spam mage dentfcaton rate of 8.5% and a false postve rate of only 5.6% for the legtmate mages. The robustness of the technque s demonstrated by an experment usng the same classfer (wthout retranng) on the mages n the TREC 2005 emal corpus. Our approach acheves 86.6% spam mage dentfcaton wth 9.% false postves. The hgher false postve rate s due to the dfferences between our legtmate corpus used for classfer tranng and the TREC 2005 data set. However, wth a comparson between sngle-class characterzaton and mult-class characterzaton for the TREC 2005 emal corpus, we acheve a much better false postve rate when a mult-class characterzaton approach s used. Our results are consstent wth the hypothess that spam mages are fundamentally dfferent from normal mages, and therefore, we can apply mage analyss technques to dstngush them. Ths s an nterestng and encouragng result snce early attempts to apply OCR algorthms to convert mages to text have been defeated by spammers applyng smple CAPTCHA technques. Our approach s a component technology that can be combned wth text-based flters to mprove the recall and precson of flterng text-and-multmeda spam messages. Beyond emal, our technques are also applcable n other spam applcaton areas such as web spam, blog spam, and nstant messagng spam, where non-text (multmeda) data has been growng n volume and mportance. We are currently workng on refnements for our mult-class MFoM-based characterzaton of spam mages. For example, addtonal propertes and a more refned class categorzaton may mprove the precson and recall of the method. On the applcaton sde, ncorporatng our approach wth text-based learnng flters for emal spam flterng s a natural next step. Other spam meda such as web spam are also nterestng but relatvely unexplored areas. Another area that wll mprove our research and the performance of our approach s the avalablty of large-scale data sets for both spam mages and legtmate mages. 8. ACKNOWLEDGMENTS Ths work has been supported by grants from the Ar Force Offce of Scentfc Research. The authors would lke to thank Flppo Vella who ntally bult up our mage processng systems. We also thank anonymous revewers for ther nsghtful feedback and comments. 9. REFERENCES [] Aradhey, H. B., et al., Image Analyss for Effcent Categorzaton of Image-based Spam E-mal, Proc. of ICDAR, 2005.

9 [2] Carbonetto, P., Fretas, N., and Barnard, K., A Statstcal Model for General Contextual Obect Recognton, Proc. of ECCV, [3] Carreras, X. and Mrquez, L., Boostng Trees for Ant-Spam Emal Flterng, Proc. of RANLP-0, 200. [4] Chen, Y. and Wang, J., Image Categorzaton by Learnng and Reasonng wth Regons, The Journal of Machne Learnng Research, vol. 5, pp , [5] Drucker, H., Wu, D., and Vapnk, V. N. Support Vector Machnes for Spam Categorzaton, IEEE Trans. on Neural Networks, vol. 0, no. 5., 999. [6] Feld, D. J., Relatons between the statstcs of natural mages and the response propertes of cortcal cells, Journal of Optcal Socety of Amerca, vol.4, No. 2., 987 [7] Fumera, G., Plla, I., and Rol, F., Spam Flterng Based on The Analyss of Text Informaton Embedded Into Images, Journal of Machne Learnng Research, vol. 7, pp , [8] Gao, S., Wang, D. H., and Lee, C. H., Automatc Image Annotaton Through Mult-Topc Text Categorzaton, Proc. of ICASSP, vol. 2, pp , [9] Gao, S., Wu, W., Lee, C. H., and Chua T. S., A MFoM Learnng Approach to Robust Multclass Mult-Label Text Categorzaton, Proc. of ICML., 2004 [0] Gonzalez, R. C. and Woods, R. E., Dgtal Image Processng 2ed, Prentce Hall Press, pp. 295., [] Gonzalez, R. C., Woods, R. E., and Eddns, S. L., Dgtal Image Processng usng Matlab, Prentce Hall Press., [2] Hu, J. and Bagga, A., Categorzng Images n Web Documents, IEEE Multmeda, vol., no., pp , [3] Koves, P. D(999), Image Feature from Phase Congruency, Vdere: Journal of Computer Vson Research, The MIT Press, vol., no. 3., 999. [4] Koves, P. D., MATLAB and Octave Functons for Computer Vson and Image Processng, [5] Leavtt, N. (2007), Vendors Fght Spam s Sudden Rse, IEEE Computer, vol. 40, no. 3, pp 6-9., [6] Ru, Y., Huang, T. S., and Chang, S. F. (999), Image Retreval: Current Technques, Promsng Drectons and Open Issues, Journal of Vsual Communcatons and Image Representaton, vol. 0, no. 4, pp , 999. [7] Saham, M., Dumas, S., Heckerman, D., and Horvtz, E., A Bayesan Approach to Flterng Junk Emal, AAAI Workshop on Learnng for Text Categorzaton., 998. [8] Strcker, M. and Orengo, M., Smlarty of Color Images, Proc. of SPIE, pp , 995. [9] Swan, M. and Ballard, D., Color Indexng, Int. Journal of Computer Vson, vol., pp -32., 99. [20] Symantec Corp., The State of Spam, A Monthly Report January, [2] Szummer, M. and Pcard, R., Indoor-Outdoor Image Classfcaton, IEEE Workshop on CAIVD, pp. 42., 998. [22] Vella, F., et al., Informaton Fuson Technques for Automatc Image Annotaton, Proc. of VISAPP., [23] Wang, D. -H., et al., Dscrmnatve Fuson Approach for Automatc Image Annotaton, Proc. of MMSP., [24] Wu, C. -T., Cheng, K. -T., Zhu, Q., and Wu, Y. -L., Usng Vsual Features For Ant-Spam Flterng, Proc. of ICIP., 2005.