Shane Dixon, Xiao-Hua Yu

Size: px

Start display at page:

Download "Shane Dixon, Xiao-Hua Yu"

Audra Henry
6 years ago
Views:

1 Proceedgs of the 2010 IEEE Iteratoal Coferece o Iformato ad Automato Jue 20-23, Harb, Cha Boformatcs Data Mg Usg Artfcal Immue Systems ad Neural Networks Shae Dxo, Xao-Hua Yu Departmet of Electrcal Egeerg Calfora Polytechc State Uversty Sa Lus Obspo, CA 93407, USA Abstract - Boformatcs s a data-tesve feld of research I a egatve selecto algorthm, detectors are geerated ad developmet. The purpose of boformatcs data mg s radomly frst, the they are evolved (.e., elmated f they to dscover the relatoshps ad patters large databases to match ay self samples) to obta a set of traed mature provde useful formato for bomedcal aalyss ad dagoss. detectors. I the testg mode, each ukow data stace s I ths research, algorthms based o artfcal mmue preseted to the detector set ad classfed as ether self or systems (AIS) ad artfcal eural etworks (ANN) are employed for boformatcs data mg. Three dfferet varatos of the o-self. That s, f the ukow data stace matches ay real-valued egatve selecto algorthm ad a mult-layer detector the detector set, the t s classfed as o-self or feedforward eural etwork model are dscussed, tested ad a aomaly; whle o the other had, f the comg data compared va computer smulatos. It s show that the ANN stace s ot recogzed by ay detector, t s cosdered to model yelds the best overall result whle the AIS algorthm s be a member of the self set. advatageous whe oly the ormal (or self ) data s Dfferet egatve selecto algorthms ca be avalable. characterzed, or dstgushed by partcular data represetato schemes, matchg rules ad detector Idex Terms - Artfcal mmue systems, Real-valued egatve geerato processes. It s kow that egatve selecto selecto algorthm, Data mg, Artfcal eural etworks. algorthms are ofte employed to classfy data; therefore, they are defed frst ad foremost by dfferet data represetato I. INTRODUCTION schemes. The early mplemetatos of egatve selecto algorthms ca oly classfy bary data. Later o, t was Boformatcs s a fast-growg, data-tesve feld that exteded to hadle data strg represetato (characters). volves the applcatos of formato techology to The focus of ths study cocers real-valued data molecular bology. The purpose of boformatcs data mg represetato, a more recet topc of research. s to dscover the relatoshps ad patters large The detector geerato ad elmato mechasms boformatcs databases to provde useful formato for mplemeted a egatve selecto algorthm are also bomedcal aalyss ad dagoss. To accomplsh ths task, mportat characterstcs of the algorthm. To date, oly may approaches have bee proposed, cludg algorthms radom-based geerato schemes have bee mplemeted for based o artfcal mmue systems ad artfcal eural real-valued vector data represetato. Other approaches to etworks. detector geerato may clude geetc algorthms ad The bologcal mmue system s a complex adaptve optmzato wth aftermath adjustmet ([2] [3] [4]). system of cells, molecules, ad orgas that ca recogze The cetral mechasm of a egatve selecto algorthm foreg substaces ad the eutralze or degrade them, wth s the selecto of a approprate matchg rule, or dstace or wthout jury to ts ow tssues. Over years, the mmue measure the case of real-valued data. The matchg rule s a system has evolved sophstcated patter recogto ad measure of affty or smlarty that two data staces share, respose mechasms usg ts etwork of chemcal ad s geerally applcato specfc ad data represetatoal messegers for commucato. Through a evolutoary depedet. learg process, the mmue system ca recogze a almost I addto to data represetatos, detector geerato lmtless varety of fectous foreg cells ad substaces processes ad matchg rules, there are also a umber of other (kow as o-self elemets), ad dstgush them from factors that affect the performaces of egatve selecto those atve ofectous cells (kow as self elemets). algorthms. For example, the umber of detectors affects the The egatve selecto algorthm (NSA) was frst effcecy of geerato ad detecto, ad cosequetly the troduced by Stephae Forrest 1994 [1]. It s a speed of the algorthm. Lked drectly to the accuracy of computatoal model based o the self/o-self dscrmato detecto, detector coverage s also a mportat factor to process performed by the T-cells atural mmue systems. cosder durg detector geerato. The stoppg crtera are Recetly, NSA has attracted the atteto of may ofte used to determe a adequate umber of detectors ad computatoal tellgece researchers ad has bee ther coverage. successfully appled to solve may egeerg problems I ths research, algorthms based o artfcal mmue recet years, such as computer etwork securty aalyss, fault systems (AIS) ad artfcal eural etworks (ANN) are detecto, ad data mg /10/$ IEEE 440

2 employed for boformatcs data mg. Three dfferet The 3-orm Mkowsk dstace metrc s smlar to the varatos of the real-valued egatve selecto algorthm (.e., Eucldea dstace, except the dfferece s cubed ad the the detectors wth fxed radus, the V-detector wth varable summato s cube-rooted: radus, ad the prolferatg V-detectors) wth fve dfferet 1 performace metrcs (.e., the Eucldea dstace, the 3 x Mahatta dstace, the 3-orm Mkowsk dstace, the y (4) =1 partal Eucldea dstace, as well as the Chebyshev dstace) ¹ are studed. As a comparso, a mult-layer feedforward eural The Chebyshev dstace s also called the fty-orm etwork model s also developed ad tested. Mkowsk dstace: II. THE REAL-VALUED NEGATIVE SELECTION ALGORITHMS The real-valued egatve selecto algorthm (RNSA) was frst proposed 2002 [5]. I ths algorthm, data (cludg both trag ad testg data), detectors, affty (or performace metrcs), ad the matchg threshold are both represeted by real-valued data a -dmesoal real vector space. The most commoly used dstace metrc RNSA s the Eucldea dstace, but may others exst. I fact, the selecto of a approprate dstace measure s crucal to the overall performace of the algorthm, due to the fact that the etre process of a egatve selecto algorthm s bult upo the cocept of affty or dstace. I the detector geerato process, the umber of detectors ad the estmato of detector coverage are both related wth the dstace metrc. Durg the detecto phase, the decso rule to classfy the ukow comg data as ether self or o-self s also based o the dstace measure. Oe uque feature of the dstace metrc chose for a real-valued egatve selecto algorthm s the mpact t has o the shape of the detectors. The detectors are assged a realvalued threshold self/o-self dscrmato, whch ca be evsoed as a radus of detecto. If the dstace betwee the detector ad a data pot s less tha ths threshold, the ths sample s detected by the detector ad thus classfed as a member of the o-self set. Cosder a pot (x 1, x 2, x 3,, x ) ad aother pot (y 1, y 2, y 3,, y ) -dmesoal real vector space. The Mkowsk dstace, or the m-orm dstace betwee two pots, s defed as ([2] [6]): 1 m m y (1) ¹ 1 m m lm x y = max( x y ) (5) m =1 ¹ where = 1, 2,,. Note that by takg the lmt as m approaches fte, t yelds the maxmum dstace betwee two pots; ad thus ofte smply referred as the maxmum dstace metrc. A partal Eucldea dstace, or the Eucldea dstace wth a sldg wdow s also employed ths research (smply referred as the wdow dstace metrc secto 4). Let s cosder a example of two arbtrary pots a fourdmesoal real space,.e., (x 1, x 2, x 3, x 4 ) ad (y 1, y 2, y 3, y 4 ), ad assume the sldg wdow has a fxed sze of 2. Frst, the Eucldea dstace s calculated, but oly for (x 1, x 2 ) ad (y 1, y 2 ). Next, the wdow of observato shfts, or sldes to (x 2, x 3 ) ad (y 2, y 3 ), ad the fally coclude wth (x 3, x 4 ) ad (y 3, y 4 ). Of the three separate dstaces calculated, oly the oe wth the smallest absolute value s retaed (ad others are dscarded). The partal Eucldea dstace determes the smallest dstace a lower-dmesoal space ( ths case, the dmeso s 2) for the data a hgher-dmesoal space ( ths case, the dmeso s 4) The real-valued egatve selecto algorthm wth fxed detector radus I the real-valued egatve selecto algorthm wth fxed detector radus proposed by Gozalez ad Dasgupta [5], the detector geerato phase begs by radomly geeratg a preset umber of pots -dmesoal real space [0, 1], wth a mea value of ½ (for smplcty, t s assumed that the put data s also ormalzed wth [0, 1] ). I other words, each detector ca be evsoed as a hypersphere wth a ceter ad fxed radus r a -dmesoal space. The detectors are x the traed wth oly self samples; that s, the postos of the =1 detectors are updated through a teratve process. The where m s also called the order of the Mkowsk dstace. detectors must rema away from the self pots ad also The commoly used Eucldea dstace ca be cosdered rema separated from other detectors order to maxmze as a specal case of the Mkowsk dstace of order 2 (or 2 the o-self space coverg. The ew locato of the detector orm): s determed by: 2 (x y ) (2) d(+1) = d() + *dr (6) =1 where d() s the curret posto (ceter) of the detector, The 1-orm dstace s called the Mahatta dstace d(+1) s the ew posto of the detector, s the adaptato metrc, ad s smply the absolute value of the dfferece rate, s the age of the detector, ad dr s the drecto of betwee the two pots -dmesoal space: movg. Sce t s udesrable for the detectors to match self pots, the shortest allowable dstace for a good detector to x y (3) the self set s r (t s also referred as the threshold for =1 matchg). 441

3 The adaptato rate ca be updated as: η =η e τ 0 (7) where the preset adaptato rate parameter o represets the tal step sze used to move the detectors. I order to guaratee that the algorthm wll coverge to a stable state, t s ecessary to decrease ths parameter each terato such a way that lmη = 0 (8) The drecto of movg dr ca be calculated based o the shortest calculated dstace to ay self pot or detector: dr = earest (d c ) =1 earest (d c ) = The V-detector algorthm I [3], Zhou ad Dasgupta proposed a dfferet scheme of detector geerato ad matchg mechasms for egatve selecto algorthms. Ths algorthm (called the V-detector algorthm) cludes a ew varable parameter, whch s the radus of each detector. The geerato phase of the V-detector algorthm begs by radomly geeratg detector caddates; but stead of geeratg a full set of detectors, t geerates detector caddates oe at a tme. Each dvdual caddate s checked usg the matchg rule determed by the choce of dstace metrc. If the dstace to the earest self pot s less tha the threshold value (whch s the radus of ths earest self pot r s ), the detector s elmated ad a ew caddate s geerated. If the mmum dstace to ay self pot s greater tha the radus of ths self pot r s, the the detector s stored temporarly ad ts radus s recorded as r d, whch s the mmum dstace to the earest self pot: (9) r d = dst_m (10) Ths s kow as the aggressve approach to assg a detector s radus [6]. Detectors are teratvely geerated ad assged a radus based o ths mechasm utl the stoppg crtera s met. A more coservatve approach to detector radus assgmet ca also be mplemeted, whch the detector radus r d s determed by the dfferece betwee the mmum dstace to the earest self pot ad the self radus r s of the earest self pot [3]: r d = dst_m - r s (11) I ths research, both mplemetatos are tally tested ad compared. The aggressve strategy produces more accurate results, ad cosequetly was the method chose for ths study The prolferatg V-detector algorthm Oe of the most recet advaces real-valued egatve selecto algorthms corporates the mplemetato of prolferatg varable-szed detectors [7]. Durg the geerato phase, the detector set s flled wth a tal set of detectors the same maer as the geerato phase for the V-detector algorthm; the oly dfferece s the assgmet of the varable radus r d. The prolferatg V-detector algorthm cludes a addtoal threshold term whch s also subtracted from the varable radus r d. Therefore, for the aggressve varable radus approach: r d = (dst_m - ) (12) whle the coservatve varable radus approach: r d = (dst_m - r s - ) (13) I ths study, the aggressve approach s chose for computer smulatos. After the geerato phase cocludes, the prolferato stage begs to prolferate (or cloe) ew detectors from the detector set tally created from the geerato stage. These ew detectors are referred to as offsprg. At the begg of the prolferato stage, the algorthm already has a set of detectors D from the prevous geerato stage. I the -th terato, t selects oe of those detectors whose ceter ad radus are x ad r from the set D, ad creates ew offsprg located at a dstace r from x. I two dmesoal vector space, the orgal detector s regarded as a crcle of radus r the o-self rego cetered aroud x, ad the offsprg detectors wll be located alog the crcle s crcumferece at a locato x + ûr, where û s some ut drecto vector [7]. The offsprg s radus s set to be equal to the mmum dstace from ts ceter to the earest self pot, but ca also be modfed to clude the addtoal threshold, as the prevous dscussos. Offsprg coverage s cotrolled the same maer as the detector geerato phase of the V-detector algorthm. Sce a ew detector has addtoal coverage value oly whe aother does ot already cover the space, oly those offsprg detectors whch are ot covered wll be retaed for the detecto phase. The detectors D are selected for prolferato a sequetal maer, wth the ut vectors û are kept to be ether parallel (+1) or at-parallel (-1) to each dmeso. The prolferato stage may ot oly volve oe stage of prolferato. Several stages of prolferato, where the offsprg from oe stage s allowed to prolferate the ext stage, are ofte desrable. Matag the threshold tally hgh durg the frst the frst geerato stage, ad lowerg t towards zero a stepwse maer durg subsequet prolferato stages, ca result much better coverage of the o-self subspace. Ths s because decremetg the threshold at the ed of each stage creates a gap betwee the self/oself boudares. Ths gap ca the be flled by the offsprg detectors of the ext prolferato stage. Steadly decreasg the gap by lowerg wll result creasgly smaller, but strategcally placed offsprg to prolferate aroud the 442

4 self/o-self boudary rego. To esure full coverage of the o-self subspace, the threshold must be set to zero durg the last stage of prolferato [3]. abormal samples are just a small porto of the etre dataset. Uder ths crcumstace, the egatve selecto algorthms may outperform the eural etwork model. III. THE NEURAL NETWORK MODEL I ths secto, a mult-layer feedforward artfcal eural etwork (ANN) model for boformatcs data classfcato s dscussed. It s well kow that ANN ca lear the putoutput mappg of a system through a teratve trag ad learg process, ad thus s a deal caddate for patter recogto ad data aalyss. The ANN model has a put layer, a output layer, ad oe or more hdde layer(s). There are puts for the - dmesoal put data; ad a output whch dcates the class of put vector ( self or o-self ). That s, the eural etwork model s a mult-put, sgle-output system. The actvato fucto for each hdde euro s chose as the sgmod fucto: 1 f ( x ) = (14) x 1+ e The weghts of the eural etwork are talzed at radom, ad the updated usg the back-propagato algorthm to mmze the followg objectve fucto: J (k) = 1 [d(k) y(k)] 2 (15) 2 where d s the desred output (class) ad y s the output of eural etwork, k s the dex of a trag par. W( k +1) = W( k ) + ΔW (16) where J ΔW = μ (17) W where μ s the learg rate. O-le learg approach employed ths study. A put sample patter s fed to the etwork, resultg a error sgal at the output. The error sgal s the back propagated through the etwork order to adjust the syaptc weghts of each euro. The above procedure repeats utl all put samples wth the trag set have bee exhausted. The order of the trag samples s the radomly rearraged ad aother trag pass s coducted, utl the maxmum umber of teratos reached or the error sgal s reduced to a acceptable level. To rema cosstet wth the egatve selecto algorthm, the output of eural etwork s also bouded betwee [0, 1], wth a decso threshold of 0.5. That s, the put data s classfed ether as 1 (self) f y 0.5, or 0 (o-self) f y < 0.5. There s a major dstcto betwee the egatve selecto ad eural etwork algorthm whch must be addressed at ths pot. Whle a egatve selecto algorthm, by desg, requres trag of oly oe class of data, the eural etwork algorthm must be traed wth samples from both classes of data. I boformatcs data mg, t s very commo that oe class of data s domat over the other class. For example, a database may cota large amout of testg results from the ormal populato whle the IV. SIMULATION RESULTS I ths secto, three dfferet real-valued egatve selecto algorthms ad a mult-layer feedforward eural etwork model are tested ad compared va computer smulatos. The data used ths research s from the bomedcal dataset" reported by Larry Cox 1982 the CMU StatLb datasets archve [8]. I a study to develop screeg methods to detfy the carrers of a rare geetc dsorder dsease, four measuremets ( m 1, m 2, m 3, ad m 4 ) were take from huma blood samples. The data cotas 209 observatos, wth 134 samples are cosdered to be ormal (or free of dsorders) ad 75 samples are detfed as the carrers of the dsorder. Two performace metrcs are employed to evaluate the effectveess of each algorthm,.e., the detecto rate ad false alarm rate [9]. The detecto rate (DR) s defed as the umber of correctly detfed o-self samples dvded by the total umber of o-self samples. Ths yelds a percetage of correctly detfed o-self pots, sgfyg how well the algorthm detected aomales. Coversely, the false alarm rate (FA) s calculated as the umber of self samples classfed correctly dvded by the total umber of self samples. Ths produces a percetage of self samples classfed correctly, sgfyg how poorly the algorthm msclassfes self data as a aomaly. A fgure of mert (FOM) s proposed by authors to determe a overall fal score for the performace of the algorthm, whch s defed as the dfferece betwee the false alarm rate ad the detecto rate (DR-FA). It s a method of comparg how well the algorthm detects aomales whle smultaeously pealzg t for self msclassfcatos. The eural etwork model s tested for two cases. The frst s the case whch the etwork s traed wth the same set of data as the egatve selecto algorthm (.e., oly self data s cluded), whle the latter case the etwork s traed wth mxed samples cludg both self ad o-self data. The results from trag wth oly self data clearly demostrate that the eural etwork fals to classfy put data. Sce the etwork s oly traed wth self data, the desred output for all trag data s always the same (.e., 1 ). That mples that the etwork s bascally traed to oly output a 1 ; ad thus ay ew ukow data s always classfed as a 1. I other words, the detecto rate s a costat zero,.e., all oself data s cosstetly classfed as self. I the secod case, the eural etwork model s traed wth 97 radomly selected samples (wth 64 of them beg ormal ad 33 of them beg carrer ), ad the tested wth the rest of samples the database. The FOM of eural etwork for the testg data s 46.99%. The computer smulato results of dfferet real-valued 443

5 egatve selecto algorthm are summarzed tables 1, 2, ad 3, where FD represets the algorthm wth fxed-radus detectors, VD represets the algorthm wth varable-radus detectors, ad PVD s for the algorthm that s wth the prolferatg detectors; Eucldea, Mahatta, Wdow, 3-Norm, ad Max dcate the dfferet dstace metrcs used for smulato, as dscussed secto 2. The last row, Average s a arthmetc average value of the performace of each algorthm wth dfferet metrcs. TABLE I The Detecto Rate (%) of Each Algorthm FD VD PVD Eucldea Mahatta Wdow Norm Max Average TABLE II The False Alarm Rate (%) of Each Algorthm FD VD PVD Eucldea Mahatta Wdow Norm Max Average TABLE III The Fgure of Mert (%) of Each Algorthm FD VD PVD Eucldea Mahatta Wdow Norm Max Average From the smulato data preseted the tables, we ca coclude that the prolferatg detectors algorthm performs the best amog the three dfferet AIS algorthms, wth the hghest average detecto rate (51.65%) ad the average FOM dex (39.93%). The varable-radus detectors algorthm yelds the lowest average detecto rate (37.05%), average false alarm rate (6.94%), as well as the average FOM (30.23%). Ths may be due to the fact that the parameters used the algorthm are ot well-selected. O the other had, t s also observed that ths algorthm requres the shortest smulato ru-tme, ad thus may be selected for certa applcatos whe quck solutos are eeded real-tme. If tme costrat s ot a cocer, the the prolferatg V-detector algorthm ca be the better choce. The performace of each AIS algorthm also dffers f dfferet dstace metrcs are used. For example, by choosg a approprate dstace measure (.e., wdow ), the fxedradus detectors algorthm ca yeld satsfactory performace (wth a FOM of 59.01%). I fact, both the fxed-radus detectors ad varable-radus detectors perform sgfcatly better (wth hgher detecto rate ad overall FOM) f the wdow metrc s chose; whle for the prolferatg V- detector algorthm, the Mahatta s deftely a better choce. V. CONCLUSIONS I ths research, algorthms based o artfcal mmue systems (AIS) ad artfcal eural etworks (ANN) are employed for boformatcs data mg. Three dfferet varatos of the real-valued egatve selecto algorthm ad a feedforward eural etwork model are tested ad compared va computer smulatos. Though the eural etwork model yelds the best overall result (wth a FOM of 46.99%), the AIS algorthm s advatageous whe oly the ormal (or self ) data s avalable whle the eural etwork has to be traed usg data of both self ad o-self sets. The accuracy of a AIS algorthm may be further mproved by optmzg (.e., fe-tug) the values of some of the parameters used the algorthm. More testg wll be coducted the future. ACKNOWLEDGMENT Ths work was supported part by the Leoard Trasportato Ceter, uder Award # GT REFERENCES [1] Forrest, S., Perelso, A., Alle, L., R., Self-oself dscrmato a computer, I Proceedgs of the 1994 IEEE Symposum o Research Securty ad Prvacy, IEEE Computer Socety Press, Los Alamtos, CA pp [2] Zhou, J., Negatve Selecto Algorthms: from the Thymus to V- detector, Ph. D. dssertato, Uversty of Memphs, Memphs, TN, USA, August [3] Zhou, J., Dasgupta, D., Revstg Negatve Selecto Algorthms, Issue Evolutoary Computato Joural. July, [4] Gozalez, F., Dasgupta, D., No, L. F., A Radomzed Real-Valued Negatve Selecto Algorthm, Secod Iteratoal Coferece o Artfcal Immue Systems. Uted Kgdom. September, [5] Gozalez, F., Dasgupta, D., Kozma, R., Combg Negatve Selecto ad Classfcato Techques for Aomaly Detecto, Volume 1, Cogress o Evolutoary Computato. Hoolulu, Hawa. May, pp [6] Zhou, J., Dasgupta, D., Applcablty Issues of the Real-Valued Negatve Selecto Algorthms, Geetc ad Evolutoary Computato Coferece (GECCO). Seattle, Washgto. July, [7] Das, S., Gu, M., Pahwa, A., Artfcal Immue Systems for Self- Noself Dscrmato: Applcato to Aomaly Detecto, Studes Computatoal Itellgece (SCI) 116, pp

6 [8] Statlb datasets archve, World Wde Web, [9] Gozalez, F., Dasgupta, D., Aomaly Detecto Usg Real-Valued Negatve Selecto, Joural of Geetc Programmg ad Evolvable Maches. Volume 4, Issue 4. December, pp [10] Hayk, S., Neural Networks: A Comprehesve Foudato, Pretce Hall, [11] Neural Network Toolbox User s Gude, The Mathworks. 445

K-Map 1. In contrast, Karnaugh map (K-map) method provides a straightforward procedure for simplifying Boolean functions.

K-Map 1. In contrast, Karnaugh map (K-map) method provides a straightforward procedure for simplifying Boolean functions. K-Map Lesso Objectves: Eve though Boolea expressos ca be smplfed by algebrac mapulato, such a approach lacks clear regular rules for each succeedg step ad t s dffcult to determe whether the smplest expresso