Intelligent and Robust Genetic Algorithm Based Classifier

Transcription

1 Intellgent and Robust Genetc Algorthm Based Classfer S. H. Zahr, H. Raab Mashhad and S. A. Seyedn Downloaded from eee.ust.ac.r at :4 IRDT on Monday September 3rd 018 Abstract: The concepts of robust classfcaton and ntellgently controllng the search process of genetc algorthm (GA) are ntroduced and ntegrated wth a conventonal genetc classfer for development of a new verson of t, whch s called Intellgent and Robust GA-classfer (IRGA-classfer). It can effcently approxmate the decson hyperplanes n the feature space. It s shown expermentally that the proposed IRGA-classfer has removed two mportant weak ponts of the conventonal GA-classfers. These problems are the large number of tranng ponts and the large number of teratons to acheve a comparable performance wth the Bayes classfer, whch s an optmal conventonal classfer. Three examples have been chosen to compare the performance of desgned IRGA-classfer to conventonal GA-classfer and Bayes classfer. They are the Irs data classfcaton, the Wne data classfcaton, and radar targets classfcaton from backscattered sgnals. The results show clearly a consderable mprovement for the performance of IRGA-classfer compared wth a conventonal GA-classfer. Keywords: Intellgent genetc classfers, robust genetc classfers, fuzzy controller, genetc algorthm, optmum decson hyperplanes. 1 Introducton 1 Genetc algorthms (GAs) have been shown to be an effectve stochastc search algorthm n hgh dmensonal spaces. They are nspred by the bologcal process of Darwn s evoluton theory, where selecton, mutaton and crossover play mportant roles [1]. GAs have been appled to solve pattern recognton and data classfcaton problems by fndng decson boundares and hyperplanes. Ths new evolutonary classfer s called GA-classfer []. It s shown theoretcally and expermentally that the performance of a GA-classfer for suffcently large number of teratons and nfntely number of tranng data ponts s comparable to Bayes classfer whch s the optmal classfer [3]. It s mportant to menton that the optmal Bayesan classfer needs a pror knowledge but GA-classfer doesn't need any mportant pror knowledge. Iranan Journal of Electrcal & Electronc Engneerng, 005. Paper frst receved 17th Aprl 005 and n revsed from 11th March 006. S. H. Zahr s wth the Department of Electroncs and Communcaton, Brand Unversty, Brand, Iran. P.O. Box: H. Raab Mashhad and S. A. Seyedn are wth the Department of Electrcal Engneerng Ferdows Unversty of Mashhad, Mashhad, Iran. E-mal: hzahr@brand.ac.r, raab@ferdows.um.ac.r, seyedn@ferdows.um.ac.r. Also a varable strng length GA-classfer (VGAclassfer) proposed evolvng the number of hyperplanes automatcally [4] and another VGA-classfer wth chromosome dfferentaton (VGACD-classfer) desgned for pxel classfcaton n [5]. The ftness functons, defned n all of these researches are the number of msclassfed tranng ponts. Although the desgned GA-classfers may classfy the tranng ponts as well as, or better than other conventonal classfers, e.g. mult-layer-perceptron (MLP), k-nearest neghbor and Bayes classfer, but ts performance has not ths strength aganst the test ponts. A multobectve GA has been recently ntroduced n [6] for smultaneously optmzaton three obectves, whch are number of msclassfed ponts, class-accuracy and the number of hyperplanes. As mentoned above, the better performance of GAclassfer, for all of these researches, happens for a large number of tranng ponts and a large number of teratons. In fact these are two condtons, whch are necessary for conventonal GA-classfer to reach a comparable performance wth Bayes classfer as an optmal classfer [-6]. Obvously, f the number of tranng ponts s suffcently large, the probablty dstrbuton functons (PDF) can be estmated and wth known PDFs, Bayes classfer s the best canddate to fnd the decson functons n feature space, because of ts smple Iranan Journal of Electrcal & Electronc Engneerng, Vol. 1, No. 3, July 005 1

2 structure and the best performance and t s not reasonable to use the GA-classfer wth low convergence rate. In ths paper a novel approach s proposed to remove these two weak ponts. The basc dea s to maxmze the margns of hyperplanes from the dfferent classes usng a proper defnton of the ftness functon. It has been shown mathematcally n [7] that maxmzng the margns of hyperplanes from dfferent classes can mnmze the rsk of error n the classfcaton. To show the stablty of the method aganst the varaton of the number of tranng ponts (n) a new ndex, named robustness ndex, s defned as a metrc. A GA-classfer wth a hgh value of robustness aganst the value of n, named n ths artcle a Robust GA-classfer (RGAclassfer). Another mportant concept ntroduced n ths artcle s to steer the GA-classfer effcently to the global soluton whle genetc algorthm s runnng. For ths purpose, an ntellgent mutaton and crossover rate controller s desgned usng a fuzzy structure to develop an Intellgent and Robust GA-classfer (IRGAclassfer). Ths ntellgent fuzzy controller, not only can chase away the genetc algorthm from the local solutons, but also can reduce the necessary number of teratons consderably. Thus a common problem of conventonal GA-classfers n prevous researches,.e. poor convergence of the search process due to the large number of teratons, s mproved. The rules for desgnng the fuzzy controller were extracted from some theoretcal and expermental results have reported n researches on GA operators [8-1]. We used Fuzzy controlled and Robust GA-classfer (FCRGA-classfer), and a Smple GA-classfer (SGAclassfer) for determnng the hyperplanes for two common benchmark problems and a specal problem n pattern recognton. Irs data and Wne data classfcaton are common problems n pattern recognton researches wth low and medum feature space dmensons, and automatc target recognton n contnuous wave radars s a specal pattern recognton problem wth hgh feature space dmensons. We compared the scores of recognton and the number of teratons s needed for convergence for FCRGAclassfer and SGA-classfer. To see the robustness of desgned IRGA-classfer, we also compared ts performance wth the Bayes classfer for dfferent tranng ponts, because t s optmal classfer when the probablty densty functon of features s known. The results show that FCRGA-classfer has more accuracy compared wth a SGA-classfer wth a less number of teratons and hgh robustness value. Also the performance of ths IRGA-classfer s comparable to Bayes classfer for a low number of tranng ponts. In ths paper, Secton explans the structure of a realvalued genetc algorthm based classfer (GAclassfer). Intellgent robust genetc classfers are then descrbed n Secton 3. Secton 4 consders expermental results on three pattern recognton problems, whch are Irs data classfcaton, Wne data classfcaton and radar target classfcaton. Fnally, Secton 5 concludes the paper. Structure of a real- valued GA-classfer A general hyperplane s n the form d (X) w1 x1 + w x w n x n + w n+ 1 = (1) Where X= ( x ) ' 1,x,...,x n, 1 and W= ( w ) ' 1,w,...,w n, w n+ 1 are called the augmented feature (pattern) and weght vectors respectvely. In a general case, there are a number of hyperplanes that separate the feature space to dfferent regons, whch each regon dstngushes an ndvdual class (Fg. 1). In Fg. 1, IR denotes the ndetermnate regon. Fg. 1 A general case, whch each regon can dentfy an ndvdual class by ts code obtaned from the sgn of hyperplanes. Some especal cases are descrbed n the text books of Pattern Recognton course [e.g. 13]. A real-valued GA classfer should fnd W (=1,,,M) n soluton space. We consder real-valued GA-classfer because bnary coded GAs are less effcent when appled to mult dmensonal problems. The bt-strngs can become very long and the search space blows up. In real-valued GAs, the varables appear drectly n the chromosome and are modfed by mutaton and recombnaton (crossover) operators. Varous real-valued-ga were revewed n [14].The basc steps for desgnng a real-valued GA-classfer are as follows: 1-Generaton At the frst step a random ntal populaton, S(0) s generated. S(0) ncludes N chromosomes and the th chromosome s of the form [ W 1,W,..., WM ] ' for M classes. Iranan Journal of Electrcal & Electronc Engneerng, Vol. 1, No. 3, July 005

3 -Ftness evaluaton At the second step the value of ftness functon of S(0) s computed. In a smple GA-classfer, whch ntroduced n prevous researches the ftness functon s defned as the number of msclassfed tranng ponts [-6]. 3-Basc loop The body of a GA s: Whle (Termnaton Condton) Loop : Compute ftness(s()); =+1; S()=Select (S(-1)); S()=Crossover S(); S()=Mutate (S()); End Loop; End Whle; The termnaton condton can be mplemented usng the best ftness value or a default maxmum number of teratons. In th teraton the ftness value s computed and then a new populaton s generated by three mportant genetc operators: selecton, crossover (recombnaton), and mutaton. These operators descrbed as follows: -Selecton Selecton s the process of determnng the number of tmes or trals that a partcular ndvdual s chosen for reproducton and, thus the number of offsprngs that an ndvdual wll produce. Many selecton technues employ roulette wheel mechansm to probablstcally select ndvduals proportonal to ther ftness value. Also the best ndvduals are transmtted to the next generaton wthout any addtonal process (eltsm strategy). -Crossover The basc operator for producng new chromosomes n the GA s the crossover. Lke ts counterpart n nature, crossover produces new ndvduals that have some parts of both parent s genetc materal. We used a smple arthmetc form of crossover, whch s sngle pont crossover, descrbed as follows: Assume W k and W p are the th and th hyperplane of k th and p th chromosomes n the th teraton respectvely. Then W k and W p are crossed over at the l th poston. The resultng offsprngs are Wk 1 l l+ 1 l+ n = (w, w,..., w, w, w,..., w ) () Wk 1 l l+ 1 l+ n = (w,w,...,w,w,w,...,w ) (3) Where l s a random number from {,,n} and n s the feature space dmenson. Crossover rate (CR) s the number of tmes that crossover operator s appled to the populaton. -Mutaton In GAs, mutaton s randomly appled wth a known probablty. It modfes elements n the chromosomes. It means that a poston n an ndvdual s selected randomly and the value n ths poston s changed. We used Gaussan mutaton mechansm that mutates some elements of an ndvdual such that = (w,w,...,w 1 l,...,w l,..., w n ) (4) + 1 W k Where l r belongs to [1,n+1] nterval and s randomly selected. Also w l = w r l + z r l. Here z r l s a random r number drown from a Gaussan dstrbuton wth zero mean and adaptve varance. 3 What s ntellgent and robust GA-classfer (IRGA-classfer)? As t mentoned n Secton 1 the large number of tranng ponts and teratons are two necessty, whch are necessary for conventonal GA-classfer to reach a comparable performance wth Bayes classfer as an optmal classfer. In ths artcle t has been tred to remove the aforesad problems n GA-classfers. For ths reason at frst a revew on optmal hyperplane for separaton of two classes s presented. Then we defne an effcent ftness functon for obtanng these optmal hyperplanes by a GA-classfer. At the next step a new concept s defned and s called the robustness of performance of a GAclassfer aganst the number of data ponts. Eventually desgnng an ntellgent mutaton rate and crossover rate controller s consdered to help GAclassfer to obtan near the optmal hyperplanes. A. Optmum Hyperplanes A conceptual problem for computng a lnear decson functon to separate two dfferent classes s determnng a hyperplane whch has a better performance aganst recevng the next data ponts (or test ponts). Snce we suppose, there s no pror knowledge about the dstrbuton of data ponts n feature space, the optmal hyperplanes are defned as the lnear decson functon wth maxmal margn between the feature vectors of dfferent classes ths margn s calculated based on the avalable tranng ponts of the classes. Ths strategy s smlar to what has been done n Support Vector Machne (SVM) studes [7]. Fg. shows three hyperplanes ( d 1, d and d opt ) that all of them can be consdered as the decson functons, whch separate two Class1 and Class successfully. It can be seen from ths fgure that f any small nose s added to tranng ponts, as a test data (or a new tranng pont) near ths decson functons, t can mpar the recognton score ( or change the poston) of d 1 and d. Among d 1, d and d opt, only d opt separates the tranng data wth a maxmal margn from two classes. Snce the margns of two classes from the are eual, the new nosy testng ponts n the d opt Iranan Journal of Electrcal & Electronc Engneerng, Vol. 1, No. 3, July 005 3

4 classes (or new tranng ponts n them) can have a lttle effect on the recognton score (or varaton of) d opt. B. Ftness Functon Defnton It should be mentoned that the ftness functon, whch has been used n prevous researches [-6] s "the number of mss-classfed data ponts". Thus a conventonal d GA-classfer may converge to each of 1( x ) or d ( x) n Fg. that have the mnmum mssclassfed data ponts, but are not optmal decson functons. In fact, ftness functon defnton as the msclassfed data ponts, need to have many data ponts n the tranng phase for convergence to the best hyperplanes n a smple GA-classfer. Ths s a weakness aspect for conventonal GA-classfers, whch has been proved theoretcally and expermentally. Fg. d opt s the optmal decson functon among other decson functons. Maxmzng the mnmum value of the average of the Eucldean dstances of all data ponts n each class s a good algorthm, whch can dsplace d 1 and d toward d opt. We named ths algorthm Max-Mn algorthm. To see the effcency of Max-Mn algorthm for chancng away d 1 and d toward d opt, suppose that a conventonal GA-classfer converged to d 1, whch has mnmum average dstance from Class. By enterng the Max-Mn algorthm n ftness functon, GA-classfer must maxmze the margn of d 1 from Class. Thus d 1 tends to d opt. Same condton s appeared f GA-classfer s converged to d. Due to above descrptons, we defned a modfed ftness functon as: ftness(w) = penalty + mn( Σd, Σd ) (5) In (5), ftness (W) s the value of ftness functon for the chromosome W. The penalty s a negatve, absolutely large value, whch s used f the hyperplane obtaned by W, doesn't classfy all data ponts n dfferent classes C and C. In ths paper penalty s defned as 10*mn( Σ d, Σ d )*Mss, whch Mss s the number of msclassfed data ponts by hyperplanes obtaned by a chromosome W. Σd s the average of the Eucldean dstances of all data ponts n class from the hyperplane W, and mn( Σ d, Σ d ) s the mnmum value of Σd and Σ d. The GA-classfer proposed n ths artcle maxmzes the ftness functon (or mnmze ts negatve value). Obvously, when the second term n defnton of ftness (W) s set to zero, the same ftness of a conventonal GA-classfer s obtaned. The frst and necessary condton for a good performance of Euaton 5 s that the classes are separable from each other. Your comment s correct when classes have overlap n some patterns. In ths case we attach the msclassfed patterns to one class and then we use Euaton 5 as the ftness functon. It means that we accept a lttle error to adust decson hyperplanes ust between classes. C. Robust GA-classfer (RGA-classfer) To evaluate the performance of modfed ftness functon defned by (5) and compare t wth conventonal ftness functon defnton n prevous researches, a new concept of robustness s ntroduced. In ths artcle the concept of robustness s defned as the nverse of the senstvty of the solutons (hyperplanes) of a GA-classfer aganst the varaton of number of data ponts for each class. More robustness for a GAclassfer concludes more stablty for the obtaned optmum soluton. We defned the robustness of a GAclassfer as follows: (Perf ) n Robustness= [ * n Perf ] 1 (6) In (6) Perf s the performance of the best hyperplanes have been found by GA-classfer. n s the number of data ponts and ΔPerf s defned as the dfference between two obtaned performance n two dfferent number of tranng ponts. A robust GA-classfer (RGA-classfer) has a low senstvty aganst changng the number of tranng ponts. In fact the robustness s a good metrc to see how the modfed ftness functon can remove one of the mportant defects n conventonal GA-classfer, whch s need to a large number of tranng ponts to reach to an optmal performance. D. Intellgent and Robust GA-classfer (IRGAclassfer) The search mechansm of an evolutonary algorthm lkeness GA s based on three mportant operators: Selecton, Mutaton and Crossover. All of these operators have probablstcally events, but by dfferent effects on search process, thus ntellgently controllng them can help the genetc algorthm to escape from 4 Iranan Journal of Electrcal & Electronc Engneerng, Vol. 1, No. 3, July 005

5 local solutons and converge to global soluton by a faster rate. A RGA-classfer wth an ntellgent CR and MR controller s called ntellgent and robust genetc classfer (IRGA-classfer). In ths paper, we desgned a fuzzy structure to control adaptvely CR and MR n each teraton and called t Fuzzy Controlled and Robust GA-classfer (FCRGA-classfer). The fuzzy controller s constructed on some fuzzy (IF antecedents THEN conseuents) rules. Each nput and output varable are defned wth ther membershp functons. We defned three nputs for fuzzy controller n a FCRGA-classfer as follows: Ft-dst: the dstance between ftness value of the best ndvdual n th teraton and maxmum of ftness functon. Based on Euaton (5) the ftness of each chromosome s related to the number of msclassfed tranng ponts and margns between dfferent classes. The number f tranng ponts s known and the mnmum dstance of the patterns whch exst n each class can be found. Thus an approxmated value of Ftdst s avalable n each problem. UN: The number of teratons whose ftness value s constant. t: the number of teratons. Two outputs of fuzzy controller are crossover rate (CR) and mutaton rate (MR). The normalzed membershp functons of Ft-dst, UN and t are shown n Fg. 3. Fg. 3 Membershp functons of nputs (Ft-dst, UN, and t) and outputs (CR, MR) n fuzzy controller. The selecton of the shapes of membershp functons and ther locatons are based on wdely study on researches were related to GA and appeared as a survey n [8] and other references (some of them are [9-1]). To extract some effectve fuzzy rules, we know that crossover facltates exploraton whle mutaton facltates explotaton n soluton space. Ths means that when the best ftness stuck at one value for a long tme (UN s Hgh), the GA s often stuck at a local mnmum, so the crossover rate should be decreased and mutaton rate should be ncreased. Low ftness values (or Hgh values of Ft-dst) often happen n the start of GA (t s low) and we need more explotaton and less exploraton. Thus crossover rate should be decreased and mutaton rate should be ncreased and contrarwse f ftness value s ncreased (Ft-dst tends to Low values) crossover rate should be ncreased and mutaton rate should be decreased. On the other hand t should be mentoned that although all of the theoretcal and expermental researches have been done on the optmal MR and CR were constructed under some condtons or for a few test functons, but they have a common aspect, whch s a decreasng schedule for MR and CR as the number of teratons (t) s ncreased. Specally n [11], Schmtt presented an annealng schedules for MR and CR wth respect to t whch guaranty the convergence to global soluton: 1 κ.l p MR(t) = φ m.t < CR(t) 1 (7) 1 m = φ.[mr(t (8) c )] + Where φm R \ {0} and κ [1, ) can be chosen and m Lp s the populaton sze and both φc (0, 1 ] and m [1, ) can be chosen. Of course decreasng the MR and CR must be based on an mprovement n ftness values (or decreasng the Ftdst). From above lngustc descrptons seven rules for the fuzzy controller are defned as below: a) IF (t) s low and (Ft-dst) s hgh THEN (MR) s hgh and (CR) s low. b) IF (t) s medum and (Ft-dst) s hgh THEN (MR) s medum and (CR) s low. c) IF (t) s hgh and (Ft-dst) s low THEN (MR) s low and (CR) s hgh. d) IF (Ft-dst) s hgh and UN s low THEN (MR) s medum and (CR) s medum. e) IF (Ft-dst) s hgh and UN s hgh THEN (MR) s hgh and (CR) s low. f) IF (Ft-dst) s low and UN s hgh THEN (MR) s medum and (CR) s low. g) IF (Ft-dst) s low and UN s low THEN (MR) s low and (CR) s hgh. It must be mentoned that another nputs, outputs, membershp functon shapes and fuzzy rules may be ntroduced and even these parameters can be optmzed by another optmzaton algorthm [15-17]. 4 Implementaton and Results Three pattern recognton problems wth dfferent augmented feature vectors dmensons (5,4,19) were used to demonstrate the effectveness of the IRGAclassfer. A descrpton of the data sets s gven here: Iranan Journal of Electrcal & Electronc Engneerng, Vol. 1, No. 3, July 005 5

6 A. Data Sets Irs Data: Irs Data contans 50 measurements of four features from each three speces Irs setosa, Irs verscolor, and Irs vrgnca [18]. Features are sepal length, sepal wdth, petal length and petal wdth. Wne Data: Wne data contans the chemcal analyss of wnes grown n the same regon n Italy but derved from dfferent cultvars [19]. The 13 contnuous attrbutes are avalable for classfcaton. Total number of nstances s 178 whch we classfed them n fve classes. Radar Targets: An applcaton of pattern recognton s Automatc Target Recognton (ATR) for contnuous wave radars. In ths paper Jet Engne Modulatons (JEM) s used for ths purpose. In ths approach the modulaton of the radar wave by rotatng propellers and et engne blades of targets s consdered [0,1]. Ten dfferent flyng obects were chosen as ntroduced n [1] for classfcaton n 0 º elevaton angle. After samplng from backscattered sgnals and data reducton preprocess, we took a 18 ponts FFT as feature vectors for each target. B. Comparson wth Exstng Methods The performance of proposed IRGA-classfer s compared wth the performance of a smple GAclassfer (SGA-classfer) and Bayes classfer. The genetc parameters n SGA-classfer were selected as conventonal GA-classfers were proposed n prevous researches [-6]. The crossover probablty s fxed at 0.8. Avalable value of mutaton probablty s selected from the range [0.015, 0.333]. A fxed populaton sze of 0 s chosen for both SGA-classfer and IRGAclassfer. For Bayes classfer, a pror probabltes eual to tr tr, for tr patterns from class, and totally tr tranng samples, and a multvarate normal dstrbuton of the samples are consdered. Ths s smlar to Bayesan classfer s used n [-5] to show the mprovements compared to SGA-classfers ntroduced n these researches. C. Expermental Results The proposed IRGA-classfer (.e. FCRGA-classfer), SGA-classfer and Bayes classfer are tested on the data sets descrbed n secton 4-A. We mplemented dfferent classfers for ten tmes wth random selected of tranng sets. Thus the results report the average score of recognton for ten tmes repeats. Table and Table 3 present the results correspondng to Irs data and Wne data classfcaton for dfferent number of tranng samples (n=5,10,15), for 5 number of hyperplanes (H=5). The calculated robustness (defned by (6)) n some dfferent number of tranng ponts for Irs data and Wne data are gven n Table 4 and Table 5, respectvely. The robustness of the IRGA-classfer and a SGAclassfer are appeared n these Tables, wth respect to 5 tranng ponts as a reference for (n, Perf). The results n Table to Table 5 have some meanngful concepts: ) The performance of an IRGA-classfer s better than or comparable wth the Bayes classfer, whch s an optmal classfer, for any number of tranng ponts, appeared n Table and Table 3, for both Irs data and wne data classfcaton. ) The performance of a SGA-classfer for these two benchmark problems depends on the number of tranng ponts [Table and Table 3]. The larger number of tranng ponts, the better performance for SGA-classfer. (It s compatble to the theorem 1 n [3] and other expermental results n [-6]. It shows low robustness of SGA-classfer [Table 4 and Table 5]. ) On the contrary, IRGA-classfer have a good robustness [Table 4 and Table 5], because a low dependence on the number of tranng ponts. Thus the large number of tranng ponts s not necessary for good performance of an IRGA-classfer. In fact t has robustness comparable to Bayesan classfer. Radar targets classfcaton s done by ten hyperplanes (H=10) and for ten numbers of tranng ponts (n=10). In ths experment we wated untl the IRGA-classfer and SGA-classfer converged to ther optmum solutons for dfferent sgnal to nose ratos (changng the varances of Gaussan nose produces dfferent powers of nose). Table 6 shows obtaned results. Table 6 shows that the hyperplanes have been found by IRGA-classfer perform more accurate than a SGAclassfer and are comparable to Bayesan classfer. Ths means that desgned ntellgent fuzzy controller steers the GA- classfer to fnd better hyperplanes, near those have been found by Bayesan classfer for a low number of tranng ponts (n=10). At another experment, to show the effectve role of fuzzy controller n the reducton of the number of teratons, 10 out of 50 measurements are consdered as tranng data and the rest as the test data. The average scores of recognton (%) wth respect to the number of generatons for FCRGA-classfer and a SGA-classfer have been shown n Fgures 4,5,6 for each case study. In ths fgures the numbers of generatons are normalzed by 10. In Fg. 6 the SNR s 10 db. These also mean that the convergence rate of an IRGAclassfer has a consderable mprovement compared wth a SGA-classfer. Snce the Euaton 5 s more complex than the ftness defnton n a SGA-classfer, the number of generatons for a RGA-classfer s more than a SGAclassfer and n turn for FCRGA-classfer and FCSGAclassfer. Thus comparng the performances of a FCRGA and FCSGA classfers we found the better reducton of number of generatons than t has been shown n Fgures Iranan Journal of Electrcal & Electronc Engneerng, Vol. 1, No. 3, July 005

7 Table Recognton scores (%) for Irs data classfcaton wth H=5. tranng ponts=5 tranng ponts=10 tranng ponts=15 Downloaded from eee.ust.ac.r at :4 IRDT on Monday September 3rd 018 SGA IRGA Bayes SGA IRGA Bayes SGA IRGA Bayes class class class average Table 3 Recognton scores (%) for Wne data classfcaton wth H=5. tranng ponts=5 tranng ponts=10 tranng ponts=15 SGA IRGA Bayes SGA IRGA Bayes SGA IRGA Bayes class class class class class average Table 4 The robustness for dfferent tranng ponts (n) for Irs data wth respect to n=5 as a reference. n=10 n=15 n=0 n=5 SGA-classfer IRGA-classfer Bayesan classfer Table 5 The robustness for dfferent tranng ponts (n) for Wne data wth respect to n=5 as a reference. n=10 n=15 n=0 n=5 SGA-classfer IRGA-classfer Bayesan classfer Table 6 Recognton scores (%) wth respect to dfferent SNRs wth n=10 and H=10. dfferent SNRs(dB) SGA IRGA Bayes Fg. 4 The average scores of recognton (%) wth respect to the number of generatons(*10) for Irs data classfcaton (n=10 and H=5). Fg. 5 The average scores of recognton (%) wth respect to the number of generatons (*10) for Wne data classfcaton (n=10 and H=5). Iranan Journal of Electrcal & Electronc Engneerng, Vol. 1, No. 3, July 005 7

8 Fg. 6 The average scores of recognton wth respect to the number of generatons (*10) for radar targets classfcaton (n=10, H=10 and SNR=10 db). 5 Conclouson An evolutonary computaton method s proposed for obtanng optmal hyperplanes n feature space, desgnng ntellgent and robust GA-classfer (IRGAclassfer). Conventonal GA-classfers, whch have been ntroduced n prevous researches, have two mportant defects. They need a large number of tranng ponts and a large number of teratons to converge to optmum hyperplanes. Both of these two prereustes are usually unreachable n practce and can restrct the performance of conventonal GA-classfers. In ths artcle a new concept, named the robustness of a GA-classfer, has been proposed. It has been defned as the nsenstvty of performance of a GA-classfer under ncreasng the number of tranng ponts, to remove the frst weakness aspect of conventonal GAclassfers. On the other hand the dea of desgnng the ntellgent controllers for adaptng the crossover and mutaton rate n a GA-classfer has been proposed to steer the GA-classfer to optmum soluton and to escape t from local solutons. It can remove another weakness of usual GA-classfers, whch s the need for large number of teratons to converge to optmum hyperplanes. The IRGA-classfer fnd the decson hyperplanes whch are fne tuned between dfferent classes, no closer to one class. The performance of desgned IRGA-classfer, whch s FCRGA-classfer compared wth a smple GAclassfer and Bayesan classfer for three pattern recognton problems wth low, medum and hgh feature space dmensons. The expermental results show a better robustness, performance and convergence rate for IRGA-classfer compared wth a SGAclassfer. Also smlar performances have been obtaned for IRGA-classfer and Bayesan classfer for low number of data ponts. Both of these results are two evdences of removng two essental problems n conventonal GA-classfers usng IRGA-classfer. Other ntellgent controllers (e.g. Neural Network structures) and other evolutonary classfers (e.g. Partcle Swarm classfer) should be studed to nvestgate ther performance compared wth proposed IRGA-classfer n ths paper. Acknowledgement Ths research has been supported by the Iranan Telecommuncaton Research Center (ITRC) through grant References [1] D. E. Goldberg, Genetc Algorthms n Search, Optmzaton and Machne Learnng, Readng, MA: Addson-Wesley, [] S. K. Pal, S. Bandyopadhyay and C.A.Murthy, Genetc Classfers for Remotely Sensed Images: Comparson wth Standard Methods, Internatonal Journal of Remote Sensng, Vol., No.13, pp , 001. [3] S. Bandyopadhyay, C. A. Murthy and S.A.Pal, Theoretcal Performance of Genetc Pattern Classfer, Journal of The Frankln Insttute 336, pp , [4] S. Bandyopadhyay, C. A. Murthy and S.k.Pal, VGA Classfer : Desgn and Applcaton, IEEE Transactons On Systems, Man, and Cybernetcs Part B: Cybernetcs, Vol.30, No.6, pp , Dec.000. [5] S. Bandyopadhyay and S.K.Pal, Pxel Classfcaton Usng Varable Strng Genetc Algorthms Wth Chromosome Dfferentaton, IEEE Transactons on Geoscence and Remote Sensng, Vol.0, No., pp , Feb.001. [6] S. Bandyopadhyay, S.K.Pal and B.Aruna, Multbectve GAs, Quanttatve Indeces, and Pattern Classfcaton, IEEE Transactons On Systems, Man, and Cybernetcs-Part B: Cybernetcs, Vol. 34, No. 5, Oct [7] C. J. C. Burges, A Tutoral on Support Vector Machnes for Pattern Recognton, Data Mnng and Knowledge Dscovery,, pp , [8] A. E. Eben, R. Hnterdng and Z. Mchalewcz, Parameter Control n Evolutonary Algorthms, IEEE Transactons on Evolutonary Computaton, Vol. 3, No., pp , July [9] T. Bäck and M. Schütz, Intellgent Mutaton Rate Control n Canoncal Genetc Algorthms, n Foundatons of Intellgent Systems (Lecture Notes n Artfcal Intellgence, 1079), Z.Ras and M. Mchalewcz, Eds. New York: Sprnger-Verlag, pp , Iranan Journal of Electrcal & Electronc Engneerng, Vol. 1, No. 3, July 005

9 [10] J. Hesser and R. Männer, Toward an Optmal Mutaton Probablty for Genetc Algorthms, n Proc. 1st conf. Parallel Problem Solvng from Nature, (Lecture Notes n Computer Scence, Vol. 496), H. Schwefel, and R. Männer, Eds. Berln, Germany: Sprnger-Verlog, pp.3-3, [11] L. M. Schmtt, Theory of Genetc Algorthm II: Models for Genetc Operators Over the Strng Tensor Representaton of Populatons and Convergence to Global Optma for Arbtrary Ftness Functon Under Scalng, Theoret. Comput. Sc. 310, 004. [1] I. Wegener, Theoretcal Aspects of Evolutonary Algorthms, a techncal report publshed by: Unversty of Dortmund, May 001. [13] J. T. Tou and R.C. Gonzalez, Pattern Recognton Prncples, Addson-Wesely, Readng MA, [14] F. Herrera, M. Lozano and J.L.Verdegay, Tacklng real-coded genetc algorthms: Operators and tools for behavoral analyss, Artfcal Intell., Rev., Vol. 1, pp , [15] Y. Sh, R. Eberhart and Y. Chen, Implementaton Of Evolutonary Fuzzy Systems, IEEE Transactons On Fuzzy Systems, Vol.7, No., pp , Aprl [16] M. Stenes and H. Robous GA-Fuzzy Modelng and Classfcaton: Complexty and Performance, IEEE Transactons on Fuzzy Systems, Vol. 8, No. 5, pp , October 000. [17] H. Ishobuch, T. Nakashma and T. Murata, Performance Evaluaton of Fuzzy Classfer Systems for Multdmensonal Pattern Classfcaton Problems, IEEE Transactons on Systems, Man and Cybernetcs, Vol. 9, No. 5, pp , October [18] R. A. Fsher, The use of Multple Measurements n Toxonomc Problems, Ann. Eugen, Vol. 7, pp ,1936. [19] Unversty of Calforna, Irvne, va anonymous ftp ftp.cs.uc.edu/pub/machne-learnng-databases. [0] M. R. Bell, and R.A. Grubbs, JEM Modelng and Measurement for Radar Target Identfcaton, IEEE Transactons on Aerospace and Electronc Systems, Vol.9, No. 1, pp , Jan [1] S. H. Zahr, H. Zaree, and M. R. Agha-Ebrahm, Automatc Radar Target Recognton Usng Modulaton of Targets on Radar Sgnal, In Proc. of ICEE 000 n Fars, pp , Isfahan, May 000. Iranan Journal of Electrcal & Electronc Engneerng, Vol. 1, No. 3, July 005 9