Flagged and Compact Fuzzy ART: Fuzzy ART in more efficient forms

Transcription

1 he Internatonal Journal of ACM Jordan (ISSN , Vol., No. 3, September Flagged and Compact Fuzzy AR: Fuzzy AR n more effcent forms Kamal R. Al-Raw, and Consuelo Gonzalo 2 ; Department of Computer Scence, Faculty of Informaton echnology, Petra Unversty, Amman, Jordan. DASI, Facultad de Informátca; Unversdad Poltécnca de Madrd, Span 2. e-mal: kamalr@uop.edu.o and chelo@f.upm.es Abstract: wo new smplfed algorthms for Fuzzy AR have been developed. Only commtted category nodes C rather than the full capacty of the category nodes N (N>>C are nvolved n the determnaton of the wnnng categoy node. In addton to that, the ntalzaton for weghts and choce values has been elmnated, and the A = calculaton of s replaced by M, snce A + =. A hs reduces a lot the tranng tme wthout alterng the categorazaton accuracy. Although, the new archtectures are presented toward the fuzzy AR ANN n ths work. However, they can be appled to all module of AR. Keywords: Compact fuzzy AR, Flagged fuzzy AR, AR ANN, Unsupervsed AR, and Unsupervsed learnng.. Introducton he Fuzzy AR (Adaptve Resonance heory s an unsupervsed AR-based ANN. Its archtecture has been desgned for learnng and categorzaton of arbtrary analog or bnary mult-valued nput patterns. It s wdely used n the lterature. Fuzzy AR s used for a wde varety of applcatons n many felds: classfyng wreless sensor data to

2 he Internatonal Journal of ACM Jordan (ISSN , Vol., No. 3, September reduce communcaton cost [, 2], evaluatng rsk prorty number n the manufacturng ndustry to mprove producton qualty and productvty [3], automatc freeway ncdent detecton [4], Electrcal Load Forecastng [5], Pattern Recognton [6, 7], analyzng Hgh-resoluton turbne spectral data [8], gene clusterng [9], fault detecton and classfcaton on transmsson lnes [0], constructng the P2P network topology [], manufacturng qualty control [2]. [3, 4] they bult VLSI archtecture for fuzzy AR. We beleve that smplfcaton of Fuzzy AR wll have a great benefcal to the scentfc communty. here are more than,500 ctatons for Fuzzy AR. For Fuzzy AR nput patterns ( complements at the nput layer F. he choce functon A t (, =,..., M [0, ] are presented wth ther commtted category node of the category layer F 2 s computed: t Where C s the number of commtted category nodes, (, =,..., C for each w the weghts that connect the nput node "" wth the commtted category node "", and α >0, the choce value parameter. node. ( A = = = α + he choce value w ; w =,..., C... (t represents the actvaton level of each commtted category he wnnng commtted category node s determned. It represents the category node wth the hghest choce value among all category nodes N n the category layer: J = max{ ; =,..., N}...2 he value of N s normally much larger than C (N>>C. All category nodes N are nvolved, whch has been employed by [5] nstead of commtted category nodes C. her reasonng for ths s to let uncommtted category nodes be commtted, when t s

3 he Internatonal Journal of ACM Jordan (ISSN , Vol., No. 3, September requred, n a sequence order (, 2,..., -,, +,..., N. o acheve ths, they assgned a very small postve value φ to each category nodes before tranng s started. he authors s assgned the term F 2 -order constants for ths operaton. hese values are decreasng as the ndex of the order of category node n the memory feld s ncreased. 0 < φ <... < φ <... < φ 0 N...3 In ths way, when all commtted category nodes are n shut off mode, as they fal to represent the current nput, the uncommtted category node ( C + wll be commtted, snce t has the hghest choce value ( F2 -order constant among all uncommtted category nodes as prearranged. hey are assgned values near zero assumng that there s no computed choce value, for any commtted category node, less thanφ. he match value s computed for the wnnng category node J by = ( A w = A J...4 he match value represents a hypothess, that the current nput A (t belongs to the wnnng category node J of the F2 feld. hs hypothess s tested aganst predetermned vglance parameter ρ [0, ]. he vglance parameter represents the mnmum confdence level that s requred to accept that the wnnng node J of the F 2 feld, represents the category of the current nput A (t. If the match value of the wnnng node s less than ρ, the hypothess s reected and ths commtted category node shuts off as far as the current nput s presented to the network. hs s to prevent the persstent selecton of the same category node durng search. Shut off s smply done by assgnng - to the choce value of the faled category node. Researchng for another wnnng commtted category node s trggered among all category nodes N. he network keeps searchng for maxmum choce value node J,

4 he Internatonal Journal of ACM Jordan (ISSN , Vol., No. 3, September dong computaton of the match functon for node J, and testng aganst the vglance parameter ρ, for each commtted node of the F2 feld. hs s done n order, accordng to ther choce values rank, untl ether one of the commtted category node can represent the current nput A (t (resonance occurs, then learnng the weght wj of the selected category node J, or f none, the uncommtted category node wth ndex C+, whch has the hghest choce value among all uncommtted category nodes as prearranged, wll be pcked by the network to represent the current nput. he match value of new commtted category node passes the value vglance parameter ρ, snce t has the value of one. he weghts of the selected category node are updated n order to ncorporate the characterstcs of the nput pattern to category J: w new J = β ( A w old J + ( β w old J...5 After learnng the weghts of the selected category node J, a check should be done to see f a commtted category node has been chosen to represent the current nput or a new category node has been commtted. hs s to ncrease the number of commtted category node C by one or not. C controls the computaton of the choce functon for commtted category nodes only. If the ndex of the selectve category node J s greater than C that means the uncommtted category node C+, has been chosen to represent the current nput, because all commtted category nodes faled to do so. hs uncommtted category node has the maxmum prearranged choce valueφ C + among the choce values φ of all other uncommtted category node, snce t s prearranged so. he full archtecture of the Fuzzy AR s shown n (fgure-.

5 he Internatonal Journal of ACM Jordan (ISSN , Vol., No. 3, September Newly developed versons of Fuzzy AR he determnaton of the wnnng category node among the full capacty of the network N, as reported by [5, 6], s tme consumng. he capacty of the system can be very large especally when t s workng n a non-homogeneous envronment. Uncommtted category nodes can be commtted n sequental order wthout usng the F2 -order constants (the prearranged choce valuesφ and wthout ncludng all the capacty of the category layer N to determne the maxmum choce value node J. wo smplfed versons of Fuzzy AR archtectures we present the Flagged Fuzzy AR, and the Compact Fuzzy AR. he frst approach nvolves the uncommtted category node wth rank C+ n the category layer together wth all commtted category nodes to determne the maxmum choce value node J. A total of C comparson s requred rather than N- as s the case n the orgnal Fuzzy AR archtecture. Whle n the second approach, only the uncommtted category nodes are nvolved n the determnaton of the maxmum choce value node. hs requres C- compressons for a current exemplar. We have to remember that N>>C. 2. Flagged Fuzzy AR 2.. he archtecture of Flagged Fuzzy AR here s no reason at all to nvolve C + 2,..., N to determne the maxmum choce value node. Only the uncommtted category node wth rank C+ n the category layer wll be nvolved. hs uncommtted category node s flagged by assgnng a value of φ C + to ts choce value [7] such that;

6 he Internatonal Journal of ACM Jordan (ISSN , Vol., No. 3, September < φ < 0 + shut off C...6 the weghts A negatve value s assgned for φ C +, because the nput features A as well as w never have negatve values. So, the choce value for any commtted category node s never a negatve value, 0 ; =,..., C. 7 However, the value of φ C + must be greater than the choce value of commtted category nodes that are n shut off mode. In ths way, when all commtted category node are n shut-off mode, the flagged node wth ndex C+ n the category layer, wll be chosen as the maxmum choce value node. We should not worry about the match value of a new commtted category node, snce the match value of any new commtted node s equal to one, whch s the hghest value that the vglance parameter ρ can have. hat s because A s normalzed to [0, ] before ts presentaton to the network, and the ntal weghts for category nodes are equal to one. So nput w A s a subset of, C +. Whch means A w, C + = A. Computng the match functon for the subset choce always leads to one as demonstrated below: 2 M M ( A w, C+ = A = = M = M = M...8 herefore, the uncommtted flagged node C+ wll not go to shut off mode. It wll pass the match test for sure. he choce value s computed as that for fuzzy AR (equ.. he determnaton of the maxmum choce value node s among C+ nodes rather than the full capacty of the category node N. he match value s: = ( A w M J...9

7 he Internatonal Journal of ACM Jordan (ISSN , Vol., No. 3, September A = he denomnator s M here rather than as n (equ.4, snce A + =. A After resonance occurs, a check should be done to see f the flagged uncommtted category node s chosen. If J>C then the flagged node has been chosen. he number of commtted category node must be ncreased by one (C=C+ and the weghts of the new flagged node w, C + should be ntated; w ; =,...,... 0, C + = he full archtecture of the Flagged-Fuzzy AR s shown n (fgure-2. In addton to the commtted category nodes only the flagged uncommtted category node are nvolved n determnaton of the maxmum choce value node ranng algorthm of the Flagged Fuzzy AR SEP Input parameters; a Dynamc parameters: - ρ (0, ]: he vglance parameter. Note that ρ 0 - β (0, ]: he dynamc learnng parameter; β = for fast learnng. note that β 0 - α >0: he choce value parameter. hs parameter s used to break the te for the choce values toward the most probable category node to represent nput patterns. However, t can be elmnated snce such occurrence s rare, and re fndng the maxmum choce value node s requred much less works than the orgnal fuzzy AR. b Data characterstcs; - M: he dmenson of the nput features. - Pt: he number of exemplars to be used n learnng.

8 he Internatonal Journal of ACM Jordan (ISSN , Vol., No. 3, September c Intalzaton; - Number of teraton t=. - Number of commtted category nodes C=. - C + = 0. SEP 2 New nput; A ( t a = a for M for M + SEP 3 Compute the choce functon for all commtted category nodes; ( A w = = α + = w, =,..., C SEP 4 Reset: Determne the node J, whch has the maxmum choce value; = max{ }, =,..., C+ J SEP 5 Matchng crteron: If ( = ( A w / M J < ρ then; - Shut off ths node to put t out of competton; J = - GOO SEP (4 SEP 6 If (J>C hen new category node has been commtted - C=J

9 he Internatonal Journal of ACM Jordan (ISSN , Vol., No. 3, September w = ; =,..., - C + = 0. SEP 7 ranng; new J J w = β ( A w + ( β w old J SEP 8 If (t<pt hen; - t=t+ - GOO SEP (2 SEP 9 ranng has been done. he network s ready for categorzaton. old J 2.2 Compact Fuzzy AR 2.2. he archtecture of Compact Fuzzy AR Uncommtted category nodes can be commtted n sequental order wthout usng even the flagged uncommtted category node. It nvolves only the commtted category node to determne the maxmum choce value node J. he choce functon s computed for commtted category nodes. he maxmum choce value node J s determned among commtted category nodes C only. = max{ }; =,..., C.. J he match value of the selected category node J s tested aganst the predetermned value of the vglance parameter ρ. If the match value of node J s less than ρ, the node s shut off by assgnng a value of to ts choce value to put t out of competton durng the current nput. Otherwse, the node s traned, all commtted category nodes are on, and new nput s presented to the network. When the maxmum choce value equals all commtted category nodes are n shut off mode. he uncommtted category node C+ should be commtted to represent

10 he Internatonal Journal of ACM Jordan (ISSN , Vol., No. 3, September the current nput n order to prevent the fragmentaton of the category layer. Smply tranng the ntal weghts of the category node wth ndex C+, and ncreasng the count of the commtted category nodes by one can do ths. hs commts the uncommtted category nodes accordng to ther order n the category layer. he number of comparson needed to determne the maxmum choce value node s (C- rather than (N- whch the orgnal Fuzzy AR algorthm requres. hs wll save a lot of computaton tme, keepng n mnd that N>>C. In the case of new category node should be commtted, ts weghts wll be updated through the next equaton: frst w = βa + ( ; =,...,...2, C + β Accordng to ths equaton weghts ntalzaton ( w ; =,..., ; =,..., N s not requred, as reported by [5]. It s mplemented mplctly n the tranng equaton tself. hs wll save tme snce ths equaton requres less arthmetc operatons. he full archtecture of Compact-Fuzzy AR s shown n (fgure 3. Commtted category nodes are shown n dark whereas uncommtted category nodes are shown n lght. Weghts connect all nput layer nodes to commtted category nodes only. Weghts are not connected to uncommtted category nodes snce they are not commtted yet (they are not assgned weghts yet ranng algorthm of Compact Fuzzy AR SEP Input parameters; a Dynamc parameters; - ρ (0, ]: he vglance parameter.

11 he Internatonal Journal of ACM Jordan (ISSN , Vol., No. 3, September β (0, ]: he dynamc learnng parameter; β = for fast learnng. - α >0: he choce value parameter. It can be elmnated. b Data characterstcs; - M: he dmenson of the nput features. - Pt: he number of exemplars to be used n learnng. c Intalzaton; - Number of teratons t=. - Number of commtted category nodes C=. SEP 2 New nput; A ( t a = a for M for M + SEP 3 Compute the choce functon for all commtted category nodes; ( A w = = α + = w ; =,..., C SEP 4 Reset: Determne the node J, whch has the maxmum choce value; = max{ } ; =,..., C J

12 he Internatonal Journal of ACM Jordan (ISSN , Vol., No. 3, September SEP 5 If = (all commtted category nodes are n shut-off mode then a J new node (the node that ts order n the category layer s C+ should be commtted; - Increase the number of commtted nodes by one; C=C+ - If n fast-learnng mode β=; Assgn the values of the nput feature to the weghts of ths node; w = A ; =,..., frst C (t Else (normal mode frst w = βa + ( β ;=,..., C - GOO SEP (2 SEP 6 Matchng crteron: If ( = ( A wj / M < ρ then; - Shut-off ths node to put t out of competton; J = - GOO SEP (4 SEP 7 Learnng; w = β( A w + ( β w new J old J old J SEP 8 If (t<pt then;

13 he Internatonal Journal of ACM Jordan (ISSN , Vol., No. 3, September t=t+ - GOO SEP (2 SEP 9 ranng has been done. he network s ready for categorzaton. 3 Categorzaton of the Flagged fuzzy AR and the Compact Fuzzy AR At the end of the tranng phase, all weghts are fxed at ther fnal update. he number of category node C s known. he network s ready for categorzaton. Whle the mach value for orgnal fuzzy AR s: = = It s for Flagged and Compact fuzzy AR: = ( A w A ( A w J J M So, the tranng algorthm for both Flagged and Compact fuzzy AR s: SEP Input: A ( t a = a for M for M + SEP 2 Compute the choce values for all commtted nodes; ( A w = = α + = w ; =,..., C

14 he Internatonal Journal of ACM Jordan (ISSN , Vol., No. 3, September 200 SEP 3 Determne the node J, whch has the maxmum choce functon among all commtted category nodes; = max{ } ; =,..., C J SEP 4 Match testng: ( A w If (the match value for the wnnng node J : = M Category node J represents the category of ths nput Else he network fals to categorze ths nput J then; ρ SEP 5 If more categorzaton s needed GOO SEP. SEP 6 Categorzaton has been done. 4 Conclusons: he comparson among the orgnal Fuzzy AR, Flagged-Fuzzy AR, and Compact-Fuzzy AR s shown n (table. It shows clearly that Flagged-Fuzzy AR and Compact-Fuzzy AR are faster than the orgnal algorthm of Fuzzy AR. he man pont nfluences the reducton of the tranng tme s the number of comparsons needed to determne the wnnng category node. hey are N-, C, and C- for the orgnal Fuzzy AR, Flagged-Fuzzy AR and Compact-Fuzzy AR, respectvely. Moreover, snce C ncreases from up to ts fnal value tranng phase, an average of C fnal C fnal at the end of /2 comparsons for Compact fuzzy AR compared to N comparsons for the fuzzy AR s requred for the determnaton of the maxmum choce value node. Keep n mnd that we repeat the process of the determnaton of the

15 he Internatonal Journal of ACM Jordan (ISSN , Vol., No. 3, September maxmum choce value node C tmes n the case of a new category node must be commtted to represent an nput. In addton to that the match values as well as the weghts update of the newly commtted category node are requred less computaton. Moreover, the ntalzaton for weghts and choce values for category nodes s elmnated. REFERENCES []- Patrkar, R. and Akowar, S. Neural Network Based Classfcaton echnques for Wreless Sensor Network wth Cooperatve Routng. 2th WSEAS Internatonal Conference on COMMUNICAIONS. 2008, [2]- Walchl, M. and Braun,. Effcent Sgnal Processng and Anomaly Detecton n Wreless Sensor Networks. Proceedngs of Evo Workshops. 2009, [3]- Keskn, G and Zkan, C. An Alternatve Evaluaton of FMEA: Fuzzy AR Algorthm Qualty and Relablty Engneerng Internatonal. 2009, 25, [4]- Al-Deek, H. Ishak, S. and Wang, M. A New Short erm raffc Predcton and Incdent Detecton System on I-4. Fnal Report Volume II, Department of Cvl and Envronmental Engneerng, Unversty of Central Florda, FL, USA 200. [5]-Lopes, M. Lotufo, A.and Mnuss, C. Applcaton of the Fuzzy AR & ARMAP Neural Network to the Electrcal Load Forecastng Problem. In: Jmmy J. Zhu. (Org.. Forecastng Model - Methods and Applcatons. 2 ed. : Concept Press. 200, [6]-Camara-Chavez, G. and Arauo, A. Invarant Pattern Recognton by Proecton Hstograms and Fuzzy AR Neural Network. Proceedngs of CLEI. 2005, -7. [7]-Saengdeeng, A. Qu, Z. and Chaeroenlap, N. 2-D Shape Recognton usng Recursve Landmark Determnaton and Fuzzy AR Network Learnng. Neural Processng Letters 2003, 8, [8]- Harrson, G. and aylor, F. Gas urbne Vbraton Analyss wth Fuzzy AR Neural Network. Jont Conference on Neural Networks(IJCNN' , [9]- omda, S. Hana,. Honda, H. and Kobayash,. Gene Expresson Analyss Usng Fuzzy AR. Genome Informatcs. 200, 2, [0]- Vaslc, S. and Kezunovc, M. Fuzzy AR Neural Network Algorthm for Classfyng the Power System Faults. IEEE rans. on Power Delvery, 2005, 20, 2,

16 he Internatonal Journal of ACM Jordan (ISSN , Vol., No. 3, September []- Wang, Y. Wang, W, On Studyng P2P opology Based on Modfed Fuzzy Adaptve Resonance heory, Proceedngs of Lecture Notes n Computer Scence, 2006, 44, [2]-Pacella, M. Semeraro, Q. and Anglan, A. Manufacturng Qualty Control by Means of a Fuzzy AR Network raned on Natural Process Data. Engneerng Applcatons of Artfcal Intellgence 2004, 7, [3]-Lubkn, J. and Cauwenberghs, G. Mxed-Mode VLSI Implementaton of Fuzzy AR and Vector Quantzaton. Proc. 7th Int. Conf. Mcroelectroncs for Neural, Fuzzy and Bo-nspred Systems (McroNeuro' , [4]-Gawarle, A. Deshmukh, A. and Keskar, A. owards Fuzzy Adaptve Resonance heory Structure Desgns wth VLSI. Dgtal echnology Journal. 2009, 2, [5]-Carpenter, G. Grossberg, S. and Rosen, D.: Fuzzy AR: Fast Stable Learnng and Categorzaton of Analog Patterns by an Adaptve Resonance System, Neural Networks, (4 (99, [6]-Carpenter, G. and Grossberg, S.: A Massvely Parallel Archtecture for a Self- Organzng Neural Pattern Recognton Machne, Computer Vson, Graphc, and Image Processng, (37 (987, [7]-Al-Raw, K. and Gonzalo, C. Flagged Fuzzy AR: A Smplfed Archtecture for Fuzzy AR Artfcal Neural Network, ICINS, 2003,

17 he Internatonal Journal of ACM Jordan (ISSN , Vol., No. 3, September F 2 2 C φ C+ φn w F a a 2 a a M a M+ a 2 M Fgure : he archtecture of Fuzzy AR. he full capacty N of the category layer s nvolved for determnaton the maxmum choce value node J. hey are shown n dark. Weghts are connected to all category nodes. Weghts that are connected to uncommtted category nodes are shown n lght. hs s because they are not learned yet. he number of comparson whch s needed to determne the maxmum choce value node J s N-, snce t s carred out among all category nodes. hs ncreases tranng tme. If J>C then the uncommtted category node wth ndex C+ has been commtted, snce ts choce value φ C + = max,{ φ ; =C+,..., N. hat because these constant are arranged as φ C + <... < φn. It has been prearranged ths way to let category nodes to be commtted n order to prevent the fragmentaton of the category layer.

18 he Internatonal Journal of ACM Jordan (ISSN , Vol., No. 3, September F 2 2 C φ C+ w F a a 2 a a M a M+ a 2 M Fgure-2: he archtecture of Flagged Fuzzy AR. Only commtted category nodes and the uncommtted category node wth ndex C+ n the category layer are nvolved n the determnaton of the maxmum choce value node J. hese category nodes are shown n dark. Weghts are connected to all these category nodes. Category nodes that are not nvolved n the determnaton of the maxmum choce value node, are shown n lght. Weghts are not connected to them. Weghts that connected to the flagged node (uncommtted category node wth ndex C+ are shown n lght. hs s because they are not ntated yet. It wll be ntated (

19 he Internatonal Journal of ACM Jordan (ISSN , Vol., No. 3, September F 2 2 C w F a a 2 a a M a M+ a 2 M Fgure-3 he archtecture of Compact Fuzzy AR. Only commtted category nodes are nvolved n the determnaton of the maxmum choce value node J. hese category nodes are shown n dark. Weghts connect all nput layer nodes to commtted category nodes only. Uncommtted category nodes are shown n lght. Weghts are not connected to them snce they are not commtted yet (they are not assgned weghts yet. he number of comparson needed to determne the maxmum choce value node s C-, snce t s carred out among commtted category nodes only. hs reduces tranng tme.

20 he Internatonal Journal of ACM Jordan (ISSN , Vol., No. 3, September Fuzzy AR Flagged Fuzzy AR Compact Fuzzy AR Intalzaton for Choce value 0 < φ <... < φ <... < φ 0 N φ C+ = 0. None Compute choce functon ( A w = = ; =,..., C α + w = Same Same Determnaton of max Check for new commtted node Number of comparson for max = max{ ;,..., N} = max{ ; =,..., C + } J = J > C J J = max{ ; =,..., C} J > C = N C C J Match testng Weghts ntalzaton Weghts update for old node Weghts update for new node = ( A w = A J ρ w = ; =,...,2 M ; =,..., N None None w = β ( A w + ( β w Same Same new J new J old J old J old J old J 2 M = ( A w M J ρ = ( A w M J ρ frst frst w = β ( A w + ( β w w C = β * A + ( β w C = β * A + ( β able-: Comparson among Orgnal, Flagged, and Compact algorthms of Fuzzy AR. he last two have been developed n ths study. Flagged and Compact algorthms are faster. Compact algorthm s recommended.