A Tool for Evolving Artificial Neural Networks

Transcription

1 A ool for Evolvng Artfcal Neural Networks Efstratos F. Georgopoulos, 3, Adam V. Adamopoulos, 3 and Sprdon D. Lkothanasss 3 Abstract. A hybrd evolutonary algorthm that combnes genetc programmng phlosophy, wth localzed Extended Kalman Flter (EKF) tranng method s presented here. hs algorthm s used for the topologcal evoluton and tranng of Mult-Layered Neural Networks. It s mplemented as a vsual software tool n C++ programmng language. he proposed hybrd evolutonary algorthm s appled on two bo-sgnal modelng tasks: the Magneto Encephalogram (MEG) of epleptc patents and the Magneto Cardogram (MCG) of normal subjects, exhbtng very satsfactory results. INRODUCION One of the man problems that are faced when Artfcal Neural Networks (ANN) and especally Multlayer Perceptrons, are appled on some tasks, s fndng the network archtecture or topology that s best suted for the task at hand. A small network for the problem mght causes poor learnng ablty, whle a large one mght cause poor generalzaton. Untl now the common method to determne the archtecture of a neural network s by tral and error. However, n the last years there have been many attempts, n the drecton of desgnng the archtecture of a neural network automatcally, that have led to a varety of methods. wo subcategores of such methods are a) the constructve and b) prunng (destructve) algorthms, [8], [9]. Roughly speakng, a constructve algorthm starts wth a mnmal network, that s an ANN wth a mnmal number of hdden layers, hdden neurons and connectons, and adds new layers, neurons or connectons, f t s necessary, durng the tranng phase. On the opposte, a prunng (destructve) algorthm does the opposte, starts wth a maxmal network and deletes the unnecessary layers, nodes and connectons durng tranng. Another approach to ths problem s by usng Genetc Algorthms. Genetc Algorthms are a class of optmzaton algorthms, whch are good n explorng a large and complex space n an ntellgent way n order to fnd values close to the global optmum (see [], [5], [0] and [] for detals). he desgn of a near optmal topology can be formulated as a search problem n the archtecture space, where each pont n the space represents network archtecture. he tranng can be formulated as a search problem n the weght space. Snce the end of the last decade, there have been several attempts to combne the technology of neural networks wth that of genetc algorthms. Gven some performance crtera, for example error, generalzaton ablty, learnng tme, archtectural complexty etc, for the archtecture, the performance level of all archtectures forms a surface n the space. he optmal archtecture desgn equals to fndng the optmum pont on ths surface. he frst attempts, descrbed n [0], [3], [5] and [7], focused manly on the problem of tranng the networks and not n the topology desgn. hey used neural networks wth fxed archtecture and genetc algorthms n order to search the weght space for some near optmum weght vector that solves the problem of network tranng. hat s, they used genetc algorthms nstead of some classcal tranng algorthm. Soon the man research nterest moved from the tranng, to the search for the optmal archtectural (or topologcal) desgn of a neural network. Some frst works used genetc algorthms n order to mtate the prunng algorthms. hey start wth a network larger than necessary for the task and then use a specally desgned genetc algorthm to defne whch combnaton of connectons s suffcent to, quckly and accurately, learn to perform the target task, usng back propagaton. Mller et al. [] dd that for some small nets. he same problem, but for larger networks, was faced by Whtley and Bogard n [6]. Bornholdt and Graudenz n [9], used a modfed GA n order to evolve a smplfed model of a bologcal neural network and then appled the algorthm to some toy Boolean functons. A dfferent approach to the desgn and evoluton of modular neural network archtectures s presented n [3]. Bllngs and Zheng n [8] used a GA for the archtectural evoluton of radal bass functon (RBF) networks. he most recent approach and maybe the most successful one, to the problem of fndng the near optmum archtecture s presented n [8]. here, Yao and Lu propose a new evolutonary system, the EPNet, for evolvng artfcal neural networks behavor. he last couple of years, there s an ncreasng nterest n the use of mult-objectve optmzaton methods and especally evolutonary mult-objectve technques for neural network tranng and structure optmzaton. wo very nterestng approaches are presented n [3] and [3]. he present work s the sequence of a seres of efforts concernng the applcaton of evolutonary algorthms for the optmzaton of neural networks. In [7] a neural network model wth bnary neurons was evolved by a modfed genetc algorthm n order to learn some Boolean functons. In [], [], [3], [4], [5], [6], [7], [], [8] and [9] genetcally evolved artfcal neural networks were successfully used for a varety of problems. In ths paper we present a hybrd evolutonary method that looks lke more to a genetc programmng technque for the evoluton of a populaton of feed-forward Mult Layered Perceptrons [4]. hs hybrd algorthm combnes a genetc programmng technque (for detals see [6]) for the evoluton of the archtecture of a neural network, wth a tranng method based on the localzed Extended echnologcal Educatonal Insttute of Kalamata, Greece, e-mal: sfg@tekal.gr Dept. of Medcne, Democrtus Unversty of hrace, Greece, e-mal: adam@med.duth.gr 3 Dept. of Computer Engneerng & Informatcs, Unversty of Patras, Greece, e-mal: lkothan@ct.gr 3

2 Kalman Flter (EKF), known as Multple Extended Kalman Algorthm (MEKA). he MEKA s descrbed n detal n [4]. he novelty of ths effort depends on, apart from the combnaton of evoluton technques wth MEKA, the capablty of the proposed method to search, not only for the optmal number of hdden unts, but also, for the number of nputs needed for the problem at hand; of course ths stands only for tme seres predcton problems where the number of needed past values, whch represent the network s nputs, s unknown. hs hybrd algorthm s an evolved and heavly enrched verson of an older algorthm that was developed by the authors and presented n [4], [7] and [9]. Furthermore ths evolutonary neural network system has been mplemented as a vsual tool n C++ wth a graphcal user nterface. In order to test the ablty of ths algorthm to produce networks that perform well, we apply the system on two bosgnals, namely the Magneto Encephalogram (MEG) recordngs of epleptc patents and Magneto Cardogram (MCG) of normal subjects. he algorthm produces networks wth small szes that perform well. he rest of the paper s organzed as follows. Secton descrbes the hybrd evolutonary algorthm, whle the numercal experments are presented n secton 3. Fnally, secton 4 dscusses the concludng remarks. HE HYBRID EVOLUIONARY ALGORIHM. HE MULIPLE EXENDED KALMAN ALGORIHM - MEKA Consder a network characterzed by a weght vector w. he average cost functon that should be mnmzed durng the tranng phase s defned n terms of N nput-output patterns as follows: E w d n y n N av = j j N () n = j C n the actual response of output neuron j when nput pattern n s presented, whle the set C ncludes all the output neurons of the network. he cost functon E w depends on the weght vector w due to the fact that Where d j ( n ) s the desred response and yj av yj ( n ) tself depends on w. Concentratng on an arbtrary neuron, whch mght be located anywhere n the network, ts behavor durng the tranng phase may be vewed as a non-lnear dynamc system, whch n the context of Kalman flter theory may be descrbed by the followng statemeasurement equatons [4], [4]: w( n ) w( n d( n) y( n) e( n) ϕ, + = ) () = + (3) y n = x n w n (4) Where the teraton n corresponds to the presentaton of the nth x n and y n are the nput and output vector of nput pattern, e ( n ) s the measurement error at the neuron respectvely and output of neuron, the nstantaneous estmate of whch s gven by: E ( n ) e ( n ) = y ( n ) (5) En = d( n) y( n) j j (6) j C he dfferentaton n equaton (5) corresponds to the backpropagaton of the global error to the output of neuron. he actvaton functon ϕ( ) s responsble for the non-lnearty n the neuron. he weght vector w of the optmum model for neuron s to be estmated through tranng wth examples. he actvaton functon s assumed to be dfferentable. Accordngly, we can use aylor seres to expand equaton (3) about the current estmate of the weght vector and thereby lnearze the equaton as follows [4]: φ( x ( nw ) ( n) ) q ( nw ) ( n) + φ x ( nw ) ( n) q ( nw ) ( n) where ( x ( n) w( n) ) φ q ( n) = = y( n) y( n) x ( n) w ( n) w ( n) = w( n) y( n) s the output of neuron that results from the use of the weght estmate. In equaton (8) we have assumed the use of the logstc functon; other sgmod functons, lke the hyperbolc tangent, can be used as well. he frst term of the rght hand sde of equaton (7) s the desred lnear term whle the remanng term represents a modelng error. hus substtutng equaton (7) and (4) n (3) and gnorng the modelng error we obtan: Where e ( n ) and (7) (8) d ( n) = q ( n) w( n) + e ( n) (9) q n are defned n equatons (5) and (8) respectvely. Equatons () and (9) descrbe the lnearzed behavor of neuron. Gven the par of equatons () and (9), we can make use of the standard Recursve Least Squares (RLS) algorthm equatons [4], whch s a specal case of the Kalman flter, to make an estmate of the weght vector w n of neuron. he resultng soluton s defned by the followng system of recursve equatons [4] that descrbe the Multple Extended Kalman Algorthm (MEKA) [4]: r( n) ( λ ) P( n ) q ( n) k( n) r( n) ( r ( n) q( n) ) w( n+ ) = w( n) + e( n) k ( n) ( ) ( λ ) = (0) = + () () P n+ = P n k n r n (3) Where, n=,,n s the teraton number and N s the total number of examples. q n represents the lnearzed neuron actvaton he vector functon gven n equaton (6), P ( n) s the current estmate of the q ( n ) nverse of the covarance matrx of and k n s the Kalman gan. he parameter λ s a forgettng factor whch takes values n the range (], and e ( n ) s the localzed measure of the global error. Equaton (3) s called the Rccatt dfference equaton. Each neuron n the network perceves ts own effectve nput P n even n the q ( n ), hence t has to mantan ts own copy of 3

3 case n whch t may share some of ts nputs wth other neurons n the network.. HE EVOLUIONARY ALGORIHM he proposed evolutonary algorthm s an mproved verson of a modfed genetc algorthm that was used aforetme by the authors. It mantans the basc workng phlosophy of evolutonary algorthms and resembles genetc programmng (see [6] for detals) snce t evolves complcated structures lke lnked lsts and not smple bt strngs as genetc algorthms do. he algorthm evolves, usng a number of genetc operators, a populaton of artfcal neural networks (multlayered perceptrons) that are represented as lnked lsts of network layers and neurons; thus t s used the drect encodng scheme. he basc steps of the algorthm are as follow:. Intalzaton: An ntal populaton of neural networks (called ndvduals) s created. Every ndvdual has a random number of neurons (or nodes) and connectons (synapses). he connecton weghts are ntalzed to some random values wthn a specfc range.. ranng: Every ndvdual (neural network) n the populaton s traned usng MEKA for a small number of tranng epochs. For populatons other than ntal, tranng occurs only for those networks that have been changed by the applcaton of genetc operators. 3. Ftness Evaluaton: As ftness functon t s used a functon that combnes the performance of the network n the tranng and/or valdaton set wth the sze of the network. he performance s evaluated usng the Mean Squared Error (MSE) or the Mean Relatve Error (MRE). Whle, the sze s the number of neurons and/or the number of actve synapses n the network. So the ftness functon for the case of MRE has a formula of the type: Ftness() = (4) + MRE + sp MRE SIZE () () () Where sp s a parameter that controls the weght of the network sze n the evaluaton of ftness, MRE() s the value of MRE of ndvdual, SIZE() s the sze of ndvdual whch can be calculated as the number of actve connectons or the number of neurons and s an ndex takng values n the range to populaton sze. 4. Selecton: Selecton operator s been used n order to create a new, ntermedate, populaton from the old one, by selectng ndvduals based on ther ftness. hs can be done usng any of the followng three dfferent selecton schemes that have been mplemented, namely: he Eltsm Roulette Wheel Selecton Operator, wth varable eltst pressure (for more detals see [], [6], [0] and []). he Rank Based Selecton (for more detals see [], [6], [0] and []). he ournament Selecton wth varable tournament sze (for more detals see [], [6], [0] and []). 5. Mutaton: It works on the members of hree dfferent mutaton operators are mplemented: Input Mutaton: t selects randomly a neural network from the populaton and changes ts number of nputs. hs operator works only on tme seres modelng and predcton problems, where the number of past values (network nputs) needed to predct future values s not usually known a prory. Hdden mutaton: t selects randomly a neural network from the populaton and changes the structure of ts hdden regon by addng or deletng a random number (selected unformly from a gven nterval) of hdden neurons. Non Unform Weght mutaton: t s responsble for the fne tunng capabltes of the system. It selects randomly a number of connecton weghts and changes ther values to new ones as follows: Let suppose that w s the old weght value then the new one s gven by the formula: ( ) (, wn+ = wn±δwtub wn) (5) Where lb and ub are the lower and upper bounds of the weght values, t s the generaton number, and Δ(t,y) s a functon that returns a value n the range [0,y], such that the probablty of Δ(t,y) beng close to 0 ncreases as t ncreases. hs property causes ths operator to search the soluton space ntally unformly (whle t s small) and very locally at the later stages. In our experments the followng functon, [0] was used: ( ) (, ) b t t y y r Δ = (6) Where r s a random number on [], s the maxmal generaton number (a parameter of the algorthm), and b s a system parameter determnng the degree of non-unformty. Gaussan weght mutaton: t works lke the Non Unform Weght mutaton operator wth the dfference that the new weght value s calculated by the formula: ( ) ( wn+ = wn+δ wn) (7) Where, Δw s a small random number followng Gaussan dstrbuton. Unform weght mutaton: t works lke the Gaussan mutaton operator wth the dfference that, Δw s a small random number followng Unform dstrbuton. 6. Crossover: It selects two parents (neural networks) and generates one or two offsprng by recombnng parts of them. he offsprng take the place of ther parents n the new populaton. In the presented algorthm crossover recombnes whole neurons wth ther ncomng connectons. But snce we have to deal wth networks wth dfferent structures, the new connectons that mght have to be produced are ntalzed wth random weght values as n the ntalzaton phase. Heren crossover works more lke a mutaton operator, lke n most genetc programmng systems, than as the recombnaton operator of genetc algorthms herefore the presented hybrd evolutonary algorthm works n bref as follows: t starts wth a populaton of randomly constructed Neural Networks (step ). Networks undergo some tranng for a couple of epochs wth MEKA, usng the tranng set (step ). Performance s measured wth the ftness functon (step 3) usng the valdaton set, n order to mprove generalzaton. hen a new, ntermedate, populaton s created, by selectng the more ft ndvduals accordng to ther ftness (step 4) usng any of the three selecton schemes. Some members of ths ntermedate populaton undergo transformatons by means of genetc operators to form the members (ndvduals) of the new populaton: mutaton (step 5) and 33 3

4 crossover (step 6) operators. he new populaton that s created s traned agan (step ); new members are traned for a couple of epochs, whle the members that have survved and passed from the old populaton may be traned wth MEKA for some more epochs, or may not be traned at all. hs s the new generaton. hs whole process contnues untl a predefned termnaton condton s fulflled; the termnaton condton mght be a maxmum number of generaton or a mnmum error (MSE or MRE) value. Once termnated the algorthm s expected to have reached a nearoptmum soluton,.e. a traned network wth near optmum archtecture..3 HE OOL hs hybrd evolutonary algorthm has been mplemented as a vsual tool n C++ programmng language, havng a graphcal user nterface (GUI). Specfcally, t was used the Borland C++ verson 6.0 IDE for Wndows. Fgure and depct two of the basc forms of the program, for the two man categores of problems that t can be used for, classfcaton and tme seres predcton. Fgure 3 s the statstcs form that llustrates the evolutonary process and prnts useful nformaton about t. Fgure. Fgure. he man form for classfcaton problems of the evolutonary neural network system he man form for predcton problems of the evolutonary neural network system he user can select between ths two problem categores. hen he/she can nsert the values of the varous genetc parameters, the tranng, valdaton and test fles, as well as the output log fles. he user can observe the evolutonary process usng some real tme graphcal dsplay of the error, the performance of the best ever network and other parameters. Fgure 3. he statstcs form 3 NUMERICAL EXPERIMENS In order to examne the ablty of the algorthm to produce networks that learn and generalze well we have tested t on two real world problems: the modelng of the MEG recordngs of epleptc patents and the modelng MCG recordngs of normal subjects. Bran dynamcs can be evaluated by recordng the changes of the neuronal electrc voltage, ether by the electroencephalogram (EEG), or by the MEG. he EEG recordngs represent the tme seres that match up to neurologcal actvty as a functon of tme. On the other hand the MEG s generated due to the tme varyng nature of the neuronal electrc actvty, snce tme-varyng electrc currents generate magnetc felds. EEG and MEG are consdered to be complementary, each one carryng a part but not the whole of the nformaton related to the underlyng neural actvty. hus, t has been suggested that the EEG s mostly related to the nter-neural electrc actvty, whereas the MEG s mostly related to the ntraneural actvty. he MEG recordngs of epleptc patents were obtaned usng a Super-conductve QUantum Interference Devce (SQUID) and were dgtzed wth a samplng frequency of 56Hz usng a -bt A/D Converter. SQUID s a very senstve magnetometer, capable to detect and record the bo-magnetc felds produced n the human bran due to the generaton of electrcal mcro-currents at neural cellular level [30]. he same stands for the MCG recordngs whch are magnetc recordngs of the heart operaton of normal subjects. MEG and MCG data were provded by the Laboratory of Medcal Physcs of the Democrtus Unversty of hrace, Greece, where a one-channel DC SQUID s operable. Both bosgnal data were normalzed n the nterval [] n order to be processed by the neural networks. In all the experments we used, for comparson reasons, the same parameter values, whch are depcted n table. For the case of the MEG modelng, as tranng set where used 04 data samples (correspondng to a four seconds epoch of the MEG) whle for the testng was used 5 data samples (correspondng to a two seconds epoch of the MEG). For the case of the MCG modelng, as tranng set where used 04 data samples and for the test set was used 04 data samples he algorthm was left to run over 000 generatons. In order to evaluate the forecastng capablty of the produced networks we used three well-known error measures, the Normalzed Root Mean Squared Error (NRMSE), the Correlaton Coeffcent (CC) and the Mean Relatve Error (MRE). he performance of the hybrd algorthm for the case of MEG modelng s depcted n tables and 3 and n fgure 4, whle for the case of MCG n tables 4 and 5 and n fgure

5 able. Parameters used for the cases of MEG and MCG modelng Parameter Value Populaton 50 Max number of Generatons 000 Crossover Probablty Input Mutaton Probablty Weght Mutaton Probablty Unform Mutaton Probablty nonunform Mutaton Probablty Gaussan Mutaton Probablty MeanGaussan StDev Gauss 0,5 Predctng Horzon able. MEG forecastng - Errors on the ranng Set Archtecture NRMSE C.C. MRE able 3. MEG forecastng - Errors on the est Set Archtecture NRMSE C.C. MRE Actual(-) vs Predcted(:) Fgure 4. MEG forecastng, performance on the test set. able 4. MCG forecastng - Errors on the ranng Set Archtecture NRMSE C.C. MRE able 5. MCG forecastng - Errors on the est Set Archtecture NRMSE C.C. MRE Actual(-) vs Predcted(:) Fgure 5. 4 CONCLUSIONS MCG forecastng, performance on the test set. In the current paper t was presented a hybrd bologcal nspred evolutonary algorthm that combnes a genetc programmng technque wth a tranng method based on the Multple Extended Kalman Algorthm. hs hybrd algorthm s mplemented n C++ as a software system wth a graphcal user nterface. he man noveltes of the proposed hybrd algorthm are the combnaton of genetc programmng technque wth MEKA, the use of a ftness functon that combnes performance wth network sze, the ablty to evolve not only the structure of the hdden layers but the number of nputs as well, and the large number of dfferent genetc operators and especally mutaton operators that have been mplemented. Another novelty s the representaton used for neural networks. As sad before, every network n the populaton s represented as a lnk lst of layers and neurons, usng the drect encodng scheme. he use of lnk lsts has some certan advantages that have to do manly wth the memory management; you use only the memory that s needed every tme and you don t have to allocate a maxmum memory sze, for maxmum network sze lke other representaton schemes. Moreover lnk lsts are dynamc data structures, whch t means that the neural network archtecture can change dynamcally durng run tme n contrast wth other data structures lke matrces that n C++ can not change durng run tme. hs hybrd algorthm was used for the modelng of two bologcal tme seres, namely the Magneto Encephalogram (MEG) recordngs of epleptc patents and Magneto Cardogram (MCG) of normal subjects. All the reported cases refer to predctons on recordngs of the dynamcs of nonlnear systems. In all the performed experments the algorthm was able to fnd a near optmum network archtecture that gave small predcton errors. herefore we can conclude that the algorthm s able to produce small and compact networks that learn and generalze well. he algorthm has only tested on tme seres predcton problems and t s n our ntenton to test t on some dffcult classfcaton problems as well. 35 5

6 One of the man drawbacks of ths knd of algorthms, namely the evolutonary algorthms, hybrd or not, s that they are computatonal expensve n terms of computer memory and CPU tme. Even though the proposed algorthm belongs to ths category, the use of MEKA for just a couple of epochs for the tranng phase of the neural networks and the representaton where each member of the populaton s a network represented as a lnk lst so that there s no need to use encodng and decodng functons for the calculaton of network s performance, makes the algorthm less computatonal expensve than other approaches to the same problem of neural networks evoluton. he algorthm could be further mproved by addng some more genetc operators for better and faster local search both to the archtecture and weght space and ths s gong to be one of our future research targets. Furthermore, n the ntegrated software system there are already mplemented a large number of genetc operators whose nfluence to the performance of the hybrd algorthm needs to be apprased; we need to see whch of the three selecton schemes, or the many mutaton operators gve better results. Another future research drecton wll be the combnaton of MEKA wth other evolutonary technques lke Partcle Swarm Optmzaton and Dfferental Evoluton for neural network evoluton. REFERENCES [] Adamopoulos, A., Andreou, A., Georgopoulos, E., Ioannou, N. and Lkothanasss, S., Currency Forecastng Usng Recurrently RBF Networks Optmzed by Genetc Algorthms, Computatonal Fnance 997 (CF 97), London Busness School, 997. [] Adamopoulos, A., Annnos, P., Lkothanasss, S., and Georgopoulos, E. On the Predctablty of MEG of Epleptc Patents Usng RBF Networks Evolved wth Genetc Algorthms, BIOSIGNAL 98, Brno, Czech Republc, June 3-5, 998a. [3] Adamopoulos, A., Georgopoulos E., Manoudaks, G. and Lkothanasss, S. An Evolutonary Method for System Structure Identfcaton Usng Neural Networks Neural Computaton 98, 998b. [4] Adamopoulos, A., G. Georgopoulos, S. Lkothanasss and P. Annnos, Forecastng the MagnetoEngephaloGram (MEG) of Epleptc Patent Usng Genetcally Optmzed Neural Networks, Genetc and Evolutonary Computaton Conference (GECCO 99), Orlando, Florda USA, July 4-7, 999 [5] Andreou, A., Georgopoulos, E., and Lkothanasss, S., and Poldoropoulos, P., Is the Greek foregn exchange-rate market predctable? A comparatve study usng chaotc dynamcs and neural networks, Proceedngs of the Fourth Internatonal Conference on Forecastng Fnancal Markets, Banque Natonale de Pars and Imperal College, London, 997. [6] Andreou, A., Georgopoulos, E., Zombanaks, G. and Lkothanasss, S., estng Currency Predctablty Usng An Evolutonary Neural Network Model, Proceedngs of the ffth Internatonal Conference on Forecastng Fnancal Markets, Banque Natonale de Pars and Imperal College, London, 998. [7] Andreou A., Georgopoulos E. and Lkothanasss, S. Exchange Rates Forecastng: A Hybrd Algorthm Based On Genetcally Optmzed Adaptve Neural Networks, Computatonal Economcs Journal, Kluwer Academc Publshers, vol. 0, ssue 3, pp. 9 0, December 00 [8] Bllngs, S. A., and Zheng, G. L. Radal bass functon network confguraton usng genetc algorthms. Neural Networks, Vol. 8, pp , 995. [9] Bornholdt S. and Graudenz, D. General asymmetrc neural networks and structure desgn by genetc algorthms. Neural Networks, Vol. 5, pp37 334, 99. [0] Davs, L. Mappng classfer systems nto neural networks. Proceedngs of the 988 Conference on Neural Informaton Processng Systems, Morgan Kaufmann, 988. [] Georgopoulos E., Lkothanasss S. and Adamopoulos A., Evolvng Artfcal Neural Networks Usng Genetc Algorthms, Neural Network World, 4/00, pp , 000. [] Goldberg, D. Genetc Algorthms n Search Optmzaton & Machne Learnng, Addson-Wesley 989. [3] Happel, B., et al. Desgn and evoluton of modular neural network archtectures. Neural Networks, Vol. 7, pp , 994. [4] Haykn, S., Neural Networks - A Comprehensve Foundaton, McMllan College Publshng Company, ch. 6, p.3, New York, 994. [5] Holland, J. Adaptaton n Natural and Artfcal Systems, MI press 99. [6] Koza J.R., Genetc programmng: on the programmng of computers by means of natural selecton, MI Press, Cambrdge, MA, 99. [7] Lkothanasss S. Georgopoulos E. and Fotaks D. (997). Optmzng the Structure of Neural Networks Usng Evoluton echnques. 5th Internatonal Conference on Applcatons of Hgh - Performance Computers n Engneerng, pp , Santago de Compostela, Span, July. [8] Lkothanasss, S. D., Georgopoulos, E. F., Manoudaks, G. D. and Adamopoulos, A.V., Currency Forecastng Usng Genetcally Optmzed Neural Networks, HERCMA Athens September 998. [9] Lkothanasss, S. D., Georgopoulos, E. F. A Novel Method for the Evoluton of Neural Networks, 3 rd IMACS/IEEE Internatonal Conference on Crcuts Systems and Computers (CSC 99), July 999. [0] Mchalewcz, Z., Genetc Algorthms + Data Structures = Evoluton Programs, Sprnger-Verlag, 996. [] Mller, G., et al. Desgnng neural networks usng genetc algorthms. Proceedngs of the 3 rd Internatonal Conference on Genetc Algorthms, Morgan Kaufmann 989. [] Mtchell M. An Introducton to Genetc Algorthms, MI Press 996. [3] Montana, D. and Davs, L. ranng feedforward neural networks usng genetc algorthms. BBN Systems and echnologes, Cambrdge, MA 989. [4] Shah, S., Palmer, F. and Datum, M., Optmal Flterng Algorthms for Fast Learnng n Feed-Forward Neural Networks, Neural Networks, Vol. 5, pp , 99. [5] Whtley, D. Applyng genetc algorthms to neural network problems, Internatonal Neural Networks Socety, p [6] Whtley, D., and Bogart, C. he evoluton of connectvty: Prunng neural networks usng genetc algorthms. Internatonal Jont Conference on Neural Networks, Washngton D.C.,. Hllsdale, NJ: Lawpence Erlbaum, pp , 990. [7] Whtley, D., and Hanson,. Optmzng neural networks usng faster, more accurate genetc search. 3 rd Intern. Conference on Genetc Algorthms, Washngton D.C., Morgan Kaufmann, pp , 989. [8] Yao, X. & Lu, Y. A New Evolutonary System for Evolvng Artfcal Neural Networks, IEEE ransactons on Neural Networks, vol. 8, no. 3, 997. [9] Yao, X. Evolvng Artfcal Neural Networks, Proceedngs of the IEEE, 87(9):43:447, September 999. [30] Annnos, P. Jacobson, J. sagas, N. Adamopoulos, A. Spatotemporal Statonarty of Epleptc Focal Actvty Evaluated by Analyzng Magneto Encephalographc (MEG) data and the heoretcal Implcatons. Panmnerva Med. 39, 89-0, 997. [3] Granng, L.; Yaochu Jn; Sendhoff, B.: Generalzaton Improvement n Mult-Objectve Learnng, Neural Networks, IJCNN apos;06. Internatonal Jont Conference on Volume, Issue, Page(s): , 006. [3] Vera, D. A. G. and J. A. Vasconcelos and W. M. Camnhas: Controllng the parallel layer perceptron complexty usng a multobjectve learnng algorthm, Neural Computng and Applcatons, vol. 6, n. 4, p.p ,