A Comprison of Artificil Neurl Networks Algorithms for Short Term Lod Forecsting in Greek Intercontinentl Power System G.J. TSEKOURAS, F.D. KANELLOS, V.T. KONTARGYRI, C.D. TSIREKIS, I.S. KARANASIOU, CH. N. ELIAS, A.D. SALIS, N. E. MASTORAKIS Deprtment of Electricl & Computer Science, School of Electricl nd Computer Engineering, Hellenic Nvl Acdemy Ntionl Technicl University of Athens Term Htzikyrikou, Pireus 9 Heroon Polytechniou Street, Zogrfou, Athens GREECE Emil: tsekours_george_j@yhoo.gr, knellos@mil.ntu.gr, vkont@centrl.ntu.gr, ktsirekis@desmie.gr, ikrn@esd.ece.ntu.gr, xelis@hlk.forthnet.gr, nstsios.slis@gmil.com, mstor@wses.org Abstrct: - The objective of this pper is to compre the performnce of different Artificil Neurl Network (ANN) trining lgorithms regrding the prediction of the hourly lod demnd of the next dy in intercontinentl Greek power system. These techniques re: () stochstic trining process nd (b) btch process with (i) constnt lerning rte, (ii) decresing functions of lerning rte nd momentum term, (iii) dptive rules of lerning rte nd momentum term, (c) conjugte grdient lgorithm with (i) Fletcher-Reeves eqution, (ii) Fletcher-Reeves eqution nd Powell-Bele restrt, (iii) Polk-Ribiere eqution, (iv) Polk- Ribiere eqution nd Powell-Bele restrt, (d) scled conjugte grdient lgorithm, (e) resilient lgorithm, (f) qusi-newton lgorithm, (g) Levenberg-Mrqurdt lgorithm. Three types of input vribles re used s inputs: () historicl lods, (b) wether relted inputs, (c) hour nd dy indictors. The trining set is consisted of the ctul historicl dt from three pst yers of the Greek power system. For ech ANN trining lgorithm clibrtion process is conducted regrding the crucil prmeters vlues, such s the number of neurons, etc. The performnce of ech lgorithm is evluted by the Men Absolute Percentge Error (MAPE) between the experimentl nd estimted vlues of the hourly lod demnd of the next dy for the evlution set in order to specify the ANN with the smllest vlue. Finlly the lod demnd for the next dy of the test set (with the historicl dt of the current yer) is estimted using the best ANN of ech trining lgorithm, so tht the verifiction of behviour of ANN lod prediction techniques should be demonstrted. Key-Words: - rtificil neurl networks, short-term lod forecsting, ANN trining bck-propgtion lgorithms Introduction In deregulted electricl energy mrket, the lod demnd hs to be predicted with the highest possible precision in different time periods: very short-term (for the next few minutes), short-term (for the next few hours to week), midterm (for the next few weeks to few months) nd long-term (for the next few months to yers). Especilly, short lod forecsting is very crucil problem, becuse its ccurcy effects to other opertionl issues of power systems, such s unit commitment [], scheduling of spinning reserve [], vilble trnsfer cpbility [3], system stbility [3], ppliction of lod demnd progrms, etc. Accurte forecsting leds to higher relibility nd lower opertionl costs for power systems. Severl forecsting methods hve been implemented for short-term lod forecsting with different levels of success, such s ARMAX models [4], regression [5], ANNs [6], fuzzy logic [7], expert systems etc. Despite the diversity of methods used, ANNs re the most common nd effective [6]. Especilly, in Greece, ANNs hve been used successfully either for the intercontinentl power system [8-], or utonomous big islnds [8, -]. Some techniques belong to clssicl ANNs [9-] or specilized ones (i.e. prllel implementtion of recurrent ANN nd zero-order regultion rdil bsis networks []) or they re combined with fuzzy logic lgorithms []. In this pper the comprison of 4 different bsic trining ANN lgorithms is crried out bsed on the bsic structure of the ANN proposed by Kirtzis et l [8-9] for the inputs nd outputs neurons for the Greek intercontinentl power system. The min gols re: the modultion of the internl neurl network structure (number of neurons of hidden lyer, initil vlue of lerning rte, etc.) for ech different trining lgorithm with respect to the best Men Absolute Percentge Error (MAPE) of the evlution set, the comprison of the respective lgorithms ccording to MAPE nd computtionl time nd ISSN: 79-55 8 ISBN: 978-96-474-36-9
the suggestion of the best trining lgorithm for this cse study. The respective results re bsed on ctul hour lod dt of the Greek intercontinentl power system for yers 997-. Proposed ANN Methodology for Ech Trining Algorithm for Shortterm Lod Forecsting The short-term lod forecsting is chieved by pplying n ANN methodology through the proper selection of the prmeters for ech trining bckpropgtion lgorithm. This methodology hs the following bsic steps nd its flow chrt is shown in Figure. Min Procedure No Dt selection Dt preprocessing ANN prmeters set ANN prmeters combintion Trining Process Evlution Process End of trils? Yes Selection of the respective prmeters with the best MAPE for the Evlution Set Finl Estimtion for Test Set Fig.. Flowchrt of the ANN methodology for the proper selection of ANN prmeters per trining lgorithm for short-term lod forecsting () Dt selection: The input vribles for lod forecsting of d-th dy re the following ccording to Kirtzis et l [9]: the hourly ctul lods of the two previous dys: L(d-,),, L(d-,4), L(d-,),, L(d-,4) (in MW), the mximum men temperture per three hours nd the minimum men temperture per three hours for Athens for the current dy nd the lst dy mx_temp Ath (d), min_temp Ath (d), mx_temp Ath (d-), min_temp Ath (d-) respectively (in o C), the mximum men temperture per three hours nd the minimum men temperture per three hours for Thesslonic for the current dy nd the lst dy mx_temp Th (d), min_temp Th (d), mx_temp Th (d-), min_temp Th (d-) respectively (in o C), the temperture difference between the mximum men temperture per three hours of the current dy nd the respective one of the lst dy for Athens dif_temp Ath nd Thesslonic dif_temp Th respectively: dif_temp Ath = mx_temp Ath (d)- mx_temp Ath (d-) () dif_temp Th = mx_temp Th (d)- mx_temp Th (d-) () the temperture dispersion from comfortble living conditions temperture for Athens for the current dy T Ath ( d ) nd the previous dy T Ath ( d ), where: ( Tc T), T < Tc Tdispersion =, Tc < T < Th (3) ( T Th), Th < T where T c =8 o C, T h =5 o C. the temperture dispersion from comfortble living conditions temperture for Thesslonic for the current dy T Th ( d ) nd the previous dy T Th ( d ), seven digit numbers, which express the kind of the week dy, where Mondy corresponds to, Tuesdy to, etc, two sinusoidl functions ( cos( π d / T), sin( π d / T) ), which express the sesonl behvior of the current dy, where T is the number of the dys of the current yer. The output vribles re the 4 hourly ctul lod demnd of the current dy L ) (d,),, L ) (d,4). (b) Dt preprocessing: Dt re exmined for normlity, in order to modify or delete the vlues tht re obviously wrong (noise suppression). Due to the gret non linerity of the problem, non liner ctivtion functions re preferbly used. In tht cse, sturtion problems my occur. These problems cn be ttributed to the use of sigmoid ctivtion functions tht present non-liner behvior outside the region [-, ]. In order to void sturtion problems, the input nd the output vlues re normlized s shown by the following expression: b xˆ = + ( x xmin) (4) xmx xmin where ˆx is the normlized vlue for vrible x, x min nd x mx re the lower nd the upper vlues of vrible x, nd b re the respective vlues of the ISSN: 79-55 9 ISBN: 978-96-474-36-9
normlized vrible. (c) Min procedure: For ech ANN trining lgorithm the respective prmeters of the neurl network re selected through set of trils. Specificlly for ech ANN prmeter (such s the neurons of the hidden lyer) the trining lgorithm is seprtely executed for the respective rnge of vlues (i.e. to 7 neurons with step ) bsed on the error function (sum of the squre of errors for ll neurons per epoch) for trining set nd the regions with stisfctory results (minimum MAPE for evlution set) re identified. Following, the trining lgorithm is repetedly executed, while ll prmeters re simultneously djusted into their respective regions, so tht the combintion with the smllest MAPE for the evlution set is selected. It is noted tht the MAPE index between the mesured nd the estimted vlues of hourly lod demnd for the evlution set dys is given by: ) m 4 ev L( d, i) L( d, i) MAPEev = % m (5) d= i= L d, i ev ( ) where L(d,i) is the mesured vlue of lod demnd for the i-th hour of d-th dy for the evlution set, L ) (d,i) the respective estimted vlue, m ev the popultion of the evlution set. This index is prcticl mesure, which reflects the pproximtion of the ctul lod demnd independently from its units. (d) Finl estimtion for the test set: The ctul lod demnd (in MW) for the dys of the test set is finlly estimted by using the respective ANN prmeters of the current trining lgorithm. This process is repeted for 4 different trining lgorithms, which re synopsized in Tble. A short description of ll trining lgorithms is presented in [3], while more nlyticl representtions cn be found in references of Tble [4-3]. The bsic steps of the bck-propgtion lgorithm hve been described in severl textbooks [4-5]. According to Kolmogorov s theorem [4], n ANN cn solve problem by using one hidden lyer, provided it the proper number of neurons. Under these circumstnces one hidden lyer is used, but the number of neurons must properly be selected properly. It is lso mentioned tht there re severl prmeters to be selected, depending on the vrition of the bck-propgtion lgorithm tht is being used ech time in order to trin the ANN. The prmeters tht re common in ll lgorithms re: the number of neurons N n in the hidden lyer, the type of the ctivtion functions (hyperbolic tngent, logistic, liner), the prmeters (, b) of the ctivtion functions, φ x = tn x+ b for hyperbolic tngent nd i.e. ( ) ( ) the mximum number of epochs (mx_epochs). For methods -6 (see Tble ) the dditionl prmeters re: the time prmeter nd the initil vlue of the lerning rte T η, η, the time prmeter nd the initil vlue of the momentum term T, (not for methods 3, 4). For methods 7- (see Tble ) the dditionl prmeters re the initil vlue s, the number of itertions T bn for the step of the bsic vector s clcultion, the number of itertions T trix for the trisection step using the golden method for minimizing the error function. TABLE DESCRIPTION OF ANN S TRAINING ALGORITHMS No. Description of ANN s trining lgorithms Stochstic trining with lerning rte nd momentum term (decresing exponentil functions) [4] Stochstic trining, use of dptive rules for the lerning rte nd the momentum term [4] 3 Stochstic trining, constnt lerning rte [4] 4 Btch mode, constnt lerning rte [4] 5 Btch mode with lerning rte nd momentum term (decresing exponentil functions) [4] 6 Btch mode, use of dptive rules for the lerning rte nd the momentum term [4] 7 Btch mode, conjugte grdient lgorithm with Fletcher-Reeves eqution [5-6] 8 Btch mode, conjugte grdient lgorithm with Fletcher-Reeves eqution & Powell-Bele restrt [5-7] 9 Btch mode, conjugte grdient lgorithm with Polk-Ribiere eqution [5, 8] Btch mode, conjugte grdient lgorithm with Polk-Ribiere eqution & Powell-Bele restrt [5, 7-8] Btch mode, scled conjugte grdient lgorithm [9] Btch mode, resilient lgorithm [] 3 Btch mode, qusi-newton lgorithm [] 4 Btch mode, Levenberg-Mrqurdt lgorithm [-3] ISSN: 79-55 ISBN: 978-96-474-36-9
For method the dditionl prmeters re σ nd λ, for method the incresing δ nd the decresing δ fctor of chnge in the vlue of the weights between two successive epochs nd for λ nd the method 4 the initil fctor ( ) multiplictive prmeter β respectively. It is lso noted tht during the trining process for ech ANN three stopping criterions re used [5]: weights stbiliztion (smller thn l imit ), the respective error function not to be improved (the vrition between two epochs should be smller thn l imit ) or the mximum number of epochs to be exceeded (bigger thn mx_ epochs ). In ech trining lgorithm, the error function is the root men squre error RMSE tr for the trining set ccording to: where out lyer, e ( ) m q out k ( ) (6) out m k RMSEtr = e m m q = = q is the number of neurons of the output k m is the error of the k-th output neuron for the m-th pttern of the trining set. If one of the three criteri is stisfied, the min core of bck propgtion lgorithm finishes. Otherwise, the number of epochs is incresed by one nd the feed forwrd nd reverse pss clcultions re repeted. Afterwrds, the results of ll ANN trining lgorithms with the respective optimized prmeters re compred, in order to choose the one leding to the smllest MAPE index within logicl computtionl time. 3 Appliction for Stochstic Trining Algorithm with Decresing Exponentil Functions for Lerning Rte nd Momentum Term Following, the forementioned method is pplied for the short-term lod forecsting in Greek intercontinentl power system. The trining nd the evlution sets consist of the 9% nd % of the norml dys (no holidys) from the yers 997-999 respectively, while the respective test set consists of the norml dys from the yer. The input r vector xin ( n) is formed with the 7 input vribles of section, where the lod nd the temperture r dt re normlized, while the output vector t ( n) is formed by the normlized 4 output ctul lod demnd of the dy under prediction. There re severl crucil ANN prmeters to be selected, such s: the number of the neurons of the hidden lyer, which rnges from to 7 with incrementl step, the initil vlue η = η() nd the time prmeter T η of the trining rte, which get vlues from the sets {.,.,,.9} nd {,,, } respectively, the initil vlue = () nd the time prmeter T of the momentum term, which get vlues from the sets {.,.,,.9} nd {,,, } respectively, the type nd the prmeters of the ctivtion functions of the hidden nd the output lyers, where the type cn be hyperbolic tngent, liner or logistic, while the, prmeters get vlues from the set {.,.,,.5} nd b, b from the set {., ±.,±.}. The prmeters of the stopping criteri re defined fter few trils s mx_ epochs =, l imit = -5, l imit = -5. The development of the bovementioned method in Visul Fortrn 6. gives the cpbility to relize ll possible combintions of the vlues of the crucil prmeters. In this study the respective combintions ccount to 836,57,5, which prcticlly cn not be exmined. This forced the uthors to pply the proposed clibrtion process grdully through consecutive steps in order to determine the vlues of the ANN s prmeters. As first step, the number of neurons vries from to 7, while the remining prmeters re ssigned with fixed vlues ( =.4, T = 8, η =.5, T η =, ctivtion functions in both lyers: hyperbolic tngent, = =.5, b =b =.). In Fig. the MAPE indexes for the trining, the evlution nd the test set re presented. The MAPE indexes of the evlution nd the test set keep step with the respective one of the trining set, even if the following reltionship is vlid for every neuron (from to 7): MAPE < MAPE < MAPE (7) trining evlution test With the neurons numbered from to 45 the MAPE index for the evlution set hs smll vlues (the smllest is for 45), while for bigger vlues it rpidly increses. As second step the initil vlue η nd the time prmeter T η of the trining rte chnge simultneously in the respective regions, while the other prmeters remin constnt (neurons=45, =.5, T = 8, ctivtion functions in hidden ISSN: 79-55 ISBN: 978-96-474-36-9
& output lyers: hyperbolic tngent, = =.5, b =b =.). In Fig. 3 the results of the MAPE index for the evlution set re stisfctory for.5 η.8 nd 4. The best result is obtined for η =.5, T η =. It is mentioned T η tht MAPE increses drmticlly for η.7 nd T η 6. As third step the initil vlue nd the time prmeter T of the momentum term chnge simultneously in the respective regions, while the other prmeters re constnt (neurons=45, η =.5, T η =, ctivtion functions in hidden & output lyers: hyperbolic tngent, = =.5, b =b =.). The results of the MAPE index for the evlution set re stisfctory for.6 nd T 6, while the best result is obtined for =.9, T =. It is mentioned tht MAPE increses drmticlly for.5. Similrly it is found tht the ANN gives better results using s hyperbolic tngent ctivtion function in both lyers with prmeters = =.5 nd b = b =.. It is mentioned tht in Tble the results for different ctivtion functions re registered. The finl clibrtion of the ANN model is relized for 4 to 5 neurons, =.8.9, T = 8--, η =.5.6, T η = -- 4, ctivtion functions in both lyers: hyperbolic tngent with prmeters = =.-.5-.3, b = b =. Μέσο Απόλυτο Εκατοστιαίο Σφάλµα ΜΑΡΕ (%) MAPE (%)..9.8.7.6.5.4.3 5 3 35 4 45 5 55 6 65 7 Neurons Νευρώνες Σύνολο Trining εκπαίδευσης set Σύνολο Evlution αξιολόγησης set Σύνολο Test set ελέγχου Fig.. MAPE(%) index for the ll sets, neurons: -7, =.4, T = 8, η =.5, T η =, ctivtion functions in both lyers: hyperbolic tngent, = =.5, b =b =. MAPE of evlution set MAPE of vlidtion set 3.5.5 8 6 4 Time prmeter of trining rte Time prmeter of trining rte Fig. 3. MAPE(%) index for the evlution set, η = {.,.,...,.9}, {,,..., } T η =, neurons: 45, =.4, T = 8, ctivtion functions in both lyers: hyperbolic tngent, = =.5, b =b =...4.6.8 Initil vlue of trining rte Initil vlue of trining rte ISSN: 79-55 ISBN: 978-96-474-36-9
The best result for the MAPE index of the evlution set is.48% nd is obtined for n ANN with 45 neurons in the hidden lyer, =.9, T =, η =.5, T η =, = =.5 nd b = b = using hyperbolic tngent ctivtion function in both lyers. 4 Appliction for the Set of 4 ANN s Trining Algorithms Following, the proposed methodology of section is pplied to ech trining lgorithm for the short-term lod forecsting in Greek intercontinentl power system properly selected ANN s prmeters (see Tble 3). The trining, the evlution nd the test sets re the sme with the respective ones of section 3. The results of ll trining lgorithms re registered in Tble 4. The best results of MAPE for evlution set re given by the stochstic trining lgorithm with the use of dptive rules for the lerning rte nd the momentum term nd by the scled conjugte grdient lgorithm. The respective results of MAPE for the test set re stisfied, even if the stochstic trining lgorithm with the use of decresing functions for the lerning rte nd the momentum term presents slightly better results. But the scled conjugte grdient lgorithm lso presents very good generliztion of the results for the test set, becuse the respective MAPE hs the second smllest vlue. The nlogy of the respective computtionl time (with the proper prmeters clibrtion) for the first trining lgorithms is: 3. 3. 4. 3.5.4. Finlly, the scled conjugte grdient lgorithm is proposed to be used. In Fig. 4 the estimted nd the mesured chronologicl lod curves on Thursdy, June 8,, re presented indictively, where the respective MAPE equls to.73%. Lod (MW) 75 7 65 6 55 5 45 4 35 Estimted lod Mesured lod 4 8 6 4 Hours Fig. 4. Chronologicl lod curve (mesured nd estimted vlues) for dy of the test set (Thursdy, June 8, ) 5 Conclusions This pper compres the performnce of 4 different Artificil Neurl Network (ANN) trining lgorithms (see Tble ) during the prediction of the hourly lod demnd of the next dy in Greek intercontinentl power system. The structure of the input nd the output neurons (7 nd 4 respectively) re determined by Kirzis et l [9]. The rest prmeters, such s number of neurons of the hidden lyer, ctivtion functions, weighting fctors, lerning rte, momentum term, etc, re determined by the proposed clibrtion methodology of section through n extensive serch. The performnce of ech lgorithm is mesured by the Men Absolute Percentge Error (MAPE) for the evlution set. Finlly, the scled conjugte grdient trining lgorithm is proposed, becuse of its smll vlues of MAPE nd the smllest computtionl time. In future the proposed methodology cn be improved (i) by using different kinds of input nd outputs, (ii) by using compression techniques for inputs, (iii) by estimting the optimiztion process nd (iv) by determining the confidence intervls of the under prediction chronologicl lod curves. TABLE MAPE(%) OF (A) TRAINING SET, (B) EVALUATION SET, (C) TEST SET FOR DIFFERENT ACTIVATION FUNCTIONS FOR NEURONS: 45, =.4, T = 8, η =.5, T η =, = =.5, b = b =. Activtion function of output lyer Activtion function of hidden lyer Hyperbolic sigmoid Hyperbolic tngent Liner (A) (B) (C) (A) (B) (C) (A) (B) (C) Hyperbolic sigmoid.3.48.65.5.6.87.788.85.9 Hyperbolic tngent.67.737.4.383.48.749.9.987. Liner.63.69.93.39.5.747.936.3.94 ISSN: 79-55 3 ISBN: 978-96-474-36-9
TABLE 3 VALUES INTERVAL DURING THE OPTIMIZATION PROCESS OF EACH PARAMETER OF EVERY ANN TRAINING ALGORITHM No. Vlues intervls of ech prmeter of the respective ANN trining lgorithm (see Tble ) - α =.,.,,.9, Τ α =,,,, η =.,.,,.9, Τ η =,,, 3 η =.,.,,.,.,,3 4 η =.,.,,3 5-6 α =.,.,,.9, Τ α =,5,,6, η =.,.,,.9, Τ η =,5,,6 7, 9 s=.4,.,., T bv =, 4, T trix =5,, e trix = -6, -5 8, s=.4,.,., T bv =, 4, T trix =5,, e trix = -6, -5, lim orthogonlity =.,.5,.9 σ= -3, -4, -5, λ = -6, -7, 5-8 δ =.,.,,.5, δ =,,.,, 3 - λ =, β =,3,...,9,,,...,5 4 ( ).,.,...,,,...,5 Common N n =,,...,7, ctivtion function for hidden nd output lyer = hyperbolic tngent, liner, logistic, =.,.,,.5, =.,.,,.5, b =., ±.,±., b =., ±.,±. TABLE 4 MAPE(%) OF TRAINING, EVALUATION & TEST SETS FOR 4 DIFFERENT ANN TRAINING ALGORITHMS WITH THE RESPECTIVE PROPERLY CALIBRATED PARAMETERS No. Of trining lgorithm MAPE(%) of trining set MAPE(%) of evlution set MAPE(%) of test set Neurons Activtion functions Rest Prmeters.383.48.749 45.3.475.89 3 3.37.489.833 48 4.356.96.6 48 5.3.94.783 5 6.9.6.475 7.798.83.47 43 8.544.595 3.39 43 9.545.6 3.35 43.544.6 3.35 43.94.487.78 5 f =tnh(.5x), f ο=tnh(.5x) f =tnh(.4x), f ο=/(+exp(-.5x)) f =tnh(.5x), f ο=tnh(.5x) f =tnh(.4x), f ο=.4x f =tnh(.5x), f ο=.5x f =tnh(.5x), f ο=.5x f =tnh(.4x), f ο=.x f =tnh(.4x), f ο=.x f =tnh(.4x), f ο=.x f =tnh(.4x), f ο=.x f =tnh(.5x), f ο=tnh(.5x) -4 # # # # # No convergence α =.4, Τ α=8, η =.5, Τ η=, e= -5 mx_epochs= α =.7, Τ α=8, η =.5, Τ η=3, e= -5, mx_epochs= η =., e= -5, mx_epochs= η =., e= -5, mx_epochs= α =.9, Τ α=6, η =.9, Τ η=6, e= -5, mx_epochs= α =.9, Τ α=6, η =.9, Τ η=6, e= -5, mx_epochs= s=.4, T bv =, T trix =5, e= -5, mx_epochs=5 s=.4, T bv =, T trix =5, e= -5, mx_epochs=5, lim orthogonlity=.9 s=.4, T bv =, T trix =5, e= -5, mx_epochs=5 s=.4, T bv =, T trix =5, e= -5, mx_epochs=5, lim orthogonlity=.9 σ= -5, λ =5x -8, e= -5, mx_epochs= Acknowledgements The uthors wnt to express their sincere grtitude to C. Anstsopoulos, D. Voumboulkis nd P. Eustthiou from PPC for the supply of ll the necessry dt for this ppliction. References: [] B.F. Hobbs, S. Jitprpikulsrn, D.J. Mrtukulm. Anlysis of the vlue for unit commitment of improved lod forecsts. IEEE Trn. on Power Systems, Vol. 4, No.4, 999, pp. 34-48. ISSN: 79-55 4 ISBN: 978-96-474-36-9
[] H.B. Gooi, D.P. Mendes, K.R.W. Bell, D.S. Kirschen. Optiml scheduling of spinning reserve. IEEE Trn. on Power Systems, Vol. 4, No.4, 999, pp. 485-49. [3] M.H. Grvener, C. Nwnkp. Avilble trnsfer cpbility nd first order sensitivity. IEEE Trn. on Power Systems, Vol. 4, No., 999, pp. 5-58. [4] H.T. Yng, C.M. Hung, C.L. Hung. Identifiction of ARMAX model for short term lod forecsting: n evolutionry progrmming pproch. IEEE Trns. on Power Systems, Vol., No., 996, pp. 43 48. [5] T. Hid, S. Muto. Regression bsed pek lod forecsting using trnsformtion technique, IEEE Trns. on Power Systems, Vol. 9, No. 4, 994, pp. 788 794. [6] H.S. Hippert, C.E. Pedreir, R.C. Souz. Neurl networks for short-term lod forecsting: A review nd evlution. IEEE Trns. on Power Systems, Vol. 6, No.,, pp. 44-55. [7] P.A.Mstorocosts, J.B.Theochris, A.G. Bkirtzis. Fuzzy modeling for short-term lod forecsting using the orthogonl lest squres method. IEEE Trns. on Power Systems, Vol. 4, No., 999, pp. 9 36. [8] S.J.Kirtzis. Artificil Intelligence Applictions on Short-term Lod Forecsting. Disserttion, Thessloniki, A.U.Th., 998 (in greek). [9] A.G. Bkirtzis, V. Petridis, S.J. Kirtzis, M.C. Alexidis, A.H. Missis. A neurl network short term lod forecsting model for the Greek power system. IEEE Trns. on Power Systems, Vol., No., 996, pp. 858-863. [] S.J. Kirtzis, C.E. Zourns, J.M. Theochris, A.G. Bkirtzis, V. Petridis. Short term lod forecsting in n utonomous power system using rtificil neurl networks. IEEE Trns. on Power Systems, Vol., No.4, 997, pp. 59-596. [] K. Klitzkis, G.S. Stvrkkis, E.M. Angnostkis. Short-term lod forecsting bsed on rtificil neurl networks prllel implementtion. Electric Power Systems Reserch, Vol. 63,, pp. 85-96. [] A.G. Bkirtzis, J.B. Theochris, S.J.Kirtzis, K.J.Stsios. Short term lod forecsting using fuzzy neurl networks. IEEE Trns. on Power Systems, Vol., No. 3, 995, pp. 58-54. [3] G.H. Asimkopoulou, V.T. Kontrgyri, G. J. Tsekours, F.H. Asimkopoulou, I.F. Gonos, I.A. Stthopulos. Artificil Neurl Network Optimiztion Methodology for the Estimtion of the Criticl Flshover Voltge on Insultors. IET Science, Mesurement & Technology, to be printed. [4] P.S. Ghosh, S. Chkrvorti, N. Chtterjee. Estimtion of Time-to-flshover Chrcteristics of Contminted Electrolytic Surfces using Neurl Network. IEEE Trns. on Dielectrics nd Electricl Insultion, Vol., No. 6, 995, pp. 64-74. [5] L.M. Sini, M.K. Soni. Artificil neurl network-bsed pek lod forecsting using conjugte grdient methods. IEEE Trns. on Power Systems, Vol. 7, No 3,, pp.97-9. [6] R. Fletcher, C.M. Reeves. Function minimiztion by conjugte grdients, Computer Journl, Vol. 7, 964, pp. 49-54. [7] M.J. Powell. Restrt procedures for the conjugte grdient method. Mthemticl Progrmming, Vol., 977, pp. 4-54. [8] E. Polk. Computtionl methods in optimiztion: unified pproch. Acdemic Press, New York, 97 [9] M.F. Moller. A scled conjugte grdient lgorithm for fst supervised lerning. Neurl Networks, Vol. 6, 993, pp. 55-533. [] M. Riedmiller, H. Brun. A direct dptive method for fster bckpropgtion lerning: The RPROP lgorithm. Proceedings of the IEEE Interntionl Conference on Neurl Networks, Sn Frncisco, Vol., Mrch 993, pp. 586-59. [] R. Bttiti. First nd second order methods for lerning: Between steepest descent nd Newton's method. Neurl Computtion, Vol. 4, No., 99, pp. 4-66. [] K. Levenberg. A method for the solution of certin problems in lest squres. The Qurterly of Applied Mthemtics, Vol., 944, pp. 64-68. [3] D. Mrqurdt. An lgorithm for lest squres estimtion of nonliner prmeters. SIAM Journl Appliction Mthemtics, Vol., 963, pp. 43-44. [4] S. Hykin. Neurl Networks: A Comprehensive Foundtion. Prentice Hll, 994. [5] G. J. Tsekours, J. Koukoulis, M.-A. Nikolinkou, N.E. Mstorkis. Prediction of fce settlement during tunneling excvtion using rtificil neurl network. WSEAS Interntionl Conference on Engineering Mechnics, Structures, Engineering Geology (EMESEG '8), Herklion, Crete Islnd, Greece, July -5, 8. ISSN: 79-55 5 ISBN: 978-96-474-36-9