th year, No., Computational Intelligence in Electrical Engineering,

Transcription

1 1 Applcaton of hybrd neural networks combned wth comprehensve learnng partcle swarm optmzaton to shortterm load forecastng Mohammadreza Emarat 1, Farshd Keyna 2, Alreza Askarzadeh 3 1 PhD Student, Department of Electrcal Engneerng, Graduate Unversty of Advanced Technology, Kerman, Iran Mr.emarat@yahoo.com 2 Assstant Professor, Department of Energy Management and Optmzaton, Insttute of Scence and Hgh Technology and Envronmental Scences, Graduate Unversty of Advanced Technology, Kerman, Iran f.keyna@kgut.ac.r 3 Assstant Professor, Department of Energy Management and Optmzaton, Insttute of Scence and Hgh Technology and Envronmental Scences, Graduate Unversty of Advanced Technology, Kerman, Iran a.askarzadeh@kgut.ac.r Abstract: Short term load forecastng s one of the key components for economcal and safe operaton of power systems. In compettve envronment of electrcty market, electrcty utltes requre more accurate load forecastng strateges to make better decsons on purchasng or generatng electrcty. Ths artcle offers a new method based on machne learnng short-term load forecastng whch s made up of a two-level feature selecton technque and a new forecast engne. The feature selecton part uses rrelevancy and redundancy flters to select best sets of nput features. The proposed forecast engne s composed of a support vector regresson machne, hybrd neural network and comprehensve learnng partcle swarm optmzaton. By applyng comprehensve learnng partcle swarm optmzaton along wth hybrd neural networks, the accuracy of forecastng s mproved and ts error decreases effectvely. The proposed strategy s tested on PJM and AEMO electrcty markets. The numercal results show the effectveness and robustness of ths method n comparson wth recent short-term load forecastng methods. Keywords: Feature selecton, Forecastng engne, Hybrd neural network, Partcle swarm optmzaton, Short-term load forecast. 1- Introducton 1 Load forecastng helps electrcal power systems to make mportant decsons on purchasng and generatng electrc power, load swtchng and substructure mprovement. Load forecasts can be dvded nto three categores: short-term forecasts whch are usually from one hour to one week, medum forecasts whch are usually from a week to a year, and long-term forecasts whch are longer than a year [1]. Short-term load forecastng (STLF) has become a serous ssue for electrcty supply. It has a sgnfcant role n securty and relablty whch are two essental necesstes for proper plannng and operaton of power systems. A relable STLF can be practcally used n power systems for meetng power consumed contnuously. In 1 Submsson date: 12,01., 2015 Acceptance date: 20, 05, 2017 Correspondng author: Mohammadreza Emarat, Electrcal Engneerng Department- Graduate Unversty of Advanced Technology- Kerman- Iran addton, mprovng economy of operaton and control of power system can be attaned by ncreasng the accuracy of STLF [2-4]. Varous methods have been used for load forecastng up to the present tme. Majorty of these approaches can be broadly dvded nto two classes: the tradtonal approaches depcted by tme seres and the modern ntellgent approaches depcted by artfcal neural networks (ANN) [5]. Tradtonal methods nclude classcal multple lnear regresson [6], ARMA (automatc regressve movng average) [7], data mnng models [8], tme-seres models [9] and exponental smoothng models [10]. However, the modern ntellgent approaches have presented hgher performance for non-lnear tme seres than the tradtonal approaches [4]. Nowadays, artfcal ntellgence (AI)-based methods such as pattern recognton [11], fuzzy feature selecton [12], fuzzy tme seres [13], neural networks(nn) [14, 15], and fuzzy NNs [16] are hghly regarded as powerful computatonal tools for solvng the problem of load forecastng [17]. Although avalable approaches have provded sgnfcant enhancement throughout the years, more precse

2 Applcaton of hybrd neural networks combned wth comprehensve learnng 3 and robust load forecastng methods are stll needed. In ths study, a new strategy based on machne learnng short-term load forecastng (ML-STLF) s proposed. Ths method employs a coalton of machne learnng (ML) for an effcent two-level feature selecton and Support Vector Regresson (SVR) for ntal tranng of the nonlnear mappng functon. The ntroduced forecast engne employs three-stage hybrd neural network (HNN) and comprehensve learnng partcle swarm optmzaton (CLPSO) smultaneously. Ths composton helps the forecast engne to create a more precse predcton. Usng CLPSO because of hgh global search ablty and ts hgh capablty n combnaton wth local search methods can have an mportant role n enhancng the precson of the forecastng engnes. The effcency of proposed strategy s proved by usng some numercal experments. The man parts of ths artcle can be summarzed n two sectons: (1) Most of prevous studes emphaszes on the forecast strategy, whereas they don t pay attenton to desgn of the nput vector. Here, an effcent data preparaton (normalzaton and shufflng) and a novel ML approach contanng two-level feature selecton technque s employed to select the prvleged canddate nputs for the proposed forecast engne. The frst level flters rrelevant nputs whle the second one removes redundant canddate features. Inputs that have been able to pass through ths two-level feature selecton are appled to the forecast engne. (2) A new powerful and effcent STLF engne s proposed. Ths engne s made up of a combnaton of three nterconnected core unts. The frst part s an auxlary predctor whch employs a SVR machne to produce ntal forecast of target varables. The second and the man part s constructed by an HNN whch uses dfferent tranng functons n each stage. And the last part, whch s a cooperatve tool for HNN, apples CLPSO method to mprove the learnng capablty of HNN. Followng ths ntroducton, the remanng parts of ths paper s organzed as follows: In Secton 2, the proposed ML-STLF approach s ntroduced. The obtaned numercal results of the proposed approach are ndcated and compared wth other new STLF methods In Secton 3. And Secton 4 presents the concluson. 2- Descrpton of the proposed ML- STLF strategy Fg(1) ndcates the general structure of proposed ML-STLF approach. As can be seen, the ntended method s made by combnng a two-level feature selecton part wth a STLF engne, as explaned n Sectons 2.1 and 2.2, respectvely. Also, n sectons 2.3 and 2.4 detaled explanatons about CLPSO and ts ntegraton wth HNN are gven. Two-level Feature Selecton Forecastng Engne Input (Canddate Feature Set) : L(t) Irrelevancy Flter Redundancy Flter Auxlary Predctor (SVR) HNN+CLPSO Output (Target Varable) Irrelevant Features Redundant Features Fg(1): Structure of proposed ML-STLF strategy Two-level feature selecton A key ssue for the achevement of any forecast strategy s a sutable choce of effectve nput varables. Feature selecton can smplfy the learnng process of the forecastng engne and mprove ts generalzaton capablty for unseen data. At the stage of feature selecton at frst, rrelevant features are removed and then redundant features are executed to create a subset coverng the best nput features. The best subset contans the least number of key features, whch are vtal to forecast more precsely. Usng feature selecton n prepossessng phase can reduce the dmenson of nput varables n an effectve way. In most of the prevous studes lke [7, 14] the authors pad attenton to forecastng models and dfferent heurstc methods were appled for selectng nput varables. Detals of correlaton approach has been explaned n [18] and t has been

3 4 employed n the feature selecton part of STLF n [4] and [19]. In correlaton approach, the relevancy among each canddate nput and the target value (n ths artcle, load of the next hour) s calculated and a dependng factor determnes the relevancy between them. More correlaton between the target varable and a canddate nput results n more chance of that varable to be selected as an nput feature. The dependng factor among two varables lke X and Y, denoted by, wth standard xy, devatons and, s computed from followng relaton: X x Y y cov( X, Y) X, Y (1) where cov X, Y s the covarance of X andy. The absolute value of dependng factor (whch s a number between 0 and 1) ndcates the amount of lnear relance among the varables. The two-level feature selecton removes the rrelevant and redundant nput features respectvely. By decreasng the number of nputs to the forecast engne, the number of optmzaton varables reduces. Ths acton not only mproves the tranng accuracy of forecastng engne but also ncreases ts tranng speed. If correlaton ndex between the output feature and a canddate varable s more than a relevancy threshold TH1, then ths canddate s regarded as the relevant feature of the forecast process. Other canddate nputs wth correlaton ndex less than TH1 are regarded as rrelevant features as shown n Fg. 1. Next, for the remanng canddates, a cross-covarance test s executed. Greater value of correlaton between two selected features ndcates more common nformaton between. In other words, these features have a remarkable level of redundancy. If the correlaton ndex across any two canddate varables s less than a predetermned value TH2, then both varables are selected; otherwse, only the varable wth the greatest correlaton accordng to the target value s remaned whle the other s not consdered any further. Selected canddate features are regarded as the fnal entres of the ML-STLF engne as ndcated n Fg. 1. Next, the proposed forecast engne must carry out the predcton procedure. Its effcent performance n short-term load forecastng utlzaton s one of the key deas of ths paper Proposed HNN The man propose of ths secton s mprovement of forecastng model through learnng from the selected canddate features whch are obtaned by two-level feature selecton strategy. SVR s a supervsed machne learnng method whch has a hgh learnng capablty. SVR models are able to deal wth dfferent knds of data patterns. For SVR, the tendency of the data whch may present fluctuaton or sustaned ncreasng or decreasng types does not make much dfference. Generally, SVR s appled for solvng non-lnear regresson and tme seres problems. A computatonal regresson functon n a hgh dmensonal feature space forms the man structure of SVR. Ths functon plots the nput data to the hgher dmensonal space. In other words, the basc noton of the SVR s to map the orgnal data nto a hgher dmensonal feature space usng a nonlnear process. In ths strategy, the structural rsk mnmzaton nductve prncple s mplemented to generate lmted numbers of learnng patterns. In ths artcle, as shown n Fg (2), SVR s appled as an auxlary predctor. Addtonal nformaton about SVR and ts operaton can be found n [20]. Furthermore, SVR has been used for short term load forecastng n [7]. In ths way, the SVR machne receves the nput features selected by two-level feature selecton and forwards the output predcted values along wth nput features to HNN. A properly desgned composton of dfferent NNs can strongly mprove ther capablty of learnng n modellng a complcated operaton. For nstance, several varous (cascaded and parallel) structures for combnaton of dfferent NNs wth enhanced capablty are presented n [21-23]. An effcent HNN, has been proposed n [24] for electrcty prce forecastng. All three NNs appled n HNN of [24] have smlar structure of mult-layer perceptron wth a hdden layer. MLP s a hgh yeld structure of forecastng neural networks. Furthermore, n accordance wth Kolmogorov s theory, by selectng approprate number of neurons, only one hdden layer s enough for MLP to deal wth a problem [25]. Therefore, one hdden layer consdered to apply n the

4 Applcaton of hybrd neural networks combned wth comprehensve learnng 5 structure of each NN. As the forecast engne of electrcty load predcton, a new HNN wth the archtecture shown n Fg (2) s proposed n ths paper. Auxlary Predctor (SVR) Weghts & Bases Weghts & Bases LMNN Target varable of LMNN BFGSNN BRNN Target varable of BFGSNN Fnal forecast for target varable HNN Fg (2): Archtecture of Hybrd Forecastng Engne. As shown n Fg (2), after the SVR machne performs the prelmnary forecast, the results of ntal predcton along wth the selected features are appled to the frst stage of HNN (LMNN). Also, n each stage of HNN a specfc NN has been used. Moreover, n ths structure each NN transmts two vectors of results to the subsequent NN. Only the frst NN should begn wth an ntal set of random values for the adjustable parameters. In other expresson, each traned NN hands ts obtaned profcency to the followng NN. Thus, nstead of begnnng from a random pont, the tranng process of each NN can be started from the place, that ts former NN has been reached. Hence, the next NN can drectly employ the weght and bas values of former one, because all NNs of the HNN have dentcal number of nputs, hdden and output neurons, the obtaned knowledge of the prevous one can be mproved. Furthermore, the second sets of consequences transferred among NNs are the predcton of target varables. By ths manner, all NNs also have a prelmnary forecast of target as an nput value, whch gets great beneft to mprove accuracy of the predcton. In addton, by sutable selecton of dfferent MLP tranng algorthms, the HNN can be learned much more than a sngle NN. Further dscussons for the most effcent NN tranng mechansms and ther mathematcal detals can be found n [25]. In vew of the above dscusson, an mproved verson of HNN forecast engne s proposed for ML-STLF n ths artcle. Three knds of tranng algorthms have been consdered for the HNN, ownng Levenberg Marquardt Neural Network (LMNN), Broyden Fletcher Goldfar Shanno neural network (BFGSNN), and Bayesan regularzaton neural network (BRNN). In [3], t s explaned that the best results are related to cascade MLPs whch beneft from LMNN In the begnnng of the tranng stage. In ths way, MLP can quckly learn about the problem and ts tranng error rapdly decreases. LMNN s a fast learnng algorthm. Therefore As seen n Fg (2) LMNN has been selected as the frst NN of HNN. BFGSNN s known as the most powerful quas-nowton method for tranng NNs. If ths algorthm starts the learnng process from a sutable ntal pont, t wll show greater ablty to fnd better solutons n the search space. Thus, n the second stage BFGSNN has been used for detectng superor weghts and bases n the soluton space. In addton, BR learnng algorthm mnmzes a combnaton of squared errors and weghts and then determnes the correct combnaton so as to produce a network that generalzes well [26]. Therefore, BR tranng mechansm s consdered as the last NN for fnal tunng of the adjustable parameters and gettng the maxmum tranng effcency CLPSO Algorthm Partcle swarm optmzaton (PSO) s a global mnmzaton algorthm, whch s a powerful tool for solvng hgh-dmensonal problems. Each potental soluton s consdered as a partcle, whch tres to make ts current poston better than ts former poston. In other words, the poston of each partcle depends on ts current poston and a velocty vector, whch s defned for the same partcle. The poston and velocty of partcle n a physcal d dmensonal search space are expressed as the vectors of X [ X 1, X 2,..., X d ] and V [ V 1, V 2,..., Vd ], ts ndvdual best poston s [ X, X,..., X ], and Pbest 1pbest 2 pbest dpbest Its global best poston s [ X, X,..., X ]. In each teraton, Gbest 1gbest 2gbest ngbest the velocty and poston of partcle s updated as follows : V ( k 1) V ( k) c r ( P X ( k)) 1 1 best c r ( G X ( k)) X 2 2 best (2) ( k 1) X ( k) V ( k 1) (3)

5 6 where V ( k) s the velocty of partcle at teraton k ; denotes nerta weght factor; C1, C are acceleraton coeffcents; 2 R1, R are unformly dstrbuted random number 2 among 0 and 1; X ( k ) s the poston of partcle at k teraton; s the best poston of Pbest partcle untl teraton k ; and G s the best global best poston of all partcles untl teraton k. In ths paper, an advanced verson of PSO called comprehensve learnng partcle swarm optmzaton (CLPSO) [27] has been used. CLPSO has demonstrated good performance n hgh dmensonal problems. In ths algorthm for updatng the velocty of each partcle, P of best all partcles wll get nvolved nstead of usng P of each partcle for the same partcle. Thus, best the equaton (2) s changed as follows: ( ) V ( 1) ( ) 1 1( f d k V k c r Pbest X ( k)) c r ( G X ( k)) 2 2 best (4) where f( d) [f(1),f(2),...,f ( d)] s a vector, whch determnes the partcle should use P of whch partcle n each dmenson. best CLPSO s completely explaned n [27]. In the process of velocty updatng, the value of some parameters such as, C1, C should be 2 decded beforehand. Expermental results show that more convergence can be acheved by reducng the nerta weght n each teraton. Therefore, the value of s reduced lnearly as the teraton k proceeds and fgured as follows: ( ) ter max mn max (5) ter max where s fnal nerta weght; max s mn ntal nerta weght; ter s current teraton number; and ter s maxmum teraton max number. In ths study, all parameters of CLPSO are fne-tuned based on the proposed method of [27] Combnaton of HNN and CLPSO for desgnng the Proposed ML-STLF Engne The hybrdzaton approach of HNN and CLPSO to create the suggested ML-STLF engne s llustrated n Fg (3). Although LMNN, BFGSNN and BRNN beneft from hgh effcent tranng algorthms, they explore the soluton space n a partcular drecton. In ths manner, these tranng algorthms may be got stuck n a local mnma wthout fndng the global mnma. However, exploraton capablty of CLPSO algorthm can broadly nvestgate the soluton space n dfferent drectons. Therefore, the proposed tranng method s more possble to escape from the local mnma. At frst LMNN s traned by the LM learnng algorthm. To avod the over fttng problem, early stoppng condton s used n the tranng procedure of all NNs, as shown n Fg (3) the obtaned weght and bas values are transferred to CLPSO. Then, CLPSO contnues the process of tranng by modellng ths process as an optmzaton problem. Weghts and bases can easly be transferred because NNs of HNN and CLPSO component have equal tranng and valdaton samples. The objectve functon of the optmzaton problem s the error functon of LMNN, whch should be mnmzed. In other words, CLPSO tres to further mnmze the valdaton error of LMNN after ts learnng algorthm s completed. The decson varables of the optmzaton problem (partcles of CLPSO) are potental solutons for weght and bas vectors of LMNN. Generally, the poston of the partcles n CLPSO are ntalzed randomly: The ntal swarm of CLPSO = X 1,0, X 2,0,..., X (6) NP,0 The structure of each partcle can be shown as follows: X W & B (7) LM where W & B LM are weght and bas vectors, whch are ntalzed randomly. where NP s the number of partcles n soluton space. In (6), X 1,0, X 2,0,..., X demonstrate the NP, 0 ntalzed postons of the NP partcles of CLPSO. In (7), W & B are obtaned results of the LM weght and bas vectors by LMNN. Also, the ntal velocty vector of NP partcles are ntalzed randomly wthn allowed specfc ranges. Then, teratve partcles of CLPSO change ther postons and explore the soluton space thoroughly. Here, f the value of the valdaton error does not reduce after four consecutve teratons, the search process wll be termnated. Next, the best partcle of CLPSO (whch s coverng a weght and bas vector) s gven back to LMNN; ths vector s regarded as the fnal weghts and bases of LMNN. Here, the learnng process of LMNN s completed.

6 Absolute value of correlaton coeffcent Applcaton of hybrd neural networks combned wth comprehensve learnng 7 Then, as seen n Fg. 3 BFGSNN accepts the fnal weghts and bases of LMNN as the ntal values to start the learnng process. For BFGSNN, tranng proses s the same as LMNN. Smlarly, after fnshng the learnng process of BFGSNN, ts fnal weghts and bases are transferred to BRNN. Startng from ths pont, the learnng process of BRNN s executed smlar to tranng process of the last two NNs. After completon of the tranng process of BRNN, all NNs of HNN are traned. At ths pont, the proposed ML-STLF engne s learned and prepared for the forecast. In ths way, LMNN usng ts obtaned weght and bas values, produces a forecast for the nput seres (Here, the electrcty load of the prevous hours) L (t), whch s appled to BFGSNN. Also, BFGSNN and BRNN generate ther forecast values untl the last predcton of L (t) s obtaned from BRNN. Auxlary Predctor (SVR) Weghts & Bases Weghts & Bases LMNN Target varable of LMNN BFGSNN BRNN Target varable of BFGSNN CLPSO respectvely. For predcton of the next hour s load, the data updates at the end of each tme nterval. Also, predcton of the next day s load s attaned by substtuton of forecasted values for nput varables called recurson method [4]. Ths procedure s repeated untl the next day s load forecast value s acheved. In ths research, hourly load forecast and one-day ahead (as the predcton horzon) have been consdered. The proposed ML-STLF strategy has been examned on load forecast of day-ahead electrcty market of PJM. PJM s a well-known ste n electrcty market that coordnates the movement of wholesale electrcty n dfferent states of US. Our test case ncludes day-ahead demand hstorcal data over the perod whch can be found at [30]. The numercal expermentatons, whch are presented n the followng, are desgned to show the hgh performance of the proposed ML-STLF engne and evaluate ts effectveness n a comparatve manner. Fg (4) shows the correlaton of canddate features of 500 hours ago accordng to the output for December 12, In Fg (4), the horzontal axs ndcates (canddate accordng to 1 hour ago) up to (canddate accordng to 500 hours ago) and the vertcal axs demonstrates the absolute amount of correlaton coeffcents. The greater amount of correlaton means more relaton between correspondng canddate and the target vector s value Fnal forecast for target varable HNN Fg(3): Archtecture of the proposed ML-STLF engne. It should be mentoned that the numercal fne-tunng of the proposed ML-STLF method s adjustable parameters, ncludng TH1 and TH2 whch are used n the two-level feature selecton process and the number of appled neurons n each NN s hdden layer, has been carred out by a computatonally effcent cross-valdaton method descrbed In [28]. 3- Numercal Results There are two prevalent types of STLF strateges n electrc power systems: hourly (next hour) and daly (next day) load forecastng [29]. Real-tme and future electrcty markets beneft from hourly and daly load forecastng, Sample Index Fg(4):Correlaton of canddate features of 500 hours ago accordng to the output for December 12, Results of two-level feature selecton process wth dfferent values of TH1 and TH2 are presented n Table (1). It can be seen by ncreasng TH1, the ntal level of feature selecton selects canddates that are more relevant and by decreasng TH2 the secondary level of feature selecton flters more redundant canddates. However, f these two terms are satsfed smultaneously, number of selected varables wll decrease. So, determnaton of TH1 and TH2 s a trade-off between qualty and number of features. In ths test case, the best values of TH1 and TH2 by the cross-valdaton method are determned 0.6 and 0.9,

7 8 respectvely. In addton, n ths study, the proper number of neurons for hdden layer of all NNs for the mnmum of the valdaton error by cross valdaton technque, occurs at NH=10. Table (1): Number of Selected Features TH1 TH2 Number of selected nputs In ths study, tranng samples (subsets of data) from electrcty load related to 39 days before the forecast day (results n 39*24=936) have been consdered. The valdaton set has been randomly selected from tranng samples. Moreover, the 10% of total samples are selected as valdaton set and the rest are consdered as tran set. In Table(2), the frst and thrd rank selected features ( L and t 1 L ) consst of nformaton t 2 about earler hours. The next more effectve features contan nformaton about the daly seasonalty (one day ago) such as L and t 24 L, whle the later terms are related to weekly t 23 seasonalty (one to three weeks ago). Farther days and weeks have less correlaton wth the target hour and so are not consdered here. Note that by ncreasng nput canddates the selected canddates provded n Table (2) wll not be changed, ndcatng that more earler canddates have less nformaton value than others. In other words, the latest selected canddate features (as valdaton set) have more resemblance to the predcton horzon. Table (2): Selected features for PJM on December 12, Rank Selected Feature 1 t 1 2 t 24 3 t 2 4 t t 23 6 t 25 7 t 335 Rank Selected Feature L 8 Lt 168 L 9 Lt 337 L 10 Lt 167 L 11 Lt 360 L 12 Lt 169 L 13 Lt 480 L 14 Lt 312 The measurement of forecastng accuracy s accomplshed by Mean Absolute Percentage Error (MAPE), whch can be computed as follows: N 1 LA LF MAPE (%) 100 ( N 1 LA 8) where: L A s the actual load, LF s the forecasted load, N s the predcton horzon (the amount of hours n the predcton perod) and s the hour ndex. Another crteron for evaluatng forecast accuracy s mean absolute error (MAE), whch s defned by the followng equaton: 1 MAE N N 1 L A L F (9) The proposed ML-STLF approach covers the predcton perod by one-hour steps up to reachng out to the end of forecast perod. Here, the predcton horzon s consdered 24 hours. For ths case study, obtaned results (MAPE and MAE) from each NN of the proposed HNN are represented n Table 3. As shown n Table 3, the values of MAPE and MAE have been reduced from 2.03% and 0.74 GW to 1.48% and 0.54 GW, respectvely. Ths table specfes the role of each stage of the forecast engne n reducng the predcton error. Table (3): MAPE (%) and MAE (MW) results of proposed forecast engne on December 12, NN output MAPE (%) MAE (MW) LMNN+CLPSO BFGSNN+CLPSO BRNN+CLPSO To compare proposed method wth the other hybrd approaches, results of dfferent hybrd methods are demonstrated n Table 4. In all of mentoned methods n Table (4) data preprocessor and feature selecton have been used. NN s a one hdden layer perceptron neural network, HNN2 s a hybrd neural network wth two stages and HNN3 s a hybrd neural network wth three stages. In Table (4) tral perod s December 12, In ths table, t s observed that the proposed ML-STLF engne beneftng from CLPSO has been able to decrease the values of MAPE and MAE by 54% and 37% (respectvely) n comparson wth HNN3. Also, In Fg (5), actual load, forecasted load and forecast error on the same day have

8 MSE Applcaton of hybrd neural networks combned wth comprehensve learnng 9 been shown. It can be seen from Fg (5) that the proposed strategy can gve an accurate forecast. Also, the convergence plot of CLPSO algorthm to mprove the results obtaned by LMNN s depcted n Fg.6. Ths fgure shows the value of Mean Squared Error (MSE) for normalzed data. The optmzed results n ths stage are gven to the next neural network (BFGSNN) for contnung the tranng process of the forecast engne. Table (4): MAPE (%) and MAE (MW) results of suggested ML-STLF and three other methods for December 12, Forecast method MAPE (%) MAE (MW) NN HNN HNN Proposed (ML-STLF) Fg (5): Actual load (sold lne), forecasted load of proposed method (dashed lne), and ts error (dotted lne) for PJM n December 12, Iteratons Fg (6): The convergence plot of CLPSO algorthm used to mprove the results obtaned by LMNN. To demonstrate the capablty of the suggested STLF, n Table (5) four test weeks of the year 2012 from the PJM electrc power market have been examned. The four test weeks are February 4 to February 11, May 5 to May 12, August 4 to August 11, and November 10 to November 17. In Table 5, the proposed forecast engne (ML-STLF) s compared wth three MLP neural networks whch are traned by LM, BFG, and BR learnng algorthms, respectvely. These nonlnear forecast methods have been used frequently n many artcles to predct electrcty load demand, electrcty prce and wnd power. As shown n Table 5, the proposed forecast engne has better forecast accuracy than the other forecast methods. As presented n the last row of Table 5 the average MAPE and MAE for the proposed ML-STLF method are 1.34% and GW respectvely. However, the other mentoned methods wth the same condtons have shown the average value of MAPE and MAE 3% and GW respectvely. The proposed method usng CLPSO n ts tranng mechansm can reach better solutons because of ts ablty to escape from local mnma. In addton, the proposed forecast engne s composed of three consecutve NNs to mprove the obtaned knowledge durng the forecast process whle the three other methods use one NN for predcton. In Table 6, the obtaned results are compared wth results of reference [31]. These results demonstrate good performance of proposed method n comparson wth the other mentoned methods. The structure of Addtve model presented n [31] s based on a three layer feedforward NN whch employs Levenberg Marquardt algorthm for tranng. For speedng up the tranng process, the hyperbolc tangent functon has been appled for hdden neurons and output neurons. The results presented n Table (6) were obtaned usng hstorcal data from Australan Energy Market Operator (AEMO) webste [32] snce October 2008 to March It can be seen from Table 6 that the proposed strategy has reached more precse results than the other mentoned methods. In the last row of Table (6), t s found out that the obtaned average values of MAPE and MAE from the proposed ML- STLF method are 8% and 19% (respectvely) less than the Addtve model. These results can show the satsfactory performance of the proposed method compared wth the recent strateges. The average computaton tme taken for tranng the proposed ML-STLF model s about 15 mnutes. The hardware confguraton of the computer used s Intel Core 5 processor wth

9 GHz CPU, 4 GB RAM and the operatng system used s Wndows 7 ultmate. 4- Concluson Load forecastng s very mportant for secure operaton of power systems. Ths paper presents a new strategy for STLF. In ths strategy, a twolevel feature selecton technque and a new hybrd forecastng engne are employed. The two-level feature selecton technque s desgned for removng both rrelevant and redundant canddate nputs. Thus, the most nstructve features are appled to forecast engne. The proposed forecastng engne s a hybrd neural network, whch benefts from good global search capablty of CLPSO. In addton, by usng CLPSO alongsde NN hgh convergencelearnng algorthms can have an mportant role for enhancng the accuracy of forecastng. The proposed strategy has been examned n PJM and AEMO electrcty markets. The results llustrate that the proposed model has a hgh level of effectveness and robustness. Also, results of proposed strategy show more accuracy compared wth methods of recent papers. Table (5): MAPE (%) and MAE (MW) results of MLP wth LM, MLP wth BFG, MLP wth BR and proposed ML-STLF engne Test week MLP wth LM MLP wth BFG MLP wth BR Proposed (ML-STLF) MAPE MAE MAPE MAE MAPE MAE MAPE MAE February May August November Average Table (6): Monthly comparson of performance. Month ANN [31] Hybrd [31] Addtve Model [31] Proposed (ML-STLF) MAPE MAE MAPE MAE MAPE MAE MAPE MAE October November December January February March Average References [1] M. Q. Raza and A. Khosrav, "A revew on artfcal ntellgence based load demand forecastng technques for smart grd and buldngs," Renewable and Sustanable Energy Revews, Vol. 50, pp , [2] R. Hu, S. Wen, Z. Zeng, and T. Huang, "A shortterm power load forecastng model based on the generalzed regresson neural network wth decreasng step frut fly optmzaton algorthm," Neurocomputng, Vol. 221, pp , [3] N. Amjady and F. Keyna, "Md-term load forecastng of power systems by a new predcton method," Energy Converson and Management, vol. 49, pp , [4] S. Kouh and F. Keyna, "A new cascade NN based method to short-term load forecast n deregulated electrcty market," Energy Converson and Management, Vol. 71, pp , [5] H. Ne, G. Lu, X. Lu, and Y. Wang, "Hybrd of ARIMA and SVMs for Short-Term Load Forecastng," Energy Proceda, Vol. 16, pp , [6] F. M. Tuamah and H. M. A. Abass, "Short-term electrcal load forecastng for Iraq power system based on Multple Lnear Regresson method," Internatonal Journal of Computer Applcatons, Vol. 100, [7] A. Kavous-Fard, H. Samet, and F. Marzban, "A new hybrd modfed frefly algorthm and support vector regresson model for accurate short term load forecastng," Expert systems wth applcatons, Vol. 41, pp , 2014.

10 10 [8] S. Park, S. Ryu, Y. Cho, and H. Km, "A framework for baselne load estmaton n demand response: Data mnng approach," n Smart Grd Communcatons (SmartGrdComm), 2014 IEEE Internatonal Conference on, 2014, pp [9] N. Amjady, "Short-term hourly load forecastng usng tme-seres modelng wth peak load estmaton capablty," Power Systems, IEEE Transactons on, Vol. 16, pp , [10] J. W. Taylor, "Short-term electrcty demand forecastng usng double seasonal exponental smoothng," Journal of the Operatonal Research Socety, Vol. 54, pp , [11] M. F. Sabah and S. Sarrfan, "A Cell Phone Postonng Method Based on the Intellgent Receved Sgnal Strength Pattern Recognton n GSM Network for Implementng Moble Advertsement Servces," Computatonal Intellgence n Electrcal Engneerng, Vol. 4, pp , [12] H. Nosrat nahook and M. Eftekhar, "A New Method for Feature Selecton Based on Fuzzy Logc," Computatonal Intellgence n Electrcal Engneerng, Vol. 4, pp , [13] Y. Hong-Tzer and H. Chao-Mng, "A new shortterm load forecastng approach usng selforganzng fuzzy ARMAX models," Power Systems, IEEE Transactons on, Vol. 13, pp , [14] N. Dng, C. Benot, G. Fogga, Y. Bésanger, and F. Wurtz, "Neural network-based model desgn for short-term load forecast n dstrbuton systems," IEEE Transactons on Power Systems, Vol. 31, pp , [15] K. Tayebeh, M. Atae, and P. Moallem," Wnd Speed Predcton Based on Chaos Theory usng RBF Neural Networks," Computatonal Intellgence n Electrcal Engneerng, Vol. 7, pp , [16] G.-C. Lao and T.-P. Tsao, "Applcaton of a fuzzy neural network combned wth a chaos genetc algorthm and smulated annealng to short-term load forecastng," IEEE Transactons on Evolutonary Computaton, Vol. 10, pp , [17] N. Amjady and F. Keyna, "Short-term load forecastng of power systems by combnaton of wavelet transform and neuro-evolutonary algorthm," Energy, Vol. 34, pp , 1// [18] P. J. Santos, A. G. Martns, and A. J. Pres, "Desgnng the nput vector to ANN-based models for short-term load forecast n electrcty dstrbuton systems," Internatonal Journal of Electrcal Power & Energy Systems, Vol. 29, pp , 5// [19] N. Amjady, "<Short-Term Hourly Load Forecastng Usng.pdf>," pp , [20] S. Haykn, Neural networks: a comprehensve foundaton: Prentce Hall PTR, [21] H. Taheran, I. Nazer-Kakhk, M. R. Aghaebrahm, M. Farshad, and S. R. Goldan, "Short Term Prce Forecastng n Electrcty Market Consderng the Effect of Wnd Unts' Generaton," Computatonal Intellgence n Electrcal Engneerng, Vol. 5, pp , [22] F. Shu, J. R. Lao, R. Yokoyama, C. Luonan, and L. We-jen, "Forecastng the Wnd Generaton Usng a Two-Stage Network Based on Meteorologcal Informaton," Energy Converson, IEEE Transactons on, Vol. 24, pp , [23] S. Z. Seyyedsaleh and S. A. Seyyedsaleh, "Bdrectonal Layer-by-layer Pre-tranng Method," Computatonal Intellgence n Electrcal Engneerng, vol. 6, pp. 1-10, [24] N. Amjady, A. Daraeepour, and F. Keyna, "Dayahead electrcty prce forecastng by modfed relef algorthm and hybrd neural network," Generaton, Transmsson & Dstrbuton, IET, Vol. 4, pp , [25] C. P. Rodrguez and G. J. Anders, "Energy prce forecastng n the Ontaro compettve power system market," Power Systems, IEEE Transactons on, Vol. 19, pp , [26] N. Amjady, Introducton to ntellgent systems. Semnan,Iran: Semnan Unversty Press, [27] J. J. Lang, A. K. Qn, P. N. Suganthan, and S. Baskar, "Comprehensve learnng partcle swarm optmzer for global optmzaton of multmodal functons," Evolutonary Computaton, IEEE Transactons on, Vol. 10, pp , [28] N. Amjady and F. Keyna, "Day-ahead prce forecastng of electrcty markets by a new feature selecton algorthm and cascaded neural network technque," Energy Converson and Management, Vol. 50, pp , [29] N. Amjady, "<Short-Term Bus Load Forecastng of Power.pdf>," pp , [30] PJM. Avalable: [31] F. Shu and R. J. Hyndman, "Short-Term Load Forecastng Based on a Sem-Parametrc Addtve Model," Power Systems, IEEE Transactons on, Vol. 27, pp , [32] AEMO. Avalable: and-demand/aggregated-prce-and-demand- Data-Fles

11

12 10