Short Term Load Forecasting based on An Optimized Architecture of Hybrid Neural Network Model

Transcription

1 Short Term Load Forecastng based on An Optmzed Archtecture of Hybrd Neural Network Model Fras Shhab Ahmed Turksh Aeronautcal Assocaton Unversty Department of Informaton Technology Ankara, Turkey Mnstry of Electrcty of Iraq, The Stat Company Electrcty Producton GEEP Shad Al Shehab Turksh Aeronautcal Assocaton Unversty Department of Computer Engneerng Ankara, Turkey Abstract Nowadays the predcaton of electrcty demand s consdred as a man crtera for estmatng the electrcty generaton costs. Companes always sought to create balancng between the supply and demand of power. We are nterested n ths paper s short-term load forecastng (1 hour 1 week). The mportant factors that are consdered, n ths paper, for the electrcty supply are the temperatures and tme. Then, dfferent neural network models (NN) are used and developped for ncreasng the accuracy of the short term load forecastng such as backpropagaton, Adaptve Neuro-Fuzzy Inference System (ANFIS), wherease Wavelets Neural Network (WNN) s used as a comparatve model. An optmzed neural network s found by defnng the sutable number of hdden layers and ther nodes and an optmzed membershp functons number assocated to each nput unt s proposed n ANFIS model. A comparson between these models shows that the optmsed ANFIS model outperfroms the other methods. Keywords: Neural network (NN), Adaptve Neuro-Fuzzy Inference System (ANFIS), Wavelets neural network (WNN) I. INTRODUCTION Power demand producton s one the of most mportant factor for the management of electrcty producton, therefore, power demand s needed for capacty plannng and mantenance arrangement [1], Long-term forecasts are requred for electrcty plannng and mantenance schedulng n one year. Medum-term demand forecasts are requred for energy system operaton and plannng, [2]. Short-term load forecasts are requred for the control and confguraton of energy systems. Indeed, accurate electrcty demand predcton methods are mportant for reducng the cost of power producton; also the dstrbuton of electrcty depends on the accurate forecastng of the demand modules. The tme of load forecastng s dvded nto three types, short-term forecasts from one hour to one week, medum-term forecasts from a week to a year and long-term forecasts more than a year [3]. We are nterested n ths paper n short-term load forecastng whch can help to estmate load flows and to make decsons that can prevent overloadng. Short-term load forecasts depend on a table of hstorcal nformaton on hourly loads and temperature observatons from ISO New England. One selected week hourly load s taken for tranng our models and hourly electrcty consumpton forecast for the next week s gven by these models. We selected short-term load forecastng because a lot of companes of power producton deepened on Short-term load forecastng to determne the capacty and the level of electrcty supplyng to meet the foreseeable load, also to determne the prces, fve factors are consdered: dry bulb temperature, wet bulb temperature, dew pont temperature, daly hours and week days (work days and weekends). We propose to mprove the short-term load forecastng performance of two neural network models such as back propagaton neural network, Adaptve Neuro-Fuzzy Inference System (ANFIS), where three hdden layers wth dfferent number of neurons are used for the frst model. The number of membershp functons s determned by fndng the relatonshp between the nput varables. The powerful of the latest model s that t enables to make a rght decson through the use of the fuzzy logc system. The thrd model s a hybrd model of wavelet transportaton wth the neural network, t s a two-stage predcton system whch nvolves wavelet decomposton of nput data at the frst stage and the decomposed data wth other nput s traned usng separate neural network to forecast the load. The forecasted load s obtaned by reconstructon of the decomposed data. [4] Volume 8 Issue 2 Aprl ISSN:

2 II. RELATED WORK Actually, many researchers have publshed n the short-term load forecastng to predct the demand and the electrcty prce. Load forecastng s done by usng some effectve tools to determne the future demand that helps the companes to estmate the electrcty producton. In [5], dfferent varants of Generalzed Neural Network (GNN) traned wth back-propagaton and adaptve GAF were used to overcome the drawbacks of artfcal neural network (ANN) and back propagaton (BKP) tranng algorthm for mprovng the accuracy of short term load forecastng. A combned GNN model wth wavelet transform and GNN-W models have been developed. In [6], the authors used multlayer perceptron (MLP) for one day ahead short term load forecastng. Ths study showed that MLP network has a good performance and reasonable forecastng accuracy, where they obtaned a Mean Absolute Percentage Error (MAPE) between actual and predcted value equal to 1.066%. In [7], the authors added several factors that affect the behavor of the consumer load and also mpact the total losses n transmsson lnes. The factors are: weather, economy and random dsturbances. Dfferent types of consumers are consdered such as ndustral, agrcultural and domestc. It showed that domestc consumers gve good statstcal rules and are perodc n nature, but on the other hand ndustral and agrcultural loads are hghly nductve and start up and shut down of such type of load nduce huge spkes to the load curve. The spkes are dsturbance of the startup and shot down because ths load s qute random. It s mpossble to predct the occurrence of these spkes so t wll affect the short-term load forecastng. In [8], the author used Support Vector Machne (SVM) kernels for Short Term Load Forecastng on data. The obtaned results of SVM are compared SVM wth LDA and QDA. SVM kernel gave hgh performance of 99%. SVM can have the adaptablty on dfferent load stuatons such as workng days and non-workng days, also on some specal events and dfferent daly hours. III. FORECASTING ALGORITHMS Three methods are used n ths paper, n order to fnd the short load forecastng for NEW England ISO, whch are artfcal neural network, Adaptve Neuro-Fuzzy Inference system (ANFIS) and wavelet transportaton wth a neural network (WNN) all those methods are used and evaluated Neural Network Neural network (NN) s a mathematcal model whch s nspred from a bologcal neural network structure, the artfcal neurons are basc buldng blocks of neural network n the mathematcal model of neural network three steps (multplcaton, summaton and actvaton),where nputs of the neural network are weghted ths means multplcaton every nput wth weght, the second step sum functon whch sums all weghts nputs, n thrd step summaton and passng of prevously weghted nputs by actvaton functon(transfer functon ) as show n the fgure (1) [9,10],. Fgur.1. Workng prncple of an artfcal neuron [8] The mathematcal model of (Neuron model) s expressed as: Volume 8 Issue 2 Aprl ISSN:

3 O = f ( w x ) (1) j j jk k k Where: ο : represents the output of a neuron j f j : represents a transfer functon w jk : represents an adjustable weght x k : represents the nput of a neuron. The actvaton functon type (sgmodal) whch makes the output value n the range (0,1) [11] Network Archtecture: there are three fully connected layers as a show n fgure (2), those layers are an nput layer, a hdden layer and an output layer, the sgnal comes from the nput layer to the hdden layer after that multpled weght wth nput value and sum those values than pass t to output layer by actvaton functon Fgure.2. Schematc of the three-layer feed-forward ANN [11] Tranng: The am of the tranng process s to get the desred output, through adjustng the network weghts. The network s traned by usng the Back-propagaton algorthm, whch conssts of three steps [12]: Frst step: s the forward step, where the output values are calculated for gven nputs Second step: s the propagaton of the error of the outputs toward the nput layer, wth the fractonal dervatves of the performance and Weghts calculated n each layer. Thrd step: Weght adjustment, where the weghts are adjusted n order to reduce the error sum-squared error SSE s computed by usng the followng functon: 1 2 E = ( tpj Opj ) (2) 2 p j Where: O pj : represents an output unt t pj : represents a desred output P: represents an actual output j: represents an nput pattern In the tranng process uses the followng learnng functon for adjustng the weghts: Volume 8 Issue 2 Aprl ISSN:

4 dw = m dw + (1 m ) l gw (3) Where: dw prev c prev c r : represents the prevous weght gw: represents a weght gradent l : represents the learnng rate, r m : represents the momentum c Optmzaton of Neural Network Internatonal Journal of Innovatons n Engneerng and Technology (IJIET) The frst am of ths study s to fnd the optmal neural network archtecture n order to use t to load forecastng wth less mean absolute percentage value (MAPE). For ths reason, we use an expermental way to fnd the optmal network archtecture. So, the desgn of our network conssts of three layers: an nput layer, three hdden layers and an output layer. The nput layer ncludes fve nput unts, whch are: the temperature dry, the temperature dew, the hours, the week work and the weekend. 3.3 Adaptve Neuro-Fuzzy Inference System (ANFIS) ANFIS s a hybrd system that combnes the human reasonng style of fuzzy logc and the connectonst learnng style of neural network [13]. The hyprd model uses the fuzzy logc through mult layer neural network because ANFIS enables to combne tranng data through neural network and to take the decson through fuzzy nference system lke the human bran. ANFIS conssts of fve layers as shown n fguer (3). Fg.3. Structure of the ANFIS Frst layer s the executng of the fuzzfcaton process n order to convert the data nto truth values makng all the values between 0 and 1, where all the nodes n ths layer are adaptve nodes. There s also a membershp functon whch plays an mportant role n ths paper. In our dataset, we have one week data n order to forecast the next week, The node functon: Q 1 or 1 = µ A( x) Q = B ( y) µ 2, =1,2 (4), =3,4 (5) Where: Volume 8 Issue 2 Aprl ISSN:

5 1 Q : represents an output x, y: represent the nput values A, B 2 : represent lngustc label assocated wth ths node Internatonal Journal of Innovatons n Engneerng and Technology (IJIET) In ths layer, there are many dfferent membershp functons such as trangular-shaped- generalzed bell functons A and B generalzed bell functon: 1 µ Ax ( ) = x c 1+ a 2b (6) Where a, b and c are parameters Second layer uses fuzzy rules (f/ and). In ths step, we use the rules wth all nput values through (f/and) functon. We combne the nputs whch are ambguous by fuzzy rules n order to fnd frng strength of a rule as output nodes. Every node n ths layer s consdered as a fxed node labeled, the output s the multplcaton of the ncomng sgnal, here we modfy these rules n order to reduce the error [14,15] 2 Q = W = µa(x) µb(y), =1,2,3 (7) Thrd layer n the prevous layer, the nodes are fxed and labelled, but here each node N s labelled n order to calculate the average of frng strength and sum of all these rules. The normalzed frng strengths s the output of ths layer: 3 w Q = w =, 1, 2,3 w + w2 = (8) - Fourth layer n ths layer we used the frng strength from the thrd layer wth a set of parameters { a, b, c }, where all the nodes are adaptve nodes wth a node functon, also n ths layer the parameters represent consequent parameters w represent the output of thrd layer. 4 Q = wz - = w( ax+ by+ c), =1,2 (9) Ffth layer In ths layer we computed the fnal output as the summaton of all ncomng sgnals Optmzaton of ANFIS In ths paper, we propose a way to determne an optmzed model of ANFIS by determnng frst the sutable number of membershp functons of the fuzzy logc system. Ths can be done by fnng the relatonshp between the nput unts. Such that, we can fnd a relatonshp between the number of hours n a day and the temperature (Dry and Dew). Secondly, determnng the type of the membershp functon s mportant for mprovng the performance of ANFIS, therefor t could be done expermentally by testng each type of the membershp functon n the sutable number of t. 3.4 Hybrd model of Wavelet Transportaton wth Neural Network Wavelet transforms (WT) s a useful technque whch decomposes the tme seres sgnal n terms of both tme and frequency. Wavelet transform of a functon s the mproved edton of Fourer transform. WT as the name suggests, uses some small wave lke functon to analyze a sgnal and hence called wavelets (mother wavelet Volume 8 Issue 2 Aprl ISSN:

6 functon). Mathematcally speakng, wavelet transform s the convoluton of wavelet functon and the sgnal. The translated and scalng of mother wavelet Ψ(t) can be represented as follows n (10) [16]: 1 t b Ψ () t = Ψ a a (10) a : dlaton parameter b : Translaton parameter Ψ : wavelet The sgnal s decomposed nto approxmaton coeffcent and detal coeffcent. Wavelet decomposed components can be assembled back nto orgnal sgnal wthout loss of nformaton. Reconstructon can be done by combnng all the decomposed wavelet [16]. IV. EXPERIMENTAL RESULTS Our test dataset s a table of hstorcal hourly loads and temperature observatons from the New England ISO. Ths dataset ncludes recorded values of two weeks wth fve varables, such that, the data of the frst week s used as tranng data of the above-mentoned models (NN, ANFIS and WNN). The nput varables are: 1. Temperature Dry 2. Temperature Dew 3. Hours: represent hours of each day (1-24) n the frst week 4. Week work: represents the day number n the week (1-7) 5. Weekend: represents the two-type day: (1) represents the workday, (0) represents the weekend The output value: represents the electrcty load durng the frst week The data of the second week s used for testng the short-term load forecastng. The forecast accuracy s evaluated usng the mean absolute percentage error (MAPE). It s defned as follows: 4.1 Results of an optmzed NN As we above-mentoned, a neural network wth one nput layer, three hdden layers and one output layer s frst determned then four models of dfferent number of neurons n the hdden layers are tested (see Table 1), the tranng s done by usng back propagaton algorthm. After fnshng the tranng of the neural network by provdng the desred load and some elements whch affect the electrcty load such as temperature, 24 hours of day and weekend, the network s ready for forecastng. It s done by applyng a new nput sgnal wthout knowng ts desred load value; the neural network can predct the load value. We appled neural network wth dfferent model n order to fnd optmal neural network for our data through usng dfferent number of hdden layers' nodes of neural network as shown n the table 1. After the tests, we found that the optmal network s found when the number of nodes of the frst layers of the hdden layers s more than the second and the thrd layers. Such that we get the less MAPE value wth less tme (MAPE = ) and (tme = 21 mn 14 sec) as compared to the other models. The optmal model contans 10 nodes n frst hdden layer, 8 nodes n second hdden layer and 8 nodes n thrd hdden layer, so we proved expermentally that t s mportant to ncrease the number of nodes n the frst hdden layer for reducng MAPE of the neural network. Fnally, we used a neural network of three hdden layers because ths technque makes network tranng faster, and wth less MAPE value. Table (1) Forecastng evaluaton of 4 dfferent neural network models Volume 8 Issue 2 Aprl ISSN:

7 Network model 1 model2 model3 model 4 Input Output layers hdden nodes n frst layers nodes second layers nodes n thrd layers learnng rate momentum confscaton Transfer functon Hyperbolc tangent Hyperbolc tangent Hyperbolc tangent Hyperbolc tangent Tranng target error ntalzaton random random random random threshold Intalzaton random random random random weght update nterval cycles MAPE Tme 21mn 14 sec 20 mn 41 sec 23 mn 13 sec 15 mn 51 sec 4.2 Results of an optmzed ANFIS As we prevously mentoned, our am s to determne an optmal number of membershp functons of the fuzzy logc system through a neural network model. It s acheved nto steps: 1) Frst, we suggest to fnd correlaton between the daly hours and the temperature, such that the correlaton value between Dew pont and daly hours s R =0.9, and the correlaton value between Dry bulb and the daly hours s R = 0.96, where fgure (3,4) shows an effcent correlaton between them by takng the daly hours of one week. 2) Secondly, accordng to the fgure (3,4) we note that the curve s a perodc curve, so we dvde the daly hours nto three perods, (Each perod s 8 hours) n order to determne low, mddle and hgh temperatures. Therefore three membershp functons can be assocated to three nput vrables whch are tempertures Dew, temperature Dry and daly hours. Fgure.3. Correlaton between the Dew pont and the daly hours Fgure.4. Correlaton between the Dry bulb and the daly hours Therefore, we found the optmal numbers of membershp functon assocated to all nputs of the network are as follows: Three membershp functons assocated the frst nput (temperatures Dry) Three membershp functons assocated to the second nput (temperatures Dew) Three membershp functons assocated to the thrd nput (daly hours) Seven membershp functons assocated to the forth nput (week day) because t accepts only seven nteger values Two membershp functons assocated to the ffth nput (work day) because t accepts two nteger values. Volume 8 Issue 2 Aprl ISSN:

8 Three membershp functons are assocated to the frst, the second and the thrd nput, because they depend on three perods of day. Each perod represents 8 hours, seven membershp functons are assocated to the forth nput dependng on seven values of week days, two membershp functons are assocated to the ffth nput dependng on two values, such as the workng weekday s (1) and weekend s (0). We can expermentally show that our way of determnng the optmal number of membershp functons assocated to each nput unt s better than the orgnal method, ANFIS, by mplementng ANFIS wth an arbtrary number of membershp functons assocated to each nput unt and by usng eght dfferent types of membershp functons Fg. 5. The structure of the optmzed ANFIS The results n Table 2 show that the best membershp functon s (Pmf) wth MAPE=0.0060, also we can see the tranng error of the Pmf membershp functon n the fgure (6) and the Short load forecastng n the fgure (7), for one week (168 hours) Table (2) Comparson of Forecastng results when applyng dfferent numbers of dfferent types of membershp functons Number of membershp functons Types of membershp 2,2,2,2,2 3,3,3,3,3 4,2,2,7,2 3,3,3,7,2 functon MAPE MAPE MAPE MAPE trmf trapmf gbellmf gaussmf gauss2mf pmf dsgmf psgmf Types of membershp functon Number of membershp functons 2,3,2,4,2 3,2,2,4,2 2,3,4,5,2 2,2,3,2,2 MAPE MAPE MAPE MAPE trmf trapmf gbellmf gaussmf gauss2mf pmf dsgmf psgmf Volume 8 Issue 2 Aprl ISSN:

9 Fg. 6. Tranng error of the optmzed model Fg. 7. Short term load forecastng for the optmzed model 4.3 Results of WNN Three steps n WNN are used for short-term load forecastng: Frst step: the fve nput varables decompose va wavelet nto an approxmaton coeffcent and a detal coeffcent, then they are separated each nput value to Hgh frequency and low frequency as shown n the fgure (8). Second step: the output of each wavelet decomposton s dvded nto two levels n the neural network, such as hgh frequency (NNH) and low frequency (NNL). Thrd step: n ths step, the neural network s able to reconstruct hgh-frequency components and lowfrequency components through combnng those components together n order to get the load forecastng. The result shows that MAPE of WNN s Fgure.8. Structure of wavelet neural network 4.4 Comparng the performance of the forecastng methods The mean absolute percentage error MAPE between the forecast and target loads presented n Table 1 for the three models, NN, optmzed ANFIS and WNN s used to compare ther performance. It shows that our optmzed ANFIS model outperforms the other methods as well as t outperforms IT2 TSK FLS (A2-C1) model presented n [17], such that MAPE of the optmzed ANFIS model s Table (3) Comparson of Forecastng results for several models Load forecastng methods Optmzed ANFIS Neural Network WNN MAPE Volume 8 Issue 2 Aprl ISSN:

10 V. CONCLUSION In ths paper we have proposed frst an optmal archtcture of neural network model by fndng a sutable number of hdden layers and a sutable number of ther nodes, then an optmsed model of ANFIS was proposed n oroder to get an accurate electrcty load forecastng. The optmsed ANFIS model s found by determnng the sutable number of membershp functons assocated to each nput node and by fndng the sutable membershp functon type. The number of membershp functon s determned by fndng the relatonshp between tempertures and the daly hours. The powerful of ANFIS comes from usng fuzzy logc system that helps the neural network model to take the decsons. Wavelet transform wth neural network model s used as a hybrd model, the powerful of ths model comes from makng the hdden layers of the neural network more actve by usng the mother functon. A comparson between the three models showed that the error of short term load forecastng was , whle the error of short-term load forecastng of the neural network was 0.056, and the last model wavelet transportaton wth a neural network the error was The results showed that our optmsed ANFIS model outperforms the other models. REFERENCES [1] J. W. Taylor and P. E. McSharry, Senor Member, "Short-Term Load Forecastng Methods: An Evaluaton Based on European Data ", IEEE Transactons on Power Systems, 22, , [2] S. M. Barakat, A. A. Gharaves and S. M. Reza Rafe, "Short-term load forecastng usng mxed lazy learnng method, Turksh Journal of Electrcal Engneerng & Computer Scences, Turk J Elec. Eng. & Comp Sc (2015) 23: [3] O. T Altnoz, E. Mengusoglu, "Cloud-based Long Term Electrcty Demand Forecastng usng Artfcal Neuro-Fuzzy and Neural Networks", IEEE ELECO November [4] S. Gupta, V Sngh, A. P. Mttal and A. Ran, "A Hybrd Model of Wavelet and Neural Network for Short Term Load Forecastng", Internatonal Journal of Electronc and Electrcal Engneerng. ISSN , Volume 7, Number 4 (2014), pp [5] D. K. Chaturved, A. P. Snh and O. P. Malk, "Short term load forecast usng fuzzy logc and wavelet transform ntegrated generalzed neural network", Internatonal Journal of Electrcal Power & Energy Systems. May [6] S. K. Shekh1 and M. G. Unde, "SHORT-TERM LOAD FORECASTING USING ANN TECHNIQUE", Internatonal Journal of Engneerng Scences & Emergng Technologes, Feb ISSN: Vol 1, Issue 2, pp: [7] M. U. Fahad and N. Arbab, "Factor Affectng Short Term Load Forecastng", Journal of Clean Energy Technologes, Vol. 2, No. 4, October [8] L. Hussan, M. S. Nadeem and S. A. Al Shah, " SHORT TERM LOAD FORECASTING SYSTEM BASED, " Internatonal Journal of Computer Scence & Informaton Technology (IJCSIT) Vol 6, No 3, June [9] A. Krenker, J. Bešter and A. Kos, Introducton to the Artfcal Neural Networks, İçnde: Suzuk, K. (Ed). Artfcal Neural Networks-Methodologcal Advances and Bomedcal Applcatons. InTech, Croata, [10] Mrs. J. P. Rothe, Dr. A. K. Wadhwan and Dr. Mrs. S. Wadhwan, "Short Term Load Forecastng Usng Mult Parameter Regresson", (IJCSIS) Internatonal Journal of Computer Scence and Informaton Securty, Vol. 6, No. 2, [11] G.J. Tsekouras, F.D. Kanellos, and N. Mastoraks, "Short Term Load Forecastng n Electrc Power Systems wth Artfcal Neural Networks", Lecture Notes n Electrcal Engneerng pp 19-58, Volume [12] H. Chen, C. A. Ca nzares and A. Sngh, "ANN-based Short-Term Load Forecastng n Electrcty Markets", Power Engneerng Socety Wnter Meetng, IEEE, 28 Jan. 1 Feb [13] M. Mordjau, B. Boudjema, M. Bouabaz, R. Dara, Short Term Electrc Load Forecastng Usng Neuro-fuzzy Modelng for Nonlnear System Identfcaton, LRPCSI Laboratory, Unversty 20 August, Skkda, [14] J. P. Rothe, A.K. Wadhwan and S. Wadhwan, "Artfcal Neural Network and ANFIS Based Short Term Load Forecastng n Real Tme Electrcal Load Envronment", Internatonal Journal of Current Engneerng and Technology, Accepted 27 May 2014, Avalable onlne 01 June 2014, Vol.4, No.3 (June 2014). [15] K. Yang, L. Zhao, "Load Forecastng Model Based on Amendment of Mamdan Fuzzy System", IEEE pp:24-26, [16] S. Gupta, V. Sngh, A. P. Mttal, Asha Ran, "A Hybrd Model of Wavelet and Neural Network for Short Term Load Forecastng", Internatonal Journal of Electronc and Electrcal Engneerng. ISSN , Volume 7, Number 4 (2014). Volume 8 Issue 2 Aprl ISSN: