Image Embedding of PMU Data for Deep Learning towards Transient Disturbance Classification

Transcription

1 Image Embeddig of PMU Data for Deep Learig towards Trasiet Disturbace Classificatio Yogli Zhu, Chegxi Liu, Kai Su Electrical Egieerig ad Computer Sciece Uiversity of Teessee (UTK) Koxville, USA Abstract This paper presets a study o power grid disturbace classificatio by Deep Learig (DL). A real sychrophasor set composig of three differet types of disturbace evets from the Frequecy Moitorig Network (FNET) is used. A image embeddig techique called Gramia Agular Field is applied to trasform each time series of evet data to a two-dimesioal image for learig. Two mai DL algorithms, i.e. CNN (Covolutioal Neural Network) ad RNN (Recurret Neural Network) are tested ad compared with two widely used data miig tools, the Support Vector Machie ad Decisio Tree. The test results demostrate the superiority of the both DL algorithms over other methods i the applicatio of power system trasiet disturbace classificatio. Keywords Covolutioal Neural Network; Deep Learig; FNET; PMU; Recurret Neural Network I. INTRODUCTION With the icreasig deploymet of sychrophasors, e.g. PMUs (Phasor Measuremet Uits) o both trasmissio ad distributio systems of power grids i may coutries, covetioal power system operatio ad plaig practice has cofroted both ew challeges ad ew opportuities. The moitorig ad cotrol of a future smart grid should be desiged ad implemeted withi the framework of Eergy Iteret []. The vast PMU measuremets geerated cotiuously durig the daily grid operatios are big data ad require the most cuttig-edge data miig ad machie leaig techiques to aalyze ad digest for useful iformatio. Those machie leaig techologies have bee utilized to improve the evet detectio speed ad system moitorig accuracy for the security of power system [], []. The PMU techology has the followig features: o oe had, PMUs have a high samplig rate, e.g. samples per secod for a 6Hz AC power system, which is much higher tha that of SCADA data. The data size for a sigle day could be extremely huge ad hece poses a challege for effective reprocessig ad utilizatio of the data. O the other had, by meas of GPS (Global Positioig System), PMU measuremets are sychroized ad itegrated ito the Wide Area Measuremet Systems (WAMS), which i tur provide potetial applicatios, icludig but ot limit to: olie idetificatio of the fault locatio ad type [4], model reductio ad model verificatio [5], oscillatio moitorig This work was supported by the ERC Program of the NSF ad DOE (EEC-4877). ad dampig cotrol [6]-[8], early warig of istabilities [9]- [], dyamic security ad stability assessmets [], etc. Amog all the above PMU applicatios, the accurate classificatio of disturbaces is critical for power system state estimatio, protectio ad cotrol [4]. The existig methods to classify disturbaces are mostly based o time series aalysis or covetioal sigal processig theory. For example, i [5] a Wavelet based method is preseted for evet detectio by utilizig the PMU measuremets. I [6], a combiatio of the Empirical Mode Decompositio ad Spectral Kurtosis methods is itroduced for evet sigal characterizatio based o PMU data stream. They are efficiet i real-time detectio. However, to carry out more advaced system aalysis (e.g. system model reductio), the accuracy of curret disturbace classificatio approaches eeds further improvemet. To detect a disturbace as early as possible, it is useful to classify ambiet sigals. Aother issue of some existig methods is that they are depedet o operators experiece, for example, classifyig the disturbace type based o a predefied set of threshold values. I additio, durig the period of disturbaces, the latet system process is i fact timevaryig due to the drastic chages of the power system states. Therefore, the solutio provided by covetioal model-based or experiece-based methods is essetially a static characterizatio of the real system. Therefore, measuremet based methods such as machie learig ca be cosidered here. Approaches from the machie learig area have bee utilized by researchers i the power idustry durig the last te years [7], [8]. Deep Learig (DL) is oe of the most active areas i the field of machie learig. It has brought a revolutioary impact o the computer sciece area. Its powerful performace is iitially demostrated i the computer visio area such as object detectio, umaed automatic drivig, etc. Recetly, its applicatio has bee exteded ito other fields, e.g. time series aalysis [9], []. The objective of this paper is to utilize deep learig methods to idetify differet types of disturbaces. The mai cotributios of this paper iclude: ) a imagigtrasformatio based Deep Learig method is created for the power system trasiet disturbace classificatio problem; ) the performace of the proposed method is verified agaist two widely used covetioal data miig methods usig real PMU datasets.

2 The remaiig parts are orgaized as follows: a real PMUbased time series dataset is described ad illustrated i Sectio II; Sectio III explais a recet reported method called GAF (Gramia Agular Field) [] for image embeddig of the origial time series. Sectio IV itroduces the basic priciples of the two major DL architectures, i.e. CNN (Covolutioal Neural Network) ad RNN (Recurret Neural Network). Test results are preseted i Sectio V with comparisos agaist the results from the Support Vector Machie (SVM) ad Decisio Tree (DT) methods. Coclusios ad discussio about future work are provided i Sectio VI. II. PROBLEM DESCRIPTION A. FNET System The frequecy moitorig etwork system (FNET) [], operated by the Uiversity of Teessee Koxville ad Oak Ridge Natioal Lab (ORNL), is a easy-deploy WAMS with good dyamic accuracy ad low istallatio cost. The GPS sychroized Frequecy Disturbace Recorder (FDR) ca measure voltage magitude, agle, ad frequecy with a high precisio from V or 5V outlets. These measured sigals are calculated at ms itervals ad the trasmitted across the public iteret to a Phasor Data Cocetrator (PDC), where the measuremets are sychroized, aalyzed, ad archived. The FNET has bee deployed sice 4. More tha 5 FDRs have bee istalled i three North America itercoectios: Easter Itercoectio (EI), Wester Electricity Coordiatig Coucil (WECC), ad Electric Reliability Coucil of Texas (ERCOT), as show i Fig.. first three types of disturbaces are available i the give period (the whole moth of November 4). The FNET offers both frequecy ad agle data. I the followig study, oly agle data are utilized. I this study, a real dataset of 74 PMU measuremets (due to the copyright agreemet, the PMU statio ames/ids are omitted here). The origial data recordig legth is miute with Hz samplig rate, thus 6 data poits for each time widow. For practical applicatios, oe-mi-widow ca be too log for pre-warig, thus i this study the first sec sapshot of the PMU measuremets are utilized. Amog those data, 4 disturbace evets are geeratio trips; 45 evets are load-sheddig; ad 87 evets are system oscillatios. The dataset of each case icludes samples from multiple sesors, each of them with a certai rage of time stamp, frequecy value ad agle value. The dataset icludes geeratio trips, load sheddig ad oscillatios from the followig list. Each of the case icludes data from 4 to 8 FDR sesors. The plots of the three types of disturbace evets are show i Fig.. TABLE I. LIST OF MEASURED DISTURBANCE EVENTS Disturbace type Date / Time Geeratio Trip 4//7 Geeratio Trip 4// Geeratio Trip 4// Load Sheddig 4// Load Sheddig 4//4 Load Sheddig 4// Oscillatio 4// Oscillatio 4//5 5 Geerator Trip Agle /deg Load Sheddig Fig.. FNET deploymet i North America B. Disturbace Classificatio Disturbace classificatio usig PMU data has sigificat meaig for system operatio, protectio, cotrol ad postevet aalysis. Uder ormal operatig coditios, multiple frequecy measuremets across a sigle itercoectio are idetical ad the variatio of frequecy is little. However, uder disturbaces, the frequecy ad phasor agle suffer sudde chages i a short time. It is critical to idetify differet types of disturbaces for operators to locate the cause of a evet ad activate proper operatios agaist them. I the FNET database, there are basically five types of power system disturbaces: ) Geeratio trip; ) Load sheddig; ) System oscillatio; 4) Lie trip; 5) Isladig. I this paper, oly the Agle /deg Agle /deg System Oscillatio Fig.. PMU measuremets for the three differet types of disturbaces III. IMAGE EMBEDDING OF PMU DATA The ext step is to map each PMU data (a time series) ito a image, i.e. so-called image embeddig. Typically, this ca be achieved by applyig a certai oliear fuctio. A good mappig usually satisfies the followig property: ) preservig the temporal-order iformatio ) the mappig is oe-to-oe

3 ad its iverse image exists ) the computatioal burde is acceptable, as small as possible to beefit olie applicatios. As reported i [], the Gramia Agular Field (GAF) is a suitable choice. The math operatio ad defiitio of GAF are as follows: xi mi( X) x i, i,,..., max( x) mi( X) arccos( x ), x X; i,,..., i i i cos( ) cos( ) G cos( ) cos( ) cos( ) cos( ) where, X = [x, x x ] is the time-series with legth ; x is the i scaled value; i is the trasformed agle values i the latet polarcoordiatio system; G is the fial Gramia Agular Field matrix correspodig to the origial time-series X. The beefits of GAF are ) the time-order icreases from its top-left corer to bottom-right corer. Thus, the temporal iformatio (correlatios amog the differet time steps for each time series) ca bee preserved, i other words, it is a matrixify of the time-series ) it is easy to calculate. No itegral or covolutio operatios but just simple cosie fuctios ivolved. To better demostrate this Image Embeddig idea, three time-series samples for differet types of disturbaces are plotted i Fig., together with their correspodig GAF-mapped images by Eq. (). displayed more vividly i the embedded image space. Similar to the kerel fuctio of SVM, the image embeddig techique here itroduces a feature augmetatio beefit, which is helpful for the classifier performace improvemet. Moreover, this GAF based feature augmetatio techique oly ivolves simple triagular fuctio ad matrix arragemet operatios, thus the computatioal cost of such data trasformatio is acceptable. IV. BASIC PRINCIPLES OF DEEP LEARNING The priciples of CNN ad RNN are briefly itroduced i this sectio. As the two maistreams i the DL area, both methods have bee successfully deployed i may cuttig-edge applicatios such as image patter recogitio ad atural laguage processig. More details ca be referred to [], [4]. Both iclude a big family of various versios, here, to give a whole picture of the two structures, the most represetative oe for each structure will be itroduced i the followig paragraphs, i.e. LeNet for CNN ad LSTM (Log Short-Term Memory) for RNN. A. CNN ad LeNet itroductio LeNet is the first successfully applied CNN structure. It takes image iputs. Two importat cocepts are Covolutio ad Poolig operatios as explaied below. Poolig/Subsamplig: Poolig is a procedure that takes iput over a certai area ad reduces that to a sigle value (subsamplig), e.g., the Max- Poolig. It partitios the iput image ito a set of ooverlappig rectagles. The, for each sub-regio, it outputs the maximum value of that regio. It is actually a dimesio reductio process for the outputs of certai hidde layers. Covolutioal layer As show i Fig. 4, it is i fact a series of liear operatio o tesors/matrices to extract high-level hidde iformatio. Ge-Trip Load-Sheddig 4 Oscillatio Agle/ deg Fig.. Image embeddig examples for PMU time series From the mapped images, it is obvious that the differeces betwee the three classes have bee visually augmeted ad w w w w w w w w w w w w w w w Fig. 4. Example of a covolutioal layer w w w Fially, the LeNet structure is depicted i Fig. 5, where C meas Covolutioal layer; S meas poolig layer; F meas fully-coected layer (i.e. a layer with each of its eural uits coected to each elemet of its iput). w

4 Fig. 5. The complete CNN (LeNet-5) architecture [] x:iput of step t t s:cell state of step t t h : output of step t t : activatio fuctio ( sigmoid) W, U, b : weight matrices or vectors ( k f, i, o, s) k k k f, i, o : differet gate outputs of step t t t t B. RNN ad LSTM itroductio RNN adds iter-layer coectios ( weights ) amog each hidde layer. Thus, it ca be regarded as a ehaced versio of traditioal multi-layer eural etwork. It is especially suitable for learig sequetial data. A basic RNN uit [4] is show i Fig. 6. Fig. 7. The iteral structure of a basic LSTM cell uit Fig. 6. A basic RNN uit ad its urolled structure I the above diagram, a block of eural etwork, i.e. B, receives iputs x t ad gives outputs h t based o a recurretstyle traiig approach as show i its urolled structure. This allows iformatio to be passed from oe step of the etwork to the ext step. Oe mai drawback of traditioal RNN is the Log-Term Depedecy issue, i.e. with the time steps icreasig, the gradiet ivolved i traiig will be explodig or vaishig quickly. For this reaso, LSTM was proposed. Differet from simple RNN, LSTM tries to remember iformatio withi log time steps. More specifically, a basic LSTM uit has several blocks iteractig i a special way as show i Fig. 7. The iovatios of LSTM are maily i two ew cocepts, gate (σ) ad cell state (s t). Gates are used to optioally allow iformatio to pass through. Mathematically, it is simply a activatio fuctio (e.g. sigmoid) with poitwise multiplicatio operatios. The gates output values betwee ad, describig the portio of iformatio to be remembered. A value of meas forget, while a value of meas remember. A LSTM has three kids of these gates: a forget gate f t, a iput gate i t ad a output gate o t, to cotrol the cell states at step-t as show i Fig.7. The complete updatig equatios for oe basic LSTM uit (euro) used i above gates are summarized i Eq. (4). C. Architecture selectio cosideratio From the above descriptio of the basic priciples for the two architectures, CNN is more suitable to the problem with some spatial correlatio like images data; while RNN (LSTM) is more flexible i hadlig time series objects with temporal dyamics. However, the RNN structure ca be also applied to the image data by certai data-reshapig trick. More specifically, for a image expressed i a -by-m matrix, RNN ca treat each row of that image as oe time step of the total steps, while each step iput is a m-dimesioal vector; thus, the image data here is aalogous to a multivariate time series. I our study, the may-to-oe structure RNN is chose. It is illustrated i Fig. 8, where the iput (bottom-level blocks) represets each row of the image mapped from the previous GAF method. After all the image rows have bee scaed ad traied, the fial output is the class iformatio. More cocretely, the fial output is a oe-hot ecodig vector represetig the probabilistic distributio of differet classes for that time series. For example, for a -class problem, assumig the true class of oe such time series is type-. The, its oe-hot ecoded vector is [,, ]. The CNN or RNN output, for example, ca be [.,.,.5], of which the compoets sum to ad each compoet stads for the correspodig probability of each class. ft ( Wfxt U fht bf ) it ( Wx i t Uiht bi) ot ( Woxt Uoht bo) st ft st it tah( Wsxt Usht bs) ht ot tah( st) Where, stads for the ier product ad: Fig. 8. Differet RNN iput/output paradigms

5 Fially, the flowcharts of the procedures usig the SVM (or DT) method ad Deep Learig method are show i Fig. 9. A. CNN Results A modified LeNet is adopted here. The oly chages are ) modificatio of the umber of the st Covolutioal Layer uits from 6 to ; ) modificatio of the umber of output dimesios from to for our three-type classificatio problem. The specific meaig of each parameters is aotated, ad more details ca be foud i [5]. TABLE II. CNN RESULTS Traiig parameters values Batch size 64 Traiig epochs Learig rate. Mometum.9 Algorithm Stochastic Gradiet Descet Neuros i st Cov. Layer Accuracy (/ data for trai) 98.9% Accuracy (/4 data for trai) 97.85% Accuracy (4/5 data for trai) 97.% B. RNN Results TABLE III. RNN RESULTS Fig. 9. Compariso of the flowcharts of the SVM/DT method (left) ad the Deep Learig method (right) for disturbace classificatio V. CASE STUDY For simplicity, three sets of experimets are desiged here, i.e. usig respectively /, /4 ad 4/5 of the origial dataset for traiig, ad the remaiig data for testig. The tests are o a Desktop PC with Itel Core i7-77 CPU (.4GHz) ad o GPU. All the experimets here are implemeted i Google s TesorFlow ad Pytho.5. Data preprocessig ad postprocessig are doe i MATLAB. I the followig tables, Accuracy meas the metric value based o the testig dataset. Comparisos with DT ad SVM are preseted as well. The umerical results for each experimet are listed i Table II to Table IV. The accuracy compariso plot for all the three experimets is depicted i Fig.. Accuracy(%) Experimet No. DT SVM CNN RNN/LSTM Fig.. The accuracy compariso plot for all the experimets Traiig parameters RNN LSTM Hidde layer umber Neuros i hidde layer 8 64 Learig rate.. Traiig epochs 5 Algorithm Adam Adam Accuracy (/ data for trai) 99.9% 99.9% Accuracy (/4 data for trai) 98.9% 98.9% Accuracy (4/5 data for trai) 98.65% 98.65% C. Compariso with other methods The details regardig the meaig of DT ad SVM parameters ca be referred to [], [8]. TABLE IV. DT AND SVM RESULTS Traiig parameters DT SVM Kerel fuctios - RBF C, gamma -,. mi_samples_split - mi_samples_leaf - criterio Gii Idex - Accuracy (/ data for trai) 86.9% 98.9% Accuracy (/4 data for trai) 87.% 97.85% Accuracy (4/5 data for trai) 97.% 97.% VI. CONCLUDING REMARKS From the above results, the followig coclusios are obtaied: ) Both CNN ad RNN (LSTM) achieve more tha 98% accuracy for this three-type disturbace classificatio problem. ) RNN ad LSTM obtai the highest accuracy (e.g. 99.9% i experimet-) amog all the classifiers. LSTM

6 eeds less traiig epoch (e.g. ) tha simple RNN ad CNN (i.e. 5) to reach the same level accuracy durig traiig. This study provides a applicatio paradigm to coect the DL with covetioal power systems studies, where the DL outperform covetioal classificatio methods i this specific power system problem i terms of the accuracy. With the powerful classificatio ad regressio capabilities of DL techiques, it ca be evisioed that a large variety of potetial applicatios i power systems ca be exploited, e.g. cotroller desig ad model idetificatio. O the other had, with the rapid progress of DL techiques, more advaced architectures will be utilized o power system applicatios, e.g. Deoise Autoecoder (DAE), Deep Boltzma Machie (DBM), Geerative Adversarial Network (GAN), etc. ACKNOWLEDGMENT The authors would like to preset appreciatios for the PMU data provided by Dr. Yilu Liu s research group at the Uiversity of Teessee, Koxville. REFERENCES [] K. Wag, H. Li, Y. Feg ad G. Tia, "Big Data Aalytics for System Stability Evaluatio Strategy i the Eergy Iteret," i IEEE Trasactios o Idustrial Iformatics, vol., o. 4, pp , Aug. 7. [] Y. Dig ad J. Liu, "Real-time false data ijectio attack detectio i eergy iteret usig olie robust pricipal compoet aalysis," 7 IEEE Coferece o Eergy Iteret ad Eergy System Itegratio (EI), Beijig, 7, pp. -6. [] C. Dou, D. Yue, Q. L. Ha ad J. M. Guerrero, "Multi-Aget System- Based Evet-Triggered Hybrid Cotrol Scheme for Eergy Iteret," i IEEE Access, vol. 5, pp. 6-7, 7. [4] Y. Sog, W. Wag, Z. Zhag, H. Qi ad Y. Liu, Multiple Evet Detectio ad Recogitio for Large-Scale Power Systems Through Cluster-Based Sparse Codig, IEEE Tras. Power Syst., vol., o. 6, pp , Nov. 7. [5] Y. Lei, G. Kou, J. Guo, Y. Liu ad R. Nuqui, Easter Itercoectio model reductio based o phasor measuremets, 4 IEEE PES T&D Coferece ad Expositio, Chicago, IL, USA, 4, pp. -5. [6] K. Su, Q. Zhou, Y. Liu, A Phase Locked Loop-based Approach to Real-time Modal Aalysis o Sychrophasor Measuremets, IEEE Tras. Smart Grid, vol. 5, No., pp. 6-69, Ja. 4 [7] Y. Zhu, C. Liu, B. Wag, K. Su. Dampig Cotrol for a Target Oscillatio Mode Usig Battery Eergy Storage. J. Mod. Power Syst. Clea. Eergy. [8] Feifei Bai, et al, Desig ad Implemetatio of a Measuremet-based Adaptive Wide-Area Dampig Cotroller Cosiderig Time Delays, Electric Power Systems Research, vol., pp. -9, Jauary 6 [9] J. Qi, K. Su, W. Kag, Optimal PMU Placemet for Power System Dyamic State Estimatio by Usig Empirical Observability Gramia, IEEE Tras. Power Systems, vol., pp. 4-54, July 5. [] F Hu, K Su, et al, Measuremet-Based Real-Time Voltage Stability Moitorig for Load Areas, IEEE Tras. Power Systems, vol., No. 4, pp , July 6 [] K. Su, S. T. Lee, P. Zhag, A Adaptive Power System Equivalet for Real-time Estimatio of Stability Margi usig Phase-Plae Trajectories, IEEE Tras. Power Systems, vol. 6, pp. 95-9, May [] Y. Liu, K. Su, Y. Liu, A Measuremet-based Power System Model for Dyamic Respose Estimatio ad Istability Warig, Electric Power Systems Research, vol. 4, pp. -9, [] C. Liu, K. Su, et al., A systematic approach for dyamic security assessmet ad the correspodig prevetive cotrol scheme based o decisio trees, IEEE Tras. Power Syst., vol. 9, o., pp. 77-7, Mar. 4. [4] D. Zhou, Y. Liu ad J. Dog, "Frequecy-based real-time lie trip detectio ad alarm trigger developmet," 4 IEEE PES Geeral Meetig, Natioal Harbor, MD, 4, pp. -5. [5] D. I. Kim, T. Y. Chu, S. H. Yoo, G. Lee ad Y. J. Shi, "Waveletbased evet detectio method usig PMU data," 7 IEEE Power & Eergy Society Geeral Meetig, Chicago, IL, 7, pp. -. [6] S. S. Negi, N. Kishor, K. Uhle ad R. Negi, "Evet Detectio ad Its Sigal Characterizatio i PMU Data Stream," i IEEE Trasactios o Idustrial Iformatics, vol., o. 6, pp. 8-8, Dec. 7. [7] Y. Zhu., S Lu, "Load Profile disaggregatio by Blid Source Separatio: A Wavelets-assisted Idepedet Compoet Aalysis Approach," I Proc. the IEEE PES Geeral Meetig, Washigto D.C., USA, July 7-, 4. [8] Y. Zhu, R. Azim, H. A. Saleem, K. Su, D. Shi ad R. Sharma, "Microgrid security assessmet ad isladig cotrol by Support Vector Machie," 5 IEEE PES Geeral Meetig, Dever, CO, 5, pp. -5. [9] M. Dalto, "Deep eural etworks for time series predictio with applicatios i ultra-short-term wid forecastig". [Olie]. Available: [] G. S. Babu, P. Zhao, X. Li, "Deep Covolutioal Neural Network Based Regressio Approach for Estimatio of Remaiig Useful Life," i Proc. Database Systems for Advaced Applicatios: st Iteratioal Coferece, Dallas, TX, USA, April 6-9, 6. [] Z. G. Wag ad, T. Oates, "Ecodig Time Series as Images for Visual Ispectio ad Classificatio Usig Tiled Covolutioal Neural Networks," i Proc. Workshops at the Twety-Nith AAAI Coferece o Artificial Itelligece, Austi, TX, USA, Jauary 5, 5. [] FNET/GridEye Web Display, [Olie]. Available: [] Covolutioal Neural Networks (LeNet) tutorial. [Olie]. Available: [4] C. Olah, "Uderstadig LSTM Networks". [Olie]. Available: [5] I. Goodfellow, Y. Begio ad A. Courville, Deep Learig. Cambridge: MIT Press, 6.