NEURO-FUZZY TECHNIQUES FOR SYSTEM MODELLING AND CONTROL

Transcription

1 Paper presented at FAE Symposum, European Unversty of Lefke, Nov 22 NEURO-FUZZY ECHNIQUES FOR SYSEM MODELLING AND CONROL Mohandas K P Faculty of Archtecture and Engneerng European Unversty of Lefke urksh Republc of North Cyprus (emal : kpmdas@eee.org) Keywords ; fuzzy logc, neural networks, neuro-fuzzy systems, modelng,control ABSRAC Fuzzy logc and neural network systems have recently proved to be very useful n the dentfcaton of complex systems. Results on modellng of systems usng back propagaton neural networks wth an extended Kalman type updatng of the weghts, modelng and control of nonlnear systems usng partal recurrent networks and adaptve neuro-fuzzy systems are dscussed. Smulaton results show that the adaptve neuro-fuzzy systems are superor to others. However, the computatonal effort s more. Search for algorthms that can be used for on-lne dentfcaton of systems s to be pursued to make these practcally useful. I. INRODUCION echnques and methods for modelng and dentfcaton of systems have gone through radcal changes n the last few years. he conventonal technques of fndng a sutable mathematcal model for a system from the nputoutput data or system dentfcaton follow the four famlar steps of selectng a class of models, determnaton of the structure of the model, fndng the parameters of the chosen model and fnally testng for the adequacy of the model for the purpose for whch t s derved [-2]. It has been proved tme and agan that ths procedure s nsuffcent for a large number of systems. Frst, the choce of the class of models was confned to those, whch could be easly dentfed, rather than the model that wll be most accurate. hs resulted n the choce of lnear tme -nvarant models for many systems. Even after a class of models was decded, determnng the structural parameters of the model was a formdable task, partcularly for real lfe, and nosy data. Many of the structural parameter dentfcaton schemes are hghly senstve to the nose and even when the sgnal to nose rato was reasonably good, many of these technques fal to dentfy the common structural parameter, the order of the system[3-4]. Because of these reasons, search for better and more realstc modelng technques was beng pursued. Nonlnear models were attempted. But dentfcaton of nonlnear systems have not been very successful as many of the reported works are applcable to specfc cases and even now, general technques are dffcult to fnd [5]. he advent of fuzzy logc systems [6] and neural networks [7] has created a new horzon n the area of system modelng and dentfcaton. he neural networks, n partcular, are known to be unversal functon approxmatons so that almost any type of nput output data can be used as tranng samples to a neural network and usng ts capablty of learnng from examples, the artfcal neutral networks (ANN) could be used for dentfcaton of any system. However, early stages n ANN development usng back propagaton networks suffered from very slow convergence and a large amount of computatonal effort n dentfyng the model [8]. he back propagaton networks are essentally statc models and modelng of the dynamcs usng them was not convenent. Use of better neural network structures such as recurrent and partal recurrent networks and radal functon networks has sgnfcantly eased the computatonal burden and neural networks can be used as effectve tools for dentfcaton of dynamc systems. Fuzzy systems, on the other hand can be used for modelng nonlnear characterstcs [9-2]. But here, the choce of the type of membershp functons s not very often transparent. he combnaton of the neural networks and fuzzy logc together called adaptve neurofuzzy nference systems (ANFIS) proposed by J S Jang[3] has been shown to be very effectve n dentfyng a model for a gven nput output data, Here, the membershp functons are tuned to the nput output data and excellent results are possble. In ths paper, some experence on applcaton of neural networks and fuzzy logc systems for dentfcaton and control of systems s reported from the work of the author and hs students. Frst, results of mprovng the rate of convergence of back propagaton algorthm by usng an extended Kalman type learnng technque are reported [4]. Applcaton of partal recurrent neural networks for dentfcaton and control s dscussed n [ 5 ] and the applcaton of an adaptve neuro-fuzzy technque s presented n [6 ]. From these studes, some useful comments on mplementaton of these technques are

2 presented. A sgnfcant number of recent references useful for any researcher are added. II. NEURAL NEWORKS FOR IDENIFICAION Because of the lmtatons of the classcal model buldng technques ndcated above, attempts were made to avod a restrctve mathematcal model by selectng a partcular class of models. he approach, some tmes called, modelfree approach s essentally an extenson of the method of fttng a curve or fndng a sutable relaton between the nput and output data avalable from experments on the system. hs approach receved an mpetus from the developments n the areas of neural networks and fuzzy logc as these are known to be model-free approaches. he capablty of neural networks to learn from examples as human bengs do routnely seemed to make t an deal choce for modelng and dentfcaton of complex dynamc systems. In the lterature, several types of neural networks are avalable [7]. However, n the current stuaton, the neural networks of nterest are the Backpropagaton networks and the recurrent networks. Back propagaton Neural Networks Mult-layer feed forward networks were hstorcally the frst to be used for dentfcaton purposes even though they had ther own lmtatons. A typcal multplayer network has an nput layer, one or more hdden layers and an output layer wth a certan number of nodes n each layer. From a system theoretc pont of vew, multplayer networks or feed forward networks can be consdered as versatle nonlnear maps wth elements of the weght matrces as parameters. he back propagaton network, whch remaned the most popular neural networks for a long tme, s based on an error correcton prncple. he algorthm starts out by assgnng a random set of weghts to the artfcal neural network (ANN). he network adjusts ts weghts each tme t comes across an nputoutput par. Each par requres two passes a forward pass and a backward pass. he forward pass nvolves presentng a sample nput to the network and lettng actvatons flow untl they reach the output layer. Durng the backward pass, the networks actual output (from the forward pass) s compared wth the target output and error estmates are computed for the output unts. he weghts connected to the output unts can be adjusted n order to reduce those errors. he error estmates of the output unts are used to derve error estmates for the unts n the hdden layers. Fnally errors are propagated back to the connectons stemmng from the nput unts and the correspondng weghts are adjusted accordngly. After an ANN has seen all the nput-output pars and has adjusted ther weghts many tmes, one epoch s sad to have completed. ranng a back propagaton neural network usually requres many epochs [8]. he major drawbacks of the requrement of the large number of epochs for tranng the ANN[ 8 ] can be overcome by modfyng the learnng rule. he archtecture of a mult-layer feed forward (MFNN) s shown n Fg.. A nonlnear Sngle Input Sngle Output (SISO) system can be descrbed by : yk ( ) = f ( yk ( ), yk ( 2),..., yk ( n), uk ( ), uk ( 2),... uk ( m)) () where y(k) and u(k) are measured output and nput at the k-th nstant and n and m are known structural parameters of the model. Nonlnear system dentfcaton problem can be defned as determnng an estmate of the unknown nonlnear functon f(.) usng the measured nput and output sequences {u(k)} and {y(k)}. Let: xk ()[( = yk ) yk ( 2)... yk ( n) uk ( ) uk ( 2)... uk ( m)] (2) hen, the neural network estmate of f(.) s : L yk ˆ( ) = fxk ˆ[ ( )] = w( k) φ [ xk ( )] (3) = and the usual sgmod functon s used as logcal actvaton functon. he network archtecture s shown n Fg.. he conventonal method of updatng the weghts Input layer Fg.. Back propagaton neural network of the neural network uses Least Mean Square (LMS) algorthm whch has been shown to be very slow n ts convergence [8 ]. Consder the cost functon to be mnmzed for a total of N nput-output samples as: ζ av w = d n yn 2N 2 ( ) [ ( ) ( )] (4) he extended Kalman algorthm s derved as follows usng the state and measurement equatons: w( n+ ) = w( n) (5) hdden layer d ( n) = φ( X ( nwn ) ( )) + e (6) where X(n) s the nput vector of neuron. w(n) s the weght vector. Expandng (6) as a aylor seres about the current estmate w(n) and thereby lnearzng the actvaton functon: Adjust weght output layer

3 φ( X ( nwn ) ( )) = q ( n) w( n) + [ φ( X ( nwn ) ˆ( ) q ( nwn ) ˆ ( )] (7) where φ ( x ( n) w( n)) ( ) = [ ] w( n) = wˆ ( n) w( n) q n usng the sgmod functon as the actvaton functon we get : q ( n) = yˆ ( n)[ yˆ ( n)] X ( n)...(8) he frst term on the rght sde of eqn (7) s the desred lnear term and the remanng terms represent the modelng error. If ths s neglected, we get from (6): d ( n) = q ( n) w( n) + e ( n) (9) eqns (5) and (9) descrbes the lnearsed dynamc behavour of the neuron. he measurement error e(n) n eqn (9) s a localzed error the nstantaneous estmate of whch s gven by : ζ ( n) en ( ) = -- () y( n) Dfferentaton of () corresponds to the back propagaton of the global error to the output neuron. From eqn(5) and (9) the standard recursve least squares algorthm s used to make an estmate of the synaptc weght vector w(n) of th neuron, he resultng soluton s defned by the followng recursve equatons [ 4 ] r( n) = P( n) q( n)...() k n r n r nq n ( ) = ( )[ + ( ) ( )]...(2) w( n+ ) = w( n) + e( n) k( n)...(3) Pn ( + ) = Pn ( ) knr ( ) ()...(4) n Recurrent Neural Networks For the purpose of dentfcaton and control of dynamc systems (both lnear and nonlnear) t s requred that the ANN used also be dynamc. If the order of the system or ts upper bound s known, all the necessary past nputs and outputs of the system beng modeled can be fed as explct nputs to the network. hs wll facltate the neural network to learn the memory less transformatons that capture the dependence of the past outputs on the past nputs and outputs. he man dsadvantage of the multplayer feed forward network s that ether the system order should be exactly known or a large number of data pars are requred for tranng of the ANN. It has been found that for nonlnear systems, the computatonal effort s enormous. A recurrent neural network has at least one feed back loop. he feed back loops nvolve the use of unt delay elements, whch results n provdng the dynamc behavor to the network structure. Partal recurrent neural networks (PRNNs) fll the requrement admrably. he commonly used PRNNs are Jordan Network wth feed back from output layer to the nput layer and the Elman network whch has feedback from hdden layer to the nput layer. he PRNNs are less B W unt delay W2 B2 lnear act.fn where n =,2,3, N where N s the total number of samples n the tranng set. P( ) s the covarance matrx and k(n) s the Kalman gan. o start wth P can be a dagonal matrx wth large elements. A results of applyng ths type of learnng rule [4 ]has been shown to sgnfcantly reduce the number of epochs requred for learnng [8 ]. For example, a SISO nonlnear system requred 85 epochs for reachng a target performance ndex of.8. A MIMO nonlnear system, a hghly nonlnear antenna trackng system requred 799 epochs as aganst 54 epochs reported for ordnary back propagaton algorthm. However, t should be remembered that the back propagaton neural networks are essentally statc nput output data nterpolaton tools and cannot be used effectvely for dentfyng the characterstcs of dynamc systems. he ANNs wth feedback between layers, called recurrent networks are useful for dentfyng dynamc systems. Fg.2. Elman Neural Network complex than recurrent networks such as Hopfeld networks and the dynamcs of the plant can be ncorporated nto the network structure at least to a certan extent. Results on applcaton of Elman type PRNNs for dentfcaton have shown that these are very useful n the dentfcaton of lnear and nonlnear dynamc systems. Both Sngle Input Sngle output systems and Multple Input Multple output systems have been successfully dentfed usng these networks [5 ]. III.FUZZY AND NEURO-FUZZY MODELLING he fuzzy logc s closer n sprt to human thnkng and natural language than conventonal logcal systems. hs provdes a means of convertng a lngustc control strategy based on expert knowledge nto an automatc control strategy.[ ] he ablty of fuzzy logc to handle mprecse and nconsstent real-world data made t sutable for a wde varety of applcatons. In partcular,

4 the methodology of the fuzzy logc controller (FLC) appears very useful when the processes are too complex for analyss by conventonal quanttatve technques or when the avalable sources of nformaton are nterpreted qualtatvely, nexactly, or wth uncertanty.[ ] hus, fuzzy logc control may be vewed as a step toward a rapprochement between conventonal precse mathematcal control and human lke decson makng. Fuzzy models of dfferent types can be used to approxmate the state-transton functon. As the state of the process s not usually measured, nput-output modelng s often convenent. he most common among the nput output models s the NARX (Nonlnear auroregressve wth exogenous nput) model descrbed n eqn () above. Snce fuzzy models can approxmate any smooth functon to any degree of accuracy, models of the above type can approxmate any observable and controllable modes of a large class of dscrete tme nonlnear systems. However, buldng fuzzy models s possble only wth pror knowledge and data (measurements). Buldng fuzzy models from data nvolves methods based on fuzzy logc and approxmate reasonng, but also deas orgnatng from the feld of neural networks, data analyss and conventonal technques of system dentfcaton. he dentfcaton nvolves both structure and parameter estmaton. he structure determnes the flexblty of the model n the approxmaton of the unknown mappngs. he parameters are then tuned to ft the data avalable. After the structure s fxed, the performance of a fuzzy model can be fne tuned by adjustng the parameters. unable parameters are the parameters of the antecedent and consequent membershp functons and the rules. Neuro-fuzzy Modelng In order to optmze the parameters whch are lnearly related to the output n a nonlnear way, tranng algorthms known from the neural networks and nonlnear optmzaton can be employed. hese technques explot the fact that at the computatonal level a fuzzy model can be seen as a layered structure (network) smlar to artfcal neural network. Hence ths approach s called neuro-fuzzy modelng. Recently, t was suggested by Roger Jang et al. [3 ] that an archtecture called Adaptve Network based Fuzzy Inference System or Adaptve Neuro Fuzzy Inference system can be used effectvely for tunng the membershp functons.. ANFIS can serve as a bass for constructng a set of fuzzy f then rules wth approprate membershp functons to generate the stpulated nput-output pars. Fundamentally, ANFIS s about takng an ntal fuzzy nference (FIS) system and tunng t wth a back propagaton algorthm based on the collecton of nputoutput data. In prncple, f the sze of avalable nputoutput data s large enough, then the fne-tunng of the membershp functons are applcable (or even necessary). Snce the human-determned membershp functons are subject to the dfferences from person to person and from tme to tme; they are rarely optmal n terms of reproducng desred outputs. However, f the data set s too small, then t probably does not contan enough nformaton of the system under consderaton. In ths stuaton, the human-determned membershp functons represent mportant knowledge obtaned through human experts experences and t mght not be reflected n the data set; therefore the membershp functons should be kept fxed throughout the learnng process. Interestngly enough, f the me mbershp functons are fxed and only the consequent part s adjusted, the ANFIS can be vewed as a functonal-lnk network, where the enhanced representaton of the nput varables are acheved by the membershp functons [3]. Practcal Consderatons In a conventonal fuzzy nference system, an expert who s famlar wth the system to be modeled decdes the number of rules. In the smulaton, however, no expert s avalable and number of membershp functons (MF s) assgned to each nput varable s chosen emprcally,.e., by examnng the desred nput-output data and /or by tral and error. After the number of MF s assocated wth each nputs are fxed, the ntal values of premse parameters are set n such a way that the MF s are equally spaced along the operatng range of each nput varable. IV. MODELLING AND CONROL Neural Networks for Control and Identfcaton he capablty of neural networks to dentfy a model from nput-output measurements of a system can be used for modelng and control of systems. It can be used to determne the mathematcal model of a real system to be controlled. It can also be used to desgn a controller once a model of the system to be controlled s avalable. If the model s an accurate representaton of the real system and the controller has been desgned correctly, then the controller wll perform satsfactorly on the real system as well. Both of these can be smulated usng ANNs. he ablty of the neural networks to model system whch are nonlnear makes ths stuaton more attractve as the mathematcal descrpton of a nonlnear system cannot always help n dervng a satsfactory controller for the system. he use of recurrent neural network for control and dentfcaton of a nonlnear system has been reported n [5 ]. he confguraton of the neural networks controller s shown n Fg.3. wo neural networks are ref nput nput NNCON nput ref NNPLN MODEL y ym

5 Fg.3.Neural Networks for modelng and control able I. Comparson of performance Nonln. elment RELAY LIMIER D. ZONE +SA.N Controller PID PRNN PID PRNN PID PRNN Delay me Rse me Peak me Max overshoot Settlng me SSE PID Prop + Integral + Dervatve Controller: PRNN Partal Recurrent N.Network used, one an dentfyng neural network for determnng the model for the plant from nput output data and another, a controller neural network whch desgns a controller from the knowledge of the desred output and actual nput of the controlled system. It has been shown that [5] very good results are obtaned for SISO and MIMO nonlnear systems usng such a controller plant confguraton. Comparson of the performance specfcatons for a typcal system are shown tabulated n able I for three typcal nonlnear elements deal relay lmter, dead zone + saturaton n cascade wth a second order plant. Adaptve Neuro-fuzzy Control Due to the adaptve capablty of ANFIS, ts applcatons to adaptve control and learnng control are mmedate. Most of all, t can replace almost any neural networks n control systems to serve the same purposes. For a controller to be desgned, a model of the system s requred. he desgn can be done usng conventonal methods or ANFIS. In the former case, a mathematcal model wll be requred, whle the latter wll be convenent f an dentfed ANFIS model of the system s avalable. he structure of the controller usng ANFIS can take the schematc shown n Fg.4. wo ANFIS networks are used. ref CANFIS SANFIS Fg.4. ANFIS Control and Modellng he frst one, called Controller ANFIS (CANFIS) s traned usng the nput output data of the controller as per the desgn specfcatons. If the mathematcal model of the plant s not avalable, a second ANFIS can be traned from the expermental nput output data from the plant and the traned ANFIS can be used n place of the model.[6]. Smulaton Results: he nput-output data pars for tranng the CANFIS and SANFIS were generated usng a conventonal PID controller. ypcal plot of the step response of the nonlnear closed loop control system wth a PID lke fuzzy controller s shown n Fg.5. See the hghly oscllatory nature of the uncontrolled system. Step response wth a conventonal PID controller and fuzzy PID controllers are also shown n Fg.5. he mprovement acheved by usng ANFIS type controller of Fg.4. s shown n Fg.6.Here, t s seen that the step response of the system s almost dead-beat type wth practcally zero rse tme and no overshoot. hs s a remarkable achevement for a nonlnear system. It s nterestng to see how the membershp functons are adapted to the nput output data, whch s shown n Fgs7a and 7b. It s obvous from these that proposed ANFIS controller s nearer to the deal one. Performance-wse, ANFIS confguraton s far superor to the conventonal PID and fuzzy controllers dscussed n [4]. Obvously the penalty s the addtonal computatonal effort n tranng the two ANFIS networks [6] >.4 Output Response of the Plant Nonlnear system wthout controller 2.Nonlnear system wth conv. PID controller 3.Nonlnear system wth Fuzzy PID controller > Number of Samples Wth s=..2 Fg.5.Comparson of Step Response of controllers Step response wth ANFIS for NLS (Saturaton as NLE)

6 the plant model s also not known. However, all these methods are essentally off-lne methods that requre collecton of nput output data for tranng the neural and neuro-fuzzy systems. Algorthms that can be used for real tme dentfcaton and control are the future drectons n whch sgnfcant work needs to be done. Fg.6. Step Response of ANFIS controller V CONCLUSIONS he neural networks are deal systems, whch can dentfy the nput output model of lnear or nonlnear systems wthout assumng a formal mathematcal descrpton. he popular mult-layer feed forward networks are not sutable for dynamc system Fg.7.a. Intal Fuzzy Membershp Functon -ANFIS Intal Membershp Functons of NLS Fg.7.b.Fnal Fuzzy Membershp Functon - ANFIS dentfcaton as there s no feedback and the slow convergence results n sgnfcant computatonal effort. An extended Kalman type algorthm can be used nstead of the gradent method, but dynamcs of the plant requre use of the recurrent networks. he use of partal recurrent networks lke Elman network can mprove the stuaton sgnfcantly. However, the ablty of fuzzy systems to use human expertse can be used to advantage n a neuro-fuzzy system. Applcaton of the combned dentfcaton and control usng neural networks and neuro-fuzzy systems can be used when VI. REFERENCES [] Astrom K J and Eykhoff P : System Identfcaton - A Survey, Automatca, Vol 7, 97, [2] Lennart Ljung : System Identfcaton theory for the user, Prentce Hall PR, 999 [3] Unbehauven H and Gohrng B: est for determnng model order n parameter estmaton, Automatca, Vol,974, [4] se E, and Wenert H L: Structure Determnaton and Parameter Identfcaton of Multvarable Stochastc Lnear Systems, IEEE rans. On Automatc Control, AC-2 (5), 975, [5] Bllngs. S.A: Identfcaton of Nonlnear Systems A survey: IEE Proceedngs, D, No 27, 98,pp [6] Zadeh L A : Outlne of a new approach to the analyss od complex systems and Decson processes, IEEE rans on Systems, Man and Cybernetcs, Vol, 973,pp [7] Haykn Smon : Neural networks, A Comprehensve foundaton, IEEE Press, 994. [8] Narendra K S and Parthasarathy K : Identfcaton and Control of Dynamc Systems Usng Neural Networks, IEEE rans on Neural Networks, Vol, No, March 99, pp 4 27 [9]Mamdan E H : Applcatons of fuzzy Algorthms for control of smple dynamc plants, Proceedngs of IEE, No 2, 974, pp [] akag, Sugeno M :Fuzzy dentfcaton of systems and ts applcaton to modelng and control,, IEEE rans on Systems, Man and Cybernetcs, Vol 5, No, 985, pp [] Babuska R and Verbruggen H B : An overvew of fuzzy modelng and control, Control Engneerng Practce, Vol 4, 996, [2] Verbruggen H B and Babuska R (Eds): Fuzzy Logc Control, Advances n Applcatons, World Scentfc, Sngapore, 998 [3] Jang J S R: ANFIS, Adaptve Network based Fuzzy Inference Systems,, IEEE rans on Systems, Man and Cybernetcs, Vol. 23, No 3, May,993, pp [4] Mohandas K P, and Jason Joseph: Extended Kalman flter speeds up Neural Networks Identfcaton of Nonlnear Systems, Proceedngs of the Natonal conference on Neural Networks, Chenna, July 997, pp59-67 [5] Mohandas K P and Deepthy A : Partal Recurrent Networks for Identfcaton and Control of Nonlnear

7 Systems, Proceedngs of the IASED Internatonal Conference on Control and Applcatons, Honolulu, USA, Aug,998, pp [6] Mohandas K P and Shak Karmulla : Fuzzy and Neuro-fuzzy Modellng and Control of Nonlnear Systems, Proceedngs of the Second Internatonal Conference on Electrcal and Electroncs Engg (ELECO 2) Bursa, urkey, 2, pp 97-2.