Hierarchical Structure for function approximation using Radial Basis Function

Transcription

1 Herarchcal Structure for functon appromaton ung Radal Ba Functon M.Awad, H.Pomare, I.Roja, L.J.Herrera, A.Gullen, O.Valenzuela Department of Computer Archtecture and Computer Technology E.T.S. Ingenería Informátca. Unverty of Granada Span Abtract: - The herarchcal network propoed (Mult-RBFNN), compoed of complete Radal Ba Functon Neural Network (RBFNN) that are n charge of a reduced et of nput varable wth the property of whch every Sub-RBFNN can take charge of a et of nput varable and not of all. For the optmzaton of the whole net, we propoe a new method to elect the more mportant nput varable, whch capable of decdng whch of the choen varable go alone or together to a Sub-RBFNN to buld the herarchc tructure Mult-RBFNN, thu reducng the dmenon of the nput varable pace for each RBFNN. We alo provde an algorthm whch automatcally fnd the mot utable topology of the propoed herarchcal tructure and elect the more mportant nput varable for t. Key-word: Herarchcal archtecture, Input varable electon, Functon appromaton, Radal ba functon.. Introducton RBFNN unveral appromaton, t correpond to a partcular cla of functon appromaton that can be traned ung a et of ample of I/O. On the other hand, when the tructure of a RBFNN comple (not herarchcal), a drect conequence the producton of a bg network, wth a bg number of hdden unt that t produce a bg quantty of computatonal parameter and th mpede the convergence of the learnng proce and ncreae the requrement of memory and tme. An effectve oluton to ncorporate a herarchcal tructure adapted to olve problem of comple model. The dea of modulatng neural network are eental for herarchcal degn [][2][3][4]. In problem of functon appromaton, when ncreae the number of nput varable, the number of parameter ncreae eponentally. A drect conequence to have a huge network, wth bg number of neuron n the hdden layer that hnder the convergence of the proce of tranng of the network. The herarchcal tructure of radal ba functon network propoed (Mult-RBFNN) dvde at frt the problem of the functon appromaton n maller problem baed on the more mportant nput varable that have been elected and whch of thee elected varable wll be go alone or together n a Sub-RBFNN. The fnal tructure a herarchcal ere of RBFNN connected n parallel wth an total output that the lnear um of all the output of Sub-RBFNN. Th topology facltate to olve problem that can be combned to gan acce to a comple oluton. Once the number of Sub-RBFNN known, the herarchcal tructure Mult- RBFNN contructed. In every Sub- RBFNN, whch preent a radal ba functon network (RBFNN), they optmzed the parameter of the RBFNN (centre, rado, weght); ung an effcent algorthm of cluterng to ntalze the value of the centre n every Sub-RBFNN. To optmze the value of the radu n every Sub- RBFNN, we ued tradtonal algorthm. When the parameter of centre and radu of every Sub-RBFNN have been ntalzed, a method of lnear calculaton ued to fnd the eact value of the weght n the whole herarchcal ytem that mnmze the cot functon calculated on the et of ample of I/O data.

2 2. The Archtecture Of The Mult-RBFNN Sytem In RBFNN every neuron n the hdden layer receve all the nput varable of the network. Neverthele, the nterconnecton n the herarchcal tructure Mult-RBFNN between nput varable and the hdden layer are lmted and located. The advantage of the herarchcal tructure Mult-RBFNN cont of the fact that the problem dvde nto many problem that connected n parallel. Every problem preent a RBFNN named Sub-RBFNN. All the Sub-RBFNN ha a total output that the output of the herarchcal tructure Mult-RBFNN. Th dvon of the ytem Mult-RBFNN lmt the quantty of the nformaton of the prevou layer. In general, to contruct a herarchc tructure Mult-RBFNN to olve problem of functon appromaton cont of two bac tep: The dentfcaton of the tructure (nput varable electon, dtrbuton of the elected nput varable to the number Sub-RBFNN, the number of Sub-RBFNN depend on the number of the elected nput varable and on whch of thee varable go alone or together n a Sub-RBFNN). The etmaton of the parameter of every Sub-RBFNN (centre c S, radu S r and weght w, and the number of radal functon RBF n each Sub- RBFNN), and the calculaton of the total output f() of the herarchcal tructure Mult-RBFNN. Fg. preent the prncpal tep of the propoed algorthm. Fg. 2 preent the propoed herarchcal Mult-RBFNN ytem. Each one of the node of Fg. 2 a Radal Ba Functon Network (ee Fg. 3). RBFNN can be een a a partcular cla of Artfcal Neural Network ANN. They are characterzed by a tranfer functon n the hdden unt layer havng radal ymmetry wth repect to a centre. The bac archtecture of an RBFNN a 3- layer network. The output of the net gven by the followng epreon: The output of the net gven by the followng epreon: m F(, Φ, w) = φ ( ) w () = where Φ = { φ : =,..., m} are the ba functon et and w the aocate weght for every RBF. The ba functon φ can be calculated a a gauan functon ung the followng epreon: c φ( cr,, ) = ep (2) r where c the central pont of the functon φ and r the radu. Select the mot mportant nput varable, ung the method propoed n th paper. (Sec.3) Select the varable that go alone or together to a Sub-RBFN (Sec.3) Optmze the Parameter of every Sub-RBFN (Centre, Radu, and Weght). (Sec.4) Optmzed the utable number of ba functon RBF n every Sub-RBFN. (Sec.4) Calculate the total output f() of the tructure Mult-RBFN and mnmze the appromaton of the ytem Mult-RBFN. (Sec.4) Fg.. Prncpal tep of the propoed algorthm d 2 2 d2 d RBFN RBFN2 RBFNS F () F 2 () F S () Fg. 2. The herarchcal tructure Mult-RBFNN. Σ f()

3 . m φ φ 2 φ m w w 2 w m Fg. 3. Each of the ub-network a RBFNN Σ F(X) The calculaton of the weght doe not depend on every output of every Sub- RBFNN {F (),,F S ()}, but t depend on the total output of the ytem Mult- RBFNN, and mut be calculated n the lnear form lke n the followng epreon: S m f (, Φ, w) = φ w = = ( ) (3) Φ= { φ : =,..., n, S=,..., } Where the of actvaton matr of the et of ba functon n all Sub-RBFNN, w the aocate weght of all Sub-RBFNN and S the number of Sub-RBFNN. The proce of the lnear optmzaton of the weght depend to the actvaton matr of the total output of the Mult-RBFNN f ( ). Th proce ue the drect method a the ngular value decompoton (SVD) to calculate the value of the weght w. The propoed herarchcal tructure Mult- RBFNN decreae the number of parameter whch decreae the complety of the appromate ytem and ncreae the effcency of the proce of functon appromaton from a et of eample of I/O data data. 3. Input Varable Selecton For The Structure Mult-RBFNN The nput varable electon (IVS) tre to reduce the dmenon of varable of nput pace and create a new et of nput varable. Th proce of dentfcaton and elmnaton of o much rrelevant and redundant nformaton a they are poble, reduce the dmenonalty of the date et and allow algorthm of learnng work more rapd and effectvely. The cure of the dmenonalty [5] refer to the eponental appromaton of the hypervolume a a functon of dmenonalty. RBFNN can be planned a nterrelaton of nput pace to output pace, t have to cover or repreent each part of h nput pace n order to know how that part of the nput pace hould be mapped. Coverng the nput pace take reource, and n the mot general cae, the amount of reource needed proportonal to the hyper-volume of the nput pace. The eact formulaton of reource and part of the nput pace depend on the type of the network and hould probably be baed on the concept of nformaton theory and dfferental geometry [6].The cure of the dmenonalty caue network wth many rrelevant nput that behave relatvely badly, when the dmenon of the nput pace hgh the network ue almot all h reource to repreent rrelevant part of the nput pace. Even f we have a network algorthm whch able to focu on mportant porton of the nput pace, the hgher the dmenonalty of the nput pace, the more data may be needed to fnd what mportant and what not. A pror nformaton can help wth the cure of dmenonalty. Input varable electon fundamentally affect the everty of the problem, a well a the electon of the neural network model. [7]. In th ecton we propoe a new method for nput varable electon for the problem of functon appromaton, and more pecfcally for our Mult-RBFNN ytem. Th method conder a mple calculaton to elect the nput varable. The electon of the nput varable wll do by the followng tep: ) Relate each poble nput dmenon of data {,, d } wth the dependent varable y (a a functon n one dmenon), a n the followng epreon: (, y),(, y),(, y),...,(, y ) { } 2 3 d

4 2) Dvde the date n each dmenon n P part (the number of the part depend on the number of the nput data n, y when the number of the nput data bg, and the number of part P mut ncreae). Th dvon obtaned by mean of the followng epreon: j k j { ( ) } P < P k =,, n; =,..., d; j =,..., p k where n the number of data of I/O, ( ) t the component th of the nput vector k th. 3) Aocate the data of each dmenon to correpondng output data a n the followng epreon: k, k j k j y P < P {( ) } ( ) 4) Ue the Kalman flter to mooth the vector of the mamum and mnmum n each part, and calculate the dtance D between the mamum and the j mnmum value of the output n each partton of the nput varable : j k k D = ma( y ) mn( y ) j =,... p j j 5) Fnally, for each nput varable we calculate the mean of dtance D. The maller D, the mot mportant nput varable for the problem, nce th mple that the other varable affect very lttle to the output varable for every fed value (partton nterval) of. Fg. 4 preent, n a chematc way, the general decrpton of the propoed IVS method. For all the part the average of the dtance calculated D. 4. SIMULATION EXAMPLES In th ecton dfferent eample wll be appear to verfy the procedure n the propoed algorthm. Two type of reult wll be preent: the tructure of the ytem Mult-RBFNN elected by the algorthm ung the method of IVS and whch of the nput varable mut go alone or together a Sub-RBFNN n the ytem Mult-RBFNN, and the reult of the valdty of algorthm n appromate functon from ample of nformaton of I/O data, compared wth reult of a typcal RBFNN that receve all the varable of the functon and wth other method propoed n the bblography. Th way, the ytem Mult-RBFNN wll be evaluated wth h charactertc n decreae the number of parameter, whch obtan the prncpal objectve n the preent work of the earch of new archtecture of calculaton capable of hapng comple ytem of functon appromaton, wthout the ncreae of the number of nput varable ha to uppoe an eponental ncreae n the complety of the ytem. Relate each dmenon of the nput data {,, d} to the target output a a functon of dmenon. Dvde the data n part P. Aocate the data of each dmenon to h correpondng output data Ue the Kalman flter to mooth the vector of the mamum and mnmum n each part. Calculate the value of dtance D between the mamum and mnmum value of the target output n each part. Ye Calculate the mean dtance n each dmenon. Remove the varable D > θ? No Select the varable Fg. 4. General decrpton of the method IVS. The preent reult of 5 eecuton; the et of radal functon ued n the Sub-RBFNN {RBF} that there conderng the algorthm (every tme there added RBF), # Param the number of parameter. NRMSE Tr the normalzed mean quared error of tranng and NRMSE Tet the normalzed mean quared error of tet. 4.. Frt Eample f () We wll take an eample wth 6 poble nput varable to chooe from. Let u conder a et of I/O data par randomly taken from the functon. 2 f( ) = 0 en( π 2) + 20 ( 3-0.5) , 2, 3, 4, 5, 6 [0,] where each nput varable defned n the nterval [0,]. The propoed algorthm elect the deal archtecture of the ytem Mult-RBFNN for the functon f (),

5 depend to the value of the varance threhold after analyzng every varable (Fg. 5). Fg. 5. The varance for each varable n f () In the functon f () few varable mut go alone to Sub-RBFNN and the ubet of the ret goe to Sub-RBFNN, a n the Fg. 6. Fg. 6. Structure Mult-RBFNN elected by the algorthm. Structure of a clac RBFNN for the current functon Table I NRMSE of tranng and tet obtaned by the propoed algorthm and by clac RBFNN for the functon f () 4.2 Second Eample f 2 () The tranng and tet et have been formed by randomly pont. ( f ) 2( ) = 2 co(2 π 8) + 6 e ( 4 5 6)+ 0 7,, 2, 3, 4, 5, 6, 7, 8 [0,] The algorthm elect the deal archtecture of the ytem Mult-RBFNN, depend to the value of the varance threhold after analyzng every varable (Fg. 8). Fg. 7. Comparon reult between Mult- RBFNN ytem and Clacal RBFNN. In the number of parameter. In the number of RBF. Fg. 8. The varance n dfferent data number. For each varable. For each ubet of two varable In the functon f 2 () tow ubet of tow varable go to Sub-RBFNN and the ubet three varable go to Sub-RBFNN, a n the Fg. 9.

6 the nput varable pace for each RBFNN. The electon of the herarchcal tructure Mult-RBFNN adapted accordng to the elected number of nput varable and whch of thee varable go alone or together n a Sub-RBFNN. Acknowledgement Th work ha been partally upported by the Project CICYT TIC E and TIN Reference: Table III NRMSE of tranng and tet obtaned by the propoed algorthm and by clac RBFNN for the functon f 2 () Fg. 9. Structure Mult-RBFNN elected by the algorthm. Structure of a clac RBFNN for the current functon 5. CONCLUSIONS A fundamental lmtaton n the problem of appromaton ytem that when the number of nput varable ncreae, the number of parameter uually ncreae n a very rapd way, even eponentally. Th phenomenon prevent the ue of the majorty of conventonal modellng technque and force u to look for more pecfc oluton. To deal wth th problem, we have earched for new archtecture for modellng comple ytem n functon appromaton problem. The new herarchcal network propoed compoed of complete Radal Ba Functon Network that are n charge of a reduced et of nput varable. For th archtecture, we have propoed a new method to elect the more mportant nput varable, thu reducng the dmenon of [] Ferrar, S., Maggon, M., Borghee, N.A.: Multcale appromaton wth herarchcal radal ba functon network. IEEE Tran. Neur. Net., vol.5, no., pp.78-88, [2] Bengo, S., Bengo, Y.: Takng on the cure of dmenonalty n jont dtrbuton ung neural network. IEEE Tran. Neur. Net., pecal ue on data mnng and knowledge dcovery, (3): , [3] Pomare, H., Roja, I., González, J., Preto, A.: Structure Identfcaton n Complete Rule-Baed Fuzzy Sytem. IEEE Tran. On fuzzy ytem, vol. 0, No. 3, June 2002 [4] S-Y.Taur.Kung,Sh-H.Ln. Herarchcal Fuzzy Neural Network for Pattern Clafcaton. Natonal Scence Councl through Grant NSC E [5] R.Bellman. Adaptve Control Procee: A Guded Tour. Prnceton Unverty Pre. 96. [6] S.Bengo and Y.Bengo. Takng on the Cure of Dmenonalty n Jont Dtrbuton Ung Neural Network IEEE Tranacton On Neural Network, Vol., No. 3, May [7] A.K. Jan and R.Chandraekaran. Dmenonalty and ample ze conderaton n pattern recognton practce. In P.R. Krhanah and L.N. Kanal, edtor, Handbook of Stattc, volume II, page North-Holland, Amterdam, The Netherland, 982.