7th European Sgnal Processng Conference (EUSIPCO 9 Glasgow, Scotland, August 4-8, 9 ROBUST IDENTIFICATION AND PREDICTION USING WILCOXON NORM AND PARTICLE SWARM OPTIMIZATION Babta Majh, G. Panda and B. Mulgrew Dept. of Electroncs & Communcaton Engneerng, Natonal Insttute of Technology, Rourkela 7698, Inda phone: +9-66-46455, fax: +9-66-46963, emal: babta.majh@gmal.com, ganapat.panda@gmal.com, web:www.ntrkl.ac.n Insttute of Dgtal Communcaton, Unversty of Ednburgh, UK, emal: B. Mulgrew@ed.ac.uk ABSTRACT The paper ntroduces a novel method of robust dentfcaton of complex plants and predcton of bench mark tme seres. It s assumed that tranng samples used contan strong outlers and the cost functon chosen n the proposed model s a robust norm called Wlcoxon norm. The weghts of the models are updated usng populaton based PSO technque whch progressvely reduces the robust norm. To demonstrate the robust performance of the proposed technque standard dentfcaton and predcton problems are smulated and the results are compared wth those obtaned by conventonal MSE norm based mnmzaton method. A sgnfcant mprovement n performance s observed n all cases.. INTRODUCTION Identfcaton of complex nonlnear plants fnds many applcatons n control, power system, nstrumentaton and telecommuncaton []. Accurate and fast dentfcaton of such real tme nonlnear processes s stll a dffcult problem. Further, buldng of proper models of a plant become challengng both for predcton and dentfcaton when outlers are present n the tranng sample. Under such adverse condtons the tranng of models becomes neffectve when conventonal mean square error based on least mean square (LMS or recursve least square (RLS [] type algorthms are used for tranng. Smlarly robust tme seres predcton s mportant n many forecastng applcatons. Varous evolutonary computng tools such as genetc algorthm (GA [3], partcle swarm optmzaton (PSO [4], bacteral foragng optmzaton (BFO [5] and ant colony optmzaton (ACO [6] have been reported and appled for optmzaton and dentfcaton tasks. In case of the dervatve free algorthms conventonally the mean square error (MSE s used as the ftness or cost functon. Use of MSE as cost functon leads to mproper tranng of adaptve models when outlers are present n the tranng samples. Therefore there s a need for robust dentfcaton of complex plants n presence of strong outlers. It s known n statstcs that lnear regressors developed usng Wlcoxon norm (W-norm [7] are robust aganst outlers. Usng such norm new robust machnes have recently been reported for approxmaton of nonlnear functons [8]. In the present nvestgaton we develop a new method of robust dentfcaton and predcton of complex tme seres by mnmzng the W-norm of errors of model usng a dervatve free PSO technque. The dentfcaton and predcton performance of the new method s evaluated through smulaton study and s compared wth the results obtaned from correspondng error square norm based PSO technque. Secton deals wth the basc prncple of system dentfcaton and tme seres predcton where as the fundamental of partcle swarm optmzaton s dealt n Secton 3. Secton 4 proposes the development of robust dentfcaton and predcton models usng mnmzaton of W-norm by PSO. Exhaustve smulaton study for dentfcaton and predcton of benchmark problems usng two dfferent norms s carred out and the results are presented n Secton 5. Fnally concludng remarks are ncluded n Secton 6.. ADAPTIVE SYSTEM IDENTIFICATION AND PREDICTION USING ROBUST NORM The block dagram of an adaptve system dentfcaton scheme s shown n Fg.. Outler + e(. y ˆ( PSO based tranng Fg. Adaptve system dentfcaton model usng W-norm and PSO algorthm At any tme nstant k,, and yˆ( represent the nput, output of the plant and estmated output of the model respectvely. The dfference of these two output produces an error, e (. The output s obtaned by combnng the system output and the outlers present at random locatons. y ( Computaton of W-norm EURASIP, 9 695
Conventonally the MSE s employed as the cost functon n dervng varous teratve learnng rules. It s observed that the learnng rules exhbt poor tranng performance when strong outlers are present n the tranng samples. The W- norm of errors of the model has proven to be a robust norm and the resultng model s expected to be robust to outlers durng tranng. In ths paper recursve mnmzaton of ths norm by PSO s chosen to obtan an mproved and robust dentfcaton model. are the orx ( z m k m Predctor PSO based tranng outlers Σ Computaton of W-norm d ( - + Σ e ( the drecton of ther pbest and gbest. The velocty and poston of th partcle s changed accordng to ( and ( respectvely so that the cost functon defned n (8 s mnmzed progressvely. V ( d = wv ( d + c * rand * ( P ( d X c * rand * ( P ( d X ( d g ( d + w, c c X ( d = X ( d + V ( d ( where V (d and X (d represent the velocty and poston of the th partcle correspondng to d th dmenson respectvely and rand s a unform random number n the range [,]. P g (d and P (d are the d th dmensonal postons of the gbest and pbest respectvely. are constants whose values are sutably chosen to acheve the best possble soluton. The entre process s repeated for some fxed number of teratons untl the global optmum s reached. At ths stage gbest provdes the desred soluton., ( Fg. Robust adaptve predctor usng W-norm based PSO algorthm Fg. shows a robust adaptve predctor usng PSO based tranng algorthm. The adaptve model employs the past samples, k m of the tmes seres and forecasts ts future samples. It computes the robust W-norm of the error vector and progressvely mnmzes the same by updatng ts weghts usng PSO. The desred sgnal s generated by addng outlers at % to 5% random locatons. The process of tme seres predcton conssts of pre-processng of the data, selecton of model to be used, parameter estmaton of the model usng tranng data and valdaton of the model usng past data. 3. FUNDAMENTALS OF PARTICLE SWARM OPTIMIZATION Partcle swarm optmzaton (PSO ntroduced by Eberhart and Kennedy s a populaton based optmzaton algorthm lke the Genetc algorthm (GA. It has already been appled successfully to functon optmzaton, mage analyss, data clusterng and structure optmzaton [9]. It mtates the flyng nature of n-dmensonal swarm (populaton of partcles (brds each of whch provdes a possble soluton to the optmzaton problem - through a problem space, n search of a sngle optmum or multple optma. The flyng behavor can be mmcked as follows: The swarms ntally have a populaton of random solutons. Each potental soluton, called a partcle, s gven a random velocty and s flown through the search space. The partcles have remnscence and each partcle keeps tracks of the prevous best poston, called pbest and the correspondng ftness functon s evaluated and stored. Agan gbest denotes the poston of the partcle havng hghest ftness value for the current teraton. All the partcles always tend to move n 4. NEW IDENTIFICATION AND PREDICTION MODELS USING WILCOXON NORM MINIMIZATION BY PSO The Wlcoxon norm robust cost functon s used for the development of dentfcaton and predcton models. The PSO s employed to teratvely mnmze ths norm of the errors of the model and hence the resultng model s expected to be robust. Robust Cost Functon (Wlcoxon Norm [6, 7] A score functon s frst defned as an ncreasng functon (u :[,] R such that (u du < (3 The score functon has the characterstcs ( u du = = and ( u du (4 The score assocated wth the score functon s defned as a ( =, l (5 l + where l s a fxed postve nteger. From (4 t may be observed that a ( a (... a ( l. The Wlcoxon norm ' [6, 7] on R s defned as l l T ' C = (6 a( R( v v = a( v, v = [ v, v,..., vl R ] = = R( v, v,..., vl, v( v(... v( l where denotes the rank of v among v 696
dered values of v, v,..., v l, a( = [ /( l + ]. In statstcs dfferent types of score functons have been dealt but the commonly used one s gven by ( u = ( u.5. The weght-updates of the models of Fgs. and are carred out by mnmzng the cost functon of the errors defned n (6 usng PSO algorthm. Subsequent steps nvolved are detaled as follows Let the error vector of p th partcle at k th generaton due to applcaton of N nput samples to the model be represented as [ e T, p (, e, p (,..., en, p ( ]. The errors are then arranged n an ncreasng manner from whch the rank R{ en, p ( } of each n th error term s obtaned. The score assocated wth each rank of the error term s evaluated as a ( = (.5 (7 N + where ( N denotes the rank assocated wth each error term. At k th generaton of each p th partcle the Wlcoxon norm s then calculated as C p N ( = a( e ( k (8 =, p The learnng strategy usng PSO contnues untl the cost functon n (8 decreases to the possble mnmum value. At ths stage the tranng s dscontnued and the correspondng global best weght vector represents the optmal weghts of these models. 5. SIMULATION STUDY Smulaton study s carred out to assess the predcton of standard tme seres and dentfcaton of some benchmark plants. The outlers are random values wthn some predefned range and are added at random locatons (% to 5% of the tranng samples. The desred sgnal s obtaned by addng outlers. The MSE and W-norm are used as the cost functons of scheme- and scheme- respectvely. The performance of the proposed scheme s obtaned from smulaton study and compared wth those obtaned by scheme- (MSE-norm. In each example the number of populaton = 3, c = c =.4 and lnearly decreasng w from.9 to.4 are taken. These optmzed values provde best performance n all cases. Example : Predcton of Mackey Glass seres The Mackey-Glass Seres (MGS s a standard benchmark system used for predcton purpose. Ths s a chaotc tme seres generated by solvng the tme-delay dfferental equaton d t = b t + a (9 dt + Ths MGS s perodc for τ < 7 and s non-perodc otherwse. Intal values are taken as random values. The dfferental equaton s solved usng Euler s method. A set of samples s generated wth b =.9 =. and τ = 3. The frst samples s dscarded due to ts random nature. Out of the remanng samples, 8 samples are used as tranng data and the rest as test samples. The model of the system s represented as x ( t + p = f { t,, t τ,..., t ( N τ} ( where p = 4 and N = 4. In ths example four sample ahead predcton of the MGS s made. Thus p = 4 and N = 4 are used. The tranng data set s corrupted by addng random values from a unform dstrbuton of [ 5, 5] to the uncorrupted data set. Smulaton s carred out n presence of % to 5% of outlers n the tranng sgnal. The response matchng obtaned from the smulaton s shown n Fgs. 3(a and (b for 5% outlers. From these fgures t s observed that scheme- based model makes sgnfcant better forecast n presence of 5% outlers n the tranng sgnal where as the scheme- based model fals to correctly predct future values..6.4..8.6.4. 4 6 8 4 6 8.6.4..8.6.4 (a Usng Scheme- (W-norm learnng wth 5% outlers. 4 6 8 4 6 8 (b Usng Scheme-(MSE-norm learnng wth 5% outlers Fg. 3 Output response matchng of Example 697
Example : Forecastng of Sunspot tme seres The seres conssts of 88 data ponts of yearly averages of sunspots startng from the year 7 to the year 987. The sunspots problem s a typcal tme seres predcton problem, n whch the task s to predct the sunspots number for the followng year. In ths example four sample ahead predcton of the sunspot s made. Out of the 88 data ponts frst 5 data s used for tranng and rest 63 data used for testng purpose. The tranng data set s corrupted by addng random values from a unform dstrbuton defned between [ 5, 5] to the uncorrupted data. Smulaton s carred out n presence of % to 5% of outlers n the tranng sgnal. The response matchng of the system wth 4% outlers s gven n Fgs. 4(a and (b. The forecastng performance of scheme- s severely degraded at 4% outlers. It s clearly observed that scheme- model provdes superor predcton n comparson to the scheme- based model partcularly n presence of outlers. Example 3 : Identfcaton of Box-Jenkn s System The 96 nput-output samples are generated wth a samplng perod of 9s. The gas combuston process has one varable, gas flow, nd one output varable, the concentraton of CO, y (. The output s nfluenced by four past output samples k, k, k 3 and x ( k.unformly dstrbuted random values between [ 3, 3] s added at % to 5% random locatons of the desred samples. Fgs. 5 (a and (b dsplay the actual and estmated output values obtaned by usng scheme- and scheme- methods of tranng respectvely. It s evdent from these fgures that scheme- provdes better dentfcaton performance n presence of strong outlers n the tranng sgnal n comparson to scheme- based method. 6.9 58 56.8.7.6.5.4.3.. 3 4 5 6 54 5 5 48 46 44 (a 3 4 5 6 7 8 9 Usng Scheme- (W-norm learnng wth 5% outlers (a..8 Usng Scheme- (W-norm learnng wth 4% outlers 6 58 56 54 5 5.6.4. 48 46 44 3 4 5 6 7 8 9 -. 3 4 5 6 (b Usng Scheme- (MSE-norm learnng wth 5% Outlers Fg. 5 Output response matchng of Example 3 (b Usng Scheme- (MSE-norm learnng wth 4% outlers Fg. 4 Output response matchng of Example Example 4 : Identfcaton of SISO dynamc system [ ] The plant n ths case s descrbed by the dfference equaton y ( k + = f [, k, k,, k ] ( where the unknown nonlnear functon f s gven by 698
f [ a 3 4 5 ] a a a a ( a. + a 3 5 3 4 = (. + a + a3 The seres-parallel model used for dentfcaton of ths plant s gven as y ˆ( k + = Ν[, k, k,, k ] (3 In case of scheme- and scheme- the basc model s a functonal lnk artfcal neural network (FLANN [] structure. Its nput and output are expanded to sx and nne terms respectvely usng trgonometrc expanson. An unformly dstrbuted random sgnal n the nterval [-, ] s used as nput. The outlers are unformly dstrbuted random values wthn the range of - to + and are added at random locatons (% to 5% of the tranng samples. Durng the testng phase, the effectveness of the proposed models are evaluated by usng the test sgnal πk sn for k 5 5 (4 = πk πk.8 sn +. sn for k > 5 5 5 Fgs. 6(a-(b show the comparatve performance of the output response of two models. The smulaton results ndcate that the dentfcaton performance s better n the proposed model than the MSE-norm based model..8.6.4. -. -.4 -.6 -.8-3 4 5 6 7 8 (a Usng Scheme- (W-norm learnng wth 4% outlers 6 CONCLUSION The adaptve dentfcaton and predcton tasks have been formulated as optmzaton problems. Instead of usng conventonal MSE as ftness functon, robust W-norm of errors s chosen for mnmzaton usng PSO technque. Robust predcton of Mackey Glass and Sunspot tme seres when strong outlers are present n tranng set s carred out through smulaton. Smlarly a PSO based method for robust dentfcaton of standard plants s also suggested by way of mnmzng W-norm. The proposed technques s robust because t provdes excellent predcton and dentfcaton performance of complex plants even when the tranng sgnal of the model contans up to 5% of outlers. The ntroducton of the new cost functon n the model and PSO based mnmzaton of these cost functon has contrbuted to robust and mproved performance compared to those obtaned from standard squared error norm based model..5.5 -.5-3 4 5 6 7 8 (busng Scheme- (MSE-normlearnng wth 4% outlers Fg. 6 Comparson of response of the dynamc plant of Example 4 REFERENCES [] Cha-Feng Juang, A TSK-type recurrent fuzzy network for dynamc systems processng by neural network and genetc algorthms, IEEE Trans. on Fuzzy Systems, vol., no., pp. 55-7, Aprl. [] B. Wdrow and S. D. Stearns, Adaptve Sgnal Process ng, Second Edton, Pearson. [3]D.H.Goldberg, Genetc algorthms n search, optmzaton and machne learnng, Addton-Wesley,989. [4]G. Panda, D. Mohanty, Babta Majh and G. Sahoo, Identfcaton of Nonlnear Systems usng Partcle Swarm Optmzaton Technque, n Proc. of IEEE Congress on Evolutonary Computaton(CEC-7, Sngapore 5-8, September, 7, pp.353-357 [5] B. Majh and G. Panda, Bacteral Foragng based Identfcaton of Nonlnear Dynamcs System, n Proc. of IEEE Congress on Evolutonary Computaton(CEC-7, Sngapore 5-8, September, 7, pp.636-64 [6] Marco Dorgo and Thomas Stützle, Ant colony optmzaton, MIT Press, June 4 [7]Joseph W. McKean, Robust analyss of Lnear models, Statstcal Scence, vol. 9, no. 4, pp. 56-57, 4. [8] Jer-Guang Hseh, Yh-Lon Ln and Jyh-Horng Jeng, Prelmnary study on Wlcoxon learnng machnes, IEEE Trans. on neural networks, vol. 9, no., pp. -, Feb. 8. [9] J. Kennedy and R. C. Eberhart, Swarm Intellgence, San Mateo, CA: Mogan Kaufmann, []K. S Narendra. and K. Parthasarathy, Identfcaton and control of dynamcal systems usng neural networks, IEEE Trans. on neural networks, vol., no., pp. 4-7, 99. [] J. C. Patra, R. N. Pal, R. Balarsngh and G. Panda, Nonlnear channel equalzaton for QAM sgnal constellaton usng artfcal neural networks, IEEE Trans. on Systems, Man and Cybernetcs- Part B, vol. 9, no., pp. 6-7, Aprl 999. 699