Wavelet Neural Network Controller for AQM in a TCP Network: Adaptive Learning Rates Approach

Transcription

1 56 International Journal Jae Man of Control, Kim, Jin Automation, Bae Park, an an Yoon Systems, Ho Choi vol. 6, no. 4, pp , August 8 Wavelet Neural Network Controller for AQM in a CP Network: Aaptive Learning Rates Approach Jae Man Kim, Jin Bae Park*, an Yoon Ho Choi Abstract: We propose a wavelet neural network (WNN) control metho for active queue management (AQM) in an en-to-en CP network, which is traine by aaptive learning rates (ALRs). In the CP network, AQM is important to regulate the queue length by passing or ropping the packets at the intermeiate routers. RED, PI, an PID algorithms have been use for AQM. But these algorithms show weaknesses in the etection an control of congestion uner ynamically changing network situations. In our metho, the WNN controller using ALRs is esigne to overcome these problems. It aaptively controls the ropping probability of the packets an is traine by graient-escent algorithm. We apply Lyapunov theorem to verify the stability of the WNN controller using ALRs. Simulations are carrie out to emonstrate the effectiveness of the propose metho. Keywors: Aaptive learning, AQM, congestion control, wavelet neural network.. INRODUCION Congestion control for a CP network has been wiely stuie since ata transmission through the Internet has been increase [-4]. In the CP network, ata transmission over the allocate capacity of link causes packet rops, which results in the retransmissions of lost packets. herefore, the core of congestion control for the CP network is to inhibit an incipient congestion an to realize retrieval from the congeste network conition. Active queue management (AQM) was introuce as a methoology to control the en-to-en congestion in the Internet. AQM primarily respons to network congestion when a queue begins to increase an then maintains a queue size at a preefine level in the router. By keeping the average queue size small, AQM has the ability to preserve the efficient queue utilization an to reuce the elays occurre by the network flow, which is particularly important for realtime interactive applications. RED algorithm has been Manuscript receive July, 7; accepte February 9, 8. Recommene by Eitor Young-Hoon Joo. his work was supporte in part by Yonsei University Institute of MS Information echnology, a Brain Korea program an in part by MOCIE through EIRC program with Yonsei Electric Power Research Center (YEPRC) at Yonsei University, Seoul, Korea. Jae Man Kim an Jin Bae Park are with the Department of Electrical an Electronic Engineering, Yonsei University, 6 Seongsanno, Seoaemun-gu, Seoul -749, Korea ( s: {utopiaro, jbpark}@yonsei.ac.kr). Yoon Ho Choi is with the School of Electronic Engineering, Kyonggi University, Suwon, Kyonggi-o , Korea ( yhchoi@kyonggi.ac.kr). * Corresponing author. propose for AQM scheme, which is the first practical an most well-known metho. RED algorithm lessens en-to-en elay time an prevents consecutive packet rops, which results in oscillation in link utilization an global synchronization [5,6]. However, it shows rawbacks in that the performance of RED algorithm is very sensitive to traffic loa an its parameter setting, moreover, it is har to reuce the queue fluctuation by only ajusting parameters in the RED algorithm [7]. PI an PID control methos also have been propose for AQM scheme. hey are control-theoretic AQM approaches, which are mostly use in classical linear control problems. Hollot [8] analyze the effect on stability cause by network parameters base PI control metho. his AQM scheme was compare with RED algorithm an emonstrate improvement on performance in [3]. he PID control metho also shows better performance than RED in []. However, PI an PID control algorithms are problematic an unrealistic as well because they take the network as a linear an constant system even though the actual network environment is a nonlinear system [9]. On the other han, wavelet neural network (WNN) has been use as the tool of ientification or control for nonlinear systems. It has the avantage of multiresolution of wavelets an fast convergence compare to multi-layer perceptron (MLP). he weights of WNN are usually traine by graient-ecent (D) metho, an the learning rates in the D metho are fixe arbitrary constants. However, since the fixe learning rates (FLRs) are not optimal learning rates, the system performance is sensitive to parameters. herefore, we propose a novel WNN control

2 Wavelet Neural Network Controller for AQM in a CP Network: Aaptive Learning Rates Approach 57 metho base on aaptive learning rates (ALRS) for AQM in the CP network. he propose WNN controller can be applie to linear systems as well as time-varying nonlinear systems, since it is not sensitive to its parameters setting. he propose metho further has an avantage in that it repeately searches the optimal learning rates using Lyapunov theory, an changes the weighting vectors. he propose WNN controller using optimal learning rates minimizes the error between the esire queue length an the network queue length. his paper is organize as follows. In Section, we present the CP nonlinear ynamic moel an WNN base control theory for AQM. We also escribe the learning metho of WNN. Section 3 iscusses the stability analysis of the system via ALRs. Simulation results are shown in Section 4, an the conclusions are presente in Section 5.. PRELIMINARIES.. CP network moel he nonlinear moel for CP flow control was propose [6-8]. he simplifie version that ignores the CP timeout mechanism is as follows: W( n+ ) = W + Rn W W( n R) pn ( Rn ), Rn ( Rn ) W qn ( + ) = qn + Nn C, Rn () where W( n ) enotes CP winow size, qn is the queue length at a router, Rn is roun trip time (R) calculate by qn / C+ p, C is link capacity, p is propagation elay, N is the number of CP sessions (loa factor), an pn is the probability of packet loss. he AQM controller for the simplifie an inaccurate linear CP moel is not optimal, because the real CP network is rapily change, that is, the network parameters, the number of CP sessions, an the capacity of the link are harly kept at constant values for a long time. herefore, it is necessary to esign an aaptive controller for nonlinear network systems... WNN structure he WNN is esigne as a four-layer structure [,]. Each layer has one or more noes. Since the objective is to esign a WNN controller which ajusts the queue length of the CP network to the esire queue length, that is, the error eq = q q shoul be minimize, where q an q represent the actual queue length an the esire queue length, respectively. he inputs of WNN are e q an p, which are previous step values. In the mother wavelet layer, each noe performs a wavelet φ j that is erive from its mother wavelet. For the j th noe, xk m φ( z ) = φ, () where m an are the translation an ilation in the j th term of the k th input to the noe of the mother wavelet layer, respectively. here are many kins of mother wavelets that can be use in WNN. In this paper, the first erivative of a aussian function, φ ( z) = zexp.5 z, is selecte as a mother wavelet. In aition, each noe l in the wavelet layer is enote by Π which multiplies the mother wavelet outputs. For the l th rule noe, Φ ( x) =Π φ( z ). (3) l he output of WNN, which is a control input of the CP moel, is compose by each wavelet an parameters as follows: l l k k (4) l= k= y =Φ ( x, θ ) = c Φ ( x) + a x + a, where a an a k are connection weight between input noes an output noes, respectively. N i an N w represent the number of noes in the input layers an the prouct layers, respectively. c l is a connection weight between wavelet noes an output noes, an Ω is the set of ajustable parameters: { m cl ak a} Ω=,,,,. (5).3. raining algorithm for WNN weights he core part of the training algorithm for WNN concerns how to obtain graient vectors. Each of the elements in the training algorithm are efine as the erivative of cost function with respect to a parameter of the network, an this is one by means of the chain rule an the back-propagation (BP) learning rule. A irect aaptive controller base on WNN is consiere in Fig.. he error, which is the ifference between the output of AQM of the CP network an the esire queue length, is the input of a irect aaptive controller. o escribe the training algorithm of WNN using the D metho, the cost function is efine as follows:

3 58 Jae Man Kim, Jin Bae Park, an Yoon Ho Choi Direct Aaptive Controller Base on WNN p CP Moel q φ( z ) φ( z ) z exp z. = = z 3. SABILIY ANALYSIS VIA ALRS D Methor Fig.. Control structure for CP moel. ref q J( Ω ) = q q = e, (6) where qn is the current output value of the CP moel an qref ( n ) is the esire output value. he training metho is base on the minimization of the cost function. he minimization is performe by the following D metho: Ω ( n+ ) =Ω Ω J( Ω) =Ω η, Ω e q q ref (7) where η is the learning rate of a WNN. he partial erivative of the cost function with respect to weighting vector can be represente as follows: J( Ω ) qn qn un = eq = eq, Ω Ω u Ω (8) qn where un = pn, an is the system sensitivity. he partial erivatives of the control input un with respect to weighting vector Ω are as follows: =, a = xk, ak =Φl( x), cl cl Φl( x) =, m z cl Φl( x) = z, z where (9) () () () (3) Φl ( x) = φ( zj) φ( zj) φ( z) φ( zjn ), (4) z i he upate rule of weighting vectors calls for the proper choice of learning rate η [3]. For a small value of η, the convergence is guarantee but the convergence spee is very slow. On the other han if η is too large, the algorithm becomes unstable. Hence, this section evelops a guieline for the selection of proper learning rates. Let us efine a iscrete Lyapunov function as follows: V = eq, (5) where eq ( n ) represents the error between the network queue length an the reference queue value. he error ifference ue to the learning rate can be expresse as follows: eq eq = eq( n+ ) eq Ω, Ω where qn Ω = ηeq, Ω (6) (7) enotes the change in an arbitrary weighting vector, an η represents the corresponing learning rate in WNN. hen, the change of Lyapunov function ue to the training process can be represente by V = V( n+ ) V = e q( n+ ) eq = eq eq + eq eq qn = ηeq Ω u Ω eq qn e q + ηeq Ω u Ω u = η eq Ω Ω eq η eq Ω Ω

4 Wavelet Neural Network Controller for AQM in a CP Network: Aaptive Learning Rates Approach 59 = eq η η Ω Ω = λeq, (8) where λ = η η, Ω Ω (9) qn =. un () heorem : Let 3 4 5,,,, a a, k c m,,, η = η η η η η = η η η η η be the learning rates for the tuning parameters of WNN, an efine U as max Umax = U,max U,max U3,max U4,max U5,max = max max max a ak c max max. n mn hen, asymptotic convergence is guarantee if chosen to satisfy ( Ui,max ) () i η is i < η <, i =,,5. () Proof: In (9), λ = η η Ω Ω ( Ui,max ) η η. Ω Since the change of Lyapunov function is q (3) V = λe, (4) the convergence of WNN is guarantee if λ >. his completes the proof. Corollary : he maximum learning rates which guarantee the convergence are η =, i =,,5. (5) i max ( Ui,max ) Proof: Since in heorem, λ = η η Ω Ω ( Ui,max ) η Ω ( Ui,max ) >. Ui,max (6) hen, from () an (6), we can obtain (5). his completes the proof. a heorem : Let η be the learning rate for weight a. hen, the asymptotic convergence is guarantee if the learning rate satisfies < < (7) a η. Proof: Since U = =, a (8) then, U = U a,max. herefore, from heorem, we fin (7). his completes the proof. a heorem 3: Let η k be the learning rate for the weight a k. hen, the asymptotic convergence is guarantee if the learning rate satisfies a < η k <. (9) N x Proof: Since i max U = = X, a k (3) where X = x, x,, x is the input of WNN, we have U U,max. herefore, from heorem, we have (9). his completes the proof. heorem 4: Let η c be the learning rate for the weight c. hen, the asymptotic convergence is guarantee if the learning rate satisfies a N w < η <. (3) Proof: Since U3 = =Φ, c (3)

5 53 Jae Man Kim, Jin Bae Park, an Yoon Ho Choi where Φ=,,, Φ Φ ΦN w is the output of the wavelet layer, we have Φl for all l, U 3 ( n ) N w. herefore, from heorem, we have (3). his completes the proof. In orer to prove heorem 5, the following lemmas are use. Lemma : f () t = texp t, then f () t <, t R. Let Lemma : g() t = t exp t, then g() t <, t R. Let m heorem 5: Let η, η be the learning rates for the translation an ilation weights, respectively. hen, the asymptotic convergence is guarantee if the learning rates satisfy m < η <, exp(.5) min < η <, exp(.5) min (33) (34) where N w is the number of noes in the prouct layer of WNN. Proof: he learning rate η m of the translation weight m : U5 = mn φ( z ) z < c max z m < max exp(.5). l l= k= cl l= k= Accoring to Lemma, we obtain z exp z, < N w exp(.5) U5 < cl l = min exp(.5 ) <. min (35) (36) (37) herefore, from heorem, we fin (33). he learning rate η of the translation weight : U4 = n φ ( z ) z < cl max l= k= z < c max exp(.5). l l= k= Accoring to Lemmas an, we obtain z ( z ) (38) exp <, (39) z exp z, < N w exp(.5) U4 < cl l= min exp(.5) <. min (4) (4) From heorem, we obtain (34). his completes the proof. Remark : From Corollary, the maximum learning rates of the WNN are as follows: a η =, (4) a η k =, xmax (43) c η =, (44) N w m η = exp(.5) min η =. exp(.5) min 4. SIMULAIONS, (45) (46) In this section, we verify the performances of the propose WNN controller using ALRs through computer simulations, an then compare the results

6 Wavelet Neural Network Controller for AQM in a CP Network: Aaptive Learning Rates Approach 53 4 Router Router 35 3 N User Fig.. CP network topology. N Server 5 5 with the performances of the PID controller an WNN controller using FLRs. We use a simple bottleneck network topology as shown in Fig.. he capacity of bottleneck link between routers an is 5Mbps. he propagation elay is 5ms. All links between users an router have Mbps capacity an those with propagation elay are also 5ms. he size of all queues is 3 packets. he number of CP sessions is, an the reference queue length is packets [4]. We get the parameters of PID controller in [] an set the values of initial ALRs an FLRs to.. We carry out the simulations on two CP network conitions. In static network conition, the simulations are performe by fixing the esire queue length an the number of CP sessions to a constant value. An, in ynamic network conition, the simulations are performe by changing the esire queue length an the number of CP sessions. 4.. Static CP network First, we compare the performances of three controllers: PID, WNN controllers using FLRs, an ALRs. he instantaneous queue lengths for these three controllers are shown in Fig. 3. As presente in Fig. 3, the plot for the PID controller extremely oscillates at the esire queue length of packets. hat is, the performance of PID controller is not aaptive to change upon the environment of the CP network. he WNN controller using FLRs makes the system more stable than the PID controller. However, it also shows a rawback in that there exists a gap between the queue length an the esire value of packets. he WNN controller using ALRs emonstrates the acceptable performance. It ecreases the packets rop, therefore transmits more packets through the router in CP network than other controllers. 4.. Dynamic CP network 4.. Dynamic reference queue length In this simulation, we investigate the ability of each scheme when the reference queue length ( q ref ) varies from packets to 5 packets. he router is more congeste by the reuction of reference length. Fig. 4 shows the queue lengths for the PID controller, WNN controllers using FLRs, an ALRs. For PID controller an WNN controller using FLRs, more ime(s) (a) PID controller (b) WNN controller using FLRs (c) WNN controller using ALRs. Fig. 3. for static CP network. packets are roppe. However, WNN controller using ALRs make the system aaptive to the change of reference queue length, therefore, the packet rop is uniform before an after the change of reference queue length. 4.. Dynamic CP sessions We consier the case that the number of CP sessions changes from to, an the value of other parameters is equal to the simulations

7 53 Jae Man Kim, Jin Bae Park, an Yoon Ho Choi ime(s) 4 (a) PID controller. 5 5 ime(s) 4 (a) PID controller (b) WNN controller using FLRs (b) WNN controller using FLRs (c) WNN controller using ALRs. Fig. 4. for ynamic CP network (Changing reference queue length). consiere in 4. Fig. 5 shows the queue lengths for three controllers. In Fig. 5, we can see that the PID controller has the overshoot at the time when the CP session changes to. But WNN controllers using FLRs an ALRs o not have the occurrence of overshoot an show the uniform queue length unrelate to the number of CP sessions. hese results also emonstrate that the WNN controller using ALRs has faster aaptability for the ynamic (c) WNN controller using ALRs. Fig. 5. for ynamic CP network (Changing CP sessions). CP sessions than others. 5. CONCLUSIONS he aaptive WNN controller using ALRs has been propose for AQM in the CP network. he WNN controller using ALRs is operate as a irect aaptive controller, where the output is a ropping probability of packets at the router. he propose controller

8 Wavelet Neural Network Controller for AQM in a CP Network: Aaptive Learning Rates Approach 533 maintains the actual queue size close to a reference queue value, which is traine by the D metho with ALRs. By using the Lyapunov theorem, we obtain the ALRs for the stable queue length, an show the stability of the whole control scheme. hrough simulation results, we show that the propose control metho is more aaptive than PID an WNN using FLR controllers in both the static an the ynamic CP network systems. REFERENCES [] R. Johari an H.. Kim, En-to-en congestion control for the internet: Delays an stability, IEEE/ACM rans. on Networking, vol. 9, no. 6, pp , December. [] S. Deb an R. Srikant, lobal stability of congestion controllers for the internet, IEEE rans. on Automatic Control, vol. 48, no. 6, pp. 55-6, June 3. [3] C. V. Hollet, V. Misra, D. owsley, an W. ong, Analysis an esign of controllers for AQM routers supporting CP flows, IEEE rans. on Automatic Control, vol. 47, no. 6, pp , June. [4] F. Yanfei, R. Fengyuan an L. Chuang, Design a PID controller for active queue management, Proc. of IEEE International Symp. on Computers an Communication, vol., pp , June 3. [5] B. Braen an D. Clark, Recommenations on queue management an congestion avoiance in the internet, IEF Request for Comments, RFC 39, 989. [6] V. Misra, W. ong, an D. owsley, Fluibase analysis of a network of AQM routers supporting CP flows with an application to RED, Proc. of ACM/SICOMM, pp. 5-6, August. [7] M. Christiansen, K. Jeffay, D. Ott, an F. D. Smith, uning RED for web traffic, Proc. of ACM/SICOMM, pp. 39-5, August. [8] C. V. Hollet, V. Misra, D. owsley, an W. ong, On esigning improve controllers for AQM routers supporting CP flows, Proc. of IEEE INFOCOM, vol. 3, pp , April. [9] H. C. Cho, M. S. Faali, an H. Lee, Neural network control for CP network congestion, Proc. of American Control Conference, vol. 5, pp , June 5. [] S. Ryu, C. Rump, an C. Qiao, A preictive an robust active queue management for internet congestion control, Proc. of. IEEE Sym. on Computers an Communication, vol., pp , June 3. [] F. J. Lin, R. J. Wai, an P. K. Huang, wo-axis motion control system using wavelet neural network for ultrasonic motor rives, IEE Proc. Electric Power Applications, vol. 5, no. 5, pp. 63-6, September 4. [] C. Ku an K. Y. Lee, Diagonal recurrent neural networks for ynamic systems control, IEEE rans. on Neural Networks, vol. 6, no., pp , January 995. [3] S. J. Yoo, J. B. Park, an Y. H. Choi, Direct aaptive control using self recurrent wavelet neural network via aaptive learning rates for stable path tracking of mobile robots, Proc. of American Control Conference, vol., pp , June 5. [4] R. Fengyuan, R. Yong, an S. Xiuming, Design of a fuzzy controller for active queue management, Computer Communications, vol. 5, no. 9, pp , June. Jae Man Kim receive the B.S. an M.S. egrees from Yonsei University, Seoul, Korea, in 5 an 7, respectively, both in Electrical an Electronic Engineering. He is currently working towar a Ph.D. egree at Yonsei University. His research interests inclue neural networks, congestion control, time-elaye systems, nonlinear theories, an aaptive control. Jin Bae Park receive the B.S. egree in Electrical Engineering from Yonsei University, Seoul, in 977 an the M.S. an Ph.D. egrees in Electrical Engineering from Kansas State University, Manhattan, in 985 an 99, respectively. Since 99 he has been with the Department of Electrical an Electronic Engineering, Yonsei University, Seoul, where he is currently a Professor. His research interests inclue robust control an filtering, nonlinear control, mobile robots, fuzzy logic control, neural networks, an genetic algorithms. He ha serve as Vice- Presient for the Institute of Control, Automation, an Systems Engineers. He currently serves as Eitor-in-Chief for the International Journal of Control, Automation, an Systems. Yoon Ho Choi receive the B.S., M.S., an Ph.D. egrees in Electrical Engineering from Yonsei University, Seoul, Korea, in 98, 98, an 99, respectively. Since 993, he has been with the School of Electronic Engineering at Kyonggi University, where he is currently an Associate Professor. From to, he was with the Department of Electrical Engineering, Ohio State University, where he was a Visiting Scholar. His research interests inclue intelligent control, mobile robots, webbase control systems, an wavelet transform. He was the Director for the Institute of Control, Automation, an Systems Engineers from 3 to 4.