Application of Control Theory to a Commercial Mobile Service Support System

Transcription

1 Appication of Contro Theory to a Coercia Mobie Service Support Syste Aani, Paya; Aspernäs, Berti; Åströ, Kar Johan; Derantz, Manfred; Kih, Maria; Radu, Gabriea; Robertsson, Anders; Torstensson, Andreas Pubished in: Internationa Journa on Advances in Teecounications 0 Lin to pubication Citation for pubished version (APA): Aani, P., Aspernäs, B., Åströ, K. J., Derantz, M., Kih, M., Radu, G.,... Torstensson, A. (0). Appication of Contro Theory to a Coercia Mobie Service Support Syste. Internationa Journa on Advances in Teecounications, 5(3&4). Genera rights Copyright and ora rights for the pubications ade accessibe in the pubic porta are retained by the authors and/or other copyright owners and it is a condition of accessing pubications that users recognise and abide by the ega requireents associated with these rights. Users ay downoad and print one copy of any pubication fro the pubic porta for the purpose of private study or research. You ay not further distribute the ateria or use it for any profit-aing activity or coercia gain You ay freey distribute the URL identifying the pubication in the pubic porta Tae down poicy If you beieve that this docuent breaches copyright pease contact us providing detais, and we wi reove access to the wor iediatey and investigate your cai. L UNDUNI VERS I TY PO Box7 00L und Downoad date: 5. Jan. 09

2 Appication of Contro Theory to a Coercia Mobie Service Support Syste Paya Aani a,, Berti Aspernäs c, Kar Johan Åströ b, Manfred Derantz b, Maria Kih a, Gabriea Radu a, Anders Robertsson b, and Andreas Torstensson c a Dept. of Eectrica and Inforation Technoogy, Lund University, Sweden b Dept. of Autoatic Contro, Lund University, Sweden c Ericsson AB, Karsrona, Sweden {paya.aani, aria.ih}@eit.th.se, {anfred.derantz, andersro, ja}@contro.th.se, uinita.radu@gai.co, berti.aspernas@teia.co, andreas.torstensson@ericsson.co Abstract The Mobie Service Support syste (MSS), which Ericsson AB deveops, handes the setup of new subscribers and services into a obie networ. Experience fro depoyed systes show that traffic onitoring and contro of the syste wi be crucia for handing overoad situations that ay occur at sudden traffic surges. In this paper we identify and expore soe iportant contro chaenges for this type of systes. Further, we present anaysis and experients showing soe advantages of proposed soutions. First, we deveop a oad-dependent server ode for the syste, which is vaidated in testbed experients. Further, we propose a contro design based on the ode, and a ethod for estiation of response ties and arriva rates. The ain contribution of this paper is that we show how contro theory ethods and anaysis can be used for coercia teeco systes. Parts of our resuts have been ipeented in coercia products, vaidating the strength of our wor. Keywords Perforance anageent; teecounication systes; queuing theory; contro theory; database servers; adission contro; Kaan fiters I. INTRODUCTION Resource anageent of coputer systes, which has gained increased attention during recent years, was expored aready in the ate 60's [][3]. It is an essentia echanis to hande oad disturbances such as traffic surges and changes in user behavior. Poory anaged resources can severey degrade the perforance of a syste with potentiay arge financia consequences. The wor presented in this paper is otivated by a coercia Mobie Service Support Syste (MSS), deveoped and produced by Ericsson AB. Mobie Service Support Systes are used by the networ operators for a processing regarding new subscribers and services in the networ. Each new subscriber or service requires processing and data storage in severa networ nodes. The systes are in genera uti-tier systes, ipeented as distributed server custers, where web and appication servers process the incoing requests and database servers are used for data This wor represents the outcoe of a ong-ter coaboration between Lund University and Ericsson AB and the contributors are isted in aphabetic order. storage. The resource anageent of these systes, based on easureents of the syste states such as actua utiization and response ties, is crucia for the optiization of operation cost and the guarantee of service eve agreeents during oad surges, for exape during areting capaigns or various events. Therefore, the chaenge is how to contro syste perforance whie providing guarantees on convergence and disturbance rejection. The soution is based on dynaic contro schees, which onitors the systes and provides actions when needed. Severa types of resource anageent echaniss have been proposed and evauated in the iterature. In arger coputer systes, oad baancing is perfored in order to distribute the deand for resources unifory over a nuber of resource units (coputers, CPUs, eory, etc.), thus avoiding the case that aong the nodes with siiar functionaities soe are under-utiized whie others are overoaded [4][5]. During overoad periods, when ore resources are requested than are avaiabe, adission contro echaniss reduce the oad to the syste by bocing or deaying soe of the requests [6][7][8][9]. For Internet appications, virtuaized server systes can be used to divide physica resources into a nuber of separated patfors where different web appications are aowed to operate without affecting one another. Dynaic resource aocation between the virtuaized patfors serves as a new and easy way to perfor resource optiization on web server systes [0][][]. In the ast years, the fied of power and energy anageent has becoe iportant. Large software systes have high energy consuption, which eans that dynaic resource optiization of these systes ay consideraby ower the operating costs for the networ operators [3][4][5][6]. However, a optiization techniques require accurate perforance odes of the invoved coputing systes. The operation region is ainy high traffic oad scenarios, which eans that the coputing systes show non-inear dynaics that needs to be characterized accuratey [7]. A software syste is basicay a networ of queues, as exapes, the CPU ready queue, seaphore queues, socet queues, and I/O device queues, which store requests in waiting of service in the processors. Therefore, queuing odes can be used when

3 describing the dynaic behavior of server systes [][8][9][0]. The concept of Load-Dependent Server (LDS) odes, in which the response tie of the jobs in the syste is a function of the service tie of the jobs and current nuber of jobs waiting to be served has, to the best of our nowedge, firsty been introduced in []. In [][][3], standard benchars were used for woroad generation and aso regression odes to capture the syste dynaics. In [4], a queuing networ ode which represents the oad dependent behavior of the LDS was presented and vaidated with siuations. In [5], a theoretica anaysis of the D/G/ and M/G/ odes with oad dependency assuptions was presented. In this paper, we investigate soutions to soe iportant contro chaenges identified for the coercia MSS deveoped by Ericsson AB. We present a oad-dependent server ode, which is vaidated in experients. The ode has been previousy pubished in []. Further, we extend [] by proposing and vaidating an adission contro echanis based on a oad-adaptive controer. A odified version of the controer has been ipeented in the Ericsson product. Finay, we show how extended Kaan fiters can be used for estiating the response ties and arriva rates in the syste. The paper is organized as foows. In Section II, the Ericsson product is described and the contro chaenges identified for the syste are presented. In Section III, the testbed used for soe of the experients is described. In Section IV, the oad-dependent server ode is presented and vaidated. In Section V, the oad-adaptive controer is presented and experients vaidating its perforance are described. In Section VI, our wor on response tie estiation based on extended Kaan fiters is presented. Finay, in Section VII, soe concusions are presented. II. SYSTEM AND PROBLEM DESCRIPTION The Mobie Service Support syste (MSS), which Ericsson AB deveops, handes the setup of new subscribers and services into a obie networ. It presents to the operator and its business support systes a unified iddeware where copex functions, such as setting up a new subscriber or odifying services for an existing subscriber, can be easiy invoed. The software architecture is copex with severa ayers and distributed infrastructures, which eans that specific parts of the syste wi not have copete nowedge of the interactions aong other parts of the syste. A. Syste architecture The syste architecture is iustrated in Figure. One request to the MSS fro an upstrea syste noray resuts in a nuber of requests downstrea out on the obie networ to severa different networ eeents (NEs). A networ eeent is usuay a database storing subscriber and service data, for exape, the Hoe Location Register (HLR). A user id, which needs to be fetched fro one database, needs to be suppied in a query to another database to get the syste consistent. Custoer adinistration syste Mobie phone users Mobie service support syste (MSS) Networ eeents (NE) Figure. Mobie service support syste (MSS) In parae to the changes and setups that the MSS perfors, the networ is aso used by the end users. Services being set up by the MSS are queried by base stations and other systes requiring that inforation. In respect to the MSS, this traffic can be considered as unnown bacground traffic, in contrast to the nown traffic fowing through the MSS. B. Contro chaenges The experience fro depoyed Ericsson systes shows that there can be probes with overoad in the NEs. The easurabe oad arriving fro the MSS and the unnown (not directy easurabe) oad arriving fro obie users ay interfere with each other, creating a race for resources that ay ead to overoad in a NE. When one NE becoes overoaded and unresponsive, this ay resut in the entire transaction requiring robac to avoid in-consistencies in the networ. Such a robac ay require anua wor which is of course costy for the operator. To protect against such situations, traffic onitoring and contro are crucia. In cooperation with Ericsson AB, soe iportant contro chaenges have been identified for this type of syste. These chaenges are described beow. In the foowing sections our coaborative wor on these chaenges wi be presented. The odes and contro designs are based on response ties, as this etric is rather easiy easurabe in the rea syste and because the response ties can be apped to the oad status of the controed syste using the proposed ode. ) Perforance odes The first chaenge is to design a perforance ode for the NEs, since good contro designs are based on sufficienty accurate syste odes. The ode shoud capture the doinant oad dynaics of the NEs. Most service perforance etrics such as response ties and service rates depend on queue state dynaics, which eans that queue odes are suitabe for these systes.

4 Figure. M/M/ ode Unnown traffic generator (obie users) For the objective of perforance contro, sipe odes, such as singe server queues, are often preferred. The ode shoud ony capture the doinating oad dynaics of the syste, since a we-designed contro syste can hande any ode uncertainties [6]. The cassica M/M/ ode, where a singe-server queue processes requests that arrive according to a Poisson process with exponentia distributed service ties, see Figure, has been shown to accuratey capture the response tie dynaics of a web server syste [7]. However, experience fro depoyed systes and ab easureents have shown that databases ay not have M/M/ dynaics [8]. Therefore, other odes are required that ore accuratey captures the dynaics of database servers. ) Adission contro in MSS The NEs are oaded by two traffic sources, the easurabe traffic coing to the MSS and the unnown (uneasurabe) traffic coing fro the obie users, as iustrated in Figure 3. The average arriva rates can be denoted as for the easurabe traffic and u for the unnown traffic. Overoad in the NEs can be detected by onitoring the response tie of requests sent to each node. When the average requests response ties exceed soe threshod, the MSS can cassify the invoved NE as overoaded and thereby start actions to ower the arriva rate to that particuar NE, in order to achieve an acceptabe arriva rate, denoted as c. Therefore, the second contro chaenge is to design an adission contro schee that can hande the unnown traffic at the NEs and further can hande the tie varying ean easured traffic rates experienced in the systes. 3) Monitoring and estiation One of the probes when designing contro echaniss in these types of systes is the ac of perforance inforation. The designed protocos basicay provide no eans of contro counication between the MSS and the NEs that can be used by a contro syste. Therefore, the third contro chaenge that has been identified is the design of onitoring and estiation echaniss that coud hep in the design of, for exape, an adission contro schee. The estiation schee can be used as feed-forward contro in the contro syste, and thereby iproving the perforance of the contro syste copared to when ony using feedbac contro. In coaboration with Ericsson AB, soe preiinary wor on the appication of extended Kaan fiters for oad estiation have been started for systes as in Figure 3. MSS Rejected requests Figure 3. Load at the NEs c NE u Measurabe traffic generator (MSS) Figure 4. Testbed for the experient. III. TESTBED MySQL server To vaidate soe of the proposed soutions, we have perfored a series of experients in our server ab. We deveoped a MSS testbed with two traffic generators, one for the easurabe traffic and one for the unnown traffic, and a MySQL 5..4 database server as depicted in Figure 4. The coputers were connected to a oca 00 Mbit/s Ethernet networ. The traffic generators were ipeented in Java, using the JDBC MySQL connector, and they were executed on coputers with an AMD Pheno II X6 055T Processor at.8 GHz and 4 GB ain eory. The operating syste was Ubuntu LTS. The traffic generators use 00 woring threads and generate MySQL queries according to a Poisson process with average rate and u queries per second. Both traffic generators were vaidated in order to guarantee that they were not a bottenec in the experients. The database server has severa reations with the sae structure but with different nuber of tupes. The axiu nuber of aowed concurrent connections is set to 00. The structure of the reations coes fro the Scaabe Wisconsin Benchar [9] with 0 iion tupes. Two basic types of queries are used, SELECT (read) and UPDATE (write). The queries oo ie this: SELECT * FROM <reation> WHERE unique=?; UPDATE <reation> SET unique=? WHERE unique=?; The question ars are repaced with unifory distributed rando nubers fro zero to ten iion. IV. PERFORMANCE MODELS In this section, we focus on the odeing aspects of database servers. The objective is to deveop a perforance ode for the database server that captures the dynaics during high oads. The perforance ode can be used in resource optiization schees, as adission contro systes, in order to axiize the throughput of the database server, whie eeping soe atency constraints. One of the chaenges for these database servers is that they have a write-heavy woroad, which eans that the CPU is not the bottenec during high oads. This eans that previous wor on

5 perforance odeing of server systes ay not be appicabe since they assue CPU-intensive woroad. A. M/M/ ode with oad dependency (M/M/-LDS) We propose to add oad-dependency to an M/M/ syste. In a oad-dependent server odes, the service tie for a request wi be dependent on the nuber of concurrent requests in the syste. This oad-dependency wi ode effects of the operating syste, eory use, etc., which ay cause service degradation when there are any concurrent jobs in a coputing syste [3]. In the experient section, we wi show that the M/M/-LDS ode accuratey captures the behavior of various database woroad. The properties of the oad dependent M/M/ ode (M/M/-LDS) are set by an exponentia distributed base processing tie, x base = / and a dependency factor, f. When a request enters the syste, it gets the base processing tie x base assigned to it. A singe request in the syste wi aways have a processing tie of x base. Each additiona request inside the syste increases the residua wor for a requests inside the syste (incuding itsef) by a percentage equa to the dependency factor f. When a request eaves the syste a other requests have their residua wor decreased by f percent again. This eans that if n concurrent requests enter the syste at the sae point, they wi a have a processing tie of ( ) ( ) n xs n xbase f () A specia case is when f = 0. It eans that there is no oad dependency, and a requests wi have processing tie x base. The syste can process a axiu of concurrent requests at each tie instance. Any additiona request wi have to wait in the queue. New requests arrive according to a Poisson process with average rate. Therefore, the syste can be odeed as a Marov chain as iustrated in Figure 5. The average service rate of the syste depends on the nuber of concurrent requests in the syste,, derived as foows: ( f ) if 0 if ( f ) By soving the baance equations, stationary probabiity distribution of existence of concurrent requests in the syste is cacuated as beow: ( ) ( f ) if 0 0! ( )( ) ( f ) 0 if! () (3) Figure 5. Iustration of M/M/-LDS ode as a Marov chain. As the su of the probabiities of a possibe states equas to one, 0 can be derived as foows: 0 0 ( ) ( ) ( f ) ( f )! ( )!( ( f ) ) The stabiity condition in this case is ( f ) (5) The average nuber of requests in the syste, N, can be cacuated as beow: N N N N N 0 ( f ) ( f ) ( ) 0 0 ( )! ( ) ( f ) ( ( )( f ) ) ( )!( ( f ) ) 0 Finay by eans of Litte s theore [30], the average tie each request spends in the syste, T, can be derived as foows. N T B. M/M//n ode with oad dependency (M/M//n-LDS) In case that the queue is iited to n positions, the probabiity for an epty syste, 0, can be deterined as foows. This queuing syste is naed as M/M//n-LDS. 0 I II III I ( f )! ( ) nn n ( f ) II f III 3 ( f ) ( f ) n n!( ( ) ) ( ) ( f ) ( )!( ( f ) ) ( ) ( f ) ( f ) ( f ) ( f ) (4) (6) (7) (8) Further, the average nuber of requests in the syste is as foows:

6 N N N N 0 ( f)! ( ) f N f 0 N N N N N N n n n 3 D D D 3 n n 3 ` nn N n f n n ` nn N f n ( f ) n ( ) f N f! f D! D f n nn n N f f f n 3 N N D 3 f f! f (9) change is sower than in the case where is decreased. On the other hand the exponentia rising rate of the graph is increased in coparison with the case where is decreased. Here, it is assued that (, ) = (3, ). The effects of changing on the/t graph whie fixing the two other paraeters is iustrated in Figure 8. As shown in the figure, by increasing in equa steps, the /T graph wi be shifted to the right in equa steps. In this case, where (, f) = (3, 0.6), the rate of rising of the graph is decreased. In cases where the M/M//n-LDS ode is used, there wi be a saturation of the response ties when the oad is high enough to overoad the queue. Here, it is assued that the defaut vaues are (, n, f, ) = (4, 5, 0.6, ). Figure 9 and Figure 0 show the effects when varying and f respectivey. In each case, the vaues of the other three paraeters are constant. The genera effect of changing the paraeters is siiar as for the case with the infinite queue, with the difference that the response ties saturate when the oad is high. Finay, the average response tie for a request can be derived using Litte s theore. C. Paraeter tuning In a teeco syste with atency constraints, the doinant dynaic of the syste is often characterized by the average response tie, T, when varying the average arriva rate,. Tuning of the paraeters of the LDS ode in a way that it fits the easured data fro the actua server syste is a necessary step in odeing of such systes. Assuing that and T are easureabe, there are three ain paraeters for the M/M/-LDS ode,, f and to tune in order to fit the ode on the easured data. Further, for the M/M//n-LDS there is an extra paraeter, n, to tune. Therefore, in Figures 6-0, the effects of changing ode paraeters on dynaics of average response tie versus ean arriva rate of queries are iustrated. In the rest of the paper, this graph wi be caed the /T graph. In each figure, it is assued that two (three) of the paraeters are fixed and the one that is entioned is the variabe. As the equations for cacuating the ean response ties are rather copex and the paraeters are interdependent, ore than one set of paraeters can be fit on the easured data. Thus using these figures, a heuristic rue for tuning the paraeters of the LDS ode can be achieved. In the cases where the M/M/-LDS ode is used, the first paraeter to be tuned is the nuber of servers,. As it can be seen in Figure 6, by increasing the axiu nuber of concurrent requests that can be processed in the syste, the inear part of the /T graph wi be shorter and the exponentia rising rate of the graph is increased. In this case it is assued that (f, ) = (0.6, ). The second paraeter to be tuned is the dependency factor, f. As shown in Figure 7, by decreasing the dependency factor, the inear part of the /T graph is increased, however, the Figure 6. Variations of the /T graph for a specia scenario with as variabe when (f,) = (0.7, ). Figure 7. Variations of /T graph for a specia scenario with f as variabe when (,) = (3, ).

7 Figure 8. Variations of /T graph for a specia scenario with as variabe when (, f) = (3, 0.6). high oad. The graphs show the average response ties of queries as a function of the arriva rate. We have fitted M/M//n-LDS odes for the data using the tuning steps described in the previous section. In both scenarios, the CPU utiization was very ow, aso for high oads. The axiu CPU oad was about 5%. In order to ode the networ deays, we have added a bias of 0.03 seconds in the average response ties of the proposed odes. In Figure, the woroad is based on 00% UPDATE queries. The fitted ode in this case has the foowing paraeters (, n, f, ) = (3, 8, 0.75, 37.). Figure depicts the sae experient setup when using a ix of 5% SELECT queries and 75% UPDATE queries. The fitted M/M//n-LDS ode in this case has the foowing paraeters (, n, f, ) = (6, 73, 0.44, 35.). In Figure 3 ony SELECT queries are used. In this case the ode paraeters are (, n, f, ) = (6, 40,.39, 38). The resuts verify that the proposed ode can represent the average dynaics of a database server with various woroads very we Figure 9. Variations of /T graph for a specia scenario with as variabe when (n, f, ) = (5, 0.6, ). Figure. Perforance of the M/M//n-LDS queuing ode in odeing steady state dynaics of a MySQL database server using UPDATE queries. Figure 0. Variations of /T graph for a specia scenario with f as variabe when (, n, ) = (4, 5, ). D. Experients In order to vaidate the ode, we have perfored a series of experients in our testbed, as described in Section III. In this case, the arriva rate of the unnown traffic was set to zero. The dynaics of the database server highy depends on the ix of requests, since SELECT and UPDATE queries require different aount of server capacity. Therefore, experients with varying woroad ix have been perfored. Figure, Figure, and Figure 3 show the resuts fro experients where the arriva rate is varied fro ow oad to Figure. Perforance of the M/M//n-LDS queuing ode in odeing steady state dynaics of a MySQL database server using ixed queries.

8 Tref Controer Measureents A Queries Reject Gate To database Figure 4. Contro syste Figure 3. Perforance of the M/M//n-LDS queuing ode in odeing steady state dynaics of a MySQL database server using SELECT queries. V. ADMISSION CONTROL As part of the coaboration with Ericsson AB, we have designed an adission contro echanis for the easurabe traffic to the NEs, as iustrated previousy in Figure 3. As a direct effect of this wor, a odified version of the contro echanis has been ipeented in the Ericsson product. In this section, the controer design and its vaidation are described. A. Contro structure The MSS incudes a contro syste, as iustrated in Figure 4, which shoud ensure that the oad on a specific NE is ept at an acceptabe eve. The contro objective is to eep the ean response ties of the NE queries beow a desired vaue whie axiizing the throughput. The contro actions ust be based on a iited aount of contro inforation, due to the standardized protocos and the ayered software architecture. The contro syste incudes a controer and a gate. The controer uses a response tie reference vaue, T ref, and easureents to deterine an acceptabe woroad to the database server. The acceptabe woroad is defined by the noraized rate of aditted queries, A, which corresponds to the ratio of the average arriva rate of the aditted requests over the higher bound of the average arriva rate of the requests. It is desired that the contro syste perfors robusty in presence of fuctuations in the average arriva rate of the queries sent to the database. Therefore, the controer design is crucia for guaranteeing the contro objectives. The gate ensures the ratio A of arriving queries is aditted to the database. In the experients, the gate rejects requests that cannot be aditted. However, in the rea product, this is not feasibe. Instead, the rea product has a traffic shaping echanis that adds deays to the responses to the custoer adinistration syste. Since the counication with the custoer adinistration syste is synchronous, adding deays to the responses wi ower the arriva rate of requests. In this paper, we focus on the controer perforance. Therefore, the ipeentation of the gate is not the ain focus as ong as it can be assued that the gate actuates the contro signa accuratey. B. Controer design We have designed a controer that can guarantee the contro objectives for the syste. The controer, caed the Load-Adaptive Controer (LAC), ony uses easureents of the query response ties. A cassica PID controer [6] incudes one Proportiona part (P), one Integra part (I), and one Derivative part (D) that deterines the contro signa based on the deviation of the input signa fro the reference vaue. For stochastic systes, the derivative part wi apify the effect of high frequency noise in the response tie error and thus deteriorate the overa perforance of the syste. Therefore, the LAC is based on a odified PI controer with anti-windup. The LAC adapts its proportiona gain with the variations in the ean arriva rates of queries sent to the database. The structure of the odified PI controer is iustrated in Figure 5. The tota oad of the NE is deterined by the aggregated arriva rates of the easurabe and the unnown traffic streas. However, assuing that the unnown traffic is stationary during a iited tie period and that the database server behaves as a conservative queuing syste [30], a specific aditted ratio of the traffic wi correspond to a specific ean response tie, as iustrated in Figure 6. T ref Load adaptive Ki T i + Low pass fiter ()dt Figure 5. Load-adaptive controer (LAC) + + T t - T A

9 T T ref ( ow,t ow ) A ( high,t high ) Figure 6. An iustration of the LAC cacuations. The controer continuousy eeps trac of two points in this graph, one ow point, ( ow, T ow ), which is situated beow the reference response tie, T ref, and one high point, ( high, T high ), which is situated above T ref. As the contro syste operates ony based on easured response ties of NE queries, ow guarantees that those easureents exist for a saping intervas. The upper iit for ean arriva rates of the queries processed by the NE whie not overoading the database is represented by high. The starting vaues for ow and high are set to 5% and 00% respectivey. The adittance rate of the incoing queries is iterativey updated so that its corresponding response tie eets the desired vaue. Every saping tie, the controer cacuates the average response tie, T, over the ast period. If the average response tie during saping period, T, is too high, (T >T ref ), the high point is updated as ( high, T high ) = (, T ) where is the noraized aditted arriva rate during interva. If the average response tie during interva is too ow, (T <T ref ), the ow point is updated as ( ow, T ow ) = (, T ). It is now assued that the optia noraized arriva rate, o, which gives a response tie of exacty T ref is in the interva [ ow high ]. Therefore, the next noraized aditted arriva rate, +, can be interpoated fro these points using cassic geoetry: high ow ( Tref T ) (0) T T high ow Therefore, the quotient ( high - ow )/(T high -T ow ) is used as proportiona gain in the P-part of the controer. The agorith wi converge to the desired response tie vaue assuing that the arriva process is stationary or sowy changing. It is obvious that the contro gate cannot adit ore queries than the incoing ones. This upper iit wi be noted in the cacuations and treated as a saturation iit of the contro signa. The integra I-part of the controer is used when the P-part is not enough for eeping the steady state error to zero. The integra part uses a controer paraeter,, which in conventiona PI controers are equa to the proportiona gain. However, in this case, as the proportiona gain changes drasticay due to the oad-adaptive agorith, using the conventiona PI structure wi ead to a reduced phase argin which wi drive the syste to unstabe region. Therefore, K i is chosen as a static gain and its suitabe vaue is deterined in tuning phase of the controer. Further, the paraeter is the integration tie constant and is the integrator s reset tie constant in the anti-windup echanis. Anti-windup is added to avoid buiding up of the integration part when the contro gate is saturated or copetey open. It is desired to choose sa vaues for so that the integrator resets quicy. Generay, is chosen to be ess than. A ow pass fiter is added after the proportiona gain to soothen the response tie error signa as it is very noisy. The bandwidth of this fiter shoud be suitaby chosen so that its effect on the in-band characteristics of the response tie errors is inor whie attenuating high frequency coponents of that signa. C. Experients To investigate the controer perforance, a Java ipeentation of the controer was depoyed as a web appication to a Gassfish appication server, paced on the server acting as traffic generator in Figure 4. The web appication aso incuded the traffic generator that generated requests for the web appication. For each request, the adission contro decides whether to aow the request to be sent to the database or rejected. The traffic generator for unnown traffic did not have an adission contro, and was set to a specific average arriva rate that coud be atered during run tie. A requests sent to the database server were SELECT queries (according to the query structure described earier). The /T graph for this particuar scenario setting is shown in Figure 7. The saturation of the syste is not shown in the graph for carity reasons, since the operation region is around the nee. To test the perforance of the controer, a scenario was chosen where the oad changed fro sight overoad to high overoad. The reference response tie, T ref, was set to 0. seconds. According to the /T graph in Figure 7, this corresponds to a tota arriva rate of approxiatey 40 queries per second. In this paper, two experients are shown, one with a step in the unnown traffic and one with a step in the easurabe traffic. The controer paraeters were set to T i =4, K i =0.5, T r =, and the saping tie h=0.5 seconds. T i was deterined as a utipe of the saping tie, chosen so that the controer was abe to axiize the throughput whie eeping the ean response ties beow T ref. K i was set equa to the saping tie. To give the controer tie to sette this state was ept for 00 seconds after which a step in the traffic was perfored. The resuting graphs are shown in Figure 8 and Figure 9. The graphs show the average dynaics fro 00 runs. In the first experient, shown in Figure 8, the starting arriva rate was set to 3 requests per second for the easured traffic and requests per second for the unnown traffic. The step increased the arriva rate of the unnown traffic by 0 (to 3) requests per second, resuting in a ore severe overoad situation.

10 Figure 7. /T graph for the adission contro experients. Figure 8. Perforance of the LAC with step in unnown traffic. The second experient, shown in Figure 9, was siiar to the first experient. However, the arriva rate step was in the easurabe traffic instead. To obtain a siiar contro signa response as in the first experient, the step in the controabe traffic had to be arger. Therefore, the observabe arriva rate was increased fro 3 requests per second to 5 requests per second. Both experients show a we-behaved controer, with a reasonabe setting tie and sooth dynaics after the step. VI. MONITORING AND ESTIMATION The syste in Figure is copicated with any different queues, caches and databases. Attepting to capture a detais gives odes that are too copex for on-ine contro. Extensive experience in the fied of contro has ceary deonstrated that sipe odes that capture essentia behavior can be very beneficia [3]. One aspect of the coaboration with Ericsson has been to expore if benefits can aso be obtained for onitoring and contro of the MSS. A crucia issue is what copexity of the odes is required for estiation and contro of the MSS. Response tie and arriva rates are variabes of prie concern. The variabes have strong variations, which can be reduced by averaging. A ore effective way is to construct estiators that expoit the dynaic behavior of the syste. Exporation of such estiators has been one of the goas of the project. A ey feature of the syste shown in Figure is that there are two traffic streas. The easured traffic, generated by the custoer adinistration syste has a nown arriva rate c, can be controed. The unnown strea, which is created by the obie phone users, has an arriva rate u that cannot be controed. Monitoring and contro of the syste can be iproved if good estiates of the average service tie are avaiabe. An abstraction of the syste in Figure is shown in Figure 0, where an estiator and the controer have been incuded. In this section, we wi focus on the estiator, which ony has access to easureents of the easured arriva strea and the response tie T. A actions by the NEs and the MSS have been represented by one queue that represents the aggregated behaviors. Figure 9. Perforance of the LAC with step in observabe traffic. The queue ength is represented by the variabe x, which captures the aggregated behavior of any different queues in the rea syste. The variabe x can be interpreted as a virtua queue ength. The queue ength cannot be easured. The actua response tie T and the actua arriva ties can, however, be easured. Variations in x refect changes in the syste s oad. A. Fow Mode To ode the syste, we wi ae an additiona abstraction by assuing that the variabes x and T are continuous and that they vary continuousy in tie. The behavior of the syste can then be captured by the sipe fow ode: dx ax f ( x) () dt where x is the virtua queue ength, c is the nown arriva rate, u is the unnown arriva rate, ax is the axiu service rate and f is a onotone function with the range [0, ]. The response tie is given by T t x) t ( f ( )) () 0( 0 where t 0 = / ax is the average tie to serve one job when the queue is epty and is the noraized service rate or the utiity =/ ax.

11 u Controer and Estiator c x ax T Figure 0. Scheatic diagra of an abstraction of the MSS in Figure with a controer and estiator. The response tie goes to infinity as approaches ax if the range of the function f is [0, ]. The function f gives significant freedo in adjusting the behavior to rea queue behavior. The ode (), () has been used extensivey to ode queuing systes [33]. The sipe M/M/ queue can be represented by () with f = x/(x + ) [3]. Even if the ode (), () is sipe it captures soe iportant features of rea queuing systes, for exape the fact that response tie increases with queue ength. The ode aso captures the behavior that the rate of change of the response tie increases with increasing arriva rate. The behavior of the syste can be shaped by the function f. In the project, we have investigated siuated odes with servers and we have deonstrated that it is possibe to find functions f which atches the steady state behavior of siuated systes. An iustration is given in Figure. B. Estiation agorith There are significant variations in the arriva and response ties due to their discrete nature. To onitor and contro the syste it is necessary to sooth these variations. For exape, the average arriva rate of the controed strea can be estiated the sipe exponentia soother tˆ i tˆ i ha tˆ 3( i ) (3) ˆ c tˆ i where t i is the arriva tie and h a is the tie since the ast arriva update. One advantage with the ode (), () is that it is possibe to use Kaan fitering [3] to cobine the ode, which captures the gross behavior of the queuing syste, with easured data. If continuous data was avaiabe, an extended Kaan fiter for the service tie is given by dxˆ c u ax f ( xˆ) ( T t0( xˆ)) dt (4) du ( T t0( xˆ)) dt This fiter wi capture the behavior that response tie increases with increasing queue ength and arriva rate. The detaied behavior can be shaped by the function f. It ust be considered that the rea easureents are events that represent arriva of a request or a copeted response. To dea with this, we have deveoped an event-based Kaan fiter. Figure. Service ties for the operations SELECT (eft) and UPDATE on an SQL server and predictions based on the ode () with f(x)=(/(+x)) n, n =.5 and ax = 880 for SELECT and n = 0.5 and ax = 3 for UPDATE. At arrivas, the queue ength is updated according to the fow ode: xˆ xˆ h ( ˆ ˆ a c u axf ( xˆ)) (5) This difference equation is sipy a forward Euer approxiation of (). Equation (5) is sipy a prediction of x based on the ode (). Inforation about x is obtained when a service is copeted. The queue ength and the unnown arriva rate are then updated as xˆ xˆ hd ( c u ax f ( xˆ) ( T Tˆ)) (6) ˆ ˆ u u hd ( T Tˆ) where h d is the tie since the ast departure update. The arriva rate can be estiated because it resuts fro the ode () and () that the arriva rate is observabe fro a easureent of service tie [3]. C. Experient The Kaan fiter estiator was evauated using a discreteevent siuation progra written in Java, The progra siuates a singe server queue with exponentiay distributed service ties with ean ax =00 requests per second. The queue has two arriva processes, representing the easurabe and unnown traffic. The Kaan fiter has been evauated for a nuber of scenarios vaidating its perforance. However, in this paper we show the resuts of one specific scenario. In this scenario, the unnown arriva process was a stationary Poisson process with ean 4.5 requests per second. The easurabe arriva process was basicay a Poisson process with changing average rate. The arriva rate,, was the su of one constant part and one part represented by a sine function as given by ( t) C a sin( t) (8) The paraeters were chosen so that the syste can hande the woroad over ong tie but with periodic overoads, hence ax a C ax (9) Therefore, the nuerica vaues used in the siuations are C=4.5 and a = 0 requests per second. The differentia equations describing the behavior of the estiates between events were approxiated using first order forward Euer discretization. Figure shows the response ties and the arriva rate, both rea vaues and estiates for a tie period of 0 seconds

12 during the siuation. The estiate error is shown in Figure 3. It can be seen how the Kaan fiter anages to foow the rea syste during the quic rises in response tie around 4 tie 44 and 47. Here the ean square error is for the period 45 < t < 40 and.0 for the period 45 < t < 430. The ean square error for the entire experient is.9 0. VII. CONCLUSIONS Accurate contro designs using contro theory are essentia for resource anageent in coputer systes. In this paper we have presented wor perfored in coaboration with Ericsson AB, investigating how contro theory can iprove the perforance of a coercia obie service support syste. Together with Ericsson AB, we have identified three ajor contro chaenges, and investigated soutions. The first chaenge is to find accurate perforance odes for the syste, with the objective to capture the syste dynaics. The second chaenge is to deveop an adission contro schee that can hande unnown traffic and oad surges. The fina chaenge is to deveop estiation ethods for accurate prediction of response ties and arriva rates in systes with unnown traffic. In this paper, the chaenges have been treated rather independent of each other. However, the future goa is to be abe to use a soutions together, in order to iprove the syste perforance and speed up the deveopent process. The perforance ode coud be tuned using rea data and then used for vaidating contro designs, which is uch easier than ipeenting the designs in testbeds or the rea syste. Aso, in the future, the estiation agoriths shoud be incorporated in the contro syste, iproving the contro decisions. ACKNOWLEDGMENT The authors at Lund University are ebers of the Lund Center for Contro of Copex Engineering Systes (LCCC). Maria Kih and Anders Robertsson are ebers of the Exceence Center at Linöping-Lund in Inforation Technoogy (elliit). The wor is party funded by the Swedish Research Counci, grant VR REFERENCES [] M. Kih, P. Aani, A. Robertsson, G. Radu, M. Derantz, and B. Aspernäs, Perforance odeing of databases in Teecounication service anageent systes, IARIA 7th Internationa Conference on Digita Counications (ICDT), 0. [] B. Brawn and F. Gustavson. Progra behavior in a paging environent, AFIPS Fa Joint Coputer Conference, pages 09-03, 968. [3] Crocus, Systees d'expoitation des Ordinateurs, Dunod, Paris, 975. [4] Y. Diao, C. Wu, J. Heerstein, A. Stor, M. Surendra, S. Lightstone, S. Pareh, C. Garcia-Areano, M. Carro, L. Chu, and J. Coaco, Coparative studies of oad baancing with contro and optiization techniques," Aerican Contro Conference, 005. Figure. Kaan fiter estiates of response ties and estiation of arriva rate. Figure 3. Proposed Kaan fiter s response tie prediction error. [5] Y. Fu, H. Wang, C. Lu, and R. Chandra, Distributed utiization contro for rea-tie custers with oad baancing," IEEE Internationa Rea Tie Systes Syposiu, 006. [6] M. Kih, A. Robertsson, M. Andersson, and B. Wittenar, Contro theoretic anaysis of adission contro echaniss for web server systes, The Word Wide Web Journa, Springer, vo., no., 008. [7] X. Chen, H. Chen, and P. Mohapatra, Aces: an efficient adission contro schee for QoS-aware web servers," Coputer Counication, vo. 6, no. 4, 003. [8] X. Liu, J. Heo, L. Sha, and X. Zhu, Adaptive contro of uti tiered web appications using queuing predictor," 0th IEEE/IFIP Networ Operation Mangeent Syposiu, 006. [9] T. Voigt and P. Gunningberg, Adaptive resource based web server adission contro, 7th Internationa Syposiu on Coputers and Counications, 00. [0] M. Kjaer, M. Kih, and A. Robertsson, Resource Aocation and Disturbance Rejection in Web Servers using SLAs and Virtuaized Servers, IEEE Transaction on Networ and Service Manageent, Vo. 6, No. 4, 009. [] W. Xu, X. Zhu, S. Singha, and Z.Wang, Predictive contro for dynaic resource aocation in enterprise data centers," 0th IEEE/IFIP Networ Operation Mangeent Syposiu, 006. [] Z. Wang, X. Liu, A. Zhang, C. Stewart, X. Zhu, T. Key, and S. Singha, AutoPara: autoated contro of appication-eve perforance in virtuaized server environents", nd IEEE Internationa Worshop on Feedbac Contro Ipeentation and Design in Coputing Systes and Networs, 007. [3] R. Bianchini and R. Rajaony, Power and energy anageent for server systes, IEEE Coputer, vo. 37, no., 004. [4] H. Caussen, L.T.W Ho, and F. Pivit, Leveraging advances in obie broadband technoogy to iprove environenta sustainabiity, Teecounications Journa of Austraia, Vo. 59, No., 009. [5] T. Horvath, T. Abdezaher, K. Sadron, and X. Liu, Dynaic votage scaing in utitier web servers with end-to-end deay contro," IEEE Transactions on Coputers, vo. 56, no. 4, 007.

13 [6] E. Enozahy, M. Kister, and R. Rajaony, Energy-efficient server custers,, Lecture Notes in Coputer Science 35. Springer-Verag Berin Heideberg, 003. [7] M. Kih, A. Robertsson, and B. Wittenar, Perforance Modeing and Contro of Server Systes using Non-inear Contro Theory, 8th Internationa Teetraffic Congress, 003. [8] J. Diey, R. Friedrich, T. Jin, and J. Roia, Web server perforance easureent and odeing techniques, Perforance Evauation, Vo. 33, No., 998. [9] D. A. Menascé and V. A. F. Aeida. Capacity Panning for Web Services, Prentice Ha, 00. [0] R. D. van der Mei, R. Hariharan, and P. K. Reeser, Web server perforance odeing, Teecounication Systes, Vo. 6, No. 3, 00. [] H. Perros, Y. Daery, and G. Pujoe, Anaysis of a queueing networ ode with cass dependent window fow contro, th Annua Joint Conference of the IEEE Coputer and Counications Societies, IEEE, pp vo., May 99. [] A. Ra, and A. Sguegia, "Instantaneous Load Dependent Servers (ilds) Mode for Web Services," Internationa Conference on Copex, Inteigent and Software Intensive Systes, 00. [3] Curie, M. and Puigjaner, R., Using oad dependent servers to reduce the copexity of arge cient-server siuation odes, Perforance Engineering, LNCS 047, Springer-Verag Berin Heideberg, 00. [4] V. Mathur and V. Apte, A coputationa copexity-aware ode for perforance anaysis of software servers, IEEE Internationa Syposiu on Modeing, Anaysis and Siuation of Coputer and Teecounication Systes (MASCOTS), 004. [5] K. K. Leung, Load-dependent service queues with appication to congestion contro in broadband networs, Perforance Evauation, Vo. 50, Issue, October 00, pp [6] K J. Åströ and B. Wittenar, Coputer Controed Systes. Upper Sadde River, NJ: Prentice Ha, 997. Dover reprint 0. [7] J. Cao, M. Andersson, C. Nyberg and M. Kih, Web Server Perforance Modeing using an M/G//K*PS Queue, Internationa Conference on Teecounication, 003. [8] M. Kih, G. Cedersjö, A. Robertsson, B. Aspernäs, Perforance easureents and odeing of database servers, Sixth Internationa Worshop on Feedbac Contro Ipeentation and Design in Coputing Systes and Networs, 0. [9] D.J. DeWitt. The Wisconsin benchar: Past, present, and future, The Benchar Handboo for Database and Transaction Processing Systes,, 99. [30] L. Keinroc, Queueing Systes, Voue I: Theory, Wiey Interscience, New Yor, 975. [3] K. J. Åströ and R. Murray. Feedbac Systes - An Introduction for Scientists and Engineers. Princeton University Press, 008. [3] D. Tipper and M.K. Sundareshan, Nuerica ethods for odeing coputer networs under nonstationary conditions, IEEE Journa on Seected Areas in Counications, 8(9), pp , Dec 990. [33] Carson E. Agnew, Dynaic odeing and contro of congestion-prone systes, Operations Research, 4(3), pp , 976.