Optimal Bandwidth Allocation with Dynamic Service Selection in Heterogeneous Wireless Networks

Transcription

1 Optimal Bandwidth Allocation Dynamic Service Selection in Heterogeneous Wireless Networs Kun Zhu, Dusit Niyato, and Ping Wang School of Computer Engineering, Nanyang Technological University NTU), Singapore Abstract Bandwidth allocation for different service classes in heterogeneous wireless networs is an important issue for service provider in terms of balancing service quality and profit. It is especially challenging when considering the dynamic competition both among service providers and among users. To address this problem, a two-level game framewor is developed in this paper. The underlying dynamic service selection is modeled as an evolutionary game based on replicator dynamics. An upper bandwidth allocation differential game is formulated to model the competition among different service providers. The service selection distribution of the underlying evolutionary game describes the state of the upper differential game. An openloop Nash equilibrium is considered to be the solution of this linear state differential game. The proposed framewor can be implemented minimum communication cost since no information broadcasting is required. Also, we observe that the selfish behavior of service providers can also maximize the social welfare. Keywords Bandwidth allocation, Replicator dynamics, Differential game, Optimal control, Open-loop Nash equilibrium. I. INTRODUCTION In recent years, the evolving different wireless access technologies and system architectures constitute a heterogeneous wireless environment where different networs complement each other in terms of coverage area, mobility support, offered data rate, and price. Naturally, two issues arise for the users and service providers in this heterogeneous wireless environment. First, the rational users select the access networ and the service class from different service providers according to the performance observation of available service classes. The decision i.e., strategy) of networ and service selection will be made dynamically so that the individual utility is maximized. Second, the service providers have to allocate the available networ capacity i.e., bandwidth) to the offered service classes. Due to the dynamic behavior of users, this bandwidth allocation has to be performed dynamically to obtain the maximum profits. To address the issues of service selection and bandwidth allocation, a hierarchical i.e., two-level) game framewor is developed to jointly obtain the strategies of users and service providers. The dynamic decision of service selection by the users is modeled by an evolutionary game [1]. This evolutionary game taes into account the bandwidth allocation control of service providers. In turn, the service providers observe the service selection of the users and allocate the bandwidth dynamically. This bandwidth allocation of service providers is modeled as a differential game. The novelty of this two-level game framewor is the consideration of the dynamic decision maing. The system parameters e.g., number of users selecting any access service class) are naturally dynamic, and hence a steady state of the networ may never be reached. Therefore, the dynamic optimal control i.e., differential game for noncooperative environment) is the suitable approach for analyzing the dynamic decision maing process of the rational service providers in heterogeneous wireless networs. A few wors studied the networ selection and rate control problems in heterogeneous wireless networs. In [1], evolutionary game based algorithms were proposed for dynamic networ selection. A Marov Decision Process MDP) based control scheme was proposed for flow assignment among different networs in [2]. In [3], a robust rate control framewor for multiple-networ simultaneous access based on H optimal control was developed. Differential game was also applied to solve data transmission issue in wireless networ. In [4], the routing in ad hoc networs was formulated as the differential game coupling constraints. However, none of the wors considered the problem of dynamic optimal bandwidth allocation in heterogeneous wireless networ in which the users can change service selection dynamically. This constitutes the main contribution of this paper. The rest of this paper is organized as follows. Section II presents the system model. The underlying service selection in heterogeneous wireless networs is formulated as an evolutionary game in Section III. The optimal bandwidth allocation control considering the dynamic networ and service selection is formulated as a differential game in Section IV. Section V presents the numerical studies. The summary of this paper is given in Section VI. II. SYSTEM MODEL AND ASSUMPTIONS We consider a particular service area a in the coverage of a heterogeneous wireless environment consisting of M access networs and Nt) active users at time t as shown in Fig. 1. Without loss of generality, each access networ is owned by each service provider 1. Service provider i {1, 2,..., M} can provide K i service classes to users for satisfying different quality of service QoS) requirements. Denote K = M i=1 K i as the total number of service classes. 1 For the rest of this paper, access networ and service provider are used interchangeably.

2 2 Due to the characteristics of wireless channels e.g., fading, interference) and the mobility of wireless users, the system capacity i.e., bandwidth, denoted by Bi for service provider i) and the number of users in area a denoted by N) are generally be time varying. In this paper, we assume that they are the smooth functions respect to t, i.e., B i t) and Nt). Similar to [3], we consider the full bandwidth utilization criterion. In particular, all users subscribed to the same service class will share the available bandwidth equally e.g., a WiMAX base station allocates equal size of time slot to the users). The bandwidth of user received from service class j of service provider i at time t is denoted as τ ij t) = B ijt)/n ij t), where B ij t) represents the allocated bandwidth of service class j from service provider i, N ij t) represents the total number of users choosing service class j of service provider i at time t and M Ki N ijt) = Nt). Users multi-mode terminals can choose different service classes from different service providers freely and independently according to the perceived instantaneous utility [5]. Service Provider 1 K1 Service Classes SC1 SC2 SCK1 Service Provider 2 K2 Service Classes SC1SC2 SCK2 Service Provider M KM Service Classes SC1 SC2 SCKM representing the QoS satisfaction level. Let x ij t) [0, 1] denote the proportion of users in area a 2 choosing service class j from service provider i at time t. Therefore, the bandwidth allocated to each of this proportion of users at time t is τ ij t) = B ijt)/nt)x ij t)) and the payoff of user is uτ ij ij t)) = ατ t) = α B ijt) Nt)x ij t), 1) where α is a constant indicating the increasing rate of utility. Then, the average payoff utility) of the population can be derived as follows: ut) = x ij t)uτ ij t)). 2) The replicator dynamics used to model the evolution process of service selection strategy for all i {1, 2,..., M}, j {1, 2,..., K i } can be described as the following differential equations ) = ẋ ij t) = δx ij t) uτ ij t t)) ut), x ij t) = 1. 3) initial condition x0) = x 0 X, 4) where xt) = [ x 11 t) x ij t) x MKM t) ] T is a vector describing the population state, δ is the learning rate of the population, and X R K is the set of all possible states. Fig. 1. AN1 AN2 a ANM SC : Service Class AN : Access Networ System model of multi-class heterogeneous wireless networs. III. EVOLUTION OF SERVICE SELECTION Users in area a compete to select the available access networs from candidate service providers. The objective of this selection is to maximize the satisfaction i.e., utility) from QoS performance. At any time instance, each user can adapt their service selection strategies according to the time-varying observed networ performance which depends on the current congestion condition. Similar to [1], an underlying evolutionary game is formulated to model the dynamic competition of service selection among users. This is the lower-level game in the proposed twolevel game framewor. In this lower-level evolutionary game model, the players are the Nt) active users in area a at time t. In the context of evolutionary game, a group of users constitute the population. The strategies of players are the choices of particular service class from certain service providers i.e., available access networs). Payoff of a player is the utility IV. DYNAMIC BANDWIDTH ALLOCATION With the dynamic service selection behavior of users, the service providers can optimally allocate the bandwidth to achieve the maximum profits. Increasing the allocated bandwidth of certain service is a natural way to improve the performance and also to attract more users for this service class. However, the limited capacity of the access networ, increasing the bandwidth allocated to one service class will decrease the bandwidth allocated to other service classes which may result in a reduced total profit of service provider. In this section, we formulate the differential game model for bandwidth allocation of service provider. This is the upperlevel game in the proposed two-level game framewor. This differential game model taes the dynamic service selection of users into account. A. Noncooperative Bandwidth Allocation as a Differential Game Each of the M noncooperative service providers competes to maximize the present value of its objective function derived over an infinite time horizon by controlling the bandwidth allocation strategy. To achieve this, a simultaneous play differential game is formulated as follows. The set of players 2 Without loss of generality, notation for area a is omitted in the rest of the paper for simplicity of the presentation.

3 3 is composed of all service providers of the available access networs. For a service provider as a player, the strategy is the dynamic control of the proportion of bandwidth allocated to different service classes. Specifically, we denote the proportion of bandwidth of service provider i allocated to service class j at time t as γ ij t). The control strategy of service provider i is denoted by vector γ i t) = [ γi1 t) γ ij t) γ iki t) ] T R K i +. Naturally, γ ij t) [0, 1], K i γ ijt) = 1 and B ij t) = B i t)γ ij t) for all t [0, + ). Similar to the notation used in game theory, Φ = { γ i t), γ i t)} denotes the strategy profile of this differential game, and γ i t) is a vector of strategies of all players except player i. Depending on different informational structure assumptions, the control strategies of service providers can be represented in different ways i.e., open-loop control strategy and closed-loop control strategy). An open-loop strategy does not need any feedbac information from the system which means that the output of control process also does not need to be observed. While a closed-loop strategy can use feedbac information to adjust the control process if the system biases the predetermined target. Therefore, the use of closed-loop strategy requires more complicated system structure. In this paper, we consider the open-loop control strategy of service provider due to its simplicity of implementation i.e., the centralized controller is not required) which is suitable for the loosely coupled heterogeneous wireless networ. In the bandwidth allocation differential game, all service providers i.e., players) choose their bandwidth allocation control strategies simultaneously, therefore influencing the evolution of the state of the differential game as well as their own and their opponents objective functions. The state of the differential game is represented by the population state xt) of the underlying service selection game. The replicator dynamics differential equations 3) describe how the current state xt) and the service providers control γ i t) at time t influence the rate of change of the state at time t. For a service provider, the problem becomes an optimal control subject to the constraints e.g., state evolution differential equations) given the control strategies of other service providers. The instantaneous payoff of service provider i choosing control strategy γ i t) is expressed as Jins γ i i t), γ i t)) = P ij Nt)x ij t) θ j γ ij t)b i t)) 2 ), where θ j is a cost factor, and P ij denotes the price charged by service provider i for service class j per user per unit of time. In noncooperative bandwidth allocation, for each rational service provider i {1, 2,..., M}, the optimal control can be expressed as follows: maximize: J i γ i t), γ i t)) 5) = e ρt 0 P ij Nt)x ij t) θ j γ ij t)b i t)) 2 )dt, subject to: ẋ ij t) = ) ) Bi t)γ ij t) δx ij t) u ut), Nt)x ij t) x0) = x 0, 6) for i {1,..., M} and j {1,..., K i }, where x ij t) = 1, x ij t) [0, 1], γ ij t) = 1, γ ij t) [0, 1], t [0, + ), 7) where ρ is the discounting rate of payoff of service provider. B. Nash Equilibrium Nash equilibrium is considered to be the solution of above bandwidth allocation differential game. First, the definition of an optimal bandwidth allocation strategy is given as follows: Definition 1: A bandwidth allocation control path γ i t) is optimal for service provider i if the inequality condition J i γ i t), γ i t)) J i γ i t), γ i t)) holds for all feasible control paths γ i t) in the noncooperative bandwidth allocation differential game. According to the Definition 1, the definition of open-loop Nash equilibrium for the bandwidth allocation differential game is given as follows: Definition 2: Denote γ i t) the open-loop bandwidth allocation strategy of service provider i. The strategy profile Φ = { γ i t), γ it)} is an open-loop Nash equilibrium if for each service provider i {1, 2,..., M}, γ i t) is an optimal control path given other service providers control strategies γ it). To obtain the open-loop Nash equilibrium, each service provider needs to solve an optimal control problem. In this case, Pontryagin s maximum principle can be used [7]. First, the definitions of the Hamiltonian function H, the maximized Hamiltonian function H, and the adjoint equation λt) for bandwidth allocation differential game are given. The Hamiltonian function of service provider i is denoted by H i as H i xt), γ i t), γ i t), λ ij t), t) 8) = + Pij Nt)x ij t) θ j γ ij t)b i t)) 2) λ ij t)δx ij t) u ) ) Bi t)γ ij t) u, Nt)x ij t) where λ ij t) is the co-state variable associated xt). Then, the corresponding maximized Hamiltonian function H is defined as H i xt), λ ijt), t) = max{h i xt), γ i t), γ i t), λ ij t), t) γ i t) [0, 1] K i }. 9) The adjoint equation is defined as λ ij t) = ρλ ij t) H i xt), λ ijt), t). 10)

4 4 Based on the above Hamiltonian functions and the linear utility function, we can obtain the following derivation: H i xt), γ i t), γ i t), λ ij t), t) = P ij Nt) αδbt)λ ijt), Nt) where Bt) = M i=1 B it). Therefore, 11) 2 H i xt), γ i t), γ i t), λ ij t), t) x 2 ij t) = 0, 12) and similarly we can obtain 2 H i xt), γ i t), γ i t), λ ij t), t) λ ij t) = 2 H i xt), γ i t), γ i t), λ ij t), t) λ ij t) = 0. 13) According to 12) and 13), we have the following property. PROPERTY 1: The bandwidth allocation differential game defined in 5)-7) is a linear state differential game which possesses the property that the open-loop Nash equilibria are Marovian perfect [6]. To solve for the optimal control strategy, the first order condition is defined as follows: H i γ ij t) = 2θ jbi 2 t)γ ij t) + λ ij t)δα B it) = 0. 14) Nt) Then, we can obtain γ ijt) = λ ijt)δα 2θ j B i t)nt). 15) We can observe that the optimal control path is independent of system state xt) and only relates to the costate variable λ ij t). This costate variable can be obtained by solving the adjoint equations as follows: λ ij t) = ρλ ij t) H i xt), λ ijt), t), 16) where the maximized Hamiltonian function Hi xt), λ ijt), t) can be obtained by substituting 15) into the Hamiltonian function defined in 8). Denote the solution of 16) as λ ij t). Substituting this λ ij t) into 15), we can obtain the optimal bandwidth allocation control path γij t) to service class j of service provider i. Similarly, the optimal control path for all service classes of all service providers can be derived. Then, we obtain the strategy profile Φ = {γij t) i {1,..., M}, j {1,..., K i }}. Since the state space X is a convex set, the solution to the state evolution differential equation 6) exists and is unique [7]. Also, for all t [0, ), the maximized Hamiltonian function H is concave and continuously differentiable respect to x. Therefore, we can state that the obtained strategy profile Φ is a Nash equilibrium for the noncooperative bandwidth allocation differential game. C. Cooperative Bandwidth Allocation as Optimal Control Next, we consider the cooperation of service providers to allocate bandwidth to service classes. In particular, the service providers adjust their bandwidth control paths in a cooperative manner to maximize the social welfare in terms of aggregated profits. Similar to the noncooperative case, the optimal control problem for the cooperative bandwidth allocation can be expressed as follows: maximize: J γ i t), γ i t)) = 17) 0 M e ρt P ij Nt)x ij t) θ j γ ij t)b i t)) 2 )dt the same constraints as defined in 6) and 7). To obtain the optimal solution of cooperative bandwidth allocation, Pontryagin s maximum principle is used. In this case, the Hamiltonian function, the maximized Hamiltonian function, and the adjoint equation of service provider i for the cooperative bandwidth allocation are defined as Hi c, H i λ, and c ij t), respectively, and can be expressed as follows: Hi c xt), γ i t), γ i t), λ c ijt), t) 18) = P ij Nt)x ij t) θ j γ ij t)b i t)) 2 ) + λ c ijt)δx ij t) u ) ) Bi t)γ ij t) u, Nt)x ij t) H i xt), λ c ijt), t) 19) and = max{h c i xt), γ i t), γ i t), λ c ijt), t) γ i t) [0, 1] K i }, λ c ijt) = ρλ c ijt) H i xt), λc ij t), t). 20) According to 18), we can verify that the cooperative bandwidth allocation is a linear state optimal control. With the similar methods used in the noncooperative case, we can obtain the cooperative optimal control γij t) and accordingly the cooperative strategy profile Φ. Observation 1: In the noncooperative bandwidth allocation differential game defined in 5)-7), the selfish behavior of service providers can also maximize the social welfare. Proof: According to the first order condition, let Hi c/ γ ijt) = 0. We can obtain γ ijt) = λ ijt)δα 2θ j B i t)nt). 21) The adjoint equation of the cooperative case is derived as λ c ijt) = ρλ c ijt) H i xt), λc ij t), t) = ρλ c ijt) + Bt)δαλc ij t) P ij Nt), 22) Nt) which is equal to the adjoint equation of the noncooperative case. Accordingly, the co-state variable λ c ij t) = λ ijt), for all i {1, 2,..., M}, j {1, 2,..., K i }. Therefore, we obtain γij t) = γ ij t), which shows that the selfish behavior of service providers can also maximize the social welfare.

5 5 A. Parameter Setting V. NUMERICAL STUDIES We consider a heterogeneous wireless networ where an IEEE b access point and an IEEE base station provide two service classes to the 20 users in the area a as shown in Fig. 1. The maximum saturation throughput of the IEEE b-based WLAN is assumed to be 7 Mbps. The available bandwidth of IEEE networ for area a is assumed to be 5 Mbps when considering the bandwidth sharing of other users in the same cell. For convenience, the WLAN networ service provider and the WiMAX service provider are denoted by service provider 1 and service provider 2, respectively. Fixed connection fees for two service classes of two service providers are set to be P 11 = 0.2, P 12 = 0.1, P 21 = 0.3, and P 22 = 0.25, respectively. For the replicator dynamics, we set the learning rate to be δ = 0.6. For the utility of users, we set α = 0.2. The discounting rate and cost factors for the objective function of service providers are set to be ρ = 0.1, and θ 1 = θ 2 = 0.01, respectively. The initial proportion of users choosing two service classes of two service providers are assumed to be x 11 0) = 0.2, x 12 0) = 0.3, x 21 0) = 0.1, and x 22 0) = 0.4, respectively. B. Numerical Results The dynamic behavior of service selection of users under the bandwidth allocation control is investigated and the strategy adaption trajectory from the initial selection distribution is shown in Fig. 2. The trajectory shows that the dynamics converges to a certain selection distribution where every user in area a receives the same utility as the average utility of population. According to the optimal control strategies, we observe that both service providers 1 and 2 allocate larger bandwidth to service class 1 due to the higher price. As a result, more users select service class 1 as shown in Fig. 2. Fig. 2. Proportion of users choosing services Service class 1 of SP1 Service class 2 of SP1 Service class 1 of SP2 Service class 2 of SP Time Dynamics of service selection. Due to the mobility of users, the total number of users in area a is time varying. The impacts of the variations of the number of users to the optimal bandwidth allocation control is shown in Fig. 3. With the increasing number of users in area a, both access networs become congested. The service providers can control the congestion by dynamically adjusting the proportion of bandwidth allocated to the service class higher price. Also, due to the higher price difference of service classes of service provider 1, we observe that the proportion of bandwidth allocated to service class 1 by service provider 1 is larger than that of service provider 2. Fig. 3. Proportion of bandwidth allocated to service class Service Provider 1 Service Provider The number of users Control strategies under different number of users. VI. SUMMARY We have presented a two-level game framewor based on differential game and evolutionary game for the optimal bandwidth allocation in heterogeneous wireless networs. The dynamic service selection behavior of users have been modeled as an evolutionary game and the strategy evolution process has been analyzed using the replicator dynamics. The bandwidth allocation among different service classes considering users dynamic service selection has been formulated as a linear state differential game. An open-loop Nash equilibrium is considered to be the solution of this differential game. In addition, we have considered the cooperative bandwidth allocation of service providers to maximize aggregated profit. It has been shown that the open-loop Nash equilibrium can also maximize the social welfare. REFERENCES [1] D. Niyato and E. Hossain, Dynamics of networs selection in heterogeneous wireless networs: An evolutionary game approach, IEEE Transactions on Vehicular Technology, vol. 58, no. 4, pp , May [2] J. P. Singh, T. Alpcan, P. Agrawal, and V. Sharma, An optimal flow assignment framewor for heterogeneous networ access, in Proc. WoWMoM, June 2007, pp [3] T. Alpcan, J. P. Singh, and T. Başar, Robust rate control for heterogeneous networ access in multihomed environments, IEEE Transactions on Mobile Computing, vol. 8, no. 1, pp , January [4] L. Lin, X. W. Zhou, L. P. Du, and X. N. Miao, Differential game model coupling constraint for routing in ad hoc networs, in Proc. WiCom, September 2009, pp [5] C. U. Saraydar, N. B. Mandayam, and D. J. Goodman, Pricing and power control in a multicell wirless data networ, IEEE Journal on Selected Areas in Communications, vol. 19, no. 10, pp , October [6] S. Jørgensen, G. Martín-Herrán, and G. Zaccour, Agreeability and time consistency in linear-state differential games, Journal of Optimization Theory and Applications, vol. 119, no. 1, pp , October [7] E. J. Docner, S. Jørgensen, N. V. Long, and G. Sorger, Differential Games in Economics and Management Science. Cambridge Univ. Press, November ACKNOWLEDGMENT This wor was done in the Centre for Multimedia and Networ Technology CeMNet) of the School of Computer Engineering, Nanyang Technological University.