Low Complexity Power and Scheduling Policies for Wireless Networks to Provide Quality of Service

Low Complexity Power and Scheduling Policies for Wireless Networks to Provide Quality of Service Satya Kumar V, Vinod Sharma Department of Electrical Communications Engineering, Indian Institute of Science, Bangalore Email: satyakumar@ece.iisc.ernet.in, vinod@ece.iisc.ernet.in Abstract We consider optimal power allocation policies for a single server, multiuser system. The power is consumed in transmission of data only. The transmission channel may experience multipath fading. We obtain very efficient, low computational complexity algorithms which minimize power and ensure stability of the data queues. We also obtain policies when the users may have mean delay constraints. If the power required is a linear function of rate then we exploit linearity and obtain linear programs with low complexity. Index Terms Fading channels, Average queue constraint, Optimal average power policies. I. INTRODUCTION Information and communications technology (ICT) industry is consuming about 3% of world power consumption. This percentage will further increase as the number of mobile users is growing at a rapid pace across the globe. Any reduction in power consumption in the ICT will reduce the ever increasing carbon footprint from this sector. It will also reduce the operational cost of the service providers and will need smaller diesel generators and batteries. Thus one of the primary challenges for Next Generation Networks (NGN) is to reduce energy consumption in the cellular network 7]. In a Base Station (BS) the power is consumed mainly in the air conditioner (25%), Radio equipment (61%), DC power (11%) and RF load and Feeder (3%). In this paper we will concentrate on reducing power in RF transmission, while providing quality of service (QoS) to the users. This indirectly reduces the power consumed in the other components also 16]. Goldsmith and Varaiya 10] obtained a power allocation policy for a single fading link which optimizes the rate. Energy efficient scheduling under average delay constraint was first addressed in 4], 5]. Goyal et al. 11] considered the same problem, proved existence of an optimal policy and obtained some structural results for the optimal policy. 24] proposed delay and power optimal policies in a single server queue when the power used in transmission is a linear function of rate. 9] proposed an energy efficient scheduling with individual packet hard delay constraints. 13] presented a policy which minimizes energy for sending a fixed number of packets under given delay constraints. Neely 18] proposed a new innovative algorithm which optimizes power while satisfying an upper bound on the sum of the average queue lengths of multiple users. The algorithm needs to learn a parameter. 25] proposed *Partially funded by a project from ANRC. a suboptimal policy which minimizes the average power under average queue constraint when there is an upper bound on packet loss also. 28] obtained asymptotic lower bounds for average queue length and average power consumption. Neely 17] presented an algorithm for a multiuser and multi channel scenario subject to an upper bound on the sum of average queue length. To meet the constraints the algorithm needs to learn two parameters. In 21] new throughput optimal energy management policies were developed when the transmitter is powered by energy harvesting sources. Explicit throughput optimal policies which also minimize mean delay were also obtained when linear power rate relation holds. 22] provides efficient MAC policies in the same setup. 19] implemented an online algorithm by modifying the value iteration equation and provides QoS by minimizing the average power for a single user. A. Problem Statement and our contribution Our objective is to minimize the average transmit power while providing QoS, such as stability of the data queue or an upper bound on the average delay. The later problem is usually formulated as an Markov Decision Problem (MDP). However, finding an optimal solution via an MDP is computationally hard and does not usually provide any insights to the solution. A few other algorithms provided in literature use learning algorithms. These can be slow to converge and also do not provide any insight into the optimal solution. We are not aware of any algorithm which guarantees an upper bound on mean delay especially in a multiuser fading environment except via MDP or learning. As against the above work, we provide algorithms which are in closed form or require very low computational complexity. Our algorithms do not require queue length information or arrival statistics. These algorithms are optimal in the class of algorithms requiring only channel gains and provide explicit guarantees on mean delays for individual users. These algorithms do not require any learning. Furthermore, either these are probably optimal or shown to be close to optimal via computations on explicit examples and perform better than other existing low complexity algorithms. The paper is organized as follows. Section II presents the system model. Section III presents our algorithm that minimizes average power and guarantees stability of the data queue. Section IV considers the problem when the users also

specifically consider the case, when f j can be approximated by a linear function, R j k = αp j k Hj k, (3) Fig. 1. System Model. require mean delay guarantees. Section V provides examples illustrating these algorithms and compares our results with other existing well known algorithms. Section VI concludes this paper. II. SYSTEM MODEL We consider a single server, discrete time queue with M user. A slot is of duration 1. Let A j k bits arrive in slot k, representing interval k, (k + 1)), k 0 for user j. The data to be transmitted arrives into the system from higher layers at the beginning of each slot and placed into an infinite buffer queue till transmission (Figure 1). Arrival process {A j k, k 0} is an independent and identically distributed (iid) sequence with mean EA j ]. We will some times also consider the case, when the arrival process is stationary, ergodic (e.g. modeled as Markov chain) We assume that channel gain H j k is constant over the duration of slot k and {H j k, k 0} is iid, where Hj k {h j 1, hj 2,..., hj L j } and 0 < h j i < hj l <, for i < l, 1 j M and h j L j <. We also assume that the channel gain H k = (Hk 1,..., HM k ) is available at the transmitter at time k. Let P H k = h] = p(h). The i.i.d assumption on {A j k } and {Hj k } are commonly made (1], 6], 27]). The H j k taking values on a finite set can be a good approximation for continuous distribution, e.g., Rayleigh, Rician, Nakagami, by taking the finite set arbitrarily large. Let R j k bits be transmitted in slot k from user j and let the power used be P j k. Let qj k denote the queue length in packets at time k for user j. Then {q j k } evolves as, q j k+1 = (qj k + Aj k Rj k )+, k 0, (1) where (x) + = max(0, x). We relate R j k with P j k via a general relationship Rj k = f j (P j k, Hj k ), where f j is a nondecreasing (in P j k and Hj k ), nonnegative function. A common function is, the Shannon function, R j k = 1 2 log 2(1 + P j k Hj k /σ2 ). (2) This assumes that the channel is Additive White Gaussian Noise (AWGN) fading channel with noise power σ 2. We will where α j > 0. This is a good approximation of Shannon formula (2) at low SNR 24] and also at high bandwidth (3], 14]). However as shown in 21], even when (2) cannot be approximated well by a linear function, (3) can still hold in a practical systems.we will show that linearity can be exploited to obtain more efficient algorithms. First, we consider the system without a delay constraint. Later on we provide the solution with delay constraints. III. OPTIMAL POLICIES : STABLE SYSTEM In this section we consider the problem of minimizing average power lim sup n M 1 n E n 1 P j k, (4) j=1 k=0 subjected to the stability of all the queues, i.e., the queue length process q k = (qk 1,..., qm k ) has a stationary distribution. This could be the QoS if all the users are best effort users in the Internet. This problem can be solved via MDP but computing the optimal solution is too complex to compute for more than one user. To reduce complexity we ignore the queue length process in obtaining the optimal policy. We will see that by obtaining an optimal policy which depends only on the channel states and the dynamics of the queue we can obtain policies which are close to the optimal policies, obtainable via MDP. For simplicity first we obtain the policy for a single user in Section III-A and will then extend to the multiuser case in Section III-B. We will call these policies greedy opportunistic policies. A. Single User In this section we remove the user index j. We want to obtain a power control policy P (h) that minimizes p(h i )P (h i ), (5) i subjected to, p(h i )f(p (h i ), h i ) EA] + ɛ, (6) i where ɛ > 0 is an arbitrarily small constant. The constraint (6) is to ensure that the queue is stable. If f is concave, e.g., Shannon function, then (5)-(6) has a linear cost function with convex constraints. Instead of solving for P (h) for each h, we can solve for rate R(h) for each h and then this becomes an optimization problem with convex cost function with linear functions. In either case it is a comparatively easy optimization problem to solve and the globally optimal solution can be obtained by solving KKT conditions. Also, it requires only channel statistics, current channel gain and EA]. Another important feature of this

policy is that when {(A k, H k ), k 0} is iid then this optimal policy retains the GI/GI/1 character of the queue. Thus, the numerous results and approximations available 2] for such queues will be applicable for this system and can be exploited to provide QoS guarantees to the wireless system. We will also see that the optimal policy obtained is very close to the overall optimal policy via MDP which requires arrival statistics and queue length information also and has a very high computational complexity. If {A k, H k } is stationary, ergodic (a generalization very useful in practice, especially to model voice and video traffic) then the solution of (5)-(6) again provides the optimal policy in the class of policies which consider only the channel gain. If the function f is linear, then a simple optimal closed form policy is available, in the whole class of policies : serve all the data in the queue whenever H k = h L, i.e., clear the queue whenever the channel gain is maximum. This is the optimal policy even when {(A k, H k )} is stationary, ergodic. This policy does not require any arrival or channel statistics; only the current channel gain and consumes average power = EA] αh L. For this optimal policy, the system has several other desirable features. When {A k, H k } is iid (in fact we can allow {A k } iid and {H k } Markov modulated with the modulating Markov chain irreducible with a finite state space), the epochs when H k = h L are regeneration epochs for the process {q k }. The process has a stationary distribution π. The queue length process converges in total variation to π exponentially. Furthermore E π q γ ] < for any γ 1 if EA γ+1 1 ] <. We also have E π q] = EA]( 1 p L 1). B. Multiuser If there are M > 1 users, the optimal policy, ignoring the queue length information is obtained as follows. Let the power needed to transmit R bits in channel state h for user j be g j (R, h) (obtainable from f j ). Then we seek an optimal policy which provides rate R j (h) in channel state h for user j with probability β(j, h). In a slot only one user will transmit. Thus, we find R j (h) and β(j, h) for each channel state vector h and j that min M p(h)g j (R j (h), h)β(j, h) j=1 h H subject to: R j (h)p(h)β(j, h) EA j ] + ɛ, j = 1,..., M, h H M β(j, h) 1, h, j=1 (7a) (7b) (7c) 0 β(j, h) 1, j, h. (7d) This is a non-convex optimization problem : the objective function as well as the constraints are non-convex even when g i are convex. But if all the functions g i are smooth enough, e.g., have continuous second order partial derivatives, then we can obtain locally optimal solutions quite easily (using MATLAB). Starting the optimization algorithm with different initial conditions and picking the best solution provides an efficient solution. Any solution obtained this way has the desirable property that each of the queues behaves as an independent GI/GI/1 queue and hence its performance, e.g., mean delay can be approximately obtained from available GI/GI/1 queue results. The problem becomes much simpler if we can approximate f j by linear functions. Then an optimal policy (among all policies) is : in slot k clear queue j if H j k = hj L j. (8) If there is more than one such queue, pick one among them with the longest queue. If even then there is more than one such queue, pick with equal probability any one of them. This policy does not require any channel or arrival statistics but only channel gains and queue lengths in the current slots. Serving the longest queue does not reduce average power; only mean queue length. If we ignore the queue length and serve one of the j users satisfying (8), with probability (say equal), then from the channel statistics we can compute the probability p j for user j with which its queue is served in any slot. In addition to being optimal, this remarkably simple policy has all the desirable properties mentioned above for a single user : The overall average power consumed = j EA j ] α jh j L j. The system has a unique stationary distribution π and starting from any initial distribution, q k converges in total variation to π exponentially fast. E π q γ j ] < whenever E(Aj 1 )γ ] < for any γ > 1. E π q j ] = EA j ]( 1 p j 1), j = 1,..., M. This policy is optimal for any {(A k, H k )} process and does not require any arrival or channel gain statistics. It does not require any fragmentation of packets. IV. OPTIMAL POLICIES : WITH MEAN DELAY CONSTRAINTS Thus far we have been concerned about minimizing constraints subject to stability of the system. For some real time and data applications, it will be desirable to have upper bounds on mean delay. Such constraints can be translated into mean queue length constraints via Little s law 2]; the exact relationship between mean delay and queue length depends on whether we consider a packet model or fluid model. In the following we assume that it has been sorted out and finally we have translated our QoS to the constraint, n ] lim sup n 1 n E k=1 q j k q j, j = 1,..., M. (9) Again, this problem can be addressed via MDP but getting optimal solution, at least for the multiuser case is computationally very demanding. Due to all the desirable properties we have discussed so far, we limit ourselves to policies which depend only on the channel state information.

Total average queue length 4.5 4 3.5 3 2.5 2 1.5 1 Fig. 2. EECA B Bernoulli MaxTrans B Bernoulli TOCA B Bernoulli Greedy 0.4 0.5 0.6 0.7 0.8 0.9 Average power Comparison of policies for three users for Bernoulli arrivals. We consider a policy which selects user j with probability β(j, h) when the system channel state is h and chooses to transmit R j (h) bits. We find β(j, h), R j (h) that solve the optimization problem (7a), (7c), (7d) with the additional constraints (9). Under our policy, left hand side of (9) can be replaced by E π q j ] where π is the stationary distribution of {q k } under the given policy. Even though under our policy each queue is like a GI/GI/1 queue with q j k satisfying q j k+1 = (qj k + Aj k Rj k )+, (10) a closed form expression for E π q j ] is not available. But several approximation are available for mean delay of a GI/GI/1 queue (which can be used here because q j k satisfies the Lindeley equation 11]) we use 12], where E π q j ] ρ(j)d(j)eaj](c2 A (j) + C2 R (j)). (11) 2(1 ρ(j)) ρ(j) = EAj ] ER j ], C2 A(j) = V araj ] (EA j ]) 2, C2 R = V ar(rj ) (ER j ]) 2 (12) 2(1 ρ(j)) (1 CR 2 exp (j))2 3ρ(j) CR 2 d(j) = (j) + C2 A (j)], if C2 R(j) < 1 (C R 2 (j) 1) exp (1 ρ(j)) CR 2 (j) + 4C2 A (j)], if C2 R(j) 1 (13) E(R j )] = h H R j (h)β(j, h)p(h). (14) We use these approximations on the LHS of (9) and solve the optimization problem (7). Again it is a non-convex optimization problem and we obtain our solution by getting locally optimal solutions from several initial conditions and choosing the best solution. We compare our results with Neely s 17] Tradeoff Optimal Control Algorithm (TOCA) and Energy Efficient Control Algorithm (EECA). We have considered three users. The arrival and channel statistics of the three users are same and independent of each other. The arrival process {A k } is a Bernoulli arrival process with average arrival rate 1 and batch size 10. The channel gains take values in the set {0.01, 1} and the respective channel gain probabilities are {0.2, 0.8}. The sum average queue length vs the corresponding average powers of our opportunistic greedy policy and the TOCA and EECA policies of 17] are plotted in Fig. 2. We see that our greedy policy works much better than TOCA and EECA. In the figure we have also plotted MaxTrans policy of 20] when we have included power control and obtained the optimal policy to provide the required mean delay. EECA becomes unstable at average power 0.54. A. Linear Case If the f i can be taken linear, then we have seen in Section III that an optimal policy to ensure stability should transmit data from user j only if its channel gain is h j L j. Since multiple users may have their best channels in a slot, then a user among them is picked with a certain probability. Thus eventually (because {h k } are iid, independent of {A k }), we obtain an optimal policy where user j will transmit all of its data in any given slot with probability p j independently of other system variables. The queue length process {q j k } is regenerative with the regeneration epochs as the epochs when the jth queue is cleared. Let N be the interval between two such epochs. It is geometrically distributed with parameter p j and independent of {A j k }. Thus, for this queue, we can explicitly compute its mean queue length under stationarity as, N 1 E q j l ]P N = k N = k] E π q j ] = = k=1 l=0 EN] EA j 1 (k 1) +... + Aj k 1 ]P N = k] k=1 = EN] = EAj ](EN 2 ] EN]) 2EN] = EA j ]( 1 p j 1). (15) L j If we did not have an upper bound on E ] π q j, then any p j > 0 is sufficient to provide stability and minimum average power. However, now it is possible that E ] π q j obtained. (15) is greater than q j for some j (otherwise we have already obtained an optimal solution for the current problem). Let ˆp j be the probability such that using this in (15) provides equality for j. Now we would like to obtain β(j, h) such that j = 1,..., M, and β(j, h) > 0, only if h j = h j L j, (16) β(j, h) = ˆp j (17) h

If such a solution does not exist (in particular this will happen when P (h j L j ) < ˆp j, we need to relax condition (16). In particular, we find β(j, h), the probability we will pick user j in state h and clear its queue that solves subject to min M j=1 h H p(h)β(j, h) EAj ] α j h j ˆp j 0 β(j, h) 1, β(j, h) = ˆp j, h β(j, h) 1. j This is a Linear Program (LP) and hence provides a global optimum with comparatively much lower complexity. If L j = L for all j then this LP has ML M variables. It s complexity can be further reduced if we consider the Dual LP which has (M +L M ) variables. The complexity of this LP is much lower than that of the LP for the MDP because in MDP, queue length state is also considered. For a single user, the solution is available in closed form. We compare our greedy policy with the optimal policy via MDP. The optimal policy is obtained via policy iteration 8]. We consider the MDP with the average cost that includes transmission power (w(h, r) r αh ) and storage cost βq with a single state cost being w(h, s) + βq (which can be interpreted as a Lagrangian relaxation of our original problem 15]). For this MDP, the new average cost optimality equation is, Fig. 3. Single user for Poisson arrivals. J(q, h) = min r {w(h, r) + βq g + EJ(Q +, H + )]}, (18) where r is the action representing data transmission in the slot, J(q, h) is the optimal relative value function, g is the optimal average cost, Q + = (q r + A) and H + are the next slot queue length and channel gain. For our example within 10 iterations of the policy iteration, we obtain the optimal policy. The β used is such that at the optimal policy the average queue length is q. We compare the greedy policy of the last section and the optimal policy for two examples. In the first example, the arrival process {A k } is Poisson with rate 2. The channel gains take values in the set {0.3, 0.5, 1, 2} and the respective channel gain probabilities are {0.2, 0.4, 0.3, 0.1}. The average queue length vs the corresponding average powers of greedy policy and optimal policy are plotted in Fig. 3. We observe that the optimal curve and the greedy curve are very close to each other. We also present results, when the incoming arrival distribution is Binomial with parameters B(30, 0.1) whose average arrival rate is 3. The channel statistics are as above. We observe from Fig. 4. that the average queue length vs average power for greedy policy and the optimal policy are very close. Fig. 4. Single user for Binomial arrivals. V. CONCLUSIONS In this paper, we have presented computational low cost optimal transmit power policies for a wireless system. The user may experience channel fading and may have an average delay constraint. Obtaining optimal policies via MDP can be very computationally complex. Thus we present a greedy suboptimal policy in closed form which is very close to optimal and does not require queue length information. Next, we have extended our greedy policy to a multiuser downlink. The users may have individual average delay constraints. This policy is close to optimal and is computationally much simpler. It can also be used for an uplink. Our study shows that if we search for optimal policies using only the channel gains and ignore the queue lengths, we obtain very efficient policies at very low computational cost which can also guarantee QOS in a multiuser fading environment.

Sum of Average Queue Lengths 140 120 100 80 60 40 TOCA Policy Opportunistic Greedy Policy EECA Policy 20 5 5.2 5.4 5.6 5.8 6 Average Power Fig. 5. Five users for Poisson arrivals. REFERENCES 1] M. S. Alouini and A. J. Goldsmith, Adaptive Modulation over Nakagami Fading Channels, Wireless Personal Communications-Springer, 2000. 2] S. Asmussen, Applied probability and queues, 2nd ed, Springer-Verlag, 2003. 3] F. Berggren and R. Jantti, Asymptotically Fair Transmission Scheduling over Fading Channels, IEEE Transactions on Wireless Communications, Vol. 3, pp. 326-336, No. 1, Jan 2004. 4] R. A. Berry and R. G. Gallager, Communication over Fading Channels with Delay Constraints, IEEE Transactions on Information Theory, Vol. 48, no. 5, pp. 1135-1149, May 2002. 5] R. A. Berry, Power and Delay Trade-offs in Fading Channels, June 2000, Phd Thesis, MIT. 6] J. K. Cavers, An Analysis of Pilot Symbol Assisted Modulation for Rayleigh Fading Channels, IEEE Transactions on Vehicular Technology, Vol. 40, No. 4, November 1991. 7] A. Chatzipapas, S. Alouf and V. Mancuso, On the Minimization of Power Consumption in Base Stations using on/off Power Amplifiers, IEEE Online Conference on Green Communications, pp. 18-23, Sept. 2011 8] Dimitri P. Bertsekas, Dynamic Programming and Optimal Control, Fourth Edition, Vol. 2, 2012. 9] W. Chen, U. Mitra and J.Neely, Energy-efficient scheduling with individual delay constraint over a fading channel, WiOpt, pp. 1-10, Apr 2007. 10] A. J. Goldsmith, P. Varaiya, Capacity of Fading Channels with Channel Side Information, IEEE Transaction on Information Theory, Vol. 43, No. 6, pp. 1986-1992, Nov 1997. 11] M. Goyal, A. Kumar and V. Sharma, Power Constrained and Delay optimal policies for Scheduling Transmission over a Fading Channel, IEEE Infocom, Vol. 1, pp. 311-320, Apr 2003. 12] W. Kramer, M. Langenbach-Beltz, Approximate formulae for the delay in the queueing systems GI/GI/1,Eighth International Telemetric Congress, Melbourne, pp. 235-1-8, 1976. 13] A. Fu. E. Modiano and J. Tsitsiklis, Optimal Energy Allocation for Delay-Constrained Data Transmission over a Time-Varying channel, Proc. IEEE INFOCOM, 2003. 14] A. Fu, E. Modiano and J. Tsitsiklis, Optimal Transmission Scheduling over a Fading Channel with Energy and Deadline Constraints, IEEE Transactions on Wireless Communications, Vol. 5, No. 3, pp. 630-641, March 2006. 15] D. J. Ma, A. M. Makowski, A. Shwartz, Estimation and Optimal Control for Constrained Markov Chains, IEEE Conf. on Decision and Control, Dec. 1986. 16] Gerson A. S. M, Low-power HF Microelectronics: A Unified Approach, The Institution Of Engineering And Technology, 1996. 17] M. J. Neely, Optimal Energy and Delay Tradeoffs for Multiuser Wireless Downlinks, IEEE Transactions on Information Theory, Vol. 53, No. 9, pp. 3095-3113, Sep 2007. 18] M. J. Neely, Intelligent Packet Dropping for Optimal Energy- Delay Tradeoffs in Wireless Downlinks, IEEE Transactions on Automatic Control, Vol. 54, no. 3, March 2009. 19] N. Salodkar, A. Bhorkar, A. Karandikar and V. S. Borkar, An On-Line Learning Algorithm for Energy Efficient Delay Constrained Scheduling over a Fading Channel, IEEE Journal on Selected Areas in Communications, Vol. 26, No. 4, pp. 732-742 May 2008. 20] V. Sharma, D. K. Prasad and E. Altman, Opportunistic scheduling of wireless links, 20th International Teletraffic Congress, Canada, June 2007. 21] V. Sharma, Utpal Mukherji, Vinay Joseph and Shrey Gupta, Optimal Energy Management Policies for Energy Harvesting Sensor Nodes, IEEE Transactions on Wireless Communications, Vol.9, No.4, pp. 1326-1336, Apr 2010. 22] V. Sharma, Utpal Mukherji and Vinay Joseph, Efficient Energy Management Policies for Networks with Energy Harvesting Sensor Nodes, 46 t h Annual Allerton COnference, pp. 375-383 Sep 2008 23] D. Tse, V. Pramod, Fundamentals of Wireless Communication, Cambridge University, 2005. 24] Satya Kumar V, A. Lalitha, V. Sharma, Power and Delay Optimal Policies for Wireless Systems, National Conference on Communications(NCC), India, pp. 1-4, Feb 2012. 25] H. Wang and N. Mandayam, A Simple Packet transmission Scheme for Wireless Data over Fading Channels, IEEE Transactions on Communications, vol. 50, no. 1, pp 125-144, 2004. 26] D. G. Luenberger, Linear and Nonlinear Programming, Springer, second edition, 2003. 27] L. Vandendorpe, Multitone Spread Spectrum Multiple Access Communications System in a Multipath Rician Fading Channel, IEEE Transactions on Vehicular Technology, Vol. 44, No. 2, May 1995. 28] B. S. Vineeth and Utpal Mukherji, Tradeoff of Average Power and Average Delay for Point-to-Point Link with Fading, National Conference on Communications(NCC), India, pp. 1-5, Feb 2013.