Power Budgeted Paket Sheduling for Wireless Multimedia Praveen Bommannavar Management Siene and Engineering Stanford University Stanford, CA 94305 USA bommanna@stanford.edu Niholas Bambos Eletrial Engineering and Management Siene and Engineering Stanford University Stanford, CA 94305 USA bambos@stanford.edu John Apostolopoulos Hewlett-Pakard Labs Palo Alto, CA 94304 USA john apostolopoulos@hp.om Abstrat In this paper we profile a partiular tradeoff between power budget and video quality that emerges in the transmission of multimedia pakets over a wireless hannel. These pakets are due to arrive to a reeiver at a partiular time, so we onsider a finite horizon problem over whih multimedia data are transmitted. Due to the lossy nature of the wireless hannel, however, not every paket an be suessfully sent aross the hannel. Hene, eah paket that is lost leads to distortion in the video that is experiened by the reeiver. We suppose that there are M pakets that must arrive at the reeiver within N time steps, but that power limitations onstrain the number of transmissions. At eah time step, we may make a measurement of the wireless hannel and deide whether or not to transmit apaketoverthehannelatthattime.firstwewillsuppose the times at whih the hannel state is sampled are spaed far enough apart so that the samples are i.i.d. Then we will ontinue by supposing that the hannel state follows a Markov hain. Index Terms Dynami Programming, Multimedia, Wireless, Power Control I. INTRODUCTION As the state-of-the-art advanes in multimedia ompression and streaming tehnologies, a number of tehnial hallenges have also arisen related to transmitter power ontrol and signal distortion. In partiular, there is a tradeoff between power usage on the transmitter side versus quality of playout on the reeiver side. That is, to mitigate adverse effets suh as underflow of video ontent resulting in disruption of the user experiene) and distortion of frames, the transmitter must inrease the power used to transmit data to the reeiver. In this paper, we fous on one aspet of this tradeoff between power onsumption and playout quality and determine an optimal poliy for balaning this tradeoff with a dynami programming approah. While the omplete piture of ommuniating multimedia over lossy hannels is more involved than the model presented here, we spotlight one partiular type of power budget onstraint and hoose to address refinements to the model in future work. Indeed, the spae of wireless multimedia is a large one with many parameters and onstraints. The speifi area of power ontrol is an area whih has seen signifiant growth in reent years. Dynami programming as well as heuristi methods are employed to ahieve optimal or near-optimal performane for a given set of parameters. Espeially noteworthy is [1], where a dynami programming framework for performing rate-distortion optimized streaming was proposed. Key ontributions offer solutions to manage the tradeoff between the delay of individual pakets and the power spent in their transmission [2]. Other approahes, suh as adaptive playout, have been used in related problems suh as streaming audio and video over the Internet [3]. Optimization of media over a network has also seen attention, as in [4] where ross-layer optimization over wireless is studied, as well as [5] where dynami programming is used to find quantization levels and perform rate ontrol. For example, [6] develops a DP framework to deide whih media units to transmit and whih to disard in order to get rate-distortion optimized streaming. Some ontributions utilize hannel flutuations to more effetively transmit video over wireless while still others use dependenies between media units. In [7], the authors jointly leverage power and playout ontrol to ahieve a ertain playout quality yet minimally stress the wireless hannel and battery. In this work, we take a different approah to power ontrol. We onsider a finite horizon problem so that all pakets that we send are sheduled to arrive before a ertain time, a situation frequently enountered in appliations suh as video onferening where large delays are not tolerated and all pakets of eah frame need to arrive within a time interval allowed for this frame. Over the duration of the problem, the number of times that pakets may be sent is onstrained. This orresponds to a ommitment to use a ertain amount of battery power over that time, but no more. Hene, if there are many opportunities remaining to make transmissions, the optimal poliy will be more liberal and will attempt to send information even in suboptimal hannel onditions. On the other hand, if the remaining hanes to transmit are sare, the poliy will be very onservative and only send data when hannel onditions are extremely good. This approah to resoure limited ontrol was introdued in [8] where the two agents are to ommuniate the state of arandomproessfromonetotheotherusingahannelthat may only be aessed a ertain number of times. Work in this diretion has also been done in the ontext of power limited surveillane/monitoring, where observations are expensive but
one would still like to trak the state of a random proess arefully [9]. Our paper is organized as follows: In Setion 2 we desribe our problem and model it mathematially. Setion 3 provides amethodforoptimallymakingtransmissionsintheasethat hannel quality states are i.i.d. and Setion 4 onsiders thease in whih they evolve aording to a Markov model. Setion 5 provides numerial results and offers a disussion of them and finally Setion 6 summarizes our findings and offers diretions for future work. II. MODEL Let us begin by desribing our problem in greater detail. We are interested in a senario in whih multimedia pakets must be transmitted aross a lossy wireless hannel in disrete time. The pakets are to be proessed by the reeiver and hene are due at a ertain time. Therefore, we utilize a framework of finite-horizon ontrol. This situation is seen in mobile video ommuniation where the nature of the appliation is intolerable to pakets arriving after some partiular deadline. In this framework, we onsider that a number of pakets, M, must be transmitted over the hannel over N time steps. In these N time steps, the hannel s quality varies so that the probability of a suessful transmission is hanging over time. Without limitations on the usage of this hannel, the optimal strategy is straightforward: one would simply utilize every time step to send a paket. If aknowledgements are present to alert the transmitter of a failed transmission, one may retransmit as many of those as possible, sine there are only M pakets to transmit and N opportunities to do so. If aknowledgements are not present, one an retransmit those pakets that were sent during the poorest hannel onditions to maximize the expeted number of pakets sent. The reality, however, is that there are limitations on the amount of power that may be utilized, espeially in the ontext of a mobile environment. In light of this, we model an additional onstraint on the number of times the hannel may be utilized. Without loss of generality, we may take this limitation to be M < N. If there are more opportunities to transmit than there are pakets to send, pakets an be retransmitted in the manner desribed above. The hannel in Fig. 1 represents one in whih there is a limit on the number of times it may be used and an assoiated hannel state at eah time k. Fig. 1. Transmission of video pakets over a wireless hannel We note that although we do not aount for dependenies between video pakets that are strung together, the ost struture an be modified and the state spae may be augmented to aount for this in the onstrution of an optimal poliy. A. Mathematial Formulation This situation an be aptured by onsidering a olletion of possible hannel states, S, and eah x S has an assoiated probability and hannel quality state as measured by the probability of a paket going through). We begin by onsidering the i.i.d. random proess {x k N 1 to model the hannel quality state of the wireless hannel. Eah x k takes avaluex S with probabilities px). Thehannelquality state shall be denoted by onsidering that eah x S has a probability of suess x) [0, 1]. Consider the lass of transmission poliies onsisting of a sequene of funtions Π={µ 0,µ 1,...µ N 1 1) where eah funtion µ k maps the information available to the ontroller at time k to an ation u from the set A = {transmit, don t transmit with the additional onstraint that one may transmit at most M times. We want to find a poliy π Π to minimize the ost { N 1 JM,N) π = E x k )I paket not sent at time k 2) where the indiator funtion I is one if it is deided not to send the paket and zero otherwise, and where J M,N) is the ost-to-go when there are N time steps remaining and M opportunities to transmit information. The searh for an optimal poliy an also be written: J M,N) =min π Π J π M,N) 3) That is, we are minimizing the hannel quality state of the hannel for the time slots in whih transmissions are not made, whih is equivalent to maximizing the hannel quality state of the hannel over time during whih transmissions are made. The problem is trivial when M = N beause in this ase one an simply send a paket at every time step whih results in zero ost or full hannel utilization). So we shall only onsider the ase in whih M<N. After first studying the ase in whih the hannel state {x k N 1 is desribed by a i.i.d. random proess, we move on to a senario in whih it evolves aording to the dynamis of a Markov hain. III. I.I.D. CASE We begin by modeling the hannel states as oming from an i.i.d. distribution. Due to the fat that the past is deoupled from the future aumulated ost in the i.i.d. ase, the optimal poliy for time k needs only onsider {x k,s k,t k where s k and t k are the number of paket transmissions left and the number of time slots remaining, respetively. For simpliity we will drop the subsript for s k and t k. We reformulate the problem so that we seek a poliy suh that we should transmit a paket if x k Λ and not send a paket otherwise. Λ is some subset of the set of all possible hannel states S. Wetakeadynamiprogrammingapproah to this end.
From the DP equation we have, J =min Λ { P [x k Λ ]J s 1,t 1) + P [x k Λ ]J s,t 1) + px)x) A. Struture of the solution Using the results above, a table Fig. 2) of optimal average errors may be onstruted offline and referened as deisions are being made in real time. That is, for eah s, t), wemay determine a orresponding Λ using 4). This an been seen by noting that the first term of the minimization is the ost-to-go from the next stage if the hannel is used in the urrent time step, the seond term is this ost-to-go when the hannel is not used and the third term is urrent stage ost. It an equivalently be written: J =min Λ { 1 P [x k Λ ] ) J s 1,t 1) + P [x k Λ ]J s,t 1) + px)x) by onsidering the omplement of the set Λ.Nowwean plug in P [x k Λ ]= x Λ px) and rearrange terms to get: { ) J =min J Λ s 1,t 1) J s,t 1) px) + px)x) + Js 1,t 1) whih we an modify by ombining like terms to get: J = J s 1,t 1) { )) +min px) x) J Λ s 1,t 1) J s,t 1) The minimization is over all possible subsets Λ whih is in general ombinatorial and intratable. In this ase, however, we an exploit the struture of the objetive to onlude that for an optimal poliy we must have: x Λ x) <J s 1,t 1) J s,t 1) 4) It is lear that 0 J s 1,t 1) J s,t 1) max x S x). Replaing our result into the previous equations, we finally get J = J s 1,t 1) + x S px) x) )) Js 1,t 1) J s,t 1) This equation holds for 0 <s<t N.Wealsomustonsider the following boundary onditions: ) J 0,t) = t x S px)x), Jt,t) =0 5) whih follow from the fats that when s =0,wearuea ost at every stage equal to the average hannel quality state. Fig. 2. Offline dynami programming solution struture One these values are obtained, one may apply the poliy u x) ={ 0 if x Λ 1 otherwise where an ation of u =0orresponds to no transmission and an ation of u =1orresponds to transmission of the paket. We finally note that although these omputations involve taking sums over a possibly large state spae, the sums orrespond to inner produts whih may be doing effiiently by using a kernal trik. IV. MARKOV CASE We turn to the ase in whih the quality of the hannel at eah time, x k,isnotdesribedbyani.i.d.randomproess,but rather as a disrete-time Markov hain. We shall now onsider asenarioinwhihthehannelstateisstilldesribedbya finite olletion of states, but the transitions between them follow Markov hain dynamis. In addition to the notation developed above, we also introdue a transition matrix P whih aptures the probability of transitioning from one state to another. We onsider the state spae S to be ordered so that we an map the rows and olumns of P to speifi states. For onveniene we also introdue the vetor, whihisavetor of the probabilities of suessful transmission, x), foreah hannel state, x. In the following development, we again seek a poliy π Π to minimize the ost { N 1 JM,N) π = E x k )I paket not sent at time k 6)
We proeed to onstrut the solution using bakwards indution. We begin with t =1,whihorrespondstoone unit of time remaining in the problem, and then ontinue for t =2, 3,... until we are able to determine a reursion. As we build bakwards in time and forward in t), we let s vary and keep trak of the ost J s,t x) where x S is the urrently observed hannel state. Note the abbreviation from x N t) ). For t =1,weaneitherhaves =0or s =1.Theseosts, respetively, are in vetor form) J 0,1) = J 1,1) =0 sine not having a transmission opportunity means that we arue a ost equal to the probability of a suessful transmission from that state. Moving on to t =2,thevaluesofs an range from s =0, s =1or s =2.Fors =0we have J 0,2) = + P sine we arue a ost equal for the urrent stage as well as the next stage, when the state has hanged aording to the transition matrix P.Whens =1,therearetwohoies:use an opportunity to make an observation so that u =1or do not observe, in whih ase u =0.Thesehoiesanbedenoted with supersripts above the ost funtion for eah stage: J 0) 1,2) = J1) 1,2) = P For u =0,wearueerrorfortheurrenttimeslotandno error afterwards. When an observation is made, no error is arued for the urrent time slot N 2, butthereiserrorin the next time slot whih depends on the urrent observation. We now introdue some new notation: 1,2) = J 0) 1,2) J 1) 1,2) =I P ) so that if 1,2) x) 0, then we should not make a transmission, whereas we should make one if 1,2) x) > 0. We proeed now by defining sets τ 1,2) and τ1,2) suh that x τ 1,2) 1,2)x) 0 x τ 1,2) 1,2) x) > 0 and we also define an assoiated vetor 1 1,2) {0, 1 S 1 1,2) x) ={ 1 if x τ 1,2) 0 otherwise Moving on to s = 2, we have J 2,2) = 0, sine there are as many opportunities to make transmissions as there are remaining time slots. We ontinue with t =3: J 0,3) = + P+ P 2 sine there are three time slots for whih ost is arued. For s =1,weagainhaveahoieofu =0and u =1.For u =0,wearueaostfortheurrentstage,andthenount the future ost depending on the urrent state: J 0) 1,3) x) =x)+ x, y) 1 1,2) y)j y SP 0) 1,2) y) and ombining terms gives us J 0) 1,3)x) =x) + P x, y) y S +1 1 1,2) y))j 1) 1,2) y) ) J 1) 1,2) y)+1 1,2)y) 1,2) y) whih after substituting the value of J 1) 1,2)x) and putting things in vetor form gives us: J 0) 1,3) = + P 2 + Pdiag1 1,2) ) 1,2) Now we onsider the u =1ase: J 1) 1,3) = P+ P 2 sine there no urrent ost, but the last two stages produe osts that depend on the urrently observed hannel state. We now write the expression for 1,3) = J 0) 1,3) J 1) 1,3) : 1,3) = P+ Pdiag1 1,2) ) 1,2) Continuing with s =2, whereas for u =1, J 0) 2,3) = +0 J 1) 2,3) x) =0+ y S P x, y) 1 1,2) y)j 0) 1,2) y) +1 1 1,2) y))j 1) 1,2) y) ) where we have aounted for the ost stage by stage: in the urrent stage, no error is arued sine an observation is made but future osts depend on the observation that is made. That is, future osts depend on whether the next state x N 2 is observed to be in the set τ 1,2). Averaging over these, we obtain the expression above. Combining like terms, we arrive at: J 1) 2,3) = P 2 + Pdiag1 1,2) ) 1,2) We use these expressions to get 2,3). 2,3) = P 2 Pdiag1 1,2) ) 1,2) Finally, letting s =3,weeasilysee:J 3,3) =0.Thisproess an be ontinued for t =4, 5,...Foreahstages, t), we may determine J 0) 1) and J.Theseoststhenallowusto determine when we should make an observation in the proess and when we should not. The implementation of this poliy is detailed in the following subsetion. )
A. Reursions We now present a method for onstruting an optimal poliy. We do this by storing for eah s, t) asubsetofs, denoted by τ,whihisthesetoflastobservedstatesforwhih we do not use an opportunity to view the proess when we are at stage s, t). Thatis,iftheurrentlyseenhannelstate x is in the set τ,thereares opportunities remaining to make transmissions and there are t time slots remaining in the horizon then we should not make a transmission at this time. On the other hand, if x τ then we should make a transmission at stage s, t) and arue zero ost for that stage. More preisely, an optimal poliy π is given by u x) ={ 0 if x τ 1 otherwise Let us introdue three vetor valued funtions: F, R S and 1 {0, 1 S. We fill in values for these funtions by using the following reursions: F = P F s 1,t 1) + 1 s 1,t 1) s 1,t 1) ) = + P F s,t 1) + diag1 s,t 1) ) s,t 1) ) F { 0 if x) > 0 1 x) = 1 otherwise for 1 <s<t<n.wealsohavetheboundaryonditions F t,t) =0, F 1,t) = t 1 m=1 P m, t,t) = These reursions allow us to determine the sets τ for s, t in the bounds speified, whih in turn defines our optimal poliy. Speifially, we assign x τ x) 0 We onlude by giving expressions for the ost-to-go from any partiular state when a partiular ation u {0, 1 is taken. The supersripts denote whether or not an observation will be made in the urrent stage. J 0) = + P F s,t 1) + diag1 s,t 1) ) s,t 1) ) J 1) = F Observe that is the differene between these two quantities. Hene, an be used to deide whether or not to make an observation in the urrent time step. V. NUMERICAL RESULTS Let us now onsider a sample problem in whih we must transmit video data over an i.i.d. wireless hannel. We suppose that there are three 1,2,3) hannel quality states and that the pakets are all due within 20 time steps. The following values are used: x) =0.1 0.5 0.9) T, px) =0.7 0.2 0.1) T Note that for the most part, the probability of suessful paket transmission is quite low 0.1), but on oasion the hannel has deent or very good quality. A. Throughput We plot the expeted number of suessfully transmitted pakets over the horizon N =20when using the algorithm above as we vary the number of pakets that may be transmitted. In the notation developed above, we plot T p J s,20) as we vary s. Wealsoplota ontentaware heuristiwhih sends the pakets over the horizon in suh a way that all the ontent has been attempted to be transmitted without onern for hannel onditions, along with a hannel aware heuristi whih only transmits if the hannel is not in the worst state, but does not neessarily send all pakets. Expeted Number Suessful Pakets 6 5 4 3 2 1 Expeted Number of Suessful Pakets vs. Number of Transmissions Optimal Content aware Channel aware 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Number of Transmissions Fig. 3. Plot of expeted number of suessful pakets vs. number of transmissions There is a region in whih the hannel aware approah performs almost as well as the optimal poliy, speifially when the number of times the hannel an be used is small ompared with the horizon of the problem. After a point, however, its performane drops off beause it does not utilize the hannel when onditions are poor, but suh a poliy might be neessary sometimes. On the other hand, the ontent aware approah does well when there are almost as many transmissions as the horizon of the problem. If this is not the ase, however, it does not differentiate between a good or poor hannel and results in more lost pakets. B. Thresholds Let us also onsider a plot of the optimal thresholds for transmitting a paket. That is, we vary s and plot the lowest hannel ondition whih will result in a transmission. In this simple example there are only three states to hoose from 0.9, 0.5 or 0.1). It an be seen in Fig. 4 that the thresholds are dereasing sine having more battery power results in more liberal usage of the hannel. We also note that when s = t we an transmit at every time step), the threshold for transmission drops to zero sine it is always optimal to transmit.
Transmission Threshold 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 Transmission Threshold vs. Number of Transmissions 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Number of Transmissions Fig. 4. C. Comparisons Plot of threshold vs. number of transmissions We finally note that the plot shapes depend greatly on the probabilities of hannel state and the probabilities of suessful transmission. In Fig. we plot the optimal tradeoff urve for p =0.1, 0.9), =0.1, 0.9), whihrepresentsa good hannel, as well as for p =0.7, 0.3), =0.1, 0.9) whih represents a bad hannel. The worse the hannel gets, the higher the optimal urve lies above the orresponding urve for a linear ontent aware approah. Hene, the worse and more varied) the hannel onditions, the greater the advantage to using the optimal poliy. Expeted Number Suessful Pakets 18 16 14 12 10 8 6 4 2 Expeted Number of Suessful Pakets vs. Number of Transmissions Good Channel, Optimal Good Channel, Content Aware Bad Channel, Optimal Bad Channel, Content Aware VI. CONCLUSIONS In this paper, we have examined a problem in ommuniation of multimedia data over lossy hannels when there are limits on hannel usage. We mathematially model this as a finite horizon ontrol problem and utilize a dynami programming framework to develop poliies that maximize throughput over the hannel for i.i.d. hannel states as well as Markov hains. We see that a thresholding poliy on the hannel state is optimal, and deisions depend on the number of opportunities remaining to transmit as well as the number of time slots left in the problem. The resulting performane utilizes the hannel when it is very good but also keeps trak of the amount of time left in the proess so that thresholds dynamially hange aording to the observed hannel states and previous ations. This ombination yields better performane than an be obtained by heuristis emphasizing a hannel-aware approah or a ontentaware approah, as an been seen in the previous setion. There are many related problems and extensions to onsider in future work. One diretion may be to investigate a senario in whih hannel states still evolve aording to a Markov model, but the state is unknown to the deision maker until a transmission is made. ACKNOWLEDGMENT The first author is supported by the Department of Defense DoD) through the National Defense Siene & Engineering Graduate Fellowship NDSEG) Program. REFERENCES [1] P. Chou and Z. Miao, Rate-distortion optimized streaming of paketized media, IEEE Trans. Multimedia, vol. 8, no. 2, pp. 390-404, Apr. 2006. [2] N. Bambos and S. Kandukuri, Power Controlled Multiple Aess PCMA) in wireless ommuniation networks, in Pro. of IEEE IN- FOCOM 2000, New York, June 2000, pp. 368-395. [3] Y. Liang, N. Faerber, and B. Girod, Adaptive playout sheduling and loss onealment for voie ommuniation over IP networks, IEEE Trans. Multimedia, vol. 5, no. 4, pp. 532-543, De. 2003. [4] S. Shankar, Z. Hu, and M. Van der Shaar, Cross layer optimized transmission of wavelet video over IEEE 802.11a/e WLANs, in Pro. IEEE Paket Video 2004, Irvine, CA, De. 2004. [5] J. Cabrera, A. Ortega, and J. I. Ronda, Stohasti rate-ontrol of video oders for wireless hannels, IEEE Trans. Ciruits Syst. Video Tehnol., vol. 12, no. 6, pp. 496-510, Jun. 2002. [6] M. Kalman, E. Steinbah, and B. Girod, Adaptive media playout for low delay video streaming over error-prone hannels, IEEE Trans. Ciruits Syst. Video Tehnol., vol. 14, no. 6, pp. 841-851, Jun. 2004. [7] Y. Li, A. Markopoulou, N. Bambos and J. Apostolopoulos, Joint Power/Playout Control Shemes for Media Streaming over Wireless Links, Pro. 14th IEEE Int l Paket Video Workshop, De. 2004. [8] O.C. Imer. Optimal Estimation and Control under Communiation Network Constraints. Ph.D. Dissertation, UIUC, 2005. [9] P. Bommannavar and N. Bambos, Optimal Surveillane with Budget Constraints, Tehnial Report, Stanford University, 2010. 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Number of Transmissions Fig. 5. poliy Plots for two hannel onditions, with gains over ontent aware