IEEE TRANSACTIONS ON COMMUNICATIONS, VOL. 62, NO. 8, AUGUST

Transcription

1 IEEE TRANSACTIONS ON COMMUNICATIONS, VO. 62, NO. 8, AUGUST Two Tme-Scale Cross-ayer Schedulng for Cellular/WAN Interworkng Amla Tharaperya Gamage, Student Member, IEEE, Hao ang, Member, IEEE, and Xuemn Sherman) Shen, Fellow, IEEE Abstract In ths paper, we nvestgate uplnk resource allocaton for wreless local area network and cellular network nterworkng to provde mult-homng voce and data servces. The problem s formulated based on the physcal layer and medum access control layer technologes of the two networks to ensure that the resource allocaton decsons are feasble and can be executed at the lower layers. Furthermore, to effcently utlze users equpment UEs) battery power, the power dstrbuton among multple network nterfaces of the UEs s ncluded n the problem formulaton. The optmal resource allocaton problem s a multple tme-scale Markov decson process MMDP) as the two networks operate at dfferent tme-scales and due to voce and data servce requrements. We derve decson polces for the upper and the lower levels of the MMDP by decomposng each resource allocaton problem over multple tme slots to a set of resource allocaton problems for ndvdual tme slots and solvng the resource allocaton problems correspondng to ndvdual tme slots usng convex optmzaton technques. To reduce the tme complexty, we further propose a heurstc resource allocaton algorthm by dervng the decson polces based on a sngle system state. The system state conssts of average square channel gans for dual varable calculaton and nstantaneous channel gans for resource allocaton based on the calculated dual varables. Smulaton results demonstrate the achevable throughput and servce qualty mprovements by employng these two algorthms. Index Terms Cellular network, cross-layer schedulng, nterworkng, multple tme-scale Markov decson process MMDP), wreless local area network WAN). I. INTRODUCTION THE monumental growth of smart moble devces n recent years has exponentally ncreased the demand for hgher data rates wth seamless servce coverage and support for dverse qualty-of-servce QoS) requrements of heterogeneous servces [1]. Interworkng of wreless networks and multhomng capablty of the users equpment UEs) can be utlzed to satsfy the capacty, coverage and QoS requrements. Most of hgh servce demandng areas, such as hotspots, offce buldngs and arports, are covered by cellular networks and Manuscrpt receved January 9, 2014; revsed May 18, 2014; accepted July 1, Date of publcaton July 10, 2014; date of current verson August 20, Ths work was presented n part at the IEEE Internatonal Conference on Communcatons, Budapest, Hungary, June The edtor coordnatng the revew of ths paper and approvng t for publcaton was. Bada. A. Tharaperya Gamage and X. Shen are wth the Department of Electrcal and Computer Engneerng, Unversty of Waterloo, Waterloo, ON N2 3G1, Canada e-mal: amla.gamage@uwaterloo.ca; sshen@uwaterloo.ca). H. ang s wth the Department of Electrcal and Computer Engneerng, Unversty of Alberta, Edmonton, AB T6G 2V4, Canada e-mal: hao2@ ualberta.ca). Dgtal Object Identfer /TCOMM wreless local area networks WANs). Also, the recent WAN technologes have ncorporated varous mechansms, such as the hybrd coordnaton functon H) n IEEE n WANs, to facltate QoS support for dfferent servces [2]. As nterworkng mechansms can be utlzed for jontly allocatng resources of multple wreless networks, nterworkng can be used at such a hgh servce demand envronment to mprove the network capacty and coverage area. Furthermore, nterworkng can provde users wth enhanced QoS by utlzng resources from multple networks n an optmal manner based on the user requrements and the attrbutes of the networks, such as supportng QoS and moblty levels [3], [4]. Mult-homng capablty of UEs wth multple rado nterfaces allows the UEs to smultaneously communcate over multple networks, and t can be used for further mprovng the effcency of resource utlzaton n the nterworkng system [5]. In ths work, we nvestgate uplnk resource allocaton for cellular/wan nterworkng n the presence of UE multhomng capablty to acheve QoS satsfacton whle maxmzng the system throughput. The cellular network s based on orthogonal frequency dvson multple access OFDMA) and the WAN operates on both contenton-based and contentonfree pollng based channel access mechansms. Resources of the system are subcarrers of the cellular network, transmsson opportuntes TXOPs) va the two channel access mechansms of the WAN, and UE transmt power. Use of ths knd of system model enables the algorthms whch are developed n ths work to allocate resources for nterworkng latest/nextgeneraton wreless networks, such as TE/TE-A cellular networks and IEEE n/802.11ac WANs. One key challenge for resource allocaton for cellular/wan nterworkng s the hgh complexty due to exstence of multple physcal layer PHY) and medum access control layer MAC) technologes. The regon of feasble transmsson rates depends on PHY and MAC technologes of dfferent networks [6]. Therefore, resource allocaton should capture dverse PHY and MAC technologes of the networks. Furthermore, resource allocaton ntervals.e., nterval between two successve resource allocatons) of exstng cellular networks are usually shorter than those of exstng WANs [2], [7]. Therefore, the resource allocaton algorthms for cellular/wan nterworkng should be desgned to perodcally allocate cellular network and WAN resources wth shorter and longer perods respectvely, where the perods correspond to the resource allocaton ntervals of the networks. That s, resources of the two networks are allocated at faster and slower tme-scales, respectvely [8] IEEE. Personal use s permtted, but republcaton/redstrbuton requres IEEE permsson. See for more nformaton.

2 2774 IEEE TRANSACTIONS ON COMMUNICATIONS, VO. 62, NO. 8, AUGUST 2014 Our contrbuton n ths work s threefold. Frst, we propose a resource allocaton framework for cellular/wan nterworkng operatng on two tme-scales. Second, we formulate the resource allocaton problem n the proposed framework as a multple tme-scale Markov decson process MMDP) based on the PHY and MAC technologes of the two networks, and derve decson polces for the MMDP. Thrd, we propose a heurstc resource allocaton algorthm for low tme complexty. The remander of ths paper s organzed as follows. Secton II summarzes related works and Secton III descrbes the nterworkng system model. Secton IV presents the MMDP based resource allocaton. Sectons V and VI dscuss resource allocatons at upper and lower levels of the proposed framework, respectvely. The heurstc resource allocaton algorthm s presented n Secton VII, whle smulaton results and the conclusons are gven n Sectons VIII and IX, respectvely. II. REATED WORK Exstng resource allocaton schemes can be classfed nto three categores: schemes usng a sngle network nterface of each UE at any gven tme [9] [11], schemes utlzng the mult-homng capablty of UEs [5], [12] [14], and schemes that are desgned based on dfferent PHY and MAC technologes [4], [15]. A load balancng scheme to mprove resource utlzaton n cellular/wan nterworkng s presented n [9]. New voce and data calls are assgned to a network based on a set of precalculated probabltes. Assgned calls are redstrbuted whenever necessary to another network by usng dynamc vertcal handoffs to reduce network congeston and mprove QoS satsfacton. To further mprove QoS satsfacton, the scheme proposed n [10] allocates voce calls preferably for the cellular network. The resource allocaton scheme proposed for WMAX/WAN nterworkng n [11] assgns all streamng calls to the WMAX network to guarantee QoS satsfacton; data calls that are served by the WMAX network are preempted to free up bandwdth for the ncomng streamng calls when requred. The man advantage of these schemes n the frst category s that they are easy to deploy as each network can use ts own/exstng resource allocaton scheme to allocate resources. Further, desgnng an effcent resource allocaton scheme s smpler for an ndvdual network than for an nterworkng system. When UEs are capable of mult-homng, restrctng a UE or a certan traffc type of a user to access only one network lmts the flexblty n dstrbutng resources of the nterworkng system among users. Thus, the resource allocaton schemes n the second category take advantage of the mult-homng capablty of UEs to effcently utlze resources of the nterworkng system. For computatonal smplcty, t s typcally assumed that the WAN uses a resource reservaton protocol to avod channel contenton collsons. Hence, resources of the WAN are modeled as frequency channels or tme slots. Bandwdth allocaton algorthms for UEs wth dfferent types of traffc requrements are presented n lterature. In [5], each network gves more prorty to satsfy ts own subscrbers QoS requrements, whle utlty farness among users n the nterworkng system s mantaned n [12]. A game theoretc approach for bandwdth allocaton and admsson control s used n [13]. Each network allocates ts bandwdth for dfferent servce areas on a long-term bass based on the statstcs of call arrvals; bandwdths for each servce area from dfferent networks are then allocated to users on a short-term bass. To ensure QoS satsfacton, a new call s accepted only f ts mnmum data rate requrement can be satsfed. Algorthms to allocate tme slots n a WAN and subcarrers n a cellular network subject to a proportonal rate constrant are presented n [14]. The thrd category ncludes the resource allocaton schemes proposed n [4], [15]. These schemes are based on PHY and MAC technologes of the dfferent networks to guarantee the feasblty of resource allocaton decsons. Specfcally, the effect of transmsson collsons caused by the contenton-based channel access n the WAN s consdered. In [4], resource allocaton and admsson control schemes are proposed for an nterworkng system consstng of a code dvson multple access CDMA) based cellular network and an IEEE dstrbuted coordnaton functon D) based WAN. Maxmzng total network welfare ensures QoS satsfacton n the system. In [15], nterworkng of an OFDMA based femtocell network and an IEEE D based WAN s consdered. Resources of both femtocell and WAN are allocated on the same tme-scale, and WAN uses basc access scheme wth two-way handshakng. The exstng resource allocaton schemes allocate resources of dfferent networks n the nterworkng system at the same tme-scale, and do not fully utlze the QoS support n WANs. Allocatng resources of dfferent networks at the same tmescale s not practcal as dfferent networks have dfferent resource allocaton ntervals. To facltate QoS n WANs, recent WAN standards offer contenton-based and contenton-free pollng based channel access mechansms. These two channel access mechansms and ther QoS capabltes should be consdered to maxmze the effcency of the nterworkng system. In addton, jontly allocatng transmt power levels for dfferent network nterfaces at mult-homng capable UEs s essental for an effcent resource utlzaton. Jont transmt power allocaton s studed n [15] wthout takng the user QoS requrements nto account. In ths work, we study the resource allocaton for cellular/ WAN nterworkng to satsfy the QoS requrements of multhomng UEs. Based on the PHY and MAC technologes of these two networks, the resources are allocated to multhomng UEs at two tme-scales: one tme-scale for allocatng resources of each network. We consder power allocaton for mult-homng UEs, and the two channel access mechansms of the WANs. III. SYSTEM MODE The nterworkng system under consderaton conssts of a sngle cell of a cellular network and a WAN wthn the coverage of the cell, as shown n Fg. 1. We focus on the resource allocaton for the uplnk. The cellular network s OFDMA based, and the set of subcarrers avalable at the base staton BS) s denoted by K C. At any tme, each subcarrer s allocated to only one user n order to avod co-channel nterference

3 THARAPERIYA GAMAGE et al.: TWO TIME-SCAE CROSS-AYER SCHEDUING FOR CEUAR/WAN INTERWORKING 2775 Fg. 1. Cellular/WAN nterworkng. among the users. The WAN supports both contenton-based and contenton-free pollng based channel access for data and voce servces. In the contenton-based channel access, fourway handshakng scheme wth request-to-send RTS) and clearto-send CTS) messages s used. The tme perods durng whch the two channel access mechansms operate are referred to as contenton perod CP) and contenton-free perod P), respectvely. The CP and P alternate over tme and they repeat once every. Resources of the cellular network and the WAN are allocated at two dfferent tme-scales. In the system, there are N users belongng to two groups: hgh-moblty users and low-moblty users. The set of all the users s denoted by S N. The set of low-moblty users wthn the WAN coverage s denoted by S M, whle the set of remanng users s denoted by S S. For example, n Fg. 1, UE 1 to UE 4 are n S M, whle UE 5 and UE 6 are n S S. Each user has voce and data traffc requrements. All the UEs are equpped wth WAN and cellular network nterfaces, and have the multhomng capablty. Users n S M are allowed to smultaneously communcate over cellular network and WAN, whle users n S S are only allowed to communcate over the cellular network. A. Two Tme-Scale Resource Allocaton Framework Resource allocaton ntervals of exstng cellular networks are shorter than those of the exstng WANs, as cellular networks and WANs are desgned to support hgh moblty and low moblty users, respectvely [2], [7]. Therefore, as shown n Fg. 2, resources n the cellular network are allocated at a faster tme-scale than n the WAN. The duraton of a tme slot n a tme-scale s the resource allocaton nterval of the correspondng network, denoted by T and T U n the fast and slow tme-scales T <T U ) for the cellular network and the WAN, respectvely. The resource allocaton processes at fast and slow tme-scales are referred to as lower and upper levels of the resource allocaton process, respectvely. As the WAN resource allocaton nterval s relatvely long, to satsfy the strct delay and jtter requrements of perodcally arrvng constant bt rate voce traffc, several short Ps are used wthn a resource allocaton nterval of the WAN nstead of usng a long P [16]. For smplcty, assume V = T U /T ) s an nteger and the boundares of the frst tme slots n the two tme-scales are algned. B. Symbols and Notatons The lth l {0, 1,...,V 1}) fast tme-scale tme slot wthn the uth slow tme-scale tme slot s referred to as u, l)th tme slot. Commonly used symbols are wrtten n the form of X,y n or Xn,y ), where superscrpt n, n {C, W,,,, U}, represents the network or the level of resource allocaton process. Superscrpts C, W, and denote the cellular network, WAN, contenton-based channel access and contenton-free channel access respectvely, whle and U denote lower and upper levels of the resource allocaton process respectvely. The subscrpts denote the user, a partcular resource of network n, and a tme slot. When n {W, }, only one subscrpt s used representng the user. Boldface letters are used for vectors and matrces, and vector X s represented as X = {X 1,...,X X } wth X beng the number of elements n X. The optmum value of varable X s denoted by X.The actve or the determned) decson polcy X and the optmal set X are denoted by X and X, respectvely. Table I summarzes the mportant symbols. C. WAN In the WAN, TXOPs for the UEs are granted usng two channel access mechansms: contenton-based channel access durng a CP and contenton-free pollng based channel access durng a P. In the former, UEs contend for the channel to obtan TXOPs, and each of these TXOPs allows transmttng a data packet of D bts. In the latter, UEs are granted TXOPs usng a centralzed pollng mechansm, and each of these TXOPs s defned as a fxed duraton of T whch allows a UE to transmt. The set of avalable contenton-free TXOPs durng a P s denoted by K. To avod co-channel nterference among the UEs, each contenton-free TXOP s allocated for only one UE at any gven tme. Contenton-based channel access s more sutable for varable bt rate data traffc, whle contenton-free channel access s more sutable for constant bt rate voce traffc [17]. In ths work, to optmze resource utlzaton subject to QoS requrements, voce traffc s served by contenton-free channel access, and data traffc s served by both channel access mechansms. The sets of users communcate usng contenton-based and contenton-free channel access are denoted by S and S respectvely, where S, S S M and possbly S S. D. Traffc Model The traffc generated by each user can be dvded nto two classes: constant bt rate voce and delay tolerant data. Every user always has at least one packet n the data traffc queue to transmt. The mnmum data rates of voce and data traffc classes requred by the th user are denoted by R V mn, and R Dmn,, respectvely. As voce traffc flows are hghly susceptble to delay and jtter, voce traffc requrements are satsfed n average sense over each tme slot at the slow tme-scale. The data traffc requrements are satsfed n average sense over an nfnte tme horzon due to ther delay tolerance.

4 2776 IEEE TRANSACTIONS ON COMMUNICATIONS, VO. 62, NO. 8, AUGUST 2014 Fg. 2. Resource allocaton at slow and fast tme-scales. TABE I SUMMARY OF IMPORTANT SYMBOS E. Channel Model We model the wreless channels as a fnte-state Markov process to capture the channel tme-correlaton [18]. The channel gan s tme nvarant.e., quas-statc fadng) wthn each coherence tme T coh ) nterval. The dfferent wreless channels vary ndependently from each other. The channel gan doman s parttoned nto K S non-overlappng states. The transton probabltes between dfferent states of a Raylegh fadng channel can be calculated as n [18], assumng that T U and T are not longer than the correspondng channel coherence tmes to ensure the states do not change wthn a tme slot. F. Throughputs Per User 1) Va Cellular Network and Contenton-Free Channel Access: The maxmum achevable error free data rate by the th user usng network n {C, } can be expressed by R,y n ) P n,y = ρ n,yb log 2 1+α n,y P n ),y, 1) y K n where α,y n s the sgnal-to-nterference plus nose rato SINR) of the channel between cellular BS/WAN access pont AP) and the th user over the yth resource of n,.e., the yth OFDM subcarrer or TXOP, wth unt transmt power; P,y n s the

5 THARAPERIYA GAMAGE et al.: TWO TIME-SCAE CROSS-AYER SCHEDUING FOR CEUAR/WAN INTERWORKING 2777 transmt power level of the th user over the yth resource of n; B represents the bandwdth of WAN B W ) or bandwdth of an OFDM subcarrer Δf = B C / K C ); B C s the system bandwdth of the cell; and ρ n,y =1f the th user s allocated the yth resource of n, and ρ n,y =0 otherwse. Furthermore, ρ,y =0, y f S M. As each resource s allocated to only one user to avod co-channel nterference, In addton, α,y S N ρ n,y 1, y K n. 2) = α = α W, y over each channel coherence tme nterval n the WAN. 2) Va Contenton-Based Channel Access: The average throughput acheved by the th user durng a CP n the WAN wth four-way handshakng scheme s gven by [2], [19] R wth τ1 τ) N W 1 D )= T 0 +N W τ1 τ) N W 1 j S, S 3) j T 0 = N W τ1 τ) N W 1 T CTS + T ACK +3T SIFS ) +1 1 τ) N W )T RT S + T AIF S )+1 τ) N W σ, where s the duraton of a packet transmsson by the th user; N W s the number of users n S ; and T SIFS, T AIF S, T ACK, T RT S, T CTS and σ are the duratons of short nterframe space, arbtraton nterframe space, acknowledgment, RTS message, CTS message and an empty slot, respectvely. The probablty τ of a user transmttng a packet n a randomly chosen ) gven by 3) cannot be drectly used n a resource allocaton problem as t s not wrtten as a functon of the transmt power levels. Therefore, we rewrte n terms of the throughput that s acheved by the th user durng a successful transmsson, and tme slot can be calculated as n [19]. The throughput R substtute t nto 3) [15]. Then, R R P )= T 0 + DN W τ1 τ) N W 1 B W ) can be rewrtten as τ1 τ) N W 1 D, 1 log j S 2 1+Pj α W j ) where P s the transmt power level of the th UE durng a CP over the WAN nterface. Note that R P ) s the same for all the users n S and s a concave functon when N W s fxed see Appendx A). G. Power Usage of Mult-Homng Devces The operatng tme of a UE s governed by the energy or average power) consumpton of the uplnk communcatons through WAN and cellular nterfaces of the UE [5]. Therefore, we lmt the total average power consumpton of each UE over each tme slot n the slow tme-scale to a predefned maxmum. We frst calculate the average power usage through the WAN nterface for contenton-based channel access, and then formulate the constrant on the total average power consumpton. Wth a successful transmsson probablty of τ1 τ) N W 1 durng a CP, the average power consumpton of the th UE through the WAN nterface s Pavg, P )= 4) τ1 τ) N W 1 P /T avg )T CP / ), where T avg s the average duraton of channel occupancy for a successful packet transmsson ncludng collson perod and empty slot n whch no UE transmts [19]. After some smplfcaton, Pavg, P ) can be expressed as P avg,p )= { TCP P R P ) B W log 2 1+α W P ), f P >0; 0, otherwse. The constrant on the total average power consumpton of each UE over the uth tme slot can then be expressed as Pavg,+P C avg,p )+ T ρ,j P,j P T,, S N, j K 6) where Pavg, C s the average power usage through the cellular nterface durng the tme slot and P T, s the total average power avalable for the th user. IV. MMDP-BASED OPTIMA RESOURCE AOCATION The objectve of resource allocaton s to maxmze the total throughput of the nterworkng system subject to the satsfacton of QoS requrements. As dscussed n Secton III-A, the resource allocaton process conssts of two upper and lower) levels operatng at slow and fast tme-scales respectvely, based on the channel state nformaton. Resources of the WAN and the cellular network are allocated at the begnnngs of the uth and the u, l)th tme slots respectvely, where u = {0, 1, 2,...} and l = {0,...,V 1}. The set of channel gans of the channels between users n S M and the WAN AP at the begnnng of the uth tme slot s referred to as the state of the upperlevel durng the uth tme slot ψu U ). The set of channel gans of the channels between all the users and the cellular BS at the begnnng of the u, l)th tme slot s referred to as the state of the lower-level durng the u, l)th tme slot ψu,l ). Whle the system state {ψu U,ψu,l } s denoted by ψ u,l, the sets of all the possble states of upper and lower levels are denoted by Ψ U and Ψ, respectvely. An overvew of the resource allocaton framework s shown n Fg. 3. The optmal resource allocaton problem for cellular/ WAN nterworkng s formulated as an MMDP [8] for three reasons: 1) the resource allocaton process operates at two tmescales as explaned n Secton III-A, 2) state transton of each level s a Markov process due to the Markov channel model, and 3) resource allocatons at multple tme slots are jontly optmzed to satsfy the user QoS requrements over multple tme slots see Secton III-D). The MMDP formulaton conssts of upper and lower level resource allocaton polces [8]. As shown n Fg. 3, the decsons of the upper-level are made consderng the throughputs acheved through and the power consumed at the lower-level. Therefore, the upper-level polcy D U ) maps system state ψ u,0 to a set of resource allocaton decsons A U u ) at the begnnng of uth tme slot, u={0, 1, 2,...}. The lower-level polcy D ) maps state ψu,l to a set of resource allocaton decsons A u,l ) at the begnnng of u, l)th tme slot, l ={0,...,V 1}. Decsons n A U u and A u,l are {P,P,j,ρ,j S M,j 5)

6 2778 IEEE TRANSACTIONS ON COMMUNICATIONS, VO. 62, NO. 8, AUGUST 2014 The data traffc requrements of the users are served through both networks whle the voce traffc requrements are served through the contenton-free channel access and the cellular network. Therefore, the QoS constrants see Secton III-D), whch ensure data and voce traffc requrement satsfacton over an nfnte tme horzon and over the uth tme slot u = {0, 1, 2,...}) respectvely, can be stated as R U ψ 0,0, D) R V mn, + R Dmn,, S N 11) R,u ψ u,0,a U u, D ) + T ρ,j R ),j P,j j K R V mn,, S N. 12) Fg. 3. Overvew of the MMDP-based two tme-scale resource allocaton framework. K } and {P,k C,ρC,k S N,k K C }, respectvely. For notaton smplcty, we use D to denote the system polcy {D U, D }. We use the summaton of dscounted throughputs SDTs) [20], [21] over an nfnte tme horzon as a reward objectve) functon. The SDT based reward functon reduces the susceptblty of the determned decson polces to the unpredctable channel changes n the future by gvng less mportance to those decsons made and rewards acheved) at far future. The SDTs acheved by the th user at the upper-level over an nfnte tme horzon wth the ntal state of ψ 0,0 and at the lowerlevel durng the uth tme slot wth the ntal state of ψu,0 are denoted by R Uψ 0,0, D) and R,u ψ u,0,a U u, D ), respectvely [8]. As the decson polces are statonary to be dscussed), R Uψ 0,0, D) and R,u ψ u,0,a U u, D ) can be nterpreted as the average throughputs that are acheved by the th user over the same perods of tme at the upper and lower levels, respectvely [21]. They are gven by [20], [21] VU R U ψ 0,0, D)= lm 1 θ) 1 θ u r,uψ U u,0,a U u, D ) 7) V U u=0 V 1 R,u ψ u,0,a U u, D ) =1 β) β l r,u,l ψ u,l,a U u,a u,l), 8) l=0 where θ 0, 1) and β 0, 1) are dscount factors; and r,u U ψ u,0,a U u, D ) and r,u,l ψ u,l,au u,a u,l ), whch denote the throughputs acheved by the th user at the upper and lower levels durng the uth and u, l)th tme slots respectvely, are gven by [22] r,u U ψu,0,a U u, D ) R,u ψ u,0,a U u, D ), f S S ; R,u ψ u,0,a U u, D ) = + T ) j K ρ,j R,j P,j, f SM \S ; R,u ψ u,0,a U u, D ) 9) + T ) j K ρ,j R,j P,j + T CP R P ), f S ; and r,u,l ψ u,l,a U u,a ) ) u,l = P C,k. 10) k K C ρ C,kR C,k As the sum of dscounted costs provdes a good approxmaton for the average cost when the polces are statonary [21], Pavg, C over the uth tme slot can be calculated by V 1 Pavg, C =1 β) β l Ptot,,l C ψ u,l,a U u,a u,l), 13) l=0 where Ptot,,l C ψ u,l,au u,a u,l ) s the total power allocated by the th user to communcate over the cellular network durng the u, l)th tme slot, and s also equvalent to k K C ρc,k P,k C over the u, l)th tme slot. The MMDP based optmal resource allocaton problem can then be stated as [8], [21] P1 : max max R D U D U ψ 0,0, D) S N s.t. 2) for n {C, }, 6), 11) and 12). To fnd the optmal D U and D solvng problem P1, resource allocaton should be optmzed over three dfferent tme ntervals: 1) resource allocaton over an nfnte tme horzon s optmzed to satsfy 11), 2) resource allocaton over each upper-level tme slot s optmzed to optmally use upper-level resources whle satsfyng 6) and 12), and 3) resource allocaton over each lower-level tme slot s optmzed to optmally use lower-level resources. Therefore, problem P1 s solved n three stages, where the frst, second and thrd stages allocate resources over an nfnte tme horzon, for each upper-level tme slot, and for each lower-level tme slot, respectvely. The resource allocaton problem for the mth stage m = {2, 3}) s derved by decomposng the m 1)th stage problem nto a set of problems, each of whch allocates resources over the resource allocaton nterval of the mth stage, and by mposng constrants that must be satsfed wthn the resource allocaton nterval of the mth stage. The optmalty of the soluton, whch s obtaned usng the three stage approach, for problem P1 s ensured by teratng the mth stage m = {1, 2}) soluton untl t reaches the optmal whle calculatng the optmum m +1)th stage soluton for each mth stage teraton. Durng the teraton process, the dual varables of the mth stage are passed to the m +1)th stage whle the throughputs/sdts acheved and power consumed at the m +1)th stage are feedback to the mth stage. At the m +1)th stage, the receved dual varables are used for

7 THARAPERIYA GAMAGE et al.: TWO TIME-SCAE CROSS-AYER SCHEDUING FOR CEUAR/WAN INTERWORKING 2779 confgurng the objectve functon of the resource allocaton problem such that the m +1)th stage asssts maxmzng the mth stage objectve. At the mth stage, the receved nformaton s used for updatng the dual varables. Problem P1 s a non-convex problem. Therefore, we determne the polces by relaxng problem P1 to reduce the computatonal complexty. Due to the relaxatons, the polces determned n ths work D U and D ) are not optmal for problem P1 n certan scenaros. Therefore, we refer to D U and D as actve or determned) upper and lower level decson polces, respectvely. Dervatons of D U, whch s found by solvng the frst and second stage resource allocaton problems, and D, whch s found by solvng the thrd stage resource allocaton problem, are dscussed n Sectons V and VI, respectvely. Usng the state transton probabltes calculated based on the channel statstcs, D U and D can be determned n advance and appled to the system based on the ntal states. The appled polces select A U u and A u,l for the uth and the u, l)th tme slots respectvely, based on the states of the two levels durng the tme slots. The polces D U and D are requred to be recalculated when the channel statstcs or the number of users n the system or ther QoS requrements change. V. U PPER-EVE RESOURCE AOCATION To determne D U, frst we maxmze the total SDT at the upper-level subject to satsfacton of 11) over an nfnte tme horzon wth the ntal system state of ψ 0,0 ; ths frst stage problem s denoted by P2. By further nvestgatng problem P2, we fnd out that problem P2 s a convex optmzaton problem and can be solved by solvng the dual problem [23]. To fnd the dual functon, the mnmum of the agrangan s determned by decomposng the agrangan nto a set of terms, each of whch s a negatve summaton of weghted throughputs for the uth tme slot u = {0, 1, 2,...}) s determned such that t maxmzes the summaton of weghted throughputs correspondng to the uth tme slot subject to satsfacton of 2) for n =, 6) and 12); ths second stage problem s denoted by P3. Frst and second stage problems are solved n Secton V-A and B, respectvely. In addton, the condtons whch the thrd stage resource allocaton at the lower-level should satsfy to ensure the optmalty of the three stage soluton for problem P1 are derved n Secton V-B. of the users correspondng to one tme slot. Then, A U u A. Frst Stage Resource Allocaton Frst stage resource allocaton problem can be stated as P2 : max D U ψ0,0, D U, D ) S N R U s.t. C1 : 11). The actve polcy D s used n problem P2 as for each teraton of the algorthm whch solves problem P2, D s calculated by solvng the thrd stage resource allocaton problem. From 7) 10), the objectve functon of problem P2 s a concave functon, and the feasble regon s a convex set. Therefore, problem P2 s a convex optmzaton problem, and s solved by maxmzng the dual functon whch s obtaned by mnmzng the agrangan of problem P2 wth respect to D U [23]. The agrangan of problem P2 s U ψ 0,0, λ, D U, D ) = [ λ R V mn, + R Dmn, ) S N ] 1 + λ )R U ψ 0,0, D U, D ), 14) where λ, are dual varables. The teratve algorthm whch solves P2 can be summarzed as follows. Frst, λ s ntalzed e.g., λ {0,...,0}). Second, we fnd D U whch mnmzes U ψ 0,0, λ, D U, D ) for the λ. To update λ for the next teraton, R U ψ 0,0, D U, D ), are also found n ths step. Thrd, λ s adjusted toward λ usng the gradent descent method [5], [24], [25]. The second and the thrd steps are repeated untl λ reaches λ. When λ reaches λ, each λ satsfes the complementary slackness condton [23] and we have found D U. To mplement the above algorthm, D U and R U ψ 0,0, D U, D ), for any λ can be calculated as follows. From 14) and snce S N λ R V mn, + R Dmn, ) does not depend on D U, D U s determned such that t maxmzes S N 1 + λ ) R U ψ 0,0, D U, D ). When S N 1 + λ )R U ψ 0,0, D U, D ) s maxmzed, by 7) and usng the Bellman optmalty equaton [21], t s gven by the followng optmalty equaton. U supψ 0,0, λ) [ =1 θ)max 1 + λ )r U A U,0 ψ0,0,a U 0, D )] 0 S N + θ P ψ0,0 1,0 U supψ 1,0, λ) 15) ψ wth P U ψ ψ1 U ΨU ψ1,0 0 U ψu 1 Ψ U supψ u,0, λ) =sup D U [ ] 1 + λ )R U ψ u,0, D U, D ), S N where P U and P are the probabltes of the upper ψ0 U ψu 1 ψ0,0 ψ 1,0 and lower level states change from ψ0 U to ψ1 U and from ψ0,0 to ψ1,0 at the end of the 0th tme slot, respectvely. Equaton 15) s a recursve formula, and A U u for the uth tme slot s determned such that the summaton of weghted throughputs gven by S N 1 + λ )r,u U ψ u,0,a U u, D ) s maxmzed [20]. Furthermore, these resource allocaton problems correspondng to dfferent tme slots are ndependent of each other. As D U s a statonary polcy to be dscussed n Secton V-B), fndng A U 0 for each ψ 0,0 {Ψ U, Ψ } at the 0th tme slot s suffcent to fnd D U.Next,R Uψ 0,0, D U, D ) can be determned by solvng the Bellman optmalty equaton for the th user wrtten usng 7). Bellman optmalty equaton solvng methods, such as Value Iteraton algorthm and ts varants, are explaned n [20].

8 2780 IEEE TRANSACTIONS ON COMMUNICATIONS, VO. 62, NO. 8, AUGUST 2014 B. Second Stage Resource Allocaton We derve Au U such that t maxmzes S N 1 + λ )r,u U ψ u,0,a U u, D ) at system state ψ u,0 subject to satsfacton of 2) for n =, 6) and 12) durng the uth tme slot. Ths optmzaton problem s a non-convex problem due to the nteger constrant whch s mposed on ρ,j see Secton III-F1). Therefore, to reduce the computatonal complexty requred to solve the problem, we relax the problem to be a convex optmzaton problem by relaxng the nteger constrant such that [0, 1]. To calculate power usage and throughputs over partally allocated TXOPs, we defne P,j = ρ,j P,j and R,j P,j,ρ,j )=T / )ρ,j R,j P,j /ρ,j ), where R,j P,j,ρ,j ) s a concave functon [22]. For notaton smplcty, we defne R P )=T CP / )R P ). Then, substtutng from 9) to r,u U ψ u,0,a U u, D ),therelaxed problem can be stated as ρ,j P3 :max 1 + λ ) A U u S N ψ u,0,a U u, D ) + s.t. R,u + S 1 + λ ) C2 : S M ρ j K R P ),j 1, j K C3 : R,u ψ u,0,a U u, D ) + R V mn,, S N C4 : P avg,p )+P C avg, + T j K j K R,j R,j P,j,ρ P,j P,j P T,, S N C5 : 0 ρ,j 1, S M,j K C6 : P,j,j ) 0,P 0, S N,j K.,ρ ),j Problem P3 s a convex optmzaton problem. Convexty of C4,.e., {Pavg, C, P,j,P C4 s satsfed, S N,j K }, s proved n Appendx B. Next, we llustrate the relatonshp between problem P3 and the thrd stage resource allocaton whch determnes D for the lower-level. Then, we derve A U u by solvng problem P3 usng Karush-Kuhn-Tucker KKT) condtons [23]. The agrangan of problem P3 s shown n 16) see equaton at the bottom of the page) where γ j, ξ and μ,, j are the dual varables assocated wth C2, C3 and C4, respectvely. As the optmal soluton for problem P3 mnmzes 16) subject to C5 and C6, D s determned such that t maxmzes S N 1 + λ + ξ )R,u ψ u,0,a U u, D ). Further, D should satsfy the followng KKT condton of P3 to ensure the optmalty of the soluton, whch s obtaned usng the three stages, for problem P λ + ξ ) R,u ψ u,0,a U u, D ) P C avg, { = μ, f P C avg, > 0; <μ, otherwse; P C avg, =P C avg, S N. 17) Dual varables μ and ξ couple the upper and the lower level resource allocatons to optmally dstrbute the transmt power avalable at the UEs among WAN and cellular network nterfaces and to optmally utlze the resources of the two networks to satsfy the users voce traffc requrements, respectvely. Due to ths couplng, once resources of the lower-level are allocated, acheved SDTs.e., R,u ψ u,0,a U u, D ), ) and the average power consumptons at the lower-level.e., Pavg, C, ) are feedback to the upper-level to solve problem P3, asshown n Fg. 3. Soluton for problem P3 s found as follows. Frst, ξ and μ are ntalzed. Second, the optmal allocatons of contentonfree TXOPs, UE transmt power levels durng contentonfree and contenton-based TXOPs, R,u ψ u,0,a U u, D ), and P C avg,, are calculated based on ξ and μ. Thrd,μ s updated toward μ usng the gradent descent method [5], [24], [25]. The second and thrd steps are repeated untl μ s found. Forth, ξ s updated toward ξ usng the gradent descent method. The last three steps are repeated untl ξ s found. In the followng, we derve the decson set A U u = {P, P,j,ρ,j S M,j K }) by solvng problem P3, and explan how S s determned. In addton, the optmalty of A U u for the ntal problem.e., the problem pror to the relaxaton) s also dscussed. U2) A U u, γ, ξ, μ ) = 1 + λ + ξ ) R,uψ u,0,a U u, D )+ S N j K R,j P,j,ρ,j ) S 1 + λ ) R P ) + γ j ρ,j + ξ R V mn, + μ Pavg,P )+Pavg, C + T S N j K j K P,j P T, γ j 16) j K

9 THARAPERIYA GAMAGE et al.: TWO TIME-SCAE CROSS-AYER SCHEDUING FOR CEUAR/WAN INTERWORKING ) Allocatons of Contenton-Free TXOPs and Transmt Power evels: Based on the KKT condtons for P3, the optmal transmt power levels of the users durng contenton-free TXOPs are gven by where P,j = ρ,j Θ, S M,j K, 18) [ B W Θ = ln2) 1 + λ + ξ ) μ 1 α W and [x] + =max{0,x}. Next, the optmal contenton-free TXOP allocaton can be determned as follows. et Γ,j =1+λ + ξ ) R,j = 1 + λ + ξ ) T B W P,j ρ,j [ log α W Θ ) 1 ln2),ρ,j ) α W ] + P,j Θ 1+α W Θ = P,j ], S M,j K. 19) Due to the fact that Γ,j s ndependent of ρ,j and from the KKT condtons, the jth TXOP s allocated to the user wth the largest Γ,j [22]. However, when there are multple users wth ther Γ,j values equal to the largest Γ,j for the jth TXOP, the optmal soluton for the problem P3 allocates fractons of the TXOP among these users allowng them to tme-share the TXOP. Snce the channel gan or α W ) and Θ reman unchanged over the uth tme slot, we can see from 19) that Γ,j of the th user s the same for all the TXOPs. Consequently, the th user s allocated the same fracton from each TXOP or s allocated all the TXOPs. When there are N users { 1, 2,..., N } wth ther Γ,j values equal to the largest, the optmal fractonal values for ρ,j,={ 1, 2,..., N } are determned based on the prmal feasblty of those ρ,j s wth respect to C2, C3 and C4. That s, the optmal set of ρ,j,={ 1, 2,..., N } s a soluton whch satsfes C2 wth equalty and the followng set of lnear nequaltes ρ,j K T ) B W log 2 1+α W Θ R V mn, R,u ψ u,0,a U u, D ),={ 1, 2,..., N } 20) and ρ,j K T Θ P T, P avg,p ) P C avg,, ={ 1, 2,..., N }. 21) As the objectve of ths work s to allocate resources based on the PHY and the MAC technologes of the networks, a near optmal TXOP allocaton for the ntal problem s found by roundng ρ,j K values to the nearest ntegers. The rounded values ndcate the number of TXOPs allocated to each user. Moreover, f ρ,j K, are ntegers, they are the optmal TXOP allocaton for the ntal problem. 2) Allocatons of Users and Transmt Power evels for Contenton-Based Channel Access: The second stage resource allocaton problem should be formulated as a convex optmzaton problem to reduce the requred computatonal capacty. However, R P ) gven by 4) s a non-concave functon when N W vares. Therefore, to formulate the second stage problem as a convex optmzaton problem, S should be determned pror to allocatng the other upper-level resources. In the MMDP based resource allocaton algorthm, S whch acheves the hghest total SDT at the upper-level s found va searchng over S M. A low complexty method to fnd a near optmal S s presented n Secton VII. From 4), t can be seen that R P ) depends not only on the th user s transmt power level, but also on the transmt power levels of the other users n S. Thus, P, S are correlated. Based on the KKT condtons for problem P3, P > 0 only f R P ) P P =P P =0 > μ 1+λ P avg, P ) P P =P P =0, 22) where P s a vector whch conssts of the power levels of the users n S except the th user. Snce these partal dervatves are not defned when P =0, we rewrte 22) by takng the lmts of the partal dervatves as P 0. Then, 22) reduces to { P > 0, f BW α W =0, otherwse; ln2) > μ 1+λ ; S. 23) Moreover, for P > 0, the two sdes of 22) become equal when the partal dervatves are evaluated at P = P. Therefore, value of P when P > 0 can be found by solvng 24) usng Newton s method f P s known [26]. 1+λ μ = T CP NW α W [ R P ) 1+P α W ln 1+P α W ln2)n W P + 1+P B W α W α W ) P 1+P R α W α W P ) ) ln 1+P Exstence of a soluton for 24) s shown n Appendx C. α W ] ). 24)

10 2782 IEEE TRANSACTIONS ON COMMUNICATIONS, VO. 62, NO. 8, AUGUST 2014 As P, S are correlated, P s found usng an teratve algorthm. In each teraton, P, S are calculated by 24) usng the P calculated n the prevous teraton. The algorthm termnates when the changes to P, S are neglgble. Convergence of ths teratve algorthm s proved n Appendx D. From 18) 24), t can be seen that A U u s ndependent of the tme slot.e., u). Therefore, A U u whch s determned for the uth tme slot can be used at the u th tme slot u = {0, 1, 2,...}) when states durng the uth and the u th tme slots are equvalent.e., ψ u,0 = ψ u,0). Thus, D U s a statonary polcy [20]. The algorthm whch determnes D U for a gven ntal state ψ 0,0 s shown n Algorthm 1. VI. OWER-EVE RESOURCE AOCATION The objectve of the lower-level resource allocaton s to maxmze S N 1 + λ + ξ )R,u ψ u,0,a U u, D ) subject to 2) for n = C and 17) over the uth tme slot see Secton V-B). To determne D, frst we decompose the resource allocaton problem over the uth tme slot to a set of ndependent subproblems, each of whch allocates resources for one lower-level tme slot. Second, A u,l for the u, l)th tme slot, l = {0, 1,...,V 1}, s found by solvng the subproblem whch corresponds to the same tme slot; ths thrd stage resource allocaton problem s denoted by P4 see Secton IV). Based on A u,l found for the u, l)th tme slot, we show that D s a statonary polcy. To decompose the resource allocaton problem whch maxmzes S N 1 + λ + ξ )R,u ψ u,0,a U u, D ) over the uth tme slot to a set of ndependent subproblems, the Bellman optmalty equaton for the lower-level s wrtten usng 8) wth the assumpton of V s very large, whch s reasonable snce T T U, as follows [2], [7]. 1 + λ + ξ )R,u ψ u,0,a U u, D ) S N [ =1 β)max 1 + λ + ξ )r A,u,0 ψ u,0,a U u,a ) ] u,0 u,0 S N + β P 2) ψ ψu,1 u,0 ψ u,1 Ψ 1 + λ + ξ )R,u ψ u,1,a U u, D ), 25) S N where P 2) s the probablty of lower-level state changes ψu,0 ψ u,1 from ψu,0 to ψu,1 at the end of the u, 0)th tme slot. As the left hand sde of 25) s maxmzed when A u,l for the u, l)th tme slot maxmzes S N 1 + λ + ξ )r,u,l ψ u,l,au u,a u,l ),resources of the lower-level.e., subcarrers and transmt power levels) are allocated for the u, l)th tme slot such that S N 1 + λ + ξ )r,u,l ψ u,l,au u,a u,l ) s maxmzed [20]. It should be noted that these resource allocaton subproblems correspondng to the lower-level tme slots are ndependent of each other. Smlar to the non-convexty caused by the nteger constrant whch s mposed on ρ,j see Secton V-B), the nteger constrant whch s mposed on ρ C,k makes the subproblem correspondng to the u, l)th tme slot non-convex see Secton III-F1). To reduce the computatonal capacty requred to solve the subproblem, we relax t by followng the same relaxaton process whch s used n Secton V-B. That s, we let ρ C,k [0, 1] and defne P,k C = ρc,k P,k C and R,k C P,k C,ρC,k )= ρ C,k RC,k P,k C /ρc,k ). The relaxed subproblem s consdered as the problem P4. Snce problem P4 s solved subject to satsfacton of 17) over the uth tme slot, 17) s frst translated nto a set of constrants, each corresponds to one lower-level tme slot, by substtutng 8), 10) and 13) nto 17). Then, the constrant correspondng to the u, l)th tme slot s gven by R ),k C P C,k,ρ C,k 1 + λ + ξ ) P,k C P C = P C {,k,k = C μ, f P,k > 0; S N,k K C. 26) <μ, otherwse; From 26), [ ] + P,k C = ρ C Δf 1 + λ + ξ ),k 1 ln2) μ α,k C. 27) Next, the remanng subcarrer allocaton problem for the u, l)th tme slot can be stated as P5 : max 1 + λ + ξ ) R C ρ C,k P C,k,ρ C,k) S N k K C s.t. C7 : ρ C,k 1, k K C S N C8 : 0 ρ C,k 1, S N,k K C.

11 THARAPERIYA GAMAGE et al.: TWO TIME-SCAE CROSS-AYER SCHEDUING FOR CEUAR/WAN INTERWORKING 2783 Problem P5 s a convex optmzaton problem. Therefore, from 27) and the KKT condtons for problem P5, and usng the same approach used for dervng ρ,j n Secton V-B1, the optmal subcarrer allocaton can be expressed as [22] { ρ C 1, f,k = =argmax {Λ,k }; S 0, otherwse; N,k K C, 28) where [ Λ,k =1+λ +ξ ) log 2 1+α C,k P,k C ) 1 α,k C P ],k C ln2) 1+α,k C P,k C. However, when there are multple users wth ther Λ,k values equvalent to the largest Λ,k for the kth subcarrer, the optmal soluton for the problem P5 requres allocaton of fractons, whch satsfes C7 wth equalty, of the kth subcarrer among these users. When such equalty of Λ,k occurs, we randomly allocate the kth subcarrer to one of the users wth the largest Λ,k due to the fact that fractonal subcarrer allocatons are not supported by the PHY. Random subcarrer allocaton n ths scenaro does not sgnfcantly devate the system throughput/qos performance from the optmum due to two reasons: 1) subcarrer bandwdth Δf) s small as there s a large number of subcarrers; 2) probablty of multple users havng equvalent Λ,k values for more than one subcarrer or for a certan subcarrer over multple tme slots s very small, because the channel gans over dfferent subcarrers are dfferent and vares over tme slots. From 27) and 28), t can be seen that A u,l s ndependent of the tme slot. Thus, A u,l whch s determned for the u, l)th tme slot can be used at the u, l )th tme slot l = {0,...,V 1}) when states durng these two tme slots are equvalent.e., ψu,l = ψ u,l ). Therefore, D s a statonary polcy [20]. As D s a statonary polcy, calculatng A u,0 for each state ψu,0 Ψ at the u, 0)th tme slot s suffcent to determne D.Next,R,u ψ u,0,a U u, D ) and Pavg, C, can be found by solvng 8) and 13), respectvely. Equatons 8) and 13) can be solved usng the methods explaned n [20] by wrtng them as Bellman optmalty equatons. Values of R,u ψ u,0,a U u, D ) and Pavg, C, are then feedback to the upper-level to update λ, ξ and μ as shown n Algorthm 2, whch determnes D. The MMDP based resource allocaton algorthm.e., D U and D ) effcently allocates resources of the nterworkng system. However, t has a hgh tme complexty as t requres to fnd A U 0 and A u,0 for each system state, where the total number of system states n a system model s gven by K S ) N KC + S M. Therefore, we propose a heurstc resource allocaton algorthm wth low tme complexty when the number of system states s sgnfcantly hgher due to the large number of users and/or OFDM subcarrers. VII. HEURISTIC RESOURCE AOCATION The heurstc algorthm conssts of two steps. The frst step s executed only once at the begnnng, and t calculates the dual varables whch correspond to data and voce traffc constrants.e., λ and ξ) based on the average square channel gans Ω s), where Ω=E{h 2 }, E{ } s the ensemble average operator and h s the channel gan. The second step uses the dual varables calculated n the frst step, and allocates upper and lower level resources based on the nstantaneous channel gans subject to total power constrants of the users.e., C4). Snce these two steps are executed based on a sngle system state whch conssts of ether Ω s or nstantaneous channel gans, solvng the Bellman optmalty equatons s not requred n the heurstc algorthm for calculatng R Uψ 0,0, D), R,u ψ u,0,a U u, D ) and P C avg,. In the MMDP based algorthm, the Bellman optmalty equatons are solved by determnng A U 0 and A u,0 for each possble system state. In addton, S s determned usng a smple method n the heurstc algorthm to be dscussed). Due to these two reasons, the heurstc algorthm has low tme complexty. In the frst step, λ and ξ are found by solvng problems P2, P3 and P4. Therefore, the solutons whch we obtaned for problems P3 and P4 n Sectons V-B and VI are used n ths step wth modfcatons to utlze Ω s as follows. Average throughput over a Raylegh fadng channel s gven by [18], [28] E{R} = 0 ) 2Bh Ω log 2 1+ h2 p e h2 Ω dh n = B ) ln2) e n Ωp n E1, 29) Ωp where p s the transmt power level, B s the bandwdth, n s the total nose plus nterference power, and E 1 x) = x e x x 1 dx. Snce 0.5e x ln1 + 2x) provdes a tght lower bound for E 1 x) [28], by 29) E{R} > B 2 log Ωp ). 30) n Thus, the solutons for the problems P3 and P4 are modfed to calculate the throughput over each wreless channel by B/2) log 2 1+2Ωp/n)). That s, the equatons n Sectons V-B and VI are modfed wth the substtutons of B/2 to B, and

12 2784 IEEE TRANSACTIONS ON COMMUNICATIONS, VO. 62, NO. 8, AUGUST Ω to h 2. The latter s also equvalent to the substtuton of 2E{α,y n } to αn,y. Moreover, as Ω s are used n ths step, the number of possble system states n the MMDP reduces to one. Consequently, R Uψ 0,0, D) =r,0 U ψ 0,0,A U 0, D ), R,0 ψ 0,0,A U 0, D )=r,0,0 ψ 0,0,A U 0,A 0,0) and P avg, C = k K C ρc,k P,k C. Furthermore, log x) > 1/2) log x), x R +. Therefore, λ and ξ calculated n the frst step wll satsfy the QoS requrements wth an addtonal margn f the throughputs are gven by B log Ωp/n)). Therefore, when the resources are allocated n the second step, QoS requrements of the users are satsfed wth a hgher satsfacton though the nstantaneous channel gans are used n ths step. Frst step of the heurstc algorthm s shown n Algorthm 3. The optmal S conssts of only a few users wth strong channel condtons due to two reasons: 1) allocaton of too many users for contenton-based channel access degrades the aggregated throughput of the users due to ncreased number of collsons [2], and 2) allocaton of a user wth a weaker channel degrades the throughputs of all the users as the weak user takes a longer tme to transmt a packet [27]. Based on these characterstcs, users for the contenton-based channel access are allocated n the frst step of the heurstc algorthm as follows. Frst, the users n S M are sorted n the descendng order of ther E{α W }. Next, the frst step of the heurstc algorthm s repeated, each tme addng the next user n S M to S, untl the total throughput acheved at the upper-level reaches the maxmum. VIII. SIMUATION RESUTS In the second step, resources of both upper and lower levels are jontly allocated at the begnnng of uth tme slot u = {0, 1, 2,...}) subject to C4 and assumng that the current lower-level state remans unchanged durng the uth tme slot.e., ψu,0 = ψu,l, l {1,...,V 1}). Wth ths assumpton, R,u ψ u,0,a U u, D )=r,u,0 ψ u,0,a U u,a u,0) and Pavg, C = k K C ρc,k P,k C. The algorthms whch solve problems P3 and P4 are used for allocatng resources whle usng λ and ξ from the frst step. Note that these algorthms need to calculate μ only and that UE power s dstrbuted between WAN and cellular network nterfaces at ths pont. At the begnnngs of the remanng u, l)th tme slots.e., l = {1,...,V 1}), lower-level resources of subcarrers and the amount of power dedcated for the cellular network nterfaces are reallocated based on the current state ψu,l to fully explot the mult-user dversty n the cellular network. The second step of the heurstc algorthm s shown n Algorthm 4. Wreless channels are modeled as Raylegh fadng channels, and ther path loss s proportonal to d 4, where d denotes the dstance between users and the WAN AP or the cellular BS. Further, the channels over the cellular network are generated at the carrer frequency of 2.1 GHz and moble speed of 50 kmh 1, whle those over the WAN are generated at the carrer frequency of 2.4 GHz and moble speed of 3 kmh 1. Based on the coherence tmes of the channels, T and T U are selected to be 4.23 ms and ms, respectvely [29], [30]. The raduses of the WAN and the cellular coverage areas are 50 m and 1000 m respectvely, and users are unformly dstrbuted over the coverage areas. The total power avalable at each user s unformly dstrbuted between 0 and 1 watt. Table II shows the remanng parameters. Frst we evaluate the performance of the MMDP based resource allocaton algorthm MM) and the heurstc algorthm HM) n a small-scale system, denoted by system-1, and compare the performance wth that of a benchmark algorthm BM1) whch resembles the frst category resource allocaton algorthms see Secton II). Algorthm BM1 allocates the resources as follows. Frst, t assgns users for the two networks va exhaustve search such that the total average system throughput s maxmzed. In ths step, average users

13 THARAPERIYA GAMAGE et al.: TWO TIME-SCAE CROSS-AYER SCHEDUING FOR CEUAR/WAN INTERWORKING 2785 TABE II SIMUATION PARAMETERS throughputs are calculated usng B/2) log Ωp/n)), as n the frst step of HM. Next, each network ndvdually allocates ts resources among the assgned users to maxmze the network throughput. Ths step utlzes nstantaneous channel gans and s repeated at the every tme slot. It should be noted that BM1 does not allow UE mult-homng and t allocates resources at two tme-scales based on the PHY and MAC technologes of the networks. In the system-1, there are four users S S = S M =2), four subcarrers and two contenton-free TXOPs. Two-state Markov channels are used [31], and the boundary between the two states of each channel s determned such that the steady state probablty of each state s 0.5. For each channel, the channel s at the frst state f h< Ωln2); otherwse, t s at the second state. When a channel s at frst and second states, the channel gans are consdered to be Ω1 ln2)) and Ω1 + ln2)), respectvely. These channel gans are determned by averagng the square of the channel gan of a contnuous-envelope Raylegh fadng channel wthn the boundares of the respectve state. Transton probabltes of the states are calculated as n [18]. Dscount factors θ and β are set to be 0.9 n MM. Fg. 4 compares the throughputs acheved by MM, HM and BM1 n system-1 for dfferent QoS requrements. Algorthm MM provdes throughput mprovement of at least 10.7% compared to HM, and both MM and HM provde hgher throughputs than BM1 as they enable mult-homng. In BM1, each user s allowed to access one network only. When multhomng s enabled, users acheve hgher throughputs due to effcent resource utlzaton, whch s a result of caterng user QoS requrements utlzng multple network resources and of dynamcally adjustng resource allocaton ncludng UE power dstrbuton for the two network nterfaces based on the nstantaneous channel gans. In addton, MM outperforms HM due to the fact that MM allocates the resources statstcally consderng the future state changes usng an MMDP, whereas HM allocates the resources based on the current states only. The satsfacton ndex SI), whch can be used for quantfyng the ablty of a resource allocaton algorthm to satsfy Fg. 4. Throughputs acheved by dfferent algorthms n system-1. Fg. 5. Satsfacton ndex acheved by dfferent algorthms for data traffc n system-1. the QoS requrements [25], s defned for a partcular traffc class as { } R SI =E 1 R Rmn +1 Rmn >R, 31) R mn where R and R mn are the acheved and the requred throughputs, and 1 x y =1f x y but t s zero otherwse. All three algorthms have acheved SI for voce traffc SI V ) of one n system-1, and the acheved SI s for data traffc SI D ) by them n system-1 are shown n Fg. 5. Smlar to the throughput performance, MM and HM acheve hgher SI D s compared to BM1, provdng users wth better QoS. Dfference between these SI D s s sgnfcant at the hgher data traffc requrements as mult-homng s partcularly useful for caterng hgher user requrements va multple networks. Complextes of the algorthms are measured n terms of the requred number of teratons n the nner most loop per user per tme slot n fast tme-scale, and Fg. 6 compares them n

14 2786 IEEE TRANSACTIONS ON COMMUNICATIONS, VO. 62, NO. 8, AUGUST 2014 Fg. 6. Complextes of the dfferent algorthms n system-1. Fg. 7. Throughputs acheved by dfferent algorthms n system-2. system-1. As MM solves an MMDP based resource allocaton problem wth 2 18 system states, t requres a large number of teratons. Algorthm BM1 requres a hgher number of teratons than HM as BM1 recalculates λ and ξ at each tme slot, whereas HM calculates λ and ξ only once n the frst step. Next, the performance of HM s evaluated n a large-scale system, denoted by system-2, and s compared wth the performance of a benchmark algorthm BM2). Snce the hghest number of resource blocks per cell n a TE system s 110, system-2 conssts of 128 subcarrers, 10 contenton-free TXOPs and 40 or 80 users. Ths system uses contnuous envelope Raylegh fadng channels generated at the same carrer frequences and moble speeds as n system-1. The performance of MM s not evaluated n ths system due ts hgh complexty. Algorthm BM2 uses a smpler user allocaton mechansm than exhaustve search whch s used n BM1, because exhaustve search s not feasble when there s a large number of users. It allocates users for the networks as follows. Frst, users n S M are sorted n the descendng order of ther E{α W }. Second, resources of the two networks are ndvdually allocated S M tmes whle calculatng users average throughputs usng B/2) log Ωp/n)); atthejth resource allocaton, frst j users n S M are allocated to the WAN whle the remanng users n S M and all the users n S S are allocated to the cellular network. Fnally, the user allocaton whch resulted n the hghest total average throughput s selected. Once the user allocaton s completed, BM2 allocates resources of the two networks at each tme slot smlar to BM1. Throughput, SI V and SI D performance of HM and BM2 n system-2 are shown n Fgs. 7 9, respectvely. Due to the advantages of user mult-homng, HM provdes better throughput and SI performance than BM2. The performance of the algorthms decreases wth the number of users, because the resources are dstrbuted among more users as each user has a certan QoS requrement. When there are 80 users n the system, HM provdes at least 14.5%, 8.4% and 8.1% of mprovements compared to BM2 n average throughput per user, SI V and SI D, respectvely. Fg. 8. Satsfacton ndex acheved by dfferent algorthms for voce traffc n system-2. Fg. 9. Satsfacton ndex acheved by dfferent algorthms for data traffc n system-2.

15 THARAPERIYA GAMAGE et al.: TWO TIME-SCAE CROSS-AYER SCHEDUING FOR CEUAR/WAN INTERWORKING 2787 where k R +, {0,...,N}; x =[x 1,...,x N ]; and x R +, {1,...,N}. Now consder fz) fx) fx)[z x] T where ) ) 2 1 = k N k =1 z k 0 + gx), 33) N k =1 x ) N k gx)= k 0 + k 0 + z =1 N =1 k z x 2 ) N k 0 + k x =1 ) 2 34) Fg. 10. Complextes of dfferent algorthms n System-2. Accordng to the complextes of the algorthms shown n Fg. 10, HM converges wthn 25 teratons per user per tme slot whle BM2 requres more than 47 teratons per user per tme slot. The complexty of HM does not consderably vary wth the QoS requrements and the number of users as HM recalculates only μ at each tme slot n the second step of the algorthm, whereas the complexty of BM2 ncreases wth the QoS requrements and the number of users as BM2 recalculates λ, ξ and μ at each tme slot. IX. CONCUSION Ths paper has presented an MMDP based resource subcarrer, TXOPs of WAN, and power) allocaton for cellular/ WAN nterworkng consderng two tme-scales; underlyng PHY and MAC layer technologes of an OFDMA based cellular network and a WAN whch operates on contenton-based and contenton-free channel access mechansms; and multhomng capable users wth voce and data traffc requrements. Further, by elmnatng the requrement to solve Bellman optmalty equatons, a low tme complexty heurstc resource allocaton algorthm has been proposed. Smulaton results have shown that the MMDP based algorthm provdes 10.7% of throughput mprovements than the heurstc algorthm, and that the MMDP based and the heurstc algorthms provde hgher throughputs and satsfacton ndexes.e., QoS) than the benchmark algorthms BM1 and BM2) whch do not consder user mult-homng. The MMDP based algorthm has a hgh tme complexty due to large number of states n the system model. The low tme complexty heurstc algorthm converges wthn 25 teratons per user per tme slot n practcal systems, whch enables t to allocate resources onlne based on the nstantaneous channel gans. et APPENDIX A PROOF OF CONVEXITY OF R P ) fx) = 1 k 0 +, 32) N k =1 x and z =[z 1,...,z N ] wth z R +, {1,...,N}. Furthermore, gx) can be rewrtten as gx) = k 0 N + + =1 k 1 z ) 2 z x N 2 k j+1 k j =2 j=1 N 1 j=2 =1 zj z j+1 x 2 j j 2 [ k N +1 k N j+ ) 2 z j+1 z j x 2 j+1 zn j+ z N +1 x 2 N j+ ) 2 ] zn +1 z N j+ x 2. N +1 35) Snce k R +, {0,...,N}, gx) 0. Thus, by 33), fx) s a convex functon as t satsfes the frst order condton [23]. Moreover, fx) s a non-ncreasng functon as ) 2 fx) = k 1 x x 2 k 0 + 0, {1,...,N}. 36) N k =1 x et x =log y ), {1,...,N}. Snce x =log y ) s a concave functon wth respect to y R +, and fx) s non-ncreasng n each of ts argument and t s a convex functon, by vector composton theory [23], fy) = 1 k 0 + N =1 k log 2 1+y ) 37) s a convex functon. Thus, R P ) s a concave functon.

16 2788 IEEE TRANSACTIONS ON COMMUNICATIONS, VO. 62, NO. 8, AUGUST 2014 APPENDIX B PROOF OF CONVEXITY OF C4 From 5), the dervatve of Pavg, P ) when P > 0 s Pavg, P ) P T CP = B W log 2 1+P [ Next, we defne R P ) + R P α W )) 2 log 2 1+P α W P ) gx) =log x) P α W ln2) 1+P P ) α W ) ) log 2 1+P α W ) ]. 38) x, x 0. 39) 1 + x)ln2) We can conclude that gx) 0 snce g0) = 0 and dgx)/ dx 0, x R +. Moreover, R P ) s a postve nondecreasng concave functon. Therefore, by 38), P P 0. Thus, Pavg, P ) s a non-decreasng functon of P. Therefore, we can show that Pavg, P ) P T, s a convex set [23], and hence C4,.e., {Pavg, C, P,j,P C4 s satsfed, S N,j K }, s a convex set. Snce Pavg, C s a lnear combnaton of P,k C, k see 13)), { P,k C, P,j, P C4 s satsfed, S N,j K,k K C } s also a convex set. avg, P )/ APPENDIX C PROOF OF EXISTENCE OF A SOUTION FOR 24) By dfferentatng 5) and then removng the non-negatve term, we have R P ) P = BW α W P avg, ln2) P ) P < BW log 2 1+α W P ) P Pavg, P ) P, at P =0;, for P >0. 40) Partal dervatves of R P ) and Pavg, P ) wth respect to P are postve and monotoncally decreasng functons of P, because R P ) and Pavg, P ) are concave ncreasng functons wth respect to P. Furthermore, the value of B W log α W P ))/P decreases from B W α W / ln2) to0asp goes from 0 to. Therefore, there exst a p, p > 0) such that at each P >p, R P ) P < μ P 1+λ avg, P ) P Thus, from 22), 23) and 41), there exst a P soluton for 24), and P 0,p ).. 41) whch s the APPENDIX D PROOF OF CONVERGENCE OF THE ITERATIVE AGORITHM WHICH CACUATES P At the nth teraton, P s calculated usng 24) such that when P = P, the left hand and the rght hand sde terms of 41) are equal. Snce 2 R P ) Pj P R P ) P ) ) > 2 Pavg, P ) Pj P P avg, P ) P ) ), P j >0, j, 42) f Pj s ncreased at the nth teraton, the left hand sde of 41) becomes larger than the rght hand sde of 41). Thus, ncreases at the n +1)th teraton. Therefore, by nducton, P, ncrease n each teraton. However, each P s upper bounded by p, where B W log α W p ))/p = μ /1 + λ ). Therefore, the algorthm converges. Furthermore, t should be noted that each P converges to a value whch s smaller than p, because f P = p, t volates 40). P REFERENCES [1] Csco vsual networkng ndex: Global moble data traffc forecast update, , Csco Syst., San Jose, CA, USA, Csco Whte Paper, Feb [2] Part 11: Wreless AN Medum Access Control MAC) and Physcal ayer PHY) Specfcatons, IEEE P REVmb/D12, Mar [3] R. Ferrus, O. Sallent, and R. Agust, Interworkng n heterogeneous wreless networks: Comprehensve framework and future trends, IEEE Wreless Commun., vol. 17, no. 2, pp , Apr [4] X. Pe, T. Jang, D. Qu, G. Zhu, and J. u, Rado-resource management and access-control mechansm based on a novel economc model n heterogeneous wreless networks, IEEE Trans. Veh. Technol., vol. 59, no. 6, pp , Jul [5] M. Ismal and W. Zhuang, A dstrbuted mult-servce resource allocaton algorthm n heterogeneous wreless access medum, IEEE J. Sel. Areas Commun., vol. 30, no. 2, pp , Feb [6] M. J. Neely, E. Modano, and C.-P., Farness and optmal stochastc control for heterogeneous networks, IEEE/ACM Trans. Netw., vol. 16, no. 2, pp , Apr [7] TE; Evolved Unversal Terrestral Rado Access E-UTRA) and Evolved Unversal Terrestral Rado Access Network E-UTRAN); Stage 2, Sopha-Antpols, France, 3GPP TS Rep. V11.6.0, [8] H. S. Chang, P. J. Fard, S. I. Marcus, and M. Shayman, Multtme scale Markov decson processes, IEEE Trans. Autom. Control, vol. 48, no. 6, pp , Jun [9] W. Song, W. Zhuang, and Y. Cheng, oad balancng for cellular/ WAN ntegrated networks, IEEE Netw., vol. 21, no. 1, pp , Jan./Feb [10] W. Song and W. Zhuang, Interworkng of Wreless ANs and Cellular Networks. New York, NY, USA: Sprnger-Verlag, [11] J. Xu, Y. Jang, and A. Perks, Mult-servce load balancng n a heterogeneous network, n Proc. WTS, 2011, pp [12] C. uo, H. J, and Y., Utlty-based mult-servce bandwdth allocaton n the 4G heterogeneous wreless access networks, n Proc. IEEE Wreless Commun. Netw. Conf., 2009, pp [13] D. Nyato and E. Hossan, A noncooperatve game-theoretc framework for rado resource management n 4G heterogeneous wreless access networks, IEEE Trans. Moble Comput., vol. 7, no. 3, pp , Mar [14] P. Xue, P. Gong, J. H. Park, D. Park, and D. K. Km, Rado resource management wth proportonal rate constrant n the heterogeneous networks, IEEE Trans. Wreless Commun., vol. 11, no. 3, pp , Mar [15] A. T. Gamage and X. Shen, Uplnk resource allocaton for nterworkng of WAN and OFDMA-based femtocell systems, n Proc. IEEE ICC, Budapest, Hungary, Jun. 2013, pp

17 THARAPERIYA GAMAGE et al.: TWO TIME-SCAE CROSS-AYER SCHEDUING FOR CEUAR/WAN INTERWORKING 2789 [16] P. Wang, H. Jang, and W. Zhuang, Capacty mprovement and analyss for voce/data traffc over WAN, IEEE Trans. Wreless Commun., vol. 6, no. 4, pp , Apr [17] J. Zhu and A. O. Fapojuwo, A new call admsson control method for provdng desred throughput and delay performance n IEEE e wreless ANs, IEEE Trans. Wreless Commun., vol. 6, no. 2, pp , Feb [18] H. S. Wang and N. Moayer, Fnte-state Markov channel A useful model for rado communcaton channels, IEEE Trans. Veh. Technol., vol. 44, no. 1, pp , Feb [19] G. Banch, Performance analyss of the IEEE dstrbuted coordnaton functon, IEEE J. Sel. Areas Commun.,vol.18,no.3,pp , Mar [20] M.. Puterman, Markov Decson Processes: Dscrete Stochastc Dynamc Programmng. New York, NY, USA: Wley, [21] E. Altman, Constraned Markov Decson Processes. Boca Raton, F, USA: CRC, [22] C. Y. Wong, R. S. Cheng, K. B. ataef, and R. D. Murch, Multuser OFDM wth adaptve subcarrer, bt, and power allocaton, IEEE J. Sel. Areas Commun., vol. 17, no. 10, pp , Oct [23] S. Boyd and. Vandenberghe, Convex Optmzaton. Cambrdge, U.K.: Cambrdge Unv. Press, [24] D. P. Bertsekas, Non-near Programmng. Belmont, MA, USA: Athena Scentfc, [25] M. S. Alam, J. W. Mark, and X. Shen, Relay selecton and resource allocaton for mult-user cooperatve OFDMA networks, IEEE Trans. Wreless Commun., vol. 12, no. 5, pp , May [26] C. Kelley, Solvng Nonlnear Equatons Wth Newton s Method. Phladelpha, PA, USA: SIAM, [27] P. u, Z. Tao, S. Narayanan, T. Koraks, and S. S. Panwar, CoopMAC: A cooperatve MAC for wreless ANs, IEEE J. Sel. Areas Commun., vol. 25, no. 2, pp , Feb [28] M. Abramowtz and I. A. Stegun, Handbook of Mathematcal Functon wth Formulas, Graphs, and Mathematcal Tables. Washngton, DC, USA: NBS, 1964, ser. Appled Mathematcs Seres 55. [29] WINNER II Interm Channel Models, WINNER II, Tech. Rep. D1.1.1 v1.0, Dec [Onlne]. Avalable: [30] T. S. Rappaport, Wreless Communcatons, 2nd ed. Upper Saddle Rver, NJ, USA: Prentce-Hall, [31] H. Shen,. Ca, and X. Shen, Performance analyss of TFRC over wreless lnk wth truncated lnk level ARQ, IEEE Trans. Wreless Commun., vol. 5, no. 6, pp , Jun Amla Tharaperya Gamage S 07) receved the B.E. degree n electroncs and telecommuncatons engneerng from Multmeda Unversty, Cyberjaya, Malaysa, n 2008 and the M.E. degree n telecommuncatons engneerng from the Asan Insttute of Technology, Pathumthan, Thaland, n He s currently workng toward the Ph.D. degree n the Department of Electrcal and Computer Engneerng, Unversty of Waterloo, Waterloo, ON, Canada. From 2008 to 2009, he was a Solutons Archtect wth Dalog Telekom PC, Sr anka. Hs current research nterests nclude resource management for nterworkng heterogeneous wreless networks, cooperatve communcaton, and cloud computng. He s a corecpent of the Best Paper Awards from the IEEE Internatonal Conference on Communcatons Hao ang S 09 M 14) receved the Ph.D. degree n electrcal and computer engneerng from the Unversty of Waterloo, Waterloo, ON, Canada, n From 2013 to 2014, he was a Postdoctoral Research Fellow wth the Broadband Communcatons Research aboratory and Electrcty Market Smulaton and Optmzaton aboratory, Unversty of Waterloo. Snce July 2014, he has been an Assstant Professor wth the Department of Electrcal and Computer Engneerng, Unversty of Alberta, Canada. Hs research nterests are n the areas of smart grd, wreless communcatons, and wreless networkng. He s the recpent of the Best Student Paper Award from IEEE 72nd Vehcular Technology Conference VTC Fall-2010), Ottawa, ON. Dr. ang was the System Admnstrator of IEEE TRANSACTIONS ON VEHICUAR TECHNOOGY ). He served as a Techncal Program Commttee Member for major nternatonal conferences n both nformaton/ communcaton system dscplne and power/energy system dscplne, ncludng IEEE Internatonal Conference on Communcatons ICC), IEEE Global Communcatons Conference Globecom), IEEE VTC, IEEE Innovatve Smart Grd Technologes Conference ISGT), and IEEE Internatonal Conference on Smart Grd Communcatons SmartGrdComm). Xuemn Sherman) Shen M 97 SM 02 F 09) receved the B.Sc. degree from Dalan Martme Unversty, Dalan, Chna, n 1982, and the M.Sc. and Ph.D. degrees from Rutgers Unversty, Camden, NJ, USA, n 1987 and 1990, all n electrcal engneerng. He s a Professor and Unversty Research Char wth the Department of Electrcal and Computer Engneerng, Unversty of Waterloo, Waterloo, ON, Canada. From 2004 to 2008, he was the Assocate Char for Graduate Studes. Hs research focuses on resource management n nterconnected wreless/wred networks, wreless network securty, socal networks, smart grd, and vehcular ad hoc and sensor networks. He s a coauthor/edtor of sx books and has publshed more than 600 papers and book chapters n wreless communcatons and networks, control, and flterng. Dr. Shen served as the Techncal Program Commttee Char/Cochar for IEEE Infocom 14, IEEE VTC 10 Fall, the Symposa Char for IEEE ICC 10, the Tutoral Char for IEEE VTC 11 Sprng and IEEE ICC 08, the Techncal Program Commttee Char for IEEE Globecom 07, the General Cochar for Chnacom 07 and QShne 06, the Char for IEEE Communcatons Socety Techncal Commttee on Wreless Communcatons, and P2P Communcatons and Networkng. He also serves/served as the Edtor-n-Chef for IEEE Network, Peer-to-Peer Networkng and Applcaton, and IET Communcatons; a Foundng Area Edtor for IEEE TRANSACTIONS ON WIREESS COMMU- NICATIONS; an Assocate Edtor for IEEE TRANSACTIONS ON VEHICUAR TECHNOOGY, Computer Networks, and ACM/Wreless Networks, etc.; and the Guest Edtor for IEEE JSAC, IEEE WIREESS COMMUNICATIONS,IEEE COMMUNICATIONS MAGAZINE, and ACM Moble Networks and Applcatons, etc. He receved the Excellent Graduate Supervson Award n 2006, and the Outstandng Performance Award n 2004, 2007, and 2010 from the Unversty of Waterloo, the Premer s Research Excellence Award PREA) n 2003 from the Provnce of Ontaro, Canada, and the Dstngushed Performance Award n 2002 and 2007 from the Faculty of Engneerng, Unversty of Waterloo. He s a Regstered Professonal Engneer of Ontaro, Canada, an Engneerng Insttute of Canada Fellow, a Canadan Academy of Engneerng Fellow, and a Dstngushed ecturer of IEEE Vehcular Technology Socety and Communcatons Socety.