COMMUNICATION networks are expected to serve a. Delay-Aware Cross-Layer Design for Network Utility Maximization in Multi-Hop Networks

Transcription

1 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 29, NO. 5, MAY Delay-Aware Cross-Layer Desgn for Network Utlty Maxmzaton n Mult-Hop Networks Haozh Xong, Ruogu L, Atlla Erylmaz, an Eylem Ekc Abstract We nvestgate the problem of esgnng elay-aware jont flow control, routng, an scheulng algorthms n general mult-hop networks for maxmzng network utlzaton. Snce the en-to-en elay performance has a complex epenence on the hgh-orer statstcs of cross-layer algorthms, earler optmzaton-base esgn methoologes that optmze the long term network utlzaton are not mmeately well-sute for elay-aware esgn. Ths motvates us n ths work to evelop a novel esgn framework an alternatve methos that take avantage of several unexplote esgn choces n the routng an the scheulng strategy spaces. In partcular, we reveal an explot a crucal characterstc of back pressure-type controllers that enables us to evelop a novel lnk rate allocaton strategy that not only optmzes long-term network utlzaton, but also yels loop free mult-path routes between each source-estnaton par. Moreover, we propose a regulate scheulng strategy, base on a token-base servce scplne, for shapng the per-hop elay strbuton to obtan hghly esrable en-to-en elay performance. We establsh that our jont flow control, routng, an scheulng algorthm acheves loop-free routes an optmal network utlzaton. Our extensve numercal stues support our theoretcal results, an further show that our jont esgn leas to substantal en-to-en elay performance mprovements n mult-hop networks compare to earler solutons. Inex Terms Loop-free routng, en-to-en mean elay reucton, throughput-optmal algorthms, elay-aware cross layer esgn, mult-hop networks I. INTRODUCTION COMMUNICATION networks are expecte to serve a varety of essental applcatons that eman hgh longterm throughput an low en-to-en elay. Over the last ecae, we have wtnesse the evelopment of ncreasngly sophstcate optmzaton an control technques targetng flow control, routng an scheulng components to aress the cross-layer resource allocaton problems for communcaton networks. Among varous fferent evelope polces, an mportant class of throughput-optmal polces has evolve from the semnal work [1] of Tassulas an Ephremes, where they propose the well-known back-pressure scheulng/routng polcy. Ths polcy utlzes properly mantane queue-length nformaton to ynamcally etermne scheulng an routng ecsons that optmze the long-term maxmal throughput levels between all source-estnaton pars. More recent works (e.g. [2], [3], [4], [5]; also see [6], [7], [8] an references theren) extene ths framework by evelopng an optmzaton-base esgn methoology for the evelopment Manuscrpt receve 30 June 2010; revse 9 December Ths work was supporte n part by DTRA Grant HDTRA , an NSF Awars: CAREER-CNS an CCF The authors are wth the Department of Electrcal an Computer Engneerng at The Oho State Unversty, Columbus, Oho 43210, USA (e-mal: {xongh, lr, erylmaz, ekc}@ece.osu.eu). Dgtal Object Ientfer /JSAC /11/$25.00 c 2011 IEEE of jont flow control, routng, an scheulng algorthms to maxmze the long-term utlzaton of the network resources, measure through proper functons of throughput. However, exstng works have omnantly concentrate only on the long-term performance metrcs of throughput or utlzaton, whle gnorng the en-to-en elay metrc that s crucal to many essental applcatons. Ths restrcton allowe for the formulaton of the esgn problem as a statc optmzaton problem n terms of the mean behavor of the network algorthm operaton. In partcular, long-term esgn objectves can easly be escrbe n terms of the mean flow rates an the mean lnk rates that the network algorthm proves to the applcatons. Unfortunately, ths approach s no longer applcable when en-to-en elay s taken nto account as ento-en elay s a complex stochastc process that epens on the hgher orer statstcs of the network algorthm operaton. Ths calls for a more sophstcate elay-aware cross-layer esgn framework, an novel strateges that prove favorable en-to-en elay performance, whle preservng long-term optmalty characterstcs. Wth ths motvaton, n ths paper, we are ntereste n the well-foune esgn of cross-layer network algorthms for general mult-hop networks that not only utlzes the network resources for long-term throughput optmalty, but also exhbts esrable en-to-en elay characterstcs. Our contrbutons n ths recton can be summarze as follows: We propose a novel esgn paragm that ecouples the objectves of long-term utlty maxmzaton from the en-to-en elay-aware resource allocaton. Ths framework s expecte to enable the systematc evelopment of future schemes, n aton to the one we evelop n ths paper. We reveal the soluton gven by the back-pressure polcy has a unque structure whch facltate us to establsh loopfree mult-path routes that guarantee long-term network utlty maxmzaton. Ths new routng-scheme not only nherts the aaptve an optmal nature of the long-term optmal algorthms, but also elmnates the unnecessary loops to reuce the en-to-en elay wthout sacrfcng from throughput. We combne ths loop-free route constructon strategy wth a token-base scheulng scplne that regulates the hgherorer statstcs of servce processes to acheve rastc reuctons n the en-to-en elay performance, whle guaranteeng long-term optmalty characterstcs. The above ponts also consttute the man fferences of ths work from the recent efforts n the esgn of algorthms wth low en-to-en elay performance (e.g. [9], [10]). We also note that there has been recent nterest n ervng funamental bouns on the elay performance ([11], [12], [13]). In our future work, we are ntereste n utlzng an extenng these

2 952 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 29, NO. 5, MAY 2011 results to stuy the gap between these funamental bouns an the elay performance of our algorthms. The remaner of the paper s organze as follows. Secton II ntrouces our system moel an objectves together wth the escrpton of several exstng algorthms. In Secton III, we propose our elay-aware esgn framework that escrbes the esre characterstcs an nterconnectons of the routng, scheulng, an flow control components of the network algorthm. In Secton IV, we bul upon the propose esgn framework to construct a elay-aware cross-layer algorthm that performs the esre tasks. The numercal result of the polcy an our conclung remarks are prove n Secton V an VI, respectvely. II. SYSTEM MODEL AND OBJECTIVE In ths work, we stuy wre networks to concentrate on the elay behavor of network algorthms wthout the atonal complcatons of nterference lmte wreless communcatons. Dscusson of possble extenson of our propose frameworks to nterference lmte networks s prove n the conclusons. We conser a fxe mult-hop network represente by graph G =(N, L, c), wheren s the set of noes, L s the set of brectonal lnks (, j) where, j N.We use c =(c j ) {(,j) L} to enote the vector of brectonal lnk capactes n packets per slot,.e., both c j an c j refer to the brectonal lnk capacty of the brectonal lnk (, j). For clarty, we use, j to enote the recte lnk from noe to noe j. Tme s slotte n our system, an external packets arrve at the begnnng of each tme slot. The network resources are to be share by a set of commotes. We stngush fferent commotes by ther estnatons. We efne D to be the set of all estnaton noes. In ths paper, we are ntereste n esgnng jont flowcontrol, scheulng, an routng polces wth esrable longterm throughput an short-term elay characterstcs. Our ual goal wll be scusse n further etal after we ntrouce some notatons. In each tme slot, the servce on the lnk, j of commoty s enote by Rj [t], whch s assume to be a statonary ergoc stochastc process. It s etermne by the scheulng an routng polcy. We let rj = lm t 1 t t τ =0 E(R j [τ]) to be the lnk rate of commoty on lnk, j, an efne r =(rj ),j, to be the vector of all such lnk rates. Uner flow control mechansm, the number of the exogenous packets that arrves at noe s estne to noe at tme slot t s enote by Xs [t], whch s also assume to be statonary an ergoc. Smlarly, x s = lm t 1 t t τ =0 E(X s [τ]) s the corresponng rate, an we let x = (x s) s, to be exogenous arrval rate vector. A utlty functon U s (x s ) s assocate wth each source-estnaton par (s, ). We make the followng typcal assumpton on the utlty functon: Assumpton 1: The utlty functons {U s (x s)} s, are strctly concave, twce fferentable an ncreasng functons. 1 A. Objectve Our goal s to evelop a jont flow-control, scheulng, an routng algorthm that not only optmzes long-term network 1 Ths s not a crtcal assumpton, but wll make our analyss clear. utlzaton, but also proves esrable elay characterstcs. We scuss these two goals next. Utlty Maxmzaton: For the network (N, L, c), the network utlzaton maxmzaton optmzaton problem s efne as: U s (x s ) (1) max {X[t],R[t]} s, s.t. Xs [t] 0, s, N, t 0, (2) Rj[t] 0, (, j) L, t 0, (3) Rj[t]+ Rj[t] c j, (, j) L, D, t 0, (4) x + rm rj, m, L,j L N, D,. (5) We efne the feasble soluton as: Defnton 1: (Feasble soluton) A soluton (X[t], R[t]) s feasble f t satsfes contons (2) to (5). We use {X [t], R [t]} to enote the optmal soluton to (1), an we efne x 1 t 1 = lm X [τ], (6) t t r 1 = lm t t τ =0 t 1 R [τ]. (7) τ =0 Note that uner Assumpton 1, x s unque whle r generally belongs to a set of optmal lnk rates enote by R. In ths paper, we efne the stablty as follows: Defnton 2: (Network stablty) We say the network s stable n the mean sense f for any queue (prce) Pnj, n, j N, 1 [ ] t 1 N,wehavelm sup t t τ =0 E Pnj [τ] <. Delay Improvement: A step beyon just solvng the above optmzaton, our secon goal s to evelop a new mechansm that reuces the en-to-en elay experence by the traffc whle mantans the utlty maxmzng nature. The en-to-en elay experence by one packet s efne as the fference between the tme nstance of njecton at the source an recepton at the estnaton, whch s a short-term metrc nstea of a long-term throughput. We also am to fn a new archtecture that can ecouple the fferent objectves. B. Backgroun The back-pressure algorthm mantans a queue for each commoty n each noe whose length at tme t s enote by P [t]. The algorthm s propose for a general network moel, an specfcally, the back-pressure algorthm uner our system moel s gven by Dscrete Tme Back-Pressure (DTBP) Polcy ([14], [5]) Queue (Prce) Evoluton: Each queue evolves as (8). Rate control: In tme slot t, theflow controller etermnes the mean of the njecton as E(X s [t] P s [t]) = mn{u 1 s (P s [t]/k),x max },

3 XIONG et al.: DELAY-AWARE CROSS-LAYER DESIGN FOR NETWORK UTILITY MAXIMIZATION IN MULTI-HOP NETWORKS 953 P [t +1]= P [t],j L + Rj [t] + X [t]+ m, L Rm [t] (8) where X max s some fnte upper boun for the arrval rate. Scheulng an Routng: Each lnk l =(, j) selects l [t] = argmax P [t] Pj [t],l L, (9) an assgn { Rj [t] = cj, f = l [t] an P [t] P j [t] > 0; (10) 0, otherwse. In the event of multple commotes satsfyng Equaton (9), we arbtrarly choose one such commoty wth equal probablty as a te-breakng rule. The constant K n the algorthm s a esgn parameter that etermnes how close the algorthm can converge to ts optmal soluton (see [14], [5]). Also we shall see n Secton IV ths soluton s relate to ual ecomposton. Remark 1: In our moel, snce there s no nterference between lnks, the back-pressure algorthm reuces to makng ecsons on each lnk nepenently. Snce each lnk can only transmt n one recton at a gven tme slot, t s necessary to use the weght efne n Equaton (9) to etermne whch one of the rectonal lnks shoul transmt. Remark 2: Note that the rate assgnment n Equaton (10) mples that we only allow transmsson on those lnks wth strctly postve maxmum back-pressure. Ths oes not affect the optmalty of the DTBP algorthm. Remark 3: The prce evoluton n Equaton (8) mples that n the case of a noe has less packets than scheule servce, ummy packets are transmtte. Thus the allocate servce Rj [t] always equals to number of packets transmtte over lnk, j at tme t as suggeste n Equaton (8). There are many follow up works of the back-pressure algorthm. In partcular, n [10], an nterestng mn-resource algorthm s propose base on the back-pressure algorthm. In ths mplementaton, nstea of usng the queue-length fference P [t] Pj [t] ( as the weght of a lnk, P [t] Pj [t] ) M s use, where M s some postve constant. Ths mofcaton scourages the use of lnks unless the queue-length fferences are suffcently hgh, an hence reuces possble loops n the network. III. DELAY-AWARE DESIGN FRAMEWORK In ths secton, we expose the elay efcences of longterm utlty maxmzng esgns such as DTBP an ts varants, both conceptually an through numercal stues. In partcular, we reveal that both the mult-path routng an the scheulng components of the earler esgns must be sgnfcantly change or enhance to obtan favorable ento-en elay performance. Base on the observatons, we propose a general elay-aware esgn framework whereby the utlty maxmzaton s combne wth elay-aware routng an scheulng components to acheve our ual objectve. Fg. 1. The strbuton of hop-count for flow 1 packets ncates the presence of unnecessary loops n ther routes A. Defcences n mult-path routng Long-term performance optmzng algorthms share the common characterstc of contnuously searchng for routes to maxmze the en-to-en throughput of the flows. However, ths potentally causes loops an unnecessarly long routes for a large subset of the packets, as we shall emonstrate n the followng example. Conser a 6 6 gr wth unt capacty lnks servng two commotes as shown n Fg. 1. The source-estnaton noe par for Commoty 1 an Commoty 2 are marke wth square an crcle respectvely. Uner the operaton of DTBP, the hop-count strbuton of Commoty 1 packets converges to the plotte strbuton. We can observe that the hop count excees the maxmum possble loop-free path length of 35 for a non-neglgble fracton of the packets. It turns out that such behavor s typcal for smlar long-term optmal polces uner fferent setups. In ths work, we are ntereste n preservng the long-term optmalty of such solutons, but also elmnatng loops an hence sgnfcantly mprovng the elay performance. We wll explore the versty n the set of optmal lnk rates R to acheve ths. B. Defcences n scheulng an flow control Long-term utlty maxmzng polces such as DTBP converge to a set of mean flow an lnk rates that solves (1). Yet, the en-to-en elay of these polces s a complex functon of hgher orer moments of the packet arrval an servce processes that are etermne by the flow controller an the scheuler. As a smple example, t s well-known ([15]) that n a G/G/1 queueng system, the mean watng tme W s boune by W σ2 a + σ2 b 2 t(1 ρ), (11)

4 954 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 29, NO. 5, MAY 2011 feasble rate assgnment. We then combne these two schemes n our elay-aware, cross layer congeston control, routng an scheulng polcy followng our propose archtecture of Secton III-C. Fg. 2. Mult-Layer Desgn Framework where σ 2 a s the varance of nter-arrval tme, σ 2 b s the varance of servce tme, t s the mean of nter-arrval tme, an ρ s the utlty factor. Ths formulaton suggests that reucng the varance of the arrval an servce processes s crucal n reucng the elay. In a large scale mult-hop network, t s not possble to formulate the en-to-en elay as a functon of the arrval an servce processes. However, motvate by the above observaton, n ths work, we are ntereste n regulatng the flow controller an scheulng components of the long-term optmal esgn to mprove elay performance. C. Propose mult-layer esgn framework The prevous two subsectons expose efcences n the routng, scheulng, an flow control components of exstng long-term utlty maxmzng algorthms, an reveal new opportuntes n the esgn of elay-aware cross layer algorthms. These motvate us n ths secton, to propose a novel multlayer esgn framework for mult-purpose polcy esgn. The qualtatve structure of our propose framework s shown n Fg. 2, where the goals of long-term optmalty, loopfree routng, an regulate flow control an scheulng are ecouple nto fferent layers of the conceptual framework. In the frst layer, we solve the problem of utlty maxmzaton. Note that uner Assumpton 1, the optmal soluton for flow rates, x s unque, whle r may have multple solutons. The elay-aware routng layer explots the freeom amongst these lnk-rate solutons an obtans a set of more esrable lnk rates that mproves elay performance whle preservng the utlty maxmzaton. Acqurng the utlty maxmzaton flow rates form the frst layer an the elay-aware lnk-rates form the secon layer, the elay-aware scheuler regulates the arrval an servce process to acheve better elay performance. As such, ths establshes a esgn framework for evelopng elay-aware cross-layer algorthms that possess the above characterstcs an hence s expecte to be useful for algorthm esgns for verse network scenaros. In partcular, we shall next present specfc algorthms wth attractve elay characterstcs. IV. DELAY-AWARE ROUTING AND SCHEDULING In ths secton, we present our elay-aware routng an elay-aware scheulng algorthms for our propose framework. Our elay-aware routng algorthm preserves utlty maxmzng propertes whle avong loops. Our elay-aware scheulng algorthm mplements a network stablzng tokenbase scheme that sgnfcantly reuces queue lengths for any A. Delay-aware route constructon To overcome the savantages observe n the backpressure algorthm, a natural ea s to restrct the recton n whch packets can be transmtte over a lnk for a certan commoty. To acheve ths, we frst ntrouce a flu approxmaton assocate wth (1) for the followng scusson. max x,r U s (x s) (12) s, s.t. x s 0, s, N, (13) rj 0, (, j) L, (14) rj + rj c j, (, j) L, D, (15) x + rm rj, m, L,j L N, D,. (16) Note that f an optmal soluton of (1) s {X [t], R [t]}, t s shown n prevous works (e.g. [14], [5]) that the assocate (x, r ) efne n (6) an (7) s an optmal soluton of (12) 2. An Alternate Optmal Soluton: To ensure the optmalty of the propose algorthm, we explore the soluton space for an alternate optmal soluton wth a partcular structure. Frst, we efne the followng mappng of the lnk rates: Defnton 3: (Delay-aware lnk rate mappng) The elayaware lnk rate mappng ˆR j [t] on the recte lnk, j for commoty at tme t s efne as ˆR j [t] =1 t ( t 1 + ( R j [τ] Rj [τ])), (17) τ =0 where N, (, j) Lan (Z) + =max(0,z). Note that ths measures the runnng average of the net rate of commoty traffc traversng lnk, j. Suppose we use Rj [t] = R j [t] n (17). Let ˆr j = lm t ˆR j [t] an ˆr =(ˆr j ),j, then ˆr s relate to r as: ˆr j = ( rj ) + r j, (18) ˆr j = ( rj ) + r j. (19) We shall show that the lnk rate mappng (18) an (19) preserves flow rate optmalty. Proposton 1: If (x, r ) s an optmal soluton to (12), then (ˆx, ˆr ) wth ˆx = x an ˆr beng as efne n Equatons (18) an (19) s also an optmal soluton. Proof: The etale proof s prove n Appenx A. 2 More precsely, (x, r ) s wthn O(1/K) of the optmal soluton of (12). Nevertheless, we stll call (x, r ) an optmal soluton of (12) snce the converge can be precse f a mnshng step-sze (n our notaton, f K grows lnearly wth tme t) s use.

5 XIONG et al.: DELAY-AWARE CROSS-LAYER DESIGN FOR NETWORK UTILITY MAXIMIZATION IN MULTI-HOP NETWORKS 955 Fg. 3. A tanem network wth N 1 hops. Remark: Note that uner ths lnk rate assgnment, for each (, j) L, at least one of ˆr j an ˆr j equals to zero as f the corresponng brectonal lnk becomes a unrectonal lnk. Clearly, those lnks wth equal rates on both rectons wll have a net rate of 0. Also, note that ths lnk rate mappng preserves the optmalty of (ˆx, ˆr ) when apple to any optmal soluton (x, r ) to problem (1) whch s not necessarly gven by the DTBP algorthm. The Steay-State Behavor of the DTBP Algorthm: Prevous works (e.g. [14]) also show the stablty of the DTBP algorthm. In other wors, the Markov chan wth the queuelengths P =(P ), beng ts states s postve recurrent an ergoc uner the DTBP algorthm. We use π(p ) to enote the steay state strbuton uner the DTBP algorthm,.e., the probablty of the queue-lengths beng P s π(p ) after the convergence of the DTBP algorthm an P π(p )=1. Also, for queue P, the followng hols: 1 t lm E(P t t [τ]) = P π(p ), (20) τ =0 P an we use P = P P π(p ) (21) to enote the optmal average queue-length of queue P. Intutvely, the lnk rate assgnment equaton (10) of the DTBP algorthm mples that f there s a postve net flow rate from noe to noe j for estnaton, then the average queue-lengths shoul be strctly ecreasng,.e., P > P j, snce can sen a packet to j only when P [t] > P j [t]. In certan smple topologes, ths statement can be proven rgorously. For a tanem network (see Fg. 3 for an example) wth unt lnk capactes an one en-to-en flow, we have the followng results: Lemma 1: In such a tanem network wth no more than 3 hops, the average queue-length of the DTBP algorthm s strctly ecreasng from the source to the estnaton for any arrval rate a (0, 1) uner a Bernoull arrval process. Proof: Ths result can be prove by constructng the queue-evoluton Markov chan uner the DTBP algorthm from zero state an solvng for the steay-state strbuton. The etale proof of Lemma 1 s prove n [16]. The metho use n the proof of Lemma 1 can be extene to a k-hop tanem network wth a Bernoull arrval. However the analyss becomes cumbersome as k ncreases. Thus we take a fferent approach to exten the above result to a more general k-hop tanem network. Lemma 2: In a tanem network uner our moel wth unt lnk capactes an one en-to-en flow wth a Bernoull arrval wth rate a ( 1 2, 1), the average queue-length P s strctly ecreasng from the source to the estnaton uner DTBP. Proof: Ths lemma s prove by expanng the average queue-length fference P P j as P π(p )(P Pj ). Ths sum can be strctly boune away from 0 uner the assumpton that the arrval rate a les n ( 1 2, 1). Also note that n the partcular topology of a tanem network, the queuelength fference between two neghborng noes can be at most 3 uner a Bernoull arrval process. The etale proof of Lemma 2 s prove n [16]. Due to the complex nteractons n the ynamcs of the queue-length n the neghborng noes n a general network, a rgorous proof of a generalzaton of Lemma 2 remans an open research problem. However t can be observe from numercal stues (see [16] for more examples) of more general networks that, uner the DTBP algorthm, the average queue-lengths are strctly ecreasng over lnks wth a postve net flow rate. Wth such results an observatons, we present the followng assumpton on the DTBP algorthm: Assumpton 2: Uner our system moel, the DTBP algorthm converges to a optmal soluton of (1) {X [t], R [t]} an the corresponng queue evoluton P [t] as efnen(8). The optmal solutons satsfy the followng property: If rj where rj >r j then P > P j, (, j) L, D, P P j an r j are efne as n (7), an an are efne as n (21). Remark: Note that the converse of Assumpton 2 s not necessarly true,.e., P > P j oes not necessarly mply that rj >r j, whch can be sprove by counter examples. Loop-Free Route Constructon: Conser the network G =(N, L, c). For each commoty, weefne a subgraph Ĝ =(ˆN, ˆL, ĉ ) as: ˆN = N ; ˆL = L\{, j L:ˆr j { =0}; ĉ 0 f, j / j = ˆL an j, / ˆL ; otherwse. c j Ths Ĝ s the restrcte network topology that commoty sees after applyng the elay-aware lnk rate assgnment. Then we have the followng proposton: Proposton 2: Uner Assumpton 2, gven a network G = (N, L, c) an ts optmal soluton (x, r ) gven by the backpressure algorthm to problem (12), the subgraph Ĝ efne as above s loop-free D. Proof: Assume Ĝ has a cycle that s compose of lnks {(n c1,n c2 ), (n c2,n c3 ),, (n ck 1,n ck ), (n ck,n c1 )} ˆL. By the efnton of subgraph Ĝ, we can assume wthout loss of generalty that ˆr c c +1 Then by Assumpton 2, we have P > 0 for 0 k 1 an ˆr c k c 1 > 0. P c 1 > P c 2 > > P c k > c 1, whch s mpossble. Therefore Ĝ s loop-free. Remark: As mentone n the proposton tself, the loopfreeness hols only when the elay-aware lnk rate assgnment s apple to the optmal soluton gven by DTBP. Example: We use the topology shown n Fg. 4(a) as an example to llustrate how the elay-aware lnk rate assgnment works wth the optmal soluton gven by DTBP. Each lnk n the network s assume to have unt capacty. Noe 1 s the source an Noe 3 s the estnaton. Shown n Fg. 4(b) s a possble optmal soluton of the problem, wth r 12 = r 23 =1an r 24 = r 45 = r 52 = r, where

6 956 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 29, NO. 5, MAY 2011 A n [t] R nj 1 [t] R m N nm [t] noe n (n, j 1 ) Fg. 4. An example to llustrate the DTBP soluton wth elay-aware lnk rate assgnment. r (0, 1]. Note that there exsts a loop ( ) n ths optmal soluton. However, analyzng the corresponng Markov chan shows that the DTBP algorthm can never converge to such an optmal soluton. In fact, the DTBP converges to a soluton shown n Fg. 4(c), where r 12 = r 23 = 1, lnk (2, 4) an (2, 5) have equal an non-zero transmsson rate n ether recton, an there s no transmsson on lnk (4, 5). Note that ths confrms Assumpton 2 n ths partcular topology, an also confrms that the optmal solutons gven by DTBP s a subset of all optmal solutons wth a specal structure. If we were to apply the elay-aware lnk rate assgnment to the optmal solutons shown n Fg. 4(b), t can be verfe that the loop ( ) stll exsts n the resultng topology. On the other han, Fg. 4() shows the resultng topology when the elay-aware lnk rate assgnment beng apple to the optmal soluton gven by DTBP algorthm, whch s a loop-free route. B. Delay-aware scheuler esgn Usng the elay-aware routng of Secton IV-A, utlty maxmzaton can be acheve whle avong loops, whch leas to lower average en-to-en elays. Here, we seek to mprove the elay performance further through elay-aware scheulng at each lnk. As argue n Secton III-B, sgnfcant per hop elay mprovement can be acheve by reshapng the servce strbuton through regulatng the scheuler operaton. In prevous works [17], the noton of regulators were ntrouce to help stablze the network wth fxe sngle-path routng. Aoptng a smlar concept, we propose a elay-aware scheuler wth regulate servce that stablzes (N, L, c) when there exsts a loop-free R[t] such that X[t] satsfes x nt ( Λ(N, L, c) ). We assume that the arrvals take place after the servce n each tme slot,.e., the arrvals cannot be serve n the arrvng tme slot. We further assume that from now on all lnks have the same lnk capacty enote by c. Defnton 4: (Delay-aware scheuler wth regulate servce) The elay-aware scheuler conssts of per-neghbor-percommoty queues,.e., the scheuler at noe n mantans queues Q nj for each next-hop neghbor j an each commoty. The servers apply regulate servce scplne to shape the traffc. Let the servce an eparture of the server be S(t) an D(t), respectvely. Let A n [t] enote the commoty packets enterng noe n n slot t. Mathematcally, A n [t] =X n [t]1 {n N source } + k N D kn [t], where Nsource enotes the source noe set wth respect to commoty. LetA nj [t] enote the number of packets of Fg. 5. A n E [ A n[t] ] R njm [t] m N A Q nj 1 nj 1 R nm [t] + δ Delay-Aware Scheuler Structure... Q nj m M nj 1 M nj m... (n, j m ) commoty arrvng at queue Q nj n slot t an A n[t] = j N A nj [t]. The queue ynamcs for each queue s gven as: ) + Q nj (Q [t +1] = nj [t] D r,nj [t] + A nj [t], n, (22) The server of the elay-aware scheuler operates wth a token-base servce scplne whch conssts of three components: Arrval Splttng: Each commoty packet arrvng at noe n goes nto the per-neghbor-per-commoty queue Q nj wth probablty R nj [t] [t], resultng n R m N nm [ ] E A R nj[t] A n[t] = A nj n[t] [t] m N R nm[t], (23) t 1 where R nj [t] = 1 t τ =0 R nj [τ]. Ths splttng preserves the same mean lnk rates as the orgnal R[t]. Asan example, the arrval splttng can be one usng a tokenbase scheme. Token Generaton: For each lnk, j L, the system mantans token counters m nj for each regulator queue Q nj as shown n Fg. 5. As the nput to the elay-aware scheuler s loop-free, the transmssons can only be unrectonal for certan commoty over lnk (, j). Hence m nj an m jn cannot be non-zero smultaneously. Then we efne Mnj = m nj + m jn. The token arrves at each counter wth rate Snj = E [ A n[t] ] R nj [t] m N R + δ, (24) nm[t] where δ>0. Note that the token arrval rates are enlarge by δ whch s allowe by the excess capacty from the utlty maxmzng soluton for (N, L, c ɛ). Servce: In each tme slot t, the token-base server chooses a wnner commoty nj wth a non-zero backlog for each

7 XIONG et al.: DELAY-AWARE CROSS-LAYER DESIGN FOR NETWORK UTILITY MAXIMIZATION IN MULTI-HOP NETWORKS 957 lnk (n, j) as follows: ( argmax M nj [t] c ), [ nj[t] = f max M nj [t] c ] > 0;, otherwse. (25) The server serves the commoty wth the largest token count that excees the lnk capacty c. Note that the lnk capacty c s share by transmssons on lnk (n,j) n both rectons. Then { Dnj[t] c, f = nj = ; 0, f nj. (26) In case of nsuffcent packets n queue, the server sens out ummy packets to ensure a eparture of c packets. After servce, the token counter M nj nj for the commoty nj s ecrease by c. The above algorthm oes not requre the knowlege of R[t] as long as R[t] :=( R nj [t]) n,j, s prove. Next we efne the network capacty regon for multhop networks. Defnton 5: (Network capacty regon) The network capacty regon s Λ(N, L, c) = Co{x}, where x s the exogenous arrval rate vector such that x 0 ansuchthat there exsts a lnk rate vector r satsfyng r 0, rnj =0, (n, j) / L, rnn =0, n N, rnj =0, n =, x n rnj rmn, n, j n rnj + rjn c nj, (n, j) L. Notce that the soluton (X[t], R[t]) s feasble f (x, r) satsfes Defnton 5 such that x Λ(N, L, c), where 1 t 1 x = lm t t τ =0 X[τ], r = lm t 1 t 1 t τ =0 R[τ]. Base on the efne network capacty regon, we now ntrouce the followng proposton: Proposton 3: The elay-aware scheuler operatng n the network (N, L, c) stablzes the queues (Q nj ) n,j,, fthere exsts any loop-free soluton (X[t], R[t]) such that x Λ(N, L, c ɛ), ɛ 0. Proof: Here, we prove the outlne of the proof to hghlght the essental steps nvolve, an refer the ntereste reaer to the etale proof avalable onlne n [16]. We frst efne the token count process, whch escrbes the token counter behavor. We prove the lemma on the fact that the token count process keeps track of the assgne token generaton rate an the evaton s always boune. From the esgn of the elay-aware scheuler, the servce rate of the next-hop neghbor s larger than ts prevous-hop neghbor by δ. An ths δ s allowe by the excess capacty when the soluton of the network (N, L, c ɛ) s use as nput to control the network (N, L, c). We make use of the servce rate fference to prove that the T -step rft of the Lyapunov functon s negatve, where T s a fnte value. Then applyng the prevous results n [14], we manage to prove the proposton. Remark: To stablze the network, the propose elay-aware scheuler benefts from the specal propertes of the token count process. In fact, we have prove that the elay-aware scheuler structure can be extene to utlze a general servce process that satsfes certan regulaton constrants an stll acheves network stablty. The exstence an characterzaton of other elay-aware servce processes remans an open research problem. C. Delay-aware cross-layer polcy The approaches we propose n Secton IV-A an IV-B can be use n the mult-layer archtecture of Secton III-C. To attan a utlty maxmzaton soluton that reuces the expecte en-to-en elay, we ncorporate back-pressure soluton to problem (1) together wth elay-aware routng an elayaware scheulng n the polcy esgn. In the followng polcy, we call the queue length seen by DTBP the prce, whch s avaluereflectng the queue length prce nstea of beng an actual queue. Delay-Aware Cross-Layer Polcy The frst layer runs DTBP polcy (see Secton II-B) to generate ( X[t], R[t]) for (N, L, c ɛ) n a vrtual mplementaton usng counters for prces. The secon layer performs the elay-aware lnk rate mappng from the frst layer an generates ˆR[t] as n (17) wth R[t] = R[t]. The thr layer uses elay-aware scheulers wth regulate servce n the network (N, L, c). Soluton X[t] = X[t] s receve from the frst layer. Soluton R[t] = ˆR[t] s receve from the secon layer. Then the elay-aware scheulng s performe as escrbe n Secton IV-B whch etermnes the actual queueng ynamcs. Ths cross-layer polcy s an onlne polcy that starts to functon whle the DTBP s convergng. All layers run fferent algorthms n parallel an ynamc soluton upates can be performe among layers. The cross-layer polcy nherts optmalty (Proposton 1) an loop-freeness (Proposton 2) characterstcs of the routng component, an the stablty (Proposton 3) of the elayaware scheulng component, whch lea to the followng funamental result: Theorem 1: Delay-aware cross-layer polcy results n loopfree an stable rate assgnments that arbtrarly approaches maxmum utlty solutons for a network (N, L, c). V. NUMERICAL RESULTS In ths secton, we present numercal results for our propose elay-aware cross-layer polcy. We compare the elay performance of our polcy wth the back-pressure polcy, whch has long term optmalty guarantees, an the more recent mn-resource algorthm (see Secton II-B), whch has superor routng characterstcs over back-pressure polcy. As a representatve topology, smulatons are carre out n a 6 6 gr network. Smulaton uraton s tme slots. Movng average metho s use to upate ˆR[t] as follows: [ t 0 ˆR nj [t] = Rnj [τ] R jn [τ] ] +,t 0 t<t 0 + T, W τ =t 0 W

8 958 IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 29, NO. 5, MAY 2011 (a) (b) (c) Fg. 6. (a) Back-Pressure Polcy (b) Mn-Resource Algorthm (c) Delay-Aware Cross-Layer Polcy In mult-layer archtecture, the elay-aware cross-layer polcy keeps real layer queue length separate from back-pressure layer queue length. Then the epenence on K s relaxe. Fg. 7 shows that n the smulate scenaro, the senstvty of the elay on K s sgnfcantly reuce wth elay-aware cross-layer polcy. As expecte, the elay uner back-pressure polcy grows as K ncreases. In ths fgure, we also show that the en-to-en elay can be reuce by elmnatng loops alone. However, further elay mprovement can be acheve by utlzng the elay-aware scheulng wth elay-aware routng. Fg. 7. En-to-En Delay uner Dfferent K Values where W s the wnow sze an T s upate pero. W an T are both set to We assume that all lnks have unt capacty. The K parameter for the flow controller of the back-pressure polcy s chosen to be 200. We mplement mn-resource algorthm as escrbe n Secton II-B wth M =3. For far comparson, all queues are mplemente as per-commoty queues. The smulate scenaro has parallel traffcs wth two commotes. For better emonstraton, the routng nformaton s embee n elay strbuton graph. Snce the sourceestnaton pars are symmetrc, results are gven for one commoty only. Squares an crcles ncate source an estnaton locatons for commoty 1 an 2, respectvely, both flowng from top to bottom. As s shown n Fg. 6(a), the back-pressure polcy results n loops. The elay strbuton s heavy-tale whch also ncates that packets may experence loopng. Fg. 6(b) emonstrates that the mn-resource algorthm utlzes mnmal number of lnks, thus attanng shortest paths from source to estnaton for each commoty. Snce loops are avoe, the elay strbuton s mprove wth the elay peaks move towars lower values. Fg. 6(c) shows the network topology for our polcy where the selecte paths are loop-free. From the elay strbuton, one observes that the majorty of packets follow the shortest path an the others are route over longer paths. The mean elay performance s mprove over other two solutons. We further stuy the effect of the esgn parameter K use n the DTBP polcy. It s known that the allocate flow rates approach the optmal rate x as K. However, ths also ncreases the equlbrum queue lengths n the DTBP an causes elay egraaton ([18], [4], etc.). The elayaware cross-layer polcy offers a soluton to ths lemma. VI. CONCLUSION In ths work, we expose the elay efcences of many long-term optmal polces when operate n mult-hop networks, an entfe several unexplote esgn choces wthn the routng an scheulng strategy spaces that yel rastc elay mprovements. In partcular, we explot: the flexblty n lnk rate assgnments to elmnate loops n routng; an the servce shapng opportuntes n scheulng to reuce queue-szes, whle preservng the long-term optmalty characterstcs. Along wth these algorthms, we also present a generc elay-aware esgn framework that can be use to evelop other long-term optmal algorthms wth favorable elay characterstcs. Ths framework proves a unque an methocal approach to the esgn of moular solutons where en-toen elay characterstcs can be mprove wthout sacrfcng from long-term optmalty. As such, our framework offers a promsng recton for tacklng the challengng problem of elay-aware algorthm esgn for wreless mult-hop networks, as well. To that en, n our future work, we wll exten the nsghts an technques evelope n ths work to the wreless oman, whle accountng for the ntrnsc challenges assocate wth the nterference-lmte nature of wreless communcatons. APPENDIX A. Proof of Proposton 1 Snce ˆx = x, the optmal value of the objectve functon (12) s not affecte. Thus, to show that (ˆx, ˆr ) s also an optmal soluton to (12), t s suffcent to show that the par (ˆx, ˆr ) satsfes the constrants (13) (16). Verfyng Equaton (13) an (14): ˆx s 0 because x s 0 for all s, D; ˆr j 0 for all (, j) Lan Dby ts efnton n Equatons (18) an (19).

9 XIONG et al.: DELAY-AWARE CROSS-LAYER DESIGN FOR NETWORK UTILITY MAXIMIZATION IN MULTI-HOP NETWORKS 959 Verfyng Equaton (15): By the efnton of ˆr j an 0, (, j) L, D, we have the fact that r j ˆr j rj, (, j) L, D.Then ˆr c j j. r j. So ˆr j Verfyng Equaton (16): Notce that for any real number Z, (Z) + = 1 2 (Z + Z ), Equatons (18) an (19) can be rewrtten as: ˆr j = ( (rj r j )+ r j r j ) /2 an ˆr j = ( (rj r j )+ r j r j ) /2. Subtractng these two equatons, we have ˆr j ˆr j = r j r j. Thus, we have the followng ˆx n + ˆr m ˆr j = ˆx = x + j + j m, L (ˆr j ) ˆr j,j L ( r j rj ) 0. [17] X. Wu an R. Srkant, Regulate maxmal matchng: A strbute scheulng algorthm for mult-hop wreless networks wth noeexclusve spectrum sharng, n Proceengs of IEEE Conference on Decson an Control., [18] A. Erylmaz an R. Srkant, Far resource allocaton n wreless networks usng queue-length base scheulng an congeston control, n Proceengs of IEEE Infocom, vol. 3, Mam, FL, March 2005, pp Haozh Xong (S 09) receve hs B.S. egree n Communcaton Engneerng from Bejng Unversty of Posts an Telecommuncatons, Bejng, Chna, n 2007, an the M.S. egree n Electrcal an Computer Engneerng from The Oho State Unversty n Hs research nterests nclue optmal control n communcaton networks, lowelay scheulng algorthm an archtecture esgn, an control theory. Therefore, (ˆx, ˆr ) s also an optmal soluton to (12). REFERENCES [1] L. Tassulas an A. Ephremes, Stablty propertes of constrane queueng systems an scheulng polces for maxmum throughput n multhop rao networks, IEEE Transactons on Automatc Control, vol. 36, pp , December [2] X. Ln an N. Shroff, Jont rate control an scheulng n multhop wreless networks, n Proceengs of IEEE Conference on Decson an Control, Parase Islan, Bahamas, December [3] A. Stolyar, Maxmzng queueng network utlty subject to stablty: Greey prmal-ual algorthm, Queueng Systems, vol. 50, no. 4, pp , [4] M. Neely, E. Moano, an C. L, Farness an optmal stochastc control for heterogeneous networks, n Proceengs of IEEE Infocom, Mam, FL, March 2005, pp [5] A. Erylmaz an R. Srkant, Jont congeston control, routng an mac for stablty an farness n wreless networks, IEEE Journal on Selecte Areas n Communcatons, specal ssue on Nonlnear Optmzaton of Communcaton Systems, vol. 14, pp , August [6] X. Ln, N. B. Shroff, an R. Srkant, A tutoral on cross-layer optmzaton n wreless networks, IEEE Journal on Selecte Areas n Communcatons, specal ssue on Nonlnear Optmzaton of Communcaton Systems, vol. 14, no. 8, Aug [7] L. Georgas, M. Neely, an L. Tassulas, Resource allocaton an cross-layer control n wreless networks, vol. 1, no. 1, pp , [8] S. Shakkotta an R. Srkant, Network optmzaton an control, vol. 2, [9] L. Yng, S. Shakkotta, an A. Rey, On combnng shortest-path an back-pressure routng over multhop wreless networks, n Proceengs of IEEE Infocom, Ro e Janero, Brazl, Aprl 2009, pp [10] L. Bu, R. Srkant, an A. Stolyar, Novel archtectures an algorthms for elay reucton n back-pressure scheulng an routng, n Proceengs of IEEE Infocom, Ro e Janero, Brazl, Aprl 2009, pp [11] G. Gupta an N. Shroff, Delay analyss for mult-hop wreless networks, n INFOCOM The 28th Conference on Computer Communcatons. IEEE, Aprl 2009, pp [12] K. Kar, X. Luo, an S. Sarkar, Delay guarantees for throughput-optmal wreless lnk scheulng, n INFOCOM The 28th Conference on Computer Communcatons. IEEE, Aprl 2009, pp [13] M. Neely, Delay analyss for maxmal scheulng n wreless networks wth bursty traffc, n INFOCOM The 27th Conference on Computer Communcatons. IEEE, Aprl 2008, pp [14], Dynamc power allocaton an routng for satellte an wreless networks wth tme varyng channels, Ph.D. ssertaton, Massachusetts Insttute of Technology, LIDS, [15] L. Klenrock, Queueng Systems, Volume 2: Computer Applcatons. New York: Wley Interscence, [16] H. Xong, R. L, A. Erylmaz, an E. Ekc, Delay-aware cross-layer esgn for network utlty maxmzaton n mult-hop networks, 2010, techncal Report, avalable onlne at Ruogu L (S 10) receve hs B.S. egree n Electronc Engneerng from Tsnghua Unversty, Bejng, n He s currently a PhD stuent n Electrcal an Computer Engneerng at The Oho State Unversty. Hs research nterests nclue optmal network control, wreless communcaton networks, low-elay scheulng scheme esgn an cross-layer algorthm esgn. Atlla Erylmaz (S 00-M 06) receve hs B.S. egree n Electrcal an Electroncs Engneerng from Boḡazç Unversty, Istanbul, n 1999, an the M.S. an Ph.D. egrees n Electrcal an Computer Engneerng from the Unversty of Illnos at Urbana-Champagn n 2001 an 2005, respectvely. Between 2005 an 2007, he worke as a Postoctoral Assocate at the Laboratory for Informaton an Decson Systems at the Massachusetts Insttute of Technology. Snce 2007, he s an Assstant Professor of Electrcal an Computer Engneerng at The Oho State Unversty. He receve the NSF-CAREER an the Lumley Research Awars n 2010, an hs research nterests nclue: esgn an analyss for communcaton networks, optmal control of stochastc networks, optmzaton theory, strbute algorthms, stochastc processes, an nformaton theory. Eylem Ekc receve the BS an MS egrees n computer engneerng from Boḡazç Unversty, Istanbul, Turkey, n 1997 an 1998, respectvely, an the PhD egree n electrcal an computer engneerng from Georga Insttute of Technology, Atlanta, n Currently, he s an assocate professor wth the Department of Electrcal an Computer Engneerng, The Oho State Unversty. Hs current research nterests nclue cogntve rao networks, nano-scale networks, vehcular communcaton systems, an wreless sensor networks, wth a focus on routng an meum access control protocols, resource management, an analyss of network archtectures an protocols. He s an assocate etor of the IEEE/ACM Transactons on Networkng, Computer Networks Journal (Elsever), an the ACM Moble Computng an Communcatons Revew. He s a member of the IEEE.