IEEE TRANSACTIONS ON MOBILE COMPUTING, VOL. 13, NO. X, XXXXX To Stay or To Switch: Multiuser Multi-Channel Dynamic Access

Transcription

1 RANSACIONS ON MOBILE COMPUING, VOL. 13, NO. X, XXXXX o Stay or o Swtch: Mulser Mult-Channel Dynamc Access Yang Lu and Mngyan Lu, Fellow, Abstract In ths paper we sdy oppornstc spectrum access (OSA) polces n a mulser mult-channel random access cogntve rado network, where users perform channel probng and swtchng n order to obtan better channel condton or hgher nstantaneous transmsson qualty. Pror sdes n ths area nclude those on channel probng and swtchng polces for a sngle user to explot spectral dversty, and those on probng and access polces for multple users over a sngle channel to explot temporal and mulser dversty. By contrast, n ths sdy we consder the collectve swtchng of multple users over multple channels. hs nevtably necesstates explct modelng of the effect of collson. Furthermore, we consder fnte arrvals, whereby users are not assumed to always have data to send and the demand for channel follows a certan arrval process. Under such a scenaro, the users ablty to oppornstcally explot temporal dversty (the temporal varaton n channel qualty over a sngle channel) and spectral dversty (qualty varaton across multple channels at a gven tme) s greatly affected by the level of congeston n the system. We nvestgate the assocated decson process n ths case, and show that the optmal polcy s gven by a nested stoppng rule whch may be vewed as a type of generalzaton to results found n exstng lterare n ths area. We analytcally and numercally evaluate the extent to whch congeston affects potental gans from oppornstc dynamc channel swtchng. Index erms Oppornstc spectrum access (OSA), cogntve rado network, dversty gan, mulser mult-channel system, optmal stoppng rule, nested stoppng rule 1 INRODUCION DYNAMIC and oppornstc spectrum access (OSA) polces have been very extensvely sded n the past few years for cogntve rado networks, aganst the backdrop of spectrum open access as well as advances n ever more agle rado transcevers, ncludng e.g., hghly effcent channel sensng technques [3], [16]. Wthn ths context, a cogntve rado s capable of quckly detectng spectrum qualty and performng channel swtchng so as to obtan good channel and transmsson qualty. At the heart of such oppornstc spectrum access s the dea of mprovng spectrum effcency through the explotaton of dversty. Wthn ths context there are three types of dversty gans commonly explored. he frst s temporal dversty, where the naral temporal varaton n the wreless channel causes a user to experence or perceve dfferent transmsson condtons over tme even when t stays on the same channel, and the dea s to have the user access the channel for data transmsson when the condton s good, whch may requre and warrant a certan amount of watng. Sdes lke [5] nvestgate the tradeoff nvolved n watng for a better condton and decdng when s the best tme to stop. he second s spectral dversty, where dfferent channels experence dfferent temporal varatons, so for a gven user at any gven tme a set of channels present dfferent transmsson condtons. he dea s then to have the user select a channel wth the best condton at any gven tme for data transmsson, he authors are wth the Department of Electrcal Engneerng and Computer Scence, Unversty of Mchgan, Ann Arbor, MI E-mal: {younglu, mngyan}@umch.edu. Manuscrpt receved 27 Oct. 2013; revsed 16 Mar. 2014; accepted 19 June Date of publcaton ; date of current verson For nformaton on obtanng reprnts of ths artcle, please send e-mal to: reprnts@eee.org, and reference the Dgtal Object Identfer below. Dgtal Object Identfer no /MC Ç whch typcally nvolves probng multple channels to fnd out ther condtons. Protocols lke [10] does exactly ths, and sdes lke [2], [21] further seek to dentfy the best sequental probng polces usng a decson makng framework. he thrd s user dversty or spatal dversty, where the same frequency band at the same tme can offer dfferent transmsson qualtes to dfferent users due to ther dfference n transcever desgn, geographc locaton, etc. he dea s to have the user wth the best condton on a channel use t. hs dversty gan can be obtaned to some degree by usng technques lke stoppng rules whereby a user essentally judges for tself whether the condton s suffcently good before transmttng, whch comes as a byproduct of utlzng temporal dversty. We note that the above forms of dverstes are often sded n solaton. For nstance, temporal dversty s sded n a mulser settng but wth a sngle channel n [19], [22]; spectral dversty s analyzed for a sngle user n [18], among others. More specfcally, [22] developed optmal stoppng polces for sngle-channel mulser access, whle an et al. [19] consdered a dstrbuted oppornstc schedulng problem for ad-hoc communcatons under delay constrants. In [18] authors exploted spectral dversty n OSA for a sngle user wth sensng errors, where the mult-channel overhead s capred by a generc penalty on each channel swtchng. hs becomes nsuffcent n a mulser settng as such overhead wll obvously depend on the level of congeston n the system whch results n dfferent amount of collson and the tme t takes to regan access to a channel. In [10] an oppornstc auto rate mult-channel MAC protocol MOAR s presented to explot spectral dversty for a mult-channel mult-rate enabled wreless ad hoc network. However, ths scheme does not allow parallel use of multple channels by dfferent users ß Personal use s permtted, but republcaton/redstrbuton requres permsson. See for more nformaton.

2 2 RANSACIONS ON MOBILE COMPUING, VOL. 13, NO. X, XXXXX 2014 due to ts reservaton mechansm. Other works that sdy mult-channel access for a sngle user nclude [2], [4], [5], [12], [13], [20]. As the number of users and ther traffc volume ncrease n such a mult-channel system, one would expect ther ablty to explot the above dversty gans to decrease sgnfcantly due to the ncreased overhead, e.g., the tme t takes to perform channel sensng or the tme t takes to regan access rght, or ncreased collson due to channel swtchng. As mentoned above, ths overhead was capred n the form of penalty cost n pror work such as [18], but s often assumed to be ndependent of the traffc volume exstng n the system. Compared to the above lterare, the man contrbuton of ths paper s two-fold: 1) We present a model that capres oppornstc spectrum access polces n a mulser mult-channel random access settng, where users are not assumed to always have data to send, whle demand for channel follows a certan arrval process, and delays due to collson and contenton are taken nto account. We then set out to nvestgate the assocated optmal decson process n ths scenaro, assumng each user follows a random sensng order. 1 We then focus on the collectve effect of channel swtchng decsons by the users, and how ther decson processes are affected by ncreasng congeston levels n the system. 2) For ths problem we characterze the nare of an optmal access polcy and dentfy condtons under whch channel swtchng acally results n transmsson gan (e.g., n terms of average data rate or throughput). We show that the optmal polcy s gven by a nested stoppng rule nvolvng a two-step stoppng decson, whch may be vewed as a type of generalzaton to those found n the lterare and mentoned above, e.g., [22]. We also show both analytcally and numercally that, unsurprsngly, wth the ncrease n user/data arrval rate, the average throughput decreases and a user becomes ncreasngly more reluctant to gve up a present transmsson oppornty n hopes for better condton later on or n a dfferent channel. he remander of ths paper s organzed as follows. he system model s gven n Secton 2. In Sectons 3 and 4, we model channel evoluton as IID and Markovan processes, respectvely, and analyze the propertes of an optmal stoppng/swtchng rule. Numercal results are gven n Secton 5, and Secton 6 concludes the paper. 2 MODEL,ASSUMPIONS AND PRELIMINARIES 2.1 Model and Assumptons Consder a wreless system wth N channels ndexed by the set V ¼f1; 2;...;Ng. We assocate each channel wth a postve reward of transmsson (e.g., transmsson rate) X j, whch s a postve random varable wth dstrbuton gven by f X jðxþ, assumed to have fnte support wth a 1. We dscuss n much greater detal the choce of random sensng order versus optmal sensng order n Secton 2.4. maxmum value of X j. here are m cogntve users (or radotranscevers),eachequppedwthasngletransmtter attemptng to send data to a base staton. Our model also capres drect peer-to-peer communcaton, where m pars of users communcate and each par can rendezvous and perform channel sensng and swtchng together through the use of a control channel [14]. However, for smplcty of exposton, for the rest of the paper we wll take the vew of m users transmttng to a base staton. We wll assume these m users are wthn a sngle nterference doman, so that at any gven tme each channel can only be occuped by one user. Consderng spatal reuse wll make the problem consderably more challengng and remans an nterestng drecton of fure research. We consder dscrete tme wth a sutably chosen tme unt, and wth all other tme values nteger multples of ths underlyng (and possbly very small) unt. We wll consder two channel models, an IID model where channel condtons over tme are assumed to form an IID process defned on ths tme unt (n Secton 3), and a Markovan model where channel condtons over tme form a Markov chan (n Secton 4). Dfferent channels are n general not dentcally dstrbuted but are assumed to evolve ndependent of each other. Strctly speakng an IID process s a specal case of a Markov process. he purpose for makng ths dstncton s to use the IID model to represent a fastvaryng channel whle usng the Markov model for a slow-varyng channel. he system operates n a way smlar to a mult-channel random access network lke , wth the followng modfcatons related to dynamc and oppornstc channel access. Each user has a pre-assgned (or self-generated) random sequence of channels; ths sequence determnes n whch order the user performs channel swtchng, an approach smlar to that used n [18]. More on ths assumpton s dscussed n Secton 2.3. Each tme a user enters a new channel, t must perform carrer sensng (CS) and compete for access (contenton resoluton) as n a regular channel. As soon as t gans the rght to transmt, the user reserves the channel (e.g., through the use of RS-CS type of handshake) and fnds out the nstantaneous data transmsson qualty (channel nformaton could be pggybacked on these control packets) t may get f t transmts rght away. Upon fndng out the channel condton, ths user faces the followng choces: 1) ransmt on the current channel rght away. Intvely ths happens f the current channel condton s deemed good enough. hs acton wll be referred to as SOP. hs s shown n Fg. 1, where the second RS-CP (denotng the Reservaton-Channel probng process) followed by DAA ndcates a SOP at the frst channel (frst lne n the fgure). 2) Forego ths transmsson oppornty, presumably due to poor channel condton, but reman on the same channel and compete for access agan n the near fure hopng to come across a better condton then. hs happens f the current channel condton s poor but the average qualty s beleved to be good, so the user wll rsk watng for possbly better condton later. hs acton wll be referred to as SAY. hs s

3 LIU AND LIU: O SAY OR O SWICH: MULIUSER MULI-CHANNEL DYNAMIC ACCESS 3 Fg. 1. System model. llustrated by the frst RS-CP on the frst lne (channel) followed by a horzontal arrow. 3) Gve up the current channel and swtch to the next one on ts lst/sequence of channels. hs happens f the current channel condton s poor, and the prospect of gettng better condtons later by stayng on the same channel s not as good as by swtchng to the next channel. hs acton wll be referred to as SWICH. An example s shown by the RS-CP on the second lne (channel) followed by a vertcal arrow ndcatng a SWICH acton. Note that opton (2) above allows the system to explot both mulser dversty (the transmsson oppornty s gven to another user under the random access) and temporal dversty (the user n queston wats for better condton to appear n tme), whle opton (3) allows the system to explot spectral dversty as users seek better condtons on other channels. Optons (1) and (2) are smlar to those used n exstng stoppng tme frameworks, see e.g., [22]. In the above decson process once a user decdes to leave a channel t cannot use the channel for transmsson wthout gong through carrer sensng and random access competton agan. More mportantly from a techncal pont of vew, ths assumpton means that the user cannot clam the same channel condton at a later tme. Once a user gets the rght to transmt on a certan channel, t can transmt for a perod of tme unts, whch s a constant. For smplcty a sngle tme unt s assumed to be the amount of tme to transmt a control packet (e.g., RS/CS type of packets.). 2.2 Caprng the Level of Congeston As mentoned earler our focus n ths paper s on understandng how the users channel access decson process s affected by ncreasng traffc load or congeston n the system. o model ths we wll frst take the vew of a sngle user, and ntroduce user arrval rates n each channel as well as the amount of delay nvolved n SAY and SWICH as parameters that need to be taken nto consderaton n ts decson process. Note that these parameter values are the result of the collectve swtchng actons of all users, and therefore cannot be obtaned pror to defnng the swtchng polces. We wll however assume that these parameters have well-defned averages to facltate our analyss. Later we show that the system under the optmal swtchng polcy converges and that these parameters ndeed have welldefned averages, thereby justfyng such an assumpton. In other words, polces derved under the assumpton that these parameters have well-defned averages lead to a stable system wth well-defned averages for these parameters. hs s not unlke a mean-feld approach where a sngle user operates aganst a background formed by other users n a system over whch ths sngle user has no control or nfluence. In practce these values may be obtaned through learnng. Specfcally, we assume that the total packet arrvals to a channel, ncludng external arrval, retransmsson, as well as arrvals swtched from other channels, form a Posson process, wth the attempt rate vector gven by G ¼½G 1 ;G 2 ;...;G N Š and a sum rate P N ¼1 G ¼ G. hese quanttes wll also be referred to as the load or traffc load on a channel. We wll not drectly deal wth the external arrval processes as our analyss entrely depends on the above nternal offered load. However, we wll assume that the external arrvals are such that the system remans stable. he level of congeston on any channel s capred by two parameters. he frst s an average contenton delay on channel j denoted by t c j ; ths s the average tme from carrer sense to ganng the rght to transmt on channel j. he more competng users there are on channel j, the hgher ths quantty s. he second s an average swtchng delay of channel j, denoted by t s j ; ths s the tme from a user swtchng nto channel j (from another channel) to ts ganng the rght to transmt on channel j. Compared to t c j the swtchng delay ncludes the addtonal tme for the rado to perform channel swtchng and addtonal watng tme n the event that the swtchng occurs durng an actve transmsson. In our characterzaton of t s j below, however, we wll gnore the hardware swtchng delay as t smply adds a constant, whch s very small compared to contenton delay, and wll not affect our subsequent analyss. For a packet arrvng at channel (from an external arrval process or by swtchng from another channel), the delay t experences between arrval and successful transmsson conssts of two parts, the average tme t takes for the channel to become dle f t happens to arrve durng an actve transmsson (ncludng ts assocated control packet exchange), denoted by t w, and the average tme t takes to compete for and gan the rght to transmt, gven by t c.we thus have t s ¼ tw þ t c. Denote by Y the random varable representng the tme between a new arrval and the completon of the current transmsson. Followng results n [14], we have f Y ðyþ ¼ S e Sy, where S s the success rate of channel contenton gven by

4 4 RANSACIONS ON MOBILE COMPUING, VOL. 13, NO. X, XXXXX 2014 t w S ¼ s then calculated as follows: t w Z 1þ G e 2G 1 þð1 þ ÞG e 2G : (1) ¼ f Y ðyþð1=z þ yþdy 0 ¼ 1 þ 1 S z þ 1 þ 1 þ 1 e ðþ1þs ; (2) S z where 1=z s the expected random backoff tme. For t c, snce a competton succeeds wth probablty e 2G we have t c ¼ðe2G 1Þð1=z þ 2Þþ2: (3) we wll seek to maxmze the rate of rern over the remanng decson process gven the current state of the process. hs may be vewed as a no-recall approxmaton to the orgnal goal by gnorng the hstory or past decsons n the same process. hs objectve can be represented by the followng dynamc program, notng that the user goes through the channels n the order 1; 2;...;N V N ðxþ ¼max x; þ t c EfV N ðx N j xþg ; N V ðxþ ¼max x; þ t c EfV ðx Þjxg; (4) þ t s EfV þ1 ðx þ1 Þjxg ; < N; þ1 Usng the above expressons, t s not dffcult to establsh the followng results. Proposton 2.1. Both t c j and ts j are non-decreasng functons of arrval rate G j, 8j 2 V. Proposton 2.2. Both t c j and ts j are non-decreasng functons of the data transmsson tme, 8j 2 V. he decson process we ntroduce next s a functon of t c and t s, so a user needs to know these parameter values n order to compute the optmal polcy. In practce, ths nformaton may be obtaned through measurement and emprcal means. 2.3 Problem Formulaton For smplcty and wthout loss of generalty, for the sngle user under consderaton we wll relabel the channels n ts sequence n the ascendng order: 1; 2;...; N. We now defne the followng rate-of-rern problem wth the objectve of maxmzng the effectve data rate over one successful data transmsson. Specfcally, let p denote a polcy p ¼fa 1 ; a 2 ;...a gðpþ g whch specfes the sequence of actons leadng up to a successful transmsson, wth a k denotng the kth acton, a k 2fSAY; SWICHg, k ¼ 1;...; gðpþ1, and a gðpþ ¼ SOP. An acton s only taken upon ganng the rght to transmt n a channel, and gðpþ denotes the stoppng tme at whch the process termnates wth a transmsson acton. Let XgðpÞ p denote the data rate obtaned at the last step when the process termnates. hen the total reward the user gets s XgðpÞ p, the total amount of data transmtted. A naral goal would be to maxmze the rato between ths reward and the total amount of tme spent n the decson process (summng up the delays nvolved n swtchng and contenton as a result of the actons),.e., the effectve or average throughput or data rate. Whle ths appears to be a standard rate-of-rern problem, an nherent dffculty arses from the fact that dfferent channels have dfferent statstcs, and thus the rewards generated and the delays experenced, respectvely, are not ndependent across channels. hs prevents the use of the renewal theorem to rn the expectaton of the aforementoned rato (average throughput) nto a rato of expectatons as s commonly done. o address ths dffculty, we wll make the followng smplfcaton: nstead of maxmzng the overall rate of rern for each successful transmsson over the entre decson process, where V ðxþ s the value functon at stage (n channel ) of the decson process when the observed channel state s x; ths s also the maxmum average throughput obtanable gven current state x (transmsson rate) n channel : In the above equaton, the frst term s the reward (current transmsson rate) f we SOP, the second the expected reward f we SAY, and the last the expected reward f we SWICH. 2.4 Crtque on the Model he model gven above capres the mulser, multchannel, and random access nare of the problem. he optmal decson process defned by (4) appears to be a fnte horzon problem,.e., the process stops at channel (or stage) N. However, ths would only be partly true, as (4) acally llustrates a two-dmensonal decson problem, where there s a fnte number of steps (N) along the spectral dmenson (the channels), but wthn each channel (for each ) the decson process s over an nfnte horzon along the tme dmenson,.e., the decson process may go on ndefntely wthn a partcular channel. hs wll be seen more clearly n Secton 3. he reason we have lmted the horzon to be fnte along the spectral dmenson the nfnte horzon verson would be where the user can contnue to swtch channels for an ndefnte number of tmes, ncludng revstng channels t has vsted n the past has to do wth the IID assumpton on the channels. Snce channel state realzatons are ndependent over tme (for the same channel), the second and thrd terms n (4) are both ndependent of the current state x. In other words, the comparson between the second and the thrd terms s ndependent of the current state x, suggestng that under the same contenton level G f the second term s larger than the thrd term, then t wll always be larger regardless of the current state. he nterpretaton of ths observaton s that f we ever decde to SAY (the second term s larger) on the same channel, then we wll never SWICH later. he opposte s also true: f we ever decde to SWICH away from a channel (the thrd term s larger), then under the optmal polcy we wll never come back to the same channel even f we are allowed to. hs means that under the objectve of maxmzng the fure rate of rern, a channel s never vsted more than once, resultng n the fnte horzon formulaton along the spectral dmenson gven above. In other words, there s no need to allow the user to revst a channel t has vsted before but swtched away from.

5 LIU AND LIU: O SAY OR O SWICH: MULIUSER MULI-CHANNEL DYNAMIC ACCESS 5 he reason why we lmt the user to a pre-determned sequence (randomly chosen) of channels has to do wth the mulser scenaro we am to analyze. If there s only a sngle user, then obvously a reasonable thng to do s to also optmze the sequence/order of channel sensng, together wth optmzng the swtchng and transmsson decsons. Indeed there has been a large volume of sdy on determnng optmal sensng orders, see e.g., [5], [6], [7], [9], wth the man dea beng that upon swtchng, a user should swtch to a channel beleved to present the best transmsson oppornty. A related problem s channel assgnment, see e.g., [1] that sded such a problem under stochastc uncertanty and wth adjacent channel nterference. However, contenton among users s not taken nto consderaton n [1]; furthermore, to acheve globally optmal assgnment the approach ether assumes statc assgnment that does not respond to random realzaton of channel condtons, or employs a central controller. A two-user model n a smlar context was ntroduced and analyzed n [7], but beyond two users the problem remans open. Compared to [1], the contenton and the oppornstc explotaton of tme-varyng channel condtons are key aspects of the model we sdy n ths paper. An optmal sensng order becomes elusve n a mulser settng because the above type of optmzaton reles on known statstcs of the channel dynamcs. However, ths s no longer applcable when there are multple competng users: one s prevously optmal sensng order may no longer be optmal dependng on what order the other users adopt. Consequently ths needs to be ether treated as a centralzed mulser optmzaton problem, where the jontly optmal sensng orders are computed smultaneously for all users, or treated as a game-theoretc problem where each user selfshly determnes ts sensng order to maxmze ts own utlty. A recent sdy [17] adopts an approach close to the frst one wth a jont desgn framework of sensng order and channel swtchng decson. he model n [17] however focuses on the nteracton between a secondary user and a prmary user, rather than on the contenton relatonshp among competng secondary users (whch our model capres), thus t does not jontly desgn channel swtchng decsons for multple users. he second, game-theoretc approach s largely an open area as t nvolves the equlbrum analyss of complex decsons (not only the sensng order of channels but also the stoppng decsons on any gven channel). Whle ths remans an nterestng drecton of fure research, n the present sdy we adopt the assumpton that a user smply follows a pre-defned (can be randomly chosen) sequence of channels and focus our attenton on the swtchng decsons nstead. In Secton 5 we compare the results between randomly selectng these sequences and users optmally selectng these sequences ndvdually wthout consderng other users behavor. For the remander of our presentaton, we wll use the terms stages and steps to descrbe the two tme scales of decson makng along the two dmensons descrbed above. Movement along the spectral dmenson (.e., swtchng from one channel to the next) occur n stages; stage means channel and ths s ndexed by the subscrpt n the value functon V ðxþ. he decson process wthn the same stage (or n the same channel) occurs n steps; the decson to reman on the same channel or swtch away occurs at the boundary of a step. he ndexng of steps s not explct n the expresson gven n (4) but wll be made explct n our subsequent analyss. 3 OPIMAL ACCESS POLICY UNDER HE IID CHANNEL MODEL In ths secton, we model the channels as fast changng, IID processes, where successve observatons of the state of the same channel are ndependent. 3.1 An Optmal Nested Stoppng Rule Snce successve channel states are ndependent, the value functon (4) s smplfed: V ðxþ ¼max x; þ t c EfV ðx Þg; þ t s þ1 EfV þ1 ðx þ1 Þg : he above three-way comparson suggests the followng. If the current state x s suffcently hgh then the optmal decson s SOP. he comparson between the second and the thrd terms s more nterestng: both terms are ndependent of the state x, so f the second term s larger then t wll always be larger. As prevously mentoned, ths mples that f we ever decde to SAY, then we wll never SWICH later. he reverse s also true: f we ever decde to SWICH then we wll never rern to the same channel. hese observatons can lead to a concrete proof of the exstence and unqueness of a threshold rule but n general cannot produce a closed form for the computaton of the threshold. Below we wll nstead use results from optmal stoppng theory [8] to obtan not only the exstence but also a closed form for the threshold. Consder the followng substton, ^X ðxþ ¼max x; þ t s EfV þ1 ðx þ1 Þg (6) þ1 wth the value functon subsequently re-wrtten as V ðxþ ¼max ^X ðxþ; þ t c EfV ðx Þg : (7) hs substton reduces the decson process to a twoway comparson, and more mportantly, a one-dmensonal decson process. Specfcally, snce the state x s IID over the same channel/stage, the frst term ^X ðxþ as defned n (6) s also IID over the same stage whle encodng the nformaton on other channels/stages. herefore, f we vew ^X ðxþ as the reward of a (meta) stoppng acton of state x and t c as the cost for contnung, then the value functon gven n (7) represents a standard stoppng tme rate-ofrern problem wth two possble actons n each step, (meta) stoppng and contnuaton, respectvely, and ths process concerns only a sngle stage/channel. he swtchng to the next stage occurs when the (meta) stoppng acton s taken (whch essentally ends the above one-dmensonal stoppng tme problem), and t s determned that SWICH s a better acton than SOP. (5)

6 6 RANSACIONS ON MOBILE COMPUING, VOL. 13, NO. X, XXXXX 2014 he followng theorem characterzes the property of the optmal decson for the problem gven n (5) or equvalently (7). heorem 3.1. he optmal acton at stage of decdng between {SOP, SWICH} and SAY s gven by a stoppng rule: the state space of the channel condton can be dvded nto a stoppng set D s and contnuaton set D c, such that whenever the channel condton s observed to be n ether set, the correspondng acton (SOP/SWICH versus. SAY) s taken. 2 Furthermore, these two sets are gven by the followng threshold property: D s ¼fx : ^X ðxþ g; 8; (8) where the threshold at the th stage s gven by the unque soluton to E½ ^X ðxþš þ ¼ tc : (9). We frst prove the exstence of an optmal stoppng rule. Defne the reward functon assocated wth step k of the stoppng decson process at stage as Z k ð; xþ ¼ ^X ðxþ k t c þ ; (10) where s a postve fnte valued varable. From [heorem 1, Chapter 3, 8] we know that an optmal stoppng rule exsts f the followng two condtons are satsfed 3 : n o ðc1þ E sup Z k ð; X Þ < 1; k ðc2þ lm Z k ð; X ÞZ 1 ð; X Þ;a:s: k!1 (11) Snce we have a fnte number of channels and the channel state realzaton s fnte, ^X ðxþ s fnte. herefore Z 1 ð; X Þ¼1. Snce ^X ðxþ, þt s EfV þ1 ðx þ1 Þg and þ1 are all fnte, (C2) s easly satsfed. Next defne Z ð; xþ ¼ ^X ðxþ ; (12) whch s agan fnte. herefore we have EfZ ð; X Þg < 1, and EfðZ ð; X ÞÞ 2 g < 1. Also notng that Z ð; X Þ s IID snce X s IID, by the domnated convergence theorem we have Efsup k Z kð; X Þg < 1, verfyng (C1). he exstence s thus establshed. Next we prove that the optmal stoppng rule s gven by a threshold. Usng the prncple of optmalty [Chapter 2, 8] and the results from [Secton 4.1, 8] (we refer the reader to [Example 6.2, 8] for further detal), our problem as expressed n (7) s equvalent to a rate-of-rern problem wth a reward of stoppng gven by Z ð; xþ and a cost of contnuaton gven by t c. he optmal stoppng rule at step k s gven by 2. he word contnuaton n ths context refers to contnung on the same channel, whereas stoppng (or the term (meta) stoppng used earler) refers to no longer stayng on the same channel ether by a transmsson or by swtchng away. 3. he nterpretaton of these two condtons s that even f we know the fure the maxmum expected reward, or the reward approachng the supremum, s fnte. D s ¼fx : Z ð ;xþ0g ¼fx : ^X ðxþ g; (13) where s such that the functon Vk ðþ, defned recursvely as [Chapter 6, 8] Vk ðþ ¼EfmaxfZ ð; xþt c ; Vk ðþtc gg, s evaluated to be zero,.e., Vk ð Þ¼0. o obtan, we take Vk ð Þ¼0 nto the above defnton and get EfmaxfZ ð; xþ; 0gg ¼ t c, or equvalently, s such that t satsfes E½ ^X ðxþ Š þ ¼ t c ; (14) whch s the same as (9). hs completes the proof of the form of the threshold. It remans to show that a unque soluton exsts to (9). Denote by DðÞ ¼E½ ^X ðxþš þ t c. It s not hard to verfy that DðÞ s a contnuous and strctly decreasng functon of. Furthermore, we have Dð ¼ 0Þ ¼E½ ^X ðxþš þ > 0 snce all channel states are postve, and DðÞ!1as!1. herefore there s a unque soluton to DðÞ ¼0,.e., the threshold exsts and s unque, completng the proof. In practce, to calculate ths threshold, we defne c ¼ þ t s EfV þ1 ðx þ1 Þg: (15) þ1 Re-wrtng (9) n the orgnal random varables, we have E max X þ ; þ t s EfV þ1 ðx þ1 Þg þ1 ¼ E maxfx ; 0gjX >c P ðx >c Þ (16) þ E maxfc ; 0gjX c P ðx c Þ ¼ t c : If the soluton <c, then t has to satsfy R X c ðx Þ f X ðxþdx þðc ÞPðX c Þ¼t c =, and thus can be R X obtaned by xf c ¼ X ðxþdx þ c P ðx c Þ 1 þ t c = and verfyng that the resultng <c. If the soluton c, then t must satsfy R X ðx Þf X ðxþdx ¼ tc =, and the soluton may be obtaned usng ¼ resultng c. R X xf X ðxþdx PðX Þþt c = and verfyng4 that the Remark 3.2. he quantty c defned above s the expected reward of SWICH, whle s the threshold for makng a decson between the set {SOP, SWICH} and SAY. he optmal polcy gven n the above theorem s llustrated n Fg. 2, whch can be vewed as a sequence of two YES/NO questons used n decson makng nvolvng two thresholds. (1) If <c, then the optmal decson s ether SOP or SWICH dependng on whether x>c. If the current condton s very good (x >c ) then the decson s SOP; otherwse SWICH. In ths case the reward from swtchng s suffcently good that we wll never consder SAY. (2) If >c, then the optmal decson s ether SOP or SAY dependng on whether 4. hs functon s a fxed pont equaton whch could be solved by teratve methods as n [22].

7 LIU AND LIU: O SAY OR O SWICH: MULIUSER MULI-CHANNEL DYNAMIC ACCESS 7 to use t for a longer perod of tme. However, practcally cannot be made too large due to the channel coherence tme: the channel condton wll lkely change over a large perod. Fg. 2. Illustraton of the decson process. x>. In ths case the reward from swtchng s nferor so that SWICH s not an opton. hs polcy wll be referred to as a nested stoppng polcy. 3.2 Propertes of the Nested Stoppng Polcy We next nvestgate a number of propertes of the mulser mult-channel system as a result of the above nested stoppng polcy. We start by examnng ts effect on the traffc load vector G. Unless otherwse noted, all proofs can be found n the appendx. Lemma 3.3 (Monotoncty of the value functon). Consder two traffc load vectors G and G 0 where G G 0 ; 8 2 V. Denote the correspondng sets of value functons by V and V 0, respectvely. hen we have EfV gefv 0 g; 8 2 V. hs lemma conveys the nton that when the load ncreases, competton ncreases leadng to longer delays. hus the expected throughput decreases n general. We next establsh the stablty of the system under the nested stoppng polcy, startng wth an assumpton. Assumpton 1. No channel s domnant,.e., there s no sngle channel that wll attract all arrvals under the nested stoppng polcy. hs assumpton excludes the extreme case where a sngle channel s of far better qualty (e.g., very hgh data rate) that even consderng the cost n competton t s benefcal to always swtch to ths channel, regardless of the condtons observed n the other channels. Lemma 3.4 (Ergodcty of the arrval process). he arrval processes are ergodc under the nested stoppng polcy and Assumpton 1. Lemma 3.5 (Load balance). 0; 8 2 V under the nested stoppng polcy. In other words, f the total traffc load ncreases, the nput/load to each channel s non-decreasng. hs property combned wth the monotoncty (Lemma 3.3) leads to the followng stronger monotoncty result on the value functon; the proof s trval and thus omtted. Lemma 3.6 (Strong monotoncty). EfV g;2 V, are all nonncreasng functons of G. We next analyze the mpact of transmsson tme and address the queston whether by reservng more tme for a sngle transmsson users gan n average throughput. Lemma 3.7 (Impact of ). EfV g;2 V, are all non-decreasng functons of. hs result reflects the nton that once a user fnds a good transmsson condton, t s benefcal for t to be able 3.3 Dscusson he performance a user obtans as a result of the precedng decson process depends on the accuracy of ts measurement over the level of contenton n the system,.e., t c j and t s j. Fornately, the performance loss due to errors n these measurements can be bounded as shown below. For smplcty we have assumed that the errors across all measurements are unformly gven by a small quantty D; f the errors are dfferent for dfferent measurements, D may be taken as the maxmum measurement error. heorem 3.8. For a gven user, the change of ts value functon at the th stage (1 N) as a result of a small (D) change to t c ; t s s bounded as follows: jv ðt c þ D; t s þ DÞV ðt c ; t s Þj jdj XN X j¼ o2fs;cg þ t o 2 C j ; (17) j where C j s a postve constant. Our model so far has assumed that each user can only access a sngle channel at a tme. If parallel transmssons are enabled (e.g., as n an OFDM system), a sngle user can access multple channels smultaneously. Under the most relaxed settng, we can model each user as havng k ndependent antennas wth no nter-channel nterference. hs then allows us to model the decson process of each user as k (or any number k 0 k dependng on how many packets t has to transmt) separate decson processes, each beng the same as that presented earler n ths secton. In other words, wthout restrcton on the use of multple nterfaces, a sngle user s now equvalent to k dfferent users and the subsequent analyss wll reman the same. If the use of these antennas are more restrctve, e.g., that a user may only use these nterfaces concurrently, and that n dong so must access contguous blocks of channels, or that there s throughput loss due to smultaneous channel access, then the resultng decson process becomes qute dfferent and combnatoral. Specfcally, due to ths couplng, the resultng access decson s over bundles of channels rather than ndvdual channels. A user may have up to N k choces of such channel bundles. If the user can sense ether sequentally or smultaneously the channel condton n each channel wthn a bundle, t can then estmate the transmsson reward from usng ths bundle. Concepally, a smlar decson process can be formulated where the user try to decde whether to swtch to a dfferent bundle or use the current bundle for transmsson, or wat. A practcal dffculty, however, les n the random access nare of a channel: f the user needs to gan access n each channel n order to use the bundle then ths could ental very sgnfcant amount of delay (thus the cost of delayng or swtchng), unless the traffc s extremely lght. hs remans a very relevant and nterestng problem of fure research.

8 8 RANSACIONS ON MOBILE COMPUING, VOL. 13, NO. X, XXXXX OPIMAL ACCESS POLICY UNDER HE MARKOVIAN CHANNEL MODEL hs secton presents a parallel effort to the prevous secton, under the assumpton that the channel condtons evolve over tme as a Markov chan. ABLE 1 Contenton Levels Load t c t s Unqueness of the Optmal Strategy Denote the state space of channel by S, and the sngle-step (over one unt of tme) state transton probablty by P ðy j xþ, x; y 2 S. he k-step transton probablty s denoted by P k ðy j xþ. he value functon representng the maxmum average throughput gven the current condton at stage s gven by the followng: ( V ðxþ ¼max ^X ðxþ; t c þ X ) P tc ðy j xþv ðyþ ; (18) where ^X ðxþ follows the same defnton as n the IID case. We make the followng approxmaton. When 1= s suffcently small, 5 we have t c þ ¼ 1 1 þ t c = ð 1 1 þ 1= Þtc. Denote b ¼ 1 1þ1= and we arrve at the followng approxmated value functon ( ) V ðxþ ¼max ^X ðxþ; b tc X P tc ðy j xþv ðyþ : (19) Denote by U¼fS; Cg the set of two actons, stoppng and contnuaton, where the stoppng acton S bundles SOP and SWICH nto a sngle acton,.e., S ¼fSOP; SWICHg due to the defnton of ^X ðxþ and as n the IID case, and the contnuaton acton C ¼fSAYg. hen the above can be re-wrtten as ( ) V ðxþ ¼max u2u rðu; xþþb tc X P u;tc ðy j xþv ðyþ ; (20) where rðs;xþ¼ ^X ðxþ, rðc;xþ¼0, P S;tc ðy j xþ ¼0, and P C;tc ðy j xþ ¼P tc ðy j xþ. heorem 4.1. he set of Equaton (19) or equvalently (20) have a unque soluton. Our proof s based on the contracton mappng theorem [11] and the next lemma. Lemma 4.2. Let F be the class of all functons v : f1; 2;...; Sg!R. Defne norm jjvjj :¼ P x2s jvðxþj and a mappng : F!Fby ( ) ð vþðxþ :¼ max u2u rðu; xþþh X vðyþp u ðy j xþ 0 < h < 1; then s a contracton. he next result also mmedately follows; the proof s omtted for brevty. 5. hs s possble snce s an nteger multple of an arbtrary tme unt, whch can be made very small. he only restrcton s that we have taken a sngle tme unt to be the tme t takes to transmt a control packet, so ths assumpton smply mples that a data transmsson s much longer than a control transmsson, whch s typcally true. ; Corollary 4.3 (hreshold polcy). he optmal stoppng rule reduces to a threshold polcy. Remark 4.4. As may be expected, ths threshold polcy works n a way very smlar to the IID case (only the numercal calculaton dffers): at stage/channel, there s a SWICH reward c (expected throughput by swtchng away from ) and by stayng on the same channel. he optmal decson s then based on the relatonshp between and c. 4.2 Propertes of the Nested Stoppng Polcy We can smlarly obtan a number of propertes for the mulser mult-channel system as a result of the nested stoppng polcy under the Markovan model. heorem 4.5 (Monotoncty). EfV g;2 V are all nonncreasng functons of G. Followng the above result we can derve smlar propertes of the nested stoppng polcy n the Markovan case as n the IID case, ncludng ergodcty of the arrval processes, load balance and the non-ncreasng value functons n. he proof of these are omtted for brevty and due to ther smlarty to those n the IID case. 5 NUMERICAL RESULS 5.1 he IID Channel Model We frst consder a scenaro of fve ndependent channels wth ther channel condton (taken to be the nstantaneous transmsson rate measured n bytes per tme unt) exponentally dstrbuted over a fnte range, wth average rates gven by f1=0:4; 1=0:6; 1=0:5; 1=0:3; 1=0:2g. A sngle transmsson perod s set to ¼ 40 tme unts. he level of contenton/ congeston measured by t c and ts (measured n tme unts) as a functon of load G (measured n packet per unt tme) s llustrated n able 1 for channel 1. hese quanttes are rounded off to the nearest ntegers when used n computng the optmal polcy. We set packet length to be 1,024 Bytes. In Fg. 3 we compare the nested stoppng polcy wth the followng three schemes, by measurng the average throughput across all channels. 1) A standard random access polcy n whch a user randomly selects a channel to use, followed by competng for channel access usng type of random access scheme. 2) A stoppng rule based random access polcy over temporal dversty (denoted emporal Dversty n the fgure) ntroduced n [19], [22]. In ths case each user s randomly assgned a channel, and follows a stoppng rule on that channel (between usng the channel now or at a later tme).

9 LIU AND LIU: O SAY OR O SWICH: MULIUSER MULI-CHANNEL DYNAMIC ACCESS 9 Fg. 3. Performance comparson: Exp. Fg. 4. ransmsson rate w.r.t. G. 3) A stoppng rule based random access polcy over spectral dversty (denoted Spectral Dversty n the fgure) ntroduced n [10], [18] where a user sequentally sense condtons over multple channels to decde whch channel to use for transmsson. 6 Fg. 3 shows that our nested stoppng polcy clearly outperforms the others. he performance gan s more promnent when the load s lght. hs s to be expected because when there s lght congeston, watng for better condton or swtchng to another channel both ncur low overhead; when there s heavy congeston a user becomes more and more reluctant to wat or swtch channels thereby underutlzng both types of dversty. Fg. 4 shows the dampenng effect of ncreased load G on each channel separately. Fg. 5a shows that the throughput ncreases n the data transmsson tme as we have characterzed, but ths ncrease becomes slower snce ncreasng also ncreases the cost n channel releasng and swtchng. Fg. 5b shows that the throughput also ncreases n the number of channels (the smulaton s done by addng channels wth same statstcs as gven for the ntal fve), as the contenton n each channel reduces. Next n able 2 we show the optmal decsons table for the optmal actons condtoned on contnuaton (SAY or SWICH) for each channel (n ths specfc experment we consder a user starts from channel 1). As can be seen, channels 2 and 3 are of low qualty so the general decson s to swtch away rather than watng on the same channel 6. here are dfferences between these two references: [18] models sensng error and derves more strucral propertes, but the man dea s the same. Fg. 5. Performance under the IID channel model. ABLE 2 Decson of IID Channels wth Dfferent Arrval Rate Load Ch 1 Ch 2 Ch 3 Ch 4 Ch SAY SWICH SWICH SWICH SAY 0.1 SAY SWICH SWICH SAY SAY 0.3 SAY SWICH SWICH SAY SAY 0.5 SAY SWICH SWICH SAY SAY f the decson s not to transmt mmedately. For channel 4, we see that the tendency to stay ncreases when the load s hgh due to the hgher cost n swtchng than stayng. he decson to stay n channel 1 s more nterestng: even though better average throughput may be obtaned n channels 4 and 5, the cost n dong so s consderable as t has to go through channels 2 and 3. By contrast, there s a SWICH decson n channel 4 even though channel 4 s on average a better channel than channel 1. We also consder a more practcal AWGN wreless channel model consderng both propagaton loss and shadowng effects. he transmsson rates are gven by the Shannon capacty formula for AWGN channels: R ¼ logð1 þ r j hj 2 Þ nats/s/hz, where h denotes the random channel gan wth a complex Gaussan dstrbuton. Moreover, the cdf of transmsson rate s gven by F R ðrþ ¼1 expð expðrþ1 r Þ;r 0. Consder a scenaro wth fve channels wth average SNR r gven by able 3. Fg. 6 shows the same performance comparson as before. Whle the nested stoppng rule contnues to

10 10 RANSACIONS ON MOBILE COMPUING, VOL. 13, NO. X, XXXXX 2014 ABLE 3 Parameter able Channels Ch1 Ch 2 Ch 3 Ch 4 Ch 5 r Fg. 7. Performance comparson: Markovan model. ABLE 5 Decson able for Markovan Channels wth Dfferent States Fg. 6. Performance comparson: AWGN. ABLE 4 Reward able for Markovan Channels wth Dfferent States States Ch 1 Ch 2 Ch 3 Ch 4 Chl outperform the other schemes, an nterestng observaton here s that the scheme based solely on temporal dversty also outperforms usng only spectral dversty and t has very smlar performance as the nested polcy. hs shows that due to the dynamc nare of AWGN channels, most of the gan s derved from explotng temporal dversty rather than spectral dversty. 5.2 he Markovan Channel Model We now smulate the nested stoppng polcy under a Markovan channel model. We model all fve channels state (agan taken to be the nstantaneous transmsson rate n bytes per tme unt) change as a brth-death chan wth fve states and the assocated transton probabltes gven as follows: P k ðmnf þ 1; 5gjÞ ¼0:8; P k ðmaxf 1; 1gjÞ ¼0:2; 1 5; 1 k 5: (21) For each channel the rewards ncrease n state ndces, and are gven n able 4. ransmsson tme s agan set to be ¼ 40 tme unts. he same performance comparson s shown n Fg. 7. akng ths result together wth prevous results under exponental and AWGN channel models, we observe somethng qute revealng. In Fg. 7 we see that the performance of the temporal and spectral schemes are reversed: explotng spectral dversty results n much hgher gan than only explotng temporal dversty. hs s because ths set of States Ch1 Ch 2 Ch 3 Ch 4 Ch 5 1 SWICH SAY SWICH SAY SAY 2 SWICH SAY SWICH SOP SAY 3 SWICH SOP SOP SOP SOP 4 SOP SOP SOP SOP SOP 5 SOP SOP SOP SOP SOP Markovan channels are relatvely slow-varyng n tme compared to the prevous models, thus stayng on the same channel watng for better condton becomes less benefcal, whle hoppng through channels seekng better condtons s more effectve. hese results show that, compared to explotng only one type of dversty, our polcy s very robust aganst dfferent dynamc propertes of the channels and can extract the largest amount of performance gan. he decson table n ths case s shown n able 5. Smlar observatons are made here: when the channel condton s good enough, the user would choose to transmt mmedately (SOP); the SWICH decson s assocated wth poor condtons and when a user hopes to get much better condtons n the next channel; the SAY decson s made on a reasonably good channel and when there s lmted prospect of gettng better condton n the next channel. 5.3 Channel Sensng Order and No-Recall Approxmaton We next examne the effect of selectng dfferent sequence of channels to use. As dscussed earler, wth multple users (m 2) t s very challengng to ether jontly determne optmal sensng orders for all users nvolved n a cooperatve settng, or determne the equlbrum sensng orders selected by selfsh ndvduals n a non-cooperatve settng. For ths reason n our analyss we have assumed that each user follows a fxed (whch can be randomly chosen) order. We now compare ths choce where each user randomly pcks a sequence wth an optmal sensng order where users sense channels ordered n an derved optmal sensng order [5] and make decsons on each channel accordng to the threshold decson derved n our paper; 7 as a result each 7. For certan channel qualty dstrbutons, e.g., exponental dstrbuton, the optmal sensng order s equvalent to a greedy sensng order.

11 LIU AND LIU: O SAY OR O SWICH: MULIUSER MULI-CHANNEL DYNAMIC ACCESS 11 Fg. 8. Channel sensng order comparson. Fg. 9. Performance of approxmaton model. user follows/cycles through the same sequence but the startng poston for each user s randomzed to avod synchronzaton. hs comparson s shown n Fg. 8 for the IID channel case; t s clear that t s far better for each user to sense n a dfferent order especally when the load s hgh. hs comparson also hghlghts some of the challenges mentoned earler n employng an optmal channel sensng order n a mult-user settng. Note that we have smulated a user-homogeneous envronment where all users perceve dentcal channel condtons. It s evdent that n ths case havng all users follow the same optmal sensng order derved for a sngle user (assumng t s the only user present) s not a good strategy, whle determnng jontly optmal sensng orders for all users s analytcally ntractable and computatonally prohbtve. In ths sense our polcy smply assumes randomzaton as a compromse to enable the mult-user sdy performed n ths paper. We end ths secton by nvestgatng the effect of the no-recall approxmaton ntroduced n Secton 2 and adopted n our analyss, by comparng t wth the exact optmal soluton. We show ths n the IID case n Fg. 9; we see that ths approxmaton has very lttle effect on the system performance. 6 CONCLUSION In ths paper we consdered the collectve swtchng of multple users over multple channels. In addton, we consdered fnte arrvals. Under such a scenaro, the users ablty to oppornstcally explot temporal dversty (the temporal varaton n channel qualty over a sngle channel) and spectral dversty (qualty varaton across multple channels at a gven tme) s greatly affected by the level of congeston n the system. We nvestgated the optmal decson process under both an IID and a Markovan channel models, and evaluate the extent to whch congeston affects potental gans from oppornstc dynamc channel swtchng. APPENDIX A PROOF OF LEMMA 3.3: MONOONICIY OF VALUE FUNCION IN G We prove ths by nducton. When ¼ N,.e., the last stage, we have N tc N ¼ R X N N ðx N Þf X N ðxþdx, where tc ¼ tc =. As t c N s a non-decreasng functon n G N, t s also non-decreasng n G. hus wth the ncrease n t c N, the soluton N cannot be ncreasng, provng that N s a non-ncreasng functon of G. Snce our value functons (EfmaxðX ; Þg) are non-decreasng functons of the thresholds s, we have now shown that they are non-ncreasng n G. Next assume the non-decreasng property holds for ¼ n þ 1;...;N 1. Consder ¼ n. We prove ths n the cases n <c n and n c n, respectvely. For the case n c n, we have n tc N ¼ R X n ðx n n Þf XnðxÞdx. Usng smlar argument as n the case ¼ N we know n s non-ncreasng n G. For the R n X case n <c n, n ¼ xf c X n ðxþdxþc npðx n c nþ n 1þt c, and we get n EfV n g¼ R X n c n xf X nðxþdx þ c n P ðx n c n Þ. akng the dervatve of EfV n g wth respect to G we n Þ R cn 0 xf X nðxþdx npðx n c n n nþ1 þ t s nþ1 EfVnþ1 nþ1 þ t s 2 : (22) nþ1 By nducton hypothess we 0 nþ1 0. herefore n 0, completng the nducton step and the proof. APPENDIX B PROOF OF LEMMA 3.4: ERGODICIY OF G By Assumpton 1 there exsts a threshold G ~ such that EfV ðg Þg <EfV ðg Þg, 8 2 V, for all G G ~, where G denotes the aggregated load on all other channel except channel, and EfV ðg Þg s defned as the average reward/rate-of-rern of all other channels except. In ths case, the arrvals to all other channels except wll not swtch to channel,.e., under loads G > G ~ the probablty of load G drftng hgher s 0 almost surely. Defne any ncreasng, unbounded Lyapunov functon LðG Þ on ½0;GŠ (e.g., LðG Þ¼ 1 GG ), we have E ~G ½Lð G ~ ÞjG ŠLðG Þ. By the Foster-Lyapunov crtera [15] we establsh the ergodcty of the system load vector.

12 12 RANSACIONS ON MOBILE COMPUING, VOL. 13, NO. X, XXXXX 2014 APPENDIX C PROOF OF LEMMA 3.5: LOAD BALANCE We prove ths by nducton on N. WhenN ¼ 1,.e.,the system degenerates to a sngle channel case, the clam holds obvously. Assume the clam holds for N ¼ 2;...; n 1, and now consder the case N ¼ n. Suppose we ncrease the total load from G to G 0, and assume that wthout loss of generalty the load to channel 1 decreases,.e., G 0 1 <G 1. By the nducton hypothess, the loads on all other channels have ncreased,.e., G 0 >G, 8 6¼ 1. Asa result, ther correspondng value functons decrease by the prevous lemma,.e., EfV 0 g <EfV g, 8 6¼ 1. hs means that the amount swtchng out of channel 1 must be non-ncreasng, due to the fact that the threshold of swtchng c 1 s a non-ncreasng functon of G, whlethe amount swtchng nto channel 1 must be non-decreasng, leadng to an overall non-decreasng load on channel 1, whch s a contradcton. APPENDIX D PROOF OF LEMMA 3.7: MONOONICIY OF VALUE FUNCIONS IN When ¼ N,.e., the last stage, we have N tc N ¼ R X N ðx N N Þf XN ðxþdx. Followng a smlar argument as n the monotoncty n G, wth the decrease n t c N, the soluton N cannot be decreasng, provng that N s a non-decreasng functon of. Assume now the clam holds for ¼ n þ 1;...;N 1. When ¼ n, consder two cases. For the case n c n,we have n tc n ¼ R X n ðx n n Þf X nðxþdx. We know n s a nondecreasng functon of. For the case n <c n, n ¼ R n X xf c X n ðxþdxþc n PðX n c n Þ n 1þt c n. We have EfV n g¼ R X n xf X nðxþdx þ c n P ðx n c n Þ and takng the dervatve of EfV n g w.r.t. we have R n n Þ 0 xf n npðx n c n n c c n þt EfV nþ1 g ; (24) þ t and moreover we þt ¼ þ t c 1 ð þ t c Þ 2 ; (25) and combne wth the 0 (nducton hypothess) 0, > 0, completng the nducton step and the proof. APPENDIX E PROOF OF LEMMA 4.2: CONRACION For v; z 2F, we have ð vþðxþðzþðxþ ( ¼ max rðu; xþþh X ) vðyþp u ðy j xþ u2u ( max rðu; xþþh X ) zðyþp u ðy j xþ : (26) u2u Let m ¼ arg max u2u frðu; xþþh P vðyþpu ðy j xþg, then ( ) ð vþðxþðzþðxþ ¼ rðm;xþþh X vðyþp m ðy j xþ ( ) max rðu; xþþh X zðyþp u ðy j xþ u2u ( ) rðm;xþþh X vðyþp m ðy j xþ ( ) rðm;xþþh X zðyþp m ðy j xþ ¼ h X ½vðyÞzðyÞŠ P m ðy j xþ h max j vðyþzðyþj ¼ hjjv zjj: (27) Smlarly by reversng the order of z; v we have ð zþðxþðvþðxþ hjjv zjj. herefore we reach at jj v zjj hjjv zjj,.e., s a contracton. APPENDIX F PROOF OF HEOREM 4.5 : MONOONICIY OF HE VALUE FUNCION IN G As proved n [11], V ðxþ;x2 S; 2 V can be nterpreted as follows: V ðxþ ¼max u E X1 k¼0 b ktc r ðu k ;x k Þ: (28) Consder the expected maxmum throughput at the last stage,.e., V N ðxþ ¼max u E P 1 k¼0 bktc N r N ðu k ;x k Þ: Consder a G 0 N G N whch gves us t c0 N tc N. Consder an arbtrary term n the above sum b ktc0 N, and there exsts a k 0 such that k 0 t c N tc0 N ðk0 þ 1Þt c N. ogether wth the fact that b t P y Pt ðy j xþy s convex w.r.t. t we know max b ðk0 þ1þt c N X P ðk0 þ1þt c N ðy j xþy; y b k0 t c N X P k0 t c N ðy j xþy (29) y b ktc0 N X y P ktc0 N ðy j xþy

13 LIU AND LIU: O SAY OR O SWICH: MULIUSER MULI-CHANNEL DYNAMIC ACCESS 13 V 0 N ðxþ ¼max E X1 ðb ktc0 N Þ k r N ðu k ;x k Þ u k¼0 max E X1 b ktc N r N ðu k ;x k Þ¼V N ðxþ: (30) u k¼0 herefore as EfV N g¼ P x p x V N ðxþ, and we know EfV N g s a non-ncreasng functon of G. hs establshes the nducton bass. Now assume that the theorem holds for ¼ n þ 1;...;N 1. Consder the case ¼ n. Assume G 0 > G. As dscussed n the IID secton we have r 0 nðu; xþ r n ðu; xþ. (hs can be proved by takng the dervatve of c wth respect to G and by nducton 0). herefore agan smlarly as argued above we have V 0 n ðxþ ¼max u E X1 b ktc0 k¼0 n r 0 n ðu k;x k Þ max E X1 b ktc n r 0 u n ðu k;x k Þ k¼0 max E X1 b ktc n r n ðu k ;x k Þ¼V n ðxþ (31) u k¼0 whch completes the nducton step. APPENDIX G BACKWARD CALCULAION OF HE WO-DIMENSIONAL NESED SOPPING POLICY We descrbe the process of calculatng the threshold for each channel. Note at the last stage of the decson process there s no more channel to swtch to; therefore the dynamc program degenerates to a standard rate-of-rern problem. he standard optmal stoppng rule thus apples and the detals are omtted. By gong backward, at a subsequent stage <N,thequanttyEfV þ1 ðx þ1 Þg s avalable, and we have V ðxþ ¼maxf ^X ðxþ; t c þ EfV ðx Þgg. We calculate c usng c ¼ þt s EfV þ1 ðx þ1 Þg, andobtan þ1 R X xf c X ðxþdxþc PðX c Þ 1þt c = R X xf X ðxþdx. If the latter s less than c, we are doneandtakethsasthethreshold.otherwse,wepro- ceed to a fxed-pont equaton ¼ P ðx Þþt c whch can = be solved teratvely to obtan the threshold. APPENDIX H PROOF OF HEOREM 3.8 : SENSIIVIY OF HE VALUE FUNCION IN t s ; t c When D s small, usng aylor approxmaton (fðdþ fð0þþf 0 ð0þd) we have jv ðt c þ D; t s þ DÞV ðt c ; t s jdj: (32) D¼0 We prove the result nductvely. Recall at stage N we have the followng fxed pont equaton for characterzng EfV N g Z N X EfV N gt c N ¼ EfV N g ðx EfV N gþf X N ðxþdx; (33) where t c ¼ tc =. akng the dervatve on both sdes w.r.t. D we N tc N þ D þ EfV t c N þ ¼ð1FðEfV N NðDÞg ; where F s the cdf of X N. Rearrangng terms we N D¼0 EfV N ðdþg ¼ EfV N g t c N þd ¼ þ 1 FðEfV N ðdþg t c N þ ð1 F ðefv NgÞ D¼0 EfV N g ¼ 1 F ðefv N gþ 1 t c N =ð1 FðEfV NgÞ þ EfV N g 1 F ðefv N gþ 1 t c N þ C 1 N þ t c ; N (35) wth C N beng constant, where the last nequalty uses the fact that EfV N g s bounded by a constant. herefore by the dynamc programmng equaton Eqn. (4) (the second term) we N C 0 N D¼0 þ t c 2 þ 1 þ t c C N N N þ t c N (36) ^C N þ t c 2 N wth C 0 N ; ^C þt c N beng constants. he frst term comes from and the second comes from Eq. (35). hs establshes the nducton bass. Suppose the result holds for þ 1;...;N, consder stage. If the decson s SAY on at same stage/channel, then usng Eqn. (4) we see ths case s smlar to stage N, ^C D¼0 þ t c 2 : (37) When the decson s SWICH, we have (the thrd term n Eqn. s þ1 þd EfV þ1ðdþg (38)

14 14 RANSACIONS ON MOBILE COMPUING, VOL. 13, NO. X, XXXXX 2014 By the nducton hypothess we þt s þ1 þd EfV þt s þ1 ¼ þd EfV þ1 D¼0 þ t s þ1 þ D C 0 þ t s 2 þ XN X j¼þ1 o2fs;cg D¼0 þ t o j Þ 2 C j: (39) Combnng Eqs. (37) and (39) completes the nducton step. ACKNOWLEDGMENS A prelmnary verson of ths paper appeared n INFOCOM n Aprl hs work was partally supported by the US NSF under grants CIF and CNS , and the ARO under Grant W911NF REFERENCES [1] M. J Abdel-Rahman, F. Lan, and M. Krunz, Spectrum-effcent stochastc channel assgnment for oppornstc networks, n Proc. Global Commun. Conf., 2013, pp [2] S. H. A. Ahmad, M. Lu,. Javd, Q. Zhao, and B. Krshnamachar, Optmalty of myopc sensng n multchannel oppornstc access, rans. Inf. heor., vol. 55, no. 9, pp , Sep [3] W. U. Bajwa, J. Haupt, A. M. Sayeed, and R. Nowak, Compressed channel sensng: A new approach to estmatng sparse multpath channels, Proc., vol. 98, no. 6, pp , Jun [4] N. B. Chang and M. Lu, Optmal compettve algorthms for oppornstc spectrum access, J. Sel. Areas Commun., vol. 26, no. 7, pp , Sep [5] N. B. Chang and M. Lu, Optmal channel probng and transmsson schedulng for oppornstc spectrum access, /ACM rans. Netw., vol. 17, no. 6, pp , Dec [6] H.. Cheng and W. Zhuang, Smple channel sensng order n cogntve rado networks, J. Sel. Areas Commun., vol. 29, no. 4, pp , Apr [7] R. Fan and H. Jang, Channel sensng-order settng n cogntve rado networks: A two-user case, rans. Veh. echnol., vol. 58, no. 9, pp , Nov [8]. S. Ferguson, Optmal stoppng and applcatons, Math. Dept., UCLA, Los Angeles, CA, USA, 2006, edu/~tom/stoppng/contents.html [9] H. Jang, L. La, R. Fan, and H. V. Poor, Optmal selecton of channel sensng order n cogntve rado, rans. Wreless Commun., vol. 8, no. 1, pp , Jan [10] V. Kanoda, A. Sabharwal, and E. Knghtly, MOAR: A mult-channel oppornstc auto-rate meda access protocol for ad hoc networks, n Proc. 1st Int. Conf. Broadband Netw., 2004, pp [11] P. R. Kumar and P. Varaya, Stochastc Systems: Estmaton, Identfcaton and Adaptve Control. Upper Saddle Rver, NJ, USA: Prentce-Hall, [12] Y. C. Lang, Y. Zeng, E. C. Y. Peh, and A.. Hoang, Sensngthroughput tradeoff for cogntve rado networks, rans. Wreless Commun., vol. 7, no. 4, pp , Apr [13] K. Lu and Q. Zhao, Indexablty of restless bandt problems and optmalty of whttle ndex for dynamc multchannel access, rans. Inf. heor., vol. 56, no. 11, pp , Nov [14] Y. Lu, M. Lu, and J. Deng, Is dversty gan worth the pan: A delay comparson between oppornstc mult-channel MAC and sngle-channel MAC, n Proc. 31st Conf. Comput. Commun., Mar. 2012, pp [15] S. P. Meyn and R. L. weede, Stablty of Markovan processes III: Foster-Lyapunov crtera for contnuous-tme processes, Adv. Appl. Probablty, vol. 25, no. 3, pp , [16] J. L. Paredes, G. R. Arce, and Z. Wang, Ultra-wdeband compressed sensng: Channel estmaton, J. Sel. opcs Sgnal Process., vol. 1, no. 3, pp , Oct [17] J. Park, P. Paweczak, and D. Cabrc, Performance of jont spectrum sensng and MAC algorthms for multchannel oppornstc spectrum access ad hoc networks, rans. Moble Comput., vol. 10, no. 7, pp , Jul [18]. Shu and M. Krunz, hroughput-effcent sequental channel sensng and probng n cogntve rado networks under sensng errors, n Proc. 15th Annu. Int. Conf. Moble Comput. Netw., 2009, pp [19] S.-S. an, D. Zheng, J. Zhang, and J. Zedler, Dstrbuted oppornstc schedulng for ad-hoc communcatons under delay constrants, n Proc. 29th Conf. Inf. Commun., 2010, pp [20] Q. Zhao, B. Krshnamachar, and K. Lu, On Myopc sensng for mult-channel oppornstc access: Strucre, optmalty, and performance, rans. Wreless Commun., vol. 7, no. 12, pp , Dec [21] Q. Zhao, L. ong, A. Swam, and Y. Chen, Decentralzed cogntve MAC for oppornstc spectrum access n ad hoc networks: A POMDP framework, J. Sel. Areas Commun., vol. 25, no. 3, pp , Apr [22] D. Zheng, W. Ge, and J. Zhang, Dstrbuted oppornstc schedulng for ad-hoc networks wth random access: An optmal stoppng approach, rans. Inf. heor., vol. 55, no. 1, pp , Jan Yang Lu s currently a fourth year PhD canddate of EE:Systems, Unversty of Mchgan, Ann Arbor. Hs advsor s Prof. Mngyan Lu. Before comng to Ann Arbor, he receved a bachelor s degree n Informaton Securty from Shangha Jao ong Unversty, Chna n Hs research nterests nclude desgn and performance evaluaton of mult-channel wreless networks, multchannel dversty ssues, effcent resource allocaton, network securty and machne learnng. Mngyan Lu (S 96-M 00-SM 11-F 14) receved her PhD degree n electrcal engneerng from the Unversty of Maryland, College Park, n She joned the Department of Electrcal Engneerng and Computer Scence at the Unversty of Mchgan, Ann Arbor, n September 2000, where she s currently a Professor. Her research nterests are n optmal resource allocaton, performance modelng and analyss, and energy effcent desgn of wreless, moble ad hoc, and sensor networks. She s the recpent of the 2002 US NSF CAREER Award, the Unversty of Mchgan Elzabeth C. Crosby Research Award n 2003, and the 2010 EECS Department Outstandng Achevement Award. She serves/has served on the edtoral boards of /ACM ransactons on Networkng, ransactons on Moble Computng, andacm ransactons on Sensor Networks. " For more nformaton on ths or any other computng topc, please vst our Dgtal Lbrary at