5048 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 12, NO. 10, OCTOBER 2013

Transcription

1 548 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL., NO., OCTOBER Wirele Acce Networ Selection Game with Negative Networ Externality Yu-Han Yang, Student Member, IEEE, Yan Chen, Member, IEEE, Chunxiao Jiang, Member, IEEE, Chih-Yu Wang, Member, IEEE, and K. J. Ray Liu, Fellow, IEEE Abtract Networ ervice acquiition in a wirele environment require the election of a wirele acce networ. A ey problem in wirele acce networ election i to tudy the rational trategy conidering the negative networ externality, i.e, the influence of ubequent uer deciion on an individual throughput due to the limited available reource. In thi wor, we formulate the wirele networ election problem a a tochatic game with negative networ externality and how that finding the optimal deciion rule can be modelled a a multi-dimenional Marov Deciion Proce (MDP. A modified value iteration algorithm i propoed to efficiently obtain the optimal deciion rule with a imple threhold tructure, which enable u to reduce the torage pace of the trategy profile. We further invetigate the mechanim deign problem with incentive compatibility contraint, which enforce the networ to reveal the truthful tate information. The formulated problem i a mixed integer programming problem which in general lac an efficient olution. Exploiting the optimality of ubtructure, we propoe a dynamic programming algorithm that can optimally olve the problem in the two-networ cenario. For the multinetwor cenario, the propoed algorithm can outperform the heuritic greedy approach in a polynomial-time complexity. Finally, imulation reult are hown to validate the analyi and demontrate the effectivene of the propoed algorithm. Index Term Game theory, tochatic game, negative networ externality, Marov deciion proce, networ election, mechanim deign, dynamic programming. I. INTRODUCTION NOWADAYS, wirele networ ervice uch a Femtocell [] and Wi-Fi acce point are widely deployed to provide Internet acce in area uch a home, office, airport, hotel, etc. While there may be multiple available wirele networ, a uer can only chooe one to join. Figure how an example of the Wi-Fi networ election from a mart phone. Since the networ can be owned by different operator, the networ election problem, which ued to be reolved in a centralized manner by admiion control [], Manucript received December, ; revied April 9 and June, ; accepted July,. The aociate editor coordinating the review of thi paper and approving it for publication wa P. Wang. Y.-H. Yang, Y. Chen, and K.J.R. Liu are with the Department of Electrical and Computer Engineering, Univerity of Maryland, College Par, MD, 74, USA ( {yhyang, yan, jrliu}@umd.edu. C. Jiang i with the Department of Electrical and Computer Engineering, Univerity of Maryland, College Par, MD 74, USA, and alo with the Department of Electronic Engineering, Tinghua Univerity, Beijing 84, P. R. China ( chx.jiang@gmail.com. C.-Y. Wang i with the Department of Electrical and Computer Engineering, Univerity of Maryland, College Par, MD 74, USA, and alo with GICE, National Taiwan Univerity, Taipei, 6, Taiwan (R.O.C. ( tomywang@gmail.com. Digital Object Identifier.9/TWC..95. Fig /$. c IEEE Wi-Fi networ election. [], hould be invetigated in a ditributed perpective by conidering uer own interet. In the wirele acce networ election problem, a myopic trategy can uually be adopted by chooing the one with the tronget ignal. A conequence of thi trategy i the congetion of uer to communicate with certain networ controller uch a acce point (AP, witche, or router. The concentration of uer create an unbalanced load in the networ, which lead to an inefficient reource utilization for ervice provider and a poor qualityof-ervice (QoS for uer. Efficient reource utilization i an important iue in modern wirele acce networ due to limited available reource uch a ignal power, temporal and patial bandwidth. On one hand, the ervice provider attempt to maximize reource utilization uch that the available reource can accommodate a many uer a poible. On the other hand, due to the individual rationality and the elfih nature, a uer aim to optimize hi/her own utility. Therefore, a uer optimal trategy in uch a reource-haring cenario inevitably ha to tae into conideration the negative networ externality [4], [5], i.e., the influence of other uer trategie on the uer own utility. Commonly referred in economic and buine, the negative networ externality i the effect that occur when more uer mae the available reource le valuable. For example, the traffic congetion overload the highway. Overwhelming cutomer degrade the quality-of-ervice in a retaurant. The negative networ externality in thee example impair the utilitie of the uer maing the ame deciion. In thi paper, we firtly focu on how a uer hould chooe

2 YANG et al.: WIRELESS ACCESS NETWORK SELECTION GAME WITH NEGATIVE NETWORK EXTERNALITY 549 one of the available wirele acce networ conidering the negative networ externality. Wirele acce networ election i an eential problem of reource utilization and ha received great attention recently [6] [7]. In [], centralized approache are invetigated to provide congetion relief by explicit channel witching and networ-directed roaming. A ditributed acce point election algorithm baed on no regret learning i propoed in [4]. The author how that the algorithm can guarantee convergence to an equilibrium. The arrival and departure of the uer in networ election problem are alo conidered in [6] and [7]. Another cla of networ election approache i baed on game theory. Game theory ha been recognized a an ideal tool to tudy the interaction among uer [8], [9]. It ha been widely ued in wirele communication and networing for many different problem [9] [] including power control [], cooperation timulation [], and ecurity enforcement [4]. In [7], Mittal et al. conider uer changing location a trategie to obtain more reource and analyze the correponding Nah equilibria (NE. In [], the networ election i modelled a a congetion game, where player mae deciion imultaneouly to optimize the interference and throughput. Alo, the congetion in the networ election game i imilar to that in the channel election game, e.g., [5] [7]. In [5], an atomic congetion game in which reource are allowed to be reued among noninterfering uer i conidered. In [6] and [7], the author invetigated game theoretic olution to the ditributed channel election problem in opportunitic pectrum acce ytem. A comprehenive review and comparion of exiting deciiontheoretic olution including Marov deciion proce, game theory and tochatic control can be found in [8]. However, mot of the exiting wor tudy the networ election problem under the cenario where uer mae deciion imultaneouly. In thi paper, we conider the problem under a different cenario where uer mae deciion equentially and their optimal deciion involve the prediction of ubequent uer deciion due to the negative networ externality. Sequential deciion conidering the negative networ externality effect are tudied in the Chinee retaurant game [9] [], in which the equilibrium of grouping under the cenario of a fixed total number of player i characterized. In thi wor, we formulate the wirele acce networ election problem a a tochatic game with negative networ externality, where uer arrive at and depart from networ in a probabilitic manner. The problem of finding the optimal deciion rule i hown to be a multi-dimenional Marov Deciion Proce (MDP. Different from the conventional MDP [], the multidimenional MDP ha multiple potential function and thu the dynamic programming (DP [] cannot be directly applied. We propoe a modified value iteration algorithm to find the equilibrium for the multi-dimenional MDP. The analyi of the propoed algorithm how that the trategy profile generated by the algorithm ha a threhold tructure, which enable u to ave the torage pace of the trategy profile from O(N to O(N log N, wheren i the number of ytem tate in the two-networ cenario. Simulation reult verify the analyi and demontrate the efficiency and effectivene of the propoed algorithm, i.e., while achieving the optimal trategy for the individual, the propoed algorithm attain imilar performance of ocial welfare compared to the centralized method that maximize the ocial welfare. The econd focu of thi paper i the truthful mechanim deign [4] [8] for the networ election game. Mechanim deign i to devie pricing and allocation rule atifying the incentive compatibility [5], [6]. In the networ election game, uer mae deciion relying on the ytem tate which conit of the information provided by the networ, poibly owned by different operator with different interet. Therefore, the reported tate may be untruthful if it i profitable to mae a deceitful claim. In thi wor, we invetigate the mechanim deign problem with incentive compatibility contraint, which enforce the networ to report truthfully, while optimizing the utility of uer. The formulated problem i a mixed integer programming problem which in general lac an efficient olution. Exploiting the optimality of ubtructure, we propoe a dynamic programming algorithm that can efficiently and optimally olve the problem in the two-networ cenario. For the multi-networ cenario, the propoed algorithm can outperform the heuritic greedy approach in a polynomial-time complexity. Finally, imulation reult are hown to validate the analyi and demontrate the effectivene of the propoed algorithm. The novelty and technical contribution of thi wor are ummarized a follow. We formulate the ditributed wirele acce networ election problem a a multi-dimenional MDP, which, to the bet of our nowledge, i new and ha not been tudied before. We propoe a modified value iteration algorithm to earch for an equilibrium. We alo analyze the propoed algorithm and how that the reulting trategy profile ha a threhold tructure. We further propoe an efficient dynamic programming algorithm to deign a truthful mechanim which enforce the networ to truthfully reveal the tate information. The ret of the paper i organized a follow. The ytem model and the formulation of the wirele acce networ election game i decribed in Section II. In Section III, we propoe a modified value iteration algorithm for the multidimenional MDP. The threhold tructure of the trategy profile generated by the propoed algorithm i analyzed in Section IV. In Section V, we decribe the mechanim deign problem for the networ election game and propoe the dynamic programming algorithm. In Section VI, the performance of the propoed algorithm i evaluated uing numerical imulation. Finally, Section VIII conclude the paper. II. SYSTEM MODEL AND PROBLEM FORMULATION In thi ection, we decribe in detail the ytem model and the problem formulation of the wirele acce networ election problem. To better illutrate the idea, we firt introduce ome neceary notation including the probabilitic model and then characterize the (approximate equilibrium. Note that a will be een, the model i quite general and hence it application i not retricted to the networ election problem but can alo be deployed in other problem with negative networ externality.

3 55 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL., NO., OCTOBER A. Sytem Model The ytem under conideration comprie K wirele acce networ and each networ ha a capacity of N uer, i.e., a networ can imultaneouly erve at mot N uer. For the ae of notational conciene, we conider that all the networ have the ame capacity. The analyi can be eaily extended to the ytem with networ of different capacity. We alo aume that the networ have no buffer room for uer, which mean when a networ i full, uer cannot mae requet of connection to the networ. Each uer in networ obtain a utility R ( per unit time, where i the current number of uer in networ. The utility function i defined a the individual throughput, i.e., R ( = P log( + S/N ( P I/N +,, which repreent the achievable data rate under inter-uer interference, where P S /N denote the ignal-to-noie power ratio, and P I /N i the interferenceto-noie power ratio. The utility repreent the quality-ofervice (QoS guaranteed by the networ but retricted to the available reource uch a the total tranmiion power and the bandwidth of radio frequency. The negative networ externality i manifeted in the decreae of the data rate a the number of uer in the networ increae due to a higher inter-uer interference. Note that the utilitie of uer in the ame networ are aumed the ame at each time lot ince the networ can provide the ame QoS to each uer by mean of reource allocation, even though the intantaneou channel condition of different uer may be different. For example, centralized downlin power control algorithm [9], [4] can be applied by the networ to attain a common ignal to interference-plu-noie ratio (SINR or to maximize the minimum SINR among the uer. The uer with Poion ditributed arrival rate λ (uer per econd have choice of connecting to one of the K networ. After a uer mae hi deciion, he/he cannot witch to any of other networ and ha to tay during a period of time with exponential ditribution of parameter μ, which i aumed the ame for all networ for implicity. The uer with arrival rate λ can only chooe networ, for =,...,K. Thee uer can be enviioned a either the uer with certain determinitic behavior, or the uer who can only have acce to one pecific networ due to the geographical ditribution. Note that incorporating thi type of uer only mae the ytem model more general ince we can imply et thee rate a zeroiftherearenouchuer. The ytem tate = (,..., K tae it value from the tate pace S = {(,..., K =,,..., N, =,...,K}, and repreent the tate that uer are in networ, for =,...,K. We conider a dicrete time Marov ytem where a time lot ha duration T (econd. Then the arrival and departure probabilitie λ = λ Te λ T and μ = μt e μt can be approximated a λ λ T, =,...,K and μ μt when T i ufficiently mall [4] [4]. Let F( = { = N, =,...,K} be the index et of the full networ which are erving the maximum number of More general type of uer, uch a uer who can only connect to one of a ubet of K networ, can be conidered. Here for implicity we only conider two type of uer, i.e., uer who have choice of connecting to any one of K networ, and uer who can only chooe one pecific networ. uer and thu cannot accept any more. The complement et of F( i denoted by F( = { <N,=,...,K}, i.e., the index et of the non-full networ. The trategy pace of networ election i retricted in F( when i a boundary tate, i.e., when σ F(. We aume that the connection requet from uer arriving at the full networ will be rejected and the traffic then goe to other non-full networ. To model uch a traffic tranition, we therefore aume that the traffic immediately flow to the non-full networ. For the two-networ cae, at mot only one non-full networ ha room for thoe uer, o the traffic goe to that nonfull networ. For the multi-networ cae, multiple non-full networ can accommodate thoe uer. In order to provide a well-defined Marov ytem and to implify the notation, we aume that the traffic goe to a pecific networ, i.e., min F(, the networ with the minimum index. Notice that if F( =φ, i.e., all networ are full, no connection requet can be accepted. The networ election trategy when the uer oberve tate i denoted a σ, which tae value in F(. Wedefineσ = j if networ j i choen. The indicator function I (σ i then defined a: if σ = j, I j (σ =; otherwie I j (σ =. We have the tate tranition probability of an arrival event a P y ( + e j { i F( = λ i + λ j + I j (σ λ, if j =min F(, λ j + I j (σ λ, if j F(\ { min F( } (, where and + e j denote the ytem tate at the current time lot and the next time lot, and e j i a tandard bai vector whoe j-th coordinate i and other coordinate are. At ytem tate, ince the number of uer in networ j i j, the tranition probability of a departure event i given by P y ( e j = j μ, j =,...,K. ( Furthermore, the probability that the ytem tate remain the ame i { K j= P y ( = λ j K j= jμ, if F( φ, K j= jμ, if F( =φ. ( The duration of a time lot T hould be choen uch that K j= λ j + KNμ, i.e., T /( K λ j= j + KN μ. For intance, when K =, N, and N, the tranition probability i given by P y { =(, } I (σ λ + λ, if =( +,, I (σ λ + λ, if =(, +, μ, if =(,, = μ, if =(,, λ λ λ μ μ, if =(,,, otherwie. (4 Similarly the correponding tranition probability for = N, N or N, = N can alo be defined. Figure depict the tate tranition diagram

4 YANG et al.: WIRELESS ACCESS NETWORK SELECTION GAME WITH NEGATIVE NETWORK EXTERNALITY 55 I, I, ( ( I, ( (, I(, (, (, I, I(, (, (, ( (, (, I (, I ( I, ( I, ( I(, I(, I,, I, ( I, I, ( ( Fig.. State diagram of the -D Marov chain. (, (, ( I(, I, ( where ( P = i F( λi + λj + Ij(σλ, if j =min F(, λ j + I j(σ λ, if j F(\ { min F( }, iμ, if = e i, i, ( μ, if = e, K j= λj K j= jμ + μ, if =,, otherwie. which i the tranition probability given that the uer till tay in networ. The probability of tranition from to e i ( μ ince uer may leave the networ. The tranition probability from to other tate i imilar to the definition of P y in (4. (7 when K =. The dynamic of the two-networ ytem can be decribed by a two-dimenional (-D Marov chain where the probability P y ( i not hown in Figure for conciene. B. Expected utility The trategy profile σ = {σ S} i a mapping from the aggregate tate pace to the action pace, i.e., σ : {,,..., N} K {,,...,K}. Given a trategy profile σ, we can obtain the ytem tranition probability in ( - (. When a rational uer arrive and oberve ytem tate, he/he mae the deciion σ = ˆ which lead the uer into the ytem tate = + eˆ. Then, the expected utility of the rational uer i given by [ ] Vˆ( =E ( μ t Rˆ( t, (5 t= where t denote the ytem tate at time t. Sinceμ i the probability that the ervice i terminated in one time lot, then ( μ can be interpreted a the probability that the uer tay in the networ in one time lot. The value ( μ can alo be regarded a the dicounting factor for the future utility a hown later in (6. The trategy σ = ˆ determine which networ the uer will enter and thu which expected utility function the uer will obtain. Denoted by Vˆ(, the expected utility function i the expected value of the dicounted um of the immediate utilitie Rˆ( t accumulated from the next time lot. Notice that = + eˆ i uniquely determined by the uer trategy σ, but the ubequent tate t,for t, are tochatic and dependent on the arrival of other uer, including uer from uer-arrival tream, K, and other rational uer. From the Bellman equation [], the expected utility in (5 can be hown to atify the following recurive expreion. V ( =R ( +( μ P ( V (, (6 C. Bet Repone of Rational Uer Due to the elfih nature, when oberving the tate, a rational uer will chooe the trategy σ to maximize hi expected utility. Thu, the rational trategy σ ha to atify σ =argmaxv ( + e. (8 It can be een that with the trategy profile in which the trategy of every tate atifie (8, no uer can obtain a higher expected utility by unilateral deviation to any other trategy. Therefore, the trategy profile atifying (6-(8 i a Nah equilibrium of the tochatic game. III. MODIFIED VALUE ITERATION ALGORITHM The problem of finding the trategy profile atifying (6-(8 i not a conventional Marov Deciion Proce problem. In a conventional MDP problem [], a ingle potential function i aociated with each ytem tate, and the optimal trategy can be obtained directly by optimizing the potential function. Such a problem can often be olved via the theory of dynamic programming (DP []. However, in our model, multiple potential function are related in a vector form: T V ( R ( p v V (. = R ( p v +( μ , V K ( R K ( K p K v K (9 where denote an all-zero vector, p and v are vector compriing P ( and V ( a element, =,...,K. The tranpoe operator i denoted by ( T. The trategy σ i determined by comparing V ( + e for all a in (8. Thu, DP cannot be directly applied in uch a problem. It i important to point out that a uer mae a deciion after he arrive and oberve the ytem tate. The trategy lead the uer into ome networ and reult in an expected utility V ( + e. In ubequent time lot, the uer cannot change from the networ he/he i taying to any other networ. The expected utility i affected by other trategie through the tranition probabilitie a given in (6. We can ee that given the expected utilitie {V } K =,the rational trategy profile σ hould atify (8. On the other hand, given a trategy profile σ, the expected utilitie {V } K =

5 55 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL., NO., OCTOBER can be found by (6, where the tranition probability P ( i a function of the trategy σ. To obtain the optimal trategy profile σ atifying (6-(8, we propoe a modified value iteration algorithm to iteratively olve the problem. At the n-th iteration, the rational trategy profile i given by σ (n+ =argmaxv (n ( + e, S. ( The expected utility function can be obtained by olving V (n+ ( =R ( +( μ S P (n+ ( V (n+ (, S, {,...,K}, ( where the tranition probability P (n+ ( i updated uing the correponding updated trategie, i.e., P (n+ ( = i F( λi + λj + Ij(σ(n+ λ, if = + e j,j =min F( λ j + I j(σ (n+ λ, if = + e j,j F(\ { min F( } jμ, if = e j,j ( μ, if = e j F P ( + e j K j= P ( e j, =,, otherwie, ( The olution to ( can be obtained through everal approache, one of which i the value iteration algorithm []. The algorithm firt initialize V (n+ ( a an arbitrary value uch a zero and iteratively update it uing (. The iteration function i a contraction mapping o the convergence to a unique fixed point i guaranteed. Another approach i to conider ( a K et of linear ytem, where each et ha N unnown variable correponding to {V (n+ (, } and N equation. Such linear ytem can be olved by linear programming or matrix inverion. In the next ection, we will theoretically how that for K =, the propoed algorithm reult in a threhold tructure of the trategy profile at each iteration, and uch a threhold tructure i alo oberved for general K >. However, the trategy profile may not converge but ocillate near the threhold due to the hard deciion rule in (8. The nonconvergence occur when the rational trategy of the tate near the threhold ocillate between different choice each time when the expected utility i updated. When uch a ituation happen, the expected utilitie correponding to different trategie are very cloe to each other. Hence, to olve thi problem, we relax the hard deciion rule by allowing a mall region of tolerance for witching among the trategie [44], which lead to the oft deciion rule a follow. σ (n+ = σ (n, if V (n σ (n arg max V (n ( + e, if V (n ( + e (n σ σ (n ( + e (n σ max V (n ( + e ɛ, < max V (n ( + e ɛ, ( where ɛ > i a mall contant. Table I ummarize the propoed modified value iteration algorithm for the multidimenional MDP. Notice that the algorithm top when an TABLE I MODIFIED VALUE ITERATION ALGORITHM (i Initialize: V ( ( =, {,...,K}, S. T = φ. (ii Loop :. Update {σ (n+ } by (. If {σ (n+ } = {σ (n }, then top loop. } T, then chooe a {σ } T,andlet{σ (n+ ele if {σ (n+ end if T = T {σ (n+ }. } = {σ }.. Update {P (n+ ( } by (.. Solve {V (n+ (} in ( by value iteration or linear programming. Until T = φ or {σ (n+ } = {σ (n }. equilibrium i found or all the trategy profile are earched. By definition, when the algorithm obtain a olution, the reulting trategy profile i an ɛ-approximate NE [8], in which the trategy at each tate ha an expected utility that i at mot ɛ le than that of any other trategy. Note that there may be multiple ɛ-approximate NE epecially for a larger ɛ when a larger region of tolerance i allowed for witching among the trategie. IV. THRESHOLD STRUCTURE OF STRATEGY PROFILE In thi ection, we how that the trategy profile produced by the propoed modified value iteration algorithm in each iteration exhibit a threhold tructure for two-networ ytem. With the aumption that R (, =,, are non-increaing, the following lemma how that V ( i non-decreaing and V ( i non-increaing along the line of + = m, m {,,..., N}. Lemma : For n, V (n ( V (n ( + e e, (4 V (n ( V (n ( + e e. (5 Proof: We ue induction to how that (4 and (5 hold for all n. i Since V ( ( and V ( ( are initialized a zero, (4 and (5 hold for n =. ii We aume the induction hypothei hold for ome n. Then it can be hown that (4 and (5 alo hold for (n+ by analyzing the following difference. Let = + e e. For N and N, V (n+ ( V (n+ ( =R ( R ( + ( +( μ [λ V (n ( + e V (n ( + e ( + λ I (σ V (n ( + e I (σ V (n ( + e ( + λ V (n ( + e V (n ( + e ( + λ I (σ V (n ( + e I (σ V (n ( + e +( μv (n ( e μv (n ( e + μv (n ( e ( μv (n ( e +( λ λ λ μ μ ( V (n ( V (n ( ]. (6

6 YANG et al.: WIRELESS ACCESS NETWORK SELECTION GAME WITH NEGATIVE NETWORK EXTERNALITY 55 Due to the fact that the utility function R ( i nonincreaing in and the induction hypothei which guarantee the non-negativene of many difference of term in (6, by rearranging a few term, it uffice to dicu the following cae. Cae : σ (n = σ (n (n =. Then, V ( + e V (n ( + e by the induction hypothei. Cae : σ (n = σ (n (n =. Then, V ( + e V (n ( + e by the induction hypothei. Cae : σ (n =and σ (n (n =. Then, V ( + e V (n ( + e =. Cae 4: σ (n =and σ (n (n =. Then, V ( + e V (n ( + e by the induction hypothei. Therefore, we have V (n+ ( V (n+ (, for N and N. Next, it can be eaily checed that the inequality till hold for the cae of = N, N a well a the cae of N, = N. Similarly, V (n ( V (n ( can alo be etablihed. The following lemma how the difference of V ( + e and V (+e i non-increaing along the line of + = m, m {,,..., N}. Lemma : V (n ( + e V (n ( + e V (n ( + e V (n ( + e,where = + e e. Proof: It can be eaily hown uing Lemma. Theorem : The trategy profile generated by the modified value iteration algorithm ha a threhold tructure for K =. Proof: The oft deciion rule in ( for K =can be rewritten a σ σ (n+ (n, if V (n ( + e V (n ( + e ɛ, =, if V (n ( + e >V (n ( + e +ɛ, (7, if V (n ( + e >V (n ( + e +ɛ. If σ (n =and σ (n+ =, i.e., the trategy of the current iteration i updated to be different from the one of the previou iteration, then we mut have V (n ( + e >V (n ( + e + ɛ. Lemma implie that V ( + e V ( + e i nonincreaing along the line of + = +. Thu, for = e + e, =,,...,min{,n },wehave V ( + e V ( + e V ( + e V ( + e >ɛ>. Therefore, σ (n+ = for = e + e, =,,..., min{,n }. Similarly, if σ (n =and σ (n+ =, then σ (n+ =, for = + e e, =,,..., min{n, }. With the above dicuion, the trategie along the line of + = m, m {,,..., N} retain a threhold tructure in each iteration. Since the initialization of the trategy profile exhibit a threhold tructure trivially, the trategy profile obtained in each iteration of the algorithm ha a threhold tructure. In a two-networ ytem, the number of ytem tate i N and thu N trategie are needed to be tored without the threhold tructure. The torage pace of each trategy i bit. Now with uch threhold tructure on each line + = m, m =,,..., N, we can imply tore the threhold point on each line. Each threhold point require the torage pace of log N bit. Therefore, The torage of the trategy profile can be reduced from O(N to O(N log N. In thi paper, we only provide the analyi for the twonetwor ytem. The analyi for ytem with more than two networ i difficult due to the lac of the optimality in a ingle potential function a in the admiion control problem [45], [46]. However, it i oberved from the imulation reult in Section VI that the multi-networ ytem alo poe the trategy profile with threhold tructure. The theoretic analyi of the threhold tructure for the multi-networ ytem i important but out of the cope of thi paper, and will erve a one of our future wor. V. TRUTHFUL MECHANISM DESIGN In the above dicuion, we have implicitly aumed the networ truthfully report their tate, and therefore the uer can oberve the true ytem tate, by which he/he can mae a deciion to maximize hi/her utility. However, without appropriate incentive, the networ may not truthfully report their tate. Intead, a networ may untruthfully report ome tate different from the true tate if profitable. In thi ection, we conider to enforce truth-telling a a dominant trategy for the networ by incorporating pricing rule into the wirele acce networ election game. A mechanim conit of pricing rule {P (} and allocation rule {a (}, wherep ( i denoted a the unit price of the expected rate V ( provided by networ at tate, and a ( i denoted a the allocation probability, which i either or, i.e., whether or not the uer enter networ. The utility of networ i given by U ( =V ( + e P ( c ( + e a (, (8 where c ( + e i the cot per uer. With the tate reported from the networ, thee rule determine the uer allocation and the price the uer ha to pay, both a function of the report from networ. For example, if networ report hi tate a and other report = { j : j }, hi utility become V (+e P (, c (+e a (,. Notice that V ( + e and c ( + e are function of true tate that do not depend on the report. Thu, the truthtelling or the incentive compatibility (IC contraint are,,,, V ( +, P (, c ( +, a (, V ( +, P (, c ( +, a (,, which mean truth-telling i a dominant trategy for each networ at each tate. The mechanim alo ha to atify the individual rationality (IR contraint, i.e.,,, V ( +, P (, c ( +, a (,, (9 which guarantee all networ would attend the mechanim. In the previou ection, we tudy the networ election game with the focu of the interdependence between the uer. In thi ection, we tudy the interplay among the networ. To thi end, we aume that uer trategie are choen baed on the ex ante optimality [8], [5], i.e., the allocation rule i baed on optimizing the expected objective over the tate probability. The truthful mechanim deign i to contruct a et of pricing and allocation rule which optimize a pecific objective while atifying IC and IR contraint. For

7 554 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL., NO., OCTOBER example, the mechanim deign problem P p for minimizing the expected payment can be formulated a follow. P p : min K π( P (V ( + e ( {P },{a } S =.t. (IC, (IR, a ( {, },,. ( K a ( =, S, ( = Other mechanim deign objective uch a the utility maximization P u can be formulated by ubtituting ( with uer expected utility function a follow. K max π( [λa (V ( + e P (V ( + e ] {P },{a } S =.t. (, (. The unit cot c ( + e /V ( + e i denoted a w (. The (IC contraint become P (, w (, a (, P (, w (, a (,,,,. ( In the following, we need a monotonicity aumption for the unit cot, i.e., w (, i non-decreaing in, i.e., w (, w (,, if.sincev (, i non-increaing in, the aumption hold when c (, i non-decreaing in. For example, if the per-uer cot i a contant in each networ, i.e., c (, =C,then the aumption hold. The monotonicity of w (, lead to the threhold tructure of a (, a in the following lemma. Lemma : Under IC contraint, there exit a threhold value of on the allocation rule a (,, i.e., given, there exit ( {,,,...,N}, uch that {, a (, = (, > ( (4. Proof: From (, we have P (, P (, w (, [a (, a (, ]. (5 Interchanging and,wealohave P (, P (, w (, [a (, a (, ]. (6 Combining the above two inequality lead to [ w (, w (, ][ a (, a (, ]. (7 Thu, ince w (, i non-decreaing in, the allocation rule a (, ha to be non-increaing in. With thi monotonicity and the fact that a (, can only have value of or, we can conclude that there exit a threhold of a (, in a decribed in (4. Corollary : If K =,then ( i non-decreaing in, and ( i non-decreaing in. Proof: Suppoe uch that ( +< (. By Lemma, we have a (, + =,for > ( +, which implie a (, +=,for > ( +, due to the contraint that a ( +a ( =,. Therefore, a ( (, +=, which implie a ( (, =by Lemma, but we alo have a ( (, =, which lead to a contradiction. The following lemma how that only adjacent IC contraint are neceary. Lemma 4: Non-adjacent IC contraint are redundant. Proof: Let u conider the two adjacent IC contraint a follow. P (, w (, a (, P (, w (, a (,, (8 P (, w (, a (, P (, w (,.a (, (9 Adding (8 and (9, we have P (, w (, a (, P (, w (, a (, w (, [a (, a (, ] + w (, [a (, a (, ] P (, w (, a (,. ( The lat inequality i due to that w (, i increaing in and a (, i decreaing in. It how that the non-adjacent IC contraint can be inferred from the adjacent one. Uing the adjacent IC contraint, we can obtain the bound for the payment, i.e., given an allocation rule {a (}, the incentive compatible payment rule {P (} atifie P (, +w (, [a (, a (, ] P (, P (, + w (, [a (, a (, ] ( In the optimization problem P p, we aim to minimize a linear combination of P (, with nonnegative coefficient. Clearly, the lower bound in ( hould be binding; otherwie, the objective function can alway be better optimized by decreaing the non-binding P (,. Hence, the payment rule can be expreed a P (, =P (N, N + w (r, [a (r, a (r, ]. ( r= + To minimize P (, while atifying the IR contraint in (9, P (N, hould be et a. Subtituting Lemma into (, we can conclude { w ( P (, =,,,, >, ( where denote ( for notational implicity. From the IC and IR contraint, the pricing rule {P } can be determined given the allocation rule {a }, which i pecified by the threhold { }. Thu ( imply mean the pricing rule {P } i alo pecified by the threhold { }.Uing{ } a optimization variable, the problem P p can be implified a K min π( P (V ( { } (4 S =.t. (, (4, (.

8 YANG et al.: WIRELESS ACCESS NETWORK SELECTION GAME WITH NEGATIVE NETWORK EXTERNALITY 555 With the implification, however, the optimization problem i till difficult to be olved optimally ince the optimization variable { } i dicrete and the exhautive earch require exponential-time complexity in N. Motivated by the optimal ubtructure in the two-networ cae, a dynamic programming algorithm i propoed for the above problem. The optimal olution to the primary problem can be broen down into olving the optimal olution to it ubproblem. The dynamic programming technique eentially perform recurive divide-and-conquer to tacle each of thee ubproblem. However, for the multi-networ cae, the propoed dynamic programming approach i uboptimal but the performance i atifactory compared to the greedy method. Other traditional optimization algorithm uch a branch-andbound can be applied to optimally olve the mixed integer programming problem, but the computational complexity i prohibitively high (exponential in the number of tate ince uch an algorithm baically perform exhautive tree earch with certain pruning trategie. In general a mixed integer program doe not have an efficient olution. In thi paper, we aim to propoe an algorithm that i able to achieve atifactory performance with reaonable complexity (polynomial in the number of tate. A. Propoed Algorithm Since the number of tate i N K, the exhautive earch over all poible allocation rule require complexity of O(K N K. Such an exponential complexity i formidably high even for a moderate N. In thi ubection, we propoe a polynomial time algorithm baed on dynamic programming to earch for the threhold { DP }.Letf ({ i : i I} { j : j J} denote the optimal value of a et of ytem tate pecified by ({ i : i I} { j : j J}, wheretheetj conit of coordinate with coordinate j beingfixeda j.the et I conit of the coordinate with range, where coordinate i range from to i.theeti ha coordinate, i.e., the conidered et of ytem tate i -dimenional. The optimal value function f DP can be computed uing lower-dimenional optimal value function. The recurive calculation i decribed by the following equation. For =,...,K, { f DP ({ i : i I} { j : j J}=min i I f DP ( i, i { j : j J}+f DP ( i { j : j J {i}} where i = { l : l i, l I}. (5 a i ( i, i,j, j =, i i, (6 i =argmin i I { f DP ( i, i { j : j J} +f DP ( i { j : j J {i}} }, }, (7 where i i denote i { l : l l,l i,l I}. The boundary condition i f DP ( i i =f DP w i( i, i ( i i w i( i, i + π( i, iv i( i +, iw i( i, i, (8 a i ( i, i,j, j =, i i, (9 where i i the minimizer in (7 when =. Notice that (8 i equivalent to f DP ( i i = i r= π(r, iv i (r + TABLE II DYNAMIC PROGRAMMING ALGORITHM FOR MECHANISM DESIGN (i Initialization: obtain {V ( (} and {π ( (} uing Table I. (ii Loop:. With initial I = {,...,K}, J = φ, evaluate f (n K (N,...,N uing (5-(9 to obtain {a(n+ (} (}. (} and {π (n+ (}. (} and {P (n+ (} converge. and {P (n+. Calculate {V (n+ Until {a (n+, i w i ( i, i, but the recurive form in (8 i more efficient in computation with the price of uing more torage pace. The propoed algorithm i to evaluate fk DP(N,...,N with I = {,...,K} and J = φ by uing (5-(9. The following propoition how the optimality of the olution obtained by the propoed algorithm when K =. The proof i omitted due to pace limitation. Propoition : For K =, the propoed algorithm optimally olve P p in O(N. For K, the olution obtained by the propoed algorithm may be ub-optimal ince monotonicity of allocation threhold in Corollary only hold when K =.However, it will be hown in Section VI that the propoed algorithm till outperform the heuritic greedy method. For a general K, the computational complexity of the propoed algorithm canbehowntobeo(n K, which i polynomial in N. Given the expected rate {V (} and the tationary probability {π(}, the propoed dynamic programming can efficiently find olution of the allocation rule {a (} and the pricing rule {P (} to the problem P p. However, {V (} and {π(} depend on {a (} ince the tate tranition probability depend on {a (}. Therefore, we propoe to iteratively update {V (}, {π(}, and{a (}. The propoed mechanim deign algorithm for the networ election game i ummarized in Table II. In the numerical imulation, we oberved that the iterative algorithm exhibit very fat convergence. The typical number of iteration to converge i between 5 to 8. The propoed algorithm can be eaily modified to olve P u by replacing the min in (5 and (7 with the max, and changing the boundary condition in (8 to be f ( i i = i r= π(r, iv (r, i (λ w( i, i. VI. NUMERICAL SIMULATION In thi ection, we ue numerical imulation to verify the analyi and evaluate the performance of the propoed modified value iteration algorithm a the rational trategy. The propoed method i compared with the following cheme. We firt define the ocial welfare given a trategy profile σ a SW σ = S πσ ( K = R (, where π σ ( i the tationary probability at ytem tate. The centralized method i to exhautively earch through all the poible trategy profile and chooe the one that achieve the larget ocial welfare, i.e., σ cent = argmax σ SW σ. Thu, the centralized method require a computational complexity of O(K S, which i exponentially increaing in the number of ytem tate and i impoible to be ued in practice. The myopic trategy i obtained by chooing the larget

9 556 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL., NO., OCTOBER Strategy profile ( iteration iteration iteration iteration (converged Fig.. The threhold tructure of the trategy profile during iteration of the propoed algorithm. Strategy profile (σ immediate utility after maing the deciion, i.e., σ myop = arg max {,...,K} R ( +. In current cellular ytem, the cell election i done by chooing the bae-tation with the highet detected SNR. Such an approach i imilar to the myopic trategy ince it only concern about the immediate utility. Finally, the random trategy i to randomly mae the deciion with equal probability, i.e., Pr { σ rand = } = F(, F(, where denote the cardinality of a et. In the following imulation, the performance of the random trategy i obtained by averaging the performance of intance for each et of parameter. The algorithm analyi in Section IV how that there exit a threhold tructure of the trategie along each line of + = m, m {,,..., N}. We verify the analyi by numerical imulation in Figure, which illutrate the trategy profile computed by the propoed algorithm in a twonetwor ytem where P /N =5, P I /N =, T =.8 (ec, λ =.5 (uer/ec, λ =.5 (uer/ec, λ =.5 (uer/ec, μ =.5 (uer/ec, ɛ =.5 and N =8.The x-axi (y-axi denote (, i.e., the number of uer in networ (networ. The number mared at the coordinate =(, denote the computed trategy σ, which i either or in thi cenario. Thi figure how the trategy profile converge in iteration. The green (dot-dah line i drawn in between different trategie to emphaize the threhold. The threhold line of certain iteration (,, and are alo hown in the figure to illutrate the evolution of the trategy profile during the iteration of the propoed algorithm. It i oberved that at each iteration, the threhold tructure of the trategie alway exit along the diagonal line a the analyi in Section IV. In the ret of imulation, intead of pecifying the arrival rate and the time lot duration, we conider the parameter a tranition probabilitie ince the relative value of thee probabilitie directly influence the reulting performance. Figure 4 how the converged trategy profile of a three-networ ytem, where P /N =5, P I /N =, λ =., λ =., λ =., λ =., μ =., ɛ =.5 and N =5. It i oberved that the trategy profile alo ha a threhold tructure. Figure 5 validate the individual rationality of the propoed Fig. 4. ytem. The threhold tructure of the trategy profile for a three-networ Expected utility of the deciion maer 4 uer, λ =., λ =., λ =., μ =.5, R ( = log(+p S /(( P I + N.4 Centralized Propoed Probability of deviation Fig. 5. Comparion of the propoed method and the centralized method for the deciion maer expected utility veru probability of deviation. method in a two-networ ytem, where the parameter are et to be P /N =5, P I /N =, λ =., λ =., λ =., μ =.5, ɛ =.5, andn =4. The deciion maer expected utility, defined a E[V σ ( + e σ ], ievaluated veru the probability of deviation p d. For computational tractability of the centralized method, the number of uer N i et to be 4. Note that the time lot duration i choen to enure that λ + λ + λ +Nμ but the relative value of thee probabilitie are retained. The uer at tate deviate from the given trategy σ with probability p d.the deciion maer expected utility can only be impaired if he deviate from the trategy profile generated by the propoed method. However, by deviating from the centralized trategy that maximize the ocial welfare, the uer can poibly obtain higher expected utility (about 7% performance improvement in Figure 5. Clearly, the individual rationality i not atified for the centralized trategy. Figure 6(a and 6(b how the comparion of the deciion maer expected utility with different trategy profile in a two-networ ytem where P /N = 5, P I /N =,

10 YANG et al.: WIRELESS ACCESS NETWORK SELECTION GAME WITH NEGATIVE NETWORK EXTERNALITY 557 Expected utility of the deciion maer (normalized w.r.t. Myopic Eexpected utility of the deciion maer (normalized w.r.t. Myopic uer, λ =., λ =., μ =.5, R ( = log(+p S /(( P I + N.6 Myopic Centralized.5 Propoed Random λ (a The deciion maer expected utility veru λ uer, λ =., λ =., μ =.5, R ( = log(+p S /(( P I + N λ Myopic Centralized Propoed Random (b The deciion maer expected utility veru λ. Fig. 7. Social Welfare (normalized w.r.t. Myopic Social Welfare (normalized w.r.t. Myopic 4 uer, λ =., λ =., μ =.5, R ( = log(+p S /(( P I + N Myopic Centralized Propoed Random λ (a The ocial welfare veru λ. 4 uer, λ =., λ =., μ =.5, R ( = log(+p S /(( P I + N Myopic Centralized Propoed Random λ (b The ocial welfare veru λ. Comparion of different trategie for the ocial welfare. Fig. 6. utility. Comparion of different trategie for the deciion maer expected 4 uer, λ =., λ =., λ =.5, μ =.5, R ( = log( + P S /(( P I + N 4 λ =., μ =.5, ɛ =.5, andn =4. We ue the myopic trategy a the baeline by normalizing the performance of other method with that of the myopic trategy. In Figure 6(a, λ =. and λ i varied from.5 to.75. In Figure 7(b, λ =. and λ i varied from.5 to.75. It can be een that the propoed method perform the bet among all the cheme ince the deciion maer optimize hi expected utility by chooing networ to hi bet advantage. The myopic trategy alway ha performance due to the normalization. The random trategy i wore than the myopic method which exploit the information of the immediate utility. The centralized method perform the wort becaue it maximize the ocial welfare and reult in acrificing the deciion maer expected utility. In Figure 7(a and 7(b, we compare the ocial welfare performance of the trategy profile generated by different approache in a two-networ ytem where the parameter are P /N =5, P I /N =, λ =., μ =.5, ɛ =.5 and N =4. In Figure 7(a, λ =. and λ i varied from.5 to.75. In Figure 7(b, λ =. and λ i varied from.5 to.75. The performance of each method i normalized by the myopic one. It can be een that the propoed method perform imilar to that of the centralized method which maximize the ocial welfare. Figure 8 how the impact of ɛ on the number of iteration for the trategy profile to converge uing the propoed modified value iteration algorithm. It can be een Number of iteration to converge ε Fig. 8. The impact of ɛ on the number of iteration for the trategy profile to converge. that when ɛ increae, it require maller number of iteration to converge ince the region of tolerance for witching among the trategy profile i larger, and poibly more ɛ-approximate NE are available. Figure 9 and how the performance comparion for different mechanim deign when K = and K =, repectively. The exhautive earch i to earch over all poible allocation rule and find out the one with the optimal objective value. The greedy algorithm i characterized by the

11 558 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL., NO., OCTOBER Expected payment λ = λ =., μ =., N=, K= Propoed Greedy Exhautive earch λ Fig. 9. Comparion of different mechanim deign for the expected payment veru λ when K =. Expected payment λ = λ = λ =., μ =., N=, K= Propoed Greedy Exhautive Search λ Fig.. Comparion of different mechanim deign for the expected payment veru λ when K =. following recurive formula. f G ({ i,i I} { j,j J} { =min f G ( i { j,j J {i}} }, (4 i I a i ( i, i, j, j =, i i, where i { =argmin f G ( i j,j J {i} }. (4 i I The boundary condition i f G ( i i =f G ( w i ( i, i i i w i ( i, i + π( i, i V i ( i +, i w i ( i, i, (4 a i ( i, i, j, j =, i i, where i i the minimizor in (4 when =. (4 The greedy algorithm i to evaluate fk G (N,...,N with I = {,...,K} and J = φ by uing (4-(4. With a imilar analyi, the computational complexity of the greedy algorithm can be hown to be O(N K. Compared with the propoed DP algorithm, the greedy method i a heuritic approach which mae a local optimal deciion according to lower dimenional reult. We can ee more clearly by conidering the cae K =, i.e., f G (, =min { f G (,f G ( }, (44 (a (,,a (, = (,,, if f G ( >f G (, (45 (a (,,a (, = (,,, if f G ( f G (. (46 For example, when evaluating f G(N,N, iff G( = N = N i larger than f G ( = N = N, then tate (N,N i allocated to networ. Due to Lemma, the tate {(,N, N} are all allocated to networ. Since the unallocated tate o far are {(,, N, N }, we can then evaluate f G (N,N, and o on. In Figure 9, we can ee that the propoed DP algorithm can achieve the ame performance a the exhautive earch when K =, but require only a polynomial time complexity. The greedy algorithm ha a wore performance ince it mae a local optimal deciion to determine the threhold of allocation rule. In Figure, different mechanim deign approache are compared for K =. It can be een that the propoed DP algorithm till outperform the greedy method. A dicued in Section V, for a general K the propoed DP algorithm may not achieve the global optimum. However, with much lower complexity compared to the exhautive earch, the propoed algorithm can achieve reaonably good reult and thu can erve a an approximate approach. VII. DISCUSSION Although we focu on the wirele acce networ election problem in thi paper, we hould notice that the model decribed in thi wor i very general and can be applied into many other problem. A cloely related cenario i the cell election problem in cellular networ [47] [49]. When a mobile tation deire to inform the cellular ytem whether it i on the air, it regiter to a bae tation which correpond to a cellular cell. In mot current cellular ytem, the cell election proce i imply accomplihed by a local ignal-tonoie ratio (SNR-baed trategy, which i to detect the SNR of each cell and chooe the cell with the larget SNR [48]. However, uch a imple trategy doe not tae into account the trategie of other, i.e., the negative networ externality. The QoS experienced by a mobile tation will be degraded if the limited reource are hared with a large number of uer. The utilization of ytem reource will alo be degraded ince uch a trategy reult in cellular cell with unbalanced load. It can be een that the cell election problem ha the ame tructure with the wirele acce networ election problem. Mobile tation equentially chooe one cellular cell (correponding to a bae tation to regiter baed on the obtained information about each available cell. The utility of a mobile tation i determined by the expected throughput during the period it tay in the cell. Furthermore, the intantaneou throughput of a mobile tation in a certain cell i affected by the crowdedne of the cell due to the limited bandwidth and the delay caued by the cheduling overhead. Thu, a rational mobile tation hould chooe a cellular cell in conideration of other mobile tation deciion to avoid the crowdedne. VIII. CONCLUSION In thi paper, we have tudied the wirele acce networ election problem a a tochatic game with negative networ

12 YANG et al.: WIRELESS ACCESS NETWORK SELECTION GAME WITH NEGATIVE NETWORK EXTERNALITY 559 externality, where a uer decide which networ to connect to by conidering ubequent uer deciion. The problem i hown to be a multi-dimenional MDP. We propoe a modified value iteration algorithm to obtain the optimal trategy profile for each elfih uer. The analyi of the propoed algorithm how that the reulting trategy profile exhibit a threhold tructure along each diagonal line. Such a threhold tructure can be ued to ave the torage pace of the trategy profile from O(N to O(N log N in the two-networ cenario. Simulation reult are hown to validate the analyi and demontrate that rational uer will not deviate from the trategy profile obtained by the propoed algorithm. For the expected utility of the deciion maer, the propoed method i uperior to other approache. Moreover, it ocial welfare performance i hown to be imilar to that of the centralized trategy which maximize the ocial welfare. We further invetigated truth-telling enforcing mechanim deign in the wirele acce networ election problem. The mechanim deign capture the incentive compatibility and individual rationality contraint while optimizing the utility of uer. The formulated problem a a mixed integer program in general doe not have an efficient olution. By exploiting the optimal ubtructure, a dynamic programming algorithm i propoed to optimally olve the mixed integer programming problem in the two-networ cenario. For the multi-networ cenario, the propoed algorithm can outperform the heuritic greedy approach in a polynomial-time complexity. Finally, imulation reult ubtantiate the optimality in the twonetwor cae and alo demontrate the effectivene of the propoed algorithm in the multi-networ cenario. REFERENCES [] V. Chandraehar, J. Andrew, and A. Gatherer, Femtocell networ: aurvey, IEEE Commun. Mag., vol. 46, no. 9, pp , Sept. 8. [] M. Ahmed, Call admiion control in wirele networ: a comprehenive urvey, IEEE Commun. Survey Tut., vol. 7, no., pp , 5. [] D. Gao, J. Cai, and K. N. Ngan, Admiion control in IEEE 8.e wirele LAN, IEEE Netw., vol. 9, no. 4, pp. 6, 5. [4] W. H. Sandholm, Negative externalitie and evolutionary implementation, Rev. Economic Studie, vol. 7, no., pp , July 5. [5] G. Fagiolo, Endogenou neighborhood formation in a local coordination model with negative networ externalitie, J. Economic Dynamic Control, vol. 9, no. -, pp. 97 9, 5. [6] J. Hou and D. C. O Brien, Vertical handover-deciion-maing algorithm uing fuzzy logic for the integrated radio-and-ow ytem, IEEE Tran. Wirele Commun., vol. 5, no., pp , Nov. 6. [7] K. Mittal, E. M. Belding, and S. Suri, A game-theoretic analyi of wirele acce point election by mobile uer, Comput. Commun., vol., no., pp. 49 6, 8. [8] D. Niyato and E. Hoain, Dynamic of networ election in heterogeneou wirele networ: an evolutionary game approach, IEEE Tran. Veh. Technol., vol. 58, no. 4, pp. 8 7, May 9. [9] Q. Song and A. Jamalipour, Networ election in an integrated wirele LAN and UMTS environment uing mathematical modeling and computing technique, IEEE Tran. Wirele Commun., vol., no., pp. 4 48, June 5. [] A. Argento, M. Ceana, N. Gatti, and I. Malanchini, A game theoretical tudy of acce point aociation in wirele meh networ, Comput. Commun., vol. 5, no. 5, pp ,. [] D. Charila, O. Marai, D. Niitopoulo, and M. Theologou, Pacetwitched networ election with the highet QoS in 4G networ, Comput. Netw., vol. 5, no., pp , 8. [] M. Ceana, N. Gatti, and I. Malanchini, Game theoretic analyi of wirele acce networ election: model, inefficiency bound, and algorithm, in Proc. 8 Intl. Conf. Performance Evaluation Methodologie Tool, pp. 6: 6:. [] A. Balachandran, P. Bahl, and G. Voeler, Hot-pot congetion relief in public-area wirele networ, in Proc. IEEE Worhop Mobile Comput. Syt. Appl., pp [4] L. Chen, A ditributed acce point election algorithm baed on noregret learning for wirele acce networ, in Proc. IEEE VTC Spring, pp. 5. [5] M. Satyanarayanan, Pervaive computing: viion and challenge, IEEE Peronal Commun., vol. 8, no. 4, pp. 7, Aug. [6] J. Konori, Multihomed wirele terminal: MAC configuration and networ election game, in Proc. Intl. Conf. Inf. Netw., pp [7] F. Xu, C. Tan, Q. Li, G. Yan, and J. Wu, Deigning a practical acce point aociation protocol, in Proc. IEEE INFOCOM. [8] D. Fudenberg and J. Tirole, Game Theory. MIT Pre, 99. [9] B. Wang, Y. Wu, and K. J. R. Liu, Game theory for cognitive radio networ: an overview, Comput. Netw., vol. 54, no. 4, pp , Oct.. [] Z. Han and K. J. R. Liu, Noncooperative power-control game and throughput game over wirele networ, IEEE Tran. Commun., vol. 5, no., pp , Oct. 5. [] B. Wang, Z. Han, and K. J. R. Liu, Ditributed relay election and power control for multiuer cooperative communication networ uing Stacelberg game, IEEE Tran. Mobile Comput., vol. 8, no. 7, pp , July 9. [] K. J. R. Liu and B. Wang, Cognitive Radio Networing and Security: A Game Theoretical View. Cambridge Univerity Pre,. [] Y. Chen and K. J. R. Liu, Indirect reciprocity game modelling for cooperation timulation in cognitive networ, IEEE Tran. Commun., vol. 59, no., pp , Jan.. [4] L. Xiao, Y. Chen, W. Lin, and K. J. R. Liu, Indirect reciprocity ecurity game for large-cale wirele networ, IEEE Tran. Inf. Forenic Security, vol. 7, no. 4, pp. 68 8, Aug.. [5] C. Tein, M. Liu, R. Southwell, J. Huang, and S. Ahmad, Atomic congetion game on graph and their application in networing, IEEE/ACM Tran. Netw., vol., no. 5, pp , Oct.. [6] Y. Xu, J. Wang, Q. Wu, A. Anpalagan, and Y.-D. Yao, Opportunitic pectrum acce in cognitive radio networ: global optimization uing local interaction game, IEEE J. Sel. Area Commun., vol. 6, no., pp. 8 94, Apr.. [7], Opportunitic pectrum acce in unnown dynamic environment: a game-theoretic tochatic learning olution, IEEE Tran. Wirele Commun., vol., no. 4, pp. 8 9, Apr.. [8] Y. Xu, A. Anpalagan, Q. Wu, L. Shen, Z. Gao, and J. Wang, Deciiontheoretic ditributed channel election for opportunitic pectrum acce: trategie, challenge and olution, IEEE Commun. Survey Tut.,. [9] C.-Y. Wang, Y. Chen, and K. J. R. Liu, Chinee retaurant game, IEEE Signal Proce. Lett., vol. 9, no., pp , Dec.. [], Sequential Chinee retaurant game, IEEE Tran. Signal Proce., vol. 6, no., pp , Feb.. [] C. Jiang, Y. Chen, Y.-H. Yang, C.-Y.Wang, and K. J. R. Liu, Dynamic Chinee retaurant game in cognitive radio networ, in Proc. IEEE INFOCOM. [] M. L. Puterman, Marov Deciion Procee: Dicrete Stochatic Dynamic Programming. John Wiley & Son, 994. [] D. Bertea, Dynamic Programming and Optimal Control, rd ed. Athena Scientific, 7. [4] R. B. Myeron, Optimal auction deign, Mathematic Operation Reearch, vol. 6, no., pp. 58 7, 98. [5] V. Krihna, Auction Theory. Academic Pre/Elevier, 9. [6] N. Nian, T. Roughgarden, E. Tardo, and V. Vazirani, Algorithmic Game Theory. Cambridge Univerity Pre, 7. [7] E. J. Friedman and D. C. Pare, Pricing WiFi at tarbuc: iue in online mechanim deign, in Proc. ACM Conf. Electron. Commerce, pp [8] R. Vohra, Mechanim Deign: A Linear Programming Approach. Cambridge Univerity Pre,. [9] J. Zander, Performance of optimum tranmitter power control in cellular radio ytem, IEEE Tran. Veh. Technol., vol. 4, no., pp. 57 6, 99. [4] T.-H. Lee, J.-C. Lin, and Y. Su, Downlin power control algorithm for cellular radio ytem, IEEE Tran. Veh. Technol., vol. 44, no., pp , 995. [4] E. Parzen, Stochatic Procee. Holden-Day, 96. [4] M. Lopez-Benitez and F. Caadevall, Dicrete-time pectrum occupancy model baed on Marov chain and duty cycle model, in Proc. IEEE DySPAN, pp

13 56 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL., NO., OCTOBER [4] J.-L. Wang, Y.-H. Xu, Z. Gao, and Q.-H. Wu, Dicrete-time queuing analyi of opportunitic pectrum acce: ingle uer cae, Frequenz, vol. 65, no. -, pp. 5 4,. [44] J. Razavilar, K. J. R. Liu, and S. Marcu, Jointly optimized bitrate/delay control policy for wirele pacet networ with fading channel, IEEE Tran. Commun., vol. 5, no., pp , Mar.. [45] G. Koole, Monotonicity in Marov reward and deciion chain: theory and application, Found. Trend. Stoch. Syt., pp. 76, 7. [46] E. Cil, E. Ormeci, and F. Karaemen, Effect of ytem parameter on the optimal policy tructure in a cla of queueing control problem, Queueing Syt., vol. 6, pp. 7 4, 9. [47] A. Sang, X. Wang, M. Madihian, and R. Gitlin, Coordinated load balancing, handoff/cell-ite election, and cheduling in multi-cell pacet data ytem, Wirele Netw., vol. 4, pp., 8. [48] D. Amzallag, R. Bar-Yehuda, D. Raz, and G. Scaloub, Cell election in 4G cellular networ, in Proc. 8 IEEE INFOCOM, pp [49] L. Gao, X. Wang, G. Sun, and Y. Xu, A game approach for cell election and reource allocation in heterogeneou wirele networ, in Proc. IEEE SECON, pp Yu-Han Yang (S 6 received hi B.S. in electrical engineering in 4, and two M.S. degree in computer cience and communication engineering in 7, from National Taiwan Univerity, Taipei, Taiwan. He i currently puruing the Ph.D. degree in Univerity of Maryland, College Par, USA. Hi reearch interet include wirele communication and ignal proceing. He received Cla A Scholarhip from the ECE department, National Taiwan Univerity in Fall 5 and Spring 6. He i a recipient of Study Abroad Scholarhip from Taiwan (R.O.C. government in 9-. Yan Chen (S 6-M received the Bachelor degree from Univerity of Science and Technology of China in 4, the M. Phil degree from Hong Kong Univerity of Science and Technology (HKUST in 7, and the Ph.D. degree from Univerity of Maryland College Par in. From to, he i a Potdoctoral reearch aociate in the Department of Electrical and Computer Engineering at Univerity of Maryland College Par. Currently, he i a Principal Technologit at Origin Wirele Communication. He i alo affiliated with Signal and Information Group of Univerity of Maryland College Par. Hi current reearch interet are in ocial learning and networing, behavior analyi and mechanim deign for networ ytem, multimedia ignal proceing and communication. Dr. Chen received the Univerity of Maryland Future Faculty Fellowhip in, Chinee Government Award for outtanding tudent abroad in, Univerity of Maryland ECE Ditinguihed Diertation Fellowhip Honorable Mention in, and wa the Finalit of A. Jame Clar School of Engineering Dean Doctoral Reearch Award in. Chunxiao Jiang (S 9-M received hi B.S. degree in information engineering from Beijing Univerity of Aeronautic and Atronautic (Beihang Univerity in 8 and the Ph.D. degree from Tinghua Univerity (THU, Beijing in, both with the highet honor. During -, he viited the Signal and Information Group (SIG at Department of Electrical & Computer Engineering (ECE of Univerity of Maryland (UMD, upported by China Scholarhip Council (CSC for one year. Dr. Jiang i currently a reearch aociate in ECE department of UMD with Prof. K. J. Ray Liu, and alo a pot-doctor in EE department of THU. Hi reearch interet include the application of game theory and queuing theory in wirele communication and networing and ocial networ. Dr. Jiang received the Beijing Ditinguihed Graduated Student Award, Chinee National Fellowhip and Tinghua Outtanding Ditinguihed Doctoral Diertation in. Chih-Yu Wang (S 7-M received the B.S. degree in Electrical Engineering from National Taiwan Univerity, Taipei, Taiwan. in 7. He ha been a viiting tudent in Univerity of Maryland, College Par in. He received the Ph.D. degree in the Graduate Intitute of Communication Engineering, National Taiwan Univerity, Taipei, Taiwan, in. Hi reearch interet mainly are application of game theory in wirele networing and ocial networing. K. J. Ray Liu (F wa named a Ditinguihed Scholar-Teacher of Univerity of Maryland, College Par, in 7, where he i Chritine Kim Eminent Profeor of Information Technology. He lead the Maryland Signal and Information Group conducting reearch encompaing broad area of ignal proceing and communication with recent focu on cooperative and cognitive communication, ocial learning and networ cience, information forenic and ecurity, and green information and communication technology. Dr. Liu i the recipient of numerou honor and award including IEEE Signal Proceing Society Technical Achievement Award and Ditinguihed Lecturer. He alo received variou teaching and reearch recognition from Univerity of Maryland including univerity-level Invention of the Year Award; and Poole and Kent Senior Faculty Teaching Award, Outtanding Faculty Reearch Award, and Outtanding Faculty Service Award, all from A. Jame Clar School of Engineering. An ISI Highly Cited Author, Dr. Liu i a Fellow of IEEE and AAAS. Dr. Liu i Preident of IEEE Signal Proceing Society where he ha erved a Vice Preident Publication and Board of Governor. He wa the Editorin-Chief of IEEE Signal Proceing Magazine and the founding Editor-in- Chief of EURASIP Journal on Advance in Signal Proceing.