Joint iterative beamforming and power adaptation for MIMO ad hoc networks

Transcription

1 RESEARCH Open Access Jont teratve beamformng and power adaptaton for MIMO ad hoc networks Engn Zeydan *, Ddem Kvanc 2, Ufuk Turel 2 and Crstna Comancu Abstract In ths paper, we present dstrbuted cooperatve and regret-matchng-based learnng schemes for jont transmt power and beamformng selecton for multple antenna wreless ad hoc networks operatng n a mult-user nterference envronment. Under the total network power mnmzaton crteron, a jont teratve approach s proposed to reduce the mutual nterference at each node whle ensurng a constant receved sgnal-to-nterference and nose rato at each recever. In cooperatve and regret-matchng-based power mnmzaton algorthms, transmt beamformers are selected from a predefned codebook to mnmze the total power. By selectng transmt beamformers judcously and performng power adaptaton, the cooperatve algorthm s shown to converge to a pure strategy Nash equlbrum wth hgh probablty n the nterference mpared network. The proposed cooperatve and regret-matchng-based dstrbuted algorthms are also compared wth centralzed solutons through smulaton results. Keywords: MIMO, ad hoc networks, game theory, beamformng Introducton Multple-nput multple-output (MIMO) communcaton technques have been shown to boost the capacty and spectral effcency of wreless communcaton systems [,2]. MIMO wreless systems can sustan more smultaneous transmssons n a reduced area through nterference management [3]. When transmsson parameters such as transmt power, beamformer selecton, frequency, modulaton, transmsson rate are modfed to adapt to the nterference envronment, MIMO systems gan an addtonal advantage. Adaptve wreless systems can acheve system effcency, lower computatonal complexty, and overhead compared to a centralzed system. Transmt beamformng has been the focus of extensve research n the lterature [4-] and desgnng optmum sgnalng at the transmtter can lead to sgnfcant mprovements for systems operatng n varyng nterference [4,6,2-6]. In spatal transmt beamformng, each communcatng node s symbol stream s multpled by a preselected transmt beamformng weght vector for transmsson through multple antennas such that the * Correspondence: ezeydan@gmal.com Department of Electrcal and Computer Engneerng, Stevens Insttute of Technology, Hoboken, NJ 73, USA Full lst of author nformaton s avalable at the end of the artcle overall nterference due to other multple nodes s mnmzed. Adaptve optmzng of transmtter beamformng mproves effcency by steerng the beam toward the ntended recever, whle placng nulls toward the unntended recevers n order to avod causng excessve nterference to them. Transmtters may adapt ther sgnals usng a low-rate feedback from the recever [7]. A power control mechansm can also be combned wth lmted rate feedback from the recever n order to satsfy certan Qualty-of-Servce (QoS) requrements at the recever [8-2]. In general, MIMO beamformng technques n communcaton systems are addressed n three dfferent systems: pont-to-pont, cellular, and ad hoc networks. The great potental of MIMO n pont-to-pont communcaton s shown n [,4,6,2] and lnear precoders (egencoders) and beamformers have been desgned for pontto-pont MIMO lnks n [5,7]. In cellular networks, beamformng algorthms mnmze the total power and enhance capacty for array-equpped base statons and sngle antenna moble transmtters [8-]. In ad hoc networks, wthout a central controller, dstrbuted beamformng technques ncrease system throughput and lower energy consumpton [2,22-24]. However, optmzaton solutons desgned for ad hoc networks need 2 Zeydan et al; lcensee Sprnger. Ths s an Open Access artcle dstrbuted under the terms of the Creatve Commons Attrbuton Lcense ( whch permts unrestrcted use, dstrbuton, and reproducton n any medum, provded the orgnal work s properly cted.

2 Page 2 of 2 careful study, because the envronment s nterference lmted and the performance of MIMO technques depends sgnfcantly on the overhead ntroduced by the proposed algorthms. Dstrbuted spatal beamformng algorthms are proposed for mult-user ad hoc MIMO networks n [23,24] under channel recprocty condtons. Channel recprocty holds when the channel matrx at the recever s the transpose of the channel matrx at the transmtter, ths s usually assumed n tme-dvson duplex (TDD) systems [24]. Bromberg [24] consder the capacty maxmzaton problem and propose a locally enabled global optmzaton (LEGO) algorthm for dstrbuted beamformng update under Gaussan other-user nterference. Ilts et al. [23] formulate the problem as a non-cooperatve game for overall power mnmzaton of the network under a constant QoS constrant (.e., target sgnal-tonterference plus nose rato (SINR)). The proposed teratve mnmum mean-square error (IMMSE) algorthm solves an optmzaton problem by computng transmt/receve beamformer pars and transmt powers n a dstrbuted manner [23]. In the IMMSE algorthm, the receve beamformer s the conjugate of the transmt beamformer and the algorthm reles on the channel recprocty condton. Hence, the IMMSE algorthm does not demand explct feedback schemes for channel state nformaton (CSI) at the recever. However, durng the updatng procedure of the IMMSE algorthm, transmsson overhead of tranng sequences and power control commands are ncurred. The amount of overhead ncreases wth teratons, snce the algorthm performs transmt/receve beamformer and power updates teratvely. Moreover, f the transmtter and recever use dfferent channels or frequences for transmsson and recepton,.e., when channel recprocty s not vald, CSI must be fed back to the transmtter, whch necesstates overhead. In order to lower the communcaton overhead between transmtter and recever when channel recprocty does not hold, a scheme to lmt feedback by quantzng the transmt beamformer n sngle user MIMO systems s proposed n [2]. The concept s based on selectng a codeword n a predetermned codebook that s known to both transmtter and recever. Selectng the transmt beamformer from a predefned codebook reduces overhead n nonrecprocal channels. Moreover, latency s reduced n hghly moble and unstable communcaton networks and user partcpaton s mnmzed. In ths scenaro, the recever only feeds back the ndex of the selected transmt beamformer to the transmtter. When there s no channel recprocty between transmtters and recevers, an teratve lmted feedback beamformng algorthm usng a predetermned codebook s proposed n [25]. The algorthm maxmzes the transmsson rate n MIMO mult-user ad hoc networks usng sequental dscrete transmt beamformer selecton updates. In each teraton, each node formulates ts best response strategy, whch maxmzes the receved SINR. However, the convergence of the algorthm has not been nvestgated. Game theory has enabled effcency and convergence proofs of some of the mportant problems n wreless communcatons such as dstrbuted power control algorthm desgn [26], jont code-dvson multple access (CDMA) waveform, and power control desgn [9,2,27] and optmum transmsson sgnalng strateges [28,29]. The applcaton of game theory to dstrbuted beamformng s problematc [23]. Lacatus and Popescu [2] and Popescu et al. [9] study jont CDMA codeword (or sequence) and power adaptaton as a noncooperatve game. The problem s formulated as a separable game usng noncooperatve convex games, wth correspondng sub-games: power control and codeword control game. However, n contrast to our jont optmzaton problem, the jont optmzaton of powers and CDMA codewords s nvestgated only over convex games (.e., the set of acton space s nonempty, compact, and convex [26,3]), and therefore the decson varables (.e., the powers and codeword sequences) are contnuous, not dscrete n these games. Optmum transmt sgnalng for rate maxmzaton n MIMO nterference systems has been studed usng game theory [2-6]. In these papers, the system s modeled as a noncooperatve game where every MIMO lnk s a player and computes aganst the others by choosng the transmt covarance matrx to maxmze ts own rate. Lang and Dandekar [3] nvestgate rate maxmzaton for MIMO ad hoc networks by performng power control. The exstence of a Nash equlbrum (NE) soluton s shown usng concave game analyss. The convergence of the proposed algorthms s not studed. Arslan et al. [4] show that ndvdual mutual nformaton maxmzaton s a concave game [3] n MIMO nterference channels, whch mples the exstence of a NE for arbtrary channel matrces. The equlbrum s provably unque when mult-user nterference (MUI) s suffcently small. Decentralzed algorthms usng local nformaton provde update strateges to determne the lnk parameters. As an extenson of ther work and for more general condtons, the unqueness of the NE soluton s provded n [5]. Scutar et al. [5] provde a unfed framework for the noncooperatve mutual nformaton maxmzaton problem for MIMO nterference systems. A unfed set of suffcent condtons guaranteeng the unqueness of the NE and the convergence of asynchronous water-fllng algorthm s provded for square nonsngular channel matrces. The analyss s based on nterpretng the MIMO water-fllng

3 Page 3 of 2 operator as a matrx projecton onto the convex and closed set of covarance matrces. In [6], same authors extend ther results for arbtrary channel matrces. However, these papers do not address the selecton of dscrete optmzed sgnalng. The exstence (or unqueness) of the NE soluton that s proven n [3,4] s vald ether for convex or concave games or for postve defnte covarance matrces that are well defned as a convex and closed set [5,6]. Cooperatve and noncooperatve algorthms for jont channel and power allocaton chosen from the dscrete strategy space are studed n [32] n the context of wreless mesh networks. However, the proposed noncooperatve algorthm s suboptmal and one of the adaptaton parameters (.e., channel adaptaton) s not followed after the frst teraton. Power mnmzaton usng dstrbuted algorthms wth transmt beamformer selecton s challengng especally n ad hoc networks. Unlke power control games, n beamformng games there s no natural orderng of the actons [23]. In MIMO ad hoc networks operatng n MUI envronments, the nterference at each user depends on the transmsson parameters of the other users. The beamformng decson of each user reshapes the nterference emtted to other lnks, n ways that may be dffcult to predct. Changng the beamformng vector mayreducenterferenceonsomelnks,whleother lnks may suffer from hgher nterference. The affected nodes wll then change ther own beamformng vectors, settng off an cascade of changes n the network. Moreover, f the node pars belong to dfferent regulaton enttes, the noncooperatve node pars may only want to mnmze ther own transmt power rather than the overall power. The analyss for the selecton of actons from the dscrete codebook set and convergence analyss s stll mssng for jont transmt beamformng and power adaptaton n the lterature. To the best of authors knowledge, the problem of jont dscrete transmt beamformng and power adaptaton has not been formalzed n mult-user MIMO ad hoc networks. In ths paper, we study a decentralzed approach for optmzng the transmt beamformer and power levels usng local nformaton and reasonable computatonal burden. We consder total power mnmzaton under a constant receved target SINR constrant. Our contrbutons n ths paper are twofold: Frst, we study an effcent cooperatve beamformng algorthm for global power mnmzaton problem wth convergence analyss. For the cooperatve algorthm, the amount of nformaton to be exchanged between nodes wll grow wth the number of teratons. Second, we study a noncooperatve regret-matchng learnng algorthm whch jontly selects transmt beamformer and power to mnmze the total power consumed by the network. The noncooperatve soluton reduces the amount of overhead by usng only local nformaton. We compare the performances of our proposed algorthms wth the optmal global soluton whch s found by exhaustvely searchng the entre feasble strategy space. The rest of ths paper s organzed as follows. Secton 2 outlnes the system model used n the paper. The optmzaton problem and ts game theoretcal nterpretaton are presented n Secton 3. The cooperatve wreless ad hoc network and noncooperatve counterpart are nvestgated Sectons 4 and 5, respectvely. The performance evaluaton of the proposed algorthms s provded n Secton 6. Fnally, Secton 7 concludes the paper. 2 System model and concepts In ths paper, we consder a wreless ad hoc network consstng of multple transmt and receve antenna node pars as shown n Fgure. All nodes are assumed to be usng the same channel. The nterference comes from the other node pars whch operate smultaneously Fgure Mult-user power control and lmted feedback transmt beamformng scheme for MIMO ad hoc networks. (t k ) m represents the mth row of the kth user s transmtter vector t k.

4 Page 4 of 2 on the same channels. In ths ad hoc network model, there are N node pars and each node par m Î {, 2,..., N} conssts of one transmtter node and one recever node. Each transmtter and recever node s equpped wth T antennas. Each node has a unt-norm receve/ transmt beamformer par (w m, t m )wthw m, t m C T. The complex symbol stream transmtted s b m C wth E{ b m 2 } =. The receved symbol stream s ˆbm C. The receved sgnal vector r m C T at the mth recevng node s gven by r m = P m H m,m t m b m + m P H m, t b + n m, () where H m, denotes the T TMIMO channel between the th transmttng node and the mth recevng node and s assumed to be quas-statc and P m s the power of the mth transmttng node. The addtve whte Gaussan nose terms n m C T have dentcal covarance matrces s 2 I T where s 2 s the nose power and I T s the T Tdentty matrx. We note that dfferent covarance matrces for nose wll not affect the performance of the proposed algorthms.notethatthefrsttermoftherght-handsdeof () s the desred sgnal, whereas the second term s the nterference from the other transmttng nodes. As we are nterested n the mnmum achevable power, we consder the worst case where all node pars always have some packets to transfer and all nodes n the network can transmt smultaneously. The network s assumed to be synchronous. The set of avalable code-book beamformers for the mth transmttng and recevng node par s denoted by m = {t m, t2 m,..., tϒ m } wth cardnalty ϒ. In a lmted feedback beamformng system, the recevng node selects a transmt beamformer from the codebook and feeds back the ndex of the selected beamformer. Each node can select between ϒ transmt beamformer vectors. Let t m Î Δ m be the selected transmt beamformer for the mth transmttng and recevng node par. Denote Θ =[t, t 2,..., t N ] T and P =[P, P 2,...,P N ] T as the transmt beamformer selecton and transmsson power vectors for N nodes, respectvely. The T Tthe nterference plus nose covarance matrx at the mth recevng node s R m ( m, P m )= m P H m, t t H H H m, + σ 2 I, (2) where Θ -m and P -m are the transmt beamformers and powers of nodes other than m. An antenna beam pattern that adjusts the antenna gans to form nulls toward the drectons of the nterferers whle keepng a constant gan toward the drectons of the mult-path of the ntended recever can be desgned usng receve antenna arrays. The mnmum varance dstortonless response beamformer [23,33] can adjust the array weghts properly such that the sum of nterference and nose s mnmzed. The normalzed receve beamformer at mth recevng node s w m = ŵ m ŵ m, (3) where ŵ m = R m H m,mt m. The resultng receved SINR at the mth recevng node due to desred transmtter of mth node par s Ɣ m = P m w H m H m,mt m 2 m P wh m H m,t 2 + σ 2, (4) where w m 2 = t m 2 = for all m. The proposed dstrbuted algorthms attempt to acheve a target SINR by adjustng transmt powers. To construct a dstrbuted teratve lmted feedback beamformng scheme, let us frst consder the case when there s only one node par n the wreless network. The recever selects the transmt beamformer from the codebook Δ as t =arg max t Ɣ, where t s the optmal transmt beamformer selecton for one node par. Then, the recever returns the ndex of the beamformer for transmt beamformer selecton t and the receved normalzed SINR, (t )H H H, R H,t, through the low-rate feedback channel where R = I snce there s no nterference n a sngle user system. The transmtter selects the transmtter beamformer n order to mnmze ts own transmsson power P, where P s updated as P = γ, (t )H H H, R H,t (6) where g s the target SINR value. Consder now the case where N node pars coexst n the wreless network. Note that for each node par m, the value of receved SINR,.e., Γ m,safunctonof(θ, P). Therefore, the transmt power of one node par depends not only on the transmt beamformer t selects, but also on the transmt power and beamformer selecton of other nodes n the network. Furthermore, n beamformng, f user m changes ts transmt beamformer t to ncrease ts own SINR Γ, t can ether ncrease or decrease Γ m,thesinroflnkm, dependng on the relatve postons of the nodes. Therefore, desgnng an optmal dstrbuted algorthm whch converges to a set of beamformers to mnmze the overall transmt power whle meetng target SINRs for all node pars s not a straghtforward task.

5 Page 5 of 2 3 Optmzaton problem and game theoretcal nterpretaton The goal s to mnmze the transmt power of all nodes m Î {,2,...,N} under constant target SINR g.the optmzaton problem can be defned as, Mnmze,P N P m, (7) m= subject to Γ m g, w m = t m =, P mn < P m P max, m Î {, 2,..., N}, where P mn and P max are the mnmum and maxmum transmt powers, respectvely. We consder the above problem as a normal form game whch can be mathematcally defned by the trplet = N, C, {Um } N m= where N = {, 2,..., N} s the fnte set of players of the game, C = C C 2 C N represents the set of all avalable actons for all the players and {U m } N m= : C R s the set of utlty functons that the players assocate wth ther strateges. The actons c m Î C m for a player m are the selecton of transmt powers P m Î [P mn, P max ]and the transmt beamformer t m Î Δ m. Players select actons to maxmze ther utlty functons. One of the questons that arse s f there exsts a convergence pont, a set of strateges, n our case a set of beamformng selectons Θ =[t,...,t m,...,t N ] T and power allocatons P =[P, P 2,..., P N ] T from whch no player would devate. In game theory such a set of strateges s called a Nash equlbrum (NE). A NE for a game s a set of strategy profles c =[c, c 2,..., c N ]fromwhchno player can ncrease hs utlty by unlateral devatons. A strategy profle (c m,c -m ) s a NE f and only f U m (c m, c m ) U m (c m, c m) m N, c m, c m C m,(8) where (c m, c m) refers to the strategy profle n whch the acton of user m s changed from c m to c m whle the actons of all the other players n the game reman the same. In the followng sectons, we wll dscuss the scenaros where the node pars are cooperatve and noncooperatve respectvely n order to search for the best results and provde convergence guarantees. 4 Cooperatve and noncooperatve beamformng for MIMO ad hoc networks 4. Optmal (centralzed) soluton In a wreless ad hoc network wth a centralzed agent, the transmt beamformers and the correspondng transmt powers can be jontly selected to mnmze the total transmt power of all transmttng antennas as, (, P ) = arg mn,p N P m (, P m ), (9) m= where =[t, t 2,..., t N ]T and P =[P, P 2,..., P N ]T are the optmal transmt beamformer and power solutons, respectvely. The transmt power P m of mth node par s defned as P m (, P m )= γ t H m HH m,m R m H m,mt m. () where R m s a functon of (Θ -m, P -m ) as shown n (2). A nave approach for solvng the problem s to nvestgate all strategy profles Θ =[t,...,t m,...,t N ] T exhaustvely (note that for a gven fxed strategy profle Θ, the correspondng power profle P can be computed usng () for each ndvdual node par m). In order to compute (9), the centralzed agent evaluates the total network power for ϒ N possble beamformng vector combnatons. For example, for a network sze wth node pars where each user has to select from a code-book of sze ϒ =6 beamformers, the search space s 6 strategy profles. Consequently, fndng the centralzed transmt beamformer s cumbersome n large-scale wreless ad hoc network. To allevate the complexty problem, whle mantanng good performance results, we propose two decentralzed power mnmzaton algorthms usng cooperatve and noncooperatve technques. 4.2 Cooperatve power mnmzaton usng beamformng In ths secton, we consder scenaros where all node pars n the wreless network are cooperatng. In a cooperatve game, nodes n the network are able to coordnate and select the transmt beamformers accordngly. We want to fnd the optmal transmt beamformer and power assgnments such that the total power by all the nodes n the network s mnmzed. The objectve functon can be wrtten as U network (, P) = N P m (, P m ) () m= We assume that each user s utlty functon s (). That s, U (, P) =U network (, P) N = P m (, P m ), N. m= (2) In other words, we model the game as an dentcal nterest game whch s a specal case of potental games [34]. It s easy to verfy that all dentcal nterest games have at least one pure NE, whch wll represent any acton profle that maxmzes U network (Θ, P) [4,32]. We analyze a cooperatve power mnmzaton algorthm (COPMA) whch can converge to the optmal NE wth

6 Page 6 of 2 arbtrarly hgh probablty. Ths method s analogous to the decentralzed negotaton method called adaptve play [4]. The key characterstc of COPMA s the randomness delberately ntroduced nto the decson-makng process to avod reachng a local soluton. In COPMA, the choces of players (n our case transmt beamformer selectons) lead the system to the optmal NE soluton wth arbtrarly hgh probablty. Motvated by Song et al. [32], COPMA can be mplemented dstrbutvely as follows: Assume that each node par m n the network has an unque ID m and mantans two varables Pm current and Pm updated whch are the transmt power of the mth node par pror to and after the change of transmt beamformer, respectvely. The node pars can be chosen randomly or n a round-robn order for updatng of the transmt beamformers. Whenever a node par changes ts strategy, t broadcasts a vector [ID m, Pm current, Pm updated ] va a backbone network. After that, all the other node pars N \m wll set P current = P updated,recalculatep updated as the new transmt power and send the vector {ID, P current, P updated } to the updatng node par m. Fnally, the mth node par wll decde whether the new transmt beamformer should be kept or changed wth some probablty whch depends on p current and p updated whch are the total transmt power n the network pror to and after the random change of the transmt beamformer, respectvely. Note that snce p current and p updated are calculated by each node par ndependently, the unque ID of each node provdes a checklst to accurately add up transmt powers. For ths paper, we assume that unque node IDs are bult nto each node and n network tmng synchronzaton s perfect so that power updates are always receved n the correct round. The detaled descrpton of COPMA s provded as follows: Intalzaton: For each transmttng and recevng par m, the ntal ndex of transmt beamformers for all node pars s selected as one and the ntal transmt powers are set as P m = P max, m N. Repeat: Randomly choose a node par m as the updatng par wth probablty /N. Denotet m (n) Î Δ m as the current transmt beamformer of the mth node par at teraton n.. Set t m (n) =t m (n-), m N.CalculatePm current as n () m N. 2. To update node par m, randomly choose a transmt beamformer, tm updated m and calculate the transmt power requred when the updated transmt beamformer s used, Pm current as n (). Then, broadcast a data vector [ID m, Pm current, Pm updated ] to all other node pars N \m. 3. After recevng the data vector, for each, -IfP changes (due to change n nterference percevedattheth recever), every other node par N \m sets P current = P updated and calculates ts new transmt power from () and sets t to P updated. -IfP does not change, P current and P updated reman unchanged. After P current and P updated are updated for every other node par N \m n the network, send back the vector {ID, P current, P updated } to node par m. 4. Node par m computes the current total network power as P current = N m= Pcurrent m and updated total network power as P updated = N m= Pupdated m wth tm updated based on the receved power values from all other node pars N \m. 5. For a smoothng factor τ >,sett m (n) = t updated m for the mth node par wth probablty +exp((p updated P current )/τ). (3).e., the updatng node par m selects tm updated wth probablty (3). 6. The mth node par broadcasts a notfyng sgnal that contans the decson about whether the new transmt beamformer s kept. If not kept, every other node par N \m keeps P updated = P current Untl : Predefned number of teraton steps n =. Note that step-5 of the updatng rule mples that f tm updated yelds a better performance,.e., (P updated -P current) <,themth node par wll change to up-dated beamformer tm updated wth hgh probablty. Otherwse, t wll keep the current transmt beamformer wth hgh probablty. Note also that the tradeoff between COP- MA s performance and convergence speed s controlled by the parameter. τ. Largeτ represents extensve space search wth slow convergence, whereas small τ represents restraned space search wth fast convergence. The smoothng factor τ s selected to be a functon of the number of teratons n such that as n ncreases, τ. For example, we chose τ nversely proportonal to n 2 n our smulatons. The long-term behavor of COPMA s characterzed n the followng theorem. Theorem : Assume that the objectve of each node par s defned as the sum power mnmzaton n the network as defned n (9). Let Θ (k) =[t (k), t 2 (k),..., t N (k)] denote the profle of choces at step (or teraton) k n COPMA and =[t, t 2,..., t N ] the optmal profle. COPMA converges to the optmal NE wth arbtrarly hgh probablty. In other words,

7 Page 7 of 2 lm τ lm k + P τ ( (k) = ) =. (4) Proof The proof of Theorem follows smlar arguments as presented n [4,32,35]. Notce that the transmt beamformer selecton wth N players, each wth ϒ codebook sze, generates an N- dmensonal Markovan chan on a fnte state space wth ϒ N states or dfferent profles. We study the analyss for two-player games,.e., N = 2 dmensonal case as shown n Fgure 2. The analyss can be easly extended for mult-player games,.e., for an N-dmensonal Markovan chan. Let t m Î Δ m and t k Î Δ k be the choces of two players say m and k, and wthout loss of generalty assume that = m = k =[t m, t2 m,..., tϒ m ]. The players m and k can choose a transmt beamformer from Δ. LetΘ j denote the state [t m, tj n] ϒ 2 where the mth user selects the th transmt beamformer t m and the nth user selects the jth transmt beamformer t j n. At an arbtrary tme nstant, for any state of the Markovan chan, only one of the players can update ther transmt beamformer. Therefore, for example n Fgure 2, state j =[t m, tj n] can only transt nto a state ether n the same row or the same column. For any fxed τ >, the transton probablty from state j =[t m, tj n] ϒ 2 to state lp =[t l m, tp n] ϒ 2 s gven by P τ ( lp j )= 2ϒ(+e (P( lp) P( j ))/τ ), (5) where Θ j and Θ lp dffer n exactly one transmt beamformer selecton,.e., Θ j Θ lp for = l or j = p, τ s the smoothng factor of COPMA and P(Θ j ) s the mnmum total network power requred to reach target SINR g for both users at state Θ j calculated usng () for each user. If Θ j and Θ lp are dfferent n more than one poston, then P τ (Θ lp Θ j ) =. In addton, P τ (Θ j Θ j ) > s always true. Therefore, for any fxed τ >, the nduced Markov chan s rreducble and aperodc. The statonary dstrbuton P τ for each state can be obtaned from the followng balance equatons (usng the arrows n Fgure 2): ϒ p=,p j P τ ( j) P τ ( p j )= ϒ p=,p j P τ ( p) P τ ( j p ), (6) for all j ϒ 2 and p ϒ 2. Substtutng (5) nto (6) gves ϒ p=,p = ϒ p=,p P τ ( j) 2ϒ( + e (P( p) P( j ))/τ ) P τ ( p) 2ϒ( + e (P( j) P( p ))/τ ). (7) Then, the statonary dstrbuton of the nduced Markov chan at step k s obtaned as P τ ( (k)) = e P( (k))/τ (k) ϒ 2 e P( (k))/τ, (8) for arbtrary state (k) ϒ 2. Hence, from rreducblty and aperodcty of the Markovan chan, we have lm τ lm k + P τ ( (k) = ) =, (9) where ϒ 2. The result valdates that COPMA converges to the optmal state wth arbtrarly hgh probablty for two-player (N =2)caseandtheanalyss can easly be extended for general mult-player (N >2) cases as well. Wth the above theorem, the transmt beamformer and power level selectons are shown to reach the optmal NE soluton wth arbtrarly hgh probablty. One dsadvantage of cooperatve-based algorthms s that the communcaton overhead ncurred to calculate the total network power ncreases wth the number of teratons. In the next secton, we study a noncooperatve learnng algorthm usng local nformaton wth less computatons. Fgure 2 Two players markov chan for COPMA. 5 Regret-matchng-based jont transmt beamformer and power selecton game (RMSG) In ths secton, our goal s to obtan a dstrbuted learnng algorthm for jont transmt beamformer and power

8 Page 8 of 2 selecton scheme n MIMO ad hoc networks that requres only local nformaton for updates. We wll use a utlty functon for noncooperatve users. Note that the nteracton among N selfsh node pars can be defned as noncooperatve power mnmzaton game where each node par s attemptng to fnd ther own transmt beamformers to mnmze ther correspondng transmt powers. In the noncooperatve jont teratve beamformng and power adaptaton, the N node pars care only about ther own power mnmzatons exclusvely, rather than accountng for the overall network power. Each player s utlty functon depends on the choce of the transmt beamformer and ts own power, as well as on the other users selectons for transmt powers and beamformers va the perceved nterference. Note that the noncooperatve dstrbuted beamformng algorthms for mult-user MIMO ad hoc networks lack the qualty of strategc complementartes [36] that s found n power controlonly games [26]. It s thus not clear how to desgn an ordered set of actons for noncooperatve beamformng games. Instead, we study a noncooperatve learnng algorthm called the regret-matchng adaptve algorthm from [37], n whch the players choose ther actons based on ther regret for not choosng partcular actons n the past. The steady-state soluton of the regret-matchngbased learnng algorthm exhbts no regret and the probablty of choosng a strategy s proportonal to the player s regret for not havng chosen other strateges. Let t m denote the vector of all strateges or actons for user m,.e., t m =[t m, t2 m,..., tϒ m ] and t m() denotethe transmt beamformer vector selected by the mth user n teraton. Defne the average regret vector R t m m (k) of user m for an acton vector t m at teraton (or tme) k as R t m m (k) = k (U m ( t m, t m ()) U m (t m ())). (2) k = In the regret-matchng-based jont transmt beamformer and power selecton game (RMSG), each user m computes R t m m for every acton t m Î Δ m n all past steps when all other player s actons reman unchanged. Each player m updates ts regret R t m m (k) for every set of actons t m based on the followng recurson formula: R t m (k +)= k R t m k (k) + k (U m( t m, t m (k)) U m (t m (k))). (2) At every step k>, each user m updates ts own average regret vector R t m m (k) for every strategy n t m. In regret matchng, after computng the average regret vector, R t m m (k), each user m chooses an acton or strategy t m (k), k>, accordng to probablty dstrbuton ϕ t m (k) defned as [R t ϕ t m m (k)] + m (k) = Prob(t m (k) = t m )= t [R t m m m (k)] +, (22) where [x] + equals x when x s postve and zero otherwse. Notce that n the regret-matchng game, each user m chooses a strategy t m Î Δ m at any step wth a probablty proportonal to the average regret for not choosng that strategy t m Î Δ m n the past steps. The detaled summary of RMSG usng a Gauss-Sedel updatng scheme [5] s gven n Table where s the predefned number of teratons. Every fnte strategy game has a mxed strategy Nash equlbrum [3]. Usng a good learnng algorthm, any fnte game can be shown to converge to a mxed strategy Nash equlbrum. Regret-matchng-based selecton s dstrbuted and requres lmted nformaton exchange between the users f the utlty functon s properly selected. The tme-averaged behavor of regret-matchng game converges almost surely (wth probablty one) to the set of coarse-correlated equlbrum [34,38]. Therefore, the jont transmt beamformer and power selectons converges to a mxed strategy equlbrum soluton. In fact, n our jont transmt beamformer and power selecton game, the average regret of a user usng regret matchng becomes asymptotcally zero, whch s confrmed by our smulatons. The utlty functon of noncooperatve or selfsh users for the transmt beamformer and power selecton game at teraton k s U m ( tm, t m (k) ) =log(t H m HH m,m R m H m,mt m ). (23) Table Regret-Matchng-based jont transmt beamformer and power selecton game (RMSG) algorthm Intalzaton: For each transmttng and recevng par m, the ntal transmt beamformers are selected wth equal probablty, the ntal transmt powers are p m = p max and the ntal average regret vector s R t m () =, m N. Iteratons: For =, 2,..., For m =, 2,..., N - Update the average regret vector R t m (k) usng the recurson n (2) - Update the probablty dstrbuton ϕ t m (k) n (22) and select the transmt beamformer t m (k) based on updated ϕ t m (k). - Calculate the new transmt power p m based on selected t m (k) usng (). Next k Next m

9 Page 9 of 2 Note that by usng the above utlty functon, each user selects the transmt beamformer t m Î Δ m to maxmze ts own normalzed SINR, t H m HH m,m R m H m,mt m. The average regret n the recurson formula (2) can be updated locally as the best transmt beamformer s beng selected. 6 Smulaton results In ths secton, we nvestgate the performance results of centralzed optmzaton, COPMA, and noncooperatve regret-matchng (RMSG). We assume that the wreless ad hoc network has N homogeneous pars where each par has one transmtter node and one recevng node. Each entry n the channel matrx H m,k m, k N s assumed to be ndependent dentcally dstrbuted complex Gaussan dstrbuton wth zero mean and unt varance. We consder a rado propagaton channel wth path-loss exponent ν = 4. Ths mples that the fadng power s attenuated by d 4 m where d m s the dstance between transmtter and recever for mth node par. The target SINR g s selected to be db. We assume that channels do not vary durng the teratons. If channel condtons vary durng an teraton, ths wll change the optmzaton problem and the proposed algorthms performance degrades. However, dependng on the network confguraton and the parameters of the algorthm, lke the smoothng factor τ for COPMA, the network optmzer can set the convergence steps to be as small as possble whle tradng aganst performance degradaton n tme-varyng channels. The Grassmannan codebook of [2] s used for the smulaton results. The codebook sze s selected to be ϒ =6wth T = 3 antennas for all users. P max = mw (2 dbm) and P mn = mw ( dbm) n our smulatons. We assume sx dfferent transmt power levels:, 5, 2, 3, 5, and mw motvated by the IEEE 82.b standard n [39]. Note that the transmt powers are selected from ths dscrete power level set whch corresponds to celng functon of (). The selected network topologes are assumed to be feasble for the gven power levels [4]. The nose power s s 2 = W (-95 dbm) whch corresponds to approxmate thermal nose power for a bandwdth of 2 MHz. 6. Comparson of centralzed optmzaton, COPMA, and RMSG for N = 4 node pars We frst consder a small wreless ad hoc network wth 4 users,.e., N = 4. All transmttng and recevng nodes are randomly located n a square of 3 m 3 m area. We choose τ =./n 2 n our smulatons, where n denotes the teraton step. The global optmum soluton obtaned by enumeratng all feasble strateges,.e., 6 4 profles, s the performance benchmark. The maxmum number of teratons s = 2 for COPMA and RMSG. The total power consumed by the network s shown n Fgure 3. The global mnmum power soluton obtaned by centralzed optmzaton s the lower bound ontheoverallpowerconsumedbythenetwork.we observe that COPMA s performance mproves over tme and settles at the global optmum combnaton after 92 teratons. Note that 68 and 76% of the gan from usng COPMA algorthm s realzed wthn the frst 59 and 83 teratons, respectvely. RMSG algorthm dscussed n Secton 5 mnmzes the total transmt power n the network defned by (9) usng the utlty functon (23) n a noncooperatve manner. Fgure 3 also shows how the total power n the network vares over 2 teratons usng RMSG. Note that RMSG yelds nferor performance compared to COPMA n terms of the acheved overall power. However, the updatng procedure s noncooperatve and requres less overhead as the teratons contnue. The total network power converges to a value of 35 mw on the 68th teraton whereas the centralzed algorthm s soluton requres 65 mw total network power. Steady state s reached when all the users select a transmt beamformer ndex wth probablty one. Fgures 4 and 5 depct the trajectores of transmt beamformer selecton ndces and power trajectores n COPMA for each user n the network topology. At the ntalzaton step, each user starts wth maxmum power levels and frst ndex of transmt beamformer selectons. Then, each user updates teratvely followng COPMA algorthm, untl the optmum Nash equlbrum s acheved. Note that when the transmt beamformers Θ and power level vectors P converge n Fgures 4 and 5, the correspondng overall transmt power obtaned by COPMA s shown n Fgure 3. The exstence of NE and Total transmt Power (W) Iteraton RMSG COPMA Centralzed Fgure 3 Total transmt power versus teraton wth N =4, T = 3, and ϒ =6.

10 Page of 2 Transmt Beamformer Index User User 2 User 3 User Iteraton Fgure 4 Transmt beamformer ndexes versus teraton n COPMA wth N =4,T = 3, and ϒ =6. the convergence toward NE n COPMA are llustrated by the curves n Fgures 4 and 5. Probablty mass functon (p.m.f): In ths subsecton, we take a look at the probablty mass functon (p.m.f) ϕ t m of the RMSG algorthm calculated n (22). Fgure 6 represents the change n the p.m.f after, 2, 5, and teratons for one user. Intally, users choose the strateges,.e., transmt beamformers, wth equal probablty. The strateges are represented by the ndces to ϒ =6 n the x-axs and the probabltes of selectng these ndces are gven on the y-axs. After 2 teratons, the probablty of choosng transmt beamformer ndex 9 s hgher than that for any other transmt beamformer ndex, although the other probabltes for ndces 3, 4, and 2 are not totally elmnated. After 25 teratons, all other probabltes except those of 4 and 9 are elmnated. A statonary pont s reached when user P.m.f P.m.f Iteraton Iteraton chooses transmt beamformer ndex 9 n the th teraton. 6.2 Comparson of COPMA and RMSG for N = node pars We now consder a larger wreless ad hoc network wth N = node pars randomly located on a m m area. The smoothng factor for COPMA s selected as τ =2/n 2 n order to search more effcently n ths large strategy space. All other smulaton parameters reman the same. An optmzaton problem wth feasble ponts exceedng 2 N when N>3 s very dffcult to fnd [4]. The centralzed approach s no longer feasble n ths scenaro due to the enormous strategy space of 6 P.m.f Iteraton 2 Iteraton Fgure 6 The probablty dstrbuton of RMSG for one of the users when N =4. P.m.f User User 2 User 3 User Recever Transmtter Transmt Powers (W) Iteraton Fgure 5 Transmt powers versus teraton n COPMA wth N = 4, T = 3, and ϒ = Fgure 7 Node confguraton and transmt beampatterns of COPMA wth N = users.

11 Page of 2 profles. Fgure 7 shows the network topology and transmt beampatterns of COPMA wth N = users. We agan nvestgate both cooperatve and regretmatchng learnng algorthms represented by COPMA and RMSG curves, where the maxmum number of teratons s set to = 4 for COPMA and = 5 for RMSG. COPMA s performance was found to be optmal soluton for the 4 lnk network, so t provdes a good benchmark to test the performance of RMSG. Fgure 8 shows the total network power versus the number of teratons for COPMA and RMSG. Ths fgure shows that RMSG s performance s wthn 75.56% of the COPMA value at the end of teratons. Furthermore, RMSG needs a larger amount of teratons compared to COPMA for the convergence. However, note that RMSG performs noncooperatve updates for transmt beamformer and powers at each teraton and thus the amount of overhead s mnmal. For the RMSG algorthm, the total power converges to a total network power of.225 W from the,296th teraton. The jont selecton of transmt beamformer ndces and transmt powers reaches steady state when no user n the network devates from ts chosen strategy. Themajortyofusersreachasteadystatewthn5 teratons. However, one user takes longer than, teratons to reach steady state. Probablty mass functon (p.m.f): Asmlarfgure for the p.m.f of RMSG for the user that takes longer convergence tme than others s shown n Fgure 9 for thelargenetworkszewthn =. As can be seen n Fgure 9, the probablty of choosng ndex 6 s hgher than other ndces at teraton 5, but the probablty of choosng ndex 5 and 2 s not totally elmnated even after, teratons. Snce the network sze s large, the learnng process s slower to converge (around,332 Total transmt Power (W) Iteraton RMSG COPMA Fgure 8 Total transmt power versus teraton wth N =, T = 3, and ϒ =6. P.m.f P.m.f Iteraton Iteraton teratons) to steady-state transmt beamformer ndces, compared to the smaller network wth N = 4 node pars. 7 Concluson In ths paper, we have consdered both cooperatve and noncooperatve jont power control and beamformng n MIMO ad hoc networks usng a game theoretc approach. Under constant SINR requrements, the jont transmt beamformer and power selecton algorthms were studed n the context of total network power mnmzaton. We frst consdered a cooperatve case where all users collaborate wth each other n order to mnmze the overall power of the network. The game was formulated as an dentcal nterest game, and a decentralzed algorthm COPMA wth hgh probablty of convergence was proposed and analyzed. To reduce the requred overhead ncurred by the cooperatve algorthm, we have also proposed a noncooperatve soluton whch requres only local nformaton. For our proposed noncooperatve algorthm, users update ther probabltes of choosng a transmt beamformer and power based on the regret of not choosng the other strateges. Numercal results llustrate the convergence propertes of the proposed algorthms and ther performance n terms of overall power mnmzaton n the network. Note Ths paper s presented n part at the IEEE Global Communcatons Conference 2 (Globecom ), Mam, FL, December 6-, 2. Author detals Department of Electrcal and Computer Engneerng, Stevens Insttute of Technology, Hoboken, NJ 73, USA 2 Department of Electrcal and P.m.f P.m.f Iteraton 5 Iteraton Fgure 9 The probablty dstrbuton for one user when N = n RMSG.

12 Page 2 of 2 Computer Engneerng, West Vrgna Unversty Insttute of Technology, Montgomery, WV, USA Competng nterests The authors declare that they have no competng nterests. Receved: 3 November 2 Accepted: 26 August 2 Publshed: 26 August 2 References. IE TelatarL, Capacty of mult-antenna Gaussan channel. Eur Trans Telecommun. (6), (999) 2. AJ Paulraj, DA Gore, RU Nabar, H Bolcske, An overvew of MIMO communcatons a key to ggabt wreless. Proc IEEE. 92(2), (24). do:.9/jproc D Gesbert, S Hanly, H Huang, SS Shtz, O Smeone, W Yu, Mult-cell MIMO cooperatve networks: a new look at nterference. IEEE J Sel Areas Commun. 28(9), (2) 4. H Sampath, P Stoca, A Paulraj, Generalzed lnear precoder and decoder desgn for MIMO channels usng the weghted MMSE crteron. IEEE Trans Commun. 49(2), (2). do:.9/ DP Palomar, JM Coff, MA Lagunas, Jont Tx-Rx beamformng desgn for multcarrer MIMO channels: a unfed framework for convex optmzaton. IEEE Trans Sgnal Process. 5(9), (23). do:.9/ TSP DP Palomar, JM Coff, MA Lagunas, Unform power allocaton n MIMO channels: a game theoretc approach. IEEE Trans Inf Theory. 49(7), (23). do:.9/tit DP Palomar, MA Lagunas, JM Coff, Optmum lnear jont transmt-receve processng for MIMO channels wth QoS constrants. IEEE Trans Sgnal Process. 52(5), (24). do:.9/tsp F Rashd-Farrokh, KJR Lu, L Tassulas, Transmt beamformng and power control for cellular wreless systems. IEEE J Sel Areas Commun. 6(), (998) 9. M Schubert, H Boche, Soluton of the multuser downlnk beamformng problem wth ndvdual SINR constrants. IEEE Trans Veh Technol. 53(), 8 28 (24). do:.9/tvt K Wong, RD Murch, KB Letaef, Performance enhancement of a multuser MIMO wreless communcaton system. IEEE Trans Commun. 5(2), (22). do:.9/tcomm F Farrokh, L Tassulas, KR Lu, Jont optmal power control and beamformng n wreless networks usng antenna arrays. IEEE Trans Commun. 46(), (998). do:.9/ S Ye, RS Blum, Optmzed sgnalng for MIMO nterference systems wth feedback. IEEE Trans Sgnal Process. 5(), (23). do:.9/ TSP C Lang, KP Dandekar, Power management n MIMO ad-hoc networks: A game-theoretc approach. IEEE Trans Wrel Commun. 6(4), 64 7 (27) 4. G Arslan, MF Demrkol, Y Song, Equlbrum effcency mprovement n MIMO nterference systems: a decentralzed stream control approach. IEEE Trans Wrel Commun. 6(8), (27) 5. G Scutar, DP Palomar, S Barbarossa, Compettve desgn of multuser MIMO systems based on game theory: a unfed vew. IEEE J Sel Areas Commun. 26(7), 89 3 (28) 6. G Scutar, DP Palomar, S Barbarossa, The MIMO teratve waterfllng algorthm. IEEE Trans Sgnal Process. 57(5), (29) 7. DJ Love, RW Heath, D Gesbert, BD Rao, M Andrews, An overvew of lmted feedback n wreless communcaton systems. IEEE J Sel Areas Commun. 26(8), (28) 8. E Zeydan, DK Turel, U Turel, Iteratve beamformng and power control for MIMO ad-hoc networks, n Proceedngs of IEEE GLOBECOM (2) 9. DC Popescu, DB Rawat, O Popescu, M Saqub, Game-theoretc approach to jont transmtter adaptaton and power control n wreless systems. IEEE Trans Syst Man Cybern Part B: Cybern. 4(3), (2) 2. C Lacatus, DC Popescu, Adaptve nterference avodance for dynamc wreless systems: a game theoretc approach. IEEE J Sel Topcs Sgnal Process. (), (27) 2. DJ Love, RW Heath, Grassmannan beamformng for multple-nput multple-output wreless systems. IEEE Trans Inf Theory 49(), (23). do:.9/tit MC Bromberg, BG Agee, Optmzaton of spatally adaptve recprocal multpont communcaton networks. IEEE Trans Commun. 5(8), (23). do:.9/tcomm R Ilts, S Km, D Hoang, Noncooperatve teratve MMSE beamformng algorthms for ad-hoc networks. IEEE Trans Commun. 54(4), (26) 24. MC Bromberg, Optmzng MIMO multpont wreless networks assumng Gaussan other-user nterference. IEEE Trans Inf Theory. 49(), (23). do:.9/tit J Lee, YG L, Iteratve lmted feedback beamformng for MIMO ad-hoc networks, n Proceedngs of IEEE GLOBE-COM 9 (29) 26. CU Saraydar, NB Mandayam, DJ Goodman, Effcent power control va prcng n wreless data networks. IEEE Trans Commun. 5(2), (22). do:.9/ S Buzz, HV Poor, Jont recever and transmtter optmzaton for energyeffcent CDMA communcatons. IEEE J Sel Areas Commun. 26(3), (28) 28. G Scutar, DP Palomar, S Barbarossa, Optmal lnear precodng strateges for wdeband non-cooperatve systems based on game theory part I: Nash equlbra. IEEE Trans Sgnal Process. 56(3), (28) 29. G Scutar, DP Palomar, S Barbarossa, Optmal lnear precodng strateges for wdeband non-cooperatve systems based on game theory part II: algorthms. IEEE Trans Sgnal Process. 56(3), (28) 3. D Fudenberg, J Trole, Game Theory (MIT Press, Cambrdge, MA, 99) 3. JB Rosen, Exstence and unqueness of equlbrum ponts for concave n- person games. Econometrca. 33, (965) 32. Y Song, C Zheng, Y Fang, Jont channel and power allocaton n wreless mesh networks: a game theoretcal perspectve. IEEE J Sel Areas Commun. 26(7), (28) 33. J Chang, L Tassulas, F Rashd-Farrokh, Jont transmtter recever dversty for effcent space dvson multaccess. IEEE Trans Wrel Commun. (), 6 27 (22). do:.9/ JR Marden, G Arslan, JS Shamma, Regret based dynamcs: convergence n weakly acyclc games. AAMAS 7: Proceedngs of the 6th Internatonal Jont Conference on Autonomous Agents and Multagent Systems (ACM, New York, NY, USA, 27), pp HP Young, Indvdual Strategy and Socal Structure (Prnce-ton Unversty Press, Prnceton, NJ, 998) 36. P Mlgrom, J Roberts, Ratonalzablty, learnng and equlbrum n games wth strategc complementartes. Econometrca. 58(6), (99). do:.237/ S Hart, A Mas-Colell, A smple adaptve procedure leadng to correlated equlbrum. Econometrca. 68(5), 27 5 (2). do:./ HP Young, Strategc Learnng and ts Lmts (Oxford Unversty Press, Oxford, UK, 24) 39. M Gruteser, A Jan, J Deng, F Zhao, D Grunwald, Explotng physcal layer power control mechansms n IEEE 82.b network nterfaces. Techncal report CU-CS-924-, Department of Computer Scence, Unversty of CO Boulder (2) 4. Z Han, FR Farrokh, KJR Lu, Jont power control and blnd beamformng over wreless networks: a cross layer approach. EURASIP J Appl Sgnal Process. 5, (24) 4. S Boyd, L Vandenberghe, Convex Optmzaton (Cambrdge Unversty Press, Cambrdge, 24) do:.86/ Cte ths artcle as: Zeydan et al.: Jont teratve beamformng and power adaptaton for MIMO ad hoc networks. EURASIP Journal on Wreless Communcatons and Networkng 2 2:79.