A Game-Theoretic Framework for Interference Avoidance in Ad hoc Networks

A Game-Theoretic Framework for Interference Avoidance in Ad hoc Networks R. Menon, A. B. MacKenzie, R. M. Buehrer and J. H. Reed The Bradley Department of Electrical and Computer Engineering Virginia Tech, Blacksburg, VA {rmenon, mackenab, buehrer, reedjh}@vt.edu Abstract A framework to construct convergent interference avoidance (IA) algorithms in networks with multiple distributed receivers (as in ad hoc networks) based on potential game theory is developed in this paper. This is motivated by the fact that direct extensions of distributed greedy IA techniques for centralized networks to these de-centralized networks do not always lead to convergence. Some channel conditions that lead to non-convergence are also identified in the paper. A waveform adaptation algorithm for IA, designed on the basis of the framework, is then proposed. It is shown that this algorithm leads to a reduction of the interference in the network and also incorporates fairness in the allocation of resources. I. INTRODUCTION Networks are becoming less structured and increasingly involve distributed decision making. Nodes are required to independently adapt in a way such that the interference in the network is minimized. This paper investigates such distributed WA strategies which reduce interference and consequently facilitate multi-user communication in ad hoc networks. Greedy Interference Avoidance (IA) algorithms by Waveform Adaptation (WA) (wherein users choose waveforms that increase their own signal to interference and noise ratio) for networks with a centralized receiver have been extensively investigated in [1], [2], and [3] (and references within). These algorithms are shown to converge to a set of waveforms which maximize the sum capacity of the multiple access channel. However, in wireless systems where users talk to multiple uncoordinated receivers (as in an ad hoc network), direct applications of the same greedy IA by WA techniques might not lead to a stable fixed point ([4]). This is caused by the asymmetry of the mutual interference between users at different receivers, leading the users to adapt their sequences in conflicting ways. In this paper, we first enumerate some channel conditions under which a greedy IA by WA algorithm does not lead to convergence. We then design a framework based on potential game theory that can lead to distributed and convergent WA algorithms that maximize some network welfare function. In these algorithms, the utility function of a user incorporates some measure of the influence caused by a particular user s actions on the other users in the network to ensure convergence. The paper is organized as follows. The system model for the network scenario under consideration is described in Section II. Section III describes some channel conditions under which greedy IA games do not converge. Section IV gives a brief overview of game-theory and potential games. Section V presents the game-theoretic framework for IA games in a decentralized network. Section VI presents the proposed WA algorithm for IA which is formulated from the framework. Section VII summarizes the paper and presents directions for future research. II. SYSTEM MODEL We consider an ad hoc network, made up of a cluster of transmit and receive node-pairs. The receive-nodes for different transmit-nodes maybe different and are assumed to not be able to directly coordinate or cooperate with each other. Figure 1 shows an example network with transmitreceive node-pairs indicated by arrows. This network model is a generalization of a network with co-located or centralized receivers and the results presented here are applicable to the centralized network scenario as well. A signal-space characterization is used to represent transmit Fig. 1. Transmitter Node Receiver Node Example network with multiple un-coordinated receivers waveforms [2] for nodes. This signal-space representation for the transmit waveform of a node specifies the waveform in orthogonal signal dimensions (e.g. time or frequency) and is referred to as the signature sequence of the node. Let N denote the number of transmission dimensions available to the network and K denote the number of transmitting nodes in the network. Vector s k R N 1 is used to denote the signature sequence associated with transmitting node k. The signature sequences are allowed to have real values (as opposed to bi-polar sequences). Without loss of generality, the signature sequences are assumed to have unit norm (s T k s k = 1). The transmit power level of the k th node is denoted by p k and the fading coefficient of the channel between the k th transmit node and the j th receive node is denoted by g kj. The channel is assumed to be constant over all signal dimensions and also constant over the time required for the adaptation process. The

data symbol (assumed to be of zero-mean and unit-variance) transmitted from the k th transmit node is denoted by b k. The received signal at the j th receive node is then given by r j = pk g kj s k b k + z, (1) where r j R N 1, and the vector z R N 1 models zero mean additive Gaussian noise. Interference is caused at a receive node by transmissions from nodes different from the one associated with the particular receive node. The interference caused is influenced by the correlation between the waveforms of user nodes, transmit power levels and the channel characteristics (as can also be observed from Equation 1). Note that in the analysis in this paper, the signature sequences from multiple users are assumed to be synchronized at the receivers. However, this is done only for the sake of notational simplicity and the scheme can directly and easily be extended to an asynchronous system (as has been shown for WA in centralized networks in [5]). III. NON-CONVERGENCE OF GREEDY BEST RESPONSE GAMES An example scenario is used to show that greedy adaptation procedures do not always converge in decentralized networks (as is the case in ad hoc networks). Greedy adaptation refers to algorithms in which each user myopically tries to maximize its own Signal-to-Interference Ratio (SIR) or some other measure of link capacity in response to adaptations by other users in the network (e.g., the iterative water-filling algorithm). Consider the cluster of three transmit-receive node-pairs shown in Figure 2. The transmit power of all user nodes is assumed to be equal. Also, the nodes are assumed to have only two signal dimensions available for transmission. Let the channel gains in the network be ordered as follows, g 21 > g 11 > g 31, g 13 > g 33 > g 23 & g 32 > g 22 > g 12. (2) Consider an adaptation process in which each user tries to maximize its SINR by water-filling over the interference and noise it sees, in a round-robin fashion. Assume that at the start of the adaptation process, transmit node 2 uses dimension 1 and transmit node 3 uses dimension 2. Transmit node 1 chooses dimension 2 as the interference seen from node 3 is smaller than that from node 2 at receive node 1. However, at receive node 3, more interference is seen from transmit node 1 than transmit node 2 and hence transmit node 3 moves to dimension 1. Receive node 2 sees more interference from transmit node 3 than transmit node 1. Hence transmit node 2 shifts to dimension 2. However, as the interference seen from node 3 is smaller than that from node 2, transmit node 1 chooses dimension 1 now. This cycle thus continues with each node choosing the two dimensions alternately. This process is illustrated in Figure 3. These results can be easily be extended to over-loaded networks (networks with more users than available signal dimensions) with more than three users to show that allocation Fig. 2. g 21 g 31 Channel 1 Channel 2 Network scenario to illustrate non-convergence g 13 g 23 g 32 g 12 g 31 g 21 Node 1 Node 3 Node 2 Node 1 Fig. 3. Channel gains at the receive node corresponding to the adapting transmit node in the two available transmission dimensions. cycles or non-convergence of greedy resource allocations occur when the channel gains between multiple users are ordered cyclically, similar to the ordering for three users given in (2). Since, each adaptation, in general, requires considerable feedback from the receiver to the transmitter, these allocation cycles are expensive with respect to the network overhead and are undesirable from a network performance perspective. IV. GAME THEORY AND POTENTIAL GAMES Consider a normal form game [6] represented as the following tuple Γ = K, {A k } k K, {u k } k K, (3) where, K = {1, 2,..., K } is the set of players of the game. The set of actions available for player k is denoted by A k and the utility function associated with each player k by u k. If the set of all available actions for all players is represented by A = A k, then u k : A R. Player k prefers an action k K profile a A over an action profile â if u k (a) u k (â). A Nash Equilibrium (NE) for a game is an action profile from which no player can increase its utility by unilateral deviations. An action profile, a A, is a NE if and only if u k (a) u k (â k, a k ) k K, â k A k, (4) where, (â k, a k ) = ( a 1,..., a k 1, â k, a k+1,..., a K ) refers to the action profile in which the action of user k is changed from a k to â k, while the actions of all the other players in the game remain the same. Nash equilibria form the steady states of the game. A potential game ([7] and [8]) is a normal form game such that any changes in the utility function of any player in the game due to a unilateral deviation by the player is also reflected in a global function referred to as the potential function. A function V : A R is called an exact potential function if k K, a A and â k A k, u k (a) u k (â k, a k ) = V (a) V (â k, a k ). A function V : A R is called an ordinal potential function if k K, a A and â k A i,

u k (a) u k (â k, a k ) V (a) V (â k, a k ). A game is an exact or ordinal potential game if for the game, there exists an exact or ordinal potential function respectively. The NE of a potential game include maximizers of the potential function. Exact and ordinal potential games with continuous utility functions and compact action spaces are shown to converge to the NE of the game while following a best response dynamic (wherein each user adapts to a sequence that maximizes its utility function). These games are also shown to converge while following a better response dynamic (wherein each user tries to adapt to a sequence that improves its utility function). However they might not necessarily converge to a NE of the game. Additional properties such as a better response with a finite minimum step size or a random better response can however be used to establish the convergence of these games to the NE [11]. V. FRAMEWORK FOR CONVERGENT IA GAMES A game-theoretic model that can be used to construct and identify IA algorithms for distributed (or ad hoc) networks is suggested here. Potential games are chosen as these are easy to analyze and give a framework where users can maximize a global network function by only trying to maximize their own utilities, leading to simple game formulations. Let the user node-pairs be the players of the game. The signature sequences (or waveforms) of users constitute the actions for the players in the game. Let the utility associated with a particular user k be given by the following, u k (s k, s k ) = f 1 (s k, p k ) f 2 (I (s j, s k ), p j, p k, g jk, g kk ) f 3 (I (s k, s j ), p k, p j, g kj, g jj ). Function f 1 quantifies the benefit associated with a particular choice of signature sequence and power. Function f 2 is a measure of the interference due to the other users present in the system perceived at the receive node for user node k. Function I is some function of two signature sequences s k and s j (e.g. the correlation between the sequences). Function f 3 is a measure of interference caused by a particular user at the receivers associated with other users in the network. Note that the utility function of a user incorporates a measure of the influence of the user s action on the other users in the system. A simple formulation of a candidate potential function is given by, f 1 (s k, p k ) α f 2 (I (s j, s k ), p j, p k, g jk, g kk ) β f 3 (I (s k, s j ), p k, p j, g kj, g jj ) (6) Coefficients α and β are weighting factors and matrix s = [s 1,..., s K ]. (5) A. Exact Potential Game (EPG) It can be shown that an EPG can be formulated under the two scenarios listed below. α = β = 1 2 in both scenarios. Scenario-1: f 2 (I(s j, s k ), p j, p k, g jk, g kk ) = f 2 (I(s k, s j ), p k, p j, g kj, g jj ) f 3 (I(s j, s k ), p j, p k, g jk, g kk ) = f 3 (I(s k, s j ), p k, p j, g kj, g kk ) (7) This scenario occurs when the interference caused by a transmit-node A, to a receive-node B, is the same as the interference caused by the transmit-node associated with receivenode B, to the receive-node associated with transmit-node A. An example is a network with co-located transmitters and colocated receivers. However, such symmetric links are not very likely in ad hoc networks. Scenario-2: f 2 ( ) = f 3 ( ). (8) This scenario requires that the interference caused at the receive-node of a particular transmit-node, be measured in the same manner as the interference caused by the transmit node at other receive-nodes. This scenario is thus more realistic and will be used for the game formulation in this paper. Let f V ( ) = f 2 ( ) = f 3 ( ). Then, when Scenario-2 holds, the potential function reduces to the following, f 1 (s k, p k ), f V (I (s k, s j ), p k, p j, g jk, g kk ) B. Ordinal Potential Game (OPG) It can be shown that an OPG can be formulated when, f 2k ( ) = f 3k ( ) = f uk ( ), where f uk ( ) is an ordinal transformation of f V ( ) and the utility function of each user is given by, u k (s k, s k ) = f 1 (s k, p k ) K K f 3k (I (s k, s j ), p k, p j ). (9) f 2k (I (s j, s k ), p j, p k ) The ordinal potential function for the game is given by, f 1 (s k, p k ) f V (I (s k, s j ), p k, p j ). (10) (11) Under this game formulation it is possible to construct convergent adaptation games with each user trying to maximize a different utility function, as long as the utility functions are ordinal transformations of each other. VI. INVERSE SINR IA GAME The Signal to Interference and Noise Ratio (SINR) at a receive-node is a good indicator of the throughput and performance of the particular user node-pair. Hence, the weighted sum-interference-and-noise (wherein the interference at each

user s receive-node is divided by the power received from its transmitter or in other words the Inverse SINR) in a distributed network is a pertinent and useful measure of network performance. In this section, we develop a WA algorithm for user node-pairs in an ad hoc network which employs the potential game framework discussed in the previous section to ensure that each user node-pair adaptation reduces the weighted suminterference-and-noise in the network. A. Formulation of the Algorithm From the system model (1), the weighted interference and noise power at the k th receiver is given by I k (s k, s k ) = st k (R ii,k) s k p k gkk 2. (12) Here, R ii,k is the interference-plus-noise-crosscorrelation matrix given by R ii,k = K s j s T j p jg 2 jk + R zz, where R zz = E [ zz T ] is the noise covariance matrix. If the noise process is white, R zz is a multiple of the identity matrix. The weighted sum-interference-and-noise or the Inverse SINR (ISINR) of the network is hence given by s T k R ii,ks k p k gkk 2. (13) To allow the WA update by each user in the network to reduce the ISINR function, the negative of the ISINR function is taken to be the exact potential function of the game. From the framework (using scenario-2 for an EPG), function f V ( ) is given by f V (I (s j, s k ), p j, p k, g jk, g kj ) = st k s js T j s kp j g 2 jk p k g 2 kk (14) and function f 1 ( ) = st k Rzzs k p k. The utility function of a user gkk 2 node-pair k as a function of its signature sequence is then given by u k (s k, s k ) = s T k X k s k, () where X k = R ii,k p k g 2 kk + s j s T j p kg 2 kj p j g 2 jj. (16) The utility for each user is thus seen to increase with a reduction in the interference caused at its receiver. In addition, it also increases with a reduction in the interference it causes at receivers corresponding to other users. Thus each user s utility function incorporates a measure of the influence of its actions on the other users in the system as opposed to utility functions for users in greedy IA games. Since, s T k s k = 1, the utility function can be written as u k (s k, s k ) = st k X ks k s T k s. (17) k Matrix X k is symmetric and positive semi-definite. Hence the utility function is a negative weighted Rayleigh quotient of X k. This is maximized by the eigenvector corresponding to the minimum eigenvalue of X k. The best response of the user to the current state of the network is, therefore, given by the inferior eigenvector of X k. The ISINR WA algorithm with the best response iteration is formally stated below: Best-response-based ISINR WA Algorithm 1) Fix the transmit-power levels and initialize codeword s k for each user. 2) Set k = 1 3) while k K a) Replace s k by the eigenvector corresponding to the minimum eigenvalue of X k b) k = k + 1 4) Repeat step 2 and 3 until a fixed point is reached. B. Convergence The NE of the game are given by signature sequences that satisfy the following expression: X k s k = a min,k s k, k {1, 2,..., K}. (18) Here, a min,k is the minimum eigenvalue of matrix X k. As established before, EPGs exhibit best response convergence to the NE of the game. Hence the fixed points of the ISINR WA algorithm are also given by Equation 18. The NE of a game include the maximizers of the potential function. Since the potential function given by (the negative of) Equation (13) is continuous and bounded, the potential function is guaranteed to have at least one maximum. Consequently, the ISIR WA algorithm is guaranteed to have at least one NE or fixed point (in other words, a set of sequences that satisfy condition 18). Note that since each user adaptation leads to an increase in the potential function, the algorithm can converge even with non-sequential updates. SINR of Users(dB) 0 50 45 35 30 25 0 100 0 300 0 500 600 700 0 100 0 300 0 500 600 700 Fig. 4. Convergence of ISINR WA algorithm with 30 user node-pairs sharing 10 dimensions. SINRs of different users are shown in the top sub-plot and the potential function is shown in the bottom sub-plot. C. Simulation-based Performance Analysis A distributed network is simulated by placing K transmit and receive nodes uniformly in a circular region with radius

1 2 3 4 5 6 7 8 9 10 45 35 30 25 10 5 Fig. 5. Weighted sum-interference-plus-noise function for a ISINR WA algorithm with 6 user node-pairs sharing 6 dimensions. Plots shows convergence to a global potential maximizer from different random initial choice of waveforms. 19.5 R (R = 5m in the simulations). The power at a receive node from a transmit-node at a distance of r from the transmit node is assumed to be given by ρ k r, where ρ α k is assumed to be the power received from a transmit node at a distance of 1m and α is the path-loss exponent (α = 3 in the simulations). All user-nodes are assumed to transmit at the same power-level of 100mW. The path loss at a distance of 1m is assumed to be db (therefore ρ k = 60dBw). The received signals are assumed to be corrupted by additive white Gaussian noise with power spectral density of 80dBw. Note that one iteration of the algorithm in the simulation results corresponds to one WA by a user-node unless indicated otherwise. Figure 4 shows the convergence of the algorithm in a network with 30 users sharing 10 signal dimensions. Figures 5 and 6 show the convergence of the weighted sum-interferenceplus-noise function from different random initial choice of signature sequences for an equally-loaded (equal number of transmit nodes and dimensions) and an over-loaded network respectively. For an equally or under-loaded network scenario, the interference in the network is minimized (or the potential function is maximized) when the transmit nodes are allocated orthogonal sequences. When starting from random initial sequences in an under-loaded or equally-loaded network, the algorithm is empirically seen to always converge to the orthogonal sequence configuration and to converge within 2 roundrobin iterations. This is illustrated in Figure 5. In the overloaded scenario, on the other hand, multiple fixed points are seen to exist for the WA algorithm and the algorithms takes longer to converge. Nevertheless, the algorithm significantly reduces the interference in the network in about - roundrobin iterations. We now compare the performance of the proposed ISINR WA algorithm with the performance of a greedy IA algorithm (similar to the algorithm suggested in [12]). In the greedy IA algorithm, each user adapts such that the interference at its receiver is minimized. The utility function for each user is hence given by u g k (s k, s k ) = st k R ii,ks k p k gkk 2. (19) Note that the utility function does not incorporate the effect of the user s action on the other users in the network. Figure 7 shows the weighted sum-interference-plus-noise 19 18.5 18 35 Proposed ISINR WA ALgorithm Greedy IA Algorithm 17.5 30 17 16.5 5 10 25 30 35 45 Number of Round robin Iterations 25 Fig. 6. Weighted sum-interference-plus-noise function for a ISINR WA algorithm with 12 user node-pairs sharing 6 dimensions. Plots shows convergence from different random initial choice of waveforms to different fixed points. 10 0 50 100 0 0 250 300 Fig. 7. Comparison of the weighted sum-interference-plus-noise function for the ISINR WA algorithm and the greedy IA algorithm in multiple arbitrary networks. The network has 12 users sharing 6 signal dimensions. SINR of Users(dB) 10 5 0 5 10 Sollid: ISINR game Dotted: Greedy IA game 0 50 100 0 0 250 300 Fig. 8. Comparison of the SINRs of users for the ISINR WA algorithm and the greedy IA algorithm. The network has 6 users sharing 3 signal dimensions. The numbers in the graph identify different user. function for the two algorithms over different arbitrary networks. It is observed that, in general, the proposed ISINR WA algorithm results in lower interference in the network as compared to the greedy IA algorithm. Note that the plot also illustrates that the greedy IA algorithm does not always converge and could lead to allocation cycles. This, as mentioned 6 2 5 6 134 34

before, could result in a large network adaptation overhead. Since the proposed ISINR WA algorithm takes into account the effect of the particular user s actions on the other users in the network, it is also observed to, in general, result in a fairer allocation of resources than the greedy IA algorithm. This is illustrated in Figure 8 which plots the SINRs of the users in the network for the two adaptation algorithms. It is seen that the SINRs of the users are more closely distributed in the case of the proposed algorithm (Note that a network scenario where the greedy algorithm converges is chosen for this particular simulated illustration example). Also, when averaged over allocations in different arbitrary network scenarios, the proposed ISINR algorithm results in a Theil s entropy measure (an inequality index [13] where measure 0 indicates equal distribution and higher values indicate more unequal distribution of resources) of 1.3998 while the greedy IA algorithm results in a measure of 2.9691. D. Distributed Implementation The implementation of the best response iteration for the ISINR game requires any node making adaptation decisions to have access to the signature sequence and received powers of all the nodes in the network. This can be accomplished by requiring each transmit node to broadcast its sequence and transmit power level and each receive-node to broadcast the channel coefficients at the beginning of the adaptation process and requiring the adapting node to broadcast its new signature sequence after each adaptation. However, this process could considerably increase the overhead of the network especially since the signature sequence can have any realvalue. It could also lead to security concerns in the network. Hence, limited-feedback schemes for the proposed algorithm based on the better-response convergence property of potential games, where a particular users adapts only according to the interference seen at its receiver and involving negligible feedback from the other receivers in the network, are currently being investigated. Such schemes have already been developed for WA algorithms in a centralized network in [11]. E. Other Example IA Games The framework can be used to construct other WA algorithms for IA. For instance, a WA algorithm, which reduces the sum interference in the network weighted by the received power of individual users, defined below, can easily be constructed using the described potential game framework. s T k s j s T j p j gjkp 2 k gkk 2 s k. () j=1 j k Note that this function is different in spirit to the function given in Equation 13, since here, there is more incentive to provide lower interference to users with larger received powers while in the previous game, stronger users are assumed to be able to tolerate more interference. Therefore, this game might not lead to fair resource allocations but could result in larger sum capacities for the network. A similar algorithm for multi-base networks is developed in [4]. Hence the framework can be directly used to construct IA algorithms for multi-base networks as well. VII. SUMMARY A framework based on potential game theory which could be used to construct convergent IA algorithms for ad hoc networks is developed in this paper. This is motivated by the fact that direct extensions of myopic greedy IA algorithms do not lead to convergence in these networks. Some conditions that lead to non-convergence are also identified in this paper. A WA algorithm based on the framework is then proposed which is shown to considerably decrease the interference in the network and also incorporate fairness in the allocation of network resources. As mentioned, the practical implementation of the adaptation algorithm developed in this paper requires considerable feedback. Hence efficient limited-feedback schemes based on the better response dynamic of potential games are currently being developed. Another approach to develop distributed adaptation strategies for these networks is to develop games in which users try to achieve a feasible target performance (and not maximize their performance). A target-performance-based WA algorithm for centralized networks is presented in [10]. However, the identification of a feasible target performance in ad hoc networks (de-centralized networks) remains an open problem. REFERENCES [1] S. Ulukus and R.D. Yates, Iterative construction of optimum signature sequence sets in synchronous CDMA systems, IEEE Trans. Inform. Theory, vol. 47, no. 5, pp. 1989-1998, Jul. 01. [2] C. Rose, S. Ulukus and R. D. Yates, wireless systems and interference avoidance, IEEE Trans. Wireless Comm., vol. 1, no. 3, pp. 4-427, Jul 02. [3] J. Hicks, A. MacKenzie, J. Neel and J. Reed, A game theory perspective on interference avoidance, IEEE Global Telecomm. Conf., vol. 1, pp. 257-261, 29 Nov.-3 Dec 04. [4] C. W. Sung and K. K. Leung, On the stability of distributed sequence adaptation for cellular asynchronous DS-CDMA systems, IEEE Trans. Inform. Theory, vol. 49, issue. 7, pp. 1828-1831, Jul. 03. [5] S. Ulukus and R. D. Yates, Signature Sequence Optimization in Asynchronous CDMA Systems, Proc. IEEE Int. Conf. Comm., vol. 2, pp. 545-549, June 01. [6] K. Dutta, Strategies and Games, Theory and Practice, MIT Press, Cambridge Massachusetts, 1999. [7] D. Monderer and L. Shapley, Potential games, Journal of Games and Economic Behavior, vol. 14, no. 0044, Pages: 124-143, 1996. [8] J. Neel, J. Reed and R. Gilles, Convergence of cognitive radio networks, IEEE Wireless Communications and Networking Conf., vol. 4, pp. 2250-2255, Mar 04. [9] Gene H. Golub, Charles F. Van Loan, Matrix Computations, Johns Hopkins University Press, 3rd edition, 1 Nov. 1996. [10] D. C. Popescu and C. Rose, Interference avoidance and power control for uplink CDMA systems, IEEE 58th Veh. Technol. Conf., vol. 3, pp. 1473-1477, 6-9 Oct. 03. [11] R. Menon, A. B. MacKenzie, J. Hicks, R. M. Buehrer and J. H. Reed, A Game-theoretic Framework for Interference Avoidance, Under Preparation. Available at http://filebox.vt.edu/ rmenon/gt/ia RM.pdf. [12] O. Popescu, D. C. Popescu and C. Rose, Interference avoidance for capacity optimization in mutually interfering wireless systems, IEEE 62nd Veh. Technol. Conf., vol. 1, pp. 593-597, 28-25 Sept. 05. [13] H.Theil, Economic and Information Theory, N. Holland, 1967.