A Distributed Mode Selection Approach Based on Evolutionary Game for Device-to-Device Communications

Transcription

1 1 A Distributed Mode Selection Approach Based on Evolutionary Game for Device-to-Device Communications Yujie Li, Wei Song, Senior Member, IEEE, Ziwen Su, Lianfen Huang, and Zhibin Gao Abstract As one of the key technologies for the fifth generation (5G) mobile networks, device-to-device (D2D) communications offer promising benefits such as high spectrum efficiency, traffic offloading and enhanced coverage. Depending on the resource sharing between D2D users and regular cellular users, a D2D user equipment (UE) can dynamically switch its communication mode to improve the quality of service (QoS) and user experience. Mode selection is an essential issue to ensure the QoS of D2D UEs while maximizing the system capacity. In this paper, we investigate the D2D mode selection problem from a novel perspective. In addition to the classic cellular mode and direct reuse mode, we further consider a relay mode for D2D UEs. Moreover, in order to address a potentially large population of D2D UEs, we propose an evolutionary game based approach for D2D mode selection. The evolutionary game is formulated with a utility function that takes into account both the achievable throughput of D2D UEs and the radio resource consumption. Based on the evolutionary game formulation, we implement selection dynamics, i.e., replication by imitation, in a devicecontrolled mode selection algorithm. To evaluate the performance of the proposed mode selection algorithm, we conduct simulations to compare it with three baseline schemes, including an approach based on maximum signal-to-interference-plus-noise (SINR), a distance-based approach, and a random approach. As shown in the simulation results, the proposed approach achieves higher utilities than the baseline schemes. Index Terms D2D communications, mode selection, evolutionary game, replicator dynamics, replication by imitation. I. INTRODUCTION Device-to-device (D2D) communications enable direct communications between user equipment (UE) in the close proximity without traversing the base station (BS) or the core network. Compared with other short-range communication technologies, such as Bluetooth, Wi-Fi Direct and ZigBee, D2D communications can operate over the licensed frequency bands (i.e., inband) and benefit from control of the BS. For instance, during a D2D communication session, the BS can assist the D2D UEs to establish a communication connection, Corresponding author. Y. Li, Z. Su, L. Huang, and Z. Gao are with the Department of Communication Engineering, Xiamen University, Xiamen, China ( s: liyujie@stu.xmu.edu.cn, suziwen1991@163.com, lfhuang@xmu.edu.cn, gaozhibin@xmu.edu.cn). W. Song is with the Faculty of Computer Science, University of New Brunswick, Fredericton, Canada ( wsong@unb.ca). The research was supported in part by the National Natural Science Foundation of China (Grant numbers: , , , ), the Key Laboratory of Digital Fujian on IoT Communication, Architecture and Security Technology (Grant number: 21499), and the Natural Sciences and Engineering Research Council (NSERC) of Canada. allocate radio resources, and coordinate interference mitigation. In addition, D2D communications can operate over unlicensed spectrum (i.e., outband) so as not to affect the performance of cellular users. In this case, a challenging issue is how to coordinate communications in different frequency bands. The earliest attempts to enable D2D communications in cellular networks was initiated by Qualcomm, which proposed FlashLinQ [1], a synchronous peer-to-peer wireless PHY/MAC network architecture. FlashLinQ uses distributed scheduling to efficiently implement D2D time synchronization, device discovery, and link management. In recent years, there are many other studies on various issues of D2D communications, such as coverage extension [2], traffic offloading [3,4], resource allocation [5,6], and content distribution [7,8]. The integration of D2D communications into the traditional cellular network results in a two-tier heterogeneous network. Depending on how the inband spectrum is shared among D2D UEs and cellular UEs, there are overlay and underlay spectrum sharing. In overlay spectrum sharing, D2D UEs are allocated dedicated radio resources in the licensed frequency band, which can resist interference from cellular UEs but may waste spectrum resources. Underlay spectrum sharing allows D2D UEs to reuse the same spectrum as cellular UEs, which may cause non-negligible interference but can achieve high spectrum efficiency. In this paper, we refer to D2D communications with overlay spectrum sharing the cellular mode, and that with underlay spectrum sharing the direct reuse mode. In addition, we consider another appealing D2D communication mode, i.e., the relay mode, in which a D2D pair is assisted by a nearby D2D-capable UE [9,1]. As the wireless communication link is highly time-varying and location-dependent, it is essential to dynamically adapt the D2D communication mode based on the network environment information to optimize the performance of the entire heterogeneous network. On one hand, mode selection can enhance the quality of service (QoS) of D2D UEs, e.g., throughput. On the other hand, it helps control mutual interference between cellular UEs and D2D UEs, while improving system spectrum efficiency. A straightforward D2D mode selection algorithm is to use the received signal strength (RSS), the received signal-tointerference-plus-noise (SINR), or the distance between D2D transceivers as the mode selection criterion. This method is easy to implement, but does not take into account channel state information (CSI) and interference information. Moreover, many existing works focus on single D2D pair or only a small number of D2D UEs. For example, there are four D2D UEs per

2 2 cell in [11,12], while a D2D group of 5 UEs are considered in [13]. In a similar scale, the numerical experiments in [9,1,14] simulate up to 15, 25, and 1 D2D pairs, respectively. As known, one prominent feature of the fifth generation (5G) mobile networks is ultra dense deployment. When there is a large population of D2D UEs, traditional mode selection approaches may not scale well or cannot make the best use of system resources to achieve the maximum capacity. In this paper, we attempt to address the challenge of a large D2D UE population in mode selection. Since each D2D UE is self-interested in maximizing its own individual utility, which may cause conflict among D2D UEs in competing for the limited spectrum resources. Hence, we employ a gametheoretic approach and formulate an evolutionary game, based on which a distributed device-controlled algorithm is designed for D2D mode selection. We use an evolutionary game model mainly because of its favourable properties in handling an interactive environment with a large population. For example, in a large conference site or shopping mall, there may be a great number of users who want to directly share content or exchange information with each other. It becomes challenging for each D2D device to reason the behaviours of other users and determine its best response. This also poses great demands on collecting the detailed information of all players. In an evolutionary game, players are not supposed to have such intelligence or coordination. Instead, strategies are viewed as being hard-wired into players. In other words, when the result of a game shows how good a strategy is and evolutions of games test alternative strategies, a player can just observe the game results and tune to the most successful strategy. Specifically, the main contributions of this paper are summarized as follows. We formulate the mode selection problem with three D2D modes. In addition to the widely considered cellular mode and direct reuse mode, we also take into account another promising D2D communication mode, i.e., the relay mode. The interference-limited D2D communication environment is properly modelled with respect to the received SINR. We investigate the mode selection problem with an evolutionary game formulation. A utility function is defined to guide mode selection of D2D UEs. The utility function incorporates both throughput and radio resource consumption, and is clearly linked to spectrum efficiency. Based on the utility function, we determine the communication modes of D2D pairs by solving the defined evolutionary game and thereby achieve high spectrum efficiency. We propose a distributed D2D mode selection algorithm using replication by imitation for the formulated evolutionary game. The proposed algorithm is compared with the baseline algorithms with extensive simulations. It is observed that the proposed approach outperforms the baseline algorithms and achieves the highest utility among the four schemes. The remainder of this paper is organized as follows. Section II reviews existing D2D mode selection methods. In Section III, we introduce the channel models for different D2D modes. In Section IV, we formulate the D2D mode selection problem as an evolutionary game and propose a distributed algorithm for the D2D UEs to approach the equilibria. Simulation results are presented in Section V, followed by conclusions in Section VI. II. RELATED WORK As discussed in Section I, mode selection is an important issue for D2D communications to ensure high performance for the cellular network. A simple method is to consider the RSS, the received SINR, or the distance between the D2D transmitter and receiver as the criterion to select the best D2D mode. Although this method is easy to implement with little information collection, its performance is poor because it does not take into account the CSI or interference information to optimize the overall performance. Advanced D2D mode selection approaches often make use of such information to maximize the system capacity or minimize the system power consumption. These mode selection methods can be broadly categorized into two classes, i.e., centralized methods [9], [15] [19] and distributed methods [1] [14], [2] [22]. In the following discussions, we further distinguish the centralized methods according to whether single D2D pair or multiple D2D pairs are considered. For distributed methods, we separate those methods based on game-theoretic ideas, as they are closely related to this work. A. Centralized Solutions with Single D2D Pair In centralized mode selection, the BS acquires the link status of the devices under its coverage, and then leverages the measurement information to select the communications modes for D2D UEs. In [16], Doppler et al. studied the D2D mode selection problem with three D2D communication modes, i.e., the reuse mode, dedicated mode, and cellular mode. The reuse mode refers to direct D2D communications that reuse the whole radio resources together with cellular UEs. In the dedicated mode, D2D UEs communicate directly but only receive half of the resources. Cellular mode uses indirect communication between devices with relay by the BS and receives half of the resources. This work models the throughput of cellular UEs and D2D UEs in different modes and proposes a greedy algorithm that selects the mode providing the highest throughput. This approach depends on centralized control of the BS, which collects channel conditions and estimates SINR values of D2D UEs in various communication modes. In [15], the authors studied three resource sharing modes for D2D UEs, including the non-orthogonal resource sharing mode, separate resource sharing mode, and cellular mode. These three D2D modes are similar to those considered in [16]. This work jointly considers mode selection and power control for D2D UEs. It is assumed that the BS has full CSI to make the best decision on the resource sharing mode. In addition, two power optimization approaches, i.e., greedy sumrate maximization and rate-constrained power control, were proposed to maximize the total system rate while satisfying the SINR (also rate) requirements for D2D UEs. The work in

3 3 [17] further extends [15] by taking into account practical constraints, such as minimum and maximum spectral efficiency restrictions, and maximum transmit power or energy limits. B. Centralized Solutions with Multiple D2D Pairs All of the above works [15] [17] are based on centralized control of the BS and consider a simple scenario with one cellular UE and one pair of D2D UEs. The control scope can be extended to a multi-cell environment or multiple D2D UEs in a single cell. In [18], the authors considered two communication modes, i.e., the reuse mode via the direct D2D link and the cellular mode via the BS, for one D2D pair within each cell of a multi-cell network. A distributed power control algorithm was proposed to iteratively determine the SINR targets and allocate transmit powers such that the overall power consumption is minimized subject to a sum-rate constraint. Also, a joint power control and mode selection algorithm was proposed, which only requires large-scale fading information of single cell. In [19], the work focuses on two communication modes similar to those in [18], and investigates mode selection for all D2D UEs within one cell. A set of system equations are derived to capture the network information, including link gains, noise levels, SINR, and communication modes of the D2D UEs. Based on the system equations, the D2D mode selection as well as power allocation can be assigned jointly and optimally for the communicating devices. In [9], mode selection takes into account three D2D communication modes, including the direct mode, relay-assisted mode, and local route mode. In the direct mode, the source and destination communicate directly, while in the relay-assisted mode and the local route mode, a D2D pair uses a relaycapable UE or a BS as the relay station. It assumed that a D2D pair in the local route mode is allocated dedicated cellular resource blocks (RBs), whereas the cellular uplink spectrum is reused and shared in the direct mode and relay-assisted mode. Though this work formulates a good unified model for D2D communications, the proposed mode selection scheme is straightforward. Basically, the BS estimates the achievable end-to-end data rates for each D2D pair in three modes, and selects the mode that achieves the maximal data rate. C. General Distributed Solutions Compared with the centralized control methods, distributed control methods often have the advantages of lower complexity and less signaling interaction. In [11], the authors combined D2D mode selection, resource scheduling and power control into one optimization problem, and proposed a distributed algorithm to minimize the system power consumption. In [1], the authors considered two D2D modes, i.e., direct mode and relay mode. It first determines the optimal power allocation when a D2D link operates in the direct or relay mode, and then solves a joint mode selection and resource allocation problem. In [14], there are three operation modes, when a subchannel is allocated only to a D2D pair (dedicated mode), only to a cellular user (cellular mode), or to both together (underlay mode). This mode selection problem is formulated from a different perspective, since it focuses on the final utilization of radio resources. This work proposes a throughput-optimal joint mode selection, user scheduling, and rate adaptation policy that exploits information about the statistics of the cross links and incorporates inter-cell interference. In [2], the authors proposed a location-aware strategy for joint mode selection and spectrum sharing. In this strategy, D2D UEs can make their own decisions on communication modes and spectrum sharing with other UEs based on relative locations. In [21], Omri and Hasna proposed a distancebased mode selection scheme. In this work, they considered a scenario with mobility and predefined an average distance threshold between two UEs to trigger the D2D mode. An analytical framework is also developed to derive key parameters such as the average D2D distance threshold, the probability of using D2D mode, the successful transmission probability, and the minimum D2D mode residence time. D. Game-Theoretic Distributed Solutions There are also some game-theoretic ideas with distributed solutions to address mode selection for D2D communications. In [12], coalition game theory is used to choose among three D2D communication modes, i.e., the reuse mode, dedicated mode, and cellular mode. For each group of D2D links in one of the three modes, a coalition is formed to cooperatively select the subchannels such that the total power is minimized while their rate requirements are satisfied. Each D2D link can make a decision to join or leave a coalition (i.e., by selecting or abandoning a mode), in order to minimize its transmission cost. The transmission cost takes into account the device s transmission power and the price of channel occupancy, which depend on the D2D link s communication mode. A distributed algorithm was proposed to obtain the stable coalitions, which aim to minimize the system power consumption. In [22], Zhu et al. proposed a joint algorithm for usercontrolled mode selection and spectrum partitioning based on a dynamic Stackelberg game. It considers three modes for D2D UEs, i.e., the reuse mode, dedicated mode, and cellular mode. In the Stackelberg game, the BS and D2D UEs are taken as leader and follower, respectively. The adaptive mode selection of D2D UEs is formulated as an evolutionary game and solved by a distributed algorithm, while the spectrum partitioning by the BS is formulated as an optimal control problem. The payoff function of a D2D UE is defined as the rate utility (as a linear function of average achievable rate in each communication mode) minus the price of access per user per unit of time charged by the service provider. Such a linear utility function is similar to that in [13]. In [13], an evolutionary game based approach is used to address resource allocation for cognitive networks with D2D communications. The secondary users (SUs) can employ BS mode or D2D mode for their transmission, while the regular cellular users are considered as the primary users (PUs). The utilities of SUs in different modes are defined as a weighted summation or subtraction of achieved data rate, power consumption, and price of unit bandwidth, where the prices increase with the spectrum consumption and interference impact. A distributed protocol is proposed for the

4 4 TABLE I THREE D2D COMMUNICATION MODES. Cellular downlink, D2D link Cellular uplink, D2D link D2D Mode Spectrum Use Relay Use Cellular mode Orthogonal spectrum BS as relay Direct reuse mode Underlay spectrum No relay Relay mode Underlay spectrum Nearby UE as relay Cellular UE BS Cellular UE SUs to converge to the equilibria in mode selection with the optimal power allocation for each mode. In [13], the utilities add incomparable measures in different dimensions together, which directs toward a balanced design objective among different aspects but induces the complexity of tuning the weighting coefficients in the utility function. III. SYSTEM MODEL WITH INTERFERENCE-LIMITED D2D COMMUNICATIONS As a promising technique not only for the LTE/LTE-A networks but also for future 5G networks, D2D communications offer various benefits in capacity augment, coverage expansion, and spectrum efficiency improvement. According to the dynamics of the network environment, there are a variety of D2D communication modes to choose from. As discussed in Section II, different D2D modes have been explored in mode selection, such as the reuse mode, dedicated mode, and cellular mode. These D2D communication modes fit for different scenarios due to their respective advantages and disadvantages. Based on previous studies, we choose to consider two popular D2D modes, i.e, the cellular mode and reuse mode, and further take into account another appealing mode, i.e., the relay mode via UEs. For completeness, we specify these three modes in detail as follows. Cellular mode: D2D UEs communicate with each other through the BS that acts as a relay node. The D2D UEs in the cellular mode are allocated spectrum resources that are orthogonal to those occupied by regular cellular UEs. Direct reuse mode: D2D UEs communicate via the direct link and reuse the uplink spectrum allocated for cellular UEs. The resource reuse can improve spectrum efficiency, but also cause interference among D2D UEs and cellular UEs. Relay mode: D2D UEs communicate via an indirect link through a nearby idle UE as a relay. That is, a D2D transmitter first sends data to a relay UE, and then the relay UE forwards the received data to the D2D receiver. Similar to the direct reuse mode, the D2D UEs in the relay mode also reuse the uplink spectrum allocated to cellular UEs. Here, we can distinguish the three D2D modes with Table I. As seen, the three modes are distinguished in two parallel aspects: 1) whether D2D UEs use orthogonal or underlay spectrum resources as to those occupied by regular cellular UEs; and 2) whether D2D UEs communicate via direct links or indirect links with relays. Regarding the spectrum allocation, the direct reuse mode and the relay mode are similar as they both reuse the uplink spectrum for cellular UEs, whereas the cellular mode uses spectrum resources orthogonal to those D2D UE D2D UE Fig. 1. Illustration of cellular mode with one D2D pair. occupied by cellular UEs. Regarding the use of relays, the direct reuse mode operates over direct links between a D2D pair without any middle node. In contrast, the cellular mode and the relay mode are similar in that both involve a relay node to assist the D2D pair. The relay node is the BS in the cellular mode and a regular UE in the relay mode. The BS as the relay can be far from the D2D pair, but the relay UE is often in the near vicinity. As specified by 3GPP for LTE/LTE- A, relays can be used to extend network coverage [23]. In addition, relays can help improve the communication quality of D2D pairs [24]. For example, the relay mode can leverage the local processing capacity of relay UEs and apply amplifyand-forward or decode-and-forward to enhance the ultimate receiving QoS. As seen, this is a promising way to exploit the close proximity and rich resources of the mobile edge. Let N denote the set of D2D UEs, which are covered by nearby BSs. These D2D UEs form n pairs, including a set of D2D transmitters and a set of D2D receivers, denoted by N s = {s 1,...,s n } and N d = {d 1,...,d n }, respectively. Assume that the ith D2D pair consists of D2D transmitter s i and D2D receiver d i, where 1 i n. In addition, each BS hasmfrequency RBs that can be allocated to cellular UEs and D2D UEs, denoted by M = {1,2,...,m}. Next, we define the channel model that determines the achievable data rate in each D2D communication mode. A. Cellular Mode As depicted in Fig. 1, in the cellular mode, the BS acts as a relay node for the D2D UEs. Here, Fig. 1 only shows one D2D pair for presentation clarity. As D2D communications take place between neighbouring devices, the transmission data do not need to be forwarded to the core network. Thus, the BS only serves as a data forwarding node, saving extra signalling overhead between the UEs and the BS. The BS is not limited to the current serving BS but can also be a neighbouring macro-bs or micro-bs. In the cellular mode, D2D UEs occupy spectrum resources that are orthogonal to those of the cellular UEs. Therefore, the D2D UEs in the cellular mode are not subject to interference from cellular UEs. However, interference occurs when D2D UEs in other modes reuse the spectrum occupied by the D2D UEs in the cellular mode.

5 5 For D2D UE pair (s i,d i ) in the cellular mode, we can express the achievable data rate at d i as follows: R c s i,d i = Blog 2 (1+min(Γ c s i,b,γc b,d i )) (1) where B is the bandwidth of the RB allocated to D2D UE pair (s i,d i ), and Γ c s i,b and Γc b,d i are the received SINR at the BS and the D2D receiver d i, respectively, given by Cellular UE BS Cellular UE D2D link Cellular downlink Cellular uplink Interference link Γ c s i,b = P si G si,b s j N s(i (1) )\s i I sj,b +σ 2 (2) D2D UE D2D UE Γ c b,d i = P b G b,di s j N s(i (2) )\s i I sj,d i +σ2. (3) Here, P si and P b are the transmit powers of D2D transmitter s i and the BS, respectively; G si,b is the channel gain from D2D transmitter s i to the BS, while G b,di is the channel gain from the BS to D2D receiver d j ; N s (i (1) ) N s and N s (i (2) ) N s are the subsets of D2D transmitters that use the same spectrum as the first hop or the second hop of D2D pair (s i,d i ), respectively; I sj,b is the interference to the BS by transmitter s j in N s (i (1) ), while I sj,d i is the interference to D2D receiver d j by transmitter s j in N s (i (2) ); and last σ 2 is the power of the white Gaussian random noise. It is worth noting that the subsets N s (i (1) ) and N s (i (2) ) can be empty if the D2D pair (s i,d i ) is allocated spectrum orthogonal to other D2D UEs. B. Direct Reuse Mode As depicted in Fig. 2, in the direct reuse mode, the D2D UEs can share the uplink and/or downlink spectrum of regular cellular UEs. The interference environment varies with the reused resources. When the D2D UEs share the uplink resources, the transmitting D2D UEs may cause interference to the BS, while the cellular UEs in the uplink can interfere with the receiving D2D UEs. On the other hand, if the downlink spectrum is reused by D2D UEs, D2D transmitters can cause interference to neighbouring cellular UEs in the downlink, while the BS will interfere with D2D receivers. Here, we assume that D2D UEs in the direct reuse mode share the uplink spectrum for two main reasons. First, the uplink resources are often less utilized than the downlink. Second, the BS is more capable than cellular UEs in handling interference. Given that the uplink spectrum is shared by D2D UEs in the direct reuse mode, a D2D receiver in this mode is subject to interference from cellular UEs in the uplink and D2D UEs in other modes that use the same RBs. In this case, we can write the achievable data rate at d i as ) Rs d P si G si,d i,d i = Blog 2 (1+ i s j N s(i)\s i I sj,d i +σ 2 (4) where G si,d i is the channel gain between D2D pair (s i,d i ), N s (i) is the set of D2D and cellular transmitters that use the same spectrum as this D2D pair, and then I sj,d i is the interference caused by transmitter s j in set N s (i). Fig. 2. Illustration of direct reuse mode with one D2D pair. C. Relay Mode Last, Fig. 3 shows the relay mode, which takes advantage of a nearby relay UE that assists with communication between two D2D UEs to improve QoS and reduce power consumption. While traditional direct D2D communications focus on singlehop transmission, the D2D relay mode extends the communication scope to multiple hops. If there exists an available relay UE that has shorter distances to the D2D transmitter and receiver, a higher data rate can be achieved with lower power consumption, of course, with the additional cost for the relay UE. To promote the relay mode for D2D communications, the relay UEs need to be incentivized appropriately by rewards to their forwarding efforts. Similar to the direct reuse mode, we assume that D2D UEs in the relay mode also reuse the uplink spectrum of the cellular system. For any D2D pair (s i,d i ), the available relay UEs in the neighbourhood are searched and the one with the closest total distance to the D2D transmitter and receiver can be selected as the forwarding device in the relay mode. Let N r denote the set of relay UEs. For D2D pair (s i,d i ) with a selected relay r i N r, we can express the achievable data rate at D2D receiver d i as R r s i,d i = Blog 2 (1+min(Γ r s i,r i,γ r r i,d i )) (5) where Γ r s i,r i and Γ r r i,d i are the received SINR at the relay UE r i and the D2D receiver d i, respectively, given by Γ r s i,r i = Γ r r i,d i = P si G si,r i s j N s(i (1) )\s i I sj,r i +σ 2 (6) P ri G ri,d i s j N s(i (2) )\s i I sj,d i +σ2. (7) Here, P ri is the transmit power of relay UE r i, G si,r i and G ri,d i are channel gains from D2D transmitter s i to relay UE r i and from relay UE r i to D2D receiver d i, respectively; and I sj,r i and I sj,d i are interferences to r i and d i, respectively, which are caused by each transmitter s j that shares the same uplink spectrum.

6 6 Cellar UE D2D UE BS Relay UE D2D UE Celluar UE Fig. 3. Illustration of relay mode with one D2D pair. D2D link Cellular downlink Cellular uplink Interference link IV. AN EVOLUTIONARY GAME BASED APPROACH FOR D2D MODE SELECTION A. Problem Formulation for D2D Mode Selection As given in the system model in Section III, D2D UEs can choose among three communication modes, i.e., the cellular mode, the direct reuse mode, and the relay mode, for their data transmission. In the cellular mode, D2D UEs are allocated a subset of orthogonal frequency RBs, denoted by M c M, and therefore they are freed from interferences caused by cellular UEs. Nonetheless, as D2D UEs in the cellular mode can be farther away from the BS, the RSSs are often lower but can be compensated by orthogonal RBs with minimal interference. In the direct reuse mode and relay mode, D2D UEs have to share spectrum with each other and with cellular UEs due to the limited orthogonal spectrum resources. To control the impact of D2D UEs in reusing the spectrum, we can restrict the frequency RBs opening to D2D UEs by a subset of RBs, denoted by M dr M. D2D UEs in the direct reuse mode and relay mode can benefit from shorter communication distances, which lead to higher RSSs. In the meantime, they have to tolerate higher interference from each other and from cellular UEs. The RSSs and interferences together determine the data rates achievable by D2D UEs. As seen above, the three D2D communication modes differ in the use of spectrum resources and relays, which leads to different achievable data rates for D2D UEs. The system spectrum efficiency also varies with the communication modes of D2D UEs. Due to the mutual influence among D2D UEs in different modes, we need to properly determine the communication mode for each D2D pair, so that the mode selection solution can balance the individual performance and overall efficiency. Here, we aim to maximize the total utility provided that acceptable data rates are achieved for individual D2D UEs. Specifically, we can express the D2D mode selection problem as follows: max z s.t. s i N s, d i N d R zi s i,d i B/m l R zi s i,d i ϕ, s i N s, d i N d. (8a) (8b) Here, z = {z i : z i {c,d,r}, s i N s,d i N d } is the decision vector. Each element z i chooses the communication mode for D2D pair (s i,d i ) from set K = {c,d,r}, which correspond to the three D2D modes, i.e., the cellular mode, the direct reuse mode, and the relay mode, respectively. In addition, m l defines the number of UEs that share RB l allocated to D2D pair (s i,d i ) from the subset of RBs for D2D UEs, i.e., M c M dr. Then, each term in the summation of the objective function in (8a) quantifies the utility of a D2D pair with the selected mode and the allocated RB. The constraint in (8b) limits the acceptable data rate for each D2D pair to be not less than threshold ϕ. B. Evolutionary Game Model As discussed in Section IV-A, the cellular system is like an ecosystem, in which the choices of D2D UEs for the communication modes influence their achievable QoS and the overall network performance. Hence, we can model the decision making of D2D UEs for mode selection as an evolutionary game. In this section, we first describe the elements of the evolutionary game model for D2D mode selection. Then, in Section IV-C, we introduce replication by imitation for the evolutionary game to pursue its evolutionary equilibrium (EE). Based on the replication dynamics, in Section IV-D, we propose a distributed mode selection algorithm based on the control of D2D UEs. Here, we formulate the following evolutionary game to model the mode selection problem of D2D UEs. Players: We consider one large population N for the evolutionary game, in which each D2D UE under consideration is a player of the population. Strategies: Each player can choose among a set of three strategies in K = {c, d, r}, which correspond to the cellular mode, the direct reuse mode, and the relay mode defined in Section III. Population state: Let n i denote the number of D2D pairs that choose strategy i K. Then, x i = n i /n, x i 1, is termed the population share of strategy i. The population state is defined as a vector of the population shares of all three D2D modes, denoted by x = [x c,x d,x r ], where (x c +x d +x r ) = 1. Utility function: A primary motivation to incorporate D2D communications into the cellular system is to improve the spectrum efficiency. Hence, given population state x, if a D2D UE selects communication mode i on the allocated RB l M, we define its utility function as u(i,x) = R(i,x) = m l R(i,x) B/m l B where R(i, x) is the achievable data rate of the D2D UE as specified in (1), (4), or (5). As seen, the utility function in (9) quantifies the spectrum efficiency of the D2D UE with the selected mode and the allocated RB. On one hand, it favours a higher data rate. On the other hand, it intends to maximize frequency reuse level. In other words, there is a larger utility if more UEs share some common resource with minimal compromise to the achievable data rate. Many previous works define utilities as linear functions of factors in different dimensions. When the utility function combines multiple incomparable parameters with significantly different (9)

7 7 scales and units, the weighting coefficients in the linear utility function need to be appropriately tuned so that the solution can be directed toward a balanced design objective. Compared with such definitions, the utility function in (9) does not involve the linear combination of metrics in incomparable units, so it is freed from the weight-tuning problem. C. Replication by Imitation for Formulated Game According to the mutual influence and interaction among D2D UEs, each D2D UE can dynamically adapt its D2D communication mode according to the achieved utility value. When more D2D UEs switch to the cellular mode, the idle spectrum resources are decreased accordingly. When there are more D2D UEs in the direct reuse mode, it results in greater interference. In addition, when the relay mode is more beneficial, there will be higher contention for the relay UEs, which reduces the availability of relay UEs in the future mode selection. As the D2D UEs change their communication modes, the population state evolves over time. Let x(t) = [x 1 (t),...,x k (t)] denote the population state at time t, where x i (t) is the population share programmed to pure strategy i, i K, at time t. To characterize the variation of population shares over time, there are a variety of selection dynamics, among which the replicator dynamics are widely used. To facilitate the introduction of the mode selection algorithm in Section IV-D, we consider a different type of selection dynamics, i.e., replication by imitation, which is a type of pure-strategy selection dynamics arising from myopic imitation. A special case of this selection dynamics will be shown to boil down to the well-known replicator dynamics with rescaling of time. Consider that all players in the population are infinitely lived and interact forever. The time can be discretized into time slots, and each player sticks to one pure strategy for slot interval τ. Then, at the end of a time slot, a player may review its strategy and decide to switch to a new strategy in the next time slot. Assume that the review times of a player follow a Poisson process with a mean rate α. That is, a player reviews its strategy with probability ατ in each slot, comparing its current utility with that of another player randomly chosen in the population. Then, the player may shift its strategy with certain probability. For instance, if the player perceives a higher utility with the chosen player s strategy, the player may decide to shift the current strategy i to strategy j of the other player with probability p ij, given by [25] p ij = { β (uj u i ), if u j > u i, otherwise (1) where u i = u(i,x(t)) and u j = u(j,x(t)) are the utilities of strategy i and strategy j at time t, respectively, and β is a sufficiently small constant so that < p ij 1 holds for all i and j. Without loss of generality, assume that the set of strategies are numbered such that u 1 u 2... u k. As derived in [25], the expected population share x i in next slot (t+τ) is given by x i (t+τ) = x i (t) ατx i (t) + k j=i+1 x j (t)β(u j u i ) i ατx j (t) x i (t)β(u i u j ) j=1 = x i (t)+ατx i (t) k βx j (t)(u i u j ) j=1 = x i (t)+ατx i (t)β(u i u) (11) where u = k j=1 x j(t)u j is the average utility for the whole population. Based on (11), it is straightforward to evaluate lim τ [x i (t+τ) x i (t)]/τ and obtain ẋ i (t) = αβ x i (t) [ u(i,x(t)) u(x(t)) ] (12) which is the well-known replicator dynamics with a rescaling constant αβ. An EE is a strategy of a player who has no incentive to change. As seen in (12), when ẋ i (t) =, the population share x i (t) does not change any more. Hence, the EE can be found by solving a system of differential equations to obtain a fixed point with ẋ i (t) = for all players. D. Distributed Mode Selection Based on Evolutionary Game As discussed in Section IV-C, the EE can be found by solving a system of differential equations. However, as the utility definition in (9) involves complex expressions for the data rates as given in (1), (4), and (5), we cannot obtain the analytical solution to the EE. Therefore, in this section, we introduce a distributed mode selection algorithm based on the process of replication by imitation to approach the EE. Algorithm 1 shows the details of the mode selection algorithm, which is controlled in a distributed manner by each D2D UE. As seen in Lines 1-9, each D2D first randomly selects a communication mode and sends its selection to the BS. Once the BS receives the communication modes and resource usage from all D2D pairs, it obtains the current population state and the usage state of RBs and sends such state information back to the D2D pairs. Then, each D2D pair estimates the SINR and achievable data rate through local measurement or from the feedback channel of the D2D receiver. Together with the state information broadcast by the BS, each D2D pair can evaluate its utility in current mode according to (9). After the initialization phase, Lines 1-25 show the iterative procedure where a D2D UE can adapt its communication mode cycle by cycle to achieve higher utility. Here, the mode adaptation process follows the dynamics of replication by imitation introduced in Section IV-C. Specifically, in each decision cycle, a D2D UE reviews its current strategy with probability α. If a D2D pair decides to review its strategy, it randomly chooses another D2D pair s strategy according to current population state. This population state is maintained by the BS and fed back to D2D UEs in the preceding decision cycle. Since the population state gives the ratio of D2D UEs in each communication mode, a D2D pair is more likely to

8 8 Algorithm 1: A distributed mode selection algorithm based on evolutionary game. Input: Selection parameters: α, β, data rate threshold: ϕ Output: Communication modes selected for all D2D pairs: z = {z i : z i K, i N s } 1 begin Initialization 2 for i = 1 : n do 3 D2D pair i randomly selects a communication mode z i and acquires frequency resource and relay UE if needed; 4 D2D pair i informs BS of its selected mode and resource usage; 5 BS maintains current modes of all D2D pairs and corresponding population shares in x() = {x k : k K, x k 1}; 6 BS maintains usage of RBs in µ() = {m l : m l, l M}; 7 BS broadcasts current population state x() and usage state of RBs µ() to all D2D pairs; 8 for i = 1 : n do 9 D2D pair i evaluates its utility u(z i,x()) in current mode based on feedback from BS; 1 begin Iterative mode selection 11 Set t ; 12 while population state is not converged do // D2D pairs review and update their modes 13 for i = 1 : n do 14 if random number > α then // D2D pair i will not review its strategy for this cycle 15 Continue with next D2D pair; 16 Select a tentative new mode j according to probability distribution x(t); 17 D2D pair i evaluates its utility in mode j and compares it with that of current mode z i ; 18 if u(j,x(t)) > u(z i,x(t)) and R j s i,d i ϕ then 19 Set change probability: λ β(u(j,x(t)) u(z i,x(t))); // Change mode z i to j with probability λ 2 Set z i j with probability λ; 21 D2D pair i informs BS of new mode and resource usage; 22 BS updates modes of all D2D pairs and corresponding population shares in x(t + 1); 23 BS updates usage of RBs in µ(t+1); 24 BS broadcasts population state x(t + 1) and usage state of RBs µ(t+1) to all D2D pairs; 25 Set t t+1; 26 Return z; Parameter TABLE II SIMULATION PARAMETERS. Value Number of D2D UEs 39 Number of D2D pairs 195 Number of relay UEs 1 Mode selection parameter α.9 Mode selection parameter β.1 Bandwidth of RB 18 khz Maximum number of orthogonal RBs 98 Maximum number of reuse RBs 1 Maximum distance of D2D UEs to BS Maximum distance of a D2D UE pair BS transmit power per RB D2D UE transmit power per RB Mean noise power density Path loss model (db) with distance d in km 15 m 5 m 25 dbm 23 dbm dbm/hz (37.6log 1 (d)) choose a mode that a majority of D2D UEs are using if that mode provides higher utility. This is the essence of replication by imitation, i.e., imitating the majority choice. After the D2D pair chooses a new communication mode, it estimates its expected utility in the new mode as discussed above and compares that with its current utility. If the new mode can bring higher utility and meet the minimum rate requirement ϕ, the D2D pair switches to the new mode with a probability λ. As seen in Line 19, this probability of changing to the new mode is proportional to the difference of the utilities in the current mode and the new mode. After receiving the new modes of D2D pairs, the BS updates the population state and resource usage state, and broadcasts the updated information to D2D UEs, which will use this information to select a new mode in the next decision cycle. This mode adaptation process continues until the population state converges, i.e., no D2D UE would change its communication mode. V. SIMULATION RESULTS AND DISCUSSIONS A. Simulation Settings To evaluate the performance of the proposed mode algorithm, we conduct computer simulations. The main simulation parameters are given in Table II. Here, we use the wireless channel model specified by 3GPP in [26] for D2D communications in LTE networks. The channel model characterizes the large-scale fading with path loss. The transmit power and noise power settings are selected by referring to [27,28]. The maximum number of RBs for the cellular mode is set to 98, while that for the direct reuse mode and relay mode is set to 1. These numbers of RBs can be adjusted according to spectrum resource availability and the D2D user population. Regarding the spatial deployment of the system, there are one BS and 39 D2D UEs that are randomly distributed within a 4m 4m square area. This can be considered as a scenario of an ultrahigh density, because the D2D UE density is about.2 per square meter, and it is far more than the threshold.6 per square meter for ultra dense networks. The spatial distribution of the UEs is illustrated in Fig. 4. The large blue dots represent

9 9 Simulation range 2 Cellular UEs D2D UEs 15 Relay UEs BS scheme are very representative for the state-of-the-art literature on D2D mode selection and they have been shown to be quite effective. The random scheme: This baseline algorithm is inspired by the model of pure imitation in evolutionary game theory [29]. In this imitation model, each player draws another player randomly from the population according to a uniform probability distribution and adopts the strategy of the so sampled player Simulation range Fig. 4. System simulation scenario. cellular UEs that share the uplink spectrum with D2D UEs. For clarity purpose, Fig. 4 does not show other cellular UEs that are not affected by D2D UEs. The small red dots represent the D2D UEs, and the grey triangles represent the relay UEs. As seen, D2D UEs are relatively far from the BS with a maximum distance of 15m. Such a scenario is typical for a cell edge, which necessitates D2D communications to enhance the QoS of edge users and improve the cell capacity. Last, the parameters α and β for Algorithm 1 are set to.9 and.1, respectively. Their values can be adjusted to control the expected convergence speed. For comparison purpose, we consider three baseline algorithms in addition to the proposed evolutionary game based algorithm. The max-sinr scheme: This baseline algorithm is based on the idea in [9]. In this scheme, each D2D pair estimates its SINR and achievable data rates in three modes. Then, each pair always selects the communication mode that offers the highest SINR and the largest achievable data rate, provided that the new mode outperforms the current mode in terms of SINR and data rate. The distance based scheme: This baseline algorithm is adapted from the idea in [21]. In [21], a D2D pair chooses between the cellular communication mode and the D2D communication mode, by comparing the distance of the D2D pair with a maximum distance threshold. Based on the log-distance path loss model, this threshold is derived by comparing the RSS over the direct D2D link and the minimum RSS of the uplink and downlink between the D2D UEs and the BS. The D2D communication mode is used only if the distance is not more than the threshold. Adapting the idea to our scenario with three modes, we compare the RSS of the direct D2D link in the direct reuse mode, the minimum RSS of the two indirect links through the BS in the cellular mode, and the minimum RSS through the potential relay UE in the relay mode. If the highest RSS among the three modes is higher than that of the current mode, the corresponding communication mode that achieves the highest RSS is selected as the new mode. The max-sinr scheme and the distance-based B. Ratios of D2D Modes First, we conduct simulation to examine the convergence performance of the mode selection algorithms. Fig. 5 compares the four mode selection algorithms and shows the variations of the ratios of D2D UEs in the cellular mode, the direct reuse mode, and the relay mode. Fig. 5(a) shows the population shares of three D2D modes with the evolutionary game based approach. In the initial state, the ratios of D2D UEs in the three modes are set to be equal. Then, each D2D UE adapts its communication mode over time according to the mode selection algorithm. As seen, the population shares converge to stable states after a number of decision cycles. First, the population share of the cellular mode decreases fast, and eventually a smallest proportion of D2D UEs stay with the cellular mode. This is because there is a longer distance between D2D UEs and the BS and the utilization of orthogonal spectrum resources is inefficient. On the other hand, the population share of the direct reuse mode increases rapidly over time and is finally stabilized to the largest ratio. This is because a shorter distance is likely to exist between a D2D pair. Also, the reuse of uplink cellular spectrum improves spectrum efficiency and the utility. Last, the population share of the relay mode fluctuates for some slots at the beginning and then approaches the second largest ratio. This is reasonable and can be explained as follows. If a D2D pair can find a good-quality relay that is close to the D2D transmitter and receiver for the relay mode, it can achieve a higher utility than those in the direct reuse mode. However, because there are a limited number of relay UEs, fewer D2D UEs end up with the relay mode than with the direct reuse mode. As Fig. 5(b) shows, with the max-sinr mode selection scheme, a majority of D2D UEs select the cellular mode. This is because there is little interference in the cellular mode with the orthogonal spectrum resources. Thus, the D2D UEs in the cellular mode can generally achieve a high SINR. Therefore, the cellular mode becomes the first choice of most D2D UEs. However, the max-sinr mode selection scheme ends up with occupying significantly more spectrum resources. It is also observed in Fig. 5(b) that the max-sinr scheme converges very fast to a stable state. This is because the max-sinr scheme only considers SINR as the selection criterion, and thus it is much simpler than the utility function in (9), which also involves the spectrum resource utilization. Fig. 5(c) shows the population shares of D2D modes with the distance based scheme. This scheme chooses the communication mode for a D2D pair based on the distances between the

10 Ratios of D2D modes Ratios of D2D modes Cellular mode Direct reuse mode Relay mode.1 Cellular mode Direct reuse mode Relay mode (a) Evolutionary game based approach. (b) Max-SINR approach Ratios of D2D modes Ratios of D2D modes Cellular mode Direct reuse mode Relay mode.1 Cellular mode Direct reuse mode Relay mode (c) Distance based approach. (d) Random approach. Fig. 5. The ratios of D2D UEs in each mode. D2D transmitter and receiver, the BS and the relay UE. The distance information further determines the large-scale path losses and RSSs. As seen, the distance based scheme performs similarly to the max-sinr scheme in that the communication modes of all D2D UEs quickly converge to the steady states. The difference is that the cellular mode is wiped out with the distance based scheme but it achieves the largest ratio with the max-sinr scheme. This is because the two schemes use different metrics, i.e., SINR or distance, in mode selection. The cellular mode is preferable in terms of SINR but inferior due to the longer distance between D2D UEs and the BS. On the other hand, the direct reuse mode and the relay mode achieve similar population shares with the distance based scheme. Due to the availability of relay UEs, there may not always exist a relay UE for a D2D pair such that the distance between the relay UE and the D2D UEs is shorter than that of the direct link. Consequently, the direct reuse mode achieves a population share slightly larger than that of the relay mode. Fig. 5(d) shows the population shares with the random mode selection scheme. For presentation clarity, we start with initial population shares {.45,.33,.22} for the cellular mode, the direct reuse mode, and the relay mode, respectively. If we use an equal population share for each mode, the three curves for three modes are mixed together and hard to examine the variations. As seen in Fig. 5(d), the population share for each mode changes randomly over time around the initial ratio. Although the population shares do not converge to stable states, it is observed that they fluctuate around the averages set by the initial ratios. This is because in the random scheme each D2D pair just imitates the communication mode of another randomly sampled D2D UE. Such pure imitation dynamics can keep the average ratio of each mode. C. Utility, Throughput, and Spectrum Efficiency Fig. 6 shows the average utility per D2D UE with the four mode selection algorithms. As seen, the proposed algorithm based evolutionary game achieves the highest utility. Also, we observe that the average utility consistently increases with time, which demonstrates the effectiveness of the mode adaptation of D2D UEs. The max-sinr scheme and the distance based scheme perform closely in terms of the average utility. Though they are worse than the evolutionary game based algorithm, both are better than the random scheme. Fig. 7 further presents the cumulative distribution function (CDF) of the utilities of D2D UEs, which complements the average performance over time in Fig. 6. As seen in Fig. 7, the evolutionary game based algorithm is still the best, and about 55% of D2D UEs achieve a utility more than 2. In contrast, with the max-sinr scheme, only about 35% of D2D UEs achieve a utility more than 2. This ratio is slightly higher with the distance based scheme, as 3% of D2E UEs have a utility more than 2. For the random scheme, this ratio

11 11 Average utility per D2D UE Evolutionary game based Max-SINR Distance based Random Average throughput per D2D UE Evolutionary game based Max-SINR Distance based Random 1 5 Fig. 6. The average utility with four mode selection algorithms. Fig. 8. The average throughput with four mode selection algorithms. CDF of utilities Evolutionary game based Max-SINR.1 Distance based Random Utility per D2D UE Overall spectrum efficiency 35 Evolutionary game based Max-SINR Distance based Random Fig. 7. The CDF of utilities with four mode selection algorithms. Fig. 9. The overall spectrum efficiency with four mode selection algorithms. is the lowest and it is only about 26%. In summary, the proposed mode selection algorithm improves both the overall utility and the individual utility. Since the evolutionary game based algorithm is device-controlled, it can effectively enhance individual utility. Furthermore, our utility definition also takes into account the overall spectrum utilization. Thus, the utilitybased decisions of D2D UEs will not compromise overall performance for individual gain. Fig. 8 compares the average throughput per D2D UE. As seen, the max-sinr scheme in fact achieves the highest average throughput. This is because each D2D UE selects the communication mode that provides the highest SINR and thus the largest data rate. Nonetheless, the high throughput of the max-sinr scheme is obtained at the expense of occupying more spectrum resources. As seen in Fig. 5(b), with the max-sinr scheme, there are more D2D UEs that choose the cellular mode. Consequently, the traffic load at the BS increases and more spectrum resources are taken in the meantime. For the evolutionary game based approach, the average throughput performance is second to the best. This approach ensures reasonable QoS as it guarantees a minimum throughput requirement for D2D UEs. For the random scheme, D2D UEs are balanced among the three modes. As a result, the throughput performance is close to that of the game based approach and ranked the third. However, similar to the max- SINR scheme, more spectrum resources are consumed to achieve the high throughput. The average throughput of the distance based scheme is the worst, because D2D UEs are concentrated in the direct reuse mode and the relay mode due to the preferences for short communication distances. Consequently, the spectrum is reused by too many D2D UEs, which leads to high interferences and thus low SINRs and data rates. On the other hand, as shown later on, the distance based scheme is quite effective in overall spectrum utilization. Fig. 9 shows the overall spectrum efficiency of the four mode selection algorithms. Here, the overall spectrum efficiency is the ratio of the total throughput of all D2D pairs in different modes and the total frequency spectrum occupied by D2D UEs. As seen, the distance based scheme achieves the highest overall spectrum efficiency. This is because in the distance based scheme all D2D UEs select either the direct reuse mode or the relay mode with spectrum sharing. As a

12 12 result, only a small number of RBs are occupied and shared by the D2D UEs. Nonetheless, due to the high interferences with spectrum reuse, the distance based scheme cannot provide high utility or high throughput to D2D UEs. For the proposed evolutionary game based algorithm, the overall spectrum efficiency is slightly lower than that of the distance based scheme, but it is significantly higher than those of the max-sinr scheme and the random scheme. In addition, the results in Fig. 8 and Fig. 9 demonstrate that the evolutionary game based scheme can effectively balance between throughput and spectrum efficiency, and achieve the highest utilities shown in Fig. 6 and Fig. 7. On one hand, the D2D UEs favour more spectrum resources to lower interferences and improve throughput. On the other hand, higher spectrum consumption reduces the spectrum efficiency of the system. To balance between the contradictory aspects, the evolutionary game based scheme leverages the natural competitions among D2D UEs based on the utility function to distribute D2D UEs over different communication modes while ensuring a reasonable throughput for each D2D pair. VI. CONCLUSION AND FUTURE WORK As D2D UEs share the radio resources with cellular UEs to improve spectrum utilization, mode selection for D2D UEs is an essential research issue to deal with the complicated network interference environment. In this paper, we considered three D2D modes, i.e., the popular cellular mode and direct reuse mode, as well as a new relay mode. As the ultra dense 5G networks are likely to have a large population of D2D UEs, we employed an evolutionary game based approach to design a distributed D2D mode selection algorithm. The evolutionary game is formulated with a utility function that takes into account both the achievable throughput of D2D UEs and the radio resource consumption. Based on the evolutionary game formulation, we implemented selection dynamics, i.e., replication by imitation, in a device-controlled mode selection algorithm. In the performance evaluation, we carried out extensive simulations to compare the proposed D2D mode selection algorithm with three representative baseline schemes in the literature. It is observed that our evolutionary game based algorithm can quickly converge to a stable state. Eventually, the direct reuse mode occupies the largest population share, followed by the relay mode and the cellular mode. It is also shown that our proposed approach can make the best use of the spectrum for different D2D modes to achieve the highest utility. On the other hand, the max-sinr scheme achieves the highest throughput, while the distance based scheme is the best in terms of overall spectrum efficiency. However, these baseline schemes cannot effectively balance between the throughput performance and spectrum cost, and thus end up with lower utilities. In the future, this work can be extended to consider other D2D communication modes, such as the direct communication mode with orthogonal resource allocation or reusing the downlink resources, or the relay mode with smallcell base stations. REFERENCES [1] X. Wu, S. Tavildar, S. Shakkottai, T. Richardson, J. Li, R. Laroia, and A. Jovicic, FlashLinQ: A synchronous distributed scheduler for peer-topeer ad hoc networks, IEEE/ACM Transactions on Networking, vol. 21, no. 14, pp , 213. [2] W. Song and W. Zhuang, Packet assignment under resource constraints with D2D communications, IEEE Network, vol. 3, no. 5, pp. 54 6, 216. [3] X. Wang, M. Chen, T. Kwon, L. Jin, and V. Leung, Mobile traffic offloading by leveraging opportunistic device-to-device sharing and exploiting social network services, IEEE Wireless Communications Magazine, vol. 21, no. 3, pp , 214. [4] J. Feng, L. Zhao, J. Du, X. Chu, and F. R. Yu, Computation offloading and resource allocation in D2D-enabled mobile edge computing, in Proc. IEEE International Confernece on Communciations, May 218, pp [5] Z. Zhou, M. Dong, K. Ota, R. Shi, Z. Liu, and T. Sato, Gametheoretic approach to energy-efficient resource allocation in device-todevice underlay communications, IET Communications, vol. 9, no. 3, pp , 215. [6] A. Sultana, L. Zhao, and X. Fernando, Efficient resource allocation in device-to-device communication using cognitive radio technology, IEEE Transaction on Vehicular Technology, vol. 66, no. 11, pp , 217. [7] N. Cheng, N. Lu, N. Zhang, T. Yang, X. Shen, and J. W. Mark, Vehicleassisted device-to-device data delivery for smart grid, IEEE Transaction on Vehicular Technology, vol. 66, no. 4, pp , 216. [8] W. Song, Y. Zhao, and W. Zhuang, Stable device pairing for collaborative data dissemination with device-to-device communications, IEEE Internet of Things Journal, vol. 5, no. 2, pp , 218. [9] R. Ma, N. Xia, H.-H. Chen, C.-Y. Chiu, and C.-S. Yang, Mode selection, radio resource allocation, and power coordination in D2D communications, IEEE Wireless Communications, vol. 24, no. 3, pp , 217. [1] T. D. Hoang, L. B. Le, and T. Le-Ngoc, Joint mode selection and resource allocation for relay-based D2D communications, IEEE Communications Letters, vol. 21, no. 2, pp , 217. [11] M. Belleschi, G. Fodor, and A. Abrardo, Performance analysis of a distributed resource allocation scheme for D2D communications, in Proc. IEEE GLOBECOM Workshops, 211, pp [12] K. Akkarajitsakul, P. Phunchongharn, E. Hossain, and V. K. Bhargava, Mode selection for energy-efficient D2D communications in LTE- Advanced networks: A coalitional game approach, in Proc. IEEE International Conference on Communication Systems (ICCS), 212, pp [13] P. Cheng, L. Deng, H. Yu, Y. Xu, and H. Wang, Resource allocation for cognitive networks with D2D communication: An evolutionary approach, in Proc. IEEE Wireless Communications and Networking Conference (WCNC), 212, pp [14] S. Bulusu, N. B. Mehta, and S. Kalyanasundaram, Rate adaptation, scheduling, and mode selection in D2D systems with partial channel knowledge, IEEE Transactions on Wireless Communications, vol. 17, no. 2, pp , 218. [15] C. Yu, O. Tirkkonen, K. Doppler, and C. Ribeiro, Power optimization of device-to-device communication underlaying cellular communication, in Proc. IEEE International Conference on Communications (ICC), 29, pp [16] K. Doppler, C.-H. Yu, C. B. Ribeiro, and P. Janis, Mode selection for device-to-device communication underlaying an LTE-Advanced network, in Proc. IEEE Wireless Communications and Networking Conference (WCNC), 21, pp [17] C. Yu, K. Doppler, C. B. Ribeiro, and O. Tirkkonen, Resource sharing optimization for device-to-device communication underlaying cellular networks, IEEE Transactions on Wireless communications, vol. 1, no. 8, pp , 211. [18] N. Reider and G. Fodor, A distributed power control and mode selection algorithm for D2D communications, EURASIP Journal on Wireless Communications and Networking, vol. 212:266, pp. 1 25, 212. [19] S. Hakola, T. Chen, J. Lehtomaki, and T. Koskela, Device-to-device (D2D) communication in cellular network-performance analysis of optimum and practical communication mode selection, in Proc. IEEE Wireless Communications and Networking Conference (WCNC), 21, pp [2] H. Chen, X. Deng, M. Gao, L. Yang, L. Guo, and M. Chi, Location related communication mode selection and spectrum sharing for D2D

13 13 communications in cellular networks, in Proc. Intelligent Transportation, Big Data & Smart City (ICITBS), 218, pp [21] A. Omri and M. O. Hasna, A distance based mode selection scheme for D2D enabled networks with mobility, IEEE Transactions on Wireless Communications, vol. 17, no. 7, pp , 218. [22] K. Zhu and E. Hossain, Joint mode selection and spectrum partitioning for device-to-device communication: A dynamic Stackelberg game, IEEE Transactions on Wireless Communications, vol. 14, no. 3, pp , 215. [23] The 3rd Generation Partnership Project (3GPP), TR 21.9 V14.1.1: Work item proposal for enhanced LTE device to device proximity services (release 13), 215. [24] J. C. Li, M. Lei, and F. Gao, Device-to-device (D2D) communication in MU-MIMO cellular networks, in Proc. IEEE Global Communications Conference (GLOBECOM), 212, pp [25] H. Gintis, Game Theory Evolving: A Problem-Centered Introduction to Modeling Strategic Interaction. Princeton University Press, 29. [26] The 3rd Generation Partnership Project (3GPP), TR V1..: Study on LTE device to device proximity services (ProSe) Radio aspects (release 12), 213. [27] Z. Zhou, M. Dong, K. Ota, G. Wang, and L. T. Yang, Energy-efficient resource allocation for D2D communications underlaying cloud-ranbased LTE-A networks, IEEE Internet of Things Journal, vol. 3, no. 3, pp , 216. [28] Z. Zhou, K. Ota, M. Dong, and C. Xu, Energy-efficient matching for resource allocation in D2D enabled cellular networks, IEEE Transactions on Vehicular Technology, vol. 66, no. 7, pp , 217. [29] J. Weibull, Evolutionary Game Theory. The MIT Press, Yujie Li received her B.S. degree in communication engineering from Changsha University of Science and Technology, Changsha, China, in 21. She received her M.S. degree, also in communication engineering, from Xiamen University, Xiamen, China, in 214. She is now a Ph.D. candidate in the Department of Communication Engineering of Xiamen University. Her current research interests include wireless communications, interference management and device-to-device communications. Ziwen Su received his B.S. degree and M.S. degree, both in communication engineering, from Xiamen University, Xiamen, China, in 213 and in 216, respectively. His research interests include resource management and mode selection in device-to-device communication networks. Lianfen Huang received her B.S. degree in radio physics in 1984 and Ph.D. degree in communication engineering in 28 from Xiamen University, Xiamen, China. She was a visiting scholar at Tsinghua University in 1997 and the Chinese University of Hong Kong in 212. She is now a Full Professor in the Department of Communication Engineering at Xiamen University. Her current research interests include wireless communications, wireless networks and signal processing. Wei Song (M 9-SM 14) received the Ph.D. degree in electrical and computer engineering from the University of Waterloo, Waterloo, ON, Canada, in 27. In 29, she joined the Faculty of Computer Science, University of New Brunswick, Fredericton, NB, Canada, where she is now an Associate Professor. Her current research interests include device-todevice communications, Internet of Things, mobile edge computing, and mobile crowd sensing. She received a Best Paper Award from the 218 IEEE ICC, a UNB Merit Award in 214, a Best Student Paper Award from the 213 IEEE CCNC, a Top 1% Award from the 29 IEEE MMSP, and a Best Paper Award from the 27 IEEE WCNC. She is the Communications/Computer Chapter Chair of IEEE New Brunswick Section. She co-chaired tracks/symposiums for IEEE VTC Fall 21, IWCMC 211, IEEE GLOBECOM 211, IEEE ICC 214, IEEE VTC Fall 216 and IEEE VTC Fall 217. Zhibin Gao received his B.S. degree in communication engineering, M.S degree in radio physics, and Ph.D. degree in communication engineering, from Xiamen University, Xiamen, China, in 23, 26, and 211, respectively. He was a visiting scholar at the University of Washington, Seattle, WA, USA, in 216. He is currently a senior research engineer at Xiamen University. His current research interests include wireless communications, wireless resource management and signal processing.