IEEE/ACM TRANSACTIONS ON NETWORKING 1

Transcription

1 IEEE/ACM TRANSACTIONS ON NETWORKING 1 Otimal Power Allocation and Scheduling Under Jamming Attacks Salvatore D Oro, Eylem Ekici, and Sergio Palazzo Abstract In this aer, we consider a jammed wireless scenario where a network oerator aims to schedule users to maximize network erformance while guaranteeing a minimum erformance level to each user. We consider the case where no information about the osition and the triggering threshold of the jammer is available. We show that the network erformance maximization roblem can be modeled as a finitehorizon joint ower control and user scheduling roblem, which is NP-hard. To find the otimal solution of the roblem, we exloit dynamic rogramming techniques. We show that the obtained roblem can be decomosed, i.e., the ower control roblem and the user scheduling roblem can be sequentially solved at each slot. We investigate the imact of uncertainty on the achievable erformance of the system and we show that such uncertainty leads to the well-known exloration-exloitation tradeoff. Due to the high comlexity of the otimal solution, we introduce an aroximation algorithm by exloiting state aggregation techniques. We also roose a erformance-aware online greedy algorithm to rovide a low-comlexity sub-otimal solution to the joint ower control and user scheduling roblem under minimum quality-of-service requirements. The efficiency of both solutions is evaluated through extensive simulations, and our results show that the roosed solutions outerform other traditional scheduling olicies. Index Terms Scheduling, ower control, jamming, QoS. I. INTRODUCTION MANY resource allocation roblems in the networking domain can be modeled as network utility maximization roblems where cross-layer otimization techniques at both PHY and MAC layers are used to maximize network erformance. Frequently, due to the time-varying behavior of both network conditions and resource availability, the allocation of network resources has to be erformed within a fixed and finite temoral window, i.e., the horizon of the otimization roblem. The resource allocation roblem is further exacerbated with the inclusion of Quality-of-Service QoS constraints and third-arty entities such as jamming attackers over which network oerators have no control. In articular, reactive jamming attacks in wireless communications have been roven Manuscrit received Aril 23, 2016; revised October 15, 2016; acceted October 23, 2016; aroved by IEEE/ACM TRANSACTIONS ON NETWORK- ING Editor X. Liu. S. D Oro and S. Palazzo are with the University of Catania, Catania, Italy sdoro@dieei.unict.it; salazzo@dieei.unict.it. E. Ekici is with The Ohio State University, Columbus, OH USA ekici.2@osu.edu. Digital Object Identifier /TNET to be one of the most threatening and harmful attacks as they can comletely or artially disrut ongoing communications [1]. Reactive jammers continuously monitor one or multile channels, searching for ongoing transmission activities by means of energy detectors [2]. Then, only when a transmission is detected, the jammer starts its attack. The attacker node is able to transmit jamming signals that interfere with those transmitted by legitimate users, thus causing a dro in the Signal-to-Interference-lus-Noise Ratio SINR of such users. Reactive jamming attacks reach a high jamming efficiency and can even imrove the energy-efficiency of the jammer in several alication scenarios [2], [3]. Also, they can easily and efficiently be imlemented on COTS hardware such as USRP radios [1], [4], [5]. But, more imortantly, reactive jamming attacks are harder to detect due to the attack model, which allows jamming signal to be hidden behind transmission activities erformed by legitimate users [4], [6], [7]. In this aer, we adot this more general and efficient attack model. For a more detailed discussion on jamming and anti-jamming techniques we refer the reader to [8] [11] and the references therein. Main challenges in dealing with reactive jamming attacks are 1 to detect the resence of a jammer and 2 to develo roer anti-jamming mechanisms to avoid the disruting actions of the jammer. It has been shown that detecting the resence of a jammer requires comlex or time-consuming oerations, such as statistical estimations and continuous acket monitoring [6], [7], [12]. Moreover, customized solutions to counteract jamming attacks are non-trivial as the exact behavior of the jammer is generally unown and unredictable. It is clear that under such hostile conditions, network erformance can be significantly reduced. Without roer antijamming techniques, it is not easy to otimize network erformance within a finite-horizon. Most anti-jamming techniques need either a riori information on jammer s behavior and channel conditions or a large number of temoral slots to become effective. Also, guaranteeing a minimum erformance level to all users in the network can be even more challenging, esecially if service guarantees must be met in a limited temoral window. As an examle, a caacity maximization roblem is investigated in [13] where authors aim to maximize the overall number of successful transmissions of a jammed network consisting of several users by exloiting distributed no-regret learning techniques. However, this aroach does not guarantee a minimum erformance level to network users IEEE. Personal use is ermitted, but reublication/redistribution requires IEEE ermission. See htt:// for more information.

2 2 IEEE/ACM TRANSACTIONS ON NETWORKING Also, to find the otimal solution, the iterative learning algorithm roosed in [13] requires a large number of iterations, a condition that cannot always be satisfied. Solutions to a variety of QoS rovisioning roblems in unjammed scenarios have been roosed in the literature. As an examle, in [14] and [15] authors roose otimal solutions for the caacity maximization roblem under minimum erformance guarantee constraints. However, such solutions do not consider ossible jamming attack, and it is shown that the roosed aroaches converge to the otimal solution when the horizon tends to infinity. Therefore, it is hard to aly these solutions to the roblem we address in this work. In this aer, we consider a multi-user multi-carrier timeslotted cellular network where the network oerator has to schedule network users to maximize achievable network erformance while guaranteeing a minimum erformance level to each user. We also consider the worst case scenario where a reactive jammer is deloyed within the base station BS coverage range. Each node is affected by its jamming activity and the transmission ower of each user cannot exceed a given maximum threshold. We further assume that channel conditions vary in time according to the block-fading model. We show that the above roblem can be modeled as a finitehorizon joint ower control and user scheduling roblem. Also, we rove that finding an otimal solution is NP-hard. We formulate the roblem by exloiting techniques from Dynamic Programming DP. The DP formulation allows us to show that the joint ower control and user scheduling is a decomosable roblem. That is, at each otimization ste we can sequentially solve the ower control and the user scheduling roblems. We show that, under some conditions, it is ossible to identify the otimal ower control olicy, i.e., conservative, exloratory or aggressive. To avoid the curse of dimensionality of the DP aroach, we exloit state aggregation techniques to roose an aroximated solution and study its comlexity. We also roose a Performance-aware Online Greedy Algorithm POGA to rovide a low-comlexity sub-otimal solution for the joint ower control and scheduling roblem under minimum QoS requirements. Accordingly, we resent an algorithmic imlementation of POGA and we show that its comutational comlexity is olynomial in the number of users in the system. We evaluate the achievable erformance of both the aroximated solution and POGA through extensive simulation results. Also, we investigate the imact of jammer s behavior and osition on achievable system s erformance. Finally, we comare the erformance of the above roosed solutions with those achieved by traditional scheduling olicies. We show that both our solutions outerform other considered aroaches. The rest of the aer is organized as follows. Related work is resented in Section II. In Section III, we first illustrate the system model and, then, we formulate the joint ower control and user scheduling roblem. In Section IV, we roose a decomosable DP formulation of the original roblem. A discretized version of the original roblem is roosed in Section V, and POGA is resented in Section VI. In Section VII, we evaluate the achievable erformance of the roosed solutions through numerical simulations. Finally, Section VIII concludes the aer. II. RELATED WORK Providing jamming-roof communications is an interesting toic and several solutions have been roosed in the literature. For examle, sread-sectrum techniques are commonly used to avoid the jammer and its attacks [16] [19]. However, to be effective, such techniques need to either share or establish a secret among network users. For this reason, such techniques cannot be alied in all wireless scenarios [20], [21]. In [20], [22], and [23] a trigger-identification aroach is resented. First, nodes whose transmission trigger the reactive jammer, i.e., the triggering nodes, are identified. Then, otimal routing aths are established, which exclude triggering nodes from the routing rocess. However, such solutions are designed for large sensor networks where multi-ho communication is feasible. Such solutions fail in scenarios where the whole network is under attack and any node can otentially be a trigger node. To detect transmission activities, the jammer has to first sense ongoing communications. Then, when an activity is detected the jammer starts its attack. Thus, there is a delay, i.e., activation time, between the detection and attack hases. All bits transmitted during the activation time escae jamming and can be exloited to establish communications under reactive jamming attacks [21]. Another aroach is roosed in [24] [26] where radio silences are exloited to establish secure communications. As the reactive jammer does not attack when no transmission activities are erformed by users, it is ossible to encode the information to be transmitted in silences between consecutive ackets. Accordingly, although transmitted ackets can be comletely damaged, the radio silence between consecutive ackets can be modulated to convey data by maing bit sequences and silence eriod durations. However, the above solutions neither aim to maximize network erformance nor rovide any minimum QoS service level to network users. Power control has been recently roosed to overcome ossible reactive jamming attacks [27], [28]. The intuition is that by controlling the transmission ower of users, it is ossible to let the received ower at the jammer remain under the triggering threshold [28]. In [27], authors consider a single sender-receiver air and roose a joint frequency hoing FH and ower control scheme to avoid reactive jamming attacks. The roosed solution consists in selecting user s transmission ower such that the sender s ower at the receiver side is higher than that of the jammer. However, the latter aroach fails when dealing with systems with multile senders and where FH is not ossible such as the one we consider in this aer. III. SYSTEM MODEL A. Network Model We consider a multi-carrier slotted wireless system where a set of users access the network and communicates with a shared BS through several non-interfering channels. Let N be

3 D ORO et al.: OPTIMAL POWER ALLOCATION AND SCHEDULING UNDER JAMMING ATTACKS 3 Fig. 1. Channel model. TABLE I NOTATION the set of users and K be the set of the channels accessed by these users. We assume that users are ower-constrained and are equied with single-antenna transceivers. Their instantaneous transmission ower is limited to a maximum ower level P and they can transmit on at most one channel at a time. Also, we assume that a given slot on a given channel can be assigned to only one user. In our aer, we focus on the uli scheduling roblem. In our model, we assume Additive White Gaussian Noise AWGN channels with channel gains defined as i.i.d. random variables. As shown in Fig. 1, let h be the channel gain coefficients between user n and the jammer on channel k, while h k indicates the channel gain coefficient between the jammer and the BS on channel k. Finally, g indicates the channel gain between user n and the BS on channel k. Relevant arameters and variables are summarized in Table I. We assume block fading, that is, channel gain coefficients remain constant for a fixed number of slots before they change. Let H be the number of slots where channel gain coefficients remain constant. Therefore, the otimal scheduling roblem has to be eriodically erformed every H slots. We refer to such time eriod as the scheduling cycle. Accordingly, H is the finite-horizon of the otimization roblem. 1 B. Attack Model We assume that a malicious user, i.e., the jammer, aims to disrut ongoing communications between legitimate users 1 Our model also alies to the case of a mobile jammer attacks the network. Since the jammer wants to be undetectable and unredictable, it moves and changes its osition every H slots. It follows that channel gain coefficients vary in time and the network oerator has to eriodically find the otimal scheduling olicy every H slots. and the BS. More secifically, in our model we consider a reactive jamming attacker where the malicious user continuously monitors all channels in K searching for transmission activities and jams only those channels where the received signal ower is higher than a given threshold. In our study, we assume that when a transmission activity is detected in a given slot, the jammer emits a jamming signal whose duration is equal to the slot duration. 2 To switch from the monitoring to the transmission hases causes a delay also referred to as the activation time. Such delay has been shown to be small [1], and are negligible comared to the duration of each slot. Thus, we assume that the jammer is able to instantaneously switch between RX and TX front-ends. Let P th and P J denote the triggering threshold and the transmission ower of the jammer, resectively. We assume that the jammer s attack strategy is indeendent of channels and users; that is, the values of P th and P J are constant and equal for all k Kand n N.However,itisworth noting that the aroach roosed in this aer also alies to the more general case where the values of P th and P J indeendently vary accross users and channels. That is, our model can be also alied to the case where the triggering threshold and the transmission ower of the jammer for user n N and channel k Kare P th and P J, resectively. To model the triggering mechanism that regulates the jammer, we define the triggering function α :R {0, 1} for user n transmitting on channel k. More secifically, α = { 1 if h P th 0 otherwise where is the transmission ower for n on channel k. Intuitively, according to eq. 1, an attack is erformed only when the received ower at the jammer side h is greater than or equal to the triggering threshold P th. Clearly, α 0 = 0. We consider the worst case scenario where α P =1, i.e., all nodes are jammed by the jammer when they transmit with full ower. However, the more general case where α P =0 can be similarly treated by exloiting the same techniques resented in this aer. Let us consider the generic channel k Kand let m and M be two feasible transmission ower levels for a given user n N such that 0 m < M P. We further assume that α m =0and α M =1, that is, the system knows that when user n transmits with ower m on channel k it does not trigger the jammer, while transmitting with ower M on the same channel activates the jammer and consequently causes a decrease in the SINR. 3 Therefore, the robability of triggering the jammer when transmitting with ower given the values 2 To consider the case where the duration T J of the jamming signal is shorter than the slot duration T, the model here roosed has to be slightly modified. Secifically, the exected rate of the system has to be rewritten as R = 1 T J log 1+ g T σ 2 + T J E T α {R πj}, wheree α {R πj} is defined in eq. 7. However, it can be easily shown that the results resented in this work also aly to this secific case. 3 The existence of both m and M is always guaranteed by our assumtions and in Section IV we rovide roer mechanisms to identify the values of both arameters. 1

4 4 IEEE/ACM TRANSACTIONS ON NETWORKING of both m and M can be written as F =Pr{h P th π } =Pr { } Pth π h where π = m, M is a tule that reresents the history or knowledge of the system. As shown in eq. 2, the robability of triggering the jammer deends on the ratio P th /h between the triggering threshold of the jammer and the channel gain coefficient between the jammer and the transmitter. Although in reality the osition and the triggering threshold of the jammer are unown and, thus, the exact value of the ratio P th /h is unown to the network oerator, the information contained in the history π = m, M is still available. Note that given π, the exact value of the ratio P th /h can be any value in the range m, M ]. Therefore, to model such uncertainty on the knowledge of such arameters we assume that the ratio P th /h is modeled as a uniformly distributed random variable 4 over the interval m, M ]. Accordingly, we rewrite eq. 2 as follows: 0 if m m F = M m if m << M 3 1 otherwise C. Problem Formulation As a consequence of the AWGN assumtion, for any given user n N scheduled on channel k K, the SINR received at the BS side is g SNR = σ α h k P J where is the transmission ower and σ 2 is the variance of the AWGN which we assume to be equal for all n N and k K. We assume that the channel gain coefficients g can be accurately estimated. Since the transmission ower of each user is chosen by the centralized entity and all g are known, the roduct h k P J can be accurately obtained by the BS by comaring the exected received signal with the actual received signal. Let H = {1, 2,...,H} be the set of slots in a scheduling cycle. Accordingly, we define the achievable rate R at slot j Has follows: g j R j = log 1+ σ α jh k P J Let θ j and j be the allocation indicator and ower control variable, resectively. If user n is allocated to channel k at slot j, the allocation indicator is set to one, i.e., θ j =1, otherwise it is set to zero, i.e., θ j =0. Similarly, j denotes the transmission ower which must take values in the range [0,P] due to the ower constraint we have discussed in Section III-A. Let θj =θ j n,k and j = j n,k be the scheduling olicy and ower 4 Note that the uniform distribution assumtion is well-suited to cature this worst-case scenario where the actual value of the triggering threshold is unown to the network oerator and all values in the range m, M ] are equirobable. 2 control olicy at slot j, resectively. Clearly, if θ j =0, then we set j =0.Also,foranyn N and k K,let π j = m j,m j be the history u to slot j, where m j =max { l j: α l = 0, l < j} M j =min { l j: α l = 1, l < j} and j = l n,k,l<j is the set of all the ower control decisions taken u to slot j. By assumtion, we have π 1 = 0,P for all n N and k K. Intuitively, at each slot the system kees track of the reaction of the jammer to different olicies chosen in the ast. To evaluate 5, we need to know the triggering function exactly. Unfortunately, the reaction of the jammer, i.e., the outcome of the triggering function α j, is known only at the end of each slot. Accordingly, from eqs. 1, 3 and 5, the exected achievable rate for user n on channel k is E α {R π j} m = j g M j log 1+ m j σ 2 + h k P J + 1 m j M j m j log 1+ g σ 2 where the exectation is taken w.r.t. the outut of the triggering function given that the history of the system at slot j is π j. We define the following finite-horizon joint ower control and scheduling roblem with minimum erformance guarantee under jamming attacks Problem A. A : max E α θ jr j θ, j H k K n N s.t. θ j 1, n N, j H 8 k K θ j 1, k K, j H 9 n N E α R j R n, n N 10 j H k K 6 7 θ j {0, 1}, n N, k K, j H 11 j [0,P], n N, k K, j H 12 where θ = θ1, θ2,...,θh; = 1, 2,...,H are the decision variables; α = α j n,k,j is the set of all outcomes of the triggering function according to the actual ower control olicy j at slot j; andrn is the minimum rate requirement of user n. In Problem A, constraint 8 guarantees that at any given time a user can be allocated to only one slot. On the other hand, constraint 9 ensures that only one user can be allocated on a given slot, thus avoiding ossible collisions and/or interferences among users. The minimum erformance constraint is imosed by the non-linear constraint 10 which ensures that the exectation of the rate achieved by any user at the end of the otimization horizon is higher than or equal to the erformance requirement Rn. Finally, constraints 11 and 12 guarantee the feasibility of the decision variables.

5 D ORO et al.: OPTIMAL POWER ALLOCATION AND SCHEDULING UNDER JAMMING ATTACKS 5 D. Hardness of the Problem A is a non-linear concave combinatorial otimization roblem with both discrete i.e., θ j and continuous i.e., j decision variables. In this section, we rove that A is NP-hard by showing that the Multirocessor Scheduling is olynomially reducible to a subroblem of A. The multirocessor scheduling is known to be NP-comlete [29] and it is stated as follows: given m rocessors, a deadline D and a set X of jobs where each job x n X has length l n,istherea m-rocessor scheduling that schedules all jobs and meets the overall deadline D? Theorem 1 NP-Hardness: Problem A is NP-hard. Proof: Let D = H, X = N, x n = n, m = K. Let us assume P th =0, i.e., any user transmission triggers the jammer and α =1for all >0, n N and k K.Let j be the otimal transmission ower level. In this secific instance of Problem A, it is straightforward that j =P for all n N, k Kand j H. Now let us assume that the minimum erformance requirement Rn is such that Rn < min k K {R P }. Accordingly, we define the length of job R n max k K {R P } x n as l n = T,whereT =1is the duration of a single slot. The above subroblem of A requires us to find the rate-otimal scheduling of users i.e., the jobs among the available channels i.e., the rocessors while satisfying their minimum erformance requirement i.e., the job s length within the considered horizon i.e., the deadline. This way, we have built a reduction of the multirocessor scheduling to an instance of a subroblem of A. Thus, the above instance is NP-comlete by reduction [29]. Also, since this reduction can be made in olynomial time, it follows that A is NP-hard and, unless P=NP, it cannot be solved in olynomial-time. IV. OPTIMAL SOLUTION In Theorem 1, we have shown that A is NP-hard. However, how to find an otimal solution still remains unsolved. In this section, we show that A is a dynamic roblem. Moreover, we show that at each slot the joint ower control and user scheduling roblem is decomosable. That is, at each slot it is ossible to searately solve the ower control and the user scheduling roblems. A. Problem Dynamism and Decomosability In revious sections, we have shown that to maximize the achievable erformance of the system, i.e., the overall transmission rate, the scheduler has to evaluate the exected achievable rate for all users at each slot. Eq. 7 shows that the exected achievable rate for a given user on a given channel deends on the history arameter π j. In turn, eq. 6 shows that the history π j deends on actions taken in the ast. Therefore, the knowledge and the state of the system dynamically evolve at each slot. Intuitively, a DP aroach is well-suited to model and solve the considered roblem. Another imortant issue is whether or not the roblem is decomosable. To maximize the overall achievable rate of network, the scheduling roblem requires us to first estimate the achievable erformance of each user. On the contrary, as shown in eq. 7, to maximize the single-slot exected rate, Fig. 2. Structure of the dynamic rogramming roblem. the ower control roblem only needs to know the history arameter π j for all n and k. Recall that π j does not deend on the scheduling olicy at the actual slot, but only deends on the scheduling decisions taken in the ast. Hence, at each slot, the ower control roblem can be solved indeendently of the actual scheduling olicy. However, the user scheduling roblem needs the outut of the ower control roblem. Therefore, as shown in Fig. 2, we first solve the ower control roblem and find the otimal transmission ower level for each user on each channel. Then, we solve the scheduling roblem. It is of extreme imortance to remark that the decomosition of the roblem does not invalidate the otimality of the solution. As we have already ointed out above, to find the otimal ower control olicy does not deend on the actual scheduling olicy but only on the decisions taken in the ast. The converse is not true as the scheduling roblem reliminarily requires to calculate the achievable rate of all users, which imlies that the ower control roblem has to be solved first. Accordingly, the decomosition of the roblem still rovides the otimal solution to the joint ower control and user scheduling roblem. Parameters shown in Fig. 2 and their imortance will be described in the two following sections. B. Otimal Power Control To solve the single-slot ower control roblem, we must find the otimal transmission ower level for all users on each channel. Constraints 8 and 9 imly that no collision may occur and we can searately solve the ower control roblem for any individual user. For each slot j and channel k K, we define the single-user ower control roblem Problem B as follows: { } B: max max E α {R π j},r P Π j where Π j =[ m j,m j]. The challenges of B are twofold as: i the roblem evolves according to ast choices; ii the reaction of the jammer is observed only at the end of the slot. Therefore, we must consider all ossible realizations of the triggering function α. Let j be defined as follows: j = arg max Π j E α {R π j} 13 From eq. 7, eq. 13 can be rewritten as follows: m j = arg max j g log 1+ Π j Δ j σ 2 + h k P J + 1 m j log 1+ g Δ j σ 2

6 6 IEEE/ACM TRANSACTIONS ON NETWORKING Fig. 3. Comarison between different realizations of the exected achievable rate solid lines as a function of the transmission ower. Gray dots reresent otimal transmission ower olicies: a conservatory; b aggressive; and c exloratory. Dashed-dotted lines show the achievable rate when no attacks are erformed α =0. Dashed lines show the achievable rate when the user is under attack α =1. where Δ j = M j m j is the Lebesgue measure of Π j. In Proosition 1, we show that B admits a unique otimal solution. Proosition 1: Problem B always admits a unique solution OPT j defined as { OPT j = j if E α {R j} R P 14 P otherwise Proof: The roof consists in showing that the function E α {R π j} is strictly concave and, therefore, it admits a unique maximizer. Let y = h k P J. By looking at the second order derivative of E α {R π j}, it can be shown that strict concavity holds if gy m [y +2σ 2 + g] σ 2 + g + y <g M m σ 2 + g + y +2yσ 2 + g 15 where, for the sake of simlicity, we have omitted all subscrits and the slot index. However, if we remove the negative term in the left hand side of eq. 15, i.e., we consider a maximization of the first term in eq. 15, and do some algebra, it follows that gy[y +2σ 2 + g] σ 2 + g + y <g M m σ 2 + g + y+2yσ 2 + g yields to y 2 < 2y 2, which always holds if y 0.Furthermore, it is straightforward to rove that also y =0, i.e., no jamming attack is erformed by the attacker, leads to strictly concavity. To rove the second art of the roosition, it suffices to note that E α {R j π j} > R P imlies that the otimal transmission ower level is OPT j = j. Instead, if E α {R j π j} = R P, both j and P are otimal. Without losing in generality, if both olicies are otimal, then we assume that OPT j = j. Finally, if E α {R j π j} <R P, it follows that OPT j =P. Proosition 1 suggests that there are some scenarios where transmitting with the highest ower, i.e., P, and triggering the jammer is the otimal ower control solution. For examle, if the jammer is in roximity of a user but far away from the BS, it is reasonable to assume that even low transmission ower levels can trigger the jammer. Therefore, to transmit with a low ower level to avoid the jammer can be inefficient. It follows that transmitting with the highest ower P and triggering the jammer can be the only otimal olicy. Clearly, j deends on the values of several arameters such as P J, channel gain coefficients and π j. Therefore, it is hard to know a riori the otimal olicy chosen by the scheduler. However, to better understand the dynamics that regulate the ower control roblem, we define three different olicies: OPT conservative: is a olicy where OPT j = m j. Recall that α m j = 0. Therefore, under such a olicy Fig. 3, the scheduler chooses to avoid the jammer by choosing a safe strategy which ensures that the jammer will not be triggered; aggressive: is a olicy where the otimal ower control olicy consists in transmitting with full ower, i.e., OPT j =P. In general, such a olicy is otimal when jamming activities does not affect the erformance of the system significantly Fig. 3; δ C π j = σ2 + m jg [ log 1+ h ] kp J h k P J g σ 2 log 1+ σ 2 + m jg 16 ˆδ π C j = M j Pσ 2 /σ 2 + h k P J 17 δπ A j = σ2 + M jg [ + h k P J log 1+ h ] kp J h k P J g σ 2 log 1+ σ 2 + M jg 18 log 1+ h kp J ˆδ π A j = j m σ log 1+ j 2 log 1+ h kp J σ 2 h k P J σ 2 +g j log σ 2 +h k P J +g P σ 2 +g j 19

7 D ORO et al.: OPTIMAL POWER ALLOCATION AND SCHEDULING UNDER JAMMING ATTACKS 7 exloratory: in this case, a transmitting ower m j,m j is otimal Fig. 3. The scheduler decides to take the risk by exloring new transmitting ower levels to which jammer s reaction is unown. The above olicies are in line with the vast body of literature on the exloration-exloitation trade-off [30], where the decision maker has to choose between gathering new information by exloring new actions, or exloit the already exlored actions whose system s reactions are already known. When an exloratory olicy is chosen, the scheduler takes the risk and exlores new ower control olicies, even though such decision could trigger the jammer. When conservative and aggressive olicies are chosen, the reaction of the jammer, together with the achievable erformance under such olicies, can be exactly redicted. Therefore, conservative and aggressive olicies are exloitation decisions. In the rest of the aer, we will refer to the exloration of new ower control olicies as the learning dynamics or learning rocess of the system. Since the system is able to detect the resence or the absence of a jamming attack only when a slot ends, the achievable rate and the jammer s reaction to a given olicy are known only at the end of each slot. Therefore, the history of the system is udated at the beginning of each slot according to the reaction of the jammer to decisions taken in the revious slot. Note that the history of the system is udated only when a user is scheduled on a given channel. That is, if k K θ j =0for agivenj Hand n N,wehavethatπ j +1 = π j. Instead, when a user is scheduled on a given channel and j>1, the history of the system is udated according to the history udate dynamics in eq. 20. π j +1 m j, OPT j if α OPT j = 1 OPT j P = m j, M j if α OPT j = 1 20 OPT j =P OPT j, M j otherwise where is the logical AND oerator and we recall that π 1 = 0,P. For any π j, letδ π j = M j m j denote the measure of the interval [ m j,m j] generated by the history π j. Furthermore, let δ C π j, ˆδC π j, δ A π j and ˆδ A π j be defined as in eqs In Proosition 2, we illustrate how system s learning dynamics imact the choice of the otimal ower control olicy. Proosition 2: For each n N, k K and j, l H, let δ {δ C j = min C π j, ˆδ } C π j, and δ {δ A j =min A π j, ˆδ } A π j. We have that 1 if Δ j δ C j a conservative olicy is otimal, 2 if Δ j δ A j an aggressive olicy is otimal, and 3 if either conservative or aggressive olicies are otimal at slot j, these olicies will be otimal for any l>j. Proof: To rove the first art of this roosition, we look at the first order derivative, say r, of the function E α {R } w.r.t. the variable [ m j,m j]. For a conservative strategy to be otimal, i.e., OPT j = m j, it must hold that i r m j 0; and ii R m j R P. It can be easily shown that i holds if Δ π j δ C π j, and ii holds if Δ π j ˆδ C π j. Therefore, a conservative olicy for user n on channel k at slot j is otimal if Δ π j δ C j. On the contrary, an aggressive olicy is otimal if a r M j 0; or b E α j {R j} < R P. It is ossible to show that a holds if Δ π j δ A π j, while b holds if Δ π j ˆδ A π j. Thus, an aggressive olicy is otimal if Δ π j δ C j. Clearly, an exloratory olicy is otimal if neither conservative nor aggressive olicies are otimal. The third and last art of the roof is a direct consequence of the history udate mechanism in eq. 20. When OPT j is conservative OPT j = m j or aggressive OPT j = P, it follows that π j +1 = π j. Thus, the same olicy will be still otimal for all l>j, i.e., π j =π l = = π H. Proosition 2 gives us an imortant insight on the learning dynamics of the system. We have shown that π j = π j for all j > j if either conservative or aggressive olicies are chosen at slot j. That is, anytime that either conservative or aggressive olicies are otimal for a given user on a given channel, the learning rocess for that user on that considered channel is stoed. Although we roved that the otimal ower control olicy is always unique and we have shown the dynamics regulating the choice of the otimal olicy, how to obtain the value of OPT j still remains unsolved. A closed-form for OPT j, can be derived only by solving eq. 13, which is not ossible for our roblem. To find the solution of eq. 13, we exloit techniques from stochastic aroximation theory and exonential maings. For the sake of simlicity, in the following of this section we omit the subscrits n, k and the slot index j. Let us define R =E α {R}, andletr denote the first derivative of R w.r.t.. From eq. 13, r can be written as r = d R g = d σ 2 + g + 1 M [ m log 1+ hp J σ 2 log 1+ hp ] J + g σ 2 m M m ghp J σ 2 + gσ 2 + g + hp J 21 We also assume m =0and M =P. However, the more general case where 0 < m < M <P can be treated similarly. 5 The measure of the feasible ower level set is Δ = M m = P. Finally, in 22 we consider the discrete-time stochastic aroximation algorithm with exonential maings z[i +1]=z[i]+γ i r[i] [i +1]=Δ ez[i+1] 22 1+e z[i+1] where i is the iteration index and γ i is a variable ste-size [31]. It is worth remarking that, at any iteration of 22, the value 5 Note that we can define an auxiliary variable = m, [0, M m ]. Our results still hold as r= d R = d R + m d d =r.

8 8 IEEE/ACM TRANSACTIONS ON NETWORKING of [i] is always bounded in [0, Δ]. That is, the roosed algorithm in 22 always generates feasible transmission ower udates. 6 Let be the otimal solution of eq. 13. Now, we first derive the continuous-time version of 22. Then, in Proosition 3, we rove that 22 converges to for any ossible feasible initial condition. Let us define the continuous-time dynamics of 22 as ż = r =Δ ez 23 1+e z Proosition 3: Let the ste-size γ i be defined such that i γ2 i < i γ i = +. Then, for any feasible initial condition, the discrete-time algorithm 22 always converges to the otimal solution of eq. 13. Proof: Let t be a solution orbit of 23. Our roof consists in 1 showing that t converges to as t +, and 2 showing that 22 is an asymtotic seudo-trajectory APT [32] for 23 which converges to when some conditions on the ste-size are satisfied. From Proosition 1, we have that R is strictly concave, therefore r < 0 for all [0, Δ] by definition. Thus, we can exloit the latter result to build a Lyaunov-candidate-function for 23. It is easy to show that the function V defined as Δ V =Δlog + log Δ Δ Δ 24 is a strict Lyaunov function for 23. One can show that this statement is true by observing that V = dv /dt = r < 0, V = 0 and V > 0 for all. Furthermore, by decouling z and, we obtain z =log ; and by rewriting V in terms of z, we Δ obtain V z which can be roved to be radially unbounded. Therefore, we can assert that the otimal solution is also globally asymtotically stable GAS, which leads to the conclusion that t converges to as t +. Now,we rove the second art of the roosition. That is, we rove that also the discrete-time algorithm converges to the otimal solution as i +. By decouling 23, we get ṗ = d/dt = 1 Δ r 25 This result will be useful to show that the discrete-time algorithm tracks the continuous-time system u to a bounded error that asymtotically tends to 0 as i increases. A second-order Taylor exansion of 22 leads to [i +1]=[i]+γ i [i] 1 [i] Δ r[i] Mγ2 i 26 for some bounded M. Note that M is bounded as d dr is bounded by definition. Intuitively, eq. 26 is the discrete version of eq. 25 u to a bounded error. Since, by assumtion, i γ2 i < i γ i =+, results contained in [32] show that [n] is an APT for 23. It still remains to rove that [n]. By considering a Taylor exansion of V z, we obtain V z[i +1]=V z[i] + γ i [i] r[i] M γ 2 i 27 6 The same result also holds for the general case where [i] [ m, M ]. for some bounded M > 0. Recall that [i] r[i] < 0 by definition and is GAS. It follows that W =[0, Δ] is a basin of attraction for. Therefore, there must exist a comact set L Wcontaining. If we rove that there also exists a large enough i such that [i ] L, then, the roof is concluded. Assume ad absurdum that such i does not exists. Therefore, some β>0 must exist such that [i] r[i] β for all i. It follows that V z[i +1] V z[i] γ i β M γ 2 i 28 which, by telescoing, yields to V z[i +1] V z[0] β i γ i M i γ 2 i 29 Sinceweassumedthat i γ2 i < i γ i = +, the latter equation leads to V z[i +1], which is a contradiction as V z is lower bounded by construction. Therefore, [32] ensures that there must exist a large i such that [i ] Land lim i + [i] =, which concludes our roof. C. Otimal User Scheduling In this section, we define the finite-horizon DP-based algorithm to solve A. To roerly define the DP framework, we must take into account the uncertainty introduced by the jammer s behavior and its imact on the dynamics of the roblem. Thus, in the language of DP, we define: System state: we define the system state at slot j as the tule πj, ρj, whereπj =π j n,k denotes the history vector at slot j. At each slot, π j is udated according to eq. 20. On the other hand, ρj =ρ n j n denotes the residual erformance vector. Each ρ n j secifies the remaining amount of erformance that has to be allocated to user n from slot j to the horizon H to satisfy its minimum erformance request Rn. At each slot, ρ n j is udated as follows: ρ n j +1 Rn if j =1 = ρ n j { θ je α R OPT j } otherwise k K 30 Action: at each slot j, the actions of the scheduler are the otimal scheduling and ower control olicies, i.e., θj and j, resectively. For any scheduling olicy θ j θj, it must hold that θ j ={0, 1}. Instead, j contains the otimal ower control olicies chosen when the history of the system is πj. The generic element j j is trivially defined as j =0 if θ j =0and j = OPT j if θ j =1, where each OPT j is given in eq. 14; Single-Slot Reward: we define the function Φπj, ρj, θj, j,j as the reward, i.e., the total transmission rate, that the system achieves at slot

9 D ORO et al.: OPTIMAL POWER ALLOCATION AND SCHEDULING UNDER JAMMING ATTACKS 9 j when olicy θj, j is chosen and the state is ρj, πj. Therefore, Φπj, ρj, θj, j,j = θ jγ πj, j,j k K n N { where Γ πj, j,j = E α R OPT j } and OPT j j and it is calculated in eq. 14. Thus, the deendence of Γ πj, j,j from πj is imlicit in the definition of OPT j. To solve B, we write the Bellman s equation [33]: J πj, ρj,j = max θj,j Φπj, ρj, θj, j,j +E{J πj +1, ρj +1,j+1 πj, ρj} 31 H { s.t. θ je α R OPT i } ρ n j 32 i=j k K θ j 1, n N 33 k K θ j 1, k K 34 n N θ j {0, 1}, j [0,P], n n, k K 35 where we set Jπj, ρj,h+1 = 0 for all πj and ρj. At each slot, the Bellman s equation consists in a binary integer linear rogramming ILP roblem. ILP roblems are known to be NP-Comlete and their exact solution can be comuted through standard Branch-and-Bound methods. Finally, by using backwards induction [33], we solve the Bellman s equation and find the otimal solution to B. In Fig. 2, we show the building blocks of our roosed solution. Backward induction imlies that we start from slot j = H and go backwards in time. At each slot, we first solve the ower control roblem by exloiting the system state arameter πj. The solution of the ower control roblem consists in the ower control vector j. Tosolvethe scheduling roblem, the system requires j and ρj. The scheduler solves the single-slot reward maximization roblem and rovides the otimal scheduling olicy θj. Finally, each element in j is modified such that each j j is set to j =0if θ j =0and the otimal joint ower control and user scheduling olicy θj, j is found. V. APPROXIMATION OF THE OPTIMAL SOLUTION In the revious sections, we have shown that DP can solve our considered roblem otimally. However, it is well known that DP suffers from the curse of dimensionality [33]. That is, when the state sace and the number of variables increase, the number of ossible combinations that we need to solve considerably increases. As an examle, the ower control variables j are defined over the continuous set [0,P]. It follows that the number of ossible combinations of both the system state and the feasible actions is infinite. Furthermore, both the history πj and the residual erformance vector ρj deend on j, which contributes to further increase the dimension of the roblem. DP can theoretically still rovide an otimal solution to such continuous sace roblem. However, from a ractical oint of view, it is unrealistic to imlement the Bellman s equation on discrete-time systems. To avoid the curse of dimensionality, we show that an aroximation of the otimal solution can be obtained by discretizing the ower control variable. Secifically, we exloit state aggregation techniques [33] which allow to aggregate one or more saces of the original roblem to create several lower dimension abstract saces. Thus, we can aggregate saces by quantizing the ower control action sace to create a discretized version of it. In the following of this section, we show that by discretizing both the ower control variable and the residual erformance vector it is ossible to significantly reduce the comlexity of the roblem while guaranteeing user QoS requirements. A. State Aggregation Aroach ξ ξ be the Let ξ be the ower quantization ste. Without losing in generality, we assume that the ower quantization ste is chosen such that ξ is a divisor for P. Let and be the ceiling and floor oerators, resec- tively. For any [0,P],let= ξ ξ and = higher and lower quantized ower level values, resectively. In the discretized version of the roblem, we use the roosed quantization-based state aggregation to discretize j. That is, we calculate its quantized equivalents and.letˆ j be defined as follows: ˆ j { j if = = arg max ={, } E α {R } otherwise Thus, ˆ OPT j = arg max ={ˆ j,p } E α {R } is the solution to the discretized version of eq Let j and θj be the otimal ower control and scheduling olicy at slot j, resectively. Similarly to the continuous sace roblem, for any j we have =ˆ OPT j iff θ j =1.Otherwise,ˆ OPT j =0. At each slot, the history of the system π j is udated according to eq. 20. So far, we have discretized the state of ower control variable. However, from eqs. 5 and 30, it is clear that both the achievable rate and the residual erformance vector have a continuous state sace. Let ξ r be the erformance quantization ste. We assume that the network oerator forces each user to submit a minimum erformance requirement, Rn, such that the latter is an integer multile of ξ r. To overcome the high-dimensionality caused by the definition of the residual erformance vector, we modify 7 Note that even though ˆ j is otimal for the discretized version of eq. 13, it is sub-otimal for the continuous sace roblem, unless that j = ˆ j.

10 10 IEEE/ACM TRANSACTIONS ON NETWORKING the udate dynamic of ρj as follows: ρ n j +1 Rn if j =1 = ρn j k K θ E α {R } ξ r otherwise ξ r ξ r 36 where j. Let N and N r denote the maximum number of ower and transmission rate quantized levels, resectively. Trivially, N =P/ξ { +1 and N} r =R max /ξ r +1,whereR max = max R1,R 2,...,R N. Now, we aly the Bellman s equation to the discretized roblem and find its otimal solution. At each slot j, the number of ossible combinations of N ρj and πj are N r and NN 1 2 N K, resectively. Finding the maximum of the single-slot reward maximization roblem has comlexity O max{ K, N } min{ K, N }. Finally, the number of ossible combinations in eq. 31 is O K N H. Therefore, the overall comlexity of the Bellman s equation is O ω K N N K N 1 K r H N N 2 K where ω = max{ K, N } min{ K, N }. However, if K < N, by exloiting constraints 33 and 34 we can reduce the comlexity of the single-slot maximization N! roblem to O N K!. Thus, the comlexity becomes N N! N O K N N K! N K N 1 K r H.Inthesecial case where N = Olog H, the roosed algorithm 2 K has olynomial comlexity. That is, to solve the roblem in olynomial time, we should consider either a low number of users that scales as the logarithm of the horizon H, oralarge value of H. In all other cases, the comlexity of the algorithm exonentially increases with the number of users in the system. VI. PERFORMANCE-AWARE ONLINE GREEDY ALGORITHM POGA In revious sections, we have roosed both otimal and aroximated solutions to the joint ower control and scheduling roblem. Unfortunately, due to comlexity issues, those solutions may not find alication to those scenarios where the number of users and channels are large. Furthermore, solutions roosed in Sections IV and V require a riori knowledge of the users in the network. However, there are some scenarios where users dynamically access and leave the network, i.e., the number of users in the network vary in time. Under the above assumtions, to aly both our otimal or aroximated solutions is not ossible. Accordingly, to account for the above issues, in this section we focus on the design of a lowcomlexity online algorithm. Let N j be the set of users at slot j Hand K = K be the number of wireless channels. Accordingly, N U j = {n Nj :ρ n j =0} indicates the set of unsatisfied users whose minimum QoS requirement has not been satisfied yet. Consequently, N S j =N j \N U j reresents the set of satisfied users u to slot j. Algorithm 1 Performance-Aware Online Greedy Algorithm POGA Inut: System state πj, ρj, Satisfied users N S j, Unsatisfied users N U j. Outut: A greedy ower control and scheduling olicy θj, j. Calculate ower control olicy j from eq. 22 Find the greedy scheduling θ U j among unsatisfied users in N U j if There are unscheduled channels then Find the greedy scheduling θ S j among satisfied users in N S j end end θj θ U j θ S j j θj j Udate πj +1, ρj +1 return θj, j Since no information about future user arrival is available, and to reduce the comlexity of the roblem, in Algorithm 1 we roose a Performance-aware Online Greedy Algorithm POGA to schedule user transmissions. We exloit the decomosability of the roblem that we have introduced in Section IV-A. Secifically, we first calculate the ower control olicy j as in eq. 14 by exloiting the discrete-time stochastic aroximation algorithm with exonential maings that we have introduced in eq. 22. Then, to account for minimum QoS requirements, we give riority to unsatisfied users in N U j and we schedule their transmissions by finding the greedy scheduling olicy θ U j. If all channels have been assigned to unsatisfied users, i.e., n N U j k K θ j=k, wesetθj=θ S and j= θj j, where is the Hadamard roduct oerator. Otherwise, if some channels have not been allocated in θ U j, i.e., n N U j k K θ j <K, we exloit those unused channels to schedule transmissions of already satisfied users in N S j and calculate the greedy scheduling olicy θ S j. Thus, we set θj=θ U j θ S j and j=θj j, where is the logical OR oerator. Finally, we udate πj+1, ρj+1 according to eqs. 20 and 30 and POGA returns the sub-otimal scheduling solution θj, j. Let N = max j H {N j} be the maximum number of users in the system. It can be shown that the comlexity of the ower control algorithm is ONK [34]. By exloiting results derived in V-A, it can be shown that to find the greedy scheduling olicies θ S j and θ U j has comlexity N! O N K! = ON K 1. Therefore, the comlexity of executing Algorithm 1 for H slots is OHN K.Thatis,the overall comlexity of POGA is olynomial in the number N of users in the system. VII. NUMERICAL RESULTS In this section, we evaluate the achievable erformance of the solutions above roosed through extensive numerical simulations. Secifically, in Section VII-A we assess the erformance of the aroximated solution to the joint ower

11 D ORO et al.: OPTIMAL POWER ALLOCATION AND SCHEDULING UNDER JAMMING ATTACKS 11 Fig. 4. Toology of the simulated 1-dimensional scenario. TABLE II SIMULATION SETTING Fig. 6. Exected rate of the system as a function of the osition of the jammer x J for different QoS requirement settings and values of P J Solid lines: P J =0.5W; Dashed lines: P J =1W. Fig. 5. Learning rocess on a single channel. Dotted lines show the lowest transmission ower that triggers the jammer. control and user scheduling roblem. Instead, the achievable erformance of POGA and its comarison with other traditional scheduling olicies are discussed in Section VII-B. A. Performance Assessment of the Aroximated Solution We consider a wireless network consisting of N = 3 legitimate users. Unless otherwise stated, we consider K = 2 channels whose gain coefficients g, h and h k are generated according to the ath-loss model. As deicted in Fig. 4, we consider a 1-dimensional scenario where users i.e., U 1,U 2,U 3, the BS and the jammer i.e., J are located along the same axis. 8 Other relevant simulation arameters are reorted in Table II. Finally, unless otherwise stated, we set the maximum transmission ower for the jammer to P J =0.6W, which is a common setting of many commercial mobile jammers. We start by illustrating how the learning rocess on a single channel evolves over time. In Fig. 5, we show the evolution of the learning rocess, i.e., the calculation of OPT over time, as a function of P J. For all users, dotted lines reresents the lowest transmission ower that triggers the jammer. The exloratory olicy chosen by U 1 at j =2causes the triggering 8 This is just an illustrative examle. However, our aroach is indeendent of the actual wireless network toology. of the jammer. Thus, U 1 reduces its transmitting ower but it again triggers the jammer. This is because the triggering threshold for user U 1 dotted-starred lines is low. Hence, U 1 chooses an aggressive olicy even though it triggers the jammer. Similarly, the exloratory olicy of U 2 at j =2does trigger the jammer. Therefore, U 2 reduces its transmission ower and exlores a new transmission ower olicy. The new exlored olicy is found to not trigger the jammer. Accordingly, U 2 evaluates eq. 14 and realizes that no further imrovements are needed, a conservative olicy is chosen and the learning rocess for U 2 is stoed. On the other hand, U 3 is far away from the jammer. Therefore, as shown in Fig. 5, the triggering threshold for this user is high. As a consequence, the first exloratory olicy does not trigger the jammer. However, the risk of triggering the jammer is high. It follows that eq. 14 suggests to choose a conservative olicy. To investigate the imact of the osition x J of the jammer on the achievable erformance of the network, in Fig. 6 we evaluate the exected transmission rate as a function of x J under two different minimum QoS requirements. Secifically, we consider the case where all users request an identical minimum QoS level Rn =10.93 Kbit/s Case A, and the case where no requirements are submitted by users Case B. Fig. 6 shows that when the jammer is at the border of the considered scenario, its attacks have limited effect on network erformance. Instead, when the jammer aroaches the BS and the users, the achievable rate of the network considerably decreases. Also, when no QoS constraints are considered and rate maximization is the only objective of the network oerator, system erformance are higher than those achieved in Case A. In Figs. 7 and 8 we comare the erformance of the system when different scheduling olicies are considered. Secifically, we comare the roosed aroximated solution to random, round-robin and greedy scheduling olicies without minimum QoS requirements. Note that the greedy scheduling olicy without QoS requirements we consider in Figs. 7 and 8 is different from POGA. POGA is erformance-aware and aims at satisfying user QoS requirements. On the contrary, a greedy scheduling olicy without QoS requirements only schedules those users whose transmission rate is the highest.

12 12 IEEE/ACM TRANSACTIONS ON NETWORKING Fig. 7. Exected rate of the system under different scheduling olicies as a function of P J. Fig. 9. Per-user exected rate of the system under POGA as a function of the number N of users Solid lines: K =4; Dashed lines: K =8. Fig. 10. Average er-user normalized residual erformance under POGA as a function of the number N of users Solid lines: K =4; Dashed lines: K =8. Fig. 8. Average er-user normalized residual erformance at the end of the joint ower control and user scheduling cycle as a function of P J. A comarison between POGA and a greedy scheduling olicy without QoS requirements is in Section VII-B. Fig. 7 shows that our roosed solution reaches high transmission rates and it outerforms random and round robin olicies. The achievable erformance of the network under the roosed aroximated solution and the greedy without QoS requirements are comarable. However, in Fig. 8 we lot the er-user normalized residual erformance variable ρ at the horizon H defined as ρ = 1 ρ n H N Rn 37 n N ρ reresents the QoS-ga, i.e., the er-user amount of erformance that has not been rovided to users at the end of the scheduling cycle. Thus, the desirable value is ρ = 0, while ρ > 0 indicates infeasible solutions. Our solution is the only one that guarantees ρ =0, i.e., all minimum QoS requirements are satisfied. Therefore, even though a greedy aroach without QoS requirements allows to achieve high erformance, it does not guarantee minimum QoS levels. In the considered simulation setting, Fig. 8 also shows that ρ for the greedy aroach without QoS requirements is constant and high. On the contrary, the value of ρ under random and round robin olicies is low for small values of the transmission ower of the jammer, but it increases when the value of P J increases as well. B. Performance Assessment of POGA In this section, we assess the erformance of the online algorithm POGA. The results being resented are averaged over 200 simulations. At each simulation run, the ositions of the users and the jammer are randomly generated within a square area of edge L = 200 according to a uniform distribution. In Fig. 9, we assume H =5and we show the er-user exected rate of the system under POGA as a function of the number N of users when K =4solid lines and K =8 dashed lines. POGA first aims at satisfying all users. Then, it greedily maximizes the rate of the system by scheduling the users that have been already satisfied. Therefore, as N increases, the er-user exected rate under POGA decreases. Also, it is worth noting that the er-user exected rate when K =8aroaches that of K =4as the number of users increases as well. Such a result stems from the fact that when the available channels are shared by a large number of users, the imrovement rovided by increasing the number K of channels reduces. It is worth noting that by increasing the number K of channels it is ossible to considerably imrove the rate of the system. Instead, worse erformance are achieved when the jammer attacks with higher transmission ower P J. In Fig. 10, we show the average er-user normalized residual erformance variable defined in eq. 37 under POGA as a function of the number N of users when K =4solid lines and K =8dashed lines. It is shown that, as N increases, the average er-user normalized residual erformance increases

13 D ORO et al.: OPTIMAL POWER ALLOCATION AND SCHEDULING UNDER JAMMING ATTACKS 13 Fig. 11. Per-user exected rate of the system under POGA as a function of the horizon duration H Solid lines: K =4; Dashed lines: K =8. Fig. 14. Average er-user normalized residual erformance at the end of the joint ower control and user scheduling cycle as a function of P J. Fig. 12. Average er-user normalized residual erformance under POGA as a function of the horizon duration H Solid lines: K =4; Dashed lines: K =8. to schedule and accommodate more users, which eventually results in better erformance. Finally, in Figs. 13 and 14 we comare POGA to other scheduling olicies. We assume an horizon duration of H = 10 and we consider N = 20 users.similarlytowhathas been shown in Figs. 7 and 8, a greedy olicy without QoS requirements achieves the highest transmission rate. However, it oorly erforms in terms of guaranteeing a minimum QoS level to users. Instead, Fig. 13 shows that POGA outerforms random and round-robin olicies in terms of er-user exected rate. And, as illustrated in Fig. 14, POGA achieves the lowest er-user normalized residual erformance. That is, among the considered scheduling olicies, POGA is the best suited to achieve high transmission rate while guaranteeing user QoS requirements. Furthermore, both Figs. 13 and 14 show that by increasing the number K of channels, all scheduling olicies achieve higher erformance. VIII. CONCLUSIONS Fig. 13. Exected rate of the system under different scheduling olicies as a function of P J. as well. Such a result is due to the fact that it is hard to accommodate all minimum QoS requirements when a large number of users is under attack. It is worth noting that, by increasing the number of channels, it is ossible to imrove the erformance of POGA and reduce the average er-user normalized residual erformance. As exected, smaller values of P J result in better erformance of the system. To investigate the imact of the horizon duration H on the achievable erformance of the system, in Figs. 11 and 12 we consider N =20users and we show the er-user exected rate of the system and the average er-user normalized residual erformance, resectively. Fig. 11 shows that higher rates are achieved when the horizon duration is large. Analogously, Fig. 12 illustrates how the average er-user normalized residual erformance variable decreases, and asymtotically tends to zero, as H increases. That is, large values of the horizon duration H allow In this aer, we studied the joint ower control and scheduling roblem in jammed networks under minimum QoS constraints. By assuming that no information on jammer s behavior and osition is available, we roved that the roblem is NP-Hard. However, we showed that it can be decomosed and modeled as a dynamic roblem whose comlexity is further reduced by exloiting state aggregation techniques. Learning through observation of jammer s reactions to transmission decisions taken in the ast has been exloited to identify otimal transmission olicies at future slots and maximize network erformance. We also roosed and discussed a low-comlexity Performance-aware Online Greedy Algorithm POGA. Simulation comarison showed that the aroaches roosed in this aer outerform other traditional scheduling olicies. REFERENCES [1] M. Wilhelm, I. Martinovic, J. B. Schmitt, and V. Lenders, Short aer: Reactive jamming in wireless networks: How realistic is the threat? in Proc. 4th ACM Conf. Wireless Netw. Secur., 2011, [2] M. Pajic and R. Mangharam, Satio-temoral techniques for antijamming in embedded wireless networks, EURASIP J. Wireless Commun. Netw., vol. 2010, Ar. 2010, Art. no. 51.

14 This article has been acceted for inclusion in a future issue of this journal. Content is final as resented, with the excetion of agination. 14 [3] A. D. Wood, J. A. Staovic, and G. Zhou, DEEJAM: Defeating energy-efficient jamming in IEEE based wireless networks, in Proc. 4th Annu. IEEE Commun. Soc. Conf. Sensor, Mesh Ad Hoc Commun. Netw. SECON, Jun. 2007, [4] D. Nguyen, C. Sahin, B. Shishkin, N. Kandasamy, and K. R. Dandekar, A real-time and rotocol-aware reactive jamming framework built on software-defined radios, in Proc. ACM Worksho Softw. Radio Imlement. Forum, New York, NY, USA, 2014, [5] E. Bayraktaroglu et al., On the erformance of IEEE under jamming, in Proc. IEEE INFOCOM, Ar. 2008, [6] A. Marttinen, A. M. Wyglinski, and R. Jäntti, Statistics-based jamming detection algorithm for jamming attacks against tactical MANETs, in Proc. IEEE Military Commun. Conf. MILCOM, Oct. 2014, [7] M. Strasser, B. Danev, and S. C akun, Detection of reactive jamming in sensor networks, ACM Trans. Sensor Netw., vol. 7, no. 2, Se. 2010, Art. no. 16. [8] W. Xu, K. Ma, W. Trae, and Y. Zhang, Jamming sensor networks: Attack and defense strategies, IEEE Netw., vol. 20, no. 3, , May/Jun [9] R. A. Poisel, Modern Communications Jamming Princiles and Techniques. Norwood, MA, USA: Artech House, [10] K. Grover, A. Lim, and Q. Yang, Jamming and anti jamming techniques in wireless networks: A survey, Int. J. Ad Hoc Ubiquitous Comut., vol. 17, no. 4, , [11] Z. Lu, W. Wang, and C. Wang, Modeling, evaluation and detection of jamming attacks in time-critical wireless alications, IEEE Trans. Mobile Comut., vol. 13, no. 8, , Aug [12] D. Giustiniano, V. Lenders, J. B. Schmitt, M. Suhler, and M. Wilhelm, Detection of reactive jamming in DSSS-based wireless networks, in Proc. 6th ACM Conf. Secur. Privacy Wireless Mobile Netw., 2013, [13] J. Dams, M. Hoefer, and T. Kesselheim, Jamming-resistant learning in wireless networks, in Automata, Languages, and Programming Comuter Science, vol Berlin, Germany: Sringer, 2014, [14] X. Liu, E. K. P. Chong, and N. B. Shroff, Oortunistic transmission scheduling with resource-sharing constraints in wireless networks, IEEE J. Sel. Areas Commun., vol. 19, no. 10, , Oct [15] X. Liu, E. K. P. Chong, and N. B. Shroff, A framework for oortunistic scheduling in wireless networks, Comut. Netw., vol. 41, no. 4, [16] C. Poer, M. Strasser, and S. Cakun, Anti-jamming broadcast communication using uncoordinated sread sectrum techniques, IEEE J. Sel. Areas Commun.,, vol. 28, no. 5, , Jun [17] E.-K. Lee, S. Y. Oh, and M. Gerla, Randomized channel hoing scheme for anti-jamming communication, in Proc. IFIP Wireless Days WD, Oct. 2010, [18] B. DeBruhl, C. Kroer, A. Datta, T. Sandholm, and P. Tague, Power naing with loud neighbors: Otimal energy-constrained jamming and anti-jamming, in Proc. ACM Conf. Secur. Privacy Wireless Mobile Netw., 2014, [19] S. Khattab, D. Mosse, and R. Melhem, Jamming mitigation in multiradio wireless networks: Reactive or roactive? in Proc. 4th Int. Conf. Secur. Privacy Commun. Netw., 2008, Art. no. 27. [20] I. Shin, Y. Shen, Y. Xuan, M. T. Thai, and T. Znati, A novel aroach against reactive jamming attacks, Ad Hoc Sensor Wireless Netw., vol. 12, nos. 1 2, , [21] Y. Liu and P. Ning, BitTrickle: Defending against broadband and highower reactive jamming attacks, in Proc. IEEE INFOCOM, Mar. 2012, [22] Y. Xuan, Y. Shen, N. Nguyen, and M. Thai, A trigger identification service for defending reactive jammers in WSN, IEEE Trans. Mobile Comut., vol. 11, no. 5, , May [23] I. Shin, Y. Shen, Y. Xuan, M. T. Thai, and T. Znati, Reactive jamming attacks in multi-radio wireless sensor networks: An efficient mitigating measure by identifying trigger nodes, in Proc. 2nd ACM Int. Worksho Found. Wireless Ad Hoc Sensor Netw. Comut., 2009, [24] W. Xu, W. Trae, and Y. Zhang, Anti-jamming timing channels for wireless networks, in Proc. 1st ACM Conf. Wireless Netw. Secur., New York, NY, USA, 2008, [25] R. D. Pietro and G. Oligeri, Silence is golden: Exloiting jamming and radio silence to communicate, ACM Trans. Inf. Syst. Secur., vol. 17, no. 3, , Mar [26] S. D Oro et al., Defeating jamming with the ower of silence: A game-theoretic analysis, IEEE Trans. Wireless Commun., vol. 14, no. 5, , May IEEE/ACM TRANSACTIONS ON NETWORKING [27] K. Xu, Q. Wang, and K. Ren, Joint UFH and ower control for effective wireless anti-jamming communication, in Proc. IEEE INFOCOM, Mar. 2012, [28] W. Xu, On adjusting ower to defend wireless networks from jamming, in Proc. 4th Annu. Int. Conf. Mobile Ubiquitous Syst., Netw. Services, MobiQuitous, Aug. 2007, [29] M. R. Garey and D. S. Johnson, Comuters and Intractability A Guide to the Theory of NP-Comleteness. San Francisco, CA, USA: Freeman, [30] J. G. March, Exloration and exloitation in organizational learning, Org. Sci., vol. 2, no. 1, , [31] H. J. Kushner and G. G. Yin, Stochastic Aroximation Algorithms and Alications Alications of Mathematics, vol. 35. New York, NY, USA: Sringer, [32] M. Benaïm, Dynamics of stochastic aroximation algorithms, in Seminaire de Probabilites XXXIII Mathematics, vol Berlin, Germany: Sringer, [33] D. P. Bertsekas, Dynamic Programming and Otimal Control. Belmont, MA, USA: Athena Scientific, [34] S. D Oro, P. Mertikooulos, A. L. Moustakas, and S. Palazzo, Interference-based ricing for oortunistic multicarrier cognitive radio systems, IEEE Trans. Wireless Commun., vol. 14, no. 12, , Dec Salvatore D Oro S 12 received the B.S. degree in comuter engineering, the M.S. degree in telecommunications engineering, and the Ph.D. degree from the University of Catania in 2011, 2012, and 2015, resectively. From 2013 to 2015, he was a Visiting Researcher with Université Paris-Sud 11, Paris, France, and The Ohio State University, Columbus, OH, USA. He is currently a Post-Doctoral Research Fellow with the University of Catania. In 2015, he organized the First Worksho on COmetitive and COoerative Aroaches for 5G networks COCOA at Euroean Wireless He served on the Technical Program Committee TPC of the CoCoNet8 worksho at the IEEE ICC In 2013, he served on the TPC of the 20th Euroean Wireless Conference EW2014. Eylem Ekici S 99 M 02 SM 11 received the B.S. and M.S. degrees in comuter engineering from Bog aziçi University, Istanbul, Turkey, in 1997 and 1998, resectively, and the Ph.D. degree in electrical and comuter engineering from the Georgia Institute of Technology, Atlanta, GA, USA, in He is currently a Professor with the Deartment of Electrical and Comuter Engineering, The Ohio State University. His current research interests include cognitive radio networks, vehicular communication systems, and next-generation wireless systems, with a focus on algorithm design, medium access control rotocols, resource management, and analysis of network architectures and rotocols. He is the TPC Co-Chair of the IEEE INFOCOM He is an Associate Editor of the IEEE T RANSACTIONS ON M OBILE C OMPUTING and Comuter Networks. He is a former Associate Editor of the IEEE/ACM T RANSACTIONS ON N ETWORKING. Sergio Palazzo M 92 SM 99 received the degree in electrical engineering from the University of Catania, Catania, Italy, in Since 1987, he has been with the University of Catania, where is currently a Professor of telecommunications networks. His current research interests include mobile systems, wireless and satellite IP networks, and rotocols for the next generation of the Internet. He is a member of the MobiHoc Steering Committee. He has also been a TPC Co-Chair of some other conferences, including IFIP Networking 2011, IWCMC 2013, and Euroean Wireless He has been the General Chair of some ACM conferences, including MobiHoc 2006 and MobiO He currently serves on the Editorial Board of Ad Hoc Networks. He was an Editor of the IEEE W IRELESS C OMMUNICATIONS M AGAZINE, the IEEE/ACM T RANSACTIONS ON N ETWORKING, the IEEE T RANSACTIONS ON M OBILE C OMPUTING, Comuter Networks, and Wireless Communications and Mobile Comuting.