Optimizing Media Access Strategy for Competing Cognitive Radio Networks Y. Gwon, S. Dastangoo, H. T. Kung

Transcription

1 Optimizing Media Access Strategy for Competing Cognitive Radio Networks Y. Gwon, S. Dastangoo, H. T. Kung December 12, 2013 Presented at IEEE GLOBECOM 2013, Atlanta, GA

2 Outline Introduction Competing Cognitive Radio Network (CCRN) Basic definitions Spectrum model Channel activity matrix, outcome, reward CCRN Multi-armed Bandit (MAB) formulation Existing solutions Our approach Numerical evaluation Conclusion

3 Introduction Mobile networks under competition Tactical setting Ally vs. Enemy Dynamic, open spectrum resource Nodes are cognitive radios Comm nodes & jammers Opportunistic channel access Strategic use of jamming Ally Radio Network Multi-channel open spectrum Intra-network cooperation Jam Jam Jam Collision Network-wide competition Enemy Radio Network

4 Competing Cognitive Radio Network (CCRN) Ally vs. Enemy networks compete Each CCRN has C comm nodes and J jammers CCRN actions a A and a E are composite a A = {a A,comm, a A,jam }, a E = {a E,comm, a E,jam } Actions determine outcome a A a E Ω Outcome maps to CCRN reward R: f(ω) R CCRN strategy Determines new action given sensing results x, set of previous actions and outcomes : x t,{a 1,...,at 1 },{Ω 1,...,Ωt 1 } a t σ t opt = arg min σ Γ t =min{e[ σ M i=1 t r j (i) ] E Rσ t } j=1

5 Spectrum Model Partitioned in time and frequency N non-overlapping channels under competition Tx opportunity <f i, B i,t,t> Each Tx opportunity used as Control or data Frequency T N channels f i B i t Time

6 Channel Activity Matrix, Outcome, Reward (1/2) Example: consider Ally and Enemy CCRNs, each with 2 comm nodes and 2 jammers Ally uses channel 10 for control, Enemy channel 1 At time t, actions are the following a A t = {a A,comm = [7 3], a A,jam = [1 5]} a A,comm = [7 3] means Ally comm node 1 transmits at channel 7, and comm node 2 at channel 3 a E t = {a E,comm = [3 5], a E,jam = [10 9]} How to compute CCRN rewards? Channel Activity Matrix

7 Channel Activity Matrix, Outcome, Reward (2/2) Blue Force Red Force Reward Outcome Comm Jammer Comm Jammer BF RF Tx BF comm Tx success +1 0 Jam Tx BF jamming success +1 0 Tx Tx BF & RF comms collide 0 0 Jam Jam BF & RF jammers both jam 0 0 Tx Jam BF misjamming 0 0 Tx Jam RF jamming success 0 +1

8 CCRN Multi-armed Bandit (MAB) Formulation MAB Problems Originated by Thompson 1933 Address sequential reward sampling and resource allocation Find many applications in WWW today Goal: find optimal strategy to maximize Cumulative reward R t = (j=1 to t) r(j) over t Lai & Robbins 1985 Introduced concept of regret Γ Minimizing Γ is equivalent to maximizing R, but mathematically more convenient Provided qualification for optimal strategy Optimal action should be chosen exponentially more often than others asymptotically Proposed asymptotically optimal algorithm (L&R 85) for stochastic MAB problem Comm Node Jammer?? Radio channels

9 Existing Solution: L&R 85 Appeared in Lai & Robbins 1985 as asymptotically optimal allocation rules Keeps track of reward history for all accessed channels Compute two candidate channels Candidate 1: channel with highest mean reward This is point estimation Candidate 2: round robin selection This is different from random selection Compute Kullback-Leibler (KL) divergence between two candidates KL divergence serves test statistic to choose between two candidate channels Test if KL divergence is approaching in log t

10 Existing Solution: UCB UCB Upper Confidence Bound L&R 85 is nontrivial and difficult Due to complexity in computing KL divergence empirically UCB maintains easily computable indexes to estimate each channel s reward Index for i th channel: g i = i + Δ i i average reward for i th channel Δ i sqrt(log t/t i ) t = current time index, reflecting how long CCRN has operated T i = total number of times accessing i th channel so far Choose channel with highest index value

11 Existing Solution: Thompson Sampling Old probability matching heuristic Revamped in contemporary machine learning literature Full proof remains open problem in theoretical computer science Algorithmically very simple Draw i th channel reward r i θ i Best channel i * = arg max i r i Observe actual reward and update θ i Thompson Sampling is Bayesian!

12 Our Solution Key idea: Thompson Sampling + Extreme Value Theory Set up Bayesian conjugate prior on extreme value distribution Why? We want to act on best M out of total N channels Ranked channel rewards r (1) > r (2) >... > r (M) > r (M+1) >... > r (N) Distribution of best M channel rewards (blue) follows Extreme Value Theory There are only three possible extreme value distributions: Frechet, Gumbel, and Weibull Reward = total bits successfully transmitted or jammed Our reward must be finite Weibull is the only finite-ended EVT distribution Given Weibull likelihood, Bayesian inference suggests Inverse-gamma prior

13 Recapitulation: Algorithmic Comparison Our algorithm works Draw θ i Inverse-gamma(a i, b i ) Compute r i = Weibull(θ i ) Access M channels with largest r i s Observe actual channel rewards and update a i, b i Algorithm Class Computational Method Complexity L&R Deterministic Point Estimation High UCB Deterministic Indexing Low Thompson Sampling Randomized Heuristic Medium Proposed Randomized Bayesian Medium

14 Numerical Evaluation Wrote custom simulator in MATLAB 10-channel spectrum (N = 10) Two CCRNs: Ally vs. Enemy Each CCRN has fixed J = 2 jammers, varying C = 2,4,8 comm nodes At each time slot, all comm nodes transmit with probability 0.5 and jammers jam with probability 1 Simulation duration = 1,000 time slots Algorithmic matchups 1. L&R 85 (Ally) vs. Static (Enemy) 2. UCB (Ally) vs. Static (Enemy) 3. Thompson Sampling (Ally) vs. Static (Enemy) 4. Our algorithm (Ally) vs. Static (Enemy) 5. L&R 85 (Ally) vs. Random (Enemy) 6. UCB (Ally) vs. Random (Enemy) 7. Thompson Sampling (Ally) vs. Random (Enemy) 8. Our algorithm (Ally) vs. Random (Enemy) Tested scenarios 1. All algorithmic matchups in centralized control 2. All algorithmic matchups in distributed control

15 Result under Centralized Control Against Static Against Random Average reward per channel Static L&R-M UCB-M Z Our algorithm Average reward per channel Random L&R-M UCB-M Z Our algorithm Number of comm nodes Number of comm nodes

16 Result under Distributed Control Against Static Against Random Average reward per channel Static L&R-M UCB-M Z Our algorithm Average reward per channel Random L&R-M UCB-M Z Our algorithm Number of comm nodes Number of comm nodes

17 Conclusion Competing Cognitive Radio Networks Strategize combined comm and jamming node actions to cope with hostile opponents Look for optimal strategy under MAB framework Our MAB-based solution is state-agnostic Overcomes traditional Markovian formulation plagued by computational complexity in tracking states Formulated MAB framework for CCRN and proposed new algorithm that outperforms existing algorithms From Lai & Robbins s asymptotically optimal rules to UCB indexing and Thompson Sampling heuristic We re extending current study for both CCRNs using MAB algorithms