COMPRESSIVE SENSING IN WIRELESS COMMUNICATIONS

Transcription

1 COMPRESSIVE SENSING IN WIRELESS COMMUNICATIONS A Dissertation Presented to the Faculty of the Electrical and Computer Engineering Department University of Houston in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in Electrical Engineering by Jia Meng December 2010

2 c Copyright by Jia Meng 2010 All Rights Reserved

3 COMPRESSIVE SENSING IN WIRELESS COMMUNICATIONS Jia Meng Approved: Chair of the Committee Dr. Zhu Han, Assistant Professor Electrical and Computer Engineering Committee Members: Dr. Richard Liu, Professor Electrical and Computer Engineering Dr. Haluk Ogmen, Professor Electrical and Computer Engineering Dr. Lijun Qian, Associate Professor Electrical and Computer Engineering Prairie View A&M University Dr. Wotao Yin, Assistant Professor Computational and Applied Mathematics Rice University Dr. Rong Zheng, Associate Professor Electrical and Computer Engineering Dr. Suresh K. Khator, Associate Dean, Cullen College of Engineering Dr. Badrinath Roysam, Professor and Chairman, Electrical and Computer Engineering

4 Acknowledgements Foremost, I would like to express my sincere gratitude to my advisor, Dr. Zhu Han, for his wise guidance, constant encouragement, and the continuous support of my PhD study and research. With his profound knowledge in the area of wireless communication and digital signal processing, he offered a great many instructive advice and constructive suggestions on my research work. Besides his energetic and productive work style, his humorous and generous disposition affected me deeply. I also want to thank him for his generous financial support that gave me several opportunities to travel to conferences. I must thank Dr. Wotao Yin for his titanic efforts in all stages of my research and scientific writing. Those informative and inspiring conversations and the invaluable help enhanced our work dramatically. From him, I learned not only numerous advanced and useful mathematical techniques, but also the crucial aspects of being a good researcher, namely, meticulousness, passion, intelligence, and hard work. I have to thank Dr. Richard Liu for his instructive advice and constructive suggestions on geoscience and well logging related areas. Those weekly individual meetings with him were inspiring and steered my research efforts to attack interesting and potentially solvable problems. I would also like to thank my thesis committee, Dr. Haluk Ogmen, Dr. Lijun Qian, and Dr. Rong Zheng, for their encouragement, insightful comments, valuable discussions, and accessibility. I am also deeply indebted to all my collaborators, Dr. Husheng Li, Dr. Javad Ahmadi- Shokouh, Dr. E. Joe Charlson, Dr. Sima Noghanian, Dr. Ekram Hossain, and Dr. Yinying Li. I am grateful for my labmates kindness and friendliness. I appreciate all the helpful discussions that I had with them over the years. My appreciation also goes to all the staff of the ECE Department. Special thanks go to Florence Flores, Adolph Flores and the entire Flores family. Their v

5 family-like love, support and encouragement are precious to me. Finally, and most importantly, I would like to express the most heartfelt acknowledgment to my family in China. Even thousands of miles away, they have been present through every step of my life, providing support in difficult times. They have been a constant source of inspiration, and this thesis is dedicated to them. vi

6 COMPRESSIVE SENSING IN WIRELESS COMMUNICATIONS An Abstract of a Dissertation Presented to the Faculty of the Electrical and Computer Engineering Department University of Houston In Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in Electrical Engineering by Jia Meng December 2010

7 Abstract Nowadays, with the increasing demand of higher resolution and increasing number of modalities, the traditional signal/image processing hardware and software are facing significant challenges, since the Nyquist rate, which is part of the dogma for signal acquisition and processing, may become too high in many applications. How to acquire, store, fuse and process these data efficiently becomes a critical problem. The most popular solution to this problem is to compress after sensing densely. However, this oversampling and then discarding procedure leads to a waste of energy and recourses. A new paradigm of signal acquisition and processing, named compressive sensing (CS), was developed around Starting with the publication of Compressed sensing by D. Donoho, and several important publications by E. J. Candès, J. Romberg, and T. Tao, the CS theory, which links data acquisition, compression, dimensionality reduction, and optimization, has attracted much research attention. The CS theory consists of three key components, namely, signal sparsity, incoherent measurement matrix, and signal recovery. It claims that, as long as the signal to be measured is sparse or can become sparse after some known transformation, the information in the signal can be preserved in a small number of incoherent measurements, and convex optimization offers overwhelming signal recovery probability. The bulk of the dissertation explores the CS framework and proposes several implementations in wireless networks. Specifically, we first proposed to apply compressive sensing for collaborative spectrum sensing in cognitive radio networks to reduce the amount of sensing and transmission overhead. This was realized by innovatively equipping each cognitive radio node with an on-board frequency-selective filter set, through which the sensing information was blended incoherently and can be decoded at the fusion center via joint sparsity recovery or the matrix completion technique. viii

8 Then, we designed a high resolution OFDM channel estimation with low-speed ADC using compressive sensing system. Aiming at increasing the channel estimation resolution without increasing the costly ADC speed, we formed a random convolution sensing scheme by carefully arranging the pilot tones and taking advantage of the channel signal convolution nature. Moreover, based on the observation that the received signals are sparse in the time domain due to the limited multipath effects at 60 GHz UWB wireless transmission, we designed a CS-based low-speed ADC to reduce the sampling rate, while still being able to reconstruct the signal with high fidelity. Finally, we implemented the CS framework for sparse events detection in wireless sensor networks, in which the sensor activation events are sparse. We proposed the use of a small number of monitoring tubes to take a very limited number of incoherent measurements, which are then decoded through the Bayesian framework with a heuristic algorithm to enhance the detection probability. ix

9 Table of Contents Acknowledgements v Abstract viii Table of Contents ix List of Figures xiv List of Figures xv List of Algorithms xvii 1 Compressive Sensing Framework Contributions of This Dissertation History and Challenges Compressive Sensing: A Theoretical Breakthrough Compressible Signals Compressed Measurement Stable Measurement Matrix Signal Reconstruction Compressive Sensing in Matrix Form: An Overview Distributed Compressive Sensing via Joint Sparsity Recovery Matrix Completion x

10 1.5 Organization of This Dissertation Collaborative Spectrum Sensing in Cognitive Radio Networks Introduction System Model CSS Matrix Completion Algorithm Nuclear Norm Min. via Fixed Point Iterative Algorithm Approximate SVD Based Fixed Point Iterative Algorithm Stopping Criterion for Iterations Channel Availability Estimation Based on the Complete Measurement Matrix CSS Joint Sparsity Recovery Algorithm Discussion Complexity Comparisons between the Two Approaches Frequency-Selective Filter Design and Adaptive Sensing Dynamic CS Update Simulation Results Simulation of Matrix Completion Recovery Joint Sparsity Recovery Simulation Comparison between Matrix Completion Algorithm and Joint Sparsity Recovery Algorithm Conclusions High Resolution OFDM Channel Estimation with Low-speed ADC 49 xi

11 3.1 Introduction OFDM System Model Compressive Sensing OFDM Channel Estimation Motivations CS Background Pilot with Random Phases Numerical Evidence of Effective Random Convolution Relationship to Existing Results Numerical Algorithm Problem Formulation Algorithm Complexity Analysis Cramér-Rao Lower Bound Numerical Simulations Simulation Settings MSE Performance CRLB Performance Multipath Delay Detection Performance Conclusions Sampling Rate Reduction for 60 GHz UWB Communication Introduction System Model xii

12 4.3 Compressive Sensing-Based ADC Proposed ADC Formulation of the Compressive Sensing Problem Bayesian Detection BER Analysis for RAKE Receiver Simulation Results and Analysis Conclusions Sparse Events Detection in Wireless Sensor Networks Introduction System Model Problem Formulation and Analysis Bayesian Detection Model Specification Marginal Likelihood Maximization Heuristic using Prior Information Simulation Results and Analysis Conclusions Conclusions and Future Work Summary and Conclusions Future Work on Wireless Communications Dynamic Compressive Sensing for Wireless Communication Applications. 99 xiii

13 6.2.2 Joint Sparsity Recovery Algorithm for MIMO Wireless Communication Applications Other Future Work Compressive Sensing for Seismic Data Acquisitions and Processing Compressive Sensing for Concrete Flaw Detection Compressive Sensing for Pipe Line Leakage Detection Compressive Sensing for Human Vision Model Compressive Sensing for Gene Regulatory Networks Multi-spectral Infrared Imaging for Offshore Oil Spill Detection using Compressive Sensing Bibliography 108 xiv

14 List of Figures 1.1 Compressive sensing illustrator Compare different measurement matrix Traditional sensing vs. Compressive sensing l 1 norm minimization offers unique and sparse solution Joint sparsity illustrator Matrix completion illustrator System model False alarm and missing probability vs. sampling rate POD vs. sampling rate Noiseless AWGN channel (No. of CR = 5) Noiseless Rayleigh fading channel (No. of CR = 5) Noiseless log-normal shadowing channel (No. of CR = 5) Noiseless AWGN channel (No. of CR = 10) Noiseless Rayleigh fading channel (No. of CR = 10) Noiseless log-normal shadowing channel (No. of CR = 10) POD, FAR, and MDR performance vs. sampling rate at different SNR POD, FAR, and MDR performance vs. noise level for different number of PR Baseband OFDM System MSE vs. No. of multipath for different cases (SNR = 30 db) xv

15 3.3 Multipath delay profile MSE performance Reconstructed SNR CS recovered channel variance vs. CRLB Probability of support detection Probability of false support detection Compressive sensing-based ADC Simulated environment in ray-tracing A simulated power delay profile for 60 GHz UWB transmission Example of data flow on the hardware Original signal vs. CS recovered signal Covariance between the original signal and reconstructed signal BER performance under RAKE reception System model for wireless sensor network Performances comparatione (a) Proposed Scheme Performances comparatione (b) l 1 Magic Scheme Illustration of (a) Correct detection Illustration of (b) Incorrect detection Heuristic improvement Noise effect Example of MIMO Received Signal Joint Sparsity Structure xvi

16 6.2 Upper: Seismic image from 50% uniformly missing shot positions; Lower: Seismic image from 50% random shot and receiver Pipe line leak and remote laser leak detection system xvii

17 List of Algorithms 2.1 Joint detection algorithm CS-OFDM Heuristic algorithm using prior information xviii

18 Chapter 1 Compressive Sensing Framework 1.1 Contributions of This Dissertation Despite the relatively short history of compressive sensing theory, the number of researches and publications in this area is amazingly huge. However, the applications of CS theory are far from being mature, and mainly concentrated on image and video acquisition and processing. We discovered the inborn nature of the signal sparsity in the wireless communication areas, e.g., the sparse channel impulse response in the time domain; the sparse unitization of the spectrum; and the time and spatial sparsity in the wireless sensor networks. For each of the sparse signal, we designed innovative signal acquisition scheme which not only satisfies the incoherent requirement by the CS theory, but also can be realized on hardware. We further provided proper signal recovery algorithm for each system, which guarantees stable signal recovery with overwhelming probability. 1.2 History and Challenges Nyquist/Shannon sampling theory has been accepted as the doctrine for signal acquisition and processing ever since it was implied by the work of Harry Nyquist in 1928 [1], and proved by Claude E. Shannon in 1949 [2]. The theorem shows that a bandlimited analog signal that has been sampled can be perfectly reconstructed from an infinite sequence of samples if the sampling rate exceeds 2B samples per second (the so-called Nyquist rate), where B is the highest frequency in the original signal. In many applications nowadays, including digital image and video cameras, the Nyquist rate is so high that the excessive number of samples makes compression a necessity prior to storage or 1

19 transmission. In other applications, including imaging systems (medical scanners and radars) and high-speed analog-to-digital converters, increasing the sampling rate is very expensive, either due to the sensor itself being expensive or the measurement process being costly. Oftentimes, we are faced with the situation in which, on the one hand, we try hard but cannot collect enough measurements required by the sampling theory, and, on the other hand, we desperately desire exact or near exact signal reconstruction. For example, in the fuel cell imaging project, neutron scattering technique is used as a scientific probe to see the inside of a fuel cell. By shooting the neutrons at the fuel cells, information from the scattering patents can be collected. However, shooting neutrons is a very expensive process, and what is worse, the measurement process could be very slow as it takes a long time to acquire data. We have to compromise and deal with this ill-posed problem, where we try to sense an n dimensional object that has n unknown values but the number of measurement m is much smaller than n. This violates what we have learned in linear algebra, and thus leads us to the question of which measurements we should take and how we should reconstruct the original signal. Another example is in the field of medical imaging using Magnetic Resonance Imaging (MRI), where measurements are taken by collecting Fourier coefficients. Problems in the traditional MRI are: if fewer measurements are collected than the number of pixels, then the image reconstruction is often impossible; while on the other hand, it is extremely difficult to acquire enough Fourier coefficients, since the process of getting Fourier coefficients from magnetic resonance (MR) takes a long time, and the patient thus has to spend a long time in the scanner. Consequently, two unwanted things will happen: First of all, the patient starts to move after a while, so the measurements become inaccurate; Second, the doctors cannot do video, because it takes a long time to get even a single frame. These problems with MRI extraordinarily limited its applicability. In order to widen the applicability of MRI, it is necessary to speed it up, which means fewer samples should be taken in the Fourier domain, while sufficient accuracy is still wanted. 2

20 1.3 Compressive Sensing: A Theoretical Breakthrough Faced with these challenges, starting from 2004, in part due to several important results by David Donoho [3], Emmanuel J. Candès, Justin Romberg and Terence Tao [4, 5], a novel sensing/sampling paradigm that goes against the common wisdom in data acquisition came into being. Relying on two principles: Sparsity, which pertains to the signals of interest, Incoherence, which pertains to the sensing modality, the theory of compressive sampling, also known as compressive sensing or compressed sensing, asserts that, based on a suitable projection measurement, one can perfectly recover certain signals or images with high probability from far fewer samples or measurements than the traditional methods use [4] Compressible Signals Here, we describe the sparse properties of the signal we are going to deal with. Generally speaking, sparse signals are those that contain much less information than their ambient dimension suggests. Consider a finite-length, one-dimensional, discrete-time signal x, which can be viewed as an N 1 column vector in R with elements x i, i = 1, 2,..., N. 1 For higher-dimensional data, we can simply vectorize it into a long one-dimensional vector. Any x R N can be represented in terms of an orthonormal basis of N 1 vectors {ψ i } N i=1. Under such basis, signal x can be written as or x = N s i ψ i, (1.1) i=1 x = Ψs, (1.2) 1 Here, we consider only real numbers for simplicity. Similar results hold for the complex case. 3

21 where s is the N 1 column vector of weighting coefficients s i = x, ψ i = ψi T x, and the superscripts T denotes transposition and Ψ is a N N basis matrix, with ψ i as a column. Apparently, x is the time or space domain representation of the signal, and s is an equivalent representation, but in the Ψ domain. Signal x is said to be K-sparse in the Ψ domain when only K out of the N s i coefficients are nonzeros and the rest N K are zeros. The study of CS is mainly concentrated on the cases where K N. And in such cases, the signals are viewed as compressible. In practice, the conditions for sparse signal is relaxed to the following: if the representation (1.1) has a few large coefficients and many small coefficients, signal x is sparse. Many natural and factitious signals are not compressible in their original form. However, we can find a basis Ψ where the representation of the signal has very limited number of significant coefficients and map the original signal to this basis to form a compressible signal. Moreover, such compressible signal can be well approximated by K-sparse representation [6]. Conventional signal acquisition systems take advantage of this compressible property and compress the signal to save storage and transmission cost. For example, in order to record an image, a digital camera first take one measurement of the intensity for each of its pixels, resulting in a relatively large image file, and then, after wavelet transform, only those large wavelet coefficients will be saved in order to reconstruct the image with little or acceptable quality loss. However, this sample-then-compress framework suffers from three inherent inefficiencies. First, the initial number of samples N may be large even if the desired K is small. Second, the set of all N transform coefficients s i must be computed even though all but K of them will be discarded. Third, the locations of the large coefficients must be encoded, thus introducing an overhead [7] Compressed Measurement Under such circumstances, it is natural to ask whether there is a clever way of obtaining the compressed version of the signal directly, by taking only a small number of measurements of the 4

22 signal. Compressive sensing addresses the aforementioned inefficiencies and provides a way of reconstructing a compressed version of the original signal by taking only a small amount of linear and non-adaptive measurements [3, 4]. Taking M linear measurements of a signal x can be done by applying a matrix Φ M N (measurement matrix) to the signal. And the measurement procedure can be written as y = Φx = ΦΨs = Θs, (1.3) where Θ is an M N matrix. The scenario of CS measurement is shown in Figure 1.1. Since Φ does not depend on signal x, this measurement is non-adaptive. And vector y with dimension M 1 (M N) is the compressed linear measurement of signal x. The remaining problems in the CS frame work design are: First, find a stable measurement matrix which can preserve the information in signal x during dimension reduction; Second, determine a signal reconstruction algorithm for solving the ill-posed linear equations which also guarantees a unique sparse solution Stable Measurement Matrix A Necessary and Sufficient Condition A necessary and sufficient condition to ensure that this M K system can be compressed and reconstructed is stated as the following property [4, 5]: Definition 1.1. Restricted Isometry Property (RIP): For any vector v sharing the same K nonzero entries as x, if 1 ϵ Θv 2 v ϵ (1.4) for some ϵ > 0, then the matrix Θ preserves the information of the K-sparse signal. In other words, matrix Θ must preserve the lengths of these particular K-sparse vectors. Besides, the restricted isometry property implies also robustness under noise on the measurements 5

23 y Φ Ψ s = x y Θ s = length-n K-sparse length-m K<M<<N M * N Figure 1.1: Compressive sensing illustrator. 6

24 [4, 5]. However, in practice we do not know the locations of the K nonzero entries in s. It has been shown in some literatures that, a sufficient condition for a stable inverse for a K-sparse signal is that the matrix Θ has to satisfy equation (1.1) for an arbitrary 3K-sparse vector v. In other words, if matrix Θ holds this property, one can find the sparse solution s of the under-determined system y = Θs using linear programming instead of a combinatorial search. An alternative approach to stability is to ensure that the measurement matrix Φ is incoherent with the sparsifying basis Ψ, in the sense that the vectors {ϕ j } cannot sparsely represent the vectors {ψ i } and vice versa [4, 3]. Although we have the RIP, which is a necessary and sufficient condition, and R. Baraniuk et al., offer a simple technique for verifying the above RIP condition for random matrices in [8], given a sparsifying basis Ψ, we have no clue on how to construct a measurement matrix Φ such that Θ = ΦΨ satisfy the RIP. What is worse, merely verifying that a given Θ holds RIP is combinatorially complex; and we must verify equation (1.1) for each of the C K N possible combinations of K nonzero entries in the length-n vector v Random Matrices Quite surprisingly, RIP is satisfied with high probability by a large class of random matrices. It was shown in [5, 4] that Gaussian, Bernoulli, and partial random Fourier matrices, or to say, almost all independent and identically distributed (i.i.d.) matrices possess the RIP condition with high probability. For example, we take measurements of an N-dimensional sparse signal x with a Gaussian random matrix Φ, whose entries are independent and identically distributed with zeromean and 1/N-variance, the resultant measurements vector y will be randomly weighted linear combinations of the N elements of x. It has been proved in [5, 4], such an M N Gaussian i.i.d. random measurement matrix hold the RIP with high probability if M ck log(n/k), where c is a small constant. This is because Φ is incoherent with the basis Ψ = I of spikes with hight probability. Besides, the properties of the i.i.d. Gaussian distribution of Φ, the matrix Θ = ΦΨ is also i.i.d. Gaussian, regardless of the choice of Ψ. Thus the Gaussian random matrix is universal 7

25 for stable compressive sensing measurement. Among many other measurement matrices, the following have been proved stable and efficient [5, 4]: ϕ ij = { +1, with probability 1 2 ; 1, with probability 1 2. (1.5) +1, with probability 1 3 ; ϕ ij = 0, with probability 1 3 ; (1.6) 1, with probability Toeplitz Matrices, Circulant Matrices and Random Convolution Randomness leads to theoretical guarantee but is hard to realize in practice, and further requires extra storage space for the measurement matrices. Besides, in some applications, random sensing matrices are not admissible. For example, in the wireless communication applications, signal transmission involves the process of transmitted signal convolving with the channel, where convolution is a circulant linear operator. This convolution leads to a structured and thus less random measurement matrix, which is not favored by CS. Nevertheless, random Toeplitz and circulant matrices can be easily or even naturally realized in various applications, and carefully designed circulant matrices can deliver the same optimal CS performance. An n n Toeplitz matrix takes the form A = a 0 a 1 a 2 a n+1 a 1 a 0 a a 2 a a 1 a a 1 a 0 a 1 a n 1 a 2 a 1 a 0, (1.7) in which each descending diagonal from left to right is constant. 8

26 An n n circulant matrix takes the form C = c 0 c n 1 c 2 c 1 c 1 c 0 c n 1 c 2. c 1 c c n c n 1 c n 1 c n 2 c 1 c 0. (1.8) A circulant matrix is fully specified by one vector, which appears as the first column of C. The remaining columns of C are each cyclic permutations of the first column vector with offset equal to the column index. From the construction of Toeplitz and circulant matrices, we can see that, compared with an m n general matrix which has mn degrees of freedom, a Toeplitz or circulant matrix of the same size has at most n degrees of freedom. Hence, a random Toeplitz or circulant matrix is generated from much fewer independent random numbers or is less random than an i.i.d. random matrix of the same size. This fact seemingly suggests that a random Toeplitz or circulant matrix would yield less incoherent projections, and consequently worse CS recovery. However, simulation results presented by Yin et al., in [9] show that circulant matrices can be equally effective as i.i.d. random matrices. For comparison purpose, we carried out a simple simulation: For the same sparse signal of length 256, we take the same number (i.e., 64) of measurements using three different measurement matrices, say the Gaussian random sensing matrix, the complex matrix with random phase and uniform magnitude, and the random circulant complex matrix, respectively. Then, we use the same l 1 -minimization algorithm for signal recovery. Figure 1.2 shows the results. Clearly, although there is a performance gap between the Gaussian random sensing matrix and the two types of structured sensing matrices, the gap is only less than 1 db on average. Despite this negligible gap, the carefullydesigned structured sensing matrices of both types perform as good as the Gaussian random sensing matrix under l 1 minimization, besides, they have the advantage of helping to simplify the sensing system. To conclude, CS senses less and computes more. A simple comparison between CS and the traditional signal acquisition and processing is shown in Figure 1.3. It is clear that CS takes fewer 9

27 10 0 Compare Different Measurement Matrix Reconstruction MSE Vs. Sparsity Level (SNR=20dB) Gaussian random complex sensing matrix, L1 minimization Random phase and uniform magnitude, L1 minimization Random circulant complex matrix, L1 minimization MSE K Sparse Figure 1.2: Compare different measurement matrix. x Sample Compress Transmit / Store Receive Decompress ˆx Traditional Compressive sensing x Compressive sensing (senses less, faster) Transmit / Store Receive Reconstruction ˆx Figure 1.3: Traditional sensing vs. Compressive sensing. 10

28 measurements than traditional sensing and requires no additional compression. The cost is that the decoding process that reconstructs the original information from the incomplete measurements becomes a computational algorithm. For many applications, such a shift of resource demands from pre transmission to post-transmission can be of great benefit. For example, a lower power demand on encoding can greatly extend the life and cost of a mobile sensor, while a large increase in decoding power at the central processing unit is affordable. CS also leads to breakthroughs in applications where the efficiency bottleneck lies in the high cost of data acquisition and motivates researchers to (more actively) exploit sparsity in other information-seeking problems arising in areas such as statistics and inverse problems Signal Reconstruction RIP provides the theoretical guarantee that a K-sparse or compressible signal can be fully described by the M measurements, but it does not tell us how to recover it. A long list of ideas and suggestions can be gleaned from papers, like [10, 11, 12, 13, 14, 15] etc., with new proposals appearing regularly. Here, we can only provide a brief and incomplete survey of existing approaches. Facing the problem of finding the sparse solution from a set of underdetermined linear equation, the first method that would come to mind is to search for the sparsest vector x which is consistent with the measurement vector y = Φx. This leads to solving the l 0 -minimization problem min ˆx 0, such that Φˆx = y. (1.9) Since the l 0 -norm of vector X counts the number of non-zero entries, it is generally NP hard [16]. The next candidate is the classical least-squares method, which is to find the vector in the translated null-space with the smallest l 2 norm min ˆx 2, such that Φˆx = y. (1.10) And there is a close form solution to this problem ˆx = Φ T (ΦΦ T ) 1 y. However, the l 2 -minimization won t be able to converge to a sparse solution [16]. 11

29 In literature, there are two types of relaxations to this problem. The first type is convex relaxation, which leads to l 1 -minimization, also referred to as basis pursuit [11]. Another group of solutions use the greedy algorithms, such as various matching pursuits (MP), Orthogonal Matching Pursuit (OMP), Regularized Orthogonal Matching Pursui (ROMP), Compressive Sampling Matching Pursuit (CoSaMP), etc. [17, 18, 19, 20]. The basic idea behind sparse signal reconstruction is to identify the smallest subset of columns of Θ whose linear span contains (approximately) the measurement y. Still there are other methods been developed from many viewpoints: iterative thresholding methods [14]; iterative support detection methods [21]; subspace methods [22]; and nonconvex optimization methods [23]. We should also realize that, for different types of approaches, various recovery algorithms are available. Depending on different designs of measurement matrix Φ, different algorithms require different numbers of measurements (M), regarding to the same signal length N and sparsity level K, and offer different probabilities of exact signal recovery. In this part, we only choose to discuss the most commonly used l 1 -minimization methods and the greedy methods, especially, the OMP method in the greedy methods group l 1 -minimization (Basis Pursuit) The l 1 -minimization approach considers the solution of min ˆx 1, such that Φˆx = y, (1.11) which is a convex optimization problem and can be viewed as a convex relaxation of equation (1.9). The l 1 norm is defined as x 1 = N 1 i=0 x i. In the real-valued case, equation (1.11) is equivalent to a linear programming and in the complex-valued case it is equivalent to a second order cone programming. Various standard convex optimization techniques can be used to solve these problems efficiently [16]. The reason why this l 1 relaxation of (1.9) leads to sparse and unique solution is intuitively explained in Figure 1.4 (b). In this case, signal x is a two dimensional signal (N = 2) and measure- 12

30 l_p ball x2 x2 y = Φx y= Φx y= Φx x2 x1 x1 x1 (a) 0 < p < 1 (b) p = 1 (c) p = 2 Figure 1.4: l 1 norm minimization offers unique and sparse solution. ment Y is one dimensional (M = 1). So we are trying to find the solution for F(y) = {ˆx : Φˆx = y} in R 2. Except for the pathological cases, where Φ is parallel to one of the faces of the polytope which is the l 1 -ball, there exists a unique sparse solution to the l 1 -minimization problem. Comparatively, in Figure 1.4 (a) when 0 < p < 1, the convergence is too slow, and in Figure 1.4 (c), when p = 2, we may end up with none sparse and multiple solutions. In the presence of noise, the original problem (1.11) becomes min ˆx 1, such that y Φˆx 2 < ξ, (1.12) where ξ is the measurement noise. As long as the measurement matrix Φ satisfies the RIP condition, solving (1.12) still lead to very good approximation of x Greedy Algorithms (Matching Pursuits) An alternative approach to sparse recovery is via greedy algorithms which have been discussed in [17,18,19,20]. These methods find the support of the signal x iteratively, and reconstruct the signal using the pseudo-inverse. Based on the assumption that the columns of measurement matrix Φ are approximately or- 13

31 thonormal, Φ Φx is locally a good approximation to x. OMP uses this idea to compute the support of a K-sparse signal by greedy search. At the beginning of the algorithm, the residual r which is defined as r = y Φx (1.13) is set to be the measurement vector y. At each iteration, the observation vector is set as x = Φ r, and the coordinate of its largest coefficient in magnitude is added to index set I. Then by solving a least-squares problem x = arg min x R I y Φx 2, (1.14) the residual is updated to remove this coordinate s contribution. Repeating this K times yields an index set of K coordinates corresponding to the support of signal x. [17] shows that OMP recovers a sparse signal with high probability. Another class of algorithms recursively solve a sequence of iteratively re-weighted linear least-squares (IRLS) problem [24]. Recent results presented in [25] consider the use of iteratively re-weighted algorithm for computing local minima of the non-convex problem. In particular, a particular regularization strategy is found to greatly improve the ability of a re-weighted least-squares algorithm to recover sparse signals, with exact recovery being observed for signals that are much less sparse than required by an un-regularized version presented in [24]. The authors in [25] further establish sufficient conditions for the noiseless case, such that the sequence converges to the sparsest solution. Other greedy algorithms share the similar basic idea, and we refer the readers to [17, 18, 19, 20] for further information. In general, greedy algorithms won t be able to outperform the l 1 -minimization algorithms in the sense of recovered mean square error (MSE) for sparse reconstruction. There are performance analysis as well as extensive computational experiment results in [26]. However, greedy algorithms are still popular and well accepted due to the following advantages: First, they are straightforward for hardware implementations and thus suitable for embedded 14

32 and real-time architectures. For example, the authors in [27, 28] offered FPGA implementation examples; Second, they usually require fewer measurements than the l 1 -minimization algorithms. 1.4 Compressive Sensing in Matrix Form: An Overview Distributed Compressive Sensing via Joint Sparsity Recovery Since distributed communication, sensing, and computing are emerging fields with numerous promising applications [29, 30], an efficient data acquisition and processing scheme becomes a necessity. A new theory distributed compressive sensing (DCS), was proposed by Wakin et al., in [31], and discussed in [32, 33], etc. Rested on a new concept that the authors term the joint sparsity of a signal ensemble, the DCS theory enables new distributed coding algorithms for multisignal ensembles that exploit both intra- and inter- signal correlation structures. The incorporation of a joint sparsity measurement allows the treatment of not only signal in matrix form but also infinite dimensional signals like analog signals and infinite resolution images. Based on the observation that in many distributed sensing applications, such as the sensor networks and arrays, the sensing information often exhibits strong spatial correlations [29, 30], DCS theory proposes a new approach for distributed coding of correlated sources whose signal correlations take the form of a sparse structure, also known as the common sparse supports. In a typical DCS scenario, as shown in Figure 1.5, a number of sensors measure signals that are each individually sparse in some basis and also correlated from sensor to sensor. Each sensor independently encodes its signal by projecting it onto another, 2 incoherent basis (such as a Gaussian random matrix) and then transmits just a few of the resulting coefficients to a central control unit. Under the right conditions, a decoder at the central control unit can reconstruct each of the signals with the same sparse pattern precisely. While the sensors operate entirely without collaboration, and the recovered signals from all sensor nodes share common sparse supports, joint decoding can recover signals using far fewer measurements per sensor than those required for separate CS 2 Note that all sensor can share the common measurement matrix, by independently we mean that there is no information exchange among sensor nodes. 15

33 X1, X2,..., Xn = Measured data at the central control unit Common measurement matrix NOTE: X1, X2,..., Xn can use different measurement matrices Each column vector X1, X2,..., Xn is sparse X1, X2,..., Xn are all different, but they share the common sparse pattern Figure 1.5: Joint sparsity illustrator. recovery. DCS signal recovery algorithms can be combinations of the existing CS signal recovery algorithms which are typically constructed as weighted l 1 -norms of vector l q -norms, where q > 1. However it is better to be designed case by case according to the sparse structure, signal variation, measurement quality, and many other factors. In Chapter 2, we show an example of distributed compressive sensing (DCS) via joint sparsity recovery with a unique and effective algorithm for our underlying problem Matrix Completion The matrix completion problems consist in studying the possibility to complete a matrix, when some of its entries are prescribed (i.e., are fixed), such that the resulting matrix satisfies certain properties [34]. Motivated by the fact that real application data often contain missing observations, corruption or even malicious errors and noise, the general matrix completion problem has been studied for some decades. Very recently, to some extent related to the success of CS, with the 16

34 + Matrix with partial observations (Black blocks are unknown) Underlying low-rank matrix Sparse error matrix Figure 1.6: Matrix completion illustrator. publication of [35, 36, 37] etc., the completion algorithms for matrices with certain property low rank became popular research topics. Recall that in order to recover a vector signal from a few measurements (less than its dimension) through convex optimization, we require the signal to hold the sparse or compressible property. Similarly, for a two dimensional signal recorded in a matrix, if it holds the low-rank property, we can fulfill it from only a small number of its revealed entries via convex optimization. Here, we only investigate the low-rank matrix completion problem and the possible solutions briefly, and leave the detailed discussion to Chapter 2, where we give an example of how to formulate and solve the collaborative spectrum sensing problem in cognitive radio networks via matrix completion and joint sparsity recovery. In a general matrix completion problem, as shown in Figure 1.6, we try to fulfill a matrix M m n, where rankm = K, and K min{m, n}. We have only a subset E m n of M s entries available. We assume that the revealed entries are uniformly distributed with high probability. Hence, we work with a model in which each entry shows up in E independently with probability ϵ/ m n. Given E m n, the partial observation of M is defined as an m n matrix given by M E ij = { Mij, if (i, j) E, 0, otherwise. (1.15) The low-rank matrix completion problem is to reconstruct M from the partial observation M E ij. 17

35 Since we have the prior information that matrix M is low-rank, it is natural to think of reconstruct M via rank minimization. However, the matrix rank function is non-convex, rank minimization is an NP hard problem [16]. Again, similar to what we have done for l 0 -minimization, we relax this rank minimization problem to a nuclear-norm minimization problem written as min M R m n M (i,j) E Mij Mij E 2. (1.16) In equation (1.16),. denotes the nuclear-norm, which is the sum of the singular values of the matrix, and the latter half of the equation takes care of the measurement noise. Solving equation (1.16) guarantees a unique low-rank solution. There are numerous algorithms for solving equation (1.16), including but not limited to, the singular value thresholding (SVT) algorithm [36] and the fixed-point continuation iterative algorithm (FPCA) [38] for fast completion of large-scale matrices (e.g., more than ), and a special trimming step introduced by Keshavan et al., in [37]. We refer the readers to [38, 36, 37] for details of the algorithms and to Chapter 2 for an application example. 1.5 Organization of This Dissertation In this dissertation, we proposed several implementations of the compressive sensing framework for wireless communications. In each of the application, we firstly investigated signal sparsity, then designed stable and physically realizable compressive sensing based measurement, and further proposed proper signal recovery algorithms according to the signal character and the designed measurement schemes. In addition, we showed simulation results for each proposed implementation. In Chapter 2, we designed a compressive sensing framework for efficient collaborative spectrum sensing in cognitive radio networks. We proposed to apply matrix completion and joint sparsity recovery to reduce sensing and transmission requirements and improve spectrum sensing capability. In our system, each cognitive radio node senses linear combinations of multiple channel information through an on board frequency selective filter set and reports them to the fusion center, where occu- 18

36 pied channels are then decoded from the reports by using novel matrix completion and joint sparsity recovery algorithms. As a result, the number of reports sent from the CRs to the fusion center was significantly reduced. We proposed two decoding approaches, one based on matrix completion and the other based on joint sparsity recovery. Both approaches allow exact spectrum sensing from incomplete sensing reports. The numerical results validated the effectiveness and robustness of our approaches. In particular, in small-scale networks, the matrix completion approach achieved exact channel detection with a number of samples no more than 50% of the number of channels in the network, while joint sparsity recovery achieved similar performance in large-scale networks. In Chapter 3, we formulated OFDM channel estimation as a compressive sensing problem, which took advantage of the sparsity of the channel impulse response and reduced the number of probing measurements, which in turn reduced the ADC speed needed for channel estimation. Specifically, we proposed to send out pilots with random phases in order to spread out the sparse taps in the impulse response over the uniformly downsampled measurements at the low-speed receiver ADC, so that the impulse response still can be recovered by sparse optimization. This contribution led to high-resolution channel estimation with low-speed ADCs, distinguishing this work from the existing attempts of OFDM channel estimation. We also proposed a novel estimator that performs better than the commonly used l 1 minimization. Specifically, it significantly reduced estimation error by combing l 1 minimization with iterative support detection and limited-support least-squares. While letting the receiver ADC running at a speed as low as 1/16 of the speed of the transmitter DAC, we simulated various numbers of multipaths and different measurement SNRs. The proposed system has channel estimation resolution as high as the system equipped with the high-speed ADCs, and the proposed algorithm provides additional 6 db gain for signal to noise ratio. In Chapter 4, we investigated the implementation of CS for the 60 GHz ultra wide-band (UWB) communication which is an emerging technology for high speed short range communications. In this scenario, the requirement of high-speed sampling increases the cost of receiver circuitry such as analog-to-digital converter (ADC). We presented a compressive sensing framework 19

37 for 60 GHz UWB communication system to achieve a significant reduction of sampling rate. The basic idea is based on the observation that the received signals are sparse in the time domain due to the limited multipath effects at 60 GHz wireless transmission. According to the theory of compressive sensing, by carefully designing the sensing scheme, sub-nyquist rate sampling of the sparse signal still enabled exact recovery with very high probability. We discussed an implementation for a low-speed A/D converter for 60 GHz UWB received signal. Moreover, we analyzed the bit error rate (BER) performance for BPSK modulation under RAKE reception. Simulation results show that in the single antenna pair system model, sampling rate can be reduced to 2.2% with 0.3 db loss of BER performance if the input sparsity is less than 1%. Consequently, the implementation cost of ADC can be significantly reduced. In Chapter 5, we proposed a compressive sensing based sparse events detection scheme in wireless sensor networks (WSN). The occurrence of sensor activation events in WSN is sparse. We took advantage of this sparse property and proposed an efficient CS based detection scheme. We also present a fully probabilistic Bayesian framework for signal recovery from CS measurements. This Bayesian framework helped us dramatically reduced the sampling rate while maintaining overwhelming detection probability. We further adopted marginal likelihood maximization algorithm and a heuristic algorithm which led to a much higher detection probability than the general linear programming solution for CS signal recovery problem. Finally, in Chapter 6, we concluded our work and explored the possible extensions of our proposed framework. We proposed some future work in wireless communication area, such as the dynamic compressive sensing for wireless communication applications, and joint sparsity recovery algorithm for MIMO wireless communication applications. Moreover, we presented the idea of implementing CS framework in miscellaneous area where we recognized the signal sparsity. Finally, we briefly described compressive sensing for seismic data acquisitions and processing, compressive sensing for concrete flaw detection, compressive sensing for pipe line leakage detection, compressive sensing for human vision model, compressive sensing for gene regulatory networks. 20

38 Chapter 2 Collaborative Spectrum Sensing in Cognitive Radio Networks Spectrum sensing which aims at detecting spectrum holes, is the precondition for the implementation of cognitive radio (CR). Collaborative spectrum sensing among the cognitive radio nodes is expected to improve the ability of checking complete spectrum usage. Due to hardware limitations, each cognitive radio node can only sense a relatively narrow band of radio spectrum. Consequently, the available channel sensing information is far from being sufficient for precisely recognizing the wide range of unoccupied channels. Aiming at breaking this bottleneck, we proposed a CS based collaborative spectrum sensing scheme. In Section 2.1, we give a brief introduction of the background of this work; In Section 2.2, the proposed system model is given. The matrix completion-based algorithm for collaborative sensing is described in Section 2.3, and the joint sparsity based algorithm is described in Section 2.4. After that, in Section 2.5 we compare the two proposed approaches, discuss their computational complexity as well as filter design and dynamic update. Simulation results are presented in Section 2.6, and conclusions are drawn in Section Introduction Ever since the 1920s, every wireless system has been required to have an exclusive license from the government in order not to interfere with other users of the radio spectrum. Today, with the emergence of new technologies which enable new wireless services, virtually all usable radio frequencies are already licensed to commercial operators and government entities. According to former U.S. Federal Communications Commission (FCC) chair William Kennard, we are facing a spectrum drought [39]. On the other hand, not every channel in every band is in use all the time; 21

39 even for premium frequencies below 3 GHz in dense, revenue-rich urban areas, most bands are quiet most of the time. The FCC in the United States and the Ofcom in the United Kingdom, as well as regulatory bodies in other countries, have found that most of the precious, licensed radio frequency spectrum resources are inefficiently utilized [40, 41]. In order to increase the efficiency of spectrum utilization, diverse types of technologies have been deployed. Cognitive radio is one of those that leads to the greatest technological gain in wireless capacity. Through the detection and utilization of the spectra that are assigned to the licensed users but standing idle at certain times, cognitive radio acts as a key enabler for spectrum sharing. Spectrum sensing, aiming at detecting spectrum holes (i.e., channels not used by any primary users), is the precondition for the implementation of cognitive radio. The Cognitive Radio (CR) nodes must constantly sense the spectrum in order to detect the presence of the Primary Radio (PR) nodes and use the spectrum holes without causing harmful interference to the PRs. Hence, sensing the spectrum in a reliable manner is of vital importance and constitutes a major challenge in CR networks. However, detection is compromised when a user experiences shadowing or fading effects or fails in an unknown way. To get a better understanding of the problem, consider the following example: a typical Digital TV receiver operating in a 6 MHz band must be able to decode a signal level of at least 83 dbm without significant errors [42]. The typical thermal noise in such bands is 106 dbm. Hence a CR which is 30 dbm more sensitive has to detect a signal level of 113 dbm, which is below the noise floor [43]. In such cases, one CR user cannot distinguish between an unused band and a deep fade. In order to combat such effects, recent studies suggest collaboration among multiple CR nodes for improving spectrum sensing performance. There has been an increasing research interest in cooperation and learning for dynamic spectrum access during the last few years. A few results are briefly explained here. In [44], the CRs collaborate by sharing their sensing decisions through a centralized fusion center that combines the CRs sensing bits. A similar approach is used in [45] using different decision-combining methods. In [46], spatial diversity techniques are proposed for improving the performance by combating the error probability due to fading on the reporting channel between the CRs and the FC. Other perfor- 22

40 mance aspects are studied in [43, 47, 48]. Collaborative spectrum sensing (CSS) techniques are introduced to improve the performance of spectrum sensing. By allowing different secondary users to collaborate and share their information, PR detection probability can be greatly increased. There are many results that address cooperative spectrum sensing schemes and challenges. The performance of hard-decision combining scheme and soft-decision combining scheme is investigated in [44, 49]. In these schemes, all secondary users send sensing reports to a common decision center. Cooperative sensing can also be done in a distributed way, where the secondary users collect reports from their neighbors and make the decision individually [50,51,52]. Optimized cooperative sensing is studied in [53,54]. When the channel that forwards sensing observations experiences fading, the sensing performance degrades significantly. This issue is investigated in [55, 56]. Furthermore, energy efficiency in collaborative spectrum sensing is addressed in [57]. CSS can be classified into two categories. The first category involves multiple users exchanging information [44, 57], and the second category uses relay transmission [51, 52]. Some recent studies on collaborative spectrum sensing include cooperative scheme design guided by game theory [50] and random matrix theory [48], cluster-based cooperative CSS [58], and distributed rule-regulated CSS [59]; studies concentrating on CSS performance improvement include [46] introducing spatial diversity techniques to combat the error probability due to fading on the reporting channel between the CR nodes and the central fusion center. There are also studies concerning other interesting aspects of CSS performance under different constraints [48, 47, 60, 61]. Very recently, there are emerging applications of the compressive sensing concept for CSS [62]. Existing literature mostly focuses on the CSS performance examination when the centralized fusion center receives and combines all CR reports. In an n channel cognitive radio network with m CR nodes, the fusion center has to deal with n m reports and combine them wisely to form a channel sensing result. However, it is known that wireless channels are subject to fading and shadowing. When secondary users experience multi-path fading or happen to be shadowed, the reports transmitted by CR users are subject to transmission loss. As a result, in practice, no entire 23

41 report data set is available at the fusion center. Besides, due to the fact that each cognitive radio can only sense a small proportion of the spectrum with limited hardware, each CR user gathers only very limited information about the entire spectrum. Contributions: We sought to release CRs from sending, and the central control unit from gathering, an excessively large number of reports, also targeted at the situations where there are only a few CR nodes in a large network and thus unable to gather enough sensing information for the traditional CSS. We proposed to equip each cognitive radio node with a frequency-selective filter, which linearly combines multiple channel information. The linear combinations are sent as reports to the fusion center, where the occupied channels are decoded from the reports by compressive sensing algorithms. As a consequence, the amount of channel sensing at CRs and the number of reports sent from the CRs to the fusion center are both significantly reduced. Following our previous work [63, 64] on compressive sensing, we proposed two approaches to collaborative spectral sensing. The first approach is based on solving a matrix completion problem [35, 36, 38, 37, 65], which seeks to efficiently reconstruct a matrix (typically low-rank) from a relatively small number of revealed entries. In this approach, the entries of the underlying matrix are linear combinations of channel powers. Each CR node takes its local spectrum measurements, but instead of directly recording channel powers, it uses its frequency-selective filters to take p linear combinations of channel powers and reports them to the fusion center. The total p m linear combinations taken by m CRs form a p m matrix at the fusion center. Considering transmission loss, we allow the matrix to be incomplete. We show that this matrix is low-rank and has the properties enabling its reconstruction from only a small number of its entries, and therefore, information about the complete spectrum usage can be recovered from a small number of reports from the CR nodes. This approach significantly reduces the amount of sensing and communication workload. The second approach is based on joint sparsity recovery [32,20,33,19,66], which is motivated by the observation that the spectrum usage information the CR nodes collect has a common sparsity 24

42 pattern: each of the few occupied channels is typically observed by multiple CRs. We develop a novel algorithm for joint sparsity signal recovery, which is more effective than existing algorithms in the compressive sensing literature since it can accommodate a large dynamic range of channel gains. In both approaches, every CR senses all channels (by taking random linear projections of the powers of all channels), and the CRs do not communicate. While they work independently, their measurements are analyzed jointly by the detection algorithms running at the fusion center. Therefore, our approaches are very different from the existing collaborative spectrum sensing schemes in which different CRs are assigned to different channels. Our approaches move from collaborative sensing to collaborative computation and shift coordination from the sensing phase to the post-sensing phase. Our work is among the first that applies matrix completion or joint sparsity recovery to collaborative spectrum sensing in cognitive radio networks. Matrix completion and joint sparsity recovery are both being intensively studied in the compressive sensing community. We presented them both because it is too early at this time to make a verdict of an eventual winner. 2.2 System Model We consider a cognitive radio network with m CR nodes that locally monitor a subset of n channels. A channel is either occupied by a PR or unoccupied, corresponding to the states 1 and 0, respectively. We assume that the number s of occupied channels is much smaller than n. The goal is to recover the occupied channels from the CR nodes observations. Since each CR node can only sense limited spectrum at a time, it is impossible for limited m CRs to observe n channels simultaneously. To overcome this problem, we propose the scheme depicted in Fig Instead of scanning all channels and sending each channel s status to the fusion center, using its frequency-selective filters, a CR takes a small number of measurements that are linear combinations of multiple channels. 25

43 The filter coefficients can be designed and implemented easily. In order to mix the different channel sensing information, the filter coefficients are designed to be random numbers. Then, these filter outputs are sent to the fusion center. Suppose that there are p frequency-selective filters in each CR node sending out p reports regarding the n channels. For the non-ideal cases, where we have relatively less measurements pm < n, i.e., the number of reports sent from all CRs is less than the total number of channels. The sensing process at each CR can be represented by a p n filter coefficient matrix F. Let an n n diagonal matrix R represent the states of all the channel sources using 0 and 1 as diagonal entries, indicating the unoccupied or occupied states, respectively. There are s nonzero entries in diag(r). In addition, channel gains between the CRs and channels are described in an m n channel gain matrix G given by [67] G i,j = P i (d i,j ) α/2 h i,j, (2.1) where P i is the i th primary user s transmitted power, d i,j is the distance between the primary transmitter using j th channel and the i th CR node, α is the propagation loss factor, and h i,j is the channel fading gain. For AWGN channel, h i,j = 1, for all{i, j}; for Rayleigh channel, h i,j follows independent Rayleigh distribution; and for shadowing fading, h i,j follows log-normal distribution [67]. Without loss of generality, we assume that all PRs use unit transmit power (otherwise, we can compensate by altering the corresponding channel gains). The measurement reports sent to the fusion center can be written as a p m matrix M p m = F p n R n n (G m n ). (2.2) Note that due to loss or errors, some of the entries of M are possibly missing. The binary numbers on the diagonal of R are the n channel states that we shall estimate from the available entries of M. 2.3 CSS Matrix Completion Algorithm It is typically difficult for the fusion center to acquire all entries of M due to transmission failure, which means that our observation is a subset E [p] [m] of M. However, it is possible to 26

44 Spectrum Licensed Band1 Unoccupied Licensed Band2 Occupied Licensed Band3 Unoccupied Primary User Equipped Frequency Selective Filter Set Primary User Cognitive Radio Node Primary User Figure 2.1: System model. Cognitive Radio Fusion Center Compressive Spectrum Sensing M=FRG recover the missing entries in M since it holds the following two important properties [35] required for matrix completion: 1. Low Rank: rank(m) equals to s, which is the number of prime users in the network and is usually very small. 2. Incoherent Property: Generate F randomly (subject to hardware limitation). From (5.2) and the fact that R has only s nonzeros on the diagonal, M s SVD factors U, Σ, and V satisfy the incoherence condition [37]. There exists a constant µ 0 > 0 such that for all i [p], j [m], we have s k=1 U2 i,k µ 0 s, s k=1 V2 i,k µ 0s. There exists µ 1 such that s k=1 U i,kσ k V j,k µ 1 s 1/2. M is in general incomplete because of transmission failure. Moreover, each CR might only be able to collect a random (up to p) number of reports due to the hardware limitation. Therefore, 27

45 the fusion center receives a subset E [p] [m] of M s entries. We assume that the received entries are uniformly distributed with high probability. 1 Hence, we work with a model in which each entry shows up in E identically and independently with probability ϵ/ p m. Given E p m, the partial observation of M is defined as a p m matrix given by M E ij = { Mij, if (i, j) E; 0, otherwise. (2.3) We shall first recover the unobserved elements of M from M E. Then, we reconstruct (RG ) from the given F and M using the fact that all but s rows of (RG ) are zero. These nonzero rows correspond to the occupied channels. Since p and m are much smaller than n, our approach requires a much less amount of sensing and transmission, compared to traditional spectrum sensing in which each channel is monitored separately. In previous research on matrix completion [36, 38, 37, 65], it was proved that under some suitable conditions, a low-rank matrix can be recovered from a random, yet small subset of its entries by nuclear norm minimization min M R p n τ M (i,j) E M i,j M E 2 i,j, (2.4) where M denotes the nuclear norm of matrix M and τ is a parameter discussed in Section below. For notational simplicity, we introduce the linear operator P that selects the components E out of a p n matrix and form them into a vector such that PM PM E 2 2 = (i,j) E M i,j M E i,j 2. The adjoint of P is denoted by P. Recent algorithms developed for (2.4) include, but not limited to, the singular value thresholding (SVT) algorithm [36] and the fixed-point continuation iterative algorithm (FPCA) [38] for fast completion of large-scale matrices (e.g., more than ), a special trimming step introduced by Keshavan et al., in [37]. 1 Depending on the different channel gain, the CRs will select different coding/modulation/power control schemes so that the received signal-to-noise ratio can be maintained about a certain threshold. Due to this reason, we can assume that the loss of information is uniformly distributed. 28

46 For our problem, we adopt FPCA, which appears to run very well for our small dimensional tests. In the following subsections, we describe this algorithm and the steps we take for nuclear norm minimization. Also, we study how to use the approximate singular value decomposition (SVD)- based iterative algorithm introduced in [38] for fast execution. We further discuss the stopping criteria for iterations to acquire optimal recovery. Finally we show how to obtain R from the estimation M of M Nuclear Norm Min. via Fixed Point Iterative Algorithm FPCA is based on the following fixed point iteration { Y k = M k δ k P (PM k PM E ) M k+1 = S τδk (Y k ), (2.5) where δ k is step size and S α ( ) is the matrix shrinkage operator defined as follows: Definition 2.1. Matrix Shrinkage Operator S α ( ): Assume M R p m and its SVD is given by M = Udiag(σ)V T, where U R p r, σ R r +, and V R m r. Given α > 0, S α ( ) is defined as S τ (M) := Udiag (s α (σ)) V T, (2.6) with the vector s α (σ) defined as s α (x) := max{x α, 0}, component-wise. (2.7) Simply speaking, S τ (M) reduces every singular values (which is nonnegative) of M by τ; if one is smaller than α, it is reduced to zero. In addition, S α (M) is the solution of where F is the Frobenius norm. min α X + 1 X R m n 2 X M 2 F, (2.8) To understand (2.5), observe that the first step of (2.5) is a gradient-descent applied to the second term in (2.4) and thus reduces its value. Because the previous gradient-descent generally 29

47 increases the nuclear norm, the second step of (2.5) involves solving (2.8) to reduce the nuclear norm of Y k. Iterations based on (2.5) converge when the step sizes δ k are properly chosen (e.g., less than 2, or select by line search) so that the first step of (2.5) is not expansive (the other step is always non-expansive) Approximate SVD Based Fixed Point Iterative Algorithm As stated in [38], the second step of (2.5) requires computing the SVD decomposition of Y k, which is the main computational cost of (2.5). However, if one can predetermine the rank of the matrix M, or have the knowledge of the approximate range of its rank, a full SVD can be simplified to computing only a rank-r approximation to Y k. Combined with the above fixed point iteration, the resulting algorithm is called fixed-point continuation algorithm with approximate SVD (FPCA). Specifically, the approximate SVD is computed by a fast Monte Carlo algorithm developed by Drineas et al., [68]. For a given matrix A R m n and parameters k s, this algorithm returns approximations to the largest k s singular values corresponding to left singular vectors of the matrix A in a linear time Stopping Criterion for Iterations We tune the parameters in FPCA for a better overall performance. Continuation is adopted by FPCA, which solves a sequence of instances of (2.4), easy to difficult, corresponding to a sequence of large to small values of τ. The final τ is the given one but solving the easier instances of (2.4) gives intermediate solutions that warm start the more difficult ones so that the entire solution time is reduced. Solving each instance of (2.4) requires proper stopping. Because our ultimate goal is to recover 0/1 values on the diagonal of R, accurate solutions of (2.4) are not required. Therefore, we use the criterion M k+1 M k F max{1, M k < mtol, (2.9) F } 30

48 where mtol is a small positive scalar. Experiments show that 1e 6 is good enough for obtaining the optimal R Channel Availability Estimation Based on the Complete Measurement Matrix Since F has more columns than rows, directly solving X := RG in (5.2) from a given M is under-determined. However, each row X i of X corresponds to the occupancy status of channel i. Ignoring noise in M for now, X i contains a positive entry if and only if channel i is used. Hence, most rows of X are completely zero, so every column X,j of X is sparse and all X,j s are jointly sparse. Such sparsity allows us to reconstruct X from (5.2) and to identify the occupied channels, which are the nonzero rows of X. Since the channel fading decays fast, the entries of X have a large dynamic range, which none of the existing algorithms can deal sufficiently well. Hence, we develop a novel joint-sparsity algorithm briefly described as follows. The algorithm is much faster than matrix completion and typically needs 1 5 iterations. At each iteration, every column X,j of X is independently reconstructed using the model min{ i w i X i,j : F X,j = M,j }, where M,j is the jth column of M. For noisy M, we instead use the constraint F X,j M,j σ. The same set of weights w i is shared by all j at each iteration. w i is set to 1 uniformly at iteration 1. After channel i is detected in an iteration, w i is set to 0. Through w i, joint sparsity information is passed to all j. Channel detection is performed on the reconstructed X,j s at each iteration. It is possible that some reconstructed X,j is wrong, so we let larger and sparser X,j s have more say. If there is a relatively large X i,j in a sparse X,j, then i is detected. We have found this algorithm to be very reliable. The detection accuracy is determined by the accuracy of M provided. 31

49 2.4 CSS Joint Sparsity Recovery Algorithm (2.10)) In this section, we describe a new, highly effective algorithm for recovering X (in equation X n m = R n n (G m n ) (2.10) and thus R by thresholding X. The algorithm allows but does not require the same F for all CRs, i.e., each CR can use a different sensing matrix F. The design of F is discussed in Section below. In X, each column (denoted by X,j ) corresponds to the channel occupancy status received by CR j, and each row X i, corresponds to the occupancy status of channel i. Ignoring noise for now, a row has a positive value (i.e., X i, > 0) if and only if channel i is used. Since there are only a small number of used channels, X is sparse in terms of the number of nonzero rows. In each nonzero row X i,, there is typically more than one nonzero entry; in other words, if X i,j 0, other entries in the same row are likely to be nonzero. Therefore, X is jointly sparse. In the case that the true X contains noise, it is approximately, rather than exactly, jointly sparse. Joint sparsity is utilized in our algorithm to recover X. While there are existing algorithms for recovering jointly sparse signals in the literature (e.g., in [32, 20, 33]), our algorithm is very different and more effective for our underlying problem. None of the existing algorithms works well to recover X because the entries of X have a very large dynamic range. This is due to the fact that, in any channel fading model, channel gains decay rapidly with distance between CRs and PRs. Most of the existing algorithms are based on minimizing i X i, p for p 1 and p =. If p = 1, it is the same as minimizing the 1-norm of each column independently, so joint sparsity is not used for recovery. If p > 1 or p =, joint sparsity is considered, but it penalizes a large dynamic range since the large values in a nonzero row of X contribute superlinearly, more than the small values in that row, to the minimizing objective. In short, p close to 1 loses joint sparsity and p bigger than 1 penalizes large dynamic ranges. Our new algorithm not only utilizes joint sparsity but also takes advantages of the large dynamic range of X. 32

50 The large dynamic range has its pros and cons in CS recovery. It makes it easy to recover the locations of large entries, which can be achieved even without recovering the locations of smaller ones. On the other hand, it makes difficult to recover both the locations and values of the smaller entries. This difficulty has been studied in our previous work [69], where we proposed a fast and accurate algorithm for recovering 1D signals x by solving several (about 5 10) subproblems in the form of Truncated l 1 minimization: min{ i T x i : Ax = b}, (2.11) where the index set T is formed iteratively as {1,..., n} excluding the identified locations of large entries of x. With techniques such as early detections and warm starts, it achieves both the state of the art speed and least requirement on the number of measurements. We integrate the idea of this algorithm with joint sparsity into the new algorithm below. Algorithm 2.1 Joint detection algorithm T {1,..., n} repeat Independence recovery: X 0 X,j min{ i T X i,j : A j X,j = b j, X,j 0} for every CR j with enough measurements (In presence of measurement noise, A j X,j = b j is replaced by A j X,j b j σ) Channel detection: select trusted X,j and detect used channels from the selections Update of T : Update T according to detected channels and X until the tail of X is small enough Report X, and R by thresholding X The framework of the proposed algorithm is shown in Table 2.1. At each iteration, every channel is first subject to independent recovery. Unlike minimizing i X i, p, which ties all CRs together, independent recovery allows large entries of X to be quickly recovered. Joint sparsity information is passed among the CRs through a shared index set T, which is updated iteratively to exclude the used channels that are already discovered. Below, we describe each step of the above algorithm in more details. In the independence recovery step, for every qualified CR, a constrained problem in the form of (2.11) with constraints A j X,j = b j in the noiseless case, or A j X,j b j σ in the noisy 33

51 case, is considered, where σ is an estimated noise level. As problem dimensions are small in our application, solvers are easily chosen: MATLAB s linprog for noiseless cases and Mosek [70] for noisy cases. Both of these solvers run in polynomial times. This step dominates the total running time of Algorithm 2.1, but up to m optimization problems can be solved in parallel. Parallelization is simple for the joint-sparsity approach. At each outer iteration, all LPs are solved independently, and they have small scales relative to today s LP solvers, like Gurobi [71] and its MATLAB interface Gurobi Mex [72], where Gurobi automatically detects and uses all CPU and cores for solving LPs. CRs without enough measurements (e.g., most of their reports are missing due to transmission losses or errors) are not qualified for independent recovery because CS recovery is known unstable in such a case. Specifically, we require the number of the available measurements from each qualified CR to exceed twice as many as used channels or n T. When measurements are ample, the first iteration will yield exact or nearly exact X,j s. Otherwise, insufficient measurements can cause a completely wrong X,j that misleads channel detection; neither the locations nor the values of the nonzero entries are correct. The algorithm, therefore, filters trusted X,j s that must be either sparse or compressible. Large entries in such X,j s likely indicate correct locations. A theoretical explanation of this argument based on stability analysis for (2.11) is given in [21]. Used channels are detected among the set of trusted X,j s. To further reduce the risk of false detections, we compute a percentage for every channel in a way that those channels corresponding to larger values in X and whose values are located in relatively sparser X,j s are given higher percentages. Here, relative sparsity is defined proportionally to the number of measurements; for fixed number of non-zeros or degree of compressibility, the more the measurements, the higher the relative sparsity. Hence, X,j corresponding to more reported CR j also tends to have a higher percentage. In short, larger and sparse solutions have more say. The channels receiving higher percentages are detected as used channels. The index set T is set as {1,..., n} excluding the used channels that are already detected. Obviously, T needs to change from one iteration to the next; otherwise, two iterations will result in 34

52 an identical X and thus the stagnation of algorithm. Therefore, if the last iteration posts no change in the set of used channels yet the stopping criterion (see next paragraph) is not met, the channels i corresponding to the larger X i, 2 are also excluded from T, and such exclusion becomes more aggressive as iteration number increases. This is not an ad hoc but a rigorous treatment. It is shown in [21] that larger entries in an inexact CS recovery tend to be the true nonzero entries, and furthermore, as long as the new T excludes more true than false nonzero entries by a certain fraction, (2.11) will yield a better solution in terms of a certain norm. In short, used channels leave T, and in case of no leaves, channels with larger joint values X i, 2 leave T. Finally, the iteration is terminated when the tail of X is small enough. One way to define the tail size of X is the fraction i T X i, p / i T X i, p, i.e., the thought unused divided by the thought used. Suppose that T precisely contains the unused channels and measurements are noiseless, then every recovered X,j in channel detection is exact, so the fraction is zero; with noise, the fraction depends on noise magnitude and is small as long as noise is small. If T includes any used channel, the numerator will be large whether or not X,j s are (nearly) exact. In a sense, the tail size measures how well X and T match the measurements b and expected sparseness. Unless the true number of used channels is known, the tail size appears to be an effective stopping indicator. 2.5 Discussion Complexity In the worst case, algorithm 2.1 reduces the cardinality of T by 1 per iteration, corresponding to recovering at least one additional used channel. Therefore, the number of iterations cannot exceed the number of total channels. However, the first couple of iterations typically recover most of the used channels. At each iteration, the independence recovery step solves up to m optimization problems, which can be independently solved in parallel, so the complexity equals a linear program (or second-order cone program) whose size is no more than n. The worst case complexity is O(n 3 ) but it is almost never observed in sparse optimization thanks to solution sparsity. The two other steps 35

53 are based on basic arithmetic and logical operations, and they run in O(p n). In practice, algorithm 2.1 is implemented and run on a workstation at the fusion center. Computational complexity will not be a bottleneck of the system. As to the matrix completion algorithm, according to [38], FPCA can recover matrices of rank 50 with a relative error of 10 5 in about 3 minutes by sampling only 20 percent of the elements Comparisons between the Two Approaches The matrix completion (Section 2.3) and joint sparsity recovery (Section 2.4) approaches both take linear channel measurements as input and both return the estimates of used channels. On the other hand, the joint sparsity approach takes the full advantage of F, so it is expected to work with smaller numbers of measurements. In addition, even though only one matrix completion problem needs to be solved in the matrix completion approach, it still takes much longer than running the entire joint sparsity recovery, and it is not easy to parallelize any of the existing matrix completion algorithms. However, in the small-scale networks, in cases where too much sensing information is lost during transmission or there are too many active PRs in the network, which increase the signal sparsity level, joint sparsity recovery algorithm with our current settings will experience degradation in performance. We, however, cannot verdict an eventual winner between the two approaches as they are both being studied and improved in the literature. For example, if a much faster matrix completion algorithm is developed which takes advantage of F, the disadvantages of the approach may no longer exist Frequency-Selective Filter Design and Adaptive Sensing The proposed method senses the channels, not by measuring the responses of individual channels one by one, but rather measures a few incoherent linear combinations of all channels responses through onboard frequency-selective filter set. The filter coefficients which perform as the sensing matrix should have entries independently sampled from a sub-gaussian distribution, since this is 36

54 known to be best for compressive sensing in terms of the number of measurements (given in order of magnitude) required for exact recovery. In other words, up to a constant, which is independent of problem dimensions, no other type of matrix is yet known to perform consistently better. However, other types of matrices (such as partial Fourier/DCT matrices [5, 4] and other random circulant matrices [9]) have been either theoretically and/or numerically demonstrated to work as effectively in many cases. These latter sensing matrices are often easier to realize physically or electrically. For example, applying a random circulant matrix performs sub-sampled convolution with a random vector. Frequency-selective surfaces (FSSs) can be used to realize frequency filtering. This can be done by designing a planar periodic structure with a unit element size around half wavelength of the frequency of interests. Both the metallic and dielectric materials can be used. To deal with the bandwidth, unit elements in different shapes will be tested Dynamic CS Update Channel occupancy evolves over time as PRs start and stop using their channels. Channel gains can also change when the PRs move. However, the CS research has so far focused on static signal sensing except the very recent path following algorithms in [73, 74]. In the future work, we can investigate recovery methods for a dynamic wireless environment where based on existing channel occupancy information, an insignificant change of channel states can be quickly and reliably discovered. Given existing channel occupancy X, each new report, which is an entry M i;j of M, is compared with (FX) i;j. If a significant number of such comparisons show differences, then there is a change in the true X. Since X = (RG T ), either R or G, or both, have changed. A change in R means new channel occupation or release. If R is unchanged, then those channel gains in G corresponding to occupied channels have changed. It is easy to deal with the latter case (i.e., G changed, but R didn t) and update the gains of occupied channels because it boils down to solving a small linear system. Let ˆF and ˆX denote the sub-matrices of F and X, respectively, formed by their columns and rows corresponding to the occupied channels. Then, the new gains are given in the 37

55 least-squares solution of M = ˆF ˆX, where M shall include new reports arrived after the previous recovery/update but may still have missing entries. This system is easy to solve since the number of occupied channels is small. In a similar way it is easy to discover released channels as long as there is no introduction of new occupied channels. The release of channel i means row X i of X turns into 0, or small numbers. Therefore, one can solve the system M = ˆF ˆX and find the released channels, which correspond to the rows of ˆX with all zero (or small) entries. When the system M = ˆF ˆX is inconsistent, it means that the received reports cannot be explained by the previously occupied channels, so there must be new channel occupation. Discovering new channel occupation is more difficult since it is to find changes in the previously unoccupied ones, which are much more than the occupied channels. However, it is computationally much easier than starting from scratch. Let X prev and X denote the previous and current channel information, respectively. Arguably, X prev X is highly sparse in the joint sense because only its rows corresponding to newly occupied or released channels can have large nonzero entries. Hence, X can be quickly recovered by performing joint sparsity recovery on X prev X over the constraints M = FX (or a relaxed version in the noisy case), a task that can be done by the algorithms for stationary recovery. 2.6 Simulation Results The Probability of Detection (POD) and False Alarm Rate (FAR) are the two most important indices associated with spectrum sensing. We also consider the Miss Detection Rate (MDR) of the proposed system. The higher the POD, the less interference will the CRs bring to the PRs, while from the CRs perspective, lower FAR will increase their chance of transmission. There is a tradeoff between POD and FAR. While designing the algorithms, we try to balance the CR nodes capability of transmission and their interferences to the PR nodes. Performance is evaluated in terms of POD, FAR and MDR defined as follows: FAR = No. False /(No. False + No. Hit); MDR = No. Miss/(No. Miss + No. Correct); POD = No. Hit/(No. Hit + No. Miss), 38

56 False Alarm Rate Miss Detection Rate False Alarm Rate vs Sampling Rate Sampling Rate Miss Detection Rate vs Sampling Rate no PR=1 no PR=2 no PR=3 no PR=4 no PR=1 no PR=2 no PR=3 no PR= Sampling Rate Figure 2.2: False alarm and missing probability vs. sampling rate. where No. False is the number of false alarms, No. Miss is the number of miss detections, No. Hit is the number of successful detections of primary users, and No. Correct is the number of correct reports of no appearance of PR. We define sampling rate as: No. received measurements at the fusion center, No. channels No. CRs where (No. channel No. CR) is the amount of total sensing workload in traditional spectrum sensing Simulation of Matrix Completion Recovery According to FCC and Defense Advanced Research Projects Agency (DARPA) reports [75, 76] data, we chose to test the proposed matrix completion recovery algorithm for spectrum utilization efficiency over a range from 3% to 12%, which is large enough in practice. Specifically, the number of active primary users is 1 to 4 on a given set of 35 channels with 20 CR nodes. Figure 2.2 shows the false alarm and miss detection rates at different sampling rates for dif- 39

57 POD POD vs Sampling Rate NO. PU=1 NO. PU=2 NO. PU=3 NO. PU= Sampling Rate Figure 2.3: POD vs. sampling rate. ferent numbers of PR nodes. Among all cases, the highest miss detection rate is no more than 5%, and this is from only 20% samples which are supposed to be gathered from the CR nodes regarding all the channels. When the sampling rate is increased to 50% and even when the channel occupancy is relatively high, i.e., 12% of the channels are occupied by the PRs, the miss detection rates can be as low as no more than 2%. From our simulation results, with a moderate channel occupancy at 9%, the false alarm rates are around 3% to 5%. Figure 2.3 shows the probability of detection at different sampling rates. When the spectrum is lightly occupied by the licensed user at 3% channels being occupied, only 20% samples offer a POD close to 100%, and when there is a slightly raise in sampling rate, POD can reach 100%. In the worst case of 12% spectrum occupancy, 20% sampling rate still can offer a POD of higher than 95%, and as the sampling rate reaches 50%, POD can reach 98%. 40

58 1 Noiseless AWGN Channel ncr5 POD vs. Sampling Rate Probability of Detection no PR=1 no PR=4 no PR=5 no PR=6 no PR=7 no PR=8 no PR=9 no PR=10 Miss Detection Rate Sampling Rate MDR vs. Sampling Rate no PR=1 no PR=4 no PR=5 no PR=6 no PR=7 no PR=8 no PR=9 no PR= Sampling Rate Figure 2.4: Noiseless AWGN channel (No. of CR = 5) Joint Sparsity Recovery Simulation Joint sparsity recovery is designed for large scale application, and simulations carried out for a larger dimensional applications with the following settings: We consider a 20-node cognitive radio network within a meter square area centered at the fusion center. The 20 CR nodes are uniformly randomly located. These cognitive radio nodes collaboratively sense the existence of primary users within a meter square area on 500 channels, which are centered also at the fusion center. We chose to test the proposed algorithm for the number of active PR nodes ranging from 1 to 15 on the given set of 500 channels. Since the fading environments of the cognitive radio networks vary, we evaluate the algorithm performance under three typical channel fading models: AWGN channel, Rayleigh fading channel, and lognormal shadowing channel. We first evaluate the POD, FAR, and MDR performance of the proposed joint sparsity recovery performance in the noiseless environment. Figure 2.4, Figure 2.5, and Figure 2.6 show the POD, FAR, and MDR performance at different sampling rate, for AWGN channel, Rayleigh fading channel, and lognormal shadowing channel, respectively, when small number of CR nodes sense 41

59 Probability of Detection Miss Detection Rate Noiseless Rayleigh Fading Channel ncr5 POD vs. Sampling Rate no PR=1 no PR=4 no PR=5 no PR=6 no PR=7 no PR=8 no PR=9 no PR= Sampling Rate MDR vs. Sampling Rate no PR=1 no PR=4 no PR=5 no PR=6 no PR=7 no PR=8 no PR=9 no PR= Sampling Rate Figure 2.5: Noiseless Rayleigh fading channel (No. of CR = 5). Probability of Detection Miss Detection Rate Noiseless Lognormal Shadowing Channel ncr5 POD vs. Sampling Rate no PR=1 no PR=4 no PR=5 no PR=6 no PR=7 no PR=8 no PR=9 no PR= Sampling Rate MDR vs. Sampling Rate no PR=1 no PR=4 no PR=5 no PR=6 no PR=7 no PR=8 no PR=9 no PR= Sampling Rate Figure 2.6: Noiseless log-normal shadowing channel (No. of CR = 5). 42

60 Miss Detection Rate Probability of Detection Noiseless AWGN Channel ncr10 POD vs. Sampling Rate Sampling Rate MDR vs. Sampling Rate no PR=1 no PR=4 no PR=5 no PR=6 no PR=7 no PR=8 no PR=9 no PR=10 no PR=1 no PR=4 no PR=5 no PR=6 no PR=7 no PR=8 no PR=9 no PR= Sampling Rate Figure 2.7: Noiseless AWGN channel (No. of CR = 10). Miss Detection Rate Probability of Detection NOISESSFREE TYPE1 ncr10 POD vs. Sampling Rate no PR=1 no PR=4 no PR=5 no PR=6 no PR=7 no PR=8 no PR=9 no PR= Sampling Rate MDR vs. Sampling Rate no PR=1 no PR=4 no PR=5 no PR=6 no PR=7 no PR=8 no PR=9 no PR= Sampling Rate Figure 2.8: Noiseless Rayleigh fading channel (No. of CR = 10). 43

61 Miss Detection Rate Probability of Detection Noiseless Lognormal Shadowing Channel ncr10 POD vs. Sampling Rate Sampling Rate MDR vs. Sampling Rate no PR=1 no PR=4 no PR=5 no PR=6 no PR=7 no PR=8 no PR=9 no PR=10 no PR=1 no PR=4 no PR=5 no PR=6 no PR=7 no PR=8 no PR=9 no PR= Sampling Rate Figure 2.9: Noiseless log-normal shadowing channel (No. of CR = 10). the spectrum collaboratively. Figure 2.7, Figure 2.8, and Figure 2.9 show the POD, FAR and MDR performance at different sampling rates for the aforementioned three types of channel models when there are more CR nodes involved in the collaborative sensing of the spectrum. We observe that the log-normal shadowing channel model shows the best POD, FAR, and MDR performance no matter how many CR nodes are involved in the spectrum sensing. In contrast, the AGWN channel model shows the worst POD, FAR, and MDR performance. With respect to POD, the performance gap between these two models is at most 10%, which happens when the sampling rate is extremely low. For the Rayleigh fading channel model, when the number of samples is 62% of the total number of channels, for all tested cases we achieved 100% POD. If there are less active PR nodes in the network, a smaller number of samples is required for exact detection. In essence, the proposed CCS system is robust to severe or poorly modeled fading environments. Cooperation among the CR nodes and the robust recovery algorithm allow us to achieve this robustness without imposing stringent requirements on individual radios. We then evaluated the POD, FAR, and MDR performance of the proposed joint sparsity re- 44

62 covery performance in noisy environments. For all the simulations considering noise, we adopted the Rayleigh fading channel model. Figure 2.10 and Figure 2.11 show the corresponding results. We observed that noise does degrade the performance. However, as shown in Figure 2.10, when the number of active PRs is small enough (e.g., number of PR = 1), even with signal-to-noise ratio as low as 15 db, we still can achieve 100% POD with a sampling rate of merely 50%. Then with an increase in the signal-to-noise ratio, lower sampling rate enables more PR nodes to be detected exactly. Figure 2.11 shows the POD, FAR and MDR performance vs. sampling rate at different noise levels. Each curve for a specific noise level is relatively flat (i.e., performance varies a little as sampling rate changes). This shows that the noise level has greater impact on the spectrum sensing performance rather than the sampling rate. At low noise level, e.g., SNR = 45 db, 40% sampling rate enables 100% POD for 4 PR nodes. As SNR reduces to 15 db, no more than 70% POD will be achieved even when the number of samples equals to the number of channels in the network Comparison between Matrix Completion Algorithm and Joint Sparsity Recovery Algorithm For comparison, we applied joint sparsity recovery algorithm on a small-scale network with the same settings as we have used to test the matrix completion recovery. Instead of a 500-channel network, we have a network with only 35 channels. Simulation results show that the joint sparsity recovery algorithm performs better than the matrix completion algorithm in the following aspects: 1. Faster computation due to lower computational complexity; 2. Higher POD for the spectrum utilization rate between 3% and 12% in the noise free simulations. To conclude, matrix completion algorithm is good for small-scale networks, with relatively high spectrum utilization, while joint sparsity recovery algorithm has the advantage of low computational complexity which enables fast computation in large-scale networks. 45

63 Probability of Detection False Alarm Rate NO. Primary User = 5 POD vs. Sampling Rate Sampling Rate Miss Detection Rate FAR vs. Sampling Rate SNR=10 SNR=15 SNR=20 SNR=25 SNR=30 SNR=35 SNR=40 SNR=45 SNR=10 SNR=15 SNR=20 SNR=25 SNR=30 SNR=35 SNR=40 SNR= Sampling Rate 6 x MDR vs. Sampling Rate 10 3 SNR=10 SNR=15 4 SNR=20 SNR=25 SNR=30 2 SNR=35 SNR=40 SNR= Sampling Rate Figure 2.10: POD, FAR, and MDR performance vs. sampling rate at different SNR. 46

64 1 Sampling Rate = 50% POD vs. SNR Probability of Detection False Alarm Rate 0.8 no PR=1 no PR=3 0.6 no PR=5 no PR=7 0.4 no PR=9 no PR= SNR FAR vs. SNR no PR=1 no PR=3 no PR=5 no PR=7 no PR=9 no PR=11 Miss Detection Rate SNR MDR vs. SNR no PR=1 no PR=3 no PR=5 no PR=7 no PR=9 no PR= SNR Figure 2.11: POD, FAR, and MDR performance vs. noise level for different number of PR. 47

65 2.7 Conclusions In this chapter, we apply compressive sensing for collaborative spectrum sensing in cognitive radio networks to reduce the amount of sensing and transmission overhead of cognitive radio (CR) nodes. We propose to equip each CR node with a frequency-selective filter set, which linearly combines multiple channel information, and let it send a small number of such linear combinations to the fusion center, where the channel occupancy information is then decoded. Consequently, the amount of channel sensing at the CRs and the number of reports sent from the CRs to the fusion center reduce significantly. Two novel decoding approaches have been proposed one based on matrix completion and the other based on joint sparsity recovery. The novel matrix completion approach recovers the complete CR to center reports from a small number of valid reports and then reconstructs the channel occupancy information. The joint sparsity approach, on the other hand, skips recovering the reports and directly reconstructs channel occupancy information by exploiting the fact that each occupied channel is observable by multiple CR nodes. Our algorithm enables faster recovery for large-scale cognitive radio networks. The primary user detection performance of the proposed approaches has been evaluated by simulations. The results of random tests show that, in noiseless cases, the number of samples required are no more than 50% of the number of channels in the network to guarantee exact primary user detection for both approaches; while in noisy environments, at low channel occupancy rate, we can still have high probability of detection. 48

66 Chapter 3 High Resolution OFDM Channel Estimation with Low-speed ADC In this chapter, we firstly review the general OFDM system model and set up the channel estimation formulation in Section 3.2. Section 3.3 relates channel estimation to CS and presents the proposed pilot design. In Section 3.4, the estimator based on iterative support detection and limitedsupport least-squares are introduced. Section 3.5 gives the simulation results. Finally, Section 3.6 concludes this work. 3.1 Introduction Orthogonal frequency division multiplexing (OFDM) is a technique that will prevail in the next generation wireless communication and has been widely applied in wireless communication systems, because it transmits at a high rate, achieves high bandwidth efficiency, and is robust to multipath fading and delay [77]. OFDM applications can be found in digital television and audio broadcasting, wireless networking, and broadband internet access. Current OFDM based WLAN standards (such as IEEE802.11a/g) use variations of QAM schemes for sub-carrier modulations which require a coherent detection at the OFDM receiver and consequently requires an accurate (or near accurate) estimation of Channel State Information (CSI). The structure of OFDM signal makes it difficult to balance complexity and performance in channel estimation. The design principles for channel estimators are to reduce the computational complexity and bandwidth overhead while maintaining sufficient estimation accuracy. Some channel estimation schemes proposed in literature are based on pilots, which form the reference signal used by both the transmitter and the receiver. This approach has two main 49

67 challenges: (i) the design of pilots; and (ii) the design of an efficient estimation algorithm (i.e., the estimator). There is a tradeoff between the spectrum efficiency and the channel estimation accuracy. Most of the existing pilot-assisted OFDM channel estimation schemes rely on the use of a large number of pilots to increase to estimation accuracy; the spectral efficiency is therefore reduced. For example, there are approaches based on time-multiplexed pilot, frequency-multiplexed pilot, and scattered pilot [78], all achieving higher estimation accuracy at the price of using more pilots. There have been attempts to reduce the number of pilots, i.e., J. Byun et al., in [79]. The solutions generally require extra test signal for channel pre-estimation. By sending out test signal, they try to find out how many pilots are needed by firstly inserting a relatively small number of pilots and then, based on the results of the test, the number of pilots are decided. Therefore, there is no guaranteed overall reduction of pilots insertion. As a sensing problem, OFDM channel estimation can benefit from the emerging technique of compressive sensing (CS), which acquires and reconstructs a signal from fewer samples than what is dictated by the Nyquist-Shannon sampling theorem, mainly by utilizing the signal s sparse or compressible property. The field has exploded since the pioneering work by Donoho [3] and Candes, Romberg and Tao [80]. The main idea is to encode a sparse signal by taking its incoherent linear projections and recover the signal through algorithms such as l 1 minimization. To maximize the benefits of CS for OFDM channel estimation, one shall skillfully perform the CS encoding and decoding steps, which are precisely the two focuses of this chapter: the designs of pilots and estimator, respectively. Contributions: CS has been applied to channel estimation in [81, 82, 83, 84], which are reviewed in subsection below. For OFDM channel estimation, there are papers [85,86,87,88], to which our work differs in various ways as follows. We skillfully design CS encoding and decoding strategies for OFDM channel estimation. Compared to existing work, we are able to obtain channel response in much higher resolutions and 50

68 from much fewer pilots (thus taking much shorter times). This is achieved by designing pilots with uniform random phases and using a novel estimator. The pilot design preserves the information of high-resolution channel response during aggressive uniform down-sampling, which means that receiver ADC can run at a much lower speed. The estimator is tailored for OFDM channel response; in particular, instead of the generic l 1 minimization, iterative support detection (ISD) [21] and limited-support least-squares are adopted in order to take advantage of the characteristics of channel response. The resulting algorithm is very simple and performs better. 3.2 OFDM System Model A baseband OFDM system is shown in Figure 3.1. In this system, the modulated signal in the frequency domain, represented by X(k), k [1, N], is inserted with pilot signal and guard band, and then an N-point IDFT transforms the signal into the time domain, denoted by x(n), n [1, N], where a cyclic extension of time length T G is added to avoid inter-symbol and inter-subcarrier interferences. The resulting time series data is converted by a digital-to-analog converter (DAC) with a clock speed of 1/T S Hz into an analog signal for transmission. We assume that the channel response comprises P propagation paths, which can be modeled by a time-domain complex baseband vector with P taps, written as P h(n) = α p δ(n τ p T S ), n = 1,..., N, (3.1) p=1 where α p is a complex multipath component and τ p is the multipath delay (0 τ p T S T G ). Since T G is less than the OFDM symbol durations, the nonzero channel response concentrates at the beginning, which translate to h = [h 1, h 2,..., hñ, 0,..., 0], i.e., only the first Ñ components of h can possibly take nonzero values and Ñ < N. Assuming that interferences are eliminated, what arrives at the receiver is the convolution of the transmitted signal and the channel response plus noise, denoted by z(n) given by z = x h + ξ, (3.2) 51

69 Input data with channel coding and modulation Pilot Sequence Insertion Guard Band Insertion Serial to Parallel I D F T Parallel to Serial Cyclic Extension of Time Length TG D/A With Sampling Interval Ts Fading Channel h(n) + AWGN Noise Output Data Pilot Sequence Removal Guard Band Removal Parallel to Serial D F T Serial to Parallel Cyclic Extension Removal A/D Non-uniformaly Spaced Samples Figure 3.1: Baseband OFDM System where denotes convolution and ξ(n), n [1, N] denote the sampled AWGN noise. Passing through the analog-to-digital converter (ADC), z(n), n [1, N] is sampled as y(m), m [1, M], and the cyclic prefix (CP) is removed. Traditional OFDM channel estimation schemes assume M = N. If M < N, then y is a downsample of z. An M-point DFT converts y to Y(k), k [1, M], where the guard band and pilot signal will be removed. For pilot assisted OFDM channel estimation, we shall design the pilots X (and thus x) and recover h from the measurements Y (or, equivalently y). 3.3 Compressive Sensing OFDM Channel Estimation Motivations CS, which will be reviewed in the next subsection, allows sparse signals to be recovered from very few measurements, which translates to slower sampling rates and shorter sensing times. Because the channel impulse response h is very sparse, we are motivated to apply CS to recover h by using a reduced number of pilots so that the estimation becomes much easier. Furthermore, in sharp contrast to conventional OFDM channel estimation in which ADC and DAC run at the same sampling rate, we can obtain a higher-resolution h by increasing the resolution of only the transmitter DAC, or we can reduce the receiver ADC speed which often defines the system cost. In 52

70 other words, we have N > M. The rest of this section reviews CS and introduces our proposed approach for OFDM channel estimation CS Background CS theories [3, 4, 5] state that a S-sparse signal 1 h can be stably recovered from linear measurements y = Φh + ξ, where Φ is a certain matrix with M rows and N columns, M < N, by minimizing the l 1 -norm of h. Classic CS often assumes that Φ, after scaling, satisfies the restricted isometry property (RIP) (1 δ) h 2 2 Φh 2 2 (1 + δ) h 2 2 for all S-sparse h, where δ > 0 is the RIP parameter. The RIP is satisfied with a high probability by a large class of random matrices, e.g., those with entries independently sampled from a sub-gaussian distribution. By minimizing the l 1 -norm, one can stably recover h as long as M O(S log N). The classic random sensing matrices are not admissible in OFDM channel estimation because the channel response h is not directly multiplied by a random matrix; instead, as being described in Section 3.2, h is first convoluted with x, the noise is added, and then the received signal z is uniformly down-sampled to y. Because convolution is a circulant linear operator, we can present this process by y = Ω z = Ω (Ch + ξ), (3.3) where the sensing matrix C is a full circulant (convolution) matrix determined by x, and Ω denotes the uniform down sampling action at points in Ω = [1, 1 + N/M,..., N N/M + 1]. As is widely perceived, CS favors fully random matrices, which admit stable recovery from fewest measurements (in terms of order of magnitude), but C in our case is structured and thus much less random. This factor seemingly suggests that C would unlikely to be favored by CS. Nevertheless, carefully designed circulant matrices can deliver the same optimal CS performance. 1 In our case, S is equal to P, the number of non-zero taps in (3.1). 53

71 3.3.3 Pilot with Random Phases To design the sensing matrix C, we propose to generate pilots X in either one of the following two ways: (i) the real and imaginary parts of X(k) are sampled independently from a Gaussian distribution, k = 1,..., N; (ii) (same as [83]) X(k), k = 1,..., N, have independent random phases but a uniform amplitude. Note that X(k) of type (i) also have independent random phases. Let F denote the discrete Fourier transform. Following from the convolution theorem x h = F 1 (F (x) F (h)) and x = F 1 (X), we have x h = F 1 diag(x)f h, so the measurements y can be written as y = Ω (Ch + ξ) = Ω ( F 1 diag(x)f h + ξ ). (3.4) Let us explain intuitively why (3.4) is an effective encoding scheme for a sparse vector h. First, it is commonly known that F h is non-sparse and its mass is somewhat evenly spread over all its components. The random phases of X by design are of critical importance. They scramble F h component wisely and break the delicate relationships among F h s components; as a result, in contrast to the sparse F 1 F h = h, F 1 diag(x)f h is not sparse at all. Furthermore, X has a random-spreading effect. Due to a phenomenon called concentration of measures [89], the mass of Ch spreads over its components in a way that, with a high probability, the information of h is preserved by down sampling of a size essentially linear in P the sparsity of h (whether or not the down-sampling is equally spaced, i.e., uniform). Up to a log factor, the down sampled measurements permit stable l 1 recovery. Both types (i) and (ii) of X have similar encoding strength, but X of type (ii) gives an orthogonal C, i.e., C C = I, so x h transforms h into a random ortho-basis. Such orthogonality results in multiple benefits such as faster convergence of our recovery algorithm. Due to the page limitation, we omit rigorous mathematical analysis of (3.4) and its guaranteed recovery. Note that the proposed sampling Ω (F 1 diag(x)f ) is very different from partial Fourier sampling Ω F. The latter requires a random Ω to avoid the aliasing artifacts in the recovery but the former, with random-phased X, permits both random and uniform Ω. Below we numerically 54

72 10 1 MSE Vs. No. of Multipath for Different Cases (SNR=30dB) MSE 10 2 Gaussian random complex, l1 minimization Random circulant complex w/ uniform magnitude, l1 minimization Random circulant complex, l1 minimization Random circulant complex w/ uniform magnitude, CS OFDM Random circulant complex, CS OFDM No. of Multipath Figure 3.2: MSE vs. No. of multipath for different cases (SNR = 30 db). demonstrate its encoding efficiency Numerical Evidence of Effective Random Convolution CS performance is measured by the number of measurements required for stable recovery. To compare the proposed sensing schemes with the well-established Gaussian random sensing, we conduct numerical simulations and show its results in Figure 3.2. We compare three types of CS encoding matrices: the i.i.d. Gaussian random complex matrix, and the circulant random complex matrices corresponding to X of types (i) and (ii) above, respectively. In addition, l 1 minimization is compared to our proposed algorithm CS-OFDM, which is detailed in the next section. The simulations results show that the random convolutions of both types perform as well as the Gaussian random sensing matrix under l 1 minimization, and our algorithm CS-OFDM further improves the performance by half of a magnitude. 55

73 3.3.5 Relationship to Existing Results Random Convolution CS Random convolution has been used and proved to be an effective way of taking CS measurements that allow the signal to be recovered using l 1 minimization. In [81], Toeplitz 2 measurement matrices are constructed with i.i.d random row 1 (the same as type (i)) but with only ±1 or { 1, 0, 1}; their down sampling effectively takes the first M rows; and the number of measurements needed for stable l 1 recovery is shown as M O(S 3 log N/S). [82] uses a partial Toeplitz matrix, with i.i.d. Bernoulli or Gaussian row 1, for sparse channel estimation where the down sampling effectively also takes the first M rows. Their scheme requires M O(S 2 log N) for stable l 1 recovery. In [83], random convolution of type (ii) above with either random downsampling or random demodulation is proposed and studied. It is shown that the resulting measurement matrix is incoherent with any given sparse basis with a high probability and l 1 recovery is stable given M O(S log N + log 3 N). Our proposed type (ii) is motivated by [83]. On the other hand, no existing work proposes uniform down-sampling or shows its recovery guarantees. In addition, most existing analysis is limited to real-valued matrices and signals CS-based Channel Estimation Our work is closely related to [82] and [84]. In [82], i.i.d. Bernoulli or Gaussian vector is used as training sequence, and downsample is carried out by taking only the first M rows. While channel estimation is obtained as a solution to the Dantzig selector. In [84], MIMO channels are estimated by activating all sources simultaneously. The receivers measure the cumulative response, which consists of random convolutions between multiple pairs of source signals and channel responses. Their goal is to reduce the channel estimation time. l 1 minimization is used to recover channel response. Our current work is limited to estimating a signal h-vector. While our work is based on similar 2 which is slightly more general than circulant. 56

74 random convolution techniques, we have proposed to use a pair of high-speed source and low-speed receiver for the novel goal of high resolution channel estimation. Furthermore, we apply a novel algorithm for the channel response recovery based on iterative support detection and limited-support least-squares, which is described in details in Section 3.4 below. 3.4 Numerical Algorithm Problem Formulation As a result of rapid decaying of wireless channels, P the number of significant multipaths is small, so the channel response h is a highly sparse signal. Recall that the non-zero components of h only appear in the first Ñ components. We shall recover a sparse high-resolution signal h with a constraint from the measurements y at a lower resolution of M. We define operation as the amplitude of a complex number, h 0 as the total number of nonzeros of h and h 1 = i h i. The corresponding model is min h 0, s.t. ϕh = y and h i = 0 i > Ñ, (3.5) h R N where ϕ denotes Ω C in (3.4), the submatrix of C formed by its rows corresponding to the downsampling points in Ω. Generally speaking, problem (3.5) is NP-hard and is impossible to solve even for moderate N. A common alternative is its l 1 relaxation model with the same constraints. min h 1, s.t. ϕh = y and h i = 0 i > Ñ, (3.6) h R N which is convex and has polynomial time algorithms. If y has no noise, both (3.5) and (3.6) can recover h exactly given enough measurements, but (3.6) requires more measurements than (3.5) Algorithm Instead of using a generic algorithm for (3.6), we design an algorithm to exploit the OFDM system features, including the special structure of h and noisy measurements y. At the same time, we maintain its simplicity to achieve low complexity and match with easy hardware implementation. 57

75 First of all, we can simply collaborate two constraints into one by letting the variables be h = [h 1, h 2,..., hñ] and dropping the rest components of h. Let ϕ be the matrix formed by first Ñ columns of ϕ. Hence, the only constraints are ϕ h = y, which reduces the size of our problem. We also develop our algorithm CS-OFDM for the purpose of handling noisy measurements. The iterative support detection (ISD) scheme proposed in [21] has a very good performance for solving (3.6) even with noisy measurements. Our algorithm uses the ISD, as well as a final denoising step. In the main iterative loop, it estimates a support set I from the current reconstruction and reconstructs a new candidate solution by solving the minimization problem min{ i I c h i : ϕ h = y}, and it iterates these two steps for a small number of iterations. The idea of iteratively updating the index set I helps catch missing spikes and erase fake spikes. This is an l 1 -based method but outperforms l 1. Analysis and numerical performance of ISD can be found in [90]. Because the measurements have noise, so reconstruction is never exact. Our algorithm uses a final denoising step, which solves least-squares over the final support T, to eliminate tiny spikes likely due to noise. Our pseudo code is listed in Algorithm 3.1. Algorithm 3.1 CS-OFDM Input: ϕ, y; Initalize: ϕ as the first Ñ columns of ϕ. I 0 and wi 0 = 1, i {1, 2,..., Ñ} while the stopping condition is not met, do Subproblem: h arg min h i, s.t. ϕ h = y. (3.7) i I j Support detection:i j+1 {i : h j i 2 j h j }, where h j = max i { h j i }. Weights update: w j+1 i 0, i I j+1 ; otherwise w j+1 i 1. j j + 1 end while Final least-squares: let T = {i : h i > threshold}, then: Return h h T arg min ϕ T h y 2 2, and h T c 0. (3.8) h In Algorithm 3.1, at each iteration j, (3.7) solves a weighted l 1 problem, and the solution 58

76 h j is used for support detection to generate a new I j+1. After the main loop is done, a support T is estimated above a threshold, which is selected based on empirical experiences. If the support detection is executed successfully, T would be the set of all channel multipath delay. Finally, h is constructed by solving a small least-squares problem, and h i, i T fall to zero Complexity Analysis This algorithm is efficient since every step is simple and the total number of iterations needed is small. The subproblem is a standard weighted l 1 minimization problem, which can be solved by various l 1 solvers. Since ϕ is a convolution operator, we choose YALL1 [91] since (i) it allows us to customize the operators involving ϕ and its adjoint to take advantages of DFTs, making it easier to implement the algorithm on hardware, (ii) YALL1 is asymptotically geometrically convergent and efficient even when the measurements are noisy. With our customization, all YALL1 operations are either an DFT/IDFT or one dimensional vector operations, so the overall complexity is O(N log N). Moreover, for support detection, we run YALL1 with a more forgiving stopping tolerance and always restart it from the last step solution. Furthermore, YALL1 converges faster as the index I j gets closer to the true support. The total number of YALL1 calls is also small since the detect support threshold decays exponentially and bounded below by a positive number. Numerical experience shows that the total number of YALL1 calls never exceeds P. The computational cost of the final least-squares step is negligible because the associated matrix ϕ T has its number of columns approximately equal to the number of spikes in h, which is far less than its rows. For example, if the system has P multipaths, the associated matrix for leastsquares has size M P. Generally speaking, the complexity for this least-squares is O(MP +P 3 ). Since P and M are much smaller than N, the complexity of the entire algorithm is dominated by that of YALL1, which O(N log N). 59

77 3 2.5 Multipath Delay Profile (SNR =3 0 db) True Recovered 2 Megnitude Time Figure 3.3: Multipath delay profile. 60

78 3.4.4 Cramér-Rao Lower Bound The Cramér-Rao Lower Bound (CRLB) is an indicator of how good an unbiased estimator performs. In this subsection we derive the CRLB for pilot assisted OFDM channel estimation. The CRLB for each entry of y is CRLB(h i ) = (I 1 (h)) ii, 0 i M 1, (3.9) where I(h) is the Fisher information matrix, written as I(h) = E{ log f(y h)( h h log f(y h)) }. (3.10) In equation (3.10), E denotes the expectation, and f(y h) is the conditional PDF of y given h. Recall that the OFDM system model can be written as y = ϕh + ξ, (3.11) where ϕ denotes Ω C in (3.4) and ξ is the AWGN noise with σ variance and zero mean. Following equation (3.11), we can derive the conditional PDF of y given h f(y h) = 1 (πσ 2 ) exp{ 1 σ 2 y ϕh 2 }. (3.12) Then the partial derivative of the logarithm of equation (3.12) with respect to h is h log f(y h)) = 1 σ 2 [y h ϕ ] T. (3.13) Substitute equation (3.13) into equation (3.10), the Fisher information matrix is I(h) = 1 σ 2 ϕ ϕ. (3.14) Since the uniform down-sample operation Ω C does not affect the orthogonality, since C C = I we have ϕ ϕ = I, where I is a M M identity matrix. Then the Fisher information matrix in equation (3.14) can be written as I(h) = 1 I. (3.15) σ2 61

79 The CRLB for the i th entry of y is I 1 (h) ii. Then the overall CRLB is CRLB(h) = M CRLB(h i ) = trace(i 1 (h)). (3.16) i=1 3.5 Numerical Simulations In this section, we performed numerical simulations to illustrate the performance of the proposed CS-OFDM algorithm for high resolution OFDM channel estimation. We focused on the mean square error (MSE) of channel estimation as well as the multipath delay detection when channel profile and signal to noise ratio (SNR) changes Simulation Settings We considered an OFDM system with 1k-point IDFT (N = 1024) at the transmitter and 64- point DFT (M = 64) at the receiver, where the compression ratio is 16. The number of silent subcarrier which acts as guard band is 256 among 1024 sub-carriers. The channel is estimated based on 768 pilot tones with uniformly random phases and a unit amplitude, with the Gaussian noise level ranging from 10 db to 30 db. We assume the usage of cyclic prefix and the impulse response of the channel is shorter than cyclic prefix which means there is no inter-symbol interference. For all simulations, we test the total numbers of multipath from 5 to 15. Moreover, we use only one OFDM symbol, i.e., use all non-silent subcarriers only once to carry pilot signals. All reported performances will substantially improve if more pilots are inserted MSE Performance Figure 3.3 is a snapshot of one channel estimation simulation. It suggests that the proposed pilot arrangement and CS-OFDM successfully detect an OFDM channel with 7 multipaths when the signal to noise ratio is 30 db. Our method not only exactly estimates the multipath delays, but also correctly estimates the values of the corresponding multipath components. 62

80 10 0 MSE Vs. No. of Different SNR 10 1 MSE 10 2 SNR=10 db SNR=20 db SNR=30 db No. of Multipath Figure 3.4: MSE performance. Figure 3.4 depicts the MSE performance on OFDM channels with the number of multipaths ranging from 5 to 15 and noise level ranging from 10 db to 30 db. When there are only a moderate number of multipaths on the OFDM channel, e.g., 10 multipaths, even when SNR is 20 db, MSE is as low as 17 db. Figure 3.5 shows the reconstructed SNR vs. the number of multipaths when the input SNR changes. We can see that CS-OFDM achieves a gain in SNR. For example, when the input SNR is 10 db, we obtain a reconstructed SNR higher than 20 db when there are 5 multipaths. As the number of multipaths increases, the SNR gain from the reconstructed signal to the input signal decreases. However, even when the number of multipaths is 10, we still have a 5 db gain, e.g., reconstructed SNR is 15 db when the input signal SNR is 10 db. The similar SNR gain appears for input SNR= 20 db and SNR= 30 db cases. From the entire input SNR and the number of multipath range we have tested, there is an average gain of 6 db from the input SNR to the recovered SNR. 63

81 45 Recovery SNR Vs. No. of Different SNR Recovery SNR SNR=10 db SNR=20 db SNR=30 db No. of Multipath Figure 3.5: Reconstructed SNR. 64

82 10 3 CR Recovered Channel Variance Vs. CRLB CRLB No. of multipath = 4 CRLB No. of multipath = 8 CRLB No. of multipath = 12 CS No. of multipath = 4 CS No. of multipath = 8 CS No. of multipath = Signal Variance SNR Figure 3.6: CS recovered channel variance vs. CRLB CRLB Performance Figure 3.6 shows the CS estimated channel variance Vs. CRLB with the change of SNR and number of multipath. The gap between CS estimated channel variance and the CRLB is relatively small at low SNR and small number of multipath. For example, when SNR = 5 db, and number of multipath equals to 4, there is only 4 db performance gap. While as SNR increases, at the same number of multipath, when SNR increases to 29 db, performance loss increases to 22 db. Besides, the performance gap increases with the increase of number of multipath. For instance, when SNR = 5 db, as number of multipath increases from 4 to 12, the performance gap increases from 4 db to 9 db. 65

83 1 Probability of Detection Vs. No. of Different SNR Probability of Detection SNR=10 db SNR=20 db SNR=30 db Noise free No. of Multipath Figure 3.7: Probability of support detection Multipath Delay Detection Performance Figure 3.7 and Figure 3.8 depict the probability of correct detection (POD) of the multipath delay and the false detection rate (FAR) while we change the SNR and the number of multipaths. When the SNR is above 10 db, simulation shows almost 100% POD when the number of multipaths changes from 5 to 12. When there are a relatively large number of multipaths, e.g., 15, the probability of correct multipath delay detection is higher than 95% as SNR 10 db. Even when SNR is low, as long as the number of multipaths does not exceed 10, we still have a POD of greater than 95%. The FAR performance shows the similar results, as the SNR decreases and the number of multipaths increases, performance decreases. But, in a large range, e.g., SNR 10 db, the number of multipath 10, we have almost zero FAR. 66

84 False Alarm Rate Vs. No. of Different SNR SNR=10 db SNR=20 db SNR=30 db Noise free False Alarm Rate No. of Multipath Figure 3.8: Probability of false support detection. 67

85 3.6 Conclusions Efficient OFDM channel estimation will drive OFDM to carry the future of wireless networking. A great opportunity for high-efficiency OFDM channel estimation is lent by the sparse nature of channel response. Riding on the recent development of CS, we propose, in this chapter, a design of probing pilots with random phases, which preserves the information of channel response during the convolution and down-sampling processes, and a sparse recovery algorithm, which returns the channel response in high SNR. These benefits translate to the high resolution of channel estimation, the lower speed of the receiver ADC, as well as shorter probing times. In this chapter, the presentation is limited to an idealized OFDM model, intuitive explanations, and simulated experiments. In the future, we will formalize the work with rigorous theorems and fuse it into more realistic OFDM frameworks. The results presented here hint a high efficiency improvement for OFMD in practice. 68

86 Chapter 4 Sampling Rate Reduction for 60 GHz UWB Communication In this chapter, we first review briefly the 60 GHz communication and the challenges in this area. Then, in Section 4.2 we present the low-speed A/D conversion system model for 60 GHz UWB communication system. Section 4.3 explains prototype implementation of the proposed ADC using compressive sensing framework, and investigate the performance of RAKE BPSK receiver. Simulation results are presented and analyzed in Section 4.4. Finally, conclusions are drawn in Section Introduction The dogma of signal processing maintains that a signal must be sampled at a frequency, at least twice its bandwidth in order to be represented without error. However, in practice, we often compress the data soon after sensing, trading off signal representation complexity (bits) for some error (consider JPEG image compression in digital cameras, for example). Clearly, this is wasteful of valuable sensing/sampling resources. Over the past few years, a new theory of compressive sensing [4, 5, 3] has begun to emerge, according to which the signal is sampled (and simultaneously compressed) at a greatly reduced rate if the input signal has the sparse property. Very recently, compressive sensing has been used in several wireless communication and networking applications [92, 62, 64]. 60 GHz communication is an emerging technology for high-speed short-range wireless communications. Recently, there has been an increasing interest in providing broadband communications in the 60 GHz unlicensed band for wireless personal area networks (WPANs). A wide 69

87 available spectrum, e.g., GHz for North American services, surrounding the 60 GHz operating frequency has the ability to support high-rate broadband wireless systems such as voice, data, and full-motion real-time video for home entertainment applications [93]. Due to special propagation characteristics at this frequency range multipath effects still exist for indoor applications and they must be precisely characterized for diversity transmission purposes. Ultra wide-band (UWB) signals are much less sensitive to these effects [93]. Hence, to take the benefits of both 60 GHz and UWB communications we up-covert the UWB signal by 60 GHz carrier. For this system, socalled 60 GHz UWB, the requirement of high-speed sampling significantly increases the cost of RF circuitry such as the analog-to-digital converter (ADC) at the receiver. In this paper, we propose to utilize the concept of compressive sensing to achieve a significant reduction of sampling rate. The basic idea is based on the observation that the received signals are sparse in the time domain due to the limited multipath effects at 60 GHz wireless transmission. The low information rate enables us to employ much lower sampling rate while still be able to reconstruct the signal with a low probability of error. We describe an implementation for a lowspeed A/D converter for 60 GHz UWB received signal. We also analyze the bit error rate (BER) performance for BPSK under RAKE reception. Simulation results show that, the sampling rate can be reduced to only 2.2% for the signal with sparsity of 1%, and the BER performance loss is only 0.3 db. 4.2 System Model We consider a system with the transmission signal s(t) and the received signal written as K x(t) = h k s(t τ k ) + ϵ(t), (4.1) k=1 where K is the number of multipaths, ϵ(t) is the thermal noise. h k is the k th multipath response, and τ k is its delay. There are two distinct characteristics for 60 GHz UWB transmission. First, due to the narrow impulse in the time domain, the receiver needs high sampling rate. Second, because of 70

88 Figure 4.1: Compressive sensing-based ADC. the line-of-sight transmission and high attenuation due to reflection, there are few multipaths (with sufficiently high power). As a result, the number of multipaths K is much smaller than the sampling space. This signal sparsity motivates and enables us to utilize a compressive sensing framework. In the 60 GHz UWB system, the transmitted pulse is an up-converted UWB pulse at 60 GHz carrier. An accurate ray-tracing model is used to simulate the multipath propagation channel. It was confirmed that at 60 GHz frequency, the channel characterization results obtained by accurate raytracing method are quite reliable and can be verified by measured data [94, 95]. Since in the indoor applications at 60 GHz, the high penetration loss of the construction material isolates adjacent rooms and significantly limits the received interference, the ray-tracing simulation needs to be applied only for the objects inside the room. In this simulation, we consider a typical furnished office room as a propagation environment, shown in Figure 4.2. In this model, a patch antenna transmitter is mounted a few centimeters off the ceiling at the center and beam facing down. The patch antenna receiver is located right on the center of the meeting table and facing upward. We consider the antenna pattern as a part of the channel. The transmitter has 10 dbm transmit power. The transmitter/receiver antenna gain is set to 7 db. All the materials in the room set for 60 GHz frequency in terms of complex permittivity. 71

89 The 60 GHz UWB pulses received by the receiver antenna are then amplified using a wideband low noise amplifier and processed via a compressive sensing receiver described in the next section. 4.3 Compressive Sensing-Based ADC In this section, we first propose an implementation for a low-speed A/D converter for 60 GHz UWB received signal. Then we formulate the compressive sensing problem, and investigate how to use the a priori information for signal recovery at the receiver based on the compressive sensing model. Finally, the BER performance of BPSK is analyzed under RAKE reception Proposed ADC Viewed as composition of a discrete, finite number of weighted continuous basis or dictionary components, described in N x(t) = α n δ n (t), (4.2) n=1 multipath received signal is sparse in the time domain. Due to the line-of-sight transmission and high attenuation due to reflection, there are few multipaths with sufficient high power. In Figure 4.3, we show a multipath power delay profile obtained using the accurate ray-tracing modeling. When the noise floor is assumed to be 100 dbm, as can be observed from the figure, only a few multipaths exist. The average delay for indoor environment is 100 ns, and the sampling rate for 60 GHz UWB can be 100 ps. As a result, in a time frame, we have a sampling space of N = 1000, within which only a small number of significant multipath coefficients exist. Obviously X is sparse. The proposed system, shown in Figure 4.1, is composed of three major parts: 1) The mixer, in which, the multipath received signal is modulated by a set of pseudo-random (PN) Bernoulli sequence, and the alternation between values in one sequence is at or faster than the Nyquist rate of 72

90 Figure 4.2: Simulated environment in ray-tracing. the input signal; 2) The integrator, which integrate the mixer output throughout each time frame; 3) Multiple copies of mixer and integrator chain, with total number of copies needed M decided by the signal sparsity level. Usually, 3 5 times of the number of significant component in a time frame is enough for exact signal recovery. After these three parts, a low-speed ADC is implemented, its input simply switches among the M copies of mixer and integrator chain, and thus the A/D conversion speed decreases to M per time frame. We introduce a delay line after the PN Bernoulli sequence generator. Each Pi (t) shown in Figure 4.1 is a row vector with Bernoulli i.i.d. entries, and by delaying the output of the PN Bernoulli sequence generator, P2 (t),, Pm (t) are time shifted copies of P1 (t). These time shifted copies used for signal modulation not only simplify the PN generation part, but also help save the storage space in the CS recovery part. Besides, we introduce sample and hold at the integrator output which can hold the integrator output at the end of each time frame for the ADC switch to arrive. In such a way, only one ADC is needed for M chains of mixer and integrator. 73

91 50 60 Magnitude (dbm) Noise Floor Delay (ns) Figure 4.3: A simulated power delay profile for 60 GHz UWB transmission. Output of the ADC is a matrix Y, which is given by Y M 1 = P M N X N 1. (4.3) Each column vector with length M holds the quantized integrator output within a time frame from M copies of the compressive sensing chain. Each row vector is the quantized integrator output throughout the time. At the end of each time frame, one column will be taken out, together with the PN matrix P for compressive sensing signal reconstruction Formulation of the Compressive Sensing Problem The problem is to obtain the K significant components of X using the limited number (M) of measurements. The first question is whether or not the information of K-sparse signal is damaged by the reduction of dimension from X R N down to Y R M. In general, if X is not sparse enough, as long as M < N, the signal is damaged since there are fewer equations than unknowns. On the other hand, for the K-sparse signal, Y is just a linear combination of K columns of P. 74

92 A necessary and sufficient condition to ensure that this M K system can be compressed and reconstructed is stated as the following property: Definition 4.1. Restricted Isometry Property (RIP) [4, 5]: For any vector V sharing the same K nonzero entries as X, if 1 ϵ PV 2 V ϵ (4.4) for some ϵ > 0, then the matrix P preserves the information of the K-sparse signal. A sufficient condition for stable inverse in practice is that P satisfies (4.1) for an arbitrary 3K-sparse vector V. In [4,5], it was proved that if P is a matrix with random ±1 entries, then the K-sparse signal is compressible with high probability if K cm log(n/m), where c is a constant. It was later proved that Toeplitz-structured matrices also satisfy the RIP condition. For our problem, we design P in such a way that, the first row of P is a PN Bernoulli sequence, and the rows after are time shifted copies of row one, therefore it is a Toeplitz matrix with Bernoulli i.i.d entries which satisfies the RIP condition. The RIP condition tells us that, under certain conditions the K significant entries in the N- dimensional signal are well preserved in the M-dimensional measurement vector. The next question is how to develop a reconstruction algorithm to recover X from the measured Y. Since M < N there are infinite number of ˆX satisfying Y = P ˆX. All solutions lie on the N M dimension hyperplane H := N (P)+X which corresponds to the null space N (P) translated to X. Therefore, the problem is to find the sparse reconstructed signal ˆX in the translated null space as ˆX = arg min ˆX 1, (4.5) Y=P ˆX where 1 is norm one. It was shown in [96] that for norm two, there might be many solutions, and for norm zero, the complexity is NP hard [4, 5, 3]. The problem in equation (4.5) is a convex optimization problem, which can be solved via the l 1 -magic solver with computational complexity O(N 3 ). Besides, other simpler algorithms such as the simplex algorithm [97, 98] can be easily 75

93 employed Bayesian Detection However, to directly solve the problem in equation (4.5) using the aforementioned l 1 -minimization algorithms, the performance may not be good. This is because those solutions do not utilize the prior information about the signal. We propose to adopt the Bayesian compressive sensing framework [99, 100], which is fully probabilistic. The Bayesian framework introduces a set of hyperparameters which is viewed as a priori probability of the signal, and the most probable values are iteratively estimated from the received data Model Specification In equation (4.1), the noise in the system is composed of Gaussian random propagation loss with zero mean and variance σ 2. The probability density function thus can be approximated as a Gaussian distribution written as M p(ϵ) = N (ϵ i 0, σ 2 ). (4.6) i=1 Due to the assumption of independence of Y n, the likelihood of the complete data set can be written as ( p(y P, σ 2 ) = (2πσ 2 ) M/2 exp 1 ) Y PX 2. (4.7) 2σ2 By adopting a Bayesian perspective, we constrain the parameters by defining an explicit a priori probability distribution over them. Here, we assume a zero-mean Gaussian a priori distribution over the signal X 1 p(x α) = N n=1 N (X i 0, α 1 i ), (4.8) where α is a vector of N independent hyper-parameters. 1 Even though X is not Gaussian distributed, simulation results show that the performance indeed improves. 76

94 Given α, the posterior parameter distribution conditioned over the signal is given by combining the likelihood and prior with Bayes rule p(x Y, α, σ 2 ) = p(y X, σ2 )p(x α) p(y α, σ 2, (4.9) ) which is a Gaussian distribution N (µ, Σ) with covariance and mean of Σ = (A + σ 2 P T P) 1, and µ = σ 2 ΣP T Y, respectively, where A = diag(α 1,..., α N ) Marginal Likelihood Maximization A most-probable point estimate α MP may be found via a type-ii maximum likelihood procedure [101]. The sparse Bayesian model is formulated as the local maximization with respect to α of the marginal likelihood, or equivalently its logarithm L(α) = log p(y α, σ 2 ) (4.10) ˆ = log p(y X, σ 2 )p(x α)dx = 1 2 ( M log 2π + log C + Y T C 1 Y ), with C = σ 2 + I + PA 1 P T. A point estimate µ MP for the parameters is then obtained by evaluating µ with α = α MP, giving a posterior mean approximator PX = Pµ MP. However, marginal likelihoods are generally difficult to compute, i.e., values of α and σ 2 which maximize L(α) cannot be obtained in closed form. Thus, we need to re-estimate them iteratively. For updating α, following the approach in [101], we differentiate equation (4.10), and then equate it to 0. After rearranging, we have α new i = γi, where µ µ 2 i is the i th posterior mean signal µ, i and γ i is defined as γ i = 1 α i N ii with N ii being the i th diagonal element of the posterior signal covariance Σ computed with current α and σ 2 values. Each γ i can be treated as a measure of how well-determined its corresponding parameter X i is by the data. For the variance σ 2, differentiation 77

95 leads to re-estimate σ 2 new = Y Pµ 2 M i γ. (4.11) i We repeat calculation of α and σ 2 with iteratively updating µ and Σ until certain convergence criteria have been reached. This procedure leads to the maximization of marginal likelihood. Then at the convergence of α estimation procedure, we make predictions based on the posterior distribution over the signal, conditioned on the maximizing values α MP and σmp 2. In other words, by doing this, we could pick up those entries in the projection matrix P which after projection preserves the information of the signal in Y. By utilizing the corresponding elements in the measurements Y and projection matrix P, we could reconstruct our signal with a better probability BER Analysis for RAKE Receiver A RAKE receiver is a radio receiver designed to counter the effects of multipath fading, using several several correlators (fingers) each assigned to a different multipath component. Each finger independently decodes a single multipath component, and then the contribution of all fingers are combined in order to make the most use of the different transmission characteristics of each transmission path. Rake receivers are common in a wide variety devices such as mobile phones and wireless LAN equipments. In this subsection, we study the BER performance of BPSK under RAKE reception. For traditional high data rate ADC, the received SNR under RAKE reception can be written as Γ = i N X i 2 N 0, (4.12) where N is the set of multipath components with sufficiently high power. For the proposed ADC, after reconstruction, we have ˆX i. Therefore, the reconstructed SNR is given by ˆΓ = X i ˆX i, (4.13) N 0 i N 78

96 Figure 4.4: Example of data flow on the hardware. where N is the set of multipath components found by the reconstructed algorithm. If BPSK is employed, the BER is given by BER= Q ( 2ˆΓ ). 4.4 Simulation Results and Analysis To verify the effectiveness and the efficiency of our design, diverse types of simulations are carried out. We firstly simulate the data flow on the hardware, shown in Figure 4.4. PN Bernoulli sequence we used to modulate multipath received signal is shown at the top, the modulated signal is shown in the middle, and compressive sensing received signal at the low-speed ADC within a time frame is shown at the bottom. We also test the proposed Bayesian frame for CS signal recovery and one example of exact signal recovery is shown in Figure 4.5. In Figure 4.6, we show the covariance between the original signal and the reconstructed signal by compressive sensing under different SNR. Figure shows that in the high SNR regime, the correlation is high and the sampling rate can be reduced to 1.6% without excessive performance loss when compared with higher sampling rate results. When the SNR is low, by increasing sampling rate we can still achieve very good performance. When the sampling rate is around 0.4%, the whole 79

97 Figure 4.5: Original signal vs. CS recovered signal. 1 cov(x,x * ) vs SNR cov(x,x * ) No. of delay line=4 (sampling rate=0.4%) No. of delay line=16 (sampling rate=1.6%) No. of delay line=22 (sampling rate=2.2%) No. of delay line=40 (sampling rate=4%) SNR (db) Figure 4.6: Covariance between the original signal and reconstructed signal. 80

98 10 0 BER vs SNR BER No. of delay line=4 (sampling rate=0.4%) No. of delay line=16 (sampling rate=1.6%) No. of delay line=22 (sampling rate=2.2%) No. of delay line=40 (sampling rate=4%) Fully sampled SNR (db) Figure 4.7: BER performance under RAKE reception. 81

99 system collapses. Clearly, a tradeoff exists for M as a function of K and N. In Figure 4.7, we show the BER performance of BPSK under RAKE reception. For comparison, we show the BER performance of the proposed scheme. We can see that with a sampling rate of 4%, the proposed scheme almost has the optimal performance. When the sampling rate is reduced to 2.2%, in low SNR regime, the performance degrades. For sampling rate equal to 1.6%, the performance degrades even for high SNR. Again, when the sampling rate is 0.4%, the system collapses. 4.5 Conclusions To reduce the sampling rate while still achieving the performance of high-speed ADC, we propose a compressive sensing-based system utilizing one low-speed ADC in chapter 4. Based on the sparse property of the multipath received signal, a new structure of low-speed sampling has been proposed and a compressive sensing problem has been formulated. To efficiently solve the problem, a Bayesian solution has been investigated. Simulation results have revealed that the BER of BPSK RAKE receiver has similar performance (0.3 db loss) as the high speed one with the sampling rate as low as 2.2% in case the multipath received signal is sparse enough. 82

100 Chapter 5 Sparse Events Detection in Wireless Sensor Networks In Section 5.1, we introduce the scenario of wireless sensor networks and the sensing problem lies in it. Then, we present in Section 5.2 the proposed compressive sensing system model. Section 5.3 formulates the CS problem. Section 5.4 conducts analysis and exposes the proposed Bayesian detection algorithm under compressive sensing framework. Simulation results are presented and analyzed in Section 5.5. Finally, we conclude this work in Section Introduction A typical WSN consists of spatially distributed autonomous sensors to cooperatively monitor physical or environmental conditions, such as temperature, sound, vibration, pressure, motion or pollutants [102, 103]. Most WSN involve large number of sensor nodes distributed across a large area. As individual nodes in such a network are often battery-operated, power consumption is a limiting factor, and the reduction of communication costs is crucial. As a result, only when there is a bursty event in an area, sensor nodes will be activated accordingly, and the number of active sensor nodes at a given time is very limited. Consequently, the sensor status signal for the entire sensor network is a time and spatial sparse signal. In this chapter, we investigate how to employ compressive sensing for sparse events detection in WSN. Specifically, we focus on solving two problems in WSN: (1) Sensor activities are sparse which makes it a waste of resources to monitor each one of them individually at all times; (2) Several sensor activities may happen simultaneously and form a superimposed sensing result, it is difficult but critical to identify individual signal. 83

101 Aiming at solving the aforementioned problems, we propose a sparse event detection scheme in WSN employing compressive sensing framework. Our contributions are listed as follows: 1. We take advantage of the sparse nature of wireless sensor networks to formulate the compressive sensing problem, which is known to be able to dramatically reduce sampling rate while guarantee exact signal recovery with high probability. 2. We employ the Bayesian framework and a heuristic algorithm for signal recovery. By using the prior information that the events are binary, we managed to increase the estimation probability substantially compared to the widely used l 1 -magic algorithm in the literature [104]. 3. We conduct simulations to investigate the effects of the noise, since most compressive sensing schemes suffer susceptibility under unbounded Gaussian noise. We show that the performance decays with the signal to noise ratio (SNR) approaching 20 db. 5.2 System Model We consider the system model as shown in Figure 5.1. There are N wireless sensor nodes randomly located in a field. At a given time, K out of the N s X N 1 denotes the event vector, in which each component has a binary value, i.e., X n {0, 1}. Obviously X is a sparse vector since K N. In the system, there are M active monitoring sensors trying to capture these events. There are two challenges for those monitoring sensors. First, all those events happen simultaneously. As a result, the received signals are interfering with each other. Second, the received signal is deteriorated by propagation loss and thermal noise. The received signal vector can be written as Y M 1 = G M N X N 1 + ϵ M 1, (5.1) where ϵ M 1 is the thermal noise vector whose component is independent and has zero mean and variance of σ 2. G M N is the channel response matrix whose component can be written as G m,n = (d m,n ) α/2 h m,n, (5.2) 84

102 Monitoring tube Scilent sensor nodes Active sensor nodes Figure 5.1: System model for wireless sensor network. where d m,n is the distance from the n th source to the m th sensing device, α is the propagation loss factor, and h m,n is the Raleigh fading modeled as complex Gaussian Noise with zero mean and unit variance. Notice that the number of events, the number of sensors, and total number of sources have the following relation K < M N. Consequently, the received signal vector Y is an condensed representation of the event. In other words, vector Y has aliasing of vector X, due to the low sampling rate M. From the algorithm proposed in the next section, we can estimate X from Y. 85

103 5.3 Problem Formulation and Analysis The problem is to obtain the K information of X using the limited number of sensors (M). The first question is that whether or not the information of K-sparse signal is damaged by the dimensionally reduction from X R N down to Y R M. In general, if X is not sparse enough, as long as M < N, the signal is damaged since there are fewer equations than unknowns. On the other hand, for the K-sparse signal, Y is just a linear combination of K columns of G. From the compressive sensing theory, we know that under a certain condition the signal is still preserved in M dimensions. The next question is how to develop a reconstruction algorithm to recover X from the measurement Y. Since M < N there are infinite number of ˆX satisfy Y = G ˆX. All solutions lie on the N M dimension hyperplane H := N (G) + X which corresponding to the null space N (G) translated to X. So the problem is to find the sparse reconstructed signal ˆX in the translated null space as ˆX = arg min ˆX 1. (5.3) Y=G ˆX Define ˆX = U + W. We can change (5.3) to min(u + W) (5.4) s.t. { (GU GW) = Y, U, W > Bayesian Detection Directly solving equation (5.3) with l 1 minimization algorithm without considering the fact that the components of X are either 0 or 1 may give us an acceptable solution, but we would like to take this 0 or 1 information into consideration for better result. Then, instead of using the traditional signal recovery algorithm like the l 1 -magic or linear programming, we adopt the Bayesian compressive sensing [101, 99, 100, 105], which is fully probabilistic and introducing a set of hyper-parameters which is viewed as a prior over the signal, and the most probable values are 86

104 iteratively estimated from the received data. The main reason why this algorithm fits our needs in the sensor network compressive sensing is that, the posterior distributions of many of the signals are sharply peaked around zero, which matches exactly our sparse binary signal system model. By exploiting such a probabilistic Bayesian framework, we can achieve accurate reconstruction with dramatically fewer samples than using other recovering algorithms, which will be shown in simulations in Section 5.5. In the following, we first propose the model, then the iterative marginal likelihood maximization, and finally we propose a heuristic algorithm based on some observations Model Specification In (5.1), the noise in the system is composed of propagation loss with zero mean and variance σ 2. The probability density function can be approximated as Gaussian distribution and written as M p(ϵ) = N (ϵ i 0, σ 2 ). (5.5) i=1 Due to the assumption of independence of Y n, the likelihood of the complete data set can be written as ( p(y G, σ 2 ) = (2πσ 2 ) M/2 exp 1 Y GX 2 2σ2 ). (5.6) By adopting a Bayesian perspective, we constrain the parameters by defining an explicit prior probability distribution over them. X has Bernoulli distribution. However, it is hard to obtain a close form solution in our problem with Bernoulli distribution. Instead, we assume a zero-mean Gaussian prior distribution over the signal X p(x α) = N n=1 N (X i 0, α 1 i ) (5.7) = (2π) N/2 N n=1 ( αn 1/2 exp α nx 2 ) n, 2 87

105 where α is a vector of N independent hyper-parameters. We will show in the simulations that this assumption can improve the performance. For the Bernoulli distribution case, we will investigate at the end of this section to further utilize the prior information. Given α, the posterior parameter distribution conditioned over the signal is given by combining the likelihood and prior with Bayes rule p(x Y, α, σ 2 ) = p(y X, σ2 )p(x α) p(y α, σ 2, (5.8) ) which is a Gaussian distribution N (µ, Σ) with covariance and mean of Σ = (A + σ 2 G T G) 1, (5.9) and µ = σ 2 ΣG T Y, (5.10) respectively, where A = diag(α 1,..., α N ) Marginal Likelihood Maximization A most-probable point estimate α MP may be found via a type-ii maximum likelihood procedure [101]. The sparse Bayesian model is formulated as the local maximization with respect to α of the marginal likelihood, or equivalently its logarithm L(α) = log p(y α, σ 2 ) (5.11) ˆ = log p(y X, σ 2 )p(x α)dx = 1 2 ( M log 2π + log C + Y T C 1 Y ), with C = σ 2 + I + GA 1 G T. (5.12) A point estimate µ MP for the parameters is then obtained by evaluating (5.10) with α = α MP, giving a posterior mean approximator GX = Gµ MP. 88

106 However, marginal likelihoods are generally difficult to compute, i.e., values of α and σ 2 which maximize L(α) cannot be obtained in closed form. Thus, we need to re-estimate them iteratively. For the updating of α, following the approach in [101], we differentiate (5.11), and then equate it to 0. After rearranging, we have α new i = γi µ 2, (5.13) i where µ i is the i th posterior mean signal from (5.10), and γ i is defined as γ i = 1 α i N ii, (5.14) with N ii being the i th diagonal element of the posterior signal covariance from (5.9) computed with current α and σ 2 values. Each γ i can be treated as a measure of how well-determined its corresponding parameter X i is by the data. For the variance σ 2, differentiation leads to re-estimate σ 2 new = Y Gµ 2 M i γ. (5.15) i We repeat calculation of α and σ 2 with iteratively updating µ and Σ until certain convergence criteria have been reached. This procedure leads to the maximization the marginal likelihoods. Then at the convergence of α estimation procedure, we make predictions based on the posterior distribution over the signal, conditioned on the maximizing values α MP and σmp 2. In other words, by doing this, we could pick up those entries in the projection matrix G which after projection preserves the information of the signal in Y. With the utilizing of the correspondent elements in the measurements Y and projection matrix G, we could reconstruct our signal with an overwhelming probability. Experimental data in the following Section show that, compared with l 1 -minimization, sampling rate could be reduced dramatically. 89

107 5.4.3 Heuristic using Prior Information After the reconstruction of ˆX, if the algorithm converges to wrong results (which will be shown in the following section), there are two possible situations. First, the algorithm can converge to either around 0 and 1, but with the wrong position for the sparse events. This type of errors could not be easily distinguished. The other type of error would let ˆX have values deviating from 0 or 1. Under this condition, it is very easy to find the error using threshold methods. Then the system can wait until the next time slot (which has different fading parameters) to make a decision, hoping the channel matrix G could be changed so that the reconstructed ˆX can be improved. Here we propose a heuristic algorithm to achieve the above ideas as shown in Table 5.1. Here δ is a small positive constant less than one. If the maximal value of ˆX is within [1 δ, 1+δ], we assume the reconstructed signal is correct. Otherwise, we obtain the new Y. Notice that the cost for the heuristic algorithm is the possible delay for the responses, since the new Y needs to be obtained after the channel states have been changed. Algorithm 5.1 Heuristic algorithm using prior information if 1 δ max X 1 + δ then Report the decision of X and Exit else if max X > 1 + δ or X < 1 δ then Wait until next time slot to obtain new Y Redo Bayesian detection end if Exit after a certain number of unsuccessful trials end if 5.5 Simulation Results and Analysis In this section, we depict some preliminary simulation results. Simulation settings are as follows: we consider a total of N = 256 events occur at random locations within a 500 m by 500 m area. The M wireless sensors are also randomly located within this area. The minimal distance between a event and a sensor is 5 m. The propagation loss factor is 3. The transmitted power is normalized to 1 and the thermal noise is The number of random events is K which is a small number. 90

108 100 Detection Probability VS Sampling Rate at Different K (BCS) 95 Detection Probability (%) K=4.5 K=5 K=5.5 K= Sampling Rate (%) Figure 5.2: Performances comparatione (a) Proposed Scheme. 91

109 100 Detection Probability VS Sampling Rate at Different K (l1) Detection Probability (%) K=4.5 K=5 K=5.5 K= Sampling Rate (%) Figure 5.3: Performances comparatione (b) l 1 Magic Scheme. Figure 5.2 shows the detection probability of the proposed algorithm. We define the sampling rate as M/N. We can see that if the sampling rate is higher than 25%, the detection probability is almost 100%. The performance gradually reduces as the sampling rate reduces and as K (the number of event) increases. For comparison, Figure 5.3 shows the traditional l 1 algorithm. Compared with the conventional linear programming recovery algorithm for compressive sensing, the proposed Bayesian framework with fast marginal likelihood maximization algorithm brings much higher detection probability especially at extremely low sampling rate. For example, with sampling rate as 10%, the proposed scheme is more than 2 times better than the l 1 algorithm. Notice that this performance gain is based on the assumption that the source is Gaussian distribution. If we model the distribution to be more realistic Bernoulli distribution, the performance can be further improved. Figure 5.4 shows an illustrative example with M = 20 and N = 256. We can see that the 92

110 Figure 5.4: Illustration of (a) Correct detection. 93

111 Figure 5.5: Illustration of (b) Incorrect detection. original signals are very sparse (K = 4). The received measurement from different sensors could not tell the original signals, which is shown in the middle of the figure. With the compressed sensing algorithm proposed in the previous sections, the reconstructed signal is shown to match the original signal perfectly. Figure 5.5 shows a possible mistake, in which the reconstructed signals do not match the original signals. However, we can see that the converged results conflict with the prior information of binary sources. As a result, we can easily tell the results are not valid. In Figure 5.6, we show the performance improvement of our proposed heuristic scheme. Here δ = 0.5, and maximal number of unsuccessful trail is 20. We can see that we can reduce the sampling rate further. For example for detection probability 80%, we can reduce sampling rate about 20% more. Figure 5.7 shows the effect of noise on the proposed scheme. We can see that when the SNR 94

112 100 Performance before and after Correction 90 Detection Probability (%) K=4 after K=4 before K=7 after K=7 before Sampling Rate (%) Figure 5.6: Heuristic improvement. reduces, the detection probability is significantly reduced. The detection probabilities for different sampling rates are similarly bad when the SNR is reduced to around 20 db. This is due to the reason that Gaussian distribution is unbounded and can cause significant detection error. In the future work, we will propose some denoise method. 5.6 Conclusions In chapter 5 we propose a compressive sensing method for sparse event detection in wireless sensor networks. We formulate the problem and propose solutions. We propose the use of a small number of monitoring tube to take blended incoherent measurement. As to the signal reconstruction part, we introduced a fully probabilistic Bayesian framework which helps dramatically reduce the sampling rate while still guaranty an overwhelming detection probability. Moreover, we adopt a marginal likelihood maximization algorithm and a heuristic algorithm for the Bayesian framework, which leads to higher detection probability than the traditional linear programming solution for this problem. The noisy condition is also analyzed. 95

113 Figure 5.7: Noise effect. 96

114 Chapter 6 Conclusions and Future Work 6.1 Summary and Conclusions In this dissertation, we firstly survey the history and challenges in signal acquisition and processing and review the emerging field of compressive sensing; We then propose several implementations of CS framework in wireless communications, e.g., high-resolution OFDM channel estimation with low-speed ADC; Collaborative spectrum sensing in cognitive radio networks; Sampling rate reduction for 60 GHz UWB communication and sparse events detection in wireless sensor networks. Since the key components of CS framework are: signal sparsity, linear and non-adaptive measurements and an efficient signal recovery algorithm, for each implementation we focus on presenting these components. Specifically, we do the following for each CS implementations: 1. Analyze signal sparsity in a proper domain; 2. Design a measurement scheme with physical meaning while satisfies the requirements presented in CS theories; 3. Propose a customized signal recovery algorithm for the designed system; 4. Perform numerical simulations and demonstrate the results. In Chapter 2, through innovatively equipping each cognitive radio nodes with a set of frequency selective filter, we are able to get a linear combination of multiple channel sensing information, which is then decoded at the fusion center by either joint sparsity recovery algorithm of matrix completion algorithm. The proposed system greatly reduced the amount of sensing and transmission overhead of cognitive radio (CR) nodes, and simulation results show that, in noiseless cases, 97

115 the number of samples required are no more than 50% of the number of channels in the network to guarantee exact primary user detection for both approaches; while in noisy environments, at low channel occupancy rate, we can still have high probability of detection. In Chapter 3, we successfully designed an OFDM channel estimation scheme using probing pilots with random phases, which preserves the information of channel response during the convolution and uniformly down-sampling processes. We also proposed a sparse recovery algorithm, which returns a channel impulse response estimation close to the Cramér-Rao Lower Bound. Aiming at reducing the sampling rate in 60 GHz UWB communication system, we proposed in Chapter 4 a hardware configuration to acquire high frequency signal with low-speed ADC. The system consists of: 1) a mixer, in which, the multipath received signal is modulated by a set of pseudo-random (PN) Bernoulli sequence; 2) An integrator, which integrate the mixer output throughout each time frame; and 3) Multiple copies of mixer and integrator chain, which finished the signal acquisition. Signal recovery using the Bayesian framework offers overwhelming recovery probability. Simulation results show that, in case the multipath received signal is sparse enough, the bit error rate of BPSK RAKE receiver has similar performance (0.3 db loss) as the high-speed ADC receiver with the sampling rate as low as 2.2%. Finally, we applied the CS framework in wireless sensor networks to detect the spatial and time sparse sensor activation events (Chapter 5). By deploying a set of randomly located monitoring tubes to take in-coherent, linear measurements of the sensor status, we were able to formulate a compressive sensing problem. As to the signal reconstruction part, we adopted the fully probabilistic Bayesian framework and a heuristic algorithm, which working together, leads to higher detection probability than the traditional linear programming solution for this problem. 98

116 6.2 Future Work on Wireless Communications Dynamic Compressive Sensing for Wireless Communication Applications In many wireless communication applications, we are facing dynamic environments. Either the communication channel status will evolve throughout the time or the transmitter and receiver status will change. The assumption of stationary environment may not hold, thus algorithms which can detect the change of the environment and make changes according are preferred. We propose to study this situation and develop dynamic compressive sensing scheme for wireless communication applications. Specifically, some preliminary thoughts on the cognitive radio dynamic channel estimation are described as follows. Channel occupancy evolves over time as PRs start and stop using their channels. Channel gains can also change when the PRs move. However, the CS research has so far focused on static signal sensing except the very recent path following algorithms in [73, 106]. In the future work, we can investigate recovery methods for a dynamic wireless environment where based on existing channel occupancy information, an insignificant change of channel states can be quickly and reliably discovered. Given existing channel occupancy X, each new report, which is an entry M i;j of M, is compared with (FX) i;j. If a significant number of such comparisons show differences, then there is a change in the true X. Since X = (RG T ), either R or G, or both, have changed. A change in R means new channel occupation or release. If R is unchanged, then those channel gains in G corresponding to occupied channels have changed. It is easy to deal with the latter case (i.e., G changed, but R didn t) and update the gains of occupied channels because it boils down to solving a small linear system. Let ˆF and ˆX denote the sub-matrices of F and X, respectively, formed by their columns and rows corresponding to the occupied channels. Then, the new gains are given in the least-squares solution of M = ˆF ˆX, where M shall include new reports arrived after the previous recovery/update but may still have missing entries. This system is easy to solve since the number of occupied channels is small. It is similarly easy to discover released channels as long as there is no introduction of new occupied channels. The release of channel i means the row X i 99

117 of X turns into 0, or small numbers. Therefore, one can solve the system M = ˆF ˆX and find the released channels, which correspond to the rows of ˆX with all zero (or small) entries. When the system M = ˆF ˆX is inconsistent, it means that the received reports cannot be explained by the previously occupied channels, so there must be new channel occupation. Discovering new channel occupation is more difficult since it is to find changes in the previously unoccupied ones, which are much more than the occupied channels. However, it is computationally much easier than starting from scratch. Let X prev and X denote the previous and current channel information, respectively. Arguably, X prev X is highly sparse in the joint sense because only its rows corresponding to newly occupied or released channels can have large nonzero entries. Hence, X can be quickly recovered by performing joint sparsity recovery on X prev X over the constraints M = FX (or a relax version in the noisy case), a task that can be done by the algorithms for stationary recovery Joint Sparsity Recovery Algorithm for MIMO Wireless Communication Applications Since multi-input-multi-output (MIMO) system offers additional parallel channels in spatial domain to boost the data rate and enhancing system performance in terms of capacity and diversity, it is widely adopted in the wireless communication systems. However, the increase in the number of transmitter and receiver antennas leads to the overall higher system complexity, and requires more energy to estimate more channels. On the other hand, since the antenna element in the MIMO system are not spatially far away separated, correlation from spatial domain exists. This enables the use of joint sparsity recovery algorithm to carry out the channel estimation work efficiently. From a 5GHz 40-transmitter-40-receiver ray tracing experiment results shown in Fig. 6.1, the joint sparsity structure is clear. This enables us to lower the sampling rate for each of the individual channel estimation, since the adjacent channel estimation results can be used to offer more information or act as cross check for each other. Keeping this idea in mind, we are going to extend two of our previous works [63] and [107] to MIMO system with joint sparsity recovery algorithm. 100

118 Figure 6.1: Example of MIMO Received Signal Joint Sparsity Structure. 6.3 Other Future Work Compressive Sensing for Seismic Data Acquisitions and Processing Since CS is a technique for acquiring and reconstructing a signal which is sparse or compressible by itself or after some known transformation, it has the potential of simplify the signal acquisition procedure, while still guarantees exact signal recovery, in recent years, researches and applications in this field has exploded. And we are now looking beyond the wireless communications and trying to find other applications for CS technique. In this section, we focus on seismic data acquisition and processing, and the reasons are: the most important information we want to get from the reflection seismology is the reflected signals from the formation boundaries; and compared with the record length, records which show the significant reflections are sparse. This characteristic of seismic data fits the pre-request for applying CS very well. Besides, as reducing data acquisition time is of great importance, geophysicists are trying to fire the sources simultaneously instead of fire them sequentially. This brings in the problem of 101

119 separating the blended receiving signals from different sources. Fortunately, compressive sensing seems to be able to solve this problem in simultaneous shooting while reduce the total number of shots needed. Some pioneer works [108, 109, 110] carried out by geoscientists have proved the feasibility of using CS to help acquire and process seismic data efficiently, also compressive sensing framework for simultaneous acquisition has been studied with primary results. In the previous attempts of using CS on 2 D seismic data acquisition, the random sampling compressive sensing requires is realized by irregular source and receiver locations, or in other words, sources and receivers are put at random locations. The authors in [109] have simulated the unified compressive sensing for simultaneous acquisition with primary estimation, and Figure 6.2 [109] shows part of their results. Based on the information offered in those works, we proposed to study: Besides randomize the source and receiver, we can take into consideration the shot time and depth of the source; Instead of random source and receiver location, we will study the possibility of putting them in a circulant pattern, which may reduce the system complexity; As to the data processing, we will try to apply matrix completion algorithm Compressive Sensing for Concrete Flaw Detection Nondestructive test of heterogeneous materials like concrete of a building or a bridge is crucial. The aim is to identify non-homogeneous conditions such as voids, cracks, honeycombs and frozen concrete without breaking the concrete. In such application, ultrasonic or other mechanical waves are sent into the concrete body, and by recording and analogizing the reflected wave from the flaws and the boundaries of the concrete structure, flaw shape and type and other information can be revealed. This reflective detection method, can be viewed as a small dimension version of the underground earth formation boundary detection problem. Thus, the similar methodology can be borrowed to reduce the measurement while still maintain accurate flaw detection. Besides, when we 102

120 Figure 6.2: Upper: Seismic image from 50% uniformly missing shot positions; Lower: Seismic image from 50% random shot and receiver. 103

121 managed to reduce the sampling points, we can use a small number of built-in sensors to monitoring the entire structure in the real-time manner. However, due to the differences lying in the nature of the material, the frequency of the probing wave, and the dimension of the problem, we have to redesign the scheme for concrete flaw detection using compressive sensing which is different from the seismic data application. Some of the guidelines are listed as follows: Sizes of the flaws are small, usually at the magnitude of µm. As a result, the designed system should have much higher resolution than the geophysical data processing system. Measurements might be extremely limited, due to the size of the concrete structure. Data efficient algorithm will be necessary. Small number of built-in sensors for real time monitoring with our dynamic CS algorithm is the final aim Compressive Sensing for Pipe Line Leakage Detection In order to reduce product loss and damage to the environment, early detection of leaks in the pipelines for oil, gas or other hazardous material is essential. Small undetected leaks can result in very high clean-up costs and have the potential to grow to more serious failures. Figure 6.3 shows the scenario of pipeline leakage and one of the leakage detection technique. Nowadays, there are a variety of methods that can detect leaks in pipelines, ranging from manual inspection to advanced satellite based imaging. Challenges for carrying out this task, including extremely large detection range, usually in harsh area, and the leak events may burst at any time, so fast reaction is required and energy and resources consumption should be taken into consideration. Since wireless sensor network enables the collection of diverse types of data at frequent interval, over large areas, it fits well the leak detection problem. Besides, for the detection work, we just need to know when and where there is an event. Therefore, it is reasonable to deploy a set wireless sensors, which could be triggered to turn on by the changing of the parameter, which are indicators of leak, e.g., temperature, pressure, flow rate, etc. And the scenario becomes exactly the problem 104

122 Figure 6.3: Pipe line leak and remote laser leak detection system what we have modeled and solved in Sparse events detection in wireless sensor networks using compressive sensing. We could extend our work for more specific use Compressive Sensing for Human Vision Model In the active research field of Machine Vision, computing devices see by examining individual pixels of images, processing them and attempting to develop conclusions with the assistance of knowledge bases and features such as pattern recognition engines. However, no machine vision or computer vision system can yet perform image comprehension as human vision model, or could tolerance to lighting variations and image degradation, parts variability etc. While, for video or image transmission, compressive sensing is employed to send the heavy hitters to the destination using limited bits. And compressive sensing are able to explore dynamics of conscious and unconscious processing of visual information efficiently and accurately. It is natural to think if side information acquired via compressive sensing method could help reconstruct signal at the destination for realize human visual Compressive Sensing for Gene Regulatory Networks A key issue in genomic signal processing is the inference of gene regulatory networks (GRNs). Some researchers have successfully built the model of Inference and intervention of gene regulatory networks using systems and control theory. In the GRNs discussed in that model, the amount 105