Beyond Nyquist Joel A. Tropp Applied and Computational Mathematics California Institute of Technology jtropp@acm.caltech.edu With M. Duarte, J. Laska, R. Baraniuk (Rice DSP), D. Needell (UC-Davis), and J. Romberg (Georgia Tech) Research supported in part by NSF, DARPA, and ONR 1
The Sampling Theorem Theorem 1. Suppose f is a continuous-time signal whose highest frequency is at most W/2 Hz. Then f(t) = ( n ) f sinc(w t n). n Z W where sinc(x) = sin(πx)/πx. The Nyquist rate W is twice the highest frequency The cardinal series represents a bandlimited signal by uniform samples taken at the Nyquist rate Reference: [Oppenheim et al. 2000] Beyond Nyquist (UC-Bolder, Sept. 2008) 2
Analog-to-Digital Converters (ADCs) An ADC consists of a low-pass filter, a sampler and a quantizer For sampling rate R, low-pass filter has cutoff R/2 to prevent aliasing Ideal sampler produces a sequence of amplitude values: f {f(nt ) : n Z} where the sampling interval T = R 1 The quantizer maps the real sample values to a discrete set of levels Commonly, analog signals are acquired by sampling at the Nyquist rate and processing information with digital technology Beyond Nyquist (UC-Bolder, Sept. 2008) 3
ADCs: State of the Art The best current technology (2005) gives 18 effective bits at 2.5 MS/s (MegaSamples/sec) 13 effective bits at 100 MS/s Performance degradation about 1 effective bit per frequency octave The standard performance metric is P = 2 # effective bits sampling frequency At all sampling rates, one effective bit improvement every 6 years References: [Walden 1999, 2006] Beyond Nyquist (UC-Bolder, Sept. 2008) 4
SNRbits (effective number of bits) 24 22 20 18 16 14 12 10 8 6 4 2 P=4.1x10 11 P=9.13x10 11 Data from 1978 to 1999 Data from 2000 to 2005 Analog Devices: 24 bit 2.5MS/s 16 bit 100 MS/s 2005 1999 0 10 0 10 2 10 4 10 6 10 8 10 10 F sample (HZ) Beyond Nyquist (UC-Bolder, Sept. 2008) 5
Train Wreck Modern applications already exceed ADC capabilities The Moore s Law for ADCs is too shallow to help Conclusion: We need fundamentally new approaches Beyond Nyquist (UC-Bolder, Sept. 2008) 6
Idea: Exploit Structure Absent additional structure, Nyquist-rate sampling is optimal Need to identify and exploit other properties of signals Signals of interest do not contain much information relative to their bandwidth In communications applications, signals often contain few significant frequencies Beyond Nyquist (UC-Bolder, Sept. 2008) 7
Example: An FM Signal 0.01 Frequency (MHz) 0.02 0.04 0.05 0.06 0.07 40.08 80.16 120.23 160.31 200.39 Time (µ s) Data provided by L3 Communications Beyond Nyquist (UC-Bolder, Sept. 2008) 8
Sparse, Bandlimited Signals A normalized model for signals sparse in time frequency: Let W exceed the signal bandwidth (in Hz) Let Ω { W/2 + 1,..., 1, 0, 1,..., W/2} be integer frequencies For each one-second time interval, signal has the form f(t) = a(ω) e 2πiωt for t [0, 1) ω Ω The set Ω of frequencies can change every second In each time interval, number of frequencies Ω = K W Beyond Nyquist (UC-Bolder, Sept. 2008) 9
Information and Signal Acquisition Signals in our model contain little information In each time interval, have K frequencies and K coefficients Total: About K log W bits of information Idea: We should be able to acquire signals with about K log W nonadaptive measurements Challenge: Achieve goal with current ADC hardware Approach: Use randomness! Beyond Nyquist (UC-Bolder, Sept. 2008) 10
Random Demodulator: Intuition With clustered frequencies, demodulate to baseband and low-pass filter demodulation + low-pass filtering 0 0 Don t know locations, so demodulate randomly and low-pass filter Beyond Nyquist (UC-Bolder, Sept. 2008) 11
input signal x(t) input signal X(!) pseudorandom modulating sequence p c (t) pseudorandom modulating sequence P c (!) modulated input signal x(t) modulated signal X(!) and integrator (lowpass filter) Beyond Nyquist (UC-Bolder, Sept. 2008) 12
Exploded View of Passband Beyond Nyquist (UC-Bolder, Sept. 2008) 13
Random Demodulator: System Model Seed Pseudorandom Number Generator p c (t) alternates randomly between levels ±1 at Nyquist rate W Sampler runs at rate R W Beyond Nyquist (UC-Bolder, Sept. 2008) 14
Matrix Formulation I The continuous signal has the form f(t) = ω Ω a(ω) e2πiωt for t [0, 1) Time-averaging for 1/W seconds at t n = n/w yields tn +1/W t n f(t) dt = [ e 2πiω/W ] a(ω) 1 ω Ω 2πiω = ω Ω s(ω) e2πiωt n Can express time-averaged signal as a vector x = F s C W s is sparse and supported on Ω F is essentially a DFT matrix x contains the same (discrete) frequencies as f e 2πiωt n Beyond Nyquist (UC-Bolder, Sept. 2008) 15
Matrix Formulation II The (ideal) action of the multiplier is given by ±1 ±1 D = ±1... ±1 The (ideal) action of the accumulate-and-dump sampler is given by H = 1 1... 1 1 1... 1...... 1 1... 1 R W. Beyond Nyquist (UC-Bolder, Sept. 2008) 16
Reconstruction from Samples The matrix Φ summarizes the action of the random demodulator Φ = HDF : C W C R Maps a (sparse) amplitude vector s to a vector of samples y Given samples y = Φs, signal reconstruction can be formulated as ŝ = arg min c 0 subject to Φc = y The l 0 function counts nonzero entries of a vector Beyond Nyquist (UC-Bolder, Sept. 2008) 17
Signal Reconstruction Algorithms Approach 1: Convex Relaxation Can often find sparsest amplitude vector by solving ŝ = arg min c 1 subject to Φc = y (P1) Approach 2: Greedy Pursuit Identify a small set of significant frequencies and iteratively refine Examples: OMP and CoSaMP References: [Candès et al. 2006, Donoho 2006, Tropp Gilbert 2007, Tropp Needell 2008] Beyond Nyquist (UC-Bolder, Sept. 2008) 18
Shifting the Burden These algorithms are much more computationally intensive than linear reconstruction via cardinal series Move the work from the analog front end to the digital back end Moore s Law for ICs saves us from Moore s Law for ADCs! Beyond Nyquist (UC-Bolder, Sept. 2008) 19
Theoretical Analysis Theorem 2. [T 2007] Suppose the sampling rate satisfies R C K log 6 W Then the matrix Φ has the restricted isometry property (1 c) x 2 2 Φx 2 2 (1 + c) x 2 2 when x 0 2K except with probability W 1. Abstract property supports efficient sampling and reconstruction Intuition: Sampling operator preserves geometry of sparse vectors Beyond Nyquist (UC-Bolder, Sept. 2008) 20
Recovery via Convex Optimization Theorem 3. [Candès Romberg Tao 2006] Suppose that the sampling matrix Φ has the RIP, the sample vector y = Φs + e, and the error e 2 η. Then the solution ŝ to the program min c 1 subject to y Φc 2 η satisfies s ŝ 2 C [ ] 1 s s K 1 + η. K Beyond Nyquist (UC-Bolder, Sept. 2008) 21
Recovery via Greedy Pursuit Theorem 4. [Needell T 2008] Suppose that the sampling matrix Φ has the RIP, the sample vector y = Φs + e, η is a precision parameter, L bounds the cost of a matrix vector multiply with Φ or Φ. Then CoSaMP produces a 2K-sparse approximation ŝ such that s ŝ 2 C max { η, } 1 s s K 1 + e 2 K with execution time O(L log( s 2 /η)). Beyond Nyquist (UC-Bolder, Sept. 2008) 22
Simulations Goal: Estimate sampling rate R to achieve success probability 99% For each of 500 trials, Draw a random demodulator with dimensions R W Choose a random set of K frequencies Set their amplitudes equal to one Take measurements of the signal Recover with l 1 minimization (via IRLS) Define success at rate R when 99% of trials result in s ŝ < ε mach Beyond Nyquist (UC-Bolder, Sept. 2008) 23
Sampling Rate Hz (R) 60 55 50 45 40 35 30 25 10 2 10 3 Signal Bandwidth Hz (W) K = 5, regression line R = 1.69K log(w/k + 1) + 4.51 Beyond Nyquist (UC-Bolder, Sept. 2008) 24
400 350 Sampling Rate Hz (R) 300 250 200 150 100 50 0 20 40 60 80 100 120 140 Number of Nonzero Components (K) W = 512, regression line R = 1.71K log(w/k + 1) + 1.00 Beyond Nyquist (UC-Bolder, Sept. 2008) 25
1 1 Sampling Efficiency (K/R) 0.8 0.6 0.4 0.2 0.8 0.6 0.4 0.2 0.2 0.4 0.6 0.8 1 Compression Factor (R/W) 0 Beyond Nyquist (UC-Bolder, Sept. 2008) 26
Reconstruction of FM Signal 0.01 0.01 Frequency (MHz) 0.02 0.04 0.05 0.06 Frequency (MHz) 0.02 0.04 0.05 0.06 0.07 0.07 40.08 80.16 120.23 160.31 200.39 Time (µ s) (a) Original Signal (1.25 MHz) 40.08 80.16 120.23 160.31 200.39 Time (µ s) (b) Rand Demod (0.63 MHz) 0.01 0.01 Frequency (MHz) 0.02 0.04 0.05 0.06 Frequency (MHz) 0.02 0.04 0.05 0.06 0.07 0.07 40.08 80.16 120.23 160.31 200.39 Time (µ s) (c) Rand Demod (0.31 MHz) 40.08 80.16 120.23 160.31 200.39 Time (µ s) (d) Rand Demod (0.16 MHz) Beyond Nyquist (UC-Bolder, Sept. 2008) 27
On Walden Pond 40 35 30 ENOB 25 20 15 Random Demodulator back-end ADC 10 5 ADC State of the art 1999 0 10 4 10 6 10 8 10 10 Signal Bandwidth Hz (W) Fixed sparsity K = 5000 Beyond Nyquist (UC-Bolder, Sept. 2008) 28
To learn more... E-mail: jtropp@acm.caltech.edu Web: http://acm.caltech.edu/~jtropp http://www.dsp.rice.edu/cs/ http://www.dsp.rice.edu/a2i/ Papers Needell and T, CoSaMP: Iterative Signal Recovery from Incomplete and Inaccurate Measurements, ACHA 2008 T, Romberg, Rice CSP, Beyond Nyquist: Efficient Sampling of Sparse, Bandlimited Signals. In preparation. Beyond Nyquist (UC-Bolder, Sept. 2008) 29