Phil Schniter and Jason Parker

Transcription

1 Parametric Bilinear Generalized Approximate Message Passing Phil Schniter and Jason Parker With support from NSF CCF and an AFOSR Lab Task (under Dr. Arje Nachman). ITA Feb 6, 25

2 Approximate Message Passing (AMP) & Generalizations Previously, AMP algorithms have been proposed... for the linear model: Infer x npx(xn) from y = Φx + w with AWGN w and known Φ. [Donoho/Maleki/Montanari 9] for the generalized linear model: Infer x n px(xn) from y m p y z(y m z m) with hidden z = Φx and known Φ. [Rangan ] and for the generalized bilinear model: Infer A m,n pa(amn) and X n,l px(x nl) from Y m,l p y z(y ml z ml ) with hidden Z = AX. [Schniter/Cevher/Parker ] In this talk, we describe recent work extending AMP... to the parametric generalized bilinear model: Infer b i p b(b i) and c j pc(cj) from Y m,l p y z(y ml z ml ) with hidden Z = A(b)X(c) and known matrix-valued linear A( ), X( ). [Parker/Schniter 4] Schniter & Parker (OSU) Parametric BiG-AMP ITA Feb 6, 25 2 /

3 Example Applications of BiG-AMP Matrix Completion: Recover low-rank matrix AX from noise-corrupted incomplete observations Y = P Ω ( AX +W ). 2 Robust PCA: Recover low-rank matrix AX and sparse matrix S from noise-corrupted observations Y = AX +(S+W) = [A I][ X S ]+W. 3 Dictionary Learning: Recover dictionary A and sparse matrix X from noise-corrupted observations Y = AX + W. 4 Non-negative Matrix Factorization: Recover non-negative matrices A and X from noise-corrupted observations Y = AX + W. A detailed numerical comparison against state-of-the-art algorithms suggests BiG-AMP gives best-in-class phase transitions, BiG-AMP gives competitive runtimes. Parker,Schniter,Cevher, IEEE-TSP 4 Schniter & Parker (OSU) Parametric BiG-AMP ITA Feb 6, 25 3 /

4 Example Applications of Parametric BiG-AMP Nonlinear Compressed Sensing with Structured Matrix Uncertainty Observe y = f ( ( i biφi)c+w) with known Φ i. Recover sparse vector c. 2 Generalized Matrix Recovery: Observe Y = f(φbc + W) with known Φ and separable nonlinearity f( ). Recover low-rank matrix BC. 3 Array Calibration: Observe Y = Diag(b )ΦC + W with known Φ. Recover calibration parameters b and signal matrix C. 4 Blind Deconvolution: Observe Y = ΦConv(b)ΨC +W with known Φ and dictionary Ψ. Recover filter b and sparse signal coefficients C. 5 Data Fusion: Observe Y i = Φ ibcω i +W i for i =,2,...,T, with known Φ i and Ω i. Estimate tall B and wide/sparse C 6 and many more... Schniter & Parker (OSU) Parametric BiG-AMP ITA Feb 6, 25 4 /

5 Parametric BiG-AMP: Derivation The functions A( ) and X( ) are treated as random affine transformations. In particular, if b R N b and c R Nc, then a mn (b) = Nb a () x nl (c) = Nc x () N b mn + i= N c nl + j= b i a (i) mn = c j x (j) nl = N b i= N c j= b i a (i) mn c j x (j) nl, where a (i) mn and x (j) nl are realizations of independent zero-mean r.v.s. We then consider the large-system limit where N,M,L,N b,n c such that M/N, L/N, N b /N 2, and N c /N 2 converge to fixed positive constants. The remainder of the derivation follows along the lines of BiG-AMP, 2 but is more involved/tedious. In practice, we also consider smooth, non-linear A( ) and X( ) with partial derivatives a mn(b) (i) and x (j) (c), although without rigorous justification. nl 2 Parker,Schniter,Cevher, IEEE-TSP 4 Schniter & Parker (OSU) Parametric BiG-AMP ITA Feb 6, 25 5 /

6 Parametric BiG-AMP: Features & Extensions The P-BiG-AMP algorithm exploits fast implementations of A( ) and X( ) (e.g., FFT-based). Although P-BiG-AMP requires knowledge of the priors on b and c and the likelihood function p Y Z (y ), the hyper-parameters can be learned from the data using the expectation maximization approach proposed for AMP in [Schniter/Vila ]. Although P-BiG-AMP assumes independent {b i }, independent {c j }, and conditionally independent {y m,n z m,n }, more general models can be handled using the turbo-amp approach proposed in [Schniter ]. Schniter & Parker (OSU) Parametric BiG-AMP ITA Feb 6, 25 6 /

7 Example : CS with Structured Matrix Uncertainty P-BiG-AMP EM-P-BiG-AMP WSS-TLS c-oracle P-BiG-AMP EM-P-BiG-AMP WSS-TLS b,support-oracle NMSE (b) NMSE (c) M/N M/N Measure: y = ( A + N b i= b ia i )c+w, (N = 256, N b =, SNR = 4dB) Unknown (all iid): w m N(,ν w ), b i N(,), c j BG(.4,,) Known (drawn iid): [A ] mn N(,N b ), [A i ] mn N(,) EM-P-BiG-AMP outperforms oracle-tuned WSS-TLS [Zhu/Leus/Giannakis ] Schniter & Parker (OSU) Parametric BiG-AMP ITA Feb 6, 25 7 /

8 Example 2: Random 2D Fourier Measurements of a Sparse Image with Row-Wise Phase Errors GAMP: Phase errors ignored P-BiG-AMP recovery P-BiG-AMP: Phase corrections Phase error (degrees) Randomly sample % of the AWGN-corrupted (@4dB SNR) 2D Fourier measurements of a image with 3 non-zero pixels An unknown random phase (uniformly distributed on [ 9,+9 ]) is added to all the measurements from each row of the observations P-BiG-AMP jointly estimates phase errors and sparse image to 5dB NMSE. Surrogate for simultaneous sparse imaging and autofocus [Önhon/Çetin 2] Schniter & Parker (OSU) Parametric BiG-AMP ITA Feb 6, 25 8 /

9 Summary Presented preliminary work on an algorithm for parametric, bilinear, generalized inference based on AMP principles. Assumes unknown independent random vectors b and c are related to observations Y through a conditionally independent likelihood of the form p(y A(b)X(c)) with known affine A( ) and X( ). Builds on previous Bilinear Generalized AMP work. Can be combined with EM and turbo AMP methods. Numerical experiments demonstrate performance near oracle bounds. Schniter & Parker (OSU) Parametric BiG-AMP ITA Feb 6, 25 9 /

10 References D.L. Donoho, A. Maleki, and A. Montanari, Message passing algorithms for compressed sensing, PNAS, S. Rangan, Generalized approximate message passing for estimation with random linear mixing, ISIT, 2. (See also arxiv:.54). 3 P. Schniter and V. Cevher, Approximate message passing for bilinear models, SPARS, 2. 4 J. T. Parker, P. Schniter, and V. Cevher, Bilinear Generalized Message Passing Part : Derivation, IEEE Trans. Signal Process., J. T. Parker, P. Schniter, and V. Cevher, Bilinear Generalized Message Passing Part 2: Applications, IEEE Trans. Signal Process., J. T. Parker, Approximate Message Passing Algorithms for Generalized Bilinear Inference, Ph.D. Dissertation, Dept. ECE, The Ohio State University, Columbus OH, J. P. Vila and P. Schniter, Expectation-Maximization Gaussian-Mixture Approximate Message Passing, IEEE Trans. Signal Process., P. Schniter, Turbo reconstruction of structured sparse signals, Proc. Conf. Inform. Science & Syst., 2. 9 H. Zhu, G. Leus, and G. Giannakis, "Sparsity-Cognizant Total Least-Squares for Perturbed Compressive Sampling," IEEE Trans. Signal Process., 2. N. Önhon and M. Çetin, A Sparsity-Driven Approach for Joint SAR Imaging and Phase Error Correction, IEEE Trans. Image Process., 22. Schniter & Parker (OSU) Parametric BiG-AMP ITA Feb 6, 25 /

11 Thanks for listening! Schniter & Parker (OSU) Parametric BiG-AMP ITA Feb 6, 25 /

12 Matrix Completion: Phase Transitions The following plots show empirical probability that NMSE < db (over realizations) for noiseless completion of an M L matrix with M=L=. IALM VSBL GROUSE sampling ratio Ω /ML sampling ratio Ω /ML sampling ratio Ω /ML LMaFit BiG AMP Lite EM BiG AMP sampling ratio Ω /ML sampling ratio Ω /ML sampling ratio Ω /ML Note that BiG-AMP-Lite and EM-BiG-AMP have the best phase transitions. Schniter & Parker (OSU) Parametric BiG-AMP ITA Feb 6, 25 /

13 Matrix Completion: Runtime to NMSE=- db sampling ratio Ω /ML =.5 sampling ratio Ω /ML = runtime (sec) runtime (sec) 2 Matrix ALPS GROUSE VSBL LMaFit IALM BiG AMP Lite BiG AMP EM BiG AMP Although LMaFit is the fastest algorithm at small, BiG-AMP-Lite s superior complexity-scaling-with-n eventually wins out. BiG-AMP runs to 2 orders-of-magnitude faster than IALM and VSBL. Schniter & Parker (OSU) Parametric BiG-AMP ITA Feb 6, 25 /

14 Robust PCA: Phase Transitions Empirical probability of NMSE < 8 db over realizations for noiseless recovery of the low-rank component of a 2 2 outlier-corrupted matrix VSBL outlier fraction λ GRASTA outlier fraction λ IALM outlier fraction λ LMaFit 9 BiG AMP 2 9 EM BiG AMP outlier fraction λ outlier fraction λ outlier fraction λ As before, the BiG-AMP methods yield the best phase transitions. Schniter & Parker (OSU) Parametric BiG-AMP ITA Feb 6, 25 /

15 Overcomplete Dictionary Recovery: Phase Transitions Mean NMSE over 5 realizations for recovery of an M (2M) dictionary from L = M log(2m) examples with sparsity K: K-SVD SPAMS EM-BiG-AMP NOISELESS sparsity K NOISY sparsity K dictionary rows M dictionary rows M dictionary rows M 6 As before, the BiG-AMP methods yield the best phase transitions. Schniter & Parker (OSU) Parametric BiG-AMP ITA Feb 6, 25 /