Separable Cosparse Analysis Operator Learning

Transcription

1 Slide 1/12 Separable Cosparse Analysis Operator Learning Julian Wörmann September 3rd, 2014 Separable Cosparse Analysis Operator Learning Julian Wörmann In collaboration with Matthias Seibert, Rémi Gribonval, and Martin Kleinsteuber Technische Universität München Dep. Electrical Engineering and Information Technology September 3rd, 2014

2 Slide 2/12 Separable Cosparse Analysis Operator Learning Julian Wörmann September 3rd, 2014 Contents Sparse signal models Separable Analysis Operator Learning Experiments and conclusion

3 Slide 3/12 Separable Cosparse Analysis Operator Learning Julian Wörmann September 3rd, 2014 Inverse Problems in Image Processing Informative signals are structured

4 Slide 3/12 Separable Cosparse Analysis Operator Learning Julian Wörmann September 3rd, 2014 Inverse Problems in Image Processing Informative signals are structured

5 Inverse Problems in Image Processing I Informative signals are structured I Structure can be captured in adequate signal representation Slide 3/12 Separable Cosparse Analysis Operator Learning Julian Wörmann September 3rd, 2014

6 Slide 4/12 Separable Cosparse Analysis Operator Learning Julian Wörmann September 3rd, 2014 Synthesis versus Analysis Synthesis Model Analysis Model s Dz α Ωs

7 Slide 5/12 Separable Cosparse Analysis Operator Learning Julian Wörmann September 3rd, 2014 Unsupervised Analysis Operator Learning Learn analysis operator from given set of training samples {s i} T i=1 minimize Ω C T g(ωs i) i=1 with the sparsity promoting function 1.5 L0 L1 log square, ν=100 log square, ν=1e6 g(α) := k log ( 1 + να 2 k) The operator Ω is restricted to a set of constraints C

8 Slide 6/12 Separable Cosparse Analysis Operator Learning Julian Wörmann September 3rd, 2014 Modeling the constraints normalized rows ω i = 1 feasible set has product of spheres structure (oblique manifold) Ω OB(m, n) := {Ω R m n : (ΩΩ ) ii = 1, i = 1,..., m} enforced by optimization full rank we can enforce full rank with h(ω) = 1 n log(n) log det( 1 m Ω Ω) controls the condition number of the operator no row repetitions ω k ±ω l, k l r(ω) = k<l log(1 (ω k ω l ) 2 ) related to the mutual coherence of the operator

9 Slide 7/12 Separable Cosparse Analysis Operator Learning Julian Wörmann September 3rd, 2014 Separable structure constraint ) Ω = (Ω (1)... Ω (N) Separable filters dramatically reduce the numerical complexity.

10 Slide 8/12 Separable Cosparse Analysis Operator Learning Julian Wörmann September 3rd, 2014 Separable Co-sparse Analysis Operator Learning A = Ω (1) S Ω (2) ( vec(a) = Ω (2) Ω (1)) vec(s) If the rows of Ω (1) and Ω (2) have unit norm, then the rows of Ω (2) Ω (1) have unit norm as well. each Ω (i) is an element of the oblique manifold The rank of Ω (2) Ω (1) is the product of the rank of Ω (1) and Ω (2) apply full rank penalty on each Ω (i) If the operators Ω (1) and Ω (2) do not exhibit trivially linearly dependent rows, then neither does Ω (2) Ω (1) coherence of the Kronecker product of each Ω (i) is equal to the maximum of the individual mutual coherences

11 Slide 9/12 Separable Cosparse Analysis Operator Learning Julian Wörmann September 3rd, 2014 Separable Co-sparse Analysis Operator Learning Enforcing all the constraints on the operator components Ω (i) is equivalent to enforcing them on the separable operator (Ω (1)... Ω (N) ). Learning a separable operator via: Ω (i) arg min f (Ω (1),..., Ω (i),..., Ω (N) ) for i = 1,..., N Ω (i) f (Ω (1),..., Ω (i),..., Ω (N) ) = T j=1 g (S j 1 Ω (1)... N Ω (N)) N N + µ r(ω (j) ) + κ h(ω (j) ) j=1 j=1 subject to: Ω (i) OB(m i, n i), i = 1,..., N. g : sparsity objective r : incoherence penalty h : full-rank penalty

12 Slide 10/12 Separable Cosparse Analysis Operator Learning Julian Wörmann September 3rd, 2014 Optimization on manifolds Optimization task is tackled using a geometric conjugate gradient on manifolds approach 1 2 Euclidean gradient is projected onto the manifold Search direction determined in tangent space Optimization along geodesics 1 Absil et al., Optimization Algorithms on Matrix Manifolds, Princeton University Press, Princeton NJ, Hawe et al., Analysis Operator Learning and its Application to Image Reconstruction, IEEE Trans. Image Process., 22(6), 2013.

13 Slide 11/12 Separable Cosparse Analysis Operator Learning Julian Wörmann September 3rd, 2014 Reconstruction of volumetric MRI signals Given: {Ω (1), Ω (2), Ω (3) }, with Ω (i) R 6 5 learned from training examples. Reconstruction with standard conjugate gradient optimization. MRI volume data reconstruction from measurements corrupted by additive white Gaussian noise with standard deviation σ noise. Method σnoise PSNR (db) MSSIM # entries in Ω time factor AKSVD our method (6 5) Noisy our method Reconstruction quality from Gaussian noise corrupted measurements. 1 Rubinstein et al., Analysis K-SVD: A Dictionary Learning Algorithm for the Analysis Sparse Model, IEEE Trans. Sig. Proc., 61(3), 2013

14 Slide 12/12 Separable Cosparse Analysis Operator Learning Julian Wörmann September 3rd, 2014 Conclusion Interesting alternative to the synthesis model Separable structure of the filters is highly desirable for computational efficiency The separability constraint is easily integrable into the manifold optimization framework Properties of the operator can be enforced directly during optimization Preprint and MATLAB Code (will be provided soon) can be found at: