ON THE AMPLITUDE AND PHASE COMPUTATION OF THE AM-FM IMAGE MODEL. Chuong T. Nguyen and Joseph P. Havlicek

Size: px

Start display at page:

Download "ON THE AMPLITUDE AND PHASE COMPUTATION OF THE AM-FM IMAGE MODEL. Chuong T. Nguyen and Joseph P. Havlicek"

Jacob Hamilton
5 years ago
Views:

1 ON THE AMPLITUDE AND PHASE COMPUTATION OF THE AM-FM IMAGE MODEL Chuong T. Nguyen and Joseph P. Havlicek School of Electrical and Computer Engineering University of Oklahoma, Norman, OK USA ABSTRACT We propose an algorithm to compute modulation functions of the AM-FM image model. We show that previous algorithms suffer from the wrapped orientation problem where the discontinuities in the orientation map of the phase gradient cause undesirable artifacts in the computed modulation functions. We address this wrapped orientation problem by imposing a local smoothness constraint on the phase function and solve for the unwrapped phase. The proposed algorithm produces artifact free modulation functions that are in good agreement with underlying image structure. Finally, we demonstrate the superiority of proposed algorithm with visual examples as well as quantitative measurement. Index Terms AM-FM image model, amplitude modulation, frequency modulation, phase unwrapping, Riesz transform. 1. INTRODUCTION The AM-FM model has been used successfully in a wide variety of 1D and 2D signal processing applications. The model can be used to extract time varying frequencies of nonstationary narrow-band signals [1, 2]. The time varying frequency, also known as instantaneous frequency (IF) or the frequency modulation (FM), is captured from the input signal f(x) by modeling f(x) = a(x) cos[ϕ(x)], (1) where a(x) : R [0, ) is the amplitude modulation (AM) function and ϕ(x) : R R is the phase modulation (PM) function. The FM function is defined as the derivative of the phase modulation signal ϕ(x). Prior to the 1990, the AM-FM model was primarily used for 1D time series analysis. Bovik, Clark, and Geisler [3] were the first to propose a 2D AM-FM model to represent images and textures. This is challenging because 1) the multidimensional IF is not well-defined and 2) the extracted AM and FM functions should agree with physical interpretations of the underlying image structure. Hahn [4] introduced the single orthant complex signal. Havlicek, Havlicek, and Bovik [5] proposed the analytic image model. Bülow and Sommer [6] proposed a nd hypercomplex signal model. Felsberg and Sommer [7] introduced the monogenic signal model that uses the Riesz transform as an extension of the 1D Hilbert transform. Independently, Larkin, Bone, and Oldfield [8] arrived at the same signal model as the monogenic. Recently, Demarcq et al. [9] extended the monogenic signal to color images. In this paper, we consider the 2D formulation of the signal model (1). Most of the 2D algorithms such as [4, 6 9] are restricted to image analysis applications. The computed AM, PM, and FM functions of these algorithms are used as features in image processing tasks such as segmentation, optical flow estimation, texture classification, and target tracking [10]. In these models, the computed PM functions are wrapped in [ π, π). Therefore, they are not ideal for image filtering and image synthesis. For example, direct filtering on the computed PM and FM results in artifacts in the output image because of discontinuities associated with branch cuts in the wrapped phase. Even after application of a practical 2D phase unwrapping algorithm, branch cuts and discontinuities will still remain in the unwrapped phase [10, 11]. These branch cuts will introduce artifacts in the output image when we do AM-FM filtering or reconstruction. In this paper, we impose the smoothness contraint of the PM function to reduce the gradient mismatch at the discontinuous locations. Once the discontinuities are resolved, we reconstruct the PM function from the modified gradient field. The proposed algorithm produces AM, PM, and FM functions in good agreement with human perception of local image structure. Finally, we evaluate the effectiveness of the proposed algorithm against existing models [4 7] by quantitatively comparing the errors in the computed AM, PM, and FM functions. 2. THE WRAPPED ORIENTATION PROBLEM In the classic paper in 1946, Gabor [12] solved the model (1) by constructing the analytic signal. The analytic signal is a complex signal having the input signal as the real part and the quadrature signal as the imaginary part. The quadrature signal is computed by applying the Hilbert transform (HT) to the real input signal. The spectrum of the analytic signal is supported only in the right half of the frequency plane [2,12]. Most successful multi-dimensional AM-FM signal models adopt the Gabor s complex signal model. Their goals are to construct the multi-dimensional imaginary signals.

Hahn [4] proposed the single orthant complex signal model where the spectrum of a nd complex signal is zero in all but one orthant.

Bülow and Sommer [6] introduced the nd hypercomplex signal model. The hypercomplex signal model is constructed by combining the hypercomplex Fourier transform and Hahn s single orthant approach [4].

However, the PM functions of the 2D hypercomplex signal may not exist for certain phase angles [6]. The construction kernel is still the partial Hilbert transform.

2 Hahn [4] proposed the single orthant complex signal model where the spectrum of a nd complex signal is zero in all but one orthant. Havlicek, Havlicek, and Bovik [5, 13] reasoned that the spectra of nd signals have support in 2 n 1 orthants in order to completely characterize the local structure of the underlying signals. Bülow and Sommer [6] introduced the nd hypercomplex signal model. The hypercomplex signal model is constructed by combining the hypercomplex Fourier transform and Hahn s single orthant approach [4]. In the 2D case, the hypercomplex signal is a quaternionic signal which consists of one AM function and three PM functions. However, the PM functions of the 2D hypercomplex signal may not exist for certain phase angles [6]. The construction kernel is still the partial Hilbert transform. Felsberg and Sommer [7] and Larkin, Bone, and Oldfield [8] independently proposed the monogenic signal model. They used the Riesz transform to construct a multi-component vector signal model. The AM component is computed as the norm of this multi-component vector. The FM is computed as the argument of signal components generated by the Riesz transform. Similar to all the partial Hilbert transform approaches, the Riesz kernels are also directional. In all of these models, the directional nature of the underlying kernels used to generate the complex signals introduces undesirable artifacts that do not agree with visual perception of the image structure. For example, the computed AM functions produced by the single orthant (SO), the adjusted Hilbert transform (aht), and the hypercomplex signal (Hyper) exhibit artifacts at structures perpendicular to the direction of action of the directional kernels. Despite using directional kernels, the magnitude of the two Riesz kernels is isotropic. Therefore, the monogenic signal produces an artifact free AM component. Nevertheless, the PM and FM computed from all of these methods exhibit ambiguity in the orientation map of the phase gradient. We illustrate this problem in Fig. 1 where we compute the AM and PM of the original image (Fig. 1(a)). The computed orientation map of the phase gradient is shown in Fig. 1(d). In contrast to the smoothly varying nature of the ground truth orientation map in Fig. 1(c), the computed orientation map has multiple disjoint circular rings. We refer these discontinuities as the wrapped orientation problem. These disjoint rings are the source undesirable artifacts observed in the filter output when image processing filters operate directly on the computed AM and FM functions [10]. Here, we impose a local smoothness constraint on the local phase function to unwrap the local orientations. This constraint allows us to compute artifact free AM, PM, and FM functions. 3. SMOOTH PHASE CONSTRAINT UNWRAPPING We adopt the monogenic signal model as the base of our algorithm. The monogenic signal model uses the 2D Riesz transform to map an input image f(x) into a three-component vector of images (f(x), f x (x), f y (x)). The two Riesz transform components f x (x) and f y (x) are computed by convolving the Riesz kernels with the input image ( ) ( ) fx (x) hx (x) f(x) =, (2) f y (x) h y (x) f(x) where h x (x) and h y (x) are two directional filters acting on the x and y direction [7, 14]. (a) (c) Fig. 1. Chirp image: (a) Original Gaussian chirp image. (b) Original phase gradient field. (c) Original orientation map. (d) Computed wrapped orientation map. The monogenic signal models each component as f(x) = A(x) cos[ϕ(x)], f x (x) = A(x) sin[ϕ(x)] cos[θ(x)], and f y (x) = A(x) sin[ϕ(x)] sin[θ(x)]. In this model, ϕ(x) is the original phase signal and θ(x) is referred to as the local orientation. The AM component A(x) is computed as A(x) = f 2 (x) + fx(x) 2 + fy 2 (x). (3) The local orientation θ(x) can be obtained simply as θ(x) = tan 1 (f y (x)/f x (x)). Here, we show that we can compute the phase function ϕ(x) and the local orientation θ(x) by differentiating the monogenic model. Let [A x, A y ] = A and [f x, f y ] = f. Let R be the magnitude of the FM functions. R can be obtained by (b) (d) R 2 = (A xf Af x ) 2 + (A y f Af y ) 2 A 2 (f f 2 2 ), (4) where we drop the spatial arguments for convenience. The wrapped orientation can be computed as θ w = tan 1 [ Ay f Af y A x f Af x ]. (5)

3 Chirp Diamond Method AM MSE FM MSE AM MSE FM MSE SO [4] TKEO [15] aht [5] Hyper [6] Mono [8] Ours Table 1. The mean square error (MSE) of competing methods with respect to the ground truth AM-FM signal models. The FM component ϕ can be computed directly from the R and θ w as ϕ = [R cos(θ w ) R sin(θ w )]. Finally, the PM is obtained by integrating ϕ. We illustrate the wrapped orientation problem in Fig. 1. The original Chirp image is in Fig. 1(a). The original FM field and the orientation map are shown in Fig. 1(b) and (c). The wrapped orientation map computed using (5) is shown in Fig. 1(d). The branch cuts in the wrapped orientation map θ w will cause artifacts in the computed PM and FM. We address the wrapped orientation problem by enforcing a local phase smoothness constraint on the wrapped orientation θ w. In particular, the local orientation difference computed at two neighbor pixels should not exceed π/2. Based on this constraint, we start the algorithm at seed points where the AM is large in magnitude and subsequently adjust the orientations of the pixels in neighborhoods of the seed points. Let (i, j) be the spatial coordinate of a seed pixel. Let = θ w (i, j) θ(i, j 1) be the orientation difference of the seed point and its neighbor. The local orientation at (i, j 1) is corrected as { θ w (i, j 1) + π if cos( ) < 0 θ w (i, j 1) = (6) θ w (i, j 1) otherwise. This algorithm is initialized by selecting as seeds all the pixels where the AM exceeds a predefined threshold. A binary mask Ψ is then initialized to 1 at the seed points and 0 else where. Starting at the seed points, the algorithm performs orientation correction as in (6). For each corrected pixel (m, n), we set Ψ(m, n) = 1. The algorithm proceeds similar to a binary region-growing technique and stops when all pixels in the mask Ψ are set to 1. This proposed algorithm has complexity O(N 2 ), where N is the row or column dimension of the input image. 4. RESULTS AND DISCUSSION We evaluate the effectiveness of the proposed algorithm against five main algorithms, i.e., the single orthant complex signal (SO) [4], the Teager-Kaiser energy operator (TKEO) [15], the adjusted Hilbert transform (aht) [13], the hypercomplex signal (Hyper) [6], and the monogenic signal (Mono) [7]. We use two test images with known mathematical ground truth for the AM, PM, and FM functions. The Chirp image, shown in Fig. 3(a), has a Gaussian-shaped AM and a quadratic PM. Their formulae are given by a(m, n) = exp [ (m M/2)2 + (n N/2) 2 ], MN p(m, n) = 144 (m M/2)2 + (n N/2) 2 ). (7) MN The Diamond image, shown in Fig. 3(e), consists of a constant AM function and a linear PM function p(m, n) = 4π [3m/M + 2n/N], (8) where M and N are the row and column of the image. In our experiment, we set M = N = 256. The correction neighborhood is a 3x3 window. Ideally, the threshold level can set to 90 95% of the maximum value of the AM component to find the seed points. Our experiments indicates that we can lower this threshold level to 75% to save computational runtime. The visual results of the Chirp image is illustrated in Fig. 2, where each column contains the computed AM and FM vector field. We compute the modulation functions for all six methods but only show the best three method in Fig. 2. The original image is shown in the first column. The results of aht and Mono methods are given in the second and third column. Results of the proposed algorithm are illustrated in the last column. Since the aht approach is directional, the computed AM in Fig. 2(b) contains undesirable artifacts in the region around the center of the image. These artifacts is directly caused by the sharp transition of the directional Hilbert transform kernel. The AM computed by Mono does not suffer from this problem because the magnitude of the 2D Riesz kernels is isotropic [7]. The FM computed by aht and Mono exhibit noticeable artifacts in the vertical stripes across the middle portion of the FM field in Fig. 2(f) and (g). The reason is that both of these approaches use directional kernels. By applying the orientation unwrapping process proposed in this paper, we are able to produce FM functions that are free of such artifacts. For quantitative evaluation, we compute the mean square error (MSE) of the computed AM and FM functions against the true signal models in (7) and (8) for six algorithms. The MSE results are tabulated in Table 1 where bold face indicates the best performance. These results in Fig. 1 demonstrate that the proposed algorithm delivers a consistent performance advantage for the FM computation relative to the competing methods. Since we use the Riesz kernels to obtain the AM, the AM result of our approach and Mono are identical and are better than other five competing methods. We note that the MSE are computed over the whole image. As the result, we do not notice much differences in the FM errors quantitatively, e.g., in the Diamond image, the FM MSE of Mono is and the FM MSE of the proposed method

(a) (b) (c) (d) (e) (f) (g) (h) Fig. 2. Chirp: The computed AM and FM functions.

(e) Original AM. (f) aht FM. (g) Mono FM. (h) Ours FM. (a) (b) (c) (d) (e) (f) (g) (h) Fig. 3.

(c) Mono imaginary image of (a). (d) Our imaginary image of (a). (e) Original Diamond image.

However, we can see clear improvements graphically in Fig.

In relation to the classical Gabor complex signal model, we compare the imaginary images generated by aht

3, where the original images are in the first column, the imaginary images of aht and Mono are shown in

We observe that there are texture mismatches in the imaginary images of aht and Mono in the middle region

Our algorithm is able to eliminate these mismatches due to the orientation unwrapping process.

3(d) and (e) do not suffer from such disruption of the local texture coherency. 5.

The proposed algorithm overcomes the wrapped orientation problem characteristic of existing approaches by

4 (a) (b) (c) (d) (e) (f) (g) (h) Fig. 2. Chirp: The computed AM and FM functions. The FM functions are illustrated as vector fields. (a) Original AM. (b) aht AM. (c) Mono AM. (d) Ours AM. (e) Original AM. (f) aht FM. (g) Mono FM. (h) Ours FM. (a) (b) (c) (d) (e) (f) (g) (h) Fig. 3. Computed imaginary image for Chirp and Diamond: (a) Original Chirp image. (b) aht imaginary image of (a). (c) Mono imaginary image of (a). (d) Our imaginary image of (a). (e) Original Diamond image. (f) aht imaginary image of (e). (g) Mono imaginary image of (e). (h) Our imaginary image of (e) However, we can see clear improvements graphically in Fig. 2(g) and (h), especially in the discontinuous regions in the middle portion of the image. In relation to the classical Gabor complex signal model, we compare the imaginary images generated by aht and Mono using our orientation unwrapping approach. These imaginary images are shown in Fig. 3, where the original images are in the first column, the imaginary images of aht and Mono are shown in the second and third column. The imaginary image obtained by our algorithm is shown in the last column. We observe that there are texture mismatches in the imaginary images of aht and Mono in the middle region of the image. These mismatches are expected because of the directional kernels. Our algorithm is able to eliminate these mismatches due to the orientation unwrapping process. Our computed imaginary images in Fig. 3(d) and (e) do not suffer from such disruption of the local texture coherency. 5. CONCLUSION We proposed an algorithm to compute the classical AM- FM image model. The proposed algorithm overcomes the wrapped orientation problem characteristic of existing approaches by imposing a local phase smoothness constraint. The new algorithm produces artifact free AM and FM components. In addition, the computed AM and FM components are in good agreement with physical interpretation of local image structure. We show quantitatively that the proposed algorithm outperforms the best current competing methods in terms of mean square error relative to ground truth.

5 6. REFERENCES [1] B. Boashash, Estimating and interpreting the instantaneous frequency of a signal Part I. Fundamentals, Proc. IEEE, vol. 80, no. 4, pp , Apr [2] L. Cohen, Time-frequency Analysis, Englewood Cliffs, Prentice-Hall, NJ, [3] A. C. Bovik, M. Clark, and W. S. Geisler, Multichannel texture analysis using localized spatial filters, IEEE Trans. Pattern Anal. Machine Intell., vol. 12, no. 1, pp , Jan [14] M. Unser, D. Sage, and D.V.D Ville, Multiresolution monogenic signal analysis using the Riesz-Laplace wavelet transform, IEEE Trans. Imag. Proc., vol. 18, no. 11, pp , Nov [15] P. Maragos, J. F. Kaiser, and T. F. Quatieri, On amplitude and frequency demodulation using energy operators, IEEE Trans. Signal Proc., vol. 41, no. 4, pp , Apr [4] S. L. Hahn, Multidimensional complex signals with single-orthant spectra, Proc. IEEE, vol. 80, no. 8, pp , Aug [5] J. P. Havlicek, J. W. Havlicek, and A. C. Bovik, The analytic image, in Proc. IEEE Int l. Conf. Image Proc., Santa Barbara, CA, Oct , [6] T. Bülow and G. Sommer, The hypercomplex signal - a novel extension of the analytic signal to the multidimensional case, IEEE Trans. Signal Proc., vol. 49, no. 11, pp , Dec [7] M. Felsberg and G. Sommer, The monogenic signal, IEEE Trans. Signal Proc., vol. 49, no. 12, pp , Dec [8] K. G. Larkin, D. J. Bone, and M. A. Oldfield, Natural demodulation of two-dimensional fringe patterns. I. General background on the spiral phase quadrature transform, J. Opt. Soc. Am. A, vol. 18, no. 8, pp , Aug [9] G. Demarcq, L. Mascarilla, M. Berthier, and P. Courtellemont, The color monogenic signal: Application to color edge detection and color optical flow, Journal of Math. Imag. and Vision, vol. 40, no. 3, pp , [10] C. T. Nguyen and J. P. Havlicek, AM-FM image filters, in Proc. IEEE Int l. Conf. Image Proc., San Diego, CA, Oct , 2008, pp [11] C. T. Nguyen, P.A. Campbell, and J. P. Havlicek, FM filters for modulation domain image processing, in Proc. IEEE Int l. Conf. Image Proc., Cairo, Egypt, Nov. 7-11, 2009, pp [12] G. Gabor, Theory of communication, J. Inst. Elec. Engr., vol. 93, pp , [13] J. P. Havlicek, D. S. Harding, and A. C. Bovik, Multidimensional quasi-eigenfunction approximations and multicomponent AM-FM models, IEEE Trans. Image Proc., vol. 9, no. 2, pp , Feb

Multicomponent Multidimensional Signals

Multidimensional Systems and Signal Processing, 9, 391 398 (1998) c 1998 Kluwer Academic Publishers, Boston. Manufactured in The Netherlands. Multicomponent Multidimensional Signals JOSEPH P. HAVLICEK*