A Directional Extension for Multidimensional Wavelet Transforms

Transcription

1 A Directional Extension for Multidimensional Wavelet Transforms Yue Lu and Minh N. Do IP EDICS: 2-WAVP (Wavelets and Multiresolution Processing). Abstract Directional information is an important and unique feature of multidimensional signals. As a result of a separable extension from one-dimensional (-D) bases, multidimensional wavelet transforms have very limited directionality. Furthermore, different directions are mixed in certain wavelet subbands. In this paper, we propose a simple directional extension for wavelets (DEW) that fixes this subband mixing problem and improves the directionality. The building block of the DEW is a two-channel 2-D filter bank with a checkerboard-shaped frequency partition. The DEW works with both the critically-sampled wavelet transform as well as the undecimated wavelet transform. In the 2-D case, it further divides the three wavelet subbands (i.e. horizontal, vertical, and diagonal) at each scale into six finer directional subbands. The DEW itself is critically-sampled, and hence will not increase the redundancy of the overall transform. Though nonseparable in essence, the proposed DEW has an efficient implementation that only requires -D filtering. Meanwhile, the DEW can be easily generalized to higher dimensions. In a nutshell, the proposed directional extension provides an optional tool to efficiently enhance the directionality of multidimensional wavelet transforms. Numerical experiments show that certain wavelet-based image processing applications will benefit from this improved directionality. Index Terms Wavelet transform, directional information, checkerboard filter bank, filter design, multidimensional signal processing, image denoising, feature extraction. Y. Lu is with the Department of Electrical and Computer Engineering and the Coordinated Science Laboratory, University of Illinois at Urbana-Champaign, Urbana IL 68 ( yuelu@uiuc.edu; WWW: yuelu). M. N. Do is with the Department of Electrical and Computer Engineering, the Coordinated Science Laboratory, and the Beckman Institute, University of Illinois at Urbana-Champaign, Urbana IL 68 ( minhdo@uiuc.edu; WWW: minhdo). This work was supported by the National Science Foundation under Grant CCR (CAREER). April, 25

2 2 I. INTRODUCTION Directional information is a unique feature of multidimensional (MD) signals. Recently, the importance of directional information has been recognized by many image processing applications, including feature extraction, enhancement, denoising, classification, and compression. The wavelet transform [], [2], [3] has a long and successful history as an efficient image processing tool. However, as a result of a separable extension from one-dimensional (-D) bases, wavelets in higher dimensions can only capture very limited directional information. For instance, 2-D wavelets only provide three directional components, namely horizontal, vertical, and diagonal. Furthermore, the 45 and 35 directions are mixed in diagonal subbands. There have been a number of approaches in providing finer directional decomposition. Some notable examples include 2-D Gabor wavelets [4], the steerable pyramid [5], the directional filter bank [6], 2-D directional wavelets [7], brushlets [8], complex wavelets [9], [], [], curvelets [2], and contourlets [3], [4]. However, the wavelet transform is still very attractive for image processing applications for a number of reasons. First, the wavelet transform has a critically-sampled implementation, and can be easily extended to multidimensional cases. In contrast, most of the approaches mentioned above are expansive systems with various redundancy ratios. For example, the complex wavelet transform is 4- times redundant for images, and in general 2 N -times redundant for N-dimensional signals. In terms of implementation complexity, the multidimensional wavelet transforms can be implemented efficiently in a separable fashion. In contrast, systems such as the directional filter bank involve nonseparable filtering and sampling and have high computational complexity. Last but not least, the theory and applications of wavelets have already been extensively studied, offering us a plethora of ready-to-use filters and processing algorithms. Therefore the natural question is: Can we extend the wavelet transform with finer directionality, while still retain its structure and desirable features? We give an affirmative answer in this paper by proposing a simple directional extension for wavelets (DEW). Based on a nonredundant checkerboard filter bank, the proposed DEW works with both the critically-sampled wavelet transform as well as the undecimated wavelet transform. In the 2-D case, the DEW leads to one lowpass subband and six directional highpass subbands at each scale, and fixes the subband mixing problem of wavelets (see Figure ). Being criticallysampled itself, the DEW will not increase the redundancy of the overall transform. Though nonseparable in essence, the DEW has an efficient implementation based on -D operations only. Finally, the DEW can be easily generalized to higher dimensions. April, 25

3 LU AND DO: A DIRECTIONAL EXTENSION FOR MULTIDIMENSIONAL WAVELET TRANSFORMS 3 (π, π) ω 2 (π, π) HH LH HH HL LL HL HH LH HH ω (π, π) ω 2 (π, π) ω (a) (b) Fig.. Division of the 2-D frequency spectrum. (a) The frequency decomposition of the wavelet transform. (b) The frequency decomposition of the wavelet transform with the proposed directional extension. The outline of the paper is as follows. Section II presents the filter bank construction of the directional extension in 2-D, with emphasis on intuition and illustration. The generalization of the results to higher dimensional cases is given in Section III with a more rigorous treatment. Section IV discusses filter design and efficient implementation. We will present some numerical results in Section V and conclude the paper in Section VI. Some preliminary results of this paper were reported in an earlier conference paper [5]. II. THE DIRECTIONAL EXTENSION IN THE 2-D CASE The traditional way to construct 2-D wavelets is to use tensor products of their -D counterparts. The advantage of this approach is its simple separable implementation. Unfortunately, this also imposes serious limits on the directionality of the resulting frequency partitioning. As shown in Figure (a), the 2-D wavelet transform produces one lowpass subband (LL), and three highpass subbands (HL, LH, HH), corresponding to the horizontal, vertical, and diagonal directions. Furthermore, diagonal subbands mixes the directional information oriented at 45 and 35. The main idea here is to find some directional extension to further divide each highpass subband of the wavelets into two branches. In particular, we want to have a system with the frequency partitioning shown in Figure (b), which contains six directional subbands roughly oriented at 5, 45, 75, 5, 35 and 65. This is the same frequency decomposition provided by the 2-D dual-tree complex wavelet transform [9], [], [], which has been shown to be successful in several image processing applications. However, the 2-D complex wavelet transform is 4-times redundant and uses a different filter bank structure compared to wavelets. In the following, we will discuss our directional extension for wavelets in two cases, i.e., when the wavelet transform is critically-sampled or undecimated. April, 25

4 4 F (z 2 ) F (z ) D 2 LL D F (z 2 ) F (z 2 ) D 2 LH F (z ) horizontal D F (z 2 ) vertical D 2 D 2 HL HH Fig. 2. The filter bank construction of the 2-D DWT for one level of decomposition. The filtering and downsampling operations are performed in a separable fashion. F and F denote the -D lowpass and highpass filters, with dark regions representing the ideal passband. D =diag(2, ) and D 2 =diag(, 2) are the 2-fold downsampling matrices along the horizontal and vertical directions, respectively. A. Working with the Critically-Sampled Discrete Wavelet Transform (DWT) This case corresponds to the most widely-used wavelet implementation in which the lowpass and highpass filters are always followed by a 2-fold downsampling. Figure 2 shows the filter bank implementation of the DWT for one level of decomposition. For notations used in the figure and hereafter, we use lower-case letters x[n], where n =(n,n 2 ) T, to denote 2-D discrete signals or filters. We use the corresponding uppercase letters X(z,z 2 ) and X(e jω,e jω2 ) for their z-transforms and discrete-time Fourier transforms, respectively. When the signal or filter is -D, the above notations will be simplified as x[n i ], X(z i ) and X(e jωi ), with i =or 2 specifying the particular dimension. We use the downsampling matrices D =diag(2, ) = 2 and D 2 =diag(, 2) = 2 to represent the 2-fold downsampling operations along the horizontal and vertical directions, respectively. In general, the multidimensional (MD) downsampling operation [6] is specified by an integer matrix M as y[n] =x[mn], where x[n] and y[n] are the MD input and output signals. To see what the desired directional extension should be, we can examine the frequency contents in the wavelet subbands. For simplicity, we first consider the case of using ideal filters, where the frequency response of each filter is constant in the passband and exactly zero in the stopband. We know the diagonal subband (HH) captures certain directional highpass frequency information (illustrated as regions {a, b, c, d} in Figure 3(a)) in the input image, where {a, d} and {b, c} correspond to directional information oriented at 45 and 35, respectively. With the downsampling operations in the wavelet transform, these April, 25

5 LU AND DO: A DIRECTIONAL EXTENSION FOR MULTIDIMENSIONAL WAVELET TRANSFORMS 5 ω 2 (π, π) ω 2 (π, π) a b d c ω ω (π, π) c d (π, π) b a (a) (b) Fig. 3. (a) The diagonal highpass frequency regions of the input signal. (b) The frequency contents of the diagonal subband (HH) after being downsampled by 2 in each dimension. H (z,z ) G (z,z 2 ) D D H (z,z 2 ) G (z,z 2 ) D D Fig. 4. The two-channel 2-D filter bank with a checkerboard-shaped frequency partition. The dark regions represent the ideal passband. frequency regions will be scrambled and mapped to the actual frequency contents in the HH subband, as shown in Figure 3(b). Now to separate regions {a, d} from {c, b}, we can see that a natural choice is to use a two-channel 2-D filter bank with a checkerboard-shaped frequency partition, illustrated in Figure 4. The downsampling matrix used in the filter bank is the simple diagonal matrix D. Similarly, it can be checked that the same checkerboard filter bank can also be used to divide the other two wavelets subbands (HL and LH). This suggests that we can use the checkerboard filter bank as the building block for a directional extension for the DWT to improve its directionality, as shown in Figure 5, where only the analysis part is given. The original wavelet transform is kept in the first two levels. In the third level, the three highpass subbands are further split by the checkerboard filter bank. Applying the multirate identities [6] in multirate signal processing, we can get the equivalent filter of each subband and verify that the system indeed achieves the desired frequency partitioning (Figure (b)) with ideal filters. Since each individual component of the proposed system, i.e., the DWT and the checkerboard filter We can also use D 2 as the downsampling matrix. April, 25

6 6 LL D LH D 2 WT HL D 3 HH D 4 D 5 D 6 separable 2 D wavelet transform directional extension Fig. 5. The filter bank construction of the proposed system for one level of decomposition. The system can be iterated on the lowpass subband for multiple levels of decomposition. The synthesis part, i.e., the inverse transform, is given by a concatenation of the synthesis part of the checkerboard filter banks and the inverse wavelet transform. bank, is critically sampled, the overall system is also critically sampled. Furthermore, if we design the checkerboard filter bank to be perfect reconstruction, then the whole system is also perfect reconstruction. A nice property of the wavelet transform is that it has an efficient separable implementation. As shown in Section IV, the proposed directional extension also has an efficient -D implementation in the polyphase domain, and hence will only moderately increase the computational complexity of the wavelet transform. We recognize that the proposed system has a fundamental limit in its frequency response, when we use non-ideal filters. Consider the chain of operations in a certain branch, say subband 5, in Figure 5. F (z ) F (z 2) H (z,z 2) D D 2 D Here, F (z ),F (z 2 ) are the wavelet highpass filters along the horizontal and vertical directions, and H (z,z 2 ) is one of the checkerboard filters. Using the multirate identities, we can rewrite this branch in its equivalent form F (z )F (z 2)H (z 2,z 2 2) M where the equivalent downsampling matrix is M = D D 2 D =diag(4, 2). April, 25

7 LU AND DO: A DIRECTIONAL EXTENSION FOR MULTIDIMENSIONAL WAVELET TRANSFORMS (a) F (e jω )F (e jω 2 ) (b) H (e j2ω,e j2ω 2 ) (c) F eq(e jω,e jω 2 ) Fig. 6. The magnitude frequency responses of F (e jω )F (e jω 2 ), H (e j2ω,e j2ω 2 ) and F eq(e jω,e jω 2 ) for the DWT case. In the Fourier domain, the equivalent filter of this branch is F eq (e jω,e jω2 )=F (e jω )F (e jω2 )H (e j2ω,e j2ω2 ). () In Figure 6(a) and Figure 6(b), we show the typical magnitude frequency responses of some non-ideal F (e jω )F (e jω2 ) and H (e j2ω,e j2ω2 ). The magnitude frequency response of the equivalent filter F eq is shown in Figure 6(c) as the multiplication of the two previous responses. We can see that F eq (e jω,e jω2 ) is concentrated mostly along the 35 direction. However, there are also some unwanted bumps (aliasing components) along the 45 direction. A similar phenomenon was mentioned in [] for a nonredundant construction of the complex wavelet transform. Similar to the observations made there, it is only possible to reduce the area of those bumps by using longer filters with sharper cut-off behavior; however, the height of the bumps cannot be reduced. It is expected that this aliasing problem will have a negative impact on the performance of the proposed system. However, as shown in Section V, the overall system still benefits from the improved directionality in some applications, since the equivalent frequency responses are still mostly concentrated along single directions. B. Working with the Undecimated Wavelet Transform (UWT) Another useful wavelet transform is called the undecimated wavelet transform [7], [2], which discards all the downsampling operations in the DWT and use appropriately upsampled filters via the algorithme à trous [7]. With J denoting the number of decomposition levels, the UWT is a (3J +)-times redundant transform in 2-D. Note that the lowpass/highpass filters in coarser levels are upsampled accordingly so that the overall equivalent filters are the same as those from the DWT. Similar to the discussion above, we can improve the directionality of the UWT by attaching checkerboard filter banks (directional extension) to its highpass branches. The construction is similar to the one shown in Figure 5, with the DWT replaced April, 25

8 (a) H (e jω,e jω 2 ) (b) F eq(e jω,e jω 2 ) Fig. 7. The magnitude frequency responses of H (e jω,e jω 2 ) and F eq(e jω,e jω 2 ) for the UWT case. by the UWT in the wavelet part. Since the checkerboard filter banks are still downsampled, the directional extension will not increase the redundancy of the overall system. 2 For the UWT case, the frequency aliasing problem is eliminated. For example, the chain of operations in subband 5 of Figure 5 will now be F (z ) F (z 2) H (z,z 2) D The Fourier transform of the equivalent filter is F eq (e jω,e jω2 )=F (e jω )F (e jω2 )H (e jω,e jω2 ). Compared with (), here we are using H (e jω,e jω2 ) (Figure 7(a)) instead of its upsampled version H (e j2ω,e j2ω2 ) (Figure 6(b)) to divide the wavelet subband. This makes a big difference when we use filters with non-ideal frequency responses. As shown in Figure 7(b), the frequency response F eq (e jω,e jω2 ) for the UWT case is single directional and free of the unwanted aliasing components. C. Connections with Other Related Systems As mentioned before, the 2-D wavelet transform with the proposed DEW achieves the same frequency partitioning as that from the 2-D dual-tree complex wavelet transform (CWT). However, there are several differences that are important to mention. The CWT produces complex coefficients and hence provides the phase information, which has be shown to be very useful for some signal processing applications. 2 However, the overall system is no longer shift-invariant due to the downsampling. April, 25

9 LU AND DO: A DIRECTIONAL EXTENSION FOR MULTIDIMENSIONAL WAVELET TRANSFORMS 9 As a price to pay for that, the CWT is an expansive system, being 4-times redundant for images, and in general 2 N -times redundant for N-dimensional signals. In contrast, the proposed DEW is a real-valued transform, and hence discards the phase information. However, it can be nonredundant (when working with the DWT) for arbitrary dimensions. More importantly, the scaling filters in the two trees of the CWT have to satisfy the half-sample delay condition, and hence cannot be arbitrarily chosen. This leads to new challenging joint filter design problems. However, our proposed DEW can work with any existing wavelet filters to suit different applications. No filter design on the wavelet part is needed. Actually this difference reflects the philosophy of this work: we try to improve the directionality of wavelets in an efficient and unintrusive way. Any existing software or hardware implementation of the wavelet transform can continue to be used. Just attach the directional extension! In [8], [9], Fernandes et al. proposed a new non-redundant version of the complex wavelet transform through either a pre-mapping or a post-mapping step. In that interesting construction, the phase information can be obtained without sacrificing in redundancy. One major difference between the two approaches is that the inverse pre-projection and post-projection steps need to use infinite impulse response (IIR) filters to achieve perfect reconstruction. In contrast, our proposed system has perfect reconstruction with linear-phase finite impulse response (FIR) filters. III. GENERALIZATION TO HIGHER DIMENSIONS In this section, we generalize the proposed directional extension for wavelets to arbitrary dimensions. Let us first take a look at the 3-D case. We know the 3-D discrete wavelet transform gives one lowpass subband and seven highpass subbands. Just like its 2-D counterpart, it suffers from limited directionality and the subband mixing problems. Figure 8 shows that the situation here is actually even more severe. Analog to the 2-D case, our idea is to find a DEW to improve the directional selectivity of these highpass subbands. A. Partitioning the N-Dimensional Frequency Spectrum We start with some notations. Let B denote the set of frequencies in [ π, π). The frequency spectrum of N-dimensional (N-D) discrete signals is the Cartesian product B N = } B B {{... B }. Let B + = [,π/2), B = [ π/2, ), B + = [π/2,π), and B = [ π, π/2). One can easily verify that the one-level separable N-D wavelet transform (both the DWT and the UWT) decomposes B N into 2 N N April, 25

10 (a) (b) Fig. 8. Frequency-domain support of two 3D-DWT subbands. Dark regions represent the ideal passband. In the extreme case, information corresponding to four completely different directions is mixed in subband (a). subbands B N = k i {,} B ± k B ± k 2... B ± k N def = k i {,} B ± k B ± k 2, (2) where B ± def k i = B + k i B k i for i =,...,N. In other words, if we calculate the equivalent filter 3 of each wavelet subband, the frequency-domain support of that filter will be one of the 2 N regions in (2). For each highpass subband B ± k B ± k 2 law B ± k B ± k 2 with at least one k i =, we can expand it using the distributive def =(B + k B k )(B + k 2 B k 2 )...(B + k N B k N ) = B s k B s2 k 2...B sn k N (3) s i {+, } = s i {+, } ( B s k B s2 k 2...B sn k N ) B s k B s2 k 2...B sn k N. (4) We group the 2 N terms in (3) into 2 N different polarized pairs in (4), so that each of them corresponds to the frequency support of some real-valued filter. For the 3-D case, we show in Figure 9(a) and Figure 9(b) two possible subbands from (4). We can see that each of them corresponds to a single direction, which is in sharp contrast to the situation in Figure 8(a). In addition to the full splitting in ) (4), it is also possible to have other partial splitting. For example, B ± k B ± k 2 = ( B s k...b sn+ k n+ B s k...b sn+ k n+ B ± k n+2, (5) s i {+, } where n =,...,N. The full splitting is a special case when n = N. Figure 9(c) shows one of the possible subbands from the partial splitting in 3-D. These subbands are useful, since they 3 In this section we only consider ideal filters. April, 25

11 LU AND DO: A DIRECTIONAL EXTENSION FOR MULTIDIMENSIONAL WAVELET TRANSFORMS (a) (b) (c) Fig. 9. Frequency-domain support of several subbands, with dark regions representing the ideal passbands. Part (a) and Part (b): Two possible subbands from the full splitting in (4). Each of them corresponds to a single direction, which is in sharp contrast to the situation in Figure 8(a). Part (c): One possible subband from the partial splitting in (5). correspond to filters that can capture signals with singularities lying on some low-dimensional manifold structures (e.g. curves, surfaces, etc.) [2]. B. The Directional Extension for Wavelets in N-D The main result in this section is that we can achieve the subband splitting in (4) and (5) by attaching some directional extension to the highpass subbands of the wavelet transform. The basic building blocks of the N-D DEW are checkerboard filter banks shown in Figure (a). For notations, we use CB i,j to denote a checkerboard filter bank whose filtering operations are carried out along the ith and jth dimensions. The downsampling matrix is defined to be D i =diag(,...,, 2,,...,), which keeps }{{} every other sample along the ith dimension. The two channels of the (analysis) filter bank are denoted by CB i,j () and CB i,j (), respectively. Lemma : Suppose we attach the filter bank CB i,i+ ( i N ) to one of the highpass subbands of the N-D wavelet transform (either the DWT or the UWT) with an ideal frequency support B ± k B ± k 2. If all filters used are ideal, the wavelet subband will be split into two channels with passband support B ± k...b ± k i (B + k i B + k i+ B k i B k i+ )B ± k i+2 and B ± k...b ± k i (B + k i B k i+ B k i B + k i+ )B ± k i+2, respectively. Remark : This is the generalization of the 2-D result in Section II and is proved by applying the multirate identities. See Appendix for details. As shown in Figure (b), an n-level DEW is a binary tree-structured expansion of filter banks CB,2, CB 2,3,..., and CB n,n+, for which we have the following theorem: Theorem : Suppose we attach the n-level DEW ( n N ) to one of the highpass subbands of the N-D wavelet transform (either the DWT or the UWT) with an ideal frequency support B ± k B ± k 2. i April, 25

12 2 H (z i,z j ) H (z i,z j ) D i D i CB 2,3 CB,2 CB 2,3 CB n,n+ 2 n subbands CB i,j n-level DEW CB n,n+ (a) (b) Fig.. (a) CB i,j is the analysis part of the checkerboard filter bank. The subscripts i, j (i j) specify that the filters H (z i,z j) and H (z i,z j) operate along the ith and jth dimensions. (b) The n-level DEW is a binary tree-structured concatenation of checkerboard filter banks CB,2, CB 2,3,..., and CB n,n+. Notice the arrangement of subscripts for the simple expansion rule. If all filters used are ideal, the wavelet subband will be split into 2 n channels as in (5). Remark 2: This is proved by successively applying the result in Lemma. See Appendix for details. Note that the partial splitting in (5) is along dimensions, 2,...,n+. However, through a permutation of indices, we can construct a similar n-level DEW to generate partial splitting along n + arbitrary dimensions d,d 2,...,d n+, where d i {,...,N}. Moreover, the full splitting in (4) is a special case when n = N. In some applications, e.g., enhancing a particular direction in the signal, one might be only interested in getting a single frequency region. Instead of using the full binary tree expansion of 2 n checkerboard filter banks in Figure (b) to get all 2 n subbands, we can use a connected sequence of n checkerboard filter banks for that specific subband. Such a sequence of checkerboard filter banks is specified by the following proposition, which can be verified in the proof of Theorem (Appendix B). ( ) Proposition : Suppose the subband of interest is B s k...b sn+ k n+ B s k...b sn+ k n+ B ± k n+2, where s i {+, }. The desired adaptive DEW for that subband will be CB,2 (c ) CB 2,3 (c 2 )... CB n,n+ (c n ), where CB i,i+ is connected to channel # c i (either or ) of CB i,i (i =2,...,n) with the following expansion rule: for DWT: δ(k i k i+ ), if s i s i+ c i = ; δ(k i k i+ ), if s i = s i+ for UWT:, if s i s i+ c i =,, if s i = s i+ where i =,...,n, and δ( ) is the Kronecker delta function. April, 25

13 LU AND DO: A DIRECTIONAL EXTENSION FOR MULTIDIMENSIONAL WAVELET TRANSFORMS 3 H (z) y D D G (z) x ˆx H (z) D y D G (z) Fig.. A maximally decimated two-channel 2-D filter bank with analysis filters H (z) (z def =(z,z 2)), H (z), synthesis filters G (z),g (z), and the downsampling matrix D. x and ˆx are the input and reconstructed signals, respectively. y and y are the two output signals of the analysis part. x D y D ˆx z H(z) G(z) z D y D Fig. 2. The equivalent polyphase form of the filter bank in Figure. H(z) and G(z) are 2-by-2 polyphase matrices. IV. FILTER DESIGN AND EFFICIENT IMPLEMENTATION As stated before, there is no need to modify the wavelet transform part and we can choose any existing wavelet filters to suit different applications. The remaining task is to design a 2-D checkerboard filter bank with the desired frequency response and perfect reconstruction. Here we will present a design based on a parameterization of the polyphase matrices. The advantage of this design is that the resulting filter banks have efficient implementation that only requires -D filtering. A more general design of checkerboard filter banks, in particular the ones that are tailored to work in combination with wavelets, will be investigated in a future work. A. Design by Polyphase Matrices Parameterization Let us consider a maximally decimated two-channel filter bank in 2-D with the downsampling matrix D, shown in Figure. The well-known equivalent polyphase form of the system is given in Figure 2. To simplify notation, we denote z =(z,z 2 ). The relationship between the analysis and synthesis filters {H k (z),g k (z)} and the polyphase matrices H(z) and G(z) can be expressed as [6] H k (z) =H k, (z 2,z 2 )+z H k,(z 2,z 2 ) (6) G k (z) =G,k (z 2,z 2 )+z G,k (z 2,z 2 ), April, 25

14 4 for k =,. We can see from Figure 2 that a sufficient condition for the filter bank to have perfect reconstruction is H(z) G(z) =I, (7) where I is the identity matrix. In our design, we choose H(z) and G(z) to be H(z) = 2.5 α(z) (8).5α(z) z and G(z) = 2 z α(z) z 2, α(z) where α(z) is a free design parameter. It is easy to verify that the perfect reconstruction condition (7) is structurally guaranteed for arbitrary choice of α(z). This form of polyphase parameterization was first proposed by Phoong et al. [2] for the -D filter bank. However, the -D to 2-D mapping method proposed in that work can only be used to design 2-D filters banks with a single parallelogram-shaped support, such as the diamond shape. While the checkerboard shape we want here does not belong to this class. In the following, we will propose a novel -D to 2-D mapping for the checkerboard filter bank in the polyphase domain. Substituting (8) into (6), we get H (z) = +z α(z2,z 2) 2 (9) Similarly, we can also write down H (z),g (z) and G (z). Actually, they are all related to H (z) as follows. ( ( ) ) H (z) =z 2 2H (z) H (z) () G (z) = z H ( z,z 2 ) () G (z) =z H ( z,z 2 ). (2) If H (z) is the ideal filter with the desired checkerboard-shaped frequency support shown in Figure 4, i.e., if its Fourier transform takes the constant value 2 in the passband and in the stopband, we can then verify from () - (2) that the other three filters H (z),g (z) and G (z) will also achieve the desired frequency response. For instance, G (e jω,e jω2 )=e jω H (e j(ω π),e jω2 ), i.e., G (e jω,e jω2 ) is obtained by shifting H (e jω,e jω2 ) horizontally by π and multiplying a linear phase. April, 25

15 LU AND DO: A DIRECTIONAL EXTENSION FOR MULTIDIMENSIONAL WAVELET TRANSFORMS 5 ω 2 ω 2 ω 2 (π, π) (π, π) (π, π) + +j j ω ω j j ω + +j j +j +j (π, π) (π, π) (π, π) z α(z2,z 2) M(z ) M(z 2) Fig. 3. figure. The separable decomposition of z α(z2,z 2). The values of the 2-D Fourier transform of the filters are shown in the Therefore, we only need to design H (z) to approximate the ideal filter on the checkerboard support. In turn, this implies that the Fourier transform of the filter z α(z2,z 2) should take constant values (,,, ) in the four quadrants of the 2-D frequency plane, as illustrated in Figure 3. Since this is a separable filter, we can decompose it as the product of two -D filters M(z ) and M(z 2 ), i.e., z α(z2,z 2 )=M(z ) M(z 2 ). (3) If we further constrain M(z) to have real coefficients, i.e., M(e jω )=M (e jω ), then one of the only two choices is 4 Meanwhile, the decomposition form in (3) also implies that j, for ω (,π] M(e jω )=. (4) +j, for ω ( π, ] M(z) =z β( z 2 ), (5) for some -D filter β(z), and α(z) =z 2 β( z )β( z 2 2). (6) From (4) and (5), we want β(z) to be an allpass filter with half-sample delay specified as: β(e jω ) =, ω (7) β(e jω )=.5ω, ω ( π, π); (8) In summary, the proposed filter bank design process is given as follows. () Design a filter β(z) to approximate the conditions in (7) and (8). 4 The only other choice is to use M(e jω ). April, 25

16 6 TABLE I THE COEFFICIENTS OF THE FILTERS β(z) WITH d =3, 4, 5, 6. b(n) d =3 d =4 d =5 d =6 b() b() b(2) b(3) b(4) b(5) -.44 (2) Obtain the analysis and synthesis filters H,H,G,G from (6) and (9) - (2). Note that step (2) is only needed for filter bank analysis. As we will see later, knowing β(z) is sufficient for actual implementation (see Figure 6). B. Designing the Allpass Filter with Half-Sample Delay The half-sample delay conditions (7) and (8) can only be realized by an IIR filter with an irrational system function. There are various ways to design FIR filters to approximate the conditions. See [22] for a review. Here we use a simple approach by choosing β(z) to be a type-ii linear phase filter [23] of length-2d: β(z) =b ( + z)+b (z + z 2 )+...+ b d (z (d ) + z d ). Note that the phase condition (8) is satisfied exactly, and we need to design the coefficients b,...,b d such that the magnitude of the frequency response d β(e jω ) = 2b k cos ((k + 2 ) )ω k= is as close to as possible. In Table I, we list the coefficients of the filters β(z) of different lengths designed by the Parks- McClellan algorithm. We show in Figure 4 the magnitude responses of those β(z). From (6), (9), and (), we can verify that the resulting 2-D filters H and H are of sizes (4d ) (4d ) and (8d 3) (8d 3), respectively. In Figure 5, we give the magnitude response of the 2-D filter H (z) obtained from a β(z) with d =6. We can see that it is a good approximation to the ideal checkerboard filter. April, 25

17 LU AND DO: A DIRECTIONAL EXTENSION FOR MULTIDIMENSIONAL WAVELET TRANSFORMS 7 Magnitude Response (db).5 Magnitude (db) d = 3 d = 4 d = 5 d = 6 Magnitude Normalized Frequency ( π rad/sample) F y.5 F x Fig. 4. The magnitude frequency responses of β(z) with d =3, 4, 5, and 6. Note that the phase responses (not shown in the figure) satisfy (8) exactly. Fig. 5. The magnitude frequency response of the filter H (z) obtained from a β(z) with d =6. x y D D 2 2 ˆx z z 2 β( z )β( z2 2 ) z 2 β( z )β( z2 2 ) z 2 β( z )β( z2 2 ) z 2 β( z )β( z2 2 ) z D 2z y 2 z D H(z) G(z) Fig. 6. The efficient polyphase implementation of the 2-D checkerboard filter bank. C. Efficient Implementation A nice property of the proposed filter design is that the structure is similar to a ladder network [2]. In Figure 6, we show the polyphase implementation of the analysis and synthesis parts of the filter bank. Although the designed 2-D filters are nonseparable, their polyphase components are separable. Meanwhile, the downsampling matrix D =diag(2, ) is also separable. Therefore, the entire system can be implemented by efficient -D operations only. To have a rough estimate of the computational complexity of the proposed DEW, we can count the number of arithmetic operations needed for each new input sample to the DEW. In 2-D, the DEW is just a single-level checkerboard filter bank implemented in the polyphase domain as in Figure 6. For simplicity, we assume the convolution operations are implemented as straight polynomial products, while similar analysis can be done for FFT-based convolutions. Suppose the filter β(z) used in Figure 6 is of April, 25

18 8 TABLE II NUMBER OF ARITHMETIC OPERATIONS/INPUT SAMPLE FOR DIFFERENT SYSTEMS. BOTH THE DWT AND THE UWT PERFORM A J-LEVEL DECOMPOSITION OF A N -DIMENSIONAL SIGNAL. THE TWO -D FILTERS USED IN THE WAVELET TRANSFORM ARE OF LENGTH L AND L 2.THE CHECKERBOARD FILTERS USED IN THE DEW ARE DERIVED FROM A β(z) OF LENGTH 2d. # of multiplications/input sample # of additions/input sample DWT ( L +L 2 )N 2 NJ ( L +L 2 )N 2 NJ 2 2 N 2 2 N DEW for DWT (2d +)(N )( 2 NJ ) (4d )(N )( 2 NJ ) UWT (L + L 2)(2 N )J (L + L 2 2)(2 N )J DEW for UWT (2d + )(2 N )J (4d )(2 N )J length 2d. Since it is a symmetric filter, the filtering by β( z ) requires d multiplications/input sample and 2d additions/input sample. Moreover, the filtering by β( z2 2 ) can be efficiently implemented in the polyphase domain and still requires d multiplications/input sample and 2d additions/input sample. We can verify that the total number of operations needed for the analysis part of the DEW is 2d + multiplications/input sample and 4d additions/input sample. The number of operations needed for the synthesis part is the same, since the analysis and synthesis parts are duals of each other. For the general N-dimensional case, the DEW (for full splitting) is a (N )-level binary tree expansion of checkerboard filter banks. The number of operations becomes (2d +)(N ) multiplications/input sample and (4d )(N ) additions/input sample. For comparison purposes, we show in Table II the number of arithmetic operations/input sample for the DWT, the UWT, and the corresponding DEW in either case. We assume both wavelet transforms are implemented in the polyphase domain 5 and perform a J-level decomposition of a N-dimensional signal. The lowpass and highpass filters used in the wavelet transforms are of length L and L 2, respectively. We do not assume special structures, such as linear phase or orthogonality, on them. The DEW is attached to all the highpass subbands of the wavelet decomposition tree. Here we just list the numbers without derivation. A detailed analysis on the computational complexity of multirate systems can be found in [2]. We can see from Table II that the relative increase in computational complexity (by attaching the DEW) is mainly determined by the ratio d/(l +L 2 ). In 2-D and for some typical numbers L =9,L 2 =7,d=6, 5 This is especially important for the computational efficiency of the UWT. April, 25

19 LU AND DO: A DIRECTIONAL EXTENSION FOR MULTIDIMENSIONAL WAVELET TRANSFORMS (a) DWT: frequency (b) DWT-DEW: frequency (c) UWT-DEW: frequency (d) DWT: spatial (e) DWT-DEW: spatial (f) UWT-DEW: spatial Fig. 7. A few basis images from different transforms. (a) - (c): Some basis images in the frequency domain. (d) - (f): Some basis images in the spatial domain. applying the DEW to the DWT corresponds to an increase of 6% multiplications and 23% additions. Similarly, applying the DEW to the UWT brings an increase of 8% multiplications and 64% additions. V. NUMERICAL RESULTS All experiments in this section use the checkerboard filters designed in Section IV with a β(z) of length 2 (i.e. d =6). These checkerboard filters are close to having the ideal frequency response and thus can provide better separation of the directional information. Meanwhile, the sharp cut-off of these filters can help alleviate the aliasing problem mentioned in Section II. For the wavelet transform, we use three levels of decomposition and employ Daubechies s symmlet of length 2 []. As before, we use DWT and UWT to represent the critically-sampled and the undecimated wavelet transforms, respectively. Correspondingly, we use DWT-DEW and UWT-DEW to denote the two wavelet transforms equipped with the proposed directional extension. A. Basis Images Figure 7 shows a few basis images from different transforms in both frequency and spatial domains. The basis images of UWT are not shown here, since they are equivalent to those of DWT. We can see that the basis images of DWT-DEW and UWT-DEW have better directionality than those of DWT. April, 25

20 2 (a) (b) (c) (d) Fig. 8. (a) A synthetic image consisting of diagonal lines. (b) The diagonal subband of the DWT. (c) Subband 5 of the DWT-DEW. (d) Subband 6 of the DWT-DEW. As expected, the critically-sampled DWT-DEW suffers from the frequency aliasing problem, resulting in some unwanted bumps in the opposite direction (Figure 7(b)). Although the height of these bumps cannot be reduced, it is possible to make the area of the bumps small by using filters with sharp cutoff. However, we should see that most of the subband energy of DWT-DEW concentrates on a single direction, and hence it might still be useful for some image processing applications. Figure 7(c) shows that the aliasing problem is eliminated for UWT-DEW, but at the cost of a redundant system. In the spatial domain, the basis image of DWT from the diagonal (HH) subband exhibits an apparent checkerboard artifact (Figure 7(d)). In contrast, DWT-DEW (Figure 7(e)) and UWT-DEW (Figure 7(f)) succeed in isolating different directions, and no checkerboard effect appears. B. Directional Feature Extraction We first apply the DWT and DWT-DEW on a synthetic image (Figure 8(a)), which consists of lines oriented at both diagonal directions. The result of the UWT-DEW is not shown here, since it is very close to that of the DWT-DEW. Figure 8(b) shows the diagonal (HH) subband of the DWT. As discussed before, 45 and 35 directions are mixed in this subband and the separable wavelets cannot discriminate April, 25

21 LU AND DO: A DIRECTIONAL EXTENSION FOR MULTIDIMENSIONAL WAVELET TRANSFORMS 2 (a) (b) (c) (d) Fig. 9. (a) The cameraman image. (b) The diagonal subband of the DWT. (c) Subband 5 of the DWT-DEW. (d) Subband 6 of the DWT-DEW. between them. In Figure 8(c) and Figure 8(d), we show the subbands 5 and 6 of the DWT-DEW. Clearly the subband mixing problem is solved, and the two subbands correctly capture the corresponding directional information. We then do the same experiment on the cameraman image. As shown in Figure 9(c) and Figure 9(d), the DWT-DEW correctly captures and separates the directional information at 45 and 35 in two subbands. C. Denoising The improvement in directionality of the transform will lead to improvements in many applications. As an example, we compare the performance of the DWT and the UWT, both with and without the proposed DEW, in image denoising. In this experiment, we apply additive white Gaussian noise of different variances σ 2 to the Barbara image. A simple hard thresholding scheme [24] with threshold T =3σ is used for all 4 transforms. In Figure 2, we show the peak signal-to-noise ratio (PSNR) of denoised Barbara images using different transforms under various noise levels. We can see that in all cases the DEW help improve the performance of the original transforms. For a noisy image with PSNR April, 25

22 22 3 PSNR (db) of Denoised Images UWT DEW UWT DWT DEW DWT PSNR (db) of Noisy Images Fig. 2. PSNR (db) of denoised Barbara images by using different transforms under various noise levels. = 2.9 db, the improvement (by using the proposed DEW) is about.3 db for DWT, and a more significant. db for UWT. Figure 2 displays a zoom-in comparison of the denoising results. We can see that edges and directional textures are better restored when we apply the proposed DEW to the DWT and UWT. Despite the relatively small improvement in PSNR, the DWT-DEW clearly outperforms the DWT (especially in regions with directional textures) in terms of visual quality. VI. CONCLUSION In this work, we proposed a simple directional extension for wavelets that equips the multidimensional wavelet transform (both the DWT and the UWT) with finer directionality. The filter bank construction of the overall system is a concatenation of the separable wavelet transform with a sequence of checkerboard filter banks. No modification on the wavelet part is necessary and hence any existing wavelet implementation can continue to be used. In 2-D, the proposed DEW further divides the three wavelet subbands (i.e. horizontal, vertical, and diagonal) at each scale into six finer directional subbands. The DEW is nonredundant, and can be easily generalized to higher dimensions. Moreover, the DEW can be optional and adaptive to specific directions. We also proposed a design of the checkerboard filter banks based on a parameterization of the polyphase matrices and a novel -D to 2-D mapping. The resulting filter banks have efficient implementation that only requires -D filtering. Numerical experiments indicate the April, 25

23 LU AND DO: A DIRECTIONAL EXTENSION FOR MULTIDIMENSIONAL WAVELET TRANSFORMS 23 (a) Original image (b) Noisy image: PSNR = 2.9 db (c) DWT: PSNR = db (d) DWT-DEW: PSNR = 25.4 db (e) UWT: PSNR = db (f) UWT-DEW: PSNR = db Fig. 2. Comparison of denoising results when applying DWT, DWT-DEW, UWT, and UWT-DEW on the Barbara image. potential of the proposed DEW in several wavelet-based image processing applications. APPENDIX A. Proof of Lemma We assume the wavelet transform is the DWT. The proof for the case of UWT is similar. Suppose we attach CB i,i+ to one of the wavelet highpass subbands with an ideal frequency support B ± k B ± k 2. The chain of operations from the input to channel # of CB i,i+ is F k (z ) D... F kn (z N ) D N }{{} N-D DWT H (z i,z i+ ) D i, }{{} CB i,i+ April, 25

24 24 ω i+ (π, π) ω i (π, π) Fig. 22. The frequency-support of H (e j2ω i,e j2ω i+ ). Dark regions represent the passband. where F kj with k j {, } are the ideal lowpass and highpass wavelet filters, and H is one of the checkerboard filters in Figure (a). Applying the multirate identities, we can rewrite this channel as F k (z )...F kn (z N ) H (zi 2,zi+) 2 }{{} D...D N D i }{{}. equivalent filter F eq(z,...,z N ) equivalent downsampling matrix The frequency-domain support of the equivalent filter is supp (F eq ) = supp ( F k (e jω )...F kn (e jωn ) H (e j2ωi,e j2ωi+ ) ) = supp ( F k (e jω )...F kn (e jωn ) ) supp ( H (e j2ωi,e j2ωi+ ) ) = B ± k B ± k 2 supp ( H (e j2ωi,e j2ωi+ ) ). (9) The frequency support of H (e j2ωi,e j2ωi+ ) is shown in Figure 22. One can easily verify that B ± k supp (F eq )=...B ± k i (B + k i B + k i+ B k i B k i+ )B ± k i+2, if k i k i+ B ± k...b ± k i (B + k i B k i+ B k i B + k i+ )B ± k i+2, if k i = k i+. (2) In either case, the two channels of CB i,i+ split the wavelet subband into two subbands as described in Lemma. B. Proof of Theorem Again, we assume the wavelet transform is the DWT. The proof for the case of UWT is similar. The n-level DEW has 2 n subband channels. The chain of operations from the input to any one of the channels is N-D DWT H c (z,z 2 ) D... H cn (z n,z n+ ) D n, }{{} one channel of the n-level DEW April, 25

25 LU AND DO: A DIRECTIONAL EXTENSION FOR MULTIDIMENSIONAL WAVELET TRANSFORMS 25 where c i {, }. Applying the multirate identities, we can get the frequency-domain support of the equivalent filter as supp ( Feq n ) ( = B ± k B ± k 2...B ± ) ( k N supp Hc (e j2ω,e j2ω2 )...H cn (e j2ωn,e j2ωn+ ) ) = ( ( B ± k B ± k 2...B ± ) n k N supp ( H ci (e j2ωi,e j2ωi+ ) )) = n i= i= ( B ± k B ± k 2 supp ( H ci (e j2ωi,e j2ωi+ ) )) (2) = supp ( F n eq ) ( B ± k B ± k 2 supp ( H cn (e j2ωn,e j2ωn+ ) )), (22) where F i eq (i =,...,n) is the equivalent filter from an i-level DEW. Note that each term in (2) is the frequency support of a subband produced by attaching a single-level DEW (CB i,i+ ) to the wavelet subband (e.g. see (9)). This implies that we can successively apply the one-step splitting result in Lemma. More formally, we will use an induction proof. () The claim in Theorem is true for n =, as ensured by Lemma. (2) Now by induction, we suppose that the (n )-level DEW (2 n N ) can split the wavelet highpass subband into 2 n regions as s i {+, } ( B s ) k...b sn k n B s k...b sn k n B ± k n+, (23) and we want to check if the splitting result works for the n-level DEW. (3) From (22), (9), and (2), each subband in (23) will be split into two in the n-level DEW: ( (B s ) ) ) k...b sn k n B s k...b sn k n B ± k n+ ( (B + k n B + k n+ B k n B k n+ ), and ( (B s ) ) ) k...b sn k n B s k...b sn k n B ± k n+ ( (B + k n B k n+ B k n B + k n+ ), which are equivalent to ( B s k...b sn k n B sn k n+ B s k )...B sn k n B sn k n+ B ± k n+2, and ( B s k...b sn k n B sn k n+ B s k )...B sn k n B sn k n+ B ± k n+2, respectively. Therefore, the 2 n subbands of the n-level DEW can be written as ( ) B s k...b sn+ k n+ B s k...b sn+ k n+ B ± k n+2, s i {+, } April, 25

26 26 which proves the claim in the theorem for all n {,...,N }. REFERENCES [] I. Daubechies, Ten Lectures on Wavelets. Philadelphia, PA: SIAM, 992, notes from the 99 CBMS-NSF Conference on Wavelets and Applications at Lowell, MA. [2] M. Vetterli and J. Kovačević, Wavelets and Subband Coding. Prentice-Hall, 995. [3] S. Mallat, A Wavelet Tour of Signal Processing. Academic Press, 998. [4] J. Daugman, Two-dimensional spectral analysis of cortical receptive field profile, Vision Research, vol. 2, pp , 98. [5] E. P. Simoncelli, W. T. Freeman, E. H. Adelson, and D. J. Heeger, Shiftable multiscale transforms, IEEE Trans. Inform. Th., Special Issue on Wavelet Transforms and Multiresolution Signal Analysis, vol. 38, no. 2, pp , March 992. [6] R. H. Bamberger and M. J. T. Smith, A filter bank for the directional decomposition of images: theory and design, IEEE Trans. Signal Proc., vol. 4, no. 4, pp , April 992. [7] J. P. Antoine, P. Carrette, R. Murenzi, and B. Piette, Image analysis with two-dimensional continuous wavelet transform, Signal Processing, vol. 3, pp , 993. [8] F. G. Meyer and R. R. Coifman, Brushlets: a tool for directional image analysis and image compression, Journal of Appl. and Comput. Harmonic Analysis, vol. 5, pp , 997. [9] N. Kingsbury, Complex wavelets for shift invariant analysis and filtering of signals, Journal of Appl. and Comput. Harmonic Analysis, vol., pp , 2. [] I. W. Selesnick, The double-density dual-tree DWT, IEEE Trans. Signal Proc., vol. 52, no. 5, pp , May 24. [] R. G. Baraniuk, N. Kingsbury, and I. W. Selesnick, The dual-tree complex wavelet transform - a coherent framework for multiscale signal and image processing, IEEE SP Mag., submitted, 24. [2] E. J. Candès and D. L. Donoho, Curvelets a suprisingly effective nonadaptive representation for objects with edges, in Curve and Surface Fitting, A. Cohen, C. Rabut, and L. L. Schumaker, Eds. Saint-Malo: Vanderbilt University Press, 999. [3] M. N. Do and M. Vetterli, The contourlet transform: an efficient directional multiresolution image representation, IEEE Trans. Image Proc., to appear, minhdo/publications. [4] Y. Lu and M. N. Do, CRISP-Contourlets: a critically-sampled directional multiresultion image representation, in Proc. of SPIE conference on Wavelet Applications in Signal and Image Processing X, San Diego, USA, 23. [5], The finer directional wavelet transform, in Proc. IEEE Int. Conf. Acoust., Speech, and Signal Proc., Philadelphia, 25. [6] P. P. Vaidyanathan, Multirate Systems and Filter Banks. Prentice Hall, 993. [7] M. Holschneider, R. Kronland-Martinet, J. Morlet, and P. Tchamitchian, A real-time algorithm for signal analysis with the help of the wavelet transform, in Wavelets, Time-Frequency Methods and Phase Space. Springer-Verlag, Berlin, 989, pp [8] F. Fernandes, R. van Spaendonck, and C. Burrus, A new framework for complex wavelet transforms, IEEE Trans. Signal Proc., vol. 5, no. 7, pp , July 23. April, 25

27 LU AND DO: A DIRECTIONAL EXTENSION FOR MULTIDIMENSIONAL WAVELET TRANSFORMS 27 [9], Multidimensional, mapping-based complex wavelet transforms, IEEE Trans. Image Proc., vol. 4, no., pp. 24, January 25. [2] W. Chan, H. Choi, and R. Baraniuk, Directional hypercomplex wavelets for multidimensional signal analysis and processing, in Proc. IEEE Int. Conf. Acoust., Speech, and Signal Proc., Montreal, Canada, May 24. [2] S.-M. Phoong, C. W. Kim, P. P. Vaidyanathan, and R. Ansari, A new class of two-channel biorthogonal filter banks and wavelet bases, IEEE Trans. Signal Proc., vol. 43, no. 3, pp , Mar [22] T. Laakso, V. Valimaki, M. Karjalainen, and U. Laine, Splitting the unit delay - tools for fractional delay filter design, IEEE SP Mag., vol. 3, no., pp. 3 6, January 996. [23] A. V. Oppenheim, R. W. Schafer, and J. R. Buck, Discrete-Time Signal Processing, 2nd ed. Prentice-Hall, 999. [24] D. Donoho and I. Johnstone, Ideal spatial adaptation via wavelet shrinkage, Biometrika, pp , December 994. April, 25