MULTIPLE-DESCRIPTION coding (MDC) has recently

Transcription

1 646 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 18, NO. 5, MAY 2008 Efficient Multiple-Description Image Coding Using Directional Lifting-Based Transform Nan Zhang, Yan Lu, Member, IEEE, Feng Wu, Senior Member, IEEE, Xiaolin Wu, Senior Member, IEEE, and Baocai Yin Abstract This paper proposes an efficient two-description image coding technique. The two side descriptions of an image are generated by quincunx subsampling. The decoding from any side description is done by an interpolation process that exploits sample correlation. Although the quincunx subsampling is a natural choice for the best use of sample correlations in image multiple-description coding (MDC), each side description is not amenable to existing image coding techniques because the pixels are not aligned rectilinearly. We show how this difficulty can be overcome by an adaptive directional lifting (ADL) transform that is particularly suitable for decorrelating samples on the quincunx lattice. The ADL transform can be embedded into JPEG 2000 to construct a practical MD image encoder. Experimental results demonstrate that the proposed image MDC scheme can achieve good coding performance. Index Terms Image compression, lifting-based transform, multiple-description coding (MDC), wavelet. I. INTRODUCTION MULTIPLE-DESCRIPTION coding (MDC) has recently gained popularity as an effective technique to cope with transmission errors when compressed media contents are delivered via error-prone channels/networks [1], [2]. The typical scenario of MDC with two descriptions, and two side decoders and one central decoder is depicted in Fig. 1. An input signal is compressed by MDC into two descriptions of rates and. If both descriptions are received, then the central decoder can reconstruct the signal at distortion. But if either one is lost, the side decoder can still reconstruct the signal at a higher distortion or. Manuscript received February 8, 2007; revised August 3, 2007 and November 5, This work was performed at Microsoft Research Asia, Beijing, China, and was supported in part by the National Natural Science Foundation of China ( ) and National Hi-Technology Research and Development Program (863) of China (2006AA01Z317). This paper was recommended by Associate Editor J. F. Arnold. N. Zhang was with College of Computer Science and Technology, Beijing University of Technology, Beijing , China. She is now with the Institute of Digital Media, Peking University, Beijing , China ( zhangnan@pku.edu.cn). Y. Lu and F. Wu are with Microsoft Research Asia, Beijing , China ( yanlu@microsoft.com; fengwu@microsoft.com). X. Wu is with the Department of Electrical and Computer Engineering, Mc- Master University, Hamilton, ON L8S 4K1, Canada ( xwu@mail.ece. mcmaster.ca). B. Yin is with College of Computer Science and Technology, Beijing University of Technology, Beijing , China ( ybc@bjut.edu.cn). Color versions of one or more of the figures in this paper are available online at Digital Object Identifier /TCSVT Fig. 1. Typical scenario of MDC with two descriptions and three decoders. A core theoretical problem of MDC is the set of achievable values for the five-tuple. Assume that the distortion-rate function of the signal is. The side decoder 1, when receiving bits, cannot have distortion less than. With similar arguments for the other two decoders, the bounds for the MD distortions are given as To provide protection against channel losses, there has to be some redundancies between the two side descriptions. This makes it impossible to achieve the equalities in three equations of (1) simultaneously. Therefore, making side descriptions good enough and yet sufficiently different is a tradeoff to be optimized in MDC design. The MDC problem was first studied by information theorists back in the eighties [3] [5]. Many works have also been reported on the design of practical MDC systems, which mainly focus on quantization and transform parts of a signal compression system. MD scalar quantization (MDSQ) is one of the most popular techniques [6], [7]. It creates two coarse side quantizers, each of which produces acceptable side distortion by itself. The two coarse side quantizers can be combined to produce a finer central quantizer, which provides lower distortion than the side quantizers. In [8], the universality of MDSQ was studied, and MDSQ was shown to achieve almost the same performance as the fully optimized entropy-constrained MDSQ [7]. To take the advantage of vector quantization, MD vector quantization is also investigated and reported in [9] [11]. Most of the MD image coding schemes are implemented in a transform domain. Vaishampayan proposed an image MDC technique that applies MDSQ to the quantization of wavelet coefficients in JPEG 2000 [12]. Wang et al. used the pairwise correlating transform to generate multiple correlated descriptions (1) /$ IEEE

2 ZHANG et al.: EFFICIENT MULTIPLE-DESCRIPTION IMAGE CODING 647 in the framework of standard DCT-based image coding [13], [14]. Goyal et al. extended the pairwise correlating transforms to more than two descriptions [15]. Chung et al. introduced a JPEG-like MDC scheme based on lapped orthogonal transforms [16]. MDC versions of many popular image coding techniques were also studied, such as SPIHT [17], subband coding [18], [19], and wavelet coding [20], [21]. The simplest way of generating MDs is interleaved down sampling in the image domain. For instance, the checker board down sampling scheme forms two descriptions, each being a quincunx lattice. Unlike other MDC techniques this approach will not produce any extra samples or symbols to code in the sense that the total number of the samples of the two descriptions is the same as in the input image. The correlations between spatially interleaved descriptions are utilized by the decoders to improve robustness of compressed code streams against channel/network errors. Wang et al. proposed to split an image into four subsampled versions prior to the JPEG coding [22]. Unfortunately, the results in [22] have coding efficiency losses over 3 db compared with those of the single-description JPEG. This is apparently due to that the subsampling process greatly deteriorates the sample correlation within each side description. Bajic et al. proposed to perform subsampling after subband/wavelet transform [23]. The resulting MDs in the transform domain can benefit from the decorrelation function of the transform, and the loss of coding efficiency is smaller than [22]. In this paper, we reexamine the image MDC approach of generating MDs directly in image domain rather than performing a decorrelation transform first. The previous method of [22] down samples an image by dropping every other row and column, and creates four side descriptions each of which remains to be a square lattice. Note that the square lattice is a poor spatial sampling scheme that is widely used solely for device reasons. However, when designing image MDC, we have the freedom to partition an image of square lattice into sublattices of any structure. A good lattice structure for image MDC should make each side description a correlated source so that it can be compressed well, while also leaving sufficient inter-description correlations so that a side decoder can estimate any missing side description(s). The above reasoning naturally leads us to the MDC design of partitioning an image into two quincunx subimages. Quincunx lattice is not only a superior spatial sampling scheme than square lattice. It also keeps maximum correlation between two resulting side descriptions. As an example, quincunx lattice has been adopted in the MD of motion vectors of video [24]. A limitation of the quincunx scheme seems to be that it only allows for two descriptions. However, one can combine quincunx subsampling scheme with MD quantization to produce more than two descriptions. For natural images the correlation between two quincunx subimages is very strong. A side decoder can reconstruct the input image via interpolation when the other side description is missing. The key to the rate-distortion performance of the proposed image MDC system is how to best compress each quincunx subimage, which is the main focus of this paper. It may be tempting to apply an existing good image coding technique, such as JPEG 2000 [25], to the quincunx subimage. But any separable 2-D transform designed for rectilinear pixel grid will perform poorly on quincunx subimages. The cause is pixel misalignment, i.e., a pixel is not in the same column or row as its closest neighbors. For instance, conventional wavelet transform cannot effectively pack signal energy when acting on quincunx, resulting in severe loss in coding efficiency. To overcome this difficulty, we propose an adaptive directional lifting-based transform tailored to the quincunx lattice. The new transform can effectively decorrelate samples on the quincunx lattice. It can replace the conventional wavelet transform in JPEG 2000 and significantly improve its rate-distortion performance on quincunx subimages. Similar ideas of adaptive directional lifting (ADL) have also proven to be successful on conventional images of square lattice [26], [27]. Our preliminary results about applying the ADL to the MD coding have been reported in [28]. The rest of this paper is organized as follows. Section II discusses the quincunx subsampling scheme and exposes the handicap of conventional wavelet transform on quincunx subimages. Section III proposes the adaptive directional lifting transform on the quincunx lattice and discusses its implementation details, in particular the organization and compression of directional information. Section IV describes the MDC decoding operations, which are essentially adaptive image interpolation using a 2-D piecewise autoregressive image model. Experimental results are given in Section V. Finally, Section VI concludes. II. SUBLATTICE ON MDC This section first discusses the sublattice issue on the proposed MDC scheme and then points out the problem after the quincunx subsampling. Assume that a signal is defined as, which is typically a subset of the lattice in the case of digital image. In the proposed MDC scheme, the signal samples will be partitioned into two descriptions (, 2) subject to In general, some redundancies may be introduced in these two descriptions, i.e., information in one description is also contained partially in another description and vice versa. It will definitely enhance the effect of error concealment when one description is corrupted or lost. In this paper, we do not intend to introduce any redundancy into these two descriptions because the spatial correlation among samples should be strong enough for error concealment. In other words, the descriptions are disjoint, i.e., (2) if (3) Obviously, there are two different methods to partition input image into two parts in pixel domain as shown in Fig. 2, where the horizontal subsampling is taken as an example. Fig. 2(a) illustrates the quincunx subsampling and Fig. 2(b) illustrates the orthogonal subsampling. After the splitting process, each description is transformed vertically and independently to generate its low subband and high subband.

3 648 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 18, NO. 5, MAY 2008 Fig. 3. Sublattice is converted from the input description and that used in transform. that is a sample of one description, where and are the coordinates in the original input image. The correlation of with its four neighbors in the same description plays an important role on coding efficiency. In general, the smaller the Euclidian distance among them is, the stronger the correlation will be. Therefore, the Euclidian distance function is defined as (4) Here is the set of four-link neighbors around and is the number of neighbors. and are also the coordinates in the original input image. The function is the Euclidian distance between two samples. According to the criterion (4), sample in one description has the distance with its four neighbors after the quincunx subsampling, whereas the distance is 3/2 after the orthogonal subsampling. Therefore, the proposed MDC scheme adopts the quincunx subsampling. In this paper, we select JPEG 2000 as a codec for each description compression. However, after the quincunx subsample, odd rows of one description are misaligned with even rows as shown in Fig. 2(a). Obviously, JPEG 2000 is not designed for such sampling images. The straightforward way is to shift odd rows right or left and align them with even rows first. Then the 1-D transform is performed vertically on samples of each description. In this way, the sublattice of each description is actually converted to another sublattice in the coding process, as shown in Fig. 3. If the 5/3 filter is used in the vertical wavelet transform, the samples of odd rows are predicted from those of even rows as indicated by the arrows with solid lines in Fig. 3. Unfortunately, the samples of odd rows do not locate at the same column with the corresponding samples of even rows. It will greatly deteriorate the energy compaction property of wavelet transform, thus resulting in low coding performance. Fig. 2. Illustrated subsampling and the first 1-D transform. (a) Quincunx subsampling. (b) Orthogonal subsampling. In order to evaluate these two subsampling methods in statistics, here we define a Euclidian distance criterion. Assume III. DIRECTIONAL LIFTING TRANSFORM ON QUINCUNX LATTICE As discussed above, the quincunx subsampling makes samples of one description to have smaller distance in statistic with its neighbors. However, the subsequent wavelet transform cannot take this advantage because of sample misalignment. To solve this problem, this section first discusses the proposed adaptive directional lifting-based transform on the quincunx lattice and then explains how to estimate and compress directional data.

4 ZHANG et al.: EFFICIENT MULTIPLE-DESCRIPTION IMAGE CODING 649 Once one direction is selected, the prediction value in (6) is taken as a linear combination of the samples at even rows indicated by the arrows in Fig. 4. In particular, we have Fig. 4. Directions used in the proposed directional lifting transform at the quincunx lattice. A. Directional Lifting Transform Consider one description as. It will be first decomposed into high and low subbands by a 1-D vertical wavelet transform. With the technique given in [29], each 1-D wavelet transform can be factored into one or multiple lifting stages. A typical lifting stage consists of three steps: split, prediction, and update. First, all samples of one description are split into two parts: the even polyphase samples and the odd polyphase samples Second, the samples at odd rows are predicted from the samples at the neighboring even rows to produce the high subband coefficients, i.e., where is the predicting value. Finally, the samples at even rows are updated with the updating value to produce the low subband coefficients, i.e., The proposed directional lifting technique aims at fully exploiting the spatial correlation among neighboring samples at the quincunx lattice. The fundamental difference lies in the predicting and updating. Instead of always making prediction and update operations in the horizontal or vertical direction, the proposed transform can adapt to the quincunx lattice and choose a direction of prediction and update to minimize the prediction error. As shown in Fig. 4, eight different directions are pre-defined for prediction and update in the proposed transform, illustrated by 0 7. White circles denote the samples in this description and black circles (denote the half-pixels) the samples in another description that is not available in the coding of this description. The mark denotes the quarter-pixel. When a sample is predicted from neighboring samples according to (6), each candidate direction is checked and the direction with the smallest prediction error is finally selected. Except for the directions 0 and 4, the pixels that all other directions point to do not exist or are not available in the description. The Sinc interpolation is employed in this paper to get these missing sample values. (5) (6) (7) where the weights are given by the filter taps, denotes the prediction direction from 0 to 7, and, which is in halfpixel or quarter-pixel precision, is interpolated using adjacent integer samples by the function. The corresponding finite impulse response function is (9) where and delimit the finite support of the FIR wavelet filter. Since the prediction is still calculated from the samples at even rows, if the prediction direction is known, the proposed lifting transform can still perfectly reconstruct the samples at odd rows without any quantization. The updating step is carried out in the same direction as that in the prediction step. Note that the proposed directional lifting technique is very general, and it does not have any restriction on the update direction. We would like to use the same direction for both prediction and update so as to save the bits for coding directional information. Actually, the optimal updating direction should also be consistent with the prediction direction in most cases. Consequently, in the updating step, the updating signal is generated by (8) (10) where depends on the employed filter taps. The corresponding finite impulse response function is (11) where and specify the kernel of the FIR wavelet filter. Equations (8) (11) show how to introduce direction of prediction and update into a lifting step. It is no difficult to derive the directional lifting transform with a special filter. In JPEG 2000, the performance of the 9/7 filter is usually much better than those of the 5/3 filter and the Haar filter. Therefore, it is

5 650 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 18, NO. 5, MAY 2008 Fig. 6. Multilevel decomposition structures. (a) Mallat s structure. (b) Proposed structure. Fig. 5. Results of the quincunx subsampled description after 1-D vertical transform: (a) low subband and (b) high subband from the conventional lifting; and (c) low subband and (d) high subband from the proposed lifting. adopted in this paper and the proposed FIR functions of 9/7 filters are given as (12), shown at the bottom of the page. Fig. 5 depicts the Lena s results of one quincunx subsampled description after 1-D vertical transform. The low subbands generated by the conventional lifting and the proposed directional lifting look very similar. But the high subbands demonstrate the significant difference between them. The high subband generated by the conventional lifting contains much texture informa- tion. However, the texture information with the proposed directional lifting is almost removed from the highpass subband. After 1-D vertical transform on each description from the quincunx subsampling, the generated low subbands and high subbands are already sample alignment. In the following decomposition, two structures can be applied in the proposed MDC scheme. In the first one, the highpass subband is also further decomposed, as shown in Fig. 6(a), which results in the Mallat s decomposition structure. Since the correlation of samples in the high subband has been removed well by the proposed directional lifting, it is usually not necessary to decompose it again. Therefore, we prefer the second structure for the multilevel wavelet transform in this paper, as shown in Fig. 6(b), where the highpass subband is not decomposed further. From the second level decomposition, the conventional horizontal and vertical wavelet transform can be employed because (12)

6 ZHANG et al.: EFFICIENT MULTIPLE-DESCRIPTION IMAGE CODING 651 Fig. 7. Directions used at the square grid lattice. the samples are already aligned. But from the results reported in [27], the directional lifting is able to outperform the conventional lifting transform up to 2.0 db on images with rich orientation features. Therefore, the directional lifting transform is still applied to the later decompositions. The difference is that the directional lifting transform is designed on the orthogonal lattice, which is exactly the same as that in [27]. Particularly, the first level of decomposition is done on the quincunx lattice with eight directions ranging from 0 to 360 in the quarter-pixel precision. The following levels of decomposition are done on the square grid lattices with nine directions ranging from 135 to 45 and from 45 to 135, as shown in Fig. 7. B. Direction Estimation and Compression How to select the directions in the proposed directional lifting is a rate-distortion optimization problem. Similar to [27], for easy implementation, we partition the image recursively into blocks of variable sizes of quad-tree. All pixels in a quad-tree block will be subject to the same directional lifting transform. The finer the segmentation, the greater degree of gradient uniformity in the resulting blocks. This leads to better directional prediction of the image signal, hence lower distortion. However, the improved signal approximation of the proposed directional lifting transform is achieved at the expense of increased side information to describe the segmentation tree and the lifting directions of individual blocks. To find the quad-tree of optimal balance between the cost of coding the segmentation tree and the cost of coding transform coefficients, we apply the well-known BFOS algorithm for optimal tree pruning [30]. Fig. 8 shows the partitioning results of Lena at different rates. Obviously, the partitions will become finer with the rate increasing. The coding of directional data is also an essential part in the proposed MDC scheme. The direction of the current block is highly correlated with the directions of the up and left adjacent blocks. Therefore, it is coded after prediction. If the up or left block is outside the image, we set the prediction direction as 2 or 4. The coding method of predicted direction residues is similar to that of intra prediction modes in H.264/MPEG-4 AVC [31]. IV. PROPOSED DECODING PROCESS In this section, we describe the decoding process of the proposed image MDC system, including the side decoding when a Fig. 8. Quadtree partitions of Lena at different rates. (a) 0.5 bpp. (b) 1.0 bpp. (c) 2.0 bpp. description is received and the central decoding when two descriptions are received. A. The Side Decoding Our design of the quincunx-based image MDC reduces the side decoding to a problem of image interpolation, which has been extensively studied [32], [33]. When only one side description is received, the MDC decoding task is to interpolate the absent quincunx subimage from the received quincunx subimage. This appears to be relatively easy because each missing pixel has the decoded values of its four 4-connected neighbors. For instance, the method of [33] can be directly used to perform the required image interpolation. It can be done fairly easily in smooth regions but suffer from the artifacts of block effects, blurred details, and ringing effect around edges. Although the proposed directional lifting transform contains directional data, all pixels of a partitioned block share a common direction. They may be not accurate enough for interpolation pixel by pixel. Therefore, we adopt the so-called texture orientation map interpolation in this paper [34]. The decoded description is first analyzed by a bank of Gabor filters to detect the presence or lack of an edge at each pixel and determine the direction of the edge. We choose the following bank of Gabor filters (13) with,, and,. and are standard deviations of the Gaussian envelope, and is the frequency of the sinusoid. As we have done in directional lifting transform, we use eight directions because such an angular resolution is sufficient for our purpose. The scale parameters and should be properly chosen to make the detected texture orientation more robust to noises. In fact, this is the main reason for choosing Gabor filters instead of conventional gradient operator. For the application of image interpolation we are mainly concerned about direction of high frequency signal, and hence choose cycles per pixel, which is the highest frequency suggested in [35].

7 652 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 18, NO. 5, MAY 2008 TABLE I COMPARISONS BETWEEN JPEG 2000 MDC AND THE PROPOSED MDC SCHEME WHEN ONLY ONE DESCRIPTION IS RECEIVED TABLE II COMPARISONS BETWEEN THE CUBIC INTERPOLATION AND THE TEXTURE ORIENTATION INTERPOLATION ON DECODED SAMPLES FROM THE DESCRIPTION 1 Fig. 9. Visual quality comparison on the interpolated images (the left is the result of Cubic interpolation and the right is that of the texture orientation interpolation). the two descriptions respectively and then merge the two decoded quincunx subimages and. However, since each side description is compressed after a certain quantization, any decoded pixel value from one description is only an approximation of the original. Therefore, the central decoder has two approximations for each pixel. One is that from a decoded quincunx image and the other is the interpolated pixel value from the description, or vice verse. It can improve the quality of the joint description by optimally fusing these two approximations, instead of simply spatially interleaving and. In this paper, we adopt a simple linear minimum mean-square data fusion technique. Suppose and denote the pixels in the interpolated images from description and description, respectively. Specifically, the final reconstruction value is calculated as Given a decoded description,wedefine its texture orientation map (TOM) by the magnitudes of the responses of Gabor filters (14) where is a threshold for smoothness, and stands for the angle perpendicular to. Namely, for the pixel position, if the responses of all eight directional filters are close to each other in magnitude, then, indicating that the image waveform is smooth at ; otherwise, the pixel is deemed textural and the texture orientation is set to be perpendicular to the direction in which the Gabor filter has the maximum response. Finally, the interpolation direction is determined by a local window of TOM according to the method proposed in [34]. B. The Central Decoding When two descriptions are simultaneously available at the decoder, a straightforward method of central decoding is to decode (15) where and are the weighing factors which are determined at the encoder and coded into the streams of and, respectively. V. EXPERIMENTAL RESULTS Experiments are conducted and the results are presented in this section to evaluate the performance of the proposed image MDC scheme. The implementation of the new MDC scheme is integrated into the JPEG 2000 reference software VM9.0, with the directional transform replacing the original 2-D separable rectilinear wavelet transform. Naturally, similar to [22], the JPEG 2000 MDC scheme is used as a benchmark for performance comparisons in terms of peak signal-to-noise ratio (PSNR), where input image is first split into two descriptions by quincunx subsampling and then they are coded individually by JPEG To make fair comparison, we also adopt the proposed texture-oriented interpolation and data fusion algorithms in the central decoding of JPEG 2000 MDC.

8 ZHANG et al.: EFFICIENT MULTIPLE-DESCRIPTION IMAGE CODING 653 Fig. 10. PSNR and rate curves of side decoding. Four JPEG test images (Lena, Barbara, Baboon and Finger) of the resolution are used in our experiments. They are split into two descriptions, each of which is a quincunx lattice. Each description is compressed by JPEG 2000 MDC and the proposed scheme, respectively. In the case of JPEG 2000 MDC, images are decomposed into five levels by the 9/7 filter. In the proposed scheme, the first level of decomposition is done by the proposed directional lifting transform on the quincunx lattice, and the following levels of decomposition are done by the adaptive directional lifting transform on the square grid lattice. Table I shows the PSNR results of JPEG 2000 MDC and the proposed MDC scheme when only one description is received. The number 1 and 2 after the image name indicate the first description and the second description, respectively. The PSNR is calculated on all samples of one description and the rate is also that for coding these samples. One can observe that the proposed MDC scheme can outperform JPEG 2000 MDC by up to 2.0 db (e.g., Lena at 0.5 bpp, Barbara at 1.0 bpp, and Finger at 1.0 bpp), because the proposed directional lifting transform can exploit the pixel correlation in the quincunx lattice much better. Baboon is a very difficult test image for its rich facial hire texture. Even in this case, the proposed MDC scheme still outperforms JPEG 2000 by 0.5 db or higher at 1.0 bpp or higher bit rates. Next we evaluate the PSNR performance of a side decoder when only one description is received. As proposed the full-resolution images are reconstructed from the received side description by the texture orientation interpolation method depicted in Table II. For comparison purpose, we also compute the PSNR results of the popular cubic interpolation when applied to the received quincunx image. These two methods are compared in Table II. Since the two descriptions are balanced in our experiments, it suffices to list the PSNR values for the description 1 of the proposed MDC scheme. The PSNR values in this table are calculated over all samples including both the decoded and the interpolated. The rates are also calculated over all samples in terms of full-resolution image. One can observe that, at low rates (e.g., bpp), the two interpolation methods perform roughly the same, with only a small advantage to the texture orientation method. This is due to the lack of high frequency components in the received side description of low rate. As the side rate increases, the text orientation interpolation offers significant gains over the cubic interpolation in PSNR. This is because the former, being adaptive and directional, can reconstructed the edges and textures much better than the latter. A distinct advantage of the texture orientation interpolation when used in side decoding is its superior visual quality to other interpolation methods at the same side bit rate. The visual quality comparison of the interpolated images is given in Fig. 9.

9 654 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 18, NO. 5, MAY 2008 For further evaluation, we compare the proposed MDC scheme with the original JPEG 2000 single description coding (SDC). For the proposed scheme, the full-resolution images are reconstructed from the received single side description by the texture orientation interpolation method. For the JPEG 2000 SDC scheme, the whole image is coded into a single code stream, without offering MDC protection against packet losses. The PSNR-rate curves of different coding schemes are plotted in Fig. 10, where the rates in bpp are calculated in terms of the full-resolution images. It can be seen that the proposed scheme significantly outperforms the JPEG 2000 MDC solution in side decoding. It even outperforms the JPEG 2000 SDC scheme at low rates in some cases. Understandably, the latter outperforms the former at high rates since the JPEG 2000 SDC scheme does not have the redundancy of MDC. But the JPEG 2000 SDC risks complete decoder failure if one of the two transmission channels is broken, whereas the MDC decoder can still survive. The last experiment is designed to evaluate the performance of the proposed image MDC when two descriptions are received. As we have discussed in previous section, each pixel has two estimates in this case. The fusion method in (15) is used to form the final pixel value. The parameters and calculated at the encoder side are listed in Table III. One can observe that and increase with the rate of the side description. This should be expected. When the side rate is high, the decoded samples are of higher quality and thus should have a larger weight on the final result. In addition, the weights in the proposed MDC scheme are usually larger than those in the JPEG 2000 MDC, because the decoded samples have a better quality in the proposed scheme. In this evaluation, the JPEG 2000 SDC results are taken as the upper bounds. It should be noted that the JPEG 2000 SDC does not own the advantages of the MDC in error-robust video transmission. The PSNR and rate curves of different coding schemes are depicted in Fig. 11. In the central decoding, the proposed scheme also significantly outperforms the JPEG 2000 MDC solution (up to 2.0 db in Lena, Barbara, and Finger). VI. CONCLUSION Fig. 11. PSNR and rate curves of central decoding. By inspecting the reconstructions of Lena images at 0.5 bpp by the cubic interpolation and texture orientation interpolation we see that the feathers reconstructed by the cubic interpolation (the left side of Fig. 9) have jagged and broken edges. In contrast, the feathers reconstructed by the texture orientation interpolation (the right side of Fig. 9) appear very close to the original. This paper proposes a new image MDC scheme based on directional lifting transform. The input image is directly split into two descriptions in pixel domain by the quincunx subsampling because it is a natural choice for best use of sample correlations. The correlations of pixels of one description at the quincunx lattice can be exploited very efficiently by the proposed directional lifting transform. The experimental results have shown that the proposed MDC scheme can outperform the JPEG 2000 MDC scheme by up to 2.0 in both side decoding and central decoding. There are some issues to be further investigated in the future. First, the proposed directional transform already contains the direction of a block. It should be used with the estimated direction of each pixel together for interpolation of side decoding. Second, only the two-description case is investigated in this paper. We should consider how to efficiently combine the sample splitting and the MDSQ together so that more descrip-

10 ZHANG et al.: EFFICIENT MULTIPLE-DESCRIPTION IMAGE CODING 655 TABLE III PARAMETERS AND FOR DATA FUSION WHEN THE DESCRIPTIONS ARE RECEIVED tions can be provided in applications. Finally, the fusing technique should be further investigated in the central decoder. REFERENCES [1] V. K. Goyal, Multiple description coding: Compression meets the network, IEEE Signal Process. Mag., vol. 18, no. 5, pp , [2] N. Franchi, M. Fumagalli, R. Lancini, and S. Tubaro, Multiple description video coding for scalable and robust transmission over IP, IEEE Trans. Circuits Syst. Video Technol., vol. 15, no. 3, pp , Mar [3] A. A. EI Gamal and T. M. Cover, Achievable rates for multiple descriptions, IEEE Trans. Inf. Theory, vol. IT-28, no. 6, pp , Nov [4] L. Ozarow, On a source-coding problem with two channels and three receivers, Bell Syst. Tech. J., vol. 59, pp , [5] Z. Zhang and T. Berger, New results in binary multiple descriptions, IEEE Trans. Inf. Theory, vol. 33, no. 4, pp , Jul [6] V. A. Vaishampayan, Design of multiple description scalar quantizers, IEEE Trans. Inf. Theory, vol. 39, no. 3, pp , Mar [7] V. A. Vaishampayan and J. Domaszewicz, Design of entropy-constrained multiple-description scalar quantizers, IEEE Trans. Inf. Theory, vol. 40, no. 1, pp , Jan [8] C. Tian and S. Hemami, Universal multiple description scalar quantization: Analysis and design, IEEE Trans. Inf. Theory, vol. 50, no. 9, pp , Sep [9] H. Jafarkhani and W. Tarokh, Multiple description trellis coded quantization, in Proc. Int. Conf. Image Process., 1998, vol. 1, pp [10] M. Fleming and M. Effros, Generalized multiple description vector quantization, in Proc. Data Compression Conf., 1999, pp [11] V. A. Vaishampayan, N. J. A. Sloane, and S. D. Servetto, Multiple-description vector quantization with lattice codebooks: Design and analysis, IEEE Trans. Inf. Theory, vol. 47, no. 5, pp , May [12] V. A. Vaishampayan, Application of multiple description codes to image and video transmission over lossy networks, in Proc. Int. Workshop Packet Video, Australia, 1996, pp [13] Y. Wang, M. T. Orchard, and A. R. Reibman, Multiple description coding for noisy channels by pairwise correlating transforms, in Proc. Workshop Multimedia Signal Process., 1997, pp [14] Y. Wang, M. T. Orchard, V. A. Vaishampayan, and A. R. Reibman, Multiple description coding using pairwise correlating transforms, IEEE Trans. Image Process., vol. 10, no. 3, pp , [15] V. K. Goyal and J. Kovacevic, Generalized multiple description coding with correlating transforms, IEEE Trans. Inf. Theory, vol. 47, no. 6, pp , Jun [16] D. M. Chung and Y. Wang, Multiple description image coding using signal decomposition and reconstruction based on lapped orthogonal transforms, IEEE Trans. Circuits Syst. Video Technol., vol. 9, no. 6, pp , Sep [17] A. C. Miquel, A. E. Mohr, and A. E. Riskin, SPIHT for generalized multiple description coding, in Proc. Int. Conf. Image Process., 1999, vol. 3, pp [18] M. Srinivasan and R. Chellappa, Multiple description subband coding, in Proc. Int. Conf. Image Process., 1998, vol. 1, pp [19] Y. Yang and K. Ramchandran, Optimal subband filter banks for multiple description coding, IEEE Trans. Inf. Theory, vol. 46, no. 7, pp , Jul., [20] S. D. Servetto, K. Ramchandran, V. A. Vaishampayan, and K. Nahrstedt, Multiple description wavelet based image coding, IEEE Trans. Image Process., vol. 9, no. 5, pp , [21] S. S. Channappayya, J. Lee, R. W. Heath, and A. C. Bovik, Frame based multiple description image coding in the wavelet domain, in Proc. Int. Conf. Image Process., 2005, vol. 3, pp [22] Y. Wang and D. M. Chung, Robust image coding and transport in wireless networks using nonhierarchical decomposition, in Mobile Multimedia Communications, D. J. Goodman and D. Raychaudhury, Eds. New York: Plenum, [23] I. V. Bajic and J. W. Woods, Domain-based multiple description coding of images and video, IEEE Trans. Image Process., vol. 12, no. 10, pp , Oct [24] C. Kim and S. Lee, Multiple description coding of motion fields for robust video transmission, IEEE Trans. Circuits Syst. Video Technol., vol. 11, no. 9, pp , Sep [25] D. Taubman and M. Marcellin, JPEG2000: Image Compression Fundamentals, Standards, and Practice. Norwell, MA: Kluwer, [26] W. Ding, F. Wu, and S. Li, Lifting-based wavelet transform with directionally spatial prediction, presented at the Picture Coding Symp., San Francisco, CA, [27] W. Ding, F. Wu, X. Wu, S. Li, and H. Li, Adaptive directional lifting-based wavelet transform for image coding, IEEE Trans. Image Process., vol. 16, no. 2, pp , Feb [28] N. Zhang, Y. Lu, F. Wu, and B. Yin, Directional lifting-based wavelet transform for multiple description image coding with quincunx segmentation, Proc. PCM 2005 pt. 2, pp , [29] W. Sweldens, The lifting scheme: A custom-design construction of biorthogonal wavelets, Industrial Mathematics Initiative, Dept. Mathematics, Univ. South Carolina, Tech. Rep. 1994:7, [30] L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone, Classification and regression trees, in The Wadsworth Statistics/Probability Series, Belmont, CA, [31] T. Wiegand, G. J. Sullivan, G. Bjntegaard, and A. Luthra, Overview of the H.264/AVC video coding standard, IEEE Trans. Circuits Syst. Video Technol., vol. 13, no. 7, pp , Jul., [32] H. Hou and H. Andrews, Cubic splines for image interpolation and digital filtering, IEEE Trans. Acoust., Speech Signal Process., vol. 26, no. 6, pp , Dec [33] D. D. Muresan and T. W. Parks, Adaptively quadratic (Aqua) image interpolation, IEEE Trans. Image Process., vol. 13, no. 5, pp , May [34] X. Wu and X. Zhang, Image interpolation using texture orientation map and kernel Fisher discriminant, in Proc. Int. Conf. Image Process., Sep. 2005, vol. 1, pp [35] A. K. Jain and F. Farrokhnia, Unsupervised texture segmentation using Gabor filteres, Pattern Recognition, vol. 24, no. 12, pp , Nan Zhang received the B.S. and M.S. degrees from Henan University, Kaifeng, China, in 1997 and 2001, respectively. She received the Ph.D. degree from the College of Computer Science and Technology at Beijing University of Technology, Beijing, China, in She is now a Postdoctoral Researcher in the Institute of Digital Media, Peking University, Beijing, China. Her research interests include image and video coding, 3-D visual data extraction, representation and coding.

11 656 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 18, NO. 5, MAY 2008 Yan Lu (S 02 M 07) received the B.S., M.S., and Ph.D. degrees from Harbin Institute of Technology (HIT), Harbin, China, in 1997, 1999, and 2003, respectively, all in computer science. He was a Research Assistant with the Department of Computer Science, City University of Hong Kong, Hong Kong, during 1999 to He was with the Joint R&D Lab (JDL) for advanced computing and communication, Chinese Academy of Sciences, Beijing, China, during 2001 to Since April 2004, he has been with Microsoft Research Asia, Beijing, China, where he is now a Researcher. Dr. Lu won the IS&T/SPIE Visual Communications and Image Processing Best Paper Awards in His research interests include image and video coding, multimedia streaming, and compression-enabled graphics applications. Xiaolin Wu (M 89-SM 96) received the B.Sc. degree from Wuhan University, Wuhan, China, in 1982, and the Ph.D. degree from the University of Calgary, Calgary, AB, Canada, in He is currently a Professor with the Department of Electrical and Computer Engineering, McMaster University, Hamilton, ON, Canada, and a Research Professor of Computer Science with Polytechnic University, Brooklyn, NY, and he holds the NSERC-DALSA research chair in Digital Cinema. His research interests include multimedia coding and communications, image processing, signal quantization and compression, and joint source-channel coding. He has published over 100 research papers, and holds two patents in these fields. Dr. Wu s awards include the 2003 Nokia Visiting Fellowship, the 2000 Monsteds Fellowship, and the 1998 UWO Distinguished Research Professorship. Feng Wu (M 99 SM 06) received the B.S. degree in electrical engineering from Xidian University, Xidian, China, in He received the M.S. and Ph.D. degrees in Computer Science from Harbin Institute of Technology, Harbin, China, in 1996 and 1999, respectively. He joined in Microsoft Research China, Beijing, China, as an Associate Researcher in He has been a Researcher with Microsoft Research Asia since His research interests include image and video representation, media compression and communication, computer vision, and graphics. He has been an active contributor to ISO/MPEG and ITU-T standards. Some techniques have been adopted by MPEG-4 FGS, H.264/MPEG-4 AVC and the coming H.264 SVC standard. He served as the chairman of China AVS video group in and led the efforts on developing China AVS video standard 1.0. He has authored or co-authored over 100 conference and journal papers. He has about 30 U.S. patents granted or pending in video and image coding. Baocai Yin received the B.S., M.S., and Ph.D. degrees from Dalian University of Technology, Liaoning, China, in 1985, 1988, and 1993, respectively, all in mathematics. He was a Postdoctoral Researcher in the Computer Science Department of Harbin Institute of Technology (HIT), Harbin, China, during He was an Associate Professor in the Department of Computer Science and Engineering, Beijing University of Technology, Beijing, China, from 1995 to Since 1998, he has been a Professor in the College of Computer Science and Technology of Beijing University of Technology. He was a Visiting Scholar at the Hong Kong Polytechnic University in 1999 and a Visiting Scholar in Trier University, Germany, in His research interests include digital multimedia, multifunctional perception, virtual reality and computer graphics.