Direction-Adaptive Partitioned Block Transform for Color Image Coding

Transcription

1 Direction-Adaptive Partitioned Block Transform for Color Image Coding Mina Makar, Sam Tsai Final Project, EE 98, Stanford University Abstract - In this report, we investigate the application of Direction Adaptive-Partitioned Block Transform () to color images and propose a basic framework for Color Image Coding. First, we investigate how performs in color spaces using the SCIELAB color metric. Comparison is also made with the other coding methods. Then, rate allocation schemes are then explored across color components. Possibility of incorporating quantization matrix is also studied. Furthermore, spatial correlation between color components is exploited in joint encoding and faster mode decision. Deblocking filter is also incorporated in the framework for subjective visual quality. Index Terms, Color Image Coding I. Introduction Direction Adaptive-Partitioned Block Transform () [] is a grayscale image compression technique that exploits the best direction that represents an image block and also performs block partitioning to improve coding performance over traditional DCT-based transforms. Applying DCT along the direction of an edge gives better energy compaction. Hence, images can be compressed more efficiently; furthermore, directions and image structure are preserved. The main objective of this project is to propose a framework for applying to color image coding. Color images have two additional layers of information compared to grayscale images. Thus, different transforms between color spaces have different compression benefits [-]. We will show YCbCr color space is a good choice. We will also demonstrate that different rate allocation schemes across the three transformed layers of images have different effects. Furthermore, the spatial correlation in the three images can be exploited to improve the encoding performance. In studying the performance of color image compression, we have adopted SCIELAB as our metric for measuring the visual quality of compressed images []. The higher the value it gives, the higher the distortion. SCIELAB extends CIELAB by applying spatial filtering using a pattern color separable method. Thus, it incorporates subjective quality of the human eye to measure distortion levels in color image reproduction. Note here that we are taking the average value of across the whole image. The rest of the report is organized as follows. In Section II, we study the application of on color images. In Section III, further extensions to the direct application of on color components are studied in order to improve the encoder, making use of the correlation between color components. In Section IV, we propose a general framework for applying on color images based on all our findings. We conclude in Section V by reviewing the benefits of our new coding framework. II. in Color Image Coding A. Choice of Color Space As stated in literature [,], we know that components of RGB color space are more correlated than components of YCbCr color space. This suggested using YCbCr color space to apply. A comparison between RGB and YCbCr color spaces was performed. We compressed 'Lena' image with different values of quantization step size (Q) in RGB domain and in YCbCr domain and each time we measured the resulting. Fig. shows the result of this experiment. YCbCr color space gives a better visual quality for the same total rate. We adopted YCbCr through the rest of our work YCbCr RGB Fig.. 'Lena' compressed in RGB and YCbCr

2 B. Chroma Sub-sampling It is known that chroma sub-sampling is used in many image and video compression standards to allocate less rate for chroma components. However, in [] it is stated that subsampling chroma introduces irreversible losses and that performing rate allocation by varying the quantization step size gives you better flexibility. This phenomenon was also observed in our experiments. We applied and DCT with 8x8 block size () to Lena both for sub-sampled and not sub-sampled chroma components. As we see in Fig., the effect of adaptive block transform diminishes the gain difference between the subsampled image and the full-sampled one especially for rates higher than 0. bpp, which are reasonable for image processing applications. Therefore, for, it is generally better not to sub-sample the chroma components Full-sampled Sub-sampled Total rate (bpp) Fig.. Sub-sampling vs. not sub-sampling chroma for Lena C. Color Quality along Directions In this section, we study the visual quality of the color images and how it is affected by the directions along which these colors are aligned. The motivation is that chooses the best direction that represents a certain block via minimizing a cost function. An image of 9 horizontal color stripes was generated. This image is shown in Fig.. The image was rotated every from 0 to 7 and each time it was compressed using and with Q = 60. The visual quality was estimated by measuring for all the compressed images. The results are shown in Fig.. We see that visual quality of images is better than in all directions even the horizontal and vertical directions which are well suited for. Also, DA- PBT significantly outperforms for angles near its supported directions (,,6,, and ). Fig. compares the results for this experiment Fig.. Image used for color quality along directions experiment Performance with Directions Angle ( ) Fig.. Relation between and color directions for and compressed images (Q = 60) Fig.. (left) and (right) compressed images with angle = D. Comparing with Other Transforms We used an image set of five images. Each image was encoded with, DCT with adaptive block size, and with spatial prediction, in which we predict the block from its neighbors and then encode the residual error in prediction. Each image was encoded with the same value of Q for all color components. For each compressed image we

3 measured PSNR for each color component and also measured to express the visual quality. Fig.6. shows the results for image 'Lena'. outperforms all non-directional transforms in PSNR for all color components as well as. with spatial prediction has better results than applied on the actual block because it takes advantage of predictive coding. PSNR (db) Y-component PSNR (db) Average Cb/Cr Lena DCT SP (a) Lena DCT 8 SP (b) Lena DCT SP (c) Fig.6. Comparing different transforms on 'Lena' image (a) measurement (b) PSNR Y component (c) Average PSNR Cb/Cr components E. Rate Allocation With the separation in color space, we can allocate rate across the three color components. For equal quantization step size, we observed that Y component has the highest rate. On the other hand, Cb & Cr have relatively the same rate. In addition, the subjective qualities of the Cb & Cr components are quite similar to each other than they are to Y component. This implies that Cb & Cr should have the same quantization step size. Thus, we set up an experiment to explore how we should allocate rate across Y vs. Cb & Cr. To find the best ratios, we test different ratios of Y rate over a given total rate, and measure. In Fig.7, we observe that the optimal value is obtained when the ratio of Y is 0.6~0.7, and corresponds to the rate of (::) for (Y:Cb:Cr). Thus, we should allocate rate in this ratio. We also observed that the quantization step size for Cb & Cr components achieving this ratio is less than that of Y component, and is roughly a little more than half of Y component quantization step size. The effect of quantizing Y component more is seen in blocking artifacts which can be easily removed using simple deblocking scheme bpp 0.9 bpp 0.8 bpp 0.7 bpp 0.6 bpp 0. bpp Y rate/ Total rate Fig.7. vs. Y rate over total rate for the image Goldhill III. Extensions for Encoder Improvement In this section, we investigate modifications to the direct application of on color components. The aim of this part is to collect the best tools in one framework for the encoding process. A. Effect of Quantization Matrix JPEG standard [,6] increases the compression ratio by assigning a higher quantization step size for higher frequencies. We tried to extend this to working on color images but the problem is that the localization of high and low frequencies is no longer valid because of the directions and partitions defined by the transform. We tried encoding Y

4 component in while encoding Cb & Cr components in DCT with adaptive block size using quantization matrices that are extended versions from the default quantization matrix [6] used for Cb & Cr in JPEG standard. In order to judge the resulting images, we compared them to ones where all color components are encoded in. We varied the quantization step sizes until we get an equal total rate between images under comparison. We found that images where only Y component is encoded and Cb & Cr are 'DCT with quantization matrices' encoded always give higher than images where all components are encoded. This suggests using fixed Q with algorithm. B. Joint Encoding for Cb & Cr Color Components We exploit the possibility of encoding directions and modes jointly in different color components. Since color transform is applied on each pixel individually, the spatial positioning is preserved after color transformation. Examining the color transformed image revealed different energy distributions in luminance and chrominance domains. This suggests that joint direction and mode decision is applied to Cb & Cr only. We thus incorporated the joint direction and mode decision. When encoding the image, same directions are applied to the two color components. The advantage is that we reduce the overhead of sending best modes and directions because we are using one overhead for both color components. Table.. shows the rate reductions that were achieved when applying joint encoding for Cb & Cr color components. Q = Rate (Cb,Cr) (bpp) Joint Rate (bpp) Rate [overhead] [overhead] Reduction House (0.07, 0.09) % [0.0, 0.0] [0.0] Kodak (0.09, 0.070) 0..9% Image 8 [0.00, 0.0] [0.00] Lena (0.069, 0.07) 0.8.% [0.0, 0.0] [0.0] Goldhill (0.0, 0.06) [0.00, 0.0] 0. [0.0].% TABLE. Comparison between individually and jointly encoding Cb & Cr C. Fast Mode Decision for Color Image Coding In exploring the spatial relation between color components, we also found that the optimal block sizes are related in luma and chroma components. For example, if the best block size for Y at certain location is not 6x6 but further subdivided, this suggests that there is an edge in the image at that location and thus the best block size for Cb & Cr is most probably also subdivided. We developed a rule for faster mode decision for Cb & Cr color components. After we encode luma component, we pass the optimal modes to the Cb & Cr encoder. When the encoder searches for the best mode for chroma, it follows the rules stated in Fig.8. In order to show the effectiveness of these rules, we looked at the optimal modes for Cb & Cr for our image set and calculated for how many blocks these rules skip the optimal mode. From a set of 760 blocks, only 6 blocks violated the rules. The degradation in rate and was negligible. Fig.9. shows an example of an image containing sub-optimal blocks for its Cr component. We can say that the blocks violating the rules are always less than % of the total number of blocks. if Luma = 6x6 search Chroma 6x6 mode only else if Luma = 6x8 or 8x6 search Chroma 6x6, 6x8, 8x6 else if Luma = 8x8 search Chroma no smaller than 8x8 Fig. 8 Rules for faster mode decision Fig. 9 Y (left) and Cr (right) optimal block sizes with blocks violating the rule D. Effect of Post-filtering The most perceptible degradation that we see in compressed images at very low rates is the blocking artifacts. This encouraged us to study the effect of applying a deblocking filter on the compressed images that use a high value of Q. We used a deblocking filter called 'UnBlock' implemented in [7]. The filter was applied as a post-processing operation on images compressed using Q = 60. The values for these images compressed with and are shown in Table.. We can see that in general there is an improvement in the visual quality, and in most cases, this improvement is more in the case of. The image 'Lena' before and after filter is shown in Fig.0. The filter improved the visual quality by successfully removing the blocking artifacts. Image Before After Diff. Before After Diff. Lena Airplane House Goldhill TABLE.. values before and after deblocking filter for Q = 60

5 Fig.0. 'Lena' before deblocking filter (left) and after filter (right), compressed (Q = 60) IV. Proposed Framework for in Color Image Coding Based on all the previous results, the following framework is proposed for the application of to color images, as shown in Fig.. The image is first transformed to YCbCr color space. Rate allocation unit decides the best values of Q for the Y component and the Cb & Cr components. These values are supposed to achieve the :: ratio in rate allocation. Y component is encoded first, and then the best modes information is passed to chroma encoder for applying the fast mode decision rule. The chroma encoder has the option to encode Cb & Cr either jointly or independently, based on the image characteristics. Then, all the encoded data is multiplexed into one bit stream. For the decoder side, postprocessing is recommended for visual quality improvement. Fig. Sample results of the proposed framework Sample results of the proposed framework are shown in Fig.. Fig. (a) is Airplane original image. Fig. (b) is Airplane image compressed for min. and total rate of 0. bpp. This image was jointly encoded for Cb & Cr components and the deblocking filter was applied on the resulting compressed image. Fig. (c) is Goldhill original image. Fig. (d) is Goldhill image compressed with the same parameters used for Fig. (b) V. Conclusion We first investigated how works on color images. We found that works well in YCbCr color space without sub-sampling the chroma components. Under the same quantization level, has less color degradation along edges, and outperforms DCT and. We also explored rate allocation, and found that a rate ratio of :: for Y:Cb:Cr provides better visual quality, which is generally achieved by setting larger quantization step size for Y component. We have also shown that quantization matrices are not improving the results. Furthermore, we proposed a framework for applying in color image coding. Rate allocation is applied and provides the quantization step sizes to minimize. Joint encoding for Cb & Cr components is optionally used to reduce the overhead, with fast mode selection assisted from independently encoding Y component. Finally, deblocking filter is applied to reduce the blocking artifacts. Fig.. Framework for in color image coding

6 VI. Acknowledgments The authors would like to thank Chuo-Ling Chang for his guidance throughout the project, Dr. Joyce Farrell and Jiajing Xu for their help in SCIELAB measurements. Thanks also to Prof. Bernd Girod, Dr. Markus Flierl and David Varodayan for their helpful suggestions. VII. References [] C.-L. Chang and B. Girod, Direction-Adaptive Partitioned Block Transform for Image Coding, submitted to IEEE International Conference on Image Processing, 008 [] D. S. Taubman and M. W. Marcellin, JPEG000: Image Compression Fundamentals, Standards and Practice, Kluwer Academic Publishers, 00 [] G. Sharma, Digital Color Imaging Handbook, CRC Press, 00 [] J. Guo, P. Shrivastava, K. Kepley, S. Yang, S. Mitra, B. Nutter, Bit-rate allocation control and quality improvement for color channels in HMVQ image compression, in Proceedings of the 7th IEEE Symposium on Computer-Based Medical Systems, 00 [] X. Zhang and B. Wandell, A spatial extension to CIELAB for digital color image reproduction, in Proceedings of Society for Information Display Symposium, San Diego, 996 [6] ISO/IEC 098- ITU-T Recommendation T.8, Information Technology: Digital Compression and Coding of Continuous-Tone Still Images, 99. [7] J. P. Costella, The UnBlock algorithm, January 006, block_paper.pdf Work Distribution The work in the project in evenly distributed among the two group members. Task Contribution Sam Tsai Mina Makar.a. Choice of Color Space 0% 0%.b. Chroma Sub-sampling 70% 0%.c. Color Quality 0% 80% along Directions.d. Comparing with 0% 0% Other Transforms.e. Rate Allocation 0% 0%.a. Effect of Quantization 0% 0% Matrix.b. Joint Encoding for Cb & Cr 80% 0% Color Components.c. Fast Mode Decision for 0% 0% Color Image Coding.d. Effect of Post-filtering 0% 70%. Proposed Framework for 0% 0% in Color Image Coding Powerpoint Preparation 0% 0% Report Writing 0% 0% 6