Color Image Encoding Using Morphological Decolorization Noura.A.Semary

Fifth International Conference on Intelligent Computing and Information Systems (ICICIS 20) 30 June 3 July, 20, Cairo, Egypt Color Image Encoding Using Morphological Decolorization Noura.A.Semary Mohiy.M.Hadhoud Faculty of Computers and Information Faculty of Computers and Information Menoufia University Menoufia University Shebeen ElKom Egypt Shebeen ElKom Egypt 002063433040 002006639290 noura.samri@ci.menofia.edu.eg mmhadhoud@yahoo.com Alaa.M.Abbas Faculty of Electronic Engineering Menoufia University Monouf Menoufia-Egypt 0020934404 aladin_abbas@yahoo.com ABSTRACT Image colorization is a new image processing topic which refers to recolor gray images to look like the original color images as possible. Different methods appeared in the literature to solve this problem, the way which leads to thinking about decolorization which means eliminating the colors of color images to just small color keys, aid in the colorization process. Due to this idea, decolorization is considered as a color image encoding mechanism. In this paper we propose a new decolorization system depends on extracting the color seeds using morphology operations. Different decolorization methods was studied and compared to our system results using different quality metrics. Categories and Subject Descriptors I.4.2 [IMAGE PROCESSING AND COMPUTER VISION]: Compression (Coding)--- Approximate methods I.4.3 [IMAGE PROCESSING AND COMPUTER VISION]: Enhancement ---Grayscale manipulation I.4.8 [IMAGE PROCESSING AND COMPUTER VISION]: Scene Analysis---Color I.4.0 [IMAGE PROCESSING AND COMPUTER VISION]: Image Representation---Morphological General Terms Algorithms, Measurement, Performance, Reliability, and Experimentation. Keywords Colorization, Decolorization, Morphology, Compression, Encoding.. INTRODUCTION Image colorization is the process of adding colors to black and white images. One of the problems of colorization is how to select the suitable colors, mainly the original colors, to generate a colored image similar to the original color one. Different colorization methods treats this problem in the literature, most of these trials are covered and discussed in our previous publications Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Conference 04, Month 2, 2004, City, State, Country. Copyright 2004 ACM -583-000-0/00/0004 $5.00. [0,]. One of the common colorization methods is Anat Levin.et.al technique [7] where the user draws some scribbles on a gray image using a color palette and a brush like tool, then the computer colorizes the image using optimization depending on the fact, the nearby pixels in gray should have the same color. Levin's method depends on user selection of the position and the color of the scribbles. So it's an important aspect in his method to practice well; where to draw the scribbles and what is the suitable color should be selected to obtain a color image as the original one. Because of these drawbacks, many researchers like Yao Li et.al [8] and Liron. Yatziv et.al [2] and more appeared in the literature; improve Levin's algorithm quality; speed and the needed number and positions of the scribbles. Decolorization is a new research field appeared after the fast development in image and video Colorization area. Decolorization means to convert the color image to gray by eliminating the colors in the image but keeping some keys refer to the original colors to be used in the Recolorization process; to obtain a recolored image looks like the original one. From our point of view and literature study, there are two trends for decolorization:. Color Embedding: Where the chromatic channels are processed to be hidden or embedded in the gray image. The inverse of the hiding process is performed to extract the color channels back and merge them with the gray (lightness channel) to obtain the color image again, but with less quality from the original one. This field leads to thinking about encoding color images using decolorization. Ricardo [4][5] uses the wavelet sub-bands to hide chromatic channels. The resulted gray image is a textured one differs a lot from the original gray image but they proposed their method for faxing applications that enables sending the color image by the traditional fax and recolorize them back after receiving. 2. Automatic scribbles or seeds selection: where the decolorization process usually made automatically without user scribbles selection [6]. The algorithm searches for the best scribbles or seeds can be extracted and sent with the gray image to be used in the colorization process. T.Miyata.et.al [9] proposed a decolorization methodology depends on extracting line segments from the color image to be the scribbles for the colorization process. Their process based on Levin's method, where the colorization process is a basic block in their system framework what leads to a lot of computational time. The authors compared their algorithm results with Cheng's method [3] where the last one algorithm uses the active learning feedback to propagate the selection 39

Fifth International Conference on Intelligent Computing and Information Systems (ICICIS 20) 30 June 3 July, 20, Cairo, Egypt of the seeds and extracted seeds (pixels) instead of lines and scribbles. Another simple method for automatic seeds selection using up sampling was proposed by Stephen Brooks [2]. After this short survey on the decolorization in the literature, we propose a new decolorization system by automatically seeds selection method depends on Image Morphological Operations. Our proposed technique has the following advantages: Extracts seeds not scribbles, which results in less number of pixels and thereby a good color compression technique. The extracted seeds are selected well to suit A.Levin [7],Y.Li [8] and L.Yatziv [2] colorization systems or any colorization system depends on seeds colorization. There is no learning or feedbacks what make it very fast seed selection algorithm. The system parameters affect the quality of both decolorization and colorization but even with high compression ratio/low quality parameters the colorization process obtains very good results. The system was implemented in matlab9 and tested on standard images. PSNR, MSE and Colorfulness were measured for quality assessment. A comparison between our method and T.Miyata [9] and Cheng's [3] methods were performed according to the compression aspect, and between our system and Levin's [7] and Liron's [2] according to the quality. 2. FUNDAMENTALS OF MORPHOLOGY Morphology is a broad set of image processing operations that process bicolor (black and white) images based on shapes. Morphological operations apply a structuring element to an input image, creating an output image of the same size. In a morphological operation, the value of each pixel in the output image is based on a comparison of the corresponding pixel in the input image with its neighbors according the structure element, which presents the positions of the pixel neighbors. The most basic morphological operations are dilation and erosion. For describing their effect simply, for a binary image presents white objects on black background, dilation adds pixels to the boundaries of the image objects, while erosion removes pixels from objects boundaries. Next subsections present the morphological operations in more details. 2. Dilation It means that, the value of the output pixel is the maximum value of all the pixels in the input pixel's neighborhood (structure element). For a binary image A, a structure element B can be presented as binary matrix or as a group of pair wise positions of s. The dilation of A by B is defined by: A B = U A Eq. b B For example let B is 3 by 3 structure element where: (, ) B = (,0) (,) b (0, ) (0,0) (0,) (, ) (,0) = (,) Eq. 2 So, A dilated by B means that each object pixel (white=) will be replaces by its 9 neighbours, so any white object will be enlarged. 2.2 Erosion Erosion is the inverse operation of dilation. It means that, the value of the output pixel is the minimum value of all the pixels in the input pixel's neighborhood. In a binary image, if any of the pixels is set to 0, the output pixel is set to 0. The erosion of the binary image A by the structuring element B is defined by:ө AӨB = I Eq. 3 A b b B So, for the same structure element in eq.2, the effect of eroding A by B is to remove any foreground pixel that is not completely surrounded by other white pixels. Such pixels must lie at the edges of white regions, and so the white regions will shrink. 2.3 Opening In mathematical morphology, opening is the dilation of the erosion of a set A by a structuring element B: A o B = (A Ө B) B Eq. 4 The effect of the operator is to preserve foreground regions that have a similar shape to this structuring element, or that can completely contain the structuring element, while eliminating all other regions of foreground pixels. 2.4 Closing In mathematical morphology, the closing of a set (binary image) A by a structuring element B is the erosion of the dilation of that set, A B = (A B) Ө B Eq. 5 The effect of the operator is to preserve background regions that have a similar shape to this structuring element, or that can completely contain the structuring element, while eliminating all other regions of background pixels. 2.5 Skeletonization Skeletonization or Medial axis transform means, to reduce all objects in an image to lines, without changing the essential structure of the image. In (Lantuéjoul 977) 2, Lantuéjoul derived the following morphological formula for the skeleton of a continuous binary image A: S(A)= U I [(AӨρB)\(AӨρB)oµB] Eq. 6 ρ > 0 µ >0 where ρb is an open ball of radius ρ. Another way to think about the skeleton is as the loci of centers of bi-tangent circles that fit entirely within the foreground region being considered. Figure. illustrates this for a rectangular shape. Berkeley Benchmark: (Online) http://www.eecs.berkeley.edu/ Research/Projects/CS/vision/grouping/segbench/ 2 Image Analysis and Mathematical Morphology by Jean Serra, ISBN 026372403 (982) 320

Fifth International Conference on Intelligent Computing and Information Systems (ICICIS 20) 30 June 3 July, 20, Cairo, Egypt Figure. Skeleton of a rectangle defined in terms of bi-tangent circles Original Eroded Dilated Inner Boundary Outer Boundary Skelton Figure 2. Morphology operations 3. THE PROPOSED SYSTEM To extract the suitable seeds, after practicing on seeds colorization systems, we found that most manual seeds or scribbles should be put either near the objects boundaries or in the objects centers or both to obtain better colorization results. From this notice, we thought about extracting the boundaries of each color cluster. This step can be performed by different ways. One of the fast ways to extract objects boundaries is by using the morphological operations. Morphology can be used to extract the inner boundary and the outer boundary of a binary image. Where the inner boundary D i is the difference between the original binary image A and the eroded image A e by a structure of size R R of ones and the outer boundary D o is the difference between the original binary image A and the dilated image A d D i = A A e Eq. 7 D o = A d A Eq. 8 Figure.2 presents an example on object after performing different morphology operations. For more accurate colorization, more seeds may be needed in the center of the object, so the skeleton of the image A s is obtained. The skeleton is extracted using morphological skeletonization. The proposed system (figures 3, 4) depends on extracting the image seeds from the boundaries of each color part. This is done by segmenting the color image into clusters. The segmentation is performed in the Cb and Cr chromatic channels of the image. The K-Mean segmentation technique proposed by D. Arthur [] is used due to its computation acceleration. The user needs to select the number of clusters K given to the segmentation technique. The final seeds are extracted from both the inner boundary and/or the skeleton by sampling, with a sampling rate = S. From the above paragraph, the quality and the compression ratio are relatives to the system parameters K, R, and S. More additional parameter M is used for eliminating small clusters less than some threshold. The effect of the system variables on the quality and compression ration is shown in table : Table. Relation between system variables and quality and compression ration Q ~ K CR ~ /K Q ~ /S CR ~ S Q ~ R CR ~ /R Q ~ /M CR ~ M Where, Q means the quality which measured by quality assessments; Peak Signal to Noise Ration (PSNR) and Mean Square Error (MSE) and CR refers to the compression ratio. PSNR = MSE = mn 2 Max.log 0 MSE 0 Eq. 9 m n [ I( i, j) K( i, j) ] i= 0 j= 0 2 Eq.0 The compression ration (CR) measured in the paper is calculated by dividing the size of the encoded image by the size of the original image. Many references use this way to calculate the CR, while other many references use the inverse operation (Size of Original / Size of Encoded), also another formula to display CR is ' N: ' where N = (Size of Original / Size of Encoded). As we care about color compression, common quality metrics like MSE and PSNR are not sufficient. A quality metric called "Colorfulness" measures the quality of the color in the image. The colorfulness metric presented by [3] is used. The colorfulness value for an image is defined as C = where 2 2 σ + σ α α = R G + 0.3 β = (( R + G / 2)) B β 2 2 µ + µ α β Eq. where µ and σ are the mean and standard deviations of the pixel cloud along two axes α and β in a simple opponent space. For comparing the colorfulness quality between an image i (Original) and the recolored image j, the colorfulness metric is defined by: CM(i,j) = C i -C j Eq. 2 Where the smaller the value of CM the closer the recolored image to the original one. Each pixel for which we store color information requires 4 bytes of additional storage: 2 bytes for the color information (the luminance channel is already present in the grayscale image), and 2 bytes to encode its location. 32

Fifth International Conference on Intelligent Computing and Information Systems (ICICIS 20) 30 June 3 July, 20, Cairo, Egypt Original RGB (A) Convert to YCbCr Y (A) Cb, Cr K KMeans Clustering Technique Clustered image (B) (B) Split Clusters to regions (C) Regions (C)... Inner Boundary Extraction (D) Boundary Image (D) S Extract Seeds (E) Marked Image (E) Figure 3 Proposed System Block Diagram Figure 4 Proposed System Steps 322

Fifth International Conference on Intelligent Computing and Information Systems (ICICIS 20) 30 June 3 July, 20, Cairo, Egypt 4. RESULTS AND DISCUSSIONS In this section two types of experiments will be discussed, first; our system results with respect to the quality of seeds selection. In these experiments, our system result will be compared to Levin's [7] seeds colorization quality and Liron's [2]. Experiment (2): The main contribution of Liron is the quality of colorization with minimum number of scribbles that are not sufficient for Liven's system. Figure.9 present one of the images Liron.et.al succeeded to colorize by their system rather than Liven's one. This is a hard test for Levin to put many scribbles in such image. Our system could solve this problem quickly and efficiently figure.0. Experiment (): Figure.5 presents the original image and the seeds used in Levin's paper and compared by Liron's colorization system. Liron was care about the quality of colorization visually without computing any quality metrics. Figure.6 presents a comparison between both Levin's and Liron's results, with quality measures displayed bellow each. Figure.7 presents our system results with inner boundary seeds with quality parameters (K=0, S=%, R=3). From the quality measures presented bellow the figure, our system shows better seeds selection than manually seeds. In figure.8 skeleton seeds are used too to enhance the colorization quality, but this will increase the compression ratio. Figure 5. (left) Original, (right) Levin's scripples Figure 9. (left)liron Seeds, (middle)levin results,(right)liron's Figure 0. Q=(3,0.0,3) :PSNR=37.79, MSE=8.58,CM:.88 The second type of experimentation tests the system compression professionalism. Comparison between our system and L. Cheng [3] and T. Miyata [9] will be discussed in these experiments. Experiment (3): The girl photo 52 683 used in both methods is used in the experiment. The image was scaled by a scale factor 0.5 as performed in T.Miyata [9]. Figure. presents Cheng's results with quality parameters and the numbers of pixels (N) measured and attached to the figure. While Miyata results are presented in figure.2. Figure 6. Results of Levin (left): PSNR=27.78, MSE=33.09, CM=2.349, Liron (right): PSNR=26.42, MSE=33.46, CM= 2.73 Figure. N=2766, CR= 0.754, PSNR=32.2, MSE=7.396, CM=.20 Figure 7. PSNR=36.79, MSE=0.45, CM= 5.44, CR = 0.45 Figure 8. PSNR=38.87, MSE=5.23, CM= 2.32, CR =0.5 Figure 2. N= 856, CR=0.335,PSNR=29.448, MSE=26.23, CM=3.4 323

Fifth International Conference on Intelligent Computing and Information Systems (ICICIS 20) 30 June 3 July, 20, Cairo, Egypt Our system results are presented in figure.3 with parameters (K=5, S=%, R=3). Different system parameters were used to measure the quality of coloring with respect to compression ratio in table.2. Values reflect the effect of system parameters as assumed in table.. Reader should notice that CR usually more than 0.333; the size of gray channel, so the colors compression ratio is the difference. Figure 3. Clustered (left), Seeds (middle), Marked image (right) Table 2. Different system parameters (a) (b) (c) (d) (e) K S R M N PSNR MSE CM CR (a) 5 % 3 00 3598 40.698 4.348.04 0.4 (b) 5 0% 3 00 467 35.97 20.64 4.30 0.345 (c) 7 50% 3 00 26 32.74 33.68 6.30 0.337 (d) 0 50% 3 00 73 37.327 26.675 3.927 0.338 (e) 0 00% 3 00 78 33.06 34.966 5.907 0.336 Experiment 4: Another example from Berkeley DB ; the Quala photo 32 48 is tested by our system with quality (3,%,3) and (4,50%,3) and presented in figure.4 and table.3. Figure 4. Original (left), Decoded(3,,3), Decoded(4,50,3) Table 3. Quala results Q N CR PSNR MSE CM (3, %, 3) 5053 0.377 40.92 5.044 2.79 (4, 50%, 3) 9 0.344 38.267.86 7.3 5. CONCLUSION In this paper we have proposed a new encoding system using morphological decolorization. The proposed system based on automatically color seeds selection from the inner boundaries and the skeleton of the image color clusters. Using morphology, lead to computation time less than methods in the literature besides obtaining the minimum better seeds for efficient recolorization. 6. ACKNOWLEDGMENTS Our thanks to the ambiguous reviewers, who help us completing this paper perfectly. 7. REFERENCES [] Arthur, D. & Vassilvitskii, S. k-means++: the advantages of careful seeding Proceedings of the eighteenth annual ACM- SIAM symposium on Discrete algorithms, Society for Industrial and Applied Mathematics, 2007, 027-035 [2] Brooks, S.; Saunders, I. & Dodgson, N. A. Image compression using sparse colour sampling combined with nonlinear image processing The 9th Symposium on Electronic Imaging (SPIE), January 2007, 6492, -2 [3] Cheng, L. & Vishwanathan, S. Learning to compress images and videos Proceedings of the 24th international conference on Machine learning (ICML), ACM, 2007, 6-68 [4] de Queiroz, R. L. & Braun, K. M. Color to gray and back: color embedding into textured gray images IEEE Transaction on Image Processing, Vol. 5, No. 6, pp., 2006., 2006, 5, 464 470 [5] de Queiroz, R. L. Reversible color-to-gray mapping using subband domain texturization Pattern Recognition Letters, 200, 3, 269 276 [6] Horiuchi, T. & Tominaga, S. Color Image Coding by Colorization Approach EURASIP Journal on Image and Video Processing, 2008, 2008, 9 [7] Levin, A.; Lischinski, D. & Weiss, Y. Colorization using Optimization. Proc. SIGGRAPH 2004 in ACM Transactions on Graphics, July 2004, 23, 689 694 [8] Li, Y.; Lizhuang, M. & Di, W. Fast Colorization Using Edge and Gradient Constraints Proceedings of (WSCG) The 5th International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision 2007. February 2007, 309-35 [9] Miyata, T.; Komiyama, Y.; Sakai, Y. & Inazumi, Y. Novel inverse colorization for image compression Proceedings of the 27th conference on Picture Coding Symposium, IEEE Press, 2009, 0-04 [0] Semary, N. A. Gray Image Coloring Using Texture Similarity Measures Master thesis. Menofia University, 2007 (online) http://www.docstoc.com/docs/585337/gray-image- Coloring-Using-Texture-Similarity-Measures [] Semary, N. A.; Hadhoud, M. M.; Ismail, N. A. & Al-Kelani, W., S. A Texture Recognition Coloring Technique for Natural Gray Images The 2007 International Conference on Computer Engineering & Systems (ICCES'07), November 27-29, 2007 [2] Yatziv, L. & Sapiro, G. Fast image and video colorization using chrominance blending IEEE Transactions On Image Processing, May 2006, 5, 20-29 [3] XIANG, Y., ZOU, B., AND LI, H. Selective color transfer with multi-source images. Pattern Recognition Letters 30, 7 ( May 2009), 682 689 324