IEICE TRANS. INF. & SYST., VOL.E89 D, NO.3 MARCH 2006 1231 PAPER Grayscale Image Segmentation Using Color Space Takahiko HORIUCHI a), Member SUMMARY A novel approach for segmentation of grayscale images, which are color scene originally, is proposed. Many algorithms have been elaborated for a grayscale image segmentation. All those approaches have been discussed in a luminance space, because it has been considered that grayscale images do not have any color information. However, a luminance value has color information as a set of corresponding colors. In this paper, an inverse mapping of luminance values is carried out to CIELAB color space, and the image segmentation for grayscale images is performed based on a distance in the color space. The proposed scheme is applied to a region growing segmentation and the performance is verified. key words: image segmentation, color space, grayscale image, region growing 1. Introduction Image segmentation is to partition the images into meaningful parts that are relatively homogenous in certain sense. In a large number of applications in image processing and computer vision, segmentation plays a fundamental role as the first step before applying to images higher-level operations such as recognition, semantic interpretation and so on. Many image segmentation techniques have been proposed mainly for grayscale images [1] [5]. The reason is that, although color information permits a more complete representation of images and a more reliable segmentation of them, processing of color images requires computation times considerably larger than those needed for grayscale images. Since relatively inexpensive color cameras are nowadays available, there has been a remarkable growth of algorithms for segmentation of color images in the last decade [5], [6]. Of course, color information becomes more effective than monochrome information. So, if it is possible to get a color image, the color image must be used for segmentation. However, according to various restricted conditions, monochrome cameras are still used in many cases in the industrial world, and segmentation for grayscale images remains as an important subject. Figure 1 shows an example of well-segmented result for a color image. When a color image in Fig. 1 (a) is given, human will classify the image into blue and red roughly as shown by yellow line in Fig. 1 (b). Then a segmentation algorithm will be designed so that a machine may also segment the color image as shown in Fig. 1 (b). Figure 2 (a) Manuscript received March 1, 2005. Manuscript revised September 28, 2005. The author is with the Department of Information and Image Sciences, Chiba University, Chiba-shi, 263 8522 Japan. a) E-mail: horiuchi@faculty.chiba-u.jp DOI: 10.1093/ietisy/e89 d.3.1231 shows the grayscale image converted from Fig. 1 (a) by the YIQ conversion. That is, if the color image in Fig. 1 (a) is taken an image with a monochrome camera, the grayscale image in Fig. 2 (a) will be obtained. When grayscale image in Fig. 2 (a) is given, human will classify the image into high luminance value and low one roughly as shown in Fig. 2 (b). Then the conventional segmentation algorithms have been designed so that a machine may also segment the grayscale image as shown in Fig. 2 (b). However, in this paper, we try to segment the grayscale image in Fig. 2 (a) as same as the segmented image in Fig. 1 (a). Since objects in a scene are originally distinguished by the color, the desire which brings the segmented result for a grayscale image close to the segmented result for a color image is very natural. All of the conventional segmentation techniques for grayscale images have been developed in the luminance space, because it has been considered that grayscale images do not have any color information. However, an luminance value has actually color information as a set of corresponding colors. Recently, there are a few studies for investigating the relation between luminance values and colors [7] [9]. Thus, we assumed that segmentation techniques developed for color images could be used for the segmentation for grayscale images by carrying out the inverse mapping of the grayscale images into color space. In this paper, we propose a novel scheme for grayscale image segmentation using CIELAB color space. In the case for color images, many techniques have been proposed using CIELAB space [10], [11]. But, there are not any techniques for grayscale images. In our approach, each luminance value on a grayscale image is mapped into a set of components in CIELAB color space. Since luminance values were determined by original color values, we call the image mapping as an inverse mapping. Then the segmentation is performed by using the color information. In Sect. 2, we present the concept of our segmentation. A spatial segmentation algorithm based on region growing is then proposed in Sect. 3. Experimental results are demonstrated in Sect. 4. We conclude with some future directions in Sect. 5. 2. Concept of the Proposed Approach In this paper, we try to express the grayscale images in color space. At first, we need to know what conversion formula transferred the color image into the grayscale image. Then, we consider the inverse mapping from luminance space into color space. In this paper, we suppose the YIQ conversion. Copyright c 2006 The Institute of Electronics, Information and Communication Engineers
1232 IEICE TRANS. INF. & SYST., VOL.E89 D, NO.3 MARCH 2006 The YIQ system is the color primary system adopted by NTSC for color television broadcasting. By using the YIQ conversion, the luminance value Y is calculated by the following formula [5]: Y = 0.299R + 0.587G + 0.114B. (1) By Eq. (1), when an luminance value Y is given, a corresponding plane is determined in the RGB color space. This is just essential color information given by luminance information. Several color representations are currently in use in color image processing. The most common is the RGB space where colors are represented by their red, green and blue components in an orthogonal Cartesian space. However, the RGB space though are not perceptually uniform; this means that differences among color perceived by the human eye as begin of the same entity are not mirrored by similar distances between the points representing those colors in such space. In this paper, we deal with a uniform color space CIELAB. Ideally the color image would be a characterized linear RGB. In order to convert the RGB space into the CIELAB space, XYZ tristimulus coordinates should be calculated through device characterization (or assumption if necessary). The coordinates are determined empirically. For instance, the matrix for NTSC receiver primary system is: X Y Z = 0.607 0.174 0.200 0.299 0.587 0.114 0.000 0.066 1.116 R G B. (2) If the XYZ coordinates are known, CIELAB value is determined as: L = 25(100Y/Y 0 ) 1/3 16, a = 500 [ (X/X 0 ) 1/3 (Y/Y 0 ) 1/3], (3) b = 200 [ (Y/Y 0 ) 1/3 (Z/Z 0 ) 1/3], where (X 0, Y 0, Z 0 ) are XYZ values for the reference white. From Eqs. (1) to (3), each luminance value Y in a grayscale image can be expressed as an L a b plane in the CIELAB space. Figure 3 shows all plotted luminance values in the CIELAB space (RGB gamut). The z-axis shows L. Then the plane for each intensity lies at right angles to the z-axis. By using this space, segmentation techniques for color images can apply to the grayscale images. In the next section, a segmentation method based on region growing is introduced as one of examples. 3. Proposed Segmentation Algorithm Based on Region Growing In this section, a segmentation method based on region growing approach using CIELAB color space is introduced. The region growing is one of the basic segmentation approaches and it can be considered a sequential clustering or classification process. The main advantage is that the regions obtained are certainly spatially connected and rather compact. Typical steps of region growing are as follows: (1) Selecting seed pixels within the image. (2) Each seed pixel grows a region based on a criterion for homogeneity. The homogeneity is an important property in region growing. Conventional approaches of segmentation for grayscale images have discussed the criterion in the luminance space. For example, the simplest homogeneity criterion uses a difference of luminance values between a seed pixel and a target pixel which is examined whether it belongs to the region or not. In this paper, a new homogeneity criterion is introduced using CIELAB color space. As a measure of color difference between (L 1, a 1, b 1 ) and (L 2, a 2, b 2 ) in CIELAB space, E ab = (L 1 L 2 )2 + (a 1 a 2 )2 + (b 1 b 2 )2 is generally used. It must be natural to use the distance as the homogeneity criterion. However, each luminance value Y is not expressed as a certain point in the CIELAB space, because of the relation of one-to-many inverse mapping. Therefore, the difference E ab is incalculable. But, Y = const. is still a plane in the CIELAB space (which is not true for X = const and Z = const). So, we have only to define a distance between planes. In this paper, a representative color on a plane is defined by the center of gravity instead of luminance value which is (a = b = 0). Then, the Euclidian distance between the gravities is defined as the homogeneity criterion instead of luminance difference. Of course, the color on the center of gravity is actually not true. But, the purpose of this study is not estimation of the color from luminance value. We believe that the center of gravity is more natural in the CIELAB space than luminance value in a partial place, and the center of gravity is suitable as the representative color with the same luminance value. Figure 4 illustrates the definition of the proposed distance. In Fig. 4, Y shows the luminance difference between two pixels in XYZ space. In this case, Y 1 = Y 2. Each luminance Y was defined in Eq. (1). Another luminance difference L is the difference in the CIELAB space. Each luminance L was calculated by Eq. (3). In this case, L 1 > L 2. All conventional approaches have been developed in those spaces. In both spaces, an addition theorem is satisfied, i.e., Y 3 = Y 1 + Y 2 and L 3 = L 1 + L 2. On the other hand, the proposed distance does not satisfy the addition thorem. The proposed distance can be shown as E in Fig. 4. As well as the property E 3 E 1 + E 2, E 3 < E 1 can be obtained by the proposed distance. Each circle in Fig. 4 shows a gamut with the same luminance. Since there is really deflection of gamut for luminance value as shown in Fig. 3, the proposed distance must be appropriate. Figure 5 illustrates the characteristics of each distance with real values. The color shows the center of gravity. In this example, Y 1 = Y 2 = 20 and L 1 > L 2. However, E 1 E 2. So, conventional segmentation algorithms decide that a pair of the right side in Fig. 5 is very likely to be a region. However, the proposed algorithm asserts that it is the objection.
HORIUCHI: GRAYSCALE IMAGE SEGMENTATION USING COLOR SPACE 1233 (a) Color image. Fig. 1 (b) A segmented result. Well-segmented result for a color image. Fig. 5 (a) The grayscale image for Fig. 1 (a). Fig. 2 An example that the luminance difference and color difference. (b) A segmented result. Conventional well-segmented result for a grayscale image. Fig. 6 Fig. 3 Plotted luminance values in CIELAB space. Fig. 4 Definition of distances. Thus by setting the color difference E ab which is the criterion of homogeneity, a homogeneity range in the luminance space is determined for each Y. Figure 6 shows the Plotted luminance values in CIELAB space. calculated homogeneity range of 8 bit luminance in the case Eab = 5. By Fig. 6, if Y = 184, the range becomes [ 20, 6]. This shows that if the luminance value of a seed pixel is 184, a target pixel, which is usually selected from adjacent pixels, with the luminance value within [164, 190] is merged by the region of the seed pixel. Though the homogeneity range of conventional techniques is constant, the range of the proposed method changes depending on the luminance value Y. The proposed segmentation algorithm is as follows: (STEP 0) Set the homogeneity criterion which is a tolerance color difference Eab in CIELAB space, and calculate a corresponding homogeneity range. (STEP 1) Select seed pixels within an input grayscale image. In our method, the seed is selected randomly from pixels with the most frequent luminance value in the grayscale histogram. (STEP 2) The seed pixel grows a region to a target pixel if the luminance value of the target pixel is within the homogeneity range. If the region growing stops, back to step (STEP 1). Note that in this algorithm, any region merging algorithms are not used after region growing.
1234 IEICE TRANS. INF. & SYST., VOL.E89 D, NO.3 MARCH 2006 4. Experimental Results The proposed algorithm was tested on a variety of images. Figure 7 shows the results of the segmentation on the image in Fig. 2 (a). The homogeneity criterion was set as E ab = 10. Considering with Figs. 1 and 2, the results in Fig. 7 are quite suitable for our aim by segmented in CIELAB color space. Next, a comparison experiment with the proposed technique and a conventional technique was conducted using natural scenes. As the conventional technique, we used a typical region growing method [5] of which homogeneity range is constant. The followings are criteria: (1) The grayscale difference between any pixel and a seed had to be less than a constant threshold. In the proposed method, the threshold can be changed dynamically depending on the grayscale value. (2) The pixel had to be 8-connected to at least one pixel in the region. Figure 8 (a) shows an original color image (with size 768 512) and Fig. 8 (b) was converted from Fig. 8 (a) by the YIQ conversion. For this test image, we focused on two parts such as bright and dark parts. One is boundaries between the sky and the clouds, and another is boundaries of a door and a pillar of the shack. Figure 9 shows enlarged image of the shack. The chroma is made high in order to make it easy to understand. It turns out that there are a door and a pillar of the shack. Segmented results by the conventional method are shown in Fig. 10. In Fig. 10 (a), the range was set as [ 5, 5]. The result shows that the door and the pillar are segmented, but the result is surely an over-segmentation. Figure 10 (b) and (c) show the results by setting the threshold range [ 25, 25] and [ 35, 35], respectively. Though the number of over- segmented regions decreased, the door and the pillar cannot be segmented. As compared with the conventional segmented results, Fig. 9 Enlarged image of Fig. 8 (a). (a) Constant range [ 5, 5]. Fig. 7 Segmented result for Fig. 2 (a) by the proposed method. (b) Constant range [ 25, 25]. (a) (b) Fig. 8 An example of test images. (a) A color image. (b) The grayscale image of (a) by YIQ conversion. Fig. 10 (c) Constant range [ 35, 35]. Segmented results by a conventional method.
HORIUCHI: GRAYSCALE IMAGE SEGMENTATION USING COLOR SPACE 1235 Table 1 Quantitative evaluation by FOM. (a) For the color image. (b) For the grayscale image. Fig. 12 Examples of reference image which were segmented by a user. Fig. 11 (a) E ab = 20. (b) E ab = 25. (c) E ab = 30. Segmented results by the proposed method. proposed method obtained suitable segmented results as shown in Fig. 11. Figure 11 (a) shows a result by setting the homogeneity criterion as E ab = 20. Over-segmentation is also found as same as Fig. 10 (b). As increasing the threshold distance E ab, the number of over-segmented regions decreased. Moreover, we can confirm that both bright and dark parts are segmented appropriately. In order to perform a quantitative evaluation, the wellknown Pratt s Figure of Merit (FOM) [12] was used to com- pare segmentation results. The FOM attempts to balance three types of errors that can produce erroneous edge maps: missing valid edge points, failure to localize edge points and classification of noise fluctuations as edge points. Here, the edge means the boundary of segmented regions. The FOM is defined as R = 1 I N I A i=1 1 1 + ad 2. In this equation, I N is the maximum of I A and I I. I A represents the total number of actual edge pixels; i.e., those edge pixels that were found. I I represents the total number of ideal pixels in the image; i.e. the number of edge pixels in the reference image. The parameter a is a scaling constant while d is the distance from an actual edge point to the nearest ideal edge point (in this paper a = 0.9). The scaling factor is used to penalize edges that are localized, but offset from the true position. The rating factor is normalized so that a value of one means that the edge has been detected perfectly. The FOM is normalized with the maximum of the actual I I < I A and ideal number of edge pixels in order to ensure a penalty for smeared (i.e., I I < I A )or fragmented edges (i.e., I A < I I ). The reference segmented image was constructed by five users. They segmented the image by piecewise polygons. In order to investigate the property, each user segmented both a color image and its grayscale image. Figure 12 shows a set of reference images which were segmented by a user. Table 1 summarizes the average of FOM by five users. For color image, the FOM values of the proposed method were higher than the values of the conventional method. For the grayscale image, an opposite result was obtained. The result shows that the conventional method is suitable to just divide a grayscale image. But, the proposed method has possibility to perform image segmentation to original scene faithfully, if the original scene of the grayscale image was color. (4)
IEICE TRANS. INF. & SYST., VOL.E89 D, NO.3 MARCH 2006 1236 (a) (b) Fig. 13 An example of test images. (a) Eab = 15. (a) Constant range [ 15, 15]. (b) Eab = 20. (b) Constant range [ 20, 20]. (c) Eab = 30. Fig. 15 (c) Constant range [ 40, 40]. Fig. 14 Segmented results by a conventional method. We show other test data. Figure 13 (a) shows an original color image (with size 240 160) and Fig. 13 (b) was converted from Fig. 13 (a) by the YIQ conversion. For this Segmented results by the proposed method. test image, we focused on bright and dark parts, that is, boundaries between the sky and the clouds, and boundaries between the mountain and the tree. Segmented results by the conventional method are shown in Fig. 14. In Fig. 14 (a), the range was set as [ 15, 15]. The result shows an over-segmentation. Figure 14 (b) and (c) show the results based on the range [ 20, 20] and [ 40, 40], respectively. Though the number of oversegmented regions decreased, a lower left tree and mountain were merged as shown by the red circle. As compared with this, the results by proposed method obtained suitable segmented results as shown in Fig. 15. Figure 15 (a) shows a result by setting the homogeneity criterion as Eab = 15. Over-segmentation is also found as
HORIUCHI: GRAYSCALE IMAGE SEGMENTATION USING COLOR SPACE 1237 same as Fig. 14 (a). As increasing the distance threshold E ab, segmentation of sky is appropriate and does not occur the region merging between the tree and the mountain as shown in Fig. 14 (b) and (c). lip images by spatial fuzzy clustering, IEEE Trans. Fuzzy Syst., vol.11, no.4, pp.542 549, 2003. [12] I.E. Abdou and K. Pratt, Quantitative design and evaluation of enhancement/thresholding edge detectors, Proc. IEEE, vol.67, no.5, pp.753 763, 1979. 5. Conclusion In this paper, a new segmentation approach for grayscale images was presented. In our approach, each luminance values in the grayscale image were mapped inversely into a color space. Then segmentation is performed by considering with a homogeneity criterion in the color space. As one of the algorithms, a simple region growing method by using CIELAB color space was proposed. In our experiments, some effectsby usingcolorspace wereconfirmed. However, there is a limit in the simple region growing. For example, the proposed technique cannot apply to texture images because of the property of the region growing. Recently, many techniques for color segmentation have been proposed. The proposed concept can apply to those techniques. In this paper, we calculate the distance in the color space using gravities of mapped colors. It may not be suitable for any pictures to use the center of gravity. Of course, the segmented results depend on the input image. This paper only found out the new possibility. It is a future subject to investigate the relation between image property and the proposed approach. Takahiko Horiuchi received his B.E., M.E. and Ph.D. degrees from University of Tsukuba in 1990, 1993 and 1995, respectively. He was a member of the Promotion of Science for Japanese Junior Scientists from 1993 to 1995. From 1995 to 1998, he was an Assistant Professor with the Institute of Information Sciences and Electronics, University of Tsukuba. From 1998 to 2003, he was an Associate Professor with the Faculty of Software and Information Sciences, Iwate Prefectural University. In 2003, he moved to Chiba University. He is an Associate Professor at Dept. of Information and Image Sciences. He is a member of IEEE, IS&T and IIEEJ. Acknowledgments Part of this work has been supported by the Tateisi Science and Technology Foundation in Kyoto, Japan. References [1] R.M. Haralick and L.G. Shapiro, Image segmentation techniques, Comput. Vis., Graph. Image Process., vol.29, no.1, pp.100 132, 1985. [2] K.S. Fu and J.K. Mui, A survey on image segmentation, Pattern Recognit., vol.13, pp.3 16, 1981. [3] A. Rosenfeld and A. Kak, Digital Picture Processing, vol.2, Academic Press, New York, 1982. [4] N.R. Pal and S.K. Pal, A review on image segmentation, Pattern Recognit., vol.26, no.9, pp.1277 1294, 1993. [5] R. Gonzalez and R. Woods, Digital Image Processing, Second ed., Prentice Hall, 2001. [6] W. Skarbek and A. Koschan, Color image segmentation A survey, Technisher Bericht 94-32, Technical University of Berlin, 1994. [7] T. Welsh, M. Ashikhmin, and K. Mueller, Transferring color to grayscale images, Proc. ACM SIGGRAPH 2002, pp.277 280, 2002. [8] T. Horiuchi and S. Hirano, Colorization algorithm for grayscale image by propagating seed pixels, Proc. IEEE ICIP 2003, pp.457 460, 2003. [9] T. Horiuchi, Colorization algorithm using probabilistic relaxation, Image Vis. Comput., vol.22, no.3, pp.197 202, 2004. [10] L.Xu,M.Jackowski,A.Goshtasby,C.Yu,D.Roseman,S.Bines, A. Dhawan, and A. Huntley, Segmentation of skin cancer images, Image Vis. Comput., vol.17, no.1, pp.65 74, 1999. [11] A.W.-C. Liew, S.H. Leung, and W.H. Lau, Segmentation of color