Morphological rational multi-scale algorithm for color contrast enhancement

Transcription

1 Morphological rational multi-scale algorithm or color contrast enhancement Hayde Peregrina-Barreto* a, Iván R. Terol-Villalobos b a Universidad Autónoma de Querétaro, 249 Rio Moctezuma, San Juan del Rio, QRO, MEX; b CIDETEQ, Parque Tecnológico Querétaro, San Fandila-Pedro Escobedo, QRO, MEX ABSTRACT Contrast enhancement main goal consists on improving the image visual appearance but also it is used or providing a transormed image in order to segment it. In mathematical morphology several works have been derived rom the ramework theory or contrast enhancement proposed by Meyer and Serra. However, when working with images with a wide range o scene brightness, as or example when strong highlights and deep shadows appear in the same image, the proposed morphological methods do not allow the enhancement. In this work, a rational multi-scale method, which uses a class o morphological connected ilters called ilters by reconstruction, is proposed. Granulometry is used by inding the more accurate scales or ilters and with the aim o avoiding the use o other little signiicant scales. The CIE-u v Y space was used to introduce our results since it takes into account the Weber s Law and by avoiding the creation o new colors it permits to modiy the luminance values without aecting the hue. The luminance component ( Y) is enhanced separately using the proposed method, next it is used or enhancing the chromatic components (u', v ) by means o the center o gravity law o color mixing. Keywords: Color contrast enhancement, rational operations, morphological ilters, mathematical morphology. 1. INTRODUCTION Contrast enhancement plays a undamental role in many image processing tasks. Its main goal consists in improving the visual appearance o an image. However, it also plays a main role in other problems; or example, providing a transormed image in order to segment it. Actually, there are many methods or image enhancement that can be classiied into two categories: spatial domain methods and requency domain methods. In the present work, the study is ocused on spatial domain methods, and particularly on the use o the morphological image processing technique. In mathematical morphology ew works have been made on this subject, even though a well-deined ramework theory or contrast enhancement proposed by Meyer and Serra 1 exists. In their work, an activity ordering to work on a complete lattice is proposed. In order to build the contrast mappings, Meyer and Serra use the morphological opening and closing as primitives and a proximity criterion or selecting them. Several works have been derived rom this theory enabling us to have dierent tools or enhancing images. However, when working with images with a wide range o scene brightness, as or example when strong highlights and deep shadows appear in the same image, the morphological methods previously proposed in the literature do not allow the enhancement o this type o images. Some interesting multi-scale methods adapted or the contrasting o images containing a wide range o brightness levels are derived rom the well-known Retinex approach. The main idea in the Retinex theory 2 addresses the problem o separating illumination rom relectance. In this manner, the Retinex theory deals with compensation o illumination eects in images. In the present work is proposed a rational multi-scale method, derived o the process made by Espino et. al 3, which ollows the Retinex approach. However, instead o using the convolution o the Gaussian to enhance regions in a given scale, as in the Retinex case, a class o morphological connected ilters called ilters by reconstruction is applied. On the other hand, a model or human visual contrast perception called Weber's law is used to build morphological rational operators or detecting the principal regions and to enhance them. Thus, one proposes a ratio between the original image and the opened image to enhance bright structures. Then, the Weber's law is taken into account, and to enhance the regions the opened image serves as background to identiy the principal regions o the original image. *hperegrina@ieee.org; phone +52 (427) ; ax +52 (427) Image Processing: Algorithms and Systems VIII, edited by Jaakko T. Astola, Karen O. Egiazarian, Proc. o SPIE-IS&T Electronic Imaging, SPIE Vol. 7532, 75320Q 2010 SPIE-IS&T CCC code: X/10/$18 doi: / SPIE-IS&T/ Vol Q-1

2 Similarly, a ratio between the original image and the closed image is used to contrast dark structures. In a multi-scale situation, the opened (closed) image at a coarser scale serves as the background or detecting and enhancing structures at a iner scale. Many scales could be applied in a transormation but depending on the image, just some o them imply important results. The importance o knowing those scales is with the aim o avoiding the use o little signiicant scales. Granulometry 4, 5 is used as a tool with the aim o inding the more accurate scales. Once both structures, bright and dark, are enhanced at dierent scales, they are combined in a single inal image. Among the dierent alternatives or the combination, a barycentric linear combination o both images is carried out. The u v Y space, proposed by Lucchese and Mitra 6, was used to introduce our results. Since this space was built using the notion o just noticeable dierence and our proposal or contrasting images takes into account the Weber's Law, which is based on the notion o just noticeable dierence (JND), it seems natural to use this space to develop our approach. Furthermore, this space permits modiy the luminance values without aecting the hue; this avoid the creation o new colors. Thus, colors change only on intensity which permits an improvement but keeping a natural appearance. The luminance component is enhanced separately using the proposed method, next it is used or enhancing the chromatic components (u', v ) by means o the center o gravity law o color mixing. Our results are compared with Retinex showing a better contrast enhancement. Particularly, the proposed method does not modiy the original structures when the improvement is made; in addition, i improvement is not necessary the changes are little signiicant in contrast with Retinex where changes are more noticeable. 2. METHODOLOGY 2.1 Mathematical morphology Morphological ilters have two important properties: they are increasing and idempotent 7, 8. Former property means that the order must be preserved; latter property establish that a transormation ψ is idempotent i and only i ψ[ψ()]= ψ or all image. Opening (γ μb ) and closing (ϕ μb ) are basic morphological ilters; where B is the 3x3 basic structuring element and μ is an homothetic parameter. Thus, the inal size (μb) o the structuring element, which is applied to, is given by (2μ+1)(2μ+1). These ilters are based on the morphological basic transormations: dilation (δ μb ) and erosion (ε μb ). Erosion is expressed as ε μb = {(y): y μb} where is the lowest value, and dilation is expressed as δ μb = {(y): y μb} where is the higher value. Then, γ μb = δ ε ] μb[ μb ϕ μb = ε δ ] (1) μb[ μb Erosion and dilation are used or delete or remark the image structures; nevertheless, the modiication o original structures is an undesirable eect on image improvement. Mathematical morphology has other kind o ilters called ilters by reconstruction 9, 10 which main eature is to conserve the original structures. These ilters are built by iterating until idempotence the dilation and erosion by reconstruction, which are deined as δ 1 ( g) = δ ( g) with g and 1 ε ( g) = ε B( g) with g, respectively. Where g is the resulting image by applying an erosion or dilation to and it is called marker. Based on this, it is possible to deine the opening and closing by reconstruction as, B n ϕ μb ( ) = limε [ δ μb ( )] = ε ε... ε [ δ μb( )] n n γ ( ) = limδ [ ε ( )] = δ δ... δ [ ε ( )] μb n μb μb (2) Remark: Frequently, B is omitted due to its size is known and constant. Thus, the expression ϕ μb is equivalent to ϕ μ ; in the same way i μ=1 then ϕ μb =ϕ μ =ϕ. 2.2 Color properties and the u v Y color space Colors have some properties which allow distinguish one rom another. On color image improvement, these properties must be manipulated, so, it is necessary to know and to dierence them. There are three important properties: hue, SPIE-IS&T/ Vol Q-2

3 saturation and luminance 11, 12. Hue is associated with the dominant wavelength in a mixture o primary colors; or example, the green yellowish and the green bluish are two dierent hues o the green. Saturation reers to the intensity o a color and it is judged in proportion to its gray content; the less gray content the more saturated color. Luminance is the notion o intensity and it describes how much dark or bright is a color. Distinguish luminance o brightness is important. According to the International Commission o Illumination (CIE), brightness is the quantity o light produced by a source and luminance is the radiated intensity which impacts to human eye. It means that, the perceived luminance permits to distinguish i a color is brighter or darker than other. Chromaticity or chroma is produced by the combination o hue and saturation; so, a color could be characterized by its chroma and luminance. For visual acquisition, color is an important characteristic since this allows distinguish one object to another with same shape, texture or size. However, color manipulation is not a simple task; or example, i a RGB color needs to be saturate and its values are modiied in a not accurate way, instead o obtaining a saturated color, a dierent color could be obtained. Then, it is necessary to work in the adequate color space in order to manipulate the chromatic image values. There are many color spaces used on image processing but some o them work more according to the human visual system. u v Y is a space which permits to separate the chromatic and achromatic inormation 6. This is important because on the one hand contrast can be improved by changing the luminance values (channel Y) but this improvement must not change the original hue; hence, when luminance is manipulated individually, the color becomes lighter but its hue is the same. On the other hand, color enhancement needs to saturate the hue by manipulating the chromatic channels u v ; it means that a pixel x must decrease its gray content or moves away rom the achromatic values. Oten, images are represented on RGB color space, thereore it is necessary to translate the RGB values to u v Y values by using its respective transormation matrix (3). Y = u v' R G B ' (3) 2.3 Morphological Rational Filters Morphological rational ilters (MRF) are a combination o the basic morphological ilters and, according with Kogan et al. 13, these ilters provide more robust results. At the beginning, the MRF were applied or edge detection and later Espino-Gudiño et al 3. proposed their use with openings by reconstruction in a multi-scale process (4). Latter application allows to improve the contrast o dark regions, where the opening by reconstruction operates as background with the aim o identiies the main regions and improves the contrast at the scale μ n. This multi-scale process considers the Weber s law which establishes that, in order to produce a change in the visual perception, it is necessary an increment proportional to the intensity o the original stimulus. N γ ( ) μ n 1 RM ( x, μ) =, with γ ( ) = 0( ) = μ γ 0 n= 1 γ ( ) μ n (4) 2.4 Granulometry An image contains many structures which give orm to its regions and objects; the structures on an image could have many sizes or scales, some o them more common than others. When a morphological transormation or ilter is applied it aects only those regions content on a size μ. For this reason, it is essential to identiy the most important scales in order to reach them when a transormation is applied. Granulometry is a concept ormalized by Matheron 4 in the binary case and extended to the gray-scale case by Serra 5 ; it is also the usual tool or inding the scales in mathematical morphology. Granulometric analysis consists on consecutive morphological transormations with an incremental size o μ and it provides inormation about how much μ aects the image. A ormal granulometry deinition is expressed next. Deinition 1: A amily o openings {γ λ }, where λ {1,, n}, is a granulometry i or all λ μ {1,,n}, and all unction, λ_ _μ_ _γ λ ()_ _ γ μ (). SPIE-IS&T/ Vol Q-3

4 mes( γ ( )) mes( + Δ ( )) G( ) = λ γ λ λ (5) mes( ) On contrast enhancement the objective is to reach the majority o the regions with the aim o improving the luminance; the granulometric analysis is used or knowing what structure sizes are contained on the image and could be reached by applying MRF. There are many scales on one image but only some o them generate a signiicant change. Equation (5) is used or granulometric analysis by using openings; where mes is the volume measure, deined as the sum o all the pixel values, λ is the size o the opening, Δ is a increment value and G(λ) is the granulometric unction. Anti-granulometric analysis uses the same expression but applying closings instead o openings. In both cases the higher G(λ) values, the more adequate λ sizes or the transormation. 3. PROPOSED METHODS 3.1 Contrast enhancement The objective o the proposed method is ocused on improve the contrast o dark regions but without aecting good contrast region. MRF by openings demonstrates that it can improve the contrast 3 and it result is analyzed and also MRF by closings is explored. Figure 1 shows an image with two dierent illuminations dynamic which is our study case. Outside region has a good luminance and it is possible to distinguish among all its elements eortless; yet, inside region is dark and the discrimination among elements is hardest. Luminance histogram (Fig. 1) shows that, although luminance values are distributed all range long, an important quantity o pixels are in the darker region, others are in the middle range having good luminance and just a ew are on the lightest region. Figure 1. Original image with two illumination dynamics and its luminance histogram. Figure 1 was analyzed with a granulometric analysis in order to ind those scales that aect it most. The results obtained by using openings and closings (Fig. 2) show many important scales but it is necessary probe them and choose which o these present an adequate improvement. It is important to say that some images require more scales than others and might even require only one. In this case, two scales were used or MRF by openings, λ 1 =72 and λ 2 =136, according to the granulometric inormation and the tests with dierent scale values (Fig. 2). MRF by closings used one scale λ 1 =148, which provides the more accurate improvement (Fig. 2). Notice that, among the more important scales to choose, not always the highest G(λ) is used, usually the more accurate scales are among the irst iteen higher G(λ) values; the election depends more on obtaining an appropriate improvement and not to exceed with the new luminance. SPIE-IS&T/ Vol Q-4

5 Figure 2. Granulometric analysis o Fig. 1 using openings and closings. It was observed that MRF by openings achieve a best improvement on dark regions, respect to MRF by closings. On the one hand, MRF by closings (6) was used or improving low luminance regions, but some elements on light region could be vanished; resulting image shows a signiicant luminance change on dark regions while in light regions it was too much (Fig. 3). On the other hand, MFR by openings (4) was used or emphasizing and conserving the elements with good luminance; igure 3 shows how, although with small improvement, structures in light regions conserve a good luminance on the image. By now, each ilter has been used individually but in this way no one can provide an accurate result or contrast enhancement. Nevertheless, by combining both MRF results is possible to obtain a satisactory improvement; this combination is called combined MRF. N ϕ ( ) μ n 1 RM ( x, μ) =, with ϕ ( ) = 0( ) = μ ϕ 0 n= 1 ϕ ( ) μ n (6) Figure 3. MRF with openings using λ 1 =72 and λ 2 =136 and MRF with closings using λ 1 =148. In this case, both MRF images were combined in order to obtain the inal improved image. It was taken 8% o Fig. 3 and 2% o Fig. 3 or combined MRF. Final image shows the new luminance where it is easier to make a distinction among the elements inside the room but also outside elements are visible (Fig. 4). Luminance histogram demonstrates that luminance still covers all the range but it has been better distributed resulting in a more uniorm contrast. SPIE-IS&T/ Vol Q-5

6 Figure 4. Combined MRF result and its luminance histogram. Color images permits a better inspection o the results, so a comparison with two methods commonly used in contrast enhancement was also made. First method was multi-scale retinex which improve dark regions but it vanish the light. Second method was histogram equalization, which distributes all the luminance values along the range they need it or not; hue might be changed so its result might more adequate on gray-level processing. In both cases structures are no conserved. (c) (d) Figure 5. Original color image and its contrast improvement by combined MRF, (c) Retinex and (d) histogram equalization. SPIE-IS&T/ Vol Q-6

7 3.2 Color enhancement As well illumination aects the contrast also aects its saturation. In processes as color segmentation, saturation is an important eature and its enhancement could make easier the object recognition. However, illumination conditions are not the unique reason o aded colors; some processes, as contrast enhancement, also originate them. The solution to this condition consists o increase saturation but in this process hue might be changed. Moreover, it is important to consider that not all regions are aded reason why a same saturation increase cannot be applied over all pixels. Although the main subject on this work is contrast, color enhancement is also a topic o interest in order to provide a more complete enhancement process. Color analysis As mentioned previously, color space u v Y separates chroma and luminance so in this case it is also used or color enhancement. Image color is distributed inside the chromatic triangle u v (CT) with the aim o observe its colors and saturation (Fig. 6) [Lucchese and Mitra, 2004; Espino and Terol, 2007]. In the CT center are located the colors nearest to white, it means the achromatic region; the greater distance o the white, the greater color saturation. On saturation process, colors must not change suddenly rom achromatic region to the limits o CT since this would cause an oversaturation. The proposed methodology consists on a multi-scale saturation which permits gradual changes according with the pixels location within the TC. This means that, with the aim o saturate aded colors it must establish a scale as a saturation limit in order to avoid oversaturation. The number o scales depends on image inormation. Figure 6. R, G, B chromatic coordinates on the u v Y color space and the u v chromatic triangle. For example, color distribution in Fig. 7 shows that most o the pixels are near to the achromatic region and it produces a low saturated image as Fig. 7. Although it is necessary saturate the colors also it must be careul not to change its hue. Figure 7(d) is the result o a saturation which moves away the pixels rom the achromatic region generating a new color distribution (Fig. 7(c)). Colors were enhanced and have a high intensity; yet, the main building color, originally white, was changed and now it has a reddish color. This is an undesired eect because a color enhancement must not change the chromatic inormation o the image. Multi-scale color saturation Enhance color without change the hue is the main objective o our proposed method. It is important remember that images contains chromatic and achromatic inormation and both are equally important or the image realism. In this case, the saturation o chromatic inormation must be increased and achromatic inormation must be keep. For this, irstly the access to the lightest pixels was limited by drawing an achromatic triangle (AT) around them; in this way, it is avoided that lightest pixels acquire color. Secondly, pixels that do not belong to the AT should move away o it but at their same hue direction. Then, it is important to know the location o pixel x and its distance to each side o CT SPIE-IS&T/ Vol Q-7

8 ( RG, GB, BR ) in order to know its direction o saturation; x must be moved toward the nearest side. Once known which pixels are going to move and to where, it is necessary to establish also in what extend will be saturated. For example, i the pixels are moved in the right direction until the TC limits saturation result is not accurate because image goes rom aded to oversaturated. In order to avoid this, a gradual color enhancement which uses scales is proposed and based on each scale size its respective enhance triangle is drawn. The result is a set o triangles contained one within another. (c) (d) (e) () Figure 7. (a, c, e) Color distribution triangles o original aded image, (d) color enhancement without limits on achromatic region and () multi-scale color enhancement with scales α1=20, α2=50, respectively. Once direction o saturation is known, the intersection points between pixel x and the AT, enhance triangles and the TC must be ound. Ater, it is determined which triangles are adjacent to x and move it toward the arthest (ps) in the proportion calculated by (7) (Espino-Gudiño et al., 2007); where u and v are the current chromatic values o x, u s and v s are the chromatic values o ps, Y is the x luminance and u new and v new are the new saturated values o x. SPIE-IS&T/ Vol Q-8

9 u' u ' new = Y Y + u's v' v' s Y Y + v' v' s and v' new = 2Y Y Y + v' v' s (7) Multi-scale color enhancement o Fig. 7 moves central pixels in the direction o its hue, which will be maniested on new color intensity. However, since original pixels are close to the achromatic region, their AT must be small in order to conserve only the lightest. For this image, the AT have a size o 0.03 o the CT total size and two scales were used α1=20 y α1=50 (Fig. 7(e)); it means that, the size or each enhance triangle (Tα1 y Tα2) is 0.2 and 0.5 o the distance rom the CT to the AT, respectively. First scale objective is to saturate the more aded pixels but avoiding oversaturation; second scale improvement the notable but still aded colors. In order to dierentiate each triangle, it was establishing that: AT is in the center, enhance triangles (Tαn) are marked with solid line and CT is marked with dotted line; enhance triangles are numerated in ascendant order rom the inside out, where Tα1 is the closest one to AT and Tαn is the closest one to TC. Finally, Fig. 7() is the result o the new saturation in which one can appreciate details not visible on original image. Other results o multi-scale color enhancement are shown on Fig. 8. (c) (d) Figure 8. (a, c) Faded images and its multi-scale color enhancement with scales Tα1=5, Tα2=25, Tα3=35 and (d) Tα1,2=10, Tα1=23, respectively. SPIE-IS&T/ Vol Q-9

10 4. CONCLUSION Morphological rational ilters are helpul on contrast enhancement and, due to they use transormations by reconstruction, original structures are conserved. MRF with closings could enhance low luminance values but it vanish light elements. MRF with openings could be used or remarking dark structures and restoring the lost inormation on light regions. Moreover, the combination o both ilters provides a more uniorm result when dierent luminance aects the image. In addition, by using a multi-scale color enhancement, a satisactory color improvement can be achieved and oversaturation can be controlled. This is a simple process which also allows to choose only those aded colors or its saturation. Finally, contrast and color improvement could be applied as a pre-processing step to acilitate a urther task. REFERENCES [1] Meyer, F. and Serra, J., Activity Mappings, Signal Processing. Papers 16, (1989). [2] Land, E. and McCann, J. J., Lightness and retinex theory, Journal o the Optical Society o America. Papers 61(1), 1-11 (1971). [3] Espino-Gudiño, M., Santillan, I. and Terol-Villalobos, I. R., Morphological Multiscale contrast approach or gray and color images consistent with visual perception, Optical Engineering. Papers 46(6), 1-14 (2007). [4] Matheron G., [Eléments pour une théorie des Milieux poreux], Masson Ed., Paris, (1967). [5] Serra J., Toggle Mappings, Technical report N-18/88/MM, Centre de Morphologie Matematique, ENSMP, Fontainebleau, France, (1988). [6] Lucchese, L. and Mitra, S. K., A new class o chromatic ilters or color image processing. Theory and applications, IEEE Transactions on Image Processing. Papers 13(4), (2004). [7] Serra J., [Image Analysis and Mathematical Morphology], Ed. Academic Press, Ney York, USA, Vol.2, (1988). [8] Soille P., [Morphological Image Analysis], 2nd. ed., Heidelberg: Springer-Verlag, (2003). [9] Vincent L., [Current Trends in Stochastics Geometry and its Applications], Chapman&Hall editors, (1997). [10] Lantuéjoul C. and Maisonneuve, F., Geodesic methods in quantitative image analysis, Pattern Recognition. Papers 17(2), (1984). [11] Pajares, G. and de la Cruz, J. M., [Visión por computador: imágenes digitales y aplicaciones], Alaomega, (2002). [12] Fairchild, M. D., [Color Appearance Models], Wiley Ed., N.Y., (2005). [13] Kogan, R. G., Agaian, S. and Panetta, K., Visualization using rational morphology and magnitude reduction, Proc. SPIE 3387, (1998). SPIE-IS&T/ Vol Q-10