Image Contrast Enhancement Using Joint Segmentation

Transcription

1 Image Contrast Enhancement Using Joint Segmentation Mr. Pankaj A. Mohrut Department of Computer Science and Engineering Visvesvaraya National Institute of Technology, Nagpur, India Abstract Image contrast enhancement without affecting other parameters of an image is one of the challenging tasks in image processing. The quality of poor images can be improved using various image contrast enhancement techniques. Contrast is the visual difference that makes an object distinguishable from background. The basic aim of this topic is to provide an improved and good quality image by adjusting the amount of saturation and illumination to achieve more realistic and clear image. This paper presents a new method of contrast enhancement of low contrast color image along with object detection in an image. In this method a final saliency map is obtained from the input low contrast image from which region of interest is detected. After that using the gray image of input and object detected image, depth map is being generated. Finally A laplacian method is implemented to enhance the input image using depth map information. The image enhancement methods based on histogram equalization (HE) often fail to improve local information and sometimes have the fatal flaw of overenhancement when a quantum jump occurs in the cumulative distribution function of the histogram. To overcome these shortcomings, we propose an image enhancement method based on binary segmentation. The proposed method shows natural and robust image quality when we compare with other existing methods. The PSNR and MSE calculated after experimentation show better result than existing method. In this method global image contrast is improved, along with local image contrast is also consistently improved without the over-enhancement. Index Terms : Contrast enhancement, Color and Depth image, Histogram equalization and Saliency map. I. INTRODUCTION Due to the advent of computer technology, imageprocessing techniques have became increasingly important in a wide variety of applications. Contrast enhancement produces an image that subjectively looks better than the original image. In image processing technology, image enhancement means improving image quality through a broad range of techniques such as contrast enhancement, color enhancement, dynamic range expansion, edge emphasis, and so on. Contrast enhancement plays a crucial Dr. Deepti Shrimankar Department of Computer Science and Engineering Visvesvaraya National Institute of Technology, Nagpur, India dshrimankar@cse.vnit.ac.in role in image processing applications such as digital photography, medical image analysis, remote sensing, etc. There are many enhancement techniques used widely, among which histogram equalization (HE) is widely used because of its simplicity and effectiveness [1]. In particular, since global histogram equalization (GHE) tends to over-enhance the image details, the approaches of dividing an image histogram into several sub-intervals and modifying each subinterval separately have been considered as an alternative to GHE. The effectiveness of these sub-histogram based methods is highly dependent on how the image histogram is divided. These image histograms are modeled using Gaussian mixture model (GMM) and divide the histogram using the intersection points of the gaussian components [4]. Histogram equalization (HE) is the representative method for contrast enhancement, has been developed to satisfy humans with its resultant images [1]. The classic HE provides the best visual performance in certain conditions compared to other state-of-the-art techniques, but has the fatal flaw of over-enhancement when a quantum jump occurs in the cumulative distribution function (CDF) which is derived from the probability density function (PDF) in the histogram of an image. This quantum jump arises when a few consecutive luminance levels of the histogram occupy substantial areas in an image, and various alternatives have been proposed to solve this issue [5]. Contrast-limited adaptive histogram equalization (CLAHE) [2], separates an image into tiles, equalizes each of them, and interpolates their boundaries. To improve its performance, bins with values over a predefined threshold are clipped and the residual is redistributed uniformly to the histogram. This method shows dramatic contrast enhancement but biggest disadvantage of this is it produce the output image which looks unnatural. Histogram specification (HS) is another method that takes a desired histogram by which the expected output image histogram can be controlled. However specifying the output histogram is not a easy task as it changes from image to image. Another method called dynamic histogram specification (DHS) is presented which generates the specified histogram dynamically from the input image. This method can 823

2 preserve the original input image histogram characteristics. However, the degree of enhancement is not that much significant [3]. Some researchers have also focused on improvement of histogram equalization based contrast enhancement such as Mean Preserving Bi-histogram equalization (BBHE), equal area dualistic sub-image histogram equalization (DSIHE) and minimum mean brightness error bi-histogram equalization (MMBEBHE). This method tries to overcome the brightness preservation problem. DSIHE method uses entropy value for histogram separation. MMBEBHE is the extension of BBHE method that provides good contrast enhancement but they also cause more annoying side effects depending on the variation of gray level distribution in the histogram [5]. Recursive mean separate histogram equalization (RMSHE) is another improvement of BBHE. However, it is also not free from side effects [6]-[9]. Recently, technical breakthroughs of the color image enhancement have been found using depth or stereo as side information [10]. Stereo matching algorithms and depth sensors are now providing highly accurate depth images, and thus the use of the depth image for the color image enhancement becomes an important research issue [12]. In this letter, we propose a new contrast enhancement method that enhances the low contrast image using color and depth images. The final saliency map is generated from the original low contrast image from which object of interest is found. After that its gray image is being obtained. The gray image of input image and this image is being used for producing depth map image. The depth map information is being used for object detection. The background and foreground image combine to generate the depth map. Finally laplacian pyramid method is being used for enhancement [5]. The rest of the paper is organized as follows. Section 2 gives the proposed framework. In section 3, experimental results and discussion are explained followed by conclusion in section 4. II. PROPOSED METHODOLOGY We use a low contrast color image as a input as shown in fig.1 (a). Our aim is to detect object of interest and enhance this object. Finally enhance the image as a whole. The proposed algorithm generates the saliency map and detects the foreground objects in an image as shown in fig. 1(b) and (c). Detection of salient image regions is useful for applications like image segmentation, adaptive compression, and region-based image retrieval. One specific problem of computer vision algorithms in extracting information from images is to find objects of interest. Humans have an uncanny knack of spotting objects of interest almost instantaneously in a scene. Such objects capture our attention whether we are aimlessly gazing at scene or whether we are searching for something specific. Sometimes, objects in the visual scene pop-out because of their distinctiveness with respect to the rest of the environment in terms of their shape, symmetry, color, brightness, etc. Such objects are noticed involuntarily by virtue of their relative contrast. This perceptual quality of an object, person, or pixel, which makes it stand out relative to its neighbors and thus direct our attention, is called saliency [11]. Fig. 1: (a) Low contrast color image (b) Final saliency map (c) detected foreground object In this paper we use a novel method to determine salient regions in images using low-level features of luminance and color. The method is fast, easy to implement and generates high quality saliency maps of the same size and resolution as the input image [11]. Using the saliency calculation method described in [11], saliency maps are created at different scales. These maps are added pixel-wise to get the final saliency maps. The input image is then oversegmented and the segments whose average saliency exceeds a certain threshold are chosen. The output image containing the salient object that is made of only those segments that have an average saliency value greater than the threshold. This method generates saliency maps at the same resolution as the input image. This approach is at least five times as fast as a prominent approach to finding saliency maps and generates high resolution saliency maps that allow better salient object segmentation [11]. After finding saliency map and detected object, we have to obtain the gray image of the above obtained output using the formula *R *G *B where R, G, B are true color values. Next aim of this method is to find out depth map of the input low contrast image. To find out depth map we require the gray image of original input image. Generally in order to find out the depth map of an image, two views of an image are required viz. Left view and right view. But whenever we are going to enhance any input image we will get only single view. In that case we can t use existing methods which are easily available. Therefore a New procedure of finding depth map from single view of an image is being implemented in our paper which is described as below: 824

3 Procedure to Find Depth Image a. Get gray image of original RGB image. b. Get gray image of final saliency object detected image. c. Set parameters fb_map=255 and bg_map=127, quant=10, where fb_map, bg_map and quant are the parameters going to be used. For background image 1. Find indices of non zero pixel values using above steps. 2. Set pixel values of all (non zero) indices to zero so we get only background image (in original gray image). 3. Find maximum value of pixel in that image. 4. Divide each pixel by maximum value. (Normalize to 0..1 range) 5. Now multiply each pixel to bg_map. (Normalize to range) 6. Rounding pixel value - round(map_bg/quant)*quant. For foreground image 1. Do all previous steps except steps In last step bg_map is added to all foreground pixel value. map_fg = (map_fg * bg_map)+bg_map After finding the background and foreground image, add both image to get depth image which highlight foreground object. threshold we assign value less than threshold to 0 and No change in greater than threshold value. The next step is to add this depth image to original color red, green and blue intensity image individually and combine them to restore into RGB image. Then apply this image as input to enhancement algorithm. Enhancement using Laplacian and Gaussian method The original low contrast image is being modified by the depth map obtained above. The modified image is taken as an input to the laplacian pyramid method. HE-based algorithms generally fail to improve local contrast and edges of an image, since it focuses on improving the global contrast of an image [5]. This issue is considered to be the inherent limitation of HE methods. To overcome the above problem, we chose a laplacian framework for enhancement. In this method RGB input image is transformed to I 0 luminance and then decomposes I 0 into band-pass images. The RGB input image and I 0 are reused in the final step of the color restoration. Generate the laplacian pyramid for the given image, for N particular levels. So using the equation, I o =I N+ D n (1) Where N is the highest decomposition layer. In this framework, first contrast is enhanced using HE based algorithm. In order to generate the histogram with luminance levels in the range K [0, L-1] as a discrete function is described as h(l k)= n k (2) Where l k is the k th luminance level in K and n k represents the number of pixels having luminance level l k. Fig. 2 : (a) background image of fig. 1(a), (b) foreground image of fig. 1(a), (c) final depth map obtained after addition of fig. 2(a) and (b) After finding the depth map image from the above steps, we are going to find maximum pixel value in the given depth map. Thereafter finding its maximum value, divide all pixel values by maximum pixel value so that we will get the entire pixel in the range of 0 to 1. Later on integrate these pixel values by some constant value and find the mean of the entire pixel. Here average value is treated as meant and using 825

4 strategies of boosting noticeable minor areas and slantwise clipping bins in the histogram [5]. Fig. 3 : Generating Histogram of R image Smooth the histogram with a Gaussian filter: In the histogram, a ridge shape with some consecutive luminance levels can be regarded as the feature area of an image. To globally distinguish between ridges and valleys and remove their ripples, we smooth the histogram [13] like as follows: h g(l k)=h(l k)*g(l k) (3) Where g (x) = Where g(x) is a Gaussian function, x is the corresponding location to a bin of the histogram, and coefficients of the Gaussian filter are normalized. Fig. 5: Histogram of R image with Local Boosting First, the peak value in the smoothed histogram h g ( l k) is found as p(k)=max k K {h g(l k)} (4) Second, the ridges between valleys are searched and boosted. Ridge boundary is defined as the bins between the first point of the positive slope and the last point of the negative slope Slantwise clipping: The clipping technique is used as it effectively suppresses the quantum jump. We find the mean of newly generated histogram and then find the mid value and then we gather the residual from local and global clipping. Fig. 4: Smoothed Histogram of R image To effectively reduce the quantum jump, the laplacian contrast enhancement algorithm has two key 826

5 Fig. 7: (a) input image, (b) Enhanced detected object, (c) Final enhanced Image Fig. 6: Histogram of R image with clipping Images PSNR(dB) MMSE(dB) Time for Enhancement(sec) Flower Girl Plant Twin Insect Generating new image: Find the normalized cumulative histogram h= (cdf- cdf(min)/mn- cdf(min)) * 255 And replace the values with new equalized values. By adding the output of contrast enhancement and detail enhancement, the final enhanced image is obtained after color restoration phase. Using (1), the enhanced luminance image is obtained as I = I N + D (5) Where the right hand components are obtained by combining the enhanced images. The same procedure is repeated for green intensity and blue intensity image. Finally we get more enhanced image with improved quality and high PSNR value. III. EXPERIMENTAL RESULT In order to evaluate the performance of the proposed algorithm, following low contrast image were used in our experiment. The table shows the performance details and effectiveness of the proposed method. Peak signal to noise ratio (PSNR) indicates enhancement of an image. This ratio is often used as quality measurement between original and reconstructed image. Higher the PSNR value greater the quality of reconstructed image. The number of layers (N) used in the process of decomposition of the proposed framework is also important. For the layer selection, mean squared error (MSE) is computed to observe variations among Laplacian image layers. The total time required for enhancement is less as compare to existing methods. Image enhancement using color and depth map using layer labeling approach takes more than 1 minute for enhancement [10]. On the other hand our segment based approach takes less than a minute for the process of enhancement. Table 1: Result obtained after experimenting on various low contrast color image. PSNR, MSE and Time required for enhancement. IV. CONCLUSION In this paper we proposed a new image contrast enhancement technique using joint segmentation. The proposed method features robust local as well as global contrast enhancement. A quick method of generating depth map is also proposed. We also generate saliency maps at the same resolution as the input image. We demonstrated the effectiveness of the method in detecting and segmenting salient regions in a wide range of images. This method is fast, easy to implement and generates high quality enhanced output image without over enhancement 827

6 REFERENCES [1] R. C. Gonzalez, and R. E. Woods, Digital image processing, 3rd ed., Upper Saddle River, N.J., Prentice Hall, [2] S. M. Pizer, E. P. Amburn, J. D. Austin, R. Cromartie, A. Geselowitz et al., Adaptive Histogram Equalization and Its Variations, Computer Vision Graphics and Image Processing, vol. 39, no. 3, pp , Sep, [3] Anil K. Jain, Fundamentals of digital image processing Prentice Hall, [4] T. Celik and T. Tjahjadi, Automatic image equalization and contrast enhancement using Gaussian mixture modeling, IEEE Trans. Image Process., vol. 21, no. 1, pp , Jan [5] S. Yun, J. Kim, and S. Kim,, Image Enhancement using a Fusion Framework of Histogram Equalization and Laplacian Pyramid IEEE Transactions on Consumer Electronics, Vol. 56, No. 4, pp , November [6] K. Yeong-Taeg, Contrast enhancement using brightness preserving bi-histogram equalization, IEEE Trans. Consum. Electron., vol. 43, no. 1, pp.1-8, [7] Soong-Der Chen, Abd. Rahman Ramli, Contrast Enhancement Using Recursive Mean-Separate Histogram Equalization for IEEE Scalable Brightness Preservation, Transactions on Consumer Electronics, Vol.49, No.4, (2003), pp [8] W. Yu, C. Qian, and Z. Baeomin, Image enhancement based on equal area dualistic sub-image histogram equalization method, IEEE Trans. Consum. Electron., vol. 45, no. 1, pp , [9] C. Soong-Der, and A. R. Ramli, Minimum mean brightness error bi-histogram equalization in contrast enhancement, IEEE Trans. Consum. Electron., vol. 49, no. 4, pp , [10] Seung-Won Jung, Member,IEEE Image Contrast Enhancement Using Color and Depth Histograms,. IEEE Signal Processing Letters, Vol.21, No. 4, pp [11] R. Achanta, F. Estrada, Patricia Wils, and S. Sfusstrun, Ecole Polytechnique Federale de Lausanne (EPFL), School of Computer and Communication Sciences (I&C). [12] W. Hachicha, A. Beghdadi, and F. A. Cheikh, Combining depth information and local edge detection for stereo image enhancement, in Proc. Eur. Signal Process. Conf. (EUSIPCO), 2012, pp [13] T. Arici, S. Dikbas, and Y. Altunbasak, A Histogram Modification Framework and Its Application for Image Contrast Enhancement, IEEE Trans. Image Process., vol. 18, no. 9, pp , Authors Profile Pankaj A. Mohrut has received bachelor of engineering degree in Information Technology from SGBAU university and currently pursuing M.Tech in Computer Science And Engineering from Visvesvaraya National Institute of Technology. The area of interest is image processing. Dr. Deepti Shrimankar is currently working as assistant professor in Visvesvaraya National Institute of Technology. She has done her M.tech in computer science and engineering. Her area of interest is Parallel and Distributed Systems, Embedded Systems, Computer Networks. She has received Ph.D in parallel and distributed system