A Saturation-based Image Fusion Method for Static Scenes

Transcription

1 2015 6th International Conference of Information and Communication Technology for Embedded Systems (IC-ICTES) A Saturation-based Image Fusion Method for Static Scenes Geley Peljor and Toshiaki Kondo Sirindhorn International Institute of Technology, Thammasat University, Thailand geley23@gmail.com, tkondo@siit.tu.ac.th Abstract In this paper, we present a saturation-based image fusion method for static scenes, which is aimed at generating high-dynamic range images. A set of images of the same stationary scene with different exposure levels, such as, overexposed and underexposed images is blended together into one. The weighting factors, for blending the images, are dependent on the saturation values of the images. Higher saturation indicates more vivid colors. Thus, we assign a heavier weight to the pixels with higher saturation while lighter weights are used for those pixels whose saturations are lower. Our experimental results show that the proposed method can produce satisfactory results without adjusting any parameters at a low computational cost. Moreover, the output images are fairly free from the halo effect problem. Index Terms Saturation, Image Fusion, Exposure Fusion, HDR, Tone mapping. I. INTRODUCTION Digital photographs are becoming hugely popular all over the world with the advancement in technology and communication, but it has a limited dynamic range. Thus, limits the computer vision potential. In order to provide for full dynamic range, one can take multiple images and fuse them together to create a single image retaining only the wellexposed regions. This process is known as image fusion. Image fusion is the process of combining relevant information from two or more images into a single image, where the resultant fused image will be more informative than any of the input images. High dynamic range (HDR) images can be obtained from the set of multi-exposure images [1, 2]. Debevec et al. presented a technique to recover the high dynamic range radiance maps from the photographs taken with conventional imaging equipment [1].Their method first identifies the response curve of the imaging device which is used to recover the high dynamic range radiance map and then used logarithmic mapping to display the high dynamic range radiance map on normal display devices. Goshtasby has used the block based approach to fuse multiexposure images [2]. Their method first partitions the image domain into uniform blocks and for each block selects the image that contains the most information within that block and the selected images are then blended together using monotonically decreasing blending functions. The entropy has been used as a measure for optimization when fusing the image. Most display devices have a limited dynamic range and cannot directly display HDR images. To this end, the process known as tone mapping [3, 4] is applied to HDR image which reduces the dynamic range of the image thus making it suitable for display. Apart from tone mapping, image fusion techniques have also been used for many years. For example, multimodal imaging [5] and video enhancement [6]. Burt et al. have used pyramid transform to perform fusion of multi-exposure images [5]. The other approach is, fusing the multiple exposures into a high quality, low dynamic range image, ready for display which is guided by simple quality measures like saturation, contrast and well exposedness [7]. Their technique consists of two main parts: 1. Computing Weight Map from metrics like Saturation, Contrast and Well Exposedness 2. Applying the Pyramidal Image Decomposition for multiresolution blending and for fusing the images. Malik et al. have extended the Merten et al. approach and used wavelet decomposition instead of pyramidal decomposition as wavelet offers more efficient and robust representation [8]. In the present work, our approach is similar to the Merten et al. approach of Computing Weight Map from metrics, but we compute weight map only to the saturation channel followed by a Gaussian low-pass filter and some post processing techniques to the final fused image. We compare our result with histogram equalization as histogram equalization is a method in image processing of contrast /15/$ IEEE

2 adjustment using the image s histogram [9], followed by MATLAB HDR built-in function since Image Processing Toolbox supports a diverse set of image types, including high dynamic range [10]. We require no information about camera or shutting conditions, such as shutter speed or aperture size. Similar to Merten et al. approach, our technique merely relies on simple quality measures, but we compute only saturation, which prove to be very effective. Our method generates fully automatically without adjusting any parameters, unlike commercially available software, such as, Photoshop. Thus, we do not compare our method with it. II. PROPOSED ALGORITHM Finally, the images are converted back to the RGB space (Fig.1 (f) ). Weighted averaging= w11 Io + w22 Iu, where, w11 w22 Io Iu a) RGB Overexposed input image (I o) RGB Underexposed input image (I u ) The procedure of the proposed method is summarized in several steps as follows: b) RGB -> HSV RGB -> HSV 1. We load a set of static input images, I o (overexposed) and I u (underexposed) ( Fig. 1 (a) ). 2. Input images I o and I u are converted from the RGB to the HSV color space, respectively ( Fig. 1 (b) ). 3. Then we extract the saturation channels from the HSV color space of the input images, respectively, as s1 (saturation of overexposed) and s2 (saturation of underexposed) ( Fig.1 (c) ). 4. Next, we take the sum of the two saturation images together as s=s1+s2. 5. The pixels whose saturation is equal to 1 are often underexposed and colorless. Hence, we replace the saturation value 1 with 0 to exclude those pixels from the image blending. Note that the pixels with saturation 0 are automatically disabled from the blending as well. 6. Then, we define the weighting factor on the overexposed image as w1= s1 and the weighting s1+s2 factor on the underexposed image as w2= s2 s1+s2 ( Fig.1 (d) ). In the case of s1+s2=0, we assign 0.5 to both w1 and w2. 7. We apply a Gaussian low-pass filter (LPF), G, to the weighting factor map ( w1 ) to smoothen it and stored as w11. w22 is obtained by w22=1 w11 (Fig. 1 (e) ). The size of the LPF is the one tenth of the vertical size of the input image. 8. We blend two input RGB full color images using the weighting factors w11 and w We apply post processing to the fused image. First, we sharpen the final fused image using the unsharp masking technique, though this is optional. Then we convert the final fused sharpened image to the HSV color space. We then apply the gamma transform to the image intensity and saturation values where we currently use the fixed gamma value 0.7 for both. This step helps to brighten the output images. c) Saturation channel d) Weight e) Gaussian low pass filter f) Fused image Saturation channel Weight Figure 1. Methodology flow chart III. EXPERIMENTAL RESULTS Gaussian low pass filter Figure 2 shows the result of the proposed method. We have obtained the sample images from the website [11]. (a) Overexposed image, (b) under-exposed image, (c) the saturation channel of the image (a), (d) the saturation channel of the image (b), (e) the weight for (a), (f) the weight for (b), (g) fused image, and (h) fused image after post processing. It is obvious that the final fused image, (h) looks pleasing, natural and realistic.

3 Further, we compare our result with the MATLAB HDR built-in function and histogram equalization techniques. Since we require no information about camera or shutting conditions, such as shutter speed or aperture size, we compare our proposed result with the MATLAB HDR built-in function without adjusting any parameters. Figure 3 shows the comparison with the MATLAB HDR built-in function [10] and histogram equalization techniques [9]. We have obtained the sample images from the website [12]. (a) Over-exposed input image, (b) under-exposed input image, (c) the histogram equalized image of (a), (d) the histogram equalized image of (b), (e) output by the MATLAB function makehdr, and (f) output by the proposed method with post processing. As we can see that, (c) histogram equalization of the overexposed image produces lots of blocks and (d) produce good color contrast, but with less information in some parts of the region. On the other hand, (e) produce faded color, which lead to the unnatural look of the image. (f) strongly shows that our method can generate highdynamic range image with complete details. Figure 4 shows another comparison with the MATLAB HDR built-in function and histogram equalization techniques. We have obtained the sample images from the website [11]. (a) Over-exposed input image, (b) under-exposed input image, (c) the histogram equalized image of (a), (d) the histogram equalized image of (b), (e) output by the MATLAB function makehdr, and (f) output by the proposed method with post processing. As we can see that, (c) histogram equalization of the overexposed image also produces lots of blocks and (d) produce good color contrast, but with less information in some parts of the region. On the other hand, (e) produce faded color, which lead to the unnatural look of the image. (f) also shows that our method can generate high-dynamic range image with complete details. We assume that the stationary images are properly aligned and every part of the image is captured by at least one image having enough information stored in it. From our initial experimental results, we have found that, when we used a small Gaussian low-pass filter, which smoothen the images, the output image shows the vivid halo effect. Halo is appeared as a white glow around the subject when the darker pixels meet the lighter pixels. Thus, we have applied a large Gaussian low-pass filter to suppress the halo effect problem. Our method is limited to the objective method due to the fact that the traditional quality measure like Image Quality Index (IQI), measures the quality of the fused image based on how much details it has obtained from different images of the same scene [8]. In some cases of multi-image fusion, due to either overexposed or underexposed, some information from the sequence does not contribute towards the fusion. So we have compared our result subjectively with the existing method and, in general, our techniques can produce a satisfactory result. IV. CONCLUSION This paper presents a saturation-based image fusion method for static scenes, which is designed for generating highdynamic range images fully automatically at a low computational cost. The weighting factors, for blending the images, are dependent on the saturation values of the images. Higher saturation indicates more vivid colors. Thus, we assign a heavier weight to the pixels with higher saturation, while lighter weights are used for those pixels whose saturations are lower. Experimental results show that the proposed method can produce realistic high-dynamic range image automatically. Moreover, since we used a large low-pass filter, which smoothen the images, the output images are fairly free from the halo effect problem. One limitation of the proposed method is that, it has a difficulty in blending images when they have a common region that has little image details because of overexposure or underexposure. In this case, a use of more than two images may be necessary. As a future work, we plan to deal with image registration. ACKNOWLEDGMENT This Research is financially supported by EFS (Excellent Foreign Student) Scholarships, Sirindhorn International Institute of Technology (SIIT), Thammasat University (TU) and National Research University Project, (NRU), Thailand Office of the Higher Education Commission. REFERENCES [1] Debevec, P.E. and J. Malik. Recovering high dynamic range radiance maps from photographs. in ACM SIGGRAPH 2008 classes ACM. [2] Goshtasby, A.A., Fusion of multi-exposure images. Image and Vision Computing, (6): p [3] Reinhard, E., et al. Photographic tone reproduction for digital images. in ACM Transactions on Graphics (TOG) ACM. [4] Reinhard, E., et al., High dynamic range imaging: acquisition, display, and image-based lighting. 2010: Morgan Kaufmann. [5] Burt, P.J. and R.J. Kolczynski. Enhanced image capture through fusion. in Computer Vision, Proceedings., Fourth International Conference on IEEE. [6] Raskar, R., A. Ilie, and J. Yu. Image fusion for context enhancement and video surrealism. in ACM SIGGRAPH 2005 Courses ACM. [7] Mertens, T., J. Kautz, and F. Van Reeth. Exposure fusion. in Computer Graphics and Applications, PG'07. 15th Pacific Conference on IEEE. [8] Malik, M.H., S. Asif, and M. Gilani, Wavelet based exposure fusion. Proceedings of The World Congress on Engineering 2008, pp [9] Gonzalez, R.C. and R.E. Woods, Digital image processing. 2002, Prentice hall Upper Saddle River, NJ:. [10] Mathworks. Image Processing Toolbox. Available at [11] [12]

4 (a) (b) (c) (d) (e) (f) (g) (h) Figure 2. (a) Over-exposed image, (b) under-exposed image, (c) the saturation channel of the image (a), (d) the saturation channel of the image (b), (e) the weight for (a), (f) the weight for (b), (g) fused image, and (h) fused image after post processing.

5 (a) (b) (c) (d) (e) (f) Figure 3. (a) Over-exposed input image, (b) under-exposed input image, (c) the histogram equalized image of (a), (d) the histogram equalized image of (b), (e) output by the MATLAB function makehdr, and (f) output by the proposed method with post processing.

6 (a) (b) (c) (d) (e) (f) Figure 4. (a) Over-exposed input image, (b) under-exposed input image, (c) the histogram equalized image of (a), (d) the histogram equalized image of (b), (e) output by the MATLAB function makehdr, and (f) output by the proposed method with post processing.