Fuzzy Fusion Based High Dynamic Range Imaging using Adaptive Histogram Separation

Similar documents
ISSCC 2006 / SESSION 19 / ANALOG TECHNIQUES / 19.1

THE magnetic field has been widely used for the transfer of

IEEE TRANSACTIONS ON POWER ELECTRONICS, VOL. 33, NO. 6, JUNE

Scene-Adaptive RGB-to-RGBW Conversion Using Retinex Theory-Based Color Preservation

Franke Three-Dimensional Molded Interconnect Devices (3D-MID)

10 Input Filter Design

The Hemispherical Resonator Gyro for precision pointing applications A. Matthews and D. A. Bauer

178 IEEE TRANSACTIONS ON ELECTROMAGNETIC COMPATIBILITY, VOL. 48, NO. 1, FEBRUARY Mohit Kumar and Vivek Agarwal, Senior Member, IEEE EMI.

A Cooperative Localization Algorithm for UWB Indoor Sensor Networks

1150 IEEE JOURNAL OF SOLID-STATE CIRCUITS, VOL. 45, NO. 6, JUNE 2010

ADVANCED radar systems benefit from the ability to have

E tions usually derives its bursts of energy by rapidly

Modeling and Evaluation of the Effect of Obstacles on the Performance of Wireless Sensor Networks

2-D Scanning Magneto-Electric Dipole Antenna Array Fed by RGW Butler Matrix

Frugal Innovation and Knowledge Transferability

THE ENVIRONMENTAL concerns and electric utility

Renewable Energy 43 (2012) 90e100. Contents lists available at SciVerse ScienceDirect. Renewable Energy

Automatic Selection of Brackets for HDR Image Creation

CHROMATIC aberration (CA) commonly arises from the

Influence of Avatar Creation on Attitude, Empathy, Presence, and Para-Social Interaction

INDUCTIVE power transfer (IPT) systems have found application

Optical-Inertial System for Railway Track Diagnostics

MULTICELL battery is a widely adopted energy source

On-line Junction Temperature Estimation of SiC Power MOSFETs through On-state Voltage Mapping

ANALYSES SUPPORTING SURVEILLANCE REQUIREMENTS FOR A CATEGORY I PAIRED APPROACH PROCEDURE

Selective Detail Enhanced Fusion with Photocropping

DUE to the growing penetration of distributed generation

Common-mode Overvoltage Mitigation in a Medium Voltage Pump Motor Transformerless Drive in a Mining Plant. Brenno Marcus Prado

An 8.2 Gb/s-to-10.3 Gb/s Full-Rate Linear Referenceless CDR Without Frequency Detector in 0.18 μm CMOS

A Novel Hybrid Exposure Fusion Using Boosting Laplacian Pyramid

Electric Drive System of Dual-Winding Fault-Tolerant Permanent-Magnet Motor for Aerospace Applications

4438 IEEE TRANSACTIONS ON POWER ELECTRONICS, VOL. 29, NO. 8, AUGUST 2014

Agile Multiple Pulse Coherent Lidar for Range and Micro-Doppler Measurement

IEEE TRANSACTIONS ON MAGNETICS, VOL. 50, NO. 5, MAY

MODERN wireless communication systems are required

Visual Occlusion Decreases Motion Sickness in a Flight Simulator

A Saturation-based Image Fusion Method for Static Scenes

ISSN: (Online) Volume 2, Issue 2, February 2014 International Journal of Advance Research in Computer Science and Management Studies

FROM DYADIC CHANGE TO CHANGING BUSINESS NETWORKS: AN ANALYTICAL FRAMEWORK* AINO HALINEN. Turku School of Economics and Business Administration

Improved Region of Interest for Infrared Images Using. Rayleigh Contrast-Limited Adaptive Histogram Equalization

International Journal of Innovative Research in Engineering Science and Technology APRIL 2018 ISSN X

ISSN Vol.03,Issue.29 October-2014, Pages:

Coil Design and Shielding Methods for a Magnetic Resonant Wireless Power Transfer System

Denoising and Effective Contrast Enhancement for Dynamic Range Mapping

IEEE TRANSACTIONS ON INDUSTRY APPLICATIONS, VOL. 50, NO. 3, MAY/JUNE

Main Subject Detection of Image by Cropping Specific Sharp Area

A Kalman-Filtering Approach to High Dynamic Range Imaging for Measurement Applications

A Multi-resolution Image Fusion Algorithm Based on Multi-factor Weights

arxiv: v1 [cs.cv] 8 Nov 2018

HDR imaging Automatic Exposure Time Estimation A novel approach

Automatic High Dynamic Range Image Generation for Dynamic Scenes

Contrast Enhancement for Fog Degraded Video Sequences Using BPDFHE

Probabilistic motion pixel detection for the reduction of ghost artifacts in high dynamic range images from multiple exposures

THE consumer electronics market demands high speed,

FEATURE BASED GHOST REMOVAL IN HIGH DYNAMIC RANGE IMAGING

IEEE JOURNAL ON SELECTED AREAS IN COMMUNICATIONS, VOL. 35, NO. 8, AUGUST

AUTOMATIC modulation classification is a procedure

Invited Paper ABSTRACT. Keywords: Fiber gyro, fiber optic gyro, FOG, IFOG, RFOG, fiber resonator, resonator fiber optic gyro, laser gyro.

Noise Adaptive and Similarity Based Switching Median Filter for Salt & Pepper Noise

High Dynamic Range image capturing by Spatial Varying Exposed Color Filter Array with specific Demosaicking Algorithm

Applications of Flash and No-Flash Image Pairs in Mobile Phone Photography

A 0.65-to-10.5 Gb/s Reference-Less CDR With Asynchronous Baud-Rate Sampling for Frequency Acquisition and Adaptive Equalization

Burst Photography! EE367/CS448I: Computational Imaging and Display! stanford.edu/class/ee367! Lecture 7! Gordon Wetzstein! Stanford University!

arxiv: v1 [cs.cv] 29 May 2018

A Review Paper on Image Processing based Algorithms for De-noising and Enhancement of Underwater Images

High Dynamic Range (HDR) Photography in Photoshop CS2

Interpolation of CFA Color Images with Hybrid Image Denoising

MODIFICATION OF ADAPTIVE LOGARITHMIC METHOD FOR DISPLAYING HIGH CONTRAST SCENES BY AUTOMATING THE BIAS VALUE PARAMETER

Correction of Clipped Pixels in Color Images

NEIGHBORHOOD electric vehicles (NEVs) are propelled

A New Auto Exposure System to Detect High Dynamic Range Conditions Using CMOS Technology

EFFICIENT CONTRAST ENHANCEMENT USING GAMMA CORRECTION WITH MULTILEVEL THRESHOLDING AND PROBABILITY BASED ENTROPY

Contrast Enhancement Using Bi-Histogram Equalization With Brightness Preservation

T direct digital synthesizer. The most elementary technique

Artifacts Reduced Interpolation Method for Single-Sensor Imaging System

Recursive Plateau Histogram Equalization for the Contrast Enhancement of the Infrared Images

OSCILLATORS and timers are critical to all systems with

Extended Dynamic Range Imaging: A Spatial Down-Sampling Approach

Mahjong Gambling in the Chinese-Australian Community in Sydney: A Prevalence Study

A No Reference Image Blur Detection using CPBD Metric and Deblurring of Gaussian Blurred Images using Lucy-Richardson Algorithm

Removal of High Density Salt and Pepper Noise through Modified Decision based Un Symmetric Trimmed Median Filter

Measure of image enhancement by parameter controlled histogram distribution using color image

Colour correction for panoramic imaging

An Improved Bernsen Algorithm Approaches For License Plate Recognition

Pixel Classification Algorithms for Noise Removal and Signal Preservation in Low-Pass Filtering for Contrast Enhancement

PERCEPTUAL QUALITY ASSESSMENT OF HDR DEGHOSTING ALGORITHMS

Batch Fabrication of Metasurface Holograms Enabled by Plasmonic Cavity Lithography

A Single Image Haze Removal Algorithm Using Color Attenuation Prior

FOG REMOVAL ALGORITHM USING ANISOTROPIC DIFFUSION AND HISTOGRAM STRETCHING

High dynamic range imaging and tonemapping

130 IEEE TRANSACTIONS ON ANTENNAS AND PROPAGATION, VOL. 60, NO. 1, JANUARY 2012

Efficient Image Retargeting for High Dynamic Range Scenes

Fuzzy Statistics Based Multi-HE for Image Enhancement with Brightness Preserving Behaviour

A Gentle Introduction to Bilateral Filtering and its Applications 08/10: Applications: Advanced uses of Bilateral Filters

Fast and High-Quality Image Blending on Mobile Phones

PERCEPTUAL QUALITY ASSESSMENT OF HDR DEGHOSTING ALGORITHMS

Continuous Flash. October 1, Technical Report MSR-TR Microsoft Research Microsoft Corporation One Microsoft Way Redmond, WA 98052

Figure 1 HDR image fusion example

Reversible Data Hiding in Encrypted color images by Reserving Room before Encryption with LSB Method

Realistic Image Synthesis

Transcription:

A. Taşyapı et al.: Fuzzy Fusion Based High Dynamic Range Imaging using Adaptive Histogram Separation 119 Fuzzy Fusion Based High Dynamic Range Imaging using Adaptive Histogram Separation Aysun Taşyapı ÇELEBİ, Ramazan DUAR, and Oğuzhan URHAN, Member, IEEE Abstract In this work, a high dynamic range (HDR) image generation method using a single input image is presented. The proposed approach generates over- and under-exposed images by making use of a novel adaptive histogram separation scheme. Thus, it becomes possible to eliminate ghosting effects which generally occur when several input image containing camera/object motion are utilized in HDR imaging. Additionally, it is proposed to utilize a fuzzy logic based approach at the fusion stage which takes visibility of the inputs pixels into account. Since the proposed approach is computationally light-weight, it is possible to implement it on mobile devices such as smart phones and compact cameras. Experimental results show that the proposed approach is able to provide ghost-free and improved HDR performance compared to the existing methods 1. Index Terms High dynamic range, HDR, Histogram separation, Ghost-free. I. INTRODUCTION Traditional imaging devices can capture limited range of intensities. A conventional consumer grade camera has typically 24-bit depth per pixel and 256 discrete intensity levels for each color channel of the image. However, light intensity in the real world has much larger dynamic range compared to that can be shown on digital images or computer screens. In order to overcome dynamic range limitation of the typical low dynamic range (LDR) images, High Dynamic Range (HDR) imaging techniques have been utilized in recent years as an alternative approach for digital imaging [1]. HDR images generally have higher quality and powerful appearance. In HDR imaging, usually a set of differently exposed low dynamic range (LDR) images are fused into a single image to overcome the dynamic range limitation of existing imaging sensors. As a result, the fused image may have higher amount of information and enhanced visual quality because of its extended dynamic range. Recently, various approaches have been proposed in the literature to produce HDR images with extended dynamic range using LDR images. In these techniques, generally multiple images of the scene are captured at different exposure times via a single camera and fused to form the HDR image 1 A.T. Celebi is with the Electronics and Telecom. Eng. Dept., University of Kocaeli, Umuttepe Campus, 41380, Izmit/Kocaeli, Turkey (e-mail: aysun.tasyapi@kocaeli.edu.tr) R. Duvar and O. Urhan are with the Kocaeli University Laboratory of Embedded and ision Systems (KULE), Electronics and Telecom. Eng. Dept., Umuttepe Campus, 41380, İzmit/Kocaeli, Turkey (e-mail: ramazan.duvar@kocaeli.edu.tr, urhano@kocaeli.edu.tr). Contributed Paper Manuscript received 12/31/14 Current version published 03/30/15 Electronic version published 03/30/15. 0098 3063/15/$20.00 2015 IEEE [2], [3]. Shen et al [4] proposed a novel algorithm based on extended Retinex model to fuse multi-exposure images. Instead of composing the input image intensities, this approach decomposes them into the luminance and reflectance components which are composed independently before being merged together to produce the final result. Gu et al [5] proposed an algorithm to fuse multi-exposure images in the gradient field. Fused gradient field was derived from the structure tensor of inputs based on multi-dimensional Riemannian geometry with a Euclidean metric. However, these approaches are mostly suitable for static scenes only. In case of dynamic scenes with camera motion and moving objects such as car, people etc., various artifacts such as ghosting cannot be avoided due to the sequential capture of differently exposed images. There are various methods in which the camera and local motion are compensated to eliminate ghosting effect. Jacobs et al [6] proposed a ghost detection approach based on entropy computation. The local entropy of each LDR image is extracted and then added to the difference of local entropy values with weighting factors computed between LDR images in this method. Khan et al [7] presented an iterative ghost effect removal approach. This method generates HDR image by weighting the probability of the pixels belonging to the background or the moving objects. Gallo et al [8] removed the ghost using a patch based method where the Poisson blending is performed for removing the boundary discontinuity. Im et al [9] proposed LDR image registration where the camera motion is compensated to align multiple LDR images using elastic registration method for removing ghost effect. Zheng et al [10] proposed a hybrid patching algorithm to prevent ghosting effect where a new optimization problem is formulated to correct the motion regions of differently exposed images by considering both spatial and temporal consistencies. Zhang et al [11] proposed to use gradient direction changes among the different images for object motion detection. The main drawback of this approach is that it estimates the moving objects without considering the local properties of ghost regions. In addition, this approach assumes that the moving object occupies only a small portion of the image. Zhang et al [12] proposed a new de-ghosting algorithm to overcome these shortcomings of the method presented by Zhang et al [11] by taking not only the temporal consistency but also spatial consistency. Thus, this approach generalizes the previous assumption and allows higher amount of local motion. However, if an inappropriate exposed image is selected as the reference view, this approach may fail to produce acceptable HDR images. Lee et al [13] proposed to obtain a ghost-free

120 IEEE Transactions on Consumer Electronics, ol. 61, No. 1, February 2015 N-Exposed Image I NE Input Image I RGB/HS H S Adaptive WHS 0 LS 0_LS CLAHE OE O-Exposed Image HS/ RGB I OE Fuzzy Based Fusion HDR Image 1 LS 1_LS CLAHE UE U-Exposed Image HS / RGB I UE Fig. 1. The flowchart of proposed method. high dynamic range image based on a low-rank matrix completion. The background and moving objects are represented as a low-rank matrix and sparse matrix, respectively. Next, the ghost region detection is formulated as the low-rank matrix completion problem with multiple physical constraints on the properties of the ghost regions. However, these kind of approaches have considerably higher computational complexity which may not be suitable for mobile/consumer level cameras and real-time applications. Recently, Im et al [14] proposed to generate high dynamic range (HDR) images using a single image instead of acquiring differently exposed images in order to avoid ghosting effect. This method utilizes weighted histogram separation (WHS) to estimate the threshold for histogram separation. Then it divides the histogram into two sub-histograms and generates differently exposed (over and under) LDR images from a single image. This method is not affected by ghosting effect inherently since it generates differently exposed images from a single image. However, this method utilizes a fixed weighting factor for histogram separation which is not suitable for images displaying different characteristics. Im et al [15] proposed a similar approach for generating ghost-free HDR images. This method utilizes histogram quantization based spatially adaptive histogram equalization approach and generates two LDR images using single input image by performing two sub-histogram equalizations. Finally, it fuses these LDR images with the input image to produce the HDR image by making use of a fixed weight for all input images. In this paper, it is proposed to produce HDR image using a single image similar to [14]. However, the proposed approach computes an adaptive weight for each image to improve HDR performance. Additionally, contrast limited adaptive histogram equalization (CLAHE) is utilized to improve overall appearance of the HDR image in local dark and bright regions. As a final contribution, fuzzy logic based fusion of LDR images is presented which takes pixel visibility into account. Thus, the proposed single image based HDR approach produces an image which has increased dynamic range and more local details. II. PROPOSED ADAPTIE HDR APPROACH The proposed single image based HDR approach has three main steps. The first step is to generate LDR images from a single original image by making use of an adaptive approach. At the second step, these LDR images are enhanced using a hybrid enhancement approach which includes global Linear Stretching (LS) and local CLAHE. Finally, the LDR images are fused employing a fuzzy logic based approach to obtain output HDR image. The block diagram of the proposed method is shown in Fig. 1. The novel contributions of this work are highlighted in this figure. The proposed method utilizes adaptive weighted histogram separation firstly in order to perform single image based HDR. The weighted histogram separation (WHS) is presented by Pei et al [16]. However, a fixed weight at the WHS process is not suitable for wide range of images. Thus, it is proposed to obtain adaptive weights specific to input images. It is important to note that only the (alue) channel is used along the HDR process. The WHS method with the proposed adaptive weight computation is applied to the channel to obtain initial under- and over-exposed images. LS and CLAHE are executed on these images to obtain final under- and overexposed images. It is important to note that, by the introduction of the CLAHE, local details in the input images are extracted efficiently. Finally, the HDR image is constructed from these three differently exposed LDR images by making use of a fuzzy based fusion approach which takes pixel visibility into account. A. Adaptive Weighted Histogram Separation Histogram separation stage is one of the most important part for single image based HDR methods because it directly affects the generation of LDR images and subsequent stages in HDR process. In the proposed approach, two differently exposed LDR images are generated from the channel of the input image. The WHS as explained by Pei et al [16] is used to divide the input image into two separate images by making use of an adaptively computed weight. WHS can be carried

A. Taşyapı et al.: Fuzzy Fusion Based High Dynamic Range Imaging using Adaptive Histogram Separation 121 out using so-called data separation units (DSUs) which separate an image dataset into two subsets. While in practice it is possible to further separate each subset into subsequent subsets using a DSU hierarchy, it has been observed that using only one level is sufficient in terms of image quality and computational complexity. Note that H is used to denote the histogram of the channel of the input image, that is the dataset used in WHS, and H () l denotes the number of pixels with gray level l in, where l ranges from 0 to 255 for an 8-bit image. The following steps describe the WHS. Step 1: Compute the threshold for separating : t 1 arg min w H ( l) (1) n 0 t l0 where t denotes a gray-level value, and n is the total number of pixels in the dataset, w is an experimentally determined weighting factor controlling the size of the two separated subsets. Im et al [14] sets this weighting factor to a fixed number. Thus, this approach is not able to separate histograms efficiently for images with different characteristics. In this paper, it is proposed to utilize an adaptive weighting scheme to separate input image histograms in order to obtain LDR images for a better HDR image. The experiments performed show that a relatively small weighting factor results in detail loss in the HDR image whereas a higher weighting factor is not able to improve dynamic range of the input image. On the other hand, a mid-range fixed weight cannot improve the image quality for relatively brighter and darker images. Thus, it is very important to decide a proper weight for different kind of input images. Thus, in this paper, it is proposed to utilize an adaptive weight based histogram separation approach. The experiments carried out reveal that there is a nearly linear relation between the amount of the pixels in relatively dark region and the optimal weights. The ratio of dark region to overall pixel intensities is defined as r b ma l H ( m) H () l where a and b are fixed intensity levels that correspond to the dark region in the input image. Fig. 2 shows optimal weights obtained by means of visual examination versus r for twelve input images having different characteristics. Note that a and b are set to 0 and 64, respectively for this examination. As seen from Fig. 2, optimal weights and r values have relatively linear relationship with some exceptions. In order to estimate linear model parameters, two approaches are examined: least squares and RANSAC (RANdom SAmple Consensus) [17]. Since the least squares approach takes all (2) Fig. 2. Optimal weights versus r values and their linear relation estimated by least squares fitting and RANSAC methods. input data into account, it suffers from outliers. On the other hand, RANSAC eliminates outliers by making use of an iterative approach. Because of this improvement, RANSAC is able to fit a better model between optimal weight and r as seen from Fig. 2. The adaptive weight w in the proposed approach is determined as follow: w r (3) where and are the estimated model parameters by RANSAC. Step 2: After estimating the adaptive weight factor (w) and computing the threshold ( ), divide the input histogram into two sub histograms based on the threshold value: H H 0 1 l l H l, if l (4) 0, otherwise H l, if l (5) 0, otherwise Two images are generated by dividing the histogram into two subsets making use of the adaptive WHS approach. While the first image referred to as 0 contains intensity levels from 0 to the threshold, the second image referred to as 1 contains intensity levels from the threshold to 255, for an 8-bit depth image. Note that, 0 and 1 is used to obtain the over- and under-exposed images, respectively. B. Linear Stretching The linear stretching enables efficient utilization of whole dynamic range and is applied to 0 and 1. The linear stretching can be formulated as 0_ LS 0, min 0, 0x y 0x y x y x y x, y 255 max, min, (6)

122 IEEE Transactions on Consumer Electronics, ol. 61, No. 1, February 2015 where 0 x, y denotes the images obtained by applying WHS, and x y shows the over-exposed 0_ LS, image obtained after the linear stretching. The under-exposed image 1_ LS x, y is obtained by processing 1 in the same way. C. Contrast Limited Adaptive Histogram Equalization Im et al [14] propose to apply only linear stretching to 0 and 1 for obtaining differently exposed images. In this paper, it is proposed to apply CLAHE [18] after linear stretching in order to improve appearance of the local details in HDR image. Note that a recently presented approach by Çelebi [19] proposes to utilize CLAHE together with linear stretching specifically for dynamic range improvement on underwater images. Adaptive histogram equalization (AHE) is a method for local contrast enhancement and it is an extension to the traditional histogram equalization technique. AHE simply partitions the image into non-overlapping regions and applies histogram equalization to each sub-region in order to redefine the pixel values of the image. As a result, contrast of each region is enhanced by improving local details. One of the important problem with the AHE is the over amplified noise in relatively homogeneous regions. CLAHE is an improved version of AHE which prevents over amplification of noise in homogenous regions by limiting the contrast. In CLAHE, each sub-histogram is partially flattened by clipping the values which are larger than a given threshold, and distributing them to other bins in the histogram. By selecting the clipping level of the histogram, undesired noise amplification can be reduced. In the proposed HDR approach, CLAHE is applied to the 0_LS and 1_ LS separately to enable enhancement of local details. Thanks to CLAHE, a higher dynamic range is achieved on local regions by virtue of local pixel statistics. D. Fuzzy Logic Based Exposure Fusion The final stage of the proposed method is the fusion of over-, under- and normally-exposed images to generate the HDR image. Firstly, the over-exposed channel ( OE ) and under-exposed channel ( UE ) are combined individually with the unprocessed H and S channels and transformed back into the RGB color space to obtain the over-exposed and under-exposed images for fusion process. Fig. 3 shows an example of under-exposed ( I UE ) and over-exposed ( I OE ) images together with the normally-exposed image ( I NE ). The idea of exposure fusion is to exploit the best parts of the differently exposed images and fuse them to generate a high quality image. There are several quality metrics for this purpose, such as well-exposedness, saturation, contrast, or variance. In this paper, it is proposed to utilize pixel visibility criterion to determine contribution of LDR images (i.e. I UE, I OE and I NE ) on the final HDR image. The pixel visibility can be (a) (b) (c) Fig. 3. (a) The normally-exposed image ( I NE ), (b) the over-exposed image, (c) the under-exposed image ( I UE ). (a) The normalized difference between local max and min values (b) Normalized pixel intensity (c) Output Fig. 4. Membership functions of inputs and output of the fuzzy fusion method. assessed by taking only the pixel values into account as in the method presented by Im et al [14]. However, the detailed experiments reveal that it might be possible to improve image quality when local properties of the pixels are also considered. Based on this observation, it is proposed to analyze the amount of the local details surrounding each pixel in LDR images. Local variance computation for each pixel provides important information about the amount of details in the local region. However, its computational cost is not suitable for devices having limited processing capabilities. Thus, it is proposed to capture same information by making use of local min-max operations with a fixed window length. The difference between local extreme values provides evidence about the amount of local details within the examined region. Another important information about the pixel visibility is its intensity value. In this paper, it is proposed to utilize these two important evidences about the pixel visibility by making use of a fuzzy approach to decide contribution level of each pixel in LDR images to the HDR image. The smoothed and normalized difference between the local extremes is assigned as the first input of the fuzzy system whereas the normalized pixel value of the channel is considered as the second input. The smoothing is required to eliminate unwanted local noise in the fuzzy inputs. The membership functions for the inputs and output are shown in Fig. 4.

A. Taşyapı et al.: Fuzzy Fusion Based High Dynamic Range Imaging using Adaptive Histogram Separation 123 TABLE I. RULE BASE OF THE PROPOSED FUZZY FUSION APPROACH Rule Number I1 I2 Output 1 mf1 mf1 mf1 2 mf1 mf2 mf1 3 mf1 mf3 mf1 4 mf2 mf1 mf3 5 mf2 mf2 mf2 6 mf2 mf3 mf1 7 mf3 mf1 mf3 8 mf3 mf2 mf2 9 mf3 mf3 mf1 The rule base for the proposed fuzzy fusion approach is given in Table I. As seen from the rule base, when the difference between local extremes around a pixel is low then the contribution of this pixel on the HDR image will be low even though the pixel intensity is relatively high. These rules eliminate erroneous contribution of the pixels located at the homogenous region. It is important to note that the contribution of the pixels having mid-level values are higher compared to that of relatively low-level and high-level pixel values (see Fig. 4(b)). Once the contribution level of each pixel in the LDR images is determined the final HDR image is constructed as I c HDR,, c Ik x y FuzzyWeight k x y koe, UE, NE x, y, FuzzyWeight x, y for c R, G, B k OE, UE, NE c c where Ik and I HDR are denote LDR images and the output HDR image, respectively. III. EXPERIMENTAL RESULTS The proposed method is tested using various input images that have different characteristics. Fig. 5 shows comparative results of the approach presented by Im et al [14] and proposed approach for different input images. As described in previous section, Im et al [14] proposed to generate HDR image using histogram separation with a fixed weight for all input images. k (7) (a) (b) (c) Fig. 5. Performance comparison. (a) Original images, (b) HDR images obtained by Im et al [14], (c) HDR images obtained by the proposed method.

124 IEEE Transactions on Consumer Electronics, ol. 61, No. 1, February 2015 However, in this paper, the weight is estimated adaptively for each input image. The results shown in Fig. 5 for the approach presented by Im et al [14] are obtained for a fixed weight of 0.3 which is determined experimentally as the average optimum weight. These results show that the fixed weight does not perform well for different type of images. But the proposed method obtains better results thanks to adaptive weight. The adaptive weights of these images are computed as 0.44, 0.16 and 0.18, respectively by the proposed approach. It is seen from these results that there is considerable improvement in the dynamic range and visual quality of the images. In addition, HDR images generated by the proposed method have more details, because the regions are properly selected from the LDR images. When the first row of the Fig. 5 is evaluated, it is obvious that the proposed method provides superior results compared to the approach presented by Im et al [14]. Especially, the details on the sand are extracted efficiently compared to the approach proposed by Im et al [14]. At the second row of the Fig. 5, the proposed method gives more details on the floor (i.e. paving stone) and shadow region. Additionally, the color of the trees is more appealing compared to the original image and HDR image produced by Im et al [14]. At the third row of the Fig. 5, the clouds on the sky provide good visual quality compared to the original image and the method presented by Im et al [14]. Furthermore, the output image of the proposed method is more brilliant and crisp. As described in the previous section, it is proposed to apply CLAHE after linear stretching in this paper. In Fig. 6, images are shown together with the zoomed area from the highlighted portions of the images. In this figure, the first column shows the original image, whereas second and third column denote the HDR result of only LS and LS+CLAHE cases, respectively. As seen from these images, the proposed hybrid scheme enhances the details, particularly in the shadow and bright regions. (a) (b) (c) Fig. 6. The contribution of the CLAHE. (a) Original images, (b) Images obtained after applying LS only for generating HDR, (c) Images obtained after applying the proposed method (LS+CLAHE).

A. Taşyapı et al.: Fuzzy Fusion Based High Dynamic Range Imaging using Adaptive Histogram Separation In addition to the method presented by Im et al [14], the HDR performance of the proposed approach is compared against to some recent techniques [11]-[13] where multiple input images are required to generate HDR image. In Fig. 8 and Fig. 9, original differently exposed input images and HDR images generated by the methods presented by Zhang et al [11], Zhang et al [12], Lee et al [13], Im et al [14] and the proposed method are shown for visual assessment. As seen from these figures, the methods presented by Zhang et al [12] and Lee et al [13] still suffer from ghost effect even though these approaches have anti-ghost schemes. In order to evaluate the contribution of fuzzy based fusion of LDR images introduced in this work, appearance based fusion approach presented by Im et al [14] is integrated into the proposed method. The visual results are shown in Fig. 7 for comparison. As seen from this figure, the proposed fuzzy logic based fusion approach not only improves the general HDR performance but also enables efficient HDR image generation for extremely dark regions. In general, these experiments show that the proposed method is able to improve visual quality of the images having different characteristics compared to original images and a recently proposed single image based HDR technique proposed by Im et al [14]. (a) 125 (b) (c) Fig. 7. The contribution of the fuzzy based fusion. (a) Original image, (b) HDR image obtained after applying exposure fusion by Im et al [14], Image obtained after applying the proposed method (Fuzzy based fusion). (a) (b) (c) (d) (e) (f) (g) (h) (i) (j) (c) Fig. 8. (a-e) The differently exposed images (Image1 sequence), HDR image obtained after applying the approach by (f) Zhang et al [11], (g) Zhang et al [12], (h) Lee et al [13], (i) Im et al [14], (j) proposed method.

126 IEEE Transactions on Consumer Electronics, ol. 61, No. 1, February 2015 (a) (b) (c) (d) (e) (f) (g) (h) (i) Fig. 9. (a-d) The differently exposed images (Image2 sequence), HDR image obtained after applying the approach by (e) Zhang et al [11], (f) Zhang et al [12], (g) Lee et al [13], (h) Im et al [14], (i) proposed method. TABLE II. OBJECTIE IMAGE QUALITY ASSESSMENT Image 1 Image 2 Entropy CPBD Entropy CPBD Zhang et al [11] 7.752 0.758 7.171 0.768 Zhang et al [12] 7.385 0.775 7.423 0.764 Lee et al [13] 7.623 0.773 7.629 0.715 Im et al [14] 7.358 0.774 7.637 0.784 Proposed Method 7.648 0.743 7.644 0.789 When the visual HDR performance of the other compared methods is evaluated, the proposed method has better visualization results compared to the methods presented by Zhang et al [11] and Im et al [14]. The HDR performance of the proposed method is also examined by making use of some objective blind image quality criteria such as entropy and Cumulative Probability of Blur Detection (CPBD) [20]. The entropy measures the richness of information in HDR image. Hence, the higher entropy simply means better performance. The CPBD is based on the just noticeable blur (JNB) and is able to predict the relative amount of blurriness in images with different content. A higher value of this metric depicts a sharper image. As the blurriness in an image increases the value of this metric is expected to decrease. Table II shows the entropy and CPBD values of the proposed approach and other methods. As seen from this table the proposed approach provides good performance in this objective evaluation as well. The approach presented by Im et al [14] and the proposed approach are implemented on a smart phone with a 1.9GHz CPU using C++ language at the same software optimization level. Table III shows average execution times in milliseconds for full HD (1920 1080 pixel) input images. As seen from this table, the computational complexity before the fusion stage is similar. The additional processing time at this stage for the proposed method originates from the implementation of CLAHE introduced in this paper. At the fusion stage, the approach presented by Im et al [14] is implemented by making use of a 1-dimensional Look-Up Table (LUT) based weighting factor computation in addition to color space conversion. However, in the proposed approach, additional local extreme computation and filtering is required to decide contribution level of each pixel in the LDR images to final HDR image. TABLE III. AERAGE EXECUTION TIMES ON A SMART PHONE Im et al [14] Proposed Approach Before fusion stage 814 ms 943 ms Fusion stage 527 ms 1684 ms Total 1341 ms 2627 ms These extra computations consume around additional 1300ms compared to the method presented by Im et al [14]. For the implementation of the fuzzy fusion stage in the proposed method, a 2-dimensional LUT is generated by sampling inputs at 0.001 resolution. Thus, the size of LUT is 1000 1000 which requires an additional 8MB of memory. Compared to the 1-dimensional LUT utilized for the implementation of the method presented by Im et al [14], the proposed fuzzy based fusion requires an additional 100ms processing time. Thus, the proposed approach requires approximately two times higher processing time in total for HDR image generation process. However, it is important to note that the proposed approach provides significantly better HDR performance compared to the method presented by Im et al [14]. Additionally, it might be possible to improve computation time of the local extreme calculations by making use of some software optimizations. I. CONCLUSION In this paper, a novel method to generate HDR image using a single input image is presented. This method utilizes an adaptive histogram separation approach to produce over- and under-exposed images. The CLAHE based image enhancement enables efficient extraction of local details. Finally, the proposed fuzzy logic based fusion approach facilitates to construct an attractive HDR image with the efficient combination of the LDR images obtained in the previous stages. The proposed method generates ghost-free HDR images inherently because it uses a single image instead of using multiple differently exposed images. Hence, this method is not affected by the moving objects and camera movement. Additionally, the low computational complexity of the proposed approach makes it suitable for smart phones and compact consumer cameras.

A. Taşyapı et al.: Fuzzy Fusion Based High Dynamic Range Imaging using Adaptive Histogram Separation 127 REFERENCES [1] G. Ward, E. Reinhard, S. Pattanaik, and P. Debevec, High dynamic range imaging: acquisition, display, and image-based lighting, 2nd ed., Morgan Kaufmann Publisher, 2005. [2] P. Debevec, and J. Malik, Recovering high dynamic range radiance maps from photographs, in Proc. ACM SIGGRAPH, Los Angeles, USA, pp. 369-378, Aug. 1997. [3] Y. Bandoh, G. Qiu, M. Okuda, S. Daly, T. Aach, and O. Au, Recent advances in high dynamic range imaging technology, in Proc. IEEE Int. Conf. on Image Processing, Hong Kong, pp. 3125-3128, Sep. 2010. [4] F. Shen, Y. Zhao, X. Jiang, M. Suwa, Recovering high dynamic range by Multi-Exposure Retinex, J. is. Commun. Image Represent., vol. 20, no. 8, pp. 521-531, Nov. 2009. [5] B. Gu, W. Li, J. Wong, M. Zhu, and M. Wang, Gradient field multiexposure images fusion for high dynamic range image visualization, J. is. Commun. Image Represent., vol. 23, no. 4, pp. 604-610, May 2012. [6] K. Jacobs, C. Loscos, and G. Ward, Automatic high dynamic range image generation for dynamic scenes, IEEE Comput. Graph. Appl., vol. 28, no. 2, pp. 84 93, Mar. 2008. [7] E. A. Khan, A. O. Akyuz, and E. Reinhard, Robust generation of high dynamic range images, in Proc. IEEE Int. Conf. on Image Processing, pp. 2005 2008, Atlanta, GA, USA, Oct. 2006. [8] O. Gallo, N. Gelfand, W. Chen, M. Tico, and K. Pulli, Artifact-free high dynamic range imaging, in Proc. Int. Conf. Computational Photography, San Francisco, USA, pp. Apr. 2009. [9] J. Im, S. Lee, and J. Paik, Improved elastic registration for ghost artifact free high dynamic range imaging, IEEE Trans. Consum. Electron., vol. 57, May 2011. [10] J. Zheng, Z. Li, Z. Zhu, S. Wu, and S. Rahardja, Hybrid patching for a sequence of differently exposed images with moving objects, IEEE Trans. Image Process., vol. 22, no. 12, pp. 5190 5201, Dec. 2013. [11] W. Zhang and W-K. Cham, Gradient-directed multi-exposure composition, IEEE Trans. on Image Process., vol. 21, no. 4,pp. 2318-2323, Apr. 2012. [12] W. Zhang and W-K. Cham, Reference-guided exposure fusion in dynamic scenes, J. is. Commun. Image Represent., vol. 23, no. 3, pp. 467-475, Apr. 2012. [13] C. Lee, Y. Li, and. Monga, Ghost-free high dynamic range imaging via rank minimization, IEEE Signal Process. Lett., vol. 21, no. 9, Sep. 2014. [14] J. Im, J. Jeon, M. Hayes, and J. Paik, Single image-based ghost-free high dynamic range imaging using local histogram stretching and spatially-adaptive denoising, IEEE Trans. Consum. Electron., vol. 57, no. 4, pp.1478-1484, Nov. 2011. [15] J. Im, and J. Paik, Spatially adaptive histogram equalization for single image-based ghost-free high dynamic range imaging, J. Arts, Imaging Science, vol. 1, no. 1, pp. 6-13, Feb. 2014. [16] S. Pei, Y. Zeng, and J. Ding, Color images enhancement using weighted histogram separation, in Proc. IEEE Int. Conf. on Image Processing, Atlanta, GA, USA, pp. 2889-2892, Oct. 2006. [17] M.A. Fischler and R. C. Bolles, Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography, Comm. of the ACM vol. 24, no. 6, pp. 381 395, 1981. [18] S. M. Pizer, Adaptive histogram equalization and its variations, Comput. ision, Graph. Image Process, vol. 39, pp. 335-368, 1987. [19] A.T. Çelebi, Image Enhancement and Target Detection in Underwater Images, PhD Dissertation, Electronics and Telecom. Engineering Dept. of University of Kocaeli, 2011. [20] N.D. Narvekar, L.J. Karam, A no-reference image blur metric based on the cumulative probability of blur detection (CPBD), IEEE Trans. on Image Process., vol. 20, no. 9, pp. 2678-2683, Mar. 2011. BIOGRAPHIES Aysun Taşyapı Çelebi was born in Çanakkale, Turkey. She received her B.Sc., M.Sc. and Ph.D. degrees in Electronics and Telecommunication engineering from Kocaeli University, Kocaeli, Turkey, in 2002, 2008, and 2012, respectively. Since 2013 she has been with the Department of Electronics and Telecommunications Engineering, University of Kocaeli, Turkey, where she is currently working as an Assistant Professor. Her research interests include digital signal and image/video processing. Ramazan Duvar received his B.Sc., M.Sc., degrees in Electronics and Telecommunication engineering from the University of Kocaeli, Kocaeli, Turkey, in 2010, 2012, respectively. Currently, he is pursing towards to Ph.D. degree. Since 2011, he has been with the Department of Electronics and Telecommunications Engineering, University of Kocaeli, Turkey as a research assistant. His research interests include image processing and embedded systems. Oğuzhan Urhan (S 02-M 06) received his B.Sc., M.Sc., and Ph.D. degrees in Electronics and Telecommunication engineering from the University of Kocaeli, Kocaeli, Turkey, in 2001, 2003, and 2006, respectively. Since 2001, he has been with the Department of Electronics and Telecommunications Engineering, University of Kocaeli, Turkey, where he is currently associate professor. He was a visiting professor at Chung-Ang University, South Korea, from 2006 to 2007. He is the director of Kocaeli University Laboratory of Embedded and ision Systems (KULE). His research interests include digital signal, image/video processing and embedded systems.

本文献由 学霸图书馆 - 文献云下载 收集自网络, 仅供学习交流使用 学霸图书馆 (www.xuebalib.com) 是一个 整合众多图书馆数据库资源, 提供一站式文献检索和下载服务 的 24 小时在线不限 IP 图书馆 图书馆致力于便利 促进学习与科研, 提供最强文献下载服务 图书馆导航 : 图书馆首页文献云下载图书馆入口外文数据库大全疑难文献辅助工具