A Fuzzy Logic Based Approach to De-Weather Fog-Degraded Images

Transcription

1 2009 Sixth International Conference on Computer Graphics, Imaging and Visualization A Fuzzy Logic Based Approach to De-Weather Fog-Degraded Images Nachiket Desai,Aritra Chatterjee,Shaunak Mishra, Dhaval Chudasama,Sunav Choudhary and Sudhirkumar Barai Dept. of Electronics and Electrical Communication Engineering Dept. of Electrical Engineering Dept. of Civil Engineering Indian Institute of Technology, Kharagpur, West Bengal, India nachiketd@iitkgp.ac.in, {yours.aritra, shaunak.mishra.iitkgp}@gmail.com, {dhavalchudasama, sunavch, sudhirkumar.barai}@gmail.com Abstract Poor visibility in foggy weather stems from the fact that particles in atmosphere scatter and absorb light from the environment and light reflected from the objects. Mathematically, de-weathering a fog degraded image is an ill posed problem and existing approaches are of high complexity and low versatility. In this paper, a novel fuzzy logic based algorithm, to de-weather fog-degraded images, is proposed. Specifically, air-light estimation is carried out using fuzzy logic followed by color correction for enhanced visibility. Experimental results show that the algorithm works effectively for images with a sky region. Due to its low complexity compared to conventional physics based solutions, the algorithm makes real-time implementation possible on a mobile platform which is crucial from a road safety viewpoint. Keywords-Fuzzy logic, Image enhancement. and VII describe the two aspects of the algorithm viz. visibility enhancement and color enhancement in detail. Section VIII presents simulation results and Section IX concludes the paper. II. PROBLEM DEFINITION Given a fog-degraded digital image taken outdoors with a sufficiently large region consisting of the sky, the underlying objective is to de-weather the digital image leading to enhanced visibility in the image. It has been further assumed that the image has been taken by a standard digital camera which works in the visible spectrum only and that multiple copies of the same image taken from different views are unavailable. I. INTRODUCTION Fog and haze are integral and unavoidable features of nature which affect our outdoor visual cognition capabilities. Unfortunately, they pose serious problems when poor visibility in foggy conditions leads to road accidents. Apart from road safety issues, they are detrimental to the performance and reliability of any outdoor visual surveillance system. Most often, the effects of fog lead to a significant loss in clarity within the picture. The challenge is to restore the clarity of such images by solely relying on image processing tools applied on a single picture of the scene taken by a visible spectrum sensor (viz. an ordinary camera). Specialized equipment (viz. infrared sensors) and images taken from multiple viewpoints are expensive, and algorithms based on multiple images of the same scene taken at different points of time are unsuitable for real-time applications. This paper proposes a novel fuzzy logic-aided physics-based solution which enhances visibility in fog degraded images and has a lower complexity than conventional physics-based solutions. The outlined approach aims to achieve the desired objective without compromising on cost or suitability for realtime applications. The remainder of the paper is organized as follows. Section II elucidates the problem, followed by a literature review in section III describing the existing methodologies to solve the problem at hand. Section IV describes the physical model of fog considered to arrive at a solution followed by section V focussing on the solution methodology. Sections VI III. LITERATURE REVIEW Different approaches exist in literature to remove the effects of fog from images. Physics-based solutions, which involve taking multiple images of the same scene by rotating a polarizer attached to the lens of the camera, have been proposed [1], [2]. A major disadvantage of this approach is the limited speed of rotation of the polarizer. Similar approaches, involving the use of multiple images of the same scene in different weather conditions, too have been proposed [3] [5]. Again, such approaches are unsuitable for real-time applications. Another way to solve the problem is to use information about the depth of an image through a user interface [6], [7]. A more efficient approach is visibility enhancement by prediction of air-light using color and intensity information [8], [9], the first one of which uses Hough Transform based methods [10]. This method predicts the air-light, based on different physical model parameters like image chromaticity, light chromaticity and object chromaticity. However, its mathematical complexity demands extensive computation capability. References [11], [12] use wavelet transform to achieve the objective. Among these, the latter works only on gray-scale images. Reference [13] uses extensive mathematical modeling to obtain the characteristics of fog, thus making it computationally complex. Fuzzy logic [14], [15] is a powerful tool for image processing, as outlined in [16]. However, to the best of the authors knowledge, it has not been used to enhance fog-affected color images /09 $ IEEE DOI /CGIV

2 is the simplest way to estimate the air-light since it is constant throughout the image. Fig. 1. Light Transmission under Foggy Conditions The proposed approach aims at taking a single image as input and giving the de-weathered image as output in real time, using fuzzy logic to give reduced complexity, without any external information on depth of the image. IV. PHYSICAL MODEL In clear weather, the atmosphere has very little suspended matter as compared to foggy conditions. Hence, the light transmitted from the object to the observer is scattered minimally, which enables clear vision. However, under foggy conditions the quality of vision degrades due to the light scattering properties of suspended matter in the atmosphere. Scattering causes the object to appear dim while also causing a loss of contrast in the image. This is shown in Fig.1. The light scattering mechanism under foggy conditions as described above has been mathematically modeled here as a linear combination of direct transmission light and air-light. The equation modeling the observed pixel intensity can thus be written as below. E(x) = B(x) Λ(x) + F (x) Γ (1) In (1), x denotes the distance of the point being observed in the image from the observer. Direct transmission light is the intensity of the component of light, from the objects, which reaches the observer. It depends on the ambient light intensity, reflectance of the object being viewed and the attenuation due to the fog. Air-light is the scattered ambient light that reaches the observer. The quantities B and F are scalar constants which depend on the ambient light intensity. B varies as e βx, where β is the atmospheric attenuation coefficient, while F varies as (1 e βx ). This satisfies the common experience that distant objects appear hazier in fog. The parameters Λ and Γ, called the color chromaticity and light chromaticity respectively, are color vectors of the light emanating from the object and the light vector of the ambient light respectively. It must be noted that one cannot apply conventional techniques like histogram equalization because its assumptions are invalid when depth is a variable in the scene [17]. Modeling fog as noise to apply conventional filtering techniques would also be unsuitable. Hence, the objective is to find a suitable transformation for each pixel which selectively attenuates the air-light component and enhances the direct transmission component. Equation (1) suggests that the light chromaticity Γ V. SOLUTION METHODOLOGY A two-stage process is employed to de-weather a given fogaffected image. The first stage involves estimation of the light chromaticity of the image and partially removing the effects of air-light. This stage outputs an image with enhanced visibility but still with a lesser contrast than an ordinary image taken in clear weather. Once the visibility of the image is enhanced, the second stage aims at restoring the natural contrast of the image. This step, called color enhancement, uses an unsupervised color correction algorithm similar to [18]. This is based on a computational model of the human optical cognitive system. It realizes a local filtering effect by taking into account the spatial color distribution in the image, and is able to adapt to widely varying light conditions to extract visual information from the environment efficaciously, similar to the human visual system. It makes a perception of objects reflectance values dependent on the chromatic and spatial composition of the scene. The basic idea of using color enhancement is to mimic some characteristics of the human visual system, in order to increase the apparent level of detail in the resultant image. VI. VISIBILITY ENHANCEMENT Equation (1) shows that as the distance from the object to the observer increases, the component of air-light in the total intensity increases. Hence, to find the characteristics of airlight, one must aim to locate the sky region in the picture. The existence of a sufficiently large sky region is assumed since the input image was taken outdoors. Generally, the sky region, which totally consists of fog-dispersed air-light, is found in the upper part of the image (forming the horizon of a scenery). The pixels in this region also correspond to peaks in the image histogram, corresponding to high gray scale values since the air-light is mostly bright. These properties lead to the definition of three fuzzy input variables, namely image region, pixel value and gray-scale difference. The image region is characterized as one of three fuzzy sets, namely top, middle or bottom, according to the normalized row number. The pixel value is characterized as one of dark, medium or bright. The gray-scale difference is the absolute difference between a pixel value and the rightmost peak in the image histogram. It makes use of the fact that the gray-scaled version of an image taken outdoors, and thus having a significant sky region, would have a peak in its histogram close to the value corresponding to absolute white, with the histogram falling rapidly on both sides of the peak. The gray-scale difference is characterized as one of almost equal, slightly different or highly different. The membership functions for these fuzzy sets are shown in Fig.2, Fig.3 and Fig.4. The Air-light content of a pixel is also characterized as one of three fuzzy sets, namely Very Little, Moderate or High, as shown in Fig.5. From the observations made above, it is clear that a pixel has large air-light component if it is in the top 384

3 Fig. 2. Fig. 3. Fig. 4. Fig. 5. Membership function for the fuzzy set Gray-Scale Difference Membership function for the fuzzy set Pixel Region Membership function for the fuzzy set Image Region Membership function for the fuzzy set Air-Light Content region of the image, and has a high gray-scale value close to the peak in the histogram of the image. This leads to the formulation of three rules by combining the conditions that are equally favorable for a given pixel containing large air-light component. Exploitation of this property gives a fairly good estimate of the average air-light value which drastically reduces the complexity of the image enhancement algorithm. The rule base thus obtained is given below. Rule 1 : If (Grayscale Difference is Almost Equal) and (Pixel value is Bright) and (Image Region is Top), then (Air-light content is High). Rule 2 : If (Grayscale Difference is Highly Different) and (Image Region is Bottom), then (Air-light content is Very Little). Rule 3 : If (Grayscale Difference is Slightly Different) and (Pixel value is Bright) and (Image Region is Middle), then (Air-light content is Moderate). A Mamdani-type Fuzzy Inference System (FIS) has been used here [15]. A weighted addition of the de-fuzzified values of the air-light content yields the ambient light intensity for each of the three colors, namely red, green and blue. The light chromaticity vector Γ is made up of three components (Γ r, Γ g, Γ b ), which correspond to the chromaticity values for red, green and blue respectively. The chromaticity value for a color is calculated as I i Γ i =, i {r, g, b} (2) I r + I b + I g The calculated light chromaticity is then removed from the image by directly dividing the image intensity by the average light chromaticity. This process is called visibility enhancement since it aims at removing air-light, thus improving visibility and restoring the original hue in case of smog. However, the overall saturation of the image still remains high as an effect of fog, which is taken care of in the next stage, i.e. color enhancement. VII. COLOR ENHANCEMENT Although it is known that the saturation of the image needs to be reduced, the magnitude of reduction needs to be determined for every pixel in the image. To achieve this, the image is processed through a color enhancement step to make the features in the image more distinct. In the first step, a chromatic adjustment is done to produce an output image R, whose every pixel is recomputed using the formula: R c (p) = r(i c (p) I c (j)) Where I c (p) and I c (j) are intensity values for a particular color channel, and d(.) function finds the distance of the pixel under consideration from all its surrounding pixels. Every pixel gets an intensity value that is dependent on the intensity of pixels surrounding it. This step therefore, makes a chromatic comparison and does a local/global balancing of color. The above equation can be normalized to the form: R c (p) = r(i c (p) I c (j)) r max Here, r max is the maximum value of r(.). The function r(.) is a distance function of the pixel under consideration from its (3) (4) 385

4 Fig. 8. Results of Fuzzy Logic Enhancement for the same image: (a) Output after Visibility Enhancement, (b) Final Output after Color Enhancement Fig. 6. Flowchart of the Process used to Enhance Fog-affected Images Fig. 9. Typical scene on a road on a foggy day ( beijingsmog.jpg): (a) Input Image, (b) Output Image Fig. 7. (a) Test Image [8], (b) Output by Conventional Algorithms [8] neighboring pixels. By tuning for different distance functions like Euclidean, inverse exponential, Manhattan, etc. the output is observed. The best result is observed on choosing r(.) as Euclidean distance. A flowchart of the complete process is shown in Fig.6. VIII. RESULTS AND DISCUSSION The proposed algorithm has been tested on images of roads and cities during fog, borrowed from various websites on the internet. These images are in standard JPEG format of approximate size 400 by 300 pixels. MATLAB R, R2008b platform was used for simulation purposes [19]. Fig.7(a) is a typical image of an urban skyline affected by fog. Fig.7(b) shows the output of [8] after enhancing this image. Although the image has been cleared to a large extent, the bluish hue of the fog still lingers. Also, the foreground can be cleared further. Fig.8(a) shows the output of visibility enhancement as described here for Fig.7(a). The visibility is clearly increased (since objects much further in the background can be seen) and the original hue is restored (by removal of the bluish tinge). In Fig.8(b), the saturation is effectively reduced for each pixel by color enhancement to introduce a sense of realness in the image. Fig.9(b) demonstrates the performance of the proposed algorithm for another image depicting a vehicle on a road. Although slight distortions appear in the image, it is clearly more detailed and sharp than the original. The vehicles in the image become distinct after de-weathering, showing its relevance in road safety. A notable aspect shown by experimental simulations was that the algorithm works best when there is a hue characterizing the fog since it completely removes the hue and gives the image its natural look. IX. CONCLUSIONS For real-time image processing, digital signal processors are needed along with a digital camera and a display unit. The processing time increases with increasing image size and thus larger images process slower. The image clarity is dependent on the complexity of processing which contributes to the processing time. An option is to use parallel architectures along with multiple cameras which can reduce the processing time but increase the cost. It should be noted that for offline computations where speed is not an issue, the image clarity is the highest priority. The strength of the algorithm lies in its simplicity. The fuzzy logic method used here clearly outperforms all other mathematics and physics-based methods when it comes to computational complexity while obtaining comparable results. The proposed algorithm is useful for outdoor applications only where the images contain a sky region. An indoor image affected by smoke cannot be processed using the same approach to obtain better visibility in the image. This stems from the fact that the algorithm estimates the air-light in an outdoor scene and process the image to remove its effect. The same reason makes it inappropriate for use in nocturnal conditions. Apart from this, the proposed algorithm is incapable of deciding if an image needs deweathering, requiring user intervention to start/stop the deweathering process. Future work may be directed along these 386

5 possibilities. An interesting use of soft computing tools, like fuzzy logic and neural networks, may be for classification of images as foggy or clear. The algorithm may also be extended to other natural conditions like rain, haze, sleet, etc. De-weathering a fog-degraded image is an ill-posed problem. To solve this challenging problem, a novel fuzzy logic based algorithm was proposed in this paper. As far as versatility is concerned, the fuzzy logic framework ensures that for images with a sky-like region, the degradation due to fog is appreciably reduced along with lower computational complexity compared to conventional physics based algorithms. This is acceptable, since most of the fog-affected images are taken outdoors. The proposed algorithm is thus a novel approach to de-weather fog affected images and realize the immense humanitarian value associated with it. REFERENCES [1] S. G. Narasimhan, S. K. Nayar, and Y. Y. Schechner, Instant dehazing of images using polarization, in Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2001., vol. 1, 2001, pp [2] S. Shwartz, E. Namer, and Y. Y. Schechner, Blind haze separation, in IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2006, vol. 2, 2006, pp [3] S. K. Nayar and S. G. Narasimhan, Vision in bad weather, in The Proceedings of the Seventh IEEE International Conference on Computer Vision, 1999, vol. 2, 1999, pp [4] S. G. Narasimhan and S. K. Nayar, Contrast restoration of weather degraded images, IEEE Trans. Pattern Anal. Mach. Intell., vol. 25, no. 6, pp , Jun [5] F. Cozman and E. Krotkov, Depth from scattering, in IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Proceedings., 1997, Jun , pp [6] S. G. Narasimhan and S. K. Nayar, Interactive de-weathering of an image using physical models, in The IEEE Workshop on Color and Photometric Methods in Computer Vision, In Conjunction with ICCV, Oct [7] N. Hautiere, J. Tarel, and D. Aubert, Towards fog-free in-vehicle vision systems through contrast restoration, in IEEE Conference on Computer Vision and Pattern Recognition, CVPR 07, Jun , pp [8] R. T. Tan, N. Petersson, and L. Petersson, Visibility enhancement for roads with foggy or hazy scenes, in IEEE Intelligent Vehicles Symposium, 2007, Jun , pp [9] R. T. Tan, Visibility in bad weather from a single image, in IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2008, Jun , pp [10] R. T. Tan, K. Nishino, and K. Ikeuchi, Color constancy through inverseintensity chromaticity space, J. Optical Society of America A, vol. 21, no. 3, pp , [11] C. Busch and E. Debes, Wavelet transform for analyzing fog visibility, IEEE Intell. Syst., vol. 13, no. 6, pp , Dec [12] Y. Zhai and X. Liu, An improved fog-degraded image enhancement algorithm, in IEEE International Conference on Wavelet Analysis and Pattern Recognition, ICWAPR 07, vol. 2, Nov , pp [13] R. Fattal, Single image dehazing, ACM Trans. Graphics, vol. 27, no. 3, Aug [14] L. Zadeh, Fuzzy sets, Information and Control, vol. 8, no. 3, pp , [15] G. J. Klir and B. Yuan, Fuzzy Sets and Fuzzy Logic - Theory and Applications. Prentice-Hall India, [16] M. Nachtegael, T. Melange, and E. E. Kerre, The possibilities of fuzzy logic in image processing, in Pattern Recognition and Machine Intelligence, ser. Lecture Notes in Computer Science, vol Springer Berlin / Heidelberg, Nov , pp [17] S. Schulte, V. D. Witte, M. Nachtegael, D. V. der Weken, and E. E. Kerre, Histogram-based fuzzy colour filter for image restoration, Image and Vision Computing, vol. 25, no. 9, pp , Sep [18] A. Rizzi, C. Gatta, and D. Marini, A new algorithm for unsupervised global and local color correction, Pattern Recognition Letters, vol. 24, pp , [19] Fuzzy Logic Toolbox Manual, 2nd ed., The Mathworks, Jun