SINGLE UNDERWATER IMAGE RESTORATION BY BLUE-GREEN CHANNELS DEHAZING AND RED CHANNEL CORRECTION

Transcription

1 SINGLE UNDERWATER IMAGE RESTORATION BY BLUE-GREEN CHANNELS DEHAZING AND RED CHANNEL CORRECTION Chongyi Li 1, Jihang Guo 1, Yanwei Pang 1, Shanji Chen, Jian Wang 1,3 1 Tianjin University, Shool of Eletroni Information Engineering Qinghai Nationalities University, College of Physis and Eletroni Information Engineering 3 National Oean Tehnology Center ABSTRACT Restoring underwater image from a single image is know to be ill-posed, and some assumptions made in previous methods are not suitable for many situations. In this paper, we propose a method based on blue-green hannels dehazing and red hannel orretion for underwater image restoration. Firstly, blue-green hannels are reovered via dehazing algorithm based on an extension and modifiation of Dark Channel Prior algorithm. Then, red hannel is orreted following the Gray-World assumption theory. Finally, in order to resolve the problem whih some reovered image regions may look too dim or too bright, an adaptive exposure map is built. Qualitative analysis demonstrates that our method signifiantly improves visibility and ontrast, and redues the effets of light absorption and sattering. For quantitative analysis, our results obtain best values in terms of entropy, loal feature points and average gradient, whih outperform three existing physial model available methods. Index Terms Underwater image restoration, image dehazing, image enhanement, visibility reovery 1. INTRODUCTION Sine the mysterious underwater world ontains abundant resoures, the study of underwater image enhanement and restoration is meaningful, and thus desired in both onsumer photography and omputer vision appliations. However, apturing lear underwater images is hallenging due to physial properties of the underwater environment. The effets of absorption and sattering as well as the varying attenuation of light in different wavelengths ause the degradation of underwater images. Therefore, single underwater image enhanement and restoration have beome a hot spot of researh given its wider appliation range. An underwater image an be represented as a linear superposition of a diret omponent, a forward sattering omponent and a bak sattering omponent [1]. Suh a forward This work was supported by the National Key Basi Researh Program of China (014CB340400) and the Natural Siene Foundation of Qinghai Provine of China (015-ZJ-71). sattering auses blurring of the image features while the bak sattering masks the details of the senario. As shown in Fig. 1, the light intensity dereases with the distane from objets in water by light attenuation depending on the wavelength of light []. Suh a varying attenuation of light in different wavelengths auses olor asts. The overall poor visibility aused by the above-disussed effets limits to the appliations of underwater images, suh as marine biology and arhaeology [3], marine eologial researh [4] and aquati robot inspetion [5]. Fig. 1. Light intensity in water. Numerous approahes are proposed to proess the degraded underwater images and an be desribed from two different perspetives. One is based on image restoration tehnique. Truo and Olmos [6] devised a self-tuning image restoration filter based on a simplified version of Jaffe [7] and M- Glamery [8] underwater imaging formation model. Optimal filter parameter are estimated by optimizing a quality riterion based on a global ontrast measure. et al. [9] proposed a simple prior that exploits the strong differene in attenuation among the three olor hannels of an underwater image in water to estimate the depth of the sene. As a result, the effets of light sattering in underwater images an be removed. et al. [10] proposed an effiient and low omplexity underwater image restoration method based on the Dark Channel Prior algorithm [11]. The median filter is used to estimate the depth map of image instead of the soft matting proedure. Moreover, a olor orretion algorithm is adopted to enhane the olor ontrast of underwater images. Chiang and Chen [1] restored underwater images by om /16/$ IEEE 1731 ICASSP 016

2 bining a dehazing algorithm with wavelength ompensation. The haze effets from olor satter are removed by the Dark Channel Prior algorithm [11]. Aording to the amount of attenuation of eah wavelength, reverse ompensation is onduted to restore the distortion from olor asts. Galdran et al. [13] proposed a Red Channel method, where olor assoiated with short wavelength is reovered and leads to a reovery of the lost ontrast. In sum, the image restoration tehnique an remove the haze in underwater images to some extent. However, those tehniques are limited by the auray of the assumption, optial model and estimated parameters. Another kind of tehnique is based on image enhanement. Anuti et al. [14] proposed a novel strategy to enhane visual quality of underwater images and videos based on the fusion priniples. Chani and Isa [15] improved the ontrast and redued the noise of underwater images through integrated olor model with Rayleigh distribution. Li and Guo [16] proposed an underwater image enhanement method based on dehazing and olor orretion. However, underwater image enhanement tehnique usually produes under-enhaned or over-enhaned regions beause this kind of tehnique is not based on the underwater imaging model. Fig.. Flowhart of the proposed method. 3. OUR METHOD. RELATION TO PRIOR WORK Aording to the seletive absorption theory of water, the red light is muh easier to be absorbed than the blue light and the green light. Moreover, sattering intensity is inversely proportional to the fourth power of wavelength aording to the Rayleigh sattering theory. The shorter wavelengths of the green light and the blue light will satter muh more than the longer wavelength of the red light [17]. Therefore, we an assume that the attenuation of the red light only results from absorption while the attenuation of the blue light and the green light only result from sattering. Unlike previous underwater image restoration works whih apply original Dark Channel Prior algorithm to restore RGB three olor hannels with the same equation, we reover underwater images by bluegreen hannels dehazing and red hannel orretion. The blue-green hannels are reovered using a dehazing algorithm based on an extension and modifiation of the Dark Channel Prior algorithm. Then, the red hannel is orreted following the Gray-World assumption theory. In order to resolve the problem whih some reovered image regions may look too dim or too bright, an adaptive exposure map is built for better visual quality. Fig. shows the flow of the proposed method. The rest of the paper is organized as follows: Setion 3 desribes our method. Setion 4 evaluates and ompares experimental results. Setion 5 onludes the paper Blue-Green Channels Dehazing As disussed above, the attenuation of the blue-green hannels only results from sattering, whih is similar to the hazy images [18]. ne, we proess the blue-green hannels of an underwater image via a dehazing algorithm based on the remarkable progress on single image dehazing theory. The underwater imaging model an be desribed as: I (x) = J (x)t(x) + B (1 t(x)), {g, b}, (1) where x denotes a pixel, I(x) is the observed image, J(x) is the restored image, B is the bakground light, and t(x) [0,1] is the medium transmission map whih represents the perentage of the sene radiane reahing the amera. The purpose of dehazing is to reover J (x), B and t(x) from I (x). The bakground light B an be estimated based on the fat that red hannel attenuates muh faster than green and blue hannels in an underwater image. To determine the differenes among the three olor hannels, the maximum intensity of the red hannel and that of the maximum one of the green and blue hannels are ompared as: D(x) = max x Ω, r I (x) max x Ω, {g,b} I (x), () where D(x) denotes the largest differenes among three different olor hannels, I (x) refers to a pixel x in the observed image, and Ω is a loal path in the image. The bakground light an be estimated as follows: B = avg(i (arg min x D(x))), {g, b}. (3) 173

3 Aording to the Rayleigh sattering theory, the attenuating of the blue light and the green light is the same in water. Thus, we assume the medium transmissions map of the blue and green hannels are idential. Furthermore, we also assume that the medium transmissions map in a loal path is onstant. The Eq. (1) is rearranged and taken the minimum operation in a loal path: min(min( x Ω I (x) J (x) )) = t(x) min(min( ) + 1 t(x), (4) x Ω B B The first term on the right side of the Eq. (4) should tend to be zero based on the Dark Channel Prior theory. Thus, the medium transmission map of the green and blue hannels an be written as: t(x) = 1 min (min( {g,b} x Ω I (x) )). B (5) As shown in Fig. 3, there are some halos and blok artifats in the map t(x). The halos and blok artifats are produed beause t(x) is alulated over an image path, whih produes a oarse initial estimate of the map. To address the problem, the guided filter [19] is applied to refine the oarse map. Fig. 3 displays the refined medium transmission map. where avgrr, avgbr and avggr are the normalized average values of the reovered red hannel, blue hannel and green hannel, respetively. The average value of the reovered red hannel an be estimated as follows: avgrr = 1.5 avgbr avggr. Then, the ompensation oeffiient δ an be alulated as: δ = avgrr/avgr, Fig. 3. Medium transmission map. Original underwater images with a size Coarse medium transmission maps. Refined medium transmission maps. With the refined medium transmission map and the obtained bakground light, we an restore the haze-free green and blue hannels aording to the Eq. (1). Speifially, the haze-free hannel an be restored by: J (x) = I (x) B + B, {g, b}, t (x) (6) where R is the normalized original red hannel, and δ is the estimated ompensation oeffiient. As shown in Fig. 4, after blue-green hannels dehazing and red hannel orretion proessing, larified visibility, alibrated olor asts and enhaned ontrast are ahieved. Based on the observation that the dark or bright regions in underwater images beome too dark or too bright after proessing by our method, we take an adaptive exposure map [0] to adjust our results. The adaptive exposure map s(x) an be obtained by solving optimization problem: s (avgrr + avgbr + avggr)/3 = 0.5, (7) Adaptive Exposure Map Estimation min The absorption rate of red light is hard to be obtained for single underwater image. Inspired by the Gray-World assumption theory that the average value of objet olor in an ideal image is gray, we orret the red hannel following the assumption. It an be written as: Fig. 4. Reovered results. Original underwater images with a size The reovered results without an adaptive exposure map. The reovered results with an adaptive exposure map. Red retangles indiate the details. where J (x) represents the restored hannel. 3.. Red Channel Corretion (9) where avgr is the normalized average value of the original red hannel. The reovered red hannel Rre an be obtained by: Rre = R. δ, (10) (8) YJ(x) {[1 s(x) ] + λ[s(x) 1] } + Φ(s), (11) Y I(x) x where s(x) is the adaptive exposure map, YJ is the illumination intensity of the restored image, YI is the illumination intensity of input image, λ = 0.3 is a onstant, and Φ( ) is a smoothness regularization. The optimization problem an be approximately solved using a two-step approah. First, solve s(x) without the smoothness regularization, whih has a losed-form solution. Then, apply guided filter GFI [19] to

4 smooth the solution. Therefore, we an get a fast approximate solution as: s(x) = GFI [ YJ(x) YI(x) + λyi(x) YJ(x) + λyi(x) ]. (1) The exposed output an be written as: OutputExp = J (x). s(x), {r, g, b}, (13) where J is the restored image and s(x) is the adaptive exposure map. Figure 4 shows the results of applying an adaptive exposure map. 4.. Quantitative result Unlike the ommon image quality assessment or ommon image restoration areas, there is no easy way to have a referene image, whih makes underwater images diffiult to evaluate. We onsider the main goal of image restoration as to emphasize the image features and information ontent. Table 1 shows the omparative values in terms of entropy, SIFT (Sale-Invariant Feature Transform) loal feature points [1] and the average value of gradient (AVG) for the underwater images shown in Fig. 5. The value of entropy represents the valuable information ontained in the reovered images. The SIFT loal feature points indiate the global ontrast and loal features while the AVG denotes the ontrast and details hanges. The best results are represented by bold fae values. 4. EXPERIMENT RESULTS In order to assess the performane of the proposed underwater image restoration method, we ompared our method with three existing methods: et al. [9], et al. [10] and et al. [11], whih are based on underwater imaging optial model and dehazing algorithm. The qualitative and quantitative evaluations are arried out to assess the performane of different methods. We just show some examples of the results owing to the limited spae. Table 1. Comparison in terms of entropy, SIFT and AVG. Images Image1 Image Image Qualitative result Figure 5 shows that s work has little or no effet on underwater images due to the distintion between the atmospheri sattering model and the atual underwater optial model. The method of an remove the haze. However, the solution of an unveil little details in the foreground. The results of usually ontain evident olor asts and artifats beause the assumption of the olor orretion is unavailable in some ases. The proposed method produes aesthetially natural image versions and improved ontrast and details without artifats. Method entropy SIFT AVG Table 1 shows that the quantitative performane of our method stands out among the other methods in terms of entropy, SIFT and AVG. The results demonstrate that our method an inrease the valuable information, global ontrast, loal features and details of the underwater images. Therefore, the qualitative and quantitative evaluations prove that our method outperforms the three ompared methods and an effetively improve the visual quality of underwater images. 5. CONCLUSION (d) An underwater image restoration method is proposed based on blue-green hannels dehazing and red hannel orretion. The qualitative and quantitative evaluations show that the proposed method an effetively remove haze, restore natural appearane and inrease ontrast, gradient and loal features of underwater images. Moreover, our method outperforms eah of the three existing methods. (e) Fig. 5. Qualitative omparisons. Original underwater images with a size : Image 1, Image and Image 3 from top to bottom. results. results. (d) results. (e) results. 1734

5 6. REFERENCES [1] R. Shettini and S. Corhs, Underwater image proessing: state of the art of restoration and image enhanement methods, EURASIP Journal on Advanes in Signal Proessing, vol. 010, pp. 14, 010. [] A. Yamashita, M. Fujii, and T. Kaneko, Color registration of underwater images for underwater sensing with onsideration of light attenuation, in Robotis and automation, 007 IEEE international onferene on. IEEE, 007, pp [3] M. Ludvigsen, B. Sortland, G. Johnsen, and H. Singh, Appliations of geo-referened underwater photo mosais in marine biology and arhaeology, Oeanography, vol. 0, pp , 007. [4] N. Strahan, Reognition of fish speies by olour and shape, Image and vision omputing, vol. 11, pp. 10, [5] L. A. Torres-Méndez and G Dudek, Color orretion of underwater images for aquati robot inspetion, in Energy Minimization Methods in Computer Vision and Pattern Reognition. Springer, 005, pp [6] E. Truo and A. T. Olmos-Antillon, Self-tuning underwater image restoration, Oeani Engineering, IEEE Journal of, vol. 31, pp , 006. [7] J. S. Jaffe, Computer modeling and the design of optimal underwater imaging systems, Oeani Engineering, IEEE Journal of, vol. 15, pp , [8] B. L MGlamery, A omputer model for underwater amera systems, in Oean Optis VI. International Soiety for Optis and Photonis, 1980, pp [9] N. -Biano, A. Mohan, and R. M. Eustie, Initial results in underwater single image dehazing, in OCEANS 010. IEEE, 010, pp [13] A. Galdran, D. Pardo, A. Pión, and A. Alvarez-Gila, Automati red-hannel underwater image restoration, Journal of Visual Communiation and Image Representation, vol. 6, pp , 015. [14] C. Anuti, C. O. Anuti, T. Haber, and P. Bekaert, Enhaning underwater images and videos by fusion, in Computer Vision and Pattern Reognition, 01 IEEE Conferene on. IEEE, 01, pp [15] A. S. A. Ghani and N. A. M. Isa, Underwater image quality enhanement through integrated olor model with rayleigh distribution, Applied Soft Computing, vol. 7, pp , 015. [16] C. Li and J. Guo, Underwater image enhanement by dehazing and olor orretion, Journal of Eletroni Imaging, vol. 4, pp , 015. [17] H. Wen, Y. Tian, T. Huang, and W. Gao, Single underwater image enhanement with a new optial model, in Ciruits and Systems, 013 IEEE International Symposium on. IEEE, 013, pp [18] S. G. Narasimhan and S. K. Nayar, Chromati framework for vision in bad weather, in Computer Vision and Pattern Reognition, 000. Proeedings. IEEE Conferene on. IEEE, 000, pp [19] K., J. Sun, and X. Tang, Guided image filtering, Pattern Analysis and Mahine Intelligene, IEEE Transations on, vol. 35, pp , 013. [0] K. Tang, J., and J. Wang, Investigating hazerelevant features in a learning framework for image dehazing, in Computer Vision and Pattern Reognition, 014 IEEE Conferene on. IEEE, 014, pp [1] D. G. Lowe, Distintive image features from saleinvariant keypoints, International journal of omputer vision, vol. 60, pp , 004. [10] H., P. Chen, C. Huang, Y. Zhuang, and Y. Shiau, Low omplexity underwater image enhanement based on dark hannel prior, in Innovations in Bio-inspired Computing and Appliations (IBICA), 011 Seond International Conferene on. IEEE, 011, pp [11] K., J. Sun, and X. Tang, Single image haze removal using dark hannel prior, Pattern Analysis and Mahine Intelligene, IEEE Transations on, vol. 33, pp , 011. [1] J. Y. Chiang and Y. C. Chen, Underwater image enhanement by wavelength ompensation and dehazing, Image Proessing, IEEE Transations on, vol. 1, pp ,