Superresolution Method Approach for Vietnam Remote Sensing Imagery

Transcription

1 doi: /ijrsa Superresolution Method Approach for Vietnam Remote Sensing Imagery Le Quoc Hung* 1, Dang Truong Giang 2, Nguyen Ngoc Quang 3 1,2,3 Department of National Remote Sensing, Vietnam Ministry of Natural Resources and Environment (MONRE) Nguyen Chi Thanh Street, Lang Ha Ward, Dong Da District, Hanoi, Vietnam *1 quochungrs@gmail.com; 2 longriver8x@gmail.com; 3 quangavril@yahoo.com Abstract Spatial resolution enhancement of satellite imagery is one of the most important aspects in the field of remote sensing science. Resolution enhancement by up-grading satellite imaging or developing advanced optical instrument it is very costly to obtain the high resolution. On the other hand, the increase in spatial resolution has to be balanced with the state capacity in transmission rates, archiving and processing capabilities. Thus, the other parameters of satellite system must be reduced such as swath width, spectral and radiometric resolution, observation and data transmission duration. These reasons promote researchers in order to propose the approach of using multiple images for enhancing spatial resolution from low to high. VNREDSAT-1 is the first Vietnamese remote sensing satellite which was launched and has operated since This paper will present some very first result of enhancing its spatial resolution based on the super-resolution method. With this method, the 1.25m resolution was created from the 2.5m original resolution of VNREDSAT-1 image. The result shows how the improved resolution can help to explore more information of objects on the earth for serving the mission of natural resources and environment monitoring. Keywords Super-resolution (SR); Low Resolution (LR); VNREDSat-1; NinhThuan; Vietnam Introduction VNREDSAT-1 is the first earth observation satellite of Vietnam. Currently, Vietnam supplies actively high resolution satellite imagery on demand to ministries and local governments who wish to apply them for socialeconomic development, response to natural disasters and climate change. In order to effectively apply Vietnamese satellite imagery for natural resources and environment monitoring as well as territory protection, research on improving spatial resolution is necessary to maximize the exploration of remotely sensed data and to the raise impact of this satellite which is expected to have 5-years lifetime [1], [2]. Resolution enhancement by using multiple images with the same area of interest (AOI), and same resolution in order to process a higher resolution image, is called Super-resolution method. This method has been researched and applied widely for popular satellite images such as SPOT-5, Landsat, Quickbird, Egyptsat-1, etc [3], [7], [10]. Hence, it is realizable to apply this method for Vietnamese satellite image. However, it requires consideration of elements come from mission planning, receiving and processing, image matching, image reconstruction, etc. Nowadays, the advantage of using LR data is that SR results can be easily assessed numerically and quality; that is, most current SR algorithms are tested for LR data which will be easy to approach [3], [4], [5], [6]. One of the most important techniques of SR is the super-resolution reconstruction (SRR) which obtains HR image from the set of LR images by increasing the high frequency of components from the non-repetitive and redundant information of LR images. That information can be extracted by utilizing the sub-pixel spatial disparity between LR images. This spatial disparity can be determined by objects in images or following the movement under control, for instance, the satellite s instrument has been defined by speed and direction on its orbit [2]. Since Tsai and Huang [8] found SSR solution in 1984, many methods have been proposed with some main approaches such as frequency domain, spatial domain [4], [9]. Currently, spatial domain approaches are more complex than others but they are used widely to reconstruct HR image because of many advantages in [9]. SSR for VNREDSAT-1 images was based on 118

2 International Journal of Remote Sensing Applications (IJRSA) Volume 6, the Variational Bayesian reconstruction method which is one of the powerful methods in spatial domain approach. Materials and Methods. Observation Sites The study area is typical for some types of basic terrain in Vietnam, in NinhThuan province, South Central Coast of Vietnam, located at '14" to '15"N latitude and '08" to '25"E longitude. NinhThuan is the last region of Annamite Range (Day Truong Son) with many mountains crashed into the East Sea, the terrain descending from northwest to southeast, surrounded by mountains on 3 sides with 3 types of terrain including mountains, semi-mountain hills and coastal plains. In particular, hilly area accounted for 63.2% of the total area of the province, mostly low mountains, average height of meters. Semi-mountain hills is 14.4% and the coastal plains is 22.4% of natural land area. Data Set. FIG. 1 STUDY AREA NINH THUAN, VIETNAM (SOURE: DEPARTMENT OF SURVEY AND MAPPING VIETNAM) In this study, 04 VNREDSAT-1 images were acquired on March 21, 2014; July 4, 2014; April 12, 2015 and August 6, 2015 (Fig.2a, 2b, 2c, 2d) with the difference of orbit and satellite incidence angle varying from to degree. VNREDSAT-1 was launched and operated from May 7, 2013 with ground resolution are 2.5m (Panchromatic) and 10m (multi-spectral); sun synchronous orbit, 680km altitude, 17.5x17.5 km wide swath, spectrum resolution is 10 bits [1], [2]. Method. In general, super resolution consists of two parts: registration and image estimation. Registration estimates the motion between the LR images. Image estimation is the process where HR image is constructed from the LR images using information about the motion and blurring. The most popular model which describes the relationship between LR images and the HR image was introduced by Elad and Feuer [6]. In this model, each LR image Yk is the measured image that is the result of a geometric warping, blurring and downsampling performed on the ideal HR image X. Besides, assuming that error of their measurements is nonhomogeneous additive Gaussian noise, uncorrelated between different measurements. Considering K LR images will be written in lexicographical notation as the vector Yk = [yk1, yk2, yk3,.,ym], where k is k th LR image in the set of n LR images; 119

3 M = N1x N2 with N1 and N2 corresponding to pixel sizes in two dimensions of LR images. We desire a highresolution image of pixel size N=L1.N1 x L2.N2 in the same scene with LR images, where L1 and L2 are downsampling factors in two dimensions of images, respectively. HR image will also be written in lexicographical notation as the vector X = [x1, x2, x3,.,xn].all pixel values of HR image are contained within X. The model is written in lexicographical following below: Yk = DkHkFkX + Vk, k =1, 2, 3 K (1) FIG. 2A IMAGE ACQUIRED ON MARCH 21, 2014 SATELLITE INCIDENCE ANGLE: 21.36O FIG. 2B IMAGE ACQUIRED ON JULY 4, 2014 SATELLITE INCIDENCE ANGLE: 33.86O FIG. 2C IMAGE ACQUIRED ON APRIL 12, 2015 SATELLITE INCIDENCE ANGLE: O FIG. 2D IMAGE ACQUIRED ONAUGUST 6, 2015 SATELLITE INCIDENCE ANGLE: 24.95O In this formula (1), Ykis the matrix k th LR image with Mx1 elements. X is an HR image matrix Nx1 elements, N= P.M with P = N1 x N2, P is called magnification factor. In spatial enhancement, N1 is equal to N2 and P= N1 2 = N2 2 ). Fk is the function describing the motion of the k th image: Fk = sk(θk, ck, dk) T with θk, ck, dk respectively are parameters of image shift: rotation, offset horizontally and vertically. The size of matrix Fk is PNxPN. Hk is the blurring matrix PNxPN elements. Dk is the matrix NxPN and the down-sampling operator (the operator to reduce the resolution). Vk is the noise matrix with size NxN. The effects of downsampling, blurring, and warping can be combined into a single system matrix B(sk) and then (1) can be written as: 120

4 International Journal of Remote Sensing Applications (IJRSA) Volume 6, Yk = B(sk)X + Vk, k =1, 2, 3 K (2) From (2), the HR image can be estimated from the set of LR images Yk using prior knowledge about the set of B(sk), Vk and X. Thus, hierarchical Bayesian model has two stages. The first stage is used to model the acquisition process, the unknown HR image X and the set of motion vectors {sk}. The unknowns X and sk are assigned prior distributions Pr(X αim) and Pr(sk), respectively. The observation Y={Yk} is also a random process with the corresponding conditional distribution Pr(Y X, sk, βk). The observation model [11] is written as below: ( ) { ( ) } (3) Using Total variation (TV) prior, the image model is given by: ( ) [ ( )](4) In (4), c is a constant and ( ) ( ( )) ( ( )) (5), where the operators ( ) ( ) and ( ) ( ) is repectively horizontal and vertical first order differences. These distributions depend on additional parameters αim and βk(called hyperparameters), which are modeled by assigning hyperprior distributions in the second stage of the hierarchical model. Using flat hyperpriors on αim and βk, Pr(αim) const, Pr({βk}) (2.32) const. Using Gamma distribution, hyperprior distributions were written as: Where, ω>0 denotes a hyperparameter, > 0, >0 are the shape and scale parameters, respectively. Finally, combining two stages, the joint probability distribution of all variables is written as: ( ) ( ) ( ) ( ) [ ](6) Using Bayesian inference, variational approximation for the formula (6), the distributions of HR image, registration parameter and the hyperparameter were estimated [11], [12]. Based on Above Algorithm, Vietnamese Scientists Built the SR Software for VNREDSat-1 with Its Characteristics by Mathlab. Following the Variational Bayesian SR, the method SR for VNREDSAT-1 includes two stages: geometric correction and Variational Bayesian reconstruction. In the first stage, VNREDSAT-1 images were geometrically corrected to remove the effects of topography and the difference of the satellite incidence angle. The result is the set of LR images which cover the same area of interest. This stage is run on image processing system of Vietnam Ground Station. In the second stage, HR image was reconstructed by SR software using Variational Bayesian method. Figure 3 shows the steps to obtain HR image from LR images that were applied for VNREDSAT-1. For initial values, with 4 input images, P is given 4 with N1= N2 =2. That means the spatial resolution of HR image is increased twice in comparison with the original VNREDSAT-1 imagery. Besides, using flat hyperpriors on αim and βk, hyperparameters are put the first values [12]. Results and Discussion. Image quality evaluation using super-resolution software: Histogram Image histogram after running super-resolution (SR) software contains much more information than the histogram before running the software with the total number of pixels is nearly 22,000,000,comparingto 5,500,000 pixels of the original image. Image histogram after running SR software has pixel gray-scale value range from 40 to 160 which is larger than the original image s pixel range from 40 to 90), so it can provide more information. 121

5 FIG. 3 SUPPER-RESOLUTION APPROACH FOR VNREDSAT-1 FIG. 3a ORIGINAL IMAGE FIG. 3b SR IMAGE FIG. 4a ORIGINAL IMAGE S HISTOGRAM FIG. 4b SR IMAGE S HISTOGRAM 122

6 International Journal of Remote Sensing Applications (IJRSA) Volume 6, Image Resolution. FIG. 5a ORIGINAL IMAGE FIG. 5b ORIGINAL IMAGE TO 02 TIMES FIG. 5c SR IMAGE The image after processing by SR software has the resolution increased to 02 times. For example, the original image has the resolution of 2.5m. The processed image has the resolution of 1.25m. The original image after zooming in twice times appeared aliasing and blurring and the quality is worse than the image processed by SR software. Image Detail Level. Example 1: FIG. 6a ORIGINAL IMAGE FIG. 6b ORIGINAL IMAGE TO 02 TIMES FIG. 6c SR IMAGE Image processed by SR software can show more detail of objects than the original image. The picture above shows the detail of the fields with rows from east to west 45 o incline; although the original image is enlarged to 02 times, it can not display as good as the processed image because of appearing aliasing and blurring. Example 2: FIG. 7a ORIGINAL IMAGE FIG. 7b ORIGINAL IMAGE TO 02 TIMES FIG. 7c SR IMAGE 123

7 The image processed by SR software can show more detail of linear objects and field plots much better than the original image. The picture on the left shows the detail of linear objects and plots clearer than the picture on the right. When enlarging the original image to 2 times to compare to the processed image, those objects appear aliasing and blurring. Image Quality. Example 1 FIG. 8a ORIGINAL IMAGE FIG. 8b ORIGINAL IMAGE TO 02 TIMES FIG. 8c SR IMAGE One of the advantages of this processing method is removing the impact of weather or physical phenomenon such as clouds, internal reflection, glare caused by the sun, The picture of image before processing shows the lake dazzled by the sun, so the color of the lake is bright white. This issue has been eliminated after running SR software. Example 2 FIG. 9a ORIGINAL IMAGE FIG. 9b SR IMAGE SR software could increase to 02 times of spatial resolution without affecting the spectral quality of the image. The algorithm of this SR software is proved to be good to use. ACKNOWLEDGMENT We would like to thank Dr. Nguyen Xuan Lam, Director General of Vietnam National Remote Sensing Department for his suggestion, especially many thanks to research group of the project Research on using the quality enhancement method of remote sensing imagery VNREDSat-1, VT/UD-01/13-15, the Vietnam National Program of Space Science and Technology (code: KHCN-VT/12-15). 124

8 International Journal of Remote Sensing Applications (IJRSA) Volume 6, REFERENCES [1] L.Q. Hung, T.T.Tuyet Some methods for enhancing resolution of Vietnam remote sensing imagery by superresolution method. Scientific Journal of Geodesy and Cartography of Vietnam. vol. 20, [2] L.Q. Hung et al. Some initial results enhancing resolution of remote sensing image VNREDSat Scientific collection of Space Technology. Publisher of Natural Resources and Technology of Vietnam [3] T. Akgun, Y. Altunbasak, and R. M. Mersereau Super-resolution reconstruction of hyperspectral images. IEEE Trans. Image Process. vol. 14 (11), [4] S. C. Park, M. K. Park, and M. G. Kang Super-resolution image reconstruction: A technical overview. IEEE Signal Process. Mag. vol. 20 (3), 21 36, [5] A. Galbraith, J. Theiler, K. Thome, and R. Ziolkowski, Resolution enhancement of multi-look imagery for the multispectral thermal imager. IEEE Trans. Geosci. Remote Sens. vol. 43, [6] M. Elad and A. Feuer Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images IEEE Trans. Image Process, vol. 6 (12), [7] J. Ma, J. C.-W. Chan, and F. Canters An operational superresolution approach for multi-temporal and multi-angle remotely sensed imagery. IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 5 (1), [8] R. Y. Tsai and T. S. Huang Multipleframe image restoration and registration.in Advances in Computer Vision and Image Processing. Greenwich, CT: JAI Press Inc., [9] Sean Borman, Robert L. Stevenson Super-Resolution from Image Sequences - A Review. In Proceedings of the 1998 Midwest Symposium on Circuits and Systems [10] J. C.-W. Chan, J. Ma, P. Kempeneers, and F. Canters Superresolution enhancement of hyperspectral CHRIS/Proba images with a thin-plate spline nonrigid transform model. IEEE Trans. Geosci. Remote Sens. vol. 48, [11] S. DerinBabacan; Rafael Molina ;Aggelos K. Katsaggelos Total Variation Image Restoration and Parameter Estimation using Variational Posterior Distribution Approximation IEEE International Conference on Image Processing. vol. 1, [12] S. DerinBabacan, Rafael Molina, Aggelos K. Katsaggelos Variational Bayesian Super Resolution. IEEE Transactions on image proceesing, vol. 20 (4). [13] S. Borman and R. Stevenson Spatial resolution enhancement of lowresolution image sequences - a comprehensive review with directions for future research. Technical report, University of Notre Dame. [14] S. Farsiu, M. Elad, and P. Milanfar Multiframedemosaicing and superresolution of color images. IEEE Transactions on Image Processing. vol. 15(1), [15] K. Aizawa, T. Komatsu, and T. Saito A Scheme for Acquiring Very High Resolution Images Using Multiple Cameras. IEEE Proc. ICASSP-92, San Francisco, CA. vol. 3, [16] K. Aizawa, T. Komatsu, T. Saito, and M. Hatori Subpixel Registration for a High Resolution Imaging System Using Multiple Imagers. IEEE Proc. ICASSP-93, Minneapolis, MN. vol. 5, [17] S. Villenaet. al Bayesian Super-Resolution Image Reconstruction using an 1 prior. Proceedings of the 6th International Symposium on Image and Signal Processing and Analysis [18] M. Tipping and C. Bishop. Bayesian image super-resolution, Advances in Neural Information Processing Systems. vol. 15, Le Quoc Hung was born in Vietnam, in He received the B.Eng. degree in 1995 from Hanoi University of Mining and Geology, Vietnam, where he also received the M.Eng. degree in Cartography and Remote sensing in He got the Ph.D. degree in the Department of Environmental Science and Technology in 2007 in Saitama University, Japan. In 1995, he was a researcher in Vietnam Academy of Science and Technology. He studied using remote sensing, GIS, biotechnology in monitoring coastal area, wetland area. He has published in Taylor & Francis, Chemistry and Ecology (Volume 125

9 24, Issue 5, 2008, pp ), in Springer-Verlag, Wetlands Ecology and Management (Volume 15, Number 2, ). In 2010, he joined the Department of National Remote Sensing, Vietnam Ministry of Natural Resources and Environment. Currently, he is the Head of Division of Remote Sensing Technology. His primary research interests including satellite imagery processing and applications of remote sensing in monitoring environment and resources. Dang Truong Giang was born in Hanoi, Vietnam, in He obtained B.Eng. degree in 2007 from Hanoi University of Mining and Geology, Vietnam. He received M.Eng. degree in the Department of Environmental Science and Technology in 2007 in Saitama University, Japan in In 2007, he joined National Remote Sensing Station, Department of National Remote Sensing, Vietnam Ministry of Natural Resources and Environment where he was accumulated lot of experience in satellite imagery processing and analysis. His areas of research interest are image processing and analysis, applying remote sensing, GIS to ecological modelling, monitoring urban, coastal environment. Nguyen Ngoc Quang was born in Hanoi, Vietnam, in He got B.Eng. degree in 2004 from Hanoi University of Mining and Geology, Vietnam. He received M.Sc. degree in Department of Geography, Vietnam National University, Hanoi, Vietnam in He was a research assistant in Remote Sensing and GIS Department, Institute of Geography (IG), Vietnamese Academy of Science and Technology (VAST) in Since 2006 to now, he responds for Satellite Images Receiving at the Vietnam Ground Station, Department of National Remote Sensing, Ministry of Natural Resources Environment (MONRE). He has lot of experiences in image processing; image processing system and using remote sensing to monitoring flood and forest fire. He concerns to image processing system, satellite system and applications of satellite imagery in the environment. 126