1 Image Fusion Sensor Merging Magsud Mehdiyev Geoinfomatics Center, AIT Image Fusion is a combination of two or more different images to form a new image by using certain algorithms. ( Pohl et al 1998) Pan sharpening is a type of data fusion Low resolution color data is fused with high resolution monochrome data Result is a high resolution color image 2 : ENVI PAN SHARPEN True Color Composite, Spatial Resolution 2.4 meters Gram-Schmidt Pansharpened Image, Spatial Resolution 0.67 meters 3 Panchromatic Image, Spatial Resolution 0.67 meters 4 QuickBird Sensor (similar to Ikonos) Multi-spectral and PAN It takes roughly 4 seconds to collect a 16.5 km x 16.5 km image The QB sensor collects: Panchromatic (Black & White) Multi-spectral Blue Green Red Near-infrared Linear Array Sensor 5 6 Multi-spectral includes Visible (Blue, Green and Red) and Near-infrared
7 QuickBird Spectral Bands Engineering Challenges For Quickbird and Ikonos, pan and MS bands are collected almost simultaneously Depending on the scanning mode of the instrument there is a time delay between the collects (~0.2 seconds) 0.2 sec * 7.5 Km/sec = 1.5 km : the satellite moves ~1.5 Km on orbit between the collects Blue -Band 1 450-520 nm Green -Band 2 520-600 nm Red -Band 3 630-690 nm NIR -Band 4 760-900 nm 8 Engineering Challenges Parallax Random spacecraft motion between collects can cause misregistration between the pan and MS bands Misregistration results in a blurry pan sharpened image In addition parallax between the bands can cause misregistration, especially when there are errors in the elevation model 9 10 Automatic Band Correlation (ABC) Misregistration between the panchromatic and MS bands due to sensor motion between the collects causes image artifacts (bluriness) In addition, parallax between the bands causes misregistration (different look angles) The solution is to perform Automatic Band Correlation (ABC) before the fusion ABC is currently under development at DigitalGlobe and, when ready, will be applied to all Pan Sharp products Process downward :ENVI 1. Choose two images (Multi-spectral and PAN) 2. Geometric correction tools can be used to quickly ortho-rectify a single image using RPC file The resulting ortho-image is accurate to realworld coordinates 3. Pan sharp algorithms 11 12
13 Why Ortho-rectification? Geo-corrected aerial photography and satellite imagery have large geometric distortion that is caused by various systematic and nonsystematic factors. Photogrammetric techniques used in orthorectification eliminate these error most efficiently, and create the most reliable and accurate imagery from the raw imagery. What is Ortho-rectification? Ortho-rectification is the process of reducing geometry errors inherent with photography and imagery. The variable contributing to geometric errors include: camera and sensor orientation systematic error associated with the camera or sensor topographic relief displacement earth curvature 14 Ortho-rectification Ortho-rectification To rectify aerial photographs and SPOT, QuickBird, or IKONOS data using a digital elevation model (DEM). The ortho-rectification uses geometric projections to produce geometrically correct images for mapping and measurement. Once an ortho-rectified image is created, each pixel within the image possess geometric fidelity. Thus, measurement taken off an ortho-rectified image represent the corresponding measurements as if they were taken on the earth s surface. 15 16 Digital Ortho-image Ortho-rectification: RPC P = ground point P 1 = image point O = perspective center (origin) X,Z = ground control point (in DTM file) f = focal length An image or photograph with an orthographic projection is one for which every point looks as if an observer were looking straight down at it, along a line of sight that is orthogonal (perpendicular) to the earth. The resulting ortho-rectified image is known as a digital ortho-image. 17 The RPC (Rational Polynomial Coefficients or Rapid Positioning Coordinates) sensor model is used to ortho-rectify data from either the IKONOS or QuickBird sensors. The RPC ortho-rectification process combines several sets of input data to place each pixel in the correct ground location. 18
19 Ortho-rectification: RPC Ortho-rectification Parameters Window: ENVI The following inputs are required: the image to be rectified, the RPC coefficients, and elevation information. Furthermore, the offset between mean sea level and the gravitational potential surface known as the geoid is required so the elevation can be correctly interpreted. Finally, if the source image does not have approximate geo-location information available, the rough location of the image on the earth's surface must be computed to provide a location base needed for the RPC transformation. 20 RPC file Rational Polynomial Coefficients To determining interior and exterior orientation Information of the geographic coordinates associated with the coordinates of the imagery Information of the projection Algorithms: ENVI There are many pan sharpening algorithms available today in commercial packages Such as: HSV Sharpening Color Normalized (Brovey) Sharpening Gram- Schmidt Spectral Sharpening 21 22 HSV(or HSI) Sharpening HSV(or HSI) Sharpening HSI(HSV) stands for Hue Saturation Intensity (Hue Saturation Value) The low resolution RGB image is upsampled and converted to HSI space The panchromatic band is then matched and substituted for the Intensity band The HIS image is converted back to RGB space 23 24
25 HSV(or HSI) Sharpening Result HSV(or HSI) Sharpening Result Spatial information and sharpness is good, color wise not good Image is sharp, however, vegetation colors are off. Why does this occur? 26 Explanation QB Spectral Response Vegetation when seen in visible light (RGB) is dark (lower reflectance) Vegetation when seen in the near IR is very bright (higher reflectance) Since the QuickBird pan band contains a lot of NIR, the vegetation pixels come out MUCH too bright and small color noise values are amplified 27 28 Explanation HSV Improvements (Vis Pan) Since the spectral overlap between the panchromatic band and the multi-spectralbands is known, we can subtract out the estimated NIR contribution from the pan band This is called the visible pan band, and more closely matches the visible intensity 29 30
31 HSV Result (Vis Pan) Vis Pan HSV Result Color recovery better but still not perfect 32 HSV Sharpening Conclusions HIS sharpening produces very sharp imagery Color recovery, especially over vegetation, is poor and won t satisfy most customers Improved color recovery is realized by subtracting out the known NIR contribution to the Pan band (VisPan), however, color recovery still not perfect Use to sharpen spectral image data with high spatial resolution data. A principal components transformation is performed on the multi-spectral data. The PC band 1 is replaced with the high resolution band, which is scaled to match the PC band 1 so no distortion of the spectral information occurs. Then, an inverse transform is performed. The multi-spectral data is automatically resampled to the high resolution pixel size using a nearest neighbor, bilinear, or cubic convolution technique 33 34 Result 35 36
37 Result:ENVI The PCA result is much better than the HIS result Significant misregistrations apparent around bright objects Color wise the recovery is acceptable on this image PCA algorithm is dependant on scene content high vegetation content causes poor performance High NIR contribution tends to distort the PCA transformation and cause blurriness Sharpness is acceptable, color is good, spatial information distorted 38 Color Normalized (Brovey) Sharpening Color Normalized Sharpening Result: ENVI Use Color Normalized (Brovey) sharpening to apply a sharpening technique that uses a mathematical combination of the color image and high resolution data. Each band in the color image is multiplied by a ratio of the high resolution data divided by the sum of the color bands. The function automatically resample the three color bands to the highresolution pixel size using a nearest neighbor, bilinear, or cubic convolution technique. The output RGB images will have the pixel size of the input high-resolution data. 39 Spatial information and sharpness is good, color wise not good 40 Gram Schmidt Algorithm Gram Schmidt Sharpening Result: ENVI This is a Kodak / RSI proprietary sharpening algorithm The algorithm is based on a rotation similar in nature to PCA The results are quite similar to PCA robustness is an issue especially in heavily vegetated scenes Questions and Comments are Welcome 41 Spatial information and sharpness is good, color wise is also good 42