TM1 TM2 337 TM3 TM4 TM5 TM6 Dr. Robert A. Schowengerdt TM7 Landsat Thematic Mapper (TM) multispectral images of desert and agriculture near Yuma, Arizona MULTISPECTRAL IMAGE PROCESSING I SENSORS Multispectral relatively low spectral resolution (typically 5 1nm) and small number (typically 5-1) of discrete spectral bands typically acquired with multipath or multiple, filtered optical systems first satellite multispectral sensor was Landsat in 1972 Landsat image size 25MB Common color images, e.g. video or camera photos, are 3band examples of multispectral images ECE/OPTI533 Digital Image Processing class notes 23
Combine band images in color composite for interpretation For example, Color InfraRed (CIR) composite uses: red color assigned to near IR sensor band green color assigned to red sensor band blue color assigned to green sensor band vegetation appears red, soil appears yellow - grey, water appears blue - black ECE/OPTI533 Digital Image Processing class notes 338 Dr. Robert A. Schowengerdt 23
Hyperspectral relatively high spectral resolution (typically 5 1nm) and large number (typically 2) of nearly-contiguous bands AvIRIS hyperspectral image cube of Los Alamos, NM (courtesy Chris Borel, LANL) typically acquired with an imaging spectrometer over the wavelength range 4 to 24nm high spectral resolution potentially allows high discrimination of surface features primarily airborne - e.g. Airborne Visible- InfraRed Imaging Spectrometer (AVIRIS) operated by NASA/JPL, 1989 - date first satellite hyperspectral sensor is Hyperion on NASA EO-1, 2 - date Hyperion image size 1.1GB ECE/OPTI533 Digital Image Processing class notes 339 Dr. Robert A. Schowengerdt 23
ECE/OPTI533 Digital Image Processing class notes 34 Dr. Robert A. Schowengerdt 23 4 8 12 16 2 24 wavelength (nm) building soil 5 1 DN 15 building water 2 soil grass 25 Airborne Visible-InfraRed Imaging Spectrometer (AVIRIS) of Palo Alto, CA grass water Feature discrimination using hyperspectral imagery MULTISPECTRAL IMAGE PROCESSING I
ECE/OPTI533 Digital Image Processing class notes 341 Dr. Robert A. Schowengerdt 23.4.8 1.2 1.6 2 2.4 wavelength (µm) H 2 O H2 O.2 H 2 O CO 2 transmittance CO 2.4 CO 2, CO, 2 H O 2 CO 2 H 2 O.6.8 1 Spectral movie of AVIRIS image, Palo Alto, CA MULTISPECTRAL IMAGE PROCESSING I
DATA REPRESENTATION K-dimensional space formed by DNs in K spectral bands of a multi- or hyperspectral image K-dimensional column vector DN ij K [ ] = DN ij1 DN ij2 DN ijk T pixel at (i,j): = DN ij1 DN ij2 DN ijk ECE/OPTI533 Digital Image Processing class notes 342 Dr. Robert A. Schowengerdt 23
Example multsipectral data vector at pixel p for 3-bands DN 3 In three dimensions, each pixel plots as a vector DN p3 DN p1 DN p = DN p2 DN p3 DN p1 DNp2 DN 1 DN 2 Data space is quantized into (2 Q ) K volume elements, or bins With 3 bands and 8 bits/pixel, number of possible data vectors 256 3 = 16,777,216 With 2 bands and 11 bits/pixel, number of possible data vectors = 248 2 ECE/OPTI533 Digital Image Processing class notes 343 Dr. Robert A. Schowengerdt 23
K-dimensional histogram hist DN = count ( DN ) N PDF ( DN ) the value of hist DN at a particular DN vector is the number of pixels that have that vector scalar function of a vector ECE/OPTI533 Digital Image Processing class notes 344 Dr. Robert A. Schowengerdt 23
ECE/OPTI533 Digital Image Processing class notes 345 Dr. Robert A. Schowengerdt 23 DN2 DN2 Note, the features in scatter-space, corresponding to objects in image space DN2 DN2 2 2 5 5 1 1 4 4 1 1 6 6 2 2 8 8 pixels 15 15 pixels 1 1 pixels 3 3 pixels pixels 2 2 pixels 12 12 25 25 4 4 14 14 3 3 16 16 5 5 35 35 18 18 6 6 4 4 scattergram in 2-D Scattergram is greylevel visualization of K-D histogram Scattergrams and Scatterplots MULTISPECTRAL IMAGE PROCESSING I
MULTISPECTRAL IMAGE PROCESSING I Scatterplot is a binary plot of the K-D histogram Example for K = 3 viewed from different directions ECE/OPTI533 Digital Image Processing class notes 346 Dr. Robert A. Schowengerdt 23 7 6 5 4 3 2 1 7 6 5 4 3 2 1 DN2 2 4 6 8 1 14 2 4 12 6 8 16 1 12 7 6 18 5 4 3 2 1 7 6 5 4 3 2 1 14 DN2 16 18 2 2 4 4 6 6 8 8 1 1 12 12 14 7 6 5 4 7 6 5 4 3 2 1 14 DN2 16 16 18 18 3 2 1 2 4 2 6 4 8 6 1 8 12 1 14 12 14 16 16 18 18
ECE/OPTI533 Digital Image Processing class notes 347 Dr. Robert A. Schowengerdt 23 band 3 vs band 2 band 4 vs band 2 band 4 vs band 3 DN2 DN2 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 2 1 2 2 4 3 5 4 4 6 6 1 8 1 8 12 6 14 16 12 14 16 7 18 18 7 6 5 4 3 2 1 7 6 5 4 3 2 1 7 6 DN2 5 4 3 2 1 7 6 5 4 3 2 1 7 6 DN2 5 4 3 2 1 7 6 5 4 3 2 1 DN2 2 2 2 2 2 2 4 4 4 4 4 4 6 6 6 8 6 6 6 8 8 1 1 8 8 1 1 1 8 1 12 12 12 12 12 12 14 14 14 14 14 14 16 16 16 16 16 16 18 18 18 18 18 18 can also be obtained by thresholding the 2-D scattergram 2-D scatterplot obtained by projecting 3-D scatterplot MULTISPECTRAL IMAGE PROCESSING I
2 3 4 5 6 7 1 All possible 2-D scatterplots for a 7- band Landsat TM image 2 3 4 5 6 ECE/OPTI533 Digital Image Processing class notes 348 Dr. Robert A. Schowengerdt 23
Spectral Covariance and Correlation Multivariate DN mean vector N DN T µ = DN p N = p = 1 Multivariate DN covariance matrix C c 11 c 1K = = ( DN µ )( DN µ ) T c K1 c KK Elements are covariance between bands m and n: N c mn = DN pm µ m ( )( DN pn µ ) ( N 1 ) n p = 1 Diagonal elements are variance of band k c kk ECE/OPTI533 Digital Image Processing class notes 349 Dr. Robert A. Schowengerdt 23
Multivariate DN correlation matrix scatterplot shape and correlation R = 1 ρ 1K ρ K1 1, 1 ρ mn 1 or ρ mn 1 high correlation DN m ρ 1 ρ 1 mn mn Correlation between bands m and n: DN n ρ mn = c mn ( c mm c nn ) 1 2 < ρ mn 1 > ρ mn 1 Normalized by variance in each band moderate correlation Special properties of covariance and correlation matrices C and R are symmetric, i.e., ρ, c = c mn mm nn ρ, c < c mn mm nn c mn c nm = and ρ mn = ρ nm no correlation If C and R are diagonal, the pixel values in bands m and n are uncorrelated ECE/OPTI533 Digital Image Processing class notes 35 Dr. Robert A. Schowengerdt 23
Principal Component Transform AKA Karhunen-Loeve or Hotelling transform linear matrix transform PC = W PC DN where: PC is the K dimensional principal component vector W PC is a K x K transformation matrix DN is the original multi- or hyperspectral pixel vector each principal component is a weighted average of all spectral bands ECE/OPTI533 Digital Image Processing class notes 351 Dr. Robert A. Schowengerdt 23
Properties of the PCT W PC diagonalizes the covariance matrix, C, of the original image, T CPC = W PC CW PC since CPC is diagonal, the PC components are uncorrelated The diagonal elements of CPC are the eigenvalues of the data CPC = eigenvalue 1 eigenvalue K = λ 1 λ K λ k C λ k I = each eigenvalue,, is equal to the variance of the k th PC and is found by solving the characteristic equation, Note, trace of C pc equals trace of C, i.e. the sum of the eigenvalues equals the sum of the original image band variances ECE/OPTI533 Digital Image Processing class notes 352 Dr. Robert A. Schowengerdt 23
W PC consists of the eigenvectors of the data along its rows, WPC t t eigenvector 1 e 1 = : = : = eigenvector K t e K t e 11 e 1K : : e K1 e KK each eigenvector, e k, consists of the weights applied to the original bands to obtain the k th PC and is found by solving the equation, ( C λ k I )e k = ECE/OPTI533 Digital Image Processing class notes 353 Dr. Robert A. Schowengerdt 23
ECE/OPTI533 Digital Image Processing class notes 354 Dr. Robert A. Schowengerdt 23 6 5 5 5 5 4 4 4 3 3 3 4 2 2 3 which has solutions λ = 2.67 and λ =.33 1 2 2 λ 2 3λ +.88 = pixel DN1 DN2 1.9 λ 1.1 1.1 1.1 λ = 1 2 3 4 5 DN1 C λ k I = 1 Characteristic Equation 2 DN2 3 R = 1.761.761 1 4 5 C = 1.9 1.1 1.1 1.1 Data Example µ = 3.5 3.5 MULTISPECTRAL IMAGE PROCESSING I
Therefore CPC = 2.67.33 Note that PC1 accounts for 2.67/3 =.89 of total variance in data Find eigenvectors using ( C λ k I )e k = For example, for eigenvector e 1, which are not independent. From either equation e 11 = 1.43e 12.77e 11 + 1.1e 12 = 1.1e 11 1.57e 12 = 2 2 Eigenvectors are orthogonal unit vectors implies e 11 + e 12 = 1. Solving simultaneously with above equation gives eigenvector e 1.82.57.82 = and, with a similar analysis, eigenvector e.57 2 = ECE/OPTI533 Digital Image Processing class notes 355 Dr. Robert A. Schowengerdt 23
Final transformation matrix W.82.57 PC =.57.82 Find and plot new coordinates of data points in PC space ECE/OPTI533 Digital Image Processing class notes 356 Dr. Robert A. Schowengerdt 23
ECE/OPTI533 Digital Image Processing class notes 357 Dr. Robert A. Schowengerdt 23 band or PC index 1 2 3 4 5 6 2 DN variance 4 6 TM bands PC bands 8 Compresses the variance λ 1 > λ 2 > λ K DN 1 PC 1 PC 2 DN 2 Decorrelates the spectral data optimally MULTISPECTRAL IMAGE PROCESSING I Why use the PCT?
Why not use the PCT? data-dependent W coefficients change from scene-to-scene Makes consistent interpretation of PC images difficult spectral details, particularly in small areas, may be lost if higher-order PCs are ignored computationally expensive for large images or for many spectral bands ECE/OPTI533 Digital Image Processing class notes 358 Dr. Robert A. Schowengerdt 23