Generaized constrained energy minimization approach to subpixe target detection for mutispectra imagery Chein-I Chang, MEMBER SPIE University of Maryand Batimore County Department of Computer Science and Eectrica Engineering Batimore, Maryand 21250 E-mai: cchang@umbc.edu Jih-Ming Liu Bin-Chang Chieu ationa Taiwan Science and Technoogy University Department of Eectrica Engineering Taipei, Taiwan Hsuan Ren University of Maryand Batimore County Department of Computer Science and Eectrica Engineering Batimore, Maryand 21250 Chuin-Mu Wang Chien-Shun Lo Pau-Choo Chung ationa Cheng-Kung University Department of Eectrica Engineering Tainan, Taiwan Ching-Wen Yang Taichung Veterans Genera Hospita Computer Center Taichung, Taiwan Abstract. Subpixe detection in mutispectra imagery presents a chaenging probem due to reativey ow spatia and spectra resoution. We present a generaized constrained energy minimization (GCEM) approach to detecting targets in mutispectra imagery at subpixe eve. GCEM is a hybrid technique that combines a constrained energy minimization (CEM) method deveoped for hyperspectra image cassification with a dimensionaity expansion (DE) approach resuting from a generaized orthogona subspace projection (GOSP) deveoped for mutispectra image cassification. DE enabes us to generate additiona bands from origina mutispectra images nonineary so that CEM can be used for subpixe detection to extract targets embedded in mutispectra images. CEM has been successfuy appied to hyperspectra target detection and image cassification. Its appicabiity to mutispectra imagery is yet to be investigated. A potentia imitation of CEM on mutispectra imagery is the effectiveness of interference eimination due to the ack of sufficient dimensionaity. DE is introduced to mitigate this probem by expanding the origina data dimensionaity. Experiments show that the proposed GCEM detects targets more effectivey than GOSP and CEM without dimensionaity expansion. 2000 Society of Photo-Optica Instrumentation Engineers. [S0091-3286(00)01205-8] Subject terms: cassification; constrained energy minimization; dimensionaity expansion; generaized constrained energy minimization; generaized orthogona subspace projection; hyperspectra image; mutispectra image; subpixe detection. Paper ATR-013 received Sep. 3, 1999; revised manuscript received ov. 23, 1999; accepted for pubication Dec. 9, 1999. Dye-Jyun Ma ationa Chung-Hsing University Taichung, Taiwan 1 Introduction Mutispectra images differ from hyperspectra images in the sense that the former is acquired by tens of spectra bands channes compared to the atter by hundreds of spectra bands. Such ow spectra resoution resuting from a sma number of spectra bands presents a chaenging probem for subpixe detection and cassification in mutispectra imagery. Intuitivey, if there are m materias or endmembers, it requires at east n spectra bands with n m to produce satisfactory cassification resuts. This phenomenon was demonstrated in Ref. 1 and is referred to the band number constraint BC. More precisey, it requires at east more than m spectra bands to cassify m endmembers so that each endmember can be diagnosed by a separate spectra band. This fact is simiar to the we-known pigeon-hoe principe in discrete mathematics. 2 To resove this issue, Ren and Chang 3 recenty proposed a generaized orthogona subspace projection GOSP approach, which deveoped a band generation process to produce additiona images so that the origina mutispectra imagery can be expanded. These newy generated images are produced by making use of various noninear correations among a given set of origina mutispectra images. Combining these extra generated images with the origina images resuts in sufficient dimensions that can be used to accommodate more materia substances that must be cassified. However, there is aso a trade-off due to such image data expansion. Some unwanted signatures may be aso generated and mix with the materia signatures of interest. These undesired signatures are usuay not known a priori. Therefore, many existing mixed pixe cassification methods such as orthogona subspace projection OSP -based cassifier 3,4 and maximum ikeihood cassifier, 5,6 may not be appropriate because they require a compete knowedge of materia signatures present in images. Opt. Eng. 39(5) 1275 1281 (May 2000) 0091-3286/2000/$15.00 2000 Society of Photo-Optica Instrumentation Engineers 1275
Chang et a.: Generaized constrained energy minimization approach... To aeviate the requirement of prior knowedge about materia signatures, a recent approach, caed constrained energy minimization CEM was proposed. 7 9 The idea of CEM arises in Frost s ineary constrained adaptive beamforming approach deveoped for array processing. 10 It first seects a materia signature as a target signature to be detected and cassified. Since the target signature is the ony signature we are interested in, we coud design an adaptive fiter to pass the desired target with a specific gain whie the fiter output resuting from unknown signa sources can be minimized. To accompish this task, CEM interpreted the target signature of interest as the signa arrived from a desired direction in the context of a ineary constrained minimum variance LCMV beamforming probem 10 12 so that finding a CEM fiter is equivaent to seeking an adaptive beamformer, which ocks on the desired direction of signa arriva with a specific constraint. The weights chosen for the desired adaptive beamformer minimizes its output variance or energy subject to this specific response constraint. As a consequence, the effects of signas from directions other than the desired one is minimized. When the specific gain is chosen to be unity, the LCMV beamformer becomes the minimum variance distortioness response MVDR beamformer, which is the precise mode on which CEM is based. Using the same approach carried out by the MVDR beamformer, a CEM-based detector was designed by a finite impuse response FIR fiter in a simiar fashion so that the desired target was passed through the fiter whie energies caused by the unknown signa sources were minimized. 7 For the purpose of simpicity, the term of CEM is referred throughout this paper to either a CEMbased detector or the CEM approach, depending on the context. To use CEM for target detection, the data dimensionaity must be sufficienty arge. For a mutispectra image its data dimensionaity is generay too sma to make CEM effective. In this paper, we combine CEM with a dimensionaity expansion DE approach that was deveoped 3 in GOSP to derive a generaized CEM GCEM that can extend the target capabiity of CEM to mutispectra images. The process of GCEM can be briefy described as foows. It first uses DE to produce a new set of nonineary correated images from the origina mutispectra images to expand data dimensionaity. It is important to note that images generated by inear correation do not provide any new information for CEM since CEM is a inear FIR fiter. The concept of creating nonineary correated images can be traced back to mutivariate anaysis where a data correation matrix is generay used to capture the second-order statistics of the data. Recenty, this idea was aso appied to create new sampes for target detection and cassification in hyperspectra images, 13 where ony very few training sampes were avaiabe for each target of interest and the data dimensionaity was reativey arge compared to the number of sampes that coud be used for training. In this case, the data sampe correation matrix was generay not of fu rank. To resove this probem, a new set of correated sampes was generated by the training sampes using noninear correation functions, e.g., autocorreation and crosscorreation. Thus, by incorporating these newy generated noninear-correated images into the origina image data, the origina data dimensionaity is augmented in the sense that the number of spectra band images that can be used for data anaysis is increased. With taking advantage of this new augmented set of images CEM can effectivey eiminate unknown interference and undesired signa sources. This wi be demonstrated by experiments conducted in this paper using a three-band SPOT Le Systeme Pour Observation de a Terra Earth Observation System image scene. The experimenta resuts show that GCEM greaty improves CEM with no data dimensionaity expansion. In order to further evauate the performance of GCEM, GCEM is aso compared 3 to GOSP. The experimenta resuts aso show that GCEM performs better than GOSP. The remainder of this paper is organized as foows. Section 2 describes an approach, referred to as DE derived 2 from GOSP. Section 3 briefy reviews the CEM approach, then we present GCEM in Sec. 4. Section 5 reports a set of experiments conducted to evauate the effectiveness of GCEM in cassification performance using SPOT images for anaysis. Section 6 presents some concuding comments. 2 DE The idea of the DE approach presented in this section arises from a fact that a second-order random process is generay specified by its first-order and second-order statistics. If we view the origina spectra band images as the first-order images, we can generate a set of second-order statistics spectra band images by capturing noninear correations between these spectra band images. These correated images generated by second-order statistics provide usefu noninear correation information about spectra band images that is missing in the set of the origina spectra band images. The desired second-order statistics used for DE incude autocorreation, cross-correation, and noninear correations. The concept of producing second-order correated spectra band images coincides that used to generate covariance function for a random process. Let B i i 1 be the set of a origina spectra band images. The first set of second-order statistics spectra band images is generated based on autocorreation. They are constructed by mutipying each individua spectra band image itsef, i.e., B 2 i i 1. A second set of second-order statistics spectra band images are made up of a crosscorreated spectra band images that are produced by correating any arbitrary two different spectra band images, i.e., B i B j i, j 1,i j. Adding these two sets of second orderstatistics spectra band images to B i i 1 produces a tota of (/2) ( 2 3)/2 spectra band images. In the case where more images are required, noninear functions can be used to generate so caed noninear correated spectra band images. For exampe, we may use the square-root or ogarithm, i.e., B i i 1 or og B i i 1 to stretch out ower gray-eve vaues. In the foowing, we describe severa ways to generate second-order correated and noninear correated spectra band images. 1. first-order spectra band image: B i i 1 set of origina spectra band images 2. second-order correated spectra band images 1276 Optica Engineering, Vo. 39 o. 5, May 2000
Chang et a.: Generaized constrained energy minimization approach... a. B 2 i i 1 set of auto-correated spectra band images b. B i B j i, j 1,i j set of cross-correated spectra band images 3. oninear correated spectra band images a. B i i 1 set of spectra band images stretched out by the square-root b. og B i i 1 set of spectra band images stretched out by the ogarithmic function As noted in DE, a the images generated as just isted are produced nonineary. These images shoud offer usefu information for target detection and cassification because the cassifier to be used for target detection and cassification is inear and ineary generated spectra band images wi not provide extra new information to hep the cassifier improve performance. 3 CEM Approach The CEM approach 7 9 was previousy deveoped for the case that the ony required knowedge is the signature of the target to be detected. It used an FIR fiter to constrain the desired target signature by a specific gain whie minimizing the fiter output energy. It was derived from MVDR in sensor array processing 10,11,14 with the desired signature interpreted as the desired direction of signa arriva and can be derived as foows. Assume that we are given a finite set of observations S r 1 r 2 r where r i (r i1 r i2 r i ) T for 1 i is a sampe pixe vector. Suppose that the desired signature d is aso known a priori. The objective of CEM is to design an FIR inear fiter with fiter coefficients w 1 w 2 w, denoted by an -dimensiona vector w (w 1 w 2 w ) T that minimizes the fiter output energy subject to the foowing unity constraint: d T w d k w k 1. k 1 ote that the constraint constant 1 in Eq. 1 can be repaced 11,12 by any scaar c. Let y i denote the output of the designed FIR fiter resuting from the output r i. Then y i can be expressed by y i k 1 w k r ik w T r i r i T w. Thus, the average output energy produced by the observation set S using the FIR fiter with coefficient vector w (w 1 w 2 w L ) T specified by Eq. 2 is given by 1 y i 2 1 i 1 i 1 w T 1 i 1 r i T w T r i T w r i r i T w w T R L L w, 1 2 3 where R (1/)( i 1 r i r T i ) turns out to be the sampe autocorreation matrix of S. Minimizing Eq. 3 with the fiter response constraint d T w k 1 d k w k 1 yieds min 1 w y i i 1 2 min w T R w 4 w subject to d T w 1. Soving for Eq. 4 is caed CEM approach 7,8 with the weight vector w* given by w* R 1 d d T R 1 d. 4 GCEM OSP and CEM have shown success in hyperspectra image cassification. 4,7,8 However, when they are appied to mutispectra images, both suffer from their sma data dimensionaity. To expand data dimensions, DE was introduced in GOSP to generate extra spectra band images for the purpose of orthogona subspace projection. In anaogy with GOSP, GCEM aso makes use of DE to expand the origina mutispectra image data to accommodate unknown signa sources such as interferers. Therefore, GCEM is a twostage process with the first stage carried out by DE, then foowed by the second stage, which uses CEM to detect desired targets. A brief description of the procedure impementing GCEM is given as foows. 4.1 GCEM Agorithm 5 1. Appy DE to generate nonineary correated spectra band images. 2. Identify a desired target signature to be detected d. 3. Appy CEM to detect the desired target d. 5 Experimenta Resuts The data used for the foowing experiments are the SPOT image with three bands, two of which are from the visibe region of eectromagnetic spectrum referred to as band 1 0.5 to 0.59 m and band 2 0.61 to 0.68 m, and the third band is from the near IR region of eectromagnetic spectrum referred to as band 3 0.79 to 0.89 m. The ground samping distance is 20 m. These three bands are shown in Fig. 1. They are registered and combined into an image cube where each pixe is represented by a 3 1 coumn vector with each component corresponding to one band of the SPOT data. In the scene, there is a river at the bottom eft corner. At the center is a arge ake and at the right edge are aso some sma akes. Between the arge ake and many sma akes is a rairoad crossing from north to south. Shown at the eft of the image scene is an urban area, which has roads and buidings. In addition, there is aso a road running on the right edge of the scene. According to the ground truth, there are two arge major factories, referred to as site a two very bright spots and site b not visibe ocated at the center of the area. Thus, a tota of six target signatures are of interest, two factory sites, Optica Engineering, Vo. 39 o. 5, May 2000 1277
Chang et a.: Generaized constrained energy minimization approach... Fig. 2 Six signatures extracted from the image. Fig. 1 Three-band SPOT image. site a and site b, akes, river, roads, and vegetation and the spectra are shown in Fig. 2. As we can see from Fig. 2, factory site a has a very distinct spectrum from a the others. Thus, we can expect that it can be easiy detected. Figure 3 shows nine images resuting from the DE described in Sec. 2, where the images of Figs. 3 a to 3 c, 3 d to 3 f, and 3 g to 3 i were obtained by autocorreation, cross-correation, and the square root, respectivey. Thus combining the 3 origina spectra band images in Fig. 1 with those in Fig. 3 resuts in a tota of 12 spectra band images that can be used for GCEM and GOSP. Figures 4 and 5 show the resuts of GCEM and GOSP, respectivey, where the images as abeed as Figs. 4 a to 4 f are the detection and cassification resuts of six targets: factory Fig. 3 ine images resuting from DE where the images (a) to (c), (d) to (f), and (g) to (i) were obtained by autocorreation, cross-correation, and the square root, respectivey. 1278 Optica Engineering, Vo. 39 o. 5, May 2000
Chang et a.: Generaized constrained energy minimization approach... Fig. 4 Cassification resuts of GCEM. site a, and factory site b, akes, river, roads, and vegetation, respectivey. Comparing the resuts in Figs. 4 and 5 GCEM ceary outperformed GOSP in a the cases, particuary for detection and cassification of factory site b, river, and roads. To further demonstrate advantages of GCEM over CEM without using DE, we appied CEM to three origina spectra band images in Fig. 1. The detection and cassification resuts are shown in Fig. 6 where the images abeed as Figs. 6 a to 6 f are the detection and cassification resuts of factory site a, factory site b, akes, river, roads, and vegetation, respectivey. Obviousy, GCEM performed significanty better than CEM. For exampe, CEM faied to detect the factory site b and had troube with cassifying akes, river, and roads in Figs. 6 c to 6 e. This was because their spectra in Fig. 2 were very simiar. In addition, from Figs. 5 and 6, it is easy to see that GOSP performed better than CEM in cassifying a targets except roads with which GOSP aso had troube. A the preceding experiments demonstrated that to appy hyperspectra image processing techniques such as OSP Ref. 4 and CEM Refs. 7, 8 to mutispectra imagery data DE is an effective means to extended their appicabiity and capabiity. It was noted in Refs. 1 and 3 that OSP performed poory for SPOT data, which was aso the case for the data in Fig. 1. Therefore, the experiments using OSP were not incuded for comparison. Fig. 5 Cassification resuts of GOSP. Optica Engineering, Vo. 39 o. 5, May 2000 1279
Chang et a.: Generaized constrained energy minimization approach... Fig. 6 Cassification resuts of CEM. 6 Concusion Despite that CEM has been successfuy appied to hyperspectra image cassification, 7 9 its appicabiity to mutispectra imagery is yet to be investigated because it has been taken for granted by assuming that CEM wi perform as we as it does for hyperspectra imagery. This paper shows that this is not the case. This is so argey due to the fact that CEM suffers from the same probem encountered in the OSP approach, 3,4 that is, data dimensions are insufficient. For CEM to work for mutispectra imagery, a GCEM is presented in this paper and can be viewed as a mutispectra version of CEM. GCEM incorporates an approach proposed in the GOSP Ref. 3, DE to expand the origina image data so that there are enough spectra band images to make CEM effective. Specificay, GCEM is a two-stage process with the first stage impemented by DE to expand image data, then foowed by using CEM in the second stage. The experiments show that GCEM overcomes the inherent imitation of CEM on data dimensionaity and performs significanty better than CEM without using DE. This is so because the spectra band images generated by DE are nonineary correated images that provide usefu information to improve CEM performance. Additiona experiments aso show that GCEM outperforms GOSP since GCEM requires ony the knowedge of the desired target signature rather than the compete knowedge of target signatures in the image scene required for GOSP, a situation that is rarey satisfied in many rea appications. However, ike CEM, GCEM is very sensitive to noise and the used desired target signature. Recenty, this probem is aeviated by an approach proposed in Ref. 12, caed the LCMV method, which constrains mutipe target signatures instead of a singe desired target signature. As a resut, LCMV performs more robusty than CEM. By taking advantage of LCMV, GCEM can be further extended to GLCMV. References 1. C.-I. Chang and C. Brumbey, A Kaman fitering approach to mutispectra image cassification and detection of changes in signature abundance, IEEE Trans. Geosci. Remote Sens. 37 1, 257 268 1999. 2. S. S. Epp, Discrete Mathematics with Appication, 2nd ed., Brooks/ Coe Pubishing 1995. 3. H. Ren and C.-I. Chang, Generaized orthogona subspace projection approach to unsupervised mutispectra image cassification, Proc. SPIE 3500, pp. 42 53, Spain 1998. 4. J. Harsanyi and C.-I. Chang, Hyperspectra image cassification and dimensionaity reduction: an orthogona subspace projection approach, IEEE Trans. Geosci. Remote Sens. 32, 779 785 1994. 5. J. J. Sette, On the reationship between spectra unmixing and subspace projection, IEEE Trans. Geosci. Remote Sens. 34, 1045 1046 1996. 6. C.-I. Chang, Further resuts on reationship between spectra unmixing and subspace projection, IEEE Trans. Geosci. Remote Sens. 36 3, 1030 1032 1998. 7. J. C. Harsanyi, Detection and cassification of subpixe spectra signatures in hyperspectra image sequences, PhD Dissertation, Department of Eectrica Engineering, University of Maryand Batimore County Batimore 1993. 8. J. C. Harsanyi, W. Farrand, and C.-I. Chang, Detection of subpixe spectra signatures in hyperspectra image sequences, in Proc. Annu. M. Am. Soc. Photogrammetry and Remote Sensing, pp. 236 247, Reno 1994. 9. W. Farrand and J. C. Harsanyi, Mapping the distribution of mine taiing in the coeur d Aene river vaey, Idaho, through the use of constrained energy minimization technique, Remote Sens. Environ. 59, 64 76 1997. 10. O. L. Frost III, An agorithm for ineary constrained adaptive array processing, Proc. IEEE 60, 926 935 1972. 11. B. D. Van Veen and K. M. Buckey, Beamforming: a versatie approach to spatia fitering, IEEE ASSP Mag., 4 24 Apr. 1998. 12. C.-I. Chang and H. Ren, Lineary constrained minimum variance beamforming for target detection and cassification in hyperspectra imagery, in IEEE Int. Geoscience and Remote Sensing Symp. 99, pp. 1241 1243, Hamburg, Germany 1999. 13. H. Ren, A comparative study of mixed pixe cassification versus pure pixe cassification for muti/hyperspectra imagery, M S Thesis, Department of Computer Science and Eectrica Engineering, University of Maryand Batimore County, Batimore May 1998. 14. S. Haykin, Adaptive Fiter Theory, 3rd. ed., Prentice-Ha, Engewood Ciffs, J 1996. 1280 Optica Engineering, Vo. 39 o. 5, May 2000
Chang et a.: Generaized constrained energy minimization approach... Chein-I Chang received his BS, MS, and MA degrees from Soochow University, Taipei, Taiwan, in 1973, the Institute of Mathematics at ationa Tsing Hua University, Hsinchu, Taiwan, in 1975, and the State University of ew York at Stony Brook, 1977, respectivey, a in mathematics, his MS and MSEE degrees from the University of Iinois at Urbana-Champaign in 1982, respectivey, and his PhD in eectrica engineering from the University of Maryand, Coege Park, in 1987. He was a visiting assistant professor from January 1987 to August 1987, assistant professor from 1987 to 1993, and is currenty an associate professor in the Department of Computer Science and Eectrica Engineering at the University of Maryand, Batimore County. He was a visiting speciaist in the Institute of Information Engineering at the ationa Cheng Kung University, Tainan, Taiwan, from 1994 to 1995. His research interests incude automatic target recognition, mutispectra/ hyperspectra image processing, medica imaging, information theory and coding, signa detection and estimation, and neura networks. Dr. Chang is a senior member of IEEE and a member of SPIE, IS, Phi Kappa Phi, and Eta Kappa u. Jih-Ming Liu received his BSEE and MSEE degrees from Chung Cheng Institute of Technoogy, Taipei, Taiwan, in 1987 and 1992, respectivey. He is currenty pursuing his PhD degree at the ationa Taiwan University of Science and Technoogy. His research interests incude remote sensing, signa/image processing, neutra networks, and statistica pattern recognition. Bin-Chang Chieu received his PhD degree in eectrica engineering from Rensseaer Poytechnic Institute, Troy, ew York, in 1989. He is currenty a professor in the Department of Eectronic Engineering, ationa Taiwan University of Science and Technoogy, Taipei, Taiwan. His current research interests are image processing, digita signa processing, neura networks, and computer vision. Hsuan Ren received his BS degree in eectrica engineering from the ationa Taiwan University, Taipei, Taiwan, in 1994 and his MS degree in computer science and eectrica engineering from University of Maryand, Batimore County, in 1998, where he is currenty a PhD candidate. He is currenty a research assistant in the Remote Sensing, Signa and Image Processing Laboratory, University of Maryand, Batimore County. His research interests incude data compression, signa and image processing, and pattern recognition. He is aso a member of Phi Kappa Phi. Chuin-Mu Wang received his BS degree in eectronic engineering from ationa Taipei Institute of Technoogy and his MS degree in information engineering from Tatung University of Taiwan in 1984 and 1990, respectivey. From 1984 to 1990, he was a system programmer on an IBM mainframe system and from 1990 to 1992 he was a marketing engineer on computer products at Tatung Company. Since 1992, he has been a ecturer at the Chinyi Institute of Technoogy. His research interests incude database, digita image process, and neura network. Chien-Shun Lo received his BS and MS degrees in information engineering and computer science from Feng-Chia University, Taiwan, in 1992 and 1994, respectivey. He is now working toward a PhD degree at the Institute of Eectrica Engineering, ationa Cheng-Kung University, Taiwan. He is aso a research assistant in the Department of Radioogy, Taichung Veterans Genera Hospita, Taiwan. His current research interests are image processing, medica image processing and anaysis, and computeraided diagnostic system. Pau-Choo Chung received her BS and MS degrees in eectrica engineering from ationa Cheng Kung University, Tainan, Taiwan, in 1981 and 1983, respectivey, and her PhD degree in eectrica engineering from Texas Tech University, Lubbock, in 1991. From 1983 to 1986, she was with the Chung Shan Institute of Science and Technoogy, Taiwan. Since 1991, she has been with the Department of Eectrica Engineering, ationa Cheng Kung University, where she is currenty a fu professor. Her current research incudes neura networks and their appications to medica image processing, medica image anaysis, teemedicine, and video image anaysis. Chin-Wen Yang received his BS degree in information engineering sciences from Feng-Chia University, Taiwan, in 1987 and his MS degree in information engineering in 1989 and his PhD degree in eectrica engineering in 1996 from ationa Cheng Kung University, Taiwan. He is currenty a system management chief in the Computer and Communication Center at Taichung Veterans Genera Hospita, Taiwan. His research interests incude image processing, biomedica image processing, and computer network. Dye-Jyun Ma received his BS degree in eectronics engineering from ationa Chiao-Tung University, Taiwan, in 1979 and has MS and PhD degrees in eectrica engineering from the University of Maryand, Coege Park, in 1984 and 1988, respectivey. From May 1988 to June 1993, he was with Digita Equipment Corporation, Massachusetts, as a principa engineer, where he was responsibe for performance modeing and anaysis of computer communication networks. Since August 1993, he has been with the Department of Eectrica Engineering, ationa Chung-Hsing University, Taichung, Taiwan, where he is presenty a professor and chairs the department. His research interests incude stochastic contro and optimization, modeing and optima design of communication networks, remote sensing, and medica image processing. Optica Engineering, Vo. 39 o. 5, May 2000 1281