Applcaton of Lnear Dscrmnant Analyss to Doppler Classfcaton M. Jahangr QnetQ St Andrews Road, Malvern WORCS, UK, WR14 3PS Unted Kngdom mjahangr@qnetq.com ABSTRACT In ths wor the author demonstrated a robust and effcent method for mplementng Doppler classfcaton through the use of Lnear Dscrmnant Analyss (LDA). LDAs were used to reduce dramatcally the data dmensonalty and thereby elmnate redundancy and mprove the effcency of the classfer. The performance was assessed on a three-class problem of personnel, traced and wheeled vehcles. Real radar data from a ground based system were used n the desgn and testng of the classfer. The classfer algorthm was optmsed by choosng the best set of features that maxmsed the performance and the bootstrap method was used to measure the confdence nterval. It was shown that only the frst few LDA features were relevant. At the very least these were shown to contan nformaton regardng the frequency extent of target Doppler sdebands. The classfer was shown to be robust to changes n target vewng geometry and speed. Overall, good classfcaton was acheved for personnel wth some msclassfcaton between traced and wheeled vehcles. 1.0 INTRODUCTION MTI (Movng Target Indcaton) radars can provde an all-weather, day/nght, survellance capablty. Such radar systems provde very effcent locaton nformaton on movng targets but tradtonally have lmted recognton capablty. Automatc recognton algorthms developed for magng radars, whch explot target spatal nformaton, are not applcable for MTI systems because they operate n a low resoluton mode. However, there s potental for classfcaton based on target Doppler sgnatures. The Doppler sgnatures are shfted n frequency n proporton to the target radal velocty. Movement or rotaton of structures on a target may nduce addtonal frequency modulatons on the returned radar sgnal and generate sdebands about the Doppler frequency shft of the target s body. The sgnature characterstcs of these Doppler sdebands provde a mechansm for classfyng the target of nterest. The Doppler classfer models each target class as a multvarate Gaussan mxture dstrbuton (GMD). The parameters of the GMD model are estmated usng labelled tranng data. The nput feature vectors are generated from the radar Doppler spectra. It s assumed that each Doppler spectrum provdes an ndependent feature vector. Tranng uses multple Doppler spectra per target class. Recognton s performed usng a sngle Doppler spectrum (feature vector). The sze (and therefore the dmensonalty) of the nput feature vector depends upon the number of separate frequency bns n the Doppler spectra. Heren les the lmtaton of a classfcaton technque that uses the Doppler spectra drectly for nput feature vectors. Doppler spectra can comprse a large number of frequency bns (several tens, possbly hundreds) to cover suffcently the full range of Doppler frequences Paper presented at the RTO SET Symposum on Target Identfcaton and Recognton Usng RF Systems, held n Oslo, Norway, 11-13 October 2004, and publshed n RTO-MP-SET-080. RTO-MP-SET-080 P14-1
Applcaton of Lnear Dscrmnant Analyss to Doppler Classfcaton at enough resoluton to be able to provde meanngful classfcaton performance. Hgh dmensonalty leads to ncreased classfer complexty. There are more parameters to estmate per target model whch results n an ncreased processng load. Reducng dmensonalty maes the classfcaton calculatons qucer and saves on data storage space. Furthermore, the orgnal set of varables may contan redundant and rrelevant nformaton. Redundancy would result n the classfer havng extra parameters over and above the mnmum requred to capture the structure wthn the data. For a fnte tranng set ths would lead to poorer estmaton of the classfer parameters. Therefore, reducng the dmensonalty could also mprove classfer robustness. Lnear Dscrmnant Analyss (LDA) s a well establshed technque for obtanng a reduced-dmenson representaton of the data. LDA defnes (a few) new varables as lnear combnatons of the orgnal ones. Evdence from speech recognton has shown that the classfcaton performance mproves f features are extracted usng LDA [1]. There s a ey smlarty between speech processng and Doppler processng.e., both use the spectrogram as the nput measurement. LDA could potentally offer a good approach for reducng the number of varables n the Doppler spectra. The technque conssts of transformng the Doppler spectra varables usng lnear combnaton nto a set of features (the feature vector) that are mutually orthogonal. The ndvdual features are assumed to be ndependent. The transformaton s desgned to maxmse the between-class covarance and mnmse the average wthn-class covarance. The transformed features are raned n order of the class separablty. In theory, the classfcaton performance should ncrease monotoncally as the number of features ncreases. Ths allows smple tradeoffs to be made between complexty (number of features) and vablty (classfcaton performance). The classfcaton algorthm s developed for a three class problem based on personnel, wheeled vehcles and traced vehcles. Secton 2 gves an outlne of the algorthm. It descrbes the pre-processng, the LDA feature extracton and the Doppler classfcaton stages of the algorthm. The data sets used n ths study are descrbed n Secton 3. Results are presented n Secton 4. Secton 5 summarses the conclusons. 2.0 CLASSIFICATION ALGORITHM 2.1 Pre-processng The objectve of Doppler classfcaton s to classfy an unnown target as belongng to one of a predefned set of classes based on the measured Doppler spectra. The Doppler spectra are obtaned by Fourer transformng a sequence of samples obtaned from a sngle range cell durng the radar dwell. Fgure 1 compares typcal spectra of a wheeled vehcle, a traced vehcle and a man joggng. The pea n the spectra corresponds to the Doppler shft due to the body of the target. The Doppler sdebands, f present, are due to any parts of the target whch are movng ndependently of the man body at that moment. For the wheeled vehcle there are no Doppler sdebands vsble. Ths can be contrasted wth the much more complex, but asymmetrcal, spectrum of the traced vehcle, and ths can agan be dstngushed from the more symmetrcal spectrum of the walng man. The nformaton n the Doppler spectra, however, cannot be used drectly for classfcaton. Ths s because the Doppler radar sgnature s affected by certan factors such as the radar gan, nose level, etc., that are unrelated to the target class but can confuse the classfcaton process. The data can be transformed so that the Doppler sgnatures are nvarant to these factors. Ths process that s performed pror to classfcaton s termed pre-processng. The pre-processng ams to obtan a 2D spectrogram from a long sequence of temporal samples and process each ndvdual spectrum to extract a target Doppler-profle that s ndependent of radarcalbraton and target-velocty. The spectrogram s generated usng a short-tme Fourer transform. Clutter frequency bns are mased and those that contan nose only are clpped to a mnmum value. The pea n P14-2 RTO-MP-SET-080
Applcaton of Lnear Dscrmnant Analyss to Doppler Classfcaton each Doppler-profle s centred whch maes the spectrum nvarant to target velocty. Fnally, the data are normalsed wth respect to receved power and transformed usng natural logarthms. Fgure 1: Doppler spectra from land targets Fgure 2 plots the Doppler sgnatures shown n Fgure 1 followng pre-processng. All the spectra have equal pea values and are centred on the same Doppler frequency. The pre-processed Doppler sgnatures are now nvarant to changes n radar gan and target bul velocty. The pre-processng also parttons the data nto fve separate velocty bands based on the estmate of the target body velocty obtaned usng the pea n the Doppler spectrum. Ths s desgned to enable the algorthm to model some aspects of the velocty dependent data attrbutes. A separate classfer s traned and tested for data from each velocty band. Fgure 2: Doppler spectra from land targets followng pre-processng RTO-MP-SET-080 P14-3
Applcaton of Lnear Dscrmnant Analyss to Doppler Classfcaton 2.2 LDA feature extracton The pre-processed Doppler spectra are put through the LDA data reducton process usng the transformaton T y = A x (1) where x s the log-normalsed Doppler spectrum wth p varables, y s the LDA transformed feature vector wth d varables and A s the p d lnear transformaton matrx. The latter s the feature transformaton matrx A [ a K ] egenvector equaton [2] = 1 a d, where j a are the egenvectors of the generalsed symmetrc S a = λs a (2) B The LDA process obtans the transformaton that maxmses the rato of between class covarance to average wthn-class covarance. S s the average wthn-class covarance matrx gven by: W S W C = = 1 W n ˆΣ n where n s the number of measurements n the -th class, n the total number of measurements n the data set, C the number of classes and Σˆ s the sample covarance of class gven by: n ( 1 ) ( x m )( x m ) n q = 1 T Σ ˆ = (4) q where x q and m are the measurement vector and the sample mean for the -th class respectvely. Each of these s a p-dmenson vector. The latter s gven by: = ( ) q = n S B s the between-class covarance matrx gven by: n 1 q q (3) m 1 x (5) S B = C = 1 where m s the sample mean of the entre data. n n ( m m)( m m) T (6) The number of columns (egenvectors) n the matrx A defnes the sze of the LDA feature vector y. The upper lmt for d s the maxmum number of non-zero egenvalues for (2) gven by: ( p, 1) d = mn C (7) max Snce the egenvalues for (2) are ordered n terms of class separablty, n theory the classfcaton performance should ncrease monotoncally as the sze of the LDA feature vector y s ncreased. The transformaton matrx A s estmated usng the same tranng data that s used for estmatng the classfer parameters. As pre-processng parttons the data n to V b (=5) dfferent velocty-bands a separate transformaton matrx A, where = 1, K,Vb s estmated for each velocty-band. Furthermore, the estmaton process requres that the data are class-labelled. One opton would have been to use the three P14-4 RTO-MP-SET-080
Applcaton of Lnear Dscrmnant Analyss to Doppler Classfcaton broad-class labels, personnel, traced vehcles and wheeled vehcles. However, ths would have lmted d to just a maxmum of two features. It was felt that ths would not have been suffcent to fully explot max the structure n the data. For ths reason a fne-class labellng mechansm was adopted to ncrease C and thereby allow for a hgher value for d for the transformed feature vector y. The fne-class labellng max was based on the target type, ts aspect angle and ts nomnal speed. It may be possble, although ths was not proven, that the fne-class categores have some physcal justfcaton. 2.3 Doppler classfer The LDA feature vectors are used as nputs to the classfer. A separate classfer s defned for each of the velocty-bands V. For C broad classes the class membershp s denoted by ω, { 1, K, C}, { 1, K, V}. For an unnown feature vector y the class membershp wll be one that P ω y. Accordng to Bayes rule ths s equvalent to: maxmses the posteror probablty ( ) P ( ω y ) = P ( ω ) P( y ω ) P( ω ) P( y ω ) = P ( y ω ) P( y ω ) where P( y ω ) s the probablty of the feature-vector y from velocty-band arsng from class ω, and P( ω ) s the pror probablty of class ω beng present. All the tranng classes were assumed to be equally lely. Thus class membershp s based on the probablty value P y ω ) calculated for each broad-class. ( Each broad class probablty was modelled as a multvarate Gaussan mxture model wth a dagonal covarance matrx. The mxture dstrbuton has the same dmensonalty as the LDA feature vector. Four mxture components were used. The parameters of the model (mean, varance and weghts) were estmated usng tranng data. Performance was evaluated usng ndependent test data. (8) 2.0 DATA SET Radar data from movng targets were collected usng a J-band, horzontal polarsaton, short range, ground based system usng a 4 Hz pulse repetton frequency. The radar measurements were taen wth the antenna pontng n a fxed drecton and a control target movng through the radar swath at a specfed aspect angle and speed. Ths consttuted a sngle magng run and the process was repeated for a number of dfferent target types belongng to the three broad classes. The personnel data were obtaned from a tral where two subjects were maged walng and joggng ether towards the radar or movng drectly away from t. The vehcle data were obtaned from a separate tral where three traced and two wheeled vehcle types were maged along 9 dfferent aspect angles travellng at a nomnal constant speed. Ths provded 53 dfferent magng runs from whch data were extracted. For each magng run, a number of ndependent target sgnature fles of four seconds dwell were generated by processng data from dfferent locatons along the range swath. The processed range resoluton was chosen such that t was wder than the dmensons of the largest target n the data set. All the data fles were pre-processed and parttoned nto velocty bands. There was an uneven dstrbuton of classes over the velocty bands. The lowest two velocty bands contaned manly personnel targets. All three target classes were represented n the next two hghest velocty bands. Velocty band V (targets wth velocty 12mph and above) on the other hand had only vehcle targets. The data fles were gven two dfferent types of labels. Fne labels were used n the estmaton of the LDA transformaton matrx. A total of 53 RTO-MP-SET-080 P14-5
Applcaton of Lnear Dscrmnant Analyss to Doppler Classfcaton fne-class labels were defned as summarsed by Table 1. Only broad class labels were used n the tranng and testng of the classfer. Broad Class Target Type Aspect Angle Speed Total per broad class Personnel 2 2 2 8 Traced Vehcles 3 9 1 27 Wheeled Vehcles 2 9 1 18 Total 53 Table 1: Breadown of fne-class categores for the entre database 3.0 RESULTS Fgure 3: LDA transformaton matrx egenvalues plotted for each velocty-band Fgure 3 shows the comparson of the egenvalues for the dfferent velocty-band data. The egenvalues provded some ndcaton of the class separablty. As the LDA theory stated, the egenvalues were monotoncally decreasng. Egenvalues wth values close to zero can be assumed to be rrelevant. Velocty-band I and II had data prmarly just from the personnel class and therefore there was just one sngle domnant egenvalue. Velocty-band III and IV also had a relatvely hgh frst egenvalue. Ths P14-6 RTO-MP-SET-080
Applcaton of Lnear Dscrmnant Analyss to Doppler Classfcaton suggested that the frst egenvector should provde good class separablty. For velocty-band V there were no domnant egenvalues, however, the frst few egenvalues were non-zero. Ths suggested that just a few features would probably be suffcent for optmum classfcaton. Further useful nsght nto the class separablty can be obtaned usng 2-dmensonal scatter plots of the feature vectors. Fgure 4 compares the results obtaned for plottng the frst two LDA features for two dfferent velocty-bands. The left-hand result s for velocty-band III whch had data for all three target classes and the rght hand result s velocty-band V that had only vehcle data. The feature values are labelled d0 and d1 respectvely. Each pont n the scatter plot s data from one feature vector. Personnel Traced Vehcles Wheeled Vehcles Fgure 4: Scatter plot for the frst two LDA features. (left mage) Velocty-band III (rght mage) Velocty-band V From Fgure 4 t can be seen that for velocty-band III the personnel class separated completely from the vehcle classes. The vehcle classes also showed some degree of separaton but there was some overlap between the traced and wheeled vehcles. The same result for velocty-band V showed that there was a relatvely small regon of the feature space occuped by both traced and wheeled classes. However, ths was contrasted by a sgnfcantly larger regon of the feature space that was occuped exclusvely by the traced class. It s not trval to nterpret the egenvectors n a physcal manner. One possble method for determnng what nformaton s captured by an egenvector (and therefore the LDA feature) s to loo for evdence for any correlaton between the LDA feature and ad hoc features that have a physcal nterpretaton. The target Doppler sdeband extent can be measured as an ad hoc feature. Emprcal analyss showed that traced vehcles tended to have broad extent whereas wheeled vehcles generally had a narrow Doppler extent. Fgure 5 replots the scatter plot of the frst two LDA features for velocty-band V hghlghtng data that has broad Doppler extent. It showed that a majorty of the regon, that separated the traced from the wheeled class, was explaned n terms of the Doppler extent. Thus the frst two LDA features were RTO-MP-SET-080 P14-7
Applcaton of Lnear Dscrmnant Analyss to Doppler Classfcaton capturng nformaton regardng the Doppler sdeband extent n some way. The LDA features, however, cannot exactly represent ths ad hoc feature snce the latter s a non-lnear feature. Personnel Traced Vehcles Wheeled Vehcles Data wth wde Doppler extent Fgure 5: Scatter plot for the frst two LDA features for velocty-band V. Data ponts that corresponded to a wde Doppler extent are hghlghted n purple A separate classfer was mplemented for each velocty band. The frst two velocty bands had only data from the personnel class and therefore were excluded from the calculatons. The data n each of the other three velocty bands were splt nto tranng and test sets usng a 3-to-1 rato. Performance results were averaged over all three velocty bands. Fgure 6 plots the percentage correct classfcaton averaged over all three broad classes (personnel, traced and wheeled vehcles) as the number of features was ncreased. Results are shown for two cases, (a) feature vectors based upon only LDA features (blac curve), and (b) feature vectors that ncluded Doppler sdeband extent as an addtonal ad hoc feature (purple curve). From the frst result t can be seen that wth just two LDA features near maxmum performance was acheved. For sx or hgher number of LDA features the performance flattened out. Ths mples that the useful nformaton s contaned n just the frst few features. A classfer wth just sx LDA features would gve optmum performance. Ths equated to a consderable reducton n the data dmensonalty and therefore the classfer complexty. Such mprovements greatly enhance the vablty of the classfer for real-tme mplementaton. Wth the addton of the Doppler extent feature, just the frst two features alone provded the optmum performance. Ths ponted toward Doppler sdeband extent beng an mportant dscrmnatng feature. It tes n wth the observaton from the feature analyss whch showed a trend for traced vehcles to have broad extent and wheeled vehcles to have narrow extent. It suggested that the LDA features are capturng the same nformaton as n the Doppler extent of the sdebands albet usng more features. Unle ad hoc features whch are data specfc and would often requre lengthy and expansve data analyss, the LDA feature extracton process on the other hand would generalse for data wth arbtrary attrbutes. P14-8 RTO-MP-SET-080
Applcaton of Lnear Dscrmnant Analyss to Doppler Classfcaton Doppler Classfcaton 70 60 % Correct 50 40 30 20 LDA Only LDA + Extent 10 0 0 2 4 6 8 10 Number of Features Fgure 6: Doppler classfcaton as a functon of number of features OVERALL Classfcaton Decson (%) 62.7% [58.3,67.1] Personnel Traced Wheeled Personnel 96.4 [99.8,93.0] 2.7 [0.1,5.3] 0.9 [0,2.9] Actual Class Traced 0.8 [0.1,1.5] 51.4 [41.7,61.1]] 47.8 [38.2,57.4] Wheeled 0.2 [0,0.6] 21.4 [12.6,30.2] 78.4 [69.7,87.1] Table 2: Confuson Matrx of a Doppler classfer usng 6 LDA features. Results averaged over 50 bootstrap replcates Table 2 provdes the confuson matrx for the classfer wth sx LDA features. The results were generated usng 50 bootstrap replcates. Bootstrap s a statstcal nference technque, frst proposed by Efron [3], whch allows a confdence nterval to be assgned to the estmated quantty. Table 2 lsts the mean of the bootstrap replcates along wth the 90% confdence nterval shown n square bracets. The results for the confdence nterval were only approxmate snce far more bootstrap replcates (>1000) would be requred for a more accurate measure. Nevertheless, the results were useful n determnng general performance trends. Earler the selecton of the sx LDA feature classfer was based on results that were essentally a sngle bootstrap replcate. Ths choce s lent support by the estmate of the 90% confdence nterval for ths classfer. Snce the performance of the other classfers wth fewer LDA features was outsde ths range t can be concluded that the choce of the optmum s statstcally sgnfcant. A per class comparson of the confuson matrx shows that just under half the traced vehcles are msclassfed as wheeled. Ths s not very surprsng gven the fact that a substantal proporton of the traced vehcle data n the data set dd not have the dstnctve broad Doppler extent that dfferentated t from wheeled vehcles. At ths stage t can only be hypothessed that the confuson between the two vehcle classes s due to the absence of the trac returns. The data were collected from vehcles that had RTO-MP-SET-080 P14-9
Applcaton of Lnear Dscrmnant Analyss to Doppler Classfcaton srts coverng the tracs. Ths would mae the movng parts of the tracs more lely to be vsble when vewed front-to-bac, and vce versa, but less so at oblque angles. The data supported ths nference, wth far fewer of the measurements taen for vehcles travellng at oblque angles to the radar reportng the presence of the broad Doppler extent. Ths was n contrast to traced vehcles travellng ether drectly toward or away from the radar, for whch the majorty of the data had the broad Doppler extent present. Unle the two vehcle classes the personnel class separated very well. Some msclassfcaton between personnel and traced class may be expected snce both possess broadenng of the Doppler spectra. However, the manner n whch the vehcle data were collected (constant velocty wth aspect changng between measurements) meant that the personnel class was only beng classfed aganst vehcles that were travellng at very oblque angles. From the data t was observed that trac returns were often absent when the vehcles were maged at oblque angles. Ths may therefore explan the very good separaton between the personnel and vehcle classes. More representatve data that contans data from slowly movng vehcles wth vsble tracs would enable a better measure of the true performance. 4.0 CONCLUSION For the three-class problem the classfer had no dffculty n recognsng the personnel class but produced some degree of confuson between the wheeled and traced classes. The classfer algorthm was optmsed by choosng the best set of features that maxmsed the performance and the bootstrap method was used to measure the confdence nterval. It was shown that only the frst few LDA features were relevant for Doppler classfcaton. At the very least these were shown to contan nformaton regardng the frequency extent of target s Doppler sdebands. The classfer was shown to be nvarant to target aspect angle and speed and was able to model multple target types. Models for addtonal classes that have dstnct Doppler characterstcs, le helcopters, can be easly ncorporated nto the algorthm. The LDA feature extracton represents a consderable reducton n data dmensonalty and therefore s able to provde for very effcent mplementaton of the classfcaton algorthm. The LDA based classfer, therefore, offers a very powerful tool for the automatc classfcaton of movng targets from ther Doppler sgnatures. ACKNOWLEDGEMENTS The author would le to acnowledge the support of UK MOD for fundng ths research. He would also le to than Dr Keth Pontng of 20 20 Speech Ltd n provdng support wth the development of the LDA classfer. Thans also go to Ian Fnley for the numerous helpful dscussons wth the author durng the course of ths wor. REFERENCES [1] Kumar N., Investgaton of slcon-audtory models and generalzaton of lnear dscrmnant analyss for mproved speech recognton, Ph.D. dssertaton, John Hopns Unv., Baltmore, MD, 1997. [2] Devjver P. A. and Kttler J., Pattern Recognton, A Statstcal Approach, Prentce-Hall, Inc., London, 1982. [3] Efron B., Boostrap Methods. Another Loo at the Jacnfe, The Annals of Statstcs, Vol:7, 1979, pp1-26. P14-10 RTO-MP-SET-080